In our last post Mark explained the software components and structure of KLAS (Kingsbury Location Awareness Systems). These components allow the system to take in data from an external source, estimate the user’s location (more on this in a minute) and then utilize that data to provide both navigation and tour guide capabilities.

The location aware portion of the system had to have a method of determining the user’s location through the use of some external infrastructure (e.g. GPS, Wi-Fi, RFID, etc.). After some reading and discussion, Mark and I decided that we needed a reliable indoor infrastructure, one that could either be easily developed, or ideally, one that was already established. We found, in the reading, that GPS indoors was considered mostly as an unreliable method, and RFID, although very accurate, was relatively expensive and too short range for our application. Looking at Wi-Fi, we found that it was acceptably reliable in this type of environment. The best part is that because Wireless Internet is so prevalent, the infrastructure in many locations is already established. It was thus why we chose to use Wi-Fi (IEEE 802.11) for our system.

To develop our specific location system using Wi-Fi, we had to investigate our target location (Kingsbury Hall) to determine the coverage and location of the Wi-Fi Access Points. We found eight UNH Wireless Access Points located on the second floor, equally spaced to cover the majority of the area. We then added two more Access Points in a few weak areas to give our system a total of ten Access Points to utilize.



Here we see two examples of the KLAS Access Points. The one on the left is a typical UNH Wireless Access Point, and on the right is one of the two Access Points that we added to provide stronger coverage in a few areas.

Our next step was to create an algorithm which would read in the Relative Signal Strength Indicator (RSSI) values from the Wi-Fi card on our PDAs and then determine the user’s location based on that data. To do this we spent a bit of time investigating the pros and cons of the methods used by other researchers, utilizing either probabilistic or deterministic models. We found, through testing and our reading, that Wi-Fi signals are highly susceptible to environmental changes such as the thickness and density of walls, the amount of people between the Access Point and the user, and even the relative humidity of the area. We decided, with the help of Professor Andrew Kun and Oskar Palinko, that due to the size of the area and the system’s environment, using a calibrated deterministic model would be best. This essential meant that we would evenly space master location points (also called reference points) throughout the floor. These points would then be compared to the user’s actual location to determine which master location they are closest to.

Through this testing we found that although the Wi-Fi structure was able to provide a reasonable estimate of the location, our initial points were too close, causing multiple real-time points to map to one reference point. This allowed use to gauge the reference location spreading, which eventually provided us with a total of 46 points for the entire second floor. After further testing with these points, it was found that there were still areas that had a tendency to “group” together. This was seen because of the fact that the building’s features caused certain areas to estimate the same master location. Recognizing this, we were able to break up the floor even more generally into 16 areas, which could be used by our system to categorize the user’s current location.



This map shows the even spread of the 16 user areas throughout the floor, as well as the locations of the ten KLAS Access Points. Dividing the floor into these areas allows for our system to have a built in buffer for the variations in the environment, as well as any variations that might be present in the user’s device itself. This allows for a more stable, reliable system, which is used, as Mark mentioned in his post, to provide navigation and tour guide capabilities for our users.

We hope to continue to refine our system in the next few months, making it even more reliable and stable. This will allow us to conduct more research in the area of the user interface, and hopefully give us the opportunity to develop a simple, intuitive system which will make KLAS a useful tool for all.

Matthew Lape

Hi ecebloggers,

In the past I’ve discussed the imaging application, and in particular, the tagging capabilities it provides.  Now it’s time to put the application to work.  Tagging of media, particularly photographs, has become a very popular and efficient means of organizing material on the internet and on personal computers.  Over a short period of time the technique has evolved from an optional feature to a must-provide service, and can be found within modern desktop and internet photo galleries.  However, tagging is normally accomplished long after the images have been captured, and possibly at the expense of in-the-moment information.  In this view, we hypothesize that tagging photos right after they are taken, or on the spot, will result in a larger number of tags than tagging photos long after they are taken.  We also hypothesize that the tags created on the spot will be perceived to better describe the photos by consumers of the photos.  Finally, we hypothesize that a convenient way of tagging photos on the spot is by using voice commands.  

To test these hypotheses we will conduct a study in which participants will be asked to introduce the University of New Hampshire campus in a number of low-resolution pictures captured using a Symbol MC50 PDA. Participants will be divided into two groups.  One group will be able to issue voice commands to select tags from a list while the other group will have to manually select or type in tags from the same list.  The images and their tags will be posted to an internet photo gallery, such as Flickr, which will allow us to recruit a third group of participants who will compare the quality of the tags created by the first two participant groups.  The study will be conducted throughout the months of November and December, so check back for the results soon after.  Below is a sample image of UNH posted on Flickr.  The tagging section is highlighted in red.

Michael Farrar

We have already reported on the arrival of the TouchKit as well as on its initial run. This time, we present our sturdy new mounting solution for the whole system. We have accomodated the display screen of the TouchKit in the middle of a hollow desktop. Here is the author with the solution.

It can be noticed that the TTT is tilted. This allows easier manipulation by a single user either by standing in front of the screen or sitting on a higher chair. The tilt angle is adjustable. The TTT can also lie flat which would allow easier information sharing around the table. The tabletop is mounted using hinges and a simple prop-up mechanism shown in the following image.

The table itself is a standard office desk which was customized for our needs.
Besides for running TouchKit projects, this setup can also be used for TouchKit development. The image below shows the mount of a keyboard and mouse on the tilted table, which provide a basic development environment.

Further action plans include buying and installing a short-throw projector device, which could be mounted onto the bottom of the tabletop. This would allow the screen to be lowered to a horizontal position and still have a maximum projection window size.

Let me know below, if you have any comments on the solution. Thanks,

Oszkar Palinko

After a couple of days fighting with scarce documentation, the TouchKit project, which I have written about here, is finally operational. It was tested with a helloworld and a drawing application, which came with the system. Here, you can see my colleague Mark operating the touch screen, which reacts by putting a red circle at every location where the fingers touch the display. The size of the circles depend on the amount of force exerted. Very nice!

We have also tested how the system works in dark. It seems that there is not much difference, since the TouchKit camera reacts only to infra-red light. The system appears to be fairly robust in this sense. Here, Matt is experimenting with the multi-touch display:

The installation process was a little bit bumpy, because it is described on the TouchKit web site in a concise manner. After “googling” and downloading the drivers for the firewire camera, the exemplar code had to be edited before it became functional with the hardware. The TouchKit Forum was very helpful in keeping me on the right track.

After the basic functionality of the system was proven, Mark Taipan, Matt Lape (receivers of the SURF award, and distinguished P54 undergrad students) and I figured out, that the temporary screen clamping setup and camera mount are not robust enough to operate the system easily: if any part of the system (screen, projector, camera) moves, it has to be recalibrated. Therefore we began brainstorming about possible mounting solutions. For starts, we came up with a horizontal table-top mount which would allow the system to be securely fastened. Here is Matt with the proposed TouchKit stand:

It would be useful if the table would be “tiltable” for different applications. We are working on modeling and designing such a solution. Further results will follow here, on eceblogger.

Thanks to Matt and Mark for their help!

Oszkar Palinko

Check out this video showing the Microsoft Surface in Sheraton hotels:

Apparently there’s one in Boston, and it looks like it’s the Sheraton next to the Prudential.

[via Interactive Multimedia Technology]

Andrew Kun

I was just talking to Mark Taipan and Matt Lape who are working on an indoor navigation project that utilizes handheld computers. Their application allows users to see where they are on a map. Of course the small size of handheld computer displays limits the type of maps you can use. This is even more of a problem for many mobile phones (even smaller displays), which are likely to be the actual devices used in any commercially viable indoor navigation application.

One solution may come from projector phones such as this one. If you had a projector phone you could display a map on the wall and presumably get better resolution, or at least larger characters, than what you can get on a mobile phone display. It would be interesting to see if people would be willing to use such a system.

Andrew Kun

Our lab has recently acquired a TouchKit development system. It serves as a basic building block for creating multi-touch screen user interfaces. The TouchKit consists of an infra-red illuminated projection screen and a FireWire camera board. A projector has to be added to complete the system. Here is the initial setup:

The image shows the projection screen in front which is held upright by an improvised clamping board on its right side. The projector throws the image onto the screen from the back, allowing the user to interact from the front. In this initial setup the camera board is mounted on a box and connected to a PC using FireWire. The system has a really interesting principle of operation: as the user touches the front of the screen, the infra-red illumination coming from the inside of the screen changes. This change is picked up by the camera and is further processed by the software API, which is based on the OpenFrameworks C++ library.

Multi-touch user interfaces have become more popular recently with the introduction of technologies like Microsoft Surface, iPhone and CNN’s Magic Wall created by Peceptive Pixel Inc. The last one claims to be the most advanced multi-touch user interface. Here is a video showing its impressive capabilities.

We are still in the process of setting up and configuring our TouchKit system. We will post more information about it here when it becomes operational. Once this is done, it will be a great asset for students in the UbiComp course this fall at UNH taught by Prof. Andrew Kun. Students will be developing innovative user experiences based on the TouchKit as part of their course projects.

Oszkar Palinko

Check out this post by Todd Bishop, and the very interesting video. It talks about the Microsoft Sphere, an adaptation of the multitouch Microsoft Surface to a spherical display, using Global Imagination hardware.

Andrew Kun

You might know, from previous posts, that I’m working with different kinds of push-to-talk (PTT) activation solutions. One of those is the PTT glove, which I will present at the Intelligent Environments Conference next week (07/21/08).

Today, we just got the first media attention event for the glove: it is featured on a very interesting blog about wearable computing, talk2myshirt.com/blog.

This blog came to my attention, while I was searching the web for commercial glove interaction solutions. The site has a very nice collection of these products. I like it, that wearable IT accessories are neatly categorized: gloves, hats, shoes, skirts, etc, even ties!

This may sound like a cliché, but it’s always good to communicate with other bloggers, professionals, researchers, etc. If I’m not aware of other solutions similar to mine, I might think that I invented PTT gloves, but it’s always good to get a reality check by seeing if anyone else had a similar idea before. Communication can also help co-operate with others and figure out what the real problems of a particular field of research are.

A good contra-example for this is the Miss Universe event. How can we have a Miss Universe, when only humans participate? Did we research all possibilities of life on other planets (not just in the Solar system) before declaring a human to be Miss Universe? What would the Klingons have to say about this (if they exist/ed)?

Oszkar Palinko

Sony revealed an egg-shaped digital music player named Rolly (picture below) at the end of 2007, but I haven’t had the chance to see it until last weekend. It plays MP3 and AAC music files and supports direct music streaming over a Bluetooth connection. And it is able to dance.

The Rolly is more than an ordinary music player. Thus, it is motion-controlled robot with a bunch of sensors, color lights, and two flapping wings. It uses two wheels that surround the body to roll, wiggle, and spin. In vertical position the wheels can be used to change songs and adjust volume. The Rolly creates motion automatically by analyzing the music, so it can dance to any song. Also, there is a possibility of creating new motions or customizing exiting ones using PC software. You can see the Rolly’s dance in the video below. It is amazing how the sound and motion are synchronized.

Ivan Elhart