Hello ecebloggers,

Last week, a group of us from the lab had traveled to Boston’s Northeastern University, attending the NECHFES student conference.  It was my first conference experience and I do have to say that it surpassed my expectations.  The atmosphere was very relaxed, and presentations were nicely sequenced with 10 – 15 minute breaks.  Of course, breakfast and lunch were served, both of which were outstanding, and free!  The keynote speaker, Daniel Serfaty from Aptima Inc, had some unique perspectives on the ubiquitous computing world of today, and tomorrow, very interesting!  Photos of my speech, entitle “Using voice to tag digital photographs on the spot” can be found here, and the complete presentation here.  I look forward to attending similar events.

Michael Farrar

In our last post Mark explained the software components and structure of KLAS (Kingsbury Location Awareness Systems). These components allow the system to take in data from an external source, estimate the user’s location (more on this in a minute) and then utilize that data to provide both navigation and tour guide capabilities.

The location aware portion of the system had to have a method of determining the user’s location through the use of some external infrastructure (e.g. GPS, Wi-Fi, RFID, etc.). After some reading and discussion, Mark and I decided that we needed a reliable indoor infrastructure, one that could either be easily developed, or ideally, one that was already established. We found, in the reading, that GPS indoors was considered mostly as an unreliable method, and RFID, although very accurate, was relatively expensive and too short range for our application. Looking at Wi-Fi, we found that it was acceptably reliable in this type of environment. The best part is that because Wireless Internet is so prevalent, the infrastructure in many locations is already established. It was thus why we chose to use Wi-Fi (IEEE 802.11) for our system.

To develop our specific location system using Wi-Fi, we had to investigate our target location (Kingsbury Hall) to determine the coverage and location of the Wi-Fi Access Points. We found eight UNH Wireless Access Points located on the second floor, equally spaced to cover the majority of the area. We then added two more Access Points in a few weak areas to give our system a total of ten Access Points to utilize.



Here we see two examples of the KLAS Access Points. The one on the left is a typical UNH Wireless Access Point, and on the right is one of the two Access Points that we added to provide stronger coverage in a few areas.

Our next step was to create an algorithm which would read in the Relative Signal Strength Indicator (RSSI) values from the Wi-Fi card on our PDAs and then determine the user’s location based on that data. To do this we spent a bit of time investigating the pros and cons of the methods used by other researchers, utilizing either probabilistic or deterministic models. We found, through testing and our reading, that Wi-Fi signals are highly susceptible to environmental changes such as the thickness and density of walls, the amount of people between the Access Point and the user, and even the relative humidity of the area. We decided, with the help of Professor Andrew Kun and Oskar Palinko, that due to the size of the area and the system’s environment, using a calibrated deterministic model would be best. This essential meant that we would evenly space master location points (also called reference points) throughout the floor. These points would then be compared to the user’s actual location to determine which master location they are closest to.

Through this testing we found that although the Wi-Fi structure was able to provide a reasonable estimate of the location, our initial points were too close, causing multiple real-time points to map to one reference point. This allowed use to gauge the reference location spreading, which eventually provided us with a total of 46 points for the entire second floor. After further testing with these points, it was found that there were still areas that had a tendency to “group” together. This was seen because of the fact that the building’s features caused certain areas to estimate the same master location. Recognizing this, we were able to break up the floor even more generally into 16 areas, which could be used by our system to categorize the user’s current location.



This map shows the even spread of the 16 user areas throughout the floor, as well as the locations of the ten KLAS Access Points. Dividing the floor into these areas allows for our system to have a built in buffer for the variations in the environment, as well as any variations that might be present in the user’s device itself. This allows for a more stable, reliable system, which is used, as Mark mentioned in his post, to provide navigation and tour guide capabilities for our users.

We hope to continue to refine our system in the next few months, making it even more reliable and stable. This will allow us to conduct more research in the area of the user interface, and hopefully give us the opportunity to develop a simple, intuitive system which will make KLAS a useful tool for all.

Matthew Lape

Hi ecebloggers,

In the past I’ve discussed the imaging application, and in particular, the tagging capabilities it provides.  Now it’s time to put the application to work.  Tagging of media, particularly photographs, has become a very popular and efficient means of organizing material on the internet and on personal computers.  Over a short period of time the technique has evolved from an optional feature to a must-provide service, and can be found within modern desktop and internet photo galleries.  However, tagging is normally accomplished long after the images have been captured, and possibly at the expense of in-the-moment information.  In this view, we hypothesize that tagging photos right after they are taken, or on the spot, will result in a larger number of tags than tagging photos long after they are taken.  We also hypothesize that the tags created on the spot will be perceived to better describe the photos by consumers of the photos.  Finally, we hypothesize that a convenient way of tagging photos on the spot is by using voice commands.  

To test these hypotheses we will conduct a study in which participants will be asked to introduce the University of New Hampshire campus in a number of low-resolution pictures captured using a Symbol MC50 PDA. Participants will be divided into two groups.  One group will be able to issue voice commands to select tags from a list while the other group will have to manually select or type in tags from the same list.  The images and their tags will be posted to an internet photo gallery, such as Flickr, which will allow us to recruit a third group of participants who will compare the quality of the tags created by the first two participant groups.  The study will be conducted throughout the months of November and December, so check back for the results soon after.  Below is a sample image of UNH posted on Flickr.  The tagging section is highlighted in red.

Michael Farrar

A few day ago, Prof Andrew Kun, Andras Fekete and I visited the Intelligent Environments ‘08 Conference in Seattle, WA. An earlier post already introduced this conference on eceblogger. We presented three works there. Andras had a great poster on the deployment of his new P54 PDA software. The poster session took place in the afternoon of the first day. I think his work drew the biggest crowd.

Andras presented the PDA study with great confidence and answered the questions flawlessly. Besides him, I also presented my research results from the past year. I had two oral presentations. The first one was on the steering wheel sensor device. This was a mixture of a regular slide-show presentation and a demonstration. For this purpose we shipped out a scaled down version of our driving simulator equipped with the new sensor. Here, we are testing the system right before the the start of the presentation.

Luckily, none of the equipment got broken during transportation, so everything worked perfectly. My other presentation took place in the afternoon of the second day. It was on the results of the PTT glove experiment that we mentioned here before. This presentation also went smoothly.

The conference was organized very nicely, with helpful hosts and great food. They even scheduled a visit for us to see the Microsoft Home project. The location of the conference was on the campus of the University of Washington in Seattle. It proved to be a beautiful place. I didn’t even know that there are campuses in the USA that are built in gothic style. I have seen this before only in Europe. Here Andras and Andrew explore the square in front of the landmark library building of the university, that looks more like a gothic cathedral.

Thanks go to Prof Kun for helping us and actively participating in writing all three papers (second author on all of them). Also, thanks to Erika Clifford for doing all the logistics for the trip and shipping the equipment.

Oszkar

Over the past few months I’ve spent much of my time developing the Project54PDA imaging applications.  Finally, after some coding revisions and a small in-lab experiment, we’re just about ready for deployment.  From the results, it seems as though tagging of images using handheld devices has a bright future, along with the note-taking and memo-recording capabilities the application provides.

Back in April we met with Lieutenant Mark Liebl of the NH State Police who gave his insight to the applications’ possible uses.  Mark had stated that he occasionally comes across a driver who has forgotten to carry or had misplaced their driver’s license.  In these circumstances information may be taken verbally from the subject and checked against the records database as normally would be done.  However, this assumes that the driver was being truthful.  This is where the imaging application comes into play.  At the time of questioning an officer may take a photograph of the subject and pair it with tagging, note, and/or memo data that may be reviewed later if necessary, providing some physical data on the event.  Along with this, Mark also believed that such data may be gathered at the scene of minor motor vehicle accidents.

Below are some screenshots of the imaging application in use.  Since most people are fairly familiar with the operation of a digital camera, I tried to have the imaging GUI replicate that functionality as much as possible.  Like all commands the application provides, images may be captured using the graphical buttons or through speech interaction and reviewed by using the scrolling features shown in the upper corners of the shot.  In the adjacent figure, the tagging mode of the application is depicted with the tags “poster” and “project54 logo” already paired with image (lower text area).  The upper text area displays previous tags the user has entered into the system, and allows the tagging process to be completed (through graphical or speech interaction) more rapidly.  That’s right, speech tagging!  Note-taking is performed using the text editor of the PDA, and the memo feature allows for a maximum 30 second audio recording.

With all of this data now stored on the PDA, I found it necessary to develop a set of applications which allow for extraction to and easy viewing on a desktop machine, because viewing data on a PDA is about as slow and painful as it gets.  The transfer application (not shown) handles the extraction, and the management application offers a centralized interface for data review (see the figure below).  All-in-all it was a lengthy process to get these applications working in harmony, but I think the data collection and organization methods they provide will prove to be quite valuable.

Michael Farrar

The majority of law enforcement officers patrol areas while using a police vehicle for transportation.  Inside the vehicle, if it has been equipped with the Project54 system, the officer is provided with multiple safe-and-convenient interfaces which allow him or her to complete a variety of daily tasks.  What happens when the officer leaves the vehicle?  Well, without the Project54 system, access to many, if not all, of these features is lost.  However, with the system in place, a PDA may be added to the mix and an Ad-hoc network may be established between the device and the in-car computer of the police cruiser.  Under this configuration the officer remains in complete control over the checking of records, operating the lights and radar units, and even vehicle diagnostics to name a few.  Here is a video demonstration:

Michael A. Farrar

One of the most notable features of the Project54 system is its ability to provide officers with an organized means of checking records.  Once the required information on an individual has been entered into the vehicle computer, Project54 forwards the request to the mobile radio of the police cruiser for transmission.  On return, the information is received through the mobile radio and decoded by the in-car computer, where it is then neatly displayed to the officer for analysis.  All it takes it a day-trip to the beach to realize that not all officers are equipped with vehicles and RF radios.  Typically, in such environments, transportation is accomplished through all-terrain vehicle, bicycle, or simply by walking.  Therefore, a more manageable scenario for such patrolmen must be developed.

The Symbol MC50 (PDA) devices currently deployed are equipped with a 2D-imager/scanner, but do not provide cellular connectivity.  With the scanner, information is accurately and quicky entered into the data fields, but the checking of records under these configurations is only possible through the PDA-to-vehicle network, where the request is generated on the PDA, forwarded to the vehicle computer, and again forwarded to the RF radio.  These communications do present the officer with the desired information, but the requirement of the police cruiser is still present. 

Alternatively, we have purchased two Symbol MC70 (PDA) devices, each equipped with a 2D-imager/scanner and cellular modem (One with GPS – check back for future post relating to this).  After some slight modification to the Records Application implemented on the desktop machines, a mobile counterpart had been developed, providing record queries to the mobile officer.  Here is a video demonstration:

Michael A. Farrar

Over the past few months I have been developing several applications which allow for image capture, tagging, and organization on PDA and desktop devices, all of which were demonstrated at the National Institute of Justice (NIJ) conference in Boulder, CO in early April.  Last week I had presented the imaging application to Lieutenant Mark Liebl of the NH State Police, who immediately recognized its value.  During a routine traffic stop, an officer will request identification from the individual under question, and as we all have done, he or she may have forgotten their drivers license at home.  In this situation Mark had stated that the information may be taken verbally and the individual’s status may be checked using the Records application of the Project54 system.  But what if the subject had given false information?  This is where the imager of the PDA comes into play.  After the verbal exchange, a photograph of the individual may be taken, and if necessary, reviewed at a later time to reveal the subject’s true identity.  Mark also believes that the imager could be used at the scene of most minor vehicle accidents. Here is a video which demonstrates the capturing and tagging techniques discussed above.

In addition to the imaging application, I have been developing two desktop applications which transfer and organize the images stored on the PDA.  The transfer application neatly synchronizes files stored on the desktop with those on the PDA, ensuring that duplicate files are not copied and that existing files are not overwritten.  Once the photographs have been transferred to the desktop PC, they and their tagging data may be easily viewed using the image management application.

 Michael A. Farrar

Last week, Lieutenant Mark Liebl of the NH State Police started testing new versions of the Project54 navigation application and the Project54 handheld application.

Jon Oppelaar installed a USB GPS unit (GlobalSat BU-353) in Mark’s cruiser. This unit performs significantly better than an older GPS unit that was intalled several years ago, and it will enable Mark to test Jon’s navigation application. The application integrates Microsoft MapPoint into the Project54 environment. Here’s Jon (in the passenger seat) tweaking software settings in Mark’s cruiser during the installation of the USB GPS unit:

Andras Fekete worked on updating the software that allows a handheld unit to control the in-car devices (see picture below). Andras has already successfully deployed handheld technology in the Lee, NH police department, which we describe in a paper that will be presented at IE08. We’re now looking forward to getting Mark’s feedback about Project54 handheld software.

While Jon and Andras were installing new hardware and software, Mike Farrar talked to Mark about using the Project54 handheld computer’s imager, not only as a 2D barcode reader (e.g. for driver licenses), but as a camera. Mike is developing software that will allow officers to take pictures and tag them. Tagging can be done using voice commands, which should speed up data entry. It’s worth mentioning that Mike is developing his software using the Symbol MC-70, which comes with a cell modem. This opens new opportunities for getting data to officers in the field, especially in urban areas. In fact, the new version of the MC-70 also has a built-in GPS unit, again presenting interesting opportunities for law enforcement applications.

Thanks for testing all this harware and software Mark!

Andrew Kun

The Project54 lab will have three papers at the Intelligent Environments 2008 conference (to be held in Seattle, WA, July 21 and 22).

Oszkar Palinko and I have a short paper and a demo paper, both related to our research on how press-to-talk buttons influence driving performance when used with in-car speech user interfaces. The short paper describes Oszkar’s PTT glove and the demo will show his work on using pressure sensors mounted on a steering wheel as a PTT solution.

Andras Fekete and I have a short paper which describes the deployment of a Project54 handheld computer in the Lee, NH police department.

Great job Andras and Oszkar!

Andrew Kun