This summer I have had a great opportunity to be an intern at Mitsubishi Electric Research Labs (MERL) in Cambridge, MA. This is not only a great prospect for my professional career, but also a chance to experience how working in a real company looks like.

MERL is a daughter company of the Mitsubishi Electric Corporation from Japan and therefore they have close collaboration. As the name implies, it is a research facility where people work on many different areas, such as digital communications, multimedia, user interfaces, speech interaction, mechatronics and many others. Their overwhelming publications page just confirms how important research is in this institution.

As a MERL intern, I am a member of the speech group. My advisor here is Bret Harsham and for this summer we have been working on testing an in-car speech user interface which was developed here at MERL. This interface enables contacts, music and points-of-interest selection using voice commands. The experiments will be concerned with the influence this interface may have on driving performance and will be performed on their driving simulator (shown in the picture below).

The driving simulator is based on a racing game simulation and consists of three huge DLP projector screens which create a very wide field-of-view, force feedback steering wheel and pedals, and a motion chair. The motion chair is very powerful, because it simulates the vibrations caused by road surface and engine, as well as the tilting of the car caused by acceleration and deceleration. The feeling it produces is very realistic and it may help prevent simulator sickness which was so common in our simulator studies.

We are looking forward to publishing the results of this interesting study. So, stay tuned for more info on this topic.

Zeljko Medenica

Microsoft is either loved or hated by most of society these days. But no matter where you stand on the company, you have to agree that the vision they show in the video below is amazing, and displays great contributions to society. The video is an envisioned future of 2019, just ten years ahead. It’s amazing how Microsoft takes the time to make such a production out of their ideas for future technology, so that the rest of society can understand their vision. My favorite part is the personal devices that people carry around, which interact with many other devices wherever they go. Enjoy.

Mike Litchfield

For the majority summer, the bulk of my work on the multi-touch table has been working with Ankit and Oskar on the Google Earth application, in which we have overlayed geotagged photos with additional orientation data as 3-D models. We have added the ability to control the Google Earth application with the multi-touch table using different gesture we’ve designed. We will be using this application in an experiment that will compare two methods of storytelling, one using the Google Earth application and the other using a traditional picture slideshow with an additional map. The purpose of this is to explore the advantages of the overlayed photos as well as the application on the multi-touch table.

We have recently submitted the project as a poster and short paper submission to the Ubicomp 2009 conference and it has been accepted. The conference will be held in Orlando,Fla at the end of September. Below is a video that Ankit and I created to go along with the short paper/poster submission.

Mike Litchfield

Last Wednesday the Project54 lab hosted the April meeting of the New Hampshire Usability Professionals Association (the NH section of the Usability Professionals Association). What does the NHUPA do? Here is what Michael Hawley, Senior VP for Experience Design at Mad*Pow, and NHUPA president, has to say on this topic:

The New Hampshire Chapter of the Usability Professionals’ Association (UPA) provides networking and professional development opportunities for usability professionals, information architects, interaction designers, and other user experience professionals and students in the Seacoast and Southern NH regions. The professionals are dedicated to designing technology and software that is easy to use, useful and engaging. Our mission is to foster the growth of the local usability community and to provide an environment for members to exchange information on job opportunities, tips, tools, methodologies, and technologies related to usability.

The meeting started with a lab open house during which Oszkar Palinko, Zeljko Medenica and I discussed aspects of the Project54 effort related to user interface development and deployment in vehicles. After the open house I gave a presentation introducing two of our studies (Oszkar’s PTT glove study and a study exploring the use of navigation aids in cars). Here are a couple of pictures from the event (more on Flickr):

The 20 people who attended this meeting had excellent questions and suggestions for us, and it was a pleasure to talk to so many of them. Thanks to Michael Hawley for bringing this meeting to UNH, thanks to Kyle Soucy of Usable Interface for recommending the Project54 lab as a good venue for a NHUPA meeting, and thanks to UPA members who visited and spent time talking to us.

Andrew Kun

UPDATE: More pictures on Flickr by Kyle Soucy.

Recently Mark Taipan and I completed the development of our Kingsbury Location Awareness System (KLAS) Video. Here you will see the development and operation of KLAS, and its principle functions, giving an example usage for both the Tour Guide and Navigation Applications. Hope you enjoy the video!

Matthew Lape

A few days ago I ran across an interesting device in a local electronic store. Finally it is time when such device arrived at our lives with satisfying performance for an affordable price. I found that buying it as a new PDA/phone accessory would be worth considering. It is an ultra-mobile (size: 2.0×0.9×4.5″, weight: 5.6 oz, battery life: 40-60 minutes) handheld projector which uses LED technology to project an image up to 50 inches in diagonal. It might potentially solve the burden of a small phone screen.

Although, the video shows some potential applications, it is still not clear what the killer app will be, how the mobile phone users will accept mobile projectors (regarding privacy), and how the projectors will enhance user experience and multi-user interaction.

Ivan Elhart

Last Friday, April 10th 2009, Project54 students visited MIT to hear Patrick Baudisch’s lecture on Back-of-Device Interaction. Patrick is a research scientist at Microsoft Research and an Affiliate Professor at the University of Washington.

He presented his findings on researching back-of-device interfaces, which is an interesting new concept of human-computer interaction. A predecessor of the current solution was the lucidTouch project, which provided a semi transparent display screen using a camera mounted on a boom. This solution was novel but not too practical. Since then, Baudisch and his group developed a more sophisticated method for back-of-device interaction: the nanoTouch. Here, the camera based system is replaced with a clickable touchpad on the back of the device.

This is an interesting solution borrowed from the Microsoft Zune MP3 player. It is a touchpad which can be clicked as a momentary push-button switch.

What is the advantage of an interface like this, compared to conventional touch devices that can be operated from the front side, like the iPhone for example? When trying to activate a virtual button from the front, the fingers obscure the critical part of the screen. This makes typing on a virtual keyboard challenging. Patrick presented the results of their study where they found that using the back-of-device interface produces less button-press errors compared to front-of-device touch screens. Also it seems that the new interface will allow reduction in screen and device sizes. Patrick even showed us concept images of wrist watches and rings which could benefit from this interface. As always, Baudisch’s presentation was visually very sophisticated with elaborate images and design.

All in all it was a very interesting talk. I can’t wait to see this technology in an actual product.

Oszkar

An interesting poster was presented by Mark Micire from the University of Massachusetts at Lowell and Brenden Keyes from Mitre Corp. at last week’s IUI’09 Conference. It dealt with using a multi-touch table to control a mobile robot agent. Multi-touch displays are getting more prominent in different interaction tasks. Here, a physical system is controlled using a highly virtual interface, unlike a joystick, which uses physical motion to control a moving agent.

Mark and his colleagues used a Mitsubishi DiamondTouch display developed at MERL. Unlike our multi-touch table, this one features front projection and capacitive sensors. Front projection allows the system to be placed closer to the ground, but users can occlude the projection with their hands. Our back projection system needs more space in the back, but is not affected by occlusion. In the MERL system, the receiving circuit is located under each user (in the chair or under the feet), which allows the users to be identified by position at the table, but needs a closed circuit through the chair or under the feet.

The research showed that a new user interface can be used by different subjects in different, sometimes unexpected, ways. While the designers wanted to create a ‘virtual joystick’, the users have found analogies with different devices: mouse track-pads, piano keys, touch-typing, steering wheel, etc. This research is great in pioneering novel applications for multi-touch displays.

Oszkar

Yesterday brought an inspiring talk by Jun Rekimoto, the director of the Interaction Lab at Sony CSL and Professor at the University of Tokyo. The topic of the talk was Sensonomy, a new term coined by Jun which fuses Folksonomy with wireless sensor networks.

The basic idea behind Sensonomy is to integrate the virtual and real world environments. For this, determining the location of the user is of particular importance. Jun has found, that using WiFi for this purpose gives very good results in urban areas. Their research was tested in Tokyo, where there is more than one million WiFi access points! Check out this map. All of these points can be used to determine the location of the user. Here, Mr. Rekimoto compares the quality of GPS position determination vs. WiFi in downtown Tokyo.

It can be seen that the WiFi red line is usually more precise than the GPS black line. WiFi can also be used indoors unlike GPS and it has a much shorter startup time for determining location.

They call this technology the PlaceEngine. Anyone who downloads the client can use it to determine their location while they are online. This technology was used in their lifelogging application. Further information on this can be found at the web site of Mr. Rekimoto’s lab, which is partly in English. Check it out, it’s really interesting.

Furthermore, this research connects very well to the KLAS project which was developed by Mark Taipan and Matt Lape at our lab, Project54. They have used WiFi for determining indoor location. It seems that it is not impossible to take the system out of Kingsbury Hall and “wire up” the whole UNH campus!

Oszkar Palinko

Trevor Darrell a distinguished professor from UC Berkeley gave a very interesting invited talk yesterday at IUI’09 on the role of image processing in making user interfaces more intelligent.

He elaborated on the state-of-the-art of image recognition. There are essentially two categories of this field: 1) instance level recognition and 2) category level retrieval. The first category is almost a solved problem, while the second one sill causes headaches to scientists: how could computers recognize that a bar stool and a rocking chair belong to the same category?

Trevor also showed a few new cutting edge technologies that are starting to change our everyday life. For example, the SnapTell application for smartphones can recognize snapshots of book covers, CDs, etc. made with your cameraphone and find the product on the internet. The goal of this program is of course commercial, to sell the items that you are looking for. But such applications can also enable you to find out more about e.g. a new movie, based on a snapshot of its poster or even a TV advertisement.

This idea seems relevant to the work of my colleague, Michael Farrar, at Project54. His photo tagging application could be further improved to enable location, object or even suspect recognition (in the long run) for police officers.

Oszkar Palinko