Imagine spending your vacation in the Swiss Alps. Wouldn’t it be nice if you could share these pictures among your friends in a easy and cool way? Imagine a big touch table where you can show around your pictures, manipulate, zoom them and pass them on to other users. Well, this has now become possible to do with multi-touch table technology. One such device is the Surface multi-touch table at Project54. The Microsoft Surface offers a unique environment where multiple users can collaboratively and simultaneously interact with data and each other. The Surface lets you grab digital content with your hands and perform various functions using simple gestures and touch.  A Surface Software Development Kit is also shipped with the Surface table which allows researchers and other software developers to develop their own applications for the Surface.

As a student intern under Andrew Kun at the University of New Hampshire for the summer of 2010, I looked into what the Surface offers. For the majority of my work, I explored the world of Natural User Interfaceoffered by the Surface by developing an application which allows users to share pictures in an easy and appealing environment. Apart from this, I also worked on developing an application that allows Windows Mobile devices to upload pictures to my Surface application. My work on the handhelds was done in collaboration with Tim April and Dylan Fransway.

My picture sharing application allows users to upload pictures on the Surface from their mobile devices (or any wireless device) and with the help of some very simple and intuitive gestures, the pictures can be moved, zoomed and played with. The users can make collages by manipulating and placing pictures anywhere on the Surface and taking a screenshot of the screen. The application allows users to transfer pictures to other users(thus transferring images to their Mobile devices), send them to Bluetooth enabled devices or upload them to web based services such as Flickr. With more and more mobile devices and cameras having a built-in GPS, pictures today are usually geo-tagged. Taking advantage of this fact, geo-tagged images can be viewed on a virtual globe. The Surface environment allows interaction with Microsoft’s virtual globe and thus storytelling can be made more immersive and interesting with our application.

Check out the video feature a demo of our Picture Sharing Application in which two users exchange images, create a collage and upload a few pictures to Flickr:

Pilot studies indicate that the photo sharing application is easy to use. The gestures required are natural, intuitive and effective. Digging deeper, the application presents very low workload statistics as compared to when the same functionalities are achieved in conventional plug and play picture sharing scenario. Note that my work can be extended to include features from the work of Mike Farrar on voice tagging and Trupti Telang’s Multi Touch Dispatch System.

Jatin Matani

A pointing device, such as a mouse, touchpad or the pointing stick, has become an essential component of the modern computer system. As the number of portable computer users steadily increases so does the need for a compact and highly portable pointing device. The problem evident with existing pointing devices is that an increase in the portability of the device often results in the decrease of performance of the device and/or the comfort for the user.

The MoGo Mouse BT^TM of Newton Peripherals and the Slim G4 Mouse of DaoKorea were developed to improve the portability of the mouse by reducing its thickness down to 5mm enabling it to be neatly stored inside a PC card slot. However reduced graspability was indicated as their weak point, which ensued due to their very slim form.

To address this usability-portability trade-off problem a 5 person team of the Department of Industrial Design of the Korea Advanced Institute of Science and Technology has developed the Inflatable Mouse by essentially attaching an inflatable balloon on top of a slim mouse. The Inflatable Mouse is a volume-adjustable user interface. It connects to the computer through a USB port which can also be used to power its air pump. The air pump inflates the balloon up to the volume of a familiar mouse as needed, and deflates it entirely so it could be stored in the PC card slot of a portable computer when not in use. Apart from the standard functionalities of a mouse it also provides a new input modality of squeezing the mouse by sensing the air pressure inside the balloon, and also has multiple touch sensors on the top and both sides of the mouse. Changing pressure can be used for scrolling, selecting from a list etc.

Since the volume of the mouse can be promptly changed it can also be used as an output/display device, where a variety of dynamic expressions can be made, for instance simulating the beating motion of the heart.

For further reading and a video, check out:

• Inflatable mouse CHI 2008 paper
• Inflatable mouse youtube video

David Filipovic

Researchers at the Massachusetts Institute of Technology have developed a device that allows a user to transform virtually any surface into a multitouch computer. The name of this new technology is Project LuminAR, which is basically an embedded computer in a lamp. The LuminAR bulb contains a pico projector, a camera, and a computer with wireless connectivity - all packaged into an area not much larger than a digital camera. The LuminAR bulb uses a pico projector to display a computer interface (which would normally be seen on a liquid crystal display) onto any flat surface. A built-in camera detects and interprets movement from the user, and sends that data to the computer. The user can navigate and type on the interface using gestures and a graphical keyboard. This reduces the need for hardware such as a mouse, keyboard, and display that are necessities on any desktop or laptop PC. Another great feature of this device is that it plugs into a robotic lamp which has the ability to move around based on the user’s movements and gestures. The LuminAR bulb can also be installed in any household lamp.

Take a look at the original article from the IEEE here.

Devin Mullen

A recent post on Slashdot led me to this interesting video produced by the folks at Ishikawa Komuro Laboratory at the University of Tokyo. Using their Vision Chip, a CMOS image sensor with a parallel image processor, they created an in-air, finger motion based interface for a mobile phone. While I do not believe that in-air typing is the right application for this interface (a lack of tactile feel will I think turn off users), this certainly is a fascinating project that may be what we see commercially in the next couple of decades! Take a look at their video below. Also, if you haven’t done so, check out Zeljko’s post about Project Natal – a similar input interface for the future Xbox 360.

Mark Taipan

A couple of weeks ago our ECE department hosted Bret Harsham of Mitsubishi Electric Research Labs (MERL) as a part of the ECE900 graduate seminar. Bret gave a very interesting lecture about the ways of shortening voice dialogs through using a contextual push-to-talk button. The title of the lecture was “Contextual Push-to-Talk: Shortening Voice Dialogs to Improve Driving Performance.”

The focus of the lecture was on a prototype in-car voice user interface (VUI) which was tested during my MERL internship last summer. As opposed to the contemporary in-car VUIs, which use only one push-to-talk button for issuing commands, this work presents a way of utilizing multiple push-to-talk buttons depending on the context of the query. For example, if we have three domains of interest, we can associate one push to talk button for each of them. Therefore, we skip multiple steps which are otherwise required in order to switch to a desired domain and initiate a voice search.

This work was recently accepted for publication at MobileHCI 2010 conference.

Zeljko Medenica

This summer I have had a great opportunity to be an intern at Mitsubishi Electric Research Labs (MERL) in Cambridge, MA. This is not only a great prospect for my professional career, but also a chance to experience how working in a real company looks like.

MERL is a daughter company of the Mitsubishi Electric Corporation from Japan and therefore they have close collaboration. As the name implies, it is a research facility where people work on many different areas, such as digital communications, multimedia, user interfaces, speech interaction, mechatronics and many others. Their overwhelming publications page just confirms how important research is in this institution.

As a MERL intern, I am a member of the speech group. My advisor here is Bret Harsham and for this summer we have been working on testing an in-car speech user interface which was developed here at MERL. This interface enables contacts, music and points-of-interest selection using voice commands. The experiments will be concerned with the influence this interface may have on driving performance and will be performed on their driving simulator (shown in the picture below).

The driving simulator is based on a racing game simulation and consists of three huge DLP projector screens which create a very wide field-of-view, force feedback steering wheel and pedals, and a motion chair. The motion chair is very powerful, because it simulates the vibrations caused by road surface and engine, as well as the tilting of the car caused by acceleration and deceleration. The feeling it produces is very realistic and it may help prevent simulator sickness which was so common in our simulator studies.

We are looking forward to publishing the results of this interesting study. So, stay tuned for more info on this topic.

Zeljko Medenica

Microsoft is either loved or hated by most of society these days. But no matter where you stand on the company, you have to agree that the vision they show in the video below is amazing, and displays great contributions to society. The video is an envisioned future of 2019, just ten years ahead. It’s amazing how Microsoft takes the time to make such a production out of their ideas for future technology, so that the rest of society can understand their vision. My favorite part is the personal devices that people carry around, which interact with many other devices wherever they go. Enjoy.

Mike Litchfield

For the majority summer, the bulk of my work on the multi-touch table has been working with Ankit and Oskar on the Google Earth application, in which we have overlayed geotagged photos with additional orientation data as 3-D models. We have added the ability to control the Google Earth application with the multi-touch table using different gesture we’ve designed. We will be using this application in an experiment that will compare two methods of storytelling, one using the Google Earth application and the other using a traditional picture slideshow with an additional map. The purpose of this is to explore the advantages of the overlayed photos as well as the application on the multi-touch table.

We have recently submitted the project as a poster and short paper submission to the Ubicomp 2009 conference and it has been accepted. The conference will be held in Orlando,Fla at the end of September. Below is a video that Ankit and I created to go along with the short paper/poster submission.

Mike Litchfield

Last Wednesday the Project54 lab hosted the April meeting of the New Hampshire Usability Professionals Association (the NH section of the Usability Professionals Association). What does the NHUPA do? Here is what Michael Hawley, Senior VP for Experience Design at Mad*Pow, and NHUPA president, has to say on this topic:

The New Hampshire Chapter of the Usability Professionals’ Association (UPA) provides networking and professional development opportunities for usability professionals, information architects, interaction designers, and other user experience professionals and students in the Seacoast and Southern NH regions. The professionals are dedicated to designing technology and software that is easy to use, useful and engaging. Our mission is to foster the growth of the local usability community and to provide an environment for members to exchange information on job opportunities, tips, tools, methodologies, and technologies related to usability.

The meeting started with a lab open house during which Oszkar Palinko, Zeljko Medenica and I discussed aspects of the Project54 effort related to user interface development and deployment in vehicles. After the open house I gave a presentation introducing two of our studies (Oszkar’s PTT glove study and a study exploring the use of navigation aids in cars). Here are a couple of pictures from the event (more on Flickr):

The 20 people who attended this meeting had excellent questions and suggestions for us, and it was a pleasure to talk to so many of them. Thanks to Michael Hawley for bringing this meeting to UNH, thanks to Kyle Soucy of Usable Interface for recommending the Project54 lab as a good venue for a NHUPA meeting, and thanks to UPA members who visited and spent time talking to us.

Andrew Kun

UPDATE: More pictures on Flickr by Kyle Soucy.

Recently Mark Taipan and I completed the development of our Kingsbury Location Awareness System (KLAS) Video. Here you will see the development and operation of KLAS, and its principle functions, giving an example usage for both the Tour Guide and Navigation Applications. Hope you enjoy the video!

Matthew Lape