Imagine spending your vacation in the Swiss Alps. Wouldn’t it be nice if you could share these pictures among your friends in a easy and cool way? Imagine a big touch table where you can show around your pictures, manipulate, zoom them and pass them on to other users. Well, this has now become possible to do with multi-touch table technology. One such device is the Surface multi-touch table at Project54. The Microsoft Surface offers a unique environment where multiple users can collaboratively and simultaneously interact with data and each other. The Surface lets you grab digital content with your hands and perform various functions using simple gestures and touch.  A Surface Software Development Kit is also shipped with the Surface table which allows researchers and other software developers to develop their own applications for the Surface.

As a student intern under Andrew Kun at the University of New Hampshire for the summer of 2010, I looked into what the Surface offers. For the majority of my work, I explored the world of Natural User Interfaceoffered by the Surface by developing an application which allows users to share pictures in an easy and appealing environment. Apart from this, I also worked on developing an application that allows Windows Mobile devices to upload pictures to my Surface application. My work on the handhelds was done in collaboration with Tim April and Dylan Fransway.

My picture sharing application allows users to upload pictures on the Surface from their mobile devices (or any wireless device) and with the help of some very simple and intuitive gestures, the pictures can be moved, zoomed and played with. The users can make collages by manipulating and placing pictures anywhere on the Surface and taking a screenshot of the screen. The application allows users to transfer pictures to other users(thus transferring images to their Mobile devices), send them to Bluetooth enabled devices or upload them to web based services such as Flickr. With more and more mobile devices and cameras having a built-in GPS, pictures today are usually geo-tagged. Taking advantage of this fact, geo-tagged images can be viewed on a virtual globe. The Surface environment allows interaction with Microsoft’s virtual globe and thus storytelling can be made more immersive and interesting with our application.

Check out the video feature a demo of our Picture Sharing Application in which two users exchange images, create a collage and upload a few pictures to Flickr:

Pilot studies indicate that the photo sharing application is easy to use. The gestures required are natural, intuitive and effective. Digging deeper, the application presents very low workload statistics as compared to when the same functionalities are achieved in conventional plug and play picture sharing scenario. Note that my work can be extended to include features from the work of Mike Farrar on voice tagging and Trupti Telang’s Multi Touch Dispatch System.

Jatin Matani

Researchers at the Massachusetts Institute of Technology have developed a device that allows a user to transform virtually any surface into a multitouch computer. The name of this new technology is Project LuminAR, which is basically an embedded computer in a lamp. The LuminAR bulb contains a pico projector, a camera, and a computer with wireless connectivity - all packaged into an area not much larger than a digital camera. The LuminAR bulb uses a pico projector to display a computer interface (which would normally be seen on a liquid crystal display) onto any flat surface. A built-in camera detects and interprets movement from the user, and sends that data to the computer. The user can navigate and type on the interface using gestures and a graphical keyboard. This reduces the need for hardware such as a mouse, keyboard, and display that are necessities on any desktop or laptop PC. Another great feature of this device is that it plugs into a robotic lamp which has the ability to move around based on the user’s movements and gestures. The LuminAR bulb can also be installed in any household lamp.

Take a look at the original article from the IEEE here.

Devin Mullen

The image above is the combination of a transparent FTIR-based multi-touch surface and a motion control board, such as the one from Nintendo BalanceBoard, to create a navigation metaphor. Dimitar Valkov, Dr. Frank Steinicke, Gerd Bruder and Prof. Dr. Klaus Hinrichs built this while working for the Visualization and Computer Graphics Research Group at the University of Munster. They created it as an easier way to navigate through a 3D virtual world. It allows the users to visualize movement through tight spaces such as caves where actual locomotion would be difficult. While this illustration includes wheels, the user of the device is not actually moving on wheels. The wheels are just a metaphor for the movement controlled by the user with hand and foot gestures.

I was drawn to this article because of the multi-touch surface similar to the one we have here at Project54, although it is not the main focus of the apparatus. The key to this navigational device is the use of a motion control board because it’s less strenuous to use feet to specify direction and speed than continuously using hands. They have presented this device to six subjects who were able to quickly adapt to using their feet and provided positive feedback on the device. The users appeared to have a natural ability to incorporate the use of feet and hand gestures for navigating in a virtual environment. With intuitive ease of use, reduced strenuous and continuous motion control requirements, the device seems to have great potential in becoming incorporated in more interaction metaphors.

Carol Perkins

These Lycra gloves are part of a new MIT prototype for a new gesture-based computing system. Graduate student Robert Wang and Associate Professor Jovan Popović developed this system that improves upon other techniques that use interfaces that adapt to human movement, such as the Nintendo Wii. Lycra gloves made of irregularly shaped patches are part of a system that rapidly produces 3-D images of the gloves on the screen using a new algorithm to produce images in a fraction of a second. Rather than incessantly calculating individual sets of data to determine the position of a hand, instead the algorithm searches through a database of various hand positions and selects the corresponding digital model.

I think it’s especially interesting that the motions used to control the screen on a Multi-Touch Table, in my experience, are very similar to the motions used with this gesture-based system in a virtual world, as shown in the video below. This prototype is faster and less expensive than the current methods to track the movements of the human body, and has the potential to increase the usage of such technology in areas like evaluations of athletic performances, animation, and of course video games.

Erika Swanson

I found a paper the other day, which utilizes a multitouch interface for controlling and communicating with tasking robots. The device used is a Dell tablet PC, which is mobile enough for the in-field application. The application allows the user to choose a robot and specify a task: go to X, perform reconnaissance, or cross dangerous area. The existing interface below shows that it is not touch friendly, due to the pull down menus and small buttons.

The multitouch interface was redesigned to solve some of the precision issues with the existing interface. Firstly, Windows 7 was used to get better multitouch capabilities. The buttons were made larger, pan and pinch gestures were utilized, and other tools were changed to benefit from touch interactions.

The study consisted of performing several tasks with each interface. The main metric was the speed of completion of each task, and NASA TLX surveys were also used to measure loads with each interface. The results favor the new multitouch interface greatly. This seems like a similar application and study as we are headed toward here at the P54 lab.

Mike Litchfield

Many times when I go to Boston I try to pay a visit to a little shrine of technology called the Sony store in the Prudential Center. They always have some cool gadgetry on display. I was not disappointed a few weeks ago either. This time it was an all-in-one desktop computer that caught my eye, the Sony Vaio L Series. The best part: it is multitouch enabled running Windows 7.

Here at the lab we are very interested in multitouch technology as reported before. Our research is made even more relevant with the commercialization of affordable touch computers. Other companies have also come up with multitouch desktop systems for personal use: HP, Dell, etc. But Sony always puts a bit of extra effort in making their products even more stylish. And with the advent of Windows 7, an inherently multitouch operating system, touch computers start to make perfect sense. The Vaio L also dubs as an HDTV and a Blue-ray disk player/recorder.

I quickly double tapped the virtual globe application on the desktop. The Virtual Earth / Bing Maps style environment popped up quickly. I was multitouch navigating to my home town in no time. The feel of touching the computer screen was a little bit strange, because until now we all tried to keep fingers off from our beloved monitors. The feel of it was sleek, like our TouchKit table and unlike the Microsoft Surface, which is textured. All in all it was a very pleasant experience using this computer. It is going to be interesting to see how this new concept of human computer interaction will change the perception and use of desktop computers in the coming years.

Oszkar

Last Friday, a few of us from P54 attended a day long student conference geared towards human factors and ergonomics. The conference was very interesting, showcasing about 20 presentations dealing with various fields. Three of us from P54 (only ones from NH) gave presentations on our current research. I gave my presentation on our research on the multitouch display with geotagged & oriented photos.

It was a valuable experience for me, since I have not yet given a presentation to so many people whom I did not know. Of course, I was nervous at first, but it went away as my presentation started, and went well overall. I’m sure I will have to make more of these types of presentations in the future, so it was very good practice. Aside from my presentation, I also enjoyed a lot of other things from the conference. One of the other presentations was focused on controlling fully automated search and destroy robots in the field, and how often the person controlling them should reassess the goals and priorities of the robots.

Also the girl in the picture (one of three presenting) had an amazing voice. We all agreed she should be on radio or something, or a professional presenter. Mark and Zelijko gave awesome presentations too, and we all enjoyed some Friday evening rush hour traffic from Boston. It was great!

Mike Litchfield

A few days ago a new Microsoft Surface arrived to our labs. It is a great tool for user interaction studies. It’s surface material has a very pleasant non-stickiy textured feel to it.

Touch interactions on the Surface are recognized by multiple underlying infra-red cameras and transformed into user actions. The image is rendered by a projector located underneath the tabletop. The Surface is a really exciting user interface since it allows incredibly fast multi-touch and multi-user interactions. Not only can it detect the location of a finger on the tabletop, but also its orientation based on the shape of the infra-red blob.

We hope that this interface will allow researchers as well as students at UNH ECE to develop exciting new user experiences.

Oszkar

Apple recently unveiled one of its newest creations: a mouse that responds to gestures and finger movements instead of the usual clicks and scroll wheel that come standard on a regular mouse. Dubbed “Magic Mouse“, this mouse uses a special optical laser that can read movement on any surface. This means the user does not have to have a mouse pad; the Magic Mouse can be used on just about any surface. One of the best features of this mouse is that it allows the user to have many different mouse configurations in one. For example, a left handed person can configure the mouse so that right click will be on the left side of the mouse, which fits a left handed person better. The mouse allows its user to click and double click anywhere on the touch surface, as well as utilize a scroll function that is harnessed by dragging a finger along the surface of the mouse. Another great feature of this mouse is that it allows a user to hold down the control key on the keyboard and drag a finger along the touchpad to zoom in. The Magic Mouse also offers a “momentum scrolling” option, where the user can increase the scrolling speed of the mouse by dragging their finger faster or slower along the touchpad.

 Devin Mullen

As my colleagues Zeljko and Mike have already blogged about, we attended the 2009 Ubiquitous Computing Conference in Orlando, Florida. It was a very interesting gathering with some vanguard research topics. For me, the one-minute madness was a new and exciting experience. Poster authors have to present their research in a single minute. At prior conferences I observed two distinct classes of one-minute presentations: serious ones and funny ones. The purpose of the first class is obvious, while the second type tries to convince people to visit the poster stand by using some witty slides and usually an open question that would be answered at the time of visit. For this conference I decided to try my luck with the funny presentation. Here are my slides, so you can judge for yourself:

The following text accompanies the slides:

Slide2: Everyone likes to take fun vacation pictures.
Slide3: But an hour long slide show of these pictures might not be as much fun for your friends as it is for you.
Slide4: So how can you stop your friends from falling asleep? We propose a new storytelling environment which integrates 3 new technologies: GPS enabled smart phones, virtual globe applications and multitouch interaction.
Slide5: This environment could create a much more enjoyable user experience for your friends compared to a slide show. It might even be the killer app for multitouch!
Slide6: If you would like to know more about our research please come by to see our poster and hear about some much more serious applications.

Acknowledements to Hyomin Kim and Alex Vapirev for their great acting skills shown in the slides.

Here you can also see the layout of our poster. If you click on it, a larger view will appear.

Let me know if you have any comments on the presentation and poster. Thanks!

Oszkar