As a student intern under Andrew Kun at the University of New Hampshire for the summer of 2010, I looked into what the Surface offers. For the majority of my work, I explored the world of Natural User Interfaceoffered by the Surface by developing an application which allows users to share pictures in an easy and appealing environment. Apart from this, I also worked on developing an application that allows Windows Mobile devices to upload pictures to my Surface application. My work on the handhelds was done in collaboration with Tim April and Dylan Fransway.

My picture sharing application allows users to upload pictures on the Surface from their mobile devices (or any wireless device) and with the help of some very simple and intuitive gestures, the pictures can be moved, zoomed and played with. The users can make collages by manipulating and placing pictures anywhere on the Surface and taking a screenshot of the screen. The application allows users to transfer pictures to other users(thus transferring images to their Mobile devices), send them to Bluetooth enabled devices or upload them to web based services such as Flickr. With more and more mobile devices and cameras having a built-in GPS, pictures today are usually geo-tagged. Taking advantage of this fact, geo-tagged images can be viewed on a virtual globe. The Surface environment allows interaction with Microsoft’s virtual globe and thus storytelling can be made more immersive and interesting with our application.

Check out the video feature a demo of our Picture Sharing Application in which two users exchange images, create a collage and upload a few pictures to Flickr:

Pilot studies indicate that the photo sharing application is easy to use. The gestures required are natural, intuitive and effective. Digging deeper, the application presents very low workload statistics as compared to when the same functionalities are achieved in conventional plug and play picture sharing scenario. Note that my work can be extended to include features from the work of Mike Farrar on voice tagging and Trupti Telang’s Multi Touch Dispatch System.

Jatin Matani

In the video above, John Underkoffler talks to enthusiasts at TED about the need to move away from the traditional, one machine for one human with one mouse and one screen flat display, to building machines that aid humans in the task of creation in multiple dimensions. He has taken a big step in this direction working at his company, Oblong Industries. He first displays a luminous room where all objects display and sense, and an optics prototyping workbench. Then he shows how he has brought the Minority Report data interface to life as a point-and-touch interface called g-speak. With this he demonstrates the apps for manipulating and analyzing pictures, combining traditional tabular data with 3D geospatial information, video editing, and utilizing explosive views of tools and objects using 6 degrees of control with a hand.

The set-up shown in the video seems bulky and non-dispersible but John Underkoffler claims that will quickly change. Oblongs aims at ubiquity, with g-speak making its way onto each lap-top, desktop, onto household items like microwaves, ovens, TVs, and then onto vehicle dashboards. It seems the g-speak has come a long way from merely making science fiction real at the MIT media lab, and has leaped into the realm of our everyday life. Although some of these ideas are being integrated by other companies, like the Microsoft’s Kinect, I think Oblong’s creations will be indispensable because of the co-located input and output space and the innovative apps.

Carol Perkins

-Erika Swanson

Carol Perkins

Mark Taipan

Not only do these glasses display information, but they also read the displayed data through cameras and send it back to the host computer. These glasses give the user the ability to display any information the user needs, less than an inch away. This technology could prove useful for situations ranging from reading a book, to viewing schematics or instructions when working on a project.

Devin Mullen

Last week, undergraduates from the Project54 lab visited the KEEPERS camp. KEEPERS stands for Kids Eager for Engineering Program with Elementary Research-based Science. The purpose of the camp is to get young children interested in engineering. One of the tours that the camp went on was a trip to Professor Wayne Smith’s Biomedical Engineering lab, where campers learned about how electrical engineers design prosthetic limbs that react to signals from the brain. Chris Bancroft, a recent graduate of the UNH ECE masters program, gave a demonstration to the campers of how the technology works. He attached electrical leads to his arm and explained to the campers that when a person with a prosthetic limb moves their arm, the brain sends signals to the muscle. That is where the leads pick up the signals and send it to a microcontroller. In the demonstration in the lab, Chris had the microcontroller attached to pneumatic pumps that forced air into a muscle-like air bladder. He explained that when the signals from the brain are picked up by the muscle, they are amplified and then sent to the microcontroller which decides which pumps to turn on and move the muscle, lifting a wooden arm.

Devin Mullen

A pointing device, such as a mouse, touchpad or the pointing stick, has become an essential component of the modern computer system. As the number of portable computer users steadily increases so does the need for a compact and highly portable pointing device. The problem evident with existing pointing devices is that an increase in the portability of the device often results in the decrease of performance of the device and/or the comfort for the user.

The MoGo Mouse BT^TM of Newton Peripherals and the Slim G4 Mouse of DaoKorea were developed to improve the portability of the mouse by reducing its thickness down to 5mm enabling it to be neatly stored inside a PC card slot. However reduced graspability was indicated as their weak point, which ensued due to their very slim form.

To address this usability-portability trade-off problem a 5 person team of the Department of Industrial Design of the Korea Advanced Institute of Science and Technology has developed the Inflatable Mouse by essentially attaching an inflatable balloon on top of a slim mouse. The Inflatable Mouse is a volume-adjustable user interface. It connects to the computer through a USB port which can also be used to power its air pump. The air pump inflates the balloon up to the volume of a familiar mouse as needed, and deflates it entirely so it could be stored in the PC card slot of a portable computer when not in use. Apart from the standard functionalities of a mouse it also provides a new input modality of squeezing the mouse by sensing the air pressure inside the balloon, and also has multiple touch sensors on the top and both sides of the mouse. Changing pressure can be used for scrolling, selecting from a list etc.

Since the volume of the mouse can be promptly changed it can also be used as an output/display device, where a variety of dynamic expressions can be made, for instance simulating the beating motion of the heart.

For further reading and a video, check out:

• Inflatable mouse CHI 2008 paper
• Inflatable mouse youtube video

David Filipovic

Researchers at the Massachusetts Institute of Technology have developed a device that allows a user to transform virtually any surface into a multitouch computer. The name of this new technology is Project LuminAR, which is basically an embedded computer in a lamp. The LuminAR bulb contains a pico projector, a camera, and a computer with wireless connectivity - all packaged into an area not much larger than a digital camera. The LuminAR bulb uses a pico projector to display a computer interface (which would normally be seen on a liquid crystal display) onto any flat surface. A built-in camera detects and interprets movement from the user, and sends that data to the computer. The user can navigate and type on the interface using gestures and a graphical keyboard. This reduces the need for hardware such as a mouse, keyboard, and display that are necessities on any desktop or laptop PC. Another great feature of this device is that it plugs into a robotic lamp which has the ability to move around based on the user’s movements and gestures. The LuminAR bulb can also be installed in any household lamp.

Take a look at the original article from the IEEE here.

Devin Mullen

Recently, users of Project54 have reported that the USB devices connected to their in-car computers at random would not power up correctly. Typically, a system reboot would solve the problem and power up the devices, but this situation is not ideal. The police officers using the Project54 application need all USB devices to consistently perform, and rebooting their computers is time consuming. Carol Perkins and I set up a diagnostic test in which 5 USB devices, including a GPS, wireless antenna, touch screen monitor, video adapter, and scanner, were set up in 5 different USB port configurations with a CoolGear USB hub. As shown in the figure below, two devices, the video adapter and the Symbol Scanner DS6707, had issues booting up. The Symbol Scanner did not power on 14% of the 50 trial runs. Of these 7 failures, twice the device booted initially, but eventually powered down without instruction. The figure below also shows that the power failures occurred while connected to 4 different ports, so we concluded that the reported problem was not due to any power allocation problems within the USB hub. When powered correctly, this scanner performs exceptionally well, so users will be asked to unplug and then replug the scanner when a power up is not successful. 

- Erika Swanson