With Windows 7 supporting multi-touch gestures and devices you can now make your own FTIR, DI or even a simple webcam based system to communicate directly with Windows 7. Thus, making your complete OS work with multi-touch.

Community Core Vision (formerly known as tbeta) is program which gets feed from your camera and applies filters for blob tracking. This way you can actually make very cheap multi-touch interfaces, you can find a simple tutorial for that here. After that it sends the fingers or touch data using the TUIO protocol. Now all we need is a driver to convert those TUIO multi-touch signals into Windows 7 multi-touch messages. Here is a video of CCV demo and a link to the tutorial on how to set it up.

CCV Beta Preview (tbeta) from ~ on Vimeo.

After this is done follow the tutorial which uses another project called Multi-touch Vista. This project requires you to have Microsoft Visual Studio 2008. You can follow this tutorial and install the Windows 7 driver for Multi-touch Vista. If you follow the tutorial correctly you will be able to run your Multi-touch hardware on Windows 7 with all the gesture recognition done by the OS. You can try XPS Viewer, Photo Preview, Onscreen Keyboard and other applications to see the multi-touch in action (in other words you won’t have to use keyboard or a mouse to work on your computer). But again you need to have hardware for that. Here is a video taken by me demonstrating the multi-touch features of Windows 7.

Ankit Singh

During the fourth week of June 2009, the biannual Driving Assessment conference was held at the beautiful Big Sky Resort, Montana. This is one of the most important driving related conferences. Andrew Kun and I had a poster presentation here, with the following title: Comparison of the Effects of Two Push-to-Talk Button Implementations on Driver Hand Position and Visual Attention. The poster session went very well with lots of interested visitors.

There were also many other interesting presentations and posters. Most of the participants came from North America and interestingly, Sweden. Also, driving assessment related companies were present. Realtime Technologies and SeeingMachines showed a new feature of the RTI simulator, which allows constant monitoring of the object in the simulation at which the driver is looking at.

The nature around Big Sky is beautiful. It is primarily a ski center, but there is plenty to see during the summer too. Its proximity to Yellowstone National Park gives it even more value.

Oszkar

I’ve been working with the Touchkit for a while and have found that there are some tracking issues with the hardware. Particularly, when sliding a finger across the surface the tracker can lose the finger and pick it back up later. This issue causes a problem when writing an application for the Touchkit, since the programmer will expect an action like this to be continuous.

Testing:

I performed 65 trials of dragging my finger across the Touchkit in one continuous motion with average pressure. I recorded the points of presses and drags and plotted them in Matlab, while I varied the lighting, background color, and sensitivity of the table and found that white background, lights on and 30 vision threshold gave the best results. Next, I did 15 more trials with these or similar settings, and calculated the distance and time between the last drag point recorded and a new touch for the optimal settings. Finally, I wrote an algorithm that uses a pointer to the default finger tracking object, which I can update when I find that a new touch is not the same as the previously lost one, based on its time and distance apart.

Results:

I found that with the optimal settings, 57% of the trials yielded one continuous touch. With a distance parameter of 25 pixels and a time of 40 ms in my algorithm the success rate would be 93% (only one failed trial). Below are two Matlab graphs of the same continuous touch. The first is plotted  using the default tracking object’s data and the second is plotted using my pointer’s data.

figure 1. Matlab graph of tracked data from the default object

figure 2. Matlab graph of tracked data from my pointer to the default object

The blue circles on the figure indicate a detected press. Notice the first figure shows several detected presses, while the second figure only detects one. Figure 2 shows that my algorithm correctly tracks the finger gesture.

Mike Litchfield

One upcoming necessity here at Project54 is the need to be able to scan licenses using a mobile phone camera. I designed an experiment involving many different camera capture formats and post camera capture image processing techniques to see which of these produced the most successful results using the DTKBarcode Barcode Reader SDK.

This scenario involved the following camera parameters:

• HTC Touch Pro
• 3MeagPixel Picture
• Default Saturation
• Default Sharpness
• Default Contrast
• Distance between camera and barcode: About 7 inches

Results:

These two barcodes are from the same picture, the bottom one (Figure 2) being the original. The only property physically different between them is the top right and bottom left corners of the top barcode were skewed two pixels inward using Adobe Photoshop CS3. The result of this skew is shown below (Figure 1). This minute change enabled the barcode to be read by the DTK Barcode Reader SDK.

Figure 1 : SUCCESSFUL Scan by the DTK Barcode Reader (Photoshop Skewing)

Figure 2: FAILED Scan by the DTK Barcode Reader (Original HTC Camera Photo)

Conclusion:

I took well over 300 pictures with varying settings. I took my best three cases (Grayscale, Sharpness +5, Saturation +5), each with only 50% success, and applied additional image processing, post capture, with Photoshop. The most successful Camera settings and Photoshop techniques were additional Saturation, Sharpening and Color Level manipulation, with none being much more successful than the others. The best I could raise the scan success rate to was 75%, for one test case, using a combination of all three techniques. Keep in mind that I was sitting, using two hands, and stabilizing the camera with my elbows on the table. The realistic case would be more along the lines of someone standing, holding the phone with one hand and the license with another, with no additional stability support. It seems with this margin of error, the DTK Barcode Software will not be able to meet our needs.

Perhaps someone else has an answer or suggestion to solve this problem?

Eric Ojala

Alyssa Shooshan, Senator Gregg’s chief of staff and Dean Hingson Senator Gregg’s new counsel, visited Project54 earlier this week. They toured our labs and the new Project54 garage (the location of our recent open house) and had a chance to see in person the results of Senator Gregg’s continued support for Project54. In the picture below you can see Alyssa and Dean (right) looking on as Michael Farrar demonstrates the Project54 handheld application.

Thanks for visiting Alyssa and Dean, we hope to see you again soon.

You can see more pictures from this visit on Flickr.

Andrew Kun

On Wednesday June 3rd the New Hampshire EPSCoR Conference and Proposal Writing Workshop was held at the Common Man Inn in Plymouth, NH.

EPSCoR stands for Experimental Project to Stimulate Competitive Research, which is not very descriptive of its purpose. The goal of this huge project is to stimulate research in science and engineering in states of the USA, which have lower concentration of research funding than others. Since 2004, New Hampshire also participates in this program. It was started as a project of the NSF (National Science Foundation) but since then other government agencies have also implemented their EPSCoR initiatives (e.g. NASA EPSCoR). As it is known, the NSF and similar institutions provide some of the most important funds for research in the country, therefore EPSCoR is also of vital importance for research labs as ours, Project54.

The conference featured a number of NSF officials who presented the most important funding aids that their institution provides. Dr. Uma Venkateswaran, the national Program Director of NSF EPSCoR gave an introductory presentation, talking about the ways they can help fund research labs in New Hampshire. The main vehicles of funding are Research Infrastructure Improvement, Co-funding with NSF Directorates and Conference and Workshop Funding.

After her, officials of different NSF Directorates held presentations of their fields. For us the most important might be the presentation of Dr. Suzanne Iacono from the CISE Directorate (Computer and Information Science and Engineering). Among other topics, she mentioned the Human Centered Computing initiative at her Directorate which funds projects aiming to improve usability and introduce human values into interaction with computers. Our work on researching in-car user interfaces, handhelds and mutitouch surfaces shares many common ideas with this program.

It is important to mention that NSF provides funding through a grant system awarded to successful proposals. The success rate of proposals is only 25% percent, but as we figured out at the conference one can improve his/her chances by elaborately studying all of NSF’ guidelines and the vast number of different programs. I found the conference to be very informative. It was also very encouraging to hear, that so much is invested in advancing science and technology in the US.

Oszkar

We often use photo-sharing websites like Flickr, Picasa, Panoramio to show our friends our travel photos and interesting places. Many of these websites have the extra feature to geo-tag photos by placing them on a map (signifying the place they were taken). They also give kml feeds, which are files we can open in Google Earth and can actually see our photos.

“LIVE” TOUR USING GOOGLE EARTH

If you have a GPS enabled phone with a camera, you can show your friends photos almost immediately after you have clicked them on Google Earth. For that you will require a GPS-tagging app like GPS Today on your Phone and an account in Shozu linked to your account in Flickr. When you sign up for Shozu and install the app on your phone you can add sites like Flickr (you will have to authorize first) to which you can upload photos just after you have taken them. GPS today tags your photos with your current GPS coordinate. So, your friends just have to subscribe to your photostream kml feed, open with Google Earth and change the refresh in the file properties to a desirable value. This way your friends and family can actually track your adventure “live”.

EXTRA FEATURES YOU COULD IMPLEMENT

There are many other features which we can utilize in Google Earth besides just showing the location of the photograph:

1.     Orientation: We can also add the orientation tags like heading, tilt and roll to our photos specifying the direction in which the photo was taken. This becomes helpful when we have a lot of photos on one particular location and identifying them is a difficult task.

2.    TimeStamp: When people have a lot of photos for a small location it sometimes become untidy or confusing to see all the photos. In Google Earth the Time Slider creates a 4th dimension for viewing the photos. We can view photos over a user specified time interval which can be in seconds and even years. Here is a Yahoo Pipe (used to generate feeds for different websites) which gives the user the same kml feed which the Flickr provides, but adds the extra feature of TimeStamp and removes the limitation of 20 photos which the official Flickr feed has. You just have to give the user’s ID in the second field (e.g. - the ID of the user with the Flickr URL www.flickr.com/photos/31413502@N06/ is ID 31413502@N06) and after running the Pipe, select “Get as kml” option. (Sometimes it may not work the first time, try running the pipe a couple of times to make it work)

The photo below shows two pics taken almost at the same location but marked differently with orientation tag, and also showing a Timeline from the first pic to the second pic.

Do try these exiting features of Google Earth.

Ankit Singh