Last Friday Oskar Palinko defended his MS thesis. Oszkar’s thesis was centered around the cool push-to-talk (PTT) glove he has designed.

Oszkar ran a rather large user study (24 participants) to evaluate if the PTT glove outperforms a fixed PTT button. While in comparing driving performance when using the two PTT solutions Oszkar didn’t find a main effect, he did find that the experiment participants looked down at the steering wheel more often when using the fixed PTT. Is this a problem? Maybe. While the total amount of time subjects spent looking at the steering wheel when using the fixed PTT button amounted to about 1% of the total experiment time, the average fixation was around 300 ms long. If such a fixation came at the wrong time (e.g. at the moment a leading vehicle started to brake), this could be a problem.

Congratulations Oszkar on a job well done!

Andrew Kun

Last week Tim Paek of Microsoft Research visited the Project54 lab at UNH. Tim and I have been collaborating for about two years now, and this summer Tim is the supervisor of my PhD student, Zeljko Medenica, during Zeljko’s summer internship with MSR.

As part of his visit Tim gave a talk on his research in the field of mobile speech interaction. The talk covered three topics. First, Tim discussed his work on the Voice Command. Tim worked on utilizing user models to reduce the semantic error rate. Next, he talked about using reinforcement learning in a voice enabled browser. Finally, Tim talked about a mobile voice search application. The application provides intuitive ways to help the search application find the right answer. One way this is accomplished is by allowing multimodal refinements to the search. E.g. the user can help the speech recognizer by typing in the first letters of a problem word in an utterance. Another way is by allowing uncertain queries in the search (you can say “something” as part of the search to indicate you’re not sure what the exact search term should be). As part of the talk Tim gave a live demo of the mobile voice search application:

The talk was very interesting and the demo was impressive - thanks Tim! For more pictures click here.

Andrew Kun

A couple of weeks ago I spent a day visiting Microsoft Research and gave a talk on our lab’s work on in-car speech user interfaces. My hosts were two Microsoft researchers, Tim Paek and Ivan Tashev. I was also accompanied by one of my graduate students, Oszkar Palinko.

As part of the visit Ivan gave us a tour of two Microsoft Research labs. First we visited the acoustic anechoic chamber. This was a really neat experience. Check out the video below in which Ivan introduces this lab.

While it’s impossible to show in a video how quiet it gets in the anechoic chamber, check out the following video which shows me clapping inside the chamber. You may be able to notice that there’s no echo (thus anechoic!). In the chamber you could also notice this just by listening to people and hearing the dramatic change in perceived speaking volume as they (or you) turned.

Ivan also showed us his newly installed driving simulator. The simulator is made by STI and Ivan plans to use it very soon in his ongoing work on Commute UX, a “telephone dialog system for location-based services.” Oszkar and I had a chance to test the simulator and some of the cool “traps” or scenarios in which a distracted driver may end up in an accident. Here’s a video of me testing the MSR driving simulator.

I uploaded some picture from this visit here.

Thanks Ivan for the tour and thanks to both Ivan and Tim for hosting.

Andrew Kun

This year’s Young Researchers’ Roundtable on Spoken Dialog Systems was a great place to share ideas with fellow speech research enthusiasts. Lots of attention was devoted to the design of spoken dialog systems in general (as the name of the event might suggest), but also to more specific areas, as automotive speech user interfaces. Besides the main program, the pleasant meal breaks provided a great environment for people to discuss research ideas in an informal manner.

I had some very interesting conversations concerning automotive topics during these breaks with Ben Reaves from Toyota ITC, Stefan Hamerich from Harman/Becker and Zeljko Medenica from Project54/UNH. We discussed the state-of-the-art of automotive speech user interfaces and where the field is headed. Most of us heard about Microsoft’s Sync Technology for Ford vehicles, thanks to their advertising campaign in the USA, but other big auto makers are also selling their cars equipped with speech recognition systems (e.g. Toyota, Honda, etc). Ben proposed, that now is the time to set standards for automotive speech user interfaces, which could be accepted by all relevant parties in business and research. We all agreed that the accepted standard might not be the best one (see VHS vs. Betamax), but still it could be very beneficial to the field, by focusing its development.

The poster session was cleverly piggy-backed onto the afternoon coffee break. This way people didn’t even notice that their minds were “on-duty” even during “recess”. I presented a poster on my ongoing research concerning push-to-talk button solutions for in-car speech user interfaces. It drew a bigger crowd than I imagined. Professor Alex Rudnicky from CMU was inquiring about the premises and methods of my research. Then, the automotive specialists, Stefan and Ben were joined by several other participants in discussing details about the poster with me.

Stefan Hamerich, Ben Reaves, Antoine Raux, Oskar Palinko, Milica Gasic

We had a very good conversation on the advantages/disadvantages of high fidelity driving simulators in automotive research. They can provide lots of measured variables (lane position, steering wheel angle, distances, speeds, etc.), but in the same time researchers must cope with their possible undesired effects (e.g. motion sickness).

I found out, that informal discussions are a very effective way of sharing ideas within small groups of people. I enjoyed a lot to talk to fellow researchers about common interests at YRRSDS’08.

Oskar Palinko
Project54, UNH

On Tuesday, Ian Cassias defended his MS thesis. Ian worked in the field of telematics and he was interested in three topics: remote diagnostics of vehicles, vehicle fleet management and traffic monitoring.

My favorite part of Ian’s thesis is his work on traffic monitoring. Ian looked at how the police radar could be used to estimate traffic volume for a given segment of road and how fast the traffic is moving. In order to do this, Ian looked at the number of car velocity readings the radar reports, and the actual values reported. From these numbers he attempted to characterize road conditions along two axes: the slow-fast axis and the light traffic-heavy traffic axis. Ian’s pilot study shows that the police radar could very well be used to monitor traffic. If we can further develop this system we could make police cruisers into a set of roaming traffic probes. Data from the cruisers could be used for traffic prediction and, if wireless communication is available, for real-time traffic reports.

Nice work Ian!

Andrew Kun

Last week Pasha Melnyk defended his PhD dissertation, entitled “Biologically inspired composite image sensor for deep field target tracking.”

Pasha was interested in the problem of deep field tracking, or more specifically, he was interested in using image sensors to track objects from when they are very far from the observer all the way into the near field, when they are close to the observer. Pasha proposed a system in which multiple image sensors of different focal lengths create a composite image sensor that can achieve this type of tracking. However, he then ran into the problem of how to recognize and track objects in this new composite image. Will objects have different characteristics as they move through space and get picked up by different parts of the composite sensor? Pasha found an elegant solution to this problem. He described the composite image by nesting the log polar representations of individual cameras. One result is that objects do not significantly change shape as they are tracked by the multiple cameras.

Pasha successfully applied his idea to the problem of vehicle tracking. He was able to track vehicles from several hundred meters and then capture license plates as the vehicles drove by. The videos of this were really impressive.

Great job Pasha (and Rich Messner, Pasha’s advisor).

Andrew Kun

Ray Kurzweil is well known for his work on optical character recognition, text-to-speech synthesis and speech recognition. Recently, he was invited by the National Academy of Engineering to be part of a committee charged with outlining the grand challenges for engineering in the 21st century. Last Thursday, Ray gave a talk on this subject at the Broad Institute in Cambridge, MA, and four of my students and I went to hear him speak.

A good portion of the 2 hour talk was devoted to two ideas. One is that careful analysis of data can allow you to predict the growth of particular technologies. The second is that the growth of most, if not all, (successful) technologies is exponential. E.g., the number of Internet hosts has risen exponentially over the years (see slide below). Note that the slide tells us that this trend wasn’t perceptibly altered by the boom and bust of the dot-coms. Ray’s take on this: Wall Street didn’t realize that the Internet was growing exponentially, extrapolated the seemingly linear growth of the early exponential curve and came to the wrong conclusion that the Internet is not a viable place to make money.

The rest of the talk was devoted to several technologies that in Ray’s opinion will make a difference in the next 20-25 years (note that he prefers to talk about this shorter time horizon rather than the next century). One set of technologies is related to biology. To frame this discussion, it’s worth taking in this quote:

“As remarkable as biology is, it’s full of downsides, and needs to be improved upon.”

That’s the spirit! So how will we do this? Well, in Ray’s opinion, our computational powers are rising fast enough that by 2029 we’ll have computers that will be able to pass the Turing test (btw, this will be OK, we’ll use them to extend our own abilities and the whole thing will not result in some takeover by the machines). Our abilities to simulate biological processes are already very powerful, and will only get more powerful, which in turn will allow us to engineer new cures rapidly. We’re in the process of developing miniature robots to be deployed in our bodies to fight disease and generally make us stronger (e.g. provide us with an extra boost of oxygen). And we also need to develop a way to quickly respond to any viral outbreak (man-made or natural).

Ray also predicted that ubicomp will play a central role in our future, since we’ll be “online all the time with augmented reality.” This was fun to hear for my students and me, since our Project54 research relies heavily on the ubicomp field. And, talking about students, the topic of education also came up. Ray pointed out that “passion, desire and skill to learn is what we nan give our students.” Very well put.

Overall, the talk was inspirational. Now I want to read this book! Thanks Ray,

Andrew Kun

p.s. See more pictures from the event here - thanks Ivan!

Last week, Lieutenant Mark Liebl of the NH State Police started testing new versions of the Project54 navigation application and the Project54 handheld application.

Jon Oppelaar installed a USB GPS unit (GlobalSat BU-353) in Mark’s cruiser. This unit performs significantly better than an older GPS unit that was intalled several years ago, and it will enable Mark to test Jon’s navigation application. The application integrates Microsoft MapPoint into the Project54 environment. Here’s Jon (in the passenger seat) tweaking software settings in Mark’s cruiser during the installation of the USB GPS unit:

Andras Fekete worked on updating the software that allows a handheld unit to control the in-car devices (see picture below). Andras has already successfully deployed handheld technology in the Lee, NH police department, which we describe in a paper that will be presented at IE08. We’re now looking forward to getting Mark’s feedback about Project54 handheld software.

While Jon and Andras were installing new hardware and software, Mike Farrar talked to Mark about using the Project54 handheld computer’s imager, not only as a 2D barcode reader (e.g. for driver licenses), but as a camera. Mike is developing software that will allow officers to take pictures and tag them. Tagging can be done using voice commands, which should speed up data entry. It’s worth mentioning that Mike is developing his software using the Symbol MC-70, which comes with a cell modem. This opens new opportunities for getting data to officers in the field, especially in urban areas. In fact, the new version of the MC-70 also has a built-in GPS unit, again presenting interesting opportunities for law enforcement applications.

Thanks for testing all this harware and software Mark!

Andrew Kun

The Project54 lab will have three papers at the Intelligent Environments 2008 conference (to be held in Seattle, WA, July 21 and 22).

Oszkar Palinko and I have a short paper and a demo paper, both related to our research on how press-to-talk buttons influence driving performance when used with in-car speech user interfaces. The short paper describes Oszkar’s PTT glove and the demo will show his work on using pressure sensors mounted on a steering wheel as a PTT solution.

Andras Fekete and I have a short paper which describes the deployment of a Project54 handheld computer in the Lee, NH police department.

Great job Andras and Oszkar!

Andrew Kun

Congratulations to Matt and Mark on receiving summer undergradate research fellowships! They will work with me on a ubicomp project that Matt described in an earlier post.

Andrew Kun