I’m reporting today from the NECHFES Student Conference held this year at the Northeastern University in Boston. Here you can see our participants: Matt, Mark, Zeljko, Nemanja and Mike. Alex was also present.

So far we have heard very interesting driving simulator talks from NEU students. Hui Cai presented his results on estimating driver emotional state using various physical measurements (heart rate variability, skin conductance, etc.)

After him, Z. Yin presented his study on driving simulator sickness. He has found that, optical flow affects largely this type of motion sickness.

That’s all for now. We will have more posts about today’s very interesting conference.

Oszkar

The Fall 2008 e-Newsletter of the Speech and Language Technical Committee of the IEEE Signal Processing Society includes an article about the driving simulator-based research going on within Project54. Thanks to Mike Seltzer for his help with editing the article.

Btw, the image that appears in the article was created by Alex Shyrokov and may be included in an upcoming book by Matthias Wölfel and John McDonough entitled Distand Speech Recognition.

Andrew Kun

Finally, the Project54 video for the IACP is finished. This meeting is a major conference and exposition organized by the International Association of Chiefs of Police happening in San Diego this year. As the name says it gathers top police officials from all over the world. Project54 will have its booth there, presenting the P54 system and our driving simulator. This is also the topic of our newly made video: the driving simulator and its capabilities for in-car device evaluation. Here it is:

I would like to thank Nate Purmort, the lead role actor/narrator, for his patience with taking, retaking and re-retaking scenes for the video, and professors Andrew Kun and Thomas Miller for their suggestions and guidance.

Oszkar

Often times we include a picture of our simulator in our publications. It turned out to be an interesting problem, because if we want to show a cab - we need light; if we want to show the projected scenery - we need to have a dark room:

The obvious solution is to compose parts that we need from both images:

But we decided to take it a little further and use actual screen shots from the simulation instead of the pictures of the screen in a dark room:

Gimp has a perspective tool that is designed to do something like that and here is the result:

The extra benefit of this approach is that it’s possible to put any scenery into the picture using a template, without taking pictures, but by simply taking screen shots:

Alexander Shyrokov

Hello ecebloggers,

I’m very proud today because I’m writing about my first experiment. The experiment builds on two of our previous studies. In one of them Zeljko Medenica found that when speech recognition performance is poor, having to use a push-to-talk button results in worse driving performance than using ambient recognition (you can find the paper on his study here). In his thesis Oskar Palinko found that drivers divert their visual attention from the outside world when using the push-to-talk button. This happens because drivers often cast a glance away from the road in order to locate the push-to-talk button before operating it.

A simple solution to this latter problem may be training drivers not to look for the push-to-talk button. However, just like users of consumer electronics in general, drivers are not willing to be trained for long periods of time in order to use an in-car device. The goal of this study is to determine if simple training, such as verbal instructions issued before driving, would improve drivers’ visual attention directed at the road ahead.

The experiment setup is shown in the picture below.

The experiment involves our high fidelity driving simulator and eye tracker. I’m also using three cameras: one to record subjects’ head movements, one to record hand positions while driving, and one to record the eye tracker laptop screen. The eye tracker laptop shows the video captured by the two eye tracking cameras but there is no elegant way of saving this video.

I’ve started the experiments this week with Oskar’s help so I’ll be able to post about results very soon!

Have a good one,

Nemanja Memarovic

The last few weeks I have been working on designing a new driving experiment which would compare different user interfaces for entering license plate numbers into the police cruiser system for checking records. The two user interfaces are manual and speech. To be able to achieve this, license plates had to be added into our simulations. I have built on the results of Zeljko Medenica’s work to put in license plates as textures in the DriveSafety simulator computers. These worked only for some car models, because of the way textures are applied to them. Here is an example of successful applications:

The plates had to be made larger than in real life, because of the limited resolution of the projection system (1024×768). This way, the plates became readable from moderate distances and still preserved a realistic feel.
The experiment hypothesizes that a voice input system would have a beneficial effect on driving performance compared to using a manual interface.

We would also research if completion time would be longer for either of the two methods. Further, looking at the eye-tracker data, it would be possible to say which method demands more visual attention. My bet is on the manual UI. We will post all interesting results on this blog.

Oszkar Palinko

Probably everybody has at least some experience with the GPS-based personal navigation aids. They usually provide directions using voice prompts and information displayed on LCD screens. While such devices appear to be less distracting to use than paper directions, in-car displays may distract drivers from their primary task, driving.

In order to assess how different in-car navigation aids affect driving performance and visual attention, we conducted an experiment using our high-fidelity driving simulator. The simulator is also equipped with an eye-tracker which provides information about subjects’ visual attention. The following picture illustrates how the experimental setup looked like.

There were three navigation aids that subjects tested in this experiment: paper directions (turn-by-turn directions with a map printed on a sheet of paper), standard PND directions (a map displayed on an LCD screen with turn-by-turn directions delivered through voice prompts), and voice-only directions (just voice-delivered turn-by-turn directions). Participants were driving on a two-lane city road with markings and a light traffic was introduced. The following picture shows one snapshot from the simulation.

We recorded three measures of driving performance from our driving simulator: lane position, steering wheel angle, and velocity. Higher variances of these variables represent worse driving performance. We also calculated the percent dwell time on the outside world for each subject using the data collected by the eye-tracker.

Our results showed that using paper directions degrades driving performance (lane position variance, steering wheel angle variance, and the mean velocity were significantly different between the paper and other navigation aids) and visual attention significantly more than using either a navigation device that provides voice prompts with a map or a device that provides voice prompts only.

Regarding the visual attention, the time participants spent looking at the road ahead was significantly higher when using the voice-only aid than when using either the paper or the voice and visual aid. On average participants spent around 94% of time looking at the road when using the voice only aid and around 89% when using the voice and visual aid.

While these findings supported our hypotheses, one interesting thing was that the majority of the participants expressed that they would prefer using the navigation aid which provides both the visual and voice directions. Based on this, we propose for our future work to model the glancing behavior of the drivers, which would enable us to predict when one would require next instruction, so we would be able to issue a voice prompt. This in turn would enable drivers to keep their eyes on the road at all times, while still being able to have the next instruction in a timely manner.

Lots of other experiments regarding this issue will follow, so we’ll keep you posted!

Zeljko Medenica

Hello ecebloggers,

In my previous post I wrote about Naturalistic Driving and Analysis Symposium at Virginia Tech University. The most interesting event on the symposium was demonstration of the Smart road, a research facility managed by Virginia Tech Transportation Institute. It is a 2.2 mile twp-lane road with a capability to produce a wide range of weather conditions (see pictures below).

VTTI’s Smart road (just the bridge)

Fog

Rain

Smart road uses 500,000 gallon water tank which supplies 75 water towers to produce rain, a fog like mist, or snow. It is also equipped with variable lighting to study effects of lighting technologies on visibility (see pictures below).

Smart road’s lighting system

Rain by night

Of course, everything needs to be coordinated (weather, traffic, lighting). This is done in control room.

Control room

Control room (downstairs)

VTTI uses real cars in their experiments on the Smart road. They are measuring speed, traveled distance, lighting conditions… Almost anything you can think of. Examples of the equipment you can find in the cars are in the pictures below.

Thousands of hours of on road research have been conducted on th Smart road. It is intended to become a part of the public transportation system connecting Blacksburg, VA to Interstate 81.

Pictures from the conference can be found here.

Have a good one,

Nemanja Memarovic

Hi Ecebloggers,

Around 2 weeks ago, I completed my summer internship at UNH. During a roughly two month timespan, I worked on how interacting with a keyboard and an attached display in the vehicle might effect the driver performance. At certain fixed locations, a text appeared on the screen for a few seconds and the subject was required to type that text using the keyboard. However, to make the study interesting, some events were designed that would be evoked when the subject would be diverted looking at the keyboard. One of the five designed events, which I call “A barrel falling from a truck”, was quite interesting. The name of the event itself indicates what actually happens during the event. A barrel from a moving truck would fall just short of the subject when he came close to the truck driving in the same lane. Here is a video that demonstrates the incident as it occurs in the Project54 driving simulator.

It was actually a little arduous to make a falling barrel look realistic as such a support is not provided in Hyperdrive , the software used for designing the scenarios. A barrel, as the ecebloggers might know, is a static entity. It was not possible to keep it in motion with the truck, a kinematic entity. I tried to create a barrel with dynamic properties and then relocate it along the roadway at a certain height above it. However, while Hyperdrive relocated the barrel along the roadway, the barrel could not be placed above the level of the roadway. A bug was noticed in Hyperdrive when the log files created indicated that the barrel was moved above the roadway level as well when actually it didn’t happen. I contacted Hyperdrive forums support and they clarified that all the kinematic and dynamic entities in the scenario were so designed to stay adhered to the roadway and such a displacement above the road was not possible for the barrel during the simulation.

To overcome this problem, I estimated the possible positions of the barrel, assuming it was in motion with the truck, and placed the barrels in midair at those specific coordinates. A script was then created to implement the planned scene. Initially, all the barrels placed in midair were made invisible and as the script progressed, the barrels were made to switch between visible and invisible states in a consecutive fashion to achieve synchronisation with the motion of the truck. Finally, when the subject came within a suitable distance relative to the truck, a separate set of barrels set for a projectile-like motion was triggered in the same visible and invisible fashion to achieve an appropriate declivity. Since the barrels that were pre-placed were limited and not certain,  the desired effect might not be obtained in case the number of barrels in midair reduced to zero when a subject approached the truck too slowly or he went on to a different lane. Thus, the person had to stay in that lane for achieving the desired effect, which otherwise, is quite a cool event to watch and enjoy.

Puneet Lakhanpal

A couple of weeks ago I spent a day visiting Microsoft Research and gave a talk on our lab’s work on in-car speech user interfaces. My hosts were two Microsoft researchers, Tim Paek and Ivan Tashev. I was also accompanied by one of my graduate students, Oszkar Palinko.

As part of the visit Ivan gave us a tour of two Microsoft Research labs. First we visited the acoustic anechoic chamber. This was a really neat experience. Check out the video below in which Ivan introduces this lab.

While it’s impossible to show in a video how quiet it gets in the anechoic chamber, check out the following video which shows me clapping inside the chamber. You may be able to notice that there’s no echo (thus anechoic!). In the chamber you could also notice this just by listening to people and hearing the dramatic change in perceived speaking volume as they (or you) turned.

Ivan also showed us his newly installed driving simulator. The simulator is made by STI and Ivan plans to use it very soon in his ongoing work on Commute UX, a “telephone dialog system for location-based services.” Oszkar and I had a chance to test the simulator and some of the cool “traps” or scenarios in which a distracted driver may end up in an accident. Here’s a video of me testing the MSR driving simulator.

I uploaded some picture from this visit here.

Thanks Ivan for the tour and thanks to both Ivan and Tim for hosting.

Andrew Kun