Hello eceblogger readers! This time we have a new video about Project54 and some of the interesting things that some of us are working on here. The video shows a brief description of our driving simulator, how the Project54 system is being used and tested in the simulator and some research studies that are currently underway. The studies shown are in-car navigation, comparison of manual and speech interaction with mobile radios, and advanced interaction techniques such as human-human interaction experiments and work with the push-to-talk glove.

Let us know how you like the video or if you have any comments or ideas about it!

Zeljko Medenica

UPDATE 10/12/2007: The movie now has sound.

On August 23 Project54 was installed for the first time on a NH Marine Patrol boat. This installation is Project54’s first attempt to adapt to the marine environment and is the result of close collaboration with NH Marine Patrol officers.

For those of you who aren’t familiar with Project54, the system was created to improve the safety and effectiveness of police officers by introducing advanced technologies into police cruisers. The project has been very successful in this domain, however, the police vehicle environment differs greatly from that of the marine patrol boat. Correspondingly, these differences present a new set of challenges in the development of a Project54 solution.

Pictures of the marine patrol installation can be seen below. The top picture shows shows the ruggedized waterproof tablet that is being used to run the Project54 software, and the bottom picture shows the hardware that we have installed. This includes third party hardware (e.g. control boxes for lights) as well as Project54 hardware, such as the three smaller black boxes in the bottom right of the picture. These boxes act as translators between the CAN bus that Project54 uses for inter-device messaging, and the native messaging of the individual devices (e.g. RS-232). The physical installation is the work of Dan Jones, Project54 technician.

The system that was installed is only a prototype. There are a number of questions that remain about the effect the environment will have on the use of Project54. Currently, Marine Patrol officers are evaluating the system. As we receive feedback the system will evolve to meet officers’ needs and improve their effectiveness and safety. I’ll keep you posted on our progress.

Brad Bell

I have an HP Deskjet 6540 with a duplexer unit. When I upgraded my computer to Vista I could not use the duplexer. There was no way to select double-sided printing in programs such as Word and Acrobat. This problem can be fixed by going to Printers in the Control Panel, right-clicking on the HP printer, and selecting Properties. In Properties, go to the Device Settings pane and select Installed for Duplex Unit.

However, if Vista doesn’t recognize you as the owner of the printer (that is the user who has rights to administer it), this option will be grayed out. Interstingly, this can happen even if you’re logged in as an administrator (yes, this would be a bug). This is what happened to me when I reinstalled Vista (indexing stopped working - I’ll have another post on this subject soon). After reinstalling Vista, I was back to not being able to use double-sided printing. I tried to select Installed as explained above, but this option was not selectable. To fix this problem I first unplugged the printer. Next, I deleted it from the Printers list in Control Panel. Finally, I plugged it back in, which reinstalled the drivers. Once this happened, Vista allowed me to administer the printer and I could select the Installed option for the duplexing unit.

Andrew Kun

Another successful conference is over. This time professor Andrew Kun, Tim Paek of Microsoft Research, and I had a paper at Interspeech 2007 which took place in Antwerp, Belgium from 27^th to 31^st of August.
The conference was very well organized, and a lot of people from all over the world attended it. We saw many interesting lectures and papers. This picture will show how crowded it was during one of the receptions. Almost every day after the lectures and poster presentations were over, the hosts organized receptions with famous Belgian beer (and less-famous wine) tasting, which I think many people enjoyed very much.

Our paper was scheduled for a poster presentation on the third day of the conference. In our paper we investigated the influence of speech recognizer accuracy, dialog repair, and press-to-talk (PTT) button usage on driving performance. Our results showed two important effects. First, there is a statistically significant influence of low speech recognizer accuracy on steering wheel angle variance. Second, the interaction between low speech recognizer accuracy and PTT button usage has a significant influence on lane position variance. We had a chance to talk to many people who were interested in our poster and gave us some good ideas and suggestions.
We also had enough time to explore Antwerp and the surroundings. Antwerp is a nice city with plenty of museums and sightseeing areas. One of the museums that we visited is the house of the famous Belgian painter Paul Rubens, which is shown in the next picture. Paul Rubens himself designed the house and inside we could see many of his paintings.

One day we also went to Brussels, which is only 45 minutes from Antwerp by train. Although we saw many interesting things over there as well, one of the most exciting was the Atomium, shown below. It was built for the World’s Fair in 1958 and represents a crystal molecule of metal. It looks very big from the outside but it is even bigger from the inside. We went to the highest ball where we had a beautiful view of Brussels.

This was the biggest conference I have attended so far. We had a chance to meet a lot of people in our research area and exchange ideas with them which was a valuable experience. Last, but not the least, we enjoyed a lot of famous Belgian beer and chocolate! So, I hope everybody will get a chance to go and visit Belgium and enjoy it as I did.

Zeljko Medenica

Andrew Kun have already made a post with the description of SIGdial 2007. Here I would like to mention my experience as a presenter. I had a presentation on the second day of the workshops (the event was two days long). On the first day I checked if I can use their computer for my presentation. That computer did not have PowerPoint 2007, hence I decided to use Andrew’s laptop. I hooked the laptop and checked if it works. I repeated this procedure a few times to make sure that everything did work. When the time of my presentation came I hooked up the laptop and… everything worked just fine. I started my talk and at the middle of the presentation I realized that my slides were not the latest slides that I had. They were good slides but I already had an improved version… Well, I finished the talk with what I had and I doubt people realized what had happened. In the feature though I will put a version number of my slides on the first slide. The first half of the presentation was not changed and when I was testing the laptop I did not pay enough attention to the last slides and hence I did not notice what they actually contained. In retrospect I realize that I’ve heard a lot of stories about having troubles with hardware, but I did not hear any stories about using improper versions of presetnations. Now I know better.

Project54 hosted Shaun Cotter of Seeing Machines, who demonstrated the Seeing Machines eye tracker called faceLAB. The eye tracker consists of two cameras, a laptop running image processing software, and an IR illuminator. You can see the entire setup in this picture:

The two cameras are sitting on the laptop, and between them you can see IR diodes which are used to illuminate the subject’s face. The screen displays some of the results of the real-time processing, such as gaze direction, PERCLOS measure, etc, as well as the two camera views of the subject. The subject is Nemanja Memarovic, Project54 graduate student, who tested the system wearing glasses. As we found out glasses do not cause faceLAB any problems, unless they are quite thick.

One way to use the eye tracker is to install it in a driving simulator (e.g. the Project54 driving sim) and track where drivers look during different parts of driving scenarios. This way, one can find out if a distraction in the car (e.g. pushing buttons on an in-car device) reduces the amount of visual scanning the driver does. faceLAB allows creating a 3D model of the world around the driver, and can detect when the driver is looking at a specific part of this world model. This can even be done in real time and thus used to trigger events in the simulator. E.g. one can include the in-car navigation system’s screen in the 3D model and then detect when the driver is looking at it. The driving simulator scenario could then be set up such that when the driver looks at the navigation system’s screen a leading vehicle brakes.

The faceLAB demo was very interesting and over ten Project54 faculty, staff and students got a chance to play with the technology. Here’s another picture from the demo (Oskar Palinko, Project54 graduate student is asking Shaun a question):

I think that all who attended the demo were very impressed. Thanks Shaun! 

Andrew Kun

I have the “father of CLR (Common Language Runtime)”, Jim Miller, coming in to speak at UNH on Oct 9 12:30-2:00 in Kingsbury N101. Really brilliant guy and partly responsible for the birth of .NET. He definitely is the guy to speak with regards to anything .NET related.

There topics that he may talk about, here are the two talks.

Talk1:

“Microsoft’s Common Language Runtime: Is It Dynamic Enough?”

Microsoft’s Common Language Runtime (CLR) was released to the public in 2000 and was touted as a multi-language runtime. But for the first five years of its commercial life it has been mostly used for executing statically typed languages (Visual Basic, Java, C#, Eiffel, C++). There has been considerable skepticism about its ability to support more dynamic languages like Python, Perl, Ruby, and Scheme. This talk, by one of the designers of the CLR type system, introduces recent work on implementing Python on top of the CLR. I’ll also discuss how the CLR is likely to evolve in the future to make it easy to build other dynamic languages on top of it.

Talk 2:

“Microsoft .NET: What is It and What’s Next”

CLR: Internals and Future Direction: Earlier this year, Microsoft released a new version of Visual Studio including the latest version of the Common Language Runtime. I’ll talk about the implementation of some of the new technology (generics, for example). Then I’ll talk a little bit about future directions for the CLR. But I’ll leave most of the time for discussion purposes. What would you like to see in the next version of the CLR? What have you always wanted to know about how the CLR evolves over time? What technologies do you think should become part of a future CLR?

Moonlight will be mentioned

The goals of MoonLight are:

• To run Silverlight applications on Linux.
• To provide a Linux SDK to build Silverlight applications.
• To reuse the Silverlight engine we have built for desktop applications.

Jim Miller Bio:

Jim Miller is a senior architect on Microsoft’s Common Language Runtime (CLR) team. His current work is on architectural changes to allow innovation in the core of the CLR and the managed Frameworks while preserving backward compatibility. He also serves as liaison with the academic, research, and compiler communities for the CLR team.

Jim holds a PhD in Computer Science from MIT and served on the faculty at Brandeis University as well as on the research staff at MIT. He has been on the research staff at Digital Equipment Corporation and the Open Software Foundation. Before joining Microsoft, he was on the senior management team of the World Wide Web Consortium, reporting to Tim Berners-Lee and in charge of work on security, electronic commerce, child protection, privacy protection, accessibility, and intellectual property protection.

Jim joined Microsoft in 1998, leading the program management team for the kernel of the .NET Common Language Runtime (CLR). His responsibility included garbage collection, metadata definition and file formats, intermediate language (IL) definition, IL-to-native code compilation, and remote objects. He also serves as editor for ECMA TC39/TG3, which is charged with creating an international standard for a Common Language Infrastructure. To validate this standard, Jim helped create the Shared Source CLI (also known as Rotor), a complete implementation of the standard, runnable on Windows, Macintosh, and Unix operating systems, available in source form for teaching and non-commercial purposes.

Jim Miller has designed and implemented a number of novel and useful real-world systems over more than thirty years, including:

- the Microsoft Common Language Runtime (and its shared source implementation, ROTOR);

- the PICS system for Internet content selection (1995);

- the first public implementation of the Dylan programming language (Thomas, 1993);

- an early complete programming system for a parallel computer (MultiScheme, 1989);

- the first portable implementation of the programming language Scheme (CScheme, 1983);

- the first full-function electronic mail system (Hermes, 1976);

- the first source-level debugging system for a high-level language (BDDT, 1972).

Several attendees at Sigdial 2007, e.g. co-chair Harry Bunt and local chair Simon Keiser, were from Tilburg University. The list of Tilburg people included Piroska Lendvai, who gave a talk on dialogue processing. Piroska also told me about a project she’s just started working on, called MITCH, for Mining for information in texts from the cultural heritage. Piroska et al. are looking at textual and semi-textual data related to animal and plant specimens, as well as stones and minerals, available at Naturalis, the Dutch National Museum of Natural History. The Naturalis collection comes from bilogists and geologists who have travelled the planet (Holland had a great trade fleet), and it’s accompanied by handwritten documents created by these biologists and geologists. MITCH aims to create tools that will let scientists search search and query the handwritten documents transcriptions of the handwritten documents in digital form (the transcriptions are the work of researchers at SCRATCH). Having the ability to search the documents and understand the relations between these documents, will empower research on the specimens.

I really like this project. It’s a multi-disciplinary effort with huge potential payoffs in multiple fields. Clearly, MITCH would be a powerful tool for biologists and geologists, as well as historians. At the same time, developing MITCH will requre new ideas in language processing, machine learning, etc.

Andrew Kun

Immediately following Interspeech 2007, Alex Shyrokov, Zeljko Medenica and myself attended Sigdial 2007. Alex and I had a paper at the conference, written in collaboration with Peter Heeman. Alex was in charge of giving the presentation and he did an excellent job.

The venue for Sigdial 2007 was the University of Antwerp, specifically a building constructed in 1516. For those of us used to American history, this is pretty impressive Here is a picture from the venue. It shows the courtyard with the sign for the workshop (promoted to a conference in the sign!).

The workshop started with an invited talk by Dr. Herbert Clark. Dr. Clark was Tim Paek’s advisor at Stanford and Tim was one of the co-organizers of Sigdial 2007. The talk dealt with “some of the consequences of bounded rationality in language,” as Dr. Clark put it in his abstract. Specifically, when we follow (Herbert) Paul Grice’s model of language use we assume that people act rationally. However, rational action has its bounds. E.g. people don’t actually have an infinite capacity for language processing which would be necessary to always select the spoken expressions dictated by rationality. The talk was very interesting and it was also interesting to hear Dr. Clark’s comparison of his work to that of Herbert Simon who won the 1978 Nobel prize in economics for his ideas on how bounded rationality influences our choices. Btw, this talk was given by one Herbert (Clark) and it touched on the ideas of two other Herberts (Grice and Simon), which is a fact Dr. Clark was happy to point out.

While the workshop would have been worth attending for the invited talk only, there were many more talks that I really enjoyed and learned from. Happily, all the workshop papers will be are available on the Sigdial 2007 website. I will end this post with another picture from the workshop venue. Here is a picture of the workshop meeting room.

Andrew Kun

Hi all,

Last week I attended Interspeech 2007, along with Zeljko Medenica and Alex Shyrokov. Zeljko and I had a paper at Interspeech, along with Tim Paek of Microsoft Research.

This was a pretty large conference, with around 1,200 attendees from all over the world. The conference was somewhat selective, with an acceptance rate of just under 60%. While this is not a very low acceptance rate, the papers we saw at the conference (as posters or as presentations) were of high quality. We presented our paper in poster format. Zeljko did a great job of creating the poster, and he and I talked to a large number of people about it. Here is a picture of us and our poster (thanks Alex!):

Tim Paek was busy with another poster, displayed right next to ours. His poster was about People Watcher, a computer game which elicited transcribed speech data from users, while at the same time entertaining these same users. The data was used to train a system for automated (spoken) directory assistance. The paper received the ELRA best paper award for Interspeech 2007. Congratulations Tim et al! Here’s a picture of Tim (facing us on the right) and co-author Yun-Cheng Ju (facing, left) discussing their poster with attendees:

In this post I also wanted to mention another collaborator, Peter Heeman. Peter presented a poster on a simulation study in which he and a student explored initiative in human-human dialogues. Understanding how initiative is treated in these dialogues is important if one wants to build a system for mixed-initiative human-computer dialogues. Here’s a picture of Peter discussing his poster:

Peter, Alex and I also had a paper at the Sigdial 2007 workshop, which was held immediately after Interspeech. I’ll talk more about Sigdial in another post.

Andrew Kun