DFKI
Home
Selected Public Appearances
Current Work (Selection)
  VOICE Award 2006
  Project HALO 2
  GNAB
  juris
  Mobile Multimodal Systems
Recent Work (Selection)
  VOICE Award 2005
  Improved Internet Search
  WYSIWYM
  Document Manager
  Multitasking with Xybernaut
  Automatic Hyperlinking
  Speech-Based Home Cooking Assistant
  Sonicson Music Retrieval System
Equipment
  Tips and Tricks for Mobile Eye Tracking
Publications
Selected Videos
Services and Contact

German Competence Center for Speech Technology The other parts of the German Competence Center for Speech Technology:
Virtual Information Center LT-World
German Demonstration Center for LT-Systems

The COLLATE Project

Last modified: 2006-11-08

The DFKI Evaluation Center for Language Technology

 

Recent Work


 
 

Speech-Based Home Cooking Assistant

 
 
Screenshot Home Cooking Assistant
  • Nature of the system:
    • The system runs on a laptop placed on a kitchen counter
    • While cooking, the user requests information from it via speech
    • The system presents information either
      • Via speech (only for texts whose length is below a given threshold)
      • Via printed text and/or graphics on the laptop screen
    • Designer: Rainer Wasinger, now with the COLLATE project
  • Examples of questions studied:
    • Users really keep working on their cooking while speaking or listening to the system,
      • or do they stop work briefly?
    • When the system is speaking, do users sometimes stop to read the text on the screen?
    • What should be the maximum length of spoken texts?
      • Does the answer depend on the nature of the texts?
    • Data yielded in this study:
      • A stream of images showing the current scene and the objects that the user is looking at
    • Expected insights:
      • Qualitative answers to questions such as those listed above
  • Results:
    • Why might users look at the screen when it is not strictly necessary to do so?
      • "I didn’t quite understand that (complicated) instruction"
      • "Is what I just did what I was supposed to do?"
      • "How far along am I in the whole set of instructions?"
      • ("How can I pass the time while waiting for the water to boil?")
  • Study performed with the ASL Model 501 mobile eye tracker