The DFKI Evaluation Center for Language Technology

Recent Work

Speech-Based Home Cooking Assistant

• Nature of the system:
  • The system runs on a laptop placed on a kitchen counter
  • While cooking, the user requests information from it via speech
  • The system presents information either
    • Via speech (only for texts whose length is below a given threshold)
    • Via printed text and/or graphics on the laptop screen
  • Designer: Rainer Wasinger, now with the COLLATE project
• Examples of questions studied:
  • Users really keep working on their cooking while speaking or listening to the system,
    • or do they stop work briefly?
  • When the system is speaking, do users sometimes stop to read the text on the screen?
  • What should be the maximum length of spoken texts?
    • Does the answer depend on the nature of the texts?
  • Data yielded in this study:
    • A stream of images showing the current scene and the objects that the user is looking at
  • Expected insights:
    • Qualitative answers to questions such as those listed above
• Results:
  • Why might users look at the screen when it is not strictly necessary to do so?
    • "I didn’t quite understand that (complicated) instruction"
    • "Is what I just did what I was supposed to do?"
    • "How far along am I in the whole set of instructions?"
    • ("How can I pass the time while waiting for the water to boil?")
• Study performed with the ASL Model 501 mobile eye tracker