Robotic Roommates Making Pancakes

author: Ulrich Klank, Technische Universität München
author: Ingo Kresse, Technische Universität München
author: Alexis Maldonado, Technische Universität München
author: Lorenz Moesenlechner, Technische Universität München
author: Dejan Pangerčič, Technische Universität München
author: Thomas Ruehr, Technische Universität München
author: Moritz Tenorth, Technische Universität München
author: Michael Beetz, Technische Universität München
published: Dec. 19, 2011,   recorded: August 2011,   views: 2705

Related Open Educational Resources

Related content

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.
Lecture popularity: You need to login to cast your vote.


In this video we show a recent public experiment that shows two robots making pancakes using web instructions. In the experiment, the robots retrieve instructions for making pancakes from the World Wide Web and generate robot action plans from the instructions. This task is jointly performed by two autonomous robots: The first robot opens and closes cupboards and drawers, takes a pancake mix from the refrigerator, and hands it to the robot B. The second robot cooks and flips the pancakes, and then delivers them back to the first robot. AI-related in the experiment is using of the encyclopedic knowledge base for the translation of instructions into robot plans which is performed by the following sequence of steps. First, the sentences are parsed using a common natural-language parser to generate a syntax tree of the instructions. The branches of the tree are then recursively combined into more complex descriptions to create an internal representation of the instructions describing the actions, the objects involved, locations, time constraints, the amount of ingredients to be used etc. The words in the original text are resolved to concepts in the robot's knowledge base by first looking up their meanings in the WordNet lexical database, and by then exploiting mappings between WordNet and the Cyc ontology. Our system employs a simple method based on the phrase context and on information about object-action pairs obtained from Cyc to disambiguate between possible word meanings.

Link this page

Would you like to put a link to this lecture on your homepage?
Go ahead! Copy the HTML snippet !

Write your own review or comment:

make sure you have javascript enabled or clear this field: