Robotic Roommates Shopping for and Preparing Bavarian Breakfast
The robotic companions from the Munich-based cluster of excellence CoTeSys (Cognition for Technical Systems) have shown their skills and capabilities as chefs in the kitchen and shopping better halves. The aim of this feasability study was to test generalizability of perception and controlling mechanisms that were earlier employed in a demonstration where both robots made pancakes and set the table.
In the first part of the video below, you can see TUM-James, a PR2 Beta Programm robot, and TUM-Rosie cooperatively preparing the traditional Bavarian Weisswurst Frühstück (Bavarian Sausage Breakfast). TUM-Rosie is collecting the sausages, putting them into the pot with boiling water, waiting for them to be cooked and, finally, finding and getting them out of the pot into the serving bowl. TUM-James is meanwhile slicing the french baguette using a regular electric bread slicer and in the end serving the sausages and the bread to the class of highly regarded roboticists.
This demonstration builds off the one of pancake making and is on one hand fulfilling Intelligent Autonomous Systems Group's commitment towards the PR2 Beta Programm in terms of having the robots to prepare two simple meals. On the other hand it is a demonstration of the bleeding edge perception, manipulation and reasoning capabilities of high-end service robots.
TUM-James makes use of the recent advances in the field of real-time RGB-D sensing using Kinect sensor for the detection of the bread slicer and the baguette. In the serving task it uses PR2's haptic capabilities in order to grasp and manipulate the plate.
TUM-Rosie is as well using Kinect and perception algorithms from COP module in order to calibrate the skimmer and use it as a new tool center point of the arm. Furthermore it learns the 3D models for the pot and the bowls in order to be able to localize them at any arbitrary pose on the table. Lastly, it uses the torque sensors to resolve depth measurement inaccuracies through contact detection with the objects and blob segmentation in order to localize sausages inside the pot.
In the second part of the video, TUM-James is shown simulating the shopping task, bringing the groceries home and placing them to the places they are meant to be stored in.
It uses 3D perception algorithms from PCL in order to detect object candidates in the storage rack and ODUfinder system to recognize them. Grasping and manipulation
of the shopping basket is done using James's haptic capabilities. Finally, to infer where do the objects belong to, TUM-James queries KnowRob system and computes similarities using techniques from semantic information retrieval (WUP similarity) to the other objects in the kitchen's ontology.
In the course of the demonstration, both robots cope with mechanical inaccuracies, obstacles, and execution flaws. Failures in the task execution are dealt with using learned knowledge about the processes involved, the tools and their usage. Thus, if the robot e.g. failed to get the sausage out of the pot, the perception system notifies this and instructs the robot to retrieve the action.
Much of the software used in this demonstration is already or will soon be made available to robot researchers in the TUM ROS repositories.
For longer version of the demo please visit the breakfast making and the shopping videos.
