6D Scene Analysis |
The Articulated Scene Model
In this paper we present a new system for a mobile robot to generate an articulated scene model by analyzing complex dynamic 3D scenes. The system extracts essential knowledge about the foreground, like moving persons, and the background, which consists of all visible static scene parts. In contrast to other 3D reconstruction approaches, we suggest to additionally distinguish between static parts, like walls, and movable objects like chairs or doors. The discrimination supports the reconstruction process and additionally, delivers important information about interaction objects. Here, the movable object detection is realized object independent by analyzing changes in the scenery. Furthermore, in the proposed system the background scene is feedbacked to the tracking part yielding a much better tracking and detection result which improves again the 3D reconstruction. We show in our experiments that we are able to provide a sound background model and to extract simultaneously persons and object regions representing chairs, doors, and even smaller movable objects.
This work has been presented on the International Conference on Robotics and Automation 2010 in Anchorage, Alaska, USA. [paper, slides]
Some source code is now available in the Cognitive Interaction Toolkit.
Video 1
Video 2
Watch also the video below showing the formation of the articulated scene model on a test sequence:
Previous work
First, we have focused on two important aspects in a human-robot interaction scenario: The localization of possible interaction partners and the reconstruction of the surrounding environment in order to use it for, e.g., navigation purposes and room categorization. Although these processes can be addressed independent of each other, we show that using the available data in exchange enables a more exact reconstruction of the static scene, especially in the case of a short sequence containing a lot motion. We present an approach that solves both tasks based on the detection and tracking of moving objects. A 6D data representation consisting of 3D Time-of-Flight (ToF) sensor data and computed 3D velocities allows segmenting the scene into clusters with consistent velocities. A weak object model is applied to localize and track objects utilizing a particle filter framework. Consequently, points emerging from moving objects can be neglected during reconstruction. Experiments demonstrate enhanced reconstruction results in comparison to pure bottom-up methods and earlier systems, especially for very short image sequences that are typical for a human robot interaction scenario.
This work is done in collaboration with Joachim Schmidt and Niklas Beuter. I presented the initial paper "Tracking Objects in 6D for Static Scene Reconstruction" at the CVPR'08 Workshop Time-of-Flight Based Computer Vision (TOF-CV) [ paper, slides].