Research

• Current research
  • Person Localization
  • 3D Body Model Tracking
  • Pose Tracking for Gesture Recognition and Object Learning
  • Automatic Initialization of a 3D Body Model
  • Motionese
• Publications

Current research                                                                                                                

My current work is on markerless localization and tracking of the human body. Furthermore, I am also interested in human-computer- and human-robot-interaction and in perception, learning and understanding of actions in humans. I am currently writing my Ph.D. thesis, which i hope to finish within the year.

Below, you will find some of the projects that I have worked on as a graduate student.

Person Localization                                                                                                           

The goal of this project is to develop a vision-based system for the 3D detection and tracking of moving persons and objects in complex scenes. Industry cooperation with DaimlerChrysler Research Ulm.

For each multiocular image, generate a motion-attributed 3D point cloud utilizing spatio-temporal local intensity modeling, also called "Spacetime Stereo". Hierarchical clustering is applied to Segment objects/persons, using velocity information as an additional discrimination criterion. A particle filter selects and tracks object hypotheses forming consistent trajectories.

[video:index=0: width=540:height=452]

Poster and Video (MPEG-4, 17 Mb) presented at ICVS07.

3D Body Model Tracking                                                                                                    

Markerless 3D body tracking using a single uncalibrated camera.

The human is modeled by an articulated 3D body model with each limb represented by cylinders. Intensity-based and color-based cues match the observation model with the images. A kernel particle filter is used to search the 14-dimensional parameter space of the joint angles.

 The approach is intended for use on a mobile robot to improve human robot interaction.

Official project page.

Results

Poster and Video (MPEG-4, 7 Mb) presented at FG06.

[video:index=2:width=480:height=380]

Results of realtime 3D-tracking out of monocular color images: Video (MPEG-2, 4 MB)

Pose Tracking for Gesture Recognition and Object Learning                                        

From observing different humans performing the same actions, e.g., pointing towards an object, a model for this gesture can be learned. The 3D body tracking framework delivers the data both for learning and recognizing actions.

Furthermore, the recognized pointing gesture can be used to guide attention towards the object of interest, thus making object recognition more robust even in the presence of multiple objects or cluttered scenes.

Cooperation with Nils Hofemann and Axel Haasch.

[video:index=3:width=452:height=360]

Video (MPEG-4, 7 Mb) of a human (me by name) pointing at different objects, this video has been presented at Nils Hofemann's disputation.

Automatic Initialization of a 3D Body Model                                                                   

Applying the ENCARA2 face detector to learn skin and shirt color. Use detected face and hands for automatic 3D body model initialization.

Nothing official yet. You may also have a look at Modesto Castrillon's page.

Motionese                                                                                                                            

The goal of the project "Motionese" is to apply the automatic 3D-tracking system for manipulative gestures in order to explore learning processes and representational mechanisms in human development.

Official project page.

Results

[video:index=4:width=451:height=360]

Markerless 3D body tracking applied for tracking parents while performing manipulative actions: Video (MPEG-4, 8 Mb)

Example from the Motionese corpus: Tracking persons demonstrating an action.

Publications                                                                                                                        

(also avaiable at the official page)

Conferences

• J. Schmidt, C. Wöhler, L. Krüger, T. Gövert, and C. Hermes, 3D Scene Segmentation and Object Tracking in Multiocular Image Sequences, In Proc. of 5th International Conference on Computer Vision Systems (ICVS'07), Bielefeld, Germany, March 2007. [BibTex] [PDF] [poster]
• J. Schmidt, B. Kwolek, and J. Fritsch, Kernel Particle Filter for Real-Time 3D Body Tracking in Monocular Color Images, In Proc. of Automatic Face and Gesture Recognition, pages 567-572, Southampton, UK, April 2006. [BibTex] [PDF] [poster]

Diploma Thesis

• J. Schmidt, Inkrementelle Adaption eines 3D-Körpermodells aus Bildsequenzdaten. (german only) Diploma thesis, Bielefeld University, 2004. [BibTex] [PDF]