Technische Universität München Robotics and Embedded Systems

Model-based Visual Tracking

Veranstalter Giorgio Panin, Ph.D.
Modul IN3150
Typ Vorlesung
Sprache Englisch
Semester WS 2009/2010
ECTS 3.0
Hörerkreis Wahlfach für Studenten der Informatik (Master, Diploma)
Zeit & Ort Di 10:00 - 12:00 MI 03.07.023
Schein Nach erfolgreicher mündlichen Prüfung


Exam: Thursday, 04.03.10 startgin from 14:00, in Seminarraum 03.07.023.

Registration: through TUM-Online.

Open and currently running Theses

Thesis proposals can be found at the Vision section of our student projects webpage.

For information about our research group, see also the ITrackU webpage, and the OpenTL library.

Course description

The course aims to provide a structured overview of model-based object tracking, with the purpose of estimating and following in real-time the spatial pose (rotation, translation etc.) of one or more objects, by using digital cameras and fast computer vision techniques.

The first part of the course will introduce the general tools for object tracking:

1. Pose and deformation models, and camera projection

2. Methods for pose estimation from geometric feature correspondences

3. Bayesian tracking concepts (state dynamics, measurement likelihood)

4. Bayesian filters for linear and nonlinear models, with single or multi-hypothesis state distributions

Afterwards, we will concentrate on the visual part: among the many modalities available, we will focus in particular on the following ones:

1. Color-based: Matching color statistics, from the visible object surface to the underlying image area.

2. Keypoint- and Motion-based: Detection and tracking of single point features, possibly making use of image motion information (optical flow).

3. Contour-based: Matching the object boundary line, as it deforms with the object roto-translation (also called Active Contours).

4. Template-based: Registration of a fully textured surface (Template) to the image gray-level intensities.

Finally, the last lecture will introduce advanced topics, concerning: multiple cameras, multiple simultaneous objects, and data fusion with multiple modalities (colors, edges, ...).


The course will also provide the following pre-requisites in a self-contained fashion (a basic knowledge would be in any case recommended):


The reference text for this course is

Giorgio Panin, Model-based visual tracking, ed. Wiley-Blackwell (to appear around end 2010).

Please check also the OpenTL webpage for more information.


Lecture slides for WS09/10 are currently in preparation:

IMPORTANT: now they are protected by password, and restricted only to the course participants.

Please contact me by email in order to obtain the password.

Part I - General tools for object tracking

Part II - Visual modalities

Bibliographical references

Lecture 1:

Lecture 2:

Lecture 3:

Lecture 4:

Lecture 5:

Kalman Filter:

Lecture 6:

Color spaces: (wikipedia links)

Color distributions:

Mean-shift: (for more recent applications and videos, look at the homepage of D. Comaniciu)

Blob matching: [2], Chapter 5 (morphology) and 8 (matching contours)

Color-based particle filter: [17] (feature-level) and [18] (pixel-level)

Lecture 7:

A general list of keypoint detectors (for this and the next lecture) can be found here.

KLT algorithm: material can be found at the KLT homepage by Stan Birchfield. In particular, [19] is the reference paper for this method.

Back-projection: for more information about depth maps, see the Z-buffering webpage.

Feature detection vs. tracking: see also [1]

Optical flow: see the Wikipedia page and the Lucas-Kanade original paper [21].

Harris detector: see the paper [20]

Lecture 8:

Harris corners: (see Lecture 7)

Scale-space theory: the book of T. Lindeberg [9], plus some references (for a quicker introduction), at this webpage.

SIFT webpage (by D. Lowe).

Lecture 9:

Lecture 10:

Lecture 11:

Lecture 12: