Description
We propose a new method to both track and segment multiple objects in
videos using min-cut/max-flow optimizations.
We introduce objective functions that combine low-level pixel-wise measures (color, motion), high-level
observations obtained via an independent detection module (connected
components of foreground detection masks in the experiments), motion
prediction and contrast-sensitive contextual regularization. The minimization
of these cost functions simultaneously allows "detection-before-track"
tracking (track-to-observation assignment and automatic initialization of new
tracks) and segmentation of tracked objects. When several tracked objects get
mixed up by the detection module (e.g., single foreground detection mask for
objects close to each other), a second stage of minimization allows the proper
tracking and segmentation of these individual entities despite the observation
confusion.
Our method is able to detect, track and precisely segment objects as they
enter and traverse the field of view, even in cases of occlusions (partial or
total), temporary grouping and frame dropping.
Results
Results with moving object detector (more)
Water skier sequence ; Original images, Object detection with mean shift clustering, Moving object segmentation and tracking
Results with background subtraction
PETS 2006 Sequence; Original images, Object detection using background subtraction, Moving object segmentation and tracking
Reference
A. Bugeau, P. Pérez. Track and Cut: simultaneous tracking and segmentation of multiple objects with graph cuts. International Conference on
Computer Vision Theory and Applications (VISAPP' 08), Madeira, Portugal, January 2008. 
(pdf)
A. Bugeau, P. Pérez. Joint Tracking and Segmentation of Objects using Graph Cuts. Proc. Conf. Advanced Concepts for Intelligent Vision Systems (ACIVS' 07), Delft, the Netherlands, August 2007. (pdf)
A. Bugeau, P. Pérez. Track and Cut: simultaneous tracking and segmentation of
multiple objects with graph cuts
.Technical report, IRISA, PI 6337, 2007.(pdf)
