Object and motion tracking (back)

Description

We propose a method to track an object and its motion field along an image sequence.

The method is formalized within a variational data assimilation framework, in order to be robust to noise and occlusions. This technique allows us to enforce the temporal coherence for the object shape and its velocity fields along the whole sequence. Such a temporal coherence can not be garanted by any usual active contours methods.

Data assimilation principle consists in combining an initialization, a dynamical law and noisy observations. This setting comes to the minimization of a global spatio-temporal functional that is realized through an adjoint formulation.

The object is described by a level set function. We have distinguished two kind of application: real world sequences and fluid motion sequences. In the first case, the motion is assumed to be preserved along the trajectory of the object. In the second case, the motion fluid structure is described by the 2D vorticity-velocity form of the Navier-Stokes equation.

The observations considered are (directly or not) extracted from the images. They can be contours, histograms or optical flow motion fields.

Results

Natural object tracking:

Tracking of a surfer in a pipeline

Initial curve	Animation:Tracked curves

The motion is first assimilated in the image areas corresponding to the surfer histograms. The dynamical law used is the conservation of the velocity. The curve is then guided by the image measurement and the a priori dynamic law.

Hand sequence

Observed curves (external segmentation process)	Tracked motion field	Tracked curve

Here, an external segmentation process provides noisy curves measurements as observations. The same kind of dynamics as for the previous example is considered.

Fluid tracking:

In the following examples, a noisy version of the vorticity-velocity formulation of the Navier Stokes equation is considered as the dynamical evolution law.

Tracking of a cyclone

Tracked curves	Observed vorticity (from optic flow)	Tracked vorticity

The motion field is here visualized through its vorticity. The image measurements are histograms of the luminance function inside and outside the region delimited by the curve.

DNS Particles sequence

The tracking of vorticity is applied on synthetic sequences in order to evaluate the quality of the estimated motions. The observations can be provided by an external optical flow estimator or directely by the images

Particle sequence	Actual vorticity
Observed vorticity (Optic flow technique)	Vorticity assimilated with optical flow observation
Vorticity assimilated with image observation (Optic flow technique)

The spectral representation of the mean vorticity shows that the assimilation process rallows to improve the recovery of the small motion structures:

Spectral analysis

Reference

N. Papadakis and E. Mémin. Variational optimal control technique for the tracking of deformable objects . IEEE International Conference on Computer Vision, ICCV'07, Rio de Janeiro, Brasil, October 2007. (pdf)

N. Papadakis, T. Corpetti and E. Mémin. Dynamically consistent optical flow estimation . IEEE International Conference on Computer Vision, ICCV'07, Rio de Janeiro, Brasil, October 2007. (pdf)

N. Papadakis and E. Mémin. A variational method for joint tracking of curve and motion . INRIA Resarch report number 6283, September 2007. (pdf)

N. Papadakis, E. Mémin. A variational framework for spatio-temporal smoothing of fluid motions . Scale Space and Variational Methods, Ischia, Italy, May 2007.(pdf)

N. Papadakis, E. Mémin, F. Cao. A variational approach for object contour tracking. In Proc. ICCV'05 Workshop on Variational, Geometric and Level Set Methods in Computer Vision, Beijing, China, October 2005. (pdf)