Wavelet-based Motion Estimation

My main topic is the design and implementation of a wavelet-based optical-flow estimator, dedicated to the motion estimation and analysis of turbulent fluid flows.

Flow visualization methods are commonly used to study fluid dynamics. The animation above consists in two successive frames of an image sequence showing the dispersion of a passive scalar (e.g. coloring agent) in a 2D turbulent flow. Computer vision methods are then used in order to extract information, such as the apparent velocity, from flow visualization pictures.

Optical Flow Concept

Optical flow estimation consists in estimating the 2D motion (or displacement, deformation field, ...) explaining the transformation of an image (at time t) of a given sequence into the next one (time t+1).

Optical flow concept: we want to recover the apparent 2D motion v that transforms the first image into the second one.

Contrary to correlations method, optical flow produces a dense field, i.e. one velocity vector per image pixel. Furthermore, both the data model (the equation linking images and motion field) and the regularization term (which enables to close the problem) may be customized in adequacy to the studied configuration -- here, fluid flows.

[Back to top of page]

Wavelet-Based Optical Flow

In this work, we consider the discrete wavelet transform of each scalar component of the 2D motion field v. Wavelets give a multiscale representation of the considered signal, which consists in a coarse approximation and several scales of details.

The 2D wavelet transform of each component is considered. Here, the isotropic transform is illustrated.

The unknowns of the optical flow problem are the wavelet coefficients representing motion v in the considered wavelet basis. They are estimated sequentially, from the coarsest scale to the finest one, using a quasi-Newton gradient descent algorithm (L-BFGS). In comparison to traditional multiresolution optical flow algorithms, or to other wavelet-based algorithms, our configuration has the following advantages:

A "natural" multiscale decomposition of the motion, avoiding to resort to ad-hoc pyramidal schemes.
Thanks to the gradient-based optimization routine, no need to compute the Hessian matrix. This results in a much lighter algorithm, enabling to access to smallest scales and/or to process large images (1024x1024 pixels => 2^21 unknowns!).
A very simple closure mechanism, by neglecting smallest scales, resulting in a polynomial approximation of the sought velocity field.
Discrete or exact high-order regularization schemes, such as Laplacian or gradient(curl(v)).
Regularizations based upon turbulence physics (e.g. power-law spectrum), thanks to wavelet coefficients properties (see TSFP-7 paper).
Divergence-free wavelet basis, for specific 2D flow configurations.
And more...

Up to now, the software features:

Orthogonal wavelets: Daubechies, Coiflets, Battle-Lemarie. Bi-orthogonal: splines.
Many discrete or exact regularization schemes.
C++ source code, based upon libLBFGS for the optimization routine, CImg for the image processing part and a home-made wavelet library. Developed on MacOS X 10.6.

This wavelet-based estimator has proved to outperform some state-of-the-art estimators (see e.g. SSVM2011 paper). It has also been applied to several datasets, some examples may be seen here.

[Back to top of page]

Work in Progress

Current developments aim at setting up a well-defined framework for multiresolution data-models, using a multiscale wavelet representation of input images and the bayesian theory.

[Back to top of page]

References

Dérian, P., P. Héas, C. Herzet and E. Mémin, 2011: Wavelets to Reconstruct Turbulence Multifractals from Experimental Image Sequences..
in 7th Int. Symp. on Turbulence and Shear Flow Phenomena (TSFP-7), Ottawa (Canada), August 2011.

In the context of turbulent fluid motion measurement from image sequences, we propose in this paper to reverse the traditional point of view of wavelets perceived as an analyzing tool: wavelets and their properties are now considered as prior regularization models for the motion estimation problem, in order to exhibit some well-known turbulence regularities and multifractal behaviors on the reconstructed motion field.

Dérian, P., P. Héas, C. Herzet and E. Mémin, 2011: Wavelet-Based Fluid Motion Estimation.
in Scale Space Methods and Variational Methods (SSVM) in Computer Vision, Ein-Gedi (Israel), June 2011.

Based on a wavelet expansion of the velocity field, we present a novel optical flow algorithm dedicated to the estimation of continuous motion fields such as fluid flows. This scale-space representation, associated to a simple gradient-based optimization algorithm, naturally sets up a well-defined multi-resolution analysis framework for the optical flow estimation problem, thus avoiding the common drawbacks of standard multi-resolution schemes. Moreover, wavelet properties enable the design of simple yet efficient high-order regularizers or polynomial approximations associated to a low computational complexity. Accuracy of proposed methods is assessed on challenging sequences of turbulent fluids flows.

Dérian, P., P. Héas, C. Herzet and E. Mémin, 2010: Wavelet Expansion and High-order Regularization for Multiscale Fluid-motion Estimation.
INRIA research report #7348.

See Publications of Pierre Dérian for an exhaustive list with PDF documents available.