Real-time 3D localisation and tracking using virtual visual servoing

Contact : Eric Marchand Creation date : July, 31 2004 Last modification : January, 12 2005

The two softwares used in these videos: Marker (Marker-based augmented reality kernel) and Markerless (Markerless-based augmented reality kernel) have been implemented at IRISA-INRIA Rennes within the Lagadic research team. These softwares are licenced to a french company: Total Immersion.

In all the videos, tracking is achieved at video rate (each frame is processed in less than 20 ms).

Related pages

• Virtual visual servoing (Eurographics 2002)
• Robust model-based tracking (IROS'04)
• Model-free augmented reality by virtual visual servoing (ICPR'04)

3D localisation and tracking: overview of the method

Virtual visual servoing: overview see [Eurographics 02] for explanations of the videos 3D Localisation wrt. straight lines 3D Localisation wrt. points, straight lines and circles

Marker-based tracking

Virtual visual servoing: Marker based tracking (the Marker software) Marker based augmented reality Interactive visio-conference (Marker within D'Fusion by Total Immersion)

Markerless tracking

Robust tracking see the specific (old) demo page and [Eurographics 02][ISMAR'03] papers for details Chair tracking (4.9 Mo) Tracking of an industrial object (8.8 Mo) Tracking within a building (6.43 Mo) Tracking a "castle" image from Inria (4.8 Mo) Tracking a "castle" (close view) image from Inria (6.1 Mo) Tracking a "castle" (Integration within an animation module) image from Total-Immersion (35 Mo)

Tracking in robotic experiments see the specific demo page and [IROS'04] paper for details Tracking of a micro controller (occlusions, various lighting conditions, etc.) Tracking in visual servoing experiments (1) Tracking in visual servoing experiments (2)

Markerless in augmented reality applications Markerless within D'Fusion by Total Immersion All these videos have been done by Total Immersion see [ISMAR'03] paper for details Tracking a BMW 740i armrest (34 Mo) Tracking within a building (11.2 Mo) Integration within an animation module (1.2 Mo) Tracking a PC (10.1 Mo)

Model-free augmented reality

Model-free augmented reality see the specific demo page and [ICPR'04] paper for details Pure rotation camera motion Pure rotation camera motion Translation and Rotation camera motion

References

• A.I. Comport, E. Marchand, M. Pressigout, F. Chaumette. Real-time markerless tracking for augmented reality: the virtual visual servoing framework. IEEE Trans. on Visualization and Computer Graphics, 12(4):615-628, July 2006.
• A. Comport, E. Marchand, F. Chaumette. Robust model-based tracking for robot vision. In IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, IROS'04, Sendai, Japon, Septembre 2004.
• A. Comport, E. Marchand, F. Chaumette. A real-time tracker for markerless augmented reality. In ACM/IEEE Int. Symp. on Mixed and Augmented Reality, ISMAR'03, Pages 36-45, Tokyo, Japon, Octobre 2003.
• E. Marchand, F. Chaumette. Virtual Visual Servoing: a framework for real-time augmented reality. In EUROGRAPHICS 2002 Conference Proceeding, G. Drettakis, H.-P. Seidel (eds.), Computer Graphics Forum, Volume 21(3), Pages 289-298, Sarrebruck, Allemagne, Septembre 2002.
• M. Pressigout, E. Marchand. Model-free augmented reality by virtual visual servoing. In IAPR Int. Conf. on Pattern Recognition, ICPR'04, Cambridge, Royaume-Uni, Août 2004.