Real Target Tracking using d2D/dt visual servoing

Contact: François Chaumette, Armel Crétual

Creation date Décembre 1999

Description of the demonstration

The aim of this application consist in controlling a camera mounted on a robot manipulator in order to maintain a mobile object at the image center. The controlled camera motion are the two degrees of freedom in panoramic and tilt rotations, which respectively corresponds to displacements along x et y axes in the image.

The task is divided into two parts:

• Detection of the object of interest according to a motion criteria by an averaged images difference.

• Estimation of the position of the mobile object center and control of the camera orientation in order to bring back this point to the image center. The estimated position is obtained by temporal integration of the apparent motion of the object. The displacement is computed in the including window of the binary mask of the object given by the detection step. At each iteration, this window is shifted in the computed movement direction. A filtering of the estimated object center allows to compensate tracking errors.

Mask of the detected object	Pursuit of the estimation window

Results

Designed, in the ideal case, for the tracking of a priori planar and undistortionable objects, this application has given efficient results in the tracking of more complex objects such as pedestrians. In the images displayed below, the mobile rigid object is moving at 30 cm/s, the pedestrian walks at a "normal" speed and both of them are approximately 2 m away from the camera. Th rate of images acquisition and treatement, 20 per second, is close to the video rate.

Rigid object tracking	Pedestrian tracking

Applications

The field of applications of this demonstration includes those of robotic visual tracking, such as surveillance.

Scientific context

This demonstration is part of the visual servoing theme developped in the VISTA team and which consists in regulating errors between current and desired situations directly in the image. The algorithm of motion estimation used in this task has also been perfected in the team, within the framework of the research of image motion analysis. Its principle is to determine the polynomial model (constant, affine, quadratic), which closest describe displacements in a specified zone of the image by multi-resolution techniques.

Collaborations

This work has lead to a study entitled "Dynamic stabilisation of a pan and tilt camera by visual servoing", carried out in collaboration with Ifremer (French Institute of Research and Exploitation of the Sea).

References

1. A.Crétual, F. Chaumette. Image-based visual servoing by integration of dynamic measurements - IEEE International Conference on Robotics and Automation, ICRA'98, Lueven, Belgium, May 1998