Visual servoing for ultrasound-guided needle insertion

Contact: Pierre Chatelain
Last update: May 2016

Context

This project deals with control strategies to assist the insertion of a biospy needle under ultrasound guidance. One of the challenges of ultrasound-guided needle insertion is the localization of the needle. We have developed tracking algorithms to detect the full needle shape in 3D ultrasound volumes without requiring any external tracker. Using the result of this real-time tracking, we have designed two different control frameworks which show how ultrasound-based visual servoing can be used to assist the insertion of a biopsy needle:

• For probe control: the needle is inserted manually, and the ultrasound probe, mounted on a robotic arm, is controlled in order to follow the needle during the insertion. In addition, the orientation of the probe is adapted so that the needle is aligned with the central ultrasound plane, where the resolution is the highest.
• For needle steering: the needle is inserted automatically by a robotic arm. Using the current 3D shape of the probe, we perform a closed-loop duty cycling control of the needle to steer it to a predefined target.

Method overview

Needle tracking

The needle is tracked directly in the 3D ultrasound volumes acquired by the wobbler probe. In a first work (ICRA 2013), we used a combination of the random sample consensus (RANSAC) algorithm with Kalman filtering to track the needle. The needle shape is modeled as a polynomial curve. The underlying assumption for the RANSAC algorithm is that voxels belonging the needle are brighter than the background. Kalman filtering allows both to smooth the detection results and to refine the search space of RANSAC, in a closed loop fashion. We also proposed a method to automatically detect the insertion point of the needle, using difference images.

In a later work (ICRA 2015), we used particle filtering to track the needle. The state space of the particle filter represents the full needle shape, and is refined iteratively based on the ultrasound image feedback. In a sense, the particle filter combines the capabilities of RANSAC and Kalman filtering in a single model. It is also less computationally expensive than the previous approach.

Probe guidance

In order to automatically guide the probe during manual needle insertion, we use a geometric visual servoing approach. The geometric feature is defined as the principal direction of the needle shape, and the desired configuration is to have the needle tip at the center of the image, and its axis aligned with the central ultrasound frame.

Needle steering

Beveled tip needles are subject to asymmetrical contact forces with surrounding tissues, which induce a natural bending during insertion. It is also possible to insert the needle in a straight trajectory by rotating around its axis while inserting. Duty cycling control consists in combining these two motion types (pure insertion and insertion/rotation) in order to obtain the desired curvature. Using the tracked position of the needle, we perform an adaptive duty cycling control of the needle insertion by visual servoing. Thus, we are able to perform a closed loop 3D steering of the needle towards a target.

Experimental results

References

1. P. Chatelain, A. Krupa, M. Marchal. Real-time needle detection and tracking using a visually servoed 3D ultrasound probe. In IEEE Int. Conf. on Robotics and Automation, ICRA'13, Pages 1668-1673, Karlsruhe, Germany, May 2013.
2. P. Chatelain, A. Krupa, N. Navab. 3D ultrasound-guided robotic steering of a flexible needle via visual servoing. In IEEE Int. Conf. on Robotics and Automation, ICRA'15, Seattle, WA, May 2015.