Vous êtes ici

Shared control of flexible needles for robot-assisted biopsies

Equipe et encadrants
Département / Equipe: 
Site Web Equipe: 
Directeur de thèse
Alexandre Krupa
Co-directeur(s), co-encadrant(s)
Marie Babel
NomAdresse e-mailTéléphone
Alexandre Krupa
Sujet de thèse

Needle insertion in soft-tissue is a minimally invasive surgical (MIS) procedure used for diagnostic and therapeutic purposes, and it is one of the many surgical procedures that may greatly benefit from the use of teleoperated robotic systems. Hence, researchers have been constantly trying to develop new techniques and systems able to improve its safety and accuracy. Flexible needles and haptic feedback are two of these technological advancements. Flexible needles provide the clinician with enhanced steering capabilities, and haptic feedback enables the clinician to receive information about the forces exerted by the needle on the soft tissue being penetrated.

In our previous works, we studied different approaches to automatically steer a flexible needle actuated by a robotic arm, in order to accurately position its tip on a desired target by visual servoing and 3D ultrasound imaging. However, for reasons of safety and responsibility, it would be beneficial to provide clinicians with direct control of the motion of the medical instrument.

We propose to study innovative teleoperation systems for steering flexible needles, exploiting grounded and ungrounded haptic stimuli for our vision-based needle insertion system, with the final objective of maximizing the information provided, the clinician comfort, and the medical procedure’s safety and effectiveness. The project will proceed by developing four main key aspects:

  • Perception of multiple haptic stimuli. At first, we will study the effectiveness of combining multiple haptic stimuli, focusing on force, vibrations, normal indentation, and skin stretch. We will focus on stimuli being able to provide multi-directional information, applied to different parts of the body, such as the hand, wrist, and forearm.
  • Visual servoing. We propose to develop new ways of assistance solutions where the clinician will keep total or partial manual control of the needle positioning. This could be achieved by sharing different degrees of liberty of the needle between the robot and the clinician through teleoperation.
  • Shared control with haptic guidance. To help clinicians steer the needle toward the target, we will study how to provide effective guiding haptic stimuli. Haptic feedback will be used to enforce active constrains aimed at safely positioning the needle without damaging the tissues and also to provide guiding information extracted from the current image.
  • Safety and stability.   We will work to improve existing stability control approaches to take into account for the additional tactile stimuli, focusing on time-domain energy-based techniques, with the objective of maximizing transparency while guaranteeing the overall safety of the system.


Below we show a mock-up representing the envisaged teleoperation system.


Robotic platform for needle steering: 6-dof robot holding an 3D ultrasound probe, 6-dof robot actuating a flexible needle, Virtuose 6D haptic device.



[1] N. Abolhassani, R. V. Patel, and M. Moallem, “Needle insertion into soft tissue: A survey,” Medical engineering & physics, vol. 29, no. 4, pp. 413–431, 2007.

[2] C. Papalazarou, P. M. Rongen, et al., “Surgical needle reconstruction using small-angle multi-view x-ray,” in Proc. IEEE International Conference on Image Processing (ICIP), pp. 4193–4196, 2010.

[3] V. Duindam, R. Alterovitz, S. Sastry, and K. Goldberg, “Screwbased motion planning for bevel-tip flexible needles in 3d environments with obstacles,” in Proc. IEEE International Conference on Robotics and Automation (ICRA), pp. 2483–2488, 2008.

[4] K. Hauser, R. Alterovitz, N. Chentanez, A. M. Okamura, and K. Goldberg, “Feedback control for steering needles through 3d deformable tissue using helical paths,” in Proc. Robotics: Science and Systems (RSS), vol. 37, 2009.

[5] M. Abayazid, R. J. Roesthuis, R. Reilink, and S. Misra, “Integrating deflection models and image feedback for real-time flexible needle steering,” IEEE Transactions on Robotics, vol. 29, no. 2, pp. 542–553, 2013.

[6] M. Abayazid, M. Kemp, and S. Misra, “3D flexible needle steering in soft-tissue phantoms using Fiber Bragg Grating sensors,” in Proc. IEEE International Conference on Robotics and Automation (ICRA), pp. 5823–5829, 2013.

[7] J. Troccaz and Y. Delnondedieu, “Semi-active guiding systems in surgery. a two-dof prototype of the passive arm with dynamic constraints (padyc),” Mechatronics, vol. 6, no. 4, pp. 399 – 421, 1996. Mechatronics in Surgery.

[8] M. Jakopec, F. Rodriguez y Baena, S. Harris, P. Gomes, J. Cobb, and B. L. Davies, “The hands-on orthopaedic robot ”acrobot”: Early clinical trials of total knee replacement surgery,” IEEE Transactions on Robotics and Automation, vol. 19, no. 5, pp. 902– 911, 2003.

[9] D. Prattichizzo, C. Pacchierotti, and G. Rosati, “Cutaneous force feedback as a sensory subtraction technique in haptics,” IEEE Transactions on Haptics, vol. 5, no. 4, pp. 289–300, 2012.

[10] S. E. Salcudean, S. Ku, and G. Bell, “Performance measurement in scaled teleoperation for microsurgery,” in Proc. First Joint Conference on Computer Vision, Virtual Reality and Robotics in Medicine and Medial Robotics and Computer-Assisted Surgery, pp. 789–798, 1997.

[11] P. Chatelain, A. Krupa, N. Navab. 3D ultrasound-guided robotic steering of a flexible needle via visual servoing. In IEEE Int. Conf. on Robotics and Automation, ICRA'15, pp. 2250-2255, Seattle, WA, May 2015.

[12] J. Chevrie, A. Krupa, M. Babel. Needle steering fusing direct base manipulation and tip-based control. In IEEE Int. Conf. on Robotics and Automation, ICRA'16, pp. 4450-4455, Stockholm, Sweden, May 2016.

[13] J. Chevrie, A. Krupa, M. Babel. Online prediction of needle shape deformation in moving soft tissues from visual feedback. In IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, IROS'16, pp. 2375-2380, Daejeon, South Korea, October 2016.

Début des travaux: 
October 1st, 2018
Mots clés: 
Medical robotics, shared control, visual servoing, haptic feedback, ultrasound imaging, needle insertion
IRISA - Campus universitaire de Beaulieu, Rennes