Surgical Tool Pose Estimation using Monocular Video

Surgical tool pose estimation has been proven to be useful for high- and low- level feedback tasks including safety enhancement, semantic feedback and surgical skill assessment. Tool pose estimation using monocular camera input is a well studied research problem as the monocular camera is one of the ubiquitous sensor across the spectrum of robotic devices. Current state-of-the art methods for visual tool pose estimation are computationally expensive and require elaborate geometric and appearance models of surgical tools. We propose a visual tool pose estimation method that maps the visual bounding box to the 3D tool pose without any explicit knowledge of tool geometry using Gaussian process regression. The proposed approach can be generalized to any surgical tool and provides tool pose estimates with a variance estimate in real-time. We demonstrate rigorous evaluation of the method under various conditions that might effect the estimation process. In order to evaluate the algorithm, we have instrumented a standard box trainer kit with two laparoscopic tools to get simultaneous ground truth pose and a video feed.

Fig.1 Flow chart of the pose estimation framework

Some of the important guiding principles in choosing Gaussian process for pose estimation were to get 1) Real-time pose estimates 2) Uncertainty in pose values

Fig.2 Data Collection Setup


 Students Involved:

- Suren Kumar, PhD Candidate, University at Buffalo

- Javad Sovizi, PhD Candidate, University at Buffalo


 Related Publications - Conference Proceedings:


Kumar,S., Sovizi, J., Narayanan, M.S., Krovi, V. “Surgical Tool Pose Estimation from Monocular Endoscopic Video”, IEEE International Conference on Robotics and Automation (ICRA), 2015


Questions or comments regarding the website, please contact the webmaster.

Last Updated: May 19, 2015