Evaluation of Robotic Minimally Invasive Surgical Skills (rMIS) using Motion Studies


  Robotic minimally-invasive-surgery (rMIS) is the fastest growing segment of computer-aided surgical systems today and has often been heralded as the new revolution in healthcare industry. However, the surgical performance-evaluation paradigms have always failed to keep pace with the advances of surgical technology. In this work, we examine extension of traditional manipulative skill assessment with deep roots in performance evaluation in manufacturing industries for applicability to robotic surgical skill evaluation. This method relies on defining task-level segmentation of modular sub-procedures called “Therbligs” that can be combined to perform a given task. Performance metrics including intra- and inter-user performance variance can by analyzed by studying surgeons’ performance over each sub-tasks. Additional metrics on tool-motion measurements, motion economy, and handed-symmetry can be similarly expanded over this temporal segmentation to help characterize performance. Our studies analyzed video recordings of surgical task performance in two settings:  First, we examine performance of two representative manipulation exercises (peg board and pick-and-place) on a da Vinci surgical (SKILLS) simulator to afford a relatively-controlled and standardized testbed for surgeons with varied experience-levels. Second task-sequences from real surgical videos were analyzed with a list of predefined “Therbligs” in order to investigate its usefulness for real implementation.


In this work we propose an automated surgical expertise evaluation method, by adapting well-established motion studies methodologies, especially for MIS evaluation. This method relies on segmenting of a primary task into sub-tasks, which can subsequently be analyzed by statistical analyses of micromotions. Motion studies were developed by 2 methods: (A) Manual annotation process by experts (to serve as a benchmark); and (B) automated kinematic-analysis-of-video techniques; for economy, repeatability as well as dexterity. The da Vinci SKILLS simulator was used to serve as a uniform testbed. Surgeons with varied levels of expertise were recruited to perform two representative simplified tasks (Peg Board and Pick & Place). The automated kinematic analysis of video was compared with the ground truth data (obtained by manual labeling) using misclassification rate and true classification confusion matrix. Future studies aimed towards analyzing real surgical procedures are already underway.

New: visit ARMLAB YouTube Channel - , and research progress on


 Students Involved:

- Seung-kook Jun, PhD Candidate, University at Buffalo

- Madususdanan Sathia Narayanan, PhD Candidate, University at Buffalo

- Priyanshu Agarwal, MS, University at Buffalo


 Movies :

Process of Time Study

- This video shows over all process of time study

- File Size: 23MB [Download]







 Related Publications - Conference Proceedings:


Seung-kook. Jun, Madususdanan Sathia Narayanan et al., Robotic Minimally Invasive Surgical Skill Assessment based on Automated Video-Analysis Motion Studies, 2011 IEEE International Conference on Biomedical Robotics and Biomechatronics, Roma, Italy, Jun 24-28, 2012



Seung-kook. Jun,  Madususdanan Sathia Narayanan et al., Evaluation of Robotic Minimally Invasive Surgical Skills using Motion Studies, Performance Metrics for Intelligent Systems (PerMIS'12) Workshop, Baltimore, Maryland, USA, March 20-22. 2012.



Sponsor: This work was supported in part by the National Science Foundation under Grants, CNS 0751132 and CNS 1135660. .

Last Updated: April 19, 2012