Neurosurgical Skills Assessment: Measuring Technical Proficiency in Neurosurgery Residents through Intraoperative Video Evaluations

World Neurosurgery. dx.doi.org/10.1016/j.wneu.2015.12.052

Neurosurgical Skills Assessment: Measuring Technical Proficiency in Neurosurgery Residents through Intraoperative Video Evaluations

Christopher A. Sarkiss, M.D., Steven Philemond, B.A., James Lee, M.D., Stanislaw Sobotka, Ph.D., Terrell D. Holloway, B.S., Maximillian Moore, Anthony B. Costa, Ph.D., Errol L. Gordon, M.D., Joshua B. Bederson, M.D.

BACKGROUND: Although technical skills are fundamental in neurosurgery, there is little agreement on how to describe, measure, or compare skills among surgeons. The primary goal of this study was to develop a quantitative grading scale for technical surgical performance that distinguishes operator skill when graded by domain experts (residents, attendings, and non-surgeons). Scores provided by raters should be highly reliable with respect to scores from other observers.

METHODS: Neurosurgery residents were fitted with a head-mounted video camera while performing craniotomies under attending supervision. Seven videos, one from each PGY level (1-7), were anonymized and scored by 16 attendings, 8 residents, and 7 non-surgeons using a grading scale. Seven skills were graded; these were incision, efficiency of instrument use, cauterization, tissue handling, drilling/craniotomy, confidence, and training level.

RESULTS: A strong correlation was found between skills score and PGY year (p< .001, ANOVA). Junior residents (PGY 1-3) had significantly lower scores than senior residents (PGY 4-7, p< .001, t-test). Significant variation among junior residents was observed, while senior residents’ scores were not significantly different from one another. Inter-rater reliability, measured against other observers, was high (r= 0.581 ± 0.245, Spearman) as was assessment of resident training level (r= 0.583 ± 0.278, Spearman). Both variables were strongly correlated (r = 0.90, Pearson). Attendings, residents, and non-surgeons did not score differently (p=0.46, ANOVA).

CONCLUSIONS: Technical skills of neurosurgery residents recorded during craniotomy can be measured with high inter-rater reliability. Surgeons and non-surgeons alike readily distinguish different skill levels. This type of assessment could be used to coach residents, to track performance over time, and potentially to compare skill levels. Developing an objective tool to evaluate surgical performance would be useful in several areas of neurosurgery education.

KEYWORDS: neurosurgery simulation, intraoperative evaluations, surgical skills, video-based resident training