Ryan Lohre, Aaron J. Bois, J. W. Pollock, pater Lapner, Katie McIlquham, George S. Athwal, Danny P. Goel
JAMA Netw Open. 2020;3(12):e2031217.
DOI: 10.1001/jamanetworkopen.2020.31217
Abstract
Importance: Video learning prior to surgery is common practice for trainees and surgeons, and immersive virtual reality (IVR) simulators are of increasing interest for surgical training. The training effectiveness of IVR compared with video training in complex skill acquisition should be studied.
Objectives: To evaluate whether IVR improves learning effectiveness for surgical trainees and to validate a VR rating scale through correlation to real-world performance.
Design, setting, and participants: This block randomized, intervention-controlled clinical trial included senior (ie, postgraduate year 4 and 5) orthopedic surgery residents from multiple institutions in Canada during a single training course. An intention-to-treat analysis was performed. Data were collected from January 30 to February 1, 2020.
Intervention: An IVR training platform providing a case-based module for reverse shoulder arthroplasty (RSA) for advanced rotator cuff tear arthropathy. Participants were permitted to repeat the module indefinitely.
Main outcomes and measures: The primary outcome measure was a validated performance metric for both the intervention and control groups (Objective Structured Assessment of Technical Skills [OSATS]). Secondary measures included transfer of training (ToT), transfer effectiveness ratio (TER), and cost-effectiveness (CER) ratios of IVR training compared with control. Additional secondary measures included IVR performance metrics measured on a novel rating scale compared with real-world performance.
Results: A total of 18 senior surgical residents participated; 9 (50%) were randomized to the IVR group and 9 (50%) to the control group. Participant demographic characteristics were not different for age (mean [SD] age: IVR group, 31.1 [2.8] years; control group, 31.0 [2.7] years), gender (IVR group, 8 [89%] men; control group, 6 [67%] men), surgical experience (mean [SD] experience with RSA: IVR group, 3.3 [0.9]; control group, 3.2 [0.4]), or prior simulator use (had experience: IVR group 6 [67%]; control group, 4 [44%]). The IVR group completed training 387% faster considering a single repetition (mean [SD] time for IVR group: 4.1 [2.5] minutes; mean [SD] time for control group: 16.1 [2.6] minutes; difference, 12.0 minutes; 95% CI, 8.8-14.0 minutes; P < .001). The IVR group had significantly better mean (SD) OSATS scores than the control group (15.9 [2.5] vs 9.4 [3.2]; difference, 6.9; 95% CI, 3.3-9.7; P < .001). The IVR group also demonstrated higher mean (SD) verbal questioning scores (4.1 [1.0] vs 2.2 [1.7]; difference, 1.9; 95% CI, 0.1-3.3; P = .03). The IVR score (ie, Precision Score) had a strong correlation to real-world OSATS scores (r = 0.74) and final implant position (r = 0.73). The ToT was 59.4%, based on the OSATS score. The TER was 0.79, and the system was 34 times more cost-effective than control, based on CER.
Conclusions and relevance: In this study, surgical training with IVR demonstrated superior learning efficiency, knowledge, and skill transfer. The TER of 0.79 substituted for 47.4 minutes of operating room time when IVR was used for 60 minutes.