Edgar Rojas, Kyle Couperus, Juan Wachs
Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization. 1-9.
DOI: 10.1080/21681163.2020.1835548
Abstract
Telementoring generalist surgeons as they treat patients can be essential when in situ expertise is not available. However, unreliable network conditions, poor infrastructure, and lack of remote mentors availability can significantly hinder remote intervention. To guide medical practitioners when mentors are unavailable, we present the AI-Medic, the initial steps towards an intelligent artificial system for autonomous medical mentoring. A Deep Learning model is used to predict medical instructions from images of surgical procedures. An encoder-decoder model was trained to predict medical instructions given a view of a surgery. The training was done using the Dataset for AI Surgical Instruction (DAISI), a dataset including images and instructions providing step-by-step demonstrations of 290 different surgical procedures from 20 medical disciplines. The predicted instructions were evaluated using cumulative BLEU scores and input from expert physicians. The evaluation was performed under two settings: with and without providing the model with prior information from test set procedures. According to the BLEU scores, the predicted and ground truth instructions were as high as 86 ± 1 % similar. Additionally, expert physicians subjectively assessed the algorithm subjetively and considered that the predicted descriptions were related to the images. This work provides a baseline for AI algorithms assisting in autonomous medical mentoring.