In this project we use body-worn cameras and Mixed Reality (MR) technologies to augment activities in the operating theatre, and provide additional information to surgical staff during an operation.
We have a particular focus on improving the training of junior surgeons, and improving their interaction with the instructing surgeon during the operation. By streaming the surgeon’s first-person view to display devices in the environment, trainees can follow the procedure up close without issues of occlusion or encumbering the theatre’s personnel.
Additionally, we aim to enable the annotation of the video stream, for example to highlight areas of interest, point out details, and thus enhance communication during the operation. Using Augmented Reality technology, those digital artefacts can be perceived by everyone in the room, thus improving the communication amongst the surgical team.
Finally, we are exploring the use of gestures and of physical markers as a means for the team to interact this the system. For example a physical marker may be digitally linked to the patient’s x-rays; positioning the marker in the room can allow the team to see the x-rays appear at the location of the marker. By repositioning the marker, it is possible for the team to adjust the physical layout of the information, and locate it in a more convenient place.