Editor's note: Frank Chang earned an honorable mention for Minority Report Imaging is the New MRI, his entry into the Synapse Storytelling Contest science category.
Augmented reality (AR) is a powerful application that allows physicians to visualize and interact with patient data.
By definition, augmented reality is a real-time interactive experience of the real-world environment using computer-generated perceptual information. The information can be overlaid by adding to or masking from the real-world environment.
AR, however, should not be confused with virtual reality (VR). Although the concepts are similar, there is a major distinction between the two in that VR takes place completely in a simulated environment.
Think Minority Report vs. The Matrix where Tom Cruise manipulates and interacts with floating screens in the real world using special gloves, whereas Keanu Reeves sees everything in green code and has to get transferred back and forth between the Matrix and Zion.
So what’s so special about AR in medicine? Are doctors now also full-time video gamers?
Not quite but they might look like one!
While VR might be beneficial for training purposes through simulation experiences, practicing medical professionals still need to interact with real patients because after all, that is what they do best.
AR has the potential to provide immediate feedback in the real world that could significantly cut down valuable time and ultimately increase the efficiency of hospital workflow.
Multiple studies have tested the feasibility of AR in the clinical setting with promising results, but the accuracy and precision of this modality have yet to be explored in more detail.
To date, augmented reality has been carried out using techniques such as image projection and registration with an optical tracker.
These methods introduce unwanted projection and registration errors which could lead to misinterpretation of the true morphology of the anatomy or structure of interest.
Further robust methods for developing precise and accurate computer-generated information could greatly benefit current treatment and standard of care.
Currently, the concept of generating holograms for augmented reality applications is being explored. The idea of holograms itself is not new as has been demonstrated extensively in movies such as the Star Wars and Iron Man series. Hopefully, this technology can now become a reality.
My research project specifically explores the precision and accuracy of directly translated three-dimensional renderings (or 3D holograms) of phantom models as displayed on the HoloLens platform, an AR headset developed by Microsoft, from various 3D magnetic resonance imaging (MRI) acquisitions.
The ability of the HoloLens to make accurate and reproducible measurements as an instrument has previously been characterized in the field of surgery. In addition, the HoloLens headset is capable of accurately following hand gestures and voice commands for manipulation and interaction of 3D holograms.
Together, the HoloLens has been demonstrated as a competent tool to assist surgeons and other medical professionals with preoperative surgical planning, such as the localization of subsurface vascular perforators, as well as intraoperative landmarking.
From a usability standpoint, the HoloLens AR headset device has received positive reception with users reporting ease of use and the intuitive nature of the interface.
A pipeline for acquiring computed tomography (CT) phantom scans and translating the resulting two-dimensional images into hologram renderings have recently been done by my principal investigator Dr. Jesse Courtier, MD.
Using physical dimensions of the phantom model measured with a caliper as the gold standard, initial findings suggest that there is no statistically significant difference between the gold standard measurements and measurement readings made on the 3D hologram.
We plan to follow and utilize a similar set of pipeline to determine the accuracy and precision of the direct translation of 3D holograms from 3D MRI acquisitions.
As in the case of translating phantom models from CT scans, we hypothesize that there will be no statistically significant difference between gold standard measurements and augmented reality holographic measurement readings of MRI phantom scans.
