You’ve heard of virtual reality, maybe you’ve even tried Google Cardboard, but the world of augmented and mixed reality is still a mystery, as is its potential impact on healthcare delivery. Justin Barad breaks down what physicians need to know.
In 2012 a Kickstarter launched for a virtual reality headset called the Oculus Rift. The campaign raised $2.4 million – shooting past its $250,000 goal. What started as a garage-based pet project has turned into a worldwide phenomenon and two new computing platforms: Virtual and augmented reality. The market for virtual reality (VR) and augmented reality (AR) is projected to reach $160 billion by 2020. One of the major useful applications for this technology is in the area of healthcare. In this article, I will try and list a few of the interesting uses of VR and AR in medicine which I have broken down into therapeutics, visualization, surgical navigation, patient education, training, telepresence, telementoring and workflow/EMR-integration.
To begin with I would like to explain the difference between Virtual Reality, Augmented Reality, and the somewhat newer term Mixed Reality. Virtual Reality implies the use of a fully immersive headset display that completely replaces the world around you. Virtual Reality includes 360 video and film, in which you are experiencing real world camera-captured content, or simulated content, in which everything you look at or interact with is created by a computer. “Mobile” VR is a term used to describe VR content that is experienced by strapping a phone to your face with a compatible headset.
Things like Google cardboard, Google daydream and the Samsung GearVR are examples of Mobile VR. Mobile VR currently lacks positional tracking, so your movement in the virtual world is restricted to the rotation of your head. There is also currently no 1:1 hand-tracking available for mobile VR. On the higher quality end of the VR spectrum is what’s commonly referred to as “Tethered” VR, which would be the Oculus Rift with Touch Controllers, the HTC Vive, and Playstation VR. These devices use tracking systems to allow you to “move around” the virtual world, which tremendously increases the immersion and overall VR experience. Furthermore, your hands are present in the virtual world and you can interact with it in a very natural way that needs to be experienced to be believed. The Vive and Rift require relatively powerful computers and GPUs to support them, however the cost and system requirements are decreasing steadily. Playstation VR runs off of the PS4.
Now let’s talk about Augmented Reality, or AR. AR exploded in a big way in 2016 with the release of Pokemon GO, but this also led to a lot of confusion as to what is or isn’t augmented reality. The term augmented reality, meaning a view of the real world “augmented” with computer generated visuals could apply to anything from a phone to a highly sophisticated holographic headset display. For this and other reasons Microsoft started pushing the term “Mixed Reality” to describe its holographic headset display technology, in which holograms and information can be placed into the real world around you in a way that makes it seemingly become a part of the environment. Other holographic display headsets include the Meta 2, the DAQRI, the Vuzix M100/300 and Magic Leap among others.
Therapeutics is an exciting area that is laser focused in the VR arena at the moment because it gives you the power to really take control of a patient’s brain in order to ease their pain, anxiety and many other conditions. One of the more exciting companies in the space is Applied VR, which has been conducting some interesting clinical research in partnership with Dr. Brennan Spiegel at Cedars Sinai. Applied VR is creating a validated VR “pharmacy” to treat patients in and out of the hospital. Some of their early results have to be seen to be believed, and the testimonials and videos they have are truly moving. Early results show that VR can decrease the use of deadly and addicting narcotic medications. VR is also being used to treat some psychological conditions such as PTSD and various phobias. Vivid Vision is another interesting company in the therapeutics space. They are using VR to treat various ophthalmologic conditions such as strabismus, amblyopia, and convergence disorders.
In terms of therapeutic AR, an exciting company called Brainpower is using this technology to help patients with autism. I had the opportunity to see psychiatrist Dr. Arshya Vahbzadeh, Brainpower’s CMO, and discuss a demo of the technology in which Brainpower enables a young child to make eye contact with his mother and to recognize her emotions. The mother is brought to tears as she exclaims that he has never before made eye contact in this way with her.
VR and AR companies are taking on medical image visualization in a big way. Seeing a 3D reconstructed MRI or CT in VR and AR can give you insight into anatomy, approaches and pathology that you sometimes could not have seen otherwise. It is also tremendously cool. Companies in this space that are approaching VR visualization include Echopixel, Surgical Theater, and Bioflight VR. In the AR space NYC-based MediVis is bringing 3d imaging to mixed reality.
Surgical navigation may see a major utility shift with the use of AR technology. Currently when using navigation systems a surgeon must look up or to the side at a screen which is ergonomically suboptimal and also requires a lot of different “views” to figure out where s/he really is in physical and virtual space. Integrating a holographic headset will allow surgeons to “see through” the patient in a natural stereoscopic way that should streamline and hopefully improve the utility of navigation technologies. Start-ups like Augmedics are using this approach for spine surgery, and Phillips just recently announced an interesting augmented application for their navigated Hybrid OR.
Patient education will likely be an early area of VR and AR innovation. It is often hard for patients to understand the complex nature of their disease states, treatment options, and also for them to have a spatial sense of the 3D aspects of their anatomy. Applications like The Body VR allow patients to get a better understanding of these challenging concepts with creative techniques like allowing them to “walk-through” their own anatomy. Another example comes from Lighthaus, which produced an interactive simulation for pediatric cardiac surgery for Lucile Packard Children’s Hospital.
I think VR and AR will have one of the biggest impacts in the world of medical training. The apprenticeship model has been used for physician training for over a century, yet it is starting to show its age.
My personal experience in VR comes from the work I’ve done at Osso VR, where we are providing a way for physicians to practice critical procedures wherever and whenever they want. This will provide a way to standardize physician training and ensure that there are no “training gaps.” VR will also allow more established surgeons to transition to newer and more effective technology, as often the lack of time or ability to travel to expensive training courses limits the adoption of new devices and techniques.
Telepresence is something that has existed in hospitals for some time both for patients and physicians, however VR is going to increase its effectiveness in a major way. Dr. Spiegel from Cedars-Sinai showed a demonstration at CES’s Digital Health Summit in which a patient who had been in the hospital for months was “transported” home via a live-streaming 360 camera and a mobile VR headset. The reaction was incredibly moving. This demonstration is only scratching the surface of what this technology is capable of.
Telementoring is an interesting area that is useful for procedures and surgeries. The concept is that if the performing physician is new or has any questions they can reach out to an expert who can see what’s happening, assess, and provide advice in addition to showing the provider what to do. An early example of this technology was VIPAAR, which used Google Glass and an iPad to accomplish telementoring. It is likely that this technology will start to become more mainstream with some of the newer holographic headsets coming down the pipeline to help democratize access to more advanced surgical treatments.
EMRs have been making the lives of physicians and healthcare providers quite difficult over the past few years. AR technology may be a way to ease that burden. Companies like Augmedix use AR technology like Google Glass to provide “virtual scribes” that can document and code a patient visit without requiring a physical scribe to be present. Furthermore, they can fetch data from compatible EMR systems to be displayed on your headset to allow providers to focus more on patients and less on their computers.
Honestly these are only a few of the applications of VR and AR we’re going to be seeing in the next few years. As physicians, it will be up to us to be open-minded about these new applications, provide resources and validation to test their effectiveness, and provide insight into where the true needs are and what innovators should be focusing on. I have no doubt that VR and AR are going improve outcomes and decrease costs for patients in addition to improving the quality of life for providers.