Holodeck Medicine: How Immersive Technology Is Changing the Doctor’s Point Of View

No Comments

One of the hottest trends in the computer industry is augmented and virtual reality, a computing environment which partially or completely skews reality with computer-generated imagery.

Somewhat like a primitive version of Star Trek’s “holodeck,” this technology allows a user an escape from real life with immersive sound, lifelike visuals, and interactive, gesture-based interaction.

While virtual reality is finding itself increasingly in the next generation of video games, both augmented reality, in which images and information is superimposed over real-world images, and virtual reality, in which one’s entire field of view is completely computer-generated, is also being trialed in the healthcare industry, namely medical education.

While an experienced doctor is still absolutely essential, “augmented medicine” and other new technologies are allowing them to extend their talents even further and learn even more about the human body and disease. For patients, this could mean reduced costs and better outcomes for surgical procedures and treatments.

Here’s our list of notable technologies that you might someday see in a medical school or doctor’s office near you:

Smartglass Learning
Ever since the launch of Google Glass in 2013, the tech industry has eagerly speculated about how each field of medicine would embrace it. In August 2013, a surgeon at the Ohio State University Medical Center wore Google Glass during an ACL repair as medical students watched the live-stream on laptops in another area, and since then, a number of doctors have started using Google Glass to stream and record their surgical procedures for educational purposes.

Google Glass was officially discontinued in its current form in early 2015, but some are still hopeful that smartglasses might still be the next revolution in surgery technology. One company, Pristine, has developed a program called EyeSight to stream and record live video and photos from the smartglass user’s point of view. This content can be shared with a colleague with a simple voice command.

Another company, inSight Augmented Medicine, has developed a similar app called Telepresence that can transmit the smartglass user’s point of view to a remote user’s tablet. The tablet user can annotate the video or picture with their finger, and the video feed with annotations, drawings, and text is transmitted back to the smartglasses and seen through its display.

Both EyeSight and Telepresence were developed for multiple smartglass platforms, so Google Glass is not required.

Microsoft Kinect
One gadget that could prove beneficial during surgery may already be sitting next to your TV. Microsoft’s Kinect motion-sensing camera has already shown itself to be a useful tool to help surgeons view and manipulate medical images. Within the sterile field, surgeons are usually unable to operate a mouse, keyboard, or even a touch screen with their hands. The Kinect makes viewing and manipulating images and patient data as simple as waving their arms.

Moreover, the camera in the Kinect is sensitive enough even to detect subtle changes in a patient’s skin color. An additional Kinect device could theoretically be aimed at the patient being operated on to monitor a user’s heart rate or track radiation exposure from x-rays or CT scans.

Immersive Surgical Navigation & Planning
Brain surgery has always been a risky undertaking. Not only because the brain is the control center for the entire body, but also because it contains a dense network of blood vessels and anatomical features that are unique for every person and extremely delicate to navigate.

One company, Surgical Theater, took inspiration from flight simulation technology and developed a platform which integrates CT and MRI scans and traditional x-rays to create a highly detailed three-dimensional model of the part of the brain being operated on. These models can be manipulated and used for planning the best entry/incision point, the best path around the patient’s vasculature and the minimum amount of skull bone needed to be removed to facilitate faster healing. Recently, Surgical Theater received approval in the EU to incorporate virtual reality headsets for enhanced navigation and planning.

Once the procedure is planned, the details can be imported into another Surgical Theater program which allows the surgeon to see the 3D model and plan in real time from inside the operating room.

Tablet-Based Augmented Reality
Tablet computers have transformed the computing industry because of their portability and ever-increasing processing power.

Fraunhofer MEVIS research center in Germany has harnessed the power of the tablet computer and developed an augmented reality app to assist with liver cancer surgery. CT scans are used to create a model of the liver and its cancerous site to assist with preoperative planning. However, instead ofbeing sent to a stationary computer monitor, the model is sent to a tablet computer. As a result, a surgeon can superimpose the exact locations of important blood vessels and anatomical features during a procedure when the tablet is held over the patient’s actual liver. The liver can be filmed with the built-in camera, and the tablet can be operated as normal with touch gestures.

This technology helps ensure that the surgeon doesn’t make any unnecessary cuts, can make adjustments quickly and flexibly, and completely remove the cancer.

Oculus Rift Goes Way Beyond Gaming
Like the Kinect, the Oculus Rift is another gaming device that is dabbling in the medical technology industry. The Oculus Rift is a virtual reality headset that displays a fully immersive 3D experience that makes you feel like you are actually in the middle of an environment. Motion sensors detect when you move your head and change your perspective accordingly. Now owned by Facebook, Oculus Rift is still looking for its niche in gaming and consumer media, but it’s already been used as an immersive medical learning tool.

Last July, a surgeon in France wore a GoPro Dual Hero camera system on his head as he performed a total hip replacement surgery. The cameras created a stereoscopic 3D video that could be viewed through the Oculus Rift. Moreover, the sensors in the Oculus Rift allowed the viewer to move his or her head and focus on different aspects of the procedure, such as the surgical site or the assistants. The team behind the surgery hopes that doctors anywhere in the world would someday be able to observe a surgical procedure by simply donning an Oculus Rift. They believe that the headset can be a revolutionary learning tool for both new and experienced surgeons.

Doctors aren’t the only ones that have all the fun when it comes to Oculus Rift, however. They’re also being evaluated in patients as a form of virtual reality therapy (VRT). Dr. Albert “Skip” Rizzo of the University of Southern California’s Institute for Creative Technologies has developed an Oculus Rift version of the Virtual Iraq/Afghanistan PTSD Exposure Therapy System that he created in 2005 to help treat war veterans suffering from PTSD. The system uses virtual reality to recreate combat situations to help alleviate fearful associations linked to traumatic memories and has been shown to significantly reduce PTSD symptoms. According to Rizzo, the lower price point of the Oculus Rift will make it much more affordable for clinics and allow for research to expand to other areas of mental health.


Leave A Reply