Augmented Reality can change brain surgery thanks to powerful diagnostic platforms, revolutionize radiology, and open new doors to reconstructive surgery.

Augmented Reality (AR), also known as spatial computing –a merging of digital and physical spaces,– is one of the current technology trends that, together with Virtual Reality (VR) and Mixed Reality (MR), is changing all industries, includinghealthcare and medical education. 

Even though these technologies are enjoying a lot of attention today, they have been around for a long while. In 1965, Robert Mann introduced the first virtual system in medicine (The Thousand Faces of Virtual Reality). It was used to decide what the best procedure for an orthopedic disease would be. It was also used to facilitate a new training environment for orthopedic residents.

In the 1960s, the first simulators with 3D images appeared. Later on, in the late 1980s, the head-mounted display (HMD) (a wearable device) was introduced for VR visualizations in medicine. 

A little over a decade later, the first pioneering applications in medical educationstarted to appear in the form of some hands-on procedures. AR and VR share some technical aspects.

And, even though the development of Augmented Reality starts already in the 1960s, it was not until 1990 that the term Augmented Reality was established as such.

With it, a new era of spatial computing began in the clinical setting. Augmented Reality, today, provides augmented information for the physician and the surgeon during interventional procedures such as Computerized Tomography (CT) and Magnetic Resonance Imaging (MRI) visualization paths. 
Augmented Reality is of great value in the educational world for both students and teachers. The main applications in medical education include representation of mechanisms in space and time dimensions in physiology, or 4D; and 3D visualizations of difficult structures in anatomy. 

Augmented Reality in medicine shows what can be done in the 21st century’s medical environment with digital health.

From brain surgery to reconnecting blood vessels research teams across the world are working closely with Augmented Reality technology companies.

Together, they bring the best of the physical and virtual world to the operation theater. 

Removing brain tumors is soon going to become easier thanks to an augmentation of surgeons’ skills: They soon are going to be able to operate while simultaneously seeing an image floating around in the air of the inside of the patient’s head. 

In the video above, the surgeon used Augmented Reality for craniotomy planningon the skin, the bone, the dura, and also after the craniotomy on the cortex.

A partnership between U.S.-based Magic Leap and Munich-based Brainlab is hoping to add this innovative dimension to surgical procedures and medical imaging in an operating system.

This operating system combines medical technology company Brainlab’s data management, cloud computing, visualization, and data pre-processing software with Augmented Reality technology company Magic Leap’s spatial computing and experiential platforms.

According to the companies, the first release of the product promises to enable surgical planning and simulation in an office setting. 

The companies say this will quickly be expanded to include a next-generation mix of virtual and physical worlds for the operating room, radiotherapy treatment room, intensive care unit, and radiology suite.  

Computing Science graduate students Ian Watts and Michael Fiest at the University of Alberta in Canada developed an Augmented Reality system called ProjectDR.

They were supervised by Pierre Boulanger, Cisco Chair in Healthcare Solutions and Professor in the Faculty of Science, and Greg Kawchuk, Professor in the Faculty of Rehabilitation Medicine at University of Alberta.

ProjectDR allows medical images such as CT scans and MRI data to be displayed directly on a patient’s body. The image then moves as the patient moves thanks to custom software written by Ian Watts. This gives the physician a full view of the patient’s internal anatomy.

The video above shows how ProjectDR works. ProjectDR can be used in surgical planning, accuracy in surgery, in rehabilitation, laparoscopic surgery, physiotherapy, and also in education.

According to the developers, ProjectDR has also the capacity to present segmented images. This means it can only show the lungs or the blood vessels depending on what a clinician needs to see, leaving out any other distraction and achieving more focus in the targeted area. 

At Imperial College London at St. Mary’s Hospital, a major trauma center in London, England, a team of researchers have shown for the first time how Mixed Reality and Microsoft HoloLens headsets can be used while operating on patients undergoing reconstructive lower limb surgery.

According to the Imperial Team, the technology can help surgeons locate and reconnect key blood vessels during reconstructive surgery. This technique could help to improve outcomes for patients. 

According to Dr. Philip Pratt, a Research Fellow in the Department of Surgery and Cancer, his team is one of the first groups in the world using HoloLenssuccessfully in the operating theater.

Using HoloLens, surgeons can overlay scan images onto the patient during the operation; they can see then the bones, the course of the blood vessels, and identify exactly where the targets are located. The technology potentially reduces the time a patient spends under anesthetic and reduce as well as reducing the margin of human error. 

According to a report by Imperial College London, the surgical team reported success in all first five cases with human patients who ranged from a 41-year-old man who had sustained leg injuries during a car accident, to an 85-year-old woman with a compound fractured ankle.

The surgeons found the HoloLens to be a powerful tool in the theater and better than the ultrasound method used to locate blood vessels. And this is just the beginning of how Augmented Reality technology can be used in medicine.