Mixed-Reality and Midnight Diner: A Surgeon's Dreams

Published on 13 March 2024

When not harnessing technology to perform complex liver transplants, Dr Gao Yujia dreams of running a hole-in-the-wall eatery.

With utmost precision, Dr Gao Yujia can deftly scale and fillet a whole fish for sashimi. In fact, he finds it much easier than making sushi rice. “I like working with my hands, that’s why I decided to specialise in surgery,” he said. “It was either that or become a chef!”

His desire to help others through medicine prevailed, due to his interest in biology and frequent visits to the hospital when his mother was ill during his teens. “Seeing the passion and drive of the specialists and other doctors — and how they tried their best to help my mum — drove me to want to do something to help the people around me, should they get sick,” he recalled.

Dr Gao Yujia is a Consultant at Division of Hepatobiliary & Pancreatic Surgery, Department of Surgery,  National University Hospital (NUH) and a Consultant, Adult Liver Transplantation Programme,  National University Centre for Organ Transplantation (NUCOT), NUH. He is also a Consultant at Division of Hepatobiliary & Pancreatic Surgery, Department of General Surgery,  Ng Teng Fong General Hospital (NTFGH), and the Assistant Group Chief Technology Officer at  National University Health System (NUHS).

Interestingly, Dr Gao is the only medical practitioner in his family of engineers. His father worked in the telecoms industry, and his mother is a retired computer scientist. He nurtured a passion for technology from a young age, helping his dad test the latest mobile phones, and building his own computer from scratch.

Then in 2019, his professional and personal interests intersected when a corridor chat with A/Prof Ngiam Kee Yuan, Group Chief Technology Officer, NUHS, led to an invite to a workshop on HoloLens — a mixed reality (MR) headset used in healthcare education and simulation.

“We purchased three devices and went around to all the different specialties within NUH, gathering feedback over the course of five or six months,” he said.

“The positive response was unanimous and we received approval to begin the HoloMedicine research and development programme in 2021.” The programme aims to explore the use of MR technology — which merges virtual and augmented reality — in clinical care.

From a two-person team comprising Dr Gao and then-final-year NUS engineering student Ng Kian Wei, the HoloMedicine team today is eight-people-strong, comprising software developers, engineers, data scientists, clinicians, a network architect and even 5G specialists who oversaw the recent deployment of the private 5G network within NUH to support the project.

Healing through a HoloLens

So, how exactly does mixed-reality enable healthcare professionals such as Dr Gao to better perform their jobs?

Imagine you are in an Iron Man movie, where Tony Stark can move the projected components of a hologram around in the room — that’s essentially what you can see through the HoloLens,” Dr Gao explained. “Using software, we can convert a patient’s CT or MRI scans into 3D holograms, which can be zoomed in or out, moved around, or even superimposed directly onto the patient.

While this experimental technology has yet to become mainstream, NUH has already used the HoloLens for some 120 surgeries across different specialities. It is especially beneficial when preparing for an operation, providing a more accurate visualisation of the patient’s spatial anatomy, especially for surgeries involving more complex organs. “Sometimes, we also wear the device during the operation itself, which gives us ‘X-Ray vision’ of where a specific tumour or growth is,” added Dr Gao. 

He illustrated the case of adrenal keyhole surgery, where a camera and small instruments are inserted into the abdomen through minor incisions. This procedure enables the surgeon to fully extract the diseased adrenal gland. The challenge arises from the difficulty in locating tumours in this region due to the presence of surrounding fat, upon which the adrenal gland rests. Furthermore, the tumour's camouflage effect is compounded by its nearly identical colour to that of fat tissue.

However, with mixed-reality, surgeons can achieve almost to pinpoint accuracy, reducing the risks of accidentally injuring a nearby structure such as a blood vessel. This can halve operating time from two to one hour or less, and can reduce the patient’s healing time.

Beyond the operating theatre, mixed-reality has myriad applications. One of the HoloMedicine team’s first projects involved using the infrared camera on the HoloLens to locate patients’ veins while drawing blood more accurately. Patients themselves can also use the HoloLens to guide them during rehab, or to gain a better understanding when discussing their upcoming surgery.

Dr Gao hopes that MR devices will eventually obtain the necessary accreditation and medical licences to be considered a standard medical device, as he has already witnessed its positive impact on patients.

“We envision a future where doctors and nurses walk around the hospital wearing MR glasses, on which information pops up when they look at patients,” said Dr Gao. He added that his team’s work is still in their developmental phases, and it will take at least five to 10 years for this vision to materialise.

Nursing big aspirations

During his off-duty hours, Dr Gao dedicates much of his time to his family. He enjoys baking alongside his five-year-old daughter during weekends, tending to his newborn son and enjoys watching children’s programmes with them.

Such family time is precious, given that Dr Gao was recently posted on a six-month fellowship at a renowned medical centre in South Korea, where he gained experience performing minimally-invasive liver transplant surgeries. 

With such a busy schedule, it isn’t surprising that Dr Gao’s hobbies have taken a back seat. The once avid cook admitted that he has not touched a stove for over two years. His dream is to set up a small eatery in a low-key location when he retires. “It will have limited seating — perhaps 20 guests a night — with a fixed menu. Just come and eat whatever we prepare, mingle and chit-chat, you know, like in the Netflix series Midnight Diner: Tokyo Stories,” he mused. “It’s still sort of my pipedream to become a chef.”

In consultation with Dr Gao Yujia, Consultant, Division of Hepatobiliary & Pancreatic Surgery, Department of Surgery, NUH; Consultant, Adult Liver Transplantation Programme, NUCOT, NUH; and Consultant, Division of Hepatobiliary & Pancreatic Surgery, Department of General Surgery,  NTFGH; Assistant Group Chief Technology Officer at  National University Health System (NUHS).