Premium
This is an archive article published on September 6, 2022

Cutting Edge: How HoloLens2, a new imaging device, guided an innovative shoulder replacement

‘A helmeted device made by Microsoft with a computer and eye-tracking camera within, it allows the surgeon to view 3D images in real time,’ says Pune-based joint replacement surgeon Ashish Babhulkar

Dr Ashish Babhulkar began training in the HoloLens 2 in July 2020 and after mastering the skill curve, started integrating patient data. (Express photo)Dr Ashish Babhulkar began training in the HoloLens 2 in July 2020 and after mastering the skill curve, started integrating patient data. (Express photo)

As a surgeon performing a complex shoulder replacement, Pune’s Dr Ashish Babhulkar often had to revisit printed films on a viewing box and twist his head almost 180 degrees for that precise detail. HoloLens2 has changed all that. A helmeted device made by Microsoft with a computer and eye-tracking camera within, it allows the surgeon to view 3D images in real time.

“Not only do I have the entire set of patient CT scans and X-ray within my eyesight, I can also increase, widen, rotate, twist and move the 3D virtual image far or close to me. My iris and finger control can resize the image to different proportions and tuck the image in a corner, only to be retrieved at will,” says the founder-president, Shoulder and Elbow Society, and consultant shoulder and joint replacement surgeon at Deenanath Mangeshkar hospital.

CHEAPER TECH TOOL THAT HELPS IN PRECISE SURGERY

“So far we have always experienced cutting edge technology introduced at a premium cost. A surgical robot costs anything between Rs 6 to 10 crore. The HoloLens 2 comes at a fraction of that cost and for an economically-conscious country like India, where health costs are unaffordable, HoloLens 2 is going to be a game changer,” Dr Babhulkar says.

When he is operating with the HoloLens2, he has several screens in front of him and the most vital is the 3D imagery of the patient’s CT scan. “I can rotate, enlarge and superimpose this image over the exact bone that I am expected to replace. I use gesture controls in the air to manipulate the 3D hologram without compromising the sterility of the field. That way I have this pictorial information at my finger tips and keep to the sterile requirements of surgery. Second, I upload the patients’ X-rays and clinical information in a separate space. Third, I have a 3D graphic step-by-step guide of the surgical procedure at hand, in case a young surgeon needs it. Far more significant than these is the ability of a remote surgeon or student to have a seamless, unobstructed view of my entire technique, sitting in any city of the world. That to me is unlimited access to the highest level of mentoring, unparalleled in any branch of education. Imagine how a young surgeon in Kenya with a Hololens2 can be trained, tutored and assessed without compromise by a surgeon in India without leaving his premises. All both would need is a good seamless internet connection,” Dr Babhulkar adds.

HOW SUCCESSFUL HAS THE DEVICE BEEN?

So far, the surgeon has performed nearly 12 mixed reality assisted replacements that involve uploading the patient’s CT scan to the Hololens2 with special protocols.

Dr Babhulkar began training in the HoloLens 2 in July 2020 and after mastering the skill curve, started integrating patient data. Conventional CT scan files must be converted with a special software and uploaded to the HoloLens 2 chip. It takes about two hours to prepare a case and integrate it into the HoloLens2. After a trial run, two patients were operated in November and December with HoloLens assisted Reverse Shoulder replacement.

The first patient was a 72-year-old woman with a massive tear of her rotator cuff tendons that led to secondary arthritis of her shoulder joint. The woman came with a very stiff shoulder joint. She was unable to comb her hair or touch her back. Besides, she also had severe pain in her shoulder which became excruciating at night. She was from a different city and was delayed due to the lockdown restrictions. Her CT scan images were converted with a special protocol to create HoloLens 2 compatible 3D images.

Story continues below this ad

The team of surgeons had to establish a secure broadband, providing a 90 MBPS seamless transfer of data. The plan also included a watch-in by two eminent shoulder surgeons from Paris, France and London, UK almost 7,300 km away. away. Among them were Thomas Gregory from Paris, the brain behind the concept of integrating HoloLens2 and Shoulder Replacement protocol and Prof Roger Emery, London. They were receiving the exact surgical field that Dr Babhulkar was looking through the HoloLens2 Visor. “They had the perfect, unblemished, live picture and could follow my every gaze and movement. Several times during surgery, they stopped me and asked me to explain my atypical technique of performing a particular step during surgery. The surgery – A Reverse Shoulder replacement – went through smoothly. The patient recovered and has now started her exercises. She is almost pain-free and unshackled of her stiffness and disability,” Dr Babhulkar says.

WHAT IS SHOULDER ARTHRITIS?

Shoulder arthritis is still unrecognised due to a lack of focus and the availability of a few shoulder specialists. It, however, remains the third most common cause of arthritis and can be very disabling. “A perfect surgery, pain relief and function restoration is a big challenge. After the initial success with knee and hip replacement, other joints such as shoulder and elbow were replicated with almost similar success. It was appropriate to push the envelope further and this century, technology was married to surgical skills, to introduce the engineers to enhance surgical skills and precision,” Dr Babhulkar says.

HOW DO THE SCANS WORK?

Dr Gregory collaborated with Microsoft and helped develop algorithms that would be led by the operating surgeon’s eye movements and integrate real 3D imaging. The concept was to improve accuracy to the last millimetre. Alex Kipman, a Microsoft scientist, elevated the original Kinect line of motion sensing device to an elite, powerful and technically superior HoloLens 2. The visor can integrate a virtual image in 3D and also allow gesture control and simulation. “This level of mixed reality has never yet been integrated into surgical planning and execution. I have always used 3D CT scan data to plan my shoulder replacements. These advanced 128 slice 3D CT scans (Till now the most advanced CT scans at Deenanath Mangeshkar hospital, Pune) are then printed on plain paper with multiple images. With Hololens2, I have the entire set of patients’ CT scans and X-ray within my eyesight,” Dr Babhulkar says.

Anuradha Mascarenhas is a Senior Editor at The Indian Express, based in Pune. With a career spanning three decades, she is one of the most respected voices in Indian journalism regarding healthcare, science and environment and research developments. She also takes a keen interest in covering women's issues . Professional Background Education: A gold medalist in Communication and Journalism from Savitribai Phule Pune University and a Master’s degree in Literature. Author: She authored the biography At The Wheel Of Research, which chronicles the life and work of Dr. Soumya Swaminathan, the former Chief Scientist at the WHO. Key Focus: She combines scientific accuracy with storytelling, translating complex medical research into compelling public and human-interest narratives. Awards and Recognition Anuradha has won several awards including the Press Council of India's national award for excellence in journalism under the gender based reporting category in 2019 and the Laadli Media award (gender sensitivity -2024). A recipient of the Lokmat journalism award (gender category-2022), she was also shortlisted for the RedInk awards for excellence in journalism-2021. Her debut book At The Wheel Of Research, an exclusive biography of Dr Soumya Swaminathan the inaugural chief scientist of World Health Organisation was also nominated in the Popular Choice Category of JK Paper AUTHER awards. She has also secured competitive fellowships including the Laadli Media Fellowship (2022), the Survivors Against TB – New Research in TB Media Fellowship (2023) and is part of the prestigious 2025 India Cohort of the WomenLift Health Leadership Journey.” Recent Notable Articles (Late 2025) 1. Cancer & Specialized Medical Care "Tata Memorial finds way to kill drug-resistant cancer cells" (Nov 26, 2025): Reporting on a breakthrough for triple-negative breast cancer, one of the most aggressive forms of the disease. Discipline, diet and purpose; How a 97-year-old professor defies ageing'' (Nov 15, 2025) Report about Prof Gururaj Mutalik, the first Head of Department at Pune's B J Government Medical College who at 97 credits his longevity to healthy habits and a strong sense of purpose. 2. Environmental Health (The "Breathless Pune" Series) Long-term exposure even to 'moderate' air leads to chronic heart, lung, kidney issues" (Nov 26, 2025): Part of an investigative series highlighting that even "safe" pollution levels are damaging to vital organs. "For every 10 µg/m3 increase in PM2.5 level, there was 6-8% jump in medicine sales" (Nov 23, 2025): Using commercial data to prove the direct link between air quality and respiratory illnesses in Pune. 3. Lifestyle & Wellness News "They didn't let cancer, diabetes and heart disease stop them from travelling" (Dec 22, 2025): A collaborative piece featuring survivors who share practical tips for traveling with chronic conditions. At 17, his BP shot up to 200/120 mmHG; Lancet study flags why child and teen hypertension doubled between 2000 and 2020'' (Nov 12,2025)--A report that focusses on 17-year-old-boy's hypertensive crisis and reflects the rising global trend of high blood pressure among children and adolescents. 4. Scientific Recognition & Infrastructure For promoting sci-comm, gender diversity: IUCAA woman prof highlighted in Nature" (Nov 25, 2025): Covering the global recognition of Indian women scientists in gender studies and physics. Pune researchers find a spiral galaxy like the Milky Way from early universe'' (December 3, 2025)- A report on how Indian researchers discovered a massive galaxy that existed when the universe was just 1.5 billion years old , one of the earliest to have been observed so far. Signature Beat: Health, Science & Women in Leadership Anuradha is known for her COVID-19 reportage, where she was one of the first journalists to provide detailed insights into the Covishield and Covaxin trials. She has a dedicated interest in gender diversity in health and science, often profiling women researchers who are breaking the "leaky pipeline" in STEM fields. Her writing style is scrupulous, often featuring interviews with top-tier scientists and health experts from various institutions.   ... Read More


📣 For more lifestyle news, click here to join our WhatsApp Channel and also follow us on Instagram

Latest Comment
Post Comment
Read Comments
Advertisement
Advertisement
Advertisement
Advertisement