How AR and sensor tech is transforming emergency healthcare

In front of you, a man collapses with his hand clutching his chest. You call an ambulance, but it’s clear between you and the operator that immediate action is required before the man suffers a cardiac arrest.

Alongside traditional CPR you find yourself a defibrillator, and, on your knees, place the shock patches on his chest guided either by a 2D image or past basic first aid training.

These days, automated external defibrillators (AEDs) are relatively okay to use without professional help. AEDs can detect when to deliver a shock to a patient and, just as crucially, can detect when not to.

As fortunate as this is, you’re still very much on your own here trying to save a life. If only you had a doctor in your ear seeing through your eyes.

At the Web Summit in Lisbon, Philips chief medical officer Atul Gupta, painted a vision of the future of healthcare and raised the potential of a solution of augmented reality (AR) powered AEDs.

In this instance, with the defibrillator in hand, you could put on a pair of AR glasses, and emergency services can see what you’re seeing and speak to you directly to instruct you on what to do.

The glasses visualise where you can place the patches and can tell you whether a the person’s heart rhythm is shockable and whether or not you should move to CPR.

In this case, the system can then instruct you on the best place to put your hands, keep track of your compression rate and then tell you when there’s a shockable rhythm.

At the same time, you know how far away the ambulance is through the AR glasses, and the paramedics in the ambulance also have a pair of their own synced up to you.

From the ambulance, they’ll know what’s going on and can easily track down your location.

In the UK, ambulance delays are at a record high, with emergency patients waiting hours for an ambulance, and in the accident and emergency room, some are waiting all day just to be seen.

So, technology such as these AR glasses may play a crucial part in making a difference, but with these currently on the trajectory rather than now, healthcare still has a lot of catching up to do.

As Gupta told TechInformed, “I live in an era where we still use beepers and fax machines in hospitals.”

He added: “Hospitals and healthcare are a bit behind the times when it comes to digitalisation, but there is a reason for that: in healthcare and the regulatory process, lives are on the line. So we cannot get anything wrong, it has to be perfect.”

Accordingly, the road maps to create this kind of technology typically take longer to develop than commercial technology. But, Gupta adds, “I do think that there is a role to accelerate the use of digital health even in today’s healthcare environment and that’s what we’re trying to do.”

Augmented reality for healthcare

Gupta, a dual board-certified doctor and radiologist, has used technology throughout his career to perform X-rays, MRIs, ultrasounds and more for over two decades.

Around six and a half years ago, he joined Philips as a chief medical officer and still works in practices alongside working with Philips.

“People that go into this type of field tend to be very techie gadgeteers if you think about it,” he mused.

“Everything we’re working on and working with is digital tech devices and imaging, so of course it came naturally to me,” explains Gupta.

“The best part of my job is dreaming about the future of healthcare – so technologies like AR, robotics, and AI.”

Cameras and technology are a clear soft spot for Gupta, as he takes his passion from taking images of X-rays to taking sensor cameras in the waiting room and then to augmented reality.

“Imagine you arrive at the ER and instead of waiting your turn, what if you knew that the existing security cameras that are already in most of your waiting rooms can now be augmented with innovative sensing technology?”

He described how this tech could potentially be used to triage certain patients in the waiting room area.

The average security camera “can also monitor your temperature and your vital signals.” Plus, “it can automatically flag to medical staff which patients might need more urgent attention.”

“For over a decade, our research has been developing camera-based technology [which] could help caregivers gather patient vital signs from far away,” said Gupta, “and this technology works by looking into the infrared light band using normal cameras, it’s amazing.”

With busy hospital waiting rooms, and lists, healthcare is taking on the remote work trend more than you may think.

Not only are people taking to video calls for remote doctor appointments, but doctors and nurses should soon be able to take it to your home with Philips.

For example, a midwife can perform more virtual tasks by taking he ultrasound to the person’s home, carrying out a scan of the unborn child, and “if they see anything suspicious, they get virtual guidance from the remote expert in the hospital.”

Not only that, but its future ultrasound technology can also create photorealistic ultrasound images in which, perhaps, grandparents that live miles away could virtually “hold” the baby by wearing AR glasses.

In paediatric care, technology can also help reduce anxiety in children during MR scans in hospitals.

“We already have an existing solution that greatly helps,” said Gupta, “What we do is we bring in light and projections to comfort kids, select sound, and even bring animation.”

“It was built by our amazing Philips designers and has since been a big hit with kids.”

And now, with AR in the equation, kids can virtually interact with “Ollie the elephant,” an animation that helps children understand what the scan is, and why it’s important to stay still.

They can also virtually explore the MR scanner from their mobile device at home, “and when it’s time to have the scan in the hospital, they are again guided by their friend Ollie.”

Collaboration is key

“No one company can do it alone,” said Gupta. “Healthcare is too big, so we really do need to collaborate.”

As a case in point, Philips has collaborated with Microsoft and its Hololens 2 to launch augmented-reality powered images of real-life X-rays of patients to help with image-guided therapy such as catheters and stents.

Collaboration also helps with the making of digital technology products.

“I think the wrong thing for a company to do is to create a technology and then throw it out there to the user and make them figure it out,” Gupta explained. “We never do that. We do a process called co-creation.”

This means that every solution that Philips builds will be developed with the end user in mind  from day one. “Because that’s the only way to know if you’re on the right track.”