On Tuesday, September 14, some of healthcare’s greatest AR/VR experts convened virtually to discuss the technology and its applications during the COVID-19 pandemic. After setting the stage, panelists launched into a discussion of AR/VR’s eye tracking capabilities and the information it is providing as it relates to performance and digestion of information for consumers. Discussion of the technical followed with insight into the innovation’s clinical applications and power to transform workflows. The technology’s ability to connect those near and far also found its place in conversation as COVID-19 keeps care teams separate from their patients and each other.
Follow some of this group’s discussion below, and view the full session here
. We want to thank Tobii Tech and Magic Leap for their insights and participation in this session. Be sure to join us for our future Transformation Tuesdays
webinars and the upcoming 2020 Medical Innovation Summit
where experts like these will talk AR/VR and so much more.
Will Morris, MD, Cleveland Clinic:
With us today, we have Dr. Doug Johnston, who is a staff cardiothoracic surgeon at the Cleveland Clinic. We have Jennifer Esposito, Vice President and General Manager of Health at Magic Leap, and Johan Bouvin, Director of XR Systems Software at Tobii Tech. Before I ask the questions, I would love to hear from Johan, tell us a little bit about Tobii Tech, what you guys are actually doing in this space, why you're here on this panel, what's its relevance for you speaking with two physicians and a healthcare organization? And then we'll hear from Jennifer in Magic Leap, and then we'll get the perspective from a frontline surgeon, Doug.
Johan Bouvin, Tobii Tech:
Great, thank you for having me. Tobii has been around for 20 years providing eye tracking technologies for a number of different applications and industries, and our vision has always been that eye tracking will come into all men machine interfaces. We have spent a lot of those years creating products for research for the medical space and assistive technology, and also now layered the last five to seven years for the broader VR and XR industry. So for us, this is a super exciting time where our roots in research and different types of medical application converge with VR and AR are becoming mainstream. And there's a lot of opportunity there that we see.
Will Morris, MD:
Thank you very much. Jennifer, your perspective from Magic Leap? Tell us a little bit about Magic Leap and your overall vision.
Jennifer Esposito, Magic Leap:
Yeah, thanks very much for having me. So I lead the healthcare organization here at Magic Leap. I've been at Magic Leap, probably about 18, 20 months now. I think what some people don't realize, however, is Magic Leap has had people working on healthcare for quite some time, actually since the beginning of the company. And we really see a lot of opportunity for spatial computing across the entire spectrum of healthcare – everything from health and wellness to patient monitoring to telehealth, and all the way to highly technical use cases that involve spatial computing in surgery. So there's really a lot of opportunity in this emerging space. And certainly, with the convergence of some of the other technologies that are out there like AI and 5G and IoT, there's just a lot of opportunity to put all of these wonderful things together from a technology perspective to drive some of those use cases.
And maybe just real quick, just to sort of define spatial computing before we get too far – spatial computing, sometimes people refer to it as mixed reality, it's different from virtual reality in the sense that virtual reality means you can't see the real world around you. It's also different from augmented reality, which is something more like what you can do with your smartphone, for example, or you're able to overlay some piece of digital content in the real world, but there's really no back and forth between the digital and the physical. In spatial computing, what we're trying to do is really bringing the physical and the digital worlds together so that they're aware of each other and that you can interact with them. And so, as I said earlier, this presents a lot of really interesting opportunities to augment the workforce in healthcare and to really drive a lot of interesting new use cases. Looking forward to the rest of the chat here today.
Will Morris, MD:
Great. So maybe, actually, Doug can introduce as a leading surgeon in the number one Cardiac Center in the U.S. for 26 years, innovation in the space and how you perform surgery, how you manage teams of surgeons around a patient. What's your perspective? What have you seen? What do you see are gaps in this space?
Doug Johnston, MD, Cleveland Clinic:
I think that the most interesting thing that we're dealing with now after decades of trying to figure out how to do surgery better and largely working on other elements of the technology, surgical technique, the way we put people on cardiopulmonary bypass, and all those kind of things. We found, and we recently published a paper that there's been an iterative improvement in quality really over the last 20 years or so even at an institution like this one that's been one of the biggest in the country. We've gotten very good at taking care of people safely, but it's still not 100% safe. And I think that if we are to get to the point where other industries like commercial aviation are, in terms of patient safety and avoidance of rare events, then we have to train like these other industries do.
And one of the problems with medicine in general and surgery is that our training to practice cadence is very different. If you're a commercial pilot, you probably spend 30% of your time training. If you're a military pilot, it might be 95% of the time training. Whereas for us, we probably spend 2% of our time training and 98% of it operating, and the rest of the learning for new nurses, new techs, new physicians is really on the job learning. So to me, the potential for these technologies in terms of improving how teams function is twofold. One, and this is what we're really interested in, in the VR space now is to use VR as a tool to say, let's distill down all the stuff that goes down in the operating room, and let's take the manual piece of it out of it. So you don't have to be sowing right now. But we want you to play the role of one of the people in the operating room, take in all the information you would normally take in.
So have situational awareness of what's going on, communicate with your peers, your other members of your team, and go through scenarios that way. And we want to see how you do and how we can refine it. So what's super interesting about that is potentially the ability to use VR to train teams that aren't co-located. So in the year of COVID, you can say we have a resident who's out in the satellite hospital, but every Wednesday morning, we all get on VR together, and we can track. So rather than using heavy bricks and mortar to everybody goes across the street to the simulation center, this is a cheaper, more robust way. And because we can tweak the scenarios, we can train for rare events that people may see in the operating room once a year.
So I really think that is potentially a game-changer. And to get some of the stuff that Tobii Tech is doing, one of the things that we're really interested in is how you get from a novice to an expert at complex scenarios, managing complex scenarios under pressure, one of the ways that you get from a novice to expert is knowing what information to look at. And we've found in the physical world that, if you take a new resident, and you put them in a really stressful situation, they tend to stare at one thing, or well, they stare at the heart, even if the heart's not beating, they stare at the heart, and they don't look around the room and figure out what else is going on. So having the ability to see where their attention is, is I think a tool not only to tune up our scenarios for training in virtual reality, but to understand what are the things that are either helping or distracting people in terms of taking care of patients in a team.
So super exciting space, this is a great panel to get together. And we're very bullish on how this is going to affect not only our training, but our practice over the next several years.