AI at Scale: Does It Deliver?

Transcript

AI Usage Disclosure: This transcript was created with assistance from AI tools. It was reviewed and edited by KFF Staff.

Chip Kahn: Today we move from the wide angle to the front line. We turn to the actual application of AI to clinical care and hospital operations across a real health system at full scale. My guest is Dr. Michael Schlosser, Senior Vice President and Chief Transformation Officer at HCA Healthcare. 190 hospitals; 2,500 ambulatory sites; more than 47 million patient encounters a year. No other private sector operator is that large and no one else deploys AI at that scale. What you will hear is how AI in health care actually gets developed for everyday use. It starts with careful testing and customizing with clinicians and nurses engaged as end users from the very beginning. Then comes the path from a promising pilot to enterprise-wide deployment. And then comes what is often overlooked, the work that begins after deployment. When navigation, feedback and course correction take over. You cannot get away from the miraculous technology of AI. But where the rubber hits the road, two things matter. The first is data. Its breadth, its integrity, the infrastructure that holds it. To borrow a decades old phrase, it’s the data, stupid. The second is the human factor. What determines in the end whether a solution actually works to further the health care mission. 

Let’s get started. Mike Schlosser, welcome to KFF’s Business of Health. 

Dr. Michael Schlosser: Glad to be here Chip, thanks for inviting me. 

Chip Kahn: This is going to be a great conversation. You and I have talked before, and I know that our audience is going to learn so much today. Let’s start off with the obvious, which is HCA Healthcare is a huge organization. You’re deploying AI across 189 hospitals with 47 million patient encounters a year. When a AI solution company hands you sensitivity, specificity and accuracy numbers on a product, what criteria does that solution have to meet before you deploy it across this sort of massive set of hospitals and ambulatory surgery centers and other kinds of care settings? 

Dr. Michael Schlosser: Yeah, it’s a great question. And yes, the scale of HCA is a superpower for us, but it also can be a challenge. But to answer your question directly, the model performance metrics themselves, the sensitivity, specificity, F1 score to me really is a small piece of the story. We really think more about sort of the human AI system, because most of the tools that we’re deploying, the human is still in the loop, right? The people are taking advantage of whatever the AI is producing for them or enhancing their workflow, whatever the product might be. And so in the end it’s really the impact or the outcome of that system that we really care about. So said a much simpler way, are we actually solving the problem? Like, do we see a better clinical outcome? Do we see a better orchestration of care or operational outcome? Are we more efficient? Are we reducing cost? And so those are the KPIs that we track very carefully. Now, the model metrics matter. In fact, where we end up using them more often than not is in ongoing monitoring of models once we’ve scaled them. So, once we’ve gotten comfortable that that system is working as designed and we’re getting the outcomes we’re looking for, those metrics, which can be calculated in a much more automated fashion, can be a way of to make sure that the model continues to perform over time and at scale. And so, they’re very useful from that perspective. But really we want to focus on business problems and clinical problems, and we want to measure those problems directly, more so than just say, oh, your model performs well, therefore we will deliver it. I’m sure we’ll talk a little bit about our nurse handoff tool, but when we developed that solution, we had multiple different metrics around the conciseness, the factuality, the helpfulness of the model that we paid very close attention to, which were much closer to the problem in the bedside than just sort of the F1 score, if you will. 

Chip Kahn: Our focus here is AI, but in a sense we miss it, I think, because it isn’t, AI, it’s the data, stupid. And, you called data a strategic asset. What does that mean in a practical sense for your system? 

Dr. Michael Schlosser: You mentioned the 47 million patient encounters a year that we have the privilege of caring for. All those encounters create data, and there are incredible patterns inside that data that we can harvest the value from. So, I’ll throw another number out. We do over 210,000 deliveries in our hospitals every year. That’s in the ballpark of how many occur on the continent of Australia. So, we have these massive populations and data describing everything about them. You know, the clinical aspects of the care, but also operations, supply chain, you know, what products were used, you name it. And so that is a strategic asset. And in fact, it’s where the value and the intelligence comes from. The AI models are kind of like shells. And I’m not trying to take anything away from the brilliant people who build these things. They’re incredible, but they don’t become intelligent until you fill them with data. So ChatGPT is trained on the entire Internet. It’s pretty knowledgeable about everything, but it’s not an expert in anything. You give it 47 million patient encounters worth of data and it can become an expert in health care delivery. You then have to pair that with the human insight because it’ll learn the patterns. Some of those are good patterns, some are not good. And the humans have to help identify which is which. But in the end the power comes from the data. And so we treat data like an asset. We’ve invested heavily in building out a new data infrastructure, a data lake house on Google Cloud that allows us to capture all that data, organize it, harmonize it, and then manage it continuously as a product so we can make it available to AI models, to our analysts, to really everyone in the organization. and ultimately that’s where the real step forward is going to come from. 

Chip Kahn: Well, you’re a leader in the field, but you also work a lot with others. And how do you compare your notion of this strategic asset and how does that compare with academic health centers or other large systems that either you’re working with or that you are following as developments go in this area? 

Dr. Michael Schlosser: Well, and I think there’s great synergy here and there’s, there’s several academic systems that we work with directly we partner with on research. I was actually hearing about this in detail yesterday from the gentleman who leads our trauma research organization that a lot of academic centers come to us because of the density and the velocity of our data. And we can do studies at scale that that they couldn’t do on their own because while they may have incredible experts, they do, they have the most brilliant minds in the field usually working there. They’re often one hospital or a small hospital system. So they just don’t have the depth and breadth of the experience that we do. Also, our data is interesting because it comes from sort of every corner of health care delivery. We have rural hospitals, we have urban and academic centers, we have quaternary care facilities. You know, we have 61 bed hospitals that are truly community hospitals. And so we can see patients and care delivery in sort of every flavor of health care across the country. Which again makes the data set that much more interesting when you talk about training or testing models. So, I’d say we are really synergistic with the academic centers in terms of all of us trying to move health care forward. I will say we tend to come at this from a little bit of a different bend. I think a lot of academic centers see their purpose as advancing the science. Like how do we make the science better, how do we find the next standard of care? We tend to focus more on how can we deliver care in our current standard at the most reliable, the most efficient, the most effective level. I think of if everyone who entered an HCA hospital or any hospital got exactly what the best standard of care treatment was in a timely fashion every single time, how much better the health care system would be than the current state where we’re reliant on a lot of manual processes and local decision making. And they get to the right answer pretty darn frequently. But I think with partnering the humans and AI together, we could do it at a different velocity. 

Chip Kahn: When you look at new technologies, whether it’s in the AI area or in other areas, you have innovation hubs, UCF Lake Nona Hospital and TriStar Hendersonville Medical Center. There you have both clinicians and operators who are sort of in a development mode for your enterprise. How does that model work and what has it taught you? 

Dr. Michael Schlosser: Yeah, and we actually have three now. We just launched a third one which is HCA Florida Aventura, down in the South Florida market. They’re critical to our strategy. So, when we started this journey of transformation, one of the things that I recognized from my prior experiences as a hospital operator, a group CMO, was that a lot of technology was sort of hoisted upon the care teams, the operators in the hospital. And many times they found it just didn’t work, it didn’t fit into their workflow, it didn’t really understand the nuances of care delivery. I’d say the best or worst example of this is the electronic health record, which achieved the goals it set out to achieve. We created a lot of data, but it did not make the process of delivering care better, faster or more efficient. And so we decided from the very beginning like we were going to do the opposite of that. We are going to design products and solutions or co-develop or bring them into our system that our caregivers really say are solving problems for them, making their lives better, making care for patients better, making patients’ lives better. And to do that we knew we were going to have to get them into the middle of the process. And so we stood up these innovation hubs as a way to bring the technology and the innovation to the bedside. So, we were doing forward deployed engineering before it was cool. We would take our data scientists and our software engineers and other innovation leaders and we would make them go to the hospital, actually insist that all of my DT&I team wear scrubs when they go into our hospitals so that they blend in with the care team, they become part of the milieu and they just sit there in the nurses station, following doctors around in the ERs. And they learn from them, they understand their workflows, their lives, how they do their job, what they care about, what they’re scared of. And it’s not to say that we want to anchor on the current process. I mean, sometimes we need to blow up the current process, but you have to really understand where you’re starting from if you’re going to drive that kind of change. And so the innovation hubs were designed, if you will, to be a place where we could do that work. And it’s a two-way street. So we, you know, we had to get the engineers and the teams to want to go in and spend substantial time in the hospitals. We also had to train the folks in the hospitals to be part of an innovation process. They were not used to this. They’re used to a product being done when we turn it on in a hospital, because of the nature of that environment, it’s high risk. And so the fact that we were going to bring them products that not only weren’t done, but they might just be a prototype on an iPhone was a totally new experience. And that they were going to talk to data scientists who didn’t know anything about health care. These were all new experiences that they were going to go through multiple iterations of change management. So we spent a lot of time upskilling the people, the leaders and the care teams that work in these hospitals to really be an innovation hub. And it’s been an incredible return on investment. We get so much amazing feedback from them. It changes the trajectory of the work that we do every time we go. I wouldn’t do it any other way. And we have a very robust discovery and design and process engineering team that goes in and understands problems at a very deep level. Even if we’re just buying a solution, we want to understand the problem in a really detailed way before we ever start trying to bring a solution in. 

Chip Kahn: Let’s take a little deep dive then on some of the solutions that you’ve actually put in place and deployed at scale. You’ve got ambient clinical documentation, you got Timpani staffing and nurse handoff. Can we take each of those, and see what they mean when they leave these innovation centers and go out into the real world across the entire system? 

Dr. Michael Schlosser: And three very different problems and very different solutions. So, I think some good illustrative examples here. I want to start with ambient clinical documentation. So this is one that, I would say most people are probably familiar with or getting familiar with, using artificial intelligence and large language models to listen to a conversation between a doctor and a patient or any provider and a patient, understand what’s being talked about and then structure that data into a draft note that can go into electronic health record. And then the doctor is the human in the loop. They review that note and ensure it’s all factual and accurate, make edits, and then they can sign it. The idea being that we would create better documentation, that the AI would be more thorough in that it would capture everything that was talked about, that it would be faster for the doctors. They would spend much less time editing notes or writing notes, and so they could spend more time focused on their patients and on critical thinking and care delivery. And that also the medical record would be more timely, that we would have complete notes, complete H&P in the chart. Much faster for the rest of the care team, but also for payers and authorization that are dependent on timely medical record documentation. And so a number of different value streams there that we saw ambient clinical documentation could provide. We partnered with a company called Commure. And full disclosure, we have a small investment in Commure as well. And we’ve actually been working with them now for over a couple of years refining this approach. And so rather than buying a solution off the shelf, so, you know, going back to this, care teams being in the middle of the product, we partnered with someone who would do the forward deployed engineering model and they would bring their product, but also bring their people and learn from our physicians. In this case, for a few reasons, we actually use some hospitals in Dallas as our guinea pigs. But we had some great physicians, some hospitalists and ER doctors there, who spent a tremendous amount of time with Commure continuously refining their product. And, the CEO of Commure told me he thinks they’re on version 300 of the product and the model, which is why I think we’re getting the performance we get. And so we’re now rolling this out. We’re live in, I think, 67 hospitals as of today. We’ll hit somewhere in the neighborhood of 105 by the end of 2026. We’re doing about 200,000 notes a month on the ambient documentation platform. A lot of H&Ps, a lot of progress notes, a lot of ER provider notes. We’re about to turn on automated discharge summary, broadly here shortly. We’ve got cardiology coming up quickly. So very rapidly bringing this across our entire platform. And we’ve seen tremendous results. Our doctors are averaging between an hour and an hour and a half of time saved every 12-hour shift. We’re getting great feedback, from our patients, including in our patient experience surveys that the doctors are spending more time at the bedside and they’re more communicative with them because they have to verbalize more in the room so the AI can hear them. So the patients are benefiting. And we’ve seen a pretty significant improvement in how quickly the notes are getting signed. That’s a metric that we track. And so I think it went from something like 67% within 24 hours to almost 90% within 24 hours. So the medical record’s getting completed faster, which helps with care delivery downstream. But the best part of this entire rollout has been the quotes that I get emailed to me all the time from the physicians. I’m constantly getting these nuggets of doctors saying, this has changed my life. I go home, I don’t have to do notes, I can spend time with my family. Like, this is one of the first times that a technology has truly made my life as a physician better. So that’s one of the best parts of my day, is when I get to read those quotes that we’re achieving. One of the missions I set out to achieve when we started this, which was to reduce the administrative burden in health care for providers. So, it’s been a tremendous success, but it came at the end of a lot of hard work. We really worked with our doctors to refine this platform and this model and the application, to get it to the point where it’s being adopted at this rate. I’ll throw out one more number. We have an 81% adoption rate in our emergency rooms amongst our ER doctors and other ER providers, which I’m fairly confident is the highest number I’ve seen reported anywhere. A lot of folks are reporting something more like 25 to 50% adoption of ambient in various care settings. Most of those are outpatient clinics. So, it really has made a tremendous impact. 

I’ll pivot and talk a little bit about nurse handoff. This was a problem that our nurses brought to us and basically told us they’ve been working on this for a long time with the EHR companies. Lots of different solutions and really had never been able to crack this. And handoffs, as you know, are just a critical part of care delivery. When the responsibility for a patient is transferred from one care team to the next, there’s a lot of risk involved. We do 60,000 handoffs a day, 24 million of them a year in our acute care setting. And so, you’ve got to get that right. Aand right now, it really falls to the nurses. They do this on their own. They scrape data from the EHR, they write handwritten notes, they communicate verbally to the oncoming shift. and they’re working very hard, doing the best they can. But it’s a process that absolutely can be improved. And so we saw this as a great use case for a large language model. We thought we can teach an LLM to read the chart and figure out how to think like a nurse. And actually, the Google research team was really excited about this. And so they partnered with us to help us build out the system that is around the Gemini model to make it good at this. And it took some time. We worked with nurses. We would have them do mock handoffs, real patient data deidentified, and we would have them read the handoff note, tell us everything that was wrong with it. We’d feed that data back to the engineers and we iterated like that dozens and dozens of times with hundreds and hundreds of examples until we got to the point where the model was really performing at a high level. So that’s the pilot. Then we put it in the real world, tried it out in eight different hospitals and got a whole other set of feedback. I think we got 7,000 comments, about how they wanted the data organized differently. They wanted to view it the way they were used to viewing data. And so we went back and we iterated again. And now we’re at a beta pilot in 12 hospitals and getting just tremendous feedback from our nurses that it’s making their job easier. They feel more confident in taking care of patients. And we’re tracking our safety events. We have a safety event reporting system and we’ve seen an 80% reduction in handoff-related safety event reporting. Some of those are near misses, a lot of them, but nonetheless we’re seeing it in the data that this is making patient care safer and that brings confidence to our nurses. You know, nurses are always concerned about the patients and their safety. So it’s been a real home run. We’re in the process of hardening the data product that sits underneath it. Back to your previous comment about it’s all about the data. We’ve got to make sure that that data product is really ready for primetime if it’s going to be used 24 million times a year reliably. So a little work to do there and then we’ll start rolling out beyond those 12 hospitals probably in the middle of this summer, on a pretty aggressive strategy. That’s, I think, a really great example of how we’ve worked at the bedside built by nurses for nurses, and that is a sense of pride for them as well. 

The last one you mentioned is Timpani. And this has been a really interesting journey. It’s maybe a little bit of a cautionary tale also. This was one of the first products we actually started working on. And we did that for a very specific reason. We knew that if we didn’t have the right care team on every unit every day, if we couldn’t deploy the right people, right place, right time, right set of skills, it would be hard to innovate on top of that platform. If we had uneven staffing, that’s not a strong platform to stand on and try to drive even more innovation. And so we said, we’ve got to make sure that we’re deploying our clinical staff uniformly and also doing it with the right set of skills, the right predictive algorithms to know how busy the units are going to be. And so we built all that. And it’s a great piece of software. It’s a really sophisticated application and it’s programmed, I think, with really sort of altruistic goals, to take whoever the team is you have on this unit in this hospital, know who they are, their skills, their preferences, how they like to work, when they like to work, and deploy them on a schedule and maintain that schedule such that we have the best likelihood of providing the right team for the patients who are going to show up. We have great machine learning algorithms that predict how busy the units are going to be. And we rolled it out to 130 hospitals at this point. 

The cautionary tale piece of this is around change management. And honestly, we haven’t talked about it yet, but change management is probably the hardest part of the job that I do on any given day, because, as I mentioned, there’s always people involved here. We’re not replacing people with AI, we’re augmenting them, which means the people have to change. And that’s hard. It’s definitely hard at scale. And so what we didn’t recognize was all of the nuances that went into building a schedule, the personal relationships between the nurse leaders and their team and how scheduling and staffing was part of that personal relationship, just all of the things coming out of the pandemic and since then that they’d done to try to maintain that staff, you know, there’s a workforce crisis. And while the system had, I think, really good rules built into it, it didn’t quite understand the nuance at that level. And so, we had trouble getting people to really accept the output of the model and they would go in and edit the schedules to look more like what they would have built on their own. Which means we were losing some of the time savings it was supposed to create. We were losing some of the value in spreading the team out more evenly. So, we’re still learning. That’s one of the things that I would say is the most interesting about AI, is that it’s always about learning, the AI learning and us learning. And so we still think that this is going to be a good solution and it’s going to solve a big problem for us in the long term. But it turned out to be a little more complex than we knew what we were getting into. 

Chip Kahn: So it really sounds like you just can’t take it off the shelf when you bring these solutions on. It’s a process. It’s not just a ready-made item to implement. What’s next? What’s sort of in your stack that you’re working with your clinicians and the other staff or the members of the team. Is there anything you can talk about that’s sort of in the wings? 

Dr. Michael Schlosser: We’re doing a lot of work in supply chain, and so this is all about making sure that we have the right products in the right place at the right time. We run a lot of our own supply chain and distribution centers. But a lot of it’s manual, a lot of it’s driven by people and their experience. And so we’re bringing intelligence to those systems. So that’s exciting to me because there’s a good financial ROI attached to the supply chain use cases. And so if I’m going to continue to get funding from the organization, we have to focus on things that also improve the finances of HCA in addition to the care experience and the clinical outcomes and the patient safety. Because AI is expensive, as you know. Another one that I’m really excited about is our partnership with OpenAI. We’re just a handful of months into this strategic partnership. but they’ve already brought their engineers to work with our teams on a few really high impact use cases, one of which we refer to as our moonshot, where we’re working with them to build a team of agents that can do the orchestration of care delivery in a hospital. So, you know, if you got 100 patients in a hospital, and they each have dozens of things that they need to have done on any given day. Meds, delivered procedures, scans, therapy. The way that gets done in a hospital right now is sort of by the people like the doctors, the nurses, the therapists. They all sort of just figure it out. Like, they have their work lists and they just get through it. But if we had a team of agents that knew everything that was going on and could behind the scenes, organizing, and orchestrating what’s the most efficient and effective way to get things done. What’s the next priority that has to happen? Who needs to go to MRI right now? Like, you know, just all of those individual details could be organized by agents. And then because they’re agents, they can also take action. They can message PT and say, hey, divert from where you’re headed, go up to the sixth floor and see this patient because they’re waiting on discharge. You could actually make a hospital run way more effectively. The patients would have a better experience. They might even know, like, when their MRI scan is going to occur next. We could get them through their plan of care faster, which means we have more capacity to care for our communities. There’s a tremendous upside that we could create here, clinically, operationally, financially. It’s a moonshot because this is a highly complex use case. Incredible amount of data that we have to organize a team of agents that we have to get to work with each other and then people on either side of those. So, it’s a heavy lift, but so far so good. We’re making incremental progress. I think we’ll have something that we can pilot later this year. So, we’re really excited about that sort of operating system of a hospital of the future, if you will. 

Chip Kahn: You’re sort of getting at it now. And I’m going to ask a conceptual question of you. We had Bob Wachter on. He asked whether AI is really changing what it means to be a clinician, and he wrote a whole book about that. And from your view, I mean, you’ve got 44,000 either affiliated or employed physicians across this gigantic system. What is your perception of the gains, risks, or both for the quarterback? I mean, historically and traditionally and in terms of the practice of medicine, those physicians are the quarterback of the care. Agentic AI really coming online at some point begins to either change that or provide some kind of new factor that hadn’t been in the physician’s workflow before. 

Dr. Michael Schlosser: Yeah, my short answer to your initial question of is it going to change what it means to be a physician is, I really hope so, because I see this as an incredibly positive change if we want it to be. And I already talked about ambient documentation and the positive change that’s happening. I see now that this is making our physicians’ lives better. And I think when the workforce is happy, the patients are happier. So there’s a lot of downstream positive impact we could have. I think the more we can use AI to make the health care system run better, the more they can be that quarterback. They can spend their time thinking critically, studying, reading articles, talking to their patients, engaging in research and other scientific endeavors. A lot of the things that we became doctors to do and maybe not have to do any of the things that we don’t have to do. I remember back to my days practicing, getting on a phone call with an insurance company explaining to them why my patient needed six more weeks of physical therapy. Like, that’s not what I went to school and to become a neurosurgeon to do. So, I think there’s a first wave here where we can massively clean up a lot of the complexity and the burden we’ve injected into our health care system, which I’m excited about and we’re focused on. And then I think once we build trust with doctors and clinicians, once they come to see AI as a tool and a partner and maybe less of a threat, then I think it opens up for a lot of other possibilities. I do think in the future we will have AI and humans working together to make most of the important critical decisions in our health care industry. That when you go to make a decision for a patient, a treatment or diagnostic decision or whatever it might be, that you can tap into your own knowledge and experience, but then you can also tap into AI and the data and the patterns. And as a second opinion, like a real time, always on second opinion, I want the doctors to stay in control. I don’t think a health care system where AI makes the choices is one that I want to help develop, at least not at this point. I don’t think the AI models are ready for that, to be honest, but I think they’re very capable of being a copilot. That health care system that I just described, where you don’t spend time on administrative burden, you get to spend time thinking and maybe enjoying your life a little bit, and then you have AI helping you do your job better, sounds like one that people would want to be a part of. 

Chip Kahn: You know, this podcast is built on the premise that patient care and ultimately patient outcomes really depend on the business model. And in some ways our conversation has covered that. So when AI improves clinical quality or promotes operational efficiency for HCA Healthcare, that’s a good. But where’s the line between that and what makes the world go round, which is volume and payment frankly? 

Dr. Michael Schlosser: You’re spot on. It has to do both. And so we think of our AI investments and you and I are saying AI, it’s really digital AI data, it’s the whole technology ecosystem. But we think of our investments in digital transformation as a portfolio. And really it sort of functions like an investment portfolio with, you know, your mutual funds and individual stocks or what have you. And so we have some high risk bets, some lower risk, but much more likely to perform. We have some things that are not going to return a financial return, but they’re really important and we want to do them, we want to support them. And then we have some that we really expect significant financial ROI from like supply chain and revenue cycle. And the idea is that we spread our investments across that entire portfolio and then we manage it as a portfolio and the whole thing collectively has to perform. So it’s got to be ROI positive, right? We’ve got to have a financial return, but every individual use case doesn’t have to. So nurse handoff doesn’t have a big financial ROI. I think there’ll be a lot of downstream benefits. Nurses are our most important workforce that we hire directly. We don’t employ a lot of our physicians. And so if, you know, if we shore up that workforce and people want to work for us, maybe down the line there’s a financial impact, but in the immediate term it’s about patient safety. But then to offset that, I’ve got other use cases that are much more focused on driving an immediate financial ROI. We call this value tracking. But the discipline of value tracking I think is intrinsic to a digital transformation agenda. And it’s not a side gig for us. Like we have people who focus all of their energy on tracking the value of each individual use case, whatever that value might be. It comes in all kinds of different forms, but it’s got to be measurable at scale, not just in a pilot. Like we’ve got to be able to see the value occurring, either the money or the safety or whatever it is, numbers, progressing as we roll it out. Otherwise, we don’t keep going. 

Chip Kahn: I guess that you must see every day really miraculous solutions and you’re sort of hitting at it, I think, with the end of that last question. So what are the characteristics or criteria that make you decide not to deploy? To say, wow, that really must be a wonderful thing. But not here. 

Dr. Michael Schlosser: The most common reason is actually because it ends up not being an interesting problem to solve and the value isn’t there. Right. So a lot of times people will pitch us an idea and they’ll say, hey, this is going to solve this amazing problem for you. And when we go in and we start looking at it, we realize, well, that problem actually isn’t really something that’s strategic for us, or it’s not top of our list, or, yes, you can make that better, but we don’t really see that that’s going to create the return for us. It’s rarely that the technology doesn’t work. I mean, most of the times the technology is solid. It’s because it doesn’t sort of fit, it won’t fit in our workflow, or it doesn’t integrate well with our systems, or we don’t see people adopting because it’s creating a new app that they have to go to that’s outside of their normal workflow and that just creates burden instead of solving for it. So it’s usually at that level, at that sort of technology, human interface level, that things fail more so than the technology. So the way we avoid spending too much time going down those rabbit trails is that discovery process. We make sure we really understand the problem we’re solving at a very detailed level. And where does the problem originate from and what does the current workflow look like? If you’ve got a manual paper process, if people are already working around a piece of technology with a paper process, enhancing that technology is not going to do anything. So you’ve got to understand sort of the people and how they’re doing their jobs if we’re going to pick the right solutions. I would say the other way is that it doesn’t scale. And so there are a lot of processes or products, technologies that you can kind of force to work in a pilot. You can sort of stand up everything you need to make it work. You can get the people on board, you can sort of get the data pipes doing what you need them to do. Then you see a result in a pilot. But it just, there’s no way to take that to 189 hospitals. It becomes prohibitively expensive. People don’t adopt it because you had this really controlled environment where you generated all this excitement around it. But at beta or at scale, the excitement’s not there. And people decide it’s really not solving the problem. For me, that’s sort of the second most common way. And so we see those through the data.That’s where the value tracking and the adoption tracking comes in. And so hopefully we catch those quickly and we pivot. 

Chip Kahn: You know, no matter what happens in the systems that affect clinical care directly, humans are going to be involved. So as we get into more clinical decision support and other kinds of, I’ll call them bells and whistles, but alerts, we see in hospitals and other areas, you know, something called alert fatigue. Actually it predates AI and the new solutions, but it’s going to become a real issue in terms of the human role. Across your system as you adopt all kinds of new solutions and particularly, eventually you will get heavily into clinical decision support because it’s coming, how do you deal with the human factor, which is really what alert fatigue is all about? 

Dr. Michael Schlosser: The simple answer is you’ve got to directly manage it. So we are very sensitive to alert fatigue. I talk with my team all the time that we really can’t deploy any solutions that add alerts right now. Like, if you were to go talk to our nurses in our hospitals, they’ll tell you they’re already past the saturation point. All of our nurses have what we call imobile. They all have iPhones that we provided them and then software on those iPhones that help them communicate with their colleagues and others. And so they have a platform where we can deliver alerts to them. And we deliver a lot of alerts. And they’ll tell us like, you know, it’s work to sort through all that and find the signal from the noise. And so we have to actively manage that. And so I would say we have as much or more work going on figuring out how to reduce the number of alerts going to those phones as we do people trying to design like, new AI solutions that could potentially throw off new alerts. The nursing team did a big project around bed alarms. Like, how do we clean all that up so that we’re not overloading them with bed alarms all the time? So, you have to actively manage it. You have to have a discipline around it. Now the future state. I have a really innovative design team that has this concept that they refer to as nugentic. It’s a combination of nudge and agentic. When they talk about nugentic, I sort of, I almost think of like the old-style telephone, systems where there was a person plugging in, like the, you know, I’m talking about plugging in, like the connections. Could you have an agent that’s sitting in the middle of all of these communication streams and figuring out what does Dr. Schlosser actually want to know right now? Like which of all of these signals that are coming in that are screaming for attention are the ones that really matter? And how does he like to receive that information? Does he like to get a text message, a phone call, an email, an overhead page? Smoke signals? And the agent could be figuring out how to parse out the information in a way that it’s not overloading, that we’re not missing critical things where patients are going to get hurt if we miss them. But we’re also filtering the noise as much as we possibly can. So that’s a future state, that’s a long-term solution. But we’re starting to build that technology because this is a problem that’s going to have to be solved because the alert fatigue is a very, very real thing. And so, we’re putting some effort behind it. 

Chip Kahn: And I guess along those same lines, you’ve described a very comprehensive process of design and development and then integration into your system very carefully done. But when I had one of my other guests, Elad Walach from Aidoc, he mentioned something called drift. In a sense, his system’s a medical miracle. Over time, there’s some, either statistical or data or technological things that happen. And whether it’s drift or whether it’s other kinds of degrading, what are the feedback loops that you all are anticipating to try to keep an eye on all these new systems that are dynamic? I mean, they’re going to change over time and there may be unintended consequences. 

Dr. Michael Schlosser: Yeah, and sometimes it’s the patient population that changes. Right? I mean, if, God forbid, we ever had another pandemic, the patient population your model would be working on now would not be the same as the one it was trained on, because that disease didn’t exist. So there’s lots of reasons why model drift can occur. And yeah, your point is a great one. You have to have this ongoing monitoring. And so, I’ll go back to what was the first question about sort of the model scores. So we’ve set up a model registry with model cards. Lots of folks have done that, but we also have invested, and this is early days, but in a technology where we can bring that kind of data in in real time. I mean, it wouldn’t be like literally minute by minute, but at whatever the right cadence is, we could ingest those kinds of metrics back into the model cards and have an AI system that’s watching that so that we can see, okay, has the performance changed, like month over month? Is the accuracy of his model changing, and if so, what do we need to do about that? How do we flag that? With the large language models, we do have to rely on the people. And so because you can’t measure an F1 score, you can’t measure a model performance the way you can with like a machine learning model, like his systems use when you’re talking about a big foundation model. So we built into the nurse handoff tool, like UI that makes it really easy for them to tell us if they see something wrong. If they see a hallucination or an error or an omission that they think is material, they basically just press a button on the interface and that data goes back to us. And we capture that. In our ambient clinical documentation, we worked with Commure to build multiple layers of safety nets. So we have a hallucination checker, which is an LLM that’s really good at seeing the hallucinations from other LLMs. Then we have a subset of notes that are just hand reviewed by people on a regular basis. And then we archive de-identified data so that we can use it as a QA testing environment in the future. And so it depends on the type of solution, what kind of, program you have to put into place. But this concept of having sort of ongoing machine learning operations, LLM ops, ongoing monitoring, I think is part of the challenge of scale and part of why going from pilot to scale is actually really hard. 

Chip Kahn: What I understand here is first there’s choices about even looking at solutions. Then there’s development and design and bringing the staff in. The evolutionary process that you go through with your staff, with the solution producers, and then there’s implementation and scaling, and then there’s the feedback loop. HCA is a very large organization that can do these kinds of things. What’s your view about how smaller systems, because if we look across the country, there are a few, but not that many systems at your scale, that have the management structure and the development structure. You know, what’s your observation about how the solutions that you’re talking about that are so necessary for the future of health care to ensure the quality, to bring the cost down. What’s your observation about what smaller systems should do in this environment when these solutions are coming out every day? 

Dr. Michael Schlosser: It’s gonna be a challenge. I would agree. I think that we are sort of the tip of the spear right now. And I would hope that they will benefit from our learnings that as we build these solutions, as we work with partners like Commure, we’re working with Google, with GE, with OpenAI, that we’re helping sort of the whole ecosystem of health care technology get better at this. And therefore the products will be more complete products. They won’t just be an algorithm or, you know, an AI dictation system, but they’ll come with the kinds of belts and suspenders, safety systems that you need, because that’s going to be necessary for these smaller health systems to be able to adopt that technology. The other thing that I think about, sometimes, and I’ve talked with some folks in Washington D.C. about this, is should we be thinking about the next version of what meaningful use was, what the HITECH bill did? Is this important enough that there should be some ways that government supports funding this kind of technology to get down to sort of every nook and cranny of health care? And we don’t need that. Like we, as you said, we’re big enough and successful enough that we can afford this, but we are definitely the outlier. The vast majority of systems would struggle to make these kinds of investments. And so, if in the next few years we really see health care changing for the better because of AI and technology, should there be some way of trying to ensure that all patients and all systems have a way to access that kind of technology? That might end up being the case. We’ll see. But in the meantime, I’m hoping what we learn can benefit all healthcare systems in the future. 

Chip Kahn: So you’re actually bringing up one, but let me ask, are there other changes, one or two in payment regulation, what AI and developers do, that you think you’d like to see, to accelerate the dissemination of all these really medical miracles that AI and health care can produce? 

Dr. Michael Schlosser: I’ll give you two. One is something that we’ve been pretty consistent about in our talking about this, and that is I think we would benefit from some common sense federal regulations or standards, just some guardrails, around what good looks like for deploying AI across health care systems. If for no other reason then it would help us not have to deal with 50 different sets of rules that every state will feel like they need to enact in order to protect patients or whatever the reasons are. I’m not trying to wade into a political space here. I can just leave it at that. But I think that would actually help drive innovation if we had some agreed-upon guardrails. The second thing I would say is that I wish the conversation and reporting around AI was a little more transparent and a little less hype, cycle driven and that this is never going to happen because it’s just the way the media works. But the constant sort of frenzy around what AI is capable of or could do versus what it actually takes to deploy agentic AI in a health system, there’s an enormous gap between those two things. And so every day there’s some new article or something about what these AI agents are capable of and the dramatic change it’s going to create and all these things. And it completely ignores the fact that you have to do all those steps that you really clearly outlined here just a minute ago if you’re going to put AI in a heavily regulated, high-risk environment like health care delivery. And so I’m incredibly bullish about AI. I’m a huge tech optimist. I think it’s going to create enormous change. It is not going to come overnight, it’s not going to be easy, but we’re going to do it because I think it’s worth doing. I wish that gap was a little more narrow so that people could work on reality of AI rather than fantasy. 

Chip Kahn: Mike to close out, we’ve really gone over the whole waterfront, I think, in terms of implementation of AI at HCA healthcare. What keeps you up at night, in this AI area, either in terms of your environment, your ecosystem or even larger scale. 

Dr. Michael Schlosser: I’m going to cheat and give you two here also. So the number one has to be change management because we have to adopt these changes and all the people who make up health care have to become open to change and open to innovation if this is going to work. And that’s a challenge. And so that worries me. The second thing that worries me is that I don’t really know that we fully understand AI even yet at this point. I’ll feel schizophrenic where sometimes I feel like, hey, these models make mistakes all the time and are they really going to be any good? And then I’ll read something where I’m like, oh my gosh, humanity is already in trouble because the AI is smarter than us. And the reality is somewhere in between. But that keeps me up at night. I don’t know that we fully understand the technology that we’re using and it might surprise us one day. So, we’ll see. 

Chip Kahn: Well, hopefully we can avoid the downside and enjoy all the value from everything you’ve described here today. Mike, this is terrific and I just so appreciate you coming on the podcast. 

Dr. Michael Schlosser: Chip, thanks for having me. Enjoyed the conversation.