Who Controls Medical AI and What Do They Want?
Dr. Eric Topol, a cardiologist, bestselling author and founder and director of the Scripps Research Translational Institute, talks about why proven medical AI is still not used in everyday care and who will control it. He argues that AI's biggest contribution should be preventing disease rather than curing it, and warns that hospital administrators focused on revenue could waste the chance to restore the patient-doctor relationship. The hosts then debate whether prevention can reach the people who need it most.
Transcript
The conversation in this video is for information purposes only and does not constitute medical advice. Always consult a licensed healthcare professional before making any health decisions.
There are nearly 1,000 FDA-approved AI systems in medicine, but almost none are used in everyday clinical care. At the same time though, 40 million people a day are using ChatGPT for health information with no regulation, no validation, and no one accountable when it gets it wrong. ECRI, the Emergency Care Research Institute, an independent patient safety organization, has just named AI chatbot misuse as the number one health technology hazard of 2026.
Dr. Eric Topol believes AI can predict disease decades before symptoms appear, and he's argued that AI without doctors may outperform doctors with AI. But someone has to set the rules. And right now, the people most likely to write them aren't doctors or patients, but rather insurers, tech companies, and hospital administrators.
This week on Agents of Tech, prevention, power, and who really controls medical AI, with cardiologist and bestselling author Dr. Eric Topol.
All right, Stephen, if we have the capabilities now to predict these diseases possibly decades before symptoms appear, why are we not using it?
You know, we've had lots of studies recently that show, you know, really how much higher detection rates of particular cancers. And there's a big Swedish study that I think are 29% more detection of cancer with AI than just the physicians on their own. And that comes to the back of big German study, big Hungarian study. And so I think, as Eric called it in one of his interviews, it's institutional inertia. You know, maybe that we just have not, you know, that we know this works, but we're waiting for another set of results. We're waiting for more proof. We're waiting for a different system. So I personally think we should be using it, but I think it's about how systems work.
Just slow to adopt?
Yeah, I think it's just really hard to implement things and ensure that they are kind of seeping through these systems, which are often bureaucratic. They don't operate in accordance to, like, what's most efficient, what's most effective, as in who they serve is actually really complicated. It's not usually the patients. Like, that's not the thing that's first and foremost. It's, commercially, are they profitable according to the accountants? Like, the accountants have a very big role in these organizations. The hospital administrators, it'll be interesting to hear how he thinks. He just detangles this really, really big feat, basically.
Right, big problem. Alright, well, we also want to hear what you have to think. If AI can predict your health risks decades early, is that a gift or is it actually a threat? Tell us in the comments below and then don't forget to like and subscribe. Alright, let's get into it. Here now is our conversation with Dr. Eric Topol.
Dr. Topol, thank you so much for joining us today. We appreciate your time.
Sure.
My first question for you really revolves around accountability and holding AI agents accountable when it comes to medical AI.
Well, it has to be accountable just the same as if it was from a physician. The problem most people don't realize is there's lots of errors by physicians. In the US, you know, 800,000 serious diagnostic errors a year that result in disability or death. So we're trying to improve accuracy. So, yes, there has to be accountability to using AI, but there's room for improvement.
I was reading about the analysis that you'd done and others had done on the study in Sweden in terms of detecting breast cancer, and the Hungarian study and the German study. And I think you were saying something on the lines of 29%, you know, improvement when you bring AI into detecting. But that seems like a big deal to me. 29% seems like a lot, it's a lot of people. And I guess my question is how many times, when are we going to actually see this implemented across the board as opposed to just in all these isolated studies?
We have the largest randomized trial in the history of medical AI. It's remarkably compelling, positive for picking up important breast cancer and reducing subsequent, over the years, of follow up appearance of breast cancer. So it's preventive, it's early diagnostic, it's just what we want. It should be part of medical practice, and the problem is it isn't yet. So what are we waiting for, I guess, is the real question.
So what are we waiting for?
Well, the medical community is kind of sclerotic or ossified and, you know, it's in a state of disequilibrium. I'm hoping that with this type of data, which is rare, we don't have many examples like this in medicine, that we will make the shift. There are some issues though. Like, for example, there's several different algorithms that are used and only one has had this remarkable validation, the one from the Netherlands called Transpara. And we don't know for sure if other algorithms in other countries are quite the same.
So you can always come up with excuses, right? But the point is we should be pushing to get this. You know, what women undergoing mammography would not want to get the best data? And we're also talking about a normal mammogram which picks up high risk, as well as heart disease risk by the mammogram. So all these things should be what I would consider a routine part of a mammogram readout with the help of AI.
And, Dr. Topol, picking up on that and the obstacles to implementation, when we're talking about the real world and practicalities, as in what does it actually look like taking it from that study and putting it in the hands of physicians and allowing patients to benefit from that? Because there are so many factors in our healthcare system when it comes to hospitals, administrators, the people who produce the tech behind the AI. So what does that actually look like, taking it from start to finish?
Yeah, there's always these issues, like, you know, in the US, the reimbursement, who's going to pay to get access to these algorithms? There's also the physician, here, the radiologist for example, who might have automation bias, to not want to use the AI, or to, on the other hand, more heavily rely on the AI, or to be de-skilled by using the AI, you know, so extensively. So there are issues, sure, but the point is that when you have compelling data, you can't continue a state of denial. You should be able to figure out, these are all logistical issues, but we should be moving, this is a precedent. And of course, once we can get through these hurdles, hopefully other things that have really strong data sets will also take hold. But it's tricky. Implementation is not easy, particularly in medicine.
Yeah. And in terms of what would be the hurdle, if you had to identify one to the medical and healthcare community that we need to target first, that we need to tackle this right now and remove that obstacle?
I think the main thing is health systems, they have to figure out how to work with these companies.
Yeah.
You know, are they going to give them a small amount for every mammogram that's interpreted, for example, or are patients going to have to pay out of pocket? I think that is not acceptable. So we have to come up with a way where everybody has a win-win, right? And that hasn't been figured out yet.
No, I was going to say, obviously you said patients paying out of pocket and, like, you also, like, these systems are going to do great things, but you have the risk of huge inequity, where actually only the people who are able to pay for, like, the newest technology, the best treatment, are going to be the ones that receive it, as in the dynamic between these tech companies and the healthcare system is going to be complicated and people aren't going to like it.
Absolutely. Any time we make progress, it's got to be for all people. Because almost always, it's the people that can't afford these things that need and benefit the most. And so we can't have this catering to the affluenza. That just doesn't work.
Dr. Topol, I want to play devil's advocate for just a second from the patient perspective. Because if we have the AI systems that could predict disease or even symptom years before symptoms arise, but yet we don't a cure for it, is ignorance bliss at that point? I mean, do I want to know that I might be at risk for Alzheimer's if I know that there is no cure for it 30 years down the road?
Well, what you're asking about is a cure. I'm talking about prevention, okay? So for Alzheimer's, cancer, cardiovascular, we have a whole lot of tools to prevent these diseases. That's a whole lot better than a cure. We're not very good at curing, but we could be much, much better about prevention. So the point here is we want to identify who's at high risk because these three big age-related diseases incubate for at least 15 to 20 years in our bodies before they ever develop the disease. That's the time we have, this great runway, to go into high prevent mode. And there are things we can do for each of these major diseases that we're not doing. And so that's what's exciting right now, and that's where AI, taking all of the person's data, can define who's at high risk for which of these diseases and also help in surveillance and all the things that we do to go into prevent mode.
Okay, that's very encouraging. Thank you for that. You've argued that the future, I think most people believe that the future when it comes to medical AI is a combination of both doctors utilizing AI, working together, but you've actually argued that AI, at times, without doctors, may actually be better. I think a lot of the hesitancy and reluctancy from people to adopt medical AI is they always want that human in the room. They always want to have that human component. What do you say to that?
Well, that's what I thought too. That's what most of us thought. And then there were seven studies that came out that showed the AI was better than the doctor with the AI for accuracy. And that caused us to pause and think, well, why is that? And it could be 'cause we're very early in this and these were not real world studies. They were simulated with, you know, things like patient actors and, you know, case studies. It could be because doctors are not grounded in how to use AI or it could be they have automation bias against AI. But the point is that there may be instances where the AI does a better job than when the doctors cancel out the AI utility. So we got to keep an eye on that. You got to keep an open mind.
Can I take you back to your point about prevention of some of the age-related diseases? I'm kind of curious to know how the data collection is going to work. I can understand how, you know, the AI can look for patterns within the data, but if somebody is not ill, how is the data going to be collected?
Yeah, well, that's what's so exciting right now is that, for example, a very large study done in 400,000 people from the UK Biobank and 1.9 million people from Denmark showed that just the electronic health record and the lifestyle factors predicted 1,200 diseases, almost every disease we know, not only predicted the risk, but when, when it would show up. So that was just two layers of data, right? Now we have proteins, biomarkers, like the breakthrough one for Alzheimer's disease, p-tau217, that tells us in advance 15 or 20 years about people who are destined to have a high risk.
We also have genes, of course, that we're not using. I mean, the UK is the genomic capital of the world, but still we could do much better, all around the world, to use our genomic data, like polygenic risk scores and sequence. So there are all these layers of data that we're not using that no human could integrate all that, process it. But that's perfect for multimodal AI. So, in the future, if a person wants to know their risk, and of course we have to dispel this notion that knowing a risk means they're going to get the disease and that there's nothing you can do about it.
All right, we're going to keep the conversation about data going coming up on the other side of the break because we're going to ask the question, who should control medical AI? Who is in control of all the data, physicians or should it be executives? Dr. Eric Topol will weigh in on his biggest fear when it comes to who ends up in control. Coming up.
Dr. Topol, you have written that your biggest fear is that hospital administrators will end up being the ones to use AI to squeeze clinicians rather than give time back to the patients. In fact, if we think about what electronic health records were supposed to do, they were supposed to help doctors, but instead we really saw an epidemic of burnout. What makes you confident that the inclusion of AI and medical AI won't end up being the electronic health records 2.0?
Yeah, well, I'm not. I'm worried about this. The reason you cited, which is that hospital administrators have, by and large, only one objective, is to increase revenue and to use AI to maximize productivity. But on the other hand, what we can do is give the gift of time back to the patient-doctor relationship. Because right now, it's a de minimis of time and lack of presence, which engenders lack of trust and lack of empathy. So in order to get that relationship back, we have to make the gift of time the number one priority, which is the opposite of what administrators want and will do.
So we have to work against this potential because it is a default mode. Make more money. Medicine as a business rather than caring for patients. And so this is something that is my biggest concern about the AI era in medicine, is we have this great chance to restore a remarkable patient-doctor relationship. We may never see it again for a long, long time, if ever. And we could blow it because of business-centric issues.
It's so unfortunate that the global name of the game at the moment in AI, medicine, and everything is kind of make more money, basically. And I wonder if you have any examples of someone, or a country, or an organization in healthcare that is actually doing it right, that is implementing new technology, including electronic health records or AI correctly, and is going down the right path.
You know, one is the use of these ambient conversations to not only create the note, which is a synthetic note better than the notes that doctors make and without taking all that data clerk function or time. And then using that automated note to interact with the patient subsequently and do all the things that are needed, like follow up appointments and, you know, tests, and labs, and all these things that are keyboard time. So that's getting very big in some countries. And I think that's good because it increases the time that doctors have with patients and their own lives. It's increasing morale. It's good.
There are lots of examples. Like in China, for example, they've already implemented using the abdominal and chest CT to pick up pancreatic cancer from an algorithm that they validated and published. So they're using what we call opportunistic AI, finding things that were not the reason why these scans were done, that doctors miss. And that's a big deal because, as you know, pancreatic cancer is deadly, it's picked up too late.
Can I just take you back just one second to the Super Agers. I'm really fascinated by what you have to say and I'm really interested in this concept, as you say, about the 15 years, you know, of unproductive, should we call it life, because of ill health, that something can be done about. Coming back to your point about kind of how all those advancements need to be for everybody, surely this is a problem where we have well educated people who have resources are going to be the people who are going to understand how to plug into this. They're going to go to physicians regularly, are going to have different sources of data. Well, the people who really need this arm, they may not have health insurance at all. So how are we going to give those 15 years back to those people who really need it?
What we've seen with AI is when there's intent to reach people who are the least likely, like a mental health app that was done in the UK specifically, what we're seeing in many low and middle income countries around the world when AI is being purposefully implemented to help these individuals, it can be done, but it's not going to happen by accident. It takes work. I mean, just think about the smartphone era and broadband connectivity. It would've been wise for us to give smartphones and data plans to people who couldn't afford them to avoid emergency room visits, which are very costly, no less hospitalizations. But we've never done that.
I mean, we can use software which is very cheap theoretically, right? And even, you know, these smartphones that don't have all the bells and whistles but are very inexpensive, we can use these things to empower people who otherwise don't have access to technology. And there are some remarkable things that are happening right now whereby people with no training are able to do ultrasound of any part of the body, except the brain, just by AI guidance. So you could actually acquire the images by AI guidance, as long as you can put the transducer on the body, and the AI will tell you move it up, or down, or clockwise, counterclockwise, and get you the report. So we're seeing some empowerment that, you know, a democratization of technology, in places you would never expect it would happen. It's actually quite formidable.
That's always the hope that we've had when technology has, or new technology has entered any industry. But what we've unfortunately found is that often the technology companies which start with this mission of kind of democratization end up being the ones that dictate those industries. And that's unfortunately happened with social media. You're seeing tech companies come into the military space, which strikes fear into people. So is healthcare going to be the final one? And how do we set up frameworks? How do we have intervention to ensure that doesn't end up being the case, where a tech company can just cut off someone who doesn't have the resource without AI to do ultrasounds in the middle of Africa because they decided that they don't want that person to have access to their technology anymore.
I think it takes deliberate intent. You know, I think that what we're talking about here, this is really something that doesn't have to be very expensive. It's just the companies that are involved. How willing are they to make their access and tools affordable and accessible? That's even more important to everyone. If you asked about worries, that would be pretty high on my list. Not as high as the administrators, but it's up there. I mean, as you say, there's many examples where it's gone the wrong way. It's going to be deliberate hard work. But we got to do it because, as I said, the people that need and benefit the most are going to be the least likely to access.
Dr. Topol, you are the expert on aging well. You literally wrote the book on it. So I would be interested to know if you could pinpoint perhaps one disease or one condition that you think AI could have the biggest impact on of us avoiding, what would that be?
I think all three: cardiovascular, it's heart disease, Alzheimer's, and cancer are eminently preventable now. The reason why we're starting with Alzheimer's, and we'll start with a randomized trial for prevention, words we never used before, prevention of Alzheimer's. We're starting a trial next month in 1,200 people who are very high risk. They have APOE4 allele, they have family history, they have polygenic risk score, and they have this high p-tau217 marker that tells us way in advance that they are very high risk. And we're going to randomize them to intensive lifestyle improvement, that's with sleep coaching, physical activity coaching, diet coaching, continuous coaching, versus just some educational materials, and see whether this aggressive lifestyle can lower, as we've seen with small anecdotes, lower their risk for p-tau217, bring it down to normal levels, and also the brain clock.
This is a really big striking advance. We have organ clocks now that tell us the pace of aging of every organ in our body, including our immune system. So if we can slow the pace of brain aging with lifestyle improvement, no less drugs in the future, that's a big plus. So I think Alzheimer's is going to be a preventable disease, and there are lots of really great drug candidates to add to the lifestyle. And so it's first defining the people at high risk and then getting all over it so that you change the natural history of a person developing mild cognitive impairment.
That's very encouraging. Okay, we have a couple of rapid fire questions that we pose to all of our guests to get varying perspectives. My first one is, where do you think people will draw the line on allowing AI into their personal lives?
That's a good one because there's a big backlash of AI right now. You know, the public is against AI. And it's seen all sorts of things from, you know, suicides among, you know, children and teenagers. So I understand that. I'm in touch with that. But I think what we're going to see eventually is we won't even talk about AI in medicine. It'll be embedded. It'll be part of everything we do, right? It'll be, you know, almost invisible. But we're kind of ways to go.
I do think, like Demis Hassabis and, you know, almost all the leaders, Geoffrey Hinton, this will be the biggest positive impact of AI. It'll be in healthcare and medicine, and in my view, especially in prevention. So I think, yes, it's natural to be worried about AI and there's lots of issues we haven't even touched on, like privacy, security of data, and we did touch on inequities, but, you know, all sorts of other things that are in the way. Eventually we'll get there. It'll take longer than it should, but we will get there.
And my question is, in your specific field, what is something that is universally accepted that you actually disagree with?
What I disagree with is a lot of the people in the genomics field say that we shouldn't be using polygenic risk scores, which is, I think, ludicrous. Because they are very inexpensive. They tell us about all the common cancers, about heart disease and Alzheimer's. They help guide therapy and prevention. So that one is easy for me to say.
Mine to finish off with is, what can you see coming down the pike that nobody's talking about now?
Well, I think that no one's really talking about this prevention opportunity. I'm kind of the lone wolf out there. But I think they're going to come along when they start to see, you know, the outputs of being able to prevent diseases. That will be, in my view, the singular biggest contribution. But it'll take years to fully, like the mammography study of a hundred thousand women, randomized, it's going to take those kind of big studies to show we prevented this disease or this other dreaded disease. But I do think, you know, we're so focused on treatment, treatment, treatment. And we got to make this pivot because we can do it now and we couldn't have done it previously.
Well, thank you so much. We appreciate you speaking with us today and for all your insights. Thank you.
All right, I think Dr. Topol made a really, really strong argument for how AI is a better predictor of potential disease or some sort of a condition, more so than physicians and doctors, because of errors. What do you think?
Yeah, I mean, it's like calculators. They're better than us.
You can't take my example.
They're not going to make a mistake.
You didn't respond quick enough.
But, like, people aren't that good at maths, so why bother trying? Like, it's just like a vanity thing if you want to say that, "Oh, yeah, I'm way better." I mean, you're not.
I think he's great, isn't he, Dr. Topol, you know? And he's talking about a lot of things that we really want to get behind, don't we? You know, we want to prevent these diseases and all this kind of stuff. But, you know, I'm going to be honest, it does feel a little bit like wishful thinking to me, right? You know, that we're going to collect all this data and then we're going to use this data to prevent these diseases, which means exercise and diet. Well, we know we need to exercise, we know we need diet. But the people who really need that help most are not going to get it.
Have access.
We don't collect the data. They're not going to be part of the trial. So I don't see the systematic organization that's needed to do both things, to help with the AI for the mammogram and, you know, for the lifestyle thing. And that to me, this is sort of societal organization, if you like, is...
I hope that he's right though in that eventually...
Well, I hope he's right.
The adoption will...
I'd like to be a super ager too.
Yes, yes.
I'm the one who needs it most out of the three of us.
But data wise, right, as in everyone has an iPhone. So if you do have things like wearables, which are an accessible point, and that's dependent on the companies, that's kind of what he said, that it's kind of contingent on the people who are making the stuff to be good, but that's not really a strategy, is it, where you say, "Please, Mr. Altman, don't bankrupt me."
Yeah, I want this, but I have to be able to afford it.
I don't see the profit motive and I don't see it happening.
Ever?
I'm not saying I don't ever or in pockets, but I don't see it universally happening 'cause I don't see the profit motive.
Interesting. All right, well, we'll have to see. A huge thank you to Dr. Eric Topol for joining us today. If AI could predict your disease decades early, but the price was that that data is sitting in the hands of tech companies and insurers, not necessarily your doctor, would you still want it? We would love to hear from you. Drop your comments below, subscribe. And then we will see you next time right here on Agents of Tech.