Real stupidity beats artificial intelligence every time.
Terry Pratchett, Hogfather (1996)
Thesis: We're employing AI1 to fix problems in medicine we already know how to fix without it but can’t be arsed to.
For an ungainly amount of time now, regulators, scientists, clinical quality people, policymakers and others have been trying and failing at getting doctors to use antibiotic prescription guides to decide what first-line drug to give and what to use as second-line. no.one.cares.
every hospital and every department has their own way. I’ve seen places where different consultants in the same department had their own preference for the same diagnosis. And somehow it was the resident’s job to remember that Dr. Mehta wants ampi+genta for his appendicitis patients and Dr.Thomas would scream at you if you didn’t write cirpo+ceftriaxone+sulbactam for his appendices.
this is not limited to antibiotics; the first line anti-hypertensives used, choice of drugs for diabetes, basic investigations for low back pain and many many more things have very poor adherence to standards and evidence. reading a prescription audit paper from India is depressing. brothers and sisters, we are terrible at doing medicine according to standards or evidence.
this is because in India as in many other countries, many, perhaps most, doctors stand to make money from using specific drugs and from writing investigations2 and this, among other things, makes doctors poor governors of other doctors.
even when this direct economic benefit is not present, the science of medicine is influenced by the superstitions of medicine and the egos of the hotshots making life and death decisions. there is neither any accountability, nor any punishment3.
No amount of tech will fix this, for as long as the incentives remain broken.4
for a lot of people pushing the AI pill on to doctors, convincing us that this supposed superintelligence is to be trusted is a trick to get us to do the basics things right5. Don’t write antibiotics for someone with a viral cold FFS.
for others, it is because there is money to be made here. How many hospitals can really afford to build train and deploy their own models or even have a data-science team? In most of the world, and not just in hospitals, the datascientist is someone with an excel formula addiction and the database is the company google drive filled with weekly reports.
for a few it is about research. And, I think, that is the stage at which AI works best. that is the level of maturity of the tech itself. it is far too early to do AI (read: LLM) interventions in medicine that reach the bedside. for one, the stack keeps changing. for another, the entire tech industry is now all about how to make LLMs useful, which has nothing to do with how to make useful things.
the exception to this (no bedside rule) is probably computer vision, but still, the long road ahead there is about figuring out how it can be useful to the GP and the intern and the resident who hasn’t slept in two days and has 9 more patients to see before she gets to sleep.
Most of the work in computer vision in medicine seems to be in how to make it work for the radiologist. Which seems foolish to me, they already know how to read CT scans and MRIs, it’s the rest of us who need help. But the complexity of the operational aspects of this prevent the interventions from reaching the people who need it the most.
It’s like we’re installing more lights in well-lit areas because that’s the only place that has electricity connections.
sure, if tele-radiology can churn out more reports every day thanks to AI, this will help a lot of people everywhere, and there is a legit business opportunity and it should be taken. But it should not be confused with “democratizing healthcare”6.
The industry calls this the last mile problem; someone ought to tell them that in India as in life, the last mile is 90% of the distance7.
What I am saying is that before we swallow the AI pill, we must with brutal honesty answer the following questions.
- Will this help the care provider or the patient in a way that improves outcomes?
- Does this solve a problem the provider and the patient actually wants solved?8
- Is AI the right way to solve this?
And if the answer isn’t a resounding yes to the first two of those questions, maybe don’t do it.
Footnotes
- I know AI isn’t just language models, but well, that’s what the industry is claiming, so why should we not treat chatgpt as the alpha and omega of AI? ↩︎
- because many if not most doctors get kickbacks or baksheesh or “referral fees” as it is called in India. And this can account for up-to a fourth of the doctor’s income ↩︎
- we’ve all seen the HOD’s favorite SR get away with more sins than politicians in India do. ↩︎
- Medicine isn’t unique in having ethical challenges, the world of AI has its own ethical issues, I recommend my slightly dated guide as a starting point: Ethical AI, Basic Readings ↩︎
- I am sympathetic to this camp. in fact, i think we should stop letting doctors decide first line medicines and dosages for most problems. doctors should only be allowed to tweak the prescription to the patient’s preferences or needs. ↩︎
- that term gives me reflux ↩︎
- also, we measure things in kilometers here because we are a civilized people ↩︎
- there are, of course any number of important questions to ask; will this intervention marginalize people who are already have a raw deal? what harms will this cause? is this sustainable? will this benefit the last woman standing in the line? ↩︎





