The Hidden Dangers of AI in Healthcare

Artificial Intelligence in healthcare gets a lot of praise — and often for good reason. More and more patients, and even doctors, are placing their trust in AI tools. What many of us fail to notice is that the same technology can also cause real harm, sometimes with grave consequences. That cuts against the very first principle of medicine: first, do no harm.

Most such cases never make the news. The five cases below — each showing a different kind of danger, and all reported in the media or documented by researchers — reveal how AI advice or AI-driven tools malfunctioned and left patients seriously harmed. They are almost certainly a small fraction of the true number. Together, they point to a single lesson: in healthcare, AI must be used with the utmost caution.

10+patients injured during surgeries that used one AI navigation tool

14×rise in fault reports for that tool after AI was added (8 to 100+)

40–50patients per clinic now arriving with harm from AI self-treatment

Case 01It can tell patients to stop their prescribed medicine

This is the most direct danger of all. A patient asks an AI tool about their treatment, takes its words as medical permission, and stops a drug the doctor told them to keep taking.

Reported case · India

A young transplant patient lost her new kidney

A 30-year-old woman had received a kidney transplant. After an AI tool told her that her “normal creatinine” meant she no longer needed her medicines, she reportedly stopped taking her antibiotics. Within weeks, her transplanted kidney began to fail, her creatinine shot up, and she was back on dialysis.

Senior kidney doctors at NIMS pointed to a worrying pattern: even well-educated patients are now acting on AI answers without checking with their own care team.

As reported in Indian news media (2025).

Case 02It gives one-size-fits-all advice that ignores your condition

An AI tool does not examine you. It does not know your other illnesses, your medicines, or your test results. So its “general health tips” can be exactly wrong for the person reading them.

Reported case · India

A diabetic man's sodium dropped dangerously after cutting out salt

A 62-year-old man with diabetes followed a plan from an AI tool that told him to cut out salt completely. He lost weight rapidly and his blood sodium fell to a dangerous level. As one government kidney specialist put it: “General tips ignore the patient in front of you.”

As reported in Indian news media (2025).

Reported case · USA · 2025

A man poisoned himself swapping salt for a chemical

Wanting to cut salt from his diet, a 60-year-old man asked an AI tool what to use instead and understood it to suggest a chemical called sodium bromide. He bought some, used it for three months, and ended up in hospital for three weeks with poisoning, paranoia, and hallucinations. The AI tool never warned him the chemical was unsafe to swallow.

Reported as a medical case in the Annals of Internal Medicine: Clinical Cases (2025).

Case 03It can hand out dangerous do-it-yourself advice

Some AI tools do not stop at bad diet tips — they tell people to carry out risky procedures on themselves.

Reported case · Morocco

A man was injured trying to treat his own piles

A 35-year-old man reportedly followed an AI tool's instructions to place rubber bands around his haemorrhoids (piles) himself. The result was an injury serious enough to need emergency medical treatment.

As reported in news media.

Case 04It can make people put off seeing a real doctor

Perhaps the quietest harm of all is delay. A confident answer makes people feel reassured, so they treat themselves at home and reach a doctor only once things have turned serious.

Reported case · India

Self-treatment delayed care until it became an emergency

A 42-year-old office worker had ongoing tiredness and mild stomach discomfort. Instead of seeing a doctor, he turned to an AI tool and began treating himself based on its suggestions. Weeks later his condition worsened and he needed emergency care.

Doctors say this is no longer a rare event. Some clinics now report 40 to 50 patients each with problems caused by AI-guided self-diagnosis — wrong drug doses, stopping prescribed medicines, or starting new treatments without asking a qualified doctor.

As reported in Indian news media (2025).

A confident answer is not the same as a correct one. An AI tool can sound calm and certain while being completely wrong about the person reading it.

Case 05Even the machines in the operating room can be wrong

The danger is not limited to the AI tools patients use at home. AI is now built into surgical equipment too — and when one of those tools is wrong, the surgeon may not realise it until the damage is done.

Reported case · USA · 2021–2025

An AI surgery tool linked to strokes and skull injuries

After a popular surgical navigation system (the TruDi system, used in sinus operations) added AI in 2021, safety reports to the US regulator jumped from about 8 to more than 100 — roughly a 14-fold rise by late 2025. At least 10 patients were reportedly injured. The tool allegedly misled surgeons about where their instruments were inside patients' heads, leading to leaks of spinal fluid, a punctured skull base, and at least two strokes after a major artery was damaged.

The makers deny that the technology directly caused the injuries, and the lawsuits are still ongoing — but the pattern alarmed safety experts.

Reuters investigation (February 2026).

Safety reports for one AI surgical tool, before and after AI was added

When an AI tool tells a surgeon where to cut and is wrong, there is often no way to know until the harm is done.

---

The Bigger PictureQuieter dangers behind the scenes

Beyond these individual patient stories, AI carries deeper risks inside hospitals and health systems:

It can be unfair. A widely used US tool that decided who got extra care rated many Black patients as healthier than they actually were, because it used past spending as a stand-in for how sick someone was. It affected around 200 million people. (Science, 2019)

It can be used to deny care. US insurers have been sued over AI tools used to cut short care for elderly patients; in one case, about 9 out of 10 denials were overturned on appeal — yet very few patients ever appealed. (STAT & CBS News; ongoing lawsuits)

It can fail quietly and cry wolf. A widely used hospital tool meant to warn staff about sepsis (a dangerous infection) missed about two-thirds of cases when tested independently, while raising so many false alarms that staff stopped trusting it. (JAMA Internal Medicine, 2021)

It can make skilled people rusty — the “lazy doctor” phenomenon. After a few months of leaning on AI, experienced doctors became worse at spotting growths on their own — their unaided detection fell from about 28% to 22%. (The Lancet Gastroenterology & Hepatology, 2025)

What To DoHow to use AI without getting hurt by it

The goal is not fear — it is care. If you are a healthcare provider, a few simple habits can protect both your patients and your practice:

1

Ask about AI use when taking a history

Make a patient's use of AI for medical advice a routine, mandatory question during history-taking and initial assessment.

2

Tell patients plainly: an AI tool is not a doctor

Warn them never to stop, start, or change a prescribed medicine — or treat themselves — on the strength of an AI answer. Give them an easy way to check with your team first.

3

Keep a qualified person in charge of every clinical decision

Treat every AI output — whether an AI tool's answer or a surgical system's guidance — as a suggestion that a trained professional must check, never the final word.

4

Verify AI output before it reaches the patient

A qualified person should review any advice or information an AI produces before it is passed on to a patient.

5

Be open, and protect patient data

Tell patients when AI is used in their care, obtain consent where needed, and guard their information carefully.

The bottom line

AI is a tool, not a colleague

None of these stories mean you should avoid AI. The same technology that caused these harms is also catching cancers earlier and giving doctors their time back. The difference between help and harm is almost never the tool itself. It is whether a careful, well-trained team — and a well-informed patient — is watching over it.

So use AI — but keep your eyes open, keep a human in the loop, and never let a confident screen replace good judgement.

A note on the cases: several of the patient stories above are drawn from news reports and may not be independently verified in every detail. Please confirm them against the original sources before publishing.

Sources & further reading: Transplant, diabetic-sodium and self-diagnosis cases — Indian news media reports (2025); chemical-poisoning case — Annals of Internal Medicine: Clinical Cases (2025); haemorrhoid-banding case — news media report; AI surgical navigation tool (TruDi) — Reuters investigation (February 2026); unfair algorithm — Science, Obermeyer et al. (2019); insurance denial tools — STAT & CBS News, class-action lawsuits (2023 onwards); sepsis warning tool — JAMA Internal Medicine (2021); AI deskilling in colonoscopy — The Lancet Gastroenterology & Hepatology (2025).

Patient’s Fall Risk Assessment

One of the common risk to safety of patients in hospital is the risk of fall. Several epidemiological studies has found that on an average 3 to 5 patient fall incidence occur in every 1000 bed-days. It is also estimated that a third of fall results into injuries which could be severe such as fracture. Due to the widespread prevalence and resulting harm, prevention of patient fall is included as one of the International Patient Safety Goals (IPSG) of JCI standards for hospitals.

The first step to prevention of fall is identifying patient who is at a risk of fall. Most accreditation bodies, including NABH and JCI expects hospital to undertake a fall risk assessment of all admitted patient and take preventive measures for those who are at a higher risk of fall.

The table below describes the points that should be used for assessing risk of fall, and classifies features into very high, high, moderate and low risk categories.

Code Pink system in hospital

Code pink is an emergency code which is used to activate a set of action in case a child/baby is missing from the ward/room. As there is a fair possibility that the missing child/baby is abducted, it is considered as an emergency situation and a predefined set of actions are taken on an urgent basis to safeguard the missing child/baby. The entire system of coordination, communications, decisions and actions followed during such situation is called as a code pink system. (Also check Code Blue, Code Red)

Depending upon how a hospital is structured and organized the code pink system may vary from one hospital to another. The objective, however, remains the same, i.e. ‘Find the missing child as soon as possible’. The objective should be achieved in a manner that other critical activities in the hospital do not get hampered and that unnecessary scare or commotion of public be avoided. An illustrative code pink system has been described below. This can be taken as a reference for developing code pink system by various hospitals. 

Critical findings in Imaging – Policy and procedure

Imaging and Medical laboratory frequently come across critical results or finding of a patient’s diagnostic tests which requires immediate intervention from the doctor to bring the patient out of the criticality. Hence, a hospital must have a policy and procedure in place for identification and quick communication of such results. This posts describes critical findings in imaging (Check critical test results in laboratory –policy and procedure).

Standard Precautions for Infection Control in Hospitals

Standard precautions are the basic infection control practices which must be adhered to while caring any patient in hospital. If fully implemented, standard precaution can drastically reduce the risk of infection to healthcare providers and patients. They are minimum level of precaution and may not be sufficient for special situations which requires special precautions. As these precautions should be taken in all kinds of patient care process they are also called as universal precautions.

Elements of standards precautions are as follows

      

Critical test results in laboratory – Policy and process for identification and communication

Medical laboratory and Imaging frequently come across critical results or finding of a patient’s diagnostic tests which requires immediate intervention from the doctor to bring the patient out of the criticality. Hence, a hospital must have a policy and procedure in place for identification and quick communication of such results. This posts describes critical test results in medical laboratory (Check critical findings in imaging–policy and procedure). 

Critical test results in laboratory are the findings in the lab tests of a patient which indicates that condition of the patient may be critical or even life-threatening.  Such results, when found shall urgently be informed to the treating physician of the patient so that required interventions can be carried out on time and patient can be saved from any possible adversities. To be able to do so, a hospital needs to have a well-developed process of ‘identification and urgent communication of critical test results in lab’. The process should be able to achieve following objectives.

1. Critical test results gets identified within the lab as soon as the test results are obtained
2. No critical test results gets missed from identification
3. Non-critical test results do not get identified as critical
4. The treating doctor of the concerned patients gets to know about the critical test result on an urgent basis

Any lapses in identification and/or communication of critical test results to concerned doctor may lead to severe consequences for patient, including death. Hence, in addition to the process, hospitals must have a policy that mandates compliance to this process by laboratory staff. Following points shall be taken into consideration for policy and system on identification and communication of critical test results in laboratory.

Patient identification Policy and Procedure

One of the most common causes of medical errors in healthcare is the incorrect identification of patients. The error can lead to potentially serious consequences such as surgery of a wrong patient or transfusion of wrong blood into a patient. It is vital that hospitals must put into place an effective policy and procedure for identifying a patient. Below are the internationally recommended practices for accurately identifying a patient.

Safe transfer of unstable patient from hospital

One of the critical task that hospitals have to frequently undertake is to transfer a critically ill or unstable patient from one hospital to another. Transfer of such patient are likely to induce various physiological changes, which may adversely affect the health of patient even leading up-to death. Hence, such transfers shall be undertaken with great care and as per evidenced-based guidelines. Following are the key elements and guidelines for safely executing transfer for an unstable patient.

List of medical errors leading to patient harm

(Also check 'Patient Identification Policy and Procedure)

Medication related

1.       Administration of wrong medicine

2.       Administration of medicine to a wrong person

3.       Administration of wrong dose of medicine

4.       Administration of medicine through wrong route

5.       Administration of medicine at wrong time

6.       Administration of medicine with wrong rate of administration

7.       Administration of expired medicine

Taking Care of Vulnerable Patients

Vulnerable patients are those patients who, for any reason, are not able to protect or take care of himself/herself, against exploitation or harm. Such patients are prone to various risks within the hospital, such as fall, injury, neglect, abuse, medical errors and acquiring of infections. Vulnerability of a patient may be due to his/her age, physical or mental condition. It is the duty of the hospital to identify such patients and provide them with necessary support so that they are safe in the hospital surroundings.

Following type of patients can be identified as vulnerable

      1.       Patients who are old (above 65 or a certain age, as decided by the hospital)

      2.       Patients who are of minor age (Patients below 12 years or as decided by the hospital)

      3.       Patients with physical problems, such as limited mobility, blindness, deafness, speech limitations etc.

      4.       Patients with mental problems, such as depression, mental retardation, forgetfulness etc.

      5.       Patients who are unconscious or in semi-conscious stage

      6.       Patients who are illiterate and have difficulty in understanding written instructions

Code blue system in hospitals

Code blue is perhaps the most popular codes used in hospital for managing emergency situation. Code blue is a code given to identify and communicate that a medical emergency, of the nature of cardiac arrest, has occurred and the patient needs to be attended immediately for life saving measures. (Other codes - Code Pink, Code Red). Since it deals with the life threatening situation, swift and coordinated action by a team of professionals is of paramount importance. This calls for designing and implementing a highly efficient system, which can work in round the clock and can cover entire hospital. This posts elaborate on all considerations that should be made while designing a code blue system. Also check code blue form and crash cart checklist along-with this post.

Checklist of facility safety inspection for NABH accreditation preparation

NABH puts high importance to safety in hospitals and ask the preparing hospitals to ensure that the facility is safe for patients, staff and visitors. As per one of the objective elements (FMS.1 e. of NABH 4^th edition), the preparing hospital is required to conduct facility inspection rounds to ensure safety. Further, it also states that the round should be conducted at-least twice in patient care areas and once in non-patient care areas. In-order for these rounds to be effective, it is important that a comprehensive checklist be used so that common safety hazards and potential risks could be checked and necessary remedial measures be taken. Below, I have prepared a checklist of points that could be used for such rounds. The checklist also addresses various other safety parameters that has been asked in different standards and objective elements of NABH standards book.

This checklist would be of much more value if used along-with the Infrastructure checklist for NABH accreditation preparation.