Prompt injection is an attack on AI systems where text hidden in the input makes a model follow instructions it was never meant to obey. In a clinical system, that input can be a scanned slide, a pasted referral, or a record field — blurring the line between data and command. As of September 2026.
The reason the flaw is so stubborn is structural: a language model reads its instructions and its data through the same channel, so anything that looks like a command can be treated as one. OWASP ranks it as the number-one risk in its Top 10 for LLM applications, defining a prompt injection vulnerability as one that occurs when prompts alter the model's behavior or output in unintended ways 1.
Why it matters in healthcare
Clinical AI systems are built to read untrusted content. A documentation assistant summarizes a referral letter someone else wrote. A triage tool ingests a PDF uploaded by a patient. A vision-language model reads a slide scanned in another lab. Every one of those is external content the system did not author — exactly the surface where injection lives. When the output feeds a note, an order, or a diagnosis, a successful injection stops being a chatbot curiosity and becomes a patient-safety event.
How it works in practice
There are two forms, and both matter clinically. Direct prompt injection comes through a user's own input — the classic "ignore your previous instructions" typed into a prompt. Indirect prompt injection is subtler: the model accepts input from an external source such as a website or file, and hidden instructions in that source alter its behavior when interpreted 1. The NIST adversarial machine-learning taxonomy formalizes both as generative-AI integrity attacks, defining indirect injection as malicious instructions embedded in the external data a model retrieves 2. The foundational security paper on the technique showed adversaries could remotely compromise LLM-integrated applications by planting instructions in data the model was likely to retrieve — turning ordinary documents into attack vectors 3.
Where it appears today
Two peer-reviewed studies have already demonstrated the risk on real clinical inputs. In an oncology study, researchers ran 594 attacks against four vision-language models and showed that sub-visual prompts embedded in medical imaging data could drive harmful output while remaining imperceptible to a human reader 4. More striking still, a histopathology study found that injection can happen by accident: handwritten labels and watermarks that naturally appear on real slides acted as unintended prompts, pushing model accuracy on misleading labels down to 0-10% even though baseline accuracy sat between 30 and 65% 5. No attacker was required — the ordinary artifacts of clinical work were enough.
Common misunderstandings
The first misconception is that prompt injection is a prompt-writing problem solved by a better system prompt. It is an architectural weakness in how models separate instructions from data, and instruction tuning alone does not close it. The second is that only a malicious insider can trigger it; the histopathology result shows incidental text does the same. The third is that output filtering is enough — filtering helps, but the durable controls are least-privilege tool access, keeping untrusted content out of the instruction channel, and human review of any consequential output.
Related terms
Prompt injection targets the clinical LLM at the center of these systems, which is why a human in the loop remains the last line of defense before output affects care. The risk grows sharply once models gain tools and connections through the Model Context Protocol, because an injected instruction can then reach real actions. For the wider view of where clinical AI is deployed and what is at stake, see our AI in healthcare statistics.
We refresh this page yearly, and sooner when OWASP revises its LLM Top 10 or a new clinical attack-or-defense study lands.