Ask a vendor how their clinical agent is kept safe and you will often get the name of a single framework, said as though it settles the question. It does not, because no published standard is specific to clinical AI agents as of July 2026. What exists instead is a stack of frameworks, each built for a different layer of the problem, that a hospital has to assemble and reconcile itself. That assembly is the real safety work, and doing it well starts with knowing which published standard governs which layer. This guide is that map. As of July 2026.
The stack at a glance
The frameworks below were written by different bodies for different purposes, and they overlap at the edges. Read the table as a division of labor rather than a menu: a serious program draws on most of these rows at once, because a gap in any one layer is a gap in the whole.
| Layer | Published standard | What it governs |
|---|---|---|
| General AI risk | NIST AI RMF 1.0 1; ISO/IEC 42001 9 | Governance scaffolding and lifecycle management for any AI system |
| Generative-AI risk | NIST Generative AI Profile 2 | Risks specific to generative models, mapped to the RMF functions |
| LLM & agent security | NIST adversarial-ML taxonomy 3; OWASP LLM Top 10 & Agentic guide 45 | Attacks and agent-layer threats: injection, tool use, excessive agency |
| Human oversight | EU AI Act, Article 14 6 | Legal duty of effective oversight and automation-bias awareness |
| Healthcare ethics & assurance | WHO principles & LMM guidance 78; CHAI 10 | Health-specific ethics, equity, and assurance across the lifecycle |
| Regulatory lifecycle | US FDA DHAC materials 11 | How generative-AI devices may be evaluated premarket to postmarket |
Layer one: general AI risk
Two documents anchor most healthcare AI governance programs. The NIST AI Risk Management Framework organizes the work into four functions — Govern, Map, Measure, and Manage — a scaffold general enough to hold any AI system and specific enough to drive a real program 1. Alongside it, ISO/IEC 42001:2023, the first AI management-system standard, specifies requirements for establishing, maintaining, and continually improving an AI management system, including impact assessment and lifecycle oversight, and it is certifiable by an independent body 9. The NIST framework tells you what to reason about; the ISO standard gives you an auditable management system to do it inside. Neither was written for agents specifically, which is exactly why they sit at the base of the stack rather than the top.
Layer two: generative-AI risk
Agents are built on generative models, and generative models carry risks the classical frameworks did not anticipate. NIST's Generative AI Profile catalogs twelve generative-AI-specific risks — among them confabulation (the confident fabrication a clinician would call hallucination) and human-AI configuration (how the handoff between person and system is arranged) — and maps mitigation actions back onto the RMF's four functions 2. For a clinical agent, two of those risk names do heavy lifting: confabulation is the failure a human in the loop exists to catch, and human-AI configuration is the design question of whether that human can actually catch it.
Layer three: LLM and agent security
This is the layer where agent-specific danger concentrates, and where the most targeted published guidance now lives. The NIST adversarial-ML taxonomy formalizes prompt injection as a generative-AI integrity attack and distinguishes direct injection from indirect injection embedded in retrieved data 3 — the precise threat that grows once an agent reads record content, covered in depth in prompt injection in clinical systems. OWASP's work goes further into agent territory than any general framework. Its Agentic AI threats guide provides a threat taxonomy across agent design, memory, planning and autonomy, tool use, and deployment 4 — the layers a system acquires precisely when it stops being a chatbot and starts being an agent. And its LLM Top 10 names excessive agency, "the vulnerability that enables damaging actions to be performed in response to unexpected, ambiguous or manipulated outputs," arising from excessive functionality, permissions, or autonomy 5. The mitigation is a discipline: grant the least capability a task requires, and no more. This is the one layer where a hospital cannot lean on its general AI governance and call the agent covered.
Layer four: human oversight, as law
Oversight of a high-stakes clinical agent is more than good practice in some jurisdictions — it is a legal design requirement. The EU AI Act requires that high-risk AI systems, a category that captures most consequential clinical uses, be designed so they "can be effectively overseen by natural persons," and it names the failure mode directly: overseers must "remain aware of the possible tendency of automatically relying or over-relying on the output," which is automation bias 6. The distinction that matters for agents is between a human who approves every action and a human who monitors a running system and intervenes on exception — and the more autonomy an agent has, the more the law's "effective" qualifier has to be engineered rather than assumed. Ongoing monitoring of a deployed agent for drift and emerging failure — algorithmovigilance — is the operational other half of this oversight duty.
Layer five: healthcare ethics and assurance
The layers above are domain-general; healthcare adds its own. WHO's foundational 2021 guidance sets six principles for health AI — including protecting autonomy and fostering responsibility and accountability — that later, narrower controls inherit 8. Its 2024 guidance on large multi-modal models carries the warning forward into the generative era, cautioning that such systems can encourage automation bias, whereby errors are overlooked that would otherwise have been identified 7. On the assurance side, the Coalition for Health AI (CHAI) publishes healthcare-specific guidance — a Responsible AI Guide and assurance standards organized around the AI lifecycle, with transparency carried through model cards — aimed at helping health systems evaluate and deploy AI responsibly 10. These do not replace the security layer; they add the equity, consent, and clinical-context questions that a generic security framework never asks.
Layer six: the regulatory lifecycle
Finally, regulators have begun to reason about generative AI across the whole life of a product rather than at a single approval gate. The US FDA's Digital Health Advisory Committee convened in November 2024 to weigh how generative-AI-enabled devices should be evaluated across the total product lifecycle — premarket performance evaluation, risk management, and postmarket performance monitoring 11. The signal for agent builders is that "we validated it once" ages badly: a generative system that updates or drifts is expected to be watched after deployment, rather than cleared once before it. Which regulatory regime binds a given deployment depends on jurisdiction and on whether the system meets a device definition — a determination for regulatory counsel, tracked in our global AI in health regulation tracker.
Composing the stack: a worked example
Consider a hospital deploying an agent that retrieves infection-control data from its record — the kind of system a real-hospital study has shown is buildable. What does the composed safety case look like? From the base layer, the program runs inside an ISO/IEC 42001 management system 9 and uses the NIST RMF's four functions to structure its risk work 1. From the generative layer, it treats confabulation as a named risk and designs the human handoff — the human-AI configuration — deliberately 2. From the security layer, it maps the agent's tool use and memory against the OWASP agentic taxonomy 4, grants the least agency the task needs 5, and hardens the retrieval path against the injection the NIST adversarial-ML taxonomy formalizes 3. From the oversight layer, it designs review so a clinician can actually catch an error rather than wave it through, as the EU AI Act requires 6. From the healthcare layer, it checks the deployment against WHO's equity and accountability principles 8 and CHAI's lifecycle assurance 10. And from the regulatory layer, it plans for postmarket monitoring in the spirit of the FDA's total-product-lifecycle direction 11. No single document told the hospital to do all of that; the composition did. Skip a layer and the omission stays invisible until it fails — which is the whole argument for treating the stack as one system rather than a menu to pick from. The corollary is a due-diligence question a buyer can ask any vendor: show me, layer by layer, which control you satisfy and how you tested it. A confident answer names each layer; a weak one names one framework and hopes the rest go unmentioned.
The gap, stated plainly
Lay the six layers side by side and the missing piece is obvious: none of them is a clinical-agent safety standard. NIST is domain-general; OWASP is domain-general; ISO is domain-general; the healthcare bodies are strong on ethics and assurance and lighter on agent-specific tool-use and autonomy controls; the regulators are early. A hospital deploying an agentic system therefore composes its own safety case by pulling the relevant row from each layer — governance from NIST or ISO, injection and agency controls from NIST adversarial-ML and OWASP, oversight from the EU AI Act, ethics and assurance from WHO and CHAI, and lifecycle expectations from the FDA's direction of travel. The composition is the deliverable. Anyone who hands you one framework as the answer has skipped five layers.
How to read this map
Three cautions. First, these standards evolve on independent clocks — the specific versions and risk names here are current as of July 2026 and will change, though the layered structure is more durable than any single revision. Second, adopting a framework is not implementing it: most real failures come from controls that exist on paper and are misconfigured in practice, which is why the security layer rewards testing over attestation. Third, this map assesses the frameworks, not any product — a system audited against a weak subset of layers is less safe than its paperwork suggests, and the burden is on the deployment to show which layers it actually satisfies. For where these controls meet real systems, see our documented agentic deployments tracker and the guide on MCP and interoperability for hospital agents. For clearance and reimbursement decisions that turn on any of this, confirm current requirements with your compliance or regulatory counsel before acting.
Sources and method
This guide maps eleven published frameworks and standards to the agent layers they govern: the NIST AI RMF 1, its Generative AI Profile 2, and its adversarial-ML taxonomy 3; OWASP's Agentic AI threats guide 4 and excessive-agency risk 5; the EU AI Act's oversight article 6; WHO's large-multi-modal-model guidance 7 and 2021 principles 8; ISO/IEC 42001 9; CHAI's assurance guidance 10; and US FDA Digital Health Advisory Committee materials 11. Every claim traces to the source cited beside it, and each citation was checked to resolve before publication. We revisit this page on a 180-day cycle and whenever a governing standard is issued or revised.