Connecting an AI agent to the electronic health record is where most of the promise and most of the difficulty of clinical AI now live. A model that only chats is bounded by what it was told; a model wired into the record can pull a patient's labs, weigh them against a history, and draft the next step. That reach is the whole point — and it is also what turns a demo into a safety problem. This guide breaks the task into the three jobs an EHR-integrated agent actually has to do, and reports what the published evidence says about how well today's systems do each. As of July 2026.
Three jobs: read, reason, act
Strip away the vocabulary and an EHR agent does three things in sequence. It reads — retrieves the relevant slice of a record that may hold thousands of data elements. It reasons — combines those elements into an answer, a risk, or a plan. And in the most capable designs it acts — drafts an order, a referral, or a note for a clinician to approve. Each job is governed by a different standard and measured by a different result, and conflating them is how vendors make a system that does the first job sound like it does the third. The honest reading of the evidence is that the three jobs are not equally solved.
The substrate: how an agent reaches the record
Before an agent can read anything, it needs a sanctioned way in. In practice that is SMART App Launch, the HL7 profile that layers OAuth 2.0 over FHIR so a client application can be granted precisely scoped access — which resource types it may read or write, for which patient, under whose authorization — including a backend-service pattern for automated clients with no human at the keyboard 7. This matters for agents specifically: the scope you grant is the blast radius you accept. An agent handed broad write scopes is a larger risk than one granted read-only access to two resource types, and the standard is where that line gets drawn. The newer connection layer — wrapping those FHIR calls as Model Context Protocol tools — sits on top of this substrate rather than replacing it; our companion guide on MCP and interoperability covers that seam in depth.
What the benchmarks show
The most systematic evidence comes from benchmarks that put agents inside virtual or real records and score them. Read as a group, they tell a consistent story: retrieval quality falls as task complexity rises.
| Evidence source | What it tests | Scale | Headline result |
|---|---|---|---|
| Reference EHR-agent benchmark 1 | Physician-written tasks in a FHIR virtual record | 300 tasks · 100 patients · 700,000+ data elements | Best of 12 models completed 69.67% |
| FHIR question-answering benchmark 2 | Agent QA over interoperable FHIR data | 2,931 real-world questions | Intricate resources and reasoning "critically affect" performance |
| Real-hospital retrieval study 3 | LLM + MCP tools on a live EHR | 6 infection-control tasks | Near-perfect on simple tasks; fails on complex time-dependent one |
The reference benchmark anchors the capability conversation: 300 physician-written tasks run inside a FHIR-compliant virtual record of 100 patients and more than 700,000 data elements, with the best of 12 evaluated models completing 69.67% 1. That is strong enough to be useful and far enough from 100% that roughly three in ten tasks failed. A second benchmark isolates the reading job and shows why: grounding 2,931 real-world clinical questions in the HL7 FHIR standard, it found that "the practical challenges of retrieving data from intricate FHIR resources and the difficulty of reasoning over them" both critically limit question-answering performance 2. The data model itself — deeply nested, reference-linked, verbose — is part of the difficulty, before any clinical judgment enters.
The reference benchmark's headline invites a mistake worth naming: reading 69.67% as a grade the way one reads an exam score. It is closer to a coverage figure — the share of realistic tasks a top model handled in a controlled record — and the roughly three in ten it missed are the ones that decide whether a system is deployable. A production setting also adds what the benchmark strips out: incomplete notes, interrupted sessions, and inputs no test harness anticipates. Read the number as a ceiling for the easier version of the job, rather than a passing mark.
The real-hospital data point
Benchmarks run on curated environments; one study ran on a live one. Researchers connected an LLM (GPT-4.1) through a planning-and-acting agent to a hospital's actual EHR using custom Model Context Protocol tools, and tested six tasks drawn from the infection-control team's real questions. The agent achieved near-perfect accuracy on simple retrieval and degraded on a complex, time-dependent calculation, with most errors traced to incorrect tool arguments or misinterpreted tool results; the authors position the framework as one that "may serve as a foundation for hospital AI agents" 3. Two lessons carry over. First, the failures were not hallucinated facts so much as tool-use mistakes — the agent called the right instrument with the wrong parameters — which is a different engineering problem than model accuracy alone. Second, "simple" and "complex" are doing real work here: the gap between them is the gap between a lookup and a calculation, and it maps directly onto the read-versus-reason split.
Making the reach reusable
A recurring cost of EHR integration is that every record has historically demanded its own bespoke connector. A peer-reviewed open-source framework attacks that directly by wrapping HL7 FHIR resources as Model Context Protocol tools, giving a model declarative access to diverse FHIR resources through JSON configuration rather than per-EHR hardcoded retrieval logic 4. The significance is economic as much as technical: if the reading job can be standardized behind a protocol, a hospital's integration effort shifts from rebuilding plumbing to governing access. That is the same consolidation logic that makes the connection layer worth understanding on its own terms.
The one deployment that changed care
Benchmarks and frameworks describe capability; one real-world study measured outcomes. A generative decision-support agent built into a primary-care network's EHR was studied across 39,849 visits at 15 clinics, and clinicians who had access made 16% fewer diagnostic errors and 13% fewer treatment errors than those who did not, with the system designed to activate "only when needed and preserving clinician autonomy" 5. It is worth being precise about what this is: an advisory system that reads the note as it is written and surfaces a flag, sitting on the decision-support end of the spectrum rather than acting on the record itself. The full record of this and other live systems is in our documented agentic deployments tracker.
The act job: drafting for approval
Reading and reasoning are visible in benchmarks; acting is where the evidence thins and the design stakes rise. An agent that drafts an order, a referral, or a note is doing the third job — and every documented system that touches it routes the action through a clinician rather than executing it. The Nairobi deployment is instructive precisely because it stops short: it surfaces an advisory flag a clinician acts on or dismisses, "preserving clinician autonomy," rather than placing an order itself 5. That restraint is a feature. The moment an agent writes to the record, two things change at once — the SMART scope it needs widens from read to write 7, and the cost of a wrong action stops being a bad answer and becomes a bad event. This is why the working definition of an agentic system treats acting as the capability that raises governance weight, and why no documented EHR deployment lets the agent act unsupervised. The practical lesson for anyone scoping the third job is to design the approval step first, because it is the control that makes the other two jobs safe to deploy.
Cost and oversight are load-bearing
Two constraints shape every EHR-agent design and rarely make the demo. The first is cost. A 2026 benchmark of clinical decision tasks measured the agent pattern consuming more than ten times the token usage and more than twice the latency of the baseline models inside it 6 — the cost of a system that decomposes a task, calls tools, checks itself, and retries. The second is oversight, and it is not free either. Every documented EHR deployment keeps a human in the loop, which is the right posture and also an imperfect one: a systematic review of 74 studies documented automation bias — the tendency to over-rely on automated advice — across clinical decision-support settings 8. A clinician present is not the same as a clinician catching errors, and the strength of the control depends on keeping error rates low enough that review stays meaningful. This is the same logic that governs any clinical decision support system.
A third constraint is quieter and just as real: an EHR agent is never finished at go-live. Records change shape as templates and coding practices shift, models are updated, and a system that retrieved correctly in a pilot can degrade as the data around it moves. This is why the documented deployments pair the tool with ongoing review rather than a one-time validation — the monitoring posture that separates a demo from a dependable system. None of it shows up in a benchmark score, and all of it decides whether an agent that reads the record correctly on Monday still reads it correctly six months later. Building the agent is the smaller half of the work; keeping it correct in a moving record is the larger one.
How to read this evidence
Three cautions apply. The benchmarks are young and use different task sets, so a score from one does not transfer cleanly to another or to your record; read each for what it actually measured. "Agent" spans a wide range here — from an advisory prompt to a tool-using retriever — and the read/reason/act frame is more informative than the label. And a single real-hospital study, however careful, is one site: the near-perfect simple-task result is encouraging and not yet a general claim. We revisit this guide on a 180-day cycle and whenever a new benchmark or a major EHR vendor's agent specification lands.
Sources and method
This guide synthesizes the EHR-agent evidence base: the reference virtual-EHR benchmark 1, a FHIR question-answering benchmark 2, a real-hospital Model Context Protocol study 3, an open-source MCP-FHIR integration framework 4, a real-world decision-support deployment 5, an agent-cost benchmark 6, the SMART App Launch standard that governs access 7, and the automation-bias literature behind the oversight caution 8. Every figure is drawn from the source cited beside it, and each citation was checked to resolve before publication. For the wider adoption picture, see our AI in healthcare statistics.