Algorithmovigilance is the set of scientific methods and activities for evaluating, monitoring, understanding, and preventing adverse effects of algorithms used in healthcare 1. Modeled on pharmacovigilance in drug safety, it treats every deployed clinical algorithm as something to be watched continuously after deployment — for degraded, unsafe, or inequitable performance.
Why it matters in healthcare
Clinical algorithms fail quietly. A drug that harms a patient produces a visible adverse event; a risk model that silently under-triages a subgroup produces statistics no single clinician can see. Peter Embi proposed the term in 2021 precisely because health systems had begun running predictive models at scale with nothing equivalent to the surveillance infrastructure drugs get 1. A 2024 analysis pushed the analogy further, mapping pharmacovigilance concepts — signal detection, causality assessment, incident reporting — onto deployed AI systems 2.
Two published failures anchor the field. An external validation of a widely implemented proprietary sepsis model at one academic health system, on 38,455 hospitalizations from 2018–2019, found a hospitalization-level AUC of 0.63 and showed the model generated alerts for 18% of all hospitalized patients while missing 67% of those with sepsis 3. And a 2019 analysis of a care-management algorithm applied to millions of patients found that, at a given risk score, Black patients were considerably sicker than White patients — because the model used healthcare cost as its proxy for need 4. Both models had passed their developers' internal checks. Both failed in ways only systematic surveillance could catch.
How it works in practice
Operationally, algorithmovigilance combines several activities: baseline external validation before go-live, ongoing dashboards tracking discrimination, calibration, and alert burden, subgroup breakdowns to surface inequity, detection of model drift as populations and data pipelines change, and a route for clinicians to report suspected algorithm-related harm 2. The pharmacovigilance parallel supplies the missing machinery: drug safety has decades of infrastructure for spontaneous adverse event reporting, signal detection across institutions, and graded responses from label changes to withdrawal — and each of those has a plausible analogue for deployed algorithms, from incident registries to model recalls 2. When a monitored model is a regulated device with a predetermined change control plan, monitoring results feed the pre-agreed criteria that govern retraining.
Where it appears today
The term is moving from viewpoint pieces into infrastructure. Vanderbilt University Medical Center designed the Vanderbilt Algorithmovigilance Monitoring and Operations System (VAMOS) through participatory design sessions, publishing the work in 2025 and noting that few systems yet support systematic monitoring and governance of AI across a health system 5. The design process distilled the interfaces such a platform needs — a performance monitoring dashboard, model-specific pages, and snapshot views for governance review 5. On the policy side, the ONC's HTI-1 final rule, published January 2024, revised certification criteria for decision support interventions — requiring certified health IT developers to disclose standardized transparency information about the predictive algorithms they ship 6. Regulatory expectations for post-market AI monitoring worldwide are tracked in our global AI-in-health regulation tracker.
Common misunderstandings
Vigilance duplicates validation. Validation is a point-in-time event; vigilance is the standing process that catches what validation cannot — drift, new populations, upstream data changes 2.
It concerns accuracy alone. Embi's framing explicitly targets effectiveness and equity: a model can hold its overall AUC while failing a subgroup, which is exactly what the 2019 bias case demonstrated 14.
It is the vendor's job. Post-market obligations sit with manufacturers, but local performance depends on local data, workflows, and populations — which is why health systems are building their own monitoring capacity 5.
Related terms
See model drift for the main phenomenon vigilance detects, external validation for the pre-deployment counterpart, and predetermined change control plan for how monitoring connects to authorized model updates.