Model drift is the gradual decline in a deployed clinical AI model's accuracy when the real-world data it sees no longer matches the data it was trained on. As patients, clinical practice, coding conventions, and instruments change, the model's predictions grow steadily less reliable — often without any visible warning. As of July 2026.
Why it matters in healthcare
Most machine-learning systems are validated once, at a single moment, on data already collected. Healthcare then keeps moving. New treatments change who ends up in hospital; a coding update relabels a diagnosis; a scanner is replaced; referral patterns shift with the seasons. Each of these can pull live data away from the training distribution. Researchers describe this formally as dataset shift — the situation where "a machine-learning system underperforms because of a mismatch between the data set with which it was developed and the data on which it is deployed" 1. Because clinical models inform decisions about real people, silent degradation carries direct patient-safety weight, which is why drift is treated as a governance problem rather than a modelling footnote.
How it works
Drift usually arrives through one of three doors. In covariate or data drift, the mix of inputs changes while their meaning holds. In label drift, the base rate of the outcome moves. In concept drift, the relationship between inputs and outcome itself changes. A study of emergency-department admission models captured the last two together, noting that "the underlying data distributions characterising episodes can change with time (data drift), and so can the relationship between episode characteristics and associated clinical outcomes" 3. Any of the three can degrade performance, and they frequently travel together.
Where it appears today
The clearest cautionary case is a sepsis-alerting model at a US academic health system: in April 2020 it "had to be deactivated because of spurious alerting" after COVID-19 changed patient characteristics beyond what the model had learned 1. Slower drift is just as real. Across nine years of Veterans Affairs data, seven acute-kidney-injury models kept their ability to rank patients but increasingly over-predicted risk — "discrimination was maintained for all models" while "calibration declined" as event rates rose 4. The failure was invisible to a discrimination-only check and obvious once calibration was tracked over time.
Common misunderstandings
Drift is the same as a one-time validation gap. It is a distinct problem. A widely deployed proprietary sepsis model scored an area under the curve of only 0.63 and missed 67% of sepsis cases when tested outside its origin site 2 — a spatial gap between development and a new setting, present from day one, rather than a temporal decline. Both matter, but they are diagnosed and fixed differently, and confusing them leads teams to re-validate when they should be monitoring.
Good discrimination means the model is fine. The kidney-injury work shows a model can keep its ranking ability while its probabilities drift out of alignment 4. Watching only an AUROC will miss it; the answer is to monitor calibration alongside discrimination.
Drift is a surprise. It is expected. FDA, Health Canada, and the MHRA jointly advise that "deployed models have the capability to be monitored" with controls to manage "degradation of the model (for example, dataset drift)" 5. The systematic version of that monitoring — continuous post-market surveillance of AI performance and equity — is now called algorithmovigilance 6.
Related terms
Drift is best understood alongside model calibration, which is often the first metric to slip; external validation, the spatial cousin of temporal drift; and algorithmovigilance, the discipline of catching both after deployment. For the wider picture of how far clinical AI has spread and where it strains, see our AI in healthcare statistics.