Evaluation

Prospective vs retrospective evaluation

A model that looks brilliant on curated historical data has cleared the lowest bar, not the last one. This is a working guide to prospective versus retrospective evaluation — what each can prove, the documented cases where retrospective promise did not survive prospective testing, and the one where it did. As of July 2026.

By Jonas WeirReviewed by Jonas Weir · editorial reviewUpdated

The short version

  • Retrospective evaluation runs a model on historical, often curated data; prospective evaluation runs it forward on new patients in the real workflow. The first is a starting point; the second is where most tools are made or broken.
  • In breast-screening AI, no prospective accuracy studies existed, and where AI was compared with real radiologist decisions, 94% of systems were less accurate than a single radiologist — even though laboratory test-set studies claimed the opposite.
  • A deep-learning eye-screening system that scored at specialist level on curated images hit friction in eleven Thai clinics, where real-world image quality collided with the model's data-quality thresholds.
  • The base rate of prospective evidence is thin: one review found only 10 registered deep-learning RCTs with two published; another found 41 RCTs with a median of just 294 participants.
  • Prospective testing can confirm as well as refute: a randomised trial of 1,058 colonoscopies raised the adenoma detection rate from 20.3% to 29.1% with real-time AI.

The most common way an AI result misleads is timing. A model is trained and then tested on data that already exists — historical, tidy, often hand-curated for the study — and it posts a number that looks like proof. What it has actually earned is a promising retrospective result, which is the lowest bar in clinical evaluation, never the last one. Whether that promise survives depends on a different kind of study: a prospective one, run forward on new patients inside the real workflow. This guide sets out what each design can and cannot show, walks through documented cases where retrospective promise did not hold prospectively, and ends with one where it did. As of July 2026.

What each design can prove

Retrospective evaluation is fast, cheap, and repeatable. You take existing data, apply the model, and compute accuracy. Its weakness is baked into its convenience: historical datasets are usually curated. Poor-quality images are excluded, labels are cleaned, the case mix is whatever was easy to assemble, and the model never has to cope with an input it cannot handle. Every one of those conveniences flatters the result.

Prospective evaluation runs the model forward on patients as they arrive, in the setting and workflow where it will be used. It is slower and costlier, and it exposes everything the curated study hid: ungradable inputs, workflow friction, shifting prevalence, and the human response to the model's output. This is the timing counterpart to the place question covered in our guide on internal versus external validation — and a model needs to clear both. The reporting standards treat the move from retrospective, "silent" testing to live prospective use as its own stage of evidence: DECIDE-AI exists specifically as the reporting guideline for the early stage clinical evaluation of decision-support systems driven by artificial intelligence 6.

The mammography ledger

The cleanest illustration that retrospective and prospective evidence can point in opposite directions comes from breast-cancer screening. A systematic review of test-accuracy studies gathered every eligible study of AI in mammography and sorted them by design. Its first finding was an absence: no prospective studies measuring the test accuracy of AI in screening practice were found, and the studies that did exist were of poor methodological quality 1.

The contrast inside the review is the lesson. Five smaller laboratory studies — AI reading a test set of images in isolation, at high risk of bias — reported that all five evaluated AI systems were more accurate than a single radiologist. But the studies that compared AI against radiologists' real clinical decisions on screened women told the opposite story: 34 (94%) of 36 AI systems were less accurate than a single radiologist, and every one was less accurate than the consensus of two or more 1. The laboratory setup — clean images, no workflow, a handicapped human comparator — produced exactly the flattering result retrospective designs are prone to. The evaluation closer to practice reversed it.

DesignSettingVerdict on AI
Laboratory test set (5 studies, high bias) 1Curated images, isolated readAll 5 systems beat a single radiologist
Comparison with real radiologist decisions 1Screening records94% of 36 systems worse than a single radiologist
Prospective accuracy in screening practice 1Live programmeNo studies existed

Why did the laboratory studies flatter the AI so badly? Two mechanisms, both characteristic of retrospective design. First, the comparator was handicapped: a single radiologist reading an isolated image on a test set has none of the prior notes, priors, and consultation a working radiologist draws on, so the human baseline is artificially low. Second, the image sets were curated — enriched with clearer cases and cleaned of the ambiguous, poor-exposure films that fill a real programme — which inflates any reader's apparent accuracy and shifts what a given threshold means. Strip both artefacts away by comparing against real clinical decisions, and the ranking flips. The retrospective number was measuring the study design as much as the model.

The field-deployment gap

Numbers on a page also hide the physical reality of a clinic. A deep-learning system for diabetic eye disease — the kind of tool that scores at specialist level on curated retinal images — was studied prospectively across eleven clinics in Thailand, with researchers observing nurses and patients rather than only counting correct classifications. The evaluation surfaced a gap that no retrospective run could: in the authors' words, there were "tensions between the model's thresholds for data quality, and the quality of data that arise from an imperfect, resource-constrained environment" 2. Images that a curated dataset would have quietly excluded were the everyday output of real clinics with variable lighting and equipment, and the system's insistence on high-quality inputs collided with the pace and constraints of frontline work. The model was capable; the deployment was where capability met friction.

The sepsis model

Sometimes the prospective reckoning arrives after the tool is already switched on at scale. A proprietary sepsis prediction model, embedded in a widely used electronic health record and enabled at hundreds of hospitals largely on internal figures, was validated by independent researchers on more than 27,000 real patients. It predicted sepsis onset with an area under the curve of 0.63 and, at the alert threshold hospitals actually ran, missed 1,709 patients with sepsis — 67% of cases — while firing frequently on patients who never developed it 3. The ordering was backwards: population-scale deployment came first, independent forward-looking evaluation second. Post-deployment monitoring of exactly this kind is the discipline our glossary entry on algorithmovigilance describes.

When prospective evidence confirms

None of this means retrospective promise always evaporates. The point of prospective evaluation is to find out, and sometimes the answer is yes. In a prospective, randomised, controlled trial, 1,058 patients undergoing colonoscopy were assigned to a standard procedure or to one assisted by a real-time deep-learning polyp-detection system. The AI arm raised the adenoma detection rate from 20.3% to 29.1% — a large, statistically significant improvement in the measure that matters for preventing colorectal cancer 8. The study had limits its authors named — it was unblinded, single-country, and the extra detections were weighted toward small lesions — but it did the essential thing: it tested the tool forward, on real patients, against a randomised control, and the benefit held. A retrospective claim graduated into prospective evidence.

The difference between this case and the failures above is instructive. The colonoscopy trial measured a clinically meaningful endpoint prospectively with a control group; the mammography laboratory studies measured accuracy on curated images with a handicapped comparator. Same technology family, opposite evidentiary weight.

How thin the prospective base rate is

Step back from individual cases and the structural problem appears: prospective, randomised evidence for clinical AI is scarce. A systematic review searching through mid-2019 found only 10 registrations for deep-learning randomised clinical trials, of which just two had been published 4. A later review that focused specifically on randomised trials of machine-learning interventions found 41 RCTs, with a median of 294 participants (range 17 to 2,488); 51% were single-site, no trial adhered to all CONSORT-AI reporting standards, and 38 (93%) did not assess poor-quality or unavailable input data — the very failure mode the Thai clinic study made visible 5. A review of translation challenges put the situation plainly: there are still limited examples of AI techniques successfully deployed into clinical practice, and randomised controlled trials should be viewed as the gold standard for evidence generation, even where running them is hard 7.

EvidenceWhat it countedFinding
DL RCT registrations to mid-2019 4Registered / published10 registered, 2 published
ML intervention RCTs 5Trials / median size41 RCTs, median 294 participants
ML intervention RCTs 5Adhered to all CONSORT-AINone
ML intervention RCTs 5Did not check input-data quality93%

The shape of that RCT evidence is as telling as its scarcity. In the same review, just over half the trials ran at a single site, and more than a third clustered in a single specialty — endoscopy — where a real-time detection task is unusually well suited to a trial. Overall risk of bias was high in about one in six trials, and none reported everything the CONSORT-AI standard asks for 5. So even the prospective evidence that exists is concentrated, single-site, and incompletely reported. A field cannot be judged mature on the strength of a few dozen small trials huddled around the one task that was easiest to test.

The bridge from silent to live

The way to avoid both the mammography trap and the sepsis ordering error is to treat evaluation as a staged path, and to know which stage a study sits on. Retrospective accuracy comes first. Then silent prospective testing, where the model runs live but no one acts on its output, so you can watch it meet real inputs without risk. Then live prospective evaluation, where clinicians see and respond to the output — the stage DECIDE-AI was written to structure and report 6. Most tools that fail do so at the transition into live use, because that is where human trust, workflow, and messy data all arrive at once. A model evaluated only retrospectively has yet to meet any of them.

How to read this

Three cautions travel with everything above. First, retrospective is a legitimate and necessary starting point — the failure is treating it as a verdict, never running the study at all. Second, prospective is stronger evidence but carries its own weaknesses: a prospective study can be small, single-site, unblinded, or measure a proxy rather than patient benefit, so read its design with the same scepticism, using our checklist for reading a validation study. Third, small prospective samples produce wide uncertainty; a median of 294 participants leaves confidence intervals broad enough to matter, which is the subject of our companion guide on sample size and confidence intervals. For where the field's prospective numbers stand overall, see our clinical AI trial results tracker, and for the related question of whether a result holds at a different place, see internal versus external validation and dataset shift and model decay.

Sources and method

This guide synthesises one systematic review that separates laboratory from real-decision breast-screening evidence 1, one prospective human-centered field study 2, one external-validation study used as a worked example 3, two reviews that quantify how scarce prospective randomised evidence is 45, one reporting guideline for early live evaluation 6, one review of translation challenges 7, and one prospective randomised trial used as a confirming counter-example 8. Every figure is drawn from the primary source cited beside it. We revisit this page on a 180-day cycle and whenever a governing guideline or a major new trial lands. For decisions that turn on regulatory clearance or reimbursement, confirm the current evidentiary requirements with your compliance or regulatory counsel before acting — a retrospective clearance record is a separate question from prospective clinical benefit.

Questions & answers

  • What is the difference between retrospective and prospective evaluation?

    Retrospective evaluation applies a model to data that already exists — historical records, often cleaned and curated for the study. Prospective evaluation runs the model forward on new patients as they present, inside the real clinical workflow. Retrospective work is faster and cheaper and is a reasonable first step; prospective work is where a model meets messy inputs, human responses, and the population it will actually serve.

  • Why do AI models that look strong retrospectively fail prospectively?

    Curated historical datasets remove much of what makes deployment hard: poor-quality images are filtered out, labels are clean, and the case mix is convenient. Prospectively, the model faces ungradable inputs, workflow friction, shifting prevalence, and clinicians who may over- or under-trust its output. Each of these can erode a headline number that looked settled on paper.

  • Does a prospective study guarantee a model works?

    It is much stronger evidence, but read the design. A prospective study can still be single-site, small, unblinded, or measure a proxy endpoint rather than patient benefit. One review of machine-learning trials found a median of just 294 participants and no trial meeting every reporting standard, so prospective is a necessary step rather than a finish line.

Sources

  1. Freeman K, Geppert J, Stinton C, et al. Use of artificial intelligence for image analysis in breast cancer screening programmes: systematic review of test accuracy. BMJ. 2021;374:n1872. doi.org/10.1136/bmj.n1872
  2. Beede E, Baylor E, Hersch F, et al. A Human-Centered Evaluation of a Deep Learning System Deployed in Clinics for the Detection of Diabetic Retinopathy. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 2020:1-12. doi.org/10.1145/3313831.3376718
  3. Wong A, Otles E, Donnelly JP, et al. External Validation of a Widely Implemented Proprietary Sepsis Prediction Model in Hospitalized Patients. JAMA Internal Medicine. 2021;181(8):1065-1070. doi.org/10.1001/jamainternmed.2021.2626
  4. Nagendran M, Chen Y, Lovejoy CA, et al. Artificial intelligence versus clinicians: systematic review of design, reporting standards, and claims of deep learning studies in medical imaging. BMJ. 2020;368:m689. doi.org/10.1136/bmj.m689
  5. Plana D, Shung DL, Grimshaw AA, Saraf A, Sung JJY, Kann BH. Randomized Clinical Trials of Machine Learning Interventions in Health Care: A Systematic Review. JAMA Network Open. 2022;5(9):e2233946. doi.org/10.1001/jamanetworkopen.2022.33946
  6. Vasey B, Nagendran M, Campbell B, et al. Reporting guideline for the early stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. BMJ. 2022;377:e070904. doi.org/10.1136/bmj-2022-070904
  7. Kelly CJ, Karthikesalingam A, Suleyman M, Corrado G, King D. Key challenges for delivering clinical impact with artificial intelligence. BMC Medicine. 2019;17:195. doi.org/10.1186/s12916-019-1426-2
  8. Wang P, Berzin TM, Glissen Brown JR, et al. Real-time automatic detection system increases colonoscopic polyp and adenoma detection rates: a prospective randomised controlled study. Gut. 2019;68(10):1813-1819. doi.org/10.1136/gutjnl-2018-317500