Fine-tuning

Fine-tuning is further training of an already pretrained model on a smaller, task- or domain-specific dataset, which changes the model's weights. It is one of three common ways to adapt a general model to clinical work. The others are in-context learning, which puts examples in the prompt and leaves the weights untouched 1, and retrieval-augmented generation, which adds trusted sources at answer time.

Why does fine-tuning matter in healthcare?

Most clinical language tools start as a general foundation model. Fine-tuning is the heaviest way to specialize one. Chat assistants are themselves products of it: InstructGPT was built by fine-tuning GPT-3 first on human-written demonstrations and then with reinforcement learning from human feedback, because "making language models bigger does not inherently make them better at following a user's intent" 2.

Clinical models followed. Med-PaLM used instruction prompt tuning, a parameter-efficient approach that aligns a model to a new domain with a few examples 3. Med-PaLM 2 combined a stronger base model, medical domain fine-tuning, and new reasoning strategies to reach 86.5% on the MedQA exam benchmark 4; the wider story is in clinical LLM.

How does it work in practice?

Full fine-tuning updates every weight, which is costly for large models. Parameter-efficient methods train a small add-on instead. The best known, LoRA, cut trainable parameters by 10,000 times and GPU memory by 3 times compared with fully fine-tuning GPT-3 175B, with quality on par or better in the authors' tests 5.

A clinical study shows the trade-offs. Van Veen and colleagues adapted eight language models to summarize radiology reports, patient questions, progress notes, and doctor-patient dialogue, using either in-context learning or QLoRA, a quantized form of LoRA 1. Three findings stand out:

• Med-Alpaca, a model instruction-tuned on medical question-and-answer text, did worse at clinical summarization than its general parent. The authors called this "a distinction between domain adaptation and task adaptation" 1.
• Given enough in-context examples, most models surpassed the best QLoRA fine-tuned model. The best configuration overall was GPT-4 with in-context examples 1.
• In a reader study with 10 physicians, summaries from the best-adapted models were judged equivalent (45%) or superior (36%) to those from medical experts in most cases 1.

Common misunderstandings

A medical fine-tune beats a general model. The Med-Alpaca result says otherwise for summarization 1. On MedQA, prompting GPT-4 with the Medprompt strategy passed 90% and beat the best prior results from specialist models 6.

Fine-tuning is a one-time step. A fine-tuned model is a new model. When the base model or the data changes, the fine-tune has to be redone and revalidated. For a regulated device, retraining is a modification that can require a new marketing submission; an authorized predetermined change control plan lets pre-specified changes ship without one 7.

Our own notes are safe training data. Patient records used for training bring privacy and governance obligations of their own; see de-identification vs anonymization.

What to ask a vendor

• Is the model fine-tuned, prompted, retrieval-grounded, or a mix? On what data?
• Did the training data include patient records? Under what de-identification and agreements?
• How does the fine-tuned model compare with the base model plus good prompting on our task?
• When the base model is updated, do you re-tune and revalidate, and is that covered by a PCCP?
• Can we run the model on our own infrastructure? See open vs closed models for hospital deployment.

Related terms

• Foundation model: the pretrained base that fine-tuning adapts.
• Retrieval-augmented generation: adaptation at answer time instead of training time.
• Context window: what limits how many in-context examples a model can take.