Domain Adaptation
Fine-tuning a general model on data from a specific industry or subject area - the step that turns a broadly capable AI into one that speaks the language of your field.
Added May 18, 2026 · 3 min read
Most high-value professional applications of AI require domain adaptation. A general model is rarely good enough for specialised professional use, but training from scratch on domain data is usually not cost-effective. Domain adaptation provides the practical middle path - retaining general capability while gaining the domain fluency that makes a model actually useful for specialist work.
A model trained on general internet text knows about medicine the way a well-read non-expert knows about medicine: it can discuss concepts, explain terminology, and reason at a general level. But it has never absorbed the dense, precise, jargon-rich text that medical practitioners actually use - clinical notes, drug trial protocols, pathology reports, specialist guidelines. It does not have the intuitions that come from deep domain immersion. Domain adaptation is the process of closing that gap.
Domain adaptation typically proceeds in two stages. First, continued pre-training on domain-specific text: the model is trained on a large corpus of in-domain material - medical literature, legal contracts, financial filings, code repositories - using the same next-token prediction objective as original pre-training. This updates the model''s representations to reflect the statistical patterns, vocabulary, and reasoning styles of the target domain. Second, fine-tuning on domain-specific labelled tasks: question answering over clinical notes, contract clause classification, financial entity extraction. This adapts the model to the specific outputs needed in the domain.
The first stage is particularly powerful for domains with highly specialised vocabulary. General models handle common medical terms adequately, but rare drug names, anatomical terms, procedure codes, and specialist abbreviations may not be well represented in general training data. Continued pre-training on medical text re-calibrates the model''s statistical intuitions toward medical language patterns, improving its handling of these specialised elements.
The amount of domain-specific data needed varies. For domains that are genuinely distinct from general internet text - clinical notes are written in highly abbreviated, non-standard language that general models find opaque - even a few million tokens of continued pre-training can make a significant difference. For domains that are well represented in general training data (mainstream law, standard financial reporting), the benefits of domain adaptation are more incremental.
Domain-adapted models often outperform both the general model they are based on and models trained exclusively on domain-specific data - the combination of broad world knowledge from general pre-training plus deep domain fluency from adaptation gives the best of both.
Analogy
A bilingual person who is fluent in everyday English and then immerses themselves for six months in a specialist subculture - the legal profession, medical practice, or academic research. They already understand English; the immersion period updates their vocabulary, idiomatic expressions, and intuitions to match the specialist community. Domain adaptation does this for language models.
Real-world example
BioBERT and PubMedBERT are domain-adapted versions of BERT trained on biomedical literature. On biomedical NER (recognising named entities like genes, diseases, and drugs) and question answering tasks, they substantially outperform general BERT, despite starting from the same architecture. The continued pre-training on PubMed abstracts and full-text articles updated the model's representations to capture biomedical language patterns that general text did not adequately encode.
Why it matters
Most high-value professional applications of AI require domain adaptation. A general model is rarely good enough for specialised professional use, but training from scratch on domain data is usually not cost-effective. Domain adaptation provides the practical middle path - retaining general capability while gaining the domain fluency that makes a model actually useful for specialist work.
In the news
No recent coverage - search for Domain Adaptation.
Related concepts
Fine-tuning
Taking a general-purpose AI model and giving it additional training on a specific subject, so it becomes noticeably better at that particular domain.
Instruction Datasets
Curated collections of instruction-response pairs used to fine-tune language models into helpful assistants - the training data that teaches models what being useful looks like.
Transfer Learning
The foundational practice of starting model training from pre-existing knowledge rather than from scratch - the reason fine-tuning a model costs far less than training one.