Overfitting
When a model learns its training data too well and fails to generalise - the central challenge of machine learning.
Added May 21, 2026 · 2 min read
Overfitting is why you cannot simply make models more accurate by training them longer or making them bigger. A model that reports high training accuracy but fails on real data is useless and potentially dangerous. Understanding overfitting explains why machine learning practitioners always evaluate models on separate test sets, why train/validation/test splits are standard practice, and why regularisation is a fundamental part of the training toolkit.
The goal of machine learning is generalisation: building a model that performs well on data it has never seen, not just on the data it was trained on. Overfitting is the enemy of generalisation. An overfit model has memorised its training data rather than learning the underlying patterns, and its performance degrades badly on new examples.
The concept becomes intuitive with an extreme case. Imagine a student who memorises every question and answer from past exams without understanding the underlying material. They score perfectly on practice tests but fail on an exam with new questions. A machine learning model can do the same: achieve very low training loss by memorising training examples, then fail on new data because it has learned the specific noise and idiosyncrasies of the training set rather than the signal.
Overfitting happens because neural networks are highly expressive - they have enough parameters to memorise large datasets if given the chance. The more complex the model and the smaller the dataset, the higher the risk. Conversely, very large datasets tend to mitigate overfitting: with billions of training examples, the model has to learn general patterns because no specific example will appear often enough to be worth memorising.
Several techniques reduce overfitting. Regularisation adds a penalty to the loss function for large parameter values, discouraging the extreme specific fits associated with memorisation. Dropout randomly disables nodes during training, preventing the network from becoming dependent on any single path. Early stopping halts training when performance on a held-out validation set starts to degrade. Data augmentation artificially expands the training set by creating modified versions of existing examples.
The deep learning era has complicated the classical understanding of overfitting. Very large neural networks often memorise training data and still generalise well - a phenomenon called benign overfitting that does not fit the classical theory. Research into when and why this happens is ongoing.
Analogy
The difference between a friend who understood a film well enough to explain it in many different ways versus one who memorised specific lines from the review and repeated them back. The first has generalised; the second has overfit. When you ask a different question about the film, the first can adapt; the second cannot.
Real-world example
A medical image classifier trained on a small dataset from one hospital may overfit to the specific imaging equipment, patient demographics, or processing artefacts at that hospital. Deployed at a different hospital with different equipment, its performance may drop sharply - not because the task changed, but because the model learned features specific to the training environment rather than the actual diagnostic signals.
Why it matters
Overfitting is why you cannot simply make models more accurate by training them longer or making them bigger. A model that reports high training accuracy but fails on real data is useless and potentially dangerous. Understanding overfitting explains why machine learning practitioners always evaluate models on separate test sets, why train/validation/test splits are standard practice, and why regularisation is a fundamental part of the training toolkit.
In the news
No recent coverage - search for Overfitting.
Related concepts
Gradient Descent
The algorithm that trains neural networks - iteratively adjusting parameters in the direction that reduces the model's error.
Loss Function
The measure of how wrong a model's predictions are - the signal that training uses to decide how to improve.
Machine Learning
A way of teaching computers by showing them examples, rather than writing explicit rules - the engine behind almost everything labelled AI today.
Training
The process of teaching an AI model by adjusting its internal parameters until it gets better at its task - the computational work that creates intelligence.