Loss Function

A neural network begins with random parameters and produces useless outputs. Training improves it. But "improve" needs to be operationalised: how do you measure, precisely, how wrong the model's current predictions are? The loss function is the answer.

The loss function takes the model's output and the correct answer and returns a single number - the loss, sometimes called the cost or error. A high loss means the model is very wrong. A low loss means it is close to correct. The goal of training is to minimise the loss across all examples in the training dataset.

For language models, the most common loss function is cross-entropy loss. The model predicts a probability distribution over all possible next tokens. Cross-entropy measures how much probability the model assigned to the actual next token. If the model confidently predicts "the" and the answer is "the," the loss is very low. If the model thought "cat" was most likely and the answer was "the," the loss is high. Every training step nudges parameters to assign more probability to the correct next token.

The choice of loss function is a design decision that shapes what the model learns to do. Change the loss function and you change what behaviour gets rewarded. Training a language model to be helpful requires not just cross-entropy on correct predictions but additional loss terms that penalise harmful outputs, reward human preferences, or encourage factual accuracy. The training techniques known as RLHF, DPO, and Constitutional AI are all fundamentally ways to modify the effective loss function so that model behaviour aligns with human values.

An important subtlety: a model that achieves very low loss on its training data is not necessarily good. It might be memorising training examples rather than learning general patterns - a problem called overfitting. The loss on a separate validation set is the more meaningful measure of whether the model will work in the real world.