Training

Training is the core activity in machine learning: the process by which a model becomes useful. It is iterative, compute-intensive, and the results depend critically on the data and choices made during it.

A model begins with randomly initialised parameters - numbers that define its behaviour but start out producing nothing useful. Training changes these numbers. The process works through a feedback loop: the model makes a prediction, the error of that prediction is computed against the correct answer, and the parameters are adjusted slightly in the direction that would have produced a less wrong answer. This adjustment happens via gradient descent. Repeat this across millions or billions of examples and the model's predictions improve.

Training happens across epochs - complete passes through the training dataset - and in batches - small subsets of the data processed together to estimate the gradient efficiently. The learning rate controls how large each adjustment step is: too large and the model overshoots; too small and training takes forever or gets stuck.

Large language models are trained in stages. Pre-training is the longest and most expensive phase: the model is trained on enormous quantities of text (often trillions of words) to predict what comes next. This can require months of compute on thousands of specialised chips, at costs reaching hundreds of millions of dollars. Fine-tuning follows, adjusting the pre-trained model on smaller, curated datasets to produce specific behaviours - like following instructions or declining harmful requests.

Once training is complete, the parameters are frozen. The model is deployed for inference - answering questions, generating text, classifying inputs - without further learning. This distinction between training time and inference time is fundamental to understanding how AI systems work and where their knowledge comes from.