Contrastive Learning

Most supervised learning requires labelled data: for each input, a human has specified the correct output. Labelling is expensive, time-consuming, and limited by human capacity. Contrastive learning is a powerful alternative that sidesteps this requirement by defining a training signal from the structure of the data itself: similar things should produce similar representations; dissimilar things should produce different ones.

The core contrastive learning loop works as follows. For each example, create two "views" of it - two different augmented or transformed versions that both represent the same underlying content. For images, these might be random crops of the same image. For text, they might be two sentences from the same paragraph, or two paraphrases of the same statement. These two views form a positive pair - things that should be close together in representation space. Negative examples are other items in the batch, which should be far away.

The training objective then pushes positive pairs close together in the model''s representation space and pushes negative pairs apart. The model cannot do this by copying raw inputs (since the two views are different) - it must learn to extract the underlying invariant content that both views share. That extraction process produces rich, meaningful representations that generalise well to downstream tasks.

In language models, contrastive learning has been applied to train powerful embedding models. SimCSE, a particularly influential technique, creates positive pairs by running the same sentence through the model twice with different dropout masks - two views that differ only by which neurons are randomly dropped during each pass. The model learns representations that are consistent across dropout, which turns out to produce excellent sentence embeddings for semantic search and similarity tasks.

Contrastive learning has also influenced preference learning in alignment. The idea of training a model to distinguish preferred from non-preferred outputs - pulling good responses toward the model and pushing bad responses away - shares mathematical structure with contrastive objectives. RLHF''s reward model is in some ways a contrastive learner: it learns to score outputs such that preferred ones score higher than rejected ones.