Concept
Latent Space
The internal numerical world where an AI model represents meaning - a high-dimensional space where similar concepts cluster together and mathematical operations on numbers produce meaningful semantic results.
Added May 18, 2026
When a language model processes text, it does not work with words and sentences in any form a human would recognise. Every piece of text - every token - gets converted into a long list of numbers called a vector. These vectors are not arbitrary; they are learned during training to encode meaning in a way that is useful for the model's tasks. The space defined by all these possible vectors is the latent space.
What makes latent spaces remarkable is their structure. Similar concepts end up geometrically close to each other. "King" and "queen" are near each other. "Paris" and "Rome" are near each other and both far from "banana." But the structure goes deeper than mere similarity. Directions in the latent space encode semantic relationships. There is a direction that points from countries toward their capitals, so adding the vector for that direction to "France" lands you near "Paris." There is a direction encoding gender, so subtracting "man" and adding "woman" moves you from "king" to somewhere close to "queen." These relationships emerge from training without anyone programming them in.
The latent space also encodes relationships between concepts across different domains. The structure of "big" versus "small" in physical objects turns out to be related to "expensive" versus "cheap" in products, because these scale relationships share underlying mathematical structure. This is part of why language models can reason by analogy: the semantic structure of the analogy is encoded as geometric relationships in the latent space.
Every major component of an AI system - embeddings, attention, the feed-forward layers - is doing operations in some latent space. Embeddings are a relatively shallow latent space of token meanings. The residual stream running through a transformer is a deeper, richer latent space where more abstract reasoning happens. Understanding this framework helps explain both what AI models are good at and why their failures sometimes have a distinctly "semantic" character.
Analogy
A map where distances represent conceptual similarity rather than physical distance. Cities close together on the map are conceptually related - medical terms cluster together, sports terms cluster together, programming terms cluster together. And just as on a real map, relationships can be expressed as directions: "northward" means "more abstract," "eastward" means "more technical." The latent space is this kind of semantic map, in hundreds of dimensions rather than two.
Real-world example
The word2vec demonstration that "king - man + woman = queen" was an early striking illustration of latent space structure. These relationships emerged purely from training on text prediction - the model had never been explicitly told anything about gender or royalty. The semantic structure encoded in the latent space was a byproduct of learning to predict language.
Why it matters
Latent space is the conceptual foundation for understanding what happens inside AI models. Interpretability research - the field trying to understand how AI models work - is largely an effort to make sense of the structure in latent space: what gets encoded where, how information flows between layers, and why models behave the way they do.
In the news
No recent coverage - check back later.
Related concepts