latentbrief
← Back to concepts

Agent Memory

The mechanisms by which AI agents store and retrieve information across tasks, sessions, and time - enabling them to learn from experience and maintain context beyond a single conversation.

Added May 21, 2026 · 2 min read

Without persistent memory, AI agents cannot improve through experience, cannot maintain continuity across sessions, and must rediscover context each time they are invoked. Memory is what separates a capable assistant from a capable tool.

Memory is one of the defining challenges for AI agents. A single conversation with a language model is bounded by its context window - everything the model needs to know must fit within that window. But real-world agents need to operate across many interactions, remember what happened in previous sessions, and accumulate knowledge over time.

Different types of memory serve different purposes. In-context memory is information held in the active context window - the conversation history, a document being worked on, tool outputs from the current task. This is temporary and lost when the session ends. External memory is information stored outside the model and retrieved when needed - databases, files, vector stores. Parametric memory is knowledge baked into the models weights during training, which cannot be easily updated without retraining.

For practical agents, the most important memory architecture is the combination of a retrieval system (usually vector search) with an external store. The agent can write notes, store observations, and summarise completed tasks into a persistent memory store. When starting a new task, it retrieves relevant memories using semantic search - finding things that are conceptually similar to the current situation rather than exact keyword matches.

Memory management introduces hard problems: what is worth storing, how to prevent the memory from growing without bound, how to handle contradictory memories, and how to ensure that retrieved memories are actually relevant rather than misleadingly similar. The quality of an agents memory system often determines whether it degrades gracefully over time or accumulates errors.

Analogy

How a professional manages their working knowledge: short-term notes for the current project (in-context memory), a filing system for past projects (external memory), and the accumulated expertise from their career that they dont consciously recall but apply automatically (parametric memory). Effective agents need all three.

Real-world example

A customer support agent might use external memory to store: a summary of each resolved ticket, the customers expressed preferences, and any workarounds discovered for recurring issues. When a customer contacts support again, the agent retrieves their history and picks up with appropriate context - rather than starting each interaction from scratch.

Why it matters

Without persistent memory, AI agents cannot improve through experience, cannot maintain continuity across sessions, and must rediscover context each time they are invoked. Memory is what separates a capable assistant from a capable tool.

In the news

Related concepts