AI Agent Memory and Context: What an Agent Remembers and What It Doesn't

An AI agent does not remember the way a person does. It remembers whatever you put in front of it at each moment and whatever you deliberately chose to store. Everything else is forgotten the instant the step ends. Confusing those two things — what the model has in view right now versus what the operation keeps between cases — is behind half the scares with agents: systems that “forget” what they were told ten minutes ago, or that “remember” a personal detail that should have been deleted.

For anyone weighing whether to put an agent to work, this is not a technicality. What an agent remembers and what it does not is a design decision with business and risk consequences. It is worth understanding the two memories separately before deciding what your operation should keep.

The two memories you have to tell apart

The context window is what the model has in front of it while it works: the instruction, the case data, the documents you handed it, the steps taken so far on this same task. It is working memory, and it is finite. It has a size limit, and when the conversation or the case runs long, the earliest parts stop fitting. The model does not “know” it has forgotten. It simply answers with what it still has in view. When the task ends, that window is discarded. By default, the agent starts the next case blank.

Persistent memory is what you choose to store outside that window to retrieve later: the state of an open case, a customer’s preferences, a rule that was set, the history of what has already been done. It is not something the model has by nature. It is a layer built around it — a database, a store, an index — from which the right pieces are brought into the context window when they are needed.

The practical distinction: the context window is working memory, limited and ephemeral; persistent memory is archival memory, deliberate and governed. A serious agent uses both, and the design question is what goes in each.

	Context window	Persistent memory
What it is	What the model has in view now	What is stored outside to retrieve later
Duration	Discarded when the task ends	Persists across cases and sessions
Limit	Finite: not everything fits	Whatever you decide, with its governance cost
Typical risk	”Forgets” what matters as it fills	”Remembers” what it should not keep

Bringing information from a document corpus into that window to answer is a neighboring but different pattern — that is enterprise RAG: RAG retrieves from a stable archive of documents. Memory keeps what the agent lives case by case. They get confused because both “put data into the context”, but one queries a manual and the other recalls its own history.

How to design what an operation should remember

This is where the work stops being technical and becomes a business matter. “Remember everything” is the lazy answer and the wrong one. A memory that stores it all fills with noise, makes every lookup more expensive, and accumulates data you later have to justify. The discipline is deciding, for each process, what deserves to be kept. In my deployments I separate three kinds.

Case state. A process that lasts more than one step needs to know where it stands: what has been requested, what has been resolved, what is pending, what it is waiting on. Without this the agent cannot resume a half-finished case, and any real operation is full of half-finished cases. It is the most obvious memory and the most neglected.

Rules. The policies, limits and criteria governing how decisions are made do not change case by case, but they have to be available in every one. This is less “memory” in the sense of history than the explicit definition of the operation, and it is one of the reasons I insist on treating the operating model as code, because the rules an agent applies should be written and versioned, not scattered in the head of whoever used to know them.

History. What already happened with this customer, this supplier, this file. It is at once the most useful and the most delicate: useful because it gives continuity, delicate because it accumulates data — often personal — whose retention has to be justified and limited.

The rule I apply is easy to state and hard to hold. Keep what a well-run operation would need to remember to give continuity and be accountable, and no more. Everything stored “just in case” is cost and risk waiting to materialize. How this layer assembles with the rest — model, orchestration, connections — I develop in AI agent architecture, and how an agent reaches the systems where part of that state lives, in the Model Context Protocol (MCP).

The risks: poisoned memory and retained data

Persistent memory introduces two risks the ephemeral context window does not have, and neither governs itself.

Poisoned memory. If an agent stores as true something that was false — a wrong fact, an instruction slipped into a document it processed, a badly drawn conclusion — that error stops being an isolated slip and becomes a premise it drags into every subsequent case. Worse still, because it comes from its own memory, the system treats it with more confidence than a fresh input. A memory without validation is a channel through which an error, or a deliberate attack, becomes permanent. That is why what goes into persistent memory deserves the same scrutiny as an agent’s action on a system: it is part of the security surface, a matter I cover in AI agent governance.

Retained personal data. A memory that accumulates history almost always accumulates personal data: names, emails, what a customer said, the content of a case. That falls squarely under data protection obligations: a legal basis to keep it, a time limit, the person’s right to have it deleted, control over who accesses it. An agent memory built without thinking about this is a compliance problem waiting for an audit. I develop it in AI agents and data privacy. Here it is enough to set the principle: retention is a decision you justify, not a side effect you tolerate.

Well-designed memory, an operation that holds

I gather it all into one idea I repeat in every boardroom: an agent’s memory is not a feature that ships from the factory and gets switched on, it is a design decision you make about your operation. What it remembers defines whether the agent gives continuity or repeats questions, what it forgets defines whether it drags errors or starts clean, and what it retains defines your legal exposure. Designing it well is part of the specific work of encoding your operation into agents that work, the same work I describe in the pillar on AI agents for business. An agent with good memory is not the one that stores the most, but the one that stores exactly what its work requires and can account for it.

Frequently Asked Questions

What is the difference between the context window and memory in an agent?

The context window is what the model has in view while solving a task: instruction, case data, steps taken. It is finite and discarded when the task ends. Persistent memory is what you choose to store outside that window — case state, rules, history — to retrieve in future cases. One is working memory. The other is archival.

Does an AI agent remember previous conversations?

Only if it was designed to. By default, the agent starts each task with no memory of earlier ones. Remembering across sessions requires building a persistent memory layer that stores what matters and brings it back when relevant. It is not automatic model behavior, but something you decide and build.

What is poisoned memory?

It is when an agent stores as true something false — a wrong fact, a malicious instruction slipped into a document, a badly drawn conclusion — and that error contaminates every subsequent case, because the system trusts its own memory. You prevent it by validating what enters memory and treating that input as part of the security surface.

Does an agent’s memory affect data protection?

Yes, considerably. A memory that keeps history usually accumulates personal data, and that requires a legal basis to retain it, a time limit, access control, and the ability to delete it if the person asks. Retention has to be a justified decision, not a byproduct of “storing everything just in case”.