Memory poisoning: defending an agent's memory in production
How to defend a production agent's long-term memory and retrieval store against poisoning — validation, provenance, segmentation and expiry that stop false data biasing future decisions.
Definition
Memory poisoning is an attack that corrupts an agent's long-term memory, embeddings or retrieval store, so that false or malicious content is later recalled as trusted context. Because the payload persists across sessions, a single poisoning event can bias many future decisions.
Most agent attacks last one run. Memory poisoning lasts much longer: it plants something in the agent’s memory that is recalled, as trusted context, again and again — long after the attacker has gone.
Why persistence changes everything
A prompt injection is dangerous in the moment. Memory poisoning is dangerous over time. Once false or malicious content lands in long-term memory, embeddings or a retrieval store, the agent treats it as established context on every future run that retrieves it. One poisoning event becomes a standing bias.
That makes it hard to detect. The original attack may be weeks in the past; what you see is an agent that subtly, consistently makes a particular wrong call — and the cause is buried in a store you rarely inspect.
How the persistent attacks compare with the one-shot kind:
| Attack | How long it lasts | Where it lives | When you notice |
|---|---|---|---|
| Prompt injection | A single run | The current context | Often immediately |
| Tool poisoning | Until the tool is fixed | The tool layer | When outputs look wrong |
| Memory poisoning | Across sessions, indefinitely | Long-term memory or RAG store | Weeks later, if at all |
How does content get poisoned?
- Unvalidated writes. The agent summarizes a poisoned document and stores the summary — payload included.
- RAG store contamination. Attacker-influenced data enters the retrieval index and is later surfaced as relevant, trusted context.
- Manipulated tool outputs that the agent decides are worth remembering, carrying tool poisoning into permanent memory.
The common thread: the memory write path is treated as trusted when it should be treated as untrusted.
Defenses
- Validate on write. Apply the same scrutiny to anything entering memory as to any other untrusted input — do not store raw, attacker-reachable content as fact.
- Segment by trust level. Keep untrusted-source memory separate from trusted knowledge, so retrieval cannot silently promote one to the other.
- Expire and review. Give long-lived entries a lifetime; periodically audit what the agent has chosen to remember.
- Attribute and trace. Record where each memory came from. When behavior drifts, provenance lets you find the poisoned entry — this is the memory side of observability.
- Least privilege on memory. Not every agent or tool should be able to write to long-term memory.
The principle
Memory is an attack surface, not a safe place. Anything an agent stores can be recalled as truth later, so guard the write path as carefully as the read path. This is one of the quieter risks in the OWASP Top 10 for Agentic Applications, and one of the hardest to notice once it lands. Terms are in the glossary.
Frequently asked questions
Why is memory poisoning especially dangerous?
Because it persists, where most agent attacks do not. A prompt injection affects a single run and is gone when the context clears; memory poisoning plants content that the agent recalls as trusted context across many later runs, often long after the attacker has left. That turns one event into a standing bias: the same false fact can quietly skew decisions for weeks. Persistence also makes it unusually hard to detect, because by the time you notice the agent consistently making a particular wrong call, the original attack is buried in a store you rarely inspect — a vector database, an embeddings index, a long-term memory table. Unlike a live injection, there is no moment of attack to catch; there is only the slow drift of an agent that has quietly learned something false and treats it as established knowledge.
How does poisoned content get into memory?
Through any write to long-term memory or a retrieval store that happens without validation. The most common path is an unvalidated summary: the agent reads a poisoned document, summarises it, and stores the summary with the payload still inside. Retrieval-augmented generation stores are a frequent target, because attacker-influenced data that lands in the index is later surfaced as relevant, trusted context. Manipulated tool outputs are another route — if the agent decides a crafted response is worth remembering, tool poisoning becomes permanent. The unifying problem is that the memory write path is usually treated as trusted when it should be treated as untrusted: whatever can reach the agent's input can, by the same token, reach its memory. Anything the agent is allowed to remember without scrutiny becomes a candidate for tomorrow's wrong decision.
What is the core defense?
Guard the write path with the same suspicion you apply to any untrusted input, because what goes into memory is what comes back out as fact. Validate content on write rather than storing raw, attacker-reachable text. Segment memory by trust level so that material from untrusted sources cannot silently become trusted context during retrieval. Give long-lived entries an expiry and review them periodically, so a poisoning does not live forever by default. Record provenance for every entry: when behaviour drifts, knowing where a memory came from is what lets you find and remove the poisoned one. Finally, apply least privilege to the memory itself — not every agent or tool should be able to write to a long-term store. None of these detects a clever payload directly; together they ensure that even if something false gets in, it is contained, attributable and removable rather than permanent and invisible.
How do I detect memory that is already poisoned?
Detection is hard precisely because the symptom is subtle and delayed, so you rely on provenance and review rather than catching the attack live. The practical signal is behavioural drift: an agent that has started, consistently, to make a specific wrong call or cite a specific false fact it did not before. When you see that pattern, provenance metadata is what makes the search tractable — if every memory entry records its source, timestamp and the run that wrote it, you can trace the bad behaviour back to the entry that introduced it. Periodic audits of what the agent has chosen to remember catch poisoning before it does much damage, especially for high-trust stores. Treat unexplained, persistent changes in agent judgment as a memory-integrity incident, not just a quality regression, and inspect the retrieval store the same way you would inspect logs after a breach.