Agent Memory Poisoning

Definition

Memory poisoning is the introduction of misleading or adversarial information into stored context that later influences an agent. It becomes especially consequential when an untrusted statement is promoted into a durable preference, policy or instruction without appropriate validation.

Persistent context can extend the effect of one encounter across many future tasks. The attacker may no longer be present when the harmful action occurs. Evaluation therefore needs multi-session scenarios and a clear account of how memory is written and trusted.

What it covers

Write path

Who can cause memory to change.

Classification

Fact, preference, instruction or unverified claim.

Reuse

Which future tasks can consume it.

Recovery

How contaminated entries and derivatives are corrected.

Not all memory deserves the same authority. A user preference may be editable by that user, while an organizational rule needs a controlled source. A remembered model inference should not silently acquire the status of an approved business fact.

In practice

Practical design patterns by Min.

• Require stronger validation for durable instruction-like memory.
• Retain source and trust labels through summarization.
• Test a malicious update followed by a separate later task.

Define write permissions, validation conditions and expiry by memory type. Keep proposed updates separate from accepted durable records where consequences warrant it. Make it possible to inspect, correct and invalidate memory without exposing unrelated private context.

Research notes

Source-grounded details and editorial applications. Follow each link for the original document.

Working example

Why the controls matter

The destination example is a permission change disguised as helpful information. A later task that reads the remembered address might otherwise send data confidently. The important control is the promotion boundary between encountered content and authorized operational settings.

Common pitfalls

• Checking only the current conversation for injection.
• Letting summaries erase the distinction between quoted instructions and policy.
• Deleting one bad entry while derived memories persist.

Memory validation can also reject legitimate updates too aggressively. Track both harmful acceptance and useful-update failures. A recovery plan should identify affected tasks and derivatives, not merely reset the visible chat and assume the influence has disappeared.

Questions for operational review

Use these questions to decide what evidence your team needs. They are editorial prompts, not a compliance checklist.

• Who can write durable instruction-like memory?
• Can a summary remove source trust labels?
• How far can one poisoned entry spread?