·5 min read·agent-memory

Agent Memory Is Infrastructure, Not a Feature

Most teams build an agent knowledge base and stop. Keeping it from rotting in production is the hard part. Agent memory is infrastructure, and infrastructure needs hygiene.

Most advice about giving an AI a knowledge base stops at building it. Collect the documents, embed them, point a model at the pile, done. The hard part is not building the knowledge base. It is running one in production for months without it quietly rotting.

Agent memory is infrastructure. And infrastructure nobody maintains does not stay neutral, it decays. A store that grows without discipline gets slower and less trustworthy the more you put into it. The notes are cheap. The discipline of how you keep and retrieve them is the whole game.

#More memory is not a smarter agent

The seductive idea is that a bigger context window or a fatter knowledge base makes an agent smarter. The research says the opposite once you cross a threshold.

The canonical result is Lost in the Middle (Liu et al.): models use information best at the very start and end of their context and measurably degrade when the relevant fact sits in the middle, even in models built for long context. Piling more in does not help if the model cannot reliably reach the part that matters.

It compounds in retrieval systems. Long-Context LLMs Meet RAG (Jin et al.) found that as you feed in more retrieved passages, answer quality improves at first and then declines, because the extra passages are mostly noise the model has to fight through. Databricks ran the experiment at scale and saw the same shape.

2,000+
experiments across 13 LLMs found that for most models, RAG quality decreases after a certain context size (Databricks, 2024)

This is not a 2023 finding that newer models have outgrown. Each model generation has reconfirmed it. A 2025 benchmark called NoLiMa showed 11 of 13 long-context models falling below half their short-context accuracy by 32K tokens. Even GPT-4o dropped from 99.3 to 69.7 percent. The same year, Chroma's "Context Rot" study reproduced the decline in current frontier systems like Claude Sonnet 4, GPT-4.1, and Gemini 2.5 Flash. And in January 2026, a fresh analysis found models collapsing by more than 30 percent once input crossed a critical length, even when every fact in the context stayed relevant. Bigger windows keep raising the ceiling. They have not removed the floor.

30%+
performance collapse once input crosses a critical length threshold, even when every fact stays relevant, in a January 2026 analysis of long-context degradation

The discipline that follows is simple to say and easy to skip: index first, then fetch only the slice the task needs. A bigger pile is not a better toolbox.

#Hygiene is a cadence, not a cleanup

Knowledge debt compounds quietly. A broken link here, a duplicated note there, a page that grew too big to retrieve cleanly. None of it hurts on the day it happens. All of it hurts six months later, when someone has to do a heroic weekend cleanup to make the system trustworthy again.

The fix is to stop treating maintenance as an event. You run it as a cadence: a light check on every change, a deeper sweep on a schedule. The rot never gets the chance to accumulate, because nothing is ever far from its last inspection.

#The system proposes, a human disposes

There is a strong temptation to let the maintenance automation also do the fixing: find the stale page, delete it; find the duplicate, merge it. Resist it.

Maintenance should surface a worklist, things to add, fix, or review, and stop there. It should not destructively mutate your knowledge on its own. The reason is the oldest rule in quality control: the thing that produced a change should never be the only thing that judges it. A sweep that can silently delete is a liability. One that can only recommend, and leaves the call to a human or a separate reviewer, is an asset.

#Provenance, or it does not count

Every load-bearing claim in the store should carry its source and a sense of how fresh it is. When a claim goes stale, the right move is to re-derive it from the source, not to trust the version you remembered.

This is the difference between a system that gets more confident over time and one that gets more correct. Confidence is not truth. A memory that cannot show where it came from is not knowledge, it is a rumor that has been repeated enough times to sound official.

#The real work

The teams that win with agents will not be the ones with the biggest memory. They will be the ones who treat memory like production infrastructure: retrieved with discipline, maintained on a cadence, never silently mutated, always sourced.

That is the part the demos skip, and it is the part that decides whether the system is still trustworthy a year from now.

We run this as a standing practice, not a one-time setup. Here is how we run agent memory in production.

#Sources