Memory and Context Management in Long Research Sessions

Long research sessions with AI agents fall apart even when the context window is huge. Token count alone doesn't explain why; the real culprit is what happens to a model's attention as that window fills, plus a hard ceiling on how many things any model can juggle at once. Fix that, and the token limit stops mattering nearly as much as everyone thinks. Back in early 2023, most models capped out around 4,000 to 8,000 tokens. By 2025, the leading models handle context windows anywhere from 128,000 tokens up to 2 million. On paper, that should've solved the "agent forgets everything" problem for good, but it didn't, and the reason is a thing people in this field call context rot: bigger windows don't fix long sessions, because the failure was never about window size. ## What "memory" actually means for an AI research agent Start with the awkward part: a stateless language model has zero memory by default, and every call starts from scratch. There's no thread connecting facts, no memory of what worked an hour ago, no record of the dumb mistake it made three steps back. Each prompt is a fresh amnesiac waking up and getting handed a stack of papers with no context for any of it. The field borrowed a three-part memory split from cognitive science, and it holds up: - **Episodic memory** covers specific events: what happened, when, in what order. Old session transcripts live here. - **Semantic memory** holds distilled facts, the stuff you've pulled out of those events once you know it's actually true. - **Procedural memory** is know-how. It covers which tool to try first, how to recover when a search comes back empty, the playbook the agent's built up over time. Why split these apart at all? Because an agent that treats "the thing I skimmed on a random blog" the same as "the thing I've checked three separate times" will contradict itself mid-session, or worse, state a wrong answer with total confidence. Deciding which bucket a piece of information belongs in is the first real architecture decision you make, and most teams never make it on purpose. The in-context window is working memory and nothing more. Anything you want to survive past this one exchange has to get written somewhere else, deliberately. No single storage system does all three jobs well, and that's not a flaw waiting to be optimized out. It's the reason layered memory exists as a category at all. ## The simplest strategies and why they fail at depth The sliding window is the first thing everybody tries: keep the last N turns, drop the rest. That's fine for small talk, but it breaks the moment a finding from turn four still matters at turn forty, because by then it's gone. You shredded the thread one dropped turn at a time and didn't notice until the agent asked you the same question twice. The two-buffer setup is a step up: a raw buffer for recent turns, a compressed summary buffer for the older stuff. Better, but summarization loses things by nature. Exact numbers, source citations, the difference between "probably true" and "confirmed" often don't make it through the compression pass. And it's still one tier pretending to be three; it can't tell an episode from a fact even if it wanted to. Then there's plain retrieval-augmented generation: pull relevant chunks on demand, drop them in context. This helps with raw token count, but flat retrieval on basic keyword search does badly on research tasks that need several sources connected together; some benchmarks show it scoring in the single digits on multi-hop questions. The reason is structural, not a tuning problem. Research questions live in the relationships between things, not inside isolated paragraphs. Flat retrieval gives you the paragraphs and throws away the wiring between them. All three of these treat the symptom, the window filling up, instead of the actual disease, which is signal getting buried under noise. ## The layered memory architecture that research agents actually need The fix isn't a bigger bucket. It's routing information to the right bucket at the right time. **Layer 1, in-context working memory**, holds only the live reasoning thread: the current question, the most recent sources, whatever hypothesis is active right now. It needs curating, not just piling up like a junk drawer nobody cleans out. Newer systems like DeepSeek-V3.2 and GLM-4.7 build pruning right into the reasoning loop, compressing what's already been seen and cutting redundant history as it goes instead of letting it stack up. **Layer 2, the episodic store**, writes state snapshots after each meaningful step. That lets a session pause and pick back up later, or lets a developer rewind and see exactly what the agent knew at step twelve. LangGraph's checkpointing does this for real, writing snapshots out to Postgres, SQLite, Redis, or MongoDB. **Layer 3, the semantic store**, is the distilled, checked knowledge base. Facts and structured extractions pulled out of finished research steps, queried when needed instead of crammed into context up front. GraphRAG is the real upgrade over flat RAG here, because instead of retrieving isolated chunks it retrieves subgraphs: entities, their relationships, and the context wrapped around them. That's what actually makes multi-hop reasoning possible. **Layer 4, the procedural store**, holds the agent's accumulated know-how, including which tools to call in what order and how it's learned to recover when a search dead-ends. Update it across sessions and the agent gets better over time instead of relearning the same lesson every Monday. Put all four together and a research step looks like: read from semantic and episodic memory, reason inside working memory, write new findings back to whichever layer owns them, prune whatever stopped earning its spot. Done well, this kind of trajectory compression can cut context size by 68% while keeping 91% of what matters, per the RE-TRAC paper. ## How production frameworks implement this architecture today Nobody builds all four layers from scratch. A handful of tools have staked out real ground here instead. Mem0 (arXiv: 2504.19413) is a long-term memory layer built for production agents specifically; it learns user preferences and pulls context back across sessions, and by mid-2025 it was running at real scale. LangGraph handles the episodic side well through checkpointing to durable storage, which is exactly what you want for time-travel debugging or picking a session back up after it got interrupted overnight. HiAgent, shown at ACL 2025 in Vienna, tackles hierarchical working memory for tasks that stretch across a lot of steps, aimed squarely at the degradation that wrecks long research sessions. Letta takes a different route: LLM-managed memory paging for conversations that outgrow the window. Worth flagging the cost here, though: every paging decision eats latency and burns tokens, and if you're running high-frequency calls, that overhead piles up fast. Cognee builds and keeps updating a structured memory graph of user history, a working version of the semantic layer aimed at customer-facing agents. On the research side, systems like IterResearch and MemAgent rebuild task state at each step using dynamic memory structures, tossing out generic history to fake something close to an unlimited horizon. No single tool does all four layers. Most real deployments stitch together a checkpointing framework, a dedicated memory layer, and a vector or graph retrieval backend, then wire the three together by hand. ## Why the quality of ingested data determines whether memory architecture pays off Here's the part people skip past. A beautifully layered memory system fed raw, messy HTML still poisons every retrieval that comes after it, and garbage in means garbage retrieved and garbage reasoned over. Format isn't a preprocessing footnote; it's load-bearing. Raw HTML off a web page averages 38,381 tokens, and most of that is nav bars, ad scripts, footers, cookie banners: noise fighting for space in a window that isn't infinite no matter what the spec sheet promises. Run that same page through a scraper built to output clean Markdown and you land around 2,788 tokens, a 94% cut, saving roughly 35,980 tokens per page. At Claude Sonnet 4.6 pricing that gap works out to about $1,079 per 10,000 scrapes, based on Vellum's 2026 analysis. The savings don't stop at the bill. Cleaner input means tighter chunks and sharper vector embeddings, which means retrieval gets more accurate across the board. Summarization compresses better starting from structured text instead of soup. GraphRAG's entity extraction is noticeably more accurate on clean Markdown or JSON than on a page full of `