Iterative Retrieval Loops and Stopping Criteria

This piece is about the one architectural question every AI agent building on live web data eventually has to answer: how do you know when to stop retrieving? Loops are the right pattern once a task gets more complicated than a single fact lookup. But building the loop is the easy part. Knowing when it's done is where most production systems quietly fall apart.

Single-pass RAG, retrieve once, generate once, works fine for questions with a clean answer sitting in one document. Ask it something multi-step, though, and it breaks fast. The model doesn't know what it doesn't know until it's already halfway into the wrong answer.

Live web data makes this worse. A vector store is tidy. The web is not. One query rarely gives you the full picture, and results come back noisy, partial, sometimes flatly contradictory. Every round of retrieval also tends to surface new sub-questions nobody knew to ask at the start. You go looking for a company's Q3 earnings and end up three hops deep in a supplier dispute you didn't know existed.

Some tasks genuinely need this. Competitive intelligence work that means cross-referencing five sources by hand. Multi-hop research where finding A tells you that you now need B and C before A even means anything. Monitoring tasks where the same page gets checked again and again because yesterday's answer already expired. Researchers call this iterative RAG, a cyclic pipeline that keeps retrieving based on the original question plus whatever the model has generated so far, piling up context round by round. ITER-RETGEN and IRCoT are the two names people usually point to here.

Loops solve the multi-step problem. Then they hand you a harder one: when do they stop?

What the retrieve-reason-act loop actually looks like inside an agent

Strip the jargon away and it's just a cycle. The agent looks at where things stand, picks an action, runs it, reads what came back, and does it again. Observe, decide, act, observe, over and over, like a dog checking the same spot in the yard for a squirrel that left an hour ago.

Retrieval slots into the "act" step. Web search, scraping, crawling, these are the actions. What comes back, whether that's a page of raw HTML or a structured JSON blob, is the observation.

The interesting part sits in the reasoning layer on top. IRCoT is the clearest demonstration: the model's own chain-of-thought reshapes what it searches for next, and what it retrieves next reshapes the chain-of-thought after that. A feedback loop inside a feedback loop.

Agentic RAG takes this further. Instead of following a fixed retrieve-then-generate pipeline, the model decides for itself when to pull in more information and how. It plans, manages its own memory, inspects the evidence it's gathered, and judges whether one more retrieval round is worth the trouble. Retrieval stops being a step and becomes a decision, made over and over. Newer reasoning models, Search-o1, Search-r1, R1-Searcher among them, push this further still by training the model itself to govern when it searches, instead of bolting search on as an external tool.

Multi-agent setups split the job further: one agent handles external retrieval, another manages what's already known internally, and others chase down whatever gaps remain.

Here's the architectural fact sitting underneath all of it: nothing in this loop guarantees it ever ends. Left alone, it runs until something outside the loop forces it to quit.

Why the base architecture is biased toward continuing rather than stopping

Language models get trained to produce output. That's the job. Nobody trains them to sit still and say "nothing more needed here," because in most training setups, silence just isn't a rewarded behavior. Drop that model into an agent framework where it can call tools again and again, and that habit turns into a structural tendency: keep acting, keep generating, keep going.

Researchers have spent a lot of energy on how to retrieve iteratively and how to call tools across multiple turns. Comparatively little effort has gone into the flip side: knowing when enough is enough.

Three things tend to cause runaway loops in practice. First, the agent has no internal definition of "done," so it keeps working toward a goal it can't recognize even once it's reached. Second, tools hand back ambiguous feedback, results that are partial or a vague line like "more data may be available," giving the agent nothing solid to treat as a finish line. Third, and most basic: there's often no hard limit at all. No cap on iterations, no cap on time, no cap on tokens.

Watch for the verification spiral. Tell an agent to check its own work, and it finds something marginally better it could do, revises, checks again, finds another marginal tweak, and keeps going. This happens constantly when the instruction to verify shows up without any definition of good enough attached. One documented case had an agent calling the same search tool dozens of times back to back, each query barely different from the last, never once deciding it had seen enough. Tokens burned, costs climbed, and the user got nothing back for it.

Stopping isn't a line you tune in a prompt. It's a structural requirement, and someone has to design it in on purpose.

The main families of stopping criteria and what each one measures

Nobody solves this with one trick. Here's what's actually out there, grouped loosely by how much engineering effort each one takes to build.

Hard budget constraints are the blunt instrument, and also the one every serious deployment needs no matter what else it uses. Cap total tokens per task and cut things off once you hit the ceiling. Cap the number of retrieval rounds too, and calibrate by task: simple lookups usually wrap up in a handful of rounds, multi-step research needs more, and if a task is chewing through an enormous number of iterations, that's usually a sign the task itself needs breaking into smaller pieces, not a sign the loop needs more room. Cap wall-clock time or API calls as well, which matters a lot for anything user-facing where speed is part of the deal. The weakness is obvious: none of these care whether the answer is any good. They just stop the clock.

Confidence-based stopping tries to fix that by watching how sure the model seems. Signals here include the probability gap between the top token and the runner-up, entropy across the output distribution, or how much several independently sampled answers agree with each other. FLARE uses token-level probability thresholds to decide when a fresh retrieval is warranted. DRAGIN blends entropy with attention weights for the same call. The catch: confidence measures what's happening inside the model's head, not whether pulling one more document would actually change the answer.

Trained stopping modules go a step further and teach a model to make the call directly. Self-RAG has the model emit explicit reflection tokens at each step, trained on data distilled from a larger model. Probing-RAG uses a separate small model, a "prober," trained on correctness labels, to score whether more retrieval is worth doing. Adaptive-RAG trains a classifier that judges how complex a question is, but only before the loop starts, never while it's running.

Satisfaction thresholds and coverage heuristics are simpler: stop once an internal confidence score clears a preset bar, or require a minimum number of reasoning steps to get resolved before the agent is allowed to call it quits.

Utility-based stopping treats "stop" as just another move on the board. The agent scores every candidate next action, including doing nothing further, and picks whichever scores highest. If stopping wins, it stops. Termination gets weighed the same way any other decision does, instead of getting left to a vague line buried in a prompt.

Then there are the practical guardrails underneath all of it: tracking the last several tool calls and forcing a stop if the same call with the same arguments keeps repeating, or injecting a check-in every few rounds that asks the agent, plainly, whether it's actually making progress.

Why present-focused stopping signals mislead and what forward-looking control offers instead

Almost every trained stopping module shares one blind spot. They get trained to judge answer quality as if the agent stopped right this second. Present-focused, every one of them, with no eye on where another round might lead.

That framing misleads in two opposite directions. Sometimes the agent already has everything it needs, but the mix also contains a bunch of irrelevant junk it picked up along the way, so the quality score reads low, the system decides it needs more retrieval, and that extra round just adds noise on top of noise. Other times an early answer sounds confident enough that the agent stops too soon, before it ever gathers the one piece of evidence that would have flipped the conclusion entirely.

A paper called Stop-RAG, presented at the NeurIPS 2025 Workshop on Multi-Turn Interactions, tackles this by reframing the whole loop as a finite-horizon Markov decision process. It trains a Q-network, using a variant of Q-learning called Q(λ), to estimate whether continuing to retrieve is actually likely to make the final answer better, weighing immediate payoff against expected future payoff before deciding whether to keep going. It doesn't need to peek inside the language model's internals either, so it can slot in as a modular piece even with closed, black-box models. On multi-hop QA benchmarks, it beat both fixed-iteration cutoffs and prompt-based stopping approaches.

Most teams aren't about to go train their own Q-network next quarter, and that's fine. The underlying principle still holds without the machinery: stopping decisions need some estimate of future value baked in, not just a snapshot of how things look right now. Simpler stand-ins get you partway there, structured progress checks, coverage heuristics, even without the full reinforcement-learning setup.

The harder version of this problem, though, isn't an agent that loops by accident.

How adversarial content in retrieved web pages can trap an agent in a permanent loop

Agents judge their own progress using signals pulled partly from the content they retrieve. That content comes from the open web. Which means, if you're an attacker, that content is yours to write.

Researchers at Zhejiang University documented exactly this in a paper on what they call termination poisoning, released under the name LoopTrap in May 2026. Slip malicious instructions into a page the agent is likely to pull, and you can convince it the task isn't finished yet, even when it plainly is, pushing it into execution that never ends on its own. The study ran across multiple language model agents and multiple tasks and found each agent carries its own behavioral fingerprint, meaning the attack that traps one model completely misses another. Multi-agent setups widen the crack further: a single compromised peer agent can feed corrupted progress signals straight into another agent's context, no need to touch any external web content at all.

Agent Security Bench, presented at ICLR 2025, ran a broad test across many model backbones and attack types and found agents are widely vulnerable, with success rates for some attacks reaching high levels, and existing defenses often doing little to stop them.

Here's the part that should worry anyone running these systems at scale: an agent caught in a loop by poisoned content can burn through a serious chunk of a session's compute budget before anyone notices, and this is already a live concern inside enterprise deployments. Hard budget caps earn their keep here well beyond mere efficiency. They're the last line standing between you and an agent that's been talked into believing its job is never done.

Failure modes that appear in production retrieval loops before stopping criteria are tightened

A handful of patterns show up again and again once you start watching these systems run in production.

The same-query spiral is the most common. The agent fires off nearly the same retrieval call over and over, each time with a slightly different phrasing, never actually converging on anything new. It's a clean tell that the agent has no working definition of "I've retrieved enough."

The verification spiral looks similar but comes from a different place. The agent checks its work, spots something it could nudge slightly better, revises, checks again, and repeats indefinitely, because the instruction to verify never came with a line marking where good enough actually sits.

Oscillation is its own animal. The agent bounces back and forth between two actions, or two nearly identical queries, forever. Usually this traces back to a tool that can't tell the agent the difference between "there's nothing more here" and "something broke."

Cascading errors are the nastiest of the bunch. One wrong turn early in the loop, a bad retrieval or a bad plan, compounds with every round that follows, dragging the agent further from the right answer each time. Once that starts, there's no fixing it without resetting the run or breaking the task down differently from scratch.

And then there's the quieter failure: context just piling up. As retrieved chunks accumulate round after round, models start losing track of anything sitting in the middle of that pile, a known issue sometimes called "lost in the middle." A method called FACT tackles this by rewriting the accumulated context on each pass and masking facts already found, so the model's attention doesn't keep drifting back to information it already covered.

All of this shows up eventually as infrastructure pain: API quotas run dry, memory climbs as context keeps growing, CPU usage spikes, users start seeing timeouts. Those are reasoning failures wearing an infrastructure costume.

Concrete stopping strategies to build into a retrieval pipeline today

No single stopping rule covers every failure mode above. Layer them. Each one has a blind spot, and the trick is making sure the others cover it.

Start with hard budgets, because they're non-negotiable. Set an iteration cap that matches the task: a handful of rounds for simple lookups, more for genuine multi-step research. Set a token budget that accounts for everything piled up across every round, not just what one round pulls in. And put a wall-clock or API-call limit on anything touching live web endpoints, full stop.

Layer semantic duplicate detection on top of that. Keep a rolling window of the last several tool calls and force a stop if the same call, or something close enough to it, keeps showing up. Normalize the query text before comparing so near-identical rewordings don't slip past the check.

Build in structured progress checks. Every few rounds, ask the agent directly: has anything genuinely new turned up since the last check? If the answer is no, stop, or hand the task off for decomposition instead of letting it keep spinning.

Spell out what "done" actually means in the system prompt, and make it concrete: a condition the agent can check off, not an instruction like "be thorough" that gives it nothing to test against.

Treat stopping as a real action the agent can choose, not something that only kicks in once everything else fails. The utility framing gets this right: stopping should win outright once its expected payoff beats whatever another round of retrieval promises.

For web retrieval specifically, tools need to hand back a clear terminal signal. "No further results" and "something broke" are two completely different situations, and an agent needs to treat them that way instead of lumping both into a shrug.

And build for the adversarial case directly. Budget caps enforce a stop no matter what the retrieved content is trying to convince the agent to believe about how finished its task actually is.

A lot of this comes down to what the tools hand back in the first place. A pipeline built on a web data API that returns clean, structured Markdown or JSON, which is what Olostep provides, removes a big source of the ambiguity that drives runaway loops to begin with. An agent parsing raw, inconsistent HTML has to guess what "no more results" even looks like. An agent reading a clean, structured response has a real signal to work from instead. That difference alone closes off one of the most common paths into a loop that never ends.