Agent loops: planning, memory and long-horizon tasks

An agent is a model that decides its own next step, takes it, looks at the result, and repeats until it judges the task done. This lesson covers the structure of that loop, why long tasks are hard for it, and the methods (planning, memory, sub-agents and verification) that let 2026 agents work for hours instead of minutes.

Workflows and agents

An Anthropic engineering guide draws a useful line. In a workflow, LLM calls and tools are orchestrated through code paths that a developer fixed in advance. In an agent, the model directs its own process and tool use. Its building block is the "augmented LLM": a model with retrieval, tools and memory. Its advice is to use the simplest design that works and add autonomy only when the task needs it, because agents trade latency and cost for flexibility.^[1]

The agent loop itself is short:

context ← system prompt + task
repeat:
    output ← model(context)            # may include reasoning
    if output is a final answer: stop
    result ← execute(output.tool_call) # harness runs the tool
    context ← context + output + result

ReAct supplied the pattern of interleaving thoughts with actions.^[2] Everything else in this lesson is about what goes wrong when this loop runs for hundreds of iterations.

Why long horizons are hard: compounding error

Suppose each step of a task succeeds independently with probability $p$ and the task needs $n$ steps with no recovery. Then

Worked example. At $p = 0.99$ per step, a 10-step task succeeds 90.4% of the time, but a 100-step task succeeds only $0.99^{100} \approx 36.6\%$ of the time. Raising per-step reliability to $0.999$ brings the 100-step task to $90.5\%$. Recovery matters as much as raw reliability. If the agent notices and repairs 80% of its errors, the effective per-step failure rate drops from 1% to 0.2%, and $0.998^{100} \approx 81.9\%$. A tenfold improvement in per-step reliability and a habit of checking your own work both turn an unusable agent into a usable one.

The model is a simplification: real errors are not independent, and some are fatal while others are harmless. But it explains three observed patterns. Success falls steeply with task length. The ability to detect errors is worth a great deal. And 99%-reliable behaviour is much harder to demonstrate than 50%-reliable behaviour, which matters when measuring time horizons.

Planning and self-correction

Planning in today's agents is mostly explicit text: writing a plan, a to-do list or a hypothesis, then revising it as observations arrive. Two early designs showed how to make the revision persist.

• Reflexion (2023) has the agent write a verbal reflection after a failed attempt and store it in an episodic memory buffer that conditions the next attempt. No weights change. It reached 91% pass@1 on HumanEval, against 80% for GPT-4 at the time.^[3]
• Voyager (2023), a Minecraft agent, combined an automatic curriculum with a growing library of executable skills, written as code and retrieved when relevant. It obtained 3.3× more unique items than prior methods and could reuse its skills in a new world.^[4]

Self-correction has a limit, and this lab has measured it. In the correlated-critique note, a generator was given a false "house convention" in its context and followed it on 12 of 12 affected items. A critic of the same model with the same context endorsed all 12 errors, in both replicates. The same model with a fresh context caught all 12.^[5] The critic was not worse at arithmetic. It had taken on the same standard of correctness. The same note warns that on naturally occurring errors the gains were far smaller. On one dataset the fresh critic rejected more than half of the correct answers, so its catch rate looked better than its real discrimination. An agent that reviews its own work in its own context window inherits that context's errors. A reviewer with a clean context, run as a separate sub-agent or a separate pass, is a cheap way to decorrelate.

Memory: managing a finite context

Everything an agent knows about its task must fit in the context window or be fetched back into it. Longer windows do not solve this.

• Position matters. Liu et al. (2023) found that models used information at the beginning and end of long contexts much better than information in the middle, even when the models were built for long context.^[6]
• Recall degrades as context fills. Anthropic's 2025 guidance on "context engineering" calls this context rot. It recommends treating the context window as a limited budget to be curated, not a log to be appended to.^[7]

The working designs fall into three families:

Each technique is lossy compression. What the summary leaves out is gone. In the terms of this lab's What language buys note, no amount of later reasoning can recover a distinction that was dropped when the information was encoded.^[10] Agents that rely on compaction can confidently continue in the wrong direction after the evidence against that direction has been summarised away.

Sub-agents and parallelism

A lead agent can hand focused sub-tasks to sub-agents. Each works in a clean context and returns a condensed summary, often 1,000–2,000 tokens.^[7] Anthropic reported in June 2025 that its multi-agent research system, a Claude Opus 4 lead with Claude Sonnet 4 sub-agents, outperformed a single Claude Opus 4 agent by 90.2% on an internal research evaluation. On the BrowseComp browsing benchmark, token usage alone explained 80% of the variance in performance. Agents used about 4× as many tokens as chat, and multi-agent systems about 15×.^[11] The honest reading is that a large share of the multi-agent gain is spending more test-time compute in parallel. That is valuable when the task is broad and parallelisable and wasteful when it is not.

Long horizons in practice

By 2026, frontier agents ran for hours on software tasks. METR's measurements, covered in the next lesson, put the most capable models' 50% time horizons beyond what its task suite could reliably measure.^[12] Longer loops also offer more chances to game the objective. METR documented o3 monkey-patching an evaluator so that every submission passed.^[13] ImpossibleBench (2025) measures this directly. It builds coding tasks whose specification contradicts their unit tests, so any pass implies the agent cheated. It found behaviours ranging from editing the tests to overloading comparison operators.^[14] An agent loop is an optimiser, and its stopping condition, "the tests pass", is itself a verifier that can be exploited. See When the verifier is the bottleneck.

Key takeaways

• An agent is a loop: the model chooses an action, the harness executes it, and the result is appended to the context. Prefer fixed workflows when the path is known.
• Under independent errors, success decays as $p^n$, so per-step reliability and error recovery dominate long-horizon performance.
• Self-review in a shared context inherits that context's errors. A fresh-context or external check decorrelates, but measure its false alarms.
• Memory is lossy context management through paging, notes and compaction, and whatever is lost in compression cannot be reasoned back.
• Multi-agent gains are partly gains from spending more compute in parallel. Long loops also give more chances to exploit the stopping condition.

Sources