Loops, detours & dead ends

The five shapes

Long-horizon agents fail in shapes. Learn to recognize the recurring ones in your traces:

• Loops: the same tool with the same arguments, again and again, hoping for a different result. Detectable in code: flag any repeated (tool, args) pair.
• Detours: steps that succeed but contribute nothing; ten searches when two would do. Costs latency and tokens, and each extra step is a new chance to derail.
• Dead ends: a tool errors and the agent gives up, apologizes, or worst of all, proceeds as if the call had succeeded.
• Context loss: step 14 contradicts what the user said at step 2. Sometimes the trajectory literally overflows the context window; more often the model stops attending to early turns, or a compaction step dropped them.
• Error cascades: one bad step (wrong ID retrieved) silently poisons every step after it.

Each shape is recognizable in two or three lines of trace. These are the patterns to burn into your eyes, all drawn from the refund agent:

# LOOP: same call, same args, no new information
    07 search_orders(email="kim@example.com")  -> 0 results
    08 search_orders(email="kim@example.com")  -> 0 results
    09 search_orders(email="kim@example.com")  -> 0 results
  
    # DETOUR: succeeding steps that add nothing
    04 retrieval "refund policy"       -> returns-policy#damaged
    05 retrieval "return policy"       -> same chunks
    06 retrieval "money back policy"   -> same chunks again
  
    # DEAD END: error, then pretend it worked
    05 create_refund(order_id=4021)    -> ERROR eligibility_not_checked
    06 model_call respond -> "Your refund has been processed!"
  
    # CONTEXT LOSS: step 14 forgets step 2
    02 user: "the blender, order 4021, NOT the toaster (4019)"
    14 create_refund(order_id=4019)
  
    # ERROR CASCADE: one wrong ID poisons the rest
    03 get_order(order_id=4012)        -> succeeds (wrong order: typo)
    04-11 eligibility, refund, reply: all computed for 4012

Notice what the snippets have in common: none of them needs a judge. A loop is a repeated pair, a detour is redundant retrieval results, a dead end is an error event followed by a success claim, context loss is an argument that contradicts an earlier user turn, and a cascade is a wrong value flowing forward. All five are detectable by code over structured traces, which is exactly why Lesson 3.1 insisted on structured traces.

A first-divergence walk

That last shape is why the core debugging skill is finding the first divergence: walk the trace from the top and mark the earliest step where things went wrong. Everything downstream is contamination, not signal. Grade and fix the first failure; re-run; repeat. Fixing step 9 while step 3 is broken is wasted work.

Here is the walk on the cascade trace above. Step 1, the plan, reads fine. Step 2, the model's extraction of the order number, also reads fine, until you compare step 3's argument against the user's message: the user wrote #4021 and the model transcribed 4012. Step 3 executes and succeeds, because 4012 is a real order, and from here every step is internally consistent and globally wrong: the eligibility check approves the wrong purchase, the refund pays the wrong amount, and the reply confirms all of it fluently. The first divergence is the extraction feeding step 3. That is the step you grade, the step you fix (stricter ID handling in the prompt, or an echo-back confirmation before any write), and the step you turn into a golden case. Patching the visibly wrong refund at step 9 would treat a symptom eight steps downstream of the disease.

Process metrics, defined

Score long tasks with partial progress, not just success/failure. A binary metric says two agents both failed; a progress metric (4 of 6 subgoals, first divergence at step 3 vs step 11) says one is close and one is lost. Define the milestones a correct run must pass through and measure how far each run gets. Cheap process metrics help too: steps per task, tokens per task, repeated-call count, error-recovery rate. A rising step count with flat success rate is an agent quietly getting lost.

Those process metrics deserve exact definitions, because vague ones get ignored. Steps per task: the count of model calls plus tool calls in a run; track the median and the p95 separately, because the median hides the runaway tail. A healthy median sits within about 1.5× the hand-solved minimum: the refund case takes 5 to 6 steps done well, so a median of 8 is fine and a p95 of 30 is a fire. Repeated-call count: the number of (tool, normalized-arguments) pairs occurring more than once in a trace. Healthy is near zero; averaging above one repeat per trace means looping is routine. Legitimate repeats (polling a job status) get whitelisted per tool, not averaged away. Error-recovery rate: of tool calls that returned an error, the fraction where the agent's next relevant action was corrective (different arguments, a different tool, or a question to the user) rather than a verbatim retry, a giving-up apology, or proceeding as if the call had succeeded. Healthy agents recover from most transient errors; below roughly half, error handling is your top fix. Treat these ranges as rules of thumb to calibrate on your own traces, then alert on drift. The metrics also guard a second axis: an agent can look accurate while being wildly inefficient, and an accuracy gain bought with four times the steps and tokens is often no gain at all; cost belongs on the scoreboard next to success.

Where teams go wrong

Average blindness: watching mean steps-per-task while the p95 explodes; most runs look fine while the worst tenth burns the latency budget and the token bill. Survivor-only grading: computing metrics only over runs that finished, so timeouts and crashes quietly leave the denominator and the dashboard improves as the agent gets worse; count every started run. Downstream patching: treating the loudest failure in the trace as the failure, adding a retry at step 9 when the divergence was at step 3, which converts one wrong refund into three attempts at it.