Categorizing Agent Failure Modes Across Harness Layers

Agent failures look uniform on the surface: the model gives a wrong answer, stalls out, or loops forever. Those symptoms trace back to five different places in the system, and each place needs a different fix. DigitalApplied's 2026 figures put the rate of projects failing to reach production at 88%, and the usual explanation, that models aren't good enough yet, misses what's actually going on. The model is rarely the problem. The scaffolding around it, the harness, is where things come apart.

The same broken run, an agent that produces the wrong answer, stops early, or loops on a task it already finished, can come from prompt ambiguity, a tool schema that changed without anyone noticing, a memory read that got dropped, or an orchestration loop with no exit condition. Four different causes, one visible failure. Raj et al. (arXiv:2607.28802, 2026) make a sharp point about this: labeling something an "execution failure" can lump together an external service that's genuinely down with a model that just gave up after a hiccup it could have retried. Those are not the same bug, and they don't get fixed the same way.

That distinction matters because repair work has to go somewhere specific. Post-training, harness engineering, environment redesign: pick the wrong bucket and the fix does nothing. Before any of that is possible, someone has to say precisely where the failure happened, and that requires a shared vocabulary for the parts of an agent system that can break. That's what this piece builds out, layer by layer.

Three taxonomies engineers should know

Three research efforts have tried to map this space, and each one adds something the others don't.

MAST, from Cemri et al. at UC Berkeley (2025), is the empirical anchor. The team annotated more than 1,600 execution traces across seven multi-agent frameworks, including AutoGen, ChatDev, and CrewAI, and pulled out 14 distinct failure modes. Those modes cluster into three groups: System Design Issues, Inter-Agent Misalignment, and Task Verification. The most-cited number from that work is stark: System Design Issues account for 44.2% of failures in one agent trajectory study, with Inter-Agent Misalignment close behind at 32.3%. The limit of MAST is that it sorts failures by what they look like, not by which component actually needs to be touched to fix them.

Raj et al.'s 2026 paper (arXiv:2607.28802) closes that gap. Instead of grouping failures by symptom, the paper treats the interaction edge between two components, model and tool, model and memory, harness and model, as the unit worth studying. That shift produces 41 named failure modes, each one tagged with an edge and a fault side: model-side or harness-side. It's a taxonomy built to point at a fix as well as describe a symptom. The reproducibility numbers back it up, too: across four frontier models, independent reasoning-agent judges recovered human labels with the strongest judge hitting a Cohen's κ of 0.76. That's a real signal that these categories reflect actual structure in the failures, not just one annotator's taste.

HarnessFix (Chen et al., arXiv:2606.06324, 2026) takes the layer idea and turns it into something buildable. The paper lays out seven harness layers: Execution Environment and Sandbox, Tool Interface, Context and Memory, Lifecycle and Orchestration, Observability, Verification and Evaluation, and Governance and Security. It compiles raw execution traces into what the authors call a Harness-aware Trace Intermediate Representation, which lines up each runtime step with the harness artifact that shaped it. The payoff is concrete: HarnessFix improved performance over initial harnesses by 6.3% to 18.4% across four benchmarks, beating both human-designed harnesses and self-evolution baselines.

Between the three, a shared vocabulary emerges: prompt, tool, memory, workflow, environment. That's the map the rest of this piece follows.

The prompt layer: how instruction ambiguity and specification drift produce silent misbehavior

The prompt layer covers system prompts, user instructions, task specs, and any plain-language guidance the model gets while it's running. It's the harness's main lever for steering behavior, and it fails in two opposite directions.

Under-specification is the familiar one: an instruction is vague, and the model fills the gap in a way nobody expected. But over-specification is just as common and gets far less attention. A model with stronger instruction-following can latch onto sloppy system-prompt language and follow it too literally, or it can start refusing requests that an earlier version of the same model would have handled without complaint. Practitioners maintaining harnesses in production flag this pattern regularly. Both directions break the workflow. The only way to tell which one happened is to read the trace.

One signature occurs often enough in agent traces to have a name: the verification spiral. Research on multi-step agent traces describes agents that solve a task correctly, early, then burn many extra steps re-checking their own work because each verification command resets a state flag. That reset forces another pass through the task_complete-checklist-verify cycle, and the loop doesn't stop on its own. In the trace, this looks unmistakable: the right answer appears early, then the step count balloons with redundant verification actions before the run finally ends. The cause is a prompt that specifies verification logic without accounting for what that state flag actually does on reset, a design defect sitting squarely in the prompt layer. It's a prompt that specifies verification logic without accounting for what that state flag actually does on reset, a design defect sitting squarely in the prompt layer.

There's also a trap tied to model updates. When a provider improves the underlying model, a prompt written for the old version's quirks can suddenly stop working the way it used to. The harness and the model have drifted apart, and the prompt is usually the first place that drift becomes visible as an actual failure. Watch for step counts that balloon on tasks that used to terminate quickly, refusals that never happened before an update, and completion rates that swing depending on which prompt variant is running.

The tool layer: schema drift, malformed arguments, and the failure modes hidden in the interface

The tool layer governs how an agent finds, understands, picks, and calls tools: schemas, documentation, argument formatting, and whatever error text comes back to the model after a call. Raj et al. (arXiv:2607.28802, 2026) name several failure modes that live at this edge. Incorrect Tool Selection is the model reaching for the wrong tool. Tool Hallucination covers a model invoking a tool that doesn't exist or isn't available. Tool Feedback Neglect is the model ignoring what a tool actually returned and running on its old assumptions instead. Tool Recovery Failure is the model failing to retry or work around a transient error. Malformed Arguments covers syntactically broken calls to a real tool, while Suboptimal Arguments covers calls that parse fine but carry weak or poorly chosen values.

Schema drift deserves its own attention as a recurring production problem. Chen et al. (2025) built ToolQA-D specifically to study this: a benchmark where API names, signatures, and behavior shift over time while the agent has no idea anything changed. A harness built and tuned against one model version can behave differently against its successor even on an identical task, because parameter naming, which fields are optional versus required, and how ambiguous tool selection gets handled all shift between versions. Token length adds another wrinkle: if a new model version produces noticeably longer responses to the same prompt, timeout logic and rate-limit handling built for the old version can misfire without any obvious error message.

There's a useful ordering principle here too. A priority-based annotation protocol described in arXiv:2605.22166 checks for action realization failures first, then environment contract mismatches, then trajectory degeneration, specifically so that a later symptom doesn't paper over an earlier interface failure. Take an agent that writes a tool call as plain text instead of a structured invocation. The environment never executes it, the episode runs out its step budget, and it would be easy to blame the orchestration layer for running out of room. The actual fault is upstream, at the tool interface. Watch for tool calls that succeed on syntax but return payloads nobody expected, rising malformed-argument rates right after a model update, and tool calls that show up in the model's output as plain text instead of a real function call.

The memory layer: context degradation, stale state, and the 2% per-step decay problem

Memory and context decide what the model actually sees at each step: the contents of the context window, session state, summaries, retrieved documents, and anything stored persistently that outlives the active window. This is the layer where degradation is quiet and cumulative rather than loud and immediate.

MemU's 2026 figures put a number on it: 65% of enterprise AI agent failures trace back to context drift rather than any architectural defect, and the underlying mechanism gets quantified at roughly 2% context retention loss per step. Running that forward makes the consequence blunt. MemU figures cited in Atlan's reporting show that by five cycles into a multi-step workflow, less than 60% of the original context remains reliably available to the agent. Nothing crashes. The agent keeps going, just on a progressively thinner picture of what it's supposed to be doing.

Raj et al. name several failure modes at this edge. Memory Rationale Erosion is when the reasoning behind a stored fact gets lost, a memory-write failure the paper attributes to Garg et al. (2026); a related but separate mode, Context Rationale Erosion, covers reasoning that drops out of context due to compaction. Missed Read is the agent failing to pull up a memory that would have changed what it did next. Memory Following Failure is retrieving the right memory and then not acting on it anyway.

A deeper question follows from all of this: can an agent tell when its own memory has gone stale? The STALE benchmark (arXiv:2605.06527, 2026) asks exactly that, and the fact that it needs asking says staleness detection is still an unsolved responsibility of the harness, not something models handle on their own.

Raj et al. also describe a case, using Claude Code as the example, where a model ignores an earlier user instruction. That can happen for two entirely different reasons: the harness's context compaction physically removed the instruction, or the instruction stayed available and the model simply failed to follow it. The observed behavior in both cases is identical. The fix is not. One is a harness change, the other is a model-level problem. Watch for decisions that contradict something stated early in the session, correct outputs early on followed by contradictions later, and error rates that climb as the session gets longer.

The workflow and orchestration layer: loops, cascading errors, and unchecked autonomy

Lifecycle and orchestration control the actual execution flow: think-act-observe loops, retries, task-state tracking, coordination between multiple agents, and the conditions that decide when a run stops. The math here is unforgiving. At 85% per-step accuracy across a 10-step workflow, task success falls to roughly 20% (0.85 raised to the 10th power). That compounding effect is what MAST's System Design Issues category captures at 44.2% of observed failures, and within that number, the breakdown gets specific: Step Repetition accounts for 15.7%, Unaware of Termination Conditions for 12.4%, Disobey Task Specification for 11.8%, and Loss of Conversation History for 2.8%.

One anti-pattern occurs often enough in agent design to deserve a name: All-or-Nothing Autonomy. That's handing an agent full control over a multi-step task with no approval gates and no human checkpoint at the moments where a wrong call actually matters. Without a checkpoint somewhere in the middle, one bad decision cascades through everything downstream. The Replit incident from July 2025 gets cited repeatedly in practitioner writing as the clearest public example of what unchecked autonomy looks like when it goes wrong.

Raj et al. name two more failure modes specific to orchestration edges. Delegation Failure is an orchestrator handing a subtask to a subagent that doesn't have the capability or context to actually do it. Communication Failure is agents in a multi-agent setup failing to pass state or intermediate results at a handoff.

Termination conditions deserve a closer look on their own. Agents without a clear stopping rule will loop on a task that's already done, and the verification spiral described in the prompt layer gets worse, not better, when the orchestration layer has no loop detection or step-budget enforcement sitting on top of it. Watch for step counts that climb steadily with no change in output, identical tool calls repeating, task-state flags toggling without the task actually advancing, and multi-agent handoffs that produce nothing on the other end.

Harness drift: when model improvements silently break a working harness

Model drift and harness drift are not the same thing, and mixing them up leads teams to the wrong fix. Model drift is a change in what the model actually does, usually from a provider pushing an update. Harness drift is the growing gap between what the harness assumes about the model and what the model now actually does. Harness drift is almost always caused by model drift, but research consistently points to the harness, not the model, as what makes the agent fail.

That distinction has a practical consequence: rolling back or swapping the model will not fix harness drift. The harness itself has to change to match how the model now behaves.

Drift appears differently in each layer covered above. At the prompt layer, a more instruction-literal model can over-adhere to a loosely written system prompt, or start refusing requests it used to handle fine. At the tool layer, parameter naming or required-versus-optional field handling shifts between versions, and calls that worked before start failing with malformed-argument errors. At the memory layer, a model that produces longer output per step burns through context faster, which speeds up that 2% per-step retention decay. At the orchestration layer, a model that reasons more carefully takes more steps to finish a subtask, which can blow through step budgets that were calibrated for a faster, less careful predecessor.

None of these are model failures in the sense of the model being wrong. They're mismatches between what the harness expects and what the model, now improved, actually does. The fix in every case sits in the harness, not in the model.