Trace-Driven Debugging vs Log-Based Debugging for LLM Agents
Traces preserve the causal chain that flat logs miss entirely.

An LLM agent can answer a question confidently, cite a tool result that doesn't exist, and return output that passes every check a log file knows how to run. Every log line says success. That gap between a clean log and a wrong answer is the subject of this piece: why log-based debugging fails for agents, and why trace-driven debugging works where it doesn't.
Why log-based debugging breaks for LLM agents
Log-based debugging depends on one assumption holding true: identical inputs produce identical outputs, so a log line plus a stack trace is enough to find where things went wrong. That assumption holds for deterministic code. It does not hold for LLM agents, which are probabilistic by design: two runs given the same prompt can take different paths, call different tools, and land on different answers. In deterministic systems, failure announces itself, a crash, an exception, a stack trace pointing at a line number. In an agent, failure looks like drift, not exception, and drift leaves no error to catch. The agent doesn't throw an exception when it hallucinates a tool name or follows a reasonable-sounding but wrong chain of logic. It returns a well-formed answer, the log records a completed request, and nothing in that log says the answer is wrong. That's the break, and every structural problem that follows in agent debugging traces back to it.
Sources of multi-step agent failure that flat logs cannot capture
The shortfall here isn't that logs record too little data. Flat logs cannot represent causality across a chain of decisions made by autonomous, interacting components, no matter how much you log. Each step finishes cleanly. The eligibility check passes, the necessity validation passes, the coding step completes. Then the final authorization comes back denied, because documentation that should have been flagged three steps earlier never was, a divergence that only becomes visible across the sequence of steps. Generic logging treats each of those steps as an opaque HTTP request, with no semantic read on quality and no way to attribute token cost across the workflow as a whole. Errors compound silently this way: each agent inherits a small flaw from the one before it, and by the final step the output is unrecognizable even though no single log line was ever technically wrong (restated here as the compounding problem that section's principle describes). Teams often respond to this by logging more: every input, every output, at every step. That still misses the structure that matters. A complete list of calls is not a tree, and without the tree you cannot see which LLM call triggered which tool call, or which sub-agent made which decision that propagated downstream. The causal structure connecting the data that already exists is what's missing from a flat log.
What a trace captures that a log cannot
A trace succeeds where a log fails because it preserves the full causal execution context, prompts, completions, tool calls, retrieval results, token counts, and the nested relationships between every step, in a form that can be queried after the fact. The parent-child span relationship is what matters most structurally: each LLM call, tool call, retriever hit, and sub-agent invocation becomes a span nested under whatever called it, so the resulting tree reconstructs the actual sequence of decisions rather than just the list of operations. That nesting is what the authorization example above lacked, and it's what lets an engineer trace a denied claim back to the step that should have flagged missing documentation. Full, verbatim preservation of each span matters too: summarizing or sampling the payloads throws away the evidence needed to diagnose prompt ambiguity, a tool schema mismatch, or a context window getting truncated mid-run, so the full input and output of every LLM and tool span has to survive intact. This standard is maturing into shared tooling rather than staying locked inside any one vendor's proprietary format, which is part of why trace-based observability is becoming a baseline expectation for production agent systems rather than a specialty add-on. A single-shot, stateless LLM call with no tools and no retrieval step doesn't need this infrastructure. A plain log of the prompt, the completion, the model, and the token count covers nearly all the value in that narrower case. The case for full tracing is specific to agentic, multi-step, tool-using runs, where the chain of decisions is the thing that needs explaining. Platforms ingest production traces rather than flat logs precisely to capture this causal structure, attributing a given failure to the layer responsible for it, whether that's the prompt, the tool, the workflow, or memory, so an engineer can move from noticing a wrong answer to naming the decision step that caused it.
The three failure categories where the log-vs-trace gap is most costly
Tool schema drift happens when a version upgrade changes how a tool's schema gets generated, and the agent starts inventing tool names that don't exist or supplying arguments in the wrong shape. A log records that the invocation was attempted and that it failed, but it does not record that the schema the agent was handed no longer matches what the API now expects. Diagnosing that mismatch means comparing the schema the agent received against what the tool currently accepts, a comparison that only a trace, with both schemas preserved as part of the span, makes possible.
Prompt ambiguity and the task drift it produces cover a wider family of failures: incorrect tool invocation, reward hacking, positional bias, mode collapse, degeneration loops, alignment faking, and version drift are patterns that appear only in agent loops and long-horizon tasks. What ties them together is that they're goal-alignment failures, detectable only by comparing what the agent said it was trying to do across multiple spans, not by inspecting any single output in isolation.
Workflow planning loops are the third category, and an April 2025 incident with Claude Code illustrates how compounding, multi-cause failures actually unfold in production. Anthropic's postmortem traced the problem to three separate bugs layered on top of each other: a March 4 change that downgraded reasoning effort, a March 26 caching defect that cleared thinking blocks on every turn until it was fixed on April 10, and an April 16 change to a verbosity prompt. No single log line flagged any of these as the cause. Diagnosing the interaction required reconstructing how three separate changes, introduced weeks apart, combined to degrade behavior, the kind of multi-span, multi-date reconstruction a trace is built for and a flat log is not.
What unites all three categories is that they are failures of process rather than failures of any individual call. Seeing them requires seeing the sequence, the accumulation, and the point of divergence across spans.
Root cause attribution in multi-agent traces: finding the causally decisive event
Capturing the trace is only the first problem. Once a long agent run is fully traced, often hundreds of spans deep, you still need to find which few events in that run actually caused the failure. The most direct tool available is counterfactual replay: rewind the trajectory, edit one candidate event, re-run it, and measure what changes downstream. That approach works, but its cost scales linearly with the number of candidate events under suspicion, which makes exhaustive replay impractical once a system runs at production scale.
Recent research addresses this head-on. Knowledge-Based Zero-Replay Debugging, from Kang and colleagues (arXiv:2606.14805), compiles each trace into a structured event knowledge graph spanning routing decisions, memory state, tool use, model uncertainty, and latent evidence. A calibrated predictor then decides where a limited replay budget should be spent, estimating in advance which events an oracle would judge high-effect, before any replay is actually run. A learning-to-rank, gradient-boosted predictor raises per-trace localization, measured as Branch Recall@5, from 0.73 to 0.93 on held-out families, achieved at zero oracle-replay cost.
That attribution remains an open problem, not a settled one, is confirmed by a survey of papers published between early 2025 and April 2026, which frames failure attribution as identifying the specific agent and execution step responsible for a task failure based on observable execution traces. The field hasn't converged on one method either. Fine-tuned models such as AgenTracer deliver strong accuracy but demand costly annotation and retraining, while instruction-based methods that analyze traces through prompting cost less upfront but sacrifice precision. That tradeoff is a live engineering decision for any team building this kind of tooling. What matters in practice is that attribution has to name a layer. Knowing a run failed tells an engineer nothing useful without knowing whether the cause sat in the prompt, the tool, the workflow, the memory, or the model itself. A trace without attribution tooling on top of it is evidence waiting for a verdict.
How TraceCoder shows trace-driven debugging working in practice
TraceCoder, from Huang and colleagues (arXiv:2602.06875, presented at ICSE 2026), gives this argument a concrete, published demonstration rather than leaving it as theory. The problem it targets is familiar: most automated repair tools rely on pass/fail test signals alone, which gives almost no visibility into what the program actually did internally and makes precise error localization hard. If a repair loop built on pass/fail signals doesn't learn from previous failed attempts, it tends to cycle through the same mistakes repeatedly and inefficiently.
TraceCoder's architecture replaces that signal with runtime traces as the primary evidence. The framework instruments code with diagnostic probes that capture fine-grained execution traces, runs causal analysis on those traces to find the actual root cause of a failure, and feeds that analysis into a Historical Lesson Learning Mechanism that distills lessons from prior failed repair attempts so the same mistake doesn't recur. A Rollback Mechanism sits alongside this: it enforces that each repair iteration makes genuine progress toward a correct solution, and it reverts to the last known best state whenever a new attempt fails to make progress. The reported numbers back the architecture: TraceCoder achieves up to a 34.43% relative improvement in Pass@1 accuracy over existing advanced baselines, with the iterative repair loop itself contributing a substantial share of that gain, figures that describe this paper's benchmark rather than a universal guarantee for every codebase.
The observe-analyze-repair loop TraceCoder runs is trace-native: it depends on having the full causal execution context available, not just a pass/fail result at the end. That same loop cannot be built from flat logs, because flat logs don't preserve the causal chain the analysis step needs to run. The loop TraceCoder demonstrates at the level of code repair has a direct counterpart one layer up, at the level of the agent harness itself.
Harness iteration over model retraining for trace findings
Most of what a trace surfaces, prompt ambiguity, tool schema mismatch, a workflow stuck in a loop, memory management gone wrong, originates in the agent harness rather than in the model's weights. The harness is the control layer that externalizes prompts, tools, memory, workflows, execution loops, constraints, tracing, and evaluation interfaces, and as agents move from single isolated prompts into multi-step execution, it's this layer, more than the underlying model, that shapes how the agent actually behaves.
Retrospective Harness Optimization, published in 2026, makes the case for why this matters directly: an agent's own past trajectories already contain the signal needed to improve it. Re-solving tasks the agent has already attempted and comparing the outcomes exposes where the harness is failing and what change would fix it, without collecting new data or retraining the model at all. A separate 2026 paper, "Adapting the Interface, Not the Model: Runtime Harness Adaptation for Deterministic LLM Agents" (arXiv:2605.22166), argues the same point from a different angle: harness iteration should be the first resort for most production failures, and its framing states the principle outright, adapt the interface, not the model. Self-Harness (Zhang et al., 2026) and Harness-Aware Self-Evolving (Luo et al., arXiv:2607.03935) are concurrent approaches that push this further by having agents propose, evaluate, and accept changes to their own harness, co-evolving model weights, harness configuration, and task solutions together. Taken together, these efforts point to harness iteration becoming a standard engineering discipline in its own right.
The parent-child span relationships and full payload preservation that traces provide, the prompts, completions, tool calls, and retrieval results nested in the order they actually occurred, form the evidence base that makes this kind of fix possible. Without that structure, a team can observe that a harness is failing but has no way to isolate which part of it to change, or to confirm that a proposed change actually fixed the problem rather than shifting it somewhere else in the chain. That's the baseline requirement for diagnosing any agentic, multi-step, tool-using system, and it's the standard the field is converging toward as agent harnesses take on more responsibility for how these systems behave in production.
Sources
- TraceCoder: A Trace-Driven Multi-Agent Framework for Automated Debugging of LLM-Generated Code
- How to Debug AI Agents in 2026: Traces, Spans, and Fix Recipes
- Knowledge-Based Zero-Replay Debugging of Multi-Agent LLM Traces
- TraceCoder: A Trace-Driven Multi-Agent Framework for Automated Debugging of LLM-Generated Code
- TraceCoder: A Trace-Driven Multi-Agent Framework for Automated Debugging of LLM-Generated Code (ICSE 2026 - Research Track) - ICSE 2026
- [2602.06875] TraceCoder: A Trace-Driven Multi-Agent Framework for Automated Debugging of LLM-Generated Code


