Agentic LLMs often fail the same way, again and again. A Stanford research team traced this to missing, reusable capabilities. Their system, TRACE, diagnoses those gaps and trains for them directly.
TRACE stands for Turning Recurrent Agent failures into Capability-targeted training Environments. It was released open-source under an MIT license.
What problem does TRACE solve?
To understand the design, first consider why agents fail. They lack specific skills that tasks demand, like retrieving the right record or verifying a precondition.
Two mainstream fixes spend compute poorly. Direct RL or SFT gives sparse rewards that never say which skill was missing. Broad synthetic data is untargeted, so budget flows to skills the model already has.
However, TRACE observes that failures are not random. A small set of deficits accounts for most failed trajectories. Therefore, each recurring deficit can become its own dense, verifiable training signal.
How does TRACE work?
Given that findings, TRACE runs an automated four-step pipeline. Each step is driven by an LLM agent following a markdown prompt.
Step 1: Contrastive capability analysis
The base agent generates rollouts in the target environment. An analysis agent splits them into successful and failed sets. It then labels every trajectory-capability pair as NA, PRESENT, or LACKING.
A capability is retained only when it is contrastive and high-coverage. Specifically, its contrastive gap must clear δ = 0.20 and coverage must clear ρ = 0.10. Consequently, the pipeline keeps skills whose absence concentrates in failures.
Step 2: Targeted environment synthesis
Next, a generation agent builds one synthetic environment per retained capability. Each environment isolates a single capability while preserving the target’s tool schemas and format.
Task instances are procedurally generated from random seeds. Because generation and verification are algorithmic, rewards need no human labels or LLM judge.
Step 3: Capability adapter training
Then each capability gets one LoRA (Low-Rank Adaptation) adapter, trained on its synthetic environment. The training algorithm is GRPO (Group Relative Policy Optimization). The base model stays frozen throughout.
GRPO groups rollouts by shared seed, so scenarios are identical within a group. Rewards are then normalized within each group to isolate the policy’s contribution.
Step 4: MoE composition with token-level routing
Finally, TRACE composes the adapters into a Mixture-of-Experts (MoE) model. The backbone and adapters stay frozen, and only lightweight token-level gates are trained.
At inference, each token is routed top-1 to a single capability adapter. This lets the model switch experts mid-trajectory.
How TRACE Works — Interactive Explainer
#trace-explainer * { margin:0; padding:0; box-sizing:border-box; }
#trace-explainer {
–bg:#ffffff; –panel:#f7f8fc; –panel2:#eef1f8; –ink:#151a2e; –muted:#5a6379;
–line:#e2e6f0; –indigo:#4f46e5; –cyan:#0891b2; –amber:#d97706; –rose:#db2777;
–green:#16a34a; –red:#dc2626; –shadow:0 6px 22px rgba(30,40,90,.08);
font-family:-apple-system,BlinkMacSystemFont,”Segoe UI”,Roboto,Helvetica,Arial,sans-serif;
color:var(–ink); background:var(–bg); max-width:960px; margin:0 auto; padding:22px 18px 30px;
line-height:1.5; -webkit-font-smoothing:antialiased;
}
#trace-explainer .te-eyebrow{ font-size:12px; font-weight:700; letter-spacing:.14em; text-transform:uppercase; color:var(–indigo); }
#trace-explainer h1{ font-size:26px; line-height:1.2; margin:6px 0 6px; letter-spacing:-.02em; }
#trace-explainer .te-sub{ color:var(–muted); font-size:14.5px; max-width:640px; }
/* stepper */
#trace-explainer .te-steps{ display:flex; gap:8px; margin:20px 0 14px; flex-wrap:wrap; }
#trace-explainer .te-step{
flex:1 1 160px; min-width:150px; text-align:left; cursor:pointer; border:1px solid var(–line);
background:var(–panel); border-radius:12px; padding:11px 13px; transition:.25s; position:relative; overflow:hidden;
}
#trace-explainer .te-step:hover{ border-color:#c7cee0; }
#trace-explainer .te-step.on{ background:#fff; border-color:var(–indigo); box-shadow:var(–shadow); }
#trace-explainer .te-step.on::before{ content:””; position:absolute; left:0; top:0; height:3px; width:100%;
background:linear-gradient(90deg,var(–indigo),var(–cyan)); }
#trace-explainer .te-step .te-num{ font-size:11px; font-weight:700; color:var(–muted); }
#trace-explainer .te-step.on .te-num{ color:var(–indigo); }
#trace-explainer .te-step .te-t{ font-size:13.5px; font-weight:650; margin-top:3px; }
/* stage */
#trace-explainer .te-stage{
border:1px solid var(–line); border-radius:16px; background:linear-gradient(180deg,#fff,#fbfcff);
box-shadow:var(–shadow); padding:18px; min-height:340px; position:relative;
}
#trace-explainer .te-caption{ display:flex; align-items:center; gap:9px; margin-bottom:14px; }
#trace-explainer .te-dot{ width:10px; height:10px; border-radius:50%; background:var(–indigo); box-shadow:0 0 0 4px rgba(79,70,229,.12); }
#trace-explainer .te-caption h3{ font-size:16px; }
#trace-explainer .te-caption p{ font-size:13px; color:var(–muted); }
#trace-explainer .te-view{ display:none; animation:tefade .4s ease; }
#trace-explainer .te-view.show{ display:block; }
@keyframes tefade{ from{opacity:0; transform:translateY(6px);} to{opacity:1; transform:none;} }
/* controls */
#trace-explainer .te-ctrl{ display:flex; gap:8px; align-items:center; margin-top:16px; flex-wrap:wrap; }
#trace-explainer button.te-btn{
font:inherit; font-size:13px; font-weight:600; cursor:pointer; border:1px solid var(–line);
background:#fff; color:var(–ink); border-radius:9px; padding:8px 14px; transition:.2s;
}
#trace-explainer button.te-btn:hover{ border-color:var(–indigo); color:var(–indigo); }
#trace-explainer button.te-btn.primary{ background:var(–indigo); color:#fff; border-color:var(–indigo); }
#trace-explainer button.te-btn.primary:hover{ background:#4338ca; color:#fff; }
#trace-explainer .te-hint{ font-size:12px; color:var(–muted); }
/* step1 trajectories */
#trace-explainer .te-traj-wrap{ display:grid; grid-template-columns:1fr 1.1fr; gap:16px; }
#trace-explainer .te-pool{ border:1px dashed var(–line); border-radius:12px; padding:10px; background:var(–panel); }
#trace-explainer .te-pool h4{ font-size:12px; text-transform:uppercase; letter-spacing:.08em; color:var(–muted); margin-bottom:8px; }
#trace-explainer .te-traj{ display:flex; align-items:center; gap:7px; font-size:12px; padding:6px 8px; border-radius:8px;
margin-bottom:6px; background:#fff; border:1px solid var(–line); opacity:0; transform:translateX(-8px); }
#trace-explainer .te-traj.in{ opacity:1; transform:none; transition:.4s; }
#trace-explainer .te-traj .te-tick{ width:16px; height:16px; border-radius:50%; display:grid; place-items:center; color:#fff; font-size:10px; font-weight:800; flex:none; }
#trace-explainer .te-ok{ background:var(–green); }
#trace-explainer .te-no{ background:var(–red); }
#trace-explainer .te-caps h4{ font-size:12px; text-transform:uppercase; letter-spacing:.08em; color:var(–muted); margin-bottom:10px; }
#trace-explainer .te-cap{ margin-bottom:12px; }
#trace-explainer .te-cap .te-cn{ display:flex; justify-content:space-between; font-size:12.5px; font-weight:600; margin-bottom:4px; }
#trace-explainer .te-cap .te-cn small{ color:var(–muted); font-weight:500; }
#trace-explainer .te-bar{ height:9px; border-radius:6px; background:var(–panel2); overflow:hidden; }
#trace-explainer .te-bar > i{ display:block; height:100%; width:0; border-radius:6px; transition:width 1.1s cubic-bezier(.2,.7,.2,1); }
#trace-explainer .te-keep{ font-size:11px; font-weight:700; margin-left:6px; padding:1px 6px; border-radius:20px; opacity:0; transition:.4s; }
#trace-explainer .te-keep.y{ background:rgba(22,163,74,.12); color:var(–green); opacity:1; }
/* step2 env cards */
#trace-explainer .te-envgrid{ display:grid; grid-template-columns:repeat(2,1fr); gap:12px; }
#trace-explainer .te-env{ border:1px solid var(–line); border-radius:12px; padding:12px; background:#fff; position:relative;
opacity:0; transform:scale(.94); }
#trace-explainer .te-env.in{ opacity:1; transform:none; transition:.45s; }
#trace-explainer .te-env .te-etag{ font-size:11px; font-weight:700; color:#fff; padding:2px 8px; border-radius:20px; display:inline-block; }
#trace-explainer .te-env h5{ font-size:13.5px; margin:8px 0 3px; }
#trace-explainer .te-env code{ font-size:11px; color:var(–muted); background:var(–panel); padding:2px 6px; border-radius:5px; display:inline-block; margin-top:2px;}
#trace-explainer .te-env .te-seed{ font-size:11px; color:var(–muted); margin-top:8px; display:flex; gap:6px; align-items:center; }
#trace-explainer .te-seedpill{ font-family:ui-monospace,Menlo,monospace; background:var(–panel2); padding:1px 6px; border-radius:5px; }
/* step3 grpo */
#trace-explainer .te-grpo{ display:grid; grid-template-columns:1.2fr 1fr; gap:18px; align-items:center; }
#trace-explainer .te-group{ border:1px solid var(–line); border-radius:12px; padding:12px; background:var(–panel); }
#trace-explainer .te-roll{ display:flex; align-items:center; gap:9px; margin-bottom:8px; font-size:12px; }
#trace-explainer .te-rollbar{ flex:1; height:12px; border-radius:6px; background:#fff; border:1px solid var(–line); overflow:hidden; }
#trace-explainer .te-rollbar > i{ display:block; height:100%; width:0; background:linear-gradient(90deg,var(–cyan),var(–indigo)); transition:width .8s; }
#trace-explainer .te-rew{ width:34px; text-align:right; font-family:ui-monospace,Menlo,monospace; font-size:11.5px; color:var(–muted); }
#trace-explainer .te-adapter{ text-align:center; }
#trace-explainer .te-lora{ width:130px; height:130px; margin:0 auto 8px; border-radius:16px; border:2px solid var(–indigo);
display:grid; place-items:center; position:relative; background:linear-gradient(180deg,#fff,#f2f0ff); overflow:hidden; }
#trace-explainer .te-lora .te-fill{ position:absolute; bottom:0; left:0; width:100%; height:0; background:rgba(79,70,229,.14); transition:height 1s; }
#trace-explainer .te-lora b{ font-size:22px; color:var(–indigo); position:relative; }
#trace-explainer .te-lora span{ font-size:10.5px; color:var(–muted); position:relative; }
#trace-explainer .te-frozen{ font-size:11px; color:var(–muted); }
/* step4 routing */
#trace-explainer .te-route{ display:grid; grid-template-columns:1fr; gap:14px; }
#trace-explainer .te-tokline{ display:flex; flex-wrap:wrap; gap:6px; min-height:34px; padding:8px; border:1px solid var(–line); border-radius:10px; background:var(–panel); }
#trace-explainer .te-tok{ font-size:12px; padding:4px 9px; border-radius:7px; background:#fff; border:1px solid var(–line); transition:.3s; }
#trace-explainer .te-tok.route{ color:#fff; transform:translateY(-2px); box-shadow:var(–shadow); }
#trace-explainer .te-experts{ display:grid; grid-template-columns:repeat(4,1fr); gap:10px; }
#trace-explainer .te-exp{ border:1px solid var(–line); border-radius:11px; padding:11px 9px; text-align:center; background:#fff; transition:.25s; }
#trace-explainer .te-exp .te-eicon{ width:30px; height:30px; border-radius:9px; margin:0 auto 6px; display:grid; place-items:center; color:#fff; font-weight:800; font-size:13px; }
#trace-explainer .te-exp .te-en{ font-size:11.5px; font-weight:600; line-height:1.25; }
#trace-explainer .te-exp .te-ec{ font-size:11px; color:var(–muted); margin-top:4px; font-family:ui-monospace,monospace; }
#trace-explainer .te-exp.hot{ transform:translateY(-3px); box-shadow:var(–shadow); }
#trace-explainer .te-tasks{ display:flex; gap:8px; flex-wrap:wrap; }
/* results */
#trace-explainer .te-results{ margin-top:22px; border:1px solid var(–line); border-radius:16px; padding:18px; background:var(–panel); }
#trace-explainer .te-results h3{ font-size:15px; margin-bottom:3px; }
#trace-explainer .te-results p.te-rp{ font-size:12.5px; color:var(–muted); margin-bottom:14px; }
#trace-explainer .te-rrow{ display:flex; align-items:center; gap:10px; margin-bottom:11px; font-size:12.5px; }
#trace-explainer .te-rlabel{ width:92px; font-weight:600; flex:none; }
#trace-explainer .te-rtrack{ flex:1; height:22px; background:#fff; border:1px solid var(–line); border-radius:6px; overflow:hidden; position:relative; }
#trace-explainer .te-rtrack > i{ display:block; height:100%; width:0; border-radius:6px 0 0 6px; transition:width 1.3s cubic-bezier(.2,.7,.2,1); }
#trace-explainer .te-rval{ position:absolute; right:8px; top:50%; transform:translateY(-50%); font-size:11.5px; font-weight:700; font-family:ui-monospace,monospace; }
#trace-explainer .te-foot{ margin-top:16px; font-size:11.5px; color:var(–muted); text-align:center; }
#trace-explainer .te-foot b{ color:var(–indigo); }
#trace-explainer .te-src{ color:var(–indigo); text-decoration:none; }
@media(max-width:640px){
#trace-explainer .te-traj-wrap{ grid-template-columns:1fr; }
#trace-explainer .te-grpo{ grid-template-columns:1fr; }
#trace-explainer .te-envgrid{ grid-template-columns:1fr; }
#trace-explainer .te-experts{ grid-template-columns:repeat(2,1fr); }
#trace-explainer h1{ font-size:22px; }
}
Interactive Explainer
How TRACE Turns Agent Failures Into Targeted Training
TRACE diagnoses the capabilities an agent lacks, builds one verifiable environment per gap, trains a LoRA expert for each, then routes tokens across experts. Step through the pipeline below.
1 · Contrastive Capability AnalysisSplit rollouts into pass / fail, then keep gaps that separate them.
Passed (D⁺)
Failed (D⁻)
Retained if Δ ≥ 0.20 and Cov ≥ 0.10
2 · Targeted Environment SynthesisOne seeded, auto-verifiable environment is generated per capability.
3 · Capability Adapter Training (GRPO)Rollouts share a seed; rewards are normalized within the group.
0%LoRA Δc (~5.3%)
Base model frozen · only Δc updates
4 · MoE Composition · Token-Level RoutingA learned gate routes each token top-1 to a single capability expert.
Pick a task above to route its tokens.
Play step
Next step →
Result · τ²-Bench overall pass rate (Qwen3-30B-A3B)
Targeted training and MoE composition beat prompt optimization and single-adapter baselines.
Built from arXiv:2604.05336 · code. Numbers are from the paper. • Marktechpost
(function(){
var root = document.getElementById(‘trace-explainer’);
var CAPS = [
{name:’Structured data reasoning’, short:’SDR’, color:’#4f46e5′, delta:0.44, cov:0.41, code:’parse / cross-reference JSON records’},
{name:’Multi-step task completion’, short:’MSC’, color:’#0891b2′, delta:0.33, cov:0.25, code:’finish every sub-task, no early stop’},
{name:’Precondition verification’, short:’PV’, color:’#d97706′, delta:0.28, cov:0.16, code:’check policy before state change’},
{name:’Tool calling precision’, short:’TCP’, color:’#db2777′, delta:0.24, cov:0.20, code:’pass the correct arguments’}
];
var STEPS = [‘Capability Analysis’,’Environment Creation’,’Adapter Training (GRPO)’,’MoE Routing’];
var HINTS = [
‘Trajectories stream in, then each capability’s Δ (contrastive gap) and Cov (failure coverage) fill.’,
‘A distinct seeded environment is synthesized for each retained capability.’,
‘Watch grouped rollouts get normalized, then the LoRA adapter train to convergence.’,
‘Click a task to see its tokens routed top-1 to the matching expert.’
];
var cur = 0;
// Build stepper
var stepsEl = document.getElementById(‘teSteps’);
STEPS.forEach(function(s,i){
var d = document.createElement(‘div’);
d.className=’te-step’+(i===0?’ on’:”);
d.innerHTML=’STEP ‘+(i+1)+’