01
Record
One decorator captures every LLM call, tool call, and error. Now a production failure becomes a permanent regression test you can re-run.
@retrace.record()CI for AI agent behavior
Retrace records every model and tool call your agent makes, so you can re-run a failure, branch from the exact step it went wrong, and check your fix before you ship.
// the core loop
From a single decorator to a verdict. This is the reliability loop Retrace owns, end to end.
One decorator captures every LLM call, tool call, and error. Now a production failure becomes a permanent regression test you can re-run.
@retrace.record()Re-run any recorded run, or fork from the exact step it broke and trace what actually caused it.
Change the step that broke (the prompt, a tool input, or the model) and re-run the fork to see the corrected path before you ship.
Re-run a change against the failed run and get a verdict before you ship.
↻ every run feeds the next. record, fix, repeat.
// see it for real
The actual trace UI shows a recorded incident-RCA run that stopped at the OOM symptom and was marked failed, then forked from that step and traced the real cause: a deploy that raised the batch size 50×. Scrub the timeline, open any span, watch the fork diverge.
6 spans · 1 error at step 4 · forked at step 4 · fix verified
Interactive. A real trace exported from the product (incident-rca · INC-4835). Spans, costs and timings are the recorded values.
// the platform
Detect failures, enforce limits, evaluate quality, and understand multi-agent systems, on every recorded run.
Flag groundedness gaps, statistical drift, failure clusters, and MAST failure types automatically — so you learn why a run failed, before a user does, not just that it did.
Halt a run at a budget, loop, or step limit, and block a bad action before it runs via a pre-call gateway with hold-for-approval — so a bug can't quietly cascade into a cloud bill.
Quality you can gate on — evaluations, auto eval-rules, datasets, and prove-the-fix verdicts, plus a CI gate so a regression fails the build instead of reaching users.
See the whole system — sessions, agent topology, agent memory, semantic search, prompt versioning, and shareable tapes — so a multi-agent failure isn't a black box.
Flag groundedness gaps, statistical drift, failure clusters, and MAST failure types automatically — so you learn why a run failed, before a user does, not just that it did.
Halt a run at a budget, loop, or step limit, and block a bad action before it runs via a pre-call gateway with hold-for-approval — so a bug can't quietly cascade into a cloud bill.
Quality you can gate on — evaluations, auto eval-rules, datasets, and prove-the-fix verdicts, plus a CI gate so a regression fails the build instead of reaching users.
See the whole system — sessions, agent topology, agent memory, semantic search, prompt versioning, and shareable tapes — so a multi-agent failure isn't a black box.
// how it's different
Aggregate metrics & dashboards
Replay the exact run. Fork from the failed step and cascade-re-execute it to find what actually caused the failure
Search through raw JSON spans
Semantic search: describe the bug in plain language and jump to the failing run in seconds
No way to test a fix without re-running everything
Prove-the-fix: re-run a change against the failed run and get a verdict before you ship
Static alert thresholds
Guardrails and circuit breakers that halt a run when it blows a budget or loops, so a bug can't quietly run up your bill
Post-mortem analysis only
Runtime enforcement: block a bad action before it runs instead of diagnosing it afterward
Tied to one framework
One decorator works with any Python or TS agent (LangChain, CrewAI, LlamaIndex). No lock-in
// how it works
Sign in with GitHub, copy ~3 lines, run your agent. The first trace streams in live. No infrastructure to manage.
Install the SDK and add one decorator. Calls to OpenAI, Anthropic and Gemini are captured automatically.
Framework-agnostic. Works with LangChain, CrewAI, and LlamaIndex.
import retraceretrace.configure(api_key="rt_...")@retrace.record(name="my-agent")def run_agent(prompt): return agent.invoke(prompt)Run your agent. Every LLM call, tool call, cost and error streams onto the timeline as it happens.
Or watch it replay step-by-step in the dashboard.
retrace traces tailFork from the exact step that broke, change the input, re-run, then prove the fix actually worked.
verify-fix returns a verdict: improved, regressed, or unchanged.
retrace forks create --trace <id> --span <id> --input "grounded prompt"retrace forks replay <id> --waitretrace traces verify-fix <id>// ci for ai agent behavior
A prompt edit, tool change, or model upgrade can quietly break multi-step behavior. Retrace turns each production failure into a regression test and runs it as an eval gate on every PR, so the build fails when behavior breaks. It diffs each run against a golden baseline, so you catch “relevance dropped vs last release,” not just a red or green dot.
- name: Eval Gate run: retrace eval gate --evaluation $EVAL_ID --trace $TRACE_ID --threshold 0.8 env: RETRACE_API_KEY: ${{ secrets.RETRACE_API_KEY }}the closed loop
what it gates on (that unit tests can't)
The build fails when behavior breaks, so that exact failure is harder to ship again. (retrace eval gate exits 1.)
// your model account
Add your Google (Gemini) API key in Settings and the eval-gate judge and every server-side replay (fork, cascade, and prove-the-fix) call the model through your key, so the tokens are billed to your own provider account. The key is validated on save, encrypted at rest (AES-256-GCM), shown only as its last four characters, and never returned again. Remove it any time and Retrace falls back to the platform key.
what your key powers
how the key is handled
provider support
// pricing
No credit card required. Upgrade when you need more traces or AI requests.
For solo builders
For shipping
For teams
For scale
// questions
Under 2 minutes. Install the SDK, add one decorator, and traces stream immediately. No infrastructure to manage.
Python and TypeScript SDKs with auto-instrumentation for OpenAI, Anthropic, and Google Gemini. Works with any agent framework: LangChain, CrewAI, Vercel AI SDK, AutoGen, LlamaIndex.
Select any span in a trace, modify its input, and Retrace cascade-replays from that point forward. Context from the fork flows into subsequent LLM calls. You get a side-by-side diff with cost and latency deltas.
Runtime policies that monitor your agent in real-time. Set cost budgets, loop detection, context overflow limits, or latency caps. When a limit is crossed, the agent receives a HALT command, so a runaway loop or budget blow-out stops at the limit instead of running up your bill.
TLS in transit, encrypted at rest. API keys are SHA-256 hashed. PII auto-redaction runs on every plan as a security baseline. Tenant isolation is enforced at the application layer. Every query is scoped per user and backed by a guardrail regression test.
Yes. The eval gate endpoint (POST /evaluations/:id/gate) returns pass/fail against a threshold. The CLI command `retrace eval gate` exits with code 1 on failure, perfect for GitHub Actions.
LangSmith focuses on tracing and observability. Retrace adds interactive fork & cascade-replay from any step, runtime guardrails that halt runaway agents, groundedness detection, and prove-the-fix verification.
Yes. Each span carries an agent id, sessions group multi-turn conversations, and an agent topology graph shows cross-agent ordering and inter-agent failure modes.
Your agent failed 3 steps before the error surfaced. Fork from the real cause instead of the symptom.
No credit card · 2-min setup