Detect and fix your agent failures
Find the failures hiding in your agent's live traffic, and fix the ones that matter first.
How it works
A short film of three.dev with checkout-copilot, an example support assistant for an online shop. Live conversations stream in, and three.dev scores every one. One reply promises a refund within 14 days while the policy says 30, and it becomes an issue: Wrong refund window. The issue joins its failure mode, Wrong refund window, in 386 of 8,124 scored requests. Fix with your agent hands it to your coding agent, which reads the failure mode from three.dev, finds the refund window hardcoded in support/assistant.ts, and wires it to the policy. The fix ships as v4, and the week before is compared with the week after. Wrong refund window drops from 386 of 8,124 to 16 of 8,029 scored requests. Ignored language request rises a little, and one new failure mode appears, Refund policy pasted in full.
How it works
Chapter 1 of 4: We score your live traffic
"Using three.dev has been a huge leap for us in providing our users the highest quality while maintaining fast responses."
Don PottingerCo-founder and CTO, LanguaTalkRead the case studySet up with one prompt.
Then start fixing your failures.
Animation: a coding agent takes the prompt, installs the three.dev skills, finds the LLM calls, routes the LLM calls through the three.dev proxy, names the use case, and confirms the build and tests pass.
Paste one prompt
Paste it into Claude Code, Codex, or Cursor. Your agent installs the three.dev skills and adds three.dev to your code.
Animation: requests travel from your app through the three.dev proxy to your AI provider (OpenAI, Anthropic, Gemini, or Azure OpenAI) and back. As each response passes the proxy, it goes to your app and to the evaluator at once, and the evaluator marks it pass or fail. Counts: 8,124 recorded, 8,124 scored.
Traffic flows through three.dev
Your requests pass through the three.dev proxy to your AI provider. three.dev sends each response to your app and to its evaluator at once, so scoring adds no latency.
Animation: single issues three.dev found arrive as a list, then each one slides into its failure mode. Counts: Wrong refund window 386; Ignored language request 104; Invented discount code 35; Unrequested bullet lists 54.
Read your failure modes
Once enough failures collect, three.dev groups them into failure modes and ranks them by severity.
Install github.com/round3ai/skills and run its three-dev-setup skill in this repository.
Every feature you need.
Evidence-backed issues
Every issue cites where it went wrong, down to one line of your system prompt or one message in a long conversation.
An assistant reply promises a refund within 14 days. three.dev underlines it as the rule violated and cites the system prompt, which says 30 days.
Failure modes
Failures group into failure modes, ranked by importance and frequency.
A failure mode table counts up, each row gets a severity, and the rows sort by severity. Wrong refund window is High and leads with 386 of 8,124 scored requests. Invented discount code is Medium and climbs above Unrequested bullet lists, which is Low, though it has fewer failures.
MCP integration
Integrate with the MCP server to supercharge your coding agent.
A coding agent in a terminal calls list_failure_modes, get_failure_mode, and get_request_conversation, and each call returns one line.
Manage failure modes
Reshape the failure modes until the list matches your team's mental model.
A person ticks Ignored language request and Replied in English, then clicks Merge. The two rows become one, Ignored language request, with 104 of 8,124.
Chat
Ask about your traffic in plain words. Answers link to the requests behind them.
In Chat, a person clicks Show me how my agent is failing. The answer names Wrong refund window, 386 of 8,124 scored requests, and links one request.
- Early access
Experiments
Know a change works before every user gets it. Try it safely on recorded traffic, then on a share of live sessions, and see how it moves quality and the business metrics you already track.
- Early access
Specialized language models (SLM)
A model trained on your own traffic and built for your use case. Built for one task, it can beat frontier models on quality, latency, and cost. See how LanguaTalk built theirs.
Ship on evidence, not vibes.
See how your agent fails. Hand the fix to your coding agent.