three.dev vs Braintrust (2026): which one fits your AI feature?

The short answer

Pick three.dev if you have a live AI feature, no eval suite, and a coding agent such as Claude Code, Codex, or Cursor. Braintrust's evals start from datasets and scorers you build, and its failure discovery starts when someone turns it on in each project. three.dev starts from your traffic. Change one base URL, and it scores a sample of live requests, writes up the failure modes it sees with a severity on each, and hands them to your coding agent.

Pick Braintrust if you want a mature eval and observability platform: datasets, scorers, experiments, production tracing, and CI checks. If you already run Braintrust, it can group your production traces with Topics and look for recurring problems with Patterns, once someone with the right permissions turns them on in each project.

At a glance

What each product is

Braintrust describes itself as "the active observability platform for instrumenting, understanding, and improving agents". You trace your application with its SDKs, build datasets, write scorers (autoevals, LLM-as-a-judge, or custom code), run evals as experiments, and compare runs. Online scoring rules score production traces in the background. Loop, Braintrust's AI agent, investigates logs and experiments, and it can create or edit prompts, scorers, datasets, facets, dashboards, and automations. Topics and Patterns look for problems that no scorer measures yet.

three.dev looks for failures in the requests themselves, with no dataset or scorer in between. It sits on your LLM traffic as a proxy at gate.three.dev:

1. Connect. Change the base URL to gate.three.dev, send a three.dev API key, and tag the use case with the X-Three-Use-Case header.
2. Find. three.dev scores a share of live requests in the background, after the reply is sent. An issue records each defect, the reason for it, and the quoted lines that show it. Enough issues and three.dev draws up failure modes, with a title, a description, and a high, medium, or low severity, listed from most to least severe. Anything that fits no failure mode lands in Unmatched.
3. Manage. Your team can raise or lower a severity, describe a new failure mode in plain words, split or merge existing ones, or dismiss one that does not matter.
4. Fix. Over the three.dev MCP server, Claude Code, Codex, or Cursor reads the failure modes, the failing requests behind them, and the docs.
5. Compare. Tag requests with a release. The Overview then compares two releases or two date ranges on quality, latency, and cost.

Chat sits beside all of this: ask a question about your traffic in plain words, and the answer links to the requests it came from.

Where Braintrust is the better choice

• You already use Braintrust. Topics already groups your production traces into clusters, and Patterns looks for recurring problems and orders them by severity, so you may already have the first thing three.dev does.
• You need quality gates in CI. Braintrust's GitHub Action runs evals on every pull request, posts the results as a comment, and can gate merges on them. three.dev has no CI gates today.
• You want to build and run eval datasets. Braintrust's datasets, scorers, experiments, and playgrounds are deep, and it covers multi-turn chat evals.
• Your team works in many languages and frameworks. Braintrust ships SDKs for TypeScript, Python, Go, Java, Ruby, and C#, accepts OpenTelemetry, and integrates with LangChain, LangGraph, the Vercel AI SDK, the OpenAI Agents SDK, Google ADK, Mastra, Pydantic AI, LlamaIndex, CrewAI, and the Claude Agent SDK.
• You need alerts on new problems. Braintrust sends alerts to Slack, webhooks, or PagerDuty, and Patterns can post each new pattern to Slack. three.dev has no alerts today.
• You want an agent that writes scorers or opens the pull request. Loop can draft scorers and prompts, and with GitHub connected (public preview), it can push a change to a branch and open a pull request for your team to review.
• Going a little past the free plan costs a little. Braintrust's free Starter plan bills extra usage per GB and per 1,000 scores. three.dev Free has no extra usage, so the next step up is Starter at $79 a month.
• You need self-hosting or a BAA today. Enterprise can self-host the data plane or run it in your own cloud (BYOC), with SAML SSO, audit logging, SOC 2 attestation, and a BAA. three.dev has no self-hosting and does not take health data (PHI) or government workloads yet.

Where three.dev is the better choice

• You want failure modes without turning anything on. In Braintrust, Topics needs a member of the Owners group to turn it on in each project, traces that render in the Thread view, and 100 facet summaries before clusters appear. It runs daily on model credits that do not roll over. Patterns is in public preview and also runs on a schedule. three.dev scores from the first request and groups the issues itself.
• You have no eval suite and do not want to build one. There is no dataset to label and no scorer to write. Each issue gives the reason and the quoted lines, so you can check it yourself.
• You want your existing SDK to keep working. One base URL change covers OpenAI, Anthropic, Gemini, Azure OpenAI, OpenRouter, Concentrate, Amazon Bedrock, LiteLLM, the Vercel AI SDK, and LangChain.
• You do not want a model bill for scoring. Braintrust's LLM scorers run on your own model key or on its monthly model credit ($10 on Starter, $100 on Pro). three.dev's plans include its scoring calls.
• You want to see a fix hold in production. Release tags let the Overview set the fixed release beside the old one on quality, latency, and cost.

Pricing

What the same traffic costs

LLM requests a month

25K

The calls your AI feature makes to its model.

1K10M

Share of requests scored

20%

5K scored requests a month, in three.dev or in Braintrust.

1%100%

Scorers per request in Braintrust

3

In Braintrust, a scorer usually checks one concern, such as accuracy or tone, and each one is a separate score and model call. three.dev scores each request once and reports every issue it finds, so its price stays the same.

110

Model bill per 1,000 scorer calls

$9

What Braintrust's scorer calls cost on your model key. We start at $9, what 1,000 calls cost on the model three.dev scores with. three.dev's price includes its scoring.

$0$50

three.dev

$0a month on the Free plan

• Scoring and failure modes included
• Unlimited seats
• 30 days of history

Braintrust

$137.50a month on the Starter plan

• $12.50 for the plan and usage
• $125 model bill for scoring (estimate)
• Unlimited users
• 14 days of logs

How we count

• A request is about 20 KB of data (5K tokens), as on our pricing page.
• We start at 3 scorers per request. A scorer usually checks one concern, so a team likely needs several to cover what three.dev checks in one pass. Braintrust's own pricing calculator uses the same number: it counts 3 scores for each sampled request. If you write one scorer that looks for any problem, set the slider to 1. We count each scorer as an LLM call; scorers written as code have no model bill.
• Processed data uses the same size per request as three.dev. Apps that log several steps per request process more data. Each scorer records 1 score on each scored request.
• Starter is free with 1 GB and 10k scores, then $4 a GB and $2.50 per 1,000 scores. Pro is $249 with 5 GB and 50k scores, then $3 a GB and $1.50 per 1,000 scores.
• LLM scorers run on your own model key or on Braintrust's built-in models, which draw on a monthly model credit ($10 on Starter, $100 on Pro). We take the credit off the model bill.
• The model bill is our estimate at the rate you set. $9 per 1,000 is the price of the model three.dev scores with ($2 in, $10 out per million tokens) at about 3,500 tokens in and 200 out per scorer call.

Automatic failure discovery. Braintrust's Topics and Patterns find problems on their own once someone turns them on in each project. They draw on the same monthly model credit. three.dev includes it in every plan.

Braintrust prices checked October 29, 2026: Braintrust pricing, Plans and limits.

Braintrust pricing:

Plan	Price	Included	Overage
Starter	$0 a month	$10 in model credits, 1 GB processed data, 10k scores, 14-day log retention, unlimited users	$4 per GB, $2.50 per 1k scores
Pro	$249 a month	$100 in model credits, 5 GB processed data, 50k scores, 30-day log retention, unlimited users, custom charts, environments, RBAC, priority support, click-through DPA	$3 per GB, $1.50 per 1k scores, $0.50 per GB a month for retention past 30 days, up to 180 days
Enterprise	Custom	Custom retention and export, on-prem or hosted deployment, SAML SSO, custom RBAC roles, DPA, BAA, uptime SLA	Custom

Starter

Price$0 a month

Included$10 in model credits, 1 GB processed data, 10k scores, 14-day log retention, unlimited users

Overage$4 per GB, $2.50 per 1k scores

Pro

Price$249 a month

Included$100 in model credits, 5 GB processed data, 50k scores, 30-day log retention, unlimited users, custom charts, environments, RBAC, priority support, click-through DPA

Overage$3 per GB, $1.50 per 1k scores, $0.50 per GB a month for retention past 30 days, up to 180 days

Enterprise

PriceCustom

IncludedCustom retention and export, on-prem or hosted deployment, SAML SSO, custom RBAC roles, DPA, BAA, uptime SLA

OverageCustom

Processed data is measured at ingestion, so deleting data does not lower it. Experiments keep up to 365 days on every plan. Model credits cover AI features such as Topics and any built-in models you use, and unused credits do not roll over. On Starter without a card, Topics pauses when the monthly credit runs out. Qualifying startups (new customers at Series A or earlier with at least $100K raised) can apply for 6 to 12 months of Pro free.

three.dev plans: Free, Starter ($79 a month), and Pro ($249 a month). Each includes scoring and unlimited seats. While three.dev is in early access, you can read the plan limits or request early access for your own traffic.

Our recommendation

If you already run Braintrust, or you need CI quality gates, eval datasets, and production tracing across a large engineering team, choose Braintrust. It is the more complete tool for that job. Turn on Topics and Patterns in your projects before you look elsewhere.

If you have a feature in production, no datasets or scorers, and a coding agent that writes most of your code, choose three.dev. It works from the traffic you already have and gives the agent each failure mode with its evidence.

Key facts

• three.dev: a proxy for LLM traffic that scores a sample of live requests in the background, explains each issue with quoted lines, and groups issues into failure modes ranked by severity. Teams can change a severity and create, split, merge, or dismiss failure modes. Chat answers questions about the traffic. Early access.
• Braintrust: an eval and observability platform with datasets, scorers, experiments, CI checks, and alerts. Free Starter plan, Pro at $249 a month, custom Enterprise.
• Failure discovery: Braintrust's Topics and Patterns run once someone turns them on in each project, on a daily schedule and model credits. three.dev's runs with no switch.
• three.dev prices: Free $0, Starter $79, Pro $249 a month, with scoring in the price.
• What three.dev lacks today: CI gates, alerts, datasets, experiments, multi-turn evals, and self-hosting. No PHI or government workloads yet.

Sources

• Braintrust pricing
• Plans and limits
• Topics
• Enable Topics
• Review Topics
• Patterns
• Enable Patterns
• Review Patterns
• Loop
• Loop GitHub integration
• Online scoring
• Run evals in CI
• SDKs
• MCP server
• Alerts
• Self-hosting
• Deployment options
• Feature lifecycle
• Braintrust llms.txt
• LangSmith vs Braintrust
• Evaluating a chat assistant
• Cookbook: production issues with Topics
• Braintrust changelog
• three.dev docs: AI providers

FAQ