three.dev vs LangSmith (2026): which one fits your AI feature?

The short answer

Pick three.dev if you have a live AI feature, no eval or observability tool yet, and a coding agent such as Claude Code, Codex, or Cursor, and you would rather not move your stack onto one platform to learn how your feature fails. There is no SDK to add, no plan upgrade to buy, and no eval suite to write. Send your LLM calls through three.dev, and it scores a sample of them, gathers the issues into failure modes with a severity on each, and gives them to your own coding agent.

Pick LangSmith if you want one platform for the whole agent lifecycle: tracing, evals, CI checks, human review, agent deployment, and alerts, with self-hosting on the Enterprise plan. If you already run LangSmith on Plus or Enterprise, its Engine finds recurring failures in your traces, ranks them by severity, and drafts the fix. Engine works inside LangSmith: your traces go through its SDK or OpenTelemetry, it needs a Plus or Enterprise plan, and each run is metered.

At a glance

What each product is

LangSmith is LangChain's platform for building and running agents. In its own words: "LangSmith is the framework agnostic agent engineering platform for observing, evaluating, and deploying agents". You trace your agent with the LangSmith SDK or OpenTelemetry, score traffic with online evals (LLM-as-judge or code, including multi-turn evaluators that score whole conversations), and run offline evals on datasets. Engine, which LangSmith calls "your proactive agent engineer", audits your traces, clusters issues and prioritizes them by severity, proposes prompt or code fixes, and reopens an issue if it comes back. LangSmith also deploys and runs agents for you.

three.dev reads the LLM traffic your feature already sends, so nothing moves to a new platform. Requests pass through a proxy at gate.three.dev:

1. Connect. Set the base URL to gate.three.dev, authenticate with a three.dev API key, and name the use case in the X-Three-Use-Case header. Keep your SDK and your framework, LangChain included.
2. Find. A sample of requests gets scored in the background once each response is back. three.dev reports each defect as an issue, says why the request failed, and quotes the lines that show it. When enough issues collect, it writes a title and a description for each failure mode, rates its severity high, medium, or low, and sorts the most severe to the top. Leftover issues sit in Unmatched.
3. Manage. Severity is three.dev's call to start with, and your team can override it. The team can also add a failure mode described in plain language, split one in two, merge two into one, or dismiss one.
4. Fix. Your own coding agent does the work. The Claude Code and Codex plugins install the three.dev MCP server, which gives read access to failure modes, failing requests, conversations, and the docs; Cursor takes the URL in its config.
5. Compare. Put a release tag on requests, then set the old and new release side by side on the Overview by quality, latency, and cost.

For questions about the traffic itself, there is Chat: ask in plain words, and each answer links to the requests behind it.

Where LangSmith is the better choice

• You already use LangSmith on Plus or Enterprise. Engine already finds recurring failures in your traces, ranks them by severity, diagnoses them, and drafts the fix, so you can turn on the first thing three.dev does inside the platform you have.
• You want the tool to write the fix and open the pull request. Engine connects to GitHub or GitLab, writes the prompt or code change, opens a pull or merge request for your review, and pushes revisions when its own commits fail CI. It never merges.
• You need alerts. LangSmith sends threshold alerts to Slack, PagerDuty, Dynatrace, or a webhook, and Engine notifies Slack, Jira, or a webhook when it finds a new issue. three.dev has no alerts today.
• You need quality gates in CI. LangSmith runs evals in pytest, Vitest, and GitHub workflows and fails the pipeline when scores drop below a threshold. three.dev has no CI gates today.
• You want one vendor for the whole agent lifecycle. Tracing, monitoring, online and offline evals, multi-turn evaluators, annotation queues, Prompt Hub and a Playground, agent deployment, sandboxes, and an LLM gateway, all in one platform.
• You need self-hosting or compliance paperwork today. Enterprise offers hybrid, BYOC, and self-hosted options, and LangSmith states SOC 2 Type II, HIPAA, and GDPR certifications. three.dev has no self-hosting and does not take health data (PHI) or government workloads yet.
• You want nothing in your request path. The LangSmith SDK sends traces asynchronously, and LangSmith says your agent keeps running if LangSmith has an incident. three.dev is a proxy, so it sits in the path of every LLM call.

What it takes to get Engine

None of this is a cost if you already run LangSmith on Plus or Enterprise. If you do not, this is what Engine asks of you, from LangSmith's own docs and pricing page as of October 29, 2026:

• Your traces in LangSmith. Engine scans LangSmith tracing projects, so every LLM call first goes through the LangSmith SDK or OpenTelemetry.
• A paid plan, plus usage. The pricing page lists Engine on Plus and Enterprise; the free Developer plan shows it as N/A. On Plus ($39 per seat a month), Engine bills in LSUs on top of the license: about 7 to 45 LSUs ($7 to $45) a run, with runs scheduled every 6 hours. A default org-wide cap of 750 LSUs ($750) a month pauses new runs until an admin raises it or the next billing month starts.
• Setup by an admin, then per project. An organization admin enables Engine and accepts the AI terms. Someone who can update tracing projects turns it on for each project and picks an analysis level and optional focus areas. Connecting a GitHub or GitLab repository is optional but recommended.
• A wait and an approval. The first analysis can take up to 20 minutes. Engine writes an agent overview, and issues appear after you accept it.
• LangChain's models on Cloud. On LangSmith Cloud, Engine runs on LangChain-managed models and keys. Self-hosted and BYOC deployments can use their own model providers.
• LangSmith Cloud deployments to replay a fix. Fix validation is in private beta and supports only agents running on LangSmith Cloud deployments.

three.dev asks for one base URL change. Your AI SDK, your framework, and your coding agent stay the same.

Where three.dev is the better choice

• You want results without an admin, a plan upgrade, or a per-project switch. There is no overview to approve either. Send requests through three.dev, and scoring starts on them. Once enough issues collect, they group into failure modes, most severe first.
• You are not on LangSmith and do not want to rebuild around it. three.dev works with the AI SDK and framework you already use. Your coding agent writes the fix in your codebase.
• You would rather change a setting than add a client library. Point the base URL at gate.three.dev and add a key and a header. Documented frameworks include LangChain and the Vercel AI SDK.
• Your coding agent is Codex. LangSmith's docs say the Codex CLI does not work with its Remote MCP, and they point Codex users to the LangSmith CLI. three.dev ships a Codex plugin next to the Claude Code one, and a "Fix with your agent" button starts the agent on one failure mode with its failing requests.
• You want scoring and seats in one plan price. Every three.dev plan includes scoring and unlimited seats, from Starter at $79 a month. LangSmith Plus charges $39 per seat, counting invited users, and meters Engine runs on top.
• You want production proof that a fix held. Tag the release that carries the fix, and three.dev's Overview shows it against the release before on quality, latency, and cost.

Pricing

What the same traffic costs

LLM requests a month

25K

The calls your AI feature makes to its model.

1K10M

Share of requests scored

20%

5K scored requests a month, in three.dev or in LangSmith.

1%100%

Scorers per request in LangSmith

3

In LangSmith, a scorer usually checks one concern, such as accuracy or tone, and each one is a separate score and model call. three.dev scores each request once and reports every issue it finds, so its price stays the same.

110

Team members

3

Everyone who signs in.

1100

Model bill per 1,000 scorer calls

$9

What LangSmith's scorer calls cost on your model key. We start at $9, what 1,000 calls cost on the model three.dev scores with. three.dev's price includes its scoring.

$0$50

three.dev

$0a month on the Free plan

• Scoring and failure modes included
• Unlimited seats
• 30 days of history

LangSmith

$339.50a month on the Plus plan

• $204.50 for the plan and usage
• $135 model bill for scoring (estimate)
• 3 seats at $39 each
• 14 days; scored traces 180 days

How we count

• A request is about 20 KB of data (5K tokens), as on our pricing page.
• We start at 3 scorers per request. A scorer usually checks one concern, so a team likely needs several to cover what three.dev checks in one pass. If you write one scorer that looks for any problem, set the slider to 1. We count each scorer as an LLM call; scorers written as code have no model bill.
• Each request is 1 trace. Developer covers 5k traces and Plus 10k. Each extra trace costs $0.005.
• Developer has 1 seat. Plus costs $39 a seat, and every user counts, invited users too.
• Each scorer is an online evaluator. It runs on a model you pick, and LangSmith tracks that LLM spend against model prices you set. New evaluators move the traces they score to 180-day retention by default, at $2.50 per 1,000. You can turn that off.
• The model bill is our estimate at the rate you set. $9 per 1,000 is the price of the model three.dev scores with ($2 in, $10 out per million tokens) at about 3,500 tokens in and 200 out per scorer call.

Automatic failure discovery. Engine, which finds failures in your traces on Plus and Enterprise, is extra and not in this total: about 7 to 45 units ($7 to $45) a run, and its pricing page says it runs every 6 hours. A default cap of 750 units ($750) a month pauses the runs. three.dev includes it in every plan.

LangSmith prices checked October 29, 2026: LangSmith pricing, Usage and billing, Engine, Evaluator spend.

LangSmith pricing:

Plan	Price	Included	Usage after that
Developer	$0 a month	1 seat, 5k base traces a month, community support	Pay as you go. Without a card on file, tracing stops at 5,000 traces a month
Plus	$39 per seat a month	Unlimited seats, 10k base traces a month, Engine (metered), Insights, Deployment with 1 free small serverless deployment	Pay as you go
Enterprise	Custom	Self-hosted and hybrid options, custom SSO, ABAC and RBAC, support SLA, custom seats and workspaces	Custom, drawn from committed credits

Developer

Price$0 a month

Included1 seat, 5k base traces a month, community support

Usage after thatPay as you go. Without a card on file, tracing stops at 5,000 traces a month

Plus

Price$39 per seat a month

IncludedUnlimited seats, 10k base traces a month, Engine (metered), Insights, Deployment with 1 free small serverless deployment

Usage after thatPay as you go

Enterprise

PriceCustom

IncludedSelf-hosted and hybrid options, custom SSO, ABAC and RBAC, support SLA, custom seats and workspaces

Usage after thatCustom, drawn from committed credits

LangSmith meters usage in LangChain Standard Units (LSUs), and 1 LSU costs $1. Extra traces cost 0.005 LSU each, which is $5 per 1,000 base traces. Base traces keep 14 days. Upgrading a trace to 180-day retention costs 0.0025 LSU more, which is $2.50 per 1,000 traces, and new online evaluators and automation rules turn that upgrade on by default. Enterprise and self-hosted traces bill at a flat 0.0075 LSU each unless the contract sets other rates. Seats count every user, including invited users. Engine bills separately: about 7 to 45 LSUs a run, with a default org-wide cap of 750 LSUs ($750) a month. Startups at Series A or earlier and backed by a LangChain partner VC can get $10,000 in credits for one year.

three.dev charges by plan, with no seat fee: Free, then Starter at $79 or Pro at $249 a month, scoring included. It is in early access. The plan details list each limit; to run it on your traffic, request early access.

Our recommendation

If you already run LangSmith, or you want tracing, evals, CI gates, alerts, and agent deployment from one vendor, choose LangSmith. If you are on Plus or Enterprise, turn on Engine before you look elsewhere: it finds, ranks, and drafts fixes for recurring failures inside the platform you already pay for.

If you have no eval or observability tool, your users tell you when the feature breaks, and you would rather not rebuild around a platform to find out why, choose three.dev. Your SDK and framework stay put, and your own coding agent gets the failure modes with their evidence.

Key facts

• three.dev: scores a sample of the LLM calls that pass through its proxy, explains each issue with quoted lines, and sorts the issues into failure modes by severity. The team can override a severity and create, split, merge, or dismiss failure modes. Coding agents read them over a read-only MCP server. Early access.
• LangSmith: LangChain's agent engineering platform for tracing, evals, deployment, and more. Developer is free for 1 seat, Plus is $39 per seat a month, Enterprise is custom, and usage bills in LSUs at $1 each.
• Engine: on Plus and Enterprise, scans LangSmith traces on a schedule, ranks issues by severity, and drafts fixes. About 7 to 45 LSUs a run, capped at 750 LSUs a month by default.
• three.dev prices: Free $0, Starter $79, Pro $249 a month, with no per-seat charge.
• Not in three.dev today: alerts, CI gates, prompt management, datasets, experiments, self-hosting, and agent deployment. No PHI or government workloads yet.

Sources

• LangSmith pricing
• LangSmith Observability
• LangSmith Evaluation
• LangSmith platform
• LangSmith Engine
• Engine docs
• Engine setup
• Engine issues
• Engine notifications
• Engine fix validation
• Insights
• Usage and billing
• Multi-turn online evaluations
• Alerts
• Remote MCP
• Self-hosted LangSmith
• Enterprise
• LangSmith changelog
• Startup program
• three.dev docs: AI providers

FAQ