Compare
three.dev vs Langfuse (2026): which one fits your AI feature?
Langfuse gives you open-source tracing and evals that you set up and run. three.dev names your failure modes from live traffic with nothing to configure. Here is where each one fits.
Updated
The short answer
Pick three.dev if your AI feature is live, nobody has written an evaluator for it yet, and your team codes with Claude Code, Codex, or Cursor. In Langfuse you would first write the judge prompt and the rule that picks what to score. three.dev skips that step. You change one base URL, and it scores a sample of live requests, sorts the issues into named failure modes with the most severe on top, and passes each one to your coding agent.
Pick Langfuse if you want an open-source platform to trace, evaluate, and manage prompts for your LLM app, and you want to self-host it for free or start on a low-cost cloud plan. It covers full agent traces, evaluators you write, datasets, experiments, CI checks, and alerts.
At a glance
A live AI feature with no evaluators yet, and a team that fixes with a coding agent
Teams that want open-source tracing, evals, and prompt management they can self-host or run on a low-cost cloud plan
A proxy that scores live LLM traffic and names its failure modes
An open-source AI engineering platform: tracing, prompt management, evaluation, datasets, experiments, dashboards, and alerts
None. Scoring begins with your first requests
Add an LLM connection, write or adapt a judge prompt (or write a code evaluator), test it, and attach a rule with filters and a sampling rate
Each issue gives the reason and quotes the lines behind it. three.dev groups issues into failure modes and sets a severity your team can change
A documented error-analysis method you run
Automatic grouping (Topics) is "In progress" on the public roadmap, and GitHub release notes show it in closed beta for allowlisted Langfuse Cloud projects in October 2026
Chat answers plain questions about your traffic and links to the requests behind each answer
Halo answers questions and clusters failed observations when you ask
Base URL gate.three.dev, a three.dev API key, and one header. Works through LiteLLM and the other documented providers
The Python or TypeScript SDK, OpenTelemetry for Go, Java, .NET, Ruby, PHP, and Swift, 100+ integrations, or logging through the LiteLLM proxy. The agent skill can add the tracing for you
LLM calls and the tool calls the model asks for, grouped into sessions. Tool runs and retrieval are not separate spans
Hierarchical traces: every LLM call, tool invocation, and retrieval step
In the path, as a proxy. Scoring waits until the response has returned
The SDKs queue and batch trace data in the background, off the request path
A read-only MCP server with failure modes, failing requests, and docs, plus a Fix with your agent button
An MCP server with read and write tools, a CLI, an agent skill, and a Cursor plugin give your agent traces, scores, prompts, and datasets
Tag a release, then compare it with the one before on quality, latency, and cost
Release and version tags track cost, latency, and quality by release; experiments compare versions on a dataset
Not offered today
Yes, on every plan
Neither, today
A GitHub Action fails the build when an experiment score breaks your threshold. Alerts on observation metrics and scores go to Slack, webhooks, or GitHub Actions, 2 to 100 per organization by plan
Not open source. No self-hosting today
MIT-licensed core. The self-hosted Open Source tier is free with unlimited usage
three.dev does not train models on your traffic for anyone other than you. Not for health data (PHI) or government workloads yet
Self-hosting keeps all data in your environment; Cloud regions in the US, EU, and Japan. SOC 2 Type II and ISO 27001 audits, and a HIPAA region with a BAA
Free, Starter $79, or Pro $249 a month, each with scoring and unlimited seats
Free Hobby plan; Core $29, Pro $199, Enterprise $2,499 a month; self-hosting free
What each product is
Langfuse calls itself "an open-source AI engineering platform for agent evaluation and observability". You instrument your app with its SDKs or OpenTelemetry, then trace, score, and improve it: evaluators on production traffic or in experiments, datasets, prompt versions, annotation queues, dashboards, and alerts. Online evaluation rules pick which incoming observations to score. On Langfuse Cloud, trace-level evaluators stop producing results on November 16, 2026, when Cloud moves to v4 only. ClickHouse acquired Langfuse in January 2026; the announcement says Langfuse stays open source and self-hostable, with no planned changes to licensing.
three.dev works on the LLM calls your feature already makes, with no evaluator for you to write. Your requests go through its proxy at gate.three.dev:
- Connect. Point the base URL at gate.three.dev, swap in a three.dev API key, and add the X-Three-Use-Case header. A setup skill lets your coding agent make the change for you.
- Find. After each response returns, three.dev scores a sample of requests in the background. Every defect is an issue with its reason and the quoted lines behind it. As issues pile up, three.dev names failure modes, describes them, gives each a severity of high, medium, or low, and lists the most severe first. Issues that fit no failure mode wait in Unmatched.
- Manage. Your team can change a severity, write a new failure mode in plain language, split or merge failure modes, or dismiss one.
- Fix. Claude Code, Codex, or Cursor reads a failure mode and its failing requests over the MCP server at
https://mcp.three.dev/mcp. - Compare. Tag requests with a release, then set two releases or two date ranges side by side on quality (the share of scored requests without issues), latency (p50 and p95), and cost.
Chat lets you ask about your traffic in plain words, and its answers link to the requests behind them.
Where Langfuse is the better choice
- You want open source or self-hosting. Langfuse's core is MIT licensed, and the self-hosted Open Source tier is free with unlimited usage, including Enterprise SSO and RBAC. Langfuse Self-Hosted can run fully offline. three.dev is not open source and has no self-hosting today.
- You already run Langfuse. If you have evaluators and an error-analysis habit in place, Langfuse already gives you a failure taxonomy and the scores to track it.
- You want one platform for tracing, prompts, and evals. Langfuse covers full agent traces, prompt management, datasets, experiments, annotation queues, dashboards, and alerts. three.dev today covers failure modes and the fix loop.
- You need CI gates or alerts. Langfuse ships a GitHub Action that fails a build when an experiment score breaks your threshold, and it sends alerts to Slack, webhooks, or GitHub Actions. three.dev offers neither today.
- You need to trace tool runs and retrieval steps. Langfuse records every step of an agent run as a hierarchical trace. three.dev records each LLM call, including the tool calls the model asks for, and does not trace tool runs or retrieval steps as their own spans.
- You want nothing in your request path. Langfuse SDKs send trace data in the background. three.dev is a proxy, so your LLM requests pass through it.
- You handle regulated data. Langfuse offers SOC 2 Type II and ISO 27001 audits, a HIPAA region, and a BAA. three.dev does not take health data (PHI) or government workloads yet.
- You want the lowest paid entry price. Langfuse Core is $29 a month, and early-stage startups can get 50% off the Cloud bill for 12 months. three.dev's first paid plan, Starter, is $79 a month.
Where three.dev is the better choice
- You have no evaluator and want results anyway. In Langfuse, you add an LLM connection, write or adapt the judge prompt, test it, and attach a rule before a score appears. The failure taxonomy then comes from an error-analysis session you run, and run again as the app changes, or from asking Halo. Topics, which would group failures for you, is in closed beta as of October 2026. three.dev starts scoring with your first requests and names the failure modes itself.
- You want to check each failure before you act on it. Every three.dev issue gives the reason the request failed and quotes the lines behind it.
- You would rather change a setting than add a tracing SDK. Your AI SDK stays as it is. If you already call models through LiteLLM, three.dev documents that route too.
- You want your coding agent to start from the answer. A "Fix with your agent" button copies a prompt that starts the agent on one failure mode, and the MCP server hands it the failing requests with each issue's summary and reasoning.
- You want scoring in the plan price. Langfuse judge calls run on your own LLM connection, so your model provider bills you for them, and each score counts as a billable unit. Every three.dev plan includes scoring.
Pricing
What the same traffic costs
The calls your AI feature makes to its model.
5K scored requests a month, in three.dev or in Langfuse.
In Langfuse, a scorer usually checks one concern, such as accuracy or tone, and each one is a separate score and model call. three.dev scores each request once and reports every issue it finds, so its price stays the same.
Everyone who signs in.
What Langfuse's scorer calls cost on your model key. We start at $9, what 1,000 calls cost on the model three.dev scores with. three.dev's price includes its scoring.
three.dev
$0a month on the Free plan
- Scoring and failure modes included
- Unlimited seats
- 30 days of history
Langfuse
$164a month on the Core plan
- $29 for the plan and usage
- $135 model bill for scoring (estimate)
- Unlimited users
- 90 days of data
How we count
- A request is about 20 KB of data (5K tokens), as on our pricing page.
- We start at 3 scorers per request. A scorer usually checks one concern, so a team likely needs several to cover what three.dev checks in one pass. If you write one scorer that looks for any problem, set the slider to 1. We count each scorer as an LLM call; scorers written as code have no model bill.
- Each request is 1 trace and 1 observation, so 2 units. Each scorer adds 1 score unit to each scored request.
- Apps that trace several steps per request log more observations, so they count more units.
- Hobby covers 50k units and 2 users, and Langfuse lists no extra usage on it. Past either limit, we price Core: $29 with 100k units, then $8 per 100k up to 1M units, then less.
- Langfuse's LLM-as-a-judge calls run on your own LLM connection, so your model provider bills them.
- The model bill is our estimate at the rate you set. $9 per 1,000 is the price of the model three.dev scores with ($2 in, $10 out per million tokens) at about 3,500 tokens in and 200 out per scorer call.
Automatic failure discovery. Langfuse's automatic grouping (Topics) is in closed beta for allowlisted Cloud projects as of October 2026, so it has no public price. three.dev includes it in every plan.
Langfuse prices checked October 29, 2026: Langfuse pricing, Billable units, LLM-as-a-judge.
Langfuse Cloud pricing:
| Plan | Price | Included | Extra usage |
|---|---|---|---|
Hobby | $0 a month | 50k units a month, 30 days of data, 2 users, community support | None listed |
Core | $29 a month | 100k units a month, 90 days of data, unlimited users, in-app support | $8 per 100k units, lower with volume |
Pro | $199 a month | 100k units a month, 3 years of data, data retention management, unlimited annotation queues, SOC 2 and ISO 27001 reports | $8 per 100k units, lower with volume |
Pro + Teams | $199 + $300 a month | Enterprise SSO, SSO enforcement, fine-grained RBAC | Same as Pro |
Enterprise | $2,499 a month | Audit logs, SCIM API, custom rate limits, uptime and support SLAs | Same rates; custom volume pricing with a yearly commitment |
A unit is any trace, observation, or score stored in Langfuse, including the scores its LLM-as-a-judge evaluators create. Extra units cost $8 per 100k up to 1M units, $7 per 100k up to 10M, $6.50 per 100k up to 50M, and $6 per 100k above that. Early-stage startups (bootstrapped or up to $5M raised, and incorporated within the last 5 years) get 50% off the Cloud bill for 12 months.
Langfuse self-hosted pricing: the Open Source tier is free, with all core features and unlimited usage. Self-Hosted Enterprise is custom pricing, bundled with a ClickHouse commercial plan. Self-Hosted Government adds NIST hardening, FIPS 140-3, and air-gapped deployment. You run your own infrastructure in each case.
three.dev has three plans: Free, Starter at $79 a month, and Pro at $249 a month. Each plan includes scoring and unlimited seats. three.dev is in early access: compare its plans, or request early access and point it at your own traffic.
Our recommendation
Choose Langfuse if you want open source, self-hosting, or one platform for tracing, prompts, datasets, and evals. If your Langfuse evaluators already run, stay on it: it does more today.
Choose three.dev if nobody on your team has time to write and tune evaluators, your users are the ones reporting failures, and a coding agent already writes most of your fixes. One base URL change gets you failure modes with a severity on each, ready for the agent.
Key facts
- three.dev: a proxy at gate.three.dev that scores a sample of live LLM requests, explains each issue with quoted lines, and groups the issues into failure modes ranked by severity. Your team can change a severity, and create, split, merge, or dismiss failure modes. Early access.
- Langfuse: an open-source (MIT core) AI engineering platform for tracing, evaluation, prompt management, datasets, and experiments, owned by ClickHouse since January 2026. Automatic grouping (Topics) is in closed beta as of October 2026.
- Work before the first failure shows up: in Langfuse, an LLM connection, a judge prompt or code evaluator, and a rule. In three.dev, a base URL, an API key, and one header.
- Prices: Langfuse Cloud Hobby is free, Core $29, Pro $199, and Enterprise $2,499 a month, and self-hosting is free. three.dev Free is $0, Starter $79, and Pro $249 a month, with scoring included.
- Missing from three.dev today: alerts, CI gates, prompt management, datasets, experiments, and self-hosting. It does not take PHI or government workloads yet.
Sources
- Langfuse homepage
- Langfuse pricing
- Self-hosted pricing
- License
- Self-hosting
- Roadmap
- Topics visibility gate (GitHub)
- Topics API, closed beta (GitHub)
- Evaluate production traffic
- LLM-as-a-judge
- Billable units
- Error analysis
- Halo
- MCP server
- Agent skill
- Alerts
- Experiments in CI/CD
- Releases and versioning
- Queuing and batching
- Security
- HIPAA
- Startup program
- Langfuse v4
- Langfuse joins ClickHouse
- three.dev docs: AI providers
FAQ
Is three.dev a Langfuse alternative?
For a team with no evaluators yet that wants its failure modes found for it, yes. You change one base URL, and three.dev scores live traffic, sorts what it finds into failure modes by severity, and gives them to your coding agent. For a team that wants open source, self-hosting, prompt management, datasets, or experiments, Langfuse covers more today.
Is Langfuse free?
Langfuse is open source, and the self-hosted Open Source tier is free with unlimited usage. You run the infrastructure: Postgres, ClickHouse, Redis or Valkey, and S3-type storage. Langfuse Cloud has a free Hobby plan with 50k units a month and 2 users. Paid Cloud plans start at $29 a month (Core).
Does Langfuse find production failures automatically?
Not in a generally available feature, as of October 29, 2026. Automatic grouping, called Topics, is "In progress" on Langfuse's public roadmap, and Langfuse's GitHub release notes show it in closed beta on Langfuse Cloud for allowlisted projects in October 2026. Today Langfuse documents error analysis as a method you run, and its in-app assistant, Halo, can cluster failed observations when you ask. three.dev groups its issues into failure modes on its own, and three.dev's Chat answers questions about your traffic with links to the requests behind each answer.
What is the difference between Langfuse error analysis and three.dev failure modes?
Both end with named failure categories. Langfuse documents error analysis as a method you run: sample traces, write notes on what went wrong, let an LLM draft a taxonomy, refine the names, and measure failure rates, then repeat as your app changes. three.dev scores a sample of your live traffic, and once enough issues collect, it names the failure modes, writes their descriptions, and sets each one's severity. Nothing needs naming by hand to get started, and your team can still change a severity, create a failure mode in plain language, or split, merge, or dismiss one.
Langfuse vs LangSmith: which is better?
Langfuse is open source (MIT core), free to self-host, and bills its Cloud plans by units. LangSmith charges $39 per seat a month on its Plus plan, offers self-hosting as an add-on to its Enterprise plan, and includes Engine on Plus and Enterprise, which looks through your traces on a schedule, orders the problems it finds by severity, and proposes fixes. Pick Langfuse for open source and cost. Pick LangSmith for one vendor from tracing to agent deployment, or for Engine.
Is Langfuse still open source after ClickHouse acquired it?
Yes, as of October 29, 2026. ClickHouse acquired Langfuse in January 2026, and the announcement says there are no planned licensing changes. The repository's core is MIT licensed; code in its ee folders is under a separate commercial license.
Does Langfuse work with coding agents like Claude Code or Cursor?
Yes. Langfuse ships an MCP server with read and write tools, a CLI, and an agent skill that works with Claude Code, Cursor, Windsurf, and other agents, plus a Cursor plugin. three.dev's MCP server is read-only: Claude Code, Codex, Cursor, and other clients can read failure modes and their failing requests, and a Fix with your agent button copies a prompt that starts the agent on one failure mode.
Do I have to write an evaluator to use three.dev?
No. three.dev scores your requests itself, so there is no prompt, rule, or LLM connection for you to set up. You change the base URL to gate.three.dev, use a three.dev API key, and add the X-Three-Use-Case header. Or paste this prompt into your coding agent and let it make the change: "Install github.com/round3ai/skills and run its three-dev-setup skill in this repository."
Langfuse stays out of the request path. Does three.dev?
No. three.dev is a proxy, so your LLM requests pass through gate.three.dev. Scoring waits until the response has returned, so it never changes a request or adds latency to it. three.dev does not train models on your traffic for anyone other than you.
How much does Langfuse cost compared to three.dev?
Langfuse Cloud has a free Hobby plan, Core at $29 a month, Pro at $199 a month, and Enterprise at $2,499 a month, plus $8 per 100k units above the 100k included on paid plans. Self-hosting the open-source tier is free. three.dev costs $0 on Free, $79 a month on Starter, and $249 a month on Pro, with scoring and unlimited seats in each. The calculator on this page prices the same traffic on both.