Compare
three.dev vs Arize AX (2026): which one fits your AI feature?
Arize AX traces, evaluates, and monitors your agents, and its Signal finds recurring failures on a schedule. three.dev names your failure modes from live traffic with no SDK and nothing to turn on. Here is where each one fits.
Updated
The short answer
Pick three.dev if your AI feature is live, you have no tracing or evals yet, and your team codes with Claude Code, Codex, or Cursor. To get Arize AX's Signal working, you first add OpenInference tracing to your app and turn Signal on in each project, and then it sweeps your traces every 6 hours. three.dev skips the SDK. You change one base URL, and it scores a sample of live requests, sorts the issues into named failure modes with the most severe on top, and passes each one to your coding agent.
Pick Arize AX if you want a full hosted platform to trace, evaluate, and monitor your agents: every agent and tool step, online evals you define, datasets, experiments, prompts, monitors with alerts, and CI gates. Its Signal also finds recurring failures on its own, on every plan, Free included. If you must self-host open source, Arize's free Phoenix edition covers tracing, evals, and experiments, without Signal.
At a glance
A live AI feature with no evaluators yet, and a team that fixes with a coding agent
Teams that want hosted tracing, evals, and monitors on OpenTelemetry, with Signal to find recurring failures and an open-source edition (Phoenix) they can self-host
A proxy that scores live LLM traffic, names its failure modes, and hands the fix to your coding agent
A hosted agent observability and evaluation platform: tracing, online and offline evals, datasets, experiments, prompts, monitors, Signal, and the Alyx agent. Owned by Dynatrace since October 2026
Base URL gate.three.dev and three headers: a three.dev API key, a use case, and your provider. Your AI SDK and provider key stay the same. Works through LiteLLM and the other documented providers
OpenInference instrumentors on OpenTelemetry for 30+ providers and frameworks, in Python, TypeScript, Go, and Java. One command (npx evals or ax instrument) has your coding agent add the tracing
None to turn on, in any project
For Signal: traces in an AX tracing project, then Enable Signal in each project
For online evals: an LLM provider connection, an evaluator, and a task with a sampling rate on each project
A sample of requests is scored after each response returns. Each issue gives the reason and quotes the lines behind it
Issues group into named failure modes, each with a severity, most severe first
Signal scans a project's traces on a schedule (every 6 hours by default, from every 3 hours to monthly), groups recurring failures into issues, and ranks them. Each issue has an overview, trace evidence, and a proposed fix
Capped at 10 issues a month on Free and 25 on Pro, account-wide; at the cap, Signal pauses until the next month. Not in Phoenix
three.dev sets each severity, and your team can change it. Your team can also dismiss a failure mode that is not worth acting on, and restore it later
In Configure Signal, you set the schedule and describe what your agent does and what counts as a failure. Issues sort by severity or last seen, and you can resolve them
Chat answers plain questions about your traffic and links each answer to the requests behind it
Alyx, the agent built into AX, answers questions about your traces and Signal issues, builds evals and dashboards, and tests prompt changes
A Fix with your agent button copies a prompt that starts your coding agent on one failure mode. The MCP server gives it the failing requests and each issue's reasoning
Create Evaluator turns a Signal issue into an online eval on new traces, and Add to Dataset feeds an experiment
On Enterprise, with a GitHub repo attached, Signal opens a pull request for the proposed fix
An MCP server with failure modes, failing requests, and docs. It reads your data, and changes only a failure mode's severity or whether it is dismissed, when you ask. Plugins for Claude Code and Codex install it; Cursor and other clients paste its URL
A hosted MCP server (alpha) that reads and writes AX data, Arize Skills for Claude Code, Cursor, Codex, and 40+ other agents, and the ax and ag CLIs
Tag requests with a release, then compare two releases or two date ranges on quality, each failure mode, p50 and p95 latency, and cost
Experiments on datasets, which can run as regression gates in CI, plus online evals and monitors on live traffic
LLM calls and the tool calls the model asks for, grouped into sessions. Tool runs and retrieval are not separate spans
OpenTelemetry traces with LLM, agent, and tool spans, grouped into sessions, including images, voice, and PDFs
In the path, as a proxy. Scoring waits until the response has returned, so it adds no latency
Not in the path. Instrumentation in your app sends spans to Arize, and your LLM calls go straight to your provider
Not offered today
Yes, on every AX plan: evaluators, datasets, experiments, playgrounds, and prompt versioning. Custom code evaluators need Enterprise
Not offered. Sessions keep whole conversations for review
Session Evals score whole multi-turn conversations, and Trace Evals score agent trajectories, on every plan
Not offered today
Code experiments run as regression gates on each pull request or deploy, with guides for GitHub Actions, GitLab, Azure DevOps, and Harness. Phoenix evals run as pytest, Vitest, or Jest tests that gate CI
Not offered today
Monitors on any span attribute or custom metric alert Slack, PagerDuty, OpsGenie, Microsoft Teams, or a webhook, on every plan
Scoring is included in the plan price
Signal runs on an Arize-managed model, inside the plan's issue cap. Online evals and LLM judges call the LLM provider you connect, which may bill you
Unlimited on every plan
Unlimited users on every plan
30 days on Free, 90 on Starter, 180 on Pro
15 days on Free, 30 days on Pro, custom on Enterprise
Not open source. No self-hosting today
AX self-hosts on Enterprise only. Phoenix is free to self-host under the Elastic License 2.0, with no feature gates, but it has no Signal
three.dev does not train models on your traffic for anyone other than you
Data regions in the US, EU, or Canada on every plan. Arize says it does not train its models on what you send AI features such as Alyx unless you consent
Not for health data (PHI) or government workloads yet
SOC 2 Type II and GDPR on Pro and Enterprise; HIPAA on Enterprise. The Trust Center also lists ISO 27001 and PCI DSS
Free, Starter $79, or Pro $249 a month, each with scoring and unlimited seats
AX Free; Pro $50 a month with 50k spans and 10 GB; Enterprise custom. Arize no longer lists a price for extra spans or GB. Phoenix is free to self-host
What each product is
Arize AX calls itself "the agent observability and evaluation platform." You instrument your app with OpenInference, Arize's open standard built on OpenTelemetry, then trace, evaluate, and improve it in AX: online evals on production traces, offline evals on datasets, experiments, prompt versions, labeling queues, dashboards, and monitors. Signal, which Arize calls "an always-on agent SRE," scans a project's traces on a schedule, groups recurring failures into ranked issues, and proposes a fix for each. Alyx, the agent built into AX, answers questions and builds evals and dashboards for you.
Arize Phoenix is the open-source edition: tracing, evals, datasets, experiments, and prompts on servers you run, free under the Elastic License 2.0. Online evals with alerting, Signal, and Alyx live in AX. Dynatrace completed its purchase of Arize on October 1, 2026, and says it will keep supporting both products.
three.dev works on the LLM calls your feature already makes, with no tracing SDK and no evaluator for you to write. Your requests go through its proxy at gate.three.dev:
- Connect. Point the base URL at gate.three.dev and add three headers: your three.dev API key, a use case, and your provider. Your AI SDK and provider key stay the same. A setup skill lets your coding agent make the change for you.
- Find. After each response returns, three.dev scores a sample of requests in the background. Every defect is an issue with its reason and the quoted lines behind it. As issues pile up, three.dev names failure modes, describes them, gives each a severity of high, medium, or low, and lists the most severe first. Issues that fit no failure mode wait in Unmatched.
- Manage. Your team can change a severity, or dismiss a failure mode that does not matter and restore it later.
- Fix. Claude Code, Codex, or Cursor reads a failure mode and its failing requests over the MCP server at
https://mcp.three.dev/mcp. - Compare. Tag requests with a release, then set two releases or two date ranges side by side on quality (the share of scored requests without issues), latency (p50 and p95), and cost.
Chat lets you ask about your traffic in plain words, and its answers link to the requests behind them.
Where Arize AX is the better choice
- You already send traces to Arize AX. Signal is on every AX plan, Free included. Turn it on in your projects, and it groups recurring failures into ranked issues with a proposed fix, the first thing three.dev does.
- You want the tool to open the fix pull request. On Enterprise, Signal can take a GitHub repo and open a pull request for its proposed fix. A Signal issue can also become a running online eval in one click, on every plan. three.dev hands the failure mode to your own coding agent and opens no pull request itself.
- You want a full eval platform. AX covers online evals with LLM judges, code evaluators, and agent-as-a-judge, session evals for multi-turn conversations, trace evals for agent paths, datasets, experiments, playgrounds, prompt versioning, and labeling queues. three.dev offers no evals, datasets, experiments, or prompt management today.
- You need CI gates or alerts. AX runs experiments as regression gates on each pull request or deploy, and Phoenix evals run as pytest, Vitest, or Jest tests that gate CI. AX monitors send alerts to Slack, PagerDuty, OpsGenie, Microsoft Teams, or a webhook. three.dev offers neither today.
- You need to trace tool runs and agent steps. OpenInference records LLM, agent, and tool spans for 30+ providers and frameworks, including images, voice, and PDFs. three.dev records each LLM call, including the tool calls the model asks for, and does not trace tool runs or retrieval steps as their own spans.
- You want open source or self-hosting. Phoenix is free to self-host with no feature gates, and AX self-hosts on Enterprise. three.dev is not open source and has no self-hosting today.
- You want nothing in your request path. Arize instrumentation sends spans from your app, and your LLM calls go straight to your provider. three.dev is a proxy, so your LLM requests pass through it.
- You handle regulated data. AX Pro includes SOC 2 Type II and GDPR, and Enterprise adds HIPAA. three.dev does not take health data (PHI) or government workloads yet.
- You want the lowest paid entry price. AX Pro is $50 a month, and Arize offers startup pricing. three.dev's first paid plan, Starter, is $79 a month.
- You run Dynatrace. Dynatrace now owns Arize and plans to build Arize's features into its platform over time.
Where three.dev is the better choice
- You would rather change a setting than add tracing. Arize needs OpenInference instrumentors in your app before Signal has traces to read. With three.dev, your AI SDK stays as it is. If you already call models through LiteLLM, three.dev documents that route too.
- You want failure modes without a switch or a monthly cap. Signal is turned on per project, sweeps every 6 hours by default, and stops at 10 issues a month on AX Free and 25 on Pro, across your whole account. three.dev starts scoring with your first requests, and its plans meter scored requests, not issues.
- You want to check each failure on the request itself. Every three.dev issue gives the reason one request failed and quotes the lines behind it, and the issues roll up into failure modes with a severity on each.
- You want to rank the list yourself. Your team can change any severity, and dismiss a failure mode that does not matter. three.dev keeps scoring it, so you can restore it later.
- You want your own coding agent to write the fix, on every plan. A Fix with your agent button copies a prompt that starts the agent on one failure mode, and the MCP server hands it the failing requests with each issue's summary and reasoning. Signal's repo-backed fix pull requests need AX Enterprise.
- You want scoring in the plan price. In AX, online evals and LLM judges call the LLM provider you connect, and that provider may bill you. Every three.dev plan includes scoring.
- You want more history on the lower plans. AX keeps data 15 days on Free and 30 days on Pro. three.dev keeps it longer on each of its plans: see the History kept row above.
Pricing
Arize AX pricing:
| Plan | Price | Included | Extra usage |
|---|---|---|---|
AX Free | $0 a month | 25k spans and 1 GB a month, 15 days of data, 10 Signal issues a month, unlimited users, 1 space, community support | None listed; upgrade to Pro for more |
AX Pro | $50 a month | 50k spans and 10 GB a month, 30 days of data, 25 Signal issues a month, unlimited users, 1 org and 2 spaces, SOC 2 Type II, GDPR, email support | Not listed by Arize |
AX Enterprise | Custom | Custom volume and retention, unlimited Signal issues, repo-backed fix pull requests, custom managed agents, self-hosting, Enterprise SSO, audit logs, HIPAA, bring your own AI provider key | Custom |
Spans, ingestion volume in GB, and Signal issues are the meters. Users, evals, experiments, datasets, prompts, and labeling queues are unlimited on every plan. Signal's issue caps are account-wide and reset each calendar month; Arize says a way to buy more Free or Pro issues is coming soon. Online evals and LLM judges run on the LLM provider you connect, so your provider may bill you for them. Startup pricing is available.
Arize Phoenix costs nothing to self-host: no license fees, no usage limits, and no feature gates. You run your own servers.
three.dev has three plans: Free, Starter at $79 a month, and Pro at $249 a month. Each plan includes scoring and unlimited seats. three.dev is generally available: compare its plans, or start free and point it at your own traffic.
Our recommendation
Choose Arize AX if you want one hosted platform for tracing, evals, experiments, prompts, monitors, and CI gates, or if you already send traces to it. In that case, turn on Signal in your projects before you look elsewhere: it does more today. Choose Phoenix if you must self-host open source.
Choose three.dev if your feature is live, nobody on your team has time to add tracing or write evaluators, and a coding agent already writes most of your fixes. One base URL change gets you failure modes with a severity on each, ready for the agent.
Key facts
- three.dev: a proxy at gate.three.dev that scores a sample of live LLM requests, explains each issue with quoted lines, and groups the issues into failure modes ranked by severity. Your team can change a severity, and dismiss or restore a failure mode. Generally available since November 2026.
- Arize AX: a hosted agent observability and evaluation platform on OpenTelemetry, with online and offline evals, datasets, experiments, prompts, monitors, Alyx, and Signal. Owned by Dynatrace since October 1, 2026. Phoenix is its free, self-hosted edition, without Signal.
- Work before the first failure shows up: in Arize AX, OpenInference tracing in your app and Signal turned on in each project, then a sweep every 6 hours by default. In three.dev, a base URL and three headers.
- Prices: AX Free is $0, Pro $50 a month, and Enterprise custom; Phoenix is free to self-host. three.dev Free is $0, Starter $79, and Pro $249 a month, with scoring included.
- Missing from three.dev today: alerts, CI gates, prompt management, datasets, experiments, and self-hosting. It does not take PHI or government workloads yet.
Sources
- Arize pricing
- Pricing and usage
- Arize AX homepage
- Arize AX Signal
- Signal to find issues
- Alyx
- Online evals
- LLM as a judge
- AI provider integrations
- Set up tracing
- Set up sessions
- Arize AX for coding agents
- Arize Skills
- Arize AX MCP server
- MCP tool catalog
- Find and fix issues with a coding agent
- Monitors and alerts
- CI/CD with experiments
- GitHub Action basics
- Eval Health agent
- Arize AX changelog
- Arize Trust Center
- Arize privacy policy
- Arize Phoenix homepage
- Phoenix self-hosting
- Phoenix license
- Phoenix evaluation
- Phoenix docs index
- Dynatrace completes Arize acquisition
- three.dev docs: sending requests
- three.dev docs: AI providers
- three.dev docs: live scoring
- three.dev docs: failure modes
FAQ
Is three.dev an Arize alternative?
For a team with a live AI feature, no tracing or evals yet, and a coding agent, yes. You change one base URL, and three.dev scores live traffic, sorts what it finds into failure modes by severity, and gives them to your coding agent. For a team that wants tracing of every agent step, online evals it writes, datasets, experiments, monitors, CI gates, or self-hosting, Arize AX and Phoenix cover more today.
Is Arize free?
Yes, in two ways. Arize Phoenix is free to self-host, with no license fees, usage limits, or feature gates; you run the servers. Arize AX, the hosted product, has a free plan with 25k spans and 1 GB a month, 15 days of retention, and 10 Signal issues a month, with no credit card. AX Pro is $50 a month. three.dev also has a free plan, at $0.
Does Arize find production failures automatically?
Yes, in Arize AX. Signal scans a tracing project's traces on a schedule, every 6 hours by default, groups recurring failures into ranked issues, and writes up each one with trace evidence and a proposed fix. You turn it on in each project. It stops at 10 issues a month on AX Free and 25 on Pro, account-wide, and pauses until the next month at the cap. Enterprise has no cap. Phoenix, the open-source edition, does not include Signal, as of November 18, 2026.
What is the difference between Arize Signal and three.dev failure modes?
Both find recurring failures in production without an eval you write first, and both rank what they find. Signal reads the traces your OpenInference instrumentation sends to Arize AX, runs as a scheduled sweep, and proposes a prompt, code, config, or eval change for each issue. On Enterprise, it can open the pull request. three.dev reads the LLM calls that pass through its proxy, with no tracing SDK. It scores a sample of requests after each response returns, explains each issue with the quoted lines behind it, and groups the issues into failure modes with a severity of high, medium, or low. Your team can change a severity or dismiss a failure mode, and your own coding agent writes the fix.
What is the difference between Arize AX and Arize Phoenix?
Phoenix is Arize's open-source, self-hosted tool for tracing, evals, datasets, experiments, and prompts. Arize AX is the hosted platform. AX adds online evals on live traffic, monitors with alerts, Signal, and the Alyx agent, and it self-hosts only on Enterprise. Phoenix's own docs send you to AX for evals on production traffic with alerting.
Is Arize Phoenix open source?
Arize calls Phoenix fully open source. Its license is the Elastic License 2.0 (ELv2). You may self-host it on your own servers or cloud account for free, but the license bars offering Phoenix to others as a hosted or managed service. three.dev is not open source and has no self-hosting today.
How much does Arize AX cost compared to three.dev?
Arize AX has a free plan, Pro at $50 a month with 50k spans and 10 GB, and custom Enterprise pricing. Arize does not list a price for extra spans or GB. Online evals call the LLM provider you connect, which may bill you. Phoenix is free to self-host. three.dev costs $0 on Free, $79 a month on Starter, and $249 a month on Pro, with scoring and unlimited seats in each. Arize meters spans and GB, and three.dev meters GB and scored requests, so price your own traffic on both pricing pages.
Does Arize work with coding agents like Claude Code or Cursor?
Yes. Arize AX has a hosted MCP server, in alpha, that can read and write AX data, plus Arize Skills for Claude Code, Cursor, Codex, Windsurf, and 40+ other agents, and the ax and ag command-line tools. One command, npx evals, has your coding agent add the tracing. three.dev's MCP server reads failure modes and their failing requests in Claude Code, Codex, Cursor, and other clients, and changes only a failure mode's severity or whether it is dismissed, when you ask, and a Fix with your agent button copies a prompt that starts the agent on one failure mode.
What happens to Arize now that Dynatrace owns it?
Dynatrace completed its purchase of Arize on October 1, 2026. Its press release says Arize will keep supporting both Phoenix and AX, and that over time Arize's features will be built into Dynatrace. The release makes no promise about prices or the Phoenix license.