Launch offerSix months of the Starter plan, free for teams who sign up by November 22Claim it →

Compare

three.dev vs Arize AX (2026): which one fits your AI feature?

Arize AX traces, evaluates, and monitors your agents, and its Signal finds recurring failures on a schedule. three.dev names your failure modes from live traffic with no SDK and nothing to turn on. Here is where each one fits.

Updated

The short answer

Pick three.dev if your AI feature is live, you have no tracing or evals yet, and your team codes with Claude Code, Codex, or Cursor. To get Arize AX's Signal working, you first add OpenInference tracing to your app and turn Signal on in each project, and then it sweeps your traces every 6 hours. three.dev skips the SDK. You change one base URL, and it scores a sample of live requests, sorts the issues into named failure modes with the most severe on top, and passes each one to your coding agent.

Pick Arize AX if you want a full hosted platform to trace, evaluate, and monitor your agents: every agent and tool step, online evals you define, datasets, experiments, prompts, monitors with alerts, and CI gates. Its Signal also finds recurring failures on its own, on every plan, Free included. If you must self-host open source, Arize's free Phoenix edition covers tracing, evals, and experiments, without Signal.

At a glance

Best for
three.dev

A live AI feature with no evaluators yet, and a team that fixes with a coding agent

Arize AX

Teams that want hosted tracing, evals, and monitors on OpenTelemetry, with Signal to find recurring failures and an open-source edition (Phoenix) they can self-host

What it is
three.dev

A proxy that scores live LLM traffic, names its failure modes, and hands the fix to your coding agent

Arize AX

A hosted agent observability and evaluation platform: tracing, online and offline evals, datasets, experiments, prompts, monitors, Signal, and the Alyx agent. Owned by Dynatrace since October 2026

How you connect
three.dev

Base URL gate.three.dev and three headers: a three.dev API key, a use case, and your provider. Your AI SDK and provider key stay the same. Works through LiteLLM and the other documented providers

Arize AX

OpenInference instrumentors on OpenTelemetry for 30+ providers and frameworks, in Python, TypeScript, Go, and Java. One command (npx evals or ax instrument) has your coding agent add the tracing

Setup before you see failures
three.dev

None to turn on, in any project

Arize AX

For Signal: traces in an AX tracing project, then Enable Signal in each project

For online evals: an LLM provider connection, an evaluator, and a task with a sampling rate on each project

Finding failures
three.dev

A sample of requests is scored after each response returns. Each issue gives the reason and quotes the lines behind it

Issues group into named failure modes, each with a severity, most severe first

Arize AX

Signal scans a project's traces on a schedule (every 6 hours by default, from every 3 hours to monthly), groups recurring failures into issues, and ranks them. Each issue has an overview, trace evidence, and a proposed fix

Capped at 10 issues a month on Free and 25 on Pro, account-wide; at the cap, Signal pauses until the next month. Not in Phoenix

Shaping the list
three.dev

three.dev sets each severity, and your team can change it. Your team can also dismiss a failure mode that is not worth acting on, and restore it later

Arize AX

In Configure Signal, you set the schedule and describe what your agent does and what counts as a failure. Issues sort by severity or last seen, and you can resolve them

Asking about your traffic
three.dev

Chat answers plain questions about your traffic and links each answer to the requests behind it

Arize AX

Alyx, the agent built into AX, answers questions about your traces and Signal issues, builds evals and dashboards, and tests prompt changes

After you find a failure
three.dev

A Fix with your agent button copies a prompt that starts your coding agent on one failure mode. The MCP server gives it the failing requests and each issue's reasoning

Arize AX

Create Evaluator turns a Signal issue into an online eval on new traces, and Add to Dataset feeds an experiment

On Enterprise, with a GitHub repo attached, Signal opens a pull request for the proposed fix

Coding-agent access
three.dev

An MCP server with failure modes, failing requests, and docs. It reads your data, and changes only a failure mode's severity or whether it is dismissed, when you ask. Plugins for Claude Code and Codex install it; Cursor and other clients paste its URL

Arize AX

A hosted MCP server (alpha) that reads and writes AX data, Arize Skills for Claude Code, Cursor, Codex, and 40+ other agents, and the ax and ag CLIs

Checking a fix
three.dev

Tag requests with a release, then compare two releases or two date ranges on quality, each failure mode, p50 and p95 latency, and cost

Arize AX

Experiments on datasets, which can run as regression gates in CI, plus online evals and monitors on live traffic

What gets recorded
three.dev

LLM calls and the tool calls the model asks for, grouped into sessions. Tool runs and retrieval are not separate spans

Arize AX

OpenTelemetry traces with LLM, agent, and tool spans, grouped into sessions, including images, voice, and PDFs

Effect on your requests
three.dev

In the path, as a proxy. Scoring waits until the response has returned, so it adds no latency

Arize AX

Not in the path. Instrumentation in your app sends spans to Arize, and your LLM calls go straight to your provider

Evals, prompts, datasets
three.dev

Not offered today

Arize AX

Yes, on every AX plan: evaluators, datasets, experiments, playgrounds, and prompt versioning. Custom code evaluators need Enterprise

Multi-turn evals
three.dev

Not offered. Sessions keep whole conversations for review

Arize AX

Session Evals score whole multi-turn conversations, and Trace Evals score agent trajectories, on every plan

CI gates
three.dev

Not offered today

Arize AX

Code experiments run as regression gates on each pull request or deploy, with guides for GitHub Actions, GitLab, Azure DevOps, and Harness. Phoenix evals run as pytest, Vitest, or Jest tests that gate CI

Alerts
three.dev

Not offered today

Arize AX

Monitors on any span attribute or custom metric alert Slack, PagerDuty, OpsGenie, Microsoft Teams, or a webhook, on every plan

What it costs to find failures
three.dev

Scoring is included in the plan price

Arize AX

Signal runs on an Arize-managed model, inside the plan's issue cap. Online evals and LLM judges call the LLM provider you connect, which may bill you

Seats
three.dev

Unlimited on every plan

Arize AX

Unlimited users on every plan

History kept
three.dev

30 days on Free, 90 on Starter, 180 on Pro

Arize AX

15 days on Free, 30 days on Pro, custom on Enterprise

Open source and self-hosting
three.dev

Not open source. No self-hosting today

Arize AX

AX self-hosts on Enterprise only. Phoenix is free to self-host under the Elastic License 2.0, with no feature gates, but it has no Signal

Data use
three.dev

three.dev does not train models on your traffic for anyone other than you

Arize AX

Data regions in the US, EU, or Canada on every plan. Arize says it does not train its models on what you send AI features such as Alyx unless you consent

Regulated data
three.dev

Not for health data (PHI) or government workloads yet

Arize AX

SOC 2 Type II and GDPR on Pro and Enterprise; HIPAA on Enterprise. The Trust Center also lists ISO 27001 and PCI DSS

Pricing
three.dev

Free, Starter $79, or Pro $249 a month, each with scoring and unlimited seats

Arize AX

AX Free; Pro $50 a month with 50k spans and 10 GB; Enterprise custom. Arize no longer lists a price for extra spans or GB. Phoenix is free to self-host

What each product is

Arize AX calls itself "the agent observability and evaluation platform." You instrument your app with OpenInference, Arize's open standard built on OpenTelemetry, then trace, evaluate, and improve it in AX: online evals on production traces, offline evals on datasets, experiments, prompt versions, labeling queues, dashboards, and monitors. Signal, which Arize calls "an always-on agent SRE," scans a project's traces on a schedule, groups recurring failures into ranked issues, and proposes a fix for each. Alyx, the agent built into AX, answers questions and builds evals and dashboards for you.

Arize Phoenix is the open-source edition: tracing, evals, datasets, experiments, and prompts on servers you run, free under the Elastic License 2.0. Online evals with alerting, Signal, and Alyx live in AX. Dynatrace completed its purchase of Arize on October 1, 2026, and says it will keep supporting both products.

three.dev works on the LLM calls your feature already makes, with no tracing SDK and no evaluator for you to write. Your requests go through its proxy at gate.three.dev:

  1. Connect. Point the base URL at gate.three.dev and add three headers: your three.dev API key, a use case, and your provider. Your AI SDK and provider key stay the same. A setup skill lets your coding agent make the change for you.
  2. Find. After each response returns, three.dev scores a sample of requests in the background. Every defect is an issue with its reason and the quoted lines behind it. As issues pile up, three.dev names failure modes, describes them, gives each a severity of high, medium, or low, and lists the most severe first. Issues that fit no failure mode wait in Unmatched.
  3. Manage. Your team can change a severity, or dismiss a failure mode that does not matter and restore it later.
  4. Fix. Claude Code, Codex, or Cursor reads a failure mode and its failing requests over the MCP server at https://mcp.three.dev/mcp.
  5. Compare. Tag requests with a release, then set two releases or two date ranges side by side on quality (the share of scored requests without issues), latency (p50 and p95), and cost.

Chat lets you ask about your traffic in plain words, and its answers link to the requests behind them.

Where Arize AX is the better choice

  • You already send traces to Arize AX. Signal is on every AX plan, Free included. Turn it on in your projects, and it groups recurring failures into ranked issues with a proposed fix, the first thing three.dev does.
  • You want the tool to open the fix pull request. On Enterprise, Signal can take a GitHub repo and open a pull request for its proposed fix. A Signal issue can also become a running online eval in one click, on every plan. three.dev hands the failure mode to your own coding agent and opens no pull request itself.
  • You want a full eval platform. AX covers online evals with LLM judges, code evaluators, and agent-as-a-judge, session evals for multi-turn conversations, trace evals for agent paths, datasets, experiments, playgrounds, prompt versioning, and labeling queues. three.dev offers no evals, datasets, experiments, or prompt management today.
  • You need CI gates or alerts. AX runs experiments as regression gates on each pull request or deploy, and Phoenix evals run as pytest, Vitest, or Jest tests that gate CI. AX monitors send alerts to Slack, PagerDuty, OpsGenie, Microsoft Teams, or a webhook. three.dev offers neither today.
  • You need to trace tool runs and agent steps. OpenInference records LLM, agent, and tool spans for 30+ providers and frameworks, including images, voice, and PDFs. three.dev records each LLM call, including the tool calls the model asks for, and does not trace tool runs or retrieval steps as their own spans.
  • You want open source or self-hosting. Phoenix is free to self-host with no feature gates, and AX self-hosts on Enterprise. three.dev is not open source and has no self-hosting today.
  • You want nothing in your request path. Arize instrumentation sends spans from your app, and your LLM calls go straight to your provider. three.dev is a proxy, so your LLM requests pass through it.
  • You handle regulated data. AX Pro includes SOC 2 Type II and GDPR, and Enterprise adds HIPAA. three.dev does not take health data (PHI) or government workloads yet.
  • You want the lowest paid entry price. AX Pro is $50 a month, and Arize offers startup pricing. three.dev's first paid plan, Starter, is $79 a month.
  • You run Dynatrace. Dynatrace now owns Arize and plans to build Arize's features into its platform over time.

Where three.dev is the better choice

  • You would rather change a setting than add tracing. Arize needs OpenInference instrumentors in your app before Signal has traces to read. With three.dev, your AI SDK stays as it is. If you already call models through LiteLLM, three.dev documents that route too.
  • You want failure modes without a switch or a monthly cap. Signal is turned on per project, sweeps every 6 hours by default, and stops at 10 issues a month on AX Free and 25 on Pro, across your whole account. three.dev starts scoring with your first requests, and its plans meter scored requests, not issues.
  • You want to check each failure on the request itself. Every three.dev issue gives the reason one request failed and quotes the lines behind it, and the issues roll up into failure modes with a severity on each.
  • You want to rank the list yourself. Your team can change any severity, and dismiss a failure mode that does not matter. three.dev keeps scoring it, so you can restore it later.
  • You want your own coding agent to write the fix, on every plan. A Fix with your agent button copies a prompt that starts the agent on one failure mode, and the MCP server hands it the failing requests with each issue's summary and reasoning. Signal's repo-backed fix pull requests need AX Enterprise.
  • You want scoring in the plan price. In AX, online evals and LLM judges call the LLM provider you connect, and that provider may bill you. Every three.dev plan includes scoring.
  • You want more history on the lower plans. AX keeps data 15 days on Free and 30 days on Pro. three.dev keeps it longer on each of its plans: see the History kept row above.

Pricing

Arize AX pricing:

AX Free
Price$0 a month
Included25k spans and 1 GB a month, 15 days of data, 10 Signal issues a month, unlimited users, 1 space, community support
Extra usageNone listed; upgrade to Pro for more
AX Pro
Price$50 a month
Included50k spans and 10 GB a month, 30 days of data, 25 Signal issues a month, unlimited users, 1 org and 2 spaces, SOC 2 Type II, GDPR, email support
Extra usageNot listed by Arize
AX Enterprise
PriceCustom
IncludedCustom volume and retention, unlimited Signal issues, repo-backed fix pull requests, custom managed agents, self-hosting, Enterprise SSO, audit logs, HIPAA, bring your own AI provider key
Extra usageCustom

Spans, ingestion volume in GB, and Signal issues are the meters. Users, evals, experiments, datasets, prompts, and labeling queues are unlimited on every plan. Signal's issue caps are account-wide and reset each calendar month; Arize says a way to buy more Free or Pro issues is coming soon. Online evals and LLM judges run on the LLM provider you connect, so your provider may bill you for them. Startup pricing is available.

Arize Phoenix costs nothing to self-host: no license fees, no usage limits, and no feature gates. You run your own servers.

three.dev has three plans: Free, Starter at $79 a month, and Pro at $249 a month. Each plan includes scoring and unlimited seats. three.dev is generally available: compare its plans, or start free and point it at your own traffic.

Our recommendation

Choose Arize AX if you want one hosted platform for tracing, evals, experiments, prompts, monitors, and CI gates, or if you already send traces to it. In that case, turn on Signal in your projects before you look elsewhere: it does more today. Choose Phoenix if you must self-host open source.

Choose three.dev if your feature is live, nobody on your team has time to add tracing or write evaluators, and a coding agent already writes most of your fixes. One base URL change gets you failure modes with a severity on each, ready for the agent.

Key facts

  • three.dev: a proxy at gate.three.dev that scores a sample of live LLM requests, explains each issue with quoted lines, and groups the issues into failure modes ranked by severity. Your team can change a severity, and dismiss or restore a failure mode. Generally available since November 2026.
  • Arize AX: a hosted agent observability and evaluation platform on OpenTelemetry, with online and offline evals, datasets, experiments, prompts, monitors, Alyx, and Signal. Owned by Dynatrace since October 1, 2026. Phoenix is its free, self-hosted edition, without Signal.
  • Work before the first failure shows up: in Arize AX, OpenInference tracing in your app and Signal turned on in each project, then a sweep every 6 hours by default. In three.dev, a base URL and three headers.
  • Prices: AX Free is $0, Pro $50 a month, and Enterprise custom; Phoenix is free to self-host. three.dev Free is $0, Starter $79, and Pro $249 a month, with scoring included.
  • Missing from three.dev today: alerts, CI gates, prompt management, datasets, experiments, and self-hosting. It does not take PHI or government workloads yet.

Sources

FAQ

Is three.dev an Arize alternative?

For a team with a live AI feature, no tracing or evals yet, and a coding agent, yes. You change one base URL, and three.dev scores live traffic, sorts what it finds into failure modes by severity, and gives them to your coding agent. For a team that wants tracing of every agent step, online evals it writes, datasets, experiments, monitors, CI gates, or self-hosting, Arize AX and Phoenix cover more today.

Is Arize free?

Yes, in two ways. Arize Phoenix is free to self-host, with no license fees, usage limits, or feature gates; you run the servers. Arize AX, the hosted product, has a free plan with 25k spans and 1 GB a month, 15 days of retention, and 10 Signal issues a month, with no credit card. AX Pro is $50 a month. three.dev also has a free plan, at $0.

Does Arize find production failures automatically?

Yes, in Arize AX. Signal scans a tracing project's traces on a schedule, every 6 hours by default, groups recurring failures into ranked issues, and writes up each one with trace evidence and a proposed fix. You turn it on in each project. It stops at 10 issues a month on AX Free and 25 on Pro, account-wide, and pauses until the next month at the cap. Enterprise has no cap. Phoenix, the open-source edition, does not include Signal, as of November 18, 2026.

What is the difference between Arize Signal and three.dev failure modes?

Both find recurring failures in production without an eval you write first, and both rank what they find. Signal reads the traces your OpenInference instrumentation sends to Arize AX, runs as a scheduled sweep, and proposes a prompt, code, config, or eval change for each issue. On Enterprise, it can open the pull request. three.dev reads the LLM calls that pass through its proxy, with no tracing SDK. It scores a sample of requests after each response returns, explains each issue with the quoted lines behind it, and groups the issues into failure modes with a severity of high, medium, or low. Your team can change a severity or dismiss a failure mode, and your own coding agent writes the fix.

What is the difference between Arize AX and Arize Phoenix?

Phoenix is Arize's open-source, self-hosted tool for tracing, evals, datasets, experiments, and prompts. Arize AX is the hosted platform. AX adds online evals on live traffic, monitors with alerts, Signal, and the Alyx agent, and it self-hosts only on Enterprise. Phoenix's own docs send you to AX for evals on production traffic with alerting.

Is Arize Phoenix open source?

Arize calls Phoenix fully open source. Its license is the Elastic License 2.0 (ELv2). You may self-host it on your own servers or cloud account for free, but the license bars offering Phoenix to others as a hosted or managed service. three.dev is not open source and has no self-hosting today.

How much does Arize AX cost compared to three.dev?

Arize AX has a free plan, Pro at $50 a month with 50k spans and 10 GB, and custom Enterprise pricing. Arize does not list a price for extra spans or GB. Online evals call the LLM provider you connect, which may bill you. Phoenix is free to self-host. three.dev costs $0 on Free, $79 a month on Starter, and $249 a month on Pro, with scoring and unlimited seats in each. Arize meters spans and GB, and three.dev meters GB and scored requests, so price your own traffic on both pricing pages.

Does Arize work with coding agents like Claude Code or Cursor?

Yes. Arize AX has a hosted MCP server, in alpha, that can read and write AX data, plus Arize Skills for Claude Code, Cursor, Codex, Windsurf, and 40+ other agents, and the ax and ag command-line tools. One command, npx evals, has your coding agent add the tracing. three.dev's MCP server reads failure modes and their failing requests in Claude Code, Codex, Cursor, and other clients, and changes only a failure mode's severity or whether it is dismissed, when you ask, and a Fix with your agent button copies a prompt that starts the agent on one failure mode.

What happens to Arize now that Dynatrace owns it?

Dynatrace completed its purchase of Arize on October 1, 2026. Its press release says Arize will keep supporting both Phoenix and AX, and that over time Arize's features will be built into Dynatrace. The release makes no promise about prices or the Phoenix license.

More comparisons

three.dev vs Braintrust

three.dev vs Braintrust (2026): which one fits your AI feature?

Braintrust is built around datasets, scorers, and CI, with failure discovery you turn on per project. three.dev scores live traffic and names the failure modes with no setup. Here is where each one fits.

As of November 18, 2026Read

three.dev vs Langfuse

three.dev vs Langfuse (2026): which one fits your AI feature?

Langfuse gives you open-source tracing and evals that you set up and run. three.dev names your failure modes from live traffic with nothing to configure. Here is where each one fits.

As of November 18, 2026Read

three.dev vs LangSmith

three.dev vs LangSmith (2026): which one fits your AI feature?

LangSmith runs the whole agent lifecycle, and its Engine finds failures on paid plans. three.dev finds them without moving your stack onto a platform. Here is where each one fits.

As of November 18, 2026Read

three.dev vs Raindrop

three.dev vs Raindrop (2026): which one fits your AI feature?

Raindrop and three.dev both find failures in production that nobody wrote a test for. Raindrop does it on its Pro plan, with alerts, full agent traces, and simulations around it. three.dev does it on every plan, from one base URL change. Here is where each one fits.

As of November 18, 2026Read

three.dev vs Comet Opik

three.dev vs Comet Opik (2026): which one fits your AI feature?

Opik gives you tracing, evals, test suites, and Diagnostics scans run by its Ollie assistant on credits. three.dev scores live traffic and names the failure modes with nothing to turn on. Here is where each one fits.

As of November 18, 2026Read

three.dev vs Galileo

three.dev vs Galileo (2026): which one fits your AI feature?

Galileo is now Splunk Agent Observability: evals, Luna models, Signals, and runtime guardrails, sold through Splunk. three.dev names your failure modes from live traffic with nothing to configure. Here is where each one fits.

As of November 18, 2026Read

See how your AI feature fails.

Point the AI SDK you already use at one base URL, and get your failure modes named and ranked by severity.