Launch offerSix months of the Starter plan, free for teams who sign up by November 22Claim it →

Compare

three.dev vs Galileo (2026): which one fits your AI feature?

Galileo is now Splunk Agent Observability: evals, Luna models, Signals, and runtime guardrails, sold through Splunk. three.dev names your failure modes from live traffic with nothing to configure. Here is where each one fits.

Updated

The short answer

Pick three.dev if your AI feature is live, nobody has written an evaluator for it yet, and your team codes with Claude Code, Codex, or Cursor. In Galileo you first log traces with its SDK, connect your own model key, and turn on evaluators. three.dev skips that step. You change one base URL, and it scores a sample of live requests, sorts the issues into named failure modes with the most severe on top, and passes each one to your coding agent.

Pick Galileo, now sold as Splunk Agent Observability, if you need guardrails that block bad actions while the agent runs, evaluation models that score every request, and evals, datasets, experiments, and alerts in one place. It fits best at a larger company that already runs Splunk or Cisco tools, or that needs an on-premises or air-gapped install.

At a glance

Best for
three.dev

A live AI feature with no evaluators yet, and a team that fixes with a coding agent

Galileo

Larger companies on Splunk or Cisco tools that want agent tracing, evals, and runtime guardrails in one place, in the cloud or on-premises

What it is
three.dev

A proxy that scores live LLM traffic, names its failure modes, and hands the fix to your coding agent

Galileo

Splunk Agent Observability, formerly Galileo: tracing, evaluators, Luna evaluation models, datasets, experiments, Signals, an AI Assistant, and runtime guardrails

How you connect
three.dev

Base URL gate.three.dev and three headers: a three.dev API key, a use case, and your provider. Your AI SDK and provider key stay the same. Works through LiteLLM and the other documented providers

Galileo

The Python SDK (splunk-ao), an OpenAI SDK wrapper, OpenTelemetry or OpenInference, or the API. Integrations for LangChain and LangGraph, CrewAI, the OpenAI Agents SDK, Google ADK, Claude, and more

Setup before you see failures
three.dev

None. No SDK, LLM integration, or evaluator to turn on. Scoring begins with your first requests

Galileo

Log traces with the SDK or OpenTelemetry. Add an LLM integration with your own model key for LLM-based evaluators, Signals, and the AI Assistant. Turn on evaluators for each Agent Stream, with an optional sampling rate

Finding failures
three.dev

A sample of requests is scored after each response returns. Each issue gives the reason and quotes the lines behind it

Issues group into named failure modes, each with a severity, most severe first

Galileo

Signals groups related failures across production traces and gives each a severity of Low, High, or Critical, labels from 45 failure types, a root cause, and a recommended action. Scans run on demand or on a schedule at least 12 hours apart

Evaluators you turn on score traces for the checks they cover

Shaping the list
three.dev

three.dev sets each severity, and your team can change it. Your team can also dismiss a failure mode that is not worth acting on, and restore it later

Galileo

You can archive, merge, or unmerge Signals, and rate each one. The Remediate tab suggests an evaluator, alert, or trend view that tracks a Signal if it comes back

Asking about your traffic
three.dev

Chat answers plain questions about your traffic and links each answer to the requests behind it

Galileo

The AI Assistant answers questions about traces, sessions, and evaluator scores, with links to the evidence. It runs on your LLM integration and is read-only

After you find a failure
three.dev

A Fix with your agent button copies a prompt that starts your coding agent on one failure mode. The MCP server gives it the failing requests and each issue's reasoning

Galileo

The Remediate tab lists recommended fixes. Fix in your codebase opens a brief in Codex or Cursor, or copies it to your coding agent, which reads the Signal over MCP. The agent proposes the change for you to apply

Coding-agent access
three.dev

An MCP server with failure modes, failing requests, and docs. It reads your data, and changes only a failure mode's severity or whether it is dismissed, when you ask. Plugins for Claude Code and Codex install it; Cursor and other clients paste its URL

Galileo

An MCP server for Cursor and VS Code with GitHub Copilot: datasets, experiments, prompt templates, and Signals, including the fixes a Signal recommends

Checking a fix
three.dev

Tag requests with a release, then compare two releases or two date ranges on quality, each failure mode, p50 and p95 latency, and cost

Galileo

Experiments run your app on a dataset with your evaluators, and you compare the runs

What gets recorded
three.dev

LLM calls and the tool calls the model asks for, grouped into sessions. Tool runs and retrieval are not separate spans

Galileo

Traces, spans, and sessions: LLM calls, tool calls, retrieval, and agent steps, with distributed tracing across services. Splunk shows the GPU and infrastructure behind them

Effect on your requests
three.dev

In the path, as a proxy. Scoring waits until the response has returned, so it adds no latency

Galileo

Not a proxy: traces go out through the SDK or OpenTelemetry. Guardrails (Agent Control), if you add them, check inputs, outputs, and tool calls during the run and can block them

Evals, prompts, datasets
three.dev

Not offered today

Galileo

Yes: out-of-the-box and custom evaluators, datasets, prompts, playgrounds, experiments, and annotations

Multi-turn evals
three.dev

Not offered. Sessions keep whole conversations for review

Galileo

Session-level evaluators, such as Conversation Quality, score whole conversations

CI gates
three.dev

Not offered today

Galileo

Experiments run inside unit tests in your CI/CD pipeline, and the test asserts on average evaluator scores

Alerts
three.dev

Not offered today

Galileo

Alerts on evaluator, cost, latency, and trace-count thresholds, sent by email, Slack, or a generic webhook such as PagerDuty

What it costs to find failures
three.dev

Scoring is included in the plan price

Galileo

Billing counts spans and Luna tokens. LLM-based evaluators, Signals, and the AI Assistant run on your own LLM integration, so your model provider bills those calls. Sampling lowers the evaluator cost

History kept
three.dev

30 days on Free, 90 on Starter, 180 on Pro

Galileo

SaaS traces: 30 days by default, or 60, 90, or 180 days through your Splunk account team. Experiments: 2 years

Open source and self-hosting
three.dev

Not open source. No self-hosting today

Galileo

An on-premises deployment that you install and run, including air-gapped. No open-source edition on the pages we read

Data use
three.dev

three.dev does not train models on your traffic for anyone other than you

Galileo

SaaS regions in the US, Europe, and Asia Pacific. Splunk offers a data processing addendum (DPA)

Regulated data
three.dev

Not for health data (PHI) or government workloads yet

Galileo

On-premises or air-gapped deployment, and guardrails that block PII, PHI, and PCI leaks. The Agent Observability pages we read list no audit reports of their own

Pricing
three.dev

Free, Starter $79, or Pro $249 a month, each with scoring and unlimited seats

Galileo

Splunk Observability Cloud Free Edition includes Agent Observability. Paid plans start at "$100 per month per 1.2 million spans annually, billed annually", with 500 million Luna tokens a year

What each product is

Galileo was an AI evaluation and observability company. Cisco announced it would buy Galileo on April 9, 2026, and completed the deal by May 22, 2026. Galileo's docs say that as of August 7, 2026, "Galileo is now Splunk Agent Observability". Splunk calls it "the leading observability, evaluation, and guardrail platform for GenAI and agentic applications". The hosted version runs inside Splunk Observability Cloud. An on-premises version can run in your own environment, air-gapped if you need.

You log your app with the Python SDK, an OpenAI wrapper, OpenTelemetry, or the API. You turn on evaluators for each Agent Stream, from a library of out-of-the-box checks or your own LLM-as-a-judge, decision model, or code evaluators, and you can sample to save cost. Luna, Galileo's family of small evaluation models, scores traffic faster and for less than a large model, on the Enterprise tier. Signals, which the Galileo site calls its insights engine, scans your production traces for failure patterns nobody wrote an evaluator for. The AI Assistant answers questions about your traces. Agent Control adds guardrails that check and block actions at runtime. Datasets, experiments, playgrounds, annotations, and alerts round it out.

three.dev works on the LLM calls your feature already makes, with no evaluator for you to write. Your requests go through its proxy at gate.three.dev:

  1. Connect. Point the base URL at gate.three.dev and add three headers: your three.dev API key, a use case, and your provider. Your AI SDK and provider key stay the same. A setup skill lets your coding agent make the change for you.
  2. Find. After each response returns, three.dev scores a sample of requests in the background. Every defect is an issue with its reason and the quoted lines behind it. As issues pile up, three.dev names failure modes, describes them, gives each a severity of high, medium, or low, and lists the most severe first. Issues that fit no failure mode wait in Unmatched.
  3. Manage. Your team can change a severity, or dismiss a failure mode that does not matter and restore it later.
  4. Fix. Claude Code, Codex, or Cursor reads a failure mode and its failing requests over the MCP server at https://mcp.three.dev/mcp.
  5. Compare. Tag requests with a release, then set two releases or two date ranges side by side on quality (the share of scored requests without issues), latency (p50 and p95), and cost.

Chat lets you ask about your traffic in plain words, and its answers link to the requests behind them.

Where Galileo is the better choice

  • You need to stop bad actions, not only find them. Galileo's guardrails sit between the agent and its tools. They check each input, output, and tool call against your policies and block prompt injection, PII leaks, tool misuse, and off-path actions before they run. three.dev scores after the response returns and blocks nothing.
  • You want every request scored. Splunk says its Luna models score and protect 100% of traffic at a fraction of the cost of a large model, and Signals reads every trace in a scan. three.dev scores a sample. Note that Luna needs the Enterprise tier, and the hosted version supports five preset Luna evaluators: Prompt Injection, Toxicity, Sexism, PII, and Context Adherence.
  • You want a full eval workflow. Out-of-the-box evaluators for RAG, agents, safety, and text-to-SQL, custom LLM-as-a-judge and decision model evaluators (code evaluators too, on-premises), datasets, prompts, playgrounds, experiments, and annotations. Experiments can run inside unit tests in your CI/CD pipeline. three.dev offers no evals, datasets, experiments, or CI gates today.
  • You need alerts. Galileo alerts on evaluator scores, cost, latency, and trace counts by email, Slack, or a webhook such as PagerDuty. three.dev has no alerts today.
  • You need to trace whole agents and the stack under them. Galileo records traces, spans, and sessions across tool calls, retrieval, and agent steps, with distributed tracing across services. Inside Splunk, you see the GPUs and infrastructure on the same timeline. three.dev records each LLM call, including the tool calls the model asks for, and does not trace tool runs or retrieval as their own spans.
  • You need on-premises or air-gapped hosting. Galileo's on-premises version runs in your own environment and can run air-gapped. three.dev has no self-hosting and does not take health data (PHI) or government workloads yet.
  • You already pay for Splunk or Cisco. Agent Observability sits in Splunk Observability Cloud and Cisco Cloud Control, next to the tools your operations team already uses.
  • You want nothing in your request path. Galileo logs through its SDK or OpenTelemetry. three.dev is a proxy, so your LLM requests pass through it.

Where three.dev is the better choice

  • You have no evaluator and want results anyway. In Galileo, you log traces through its SDK or OpenTelemetry, add an LLM integration with your own model key, and turn on evaluators for each Agent Stream. Signals needs that LLM integration too, and it runs on demand or on a schedule you set. three.dev starts scoring with your first requests and names the failure modes itself.
  • You want scoring in the plan price. Galileo bills spans and Luna tokens, and its LLM-based evaluators, Signals, and the AI Assistant run on your own LLM integration, so your model provider bills those calls as well. Every three.dev plan includes scoring.
  • You want to start this week without a contract. Paid Agent Observability is billed annually, on a quote from Splunk. three.dev has a Free plan and monthly plans at $79 and $249.
  • You want to check each failure before you act on it. Every three.dev issue gives the reason the request failed and quotes the lines behind it.
  • You want your coding agent to start from the answer. A "Fix with your agent" button copies a prompt that starts the agent on one failure mode, and the MCP server works with Claude Code, Codex, Cursor, and other clients. Galileo's MCP server has setup steps for Cursor and VS Code, and its Signals can open a fix brief in Codex or Cursor.
  • You want to see a fix hold in production. Release tags let you set the fixed release beside the old one on quality, latency, and cost, on live traffic. Galileo compares experiments run on a dataset.
  • You would rather change a setting than add a tracing SDK. Your AI SDK stays as it is. three.dev documents OpenAI, Anthropic, Gemini, Azure OpenAI, OpenRouter, Concentrate, Amazon Bedrock, LiteLLM, the Vercel AI SDK, and LangChain. Splunk Agent Observability's current docs list a Python SDK and no TypeScript SDK.

Pricing

Splunk Agent Observability (Galileo) pricing, from Splunk's pricing page:

Observability Cloud Free Edition
Price$0
IncludedAgent Observability inside Splunk Observability Cloud, for up to 15 hosts, with no time limit and no credit card
NotesLuna models are only in the Enterprise tier, per Splunk's docs
Agent Observability
PriceFrom $100 a month per 1.2 million spans a year, billed annually
Included500 million Luna tokens and 5 Luna fine-tuning runs a year. Runs in Observability Cloud or self-hosted
NotesExtra Luna tokens and longer data retention cost more. Get a quote from Splunk

Billing counts the agent spans you send and the Luna tokens your evaluations use. LLM-based evaluators, Signals, and the AI Assistant run on the LLM integration you connect, so your model provider bills those calls too. On the hosted version, traces are kept 30 days by default and up to 180 days on request, and experiments are kept 2 years. Splunk's docs say Luna fine-tuning is not available on the hosted version. The old Galileo plan table, with its free tier and $100 Pro plan, is gone: galileo.ai/pricing now opens Splunk's pricing page.

three.dev has three plans: Free, Starter at $79 a month, and Pro at $249 a month. Each plan includes scoring and unlimited seats. three.dev is generally available: compare its plans, or start free and point it at your own traffic.

Our recommendation

Choose Galileo if you need runtime guardrails, Luna models that score every request, an on-premises or air-gapped install, or one Splunk contract for agents and the infrastructure under them. If your company already runs Splunk, start there: it does more today, and your operations team already knows it.

Choose three.dev if nobody on your team has time to write and tune evaluators, your users are the ones reporting failures, and a coding agent already writes most of your fixes. One base URL change gets you failure modes with a severity on each, ready for the agent, on a monthly plan.

Key facts

  • three.dev: a proxy at gate.three.dev that scores a sample of live LLM requests, explains each issue with quoted lines, and groups the issues into failure modes ranked by severity. Your team can change a severity, and dismiss or restore a failure mode. Generally available since November 2026.
  • Galileo: now Splunk Agent Observability, owned by Cisco since May 2026 and renamed on August 7, 2026. Tracing, evaluators, Luna models, datasets, experiments, Signals, an AI Assistant, alerts, and runtime guardrails, hosted in Splunk Observability Cloud or on-premises.
  • Work before the first failure shows up: in Galileo, the SDK or OpenTelemetry, an LLM integration with your own model key, and evaluators turned on for each Agent Stream. In three.dev, a base URL and three headers.
  • Prices: Galileo has a free way in through Splunk Observability Cloud Free Edition, and paid Agent Observability from $100 a month per 1.2 million spans a year, billed annually. three.dev Free is $0, Starter $79, and Pro $249 a month, with scoring included.
  • Missing from three.dev today: guardrails, alerts, CI gates, prompt management, datasets, experiments, and self-hosting. It does not take PHI or government workloads yet.

Sources

FAQ

Is three.dev a Galileo alternative?

For a team with a live AI feature, no evaluators yet, and a coding agent, yes. You change one base URL, and three.dev scores a sample of live traffic, sorts what it finds into failure modes by severity, and gives them to your coding agent. For a team that needs runtime guardrails, Luna models on every request, datasets, experiments, alerts, or an on-premises install, Galileo (now Splunk Agent Observability) covers more today.

Is Galileo the same as Splunk Agent Observability?

Yes. Cisco announced it would buy Galileo on April 9, 2026, and completed the deal by May 22, 2026. Galileo's docs say that as of August 7, 2026, Galileo is Splunk Agent Observability. The hosted version runs inside Splunk Observability Cloud, and an on-premises version is also offered. galileo.ai/pricing now opens Splunk's pricing page. The old Galileo docs still serve customers who joined before August 7, 2026.

Is Galileo free?

There is a free way in. Splunk Observability Cloud Free Edition includes Agent Observability, for up to 15 hosts, with no time limit and no credit card. Splunk's docs say Luna evaluation models are only in the Enterprise tier. Paid Agent Observability starts at $100 a month per 1.2 million spans a year, billed annually. three.dev has a Free plan at $0, with scoring included.

Does Galileo find production failures automatically?

Yes, with Signals, once you set it up. You log your traces through the Splunk Agent Observability SDK or OpenTelemetry and add an LLM integration with a reasoning model, which your model provider bills. Signals then scans your traces on demand or on a schedule at least 12 hours apart, groups related failures, and gives each a severity of Low, High, or Critical, labels, a root cause, and a recommended action. three.dev needs no SDK and no model key: it scores a sample of the traffic that passes through it and names the failure modes itself.

What is the difference between Galileo Signals and three.dev failure modes?

Both group failures from production into named patterns and rank them. Signals reads 100% of the traces in a scan, runs on your own LLM integration, and suggests an evaluator, alert, or trend view that tracks a Signal if it comes back. You can archive, merge, or rate Signals. three.dev scores a sample of live requests in the background, explains each issue with the quoted lines behind it, and sets a high, medium, or low severity on each failure mode. Your team can change a severity or dismiss a failure mode, and your coding agent reads them over MCP.

Can Galileo block a bad response before the user sees it? Can three.dev?

Galileo can. Its guardrails (Agent Control) check inputs, outputs, and tool calls while the agent runs, and can block prompt injection, PII leaks, tool misuse, and other risks before a tool fires. three.dev cannot. It scores each request after the response has returned, so it never changes a request or adds latency, and it does not block anything.

Does Galileo work with coding agents like Claude Code or Cursor?

Partly. Splunk Agent Observability has an MCP server, with setup steps for Cursor and VS Code with GitHub Copilot. It covers datasets, experiments, prompt templates, and Signals. On a Signal, Fix in your codebase opens a brief in Codex or Cursor, or copies it to your coding agent. three.dev's MCP server works with Claude Code, Codex, Cursor, and other clients, and changes only a failure mode's severity or whether it is dismissed, when you ask, and a Fix with your agent button copies a prompt that starts the agent on one failure mode.

Do I need an SDK to use three.dev?

No. You keep the AI SDK you call your model with and point its base URL at gate.three.dev, and add three headers: your three.dev API key, the use case, and your provider. Or paste this prompt into your coding agent and let it make the change: "Install github.com/round3ai/skills and run its three-dev-setup skill in this repository." three.dev is a proxy, so your LLM requests pass through it. Galileo logs through its SDK or OpenTelemetry instead.

How much does Galileo cost compared to three.dev?

Splunk lists Agent Observability from $100 a month per 1.2 million spans a year, billed annually, with 500 million Luna tokens a year. Extra Luna tokens and longer retention cost more, and you get a quote from Splunk. LLM-based evaluators, Signals, and the AI Assistant also run on your own model key. three.dev costs $0 on Free, $79 a month on Starter, and $249 a month on Pro, with scoring and unlimited seats in each, and no annual contract to start.

More comparisons

three.dev vs Braintrust

three.dev vs Braintrust (2026): which one fits your AI feature?

Braintrust is built around datasets, scorers, and CI, with failure discovery you turn on per project. three.dev scores live traffic and names the failure modes with no setup. Here is where each one fits.

As of November 18, 2026Read

three.dev vs Langfuse

three.dev vs Langfuse (2026): which one fits your AI feature?

Langfuse gives you open-source tracing and evals that you set up and run. three.dev names your failure modes from live traffic with nothing to configure. Here is where each one fits.

As of November 18, 2026Read

three.dev vs LangSmith

three.dev vs LangSmith (2026): which one fits your AI feature?

LangSmith runs the whole agent lifecycle, and its Engine finds failures on paid plans. three.dev finds them without moving your stack onto a platform. Here is where each one fits.

As of November 18, 2026Read

three.dev vs Raindrop

three.dev vs Raindrop (2026): which one fits your AI feature?

Raindrop and three.dev both find failures in production that nobody wrote a test for. Raindrop does it on its Pro plan, with alerts, full agent traces, and simulations around it. three.dev does it on every plan, from one base URL change. Here is where each one fits.

As of November 18, 2026Read

three.dev vs Arize AX

three.dev vs Arize AX (2026): which one fits your AI feature?

Arize AX traces, evaluates, and monitors your agents, and its Signal finds recurring failures on a schedule. three.dev names your failure modes from live traffic with no SDK and nothing to turn on. Here is where each one fits.

As of November 18, 2026Read

three.dev vs Comet Opik

three.dev vs Comet Opik (2026): which one fits your AI feature?

Opik gives you tracing, evals, test suites, and Diagnostics scans run by its Ollie assistant on credits. three.dev scores live traffic and names the failure modes with nothing to turn on. Here is where each one fits.

As of November 18, 2026Read

See how your AI feature fails.

Point the AI SDK you already use at one base URL, and get your failure modes named and ranked by severity.