Compare
three.dev vs Galileo (2026): which one fits your AI feature?
Galileo is now Splunk Agent Observability: evals, Luna models, Signals, and runtime guardrails, sold through Splunk. three.dev names your failure modes from live traffic with nothing to configure. Here is where each one fits.
Updated
The short answer
Pick three.dev if your AI feature is live, nobody has written an evaluator for it yet, and your team codes with Claude Code, Codex, or Cursor. In Galileo you first log traces with its SDK, connect your own model key, and turn on evaluators. three.dev skips that step. You change one base URL, and it scores a sample of live requests, sorts the issues into named failure modes with the most severe on top, and passes each one to your coding agent.
Pick Galileo, now sold as Splunk Agent Observability, if you need guardrails that block bad actions while the agent runs, evaluation models that score every request, and evals, datasets, experiments, and alerts in one place. It fits best at a larger company that already runs Splunk or Cisco tools, or that needs an on-premises or air-gapped install.
At a glance
A live AI feature with no evaluators yet, and a team that fixes with a coding agent
Larger companies on Splunk or Cisco tools that want agent tracing, evals, and runtime guardrails in one place, in the cloud or on-premises
A proxy that scores live LLM traffic, names its failure modes, and hands the fix to your coding agent
Splunk Agent Observability, formerly Galileo: tracing, evaluators, Luna evaluation models, datasets, experiments, Signals, an AI Assistant, and runtime guardrails
Base URL gate.three.dev and three headers: a three.dev API key, a use case, and your provider. Your AI SDK and provider key stay the same. Works through LiteLLM and the other documented providers
The Python SDK (splunk-ao), an OpenAI SDK wrapper, OpenTelemetry or OpenInference, or the API. Integrations for LangChain and LangGraph, CrewAI, the OpenAI Agents SDK, Google ADK, Claude, and more
None. No SDK, LLM integration, or evaluator to turn on. Scoring begins with your first requests
Log traces with the SDK or OpenTelemetry. Add an LLM integration with your own model key for LLM-based evaluators, Signals, and the AI Assistant. Turn on evaluators for each Agent Stream, with an optional sampling rate
A sample of requests is scored after each response returns. Each issue gives the reason and quotes the lines behind it
Issues group into named failure modes, each with a severity, most severe first
Signals groups related failures across production traces and gives each a severity of Low, High, or Critical, labels from 45 failure types, a root cause, and a recommended action. Scans run on demand or on a schedule at least 12 hours apart
Evaluators you turn on score traces for the checks they cover
three.dev sets each severity, and your team can change it. Your team can also dismiss a failure mode that is not worth acting on, and restore it later
You can archive, merge, or unmerge Signals, and rate each one. The Remediate tab suggests an evaluator, alert, or trend view that tracks a Signal if it comes back
Chat answers plain questions about your traffic and links each answer to the requests behind it
The AI Assistant answers questions about traces, sessions, and evaluator scores, with links to the evidence. It runs on your LLM integration and is read-only
A Fix with your agent button copies a prompt that starts your coding agent on one failure mode. The MCP server gives it the failing requests and each issue's reasoning
The Remediate tab lists recommended fixes. Fix in your codebase opens a brief in Codex or Cursor, or copies it to your coding agent, which reads the Signal over MCP. The agent proposes the change for you to apply
An MCP server with failure modes, failing requests, and docs. It reads your data, and changes only a failure mode's severity or whether it is dismissed, when you ask. Plugins for Claude Code and Codex install it; Cursor and other clients paste its URL
An MCP server for Cursor and VS Code with GitHub Copilot: datasets, experiments, prompt templates, and Signals, including the fixes a Signal recommends
Tag requests with a release, then compare two releases or two date ranges on quality, each failure mode, p50 and p95 latency, and cost
Experiments run your app on a dataset with your evaluators, and you compare the runs
LLM calls and the tool calls the model asks for, grouped into sessions. Tool runs and retrieval are not separate spans
Traces, spans, and sessions: LLM calls, tool calls, retrieval, and agent steps, with distributed tracing across services. Splunk shows the GPU and infrastructure behind them
In the path, as a proxy. Scoring waits until the response has returned, so it adds no latency
Not a proxy: traces go out through the SDK or OpenTelemetry. Guardrails (Agent Control), if you add them, check inputs, outputs, and tool calls during the run and can block them
Not offered today
Yes: out-of-the-box and custom evaluators, datasets, prompts, playgrounds, experiments, and annotations
Not offered. Sessions keep whole conversations for review
Session-level evaluators, such as Conversation Quality, score whole conversations
Not offered today
Experiments run inside unit tests in your CI/CD pipeline, and the test asserts on average evaluator scores
Not offered today
Alerts on evaluator, cost, latency, and trace-count thresholds, sent by email, Slack, or a generic webhook such as PagerDuty
Scoring is included in the plan price
Billing counts spans and Luna tokens. LLM-based evaluators, Signals, and the AI Assistant run on your own LLM integration, so your model provider bills those calls. Sampling lowers the evaluator cost
30 days on Free, 90 on Starter, 180 on Pro
SaaS traces: 30 days by default, or 60, 90, or 180 days through your Splunk account team. Experiments: 2 years
Not open source. No self-hosting today
An on-premises deployment that you install and run, including air-gapped. No open-source edition on the pages we read
three.dev does not train models on your traffic for anyone other than you
SaaS regions in the US, Europe, and Asia Pacific. Splunk offers a data processing addendum (DPA)
Not for health data (PHI) or government workloads yet
On-premises or air-gapped deployment, and guardrails that block PII, PHI, and PCI leaks. The Agent Observability pages we read list no audit reports of their own
Free, Starter $79, or Pro $249 a month, each with scoring and unlimited seats
Splunk Observability Cloud Free Edition includes Agent Observability. Paid plans start at "$100 per month per 1.2 million spans annually, billed annually", with 500 million Luna tokens a year
What each product is
Galileo was an AI evaluation and observability company. Cisco announced it would buy Galileo on April 9, 2026, and completed the deal by May 22, 2026. Galileo's docs say that as of August 7, 2026, "Galileo is now Splunk Agent Observability". Splunk calls it "the leading observability, evaluation, and guardrail platform for GenAI and agentic applications". The hosted version runs inside Splunk Observability Cloud. An on-premises version can run in your own environment, air-gapped if you need.
You log your app with the Python SDK, an OpenAI wrapper, OpenTelemetry, or the API. You turn on evaluators for each Agent Stream, from a library of out-of-the-box checks or your own LLM-as-a-judge, decision model, or code evaluators, and you can sample to save cost. Luna, Galileo's family of small evaluation models, scores traffic faster and for less than a large model, on the Enterprise tier. Signals, which the Galileo site calls its insights engine, scans your production traces for failure patterns nobody wrote an evaluator for. The AI Assistant answers questions about your traces. Agent Control adds guardrails that check and block actions at runtime. Datasets, experiments, playgrounds, annotations, and alerts round it out.
three.dev works on the LLM calls your feature already makes, with no evaluator for you to write. Your requests go through its proxy at gate.three.dev:
- Connect. Point the base URL at gate.three.dev and add three headers: your three.dev API key, a use case, and your provider. Your AI SDK and provider key stay the same. A setup skill lets your coding agent make the change for you.
- Find. After each response returns, three.dev scores a sample of requests in the background. Every defect is an issue with its reason and the quoted lines behind it. As issues pile up, three.dev names failure modes, describes them, gives each a severity of high, medium, or low, and lists the most severe first. Issues that fit no failure mode wait in Unmatched.
- Manage. Your team can change a severity, or dismiss a failure mode that does not matter and restore it later.
- Fix. Claude Code, Codex, or Cursor reads a failure mode and its failing requests over the MCP server at
https://mcp.three.dev/mcp. - Compare. Tag requests with a release, then set two releases or two date ranges side by side on quality (the share of scored requests without issues), latency (p50 and p95), and cost.
Chat lets you ask about your traffic in plain words, and its answers link to the requests behind them.
Where Galileo is the better choice
- You need to stop bad actions, not only find them. Galileo's guardrails sit between the agent and its tools. They check each input, output, and tool call against your policies and block prompt injection, PII leaks, tool misuse, and off-path actions before they run. three.dev scores after the response returns and blocks nothing.
- You want every request scored. Splunk says its Luna models score and protect 100% of traffic at a fraction of the cost of a large model, and Signals reads every trace in a scan. three.dev scores a sample. Note that Luna needs the Enterprise tier, and the hosted version supports five preset Luna evaluators: Prompt Injection, Toxicity, Sexism, PII, and Context Adherence.
- You want a full eval workflow. Out-of-the-box evaluators for RAG, agents, safety, and text-to-SQL, custom LLM-as-a-judge and decision model evaluators (code evaluators too, on-premises), datasets, prompts, playgrounds, experiments, and annotations. Experiments can run inside unit tests in your CI/CD pipeline. three.dev offers no evals, datasets, experiments, or CI gates today.
- You need alerts. Galileo alerts on evaluator scores, cost, latency, and trace counts by email, Slack, or a webhook such as PagerDuty. three.dev has no alerts today.
- You need to trace whole agents and the stack under them. Galileo records traces, spans, and sessions across tool calls, retrieval, and agent steps, with distributed tracing across services. Inside Splunk, you see the GPUs and infrastructure on the same timeline. three.dev records each LLM call, including the tool calls the model asks for, and does not trace tool runs or retrieval as their own spans.
- You need on-premises or air-gapped hosting. Galileo's on-premises version runs in your own environment and can run air-gapped. three.dev has no self-hosting and does not take health data (PHI) or government workloads yet.
- You already pay for Splunk or Cisco. Agent Observability sits in Splunk Observability Cloud and Cisco Cloud Control, next to the tools your operations team already uses.
- You want nothing in your request path. Galileo logs through its SDK or OpenTelemetry. three.dev is a proxy, so your LLM requests pass through it.
Where three.dev is the better choice
- You have no evaluator and want results anyway. In Galileo, you log traces through its SDK or OpenTelemetry, add an LLM integration with your own model key, and turn on evaluators for each Agent Stream. Signals needs that LLM integration too, and it runs on demand or on a schedule you set. three.dev starts scoring with your first requests and names the failure modes itself.
- You want scoring in the plan price. Galileo bills spans and Luna tokens, and its LLM-based evaluators, Signals, and the AI Assistant run on your own LLM integration, so your model provider bills those calls as well. Every three.dev plan includes scoring.
- You want to start this week without a contract. Paid Agent Observability is billed annually, on a quote from Splunk. three.dev has a Free plan and monthly plans at $79 and $249.
- You want to check each failure before you act on it. Every three.dev issue gives the reason the request failed and quotes the lines behind it.
- You want your coding agent to start from the answer. A "Fix with your agent" button copies a prompt that starts the agent on one failure mode, and the MCP server works with Claude Code, Codex, Cursor, and other clients. Galileo's MCP server has setup steps for Cursor and VS Code, and its Signals can open a fix brief in Codex or Cursor.
- You want to see a fix hold in production. Release tags let you set the fixed release beside the old one on quality, latency, and cost, on live traffic. Galileo compares experiments run on a dataset.
- You would rather change a setting than add a tracing SDK. Your AI SDK stays as it is. three.dev documents OpenAI, Anthropic, Gemini, Azure OpenAI, OpenRouter, Concentrate, Amazon Bedrock, LiteLLM, the Vercel AI SDK, and LangChain. Splunk Agent Observability's current docs list a Python SDK and no TypeScript SDK.
Pricing
Splunk Agent Observability (Galileo) pricing, from Splunk's pricing page:
| Plan | Price | Included | Notes |
|---|---|---|---|
Observability Cloud Free Edition | $0 | Agent Observability inside Splunk Observability Cloud, for up to 15 hosts, with no time limit and no credit card | Luna models are only in the Enterprise tier, per Splunk's docs |
Agent Observability | From $100 a month per 1.2 million spans a year, billed annually | 500 million Luna tokens and 5 Luna fine-tuning runs a year. Runs in Observability Cloud or self-hosted | Extra Luna tokens and longer data retention cost more. Get a quote from Splunk |
Billing counts the agent spans you send and the Luna tokens your evaluations use. LLM-based evaluators, Signals, and the AI Assistant run on the LLM integration you connect, so your model provider bills those calls too. On the hosted version, traces are kept 30 days by default and up to 180 days on request, and experiments are kept 2 years. Splunk's docs say Luna fine-tuning is not available on the hosted version. The old Galileo plan table, with its free tier and $100 Pro plan, is gone: galileo.ai/pricing now opens Splunk's pricing page.
three.dev has three plans: Free, Starter at $79 a month, and Pro at $249 a month. Each plan includes scoring and unlimited seats. three.dev is generally available: compare its plans, or start free and point it at your own traffic.
Our recommendation
Choose Galileo if you need runtime guardrails, Luna models that score every request, an on-premises or air-gapped install, or one Splunk contract for agents and the infrastructure under them. If your company already runs Splunk, start there: it does more today, and your operations team already knows it.
Choose three.dev if nobody on your team has time to write and tune evaluators, your users are the ones reporting failures, and a coding agent already writes most of your fixes. One base URL change gets you failure modes with a severity on each, ready for the agent, on a monthly plan.
Key facts
- three.dev: a proxy at gate.three.dev that scores a sample of live LLM requests, explains each issue with quoted lines, and groups the issues into failure modes ranked by severity. Your team can change a severity, and dismiss or restore a failure mode. Generally available since November 2026.
- Galileo: now Splunk Agent Observability, owned by Cisco since May 2026 and renamed on August 7, 2026. Tracing, evaluators, Luna models, datasets, experiments, Signals, an AI Assistant, alerts, and runtime guardrails, hosted in Splunk Observability Cloud or on-premises.
- Work before the first failure shows up: in Galileo, the SDK or OpenTelemetry, an LLM integration with your own model key, and evaluators turned on for each Agent Stream. In three.dev, a base URL and three headers.
- Prices: Galileo has a free way in through Splunk Observability Cloud Free Edition, and paid Agent Observability from $100 a month per 1.2 million spans a year, billed annually. three.dev Free is $0, Starter $79, and Pro $249 a month, with scoring included.
- Missing from three.dev today: guardrails, alerts, CI gates, prompt management, datasets, experiments, and self-hosting. It does not take PHI or government workloads yet.
Sources
- Galileo homepage
- Galileo Signals
- What is Galileo? (rename notice)
- Cisco: intent to acquire Galileo, with the May 22 completion update
- Splunk Observability + Galileo
- Splunk Agent Observability
- Splunk pricing
- Splunk Observability Cloud Free Edition
- Splunk runtime guardrails
- Splunk data security and privacy
- What is Splunk Agent Observability?
- Agent Observability (SaaS)
- Billing and usage
- Signals
- Signals configuration and schedules (API)
- AI Assistant
- MCP server
- Log your first trace
- Evaluate your traces
- Evaluators overview
- Agent Stream evaluators and sampling
- Luna-3
- Integrations
- Agent Control
- Alerts
- Experiments in unit tests
- Release notes (SaaS)
- Release notes (on-premises)
- three.dev docs: sending requests
- three.dev docs: AI providers
- three.dev docs: live scoring
- three.dev docs: failure modes
FAQ
Is three.dev a Galileo alternative?
For a team with a live AI feature, no evaluators yet, and a coding agent, yes. You change one base URL, and three.dev scores a sample of live traffic, sorts what it finds into failure modes by severity, and gives them to your coding agent. For a team that needs runtime guardrails, Luna models on every request, datasets, experiments, alerts, or an on-premises install, Galileo (now Splunk Agent Observability) covers more today.
Is Galileo the same as Splunk Agent Observability?
Yes. Cisco announced it would buy Galileo on April 9, 2026, and completed the deal by May 22, 2026. Galileo's docs say that as of August 7, 2026, Galileo is Splunk Agent Observability. The hosted version runs inside Splunk Observability Cloud, and an on-premises version is also offered. galileo.ai/pricing now opens Splunk's pricing page. The old Galileo docs still serve customers who joined before August 7, 2026.
Is Galileo free?
There is a free way in. Splunk Observability Cloud Free Edition includes Agent Observability, for up to 15 hosts, with no time limit and no credit card. Splunk's docs say Luna evaluation models are only in the Enterprise tier. Paid Agent Observability starts at $100 a month per 1.2 million spans a year, billed annually. three.dev has a Free plan at $0, with scoring included.
Does Galileo find production failures automatically?
Yes, with Signals, once you set it up. You log your traces through the Splunk Agent Observability SDK or OpenTelemetry and add an LLM integration with a reasoning model, which your model provider bills. Signals then scans your traces on demand or on a schedule at least 12 hours apart, groups related failures, and gives each a severity of Low, High, or Critical, labels, a root cause, and a recommended action. three.dev needs no SDK and no model key: it scores a sample of the traffic that passes through it and names the failure modes itself.
What is the difference between Galileo Signals and three.dev failure modes?
Both group failures from production into named patterns and rank them. Signals reads 100% of the traces in a scan, runs on your own LLM integration, and suggests an evaluator, alert, or trend view that tracks a Signal if it comes back. You can archive, merge, or rate Signals. three.dev scores a sample of live requests in the background, explains each issue with the quoted lines behind it, and sets a high, medium, or low severity on each failure mode. Your team can change a severity or dismiss a failure mode, and your coding agent reads them over MCP.
Can Galileo block a bad response before the user sees it? Can three.dev?
Galileo can. Its guardrails (Agent Control) check inputs, outputs, and tool calls while the agent runs, and can block prompt injection, PII leaks, tool misuse, and other risks before a tool fires. three.dev cannot. It scores each request after the response has returned, so it never changes a request or adds latency, and it does not block anything.
Does Galileo work with coding agents like Claude Code or Cursor?
Partly. Splunk Agent Observability has an MCP server, with setup steps for Cursor and VS Code with GitHub Copilot. It covers datasets, experiments, prompt templates, and Signals. On a Signal, Fix in your codebase opens a brief in Codex or Cursor, or copies it to your coding agent. three.dev's MCP server works with Claude Code, Codex, Cursor, and other clients, and changes only a failure mode's severity or whether it is dismissed, when you ask, and a Fix with your agent button copies a prompt that starts the agent on one failure mode.
Do I need an SDK to use three.dev?
No. You keep the AI SDK you call your model with and point its base URL at gate.three.dev, and add three headers: your three.dev API key, the use case, and your provider. Or paste this prompt into your coding agent and let it make the change: "Install github.com/round3ai/skills and run its three-dev-setup skill in this repository." three.dev is a proxy, so your LLM requests pass through it. Galileo logs through its SDK or OpenTelemetry instead.
How much does Galileo cost compared to three.dev?
Splunk lists Agent Observability from $100 a month per 1.2 million spans a year, billed annually, with 500 million Luna tokens a year. Extra Luna tokens and longer retention cost more, and you get a quote from Splunk. LLM-based evaluators, Signals, and the AI Assistant also run on your own model key. three.dev costs $0 on Free, $79 a month on Starter, and $249 a month on Pro, with scoring and unlimited seats in each, and no annual contract to start.