Holmes vs Bug0
Both write tests in plain language, heal them when the UI changes and run on every pull request. Bug0 assigns a dedicated QA engineer who builds tests, checks every failure and gates releases, across Chrome, Firefox, Safari and Edge. Holmes is self-serve from a free plan, and adds agents that explore on their own, native mobile and API testing, and context from your specs and designs.
Holmes is self-serve, sets up in under ten minutes and has a Free plan with €250 in free credits, no credit card. Bug0's pricing page lists Managed at $2,500 a month, with a discounted 60-day pilot.
On Bug0's pages, the forward-deployed engineer explores your app and maps the critical flows. Holmes agents also explore by themselves and test paths nobody wrote down, on every change.
Bug0's pages describe web testing. Holmes also tests native Android and iOS apps on emulators and simulators and checks every request behind a journey, so web, mobile and API sit in one place.
Holmes reads the code change, the ticket, the spec in Notion or Confluence, the design in Figma and your analytics in PostHog before it tests. Jira, Linear, Figma and Notion aren't on Bug0's integrations page.
Based on each product's public pages on 17 November 2026. A red cross means Bug0's pages don't describe it, not that it can't be done.
| Capability | Holmes | Bug0 |
|---|---|---|
| Testing | ||
| Managed service | Not covered: Not covered. Self-serve, with guided onboarding on Team and up | Yes: Managed: a dedicated forward-deployed engineer plans, builds and maintains your tests and gates releases[1][2] |
| How tests are written | Yes: Plain-language goals. No test code to write | Yes: Plain-English steps run as Playwright actions. Studio: you create them from a screen recording, a text description or by hand[3][4] |
| Finds bugs nobody scripted | Yes: Agents explore and test paths nobody wrote down | Partly: Managed: the engineer explores the app to map critical flows. An agent that explores on its own isn't on its pages[5] |
| Keeps tests working when the UI changes | Yes: Flows repair themselves when a label or layout changes | Yes: Steps run from intent and heal when the UI changes. Managed: the engineer confirms each heal[6][7] |
| Browsers | Yes: Chromium, Firefox and WebKit, the engine behind Safari | Yes: Cloud-hosted Chromium. Managed: Chrome, Firefox, Safari and Edge, plus common viewports[8][9] |
| Real devices | Not covered: Not covered. Android emulators and iOS simulators | Yes: Managed: mobile web on real devices, including iOS Safari and Android Chrome[10] |
| Native Android apps | Yes: Native Android apps on an Android emulator | Not described: Not on its pages. Native iOS or Android apps |
| APIs | Yes: Every request behind a journey, checked in context | Not described: Not on its pages. API testing as a feature |
| Accessibility | Yes: WCAG audits in a real browser, with keyboard navigation | Not described: Not on its pages. WCAG audits |
| Workflow | ||
| Pull requests and previews | Yes: Check and comment on GitHub and Bitbucket pull requests. GitLab supported | Yes: Managed: runs on every pull request or commit through GitHub Actions, GitLab or any CI. Studio: validates preview deployments[11][12] |
| Schedules | Yes: On every deploy, on a schedule, or by webhook | Yes: Managed: on a schedule. Studio: every 15 minutes, hourly, daily or weekly, or from any CI through its API[11][13][12] |
| Context it reads before testing | Yes: Reads the code change, ticket, spec (Notion, Confluence), design (Figma) and analytics (PostHog) | Not described: Not on its pages. Jira, Linear, Figma, Notion or analytics |
| Coding agents (MCP) | Yes: Start runs and read findings from Claude and Cursor via MCP | Not described: Not on its pages. MCP server or coding-agent integration |
| Evidence with each bug | Yes: Recording, screenshots, console and network logs, steps to reproduce | Yes: Managed: video, network and console logs, repro steps. Studio: per-step status and screenshots[14][15] |
| Where findings go | Yes: Findings on the pull request, the ticket, Slack, Microsoft Teams and email | Yes: GitHub and Slack. Studio also lists email, Vercel and Netlify[16][17] |
| Test code you own | Yes: Plain-language goals. No test code to write | Partly: Tests run on its source-available engine, and its pricing page says tests live in your repos and stay portable. A February 2026 post says it won't export standalone Playwright scripts[29][18][19] |
| Pricing and security | ||
| What you pay for | Yes: Priced on how much Holmes tests, never on seats | Note: Managed: a flat monthly fee for one engineer and up to 500 user flows, pro rata beyond[20][21] |
| Entry price | Yes: Free plan. Paid plans from €199 ($219) a month | Note: Managed $2,500 a month, month-to-month. Studio isn't on the pricing page; Bug0's general terms (last updated June 2025) say pricing starts at $250 a month[20][22] |
| Getting started | Yes: €250 in free credits, no credit card | Yes: Managed: discounted 60-day pilot. Studio: sign in with Google[23][24] |
Holmes sells QA software, so we compete with Bug0. We've tried to be fair: every statement about Bug0 comes from its own public pages, linked below with the date we checked them. Products change. If something here is out of date, email support@useholmes.com and we'll fix it.
Bug0 is a managed QA testing company built on its own AI platform, not a DIY tool.Checked 17 November 2026.
A forward-deployed QA engineer plans coverage, builds your tests on Bug0's agentic AI engine, verifies results, files bugs, and gates releases.Checked 17 November 2026.
Tests are written as plain-English steps and executed as Playwright actions by AI.Checked 17 November 2026.
Record your screen performing the flow, and AI extracts test steps automatically.Checked 17 November 2026.
Your FDE explores the app, maps your critical flows (signups, checkouts, dashboards) and drafts the coverage plan.Checked 17 November 2026.
Tests auto-heal. 0% flake rate. Bug0 AI, our open-source engine, executes each step from intent.Checked 17 November 2026.
The engineer confirms each heal, which is what stops a healed test from passing while it checks nothing.Checked 17 November 2026.
Cloud-hosted Chromium tests with realistic data and fixtures.Checked 17 November 2026.
Validate your app across major browsers (Chrome, Firefox, Safari, Edge) and common viewports.Checked 17 November 2026.
Running suites on macOS Safari, Windows Edge, iOS Safari, and Android Chrome.Checked 17 November 2026.
GitHub Actions and the Bug0 app. Tests run on every PR, every commit, or on a schedule.Checked 17 November 2026.
Call the /api/run-tests endpoint directly from any CI/CD system.Checked 17 November 2026.
Every 15 minutes Critical path monitoringChecked 17 November 2026.
files a report with video recording, network logs, console logs, and repro steps.Checked 17 November 2026.
Screenshots - Visual state at key pointsChecked 17 November 2026.
Reports go to GitHub, Slack, or wherever you need them.Checked 17 November 2026.
Connect your GitHub repository for:Checked 17 November 2026.
Can I export my Bug0 tests as Playwright scripts?Checked 17 November 2026.
Bug0 AI is licensed under FSL-1.1-Apache-2.0.Checked 17 November 2026.
$2,500/mo flat. Month-to-month. No annual contracts.Checked 17 November 2026.
Up to 500 user flows. Pro rata beyond.Checked 17 November 2026.
Pricing starts at $250/month and scales based on usage.Checked 17 November 2026.
We offer a discounted 60-day pilot. Month-to-month, no annual commitment. Cancel anytime.Checked 17 November 2026.
Click Sign in with Google (use your work email)Checked 17 November 2026.
We are opening up Bug0 Studio v0.1 in research preview. This is the internal tool our FDE team uses to turn natural language and video into clean, reliable Playwright tests.Checked 17 November 2026.
BAAs available for healthcare customers.Checked 17 November 2026.
Bug0's AI agents and forward-deployed engineers catch what breaks before your customers hear it.Checked 17 November 2026.
We're not a fit for mobile-native apps.Checked 17 November 2026.
You own the artifacts. Tests live in your repos and CI.Checked 17 November 2026.
Bug0 is first a managed service. A dedicated forward-deployed engineer plans coverage, builds tests on Bug0's AI engine, checks every failure and gates your releases. Holmes is a self-serve product: its agents test your web app, mobile app and APIs, explore on their own, and read your specs and designs, with results on the pull request.[1][2]
Yes. Bug0 Studio is the tool its own engineers use, opened as a v0.1 research preview. Its docs cover tests from a screen recording or a text description, pull request runs, schedules, and Slack and email results. Bug0's pricing page lists only Managed. Its general terms of service, last updated in June 2025, say pricing starts at $250 a month and scales with usage; they don't name Studio.[25][4][20][22]
Bug0 Managed is $2,500 a month flat, month-to-month, for one engineer and up to 500 user flows, with pro rata pricing beyond that and a discounted 60-day pilot. Holmes has a Free plan, Starter at €199 a month for up to 3 products, and Team from €499 a month, all with unlimited users. Holmes is priced on how much it tests, never on seats.[20][21][23]
Bug0 says your tests stay portable: its pricing page says tests live in your repos and CI, and its engine, Bug0 AI, is source-available under FSL-1.1-Apache-2.0 and runs in any Playwright project. A February 2026 post says it won't build a button that exports standalone Playwright scripts. Ask Bug0 what you keep if you leave. Holmes tests are plain-language goals, with no test code to maintain.[29][18][19]
If you want a dedicated QA engineer to plan, write, maintain and triage your tests for you, Bug0 is built for that. Its managed service also covers Chrome, Firefox, Safari and Edge, mobile web on real devices including iOS Safari, BAAs for healthcare customers, and testing for voice and chat AI agents. Its managed page says it isn't a fit for mobile-native apps. If you want to run QA yourself, starting free, with native mobile apps, API testing and context from your specs and designs, try Holmes.[2][9][10][26][27][28]
Evaluating Bug0? See what Holmes finds first.
Free, no scripts, no SDK, and no test account needed to start.