Holmes vs Katalon
Both cover web, mobile and API testing, with AI that helps create tests and repair them. Katalon goes wider, with real devices, desktop apps, test management and on-premises deployment. Holmes adds agents that write and explore tests from plain-language goals, pull request checks on previews, and unlimited users.
In Katalon, your team writes tests in Studio, with AI to help. Holmes takes a plain-language goal, and its agents write, run and repair the test, with no test code to maintain.
Katalon TrueTest finds coverage gaps by watching real users in production. Holmes agents explore your app, previews included, and test paths nobody wrote down.
Holmes checks and comments on GitHub and Bitbucket pull requests and tests the preview deployment before you merge. Katalon runs in your CI pipeline on each change; pull request comments and preview deployments aren't on its pages.
Katalon prices by seats, sessions and licenses, and CI runs need Runtime Engine add-ons. Holmes has unlimited users on every plan and is priced on how much it tests.
Based on each product's public pages on 17 November 2026. A red cross means Katalon's pages don't describe it, not that it can't be done.
| Capability | Holmes | Katalon |
|---|---|---|
| Testing | ||
| How tests are written | Yes: Plain-language goals. No test code to write | Yes: No-code, low-code, full code or AI-assisted, written by your team in Studio[1] |
| Finds bugs nobody scripted | Yes: Agents explore and test paths nobody wrote down | Not described: Not on its pages. Exploring the app without scripted tests or production data |
| Keeps tests working when the UI changes | Yes: Flows repair themselves when a label or layout changes | Yes: Self-healing locators; Katalon AI proposes the fix and you review it[1] |
| Other browsers | Yes: Chromium, Firefox and WebKit, the engine behind Safari | Yes: Chrome, Firefox, Safari and Edge on Windows and macOS[2] |
| Native Android apps | Yes: Native Android apps on an Android emulator | Yes: Native, hybrid and mobile web, on emulators or real devices[3] |
| Native iOS apps | Yes: Native iOS apps on iPhone and iPad simulators | Yes: Native, hybrid and mobile web, on simulators or real devices[3] |
| Real devices | Not covered: Not covered. Android emulators and iOS simulators | Yes: Real iOS and Android hardware in TestCloud[2] |
| APIs | Yes: Every request behind a journey, checked in context | Yes: REST, SOAP and GraphQL, with OpenAPI, Postman or WSDL import[4] |
| Visual testing | Partly: Compares the Figma design with what shipped. No pixel-diff baselines | Yes: Pixel, layout and text-based image comparison[5] |
| Accessibility | Yes: WCAG audits in a real browser, with keyboard navigation | Yes: WCAG checks through the axe-core integration[6] |
| Workflow | ||
| Pull requests and previews | Yes: Check and comment on GitHub and Bitbucket pull requests. GitLab supported | Partly: Runs as a CI check on pull requests through Runtime Engine. Pull request comments and preview deployments aren't on its pages[6][7] |
| CI and schedules | Yes: On every deploy, on a schedule, or by webhook | Yes: Runs in GitHub Actions, GitLab, Jenkins, Azure DevOps and more; headless runs need Runtime Engine add-ons[6][7] |
| Context it reads before testing | Yes: Reads the code change, ticket, spec (Notion, Confluence), design (Figma) and analytics (PostHog) | Partly: Requirements, Jira and Azure DevOps tickets, Confluence pages and Figma designs. Notion, Linear or analytics aren't on its integrations page[1][6] |
| Coding agents (MCP) | Yes: Start runs and read findings from Claude and Cursor via MCP | Yes: MCP server for Copilot, Claude, Codex, Cursor and others[1] |
| Evidence with each bug | Yes: Recording, screenshots, console and network logs, steps to reproduce | Yes: Screenshots, device logs and an AI log on every step[2][8] |
| Where findings go | Yes: Findings on the pull request, the ticket, Slack, Microsoft Teams and email | Yes: Jira, Slack and Microsoft Teams[6] |
| Pricing and security | ||
| What you pay for | Yes: Priced on how much Holmes tests, never on seats | Note: Seats, TestCloud sessions and Runtime Engine licenses; no per-run fees[7] |
| Entry price | Yes: Free plan. Paid plans from €199 ($219) a month | Note: Free plan; Studio Professional $1,800 a seat a year, or $180 a seat a month billed monthly[7] |
| Where it runs | Not covered: Not covered. Cloud only | Yes: Private cloud or on-premises on Enterprise; US and EU hosting[7][9] |
Holmes sells QA software, so we compete with Katalon. We've tried to be fair: every statement about Katalon comes from its own public pages, linked below with the date we checked them. Products change. If something here is out of date, email support@useholmes.com and we'll fix it.
Web, mobile, API and desktop applications, from one platform, with no-code, low-code, full-code and AI-assisted authoring in the same project.Checked 17 November 2026.
Cross-browser test execution against Chrome, Firefox, Safari and Edge on Windows and macOS, plus real iOS and Android devices, all at once.Checked 17 November 2026.
Build tests on an Android emulator, an iOS simulator, or a phone plugged into your machine, then run releases on real devices.Checked 17 November 2026.
Import your OpenAPI, Postman, or WSDL definitions, verify every response, and run the same API suite in CI, alongside your web and mobile tests.Checked 17 November 2026.
Aside from the pixel comparison approach, we provide two new image comparison methods using AI: layout-based and text/content-based.Checked 17 November 2026.
Katalon integrates with 86 tools across 12 categoriesChecked 17 November 2026.
There are no per-run or usage fees. Katalon prices by seats, sessions and licenses, not by how much you run.Checked 17 November 2026.
Screenshots and an AI log on every step, ready for review, defects, and audits.Checked 17 November 2026.
Regional hosting options are available in the United States and Europe; availability depends on the product and deployment model.Checked 17 November 2026.
TrueTest watches real users in production, finds the journeys that matter, and generates tests for your regression coverage gaps.Checked 17 November 2026.
Katalon is a full testing platform your team writes tests in, with no-code, low-code and full-code authoring plus AI help, and it covers real devices and desktop apps. Holmes is an AI agent: you describe goals in plain language, and it tests web, mobile and APIs, explores paths nobody wrote down, and checks every pull request and preview.[1]
Katalon tests native, hybrid and mobile web apps on Android and iOS, on emulators, simulators and real devices in TestCloud. Holmes tests native Android and iOS apps on emulators and simulators; real devices are not covered.[3][2]
Katalon prices by seats, sessions and licenses, with no per-run fees. There is a Free plan, and Studio Professional is $1,800 a seat a year, or $180 a seat a month billed monthly; running in CI needs Runtime Engine add-ons. Holmes has a Free plan, Starter at €199 a month for up to 3 products, and Team from €499 a month, with unlimited users on every plan.[7]
TrueTest watches real users in production, finds the journeys that matter and generates tests for gaps in your regression suite. Holmes does not need production traffic: its agents explore your app, preview deployments included, and test paths nobody wrote down.[10]
If you need real-device testing, Windows desktop apps, one system of record for manual and automated tests, or on-premises deployment, Katalon covers those on its own pages. It also suits teams who want to keep writing tests in code. If you want agents that write and explore tests from plain-language goals, pull request checks on previews and unlimited users, try Holmes.[2][7][8]
Evaluating Katalon? See what Holmes finds first.
Free, no scripts, no SDK, and no test account needed to start.