TestMaze
MCP Verifier
THE VERIFIER HARNESS FOR AI CODING AGENTS

Stop letting AI grade
its own homework.

TestMaze is the MCP-native verifier for Claude Code, Cursor, Cline, and every coding agent that speaks Model Context Protocol. Your agent writes the code. We author the tests, run them headlessly, and return deterministic verdicts — with KPI gates the agent can't override.

Source-blind by design
Deterministic verdicts
KPI gates enforced
Zero self-grading
gitSha-pinned runs

Works with any MCP client. No vendor lock-in. Self-hosted or managed.

~/checkout-v2 · claude code
$ implement checkout flow
→ edited 3 files, ran unit tests ✓
$ testrun.create && pdlc.verify
verdict: fail · 14 passed, 2 failed
text-mismatch · tc_204
nextStep: repair_code
caseId tc_204 · failureBucket text-mismatch
Speaks MCP · works withClaude Code·Cursor·Cline·Aider·Continue·Goose
THE GAP

AI coding agents are great writers.
They're terrible judges.

When the same agent that writes the code also decides whether it works, every CI green-light is a self-graded exam. You ship faster — and you ship more bugs.

Self-grading drift

Agent edits the assertion until the test passes. The diff looks clean. The bug ships.

No shared quality bar

Every agent invents its own KPI. Selector-failure share, snippet quality, reliability — none of it is enforced.

Hidden flakiness

Retries hide real regressions. "Transient" failures pile up until the suite is unrunnable.

THE PDLC LOOP

One protocol. Six states. One source of truth.

Every coding session becomes a PdlcSession — a finite state machine that only the MCP server can advance. The agent drives; we verify.

01 · INTAKE
project.initialize

Repo surveyed, product metadata populated, PdlcSession opened. JWT bound to the space.

02 · PLAN
feature.implement

feature.list runs first so duplicates are skipped. Then the feature-spec prompt expands the spec into user stories + 8-facet acceptance criteria — rendered locally by your IDE’s LLM.

03 · AUTHOR
feature.verify

ac-to-testcase prompt fleshes out AC shells into complete test bodies with facet-appropriate assertions. Both prompts run client-side via prompts/get — we never see the rendered expansion, only the structured JSON your agent posts back.

04 · CODE
case.update + your agent

Your coding agent writes the application code that satisfies each AC. We hold the line.

05 · VERIFY
pdlc.verify

testrun.create executes headlessly; per-case evidence screenshots attach via testrun.record_results. pdlc.verify composes the verdict — failure buckets, reliability KPI, deterministic nextStep.

06 · SHIP
release.ship

Verdict=pass. gitSha pinned. Working tree clean. The release moves to status=shipped.

On failure: bucketed hints feed back into CODE. On success: only the verifier can advance to SHIP.
Throughout: every tool call narrated in plain English on the live MCP Sessions view inside your space — “Created test case ‘Pay button disabled when cart empty’ for feature ‘Apple Pay’.”
SOURCE-BLIND BY DESIGN

We verify the work.
We never see the code.

Your repository lives on your laptop, in your IDE, with your own LLM. Test Maze only ever sees what your agent explicitly hands us: test artefacts, run results, and git identifiers. No diffs, no files, no streamed source — by design.

ON YOUR MACHINE

Stays right here.

Every source file
.ts / .py / .go / .sql — whatever the agent edits, stays on disk.
.env, secrets, credentials
We never read your environment. Your secrets are not in scope.
The LLM that powers your IDE
Claude / GPT / Gemini runs in your client. We never proxy LLM traffic.
Rendered prompt output
feature-spec and ac-to-testcase expansions execute client-side. The diff between template and rendered prompt never reaches us.
Git diffs & file contents
We receive the gitSha + branch + workingTreeClean flag — identifiers, not contents.
REACHES TEST MAZE

Just the verdict layer.

Test case rows
Titles, AC labels, steps, expected results — the contract your code is built against.
Test run results
Pass / fail status per case, failure-bucket classification, KPI scoring.
Git identifiers
gitSha, branch name, workingTreeClean flag. We pin verdicts to revisions, never to file contents.
Evidence — opt-in only
Screenshots attach to a failed run only when your agent explicitly base64-encodes one in testrun.record_results.
Tool-call args & results
Visible inside your Space and only your Space. Per-tenant tmt_* tokens isolate every customer.
If even metadata can’t leave — self-host.

The entire MCP server is open source and lives in your repo. PointTESTMAZE_MCP_URLat your own Test Maze deployment and the verifier loop runs inside your perimeter.

AES-256-GCM at rest· Per-space tmt_* tokens· Multi-tenant isolation
WORKING WITH YOUR AGENT

From feature spec to shipped — the same loop, every time.

Whether you're starting a fresh project in Claude Code or pushing a new feature into an existing repo with Codex, the workflow is the same: hand us the description, get back a tested, graded build.

Starting a fresh project
Adding a feature to an existing app
01

Drop the feature description in

A PRD, a Linear ticket, a back-of-napkin one-pager — whatever you fed your agent. Fresh project? Use the high-level spec. New branch? Hand us just the delta.

02

We map it to a Test Suite

Each feature becomes a structured Test Suite inside your space — the contract your agent is building toward. Existing repo? We graft the suite into your current test plan; we don't duplicate what's already covered.

03

Extensive test cases — every facet covered

For each suite we generate a wide test case set. Not just the happy path. Coverage spans every facet of how the feature can be used, abused, and broken.

Happy-path flows
Edge cases & boundaries
Error & recovery states
Permissions & roles
Accessibility (a11y)
Performance budgets
Browser & viewport matrix
Data-shape mutations
04

Your agent codes; we run the Test Runs

Claude Code or Cursor implements the acceptance criteria. When ready to grade, the agent calls testrun.create to execute the suite headlessly, then pdlc.verify composes the verdict — failure buckets, reliability KPI, deterministic nextStep — and feeds the result back into your IDE.

05

A quality grade you can defend

Every Test Run produces a verdict pinned to a specific gitSha. Pass — merge it. Fail — categorised hints route back to your agent. You ship with a per-feature reliability grade, not just a green CI badge you can't explain in standup.

# greenfield product — initialise once
> project.initialize { repoRoot: "/Users/me/checkout-v2" }

# new feature on an existing product
> feature.implement { title: "Apple Pay button", sessionId: "pdlc_…" }

# after your agent codes — grade the work
> pdlc.verify { testRunId: "tr_…", sessionId: "pdlc_…" }
THE VERDICT PAYLOAD

Every verify call returns the same contract.

No LLM in the pass/fail decision path. Failure buckets are machine-categorised. Every run carries gitSha, branch, and workingTreeClean — so the verdict is bound to exactly the code that ran.

Deterministic verdict path (no LLM in pass/fail)
KPI gates from backend/src/utils/constants.ts
Selector-failure share + reliability scoring
Pinned to git state — no untracked-file surprises
verify.response.json
// pdlc.verify → returns
{
  "verdict": "fail",
  "passed": 14,
  "failed": 2,
  "failureBuckets": {
    "selector-not-found": 1,
    "text-mismatch": 1
  },
  "reliabilityKpi": {
    "passRateMet": false,
    "selectorShareMet": true,
    "passRate": 0.875,
    "selectorFailureShare": 0.5
  },
  "gitSha": "a4f9e2c",
  "branch": "feat/checkout-v2",
  "workingTreeClean": true,
  "testRunId": "tr_a4f9e2c-001",
  "sessionId": "pdlc_q4-checkout-v2",
  "nextStep": {
    "action": "repair_code",
    "caseId": "tc_204",
    "failureBucket": "text-mismatch",
    "hint": null
  }
}
WIRE IT UP

One command. Your agent now has a verifier.

Issue an MCP token from your TestMaze space and point any MCP-compatible client at the server. Stdio for IDE plugins and stateless HTTP for hosted scale — both live today, sharing the same tool surface.

@testmaze/mcp v0.1 — coming soon on npm. Build from source today below.
terminal · zsh
claude mcp add testmaze \
  --env TESTMAZE_MCP_TOKEN=tmt_xxx \
  -- npx -y @testmaze/mcp
@testmaze/mcp is a thin stdio wrapper that exchanges your token and forwards to your Test Maze backend (defaults to https://testmaze.com; override with TESTMAZE_MCP_URL). Published on npm.
COMPATIBILITY

Speaks MCP. So it speaks every coding agent.

One canonical contract — tools/list, resources/read, sampling — that every modern coding agent already speaks. No plugins to maintain. No bespoke integrations.

CClaude Code
◉Cursor
clCline
AAider
⟶Continue
GGoose
{}Custom
"The coding agent never grades its own work."— TestMaze design contract

Wire your coding agent
to a real verifier.

Five minutes to install. Deterministic verdicts from the first run. Your source code never leaves your machine — only the verdicts come back. Free for solo developers; pay only when your team scales.