Nobody on your team can say what you actually test

Ask a team of any size what they cover, and watch what happens. Someone opens the test-management tool and reads a number of manual cases. Someone else opens CI and reads a number of green tests. Nobody adds them up, because they can't: the two halves live in different systems, in different formats, owned by different people, and the overlap between them is unknown.

That's not a reporting inconvenience. It's why the same scenario gets automated and run by hand every regression, and why a feature ships with a gap both sides assumed the other had covered.

The instinct is to consolidate — put everything in one place. That instinct is what leads to either "all the cases go in the tool" (and automation drifts away from them within a month) or "all the cases go in the repo" (and your manual testers, who don't use git, quietly stop participating). I've settled on a third arrangement, and the rule is simple: one owner per half, and no artifact edited on both sides.

Who owns what

The automated half belongs to the repository. A case that a machine will run is a Markdown file next to the test that runs it. It is created, reviewed, and changed in pull requests, and it gets pushed up into the test-management tool — once as a case, then continuously as results, run after run.

The manual half belongs to the tool. A case that a human will execute is written where that human works. Manual testers get a proper editor, attachments, run assignment, and none of my opinions about git. That half gets pulled down into the repository as a read-only copy, purely so the AI design layer knows it exists.

Each artifact has exactly one place where it can be edited, and each flows one way. There is no merge, no conflict resolution, no sync daemon holding two databases in an unhappy marriage — the failure mode that makes people say "we tried syncing our TMS once, never again." They didn't try this; they tried two-way sync of the same artifact, which nobody should do.

And because both halves land in the tool, there's finally a place where the coverage question has one answer instead of two.

The repo half, briefly

One case, one Markdown file. The frontmatter carries what a TMS wants when you create a case through its API — title, section, priority — plus the sources the case was derived from and, once decided, whether it's automatable at all.

The body has a hard boundary in it, a heading called ## Automation:

• Above the line is the case as a human executes it — product vocabulary, no endpoints, no locators, no function names. "Submit the sign-in form", not POST /auth/sign-in. This half ships to the tool verbatim, so if it's written in automator's language, somebody has to rewrite it during export, and a rewritten case is a second version of the truth.
• Below the line is the automator's cheat sheet — layer, endpoint or locator, pseudocode, traps. It never leaves the repository.

The design work itself is done by a chain of narrow AI steps rather than one prompt: one turns sourced facts into atomic cases and refuses to guess whether they're automatable; the next decides the layer and the verdict from what's actually in the repo; the third writes implementation notes, or points at the test that already covers it. Every step must trace to a numbered rule in a fact base; anything unsourced gets labelled a hypothesis rather than dressed up as a requirement. The AI designs cases — it doesn't write the Go until enough tests exist by hand to make the patterns obvious.

What the pulled-down manual half is actually for

This is the part that surprised me, so it's the part worth stealing.

An AI design layer that can't see the manual cases will confidently propose coverage that a human has been executing every release for two years. It has no way to know. Give it the manual half as context and three things change: it stops proposing duplicates, it can be asked the genuinely useful question — what does neither half cover? — and it can flag the opposite waste, the scenario that's automated and still sitting in somebody's manual regression checklist.

There are two honest ways to hand it that context. Export the cases into a read-only directory in the repo on a schedule, which is a single CLI call. Or let the assistant read the tool directly over MCP, which is where the tools have started to diverge sharply — more on that below.

Wiring it up

Up, from the repo. The token lives in the environment for exactly one command; in CI it's a secret:

export QAS_URL=https://your-org.eu1.qasphere.com

# create the case from the half above the line
QAS_TOKEN=… npx -y qas-cli@latest api test-cases create \
  --project-code PROJ --folder-id 11 \
  --title "Second withdrawal within the cooling-off window is rejected" \
  --priority high \
  --steps '[{"description":"Open the wallet page","expected":""},
            {"description":"Request a withdrawal","expected":"The request is rejected and the cooling-off period is shown"}]'

# after every run, the results CI already produces
QAS_TOKEN=… npx -y qas-cli@latest junit-upload --run-name "nightly" reports/junit.xml

Two details cost me time. The steps array wants an expected on every entry, but a case in this format has one expected result for the whole scenario — so every step carries an empty string except the last. And matching an uploaded result to a case requires the case id to appear in the test's name, which means renaming every covering test. That's a naming-convention decision, not a scripting one; make it before you have three hundred tests, not after.

Down, into the repo, is the mirror image — list the cases in the project and write them into a directory nobody hand-edits:

QAS_TOKEN=… npx -y qas-cli@latest api test-cases list --project-code PROJ

Why TestRail looks like an expensive atavism now

TestRail isn't a bad tool. It's a tool shaped for a world where every reader of test data was a person with a login, and that assumption is now the expensive part.

It charges by the seat, and the readers multiplied. List prices as I write this, September 2026: TestRail is €39 per seat per month on Professional, €78 on Enterprise. QA Sphere is $12 per user per month on Standard — where read-only users are unlimited — and free up to three users. The whole point of pushing automated results into a tool is that people who never write a case get to look at the dashboard: developers, the PM, the founder who wants to know if last night was green. Under per-seat pricing every one of those looks costs a licence, so teams ration access, and a coverage picture nobody is allowed to open isn't a coverage picture.

Its integration surface predates agents. Both tools have a CLI — TestRail's trcli is real and works. The difference is what happens when an assistant needs to read your test data. QA Sphere serves an MCP server itself: you go to settings, pick the assistant, pick an access level, add a key, and the agent gets exactly the tools your role allows, hosted and always matching the API. For TestRail, the MCP servers on npm today are community projects — several of them, by unrelated authors, at different API versions. You can absolutely make one work. You're also making a third-party package the thing standing between an LLM and your test management, and you own that decision at review time.

And the rate limits give away the design era. 180 API requests per minute on Professional, 300 on Enterprise. That's generous for a human clicking around, and thin for a pipeline pushing results case by case, on every run, across suites.

None of that makes the concept of a test-management tool obsolete — the manual half genuinely needs one, which is the whole premise of this architecture. What's obsolete is paying per human head, in an era when the heaviest reader of your test data is a machine, through an integration written by a stranger.

The exit argument, which is the one that actually saves money

When cases live only inside a tool, you can't leave it. Export gives you a CSV whose fields map to nothing, rebuilding is a quarter of somebody's year, and the honest answer to any renewal quote becomes yes.

Split ownership fixes half of that permanently. The automated half — the half that grows, the half tied to your code — has its original in your repository. Moving it to another tool is a new field mapping, a dozen lines of script. The manual half still lives in the vendor's database, and that's a real dependency, but it's the smaller, slower-growing half, and you now have a copy of it in the repo as a side effect of feeding your AI layer.

What it costs you

Someone owns the pull job, and it has to run on a schedule or the AI layer reasons about stale coverage. Linking results to cases means case ids in test names — a rename you should do early. And nothing here improves the manual cases themselves: if they were vague before, they're vague in the repo too, only now the machine reads them.

What you get for that is one number instead of two, and a design layer that knows what a human already checks. Which is the actual question behind "what do we test?" — and the one nobody could answer at the start.