qa-agent-kit 0.6.0__py3-none-any.whl

qa_agent_kit-0.6.0.dist-info/METADATA

@@ -0,0 +1,157 @@
+Metadata-Version: 2.5
+Name: qa-agent-kit
+Version: 0.6.0
+Summary: The QA agent kit: mutation testing for any repo, prose gates with a claims ledger, and real-device QA for Apple apps, with the procedures an AI coding agent follows.
+Project-URL: Homepage, https://github.com/younatics/QAKit
+Project-URL: Changelog, https://github.com/younatics/QAKit/blob/main/CHANGELOG.md
+Project-URL: Documentation, https://github.com/younatics/QAKit/blob/main/docs/getting-started.md
+Author: younatics
+License-Expression: MIT
+License-File: LICENSE
+Keywords: agents,claude-code,codex,ios,mutation-testing,prose,qa,xcuitest
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: MacOS
+Classifier: Operating System :: POSIX
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Topic :: Software Development :: Quality Assurance
+Classifier: Topic :: Software Development :: Testing
+Requires-Python: >=3.10
+Provides-Extra: test
+Requires-Dist: pyflakes; extra == 'test'
+Requires-Dist: pytest; extra == 'test'
+Description-Content-Type: text/markdown
+
+<p align="center">
+  <img src="https://raw.githubusercontent.com/younatics/QAKit/main/assets/banner.svg" alt="QAKit: the QA agent kit" width="100%">
+</p>
+
+# QAKit
+
+**The QA agent kit.** Procedures an AI coding agent follows to test like a QA engineer, the tools that run the checks,
+and the evidence format a release verdict cites. Three layers, one repo:
+
+| Layer | For | Skill | Tools |
+|---|---|---|---|
+| Code | any repo with tests | `skills/qa-rigor` | `tools/mutate.py` (mutation testing, Python and C-family syntax) |
+| Real device | Apple apps | `skills/device-qa` | `tools/device_qa.py` (test, traits, perf, upgrade, release, one lock per phone), `tools/release_checks.py`, `recording_frames.swift`, `screenshot_contrast.swift`, Swift package `QAKit` for XCUITest |
+| Prose | memos, READMEs, papers | `skills/prose-qa` | `tools/prose_check.py` (gates and a claims ledger) |
+
+QAKit grew out of shipping several App Store apps as a solo developer whose QA team is an agent. Every check in it was
+added after a defect that green tests let through: a `>=` that became `>` at every grading boundary with 13 mutants
+surviving, a review note over the 4,000-character limit, a clipped caption nobody looked at, a What's New that still
+announced the first release in version 1.1.
+
+## See it
+
+<p align="center"><img src="https://raw.githubusercontent.com/younatics/QAKit/main/assets/demo-walk.gif" alt="An agent walking an app on a real iPhone: onboarding, the home screen at every scroll position, each button tapped once" width="300"></p>
+
+`QA.walk` on a real iPhone: the onboarding pages, the home screen at every scroll position, every visible button tapped once
+and screenshotted, the accessibility audit per screen. These 29 frames are one walkthrough test's attachments, assembled by
+`tools/shots_gif.py` from the result bundle (the app is the author's, live on the App Store).
+
+<p align="center"><img src="https://raw.githubusercontent.com/younatics/QAKit/main/assets/demo-variants.gif" alt="The Notes example app's list, settings and search screens in the default, dark and large-text variants, one phone, one command" width="300"></p>
+
+The same walk in three variants: `device_qa.py test --variant default --variant dark --variant large-text` switches the
+phone's appearance and text size between runs. These nine frames are the `examples/Notes` app's list, settings and
+search screens from one such run.
+
+## What makes it different
+
+- **Systematic, not random.** `QA.sweepButtons` taps every visible button on every screen at every scroll position,
+  screenshots what opened, checks the app is still in front and goes back. Monkey testers tap at random; agents reviewing a
+  contact sheet need every control seen once.
+- **The whole phone changes, not the app.** `device_qa.py test --variant dark` switches the device to dark mode, large
+  accessibility text, a simulated location or VoiceOver through Xcode 27's `devicectl`, runs the suite, and restores iOS
+  defaults. System alerts and other apps render the same way the user will see them.
+- **Apple's accessibility audit on every screen,** collected into one report per screen instead of stopping at the first
+  issue, with an allow list for reviewed exceptions.
+- **Test strength is measured.** Coverage says a line ran. Mutation testing says whether a test would notice if the
+  line were wrong. The gate is "every mutant on a changed line is dead or explained".
+- **Prose gets the same treatment.** Deterministic gates, then a claims ledger where every number is traced to a quote,
+  then a cold read by a reader who did not write it.
+- **Written for agents.** The skills are `SKILL.md` files a coding agent loads. The protocol has 180 checks with IDs,
+  pass criteria, methods and sources, so an agent can cite what it did and what it skipped.
+
+## Quick start
+
+```bash
+pip install qa-agent-kit                           # qakit-mutate, qakit-prose, qakit-device, qakit-release-checks, qakit-install-skills
+# or from source
+git clone https://github.com/younatics/QAKit.git
+cd QAKit && python3 -m pytest -q tests          # 150 tests, no dependency beyond pytest
+
+# mutation testing on any project
+python3 tools/mutate.py --root ../myproject --source 'src/**/*.py' --test '{orig}/.venv/bin/python -m pytest -x -q'
+
+# prose
+python3 tools/prose_check.py ../memo.md --claims claims.csv
+
+# an Apple app on a USB-attached test iPhone (macOS, Xcode 27)
+python3 tools/device_qa.py doctor --manifest ../myapp/release/testing/device_qa.json
+python3 tools/device_qa.py test --manifest ../myapp/release/testing/device_qa.json --suite qa --variant dark
+```
+
+Give the procedures to your agent, whichever one you use:
+
+```bash
+claude plugin marketplace add younatics/QAKit && claude plugin install qakit@younatics   # Claude Code: skills + MCP tools
+python3 tools/install_skills.py --target claude --scope project      # or codex, cursor, gemini, all; --scope user
+claude mcp add qakit -- python3 /path/to/qakit/mcp/server.py        # any MCP client: doctor, test, shots, crashes, release checks, mutation, prose, verdict
+```
+
+Then read [docs/getting-started.md](docs/getting-started.md). [AGENTS.md](AGENTS.md) tells any agent which skill to load
+for which job.
+
+## Documentation
+
+| Document | What it covers |
+|---|---|
+| [docs/getting-started.md](docs/getting-started.md) | Installing, linking the Swift package, the first walkthrough test, the first device run |
+| [docs/device-qa-protocol.md](docs/device-qa-protocol.md) | The protocol: 24 areas, 180 checks with IDs, exit criteria, what only a human can do ([한국어판](docs/ko/device-qa-protocol.md)) |
+| [docs/manifest.md](docs/manifest.md) | Every key of the `device_qa.json` manifest |
+| [docs/report.md](docs/report.md) | What a run leaves behind: `summary.json`, contact sheets, crash reports |
+| [docs/mutation-testing.md](docs/mutation-testing.md) | `mutate.py` operators, config, reports, how to read a score |
+| [docs/prose-qa.md](docs/prose-qa.md) | The prose gates and the claims ledger |
+| [docs/case-studies.md](docs/case-studies.md) | Does the mutation layer find real gaps? Measured on humanize, tenacity and swift-argument-parser |
+| [docs/pypi.md](docs/pypi.md) | The pip package: the five commands, where the skills land, the release procedure |
+| [examples/README.md](examples/README.md) | The Notes example app: build it, walk it, see the two planted defects reported |
+| [docs/agents.md](docs/agents.md) | Attaching the kit to Claude Code (plugin), Codex, Cursor, Gemini CLI and any MCP client |
+| [mcp/README.md](mcp/README.md) | The MCP server: nine tools, one stdio process, no dependencies |
+| [skills/README.md](skills/README.md) | Installing the skills by hand |
+| [ROADMAP.md](ROADMAP.md) | What comes next and the loop that keeps the kit honest |
+| [RELEASING.md](RELEASING.md) | How a change becomes a tagged release |
+
+## The Swift package
+
+```swift
+import XCTest
+import QAKit
+
+QA.snap("settings", in: self)                       // screenshot into the result bundle, named by variant
+QA.audit(app, "settings", in: self)                 // Apple's accessibility audit, every issue in one report
+QA.walk(app, "settings", in: self, deny: ["Delete"]) { QA.dismissTopmost(app) }   // snap, audit, sweep, scroll
+QA.answerSystemAlert()                              // permission prompts, in the phone's language
+QA.settle(app, on: screen, in: self) { relaunch() } // get back to a known screen, relaunch as the last resort
+```
+
+Requirements: iOS 17 or later for the accessibility audit, Xcode 15 or later to build, Xcode 27 or later for the
+device variants in `device_qa.py`.
+
+## Who runs it
+
+- The author's app monorepo pins the kit as a submodule and runs every layer on a USB-attached test iPhone.
+- [forge](https://github.com/younatics/forge), the multi-agent coding loop, runs the mutation and prose gates as merge
+  authority on every task and gives its reviewer the qa-rigor principles.
+- Gaps either of them hits come back here as a fix with a test and a tag (`RELEASING.md`).
+
+## Status
+
+Private beta. The three layers are in daily use on the author's apps and documents; the API and the manifest keys may
+still change between minor versions until 1.0. See [CHANGELOG.md](CHANGELOG.md).
+
+## License
+
+MIT. See [LICENSE](LICENSE).

qa_agent_kit-0.6.0.dist-info/RECORD

@@ -0,0 +1,32 @@
+qakit/docs/README.md,sha256=Sgetyf6slmIkMV6_LIlLsxk3vUTDZe_8ggaSUiPQPmE,860
+qakit/docs/agents.md,sha256=UK0uVYGJFS3XBLa90WvTJ4ic6_GWwEjD5E880EX04a0,2109
+qakit/docs/case-studies.md,sha256=TYHNY42Tcf2he2fhyea6nI3N7kOPBEZ8L_5No2jBd9w,17351
+qakit/docs/device-qa-protocol.md,sha256=oRB1aeGEIcbInsV2PwUqww5fN1DBJixrmdGmKWzcTtg,116937
+qakit/docs/getting-started.md,sha256=DheNuCXY5fmX-q4kvgN-1umUAjGAVZQ71svA4HBijRI,4624
+qakit/docs/manifest.md,sha256=H3YZcBsN68wU4sKnIPZp8_aV9zIIaZm_P3G4Fw6D178,7021
+qakit/docs/mutation-testing.md,sha256=Tnjy7oX84NT7LsDHw_1qyPDvMmmiDALlb_sNOmRZt9Q,4170
+qakit/docs/prose-qa.md,sha256=9wMbDy2OsPbiUMIhA1MSAVZthELHNmjThavkmD9CwXI,2543
+qakit/docs/pypi.md,sha256=aHLPRhBP7cE0MWHTevLIXMUMSqALqx46P62flDY8rvc,2052
+qakit/docs/report.md,sha256=lHsMsj2gIMGTgF9h9SjFwXj252VFc9-WtG8y7xolsu8,3109
+qakit/docs/ko/device-qa-protocol.md,sha256=Bl-Px9ROl1eFRErI4LPDfSkghsobhbBulIY5rRKCYB0,119864
+qakit/skills/README.md,sha256=DsYRoF--FPXt9oYi3suHpwtD2phRvcBx7whMblYxxGk,2464
+qakit/skills/device-qa/SKILL.md,sha256=YJdkSeKt_ETZGbi8IyGF6OCkrHco8S_oKFmmW8oj86U,9805
+qakit/skills/prose-qa/SKILL.md,sha256=Fn_GqP-ZMwNMbMyMRXnSmEVXa-d0jB62Vr5Eui-0QQw,3947
+qakit/skills/qa-rigor/SKILL.md,sha256=eKXWnMjgnZ7UVLSSVv3t1jsDnVIExAUzZHHcVOnwP1M,7569
+qakit/tools/device_lock.py,sha256=aYEjfa9h6QN9gYnapQDlcpSRe8wpW8_9ml9n2GlfiqQ,7171
+qakit/tools/device_qa.py,sha256=-7TY2vfvo7aCq42zsN7_oYccZEKh4RRWCe5X4nCIowg,50858
+qakit/tools/install_skills.py,sha256=BPq6e_w7nyKNgaALfP2eHgAskIUipN9vMU-LP3GMVLc,5936
+qakit/tools/mutate.py,sha256=z5ZIXfyUixIX0y0ERZzpL9wFvuHHAkpAcmLG2E2CmeY,35779
+qakit/tools/prose_check.py,sha256=KMbSsZ1quS4b5l1shSX6RunEiviD9gsds2uRw8yQAcI,13428
+qakit/tools/recording_frames.swift,sha256=rYxVX7CIrzb6ISBKXxmS4Biwkg-25LFRAYVjUwqJiTc,4441
+qakit/tools/release_checks.py,sha256=dGUlqH-E77kWMxsXvTPQDrvg-wzoqXLNLkDbs86oCp0,16991
+qakit/tools/screenshot_contrast.swift,sha256=LKD8P8b6DdbilNoZWLuLuY0RpdfHmQI-8goF8D4maco,2488
+qakit/tools/shots_gif.py,sha256=wPAGztB4Bovt8rz8vNZJIpwfwA69bUIKHUmyAK5jYAI,3924
+qakit/__init__.py,sha256=kXicK-_xzqXKlztYMBp1oBUsYr17Clq5kwKb6Yh2jqQ,904
+qakit/_cli.py,sha256=jUkesrwLE_W6_OzlSeHaIWyaqx8qRgmhFhMY2J4SNGU,1270
+qakit/AGENTS.md,sha256=Z3LUc47SL7zWlbzh9Z9mXDIDzn0kEF-x5DoAcEuFrpc,2903
+qa_agent_kit-0.6.0.dist-info/METADATA,sha256=c4BJCc_MyRPVYi3FAfmJKmvY31ATAOKXBo2NSBAyO0c,9466
+qa_agent_kit-0.6.0.dist-info/WHEEL,sha256=2iZrJkDIkeDh0k9SikKMqFuRP-ua6mb4Ir1AsER4BEw,87
+qa_agent_kit-0.6.0.dist-info/entry_points.txt,sha256=eBgRi1nFI43Z8ZnQQDJ_MWNY5sdW45BjOiEwRkldxpY,213
+qa_agent_kit-0.6.0.dist-info/licenses/LICENSE,sha256=E8n9SVFFMrIyyNvihyJ0VRy-a7WWdYBA5z2A_Mt38m4,1081
+qa_agent_kit-0.6.0.dist-info/RECORD,,

qa_agent_kit-0.6.0.dist-info/WHEEL

@@ -0,0 +1,4 @@
+Wheel-Version: 1.0
+Generator: hatchling 1.33.0
+Root-Is-Purelib: true
+Tag: py3-none-any

qa_agent_kit-0.6.0.dist-info/entry_points.txt

@@ -0,0 +1,6 @@
+[console_scripts]
+qakit-device = qakit._cli:device
+qakit-install-skills = qakit._cli:install_skills
+qakit-mutate = qakit._cli:mutate
+qakit-prose = qakit._cli:prose
+qakit-release-checks = qakit._cli:release_checks

qa_agent_kit-0.6.0.dist-info/licenses/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 SeungYoun Yi (younatics)
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

qakit/AGENTS.md

@@ -0,0 +1,48 @@
+# AGENTS.md: how an AI coding agent uses QAKit
+
+This file is read by agents that follow the `AGENTS.md` convention (Codex, Cursor, Aider, OpenHands, Gemini CLI and
+others). Claude Code reads it through `CLAUDE.md`. It says what the kit is and which procedure to load for which job.
+Copy or link it into the project that uses the kit, or point that project's own AGENTS.md here.
+
+## What this is
+
+QAKit is a QA agent kit: procedures (`skills/*/SKILL.md`), deterministic tools (`tools/*.py`, `Sources/QAKit`), and
+the evidence formats a verdict cites (`docs/report.md`). The agent is the QA team; the tools make its findings
+reproducible and its verdicts citable.
+
+## Pick the procedure
+
+| The job | Load | Run |
+|---|---|---|
+| Any code change before merge or release | `skills/qa-rigor/SKILL.md` | `python3 tools/mutate.py --config <mutation.json> --changed-since origin/main` |
+| An Apple app before TestFlight or App Review | `skills/device-qa/SKILL.md` and `docs/device-qa-protocol.md` | `python3 tools/device_qa.py doctor --manifest <device_qa.json>`, then `test`, then `python3 tools/release_checks.py` |
+| A memo, README, design doc, paper or post | `skills/prose-qa/SKILL.md` | `python3 tools/prose_check.py <file.md> --claims claims.csv` |
+
+Read the skill fully before running a tool. Each skill ends with the verdict format; produce it, including what you did
+not run and why.
+
+## Rules the kit assumes
+
+- Expected values are written before the test, the fix or the edit. Three values at every boundary.
+- Green tests are not a verdict. The verdict is the ledger: gates run, results, gates skipped with one line of risk each.
+- A survivor in a mutation report gets a test or a `mutation-ok: <reason>` on the line. Never an exclusion to lift a score.
+- On a phone, evidence goes into the result bundle (`QA.snap`, `QA.note`), never to a Mac path.
+- Submission to review, a release request or a destructive action waits for the owner's explicit word.
+- A gap in the kit (a tool that needs a workaround, a protocol row that is wrong) is fixed in the kit with a test and
+  released by tag (`RELEASING.md`), not patched in the consumer project.
+
+## Install the skills into your agent
+
+`python3 tools/install_skills.py --target claude --scope project` (or `codex`, `cursor`, `gemini`, `all`; `--scope user`
+for your home folder). It links `skills/*` into the folder each tool reads and writes a one-line rule file where the
+tool needs one. `skills/README.md` has the manual steps.
+
+## Layout
+
+```
+skills/        qa-rigor · device-qa · prose-qa
+tools/         mutate.py · device_qa.py · release_checks.py · prose_check.py · install_skills.py
+Sources/QAKit  Swift helpers for XCUITest targets (snap, audit, sweep, alerts, settle)
+docs/          protocol (en, ko), manifest, report, getting started, mutation testing, prose QA
+tests/         pytest; mutation.json measures the kit's own tests
+```

qakit/__init__.py

@@ -0,0 +1,21 @@
+"""QAKit, the QA agent kit (https://github.com/younatics/QAKit).
+
+The tools live under ``qakit/tools`` as the same plain scripts a checkout runs with ``python3 tools/<name>.py``; the
+console scripts (``qakit-mutate``, ``qakit-prose``, ``qakit-device``, ``qakit-release-checks``, ``qakit-install-skills``)
+are the shims in :mod:`qakit._cli`. The skills a coding agent loads are under ``qakit/skills``.
+"""
+from __future__ import annotations
+
+from importlib.metadata import PackageNotFoundError, version
+from pathlib import Path
+
+KIT = Path(__file__).resolve().parent          # qakit/ (tools/, skills/, docs/, AGENTS.md live here after a pip install)
+TOOLS = KIT / "tools"
+SKILLS = KIT / "skills"
+
+try:
+    __version__ = version("qakit")
+except PackageNotFoundError:   # imported from a checkout's src/ without an install
+    __version__ = "0+unknown"
+
+__all__ = ["KIT", "SKILLS", "TOOLS", "__version__"]

qakit/_cli.py

@@ -0,0 +1,46 @@
+"""Console-script shims: one function per tool, each the target of a `[project.scripts]` entry in pyproject.toml.
+
+The tools under qakit/tools import each other as siblings (`device_qa` imports `device_lock`, `release_checks` imports
+`device_qa`), exactly as they do when run as `python3 tools/<name>.py` from a checkout, where the script's own folder is
+the first sys.path entry. A shim reproduces that: it puts qakit/tools first on sys.path, imports the tool as a top-level
+module and exits with its main(). The tools themselves are unchanged.
+"""
+from __future__ import annotations
+
+import importlib
+import sys
+from types import ModuleType
+
+from qakit import TOOLS
+
+
+def load(name: str) -> ModuleType:
+    """Import tools/<name>.py the way a script run would: by bare module name, with the tools folder first on sys.path."""
+    folder = str(TOOLS)
+    if folder not in sys.path:
+        sys.path.insert(0, folder)
+    return importlib.import_module(name)
+
+
+def run(name: str) -> None:
+    sys.exit(load(name).main())
+
+
+def mutate() -> None:
+    run("mutate")
+
+
+def prose() -> None:
+    run("prose_check")
+
+
+def device() -> None:
+    run("device_qa")
+
+
+def release_checks() -> None:
+    run("release_checks")
+
+
+def install_skills() -> None:
+    run("install_skills")

qakit/docs/README.md

@@ -0,0 +1,14 @@
+# Docs
+
+| Read this | When |
+|---|---|
+| [getting-started.md](getting-started.md) | First hour: install, link the package, first walkthrough, first device run |
+| [device-qa-protocol.md](device-qa-protocol.md) | Running a release QA: 24 areas, 180 checks, exit criteria. Korean edition in [ko/](ko/device-qa-protocol.md) |
+| [manifest.md](manifest.md) | Writing or debugging a `device_qa.json` |
+| [report.md](report.md) | Reading what a run left behind |
+| [mutation-testing.md](mutation-testing.md) | Configuring `mutate.py`, reading survivors |
+| [prose-qa.md](prose-qa.md) | The prose gates and the claims ledger |
+| [case-studies.md](case-studies.md) | Evidence that the mutation layer finds real test gaps on public projects |
+
+The skills under [../skills](../skills) are the procedures an agent follows; these documents are the reference they
+point at.

qakit/docs/agents.md

@@ -0,0 +1,40 @@
+# Attach QAKit to your agent
+
+QAKit gives an agent two things: the procedures (`skills/*/SKILL.md`) and the tools that run the checks. The skills say
+what to do; the server for the Model Context Protocol (MCP) lets the agent run the kit without a shell. Use one or both.
+
+## Claude Code
+
+The plugin installs the skills and the server together:
+```bash
+claude plugin marketplace add younatics/QAKit
+claude plugin install qakit@younatics
+```
+
+The skills appear as `/qakit:qa-rigor`, `/qakit:device-qa` and `/qakit:prose-qa`; the server as `plugin:qakit:qakit`.
+Without the plugin, link the skills from a checkout and add the server by hand:
+
+```bash
+python3 tools/install_skills.py --target claude --scope project     # or --scope user
+claude mcp add qakit -- python3 /path/to/qakit/mcp/server.py
+```
+
+## Codex, Cursor, Gemini CLI
+
+Skills first: `python3 tools/install_skills.py --target codex` (or `cursor`, `gemini`, `all`) links `skills/*` into the
+folder each tool reads and leaves a pointer line in `AGENTS.md`, a `.cursor/rules` file or `GEMINI.md`.
+Then the server, plain Python with no dependency. Codex: `codex mcp add qakit -- python3 /path/to/qakit/mcp/server.py`,
+or `[mcp_servers.qakit]` with `command = "python3"` and `args = ["/path/to/qakit/mcp/server.py"]` in `~/.codex/config.toml`.
+Cursor: the same server under `mcpServers` in `.cursor/mcp.json`. Gemini CLI: `.gemini/settings.json`. Snippets: `mcp/README.md`.
+
+## Any MCP client
+
+Transport stdio, command `python3`, one argument: the absolute path of `mcp/server.py`. The server answers `initialize`,
+`tools/list` and `tools/call`. Point the client's instructions at the kit's `AGENTS.md`, which names the procedure for
+each kind of job, and at `docs/report.md` for what the artifacts mean.
+
+## What the agent should do with it
+
+Read the skill fully, run `device_doctor` before `device_test`, keep every path absolute, and end with `verdict` on the
+artifact paths the tools returned. The ledger names what ran, what failed and what was skipped, with one line of risk per
+skipped gate. That ledger, not green output, is the release verdict.

qakit/docs/case-studies.md

@@ -0,0 +1,146 @@
+# Case studies: does the mutation layer find real test gaps?
+
+Date: 2026-10-28. Engine: `tools/mutate.py` at the kit's current checkout. Machine: MacBook Pro M5 Max (18 cores), Python 3.15, Swift 6.4, Xcode 27.1.
+Scratch area `/private/tmp/qakit-cases/` (shallow clones, venvs, configs, reports). Nothing was written inside a studied repo except what `pip install -e` itself drops (`_version.py`).
+
+Method. Each project got one config, one `--list` preview, one sweep, and a read of 5 surviving mutants chosen one per file or operator family across the survivor table (not the first five rows). Each sampled survivor was classed by opening the code and, where the sentence below says "returns", by running the mutant by hand in a copy outside the repo:
+
+- (a) real test gap: the mutant changes behaviour and no test notices.
+- (b) equivalent: behaviour cannot change.
+- (c) engine noise: a mutant the engine should not have generated.
+
+A note on the report files: the engine writes `results.jsonl`, `mutation.md` and `mutation.json` (with a `summary` key). `docs/mutation-testing.md` said `summary.json` at the time; fixed in 0.2.0.
+
+## 1. younatics/ExpandableCell: no runnable test target here (skipped)
+
+Repo `https://github.com/younatics/ExpandableCell`, commit `9f1b9c381952d2b97577f71b5baa85fa7c8b892d` (2026-10-24, "Release 2.1.0").
+
+Finding: the xcodeproj has an `ExpandableCellTests` target with 11 tests (7 on `ExpandableProcessor`, 4 memory-leak tests on the delegate), but `Package.swift` declares no `testTarget`, the library imports UIKit in every file, and this Mac cannot run iOS tests: Xcode 27.1 refuses to load simulators ("CoreSimulator 1171.7.0 is older than build version 1174.9.2"). So `swift test` has nothing to run and `xcodebuild test` has no destination.
+
+Fallbacks: `younatics/YNDropDownMenu` (commit `08361a35`) does declare a SwiftPM test target, but it is iOS-only too; `swift test` on macOS stops at `YNDropDownButton.swift:8: unable to resolve module dependency: 'UIKit'`. `younatics/MotionBook` is a content repo with no Swift package. Ten minutes were not enough to make any of the three build, so this slot is recorded as "no runnable test target on this Mac" and the Swift measurement comes from project 4.
+
+What this means for Kyle's libraries: the mutation layer needs a test command that runs on the machine. For these iOS-only packages that is `xcodebuild test -destination 'platform=iOS Simulator,...'` on a Mac whose CoreSimulator matches Xcode, with `sweep_sample` set, as `docs/mutation-testing.md` already says. Adding a SwiftPM `testTarget` for `ExpandableProcessor` would not help on macOS either, because `ExpandableProcessor.swift` itself imports UIKit. The YNDropDownMenu attempt left an untracked `.build/` folder in its clone; no source file in any studied repo was changed.
+
+## 2. python-humanize/humanize
+
+Repo `https://github.com/python-humanize/humanize`, commit `6b817b963dbb091b9ad20a746cb6a26653a93b95` (2026-10-25). Sources: `src/humanize/*.py`, 1,751 lines.
+
+Setup: `python3 -m venv venv-humanize`, `pip install -e ./humanize pytest freezegun`. `tests/test_benchmarks.py` is ignored because pytest-benchmark 5.3.0 does not work with pytest 9.2 (`'Function' object has no attribute '_nodeid'`). Baseline: 731 passed, 132 skipped, 12.5 s. The 132 skips are every locale test: a fresh clone has no compiled `.mo` files (`scripts/generate-translation-binaries.sh` writes them into the repo, so it was not run).
+
+Config (`configs/humanize.json`, kept outside the repo; `PYTHONPATH=src` makes the mutated copy win over the editable install, checked by printing `humanize.__file__` from a copy):
+
+```json
+{"name": "humanize", "root": "../humanize", "sources": ["src/humanize/*.py"], "exclude": ["src/humanize/_version.py"],
+ "test": "PYTHONPATH=src /private/tmp/qakit-cases/venv-humanize/bin/python -m pytest -x -q -p no:cacheprovider --ignore=tests/test_benchmarks.py",
+ "jobs": 3}
+```
+
+| mutants | killed | timeout | survived | invalid | uncovered | score | wall |
+|---|---|---|---|---|---|---|---|
+| 235 | 202 | 0 | 33 | 0 | 0 (no `coverage` command) | 86% | 140 s, 3 workers, overlapping the tenacity run and the Swift build |
+
+By operator: arithmetic 66, equality 43, boundary 40, negation 36, logical 34, boolean 11, drop-call 5.
+
+Sampled survivors (5):
+
+| Where | Mutant | Class | Judgement |
+|---|---|---|---|
+| `filesize.py:106` | `exp < len(suffix)` to `<=` | (a) | `naturalsize(999.9995e30)` raises `IndexError: tuple index out of range` instead of returning `"1000.0 QB"`; the rounding-carry branch is tested only below the last suffix. |
+| `number.py:515` | `value < floor` to `<=` | (a) | `clamp(0.01, "{:.0%}", floor=0.01)` returns `"<1%"` instead of `"1%"`; `test_clamp` has a below-floor case and no on-floor case. The twin on line 518 (`> ceil`) is the same gap. |
+| `time.py:196` | `delta.seconds < 60` to `<=` | (a) | `naturaldelta(timedelta(seconds=60))` returns `"60 seconds"` instead of `"a minute"`. Lines 177, 184 and 199 are the same family: nothing tests exactly 1000 microseconds or exactly 60 seconds. |
+| `time.py:661` | `1 < fmt_value < 2` to `1 < fmt_value <= 2` | (b) | At `fmt_value == 2` both branches give 2. Cannot change output. |
+| `i18n.py:8` | `TYPE_CHECKING = False` to `True` | (c) | A typing guard. Flipping it only imports `os` and `pathlib` at runtime. The same mutant survives on `lists.py:5`, `number.py:16` and `time.py:17`: 4 of the 33 survivors are this one idiom. |
+
+Count in the sample: (a) 3, (b) 1, (c) 1.
+
+Beyond the sample: 5 survivors (`number.py:138`, three on `number.py:201`, `number.py:202`) are in the ordinal-gender and locale-separator code that only the 132 skipped locale tests reach. They are an artefact of this run, not a finding about the project. By reading, `number.py:86`, `time.py:384` and `time.py:454` are equivalent too (`inf` is never 0; `days` is an int compared with 152.08; `unit == min_unit` is already in `suppress`). The boundary family in `time.py` (8 survivors) is the strongest signal: humanize has many thresholds and its tests sit between them, not on them.
+
+## 3. jd/tenacity
+
+Repo `https://github.com/jd/tenacity`, commit `32ac68f0941319c0d2f4fb927f96e4ee29bb46c6` (2026-10-12). Sources: `tenacity/*.py` and `tenacity/asyncio/*.py`, 2,466 lines. (markupsafe was not needed; tenacity installed cleanly.)
+
+Setup: `python3 -m venv venv-tenacity`, `pip install -e ./tenacity pytest typeguard tornado trio`. Baseline: 185 passed, 15 subtests, 4.9 s.
+
+Config (`configs/tenacity.json`):
+
+```json
+{"name": "tenacity", "root": "../tenacity", "sources": ["tenacity/*.py", "tenacity/asyncio/*.py"], "exclude": ["tenacity/_version.py"],
+ "test": "PYTHONPATH=. /private/tmp/qakit-cases/venv-tenacity/bin/python -m pytest -x -q -p no:cacheprovider",
+ "jobs": 3}
+```
+
+| mutants | killed | timeout | survived | invalid | uncovered | score | wall |
+|---|---|---|---|---|---|---|---|
+| 212 | 152 | 23 | 37 | 0 | 0 (no `coverage` command) | 83% | 391 s, 3 workers, overlapping the humanize run and the Swift build |
+
+By operator: drop-call 55, negation 45, boolean 39, logical 25, arithmetic 22, boundary 13, equality 13. The 23 timeouts are retry loops that never stop once a `set_result` or `prepare_for_next_attempt` call is dropped; the engine counts them as caught, which is right (a test that hangs is a failing test).
+
+Sampled survivors (5):
+
+| Where | Mutant | Class | Judgement |
+|---|---|---|---|
+| `__init__.py:519` | `self.sleep(do)` dropped | (a) | `Retrying(wait=wait_fixed(0.5), stop=stop_after_attempt(3), sleep=record)` makes zero sleep calls instead of two; no test asserts that the configured `sleep` is ever invoked. The async twin `asyncio/__init__.py:206` (`await self.sleep(do)`) and `nap.py:43` (`self.event.wait(...)`) survive for the same reason. A retry library that never waits passes its whole suite. |
+| `__init__.py:492` | `statistics["idle_for"] += sleep` to `-=` | (a) | The same run reports `idle_for == -1.0` instead of `1.0`; the tests match `idle_for` with `mock.ANY` (`test_tenacity.py:1595`). Line 491 (`rs.idle_for`) is the same gap. |
+| `__init__.py:125` | `retry_run_result: bool = False` to `True` | (b) | `iter_state.reset()` runs first in every `_begin_iter`, and each field is written before it is read inside one iteration. The six survivors on lines 125 to 133 are all this. |
+| `_utils.py:75` | `4 <= pos_num <= 20` to `< 20` | (b) | `find_ordinal(20)` still returns `"th"`: 20 falls through to `find_ordinal(0)`, which is `"th"`. |
+| `__init__.py:765` | `reraise: bool = False` to `True` | (c) | This is a `@t.overload` stub whose body is `...`; its defaults are never executed. Line 769 is the same stub, line 101 (`sys.version_info >= (3, 11)`) is inside `if t.TYPE_CHECKING:`, and `_utils.py:29` is a `sys.version_info` guard. Four of the 37 survivors are typing-only code. |
+
+Count in the sample: (a) 2, (b) 2, (c) 1.
+
+Beyond the sample: `retry.py:82` (`retry_always` returning `False`) and `retry.py:229` (`retry_if_not_result` retrying on an exception) survived too, which says the predicate classes are tested in isolation and not through a full `Retrying` loop. `wait.py:91` (`random.random() * (max - min)` to `/`) says nobody asserts that `wait_random` stays inside its range. `__init__.py:793` (`sleep is not None and is_coroutine_callable(sleep)` to `or`) means `@retry(sleep=time.sleep)` on a plain function would silently get `AsyncRetrying`. The `stats.clear()` survivors (401, 417, `asyncio/__init__.py:229`) are equivalent: `begin()` rewrites every key right after.
+
+## 4. apple/swift-argument-parser
+
+Repo `https://github.com/apple/swift-argument-parser`, commit `38ca4eceb349155836701b88da3e0986eb9d1e1c` (2026-10-24). Sources restricted to `Sources/ArgumentParser/Parsing/*.swift`, 3,556 lines.
+
+Setup: `swift build --build-tests -j 4` took 39 s from a clean clone. The first sweep stopped at exit 2: `swift test` fails at baseline in `ArgumentParserGenerateManualTests` and `ArgumentParserGenerateDoccReferenceTests`, which look for an Xcode-layout product (`.build/out/Products/Debug/generate-manual`) that SwiftPM does not produce. The engine's "baseline must pass" rule caught it, and the report named the failing tests. Those two targets are skipped in the config; the remaining 268 Swift Testing tests plus the XCTest targets pass in 3.7 s.
+
+Config (`configs/swift-argument-parser.json`; `sweep_sample` is the seeded 60-mutant sample, seed 1, so `--max-mutants` was not needed):
+
+```json
+{"name": "swift-argument-parser", "root": "../swift-argument-parser", "sources": ["Sources/ArgumentParser/Parsing/*.swift"],
+ "build": "swift build --build-tests -j 4",
+ "test": "swift test --skip-build --skip ArgumentParserGenerateManualTests --skip ArgumentParserGenerateDoccReferenceTests",
+ "jobs": 3, "sweep_sample": 60}
+```
+
+| mutants | killed | timeout | survived | invalid | uncovered | score | wall |
+|---|---|---|---|---|---|---|---|
+| 60 (sampled from the folder) | 43 | 1 | 11 | 5 | 0 (no `coverage` command) | 80% | 467 s, 3 workers, about 15 s per mutant after three parallel 40 s baseline builds |
+
+By operator in the sample: drop-call 17, equality 11, boolean 11, boundary 7, negation 6, arithmetic 4, logical 4.
+
+Sampled survivors (5):
+
+| Where | Mutant | Class | Judgement |
+|---|---|---|---|
+| `ArgumentSet.swift:400` | `try update(origin, parsed.name, value, &result)` dropped | (a) | In the `.allRemainingInput` strategy the `--files=a b c` form stores nothing for `a` and then consumes `b c`; by reading, the attached value is silently lost and no test uses `--option=value` with that strategy. |
+| `ArgumentSet.swift:439` | `foundAttachedValue = true` to `false` | (a) | In `.upToNextOption`, a joined value as the last argument (`-dDebug` with nothing after it) would be rejected with "missing value" because the guard at line 453 only breaks when the flag is set; the joined tests (`JoinedEndToEndTests.swift`) use `.unary` options, not `.upToNextOption` arrays. |
+| `ArgumentSet.swift:342` | `usedOrigins.formUnion(origins)` dropped | (b) | The joined tests (`-dDebug`, `-fdDebug`) pass without it; `remove(at:)` on a sub-index already removes the whole input element, so this union adds nothing the caller does not add. |
+| `SplitArguments.swift:386` | `> currentIndex.inputIndex` to `>=` | (b) | The element at the current index is the option being parsed, never a terminator, so equality can never match. Line 387 (`<` on the target index) is the same. |
+| `ArgumentDefinition.swift:210` | `isNullary` returns `true` to `false` for `.optionalUnary` | (b) | `isNullary` has no caller in `Sources/` or `Tests/`; dead code. |
+
+Count in the sample: (a) 2, (b) 3, (c) 0.
+
+Beyond the sample: `SplitArguments.swift:613` (dropping `!` in `coalescedExtraElements`) would hide a leftover sub-element such as the unknown `-c` in `-abc` from the "unexpected argument" list; by reading it is a real gap, so the honest (a) share for Swift is closer to 3 of 6. `InputOrigin.swift:117` (`l < r` to `<=` inside `static func <`) only matters when two equal elements are compared, which a `Set` never does.
+
+The engine noise in this project went into the 5 invalid mutants rather than the survivors, and each cost one 15 s build:
+
+| Where | Mutant | Why it cannot compile |
+|---|---|---|
+| `ArgumentDefinition.swift:166`, `SplitArguments.swift:206` | `+` to `-` | `String + String`; there is no `-` on strings |
+| `ArgumentSet.swift:130` | `case (false, _)` to `(true, _)` | a boolean inside a `switch` pattern; the switch stops being exhaustive |
+| `SplitArguments.swift:162` | `static func <` to `<=` | an operator declaration, not a comparison |
+| `CommandParser.swift:282` | `currentNode.firstChild(equalTo: defaultSubcommand)` dropped | the line is the right-hand side of `let subcommandNode =` on line 281; the drop leaves `let subcommandNode =` followed by `else {` |
+
+## What this says about the kit
+
+Is the layer effective? Yes, on three well-tested libraries with 80 to 86 percent scores it still handed over a short survivor list (11, 33, 37) and, in each, real bugs-in-waiting within an hour of reading. Across the 15 sampled survivors: 7 real gaps (47%), 6 equivalent (40%), 2 engine noise (13%). The real gaps are not academic: a retry library whose tests pass when it never sleeps, a duration formatter that says "60 seconds" at the minute boundary, a clamp that marks the floor value as below the floor, and an argument parser that drops `--files=a` under one strategy. Three of the seven are boundary operators on `<` or `<=`, which matches the kit's own first finding (the BMI grading survivor in `skills/qa-rigor/SKILL.md`).
+
+Two caveats on the number. First, the (a) share depends on the suite: tenacity's survivors were mostly in bookkeeping (`idle_for`, `statistics`) that its tests deliberately match with `mock.ANY`. Second, a run without `coverage` labels every untested line "survived", so 5 of humanize's 33 survivors were locale code whose tests were skipped in a fresh clone; with a `coverage` command those would have been "uncovered" and not confused with gaps.
+
+The noise class to fix next in `mutate.py` is typing-only Python code. It produced 4 of 33 humanize survivors and 4 of 37 tenacity survivors (11% of all Python survivors) and every one is a guaranteed equivalent: `TYPE_CHECKING = False` (`src/humanize/i18n.py:8`, `lists.py:5`, `number.py:16`, `time.py:17`), bodies of `if t.TYPE_CHECKING:` (`tenacity/__init__.py:101`), `@t.overload` stubs whose body is `...` (`tenacity/__init__.py:765`, `:769`), and `sys.version_info` guards (`tenacity/_utils.py:29`). In `python_mutants`, extend the `skip` set with (1) lines matching `^\s*TYPE_CHECKING\s*=\s*False`, (2) every line inside an `if TYPE_CHECKING:` or `if <alias>.TYPE_CHECKING:` block, (3) every line of a `FunctionDef` whose decorator list contains `overload` and whose body is a lone `...`, and (4) lines containing `sys.version_info`. The ast pass already walks the tree, so (2) and (3) are a `lineno` to `end_lineno` range on the matching nodes.
+
+One concrete engine improvement, from the Swift run: `CommandParser.swift:282` matched `C_CALL_LINE` because the regex looks at one line only, so a call that continues a `let x =` from the previous line is treated as a statement and its removal leaves a dangling `=`. The fix in `c_family_mutants` is to look at the previous non-blank masked line and skip the drop-call when it ends with `=`, `(`, `,`, `:`, `?`, `??`, `->`, `&&`, `||` or an arithmetic operator. The same mask can stop the two other invalid classes: skip the arithmetic swap on `+` when the character just before or after the operator in the unmasked text is `"` (a string concatenation, `ArgumentDefinition.swift:166`), and skip boolean literals on a line that starts with `case ` (`ArgumentSet.swift:130`) and binary operators on a line that contains `static func ` (`SplitArguments.swift:162`). Each of those five mutants cost a full build; on an Xcode project they would cost a minute or two apiece.
+
+Smaller notes: `docs/mutation-testing.md` says the engine writes `summary.json`; the file is `mutation.json` with a `summary` key. Timeouts counted as caught is the right call, but tenacity had 23 of them (11% of mutants) and each ran for the full 15 s limit, so a `timeout_factor` of 2 would have saved about two minutes there.