qa-agent-kit 0.6.0__py3-none-any.whl

qakit/docs/manifest.md

@@ -0,0 +1,96 @@
+# The `device_qa.json` manifest
+
+One file per app tells `tools/device_qa.py` and `tools/release_checks.py` what to build, which tests form which suite, how
+to change the phone for each variant and where the store material lives. Paths are relative to the manifest's folder.
+The conventional location is `<app>/release/testing/device_qa.json`, which is what `--app <slug>` resolves under
+`--apps-root` (default `$QAKIT_APPS_ROOT`, else `<git top>/apps`). `--manifest <path>` works anywhere.
+
+`examples/device_qa.json` is a complete example. `validate_manifest` in `tools/device_qa.py` rejects a manifest with a
+message per mistake; `tests/test_device_qa.py` pins those messages.
+
+## Top level
+
+| Key | Required | Meaning |
+|---|---|---|
+| `schema_version` | yes | `"1"` |
+| `app` | yes | Display name; names the artifact folder and the contact sheet |
+| `project` or `workspace` | one of them | `.xcodeproj` or `.xcworkspace` passed to xcodebuild |
+| `scheme` | yes | Scheme that builds the app and its test targets |
+| `process_name` | yes | The app's process name on the phone, used to pick crash and jetsam reports |
+| `extension_processes` | no | Extension process names (widget, notification service) whose crashes also count |
+| `suites` | yes | Name to list of `-only-testing` identifiers (`Target`, `Target/Class` or `Target/Class/test`) |
+| `variants` | no | Name to variant (below). `default` is applied when no `--variant` is given |
+| `bundle_id` | for perf, upgrade, release, traits | The app's bundle identifier (else read from `appstore_config`) |
+| `team_id` | for release | The Apple team id (else read from `appstore_config`) |
+| `signing` | no | `key_path`, `key_id`, `issuer_id` of an App Store Connect API key (below) |
+| `appstore_config` | no | Path to a JSON whose `api` block holds `private_key_path`, `key_id`, `issuer_id` (an alternative to `signing`) |
+| `derived_data` | no | DerivedData folder (default `<project folder>/build/DerivedData-device`); the run's build output is excluded from the source fingerprint either way |
+| `fingerprint_root` | no | Folder whose edits during a run mark the round as mixed (default the project folder); set it when the app's core sources live beside the project folder |
+| `xcodebuild_args` | no | Extra xcodebuild arguments, for example `["CODE_SIGN_STYLE=Automatic"]` |
+| `traits` | for `traits` | `screens`: screen name to the launch arguments that open it (`{"map": ["-start", "map"]}`); default one launch with no arguments |
+| `upgrade` | for `upgrade` | `seed_file` (a black-box UI test copied into the previous release's UI-test folder), `uitests_dir`, `seed` and `check` test identifiers, optional `copy` (gitignored files the old build needs) and `generator` (a project generator script to run in the old worktree) |
+| `release` | for release checks | Store material (below); `version` overrides the store config's `app_store_version` for the What's New check |
+
+## Suites
+
+A suite is what one `test --suite <name>` run executes. Common names:
+
+- `smoke`: five or fewer tests that must be green within five minutes before anything else runs.
+- `unit`, `ui`: the whole test targets.
+- `qa`: the walkthrough class (`<App>QAWalkthroughTests`) that screenshots, audits and sweeps every screen.
+- `qa-screens`: the same walkthrough split per screen, so a failing screen can be rerun alone.
+
+## Variants
+
+A variant changes the whole phone, so system alerts and other apps render the same way. The runner applies it before the
+suite and restores iOS defaults afterwards, even when the suite fails.
+
+| Key | Values | devicectl call |
+|---|---|---|
+| `appearance` | `light`, `dark` | `device settings appearance --mode` |
+| `text_size` | `extra-small` ... `extra-extra-extra-large`, `accessibility-medium` ... `accessibility-extra-extra-extra-large` | `device settings appearance --text-size` (accessibility sizes turn Larger Accessibility Sizes on) |
+| `location` | `[latitude, longitude]` | `device simulate location coordinate` |
+| `voiceover` | `true` | `device settings voiceover --enable`. The first time VoiceOver goes on, iOS 27.2 shows a welcome prompt over the app (Start VoiceOver Tutorial / Use VoiceOver / Cancel); call `QA.settleAfterLaunch(app)` right after `app.launch()` so the walkthrough's first tap is not swallowed |
+| `env` | object | Forwarded to the UI-test runner as `TEST_RUNNER_<KEY>`; QAKit reads `QA_SWEEP`, `QA_LANG`, `QA_ALLOW_LABELS`, `QA_DISMISS_LABELS`, `QA_ALERT_HOSTS`, `QA_LOCALES` |
+
+The variant name itself reaches the tests as `QA_VARIANT` and prefixes every screenshot name.
+
+Typical set: `dark` and `large-text` and `small-text` and `english` with `QA_SWEEP=0` (layout variants: every screen
+seen, no second sweep), one or two `location` variants for a location-aware app, and `voiceover` for a major release.
+
+## Signing
+
+Running on a real device needs development signing. With an App Store Connect API key xcodebuild registers the phone and
+mints profiles without an Xcode login (`-allowProvisioningUpdates -allowProvisioningDeviceRegistration` plus the
+`-authenticationKey*` flags). `key_path` may be absolute, relative to the git top level that holds the manifest, or
+relative to `$QAKIT_SECRETS_ROOT` (so a worktree can reach the key kept in the main checkout). Without a key the runner
+still passes the provisioning flags and relies on the Mac's Xcode account.
+
+## Projects that pin App Store profiles
+
+A project with `CODE_SIGN_STYLE = Manual` and `PROVISIONING_PROFILE_SPECIFIER` set (the usual shape after a release
+script took over signing) cannot be signed for the test phone with the development certificate the key mints. `doctor`
+reports it. Put the overrides in the manifest and every device command applies them:
+
+```json
+"xcodebuild_args": ["CODE_SIGN_STYLE=Automatic", "PROVISIONING_PROFILE_SPECIFIER=", "CODE_SIGN_IDENTITY=Apple Development", "DEVELOPMENT_TEAM=ABCDE12345"]
+```
+
+The archive you upload keeps the project's own settings; the overrides only reach the `test`, `traits`, `upgrade` and `perf`
+builds.
+
+## Release block
+
+Read by `tools/release_checks.py`.
+
+| Key | Meaning |
+|---|---|
+| `info_plist` | The app's Info.plist (purpose strings, `CFBundleLocalizations`) |
+| `resources_dir` | Folder that holds `<lang>.lproj/InfoPlist.strings` or `InfoPlist.xcstrings` |
+| `source_dirs` | Swift source folders scanned for protected APIs (camera, location, ...) |
+| `bundles` | One entry per code-signed bundle: `name`, `sources`, `privacy` (its `PrivacyInfo.xcprivacy`, or `null`) |
+| `metadata_dir` | Folder of `<locale>.md` store metadata files with `## Name`, `## Subtitle`, `## Description`, `## Keywords`, `## What's New`, `## Promotional Text` sections |
+| `review_notes` | The review notes markdown (4,000 character limit) |
+| `app_id` | `TEAMID.bundle.id` as it must appear in the AASA file |
+| `aasa_domain`, `aasa_paths` | Universal link domain and the paths the AASA must list (checked on Apple's CDN) |
+| `link_ignore` | URL prefixes not fetched (API bases that reject GET) |

qakit/docs/mutation-testing.md

@@ -0,0 +1,69 @@
+# Mutation testing with `tools/mutate.py`
+
+`mutate.py` is a single-file mutation testing engine for any repo with a test command. It copies the project root to a
+temporary folder, applies one mutant at a time, runs your build and test commands there and reports which mutants the
+tests killed, which survived, which did not compile and which sit on lines no test runs.
+
+## Languages
+
+- **Python**: tokenize and ast based. Comparison and arithmetic operators, boolean literals, `not` removal, call
+  statements (`store.save(x)` dropped when the call is a whole statement).
+- **C-family syntax** (`.swift .js .jsx .ts .tsx .mjs .kt .go .java .cs`): regex on binary operators that have spaces on
+  both sides, which keeps `Array<Int>`, `->`, `0..<n`, prefix `!` and `-x` intact. Strings, comments and `#if` lines are
+  skipped.
+
+## Config keys
+
+| Key | Meaning |
+|---|---|
+| `name` | Report name |
+| `root` | Folder copied for each mutant (relative to the git top level that holds the config) |
+| `sources` | Globs of files to mutate, relative to root |
+| `exclude` | Globs left alone (views, generated code) |
+| `copy_exclude` | Extra folders not copied (build output); `.git`, `.build`, `DerivedData`, `.venv`, `xcuserdata` and the like are always skipped |
+| `setup` | Shell command run once per copy before `build` |
+| `build` | Compile step; failure means the mutant is invalid (not counted) |
+| `test` | Test step; failure or timeout means killed. `{orig}` real root, `{file}`/`{stem}` the mutated file, `{lcov}` coverage path |
+| `coverage` | Command that writes lcov to `{lcov}`; mutants on unexecuted lines are reported uncovered and not run |
+| `env` | Extra environment for the commands |
+| `jobs` | Parallel copies (default 3); the engine runs under `nice -n 10` |
+| `timeout_factor` | Per-mutant timeout as a multiple of the baseline test time (default 3) |
+| `test_timeout` | Fixed per-mutant timeout in seconds when the test command differs from the baseline one |
+| `operators` | Subset of `boundary equality logical arithmetic boolean negation drop-call` |
+| `sweep_sample` | Seeded sample size for a full sweep (Xcode projects: one to two minutes per mutant) |
+
+## Command line
+
+```
+--config <json>            config file (or --root/--source/--build/--test inline)
+--changed-since <ref>      only lines added or changed since a git ref; 'latest-tag:<glob>' picks the newest matching tag
+--file <path[:line]>       limit to files or one line (repeatable)
+--operator <name>          limit operators (repeatable)
+--max-mutants N --seed S   seeded sample
+--jobs N                   parallel copies
+--max-survivors N          exit 0 up to N survivors (for a gate that is tightening over time)
+--out <dir>                report folder (default $TMPDIR/mutation-reports/<name>-<time>)
+--list                     print the mutants and stop
+--keep                     keep the scratch copies
+--resume                   reuse verdicts already in --out/results.jsonl
+```
+
+Exit codes: 0 no survivors, 1 survivors (or uncovered), 2 setup or baseline failure. A failing baseline names the failing
+test.
+
+## Reports
+
+- `results.jsonl`: one verdict per mutant (file, line, operator, before, after, status, seconds, output tail).
+- `mutation.md`: the human and agent readable report, survivors first, with the line to open.
+- `mutation.json`: counts per status and per operator, the score (`killed / (killed + survived + uncovered)`).
+
+Without `coverage`, a line no test reaches is reported as survived, not uncovered; a project whose tests skip in a fresh
+clone (locale data, network fixtures) looks weaker than it is. Give the engine a coverage command when the test suite can
+produce lcov.
+
+## Reading the score
+
+The score is not a target to maximise by excluding files. It is a map: each survivor is one behaviour nobody checks. A
+boundary survivor on a pricing or grading line is a bug waiting; a survivor in a log message is noise and takes a
+`mutation-ok` comment. The gate that works in practice is "every mutant on a changed line is dead or explained", run
+before merge (`--changed-since origin/main`) and before release (`--changed-since 'latest-tag:myapp/*'`).

qakit/docs/prose-qa.md

@@ -0,0 +1,49 @@
+# Prose QA
+
+`skills/prose-qa/SKILL.md` is the procedure. `tools/prose_check.py` is the mechanical half: seven deterministic gates
+over a Markdown file, and a claims ledger skeleton for the verification step.
+
+## Gates
+
+| Gate | Finds | Severity |
+|---|---|---|
+| `outline` | heading level jumps (H2 to H4), headings with no body, a document with exactly one H2 | WARN |
+| `sentences` | sentences over `--max-words` (35; DEFECT past 1.5 times that), paragraphs over `--max-sentences` (7) | WARN / DEFECT |
+| `duplicates` | the same sentence twice (six words or more, punctuation ignored) | DEFECT |
+| `claims` | a number, percentage, year, currency or evidence word with no source marker in that sentence or its neighbours | WARN |
+| `acronyms` | an acronym of three to six capitals used before `Full Name (ACR)` appears; common ones (API, JSON, PDF ...) skipped | WARN |
+| `links` | empty link targets, bare URLs in prose, the same link text pointing at two targets | DEFECT / WARN |
+| `tells` | em dashes, "In conclusion", "Firstly", "It is worth noting", "delve", three sentences in a row with the same first word | WARN |
+
+Source markers the `claims` gate accepts: `[1]`, `[^note]`, a Markdown link or URL, `(2024)`, `et al.`, `source:`,
+`according to`, `p. 12`, `Table 3`, `Figure 2`.
+
+## The claims ledger
+
+```
+line,claim,source_marker_nearby,source,quote,verdict
+12,"Revenue grew 42% in 2025.",no,,,
+```
+
+The reviewer fills `source` (URL, file, page), `quote` (the exact words that support the claim) and `verdict`:
+
+| Verdict | Meaning |
+|---|---|
+| SUPPORTED | the quote says what the sentence says, numbers digit for digit |
+| PARTIAL | the quote says less; note what the sentence adds beyond it |
+| NOT IN SOURCE | the source exists but does not say this |
+| NO SOURCE | nothing found; the claim gets a source or leaves the document |
+
+A document passes the claims step when every row is SUPPORTED or PARTIAL with the addition justified in the text.
+
+## Running it
+
+```bash
+python3 tools/prose_check.py memo.md                      # findings, defects first; exit 1 on any DEFECT
+python3 tools/prose_check.py memo.md --gate claims --gate links
+python3 tools/prose_check.py paper.md --claims claims.csv --max-words 40
+python3 tools/prose_check.py memo.md --json | jq '.[] | select(.severity=="DEFECT")'
+```
+
+Code blocks, front matter, tables and HTML comments are skipped. The sentence splitter understands `.`, `!`, `?` and the
+CJK full stop, so Korean and Japanese documents get the structural gates; the lexical tells are English.

qakit/docs/pypi.md

@@ -0,0 +1,40 @@
+# QAKit on PyPI
+
+```bash
+pip install qa-agent-kit   # the Python tools (the import name and the commands stay `qakit`); the Swift package stays on SwiftPM (Package.swift)
+```
+
+## The five commands
+
+| Command | Same as, from a checkout |
+|---|---|
+| `qakit-mutate` | `python3 tools/mutate.py` |
+| `qakit-prose` | `python3 tools/prose_check.py` |
+| `qakit-device` | `python3 tools/device_qa.py` |
+| `qakit-release-checks` | `python3 tools/release_checks.py` |
+| `qakit-install-skills` | `python3 tools/install_skills.py` |
+
+Flags are identical. Each command is a shim in `src/qakit/_cli.py`. The wheel copies `tools/` to `qakit/tools/`; the
+shim puts that folder first on `sys.path` and calls the tool's `main()`, so the sibling imports (`device_qa` needs
+`device_lock`) resolve as they do for a plain script. The scripts are unchanged and still run from a checkout.
+
+## Where the skills are after a pip install
+
+The wheel also copies `skills/`, `docs/` and `AGENTS.md` under the package, so `qakit-install-skills` links
+`.claude/skills/<skill>` (or the Codex, Cursor or Gemini equivalent) to `<site-packages>/qakit/skills/<skill>`. Find the
+folder with `python3 -c "import qakit; print(qakit.SKILLS)"`. Run `qakit-install-skills --target all --uninstall` before `pip uninstall`.
+
+## Releasing
+
+The version is read at build time from the newest `## [x.y.z]` heading in `CHANGELOG.md`, so a release starts with the
+changelog step in `RELEASING.md`, then:
+
+```bash
+python3 -m venv /tmp/qakit-release && /tmp/qakit-release/bin/pip install build twine
+rm -rf dist && /tmp/qakit-release/bin/python -m build          # dist/qakit-<version>.tar.gz and the wheel
+/tmp/qakit-release/bin/twine check dist/*
+/tmp/qakit-release/bin/twine upload dist/*                     # token in ~/.pypirc, or TWINE_USERNAME=__token__ TWINE_PASSWORD=pypi-...
+```
+
+Before `twine upload`, install the wheel into a clean venv and run `qakit-mutate --help` outside the checkout.
+The sdist excludes `Sources/`, `Package.swift`, `.github/` and `assets/`; the wheel excludes the tests.

qakit/docs/report.md

@@ -0,0 +1,64 @@
+# What a run leaves behind
+
+`tools/device_qa.py test` writes everything under one folder: `--out`, else `$QAKIT_ARTIFACTS/<app>/<stamp>`, else
+`/tmp/qakit/<app>/<stamp>`. The folder is the evidence an agent reviews and the evidence a verdict cites.
+
+```
+<out>/
+  summary.json                 the run's verdict and pointers (schema below)
+  <variant>-<suite>.log        full xcodebuild output
+  <variant>-<suite>.xcresult   the result bundle (open in Xcode, or export again with `shots`)
+  <variant>-<suite>-shots/
+    index.md                   contact sheet: one section per test, every attachment in order, failures marked
+    manifest.json              xcresulttool's export manifest
+    *.png, *.txt               screenshots (`QA.snap`) and notes (`QA.note`, accessibility audit reports)
+  crashes/                     .ips crash and jetsam reports from the phone that name the app or its extensions
+```
+
+## summary.json (`qakit.device-qa.summary.v1`)
+
+```json
+{
+  "schema": "qakit.device-qa.summary.v1",
+  "app": "MyApp",
+  "device": {"udid": "...", "name": "iPhone", "transport": "wired"},
+  "started": "2026-10-28T09:00:00+00:00",
+  "runs": [
+    {"variant": "dark", "suite": "qa", "executed": 9, "skipped": 0, "failures": 1,
+     "failed_cases": ["MyAppQAWalkthroughTests testWalkSettings"], "succeeded": false,
+     "screenshots": 143, "log": ".../dark-qa.log", "xcresult": ".../dark-qa.xcresult"}
+  ],
+  "crashes": [".../crashes/MyApp-2026-10-28-090501.ips"],
+  "sources_changed_during_run": false,
+  "succeeded": false
+}
+```
+
+`succeeded` is true only when every run succeeded, no crash report named the app, and no source under the app changed while
+the run was going (a round that compiled a mix of two trees describes neither). The exit code follows it.
+
+## Other commands
+
+| Command | Leaves |
+|---|---|
+| `perf` | `perf-<stamp>/launch.trace` (open in Instruments) and `perf.json` with the potential-hangs and hang-risks rows |
+| `traits` | `traits-<stamp>/<screen>.png` after the changes and `crashes/` |
+| `release` | `release-<stamp>/export/*.ipa`, `launch-<n>.png` per cold launch and `crashes/` |
+| `upgrade` | `upgrade-<stamp>/old-seed.xcresult`, `new-check.xcresult`, their logs and shot folders |
+
+## Screenshot names
+
+`QA.snap("<step>")` stores `"<variant> <step>"`. The walkthrough convention is `<screen> › <control>` for a sweep tap and
+`a11y <screen>` for an accessibility report, so a reviewer can group the contact sheet by screen with a plain text search.
+
+## Reviewing a contact sheet
+
+Read `index.md` top to bottom, one screen at a time: clipping, overlap, empty states, wrong facts, translation, contrast,
+touch targets, attribution. For every suspected defect try to refute it (is it the variant? a test fixture? a real
+bug?) before putting it on the defect list. Accessibility notes (`a11y ...` text attachments) go on the same list with
+the issue type Apple reported.
+
+## Static checks
+
+`tools/release_checks.py` prints one line per check (`PASS`, `WARN`, `FAIL`, the check ID from the protocol and the
+evidence) and `--json` gives the same as a list. Any FAIL exits 1.

qakit/skills/README.md

@@ -0,0 +1,51 @@
+# Agent skills
+
+Three skills, written for coding agents that read `SKILL.md` files (Claude Code, Codex, and any tool that follows the
+same convention). They are the operating procedure; the tools under `tools/` and the Swift package do the work.
+
+| Skill | Scope |
+|---|---|
+| `qa-rigor` | Any code: test design, mutation testing, runtime checkers, real-environment check, verdict ledger |
+| `device-qa` | Apple apps on a USB-attached test iPhone: walkthrough, variants, screenshots, static submission checks, release verdict |
+| `prose-qa` | Memos, READMEs, design docs, papers: deterministic gates, claims ledger, argument check, cold read |
+
+## Install with the installer
+
+```bash
+python3 tools/install_skills.py --target claude --scope project        # this project's .claude/skills
+python3 tools/install_skills.py --target all --scope user              # ~/.claude/skills, ~/.agents/skills, ~/.cursor/rules, ~/.gemini
+python3 tools/install_skills.py --target codex --scope project --project /path/to/repo
+python3 tools/install_skills.py --target all --uninstall
+```
+
+| Target | What it gets |
+|---|---|
+| `claude` | `.claude/skills/<skill>` symlink to `skills/<skill>`; Claude Code lists the skill and loads `SKILL.md` on use |
+| `codex` | `.agents/skills/<skill>` symlink, and a pointer line in the project's `AGENTS.md` |
+| `cursor` | `.cursor/rules/qakit-<skill>.mdc`, a rule that tells Cursor to read the `SKILL.md` |
+| `gemini` | A pointer line in `GEMINI.md` (project) or `~/.gemini/GEMINI.md` (user) |
+
+Existing files that are not QAKit links or rules are left alone and reported as `skip`. The installer is idempotent.
+
+## Install by hand
+
+Project-wide (checked into your repo):
+
+```bash
+git submodule add https://github.com/younatics/QAKit.git vendor/qakit
+mkdir -p .claude/skills
+ln -s ../../vendor/qakit/skills/qa-rigor .claude/skills/qa-rigor
+ln -s ../../vendor/qakit/skills/device-qa .claude/skills/device-qa
+ln -s ../../vendor/qakit/skills/prose-qa .claude/skills/prose-qa
+```
+
+User-wide:
+
+```bash
+git clone https://github.com/younatics/QAKit.git ~/qakit
+for s in qa-rigor device-qa prose-qa; do ln -s ~/qakit/skills/$s ~/.claude/skills/$s; done
+```
+
+Any other tool: paste or link the kit's `AGENTS.md`, which names the skill to load for each kind of job. Relative paths
+inside the skills (`tools/...`) are relative to the QAKit checkout; say where the checkout is in your project's
+instructions file so the agent can find `tools/device_qa.py`.

qakit/skills/device-qa/SKILL.md

@@ -0,0 +1,87 @@
+---
+name: device-qa
+description: QA an Apple app like a QA engineer on a USB-attached test iPhone. Walk every screen and tap every button, review screenshots, audit accessibility, run dark, large-text, language and location variants, run the static submission checks, fix and loop, and write the release verdict. Use on "QA this app", "tap through every screen", "be thorough before release", "test on a real device", and as the gate before App Review. Argument: the app slug or manifest path.
+---
+
+# device-qa: real-device QA for a solo developer whose QA team is an agent
+
+The procedure and the check list live in `docs/device-qa-protocol.md` (24 areas, each check with an ID, a pass
+criterion, a method and a source). This skill is the order in which to run the tools. "Tests green" is not "ready to
+ship". The release verdict comes from the protocol's exit criteria.
+
+## Tools
+
+| Tool | Does |
+|---|---|
+| `python3 tools/device_qa.py doctor --app <slug>` | Device, Developer Mode, passcode, developer disk image, signing identity, App Store Connect key. Prints a fix per problem |
+| `python3 tools/device_qa.py devmode reveal` | Makes Settings > Privacy & Security > Developer Mode appear (`enable` turns it on when the phone has no passcode) |
+| `python3 tools/device_qa.py test --app <slug> --suite qa --variant dark ...` | Puts the whole phone into a variant (dark, text size, location, VoiceOver, language), runs the suite, restores iOS defaults. Leaves the result bundle, a screenshot `index.md`, crash and jetsam reports and `summary.json`. `--only <test>` runs single tests while iterating on one defect; `--configuration Release` runs the UI suites on the optimized build. A source edit that lands mid-run marks the round as mixed and fails it |
+| `python3 tools/device_qa.py traits --app <slug> [--screen X]` | Changes text size and dark mode on each manifest screen in front and in the background and checks the app survived in the same process (no 0x8BADF00D) |
+| `python3 tools/device_qa.py perf --app <slug> --seconds 40` | Launches the app, attaches Time Profiler and counts Instruments' potential hangs and hang risks |
+| `python3 tools/device_qa.py upgrade --app <slug> --from <previous tag>` | Builds the previous release in a temporary worktree, seeds real data with the manifest's seed test, installs this build over it and runs the check test |
+| `python3 tools/device_qa.py release --app <slug> --archive <xcarchive>` | Exports the Release archive for the phone, installs it and cold-launches it with no debugger, then pulls crash reports |
+| `python3 tools/device_qa.py lock status` and `lock run --note "<who>" -- <command>` | One phone, many sessions: who holds it and who waits. Raw xcodebuild, devicectl or xctrace work goes through `lock run` so it takes its turn |
+| `swift tools/recording_frames.swift <video.mp4> <out-dir>` | Frames and a contact sheet from a UI-test screen recording (the `.mp4` in an xcresult or a `*-shots` folder). Look at them before calling a device UI failure an app defect. Given `<a.png> <b.png> ... <out-dir>` it tiles screenshots side by side |
+| `swift tools/screenshot_contrast.swift <shot.png> <x0> <y0> <x1> <y1>` | The real WCAG contrast of text in a screenshot box (fractions). The audit reports "Contrast failed" on glass, materials and shadows; a reviewed exception needs the number the user sees |
+| `Sources/QAKit/QAKit.swift` | UI-test helpers. Evidence: `QA.snap`, `QA.note`, `QA.assertInFront`. Accessibility: `QA.audit` (one report per screen, explained classes counted, one retry on an audit timeout). Alerts: `QA.answerSystemAlert`, `QA.dismissTopmost` (alert, share sheet, in-app Safari, action sheet, the app's closers, Back, system alert, swipe down). Walking: `QA.walk` (snap, audit, sweep, scroll loop), `QA.sweepButtons` (every visible button, from one snapshot of the tree), `QA.back`, `QA.closeInAppSafari`, `QA.exitSearch`, `QA.settle`, `QA.onScreen`, `QA.reveal` (scroll an off-screen row into view, `clearOf:` floating bars that swallow taps). Stability on a device: `QA.waitForStill`, `QA.tapUntilGone`, `QA.replaceText`, `QA.switches(labelled:)`, `QA.flip` (a Toggle's inner switch), `QA.waitForValue`, `QA.searchField` (iOS 26 bottom bar or minimised Search button), `QA.tree` (the element tree as text when something is missing), `QA.count` (rows of a lazy list that keeps only the visible ones in the tree). The app passes markers and identifiers only |
+| `python3 tools/mutate.py --config <app>/release/testing/mutation.json --changed-since 'latest-tag:<app>/*'` | The mutation gate (SUITE-08): breaks the logic lines changed since the last release one at a time and checks the unit tests notice. Every survivor gets a test |
+| `python3 tools/release_checks.py --app <slug> [--ipa X]` | Static checks right before submission: What's New, field limits, placeholders and markup, review notes, age-rating block, localized purpose strings, required-reason APIs, links, AASA, IPA entitlements |
+
+Attaching a new app: write `<app>/release/testing/device_qa.json` (`docs/manifest.md`, start from `examples/device_qa.json`)
+and link the `QAKit` package into the UI-test target (`docs/getting-started.md`).
+
+## Order
+
+1. **Readiness.** Run `doctor` until it prints `ready`. The test phone has no passcode, Developer Mode on, UI automation
+   allowed, automatic updates off.
+2. **Risk map.** From `git diff --stat` since the last release tag and the changelog, list the changed areas. Build a UI
+   matrix of every screen, control and state (one reader per area plus a completeness critic when you have parallel
+   agents). Deduplicate suspected defects, try to refute each one, keep only CONFIRMED. Product decisions go on the
+   owner's decision list.
+3. **Walkthrough test.** `<App>QAWalkthroughTests`: for every screen `QA.walk` (snap, audit, sweep, scroll until the bottom) with
+   a `settle` closure that brings the walk back to its screen (`QA.dismissTopmost` then `QA.back`; `QA.settle` relaunches only as
+   the last resort). Controls that leave the app or have real effects (call, purchase, delete) are excluded from the
+   sweep and get their own test that ends safely (a call is cancelled at the confirmation sheet). Always keep one
+   **first real use** test (no test mode: real location, network, onboarding).
+4. **Run.** Unit, UI and qa suites under the default variant, then qa under `dark`, `large-text`, `english` and location
+   variants; `traits` for the trait-change survival; the mutation gate on the changed logic lines. Read the failure summary and
+   `crashes`.
+5. **Screenshot review.** Split the screens of `index.md` and review them in parallel (clipping, overlap, empty states,
+   wrong facts, translation, contrast, touch targets, attribution). Each defect gets a refutation pass. Accessibility audit
+   notes (`a11y ...` attachments) go on the same list.
+6. **Fix.** Group defects by file set, fix each in its own worktree (failing test first), integrate, rerun 3 to 5. Stop
+   when **two consecutive rounds find zero new defects**.
+7. **Static checks.** `release_checks.py` (not offline) with 0 FAIL. After archiving, run it again with `--ipa` for the
+   entitlements.
+8. **Verdict.** Fill the protocol's exit criteria into `<app>/docs/qa/QA_<version>.md` with evidence. Checks you did not
+   run go into an "untested" list with one line of risk each. Submission to review happens only on the owner's explicit
+   word.
+
+## Pitfalls (measured on real phones)
+
+- `xcode-select` pointing at CommandLineTools makes `xcrun devicectl` and `simctl` fail. The tools set
+  `DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer` unless you override it.
+- The Developer Mode menu only appears after a Mac reveals it: `devmode reveal`.
+- On iOS 27 the confirmation before dialling a business number is presented by `com.apple.BusinessActionSheet`, not
+  SpringBoard. `QA.answerSystemAlert` watches both; add others with `QA_ALERT_HOSTS`.
+- Asking `isHittable` on an element outside the screen records a failure instead of returning false. The sweep only asks
+  elements whose frame centre is on screen.
+- Behind a full-screen cover (onboarding) the first screen is already loaded, so its identifier exists while the cover is
+  up. Decide by the cover's buttons.
+- A test that reads repo fixtures through `#filePath` finds nothing on the device. Put fixtures into the test bundle as
+  resources.
+- Device tests cannot write to Mac paths (`/tmp/...`). Evidence goes into the result bundle through `XCTAttachment`.
+- A simulator can wedge on app launch after days of use. Keep the UI gate on the device.
+- Two sessions on one phone fight over the foreground app and the phone-wide settings. Every phone command takes the lock;
+  a second session waits in arrival order and `lock status` says who holds it.
+- A sheet that closes between a query and the frame read fails a live query with "Failed to get matching snapshot". The sweep
+  reads one snapshot of the tree and taps coordinates.
+- Rows of a SwiftUI List that are off screen are not in the tree, so `waitForExistence` on them never succeeds. Wait for an
+  anchor that is always on screen, then `QA.reveal` the row.
+- A layout that is still moving swallows a tap on a device (an alert sliding with the keyboard, a bottom bar sliding in).
+  `QA.tapUntilGone` waits for the screen to be still and retries once.
+- Tapping a text field that already has keyboard focus raises the edit menu over the buttons next to it. `QA.replaceText`
+  only taps when the field has no focus.
+- The first time a variant switches VoiceOver on, iOS 27.2 shows a welcome prompt over the app and the first tap of the
+  walkthrough lands on it. `QA.settleAfterLaunch(app)` after `app.launch()` answers it (Use VoiceOver) and any permission
+  prompt that is already up.

qakit/skills/prose-qa/SKILL.md

@@ -0,0 +1,49 @@
+---
+name: prose-qa
+description: QA for prose: a memo, a README, a design doc, a paper draft, a blog post. Run the deterministic gates (tools/prose_check.py), build a claims ledger and verify every consequential claim against its source, check the argument, then do a cold read as the intended reader. Use on "review this doc", "check the paper", "is this ready to send", "fact-check", and before publishing anything a reader will act on.
+---
+
+# prose-qa: a document is a product; test it like one
+
+The same principles as `qa-rigor`, applied to text: decide what the reader must take away before editing (expected
+value), check the boundaries (the first paragraph, the numbers, the conclusion), test the failure paths (a reader who
+skips, a reader who disagrees), assert exact values (every number traced to a source), and leave a ledger.
+
+## Order
+
+1. **Expected reading.** Write one line: who reads this, what they must do or believe afterwards, and the one sentence
+   they would quote. If you cannot write it, the document is not ready for QA.
+2. **Mechanical gates.** `python3 tools/prose_check.py draft.md`. DEFECTs (duplicate sentences, empty links, runaway
+   sentences) are fixed before anything else. WARNs are judgment calls: look at each, fix or leave it on purpose.
+3. **Claims ledger.** `python3 tools/prose_check.py draft.md --claims claims.csv` lists every sentence with a number, a
+   year or an evidence word. For each row find the source, paste the quote that supports it, and write the verdict:
+   SUPPORTED (quote says it), PARTIAL (quote says less; the text adds something), NOT IN SOURCE, or NO SOURCE. A claim
+   with NO SOURCE either gets one or leaves the document. Numbers are compared digit by digit; a rounded figure says
+   "about". For a paper, the ledger also covers every citation: the cited work exists, the page or section is right, the
+   cited work says what the sentence says it says.
+4. **Argument.** For a recommendation or a paper: the problem is stated before the answer, the answer follows from the
+   evidence shown (not from evidence the author has in mind), the alternatives considered are named and the reason for
+   rejecting them is given, and the limits are stated by the author before a reviewer states them. Each gap is one
+   line with the paragraph it belongs to.
+5. **Consistency.** The same thing has one name throughout. Numbers that appear twice agree. The summary matches the
+   body. Terms defined on first use are used as defined.
+6. **Cold read.** Read it once as the intended reader with no memory of the drafting (a separate agent or a fresh
+   context when you have one). Mark where you stopped understanding, where you stopped believing, and what you would
+   ask the author. A reader who stops is a defect, not a style note.
+7. **Verdict.** The ledger: gates run and results, claims verified and their verdicts, argument gaps, cold-read stops,
+   and what was not checked with one line of risk each.
+
+## What the tool does not do
+
+`prose_check.py` cannot judge whether the argument holds or whether a quote supports a sentence; that is steps 3 to 6.
+It also has no lexicon for a given house voice; keep that in your own skill and run it after this one.
+
+## For papers specifically
+
+- Every number in the abstract appears in the body with the same value and a table or figure reference.
+- Every table and figure is referenced in the text, and every reference points at an existing table or figure.
+- Methods give what a reader needs to rerun the experiment: data, versions, seeds, hardware, the exact metric.
+- Related work names the closest prior result and says what is different, not only what is similar.
+- Limitations are specific ("we did not test languages other than English") rather than generic.
+- Reproduce one number yourself when the artifacts are available. A paper whose headline number cannot be reproduced
+  from its own artifacts fails QA regardless of the prose.

qakit/skills/qa-rigor/SKILL.md

@@ -0,0 +1,105 @@
+---
+name: qa-rigor
+description: Common QA procedure for any code (apps, Python, scripts, other repos) before merge or release. Design tests from boundaries, states and failure paths, measure test strength with mutation testing (tools/mutate.py), run runtime checkers, confirm in the user's real environment, and leave a verdict ledger. Use on "QA this", "make the tests solid", "ship without bugs", "mutation test", and before reporting a feature done. Apple apps add the device-qa layer on top.
+---
+
+# qa-rigor: green tests and no bugs are different things
+
+A real example from the first measurement (2026-10-28): an app with every test green had 13 surviving mutants where
+`>=` became `>` in its BMI grading. The grade was wrong at every boundary weight and no test knew.
+
+## Layers
+
+| Layer | What it looks at | Where |
+|---|---|---|
+| Common (this skill) | test design, test strength (mutation), runtime checkers, real environment, verdict ledger | all code |
+| Pull request | repo rules, CI replicated locally, the edges a bot reviewer will probe | your repo's PR skill |
+| Apple app | real device, variants (dark, large text, language), screenshots, store requirements | `skills/device-qa` + `docs/device-qa-protocol.md` |
+
+A lower layer never replaces an upper one. An app follows the nine principles below first.
+
+## Principles
+
+1. **Expected values before code.** For every changed behaviour write one line: "this input gives this output". If you
+   cannot write it, the requirement is not settled yet.
+2. **Partition the inputs.** One case per group of inputs that get the same treatment, and three values at every
+   boundary (below, on, above: 14.9, 15, 15.1 km). Empty, maximum length, negative and Unicode are groups too. Screens
+   and lifecycles become a state table that covers every valid transition and every invalid attempt.
+3. **Failure paths are features.** Network gone, permission denied, disk full, bad payload, time zone and DST,
+   concurrent calls: each has a test.
+4. **Assertions are exact values.** A test that only checks `XCTAssertNotNil` or "did not crash" passes when the output is
+   wrong. Compare the result value, the stored state, the side effect that was called.
+5. **Tests are deterministic.** No fixed `sleep` (bounded waits instead); inject time, randomness, network and location;
+   fix seeds; no order dependence.
+6. **Test strength is measured by mutation testing.** Coverage only says "this line ran". Every mutant on a changed line
+   must die, or the line carries the reason it cannot. Procedure below.
+7. **Turn on a runtime checker at least once.** Swift: ASan, UBSan, Main Thread Checker, TSan (Apple platforms: simulator
+   or Mac only). Python: `-W error`, pyflakes (undefined names). JS: strict mode.
+8. **The last check happens in the user's environment.** Run the build the user receives, where they receive it: a Release
+   build on a real device for an app, the boot log after a restart for a server, the installed binary for a CLI.
+9. **Every defect leaves two things.** A regression test that catches it again (watch it fail first) and a guard that stops
+   the same class next time (a lint rule, a harness test, a checklist line).
+
+The verdict is a ledger: the gates you ran and their results, the gates you did not run and one line of risk each. Even
+when every gate is green, an unrun gate is written down.
+
+## Mutation testing
+
+Break the code on purpose, one small change at a time, and rerun the tests. A test failing means the mutant **died** (a
+test pins that behaviour). Everything passing means it **survived** (nobody checks that behaviour).
+
+```bash
+M=tools/mutate.py          # from the QAKit checkout, or <your-repo>/path/to/qakit/tools/mutate.py
+python3 $M --config <mutation.json> --list                                   # which mutants would be made
+python3 $M --config <mutation.json> --changed-since origin/main              # before merge: changed lines only
+python3 $M --config <mutation.json> --changed-since 'latest-tag:<app>/*'     # before release: lines changed since the last tag
+python3 $M --config <mutation.json> --jobs 5 --out <dir>                     # full measurement (periodic)
+python3 $M --config <mutation.json> --out <dir> --resume                     # continue a long run that was cut short
+```
+
+Config (JSON; paths are relative to the git top level that holds the config):
+
+```json
+{"name": "myapp-core", "root": "apps/myapp", "sources": ["Core/Sources/**/*.swift"], "exclude": [],
+ "setup": "", "build": "swift build --build-tests", "test": "swift test --skip-build",
+ "coverage": "swift test --enable-code-coverage ... > {lcov}", "jobs": 4}
+```
+
+- Work happens in a temporary copy of `root`. The checkout is never touched, so other sessions can keep working.
+- A `build` failure means the mutant does not compile (invalid, not counted). A `test` failure or timeout means killed.
+- With `coverage`, mutants on lines no test runs are not executed and are reported **uncovered**. They count with survivors.
+- `{orig}` is the real root (for virtualenv paths), `{file}` and `{stem}` the mutated file (to run related tests only),
+  `{lcov}` the coverage output path.
+- Python example: `"test": "{orig}/.venv/bin/python -m pytest -x -q -p no:cacheprovider"`.
+- Xcode apps take one to two minutes per mutant, so measure a full sweep with `sweep_sample` and run every changed line at
+  release time.
+
+Operators: boundary (`<` and `<=`, `>` and `>=`), equality (`==` and `!=`), logical (`&&` and `||`, `and` and `or`),
+arithmetic (`+` and `-`, `*` and `/`, `+=` and `-=`), boolean (`true` and `false`), negation removal (`!x`, `not x` to
+`x`), call statement removal (`store.save(x)` dropped; logging, print, assert, super and builder elements such as
+`Text(...)` are left alone). C-family languages (Swift, JS, TS, Kotlin, Go, Java, C#) only swap binary operators with
+spaces on both sides, which keeps generics (`Array<Int>`), arrows and unary operators intact. Strings, comments and `#if`
+lines are never touched.
+
+### Handling survivors
+
+For every line of the report (`mutation.md`) do one of two things.
+
+1. **Write a test.** Write the test that fails under that mutant, rerun, confirm it dies. Most survivors are a missing
+   boundary value (principle 2) or a loose assertion (principle 4).
+2. **Write the reason.** Only when the change cannot alter behaviour (an equivalent mutant) add `// mutation-ok: <reason>`
+   (`# mutation-ok: <reason>` in Python) on that line. Examples: `count > 0` and `count >= 1` on an integer, a cache size
+   that only changes performance, log message assembly. "Writing the test is tedious" is not a reason.
+
+"no test runs this line" means no test reaches that code at all. If it is not unit-testable (View code, for instance),
+move it to `exclude` and write next to the config which layer owns it (UI tests, screenshots, device QA).
+
+### Pitfalls (measured)
+
+- **The machine is shared.** SwiftPM builds start one compiler per core. Five jobs pushed a shared Mac's load to 174 and
+  slowed another session's simulator. The engine runs under `nice -n 10`; SwiftPM configs default to `-j 4` and 3 jobs.
+- **Same-size mutants hide behind caches.** Python `.pyc` freshness uses mtime (seconds) and size. A `*` to `/` mutant
+  written in the same second runs the old bytecode and survives falsely. The engine sets `PYTHONDONTWRITEBYTECODE=1`
+  and advances mtimes.
+- **Branches that never compile.** The other side of `#if SWIFT_PACKAGE ... #else` is not compiled in that build, so its
+  mutants always survive. Mark the line `mutation-ok`.