mutate4py 0.2.0__tar.gz → 0.3.0__tar.gz

{mutate4py-0.2.0 → mutate4py-0.3.0}/.agents/roles/QA.md

@@ -3,12 +3,6 @@
 ## Pre-Handoff Lint and Format Check
 Before handing off, MUST run `uv run ruff check src/ tests/ && uv run ruff format --check src/ tests/`. Fix any failures before handoff. Do not hand off with lint or format violations — integrator absorbs CI fix cycles.
 
-## QA Feature Step Wiring
-Before marking QA done, grep `acceptance/run_acceptance.sh` for every `*_qa.feature` file present in `features/`. Confirm each has a `QA_STEPS_MAP` entry. A missing `QA_STEPS_MAP` entry causes silent `SKIP QA: no steps module for X` output — the feature is silently skipped and failures are masked.
-
-## QA Harness: Handling CLI-Rejected Values (e.g. --max-workers 0)
-When a QA feature passes a value that the CLI rejects (e.g., `--max-workers 0` to test serial mode), handle it in `_run_qa_cmd` — not by changing the CLI validator. Changing the CLI validator to accept test-only values breaks the cli-surface acceptance test.
-
 ## Shell Counter Files in Acceptance Steps
 Shell scripts in acceptance steps must not use `$COUNT` from `ls dir | wc -l` as unique file names with parallel workers — it is not safe under concurrency. Use `$$` (PID) or content-based detection (grep for a token in the worker's source copy) instead.
 

{mutate4py-0.2.0 → mutate4py-0.3.0}/.agents/roles/hardender.md

@@ -4,25 +4,24 @@
 This project (mutate4py) is self-hosted — it generates and checks its own mutation manifests. Use `uv run mutate4py` (not `uvx mutate4py`; uvx isolates the env and breaks self-referential scanning).
 
 Key commands:
-- `uv run mutate4py src/ --check-manifest` — verify all manifests are current (CI gate; exits 0 if current, 1 if missing/stale)
+- `uv run mutate4py src/ --check-manifest --manifest-file` — verify the manifests are current: unit hashes, recorded Survivors and equivalent markers. It runs no test, so it does not check tests digests. CI gate; exits 0 if current, 1 if a manifest is missing, stale, or records an unaccepted Survivor.
 - `uv run mutate4py <file> --scan` — scan a single file and print surviving mutants
 - `uv run mutate4py <file> --update-manifest` — regenerate and embed a file's manifest
 
 `--scan`, `--update-manifest`, and `--check-manifest` are mutually exclusive. `--scan` and `--update-manifest` accept exactly ONE file; for multi-file operations loop over files or use a directory with `--check-manifest`.
 
-## Running a Scored Mutation Pass (use `just mutate`, never raw `mutate4py --mutate-all`)
-`_runner.py` prints one `[i/total] <status> line N ...` line per mutation site unconditionally on a scored run — for the ~346-site full-src/ surface that's ~20k chars an agent's Bash tool would otherwise capture in full, every hardening iteration.
+## Running a Scored Mutation Pass (use `just mutate`, never raw `mutate4py`)
+`_runner.py` prints one `[i/total] <status> line N ...` line per mutation site unconditionally on a scored run. On a full-src/ run that is many thousands of characters an agent's Bash tool would otherwise capture in full, every hardening iteration.
 
-Invoke scored runs through the justfile instead of calling `mutate4py --mutate-all` directly:
+Invoke scored runs through the justfile instead of calling `mutate4py` directly:
 ```
-just check test-unit test-integration   # once per verification pass, refreshes lcov.info
-just mutate <path>        # e.g. just mutate src/mutate4py/_runner.py --mutate-all
+just mutate <path>                      # e.g. just mutate src/mutate4py/_runner.py
 ```
-`just mutate` is `justfile`'s wrapper: it redirects the run to a temp log, then prints only the `Mutation Report` block (Killed/Survived/Uncovered counts + every Survivor's `line N desc` — still fully actionable) via `awk '/^Mutation Report$/,0'`. On a non-zero exit other than 3 it also tails the last 20 log lines, so an infra failure that happens before the summary prints is still visible. This is the same shape `perf.sh`'s (now `just perf`) `_run` helper already used — see `justfile`'s header comment.
+`just mutate` is `justfile`'s wrapper: it redirects the run to a temp log, then prints only the `Mutation Report` block (Killed/Survived/Uncovered counts + every Survivor's `line N desc` — still fully actionable) via `awk '/^Mutation Report$/,0'`. On a non-zero exit other than 3 it also tails the last 20 log lines, so an infra failure that happens before the summary prints is still visible.
 
-Exit 3 means unaccepted survivors: kill them, or mark a recognized equivalent with `# mutate4py: equivalent <category>` on the Site's line (ADR 0026).
+Exit 3 means unaccepted Survivors: kill them with a unit test (ADR 0023: code mutation is unit-only, so a Mutant that only a component test kills stays a Survivor), or mark a recognized equivalent with `# mutate4py: equivalent <category>` on the Site's line (ADR 0026). A Survivor recorded on a function whose Unit hash and tests digest are unchanged is skipped by later runs and still exits 3 (ADR 0029). It is not retried, so it has to be killed or accepted rather than waited out.
 
-`just mutate` narrows against `contexts.db`, an isolated-session test-context db (docs/adr/0021) it builds/refreshes itself every run via its `test-context-db` dependency — no separate step needed. A cache (mutate4py's own, keyed on the project tree) makes a rerun on an unchanged tree ~4 seconds; a first build, or one after a source/test edit, costs ~16 minutes on this project's full non-integration suite. That first cost is not a hang — let it run.
+The loop: a scored run re-scores the changed functions only. A changed function is one whose Unit hash or Tests digest moved since its Manifest entry was written, one that is Unscored, or one that records Survivors but has no digest yet. Each run takes one Measurement first (the unit tests, once, under coverage) and narrows each Mutant to the tests that reached its line, so there is no setup step to run before `just mutate`. An unchanged function is skipped, its recorded Survivors included, and they still fail the run. There is no flag for a full re-score: `--mutate-all` is removed (spec #115). A run with no Manifest scores every function (an uncovered one gets an `uncovered` score), so a deliberate full re-score needs a file with no sidecar Manifest.
 
 `just check` (no args) runs the full CI-equivalent gate list the same way — one ✓/✗ line per gate, full output in `check.log`, tailed only on failure.
 

mutate4py-0.3.0/.github/workflows/ci.yml

@@ -0,0 +1,61 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+permissions:
+  contents: read
+
+jobs:
+  ci:
+    name: CI
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+      - uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
+        with:
+          enable-cache: true
+          version: "0.12.3"
+      - uses: extractions/setup-just@53165ef7e734c5c07cb06b3c8e7b647c5aa16db3 # v4.0.0
+        with:
+          just-version: "1.58.0"
+      - uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
+        with:
+          go-version: "1.26.5"
+      # Gate commands live in the justfile (one source of truth for CI + local
+      # `just check`) — see justfile's header comment. Steps stay separate
+      # (rather than one `just check` call) so the Actions UI keeps per-gate
+      # timing and log folding.
+      - name: Lint
+        run: just lint
+      - name: Format check
+        run: just format-check
+      # Coverage is measured on test-unit alone: it writes lcov.info and
+      # applies the threshold, so the CRAP step below scores unit-only data.
+      # test-component appends to .coverage without touching lcov.info, and
+      # test-integration adds no coverage data at all (measured; see the
+      # justfile). Both stay here as test gates in their own right. These steps share one working directory by necessity: test-unit
+      # cannot move to a separate job from CRAP without merging coverage first.
+      - name: Tests (unit)
+        run: just test-unit
+      - name: Tests (component)
+        run: just test-component
+      - name: Tests (integration)
+        run: just test-integration
+      - name: CRAP
+        run: just crap
+      - name: Install tools
+        # Installs the pinned drywall and gherkin-parser through the justfile's
+        # `install-tools` recipe, the same one a local run uses, so the pins
+        # live in one place. Adds ~/.local/bin to PATH for the steps after it.
+        run: |
+          just install-tools
+          echo "$HOME/.local/bin" >> "$GITHUB_PATH"
+      - name: DRY
+        run: just dry
+      - name: Mutation manifest
+        run: just manifest
+      - name: Acceptance tests
+        run: just acceptance

{mutate4py-0.2.0 → mutate4py-0.3.0}/AGENTS.md

@@ -8,14 +8,15 @@ Navigation and universal invariants for all agents in this project.
 
 ## Local Gates
 - `just check` — all CI gates, same order as `.github/workflows/ci.yml` (log-per-gate in `check.log`, ✓/✗ printed, tail on failure). `just check <gate> [<gate> ...]` runs a subset; `just --list` shows public recipes.
-- `just mutate <path> [<args>]` — scored mutation run with per-mutant progress captured to a log; only the `Mutation Report` summary is printed. Use this instead of raw `uv run mutate4py <path> --mutate-all` (see `.agents/roles/hardender.md`). Depends on `test-context-db`, so a bare `just mutate <path>` also builds-or-refreshes the isolated-session test-context db first: ~16 min on a cold cache, ~4s once warm (see the `test-context-db` recipe in `justfile`).
+- `just mutate <path> [<args>]` — scored mutation run with per-mutant progress captured to a log; only the `Mutation Report` summary is printed. Use this instead of raw `uv run mutate4py <path>` (see `.agents/roles/hardender.md`).
+- While iterating on a change, run `just check test-unit` (make the behaviour work first); the full `just check` runs once before handing off or committing.
 
 ## Code Review
 - Use `/mattpocock-skills:code-review`, never bare `/code-review` (different, heavier skill).
 
 ## Tool Paths (Local Machine)
 - `crap4py`: installed from local sibling `~/workspace/addi/crap4py` via `uv tool install ~/workspace/addi/crap4py` (not PyPI, not GitHub URL)
-- `drywall`: available at `/Users/gabadi/.local/bin/drywall` (not in PyPI under that name)
+- `drywall` and `gherkin-parser` (not in PyPI): run `just install-tools` once per machine. It installs the versions pinned in the `justfile` (`DRYWALL_TAG`, `GHERKIN_PARSER_COMMIT`) into `~/.local/bin`, the same pins CI installs with, and rerunning it is a no-op. `~/.local/bin` must be on PATH. When either tool is missing, `just check` fails before its first gate and names this recipe.
 - `uv run mutate4py` is the correct invocation for mutation scan and manifest checks (not `uvx mutate4py`); uvx isolates the env and breaks self-referential scanning
 - `uv run mutate4py src/ --check-manifest --manifest-file` — CI gate; exits 0 if all manifests current, 1 if any are missing/stale. `src/` stores manifests as sidecars (`*.manifest.json`), so `--manifest-file` is required — without it the check looks for embedded footers and always fails (issue #47).
 
@@ -26,7 +27,7 @@ Navigation and universal invariants for all agents in this project.
 - `mutate4py --scan`: accepts exactly ONE file argument; loop over files separately for multi-file scans
 
 ## Acceptance Test Safety
-- Manifest QA fixtures (`acceptance/fixtures/plain.py`, `acceptance/fixtures/stale.py`) are committed inputs and MUST NOT be overwritten; all manifest QA steps must use a writable-copy pattern (see `acceptance/steps/manifest_qa_steps.py:setup_copy()`)
+- Committed fixtures in `acceptance/fixtures/` (`plain.py`, `stale.py` for `manifest.feature`; `mixed_operators.py` and the others for `site-discovery.feature`) are committed inputs and MUST NOT be overwritten; steps that run mutate4py against one work on a writable copy (see `acceptance/steps/manifest_steps.py:setup_copy()` and `site_discovery_steps.py:given_fixture_copy()`). There is no separate QA feature pass: each `X_qa.feature` was folded into `X.feature`.
 - `runner_adapter.py` must be executable (`chmod +x`) before gherkin-mutator can use it; tool does not emit a clear permission error; supports `RUNNER_WORKERS` env var (default 4) for concurrent job execution
 - When modifying source with an embedded manifest, always strip manifest first, modify body, then re-embed; never append after the manifest footer — `strip_manifest` removes everything from begin-marker to EOF
 
@@ -35,7 +36,8 @@ Navigation and universal invariants for all agents in this project.
 - Acceptance step files are boundary files (15-site mutation threshold); extractable pure logic belongs in `*_helpers.py` modules with their own unit tests.
 - `coverage_helpers.py` is the testable module for acceptance step helper functions; unit tests live in `tests/test_coverage_helpers.py`.
 - `cli_surface_helpers.py` is the testable module for CLI surface acceptance step helpers; unit tests live in `tests/test_cli_surface_helpers.py`.
-- `__main__.py` is a pure CLI adapter. `_runner.py` re-exports `scan_report`, `scan_report_with_coverage` (defined in `_report.py`), `update_manifest` and `check_manifest` (defined in `_manifest_modes.py`); test them directly. Do not move domain functions back to `__main__.py`.
+- `run_loop_helpers.py` and `parallel_workers_helpers.py` are the testable modules for the run-loop and parallel-workers acceptance steps; unit tests live in `tests/test_run_loop_helpers.py` and `tests/test_parallel_workers_helpers.py`.
+- `__main__.py` is a pure CLI adapter. `_runner.py` re-exports `scan_report` (defined in `_report.py`), `update_manifest` and `check_manifest` (defined in `_manifest_modes.py`); test them directly. Do not move domain functions back to `__main__.py`.
 
 ## References
 - `CONTEXT.md` — the project's glossary. Use its terms; don't drift to synonyms.

mutate4py-0.3.0/CHANGELOG.md

@@ -0,0 +1,52 @@
+# Changelog
+
+## 0.3.0 (unreleased)
+
+The run measures coverage itself and re-scores only what changed (spec #89). Decisions
+are in [`docs/adr/`](docs/adr/): ADR 0027 (the Measurement), ADR 0028 (executors),
+ADR 0029 (the tests digest and skip rule).
+
+### Removed flags
+
+Each flag below is refused with exit 2 and a message naming it, `<flag> removed in 0.3.0: <reason>`.
+Remove them from scripts and CI.
+
+- `--cov-cmd`: the run measures coverage itself (the Measurement, ADR 0027). No replacement.
+- `--lcov`: the run measures coverage itself; no `lcov.info` is read or written. No replacement.
+- `--reuse-coverage`: each scored run takes its Measurement; a rerun with nothing changed skips it. No replacement.
+- `--test-contexts`: selection reads the run's own Measurement and its Test map; there is no db to pass. No replacement.
+- `--build-test-contexts`: no test-context db is built. No replacement.
+- `--since-last-run`: a default run already re-scores only the changed functions. Drop the flag.
+- `--lines`: edit a function to re-score it.
+- `--mutate-all`: a run scores every changed or Unscored function itself. A run with no Manifest scores every function (an uncovered one gets an `uncovered` score). Drop the flag.
+
+### Changed
+
+- Coverage comes from one Measurement per scored run: one pytest session of the unit tests
+  under coverage.py, which also gives the Baseline and the Test map (ADR 0027).
+- New runtime dependency: coverage.py, `coverage>=7.10`.
+- `--min-timeout` default is 5.0 s, up from 1.0 s. `--timeout-factor` stays at 10.
+- The per-Mutant overhead is the Measurement's wall time minus the summed durations of its
+  tests. No `--collect-only` pass runs. The overhead hint fires at half of the Measurement's wall.
+- A default run re-scores only changed functions. A function is changed when its Unit hash
+  or its tests digest differs, it is Unscored, or it records Survivors and has no digest yet
+  (ADR 0029). A default run also scores every Unscored function, including one no covered Site reaches (spec #115). A run with no Manifest scores every function (an uncovered one gets an `uncovered` score); `--mutate-all` is removed.
+- Unchanged functions are skipped even when they record Survivors. Those Survivors print under
+  `Recorded survivors (function unchanged, not re-run):` and still count toward exit 3 (0 with `--allow-survivors`).
+- Test selection has two outcomes, `narrowed` and `static`, read from the Measurement's Test map
+  (ADR 0018). The report prints `Test selection: narrowed <n>, static <k>`.
+- Every non-accepted Survivor is re-run with the full test set, in a fresh interpreter through the
+  subprocess executor. It counts only if it survives that run too (ADR 0018 amendment #97, ADR 0028).
+- Executors: the forking executor runs Mutants by fresh import or swap mode (ADR 0028); the
+  `Executor:` line of the report says how each Mutant ran.
+- `--check-manifest` checks unit hashes, recorded Survivors and equivalent markers. It runs no
+  test, so it does not check tests digests.
+
+### Migration
+
+- Delete the seven removed flags from scripts and CI. The run needs no prior coverage step, and
+  `--pytest-args` still passes through to pytest.
+- A Survivor recorded on an unchanged function now fails the run (exit 3) until it is killed or
+  accepted with `# mutate4py: equivalent <reason>`.
+- A 0.2.x Manifest entry with Survivors and no tests digest is retried once.
+- To keep the 1.0 s floor, pass `--min-timeout 1.0`.

{mutate4py-0.2.0 → mutate4py-0.3.0}/CONTEXT.md

@@ -48,6 +48,13 @@ _Avoid_: cache, state file, lockfile
 A Function unit's structural fingerprint. Reformatting and comment edits leave it
 alone; behaviour-affecting edits change it.
 
+**Tests digest**:
+A Function unit's SHA-256 over the test inputs that can reach its covered Sites: the
+test files the Measurement shows reaching them, their conftests, the run's support
+files, the pytest table of `pyproject.toml`, the pytest ini file and `uv.lock`. Stored
+beside the Function unit's score. A stored digest that no longer matches the fresh one
+re-scores that unit (ADR 0029).
+
 **Differential rerun**:
 Re-testing only the Sites whose Function unit changed since the last Manifest. The
 default once a Manifest exists.
@@ -58,30 +65,65 @@ under the Function unit's `accepted` list with its reason; it does not fail the
 _Avoid_: ignored, suppressed
 
 **Unscored**:
-A Function unit whose Manifest entry has no `score` (written before 0.1.5, or never
-fully selected by a scored run). `--check-manifest` judges it on hashes alone.
+A Function unit with no stored entry, or whose Manifest entry has no `score` (written before
+0.1.5, or never scored by a run). A default run scores it (ADR 0029 rule 1). `--check-manifest` judges it on hashes alone.
 _Avoid_: clean
 
 ### Coverage
 
 **Coverage gate**:
-The line-coverage filter that splits discovered Sites into covered and uncovered.
-Branch data is deliberately ignored.
+The line-coverage filter that splits discovered Sites into covered and uncovered. It
+reads the Measurement's data; a line is covered when any context reached it, Empty
+context included. Branch data is deliberately ignored.
 _Avoid_: coverage filter, eligibility check
 
 **covered / uncovered**:
 The two disjoint, exhaustive partitions of the discovered Sites.
 
+**Measurement**:
+The one pytest session a scored run takes before any Mutant runs: the run's unit tests
+in the Project environment, under coverage.py's `ctrace` core, each test in its own
+context. Its data gives the Baseline, the Coverage gate and the Test map (ADR 0027).
+_Avoid_: coverage run, pre-run
+
 **Baseline**:
-One run of the test suite against unmutated source. It must pass, and its duration
-sets the Mutant timeout.
+The Measurement's pass requirement: the unmutated suite must pass in the Measurement,
+or no Mutant runs. The Measurement's wall time gives the Per-Mutant overhead (wall
+time minus summed test durations), which is part of every Mutant's timeout.
+
+**Test map**:
+The named tests that reached each covered line in the Measurement. Dispatch runs a
+Site's narrowed tests from it.
+_Avoid_: coverage contexts, test list
 
 ### Execution
 
 **Executor**:
 The interface both Mutant-execution backends implement — prime once, then run a test
-argument list under a timeout and classify. The forking executor is the fast path;
-the subprocess executor is the always-correct fallback.
+argument list under a timeout and classify. The forking executor runs Mutants from a
+Fork server, each by fresh import, Swap mode or a fresh interpreter. The subprocess
+executor is the whole-run fallback where forking is impossible or unsafe, and it runs
+every Survivor confirmation.
+
+**Fork server**:
+The `_worker_server` process a serial forking run starts over the project tree itself,
+under the Project environment. It primes the forking executor once and forks one child
+per Mutant, so the orchestrator's state never reaches a Mutant.
+
+**Harness module**:
+A file loaded before the Fork server primes: part of the server's own code, the executor
+and protocol code a child runs around pytest. Each Mutant in it runs in a fresh interpreter,
+because a swap would change the code that reports the child's result.
+
+**Swap mode**:
+The forking executor's way of running a file the prime imported (a conftest or a
+plugin): the forked child swaps the mutated function bodies into the live functions and
+runs pytest in-process. A Mutant the swap refuses runs in a fresh interpreter instead.
+
+**Survivor confirmation**:
+The re-run of every Survivor through the subprocess executor with the full test set,
+before a Survivor is reported. A narrowed selection or a swap can let a Mutant survive
+that the full test set kills, so a Survivor is not trusted until this run agrees.
 
 **Worker**:
 An isolated tree copy of the working directory with its own provisioned environment,
@@ -98,7 +140,7 @@ A symlink-only directory that supplies the `mutate4py` package to a Worker venv
 The target project's `.venv`, or a `uv run --with` overlay on it; scored runs must run under it.
 
 **Selected sites**:
-The covered Sites actually mutated, after line filtering and differential selection.
+The covered Sites actually mutated, after differential selection.
 
 **Classification**:
 The per-Site verdict: **killed**, **survived**, or **timeout**. Timeout is visible
@@ -108,46 +150,10 @@ _Avoid_: result, status, outcome (reserve *outcome* for Selection outcome)
 ### Test selection
 
 **Selection outcome**:
-What the test-context db says about a Site's line — **narrowed** (named tests cover
-it, so only those run), **static** (import-time code no single test owns, so the
-full test set runs), **under-listed** (see below; also runs the full test set), or
-a disagreement.
+What the Measurement says about a Site's line — **narrowed** (named tests reach it, so
+only those run) or **static** (import-time code no single test owns, or a line no test
+reached, so the full test set runs).
 
 **Empty context**:
 Coverage.py's whole-run context, as opposed to a named per-test one. A line matched
-only by it ran at import time. A signal, not noise.
-
-**Under-listed**:
-A Selection outcome (issue #69): the test-context db names covering tests for a
-Site's line, but at least one was recorded by a *dynamic* (`pytest
---cov-context=test`, i.e. `switch_context()`) context rather than a static
-(isolated-session) one. ADR 0021 proved a dynamic-context collector session
-silently drops every covering test but the first to reach a shared line or arc, so
-such a list can never be trusted complete — it is never returned as `narrowed`.
-mutate4py runs the full test set instead, same dispatch as `static` but tallied
-apart from it (`degraded` in the run summary) so the report can't claim `static 0`
-while Sites were actually rejected. Detected by the `|setup`/`|run`/`|teardown`
-suffix pytest-cov appends to a dynamic context's name — an isolated-session build
-(`--build-test-contexts`) never produces that suffix.
-_Avoid_: stale context, incomplete narrowing
-
-**Selection disagreement**:
-The test-context db and the Coverage gate contradicting each other. Always an input
-defect — a stale db or a path mismatch — never uncovered code, so it aborts the run
-rather than falling back. Distinct from **under-listed**: a disagreement means the db
-can't account for the line at all (no safe fallback exists); under-listed means it
-names some tests but can't prove the list complete (falling back to the full test set
-is safe).
-
-**Test-context cache**:
-The sidecar fingerprint of the inputs a test-context db was built from, letting a
-later `--build-test-contexts` run skip the rebuild when none of them changed. Named
-*cache* — not *manifest* — deliberately: the Manifest is a durable record of what was
-mutation-tested and belongs in the repo, whereas this is disposable, gitignored, and
-rebuilt from scratch whenever it can't prove itself fresh.
-_Avoid_: test-context manifest, context manifest
-
-**Freshness / staleness**:
-Whether the Test-context cache can prove nothing that would change the db has
-changed. Deliberately one-way: fresh means proven unchanged, stale means anything
-else, including "couldn't tell".
+only by it ran at import time.