bmad-module-ultracode-goal 7.0.0 → 7.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +1 -1
- package/docs/architecture.md +1 -0
- package/docs/gate-model.md +1 -1
- package/package.json +1 -1
- package/skills/ucg-resolve/SKILL.md +37 -6
- package/skills/ultracode-goal/references/define-done.md +1 -1
- package/skills/ultracode-goal/references/execute.md +1 -1
- package/skills/ultracode-goal/references/gate.md +29 -19
- package/skills/ultracode-goal/scripts/moved_ac_check.py +233 -0
- package/skills/ultracode-goal/scripts/operator_owed.py +59 -27
|
@@ -19,7 +19,7 @@
|
|
|
19
19
|
"name": "bmad-module-ultracode-goal",
|
|
20
20
|
"source": "./",
|
|
21
21
|
"description": "Supercharges BMAD with Claude Code's Auto Mode, Auto Memory, and hooks: autonomous Epic execution to a machine-checked, TEA-gated Definition-of-Done. Execute Epics. Accumulate Knowledge. Deliver Outcomes.",
|
|
22
|
-
"version": "7.
|
|
22
|
+
"version": "7.1.0",
|
|
23
23
|
"author": {
|
|
24
24
|
"name": "Armel"
|
|
25
25
|
},
|
package/docs/architecture.md
CHANGED
|
@@ -57,6 +57,7 @@ skills/ultracode-goal/
|
|
|
57
57
|
│ ├── preflight_check.py # mechanical preflight facts + blocker budget
|
|
58
58
|
│ ├── gate_eval.py # gate status -> verdict (the completion authority)
|
|
59
59
|
│ ├── operator_owed.py # is a gate FAIL owed only operator-evidence ACs' records? (fail-closed)
|
|
60
|
+
│ ├── moved_ac_check.py # does trace's matrix still score an AC the operator moved out? (fail-closed)
|
|
60
61
|
│ ├── mutate_check.py # mutation harness: attributable-red proof for the story's guards
|
|
61
62
|
│ ├── probe_servers.py # which URLs a test runner may reuse already answer, from which folder; stops a server the run left up
|
|
62
63
|
│ ├── link_worktree_deps.py # links the main tree's dependency folders into the gate worktree
|
package/docs/gate-model.md
CHANGED
|
@@ -54,7 +54,7 @@ Any unrecognized status escalates (the script's `GATE_VERDICT.get(gate_status, "
|
|
|
54
54
|
|
|
55
55
|
One `FAIL` takes a different route. When the only criteria TEA's coverage matrix holds open are acceptance criteria tagged `**Gate-ability:** operator-evidence`, whose proof is a live result a person records, no re-loop can change the gate. A plain `**Gate-ability:** operator` tag is not read this way: it marks work the dev agent builds, so an open criterion carrying it keeps the re-loop. The skill checks that with `scripts/operator_owed.py`, which fails closed, and escalates with kind `operator-evidence-owed` instead of re-looping. `gate_eval.py` still prints `reloop`: the route changes, not the verdict.
|
|
56
56
|
|
|
57
|
-
Once `/ucg-resolve` closes that escalation, with the person's records on disk at the commit the gate measured, the resume re-gates that same commit instead of re-driving the story. An `operator-evidence` criterion that already has a counted `pass` record, and that TEA still holds below `FULL`, is still owed: the record covers only part of the criterion, so the escalation names the records already counted and asks the operator to record the rest or to narrow or move the criterion in writing. Re-gating the records already counted changes nothing, so `/ucg-resolve` closes it only on a new `pass` record or on that written change.
|
|
57
|
+
Once `/ucg-resolve` closes that escalation, with the person's records on disk at the commit the gate measured, the resume re-gates that same commit instead of re-driving the story. An `operator-evidence` criterion that already has a counted `pass` record, and that TEA still holds below `FULL`, is still owed: the record covers only part of the criterion, so the escalation names the records already counted and asks the operator to record the rest or to narrow or move the criterion in writing. Re-gating the records already counted changes nothing, so `/ucg-resolve` closes it only on a new `pass` record or on that written change. A criterion the operator moves out of the story leaves the story's numbered list for a section of its own that names its owner and the ruling, its number retired and never reused, and `/ucg-resolve` closes on the move only once the story file carries it, recording each moved criterion in a per-story `.moved-<story_id>` file. A decomposed child's criterion is not moved this way, since its parent's story file is never edited. TEA's trace has no status for a moved criterion and can still score it from the epic, the PRD or the test design, so every gate cycle of a story with that record names the main-tree story file, and every recorded oracle source, to trace, and before `gate_eval.py` runs, `scripts/moved_ac_check.py --commit` reads TEA's coverage matrix for the commit under assessment: while a row still holds a moved criterion, or the trace files cannot be read or measured another commit, the story escalates `operator-evidence-owed` again, with a `decision_needed` that opens "The moved-AC check refused" and its exit code, instead of re-looping with nothing left to build.
|
|
58
58
|
|
|
59
59
|
## The production AND
|
|
60
60
|
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"$schema": "https://json.schemastore.org/package.json",
|
|
3
3
|
"name": "bmad-module-ultracode-goal",
|
|
4
|
-
"version": "7.
|
|
4
|
+
"version": "7.1.0",
|
|
5
5
|
"description": "BMAD module — Run a BMAD Epic autonomously to a machine-checked, TEA-gated Definition-of-Done.",
|
|
6
6
|
"keywords": [
|
|
7
7
|
"bmad",
|
|
@@ -230,9 +230,13 @@ the file holds more than one entry for that `id`, carry each forward, in file or
|
|
|
230
230
|
write one entry in their place. Leave `history` off an entry that replaced nothing. A pass
|
|
231
231
|
that only parks an item records one line, while the entry it replaces can hold the
|
|
232
232
|
operator's whole ruling: a plain replacement would erase that ruling, and nothing in the
|
|
233
|
-
pass would report the loss. Every reader of the file keys on `id` and `action` alone
|
|
233
|
+
pass would report the loss. Every reader of the file but one keys on `id` and `action` alone
|
|
234
234
|
(`{ucg-root}/scripts/red_ids.py`, which preflight and the resume read it through), so
|
|
235
|
-
`history` changes no suppression.
|
|
235
|
+
`history` changes no suppression. The one is the gate's re-read of rulings at the start of
|
|
236
|
+
each cycle (`{ucg-root}/references/gate.md`, "An owned `gate-gap:` line is not this
|
|
237
|
+
story's to close"), which reads the `answer` and the `history` of each `close` entry and
|
|
238
|
+
applies only the latest ruling for each line or AC: a later return or reassignment
|
|
239
|
+
replaces an earlier one.
|
|
236
240
|
|
|
237
241
|
**If an existing `.decisions.json` is unreadable, unparseable, or parsed but missing a
|
|
238
242
|
list-valued `decisions`, stop — do not overwrite it.** Say which file failed and why. This
|
|
@@ -278,18 +282,45 @@ The two dispositions are genuinely different things, not two names for one:
|
|
|
278
282
|
every AC it lists, made at the commit it names. Those are the story's
|
|
279
283
|
`**Gate-ability:** operator-evidence` ACs; a plain `operator` tag is work the dev agent
|
|
280
284
|
builds, never evidence this close waits on. If one is missing, say which and record
|
|
281
|
-
`defer`. An AC the
|
|
285
|
+
`defer`. An AC the operator moves out of the story in writing needs no record, and the
|
|
286
|
+
close waits for the move instead: the story file must carry it in the moved-AC form
|
|
287
|
+
(`{ucg-root}/references/gate.md`, "A moved AC leaves the story's list"), its number
|
|
288
|
+
starting no item under `## Acceptance Criteria` and a note under an H2 heading of its own
|
|
289
|
+
after that section naming that number, the owner and the ruling. Until it does, record
|
|
290
|
+
`defer`. When it does, write `{workflow.implementation_artifacts}/.moved-<story_id>`,
|
|
291
|
+
one line per AC it moves, opened by its `AC-<n>`, `AC-<n> to <owner> per <ruling>`,
|
|
292
|
+
adding to any lines already there, and an `oracle=<absolute path>` line for each oracle
|
|
293
|
+
source the operator edited with the move (a relative one resolved against
|
|
294
|
+
`{project-root}`, and only a path inside the work tree `git rev-parse --show-toplevel`
|
|
295
|
+
prints); log each open `gate-gap:` line on the moved AC again, under the story's key,
|
|
296
|
+
with `owned-by` the same owner per the same ruling; and log the move in the story's
|
|
297
|
+
`.decision-log.md` entry as a `moved:` line, which records it for a reader and which no
|
|
298
|
+
step reads. A written return of a moved AC to the story is applied here too, as
|
|
299
|
+
`{ucg-root}/references/gate.md`, "A moved AC leaves the story's list", says: its line
|
|
300
|
+
leaves the record, and the record goes once no AC line is left. A decomposed child's AC
|
|
301
|
+
is not moved this way: it lives in its parent's story file, which is never edited and
|
|
302
|
+
which every sibling reads; record `defer` and propose a re-plan of the split instead. An
|
|
303
|
+
escalation whose `decision_needed` opens with "The moved-AC check refused (exit 1)"
|
|
304
|
+
closes, its moved ACs owing no record, only once the operator states in writing that
|
|
305
|
+
each source it names no longer lists the AC for this story and you have added each
|
|
306
|
+
source they edited inside the work tree to `.moved-<story_id>` as an `oracle=<absolute
|
|
307
|
+
path>` line, or once they return the AC in writing; until then record `defer`. One that
|
|
308
|
+
opens with "The moved-AC check refused (exit 2)" closes at once when it is the story's
|
|
309
|
+
first exit-2 refusal at that commit; when it says one came before at that commit, it
|
|
310
|
+
closes only once the operator confirms in writing that trace can write its files there,
|
|
311
|
+
and until then record `defer`. An AC the escalation names as partly recorded, with the
|
|
312
|
+
ids of the records TEA
|
|
282
313
|
already counted for it, closes only on a `pass` record at that commit whose `id` is not
|
|
283
314
|
among the ones the escalation names, or on the operator's written narrowing or move of
|
|
284
315
|
the AC: the records already counted are not enough, since re-gating them changes
|
|
285
316
|
nothing. This surface never writes a record itself: the operator is its producer
|
|
286
317
|
(`{ucg-root}/references/gate.md`, "Operator evidence is a recorded live result"). A
|
|
287
|
-
ruling that narrows or moves the AC is the operator's story edit: record it,
|
|
318
|
+
ruling that narrows or moves the AC is the operator's story edit: record it, naming each
|
|
319
|
+
AC a move retires, then close.
|
|
288
320
|
**That close also writes the story's re-gate marker and resets its turn counter.** Write
|
|
289
321
|
`{workflow.implementation_artifacts}/.regate-<story_id>` as two lines, `sha=` the
|
|
290
322
|
assessed commit the sidecar names (in full, as `git rev-parse` prints it) and
|
|
291
|
-
`decision=` the id you closed, then delete the sidecar as for any close, and delete
|
|
292
|
-
`{workflow.implementation_artifacts}/.budget-<story_id>.json`: waiting for operator
|
|
323
|
+
`decision=` the id you closed, then delete the sidecar as for any close, and delete `{workflow.implementation_artifacts}/.budget-<story_id>.json`: waiting for operator
|
|
293
324
|
evidence is not the story's work. The marker is the re-gate entry. While its `sha=`
|
|
294
325
|
names `HEAD`, the resume's `--assert-armed` call routes the story `gate-owed`, so it
|
|
295
326
|
re-enters at Stage 5 to re-gate that commit, and is never re-driven from step 0, where
|
|
@@ -24,7 +24,7 @@ For every **in-scope** story, in sprint order (leave out a parent that preflight
|
|
|
24
24
|
|
|
25
25
|
1. **`ucg-create-story`**: produce the dedicated story file with full implementation context and *clear, testable* acceptance criteria. Pass it `implementation_artifacts` set to `{workflow.implementation_artifacts}`, so the file lands where this run reads it. If preflight already generated this story, treat the change request as a refinement, not a re-create. Vague ACs are the single most common cause of the next step halting, so sharpen them here.
|
|
26
26
|
|
|
27
|
-
**A criterion with an operator half is split here.** UCG's shaping asks every acceptance criterion for a CI-versus-operator split. When an AC's split names a part no script or test can decide (a measurement on a build or machine CI cannot run, a judgment only a person can make), have `ucg-create-story` split it. The machine-decidable part stays one AC tagged `**Gate-ability:** ci-machine`. The operator part becomes its own numbered AC, numbered after the story
|
|
27
|
+
**A criterion with an operator half is split here.** UCG's shaping asks every acceptance criterion for a CI-versus-operator split. When an AC's split names a part no script or test can decide (a measurement on a build or machine CI cannot run, a judgment only a person can make), have `ucg-create-story` split it. The machine-decidable part stays one AC tagged `**Gate-ability:** ci-machine`. The operator part becomes its own numbered AC, numbered after the highest number the story has ever used, so no existing number moves and no retired number comes back: the number of an AC the operator moved out of the story stays retired (`references/gate.md`, "A moved AC leaves the story's list"), since live records key on `AC-<n>`. It carries its own `**Gate-ability:** operator-evidence` line. An AC whose split names only such a part, with nothing a script or test can decide, is not split. Retag it in place, keeping its number, with `**Gate-ability:** operator-evidence`, whatever tag it carried, only when it asks the dev agent to build or change nothing: its whole content is a measurement, run or observation a person makes and records. The test is whether the AC asks for anything to be built or changed, not who can check it: an AC that asks for a change (code, a comment, a doc, a ledger edit) keeps its tag even when only a person's read can check it. The `operator-evidence` tag is this class's alone: a plain `**Gate-ability:** operator` tag keeps its meaning, work the dev agent builds that a person or a script verifies, and nothing reads it as owed evidence. Its **Verification** is a recorded live result (`references/gate.md`, "Operator evidence is a recorded live result"), and its anti-vacuous twin is that a `fail` record, or a record made at another commit, leaves it uncovered. Never leave the operator part inside a ci-machine AC as prose such as "recorded rather than gated": trace classifies a criterion by everything it states, so the unrecorded half holds the whole AC at PARTIAL. The split drops and relaxes nothing. Log each split in `.decision-log.md`.
|
|
28
28
|
|
|
29
29
|
**A decomposed child gets no story file.** Skip this step for a key the readiness check listed in `checks.decomposed_children` when the split is recorded (its split record (`.split-<parent key>.json` beside `sprint-status.yaml`, which `scripts/story_sizing.py --record` writes) names the child, the child's row cites the parent ACs it claims, or `.decision-log.md` names the parent and the ACs each child claims): the child has no story file and must never get one, and its criteria are the parent ACs it claims, in the parent's file (`references/execute.md`, step 1). A listed key with none of these records is a story of its own that reads like a child (above a `done` parent's row, or under a parent whose row was removed): create its file here, and log in `.decision-log.md` that the readiness check never read its criteria. One such key already has its file: a key whose title starts with a single letter or a roman numeral (`2-4-a-new-login-page`) reads one level below its story number, so its own story file named by that number alone (`2-4.md`) or by another title is listed as the parent of a removed row. When the listed parent file is that story's own (`parent_row` is `null`, no other key in the list names the same file, and the file's title and criteria are this key's), rename that file to `<key>.md` instead, in one `mv <that file> <its folder>/<key>.md` call (a filesystem command `acceptEdits` runs without asking on a path inside the project, unlike a git move, which no allow rule covers), then run this step on it as a refinement, and log the rename: the readiness check did read its criteria. A file two listed keys name is a parent's, so rename it for neither.
|
|
30
30
|
|
|
@@ -22,7 +22,7 @@ Preconditions (asserted, not assumed): you are on the Epic branch (`{workflow.ep
|
|
|
22
22
|
|
|
23
23
|
**A third case: a story already committed, green and un-gated.** The routing above sorts stories into two classes — re-drive, and do not re-run — and a session that died between its commit and its gate belongs to neither. Recognise it by artifacts, never by prose: the row carries commits past its baseline marker **and** a `.tests-ran-<story_id>` marker whose `baseline=` line still matches that marker, **and** neither `trace-<story_id>.md` nor `gate-decision-<story_id>.json` exists under `{workflow.trace_output_dir}`. That story owes **evidence and a gate, not implementation**. Re-set the `.current-story` pointer (the pointer half of step 0 only — the baseline marker is still re-read, never rewritten), skip steps 1 through 4, and enter at **step 5**, re-measuring the committed HEAD rather than reading the tests-ran marker's numbers back — at FULL scope, since no step 2 ran this session to inherit from — and once green, rewrite the tests-ran marker fresh (same `baseline=` line, `scope=full`) **even on a green first try**: the inherited marker may predate the scope contract or carry a mid-loop `scope=scoped`, and the gate reads the marker THIS session leaves behind, not the dead one's; then continue into Stage 5 as normal. Trust the artifacts and the `sprint-status.yaml` row, never the story file's own prose: such a file routinely carries a close-out asserting a gate `PASS` that no artifact backs, which is precisely the claim this stage does not get to make. **The discriminator must hold in full.** A tests-ran marker with no commits, or commits with no marker, is a story that died before or during the sweep, and it takes the ordinary loop from step 0.
|
|
24
24
|
|
|
25
|
-
**A fourth case: a closed `operator-evidence-owed` story owes a re-gate, not a re-drive.** Its FAIL gate left `trace-<story_id>.md` and `gate-decision-<story_id>.json` on disk, so the third case never fires for it. When `/ucg-resolve` closes that escalation, it deletes the sidecar, resets the story's turn counter (waiting for operator evidence is not the story's work) and writes `{workflow.implementation_artifacts}/.regate-<story_id>`, holding `sha=` the assessed commit and `decision=` the closed id. The `story` block reports that marker as `regate`: `null` when there is none, otherwise its `state`. While it reads `fresh` (its `sha=` resolves to `HEAD`) and the baseline marker exists, the route is `gate-owed`: enter exactly as the third case does, re-setting the pointer, skipping step 0's turn-ceiling read and steps 1 through 4, re-measuring `HEAD` at step 5 with any runner or harness the escalated cycle built in `.scratch-<story_id>/` (step 0 keeps that directory until the story advances), then re-gating it in Stage 5, whose trace counts the operator's records. When that directory is gone (a run under an earlier rule removed it at step 5, or an operator did), re-author any runner or harness the story's declared gate set needs before step 5 runs. Check it against the gates the escalated cycle's step-5 capture in `evidence-<story_id>/` names, when that folder holds one, so the re-gate measures the gate set that cycle measured. Commit nothing before that gate while step 5's re-measure is green, since a commit past the assessed commit makes the records stale. On a red, take step 5's red route (remediate, write a fresh `.tests-ran-<story_id>` marker, re-commit): that commit moves `HEAD` past the assessed commit, so the records no longer count and the `.regate-<story_id>` marker is stale, and the re-gate at the new `HEAD` escalates `operator-evidence-owed` again at the new commit. A `stale` marker (one naming another commit, or no commit id) routes as if it were absent, normally `mid-story`: a later commit already made the records stale. On either route the gate deletes the marker once this story's verdict is logged (`references/gate.md`, "A logged verdict settles an owed re-gate").
|
|
25
|
+
**A fourth case: a closed `operator-evidence-owed` story owes a re-gate, not a re-drive.** Its FAIL gate left `trace-<story_id>.md` and `gate-decision-<story_id>.json` on disk, so the third case never fires for it. When `/ucg-resolve` closes that escalation, it deletes the sidecar, resets the story's turn counter (waiting for operator evidence is not the story's work) and writes `{workflow.implementation_artifacts}/.regate-<story_id>`, holding `sha=` the assessed commit and `decision=` the closed id. The `story` block reports that marker as `regate`: `null` when there is none, otherwise its `state`. While it reads `fresh` (its `sha=` resolves to `HEAD`) and the baseline marker exists, the route is `gate-owed`: enter exactly as the third case does, re-setting the pointer, skipping step 0's turn-ceiling read and steps 1 through 4, re-measuring `HEAD` at step 5 with any runner or harness the escalated cycle built in `.scratch-<story_id>/` (step 0 keeps that directory until the story advances), then re-gating it in Stage 5, whose trace counts the operator's records and, for a story with a `{workflow.implementation_artifacts}/.moved-<story_id>` record, reads the main tree's story file and the sources that record lists, and whose check that trace scored no moved AC runs before `gate_eval.py` (`references/gate.md`, "After a move, check that trace scored no moved AC"). When that directory is gone (a run under an earlier rule removed it at step 5, or an operator did), re-author any runner or harness the story's declared gate set needs before step 5 runs. Check it against the gates the escalated cycle's step-5 capture in `evidence-<story_id>/` names, when that folder holds one, so the re-gate measures the gate set that cycle measured. Commit nothing before that gate while step 5's re-measure is green, since a commit past the assessed commit makes the records stale. On a red, take step 5's red route (remediate, write a fresh `.tests-ran-<story_id>` marker, re-commit): that commit moves `HEAD` past the assessed commit, so the records no longer count and the `.regate-<story_id>` marker is stale, and the re-gate at the new `HEAD` escalates `operator-evidence-owed` again at the new commit. A `stale` marker (one naming another commit, or no commit id) routes as if it were absent, normally `mid-story`: a later commit already made the records stale. On either route the gate deletes the marker once this story's verdict is logged (`references/gate.md`, "A logged verdict settles an owed re-gate").
|
|
26
26
|
|
|
27
27
|
**A fifth case: a gate stopped between automate and trace.** `preflight_check.py` reads no log, so its `story` block cannot see this case and reads such a story as `gate-owed` or `mid-story`; take this case instead of either, but never instead of `decision-pending`. It takes three forms, each read from the story's `.decision-log.md`:
|
|
28
28
|
|
|
@@ -16,7 +16,7 @@ The gate reads artifacts TEA produces. In **production**, before running the gat
|
|
|
16
16
|
|
|
17
17
|
**When automate backfills, commit the backfill before trace runs.** A test automate writes or changes lands in the main tree, as every assessor's output does ("Their OUTPUTS go home" below), so neither the commit under assessment nor the worktree made from it holds that test, and trace would count a test the commit does not contain. So never run trace, NFR or test review on that commit. While `.current-story` still names this story, log `gate-backfill-base: <sha>` in `.decision-log.md`, naming `HEAD`, the commit automate measured, then commit the backfill the way `references/execute.md` commits a story's paths, in this order:
|
|
18
18
|
|
|
19
|
-
1. Stage the test files this pass's summary (`automation-summary-<story_id>-<sha>.md`, with `<sha>` the `gate-backfill-base:` commit; "Automate's summary records its rows" below) names as written or changed, by explicit path, in one call. `git status --porcelain --untracked-files=all` lists them: the plain `git status --porcelain` lists a new directory instead of the files in it, so a test in a new folder would look absent. A path it lists that the summary does not name is not part of the backfill and stays unstaged. A path the summary names that it lists is staged from the main tree, and the worktree's copy of that file, which automate ran the test's confirming rows against ("A backfill is confirmed in the pass that writes it" below), is not a test written into the worktree: the rebuild discards it. A path the summary names that it does not list has no change in the main tree, so look for the file in the worktree: at that path when the summary gives it under `<tmp-path>`, and otherwise at the same repo-relative path under `<tmp-path>`. When the file is there, automate wrote it into the worktree: copy it to its repo-relative path in the main tree before the rebuild destroys it, and list again. When git still lists no change at that path, or neither tree holds the file, the test the summary names was never written: leave the path out, and log a `gate-gap:` line for the gap that test was written for, as "A reported gap stays open until a later pass measures it closed" below describes, in place of any `unkilled` line logged for that gap. When this step stages nothing, there is no backfill: skip steps 2 through 5, and the commit under assessment stays the `gate-backfill-base:` commit.
|
|
19
|
+
1. Stage the test files this pass's summary (`automation-summary-<story_id>-<sha>.md`, with `<sha>` the `gate-backfill-base:` commit; "Automate's summary records its rows" below) names as written or changed, by explicit path, in one call. `git status --porcelain --untracked-files=all` lists them: the plain `git status --porcelain` lists a new directory instead of the files in it, so a test in a new folder would look absent. A path it lists that the summary does not name is not part of the backfill and stays unstaged. Neither is a test the summary names for a risk or a gap an owned `gate-gap:` line names ("An owned `gate-gap:` line is not this story's to close" below): move it aside, unstaged, into the story's scratch folder, `{workflow.implementation_artifacts}/.scratch-<story_id>/`, at its repo-relative path under that folder, creating the folder when it is gone, which keeps it out of every default source glob (`references/execute.md`, step 0), and log the move in `.decision-log.md`. The gap it was written for is none the cycle backfilled, so the next cycle re-runs no row for it as one ("At most one automate pass finishes per gate cycle" below). A path the summary names that it lists is staged from the main tree, and the worktree's copy of that file, which automate ran the test's confirming rows against ("A backfill is confirmed in the pass that writes it" below), is not a test written into the worktree: the rebuild discards it. A path the summary names that it does not list has no change in the main tree, so look for the file in the worktree: at that path when the summary gives it under `<tmp-path>`, and otherwise at the same repo-relative path under `<tmp-path>`. When the file is there, automate wrote it into the worktree: copy it to its repo-relative path in the main tree before the rebuild destroys it, and list again. When git still lists no change at that path, or neither tree holds the file, the test the summary names was never written: leave the path out, and log a `gate-gap:` line for the gap that test was written for, as "A reported gap stays open until a later pass measures it closed" below describes, in place of any `unkilled` line logged for that gap. When this step stages nothing, there is no backfill: skip steps 2 through 5, and the commit under assessment stays the `gate-backfill-base:` commit.
|
|
20
20
|
2. With them staged, so that a gate reading git's file list sees a new test file, run the full three-axis sweep over the main tree, cache-defeated and printed, as `references/execute.md` step 5 runs it. A red `references/execute.md` step 2 reads as not yet a red (a timeout on a loaded machine, or a red the story's changes do not explain) is read that way here first. That step's second sweep for a red the story's changes do not explain is no fix, so it counts toward neither of the two fixes below, and when it shows no red, this sweep is green. A red that a change to the backfilled test files alone fixes (a lint or format error in one, a tracked-files gate that names one) is fixed there, staged, and re-run, at most twice in this backfill, a fix step 4 makes for a commit hook included. A red that needs a change to any other path, or that is still red after the second fix, is not the gate's to fix: commit nothing, and take "A red backfill sweep ends the cycle at a re-loop" below. A green sweep goes on to step 3, whatever `gate-gap:` lines this cycle has logged.
|
|
21
21
|
3. Once the sweep is green, write a fresh `.tests-ran-<story_id>` marker as step 5 writes one: its `baseline=` line copied from `.baseline-<story_id>`, never re-derived, and `scope=full`.
|
|
22
22
|
4. Run `references/execute.md` step 4's claim check on the commit message, then commit from the message file, in a call separate from the staging call, as step 4 requires. The guard allows this commit as it allows the evidence commit in "Route on the verdict": the story is still current, its marker was written in a prior call, and the index is not empty. A repo commit hook that rejects this commit is read as step 2's red, and `references/execute.md` step 4's own rule for a hook rejection does not apply to it: when a change to the backfilled test files alone answers the hook, make it there, stage it, and go back to step 2, whose sweep runs again before step 3 writes a fresh marker and this step retries the commit, all within step 2's budget of two fixes; when the hook needs a change to any other path, or rejects the commit again after the second fix, commit nothing, and take "A red backfill sweep ends the cycle at a re-loop" below. A `commit-msg` hook that rejects the message alone asks for no file change: Git runs it only once every `pre-commit` job has passed, so reword the message file, run the claim check again and retry the commit with the marker as it stands, re-running no sweep. That retry counts as one of step 2's two fixes, and a rejection after the second still takes "A red backfill sweep ends the cycle at a re-loop" below. A change a `pre-commit` job staged on its way there rides the retry as it would ride an accepted commit. A path the re-run claim check has you stage or edit is a file change, which takes the rule above. A rejection whose output does not show that a `commit-msg` hook made it is read as step 2's red, as above.
|
|
@@ -24,7 +24,7 @@ The gate reads artifacts TEA produces. In **production**, before running the gat
|
|
|
24
24
|
|
|
25
25
|
The new commit is now the commit under assessment: the worktree is rebuilt at it, trace, NFR and test review measure it, and the verdict's `assessed-commit` line names it. The backfill belongs to this gate cycle and is not a re-loop: it writes no `class:` line, spends no re-loop, and never meets the turn-ceiling check that runs before a re-loop, though its sweep spends turns like any wait (`references/execute.md`, "Waiting spends turns").
|
|
26
26
|
|
|
27
|
-
**A red backfill sweep ends the cycle at a re-loop.** A red in step 2's sweep, a commit hook's rejection at step 4 included, that needs a change outside the backfilled test files, or is still red after the second fix, means the backfill has found a defect in the story's code. Of what automate's pass and its backfill produce, it is the one thing that ends a cycle before trace, because a failing sweep is a script's result and not a judgment; a `gate-gap:` line never ends a cycle. A fix made here would land after automate's only pass in this cycle and skip the generalize step, so commit nothing and run no trace, NFR or test review in this cycle. Log this cycle's verdict in `.decision-log.md` as a fenced JSON object holding only `verdict` and `reasons`, `{"verdict": "reloop", "reasons": [...]}`, with one reason naming the red, and beside it an `assessed-commit:` line naming the `gate-backfill-base:` commit, the commit automate measured. That object is the shape `scripts/gate_trail.py` reads as a recorded verdict, and since no trace ran it holds no `gate_status`. It stands in for the `gate_eval.py` result this cycle never produced: the run-status heartbeat's `last_reasons` copies its `reasons` (`references/execute.md`, "Run-status heartbeat"), and an attended run prints the one verdict line "Attended runs also print the verdict, once" (under "Run the gate") describes when this verdict is logged, with `n/a` as the `gate_status`. Then take the `reloop` route under "Route on the verdict" with the red and every open `gate-gap:` line of this cycle that is neither `unmeasured
|
|
27
|
+
**A red backfill sweep ends the cycle at a re-loop.** A red in step 2's sweep, a commit hook's rejection at step 4 included, that needs a change outside the backfilled test files, or is still red after the second fix, means the backfill has found a defect in the story's code. Of what automate's pass and its backfill produce, it is the one thing that ends a cycle before trace, because a failing sweep is a script's result and not a judgment; a `gate-gap:` line never ends a cycle. A fix made here would land after automate's only pass in this cycle and skip the generalize step, so commit nothing and run no trace, NFR or test review in this cycle. Log this cycle's verdict in `.decision-log.md` as a fenced JSON object holding only `verdict` and `reasons`, `{"verdict": "reloop", "reasons": [...]}`, with one reason naming the red, and beside it an `assessed-commit:` line naming the `gate-backfill-base:` commit, the commit automate measured. That object is the shape `scripts/gate_trail.py` reads as a recorded verdict, and since no trace ran it holds no `gate_status`. It stands in for the `gate_eval.py` result this cycle never produced: the run-status heartbeat's `last_reasons` copies its `reasons` (`references/execute.md`, "Run-status heartbeat"), and an attended run prints the one verdict line "Attended runs also print the verdict, once" (under "Run the gate") describes when this verdict is logged, with `n/a` as the `gate_status`. Then take the `reloop` route under "Route on the verdict" with the red and every open `gate-gap:` line of this cycle that is neither `unmeasured`, `unkilled` nor owned as its findings, turn-ceiling check included. The backfilled files stay staged in the main tree, so the re-loop's `references/execute.md` step 4 commits them with its fix, and the next cycle's automate scope, the diff from the `assessed-commit:` commit, holds the fix and those tests. When the turn-ceiling check turns that re-loop into `escalate`, the backfilled files stay staged too, and the escalation sidecar's `decision_needed` names each of them as staged for this story. The cycle after this re-loop runs the full gate, never a delta gate: this cycle wrote no trace, NFR or test review artifact for a delta gate to re-check. A resume that finds this verdict logged and `HEAD` still at its `assessed-commit:` commit takes this re-loop where the log stopped (`references/execute.md`'s fifth resume case).
|
|
28
28
|
|
|
29
29
|
**A resume after a backfill does not run automate again.** When this story's `.decision-log.md` holds a `gate-backfill-base:` or a `gate-backfill:` line with no verdict logged after it (when it holds both for this cycle, the later one decides which bullet applies), `references/execute.md`'s fifth resume case sends the resume here, and it goes on from where the cycle stopped:
|
|
30
30
|
|
|
@@ -32,18 +32,18 @@ The new commit is now the commit under assessment: the worktree is rebuilt at it
|
|
|
32
32
|
- After `gate-backfill-base: <sha>`, when `git rev-parse HEAD` prints `<sha>`, the backfill is not committed yet: start again at step 1 above. When `git rev-parse HEAD~1` prints `<sha>`, it is committed: start at step 5.
|
|
33
33
|
- At any other `HEAD`, the story is not in this case: take the route `preflight_check.py` reports.
|
|
34
34
|
|
|
35
|
-
**At most one automate pass finishes per gate cycle, and a backfill never restarts the cycle.** After the backfill commit the same cycle goes on, on the new commit, and automate does not run in it again, so every cycle reaches a verdict whatever automate finds: trace's, or the re-loop a red backfill sweep takes. Automate runs a second time in one cycle only when a resume finds the first pass's summary unfinished, or finds none at all ("A summary already at that name is this cycle's own" below). A later cycle of the story exists only because a verdict sent it back to be gated again, and in that cycle, whether the re-gate is full or delta, automate's scope is the diff from the previous gate's `assessed-commit` line to the commit under assessment, plus the mutation rows behind the previous cycle's backfilled gaps and behind every open `gate-gap:` line:
|
|
35
|
+
**At most one automate pass finishes per gate cycle, and a backfill never restarts the cycle.** After the backfill commit the same cycle goes on, on the new commit, and automate does not run in it again, so every cycle reaches a verdict whatever automate finds: trace's, or the re-loop a red backfill sweep takes. Automate runs a second time in one cycle only when a resume finds the first pass's summary unfinished, or finds none at all ("A summary already at that name is this cycle's own" below). A later cycle of the story exists only because a verdict sent it back to be gated again, and in that cycle, whether the re-gate is full or delta, automate's scope is the diff from the previous gate's `assessed-commit` line to the commit under assessment, plus the mutation rows behind the previous cycle's backfilled gaps and behind every open `gate-gap:` line that is not owned ("An owned `gate-gap:` line is not this story's to close" below):
|
|
36
36
|
|
|
37
37
|
- It searches for new gaps only in code that diff changes. Code outside the diff was in an earlier cycle's scope already.
|
|
38
|
-
- It re-runs, at the commit under assessment, whatever the diff covers, the mutation rows behind each gap the previous cycle backfilled (the `scripts/mutate_check.py` table rows recorded in the summary named for that cycle's `gate-backfill-base:` commit) and behind each open `gate-gap:` line (the rows recorded in the summary the line names). For an `unmeasured` line ("A risk automate names on a P0 criterion" below), and for a line whose rows cannot run ("A reported gap stays open until a later pass measures it closed" below says which cannot), it measures the gap afresh instead and writes new rows for it. A row that reads `GREEN` is a gap again, and "Automate backfills every gap it finds on a P0 criterion" below says whether automate backfills it or reports it, except a row behind an open `unkilled` line, which keeps that line open and gets no new test ("A backfill is confirmed in the pass that writes it" below). A re-loop sent to close a reported gap often adds only a test, which leaves the gap's code outside the diff, so without this re-run no gate would measure whether that test closes the gap.
|
|
39
|
-
- When that diff is empty, automate still runs, for the open `gate-gap:` lines
|
|
38
|
+
- It re-runs, at the commit under assessment, whatever the diff covers, the mutation rows behind each gap the previous cycle backfilled (the `scripts/mutate_check.py` table rows recorded in the summary named for that cycle's `gate-backfill-base:` commit), never a gap whose test step 1 of "When automate backfills" moved aside for an owned line, and behind each open `gate-gap:` line that is not owned (the rows recorded in the summary the line names). For an `unmeasured` line ("A risk automate names on a P0 criterion" below), and for a line whose rows cannot run ("A reported gap stays open until a later pass measures it closed" below says which cannot), it measures the gap afresh instead and writes new rows for it. A row that reads `GREEN` is a gap again, and "Automate backfills every gap it finds on a P0 criterion" below says whether automate backfills it or reports it, except a row behind an open `unkilled` line, which keeps that line open and gets no new test ("A backfill is confirmed in the pass that writes it" below). A re-loop sent to close a reported gap often adds only a test, which leaves the gap's code outside the diff, so without this re-run no gate would measure whether that test closes the gap.
|
|
39
|
+
- When that diff is empty, automate still runs, for the open `gate-gap:` lines that are not owned and nothing else, when any of them names a commit other than the commit under assessment. Otherwise skip automate and the rebuild, and carry the open lines unchanged, `unmeasured` and owned ones included. A cap-only re-gate, and the re-gate a closed `operator-evidence-owed` escalation owes, measure the commit the previous gate measured, so automate has nothing left to look at there unless an open line that is not owned names another commit, and the gate is still the full instrument. Each automate pass leaves every open line that is not owned naming the commit that pass measured, so no cycle runs automate at a commit an earlier cycle's pass has measured, and an owned line, which keeps the commit it names, never makes automate run.
|
|
40
40
|
- When the previous verdict record has no `assessed-commit` line, automate runs in full, as in a story's first cycle (the same fallback the delta profile takes).
|
|
41
41
|
|
|
42
42
|
(Measured on one story, where each backfill started a new cycle: two cycles of about an hour each ended at automate with a new backfilled test, and a third reached a verdict only once its automate could not add tests. The second gap sat in code the first automate pass had already searched and named as a risk, outside the diff between the two cycles, which held only the first backfill's tests. A re-run of the previous backfill's mutation table alone would leave a re-loop's own new code, the code automate exists to cover, with no automate pass at all, and the diff alone would leave a gap a re-loop was sent to close unmeasured, so a later cycle takes both.)
|
|
43
43
|
|
|
44
|
-
**Automate backfills every gap it finds on a P0 criterion.** In any cycle, whatever its scope, and even while the story's live results file holds a record at the commit under assessment, a gap automate finds on a P0 criterion gets a test: the backfill is committed before trace ("When automate backfills" above), so TEA's trace counts the test and `gate_eval.py` decides. A gap on any other criterion it backfills only when the gap's code is in this cycle's search scope (the code that landed during Execute in a story's first cycle, the diff in a later one) and the story's live results file holds no record at the commit under assessment. Otherwise it reports the gap rather than backfills it: it names the gap in its summary, with its mutation evidence, and adds no test. So only a gap on a criterion other than P0 is ever reported rather than backfilled. A gap behind an open `unkilled` line already has its test, so no pass writes it another while its rows can run ("A backfill is confirmed in the pass that writes it" below). A P0 backfill made while a live record sits at the commit under assessment moves that commit past the record, and that needs no rule of its own: a `pass` counts only at the commit trace measures ("Operator evidence is a recorded live result" below), so the gate at the backfill commit owes the AC again through the routes that already owe one (the `operator-evidence-owed` escalation on a FAIL, a ledger row on `advance` or `defer`), as `references/execute.md`'s fourth resume case already says of the commit a red makes past the assessed commit. For a gap on another criterion that cost is not worth paying, which is why automate reports it instead.
|
|
44
|
+
**Automate backfills every gap it finds on a P0 criterion.** In any cycle, whatever its scope, and even while the story's live results file holds a record at the commit under assessment, a gap automate finds on a P0 criterion gets a test: the backfill is committed before trace ("When automate backfills" above), so TEA's trace counts the test and `gate_eval.py` decides. A gap on any other criterion it backfills only when the gap's code is in this cycle's search scope (the code that landed during Execute in a story's first cycle, the diff in a later one) and the story's live results file holds no record at the commit under assessment. Otherwise it reports the gap rather than backfills it: it names the gap in its summary, with its mutation evidence, and adds no test. So only a gap on a criterion other than P0 is ever reported rather than backfilled, and a gap on a P0 criterion goes without a backfill on purpose only when an owned line names it, which automate leaves alone ("An owned `gate-gap:` line is not this story's to close" below). A gap behind an open `unkilled` line already has its test, so no pass writes it another while its rows can run ("A backfill is confirmed in the pass that writes it" below). A P0 backfill made while a live record sits at the commit under assessment moves that commit past the record, and that needs no rule of its own: a `pass` counts only at the commit trace measures ("Operator evidence is a recorded live result" below), so the gate at the backfill commit owes the AC again through the routes that already owe one (the `operator-evidence-owed` escalation on a FAIL, a ledger row on `advance` or `defer`), as `references/execute.md`'s fourth resume case already says of the commit a red makes past the assessed commit. For a gap on another criterion that cost is not worth paying, which is why automate reports it instead.
|
|
45
45
|
|
|
46
|
-
**A risk automate names on a P0 criterion is measured in the same pass.** The pass is the only one its cycle finishes, so a risk it names in its summary and leaves as prose (an assertion that a mutant failing some other way would also satisfy, say) would reach the verdict unmeasured. When you delegate automate, tell it to measure each risk it names on a P0 criterion before it finishes, with a mutation row for each failure arm the criterion names, its own backfilled tests included, and to backfill each gap that measuring confirms, as it backfills every gap on a P0 criterion. A named P0 risk it still leaves unmeasured is logged as a `gate-gap:` line that says `unmeasured` in place of a summary, and carried: the story's next automate pass measures it, and backfills the gap when a row confirms one. An `unmeasured` line is not a finding, since no mutant has run for it yet, and like every `gate-gap:` line it never changes the route the verdict takes.
|
|
46
|
+
**A risk automate names on a P0 criterion is measured in the same pass.** The pass is the only one its cycle finishes, so a risk it names in its summary and leaves as prose (an assertion that a mutant failing some other way would also satisfy, say) would reach the verdict unmeasured. When you delegate automate, tell it to measure each risk it names on a P0 criterion before it finishes, with a mutation row for each failure arm the criterion names, its own backfilled tests included, and to backfill each gap that measuring confirms, as it backfills every gap on a P0 criterion. A named P0 risk it still leaves unmeasured is logged as a `gate-gap:` line that says `unmeasured` in place of a summary, and carried: the story's next automate pass measures it, and backfills the gap when a row confirms one. An `unmeasured` line is not a finding, since no mutant has run for it yet, and like every `gate-gap:` line it never changes the route the verdict takes. A risk an owned line names is the exception: automate does not measure it, and its line stays as it was logged ("An owned `gate-gap:` line is not this story's to close" below).
|
|
47
47
|
|
|
48
48
|
**A backfill is confirmed in the pass that writes it.** When you delegate automate, tell it to write each test it backfills to its repo-relative path in the main tree, copy that file unchanged to the same path in the gate worktree, where its mutation rows run, and re-run there, with the test in place, the rows behind the gap the test was written for, making any later edit to the test in the main tree and copying it again. Tell it, too, to give each row it re-runs there a `command` that runs the backfilled test's file, so that the row's own run sees the test: a row written before the test may select only the tests that existed then, and the second run with the full sweep's command (the second bullet below) can be refused in the worktree, where a tracked-files check sees the copy as a file git does not track. Read each row as "A reported gap stays open until a later pass measures it closed" below reads one. When every row reads killed, the test closed the gap. When a row still reads `GREEN` with the test in place, the test ran and did not kill that row's mutant, which may be an equivalent one that no test can tell from the code: log an `unkilled` `gate-gap:` line for the gap, naming the summary that records its rows and the test that did not kill them. The test is still committed with the backfill. No later pass writes another test for an `unkilled` line while its rows can run: each later pass re-runs them, the line closes when they read killed, a row that reads `GREEN` keeps it open, and a row that cannot run (`STALE` once the code it mutates has moved) has the gap measured afresh, as for any line. Like an `unmeasured` line, an `unkilled` line is not a finding, and it never changes the route the verdict takes.
|
|
49
49
|
|
|
@@ -51,18 +51,20 @@ The new commit is now the commit under assessment: the worktree is rebuilt at it
|
|
|
51
51
|
|
|
52
52
|
**A summary already at that name is this cycle's own.** No two cycles run automate at one commit ("When that diff is empty", above), so a summary at that name was written by this cycle's own pass, in a session that ended before this cycle's verdict was logged, and TEA adds each step's output to a summary that already exists rather than starting a new one. So before you delegate automate, look for it, and right before each time you delegate it, log `gate-automate: <sha>` in `.decision-log.md`, with `<sha>` the commit it measures, the one its summary's name carries: a session that ends while automate runs, or after it finishes but before anything is committed, then leaves a record of where its cycle stopped, and `references/execute.md`'s fifth resume case sends a resume that finds that line for `HEAD`, with no verdict and no backfill line after it, back to this paragraph instead of re-driving the story. When none is there, delegate automate. When it is there and complete (its frontmatter's `lastStep` reads `step-04-validate-and-summarize`, the last step of TEA's automate), that pass finished: do not run automate again. Go on to step 2 of the list at the top of this section with that summary as this pass's, logging only the `gate-gap:` lines this cycle's entry does not hold yet, and leave the dead session's worktree in place until the rebuild removes it, so that step 1 of "When automate backfills" can look in it for the tests the summary names. When it is there but not complete, that pass stopped part way: move it aside to `automation-summary-<story_id>-<sha>-stale-<n>.md` in the same folder, with `<n>` the first number from 1 that no file there carries yet, log the move in `.decision-log.md`, and only then delegate automate.
|
|
53
53
|
|
|
54
|
-
**A reported gap stays open until a later pass measures it closed.** Once automate has finished, log in this cycle's `.decision-log.md` entry one `gate-gap:` line for each gap it reported, for each risk on a P0 criterion it left unmeasured, for each gap it could not measure because its rows could not run even afresh, and for each backfill its confirmation left `unkilled`, beside any line step 1 of "When automate backfills" logs for a test that was never written. Each line names the criterion, its priority, the commit automate measured and the summary that recorded its rows, or `unmeasured` in place of the summary for a risk it did not measure or a gap it could not, and an `unkilled` line also says `unkilled` and names the test that did not kill its rows. Every later cycle of the story copies the previous cycle's open `gate-gap:` lines into its own entry, whatever route the previous verdict took: once its automate has finished, or before trace when automate does not run. Whether a line is open is read off `scripts/mutate_check.py`'s classification of each row behind it:
|
|
54
|
+
**A reported gap stays open until a later pass measures it closed.** Once automate has finished, log in this cycle's `.decision-log.md` entry one `gate-gap:` line for each gap it reported, for each risk on a P0 criterion it left unmeasured, for each gap it could not measure because its rows could not run even afresh, and for each backfill its confirmation left `unkilled`, beside any line step 1 of "When automate backfills" logs for a test that was never written. Each line names the criterion, its priority, the commit automate measured and the summary that recorded its rows, or `unmeasured` in place of the summary for a risk it did not measure or a gap it could not, and an `unkilled` line also says `unkilled` and names the test that did not kill its rows. An owned line ends with its `owned-by` marker ("An owned `gate-gap:` line is not this story's to close" below). Every later cycle of the story copies the previous cycle's open `gate-gap:` lines into its own entry, whatever route the previous verdict took: once its automate has finished, or before trace when automate does not run. Whether a line is open is read off `scripts/mutate_check.py`'s classification of each row behind it:
|
|
55
55
|
|
|
56
56
|
- A row reads killed when it is `KILLED`, `KILLED+others`, or `MISATTRIBUTED` with the detail that its `expect_red` never appears beside a failure token. In each, something the row's command runs failed under the mutant, and a test a re-loop added under a name other than the row's `expect_red` reads `MISATTRIBUTED` that way.
|
|
57
57
|
- A row that reads `GREEN` is still a gap unless it reads killed when re-run once more with its `command` replaced by the test command of the story's full sweep that covers its `file`, cache-defeated, as step 2 of "When automate backfills" runs it: a row's own command can select only the tests that existed when the row was written, so it misses a killing test a re-loop added in another file. When the second run reads killed, the row reads killed. When it reads `GREEN`, or cannot run (the next bullet), the row is still a gap, read by its first run: a second run that cannot run is no reason to measure the gap afresh. Automate makes that second run in the same pass, for each row it runs that reads `GREEN`, so when you delegate it, tell it to, and to record both results beside the row in its summary.
|
|
58
58
|
- A row cannot run when it reads `STALE`, `NOT UNIQUE` or `UNDECODABLE` (its file or its anchor no longer holds, so nothing was mutated), or `MISATTRIBUTED` with the detail that the output's own count says 0 failed (a collection or harness error, not a test: a mutant reads this way when pytest then cannot collect a test module or class or load a conftest.py, or jest, vitest or Node's test runner cannot load a test file, since no test in that file or class runs, or Node's test runner cannot run its global setup, since no test runs at all, or when a Node test file's process exits non-zero after the row's test passes or is skipped, or reports that a test generated asynchronous activity after it ended, whatever test that names, and no other test fails; a Node test file's process that exits or crashes during a test, before Node reports the row's test, still reads as a kill, since the row's `expect_red` then never appears beside a failure token, though it can read 0 failed when a test whose name holds the row's `expect_red` passes or is skipped in that file before the process dies or in a file Node lists before it, when that file reports such asynchronous activity before the process dies, or when the last thing Node prints for that file is an uncaught error that a child process printed; Node prints a test file whose process exits non-zero after all its tests pass as it prints one whose process dies during a test, so such a file whose lines report no pass of a test whose name holds the row's `expect_red` reads as that death, and the row as a kill though no test went red, and when you delegate automate, tell it to write each Node row's `command` to run only the test file that holds the row's test, though a row's second run, through the full sweep, can still read so; a module that calls `process.exit` while a Node test file loads, or that kills its process then by filling the heap or by a signal, stops the file loading, yet Node reports it as it reports a process that exits or crashes during the file's first test, so it reads as that kill does, and when you delegate automate, tell it to write no replacement that makes a module exit, fill the heap or raise a signal while a Node test file loads; and in output that interleaves the lines of several runs with no task-runner prefix to tell them apart, a Node test file that cannot load, or whose process exits non-zero after the row's test passes, can still read as a kill), or `MISATTRIBUTED` with the detail that no test ran (a harness error too: the runner stopped before any test because it could not start its web server, its port taken, by a server an earlier run left up among others, or its start command failing, whether or not the mutant made it fail), or when `mutate_check.py` refuses its table with exit 2 (no row ran). A row whose command runs through a server the gate set reuses, and whose mutated file that server would load, cannot run either, even when the worktree's own processes load that file too, whatever `mutate_check.py` prints, and the two bullets above do not read it ("A server the gate set reuses does not follow the commit", under "Measure the commit, not the working tree"): that server serves the tree it was started from and never the mutant, so whatever the run reads, its `GREEN` is no gap and its red no kill, no test is backfilled for it, no `unkilled` line is logged from it and no line closes on it. It measures nothing, so its line stays open, and the pass measures that gap afresh, once in the pass: when a row it writes for the gap then cannot run either, it measures that gap no further, and logs the gap's line as `unmeasured`, in place of a summary, for the story's next automate pass to measure.
|
|
59
59
|
|
|
60
|
-
A line closes when an automate pass at a later commit than the one the line names finds every row behind it reading killed: the line's own rows, re-run, or for an `unmeasured` line or one whose rows cannot run, the rows that pass writes when it measures the gap afresh, any test it backfills for them included. That pass's cycle leaves the line out. A carried line that pass does not close is logged afresh in place of the copied line, naming this commit and this pass's summary, or `unmeasured` while the risk still is or the gap's rows still cannot run, or `unkilled`, naming its test, while a test backfilled for the gap has not killed its rows. A `gate-gap:` line never changes the route the verdict takes, and each route treats the open lines this way:
|
|
60
|
+
A line closes when an automate pass at a later commit than the one the line names finds every row behind it reading killed: the line's own rows, re-run, or for an `unmeasured` line or one whose rows cannot run, the rows that pass writes when it measures the gap afresh, any test it backfills for them included. That pass's cycle leaves the line out. A carried line that is not owned and that pass does not close is logged afresh in place of the copied line, naming this commit and this pass's summary, or `unmeasured` while the risk still is or the gap's rows still cannot run, or `unkilled`, naming its test, while a test backfilled for the gap has not killed its rows. A `gate-gap:` line never changes the route the verdict takes, and each route treats the open lines this way:
|
|
61
61
|
|
|
62
|
-
- On a `reloop` that is not cap-only, each open line that is neither `unmeasured
|
|
63
|
-
- On `advance` or `defer`, each open line becomes a ledger row with `source` `gate` and `severity` `med`, whatever its criterion's priority. An `unmeasured` line's row says the risk or the gap is unmeasured, and an `unkilled` line's row says the backfilled test did not kill the mutant, which may be equivalent. No such row is a gate FAIL or a critical finding, whatever its criterion's priority, so it belongs on the ledger and never re-loops or escalates the story, and neither the INVARIANT under "Route on the verdict" nor the ledger schema's `severity` bar sends it anywhere else: a FAIL is the `gate_status` that `gate_eval.py` reads from TEA's trace, which no `gate-gap:` line changes, and a critical finding is one a code review or the NFR assessment rates critical, while a `gate-gap:` line records automate's mutation measurement, an `unmeasured` one a measurement nobody has made yet and an `unkilled` one a test that did not kill its mutant, not a defect anyone found.
|
|
62
|
+
- On a `reloop` that is not cap-only, each open line that is neither `unmeasured`, `unkilled` nor owned is one of the findings the generalize step reads. A cap-only `reloop` has no finding to generalize and an `escalate` only stops, so neither of them takes one.
|
|
63
|
+
- On `advance` or `defer`, each open line becomes a ledger row with `source` `gate` and `severity` `med`, whatever its criterion's priority, except an owned line on a moved AC, which rides that AC's row. An `unmeasured` line's row says the risk or the gap is unmeasured, and an `unkilled` line's row says the backfilled test did not kill the mutant, which may be equivalent. An owned line's row also names its owner and the ruling in `reason`, and its `suggested_action` names the owner. No such row is a gate FAIL or a critical finding, whatever its criterion's priority, so it belongs on the ledger and never re-loops or escalates the story, and neither the INVARIANT under "Route on the verdict" nor the ledger schema's `severity` bar sends it anywhere else: a FAIL is the `gate_status` that `gate_eval.py` reads from TEA's trace, which no `gate-gap:` line changes, and a critical finding is one a code review or the NFR assessment rates critical, while a `gate-gap:` line records automate's mutation measurement, an `unmeasured` one a measurement nobody has made yet and an `unkilled` one a test that did not kill its mutant, not a defect anyone found.
|
|
64
64
|
- Whatever the route, a line stays open, so the story's next gate cycle, when there is one, copies it.
|
|
65
65
|
|
|
66
|
+
**An owned `gate-gap:` line is not this story's to close.** A written operator ruling can assign a risk or a gap a `gate-gap:` line names, on a criterion the story keeps, to another story of this Epic or to a row of the deferred-work ledger. Only then, log the line with `owned-by <owner> per <ruling>` at its end, after every field above: `<owner>` is the story key of a story of this Epic not yet `done`, and any other owner, a `done` story or one outside this Epic included, takes the ledger-row form, `"<Epic heading> <ledger id>"` (a ledger id is unique only within its heading), and `<ruling>` cites the ruling as `Operator notes <entry>` or `decision <id>`, with `<id>` the `.decisions.json` id `/ucg-resolve` recorded it under. The ruling names this story, the criterion or the risk, and the owner; one that names less is not this ruling, and neither is a general or standing delegation, wherever it is written, the bar the clause move in `references/execute.md`'s "Two stories green only together" sets. A ruling that assigns a whole criterion elsewhere is a move of the AC, which takes the moved-AC form ("A moved AC leaves the story's list", under "Route on the verdict"), not this marker alone: whoever applies that form, for any written ruling (the `/ucg-resolve` close, you in an attended run, or the re-read at each cycle's start below, which takes the latest ruling for each AC as it does for each line), writes the story's `.moved-<story_id>` record and logs each open line of the criterion the move retired again with the marker, and those lines reach the ledger inside the moved AC's row and not as rows of their own. An Operator notes entry given before launch (`references/ingest-and-scope.md`, "Operator notes") counts from the story's first cycle: when you delegate automate there, name the risk or the gap the entry assigns and tell automate neither to measure nor to backfill it, then log its line `unmeasured`, with the marker. At the start of each gate cycle, before automate's scope and the empty-diff trigger are decided, read the rulings again (the Operator notes entries, and the `answer` and the `history` of each entry of `{workflow.implementation_artifacts}/.decisions.json` whose `action` is `close`, the store `/ucg-resolve` records every answer in), taking only the latest ruling for each line, an entry's `answer` over its `history` and a later entry over an earlier one, since a later return or reassignment replaces an earlier ruling, and log each open line whose marker that ruling changes in this cycle's entry, with the marker it now gives or with none, in place of the copy the carry makes. Keep the line one line of prose: never a heading, with no JSON object and no `|` in it. An owned line is outside automate's scope in every cycle: automate re-runs no row behind it, neither measures nor backfills the risk or the gap it names, and writes no row and no test for it, so when you delegate automate, name each owned line and tell it so. A test its summary names for one anyway is never staged: step 1 of "When automate backfills" above moves it aside and logs the move. An owned line never closes in this story: each cycle copies it unchanged and never logs it afresh, and only a later written ruling takes the marker off, by logging the line again without it, which puts the line back in automate's scope. It is none of a re-loop's findings, so it gets no `class:` line, and it reaches the ledger once, as the row an `advance` or a `defer` gives every open line (the bullets above), except an owned line on a moved AC, which rides that AC's row, with `source` `gate`, `severity` `med` and `story` this story, its `reason` the line's state (`unmeasured`, `unkilled`, or the summary that recorded its rows) with the owner and the ruling, and its `suggested_action` naming the owner. The owner takes the work up from that row, never from this line: no story's gate reads another story's `gate-gap:` lines.
|
|
67
|
+
|
|
66
68
|
**Operator evidence is a recorded live result, in either profile.** An AC tagged `**Gate-ability:** operator-evidence` (`references/define-done.md`, step 1) is proven only through TEA's live verification results, which trace reads as `live_results_input` and which TEA lets a person write. For a story with such an AC the file is `{workflow.trace_output_dir}/live-verification-results-<story_id>.json`; name it (absolute, main tree) when you delegate trace. Each record carries `id` `<story_id>-LIVE-<NNN>`, `requirement_id` `AC-<n>`, `status`, `evidence` and `source_sha`, under `schema_version` `0.1.0`. A `pass` counts only at the commit trace measures. Write no record the operator did not give you, and name the operator as `producer`. TEA caps a requirement covered only this way at CONCERNS, so the story defers. That is the honest outcome, never a gap to close with a test that restates the record.
|
|
67
69
|
|
|
68
70
|
## Measure the commit, not the working tree
|
|
@@ -136,7 +138,13 @@ The score **must** carry the `/100` denominator: a bare `score: 89` matches noth
|
|
|
136
138
|
|
|
137
139
|
## Run the gate
|
|
138
140
|
|
|
139
|
-
**Name the per-story paths trace writes, in either profile.** TEA 1.27.2's `bmad-testarch-trace` writes fixed, flat paths at the TEA `test_artifacts` root (`traceability-matrix.md`, `e2e-trace-summary.json`, `gate-decision.json`) and does not read `trace_output`. Left alone, its output neither lands in `{workflow.trace_output_dir}` nor names a story, so the gate below resolves nothing there, and each story's trace overwrites the last one's. When you delegate trace for a story, name the paths it must write in `{workflow.trace_output_dir}`: `trace-<story_id>.md` (frontmatter `workflowType: testarch-trace` and `gateDecisionFile: gate-decision-<story_id>.json`), `gate-decision-<story_id>.json` and `e2e-trace-summary-<story_id>.json`, with `<story_id>` the sprint-status key copied verbatim. If it writes the flat files anyway, move them to those names before anything else reads or overwrites them. Ask for nothing beyond those names, those frontmatter keys
|
|
141
|
+
**Name the per-story paths trace writes, in either profile.** TEA 1.27.2's `bmad-testarch-trace` writes fixed, flat paths at the TEA `test_artifacts` root (`traceability-matrix.md`, `e2e-trace-summary.json`, `gate-decision.json`) and does not read `trace_output`. Left alone, its output neither lands in `{workflow.trace_output_dir}` nor names a story, so the gate below resolves nothing there, and each story's trace overwrites the last one's. When you delegate trace for a story, name the paths it must write in `{workflow.trace_output_dir}`: `trace-<story_id>.md` (frontmatter `workflowType: testarch-trace` and `gateDecisionFile: gate-decision-<story_id>.json`), `gate-decision-<story_id>.json` and `e2e-trace-summary-<story_id>.json`, with `<story_id>` the sprint-status key copied verbatim. If it writes the flat files anyway, move them to those names before anything else reads or overwrites them. Ask for nothing beyond those names, those frontmatter keys, the live results file trace reads for a story with an operator-evidence AC ("Operator evidence is a recorded live result" above) and, on every gate cycle of a story with a `.moved-<story_id>` record, the story file trace reads the story's criteria from and every `oracle=` path that record lists, each named absolute in the main tree: the move, and an operator's edit of an oracle source, are edits no commit carries until the story advances ("A moved AC leaves the story's list", under "Route on the verdict"), so the worktree's copies still state the AC. Name the files and nothing of the move, which the files themselves state: the provenance stays `tea` only while TEA's own steps computed the content.
|
|
142
|
+
|
|
143
|
+
**After a move, check that trace scored no moved AC, in either profile.** On every gate cycle of a story with a `{workflow.implementation_artifacts}/.moved-<story_id>` record ("A moved AC leaves the story's list", under "Route on the verdict"), the re-gate the move owes first, move the story's previous `trace-<story_id>.md`, `gate-decision-<story_id>.json` and `e2e-trace-summary-<story_id>.json` aside, all three in one step, before you delegate trace, into `{workflow.implementation_artifacts}/evidence-<story_id>/`, each under its own name with `-cycle-<n>` before its extension, `<n>` the first number from 1 that no file there carries yet, so nothing there is overwritten, and a trace that writes nothing, or only some of its files, leaves the check nothing of an earlier cycle to read, and it exits 2. A resume that finds some of the three moved aside and some still in place moves the rest aside the same way before trace. A resume that finds them moved aside still routes `gate-owed`: through `references/execute.md`'s fourth resume case while the `.regate-<story_id>` marker is fresh, and otherwise through its third, which finds neither the trace report nor the gate decision. Once trace has written its files and before `gate_eval.py` runs, run the command below, with `--commit` the full sha `git rev-parse <the commit under assessment>` prints and one `--moved AC-<n>` for each line of the record that is not an `oracle=` line, taken from the `AC-<n>` that opens it. Run it whatever `gate_status` trace wrote. Trace scores every criterion it resolves, from the story, the epic, the PRD or the test design, and has no status for a moved one, so an AC that left the story's list can still be a row of its coverage matrix, and a row left at `NONE` fails the gate, or holds it at CONCERNS, over work this story no longer owes. Exit 0 means no row is a moved AC: run `gate_eval.py` as below and take the route its verdict gives, the ordinary routes unchanged, since what it scores is then only criteria the story still owns. Exit 1 means a row is one, and exit 2 that the trace files could not be read, are not this trace's or measured another commit, since the script reads them as `operator_owed.py` does and fails closed. On either, run no `gate_eval.py`. Log the script's JSON and, as this cycle's verdict, a fenced JSON object holding only `verdict` and `reasons`, `{"verdict": "escalate", "reasons": [...]}`, with the script's `reasons`, as a red backfill sweep logs its re-loop, with the provenance and `assessed-commit:` lines "Record who authored the artifacts the verdict rests on" below asks for. Then take `escalate` with `kind` `operator-evidence-owed` again: `source` is the trace report's path, `evidence` is the script's first reason, and `decision_needed` opens with "The moved-AC check refused (exit 1)" or "The moved-AC check refused (exit 2)", then names this run's id, the refusal's number for this story (one past the refusals of this story the decision log already records), `assessed_commit` (the script's, or the commit under assessment when it printed none), each moved AC, and what must change before a re-gate can pass, so each refusal is a decision of its own and never one `/ucg-resolve` has closed already. On exit 1, that is each source in the script's `oracle_sources` that still states a moved AC for this story (when it names none, the story file, the epic, the PRD and the test design), which the operator edits in the main tree so it no longer does, or a written return of the AC to the story. Resolve a relative source against `{project-root}`, and name one outside the work tree `git rev-parse --show-toplevel` prints, or one that names no file, as a source the operator must clear themselves, since only a path inside the work tree is recorded and committed. On exit 2, it is that the trace files are missing, not this trace's or of another commit, so the re-gate runs trace again, and `decision_needed` says whether an exit-2 refusal of this story at this commit came before. `/ucg-resolve` keys its close on that opening phrase. Write no `class:` line and take no `reloop` route, since no re-loop changes what trace reads.
|
|
144
|
+
|
|
145
|
+
```
|
|
146
|
+
uv run {skill-root}/scripts/moved_ac_check.py --trace-output {workflow.trace_output_dir} --story <story_id> --commit <the full sha git rev-parse prints for the commit under assessment> --moved AC-<n>
|
|
147
|
+
```
|
|
140
148
|
|
|
141
149
|
Production:
|
|
142
150
|
|
|
@@ -175,11 +183,11 @@ Do not recompute TEA thresholds or re-judge `gate_status` — read it as given.
|
|
|
175
183
|
|
|
176
184
|
## Route on the verdict
|
|
177
185
|
|
|
178
|
-
**A logged verdict settles an owed re-gate.** Once this story's verdict is recorded in `.decision-log.md` ("Run the gate" above), and before you take any route below, delete `{workflow.implementation_artifacts}/.regate-<story_id>` if it exists. `/ucg-resolve` wrote it on closing the story's `operator-evidence-owed` escalation, to route a resume to this re-gate (`references/execute.md`'s resume rule), and a logged verdict leaves no re-gate owed.
|
|
186
|
+
**A logged verdict settles an owed re-gate.** Once this story's verdict is recorded in `.decision-log.md` ("Run the gate" above), and before you take any route below, delete `{workflow.implementation_artifacts}/.regate-<story_id>` if it exists. `/ucg-resolve` wrote it on closing the story's `operator-evidence-owed` escalation, to route a resume to this re-gate (`references/execute.md`'s resume rule), and a logged verdict leaves no re-gate owed. A refusal of the check after a move ("After a move, check that trace scored no moved AC", under "Run the gate") is logged as this cycle's verdict too, so it settles the marker the same way, and `/ucg-resolve` writes a fresh one when it closes the escalation that refusal takes.
|
|
179
187
|
|
|
180
188
|
- **`advance`** (gate_status PASS or WAIVED) → the story passes (an operator-evidence AC still owed its evidence is parked on the ledger first, as `defer` below says). **Sync the story's row in `sprint-status.yaml` to `done` before moving on**: set `development_status[<story_id>]` to `done`, preserving the file's comments and key order, and log the transition in `.decision-log.md`. This write is the run's own, not a sub-skill's: `ucg-dev-story` leaves the row at `review`, and `bmad-code-review` (production only; it does not run under `--light` at all) writes `done` **only** when nothing was left as an action item, so a story that legitimately `defer`s a non-critical finding is left at `in-progress`. Every downstream reader keys on `done`: Stage 1's in-scope rule (`references/ingest-and-scope.md` rule 3), the Epic-level trigger in this very sentence, and `scripts/drive_epic.py`'s progress check. So an advanced story whose row still says `review` is an advanced story the next reader re-drives, and an Epic whose last story advanced without this write never satisfies the Epic-level condition below. Sync it here, at the one place the authoritative verdict is known. In the same write, move the Epic's own `epic-<id>` row from `backlog` to `in-progress` when it still reads `backlog` (never backward, and only when the key exists): `bmad-build` makes that move when a story starts, but the story skills this run delegates to skip it when handed an explicit story key, so without it a finished Epic still reads `backlog`. If the Stop hook left a turn-budget marker for this story (`{workflow.implementation_artifacts}/.escalation-<story_id>.md`) and its count is still at or past the ceiling, do **not** promote it: log the sizing note `references/execute.md` describes under "Promoting the turn-budget marker", and leave the marker in place.
|
|
181
189
|
|
|
182
|
-
**Commit the gate's evidence before the next story starts.** The `done` sync, this story's trace and gate files (`trace-<id>.md`, `gate-decision-<id>.json`, `e2e-trace-summary-<id>.json`, and in production its per-story NFR assessment and test review), its live results file when it has an operator-evidence AC, this gate's ledger rows and the decision log (with the first story, also Stage 3's test-design output) are tracked state written after Execute's commit-at-green, and any `sweep` row this story's sweeps appended (`references/execute.md` step 2) is tracked state no commit of the story carries, since those stage only its source and test paths. While `.current-story` still names this story, stage those of these paths that exist and are not ignored, by explicit path, in one call, and commit them in a second, before the next story's step 0. Before the commit call, write its message with Write to `{workflow.implementation_artifacts}/.scratch-<story_id>/message.txt`, Reading the file first when one is there (step 4's claim check leaves one, as a backfill or an earlier session may) and creating the folder when it is gone, and commit from that file (`git commit -F`), as the story's own commits do. Never put the message in the Bash command. After a double-quoted `-m`, or in a heredoc whose delimiter is unquoted, bash runs a backticked name or a `$(...)` in it as a command while it builds the arguments, and the commit still exits 0 with the text that is left. A quoted heredoc keeps the text, but the guard reads its lines as commands (a line that opens with `Git runs` is read as a git command it does not know, which it counts as a commit and a push), and the `$(cat <<'EOF' ...)` form is a command substitution, which a harness that statically analyses Bash strings refuses (`references/execute.md` step 2). Once the commit is made, the escalation below is written, or nothing is staged, remove `.scratch-<story_id>/`: this is where `references/execute.md` step 0 has it removed, and the next story's step 0 must not find it. A resume that finds `.scratch-<story_id>/` for a story whose row is already `done` removes it before the next story's step 0. For the story whose advance makes every story `done`, make this commit after the Epic-level gate below instead, so it also carries that gate's writes. The commit carries no source or test path, so the story's existing tests-ran marker covers it: reuse that marker as it stands and never rewrite it for this commit. A repo commit hook that rejects this commit is not `references/execute.md` step 4's case, whose rule for a hook rejection binds the story's own commits. Make a change the hook asks for only in a file the gate itself writes for this commit, `sprint-status.yaml` (its `done` sync) or `.decision-log.md`, then stage it and retry, with the marker as it stands, at most twice for this commit. When the hook asks for a change to any other path, such as a file TEA wrote (an edit there would make its provenance false) or a ledger row (the ledger schema below never rewrites one), or rejects the commit again after the second fix, commit nothing: log the rejection in `.decision-log.md`, write the typed escalation sidecar `escalation-<story_id>.json` (shape in `references/execute.md`, under "Escalation sidecar") with `kind` `evidence-commit-rejected` and a `decision_needed` that names the hook, its message, the paths left uncommitted and this run's id (so a rejection on a later run is a new decision, never one `/ucg-resolve` has closed already), write `.evidence-owed-<story_id>` beside it, listing those paths one per line, and stop before the next story's step 0, as the `escalate` route below stops. The story keeps the verdict it advanced on (`advance`, or `defer`, whose route advances anyway), and its evidence stays staged for the commit a resume makes. A `commit-msg` hook that rejects only this commit's message asks for no file change: the message is the gate's own, as the `done` sync and the decision log are, so reword `message.txt` and retry from it (`-F`), with the marker as it stands, and count that retry as one of this commit's two fixes, after which a rejection still takes the escalation above. Git runs that hook only once every `pre-commit` job has passed, so a change a `pre-commit` job staged on its way there rides the retry as it would ride an accepted commit. A rejection whose output does not show that a `commit-msg` hook made it keeps the rule above. Automate's backfill is not among its paths, because it was committed before trace ran ("When automate backfills", under "Backfill the gate evidence"). If it must carry an executable path all the same (a source or test file left uncommitted when trace ran, which this gate therefore never measured: log why), re-run the sweep and write a fresh marker per `references/execute.md` step 5 first. Then refresh `.commit-<story_id>` from `HEAD`, so this story's range in the gate trail includes the evidence commit. A path the project ignores is logged and left out, never forced in with `git add -f`; if nothing is staged, make no commit and leave `.commit-<story_id>` as it is. A resume that finds an advanced story's evidence still uncommitted makes this commit for it before the next story's step 0: point `.current-story` back at that story (its tests-ran marker still passes the guard), write the message file and commit from it as above, and refresh its `.commit-<story_id>`. While that story's `evidence-commit-rejected` escalation is still open, the resume commits nothing and stops on it instead (`references/execute.md`'s resume rule says how it reads one); once `/ucg-resolve` has closed it, the resume makes the commit, under the hook rule above. Once the commit is made, remove `.evidence-owed-<story_id>`: `/ucg-resolve` deletes the sidecar when it closes the escalation, so the marker is what tells a resume, and Stage 1's already-done short-circuit, that the commit is still owed. A resume that finds the marker with the sidecar gone and none of the paths it lists staged or different from `HEAD` (`git diff --quiet HEAD -- <paths>` exits 0 for them) finds the evidence committed already, by the operator or by a run that stopped between its commit and this removal: log the commit that holds it and remove the marker, making no commit. On the story whose advance makes every story `done`, this commit follows the Epic-level gate, so the `epic-<id>` row that gate's `advance` wrote stays staged with the rest, and the run ends `blocked` on that escalation all the same, because its evidence is not committed (`references/finalize.md`). While `.evidence-owed-<story_id>` is on disk, Stage 1's already-done short-circuit does not end that Epic's resume (`references/ingest-and-scope.md`, rule 4), nor does `scripts/drive_epic.py` report the Epic `all-done`: the resume waits on the escalation while it is open, makes the commit once it is closed, and then goes to Finalize with the Epic-level verdict the log holds, without running that gate again. Then move to the next story.
|
|
190
|
+
**Commit the gate's evidence before the next story starts.** The `done` sync, this story's trace and gate files (`trace-<id>.md`, `gate-decision-<id>.json`, `e2e-trace-summary-<id>.json`, and in production its per-story NFR assessment and test review), its live results file when it has an operator-evidence AC or moved one out, its story file, its ATDD checklist and every `oracle=` path its `.moved-<story_id>` lists, when it has that record ("A moved AC leaves the story's list" below), this gate's ledger rows and the decision log (with the first story, also Stage 3's test-design output) are tracked state written after Execute's commit-at-green, and any `sweep` row this story's sweeps appended (`references/execute.md` step 2) is tracked state no commit of the story carries, since those stage only its source and test paths. While `.current-story` still names this story, stage those of these paths that exist and are not ignored, by explicit path, in one call, and commit them in a second, before the next story's step 0. Before the commit call, write its message with Write to `{workflow.implementation_artifacts}/.scratch-<story_id>/message.txt`, Reading the file first when one is there (step 4's claim check leaves one, as a backfill or an earlier session may) and creating the folder when it is gone, and commit from that file (`git commit -F`), as the story's own commits do. Never put the message in the Bash command. After a double-quoted `-m`, or in a heredoc whose delimiter is unquoted, bash runs a backticked name or a `$(...)` in it as a command while it builds the arguments, and the commit still exits 0 with the text that is left. A quoted heredoc keeps the text, but the guard reads its lines as commands (a line that opens with `Git runs` is read as a git command it does not know, which it counts as a commit and a push), and the `$(cat <<'EOF' ...)` form is a command substitution, which a harness that statically analyses Bash strings refuses (`references/execute.md` step 2). Once the commit is made, the escalation below is written, or nothing is staged, remove `.scratch-<story_id>/`: this is where `references/execute.md` step 0 has it removed, and the next story's step 0 must not find it. A resume that finds `.scratch-<story_id>/` for a story whose row is already `done` removes it before the next story's step 0. For the story whose advance makes every story `done`, make this commit after the Epic-level gate below instead, so it also carries that gate's writes. The commit carries no source or test path, so the story's existing tests-ran marker covers it: reuse that marker as it stands and never rewrite it for this commit. A repo commit hook that rejects this commit is not `references/execute.md` step 4's case, whose rule for a hook rejection binds the story's own commits. Make a change the hook asks for only in a file the gate itself writes for this commit, `sprint-status.yaml` (its `done` sync) or `.decision-log.md`, then stage it and retry, with the marker as it stands, at most twice for this commit. When the hook asks for a change to any other path, such as a file TEA wrote (an edit there would make its provenance false) or a ledger row (the ledger schema below never rewrites one), or rejects the commit again after the second fix, commit nothing: log the rejection in `.decision-log.md`, write the typed escalation sidecar `escalation-<story_id>.json` (shape in `references/execute.md`, under "Escalation sidecar") with `kind` `evidence-commit-rejected` and a `decision_needed` that names the hook, its message, the paths left uncommitted and this run's id (so a rejection on a later run is a new decision, never one `/ucg-resolve` has closed already), write `.evidence-owed-<story_id>` beside it, listing those paths one per line, and stop before the next story's step 0, as the `escalate` route below stops. The story keeps the verdict it advanced on (`advance`, or `defer`, whose route advances anyway), and its evidence stays staged for the commit a resume makes. A `commit-msg` hook that rejects only this commit's message asks for no file change: the message is the gate's own, as the `done` sync and the decision log are, so reword `message.txt` and retry from it (`-F`), with the marker as it stands, and count that retry as one of this commit's two fixes, after which a rejection still takes the escalation above. Git runs that hook only once every `pre-commit` job has passed, so a change a `pre-commit` job staged on its way there rides the retry as it would ride an accepted commit. A rejection whose output does not show that a `commit-msg` hook made it keeps the rule above. Automate's backfill is not among its paths, because it was committed before trace ran ("When automate backfills", under "Backfill the gate evidence"). If it must carry an executable path all the same (a source or test file left uncommitted when trace ran, which this gate therefore never measured: log why), re-run the sweep and write a fresh marker per `references/execute.md` step 5 first. Then refresh `.commit-<story_id>` from `HEAD`, so this story's range in the gate trail includes the evidence commit. Once that commit is made, by this route or by a resume (the resume after a commit hook's escalation included), or nothing is staged for it, or a resume finds the gate's files committed already, remove `{workflow.implementation_artifacts}/.moved-<story_id>`: the story's committed file and its ledger rows now hold each move. A path the project ignores is logged and left out, never forced in with `git add -f`; if nothing is staged, make no commit and leave `.commit-<story_id>` as it is. A resume that finds an advanced story's evidence still uncommitted makes this commit for it before the next story's step 0: point `.current-story` back at that story (its tests-ran marker still passes the guard), write the message file and commit from it as above, and refresh its `.commit-<story_id>`. While that story's `evidence-commit-rejected` escalation is still open, the resume commits nothing and stops on it instead (`references/execute.md`'s resume rule says how it reads one); once `/ucg-resolve` has closed it, the resume makes the commit, under the hook rule above. Once the commit is made, remove `.evidence-owed-<story_id>`: `/ucg-resolve` deletes the sidecar when it closes the escalation, so the marker is what tells a resume, and Stage 1's already-done short-circuit, that the commit is still owed. A resume that finds the marker with the sidecar gone and none of the paths it lists staged or different from `HEAD` (`git diff --quiet HEAD -- <paths>` exits 0 for them) finds the evidence committed already, by the operator or by a run that stopped between its commit and this removal: log the commit that holds it and remove the marker, making no commit. On the story whose advance makes every story `done`, this commit follows the Epic-level gate, so the `epic-<id>` row that gate's `advance` wrote stays staged with the rest, and the run ends `blocked` on that escalation all the same, because its evidence is not committed (`references/finalize.md`). While `.evidence-owed-<story_id>` is on disk, Stage 1's already-done short-circuit does not end that Epic's resume (`references/ingest-and-scope.md`, rule 4), nor does `scripts/drive_epic.py` report the Epic `all-done`: the resume waits on the escalation while it is open, makes the commit once it is closed, and then goes to Finalize with the Epic-level verdict the log holds, without running that gate again. Then move to the next story.
|
|
183
191
|
|
|
184
192
|
When **every story of the Epic is `done`**, run the Epic-level gate: pass the epic's own id via `--story`, **add `--epic-level`**, and read its `gate_status` **only**. TEA produces NFR and test-review **per story, not per epic**, so there is no epic-level aggregate to AND, and every story's own production gate already ANDed its signals before reaching `done`. With `--rollup` the epic id is the sprint-status grouping key and the verdict is a computation over the per-story record; only in the omit-`--rollup` fallback is `--story` resolved against an epic-named artifact (scoped per the `--story` note above) and the roll-up a pure trace read. When the Epic-level gate returns `advance`, or `defer` (whose route advances anyway), set the `epic-<id>` row to `done` (when the key exists), preserving comments and key order, log it, and carry it in the last story's evidence commit. BMAD Method 6.12 has no other writer of an epic row's `done`, and `bmad-retrospective` writes only the `epic-<id>-retrospective` row. Never write it on any other route: the work bound, `partial-complete`, a handoff, a block or an escalation.
|
|
185
193
|
|
|
@@ -195,21 +203,23 @@ uv run {skill-root}/scripts/gate_eval.py --trace-output {workflow.trace_output_d
|
|
|
195
203
|
|
|
196
204
|
- **`defer`** (gate_status CONCERNS, or non-critical code-review / NFR findings that did not flip the gate) → append the open items to the ledger at `{workflow.deferred_work_path}` using the schema below, then **advance** anyway. The Epic keeps moving; the parked work is visible. **On `advance` or `defer`, an operator-evidence AC still owed its evidence is parked too**: an uncovered P2 or P3 criterion may leave the gate at PASS or CONCERNS, so the story can move without it. It still counts toward TEA's overall coverage, so it can also FAIL the gate, and a FAIL goes to `operator_owed.py` (the next bullet), never to this ledger. It is owed while its live results file holds no `pass` record for it at the commit the gate measured, or while TEA holds it below FULL with only the records it counted there; append a row with `source` `gate` naming the AC, the evidence it owes (for a partly recorded AC, the ids of the records TEA counted and the part still owed) and that file.
|
|
197
205
|
|
|
198
|
-
- **A FAIL only operator evidence can change is not a re-loop.** On every gate_status FAIL, first run the command below (for a decomposed child, `--story-file` is its parent's story file). Under `--light`, pass `--profile light` and no `--nfr` or `--test-review`; in either profile, pass `--cycle-profile` exactly as the gate took it. On `owed: true`, take `escalate` with `kind` `operator-evidence-owed`: `source` is the trace report's path, `decision_needed` names the ACs in `operator`, what each owes, `assessed_commit`, the live results file to record into, and that `/ucg-resolve` closes it once the records are there
|
|
206
|
+
- **A FAIL only operator evidence can change is not a re-loop.** On every gate_status FAIL, first run the command below (for a decomposed child, `--story-file` is its parent's story file). Under `--light`, pass `--profile light` and no `--nfr` or `--test-review`; in either profile, pass `--cycle-profile` exactly as the gate took it. On `owed: true`, take `escalate` with `kind` `operator-evidence-owed`: `source` is the trace report's path, `decision_needed` names the ACs in `operator`, what each owes, `assessed_commit`, the live results file to record into, and that `/ucg-resolve` closes it once the records are there, once the operator narrows the AC in writing, or once the story file carries the operator's written move of the AC out of the story ("A moved AC leaves the story's list" below), and `evidence` is TEA's gap line for the first of those ACs. Write no `class:` line, run no `bmad-correct-course`, spend no re-loop, and commit nothing after the assessed commit, since a later commit makes the record stale. Attended, the operator may record it now, or move the AC in that form, which you record as the `/ucg-resolve` close would, logging their in-session ruling as a named Operator notes entry in `.decision-log.md` and citing it as `Operator notes <entry>`: the form on disk before the re-gate, the owner as an owned line names one, the AC's open `gate-gap:` lines logged again as owned, and the `.moved-<story_id>` record; then re-gate the same commit in-session. For a decomposed child's AC, record `defer`, propose the re-plan and stop there, re-gating nothing. Otherwise `/ucg-resolve` closes it with the records on disk, or with the move in the story file and its `.moved-<story_id>` record, and writes `.regate-<story_id>`, and the resume re-gates that commit rather than re-driving the story (`references/execute.md`'s resume rule). An open operator-evidence AC that already holds a `pass` record TEA counted at the assessed commit is still owed: TEA held it below FULL because the record covers only part of the criterion, and no re-loop can record the rest or narrow the AC, so the script lists it in `operator` and in `partial`, which maps it to the ids of the records TEA counted. For each AC in `partial`, `decision_needed` names the record ids TEA already counted, says they cover only part of the criterion, and asks the operator to record the rest or to narrow or move the AC in writing. On `owed: false`, the `reloop` route below stands; log the script's `reasons`.
|
|
199
207
|
|
|
200
208
|
```
|
|
201
209
|
uv run {skill-root}/scripts/operator_owed.py --trace-output {workflow.trace_output_dir} --story <story_id> --story-file <story file> --profile production --nfr {nfr-assessment.md} --test-review {test-review.md} --tests-ran {workflow.implementation_artifacts}/.tests-ran-<story_id>
|
|
202
210
|
```
|
|
203
211
|
|
|
212
|
+
**A moved AC leaves the story's list.** A written operator ruling can move an AC out of the story, to another story or to a row of the deferred-work ledger, and the story file then carries the move in this form and no other. The AC's numbered item leaves `## Acceptance Criteria`, and a note under an H2 heading of its own after that section, one that does not say "acceptance criteria", names the retired number, the owner, as an owned `gate-gap:` line names one (a story of this Epic not yet `done`, or else a ledger row as `"<Epic heading> <ledger id>"`), and the ruling (`Operator notes <entry>` or `decision <id>`, with `<id>` its `.decisions.json` id). The note starts with no `<n>.` and holds no `**Gate-ability:**` text: the parsers that read the section fold an unnumbered line into the AC above it and read a tag there as that AC's, so a note left in the list, or one that quotes a tag, changes another AC. The number is retired: no other AC is renumbered and no later AC takes it (`references/define-done.md`, step 1), since live records key on `AC-<n>`. The AC also leaves the `acceptanceCriteria` list of the story's ATDD checklist; it had no scaffold, so no test moves. These edits are the operator's own, or yours at their written direction, and no commit carries them before the re-gate, since nothing may land after the assessed commit: the commit that carries the gate's files once the story advances stages them ("Commit the gate's evidence before the next story starts", above). Whoever applies this form, for any written ruling (the `/ucg-resolve` close, you at the operator's written direction in an attended run, or you for an Operator notes entry or an answer to another escalation that the re-read at each cycle's start finds), writes `{workflow.implementation_artifacts}/.moved-<story_id>`, one line per moved AC opened by its `AC-<n>`, `AC-<n> to <owner> per <ruling>`, adding to any lines already there, and logs a `moved:` line in `.decision-log.md` that records the move for a reader and that no step reads. The record also takes an `oracle=<absolute path>` line for each oracle source the operator edited, with the move or after a refusal of the check: a relative one is resolved against `{project-root}`, and only a path inside the work tree `git rev-parse --show-toplevel` prints is recorded. Every later gate cycle of the story, the commit that carries the gate's files and a resume read the moved set from the `AC-<n>` that opens each line that is not an `oracle=` line, and the sources from the `oracle=` lines, of this record alone. A written return of a moved AC to the story is applied by the `/ucg-resolve` close, or by you at the operator's written direction in an attended run: it deletes the AC's line from the record, and the record once no AC line is left, so no check runs then; puts the numbered item back under its old number; takes its entry out of the H2 note; restores it to the checklist's `acceptanceCriteria`; and logs its open `gate-gap:` lines again without the `owned-by` marker. The advance route removes the record once the commit that carries the gate's files is made ("Commit the gate's evidence before the next story starts", above). Whoever records the move also logs each open `gate-gap:` line on the moved AC again with `owned-by` the same owner per the same ruling ("An owned `gate-gap:` line is not this story's to close", under "Backfill the gate evidence"), so automate leaves it alone. On `advance` or `defer`, append one ledger row per moved AC, with `source` `gate`, `severity` `med`, `story` this story, a `reason` naming the AC, its owner, the ruling and each owned `gate-gap:` line on it, which then writes no row of its own, and a `suggested_action` naming the owner, written once, as an owned line's row is. A decomposed child's AC is not moved in this form: it lives in its parent's story file, which is never edited and which every sibling reads, so retiring it there would retire it for each of them; `/ucg-resolve` records `defer` for such a move and proposes a re-plan of the split instead. WAIVED is no route for a moved AC: TEA keeps a waived criterion in every count, and the AC is no longer this story's. Every gate cycle of a story with that record names the main-tree story file and its sources to trace and checks trace's matrix before `gate_eval.py` runs ("After a move, check that trace scored no moved AC", under "Run the gate"), so an AC the operator moved never reaches the `reloop` route.
|
|
213
|
+
|
|
204
214
|
- **`reloop`** (gate_status FAIL, or a production signal downgraded an advance, or a sweep-scope or delta-profile cap) → **generalize before fixing.** Before `bmad-correct-course` writes any fix, take the verdict's `reasons` and the review's findings and, for each finding: (1) name, in ONE line, the defect CLASS the finding is an instance of; (2) state whether any earlier review of this story or this Epic already found that class: the previous gate artifacts and `{workflow.implementation_artifacts}/.class-ledger-<epic_id>.md` are the evidence, and they are already on disk, so this is a read, not a judgment call; (3) where the class is a repeat, the fix must close the CLASS (every sibling instance the same seam admits, not the one instance the reviewer demonstrated), and the log records why the class-level fix was not taken when the class was first named. A measured run spent five of six reviews re-finding one class ("an assertion that cannot go red") one seam over each time, with the class-killing move available from day one; the generalize step is what makes cycle N read cycle N-1 instead of re-buying it. Record one `class:` line per finding in this re-loop's `.decision-log.md` entry, and append any class not already present to `.class-ledger-<epic_id>.md` (create on first use; named for the Epic, so a later Epic starts a fresh ledger while this Epic's survives across its own invocations for exactly the resume case; one line per class, `- <class, one line> (first named <story_id>, cycle <n>)`; **append-only**: Execute step 1 reads it before every story and re-loop, so a class named once is warned about everywhere after, including across stories, which no per-story artifact can do). **A score that does not move indicts the FIX STRATEGY, not the code:** when two consecutive gate evaluations of this story have moved `review_score` by less than 3 (production; under `--light` no review score exists, so the trigger there is two consecutive re-loops whose findings name any class already in the ledger), the generalize answers above are mandatory *in writing* before any further fix: a re-loop whose findings all name the same class as the previous re-loop's is a re-loop about to be spent badly. **A cap-only re-loop skips the ceremony.** When a cap alone produced the verdict (the `reasons` show only the sweep-scope cap, the delta-profile cap, or both, with no gate FAIL and no failed production signal), there is no finding to generalize and nothing to diagnose: re-run the full three-axis sweep where the sweep cap fired and refresh the marker with `scope=full`, then re-run the gate as the FULL instrument (**dropping `--cycle-profile`**, because the delta cap is lifted by the flag's absence and a re-gate that keeps it caps again forever), writing no `class:` lines, appending nothing to `.class-ledger-<epic_id>.md` and no row to `{workflow.deferred_work_path}` save the `sweep` row its re-sweep owes for a red the story's changes do not explain (`references/execute.md` step 2), and skipping `bmad-correct-course` entirely; such evaluations do not count toward the flat-score tripwire, and a capped CONCERNS keeps its findings for the next gate's own `defer` route: parking them is that gate's job, not this re-loop's. A cap-only re-gate re-measures without re-driving the story, so the turn-ceiling rule below never turns it into `escalate`. Otherwise, run `bmad-correct-course` to diagnose and adjust, then re-run the story (back through Execute, `references/execute.md`), **within the remaining turn budget**. Re-run the gate after. If the re-loop would exceed `{workflow.max_turns_per_story}`, treat it as `escalate` instead; the count is the `turns` value the Stop hook records in `{workflow.implementation_artifacts}/.budget-<story_id>.json`. When the hook left a turn-budget marker for this story, that escalation's sidecar is the marker's promotion (`references/execute.md`, "Promoting the turn-budget marker").
|
|
205
215
|
|
|
206
|
-
**A red backfill sweep and the `gate-gap:` lines take this route their own way.** A re-loop that ended its cycle before trace ("A red backfill sweep ends the cycle at a re-loop", under "Backfill the gate evidence") takes the full route above, with its red and every open `gate-gap:` line of its cycle that is neither `unmeasured
|
|
216
|
+
**A red backfill sweep and the `gate-gap:` lines take this route their own way.** A re-loop that ended its cycle before trace ("A red backfill sweep ends the cycle at a re-loop", under "Backfill the gate evidence") takes the full route above, with its red and every open `gate-gap:` line of its cycle that is neither `unmeasured`, `unkilled` nor owned as its findings, and the gate after it is a full one. On a `reloop` from `gate_eval.py` that is not cap-only, each open `gate-gap:` line that is neither `unmeasured`, `unkilled` nor owned is among the findings the generalize step reads, though none of them is a reason for the verdict. An open `gate-gap:` line in a cycle whose re-loop is cap-only is not one of that re-loop's findings, so it gets no `class:` line and no ledger row here: like a capped CONCERNS's findings, it goes to the next gate, by its `gate-gap:` line ("A reported gap stays open until a later pass measures it closed", under the same section). No `gate-gap:` line, on any criterion, sends a story to this route: only the verdict or a red backfill sweep does.
|
|
207
217
|
|
|
208
|
-
- **`escalate`** (gate_status NOT_EVALUATED, meaning the gate could not be read; budget exhausted on a FAIL;
|
|
218
|
+
- **`escalate`** (gate_status NOT_EVALUATED, meaning the gate could not be read; budget exhausted on a FAIL; a FAIL only operator evidence can change, above; or any gate cycle of a story with a `.moved-<story_id>` record whose check exits 1 or 2, under "Run the gate") → **stop.** Do not advance, do not defer the failing item. Record the reason and the verdict JSON in `.decision-log.md`, and write the typed escalation sidecar `{workflow.implementation_artifacts}/escalation-<story_id>.json` (shape in `references/execute.md`, under "Escalation sidecar") so the pending decision is readable without the transcript. In an attended run, surface the blocker to the user. In headless, this is a `blocked` outcome: emit the JSON in Stage 6 (`references/finalize.md`).
|
|
209
219
|
|
|
210
220
|
**INVARIANT: a P0/critical FAIL never defers.** A failing gate (or a P0/P1/overall threshold miss) is `reloop` or `escalate`, never `defer`. Only non-gate-blocking work (CONCERNS, non-critical findings, parked decisions, and the rows open `gate-gap:` lines become) is allowed onto the ledger. If you find yourself about to write a FAIL or a critical finding to the ledger, you are violating the gate: re-loop within budget or escalate instead. A row from a `gate-gap:` line is neither a FAIL nor a critical finding, whatever its criterion's priority, so it belongs on the ledger and never re-loops or escalates the story.
|
|
211
221
|
|
|
212
|
-
An open `gate-gap:` line is neither a FAIL nor a critical finding, whatever its criterion's priority, so the ledger row it becomes on `advance` or `defer` keeps this rule ("A reported gap stays open until a later pass measures it closed", under "Backfill the gate evidence", says why). A gap automate finds on a P0 criterion is backfilled rather than parked ("Automate backfills every gap it finds on a P0 criterion", under the same section), and `gate_eval.py` decides on the test, so a `gate-gap:` line on a P0 criterion is only ever a risk or a gap nobody has measured yet, a backfilled test that was never written,
|
|
222
|
+
An open `gate-gap:` line is neither a FAIL nor a critical finding, whatever its criterion's priority, so the ledger row it becomes on `advance` or `defer` keeps this rule ("A reported gap stays open until a later pass measures it closed", under "Backfill the gate evidence", says why). A gap automate finds on a P0 criterion is backfilled rather than parked ("Automate backfills every gap it finds on a P0 criterion", under the same section), and `gate_eval.py` decides on the test, so a `gate-gap:` line on a P0 criterion is only ever a risk or a gap nobody has measured yet, a backfilled test that was never written, a backfilled test that did not kill its mutant, or a risk or a gap a written ruling assigned to another owner ("An owned `gate-gap:` line is not this story's to close", under the same section).
|
|
213
223
|
|
|
214
224
|
If any orchestrated sub-skill blocks on interactive input mid-run, treat it as `escalate` for that story — write the typed escalation sidecar `{workflow.implementation_artifacts}/escalation-<story_id>.json` and stop; do not answer its prompt blind.
|
|
215
225
|
|
|
@@ -225,7 +235,7 @@ Append to `{workflow.deferred_work_path}` (create on first use). One heading per
|
|
|
225
235
|
| d1 | gate | low | <story-id> | <why parked> | <what to do later> | open |
|
|
226
236
|
```
|
|
227
237
|
|
|
228
|
-
- `source` ∈ `gate` (CONCERNS, an operator-evidence AC's evidence still owed, or an open `gate-gap:` line: a coverage gap automate reported rather than backfilled, a risk on a P0 criterion it left unmeasured, a gap whose rows could not run, a backfilled test that was never written,
|
|
238
|
+
- `source` ∈ `gate` (CONCERNS, an operator-evidence AC's evidence still owed, an AC the operator moved out of the story, or an open `gate-gap:` line: a coverage gap automate reported rather than backfilled, a risk on a P0 criterion it left unmeasured, a gap whose rows could not run, a backfilled test that was never written, a backfilled test that did not kill its mutant, or a risk or a gap a written ruling assigned to another owner), `code-review` (non-critical finding), `nfr` (non-FAIL finding), `decision` (parked decision), `sweep` (a non-critical finding: a case outside the story's changes that went red in one sweep and green in the next at the same tree, `references/execute.md` step 2).
|
|
229
239
|
- `severity` ∈ `low`, `med`, `high`. A `high` that maps to a gate FAIL or a P0/critical finding does **not** belong here: re-loop or escalate it. A row from an open `gate-gap:` line is always `med`: whatever its criterion's priority, it is neither a gate FAIL nor a P0/critical finding, so it belongs here and never re-loops or escalates the story.
|
|
230
240
|
- `id` is unique within the Epic heading (append `d2`, `d3`, … ). Note that the id is unique only WITHIN its heading, so `d1` under two Epics is two different decisions; a project-wide reader must key them by more than the raw column (`/ucg-resolve` mints an id for exactly this reason).
|
|
231
241
|
- `status` ∈ `open`, `resolved`. **This gate only ever appends rows, and never rewrites one** — but the column is not frozen forever: `/ucg-resolve` flips `open` to `resolved` when an operator closes a parked decision, and that single transition is the one sanctioned edit to an existing row. Everything else about a row is immutable once written: never restate its `id`, `source`, `severity`, `story`, `reason` or `suggested_action`, because the ledger is the durable account of what was parked and rewriting it loses the history the Epic close-out reads.
|
|
@@ -0,0 +1,233 @@
|
|
|
1
|
+
#!/usr/bin/env -S uv run --script
|
|
2
|
+
# /// script
|
|
3
|
+
# requires-python = ">=3.11"
|
|
4
|
+
# dependencies = []
|
|
5
|
+
# ///
|
|
6
|
+
"""Say whether TEA's trace still scores an AC the operator moved out of the story.
|
|
7
|
+
|
|
8
|
+
An `operator-evidence-owed` escalation can close on the operator's written move
|
|
9
|
+
of an AC out of the story: its numbered item leaves `## Acceptance Criteria`
|
|
10
|
+
for a section of its own, and its number is retired. TEA's trace has no status
|
|
11
|
+
for a moved criterion. It scores every criterion it resolves, from the story,
|
|
12
|
+
the epic, the PRD or the test design, so an AC that left the story's list can
|
|
13
|
+
still be a row of its coverage matrix, scored NONE, and that row fails the
|
|
14
|
+
gate, or holds it at CONCERNS, over work the story no longer owes.
|
|
15
|
+
`references/gate.md` runs this check on every gate cycle after a move, once
|
|
16
|
+
trace has written its files and before `gate_eval.py` runs, whatever the gate
|
|
17
|
+
status and in either profile. Only a pass lets the gate run; a row or an
|
|
18
|
+
unreadable matrix escalates `operator-evidence-owed` again, never a re-loop.
|
|
19
|
+
|
|
20
|
+
gate_eval.py is not changed and no verdict is added. The matrix is read through
|
|
21
|
+
operator_owed.read_matrix, with the same cross-checks: the story's own
|
|
22
|
+
`trace-<id>.md`, an absolute `tempCoverageMatrixPath` naming a PHASE_1_COMPLETE
|
|
23
|
+
matrix whose P0-P3 counts and requirement recount agree with the story's own
|
|
24
|
+
`e2e-trace-summary-<id>.json`, a summary that names the commit it measured
|
|
25
|
+
(`source_sha`), and a matrix `source_sha`, when the matrix has one, naming the
|
|
26
|
+
same commit. This script then requires that commit to be `--commit`, the
|
|
27
|
+
commit under assessment, so a previous gate's trace files never read as this
|
|
28
|
+
one's, and every requirement id to be a string. Anything it cannot read is a
|
|
29
|
+
refusal (FAIL-CLOSED).
|
|
30
|
+
|
|
31
|
+
A row is a moved AC's when an AC token anywhere in its id names a number
|
|
32
|
+
`--moved` names: `AC` not preceded by a letter or digit, then optional blanks
|
|
33
|
+
around at most one `-`, `_`, `:`, `#`, `.` or dash (U+2010 to U+2015), then the
|
|
34
|
+
number, not followed by another digit, in any case and with leading zeros
|
|
35
|
+
(`AC-7`, `AC7`, `AC 7`, `AC: 7`, `AC#7`, `AC.7`, `AC-7.1`, `AC-7: login` are AC
|
|
36
|
+
7's; `AC-7/AC-8` names both; `AC-70` and `AC-17` name neither). A token that a
|
|
37
|
+
story-key-like prefix (`<digits>-<digits>...-`) directly precedes counts only
|
|
38
|
+
when that prefix is `--story`, or the run of digit groups `--story` opens with:
|
|
39
|
+
`11-5-AC-7` is a sibling story's AC 7, never this one's. `--moved` takes
|
|
40
|
+
`AC-<n>` stripped, in any case and with leading zeros, and each AC once;
|
|
41
|
+
`--commit` takes the full sha `git rev-parse` prints, 40 or 64 hex digits, and
|
|
42
|
+
never a prefix, which could match another commit's matrix. Prints one JSON object, a usage error or a failure the
|
|
43
|
+
check did not foresee included: `moved` (the ids asked about, as `AC-<n>`),
|
|
44
|
+
`rows` (each matrix row that is a moved AC: its `id`, `priority` and
|
|
45
|
+
`coverage`), `matrix` (the matrix path, or null), `oracle_sources` (the matrix's
|
|
46
|
+
`oracle.sources`, where trace says it read its criteria, or an empty list),
|
|
47
|
+
`assessed_commit` and `reasons`.
|
|
48
|
+
|
|
49
|
+
Exit 0: no row is a moved AC. Exit 1: a row is. Exit 2: the trace files could
|
|
50
|
+
not be read, are not this trace's or measured another commit, an argument
|
|
51
|
+
error, or any other failure of the check.
|
|
52
|
+
|
|
53
|
+
uv run moved_ac_check.py --trace-output DIR --story 11-6 --commit SHA \\
|
|
54
|
+
--moved AC-7 [--moved AC-9 ...]
|
|
55
|
+
"""
|
|
56
|
+
|
|
57
|
+
from __future__ import annotations
|
|
58
|
+
|
|
59
|
+
import argparse
|
|
60
|
+
import json
|
|
61
|
+
import re
|
|
62
|
+
import sys
|
|
63
|
+
from pathlib import Path
|
|
64
|
+
|
|
65
|
+
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
|
66
|
+
# A failed import is reported by main() as a refusal with its JSON, never as an
|
|
67
|
+
# exit 1, the code that means a row was found.
|
|
68
|
+
try:
|
|
69
|
+
import operator_owed # noqa: E402 (sibling script, path set above)
|
|
70
|
+
except Exception as _exc: # noqa: BLE001
|
|
71
|
+
operator_owed = None
|
|
72
|
+
_IMPORT_ERROR: Exception | None = _exc
|
|
73
|
+
else:
|
|
74
|
+
_IMPORT_ERROR = None
|
|
75
|
+
|
|
76
|
+
# A moved id as `--moved` takes it; an AC token anywhere in a matrix row id,
|
|
77
|
+
# with a story-key-like prefix right before it when there is one; and a full
|
|
78
|
+
# commit id as git prints one (never a ref name such as `HEAD`, never a prefix).
|
|
79
|
+
_MOVED = re.compile(r"^AC-0*(\d+)$", re.IGNORECASE)
|
|
80
|
+
_AC_TOKEN = re.compile(
|
|
81
|
+
r"(?:(?<![A-Za-z0-9])(?P<key>\d+(?:-\d+)+)-)?(?<![A-Za-z0-9])AC\s*[-_:#.\u2010-\u2015]?\s*0*(?P<n>\d+)(?![0-9])",
|
|
82
|
+
re.IGNORECASE,
|
|
83
|
+
)
|
|
84
|
+
_STORY_DIGITS = re.compile(r"^\d+(?:-\d+)*")
|
|
85
|
+
_COMMIT = re.compile(r"^(?:[0-9a-f]{40}|[0-9a-f]{64})$")
|
|
86
|
+
|
|
87
|
+
NO_ROW, ROW, UNREADABLE = 0, 1, 2
|
|
88
|
+
|
|
89
|
+
|
|
90
|
+
def _empty(moved: list[str], reasons: list[str]) -> dict:
|
|
91
|
+
return {
|
|
92
|
+
"moved": moved,
|
|
93
|
+
"rows": [],
|
|
94
|
+
"matrix": None,
|
|
95
|
+
"oracle_sources": [],
|
|
96
|
+
"assessed_commit": None,
|
|
97
|
+
"reasons": reasons,
|
|
98
|
+
}
|
|
99
|
+
|
|
100
|
+
|
|
101
|
+
def _moved_id(value: str) -> str:
|
|
102
|
+
match = _MOVED.match(value.strip())
|
|
103
|
+
if not match:
|
|
104
|
+
raise argparse.ArgumentTypeError(f"{value!r} is not an AC id of the form AC-<n>")
|
|
105
|
+
return f"AC-{int(match.group(1))}"
|
|
106
|
+
|
|
107
|
+
|
|
108
|
+
def _commit_id(value: str) -> str:
|
|
109
|
+
sha = value.strip().lower()
|
|
110
|
+
if not _COMMIT.match(sha):
|
|
111
|
+
raise argparse.ArgumentTypeError(
|
|
112
|
+
f"{value!r} is not a full commit id of 40 or 64 hex digits; pass the sha git rev-parse prints"
|
|
113
|
+
)
|
|
114
|
+
return sha
|
|
115
|
+
|
|
116
|
+
|
|
117
|
+
def _named_numbers(rid: str, story: str) -> set[int]:
|
|
118
|
+
"""The AC numbers a row id names for this story: a token a story-key-like
|
|
119
|
+
prefix precedes counts only when that prefix is the story's own."""
|
|
120
|
+
own = {story.strip().lower()}
|
|
121
|
+
digits = _STORY_DIGITS.match(story.strip())
|
|
122
|
+
if digits:
|
|
123
|
+
own.add(digits.group(0))
|
|
124
|
+
return {
|
|
125
|
+
int(token.group("n"))
|
|
126
|
+
for token in _AC_TOKEN.finditer(rid)
|
|
127
|
+
if token.group("key") is None or token.group("key").lower() in own
|
|
128
|
+
}
|
|
129
|
+
|
|
130
|
+
|
|
131
|
+
class _Parser(argparse.ArgumentParser):
|
|
132
|
+
"""An argument error still prints the one JSON object, with the error as its reason."""
|
|
133
|
+
|
|
134
|
+
def error(self, message: str): # type: ignore[override]
|
|
135
|
+
self.print_usage(sys.stderr)
|
|
136
|
+
print(json.dumps(_empty([], [f"usage error: {message}"]), indent=2))
|
|
137
|
+
raise SystemExit(UNREADABLE)
|
|
138
|
+
|
|
139
|
+
|
|
140
|
+
def _moved_of(args: argparse.Namespace) -> list[int]:
|
|
141
|
+
return sorted({int(rid[len("AC-") :]) for rid in args.moved})
|
|
142
|
+
|
|
143
|
+
|
|
144
|
+
def evaluate(args: argparse.Namespace, out: dict | None = None) -> tuple[int, dict]:
|
|
145
|
+
"""The exit code and the JSON object, filled into `out` as the read goes, so
|
|
146
|
+
a failure part way leaves what was already read in it."""
|
|
147
|
+
numbers = _moved_of(args)
|
|
148
|
+
if out is None:
|
|
149
|
+
out = _empty([f"AC-{n}" for n in numbers], [])
|
|
150
|
+
reasons = out["reasons"]
|
|
151
|
+
moved = out["moved"]
|
|
152
|
+
trace = operator_owed.read_matrix(Path(args.trace_output), args.story, reasons)
|
|
153
|
+
if trace is None:
|
|
154
|
+
reasons.append("trace's coverage matrix could not be read, so no moved AC is shown to be out of it")
|
|
155
|
+
return UNREADABLE, out
|
|
156
|
+
measured = [trace.commit]
|
|
157
|
+
matrix_commit = trace.matrix.get("source_sha")
|
|
158
|
+
if isinstance(matrix_commit, str) and matrix_commit.strip():
|
|
159
|
+
measured.append(matrix_commit)
|
|
160
|
+
stale = [sha.strip() for sha in measured if not operator_owed._same_commit(sha, args.commit)]
|
|
161
|
+
if stale:
|
|
162
|
+
reasons.append(
|
|
163
|
+
f"the trace files measured {', '.join(sorted(set(stale)))}, not the commit under assessment "
|
|
164
|
+
f"{args.commit}: they are not this gate's trace"
|
|
165
|
+
)
|
|
166
|
+
return UNREADABLE, out
|
|
167
|
+
ids = [requirement.get("id") for requirement in trace.requirements]
|
|
168
|
+
if not all(isinstance(rid, str) for rid in ids):
|
|
169
|
+
reasons.append("a requirement id in the coverage matrix is not a string, so it cannot be read as an AC's")
|
|
170
|
+
return UNREADABLE, out
|
|
171
|
+
out["matrix"] = str(trace.path)
|
|
172
|
+
out["assessed_commit"] = trace.commit.strip()
|
|
173
|
+
oracle = trace.matrix.get("oracle")
|
|
174
|
+
sources = oracle.get("sources") if isinstance(oracle, dict) else None
|
|
175
|
+
out["oracle_sources"] = sources if isinstance(sources, list) else []
|
|
176
|
+
for requirement, rid in zip(trace.requirements, ids):
|
|
177
|
+
if _named_numbers(rid, args.story) & set(numbers):
|
|
178
|
+
out["rows"].append(
|
|
179
|
+
{"id": rid.strip(), "priority": requirement.get("priority"), "coverage": requirement.get("coverage")}
|
|
180
|
+
)
|
|
181
|
+
if out["rows"]:
|
|
182
|
+
for row in out["rows"]:
|
|
183
|
+
reasons.append(
|
|
184
|
+
f"{row['id']} is a row of {trace.path} ({row['priority']}, {row['coverage']}): trace still "
|
|
185
|
+
"scores an AC the operator moved out of the story"
|
|
186
|
+
)
|
|
187
|
+
return ROW, out
|
|
188
|
+
reasons.append(f"no row of {trace.path} is {', '.join(moved)}")
|
|
189
|
+
return NO_ROW, out
|
|
190
|
+
|
|
191
|
+
|
|
192
|
+
def build_parser() -> argparse.ArgumentParser:
|
|
193
|
+
parser = _Parser(description="Say whether TEA's trace still scores an AC the operator moved out of the story.")
|
|
194
|
+
parser.add_argument("--trace-output", required=True, help="The trace directory the gate's trace wrote to.")
|
|
195
|
+
parser.add_argument("--story", required=True, help="The story id, as the gate takes it.")
|
|
196
|
+
parser.add_argument(
|
|
197
|
+
"--commit", required=True, type=_commit_id, help="The full sha of the commit under assessment, as git prints it."
|
|
198
|
+
)
|
|
199
|
+
parser.add_argument(
|
|
200
|
+
"--moved",
|
|
201
|
+
required=True,
|
|
202
|
+
action="append",
|
|
203
|
+
type=_moved_id,
|
|
204
|
+
help="An AC the move retired, as AC-<n>; repeat it for each.",
|
|
205
|
+
)
|
|
206
|
+
return parser
|
|
207
|
+
|
|
208
|
+
|
|
209
|
+
def main(argv: list[str] | None = None) -> int:
|
|
210
|
+
args = build_parser().parse_args(argv)
|
|
211
|
+
out = _empty([f"AC-{n}" for n in _moved_of(args)], [])
|
|
212
|
+
code = UNREADABLE
|
|
213
|
+
try:
|
|
214
|
+
if _IMPORT_ERROR is not None:
|
|
215
|
+
raise _IMPORT_ERROR
|
|
216
|
+
code, _ = evaluate(args, out)
|
|
217
|
+
except Exception as exc: # noqa: BLE001 (any failure refuses; exit 1 would read as a row)
|
|
218
|
+
out["rows"] = []
|
|
219
|
+
out["reasons"].append(f"the check could not finish, so no moved AC is shown to be out of the matrix: {exc!r}")
|
|
220
|
+
code = UNREADABLE
|
|
221
|
+
try:
|
|
222
|
+
print(json.dumps(out, indent=2))
|
|
223
|
+
except Exception as exc: # noqa: BLE001 (a closed pipe or an unprintable object still refuses)
|
|
224
|
+
try:
|
|
225
|
+
print(f"moved_ac_check.py could not print its result: {exc!r}", file=sys.stderr)
|
|
226
|
+
except Exception: # noqa: BLE001
|
|
227
|
+
pass
|
|
228
|
+
return UNREADABLE
|
|
229
|
+
return code
|
|
230
|
+
|
|
231
|
+
|
|
232
|
+
if __name__ == "__main__":
|
|
233
|
+
sys.exit(main())
|
|
@@ -53,7 +53,8 @@ nothing.
|
|
|
53
53
|
|
|
54
54
|
TEA's own per-criterion classification is trusted as given, the same trust the
|
|
55
55
|
gate places in gate_status. The cross-checks catch a stale or foreign matrix,
|
|
56
|
-
not a criterion TEA classified wrongly.
|
|
56
|
+
not a criterion TEA classified wrongly. `read_matrix` holds them, apart from
|
|
57
|
+
the FAIL-only check above, so moved_ac_check.py reads the matrix the same way.
|
|
57
58
|
|
|
58
59
|
Prints one JSON object and exits 0: `owed`, `open` (the requirement ids below
|
|
59
60
|
FULL), `operator` (the open ones owed operator evidence), `evidence` (each of
|
|
@@ -73,6 +74,7 @@ import json
|
|
|
73
74
|
import re
|
|
74
75
|
import sys
|
|
75
76
|
from pathlib import Path
|
|
77
|
+
from typing import NamedTuple
|
|
76
78
|
|
|
77
79
|
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
|
78
80
|
import gate_eval # noqa: E402 (sibling script, path set above)
|
|
@@ -179,41 +181,50 @@ def _gate_reads_owed(args: argparse.Namespace, reasons: list[str]) -> bool:
|
|
|
179
181
|
return True
|
|
180
182
|
|
|
181
183
|
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
|
|
184
|
+
class TraceMatrix(NamedTuple):
|
|
185
|
+
"""TEA's Phase 1 coverage matrix for one story's trace, checked against its summary.
|
|
186
|
+
|
|
187
|
+
A NamedTuple, not a dataclass: the tests load this file by path, without a
|
|
188
|
+
`sys.modules` entry, which a dataclass's annotation check needs.
|
|
189
|
+
"""
|
|
190
|
+
|
|
191
|
+
path: Path
|
|
192
|
+
matrix: dict
|
|
193
|
+
requirements: list[dict]
|
|
194
|
+
live_records: list
|
|
195
|
+
commit: str
|
|
196
|
+
|
|
195
197
|
|
|
196
|
-
|
|
197
|
-
|
|
198
|
+
def read_matrix(trace_output: Path, story: str, reasons: list[str]) -> TraceMatrix | None:
|
|
199
|
+
"""The story's trace matrix, or None with the reason appended when it cannot be trusted.
|
|
200
|
+
|
|
201
|
+
The report must be exactly `trace-<story>.md` and declare itself a trace
|
|
202
|
+
report; its `tempCoverageMatrixPath` must be absolute and name a
|
|
203
|
+
PHASE_1_COMPLETE matrix whose P0-P3 counts equal the story's own
|
|
204
|
+
`e2e-trace-summary-<story>.json`, whose requirements recount to them, and
|
|
205
|
+
whose `source_sha`, when it has one, names the summary's commit, which must
|
|
206
|
+
be named. moved_ac_check.py reads the matrix through this too.
|
|
207
|
+
"""
|
|
208
|
+
report = _own_artifact(trace_output, "trace-{}.md", story)
|
|
198
209
|
if report is None or not gate_eval._declares_trace_report(report):
|
|
199
|
-
reasons.append(f"no trace report named trace-{
|
|
200
|
-
return
|
|
210
|
+
reasons.append(f"no trace report named trace-{story}.md in {trace_output}")
|
|
211
|
+
return None
|
|
201
212
|
report_text = gate_eval._read_text(report)
|
|
202
213
|
matrix_hint = gate_eval._frontmatter(report_text or "").get("tempCoverageMatrixPath", "")
|
|
203
214
|
if not matrix_hint or not Path(matrix_hint).is_absolute():
|
|
204
215
|
reasons.append(
|
|
205
216
|
f"{report.name} names no absolute tempCoverageMatrixPath, so TEA's coverage matrix cannot be read"
|
|
206
217
|
)
|
|
207
|
-
return
|
|
218
|
+
return None
|
|
208
219
|
matrix = gate_eval._read_json(Path(matrix_hint))
|
|
209
220
|
if matrix is None or matrix.get("phase") != "PHASE_1_COMPLETE":
|
|
210
221
|
reasons.append(f"{matrix_hint} is not a readable PHASE_1_COMPLETE coverage matrix")
|
|
211
|
-
return
|
|
212
|
-
summary_path = _own_artifact(trace_output, "e2e-trace-summary-{}.json",
|
|
222
|
+
return None
|
|
223
|
+
summary_path = _own_artifact(trace_output, "e2e-trace-summary-{}.json", story)
|
|
213
224
|
summary = gate_eval._read_json(summary_path) if summary_path else None
|
|
214
225
|
if summary is None:
|
|
215
|
-
reasons.append(f"no readable e2e-trace-summary-{
|
|
216
|
-
return
|
|
226
|
+
reasons.append(f"no readable e2e-trace-summary-{story}.json in {trace_output}")
|
|
227
|
+
return None
|
|
217
228
|
|
|
218
229
|
try:
|
|
219
230
|
matrix_counts = {p: _counts(matrix["coverage_statistics"]["priority_breakdown"], p) for p in PRIORITIES}
|
|
@@ -228,13 +239,13 @@ def evaluate(args: argparse.Namespace) -> dict:
|
|
|
228
239
|
raise TypeError("live_records is not a list")
|
|
229
240
|
except (KeyError, TypeError) as exc:
|
|
230
241
|
reasons.append(f"the coverage matrix or the summary lacks a coverage field ({exc})")
|
|
231
|
-
return
|
|
242
|
+
return None
|
|
232
243
|
stale = [p for p in PRIORITIES if matrix_counts[p] != summary_counts[p]]
|
|
233
244
|
if stale:
|
|
234
245
|
reasons.append(
|
|
235
246
|
f"the matrix disagrees with {summary_path.name} on {', '.join(stale)}: it is not this gate's matrix"
|
|
236
247
|
)
|
|
237
|
-
return
|
|
248
|
+
return None
|
|
238
249
|
recount = {
|
|
239
250
|
p: (
|
|
240
251
|
sum(1 for r in requirements if r.get("priority") == p),
|
|
@@ -244,15 +255,36 @@ def evaluate(args: argparse.Namespace) -> dict:
|
|
|
244
255
|
}
|
|
245
256
|
if recount != matrix_counts:
|
|
246
257
|
reasons.append("the matrix's requirements do not recount to its own priority breakdown")
|
|
247
|
-
return
|
|
258
|
+
return None
|
|
248
259
|
commit = summary.get("source_sha")
|
|
249
260
|
if not isinstance(commit, str) or not commit.strip():
|
|
250
261
|
reasons.append(f"{summary_path.name} names no source_sha, so no commit to record the evidence at")
|
|
251
|
-
return
|
|
262
|
+
return None
|
|
252
263
|
matrix_commit = matrix.get("source_sha")
|
|
253
264
|
if isinstance(matrix_commit, str) and matrix_commit.strip() and not _same_commit(matrix_commit, commit):
|
|
254
265
|
reasons.append(f"the matrix measured {matrix_commit}, not the summary's {commit}")
|
|
266
|
+
return None
|
|
267
|
+
return TraceMatrix(Path(matrix_hint), matrix, requirements, live_records, commit)
|
|
268
|
+
|
|
269
|
+
|
|
270
|
+
def evaluate(args: argparse.Namespace) -> dict:
|
|
271
|
+
reasons: list[str] = []
|
|
272
|
+
out: dict = {
|
|
273
|
+
"owed": False,
|
|
274
|
+
"open": [],
|
|
275
|
+
"operator": [],
|
|
276
|
+
"evidence": {},
|
|
277
|
+
"partial": {},
|
|
278
|
+
"assessed_commit": None,
|
|
279
|
+
"reasons": reasons,
|
|
280
|
+
}
|
|
281
|
+
if not _gate_reads_owed(args, reasons):
|
|
282
|
+
return out
|
|
283
|
+
|
|
284
|
+
trace = read_matrix(Path(args.trace_output), args.story, reasons)
|
|
285
|
+
if trace is None:
|
|
255
286
|
return out
|
|
287
|
+
requirements, live_records, commit = trace.requirements, trace.live_records, trace.commit
|
|
256
288
|
story_text = gate_eval._read_text(Path(args.story_file))
|
|
257
289
|
if story_text is None:
|
|
258
290
|
reasons.append(f"story file {args.story_file} could not be read")
|