bmad-module-ultracode-goal 3.1.0 → 3.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +1 -1
- package/README.md +1 -1
- package/docs/architecture.md +1 -1
- package/docs/gate-model.md +2 -2
- package/docs/how-it-works.md +3 -3
- package/docs/index.md +1 -1
- package/docs/operating-tips.md +3 -2
- package/docs/troubleshooting.md +1 -1
- package/docs/why-ultracode-goal.md +1 -1
- package/package.json +1 -1
- package/skills/ucg-resolve/SKILL.md +11 -1
- package/skills/ucg-status/SKILL.md +3 -2
- package/skills/ultracode-goal/SKILL.md +2 -2
- package/skills/ultracode-goal/references/execute.md +11 -5
- package/skills/ultracode-goal/references/finalize.md +5 -3
- package/skills/ultracode-goal/references/gate.md +69 -13
- package/skills/ultracode-goal/references/ingest-and-scope.md +1 -1
- package/skills/ultracode-goal/scripts/drive_epic.py +49 -0
- package/skills/ultracode-goal/scripts/gate_trail.py +5 -3
- package/skills/ultracode-goal/scripts/hooks/guard_pretooluse.py +800 -27
- package/skills/ultracode-goal/scripts/mutate_check.py +158 -32
- package/skills/ultracode-goal/scripts/preflight_check.py +5 -1
|
@@ -6,24 +6,76 @@ This is the completion authority. The `/goal` evaluator only sees the transcript
|
|
|
6
6
|
|
|
7
7
|
## Backfill the gate evidence (production only)
|
|
8
8
|
|
|
9
|
-
The gate reads artifacts TEA produces. In **production**, before running the gate, make the evidence current
|
|
9
|
+
The gate reads artifacts TEA produces. In **production**, before running the gate, make the evidence current, in this order:
|
|
10
10
|
|
|
11
|
-
1. `bmad-testarch-automate`
|
|
12
|
-
2. `
|
|
11
|
+
1. `bmad-testarch-automate` backfills coverage for code that landed during Execute. **It runs first and alone.** The mutation runs it confirms a gap with (`scripts/mutate_check.py` writes a mutation into a source file, runs a command, then restores the file) edit source in the gate worktree ("Measure the commit, not the working tree" below), and trace reads the coverage it backfills, so no other assessor reads, runs or times anything in the worktree until automate has finished.
|
|
12
|
+
2. Once automate has finished, log its `gate-gap:` lines ("A reported gap stays open until a later pass measures it closed" below), commit its backfill if it made one ("When automate backfills" below), then rebuild the gate worktree at the commit under assessment ("Rebuild the worktree once automate has finished" below). A red backfill sweep ends the cycle here instead, before trace ("A red backfill sweep ends the cycle at a re-loop" below), and nothing else automate's pass or its backfill produces does: a `gate-gap:` line never ends a cycle. A sub-skill that blocks on input or is not installed still escalates the story, at this step or any other, as `references/execute.md`'s sub-skill halt catch-all says.
|
|
13
|
+
3. `bmad-testarch-trace` (re)builds the traceability matrix and writes the gate decision.
|
|
13
14
|
|
|
14
|
-
`bmad-testarch-nfr` (produces `nfr-assessment.md`) and `bmad-testarch-test-review` are independent
|
|
15
|
+
`bmad-testarch-nfr` (produces `nfr-assessment.md`) and `bmad-testarch-test-review` are independent of each other and of trace, so once step 2 is done, run them in any order, or concurrently with trace. None of the three may edit a file in the worktree, so none of them runs a mutant: an assessor that would probe one names the mutant it would run as a finding in its own artifact instead, and when you delegate NFR or test review, tell it so. `gate_eval.py` consumes all three artifacts without caring how they were produced. (`bmad-testarch-test-review` normally runs in Execute per story: run it here only if you lack a current `test-review.md` for the story. One written before automate's backfill is not current, since no review has read the tests the backfill added.) In **`--light`**, skip all of the above and run only `bmad-testarch-trace`, then the gate with `--profile light` (trace gate only, with no NFR/review AND).
|
|
16
|
+
|
|
17
|
+
**When automate backfills, commit the backfill before trace runs.** A test automate writes or changes lands in the main tree, as every assessor's output does ("Their OUTPUTS go home" below), so neither the commit under assessment nor the worktree made from it holds that test, and trace would count a test the commit does not contain. So never run trace, NFR or test review on that commit. While `.current-story` still names this story, log `gate-backfill-base: <sha>` in `.decision-log.md`, naming `HEAD`, the commit automate measured, then commit the backfill the way `references/execute.md` commits a story's paths, in this order:
|
|
18
|
+
|
|
19
|
+
1. Stage the test files this pass's summary (`automation-summary-<story_id>-<sha>.md`, with `<sha>` the `gate-backfill-base:` commit; "Automate's summary records its rows" below) names as written or changed, by explicit path, in one call. `git status --porcelain --untracked-files=all` lists them: the plain `git status --porcelain` lists a new directory instead of the files in it, so a test in a new folder would look absent. A path it lists that the summary does not name is not part of the backfill and stays unstaged. A path the summary names that it lists is staged from the main tree, and the worktree's copy of that file, which automate ran the test's confirming rows against ("A backfill is confirmed in the pass that writes it" below), is not a test written into the worktree: the rebuild discards it. A path the summary names that it does not list has no change in the main tree, so look for the file in the worktree: at that path when the summary gives it under `<tmp-path>`, and otherwise at the same repo-relative path under `<tmp-path>`. When the file is there, automate wrote it into the worktree: copy it to its repo-relative path in the main tree before the rebuild destroys it, and list again. When git still lists no change at that path, or neither tree holds the file, the test the summary names was never written: leave the path out, and log a `gate-gap:` line for the gap that test was written for, as "A reported gap stays open until a later pass measures it closed" below describes, in place of any `unkilled` line logged for that gap. When this step stages nothing, there is no backfill: skip steps 2 through 5, and the commit under assessment stays the `gate-backfill-base:` commit.
|
|
20
|
+
2. With them staged, so that a gate reading git's file list sees a new test file, run the full three-axis sweep over the main tree, cache-defeated and printed, as `references/execute.md` step 5 runs it. A red that a change to the backfilled test files alone fixes (a lint or format error in one, a tracked-files gate that names one) is fixed there, staged, and re-run, at most twice in this backfill, a fix step 4 makes for a commit hook included. A red that needs a change to any other path, or that is still red after the second fix, is not the gate's to fix: commit nothing, and take "A red backfill sweep ends the cycle at a re-loop" below. A green sweep goes on to step 3, whatever `gate-gap:` lines this cycle has logged.
|
|
21
|
+
3. Once the sweep is green, write a fresh `.tests-ran-<story_id>` marker as step 5 writes one: its `baseline=` line copied from `.baseline-<story_id>`, never re-derived, and `scope=full`.
|
|
22
|
+
4. Run `references/execute.md` step 4's claim check on the commit message, then commit from the message file, in a call separate from the staging call, as step 4 requires. The guard allows this commit as it allows the evidence commit in "Route on the verdict": the story is still current, its marker was written in a prior call, and the index is not empty. A repo commit hook that rejects this commit is read as step 2's red, and `references/execute.md` step 4's own rule for a hook rejection does not apply to it: when a change to the backfilled test files alone answers the hook, make it there, stage it, and go back to step 2, whose sweep runs again before step 3 writes a fresh marker and this step retries the commit, all within step 2's budget of two fixes; when the hook needs a change to any other path, or rejects the commit again after the second fix, commit nothing, and take "A red backfill sweep ends the cycle at a re-loop" below.
|
|
23
|
+
5. Refresh `.commit-<story_id>` from `HEAD`, log `gate-backfill: <sha>`, naming the new commit, in `.decision-log.md`, and remove `.scratch-<story_id>/` again: the claim check's message files recreated it, and `references/execute.md` step 0 has it removed before the story advances.
|
|
24
|
+
|
|
25
|
+
The new commit is now the commit under assessment: the worktree is rebuilt at it, trace, NFR and test review measure it, and the verdict's `assessed-commit` line names it. The backfill belongs to this gate cycle and is not a re-loop: it writes no `class:` line, spends no re-loop, and never meets the turn-ceiling check that runs before a re-loop, though its sweep spends turns like any wait (`references/execute.md`, "Waiting spends turns").
|
|
26
|
+
|
|
27
|
+
**A red backfill sweep ends the cycle at a re-loop.** A red in step 2's sweep, a commit hook's rejection at step 4 included, that needs a change outside the backfilled test files, or is still red after the second fix, means the backfill has found a defect in the story's code. Of what automate's pass and its backfill produce, it is the one thing that ends a cycle before trace, because a failing sweep is a script's result and not a judgment; a `gate-gap:` line never ends a cycle. A fix made here would land after automate's only pass in this cycle and skip the generalize step, so commit nothing and run no trace, NFR or test review in this cycle. Log this cycle's verdict in `.decision-log.md` as a fenced JSON object holding only `verdict` and `reasons`, `{"verdict": "reloop", "reasons": [...]}`, with one reason naming the red, and beside it an `assessed-commit:` line naming the `gate-backfill-base:` commit, the commit automate measured. That object is the shape `scripts/gate_trail.py` reads as a recorded verdict, and since no trace ran it holds no `gate_status`. It stands in for the `gate_eval.py` result this cycle never produced: the run-status heartbeat's `last_reasons` copies its `reasons` (`references/execute.md`, "Run-status heartbeat"), and an attended run prints the one verdict line "Attended runs also print the verdict, once" (under "Run the gate") describes when this verdict is logged, with `n/a` as the `gate_status`. Then take the `reloop` route under "Route on the verdict" with the red and every open `gate-gap:` line of this cycle that is neither `unmeasured` nor `unkilled` as its findings, turn-ceiling check included. The backfilled files stay staged in the main tree, so the re-loop's `references/execute.md` step 4 commits them with its fix, and the next cycle's automate scope, the diff from the `assessed-commit:` commit, holds the fix and those tests. When the turn-ceiling check turns that re-loop into `escalate`, the backfilled files stay staged too, and the escalation sidecar's `decision_needed` names each of them as staged for this story. The cycle after this re-loop runs the full gate, never a delta gate: this cycle wrote no trace, NFR or test review artifact for a delta gate to re-check. A resume that finds this verdict logged and `HEAD` still at its `assessed-commit:` commit takes this re-loop where the log stopped (`references/execute.md`'s fifth resume case).
|
|
28
|
+
|
|
29
|
+
**A resume after a backfill does not run automate again.** When this story's `.decision-log.md` holds a `gate-backfill-base:` or a `gate-backfill:` line with no verdict logged after it (when it holds both for this cycle, the later one decides which bullet applies), `references/execute.md`'s fifth resume case sends the resume here, and it goes on from where the cycle stopped:
|
|
30
|
+
|
|
31
|
+
- After `gate-backfill: <sha>` with `HEAD` still `<sha>`, remove `.scratch-<story_id>/` if it is still there, rebuild the worktree at that commit and run trace, NFR and test review on it.
|
|
32
|
+
- After `gate-backfill-base: <sha>`, when `git rev-parse HEAD` prints `<sha>`, the backfill is not committed yet: start again at step 1 above. When `git rev-parse HEAD~1` prints `<sha>`, it is committed: start at step 5.
|
|
33
|
+
- At any other `HEAD`, the story is not in this case: take the route `preflight_check.py` reports.
|
|
34
|
+
|
|
35
|
+
**At most one automate pass finishes per gate cycle, and a backfill never restarts the cycle.** After the backfill commit the same cycle goes on, on the new commit, and automate does not run in it again, so every cycle reaches a verdict whatever automate finds: trace's, or the re-loop a red backfill sweep takes. Automate runs a second time in one cycle only when a resume finds the first pass's summary unfinished, or finds none at all ("A summary already at that name is this cycle's own" below). A later cycle of the story exists only because a verdict sent it back to be gated again, and in that cycle, whether the re-gate is full or delta, automate's scope is the diff from the previous gate's `assessed-commit` line to the commit under assessment, plus the mutation rows behind the previous cycle's backfilled gaps and behind every open `gate-gap:` line:
|
|
36
|
+
|
|
37
|
+
- It searches for new gaps only in code that diff changes. Code outside the diff was in an earlier cycle's scope already.
|
|
38
|
+
- It re-runs, at the commit under assessment, whatever the diff covers, the mutation rows behind each gap the previous cycle backfilled (the `scripts/mutate_check.py` table rows recorded in the summary named for that cycle's `gate-backfill-base:` commit) and behind each open `gate-gap:` line (the rows recorded in the summary the line names). For an `unmeasured` line ("A risk automate names on a P0 criterion" below), and for a line whose rows cannot run ("A reported gap stays open until a later pass measures it closed" below says which cannot), it measures the gap afresh instead and writes new rows for it. A row that reads `GREEN` is a gap again, and "Automate backfills every gap it finds on a P0 criterion" below says whether automate backfills it or reports it, except a row behind an open `unkilled` line, which keeps that line open and gets no new test ("A backfill is confirmed in the pass that writes it" below). A re-loop sent to close a reported gap often adds only a test, which leaves the gap's code outside the diff, so without this re-run no gate would measure whether that test closes the gap.
|
|
39
|
+
- When that diff is empty, automate still runs, for the open `gate-gap:` lines alone, when any of them names a commit other than the commit under assessment. Otherwise skip automate and the rebuild, and carry the open lines unchanged, `unmeasured` ones included. A cap-only re-gate, and the re-gate a closed `operator-evidence-owed` escalation owes, measure the commit the previous gate measured, so automate has nothing left to look at there unless an open line names another commit, and the gate is still the full instrument. Each automate pass leaves every open line naming the commit that pass measured, so no cycle runs automate at a commit an earlier cycle's pass has measured.
|
|
40
|
+
- When the previous verdict record has no `assessed-commit` line, automate runs in full, as in a story's first cycle (the same fallback the delta profile takes).
|
|
41
|
+
|
|
42
|
+
(Measured on one story, where each backfill started a new cycle: two cycles of about an hour each ended at automate with a new backfilled test, and a third reached a verdict only once its automate could not add tests. The second gap sat in code the first automate pass had already searched and named as a risk, outside the diff between the two cycles, which held only the first backfill's tests. A re-run of the previous backfill's mutation table alone would leave a re-loop's own new code, the code automate exists to cover, with no automate pass at all, and the diff alone would leave a gap a re-loop was sent to close unmeasured, so a later cycle takes both.)
|
|
43
|
+
|
|
44
|
+
**Automate backfills every gap it finds on a P0 criterion.** In any cycle, whatever its scope, and even while the story's live results file holds a record at the commit under assessment, a gap automate finds on a P0 criterion gets a test: the backfill is committed before trace ("When automate backfills" above), so TEA's trace counts the test and `gate_eval.py` decides. A gap on any other criterion it backfills only when the gap's code is in this cycle's search scope (the code that landed during Execute in a story's first cycle, the diff in a later one) and the story's live results file holds no record at the commit under assessment. Otherwise it reports the gap rather than backfills it: it names the gap in its summary, with its mutation evidence, and adds no test. So only a gap on a criterion other than P0 is ever reported rather than backfilled. A gap behind an open `unkilled` line already has its test, so no pass writes it another while its rows can run ("A backfill is confirmed in the pass that writes it" below). A P0 backfill made while a live record sits at the commit under assessment moves that commit past the record, and that needs no rule of its own: a `pass` counts only at the commit trace measures ("Operator evidence is a recorded live result" below), so the gate at the backfill commit owes the AC again through the routes that already owe one (the `operator-evidence-owed` escalation on a FAIL, a ledger row on `advance` or `defer`), as `references/execute.md`'s fourth resume case already says of the commit a red makes past the assessed commit. For a gap on another criterion that cost is not worth paying, which is why automate reports it instead.
|
|
45
|
+
|
|
46
|
+
**A risk automate names on a P0 criterion is measured in the same pass.** The pass is the only one its cycle finishes, so a risk it names in its summary and leaves as prose (an assertion that a mutant failing some other way would also satisfy, say) would reach the verdict unmeasured. When you delegate automate, tell it to measure each risk it names on a P0 criterion before it finishes, with a mutation row for each failure arm the criterion names, its own backfilled tests included, and to backfill each gap that measuring confirms, as it backfills every gap on a P0 criterion. A named P0 risk it still leaves unmeasured is logged as a `gate-gap:` line that says `unmeasured` in place of a summary, and carried: the story's next automate pass measures it, and backfills the gap when a row confirms one. An `unmeasured` line is not a finding, since no mutant has run for it yet, and like every `gate-gap:` line it never changes the route the verdict takes.
|
|
47
|
+
|
|
48
|
+
**A backfill is confirmed in the pass that writes it.** When you delegate automate, tell it to write each test it backfills to its repo-relative path in the main tree, copy that file unchanged to the same path in the gate worktree, where its mutation rows run, and re-run there, with the test in place, the rows behind the gap the test was written for, making any later edit to the test in the main tree and copying it again. Tell it, too, to give each row it re-runs there a `command` that runs the backfilled test's file, so that the row's own run sees the test: a row written before the test may select only the tests that existed then, and the second run with the full sweep's command (the second bullet below) can be refused in the worktree, where a tracked-files check sees the copy as a file git does not track. Read each row as "A reported gap stays open until a later pass measures it closed" below reads one. When every row reads killed, the test closed the gap. When a row still reads `GREEN` with the test in place, the test ran and did not kill that row's mutant, which may be an equivalent one that no test can tell from the code: log an `unkilled` `gate-gap:` line for the gap, naming the summary that records its rows and the test that did not kill them. The test is still committed with the backfill. No later pass writes another test for an `unkilled` line while its rows can run: each later pass re-runs them, the line closes when they read killed, a row that reads `GREEN` keeps it open, and a row that cannot run (`STALE` once the code it mutates has moved) has the gap measured afresh, as for any line. Like an `unmeasured` line, an `unkilled` line is not a finding, and it never changes the route the verdict takes.
|
|
49
|
+
|
|
50
|
+
**Automate's summary records its rows.** When you delegate automate, name its summary `{workflow.implementation_artifacts}/automation-summary-<story_id>-<sha>.md`, with `<sha>` the commit it measures as `git rev-parse HEAD` prints it, and tell it to copy into that summary, verbatim, each `scripts/mutate_check.py` table row (`file`, `anchor`, `replacement`, `command`, `expect_red`) behind a gap it backfilled or reported, or behind an open `gate-gap:` line it re-ran and leaves open, so a later cycle can re-run them. TEA's default, `automation-summary.md` at the TEA `test_artifacts` root, is one file every pass updates, for every story, so step 1 of a backfill would read an earlier pass's files as this pass's, and a later cycle would look for its rows in a file another pass has since updated. If automate writes the flat file anyway, move it to that name before anything else reads it. The ledger that `scripts/mutate_check.py --output` writes leaves out each row's anchor, replacement and command, so it is no substitute for the rows.
|
|
51
|
+
|
|
52
|
+
**A summary already at that name is this cycle's own.** No two cycles run automate at one commit ("When that diff is empty", above), so a summary at that name was written by this cycle's own pass, in a session that ended before this cycle's verdict was logged, and TEA adds each step's output to a summary that already exists rather than starting a new one. So before you delegate automate, look for it, and right before each time you delegate it, log `gate-automate: <sha>` in `.decision-log.md`, with `<sha>` the commit it measures, the one its summary's name carries: a session that ends while automate runs, or after it finishes but before anything is committed, then leaves a record of where its cycle stopped, and `references/execute.md`'s fifth resume case sends a resume that finds that line for `HEAD`, with no verdict and no backfill line after it, back to this paragraph instead of re-driving the story. When none is there, delegate automate. When it is there and complete (its frontmatter's `lastStep` reads `step-04-validate-and-summarize`, the last step of TEA's automate), that pass finished: do not run automate again. Go on to step 2 of the list at the top of this section with that summary as this pass's, logging only the `gate-gap:` lines this cycle's entry does not hold yet, and leave the dead session's worktree in place until the rebuild removes it, so that step 1 of "When automate backfills" can look in it for the tests the summary names. When it is there but not complete, that pass stopped part way: move it aside to `automation-summary-<story_id>-<sha>-stale-<n>.md` in the same folder, with `<n>` the first number from 1 that no file there carries yet, log the move in `.decision-log.md`, and only then delegate automate.
|
|
53
|
+
|
|
54
|
+
**A reported gap stays open until a later pass measures it closed.** Once automate has finished, log in this cycle's `.decision-log.md` entry one `gate-gap:` line for each gap it reported, for each risk on a P0 criterion it left unmeasured, for each gap it could not measure because its rows could not run even afresh, and for each backfill its confirmation left `unkilled`, beside any line step 1 of "When automate backfills" logs for a test that was never written. Each line names the criterion, its priority, the commit automate measured and the summary that recorded its rows, or `unmeasured` in place of the summary for a risk it did not measure or a gap it could not, and an `unkilled` line also says `unkilled` and names the test that did not kill its rows. Every later cycle of the story copies the previous cycle's open `gate-gap:` lines into its own entry, whatever route the previous verdict took: once its automate has finished, or before trace when automate does not run. Whether a line is open is read off `scripts/mutate_check.py`'s classification of each row behind it:
|
|
55
|
+
|
|
56
|
+
- A row reads killed when it is `KILLED`, `KILLED+others`, or `MISATTRIBUTED` with the detail that its `expect_red` never appears beside a failure token. In each, something the row's command runs failed under the mutant, and a test a re-loop added under a name other than the row's `expect_red` reads `MISATTRIBUTED` that way.
|
|
57
|
+
- A row that reads `GREEN` is still a gap unless it reads killed when re-run once more with its `command` replaced by the test command of the story's full sweep that covers its `file`, cache-defeated, as step 2 of "When automate backfills" runs it: a row's own command can select only the tests that existed when the row was written, so it misses a killing test a re-loop added in another file. When the second run reads killed, the row reads killed. When it reads `GREEN`, or cannot run (the next bullet), the row is still a gap, read by its first run: a second run that cannot run is no reason to measure the gap afresh. Automate makes that second run in the same pass, for each row it runs that reads `GREEN`, so when you delegate it, tell it to, and to record both results beside the row in its summary.
|
|
58
|
+
- A row cannot run when it reads `STALE`, `NOT UNIQUE` or `UNDECODABLE` (its file or its anchor no longer holds, so nothing was mutated), or `MISATTRIBUTED` with the detail that the output's own count says 0 failed (a collection or harness error, not a test: a mutant reads this way when pytest then cannot collect a test module or load a conftest.py, or jest or vitest cannot load a test file, and no other test fails, since no test in that file runs), or when `mutate_check.py` refuses its table with exit 2 (no row ran). It measures nothing, so its line stays open, and the pass measures that gap afresh, once in the pass: when a row it writes for the gap then cannot run either, it measures that gap no further, and logs the gap's line as `unmeasured`, in place of a summary, for the story's next automate pass to measure.
|
|
59
|
+
|
|
60
|
+
A line closes when an automate pass at a later commit than the one the line names finds every row behind it reading killed: the line's own rows, re-run, or for an `unmeasured` line or one whose rows cannot run, the rows that pass writes when it measures the gap afresh, any test it backfills for them included. That pass's cycle leaves the line out. A carried line that pass does not close is logged afresh in place of the copied line, naming this commit and this pass's summary, or `unmeasured` while the risk still is or the gap's rows still cannot run, or `unkilled`, naming its test, while a test backfilled for the gap has not killed its rows. A `gate-gap:` line never changes the route the verdict takes, and each route treats the open lines this way:
|
|
61
|
+
|
|
62
|
+
- On a `reloop` that is not cap-only, each open line that is neither `unmeasured` nor `unkilled` is one of the findings the generalize step reads. A cap-only `reloop` has no finding to generalize and an `escalate` only stops, so neither of them takes one.
|
|
63
|
+
- On `advance` or `defer`, each open line becomes a ledger row with `source` `gate` and `severity` `med`, whatever its criterion's priority. An `unmeasured` line's row says the risk or the gap is unmeasured, and an `unkilled` line's row says the backfilled test did not kill the mutant, which may be equivalent. No such row is a gate FAIL or a critical finding, whatever its criterion's priority, so it belongs on the ledger and never re-loops or escalates the story, and neither the INVARIANT under "Route on the verdict" nor the ledger schema's `severity` bar sends it anywhere else: a FAIL is the `gate_status` that `gate_eval.py` reads from TEA's trace, which no `gate-gap:` line changes, and a critical finding is one a code review or the NFR assessment rates critical, while a `gate-gap:` line records automate's mutation measurement, an `unmeasured` one a measurement nobody has made yet and an `unkilled` one a test that did not kill its mutant, not a defect anyone found.
|
|
64
|
+
- Whatever the route, a line stays open, so the story's next gate cycle, when there is one, copies it.
|
|
15
65
|
|
|
16
66
|
**Operator evidence is a recorded live result, in either profile.** An AC tagged `**Gate-ability:** operator-evidence` (`references/define-done.md`, step 1) is proven only through TEA's live verification results, which trace reads as `live_results_input` and which TEA lets a person write. For a story with such an AC the file is `{workflow.trace_output_dir}/live-verification-results-<story_id>.json`; name it (absolute, main tree) when you delegate trace. Each record carries `id` `<story_id>-LIVE-<NNN>`, `requirement_id` `AC-<n>`, `status`, `evidence` and `source_sha`, under `schema_version` `0.1.0`. A `pass` counts only at the commit trace measures. Write no record the operator did not give you, and name the operator as `producer`. TEA caps a requirement covered only this way at CONCERNS, so the story defers. That is the honest outcome, never a gap to close with a test that restates the record.
|
|
17
67
|
|
|
18
68
|
## Measure the commit, not the working tree
|
|
19
69
|
|
|
20
|
-
Stage 5's subject is a COMMIT, but an assessor that runs or times anything by default measures the WORKING TREE
|
|
70
|
+
Stage 5's subject is a COMMIT, but an assessor that runs or times anything by default measures the WORKING TREE, which is where the next remediation, or a sibling assessor's mutation window, happens. A measured run recorded the consequence three times in one story even under a strict no-concurrent-work rule: one assessor's mutation window contaminated a sibling's timing run (discarded and re-run), and two assessors invented their own ad hoc isolation mid-gate because they could not trust the shared tree. Make the isolation structural instead of a discipline each assessor re-derives: create a worktree of the commit under assessment with `git worktree add <tmp-path> <commit>`, with `<tmp-path>` under `{workflow.implementation_artifacts}/.gate-worktree-<story_id>` (the same containment Execute's scratch rule buys; never an improvised repo-root directory), and point every assessor's reads, runs and measurements THERE. Their OUTPUTS go home: every artifact an assessor writes (trace report, gate decision, NFR assessment, test review, `bmad-testarch-automate`'s backfilled coverage) lands in the MAIN tree's resolved dirs (`{workflow.trace_output_dir}`, `{workflow.implementation_artifacts}`, resolved to absolute paths against `{project-root}` BEFORE any assessor is pointed at the worktree), never under the worktree. The copy of a backfilled test that automate puts in the worktree, to run the rows that confirm it there, is an input to that measurement and not where the test lands: automate writes the test to the main tree first, the main tree's file is the one the backfill commit stages, and the rebuild after automate discards the copy ("A backfill is confirmed in the pass that writes it", under "Backfill the gate evidence"). The worktree is the measurement surface, the main tree is the record: an artifact written into the worktree is destroyed with it, after which the gate re-reads the PREVIOUS cycle's same-named artifact as current (resolution is by name, with no freshness check), which is a stale PASS advancing a story whose fresh assessment was never read. Once the story's verdict is recorded, remove the worktree with `git worktree remove --force <tmp-path>` (the copied-in untracked inputs below make the unforced form refuse, and a measurement surface holds nothing to preserve), then `git worktree prune`; a resume that finds a dead session's stale `.gate-worktree-<story_id>` removes it the same way before creating its own. A step that only authors an artifact from already-recorded evidence has nothing to measure and needs no worktree; the rule binds every assessor that runs or times anything. Three caveats the setup owes, each a measured failure when skipped:
|
|
21
71
|
|
|
22
72
|
- **Dependency directories do not follow the commit.** A worktree carries tracked files only, so an installed dependency tree (`node_modules` and its kin) is absent there. Link each such directory the repo has (root *and* nested) from the main tree into the worktree with `uv run {skill-root}/scripts/link_worktree_deps.py --project-root {project-root} --worktree <tmp-path>`, which links every `node_modules` folder (`--dir <path>`, repeatable, names a dependency folder of another name instead) and prints what it linked, skipped and failed. Never a bare `ln -s`: it is not a command `acceptEdits` runs without asking, nor on the allowlist, so an unattended gate would have it refused, where the script runs under the allowlisted `uv run`. And prefer the symlink over resolver environment variables: Node's ESM resolver ignores `NODE_PATH` (it is honoured by the CJS resolver only), so a `NODE_PATH` bridge quietly breaks exactly the modern half of a JS gate set. Where a symlink cannot serve, an offline install from the package manager's shared store is the fallback; measure both once per repo before committing to one.
|
|
23
73
|
- **Untracked inputs the gate set reads.** A scratch browser config, an env file, a generated fixture: enumerate what the declared gate commands actually read that the commit does not carry, and copy those in deliberately — an isolated gate that silently lost its config measures a different harness and reports the result as this one's.
|
|
24
|
-
- **The ordering rule stands.** Isolation removes the contamination, not the sequence: the run still gates story N to a verdict before touching story N+1 (the sequential spine is unchanged), so this section licenses no overlap. What
|
|
74
|
+
- **The ordering rule stands.** Isolation removes the contamination, not the sequence: the run still gates story N to a verdict before touching story N+1 (the sequential spine is unchanged), so this section licenses no overlap. What the worktree buys is that no stray tree state, the next remediation's included, can contaminate a gate mid-read. It does not keep the assessors apart, since they all measure the one worktree: that comes from the order "Backfill the gate evidence" sets, where automate, the one assessor that edits source, runs alone and the worktree is rebuilt before any other assessor starts. The assessors after it only read, run and time, so none of them changes the tree another one measures. They still share one machine, so a timing one of them takes beside another's runs carries the load caveat `references/execute.md` gives a wide sweep.
|
|
75
|
+
|
|
76
|
+
**Rebuild the worktree once automate has finished (production).** Automate's mutation runs edit source in the worktree, and it puts there a copy of each test it backfills to confirm the test ("A backfill is confirmed in the pass that writes it" above), so before any other assessor starts, replace the worktree instead of trusting it: remove it with `git worktree remove --force <tmp-path>`, add it again with `git worktree add <tmp-path> <commit>` at the commit under assessment (the backfill's commit when automate made one, per "When automate backfills" above), then link the dependency folders with the same `link_worktree_deps.py` call and copy the same untracked inputs in again, exactly as the first setup did. A worktree just added from the commit holds that commit's tracked files and nothing a mutation or a copied test left behind (a backfilled test the backfill commit carries is one of those tracked files), and the rebuild needs no status read of the old worktree, which would take `git -C <tmp-path> status`, a command no allowlist entry covers. Rebuild whether or not automate backfilled, and whether this session ran the pass or a resume reused it ("A summary already at that name is this cycle's own" above); skip the rebuild only when no automate pass ran in this cycle (`--light`, or a later cycle that skips it), or when a red backfill sweep ended the cycle before trace, so that no other assessor runs in it.
|
|
25
77
|
|
|
26
|
-
Record in `.decision-log.md`, once per gated story, whether the gate measured a worktree or the main tree
|
|
78
|
+
Record in `.decision-log.md`, once per gated story, whether the gate measured a worktree or the main tree: the same honesty line the sweep's `scope=` carries, since an assessment reported as isolated that actually ran on the shared tree is a false claim about the evidence. In production the line also says whether automate ran alone and the worktree was rebuilt before the other assessors started, and names the commit it was rebuilt at.
|
|
27
79
|
|
|
28
80
|
## Non-web stack: the red phase by hand, and the trace artifacts where trace cannot run
|
|
29
81
|
|
|
@@ -58,7 +110,7 @@ Write a PASS only when the story is demonstrably done to its profile's Definitio
|
|
|
58
110
|
4. **The red count recorded in the trace report, with a one-line justification for every case that was already green.** Where TEA authored the trace report, record them beside the captured output of item 3 and in `.decision-log.md` instead, never in TEA's file: an edit there would make its provenance false. Where a shared acceptance file accumulates already-shipped siblings' cases, those green-by-design cases may be justified as one line naming that class rather than case-by-case — the per-case justification owed is for THIS story's cases that never went red.
|
|
59
111
|
5. **`gate-provenance:`, logged per story** exactly as "Run the gate" below requires: `gate-provenance: hand-authored` where item 1 applied, and `tea` where trace ran. A run that takes the hand-authored branch on every row produces an Epic whose every verdict rests on a file the run wrote for itself, and nothing else in the artifacts would say so.
|
|
60
112
|
|
|
61
|
-
`bmad-testarch-nfr` and `bmad-testarch-test-review` are independent of
|
|
113
|
+
`bmad-testarch-nfr` and `bmad-testarch-test-review` are independent of trace and wait only for automate and the step after it, its backfill commit and the worktree rebuild (see "Backfill the gate evidence" above), so still produce `nfr-assessment.md` and `test-review.md` the normal way and run `gate_eval.py --profile production --story <story_id> --nfr <…/nfr-assessment-<story_id>.md> --test-review <…/test-review-<story_id>.md> --tests-ran {workflow.implementation_artifacts}/.tests-ran-<story_id>`. Pass **both** signal flags (and `--tests-ran`, which every per-story gate carries: the sweep-scope AND binds this path unchanged). `gate_eval.py` fail-closes a `--nfr`/`--test-review` path that is given-but-missing **and one you simply omit**: an omitted flag on a per-story production gate is itself a failing signal, with a reason naming the flag. That is deliberate: a forgotten flag used to be skipped in silence, so the AND never ran, the field rendered `null`, and the verdict was computed without it, which bought a *higher* verdict than supplying a failing artifact would have. If a signal genuinely cannot be produced on the stack, that is a CONCERNS/`defer` or a `reloop`, never a dropped flag. The one legitimate omission is the epic roll-up, which declares itself with `--epic-level` (see "Route on the verdict") rather than by leaving the flags off. A stack that cannot meet the production acceptance-test bar at all belongs under `--light` (see the framework fitness caveat in `references/preflight.md`), not a hand-waved production PASS. The honesty bar is unchanged: a PASS requires full AC coverage by passing tests, never to dodge an AND.
|
|
62
114
|
|
|
63
115
|
**The epic-level roll-up needs no hand-authoring at all where sprint-status is available: run it with `--rollup`** (see "Route on the verdict"), which computes the aggregate from the per-story artifacts this section already had you author — no `trace-<epic>.md` required. Hand-author the epic pair only where the gate cannot be given sprint-status. **In that fallback the roll-up needs the same hand-authoring, named for the EPIC.** Everything above is written per story, and where trace cannot run nothing ever writes the epic's own `trace-<epic>.md`, while the omit-`--rollup` form of the gate below presumes it exists: `--story <epic_id>` in a per-story-named directory resolves nothing and fail-closes a completed Epic to `escalate`. When every story is `done`, hand-author the same two files named for the epic id — `trace-<epic>.md`, its frontmatter pointing at `gate-decision-<epic>.json` — with a `gate_status` that honestly summarises the per-story record: a PASS only when every story advanced, never a roll-up dressed over a story that did not. It takes no NFR and no test-review; the roll-up declares that omission with `--epic-level`, per "Route on the verdict" below.
|
|
64
116
|
|
|
@@ -112,7 +164,7 @@ Do not recompute TEA thresholds or re-judge `gate_status` — read it as given.
|
|
|
112
164
|
|
|
113
165
|
**The sweep-scope AND (`--tests-ran`, per-story gates, both profiles).** Pass `--tests-ran {workflow.implementation_artifacts}/.tests-ran-<story_id>` on every per-story gate, exactly as the commands above show. The script reads the marker's `scope=` line (`references/execute.md` steps 2 and 5 write it beside `baseline=`): `scope=full` asserts the green run behind this gate was the full three-axis sweep and passes the verdict through; `scope=scoped` is legitimate mid-loop evidence, so the verdict is **capped at `reloop`** — a story may only advance off a full-scope sweep, and the cap moves `defer` too, because defer's route advances the story. Fail-closed like every other signal: a missing marker, a marker with no `scope=` line, or a value the reader does not recognise all get the same cap, with a reason naming what was wrong. The verdict JSON carries the recognised value as `sweep_scope` so an advance that proved its sweep is distinguishable from one that was never asked. Two boundaries: the epic roll-up gates no sweep of its own, so `--epic-level` refuses the flag outright (an invocation error, exit 2 — the same lane as `--epic-level` with `--nfr`); and *omitting* the flag skips the check rather than failing it, deliberately — making omission failing would change the verdict of documented invocations that predate the flag, the exact 2.0.0 lesson `docs/_internal/STABILITY.md` records — so the always-pass-it rule lives here, in the instruction, and a per-story gate invoked without the flag is a drift to fix, not a laxer mode to use.
|
|
114
166
|
|
|
115
|
-
**The delta cycle profile (`--cycle-profile`, mid-loop gates, production).** A gate whose previous evaluation of this story was `reloop` with findings cleanly attributed to their assessor dimension may re-run as a **delta gate**: re-run in full only the assessor(s) whose dimension produced the failing findings, while the others verify their own prior artifact against the diff from the previous gate's `assessed-commit` line to the commit under assessment (a concerns-only re-check; a previous verdict record without that line means the fallback below
|
|
167
|
+
**The delta cycle profile (`--cycle-profile`, mid-loop gates, production).** A gate whose previous evaluation of this story was `reloop` with findings cleanly attributed to their assessor dimension may re-run as a **delta gate**: re-run in full only the assessor(s) whose dimension produced the failing findings, while the others verify their own prior artifact against the diff from the previous gate's `assessed-commit` line to the commit under assessment (a concerns-only re-check; a previous verdict record without that line means the fallback below: run the full gate) instead of re-deriving the story from zero. Declare it (pass `--cycle-profile delta`) and the script caps the verdict at `reloop`: a delta gate may say "not yet, and here is why", never `advance` or `defer`. ADVANCE requires the full instrument (every assessor over a `scope=full` sweep, with automate in the scope "At most one automate pass finishes per gate cycle", under "Backfill the gate evidence", gives a later cycle), so the oracle's finality is untouched, and a delta cycle whose findings all close costs one extra full gate, the confirming cycle a multi-cycle story pays anyway. The fallback is mechanical, not judgment: when any prior finding lacks a clean dimension attribution (which assessor artifact it came from), run the full gate. A re-loop that ended its cycle before trace ("A red backfill sweep ends the cycle at a re-loop") left no trace, NFR or test review artifact to re-check, so the gate after it is always the full gate. Automate is not one of the assessors this choice sorts: in every later cycle, full or delta, it takes the scope that paragraph sets, and for a finding automate reported, that scope's re-run of the gap's mutation rows is the re-run in full of its dimension. Pair the profile with the sweep split: mid-loop cycles ride `scope=scoped` sweeps and delta gates; the advance-bound cycle runs the full sweep and the full gate. (A measured seven-cycle story paid a flat ~20 minutes of full three-assessor gating per cycle regardless of fix size: a nine-minute confirming fix gated at the same cost as a 77-minute one.) Under `--light` the gate is already a single trace read, so the profile buys nothing there; omit it.
|
|
116
168
|
|
|
117
169
|
**Record who authored the artifacts the verdict rests on.** Alongside that JSON, log one line naming the provenance of this story's `trace-<id>.md` / `gate-decision-<id>.json`: **`gate-provenance: tea`** when `bmad-testarch-trace` wrote them, or **`gate-provenance: hand-authored`** when the non-web section above did. This is not bookkeeping. The module's central non-negotiable is that completion is decided by a script reading TEA's artifact rather than by the model's judgment, and on the hand-authored path the model *writes the file the gate then reads* — a defensible substitution the non-web section sanctions, but one that is otherwise **indistinguishable** in the decision log and the run report from a verdict grounded in a TEA-authored gate. An Epic can run to completion entirely on the hand-authored path, story after story, with nothing anywhere recording that it did. Provenance is what keeps "the JSON is the truth" honest about *whose* JSON it was; it changes no verdict and gates nothing. Beside it, log `assessed-commit: <sha>` — the commit the assessors measured — because the delta cycle profile's re-check computes its diff from the PREVIOUS gate's `assessed-commit` line, and a story whose record lacks one falls back to the full gate.
|
|
118
170
|
|
|
@@ -124,7 +176,7 @@ Do not recompute TEA thresholds or re-judge `gate_status` — read it as given.
|
|
|
124
176
|
|
|
125
177
|
- **`advance`** (gate_status PASS or WAIVED) → the story passes (an operator-evidence AC still owed its evidence is parked on the ledger first, as `defer` below says). **Sync the story's row in `sprint-status.yaml` to `done` before moving on**: set `development_status[<story_id>]` to `done`, preserving the file's comments and key order, and log the transition in `.decision-log.md`. This write is the run's own, not a sub-skill's: `ucg-dev-story` leaves the row at `review`, and `bmad-code-review` (production only; it does not run under `--light` at all) writes `done` **only** when nothing was left as an action item, so a story that legitimately `defer`s a non-critical finding is left at `in-progress`. Every downstream reader keys on `done`: Stage 1's in-scope rule (`references/ingest-and-scope.md` rule 3), the Epic-level trigger in this very sentence, and `scripts/drive_epic.py`'s progress check. So an advanced story whose row still says `review` is an advanced story the next reader re-drives, and an Epic whose last story advanced without this write never satisfies the Epic-level condition below. Sync it here, at the one place the authoritative verdict is known. In the same write, move the Epic's own `epic-<id>` row from `backlog` to `in-progress` when it still reads `backlog` (never backward, and only when the key exists): `bmad-build` makes that move when a story starts, but the story skills this run delegates to skip it when handed an explicit story key, so without it a finished Epic still reads `backlog`. If the Stop hook left a turn-budget marker for this story (`{workflow.implementation_artifacts}/.escalation-<story_id>.md`) and its count is still at or past the ceiling, do **not** promote it: log the sizing note `references/execute.md` describes under "Promoting the turn-budget marker", and leave the marker in place.
|
|
126
178
|
|
|
127
|
-
**Commit the gate's evidence before the next story starts.** The `done` sync, this story's trace and gate files (`trace-<id>.md`, `gate-decision-<id>.json`, `e2e-trace-summary-<id>.json`, and in production its per-story NFR assessment and test review), its live results file when it has an operator-evidence AC, this gate's ledger rows and the decision log (with the first story, also Stage 3's test-design output) are tracked state written after Execute's commit-at-green. While `.current-story` still names this story, stage those of these paths that exist and are not ignored, by explicit path, in one call, and commit them in a second, before the next story's step 0. For the story whose advance makes every story `done`, make this commit after the Epic-level gate below instead, so it also carries that gate's writes. The commit carries no source or test path, so the story's existing tests-ran marker covers it: reuse that marker as it stands and never rewrite it for this commit. If it must carry an executable path (
|
|
179
|
+
**Commit the gate's evidence before the next story starts.** The `done` sync, this story's trace and gate files (`trace-<id>.md`, `gate-decision-<id>.json`, `e2e-trace-summary-<id>.json`, and in production its per-story NFR assessment and test review), its live results file when it has an operator-evidence AC, this gate's ledger rows and the decision log (with the first story, also Stage 3's test-design output) are tracked state written after Execute's commit-at-green. While `.current-story` still names this story, stage those of these paths that exist and are not ignored, by explicit path, in one call, and commit them in a second, before the next story's step 0. For the story whose advance makes every story `done`, make this commit after the Epic-level gate below instead, so it also carries that gate's writes. The commit carries no source or test path, so the story's existing tests-ran marker covers it: reuse that marker as it stands and never rewrite it for this commit. A repo commit hook that rejects this commit is not `references/execute.md` step 4's case, whose rule for a hook rejection binds the story's own commits. Make a change the hook asks for only in a file the gate itself writes for this commit, `sprint-status.yaml` (its `done` sync) or `.decision-log.md`, then stage it and retry, with the marker as it stands, at most twice for this commit. When the hook asks for a change to any other path, such as a file TEA wrote (an edit there would make its provenance false) or a ledger row (the ledger schema below never rewrites one), or rejects the commit again after the second fix, commit nothing: log the rejection in `.decision-log.md`, write the typed escalation sidecar `escalation-<story_id>.json` (shape in `references/execute.md`, under "Escalation sidecar") with `kind` `evidence-commit-rejected` and a `decision_needed` that names the hook, its message, the paths left uncommitted and this run's id (so a rejection on a later run is a new decision, never one `/ucg-resolve` has closed already), write `.evidence-owed-<story_id>` beside it, listing those paths one per line, and stop before the next story's step 0, as the `escalate` route below stops. The story keeps its `advance` verdict, and its evidence stays staged for the commit a resume makes. Automate's backfill is not among its paths, because it was committed before trace ran ("When automate backfills", under "Backfill the gate evidence"). If it must carry an executable path all the same (a source or test file left uncommitted when trace ran, which this gate therefore never measured: log why), re-run the sweep and write a fresh marker per `references/execute.md` step 5 first. Then refresh `.commit-<story_id>` from `HEAD`, so this story's range in the gate trail includes the evidence commit. A path the project ignores is logged and left out, never forced in with `git add -f`; if nothing is staged, make no commit and leave `.commit-<story_id>` as it is. A resume that finds an advanced story's evidence still uncommitted makes this commit for it before the next story's step 0: point `.current-story` back at that story (its tests-ran marker still passes the guard), commit, and refresh its `.commit-<story_id>`. While that story's `evidence-commit-rejected` escalation is still open, the resume commits nothing and stops on it instead (`references/execute.md`'s resume rule says how it reads one); once `/ucg-resolve` has closed it, the resume makes the commit, under the hook rule above. Once the commit is made, remove `.evidence-owed-<story_id>`: `/ucg-resolve` deletes the sidecar when it closes the escalation, so the marker is what tells a resume, and Stage 1's already-done short-circuit, that the commit is still owed. A resume that finds the marker with the sidecar gone and none of the paths it lists staged or different from `HEAD` (`git diff --quiet HEAD -- <paths>` exits 0 for them) finds the evidence committed already, by the operator or by a run that stopped between its commit and this removal: log the commit that holds it and remove the marker, making no commit. On the story whose advance makes every story `done`, this commit follows the Epic-level gate, so the `epic-<id>` row that gate's `advance` wrote stays staged with the rest, and the run ends `blocked` on that escalation all the same, because its evidence is not committed (`references/finalize.md`). While `.evidence-owed-<story_id>` is on disk, Stage 1's already-done short-circuit does not end that Epic's resume (`references/ingest-and-scope.md`, rule 4), nor does `scripts/drive_epic.py` report the Epic `all-done`: the resume waits on the escalation while it is open, makes the commit once it is closed, and then goes to Finalize with the Epic-level verdict the log holds, without running that gate again. Then move to the next story.
|
|
128
180
|
|
|
129
181
|
When **every story of the Epic is `done`**, run the Epic-level gate: pass the epic's own id via `--story`, **add `--epic-level`**, and read its `gate_status` **only**. TEA produces NFR and test-review **per story, not per epic**, so there is no epic-level aggregate to AND, and every story's own production gate already ANDed its signals before reaching `done`. With `--rollup` the epic id is the sprint-status grouping key and the verdict is a computation over the per-story record; only in the omit-`--rollup` fallback is `--story` resolved against an epic-named artifact (scoped per the `--story` note above) and the roll-up a pure trace read. When the Epic-level gate returns `advance`, or `defer` (whose route advances anyway), set the `epic-<id>` row to `done` (when the key exists), preserving comments and key order, log it, and carry it in the last story's evidence commit. BMAD Method 6.12 has no other writer of an epic row's `done`, and `bmad-retrospective` writes only the `epic-<id>-retrospective` row. Never write it on any other route: the work bound, `partial-complete`, a handoff, a block or an escalation.
|
|
130
182
|
|
|
@@ -148,9 +200,13 @@ uv run {skill-root}/scripts/operator_owed.py --trace-output {workflow.trace_outp
|
|
|
148
200
|
|
|
149
201
|
- **`reloop`** (gate_status FAIL, or a production signal downgraded an advance, or a cap — sweep-scope or delta-profile) → **generalize before fixing.** Before `bmad-correct-course` writes any fix, take the verdict's `reasons` and the review's findings and, for each finding: (1) name, in ONE line, the defect CLASS the finding is an instance of; (2) state whether any earlier review of this story or this Epic already found that class — the previous gate artifacts and `{workflow.implementation_artifacts}/.class-ledger-<epic_id>.md` are the evidence, and they are already on disk, so this is a read, not a judgment call; (3) where the class is a repeat, the fix must close the CLASS — every sibling instance the same seam admits, not the one instance the reviewer demonstrated — and the log records why the class-level fix was not taken when the class was first named. A measured run spent five of six reviews re-finding one class ("an assertion that cannot go red") one seam over each time, with the class-killing move available from day one; the generalize step is what makes cycle N read cycle N-1 instead of re-buying it. Record one `class:` line per finding in this re-loop's `.decision-log.md` entry, and append any class not already present to `.class-ledger-<epic_id>.md` (create on first use; named for the Epic, so a later Epic starts a fresh ledger while this Epic's survives across its own invocations for exactly the resume case; one line per class, `- <class, one line> — first named <story_id>, cycle <n>`; **append-only** — Execute step 1 reads it before every story and re-loop, so a class named once is warned about everywhere after, including across stories, which no per-story artifact can do). **A score that does not move indicts the FIX STRATEGY, not the code:** when two consecutive gate evaluations of this story have moved `review_score` by less than 3 (production — under `--light` no review score exists, so the trigger there is two consecutive re-loops whose findings name any class already in the ledger), the generalize answers above are mandatory *in writing* before any further fix — a re-loop whose findings all name the same class as the previous re-loop's is a re-loop about to be spent badly. **A cap-only re-loop skips the ceremony.** When a cap alone produced the verdict — the `reasons` show only the sweep-scope cap, the delta-profile cap, or both, with no gate FAIL and no failed production signal — there is no finding to generalize and nothing to diagnose: re-run the full three-axis sweep where the sweep cap fired and refresh the marker with `scope=full`, then re-run the gate as the FULL instrument — **dropping `--cycle-profile`**, because the delta cap is lifted by the flag's absence and a re-gate that keeps it caps again forever — writing no `class:` lines, appending nothing to the ledger, and skipping `bmad-correct-course` entirely; such evaluations do not count toward the flat-score tripwire, and a capped CONCERNS keeps its findings for the next gate's own `defer` route — parking them is that gate's job, not this re-loop's. A cap-only re-gate re-measures without re-driving the story, so the turn-ceiling rule below never turns it into `escalate`. Otherwise, run `bmad-correct-course` to diagnose and adjust, then re-run the story (back through Execute, `references/execute.md`) — **within the remaining turn budget**. Re-run the gate after. If the re-loop would exceed `{workflow.max_turns_per_story}`, treat it as `escalate` instead; the count is the `turns` value the Stop hook records in `{workflow.implementation_artifacts}/.budget-<story_id>.json`. When the hook left a turn-budget marker for this story, that escalation's sidecar is the marker's promotion (`references/execute.md`, "Promoting the turn-budget marker").
|
|
150
202
|
|
|
203
|
+
**A red backfill sweep and the `gate-gap:` lines take this route their own way.** A re-loop that ended its cycle before trace ("A red backfill sweep ends the cycle at a re-loop", under "Backfill the gate evidence") takes the full route above, with its red and every open `gate-gap:` line of its cycle that is neither `unmeasured` nor `unkilled` as its findings, and the gate after it is a full one. On a `reloop` from `gate_eval.py` that is not cap-only, each open `gate-gap:` line that is neither `unmeasured` nor `unkilled` is among the findings the generalize step reads, though none of them is a reason for the verdict. An open `gate-gap:` line in a cycle whose re-loop is cap-only is not one of that re-loop's findings, so it gets no `class:` line and no ledger row here: like a capped CONCERNS's findings, it goes to the next gate, by its `gate-gap:` line ("A reported gap stays open until a later pass measures it closed", under the same section). No `gate-gap:` line, on any criterion, sends a story to this route: only the verdict or a red backfill sweep does.
|
|
204
|
+
|
|
151
205
|
- **`escalate`** (gate_status NOT_EVALUATED, meaning the gate could not be read; budget exhausted on a FAIL; or a FAIL only operator evidence can change, above) → **stop.** Do not advance, do not defer the failing item. Record the reason and the verdict JSON in `.decision-log.md`, and write the typed escalation sidecar `{workflow.implementation_artifacts}/escalation-<story_id>.json` (shape in `references/execute.md`, under "Escalation sidecar") so the pending decision is readable without the transcript. In an attended run, surface the blocker to the user. In headless, this is a `blocked` outcome: emit the JSON in Stage 6 (`references/finalize.md`).
|
|
152
206
|
|
|
153
|
-
**INVARIANT
|
|
207
|
+
**INVARIANT: a P0/critical FAIL never defers.** A failing gate (or a P0/P1/overall threshold miss) is `reloop` or `escalate`, never `defer`. Only non-gate-blocking work (CONCERNS, non-critical findings, parked decisions, and the rows open `gate-gap:` lines become) is allowed onto the ledger. If you find yourself about to write a FAIL or a critical finding to the ledger, you are violating the gate: re-loop within budget or escalate instead. A row from a `gate-gap:` line is neither a FAIL nor a critical finding, whatever its criterion's priority, so it belongs on the ledger and never re-loops or escalates the story.
|
|
208
|
+
|
|
209
|
+
An open `gate-gap:` line is neither a FAIL nor a critical finding, whatever its criterion's priority, so the ledger row it becomes on `advance` or `defer` keeps this rule ("A reported gap stays open until a later pass measures it closed", under "Backfill the gate evidence", says why). A gap automate finds on a P0 criterion is backfilled rather than parked ("Automate backfills every gap it finds on a P0 criterion", under the same section), and `gate_eval.py` decides on the test, so a `gate-gap:` line on a P0 criterion is only ever a risk or a gap nobody has measured yet, a backfilled test that was never written, or a backfilled test that did not kill its mutant.
|
|
154
210
|
|
|
155
211
|
If any orchestrated sub-skill blocks on interactive input mid-run, treat it as `escalate` for that story — write the typed escalation sidecar `{workflow.implementation_artifacts}/escalation-<story_id>.json` and stop; do not answer its prompt blind.
|
|
156
212
|
|
|
@@ -166,7 +222,7 @@ Append to `{workflow.deferred_work_path}` (create on first use). One heading per
|
|
|
166
222
|
| d1 | gate | low | <story-id> | <why parked> | <what to do later> | open |
|
|
167
223
|
```
|
|
168
224
|
|
|
169
|
-
- `source` ∈ `gate` (CONCERNS,
|
|
170
|
-
- `severity` ∈ `low`, `med`, `high`. A `high` that maps to a gate FAIL or a P0/critical finding does **not** belong here
|
|
225
|
+
- `source` ∈ `gate` (CONCERNS, an operator-evidence AC's evidence still owed, or an open `gate-gap:` line: a coverage gap automate reported rather than backfilled, a risk on a P0 criterion it left unmeasured, a gap whose rows could not run, a backfilled test that was never written, or a backfilled test that did not kill its mutant), `code-review` (non-critical finding), `nfr` (non-FAIL finding), `decision` (parked decision).
|
|
226
|
+
- `severity` ∈ `low`, `med`, `high`. A `high` that maps to a gate FAIL or a P0/critical finding does **not** belong here: re-loop or escalate it. A row from an open `gate-gap:` line is always `med`: whatever its criterion's priority, it is neither a gate FAIL nor a P0/critical finding, so it belongs here and never re-loops or escalates the story.
|
|
171
227
|
- `id` is unique within the Epic heading (append `d2`, `d3`, … ). Note that the id is unique only WITHIN its heading, so `d1` under two Epics is two different decisions; a project-wide reader must key them by more than the raw column (`/ucg-resolve` mints an id for exactly this reason).
|
|
172
228
|
- `status` ∈ `open`, `resolved`. **This gate only ever appends rows, and never rewrites one** — but the column is not frozen forever: `/ucg-resolve` flips `open` to `resolved` when an operator closes a parked decision, and that single transition is the one sanctioned edit to an existing row. Everything else about a row is immutable once written: never restate its `id`, `source`, `severity`, `story`, `reason` or `suggested_action`, because the ledger is the durable account of what was parked and rewriting it loses the history the Epic close-out reads.
|
|
@@ -35,7 +35,7 @@ Resolution order:
|
|
|
35
35
|
**An operator-scoped subset is recorded by key.** Name each story in it in `.decision-log.md` by its key, as its `sprint-status.yaml` row spells it: Stage 2 passes each to the readiness check as `--story <key>` (`references/preflight.md` step 1b), so the verdict reads those stories and no others.
|
|
36
36
|
|
|
37
37
|
**A decomposed parent owes no pair.** Leave out of `done-without-gate-artifacts` a `done` row the Epic-level `--rollup` reads through its children (`references/gate.md`, "A decomposed parent is read through its children, on its split record"): its split record, `.split-<its key>.json` beside `sprint-status.yaml`, names it as the parent and names children each of which is another story row of this Epic, no chain of records leads from it back to it, and `--story <its key>` would find no trace report, gate decision or trace summary of its own. It never refuses the Epic gate. A decomposed parent with no record is listed like any other `done` row with no pair, and its way out is the record: one `scripts/story_sizing.py` run for its split with `--parent` and `--record` (`references/execute.md`).
|
|
38
|
-
4. **Already-done short-circuit.** If *every* story for the resolved Epic is already `done`, do not carry a no-op Epic into preflight: surface "this Epic is already complete
|
|
38
|
+
4. **Already-done short-circuit.** If *every* story for the resolved Epic is already `done`, do not carry a no-op Epic into preflight: surface "this Epic is already complete: re-run anyway?" and proceed only on a yes (an explicit yes re-scopes all stories deliberately). Headless: emit the same five-key blocked JSON (`report`/`deferred_work` `null`) with `reason` "epic already complete", built by the `scripts/headless_envelope.py` `build_headless_envelope` adapter with `impl_artifacts={workflow.implementation_artifacts}` (the config resolved by this point), so the adapter also writes the envelope to `{workflow.implementation_artifacts}/run-result.json` (same bytes as stdout) for an automator to read. An Epic whose every story reads `done` while one of its stories still owes its evidence commit (`{workflow.implementation_artifacts}/.evidence-owed-<story_id>` is on disk: the gate writes it when it escalates that commit as `evidence-commit-rejected`, and it stays, after `/ucg-resolve` closes the escalation, until the commit is made) is not complete: neither ask nor emit, and resume it as a not-all-done Epic, which re-enters Execute only for that commit (`references/execute.md`'s resume rule, `references/gate.md`'s "Commit the gate's evidence before the next story starts").
|
|
39
39
|
5. If `sprint-status.yaml` is absent (the rollup says so and emits no Epics) or `{planning_artifacts}` has no PRD/ADR for this Epic, that is not a hard stop at this stage — record the gap; Stage 2 preflight reports it mechanically and the semantic scan judges whether the missing planning artifact is a true RED (undecided product/architecture) or a benign absence.
|
|
40
40
|
6. **Cross-file or colliding-epic initiative.** The resolution chain keys on the epic *number*: the rollup and `formalize_check.py --epic` both read it against one `sprint-status.yaml` inside `{workflow.implementation_artifacts}`, and the Stage-5 gate (`gate_eval.py`) resolves per-story trace artifacts *by id* inside `{workflow.trace_output_dir}`. So an initiative that is **not** tracked in that sprint-status (its epic/stories live in a separate planning file, e.g. a starter-template track brought in alongside the product's own epics) — **or** whose epic number **collides** with an epic already in it — cannot be resolved by number without silently hitting the wrong, unrelated epic. Run it as an **isolated track**: point `--impl-artifacts` (and the matching `{workflow.trace_output_dir}` / test-artifacts root) at a dedicated dir that holds *only* this initiative's own `sprint-status.yaml` and stories, and give its stories **disambiguated ids that keep a numeric epic prefix** — a free, high epic *number* no shared-track epic uses (e.g. `epic-90` with stories `90-1`, `90-2`), never an alpha prefix like `st-5`. Keep the disambiguation numeric because the number-keyed readers named above group by the `^\d+` prefix: an alpha-keyed track (`epic-st-5` / `st-5-1`) silently rolls up to an empty `epics` array and formalizes to zero in-scope stories. Pass that isolated `--impl-artifacts` to the sprint-status readers (`preflight_check.py --rollup`, `formalize_check.py`); scope the gate to the isolated track by pointing `gate_eval.py --trace-output` at that dedicated trace dir with `--story <disambiguated id>` (the per-story gate keys on the story id against trace-report filenames and takes no `--impl-artifacts`; the `--rollup` epic gate DOES read a sprint-status, so pass the isolated track's own file via `--sprint-status`, mirroring the `--impl-artifacts` redirection — never the shared one). Record the isolated `--impl-artifacts` and trace-output paths in `.decision-log.md` so Stage 2 and the gate inherit them verbatim; never let a number-keyed read fall through to the shared `sprint-status.yaml`. **Where the redirection is declared depends on whether this project ALSO runs a shared track.** The scalar is project-wide, so setting it redirects *every* run, not this one: on a project that carries both tracks it would point the shared track's runs at the isolated dir too, and `/ucg-status` (which reads the configured scalar as the shared track's) would then describe the wrong run. So: **only when the isolated track is the project's sole track** — a dedicated checkout, or a project with no shared `sprint-status.yaml` — set `implementation_artifacts` (and `trace_output_dir`) to the dedicated dirs in `{project-root}/_bmad/custom/ultracode-goal.toml` and pass the flags as the pass-through of those resolved scalars. **On a project carrying both, stay flag-only** and take the caveat at the end of this rule seriously, because it is then the normal case rather than the exception. Either way the hazard to know is that a flag reaches only the script you type it on, and three consumers read the **scalar** instead: Stage 2's injected `ULTRACODE_IMPL_ARTIFACTS` (`references/preflight.md`), Stage 6's `run-result.json` write (`references/finalize.md`, `impl_artifacts={workflow.implementation_artifacts}`), and `scripts/drive_epic.py`, whose terminal poll is `--impl-artifacts` exactly. Redirect the flags alone and the run's terminal lands in the shared dir while the driver polls the isolated one, so a story that in fact completed is reported as `no-terminal`. If a track must run flag-only, record that in `.decision-log.md` and do not drive it with `drive_epic.py`.
|
|
41
41
|
|
|
@@ -263,6 +263,7 @@ STOP_BLOCKED = "blocked"
|
|
|
263
263
|
STOP_UNKNOWN_STATUS = "unknown-status"
|
|
264
264
|
STOP_NO_PROGRESS = "no-progress"
|
|
265
265
|
STOP_SIGNALLED = "signalled"
|
|
266
|
+
STOP_EVIDENCE_OWED = "evidence-owed"
|
|
266
267
|
|
|
267
268
|
# The stops that mean "this invocation finished its work", i.e. exit 0. A
|
|
268
269
|
# signalled stop is deliberately NOT one: the operator interrupted the drive.
|
|
@@ -321,6 +322,20 @@ def done_ids(stories: list[dict]) -> list[str]:
|
|
|
321
322
|
return [str(row.get("id")) for row in stories if row.get("status") == DONE and row.get("id")]
|
|
322
323
|
|
|
323
324
|
|
|
325
|
+
def evidence_owed_ids(impl_artifacts: Path, story_ids: list[str]) -> list[str]:
|
|
326
|
+
"""The stories whose gate evidence commit is still owed, in the order given.
|
|
327
|
+
|
|
328
|
+
The gate writes `.evidence-owed-<story>` when a repo commit hook's rejection
|
|
329
|
+
of a story's evidence commit escalates as `evidence-commit-rejected`, and
|
|
330
|
+
removes it once a resume makes the commit. `/ucg-resolve` deletes the
|
|
331
|
+
escalation's sidecar when it closes it and leaves this marker, so on the
|
|
332
|
+
Epic's last story it is the one sign that an Epic whose rows all read `done`
|
|
333
|
+
still owes a commit: `references/gate.md`'s advance route.
|
|
334
|
+
"""
|
|
335
|
+
root = Path(impl_artifacts)
|
|
336
|
+
return [story for story in story_ids if (root / f".evidence-owed-{story}").is_file()]
|
|
337
|
+
|
|
338
|
+
|
|
324
339
|
def result_path(impl_artifacts: Path) -> Path:
|
|
325
340
|
"""The pinned terminal path. The filename comes from the envelope module.
|
|
326
341
|
|
|
@@ -979,6 +994,20 @@ def drive(
|
|
|
979
994
|
planned = planned[:limit]
|
|
980
995
|
for position, story_id in enumerate(planned, start=1):
|
|
981
996
|
say(f"[{position}/{len(planned)}] {story_id} would run: {shlex.join(command)}")
|
|
997
|
+
owed = [] if planned else evidence_owed_ids(impl_artifacts, done_ids(stories))
|
|
998
|
+
if owed:
|
|
999
|
+
# A preview must not read a finished Epic where the drive itself
|
|
1000
|
+
# would stop on the owed commit.
|
|
1001
|
+
detail = f"would stop: {STOP_EVIDENCE_OWED} ({', '.join(owed)})"
|
|
1002
|
+
say(f"dry run: {detail}")
|
|
1003
|
+
return _summary(
|
|
1004
|
+
epic_key=epic_key,
|
|
1005
|
+
driven=[],
|
|
1006
|
+
advanced=[],
|
|
1007
|
+
skipped=skipped,
|
|
1008
|
+
stopped_because=STOP_EVIDENCE_OWED,
|
|
1009
|
+
detail=detail,
|
|
1010
|
+
)
|
|
982
1011
|
say(f"dry run: {len(planned)} spawn(s) planned, none started")
|
|
983
1012
|
return _summary(
|
|
984
1013
|
epic_key=epic_key,
|
|
@@ -991,6 +1020,26 @@ def drive(
|
|
|
991
1020
|
|
|
992
1021
|
while True:
|
|
993
1022
|
pending = pending_ids(stories)
|
|
1023
|
+
owed = [] if pending else evidence_owed_ids(impl_artifacts, done_ids(stories))
|
|
1024
|
+
if owed:
|
|
1025
|
+
# Every row is `done`, yet the run that advanced the last of them
|
|
1026
|
+
# stopped `blocked` with its evidence staged. Re-driving finds no
|
|
1027
|
+
# story to spawn, so `all-done` would report a finished Epic, exit 0,
|
|
1028
|
+
# and leave the commit owed for good.
|
|
1029
|
+
detail = (
|
|
1030
|
+
f"{', '.join(owed)}: every story is done, but its gate evidence is not committed "
|
|
1031
|
+
"(.evidence-owed-<story>); close its escalation with /ucg-resolve if it is still open, "
|
|
1032
|
+
"then resume the run once to make the commit"
|
|
1033
|
+
)
|
|
1034
|
+
say(f"stopped: {STOP_EVIDENCE_OWED} - {detail}")
|
|
1035
|
+
return _summary(
|
|
1036
|
+
epic_key=epic_key,
|
|
1037
|
+
driven=driven,
|
|
1038
|
+
advanced=advanced,
|
|
1039
|
+
skipped=skipped,
|
|
1040
|
+
stopped_because=STOP_EVIDENCE_OWED,
|
|
1041
|
+
detail=detail,
|
|
1042
|
+
)
|
|
994
1043
|
if not pending:
|
|
995
1044
|
say(f"stopped: {STOP_ALL_DONE} - every story of epic {epic} is done")
|
|
996
1045
|
return _summary(
|
|
@@ -654,9 +654,11 @@ def commit_sha(impl_artifacts: Path | None, story: str) -> str | None:
|
|
|
654
654
|
"""The story's recorded END commit, or None when the run wrote no such marker.
|
|
655
655
|
|
|
656
656
|
`references/execute.md` step 4 writes `.commit-<story>` once the commit lands,
|
|
657
|
-
step 5 refreshes it after any amend or remediation re-commit,
|
|
658
|
-
|
|
659
|
-
|
|
657
|
+
step 5 refreshes it after any amend or remediation re-commit, the gate
|
|
658
|
+
refreshes it after it commits a test `bmad-testarch-automate` backfilled
|
|
659
|
+
(`references/gate.md`, "When automate backfills"), and the gate's advance
|
|
660
|
+
route refreshes it after its evidence commit, so it names the story's LAST
|
|
661
|
+
commit. Reading it is what lets the trail stop inferring a
|
|
660
662
|
range end from the next story's baseline.
|
|
661
663
|
|
|
662
664
|
The inference it replaces could not be made correct from baselines alone. Two
|