@surlymochan/evalid 0.3.1 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +28 -0
- package/README.md +9 -9
- package/RELEASE_NOTES.md +36 -0
- package/bin/xflow.js +56 -3
- package/docs/launch-dossier.md +2 -2
- package/docs/npm-publish-handoff.md +13 -0
- package/docs/workflow-manual.md +13 -8
- package/package.json +1 -1
- package/specs/workflow.md +2 -2
- package/src/core/workflow-executor.js +16 -0
- package/workflows/corps.yaml +2 -0
- package/workflows/yolo.yaml +25 -17
- package/xflow/atoms/i6a_tdd_run.py +4 -1
- package/xflow/atoms/i6b_tdd_proof_validate.py +20 -1
- package/xflow/atoms/i6c_tdd_quality_review.py +15 -0
- package/xflow/atoms/j6_cr_review.py +39 -0
- package/xflow/atoms/tdd_lib.py +220 -11
- package/xflow/plan/SKILL.md +9 -0
- package/xflow/scripts/common.py +4 -1
- package/xflow/yolo/SKILL.md +20 -3
package/CHANGELOG.md
CHANGED
|
@@ -3,6 +3,34 @@
|
|
|
3
3
|
All notable changes to this project are documented here. Detailed notes live in
|
|
4
4
|
[RELEASE_NOTES.md](RELEASE_NOTES.md).
|
|
5
5
|
|
|
6
|
+
## [Unreleased]
|
|
7
|
+
|
|
8
|
+
## [0.5.0] - 2026-11-06
|
|
9
|
+
|
|
10
|
+
### Added
|
|
11
|
+
- **CR review seal partition (issue #5, P1-2)**: the J6 review proof (`cr/review-0.json`) now splits the changed files into `sealed_unchanged` (bytes still identical to the green TDD proof's sealed hashes) and `seal_drifted`, with a `reread_count`. The yolo skill's verify-consistency guidance uses the partition so the review pass re-reads only drifted files instead of everything the gate already verified. Reporting only — verdicts are unchanged, and without a green seal the partition is `null` and everything is re-read.
|
|
12
|
+
- **Bench Verified set extended to 15 instances (issue #5, P2-2)**: `bench/swebench/instances.json` grows from 12 to 15, adding sympy-15599, sympy-19040 and sympy-24213, and a structural validation of the instance set (`bench/swebench/test_instances.py`) now runs in CI. Gold-patch checks for the three new instances stay pending the benchmark environment.
|
|
13
|
+
- **Bench report aggregates median (IQR) (issue #5, P2-1)**: `bench/swebench/report.py` now aggregates per-arm wall-clock, token and cost results with q1/median/q3 and presents them as median (IQR); reps isolation and the aggregation itself are pinned by `bench/swebench/test_reps_report.py`. Live multi-rep runs stay pending the benchmark environment.
|
|
14
|
+
- **P0-2 acceptance pinned against saved pre-fix patches (issue #5, P0-2)**: regression tests in `bench/swebench/test_diff_filter.py` run the saved pre-fix patches through the evidence filter (`specs/`, `.xflow/`, `.as-xflow/` excluded from solution patches). The saved 9.7 MB evidence-laden patch filters down to 4.3 KB, and every saved patch lands under the 200 KB acceptance line.
|
|
15
|
+
|
|
16
|
+
## [0.4.1] - 2026-11-06
|
|
17
|
+
|
|
18
|
+
### Fixed
|
|
19
|
+
- **TDD gate counts skipped/todo tests again on Node 24**: the gate parsed skip/todo counts only from the TAP reporter's `# skipped / # todo` summary lines, but Node 24's built-in runner defaults to the spec reporter when piped, whose summary lines carry a `ℹ` marker — an all-skipped/todo run exited 0, parsed as zero skips, and was waved through as green. The parser now reads the runner's summary block in both reporter formats (anchored to the summary lines, gated on the full node signature), and fails closed: a node summary whose gate counts cannot be fully parsed classifies as `counts_unknown` and is rejected by proof validation (`tests_counts_unknown`) instead of reading as zero skipped.
|
|
20
|
+
|
|
21
|
+
### Added
|
|
22
|
+
- **Phase cost evidence on every run (issue #5 P1-1)**: the per-phase timing/token report (`src/core/phase-cost.js`, landed in 0.3.0) now also rides on the `workflow_failed` execution-log event — a run that stops mid-way still reports where its time and tokens went — and is persisted as a standalone artifact at `.as-xflow/phase-cost.json` on both completion and failure, so the breakdown is readable without mining the ndjson execution log. Acceptance rerun on the SWE-bench cells stays pending the benchmark environment; the accounting structure (phase sum vs wall clock within 5%) is covered by `test/phase-cost.test.js`.
|
|
23
|
+
|
|
24
|
+
## [0.4.0] - 2026-11-04
|
|
25
|
+
|
|
26
|
+
### Changed
|
|
27
|
+
- **Overhead cut 1 (v0.4 F2, issue #5 P1-2/P1-4)**: the lite (yolo) track seals the TDD red proof immediately after `plan` — the issue/branch ceremony no longer sits between planning and red-proof, and the planning pass now authors the failing test, so plan and red share one agent pass. `xflow workflow run --title "..."` also derives the change id from the title when the branch cannot provide one (previously a hard stop on `main` that pushed agents into manual atom-by-atom driving). TDD snapshots are scoped: inside a git work tree I6a seals only the files that differ from HEAD plus the files the test command names, and I6b materializes the rest from the recorded HEAD at replay — on the SWE-bench sympy tree the sealed manifest drops from 1,543 files / 164 KB to the changed files only (2 files / 197 B for a one-file change), and the snapshot copy from 26.9 MB to ~51 KB. Proofs sealed before this change keep verifying (the scope joins the binding only when present).
|
|
28
|
+
|
|
29
|
+
### Added
|
|
30
|
+
- **CR gate `J6.cr.review` wired into the verify stage (v0.4 F1)**: closes the v0.3.0 known gap where J6 was registered and unit-tested but no workflow phase invoked it. The yolo `verify-consistency` phase and the corps `qa` phase now run J6 on every change — after the spec-delta review, before the stage flips to verify — and declare its verdict record `specs/changes/<change_id>/cr/review-0.json` as a required artifact. A failing verdict (spec misalignment, changed code with no test follow, net assertion decrease, blown loop budget) stops the workflow at verify instead of waving the change through to archive. Orchestration evidence: `test/cr-review-wiring.test.js` drives the real yolo verify phase through `WorkflowExecutor` against a git fixture — an aligned, tested change passes with a verdict record, and an untested change is stopped at the phase.
|
|
31
|
+
- **SWE-bench predictions emitted directly by `run.py` (v0.4 F3)**: the bench harness no longer relies on a one-off converter outside the repo. A stdlib-only `bench/swebench/harness/predictions.py` builds records with exactly `instance_id`, `model_name_or_path`, `model_patch`, re-applies the P0-2 evidence filter (`specs/`, `.xflow/`, `.as-xflow/`) on both paths, and upserts per-arm files so resumable reruns never duplicate lines; `run.py --export-predictions` rebuilds the same files from saved patches with no model calls. Regression: all four pilot arms rebuilt through the new path are byte-identical to the hand-converted pilot files (`cmp` = 0 across 24 instances), and the largest openflow-arm patch stays at 9,650 bytes, well under the 200 KB line.
|
|
32
|
+
- **`xflow init` writes the project `.gitignore` (v0.4 F4)**: init now creates or merges `.gitignore` entries for evalid local state (`.as-xflow/`, `specs/changes/`), appending only the entries that are missing and never modifying existing lines; `.xflow/` stays tracked on purpose (`GOAL.md` is a reviewable project document). Re-running `init --force` tops up projects initialized before v0.4. Closes the v0.3.0 known gap.
|
|
33
|
+
|
|
6
34
|
## [0.3.1] - 2026-11-04
|
|
7
35
|
|
|
8
36
|
### Changed
|
package/README.md
CHANGED
|
@@ -22,12 +22,12 @@ Underneath, a workflow engine (spec → implement → verify loops with per-phas
|
|
|
22
22
|
|
|
23
23
|
_Renamed from OpenFlow in v0.3.1 (2026-11-04); old repository and package links redirect._
|
|
24
24
|
|
|
25
|
-
## Benchmarks:
|
|
25
|
+
## Benchmarks: evalid vs superpowers (2026-10-17)
|
|
26
26
|
|
|
27
|
-
_These results were
|
|
27
|
+
_These results were measured before the rename, when the project was named OpenFlow; product names in this section have been unified to the current names (evalid, swrm), and all versions and figures are the original measurements._
|
|
28
28
|
|
|
29
29
|
For the synthetic and evidence-check results below, the same model (`grok-4.7-high` via local `cursor-agent`) and same
|
|
30
|
-
harness are used for every arm; the SWE-bench block uses DeepSeek via Pi. Pinned:
|
|
30
|
+
harness are used for every arm; the SWE-bench block uses DeepSeek via Pi. Pinned: evalid 0.2.1,
|
|
31
31
|
[superpowers](https://github.com/obra/superpowers) v7.0.0 (`bb92a77`). Scripts, raw results and methodology:
|
|
32
32
|
[`bench/`](bench/README.md) and [`bench/e2e/`](bench/e2e/README.md). Losses are shown as measured.
|
|
33
33
|
|
|
@@ -40,11 +40,11 @@ DeepSeek-V4.1-Flash (`deepseek-flash`), n=1 per cell, 20-minute cap; details and
|
|
|
40
40
|
| --- | --- | --- | --- | --- | --- |
|
|
41
41
|
| Plain agent (baseline) | 3/6 (50%) | 124 s (110–175) | $0.0234 | $0.16 | 6 |
|
|
42
42
|
| superpowers v7.0.0 | 2/6 (33%) | 154 s (136–340) | $0.0285 | $0.31 | 6 |
|
|
43
|
-
|
|
|
44
|
-
|
|
|
43
|
+
| evalid 0.2.1 | 4/6 (67%) | 230 s (200–266) | $0.0412 | $0.23 | 6 |
|
|
44
|
+
| swrm (Pi executor) | 5/6 (83%) | 228 s (187–254) | $0.0377 | $0.21 | 6 |
|
|
45
45
|
|
|
46
46
|
**Caveats:** n=1 on 6 instances, so one or two resolves are within noise. The two workflow tools took about 1.8x the
|
|
47
|
-
baseline's median wall-clock and cost 76% (evalid) / 61% (
|
|
47
|
+
baseline's median wall-clock and cost 76% (evalid) / 61% (swrm) more per task. The first published table undercounted
|
|
48
48
|
pytest because of a shared temp-dir race in grading; it is now fixed and regraded
|
|
49
49
|
([regrade notes](bench/swebench/README.md#regrade-2026-10-18)). On the MiniMax-M3.1-Flash-Preview subset, baseline
|
|
50
50
|
resolved 3/4 and superpowers 2/4; evalid had no clean MiniMax run because the plan's usage cap was hit.
|
|
@@ -58,7 +58,7 @@ results: [#5](https://github.com/surlymochan/evalid/issues/5).
|
|
|
58
58
|
| --- | --- | --- | --- | --- |
|
|
59
59
|
| Plain agent (baseline) | 24/24 (100%) | 55 s (46–132) | 105k / 28k | 24 |
|
|
60
60
|
| superpowers v7.0.0 | 24/24 (100%) | 187 s (121–299) | 312k / 59k | 24 |
|
|
61
|
-
|
|
|
61
|
+
| evalid 0.2.1 | 11/12 (92%) | 985 s (920–1195) | 5.29M / 233k ¹ | 12 ² |
|
|
62
62
|
|
|
63
63
|
On tasks this small the plain agent already succeeds every time, and evalid's full ceremony is the slowest and most
|
|
64
64
|
token-hungry arm. 3 of 12 evalid runs hit the 20-minute cap while still running the workflow; their code already passed.
|
|
@@ -70,10 +70,10 @@ proofs, skipped tests, weak or wrong assertions, contradictory specs, …) and 1
|
|
|
70
70
|
|
|
71
71
|
| Checker | Faulty caught | Honest wrongly rejected | Cost per check |
|
|
72
72
|
| --- | --- | --- | --- |
|
|
73
|
-
|
|
|
73
|
+
| evalid 0.2.1 gates (deterministic code) | 22/25 (88%): original 16/17, held-out 6/8 | 0/11 | ~0.1–0.2 s, no model tokens |
|
|
74
74
|
| superpowers v7.0.0 skills + LLM judge | 25/25 (100%) | 1/11 | one agent call per check |
|
|
75
75
|
| LLM judge, no skills | 25/25 (100%) ³ | 0/11 | one agent call per check |
|
|
76
|
-
|
|
|
76
|
+
| evalid 0.2.0 (before this benchmark, original 24 only) | 10/17 (59%) | 3/7 | – |
|
|
77
77
|
|
|
78
78
|
An LLM reviewer that is explicitly asked to audit an episode, and is allowed to rerun the tests, caught more than evalid's static gates did.
|
|
79
79
|
evalid's gates are deterministic and cost no model calls, but they still miss plausible-but-wrong expected values,
|
package/RELEASE_NOTES.md
CHANGED
|
@@ -1,5 +1,41 @@
|
|
|
1
1
|
# Release Notes
|
|
2
2
|
|
|
3
|
+
## 0.5.0 - Review re-reads only drifted files; the bench set grows to 15
|
|
4
|
+
|
|
5
|
+
Minor release. One product change — the CR review now spends its re-reads only where the seal says bytes moved — plus three bench-hygiene landings from the issue #5 backlog, each pinned by tests rather than by a live run.
|
|
6
|
+
|
|
7
|
+
**Review re-reads only drifted files (#32, issue #5 P1-2).** The J6 review proof (`cr/review-0.json`) now carries a seal partition: every changed file is classified `sealed_unchanged` (bytes still identical to the green TDD proof's sealed hashes) or `seal_drifted`, with a `reread_count`. The yolo verify-consistency guidance consumes the partition, so the review pass re-reads only the drifted files instead of everything the gate already verified. This is reporting only — verdicts are unchanged, and without a green seal the partition is `null` and everything is re-read, exactly as before.
|
|
8
|
+
|
|
9
|
+
**The Verified bench set grows from 12 to 15 instances (#30, issue #5 P2-2).** `bench/swebench/instances.json` adds sympy-15599, sympy-19040 and sympy-24213, and a structural validation of the instance set (`bench/swebench/test_instances.py`) now runs in CI, so a malformed instance entry fails the build instead of a bench run. Gold-patch checks for the three new instances are pending the benchmark environment.
|
|
10
|
+
|
|
11
|
+
**Bench reports now aggregate as median (IQR) (#31, issue #5 P2-1).** `bench/swebench/report.py` aggregates per-arm wall-clock, token and cost results with q1/median/q3 and presents them in median (IQR) form. Reps isolation — repeat runs of the same cell never share grading state — and the aggregation itself are pinned by `bench/swebench/test_reps_report.py`.
|
|
12
|
+
|
|
13
|
+
**P0-2 acceptance is now pinned by regression tests (#29, issue #5 P0-2).** The evidence exclusion itself landed earlier (solution patches drop `specs/`, `.xflow/` and `.as-xflow/`); what was missing was a test that would notice if it regressed. `bench/swebench/test_diff_filter.py` now runs the saved pre-fix patches through the filter: the saved 9.7 MB evidence-laden patch comes out at 4.3 KB, and every saved patch lands under the 200 KB acceptance line.
|
|
14
|
+
|
|
15
|
+
Acceptance debt, stated plainly: the live-run acceptance for P1-2, P2-1 and P2-2 — multi-rep bench runs on the restored environment, including the wall-clock/token lines from issue #5 and gold checks for the new instances — is still pending the benchmark environment. No performance numbers are claimed in this release; what shipped is the reporting, the larger instance set, and the tests that pin their behaviour.
|
|
16
|
+
|
|
17
|
+
## 0.4.1 - Failed runs report phase costs; Node 24 test counts fail closed
|
|
18
|
+
|
|
19
|
+
Patch release. Two fixes, both about the gate trusting evidence it had not actually read.
|
|
20
|
+
|
|
21
|
+
**Failed runs now report where the time and tokens went (#22).** The per-phase cost report landed in 0.3.0 but only rode on successful runs. It now also rides on the `workflow_failed` execution-log event — a run that stops mid-way still reports its per-phase timing/token breakdown — and is persisted as a standalone artifact at `.as-xflow/phase-cost.json` on both completion and failure, so the breakdown is readable without mining the ndjson execution log. The accounting structure (phase sum vs wall clock within 5%) is covered by `test/phase-cost.test.js`. This lands item P1-1 from issue #5.
|
|
22
|
+
|
|
23
|
+
**TDD gate reads test counts correctly on Node 24, and fails closed when it cannot (#23).** The gate parsed skip/todo counts only from the TAP reporter's summary lines. Node 24's built-in runner defaults to the spec reporter when piped, whose summary lines carry a different marker — an all-skipped/todo run on Node 24 exited 0, parsed as zero skips, and was waved through as green. The parser now reads the runner's summary block in both reporter formats (TAP and spec), requires the full node signature before treating a block as a node summary, and anchors summary lines at line start so test names or program output cannot forge counts. When a detected node summary cannot be fully parsed, the run classifies as `counts_unknown` and proof validation rejects it (`tests_counts_unknown`) instead of reading unknown counts as zero. Measured against pinned captured outputs: the new count-parsing tests pass 9/9 under Node 22 and 9/9 under Node 24; the end-to-end TDD proof suite passes 18/18 under both (it was 17/18 on Node 24 before this fix).
|
|
24
|
+
|
|
25
|
+
With P1-1 landed here, the P0/P1 items tracked on the optimization backlog in #5 stand checked off as verified; the remaining longer-horizon items stay tracked there.
|
|
26
|
+
|
|
27
|
+
## 0.4.0 - CR gate wired in, overhead cut 1, predictions direct
|
|
28
|
+
|
|
29
|
+
**The CR gate now runs inside the pipeline (F1, #15).** In 0.3.0 `J6.cr.review` was registered and unit-tested but no workflow invoked it — the release notes said so, and this release closes that gap. The yolo `verify-consistency` phase and the corps `qa` phase run J6 on every change, after the spec-delta review and before the stage flips to verify, and the verdict record `cr/review-0.json` is a required artifact: a failing verdict stops the workflow instead of waving the change through to archive. The wiring is proven end-to-end, not just by unit tests: `test/cr-review-wiring.test.js` drives the real yolo verify phase through `WorkflowExecutor` against a git fixture — an aligned, tested change passes with a verdict record on disk, and a code change with no test follow is stopped at the phase with the `cr_code_without_test` finding.
|
|
30
|
+
|
|
31
|
+
**Overhead cut 1 (F2, #17).** The lite (yolo) track seals the TDD red proof immediately after `plan` — the issue/branch ceremony no longer sits between planning and red-proof, and the planning pass authors the failing test, so plan and red share one agent pass. `xflow workflow run --title "..."` derives the change id from the title when the branch cannot provide one. TDD snapshots are scoped: inside a git work tree, I6a seals only the files that differ from HEAD plus the files the test command names, and I6b materializes the rest from the recorded HEAD at replay. Offline measurements on the SWE-bench sympy tree: the sealed manifest drops from 1,543 files / 164 KB to the changed files only (2 files / 197 B for a one-file change), and the snapshot copy from 26.9 MB to ~51 KB. Proofs sealed before this change keep verifying.
|
|
32
|
+
|
|
33
|
+
Acceptance debt, stated plainly: the v0.4 acceptance line for F2 was a live rerun of the same 6 SWE-bench instances — wall-clock ≤ 1.3× the baseline median, total tokens ≤ 2.0M, resolved ≥ 4/6. The bench generation environment is not currently restored, so those three numbers are still owed; they are not claimed here. If the rerun misses the line when it lands, the remaining overhead work continues as v0.4.1 rather than being declared done.
|
|
34
|
+
|
|
35
|
+
**Predictions come straight out of `run.py` (F3, #16).** The official SWE-bench predictions format (`instance_id`, `model_name_or_path`, `model_patch`) is now produced by the bench harness itself, with the P0-2 evidence filter re-applied on both the live and replay paths — the one-off converter outside the repo is retired. Regression evidence: all four pilot arms rebuilt through `run.py --export-predictions` are byte-identical to the hand-converted pilot files (`cmp` = 0, 24 instances).
|
|
36
|
+
|
|
37
|
+
**`xflow init` writes `.gitignore` (F4, #18).** Init now creates or merges the project `.gitignore` for evalid local state (`.as-xflow/`, `specs/changes/`), appending only missing entries and never modifying existing lines — the first line of defense ahead of the diff filtering added in #8. This closes the second v0.3.0 known gap; both are now done.
|
|
38
|
+
|
|
3
39
|
## 0.3.1 - Renamed: OpenFlow → evalid
|
|
4
40
|
|
|
5
41
|
Same code as 0.3.0, new name and positioning. The project is now **evalid** — a runtime verification layer for coding agents: it seals each test run and replays it from a clean snapshot, so a claim only ships if the evidence replays green. Repository: `surlymochan/evalid` (old `openflow` links redirect); npm package: `evalid`. The `xflow` CLI name is unchanged in this release.
|
package/bin/xflow.js
CHANGED
|
@@ -5079,15 +5079,55 @@ async function handleInit(args) {
|
|
|
5079
5079
|
mkdirSync(configDir, { recursive: true });
|
|
5080
5080
|
writeFileSync(configPath, `${JSON.stringify(config, null, 2)}\n`, 'utf8');
|
|
5081
5081
|
|
|
5082
|
+
const gitignore = ensureXflowGitignore(projectRoot);
|
|
5083
|
+
|
|
5082
5084
|
if (parsedArgs.json) {
|
|
5083
|
-
console.log(JSON.stringify({ ok: true, config_path: configPath, config }, null, 2));
|
|
5085
|
+
console.log(JSON.stringify({ ok: true, config_path: configPath, config, gitignore }, null, 2));
|
|
5084
5086
|
return;
|
|
5085
5087
|
}
|
|
5086
5088
|
|
|
5087
5089
|
console.log(`xflow init: wrote ${configPath}`);
|
|
5090
|
+
if (gitignore.created) {
|
|
5091
|
+
console.log(`xflow init: wrote ${gitignore.path} (ignores ${gitignore.added.join(', ')})`);
|
|
5092
|
+
} else if (gitignore.added.length > 0) {
|
|
5093
|
+
console.log(`xflow init: updated ${gitignore.path} (+ ${gitignore.added.join(', ')})`);
|
|
5094
|
+
} else {
|
|
5095
|
+
console.log(`xflow init: ${gitignore.path} already covers evalid local state`);
|
|
5096
|
+
}
|
|
5088
5097
|
console.log('Next: xflow doctor --project-root <path>');
|
|
5089
5098
|
}
|
|
5090
5099
|
|
|
5100
|
+
// Local runtime state an evalid project should never commit: the .as-xflow
|
|
5101
|
+
// state dir (config, sqlite state, logs, pending gates/tasks, snapshots) and
|
|
5102
|
+
// per-change spec artifacts (proofs, status files). These mirror the entries
|
|
5103
|
+
// this repository keeps in its own .gitignore.
|
|
5104
|
+
const XFLOW_GITIGNORE_ENTRIES = ['.as-xflow/', 'specs/changes/'];
|
|
5105
|
+
const XFLOW_GITIGNORE_HEADER = '# evalid local state (added by xflow init)';
|
|
5106
|
+
|
|
5107
|
+
function normalizeGitignoreEntry(line) {
|
|
5108
|
+
return line.trim().replace(/^\//, '').replace(/\/$/, '');
|
|
5109
|
+
}
|
|
5110
|
+
|
|
5111
|
+
function ensureXflowGitignore(projectRoot) {
|
|
5112
|
+
const gitignorePath = resolve(projectRoot, '.gitignore');
|
|
5113
|
+
|
|
5114
|
+
if (!existsSync(gitignorePath)) {
|
|
5115
|
+
writeFileSync(gitignorePath, `${XFLOW_GITIGNORE_HEADER}\n${XFLOW_GITIGNORE_ENTRIES.join('\n')}\n`, 'utf8');
|
|
5116
|
+
return { path: gitignorePath, created: true, added: [...XFLOW_GITIGNORE_ENTRIES] };
|
|
5117
|
+
}
|
|
5118
|
+
|
|
5119
|
+
const original = readFileSync(gitignorePath, 'utf8');
|
|
5120
|
+
const present = new Set(original.split('\n').map(normalizeGitignoreEntry));
|
|
5121
|
+
const missing = XFLOW_GITIGNORE_ENTRIES.filter((entry) => !present.has(normalizeGitignoreEntry(entry)));
|
|
5122
|
+
if (missing.length === 0) {
|
|
5123
|
+
return { path: gitignorePath, created: false, added: [] };
|
|
5124
|
+
}
|
|
5125
|
+
|
|
5126
|
+
const separator = original.length === 0 ? '' : original.endsWith('\n') ? '\n' : '\n\n';
|
|
5127
|
+
writeFileSync(gitignorePath, `${original}${separator}${XFLOW_GITIGNORE_HEADER}\n${missing.join('\n')}\n`, 'utf8');
|
|
5128
|
+
return { path: gitignorePath, created: false, added: missing };
|
|
5129
|
+
}
|
|
5130
|
+
|
|
5091
5131
|
// ─── goal ────────────────────────────────────────────────────────────────────
|
|
5092
5132
|
|
|
5093
5133
|
async function handleGoal(sub, args) {
|
|
@@ -5421,7 +5461,17 @@ async function handleWorkflow(sub, args) {
|
|
|
5421
5461
|
}
|
|
5422
5462
|
const isDryRun = args.includes('--dry-run');
|
|
5423
5463
|
const input = buildWorkflowInput(parsedArgs);
|
|
5424
|
-
|
|
5464
|
+
let changeId = resolveChangeId({ explicit: parsedArgs.change_id, projectRoot });
|
|
5465
|
+
if (!changeId && input.title) {
|
|
5466
|
+
// Lite-track turn economy (issue #5 P1-2): the documented invocation
|
|
5467
|
+
// is `workflow run yolo --title "..."`, which dead-ended on branches
|
|
5468
|
+
// like `main` that cannot derive a change id and pushed agents into
|
|
5469
|
+
// manual atom-by-atom driving. Derive the id from the title instead.
|
|
5470
|
+
changeId = normalizeChangeId(input.title);
|
|
5471
|
+
if (changeId) {
|
|
5472
|
+
console.log(`Derived --change-id "${changeId}" from --title (pass --change-id to override).`);
|
|
5473
|
+
}
|
|
5474
|
+
}
|
|
5425
5475
|
ensureChangeId(changeId, 'workflow run', projectRoot);
|
|
5426
5476
|
const { load, dryRun } = await import('../src/core/workflow-loader.js');
|
|
5427
5477
|
if (isDryRun) {
|
|
@@ -6667,7 +6717,10 @@ function printInitHelp() {
|
|
|
6667
6717
|
Usage: xflow init [--project-root <path>] [--force] [--json]
|
|
6668
6718
|
|
|
6669
6719
|
Create a project-local .as-xflow/config.json with portable defaults for
|
|
6670
|
-
workflow paths, skill sync paths, and issue routing.
|
|
6720
|
+
workflow paths, skill sync paths, and issue routing. Also creates or updates
|
|
6721
|
+
the project .gitignore so evalid local state (.as-xflow/, specs/changes/)
|
|
6722
|
+
stays out of version control; existing entries are never modified and
|
|
6723
|
+
re-running init adds nothing twice.
|
|
6671
6724
|
|
|
6672
6725
|
Options:
|
|
6673
6726
|
--project-root PATH Initialize a specific project root
|
package/docs/launch-dossier.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# evalid Launch Dossier
|
|
2
2
|
|
|
3
|
-
Generated: 2026-11-
|
|
3
|
+
Generated: 2026-11-05T19:40:37.700Z
|
|
4
4
|
Mode: post_publish
|
|
5
5
|
Verdict: conditionally ready: engine and adoption evidence are strong, registry publication remains required for a splash launch
|
|
6
6
|
Launch ready: not yet
|
|
@@ -117,7 +117,7 @@ Evidence:
|
|
|
117
117
|
- Adoption records: 1; status: pass
|
|
118
118
|
- Goal audit: pass; goal: Keep evalid credible as an open-source agent delivery runtime: repo-owned workflow contracts, executable verification gates, durable project context, and honest benchmark evidence.
|
|
119
119
|
- Package evidence: missing; source: registry_json
|
|
120
|
-
- Package name/version: @surlymochan/evalid@0.
|
|
120
|
+
- Package name/version: @surlymochan/evalid@0.5.0
|
|
121
121
|
|
|
122
122
|
## Passing Evidence
|
|
123
123
|
|
|
@@ -6,6 +6,19 @@ Use this page for the final irreversible public publish step. Everything here
|
|
|
6
6
|
is intentionally explicit because `npm publish --access public` changes public
|
|
7
7
|
distribution state.
|
|
8
8
|
|
|
9
|
+
## Automated publish (trusted publishing)
|
|
10
|
+
|
|
11
|
+
The registry-mutating step normally runs from GitHub Actions, not from a
|
|
12
|
+
local shell: `.github/workflows/npm-publish.yml` publishes via npm trusted
|
|
13
|
+
publishing (OIDC — no stored npm token anywhere) whenever a GitHub Release
|
|
14
|
+
is published, and can also be started manually (workflow_dispatch) to
|
|
15
|
+
publish the version in `package.json` if that version is not on npm yet.
|
|
16
|
+
Before publishing it runs the same `npm run release:pack` gate as the manual
|
|
17
|
+
sequence below, and for release events it refuses to publish when the
|
|
18
|
+
release tag does not match the `package.json` version. The manual sequence
|
|
19
|
+
below remains the fallback for when Actions or the trusted-publisher
|
|
20
|
+
registration on npmjs.com is unavailable.
|
|
21
|
+
|
|
9
22
|
## Current Verified State
|
|
10
23
|
|
|
11
24
|
- Package: `evalid@0.2.0`
|
package/docs/workflow-manual.md
CHANGED
|
@@ -130,7 +130,7 @@
|
|
|
130
130
|
└──────────────────────────────────────────────────────────────────────────┘
|
|
131
131
|
|
|
132
132
|
┌──────────────────────────── Verify / Review ────────────────────────────┐
|
|
133
|
-
│ 23 verify-consistency impl vs proposal/plan/specs/AHA.md 漂移检查
|
|
133
|
+
│ 23 verify-consistency impl vs proposal/plan/specs/AHA.md 漂移检查 + J6 CR 门禁 │
|
|
134
134
|
│ 24 review 对抗性代码评审 + patch-challenge(corps) │
|
|
135
135
|
│ 25 qa 行为 + 视觉验证,截图支持(corps) │
|
|
136
136
|
│ 26 gate_final 镜像 frozen contract,pass/fail/needs-human │
|
|
@@ -195,6 +195,12 @@ CHANGE_ID=<id> xflow workflow run workflows/yolo.yaml
|
|
|
195
195
|
│ exist
|
|
196
196
|
▼
|
|
197
197
|
┌──────────────────────┐
|
|
198
|
+
│ 21a tdd-red │ atoms: I6a(red) + I6b(red) + B3 transition
|
|
199
|
+
│ gate: artifact-verify│◄─── red 必须失败(F2 起紧随 plan 封存,与规划同一轮次)
|
|
200
|
+
└──────────┬───────────┘
|
|
201
|
+
│ exist
|
|
202
|
+
▼
|
|
203
|
+
┌──────────────────────┐
|
|
198
204
|
│ 19 openissue │ atoms: A1 issue + A2 branch + B2b status
|
|
199
205
|
│ gate: deterministic │◄─── E6 post-openissue
|
|
200
206
|
└──────────┬───────────┘
|
|
@@ -207,19 +213,18 @@ CHANGE_ID=<id> xflow workflow run workflows/yolo.yaml
|
|
|
207
213
|
│ pass
|
|
208
214
|
▼
|
|
209
215
|
┌──────────────────────┐
|
|
210
|
-
│ 21a tdd-red │ atoms: I6a(red) + I6b(red)
|
|
211
|
-
│ 21b tdd-green │ atoms: I6a(green) + I6b(green) + I6c quality
|
|
212
|
-
│ gate: artifact-verify│◄─── red 必须失败;green/refactor 必须通过
|
|
213
|
-
└──────────┬───────────┘
|
|
214
|
-
│ exist
|
|
215
|
-
▼
|
|
216
|
-
┌──────────────────────┐
|
|
217
216
|
│ 22 execute │ atoms: B3 transition
|
|
218
217
|
│ gate: human ⏸ │◄─── 等待 xflow gate ack execute
|
|
219
218
|
└──────────┬───────────┘
|
|
220
219
|
│ ack
|
|
221
220
|
▼
|
|
222
221
|
┌──────────────────────┐
|
|
222
|
+
│ 21b tdd-green │ atoms: I6a(green) + I6b(green) + I6c quality
|
|
223
|
+
│ gate: artifact-verify│◄─── green/refactor 必须通过
|
|
224
|
+
└──────────┬───────────┘
|
|
225
|
+
│ exist
|
|
226
|
+
▼
|
|
227
|
+
┌──────────────────────┐
|
|
223
228
|
│ 23 verify-consist. │ atoms: J1 tests.run + J4a structural + J4b delta
|
|
224
229
|
│ gate: deterministic │◄─── J4a 漂移扫描 + J4b spec delta review
|
|
225
230
|
└──────────┬───────────┘
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@surlymochan/evalid",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.5.0",
|
|
4
4
|
"description": "Claimed != verified: a runtime verification layer for coding agents. It seals each test run and replays it from a clean snapshot, with a zero-judge gate pipeline (spec clarification, code review, TDD proof) and the xflow CLI.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"ai-agents",
|
package/specs/workflow.md
CHANGED
|
@@ -79,9 +79,9 @@ No workflow may define a phase not listed here. Phase IDs here are authoritative
|
|
|
79
79
|
|
|
80
80
|
| # | ID | Source | Description |
|
|
81
81
|
|---|----|--------|-------------|
|
|
82
|
-
| 23 | `verify-consistency` | yolo | Impl vs proposal/plan/specs/AHA.md structural drift check |
|
|
82
|
+
| 23 | `verify-consistency` | yolo | Impl vs proposal/plan/specs/AHA.md structural drift check + deterministic CR gate (`J6.cr.review`: spec alignment, test follow, assertion balance, loop budget) |
|
|
83
83
|
| 24 | `review` | agentos | Adversarial code review + patch-challenge |
|
|
84
|
-
| 25 | `qa` | agentos | Behavior + visual verification, screenshot-backed for strict UI work |
|
|
84
|
+
| 25 | `qa` | agentos | Behavior + visual verification, screenshot-backed for strict UI work; runs the same `J6.cr.review` gate (verdict record `cr/review-0.json`) |
|
|
85
85
|
| 26 | `gate_final` | agentos | Mirror frozen contract; decide pass / fail / needs-human |
|
|
86
86
|
|
|
87
87
|
### Archive
|
|
@@ -91,9 +91,24 @@ export class WorkflowExecutor {
|
|
|
91
91
|
phases: this.runtime.phaseCosts,
|
|
92
92
|
wallClockMs,
|
|
93
93
|
});
|
|
94
|
+
this.persistCostReport(this.runtime.costReport);
|
|
94
95
|
return this.runtime.costReport;
|
|
95
96
|
}
|
|
96
97
|
|
|
98
|
+
// The cost report is run evidence (issue #5 P1-1): besides riding on the
|
|
99
|
+
// terminal execution-log event, persist it as a standalone artifact so a
|
|
100
|
+
// reader never has to mine the ndjson log for the per-phase breakdown.
|
|
101
|
+
// Best-effort by design — accounting must never fail the run itself.
|
|
102
|
+
persistCostReport(report) {
|
|
103
|
+
try {
|
|
104
|
+
const dir = resolve(this.projectRoot, '.as-xflow');
|
|
105
|
+
mkdirSync(dir, { recursive: true });
|
|
106
|
+
writeFileSync(resolve(dir, 'phase-cost.json'), JSON.stringify(report, null, 2));
|
|
107
|
+
} catch {
|
|
108
|
+
// The execution-log event still carries the report.
|
|
109
|
+
}
|
|
110
|
+
}
|
|
111
|
+
|
|
97
112
|
async run() {
|
|
98
113
|
this.runtime.workflowStartedAtMs = Date.now();
|
|
99
114
|
this.runtime.currentPhaseTokens = emptyTokenUsage();
|
|
@@ -720,6 +735,7 @@ export class WorkflowExecutor {
|
|
|
720
735
|
phase_label: phase?.label || null,
|
|
721
736
|
reason,
|
|
722
737
|
detail: detail || null,
|
|
738
|
+
cost: this.buildCostReport(),
|
|
723
739
|
});
|
|
724
740
|
} catch {
|
|
725
741
|
// Failure logging is best-effort and must not hide the real stop reason.
|
package/workflows/corps.yaml
CHANGED
|
@@ -428,6 +428,7 @@ phases:
|
|
|
428
428
|
atoms:
|
|
429
429
|
- id: J1.tests.run
|
|
430
430
|
- id: J4b.spec_delta.review
|
|
431
|
+
- id: J6.cr.review
|
|
431
432
|
- id: H5.visual.review
|
|
432
433
|
with: { mode: "qa" }
|
|
433
434
|
- id: E5.artifact.verify
|
|
@@ -441,6 +442,7 @@ phases:
|
|
|
441
442
|
- id: J5.gate.evidence_collect
|
|
442
443
|
artifacts:
|
|
443
444
|
- { path: "specs/changes/${change_id}/qa_acceptance.json", optional: false }
|
|
445
|
+
- { path: "specs/changes/${change_id}/cr/review-0.json", optional: false }
|
|
444
446
|
gate:
|
|
445
447
|
type: artifact-verify
|
|
446
448
|
on_fail: stop
|
package/workflows/yolo.yaml
CHANGED
|
@@ -110,6 +110,28 @@ phases:
|
|
|
110
110
|
type: artifact-verify
|
|
111
111
|
on_fail: stop
|
|
112
112
|
|
|
113
|
+
# F2 (issue #5 P1-2): on the lite track the red proof is sealed immediately
|
|
114
|
+
# after planning, in the same agent pass that authored the plan and the
|
|
115
|
+
# failing test. The issue/branch ceremony (openissue, set-in-progress) no
|
|
116
|
+
# longer sits between planning and red-proof, where it forced a separate
|
|
117
|
+
# exploration pass; it still completes before execute.
|
|
118
|
+
- id: tdd-red
|
|
119
|
+
label: "21a · TDD red"
|
|
120
|
+
catalog_ref: 21
|
|
121
|
+
required: true
|
|
122
|
+
atoms:
|
|
123
|
+
- id: I6a.tdd.run
|
|
124
|
+
with: { phase: "red", test_command: "${input.tdd_red_command}", expect: "fail" }
|
|
125
|
+
- id: I6b.tdd.proof_validate
|
|
126
|
+
with: { phase: "red", tdd_mode: "${input.tdd_mode}" }
|
|
127
|
+
- id: B3.status.transition
|
|
128
|
+
with: { to_stage: "tdd" }
|
|
129
|
+
artifacts:
|
|
130
|
+
- { path: "specs/changes/${change_id}/tdd/red-0.json", optional: false }
|
|
131
|
+
gate:
|
|
132
|
+
type: artifact-verify
|
|
133
|
+
on_fail: stop
|
|
134
|
+
|
|
113
135
|
- id: openissue
|
|
114
136
|
label: "19 · Open issue + branch"
|
|
115
137
|
catalog_ref: 19
|
|
@@ -145,23 +167,6 @@ phases:
|
|
|
145
167
|
with: { phase: "pre-exec" }
|
|
146
168
|
on_fail: stop
|
|
147
169
|
|
|
148
|
-
- id: tdd-red
|
|
149
|
-
label: "21a · TDD red"
|
|
150
|
-
catalog_ref: 21
|
|
151
|
-
required: true
|
|
152
|
-
atoms:
|
|
153
|
-
- id: I6a.tdd.run
|
|
154
|
-
with: { phase: "red", test_command: "${input.tdd_red_command}", expect: "fail" }
|
|
155
|
-
- id: I6b.tdd.proof_validate
|
|
156
|
-
with: { phase: "red", tdd_mode: "${input.tdd_mode}" }
|
|
157
|
-
- id: B3.status.transition
|
|
158
|
-
with: { to_stage: "tdd" }
|
|
159
|
-
artifacts:
|
|
160
|
-
- { path: "specs/changes/${change_id}/tdd/red-0.json", optional: false }
|
|
161
|
-
gate:
|
|
162
|
-
type: artifact-verify
|
|
163
|
-
on_fail: stop
|
|
164
|
-
|
|
165
170
|
- id: execute
|
|
166
171
|
label: "22 · Execute"
|
|
167
172
|
catalog_ref: 22
|
|
@@ -199,12 +204,15 @@ phases:
|
|
|
199
204
|
- id: J1.tests.run
|
|
200
205
|
- id: J4a.spec_consistency.structural
|
|
201
206
|
- id: J4b.spec_delta.review
|
|
207
|
+
- id: J6.cr.review
|
|
202
208
|
- id: B3.status.transition
|
|
203
209
|
with: { to_stage: "verify" }
|
|
204
210
|
- id: B2b.status.write
|
|
205
211
|
with:
|
|
206
212
|
fields:
|
|
207
213
|
verification_status: "running"
|
|
214
|
+
artifacts:
|
|
215
|
+
- { path: "specs/changes/${change_id}/cr/review-0.json", optional: false }
|
|
208
216
|
gate:
|
|
209
217
|
type: deterministic
|
|
210
218
|
atom: J4a.spec_consistency.structural
|
|
@@ -64,7 +64,9 @@ def main():
|
|
|
64
64
|
|
|
65
65
|
passed = exit_code == 0
|
|
66
66
|
combined = f"{stdout}\n{stderr}"
|
|
67
|
-
file_hashes = tdd_lib.write_snapshot(
|
|
67
|
+
file_hashes, snapshot_scope = tdd_lib.write_snapshot(
|
|
68
|
+
project, args.change_id, args.phase, args.attempt, command=args.test_command
|
|
69
|
+
)
|
|
68
70
|
proof = {
|
|
69
71
|
'phase': args.phase,
|
|
70
72
|
'attempt': args.attempt,
|
|
@@ -79,6 +81,7 @@ def main():
|
|
|
79
81
|
'output_sha256': tdd_lib.output_digest(stdout, stderr),
|
|
80
82
|
'git_head': tdd_lib.git_head(project),
|
|
81
83
|
'file_hashes': file_hashes,
|
|
84
|
+
'snapshot_scope': snapshot_scope,
|
|
82
85
|
'failure_kind': 'timeout' if timed_out else tdd_lib.classify_output(combined, passed),
|
|
83
86
|
'recorded_at': datetime.now(timezone.utc).isoformat(),
|
|
84
87
|
}
|
|
@@ -95,8 +95,17 @@ def main():
|
|
|
95
95
|
if found is None or found != proof.get('file_hashes'):
|
|
96
96
|
emit({'ok': False, 'verdict': 'snapshot_mismatch', 'phase': args.phase}, 1)
|
|
97
97
|
|
|
98
|
+
scope = proof.get('snapshot_scope')
|
|
99
|
+
scoped = isinstance(scope, dict) and bool(proof.get('git_head'))
|
|
98
100
|
try:
|
|
99
|
-
|
|
101
|
+
if scoped:
|
|
102
|
+
actual_exit, actual_text = tdd_lib.reexec_scoped_snapshot(
|
|
103
|
+
project, destination, proof.get('command') or '', proof['git_head'], scope, timeout=120
|
|
104
|
+
)
|
|
105
|
+
else:
|
|
106
|
+
actual_exit, actual_text = tdd_lib.reexec_snapshot(destination, proof.get('command') or '', timeout=120)
|
|
107
|
+
except RuntimeError as exc:
|
|
108
|
+
emit({'ok': False, 'verdict': 'replay_source_missing', 'phase': args.phase, 'error': str(exc)[:300]}, 1)
|
|
100
109
|
except subprocess.TimeoutExpired:
|
|
101
110
|
emit({'ok': False, 'verdict': 'reexec_timeout', 'phase': args.phase}, 1)
|
|
102
111
|
|
|
@@ -127,6 +136,13 @@ def main():
|
|
|
127
136
|
path = project / rel
|
|
128
137
|
if not path.is_file() or tdd_lib.sha256_bytes(path.read_bytes()) != digest:
|
|
129
138
|
mismatched.append(rel)
|
|
139
|
+
if scoped:
|
|
140
|
+
# Test files outside the sealed scope were pinned to the recorded
|
|
141
|
+
# HEAD at seal time; any that have drifted since are unsealed edits.
|
|
142
|
+
sealed = set(proof.get('file_hashes', {}))
|
|
143
|
+
for rel in tdd_lib.changed_since(project, proof['git_head']):
|
|
144
|
+
if rel not in sealed and tdd_lib.is_test_file(rel):
|
|
145
|
+
mismatched.append(rel)
|
|
130
146
|
if mismatched:
|
|
131
147
|
emit({
|
|
132
148
|
'ok': False,
|
|
@@ -138,6 +154,9 @@ def main():
|
|
|
138
154
|
if proof.get('failure_kind') == 'skipped' or tdd_lib.output_non_result(actual_text):
|
|
139
155
|
emit({'ok': False, 'verdict': 'tests_skipped_or_todo', 'phase': args.phase}, 1)
|
|
140
156
|
|
|
157
|
+
if proof.get('failure_kind') == 'counts_unknown' or tdd_lib.output_counts_unknown(actual_text):
|
|
158
|
+
emit({'ok': False, 'verdict': 'tests_counts_unknown', 'phase': args.phase}, 1)
|
|
159
|
+
|
|
141
160
|
expected_passed = expected_result(args.phase)
|
|
142
161
|
if expected_passed is not None and proof.get('passed') is not expected_passed:
|
|
143
162
|
emit({
|
|
@@ -95,6 +95,15 @@ def proof_skipped(proof: dict) -> bool:
|
|
|
95
95
|
return tdd_lib.output_non_result(blob)
|
|
96
96
|
|
|
97
97
|
|
|
98
|
+
def proof_counts_unknown(proof: dict) -> bool:
|
|
99
|
+
if not proof:
|
|
100
|
+
return False
|
|
101
|
+
if proof.get("failure_kind") == "counts_unknown":
|
|
102
|
+
return True
|
|
103
|
+
blob = f"{proof.get('stdout_tail') or ''}\n{proof.get('stderr_tail') or ''}"
|
|
104
|
+
return tdd_lib.output_counts_unknown(blob)
|
|
105
|
+
|
|
106
|
+
|
|
98
107
|
def symbols_in(project_root: Path, code_files: list[str]) -> set[str]:
|
|
99
108
|
symbols = set()
|
|
100
109
|
for relative in code_files:
|
|
@@ -298,6 +307,12 @@ def main() -> None:
|
|
|
298
307
|
"code": "tests_skipped_or_todo",
|
|
299
308
|
"message": "A recorded run skipped or marked todo the tests it claims to have executed.",
|
|
300
309
|
})
|
|
310
|
+
if proof_counts_unknown(red_proof) or proof_counts_unknown(green_proof):
|
|
311
|
+
findings.append({
|
|
312
|
+
"severity": "fail",
|
|
313
|
+
"code": "tests_counts_unknown",
|
|
314
|
+
"message": "A recorded run's summary counts could not be fully parsed, so its skip/todo counts are unknown and the run cannot be verified.",
|
|
315
|
+
})
|
|
301
316
|
symbols = symbols_in(project_root, code_files)
|
|
302
317
|
if symbols and test_files:
|
|
303
318
|
for relative in test_files:
|
|
@@ -26,6 +26,11 @@ Checks (findings are structured JSON, severity fail | warn):
|
|
|
26
26
|
(--max-files / --max-lines, or status.json cr_max_files /
|
|
27
27
|
cr_max_lines). Over budget is a warn with a split-the-spec
|
|
28
28
|
recommendation; over 2x budget fails.
|
|
29
|
+
5. Seal partition (reporting only, never a verdict input) - the proof
|
|
30
|
+
splits the changed files into those still byte-identical to the
|
|
31
|
+
green TDD seal (file_hashes in tdd/green-0.json) and those that
|
|
32
|
+
drifted or were never sealed. A review pass re-reads only the
|
|
33
|
+
drifted set; the sealed set is exactly what the TDD gate verified.
|
|
29
34
|
|
|
30
35
|
Self-contained on purpose: this atom does not import tdd_lib so it can
|
|
31
36
|
run standalone in a reviewed tree. The file/symbol/assertion semantics
|
|
@@ -36,6 +41,7 @@ directory is excluded from the reviewed diff.
|
|
|
36
41
|
"""
|
|
37
42
|
import argparse
|
|
38
43
|
import fnmatch
|
|
44
|
+
import hashlib
|
|
39
45
|
import json
|
|
40
46
|
import os
|
|
41
47
|
import re
|
|
@@ -200,6 +206,38 @@ def assertion_delta(project_root: Path, diff_ref: str, test_files: list, deleted
|
|
|
200
206
|
return added, removed
|
|
201
207
|
|
|
202
208
|
|
|
209
|
+
def seal_partition(project_root: Path, change_dir: Path, files: list):
|
|
210
|
+
"""Split changed files by whether they still match the green seal.
|
|
211
|
+
|
|
212
|
+
The green TDD proof (tdd/green-0.json) records the sha256 of every
|
|
213
|
+
file I6a sealed when the proof was produced (issue #5, P1-2). A
|
|
214
|
+
changed file whose current bytes still hash to the sealed value is
|
|
215
|
+
byte-for-byte what the TDD gate verified, so a review pass does not
|
|
216
|
+
need to re-read it; anything else — drifted after the seal, or never
|
|
217
|
+
sealed — must be re-read. Returns None when no usable green seal
|
|
218
|
+
exists (the review then re-reads everything).
|
|
219
|
+
"""
|
|
220
|
+
green = read_json(change_dir / "tdd" / "green-0.json")
|
|
221
|
+
hashes = green.get("file_hashes")
|
|
222
|
+
if not isinstance(hashes, dict) or not hashes:
|
|
223
|
+
return None
|
|
224
|
+
sealed, drifted = [], []
|
|
225
|
+
for relative in files:
|
|
226
|
+
full = project_root / relative
|
|
227
|
+
digest = None
|
|
228
|
+
if full.is_file():
|
|
229
|
+
digest = hashlib.sha256(full.read_bytes()).hexdigest()
|
|
230
|
+
if digest is not None and hashes.get(relative) == digest:
|
|
231
|
+
sealed.append(relative)
|
|
232
|
+
else:
|
|
233
|
+
drifted.append(relative)
|
|
234
|
+
return {
|
|
235
|
+
"sealed_unchanged": sealed,
|
|
236
|
+
"seal_drifted": drifted,
|
|
237
|
+
"reread_count": len(drifted),
|
|
238
|
+
}
|
|
239
|
+
|
|
240
|
+
|
|
203
241
|
def tokenize(text: str) -> set:
|
|
204
242
|
tokens = set()
|
|
205
243
|
for raw in re.findall(r"[A-Za-z_][A-Za-z0-9_]*", text):
|
|
@@ -416,6 +454,7 @@ def main() -> None:
|
|
|
416
454
|
"changed_files": all_files,
|
|
417
455
|
"code_files": code_files,
|
|
418
456
|
"test_files": test_files,
|
|
457
|
+
"seal_partition": seal_partition(project_root, change_dir, all_files),
|
|
419
458
|
"spec_aligned_files": aligned_files,
|
|
420
459
|
"assertions": {"added": added_assertions, "removed": removed_assertions},
|
|
421
460
|
"budget": budget,
|
package/xflow/atoms/tdd_lib.py
CHANGED
|
@@ -8,6 +8,13 @@ fails unless the snapshot really exits that way and the binding is rebuilt.
|
|
|
8
8
|
An agent who can write the tree can still record a genuine failing snapshot
|
|
9
9
|
and later replace the working tree; the digest is not a trusted third party.
|
|
10
10
|
Byte-identical stdout is not required, because runners print timings.
|
|
11
|
+
|
|
12
|
+
Scoped sealing (issue #5 P1-4): inside a git work tree the snapshot copies
|
|
13
|
+
only the files that differ from HEAD plus the files the test command names.
|
|
14
|
+
Every other file is pinned to the recorded HEAD: I6b materializes it from
|
|
15
|
+
`git archive` at replay time, so the replayed tree is byte-identical to the
|
|
16
|
+
sealed one while the stored manifest stays proportional to the change.
|
|
17
|
+
Outside a git work tree the legacy full-tree copy is used unchanged.
|
|
11
18
|
"""
|
|
12
19
|
import hashlib
|
|
13
20
|
import json
|
|
@@ -44,6 +51,7 @@ BINDING_FIELDS = (
|
|
|
44
51
|
"stderr_tail",
|
|
45
52
|
"git_head",
|
|
46
53
|
"file_hashes",
|
|
54
|
+
"snapshot_scope",
|
|
47
55
|
"failure_kind",
|
|
48
56
|
"recorded_at",
|
|
49
57
|
)
|
|
@@ -134,9 +142,16 @@ def sha256_bytes(data: bytes) -> str:
|
|
|
134
142
|
return hashlib.sha256(data).hexdigest()
|
|
135
143
|
|
|
136
144
|
|
|
145
|
+
# Optional binding fields join the payload only when present, so proofs
|
|
146
|
+
# sealed before the field existed keep verifying byte-for-byte.
|
|
147
|
+
OPTIONAL_BINDING_FIELDS = {"snapshot_scope"}
|
|
148
|
+
|
|
149
|
+
|
|
137
150
|
def binding_payload(proof: dict) -> dict:
|
|
138
151
|
payload = {}
|
|
139
152
|
for field in BINDING_FIELDS:
|
|
153
|
+
if field in OPTIONAL_BINDING_FIELDS and proof.get(field) is None:
|
|
154
|
+
continue
|
|
140
155
|
payload[field] = proof.get(field)
|
|
141
156
|
return payload
|
|
142
157
|
|
|
@@ -187,20 +202,128 @@ def iter_project_files(project_root: Path, skip_prefix: str):
|
|
|
187
202
|
yield rel, full
|
|
188
203
|
|
|
189
204
|
|
|
190
|
-
def
|
|
205
|
+
def _git_status_paths(project_root: Path):
|
|
206
|
+
"""(changed, deleted) path sets from `git status --porcelain -z`.
|
|
207
|
+
|
|
208
|
+
Returns None when the project is not a git work tree. Renames report the
|
|
209
|
+
new path as changed and the old path as deleted.
|
|
210
|
+
"""
|
|
211
|
+
result = subprocess.run(
|
|
212
|
+
["git", "status", "--porcelain=v1", "-z", "--untracked-files=all"],
|
|
213
|
+
cwd=project_root,
|
|
214
|
+
capture_output=True,
|
|
215
|
+
)
|
|
216
|
+
if result.returncode != 0:
|
|
217
|
+
return None
|
|
218
|
+
entries = result.stdout.decode("utf-8", errors="replace").split("\0")
|
|
219
|
+
changed: set[str] = set()
|
|
220
|
+
deleted: set[str] = set()
|
|
221
|
+
index = 0
|
|
222
|
+
while index < len(entries):
|
|
223
|
+
entry = entries[index]
|
|
224
|
+
index += 1
|
|
225
|
+
if len(entry) < 4:
|
|
226
|
+
continue
|
|
227
|
+
code, rel = entry[:2], entry[3:]
|
|
228
|
+
if "R" in code or "C" in code:
|
|
229
|
+
old = entries[index] if index < len(entries) else ""
|
|
230
|
+
index += 1
|
|
231
|
+
if old:
|
|
232
|
+
deleted.add(old)
|
|
233
|
+
if "D" in code:
|
|
234
|
+
deleted.add(rel)
|
|
235
|
+
elif rel:
|
|
236
|
+
changed.add(rel)
|
|
237
|
+
return changed, deleted
|
|
238
|
+
|
|
239
|
+
|
|
240
|
+
def changed_since(project_root: Path, ref: str) -> set[str]:
|
|
241
|
+
"""Paths whose working-tree content differs from `ref`, plus untracked."""
|
|
242
|
+
paths: set[str] = set()
|
|
243
|
+
diff = subprocess.run(
|
|
244
|
+
["git", "diff", "--name-only", "-z", ref],
|
|
245
|
+
cwd=project_root,
|
|
246
|
+
capture_output=True,
|
|
247
|
+
)
|
|
248
|
+
if diff.returncode == 0:
|
|
249
|
+
paths.update(p for p in diff.stdout.decode("utf-8", errors="replace").split("\0") if p)
|
|
250
|
+
others = subprocess.run(
|
|
251
|
+
["git", "ls-files", "--others", "--exclude-standard", "-z"],
|
|
252
|
+
cwd=project_root,
|
|
253
|
+
capture_output=True,
|
|
254
|
+
)
|
|
255
|
+
if others.returncode == 0:
|
|
256
|
+
paths.update(p for p in others.stdout.decode("utf-8", errors="replace").split("\0") if p)
|
|
257
|
+
return paths
|
|
258
|
+
|
|
259
|
+
|
|
260
|
+
def snapshot_scope(project_root: Path, change_id: str, command: str | None) -> dict | None:
|
|
261
|
+
"""Files the proof actually depends on (issue #5 P1-4).
|
|
262
|
+
|
|
263
|
+
Scoped sealing only applies inside a git work tree: every file that
|
|
264
|
+
differs from HEAD is sealed by copy, and every other file is pinned to
|
|
265
|
+
the recorded HEAD and materialized from git at replay time. Returns None
|
|
266
|
+
when no HEAD exists to pin the rest of the tree to (legacy full copy).
|
|
267
|
+
"""
|
|
268
|
+
if git_head(project_root) is None:
|
|
269
|
+
return None
|
|
270
|
+
status = _git_status_paths(project_root)
|
|
271
|
+
if status is None:
|
|
272
|
+
return None
|
|
273
|
+
changed, deleted = status
|
|
274
|
+
evidence_prefix = (Path("specs") / "changes" / change_id).as_posix() + "/"
|
|
275
|
+
|
|
276
|
+
def excluded(rel: str) -> bool:
|
|
277
|
+
return rel.startswith(evidence_prefix)
|
|
278
|
+
|
|
279
|
+
def noise(rel: str) -> bool:
|
|
280
|
+
return any(part in SKIP_DIR_NAMES for part in rel.split("/"))
|
|
281
|
+
|
|
282
|
+
files = {rel for rel in changed if not excluded(rel) and not noise(rel)}
|
|
283
|
+
for rel in test_paths(command or ""):
|
|
284
|
+
if not excluded(rel) and (project_root / rel).is_file():
|
|
285
|
+
files.add(rel)
|
|
286
|
+
return {
|
|
287
|
+
"version": 1,
|
|
288
|
+
"files": sorted(files),
|
|
289
|
+
"deleted": sorted(rel for rel in deleted if not excluded(rel) and not noise(rel)),
|
|
290
|
+
}
|
|
291
|
+
|
|
292
|
+
|
|
293
|
+
def write_snapshot(project_root: Path, change_id: str, phase: str, attempt: int,
|
|
294
|
+
command: str | None = None) -> tuple[dict, dict | None]:
|
|
295
|
+
"""Seal the tree state a proof ran against.
|
|
296
|
+
|
|
297
|
+
With a scope (git work tree), only the scoped files are copied and
|
|
298
|
+
hashed; the manifest stays proportional to the change, not the tree.
|
|
299
|
+
Without one, the legacy full-tree copy is used.
|
|
300
|
+
"""
|
|
191
301
|
skip_prefix = (Path("specs") / "changes" / change_id / "tdd").as_posix()
|
|
192
302
|
destination = snapshot_dir(project_root, change_id, phase, attempt)
|
|
193
303
|
if destination.exists():
|
|
194
304
|
shutil.rmtree(destination)
|
|
195
305
|
destination.mkdir(parents=True, exist_ok=True)
|
|
306
|
+
scope = snapshot_scope(project_root, change_id, command)
|
|
196
307
|
hashes = {}
|
|
197
|
-
|
|
308
|
+
if scope is None:
|
|
309
|
+
for rel, full in iter_project_files(project_root, skip_prefix):
|
|
310
|
+
data = full.read_bytes()
|
|
311
|
+
hashes[rel] = sha256_bytes(data)
|
|
312
|
+
target = destination / rel
|
|
313
|
+
target.parent.mkdir(parents=True, exist_ok=True)
|
|
314
|
+
target.write_bytes(data)
|
|
315
|
+
return hashes, None
|
|
316
|
+
for rel in scope["files"]:
|
|
317
|
+
full = project_root / rel
|
|
318
|
+
if full.is_symlink() or not full.is_file():
|
|
319
|
+
continue
|
|
198
320
|
data = full.read_bytes()
|
|
199
321
|
hashes[rel] = sha256_bytes(data)
|
|
200
322
|
target = destination / rel
|
|
201
323
|
target.parent.mkdir(parents=True, exist_ok=True)
|
|
202
324
|
target.write_bytes(data)
|
|
203
|
-
|
|
325
|
+
scope = {**scope, "files": sorted(hashes)}
|
|
326
|
+
return hashes, scope
|
|
204
327
|
|
|
205
328
|
|
|
206
329
|
def snapshot_tree_hashes(destination: Path):
|
|
@@ -220,27 +343,68 @@ def output_digest(stdout: str, stderr: str) -> str:
|
|
|
220
343
|
return hashlib.sha256(f"{stdout}\0{stderr}".encode("utf-8")).hexdigest()
|
|
221
344
|
|
|
222
345
|
|
|
346
|
+
# Node's built-in test runner ends a run with a summary block of
|
|
347
|
+
# "<marker> <label> <count>" lines — the runner's own structured tally.
|
|
348
|
+
# The marker depends on the reporter: "#" for the TAP reporter (the piped
|
|
349
|
+
# default up to Node 22) and "ℹ" for the spec reporter (the piped default
|
|
350
|
+
# from Node 24, and selectable on any version). The gate reads only these
|
|
351
|
+
# line-anchored summary lines, never free-floating text, so test names or
|
|
352
|
+
# program output cannot forge a count.
|
|
353
|
+
_NODE_GATE_LABELS = ("pass", "fail", "skipped", "todo")
|
|
354
|
+
# Lines every node:test summary block carries and other TAP emitters do
|
|
355
|
+
# not; their joint presence is what marks a block as a node summary.
|
|
356
|
+
_NODE_SIGNATURE_LABELS = ("tests", "suites", "cancelled")
|
|
357
|
+
|
|
358
|
+
|
|
223
359
|
def _summary_count(text: str, label: str):
|
|
224
|
-
match = re.search(rf"#\s
|
|
360
|
+
match = re.search(rf"^[#\u2139]\s*{label}\s+(\d+)\b", text, re.MULTILINE)
|
|
225
361
|
if not match:
|
|
226
362
|
return None
|
|
227
363
|
return int(match.group(1))
|
|
228
364
|
|
|
229
365
|
|
|
366
|
+
def node_run_counts(text: str):
|
|
367
|
+
"""Summary counts from a node:test run, or None when there is no summary.
|
|
368
|
+
|
|
369
|
+
A dict is returned once a block carries the full node signature
|
|
370
|
+
(tests/suites/cancelled). A gate label the block does not carry maps
|
|
371
|
+
to None — unknown, never silently zero: a truncated or reshaped
|
|
372
|
+
block must not read as "zero skipped".
|
|
373
|
+
"""
|
|
374
|
+
if not text:
|
|
375
|
+
return None
|
|
376
|
+
if any(_summary_count(text, label) is None for label in _NODE_SIGNATURE_LABELS):
|
|
377
|
+
return None
|
|
378
|
+
labels = _NODE_SIGNATURE_LABELS + _NODE_GATE_LABELS
|
|
379
|
+
return {label: _summary_count(text, label) for label in labels}
|
|
380
|
+
|
|
381
|
+
|
|
382
|
+
def output_counts_unknown(text: str) -> bool:
|
|
383
|
+
"""True when a node:test summary is present but a gate count is unreadable."""
|
|
384
|
+
counts = node_run_counts(text)
|
|
385
|
+
if counts is None:
|
|
386
|
+
return False
|
|
387
|
+
return any(counts[label] is None for label in _NODE_GATE_LABELS)
|
|
388
|
+
|
|
389
|
+
|
|
230
390
|
def output_non_result(text: str) -> bool:
|
|
231
391
|
"""True when the runner reported skip/todo/xfail and no passing test."""
|
|
232
392
|
if not text:
|
|
233
393
|
return False
|
|
234
|
-
|
|
235
|
-
|
|
236
|
-
|
|
237
|
-
|
|
238
|
-
|
|
239
|
-
|
|
394
|
+
counts = node_run_counts(text)
|
|
395
|
+
node_failed = None
|
|
396
|
+
if counts is not None:
|
|
397
|
+
skipped = counts["skipped"]
|
|
398
|
+
todo = counts["todo"]
|
|
399
|
+
passed = counts["pass"]
|
|
400
|
+
node_failed = counts["fail"]
|
|
401
|
+
if None not in (skipped, todo, passed, node_failed):
|
|
402
|
+
if skipped + todo > 0 and passed == 0 and node_failed == 0:
|
|
403
|
+
return True
|
|
240
404
|
py_skipped = sum(int(item) for item in re.findall(r"\b(\d+)\s+skipped\b", text))
|
|
241
405
|
py_skipped += sum(int(item) for item in re.findall(r"\b(\d+)\s+xfailed\b", text))
|
|
242
406
|
py_passed = sum(int(item) for item in re.findall(r"\b(\d+)\s+passed\b", text))
|
|
243
|
-
if py_skipped > 0 and py_passed == 0 and (
|
|
407
|
+
if py_skipped > 0 and py_passed == 0 and (node_failed or 0) == 0:
|
|
244
408
|
return True
|
|
245
409
|
return False
|
|
246
410
|
|
|
@@ -249,6 +413,11 @@ def classify_output(text: str, passed: bool) -> str:
|
|
|
249
413
|
if output_non_result(text):
|
|
250
414
|
return "skipped"
|
|
251
415
|
if passed:
|
|
416
|
+
# Fail closed: a node:test summary that cannot be fully read does
|
|
417
|
+
# not prove a clean pass — the skip/todo counts may sit in the
|
|
418
|
+
# part that failed to parse, so the run is unverifiable, not green.
|
|
419
|
+
if output_counts_unknown(text):
|
|
420
|
+
return "counts_unknown"
|
|
252
421
|
return "passed"
|
|
253
422
|
if re.search(r"does not provide an export named|cannot import name\b", text, re.IGNORECASE):
|
|
254
423
|
return "test_error"
|
|
@@ -676,6 +845,46 @@ def reexec_snapshot(destination: Path, command: str, timeout: int = 120):
|
|
|
676
845
|
shutil.rmtree(temp, ignore_errors=True)
|
|
677
846
|
|
|
678
847
|
|
|
848
|
+
def reexec_scoped_snapshot(project_root: Path, destination: Path, command: str,
|
|
849
|
+
git_head_ref: str, scope: dict, timeout: int = 120):
|
|
850
|
+
"""Replay a scoped proof: rebuild the sealed tree, then re-run.
|
|
851
|
+
|
|
852
|
+
Unchanged files come from `git archive` at the recorded HEAD (their
|
|
853
|
+
seal-time content is HEAD content by construction of the scope), deleted
|
|
854
|
+
paths stay deleted, and the sealed scoped files are overlaid last so
|
|
855
|
+
they win over the archive.
|
|
856
|
+
"""
|
|
857
|
+
temp = Path(tempfile.mkdtemp(prefix="evalid-proof-"))
|
|
858
|
+
try:
|
|
859
|
+
archive = subprocess.run(
|
|
860
|
+
["git", "archive", git_head_ref],
|
|
861
|
+
cwd=project_root,
|
|
862
|
+
capture_output=True,
|
|
863
|
+
)
|
|
864
|
+
if archive.returncode != 0:
|
|
865
|
+
raise RuntimeError(f"git archive {git_head_ref} failed: {archive.stderr.decode('utf-8', errors='replace')[:300]}")
|
|
866
|
+
extract = subprocess.run(
|
|
867
|
+
["tar", "-x", "-C", str(temp)],
|
|
868
|
+
input=archive.stdout,
|
|
869
|
+
capture_output=True,
|
|
870
|
+
)
|
|
871
|
+
if extract.returncode != 0:
|
|
872
|
+
raise RuntimeError(f"replay tree extraction failed: {extract.stderr.decode('utf-8', errors='replace')[:300]}")
|
|
873
|
+
for rel in list(scope.get("deleted", [])) + list(scope.get("files", [])):
|
|
874
|
+
target = temp / rel
|
|
875
|
+
if target.is_dir() and not target.is_symlink():
|
|
876
|
+
shutil.rmtree(target)
|
|
877
|
+
elif target.exists() or target.is_symlink():
|
|
878
|
+
target.unlink()
|
|
879
|
+
if any(destination.iterdir()):
|
|
880
|
+
shutil.copytree(destination, temp, dirs_exist_ok=True)
|
|
881
|
+
result = run_shell(command, temp, timeout=timeout)
|
|
882
|
+
text = f"{result.stdout or ''}\n{result.stderr or ''}"
|
|
883
|
+
return result.returncode, text
|
|
884
|
+
finally:
|
|
885
|
+
shutil.rmtree(temp, ignore_errors=True)
|
|
886
|
+
|
|
887
|
+
|
|
679
888
|
def copy_project(project_root: Path, destination: Path) -> None:
|
|
680
889
|
def ignore(_directory, names):
|
|
681
890
|
return [name for name in names if name in SKIP_DIR_NAMES]
|
package/xflow/plan/SKILL.md
CHANGED
|
@@ -30,6 +30,15 @@ reuse it instead of rewriting the same plan again.
|
|
|
30
30
|
If the file is missing or still a scaffold placeholder, create the reusable
|
|
31
31
|
plan once and leave routing to the execution workflow.
|
|
32
32
|
|
|
33
|
+
## Lite-track merged pass
|
|
34
|
+
|
|
35
|
+
When the change will run the lite (yolo) track, do the red-proof authoring
|
|
36
|
+
in the same pass as the plan: write the failing test and fix the red test
|
|
37
|
+
command in the plan itself. The yolo workflow seals `tdd-red` immediately
|
|
38
|
+
after `plan`, so a separate test-authoring pass would only re-read the
|
|
39
|
+
same files. This does not apply to the corps track, which keeps planning
|
|
40
|
+
and proof authoring apart.
|
|
41
|
+
|
|
33
42
|
## Boundary
|
|
34
43
|
|
|
35
44
|
- This skill does not open issues.
|
package/xflow/scripts/common.py
CHANGED
|
@@ -25,7 +25,10 @@ ALLOWED_STAGE_TRANSITIONS: dict[str, set[str]] = {
|
|
|
25
25
|
"proposal-freeze": {"proposal-freeze", "proposal-consistency-check", "plan"},
|
|
26
26
|
"proposal-consistency-check": {"proposal-consistency-check", "plan"},
|
|
27
27
|
"plan": {"plan", "tdd"},
|
|
28
|
-
"
|
|
28
|
+
# "execute" is reachable from openissue on the lite track: since F2 the
|
|
29
|
+
# red proof is sealed before the issue is filed, so execution may start
|
|
30
|
+
# straight from the openissue stage.
|
|
31
|
+
"openissue": {"openissue", "tdd", "execute"},
|
|
29
32
|
"tdd": {"tdd", "execute", "verify"},
|
|
30
33
|
"execute": {"execute", "tdd", "verify"},
|
|
31
34
|
"verify": {"verify", "tdd", "execute", "archive"},
|
package/xflow/yolo/SKILL.md
CHANGED
|
@@ -22,6 +22,10 @@ For a full run:
|
|
|
22
22
|
xflow workflow run workflows/yolo.yaml --title "<中文标题>" --change-type backend
|
|
23
23
|
```
|
|
24
24
|
|
|
25
|
+
When `--change-id` is omitted and the current branch cannot provide one
|
|
26
|
+
(e.g. `main`), the run derives the change id from `--title` and prints it;
|
|
27
|
+
pass `--change-id` explicitly to override.
|
|
28
|
+
|
|
25
29
|
## Plan reuse rule
|
|
26
30
|
|
|
27
31
|
- Read `.xflow/GOAL.md` before execution when present. State whether the current
|
|
@@ -45,13 +49,26 @@ xflow workflow run workflows/yolo.yaml --title "<中文标题>" --change-type ba
|
|
|
45
49
|
4. `proposal-freeze`
|
|
46
50
|
5. `proposal-consistency-check`
|
|
47
51
|
6. `plan`
|
|
48
|
-
7. `
|
|
49
|
-
8. `
|
|
50
|
-
9. `
|
|
52
|
+
7. `tdd` (red proof — sealed in the same pass as the plan, see below)
|
|
53
|
+
8. `openissue`
|
|
54
|
+
9. `set-in-progress`
|
|
51
55
|
10. `execute`
|
|
52
56
|
11. `verify-consistency`
|
|
53
57
|
12. `archive`
|
|
54
58
|
|
|
59
|
+
On the lite track the red proof is sealed immediately after planning: the
|
|
60
|
+
pass that writes the plan also writes the failing test, so `tdd-red` only
|
|
61
|
+
runs and seals it — no second exploration pass, and the issue/branch
|
|
62
|
+
ceremony no longer sits between planning and the red proof.
|
|
63
|
+
|
|
64
|
+
In `verify-consistency`, run the deterministic gates first. The J6 proof
|
|
65
|
+
(`specs/changes/${CHANGE_ID}/cr/review-0.json`) carries a `seal_partition`:
|
|
66
|
+
files whose bytes still match the green proof's sealed hashes
|
|
67
|
+
(`sealed_unchanged`) are exactly what the TDD gate verified — the review
|
|
68
|
+
pass does not re-read them. Only `seal_drifted` files (changed after the
|
|
69
|
+
green seal, or never sealed) get a fresh read. When `seal_partition` is
|
|
70
|
+
null there is no green seal to trust, and the review re-reads everything.
|
|
71
|
+
|
|
55
72
|
The workflow YAML is the canonical execution source for atom order, gates, and
|
|
56
73
|
artifact checks. This skill only explains the contract.
|
|
57
74
|
|