@surlymochan/evalid 0.5.18 → 0.5.19
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +12 -0
- package/RELEASE_NOTES.md +14 -0
- package/bin/xflow.js +5 -0
- package/docs/evidence/m4-slice2-wallclock.md +61 -0
- package/docs/launch-dossier.md +2 -2
- package/package.json +1 -1
- package/schemas/workflow.schema.json +8 -0
- package/src/core/workflow-executor.js +69 -3
- package/workflows/yolo.yaml +10 -0
- package/xflow/atoms/lite_track.py +247 -0
package/CHANGELOG.md
CHANGED
|
@@ -3,6 +3,18 @@
|
|
|
3
3
|
All notable changes to this project are documented here. Detailed notes live in
|
|
4
4
|
[RELEASE_NOTES.md](RELEASE_NOTES.md).
|
|
5
5
|
|
|
6
|
+
## [0.5.19] - 2026-11-10
|
|
7
|
+
|
|
8
|
+
### Added
|
|
9
|
+
- **M4 slice 1: lite fast-track classifier and `--lite` switch (#98)**: `xflow/atoms/lite_track.py` classifies a diff `lite` / `full` against the merge-base (`git diff --numstat`, added + removed); thresholds `max_files = 10`, `max_lines = 300`; any touch under `specs/` or `workflows/` forces `full`; empty diff is `lite`; unified-diff text mode (`--diff-file`) also supported. `xflow workflow run` gains an explicit `--lite` flag — the classifier is the decision procedure, no silent auto-switching. Executor `lite_skip` (`src/core/workflow-executor.js` + schema): phases/atoms marked `lite_skip: true` are skipped in a `--lite` run and logged `phase_skipped` / `atom_skipped` (reason `lite_track`); a deterministic gate on a lite-skipped atom passes with verdict `lite_skipped`; `workflow_started` / `workflow_completed` record `lite_fast_track: true/false`. `workflows/yolo.yaml` wiring: `proposal-consistency-check` phase is `lite_skip`; `J4a` / `J4b` atoms in `verify-consistency` are `lite_skip`. Kept in lite mode: TDD run + proof seal + i6d gate (`tdd-green`), full test run (`J1`), code review (`J6`), and `xflow audit` replay. Test evidence: 11 new tests in `test/lite-track.test.js` (classifier on real git fixtures, yolo.yaml wiring, `WorkflowExecutor` lite/full runs with mock atoms) RED→GREEN; full `npm run verify` exit 0 (58 test files); PR CI runs verify 42664527969 and ledger 42664528087 both pass; merged main CI run 42665066351 success.
|
|
10
|
+
- **M4 slice 2: lite acceptance demo + wall-clock preliminary numbers (#99)**: `test/lite-track-acceptance.test.js` proves the lite track end to end through the real `WorkflowExecutor` and real `workflows/yolo.yaml` (dogfood fix: `classify_repo --uncommitted` in `lite_track.py` merged committed and worktree numstats with `max()`, overcounting partially reverted lines — now reads the worktree diff against the base directly; regression test in `test/lite-track.test.js`, 11/12 pass without the fix). Lite run: `proposal-consistency-check` logged `phase_skipped` (reason `lite_track`); `J4a` / `J4b` logged `atom_skipped`; the `verify-consistency` gate passed with verdict `lite_skipped`; `workflow_completed` recorded `lite_fast_track: true`. Seal + clean replay: red proof `passed: false` (`failure_kind: assertion`); green proof `passed: true` with binding hash and snapshot scope; I6b validated both. I6d, I6c, J1 and J6 all passed in both tracks. Full-track control (same flow, `--lite` off): J4a / J4d / J4b all executed and passed; no `lite_track` skip anywhere; `lite_fast_track: false`. Classifier controls: the real 2-file fix classifies `lite`; a 400-line addition classifies `full`; a one-line touch under `specs/` classifies `full` (force-full). Test evidence: 2 new tests (regression + acceptance) RED→GREEN; full `npm run verify` exit 0 — 59 test files, 536 tests, 535 pass, 0 fail (1 pre-existing todo); PR CI runs verify and ledger both pass; merged main CI run 42689050732 success.
|
|
11
|
+
|
|
12
|
+
Wall clock, stated plainly as preliminary (docs/evidence/m4-slice2-wallclock.md, n=1 per track, same demo flow, Linux VM / node v24.22.1 / python 3.12.3): standalone acceptance test lite 5,050 ms vs full 6,330 ms (ratio 0.798); inside `npm run verify` lite 10,420 ms vs full 16,191 ms (ratio 0.644). The two runs disagree by ~0.15 on the ratio minutes apart on the same machine — that spread is why these numbers are preliminary. The formal ≤1.3× verdict for issue #5 P1-2 waits for the bench full-matrix numbers; P1-2 stays unticked, and this release makes no ≤1.3× claim.
|
|
13
|
+
|
|
14
|
+
Milestone M4 "lite fast track" — **not yet accepted**: slices 1 and 2 are merged checkpoints only; the end-to-end acceptance evidence (real small change through `--lite`, real large change judged `full`, wall-clock verdict from bench numbers) is not yet in place. This release makes no M4 acceptance claim.
|
|
15
|
+
|
|
16
|
+
Acceptance debt, stated plainly: the bench-number acceptance for P1-2 / P1-3 / P2-1 / P2-3 still awaits the MiniMax full-matrix batch (15 instances × 4 arms × reps 3), which is still running. No bench numbers are claimed in this release.
|
|
17
|
+
|
|
6
18
|
## [0.5.18] - 2026-11-09
|
|
7
19
|
|
|
8
20
|
### Added
|
package/RELEASE_NOTES.md
CHANGED
|
@@ -1,5 +1,19 @@
|
|
|
1
1
|
# Release Notes
|
|
2
2
|
|
|
3
|
+
## 0.5.19 - M4 lite fast track, slices 1–2 (milestone M4 not yet accepted)
|
|
4
|
+
|
|
5
|
+
Patch release. Two merged PRs since 0.5.18, both verified end to end in CI and locally.
|
|
6
|
+
|
|
7
|
+
**M4 slice 1: lite fast-track classifier and `--lite` switch (#98).** `xflow/atoms/lite_track.py` classifies a diff `lite` / `full` against the merge-base; thresholds `max_files = 10`, `max_lines = 300`; any touch under `specs/` or `workflows/` forces `full`; empty diff is `lite`. `xflow workflow run` gains an explicit `--lite` flag — the classifier decides, no silent auto-switching. Executor `lite_skip` skips the spec ritual in a `--lite` run (`proposal-consistency-check` phase, `J4a` / `J4b` atoms) and logs the skips; kept in lite mode: TDD run + proof seal + i6d gate, full test run (`J1`), code review (`J6`), and `xflow audit` replay. Test evidence: 11 new tests in `test/lite-track.test.js` RED→GREEN; full `npm run verify` exit 0 (58 test files); PR CI runs verify 42664527969 and ledger 42664528087 both pass; merged main CI run 42665066351 success.
|
|
8
|
+
|
|
9
|
+
**M4 slice 2: lite acceptance demo + wall-clock preliminary numbers (#99).** `test/lite-track-acceptance.test.js` proves the lite track end to end through the real executor and real `workflows/yolo.yaml` (dogfood fix: `classify_repo --uncommitted` overcounted partially reverted lines — now reads the worktree diff against the base directly; regression test, 11/12 pass without the fix). Lite run: `proposal-consistency-check` skipped (reason `lite_track`), `J4a` / `J4b` skipped, `verify-consistency` gate passed with verdict `lite_skipped`, `lite_fast_track: true`; seal + clean replay (red proof `passed: false`, green proof `passed: true`, I6b validated both); I6d, I6c, J1 and J6 passed in both tracks. Full-track control ran all spec atoms, `lite_fast_track: false`. Classifier controls: 2-file fix → `lite`; 400-line addition → `full`; one-line `specs/` touch → `full`. Test evidence: 2 new tests RED→GREEN; full `npm run verify` exit 0 — 59 test files, 536 tests, 535 pass, 0 fail (1 pre-existing todo); PR CI verify and ledger both pass; merged main CI run 42689050732 success.
|
|
10
|
+
|
|
11
|
+
Wall clock, stated plainly as preliminary (docs/evidence/m4-slice2-wallclock.md, n=1 per track, same demo flow, Linux VM / node v24.22.1 / python 3.12.3): standalone acceptance test lite 5,050 ms vs full 6,330 ms (ratio 0.798); inside `npm run verify` lite 10,420 ms vs full 16,191 ms (ratio 0.644). The two runs disagree by ~0.15 on the ratio minutes apart on the same machine — that spread is why these numbers are preliminary. The formal ≤1.3× verdict for issue #5 P1-2 waits for the bench full-matrix numbers; P1-2 stays unticked, and this release makes no ≤1.3× claim.
|
|
12
|
+
|
|
13
|
+
**Milestone M4 "lite fast track" — not yet accepted.** Slices 1 and 2 are merged checkpoints only; the end-to-end acceptance evidence (real small change through `--lite`, real large change judged `full`, wall-clock verdict from bench numbers) is not yet in place.
|
|
14
|
+
|
|
15
|
+
Acceptance debt, stated plainly: the bench-number acceptance for P1-2, P1-3, P2-1 and P2-3 — the wall-clock/token/resolved lines from issue #5 — still awaits the MiniMax full-matrix batch (15 instances × 4 arms × reps 3), which is still running. No bench numbers are claimed in this release.
|
|
16
|
+
|
|
3
17
|
## 0.5.18 - Proof verdicts as SARIF for code scanning (issue #73 closed)
|
|
4
18
|
|
|
5
19
|
Patch release. One merged PR since 0.5.17, verified end to end in CI and locally, with the Security-tab acceptance evidence from a demo PR.
|
package/bin/xflow.js
CHANGED
|
@@ -6692,6 +6692,11 @@ Commands:
|
|
|
6692
6692
|
validate <yaml|yolo|corps> Validate workflow YAML as preflight only
|
|
6693
6693
|
|
|
6694
6694
|
Options:
|
|
6695
|
+
--lite M4 lite fast track: skip spec-ritual steps
|
|
6696
|
+
(proposal-consistency-check phase, J4a/J4b atoms)
|
|
6697
|
+
and keep seal + replay only. Run the classifier
|
|
6698
|
+
first (python3 xflow/atoms/lite_track.py --repo .)
|
|
6699
|
+
and add --lite only when it says lite.
|
|
6695
6700
|
help, -h, --help Show this help message
|
|
6696
6701
|
`);
|
|
6697
6702
|
}
|
|
@@ -0,0 +1,61 @@
|
|
|
1
|
+
# M4 slice 2: lite vs full wall-clock — local preliminary numbers
|
|
2
|
+
|
|
3
|
+
Date: 2026-11-10 (Asia/Singapore). Source of truth for the demo itself is
|
|
4
|
+
`test/lite-track-acceptance.test.js` (re-run it to reproduce).
|
|
5
|
+
|
|
6
|
+
## Status: preliminary — NOT the ≤1.3× conclusion
|
|
7
|
+
|
|
8
|
+
These are local n=1 measurements of the acceptance demo harness. The formal
|
|
9
|
+
wall-clock verdict for issue #5 P1-2 (small tasks ≤1.3× baseline) waits for
|
|
10
|
+
the bench full-matrix numbers; P1-2 stays unticked and no ≤1.3× claim is
|
|
11
|
+
made here.
|
|
12
|
+
|
|
13
|
+
## Method
|
|
14
|
+
|
|
15
|
+
Same demo flow per track, in a throwaway git repo holding a buggy copy of
|
|
16
|
+
`lite_track.py`: the real `WorkflowExecutor` runs the real
|
|
17
|
+
`workflows/yolo.yaml` phases (`proposal-consistency-check`, `tdd-red`,
|
|
18
|
+
`tdd-green`, `verify-consistency`) with the real atom registry. First run
|
|
19
|
+
(red seals, green stops pre-fix), fix applied, resume run (green seals,
|
|
20
|
+
verify runs). The lite run skips the spec ritual; the full run executes it.
|
|
21
|
+
Each total below is first run + resume run.
|
|
22
|
+
|
|
23
|
+
Machine: Linux x86_64 VM, AMD EPYC 9D25, 4 vCPU; node v24.22.1, python 3.12.3.
|
|
24
|
+
|
|
25
|
+
## Numbers
|
|
26
|
+
|
|
27
|
+
| Run | lite (first + resume) | full (first + resume) | lite/full |
|
|
28
|
+
| --- | --- | --- | --- |
|
|
29
|
+
| Standalone acceptance test | 2,198 ms + 2,852 ms = 5,050 ms | 2,816 ms + 3,514 ms = 6,330 ms | 0.798 |
|
|
30
|
+
| Inside `npm run verify` (full suite) | 4,120 ms + 6,300 ms = 10,420 ms | 7,601 ms + 8,590 ms = 16,191 ms | 0.644 |
|
|
31
|
+
|
|
32
|
+
n=1 per track per run. The two runs, minutes apart on the same machine,
|
|
33
|
+
disagree by ~0.15 on the ratio — that spread is itself the reason these
|
|
34
|
+
numbers are preliminary: the demo's absolute times are dominated by
|
|
35
|
+
per-process startup (node + python atoms) and machine load, not by the
|
|
36
|
+
spec ritual alone. Do not quote 0.64 or 0.80 as the lite-track speedup.
|
|
37
|
+
|
|
38
|
+
## What the demo proved (functional, not timing)
|
|
39
|
+
|
|
40
|
+
- Classifier: the real fix (2 files, `lite_track.py` + `test/demo.test.mjs`)
|
|
41
|
+
classifies `lite`; a 400-line addition classifies `full`; a one-line
|
|
42
|
+
touch under `specs/` classifies `full` (force-full).
|
|
43
|
+
- Lite run: `proposal-consistency-check` logged `phase_skipped`
|
|
44
|
+
(reason `lite_track`); `J4a` / `J4b` logged `atom_skipped` (reason
|
|
45
|
+
`lite_track`); the `verify-consistency` gate passed with verdict
|
|
46
|
+
`lite_skipped`; `workflow_completed` recorded `lite_fast_track: true`.
|
|
47
|
+
- Seal + clean replay: red proof `passed: false`
|
|
48
|
+
(`failure_kind: assertion`); green proof `passed: true` with binding
|
|
49
|
+
hash and snapshot scope; I6b validated both (binding recomputed,
|
|
50
|
+
snapshot hashes checked, command re-run on the snapshot).
|
|
51
|
+
I6d, I6c, J1 and J6 all passed in both tracks.
|
|
52
|
+
- Full run (control): J4a / J4d / J4b all executed and passed; no
|
|
53
|
+
`lite_track` skip anywhere; `workflow_completed` recorded
|
|
54
|
+
`lite_fast_track: false`.
|
|
55
|
+
|
|
56
|
+
## Reproduce
|
|
57
|
+
|
|
58
|
+
```sh
|
|
59
|
+
node test/lite-track-acceptance.test.js # prints a [m4-slice2-wallclock] line
|
|
60
|
+
npm run verify # full suite; same line appears
|
|
61
|
+
```
|
package/docs/launch-dossier.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# evalid Launch Dossier
|
|
2
2
|
|
|
3
|
-
Generated: 2026-11-
|
|
3
|
+
Generated: 2026-11-09T20:25:08.664Z
|
|
4
4
|
Mode: post_publish
|
|
5
5
|
Verdict: conditionally ready: engine and adoption evidence are strong, registry publication remains required for a splash launch
|
|
6
6
|
Launch ready: not yet
|
|
@@ -117,7 +117,7 @@ Evidence:
|
|
|
117
117
|
- Adoption records: 1; status: pass
|
|
118
118
|
- Goal audit: pass; goal: Keep evalid credible as an open-source agent delivery runtime: repo-owned workflow contracts, executable verification gates, durable project context, and honest benchmark evidence.
|
|
119
119
|
- Package evidence: missing; source: registry_json
|
|
120
|
-
- Package name/version: @surlymochan/evalid@0.5.
|
|
120
|
+
- Package name/version: @surlymochan/evalid@0.5.19
|
|
121
121
|
|
|
122
122
|
## Passing Evidence
|
|
123
123
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@surlymochan/evalid",
|
|
3
|
-
"version": "0.5.
|
|
3
|
+
"version": "0.5.19",
|
|
4
4
|
"description": "Claimed != verified: a runtime verification layer for coding agents. It seals each test run and replays it from a clean snapshot, with a zero-judge gate pipeline (spec clarification, code review, TDD proof) and the xflow CLI.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"ai-agents",
|
|
@@ -40,6 +40,10 @@
|
|
|
40
40
|
"label": { "type": "string" },
|
|
41
41
|
"catalog_ref": { "type": "integer", "minimum": 1, "maximum": 27 },
|
|
42
42
|
"required": { "type": "boolean" },
|
|
43
|
+
"lite_skip": {
|
|
44
|
+
"type": "boolean",
|
|
45
|
+
"description": "M4 lite fast track: skip this whole phase when the run has input.lite=true (spec ritual only; the verification spine never carries this flag)"
|
|
46
|
+
},
|
|
43
47
|
"condition": {
|
|
44
48
|
"type": "object",
|
|
45
49
|
"description": "Skip phase unless condition met",
|
|
@@ -56,6 +60,10 @@
|
|
|
56
60
|
"properties": {
|
|
57
61
|
"id": { "type": "string" },
|
|
58
62
|
"with": { "type": "object" },
|
|
63
|
+
"lite_skip": {
|
|
64
|
+
"type": "boolean",
|
|
65
|
+
"description": "M4 lite fast track: skip this atom when the run has input.lite=true (spec ritual atoms only; a deterministic gate on a lite-skipped atom passes with verdict lite_skipped)"
|
|
66
|
+
},
|
|
59
67
|
"parallel_group": { "type": "string" },
|
|
60
68
|
"parallel_weight": { "type": "integer", "minimum": 1, "maximum": 16 }
|
|
61
69
|
}
|
|
@@ -11,6 +11,14 @@
|
|
|
11
11
|
*
|
|
12
12
|
* E6 ≡ E1+E2 invariant enforced: executor routes ALL deterministic gate checks
|
|
13
13
|
* through E6.gate.local_precheck Python atom, never re-implementing logic here.
|
|
14
|
+
*
|
|
15
|
+
* M4 lite fast track (evalid #97): when input.lite is true, phases and atoms
|
|
16
|
+
* marked `lite_skip: true` in the workflow YAML are skipped (the spec ritual:
|
|
17
|
+
* J4a/J4b/J4d), while the verification spine (TDD run + proof seal + i6d
|
|
18
|
+
* gate + full test run + CR review) still runs. Skips are logged as
|
|
19
|
+
* `phase_skipped` / `atom_skipped` with reason `lite_track`, and a
|
|
20
|
+
* deterministic gate whose gate atom is lite-skipped passes with verdict
|
|
21
|
+
* `lite_skipped` instead of re-running the spec check.
|
|
14
22
|
*/
|
|
15
23
|
|
|
16
24
|
import { existsSync, statSync, writeFileSync, readFileSync, mkdirSync } from 'fs';
|
|
@@ -67,6 +75,13 @@ export class WorkflowExecutor {
|
|
|
67
75
|
|
|
68
76
|
// ─── Main execution loop ─────────────────────────────────────────────────
|
|
69
77
|
|
|
78
|
+
// M4 lite fast track: explicit `--lite` input only (the classifier in
|
|
79
|
+
// xflow/atoms/lite_track.py is the decision procedure; this flag is the
|
|
80
|
+
// enforcement switch). No silent auto-switching.
|
|
81
|
+
isLiteFastTrack() {
|
|
82
|
+
return this.input?.lite === true;
|
|
83
|
+
}
|
|
84
|
+
|
|
70
85
|
// ─── Phase cost accounting ─────────────────────────────────────────────
|
|
71
86
|
|
|
72
87
|
recordPhaseCost(phase, status, startedAtMs) {
|
|
@@ -121,6 +136,7 @@ export class WorkflowExecutor {
|
|
|
121
136
|
workflow_run_id: this.runtime.workflowRunId,
|
|
122
137
|
workflow: this.workflow.name,
|
|
123
138
|
track: this.workflow.track,
|
|
139
|
+
lite_fast_track: this.isLiteFastTrack(),
|
|
124
140
|
workflow_integrity: this.workflowManifest,
|
|
125
141
|
max_parallel_agents: this.maxParallelAgents,
|
|
126
142
|
parallel_policy: this.concurrencyPolicy.mode,
|
|
@@ -189,6 +205,28 @@ export class WorkflowExecutor {
|
|
|
189
205
|
}
|
|
190
206
|
}
|
|
191
207
|
|
|
208
|
+
// M4 lite fast track: skip whole phases marked lite_skip (the spec
|
|
209
|
+
// ritual). Logged like a condition skip so proof still sees the phase
|
|
210
|
+
// as accounted for; the reason distinguishes the two.
|
|
211
|
+
if (phase.lite_skip && this.isLiteFastTrack()) {
|
|
212
|
+
console.log(` ↷ Skipping (lite fast track: phase "${phase.id}" is spec ritual)`);
|
|
213
|
+
await this.logExecutionEvent('phase_skipped', {
|
|
214
|
+
workflow_run_id: this.runtime.workflowRunId,
|
|
215
|
+
workflow: this.workflow.name,
|
|
216
|
+
track: this.workflow.track,
|
|
217
|
+
phase_id: phase.id,
|
|
218
|
+
phase_label: phase.label || null,
|
|
219
|
+
phase_index: i,
|
|
220
|
+
reason: 'lite_track',
|
|
221
|
+
change_id: this.changeId || null,
|
|
222
|
+
mission_id: this.runtime.missionId || null,
|
|
223
|
+
workflow_integrity_digest: this.workflowManifest.digest,
|
|
224
|
+
});
|
|
225
|
+
this.recordPhaseCost(phase, 'skipped', phaseStartedAtMs);
|
|
226
|
+
this.saveState({ ...state, last_completed_phase: phase.id });
|
|
227
|
+
continue;
|
|
228
|
+
}
|
|
229
|
+
|
|
192
230
|
// Preconditions
|
|
193
231
|
if (phase.preconditions?.length > 0) {
|
|
194
232
|
const precondOk = this.checkPreconditions(phase.preconditions);
|
|
@@ -203,12 +241,33 @@ export class WorkflowExecutor {
|
|
|
203
241
|
this.ensurePlanArtifact();
|
|
204
242
|
}
|
|
205
243
|
|
|
206
|
-
// Run atoms
|
|
244
|
+
// Run atoms (M4 lite: atoms marked lite_skip are filtered out first
|
|
245
|
+
// and logged individually, so mixed phases keep their spine).
|
|
207
246
|
const atomsToRun = rerunRequest?.phase_id === phase.id && rerunRequest?.atom_id
|
|
208
247
|
? (phase.atoms || []).filter((atomRef) => atomRef.id === rerunRequest.atom_id)
|
|
209
248
|
: (phase.atoms || []);
|
|
210
|
-
|
|
211
|
-
|
|
249
|
+
const runnableAtoms = [];
|
|
250
|
+
for (const atomRef of atomsToRun) {
|
|
251
|
+
if (atomRef.lite_skip && this.isLiteFastTrack()) {
|
|
252
|
+
console.log(` ↷ Skipping atom (lite fast track): ${atomRef.id}`);
|
|
253
|
+
await this.logExecutionEvent('atom_skipped', {
|
|
254
|
+
workflow_run_id: this.runtime.workflowRunId,
|
|
255
|
+
workflow: this.workflow.name,
|
|
256
|
+
track: this.workflow.track,
|
|
257
|
+
phase_id: phase.id,
|
|
258
|
+
phase_label: phase.label || null,
|
|
259
|
+
atom_id: atomRef.id,
|
|
260
|
+
reason: 'lite_track',
|
|
261
|
+
change_id: this.changeId || null,
|
|
262
|
+
mission_id: this.runtime.missionId || null,
|
|
263
|
+
workflow_integrity_digest: this.workflowManifest.digest,
|
|
264
|
+
});
|
|
265
|
+
continue;
|
|
266
|
+
}
|
|
267
|
+
runnableAtoms.push(atomRef);
|
|
268
|
+
}
|
|
269
|
+
if (runnableAtoms.length > 0) {
|
|
270
|
+
for (const batch of this.buildAtomBatches(runnableAtoms)) {
|
|
212
271
|
this.ensureNotAborted();
|
|
213
272
|
const results = batch.parallel
|
|
214
273
|
? await this.runParallelBatch(batch.atoms, phase)
|
|
@@ -291,6 +350,7 @@ export class WorkflowExecutor {
|
|
|
291
350
|
workflow_run_id: this.runtime.workflowRunId,
|
|
292
351
|
workflow: this.workflow.name,
|
|
293
352
|
track: this.workflow.track,
|
|
353
|
+
lite_fast_track: this.isLiteFastTrack(),
|
|
294
354
|
workflow_integrity: this.workflowManifest,
|
|
295
355
|
change_id: this.changeId || null,
|
|
296
356
|
mission_id: this.runtime.missionId || null,
|
|
@@ -567,6 +627,12 @@ export class WorkflowExecutor {
|
|
|
567
627
|
|
|
568
628
|
switch (gate.type) {
|
|
569
629
|
case 'deterministic': {
|
|
630
|
+
// M4 lite: a gate whose atom is lite-skipped in this phase must not
|
|
631
|
+
// re-run the spec check it gates — the skip is the verdict.
|
|
632
|
+
if (gate.atom && this.isLiteFastTrack()
|
|
633
|
+
&& (phase.atoms || []).some((atomRef) => atomRef.id === gate.atom && atomRef.lite_skip)) {
|
|
634
|
+
return { ok: true, verdict: 'lite_skipped' };
|
|
635
|
+
}
|
|
570
636
|
// Route through E6 (Python) for ALL structural checks — invariant enforced
|
|
571
637
|
if (gate.atom) {
|
|
572
638
|
const result = await this.runAtom({ id: gate.atom, with: gate.with }, phase);
|
package/workflows/yolo.yaml
CHANGED
|
@@ -80,10 +80,14 @@ phases:
|
|
|
80
80
|
type: artifact-verify
|
|
81
81
|
on_fail: stop
|
|
82
82
|
|
|
83
|
+
# M4 lite fast track (evalid #97): this whole phase is spec ritual — a
|
|
84
|
+
# `--lite` run skips it (executor logs phase_skipped, reason lite_track)
|
|
85
|
+
# and keeps the verification spine (tdd seal + i6d + J1 + J6).
|
|
83
86
|
- id: proposal-consistency-check
|
|
84
87
|
label: "07 · Proposal consistency check"
|
|
85
88
|
catalog_ref: 7
|
|
86
89
|
required: true
|
|
90
|
+
lite_skip: true
|
|
87
91
|
atoms:
|
|
88
92
|
- id: J4a.spec_consistency.structural
|
|
89
93
|
- id: J4d.spec_clarify.gate
|
|
@@ -205,8 +209,14 @@ phases:
|
|
|
205
209
|
required: true
|
|
206
210
|
atoms:
|
|
207
211
|
- id: J1.tests.run
|
|
212
|
+
# M4 lite fast track (evalid #97): J4a/J4b are spec ritual — a `--lite`
|
|
213
|
+
# run skips them (atom_skipped, reason lite_track) and the phase's
|
|
214
|
+
# deterministic J4a gate passes with verdict lite_skipped. J1 (full
|
|
215
|
+
# test run) and J6 (code review) stay: they are verification, not ritual.
|
|
208
216
|
- id: J4a.spec_consistency.structural
|
|
217
|
+
lite_skip: true
|
|
209
218
|
- id: J4b.spec_delta.review
|
|
219
|
+
lite_skip: true
|
|
210
220
|
- id: J6.cr.review
|
|
211
221
|
- id: B3.status.transition
|
|
212
222
|
with: { to_stage: "verify" }
|
|
@@ -0,0 +1,247 @@
|
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
|
+
"""
|
|
3
|
+
M4 lite fast-track: diff-size classifier (slice 1).
|
|
4
|
+
|
|
5
|
+
Decides whether a change is small enough to skip the spec ritual and run
|
|
6
|
+
seal (i6b) + clean replay only. The verdict is measured against the change's
|
|
7
|
+
merge-base, reusing M3 slice 2's importable parts from
|
|
8
|
+
``i6d_assertion_execution``: ``resolve_diff_base`` for the branch point,
|
|
9
|
+
``changed_lines`` / ``changed_lines_from_diff`` for the changed-line map.
|
|
10
|
+
Line/file counts come from ``git diff --numstat`` (added + removed, which
|
|
11
|
+
also catches deletion-only and binary files that the changed-line map
|
|
12
|
+
excludes by design).
|
|
13
|
+
|
|
14
|
+
Thresholds (initial values, see evalid #97 for the calibration):
|
|
15
|
+
|
|
16
|
+
* ``max_files = 10``, ``max_lines = 300`` (added + removed). This repo's own
|
|
17
|
+
merged PRs calibrate it: routine slices / fixes / releases land at 4-28
|
|
18
|
+
lines; the only recent feature slice above the line is P1-3 (#34, 330
|
|
19
|
+
lines), which correctly stays full-track.
|
|
20
|
+
* Force-full overrides: any touched path under ``specs/`` or ``workflows/``
|
|
21
|
+
flips the verdict to ``full`` -- the spec ritual exists precisely to catch
|
|
22
|
+
drift there, so a small diff touching spec/workflow definitions must not
|
|
23
|
+
skip it. An empty diff classifies ``lite``.
|
|
24
|
+
|
|
25
|
+
CLI (always prints one JSON object; exit 0, exit 2 on git/usage errors)::
|
|
26
|
+
|
|
27
|
+
python3 lite_track.py --repo . [--diff-base main] [--json]
|
|
28
|
+
python3 lite_track.py --repo . --diff-file patch.diff [--json]
|
|
29
|
+
python3 lite_track.py --repo . --max-files 10 --max-lines 300
|
|
30
|
+
|
|
31
|
+
The classifier is the *decision procedure*; the enforcement switch is the
|
|
32
|
+
explicit ``--lite`` flag on ``xflow workflow run``. Run the classifier
|
|
33
|
+
first and add ``--lite`` only when it says ``lite`` -- slice 1 does no
|
|
34
|
+
silent auto-switching.
|
|
35
|
+
"""
|
|
36
|
+
|
|
37
|
+
import argparse
|
|
38
|
+
import json
|
|
39
|
+
import os
|
|
40
|
+
import subprocess
|
|
41
|
+
import sys
|
|
42
|
+
|
|
43
|
+
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
|
44
|
+
|
|
45
|
+
from i6d_assertion_execution import ( # noqa: E402
|
|
46
|
+
changed_lines,
|
|
47
|
+
changed_lines_from_diff,
|
|
48
|
+
resolve_diff_base,
|
|
49
|
+
)
|
|
50
|
+
|
|
51
|
+
DEFAULT_MAX_FILES = 10
|
|
52
|
+
DEFAULT_MAX_LINES = 300
|
|
53
|
+
FORCE_FULL_PREFIXES = ("specs/", "workflows/")
|
|
54
|
+
VERSION = 1
|
|
55
|
+
|
|
56
|
+
|
|
57
|
+
def _git(repo, *argv):
|
|
58
|
+
proc = subprocess.run(
|
|
59
|
+
["git", *argv], cwd=repo, capture_output=True, text=True, timeout=30,
|
|
60
|
+
)
|
|
61
|
+
if proc.returncode != 0:
|
|
62
|
+
raise RuntimeError(
|
|
63
|
+
f"git {' '.join(argv)} failed ({proc.returncode}): "
|
|
64
|
+
f"{proc.stderr.strip()}"
|
|
65
|
+
)
|
|
66
|
+
return proc.stdout
|
|
67
|
+
|
|
68
|
+
|
|
69
|
+
def _numstat(repo, base_sha):
|
|
70
|
+
"""``{relpath: (added, removed)}`` from ``git diff --numstat``.
|
|
71
|
+
|
|
72
|
+
Binary files report ``-`` counts; they count as a touched file with
|
|
73
|
+
zero counted lines.
|
|
74
|
+
"""
|
|
75
|
+
raw = _git(repo, "diff", "--numstat", base_sha, "HEAD", "--")
|
|
76
|
+
stats = {}
|
|
77
|
+
for line in raw.splitlines():
|
|
78
|
+
if not line.strip():
|
|
79
|
+
continue
|
|
80
|
+
added_s, removed_s, path = line.split("\t", 2)
|
|
81
|
+
added = 0 if added_s == "-" else int(added_s)
|
|
82
|
+
removed = 0 if removed_s == "-" else int(removed_s)
|
|
83
|
+
stats[path] = (added, removed)
|
|
84
|
+
return stats
|
|
85
|
+
|
|
86
|
+
|
|
87
|
+
def _diff_text_files_and_counts(diff_text):
|
|
88
|
+
"""``(files, added, removed)`` parsed from a unified diff text.
|
|
89
|
+
|
|
90
|
+
Counts ``+``/``-`` lines (excluding the ``+++``/``---`` headers);
|
|
91
|
+
files come from the ``+++`` headers so deletion-only files are
|
|
92
|
+
included, unlike the changed-line map.
|
|
93
|
+
"""
|
|
94
|
+
files = []
|
|
95
|
+
added = 0
|
|
96
|
+
removed = 0
|
|
97
|
+
for line in diff_text.splitlines():
|
|
98
|
+
if line.startswith("+++ "):
|
|
99
|
+
target = line[4:].strip()
|
|
100
|
+
if target in ("/dev/null", ""):
|
|
101
|
+
continue
|
|
102
|
+
if target.startswith("b/"):
|
|
103
|
+
target = target[2:]
|
|
104
|
+
if len(target) >= 2 and target.startswith('"') and target.endswith('"'):
|
|
105
|
+
target = json.loads(target)
|
|
106
|
+
files.append(target)
|
|
107
|
+
elif line.startswith("+") and not line.startswith("+++"):
|
|
108
|
+
added += 1
|
|
109
|
+
elif line.startswith("-") and not line.startswith("---"):
|
|
110
|
+
removed += 1
|
|
111
|
+
return files, added, removed
|
|
112
|
+
|
|
113
|
+
|
|
114
|
+
def _verdict(files, added, removed, max_files, max_lines):
|
|
115
|
+
"""Shared verdict logic. ``files`` are repo-relative paths."""
|
|
116
|
+
total = added + removed
|
|
117
|
+
force_full = sorted({path for path in files if path.startswith(FORCE_FULL_PREFIXES)})
|
|
118
|
+
if force_full:
|
|
119
|
+
which = "specs/" if any(p.startswith("specs/") for p in force_full) else "workflows/"
|
|
120
|
+
return "full", (
|
|
121
|
+
f"touches {which} paths ({', '.join(force_full)}): "
|
|
122
|
+
"spec/workflow drift needs the full track"
|
|
123
|
+
), force_full
|
|
124
|
+
if len(files) > max_files:
|
|
125
|
+
return "full", (
|
|
126
|
+
f"exceeds file budget: {len(files)} files > {max_files} max_files"
|
|
127
|
+
), []
|
|
128
|
+
if total > max_lines:
|
|
129
|
+
return "full", (
|
|
130
|
+
f"exceeds line budget: {total} lines > {max_lines} max_lines"
|
|
131
|
+
), []
|
|
132
|
+
if not files:
|
|
133
|
+
return "lite", "empty diff: nothing to ritualise", []
|
|
134
|
+
return "lite", (
|
|
135
|
+
f"within thresholds ({len(files)} files, {total} lines "
|
|
136
|
+
f"<= {max_files} files, {max_lines} lines)"
|
|
137
|
+
), []
|
|
138
|
+
|
|
139
|
+
|
|
140
|
+
def _numstat_worktree(repo, base_sha):
|
|
141
|
+
"""``{relpath: (added, removed)}`` from ``git diff --numstat`` of the
|
|
142
|
+
working tree (staged + unstaged) against ``base_sha``.
|
|
143
|
+
|
|
144
|
+
Binary files report ``-`` counts; they count as a touched file with
|
|
145
|
+
zero counted lines.
|
|
146
|
+
"""
|
|
147
|
+
raw = _git(repo, "diff", "--numstat", base_sha, "--")
|
|
148
|
+
stats = {}
|
|
149
|
+
for line in raw.splitlines():
|
|
150
|
+
if not line.strip():
|
|
151
|
+
continue
|
|
152
|
+
added_s, removed_s, path = line.split("\t", 2)
|
|
153
|
+
stats[path] = (
|
|
154
|
+
0 if added_s == "-" else int(added_s),
|
|
155
|
+
0 if removed_s == "-" else int(removed_s),
|
|
156
|
+
)
|
|
157
|
+
return stats
|
|
158
|
+
|
|
159
|
+
|
|
160
|
+
def classify_repo(repo, base="main", max_files=DEFAULT_MAX_FILES,
|
|
161
|
+
max_lines=DEFAULT_MAX_LINES, uncommitted=False):
|
|
162
|
+
"""Classify the committed (or, with ``uncommitted``, working-tree) diff."""
|
|
163
|
+
base_sha = resolve_diff_base(repo, base)
|
|
164
|
+
_, changed = changed_lines(repo, base, uncommitted=uncommitted)
|
|
165
|
+
if uncommitted:
|
|
166
|
+
# changed_lines(..., uncommitted=True) unions the working tree, so
|
|
167
|
+
# the numstat side must measure that same union: the worktree diff
|
|
168
|
+
# against the base. Merging the committed and worktree numstats
|
|
169
|
+
# with max() overcounts whenever the worktree partially reverts
|
|
170
|
+
# committed lines (a branch that added 10 lines, cut back by 5 in
|
|
171
|
+
# the worktree, read as 15 changed lines instead of the true 5).
|
|
172
|
+
stats = _numstat_worktree(repo, base_sha)
|
|
173
|
+
else:
|
|
174
|
+
stats = _numstat(repo, base_sha)
|
|
175
|
+
files = sorted(stats)
|
|
176
|
+
added = sum(a for a, _ in stats.values())
|
|
177
|
+
removed = sum(r for _, r in stats.values())
|
|
178
|
+
track, reason, force_full = _verdict(files, added, removed, max_files, max_lines)
|
|
179
|
+
return {
|
|
180
|
+
"version": VERSION,
|
|
181
|
+
"track": track,
|
|
182
|
+
"reason": reason,
|
|
183
|
+
"files": len(files),
|
|
184
|
+
"added": added,
|
|
185
|
+
"removed": removed,
|
|
186
|
+
"total": added + removed,
|
|
187
|
+
"max_files": max_files,
|
|
188
|
+
"max_lines": max_lines,
|
|
189
|
+
"base": base_sha,
|
|
190
|
+
"force_full": force_full,
|
|
191
|
+
"changed_new_lines": sum(len(lines) for lines in changed.values()),
|
|
192
|
+
"source": "repo",
|
|
193
|
+
}
|
|
194
|
+
|
|
195
|
+
|
|
196
|
+
def classify_diff_text(diff_text, repo, max_files=DEFAULT_MAX_FILES,
|
|
197
|
+
max_lines=DEFAULT_MAX_LINES):
|
|
198
|
+
"""Classify a unified diff *text* (e.g. ``gh pr diff`` output)."""
|
|
199
|
+
changed = changed_lines_from_diff(diff_text, repo)
|
|
200
|
+
files, added, removed = _diff_text_files_and_counts(diff_text)
|
|
201
|
+
track, reason, force_full = _verdict(files, added, removed, max_files, max_lines)
|
|
202
|
+
return {
|
|
203
|
+
"version": VERSION,
|
|
204
|
+
"track": track,
|
|
205
|
+
"reason": reason,
|
|
206
|
+
"files": len(files),
|
|
207
|
+
"added": added,
|
|
208
|
+
"removed": removed,
|
|
209
|
+
"total": added + removed,
|
|
210
|
+
"max_files": max_files,
|
|
211
|
+
"max_lines": max_lines,
|
|
212
|
+
"base": None,
|
|
213
|
+
"force_full": force_full,
|
|
214
|
+
"changed_new_lines": sum(len(lines) for lines in changed.values()),
|
|
215
|
+
"source": "diff-text",
|
|
216
|
+
}
|
|
217
|
+
|
|
218
|
+
|
|
219
|
+
def main(argv=None):
|
|
220
|
+
parser = argparse.ArgumentParser(description="M4 lite fast-track diff-size classifier.")
|
|
221
|
+
parser.add_argument("--repo", default=".", help="repo to classify (default: .)")
|
|
222
|
+
parser.add_argument("--diff-base", default="main", help="base branch (default: main)")
|
|
223
|
+
parser.add_argument("--diff-file", default=None, help="classify a unified diff file instead of the repo diff")
|
|
224
|
+
parser.add_argument("--max-files", type=int, default=DEFAULT_MAX_FILES)
|
|
225
|
+
parser.add_argument("--max-lines", type=int, default=DEFAULT_MAX_LINES)
|
|
226
|
+
parser.add_argument("--uncommitted", action="store_true",
|
|
227
|
+
help="include uncommitted working-tree edits (repo mode)")
|
|
228
|
+
parser.add_argument("--json", action="store_true", help="print the verdict as JSON (default output)")
|
|
229
|
+
args = parser.parse_args(argv)
|
|
230
|
+
|
|
231
|
+
repo = os.path.realpath(args.repo)
|
|
232
|
+
try:
|
|
233
|
+
if args.diff_file:
|
|
234
|
+
with open(args.diff_file, encoding="utf-8") as handle:
|
|
235
|
+
diff_text = handle.read()
|
|
236
|
+
result = classify_diff_text(diff_text, repo, args.max_files, args.max_lines)
|
|
237
|
+
else:
|
|
238
|
+
result = classify_repo(repo, args.diff_base, args.max_files, args.max_lines, args.uncommitted)
|
|
239
|
+
except (RuntimeError, OSError) as exc:
|
|
240
|
+
print(f"lite_track: {exc}", file=sys.stderr)
|
|
241
|
+
return 2
|
|
242
|
+
print(json.dumps(result, indent=2))
|
|
243
|
+
return 0
|
|
244
|
+
|
|
245
|
+
|
|
246
|
+
if __name__ == "__main__":
|
|
247
|
+
raise SystemExit(main())
|