feature-factory 0.10.6 → 0.10.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1 -1
- package/WORKFLOW.md +15 -3
- package/agents/spec-writer.md +5 -0
- package/agents/work-reviewer.md +32 -0
- package/bin/factory.js +55 -9
- package/observe/index.js +31 -7
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -152,7 +152,7 @@ truncation, redaction, timeout, retry, cache, payload transport, or session beha
|
|
|
152
152
|
|
|
153
153
|
### Sandbox bootstrap
|
|
154
154
|
|
|
155
|
-
Only the CLI executes configured `bootstrap`: once during fresh init after clone, containment, and PR-base observation but before manifest publication, and again on every explicit resume while the run remains parked. The exact configured string runs unchanged with `shell: true`, inherited environment and stdin, cwd exactly the selected sandbox, and child stdout and stderr both routed to CLI stderr. Init JSON stdout therefore remains exactly one response object. Bootstrap has its own configured timeout or independent `900000` millisecond default, with no retry and no use of the verify budget.
|
|
155
|
+
Only the CLI executes configured `bootstrap`: once during fresh init after clone, containment, and PR-base observation but before manifest publication, and again on every explicit resume while the run remains parked. It also runs immediately before every configured `verify` execution, in the slice or integration worktree that verify runs in, so `verify` always sees bootstrap output installed from the tree it checks. The exact configured string runs unchanged with `shell: true`, inherited environment and stdin, cwd exactly the selected sandbox, and child stdout and stderr both routed to CLI stderr. Init JSON stdout therefore remains exactly one response object. Bootstrap has its own configured timeout or independent `900000` millisecond default, with no retry and no use of the verify budget.
|
|
156
156
|
|
|
157
157
|
After every attempt, the CLI checks tracked worktree and index paths only and ignores untracked dependency output. Unobservable tracked state refuses before dirty tracked paths; dirty paths refuse before unavailable or nonzero exit, and diagnostics name exact repository-relative paths. A clean numeric zero stores paired `bootstrap_command` and `bootstrap_exit` manifest evidence. The command is exact; the exit is a non-negative integer or `null`. Ordinary transitions preserve the pair, and status output does not expose it.
|
|
158
158
|
|
package/WORKFLOW.md
CHANGED
|
@@ -723,8 +723,8 @@ Effective push-target capture and comparison are active through the package-owne
|
|
|
723
723
|
|
|
724
724
|
| Entry | Declared input | Return shape | Failure meaning | Current behavior |
|
|
725
725
|
|---|---|---|---|---|
|
|
726
|
-
| `bootstrap` | Exact configured string as one shell command with `shell: true`, inherited environment and stdin, cwd exactly the selected sandbox, and child stdout and stderr both routed to CLI stderr. Each execution receives its own `bootstrap_timeout_ms`, independently `900000` when omitted. | Numeric exit status or unavailable `null`; output is visible on CLI stderr and never parsed | Clean zero succeeds; dirty or unobservable tracked state outranks unavailable or nonzero exit | Invoked by the CLI once during configured fresh init
|
|
727
|
-
| `verify` | Ordinary shell step in the exact integration-worktree cwd with inherited environment; no structured stdin or factory-specific payload is defined. Each attempt receives the full configured `verify_timeout_ms`, silently `900000` when omitted. | Exit status is authoritative; stdout and stderr are
|
|
726
|
+
| `bootstrap` | Exact configured string as one shell command with `shell: true`, inherited environment and stdin, cwd exactly the selected sandbox, and child stdout and stderr both routed to CLI stderr. Each execution receives its own `bootstrap_timeout_ms`, independently `900000` when omitted. | Numeric exit status or unavailable `null`; output is visible on CLI stderr and never parsed | Clean zero succeeds; dirty or unobservable tracked state outranks unavailable or nonzero exit | Invoked by the CLI once during configured fresh init, again on every explicit configured resume, and immediately before every configured `verify` execution in the tree that verify runs in: the slice worktree during slice observation, and the integration worktree for post-merge verify, its replay, and direct repository verification. Never invoked by the resolver, effective push, publication, or a replay that reuses recorded verify evidence. |
|
|
727
|
+
| `verify` | Ordinary shell step in the exact integration-worktree cwd with inherited environment; no structured stdin or factory-specific payload is defined. Each attempt receives the full configured `verify_timeout_ms`, silently `900000` when omitted. | Exit status is authoritative; stdout and stderr are routed to CLI stderr, informational, and unparsed | Zero means success; non-zero means repository verification failed; no numeric child status means unavailable | Invoked after each newly recorded merge through `observe --repository-verify`, with at most two executions in that merge invocation. The timeout and retry never apply to resolver, slice, or Gate 3 commands. |
|
|
728
728
|
| `publish` | Optional file candidate for the one Step 6 publishing selection. A nonblank inherited `FACTORY_PUBLISHING_COMMAND` selects its exact string; the same variable set empty or to whitespace selects the default; when the variable is unset this entry is selected if present, otherwise the default. A selected nondefault command runs as one shell step in `RUN_REPO` cwd, with no stdin or positional arguments and inherited environment plus exact `PR_BASE`, `FEATURE_BRANCH`, `PR_DRAFT`, `PR_TITLE`, and absolute `PR_BODY_FILE`. | Exit status is authoritative; the last nonempty stdout line must be an absolute HTTPS URL and becomes `PR_URL` | Zero plus that URL is recordable; any other result is indeterminate and parks before `factory pr` | The resolved selection replaces only `gh pr create`, after the factory-owned exact push and post-push identity guard. `factory pr` is unchanged and still records the URL. |
|
|
729
729
|
| `publishing_identity` | No runtime input; read the value `status` reports for the run, recorded at init from `--publishing-identity` or the inherited `FACTORY_PUBLISHING_IDENTITY` | Exact case-sensitive string compared with the observed login | Absent at init refuses before any sandbox exists; mismatch or unobservable identity parks the run | Active at the three mandatory guards below; only a manifest written before 0.8.0 can report `null` and skip them. |
|
|
730
730
|
|
|
@@ -1527,6 +1527,18 @@ Per slice:
|
|
|
1527
1527
|
not a detail to reconcile in your head. Omit `--test-cmd` and the slice is not `review_ready`
|
|
1528
1528
|
unless its ratified `test_plan` is empty — the waiver comes from the plan, not from you.
|
|
1529
1529
|
|
|
1530
|
+
When `.factory.json` declares `verify`, this step also runs it on the slice's commit, after the
|
|
1531
|
+
ratified test command, and records the result as `repository_verify`. A declared `bootstrap` runs in
|
|
1532
|
+
the slice worktree, which never received init's bootstrap output, immediately before that verify and
|
|
1533
|
+
only when it will run; a bootstrap refusal is the evidence's blocked reason and the verify does not run. A failing repository verify makes the slice not `review_ready`, the
|
|
1534
|
+
same as a failing test, so the review rejects and the slice retries; an empty `test_plan` does not
|
|
1535
|
+
waive it. Its output appears on stderr: give the failing part of it (the lint, format, or test
|
|
1536
|
+
failure) to the builder with the rejection, because otherwise the retry cannot see what to fix. This
|
|
1537
|
+
is what keeps a one-line lint failure from surfacing only after merge, where it cannot be repaired.
|
|
1538
|
+
When a merge's tree is byte-identical to the slice commit whose repository verify passed, the
|
|
1539
|
+
post-merge verify reuses that result instead of running again, and its evidence names the slice commit
|
|
1540
|
+
in `reused_from`; any other merge, such as one after a sibling merged, runs the verify as before.
|
|
1541
|
+
|
|
1530
1542
|
`BUILDER_REPORT` is a path and not the report. Write the builder's returned report to
|
|
1531
1543
|
`BUILDER_REPORT=".factory/$R/artifacts/$SLICE_ID-builder-attempt-$SLICE_ATTEMPT.json"` and pass that path,
|
|
1532
1544
|
which keeps the report beside the run's other evidence instead of in argv. It holds `status`, `slice`,
|
|
@@ -1706,7 +1718,7 @@ Gate 3 remains a fresh independent observation.
|
|
|
1706
1718
|
|
|
1707
1719
|
### Post-merge finding routing and repair journal
|
|
1708
1720
|
|
|
1709
|
-
Route a known post-merge failure before any next-wave action. A production defect parks top-level needs-human
|
|
1721
|
+
Route a known post-merge failure before any next-wave action. A production defect parks top-level needs-human. There is no in-band resume for it: production source is never repaired on the integration branch, and a configured command reruns only after a test-only repair. Say so in the reason, and name preserving the feature branch and starting a fresh run as the recovery; do not tell the operator to resume.
|
|
1710
1722
|
Production source is never repaired on the integration branch. Unclassifiable,
|
|
1711
1723
|
interrupted-unknown, invalid-config, unsafe dirty or moved replay, unobservable, journal-invalid, or
|
|
1712
1724
|
repair-exhausted outcomes also terminalize. Clean unchanged repository-verification exhaustion follows
|
package/agents/spec-writer.md
CHANGED
|
@@ -49,6 +49,11 @@ Resolve every ambiguity so builders don't have to. Follow the repository's agent
|
|
|
49
49
|
- Auth/role gating
|
|
50
50
|
- Test plan: unit tests, and acceptance (which criterion maps to which test, and at what level)
|
|
51
51
|
- **Class-wide work:** convert the research inventory into a closed implementation matrix — one row per sink/call site, each assigned an exact primitive/policy, a compatibility (preserve/migrate) or explicit exclusion decision, and a mapped test. No sink is left to the builder to discover.
|
|
52
|
+
- **Document deliverables:** when the issue's deliverable is a specification, contract, or data artifact
|
|
53
|
+
rather than executable behavior, the brief specifies the content each AC requires and nothing more. Do
|
|
54
|
+
not expand "test vectors", "examples", or "fixtures" into an executable model, a validator, or an
|
|
55
|
+
exhaustive corpus unless a criterion says so; an edge case beyond the criteria gets a documented limit,
|
|
56
|
+
not new mechanism.
|
|
52
57
|
|
|
53
58
|
## Output contract
|
|
54
59
|
|
package/agents/work-reviewer.md
CHANGED
|
@@ -78,6 +78,35 @@ When the subject is a **class-wide** requirement — one that **cannot be establ
|
|
|
78
78
|
- **Precedence for late discoveries:** a genuinely required sink, policy, compatibility decision, migration, or test that is missing is a **blocker no matter which review round surfaces it** — record it once in `required_fixes`, carry it into every later review, and REJECT until observed evidence proves it landed. Only *unrelated* new scope or *optional* extra depth on already-decided rows is a non-blocking note; a required in-scope omission is never downgraded to optional just because it appeared in a later round.
|
|
79
79
|
- **Feasibility rule:** reject a brief whose required behavior cannot be implemented within its allowed mechanisms, dependencies, compatibility constraints, or explicit non-goals — for example, demanding grammar-complete or adversarial-input recognition while forbidding every parser, new dependency, or bounded implementation strategy. Surface the smallest explicit dependency, scope, or design decision needed before builders start; do not approve an impossible implementation envelope as "decision-complete."
|
|
80
80
|
|
|
81
|
+
## Specification subjects (documents, contracts, data artifacts)
|
|
82
|
+
|
|
83
|
+
When the deliverable is a document or a data artifact — a specification, a contract, a protocol or
|
|
84
|
+
topology description, a fixture corpus — rather than executable behavior, the class-wide rules above do
|
|
85
|
+
not apply beyond what the acceptance criteria require. For code the in-scope members already exist in the
|
|
86
|
+
repository, so enumerating them ends. For a document the members are whatever it chooses to specify, and
|
|
87
|
+
each answer creates new surface to review; exhaustive enumeration and the late-discovery precedence rule
|
|
88
|
+
then turn every fix into the next round's findings. Three contract runs in a row inflated this way (130k
|
|
89
|
+
lines of corpus JSON, 25k lines of test vectors with a validator inside a Markdown file, a 72 KB
|
|
90
|
+
topology document with its own SQL schemas and a quarantine system).
|
|
91
|
+
One boundary decides everything beyond the criteria, in both directions: **does the content commit an
|
|
92
|
+
implementer to build, store, or conform to something?** Call that an obligation.
|
|
93
|
+
- **The bar is the acceptance criteria, not completeness.** A behavior an AC requires that the document
|
|
94
|
+
omits or gets wrong is a BLOCKER, in any round.
|
|
95
|
+
- **Missing depth is not a finding.** Do not ask for an obligation no AC requires — deeper design,
|
|
96
|
+
internal schemas, tuning values, or handling for rare edge cases. Note it at most; never block on it.
|
|
97
|
+
- **Added obligation beyond the criteria is a finding.** A schema, mechanism, subsystem, executable model,
|
|
98
|
+
or corpus that no AC requires, and that an implementer would have to build, store, or conform to, is
|
|
99
|
+
unapproved scope expansion: require it removed or reduced to a documented limit, the same way unapproved
|
|
100
|
+
scope in a brief is rejected. A reviewer that stops asking for more while accepting whatever the builder
|
|
101
|
+
adds unprompted has not stopped the ratchet.
|
|
102
|
+
- **Explanatory depth is neither.** Rationale, worked examples, and clarification of a required behavior
|
|
103
|
+
add no obligation; accept them without comment.
|
|
104
|
+
- **A documented limit satisfies an edge case.** "X is out of scope" or "behavior is undefined for X" is
|
|
105
|
+
an acceptable answer unless an AC requires X to be handled.
|
|
106
|
+
- **No executable models by default.** A validator, generator, exhaustive corpus, or test vectors is an
|
|
107
|
+
obligation like any other: required only when an AC asks for one. Judge whether what is specified is
|
|
108
|
+
correct and consistent.
|
|
109
|
+
|
|
81
110
|
## What to check, by subject
|
|
82
111
|
|
|
83
112
|
- **`work-decomposer` satisfiability:** for every slice, check that its ratified `test_plan` can be made
|
|
@@ -116,6 +145,9 @@ When the subject is a **class-wide** requirement — one that **cannot be establ
|
|
|
116
145
|
- Code comments follow the target repository's documented policy; do not invent a blanket ban.
|
|
117
146
|
- **Slice discipline:** the diff stays within the slice's `paths` (out-of-lane edits are a finding).
|
|
118
147
|
- The slice's `acceptance` is actually implemented. This is never waived.
|
|
148
|
+
- A present `repository_verify` that did not exit zero is a BLOCKER: the repository's own verify
|
|
149
|
+
(lint, format, full suite) fails on this commit and will fail again after merge, where it cannot be
|
|
150
|
+
repaired. Name the failing check in `required_fixes`. An empty `test_plan` does not waive it.
|
|
119
151
|
- The observed tests cover that acceptance — **unless the slice's ratified `test_plan` is empty**,
|
|
120
152
|
which the plan decided at Gate 2 and which the workflow and `deriveReviewReady` both honour. A
|
|
121
153
|
docs-only slice reviewed against tests it was ratified not to have is rejected forever; that
|
package/bin/factory.js
CHANGED
|
@@ -394,16 +394,23 @@ function bootstrapOutcome(worktree, config, phase) {
|
|
|
394
394
|
}
|
|
395
395
|
|
|
396
396
|
|
|
397
|
+
// Enforcement (#376): verify runs on a tree bootstrap has just prepared, the same way in every place it runs.
|
|
398
|
+
// Bootstrap output is untracked, so a slice worktree never had it and a merge that changes what bootstrap
|
|
399
|
+
// installs was verified against the dependencies from init. A verify that does not execute runs no bootstrap.
|
|
400
|
+
function bootstrapBeforeVerify(worktree, config, phase) {
|
|
401
|
+
return config?.bootstrapCommand ? bootstrapOutcome(worktree, config, phase).refusal : null;
|
|
402
|
+
}
|
|
403
|
+
|
|
397
404
|
function branchPoint(run) {
|
|
398
405
|
const base = run.slices.find((slice) => Array.isArray(slice.depends_on) && slice.depends_on.length === 0)?.base_ref;
|
|
399
406
|
if (!/^[0-9a-f]{40}$/u.test(base ?? "")) throw new CliError("first seeded root slice has no immutable 40-character base_ref");
|
|
400
407
|
return base;
|
|
401
408
|
}
|
|
402
409
|
|
|
403
|
-
async function writeObservedEvidence({ repo, runDir, runId, subject, attempt, branch, baseRef, worktree, status, blockedReason, claim, testCommand, skipReason, shellCommand, testTimeoutMs }) {
|
|
410
|
+
async function writeObservedEvidence({ repo, runDir, runId, subject, attempt, branch, baseRef, worktree, status, blockedReason, claim, testCommand, skipReason, shellCommand, testTimeoutMs, repositoryVerify = null, reused = null }) {
|
|
404
411
|
const evidence = buildEvidence({
|
|
405
412
|
subject, attempt, branch, baseRef, worktree, status, blockedReason, claim, runId,
|
|
406
|
-
testCommand, skipReason, shellCommand, testTimeoutMs,
|
|
413
|
+
testCommand, skipReason, shellCommand, testTimeoutMs, repositoryVerify, reused,
|
|
407
414
|
});
|
|
408
415
|
const ancestry = observeAncestry(worktree, baseRef, "HEAD");
|
|
409
416
|
if (ancestry !== "ancestor") {
|
|
@@ -420,7 +427,8 @@ async function writeObservedEvidence({ repo, runDir, runId, subject, attempt, br
|
|
|
420
427
|
|
|
421
428
|
function canonicalRepositoryVerifyEvidence(evidence, { runId, run, integration, verifyCommand }) {
|
|
422
429
|
const baseRef = branchPoint(run);
|
|
423
|
-
|
|
430
|
+
// `repository_verify` is slice-only: null on test-verifier evidence, absent on evidence written before #372.
|
|
431
|
+
const keys = Object.keys(evidence).filter((key) => !["repository_verify", "reused_from"].includes(key)).sort();
|
|
424
432
|
const commandNames = [
|
|
425
433
|
"git rev-parse HEAD",
|
|
426
434
|
`git --literal-pathspecs diff --name-only -z ${baseRef}...HEAD`,
|
|
@@ -438,7 +446,9 @@ function canonicalRepositoryVerifyEvidence(evidence, { runId, run, integration,
|
|
|
438
446
|
&& ((tests.observed === true && Number.isInteger(tests.exit))
|
|
439
447
|
|| (tests.observed === false && tests.exit === null));
|
|
440
448
|
const reconciliation = evidence.claim_reconciliation;
|
|
441
|
-
return JSON.stringify(keys) === JSON.stringify([
|
|
449
|
+
return JSON.stringify(keys) === JSON.stringify(EVIDENCE_KEYS.filter((key) => !["repository_verify", "reused_from"].includes(key)).sort())
|
|
450
|
+
&& (evidence.repository_verify ?? null) === null
|
|
451
|
+
&& (evidence.reused_from === undefined || sameTree(integration.worktree, evidence.reused_from, evidence.commit))
|
|
442
452
|
&& evidence.subject === "test-verifier" && evidence.run_id === runId
|
|
443
453
|
&& Number.isSafeInteger(evidence.attempt) && evidence.attempt >= 1
|
|
444
454
|
&& evidence.branch === run.branch && evidence.base_ref === baseRef
|
|
@@ -521,16 +531,37 @@ function repositoryVerifyRetrySafety(repo, run, mergeCommit) {
|
|
|
521
531
|
return integration;
|
|
522
532
|
}
|
|
523
533
|
|
|
524
|
-
|
|
534
|
+
// Enforcement (#374): a post-merge verify is skipped only when the merge's tree is byte-identical to a slice
|
|
535
|
+
// commit whose own repository verify ran the same command green -- the same bytes already passed. A tree
|
|
536
|
+
// equality also implies the same committed `.factory.json`, so command and timeout cannot differ.
|
|
537
|
+
function sameTree(worktree, left, right) {
|
|
538
|
+
const tree = (sha) => git(worktree, ["rev-parse", "--verify", `${sha}^{tree}`]);
|
|
539
|
+
const [a, b] = [tree(left), tree(right)];
|
|
540
|
+
return a.ok && b.ok && a.stdout.trim() === b.stdout.trim();
|
|
541
|
+
}
|
|
542
|
+
|
|
543
|
+
function reusableSliceVerify({ runDir, runId, ref, worktree, mergeCommit, command }) {
|
|
544
|
+
let evidence;
|
|
545
|
+
try { evidence = readEvidence(runDir, ref, { runId }); } catch { return null; }
|
|
546
|
+
const verified = evidence.repository_verify;
|
|
547
|
+
if (!verified || verified.observed !== true || verified.exit !== 0 || verified.cmd !== command) return null;
|
|
548
|
+
if (!sameTree(worktree, evidence.commit, mergeCommit)) return null;
|
|
549
|
+
return { commit: evidence.commit, tests: { cmd: command, exit: 0, observed: true, skipped_reason: null } };
|
|
550
|
+
}
|
|
551
|
+
|
|
552
|
+
async function runRepositoryVerifyAttempts({ repo, runDir, runId, run, mergeCommit, verify, integration, reused = null }) {
|
|
525
553
|
const baseRef = branchPoint(run);
|
|
526
554
|
let attemptIntegration = integration;
|
|
527
555
|
// False-green enforcement: one invocation gets at most two executions, never an unbounded recovery loop.
|
|
528
556
|
for (let attempt = 1; attempt <= 2; attempt += 1) {
|
|
557
|
+
// Every execution is freshly bootstrapped, the retry included; a reused result executes nothing (#376).
|
|
558
|
+
const refusal = attempt === 1 && reused ? null : bootstrapBeforeVerify(attemptIntegration.worktree, verify, "post-merge verify");
|
|
559
|
+
if (refusal) throw new CliError(`${refusal} after recorded merge ${mergeCommit}; merged slice remains recorded; stop before advancing.`);
|
|
529
560
|
const { evidence } = await writeObservedEvidence({
|
|
530
561
|
repo, runDir, runId, subject: "test-verifier", attempt, branch: run.branch,
|
|
531
562
|
baseRef, worktree: attemptIntegration.worktree, status: "completed", blockedReason: null,
|
|
532
563
|
claim: null, testCommand: verify.command, skipReason: null, shellCommand: true,
|
|
533
|
-
testTimeoutMs: verify.timeoutMs,
|
|
564
|
+
testTimeoutMs: verify.timeoutMs, reused: attempt === 1 ? reused : null,
|
|
534
565
|
});
|
|
535
566
|
const classified = classifyRepositoryVerifyEvidence(runDir, {
|
|
536
567
|
runId, run, integration: attemptIntegration, verifyCommand: verify.command,
|
|
@@ -544,7 +575,7 @@ async function runRepositoryVerifyAttempts({ repo, runDir, runId, run, mergeComm
|
|
|
544
575
|
return null;
|
|
545
576
|
}
|
|
546
577
|
|
|
547
|
-
async function verifyRecordedMerge({ repo, runDir, runId, mergeCommit }) {
|
|
578
|
+
async function verifyRecordedMerge({ repo, runDir, runId, mergeCommit, sliceEvidenceRef = null }) {
|
|
548
579
|
const run = readRun(runDir);
|
|
549
580
|
const integration = requireIntegrationWorktree(repo, run, run.worktree);
|
|
550
581
|
if (integration.head !== mergeCommit) {
|
|
@@ -560,7 +591,9 @@ async function verifyRecordedMerge({ repo, runDir, runId, mergeCommit }) {
|
|
|
560
591
|
throw error;
|
|
561
592
|
}
|
|
562
593
|
if (verify === null) return null;
|
|
563
|
-
|
|
594
|
+
const reused = sliceEvidenceRef && reusableSliceVerify({ runDir, runId, ref: sliceEvidenceRef,
|
|
595
|
+
worktree: integration.worktree, mergeCommit, command: verify.command });
|
|
596
|
+
return runRepositoryVerifyAttempts({ repo, runDir, runId, run, mergeCommit, verify, integration, reused });
|
|
564
597
|
}
|
|
565
598
|
|
|
566
599
|
const HANDLERS = {
|
|
@@ -1090,7 +1123,7 @@ const HANDLERS = {
|
|
|
1090
1123
|
// step needs it: `observe --base` is compared for exact equality against this value at
|
|
1091
1124
|
// merge time. The skill previously said to read it from `factory status`, which does not
|
|
1092
1125
|
// expose it — so the documented path could not be followed at all.
|
|
1093
|
-
if (status === "merged") await verifyRecordedMerge({ repo, runDir, runId, mergeCommit: row.merge_commit });
|
|
1126
|
+
if (status === "merged") await verifyRecordedMerge({ repo, runDir, runId, mergeCommit: row.merge_commit, sliceEvidenceRef: row.evidence_ref });
|
|
1094
1127
|
// Slice attempts are budgeted the same way, so their rejected verdicts vanish the same way.
|
|
1095
1128
|
const sliceReviewArchive = await archiveReviewAttempt(runDir, retryReviewRef ?? row.review_ref);
|
|
1096
1129
|
await archiveReviewAttempt(runDir, retryEvidenceRef ?? row.evidence_ref);
|
|
@@ -1124,6 +1157,8 @@ const HANDLERS = {
|
|
|
1124
1157
|
if (error instanceof RepositoryConfigError) throw new CliError(error.message);
|
|
1125
1158
|
throw error;
|
|
1126
1159
|
}
|
|
1160
|
+
const refusal = bootstrapBeforeVerify(worktree, repositoryVerify, "repository verification");
|
|
1161
|
+
if (refusal) throw new CliError(refusal);
|
|
1127
1162
|
}
|
|
1128
1163
|
|
|
1129
1164
|
let claim = null;
|
|
@@ -1166,6 +1201,14 @@ const HANDLERS = {
|
|
|
1166
1201
|
+ `expected ${JSON.stringify(slice.test_plan)}; received ${JSON.stringify(flags.testCmd)}`,
|
|
1167
1202
|
);
|
|
1168
1203
|
}
|
|
1204
|
+
// A slice also runs the configured repository `verify`, read from its own committed config (#372).
|
|
1205
|
+
let sliceVerify = null;
|
|
1206
|
+
if (slice) {
|
|
1207
|
+
try { sliceVerify = readRepositoryConfig(worktree, { optional: true }); } catch (error) {
|
|
1208
|
+
if (error instanceof RepositoryConfigError) throw new CliError(error.message);
|
|
1209
|
+
throw error;
|
|
1210
|
+
}
|
|
1211
|
+
}
|
|
1169
1212
|
const skipReason = slice && slice.test_plan.length === 0
|
|
1170
1213
|
? `test_plan for '${subject}' was approved empty at slices-seed`
|
|
1171
1214
|
: null;
|
|
@@ -1179,12 +1222,15 @@ const HANDLERS = {
|
|
|
1179
1222
|
testCommand: flags.repositoryVerify ? repositoryVerify.command : flags.testCmd ? flags.testCmd.split(" ").filter(Boolean) : null,
|
|
1180
1223
|
skipReason, shellCommand: flags.repositoryVerify === true,
|
|
1181
1224
|
testTimeoutMs: flags.repositoryVerify ? repositoryVerify.timeoutMs : undefined,
|
|
1225
|
+
repositoryVerify: sliceVerify && { command: sliceVerify.command, timeoutMs: sliceVerify.timeoutMs,
|
|
1226
|
+
prepare: () => bootstrapBeforeVerify(worktree, sliceVerify, "slice observation") },
|
|
1182
1227
|
});
|
|
1183
1228
|
return emit(flags, {
|
|
1184
1229
|
run_id: runId, subject, evidence_ref: evidenceRef(subject),
|
|
1185
1230
|
review_ready: evidence.review_ready, files_changed: evidence.files_changed.length,
|
|
1186
1231
|
tests: evidence.tests.observed ? `exit ${evidence.tests.exit}` : `skipped: ${evidence.tests.skipped_reason}`,
|
|
1187
1232
|
ancestry, mismatches: evidence.claim_reconciliation.mismatches.map((entry) => entry.field),
|
|
1233
|
+
repository_verify: evidence.repository_verify && (evidence.repository_verify.observed ? `exit ${evidence.repository_verify.exit}` : "not run"),
|
|
1188
1234
|
});
|
|
1189
1235
|
},
|
|
1190
1236
|
async init(positional, flags) {
|
package/observe/index.js
CHANGED
|
@@ -14,7 +14,7 @@ export const EVIDENCE_KEYS = Object.freeze([
|
|
|
14
14
|
"subject", "run_id", "attempt", "branch", "base_ref", "worktree", "status", "blocked_reason",
|
|
15
15
|
"worktree_clean",
|
|
16
16
|
"files_changed", "diff_stat", "diff_observed", "commands", "tests", "commit",
|
|
17
|
-
"observed_by", "review_ready", "claim_reconciliation",
|
|
17
|
+
"observed_by", "review_ready", "claim_reconciliation", "repository_verify", "reused_from",
|
|
18
18
|
]);
|
|
19
19
|
|
|
20
20
|
export function git(cwd, args, { runner = spawnSync } = {}) {
|
|
@@ -109,7 +109,7 @@ export function observeAncestry(worktree, ancestor, descendant, options = {}) {
|
|
|
109
109
|
// Attack 1: the test command is run here, by us, and its exit code is recorded
|
|
110
110
|
// from the process rather than from anybody's report. `observed: false` means we
|
|
111
111
|
// could not run it, which is not the same as a pass.
|
|
112
|
-
export function runTests(worktree, command, { runner = spawnSync, skipReason = null, shellCommand = false, timeoutMs = DEFAULT_REPOSITORY_VERIFY_TIMEOUT_MS } = {}) {
|
|
112
|
+
export function runTests(worktree, command, { runner = spawnSync, skipReason = null, shellCommand = false, timeoutMs = DEFAULT_REPOSITORY_VERIFY_TIMEOUT_MS, stdio = "inherit", env = process.env } = {}) {
|
|
113
113
|
if (!command) {
|
|
114
114
|
// Finding 1: defaulting a skip reason let omission manufacture review readiness.
|
|
115
115
|
// Tests must be observed green or explicitly skipped with a caller-declared reason;
|
|
@@ -117,7 +117,7 @@ export function runTests(worktree, command, { runner = spawnSync, skipReason = n
|
|
|
117
117
|
return { cmd: null, exit: null, observed: false, skipped_reason: skipReason };
|
|
118
118
|
}
|
|
119
119
|
const result = shellCommand
|
|
120
|
-
? runner(command, [], { cwd: worktree, shell: true, stdio
|
|
120
|
+
? runner(command, [], { cwd: worktree, shell: true, stdio, env, timeout: timeoutMs })
|
|
121
121
|
: runner(command[0], command.slice(1), { cwd: worktree, encoding: "utf8", shell: false });
|
|
122
122
|
const exit = Number.isInteger(result?.status) ? result.status : null;
|
|
123
123
|
return { cmd: shellCommand ? command : command.join(" "), exit, observed: exit !== null, skipped_reason: null };
|
|
@@ -133,6 +133,9 @@ export function deriveReviewReady(evidence) {
|
|
|
133
133
|
// function alone; with the term only at write time, every mismatched record read back as
|
|
134
134
|
// tampered and wedged its slice where `slice blocked` could never record it.
|
|
135
135
|
if (evidence.claim_reconciliation?.mismatches?.length > 0) return false;
|
|
136
|
+
// Absent on evidence written before #372, and null when no `verify` is configured.
|
|
137
|
+
const verify = evidence.repository_verify;
|
|
138
|
+
if (verify && !(verify.observed === true && verify.exit === 0)) return false;
|
|
136
139
|
// A tree with uncommitted changes cannot produce evidence about the commit it
|
|
137
140
|
// claims, whatever the tests said.
|
|
138
141
|
if (evidence.worktree_clean !== true) return false;
|
|
@@ -201,7 +204,7 @@ export function privilegedPaths(filesChanged) {
|
|
|
201
204
|
|| PRIVILEGED_EXACT.includes(file));
|
|
202
205
|
}
|
|
203
206
|
|
|
204
|
-
export function buildEvidence({ subject, runId, attempt, branch, baseRef, worktree, status, blockedReason = null, claim = null, testCommand = null, skipReason = null, shellCommand = false, testTimeoutMs = DEFAULT_REPOSITORY_VERIFY_TIMEOUT_MS, options = {} }) {
|
|
207
|
+
export function buildEvidence({ subject, runId, attempt, branch, baseRef, worktree, status, blockedReason = null, claim = null, testCommand = null, skipReason = null, shellCommand = false, testTimeoutMs = DEFAULT_REPOSITORY_VERIFY_TIMEOUT_MS, repositoryVerify = null, reused = null, options = {} }) {
|
|
205
208
|
// Cleanliness is established before anything else is observed, because every later
|
|
206
209
|
// fact - the diff, the commit, and above all the test result - is only about the
|
|
207
210
|
// recorded commit if the tree has nothing uncommitted in it.
|
|
@@ -209,9 +212,27 @@ export function buildEvidence({ subject, runId, attempt, branch, baseRef, worktr
|
|
|
209
212
|
const observation = observeWorktree(worktree, baseRef, options);
|
|
210
213
|
// Tests are not run at all against a dirty tree: running them would produce a
|
|
211
214
|
// result about bytes that are not going to merge.
|
|
212
|
-
|
|
213
|
-
|
|
215
|
+
// `reused` carries a slice's green repository verify for a merge whose tree is byte-identical (#374).
|
|
216
|
+
const tests = cleanliness.clean && reused ? reused.tests : cleanliness.clean
|
|
217
|
+
? runTests(worktree, testCommand, { ...options, skipReason, shellCommand, timeoutMs: testTimeoutMs,
|
|
218
|
+
// Output to stderr, stdin inherited: `slice merged --json` and `observe --json` stay one JSON object.
|
|
219
|
+
...(shellCommand ? { stdio: ["inherit", 2, 2] } : {}) })
|
|
214
220
|
: { cmd: testCommand ? (shellCommand ? testCommand : testCommand.join(" ")) : null, exit: null, observed: false, skipped_reason: null };
|
|
221
|
+
// Enforcement (false green, #372): a slice ran only its ratified test command, so the repository's
|
|
222
|
+
// configured `verify` (lint, format, full suite) first ran after merge, where production repair is
|
|
223
|
+
// forbidden and one Clippy warning parked a 13-slice run. The same suite now runs on the slice's commit.
|
|
224
|
+
// Skipped after a failing test run, which is already not review-ready; its output goes to stderr so
|
|
225
|
+
// `observe --json` stays one JSON object. A bootstrap refusal (#376) means the tree is not prepared, so the
|
|
226
|
+
// verify does not run and the refusal is the evidence's blocked reason.
|
|
227
|
+
// Bootstrap is the verify's own first step (#376 review): it runs only when the verify will, and after the
|
|
228
|
+
// ratified test, so nothing the test does can remove its output before the verify reads it.
|
|
229
|
+
const eligible = repositoryVerify && cleanliness.clean && !(tests.observed && tests.exit !== 0);
|
|
230
|
+
const bootstrapRefusal = eligible ? repositoryVerify.prepare?.() ?? null : null;
|
|
231
|
+
const verifyRuns = eligible && !bootstrapRefusal;
|
|
232
|
+
const verified = !repositoryVerify ? null : verifyRuns
|
|
233
|
+
? (({ cmd, exit, observed }) => ({ cmd, exit, observed }))(runTests(worktree, repositoryVerify.command,
|
|
234
|
+
{ ...options, shellCommand: true, timeoutMs: repositoryVerify.timeoutMs, stdio: ["ignore", 2, 2] }))
|
|
235
|
+
: { cmd: repositoryVerify.command, exit: null, observed: false };
|
|
215
236
|
|
|
216
237
|
// Third round, finding 1: cleanliness was a pre-test snapshot, so a test that wrote
|
|
217
238
|
// tracked files left the tree dirty while the evidence still claimed a clean HEAD -
|
|
@@ -242,7 +263,8 @@ export function buildEvidence({ subject, runId, attempt, branch, baseRef, worktr
|
|
|
242
263
|
base_ref: baseRef,
|
|
243
264
|
worktree,
|
|
244
265
|
status,
|
|
245
|
-
blocked_reason: blockedReason ?? cleanliness.reason ??
|
|
266
|
+
blocked_reason: blockedReason ?? cleanliness.reason ?? bootstrapRefusal
|
|
267
|
+
?? (stableUnderTest ? null : "worktree changed while the tests ran"),
|
|
246
268
|
// Named for what it asserts: clean before the run, still clean after, and HEAD did
|
|
247
269
|
// not move. A pre-test snapshot alone was not enough.
|
|
248
270
|
worktree_clean: stableUnderTest,
|
|
@@ -255,6 +277,8 @@ export function buildEvidence({ subject, runId, attempt, branch, baseRef, worktr
|
|
|
255
277
|
observed_by: "orchestrator",
|
|
256
278
|
review_ready: false,
|
|
257
279
|
claim_reconciliation: { claimed: false, mismatches: [] },
|
|
280
|
+
repository_verify: verified,
|
|
281
|
+
...(reused ? { reused_from: reused.commit } : {}),
|
|
258
282
|
};
|
|
259
283
|
evidence.claim_reconciliation = reconcileClaim(claim, evidence);
|
|
260
284
|
evidence.review_ready = deriveReviewReady(evidence);
|