feature-factory 0.10.5 → 0.10.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/WORKFLOW.md +15 -1
- package/agents/spec-writer.md +5 -0
- package/agents/work-reviewer.md +32 -0
- package/bin/factory.js +16 -4
- package/observe/index.js +21 -5
- package/package.json +1 -1
package/WORKFLOW.md
CHANGED
|
@@ -909,6 +909,12 @@ with `git check-ref-format --branch "$FEATURE_BRANCH"`. Bind
|
|
|
909
909
|
`git -C "$O" show-ref --verify --quiet "$FEATURE_REF"` to exit exactly 1 for ref absence. Exit 0 means
|
|
910
910
|
present and every other result is a lookup error; either refuses before init.
|
|
911
911
|
|
|
912
|
+
Bind `KEY` from how `R` was bound, never from anything else: when the configured resolver returned a
|
|
913
|
+
payload, `KEY` is exactly `R`, its `run_id`; when `R` is a ticket key from the request or branch, `KEY`
|
|
914
|
+
is that key; when `R` was derived from free text, `KEY` is absent and `--issue` is omitted. Never add a
|
|
915
|
+
`#` or other decoration. `issue_key` is immutable after init, and publication reads it for the title
|
|
916
|
+
prefix and closing reference, so a run whose resolver found issue `25` must record `25`.
|
|
917
|
+
|
|
912
918
|
An explicit `PR_BASE` wins. Otherwise require the symbolic branch in the configured operator worktree;
|
|
913
919
|
detached, missing, escaping, or unprovable worktree state is refused by init. Request one fresh sandbox
|
|
914
920
|
by running exactly the command below, command first, including each bracketed flag only when its value
|
|
@@ -1521,6 +1527,14 @@ Per slice:
|
|
|
1521
1527
|
not a detail to reconcile in your head. Omit `--test-cmd` and the slice is not `review_ready`
|
|
1522
1528
|
unless its ratified `test_plan` is empty — the waiver comes from the plan, not from you.
|
|
1523
1529
|
|
|
1530
|
+
When `.factory.json` declares `verify`, this step also runs it on the slice's commit, after the
|
|
1531
|
+
ratified test command and with `FACTORY_VERIFY_SCOPE=slice` in its environment, and records the
|
|
1532
|
+
result as `repository_verify`. A failing repository verify makes the slice not `review_ready`, the
|
|
1533
|
+
same as a failing test, so the review rejects and the slice retries; an empty `test_plan` does not
|
|
1534
|
+
waive it. Its output appears on stderr: give the failing part of it (the lint, format, or test
|
|
1535
|
+
failure) to the builder with the rejection, because otherwise the retry cannot see what to fix. This
|
|
1536
|
+
is what keeps a one-line lint failure from surfacing only after merge, where it cannot be repaired.
|
|
1537
|
+
|
|
1524
1538
|
`BUILDER_REPORT` is a path and not the report. Write the builder's returned report to
|
|
1525
1539
|
`BUILDER_REPORT=".factory/$R/artifacts/$SLICE_ID-builder-attempt-$SLICE_ATTEMPT.json"` and pass that path,
|
|
1526
1540
|
which keeps the report beside the run's other evidence instead of in argv. It holds `status`, `slice`,
|
|
@@ -1700,7 +1714,7 @@ Gate 3 remains a fresh independent observation.
|
|
|
1700
1714
|
|
|
1701
1715
|
### Post-merge finding routing and repair journal
|
|
1702
1716
|
|
|
1703
|
-
Route a known post-merge failure before any next-wave action. A production defect parks top-level needs-human
|
|
1717
|
+
Route a known post-merge failure before any next-wave action. A production defect parks top-level needs-human. There is no in-band resume for it: production source is never repaired on the integration branch, and a configured command reruns only after a test-only repair. Say so in the reason, and name preserving the feature branch and starting a fresh run as the recovery; do not tell the operator to resume.
|
|
1704
1718
|
Production source is never repaired on the integration branch. Unclassifiable,
|
|
1705
1719
|
interrupted-unknown, invalid-config, unsafe dirty or moved replay, unobservable, journal-invalid, or
|
|
1706
1720
|
repair-exhausted outcomes also terminalize. Clean unchanged repository-verification exhaustion follows
|
package/agents/spec-writer.md
CHANGED
|
@@ -49,6 +49,11 @@ Resolve every ambiguity so builders don't have to. Follow the repository's agent
|
|
|
49
49
|
- Auth/role gating
|
|
50
50
|
- Test plan: unit tests, and acceptance (which criterion maps to which test, and at what level)
|
|
51
51
|
- **Class-wide work:** convert the research inventory into a closed implementation matrix — one row per sink/call site, each assigned an exact primitive/policy, a compatibility (preserve/migrate) or explicit exclusion decision, and a mapped test. No sink is left to the builder to discover.
|
|
52
|
+
- **Document deliverables:** when the issue's deliverable is a specification, contract, or data artifact
|
|
53
|
+
rather than executable behavior, the brief specifies the content each AC requires and nothing more. Do
|
|
54
|
+
not expand "test vectors", "examples", or "fixtures" into an executable model, a validator, or an
|
|
55
|
+
exhaustive corpus unless a criterion says so; an edge case beyond the criteria gets a documented limit,
|
|
56
|
+
not new mechanism.
|
|
52
57
|
|
|
53
58
|
## Output contract
|
|
54
59
|
|
package/agents/work-reviewer.md
CHANGED
|
@@ -78,6 +78,35 @@ When the subject is a **class-wide** requirement — one that **cannot be establ
|
|
|
78
78
|
- **Precedence for late discoveries:** a genuinely required sink, policy, compatibility decision, migration, or test that is missing is a **blocker no matter which review round surfaces it** — record it once in `required_fixes`, carry it into every later review, and REJECT until observed evidence proves it landed. Only *unrelated* new scope or *optional* extra depth on already-decided rows is a non-blocking note; a required in-scope omission is never downgraded to optional just because it appeared in a later round.
|
|
79
79
|
- **Feasibility rule:** reject a brief whose required behavior cannot be implemented within its allowed mechanisms, dependencies, compatibility constraints, or explicit non-goals — for example, demanding grammar-complete or adversarial-input recognition while forbidding every parser, new dependency, or bounded implementation strategy. Surface the smallest explicit dependency, scope, or design decision needed before builders start; do not approve an impossible implementation envelope as "decision-complete."
|
|
80
80
|
|
|
81
|
+
## Specification subjects (documents, contracts, data artifacts)
|
|
82
|
+
|
|
83
|
+
When the deliverable is a document or a data artifact — a specification, a contract, a protocol or
|
|
84
|
+
topology description, a fixture corpus — rather than executable behavior, the class-wide rules above do
|
|
85
|
+
not apply beyond what the acceptance criteria require. For code the in-scope members already exist in the
|
|
86
|
+
repository, so enumerating them ends. For a document the members are whatever it chooses to specify, and
|
|
87
|
+
each answer creates new surface to review; exhaustive enumeration and the late-discovery precedence rule
|
|
88
|
+
then turn every fix into the next round's findings. Three contract runs in a row inflated this way (130k
|
|
89
|
+
lines of corpus JSON, 25k lines of test vectors with a validator inside a Markdown file, a 72 KB
|
|
90
|
+
topology document with its own SQL schemas and a quarantine system).
|
|
91
|
+
One boundary decides everything beyond the criteria, in both directions: **does the content commit an
|
|
92
|
+
implementer to build, store, or conform to something?** Call that an obligation.
|
|
93
|
+
- **The bar is the acceptance criteria, not completeness.** A behavior an AC requires that the document
|
|
94
|
+
omits or gets wrong is a BLOCKER, in any round.
|
|
95
|
+
- **Missing depth is not a finding.** Do not ask for an obligation no AC requires — deeper design,
|
|
96
|
+
internal schemas, tuning values, or handling for rare edge cases. Note it at most; never block on it.
|
|
97
|
+
- **Added obligation beyond the criteria is a finding.** A schema, mechanism, subsystem, executable model,
|
|
98
|
+
or corpus that no AC requires, and that an implementer would have to build, store, or conform to, is
|
|
99
|
+
unapproved scope expansion: require it removed or reduced to a documented limit, the same way unapproved
|
|
100
|
+
scope in a brief is rejected. A reviewer that stops asking for more while accepting whatever the builder
|
|
101
|
+
adds unprompted has not stopped the ratchet.
|
|
102
|
+
- **Explanatory depth is neither.** Rationale, worked examples, and clarification of a required behavior
|
|
103
|
+
add no obligation; accept them without comment.
|
|
104
|
+
- **A documented limit satisfies an edge case.** "X is out of scope" or "behavior is undefined for X" is
|
|
105
|
+
an acceptable answer unless an AC requires X to be handled.
|
|
106
|
+
- **No executable models by default.** A validator, generator, exhaustive corpus, or test vectors is an
|
|
107
|
+
obligation like any other: required only when an AC asks for one. Judge whether what is specified is
|
|
108
|
+
correct and consistent.
|
|
109
|
+
|
|
81
110
|
## What to check, by subject
|
|
82
111
|
|
|
83
112
|
- **`work-decomposer` satisfiability:** for every slice, check that its ratified `test_plan` can be made
|
|
@@ -116,6 +145,9 @@ When the subject is a **class-wide** requirement — one that **cannot be establ
|
|
|
116
145
|
- Code comments follow the target repository's documented policy; do not invent a blanket ban.
|
|
117
146
|
- **Slice discipline:** the diff stays within the slice's `paths` (out-of-lane edits are a finding).
|
|
118
147
|
- The slice's `acceptance` is actually implemented. This is never waived.
|
|
148
|
+
- A present `repository_verify` that did not exit zero is a BLOCKER: the repository's own verify
|
|
149
|
+
(lint, format, full suite) fails on this commit and will fail again after merge, where it cannot be
|
|
150
|
+
repaired. Name the failing check in `required_fixes`. An empty `test_plan` does not waive it.
|
|
119
151
|
- The observed tests cover that acceptance — **unless the slice's ratified `test_plan` is empty**,
|
|
120
152
|
which the plan decided at Gate 2 and which the workflow and `deriveReviewReady` both honour. A
|
|
121
153
|
docs-only slice reviewed against tests it was ratified not to have is rejected forever; that
|
package/bin/factory.js
CHANGED
|
@@ -400,10 +400,10 @@ function branchPoint(run) {
|
|
|
400
400
|
return base;
|
|
401
401
|
}
|
|
402
402
|
|
|
403
|
-
async function writeObservedEvidence({ repo, runDir, runId, subject, attempt, branch, baseRef, worktree, status, blockedReason, claim, testCommand, skipReason, shellCommand, testTimeoutMs }) {
|
|
403
|
+
async function writeObservedEvidence({ repo, runDir, runId, subject, attempt, branch, baseRef, worktree, status, blockedReason, claim, testCommand, skipReason, shellCommand, testTimeoutMs, repositoryVerify = null }) {
|
|
404
404
|
const evidence = buildEvidence({
|
|
405
405
|
subject, attempt, branch, baseRef, worktree, status, blockedReason, claim, runId,
|
|
406
|
-
testCommand, skipReason, shellCommand, testTimeoutMs,
|
|
406
|
+
testCommand, skipReason, shellCommand, testTimeoutMs, repositoryVerify,
|
|
407
407
|
});
|
|
408
408
|
const ancestry = observeAncestry(worktree, baseRef, "HEAD");
|
|
409
409
|
if (ancestry !== "ancestor") {
|
|
@@ -420,7 +420,8 @@ async function writeObservedEvidence({ repo, runDir, runId, subject, attempt, br
|
|
|
420
420
|
|
|
421
421
|
function canonicalRepositoryVerifyEvidence(evidence, { runId, run, integration, verifyCommand }) {
|
|
422
422
|
const baseRef = branchPoint(run);
|
|
423
|
-
|
|
423
|
+
// `repository_verify` is slice-only: null on test-verifier evidence, absent on evidence written before #372.
|
|
424
|
+
const keys = Object.keys(evidence).filter((key) => key !== "repository_verify").sort();
|
|
424
425
|
const commandNames = [
|
|
425
426
|
"git rev-parse HEAD",
|
|
426
427
|
`git --literal-pathspecs diff --name-only -z ${baseRef}...HEAD`,
|
|
@@ -438,7 +439,8 @@ function canonicalRepositoryVerifyEvidence(evidence, { runId, run, integration,
|
|
|
438
439
|
&& ((tests.observed === true && Number.isInteger(tests.exit))
|
|
439
440
|
|| (tests.observed === false && tests.exit === null));
|
|
440
441
|
const reconciliation = evidence.claim_reconciliation;
|
|
441
|
-
return JSON.stringify(keys) === JSON.stringify(
|
|
442
|
+
return JSON.stringify(keys) === JSON.stringify(EVIDENCE_KEYS.filter((key) => key !== "repository_verify").sort())
|
|
443
|
+
&& (evidence.repository_verify ?? null) === null
|
|
442
444
|
&& evidence.subject === "test-verifier" && evidence.run_id === runId
|
|
443
445
|
&& Number.isSafeInteger(evidence.attempt) && evidence.attempt >= 1
|
|
444
446
|
&& evidence.branch === run.branch && evidence.base_ref === baseRef
|
|
@@ -1166,6 +1168,14 @@ const HANDLERS = {
|
|
|
1166
1168
|
+ `expected ${JSON.stringify(slice.test_plan)}; received ${JSON.stringify(flags.testCmd)}`,
|
|
1167
1169
|
);
|
|
1168
1170
|
}
|
|
1171
|
+
// A slice also runs the configured repository `verify`, read from its own committed config (#372).
|
|
1172
|
+
let sliceVerify = null;
|
|
1173
|
+
if (slice) {
|
|
1174
|
+
try { sliceVerify = readRepositoryConfig(worktree, { optional: true }); } catch (error) {
|
|
1175
|
+
if (error instanceof RepositoryConfigError) throw new CliError(error.message);
|
|
1176
|
+
throw error;
|
|
1177
|
+
}
|
|
1178
|
+
}
|
|
1169
1179
|
const skipReason = slice && slice.test_plan.length === 0
|
|
1170
1180
|
? `test_plan for '${subject}' was approved empty at slices-seed`
|
|
1171
1181
|
: null;
|
|
@@ -1179,12 +1189,14 @@ const HANDLERS = {
|
|
|
1179
1189
|
testCommand: flags.repositoryVerify ? repositoryVerify.command : flags.testCmd ? flags.testCmd.split(" ").filter(Boolean) : null,
|
|
1180
1190
|
skipReason, shellCommand: flags.repositoryVerify === true,
|
|
1181
1191
|
testTimeoutMs: flags.repositoryVerify ? repositoryVerify.timeoutMs : undefined,
|
|
1192
|
+
repositoryVerify: sliceVerify && { command: sliceVerify.command, timeoutMs: sliceVerify.timeoutMs },
|
|
1182
1193
|
});
|
|
1183
1194
|
return emit(flags, {
|
|
1184
1195
|
run_id: runId, subject, evidence_ref: evidenceRef(subject),
|
|
1185
1196
|
review_ready: evidence.review_ready, files_changed: evidence.files_changed.length,
|
|
1186
1197
|
tests: evidence.tests.observed ? `exit ${evidence.tests.exit}` : `skipped: ${evidence.tests.skipped_reason}`,
|
|
1187
1198
|
ancestry, mismatches: evidence.claim_reconciliation.mismatches.map((entry) => entry.field),
|
|
1199
|
+
repository_verify: evidence.repository_verify && (evidence.repository_verify.observed ? `exit ${evidence.repository_verify.exit}` : "not run"),
|
|
1188
1200
|
});
|
|
1189
1201
|
},
|
|
1190
1202
|
async init(positional, flags) {
|
package/observe/index.js
CHANGED
|
@@ -14,7 +14,7 @@ export const EVIDENCE_KEYS = Object.freeze([
|
|
|
14
14
|
"subject", "run_id", "attempt", "branch", "base_ref", "worktree", "status", "blocked_reason",
|
|
15
15
|
"worktree_clean",
|
|
16
16
|
"files_changed", "diff_stat", "diff_observed", "commands", "tests", "commit",
|
|
17
|
-
"observed_by", "review_ready", "claim_reconciliation",
|
|
17
|
+
"observed_by", "review_ready", "claim_reconciliation", "repository_verify",
|
|
18
18
|
]);
|
|
19
19
|
|
|
20
20
|
export function git(cwd, args, { runner = spawnSync } = {}) {
|
|
@@ -109,7 +109,7 @@ export function observeAncestry(worktree, ancestor, descendant, options = {}) {
|
|
|
109
109
|
// Attack 1: the test command is run here, by us, and its exit code is recorded
|
|
110
110
|
// from the process rather than from anybody's report. `observed: false` means we
|
|
111
111
|
// could not run it, which is not the same as a pass.
|
|
112
|
-
export function runTests(worktree, command, { runner = spawnSync, skipReason = null, shellCommand = false, timeoutMs = DEFAULT_REPOSITORY_VERIFY_TIMEOUT_MS } = {}) {
|
|
112
|
+
export function runTests(worktree, command, { runner = spawnSync, skipReason = null, shellCommand = false, timeoutMs = DEFAULT_REPOSITORY_VERIFY_TIMEOUT_MS, stdio = "inherit", env = process.env } = {}) {
|
|
113
113
|
if (!command) {
|
|
114
114
|
// Finding 1: defaulting a skip reason let omission manufacture review readiness.
|
|
115
115
|
// Tests must be observed green or explicitly skipped with a caller-declared reason;
|
|
@@ -117,7 +117,7 @@ export function runTests(worktree, command, { runner = spawnSync, skipReason = n
|
|
|
117
117
|
return { cmd: null, exit: null, observed: false, skipped_reason: skipReason };
|
|
118
118
|
}
|
|
119
119
|
const result = shellCommand
|
|
120
|
-
? runner(command, [], { cwd: worktree, shell: true, stdio
|
|
120
|
+
? runner(command, [], { cwd: worktree, shell: true, stdio, env, timeout: timeoutMs })
|
|
121
121
|
: runner(command[0], command.slice(1), { cwd: worktree, encoding: "utf8", shell: false });
|
|
122
122
|
const exit = Number.isInteger(result?.status) ? result.status : null;
|
|
123
123
|
return { cmd: shellCommand ? command : command.join(" "), exit, observed: exit !== null, skipped_reason: null };
|
|
@@ -133,6 +133,9 @@ export function deriveReviewReady(evidence) {
|
|
|
133
133
|
// function alone; with the term only at write time, every mismatched record read back as
|
|
134
134
|
// tampered and wedged its slice where `slice blocked` could never record it.
|
|
135
135
|
if (evidence.claim_reconciliation?.mismatches?.length > 0) return false;
|
|
136
|
+
// Absent on evidence written before #372, and null when no `verify` is configured.
|
|
137
|
+
const verify = evidence.repository_verify;
|
|
138
|
+
if (verify && !(verify.observed === true && verify.exit === 0)) return false;
|
|
136
139
|
// A tree with uncommitted changes cannot produce evidence about the commit it
|
|
137
140
|
// claims, whatever the tests said.
|
|
138
141
|
if (evidence.worktree_clean !== true) return false;
|
|
@@ -201,7 +204,7 @@ export function privilegedPaths(filesChanged) {
|
|
|
201
204
|
|| PRIVILEGED_EXACT.includes(file));
|
|
202
205
|
}
|
|
203
206
|
|
|
204
|
-
export function buildEvidence({ subject, runId, attempt, branch, baseRef, worktree, status, blockedReason = null, claim = null, testCommand = null, skipReason = null, shellCommand = false, testTimeoutMs = DEFAULT_REPOSITORY_VERIFY_TIMEOUT_MS, options = {} }) {
|
|
207
|
+
export function buildEvidence({ subject, runId, attempt, branch, baseRef, worktree, status, blockedReason = null, claim = null, testCommand = null, skipReason = null, shellCommand = false, testTimeoutMs = DEFAULT_REPOSITORY_VERIFY_TIMEOUT_MS, repositoryVerify = null, options = {} }) {
|
|
205
208
|
// Cleanliness is established before anything else is observed, because every later
|
|
206
209
|
// fact - the diff, the commit, and above all the test result - is only about the
|
|
207
210
|
// recorded commit if the tree has nothing uncommitted in it.
|
|
@@ -210,8 +213,20 @@ export function buildEvidence({ subject, runId, attempt, branch, baseRef, worktr
|
|
|
210
213
|
// Tests are not run at all against a dirty tree: running them would produce a
|
|
211
214
|
// result about bytes that are not going to merge.
|
|
212
215
|
const tests = cleanliness.clean
|
|
213
|
-
? runTests(worktree, testCommand, { ...options, skipReason, shellCommand, timeoutMs: testTimeoutMs
|
|
216
|
+
? runTests(worktree, testCommand, { ...options, skipReason, shellCommand, timeoutMs: testTimeoutMs,
|
|
217
|
+
...(shellCommand ? { env: { ...process.env, FACTORY_VERIFY_SCOPE: "integration" } } : {}) })
|
|
214
218
|
: { cmd: testCommand ? (shellCommand ? testCommand : testCommand.join(" ")) : null, exit: null, observed: false, skipped_reason: null };
|
|
219
|
+
// Enforcement (false green, #372): a slice ran only its ratified test command, so the repository's
|
|
220
|
+
// configured `verify` (lint, format, full suite) first ran after merge, where production repair is
|
|
221
|
+
// forbidden and one Clippy warning parked a 13-slice run. The same suite now runs on the slice's commit.
|
|
222
|
+
// Skipped after a failing test run, which is already not review-ready; its output goes to stderr so
|
|
223
|
+
// `observe --json` stays one JSON object. `FACTORY_VERIFY_SCOPE` tells the command which run this is.
|
|
224
|
+
const verifyRuns = repositoryVerify && cleanliness.clean && !(tests.observed && tests.exit !== 0);
|
|
225
|
+
const verified = !repositoryVerify ? null : verifyRuns
|
|
226
|
+
? (({ cmd, exit, observed }) => ({ cmd, exit, observed }))(runTests(worktree, repositoryVerify.command,
|
|
227
|
+
{ ...options, shellCommand: true, timeoutMs: repositoryVerify.timeoutMs, stdio: ["ignore", 2, 2],
|
|
228
|
+
env: { ...process.env, FACTORY_VERIFY_SCOPE: "slice" } }))
|
|
229
|
+
: { cmd: repositoryVerify.command, exit: null, observed: false };
|
|
215
230
|
|
|
216
231
|
// Third round, finding 1: cleanliness was a pre-test snapshot, so a test that wrote
|
|
217
232
|
// tracked files left the tree dirty while the evidence still claimed a clean HEAD -
|
|
@@ -255,6 +270,7 @@ export function buildEvidence({ subject, runId, attempt, branch, baseRef, worktr
|
|
|
255
270
|
observed_by: "orchestrator",
|
|
256
271
|
review_ready: false,
|
|
257
272
|
claim_reconciliation: { claimed: false, mismatches: [] },
|
|
273
|
+
repository_verify: verified,
|
|
258
274
|
};
|
|
259
275
|
evidence.claim_reconciliation = reconcileClaim(claim, evidence);
|
|
260
276
|
evidence.review_ready = deriveReviewReady(evidence);
|