orchestrator-workflow 0.30.0 → 0.32.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,432 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
+
## [0.32.0] - 2026-09-11
|
|
11
|
+
|
|
12
|
+
### Added
|
|
13
|
+
|
|
14
|
+
- A review-method axis, orthogonal to the effort tier: every reviewer
|
|
15
|
+
briefing now names `review_method: normal | rigorous | adversarial`
|
|
16
|
+
(`assets/agents/reviewer.md`, SKILL.md step 7, the kit-fence
|
|
17
|
+
Scaling-delegation text). The three methods are obligation sets, not
|
|
18
|
+
personas: `normal` reads the diff and spec and runs the declared tests
|
|
19
|
+
once, for docs/renames/batch cosmetics; `rigorous` (the default when a
|
|
20
|
+
briefing names none) adds an independent extract, a base-attribution
|
|
21
|
+
control, and mandatory reproduction of every empirical claim (the
|
|
22
|
+
pre-existing `reproduction`/`matches_implementer_claim` requirement);
|
|
23
|
+
`adversarial` adds one discriminating probe or negative control per
|
|
24
|
+
acceptance criterion, an active search of the neighbouring scenario
|
|
25
|
+
space, an attempt to break the claimed invariant, and a list of break
|
|
26
|
+
attempts that failed. `adversarial` and `rigorous` both carry a
|
|
27
|
+
withdrawal rule: a finding that does not reproduce on a second attempt
|
|
28
|
+
with a corrected harness is withdrawn in the same round and reported
|
|
29
|
+
under a `withdrawn` list with the reason, so the method cannot buy false
|
|
30
|
+
positives; emit `withdrawn: []` when nothing was withdrawn. The reviewer
|
|
31
|
+
output contract (both copies, `reviewer.md` and SKILL.md) gains
|
|
32
|
+
`method_applied` and `withdrawn`; until the grounding-mcp reader parses
|
|
33
|
+
the marker (agent-grounding follow-up 5df7b809), the orchestrator checks
|
|
34
|
+
by hand that the return's `method_applied` matches the briefing's
|
|
35
|
+
`review_method`, resupplying a mismatch or omission rather than
|
|
36
|
+
accepting it.
|
|
37
|
+
`assets/templates/05-review-findings.md` gains a `Method` line per review
|
|
38
|
+
round, outside the pinned Findings table
|
|
39
|
+
(`test/template-markers.test.ts` pins it); the grounding-mcp
|
|
40
|
+
completeness reader does not parse it yet, tracked as a follow-up in the
|
|
41
|
+
agent-grounding repo. SKILL.md's selection rule: `adversarial` at
|
|
42
|
+
minimum for security judgment, install/deploy scripts, hand-edited
|
|
43
|
+
lockfiles, cross-major overrides, or anything the operator flags
|
|
44
|
+
high-risk; `normal` only for docs, renames, or batch cosmetics;
|
|
45
|
+
`rigorous` otherwise; never `adversarial` on the `-medium` reviewer tier
|
|
46
|
+
(budget mismatch). Tiers themselves are unchanged. Anchored by the
|
|
47
|
+
pandora run `2026-09-11-cve-sweep`: reviews R1, R5, R11, R13 and R14
|
|
48
|
+
found the Critical and High findings by probing; R4, R6 and R9 on the
|
|
49
|
+
default method returned no findings and R8 only two informational lows;
|
|
50
|
+
R5 withdrew a harness artefact. The selection rule stays advisory, not
|
|
51
|
+
an AGENTS.md rule, until an A/B (same tasks, run once under `rigorous`
|
|
52
|
+
and once under `adversarial`, counting real Critical/High findings and
|
|
53
|
+
findings withdrawn) is recorded.
|
|
54
|
+
|
|
55
|
+
- A citation-sibling-drift guard (`test/docs-consistency.test.ts`, next to
|
|
56
|
+
the existing anchor-load-bearing checks) catches a citation that resolves
|
|
57
|
+
and anchors correctly on its own but names the wrong sibling among a run
|
|
58
|
+
of near-identical citations, a class neither okf-kit's `citations-resolve`
|
|
59
|
+
rule nor the local anchor guards can see, because both check an anchor
|
|
60
|
+
only inside its own cited range. Two rules, applied per paragraph: (a) the
|
|
61
|
+
same `file:range#anchor` cited twice in one paragraph, unless allowlisted
|
|
62
|
+
with a reason; (b) a string anchor's text also occurring, uncited, at
|
|
63
|
+
another line of the same target within a 20-line window (widened from 10
|
|
64
|
+
in review round 2, see below) while the paragraph cites a sibling range
|
|
65
|
+
of that file, unless allowlisted. Fixtures
|
|
66
|
+
reproduce three review findings that shared this shape and were the
|
|
67
|
+
motivation for the guard: three sibling `it`-block citations collapsing
|
|
68
|
+
onto one range twice, the third never cited; three per-harness bullet
|
|
69
|
+
citations doing the same; two logically distinct assertions collapsing
|
|
70
|
+
onto one shared range and anchor text, the second's own line never cited.
|
|
71
|
+
Run over the current bundle, every hit either rule reports is read
|
|
72
|
+
against its own target file and the citing paragraph, then fixed or
|
|
73
|
+
allowlisted; the measured per-rule hit counts live in `docs/okf/log.md`
|
|
74
|
+
with the classification that produced them, not here, so the two sites
|
|
75
|
+
cannot drift apart. The recurring coincidence shapes are a doc-wide
|
|
76
|
+
"topic sentence, then repeat as the closing list item" convention for
|
|
77
|
+
rule (a), and a short or common token -- a keyword, a mirrored field on
|
|
78
|
+
twin interfaces, a comment restating a literal, a test-assertion idiom on
|
|
79
|
+
an adjacent line, a reused local variable name -- recurring near a real
|
|
80
|
+
citation for rule (b). Lives in this file rather than as an okf-kit rule
|
|
81
|
+
because this
|
|
82
|
+
suite already runs on every PR while an okf-kit rule needs a release and
|
|
83
|
+
a fleet pin bump first; an opt-in okf-kit rule for the same class is a
|
|
84
|
+
named follow-up candidate once this guard has proven itself.
|
|
85
|
+
- Review round 2 of the citation-sibling-drift guard above (task agent-dx
|
|
86
|
+
9f72ae6d): the round-1 review found 7 of the round's 18 raw hits were not
|
|
87
|
+
coincidental at all -- real mis-pointed citations that got allowlisted
|
|
88
|
+
instead of fixed, because the round-1 pass classified every hit by range
|
|
89
|
+
and reason without re-deriving each cited claim's real evidence line by
|
|
90
|
+
line. All seven re-pointed to their real evidence, citation-only, no
|
|
91
|
+
content changes: a duplicate `docs-consistency.test.ts` self-citation
|
|
92
|
+
whose second claim's real test sat 26 lines below the first
|
|
93
|
+
(`subagent-contracts-superset.md`); an `init.test.ts` range that stopped
|
|
94
|
+
6 lines short of the `model: opus` assertion it named
|
|
95
|
+
(`model-preselection.md`); a `SKILL.md` duplicate whose first claim's
|
|
96
|
+
real text sat just above the cited line (`run-state-lifecycle-and-
|
|
97
|
+
markers.md`); an `init.ts` comment cited in place of the real
|
|
98
|
+
`installKitFile` call it restates (`install-fence-mechanics.md`); an
|
|
99
|
+
`init.ts` range crossing from one branch of `installKitFile` into
|
|
100
|
+
another branch's own record line (`install-fence-mechanics.md`); an
|
|
101
|
+
`init.ts` range naming the wrong call for an `opencodeEffortLine(...)`
|
|
102
|
+
claim (`model-preselection.md`, cited from two spellings of the same
|
|
103
|
+
path); and a milder range that stopped 1 line short of the parameter its
|
|
104
|
+
own sentence's second half named (`install-fence-mechanics.md`). Widened
|
|
105
|
+
`SIBLING_GUARD_WINDOW` from 10 to 20 once a real case fell just outside
|
|
106
|
+
it (two genuinely distinct `init.ts` notes sharing one message, 18 lines
|
|
107
|
+
apart, both legitimate and now allowlisted per doc); re-triaged every
|
|
108
|
+
additional hit the wider window surfaced against the bundle, fixing or
|
|
109
|
+
allowlisting each with a reason stating what the cited line actually
|
|
110
|
+
says (see `docs/okf/log.md` for the full re-triage and the re-measured
|
|
111
|
+
counts). Added `anchorKey` (first 8 hex chars of a sha256 over the
|
|
112
|
+
finding's own anchor text, computed at test time, never stored literally
|
|
113
|
+
in the array) to every allowlist entry and to the match, plus a test
|
|
114
|
+
asserting every entry matched at least one finding on the current
|
|
115
|
+
bundle, closing a gap where a range-only match would silently exempt any
|
|
116
|
+
future, differently-anchored finding on the same range. Added a fixture
|
|
117
|
+
at the real batch-39 S3 geometry (a literal duplicate citation, its real
|
|
118
|
+
sibling 15 lines away, not the original fixture's 10-line near-miss
|
|
119
|
+
range) asserting both rules' behaviour, and a negative fixture pinning
|
|
120
|
+
that the duplicate-citation rule fires regardless of window, since it is
|
|
121
|
+
a pairing comparison, not a windowed one. Two coverage gaps noted in the
|
|
122
|
+
guard's own comment and here rather than closed this round: a path-less
|
|
123
|
+
continuation citation (`:N-M#"..."`, whose path is implied by the
|
|
124
|
+
preceding citation) never matches the citation regex, so this guard
|
|
125
|
+
cannot see one -- extending the regex to resolve a continuation's
|
|
126
|
+
implied path is a named follow-up; and a citation-shaped string inside a
|
|
127
|
+
fenced ` ``` ` code block is now skipped rather than matched (closing the
|
|
128
|
+
reverse risk of misreading a code sample as a citation), a cheap
|
|
129
|
+
addition alongside the rest of this round's work.
|
|
130
|
+
- Review round 3 of the citation-sibling-drift guard above (task agent-dx
|
|
131
|
+
9f72ae6d): a second consecutive review round found allowlist entries that
|
|
132
|
+
certified real wrong-sibling drift, so this round changes the mechanism
|
|
133
|
+
rather than only the entries. An allowlist entry now records the
|
|
134
|
+
GEOMETRY it was cleared against -- the target-file line(s) carrying the
|
|
135
|
+
uncited identical anchor text for a rule-(b) entry, the doc line the
|
|
136
|
+
repeat sits on for a rule-(a) one -- and that geometry is part of the
|
|
137
|
+
match, so an entry exempts only the hit it was actually reviewed for: a
|
|
138
|
+
new uncited occurrence next to an already-cleared one, or a repeat that
|
|
139
|
+
moved to another doc line, fails instead of inheriting the old verdict. A
|
|
140
|
+
test re-reads those recorded lines against the current files
|
|
141
|
+
independently of the guard's own output (the anchor text is re-derived
|
|
142
|
+
from the doc's own citation, since the array deliberately stores a hash
|
|
143
|
+
rather than the literal text), so an entry whose situation no longer
|
|
144
|
+
exists goes red instead of silently exempting a different one. The
|
|
145
|
+
free-form `reason` field is replaced by `claim`: one sentence naming what
|
|
146
|
+
the citing sentence describes and why the cited line, rather than the
|
|
147
|
+
uncited sibling, is its evidence, written so a reviewer can falsify it by
|
|
148
|
+
reading exactly the two lines the entry names. Process, recorded in
|
|
149
|
+
`docs/okf/log.md` with each round's classification: an allowlist entry is
|
|
150
|
+
accepted only on an INDEPENDENT review classification of the hit, never
|
|
151
|
+
on the reading of whoever implemented or re-pointed the citation, which
|
|
152
|
+
is how both earlier rounds' wrong verdicts reached a green suite.
|
|
153
|
+
Re-pointed the pair this round's review found (a sentence about the
|
|
154
|
+
dropped-role tier-variant SUB-loop citing the enclosing loop's own note
|
|
155
|
+
range, in two docs) to the sub-loop's own note, dropped their entries,
|
|
156
|
+
and re-triaged every remaining hit at the current window against its
|
|
157
|
+
target file. Also citation-only: a fence-contract citation that stopped
|
|
158
|
+
short of the assertion its sentence names, a run-state citation one line
|
|
159
|
+
short of the sentence it supports, and a slicer-superset citation whose
|
|
160
|
+
sentence's second half is now cited from the test that actually pins it.
|
|
161
|
+
Two more fixtures: rule (b) firing at the guard's window and staying
|
|
162
|
+
silent at the round-1 value of 10 for an uncited occurrence 15 lines
|
|
163
|
+
outside the cited range (the window was previously pinned only
|
|
164
|
+
indirectly, through the no-dead-exemption test), and a doc that ends
|
|
165
|
+
inside an unclosed fence now failing loudly instead of silently dropping
|
|
166
|
+
every citation after the stray delimiter, paired with an assertion that
|
|
167
|
+
every bundle doc yields at least one citation.
|
|
168
|
+
- Review round 4 of the citation-sibling-drift guard above (task agent-dx
|
|
169
|
+
9f72ae6d): a third consecutive review classified every allowlist entry
|
|
170
|
+
by re-reading the two lines each `claim` names, rather than trusting the
|
|
171
|
+
prior round's verdicts; none certified real drift, but two claims were
|
|
172
|
+
inaccurate and one more citation was mis-paired in a shape the guard
|
|
173
|
+
itself cannot see. Rule (a)'s match compared only the finding's second
|
|
174
|
+
citation line, which a `return true;` mutant of that comparison
|
|
175
|
+
survives, and which also could not tell a two-citation entry's cleared
|
|
176
|
+
repeat from a THIRD, unreviewed repeat sharing the same second line;
|
|
177
|
+
fixed with a dedicated fixture and a repeat-count check. The allowlist
|
|
178
|
+
entry's match and its independent geometry re-check both gained
|
|
179
|
+
`paragraphLine`, the doc line of the finding's own first citation: an
|
|
180
|
+
entry was previously keyed by (doc, kind, real target, range, anchorKey,
|
|
181
|
+
recorded geometry) alone, so the same coincidence recurring in a SECOND,
|
|
182
|
+
unreviewed paragraph of a doc would silently inherit the first
|
|
183
|
+
paragraph's verdict -- exactly the shape one entry was carrying (the
|
|
184
|
+
same `init.ts` range cited, and separately drifting, from two paragraphs
|
|
185
|
+
of `model-preselection.md`); the second paragraph's citation is now
|
|
186
|
+
re-pointed to its own, different evidence instead, so the entry covers
|
|
187
|
+
one paragraph only. The geometry re-check's duplicate-citation branch
|
|
188
|
+
dropped a near-tautological "some citation exists at the recorded line"
|
|
189
|
+
check (true of any citation the extractor produces, by construction) for
|
|
190
|
+
one that reads the group's own citations, sorts them into document
|
|
191
|
+
order, and checks the recorded lines by POSITION -- closing a mutant
|
|
192
|
+
(`if (false)` on the old guard) no existing fixture caught. The bare
|
|
193
|
+
`claim.length > 40` sanity check now also rejects a claim that never
|
|
194
|
+
names one of its own entry's recorded lines, closing the gap that let
|
|
195
|
+
three `subagent-contracts-superset.md` entries carry a long claim that
|
|
196
|
+
never actually pointed at its own geometry. One inaccurate claim
|
|
197
|
+
(`model-preselection.md`) said an uncited line named a "codex-only
|
|
198
|
+
effort field"; it is opencode's own field for a non-Claude-family,
|
|
199
|
+
non-Ollama provider, not a codex field at all, and no anchor exists that
|
|
200
|
+
can widen the citation to cover it under this file's own occurrence-cap
|
|
201
|
+
rule, so the claim was corrected instead. One real mis-pairing:
|
|
202
|
+
`install-fence-mechanics.md` cited the OUTER per-dropped-role loop's
|
|
203
|
+
gate/note for a sentence about the tier-variant SUB-loop, and the
|
|
204
|
+
sub-loop's own gate/note for a sentence about the base-file note --
|
|
205
|
+
swapped, citation-only, no content change. Known limit, unclosed this
|
|
206
|
+
round: neither rule catches a citation that resolves and anchors cleanly
|
|
207
|
+
but simply names the WRONG target -- no duplication, no anchor text
|
|
208
|
+
recurring nearby -- which is exactly the shape this round's real
|
|
209
|
+
mis-pairing was; both citations passed every existing check (including
|
|
210
|
+
this guard) because nothing about either one, read alone or against its
|
|
211
|
+
paragraph's siblings, looks wrong.
|
|
212
|
+
- Citation-sibling-drift guard, continuation-citation coverage (task
|
|
213
|
+
agent-dx b50fd903): closed the guard's own documented coverage gap
|
|
214
|
+
(round 1) that a path-less continuation citation (`:N-M#"..."`, whose
|
|
215
|
+
path is implied by the preceding FULL citation earlier in the same
|
|
216
|
+
paragraph) never matched `ANCHOR_CITATION_RE`, so the guard could not
|
|
217
|
+
see one at all -- the form `model-preselection.md` alone carries four
|
|
218
|
+
of. `extractSiblingGuardCitations` now also matches the anchored,
|
|
219
|
+
path-less tail on its own (`ANCHOR_CONTINUATION_CITATION_RE`, anchor
|
|
220
|
+
group required so a bare `:N-M` digit pair in ordinary prose is never
|
|
221
|
+
mistaken for one) and resolves it against `governingPathByParagraph`,
|
|
222
|
+
the nearest preceding full citation's own `citedPath` in the same
|
|
223
|
+
paragraph -- the same "nearest preceding, same paragraph" BINDING RULE
|
|
224
|
+
okf-kit's own short-form/continuation citations use in
|
|
225
|
+
`citations-resolve.ts`. That mirrors the binding rule only, not the
|
|
226
|
+
grammar (review round 3 correction: this bundle's anchored, path-less
|
|
227
|
+
`:N-M#"..."` form IS backtick-wrapped; an earlier version of this
|
|
228
|
+
bullet said it was not). okf-kit's own `CONT_COLON_RE` still misses it
|
|
229
|
+
because its own closing backtick has to follow the digit range
|
|
230
|
+
immediately, and this form's closing backtick follows the `#"anchor"`
|
|
231
|
+
tail instead; `SHORT_FORM_COLON_RE` misses it too, both because a match
|
|
232
|
+
right after a backtick is skipped and because no serial connective
|
|
233
|
+
("and", "also", ...) precedes it either -- okf-kit sees these citations
|
|
234
|
+
as nothing at all, not merely as unresolved ones. Two fixtures pin the
|
|
235
|
+
closed gap: a path-less
|
|
236
|
+
continuation duplicating its governing citation's own range and anchor
|
|
237
|
+
is flagged by the duplicate rule (drifted/corrected), and a
|
|
238
|
+
discriminating fixture with two different full citations in one
|
|
239
|
+
paragraph before the continuation, which only passes when the
|
|
240
|
+
continuation binds to the NEARER of the two, not the paragraph's first.
|
|
241
|
+
Run against the current bundle, the newly-visible continuation
|
|
242
|
+
citations in `model-preselection.md` produced no new SIBLING-GUARD
|
|
243
|
+
finding (duplicate-citation/wrong-sibling-anchor), allowlisted or
|
|
244
|
+
otherwise -- see `docs/okf/log.md`. That measurement did not cover
|
|
245
|
+
whether each continuation's own anchor still resolved against its
|
|
246
|
+
target at head; it did not, by the time this round's own CHANGELOG line
|
|
247
|
+
shift landed a few commits later -- see the round-2 follow-up bullet
|
|
248
|
+
below.
|
|
249
|
+
- Citation-sibling-drift guard, okf-kit-porting decision (task agent-dx
|
|
250
|
+
b50fd903, run `.ai/runs/2026-09-08-open-pool-batch44` D-006): the guard
|
|
251
|
+
stays kit-local (this package's own `test/docs-consistency.test.ts`),
|
|
252
|
+
not ported to okf-kit as an opt-in `citations-sibling` check. Who pays:
|
|
253
|
+
kit-local means only this package's own OKF bundle is guarded by it,
|
|
254
|
+
and every other fleet bundle with sibling citations (a paragraph
|
|
255
|
+
repeating, or near-duplicating, one of its own citations) stays
|
|
256
|
+
unguarded until each such bundle's own docs-consistency-style suite
|
|
257
|
+
grows the same check by hand; porting would instead put the maintenance
|
|
258
|
+
on okf-kit's maintainers, who would then carry the rule itself, the
|
|
259
|
+
allowlist shape (recorded geometry, a falsifiable one-sentence claim,
|
|
260
|
+
and the independent-review-classification process this file's own
|
|
261
|
+
allowlist process block already documents) as a public, cross-repo
|
|
262
|
+
contract, and a fleet-wide pin bump on every fix to it. Trigger to
|
|
263
|
+
revisit: a second fleet bundle observed carrying real sibling-citation
|
|
264
|
+
drift in a review pass (not merely plausible in the abstract) reopens
|
|
265
|
+
the port decision.
|
|
266
|
+
- Citation-sibling-drift guard, continuation-citation coverage, review
|
|
267
|
+
round 2 (task agent-dx b50fd903): the round-1 bullet above added
|
|
268
|
+
continuation-citation EXTRACTION but not RESOLUTION at the three
|
|
269
|
+
`matchAll(ANCHOR_CITATION_RE)` sites that check "anchor on last content
|
|
270
|
+
line", "anchor <=3 times file-wide/exactly once in-range", "unanchored
|
|
271
|
+
citation", and "citation stays inside one describe/it/test block" --
|
|
272
|
+
and, separately (review round 3 correction: an earlier version of this
|
|
273
|
+
bullet blamed a same-round CHANGELOG.md line shift for staling
|
|
274
|
+
`src/init.ts`'s own line numbers; false, since editing CHANGELOG.md
|
|
275
|
+
cannot move a different file's lines, and this branch had not touched
|
|
276
|
+
`src/` yet at that point), all four of `model-preselection.md`'s
|
|
277
|
+
continuation citations were already stale at this task's own merge
|
|
278
|
+
base: `src/init.ts` last moved on 2026-09-05 (`60cb546`, task agent-dx
|
|
279
|
+
#184, native Codex routing), and nothing detected the drift, since this
|
|
280
|
+
round-1 bullet's own fix added continuation extraction only, not
|
|
281
|
+
resolution, at the three sites above -- so the round-1 bundle re-run's
|
|
282
|
+
"zero unallowlisted findings" never actually re-checked those four
|
|
283
|
+
anchors' text against `src/init.ts` at head. All four re-pointed
|
|
284
|
+
(citation-only) against `src/init.ts` at head; the fourth (into
|
|
285
|
+
`composeClaudeAgentVariant`'s own call site) needed a freshly-derived
|
|
286
|
+
anchor since its old text no longer occurs there at all (the call now
|
|
287
|
+
takes three arguments, not two). `extractSiblingGuardCitations` is now
|
|
288
|
+
also what the three resolution sites above call, instead of each
|
|
289
|
+
running its own bespoke `matchAll(ANCHOR_CITATION_RE)` loop, so a
|
|
290
|
+
resolved continuation is checked by those properties exactly like a
|
|
291
|
+
full citation is; this is what would have caught the stale
|
|
292
|
+
`model-preselection.md` anchors, had it existed in round 1. Four
|
|
293
|
+
further gaps closed in the same extractor: `governingPathByParagraph`
|
|
294
|
+
now resets (not merely leaves stale) on an unresolved/ambiguous full
|
|
295
|
+
citation, matching okf-kit's own reset behaviour; a continuation-match
|
|
296
|
+
overlap filter now also drops a match whose immediately preceding text
|
|
297
|
+
is path-shaped regardless of file extension, so a citation into an
|
|
298
|
+
extension `ANCHOR_CITATION_RE` does not recognise (a `.toml`, a `.tsx`)
|
|
299
|
+
cannot have its own tail misread as a phantom continuation; the
|
|
300
|
+
left-to-right, nearest-preceding ordering of full and continuation
|
|
301
|
+
matches on one line, and the paragraph-scoping property (a continuation
|
|
302
|
+
never resolves across a paragraph boundary), are now both pinned by
|
|
303
|
+
fixtures rather than only described in a comment; and
|
|
304
|
+
`ANCHOR_CONTINUATION_CITATION_RE`'s anchor alternation (previously a
|
|
305
|
+
hand copy of `ANCHOR_CITATION_RE`'s own group 4 pattern) is now asserted
|
|
306
|
+
to be a substring of it, throwing at module load on drift. Residual,
|
|
307
|
+
unclosed this round, named next to the pre-existing fenced-code-block
|
|
308
|
+
gap in the extractor's own comment: this guard's continuation form
|
|
309
|
+
mirrors okf-kit's short-form BINDING RULE only, not its grammar (see
|
|
310
|
+
the round-1 bullet above), so okf-kit's own `citations-resolve` rule
|
|
311
|
+
still cannot see one of these citations at all; closing that gap means
|
|
312
|
+
either changing okf-kit's own grammar (out of this task's scope) or
|
|
313
|
+
accepting the guard-only coverage as the design. Review round 3 (LOW
|
|
314
|
+
5): the three resolution sites above inherit the extractor's two other
|
|
315
|
+
latent costs too, since they now call it -- a citation inside a fenced
|
|
316
|
+
code block goes unchecked at those sites as well, and a document ending
|
|
317
|
+
inside an unclosed fence makes them throw, same as this guard -- both
|
|
318
|
+
accepted as the same currently-unused-shape cost, not a new one.
|
|
319
|
+
- Citation-sibling-drift guard, docs/okf/log.md's own citations (task
|
|
320
|
+
agent-dx b50fd903, review round 3, D-037): review round 2 found that
|
|
321
|
+
`log.md` -- excluded from `ANCHOR_OKF_DOCS` and therefore from every
|
|
322
|
+
guard above, and from okf-kit's own citation grammar too -- is read by
|
|
323
|
+
nothing, so citation-shaped historical text written into a log entry
|
|
324
|
+
goes unchecked; round 1 of this task had already removed such text from
|
|
325
|
+
one entry (`0f054d2`) and round 2 wrote the same shape into its own
|
|
326
|
+
entry again. Rather than another round of rephrasing that recurs on the
|
|
327
|
+
next entry, `log.md` gets its own guard in
|
|
328
|
+
`test/docs-consistency.test.ts`: every full, anchored citation it writes
|
|
329
|
+
must still resolve at head (the target exists, the anchor text sits
|
|
330
|
+
somewhere inside the cited range), and it may never carry the bundle's
|
|
331
|
+
path-less continuation form at all, since that form has no
|
|
332
|
+
governing-citation semantics in `log.md` -- nothing resolves a
|
|
333
|
+
continuation written there against anything, so it can only be stale
|
|
334
|
+
prose dressed as a citation. Fixtures pin both rules both ways (a stale
|
|
335
|
+
full citation fails, an unresolvable path fails, a continuation form
|
|
336
|
+
fails even when it would resolve, a clean entry passes); run against
|
|
337
|
+
the current bundle, both checks are clean, and every citation-shaped
|
|
338
|
+
historical value the run reported was rephrased as plain prose, in the
|
|
339
|
+
round-2 entry and in an older entry from task 9f72ae6d. Review round 4
|
|
340
|
+
(D-050) removed this bullet's original hit counts rather than
|
|
341
|
+
correcting them: they were typed by hand and did not match what the
|
|
342
|
+
guard produces (see the round-4 bullet below for the rule and for where
|
|
343
|
+
the live figures live instead).
|
|
344
|
+
The round-2 entry's own false same-round-CHANGELOG-line-shift narration
|
|
345
|
+
is corrected in place (see the review round 3 correction two bullets
|
|
346
|
+
above for the identical fix here); `docs/okf/index.md`'s Maintenance
|
|
347
|
+
section now names the new guard.
|
|
348
|
+
- Citation scanning is paragraph-joined, and the log guard is no longer
|
|
349
|
+
silenceable (task agent-dx b50fd903, review round 4, D-050): both
|
|
350
|
+
citation scanners in `test/docs-consistency.test.ts` matched per
|
|
351
|
+
physical line while every doc in this bundle hard-wraps its prose, so a
|
|
352
|
+
citation whose own text straddled a wrap matched neither regex and was
|
|
353
|
+
invisible to every check built on them -- and re-running the regexes
|
|
354
|
+
over raw document text cannot close that, since the string-anchor
|
|
355
|
+
alternation forbids a newline inside the anchor by construction. Both
|
|
356
|
+
scanners now consume one shared `citationScanParagraphs` helper that
|
|
357
|
+
joins each paragraph's lines the way a hard wrap split them and maps
|
|
358
|
+
every joined offset back to its physical line, so findings, allowlist
|
|
359
|
+
geometry and failure messages still name real doc lines; a wrapped full
|
|
360
|
+
citation with a stale anchor and a wrapped continuation form are each
|
|
361
|
+
pinned by their own fixture. The same helper carries the one fence
|
|
362
|
+
pass, so the `log.md` guard inherits the unbalanced-fence throw the
|
|
363
|
+
round-3 hand copy had left behind (a single stray ``` excused every
|
|
364
|
+
citation after it), and its non-vacuity floor now carries the live
|
|
365
|
+
count in its own computed test name. The unanchored-citation brake and
|
|
366
|
+
the block-straddle collector take their doc set and resolver as
|
|
367
|
+
parameters, like the string-anchor collector already did, so the
|
|
368
|
+
"a resolved continuation survives this collector" property is pinned by
|
|
369
|
+
a synthetic doc set at all three sites instead of by source text alone;
|
|
370
|
+
the brake's examined count is pinned as an exact delta (one added full
|
|
371
|
+
citation raises it by one, one added continuation by one more) rather
|
|
372
|
+
than by a floor a dropped-continuation mutant could sink under. The
|
|
373
|
+
`log.md` resolver rejects a cited path carrying a `..` segment and
|
|
374
|
+
asserts repository containment on its on-disk fallback, and a bare
|
|
375
|
+
basename that also exists at the repository root is reported ambiguous
|
|
376
|
+
with both candidates named instead of silently binding to this
|
|
377
|
+
package's own file; the deeper repo-wide basename ambiguity okf-kit
|
|
378
|
+
reports is still bound unconditionally by `anchorScopeResolve()`'s own
|
|
379
|
+
documented design, named as the residual. Convention this round
|
|
380
|
+
installs (D-050): a log entry or CHANGELOG bullet writes no hand-typed
|
|
381
|
+
count of what a guard found; the live figures are the guards' own
|
|
382
|
+
computed test names, read off a passing run.
|
|
383
|
+
|
|
384
|
+
### Fixed
|
|
385
|
+
|
|
386
|
+
- Three unpinned properties from the review round 4 lows above
|
|
387
|
+
(task agent-dx 4ece8e1e) now have a discriminating fixture each: the
|
|
388
|
+
`log.md` resolver's on-disk fallback is checked against an existing,
|
|
389
|
+
outside-the-repository absolute path so its own containment conjunct is
|
|
390
|
+
no longer redundant with the `..`-segment rejection; `extractSiblingGuard
|
|
391
|
+
Citations` gets its own wrapped-citation fixture (full citation and
|
|
392
|
+
continuation each straddling a hard line break), independent of the
|
|
393
|
+
`log.md` guard's; and the previously duplicated `PATH_SHAPED_BEFORE_RE`
|
|
394
|
+
path-shaped regex is now one module-scope const both call sites read,
|
|
395
|
+
so the two copies can no longer drift apart.
|
|
396
|
+
|
|
397
|
+
## [0.31.0] - 2026-09-07
|
|
398
|
+
|
|
399
|
+
### Changed
|
|
400
|
+
|
|
401
|
+
- Tests only, no asset wording change. The reviewer prompt's output-contract
|
|
402
|
+
yaml-block negative pin now slices from the output-contract heading (and
|
|
403
|
+
requires exactly one yaml fence after it) instead of matching the first
|
|
404
|
+
yaml fence in the file, so an unrelated fence earlier in the prompt cannot
|
|
405
|
+
be mistaken for the output contract. The CHANGELOG's own prose copy of the
|
|
406
|
+
fix-round mutation-probe replay rule is now pinned, anchored on the
|
|
407
|
+
bullet's own opening text rather than the release heading above it, so
|
|
408
|
+
the pin survives the bullet moving under a version heading. A new guard
|
|
409
|
+
asserts every reviewer checklist item mirrored in SKILL.md (Placement,
|
|
410
|
+
the GitHub Actions run-step replay, identifier drift) still carries its
|
|
411
|
+
SKILL.md counterpart sentence, from one table of paired phrases, so
|
|
412
|
+
either half going missing fails the same test.
|
|
413
|
+
|
|
414
|
+
- The implementer prompt now requires running every long test, build, or
|
|
415
|
+
mutation-probe command in the foreground and waiting for it to finish
|
|
416
|
+
before returning, never backgrounding it and ending the turn: a batch
|
|
417
|
+
review found a probe run left running in the background whose result
|
|
418
|
+
was only recovered by resuming the subagent later, which is not
|
|
419
|
+
evidence the orchestrator can trust unattended. The reviewer and
|
|
420
|
+
implementer prompts now both caution against a spawned-CLI test
|
|
421
|
+
calibrated to a byte-count ceiling that sits inside the tool's own
|
|
422
|
+
run-to-run noise (timing digits, temp-directory names): a batch review
|
|
423
|
+
found exactly such a test pass locally and fail on the next run one
|
|
424
|
+
byte off, with no code change, and the fix is to pin the argument
|
|
425
|
+
under test in-process or assert the actual contract (a bound, or the
|
|
426
|
+
presence of a warning), never a byte ceiling. SKILL.md's Delegate
|
|
427
|
+
review step now prohibits running mutation probes in place against a
|
|
428
|
+
worktree a reviewer subagent is concurrently reviewing, after a batch
|
|
429
|
+
review found a reviewer verifying findings against a tree that was
|
|
430
|
+
mutating under it from a concurrent probe; the fix is worktree
|
|
431
|
+
isolation or waiting until the reviewer has returned. Consumers refresh
|
|
432
|
+
their installs at the next release to pick up all three prompt changes.
|
|
433
|
+
Motivated by agent-tasks task 0e17fb63; see `docs/okf/log.md` for the
|
|
434
|
+
incident detail behind each of the three rules.
|
|
435
|
+
|
|
10
436
|
## [0.30.0] - 2026-09-06
|
|
11
437
|
|
|
12
438
|
### Added
|
|
@@ -53,6 +53,17 @@ Rules:
|
|
|
53
53
|
criteria name and report its summary under `tests.executed`; when a
|
|
54
54
|
mutation-probe runner is available, run the named probes through it and
|
|
55
55
|
copy its fields into `mutation_probes`.
|
|
56
|
+
- Run every long test, build, or mutation-probe command in the foreground
|
|
57
|
+
and wait for it to finish before returning. When one foreground call
|
|
58
|
+
cannot hold it to completion, poll the backgrounded run to completion
|
|
59
|
+
and report its result before ending your turn; never end your turn with
|
|
60
|
+
the run still outstanding, since a run that outlives your turn is not
|
|
61
|
+
evidence you can report.
|
|
62
|
+
- A test that spawns a CLI and asserts its output against a byte-count
|
|
63
|
+
ceiling calibrated to sit inside the output's own run-to-run noise
|
|
64
|
+
(timing digits, temporary-directory names) is not a regression test; pin
|
|
65
|
+
the argument under test in-process, or assert the actual contract (a
|
|
66
|
+
bound, or the presence of a warning), never a byte ceiling.
|
|
56
67
|
- For any diff that adds or changes a GitHub Actions `run:` step, replay it
|
|
57
68
|
locally under the shell the step actually runs: `bash --noprofile --norc
|
|
58
69
|
-eo pipefail` when `shell: bash` is set on the step or via
|
|
@@ -22,6 +22,25 @@ a version. For a recorded original string-list contract, retain the original
|
|
|
22
22
|
and `criterion_evidence` fields; keep all existing role output fields. This
|
|
23
23
|
selection governs the rules and every YAML block below.
|
|
24
24
|
|
|
25
|
+
Review method: the orchestrator names `review_method: normal | rigorous |
|
|
26
|
+
adversarial` in every briefing; treat an unnamed method as `rigorous`. The
|
|
27
|
+
three methods are obligation sets, not personas: they define what you must
|
|
28
|
+
read, reproduce, and probe, and how a non-reproducing finding is withdrawn,
|
|
29
|
+
not how skeptical to sound.
|
|
30
|
+
|
|
31
|
+
| Method | Obligations |
|
|
32
|
+
|---|---|
|
|
33
|
+
| `normal` | Read the diff and the spec; run the declared tests once; findings come only from what you read. `normal` adds nothing beyond the obligations already stated in the Check list and the Rules below, and suspends none of them: the empirical-reproduction rule and the GitHub Actions shell replay rule apply under every method. `normal` only means no further independent reproduction beyond what those already require. Fits docs, renames, and batch cosmetics. |
|
|
34
|
+
| `rigorous` (default) | Everything `normal` requires, plus: your own extract of the change, a base-attribution control, classifying every change, and reproducing every empirical claim yourself. `reproduction` and `matches_implementer_claim` are mandatory, as already required below. |
|
|
35
|
+
| `adversarial` | Everything `rigorous` requires, plus: one discriminating probe or negative control per acceptance criterion; an active search of the neighbouring scenario space (environment, install modes, platform, ordering, concurrency); an attempt to break the claimed invariant; and an explicit list of break attempts that failed. |
|
|
36
|
+
|
|
37
|
+
Withdrawal rule (`rigorous` and `adversarial`): a finding that does not
|
|
38
|
+
reproduce on a second attempt with a corrected harness is withdrawn in the
|
|
39
|
+
same round, not carried into the next one, and reported under `withdrawn`
|
|
40
|
+
with the reason; this keeps the method from buying false positives. Emit
|
|
41
|
+
`withdrawn: []` when nothing was withdrawn. Report the method you actually
|
|
42
|
+
applied in `method_applied`.
|
|
43
|
+
|
|
25
44
|
Check, at minimum:
|
|
26
45
|
|
|
27
46
|
- Acceptance baseline: for a run explicitly adopted as `acceptance-baseline/v1`,
|
|
@@ -38,7 +57,12 @@ Check, at minimum:
|
|
|
38
57
|
- Edge cases: empty inputs, error paths, concurrency, encoding, limits.
|
|
39
58
|
- Security: injection, path traversal, secrets, permissions, unsafe defaults.
|
|
40
59
|
- Test adequacy: are the new or changed behaviors covered, and would the new
|
|
41
|
-
tests actually fail if the change were reverted? Flag inert tests.
|
|
60
|
+
tests actually fail if the change were reverted? Flag inert tests. A test
|
|
61
|
+
that spawns a CLI and asserts its output against a byte-count ceiling
|
|
62
|
+
calibrated to sit inside the output's own run-to-run noise (timing digits,
|
|
63
|
+
temporary-directory names) is not a regression test; the fix is to pin
|
|
64
|
+
the argument under test in-process, or assert the actual contract (a
|
|
65
|
+
bound, or the presence of a warning), never a byte ceiling.
|
|
42
66
|
- Maintainability: naming, dead code, needless abstraction, doc drift.
|
|
43
67
|
- Placement: does the change add org-, machine-, or point-in-time-bound
|
|
44
68
|
evidence (dates, sample sizes, task ids, home paths, incident tallies) to a
|
|
@@ -140,4 +164,8 @@ reproduction:
|
|
|
140
164
|
sample_size: ""
|
|
141
165
|
result: ""
|
|
142
166
|
matches_implementer_claim: matched | mismatched | not_applicable
|
|
167
|
+
method_applied: normal | rigorous | adversarial
|
|
168
|
+
withdrawn:
|
|
169
|
+
- description: ""
|
|
170
|
+
reason: ""
|
|
143
171
|
```
|
|
@@ -49,6 +49,13 @@ default, not a ritual.
|
|
|
49
49
|
orchestrator may review it itself; reserve the reviewer subagent for
|
|
50
50
|
changes whose risk or size warrants an independent skeptical pass. Either
|
|
51
51
|
way, review is never skipped.
|
|
52
|
+
- Every reviewer briefing also names a `review_method`: `normal | rigorous |
|
|
53
|
+
adversarial`, an obligation set orthogonal to the effort tier below.
|
|
54
|
+
`adversarial` is the minimum for security judgment, install/deploy
|
|
55
|
+
scripts, hand-edited lockfiles, cross-major overrides, or anything the
|
|
56
|
+
operator flags high-risk; `normal` fits only docs, renames, or batch
|
|
57
|
+
cosmetics; `rigorous` is the default otherwise. Never pair `adversarial`
|
|
58
|
+
with the `-medium` reviewer tier; tiers themselves are unchanged.
|
|
52
59
|
- When tier variants are installed (manifest `tiers: true`), the orchestrator
|
|
53
60
|
picks the effort tier per task by complexity and risk, at its own judgment.
|
|
54
61
|
The unsuffixed default subagent is the normal case; `-high`/`-xhigh` fit
|
package/assets/skill/SKILL.md
CHANGED
|
@@ -251,7 +251,16 @@ directory and the subagents.
|
|
|
251
251
|
`reviewer-<tier>` subagents, if any) by the task's complexity and risk, at
|
|
252
252
|
your own judgment, defaulting to the unsuffixed subagent when unsure; record
|
|
253
253
|
a non-default tier choice with a one-line reason in `03-decisions.md` when
|
|
254
|
-
the task is non-trivial.
|
|
254
|
+
the task is non-trivial. Also name `review_method: normal | rigorous |
|
|
255
|
+
adversarial` in the briefing; every briefing names one. Pick it by risk
|
|
256
|
+
class: `adversarial` at minimum for security judgment, install/deploy
|
|
257
|
+
scripts, hand-edited lockfiles, cross-major overrides, or anything the
|
|
258
|
+
operator flags high-risk; `normal` only for docs, renames, or batch
|
|
259
|
+
cosmetics; `rigorous` otherwise. The method is orthogonal to the tier and
|
|
260
|
+
never substitutes for it: do not pair `adversarial` with the `-medium`
|
|
261
|
+
reviewer tier, a budget mismatch that names probes without the effort to
|
|
262
|
+
run them; tiers themselves are unchanged by this axis. When the
|
|
263
|
+
reviewer's environment cannot use version
|
|
255
264
|
control to see the diff (for example a policy-gated repository), supply the
|
|
256
265
|
diff as a pre-generated file in the briefing instead of expecting the
|
|
257
266
|
reviewer to derive it, and have the reviewer report explicitly if it could
|
|
@@ -273,8 +282,8 @@ directory and the subagents.
|
|
|
273
282
|
implementer's log — and record the method, sample size, and result against
|
|
274
283
|
the implementer's claim in the reviewer output contract's `reproduction`
|
|
275
284
|
field. This does not apply to deterministic checks (a single test run,
|
|
276
|
-
`tsc`, lint): only claims that could vary run to run trigger it. The
|
|
277
|
-
|
|
285
|
+
`tsc`, lint): only claims that could vary run to run trigger it. The GitHub
|
|
286
|
+
Actions shell replay named in step 6 is a second, explicitly
|
|
278
287
|
non-probabilistic trigger for the same field, with `sample_size:
|
|
279
288
|
not_applicable` allowed when the replay itself has no meaningful sample
|
|
280
289
|
size. A change that deletes or renames an exported identifier, type, config
|
|
@@ -282,18 +291,21 @@ directory and the subagents.
|
|
|
282
291
|
describing the old name as current), by the reviewer or by the orchestrator
|
|
283
292
|
itself when it reviews a trivial rename per Scaling delegation, using a
|
|
284
293
|
connected drift check when one exists. When this is not the task's first
|
|
285
|
-
review round, name the round
|
|
286
|
-
|
|
287
|
-
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
|
|
291
|
-
|
|
292
|
-
|
|
293
|
-
|
|
294
|
-
|
|
295
|
-
|
|
296
|
-
|
|
294
|
+
review round, name the round number in the briefing; the reviewer marks each
|
|
295
|
+
finding's `recurrence` as `new` or `repeated` against the earlier rounds it
|
|
296
|
+
was told about, which is what lets the orchestrator detect the Review-round
|
|
297
|
+
escalation budget's trigger (see below) without re-deriving it by hand. When
|
|
298
|
+
the implementer's report replays a prior round's mutation probe, the
|
|
299
|
+
orchestrator's reviewer briefing names the replayed probes the implementer
|
|
300
|
+
reports as killed together with their `mutant` and `verified_applied_via`
|
|
301
|
+
values; the reviewer may then skip re-running those. The reviewer output
|
|
302
|
+
contract itself is unchanged. Never run mutation probes in place against a
|
|
303
|
+
worktree a reviewer subagent is concurrently reviewing; isolate the probe in
|
|
304
|
+
a separate worktree or wait until the reviewer has returned before probing
|
|
305
|
+
that tree again. For an explicitly adopted v1 run, ask the reviewer to
|
|
306
|
+
compare the frozen delegated criteria with the referenced evidence and judge
|
|
307
|
+
semantic adequacy, including whether a manual check is actually concrete and
|
|
308
|
+
reasoned.
|
|
297
309
|
8. **Decide acceptance.** Accept, request fixes, defer, or escalate to the
|
|
298
310
|
operator. High or critical findings block acceptance until fixed or
|
|
299
311
|
explicitly waived: critical findings require operator sign-off; high
|
|
@@ -517,6 +529,10 @@ reproduction:
|
|
|
517
529
|
sample_size: ""
|
|
518
530
|
result: ""
|
|
519
531
|
matches_implementer_claim: matched | mismatched | not_applicable
|
|
532
|
+
method_applied: normal | rigorous | adversarial
|
|
533
|
+
withdrawn:
|
|
534
|
+
- description: ""
|
|
535
|
+
reason: ""
|
|
520
536
|
```
|
|
521
537
|
|
|
522
538
|
`acceptance_recommendation` is mandatory: every reviewer return must set it.
|
|
@@ -529,6 +545,15 @@ one that already appeared in an earlier round. On a task's first review
|
|
|
529
545
|
round every finding is `new` by definition. This is what feeds the
|
|
530
546
|
Review-round escalation budget's trigger.
|
|
531
547
|
|
|
548
|
+
`method_applied` echoes the `review_method` named in the briefing (see step
|
|
549
|
+
7); `withdrawn` lists each finding the reviewer proposed and then retracted
|
|
550
|
+
under the withdrawal rule (`rigorous` and `adversarial` only), with its
|
|
551
|
+
reason; emit `withdrawn: []` when nothing was withdrawn. Until a
|
|
552
|
+
grounding-mcp reader parses the marker (tracked as a cross-repo
|
|
553
|
+
follow-up), the orchestrator checks by hand that the return's
|
|
554
|
+
`method_applied` matches the briefing's `review_method`; a mismatch or
|
|
555
|
+
omission is resupplied, not accepted.
|
|
556
|
+
|
|
532
557
|
## Task slicer output contract
|
|
533
558
|
|
|
534
559
|
Use this v1 block subject to Contract selection above for every task.
|
|
@@ -4,6 +4,11 @@
|
|
|
4
4
|
|
|
5
5
|
<!-- Short summary. -->
|
|
6
6
|
|
|
7
|
+
<!-- review-method[<round>] = normal|rigorous|adversarial -->
|
|
8
|
+
Method: normal | rigorous | adversarial (the `review_method` named in this
|
|
9
|
+
round's briefing and the `method_applied` the reviewer returned; not parsed
|
|
10
|
+
by the grounding-mcp completeness reader yet).
|
|
11
|
+
|
|
7
12
|
## Findings
|
|
8
13
|
|
|
9
14
|
<!-- The Severity and Decision column headers below are load-bearing: the orchestrator-workflow completeness reader locates this table by its header row and verifies unresolved findings from those two columns. Do not rename or drop them. -->
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "orchestrator-workflow",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.32.0",
|
|
4
4
|
"description": "Installer for an orchestrator-led agent workflow: .ai/ run state, an AGENTS.md policy section, and per-harness subagent definitions for Claude Code, OpenAI Codex, and opencode",
|
|
5
5
|
"main": "dist/index.js",
|
|
6
6
|
"type": "module",
|