orchestrator-workflow 0.23.0 → 0.25.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +263 -0
- package/README.md +3 -4
- package/assets/agents/reviewer.md +11 -0
- package/assets/agents-md-section.md +26 -19
- package/assets/skill/SKILL.md +71 -25
- package/assets/templates/03-decisions.md +13 -0
- package/assets/templates/05-review-findings.md +3 -0
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,269 @@ All notable changes to `orchestrator-workflow` are documented here.
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
6
6
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
7
|
|
|
8
|
+
## [Unreleased]
|
|
9
|
+
|
|
10
|
+
## [0.25.0] - 2026-08-27
|
|
11
|
+
|
|
12
|
+
### Added
|
|
13
|
+
|
|
14
|
+
- A "Review-round escalation budget" (`SKILL.md`, new section right after
|
|
15
|
+
the existing Round-2 halt rule): by the second round-2 halt signal on a
|
|
16
|
+
task, or its third `fix_required` review round, whichever comes first,
|
|
17
|
+
the orchestrator now picks one of three escalations, tier/model
|
|
18
|
+
escalation, an advisor spawn, or an operator merge-hold, instead of
|
|
19
|
+
running another round unaided, and records the choice in
|
|
20
|
+
`03-decisions.md`'s new Review-round escalation section (the
|
|
21
|
+
`review-round-escalation` marker: `n/a | tier_escalation | advisor |
|
|
22
|
+
merge_hold`, `n/a` a deliberately fail-open default since most runs
|
|
23
|
+
never trigger the budget at all, unlike the `TODO` fail-closed sentinel
|
|
24
|
+
the `solution-acceptance:` marker family uses). Which of the three is
|
|
25
|
+
picked stays a judgment call; only that one is picked and recorded is
|
|
26
|
+
now mandatory, and escalating never substitutes for a review round.
|
|
27
|
+
`agents-md-section.md` carries the same rule in short form. The
|
|
28
|
+
reviewer output contract (`SKILL.md` and `reviewer.md`) gained a
|
|
29
|
+
`recurrence: new | repeated` field per finding, and step 7 now has the
|
|
30
|
+
orchestrator name the review round number in the briefing when it is
|
|
31
|
+
not the task's first, so the reviewer can classify each finding against
|
|
32
|
+
the rounds it was told about instead of the orchestrator re-deriving
|
|
33
|
+
recurrence by hand. `03-decisions.md`'s new marker is a documented
|
|
34
|
+
convention only: nothing in this package's own code reads it today
|
|
35
|
+
(unlike the `solution-acceptance:` markers, which grounding-mcp's
|
|
36
|
+
run-completeness reader does parse); wiring it into that reader, if
|
|
37
|
+
wanted, is a follow-up in the consuming project, not part of this
|
|
38
|
+
change.
|
|
39
|
+
|
|
40
|
+
Evidence: agent-tasks d03af8f6 (pandora run
|
|
41
|
+
`.ai/runs/2026-08-26-open-pool-batch30`, a harness risk-gate deletion
|
|
42
|
+
arm) ran five implementer rounds on the same effort tier (rounds 1-5, the
|
|
43
|
+
default-tier implementer, Sonnet) before this rule existed; each reviewer
|
|
44
|
+
round found one new HIGH on the permissive side of the same detection layer
|
|
45
|
+
(first-segment matching, wrapper flags, xargs flags, a find-root premise
|
|
46
|
+
twice, xargs `-i`/append), and the Round-2 halt rule's split-or-redesign
|
|
47
|
+
response was applied three times (D-014, D-019, D-022/D-023) without ever
|
|
48
|
+
escalating tier, model, or to the advisor, and without a merge-hold. Only
|
|
49
|
+
after round 5 did the operator suggest a stronger model (D-024); round 6, on
|
|
50
|
+
Fable with the `-xhigh` variant, found and closed twelve further fail-open
|
|
51
|
+
classes in one round by its own 248-probe audit, and the following review
|
|
52
|
+
found only one remaining, a documentation-only medium. A same-run comparison
|
|
53
|
+
task, agent-tasks da823721, needed three rounds with one round-2 halt and
|
|
54
|
+
was accepted after it, well inside this budget. This is the first data
|
|
55
|
+
point relating implementer model strength to round count on a
|
|
56
|
+
permissive-security-parser class of task; the 2026-08-24 tier A/B (see
|
|
57
|
+
`[0.23.0]` below) measured only
|
|
58
|
+
`implementer-low` against the default tier on a different task shape,
|
|
59
|
+
not model strength, so whether a stronger model generally shortens
|
|
60
|
+
these rounds remains open.
|
|
61
|
+
- A new `okf-anchor-guard` job in `.github/workflows/ci.yml` (this repo)
|
|
62
|
+
runs okf-kit's anchor check against `docs/okf` and fails the build (no
|
|
63
|
+
branch-protection change; master has no required checks today) when an
|
|
64
|
+
edit to `SKILL.md`, an agent template under `assets/agents/`,
|
|
65
|
+
`src/models.ts`, a `test/*.test.ts` file, any other `src/*.ts` module,
|
|
66
|
+
a run template under `assets/templates/`, or `assets/agents-md-section.md`
|
|
67
|
+
shifts a cited range out of the section or text an anchored bundle
|
|
68
|
+
citation names, instead of only the existing warn-only `okf-staleness.yml`
|
|
69
|
+
drift watch (which never blocks by design and stays untouched). Every
|
|
70
|
+
in-scope bundle citation into those source categories now carries a
|
|
71
|
+
string-form anchor (up from a first-round draft that review round 2
|
|
72
|
+
found had missed 44 bare, non-backtick-delimited citations in two of the
|
|
73
|
+
five docs/okf siblings; the citation parser was corrected to match
|
|
74
|
+
okf-kit's own backtick-optional shape; scope later widened from the
|
|
75
|
+
original four categories to every `src/*.ts` module and every
|
|
76
|
+
`assets/templates/*.md` plus `assets/agents-md-section.md`, agent-tasks
|
|
77
|
+
ca9d5048); the `CHANGELOG.md` citations still carry heading anchors from
|
|
78
|
+
the prior round (that mechanism is untouched, but every CHANGELOG entry
|
|
79
|
+
added above them, this one included, re-points all of them -- see
|
|
80
|
+
`docs/okf/log.md` for the live count and the re-point history, not
|
|
81
|
+
hand-copied here since it drifts with every CHANGELOG edit).
|
|
82
|
+
- Every anchor now satisfies two mechanically-checked properties review
|
|
83
|
+
round 2 added (a first-round anchor sitting on a wide range's first line,
|
|
84
|
+
as 107 of 121 did, survives an insertion shorter than the range itself --
|
|
85
|
+
measured: round 1 had 46 SKILL.md-targeting anchors, and a 1-line
|
|
86
|
+
insertion near the top of SKILL.md left 24 of them silently green): (a)
|
|
87
|
+
the anchor text occurs on the LAST line of its cited range (ranges were
|
|
88
|
+
narrowed where needed to end on real content rather than a
|
|
89
|
+
blank/closing-brace line), and (b) it occurs at most 3 times in the
|
|
90
|
+
whole target file (23 first-round anchors used a too-common token, e.g.
|
|
91
|
+
`describe(`, and were
|
|
92
|
+
replaced). Two anchors were additionally re-pointed because the text
|
|
93
|
+
they carried did not match the claim their citing sentence made, not
|
|
94
|
+
just its mechanical position; see `docs/okf/log.md` for both. Pinned by
|
|
95
|
+
three new tests in `test/docs-consistency.test.ts` (version-pin
|
|
96
|
+
coverage extended to every `.github/workflows/*.yml` file, not just
|
|
97
|
+
`okf-staleness.yml`; the last-line/occurrence-cap rule, verified red
|
|
98
|
+
against the first-round anchors and green against this round's; and an
|
|
99
|
+
erosion brake asserting zero unanchored in-scope citations going
|
|
100
|
+
forward). `ci.yml`'s anchor-finding jq filter now matches any
|
|
101
|
+
`anchor-*`-tagged finding by pattern instead of four hardcoded rule
|
|
102
|
+
ids, guarded by a new self-test step that builds a throwaway fixture
|
|
103
|
+
bundle with one deliberately drifted anchored citation and requires the
|
|
104
|
+
filter to catch it before the real check runs.
|
|
105
|
+
`okf-kit check` reports the same 0 errors / 13 warnings / 22 notices
|
|
106
|
+
before and after this round's full anchor rewrite (0 anchor findings
|
|
107
|
+
either way; the 13 pre-existing warnings are unrelated
|
|
108
|
+
`install-fence-mechanics.md` short-form findings against `init.test.ts`
|
|
109
|
+
and `init.ts`). Differential mutation probe, replacing the first
|
|
110
|
+
round's single 27-finding number: inserting `k` dummy lines near the
|
|
111
|
+
top of SKILL.md, 52/52 (100%) of its unique cited ranges now produce a
|
|
112
|
+
finding at `k=1`; 51/52 (98%) at `k=2`, one named exception (a
|
|
113
|
+
same-block token collision, see `docs/okf/log.md`). A `package.json`
|
|
114
|
+
patch-version bump still leaves the anchor-finding count at 0,
|
|
115
|
+
confirming no false positive. Residual gaps named in `docs/okf/log.md`:
|
|
116
|
+
a content change inside a cited range that neither shifts its line
|
|
117
|
+
count nor disturbs the anchor text stays invisible to this check
|
|
118
|
+
(mechanical, never semantic, the same limit okf-kit's own README
|
|
119
|
+
documents), plus the one named same-block token collision above
|
|
120
|
+
(agent-tasks task 578f5bfd, review round 2; following the
|
|
121
|
+
anchored-citations feature itself, task 5c8013c0, and its release, task
|
|
122
|
+
c0effc67).
|
|
123
|
+
|
|
124
|
+
### Changed
|
|
125
|
+
|
|
126
|
+
- `docs/okf/subagent-contracts-superset.md`: rewrote its 22 sibling short-
|
|
127
|
+
form citations (21 initially, plus one missed and fixed this round) from
|
|
128
|
+
the parenthesized form (`(N-M)`) to the colon form (`, :N-M`) so
|
|
129
|
+
`citations-resolve` checks them again. Neither form was ever machine-
|
|
130
|
+
checked in a released `okf-kit`: the colon-form gate and the drop of
|
|
131
|
+
paren-form collection landed in the same `[Unreleased]` entry, so
|
|
132
|
+
"dropped" overstates it. Inserting this entry shifts every later line
|
|
133
|
+
number in this file, so any absolute-line `CHANGELOG.md` citation below
|
|
134
|
+
it breaks until re-pointed; this round re-points all sixteen such
|
|
135
|
+
citations across the three touched docs/okf siblings
|
|
136
|
+
(`subagent-contracts-superset.md`, `review-gate-and-waivers.md`,
|
|
137
|
+
`run-state-lifecycle-and-markers.md`), each checked against the
|
|
138
|
+
`## [x.y.z]` section its own sentence names, not against a byte-diff of a
|
|
139
|
+
moving base. `okf-kit check` against a repo build (not the published
|
|
140
|
+
package) reports 0 errors / 13 warnings / 22 notices; all 13 are
|
|
141
|
+
pre-existing `install-fence-mechanics.md` short-form findings unrelated
|
|
142
|
+
to this change, so the three touched docs carry no citations-resolve or
|
|
143
|
+
sources-fresh finding of their own (log.md records the fuller
|
|
144
|
+
sources-fresh investigation, since that count depends on this round's
|
|
145
|
+
commit shape, not on the citations themselves). Coverage holds only
|
|
146
|
+
against a repo build: CI's `okf-staleness.yml` still pins the published
|
|
147
|
+
`okf-kit@0.5.0`, which predates short-form colon resolution, so it
|
|
148
|
+
reports 0 short-form findings until that release ships (agent-tasks task
|
|
149
|
+
2e3e5f4b).
|
|
150
|
+
- All sixteen `CHANGELOG.md` citations across the three `docs/okf` siblings
|
|
151
|
+
above now also carry a heading anchor (`` `CHANGELOG.md:N-M#x.y.z` ``, a
|
|
152
|
+
new `citations-resolve` form -- see okf-kit's own CHANGELOG for the
|
|
153
|
+
design), pinning each one to the release section its own sentence names
|
|
154
|
+
instead of only to a line range that a future top-of-file insertion can
|
|
155
|
+
silently shift into the wrong section. `okf-kit check` against a repo
|
|
156
|
+
build still reports the same 0 errors / 13 warnings / 22 notices as
|
|
157
|
+
above (all pre-existing, unrelated to this change); a mutation probe that
|
|
158
|
+
moved one migrated citation's range into its neighbouring release
|
|
159
|
+
section, and a second probe that inserted a dummy entry at the top of
|
|
160
|
+
this file (shifting every citation below it), were both caught by the
|
|
161
|
+
new anchor check and reverted (agent-tasks task 5c8013c0).
|
|
162
|
+
|
|
163
|
+
### Corrections
|
|
164
|
+
|
|
165
|
+
- Correction to the "Known limit of the pack" note in the 0.24.0 entry
|
|
166
|
+
below: this is not an `orchestrator-workflow` behavior change, it
|
|
167
|
+
documents a `slop-detector` fix. The 0.24.0 note described a known limit
|
|
168
|
+
of `placement-slop`, that an `allow` match suppressed every placement
|
|
169
|
+
rule on the line it matched, not just the marker span it was meant to
|
|
170
|
+
excuse. `slop-detector` has since fixed this (`placement.allow` is now
|
|
171
|
+
span-scoped, not line-wide). Consumer-visible effect: a config that
|
|
172
|
+
reported clean before can now report a `block`-severity finding (e.g.
|
|
173
|
+
`placement-slop/home-path`) when a home path, a date, or a tally phrase
|
|
174
|
+
shares a line with an allowed marker.
|
|
175
|
+
|
|
176
|
+
## [0.24.0] - 2026-08-24
|
|
177
|
+
|
|
178
|
+
### Changed
|
|
179
|
+
|
|
180
|
+
- **Moved org-, machine-, and point-in-time-bound evidence out of the kit's
|
|
181
|
+
reusable instruction files; rule text is unchanged.** Public,
|
|
182
|
+
tool-agnostic kit files (`SKILL.md`, `agents-md-section.md`) now carry
|
|
183
|
+
rules and procedures only; the measurements, dates, sample sizes, task
|
|
184
|
+
ids, and incident tallies that used to sit inline are recorded here in
|
|
185
|
+
the changelog instead, with a one-line pointer left in prose.
|
|
186
|
+
- `SKILL.md` step 6 ("Delegate implementation"): the parenthetical
|
|
187
|
+
`(2026-08-24 A/B measurement, n=8: implementer-low reached accept a
|
|
188
|
+
median 320 seconds slower, p=0.016, with 9 high-plus-critical review
|
|
189
|
+
findings against 1 and 8 fix rounds against 1)` is now `(anchored by an
|
|
190
|
+
A/B measurement; see CHANGELOG 0.23.0)`. The full numbers already live
|
|
191
|
+
in the 0.23.0 entry below.
|
|
192
|
+
- `agents-md-section.md`'s Scaling delegation bullet: the paragraph
|
|
193
|
+
naming the A/B's `n=8`, the median slowdown, `p=0.016`, the
|
|
194
|
+
high-plus-critical finding count, the fix-round count, the Haiku 4.5
|
|
195
|
+
model detail, and agent-tasks task `7f38899d` is replaced by "This rule
|
|
196
|
+
is anchored by an A/B measurement; the data and the model caveat are
|
|
197
|
+
recorded in the orchestrator-workflow CHANGELOG (0.23.0)." Same data,
|
|
198
|
+
same 0.23.0 entry.
|
|
199
|
+
- `SKILL.md`'s "Subagent misfire rule": the incident tally `(four so
|
|
200
|
+
far)` and the whole reviewer/model-correlation passage ("So far this
|
|
201
|
+
signal has only been observed for the reviewer role ...", including the
|
|
202
|
+
0.21.0 advisor remark and the pointer to the per-role model
|
|
203
|
+
preferences) are removed. The rule itself (the signal definition, the
|
|
204
|
+
contract-parse and near-instant detection signals, the
|
|
205
|
+
resume-over-respawn preference and its reasoning, the respawn fallback,
|
|
206
|
+
the watchdog-stall exception, the `03-decisions.md` record requirement,
|
|
207
|
+
and "a misfired review is not a review") is unchanged. The removed
|
|
208
|
+
passage, verbatim, for the durable record: "Every incident of this
|
|
209
|
+
exact signal (a return within seconds, zero tool calls, harness or
|
|
210
|
+
system boilerplate instead of the output contract) whose outcome was
|
|
211
|
+
recorded (four so far) has resolved on the first resume attempt; fall
|
|
212
|
+
back to a fresh respawn only if the resume attempt itself misfires the
|
|
213
|
+
same way. So far this signal has only been observed for the reviewer
|
|
214
|
+
role, a role whose default model differs from explorer's,
|
|
215
|
+
task-slicer's, and implementer's (since 0.21.0 the advisor shares the
|
|
216
|
+
reviewer's default model too; the advisor has had no spawns yet, so it
|
|
217
|
+
contributes no evidence either way; see the per-role model
|
|
218
|
+
preferences); treat that correlation as an open lead worth watching as
|
|
219
|
+
more incidents accumulate, not as a confirmed cause." Of the four
|
|
220
|
+
resume outcomes recorded for that signal, three were on 2026-07-16 and
|
|
221
|
+
one was on 2026-07-20. The watchdog-stall exception's removed sentence,
|
|
222
|
+
also verbatim: "This resume-over-respawn preference does not extend to
|
|
223
|
+
a structurally different misfire class: a mid-run watchdog stall (the
|
|
224
|
+
subagent goes idle partway through a run rather than returning
|
|
225
|
+
near-instantly) did not resolve on resume in the one measured incident
|
|
226
|
+
of that class, it stalled a second time, and only a fresh, explicitly
|
|
227
|
+
constrained respawn produced a contract-valid review; treat a watchdog
|
|
228
|
+
stall as outside this preference."
|
|
229
|
+
- `SKILL.md`'s Run state paragraph: "see the grounding-mcp 0.6.0 docs for
|
|
230
|
+
the full consumer semantics" is now "see the consuming gate's
|
|
231
|
+
documentation (grounding-mcp) for the full consumer semantics", dropping
|
|
232
|
+
the pinned version number.
|
|
233
|
+
|
|
234
|
+
### Added
|
|
235
|
+
|
|
236
|
+
- **Reviewer placement check.** `reviewer.md`'s "Check, at minimum" list
|
|
237
|
+
gains a check for org-, machine-, or point-in-time-bound evidence (dates,
|
|
238
|
+
sample sizes, task ids, home paths, incident tallies) leaking into a
|
|
239
|
+
reusable instruction file (a skill, an agent prompt, an AGENTS.md
|
|
240
|
+
section, a template); the fix it recommends is moving the evidence to the
|
|
241
|
+
changelog, the run files, or the consuming workspace and leaving a
|
|
242
|
+
one-line pointer.
|
|
243
|
+
- **Hand-off placement check.** `SKILL.md` step 9 ("Hand off") gains one
|
|
244
|
+
sentence for the orchestrator: check before handing off that no such
|
|
245
|
+
evidence was added to a reusable instruction file.
|
|
246
|
+
- **`placement-guard` CI job.** A dedicated job in agent-dx's
|
|
247
|
+
`.github/workflows/ci.yml`, separate from the package matrix, builds
|
|
248
|
+
`slop-detector` and runs its opt-in `placement-slop` pack against the
|
|
249
|
+
monorepo's instruction files (`packages/orchestrator-workflow/assets/**`
|
|
250
|
+
and `packages/agentic-coding-playbook/**`, configured via the repo-root
|
|
251
|
+
`slop.config.yml`), failing on block-level violations.
|
|
252
|
+
- **Root `slop.config.yml`.** Opts the `placement-slop` pack in for the CI
|
|
253
|
+
job, configures the `LanNguyenSi` org marker with `allow` entries for its
|
|
254
|
+
two legitimate repo links (`github.com/LanNguyenSi/`,
|
|
255
|
+
`raw.githubusercontent.com/LanNguyenSi/`, both anchored to a full
|
|
256
|
+
`https://` URL), widens the pack's instruction-file globs to cover this
|
|
257
|
+
package's `assets/` tree and the `agentic-coding-playbook` package, and
|
|
258
|
+
overrides `placement-slop/dated-evidence`, `placement-slop/tally-phrase`,
|
|
259
|
+
and `placement-slop/opaque-id` from their pack default of `warn` to
|
|
260
|
+
`block`, so the CI job actually fails on a leaked date, tally phrase, or
|
|
261
|
+
opaque id instead of only warning (a home path or an unlisted org marker
|
|
262
|
+
was already `block` by pack default). `packages/github-api-tool/SKILL.md`
|
|
263
|
+
carries two pre-existing dated examples that this severity change would
|
|
264
|
+
now block; it is `ignorePaths`-excluded pending a follow-up cleanup of
|
|
265
|
+
that unrelated package.
|
|
266
|
+
Known limit of the pack, unchanged here: an `allow` match suppresses
|
|
267
|
+
every placement rule on that line, so a home path or a date that shares
|
|
268
|
+
a line with an allowed repo URL is not reported; a span-scoped allow is
|
|
269
|
+
a slop-detector follow-up.
|
|
270
|
+
|
|
8
271
|
## [0.23.0] - 2026-08-24
|
|
9
272
|
|
|
10
273
|
### Changed
|
package/README.md
CHANGED
|
@@ -329,10 +329,9 @@ effort actually taking effect.
|
|
|
329
329
|
The pin is also emitted unconditionally regardless of which model the role
|
|
330
330
|
resolves to via `--models`, including a model with no effort support at all
|
|
331
331
|
(e.g. `--models reviewer=haiku` still renders `model: haiku` followed by
|
|
332
|
-
`effort: high`).
|
|
333
|
-
|
|
334
|
-
|
|
335
|
-
run-time error.
|
|
332
|
+
`effort: high`). On Haiku 4.5, which does not support the `effort`
|
|
333
|
+
parameter, the harness ignores the pinned value rather than rejecting it
|
|
334
|
+
(anchored by a measurement, see CHANGELOG 0.23.0).
|
|
336
335
|
|
|
337
336
|
## Ownership and re-runs
|
|
338
337
|
|
|
@@ -21,6 +21,16 @@ Check, at minimum:
|
|
|
21
21
|
- Test adequacy: are the new or changed behaviors covered, and would the new
|
|
22
22
|
tests actually fail if the change were reverted? Flag inert tests.
|
|
23
23
|
- Maintainability: naming, dead code, needless abstraction, doc drift.
|
|
24
|
+
- Placement: does the change add org-, machine-, or point-in-time-bound
|
|
25
|
+
evidence (dates, sample sizes, task ids, home paths, incident tallies) to a
|
|
26
|
+
reusable instruction file (a skill, an agent prompt, an AGENTS.md section, a
|
|
27
|
+
template)? Report it; the fix is to move the evidence to the changelog, the
|
|
28
|
+
run files, or the consuming workspace and leave a one-line pointer.
|
|
29
|
+
- Recurrence: when the briefing tells you this is not the task's first
|
|
30
|
+
review round, classify each finding as `new` or `repeated` against the
|
|
31
|
+
earlier rounds you were told about; on a first round every finding is
|
|
32
|
+
`new` by definition. The orchestrator uses this to detect the
|
|
33
|
+
review-round escalation budget's trigger.
|
|
24
34
|
|
|
25
35
|
Rules:
|
|
26
36
|
|
|
@@ -68,6 +78,7 @@ findings:
|
|
|
68
78
|
category: correctness | architecture | security | tests | maintainability | performance | docs
|
|
69
79
|
description: ""
|
|
70
80
|
suggested_fix: ""
|
|
81
|
+
recurrence: new | repeated
|
|
71
82
|
acceptance_recommendation: accept | accept_with_notes | fix_required | reject
|
|
72
83
|
missing_tests:
|
|
73
84
|
- ""
|
|
@@ -61,25 +61,17 @@ default, not a ritual.
|
|
|
61
61
|
those three excludes `implementer-low`, and the task runs on the unsuffixed
|
|
62
62
|
implementer or higher, even when the change looks mechanical (a bugfix
|
|
63
63
|
included); when it is unclear whether a criterion demands a run, exclude
|
|
64
|
-
`implementer-low`. This rule is anchored by an A/B measurement
|
|
65
|
-
|
|
66
|
-
(
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
mechanical work; no equivalent measurement exists for those two roles, so
|
|
76
|
-
their rule is unchanged. Not every role gets every tier: `-xhigh` exists
|
|
77
|
-
only for the implementer, the reviewer, and the advisor. The reviewer's
|
|
78
|
-
downshift is `-medium` rather than `-low`, since its default already sits at
|
|
79
|
-
high. The advisor has no downshift at all: its default `high` is already its
|
|
80
|
-
only non-`-xhigh` tier. Spawn only variants that are actually installed.
|
|
81
|
-
Tier choice is a conscious decision, not a ritual; when unsure, use the
|
|
82
|
-
default.
|
|
64
|
+
`implementer-low`. This rule is anchored by an A/B measurement; the data and
|
|
65
|
+
the model caveat are recorded in the orchestrator-workflow CHANGELOG
|
|
66
|
+
(0.23.0). For the explorer and the task-slicer, a `-low` variant still suits
|
|
67
|
+
narrowly scoped, mechanical work; no equivalent measurement exists for those
|
|
68
|
+
two roles, so their rule is unchanged. Not every role gets every tier:
|
|
69
|
+
`-xhigh` exists only for the implementer, the reviewer, and the advisor. The
|
|
70
|
+
reviewer's downshift is `-medium` rather than `-low`, since its default
|
|
71
|
+
already sits at high. The advisor has no downshift at all: its default `high`
|
|
72
|
+
is already its only non-`-xhigh` tier. Spawn only variants that are actually
|
|
73
|
+
installed. Tier choice is a conscious decision, not a ritual; when unsure,
|
|
74
|
+
use the default.
|
|
83
75
|
- Every unsuffixed default subagent carries its own pinned default effort
|
|
84
76
|
baked into its own file, not inherited from the orchestrator session:
|
|
85
77
|
medium for the explorer, the task-slicer, and the implementer; high for
|
|
@@ -111,6 +103,21 @@ trivial change.
|
|
|
111
103
|
the Accepted Waivers section of `06-handoff.md`.
|
|
112
104
|
- Medium and low findings are addressed or consciously accepted at the
|
|
113
105
|
orchestrator's judgment.
|
|
106
|
+
- Review-round escalation budget: by the second round-2 halt signal on a
|
|
107
|
+
task, or its third `fix_required` review round, whichever comes first,
|
|
108
|
+
the orchestrator picks one of tier/model escalation (raise the
|
|
109
|
+
implementer to at least `-xhigh` where installed, or to the strongest
|
|
110
|
+
model available, until that is exhausted), an advisor spawn (where the
|
|
111
|
+
advisor is installed, `full` profile only; under a `minimal` profile
|
|
112
|
+
the exhausted tier path falls straight to the merge-hold), or an
|
|
113
|
+
operator merge-hold, and adds a row (task, choice, reason) to
|
|
114
|
+
`03-decisions.md`'s Review-round escalation table, then sets the
|
|
115
|
+
`review-round-escalation` marker to the most recent choice. A counted
|
|
116
|
+
round is a completed reviewer return recommending `fix_required` or
|
|
117
|
+
`reject`; a misfired review is not a round. Which of the three is
|
|
118
|
+
picked is judgment; that one is picked and recorded is not. Escalating
|
|
119
|
+
never substitutes for a review round and comes in addition to the halt
|
|
120
|
+
rule's split-or-redesign response, not instead of it.
|
|
114
121
|
|
|
115
122
|
### Instruction trust boundary
|
|
116
123
|
|
package/assets/skill/SKILL.md
CHANGED
|
@@ -90,8 +90,8 @@ run-completeness readers, not an acceptance verdict, and it fails open:
|
|
|
90
90
|
left as `TODO` it does not block anything, the reader just falls back to a
|
|
91
91
|
tolerant day-granular date heuristic. The recorded base must resolve in the
|
|
92
92
|
repo, be an ancestor of HEAD, and must not lie behind the fork point of the
|
|
93
|
-
change (the merge-base with the remote default branch); see the
|
|
94
|
-
grounding-mcp
|
|
93
|
+
change (the merge-base with the remote default branch); see the consuming
|
|
94
|
+
gate's documentation (grounding-mcp) for the full consumer semantics.
|
|
95
95
|
|
|
96
96
|
## Workflow
|
|
97
97
|
|
|
@@ -147,10 +147,8 @@ directory and the subagents.
|
|
|
147
147
|
task assignment names mutation probes to run; or the task slicer's
|
|
148
148
|
`suggested_tests` came back non-empty. Any one of those three excludes
|
|
149
149
|
`implementer-low`, even for a change that looks mechanical (a bugfix
|
|
150
|
-
included) (
|
|
151
|
-
a
|
|
152
|
-
findings against 1 and 8 fix rounds against 1). When it is unclear
|
|
153
|
-
whether a criterion demands a run, exclude `implementer-low`. When a
|
|
150
|
+
included) (anchored by an A/B measurement; see CHANGELOG 0.23.0). When it is
|
|
151
|
+
unclear whether a criterion demands a run, exclude `implementer-low`. When a
|
|
154
152
|
task's acceptance rests on a test that must fail without the change, name
|
|
155
153
|
the mutation probes to run in the task assignment; the implementer reports
|
|
156
154
|
each one in the output contract's `mutation_probes` field (apply the mutant
|
|
@@ -187,7 +185,12 @@ directory and the subagents.
|
|
|
187
185
|
implementer's log — and record the method, sample size, and result against
|
|
188
186
|
the implementer's claim in the reviewer output contract's `reproduction`
|
|
189
187
|
field. This does not apply to deterministic checks (a single test run,
|
|
190
|
-
`tsc`, lint): only claims that could vary run to run trigger it.
|
|
188
|
+
`tsc`, lint): only claims that could vary run to run trigger it. When
|
|
189
|
+
this is not the task's first review round, name the round number in the
|
|
190
|
+
briefing; the reviewer marks each finding's `recurrence` as `new` or
|
|
191
|
+
`repeated` against the earlier rounds it was told about, which is what
|
|
192
|
+
lets the orchestrator detect the Review-round escalation budget's
|
|
193
|
+
trigger (see below) without re-deriving it by hand.
|
|
191
194
|
8. **Decide acceptance.** Accept, request fixes, defer, or escalate to the
|
|
192
195
|
operator. High or critical findings block acceptance until fixed or
|
|
193
196
|
explicitly waived: critical findings require operator sign-off; high
|
|
@@ -196,7 +199,10 @@ directory and the subagents.
|
|
|
196
199
|
all decisions and waivers in `03-decisions.md` and summarize waivers in
|
|
197
200
|
the Accepted Waivers section of `06-handoff.md`. Watch for the round-2
|
|
198
201
|
halt signal across repeated review-fix cycles (see Round-2 halt rule
|
|
199
|
-
below).
|
|
202
|
+
below). By the second round-2 halt signal or the third `fix_required`
|
|
203
|
+
review round on the same task, apply the Review-round escalation budget
|
|
204
|
+
(see below) instead of running another round unaided. At an advisor
|
|
205
|
+
trigger (architectural uncertainty, conflicting
|
|
200
206
|
requirements, a high-commitment fork among valid options, repeated
|
|
201
207
|
implementation failures, a review deadlock, a high-risk decision), the
|
|
202
208
|
orchestrator may spawn the advisor subagent before deciding; the advisor
|
|
@@ -213,7 +219,10 @@ directory and the subagents.
|
|
|
213
219
|
validator when one is available (for example `okf-kit check`). Repos
|
|
214
220
|
without a bundle are unaffected. Then fill `06-handoff.md` and report to the
|
|
215
221
|
operator: what changed, why, how it was verified, known risks, accepted
|
|
216
|
-
waivers, suggested next step.
|
|
222
|
+
waivers, suggested next step. Before handing off, check that no org-,
|
|
223
|
+
machine-, or point-in-time-bound evidence was added to a reusable
|
|
224
|
+
instruction file; such evidence belongs in the changelog, the run files,
|
|
225
|
+
or the consuming workspace, with a pointer left behind.
|
|
217
226
|
|
|
218
227
|
When finalizing `05-review-findings.md` and `06-handoff.md`, replace the `TODO`
|
|
219
228
|
in each `<!-- solution-acceptance: ... = TODO -->` marker with the chosen enum
|
|
@@ -319,6 +328,7 @@ findings:
|
|
|
319
328
|
category: correctness | architecture | security | tests | maintainability | performance | docs
|
|
320
329
|
description: ""
|
|
321
330
|
suggested_fix: ""
|
|
331
|
+
recurrence: new | repeated
|
|
322
332
|
acceptance_recommendation: accept | accept_with_notes | fix_required | reject
|
|
323
333
|
missing_tests:
|
|
324
334
|
- ""
|
|
@@ -335,6 +345,12 @@ reproduction:
|
|
|
335
345
|
When it is missing, the orchestrator asks the reviewer to resupply it
|
|
336
346
|
instead of inferring one from the findings list.
|
|
337
347
|
|
|
348
|
+
`recurrence` classifies each finding against earlier rounds on the same
|
|
349
|
+
task: `new` for a defect class not previously found here, `repeated` for
|
|
350
|
+
one that already appeared in an earlier round. On a task's first review
|
|
351
|
+
round every finding is `new` by definition. This is what feeds the
|
|
352
|
+
Review-round escalation budget's trigger.
|
|
353
|
+
|
|
338
354
|
## Task slicer output contract
|
|
339
355
|
|
|
340
356
|
```yaml
|
|
@@ -451,22 +467,14 @@ explicitly repeats the original assignment rather than a generic retry,
|
|
|
451
467
|
since resume keeps the subagent's prior turn in context while a fresh spawn
|
|
452
468
|
starts cold and risks the same misfire again. Every incident of this exact
|
|
453
469
|
signal (a return within seconds, zero tool calls, harness or system
|
|
454
|
-
boilerplate instead of the output contract) whose outcome was recorded
|
|
455
|
-
|
|
456
|
-
|
|
457
|
-
|
|
458
|
-
|
|
459
|
-
|
|
460
|
-
|
|
461
|
-
|
|
462
|
-
worth watching as more incidents accumulate, not as a confirmed cause. This
|
|
463
|
-
resume-over-respawn preference does not extend to a structurally different
|
|
464
|
-
misfire class: a mid-run watchdog stall (the subagent goes idle partway
|
|
465
|
-
through a run rather
|
|
466
|
-
than returning near-instantly) did not resolve on resume in the one
|
|
467
|
-
measured incident of that class, it stalled a second time, and only a
|
|
468
|
-
fresh, explicitly constrained respawn produced a contract-valid review;
|
|
469
|
-
treat a watchdog stall as outside this preference. Record every misfire in
|
|
470
|
+
boilerplate instead of the output contract) whose outcome was recorded has
|
|
471
|
+
resolved on the first resume attempt; fall back to a fresh respawn only if
|
|
472
|
+
the resume attempt itself misfires the same way. This resume-over-respawn
|
|
473
|
+
preference does not extend to a structurally different misfire class: a
|
|
474
|
+
mid-run watchdog stall (the subagent goes idle partway through a run rather
|
|
475
|
+
than returning near-instantly) did not resolve on resume; only a fresh,
|
|
476
|
+
explicitly constrained respawn produced a contract-valid review; treat a
|
|
477
|
+
watchdog stall as outside this preference. Record every misfire in
|
|
470
478
|
`03-decisions.md`. This matters most for review: a misfired review is not a
|
|
471
479
|
review and never satisfies the review gate, since review is never skipped.
|
|
472
480
|
|
|
@@ -485,6 +493,44 @@ the split. Acceptance criteria that cannot be satisfied this way go to the
|
|
|
485
493
|
operator as a merge-hold (hold the change unmerged and hand the decision to
|
|
486
494
|
the operator).
|
|
487
495
|
|
|
496
|
+
## Review-round escalation budget
|
|
497
|
+
|
|
498
|
+
The Round-2 halt rule above stops the first time a defect class recurs
|
|
499
|
+
within one task. This rule puts a budget on the whole task, across halts
|
|
500
|
+
and across repeated review rounds, so effort does not keep accumulating
|
|
501
|
+
unaided: by the second round-2 halt signal on the same task, or by the
|
|
502
|
+
third `fix_required` review round on the same task, whichever comes
|
|
503
|
+
first, choose one of three escalations instead of running another round
|
|
504
|
+
the same way. A counted round is a completed reviewer return whose
|
|
505
|
+
`acceptance_recommendation` is `fix_required` or `reject`; a misfired
|
|
506
|
+
review is not a round (see Subagent misfire rule); the escalation is
|
|
507
|
+
chosen once the third such round has returned, before the next attempt
|
|
508
|
+
starts. The escalation is chosen in addition to the halt rule's
|
|
509
|
+
split-or-redesign response, not instead of it.
|
|
510
|
+
|
|
511
|
+
- **Tier or model escalation**: raise the implementer to at least
|
|
512
|
+
`-xhigh` where that variant is installed, or to the strongest model
|
|
513
|
+
available in this environment. When it already runs at both, this
|
|
514
|
+
option is exhausted; under a `full` profile the choice falls to the
|
|
515
|
+
advisor spawn or the merge-hold, under a `minimal` profile (no advisor
|
|
516
|
+
subagent to spawn) it falls straight to the merge-hold.
|
|
517
|
+
- **Advisor spawn** (where the advisor is installed, `full` profile):
|
|
518
|
+
send the advisor subagent the question "redesign, split, or hold?" and
|
|
519
|
+
weigh its recommendation before deciding.
|
|
520
|
+
- **Merge-hold**: hold the change unmerged and hand the decision to the
|
|
521
|
+
operator.
|
|
522
|
+
|
|
523
|
+
Judgment governs which of the three to pick; only that one is chosen and
|
|
524
|
+
recorded is mandatory. Add a row (task, choice, reason) to
|
|
525
|
+
`03-decisions.md`'s Review-round escalation table, the record of the
|
|
526
|
+
decision, and set the `review-round-escalation` marker to the most recent
|
|
527
|
+
choice (a reader shortcut derived from that table, one of `n/a |
|
|
528
|
+
tier_escalation | advisor | merge_hold`). Escalating does not replace a
|
|
529
|
+
review round: whichever option is chosen, the next attempt still goes
|
|
530
|
+
through the reviewer subagent in full; this budget forces a change in
|
|
531
|
+
approach, not a shortcut past the review gate. Anchored by a measurement;
|
|
532
|
+
see the entry for this rule in the orchestrator-workflow CHANGELOG.
|
|
533
|
+
|
|
488
534
|
## Final acceptance rule
|
|
489
535
|
|
|
490
536
|
Subagents provide evidence. The orchestrator decides. The operator receives
|
|
@@ -3,3 +3,16 @@
|
|
|
3
3
|
| Date | Decision | Reason | Consequences |
|
|
4
4
|
|---|---|---|---|
|
|
5
5
|
| YYYY-MM-DD | <!-- decision --> | <!-- reason --> | <!-- consequences --> |
|
|
6
|
+
|
|
7
|
+
## Review-round escalation
|
|
8
|
+
|
|
9
|
+
<!-- One row per task that triggers the Review-round escalation budget in SKILL.md: the second round-2 halt signal or the third fix_required review round on that task. A run carries multiple tasks, so this table can carry multiple rows. Leave the single placeholder row as n/a when no task in this run has triggered the budget. -->
|
|
10
|
+
|
|
11
|
+
| Task | Choice | Reason |
|
|
12
|
+
|---|---|---|
|
|
13
|
+
| n/a | n/a | n/a |
|
|
14
|
+
|
|
15
|
+
<!-- Choice is one of: n/a | tier_escalation | advisor | merge_hold -->
|
|
16
|
+
|
|
17
|
+
<!-- review-round-escalation: choice = n/a -->
|
|
18
|
+
<!-- Reader marker: the most recent choice recorded in this run's table above, kept for readers that scan for a single marker rather than the table. One of: n/a | tier_escalation | advisor | merge_hold -->
|
|
@@ -28,3 +28,6 @@ accept | accept_with_notes | fix_required | reject
|
|
|
28
28
|
<!-- solution-acceptance: acceptance-recommendation = TODO -->
|
|
29
29
|
|
|
30
30
|
<!-- Reproduction note: when a finding rests on empirical or probabilistic evidence (flake rates, benchmarks, "n runs green", performance/timing numbers), record the reviewer's independent reproduction (method, sample size, result vs. the implementer's claim) in the reviewer output contract's `reproduction` field (SKILL.md step 7). Deterministic checks (a single test run, tsc, lint) do not require it. -->
|
|
31
|
+
|
|
32
|
+
<!-- Recurrence note: each finding in the reviewer output contract also carries a `recurrence` field (new or repeated), letting the orchestrator read the Review-round escalation budget's trigger (SKILL.md, Review-round escalation budget) off the reviewer's own return instead of reconstructing it by hand. A repeated finding here is what feeds that budget's round count. -->
|
|
33
|
+
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "orchestrator-workflow",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.25.0",
|
|
4
4
|
"description": "Installer for an orchestrator-led agent workflow: .ai/ run state, an AGENTS.md policy section, and per-harness subagent definitions for Claude Code, OpenAI Codex, and opencode",
|
|
5
5
|
"main": "dist/index.js",
|
|
6
6
|
"type": "module",
|