orchestrator-workflow 0.22.0 → 0.24.0

package/CHANGELOG.md

@@ -5,6 +5,127 @@ All notable changes to `orchestrator-workflow` are documented here.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.24.0] - 2026-08-24
+
+### Changed
+
+- **Moved org-, machine-, and point-in-time-bound evidence out of the kit's
+  reusable instruction files; rule text is unchanged.** Public,
+  tool-agnostic kit files (`SKILL.md`, `agents-md-section.md`) now carry
+  rules and procedures only; the measurements, dates, sample sizes, task
+  ids, and incident tallies that used to sit inline are recorded here in
+  the changelog instead, with a one-line pointer left in prose.
+  - `SKILL.md` step 6 ("Delegate implementation"): the parenthetical
+    `(2026-08-24 A/B measurement, n=8: implementer-low reached accept a
+    median 320 seconds slower, p=0.016, with 9 high-plus-critical review
+    findings against 1 and 8 fix rounds against 1)` is now `(anchored by an
+    A/B measurement; see CHANGELOG 0.23.0)`. The full numbers already live
+    in the 0.23.0 entry below.
+  - `agents-md-section.md`'s Scaling delegation bullet: the paragraph
+    naming the A/B's `n=8`, the median slowdown, `p=0.016`, the
+    high-plus-critical finding count, the fix-round count, the Haiku 4.5
+    model detail, and agent-tasks task `7f38899d` is replaced by "This rule
+    is anchored by an A/B measurement; the data and the model caveat are
+    recorded in the orchestrator-workflow CHANGELOG (0.23.0)." Same data,
+    same 0.23.0 entry.
+  - `SKILL.md`'s "Subagent misfire rule": the incident tally `(four so
+    far)` and the whole reviewer/model-correlation passage ("So far this
+    signal has only been observed for the reviewer role ...", including the
+    0.21.0 advisor remark and the pointer to the per-role model
+    preferences) are removed. The rule itself (the signal definition, the
+    contract-parse and near-instant detection signals, the
+    resume-over-respawn preference and its reasoning, the respawn fallback,
+    the watchdog-stall exception, the `03-decisions.md` record requirement,
+    and "a misfired review is not a review") is unchanged. The removed
+    passage, verbatim, for the durable record: "Every incident of this
+    exact signal (a return within seconds, zero tool calls, harness or
+    system boilerplate instead of the output contract) whose outcome was
+    recorded (four so far) has resolved on the first resume attempt; fall
+    back to a fresh respawn only if the resume attempt itself misfires the
+    same way. So far this signal has only been observed for the reviewer
+    role, a role whose default model differs from explorer's,
+    task-slicer's, and implementer's (since 0.21.0 the advisor shares the
+    reviewer's default model too; the advisor has had no spawns yet, so it
+    contributes no evidence either way; see the per-role model
+    preferences); treat that correlation as an open lead worth watching as
+    more incidents accumulate, not as a confirmed cause." Of the four
+    resume outcomes recorded for that signal, three were on 2026-07-16 and
+    one was on 2026-07-20. The watchdog-stall exception's removed sentence,
+    also verbatim: "This resume-over-respawn preference does not extend to
+    a structurally different misfire class: a mid-run watchdog stall (the
+    subagent goes idle partway through a run rather than returning
+    near-instantly) did not resolve on resume in the one measured incident
+    of that class, it stalled a second time, and only a fresh, explicitly
+    constrained respawn produced a contract-valid review; treat a watchdog
+    stall as outside this preference."
+  - `SKILL.md`'s Run state paragraph: "see the grounding-mcp 0.6.0 docs for
+    the full consumer semantics" is now "see the consuming gate's
+    documentation (grounding-mcp) for the full consumer semantics", dropping
+    the pinned version number.
+
+### Added
+
+- **Reviewer placement check.** `reviewer.md`'s "Check, at minimum" list
+  gains a check for org-, machine-, or point-in-time-bound evidence (dates,
+  sample sizes, task ids, home paths, incident tallies) leaking into a
+  reusable instruction file (a skill, an agent prompt, an AGENTS.md
+  section, a template); the fix it recommends is moving the evidence to the
+  changelog, the run files, or the consuming workspace and leaving a
+  one-line pointer.
+- **Hand-off placement check.** `SKILL.md` step 9 ("Hand off") gains one
+  sentence for the orchestrator: check before handing off that no such
+  evidence was added to a reusable instruction file.
+- **`placement-guard` CI job.** A dedicated job in agent-dx's
+  `.github/workflows/ci.yml`, separate from the package matrix, builds
+  `slop-detector` and runs its opt-in `placement-slop` pack against the
+  monorepo's instruction files (`packages/orchestrator-workflow/assets/**`
+  and `packages/agentic-coding-playbook/**`, configured via the repo-root
+  `slop.config.yml`), failing on block-level violations.
+- **Root `slop.config.yml`.** Opts the `placement-slop` pack in for the CI
+  job, configures the `LanNguyenSi` org marker with `allow` entries for its
+  two legitimate repo links (`github.com/LanNguyenSi/`,
+  `raw.githubusercontent.com/LanNguyenSi/`, both anchored to a full
+  `https://` URL), widens the pack's instruction-file globs to cover this
+  package's `assets/` tree and the `agentic-coding-playbook` package, and
+  overrides `placement-slop/dated-evidence`, `placement-slop/tally-phrase`,
+  and `placement-slop/opaque-id` from their pack default of `warn` to
+  `block`, so the CI job actually fails on a leaked date, tally phrase, or
+  opaque id instead of only warning (a home path or an unlisted org marker
+  was already `block` by pack default). `packages/github-api-tool/SKILL.md`
+  carries two pre-existing dated examples that this severity change would
+  now block; it is `ignorePaths`-excluded pending a follow-up cleanup of
+  that unrelated package.
+  Known limit of the pack, unchanged here: an `allow` match suppresses
+  every placement rule on that line, so a home path or a date that shares
+  a line with an allowed repo URL is not reported; a span-scoped allow is
+  a slop-detector follow-up.
+
+## [0.23.0] - 2026-08-24
+
+### Changed
+
+- **Tightened the `implementer-low` tier rule from discretion to a checkable
+  gate on the task contract.** `agents-md-section.md`'s Scaling delegation
+  bullet and `SKILL.md` step 6 ("Delegate implementation") no longer say
+  `-low` "fits mechanical, narrowly scoped tasks" for the implementer;
+  `implementer-low` is now spawned only when none of the following hold: an
+  acceptance criterion demands a test, typecheck, lint, or build run; the task
+  assignment names mutation probes to run; or the task slicer's
+  `suggested_tests` came back non-empty. A task with any of those, including a
+  bugfix that looks mechanical, runs on the unsuffixed implementer or higher.
+  When it is unclear whether a criterion demands a run, exclude
+  `implementer-low`. The explorer and the task-slicer keep the prior
+  discretionary `-low` guidance unchanged, since no equivalent measurement
+  exists for those roles. Operator decision 2026-08-24 after a Tier-A/B
+  measurement of implementer-low as installed (Haiku 4.5) against the default
+  implementer (Sonnet 5, effort medium) (agent-tasks task 7f38899d), blinded,
+  n=8, identical tasks in both tiers: implementer-low reached accept a median
+  320 seconds slower (p=0.016), drew 9 high-plus-critical review findings
+  against 1, and needed 8 fix rounds against 1. The A/B's implementer-low ran
+  on Haiku 4.5, which does not support the `effort` parameter, so the harness
+  ignores the pinned `effort: low` on that model; the measurement compared
+  Haiku 4.5 without effort control against Sonnet 5 at `effort: medium`.
+
 ## [0.22.0] - 2026-08-20
 
 ### Changed

package/INSTALL-AGENT.md

@@ -198,12 +198,13 @@ steps in the repository you were asked to install into.
      ```
    - `.ai/workflow/manifest.json`, exactly this shape (harnesses MUST be an
      array, `profile` is `"minimal"` or `"full"`, models keyed by role,
-     version = the kit version you installed):
+     version = the kit version you installed, read from this kit's own
+     `packages/orchestrator-workflow/package.json` `version` field):
 
      ```json
      {
        "kit": "orchestrator-workflow",
-       "version": "0.5.0",
+       "version": "<kit version you installed>",
        "harnesses": ["claude", "opencode"],
        "profile": "full",
        "tiers": false,
@@ -215,7 +216,7 @@ steps in the repository you were asked to install into.
          "advisor": "opus"
        },
        "files": {},
-       "installedAt": "2026-06-12T00:00:00.000Z"
+       "installedAt": "<ISO 8601 timestamp of this install>"
      }
      ```
 

package/assets/agents/reviewer.md

@@ -21,6 +21,11 @@ Check, at minimum:
 - Test adequacy: are the new or changed behaviors covered, and would the new
   tests actually fail if the change were reverted? Flag inert tests.
 - Maintainability: naming, dead code, needless abstraction, doc drift.
+- Placement: does the change add org-, machine-, or point-in-time-bound
+  evidence (dates, sample sizes, task ids, home paths, incident tallies) to a
+  reusable instruction file (a skill, an agent prompt, an AGENTS.md section, a
+  template)? Report it; the fix is to move the evidence to the changelog, the
+  run files, or the consuming workspace and leave a one-line pointer.
 
 Rules:
 

package/assets/agents-md-section.md

@@ -50,16 +50,28 @@ default, not a ritual.
   changes whose risk or size warrants an independent skeptical pass. Either
   way, review is never skipped.
 - When tier variants are installed (manifest `tiers: true`), the orchestrator
-  picks the effort tier per task by complexity and risk, at its own
-  judgment. The unsuffixed default subagent is the normal case; a `-low`
-  variant fits mechanical, narrowly scoped tasks; `-high`/`-xhigh` fit
-  high-risk changes, hard problems, or repeated failed attempts. Not every
-  role gets every tier: `-xhigh` exists only for the implementer, the
-  reviewer, and the advisor. The reviewer's downshift is `-medium` rather
-  than `-low`, since its default already sits at high. The advisor has no
-  downshift at all: its default `high` is already its only non-`-xhigh`
-  tier. Spawn only variants that are actually installed. Tier choice is a
-  conscious decision, not a ritual; when unsure, use the default.
+  picks the effort tier per task by complexity and risk, at its own judgment.
+  The unsuffixed default subagent is the normal case; `-high`/`-xhigh` fit
+  high-risk changes, hard problems, or repeated failed attempts. For the
+  implementer specifically, `-low` is spawned only when none of the following
+  hold: an acceptance criterion demands a test, typecheck, lint, or build run;
+  the task assignment names mutation probes to run; or the task slicer's
+  `suggested_tests` came back non-empty. This is checkable against the task
+  contract rather than a judgment about how hard the task looks: any one of
+  those three excludes `implementer-low`, and the task runs on the unsuffixed
+  implementer or higher, even when the change looks mechanical (a bugfix
+  included); when it is unclear whether a criterion demands a run, exclude
+  `implementer-low`. This rule is anchored by an A/B measurement; the data and
+  the model caveat are recorded in the orchestrator-workflow CHANGELOG
+  (0.23.0). For the explorer and the task-slicer, a `-low` variant still suits
+  narrowly scoped, mechanical work; no equivalent measurement exists for those
+  two roles, so their rule is unchanged. Not every role gets every tier:
+  `-xhigh` exists only for the implementer, the reviewer, and the advisor. The
+  reviewer's downshift is `-medium` rather than `-low`, since its default
+  already sits at high. The advisor has no downshift at all: its default `high`
+  is already its only non-`-xhigh` tier. Spawn only variants that are actually
+  installed. Tier choice is a conscious decision, not a ritual; when unsure,
+  use the default.
 - Every unsuffixed default subagent carries its own pinned default effort
   baked into its own file, not inherited from the orchestrator session:
   medium for the explorer, the task-slicer, and the implementer; high for

package/assets/skill/SKILL.md

@@ -90,8 +90,8 @@ run-completeness readers, not an acceptance verdict, and it fails open:
 left as `TODO` it does not block anything, the reader just falls back to a
 tolerant day-granular date heuristic. The recorded base must resolve in the
 repo, be an ancestor of HEAD, and must not lie behind the fork point of the
-change (the merge-base with the remote default branch); see the
-grounding-mcp 0.6.0 docs for the full consumer semantics.
+change (the merge-base with the remote default branch); see the consuming
+gate's documentation (grounding-mcp) for the full consumer semantics.
 
 ## Workflow
 
@@ -141,7 +141,14 @@ directory and the subagents.
    installed `implementer-<tier>` subagents, if any) by the task's
    complexity and risk, at your own judgment, defaulting to the unsuffixed
    subagent when unsure; record a non-default tier choice with a
-   one-line reason in `03-decisions.md` when the task is non-trivial. When a
+   one-line reason in `03-decisions.md` when the task is non-trivial.
+   `implementer-low` is spawned only when none of the following hold: an
+   acceptance criterion demands a test, typecheck, lint, or build run; the
+   task assignment names mutation probes to run; or the task slicer's
+   `suggested_tests` came back non-empty. Any one of those three excludes
+   `implementer-low`, even for a change that looks mechanical (a bugfix
+   included) (anchored by an A/B measurement; see CHANGELOG 0.23.0). When it is
+   unclear whether a criterion demands a run, exclude `implementer-low`. When a
    task's acceptance rests on a test that must fail without the change, name
    the mutation probes to run in the task assignment; the implementer reports
    each one in the output contract's `mutation_probes` field (apply the mutant
@@ -204,7 +211,10 @@ directory and the subagents.
    validator when one is available (for example `okf-kit check`). Repos
    without a bundle are unaffected. Then fill `06-handoff.md` and report to the
    operator: what changed, why, how it was verified, known risks, accepted
-   waivers, suggested next step.
+   waivers, suggested next step. Before handing off, check that no org-,
+   machine-, or point-in-time-bound evidence was added to a reusable
+   instruction file; such evidence belongs in the changelog, the run files,
+   or the consuming workspace, with a pointer left behind.
 
 When finalizing `05-review-findings.md` and `06-handoff.md`, replace the `TODO`
 in each `<!-- solution-acceptance: ... = TODO -->` marker with the chosen enum
@@ -442,22 +452,14 @@ explicitly repeats the original assignment rather than a generic retry,
 since resume keeps the subagent's prior turn in context while a fresh spawn
 starts cold and risks the same misfire again. Every incident of this exact
 signal (a return within seconds, zero tool calls, harness or system
-boilerplate instead of the output contract) whose outcome was recorded
-(four so far) has resolved on the first resume attempt; fall back to a
-fresh respawn only if the resume attempt itself misfires the same way. So
-far this signal has only been observed for the reviewer role, a role whose
-default model differs from explorer's, task-slicer's, and implementer's
-(since 0.21.0 the advisor shares the reviewer's default model too; the
-advisor has had no spawns yet, so it contributes no evidence either way; see
-the per-role model preferences); treat that correlation as an open lead
-worth watching as more incidents accumulate, not as a confirmed cause. This
-resume-over-respawn preference does not extend to a structurally different
-misfire class: a mid-run watchdog stall (the subagent goes idle partway
-through a run rather
-than returning near-instantly) did not resolve on resume in the one
-measured incident of that class, it stalled a second time, and only a
-fresh, explicitly constrained respawn produced a contract-valid review;
-treat a watchdog stall as outside this preference. Record every misfire in
+boilerplate instead of the output contract) whose outcome was recorded has
+resolved on the first resume attempt; fall back to a fresh respawn only if
+the resume attempt itself misfires the same way. This resume-over-respawn
+preference does not extend to a structurally different misfire class: a
+mid-run watchdog stall (the subagent goes idle partway through a run rather
+than returning near-instantly) did not resolve on resume; only a fresh,
+explicitly constrained respawn produced a contract-valid review; treat a
+watchdog stall as outside this preference. Record every misfire in
 `03-decisions.md`. This matters most for review: a misfired review is not a
 review and never satisfies the review gate, since review is never skipped.
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "orchestrator-workflow",
-  "version": "0.22.0",
+  "version": "0.24.0",
   "description": "Installer for an orchestrator-led agent workflow: .ai/ run state, an AGENTS.md policy section, and per-harness subagent definitions for Claude Code, OpenAI Codex, and opencode",
   "main": "dist/index.js",
   "type": "module",