@ai-dossier/sched 0.8.0 → 0.10.0

package/README.md

@@ -96,12 +96,24 @@ where every mechanical supervision decision is code, not remembered prose:
    falls back to the global `stall_timeout_ms` outright.
 5. **Immediate refill (AC5)** — a slot freed by a terminal state is refilled in the SAME
    tick; a runnable unit never waits while a slot is idle (pinned by a regression test).
+   Refill was always synchronous; what previously had no journal trace was the release
+   itself — see `slot-released` below (#525).
 6. **Journal (AC6)** — every event (assigned, spawned, exit-detected, external-advance,
    progress, stalled, redispatched, fence-written, fence-failed, unit-failed,
-   dependents-blocked, suspect-dispatch, dispatch-unhealthy, …) is appended to
-   `events.jsonl`; `sched status` shows the live phase per unit, plus each slot's `gen`
-   and `fenced` state (#504). `label-blocked`/`label-check-failed` (#507) are the one pair journaled
-   OUTSIDE the engine — `sched enqueue` appends them at enqueue time, before dispatch.
+   dependents-blocked, slot-released, suspect-dispatch, dispatch-unhealthy, …) is
+   appended to `events.jsonl`; `sched status` shows the live phase per unit, plus each
+   slot's `gen` and `fenced` state (#504). `label-blocked`/`label-check-failed` (#507)
+   are the one pair journaled OUTSIDE the engine — `sched enqueue` appends them at
+   enqueue time, before dispatch. `slot-released` (#525) marks the exact tick a held
+   slot reaches `idle` on a per-issue dispatch terminal path — verified completion,
+   external-advance, a direct failure, a blocked dependent's release, or a
+   detached-ship park — carrying the freed `slot` id and a closed `reason`
+   (`verify-complete` / `external-advance` / `unit-failed` / `report-failed` /
+   `dependents-blocked` / `parked`, the exported `SlotReleaseReason` union), journaled
+   right after that path's own cause event so an occupancy report reads release time
+   directly instead of inferring it from the next `assigned` on that slot. Not yet
+   journaled by `sched abandon` or by batch-slot release, which walk a slot to `idle`
+   through their own copies of the same edge table (tracked as a follow-up).
 7. **Dispatch-health pause (#505)** — an unverified exit within `SUSPECT_DISPATCH_WINDOW_MS`
    (60s) of a slot's last progress is `suspect-dispatch`: real work rarely produces zero
    milestones that fast, but an operator-billing quota/auth wall (a Claude Code weekly
@@ -133,8 +145,9 @@ Two engine-safety policies were explicit product decisions on #464:
   an outage holds in `verifying` until truth returns. Each pause is journaled as
   `ground-truth-unreachable`.
 
-Only `issue:<n>` units are dispatched today — batch member sequencing is a follow-up
-(#464 non-goal).
+This applies to `issue:<n>` unit dispatch (`dispatchAssignments`). `batch:<id>` units run
+through a separate pass with its own claim/reconcile logic — see
+[Batch dispatch (#523)](#batch-dispatch-523) below.
 
 ### Zombie-run fencing (#504)
 
@@ -182,9 +195,13 @@ but the unprotected redispatch is never silent.
 What happens when a batch's aggregate suite goes red, or its PR will not merge
 (RFC-0001 §F.2/F.8/F.9).
 
-**Not yet wired into `sched start`** — `tick()` still dispatches only `issue:<n>` units
-(the batch execution loop is a follow-up, see above). These modules are the library
-surface that loop will call, and they are tested standalone against real scratch repos.
+**Wired into `sched start` since #523** — the `validating → attributing → fixing/evicting`
+rail below is called directly from `batch-dispatch.ts`'s `runValidate`/`evictOffender`
+(a red AGGREGATE suite, after every member individually went green). A member that never
+went green in the first place (its own gate failed) evicts through a separate, simpler
+rail that never touches this module — see
+[Batch dispatch (#523)](#batch-dispatch-523). These modules remain independently tested
+against real scratch repos.
 
 ```
 validating → attributing → fixing (ONE bounded attempt) → validating
@@ -270,6 +287,64 @@ suspect-dispatch streak (#505 above). The two fields are a single fact and must
 on load: no suspect dispatches were ever tracked under them, so `0`/`null` is the exact
 backfill, not a guess.
 
+## Batch dispatch (#523)
+
+`batch-dispatch.ts`'s `runBatchTick` — called from `tick()` after the issue-level pass,
+only when `batchExec`/`runBatchSuite` are both configured on `EngineDeps` — drives every
+`batch:<id>` unit through:
+
+```
+ready → executing(member i/N) ⟲ → validating → reviewing → shipping
+  → awaiting-merge → merged → deployed → reported → done
+failure rails: executing → dissolving (a member self-reports blocked)
+               validating → attributing → (fixing | evicting) → validating → dissolving
+```
+
+- **One shared worktree/branch per batch**, claimed once by a deterministic (no LLM)
+  `batch-setup` step: `git branch`/`push`/`worktree add` off `base_branch`, named
+  `batch/<id>-<date>`, plus a fresh `ai-dossier runstate mint` against the anchor issue.
+- **Members run serially, one fresh `slot-cycle` agent at a time**, in the shared
+  worktree. A member's completion signal is `phase=review status=done mode=slot` on its
+  OWN issue (`slot-cycle` posts no phase of its own past `review` — ship is batch-owned);
+  its commit range on the batch branch is recomputed (`git log`) after every member and
+  kept on `BatchEntry.ranges` for eviction. An incremental gate (`ai-dossier cap run
+  typecheck.run` / `test.focused`, when the repo has a manifest) runs after each member
+  before advancing — a second, independent check that the member's self-reported "done"
+  is real.
+- **The batch's single slot is claimed FRESH for each live step** (a member, the tail
+  agent, the report agent, a bounded fix agent) — never held across a wait. The aggregate
+  suite itself runs with NO slot claimed at all (deterministic engine work, not an LLM
+  step).
+- **Two failure rails.** A member that never went green evicts directly (nothing to
+  attribute — see the #472 section above for what "directly" skips). A red AGGREGATE
+  suite (every member individually green, but integration-level conflict) routes through
+  the #472 attribution/fix/evict library.
+- **The tail**, after the last member: the aggregate suite runs deterministically; green
+  spawns ONE bounded strong-tier agent that runs `review-issue` aggregate mode then
+  `ship-issue` batch mode (rebase-merge, a `Closes` list) and parks the PR exactly like a
+  detached full-cycle run; the engine's own PR watcher (a batch-granularity mirror of the
+  per-issue one) accepts the merge and dispatches a cheap mechanical-tier agent for
+  `report-issue`'s batch variant.
+- **Scope cuts, recorded rather than discovered later:** no `git bisect` stage for an
+  ambiguous aggregate failure (an unattributable red suite dissolves instead); no
+  worktree-pool integration for batch-setup (cold `git worktree add` only); no per-phase
+  stall/escalation ladder for batch sub-agents (a dead-without-verification agent is
+  treated as blocked, not redispatched stronger).
+
+Schema 1.7.0: `BatchEntry` gains `worktree` (absolute path of the shared batch worktree,
+null until batch-setup lands), `ranges` (`MemberRange[]` — each member's commit range,
+recomputed after every member completes) and `pr` (the batch PR parked on auto-merge,
+persisted so a restart mid-watch still knows what to poll). 1.6.0 states migrate on load:
+no batch was ever dispatched under them, so `null`/`[]`/`null` is the exact backfill, not
+a guess. Config schema moves to 1.3.0: `dispatch` gains `member_prompt`,
+`batch_tail_prompt` and `batch_report_prompt` (the three new agent prompt templates).
+
+New journal events: `batch-setup-done`, `batch-setup-failed`, `member-advanced`. Member/
+tail/report/fix-agent spawn, progress, completion and park events reuse the existing
+unit-generic names (`assigned`/`spawned`/`unit-failed`/`external-advance`/`pr-parked`/
+`merge-accepted`/`report-dispatched`/`teardown-done`/`teardown-failed`) with
+`unit = batch:<id>`.
+
 ## API surface
 
 ```ts
@@ -284,9 +359,11 @@ import {
   runLoop,               // the sched start loop (tick, sleep, repeat)
   type TickResult,       // what one tick did (spawned/parked/merge-accepted/stale-reconciled/
                          //   dependents-unblocked/report-dispatched/teardown/completed/
-                         //   redispatched/failed/blocked)
+                         //   redispatched/failed/blocked) — since #523 also carries
+                         //   `batch:<id>` unit ids (issue numbers for `blocked`)
   type EngineDeps,       // inject everything the engine touches (store/journal/spawn/ground
-                         //   truth/clock/repoDir/teardownExec/fencer)
+                         //   truth/clock/repoDir/teardownExec/fencer/batchExec/runBatchSuite/
+                         //   runBatchCapability — #523)
   createSpawnDeps,       // real detached-spawn process I/O
   createExecGroundTruth, // runstate/gh/git ground truth via subprocesses (injectable exec);
                          //   since #468 also gh pr view PR state + setup info from comments
@@ -337,9 +414,24 @@ import {
   transitionIssue, transitionBatch, transitionSlot,  // typed §D transitions
   TRANSITIONS,           // the transition tables themselves (for previews)
   buildStatusReport,     // machine-readable status incl. blocked/failed sets
-  validateState,         // strict persisted-state validation (1.0.0-1.5.0 files migrate)
+  validateState,         // strict persisted-state validation (1.0.0-1.6.0 files migrate)
   IllegalTransitionError, EnqueueError, CorruptStateError, LockTimeoutError,
   SchedNotFoundError,
+  runBatchTick,          // #523: one batch reconcile+refill pass; called by tick() after
+                         //   the issue pass — loads/saves state itself, holds no lock
+                         //   across the call
+  type BatchDispatchDeps, // inject store/journal/groundTruth/spawnDeps/exec/runSuite/
+                         //   runCapability(optional)/fsExists(optional)
+  type BatchTickResult,  // spawned/completed/parked/mergeAccepted/failed (batch:<id> ids)
+                         //   + blocked (issue numbers, dissolve-requeued)
+  type CapOutcome,       // ok | task-failed | automation-broken | capability-unavailable
+  buildMemberPrompt, buildBatchTailPrompt, buildBatchReportPrompt, // #523 prompt builders
+  DEFAULT_MEMBER_PROMPT_TEMPLATE, DEFAULT_BATCH_TAIL_PROMPT_TEMPLATE,
+  DEFAULT_BATCH_REPORT_PROMPT_TEMPLATE,
+  isMemberComplete, isMemberBlocked, // member milestone predicates (mode=slot gated)
+  isBatchTailParked,     // batch-ship awaiting-merge + pr= — the batch park signal
+  isBatchPhaseDone,      // <phase> done on the anchor (batch-review/batch-report)
+  batchOfUnit,           // batch:<id> → <id>; null for issue units or malformed ids
 } from '@ai-dossier/sched';
 ```
 

package/dist/batch-dispatch.d.ts

@@ -0,0 +1,121 @@
+/**
+ * Batch dispatch (#523, RFC-0001 §C.4/D.2/D.3): the missing driver that
+ * executes `batch:<id>` units. #498 landed the batch failure-recovery library
+ * (attribution/bisect/eviction/dissolve, `recovery.ts`) and the batch state
+ * machine (`state.ts`); readiness/placement already treat a `ready` batch as
+ * a runnable unit (`readiness.ts`, `scheduler.ts`). Nothing dispatched one
+ * until now.
+ *
+ * Shape, mirroring `engine.ts`'s per-issue dispatch: claim a slot → spawn an
+ * agent → poll ground truth → verify → transition. Generalized to `BatchEntry`
+ * at batch-phase granularity instead of per-issue-phase granularity:
+ *
+ * ```
+ * ready → executing(member i/N) ⟲ → validating → reviewing → shipping
+ *   → awaiting-merge → merged → deployed → reported → done
+ * failure rails (RFC F.2/F.8/F.9):
+ *   executing → dissolving  (a member self-reports blocked, RFC F.1)
+ *   validating → attributing → (fixing | evicting) → validating
+ *              → dissolving
+ * ```
+ *
+ * NO batch claim — not the first (`ready → executing`) nor any continuation
+ * (a later member, the tail agent, the report agent, the fix agent) — ever
+ * goes through `computeAssignments`/`runnableUnits`. Every one is a bespoke
+ * free-capacity-gated assignment, the same shape `engine.ts`'s
+ * `dispatchReportAgents` already uses (`runnableUnits` only ever offers a
+ * `status === 'ready'` batch, i.e. the moment BEFORE any claim). Between
+ * steps — a suite run, a PR merge wait — the slot is released to `idle` and
+ * holds no capacity (AC5): only a live member/tail/report/fix agent holds a
+ * slot.
+ *
+ * The aggregate suite itself is deterministic engine work, not an LLM step —
+ * it runs with no slot claimed at all, matching AC5's "member or batch-LLM-step"
+ * wording precisely.
+ *
+ * Two distinct failure rails, deliberately different:
+ * - A member's OWN agent reports itself blocked (its own gate never went
+ *   green) — evicted directly, no attribution needed: the offender is already
+ *   known, and either it has no commits yet (blocked before implementing) or
+ *   its commits are exactly what gets reverted.
+ * - The AGGREGATE suite (run by the engine after every member individually
+ *   went green) comes back red — an integration-level conflict no member's own
+ *   gate caught. THIS is what `recovery.ts`'s attribution/fix/evict pipeline
+ *   exists for (RFC F.2).
+ *
+ * Scope decisions recorded here, not silently cut: no `git bisect` stage for
+ * an ambiguous aggregate failure (bisect needs a per-project "run only these
+ * tests" command this module has no generic way to construct) — an
+ * unattributable red aggregate suite dissolves the batch rather than
+ * bisecting, which `attributing → dissolving` already models. No worktree-pool
+ * integration for batch-setup — cold `git worktree add` only, mirroring
+ * `teardown.ts`'s cold path. No per-phase stall/escalation ladder for batch
+ * sub-agents — a dead-without-verification agent is treated as blocked and
+ * evicted/reported rather than redispatched stronger. Both are documented
+ * follow-ups, not gaps discovered later.
+ */
+import { type ResolvedDispatch, type SpawnDeps } from './dispatch';
+import { type GroundTruth } from './groundtruth';
+import { type Journal } from './journal';
+import type { SchedStore } from './persist';
+import type { ExecFn } from './project';
+import { type SuiteResult } from './recovery';
+import { type FsExists } from './teardown';
+import type { SchedConfig } from './types';
+/**
+ * The four `ai-dossier cap run` outcomes (docs/reference/capabilities.md):
+ * `ok` = the capability ran and passed; `task-failed` = it ran and the TASK
+ * itself failed (trust the result); `automation-broken` = do not trust the
+ * machinery (missing tool, timeout, bad manifest); `capability-unavailable` =
+ * no manifest / no such id / `lifecycle: shadow` — no fast path here.
+ */
+export type CapOutcome = 'ok' | 'task-failed' | 'automation-broken' | 'capability-unavailable';
+/** Everything batch dispatch needs from the outside world. */
+export interface BatchDispatchDeps {
+    store: SchedStore;
+    journal: Journal;
+    groundTruth: GroundTruth;
+    spawnDeps: SpawnDeps;
+    now: () => Date;
+    /** Repo root — cwd for `git`/`ai-dossier` calls that are not batch-worktree-scoped. */
+    repoDir: string;
+    /** Exec for batch git/milestone-CLI operations (never throws — the `ExecFn` contract). */
+    exec: ExecFn;
+    /** Runs the aggregate suite inside a batch worktree; batches never leave `validating` without one. */
+    runSuite: (worktree: string) => SuiteResult;
+    /**
+     * Runs one `ai-dossier cap run <capabilityId>` in a batch worktree for the
+     * per-member incremental gate (#523 AC2 — "typecheck + focused tests via
+     * `cap run test.focused` when available"). Optional and degrade-not-crash,
+     * like `batchExec`/`runBatchSuite`: without it, or on `automation-broken`/
+     * `capability-unavailable`, the gate is skipped — the member's own
+     * `slot-cycle` run already attempted this fast path (with its own reasoning
+     * fallback) before ever posting `review done`, so a repo with no manifest
+     * loses nothing but the engine's independent re-check.
+     */
+    runCapability?: (worktree: string, capabilityId: string) => CapOutcome;
+    fsExists?: FsExists;
+}
+/** What one `runBatchTick` call did, merged into `engine.ts`'s `TickResult` by the caller. */
+export interface BatchTickResult {
+    spawned: string[];
+    completed: string[];
+    parked: string[];
+    mergeAccepted: string[];
+    failed: string[];
+    /** Issue numbers requeued full-cycle by a dissolve — matches `TickResult.blocked`'s shape. */
+    blocked: number[];
+}
+/**
+ * One batch reconcile+refill pass. Called from `engine.ts`'s `tick()` after
+ * the issue-level pass — batches never compete with issues for a slot within
+ * the same tick because this pass runs strictly after `dispatchAssignments`
+ * already filled every slot it could (see the module doc: batch claims never
+ * go through `computeAssignments`/`runnableUnits` at all). Loads and saves
+ * state itself via `deps.store.withLock` — the caller holds no lock across
+ * this call. `deps.exec` and `deps.runSuite` are mandatory; `deps.
+ * runCapability` is independently optional (AC2's incremental gate is itself
+ * a "when available" fast path).
+ */
+export declare function runBatchTick(deps: BatchDispatchDeps, config: SchedConfig, dispatch: ResolvedDispatch): BatchTickResult;
+//# sourceMappingURL=batch-dispatch.d.ts.map

package/dist/batch-dispatch.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"batch-dispatch.d.ts","sourceRoot":"","sources":["../src/batch-dispatch.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GAuDG;AAWH,OAAO,EAKL,KAAK,gBAAgB,EACrB,KAAK,SAAS,EAEf,MAAM,YAAY,CAAC;AACpB,OAAO,EACL,KAAK,WAAW,EAOjB,MAAM,eAAe,CAAC;AACvB,OAAO,EAAE,KAAK,OAAO,EAAa,MAAM,WAAW,CAAC;AACpD,OAAO,KAAK,EAAE,UAAU,EAAE,MAAM,WAAW,CAAC;AAC5C,OAAO,KAAK,EAAE,MAAM,EAAE,MAAM,WAAW,CAAC;AACxC,OAAO,EASL,KAAK,WAAW,EACjB,MAAM,YAAY,CAAC;AAWpB,OAAO,EAAE,KAAK,QAAQ,EAA+B,MAAM,YAAY,CAAC;AACxE,OAAO,KAAK,EAKV,WAAW,EAIZ,MAAM,SAAS,CAAC;AAEjB;;;;;;GAMG;AACH,MAAM,MAAM,UAAU,GAAG,IAAI,GAAG,aAAa,GAAG,mBAAmB,GAAG,wBAAwB,CAAC;AAE/F,8DAA8D;AAC9D,MAAM,WAAW,iBAAiB;IAChC,KAAK,EAAE,UAAU,CAAC;IAClB,OAAO,EAAE,OAAO,CAAC;IACjB,WAAW,EAAE,WAAW,CAAC;IACzB,SAAS,EAAE,SAAS,CAAC;IACrB,GAAG,EAAE,MAAM,IAAI,CAAC;IAChB,uFAAuF;IACvF,OAAO,EAAE,MAAM,CAAC;IAChB,0FAA0F;IAC1F,IAAI,EAAE,MAAM,CAAC;IACb,sGAAsG;IACtG,QAAQ,EAAE,CAAC,QAAQ,EAAE,MAAM,KAAK,WAAW,CAAC;IAC5C;;;;;;;;;OASG;IACH,aAAa,CAAC,EAAE,CAAC,QAAQ,EAAE,MAAM,EAAE,YAAY,EAAE,MAAM,KAAK,UAAU,CAAC;IACvE,QAAQ,CAAC,EAAE,QAAQ,CAAC;CACrB;AAED,8FAA8F;AAC9F,MAAM,WAAW,eAAe;IAC9B,OAAO,EAAE,MAAM,EAAE,CAAC;IAClB,SAAS,EAAE,MAAM,EAAE,CAAC;IACpB,MAAM,EAAE,MAAM,EAAE,CAAC;IACjB,aAAa,EAAE,MAAM,EAAE,CAAC;IACxB,MAAM,EAAE,MAAM,EAAE,CAAC;IACjB,8FAA8F;IAC9F,OAAO,EAAE,MAAM,EAAE,CAAC;CACnB;AA8uCD;;;;;;;;;;GAUG;AACH,wBAAgB,YAAY,CAC1B,IAAI,EAAE,iBAAiB,EACvB,MAAM,EAAE,WAAW,EACnB,QAAQ,EAAE,gBAAgB,GACzB,eAAe,CAsEjB"}