@ai-dossier/sched 0.19.0 → 0.21.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -82,7 +82,15 @@ where every mechanical supervision decision is code, not remembered prose:
82
82
  "waiting for it to finish" abandons the run with the subprocess still going; the
83
83
  instruction tells it to run such commands in the foreground and wait, or poll until
84
84
  they finish. `DEFAULT_REPORT_PROMPT_TEMPLATE` is excluded — it never spawns a long
85
- command.
85
+ command. The prompt instruction alone was not enough (#591): agents kept arming the
86
+ `Monitor` tool to wait on a background command and ending their turn anyway, which the
87
+ engine can only see as an unverified exit. Every `claude`-family command template
88
+ (top-level `command` and each tier's own `commandTemplate`, #527) gets
89
+ `--disallowedTools Monitor` appended automatically — set `dispatch.disallowed_tools: []`
90
+ in `config.json` to opt out, or list your own tools to deny instead of the default
91
+ `["Monitor"]`. Matched on the binary's basename, so an absolute or wrapper path
92
+ (`/usr/local/bin/claude`) still gets it; never applied to a non-`claude` command or one
93
+ that already carries the flag itself, so an `opencode` tier is unaffected.
86
94
  2. **Completion verification (AC2)** — an agent exiting is never proof of completion.
87
95
  On exit, the unit completes only when ground truth confirms it: the issue's latest
88
96
  runstate milestone is `report done`, or GitHub says the issue is closed — except a
@@ -455,6 +463,8 @@ ready → executing(member i/N) ⟲ → validating → reviewing → shipping
455
463
  failure rails: executing → dissolving (a member self-reports blocked)
456
464
  validating → attributing → (fixing | evicting) → validating → dissolving
457
465
  validating → blocked (suite report unreadable, #562) → validating
466
+ executing → blocked (gate-inconclusive:<cap>, #583) → executing
467
+ (`sched resume --batch <id>` re-runs the gate; nothing requeued/reverted)
458
468
  ```
459
469
 
460
470
  - **One shared worktree/branch per batch**, claimed once by a deterministic (no LLM)
@@ -473,7 +483,12 @@ failure rails: executing → dissolving (a member self-reports blocked)
473
483
  kept on `BatchEntry.ranges` for eviction. An incremental gate (`ai-dossier cap run
474
484
  typecheck.run` / `test.focused`, when the repo has a manifest) runs after each member
475
485
  before advancing — a second, independent check that the member's self-reported "done"
476
- is real.
486
+ is real. Three-way outcome policy (#583): `task-failed` evicts the member (same rail
487
+ as a self-reported block); `automation-broken`/`capability-unavailable` — the gate
488
+ itself couldn't reach a verdict — block the batch instead of silently proceeding
489
+ (`gate-inconclusive:<cap>`, `member_gates`/`blocked_reason` on `BatchEntry`, surfaced
490
+ in `sched status`); `sched resume --batch <id>` re-runs the gate later to resolve the
491
+ block once the capability is fixed.
477
492
  - **The batch's single slot is claimed FRESH for each live step** (a member, the tail
478
493
  agent, the report agent, a bounded fix agent) — never held across a wait. The aggregate
479
494
  suite itself runs with NO slot claimed at all (deterministic engine work, not an LLM
@@ -504,10 +519,21 @@ batch-setup had pool integration). Config schema moves to 1.3.0: `dispatch` gain
504
519
  `member_prompt`, `batch_tail_prompt` and `batch_report_prompt` (the three new agent
505
520
  prompt templates).
506
521
 
522
+ Schema 1.11.0 (#583): `BatchEntry` gains `member_gates` (most recent incremental-gate
523
+ result per member, keyed by issue number as a string — `{capability, outcome,
524
+ output_tail, at}`) and `blocked_reason` (why the batch is `blocked` — persisted so
525
+ `sched status` can show it; previously `blockBatch` only journaled/posted the reason,
526
+ never stored it on the entry, so this also retroactively covers the #562 case). 1.10.0
527
+ states migrate on load: no gate has ever produced a non-`ok` verdict, and no batch has
528
+ ever been blocked, under them, so `{}`/`null` is the exact backfill, not a guess.
529
+
507
530
  New journal events: `batch-setup-done`, `batch-setup-failed`, `member-advanced`,
508
531
  `batch-warmup-done`, `batch-warmup-failed` (#561 — the cold-path warm step only; a pool
509
- claim emits neither). Member/tail/report/fix-agent spawn, progress, completion and park
510
- events reuse the existing unit-generic names (`assigned`/`spawned`/`unit-failed`/
532
+ claim emits neither). `gate-inconclusive` (#583 the incremental gate came back
533
+ `automation-broken`/`capability-unavailable` rather than a definite `ok`/`task-failed`;
534
+ sits alongside `batch-blocked` as the per-member analogue of the aggregate suite's
535
+ "block, don't dissolve" precedent). Member/tail/report/fix-agent spawn, progress,
536
+ completion and park events reuse the existing unit-generic names (`assigned`/`spawned`/`unit-failed`/
511
537
  `external-advance`/`pr-parked`/`merge-accepted`/`report-dispatched`/`teardown-done`/
512
538
  `teardown-failed`) with `unit = batch:<id>`.
513
539
 
@@ -624,10 +650,12 @@ import {
624
650
  // the issue pass — loads/saves state itself, holds no lock
625
651
  // across the call
626
652
  type BatchDispatchDeps, // inject store/journal/groundTruth/spawnDeps/exec/runSuite/
627
- // runCapability(optional)/fsExists(optional)
653
+ // runCapability(optional, returns CapabilityGateResult)/fsExists(optional)
628
654
  type BatchTickResult, // spawned/completed/parked/mergeAccepted/failed (batch:<id> ids)
629
655
  // + blocked (issue numbers, dissolve-requeued)
630
656
  type CapOutcome, // ok | task-failed | automation-broken | capability-unavailable
657
+ type CapabilityGateResult, // {outcome: CapOutcome, outputTail?, reason?} — runCapability's return shape (#583)
658
+ resumeBlockedGate, // #583: sched resume --batch <id> — re-run the gate that blocked a batch
631
659
  buildMemberPrompt, buildBatchTailPrompt, buildBatchReportPrompt, // #523 prompt builders
632
660
  DEFAULT_MEMBER_PROMPT_TEMPLATE, DEFAULT_BATCH_TAIL_PROMPT_TEMPLATE,
633
661
  DEFAULT_BATCH_REPORT_PROMPT_TEMPLATE,
@@ -656,7 +684,7 @@ telemetry" below.
656
684
  ├── config.json # durable intent: max_slots, stall_timeout_ms, reconcile_interval_ms,
657
685
  │ # pr_poll_interval_ms, dispatch (incl. report_prompt,
658
686
  │ # phase_stall_timeout_ms, fence_takeover_timeout_ms, tiers — #527,
659
- │ # suite_command — #562), auto_upgrade — #537,
687
+ │ # suite_command — #562, disallowed_tools — #591), auto_upgrade — #537,
660
688
  │ # dissolve_policy — #563
661
689
  ├── events.jsonl # append-only event journal (the operator's flight recorder)
662
690
  ├── runs/ # per-unit agent output logs (issue-<n>.log)
@@ -677,6 +705,14 @@ every place a dispatch's exit is first detected: the dead-pid rail, the
677
705
  external-advance rail, a stall-timeout kill, and a dependents-blocked kill), sourced
678
706
  from the agent's `modelUsage` map — never blended with the top-level `usage` block,
679
707
  the fix for a ~43% fabricated-saving discrepancy the two blocks were found to produce.
708
+ `recordDispatchRunLog`/`recordMemberRunLog` also return the last tool the dispatch called
709
+ (`parseLastToolUse`, `@ai-dossier/core`, #591), when the log yielded one — the exit itself
710
+ attributes to a concrete cause (e.g. `Monitor`) without opening the transcript. It rides the
711
+ non-terminal `verify-incomplete` event on every unverified exit and, once the escalation
712
+ ladder is exhausted, the terminal `unit-failed` (`agent-exited-unverified` /
713
+ `unverified-exit-at-strongest-tier`) as `last_tool`; a stall-timeout kill carries it too. Only
714
+ the dead-pid detection rail and the stall kill record a fresh log slice in the same tick —
715
+ a slot already `exited`/`verifying` when reconciled again has none to attribute.
680
716
 
681
717
  The dispatch log (`runs/<unit>.log`) is per-UNIT and opened in append mode
682
718
  (`createSpawnDeps`), so a redispatched unit's second agent writes its output AFTER the
@@ -67,15 +67,8 @@ import type { SchedStore } from './persist';
67
67
  import type { ExecFn } from './project';
68
68
  import { type SuiteResult } from './recovery';
69
69
  import { type FsExists } from './teardown';
70
- import type { SchedConfig } from './types';
71
- /**
72
- * The four `ai-dossier cap run` outcomes (docs/reference/capabilities.md):
73
- * `ok` = the capability ran and passed; `task-failed` = it ran and the TASK
74
- * itself failed (trust the result); `automation-broken` = do not trust the
75
- * machinery (missing tool, timeout, bad manifest); `capability-unavailable` =
76
- * no manifest / no such id / `lifecycle: shadow` — no fast path here.
77
- */
78
- export type CapOutcome = 'ok' | 'task-failed' | 'automation-broken' | 'capability-unavailable';
70
+ import type { CapabilityGateResult, SchedConfig } from './types';
71
+ export type { CapOutcome } from './types';
79
72
  /** Everything batch dispatch needs from the outside world. */
80
73
  export interface BatchDispatchDeps {
81
74
  store: SchedStore;
@@ -100,19 +93,24 @@ export interface BatchDispatchDeps {
100
93
  /**
101
94
  * Runs one `ai-dossier cap run <capabilityId>` in a batch worktree. Two call
102
95
  * sites, different degrade contracts:
103
- * - the per-member incremental gate (#523 AC2 "typecheck + focused tests
104
- * via `cap run test.focused` when available"): degrade-not-crash — without
105
- * this hook, or on `automation-broken`/`capability-unavailable`, the gate
106
- * is skipped (the member's own `slot-cycle` run already attempted this
107
- * fast path before ever posting `review done`, so a repo with no manifest
108
- * loses nothing but the engine's independent re-check).
96
+ * - the per-member incremental gate (#523 AC2, revised #583): without this
97
+ * hook, the gate is skipped entirely (the member's own `slot-cycle` run
98
+ * already attempted this fast path before ever posting `review done`, so
99
+ * a repo with no manifest loses nothing but the engine's independent
100
+ * re-check). With the hook: `ok` advances, `task-failed` evicts,
101
+ * `automation-broken`/`capability-unavailable` BLOCK the batch
102
+ * (`gate-inconclusive`) rather than silently proceeding — #583 found the
103
+ * old "proceed on anything but task-failed" policy let a script's own
104
+ * "I could not run this" signal (a non-zero exit reporting it never
105
+ * really tested anything) masquerade as either a pass or a real failure.
109
106
  * - batch-setup's `worktree.prepare` warm step (#561, `warmColdBatchWorktree`):
110
- * `undefined`/`capability-unavailable`/`automation-broken` fall through to
111
- * package-manager detection, but a declared-and-`task-failed` capability
112
- * hard-fails the whole batch setup — a repo that owns its warm-up should
113
- * never be silently second-guessed by a fallback underneath it.
107
+ * unaffected by #583's gate policy change — `undefined`/
108
+ * `capability-unavailable`/`automation-broken` still fall through to
109
+ * package-manager detection, and a declared-and-`task-failed` capability
110
+ * still hard-fails the whole batch setup (a repo that owns its warm-up
111
+ * should never be silently second-guessed by a fallback underneath it).
114
112
  */
115
- runCapability?: (worktree: string, capabilityId: string) => CapOutcome;
113
+ runCapability?: (worktree: string, capabilityId: string) => CapabilityGateResult;
116
114
  fsExists?: FsExists;
117
115
  /**
118
116
  * Home directory for `~/.dossier/runs.jsonl` (#564) — mirrors `EngineDeps.homeDir`.
@@ -130,6 +128,33 @@ export interface BatchTickResult {
130
128
  /** Issue numbers requeued full-cycle by a dissolve — matches `TickResult.blocked`'s shape. */
131
129
  blocked: number[];
132
130
  }
131
+ /**
132
+ * `sched resume --batch <id>` (#583 AC4): an operator-triggered, synchronous
133
+ * one-shot recheck of a batch blocked on `gate-inconclusive:<capabilityId>`
134
+ * — re-runs exactly that capability against the current member and resolves
135
+ * the block:
136
+ *
137
+ * - still `automation-broken`/`capability-unavailable` → stays `blocked`,
138
+ * no state change (the capability still isn't fixed).
139
+ * - `task-failed` → the member really is broken; evict via the same rail
140
+ * the live gate uses.
141
+ * - `ok` → the member really was fine; complete it via the same rail the
142
+ * live gate uses.
143
+ *
144
+ * Deliberately NOT a generic engine tick: `runBatchTick`'s own "executing
145
+ * with no live slot" wedge-recovery path (`spawnMemberContinuation`,
146
+ * unconditional) exists for a dispatch that never happened — reusing it here
147
+ * would redispatch a fresh agent for a member whose work is already
148
+ * committed and reviewed. This function transitions `blocked → executing`
149
+ * and immediately, synchronously, calls the same completion/eviction
150
+ * functions the live gate calls — no intervening tick ever sees the batch
151
+ * `executing` with nothing in flight.
152
+ */
153
+ export declare function resumeBlockedGate(deps: BatchDispatchDeps, config: SchedConfig, dispatch: ResolvedDispatch, batchId: string, now: Date): {
154
+ outcome: 'still-blocked' | 'evicted' | 'completed';
155
+ capability: string;
156
+ detail?: string;
157
+ };
133
158
  /**
134
159
  * One batch reconcile+refill pass. Called from `engine.ts`'s `tick()` after
135
160
  * the issue-level pass — this pass never claims a slot the issue pass already
@@ -1 +1 @@
1
- {"version":3,"file":"batch-dispatch.d.ts","sourceRoot":"","sources":["../src/batch-dispatch.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GA6DG;AAiBH,OAAO,EASL,KAAK,gBAAgB,EAErB,KAAK,SAAS,EACf,MAAM,YAAY,CAAC;AACpB,OAAO,EACL,KAAK,WAAW,EAOjB,MAAM,eAAe,CAAC;AACvB,OAAO,EAAE,KAAK,OAAO,EAAa,MAAM,WAAW,CAAC;AACpD,OAAO,KAAK,EAAE,UAAU,EAAE,MAAM,WAAW,CAAC;AAC5C,OAAO,KAAK,EAAE,MAAM,EAAE,MAAM,WAAW,CAAC;AAExC,OAAO,EAUL,KAAK,WAAW,EACjB,MAAM,YAAY,CAAC;AAYpB,OAAO,EAAE,KAAK,QAAQ,EAA2D,MAAM,YAAY,CAAC;AACpG,OAAO,KAAK,EAKV,WAAW,EAIZ,MAAM,SAAS,CAAC;AAGjB;;;;;;GAMG;AACH,MAAM,MAAM,UAAU,GAAG,IAAI,GAAG,aAAa,GAAG,mBAAmB,GAAG,wBAAwB,CAAC;AAE/F,8DAA8D;AAC9D,MAAM,WAAW,iBAAiB;IAChC,KAAK,EAAE,UAAU,CAAC;IAClB,OAAO,EAAE,OAAO,CAAC;IACjB,WAAW,EAAE,WAAW,CAAC;IACzB,SAAS,EAAE,SAAS,CAAC;IACrB,GAAG,EAAE,MAAM,IAAI,CAAC;IAChB,uFAAuF;IACvF,OAAO,EAAE,MAAM,CAAC;IAChB,0FAA0F;IAC1F,IAAI,EAAE,MAAM,CAAC;IACb;;;;;;OAMG;IACH,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,sGAAsG;IACtG,QAAQ,EAAE,CAAC,QAAQ,EAAE,MAAM,KAAK,WAAW,CAAC;IAC5C;;;;;;;;;;;;;;OAcG;IACH,aAAa,CAAC,EAAE,CAAC,QAAQ,EAAE,MAAM,EAAE,YAAY,EAAE,MAAM,KAAK,UAAU,CAAC;IACvE,QAAQ,CAAC,EAAE,QAAQ,CAAC;IACpB;;;OAGG;IACH,OAAO,CAAC,EAAE,MAAM,CAAC;CAClB;AAED,8FAA8F;AAC9F,MAAM,WAAW,eAAe;IAC9B,OAAO,EAAE,MAAM,EAAE,CAAC;IAClB,SAAS,EAAE,MAAM,EAAE,CAAC;IACpB,MAAM,EAAE,MAAM,EAAE,CAAC;IACjB,aAAa,EAAE,MAAM,EAAE,CAAC;IACxB,MAAM,EAAE,MAAM,EAAE,CAAC;IACjB,8FAA8F;IAC9F,OAAO,EAAE,MAAM,EAAE,CAAC;CACnB;AA2jDD;;;;;;;;;;;;;GAaG;AACH,wBAAgB,YAAY,CAC1B,IAAI,EAAE,iBAAiB,EACvB,MAAM,EAAE,WAAW,EACnB,QAAQ,EAAE,gBAAgB,GACzB,eAAe,CAsFjB"}
1
+ {"version":3,"file":"batch-dispatch.d.ts","sourceRoot":"","sources":["../src/batch-dispatch.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GA6DG;AAkBH,OAAO,EAUL,KAAK,gBAAgB,EAErB,KAAK,SAAS,EACf,MAAM,YAAY,CAAC;AACpB,OAAO,EACL,KAAK,WAAW,EAOjB,MAAM,eAAe,CAAC;AACvB,OAAO,EAAE,KAAK,OAAO,EAAa,MAAM,WAAW,CAAC;AACpD,OAAO,KAAK,EAAE,UAAU,EAAE,MAAM,WAAW,CAAC;AAC5C,OAAO,KAAK,EAAE,MAAM,EAAE,MAAM,WAAW,CAAC;AAExC,OAAO,EAUL,KAAK,WAAW,EACjB,MAAM,YAAY,CAAC;AAYpB,OAAO,EAAE,KAAK,QAAQ,EAA2D,MAAM,YAAY,CAAC;AACpG,OAAO,KAAK,EAGV,oBAAoB,EAIpB,WAAW,EAIZ,MAAM,SAAS,CAAC;AAMjB,YAAY,EAAE,UAAU,EAAE,MAAM,SAAS,CAAC;AAE1C,8DAA8D;AAC9D,MAAM,WAAW,iBAAiB;IAChC,KAAK,EAAE,UAAU,CAAC;IAClB,OAAO,EAAE,OAAO,CAAC;IACjB,WAAW,EAAE,WAAW,CAAC;IACzB,SAAS,EAAE,SAAS,CAAC;IACrB,GAAG,EAAE,MAAM,IAAI,CAAC;IAChB,uFAAuF;IACvF,OAAO,EAAE,MAAM,CAAC;IAChB,0FAA0F;IAC1F,IAAI,EAAE,MAAM,CAAC;IACb;;;;;;OAMG;IACH,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,sGAAsG;IACtG,QAAQ,EAAE,CAAC,QAAQ,EAAE,MAAM,KAAK,WAAW,CAAC;IAC5C;;;;;;;;;;;;;;;;;;;OAmBG;IACH,aAAa,CAAC,EAAE,CAAC,QAAQ,EAAE,MAAM,EAAE,YAAY,EAAE,MAAM,KAAK,oBAAoB,CAAC;IACjF,QAAQ,CAAC,EAAE,QAAQ,CAAC;IACpB;;;OAGG;IACH,OAAO,CAAC,EAAE,MAAM,CAAC;CAClB;AAED,8FAA8F;AAC9F,MAAM,WAAW,eAAe;IAC9B,OAAO,EAAE,MAAM,EAAE,CAAC;IAClB,SAAS,EAAE,MAAM,EAAE,CAAC;IACpB,MAAM,EAAE,MAAM,EAAE,CAAC;IACjB,aAAa,EAAE,MAAM,EAAE,CAAC;IACxB,MAAM,EAAE,MAAM,EAAE,CAAC;IACjB,8FAA8F;IAC9F,OAAO,EAAE,MAAM,EAAE,CAAC;CACnB;AAywCD;;;;;;;;;;;;;;;;;;;;;GAqBG;AACH,wBAAgB,iBAAiB,CAC/B,IAAI,EAAE,iBAAiB,EACvB,MAAM,EAAE,WAAW,EACnB,QAAQ,EAAE,gBAAgB,EAC1B,OAAO,EAAE,MAAM,EACf,GAAG,EAAE,IAAI,GACR;IAAE,OAAO,EAAE,eAAe,GAAG,SAAS,GAAG,WAAW,CAAC;IAAC,UAAU,EAAE,MAAM,CAAC;IAAC,MAAM,CAAC,EAAE,MAAM,CAAA;CAAE,CA+E7F;AAofD;;;;;;;;;;;;;GAaG;AACH,wBAAgB,YAAY,CAC1B,IAAI,EAAE,iBAAiB,EACvB,MAAM,EAAE,WAAW,EACnB,QAAQ,EAAE,gBAAgB,GACzB,eAAe,CAsFjB"}
@@ -95,9 +95,11 @@ var __importStar = (this && this.__importStar) || (function () {
95
95
  };
96
96
  })();
97
97
  Object.defineProperty(exports, "__esModule", { value: true });
98
+ exports.resumeBlockedGate = resumeBlockedGate;
98
99
  exports.runBatchTick = runBatchTick;
99
100
  const fs = __importStar(require("node:fs"));
100
101
  const path = __importStar(require("node:path"));
102
+ const core_1 = require("@ai-dossier/core");
101
103
  const worktree_pool_1 = require("@ai-dossier/worktree-pool");
102
104
  const attribution_1 = require("./attribution");
103
105
  const dispatch_1 = require("./dispatch");
@@ -140,6 +142,65 @@ function sanitizeUntrustedText(value) {
140
142
  function journalEvent(deps, event, unitId, extra = {}) {
141
143
  deps.journal.append((0, journal_1.unitEvent)(event, unitId, extra), deps.now());
142
144
  }
145
+ /**
146
+ * Last ~500 bytes of a gate's output tail, falling back to the envelope's
147
+ * `reason` when no subprocess ran (#583 AC1/AC3 review: `capability-unavailable`
148
+ * and a failed assumption probe carry no output_tail — `reason` is the only
149
+ * explanation available for those) — journal details stay compact; the full
150
+ * tail lives in the per-gate log file. UTF-8-safe (never splits a multi-byte
151
+ * character) — chars would risk exactly that, hence `Buffer`, matching
152
+ * `cli/src/capability.ts`'s `truncateTailBytes`.
153
+ */
154
+ function gateDetailExcerpt(outputTail, reason) {
155
+ const text = outputTail || reason;
156
+ if (!text)
157
+ return undefined;
158
+ const buf = Buffer.from(text, 'utf-8');
159
+ return buf.length > 500 ? buf.subarray(buf.length - 500).toString('utf-8') : text;
160
+ }
161
+ /**
162
+ * Best-effort per-gate diagnostic log (#583 AC1) — mirrors `appendCapLog`'s
163
+ * never-crash contract: a log-write failure must not interrupt the gate
164
+ * decision itself.
165
+ */
166
+ function writeGateLog(deps, batchId, capabilityId, issue, outputTail) {
167
+ if (!outputTail)
168
+ return;
169
+ try {
170
+ const logPath = (0, dispatch_1.batchGateLogPath)(deps.store.runsDir, batchId, capabilityId, issue);
171
+ fs.mkdirSync(path.dirname(logPath), { recursive: true, mode: 0o700 });
172
+ fs.writeFileSync(logPath, outputTail, { mode: 0o600 });
173
+ }
174
+ catch {
175
+ // diagnostic only — never fail the gate decision over a log write
176
+ }
177
+ }
178
+ /**
179
+ * Persist the incremental gate's most recent verdict for a member on
180
+ * `BatchEntry.member_gates` (#583 AC4) — shared by the live gate
181
+ * (`reconcileMemberSlot`) and `sched resume --batch`'s recheck
182
+ * (`resumeBlockedGate`), so a resumed batch's status never shows a stale
183
+ * outcome from before the recheck ran.
184
+ */
185
+ function recordMemberGate(deps, batchId, memberIssue, gate, now) {
186
+ deps.store.withLock((s) => {
187
+ const b = (0, state_1.findBatch)(s, batchId);
188
+ return {
189
+ state: (0, state_1.patchBatch)(s, batchId, {
190
+ member_gates: {
191
+ ...(b?.member_gates ?? {}),
192
+ [String(memberIssue)]: {
193
+ capability: gate.id,
194
+ outcome: gate.outcome,
195
+ output_tail: gate.outputTail ?? null,
196
+ at: now.toISOString(),
197
+ },
198
+ },
199
+ }, now),
200
+ result: undefined,
201
+ };
202
+ });
203
+ }
143
204
  function slotFor(state, batchId) {
144
205
  return state.slots.find((s) => s.unit === unit(batchId));
145
206
  }
@@ -288,7 +349,7 @@ poolNote = '') {
288
349
  }), now);
289
350
  return { ok: true };
290
351
  };
291
- const capOutcome = deps.runCapability?.(worktree, 'worktree.prepare');
352
+ const capOutcome = deps.runCapability?.(worktree, 'worktree.prepare')?.outcome;
292
353
  if (capOutcome === 'ok')
293
354
  return done('cap:worktree.prepare');
294
355
  if (capOutcome === 'task-failed')
@@ -932,6 +993,187 @@ function evictMemberAndContinue(deps, config, dispatch, batchId, batch, memberIs
932
993
  }
933
994
  advanceMemberOrValidate(deps, config, dispatch, batchId, batch.members.length, batch.executing_member, memberIssue, now, result);
934
995
  }
996
+ /**
997
+ * The incremental gate (#523 AC2, revised #583): typecheck + focused tests
998
+ * via `cap run`, when the repo has a manifest for them — a second,
999
+ * independent check that the member's own self-reported "done" is real,
1000
+ * matching this codebase's "never trust a claimed completion" ethos
1001
+ * (AC2/#464's `isVerifiedComplete`). Three-way policy: `task-failed` evicts
1002
+ * the member directly, same rail as a self-reported block (RFC F.1) — no
1003
+ * aggregate suite has run yet, so there is nothing to attribute.
1004
+ * `automation-broken`/`capability-unavailable` — the gate itself could not
1005
+ * reach a verdict — BLOCK the batch instead of silently proceeding (#583: a
1006
+ * script that legitimately could not run its suite must not be read as
1007
+ * either a pass or a real failure). Only both `ok` falls through.
1008
+ *
1009
+ * Returns `true` when the gate already decided the member's fate (evicted or
1010
+ * blocked, both of which `return` from the caller); `false` when no hook is
1011
+ * configured or both checks came back `ok`, meaning the caller should treat
1012
+ * the member as genuinely complete.
1013
+ */
1014
+ function runIncrementalGate(deps, config, dispatch, batchId, batch, memberIssue, now, result) {
1015
+ if (batch.worktree === null || !deps.runCapability)
1016
+ return false;
1017
+ const worktree = batch.worktree;
1018
+ const runCapability = deps.runCapability;
1019
+ const gateResults = ['typecheck.run', 'test.focused'].map((id) => ({
1020
+ id,
1021
+ ...runCapability(worktree, id),
1022
+ }));
1023
+ const gateFailure = gateResults.find((r) => r.outcome === 'task-failed');
1024
+ const gateInconclusive = gateResults.find((r) => r.outcome === 'automation-broken' || r.outcome === 'capability-unavailable');
1025
+ const worstGate = gateFailure ?? gateInconclusive;
1026
+ if (worstGate) {
1027
+ recordMemberGate(deps, batchId, memberIssue, worstGate, now);
1028
+ }
1029
+ if (gateFailure) {
1030
+ const reason = `incremental-gate-failed:${gateFailure.id}`;
1031
+ writeGateLog(deps, batchId, gateFailure.id, memberIssue, gateFailure.outputTail);
1032
+ const excerpt = gateDetailExcerpt(gateFailure.outputTail, gateFailure.reason);
1033
+ journalEvent(deps, 'unit-failed', unit(batchId), {
1034
+ issue: memberIssue,
1035
+ reason,
1036
+ detail: excerpt
1037
+ ? `cap run ${gateFailure.id} reported task-failed after member review done: ${excerpt}`
1038
+ : `cap run ${gateFailure.id} reported task-failed after member review done`,
1039
+ });
1040
+ deps.store.withLock((s) => ({ state: releaseSlot(s, batchId, now), result: undefined }));
1041
+ evictMemberAndContinue(deps, config, dispatch, batchId, batch, memberIssue, reason, now, result);
1042
+ return true;
1043
+ }
1044
+ if (gateInconclusive) {
1045
+ const reason = `gate-inconclusive:${gateInconclusive.id}`;
1046
+ writeGateLog(deps, batchId, gateInconclusive.id, memberIssue, gateInconclusive.outputTail);
1047
+ const excerpt = gateDetailExcerpt(gateInconclusive.outputTail, gateInconclusive.reason);
1048
+ journalEvent(deps, 'gate-inconclusive', unit(batchId), {
1049
+ issue: memberIssue,
1050
+ reason,
1051
+ detail: excerpt
1052
+ ? `cap run ${gateInconclusive.id} reported ${gateInconclusive.outcome} after member review done: ${excerpt}`
1053
+ : `cap run ${gateInconclusive.id} reported ${gateInconclusive.outcome} after member review done`,
1054
+ });
1055
+ deps.store.withLock((s) => ({ state: releaseSlot(s, batchId, now), result: undefined }));
1056
+ const stateNow = deps.store.load();
1057
+ const rDeps = recoveryDeps(deps, config, batch, now);
1058
+ const blocked = (0, recovery_1.blockBatch)(stateNow, batchId, { reason, milestonePhase: 'batch-review' }, rDeps);
1059
+ deps.store.withLock((s) => ({
1060
+ state: applyBatchAndIssues(s, blocked.state, batchId, []),
1061
+ result: undefined,
1062
+ }));
1063
+ result.failed.push(unit(batchId));
1064
+ return true;
1065
+ }
1066
+ return false;
1067
+ }
1068
+ /**
1069
+ * The member is verifiably done — self-reported complete AND (when checked)
1070
+ * the incremental gate agrees. Recompute the member's commit range, mark it
1071
+ * validated, and advance the batch. Shared by `reconcileMemberSlot`'s
1072
+ * immediate "both gate checks ok" fallthrough and `resumeBlockedGate` (#583)
1073
+ * — a member confirmed complete via a delayed `sched resume --batch` recheck
1074
+ * gets exactly the same treatment as one confirmed complete on the first try.
1075
+ * `releaseSlot` is idempotent (a no-op once the slot is already idle), so
1076
+ * this is safe to call whether or not the caller already released it.
1077
+ */
1078
+ function completeMemberGate(deps, config, dispatch, batchId, batch, memberIssue, now, result) {
1079
+ // The commit-range recompute (`git log`) is a blocking subprocess call — it
1080
+ // must run OUTSIDE the lock, like every other exec in this module; the
1081
+ // result then lands as a pure data patch under the lock (Convention review:
1082
+ // `recordRanges` used to run `git log` INSIDE the withLock mutator, which
1083
+ // is exactly what `engine.ts`'s own "a slow git call never holds the lock"
1084
+ // invariant exists to prevent).
1085
+ const ranges = (0, attribution_1.memberRanges)(boundaryCommits(deps, batch));
1086
+ deps.store.withLock((s) => {
1087
+ let n = releaseSlot(s, batchId, now);
1088
+ n = (0, state_1.patchBatch)(n, batchId, { ranges }, now);
1089
+ n = advanceMemberToValidated(n, memberIssue, now);
1090
+ return { state: n, result: undefined };
1091
+ });
1092
+ result.completed.push(unit(batchId));
1093
+ advanceMemberOrValidate(deps, config, dispatch, batchId, batch.members.length, batch.executing_member, memberIssue, now, result);
1094
+ }
1095
+ /**
1096
+ * `sched resume --batch <id>` (#583 AC4): an operator-triggered, synchronous
1097
+ * one-shot recheck of a batch blocked on `gate-inconclusive:<capabilityId>`
1098
+ * — re-runs exactly that capability against the current member and resolves
1099
+ * the block:
1100
+ *
1101
+ * - still `automation-broken`/`capability-unavailable` → stays `blocked`,
1102
+ * no state change (the capability still isn't fixed).
1103
+ * - `task-failed` → the member really is broken; evict via the same rail
1104
+ * the live gate uses.
1105
+ * - `ok` → the member really was fine; complete it via the same rail the
1106
+ * live gate uses.
1107
+ *
1108
+ * Deliberately NOT a generic engine tick: `runBatchTick`'s own "executing
1109
+ * with no live slot" wedge-recovery path (`spawnMemberContinuation`,
1110
+ * unconditional) exists for a dispatch that never happened — reusing it here
1111
+ * would redispatch a fresh agent for a member whose work is already
1112
+ * committed and reviewed. This function transitions `blocked → executing`
1113
+ * and immediately, synchronously, calls the same completion/eviction
1114
+ * functions the live gate calls — no intervening tick ever sees the batch
1115
+ * `executing` with nothing in flight.
1116
+ */
1117
+ function resumeBlockedGate(deps, config, dispatch, batchId, now) {
1118
+ const state = deps.store.load();
1119
+ const batch = (0, state_1.findBatch)(state, batchId);
1120
+ if (!batch)
1121
+ throw new types_1.SchedNotFoundError(`Batch not found: ${batchId}`);
1122
+ if (batch.status !== 'blocked' || !batch.blocked_reason?.startsWith('gate-inconclusive:')) {
1123
+ throw new types_1.IllegalTransitionError('batch', batch.status, 'executing');
1124
+ }
1125
+ const capabilityId = batch.blocked_reason.slice('gate-inconclusive:'.length);
1126
+ const memberIssue = batch.members[batch.executing_member - 1];
1127
+ if (batch.worktree === null || !deps.runCapability || memberIssue === undefined) {
1128
+ throw new types_1.SchedNotFoundError(`Batch ${batchId} has no worktree/current member/gate hook to recheck`);
1129
+ }
1130
+ const recheck = deps.runCapability(batch.worktree, capabilityId);
1131
+ const result = emptyResult();
1132
+ const excerpt = gateDetailExcerpt(recheck.outputTail, recheck.reason);
1133
+ recordMemberGate(deps, batchId, memberIssue, { id: capabilityId, ...recheck }, now);
1134
+ if (recheck.outcome === 'automation-broken' || recheck.outcome === 'capability-unavailable') {
1135
+ // Still inconclusive: leave an audit trail (log + journal) exactly like
1136
+ // the live gate does, even though the batch stays `blocked` — otherwise
1137
+ // a repeated `sched resume --batch` leaves no record of when it was last
1138
+ // checked or what it said (#583 review).
1139
+ writeGateLog(deps, batchId, capabilityId, memberIssue, recheck.outputTail);
1140
+ journalEvent(deps, 'gate-inconclusive', unit(batchId), {
1141
+ issue: memberIssue,
1142
+ reason: `gate-inconclusive:${capabilityId}`,
1143
+ detail: excerpt
1144
+ ? `sched resume --batch: cap run ${capabilityId} still reports ${recheck.outcome} on recheck: ${excerpt}`
1145
+ : `sched resume --batch: cap run ${capabilityId} still reports ${recheck.outcome} on recheck`,
1146
+ });
1147
+ return {
1148
+ outcome: 'still-blocked',
1149
+ capability: capabilityId,
1150
+ detail: excerpt,
1151
+ };
1152
+ }
1153
+ deps.store.withLock((s) => ({
1154
+ state: (0, state_1.patchBatch)((0, state_1.transitionBatch)(s, batchId, 'executing', {}, now), batchId, { blocked_reason: null }, now),
1155
+ result: undefined,
1156
+ }));
1157
+ if (recheck.outcome === 'task-failed') {
1158
+ const reason = `incremental-gate-failed:${capabilityId}`;
1159
+ writeGateLog(deps, batchId, capabilityId, memberIssue, recheck.outputTail);
1160
+ journalEvent(deps, 'unit-failed', unit(batchId), {
1161
+ issue: memberIssue,
1162
+ reason,
1163
+ detail: excerpt
1164
+ ? `sched resume --batch: cap run ${capabilityId} reported task-failed on recheck: ${excerpt}`
1165
+ : `sched resume --batch: cap run ${capabilityId} reported task-failed on recheck`,
1166
+ });
1167
+ evictMemberAndContinue(deps, config, dispatch, batchId, batch, memberIssue, reason, now, result);
1168
+ return { outcome: 'evicted', capability: capabilityId, detail: excerpt };
1169
+ }
1170
+ journalEvent(deps, 'external-advance', unit(batchId), {
1171
+ issue: memberIssue,
1172
+ detail: `sched resume --batch: cap run ${capabilityId} reported ok on recheck`,
1173
+ });
1174
+ completeMemberGate(deps, config, dispatch, batchId, batch, memberIssue, now, result);
1175
+ return { outcome: 'completed', capability: capabilityId };
1176
+ }
935
1177
  /**
936
1178
  * Record one member dispatch's tokens/cost to `runs.jsonl` (#564) — the
937
1179
  * `batch-dispatch.ts` analogue of `engine.ts`'s `recordDispatchRunLog`.
@@ -957,7 +1199,7 @@ function recordMemberRunLog(deps, dispatch, state, batchId, batch, memberIssue,
957
1199
  reason: slot.spawned_at === null ? 'never-spawned' : `already-recorded-${slot.status}`,
958
1200
  slot: slot.id,
959
1201
  });
960
- return;
1202
+ return null;
961
1203
  }
962
1204
  const tier = (0, state_1.findEntry)(state, memberIssue)?.tier ?? 'mid';
963
1205
  const { cmd, model } = (0, dispatch_1.resolveTierSpawn)(dispatch, tier, memberIssue);
@@ -976,6 +1218,9 @@ function recordMemberRunLog(deps, dispatch, state, batchId, batch, memberIssue,
976
1218
  tier,
977
1219
  });
978
1220
  (0, run_log_1.finalizeRunLogEntry)(runEntry, logContent, deps.homeDir, (event, extra) => journalEvent(deps, event, unit(batchId), extra), { issue: memberIssue, log: logFile });
1221
+ // #591: the last tool this member dispatch called — attributes an
1222
+ // `agent-exited-unverified` failure to a concrete cause without opening the transcript.
1223
+ return (0, core_1.parseLastToolUse)(logContent);
979
1224
  }
980
1225
  function reconcileMemberSlot(deps, config, dispatch, batchId, slot, now, result) {
981
1226
  const state0 = deps.store.load();
@@ -1016,46 +1261,15 @@ function reconcileMemberSlot(deps, config, dispatch, batchId, slot, now, result)
1016
1261
  // incremental gate below decides — record its telemetry once here (#564)
1017
1262
  // rather than at each of this branch's two later exit points.
1018
1263
  recordMemberRunLog(deps, dispatch, state0, batchId, batch, memberIssue, slot, now);
1019
- // Incremental gate (#523 AC2): typecheck + focused tests via `cap run`,
1020
- // when the repo has a manifest for them a second, independent check
1021
- // that the member's own self-reported "done" is real, matching this
1022
- // codebase's "never trust a claimed completion" ethos (AC2/#464's
1023
- // `isVerifiedComplete`). `task-failed` evicts the member directly, same
1024
- // rail as a self-reported block (RFC F.1) — no aggregate suite has run
1025
- // yet, so there is nothing to attribute.
1026
- if (batch.worktree !== null && deps.runCapability) {
1027
- const gateFailure = ['typecheck.run', 'test.focused']
1028
- .map((id) => ({ id, outcome: deps.runCapability?.(batch.worktree, id) }))
1029
- .find((r) => r.outcome === 'task-failed');
1030
- if (gateFailure) {
1031
- const reason = `incremental-gate-failed:${gateFailure.id}`;
1032
- journalEvent(deps, 'unit-failed', unit(batchId), {
1033
- issue: memberIssue,
1034
- reason,
1035
- detail: `cap run ${gateFailure.id} reported task-failed after member review done`,
1036
- });
1037
- deps.store.withLock((s) => ({ state: releaseSlot(s, batchId, now), result: undefined }));
1038
- evictMemberAndContinue(deps, config, dispatch, batchId, batch, memberIssue, reason, now, result);
1039
- return;
1040
- }
1041
- // `ok` / `automation-broken` / `capability-unavailable` all proceed —
1042
- // only a definite task failure blocks a member here.
1264
+ // Incremental gate (#523 AC2, revised #583) see `runIncrementalGate`'s
1265
+ // own doc comment for the three-way policy. A `true` return means the
1266
+ // gate already decided the member's fate (evicted or blocked) and
1267
+ // returned early; `false` means both checks were `ok` and the member is
1268
+ // genuinely done.
1269
+ if (runIncrementalGate(deps, config, dispatch, batchId, batch, memberIssue, now, result)) {
1270
+ return;
1043
1271
  }
1044
- // The commit-range recompute (`git log`) is a blocking subprocess call
1045
- // it must run OUTSIDE the lock, like every other exec in this module;
1046
- // the result then lands as a pure data patch under the lock (Convention
1047
- // review: `recordRanges` used to run `git log` INSIDE the withLock
1048
- // mutator, which is exactly what `engine.ts`'s own "a slow git call never
1049
- // holds the lock" invariant exists to prevent).
1050
- const ranges = (0, attribution_1.memberRanges)(boundaryCommits(deps, batch));
1051
- deps.store.withLock((s) => {
1052
- let n = releaseSlot(s, batchId, now);
1053
- n = (0, state_1.patchBatch)(n, batchId, { ranges }, now);
1054
- n = advanceMemberToValidated(n, memberIssue, now);
1055
- return { state: n, result: undefined };
1056
- });
1057
- result.completed.push(unit(batchId));
1058
- advanceMemberOrValidate(deps, config, dispatch, batchId, batch.members.length, batch.executing_member, memberIssue, now, result);
1272
+ completeMemberGate(deps, config, dispatch, batchId, batch, memberIssue, now, result);
1059
1273
  return;
1060
1274
  }
1061
1275
  if ((0, groundtruth_1.isMemberBlocked)(milestone) || dead) {
@@ -1065,11 +1279,17 @@ function reconcileMemberSlot(deps, config, dispatch, batchId, slot, now, result)
1065
1279
  : dead
1066
1280
  ? 'agent-exited-unverified'
1067
1281
  : 'member-blocked';
1068
- recordMemberRunLog(deps, dispatch, state0, batchId, batch, memberIssue, slot, now);
1282
+ const lastTool = recordMemberRunLog(deps, dispatch, state0, batchId, batch, memberIssue, slot, now);
1069
1283
  journalEvent(deps, 'unit-failed', unit(batchId), {
1070
1284
  issue: memberIssue,
1071
1285
  reason,
1072
1286
  detail: 'member blocked',
1287
+ // #591: attributes an unverified member exit to a concrete cause (e.g.
1288
+ // `Monitor`) without opening the transcript. Gated on the RESOLVED reason, not
1289
+ // `dead` — a member can be simultaneously `dead` AND carry a milestone-posted
1290
+ // `reason` (it posted `blocked` and then exited), and that real block has no
1291
+ // log-derived cause to attribute; only `agent-exited-unverified` does.
1292
+ ...(reason === 'agent-exited-unverified' && lastTool !== null ? { last_tool: lastTool } : {}),
1073
1293
  });
1074
1294
  deps.store.withLock((s) => ({ state: releaseSlot(s, batchId, now), result: undefined }));
1075
1295
  // A member that never went green (RFC F.1) evicts DIRECTLY — no aggregate