brl-subagent 2.5.0 → 2.5.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENT.md +9 -2
- package/README.md +6 -3
- package/package.json +6 -6
- package/src/history.ts +19 -9
- package/src/index.ts +206 -35
- package/src/kill-escalation.ts +338 -6
- package/src/metrics.ts +5 -3
- package/src/notify-completion.ts +23 -0
- package/src/proc.ts +270 -20
- package/src/recovery-engine.ts +100 -14
- package/src/recovery-surface.ts +374 -0
- package/src/recovery.ts +19 -1
- package/src/run-registry.ts +64 -3
- package/src/runner.ts +216 -13
- package/src/session-manager.ts +129 -7
- package/src/state.ts +32 -2
- package/src/tui-format.ts +57 -1
- package/src/tui.ts +102 -22
- package/src/types.ts +66 -1
- package/src/unit-run.ts +8 -1
package/AGENT.md
CHANGED
|
@@ -64,12 +64,19 @@ delegation-heavy work.
|
|
|
64
64
|
- Graph with `dependsOn`: subtasks declare edges; the scheduler runs them in waves.
|
|
65
65
|
- Background + act on wake: spawn a background run, then act when the completion message arrives. Never poll between spawn and wake.
|
|
66
66
|
- Background fan-out: `tasks: [...], background: true` — one background agent per task, one id per task in task order, one wake per agent.
|
|
67
|
-
- retryRunId: pass a failed run's id to re-run it with its recorded task and params; explicit parameters override the originals.
|
|
67
|
+
- retryRunId: pass a failed or **interrupted** run's id to re-run it with its recorded task and params; explicit parameters override the originals. A run interrupted before durable task capture (a pre-enrichment registry entry) has no recorded task — pass `task` explicitly for those.
|
|
68
|
+
|
|
69
|
+
## Interrupted runs and recovery
|
|
70
|
+
|
|
71
|
+
- An interrupted run (its conductor died mid-run) stays visible with status **Interrupted** in `/brl-subagent history` and is listed by `/brl-subagent retry` alongside failed runs — including after a crash + fresh boot, where it is reconstructed from the durable per-run registry (plus the background agent record). Recover it with `retryRunId: <run-id>` (recorded params restored), or abandon it from that menu — abandon finalizes the run as stopped, drops its persisted session file, and sends **no** completion wake.
|
|
72
|
+
- **Task persistence (O3):** a foreground/unit run persists its task, label, model, thinking level and full `originalParams` snapshot in the durable registry at dispatch, so after a crash + fresh boot its row shows the real task and `retryRunId` restores the recorded shape — the same 17 fields a same-session retry restores (including `force`; explicit caller values still win). A successful retry retires the source entry so the superseded run is not re-offered at the next boot; a failed retry start leaves it intact and retryable.
|
|
73
|
+
- **Legacy rows:** an entry written before task persistence lacks those fields, so its row carries `(task unavailable)` and `retryRunId` cannot restore a task — pass `task` explicitly on the retry call. A background run keeps its task via its agent record.
|
|
74
|
+
- The **recovery posture** (`/brl-subagent recovery`, default `offer`) controls whether a boot surfaces interrupted runs with one turn-boundary notice; `off` silences it. Nothing is auto-resumed today.
|
|
68
75
|
|
|
69
76
|
## Deadlines and retries
|
|
70
77
|
|
|
71
78
|
- `timeout` (ms) is a **deadline, not a ceiling** — an explicit value is honored as given; 30 minutes is only the default for a background run that passes none. Background expiry aborts the session (the run ends `stopped`); foreground expiry kills the process.
|
|
72
|
-
- `retryOnTimeout` applies to **foreground runs only**. A timed-out background run is not auto-retried: re-dispatch it with `retryRunId` if that is the right call.
|
|
79
|
+
- `retryOnTimeout` applies to **foreground runs only**. A timed-out background run is not auto-retried: re-dispatch it with `retryRunId` if that is the right call. The same applies to an interrupted run — recovery is always an explicit `retryRunId` call, never automatic (auto mode is not yet accepted).
|
|
73
80
|
|
|
74
81
|
## Mechanics pointer
|
|
75
82
|
|
package/README.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
> Multi-agent orchestration for [pi](https://github.com/earendil-works/pi) — chain, parallel, and dependency-graph delegation to isolated subagents with per-step model routing, preset-driven tool scoping, thinking-level control, and background execution with live monitoring, real abort, and per-agent timeouts.
|
|
4
4
|
|
|
5
|
-
**Version:** 2.5.
|
|
5
|
+
**Version:** 2.5.2 · **Author:** Azmeen Afandi / [Beeroo Labs](https://beeroolabs.com) · **License:** MIT
|
|
6
6
|
|
|
7
7
|
---
|
|
8
8
|
|
|
@@ -14,7 +14,7 @@
|
|
|
14
14
|
- **True background execution** — live monitor, real abort (`stop_subagent`), per-agent timeouts, and a 30-minute default deadline for runs that don't set one. Nothing orphans; nothing leaks.
|
|
15
15
|
- **Preset-driven tool scoping** — every subagent runs with exactly the tools its job needs, restricted by preset or per-call `tools`/`excludeTools`, with auto-route that picks the right preset when you don't.
|
|
16
16
|
- **Templates with slots** — saved, file-backed task templates with `${param}` placeholders for workflows you run again and again.
|
|
17
|
-
- **Safety by default** — task-fence injection protection, sanitized error paths, owner-only persistence, and a
|
|
17
|
+
- **Safety by default** — task-fence injection protection, sanitized error paths, owner-only persistence, and a 1500+ test suite pinning every contract against the real pi SDK.
|
|
18
18
|
|
|
19
19
|
---
|
|
20
20
|
|
|
@@ -50,6 +50,7 @@ pi remove npm:brl-subagent # uninstall
|
|
|
50
50
|
| `/brl-subagent depth` | Set max recursion depth |
|
|
51
51
|
| `/brl-subagent approval` | Set change approval mode |
|
|
52
52
|
| `/brl-subagent completionnotify` | Set completion-push wake (all/failed/off) |
|
|
53
|
+
| `/brl-subagent recovery` | Set recovery posture for interrupted runs (offer/off) |
|
|
53
54
|
| `/brl-subagent costlimit` | Set session cost limit |
|
|
54
55
|
| `/brl-subagent historyentries` | Set max history entries |
|
|
55
56
|
| `/brl-subagent sla` | Configure SLA tracking |
|
|
@@ -104,13 +105,15 @@ All settings persist across sessions.
|
|
|
104
105
|
| `gitMode` | string | — | `'branch'` creates a work branch, captures the diff, and switches back; `'none'` does nothing. Falls back to the configured default. |
|
|
105
106
|
| `approvalMode` | string | — | `auto` / `writes` / `always`. Default is user config (`/brl-subagent approval`). `always` is rejected for background runs. |
|
|
106
107
|
| `force` | boolean | `false` | Dispatch anyway past a capability pre-flight block; the block is downgraded to a warning. |
|
|
107
|
-
| `retryRunId` | string | — | Re-run a previously failed run with its recorded task and params. |
|
|
108
|
+
| `retryRunId` | string | — | Re-run a previously failed or interrupted run with its recorded task and params. |
|
|
108
109
|
| `retryOnTimeout` | boolean | `false` | Retry once if a **foreground** subagent times out. The second timeout is a final failure. Background runs are not auto-retried — re-dispatch with `retryRunId`. |
|
|
109
110
|
|
|
110
111
|
**Capability pre-flight:** before spawning, the extension checks that the resolved toolset can actually do the task — a run/execute/test/compile/benchmark task with no `bash`, or an exploration task (search/grep/find/list/locate/glob) with none of `find`/`ls`/`grep`/`bash`. Such a dispatch is **rejected** by default; `force: true` downgrades the rejection to a **warning**. `force` never suppresses an `outputFile`-without-write conflict — that stays a hard error. Warnings are surfaced in the returned result in every mode.
|
|
111
112
|
|
|
112
113
|
**Retries (`retryRunId`):** a retry rebuilds the parameter object from a fixed set of fields, falling back to the original run's recorded values, with explicit values on the retry call winning: `task`, `label`, `model`, `preset`, `systemPrompt`, `inheritSystemPrompt`, `thinkingLevel`, `priority`, `outputFile`, `timeout`, `cwd`, `tools`, `excludeTools`, `noBuiltinTools`, plus the four **execution-shape** fields `background`, `gitMode`, `approvalMode` and `force`.
|
|
113
114
|
|
|
115
|
+
**Interrupted runs:** a run whose conductor died mid-run is marked interrupted (it keeps its persisted session file) and is surfaced as **Interrupted** in `/brl-subagent history` and in `/brl-subagent retry` — including after a crash + fresh boot, when it is reconstructed from the durable per-run registry (plus the background agent record). Recover one with `retryRunId: <run-id>`, or abandon it from the retry menu — abandon finalizes it as stopped, prunes its session file, and sends no completion wake. A **foreground** run's task is not persisted, so its crashed row shows no task and a retry needs the task supplied explicitly; a background run keeps its task in its agent record. The recovery posture (`/brl-subagent recovery`, default `offer`) controls whether a boot announces interrupted runs with one turn-boundary notice; `off` silences it. Recovery is never automatic — there is no `auto` mode.
|
|
116
|
+
|
|
114
117
|
**The execution-shape fields are restored, so a retry keeps the original's shape:** retrying a background run re-runs it in the background (with its completion wake), a `gitMode: 'branch'` retry keeps its work branch, an `approvalMode`-constrained retry keeps that gating, and a retry that was capability-blocked keeps its `force`. Passing any of them explicitly on the retry call overrides the recorded value. One qualification on `gitMode`: when neither the record nor the call carries one, the **configured** default still applies — so a user with a global `gitMode: 'branch'` gets a work branch on every retry, and one with the default `'none'` gets none. Do not assume "no work branch" without checking the effective `gitMode`.
|
|
115
118
|
|
|
116
119
|
**Fan-out origins are the exception.** Parallel/chain/graph units are recorded per unit, and those records deliberately snapshot **no** `background`, so retrying a unit is a **single run in the foreground**. Pass `background: true` explicitly if you want a background retry.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "brl-subagent",
|
|
3
|
-
"version": "2.5.
|
|
3
|
+
"version": "2.5.2",
|
|
4
4
|
"description": "Subagent delegation for pi — blocking or background runs, chain/parallel/graph modes, presets and task templates, live monitor, and dispatch capability guards.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"pi-package"
|
|
@@ -37,12 +37,12 @@
|
|
|
37
37
|
"typebox": "*"
|
|
38
38
|
},
|
|
39
39
|
"devDependencies": {
|
|
40
|
-
"@earendil-works/pi-agent-core": "^1.
|
|
41
|
-
"@earendil-works/pi-ai": "^1.
|
|
42
|
-
"@earendil-works/pi-coding-agent": "^1.
|
|
43
|
-
"@earendil-works/pi-tui": "^1.
|
|
40
|
+
"@earendil-works/pi-agent-core": "^1.3.0",
|
|
41
|
+
"@earendil-works/pi-ai": "^1.3.0",
|
|
42
|
+
"@earendil-works/pi-coding-agent": "^1.3.0",
|
|
43
|
+
"@earendil-works/pi-tui": "^1.3.0",
|
|
44
44
|
"jiti": "^2.7.0",
|
|
45
|
-
"typebox": "
|
|
45
|
+
"typebox": "1.3.27",
|
|
46
46
|
"typescript": "7.0.2",
|
|
47
47
|
"vitest": "^5.0.0"
|
|
48
48
|
},
|
package/src/history.ts
CHANGED
|
@@ -16,23 +16,33 @@ import type { ExtensionContext, SessionManager } from "@earendil-works/pi-coding
|
|
|
16
16
|
// Run pruning — R2: Disk usage policy
|
|
17
17
|
// ---------------------------------------------------------------------------
|
|
18
18
|
|
|
19
|
+
/**
|
|
20
|
+
* Order run records NEWEST-FIRST by `startedAt`, without mutating the input.
|
|
21
|
+
*
|
|
22
|
+
* The ONE ordering authority for run records (issue #330): sorts by
|
|
23
|
+
* `startedAt` descending using `Date.parse` comparisons and returns a new
|
|
24
|
+
* array. `Array.prototype.sort` is stable, so records sharing a `startedAt`
|
|
25
|
+
* keep their relative order (spawn-before-terminal for a finalized run).
|
|
26
|
+
*/
|
|
27
|
+
export function sortNewestFirst(runs: SubagentRun[]): SubagentRun[] {
|
|
28
|
+
return [...runs].sort(
|
|
29
|
+
(a, b) => Date.parse(b.startedAt) - Date.parse(a.startedAt),
|
|
30
|
+
);
|
|
31
|
+
}
|
|
32
|
+
|
|
19
33
|
/**
|
|
20
34
|
* Prune a SubagentRun array to keep only the newest entries up to maxEntries.
|
|
21
|
-
*
|
|
22
|
-
*
|
|
35
|
+
* The result is always ordered NEWEST-FIRST by `startedAt` — including the
|
|
36
|
+
* `maxEntries === 0` unlimited path, which keeps every entry but still sorts.
|
|
37
|
+
* Returns a new array; does not mutate the input.
|
|
23
38
|
*/
|
|
24
39
|
export function cleanupRuns(
|
|
25
40
|
runs: SubagentRun[],
|
|
26
41
|
maxEntries: number = MAX_RUN_HISTORY_ENTRIES,
|
|
27
42
|
): SubagentRun[] {
|
|
28
43
|
if (runs.length === 0) return [];
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
// Always sort newest-first for consistency, even when not truncating
|
|
32
|
-
const sorted = [...runs].sort(
|
|
33
|
-
(a, b) => new Date(b.startedAt).getTime() - new Date(a.startedAt).getTime(),
|
|
34
|
-
);
|
|
35
|
-
|
|
44
|
+
const sorted = sortNewestFirst(runs);
|
|
45
|
+
if (maxEntries === 0) return sorted;
|
|
36
46
|
if (sorted.length <= maxEntries) return sorted;
|
|
37
47
|
return sorted.slice(0, maxEntries);
|
|
38
48
|
}
|
package/src/index.ts
CHANGED
|
@@ -51,6 +51,7 @@ import {
|
|
|
51
51
|
DEFAULT_PRIORITY,
|
|
52
52
|
type GitMode,
|
|
53
53
|
type BackgroundAgent,
|
|
54
|
+
type RecoveryMode,
|
|
54
55
|
} from "./types";
|
|
55
56
|
import { validateGraph, topologicalSort } from "./scheduler";
|
|
56
57
|
import { resolveTemplate, loadAllTemplates, loadBuiltinTemplates, validateTemplatePresetRefs, extractParamNames } from "./templates";
|
|
@@ -74,7 +75,7 @@ import { modelIsAvailable } from "./model-availability";
|
|
|
74
75
|
import { validatePreTask, diagnoseFailure, DEFAULT_BACKGROUND_DEADLINE_MS } from "./validate";
|
|
75
76
|
import { findUnknownParams, KNOWN_DELEGATE_KEYS, resolveSubagentParams, snapshotOriginalParams } from "./params";
|
|
76
77
|
import { delegateTaskParamsSchema } from "./schema";
|
|
77
|
-
import { createSessionState } from "./state";
|
|
78
|
+
import { createSessionState, resolveTerminalRunEntry } from "./state";
|
|
78
79
|
import { makeLiveOnUpdate, createUnitRun, finalizeUnitRun, finalizeUnitRunCrash, pruneHistoryIfNeeded, registerLiveRun } from "./unit-run";
|
|
79
80
|
import { buildSubagentPrompt, describePromptMode } from "./prompt";
|
|
80
81
|
import { runSubagent, cleanupTempDirs, reapActiveChildren } from "./runner";
|
|
@@ -83,7 +84,7 @@ import {
|
|
|
83
84
|
currentProcessOwner,
|
|
84
85
|
newDispatchIdentity,
|
|
85
86
|
} from "./recovery";
|
|
86
|
-
import { markInterrupted } from "./run-registry";
|
|
87
|
+
import { markInterrupted, listInflightRuns, clearInflightRun } from "./run-registry";
|
|
87
88
|
import { acquireSlot, releaseSlot, updateStatus, updateProgressStatus } from "./concurrency";
|
|
88
89
|
import { transcriptDisplayPath } from "./transcript-path";
|
|
89
90
|
import {
|
|
@@ -103,6 +104,7 @@ import {
|
|
|
103
104
|
showApprovalModeSelector,
|
|
104
105
|
showApprovalDialog,
|
|
105
106
|
showCompletionNotifySelector,
|
|
107
|
+
showRecoverySelector,
|
|
106
108
|
showPresetManager,
|
|
107
109
|
showTemplateManager,
|
|
108
110
|
showSLAConfig,
|
|
@@ -117,13 +119,22 @@ import {
|
|
|
117
119
|
renderDelegateResult,
|
|
118
120
|
} from "./tui";
|
|
119
121
|
import { createLogger, setLogCwd, type Logger } from "./logging";
|
|
122
|
+
import {
|
|
123
|
+
collectInterruptedRuns,
|
|
124
|
+
offeredInterruptedRuns,
|
|
125
|
+
buildRecoveryNotice,
|
|
126
|
+
resolveRecoveryNoticeDelivery,
|
|
127
|
+
abandonInterruptedRun,
|
|
128
|
+
visibleRunEntriesForSession,
|
|
129
|
+
type RecoveryAgentLookup,
|
|
130
|
+
} from "./recovery-surface";
|
|
120
131
|
import { Intercom } from "./messaging";
|
|
121
132
|
import * as eventBus from "./event-bus";
|
|
122
133
|
import {
|
|
123
134
|
buildCompletionMessage,
|
|
124
135
|
resolveDelivery,
|
|
125
136
|
sendCompletionNotification,
|
|
126
|
-
|
|
137
|
+
claimTerminalNotice,
|
|
127
138
|
resolveRunEntry,
|
|
128
139
|
} from "./notify-completion";
|
|
129
140
|
|
|
@@ -197,6 +208,12 @@ export default function (pi: ExtensionAPI) {
|
|
|
197
208
|
// receives no ctx, so capture it in session_start where it is available.
|
|
198
209
|
let sessionCtx: ExtensionContext | undefined;
|
|
199
210
|
|
|
211
|
+
// Option B U3 (D2): "one recovery notice per boot". The boot scan may be
|
|
212
|
+
// followed by more session_starts in the same process (reload); without this
|
|
213
|
+
// guard every one would re-list the same interrupted runs. Reset on shutdown
|
|
214
|
+
// so a genuinely new session notifies again.
|
|
215
|
+
let recoveryNoticeDelivered = false;
|
|
216
|
+
|
|
200
217
|
// -------------------------------------------------------------------
|
|
201
218
|
// Config change callback
|
|
202
219
|
// -------------------------------------------------------------------
|
|
@@ -707,7 +724,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
707
724
|
// so sweepStaleLiveSubagents finds a 'running' record for every
|
|
708
725
|
// foreground live entry (the #130 invariant gap that swept
|
|
709
726
|
// graph/chain entries immediately).
|
|
710
|
-
const { runId, run } = createUnitRun(merged, stepModel, chainPriority, globalParams.resolvedPreset?.name);
|
|
727
|
+
const { runId, run } = createUnitRun(merged, stepModel, chainPriority, globalParams.resolvedPreset?.name, state.config.recovery);
|
|
711
728
|
state.persistRun(pi, run, "unit");
|
|
712
729
|
|
|
713
730
|
// R2: Prune old run entries if history exceeds limit
|
|
@@ -1162,7 +1179,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
1162
1179
|
// Issue #119: per-subtask run entry — created at spawn so the
|
|
1163
1180
|
// monitor drill-in shows per-unit priority and a post-hoc audit
|
|
1164
1181
|
// trail exists per subtask (mirrors single-mode run persistence).
|
|
1165
|
-
const { runId, run } = createUnitRun(merged, stepModel, parallelPriority, globalParams.resolvedPreset?.name);
|
|
1182
|
+
const { runId, run } = createUnitRun(merged, stepModel, parallelPriority, globalParams.resolvedPreset?.name, state.config.recovery);
|
|
1166
1183
|
state.persistRun(pi, run, "unit");
|
|
1167
1184
|
|
|
1168
1185
|
// R2: Prune old run entries if history exceeds limit
|
|
@@ -1799,7 +1816,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
1799
1816
|
// so sweepStaleLiveSubagents finds a 'running' record for every
|
|
1800
1817
|
// foreground live entry (the #130 invariant gap that swept
|
|
1801
1818
|
// graph/chain entries immediately).
|
|
1802
|
-
const { runId, run } = createUnitRun(merged, stepModel, graphPriority, globalParams.resolvedPreset?.name);
|
|
1819
|
+
const { runId, run } = createUnitRun(merged, stepModel, graphPriority, globalParams.resolvedPreset?.name, state.config.recovery);
|
|
1803
1820
|
state.persistRun(pi, run, "unit");
|
|
1804
1821
|
|
|
1805
1822
|
// R2: Prune old run entries if history exceeds limit
|
|
@@ -2210,6 +2227,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
2210
2227
|
preset: globalParams.resolvedPreset?.name,
|
|
2211
2228
|
}),
|
|
2212
2229
|
sanitizeCwd: merged.effectiveCwd,
|
|
2230
|
+
recoveryPosture: state.config.recovery,
|
|
2213
2231
|
});
|
|
2214
2232
|
} catch (err) {
|
|
2215
2233
|
const message = sanitizeErrorMessage(
|
|
@@ -2280,6 +2298,8 @@ export default function (pi: ExtensionAPI) {
|
|
|
2280
2298
|
dispatchId?: string;
|
|
2281
2299
|
resumeOf?: string;
|
|
2282
2300
|
attempt?: number;
|
|
2301
|
+
// Option B U3 (D4): recovery posture snapshot for the background run.
|
|
2302
|
+
recoveryPosture?: RecoveryMode;
|
|
2283
2303
|
},
|
|
2284
2304
|
): Promise<BackgroundAgent> {
|
|
2285
2305
|
const { spawnBackgroundSession, setAgentFinalOutput, extractFinalOutput, updateAgentStatus } = await import('./session-manager');
|
|
@@ -2310,8 +2330,24 @@ export default function (pi: ExtensionAPI) {
|
|
|
2310
2330
|
dispatchId: spawn.dispatchId,
|
|
2311
2331
|
resumeOf: spawn.resumeOf,
|
|
2312
2332
|
attempt: spawn.attempt,
|
|
2333
|
+
// Option B U3 (D4): recovery posture snapshot for the background run.
|
|
2334
|
+
recoveryPosture: spawn.recoveryPosture,
|
|
2313
2335
|
});
|
|
2314
2336
|
|
|
2337
|
+
// Issue #318 (from #316 review): the two poller crash sites share one
|
|
2338
|
+
// claimed notification envelope. Content is the only variable; the
|
|
2339
|
+
// claim guard, display, details and deliverAs live here once.
|
|
2340
|
+
const sendCrashNotice = (content: string): void => {
|
|
2341
|
+
if (claimTerminalNotice(agent.id)) {
|
|
2342
|
+
pi.sendMessage({
|
|
2343
|
+
customType: "subagent-notification",
|
|
2344
|
+
content,
|
|
2345
|
+
display: true,
|
|
2346
|
+
details: { agentId: agent.id }
|
|
2347
|
+
}, { deliverAs: "followUp" });
|
|
2348
|
+
}
|
|
2349
|
+
};
|
|
2350
|
+
|
|
2315
2351
|
// Register for live monitor
|
|
2316
2352
|
state.registerLiveSubagent(agent.id, {
|
|
2317
2353
|
id: agent.id,
|
|
@@ -2359,12 +2395,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
2359
2395
|
}
|
|
2360
2396
|
state.failedSubagents++;
|
|
2361
2397
|
updateProgressStatus(state, ctx);
|
|
2362
|
-
|
|
2363
|
-
customType: "subagent-notification",
|
|
2364
|
-
content: `Background agent "${agent.description}" crashed.`,
|
|
2365
|
-
display: true,
|
|
2366
|
-
details: { agentId: agent.id }
|
|
2367
|
-
}, { deliverAs: "followUp" });
|
|
2398
|
+
sendCrashNotice(`Background agent "${agent.description}" crashed.`);
|
|
2368
2399
|
return;
|
|
2369
2400
|
}
|
|
2370
2401
|
|
|
@@ -2411,12 +2442,10 @@ export default function (pi: ExtensionAPI) {
|
|
|
2411
2442
|
if (agent.status === 'failed') {
|
|
2412
2443
|
state.failedSubagents++;
|
|
2413
2444
|
updateProgressStatus(state, ctx);
|
|
2414
|
-
|
|
2415
|
-
|
|
2416
|
-
|
|
2417
|
-
|
|
2418
|
-
details: { agentId: agent.id }
|
|
2419
|
-
}, { deliverAs: "followUp" });
|
|
2445
|
+
// Issue #315: the terminal wake (`subagent-completion`, emitted by
|
|
2446
|
+
// the session-manager settle) is the ONE notice for a failed run.
|
|
2447
|
+
// The status here was set by that settle, so the wake already
|
|
2448
|
+
// delivered — this legacy poller notice would duplicate it.
|
|
2420
2449
|
} else if (agent.status === 'stopped') {
|
|
2421
2450
|
// User-initiated stop (stop_subagent) or deadline abort
|
|
2422
2451
|
// (timeout/deadline) — not a failure.
|
|
@@ -2444,12 +2473,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
2444
2473
|
}
|
|
2445
2474
|
state.failedSubagents++;
|
|
2446
2475
|
updateProgressStatus(state, ctx);
|
|
2447
|
-
|
|
2448
|
-
customType: "subagent-notification",
|
|
2449
|
-
content: `Background agent "${agent.description}" crashed: ${sanitizeErrorMessage((err as Error).message, spawn.sanitizeCwd)}`,
|
|
2450
|
-
display: true,
|
|
2451
|
-
details: { agentId: agent.id }
|
|
2452
|
-
}, { deliverAs: "followUp" });
|
|
2476
|
+
sendCrashNotice(`Background agent "${agent.description}" crashed: ${sanitizeErrorMessage((err as Error).message, spawn.sanitizeCwd)}`);
|
|
2453
2477
|
}
|
|
2454
2478
|
}
|
|
2455
2479
|
}, 2000);
|
|
@@ -2646,8 +2670,15 @@ export default function (pi: ExtensionAPI) {
|
|
|
2646
2670
|
dispatchId: bgIdentity.dispatchId,
|
|
2647
2671
|
resumeOf: bgIdentity.resumeOf,
|
|
2648
2672
|
attempt: bgIdentity.attempt,
|
|
2673
|
+
recoveryPosture: state.config.recovery,
|
|
2649
2674
|
});
|
|
2650
2675
|
|
|
2676
|
+
// Retire-on-retry (O3 fold, decision 2026-10-11): the source interrupted
|
|
2677
|
+
// registry entry is superseded once the new background run is actually
|
|
2678
|
+
// spawned (a throwing spawn is caught below and leaves the source intact
|
|
2679
|
+
// and retryable).
|
|
2680
|
+
if (retrySourceRun) clearInflightRun(retrySourceRun.id);
|
|
2681
|
+
|
|
2651
2682
|
// B2: Surface auto-route decisions in background spawn result too —
|
|
2652
2683
|
// preset name + matched-keyword evidence + restriction, derived from
|
|
2653
2684
|
// the RESOLVED toolOptions (reflects per-call overrides).
|
|
@@ -2692,7 +2723,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
2692
2723
|
description: "Configure subagent model and thinking level",
|
|
2693
2724
|
getArgumentCompletions: (prefix: string) => {
|
|
2694
2725
|
const options = [
|
|
2695
|
-
"history", "historyentries", "monitor", "dashboard", "preset", "templates", "retry", "completionnotify",
|
|
2726
|
+
"history", "historyentries", "monitor", "dashboard", "preset", "templates", "retry", "completionnotify", "recovery",
|
|
2696
2727
|
];
|
|
2697
2728
|
const filtered = options.filter((o) => o.startsWith(prefix));
|
|
2698
2729
|
return filtered.length > 0
|
|
@@ -2709,6 +2740,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
2709
2740
|
depth: () => showDepthInput(ctx, state, applyConfig),
|
|
2710
2741
|
approval: () => showApprovalModeSelector(ctx, state, applyConfig),
|
|
2711
2742
|
completionnotify: () => showCompletionNotifySelector(ctx, state, applyConfig),
|
|
2743
|
+
recovery: () => showRecoverySelector(ctx, state, applyConfig),
|
|
2712
2744
|
costlimit: () => showCostLimitInput(ctx, state, applyConfig),
|
|
2713
2745
|
reset: () => resetState(ctx),
|
|
2714
2746
|
history: () => showRunHistory(ctx, state, () => state.persistState(pi)),
|
|
@@ -2717,7 +2749,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
2717
2749
|
dashboard: () => showDashboard(ctx, state),
|
|
2718
2750
|
preset: () => showPresetManager(ctx, state),
|
|
2719
2751
|
templates: () => showTemplateManager(ctx, state),
|
|
2720
|
-
retry: () => showRetryMenu(ctx, state),
|
|
2752
|
+
retry: () => showRetryMenu(ctx, state, (id) => abandonProductionRun(ctx, id)),
|
|
2721
2753
|
sla: () => showSLAConfig(ctx, state, applyConfig),
|
|
2722
2754
|
"sla-stats": () => showSLAStats(ctx, state),
|
|
2723
2755
|
// E5: the compliance report menu (file access / secrets exposure /
|
|
@@ -2922,7 +2954,18 @@ export default function (pi: ExtensionAPI) {
|
|
|
2922
2954
|
// resolving templates first keeps a combined template + retryRunId call
|
|
2923
2955
|
// from silently disabling template resolution (issue #175 review).
|
|
2924
2956
|
if (params.retryRunId) {
|
|
2925
|
-
|
|
2957
|
+
// Fix A: prefer the session spawn entry (it carries `originalParams`);
|
|
2958
|
+
// a run that exists ONLY in the durable registry after a fresh boot
|
|
2959
|
+
// resolves through the visibility bridge instead of being rejected as
|
|
2960
|
+
// not found — the id the boot notice names must resolve here.
|
|
2961
|
+
let runEntry = state.findSpawnRunById(ctx, params.retryRunId);
|
|
2962
|
+
if (!runEntry) {
|
|
2963
|
+
const { getAgent } = await import("./session-manager");
|
|
2964
|
+
runEntry = resolveTerminalRunEntry(
|
|
2965
|
+
visibleRunsFor(ctx, getAgent),
|
|
2966
|
+
params.retryRunId,
|
|
2967
|
+
);
|
|
2968
|
+
}
|
|
2926
2969
|
if (runEntry) {
|
|
2927
2970
|
retrySourceRun = runEntry;
|
|
2928
2971
|
params = resolveRetryParams(params, runEntry) as DelegateTaskParams;
|
|
@@ -3212,6 +3255,8 @@ export default function (pi: ExtensionAPI) {
|
|
|
3212
3255
|
attempt: identity.attempt,
|
|
3213
3256
|
owner: currentProcessOwner(),
|
|
3214
3257
|
childMarker: crypto.randomUUID(),
|
|
3258
|
+
// U3 (D4): the session recovery posture at dispatch.
|
|
3259
|
+
recoveryPosture: state.config.recovery,
|
|
3215
3260
|
};
|
|
3216
3261
|
state.persistRun(pi, run, "foreground");
|
|
3217
3262
|
|
|
@@ -3286,9 +3331,40 @@ export default function (pi: ExtensionAPI) {
|
|
|
3286
3331
|
}
|
|
3287
3332
|
};
|
|
3288
3333
|
|
|
3334
|
+
// #340: both pre-start gate rejections below must finalize the
|
|
3335
|
+
// `running` record persisted just above. Without this the session entry
|
|
3336
|
+
// stays non-terminal, the inflight registry row (written by
|
|
3337
|
+
// `persistRun(..., "foreground")`) is never cleared, and the live
|
|
3338
|
+
// registration is never released — a phantom interrupted run at the
|
|
3339
|
+
// next boot (and, on a retry, a dangling twin beside the source).
|
|
3340
|
+
// The fabricated result drives `finalizeRunRecord` to terminal
|
|
3341
|
+
// `failed`; `stopReason: "aborted"` makes `coherentFailureReason` keep
|
|
3342
|
+
// the D1 `aborted` category coherent. The exact gate message the caller
|
|
3343
|
+
// receives is recorded as `errorMessage`; the terminal
|
|
3344
|
+
// `state.persistRun` (no kind) clears the registry row via its
|
|
3345
|
+
// terminal-clear rule. Returned content/isError and the retry-source
|
|
3346
|
+
// retention below are unchanged.
|
|
3347
|
+
const SLOT_CANCEL_MESSAGE = "Subagent cancelled while waiting for concurrency slot.";
|
|
3348
|
+
const finalizePreStartRejection = (errorMessage: string): void => {
|
|
3349
|
+
const rejection: SubagentResult = {
|
|
3350
|
+
messages: [],
|
|
3351
|
+
usage: { ...EMPTY_USAGE },
|
|
3352
|
+
exitCode: 1,
|
|
3353
|
+
stderr: "",
|
|
3354
|
+
stopReason: "aborted",
|
|
3355
|
+
errorMessage,
|
|
3356
|
+
errorCategory: "aborted",
|
|
3357
|
+
};
|
|
3358
|
+
finalizeRunRecord(run, rejection, "", new Date(run.startedAt).getTime());
|
|
3359
|
+
run.originalParams = { ...run.originalParams, errorCategory: rejection.errorCategory };
|
|
3360
|
+
state.persistRun(pi, run);
|
|
3361
|
+
state.finalizeLiveSubagent(run.id);
|
|
3362
|
+
};
|
|
3363
|
+
|
|
3289
3364
|
// R1: Circuit breaker check — reject if circuit is open
|
|
3290
3365
|
const circuitCheck = state.checkCircuit();
|
|
3291
3366
|
if (circuitCheck.isOpen) {
|
|
3367
|
+
finalizePreStartRejection(circuitCheck.message!);
|
|
3292
3368
|
cleanupGitBranch();
|
|
3293
3369
|
return {
|
|
3294
3370
|
content: [{ type: "text" as const, text: circuitCheck.message! }],
|
|
@@ -3307,12 +3383,13 @@ export default function (pi: ExtensionAPI) {
|
|
|
3307
3383
|
// Acquire concurrency slot
|
|
3308
3384
|
const acquired = await acquireSlot(state, ctx, signal, singlePriority);
|
|
3309
3385
|
if (!acquired) {
|
|
3386
|
+
finalizePreStartRejection(SLOT_CANCEL_MESSAGE);
|
|
3310
3387
|
cleanupGitBranch();
|
|
3311
3388
|
return {
|
|
3312
3389
|
content: [
|
|
3313
3390
|
{
|
|
3314
3391
|
type: "text" as const,
|
|
3315
|
-
text:
|
|
3392
|
+
text: SLOT_CANCEL_MESSAGE,
|
|
3316
3393
|
},
|
|
3317
3394
|
],
|
|
3318
3395
|
details: undefined,
|
|
@@ -3322,6 +3399,16 @@ export default function (pi: ExtensionAPI) {
|
|
|
3322
3399
|
|
|
3323
3400
|
let success = false;
|
|
3324
3401
|
|
|
3402
|
+
// Retire-on-retry (O3 fold, decision 2026-10-11): the retry has passed
|
|
3403
|
+
// every pre-spawn gate and is about to start, so the SOURCE interrupted
|
|
3404
|
+
// registry entry is superseded — clear it here so the old run is not
|
|
3405
|
+
// re-offered at every subsequent boot. A failed pre-start (circuit open,
|
|
3406
|
+
// slot cancelled) returns ABOVE this point: the source is left intact
|
|
3407
|
+
// and retryable, while the NEW twin has already been finalized terminal
|
|
3408
|
+
// by the gate (#340) — nothing dangles. `retrySourceRun` is set only on
|
|
3409
|
+
// a `retryRunId` call.
|
|
3410
|
+
if (retrySourceRun) clearInflightRun(retrySourceRun.id);
|
|
3411
|
+
|
|
3325
3412
|
// F7 (issue #65): import BEFORE the try — the crash-path catch calls
|
|
3326
3413
|
// completeTranscript, and block-scoped consts declared inside the try
|
|
3327
3414
|
// are invisible in its catch (pre-fix, a crash here threw
|
|
@@ -3930,8 +4017,9 @@ export default function (pi: ExtensionAPI) {
|
|
|
3930
4017
|
// parse (only the one agent record a background entry names is read). For
|
|
3931
4018
|
// each in-flight run whose owner is dead, its orphaned marker-verified child
|
|
3932
4019
|
// is reaped and the record is marked: the registry entry always, plus the
|
|
3933
|
-
// agent record for background runs. SILENT in the UI —
|
|
3934
|
-
//
|
|
4020
|
+
// agent record for background runs. The scan itself is SILENT in the UI —
|
|
4021
|
+
// U3's `maybeSendRecoveryNotice` (below) is the one notice surface.
|
|
4022
|
+
// Platforms without /proc skip the whole scan (one log line):
|
|
3935
4023
|
// pid-only liveness cannot rule out pid reuse, so nothing is killed or
|
|
3936
4024
|
// marked unverified.
|
|
3937
4025
|
async function performBootRecovery(): Promise<void> {
|
|
@@ -3964,6 +4052,83 @@ export default function (pi: ExtensionAPI) {
|
|
|
3964
4052
|
}
|
|
3965
4053
|
}
|
|
3966
4054
|
|
|
4055
|
+
/**
|
|
4056
|
+
* Option B U3 Fix A: the recovery visibility bridge, wired for the two
|
|
4057
|
+
* extension consumers that resolve a run id (the retryRunId branch and the
|
|
4058
|
+
* abandon action). Session run entries UNIONED with synthetic durable
|
|
4059
|
+
* interrupted records the session has never seen (a genuine fresh boot), so
|
|
4060
|
+
* an id named by the boot notice resolves here instead of being rejected as
|
|
4061
|
+
* not found. `getAgent` is injected because the callers already load it
|
|
4062
|
+
* dynamically — this keeps the registry/agent reads in ONE place.
|
|
4063
|
+
*/
|
|
4064
|
+
function visibleRunsFor(
|
|
4065
|
+
ctx: ExtensionContext,
|
|
4066
|
+
getAgent: RecoveryAgentLookup,
|
|
4067
|
+
): SubagentRun[] {
|
|
4068
|
+
return visibleRunEntriesForSession(state.getRunEntries(ctx), getAgent);
|
|
4069
|
+
}
|
|
4070
|
+
|
|
4071
|
+
/**
|
|
4072
|
+
* Option B U3 (D2): after the boot scan, send ONE notice at the next turn
|
|
4073
|
+
* boundary listing every unresolved interrupted run whose dispatch snapshot
|
|
4074
|
+
* (and the session posture) says "offer". Delivery is `nextTurn` — the notice
|
|
4075
|
+
* can never steer or arrive mid-turn (see `resolveRecoveryNoticeDelivery`).
|
|
4076
|
+
* Silent when there is nothing to offer; at most once per session start.
|
|
4077
|
+
*/
|
|
4078
|
+
async function maybeSendRecoveryNotice(ctx: ExtensionContext): Promise<void> {
|
|
4079
|
+
if (recoveryNoticeDelivered) return;
|
|
4080
|
+
recoveryNoticeDelivered = true;
|
|
4081
|
+
// Session posture off silences the whole surface, regardless of per-run
|
|
4082
|
+
// snapshots (a user opting out expects quiet).
|
|
4083
|
+
if (state.config.recovery === "off") return;
|
|
4084
|
+
try {
|
|
4085
|
+
const { getAgent } = await import("./session-manager");
|
|
4086
|
+
const offered = offeredInterruptedRuns(
|
|
4087
|
+
collectInterruptedRuns({
|
|
4088
|
+
listInflight: () => listInflightRuns(),
|
|
4089
|
+
getAgent: (id) => getAgent(id),
|
|
4090
|
+
findRun: (id) => state.findTerminalRunById(ctx, id),
|
|
4091
|
+
}),
|
|
4092
|
+
);
|
|
4093
|
+
if (offered.length === 0) return;
|
|
4094
|
+
pi.sendMessage(buildRecoveryNotice(offered), resolveRecoveryNoticeDelivery());
|
|
4095
|
+
log.info("Recovery notice delivered", { count: offered.length });
|
|
4096
|
+
} catch (err) {
|
|
4097
|
+
// The notice is best-effort — it must never break session start.
|
|
4098
|
+
log.warn("Recovery notice failed", { error: (err as Error).message });
|
|
4099
|
+
}
|
|
4100
|
+
}
|
|
4101
|
+
|
|
4102
|
+
/**
|
|
4103
|
+
* Option B U3 (D5): finalize ONE interrupted run as the existing terminal
|
|
4104
|
+
* `stopped` outcome and prune its U2 session dir. Wired through injected deps
|
|
4105
|
+
* so the pure core (`recovery-surface.ts`) stays testable; the run record is
|
|
4106
|
+
* written terminal (the run store's `status` union has no `stopped` — a
|
|
4107
|
+
* stopped outcome is represented as `failed` + `errorCategory: "aborted"`,
|
|
4108
|
+
* exactly as every other stop path does) and the background agent record is
|
|
4109
|
+
* flipped to `stopped`.
|
|
4110
|
+
*/
|
|
4111
|
+
async function abandonProductionRun(ctx: ExtensionContext, id: string): Promise<void> {
|
|
4112
|
+
const { updateAgentStatus, pruneBackgroundSession, getAgent } = await import("./session-manager");
|
|
4113
|
+
// Fix A: the abandon lookup routes through the visibility bridge, so a run
|
|
4114
|
+
// that exists ONLY in the durable registry after a fresh boot is
|
|
4115
|
+
// abandonable instead of resolving to undefined (the silent `false`).
|
|
4116
|
+
const visible = visibleRunsFor(ctx, getAgent);
|
|
4117
|
+
abandonInterruptedRun(id, {
|
|
4118
|
+
getRun: (runId) => resolveTerminalRunEntry(visible, runId),
|
|
4119
|
+
writeRun: (run) => {
|
|
4120
|
+
state.persistRun(pi, run);
|
|
4121
|
+
pruneHistoryIfNeeded(state, ctx, log);
|
|
4122
|
+
},
|
|
4123
|
+
markAgentStopped: (agentId) => {
|
|
4124
|
+
const agent = updateAgentStatus(agentId, "stopped");
|
|
4125
|
+
if (agent) log.info("Abandoned background agent record", { agentId });
|
|
4126
|
+
},
|
|
4127
|
+
retireRegistry: (runId) => clearInflightRun(runId),
|
|
4128
|
+
pruneSession: (runId) => pruneBackgroundSession(runId),
|
|
4129
|
+
});
|
|
4130
|
+
}
|
|
4131
|
+
|
|
3967
4132
|
pi.on("session_start", async (_event, ctx) => {
|
|
3968
4133
|
// Issue #147: capture the session context for the completion-push subscriber.
|
|
3969
4134
|
sessionCtx = ctx;
|
|
@@ -4018,6 +4183,10 @@ export default function (pi: ExtensionAPI) {
|
|
|
4018
4183
|
// records interrupted BEFORE anything else touches their worktrees.
|
|
4019
4184
|
await performBootRecovery();
|
|
4020
4185
|
|
|
4186
|
+
// Option B U3 (D2): after the scan, surface every unresolved interrupted
|
|
4187
|
+
// run ONCE, at the next turn boundary (never mid-turn).
|
|
4188
|
+
await maybeSendRecoveryNotice(ctx);
|
|
4189
|
+
|
|
4021
4190
|
updateStatus(state, ctx);
|
|
4022
4191
|
});
|
|
4023
4192
|
|
|
@@ -4048,6 +4217,8 @@ export default function (pi: ExtensionAPI) {
|
|
|
4048
4217
|
state.unseenSubagents = 0;
|
|
4049
4218
|
// Clear pending queue
|
|
4050
4219
|
state.pendingQueue.length = 0;
|
|
4220
|
+
// U3: a new session boot may notify again.
|
|
4221
|
+
recoveryNoticeDelivered = false;
|
|
4051
4222
|
});
|
|
4052
4223
|
|
|
4053
4224
|
// -------------------------------------------------------------------
|
|
@@ -4057,9 +4228,10 @@ export default function (pi: ExtensionAPI) {
|
|
|
4057
4228
|
// subagent:failed / subagent:stopped), so the conductor is woken via
|
|
4058
4229
|
// pi.sendMessage + triggerTurn when a background run reaches a terminal state.
|
|
4059
4230
|
// Delivery is always-on (minimum nextTurn); the completionNotify knob controls
|
|
4060
|
-
// the WAKE (D1+D2 — see notify-completion.ts). The
|
|
4061
|
-
// pathological double-emit path (first terminal
|
|
4062
|
-
// capped simply (cleared when it
|
|
4231
|
+
// the WAKE (D1+D2 — see notify-completion.ts). The shared terminal claim
|
|
4232
|
+
// (issue #315) defends the pathological double-emit path (first terminal
|
|
4233
|
+
// event or crash notice per id wins); it is capped simply (cleared when it
|
|
4234
|
+
// exceeds 200 entries).
|
|
4063
4235
|
//
|
|
4064
4236
|
// Stopped-run degradation — ACCEPTED BY DESIGN (review #2, adjudicated
|
|
4065
4237
|
// 2026-09-01): subagent:stopped fires at stop time (updateAgentStatus) BEFORE
|
|
@@ -4068,7 +4240,6 @@ export default function (pi: ExtensionAPI) {
|
|
|
4068
4240
|
// resolveRunEntry returns may be the pre-finalize spawn entry. The wake is
|
|
4069
4241
|
// the point; details are best-effort — the notification must never throw or
|
|
4070
4242
|
// block on their absence.
|
|
4071
|
-
const terminalSeen = new Set<string>();
|
|
4072
4243
|
const terminalTypes = ["subagent:completed", "subagent:failed", "subagent:stopped"] as const;
|
|
4073
4244
|
for (const type of terminalTypes) {
|
|
4074
4245
|
eventBus.on(type, (event) => {
|
|
@@ -4078,7 +4249,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
4078
4249
|
|
|
4079
4250
|
async function deliverCompletionAlert(event: SubagentEvent): Promise<void> {
|
|
4080
4251
|
const id = event.agentId;
|
|
4081
|
-
if (!
|
|
4252
|
+
if (!claimTerminalNotice(id)) return;
|
|
4082
4253
|
const ctx = sessionCtx;
|
|
4083
4254
|
if (!ctx) return;
|
|
4084
4255
|
try {
|