@harms-haus/pi-subagents 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +362 -0
- package/docs/architecture.md +554 -0
- package/docs/changelog.md +61 -0
- package/docs/profiles.md +546 -0
- package/docs/settings.md +52 -0
- package/docs/tools-reference.md +519 -0
- package/package.json +59 -0
- package/src/cache.ts +24 -0
- package/src/commands/profile.ts +176 -0
- package/src/format-tool-call.ts +597 -0
- package/src/format-transcript.ts +151 -0
- package/src/index.ts +117 -0
- package/src/profile-editor.ts +356 -0
- package/src/profile-formatting.ts +178 -0
- package/src/profile-types.ts +73 -0
- package/src/profiles.ts +577 -0
- package/src/schemas.ts +65 -0
- package/src/settings.ts +155 -0
- package/src/skill-discovery.ts +30 -0
- package/src/spawner.ts +523 -0
- package/src/tools/delegate-render.ts +285 -0
- package/src/tools/delegate.ts +867 -0
- package/src/tools/retrieval.ts +287 -0
- package/src/types.ts +232 -0
- package/src/utils.ts +168 -0
|
@@ -0,0 +1,554 @@
|
|
|
1
|
+
# Architecture Reference
|
|
2
|
+
|
|
3
|
+
Deep-dive architecture document for the pi-subagents extension. This covers internal modules, data flows, concurrency model, and lifecycle management.
|
|
4
|
+
|
|
5
|
+
## 1. Overview
|
|
6
|
+
|
|
7
|
+
pi-subagents is a pi-coding-agent extension that enables the main agent to spawn multiple isolated sub-agent processes in parallel. Each sub-agent runs its own `pi` subprocess with an independent context window, provider/model configuration, and tool access. Live output from each sub-agent is rendered in a rolling TUI window inline with the main agent's conversation. The extension provides four tools — `delegate_to_subagents`, `get_subagent_output`, `get_subagent_session`, and `list_subagent_profiles` — plus a `/profile` slash command for interactive profile management. Session data is maintained in an in-memory store with LRU eviction, **and is also persisted to the main agent's session tree** via custom entries written through `pi.appendEntry()`. On session load (startup, resume, reload, or fork), the `session_start` handler reconstructs the in-memory store from these persisted entries. All sub-agent communication flows through JSONL-parsed stdout from the spawned processes.
|
|
8
|
+
|
|
9
|
+
## 2. Module Map
|
|
10
|
+
|
|
11
|
+
| File | Responsibility |
|
|
12
|
+
| ------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
13
|
+
| `src/index.ts` | Extension entry point; creates `sessionStore` (Map), registers tools/commands, handles `session_start` (reconstructs in-memory store from persisted custom entries) and `session_shutdown` lifecycle events |
|
|
14
|
+
| `src/types.ts` | Core type definitions (`SubAgentTask`, `SubAgentWindow`, `SubagentSessionData`, `SessionRecord`), configuration constants (`MAX_PARALLEL_TASKS`, `MAX_CONCURRENCY`, etc.), `syncState()` helper, session persistence helpers (`CUSTOM_ENTRY_TYPE`, `serializeSessionData()`, `deserializeSessionData()`). Re-exports `formatRunsForResume()` and `getTextContent()` from `format-transcript.ts` |
|
|
15
|
+
| `src/spawner.ts` | Process spawning, JSONL parsing, abort handling. `runSubAgent()` spawns `pi` subprocess, buffers stdout/stderr, parses JSON events, updates rolling window. Also processes `turn_end` events to capture ls/find tool result summaries. Also contains `getPiInvocation()` (moved from `utils.ts`) |
|
|
16
|
+
| `src/format-tool-call.ts` | `formatToolCall()` (one-line tool previews), `countNonEmptyLines()` (edit/write diff stats), `shortenPath()`, `formatBashCommand()`, `collapseCdDot()`, `shortenPathsInText()`, `formatToolResult()` (ls/find result summaries) |
|
|
17
|
+
| `src/settings.ts` | `loadMaxLinesPerWindow()`, `loadCommandPreviewWidth()`, settings file reading (global + project-local) |
|
|
18
|
+
| `src/format-transcript.ts` | `formatRunsForResume()` (resume transcript formatting), `getTextContent()` (message text extraction) |
|
|
19
|
+
| `src/profile-types.ts` | `SubagentProfile`, `SubagentProfiles`, `ThinkingLevel`, `ProfileInvocation` type definitions |
|
|
20
|
+
| `src/profile-formatting.ts` | `profileSummary()`, `formatProfileDetail()`, `serializeProfileToMarkdown()` |
|
|
21
|
+
| `src/profiles.ts` | Profile loading from `.md` files, YAML frontmatter parsing, 5s TTL cache, `profileToArgs()` CLI conversion, profile CRUD, tool validation (`validateProfileTools`, `applyExcludeTools`), skill validation (`validateProfileSkills`) and resolution (`resolveProfileSkills`). Re-exports from `profile-formatting.ts` and `profile-types.ts` |
|
|
22
|
+
| `src/profile-editor.ts` | Interactive profile creation/editing via `/profile` command |
|
|
23
|
+
| `src/commands/profile.ts` | `/profile` slash command (list, show, create, edit, delete) |
|
|
24
|
+
| `src/schemas.ts` | TypeBox schemas for `delegate_to_subagents` parameter validation |
|
|
25
|
+
| `src/tools/delegate.ts` | `delegate_to_subagents` tool registration — profile resolution, session creation, concurrency orchestration, session persistence via `persistSession()` helper. Delegates TUI rendering to `delegate-render.ts` |
|
|
26
|
+
| `src/tools/delegate-render.ts` | `colorizeToolLine()`, `renderDelegateCall()`, `renderDelegateResult()` — pure rendering functions for the delegate tool TUI display |
|
|
27
|
+
| `src/tools/retrieval.ts` | `get_subagent_output`, `get_subagent_session`, `list_subagent_profiles` tool registrations with truncating renderers |
|
|
28
|
+
| `src/utils.ts` | Shared helpers: ANSI stripping (`stripAnsi`), `appendLineToWindow()`, `getTextParts()`, `getLastAssistantText()`, `mapWithConcurrencyLimit()`, `countWindowStatuses()`, `getSummaryText()` |
|
|
29
|
+
|
|
30
|
+
### Dependency Graph
|
|
31
|
+
|
|
32
|
+
```
|
|
33
|
+
index.ts
|
|
34
|
+
├── profiles.ts
|
|
35
|
+
│ ├── profile-formatting.ts
|
|
36
|
+
│ │ └── profile-types.ts
|
|
37
|
+
│ └── profile-types.ts
|
|
38
|
+
├── commands/profile.ts
|
|
39
|
+
│ ├── profiles.ts
|
|
40
|
+
│ └── profile-editor.ts
|
|
41
|
+
│ └── profiles.ts
|
|
42
|
+
├── tools/delegate.ts
|
|
43
|
+
│ ├── profiles.ts
|
|
44
|
+
│ ├── schemas.ts
|
|
45
|
+
│ │ └── types.ts
|
|
46
|
+
│ ├── settings.ts
|
|
47
|
+
│ ├── spawner.ts
|
|
48
|
+
│ │ ├── format-tool-call.ts
|
|
49
|
+
│ │ ├── profiles.ts
|
|
50
|
+
│ │ ├── settings.ts
|
|
51
|
+
│ │ ├── types.ts
|
|
52
|
+
│ │ └── utils.ts
|
|
53
|
+
│ │ └── types.ts
|
|
54
|
+
│ ├── tools/delegate-render.ts
|
|
55
|
+
│ │ ├── types.ts
|
|
56
|
+
│ │ └── utils.ts
|
|
57
|
+
│ │ └── types.ts
|
|
58
|
+
│ ├── types.ts ← CUSTOM_ENTRY_TYPE, serializeSessionData
|
|
59
|
+
│ └── utils.ts
|
|
60
|
+
│ └── types.ts
|
|
61
|
+
├── tools/retrieval.ts
|
|
62
|
+
│ ├── profiles.ts
|
|
63
|
+
│ ├── settings.ts
|
|
64
|
+
│ ├── types.ts
|
|
65
|
+
│ └── utils.ts
|
|
66
|
+
│ └── types.ts
|
|
67
|
+
└── types.ts ← CUSTOM_ENTRY_TYPE, deserializeSessionData
|
|
68
|
+
└── format-transcript.ts
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
## 3. Session Lifecycle
|
|
72
|
+
|
|
73
|
+
### 3.1 Session Store
|
|
74
|
+
|
|
75
|
+
The session store is an in-memory `Map<string, SessionRecord>` held in `index.ts`, shared across all tool registrations.
|
|
76
|
+
|
|
77
|
+
```ts
|
|
78
|
+
const sessionStore = new Map<string, SessionRecord>();
|
|
79
|
+
const MAX_STORED_SESSIONS = 32;
|
|
80
|
+
const MAX_RUNS_PER_SESSION = 10;
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
Each `SessionRecord` maps a session ID to an array of `SubagentSessionData` runs:
|
|
84
|
+
|
|
85
|
+
```ts
|
|
86
|
+
interface SessionRecord {
|
|
87
|
+
runs: SubagentSessionData[]; // chronological, max 10
|
|
88
|
+
}
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
**Capacity management:**
|
|
92
|
+
|
|
93
|
+
- **Max sessions:** When `sessionStore.size >= 32`, the oldest session (determined by `runs[0].startedAt`) is evicted (LRU by start time).
|
|
94
|
+
- **Max runs per session:** When a session is resumed, the new run is appended to `record.runs`. If the array exceeds 10 entries, the oldest run is shifted off (`shift()`).
|
|
95
|
+
|
|
96
|
+
**Persistence:** After each sub-agent completes (or errors), `persistSession()` writes the session data to the main agent's session tree via `pi.appendEntry(CUSTOM_ENTRY_TYPE, serializeSessionData(session))`. This ensures session data survives across session reloads, resumes, and forks.
|
|
97
|
+
|
|
98
|
+
**Reconstruction:** On `session_start` (for any reason other than `"new"`), the handler iterates the session's custom entries, deserializes any with `customType === CUSTOM_ENTRY_TYPE` via `deserializeSessionData()`, and calls `registerSession()` to rebuild the in-memory store. Stale `"running"` sessions (from crashes) are automatically converted to `"error"` status during deserialization.
|
|
99
|
+
|
|
100
|
+
**Cleanup:** The store is cleared entirely on the `session_shutdown` event:
|
|
101
|
+
|
|
102
|
+
```ts
|
|
103
|
+
pi.on("session_shutdown", () => {
|
|
104
|
+
sessionStore.clear();
|
|
105
|
+
});
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
### 3.2 Full Lifecycle Sequence
|
|
109
|
+
|
|
110
|
+
```
|
|
111
|
+
1. LLM calls delegate_to_subagents({ tasks: [...] })
|
|
112
|
+
2. delegate.ts validates resume parameters (if any)
|
|
113
|
+
3. Profile resolution: loadProfiles(cwd) → resolveProfile() per task
|
|
114
|
+
3a. `excludeTools` resolution: validateProfileTools() + applyExcludeTools() per profile with excludeTools
|
|
115
|
+
3b. Skill resolution: validateProfileSkills() → discoverSkills() (once, cached) → resolveProfileSkills() per unique profile
|
|
116
|
+
4. Windows created: one SubAgentWindow per task (TUI state)
|
|
117
|
+
5. Session data created: one SubagentSessionData per task (persistent store)
|
|
118
|
+
6. registerSession() → store in sessionStore (with LRU eviction)
|
|
119
|
+
7. mapWithConcurrencyLimit(tasks, 4) → per-task execution:
|
|
120
|
+
a. Per-task AbortController with timeout (default 600s)
|
|
121
|
+
b. Parent abort signal forwarded to task controller
|
|
122
|
+
c. runSubAgent() spawns pi subprocess
|
|
123
|
+
d. stdout lines parsed as JSONL, appended to rolling window; `turn_end` events processed to capture ls/find tool result summaries
|
|
124
|
+
e. Process exit → status set to "completed" or "error"
|
|
125
|
+
f. persistSession() → serialize and write to main agent's session tree via pi.appendEntry()
|
|
126
|
+
8. Summary result returned with session IDs
|
|
127
|
+
9. LLM retrieves output via get_subagent_output(sessionId)
|
|
128
|
+
or get_subagent_session(sessionId)
|
|
129
|
+
|
|
130
|
+
**On session load** (`session_start` with reason ≠ `"new"`):
|
|
131
|
+
- Custom entries with `customType === "pi-subagents"` are deserialized via `deserializeSessionData()`
|
|
132
|
+
- Validated entries are registered into the in-memory session store
|
|
133
|
+
- Stale `"running"` entries (from crashes) are converted to `"error"` status
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
### 3.3 Resume Flow
|
|
137
|
+
|
|
138
|
+
When a task specifies a `resume` session ID:
|
|
139
|
+
|
|
140
|
+
1. The resume session must exist in the store and **not** be actively running.
|
|
141
|
+
2. `formatRunsForResume(record.runs)` produces a human-readable transcript of all previous runs, including user messages, assistant text, tool calls (truncated args to 120 chars), tool results (truncated to 500 chars), and error messages.
|
|
142
|
+
3. The transcript is prepended to the new task's prompt:
|
|
143
|
+
|
|
144
|
+
```ts
|
|
145
|
+
effectivePrompt = `Previously:\n\n${previousData}\n\nInstructions:\n\n${task.prompt}`;
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
4. The resumed session reuses the **same** session ID (from `task.resume`), so subsequent runs are appended to the same `SessionRecord`.
|
|
149
|
+
5. `get_subagent_output` returns only the **latest** run's text; `get_subagent_session` returns **all** runs concatenated with separators.
|
|
150
|
+
|
|
151
|
+
## 4. Spawner Internals
|
|
152
|
+
|
|
153
|
+
`runSubAgent()` in `src/spawner.ts` is the core process-spawning function.
|
|
154
|
+
|
|
155
|
+
### 4.1 pi Invocation
|
|
156
|
+
|
|
157
|
+
The subprocess is spawned using Node.js `child_process.spawn()`:
|
|
158
|
+
|
|
159
|
+
```ts
|
|
160
|
+
const proc = spawn(invocation.command, args, {
|
|
161
|
+
cwd: resolvedCwd,
|
|
162
|
+
shell: false,
|
|
163
|
+
stdio: ["pipe", "pipe", "pipe"],
|
|
164
|
+
env: { ...process.env, ...profileEnv },
|
|
165
|
+
});
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
**Command resolution** (`getPiInvocation()` in `spawner.ts`):
|
|
169
|
+
|
|
170
|
+
- If running as a script (`process.argv[1]` exists and isn't in `$bunfs`), uses `process.execPath` + script path.
|
|
171
|
+
- Otherwise falls back to `pi` with no args.
|
|
172
|
+
|
|
173
|
+
**Args structure:**
|
|
174
|
+
|
|
175
|
+
```
|
|
176
|
+
[base args...] --mode json -p --no-session [profile args...] [task prompt]
|
|
177
|
+
```
|
|
178
|
+
|
|
179
|
+
- `--mode json` — forces JSON output from the pi subprocess.
|
|
180
|
+
- `-p` — enables profile mode (reads prompts from argument).
|
|
181
|
+
- `--no-session` — runs without interactive session persistence.
|
|
182
|
+
- Profile args (from `profileToArgs()`) are injected **before** the prompt, so they take effect as CLI flags.
|
|
183
|
+
|
|
184
|
+
### 4.2 stdout Line Buffering & JSONL Parsing
|
|
185
|
+
|
|
186
|
+
stdout data arrives in arbitrary Buffer chunks. The spawner implements line-buffered parsing:
|
|
187
|
+
|
|
188
|
+
```ts
|
|
189
|
+
proc.stdout.on("data", (data: Buffer) => {
|
|
190
|
+
buffer += data.toString();
|
|
191
|
+
const lines = buffer.split("\n");
|
|
192
|
+
buffer = lines.pop() ?? ""; // incomplete line held for next chunk
|
|
193
|
+
for (const line of lines) {
|
|
194
|
+
processLine(line);
|
|
195
|
+
}
|
|
196
|
+
});
|
|
197
|
+
```
|
|
198
|
+
|
|
199
|
+
Each complete line is processed by `handleStdoutLine()`:
|
|
200
|
+
|
|
201
|
+
1. **Empty lines** — skipped.
|
|
202
|
+
2. **Non-JSON lines** — appended to the rolling window as plain text.
|
|
203
|
+
3. **JSON lines** — parsed. Two event types are processed:
|
|
204
|
+
- **`turn_end` events** — `toolResults` array is inspected for `ls`/`find` tool results; summaries are generated via `formatToolResult()` and appended as `"tool"`-kind lines.
|
|
205
|
+
- **`message_end` events** (with a `message` field) — Message is pushed to `session.messages` (capped at `MAX_MESSAGES_PER_SESSION = 500`).
|
|
206
|
+
- Text parts extracted via `getTextParts()` → appended to rolling window.
|
|
207
|
+
- Tool call parts formatted via `formatToolCall()` → appended as `"tool"`-kind lines.
|
|
208
|
+
- Model metadata (`model`, `stopReason`, `errorMessage`) synced to window and session via `syncState()`.
|
|
209
|
+
|
|
210
|
+
### 4.3 stderr Handling
|
|
211
|
+
|
|
212
|
+
stderr data is captured and prefixed with `[stderr]:`, then appended to the rolling window. Unlike stdout, stderr is not parsed as JSON — it's treated as raw text.
|
|
213
|
+
|
|
214
|
+
### 4.4 Process Exit & Status Determination
|
|
215
|
+
|
|
216
|
+
On `close` event, `handleProcessExit()` determines final status:
|
|
217
|
+
|
|
218
|
+
```ts
|
|
219
|
+
const isError = code !== 0 || win.stopReason === "error" || win.stopReason === "aborted";
|
|
220
|
+
const status = isError ? "error" : "completed";
|
|
221
|
+
```
|
|
222
|
+
|
|
223
|
+
Any remaining buffered content is flushed before status is set. Spawn errors (e.g., binary not found) set status to `"error"` immediately with message `"Failed to spawn sub-agent process"`.
|
|
224
|
+
|
|
225
|
+
## 5. Abort & Timeout Handling
|
|
226
|
+
|
|
227
|
+
### 5.1 Per-Task AbortController
|
|
228
|
+
|
|
229
|
+
Each task gets its own `AbortController`:
|
|
230
|
+
|
|
231
|
+
```ts
|
|
232
|
+
const taskAbortController = new AbortController();
|
|
233
|
+
const taskAbortTimeout = setTimeout(() => {
|
|
234
|
+
taskAbortController.abort();
|
|
235
|
+
}, taskTimeout * 1000); // taskTimeout defaults to 600s
|
|
236
|
+
```
|
|
237
|
+
|
|
238
|
+
### 5.2 Parent Signal Forwarding
|
|
239
|
+
|
|
240
|
+
The parent's `signal` (from the LLM tool execution context) is forwarded:
|
|
241
|
+
|
|
242
|
+
```ts
|
|
243
|
+
const onParentAbort = () => taskAbortController.abort();
|
|
244
|
+
if (signal?.aborted) {
|
|
245
|
+
taskAbortController.abort();
|
|
246
|
+
} else if (signal) {
|
|
247
|
+
signal.addEventListener("abort", onParentAbort, { once: true });
|
|
248
|
+
});
|
|
249
|
+
```
|
|
250
|
+
|
|
251
|
+
The listener is removed in the `finally` block to prevent leaks.
|
|
252
|
+
|
|
253
|
+
### 5.3 SIGTERM → SIGKILL Escalation
|
|
254
|
+
|
|
255
|
+
In `setupAbortHandler()`, when the task's abort signal fires:
|
|
256
|
+
|
|
257
|
+
```ts
|
|
258
|
+
const killProc = () => {
|
|
259
|
+
proc.kill("SIGTERM");
|
|
260
|
+
setTimeout(() => {
|
|
261
|
+
if (!proc.killed) proc.kill("SIGKILL");
|
|
262
|
+
}, 5000); // 5-second grace period
|
|
263
|
+
};
|
|
264
|
+
```
|
|
265
|
+
|
|
266
|
+
If the process was already aborted when the handler is installed (parent cancelled before spawn), `killProc()` is invoked immediately.
|
|
267
|
+
|
|
268
|
+
### 5.4 Timeout vs. Parent Abort Distinction
|
|
269
|
+
|
|
270
|
+
After `runSubAgent()` completes, the delegate tool checks whether the abort was caused by the task's own timeout (not the parent signal):
|
|
271
|
+
|
|
272
|
+
```ts
|
|
273
|
+
if (taskAbortController.signal.aborted && !signal?.aborted) {
|
|
274
|
+
win.status = "error";
|
|
275
|
+
win.errorMessage = `Timed out after ${taskTimeout}s. Consider resuming with a longer timeout.`;
|
|
276
|
+
}
|
|
277
|
+
```
|
|
278
|
+
|
|
279
|
+
This allows a clean timeout error message rather than treating it as a generic abort.
|
|
280
|
+
|
|
281
|
+
## 6. Rolling Buffer & TUI Updates
|
|
282
|
+
|
|
283
|
+
### 6.1 Dual Buffers in `appendLineToWindow()`
|
|
284
|
+
|
|
285
|
+
Each `SubAgentWindow` maintains two buffers:
|
|
286
|
+
|
|
287
|
+
| Buffer | Purpose | Limit |
|
|
288
|
+
| ----------------- | -------------------------------------------------------- | -------------------------------- |
|
|
289
|
+
| `win.lines` | Rolling window shown in **collapsed** TUI view | `maxLines` (default 15) |
|
|
290
|
+
| `win.allMessages` | Full message history shown in **expanded** view (Ctrl+O) | `MAX_MESSAGES_PER_SESSION` (500) |
|
|
291
|
+
|
|
292
|
+
```ts
|
|
293
|
+
export function appendLineToWindow(win, line, maxLines, kind = "text") {
|
|
294
|
+
const clean = stripAnsi(line).trimEnd();
|
|
295
|
+
if (!clean) return;
|
|
296
|
+
const entry = { text: clean, kind };
|
|
297
|
+
|
|
298
|
+
// Rolling window (latest N lines)
|
|
299
|
+
win.lines.push(entry);
|
|
300
|
+
while (win.lines.length > maxLines) win.lines.shift();
|
|
301
|
+
|
|
302
|
+
// Full history (capped at 500)
|
|
303
|
+
win.allMessages.push(entry);
|
|
304
|
+
while (win.allMessages.length > MAX_MESSAGES_PER_SESSION) win.allMessages.shift();
|
|
305
|
+
}
|
|
306
|
+
```
|
|
307
|
+
|
|
308
|
+
Lines are ANSI-stripped before storage. Each `WindowLine` carries a `kind` (`"text"` or `"tool"`) that affects TUI rendering — tool lines are colorized by `colorizeToolLine()` in `delegate-render.ts`: diff additions in `toolDiffAdded` (green), removals in `toolDiffRemoved` (red), line counts in `toolDiffAdded`, ls/find result summary counts in `toolDiffAdded` (green), and all other text in `muted`.
|
|
309
|
+
|
|
310
|
+
### 6.2 Debounced TUI Updates
|
|
311
|
+
|
|
312
|
+
Updates to the TUI are debounced at 50ms to avoid excessive rendering pressure during high-throughput stdout:
|
|
313
|
+
|
|
314
|
+
```ts
|
|
315
|
+
const debouncedUpdate = () => {
|
|
316
|
+
if (bufferTimeout) clearTimeout(bufferTimeout);
|
|
317
|
+
bufferTimeout = setTimeout(() => onUpdate(), 50);
|
|
318
|
+
};
|
|
319
|
+
```
|
|
320
|
+
|
|
321
|
+
The final `onUpdate()` in `handleProcessExit()` is **not** debounced — it fires immediately to ensure the final status is visible.
|
|
322
|
+
|
|
323
|
+
### 6.3 `renderResult`: Collapsed vs. Expanded View
|
|
324
|
+
|
|
325
|
+
The `renderResult` method in `delegate.ts` renders the TUI output:
|
|
326
|
+
|
|
327
|
+
- **Collapsed** (default): Shows `win.lines` (latest N lines). If empty, displays `"(starting...)"`.
|
|
328
|
+
- **Expanded** (Ctrl+O): Shows `win.allMessages` (up to 500 entries). If empty, displays `"(no output)"`.
|
|
329
|
+
|
|
330
|
+
Each window header includes the agent name, profile info (if any), and a status icon (⏳ / ✗ / ✓). Error messages are shown in red below the window. When all agents complete, a footer lists session IDs for retrieval.
|
|
331
|
+
|
|
332
|
+
### 6.4 `formatToolCall()` One-Line Previews
|
|
333
|
+
|
|
334
|
+
Tool calls in the rolling window are rendered as concise one-liners by `formatToolCall()` in `format-tool-call.ts`. Key patterns:
|
|
335
|
+
|
|
336
|
+
| Tool | Format |
|
|
337
|
+
| ---------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
|
|
338
|
+
| `edit` | `edit → path (N edits) +A/-R` |
|
|
339
|
+
| `write` | `write → path +L` |
|
|
340
|
+
| `grep` | `grep → /pattern/ → glob-or-path` |
|
|
341
|
+
| `ls` | `ls → <path>` (defaults to `.` when no path) |
|
|
342
|
+
| `find` | `find → <pattern> in <path>` or `find → <pattern>` (no path) |
|
|
343
|
+
| `bash` | `bash → first line of command (smart && splitting)` |
|
|
344
|
+
| `read` | `read → path:offset+limit (L lines)` |
|
|
345
|
+
| `delegate_to_subagents` | `delegate_to_subagents → N tasks [profile names]` |
|
|
346
|
+
| LSP tools | `lsp_diagnostics → file` |
|
|
347
|
+
| `fetch_content`/`web_search` | `fetch_content → url (truncated)` |
|
|
348
|
+
| Generic | `toolName {"key":"value",...}` (full JSON args, truncated to width budget; empty `{}` omitted) |
|
|
349
|
+
| `write_todos` | `write_todos → N todos written` |
|
|
350
|
+
| `edit_todos` | `edit_todos → description or action [indices]` |
|
|
351
|
+
| `list_todos` | `list_todos` |
|
|
352
|
+
| LSP tools (5) | `lsp_name → file:line:column` (varies; `lsp_find_symbol` → `query`; `lsp_refactor_symbol` → `file:line:col → newName`) |
|
|
353
|
+
| `lint_files` | `lint → files... +N more` or `lint → (all)` |
|
|
354
|
+
| `fetch_repo` | `fetch_repo → url` |
|
|
355
|
+
| Session retrieval (3) | `tool_name → sessionId` or just tool name |
|
|
356
|
+
| `workflow_step` | `workflow_step → action` |
|
|
357
|
+
|
|
358
|
+
Where A=lines added, R=lines removed, L=line count. The `(L lines)` suffix only appears when `limit` is specified. Diff stats are computed using `countNonEmptyLines()`, which counts non-blank lines without allocating intermediate arrays.
|
|
359
|
+
|
|
360
|
+
Diff stats (`+A/-R`), line counts (`(L lines)`), and numeric counts in ls/find result summary lines are colorized in the TUI using `colorizeToolLine()`: additions use `toolDiffAdded` (green), removals use `toolDiffRemoved` (red), and line counts use `toolDiffAdded` (green).
|
|
361
|
+
|
|
362
|
+
Paths are shortened via `shortenPath()` (replaces home prefix with `~`, uses relative paths when shorter). Bash commands are collapsed (stripping redundant `cd <cwd>` prefixes) and formatted with `formatBashCommand()` (smart `&&` splitting with `│` continuation prefixes).
|
|
363
|
+
|
|
364
|
+
## 7. Concurrency Model
|
|
365
|
+
|
|
366
|
+
### 7.1 `mapWithConcurrencyLimit()` Worker-Pool Pattern
|
|
367
|
+
|
|
368
|
+
```ts
|
|
369
|
+
export async function mapWithConcurrencyLimit<TIn, TOut>(
|
|
370
|
+
items: TIn[],
|
|
371
|
+
concurrency: number,
|
|
372
|
+
fn: (item: TIn, index: number) => Promise<TOut>,
|
|
373
|
+
): Promise<TOut[]>;
|
|
374
|
+
```
|
|
375
|
+
|
|
376
|
+
Implementation uses a shared `nextIndex` counter accessed by `concurrency` worker coroutines:
|
|
377
|
+
|
|
378
|
+
```ts
|
|
379
|
+
let nextIndex = 0;
|
|
380
|
+
const workers = new Array(limit).fill(null).map(async () => {
|
|
381
|
+
while (true) {
|
|
382
|
+
const current = nextIndex++; // atomic-ish: JS single-threaded
|
|
383
|
+
if (current >= items.length) return;
|
|
384
|
+
results[current] = await fn(items[current], current);
|
|
385
|
+
}
|
|
386
|
+
});
|
|
387
|
+
await Promise.all(workers);
|
|
388
|
+
```
|
|
389
|
+
|
|
390
|
+
This is a work-stealing pattern — as each worker completes its task, it immediately grabs the next available index. Results are stored at their original indices, preserving order.
|
|
391
|
+
|
|
392
|
+
### 7.2 Concurrency Constants
|
|
393
|
+
|
|
394
|
+
| Constant | Value | Meaning |
|
|
395
|
+
| -------------------- | ----- | ----------------------------------------------------------------------------------- |
|
|
396
|
+
| `MAX_CONCURRENCY` | 4 | Active sub-agent processes at once |
|
|
397
|
+
| `MAX_PARALLEL_TASKS` | 16 | Maximum tasks in a single `delegate_to_subagents` call (enforced by TypeBox schema) |
|
|
398
|
+
|
|
399
|
+
When a delegation call has 16 tasks with concurrency 4, the first 4 spawn immediately, and the remaining 12 queue — each starts as a slot opens.
|
|
400
|
+
|
|
401
|
+
## 8. Profile System
|
|
402
|
+
|
|
403
|
+
See [docs/profiles.md](profiles.md) for the full profile documentation. This section covers the internal architecture.
|
|
404
|
+
|
|
405
|
+
### 8.1 Profile Loading
|
|
406
|
+
|
|
407
|
+
Profiles are loaded from two directories:
|
|
408
|
+
|
|
409
|
+
| Scope | Path |
|
|
410
|
+
| ------- | ------------------------------------------------------------------- |
|
|
411
|
+
| Global | `~/.pi/agent/agent-profiles/*.md` (configurable via `PI_AGENT_DIR`) |
|
|
412
|
+
| Project | `<cwd>/.pi/agent-profiles/*.md` |
|
|
413
|
+
|
|
414
|
+
Project-local profiles **override** global profiles with the same name. Loading is synchronous via `readFileSync` — each `.md` file is parsed with `parseFrontmatter()`:
|
|
415
|
+
|
|
416
|
+
```ts
|
|
417
|
+
// Frontmatter keys → SubagentProfile fields
|
|
418
|
+
name, provider, model, thinkingLevel, appendSystemPrompt, apiKey,
|
|
419
|
+
noTools, noExtensions, noSkills, noContextFiles,
|
|
420
|
+
tools (string or comma-separated), excludeTools (comma-separated), extensions, extraArgs,
|
|
421
|
+
suggestedSkills (comma-separated), loadSkills (comma-separated)
|
|
422
|
+
|
|
423
|
+
// Body (after frontmatter) → systemPrompt
|
|
424
|
+
```
|
|
425
|
+
|
|
426
|
+
### 8.2 5-Second TTL Cache
|
|
427
|
+
|
|
428
|
+
```ts
|
|
429
|
+
let profilesCache: {
|
|
430
|
+
cwd: string | undefined;
|
|
431
|
+
profiles: SubagentProfiles;
|
|
432
|
+
timestamp: number;
|
|
433
|
+
} | null = null;
|
|
434
|
+
const CACHE_TTL = 5000;
|
|
435
|
+
```
|
|
436
|
+
|
|
437
|
+
The cache key includes the `cwd` — different working directories get separate cache entries. Cache is invalidated by `saveProfile()`, `deleteProfile()`, and `invalidateProfilesCache()`.
|
|
438
|
+
|
|
439
|
+
### 8.3 `excludeTools` Resolution (in delegate.ts)
|
|
440
|
+
|
|
441
|
+
Before `profileToArgs()` is called, profiles with `excludeTools` are resolved to concrete tool allowlists in the delegate tool's execute handler:
|
|
442
|
+
|
|
443
|
+
1. For each task whose profile has `excludeTools` set, `pi.getAllTools()` is called once to obtain the full list of available tool names.
|
|
444
|
+
2. `validateProfileTools(profile, name)` throws if both `tools` (allowlist) and `excludeTools` (blacklist) are set — they are mutually exclusive.
|
|
445
|
+
3. `applyExcludeTools(profile, allToolNames)` computes the allowlist:
|
|
446
|
+
```ts
|
|
447
|
+
const excludeSet = new Set(profile.excludeTools);
|
|
448
|
+
const computedTools = allToolNames.filter((name) => !excludeSet.has(name));
|
|
449
|
+
return { ...profile, tools: computedTools, excludeTools: undefined };
|
|
450
|
+
```
|
|
451
|
+
4. The resolved profile (now with `tools` populated and `excludeTools` cleared) replaces the original in the `resolvedProfiles` array.
|
|
452
|
+
5. This resolved profile flows through `profileToArgs()` as a normal `--tools <computed-list>` argument.
|
|
453
|
+
|
|
454
|
+
If no profile has `excludeTools` set, `pi.getAllTools()` is never called.
|
|
455
|
+
|
|
456
|
+
### 8.4 Skill Resolution (in delegate.ts)
|
|
457
|
+
|
|
458
|
+
Before `profileToArgs()` is called, profiles with `suggestedSkills` or `loadSkills` are resolved in a multi-step pipeline inside the delegate tool's execute handler:
|
|
459
|
+
|
|
460
|
+
1. **Mutual-exclusivity validation** — `validateProfileSkills(profile, name)` throws if `suggestedSkills` or `loadSkills` is combined with `noSkills`. These are mutually exclusive because `--no-skills` would either override `--skill` flags or disable skill content that was injected into the system prompt.
|
|
461
|
+
|
|
462
|
+
2. **Discovery (once per delegation)** — If any profile references skills, `discoverSkills()` is called exactly once with `{ cwd, agentDir, skillPaths: [], includeDefaults: true }`. The result is cached in a `Map<string, SkillMeta>` keyed by skill name. If no profile uses skills, discovery is skipped entirely.
|
|
463
|
+
|
|
464
|
+
3. **Per-profile resolution** — `resolveProfileSkills(profile, cwd, skillMap)` is called once per **unique** profile object (deduplicated via `skillResolvedProfiles` Map). Two resolution paths:
|
|
465
|
+
- **`suggestedSkills`** — skill names are mapped to their file paths. The resolved `suggestedSkills` array contains file paths (not names), which `profileToArgs()` converts to `--skill <path>` CLI flags. Unknown skill names throw an error listing available skills.
|
|
466
|
+
- **`loadSkills`** — skill names are mapped to their SKILL.md body content (frontmatter stripped via `stripFrontmatter()`). Each skill is wrapped in `<loaded_skill name="...">...</loaded_skill>` tags and appended to `appendSystemPrompt`. After resolution, `loadSkills` is set to `undefined` on the profile (it has been fully consumed).
|
|
467
|
+
|
|
468
|
+
4. **Deduplication via `skillResolvedProfiles` Map** — A `Map<SubagentProfile, { ok, profile } | { ok, error }>` ensures each unique profile object is resolved at most once. Multiple tasks sharing the same profile reuse the cached result.
|
|
469
|
+
|
|
470
|
+
5. **Error handling** — If `resolveProfileSkills()` throws (e.g., unknown skill name), the error is stored in `skillResolvedProfiles` as `{ ok: false, error }`. When the offending task is processed inside `mapWithConcurrencyLimit`, it is marked as `"error"` with the skill resolution error message — but **other tasks are unaffected** and continue normally.
|
|
471
|
+
|
|
472
|
+
### 8.5 `profileToArgs()` Conversion
|
|
473
|
+
|
|
474
|
+
Converts a `SubagentProfile` to CLI arguments and environment variables:
|
|
475
|
+
|
|
476
|
+
```ts
|
|
477
|
+
interface ProfileInvocation {
|
|
478
|
+
args: string[]; // CLI flags appended before the prompt
|
|
479
|
+
env: Record<string, string>; // merged into process.env
|
|
480
|
+
}
|
|
481
|
+
```
|
|
482
|
+
|
|
483
|
+
**Security note:** API keys are passed via `PI_API_KEY` environment variable, **not** as CLI arguments (which would be visible in `/proc/PID/cmdline`).
|
|
484
|
+
|
|
485
|
+
**Safety validation for `extraArgs`:**
|
|
486
|
+
|
|
487
|
+
- Null bytes (`\0`) are rejected.
|
|
488
|
+
- Characters at the start of an arg: whitespace, `|`, `&`, `;`, `$`, `\`, `` ` ``, `!`
|
|
489
|
+
- Command separators anywhere in the arg: `&&`, `||`, `;`, `>`, `>>`, `<`, `<<`
|
|
490
|
+
|
|
491
|
+
**`suggestedSkills` → `--skill` flags:** After `resolveProfileSkills()` runs, `suggestedSkills` contains resolved file paths (not skill names). `profileToArgs()` emits one `--skill <path>` per entry. `loadSkills` is never encountered here — it was already consumed by `resolveProfileSkills()` and merged into `appendSystemPrompt` during the skill resolution pipeline (§8.4).
|
|
492
|
+
|
|
493
|
+
**Tool-override blocking:** When any tool restriction is active on the profile (`noTools`, `tools`, or `excludeTools`), `extraArgs` containing tool-override flags are rejected. The blocked flags are:
|
|
494
|
+
|
|
495
|
+
| Flag | Variants |
|
|
496
|
+
| ------------ | -------------------------------- |
|
|
497
|
+
| `--tools` | `--tools`, `--tools=value` |
|
|
498
|
+
| `-t` | `-t`, `-t=value` |
|
|
499
|
+
| `--no-tools` | `--no-tools`, `--no-tools=value` |
|
|
500
|
+
| `-nt` | `-nt`, `-nt=value` |
|
|
501
|
+
|
|
502
|
+
This prevents `extraArgs` from bypassing the profile's intended tool restrictions via equals-sign forms or short flags.
|
|
503
|
+
|
|
504
|
+
### 8.6 Settings Files
|
|
505
|
+
|
|
506
|
+
Two settings locations are checked (project overrides global):
|
|
507
|
+
|
|
508
|
+
| Scope | Path |
|
|
509
|
+
| ------- | --------------------------- |
|
|
510
|
+
| Global | `~/.pi/agent/settings.json` |
|
|
511
|
+
| Project | `<cwd>/.pi/settings.json` |
|
|
512
|
+
|
|
513
|
+
Settings are read from the `subagents` key:
|
|
514
|
+
|
|
515
|
+
```json
|
|
516
|
+
{
|
|
517
|
+
"subagents": {
|
|
518
|
+
"maxLinesPerWindow": 20,
|
|
519
|
+
"commandPreviewWidth": 120
|
|
520
|
+
}
|
|
521
|
+
}
|
|
522
|
+
```
|
|
523
|
+
|
|
524
|
+
Both values default gracefully if missing: `maxLinesPerWindow` defaults to `15`, `commandPreviewWidth` falls back to TTY terminal width minus 4 (or `160` in non-TTY environments), clamped to a minimum of `20`.
|
|
525
|
+
|
|
526
|
+
## 9. Settings
|
|
527
|
+
|
|
528
|
+
### 9.1 `maxLinesPerWindow`
|
|
529
|
+
|
|
530
|
+
Controls the number of lines shown in each sub-agent's collapsed rolling window.
|
|
531
|
+
|
|
532
|
+
- **Type:** `number`
|
|
533
|
+
- **Default:** `15`
|
|
534
|
+
- **Resolution:** Global settings → project settings → default
|
|
535
|
+
|
|
536
|
+
### 9.2 `commandPreviewWidth`
|
|
537
|
+
|
|
538
|
+
Controls the character budget for bash command previews in tool call lines.
|
|
539
|
+
|
|
540
|
+
- **Type:** `number`
|
|
541
|
+
- **Default:** Terminal width minus 4 (if TTY), otherwise `160`
|
|
542
|
+
- **Minimum:** `20` (clamped)
|
|
543
|
+
- **Resolution:** Two independent paths:
|
|
544
|
+
- In TTY mode: terminal width − 4 (min 20); settings are **not consulted**
|
|
545
|
+
- In non-TTY mode: global settings → project settings → default `160` (min 20)
|
|
546
|
+
|
|
547
|
+
### 9.3 Settings File Locations
|
|
548
|
+
|
|
549
|
+
```
|
|
550
|
+
~/.pi/agent/settings.json ← global settings
|
|
551
|
+
<cwd>/.pi/settings.json ← project-local settings
|
|
552
|
+
```
|
|
553
|
+
|
|
554
|
+
Project-local values override global values. The `SubagentSettings` interface supports additional arbitrary keys (extensible via `[key: string]: unknown`).
|
|
@@ -0,0 +1,61 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project will be documented in this file.
|
|
4
|
+
|
|
5
|
+
The format is based on [Keep a Changelog](https://keepachangelog.com/), and this project adheres to [Semantic Versioning](https://semver.org/).
|
|
6
|
+
|
|
7
|
+
## [Unreleased]
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
|
|
11
|
+
- **Session persistence**: Sub-agent session data is now persisted to the main agent's session tree via `pi.appendEntry()`. Sessions survive agent restarts and crashes, and are automatically reconstructed on session load. Persistence occurs per-sub-agent after each task completes or errors. Stale "running" sessions left over from crashes are auto-converted to "error" status during reconstruction. Resume functionality now works across session restarts. Persistence is fault-tolerant — failures are logged and do not affect delegation.
|
|
12
|
+
- **Timeout extension**: Sub-agents that are actively working (making tool calls) when their timeout expires now get an automatic extension. Each tool call restarts an idle timer. The sub-agent is killed only after `extend_timeout_debounce` seconds (default 30) of no activity. The TUI always shows the original timeout value. Configurable via `subagents.extend_timeout_debounce` setting.
|
|
13
|
+
- **Loop detection**: Sub-agents that repeat the same tool calls consecutively are now automatically killed. When `looping_tool_count` (default 5) consecutive tool call signatures (serialized as JSON) are identical strings, the sub-agent is immediately stopped with an error. Configurable via `subagents.looping_tool_count` setting.
|
|
14
|
+
- `suggestedSkills` profile field — suggests skill names to the sub-agent via `--skill` CLI flags; the model chooses whether to load them.
|
|
15
|
+
- `loadSkills` profile field — pre-loads skill content (SKILL.md body) into the sub-agent's system prompt via `<loaded_skill>` XML injection.
|
|
16
|
+
- `validateProfileSkills()` — mutual exclusivity validation for `suggestedSkills`/`loadSkills` vs `noSkills`.
|
|
17
|
+
- `resolveProfileSkills()` — resolves skill names to file paths (suggestedSkills) or injected content (loadSkills) at delegation time.
|
|
18
|
+
- Skill configuration step in the interactive profile editor (`/profile create`, `/profile edit`).
|
|
19
|
+
|
|
20
|
+
### Changed
|
|
21
|
+
|
|
22
|
+
- Settings loading is now parallelized for better performance.
|
|
23
|
+
|
|
24
|
+
## [0.1.0] — 2026-05-14
|
|
25
|
+
|
|
26
|
+
### Added
|
|
27
|
+
|
|
28
|
+
- `delegate_to_subagents` tool with parallel execution (max 16 tasks, 4 concurrent)
|
|
29
|
+
- `get_subagent_output`, `get_subagent_session`, `list_subagent_profiles` retrieval tools
|
|
30
|
+
- Profile system using markdown+YAML frontmatter files (`~/.pi/agent/agent-profiles/*.md`, `.pi/agent-profiles/*.md`)
|
|
31
|
+
- `/profile` slash command with interactive profile editor
|
|
32
|
+
- Live TUI rolling window display with expand/collapse per sub-agent
|
|
33
|
+
- Per-task timeout with abort escalation (SIGTERM → SIGKILL after 5s)
|
|
34
|
+
- Session resume support via `resume` parameter
|
|
35
|
+
- Debounced TUI updates (50ms)
|
|
36
|
+
- Path shortening and tool call preview formatting
|
|
37
|
+
- In-memory session store with oldest-first eviction (max 32 sessions, 10 runs per session)
|
|
38
|
+
- Profile cache with 5-second TTL
|
|
39
|
+
- `excludeTools` profile field — blacklist of tool names to exclude from the parent session's full tool set. Mutually exclusive with `tools` (allowlist). Resolved at spawn time by computing `all tools - excluded tools` and passing the result via `--tools`.
|
|
40
|
+
- Security: `extraArgs` containing `--tools`, `--no-tools`, or their short/equals-sign forms are now blocked when tool restrictions (`tools`, `excludeTools`, or `noTools`) are active.
|
|
41
|
+
- Comprehensive test coverage: 365 tests (93% coverage) across 13 test files, covering session store, helper functions, profile editor, command handler, TUI rendering, retrieval tools, and profile/spawner modules.
|
|
42
|
+
- ESLint strict config (`eslint.config.js`) replacing Biome — 0 lint errors, no-explicit-any enforced, no-non-null-assertion enforced.
|
|
43
|
+
- GitHub Actions CI workflow (`.github/workflows/ci.yml`) — runs typecheck, lint, and tests on push/PR to main across Node 20 and 22.
|
|
44
|
+
- GitHub Actions publish workflow (`.github/workflows/publish.yml`) — dry-run publish on git tags matching `v*`.
|
|
45
|
+
- Vitest coverage reporting with `@vitest/coverage-v8` — 80% threshold for statements, branches, functions, and lines.
|
|
46
|
+
- `test:coverage` npm script for local coverage reporting.
|
|
47
|
+
- `typecheck` npm script (`tsc --noEmit`) for TypeScript validation.
|
|
48
|
+
|
|
49
|
+
### Changed
|
|
50
|
+
|
|
51
|
+
- `typebox` moved from `devDependencies` to `dependencies` (runtime usage).
|
|
52
|
+
- Added `repository`, `publishConfig`, `files`, and proper `author` to `package.json`.
|
|
53
|
+
- Added `@earendil-works/pi-ai` as `peerDependencies`.
|
|
54
|
+
|
|
55
|
+
### Fixed
|
|
56
|
+
|
|
57
|
+
- Fixed all implicit `any` parameter errors in source and test files.
|
|
58
|
+
- Fixed all non-null assertion (`!`) violations.
|
|
59
|
+
- Fixed all explicit `any` types in source files.
|
|
60
|
+
- Fixed biome formatting errors throughout codebase.
|
|
61
|
+
- Fixed Dirent type errors in profiles.test.ts.
|