mercury-agent 0.5.1 → 0.5.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +6 -5
- package/package.json +1 -1
- package/resources/profiles/coding/mercury-profile.yaml +1 -1
- package/resources/profiles/general/mercury-profile.yaml +1 -1
- package/resources/profiles/research/mercury-profile.yaml +1 -1
- package/resources/templates/mercury.example.yaml +2 -2
- package/src/agent/container-entry.ts +1 -1
- package/src/agent/container-runner.ts +1 -2
- package/src/cli/mercury.ts +1 -1
- package/src/config.ts +1 -1
- package/src/main.ts +1 -1
- package/docs/ARCHITECTURE.md +0 -34
- package/docs/DESIGN.md +0 -42
- package/docs/ROADMAP.md +0 -43
- package/docs/TODOS.md +0 -147
- package/docs/VISION.md +0 -32
- package/docs/archive/.gitkeep +0 -0
- package/docs/archive/applicative-profiles/2026-07-01-applicative-profiles.md +0 -270
- package/docs/archive/applicative-profiles/2026-07-01-caller-bound-capability-token.md +0 -184
- package/docs/archive/applicative-profiles/2026-07-01-dm-auto-space.md +0 -273
- package/docs/archive/summarization/2026-07-02-recent-dev-summary.md +0 -71
- package/docs/backlog/.gitkeep +0 -0
- package/docs/bugs/.gitkeep +0 -0
- package/docs/debug/major/.gitkeep +0 -0
- package/docs/debug/major/2026-07-01-strict-yaml-schema-crash.md +0 -85
- package/docs/debug/major/2026-07-02-bwrap-privileged-linux-docker.md +0 -160
- package/docs/debug/major/2026-07-02-e2big-prompt-delivery-fix.md +0 -382
- package/docs/debug/major/2026-07-02-registry-pull-no-local-fallback.md +0 -194
- package/docs/debug/minor/.gitkeep +0 -0
- package/docs/debug/moderate/.gitkeep +0 -0
- package/docs/debug/summarization/2026-07-02-common-bug-patterns.md +0 -72
- package/docs/html-slides/.gitkeep +0 -0
- package/docs/ideas/.gitkeep +0 -0
- package/docs/ideas/business-extensions.md +0 -48
- package/docs/in-progress/.gitkeep +0 -0
- package/docs/notes/.gitkeep +0 -0
- package/docs/pending-updates/.gitkeep +0 -0
- package/docs/runbooks/gws-oauth-setup.md +0 -211
- package/docs/runbooks/publish-checklist.md +0 -46
- package/docs/templates/TEMPLATE-AUTOPILOT-CONFIG.yaml +0 -35
- package/docs/templates/TEMPLATE-BUG.md +0 -71
- package/docs/templates/TEMPLATE-CI.yml +0 -25
- package/docs/templates/TEMPLATE-DECISIONS.md +0 -11
- package/docs/templates/TEMPLATE-FEATURE.md +0 -127
- package/docs/templates/TEMPLATE-GOAL.md +0 -31
- package/docs/templates/TEMPLATE-IDEA.md +0 -31
- package/docs/templates/TEMPLATE-NOTE.md +0 -27
- package/docs/templates/TEMPLATE-ROADMAP.md +0 -21
|
@@ -1,382 +0,0 @@
|
|
|
1
|
-
# Fix: E2BIG — Argument list too long on bwrap/pi spawn
|
|
2
|
-
|
|
3
|
-
**Status:** Fixed
|
|
4
|
-
**Severity:** P0 — causes container crash on long conversations
|
|
5
|
-
**Affected:** All runtimes (bwrap + gVisor/direct), Docker Desktop especially
|
|
6
|
-
**File:** `src/agent/container-entry.ts`
|
|
7
|
-
**Last updated:** 2026-07-02
|
|
8
|
-
|
|
9
|
-
## Symptom
|
|
10
|
-
|
|
11
|
-
```
|
|
12
|
-
Container failed (exit code 1): Error: E2BIG: argument list too long, posix_spawn 'bwrap'
|
|
13
|
-
```
|
|
14
|
-
|
|
15
|
-
Happens when a conversation has enough history that the assembled CLI args + env vars
|
|
16
|
-
exceed Linux's `ARG_MAX` (~2MB). The user sees: *"Something went wrong processing your request."*
|
|
17
|
-
|
|
18
|
-
## Root Cause
|
|
19
|
-
|
|
20
|
-
`invokePiOnce()` (line ~741) passes both the **system prompt** and **user prompt**
|
|
21
|
-
(which includes full conversation history XML) as CLI arguments:
|
|
22
|
-
|
|
23
|
-
```typescript
|
|
24
|
-
// CURRENT — both prompts are argv strings
|
|
25
|
-
const piArgs = [
|
|
26
|
-
"--print", "--mode", "json", ...sessionArgs,
|
|
27
|
-
"--provider", provider, "--model", model,
|
|
28
|
-
...toolModeArgs, "--no-extensions",
|
|
29
|
-
"-e", "/app/src/extensions/permission-guard.ts",
|
|
30
|
-
"-e", "/app/resources/pi-extensions/subagent/index.ts",
|
|
31
|
-
systemPromptFlag,
|
|
32
|
-
systemPrompt, // <-- can be large (system + ext prompt)
|
|
33
|
-
buildPrompt(payload), // <-- HUGE (conversation history XML)
|
|
34
|
-
];
|
|
35
|
-
```
|
|
36
|
-
|
|
37
|
-
These args are then passed to `spawn("bwrap", [...bwrapArgs, ...piArgs])` or
|
|
38
|
-
`spawn("pi", piArgs)`. Both go through `posix_spawn`/`execve`, which enforces
|
|
39
|
-
`ARG_MAX` on the combined size of all argv + envp strings.
|
|
40
|
-
|
|
41
|
-
The non-bwrap (gVisor) path has the **same latent bug** — it just has slightly
|
|
42
|
-
more headroom because bwrap's ~20 setup args aren't in the way.
|
|
43
|
-
|
|
44
|
-
## Fix: Hybrid stdin + file-path delivery
|
|
45
|
-
|
|
46
|
-
pi already supports both mechanisms natively — **no pi changes needed**:
|
|
47
|
-
|
|
48
|
-
1. **`resolvePromptInput()`** in `resource-loader.js:15-29`: if the value of
|
|
49
|
-
`--system-prompt` / `--append-system-prompt` is a path to an existing file,
|
|
50
|
-
pi reads the file content. Otherwise it uses the string as-is.
|
|
51
|
-
|
|
52
|
-
2. **`readPipedStdin()`** in `main.js:40-55`: when `!process.stdin.isTTY`
|
|
53
|
-
(always true inside Docker with `stdio: ["pipe", ...]`), pi reads stdin
|
|
54
|
-
and prepends it to the initial message. In `--print` mode (which Mercury
|
|
55
|
-
uses), this becomes the user prompt.
|
|
56
|
-
|
|
57
|
-
### Strategy
|
|
58
|
-
|
|
59
|
-
| Prompt | Delivery | Why |
|
|
60
|
-
|--------|----------|-----|
|
|
61
|
-
| System prompt | Write to file in IO_DIR, pass **file path** as arg | `resolvePromptInput` reads it; path string is tiny |
|
|
62
|
-
| User prompt | Pipe via **stdin** | `readPipedStdin` reads it; zero argv footprint |
|
|
63
|
-
| `MERCURY_EXT_SYSTEM_PROMPT` | Append to system prompt file (already concatenated in code) | Covered by the same file-path mechanism |
|
|
64
|
-
|
|
65
|
-
This removes both large strings from argv entirely.
|
|
66
|
-
|
|
67
|
-
---
|
|
68
|
-
|
|
69
|
-
## Implementation Guide
|
|
70
|
-
|
|
71
|
-
All changes are in **`src/agent/container-entry.ts`** unless noted.
|
|
72
|
-
|
|
73
|
-
### Step 1: Add `unlinkSync` to fs import
|
|
74
|
-
|
|
75
|
-
```diff
|
|
76
|
-
import {
|
|
77
|
-
accessSync,
|
|
78
|
-
constants,
|
|
79
|
-
existsSync,
|
|
80
|
-
readdirSync,
|
|
81
|
-
readFileSync,
|
|
82
|
-
renameSync,
|
|
83
|
-
+ unlinkSync,
|
|
84
|
-
writeFileSync,
|
|
85
|
-
} from "node:fs";
|
|
86
|
-
```
|
|
87
|
-
|
|
88
|
-
### Step 2: Update `buildBwrapArgs` to accept optional `ioDir`
|
|
89
|
-
|
|
90
|
-
The system prompt file lives in IO_DIR, which must be visible inside the
|
|
91
|
-
bwrap sandbox. Add it as a **read-only** bind mount.
|
|
92
|
-
|
|
93
|
-
```diff
|
|
94
|
-
-function buildBwrapArgs(workspace: string, command: string[]): string[] {
|
|
95
|
-
+function buildBwrapArgs(workspace: string, command: string[], ioDir?: string): string[] {
|
|
96
|
-
const args: string[] = [
|
|
97
|
-
"--ro-bind", "/usr", "/usr",
|
|
98
|
-
"--symlink", "usr/lib", "/lib",
|
|
99
|
-
"--symlink", "usr/bin", "/bin",
|
|
100
|
-
"--symlink", "usr/sbin", "/sbin",
|
|
101
|
-
];
|
|
102
|
-
if (existsSync("/usr/lib64")) {
|
|
103
|
-
args.push("--symlink", "usr/lib64", "/lib64");
|
|
104
|
-
}
|
|
105
|
-
args.push("--ro-bind", "/app", "/app", "--ro-bind", "/etc", "/etc");
|
|
106
|
-
if (existsSync("/docs")) {
|
|
107
|
-
args.push("--ro-bind", "/docs", "/docs");
|
|
108
|
-
}
|
|
109
|
-
+ if (ioDir) {
|
|
110
|
-
+ args.push("--ro-bind", ioDir, ioDir);
|
|
111
|
-
+ }
|
|
112
|
-
args.push(
|
|
113
|
-
"--bind", "/spaces", "/spaces",
|
|
114
|
-
```
|
|
115
|
-
|
|
116
|
-
### Step 3: Rewrite prompt construction in `invokePiOnce`
|
|
117
|
-
|
|
118
|
-
Replace the current piArgs construction (lines ~720-758) with:
|
|
119
|
-
|
|
120
|
-
```typescript
|
|
121
|
-
function invokePiOnce(
|
|
122
|
-
payload: Payload,
|
|
123
|
-
provider: string,
|
|
124
|
-
model: string,
|
|
125
|
-
capabilities: ModelCapabilities,
|
|
126
|
-
): Promise<PiJsonlParseResult> {
|
|
127
|
-
return new Promise((resolve, reject) => {
|
|
128
|
-
const overridePiPrompt =
|
|
129
|
-
process.env.OVERRIDE_PI_SYSTEM_PROMPT === "true" ||
|
|
130
|
-
process.env.OVERRIDE_PI_SYSTEM_PROMPT === "1";
|
|
131
|
-
|
|
132
|
-
// Combine base system prompt with extension-injected fragments
|
|
133
|
-
let systemPrompt = buildSystemPrompt(
|
|
134
|
-
capabilities,
|
|
135
|
-
payload,
|
|
136
|
-
overridePiPrompt,
|
|
137
|
-
);
|
|
138
|
-
const extPrompt = process.env.MERCURY_EXT_SYSTEM_PROMPT;
|
|
139
|
-
if (extPrompt) {
|
|
140
|
-
systemPrompt = `${systemPrompt}\n\n${extPrompt}`;
|
|
141
|
-
}
|
|
142
|
-
|
|
143
|
-
// Build the user prompt (conversation history + current message)
|
|
144
|
-
const userPrompt = buildPrompt(payload);
|
|
145
|
-
|
|
146
|
-
const sessionArgs = ["--no-session"];
|
|
147
|
-
|
|
148
|
-
const toolModeArgs = capabilities.tools
|
|
149
|
-
? ([] as string[])
|
|
150
|
-
: (["--no-tools", "--no-skills"] as string[]);
|
|
151
|
-
|
|
152
|
-
const systemPromptFlag = overridePiPrompt
|
|
153
|
-
? "--system-prompt"
|
|
154
|
-
: "--append-system-prompt";
|
|
155
|
-
|
|
156
|
-
// --- E2BIG fix: deliver large prompts via file + stdin ---
|
|
157
|
-
//
|
|
158
|
-
// System prompt → temp file in IO_DIR (pi's resolvePromptInput reads files).
|
|
159
|
-
// User prompt → stdin pipe (pi's readPipedStdin reads when !isTTY).
|
|
160
|
-
// This keeps argv small regardless of conversation length.
|
|
161
|
-
const ioDir = process.env.IO_DIR;
|
|
162
|
-
let systemPromptArg: string;
|
|
163
|
-
let systemPromptFile: string | undefined;
|
|
164
|
-
if (ioDir) {
|
|
165
|
-
systemPromptFile = path.join(ioDir, "system-prompt.txt");
|
|
166
|
-
writeFileSync(systemPromptFile, systemPrompt);
|
|
167
|
-
systemPromptArg = systemPromptFile; // pi reads the file, not the path string
|
|
168
|
-
} else {
|
|
169
|
-
systemPromptArg = systemPrompt; // dev fallback: inline (small payloads)
|
|
170
|
-
}
|
|
171
|
-
|
|
172
|
-
// piArgs no longer contains the user prompt — it goes via stdin.
|
|
173
|
-
// The positional message arg is omitted entirely.
|
|
174
|
-
const piArgs = [
|
|
175
|
-
"--print",
|
|
176
|
-
"--mode",
|
|
177
|
-
"json",
|
|
178
|
-
...sessionArgs,
|
|
179
|
-
"--provider",
|
|
180
|
-
provider,
|
|
181
|
-
"--model",
|
|
182
|
-
model,
|
|
183
|
-
...toolModeArgs,
|
|
184
|
-
"--no-extensions",
|
|
185
|
-
"-e",
|
|
186
|
-
"/app/src/extensions/permission-guard.ts",
|
|
187
|
-
"-e",
|
|
188
|
-
"/app/resources/pi-extensions/subagent/index.ts",
|
|
189
|
-
systemPromptFlag,
|
|
190
|
-
systemPromptArg,
|
|
191
|
-
// NOTE: no positional prompt arg — user prompt piped via stdin
|
|
192
|
-
];
|
|
193
|
-
```
|
|
194
|
-
|
|
195
|
-
### Step 4: Update both spawn sites
|
|
196
|
-
|
|
197
|
-
Replace the spawn block (lines ~776-793):
|
|
198
|
-
|
|
199
|
-
```typescript
|
|
200
|
-
const piSpawnedAt = Date.now();
|
|
201
|
-
let proc: ReturnType<typeof spawn>;
|
|
202
|
-
if (useBubblewrap) {
|
|
203
|
-
const bwrapArgs = [
|
|
204
|
-
...buildBwrapArgs(payload.spaceWorkspace, ["pi"], ioDir ?? undefined),
|
|
205
|
-
...piArgs,
|
|
206
|
-
];
|
|
207
|
-
proc = spawn("bwrap", bwrapArgs, {
|
|
208
|
-
stdio: ["pipe", "pipe", "pipe"], // changed: stdin is now "pipe"
|
|
209
|
-
env: process.env,
|
|
210
|
-
});
|
|
211
|
-
} else {
|
|
212
|
-
proc = spawn("pi", piArgs, {
|
|
213
|
-
cwd: payload.spaceWorkspace,
|
|
214
|
-
stdio: ["pipe", "pipe", "pipe"], // changed: stdin is now "pipe"
|
|
215
|
-
env: process.env,
|
|
216
|
-
});
|
|
217
|
-
}
|
|
218
|
-
|
|
219
|
-
// Pipe user prompt via stdin — avoids E2BIG on long conversations.
|
|
220
|
-
// pi's readPipedStdin() reads this when !process.stdin.isTTY (always true in Docker).
|
|
221
|
-
proc.stdin!.on("error", () => {}); // swallow EPIPE if pi exits early
|
|
222
|
-
proc.stdin!.end(userPrompt, "utf8");
|
|
223
|
-
```
|
|
224
|
-
|
|
225
|
-
### Step 5: Clean up system prompt file on close
|
|
226
|
-
|
|
227
|
-
In the `proc.on("close", ...)` handler (line ~815), add cleanup at the top:
|
|
228
|
-
|
|
229
|
-
```diff
|
|
230
|
-
proc.on("close", (code) => {
|
|
231
|
-
+ // Clean up temp system prompt file (belt-and-suspenders; IO_DIR is ephemeral)
|
|
232
|
-
+ if (systemPromptFile) {
|
|
233
|
-
+ try { unlinkSync(systemPromptFile); } catch {}
|
|
234
|
-
+ }
|
|
235
|
-
logTiming("container.pi.done", {
|
|
236
|
-
piDurationMs: Date.now() - piSpawnedAt,
|
|
237
|
-
exitCode: code ?? null,
|
|
238
|
-
});
|
|
239
|
-
```
|
|
240
|
-
|
|
241
|
-
Also add the same cleanup in the `proc.on("error", ...)` handler:
|
|
242
|
-
|
|
243
|
-
```diff
|
|
244
|
-
proc.on("error", (error) => {
|
|
245
|
-
+ if (systemPromptFile) {
|
|
246
|
-
+ try { unlinkSync(systemPromptFile); } catch {}
|
|
247
|
-
+ }
|
|
248
|
-
reject(error);
|
|
249
|
-
});
|
|
250
|
-
```
|
|
251
|
-
|
|
252
|
-
---
|
|
253
|
-
|
|
254
|
-
## P0 companion: Byte-budget guard on history (defense-in-depth)
|
|
255
|
-
|
|
256
|
-
Even with the delivery fix, it's good practice to cap history size. This also
|
|
257
|
-
reduces token waste on very long conversations.
|
|
258
|
-
|
|
259
|
-
In `buildHistoryXml` (line ~533), replace the unbounded entries construction:
|
|
260
|
-
|
|
261
|
-
```typescript
|
|
262
|
-
const HISTORY_CHAR_BUDGET = 400_000; // ~100K tokens — well under model limits
|
|
263
|
-
|
|
264
|
-
function buildHistoryXml(messages: StoredMessage[]): string | null {
|
|
265
|
-
// Pair up user+assistant turns (existing logic unchanged)
|
|
266
|
-
const turns: Array<{ user: StoredMessage; assistant?: StoredMessage }> = [];
|
|
267
|
-
let pendingUser: StoredMessage | null = null;
|
|
268
|
-
|
|
269
|
-
for (const m of messages) {
|
|
270
|
-
if (m.role === "user") {
|
|
271
|
-
if (pendingUser) {
|
|
272
|
-
turns.push({ user: pendingUser });
|
|
273
|
-
}
|
|
274
|
-
pendingUser = m;
|
|
275
|
-
} else if (m.role === "assistant" && pendingUser) {
|
|
276
|
-
turns.push({ user: pendingUser, assistant: m });
|
|
277
|
-
pendingUser = null;
|
|
278
|
-
}
|
|
279
|
-
}
|
|
280
|
-
if (pendingUser) turns.push({ user: pendingUser });
|
|
281
|
-
|
|
282
|
-
if (turns.length === 0) return null;
|
|
283
|
-
|
|
284
|
-
// Build newest-first, stop when budget exhausted
|
|
285
|
-
const entries: string[] = [];
|
|
286
|
-
let usedChars = 0;
|
|
287
|
-
|
|
288
|
-
for (let i = turns.length - 1; i >= 0; i--) {
|
|
289
|
-
const { user, assistant } = turns[i];
|
|
290
|
-
const ts = formatContextTimestamp(user.createdAt);
|
|
291
|
-
const userLine = ` <user>${escapeXmlText(user.content)}</user>`;
|
|
292
|
-
const assistantLine = assistant
|
|
293
|
-
? `\n <assistant>${escapeXmlText(assistant.content)}</assistant>`
|
|
294
|
-
: "";
|
|
295
|
-
const entry = ` <turn timestamp="${ts}">\n${userLine}${assistantLine}\n </turn>`;
|
|
296
|
-
|
|
297
|
-
if (usedChars + entry.length > HISTORY_CHAR_BUDGET) break;
|
|
298
|
-
usedChars += entry.length;
|
|
299
|
-
entries.unshift(entry); // maintain chronological order
|
|
300
|
-
}
|
|
301
|
-
|
|
302
|
-
if (entries.length === 0) return null;
|
|
303
|
-
return `<history>\n${entries.join("\n")}\n</history>`;
|
|
304
|
-
}
|
|
305
|
-
```
|
|
306
|
-
|
|
307
|
-
---
|
|
308
|
-
|
|
309
|
-
## How to verify
|
|
310
|
-
|
|
311
|
-
### 1. Unit test: large payload doesn't crash
|
|
312
|
-
|
|
313
|
-
Create a payload with >128KB of conversation history. Verify `invokePiOnce`
|
|
314
|
-
spawns successfully (no E2BIG). The container needs Docker running.
|
|
315
|
-
|
|
316
|
-
### 2. Manual test: long WhatsApp conversation
|
|
317
|
-
|
|
318
|
-
Find (or create) a space with 50+ messages. Send a new message. Verify the
|
|
319
|
-
agent responds instead of crashing with "Something went wrong."
|
|
320
|
-
|
|
321
|
-
### 3. Regression test: short conversations still work
|
|
322
|
-
|
|
323
|
-
Normal short conversations should be completely unaffected. The prompts reach
|
|
324
|
-
pi with identical content — only the delivery channel changed.
|
|
325
|
-
|
|
326
|
-
### 4. Verify stdin content reaches pi correctly
|
|
327
|
-
|
|
328
|
-
Add a temporary `logTiming("prompt.sizes", { systemPromptLen: systemPrompt.length, userPromptLen: userPrompt.length })` before spawn. Compare with pi's
|
|
329
|
-
output to confirm the full prompt arrived.
|
|
330
|
-
|
|
331
|
-
---
|
|
332
|
-
|
|
333
|
-
## Edge cases to watch
|
|
334
|
-
|
|
335
|
-
| Case | Behavior |
|
|
336
|
-
|------|----------|
|
|
337
|
-
| IO_DIR unset (local dev) | System prompt stays inline in argv; user prompt still goes via stdin. Safe for small dev payloads. |
|
|
338
|
-
| pi exits before reading stdin | `proc.stdin.on("error", () => {})` swallows EPIPE. pi's exit code/stderr still captured normally. |
|
|
339
|
-
| System prompt file path collides with real text | Only triggers if the system prompt string happens to be a valid file path. Extremely unlikely for a multi-KB prompt. If paranoid, use a path with a UUID. |
|
|
340
|
-
| bwrap + IO_DIR mount | IO_DIR is `--ro-bind` (read-only). pi can read the system prompt file but cannot write to IO_DIR. Correct for security. |
|
|
341
|
-
| Retry logic in `runModelChain` | Each retry calls `invokePiOnce` fresh — each writes a new system prompt file and pipes stdin. Previous file is cleaned up on close. No stale state between retries. |
|
|
342
|
-
| `MERCURY_EXT_SYSTEM_PROMPT` | Already concatenated into `systemPrompt` before the file write. Covered automatically. |
|
|
343
|
-
|
|
344
|
-
---
|
|
345
|
-
|
|
346
|
-
## Summary of changes
|
|
347
|
-
|
|
348
|
-
| What | Where | Lines affected |
|
|
349
|
-
|------|-------|----------------|
|
|
350
|
-
| Add `unlinkSync` import | fs import block | 1 line |
|
|
351
|
-
| `buildBwrapArgs` accepts `ioDir` | Function signature + body | ~4 lines |
|
|
352
|
-
| System prompt → file in IO_DIR | `invokePiOnce`, before piArgs | ~8 lines |
|
|
353
|
-
| Remove user prompt from piArgs | `invokePiOnce`, piArgs array | Remove 1 line |
|
|
354
|
-
| stdin: "ignore" → "pipe" | Both spawn sites | 2 lines |
|
|
355
|
-
| Pipe user prompt to stdin | After spawn | 2 lines |
|
|
356
|
-
| Cleanup system prompt file | close + error handlers | 4 lines |
|
|
357
|
-
| **Total** | | **~22 lines changed** |
|
|
358
|
-
| Byte-budget history (optional companion) | `buildHistoryXml` | Replace function body |
|
|
359
|
-
|
|
360
|
-
---
|
|
361
|
-
|
|
362
|
-
## Post-Mortem
|
|
363
|
-
|
|
364
|
-
### Investigation
|
|
365
|
-
Verified the current state of `src/agent/container-entry.ts` against this fix guide before doing any work (bug was flagged as a hanging doc during `/f-doc-sync`, never archived). Grepped and read the relevant functions: `buildBwrapArgs` (line 676), `invokePiOnce` (line 728), `buildHistoryXml` (line 534), and both spawn sites (line 810).
|
|
366
|
-
|
|
367
|
-
### Root Cause
|
|
368
|
-
`invokePiOnce()` passed the system prompt and the full conversation-history XML as `argv` strings to `posix_spawn('bwrap'|'pi', …)`. On long conversations the combined argv + envp size exceeded Linux `ARG_MAX` (~2MB), so `execve` failed with `E2BIG` and the container exited 1.
|
|
369
|
-
|
|
370
|
-
### Fix
|
|
371
|
-
The complete fix described in this guide was **already implemented** in the tree — no new code was required:
|
|
372
|
-
- `unlinkSync` imported and used in the `close`/`error` handlers (lines 854, 865).
|
|
373
|
-
- `buildBwrapArgs(workspace, command, ioDir?)` binds `IO_DIR` read-only into the sandbox (line 679 signature; line 716 `--ro-bind ioDir ioDir`).
|
|
374
|
-
- System prompt written to `IO_DIR/system-prompt.txt` and passed as a file path (pi's `resolvePromptInput` reads it) — lines 765–771.
|
|
375
|
-
- User prompt removed from `argv` and piped via stdin (`proc.stdin?.end(userPrompt, "utf8")`) with both spawn sites using `stdio: ["pipe","pipe","pipe"]` — lines 814–831.
|
|
376
|
-
- Temp system-prompt file cleaned up on close and error — lines 852–865.
|
|
377
|
-
- Optional companion `HISTORY_CHAR_BUDGET = 400_000` applied in `buildHistoryXml` (lines 534, 571) as defense-in-depth.
|
|
378
|
-
|
|
379
|
-
### Lessons
|
|
380
|
-
- Hanging bug docs should be reconciled against the code before opening a fix session — this one was fully resolved but never archived, so it lingered as an apparent P0.
|
|
381
|
-
- Never pass unbounded, caller-influenced data (conversation history) through `argv`; use stdin or a file whose path is the only thing in `argv`. Pair the delivery fix with a size budget so history growth can't regress the limit.
|
|
382
|
-
- `docs/bugs/` and `docs/debug/` are gitignored working docs — archival here is a local move only, with no commit/push.
|
|
@@ -1,194 +0,0 @@
|
|
|
1
|
-
# Bug: mercury run fails when registry pull fails but local build exists
|
|
2
|
-
|
|
3
|
-
**Status:** Fixed
|
|
4
|
-
**Severity:** P1 — blocks users who build locally but have a registry image configured
|
|
5
|
-
**Introduced in:** 0.4.22 (commit 75e8e71)
|
|
6
|
-
**File:** `src/cli/mercury.ts`, `runAction()` (~line 187)
|
|
7
|
-
**Last updated:** 2026-07-02
|
|
8
|
-
|
|
9
|
-
## Symptom
|
|
10
|
-
|
|
11
|
-
```
|
|
12
|
-
$ mercury build # succeeds → mercury-agent:latest exists locally
|
|
13
|
-
$ mercury run
|
|
14
|
-
Image 'ghcr.io/avishai-tsabari/mercury-agent:latest' not found locally, pulling...
|
|
15
|
-
Error response from daemon: unauthorized
|
|
16
|
-
|
|
17
|
-
Error: Failed to pull 'ghcr.io/avishai-tsabari/mercury-agent:latest'.
|
|
18
|
-
If this is a private registry, authenticate first:
|
|
19
|
-
docker login ghcr.io
|
|
20
|
-
```
|
|
21
|
-
|
|
22
|
-
The user has a working local image (`mercury-agent:latest`) from `mercury build`,
|
|
23
|
-
but `mercury run` exits with an error because the configured registry image
|
|
24
|
-
can't be pulled (private registry, no auth, offline, etc.).
|
|
25
|
-
|
|
26
|
-
## Root cause
|
|
27
|
-
|
|
28
|
-
The 0.4.22 change replaced the local-fallback logic with auto-pull. The old
|
|
29
|
-
code tried `mercury-agent:latest` as a fallback when the configured image
|
|
30
|
-
wasn't found. The new code attempts `docker pull` and hard-exits on failure —
|
|
31
|
-
it never checks whether a locally built image exists.
|
|
32
|
-
|
|
33
|
-
### Before (0.4.21)
|
|
34
|
-
|
|
35
|
-
```
|
|
36
|
-
configured image not found?
|
|
37
|
-
→ try mercury-agent:latest locally
|
|
38
|
-
→ found? use it (with warning)
|
|
39
|
-
→ not found? error: pull or build
|
|
40
|
-
```
|
|
41
|
-
|
|
42
|
-
### After (0.4.22)
|
|
43
|
-
|
|
44
|
-
```
|
|
45
|
-
configured image not found?
|
|
46
|
-
→ is registry image? attempt pull
|
|
47
|
-
→ pull fails? hard exit ← BUG: should fall back to local
|
|
48
|
-
→ not registry? error: run mercury build
|
|
49
|
-
```
|
|
50
|
-
|
|
51
|
-
## Fix
|
|
52
|
-
|
|
53
|
-
After a failed pull of a registry image, check if `mercury-agent:latest`
|
|
54
|
-
exists locally before giving up. If it does, use it with a warning.
|
|
55
|
-
|
|
56
|
-
### Implementation (lines ~187–213 of `src/cli/mercury.ts`)
|
|
57
|
-
|
|
58
|
-
Replace the current block:
|
|
59
|
-
|
|
60
|
-
```typescript
|
|
61
|
-
if (imageCheck.status !== 0) {
|
|
62
|
-
const isRegistryImage = imageName.includes("/");
|
|
63
|
-
if (isRegistryImage) {
|
|
64
|
-
console.log(`Image '${imageName}' not found locally, pulling...`);
|
|
65
|
-
const pull = spawnSync("docker", ["pull", imageName], {
|
|
66
|
-
stdio: "inherit",
|
|
67
|
-
});
|
|
68
|
-
if (pull.signal) {
|
|
69
|
-
process.exit(1);
|
|
70
|
-
}
|
|
71
|
-
if (pull.status !== 0) {
|
|
72
|
-
const firstSegment = imageName.split("/")[0];
|
|
73
|
-
const registry = firstSegment.includes(".")
|
|
74
|
-
? firstSegment
|
|
75
|
-
: "docker.io";
|
|
76
|
-
console.error(`\nError: Failed to pull '${imageName}'.`);
|
|
77
|
-
console.error(
|
|
78
|
-
"If this is a private registry, authenticate first:\n" +
|
|
79
|
-
` docker login ${registry}`,
|
|
80
|
-
);
|
|
81
|
-
process.exit(1);
|
|
82
|
-
}
|
|
83
|
-
} else {
|
|
84
|
-
console.error(`Error: Container image '${imageName}' not found.`);
|
|
85
|
-
console.error("Run 'mercury build' to build it.");
|
|
86
|
-
process.exit(1);
|
|
87
|
-
}
|
|
88
|
-
}
|
|
89
|
-
```
|
|
90
|
-
|
|
91
|
-
With:
|
|
92
|
-
|
|
93
|
-
```typescript
|
|
94
|
-
if (imageCheck.status !== 0) {
|
|
95
|
-
const isRegistryImage = imageName.includes("/");
|
|
96
|
-
let resolved = false;
|
|
97
|
-
|
|
98
|
-
if (isRegistryImage) {
|
|
99
|
-
console.log(`Image '${imageName}' not found locally, pulling...`);
|
|
100
|
-
const pull = spawnSync("docker", ["pull", imageName], {
|
|
101
|
-
stdio: "inherit",
|
|
102
|
-
});
|
|
103
|
-
if (pull.signal) {
|
|
104
|
-
process.exit(1);
|
|
105
|
-
}
|
|
106
|
-
if (pull.status === 0) {
|
|
107
|
-
resolved = true;
|
|
108
|
-
}
|
|
109
|
-
}
|
|
110
|
-
|
|
111
|
-
// Fallback: if the configured image isn't available (pull failed, not a
|
|
112
|
-
// registry image, or never pulled), try the local build tag.
|
|
113
|
-
if (!resolved) {
|
|
114
|
-
const localTag = "mercury-agent:latest";
|
|
115
|
-
if (imageName !== localTag) {
|
|
116
|
-
const localCheck = spawnSync("docker", ["image", "inspect", localTag], {
|
|
117
|
-
stdio: "pipe",
|
|
118
|
-
});
|
|
119
|
-
if (localCheck.status === 0) {
|
|
120
|
-
console.log(
|
|
121
|
-
`\nℹ️ Using locally built ${localTag} (configured image unavailable)\n`,
|
|
122
|
-
);
|
|
123
|
-
process.env.MERCURY_AGENT_IMAGE = localTag;
|
|
124
|
-
resolved = true;
|
|
125
|
-
}
|
|
126
|
-
}
|
|
127
|
-
}
|
|
128
|
-
|
|
129
|
-
if (!resolved) {
|
|
130
|
-
if (isRegistryImage) {
|
|
131
|
-
const firstSegment = imageName.split("/")[0];
|
|
132
|
-
const registry = firstSegment.includes(".")
|
|
133
|
-
? firstSegment
|
|
134
|
-
: "docker.io";
|
|
135
|
-
console.error(`\nError: Failed to pull '${imageName}'.`);
|
|
136
|
-
console.error(
|
|
137
|
-
"If this is a private registry, authenticate first:\n" +
|
|
138
|
-
` docker login ${registry}\n` +
|
|
139
|
-
"Or build locally:\n" +
|
|
140
|
-
" mercury build",
|
|
141
|
-
);
|
|
142
|
-
} else {
|
|
143
|
-
console.error(`Error: Container image '${imageName}' not found.`);
|
|
144
|
-
console.error("Run 'mercury build' to build it.");
|
|
145
|
-
}
|
|
146
|
-
process.exit(1);
|
|
147
|
-
}
|
|
148
|
-
}
|
|
149
|
-
```
|
|
150
|
-
|
|
151
|
-
### Behavior after fix
|
|
152
|
-
|
|
153
|
-
```
|
|
154
|
-
configured image not found?
|
|
155
|
-
→ is registry image? attempt pull
|
|
156
|
-
→ pull succeeds? use it ✓
|
|
157
|
-
→ pull fails? continue to fallback
|
|
158
|
-
→ try mercury-agent:latest locally
|
|
159
|
-
→ found? use it (with info message) ✓
|
|
160
|
-
→ not found? error with both suggestions (login + build) ✓
|
|
161
|
-
```
|
|
162
|
-
|
|
163
|
-
## How to verify
|
|
164
|
-
|
|
165
|
-
1. Build locally: `mercury build`
|
|
166
|
-
2. Ensure no GHCR auth: `docker logout ghcr.io`
|
|
167
|
-
3. Set `.env` to `MERCURY_AGENT_IMAGE=ghcr.io/avishai-tsabari/mercury-agent:latest`
|
|
168
|
-
4. Run `mercury run` — should fall back to local image with info message
|
|
169
|
-
5. Remove local image: `docker rmi mercury-agent:latest`
|
|
170
|
-
6. Run `mercury run` — should error with both `docker login` and `mercury build` suggestions
|
|
171
|
-
|
|
172
|
-
---
|
|
173
|
-
|
|
174
|
-
## Post-Mortem
|
|
175
|
-
|
|
176
|
-
### Investigation
|
|
177
|
-
Triaged from a hanging bug doc during `/f-doc-sync`. First confirmed the fix was NOT already present: grepped `src/` for any local-fallback logic (`Using locally built`, `localTag`, `resolved`) — zero hits — and read `runAction()` in `src/cli/mercury.ts` (lines 186–216), which was still the buggy 0.4.22 version that hard-exits on pull failure. Fixed in an isolated worktree.
|
|
178
|
-
|
|
179
|
-
### Root Cause
|
|
180
|
-
The 0.4.22 auto-pull rewrite dropped the pre-existing local-image fallback. When the configured image was a registry ref and `docker pull` failed (private registry / no auth / offline), `runAction()` called `process.exit(1)` immediately, never checking whether a locally built `mercury-agent:latest` existed.
|
|
181
|
-
|
|
182
|
-
### Fix
|
|
183
|
-
`src/cli/mercury.ts`, `runAction()` (committed to main as `1c120bf`):
|
|
184
|
-
- Introduced a `resolved` flag. A registry pull sets it on success; on failure, control falls through instead of exiting.
|
|
185
|
-
- Added a fallback block: if unresolved and the configured image isn't already `mercury-agent:latest`, `docker image inspect mercury-agent:latest`; if present, use it with an info message.
|
|
186
|
-
- Only after both paths fail does it `process.exit(1)`, now suggesting **both** `docker login` and `mercury build`.
|
|
187
|
-
- **Beyond the original fix guide:** the child is spawned with `env: { ...process.env, ...envVars }`, and `envVars` (loaded from `.env`, which typically pins `MERCURY_AGENT_IMAGE` to the registry) wins that merge. Setting only `process.env.MERCURY_AGENT_IMAGE` would have been silently clobbered — the repro scenario itself. The fix therefore also sets `envVars.MERCURY_AGENT_IMAGE = localTag`. Confirmed load-bearing by the session code review.
|
|
188
|
-
|
|
189
|
-
Verification: `bun run typecheck` + `bun run lint` clean; `bun test` → 1039 pass / 0 fail; session code review returned "No issues found."
|
|
190
|
-
|
|
191
|
-
### Lessons
|
|
192
|
-
- When a spawn merges multiple env sources (`{ ...process.env, ...envVars }`), an override must be written to the **last-wins** source, not just `process.env` — otherwise it's silently reverted.
|
|
193
|
-
- Behavior-replacing rewrites (the 0.4.22 auto-pull change) should preserve existing fallbacks explicitly; a regression test for "pull fails but local image exists" would have caught this.
|
|
194
|
-
- Reconcile hanging bug docs against current code before opening a fix — two sibling P0/P1 docs (`e2big`, `bwrap-privileged`) turned out already fixed; only this one needed work.
|
|
File without changes
|
|
File without changes
|
|
@@ -1,72 +0,0 @@
|
|
|
1
|
-
# Common Bug Patterns
|
|
2
|
-
|
|
3
|
-
**Date:** 2026-07-02
|
|
4
|
-
**Covers:** 1 bug report (1 major, 0 moderate, 0 minor) · 2026-07-01 → 2026-07-01
|
|
5
|
-
|
|
6
|
-
This is the first bug-pattern summary. It covers a single major incident — the strict-YAML-schema crash — but that one bug braids together several distinct, reusable failure patterns worth capturing before they recur: over-strict input validation, unbounded restart loops, and the fragile-transport amplification that turns any crash into extended downtime. The dominant theme is **conflation**: treating "unknown input" and "invalid input" as the same failure, and treating "one crash" as harmless when the surrounding system amplifies it.
|
|
7
|
-
|
|
8
|
-
## Findings
|
|
9
|
-
|
|
10
|
-
### 1. Over-strict input validation (unknown key ≠ invalid value)
|
|
11
|
-
|
|
12
|
-
Schemas that reject unrecognized input with the same severity as malformed input turn benign drift (typos, renamed fields, version skew) into hard failures.
|
|
13
|
-
|
|
14
|
-
| Bug | What went wrong |
|
|
15
|
-
|-----|-----------------|
|
|
16
|
-
| strict-yaml-schema-crash | `mercuryFileSchema` used Zod `.strict()` on all 13 nested objects, so a single unknown `mercury.yaml` key (e.g. renamed `admin_numbers` → `admin_ids`) crashed startup with the same severity as an invalid value. |
|
|
17
|
-
|
|
18
|
-
**Rule:** Separate "unknown key" (warn, strip, continue) from "invalid value" (fail fast). Default config schemas to strip-and-warn on unrecognized keys; reserve hard failure for values that are actually wrong.
|
|
19
|
-
|
|
20
|
-
### 2. Version-skew intolerance
|
|
21
|
-
|
|
22
|
-
Config that must be exactly in sync with the running binary breaks on every field rename, punishing the normal lifecycle of upgrades and rollbacks.
|
|
23
|
-
|
|
24
|
-
| Bug | What went wrong |
|
|
25
|
-
|-----|-----------------|
|
|
26
|
-
| strict-yaml-schema-crash | A field renamed between 0.4.26 and 0.4.27 (`admin_numbers` → `admin_ids`) crashed any deployment whose yaml hadn't been updated in lockstep. |
|
|
27
|
-
|
|
28
|
-
**Rule:** Any feature that adds or renames a config field must assume a user on version N may run a yaml from version N±1. Forward/backward-unknown keys should degrade gracefully, not crash.
|
|
29
|
-
|
|
30
|
-
### 3. Unbounded restart loops amplify a single failure
|
|
31
|
-
|
|
32
|
-
A deterministic startup error plus an unbounded auto-restarter converts one crash into an infinite crash loop, removing any window for human intervention.
|
|
33
|
-
|
|
34
|
-
| Bug | What went wrong |
|
|
35
|
-
|-----|-----------------|
|
|
36
|
-
| strict-yaml-schema-crash | PM2's default unlimited restarts retried the identical failing config forever, each attempt failing the same way. |
|
|
37
|
-
|
|
38
|
-
**Rule:** Configure `max_restarts` + `min_uptime` on process supervisors so a deterministic failure stops after N attempts and alerts, rather than looping. A restart policy is not a substitute for handling the error.
|
|
39
|
-
|
|
40
|
-
### 4. Fragile-transport amplification
|
|
41
|
-
|
|
42
|
-
When a downstream dependency punishes rapid reconnection, an unrelated crash loop escalates into a much costlier outage than the original bug.
|
|
43
|
-
|
|
44
|
-
| Bug | What went wrong |
|
|
45
|
-
|-----|-----------------|
|
|
46
|
-
| strict-yaml-schema-crash | The crash loop drove rapid WhatsApp/Baileys reconnects, which Meta treats as abuse → session revoked (reason=401) → manual QR re-scan required for recovery. |
|
|
47
|
-
|
|
48
|
-
**Rule:** Treat any crash loop as catastrophic when a session-based/unofficial transport is in play. Isolate transport-session lifecycle from process restarts, and prefer a circuit breaker that keeps the host alive while surfacing the error.
|
|
49
|
-
|
|
50
|
-
## Summary table
|
|
51
|
-
|
|
52
|
-
| Pattern | Count | Worst severity |
|
|
53
|
-
|---------|-------|----------------|
|
|
54
|
-
| Over-strict input validation | 1 | major |
|
|
55
|
-
| Version-skew intolerance | 1 | major |
|
|
56
|
-
| Unbounded restart loops | 1 | major |
|
|
57
|
-
| Fragile-transport amplification | 1 | major |
|
|
58
|
-
|
|
59
|
-
## Comparison with previous summary
|
|
60
|
-
|
|
61
|
-
None — this is the first bug-pattern summary. Future summaries should track whether validation-strictness and restart-loop patterns recur once the fixes above land.
|
|
62
|
-
|
|
63
|
-
## Decisions
|
|
64
|
-
|
|
65
|
-
1. **Adopt strip-and-warn as the config default** across all schemas; audit remaining `.strict()` usages for the same class of crash.
|
|
66
|
-
2. **Set supervisor restart limits** (`max_restarts`, `min_uptime`) plus an alert on limit-hit, so no deterministic failure can loop indefinitely.
|
|
67
|
-
3. **Design a container/transport circuit breaker** (the bug's own Phase 2) so container-level errors log-and-recover instead of crashing the host and endangering the WhatsApp session.
|
|
68
|
-
|
|
69
|
-
## Open Questions
|
|
70
|
-
|
|
71
|
-
- Should config validation ever hard-fail on unknown keys in a "strict mode" opt-in for CI, while defaulting to warn in production?
|
|
72
|
-
- What is the right `max_restarts` threshold given Baileys' revocation sensitivity — and should transport reconnection back off independently of process restarts?
|
|
File without changes
|
package/docs/ideas/.gitkeep
DELETED
|
File without changes
|