pi-anti-doom-loop 0.0.10 → 0.0.12
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +33 -0
- package/README.md +67 -14
- package/extensions/controller.ts +70 -5
- package/extensions/detector.ts +210 -3
- package/extensions/index.ts +16 -2
- package/package.json +16 -17
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,39 @@
|
|
|
2
2
|
|
|
3
3
|
All notable changes to **pi-anti-doom-loop**.
|
|
4
4
|
|
|
5
|
+
## [0.0.12] — 2026-10-01
|
|
6
|
+
|
|
7
|
+
### Added
|
|
8
|
+
|
|
9
|
+
- **E2E tests against a real pi ≥ 0.99** (`tests/e2e/pi.test.ts` + `tests/e2e/pi-rig.ts`) — the rig downloads the real `@earendil-works/pi-coding-agent@0.99.2` (cached), runs pi's RPC entry headlessly with the repo extension loaded, and drives sessions with an in-process OpenAI-compatible fake LLM, fully offline. Verified: the extension blocks a scripted identical-`bash` doom loop in a real session; a **real codemode worker** executing a script that polls a file 5× inside ONE `codemode` call is blocked by the nested 2× rule and the script fails with the `BLOCKED by anti-doom-loop` reason; the recommended distinct `Promise.allSettled` batch completes untouched. Skips cleanly when the npm registry is unreachable. Run with `pnpm run e2e:pi`.
|
|
10
|
+
- **Codemode nested-loop fixtures** — `codemodeNestedLoop` (in-script poll loop), `codemodeErrorRetryLoop` (identical retry after an in-script error), `codemodeCrossScriptRepeat` (second script re-issuing a blocked signature → escalation), and `codemodeHealthyBatch` (the documented `Promise.allSettled` pattern stays unblocked), each driven through the controller exactly as pi delivers nested events, plus a contrast test proving a 2× loop blocks nested while staying under the top-level 3× threshold.
|
|
11
|
+
|
|
12
|
+
### Changed
|
|
13
|
+
|
|
14
|
+
- **Tooling migrated to pnpm (latest, 12.8.1)** — `packageManager` pin, `pnpm-lock.yaml` with the default virtual store (`node_modules/.pnpm`), all scripts and docs use `pnpm`, the release workflow installs via `pnpm/action-setup` + `--frozen-lockfile` and publishes with `pnpm publish --no-git-checks`; the tarball e2e asserts via `pnpm pack` and pins the tooling.
|
|
15
|
+
|
|
16
|
+
### Fixed
|
|
17
|
+
|
|
18
|
+
- **Fresh installs failed under pnpm 12** — unapproved dependency build scripts made `pnpm install --frozen-lockfile` exit `ERR_PNPM_IGNORED_BUILDS`, which would have aborted the release workflow before the quality gate. The dev-only transitives pi pulls (`esbuild`, `protobufjs`, `@google/genai`) are now explicitly denied in `pnpm-workspace.yaml`; their binaries come from optional platform packages.
|
|
19
|
+
- **`oxlint-tsgolint` pin had no matching artifact** — `package.json` asked for `7.0.2003` while `@effect/tsgo` ships its patched tsgolint for `7.0.2001` only. The devDependency now matches the artifact (the workspace override already pinned it), so `prepare` no longer looks for a missing binary.
|
|
20
|
+
- **Bounded codemode memory** — a script's window now keeps only its most recent 100 calls (args are capped at 100 KB each), so a long-running script cannot pin unbounded memory; in-script loops repeat adjacent calls and still block at 2×.
|
|
21
|
+
- **Real-pi e2e reported green when it never ran** — offline runs returned early from each test, which `node:test` records as passing. They now report `skipped`, so `pnpm test` can no longer claim coverage it did not execute.
|
|
22
|
+
|
|
23
|
+
### Tests
|
|
24
|
+
|
|
25
|
+
- 122 → 133: 4 real-pi e2e sessions, 6 codemode fixture tests, pnpm-pack artifact assertions, and the bounded nested-window case.
|
|
26
|
+
|
|
27
|
+
## [0.0.11] — 2026-10-01
|
|
28
|
+
|
|
29
|
+
### Added
|
|
30
|
+
|
|
31
|
+
- **Codemode awareness (pi ≥ 0.99)** — pi's `codemode` tool runs model-written scripts that call other tools via `tools.<name>(args)`; those nested calls carry `parentToolCallId` in the `tool_call`/`tool_result` events and previously bypassed detection entirely (a script can issue hundreds of calls inside ONE top-level tool call). The extension now tracks them: identical calls inside one script block at **2×** (stricter than the top-level 3× — a repeated in-script call is never progress; store the result and reuse it), near-identical args count toward the same rule, and an identical retry of a call that already **errored** in the same script blocks immediately. Nested traffic is windowed **per script** and fully isolated from the top-level window in both directions. A NEW script re-issuing a signature blocked in an earlier script this prompt **escalates** (aborts the turn), mirroring top-level block escalation. `PI_ANTI_LOOP_TOOLS_EXCLUDE` applies to nested calls; nested identity is storage-capped (100 KB per call, most recent 100 calls per script) so huge payloads (whole-file reads) stay cheap; `/loopcheck` reports `nested: calls=N blocks=M`.
|
|
32
|
+
- **The information wall** — a nested block reason reaches the _script_ as a tool error, not the model (pi delivers only the script's return value). If a script swallows block errors, the loop signal was lost. The extension now arms a one-shot **steer** on the next assistant `message_end` after any nested block, telling the model to call each tool once per script and reuse results — so a script that ignores tool errors can no longer silently eat the loop signal.
|
|
33
|
+
|
|
34
|
+
### Tests
|
|
35
|
+
|
|
36
|
+
- 99 → 122: nested unit semantics (2× rule, per-script isolation, error-retry, cross-script escalation, payload cap, exclude-list), adapter wiring (`parentToolCallId` forwarding, `NESTED_STEER_TEXT` via `sendMessage`, escalation aborts), and nested fuzz streams (never throw, injected in-script loops always block, random unique-args streams never false-positive).
|
|
37
|
+
|
|
5
38
|
## [0.0.10] — 2026-09-23
|
|
6
39
|
|
|
7
40
|
### Fixed
|
package/README.md
CHANGED
|
@@ -51,7 +51,7 @@ Environment variables, read at session/prompt start:
|
|
|
51
51
|
| `PI_ANTI_LOOP_TIME_WINDOW` | `0` | Elapsed-time window in ms (`0` = disabled, count-only): evicts window entries older than this so slow chronic loops over a long session are caught |
|
|
52
52
|
| `PI_ANTI_LOOP_FAIL_RATE` | `0` | Fail-rate block threshold `0..1` (`0` = disabled): block when a tool's error share of its in-window calls reaches this |
|
|
53
53
|
| `PI_ANTI_LOOP_FAIL_RATE_MIN` | `3` | Minimum calls before the fail-rate window can block |
|
|
54
|
-
| `PI_ANTI_LOOP_TOOLS_EXCLUDE` | — | Comma-separated tool names to disable detection for entirely (never block, never enter the window)
|
|
54
|
+
| `PI_ANTI_LOOP_TOOLS_EXCLUDE` | — | Comma-separated tool names to disable detection for entirely (never block, never enter the window — applies to nested codemode calls too) |
|
|
55
55
|
| `PI_ANTI_LOOP_DISABLE` | — | Set to `1` to disable the extension entirely |
|
|
56
56
|
|
|
57
57
|
## Command
|
|
@@ -61,6 +61,38 @@ Environment variables, read at session/prompt start:
|
|
|
61
61
|
- `/loopcheck suspend` — pause detection until the next prompt (escape hatch for intentional repetition)
|
|
62
62
|
- `/loopcheck resume` — re-enable detection early
|
|
63
63
|
|
|
64
|
+
### Codemode (pi ≥ 0.99): nested script calls
|
|
65
|
+
|
|
66
|
+
pi's `codemode` tool lets the model run a JavaScript script that calls other
|
|
67
|
+
tools via `tools.<name>(args)` — a script can loop hundreds of calls inside
|
|
68
|
+
ONE top-level tool call, which per-call detection never sees as a streak.
|
|
69
|
+
Since pi 0.99, those nested calls still flow through the `tool_call`/
|
|
70
|
+
`tool_result` events tagged with `parentToolCallId` — and this extension
|
|
71
|
+
tracks them:
|
|
72
|
+
|
|
73
|
+
- **Stricter in-script threshold** — inside one script, the 2nd identical
|
|
74
|
+
call blocks (top-level detection needs 3). A repeated call inside a script
|
|
75
|
+
is never progress: the script can store its first result in a variable and
|
|
76
|
+
reuse it. Near-identical args (sleep-wrapped polls, flag drift) count too.
|
|
77
|
+
- **Error-retry rule** — an identical retry of a call that already errored
|
|
78
|
+
inside the same script blocks immediately with a "the identical call
|
|
79
|
+
already failed inside this script" reason.
|
|
80
|
+
- **Script-level isolation** — nested traffic never enters the top-level
|
|
81
|
+
window and vice versa, so script loops cannot hide from their own 2× rule
|
|
82
|
+
and legitimate top-level repetition is never inflated by scripts.
|
|
83
|
+
- **The information wall** — a nested block reason reaches the _script_ as a
|
|
84
|
+
tool error, not the model (only the script's return value does). When a
|
|
85
|
+
script swallows it anyway, the next assistant message gets a one-shot steer
|
|
86
|
+
telling the model to call each tool once and reuse results — so the loop
|
|
87
|
+
signal can never be silently eaten by a script that ignores tool errors.
|
|
88
|
+
- **Cross-script escalation** — counters are per-script (a new script starts
|
|
89
|
+
clean), but re-issuing a signature blocked in an earlier script this prompt
|
|
90
|
+
escalates and aborts the turn, mirroring top-level block-escalation.
|
|
91
|
+
|
|
92
|
+
Huge nested payloads (whole-file reads) are identity-capped (100 KB per call)
|
|
93
|
+
and each script keeps only its most recent 100 calls, so memory stays bounded. `/loopcheck` reports `nested: calls=N blocks=M`. Disable with
|
|
94
|
+
`PI_ANTI_LOOP_DISABLE=1` as usual.
|
|
95
|
+
|
|
64
96
|
## Token-cost awareness
|
|
65
97
|
|
|
66
98
|
The detector estimates tokens burned on redundant repeats (~4 chars/token) and
|
|
@@ -77,29 +109,50 @@ Works with any model — cheap models just trigger it more often.
|
|
|
77
109
|
|
|
78
110
|
## Development
|
|
79
111
|
|
|
80
|
-
Requires Node 22.18+ (
|
|
112
|
+
Requires Node 22.18+ and [pnpm](https://pnpm.io) 12 (pinned via `packageManager`; the install uses pnpm's virtual store at `node_modules/.pnpm`).
|
|
81
113
|
|
|
82
114
|
```bash
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
115
|
+
pnpm install
|
|
116
|
+
pnpm test # node --test: unit + fixture + fuzz + integration + e2e (incl. real-pi e2e)
|
|
117
|
+
pnpm run check # pnpm test + tsc + oxlint --deny-warnings + oxfmt
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
### E2E against real pi ≥ 0.99
|
|
121
|
+
|
|
122
|
+
`tests/e2e/pi.test.ts` downloads the real `@earendil-works/pi-coding-agent@0.99.2`
|
|
123
|
+
once (cached in the OS temp dir), then drives it headlessly: an in-process
|
|
124
|
+
OpenAI-compatible fake LLM scripts the assistant turns, pi's RPC entry runs
|
|
125
|
+
with THIS repo's extension loaded, and the assertions read the real RPC event
|
|
126
|
+
stream. Verified scenarios:
|
|
127
|
+
|
|
128
|
+
- the extension loads into pi 0.99 and blocks a scripted identical-`bash` doom
|
|
129
|
+
loop with the instructive block reason
|
|
130
|
+
- a **real codemode worker** executing a script that polls a file 5× inside
|
|
131
|
+
ONE `codemode` call gets blocked by the nested 2× rule — the script fails
|
|
132
|
+
with `BLOCKED by anti-doom-loop — inside one codemode script…`
|
|
133
|
+
- the recommended distinct `Promise.allSettled` batch completes untouched
|
|
134
|
+
|
|
135
|
+
Everything is offline (no API keys); if the npm registry is unreachable the
|
|
136
|
+
real-pi tests skip cleanly instead of failing. Run just them with:
|
|
137
|
+
|
|
138
|
+
```bash
|
|
139
|
+
pnpm run e2e:pi
|
|
86
140
|
```
|
|
87
141
|
|
|
88
142
|
### Test suite (Node built-in runner, no framework)
|
|
89
143
|
|
|
90
|
-
| Suite | File | What it proves
|
|
91
|
-
| ----------- | --------------------------------------------- |
|
|
92
|
-
| unit | `tests/unit.test.ts` | detector semantics: repeat/failure/text signals, window eviction, options clamping, helpers
|
|
93
|
-
| fixture | `tests/fixtures.ts` + `tests/fixture.test.ts` | real doom-loop transcripts (CI-log loops, verbatim repeats) are caught; healthy sessions are not
|
|
94
|
-
| fuzz | `tests/fuzz.test.ts` | seeded random streams: never throws, no false positives, injected loops always block, canonical stability
|
|
95
|
-
| integration | `tests/integration.test.ts` | controller + `index.ts` adapter driven through a fake `PiLike`: blocks, escalations, aborts, resets, `/loopcheck`
|
|
96
|
-
| e2e | `tests/e2e.test.ts`
|
|
144
|
+
| Suite | File | What it proves |
|
|
145
|
+
| ----------- | --------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
146
|
+
| unit | `tests/unit.test.ts` | detector semantics: repeat/failure/text/nested signals, window eviction, options clamping, helpers |
|
|
147
|
+
| fixture | `tests/fixtures.ts` + `tests/fixture.test.ts` | real doom-loop transcripts (CI-log loops, verbatim repeats, codemode script loops) are caught; healthy sessions are not |
|
|
148
|
+
| fuzz | `tests/fuzz.test.ts` | seeded random streams: never throws, no false positives, injected loops always block, canonical stability |
|
|
149
|
+
| integration | `tests/integration.test.ts` | controller + `index.ts` adapter driven through a fake `PiLike`: blocks, escalations, aborts, resets, `/loopcheck` |
|
|
150
|
+
| e2e | `tests/e2e.test.ts` + `tests/e2e/pi.test.ts` | real subprocesses: detector self-check, version guard, tarball contents (extensions/scripts ship, tests don't), and full sessions against a real pi ≥ 0.99 with a scripted fake LLM |
|
|
97
151
|
|
|
98
152
|
> `peerDependencies` pins `@earendil-works/pi-coding-agent` at `"*"` on purpose — the
|
|
99
153
|
> [pi packages docs](https://pi.dev/docs/latest/packages) require an unbounded range for
|
|
100
154
|
> pi-core packages (pi provides them at runtime). The extension loads `.ts` directly via
|
|
101
155
|
> jiti, so no build step ships; `prepublishOnly` runs the full quality gate before publish.
|
|
102
|
-
|
|
103
156
|
> When a call is blocked, escalation still works without recording it in the window:
|
|
104
157
|
> re-issuing the identical call increments a per-signature block counter and aborts the
|
|
105
158
|
> turn on the second block. Thresholds are clamped to a minimum of 2 so a bad config
|
|
@@ -117,7 +170,7 @@ git tag v0.0.1
|
|
|
117
170
|
git push origin v0.0.1
|
|
118
171
|
```
|
|
119
172
|
|
|
120
|
-
CI runs the quality gate, then the **version bump guard** (`
|
|
173
|
+
CI runs the quality gate with pnpm, then the **version bump guard** (`pnpm run guard`): it blocks publishing if the version is already on npm or the tag doesn't match `package.json`. After the first publish, the [pi.dev gallery](https://pi.dev/packages) picks the package up automatically via the `pi-package` keyword.
|
|
121
174
|
|
|
122
175
|
## License
|
|
123
176
|
|
package/extensions/controller.ts
CHANGED
|
@@ -20,11 +20,15 @@ export interface ToolCallEventLite {
|
|
|
20
20
|
toolName: string;
|
|
21
21
|
toolCallId: string;
|
|
22
22
|
input: unknown;
|
|
23
|
+
/** Set when another tool (for example a codemode script) issued this call (pi ≥ 0.99). */
|
|
24
|
+
parentToolCallId?: string;
|
|
23
25
|
}
|
|
24
26
|
export interface ToolResultEventLite {
|
|
25
27
|
toolName: string;
|
|
26
28
|
toolCallId: string;
|
|
27
29
|
isError: boolean;
|
|
30
|
+
/** Set when another tool (for example a codemode script) issued this call (pi ≥ 0.99). */
|
|
31
|
+
parentToolCallId?: string;
|
|
28
32
|
}
|
|
29
33
|
export interface MessageEndEventLite {
|
|
30
34
|
message: { role: string; content?: unknown };
|
|
@@ -54,6 +58,10 @@ export interface ToolCallOutcome {
|
|
|
54
58
|
reason: string;
|
|
55
59
|
/** True when this exact call was blocked before — caller should abort the turn. */
|
|
56
60
|
escalate: boolean;
|
|
61
|
+
/** True for calls a codemode script made (`parentToolCallId` set, pi ≥ 0.99).
|
|
62
|
+
* Their block reason reaches the script as a tool error, not the model —
|
|
63
|
+
* the adapter must also steer so the model learns about the loop. */
|
|
64
|
+
nested?: boolean;
|
|
57
65
|
}
|
|
58
66
|
|
|
59
67
|
export interface TextLoopOutcome {
|
|
@@ -61,16 +69,29 @@ export interface TextLoopOutcome {
|
|
|
61
69
|
action: "steer" | "abort";
|
|
62
70
|
/** When aborting: also queue a single fresh-resume directive (bounded). */
|
|
63
71
|
resume: boolean;
|
|
72
|
+
/** True when this steer surfaces a codemode script's swallowed nested block. */
|
|
73
|
+
nested?: boolean;
|
|
64
74
|
}
|
|
65
75
|
|
|
66
76
|
/** How many auto-resumes per session before we hand control back for real. */
|
|
67
77
|
export const RESUME_BUDGET = 1;
|
|
68
78
|
|
|
69
79
|
export interface AntiLoopController {
|
|
70
|
-
/** Returns a block decision for a tool call, or null to let it run.
|
|
71
|
-
|
|
80
|
+
/** Returns a block decision for a tool call, or null to let it run.
|
|
81
|
+
* `parentToolCallId` is set for calls a codemode script made (pi ≥ 0.99). */
|
|
82
|
+
onToolCall(
|
|
83
|
+
toolName: string,
|
|
84
|
+
input: ToolInput,
|
|
85
|
+
toolCallId: string,
|
|
86
|
+
parentToolCallId?: string,
|
|
87
|
+
): ToolCallOutcome | null;
|
|
72
88
|
/** Record a finished tool result (blocked calls' results are ignored). */
|
|
73
|
-
onToolResult(
|
|
89
|
+
onToolResult(
|
|
90
|
+
toolName: string,
|
|
91
|
+
toolCallId: string,
|
|
92
|
+
isError: boolean,
|
|
93
|
+
parentToolCallId?: string,
|
|
94
|
+
): void;
|
|
74
95
|
/** Detect assistant-text loops; returns a steer/abort decision or null. */
|
|
75
96
|
onMessageEnd(role: string, content: MessageContent): TextLoopOutcome | null;
|
|
76
97
|
/** Full reset (session start, user prompt, /loopcheck reset). */
|
|
@@ -87,14 +108,39 @@ export function createController(opts: LoopOptions = readOptions()): AntiLoopCon
|
|
|
87
108
|
let detector = new LoopDetector(opts);
|
|
88
109
|
const blockedIds = new Set<string>();
|
|
89
110
|
let steered = false;
|
|
111
|
+
/** Nested (codemode) blocks seen but not yet surfaced to the model: scripts
|
|
112
|
+
* receive the block reason as a tool error, but only the script's return
|
|
113
|
+
* value reaches the model — the next message_end steers once so the loop
|
|
114
|
+
* signal is never swallowed by a script that ignores tool errors. */
|
|
115
|
+
let steerAgain = false;
|
|
90
116
|
let resumes = 0;
|
|
91
117
|
let steers = 0;
|
|
92
118
|
let aborts = 0;
|
|
93
119
|
let suspended = false;
|
|
94
120
|
|
|
95
121
|
return {
|
|
96
|
-
onToolCall(toolName, input, toolCallId) {
|
|
122
|
+
onToolCall(toolName, input, toolCallId, parentToolCallId) {
|
|
97
123
|
if (suspended) return null;
|
|
124
|
+
// Nested codemode calls (pi ≥ 0.99) take the stricter per-script path:
|
|
125
|
+
// they never enter the top-level window, so a script-level loop cannot
|
|
126
|
+
// hide from its own 3× top-level threshold.
|
|
127
|
+
if (parentToolCallId !== undefined) {
|
|
128
|
+
const nested = detector.checkNested(parentToolCallId, toolName, input);
|
|
129
|
+
if (nested.isErr()) {
|
|
130
|
+
detector.recordNested(parentToolCallId, toolName, toolCallId, input);
|
|
131
|
+
return null;
|
|
132
|
+
}
|
|
133
|
+
blockedIds.add(toolCallId);
|
|
134
|
+
// The block reason becomes the script's tool error, not the model's:
|
|
135
|
+
// arm the one-shot steer so the model still learns about the loop.
|
|
136
|
+
steerAgain = true;
|
|
137
|
+
return {
|
|
138
|
+
block: true,
|
|
139
|
+
reason: nested.value.reason,
|
|
140
|
+
escalate: nested.value.escalate,
|
|
141
|
+
nested: true,
|
|
142
|
+
};
|
|
143
|
+
}
|
|
98
144
|
const decision = detector.check(toolName, input);
|
|
99
145
|
if (decision.isErr()) {
|
|
100
146
|
detector.record(toolName, input);
|
|
@@ -105,13 +151,17 @@ export function createController(opts: LoopOptions = readOptions()): AntiLoopCon
|
|
|
105
151
|
return { block: true, reason: block.reason, escalate: block.escalate };
|
|
106
152
|
},
|
|
107
153
|
|
|
108
|
-
onToolResult(toolName, toolCallId, isError) {
|
|
154
|
+
onToolResult(toolName, toolCallId, isError, parentToolCallId) {
|
|
109
155
|
// Blocked calls never ran, so their (error) result must not count as a
|
|
110
156
|
// consecutive failure.
|
|
111
157
|
if (blockedIds.has(toolCallId)) {
|
|
112
158
|
blockedIds.delete(toolCallId);
|
|
113
159
|
return;
|
|
114
160
|
}
|
|
161
|
+
if (parentToolCallId !== undefined) {
|
|
162
|
+
detector.recordNestedResult(parentToolCallId, toolCallId, isError);
|
|
163
|
+
return;
|
|
164
|
+
}
|
|
115
165
|
detector.recordResult(toolName, isError);
|
|
116
166
|
},
|
|
117
167
|
|
|
@@ -129,6 +179,20 @@ export function createController(opts: LoopOptions = readOptions()): AntiLoopCon
|
|
|
129
179
|
const batch = detector.checkDuplicateCalls(calls);
|
|
130
180
|
if (batch.isOk()) return { reason: batch.value.reason, action: "abort", resume: false };
|
|
131
181
|
|
|
182
|
+
// A nested codemode block may have been swallowed by the script (only
|
|
183
|
+
// its return value reaches the model): deliver the loop signal once.
|
|
184
|
+
if (steerAgain && !steered) {
|
|
185
|
+
steerAgain = false;
|
|
186
|
+
steered = true;
|
|
187
|
+
steers++;
|
|
188
|
+
return {
|
|
189
|
+
reason: "a codemode script was blocked for repeating tool calls inside itself",
|
|
190
|
+
action: "steer",
|
|
191
|
+
resume: false,
|
|
192
|
+
nested: true,
|
|
193
|
+
};
|
|
194
|
+
}
|
|
195
|
+
|
|
132
196
|
const text = extractText(content);
|
|
133
197
|
if (!text) return null;
|
|
134
198
|
const hit = detector.checkText(text);
|
|
@@ -154,6 +218,7 @@ export function createController(opts: LoopOptions = readOptions()): AntiLoopCon
|
|
|
154
218
|
blockedIds.clear();
|
|
155
219
|
steered = false;
|
|
156
220
|
suspended = false;
|
|
221
|
+
steerAgain = false;
|
|
157
222
|
// resumes/steers/aborts are intentionally NOT reset here: they are
|
|
158
223
|
// session-scoped so a stuck model cannot cycle steer→abort forever and
|
|
159
224
|
// /loopcheck can report lifetime counters.
|
package/extensions/detector.ts
CHANGED
|
@@ -163,6 +163,17 @@ export class LoopDetector {
|
|
|
163
163
|
private textStreak = 0;
|
|
164
164
|
private wastedTokens = 0;
|
|
165
165
|
|
|
166
|
+
// --- nested codemode calls (pi ≥ 0.99: `parentToolCallId` set) ---
|
|
167
|
+
/** Per-parent (per-script) call windows, keyed by the parent tool-call id. */
|
|
168
|
+
private nestedSigs = new Map<string, { sig: string; tool: string; args: string }[]>();
|
|
169
|
+
/** Nested call ids in flight, for wiring results back to their signature. */
|
|
170
|
+
private nestedIds = new Map<string, { tool: string; args: string }>();
|
|
171
|
+
/** Blocks per nested signature, prompt-scoped: a NEW script re-issuing a blocked signature escalates. */
|
|
172
|
+
private nestedBlockedBySig = new Map<string, number>();
|
|
173
|
+
/** Signatures whose identical nested call already errored inside that script. */
|
|
174
|
+
private nestedErrorSigs = new Map<string, Set<string>>();
|
|
175
|
+
private nestedCallCount = 0;
|
|
176
|
+
|
|
166
177
|
constructor(opts: LoopOptions = DEFAULT_OPTIONS) {
|
|
167
178
|
this.opts = opts;
|
|
168
179
|
}
|
|
@@ -227,10 +238,101 @@ export class LoopDetector {
|
|
|
227
238
|
this.evictSigs();
|
|
228
239
|
}
|
|
229
240
|
|
|
241
|
+
/**
|
|
242
|
+
* Canonical args of a nested call, capped: scripts can issue huge payloads
|
|
243
|
+
* (a whole file read back) hundreds of times; identity only needs the head.
|
|
244
|
+
*/
|
|
245
|
+
private canonicalNestedArgs(input: ToolInput): string {
|
|
246
|
+
const raw = canonical(input);
|
|
247
|
+
return raw.length > NESTED_ARG_STORAGE_LIMIT ? raw.slice(0, NESTED_ARG_STORAGE_LIMIT) : raw;
|
|
248
|
+
}
|
|
249
|
+
|
|
250
|
+
/**
|
|
251
|
+
* Stricter detection for calls a codemode script made (`parentToolCallId` set).
|
|
252
|
+
* Inside one script a repeated call is never progress — the script can store
|
|
253
|
+
* and reuse its first result — so identical args block at
|
|
254
|
+
* `NESTED_REPEAT_THRESHOLD` (2), and an identical retry of a call that
|
|
255
|
+
* already errored in this script blocks immediately. The block reason
|
|
256
|
+
* becomes the script's tool error; `escalate` is true when this signature
|
|
257
|
+
* was already blocked in an earlier script this prompt.
|
|
258
|
+
*/
|
|
259
|
+
checkNested(parentToolCallId: string, toolName: string, input: ToolInput): NestedDecision {
|
|
260
|
+
if (this.opts.toolExclude.has(toolName)) return Result.err(undefined);
|
|
261
|
+
const args = this.canonicalNestedArgs(input);
|
|
262
|
+
const sig = `${toolName}:${args}`;
|
|
263
|
+
const pool = this.nestedSigs.get(parentToolCallId) ?? [];
|
|
264
|
+
const repeats = pool.filter((s) => s.sig === sig).length;
|
|
265
|
+
const reasons = collectNestedBlockReasons(
|
|
266
|
+
toolName,
|
|
267
|
+
repeats,
|
|
268
|
+
this.nestedErrorSigs.get(parentToolCallId)?.has(sig) === true,
|
|
269
|
+
() => this.nearRepeats(toolName, args, pool),
|
|
270
|
+
);
|
|
271
|
+
if (reasons.length === 0) return Result.err(undefined);
|
|
272
|
+
|
|
273
|
+
const blockedCount = (this.nestedBlockedBySig.get(sig) ?? 0) + 1;
|
|
274
|
+
this.nestedBlockedBySig.set(sig, blockedCount);
|
|
275
|
+
return Result.ok({
|
|
276
|
+
reason:
|
|
277
|
+
reasons.join("; ") +
|
|
278
|
+
`. BLOCKED by anti-doom-loop — inside a script, store the first result in a variable and reuse it instead of calling again.`,
|
|
279
|
+
escalate: blockedCount > 1,
|
|
280
|
+
});
|
|
281
|
+
}
|
|
282
|
+
|
|
283
|
+
/** Record an allowed nested call into its parent script's window. */
|
|
284
|
+
recordNested(
|
|
285
|
+
parentToolCallId: string,
|
|
286
|
+
toolName: string,
|
|
287
|
+
toolCallId: string,
|
|
288
|
+
input: ToolInput,
|
|
289
|
+
): void {
|
|
290
|
+
if (this.opts.toolExclude.has(toolName)) return;
|
|
291
|
+
const args = this.canonicalNestedArgs(input);
|
|
292
|
+
let pool = this.nestedSigs.get(parentToolCallId);
|
|
293
|
+
if (!pool) {
|
|
294
|
+
pool = [];
|
|
295
|
+
this.nestedSigs.set(parentToolCallId, pool);
|
|
296
|
+
}
|
|
297
|
+
pool.push({ sig: `${toolName}:${args}`, tool: toolName, args });
|
|
298
|
+
if (pool.length > NESTED_POOL_LIMIT) pool.splice(0, pool.length - NESTED_POOL_LIMIT);
|
|
299
|
+
this.nestedIds.set(toolCallId, { tool: toolName, args });
|
|
300
|
+
this.nestedCallCount++;
|
|
301
|
+
}
|
|
302
|
+
|
|
303
|
+
/** Wire a nested result back: an errored signature blocks its identical retry. */
|
|
304
|
+
recordNestedResult(parentToolCallId: string, toolCallId: string, isError: boolean): void {
|
|
305
|
+
const entry = this.nestedIds.get(toolCallId);
|
|
306
|
+
this.nestedIds.delete(toolCallId);
|
|
307
|
+
if (!isError || !entry) return;
|
|
308
|
+
let errs = this.nestedErrorSigs.get(parentToolCallId);
|
|
309
|
+
if (!errs) {
|
|
310
|
+
errs = new Set();
|
|
311
|
+
this.nestedErrorSigs.set(parentToolCallId, errs);
|
|
312
|
+
}
|
|
313
|
+
errs.add(`${entry.tool}:${entry.args}`);
|
|
314
|
+
if (errs.size > NESTED_POOL_LIMIT) {
|
|
315
|
+
const oldest = errs.values().next();
|
|
316
|
+
if (!oldest.done) errs.delete(oldest.value);
|
|
317
|
+
}
|
|
318
|
+
}
|
|
319
|
+
|
|
320
|
+
/** Nested codemode counters for /loopcheck (allowed calls recorded, blocks issued). */
|
|
321
|
+
nestedStats(): NestedStats {
|
|
322
|
+
return {
|
|
323
|
+
calls: this.nestedCallCount,
|
|
324
|
+
blocks: [...this.nestedBlockedBySig.values()].reduce((a, b) => a + b, 0),
|
|
325
|
+
};
|
|
326
|
+
}
|
|
327
|
+
|
|
230
328
|
/** Windowed same-tool entries whose args are ≥55% shared tokens (identical counts). */
|
|
231
|
-
private nearRepeats(
|
|
329
|
+
private nearRepeats(
|
|
330
|
+
toolName: string,
|
|
331
|
+
args: string,
|
|
332
|
+
pool: readonly { tool: string; args: string }[] = this.recentSigs,
|
|
333
|
+
): number {
|
|
232
334
|
let n = 0;
|
|
233
|
-
for (const s of
|
|
335
|
+
for (const s of pool) {
|
|
234
336
|
if (s.tool !== toolName) continue;
|
|
235
337
|
const sim = argSimilarity(args, s.args);
|
|
236
338
|
if (sim !== null && sim >= NEAR_ARGS_SIMILARITY_THRESHOLD) n++;
|
|
@@ -424,10 +526,12 @@ export class LoopDetector {
|
|
|
424
526
|
.map(([s, n]) => `"${truncate(s, 24)}" x${n}`)
|
|
425
527
|
.join(", ");
|
|
426
528
|
|
|
529
|
+
const nested = this.nestedStats();
|
|
427
530
|
return (
|
|
428
531
|
`calls[${this.recentSigs.length}] ${topSigs || "none"}; ` +
|
|
429
532
|
`texts[${this.recentTexts.length}] ${topTexts || "none"}; ` +
|
|
430
|
-
`wastedTokens=${this.wastedTokens}`
|
|
533
|
+
`wastedTokens=${this.wastedTokens}; ` +
|
|
534
|
+
`nested: calls=${nested.calls} blocks=${nested.blocks}`
|
|
431
535
|
);
|
|
432
536
|
}
|
|
433
537
|
|
|
@@ -439,6 +543,11 @@ export class LoopDetector {
|
|
|
439
543
|
this.lastText = null;
|
|
440
544
|
this.textStreak = 0;
|
|
441
545
|
this.wastedTokens = 0;
|
|
546
|
+
this.nestedSigs.clear();
|
|
547
|
+
this.nestedIds.clear();
|
|
548
|
+
this.nestedBlockedBySig.clear();
|
|
549
|
+
this.nestedErrorSigs.clear();
|
|
550
|
+
this.nestedCallCount = 0;
|
|
442
551
|
}
|
|
443
552
|
|
|
444
553
|
summary(): string {
|
|
@@ -473,6 +582,63 @@ export const NEAR_ARGS_SIMILARITY_THRESHOLD = 0.55;
|
|
|
473
582
|
* exact matching only. Long inputs — shell commands — always clear this.
|
|
474
583
|
*/
|
|
475
584
|
export const NEAR_ARGS_MIN_UNION = 5;
|
|
585
|
+
|
|
586
|
+
/** Decision for a nested (codemode) tool call — same shape as BlockDecision. */
|
|
587
|
+
export type NestedDecision = Result<BlockDecision, undefined>;
|
|
588
|
+
|
|
589
|
+
/**
|
|
590
|
+
* Block reasons for one nested codemode call: identical repeats (2× rule),
|
|
591
|
+
* an identical retry of a call that already errored in this script, and —
|
|
592
|
+
* only when neither fired — near-identical args at the same 2× threshold.
|
|
593
|
+
* `nearRepeats` is lazy so random traffic never pays the tokenization cost.
|
|
594
|
+
*/
|
|
595
|
+
function collectNestedBlockReasons(
|
|
596
|
+
toolName: string,
|
|
597
|
+
repeats: number,
|
|
598
|
+
erroredBefore: boolean,
|
|
599
|
+
nearRepeats: () => number,
|
|
600
|
+
): string[] {
|
|
601
|
+
const reasons: string[] = [];
|
|
602
|
+
if (repeats + 1 >= NESTED_REPEAT_THRESHOLD) {
|
|
603
|
+
reasons.push(
|
|
604
|
+
`"${toolName}" was called with identical arguments ${repeats + 1} times inside one codemode script`,
|
|
605
|
+
);
|
|
606
|
+
}
|
|
607
|
+
if (erroredBefore) {
|
|
608
|
+
reasons.push(`the identical "${toolName}" call already failed inside this script`);
|
|
609
|
+
}
|
|
610
|
+
if (reasons.length === 0 && repeats === 0 && nearRepeats() + 1 >= NESTED_REPEAT_THRESHOLD) {
|
|
611
|
+
reasons.push(
|
|
612
|
+
`"${toolName}" was called with near-identical arguments ${nearRepeats() + 1} times inside one codemode script (only minor changes)`,
|
|
613
|
+
);
|
|
614
|
+
}
|
|
615
|
+
return reasons;
|
|
616
|
+
}
|
|
617
|
+
|
|
618
|
+
/** Counters for nested codemode traffic, reported by /loopcheck. */
|
|
619
|
+
export interface NestedStats {
|
|
620
|
+
calls: number;
|
|
621
|
+
blocks: number;
|
|
622
|
+
}
|
|
623
|
+
|
|
624
|
+
/**
|
|
625
|
+
* Nested codemode calls (pi ≥ 0.99, `parentToolCallId` set) block at a stricter
|
|
626
|
+
* threshold than top-level calls: inside one script a repeated call is never
|
|
627
|
+
* progress — the script can reuse its first result — so 2 occurrences block
|
|
628
|
+
* where top-level detection needs 3.
|
|
629
|
+
*/
|
|
630
|
+
export const NESTED_REPEAT_THRESHOLD = 2;
|
|
631
|
+
|
|
632
|
+
/** Max chars of one nested call's canonical args kept for identity/similarity. */
|
|
633
|
+
export const NESTED_ARG_STORAGE_LIMIT = 100_000;
|
|
634
|
+
|
|
635
|
+
/**
|
|
636
|
+
* Max recent calls a codemode script's window keeps. Entries hold up to
|
|
637
|
+
* `NESTED_ARG_STORAGE_LIMIT` chars each, so an unbounded per-script window
|
|
638
|
+
* would let one script pin hundreds of MB; in-script loops repeat adjacent
|
|
639
|
+
* calls, which a recent window still catches.
|
|
640
|
+
*/
|
|
641
|
+
export const NESTED_POOL_LIMIT = 100;
|
|
476
642
|
// ponytail: 0.55 misses ONE shape — short (~11-token) commands that change
|
|
477
643
|
// BOTH ends at once (added wrapper + changed tail glue → ~0.53); single-axis
|
|
478
644
|
// drift and longer commands clear. Upgrade path if that shape shows up:
|
|
@@ -776,5 +942,46 @@ if (import.meta.main) {
|
|
|
776
942
|
// genuinely different command with the same tool stays unblocked
|
|
777
943
|
assert.ok(d.check("bash", { command: "cargo test -p pools 2>&1 | tail -5" }).isErr());
|
|
778
944
|
|
|
945
|
+
// 15. nested codemode calls: stricter 2× threshold, error-retry block, no
|
|
946
|
+
// leakage into the top-level window
|
|
947
|
+
d.reset();
|
|
948
|
+
assert.ok(d.checkNested("p1", "read", { path: "a.ts" }).isErr(), "1st nested call passes");
|
|
949
|
+
d.recordNested("p1", "read", "p1/1", { path: "a.ts" });
|
|
950
|
+
const nHit = d.checkNested("p1", "read", { path: "a.ts" });
|
|
951
|
+
assert.ok(nHit.isOk(), "2nd identical nested call blocks");
|
|
952
|
+
if (nHit.isOk()) {
|
|
953
|
+
assert.equal(nHit.value.escalate, false, "first in-script block does not escalate");
|
|
954
|
+
assert.match(nHit.value.reason, /identical arguments 2 times inside one codemode script/);
|
|
955
|
+
}
|
|
956
|
+
assert.ok(d.checkNested("p2", "read", { path: "a.ts" }).isErr(), "a new script starts clean");
|
|
957
|
+
// identical-after-error blocks even at the first occurrence in a parent
|
|
958
|
+
d.recordNested("p2", "bash", "p2/1", { command: "npm test" });
|
|
959
|
+
d.recordNestedResult("p2", "p2/1", true);
|
|
960
|
+
const errHit = d.checkNested("p2", "bash", { command: "npm test" });
|
|
961
|
+
assert.ok(errHit.isOk(), "identical retry after a nested error blocks");
|
|
962
|
+
if (errHit.isOk()) assert.match(errHit.value.reason, /already failed inside this script/);
|
|
963
|
+
// a NEW script re-issuing a signature blocked before escalates
|
|
964
|
+
const crossScript = d.checkNested("p3", "bash", { command: "npm test" });
|
|
965
|
+
assert.ok(crossScript.isErr(), "fresh script with a fresh call passes");
|
|
966
|
+
d.recordNested("p3", "bash", "p3/1", { command: "npm test" });
|
|
967
|
+
const crossBlock = d.checkNested("p3", "bash", { command: "npm test" });
|
|
968
|
+
assert.ok(crossBlock.isOk() && crossBlock.value.escalate, "cross-script repeat escalates");
|
|
969
|
+
// nested traffic never enters the top-level window
|
|
970
|
+
d.reset();
|
|
971
|
+
d.recordNested("p", "bash", "p/1", { command: "poll" });
|
|
972
|
+
d.recordNested("p", "bash", "p/2", { command: "poll" });
|
|
973
|
+
assert.ok(
|
|
974
|
+
d.check("bash", { command: "poll" }).isErr(),
|
|
975
|
+
"top-level window empty of nested traffic",
|
|
976
|
+
);
|
|
977
|
+
// huge nested payloads stay capped and never throw
|
|
978
|
+
assert.doesNotThrow(() => {
|
|
979
|
+
const huge = "x".repeat(300_000);
|
|
980
|
+
for (let i = 0; i < 50; i++) {
|
|
981
|
+
d.checkNested("p", "read", { path: `f${i}.ts`, content: huge });
|
|
982
|
+
d.recordNested("p", "read", `p/h${i}`, { path: `f${i}.ts`, content: huge });
|
|
983
|
+
}
|
|
984
|
+
});
|
|
985
|
+
|
|
779
986
|
console.log("detector self-check: all assertions passed");
|
|
780
987
|
}
|
package/extensions/index.ts
CHANGED
|
@@ -76,6 +76,13 @@ const STEER_TEXT =
|
|
|
76
76
|
"Stop. Re-read the actual error output, pick ONE different action, and execute it. " +
|
|
77
77
|
"If you are stuck, ask the user instead of retrying.";
|
|
78
78
|
|
|
79
|
+
/** Steer delivered after a codemode script swallowed a nested block: the script
|
|
80
|
+
* got the block reason as a tool error, but only its return reaches the model. */
|
|
81
|
+
const NESTED_STEER_TEXT =
|
|
82
|
+
"Anti-doom-loop steering: a codemode script repeated identical tool calls inside itself. " +
|
|
83
|
+
"Inside a script, call each tool once, store the result in a variable, and reuse it — " +
|
|
84
|
+
"never poll or retry the same call in a loop. If the calls keep failing, stop and ask the user.";
|
|
85
|
+
|
|
79
86
|
/** Queued once after an abort so the work can continue with a fresh approach. */
|
|
80
87
|
const RESUME_TEXT =
|
|
81
88
|
"Anti-doom-loop: the previous run was aborted because it looped. " +
|
|
@@ -96,6 +103,7 @@ export default function (pi: PiLike): void {
|
|
|
96
103
|
event.toolName,
|
|
97
104
|
event.input as ToolInput,
|
|
98
105
|
event.toolCallId,
|
|
106
|
+
event.parentToolCallId,
|
|
99
107
|
);
|
|
100
108
|
if (outcome === null) return;
|
|
101
109
|
if (outcome.escalate) {
|
|
@@ -106,7 +114,12 @@ export default function (pi: PiLike): void {
|
|
|
106
114
|
});
|
|
107
115
|
|
|
108
116
|
pi.on("tool_result", (event: ToolResultEventLite) => {
|
|
109
|
-
controller.onToolResult(
|
|
117
|
+
controller.onToolResult(
|
|
118
|
+
event.toolName,
|
|
119
|
+
event.toolCallId,
|
|
120
|
+
event.isError === true,
|
|
121
|
+
event.parentToolCallId,
|
|
122
|
+
);
|
|
110
123
|
});
|
|
111
124
|
|
|
112
125
|
// Text-only doom loops (model re-emits/rephrases the same thing with no
|
|
@@ -134,9 +147,10 @@ export default function (pi: PiLike): void {
|
|
|
134
147
|
if (outcome === null) return;
|
|
135
148
|
|
|
136
149
|
if (outcome.action === "steer") {
|
|
150
|
+
const text = outcome.nested ? NESTED_STEER_TEXT : STEER_TEXT;
|
|
137
151
|
ctx.ui.notify(`Anti-doom-loop: ${outcome.reason}`, "warning");
|
|
138
152
|
pi.sendMessage?.(
|
|
139
|
-
{ customType: "anti-doom-loop", content:
|
|
153
|
+
{ customType: "anti-doom-loop", content: text, display: true },
|
|
140
154
|
{ deliverAs: "steer", triggerTurn: true },
|
|
141
155
|
);
|
|
142
156
|
return;
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "pi-anti-doom-loop",
|
|
3
|
-
"version": "0.0.
|
|
3
|
+
"version": "0.0.12",
|
|
4
4
|
"description": "Detect and break agent doom loops in pi: blocks identical repeated tool calls and blind retries before they burn tokens.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"anti-doom-loop",
|
|
@@ -29,28 +29,18 @@
|
|
|
29
29
|
"publishConfig": {
|
|
30
30
|
"access": "public"
|
|
31
31
|
},
|
|
32
|
-
"scripts": {
|
|
33
|
-
"check": "npm run test && tsc --noEmit && oxlint --deny-warnings && oxfmt --check .",
|
|
34
|
-
"prepublishOnly": "npm run check",
|
|
35
|
-
"test": "node --test tests/*.test.ts",
|
|
36
|
-
"guard": "node scripts/guard-publish.ts",
|
|
37
|
-
"lint": "oxlint --deny-warnings",
|
|
38
|
-
"format": "oxfmt .",
|
|
39
|
-
"format:check": "oxfmt --check .",
|
|
40
|
-
"prepare": "effect-tsgo patch --no-typescript --oxlint --skip-missing"
|
|
41
|
-
},
|
|
42
32
|
"dependencies": {
|
|
43
|
-
"@effect/tsgo": "
|
|
33
|
+
"@effect/tsgo": "0.36.4",
|
|
44
34
|
"better-result": "^3.0.0",
|
|
45
|
-
"effect": "^4.0.0-rc.108"
|
|
46
|
-
"oxlint": "1.77.0"
|
|
35
|
+
"effect": "^4.0.0-rc.108"
|
|
47
36
|
},
|
|
48
37
|
"devDependencies": {
|
|
49
38
|
"@earendil-works/pi-coding-agent": "*",
|
|
50
39
|
"@oxlint/plugins": "1.77.0",
|
|
51
40
|
"@types/node": "^22.0.0",
|
|
52
|
-
"oxfmt": "
|
|
53
|
-
"oxlint
|
|
41
|
+
"oxfmt": "0.62.0",
|
|
42
|
+
"oxlint": "1.77.0",
|
|
43
|
+
"oxlint-tsgolint": "7.0.2001",
|
|
54
44
|
"typescript": "^5.6.0"
|
|
55
45
|
},
|
|
56
46
|
"peerDependencies": {
|
|
@@ -63,5 +53,14 @@
|
|
|
63
53
|
"extensions": [
|
|
64
54
|
"./extensions"
|
|
65
55
|
]
|
|
56
|
+
},
|
|
57
|
+
"scripts": {
|
|
58
|
+
"check": "pnpm run test && tsc --noEmit && oxlint --deny-warnings && oxfmt --check .",
|
|
59
|
+
"test": "node --test tests/*.test.ts tests/e2e/*.test.ts",
|
|
60
|
+
"guard": "node scripts/guard-publish.ts",
|
|
61
|
+
"e2e:pi": "node --test tests/e2e/pi.test.ts",
|
|
62
|
+
"lint": "oxlint --deny-warnings",
|
|
63
|
+
"format": "oxfmt .",
|
|
64
|
+
"format:check": "oxfmt --check ."
|
|
66
65
|
}
|
|
67
|
-
}
|
|
66
|
+
}
|