pi-anti-doom-loop 0.0.9 → 0.0.12
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +47 -0
- package/README.md +67 -14
- package/extensions/controller.ts +70 -5
- package/extensions/detector.ts +301 -4
- package/extensions/index.ts +38 -9
- package/package.json +14 -15
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,53 @@
|
|
|
2
2
|
|
|
3
3
|
All notable changes to **pi-anti-doom-loop**.
|
|
4
4
|
|
|
5
|
+
## [0.0.12] — 2026-10-01
|
|
6
|
+
|
|
7
|
+
### Added
|
|
8
|
+
|
|
9
|
+
- **E2E tests against a real pi ≥ 0.99** (`tests/e2e/pi.test.ts` + `tests/e2e/pi-rig.ts`) — the rig downloads the real `@earendil-works/pi-coding-agent@0.99.2` (cached), runs pi's RPC entry headlessly with the repo extension loaded, and drives sessions with an in-process OpenAI-compatible fake LLM, fully offline. Verified: the extension blocks a scripted identical-`bash` doom loop in a real session; a **real codemode worker** executing a script that polls a file 5× inside ONE `codemode` call is blocked by the nested 2× rule and the script fails with the `BLOCKED by anti-doom-loop` reason; the recommended distinct `Promise.allSettled` batch completes untouched. Skips cleanly when the npm registry is unreachable. Run with `pnpm run e2e:pi`.
|
|
10
|
+
- **Codemode nested-loop fixtures** — `codemodeNestedLoop` (in-script poll loop), `codemodeErrorRetryLoop` (identical retry after an in-script error), `codemodeCrossScriptRepeat` (second script re-issuing a blocked signature → escalation), and `codemodeHealthyBatch` (the documented `Promise.allSettled` pattern stays unblocked), each driven through the controller exactly as pi delivers nested events, plus a contrast test proving a 2× loop blocks nested while staying under the top-level 3× threshold.
|
|
11
|
+
|
|
12
|
+
### Changed
|
|
13
|
+
|
|
14
|
+
- **Tooling migrated to pnpm (latest, 12.8.1)** — `packageManager` pin, `pnpm-lock.yaml` with the default virtual store (`node_modules/.pnpm`), all scripts and docs use `pnpm`, the release workflow installs via `pnpm/action-setup` + `--frozen-lockfile` and publishes with `pnpm publish --no-git-checks`; the tarball e2e asserts via `pnpm pack` and pins the tooling.
|
|
15
|
+
|
|
16
|
+
### Fixed
|
|
17
|
+
|
|
18
|
+
- **Fresh installs failed under pnpm 12** — unapproved dependency build scripts made `pnpm install --frozen-lockfile` exit `ERR_PNPM_IGNORED_BUILDS`, which would have aborted the release workflow before the quality gate. The dev-only transitives pi pulls (`esbuild`, `protobufjs`, `@google/genai`) are now explicitly denied in `pnpm-workspace.yaml`; their binaries come from optional platform packages.
|
|
19
|
+
- **`oxlint-tsgolint` pin had no matching artifact** — `package.json` asked for `7.0.2003` while `@effect/tsgo` ships its patched tsgolint for `7.0.2001` only. The devDependency now matches the artifact (the workspace override already pinned it), so `prepare` no longer looks for a missing binary.
|
|
20
|
+
- **Bounded codemode memory** — a script's window now keeps only its most recent 100 calls (args are capped at 100 KB each), so a long-running script cannot pin unbounded memory; in-script loops repeat adjacent calls and still block at 2×.
|
|
21
|
+
- **Real-pi e2e reported green when it never ran** — offline runs returned early from each test, which `node:test` records as passing. They now report `skipped`, so `pnpm test` can no longer claim coverage it did not execute.
|
|
22
|
+
|
|
23
|
+
### Tests
|
|
24
|
+
|
|
25
|
+
- 122 → 133: 4 real-pi e2e sessions, 6 codemode fixture tests, pnpm-pack artifact assertions, and the bounded nested-window case.
|
|
26
|
+
|
|
27
|
+
## [0.0.11] — 2026-10-01
|
|
28
|
+
|
|
29
|
+
### Added
|
|
30
|
+
|
|
31
|
+
- **Codemode awareness (pi ≥ 0.99)** — pi's `codemode` tool runs model-written scripts that call other tools via `tools.<name>(args)`; those nested calls carry `parentToolCallId` in the `tool_call`/`tool_result` events and previously bypassed detection entirely (a script can issue hundreds of calls inside ONE top-level tool call). The extension now tracks them: identical calls inside one script block at **2×** (stricter than the top-level 3× — a repeated in-script call is never progress; store the result and reuse it), near-identical args count toward the same rule, and an identical retry of a call that already **errored** in the same script blocks immediately. Nested traffic is windowed **per script** and fully isolated from the top-level window in both directions. A NEW script re-issuing a signature blocked in an earlier script this prompt **escalates** (aborts the turn), mirroring top-level block escalation. `PI_ANTI_LOOP_TOOLS_EXCLUDE` applies to nested calls; nested identity is storage-capped (100 KB per call, most recent 100 calls per script) so huge payloads (whole-file reads) stay cheap; `/loopcheck` reports `nested: calls=N blocks=M`.
|
|
32
|
+
- **The information wall** — a nested block reason reaches the _script_ as a tool error, not the model (pi delivers only the script's return value). If a script swallows block errors, the loop signal was lost. The extension now arms a one-shot **steer** on the next assistant `message_end` after any nested block, telling the model to call each tool once per script and reuse results — so a script that ignores tool errors can no longer silently eat the loop signal.
|
|
33
|
+
|
|
34
|
+
### Tests
|
|
35
|
+
|
|
36
|
+
- 99 → 122: nested unit semantics (2× rule, per-script isolation, error-retry, cross-script escalation, payload cap, exclude-list), adapter wiring (`parentToolCallId` forwarding, `NESTED_STEER_TEXT` via `sendMessage`, escalation aborts), and nested fuzz streams (never throw, injected in-script loops always block, random unique-args streams never false-positive).
|
|
37
|
+
|
|
38
|
+
## [0.0.10] — 2026-09-23
|
|
39
|
+
|
|
40
|
+
### Fixed
|
|
41
|
+
|
|
42
|
+
- **Counters reset per user prompt, not per run** — the extension reset on `before_agent_start`, which omp emits on _every_ run start: auto-continue turns (`role:"developer"`), steers/resumes, advisor cards, and queued async-result drains all restart the run, wiping counters (including block-escalation state) between the iterations of exactly the cross-run loops this extension exists to catch. Reported identical `ssh`-polling loops ran 7+ times unblocked while intra-run repeats fired correctly — replaying the incident session on detector-visible input reproduces its 9 exact blocks, with the only discrepancies landing right after post-abort run restarts, where the per-run reset had cleared the window. Reset now keys off `message_end` with `role:"user"` (emitted with the run's input messages on omp and pi alike), matching the documented per-user-prompt contract; steers and auto-continues keep counting, so the steer→abort escalation ladder survives run restarts too.
|
|
43
|
+
|
|
44
|
+
### Added
|
|
45
|
+
|
|
46
|
+
- **Near-identical tool-call detection** — when no exact signature repeats, `check()` counts same-tool window entries whose canonical args share ≥ 55% whitespace tokens (token union ≥ 5): sleep-wrapped polls (`gh run list…` → `sleep 150; gh run list…`), flag/grep-pattern/tail drift. Block reason prefix: _"…" was called with near-identical arguments N times…_. Short inputs stay exact-only via the union guard; exact repeats still block first with the exact reason.
|
|
47
|
+
|
|
48
|
+
### Notes
|
|
49
|
+
|
|
50
|
+
- omp strips the harness intent label (`i`) at parse (`extractIntent`), before extension events — this detector never sees it; no handling was added. pi has no intent field at all.
|
|
51
|
+
|
|
5
52
|
## [0.0.9] — 2026-09-07
|
|
6
53
|
|
|
7
54
|
### Added
|
package/README.md
CHANGED
|
@@ -51,7 +51,7 @@ Environment variables, read at session/prompt start:
|
|
|
51
51
|
| `PI_ANTI_LOOP_TIME_WINDOW` | `0` | Elapsed-time window in ms (`0` = disabled, count-only): evicts window entries older than this so slow chronic loops over a long session are caught |
|
|
52
52
|
| `PI_ANTI_LOOP_FAIL_RATE` | `0` | Fail-rate block threshold `0..1` (`0` = disabled): block when a tool's error share of its in-window calls reaches this |
|
|
53
53
|
| `PI_ANTI_LOOP_FAIL_RATE_MIN` | `3` | Minimum calls before the fail-rate window can block |
|
|
54
|
-
| `PI_ANTI_LOOP_TOOLS_EXCLUDE` | — | Comma-separated tool names to disable detection for entirely (never block, never enter the window)
|
|
54
|
+
| `PI_ANTI_LOOP_TOOLS_EXCLUDE` | — | Comma-separated tool names to disable detection for entirely (never block, never enter the window — applies to nested codemode calls too) |
|
|
55
55
|
| `PI_ANTI_LOOP_DISABLE` | — | Set to `1` to disable the extension entirely |
|
|
56
56
|
|
|
57
57
|
## Command
|
|
@@ -61,6 +61,38 @@ Environment variables, read at session/prompt start:
|
|
|
61
61
|
- `/loopcheck suspend` — pause detection until the next prompt (escape hatch for intentional repetition)
|
|
62
62
|
- `/loopcheck resume` — re-enable detection early
|
|
63
63
|
|
|
64
|
+
### Codemode (pi ≥ 0.99): nested script calls
|
|
65
|
+
|
|
66
|
+
pi's `codemode` tool lets the model run a JavaScript script that calls other
|
|
67
|
+
tools via `tools.<name>(args)` — a script can loop hundreds of calls inside
|
|
68
|
+
ONE top-level tool call, which per-call detection never sees as a streak.
|
|
69
|
+
Since pi 0.99, those nested calls still flow through the `tool_call`/
|
|
70
|
+
`tool_result` events tagged with `parentToolCallId` — and this extension
|
|
71
|
+
tracks them:
|
|
72
|
+
|
|
73
|
+
- **Stricter in-script threshold** — inside one script, the 2nd identical
|
|
74
|
+
call blocks (top-level detection needs 3). A repeated call inside a script
|
|
75
|
+
is never progress: the script can store its first result in a variable and
|
|
76
|
+
reuse it. Near-identical args (sleep-wrapped polls, flag drift) count too.
|
|
77
|
+
- **Error-retry rule** — an identical retry of a call that already errored
|
|
78
|
+
inside the same script blocks immediately with a "the identical call
|
|
79
|
+
already failed inside this script" reason.
|
|
80
|
+
- **Script-level isolation** — nested traffic never enters the top-level
|
|
81
|
+
window and vice versa, so script loops cannot hide from their own 2× rule
|
|
82
|
+
and legitimate top-level repetition is never inflated by scripts.
|
|
83
|
+
- **The information wall** — a nested block reason reaches the _script_ as a
|
|
84
|
+
tool error, not the model (only the script's return value does). When a
|
|
85
|
+
script swallows it anyway, the next assistant message gets a one-shot steer
|
|
86
|
+
telling the model to call each tool once and reuse results — so the loop
|
|
87
|
+
signal can never be silently eaten by a script that ignores tool errors.
|
|
88
|
+
- **Cross-script escalation** — counters are per-script (a new script starts
|
|
89
|
+
clean), but re-issuing a signature blocked in an earlier script this prompt
|
|
90
|
+
escalates and aborts the turn, mirroring top-level block-escalation.
|
|
91
|
+
|
|
92
|
+
Huge nested payloads (whole-file reads) are identity-capped (100 KB per call)
|
|
93
|
+
and each script keeps only its most recent 100 calls, so memory stays bounded. `/loopcheck` reports `nested: calls=N blocks=M`. Disable with
|
|
94
|
+
`PI_ANTI_LOOP_DISABLE=1` as usual.
|
|
95
|
+
|
|
64
96
|
## Token-cost awareness
|
|
65
97
|
|
|
66
98
|
The detector estimates tokens burned on redundant repeats (~4 chars/token) and
|
|
@@ -77,29 +109,50 @@ Works with any model — cheap models just trigger it more often.
|
|
|
77
109
|
|
|
78
110
|
## Development
|
|
79
111
|
|
|
80
|
-
Requires Node 22.18+ (
|
|
112
|
+
Requires Node 22.18+ and [pnpm](https://pnpm.io) 12 (pinned via `packageManager`; the install uses pnpm's virtual store at `node_modules/.pnpm`).
|
|
81
113
|
|
|
82
114
|
```bash
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
115
|
+
pnpm install
|
|
116
|
+
pnpm test # node --test: unit + fixture + fuzz + integration + e2e (incl. real-pi e2e)
|
|
117
|
+
pnpm run check # pnpm test + tsc + oxlint --deny-warnings + oxfmt
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
### E2E against real pi ≥ 0.99
|
|
121
|
+
|
|
122
|
+
`tests/e2e/pi.test.ts` downloads the real `@earendil-works/pi-coding-agent@0.99.2`
|
|
123
|
+
once (cached in the OS temp dir), then drives it headlessly: an in-process
|
|
124
|
+
OpenAI-compatible fake LLM scripts the assistant turns, pi's RPC entry runs
|
|
125
|
+
with THIS repo's extension loaded, and the assertions read the real RPC event
|
|
126
|
+
stream. Verified scenarios:
|
|
127
|
+
|
|
128
|
+
- the extension loads into pi 0.99 and blocks a scripted identical-`bash` doom
|
|
129
|
+
loop with the instructive block reason
|
|
130
|
+
- a **real codemode worker** executing a script that polls a file 5× inside
|
|
131
|
+
ONE `codemode` call gets blocked by the nested 2× rule — the script fails
|
|
132
|
+
with `BLOCKED by anti-doom-loop — inside one codemode script…`
|
|
133
|
+
- the recommended distinct `Promise.allSettled` batch completes untouched
|
|
134
|
+
|
|
135
|
+
Everything is offline (no API keys); if the npm registry is unreachable the
|
|
136
|
+
real-pi tests skip cleanly instead of failing. Run just them with:
|
|
137
|
+
|
|
138
|
+
```bash
|
|
139
|
+
pnpm run e2e:pi
|
|
86
140
|
```
|
|
87
141
|
|
|
88
142
|
### Test suite (Node built-in runner, no framework)
|
|
89
143
|
|
|
90
|
-
| Suite | File | What it proves
|
|
91
|
-
| ----------- | --------------------------------------------- |
|
|
92
|
-
| unit | `tests/unit.test.ts` | detector semantics: repeat/failure/text signals, window eviction, options clamping, helpers
|
|
93
|
-
| fixture | `tests/fixtures.ts` + `tests/fixture.test.ts` | real doom-loop transcripts (CI-log loops, verbatim repeats) are caught; healthy sessions are not
|
|
94
|
-
| fuzz | `tests/fuzz.test.ts` | seeded random streams: never throws, no false positives, injected loops always block, canonical stability
|
|
95
|
-
| integration | `tests/integration.test.ts` | controller + `index.ts` adapter driven through a fake `PiLike`: blocks, escalations, aborts, resets, `/loopcheck`
|
|
96
|
-
| e2e | `tests/e2e.test.ts`
|
|
144
|
+
| Suite | File | What it proves |
|
|
145
|
+
| ----------- | --------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
146
|
+
| unit | `tests/unit.test.ts` | detector semantics: repeat/failure/text/nested signals, window eviction, options clamping, helpers |
|
|
147
|
+
| fixture | `tests/fixtures.ts` + `tests/fixture.test.ts` | real doom-loop transcripts (CI-log loops, verbatim repeats, codemode script loops) are caught; healthy sessions are not |
|
|
148
|
+
| fuzz | `tests/fuzz.test.ts` | seeded random streams: never throws, no false positives, injected loops always block, canonical stability |
|
|
149
|
+
| integration | `tests/integration.test.ts` | controller + `index.ts` adapter driven through a fake `PiLike`: blocks, escalations, aborts, resets, `/loopcheck` |
|
|
150
|
+
| e2e | `tests/e2e.test.ts` + `tests/e2e/pi.test.ts` | real subprocesses: detector self-check, version guard, tarball contents (extensions/scripts ship, tests don't), and full sessions against a real pi ≥ 0.99 with a scripted fake LLM |
|
|
97
151
|
|
|
98
152
|
> `peerDependencies` pins `@earendil-works/pi-coding-agent` at `"*"` on purpose — the
|
|
99
153
|
> [pi packages docs](https://pi.dev/docs/latest/packages) require an unbounded range for
|
|
100
154
|
> pi-core packages (pi provides them at runtime). The extension loads `.ts` directly via
|
|
101
155
|
> jiti, so no build step ships; `prepublishOnly` runs the full quality gate before publish.
|
|
102
|
-
|
|
103
156
|
> When a call is blocked, escalation still works without recording it in the window:
|
|
104
157
|
> re-issuing the identical call increments a per-signature block counter and aborts the
|
|
105
158
|
> turn on the second block. Thresholds are clamped to a minimum of 2 so a bad config
|
|
@@ -117,7 +170,7 @@ git tag v0.0.1
|
|
|
117
170
|
git push origin v0.0.1
|
|
118
171
|
```
|
|
119
172
|
|
|
120
|
-
CI runs the quality gate, then the **version bump guard** (`
|
|
173
|
+
CI runs the quality gate with pnpm, then the **version bump guard** (`pnpm run guard`): it blocks publishing if the version is already on npm or the tag doesn't match `package.json`. After the first publish, the [pi.dev gallery](https://pi.dev/packages) picks the package up automatically via the `pi-package` keyword.
|
|
121
174
|
|
|
122
175
|
## License
|
|
123
176
|
|
package/extensions/controller.ts
CHANGED
|
@@ -20,11 +20,15 @@ export interface ToolCallEventLite {
|
|
|
20
20
|
toolName: string;
|
|
21
21
|
toolCallId: string;
|
|
22
22
|
input: unknown;
|
|
23
|
+
/** Set when another tool (for example a codemode script) issued this call (pi ≥ 0.99). */
|
|
24
|
+
parentToolCallId?: string;
|
|
23
25
|
}
|
|
24
26
|
export interface ToolResultEventLite {
|
|
25
27
|
toolName: string;
|
|
26
28
|
toolCallId: string;
|
|
27
29
|
isError: boolean;
|
|
30
|
+
/** Set when another tool (for example a codemode script) issued this call (pi ≥ 0.99). */
|
|
31
|
+
parentToolCallId?: string;
|
|
28
32
|
}
|
|
29
33
|
export interface MessageEndEventLite {
|
|
30
34
|
message: { role: string; content?: unknown };
|
|
@@ -54,6 +58,10 @@ export interface ToolCallOutcome {
|
|
|
54
58
|
reason: string;
|
|
55
59
|
/** True when this exact call was blocked before — caller should abort the turn. */
|
|
56
60
|
escalate: boolean;
|
|
61
|
+
/** True for calls a codemode script made (`parentToolCallId` set, pi ≥ 0.99).
|
|
62
|
+
* Their block reason reaches the script as a tool error, not the model —
|
|
63
|
+
* the adapter must also steer so the model learns about the loop. */
|
|
64
|
+
nested?: boolean;
|
|
57
65
|
}
|
|
58
66
|
|
|
59
67
|
export interface TextLoopOutcome {
|
|
@@ -61,16 +69,29 @@ export interface TextLoopOutcome {
|
|
|
61
69
|
action: "steer" | "abort";
|
|
62
70
|
/** When aborting: also queue a single fresh-resume directive (bounded). */
|
|
63
71
|
resume: boolean;
|
|
72
|
+
/** True when this steer surfaces a codemode script's swallowed nested block. */
|
|
73
|
+
nested?: boolean;
|
|
64
74
|
}
|
|
65
75
|
|
|
66
76
|
/** How many auto-resumes per session before we hand control back for real. */
|
|
67
77
|
export const RESUME_BUDGET = 1;
|
|
68
78
|
|
|
69
79
|
export interface AntiLoopController {
|
|
70
|
-
/** Returns a block decision for a tool call, or null to let it run.
|
|
71
|
-
|
|
80
|
+
/** Returns a block decision for a tool call, or null to let it run.
|
|
81
|
+
* `parentToolCallId` is set for calls a codemode script made (pi ≥ 0.99). */
|
|
82
|
+
onToolCall(
|
|
83
|
+
toolName: string,
|
|
84
|
+
input: ToolInput,
|
|
85
|
+
toolCallId: string,
|
|
86
|
+
parentToolCallId?: string,
|
|
87
|
+
): ToolCallOutcome | null;
|
|
72
88
|
/** Record a finished tool result (blocked calls' results are ignored). */
|
|
73
|
-
onToolResult(
|
|
89
|
+
onToolResult(
|
|
90
|
+
toolName: string,
|
|
91
|
+
toolCallId: string,
|
|
92
|
+
isError: boolean,
|
|
93
|
+
parentToolCallId?: string,
|
|
94
|
+
): void;
|
|
74
95
|
/** Detect assistant-text loops; returns a steer/abort decision or null. */
|
|
75
96
|
onMessageEnd(role: string, content: MessageContent): TextLoopOutcome | null;
|
|
76
97
|
/** Full reset (session start, user prompt, /loopcheck reset). */
|
|
@@ -87,14 +108,39 @@ export function createController(opts: LoopOptions = readOptions()): AntiLoopCon
|
|
|
87
108
|
let detector = new LoopDetector(opts);
|
|
88
109
|
const blockedIds = new Set<string>();
|
|
89
110
|
let steered = false;
|
|
111
|
+
/** Nested (codemode) blocks seen but not yet surfaced to the model: scripts
|
|
112
|
+
* receive the block reason as a tool error, but only the script's return
|
|
113
|
+
* value reaches the model — the next message_end steers once so the loop
|
|
114
|
+
* signal is never swallowed by a script that ignores tool errors. */
|
|
115
|
+
let steerAgain = false;
|
|
90
116
|
let resumes = 0;
|
|
91
117
|
let steers = 0;
|
|
92
118
|
let aborts = 0;
|
|
93
119
|
let suspended = false;
|
|
94
120
|
|
|
95
121
|
return {
|
|
96
|
-
onToolCall(toolName, input, toolCallId) {
|
|
122
|
+
onToolCall(toolName, input, toolCallId, parentToolCallId) {
|
|
97
123
|
if (suspended) return null;
|
|
124
|
+
// Nested codemode calls (pi ≥ 0.99) take the stricter per-script path:
|
|
125
|
+
// they never enter the top-level window, so a script-level loop cannot
|
|
126
|
+
// hide from its own 3× top-level threshold.
|
|
127
|
+
if (parentToolCallId !== undefined) {
|
|
128
|
+
const nested = detector.checkNested(parentToolCallId, toolName, input);
|
|
129
|
+
if (nested.isErr()) {
|
|
130
|
+
detector.recordNested(parentToolCallId, toolName, toolCallId, input);
|
|
131
|
+
return null;
|
|
132
|
+
}
|
|
133
|
+
blockedIds.add(toolCallId);
|
|
134
|
+
// The block reason becomes the script's tool error, not the model's:
|
|
135
|
+
// arm the one-shot steer so the model still learns about the loop.
|
|
136
|
+
steerAgain = true;
|
|
137
|
+
return {
|
|
138
|
+
block: true,
|
|
139
|
+
reason: nested.value.reason,
|
|
140
|
+
escalate: nested.value.escalate,
|
|
141
|
+
nested: true,
|
|
142
|
+
};
|
|
143
|
+
}
|
|
98
144
|
const decision = detector.check(toolName, input);
|
|
99
145
|
if (decision.isErr()) {
|
|
100
146
|
detector.record(toolName, input);
|
|
@@ -105,13 +151,17 @@ export function createController(opts: LoopOptions = readOptions()): AntiLoopCon
|
|
|
105
151
|
return { block: true, reason: block.reason, escalate: block.escalate };
|
|
106
152
|
},
|
|
107
153
|
|
|
108
|
-
onToolResult(toolName, toolCallId, isError) {
|
|
154
|
+
onToolResult(toolName, toolCallId, isError, parentToolCallId) {
|
|
109
155
|
// Blocked calls never ran, so their (error) result must not count as a
|
|
110
156
|
// consecutive failure.
|
|
111
157
|
if (blockedIds.has(toolCallId)) {
|
|
112
158
|
blockedIds.delete(toolCallId);
|
|
113
159
|
return;
|
|
114
160
|
}
|
|
161
|
+
if (parentToolCallId !== undefined) {
|
|
162
|
+
detector.recordNestedResult(parentToolCallId, toolCallId, isError);
|
|
163
|
+
return;
|
|
164
|
+
}
|
|
115
165
|
detector.recordResult(toolName, isError);
|
|
116
166
|
},
|
|
117
167
|
|
|
@@ -129,6 +179,20 @@ export function createController(opts: LoopOptions = readOptions()): AntiLoopCon
|
|
|
129
179
|
const batch = detector.checkDuplicateCalls(calls);
|
|
130
180
|
if (batch.isOk()) return { reason: batch.value.reason, action: "abort", resume: false };
|
|
131
181
|
|
|
182
|
+
// A nested codemode block may have been swallowed by the script (only
|
|
183
|
+
// its return value reaches the model): deliver the loop signal once.
|
|
184
|
+
if (steerAgain && !steered) {
|
|
185
|
+
steerAgain = false;
|
|
186
|
+
steered = true;
|
|
187
|
+
steers++;
|
|
188
|
+
return {
|
|
189
|
+
reason: "a codemode script was blocked for repeating tool calls inside itself",
|
|
190
|
+
action: "steer",
|
|
191
|
+
resume: false,
|
|
192
|
+
nested: true,
|
|
193
|
+
};
|
|
194
|
+
}
|
|
195
|
+
|
|
132
196
|
const text = extractText(content);
|
|
133
197
|
if (!text) return null;
|
|
134
198
|
const hit = detector.checkText(text);
|
|
@@ -154,6 +218,7 @@ export function createController(opts: LoopOptions = readOptions()): AntiLoopCon
|
|
|
154
218
|
blockedIds.clear();
|
|
155
219
|
steered = false;
|
|
156
220
|
suspended = false;
|
|
221
|
+
steerAgain = false;
|
|
157
222
|
// resumes/steers/aborts are intentionally NOT reset here: they are
|
|
158
223
|
// session-scoped so a stuck model cannot cycle steer→abort forever and
|
|
159
224
|
// /loopcheck can report lifetime counters.
|
package/extensions/detector.ts
CHANGED
|
@@ -102,6 +102,12 @@ export function canonical(input: ToolInput): string {
|
|
|
102
102
|
return JSON.stringify(input) ?? "";
|
|
103
103
|
}
|
|
104
104
|
|
|
105
|
+
/**
|
|
106
|
+
* Callers pass detector-visible input: hosts strip harness-internal fields
|
|
107
|
+
* (omp removes the intent label `i` at parse via `extractIntent`) before the
|
|
108
|
+
* `tool_call` event, so identity is the executed args exactly as the detector
|
|
109
|
+
* receives them.
|
|
110
|
+
*/
|
|
105
111
|
export function signature(toolName: string, input: ToolInput): string {
|
|
106
112
|
return `${toolName}:${canonical(input)}`;
|
|
107
113
|
}
|
|
@@ -149,7 +155,7 @@ function collectBlockReasons(
|
|
|
149
155
|
*/
|
|
150
156
|
export class LoopDetector {
|
|
151
157
|
readonly opts: LoopOptions;
|
|
152
|
-
private recentSigs: { sig: string; ts: number }[] = [];
|
|
158
|
+
private recentSigs: { sig: string; tool: string; args: string; ts: number }[] = [];
|
|
153
159
|
private recentResults: { tool: string; error: boolean; ts: number }[] = [];
|
|
154
160
|
private blockedBySig = new Map<string, number>();
|
|
155
161
|
private recentTexts: { text: string; ts: number }[] = [];
|
|
@@ -157,6 +163,17 @@ export class LoopDetector {
|
|
|
157
163
|
private textStreak = 0;
|
|
158
164
|
private wastedTokens = 0;
|
|
159
165
|
|
|
166
|
+
// --- nested codemode calls (pi ≥ 0.99: `parentToolCallId` set) ---
|
|
167
|
+
/** Per-parent (per-script) call windows, keyed by the parent tool-call id. */
|
|
168
|
+
private nestedSigs = new Map<string, { sig: string; tool: string; args: string }[]>();
|
|
169
|
+
/** Nested call ids in flight, for wiring results back to their signature. */
|
|
170
|
+
private nestedIds = new Map<string, { tool: string; args: string }>();
|
|
171
|
+
/** Blocks per nested signature, prompt-scoped: a NEW script re-issuing a blocked signature escalates. */
|
|
172
|
+
private nestedBlockedBySig = new Map<string, number>();
|
|
173
|
+
/** Signatures whose identical nested call already errored inside that script. */
|
|
174
|
+
private nestedErrorSigs = new Map<string, Set<string>>();
|
|
175
|
+
private nestedCallCount = 0;
|
|
176
|
+
|
|
160
177
|
constructor(opts: LoopOptions = DEFAULT_OPTIONS) {
|
|
161
178
|
this.opts = opts;
|
|
162
179
|
}
|
|
@@ -167,7 +184,8 @@ export class LoopDetector {
|
|
|
167
184
|
return Result.err(undefined);
|
|
168
185
|
}
|
|
169
186
|
this.evictSigs();
|
|
170
|
-
const
|
|
187
|
+
const args = canonical(input);
|
|
188
|
+
const sig = `${toolName}:${args}`;
|
|
171
189
|
const repeats = this.recentSigs.filter((s) => s.sig === sig).length;
|
|
172
190
|
const total = repeats + 1; // including this call
|
|
173
191
|
|
|
@@ -183,6 +201,21 @@ export class LoopDetector {
|
|
|
183
201
|
this.opts,
|
|
184
202
|
);
|
|
185
203
|
|
|
204
|
+
// Exact-signature miss: a loop that varies cosmetic fields (the `i` intent
|
|
205
|
+
// label, tail flags, timeouts) never repeats a signature. Production data:
|
|
206
|
+
// 7 of 9 commands repeated ≥3× in a live session had fully distinct args
|
|
207
|
+
// with an identical `command`. Same tool + ≥55% shared arg tokens still
|
|
208
|
+
// counts as a repeat.
|
|
209
|
+
if (reasons.length === 0) {
|
|
210
|
+
const near = this.nearRepeats(toolName, args);
|
|
211
|
+
if (near >= 1) this.wastedTokens += estimateTokens(stringify(input));
|
|
212
|
+
if (near + 1 >= this.opts.repeatThreshold) {
|
|
213
|
+
reasons.push(
|
|
214
|
+
`"${toolName}" was called with near-identical arguments ${near + 1} times in the last ${this.opts.windowSize} tool calls (only minor changes)`,
|
|
215
|
+
);
|
|
216
|
+
}
|
|
217
|
+
}
|
|
218
|
+
|
|
186
219
|
if (reasons.length === 0) return Result.err(undefined);
|
|
187
220
|
|
|
188
221
|
const blockedCount = (this.blockedBySig.get(sig) ?? 0) + 1;
|
|
@@ -200,10 +233,113 @@ export class LoopDetector {
|
|
|
200
233
|
|
|
201
234
|
record(toolName: string, input: ToolInput): void {
|
|
202
235
|
if (this.opts.toolExclude.has(toolName)) return;
|
|
203
|
-
|
|
236
|
+
const args = canonical(input);
|
|
237
|
+
this.recentSigs.push({ sig: `${toolName}:${args}`, tool: toolName, args, ts: Date.now() });
|
|
204
238
|
this.evictSigs();
|
|
205
239
|
}
|
|
206
240
|
|
|
241
|
+
/**
|
|
242
|
+
* Canonical args of a nested call, capped: scripts can issue huge payloads
|
|
243
|
+
* (a whole file read back) hundreds of times; identity only needs the head.
|
|
244
|
+
*/
|
|
245
|
+
private canonicalNestedArgs(input: ToolInput): string {
|
|
246
|
+
const raw = canonical(input);
|
|
247
|
+
return raw.length > NESTED_ARG_STORAGE_LIMIT ? raw.slice(0, NESTED_ARG_STORAGE_LIMIT) : raw;
|
|
248
|
+
}
|
|
249
|
+
|
|
250
|
+
/**
|
|
251
|
+
* Stricter detection for calls a codemode script made (`parentToolCallId` set).
|
|
252
|
+
* Inside one script a repeated call is never progress — the script can store
|
|
253
|
+
* and reuse its first result — so identical args block at
|
|
254
|
+
* `NESTED_REPEAT_THRESHOLD` (2), and an identical retry of a call that
|
|
255
|
+
* already errored in this script blocks immediately. The block reason
|
|
256
|
+
* becomes the script's tool error; `escalate` is true when this signature
|
|
257
|
+
* was already blocked in an earlier script this prompt.
|
|
258
|
+
*/
|
|
259
|
+
checkNested(parentToolCallId: string, toolName: string, input: ToolInput): NestedDecision {
|
|
260
|
+
if (this.opts.toolExclude.has(toolName)) return Result.err(undefined);
|
|
261
|
+
const args = this.canonicalNestedArgs(input);
|
|
262
|
+
const sig = `${toolName}:${args}`;
|
|
263
|
+
const pool = this.nestedSigs.get(parentToolCallId) ?? [];
|
|
264
|
+
const repeats = pool.filter((s) => s.sig === sig).length;
|
|
265
|
+
const reasons = collectNestedBlockReasons(
|
|
266
|
+
toolName,
|
|
267
|
+
repeats,
|
|
268
|
+
this.nestedErrorSigs.get(parentToolCallId)?.has(sig) === true,
|
|
269
|
+
() => this.nearRepeats(toolName, args, pool),
|
|
270
|
+
);
|
|
271
|
+
if (reasons.length === 0) return Result.err(undefined);
|
|
272
|
+
|
|
273
|
+
const blockedCount = (this.nestedBlockedBySig.get(sig) ?? 0) + 1;
|
|
274
|
+
this.nestedBlockedBySig.set(sig, blockedCount);
|
|
275
|
+
return Result.ok({
|
|
276
|
+
reason:
|
|
277
|
+
reasons.join("; ") +
|
|
278
|
+
`. BLOCKED by anti-doom-loop — inside a script, store the first result in a variable and reuse it instead of calling again.`,
|
|
279
|
+
escalate: blockedCount > 1,
|
|
280
|
+
});
|
|
281
|
+
}
|
|
282
|
+
|
|
283
|
+
/** Record an allowed nested call into its parent script's window. */
|
|
284
|
+
recordNested(
|
|
285
|
+
parentToolCallId: string,
|
|
286
|
+
toolName: string,
|
|
287
|
+
toolCallId: string,
|
|
288
|
+
input: ToolInput,
|
|
289
|
+
): void {
|
|
290
|
+
if (this.opts.toolExclude.has(toolName)) return;
|
|
291
|
+
const args = this.canonicalNestedArgs(input);
|
|
292
|
+
let pool = this.nestedSigs.get(parentToolCallId);
|
|
293
|
+
if (!pool) {
|
|
294
|
+
pool = [];
|
|
295
|
+
this.nestedSigs.set(parentToolCallId, pool);
|
|
296
|
+
}
|
|
297
|
+
pool.push({ sig: `${toolName}:${args}`, tool: toolName, args });
|
|
298
|
+
if (pool.length > NESTED_POOL_LIMIT) pool.splice(0, pool.length - NESTED_POOL_LIMIT);
|
|
299
|
+
this.nestedIds.set(toolCallId, { tool: toolName, args });
|
|
300
|
+
this.nestedCallCount++;
|
|
301
|
+
}
|
|
302
|
+
|
|
303
|
+
/** Wire a nested result back: an errored signature blocks its identical retry. */
|
|
304
|
+
recordNestedResult(parentToolCallId: string, toolCallId: string, isError: boolean): void {
|
|
305
|
+
const entry = this.nestedIds.get(toolCallId);
|
|
306
|
+
this.nestedIds.delete(toolCallId);
|
|
307
|
+
if (!isError || !entry) return;
|
|
308
|
+
let errs = this.nestedErrorSigs.get(parentToolCallId);
|
|
309
|
+
if (!errs) {
|
|
310
|
+
errs = new Set();
|
|
311
|
+
this.nestedErrorSigs.set(parentToolCallId, errs);
|
|
312
|
+
}
|
|
313
|
+
errs.add(`${entry.tool}:${entry.args}`);
|
|
314
|
+
if (errs.size > NESTED_POOL_LIMIT) {
|
|
315
|
+
const oldest = errs.values().next();
|
|
316
|
+
if (!oldest.done) errs.delete(oldest.value);
|
|
317
|
+
}
|
|
318
|
+
}
|
|
319
|
+
|
|
320
|
+
/** Nested codemode counters for /loopcheck (allowed calls recorded, blocks issued). */
|
|
321
|
+
nestedStats(): NestedStats {
|
|
322
|
+
return {
|
|
323
|
+
calls: this.nestedCallCount,
|
|
324
|
+
blocks: [...this.nestedBlockedBySig.values()].reduce((a, b) => a + b, 0),
|
|
325
|
+
};
|
|
326
|
+
}
|
|
327
|
+
|
|
328
|
+
/** Windowed same-tool entries whose args are ≥55% shared tokens (identical counts). */
|
|
329
|
+
private nearRepeats(
|
|
330
|
+
toolName: string,
|
|
331
|
+
args: string,
|
|
332
|
+
pool: readonly { tool: string; args: string }[] = this.recentSigs,
|
|
333
|
+
): number {
|
|
334
|
+
let n = 0;
|
|
335
|
+
for (const s of pool) {
|
|
336
|
+
if (s.tool !== toolName) continue;
|
|
337
|
+
const sim = argSimilarity(args, s.args);
|
|
338
|
+
if (sim !== null && sim >= NEAR_ARGS_SIMILARITY_THRESHOLD) n++;
|
|
339
|
+
}
|
|
340
|
+
return n;
|
|
341
|
+
}
|
|
342
|
+
|
|
207
343
|
recordResult(toolName: string, error: boolean): void {
|
|
208
344
|
this.recentResults.push({ tool: toolName, error, ts: Date.now() });
|
|
209
345
|
this.evictResults();
|
|
@@ -390,10 +526,12 @@ export class LoopDetector {
|
|
|
390
526
|
.map(([s, n]) => `"${truncate(s, 24)}" x${n}`)
|
|
391
527
|
.join(", ");
|
|
392
528
|
|
|
529
|
+
const nested = this.nestedStats();
|
|
393
530
|
return (
|
|
394
531
|
`calls[${this.recentSigs.length}] ${topSigs || "none"}; ` +
|
|
395
532
|
`texts[${this.recentTexts.length}] ${topTexts || "none"}; ` +
|
|
396
|
-
`wastedTokens=${this.wastedTokens}`
|
|
533
|
+
`wastedTokens=${this.wastedTokens}; ` +
|
|
534
|
+
`nested: calls=${nested.calls} blocks=${nested.blocks}`
|
|
397
535
|
);
|
|
398
536
|
}
|
|
399
537
|
|
|
@@ -405,6 +543,11 @@ export class LoopDetector {
|
|
|
405
543
|
this.lastText = null;
|
|
406
544
|
this.textStreak = 0;
|
|
407
545
|
this.wastedTokens = 0;
|
|
546
|
+
this.nestedSigs.clear();
|
|
547
|
+
this.nestedIds.clear();
|
|
548
|
+
this.nestedBlockedBySig.clear();
|
|
549
|
+
this.nestedErrorSigs.clear();
|
|
550
|
+
this.nestedCallCount = 0;
|
|
408
551
|
}
|
|
409
552
|
|
|
410
553
|
summary(): string {
|
|
@@ -430,6 +573,95 @@ export const MIN_REPEAT_CHUNK = 16;
|
|
|
430
573
|
/** Jaccard similarity threshold for "near-identical" consecutive texts. */
|
|
431
574
|
export const TEXT_SIMILARITY_THRESHOLD = 0.55;
|
|
432
575
|
|
|
576
|
+
/** Shared-whitespace-token threshold for near-identical tool-call arguments. */
|
|
577
|
+
export const NEAR_ARGS_SIMILARITY_THRESHOLD = 0.55;
|
|
578
|
+
|
|
579
|
+
/**
|
|
580
|
+
* Minimum token union before two arg strings are comparable. Short inputs
|
|
581
|
+
* ({"i":"…","op":"view"}) would score 1.0 on one shared token; they rely on
|
|
582
|
+
* exact matching only. Long inputs — shell commands — always clear this.
|
|
583
|
+
*/
|
|
584
|
+
export const NEAR_ARGS_MIN_UNION = 5;
|
|
585
|
+
|
|
586
|
+
/** Decision for a nested (codemode) tool call — same shape as BlockDecision. */
|
|
587
|
+
export type NestedDecision = Result<BlockDecision, undefined>;
|
|
588
|
+
|
|
589
|
+
/**
|
|
590
|
+
* Block reasons for one nested codemode call: identical repeats (2× rule),
|
|
591
|
+
* an identical retry of a call that already errored in this script, and —
|
|
592
|
+
* only when neither fired — near-identical args at the same 2× threshold.
|
|
593
|
+
* `nearRepeats` is lazy so random traffic never pays the tokenization cost.
|
|
594
|
+
*/
|
|
595
|
+
function collectNestedBlockReasons(
|
|
596
|
+
toolName: string,
|
|
597
|
+
repeats: number,
|
|
598
|
+
erroredBefore: boolean,
|
|
599
|
+
nearRepeats: () => number,
|
|
600
|
+
): string[] {
|
|
601
|
+
const reasons: string[] = [];
|
|
602
|
+
if (repeats + 1 >= NESTED_REPEAT_THRESHOLD) {
|
|
603
|
+
reasons.push(
|
|
604
|
+
`"${toolName}" was called with identical arguments ${repeats + 1} times inside one codemode script`,
|
|
605
|
+
);
|
|
606
|
+
}
|
|
607
|
+
if (erroredBefore) {
|
|
608
|
+
reasons.push(`the identical "${toolName}" call already failed inside this script`);
|
|
609
|
+
}
|
|
610
|
+
if (reasons.length === 0 && repeats === 0 && nearRepeats() + 1 >= NESTED_REPEAT_THRESHOLD) {
|
|
611
|
+
reasons.push(
|
|
612
|
+
`"${toolName}" was called with near-identical arguments ${nearRepeats() + 1} times inside one codemode script (only minor changes)`,
|
|
613
|
+
);
|
|
614
|
+
}
|
|
615
|
+
return reasons;
|
|
616
|
+
}
|
|
617
|
+
|
|
618
|
+
/** Counters for nested codemode traffic, reported by /loopcheck. */
|
|
619
|
+
export interface NestedStats {
|
|
620
|
+
calls: number;
|
|
621
|
+
blocks: number;
|
|
622
|
+
}
|
|
623
|
+
|
|
624
|
+
/**
|
|
625
|
+
* Nested codemode calls (pi ≥ 0.99, `parentToolCallId` set) block at a stricter
|
|
626
|
+
* threshold than top-level calls: inside one script a repeated call is never
|
|
627
|
+
* progress — the script can reuse its first result — so 2 occurrences block
|
|
628
|
+
* where top-level detection needs 3.
|
|
629
|
+
*/
|
|
630
|
+
export const NESTED_REPEAT_THRESHOLD = 2;
|
|
631
|
+
|
|
632
|
+
/** Max chars of one nested call's canonical args kept for identity/similarity. */
|
|
633
|
+
export const NESTED_ARG_STORAGE_LIMIT = 100_000;
|
|
634
|
+
|
|
635
|
+
/**
|
|
636
|
+
* Max recent calls a codemode script's window keeps. Entries hold up to
|
|
637
|
+
* `NESTED_ARG_STORAGE_LIMIT` chars each, so an unbounded per-script window
|
|
638
|
+
* would let one script pin hundreds of MB; in-script loops repeat adjacent
|
|
639
|
+
* calls, which a recent window still catches.
|
|
640
|
+
*/
|
|
641
|
+
export const NESTED_POOL_LIMIT = 100;
|
|
642
|
+
// ponytail: 0.55 misses ONE shape — short (~11-token) commands that change
|
|
643
|
+
// BOTH ends at once (added wrapper + changed tail glue → ~0.53); single-axis
|
|
644
|
+
// drift and longer commands clear. Upgrade path if that shape shows up:
|
|
645
|
+
// tokenize on punctuation boundaries instead of whitespace.
|
|
646
|
+
|
|
647
|
+
/**
|
|
648
|
+
* Case-insensitive whitespace-token Jaccard over canonical args.
|
|
649
|
+
* Full tokens (no word filtering): a changed field glues into its own token,
|
|
650
|
+
* so only genuinely shared structure counts — same command with a rephrased
|
|
651
|
+
* `i` label scores ~0.8, two different commands score far below the threshold.
|
|
652
|
+
* Returns null when the union is too small to judge.
|
|
653
|
+
*/
|
|
654
|
+
export function argSimilarity(a: string, b: string): number | null {
|
|
655
|
+
const as = new Set(a.toLowerCase().split(/\s+/).filter(Boolean));
|
|
656
|
+
const bs = new Set(b.toLowerCase().split(/\s+/).filter(Boolean));
|
|
657
|
+
if (as.size === 0 || bs.size === 0) return null;
|
|
658
|
+
let inter = 0;
|
|
659
|
+
for (const t of as) if (bs.has(t)) inter++;
|
|
660
|
+
const union = as.size + bs.size - inter;
|
|
661
|
+
if (union < NEAR_ARGS_MIN_UNION) return null;
|
|
662
|
+
return inter / union;
|
|
663
|
+
}
|
|
664
|
+
|
|
433
665
|
/** Stable string form of a tool input (used for token estimation). */
|
|
434
666
|
export function stringify(input: ToolInput): string {
|
|
435
667
|
return JSON.stringify(input) ?? "";
|
|
@@ -686,5 +918,70 @@ if (import.meta.main) {
|
|
|
686
918
|
"distinct parallel args are not spam",
|
|
687
919
|
);
|
|
688
920
|
|
|
921
|
+
// 14. command-tail/wrapper drift: same poll wrapped or slightly reworded —
|
|
922
|
+
// near-identical token overlap catches what exact identity cannot.
|
|
923
|
+
d.reset();
|
|
924
|
+
const ghTail = (command: string) => ({ command, cwd: "/w/flash", timeout: 30 });
|
|
925
|
+
assert.ok(d.check("bash", ghTail("gh run list --branch main --limit 3 2>&1 | head -6")).isErr());
|
|
926
|
+
d.record("bash", ghTail("gh run list --branch main --limit 3 2>&1 | head -6"));
|
|
927
|
+
assert.ok(
|
|
928
|
+
d
|
|
929
|
+
.check("bash", ghTail("sleep 150; gh run list --branch main --limit 3 2>&1 | head -6"))
|
|
930
|
+
.isErr(),
|
|
931
|
+
);
|
|
932
|
+
d.record("bash", ghTail("sleep 150; gh run list --branch main --limit 3 2>&1 | head -6"));
|
|
933
|
+
const nearHit = d.check(
|
|
934
|
+
"bash",
|
|
935
|
+
ghTail("timeout 30; gh run list --branch main --limit 3 2>&1 | head -6"),
|
|
936
|
+
);
|
|
937
|
+
assert.ok(nearHit.isOk(), "3rd command-tail variant should block");
|
|
938
|
+
if (nearHit.isOk()) {
|
|
939
|
+
assert.match(nearHit.value.reason, /near-identical arguments 3 times/);
|
|
940
|
+
assert.equal(nearHit.value.escalate, false, "first near block does not escalate");
|
|
941
|
+
}
|
|
942
|
+
// genuinely different command with the same tool stays unblocked
|
|
943
|
+
assert.ok(d.check("bash", { command: "cargo test -p pools 2>&1 | tail -5" }).isErr());
|
|
944
|
+
|
|
945
|
+
// 15. nested codemode calls: stricter 2× threshold, error-retry block, no
|
|
946
|
+
// leakage into the top-level window
|
|
947
|
+
d.reset();
|
|
948
|
+
assert.ok(d.checkNested("p1", "read", { path: "a.ts" }).isErr(), "1st nested call passes");
|
|
949
|
+
d.recordNested("p1", "read", "p1/1", { path: "a.ts" });
|
|
950
|
+
const nHit = d.checkNested("p1", "read", { path: "a.ts" });
|
|
951
|
+
assert.ok(nHit.isOk(), "2nd identical nested call blocks");
|
|
952
|
+
if (nHit.isOk()) {
|
|
953
|
+
assert.equal(nHit.value.escalate, false, "first in-script block does not escalate");
|
|
954
|
+
assert.match(nHit.value.reason, /identical arguments 2 times inside one codemode script/);
|
|
955
|
+
}
|
|
956
|
+
assert.ok(d.checkNested("p2", "read", { path: "a.ts" }).isErr(), "a new script starts clean");
|
|
957
|
+
// identical-after-error blocks even at the first occurrence in a parent
|
|
958
|
+
d.recordNested("p2", "bash", "p2/1", { command: "npm test" });
|
|
959
|
+
d.recordNestedResult("p2", "p2/1", true);
|
|
960
|
+
const errHit = d.checkNested("p2", "bash", { command: "npm test" });
|
|
961
|
+
assert.ok(errHit.isOk(), "identical retry after a nested error blocks");
|
|
962
|
+
if (errHit.isOk()) assert.match(errHit.value.reason, /already failed inside this script/);
|
|
963
|
+
// a NEW script re-issuing a signature blocked before escalates
|
|
964
|
+
const crossScript = d.checkNested("p3", "bash", { command: "npm test" });
|
|
965
|
+
assert.ok(crossScript.isErr(), "fresh script with a fresh call passes");
|
|
966
|
+
d.recordNested("p3", "bash", "p3/1", { command: "npm test" });
|
|
967
|
+
const crossBlock = d.checkNested("p3", "bash", { command: "npm test" });
|
|
968
|
+
assert.ok(crossBlock.isOk() && crossBlock.value.escalate, "cross-script repeat escalates");
|
|
969
|
+
// nested traffic never enters the top-level window
|
|
970
|
+
d.reset();
|
|
971
|
+
d.recordNested("p", "bash", "p/1", { command: "poll" });
|
|
972
|
+
d.recordNested("p", "bash", "p/2", { command: "poll" });
|
|
973
|
+
assert.ok(
|
|
974
|
+
d.check("bash", { command: "poll" }).isErr(),
|
|
975
|
+
"top-level window empty of nested traffic",
|
|
976
|
+
);
|
|
977
|
+
// huge nested payloads stay capped and never throw
|
|
978
|
+
assert.doesNotThrow(() => {
|
|
979
|
+
const huge = "x".repeat(300_000);
|
|
980
|
+
for (let i = 0; i < 50; i++) {
|
|
981
|
+
d.checkNested("p", "read", { path: `f${i}.ts`, content: huge });
|
|
982
|
+
d.recordNested("p", "read", `p/h${i}`, { path: `f${i}.ts`, content: huge });
|
|
983
|
+
}
|
|
984
|
+
});
|
|
985
|
+
|
|
689
986
|
console.log("detector self-check: all assertions passed");
|
|
690
987
|
}
|
package/extensions/index.ts
CHANGED
|
@@ -7,6 +7,9 @@
|
|
|
7
7
|
*
|
|
8
8
|
* - identical (tool, args) repeated `PI_ANTI_LOOP_REPEATS` times (default 3)
|
|
9
9
|
* in the last `PI_ANTI_LOOP_WINDOW` calls → block with an instructive reason
|
|
10
|
+
* - near-identical same-tool args (≥ 55% shared whitespace tokens, union ≥ 5)
|
|
11
|
+
* count toward the same threshold — sleep-wrapped polls, flag/pattern/tail
|
|
12
|
+
* drift that exact identity misses by construction
|
|
10
13
|
* - the same tool failing `PI_ANTI_LOOP_FAILS` consecutive times (default 3)
|
|
11
14
|
* → block with a "stop retrying, fix the root cause" reason
|
|
12
15
|
* - the model repeating text: verbatim, near-identical (token similarity),
|
|
@@ -19,8 +22,12 @@
|
|
|
19
22
|
* instructive reason (that is the steer); re-issuing the exact same blocked
|
|
20
23
|
* call aborts the turn.
|
|
21
24
|
*
|
|
22
|
-
* Counters reset on every user prompt
|
|
23
|
-
* in the session is never a false positive.
|
|
25
|
+
* Counters reset on every user prompt (`message_end` with `role:"user"`), so
|
|
26
|
+
* a task legitimately repeated later in the session is never a false positive.
|
|
27
|
+
* Run restarts (auto-continue, steers/resumes, async-result drains) do NOT
|
|
28
|
+
* reset — hosts emit `before_agent_start` on every run start, and resetting
|
|
29
|
+
* there wiped counters between the iterations of cross-run loops. Disable with
|
|
30
|
+
* PI_ANTI_LOOP_DISABLE=1.
|
|
24
31
|
*
|
|
25
32
|
* All logic lives in `controller.ts` (pure, pi-free, unit-tested); this file
|
|
26
33
|
* is a thin adapter wiring it to pi's event loop. The pi API is consumed
|
|
@@ -69,6 +76,13 @@ const STEER_TEXT =
|
|
|
69
76
|
"Stop. Re-read the actual error output, pick ONE different action, and execute it. " +
|
|
70
77
|
"If you are stuck, ask the user instead of retrying.";
|
|
71
78
|
|
|
79
|
+
/** Steer delivered after a codemode script swallowed a nested block: the script
|
|
80
|
+
* got the block reason as a tool error, but only its return reaches the model. */
|
|
81
|
+
const NESTED_STEER_TEXT =
|
|
82
|
+
"Anti-doom-loop steering: a codemode script repeated identical tool calls inside itself. " +
|
|
83
|
+
"Inside a script, call each tool once, store the result in a variable, and reuse it — " +
|
|
84
|
+
"never poll or retry the same call in a loop. If the calls keep failing, stop and ask the user.";
|
|
85
|
+
|
|
72
86
|
/** Queued once after an abort so the work can continue with a fresh approach. */
|
|
73
87
|
const RESUME_TEXT =
|
|
74
88
|
"Anti-doom-loop: the previous run was aborted because it looped. " +
|
|
@@ -82,10 +96,6 @@ export default function (pi: PiLike): void {
|
|
|
82
96
|
|
|
83
97
|
pi.on("session_start", () => reset());
|
|
84
98
|
|
|
85
|
-
// Fresh counters per user prompt: only the loop happening *right now* counts.
|
|
86
|
-
// Internal reset keeps session-scoped steers/aborts/resume budget.
|
|
87
|
-
pi.on("before_agent_start", () => controller.reset());
|
|
88
|
-
|
|
89
99
|
pi.on("tool_call", (event: ToolCallEventLite, ctx: CtxLite) => {
|
|
90
100
|
// SAFETY: pi delivers JSON-safe tool arguments, which is exactly the ToolInput domain.
|
|
91
101
|
// Decode them into the ToolInput domain type at this I/O boundary before the controller sees them.
|
|
@@ -93,6 +103,7 @@ export default function (pi: PiLike): void {
|
|
|
93
103
|
event.toolName,
|
|
94
104
|
event.input as ToolInput,
|
|
95
105
|
event.toolCallId,
|
|
106
|
+
event.parentToolCallId,
|
|
96
107
|
);
|
|
97
108
|
if (outcome === null) return;
|
|
98
109
|
if (outcome.escalate) {
|
|
@@ -103,13 +114,30 @@ export default function (pi: PiLike): void {
|
|
|
103
114
|
});
|
|
104
115
|
|
|
105
116
|
pi.on("tool_result", (event: ToolResultEventLite) => {
|
|
106
|
-
controller.onToolResult(
|
|
117
|
+
controller.onToolResult(
|
|
118
|
+
event.toolName,
|
|
119
|
+
event.toolCallId,
|
|
120
|
+
event.isError === true,
|
|
121
|
+
event.parentToolCallId,
|
|
122
|
+
);
|
|
107
123
|
});
|
|
108
124
|
|
|
109
125
|
// Text-only doom loops (model re-emits/rephrases the same thing with no
|
|
110
126
|
// tool calls) never reach tool_call. Steer first, abort as escalation,
|
|
111
|
-
// then a bounded auto-resume so
|
|
127
|
+
// then a bounded auto-resume so work continues.
|
|
112
128
|
pi.on("message_end", (event: MessageEndEventLite, ctx: CtxLite) => {
|
|
129
|
+
// Fresh counters per genuine user prompt: only the loop happening *right
|
|
130
|
+
// now* counts (session-scoped steers/aborts/resume budget survive).
|
|
131
|
+
// Deliberately NOT on `before_agent_start`: hosts emit that on every run
|
|
132
|
+
// start — auto-continue turns, steers/resumes, advisor cards, and queued
|
|
133
|
+
// async-result drains — which wiped counters between the iterations of
|
|
134
|
+
// cross-run doom loops, so they never reached the repeat threshold.
|
|
135
|
+
// `role:"user"` input messages are emitted with the run's input messages
|
|
136
|
+
// (agent-core `emitInputMessages`) on omp and pi alike.
|
|
137
|
+
if (event.message.role === "user") {
|
|
138
|
+
controller.reset();
|
|
139
|
+
return;
|
|
140
|
+
}
|
|
113
141
|
// SAFETY: pi delivers message content as JSON-safe blocks matching MessageContent.
|
|
114
142
|
// Decode the untyped message content into MessageContent at this boundary.
|
|
115
143
|
const outcome = controller.onMessageEnd(
|
|
@@ -119,9 +147,10 @@ export default function (pi: PiLike): void {
|
|
|
119
147
|
if (outcome === null) return;
|
|
120
148
|
|
|
121
149
|
if (outcome.action === "steer") {
|
|
150
|
+
const text = outcome.nested ? NESTED_STEER_TEXT : STEER_TEXT;
|
|
122
151
|
ctx.ui.notify(`Anti-doom-loop: ${outcome.reason}`, "warning");
|
|
123
152
|
pi.sendMessage?.(
|
|
124
|
-
{ customType: "anti-doom-loop", content:
|
|
153
|
+
{ customType: "anti-doom-loop", content: text, display: true },
|
|
125
154
|
{ deliverAs: "steer", triggerTurn: true },
|
|
126
155
|
);
|
|
127
156
|
return;
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "pi-anti-doom-loop",
|
|
3
|
-
"version": "0.0.
|
|
3
|
+
"version": "0.0.12",
|
|
4
4
|
"description": "Detect and break agent doom loops in pi: blocks identical repeated tool calls and blind retries before they burn tokens.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"anti-doom-loop",
|
|
@@ -29,28 +29,18 @@
|
|
|
29
29
|
"publishConfig": {
|
|
30
30
|
"access": "public"
|
|
31
31
|
},
|
|
32
|
-
"scripts": {
|
|
33
|
-
"check": "npm run test && tsc --noEmit && oxlint --deny-warnings && oxfmt --check .",
|
|
34
|
-
"prepublishOnly": "npm run check",
|
|
35
|
-
"test": "node --test tests/*.test.ts",
|
|
36
|
-
"guard": "node scripts/guard-publish.ts",
|
|
37
|
-
"lint": "oxlint --deny-warnings",
|
|
38
|
-
"format": "oxfmt .",
|
|
39
|
-
"format:check": "oxfmt --check .",
|
|
40
|
-
"prepare": "effect-tsgo patch --no-typescript --oxlint"
|
|
41
|
-
},
|
|
42
32
|
"dependencies": {
|
|
33
|
+
"@effect/tsgo": "0.36.4",
|
|
43
34
|
"better-result": "^3.0.0",
|
|
44
35
|
"effect": "^4.0.0-rc.108"
|
|
45
36
|
},
|
|
46
37
|
"devDependencies": {
|
|
47
38
|
"@earendil-works/pi-coding-agent": "*",
|
|
48
|
-
"@effect/tsgo": "^0.36.4",
|
|
49
39
|
"@oxlint/plugins": "1.77.0",
|
|
50
40
|
"@types/node": "^22.0.0",
|
|
51
|
-
"oxfmt": "
|
|
41
|
+
"oxfmt": "0.62.0",
|
|
52
42
|
"oxlint": "1.77.0",
|
|
53
|
-
"oxlint-tsgolint": "
|
|
43
|
+
"oxlint-tsgolint": "7.0.2001",
|
|
54
44
|
"typescript": "^5.6.0"
|
|
55
45
|
},
|
|
56
46
|
"peerDependencies": {
|
|
@@ -63,5 +53,14 @@
|
|
|
63
53
|
"extensions": [
|
|
64
54
|
"./extensions"
|
|
65
55
|
]
|
|
56
|
+
},
|
|
57
|
+
"scripts": {
|
|
58
|
+
"check": "pnpm run test && tsc --noEmit && oxlint --deny-warnings && oxfmt --check .",
|
|
59
|
+
"test": "node --test tests/*.test.ts tests/e2e/*.test.ts",
|
|
60
|
+
"guard": "node scripts/guard-publish.ts",
|
|
61
|
+
"e2e:pi": "node --test tests/e2e/pi.test.ts",
|
|
62
|
+
"lint": "oxlint --deny-warnings",
|
|
63
|
+
"format": "oxfmt .",
|
|
64
|
+
"format:check": "oxfmt --check ."
|
|
66
65
|
}
|
|
67
|
-
}
|
|
66
|
+
}
|