pi-antiloop 1.5.1 → 1.6.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +145 -11
- package/package.json +2 -2
- package/src/commands.ts +40 -1
- package/src/config.ts +11 -0
- package/src/detect.ts +437 -8
- package/src/index.ts +106 -5
- package/src/types.ts +49 -1
package/README.md
CHANGED
|
@@ -6,15 +6,16 @@
|
|
|
6
6
|
|
|
7
7
|
# Antiloop — Loop Detection and Break for pi
|
|
8
8
|
|
|
9
|
-
**Antiloop watches every assistant message, tool call and thinking block, and forces the model out of reasoning loops before they eat your context and your patience.**
|
|
9
|
+
**Antiloop watches every assistant message, tool call and thinking block, and forces the model out of reasoning loops before they eat your context and your patience.** Six simultaneous detection strategies (text similarity, tool-call sequences, thinking content, structural openings, degenerate repetition, no-progress outcome runs) find loops that humans miss — and progressive intervention (warning → force break → abort) tells the model to take a different approach, without you having to babysit it.
|
|
10
10
|
|
|
11
11
|
---
|
|
12
12
|
|
|
13
13
|
## Features
|
|
14
14
|
|
|
15
|
-
- **
|
|
15
|
+
- **Six detection strategies** — text repetition (trigram Jaccard + Levenshtein), tool-call sequences (name + near-identical arguments + same outcome — result-aware, so retries that make progress don't false-positive), thinking blocks, structural opening-phrase patterns, **degenerate repetition** (a single message/call stuck repeating one word hundreds of times — the `noguerol ×5145` meltdown — caught at `message_end` with no repeated peer needed, and degenerate bash commands blocked before they execute), and **no-progress outcome runs** (the NFS `test A…QQQ` class: dozens of near-identical re-runs of the same experiment, every one ending in the *same failing outcome* — args mutate so tool-loop can't see it; the repeated failure signature can)
|
|
16
|
+
- **Stays alive in long sessions** — tracked messages carry a monotonic sequence number, so trimming the sliding window can never make the dedupe guard skip later messages (a bug that silently blinded antiloop after ~15 tracked messages)
|
|
16
17
|
- **Task-stream recognition (batch work)** — when another extension (e.g. `punched` appending lines to pi.md, or `plan` adding tasks) makes the model call the *same* tool many times with *different* content, antiloop recognizes it as N distinct tasks of one type and stays silent — no warning, no force break. A genuine loop (the *same* call repeated verbatim) is still caught
|
|
17
|
-
- **Progressive intervention** — `warning` reminds the model to vary its approach; `force break`
|
|
18
|
+
- **Progressive intervention** — `warning` reminds the model to vary its approach; `force break` steers a real break message into the running agent before its next LLM call; `abort` stops the run entirely
|
|
18
19
|
- **Configurable thresholds** — independent dials for similarity cutoff, warning/force-break/abort counts, detection window, and which strategies are on
|
|
19
20
|
- **Sliding window** — only the last N messages are compared, so detection is O(N) in the window size, not in the full session
|
|
20
21
|
- **Live footer indicator** — `🔄 antiloop(on|off)` in the footer, per spec, with the current level (`⚠️/🛑/🚨`) and consecutive count; an interactive TUI footer adds a keyboard toggle (`esc+a` by default, configurable/off, honored only while idle) and preserves the built-in footer's pwd/branch/context/model info
|
|
@@ -98,10 +99,14 @@ Detection strategies:
|
|
|
98
99
|
Text loops: ✅
|
|
99
100
|
Tool loops: ✅
|
|
100
101
|
Thinking loops: ✅
|
|
102
|
+
Degenerate: ✅
|
|
103
|
+
Outcome (no-progress): ✅
|
|
101
104
|
|
|
102
105
|
Recent detections:
|
|
103
106
|
[text] Text similarity 85% with message 3 (2m ago)
|
|
104
107
|
[tool] repeated 3x: bash (5m ago)
|
|
108
|
+
[degenerate] bash command: degenerate repetition — "noguerol" ×5145 (5m ago)
|
|
109
|
+
[outcome] no progress: 9 near-identical bash attempts with the same failing outcome (5m ago)
|
|
105
110
|
```
|
|
106
111
|
|
|
107
112
|
### `/antiloop config`
|
|
@@ -123,6 +128,9 @@ Grouped interactive menu showing the current value in each option:
|
|
|
123
128
|
- **🔧 call similarity** — `99 / 95 / 90 / 80%` — how identical tool-call *arguments* must be to count as the same call (default 95%: only near-identical repeats loop)
|
|
124
129
|
- **🔁 call repeats** — `1 / 2 / 3` — how many times the same call must repeat before it flags (default 2)
|
|
125
130
|
- **🧾 result similarity** — `95 / 80 / 60%` — how similar captured results must be to count as the *same outcome*; a repeated command that starts producing a different result is progress, not a loop (default 80%)
|
|
131
|
+
- **🌀 degenerate run** — `8 / 16 / 24 / 48` — identical words in a row inside ONE message/call before it counts as stuck generation (default 16; the `noguerol ×5145` class)
|
|
132
|
+
- **⛔ block degenerate bash** — on/off — refuse a degenerate bash command before it executes, feeding the reason back to the model (default on)
|
|
133
|
+
- **📉 no-progress after** — `4 / 6 / 8 / 12` — same failing outcome repeated this many times (near-identical args) before flagging (default 8; the NFS `test A…QQQ` class)
|
|
126
134
|
|
|
127
135
|
**📋 Task streams** — batch work (punched_log, plan_manager, …) is N tasks of one type, not a loop
|
|
128
136
|
- **📋 task streams** — on/off — recognize that batch work and stay silent
|
|
@@ -133,6 +141,8 @@ Grouped interactive menu showing the current value in each option:
|
|
|
133
141
|
- **📝 text** — on/off — detect repeated text messages
|
|
134
142
|
- **🔧 tools** — on/off — detect repeated tool calls
|
|
135
143
|
- **🧠 thinking** — on/off — detect repeated internal reasoning
|
|
144
|
+
- **🌀 degenerate** — on/off — detect one message stuck repeating a single word/token (no repeated peer needed)
|
|
145
|
+
- **📉 outcome** — on/off — detect many near-identical attempts all ending in the same failing outcome (no progress)
|
|
136
146
|
|
|
137
147
|
**🧹 reset state** — clear all counters and history
|
|
138
148
|
|
|
@@ -160,13 +170,23 @@ batch no detections → silent (exp silent — 98.9% args would match without
|
|
|
160
170
|
loop still detected → tool (exp tool — identical repeats are NOT a stream) ✅
|
|
161
171
|
loop survives batch gate→ tool (exp tool — bash repeats are real) ✅
|
|
162
172
|
stream needs ≥3 calls → no stream (exp no stream at 2 calls) ✅
|
|
173
|
+
degenerate first sight → degenerate (bash command: "noguerol" ×402 …) ✅ ← v1.6: ONE meltdown message, no peer
|
|
174
|
+
degenerate legit cmd → ok (exp ok) ✅
|
|
175
|
+
degenerate interleaved → flag (noguerol ×150) (exp flag — freq/share clause) ✅
|
|
176
|
+
degenerate glued token → flag (noguerol ×300) (exp flag — perfect power) ✅
|
|
177
|
+
degenerate turn weight → degenerate 2, text 1 (exp 2, 1) ✅
|
|
178
|
+
outcome fires on 9th → outcome (9 near-identical bash attempts, same failing outcome) ✅ ← v1.6.1: NFS test-A…QQQ class
|
|
179
|
+
outcome needs 8 prior → silent (exp silent at 5 attempts) ✅
|
|
180
|
+
outcome converging sweep → silent (exp silent — outcomes differ = progress) ✅
|
|
181
|
+
outcome diff failures → silent (exp silent — error changed = progress) ✅
|
|
182
|
+
outcome identical OKs → silent (exp silent — success repeats ≠ loop) ✅
|
|
163
183
|
```
|
|
164
184
|
|
|
165
185
|
## How It Works
|
|
166
186
|
|
|
167
187
|
### Detection pipeline
|
|
168
188
|
|
|
169
|
-
After every assistant `message_end` event, antiloop extracts the new content (text, thinking, tool calls — including their ids) and pushes it onto a sliding window of the last `detectionWindow + 5` messages. Detection itself runs at `turn_end`, once the tool results are known: results are fingerprinted and attached to the tracked calls, then the active detection strategies run against the window:
|
|
189
|
+
After every assistant `message_end` event, antiloop extracts the new content (text, thinking, tool calls — including their ids) and pushes it onto a sliding window of the last `detectionWindow + 5` messages. Detection itself runs at `turn_end`, once the tool results are known: results are fingerprinted and attached to the tracked calls, then the active detection strategies run against the window. The one exception is the **degenerate** strategy: a single stuck message needs no peer and no tool result, so it is evaluated right at `message_end` — the only point before the message's own tool calls execute — and degenerate `bash` calls are also blocked at the `tool_call` hook.
|
|
170
190
|
|
|
171
191
|
| Strategy | What it compares | Algorithm |
|
|
172
192
|
|----------|------------------|-----------|
|
|
@@ -174,9 +194,11 @@ After every assistant `message_end` event, antiloop extracts the new content (te
|
|
|
174
194
|
| Tool | Tool name + arguments (+ captured result) | Sequence match + near-identical args (≥ `toolSimilarityThreshold`, default 95%) *and* ≥ `minToolRepeatCount` prior recurrences. **Result veto:** if both runs captured a result and the outcomes differ, it's progress, not a loop |
|
|
175
195
|
| Thinking | Internal reasoning/thinking blocks | Same as text |
|
|
176
196
|
| Structural | First 10 words of each message | Opening-phrase similarity ≥ 90% across ≥ 3 messages |
|
|
197
|
+
| Degenerate | One single message/call (no peer needed) | Run-length + frequency of identical words inside the payload: ≥ `degenerateMaxRun` (default 16) consecutive identical words, or one word ≥ `degenerateMaxFreq`× at ≥ `degenerateMaxShare` of all tokens; plus a perfect-power check for glued no-space tokens. Scanned at `message_end` — before the tool calls execute — and on every `bash` `tool_call` (blocking gate) |
|
|
198
|
+
| Outcome | Single tool calls across the window, after the last user input | ≥ `outcomeMinRepeats` (default 8) PRIOR attempts with args ≥ `outcomeArgSimilarity` (0.85) similar AND the same *failing* outcome (failure signatures compared at ≥ `outcomeSigThreshold`, 0.7; identical OK results never count — they're the norm for batches). Catches mutated re-run loops the tool detector can't see (labels/permutations change every turn) |
|
|
177
199
|
| Task stream | Same tool, many calls | When a tool appears ≥ `taskStreamMinCalls` times (default 3) in the window and *no two* calls are near-identical (`taskStreamTwinThreshold`, default 99%), the tool is an active batch: N different tasks of one type (e.g. `punched_log` appends, `plan_manager` task adds). Those calls are exempt from tool-loop detection, and text/thinking/structural patterns that only involve those batch messages are suppressed too. If even one call pair is a twin (the same task repeated), the tool is *not* a stream and detection proceeds normally |
|
|
178
200
|
|
|
179
|
-
Each detected pair becomes a `LoopDetection { type, similarity, messageIndices, description }` and the consecutive counter increases.
|
|
201
|
+
Each detected pair becomes a `LoopDetection { type, similarity, messageIndices, description }` and the consecutive counter increases (a degenerate turn counts `degenerateTurnWeight`, default 2 — warning on first sight).
|
|
180
202
|
|
|
181
203
|
### Intervention levels
|
|
182
204
|
|
|
@@ -184,11 +206,13 @@ Each detected pair becomes a `LoopDetection { type, similarity, messageIndices,
|
|
|
184
206
|
|-------|---------|----------|
|
|
185
207
|
| 0 (no loop) | — | Silent — passes the message through |
|
|
186
208
|
| 1 (warning) | `consecutiveDetections >= warningThreshold` | Notifies the user (`⚠️`) — no message is injected into the conversation, so the model's generation is never interrupted by the warning itself |
|
|
187
|
-
| 2 (force break) | `consecutiveDetections >= forceBreakThreshold` |
|
|
188
|
-
| 3 (abort) | `consecutiveDetections >= abortThreshold` | (Disabled by default)
|
|
209
|
+
| 2 (force break) | `consecutiveDetections >= forceBreakThreshold` | Steers a real break message into the running agent (`pi.sendUserMessage`, delivered right before its next LLM call) telling it to stop repeating and change approach |
|
|
210
|
+
| 3 (abort) | `consecutiveDetections >= abortThreshold` | (Disabled by default) Stops the run outright via `ctx.abort()` |
|
|
189
211
|
|
|
190
212
|
The level never de-escalates during an active loop; user input decays the consecutive counter naturally so a fresh prompt can break the cycle.
|
|
191
213
|
|
|
214
|
+
**Degenerate turns are handled earlier than the ladder:** the meltdown is detected at `message_end` (the same moment the text/tool-call payload is complete, *before* pi preflights and executes its tools). A single degenerate turn already adds `degenerateTurnWeight` (2) consecutive points → warning on first sight; the second consecutive meltdown → force break steer; after the steer, further degenerate output counts against `ignoredSteerLimit` → hard stop. Degenerate `bash` calls are additionally refused by the `tool_call` gate (`blockDegenerateBash`) — the command never runs, and the block reason is fed back to the model as the tool error so it can still change approach.
|
|
215
|
+
|
|
192
216
|
### Similarity scoring
|
|
193
217
|
|
|
194
218
|
```
|
|
@@ -262,6 +286,92 @@ Tunables: `detectTaskStreams` (master switch), `taskStreamMinCalls` (batch
|
|
|
262
286
|
size needed before recognition), `taskStreamTwinThreshold` (how similar args
|
|
263
287
|
must be to count as *the same task* — lower it to treat near-duplicate
|
|
264
288
|
entries as loops again).
|
|
289
|
+
|
|
290
|
+
### Degenerate repetition: ONE message stuck on a word ≠ a reasoning loop
|
|
291
|
+
|
|
292
|
+
All strategies above need at least two similar messages — they detect a model
|
|
293
|
+
*repeating itself across turns*. There is a different, equally destructive
|
|
294
|
+
failure mode they cannot see: the model's decoder **anchors on a token and
|
|
295
|
+
stops producing new output**, repeating the same word hundreds of times
|
|
296
|
+
*inside a single message or tool call*. Real case (session `2026-09-09T15-43`,
|
|
297
|
+
/home/j — Qwen3.8-27B on llama.cpp): one 46 KB `bash` call whose SSH
|
|
298
|
+
username wordlist repeated `noguerol` **5145 times** (a run of 5140 — 99% of
|
|
299
|
+
the payload). Every cross-message detector stayed silent (nothing to compare
|
|
300
|
+
against — it happened exactly once), and only a manual ESC stopped it.
|
|
301
|
+
|
|
302
|
+
Antiloop v1.6 detects this **degenerate repetition** directly, on the first
|
|
303
|
+
occurrence, with no peer message:
|
|
304
|
+
|
|
305
|
+
- **Run-length** — a payload of ≥ `degenerateMinTokens` (50) normalized words
|
|
306
|
+
containing ≥ `degenerateMaxRun` (16) *consecutive identical* words is a
|
|
307
|
+
meltdown.
|
|
308
|
+
- **Frequency share** — one word appearing ≥ `degenerateMaxFreq` (60) times
|
|
309
|
+
with ≥ `degenerateMaxShare` (40%) of all tokens catches interleaved
|
|
310
|
+
meltdowns (`A B A B A B…`) that have no long run.
|
|
311
|
+
- **Perfect power** — a single giant token with no separators at all
|
|
312
|
+
(`noguerolnoguerol…`) is checked for periodicity.
|
|
313
|
+
|
|
314
|
+
Tokenization uses runs of letters (unicode), so JSON stringification noise
|
|
315
|
+
(escaped `\n` in stored tool args, punctuation, digits, code symbols) never
|
|
316
|
+
fragments the repeated word — and legit payloads full of numbers or code
|
|
317
|
+
symbols don't false-positive. 1-letter tokens are ignored as candidates
|
|
318
|
+
(`{"a":1}` JSON keys can't trigger).
|
|
319
|
+
|
|
320
|
+
Because the signal is conclusive, it acts **before the tools run**: the
|
|
321
|
+
meltdown is caught at `message_end` (pi emits it once the assistant message is
|
|
322
|
+
complete, before tool preflight), escalates immediately (one degenerate turn =
|
|
323
|
+
`degenerateTurnWeight` = 2 consecutive points → warning on first sight, force
|
|
324
|
+
steer on the second consecutive meltdown), and any degenerate `bash` command
|
|
325
|
+
is **blocked in the `tool_call` hook** (`blockDegenerateBash`) — the 46 KB
|
|
326
|
+
brute-force style command never executes. The block reason is returned to the
|
|
327
|
+
model as the tool error, so the next LLM call can still change approach; only
|
|
328
|
+
if it keeps melting does antiloop steer and then hard-stop the run.
|
|
329
|
+
|
|
330
|
+
Legit commands are safe: real scripts never repeat one word 16+ times in a
|
|
331
|
+
row inside a ≥ 50-token payload (verified against the actual sequential bash
|
|
332
|
+
sweeps that motivated the tool-loop threshold). Only `bash` gets the blocking
|
|
333
|
+
gate — writing a repetitive *file* (e.g. a user-requested padding fixture) is
|
|
334
|
+
alerted and escalated, not refused.
|
|
335
|
+
|
|
336
|
+
### No-progress outcome runs: mutated re-runs, same wall
|
|
337
|
+
|
|
338
|
+
A subtler meltdown than the degenerate one: the model re-runs the SAME
|
|
339
|
+
experiment over and over, mutating a cosmetic label or permutation each time
|
|
340
|
+
so no call ever repeats verbatim — while the outcome stays the SAME FAILURE.
|
|
341
|
+
Real case (same session, rows 95–249): ~90 ssh `exportfs`/`mount` tests,
|
|
342
|
+
labels `test A` … `test QQQ`, targets alternating Javi/Compartido — every one
|
|
343
|
+
ending `access denied` / `rc=32`, with fresh journalctl noise per attempt.
|
|
344
|
+
The tool-loop detector is blind to it BY DESIGN (args mutate every turn:
|
|
345
|
+
mean adjacent trigram similarity 0.93, but the label always changes, so the
|
|
346
|
+
same call never recurs `minToolRepeatCount` times) and results only *veto*
|
|
347
|
+
tool loops today — nothing used "same outcome repeated" as a positive signal.
|
|
348
|
+
|
|
349
|
+
Antiloop v1.6.1's **outcome detector** closes exactly that gap, conservatively:
|
|
350
|
+
|
|
351
|
+
- the LAST turn's single tool call must have a captured result that is a
|
|
352
|
+
FAILURE (fingerprints carry an error signature — `ok|fail|sig|…` — extracted
|
|
353
|
+
around the first failure marker over the FULL output, because a tool run can
|
|
354
|
+
fail with `isError=false`: the ssh pipeline exits 0 while `rc=32` lives
|
|
355
|
+
inside the text);
|
|
356
|
+
- at least `outcomeMinRepeats` (default 8) PRIOR single-call turns (all after
|
|
357
|
+
the last real user message) must share BOTH args ≥ `outcomeArgSimilarity`
|
|
358
|
+
(0.85 — the same experiment reshuffled) AND the same failure signature
|
|
359
|
+
(digit-stripped signatures compared at `outcomeSigThreshold`, 0.7 — mount
|
|
360
|
+
targets that legitimately vary between attempts survive, journalctl noise
|
|
361
|
+
doesn't).
|
|
362
|
+
|
|
363
|
+
Guarantees preserved: converging sweeps change outcome → silent; different
|
|
364
|
+
failures = evolving diagnosis → silent; identical OKs (task-stream batches,
|
|
365
|
+
file writes, idempotent verifications) never count as failures → silent;
|
|
366
|
+
user-steered turns don't count → only the autonomous stretch is judged.
|
|
367
|
+
Because the signal is proven no-progress, one such turn adds
|
|
368
|
+
`degenerateTurnWeight` (2) points → warning on the crossing turn, force-break
|
|
369
|
+
steer on the next, hard stop shortly after if the same wall persists. On the
|
|
370
|
+
real session this fires at `test MM` (warn) → `NN` (steer) → `PP` (abort) —
|
|
371
|
+
~50 wasted experiment turns cut.
|
|
372
|
+
|
|
373
|
+
Tunables: `detectOutcomeLoops`, `outcomeMinRepeats` (lower = earlier cutoff),
|
|
374
|
+
`outcomeArgSimilarity`, `outcomeSigThreshold`.
|
|
265
375
|
```
|
|
266
376
|
|
|
267
377
|
### Sliding window
|
|
@@ -282,6 +392,17 @@ Persisted as JSON at `~/.pi/agent/antiloop.json`:
|
|
|
282
392
|
"toolSimilarityThreshold": 0.95,
|
|
283
393
|
"minToolRepeatCount": 2,
|
|
284
394
|
"resultSimilarityThreshold": 0.8,
|
|
395
|
+
"detectDegenerate": true,
|
|
396
|
+
"degenerateMinTokens": 50,
|
|
397
|
+
"degenerateMaxRun": 16,
|
|
398
|
+
"degenerateMaxFreq": 60,
|
|
399
|
+
"degenerateMaxShare": 0.4,
|
|
400
|
+
"degenerateTurnWeight": 2,
|
|
401
|
+
"blockDegenerateBash": true,
|
|
402
|
+
"detectOutcomeLoops": true,
|
|
403
|
+
"outcomeMinRepeats": 8,
|
|
404
|
+
"outcomeArgSimilarity": 0.85,
|
|
405
|
+
"outcomeSigThreshold": 0.7,
|
|
285
406
|
"detectTaskStreams": true,
|
|
286
407
|
"taskStreamMinCalls": 3,
|
|
287
408
|
"taskStreamTwinThreshold": 0.99,
|
|
@@ -306,6 +427,17 @@ Persisted as JSON at `~/.pi/agent/antiloop.json`:
|
|
|
306
427
|
| `toolSimilarityThreshold` | `0.95` | How close tool-call arguments must be (0.0–1.0) to count as the *same* call — see [tool loops](#how-it-works) |
|
|
307
428
|
| `minToolRepeatCount` | `2` | Prior occurrences of a near-identical call set required before a tool loop is flagged (2 = same call seen 3×) |
|
|
308
429
|
| `resultSimilarityThreshold` | `0.8` | Minimum similarity between captured result tails to still count as the *same outcome*; below this, a repeated command is treated as progress, not a loop |
|
|
430
|
+
| `detectDegenerate` | `true` | Detect intra-message degenerate repetition — one message/call stuck repeating a single word (the `noguerol ×5145` class). Needs no repeated peer message; scanned at `message_end` and on every bash `tool_call` |
|
|
431
|
+
| `degenerateMinTokens` | `50` | Minimum normalized tokens in a payload before it is scanned for degenerate repetition (shorter payloads aren't conclusive) |
|
|
432
|
+
| `degenerateMaxRun` | `16` | Consecutive identical words inside one payload that flag it as degenerate |
|
|
433
|
+
| `degenerateMaxFreq` | `60` | One word's total occurrences (with `degenerateMaxShare` of the payload) that flags interleaved meltdowns |
|
|
434
|
+
| `degenerateMaxShare` | `0.4` | Frequency share (freq/total tokens) required together with `degenerateMaxFreq` |
|
|
435
|
+
| `degenerateTurnWeight` | `2` | Consecutive-detection points added by one degenerate turn (2 = warning on first sight) |
|
|
436
|
+
| `blockDegenerateBash` | `true` | Block a degenerate `bash` command in the `tool_call` hook before it executes; the reason is fed back to the model as the tool error |
|
|
437
|
+
| `detectOutcomeLoops` | `true` | No-progress outcome runs: ≥ `outcomeMinRepeats` near-identical attempts (args ≥ `outcomeArgSimilarity`) all ending in the *same failing outcome* — the NFS `test A…QQQ` class |
|
|
438
|
+
| `outcomeMinRepeats` | `8` | Prior same-failure attempts (inside the window, after the last user input) required before the outcome detector fires |
|
|
439
|
+
| `outcomeArgSimilarity` | `0.85` | How similar args must be to count as the *same experiment reshuffled* (mutations of labels/permutations stay under it — distinct tasks don't) |
|
|
440
|
+
| `outcomeSigThreshold` | `0.7` | Minimum similarity between digit-stripped failure signatures to count as the *same failure* |
|
|
309
441
|
| `detectTaskStreams` | `true` | Recognize homogeneous batch work (same tool called with distinct content — e.g. punched/plan/obsidian extensions) and stay silent; see [task streams](#task-streams-n-tasks-of-one-type--a-loop) |
|
|
310
442
|
| `taskStreamMinCalls` | `3` | Same-tool calls required inside the window before a task stream is recognized |
|
|
311
443
|
| `taskStreamTwinThreshold` | `0.99` | Arguments this similar (or identical) count as *the same task* — a twin invalidates the stream and re-enables normal loop detection |
|
|
@@ -326,7 +458,9 @@ Persisted as JSON at `~/.pi/agent/antiloop.json`:
|
|
|
326
458
|
4. **Per-strategy toggles** — if the model's reasoning legitimately repeats (e.g. it's working through a checklist), disable `thinking` detection and leave text/tool on.
|
|
327
459
|
5. **Watch the log** — `/antiloop log` shows what's actually triggering. If you see false positives, raise `similarityThreshold` instead of disabling the strategy entirely.
|
|
328
460
|
6. **Let user input clear state** — each user message decays the consecutive counter by 2, so a fresh prompt naturally resets without `/antiloop reset`.
|
|
329
|
-
7.
|
|
461
|
+
7. **Degenerate detector needs no tuning for most setups** — a run of ≥ 16 identical words (or one word ≥ 40% of a ≥ 50-token payload) inside a single message is conclusive stuck generation; the 46 KB `noguerol ×5145` SSH-wordlist meltdown is caught on first sight (warning), its bash never executes (`blockDegenerateBash`), and a second consecutive meltdown gets the force-break steer. If a model legitimately writes repetitive payloads, raise `degenerateMaxRun` / `degenerateMaxFreq` via `/antiloop config` — don't disable the detector.
|
|
462
|
+
8. **Outcome detector catches mutated re-run loops** — a model that re-issues the same experiment with cosmetic changes (labels, permutations) while every attempt fails identically gets a warning after `outcomeMinRepeats` (8) same-failure attempts, a steer on the next, and a hard stop shortly after. Converging sweeps, evolving failures and repeated successes stay silent by design. Lower `outcomeMinRepeats` if you want earlier cutoffs.
|
|
463
|
+
9. **`/antiloop test`** — runs the real detection engine (text + tool-call + task-stream + degenerate + outcome regression cases) to verify calibration after any change.
|
|
330
464
|
|
|
331
465
|
## Architecture
|
|
332
466
|
|
|
@@ -351,10 +485,10 @@ Modular extension with zero external dependencies (only pi's bundled `@earendil-
|
|
|
351
485
|
|
|
352
486
|
- **Levenshtein + trigram Jaccard** hybrid — small texts use edit distance, large texts use n-gram overlap (each is O(N) in text length)
|
|
353
487
|
- **Sliding window** — only the last `detectionWindow` messages participate, capping memory at O(W × message_size)
|
|
354
|
-
- **Early bail** — short messages and empty tool calls skip similarity computation entirely
|
|
488
|
+
- **Early bail** — short messages and empty tool calls skip similarity computation entirely; the degenerate scan is a single linear tokenization pass
|
|
355
489
|
- **TUI integration** — uses `ctx.ui.select` for the config menu and the log viewer; `ctx.ui.notify` for state notifications; `ctx.ui.setStatus` + a custom `ctx.ui.setFooter` component for the persistent footer indicator, live level info, and the `esc+a` keyboard toggle (`ctx.ui.onTerminalInput`, never consumes input)
|
|
356
|
-
- **Hooks** — `message_end` (track messages + tool call ids), `turn_end` (attach result fingerprints, detect
|
|
357
|
-
- **Intervention runs on the turn loop, not on user prompts** — escalation is decided at `turn_end
|
|
490
|
+
- **Hooks** — `message_end` (track messages + tool call ids with a monotonic sequence so the sliding-window trim can never collide turn indices, and pre-handle degenerate meltdowns — the message is complete but its tools haven't executed yet), `tool_call` (block degenerate `bash` commands before they run), `turn_end` (attach result fingerprints with failure signatures, detect — including no-progress outcome runs — and intervene: steer the force break / abort the run), `input` (decay on real user messages only), `session_start` (load config + install footer + reset), `session_shutdown` (restore built-in footer)
|
|
491
|
+
- **Intervention runs on the turn loop, not on user prompts** — escalation is decided at `turn_end` (and at `message_end` for the self-contained degenerate signal), the break is steered into the running agent before its next LLM call, and the guaranteed hard stop aborts the run (`ctx.abort`, fire-and-forget — never awaited, so the hook can't deadlock). No custom-role messages are injected into the conversation at any level (steering a real user message + aborting are the only levers; custom-role injections were removed because a model can stall on an unexpected injected message)
|
|
358
492
|
|
|
359
493
|
## License
|
|
360
494
|
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "pi-antiloop",
|
|
3
|
-
"version": "1.
|
|
4
|
-
"description": "Antiloop: detect reasoning loops and force a break (warn
|
|
3
|
+
"version": "1.6.1",
|
|
4
|
+
"description": "Antiloop: detect reasoning loops and force a break (warn → force → abort) across text, tool, thinking, structural, degenerate-repetition, and no-progress outcome patterns. Degenerate (one message stuck repeating a word hundreds of times — noguerol ×5145) is caught at message_end and its bash blocked before executing. No-progress (the NFS test-A…QQQ class: ~90 mutated re-runs of the same experiment, every one failing identically) fires when the same failing outcome repeats ≥ outcomeMinRepeats times with near-identical args. Fixes antiloop going blind mid-session: tracked messages use a monotonic sequence so the trim of the sliding window can never collide turn indices. The force break is delivered mid-run (steer before the next LLM call) and if the model ignores it antiloop aborts the run — an autonomous tool loop always terminates. Result-aware tool-loop detection keeps sequential bash and converging sweeps quiet; task-stream recognition keeps punched/plan batch work quiet.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"pi-package",
|
|
7
7
|
"antiloop",
|
package/src/commands.ts
CHANGED
|
@@ -58,9 +58,11 @@ async function showStatus(ctx: ExtensionCommandContext, rt: Runtime): Promise<vo
|
|
|
58
58
|
` similarity: ${(rt.config.similarityThreshold * 100).toFixed(0)}% window: ${rt.config.detectionWindow}`,
|
|
59
59
|
` tool sim: ${(rt.config.toolSimilarityThreshold * 100).toFixed(0)}% tool repeat: ${rt.config.minToolRepeatCount}+ prior`,
|
|
60
60
|
` result sim: ${(rt.config.resultSimilarityThreshold * 100).toFixed(0)}% (same cmd + diff outcome = no loop)`,
|
|
61
|
+
` degenerate: run ≥ ${rt.config.degenerateMaxRun} same word · freq ≥ ${rt.config.degenerateMaxFreq} @ ${(rt.config.degenerateMaxShare * 100).toFixed(0)}% (≥ ${rt.config.degenerateMinTokens} tokens) · weight ${rt.config.degenerateTurnWeight} · block bash ${yn(rt.config.blockDegenerateBash)}`,
|
|
62
|
+
` outcome: same failing result ≥ ${rt.config.outcomeMinRepeats} attempts (args ≥ ${(rt.config.outcomeArgSimilarity * 100).toFixed(0)}% sim, sig ≥ ${(rt.config.outcomeSigThreshold * 100).toFixed(0)}%)`,
|
|
61
63
|
` task streams: ${yn(rt.config.detectTaskStreams)} (min ${rt.config.taskStreamMinCalls} calls, twins ≥ ${(rt.config.taskStreamTwinThreshold * 100).toFixed(0)}%)`,
|
|
62
64
|
"",
|
|
63
|
-
`detectors: text ${yn(rt.config.detectTextLoops)} · tool ${yn(rt.config.detectToolLoops)} · think ${yn(rt.config.detectThinkingLoops)}`,
|
|
65
|
+
`detectors: text ${yn(rt.config.detectTextLoops)} · tool ${yn(rt.config.detectToolLoops)} · think ${yn(rt.config.detectThinkingLoops)} · degenerate ${yn(rt.config.detectDegenerate)} · outcome ${yn(rt.config.detectOutcomeLoops)}`,
|
|
64
66
|
`footer: interactive ${yn(rt.config.interactiveFooter)} · toggle: ${rt.config.toggleShortcut}`,
|
|
65
67
|
];
|
|
66
68
|
if (rt.state.activeTaskStreams.length) {
|
|
@@ -91,6 +93,11 @@ async function showConfigMenu(ctx: ExtensionCommandContext, rt: Runtime): Promis
|
|
|
91
93
|
{ value: "toolSim" as const, label: `🔧 call similarity: ${(c.toolSimilarityThreshold * 100).toFixed(0)}%`, description: "how identical tool calls must be to count as the same call" },
|
|
92
94
|
{ value: "toolRepeat" as const, label: `🔁 call repeats: ${c.minToolRepeatCount}+`, description: "how many times the same call must repeat before it flags" },
|
|
93
95
|
{ value: "resultSim" as const, label: `🧾 result similarity: ${(c.resultSimilarityThreshold * 100).toFixed(0)}%`, description: "same command + different result = progress, not a loop" },
|
|
96
|
+
// ── 🌀 Degenerate (intra-message meltdown) ───────────────────────
|
|
97
|
+
{ value: "degRun" as const, label: `🌀 degenerate run: ≥ ${c.degenerateMaxRun}`, description: "identical words in a row inside ONE message/call before it counts as stuck generation (noguerol ×5145 class)" },
|
|
98
|
+
{ value: "degBlock" as const, label: `⛔ block degenerate bash: ${yn(c.blockDegenerateBash)}`, description: "stop a degenerate command before it executes (default on)" },
|
|
99
|
+
// ── 📉 Outcome (no-progress) ────────────────────────────────
|
|
100
|
+
{ value: "outcomeMin" as const, label: `📉 no-progress after: ${c.outcomeMinRepeats}`, description: "same failing outcome repeated this many times (mutated args ≥ 85% similar) before flagging — the NFS test-A…QQQ class" },
|
|
94
101
|
// ── 📋 Task streams ─────────────────────────────────────────
|
|
95
102
|
{ value: "streams" as const, label: `📋 task streams: ${yn(c.detectTaskStreams)}`, description: "batch work (punched_log / plan_manager / …) is not a loop" },
|
|
96
103
|
{ value: "streamMin" as const, label: `📋 stream min calls: ${c.taskStreamMinCalls}`, description: "calls of the same tool before a batch is recognized" },
|
|
@@ -99,6 +106,8 @@ async function showConfigMenu(ctx: ExtensionCommandContext, rt: Runtime): Promis
|
|
|
99
106
|
{ value: "text" as const, label: `📝 text: ${yn(c.detectTextLoops)}`, description: "detect repeated text messages" },
|
|
100
107
|
{ value: "tool" as const, label: `🔧 tools: ${yn(c.detectToolLoops)}`, description: "detect repeated tool calls" },
|
|
101
108
|
{ value: "think" as const, label: `🧠 thinking: ${yn(c.detectThinkingLoops)}`, description: "detect repeated internal reasoning" },
|
|
109
|
+
{ value: "deg" as const, label: `🌀 degenerate: ${yn(c.detectDegenerate)}`, description: "detect ONE message stuck repeating a single word/token (no repeated peer needed)" },
|
|
110
|
+
{ value: "outcome" as const, label: `📉 outcome: ${yn(c.detectOutcomeLoops)}`, description: "detect many near-identical attempts all ending in the SAME failing outcome (no progress)" },
|
|
102
111
|
// ── 🧹 ──────────────────────────────────────────────────────
|
|
103
112
|
{ value: "reset" as const, label: "🧹 reset state", description: "clear counters and history" },
|
|
104
113
|
]);
|
|
@@ -208,6 +217,29 @@ async function showConfigMenu(ctx: ExtensionCommandContext, rt: Runtime): Promis
|
|
|
208
217
|
if (v !== undefined) { c.resultSimilarityThreshold = v; saveConfig(c); ctx.ui.notify(`result similarity: ${(v * 100).toFixed(0)}%`, "info"); }
|
|
209
218
|
break;
|
|
210
219
|
}
|
|
220
|
+
case "degRun": {
|
|
221
|
+
const v = await selectFrom(ctx, "🌀 degenerate run (identical words in a row inside one payload)", [
|
|
222
|
+
{ value: 8, label: "⚡ 8 (sensitive)" },
|
|
223
|
+
{ value: 16, label: "🎯 16 (default)" },
|
|
224
|
+
{ value: 24, label: "24" },
|
|
225
|
+
{ value: 48, label: "🐢 48 (relaxed)" },
|
|
226
|
+
]);
|
|
227
|
+
if (v !== undefined) { c.degenerateMaxRun = v; saveConfig(c); ctx.ui.notify(`degenerate run: ≥ ${v}`, "info"); }
|
|
228
|
+
break;
|
|
229
|
+
}
|
|
230
|
+
case "degBlock":
|
|
231
|
+
c.blockDegenerateBash = !c.blockDegenerateBash; saveConfig(c);
|
|
232
|
+
ctx.ui.notify(`block degenerate bash: ${yn(c.blockDegenerateBash)}`, "info"); break;
|
|
233
|
+
case "outcomeMin": {
|
|
234
|
+
const v = await selectFrom(ctx, "📉 no-progress threshold (same failing outcome, near-identical args)", [
|
|
235
|
+
{ value: 4, label: "⚡ 4 (sensitive — long experiment series get cut early)" },
|
|
236
|
+
{ value: 6, label: "6" },
|
|
237
|
+
{ value: 8, label: "🎯 8 (default)" },
|
|
238
|
+
{ value: 12, label: "🐢 12 (relaxed)" },
|
|
239
|
+
]);
|
|
240
|
+
if (v !== undefined) { c.outcomeMinRepeats = v; saveConfig(c); ctx.ui.notify(`no-progress after: ${v}`, "info"); }
|
|
241
|
+
break;
|
|
242
|
+
}
|
|
211
243
|
case "streams":
|
|
212
244
|
c.detectTaskStreams = !c.detectTaskStreams; saveConfig(c);
|
|
213
245
|
ctx.ui.notify(`task streams: ${yn(c.detectTaskStreams)}`, "info"); break;
|
|
@@ -248,6 +280,12 @@ async function showConfigMenu(ctx: ExtensionCommandContext, rt: Runtime): Promis
|
|
|
248
280
|
case "think":
|
|
249
281
|
c.detectThinkingLoops = !c.detectThinkingLoops; saveConfig(c);
|
|
250
282
|
ctx.ui.notify(`thinking: ${yn(c.detectThinkingLoops)}`, "info"); break;
|
|
283
|
+
case "deg":
|
|
284
|
+
c.detectDegenerate = !c.detectDegenerate; saveConfig(c);
|
|
285
|
+
ctx.ui.notify(`degenerate: ${yn(c.detectDegenerate)}`, "info"); break;
|
|
286
|
+
case "outcome":
|
|
287
|
+
c.detectOutcomeLoops = !c.detectOutcomeLoops; saveConfig(c);
|
|
288
|
+
ctx.ui.notify(`outcome: ${yn(c.detectOutcomeLoops)}`, "info"); break;
|
|
251
289
|
case "reset":
|
|
252
290
|
resetState(rt.state);
|
|
253
291
|
ctx.ui.notify("🧹 state reset", "info");
|
|
@@ -280,6 +318,7 @@ export function resetState(state: AntiloopState): void {
|
|
|
280
318
|
state.lastDetectedTurnIndex = -1;
|
|
281
319
|
state.steerDelivered = false;
|
|
282
320
|
state.ignoredSteerCount = 0;
|
|
321
|
+
state.turnSeq = 0;
|
|
283
322
|
}
|
|
284
323
|
|
|
285
324
|
async function runSelfTest(ctx: ExtensionCommandContext): Promise<void> {
|
package/src/config.ts
CHANGED
|
@@ -17,6 +17,17 @@ export const DEFAULT_CONFIG: AntiloopConfig = {
|
|
|
17
17
|
toolSimilarityThreshold: 0.95,
|
|
18
18
|
minToolRepeatCount: 2,
|
|
19
19
|
resultSimilarityThreshold: 0.8,
|
|
20
|
+
detectDegenerate: true,
|
|
21
|
+
degenerateMinTokens: 50,
|
|
22
|
+
degenerateMaxRun: 16,
|
|
23
|
+
degenerateMaxFreq: 60,
|
|
24
|
+
degenerateMaxShare: 0.4,
|
|
25
|
+
degenerateTurnWeight: 2,
|
|
26
|
+
blockDegenerateBash: true,
|
|
27
|
+
detectOutcomeLoops: true,
|
|
28
|
+
outcomeMinRepeats: 8,
|
|
29
|
+
outcomeArgSimilarity: 0.85,
|
|
30
|
+
outcomeSigThreshold: 0.7,
|
|
20
31
|
detectTaskStreams: true,
|
|
21
32
|
taskStreamMinCalls: 3,
|
|
22
33
|
taskStreamTwinThreshold: 0.99,
|
package/src/detect.ts
CHANGED
|
@@ -34,6 +34,166 @@ function opening(text: string, n = 10): string {
|
|
|
34
34
|
return normalizeText(text.split(/\s+/).slice(0, n).join(" "));
|
|
35
35
|
}
|
|
36
36
|
|
|
37
|
+
// ---------------------------------------------------------------------------
|
|
38
|
+
// Intra-message degenerate repetition (v1.6).
|
|
39
|
+
//
|
|
40
|
+
// The "noguerol ×5145" class (verified against a real session — /home/j
|
|
41
|
+
// 2026-09-09T15-43: ONE 46 KB bash call whose SSH username list repeats a
|
|
42
|
+
// single word 5145 times, a run of 5140 — 99% of the payload). A model whose
|
|
43
|
+
// decoder anchors on a token stops producing NEW output: it repeats the same
|
|
44
|
+
// word hundreds of times INSIDE one message or tool call. The cross-message
|
|
45
|
+
// detectors (text / tool / thinking / structural) all need >= 2 similar
|
|
46
|
+
// messages and cannot see this — the meltdown happened exactly once, inside a
|
|
47
|
+
// single call, and antiloop stayed silent until the user ESC'd.
|
|
48
|
+
//
|
|
49
|
+
// This detector is self-contained: it flags the FIRST such payload, no peer
|
|
50
|
+
// message required, and it is cheap enough to run at message_end (before the
|
|
51
|
+
// tool calls execute) and on every bash tool_call (blocking gate).
|
|
52
|
+
// ---------------------------------------------------------------------------
|
|
53
|
+
|
|
54
|
+
/** Longest run / top frequency of ONE repeated word inside a payload. */
|
|
55
|
+
export interface DegenerateInfo {
|
|
56
|
+
token: string;
|
|
57
|
+
freq: number;
|
|
58
|
+
maxRun: number;
|
|
59
|
+
total: number;
|
|
60
|
+
}
|
|
61
|
+
|
|
62
|
+
export interface DegenerateHit extends DegenerateInfo {
|
|
63
|
+
where: string;
|
|
64
|
+
}
|
|
65
|
+
|
|
66
|
+
/** Ignore 1-letter tokens as candidates (JSON keys like {"a":1} must not flag). */
|
|
67
|
+
const MIN_TOKEN_LEN = 2;
|
|
68
|
+
|
|
69
|
+
/**
|
|
70
|
+
* Detect pathological single-word repetition in a payload (assistant text or a
|
|
71
|
+
* JSON.stringify'd tool-call argument). Returns the repeated word with its
|
|
72
|
+
* count, longest consecutive run and payload size, or undefined when the
|
|
73
|
+
* payload is normal.
|
|
74
|
+
*
|
|
75
|
+
* Tokenization: runs of unicode letters, lowercased. Stored tool args are
|
|
76
|
+
* JSON-escaped (real newlines arrived as the two characters `\n`), so escaped
|
|
77
|
+
* whitespace is normalized back to a separator first — a word list written
|
|
78
|
+
* across lines must still tokenize word by word. Digits/punctuation/code
|
|
79
|
+
* symbols split tokens instead of polluting them.
|
|
80
|
+
*
|
|
81
|
+
* Signals (both are conclusive for generation quality):
|
|
82
|
+
* - a run of >= degenerateMaxRun consecutive identical words, or
|
|
83
|
+
* - one word occurring >= degenerateMaxFreq times with >= degenerateMaxShare
|
|
84
|
+
* of all tokens (catches interleaved "A B A B" meltdowns with no run).
|
|
85
|
+
* A payload must have >= degenerateMinTokens tokens to be scanned.
|
|
86
|
+
*/
|
|
87
|
+
export function findDegenerateRepetition(
|
|
88
|
+
text: string,
|
|
89
|
+
config: AntiloopConfig,
|
|
90
|
+
): DegenerateInfo | undefined {
|
|
91
|
+
let s = String(text).replace(/\\+[nrt]/g, " ").toLowerCase();
|
|
92
|
+
const raw = s.match(/[\p{L}]+/gu);
|
|
93
|
+
if (!raw) return undefined;
|
|
94
|
+
const total = raw.length;
|
|
95
|
+
|
|
96
|
+
let maxRun = 0;
|
|
97
|
+
let runToken = "";
|
|
98
|
+
let prev = "";
|
|
99
|
+
let run = 0;
|
|
100
|
+
const freq = new Map<string, number>();
|
|
101
|
+
for (const t of raw) {
|
|
102
|
+
if (t.length < MIN_TOKEN_LEN) {
|
|
103
|
+
run = 0;
|
|
104
|
+
prev = "";
|
|
105
|
+
continue;
|
|
106
|
+
}
|
|
107
|
+
freq.set(t, (freq.get(t) ?? 0) + 1);
|
|
108
|
+
if (t === prev) run++;
|
|
109
|
+
else {
|
|
110
|
+
run = 1;
|
|
111
|
+
prev = t;
|
|
112
|
+
}
|
|
113
|
+
if (run > maxRun) {
|
|
114
|
+
maxRun = run;
|
|
115
|
+
runToken = t;
|
|
116
|
+
}
|
|
117
|
+
}
|
|
118
|
+
let topToken = "";
|
|
119
|
+
let topFreq = 0;
|
|
120
|
+
for (const [t, c] of freq) {
|
|
121
|
+
if (c > topFreq) {
|
|
122
|
+
topFreq = c;
|
|
123
|
+
topToken = t;
|
|
124
|
+
}
|
|
125
|
+
}
|
|
126
|
+
if (total >= config.degenerateMinTokens) {
|
|
127
|
+
if (maxRun >= config.degenerateMaxRun) {
|
|
128
|
+
return { token: runToken, freq: freq.get(runToken) ?? topFreq, maxRun, total };
|
|
129
|
+
}
|
|
130
|
+
if (topFreq >= config.degenerateMaxFreq && topFreq / total >= config.degenerateMaxShare) {
|
|
131
|
+
return { token: topToken, freq: topFreq, maxRun, total };
|
|
132
|
+
}
|
|
133
|
+
}
|
|
134
|
+
|
|
135
|
+
// No-space meltdown: one giant periodic token ("noguerolnoguerol…" with all
|
|
136
|
+
// separators stripped) is a perfect power of a short motif. Independent of
|
|
137
|
+
// the token-count gate: a single 1200+ char token has no word runs at all.
|
|
138
|
+
if (raw.length <= 2) {
|
|
139
|
+
const single = raw.join("");
|
|
140
|
+
const len = single.length;
|
|
141
|
+
if (len >= 1200) {
|
|
142
|
+
for (let p = 3; p <= 200 && p * 12 <= len; p++) {
|
|
143
|
+
if (len % p) continue;
|
|
144
|
+
const motif = single.slice(0, p);
|
|
145
|
+
let ok = true;
|
|
146
|
+
for (let i = p; i < len; i += p) {
|
|
147
|
+
if (!single.startsWith(motif, i)) {
|
|
148
|
+
ok = false;
|
|
149
|
+
break;
|
|
150
|
+
}
|
|
151
|
+
}
|
|
152
|
+
if (ok) {
|
|
153
|
+
const repeats = len / p;
|
|
154
|
+
if (repeats >= 12) {
|
|
155
|
+
return { token: motif.slice(0, 40), freq: repeats, maxRun: repeats, total: repeats };
|
|
156
|
+
}
|
|
157
|
+
}
|
|
158
|
+
}
|
|
159
|
+
}
|
|
160
|
+
}
|
|
161
|
+
return undefined;
|
|
162
|
+
}
|
|
163
|
+
|
|
164
|
+
/** Scan one assistant message (text + each tool-call argument) for a meltdown. */
|
|
165
|
+
export function scanMessageDegenerate(
|
|
166
|
+
content: string,
|
|
167
|
+
toolCalls: TrackedToolCall[] | undefined,
|
|
168
|
+
config: AntiloopConfig,
|
|
169
|
+
): DegenerateHit | undefined {
|
|
170
|
+
if (!config.detectDegenerate) return undefined;
|
|
171
|
+
if (content && content.length) {
|
|
172
|
+
const d = findDegenerateRepetition(content, config);
|
|
173
|
+
if (d) return { ...d, where: "message text" };
|
|
174
|
+
}
|
|
175
|
+
for (const tc of toolCalls ?? []) {
|
|
176
|
+
if (!tc.args) continue;
|
|
177
|
+
const d = findDegenerateRepetition(tc.args, config);
|
|
178
|
+
if (d) return { ...d, where: tc.name === "bash" ? "bash command" : `args(${tc.name})` };
|
|
179
|
+
}
|
|
180
|
+
return undefined;
|
|
181
|
+
}
|
|
182
|
+
|
|
183
|
+
export function degenerateDescription(hit: DegenerateHit): string {
|
|
184
|
+
const share = hit.total ? Math.round((hit.freq / hit.total) * 100) : 100;
|
|
185
|
+
return `${hit.where}: degenerate repetition — "${hit.token}" ×${hit.freq} (${share}% of ${hit.total} tokens, longest run ${hit.maxRun})`;
|
|
186
|
+
}
|
|
187
|
+
|
|
188
|
+
/** Consecutive-detection weight of a detection turn. The strong
|
|
189
|
+
* self-contained signals (degenerate meltdown, proven no-progress outcome run)
|
|
190
|
+
* add degenerateTurnWeight (default 2) points so the FIRST one already reaches
|
|
191
|
+
* the warning level and escalation is fast on repeat. */
|
|
192
|
+
export function detectionTurnWeight(detections: LoopDetection[], config: AntiloopConfig): number {
|
|
193
|
+
const strong = detections.some((d) => d.type === "degenerate" || d.type === "outcome");
|
|
194
|
+
return strong ? Math.max(1, config.degenerateTurnWeight) : 1;
|
|
195
|
+
}
|
|
196
|
+
|
|
37
197
|
function similarity(a: string, b: string): number {
|
|
38
198
|
if (a.length < MIN_CONTENT_LENGTH || b.length < MIN_CONTENT_LENGTH) return 0;
|
|
39
199
|
if (a === b) return 1;
|
|
@@ -74,7 +234,18 @@ export function resultFingerprint(
|
|
|
74
234
|
}
|
|
75
235
|
const norm = normalizeText(text);
|
|
76
236
|
if (!norm.length) return undefined;
|
|
77
|
-
|
|
237
|
+
// A tool run can FAIL while isError stays false (the NFS mount errors: the
|
|
238
|
+
// ssh pipeline exits 0 after grep/echo, rc=32 lives inside the output). The
|
|
239
|
+
// tail-only fingerprint would cut the failure markers away AND the tail is
|
|
240
|
+
// noisy (journalctl timestamps differ per attempt), so for failures we keep
|
|
241
|
+
// a SHORT ERROR SIGNATURE around the first failure marker — it repeats
|
|
242
|
+
// verbatim across attempts of the same failure and makes same-outcome
|
|
243
|
+
// comparisons robust. Formats: "err|…" / "ok|fail|sig|…|tail" / "ok|…".
|
|
244
|
+
const failed = FAIL_MARKERS.test(norm);
|
|
245
|
+
if (!failed) return `${isError ? "err" : "ok"}|${norm.slice(-400)}`;
|
|
246
|
+
const at = norm.search(FAIL_MARKERS);
|
|
247
|
+
const sig = norm.slice(Math.max(0, at - 60), at + 160);
|
|
248
|
+
return `${isError ? "err" : "ok"}|fail|${sig}|${norm.slice(-400)}`;
|
|
78
249
|
}
|
|
79
250
|
|
|
80
251
|
/** Same outcome = identical fingerprint, or high similarity of the tails. */
|
|
@@ -89,6 +260,44 @@ function sameOutcome(a: string, b: string, threshold: number): boolean {
|
|
|
89
260
|
return s >= threshold && s > 0;
|
|
90
261
|
}
|
|
91
262
|
|
|
263
|
+
// Failure markers on a NORMALIZED fingerprint tail (lowercased, punctuation
|
|
264
|
+
// stripped — "rc=32" arrives as "rc32"). A conservative "this attempt failed"
|
|
265
|
+
// test: rc ≠ 0, denials, missing files, refusals, syntax crashes… Generic
|
|
266
|
+
// words like "error"/"failed" are intentionally NOT markers (legit outputs
|
|
267
|
+
// like "0 failed, 12 passed" must not count as failures).
|
|
268
|
+
const FAIL_MARKERS =
|
|
269
|
+
/\b(denied|no such file|not found|cannot|unable|refused|syntax error|timed out|timeout|exception|traceback|fatal|core dumped|rc\s*[1-9]\d*)\b/i;
|
|
270
|
+
|
|
271
|
+
/** True when a captured result fingerprint represents a FAILED attempt.
|
|
272
|
+
* Used by the no-progress outcome detector so only repeated FAILURES (not
|
|
273
|
+
* repeated identical successes, which are the norm for task-stream batches
|
|
274
|
+
* like "logged" or file writes) count as a stuck loop. */
|
|
275
|
+
export function isFailResult(fp: string | undefined): boolean {
|
|
276
|
+
if (!fp) return false;
|
|
277
|
+
if (fp.startsWith("err|") || fp.startsWith("ok|fail|")) return true;
|
|
278
|
+
return FAIL_MARKERS.test(fp);
|
|
279
|
+
}
|
|
280
|
+
|
|
281
|
+
/** The short error signature embedded in a failure fingerprint
|
|
282
|
+
* ("ok|fail|SIG|tail"), if present. */
|
|
283
|
+
function failSig(fp: string): string | undefined {
|
|
284
|
+
const m = /^(?:err|ok)\|fail\|(.*?)\|/.exec(fp);
|
|
285
|
+
return m ? m[1] : undefined;
|
|
286
|
+
}
|
|
287
|
+
|
|
288
|
+
/** Same-FAILURE comparison for the outcome detector. Prefers the embedded error
|
|
289
|
+
* signatures (they repeat across attempts of the same failure) with digits
|
|
290
|
+
* stripped — timestamps, PIDs and rc values are noise; the target words that
|
|
291
|
+
* legitimately vary between attempts (Javi vs Compartido) survive but the
|
|
292
|
+
* threshold is looser than the veto threshold on purpose. Falls back to the
|
|
293
|
+
* full fingerprints when no signature is present. */
|
|
294
|
+
function sameFailure(a: string, b: string, threshold: number): boolean {
|
|
295
|
+
const sa = failSig(a);
|
|
296
|
+
const sb = failSig(b);
|
|
297
|
+
if (sa && sb) return sameOutcome(sa.replace(/\d+/g, ""), sb.replace(/\d+/g, ""), threshold);
|
|
298
|
+
return sameOutcome(a, b, threshold);
|
|
299
|
+
}
|
|
300
|
+
|
|
92
301
|
function toolCallsSimilar(
|
|
93
302
|
c1: TrackedToolCall[],
|
|
94
303
|
c2: TrackedToolCall[],
|
|
@@ -177,11 +386,28 @@ export function detectTaskStreams(
|
|
|
177
386
|
export function detectLoops(state: AntiloopState, config: AntiloopConfig): LoopDetection[] {
|
|
178
387
|
const out: LoopDetection[] = [];
|
|
179
388
|
const msgs = state.recentMessages;
|
|
180
|
-
if (msgs.length <
|
|
389
|
+
if (msgs.length < 1) return out;
|
|
181
390
|
const start = Math.max(0, msgs.length - config.detectionWindow);
|
|
182
391
|
const win = msgs.slice(start);
|
|
183
392
|
const now = Date.now();
|
|
184
393
|
|
|
394
|
+
// Intra-message degenerate repetition fires on a SINGLE pathological message
|
|
395
|
+
// (no peer needed) and is independent of the task-stream batch gate: a
|
|
396
|
+
// meltdown is a meltdown even mid-batch.
|
|
397
|
+
if (config.detectDegenerate) {
|
|
398
|
+
const last = win[win.length - 1];
|
|
399
|
+
const hit = scanMessageDegenerate(last.content, last.toolCalls, config);
|
|
400
|
+
if (hit) {
|
|
401
|
+
out.push({
|
|
402
|
+
type: "degenerate",
|
|
403
|
+
similarity: 1,
|
|
404
|
+
messageIndices: [msgs.length - 1],
|
|
405
|
+
description: degenerateDescription(hit),
|
|
406
|
+
timestamp: now,
|
|
407
|
+
});
|
|
408
|
+
}
|
|
409
|
+
}
|
|
410
|
+
|
|
185
411
|
// Task-stream gate: if the window is a homogeneous batch (same extension
|
|
186
412
|
// tool called with DISTINCT content ≥ taskStreamMinCalls times), that tool
|
|
187
413
|
// is exempt from tool-loop detection, and text/thinking/structural
|
|
@@ -264,6 +490,68 @@ export function detectLoops(state: AntiloopState, config: AntiloopConfig): LoopD
|
|
|
264
490
|
}
|
|
265
491
|
}
|
|
266
492
|
|
|
493
|
+
// -------------------------------------------------------------------
|
|
494
|
+
// No-progress outcome runs (v1.6.1).
|
|
495
|
+
//
|
|
496
|
+
// The NFS-test session (/home/j 2026-09-09, rows 95–249): ~90 mutated
|
|
497
|
+
// re-runs of the SAME experiment (sshpass+sudo+exportfs+mount, labels
|
|
498
|
+
// "test A"…"test QQQ"), every one failing identically (rc=32 / access
|
|
499
|
+
// denied). The tool-loop detector is blind to it BY DESIGN: args mutate
|
|
500
|
+
// every turn (mean adjacent trigram similarity 0.93, but the label always
|
|
501
|
+
// changes) so the same call never recurs >= minToolRepeatCount times, and
|
|
502
|
+
// identical results only VETO tool loops — nothing uses "same outcome
|
|
503
|
+
// repeated" as a positive signal. A human sees it instantly: many attempts,
|
|
504
|
+
// same wall, zero progress.
|
|
505
|
+
//
|
|
506
|
+
// Signal: the LAST turn's single tool call has a captured result, and at
|
|
507
|
+
// least outcomeMinRepeats PRIOR single-call turns (after the last real user
|
|
508
|
+
// message — an autonomous stretch, not user-steered iteration) share BOTH
|
|
509
|
+
// args >= outcomeArgSimilarity (the same experiment reshuffled) AND the same
|
|
510
|
+
// outcome (>= resultSimilarityThreshold). Legit work is untouched: distinct
|
|
511
|
+
// operations fail with distinct output; converging sweeps change outcome;
|
|
512
|
+
// batch/stream messages are excluded; a success interspersed resets the
|
|
513
|
+
// class. similarity 0.99 => after a force break, further same-outcome turns
|
|
514
|
+
// count as "ignoring the break" (isVerbatimRepeat) and escalate to the hard
|
|
515
|
+
// stop, exactly like verbatim tool loops.
|
|
516
|
+
// -------------------------------------------------------------------
|
|
517
|
+
if (config.detectOutcomeLoops) {
|
|
518
|
+
const last = win[win.length - 1];
|
|
519
|
+
const lastCalls = last.toolCalls;
|
|
520
|
+
const afterUser = state.lastUserMessageTime;
|
|
521
|
+
if (lastCalls && lastCalls.length === 1) {
|
|
522
|
+
const lc = lastCalls[0];
|
|
523
|
+
// Failure gate: only repeated FAILURES prove no progress. Task-stream
|
|
524
|
+
// batches (punched_log appends, obsidian/file writes) legitimately
|
|
525
|
+
// produce the SAME OK outcome every call — they must never count. And
|
|
526
|
+
// batch messages must NOT be skipped here: a "bash ×N stream" with
|
|
527
|
+
// identical failures is exactly the no-progress loop to catch.
|
|
528
|
+
if (lc.result && isFailResult(lc.result)) {
|
|
529
|
+
let matches = 0;
|
|
530
|
+
for (let i = 0; i < win.length - 1; i++) {
|
|
531
|
+
const m = win[i];
|
|
532
|
+
if (m.timestamp <= afterUser) continue; // user-steered turns don't count
|
|
533
|
+
const prev = m.toolCalls;
|
|
534
|
+
if (!prev || prev.length !== 1) continue;
|
|
535
|
+
const pc = prev[0];
|
|
536
|
+
if (pc.name !== lc.name || !pc.result) continue;
|
|
537
|
+
if (!sameFailure(lc.result, pc.result, config.outcomeSigThreshold)) continue;
|
|
538
|
+
if (!argsTwin(lc.args, pc.args, config.outcomeArgSimilarity)) continue;
|
|
539
|
+
matches++;
|
|
540
|
+
}
|
|
541
|
+
if (matches >= config.outcomeMinRepeats) {
|
|
542
|
+
out.push({
|
|
543
|
+
type: "outcome",
|
|
544
|
+
similarity: 0.99,
|
|
545
|
+
messageIndices: [msgs.length - 1],
|
|
546
|
+
description:
|
|
547
|
+
`no progress: ${matches + 1} near-identical ${lc.name} attempts (args ≥ ${(config.outcomeArgSimilarity * 100).toFixed(0)}% similar) with the same failing outcome — “${lc.result.slice(0, 90)}”`,
|
|
548
|
+
timestamp: now,
|
|
549
|
+
});
|
|
550
|
+
}
|
|
551
|
+
}
|
|
552
|
+
}
|
|
553
|
+
}
|
|
554
|
+
|
|
267
555
|
if (config.detectThinkingLoops) {
|
|
268
556
|
const last = win[win.length - 1];
|
|
269
557
|
if (last.thinking && last.thinking.length > 50) {
|
|
@@ -297,11 +585,13 @@ export function nextLevel(consecutiveDetections: number, config: AntiloopConfig)
|
|
|
297
585
|
}
|
|
298
586
|
|
|
299
587
|
/** True when detections prove the model repeated a message/tool call essentially
|
|
300
|
-
* verbatim (≥98% text similarity or an identical tool-loop)
|
|
301
|
-
*
|
|
302
|
-
*
|
|
303
|
-
*
|
|
304
|
-
*
|
|
588
|
+
* verbatim (≥98% text similarity or an identical tool-loop), OR produced a
|
|
589
|
+
* degenerate meltdown, OR kept re-running the same experiment with the same
|
|
590
|
+
* failing outcome (no-progress, sim 0.99). Weaker signals (thinking echoes,
|
|
591
|
+
* structural repeated openings at 90%) do NOT count — a model that only *thinks*
|
|
592
|
+
* in circles but varies its actual output is still making an attempt and must
|
|
593
|
+
* not be hard-stopped. Used post-force-break: only verbatim repeats and proven
|
|
594
|
+
* no-progress repeats show the model ignored the break instruction. */
|
|
305
595
|
export function isVerbatimRepeat(detections: LoopDetection[]): boolean {
|
|
306
596
|
return detections.some((d) => d.type !== "thinking" && d.similarity >= 0.98);
|
|
307
597
|
}
|
|
@@ -383,11 +673,14 @@ export function runSelfTest(): string[] {
|
|
|
383
673
|
detectTextLoops: true, notifyOnDetection: true, maxHistoryEntries: 100,
|
|
384
674
|
detectionWindow: 10, interactiveFooter: true, toggleShortcut: "esc+a",
|
|
385
675
|
detectTaskStreams: true, taskStreamMinCalls: 3, taskStreamTwinThreshold: 0.99,
|
|
676
|
+
detectDegenerate: true, degenerateMinTokens: 50, degenerateMaxRun: 16,
|
|
677
|
+
degenerateMaxFreq: 60, degenerateMaxShare: 0.4, degenerateTurnWeight: 2, blockDegenerateBash: true,
|
|
678
|
+
detectOutcomeLoops: true, outcomeMinRepeats: 8, outcomeArgSimilarity: 0.85, outcomeSigThreshold: 0.7,
|
|
386
679
|
};
|
|
387
680
|
const asState = (recentMessages: TrackedMessage[]): AntiloopState =>
|
|
388
681
|
({ recentMessages, detections: [], activeTaskStreams: [], currentLevel: 0,
|
|
389
682
|
consecutiveDetections: 0, inForcedBreak: false, totalDetections: 0,
|
|
390
|
-
lastUserMessageTime: 0, lastDetectedTurnIndex: -1,
|
|
683
|
+
lastUserMessageTime: 0, lastDetectedTurnIndex: -1, turnSeq: 0,
|
|
391
684
|
steerDelivered: false, ignoredSteerCount: 0 });
|
|
392
685
|
const NARR = "Now I will append the next decision entry to the project memory document so we keep the context.";
|
|
393
686
|
|
|
@@ -443,5 +736,141 @@ export function runSelfTest(): string[] {
|
|
|
443
736
|
out.push(`thinking-only 1.00 → ${isVerbatimRepeat(dl("thinking", 1)) ? "yes" : "no"} (exp no — output varies) ${!isVerbatimRepeat(dl("thinking", 1)) ? "✅" : "❌"}`);
|
|
444
737
|
out.push(`structural 0.90 → ${isVerbatimRepeat(dl("structural", 0.9)) ? "yes" : "no"} (exp no) ${!isVerbatimRepeat(dl("structural", 0.9)) ? "✅" : "❌"}`);
|
|
445
738
|
|
|
739
|
+
// --- v1.6: intra-message degenerate repetition (single-message meltdown) ---
|
|
740
|
+
// Regression: the real session /home/j 2026-09-09T15-43 — ONE 46 KB bash call
|
|
741
|
+
// whose username list repeats "noguerol" 5145 times (run of 5140). Every
|
|
742
|
+
// cross-message detector needs a peer message and stayed silent; the
|
|
743
|
+
// degenerate scan must fire on the FIRST such message, alone in the window.
|
|
744
|
+
const argJson = (cmd: string) => JSON.stringify({ command: cmd }); // stored args form
|
|
745
|
+
const meltdownCmd =
|
|
746
|
+
`echo "=== brute usernames with petete pw ==="; for u in noguerol noguerol@ j javi javi@ root petete ${`noguerol `.repeat(400)}; do :; done`;
|
|
747
|
+
const meltMsg = mk("", [{ name: "bash", args: argJson(meltdownCmd) }]);
|
|
748
|
+
const mDet = detectLoops(asState([meltMsg]), tcfg);
|
|
749
|
+
const mHit = mDet.find((d) => d.type === "degenerate");
|
|
750
|
+
out.push(`degenerate first sight → ${mHit ? `degenerate (${mHit.description})` : "no"} (exp degenerate — was the miss) ${mHit ? "✅" : "❌"}`);
|
|
751
|
+
|
|
752
|
+
// Legit payloads must NOT flag: short commands are under minTokens, real
|
|
753
|
+
// scripts never repeat one word 16× in a row.
|
|
754
|
+
const leg1 = findDegenerateRepetition(sweepRun1, tcfg);
|
|
755
|
+
const leg2 = findDegenerateRepetition("for i in 1 2 3; do echo step $i; done", tcfg);
|
|
756
|
+
const leg3 = findDegenerateRepetition(
|
|
757
|
+
"set -euo pipefail; mkdir -p build tmp dist logs data assets src test docs lib bin etc usr var opt srv && " +
|
|
758
|
+
"cp -r config.yaml README.md LICENSE package.json tsconfig.json src lib test docs assets && " +
|
|
759
|
+
"chmod +x scripts/deploy.sh scripts/backup.sh scripts/monitor.sh && " +
|
|
760
|
+
"./scripts/deploy.sh --env production --region eu-west-1 --tag v1.2.3 --dry-run false > deploy.log 2>&1 || echo deploy failed",
|
|
761
|
+
tcfg,
|
|
762
|
+
);
|
|
763
|
+
out.push(`degenerate legit cmd → ${leg1 || leg2 || leg3 ? "flag" : "ok"} (exp ok) ${!leg1 && !leg2 && !leg3 ? "✅" : "❌"}`);
|
|
764
|
+
|
|
765
|
+
// Interleaved meltdown ("A B A B…") has no long run — caught via freq/share.
|
|
766
|
+
const inter = findDegenerateRepetition("noguerol petete ".repeat(150), tcfg);
|
|
767
|
+
out.push(`degenerate interleaved → ${inter ? `flag (${inter.token} ×${inter.freq})` : "no"} (exp flag — freq/share clause) ${inter ? "✅" : "❌"}`);
|
|
768
|
+
|
|
769
|
+
// Word list written ACROSS lines: separators are the literal "\n" escapes
|
|
770
|
+
// inside the stored JSON args — must still tokenize word by word.
|
|
771
|
+
const acrossLines = argJson("for u in " + "noguerol\n".repeat(120) + "done");
|
|
772
|
+
const linesHit = findDegenerateRepetition(acrossLines, tcfg);
|
|
773
|
+
out.push(`degenerate across \n → ${linesHit ? `flag (${linesHit.token} ×${linesHit.freq})` : "no"} (exp flag — escaped newlines) ${linesHit ? "✅" : "❌"}`);
|
|
774
|
+
|
|
775
|
+
// No-space giant token ("noguerol" glued) — perfect-power clause.
|
|
776
|
+
const glued = findDegenerateRepetition("noguerol".repeat(300), tcfg);
|
|
777
|
+
out.push(`degenerate glued token → ${glued ? `flag (${glued.token} ×${glued.freq})` : "no"} (exp flag — perfect power) ${glued ? "✅" : "❌"}`);
|
|
778
|
+
|
|
779
|
+
// Turn weight: one degenerate turn = 2 consecutive points (warning on first
|
|
780
|
+
// sight), normal turns stay at 1.
|
|
781
|
+
const wDeg = detectionTurnWeight(mDet, tcfg);
|
|
782
|
+
const wTxt = detectionTurnWeight(dl("text", 0.8), tcfg);
|
|
783
|
+
out.push(`degenerate turn weight → degenerate ${wDeg}, text ${wTxt} (exp 2, 1) ${wDeg === 2 && wTxt === 1 ? "✅" : "❌"}`);
|
|
784
|
+
|
|
785
|
+
// --- v1.6.1: no-progress outcome runs (mutated re-runs, same outcome) ---
|
|
786
|
+
// Regression: the NFS session — ~90 mutated re-runs of the SAME experiment
|
|
787
|
+
// (ssh exportfs/mount, labels test A…test QQQ, targets alternating Javi /
|
|
788
|
+
// Compartido), every one failing rc=32. Args mutate each turn (same call
|
|
789
|
+
// never recurs → tool-loop silent) and journalctl noise varies per attempt,
|
|
790
|
+
// but the FAILURE SIGNATURE repeats: that IS the loop. Fires on the 9th
|
|
791
|
+
// attempt (8 prior same-failure matches ≥ outcomeMinRepeats).
|
|
792
|
+
const nfsCmd = (label: string, target: string) =>
|
|
793
|
+
argJson(
|
|
794
|
+
`sshpass -p X ssh -o ConnectTimeout=10 noguerol@petete 'cd /tmp && echo X | sudo -S bash -c "echo --- test ${label}: rootdir=/volume2, absolute paths, fsid=0 and 1, mount /${target} ---; ` +
|
|
795
|
+
`cat > /etc/exports << EOF\n/volume2/NAS-8TB-Javi *(rw,sync,no_subtree_check,fsid=0)\nEOF\nexportfs -ra\nsystemctl restart nfs-server\n` +
|
|
796
|
+
`mount -t nfs4 -o vers=4.2 127.0.0.1:/${target} /tmp/nfstest 2>&1; echo rc=\$?; journalctl -u nfs-mountd | tail -4"' 2>&1`,
|
|
797
|
+
);
|
|
798
|
+
const nfsFailFor = (target: string, sec: number) =>
|
|
799
|
+
resultFingerprint(
|
|
800
|
+
[
|
|
801
|
+
{
|
|
802
|
+
type: "text",
|
|
803
|
+
text:
|
|
804
|
+
`--- mount /${target} --- | mount.nfs4: access denied by server while mounting 127.0.0.1:/${target} rc=32 | ` +
|
|
805
|
+
`Sep 09 18:${sec} petete systemd[1]: Started nfs-mountd.service (PID ${1000 + sec})`,
|
|
806
|
+
},
|
|
807
|
+
],
|
|
808
|
+
false,
|
|
809
|
+
)!;
|
|
810
|
+
const nfsOk = resultFingerprint([{ type: "text", text: "rc=0 | TARGET SOURCE FSTYPE | /tmp/nfstest 127.0.0.1:/ nfs4 rw,relatime" }], false)!;
|
|
811
|
+
const targets = ["NAS-8TB-Javi", "NAS-8TB-Compartido"];
|
|
812
|
+
const nfsMsgs = [..."ABCDEFGHI"].map((l, idx) =>
|
|
813
|
+
mk("", [{ name: "bash", args: nfsCmd(l, targets[idx % 2]), result: nfsFailFor(targets[idx % 2], 100 + idx) }]),
|
|
814
|
+
);
|
|
815
|
+
const nfsDet = detectLoops(asState(nfsMsgs), tcfg);
|
|
816
|
+
const nfsHit = nfsDet.find((d) => d.type === "outcome");
|
|
817
|
+
out.push(`outcome fires on 9th → ${nfsHit ? `outcome (${nfsHit.description.slice(0, 100)}…)` : nfsDet.map((d) => d.type).join(",") || "no"} (exp outcome — mixed targets) ${nfsHit ? "✅" : "❌"}`);
|
|
818
|
+
out.push(`outcome weight → ${detectionTurnWeight(nfsDet, tcfg)} (exp 2) ${detectionTurnWeight(nfsDet, tcfg) === 2 ? "✅" : "❌"}`);
|
|
819
|
+
out.push(`outcome post-steer = ignored→ ${isVerbatimRepeat(nfsDet) ? "yes" : "no"} (exp yes — same failing outcome after the break) ${isVerbatimRepeat(nfsDet) ? "✅" : "❌"}`);
|
|
820
|
+
|
|
821
|
+
// Below the repeat count: 5 identical-failure attempts → still trying, silent.
|
|
822
|
+
const fewMsgs = [..."ABCDE"].map((l, idx) =>
|
|
823
|
+
mk("", [{ name: "bash", args: nfsCmd(l, targets[idx % 2]), result: nfsFailFor(targets[idx % 2], 100 + idx) }]),
|
|
824
|
+
);
|
|
825
|
+
const fewDet = detectLoops(asState(fewMsgs), tcfg);
|
|
826
|
+
out.push(`outcome needs 8 prior → ${fewDet.some((d) => d.type === "outcome") ? "outcome" : "silent"} (exp silent at 5 attempts) ${!fewDet.some((d) => d.type === "outcome") ? "✅" : "❌"}`);
|
|
827
|
+
|
|
828
|
+
// Converging sweep (v1.1 guarantee): similar args but the outcome CHANGES
|
|
829
|
+
// (progress!) — must stay silent even with many attempts.
|
|
830
|
+
const progMsgs = [..."ABCDEFGHIJ"].map((l, idx) =>
|
|
831
|
+
mk("", [
|
|
832
|
+
{
|
|
833
|
+
name: "bash",
|
|
834
|
+
args: nfsCmd(l, targets[idx % 2]),
|
|
835
|
+
result:
|
|
836
|
+
idx === 9
|
|
837
|
+
? nfsOk
|
|
838
|
+
: resultFingerprint(
|
|
839
|
+
[{ type: "text", text: `attempt ${idx}: failed with rc=${idx + 30} reason=${idx % 3}` }],
|
|
840
|
+
true,
|
|
841
|
+
)!,
|
|
842
|
+
},
|
|
843
|
+
]),
|
|
844
|
+
);
|
|
845
|
+
const progDet = detectLoops(asState(progMsgs), tcfg);
|
|
846
|
+
out.push(`outcome converging sweep → ${progDet.some((d) => d.type === "outcome") ? "outcome" : "silent"} (exp silent — outcomes differ = progress) ${!progDet.some((d) => d.type === "outcome") ? "✅" : "❌"}`);
|
|
847
|
+
|
|
848
|
+
// Different FAILURE kinds with similar args (denied vs timeout vs no-such-file)
|
|
849
|
+
// = evolving diagnosis, not the same wall — silent too.
|
|
850
|
+
const diffFailMsgs = [..."ABCDEFGHIJ"].map((l, idx) => {
|
|
851
|
+
const reasons = ["access denied by server", "timed out after 90 seconds", "No such file or directory"];
|
|
852
|
+
const r = reasons[idx % 3];
|
|
853
|
+
return mk("", [
|
|
854
|
+
{ name: "bash", args: nfsCmd(l, targets[idx % 2]), result: resultFingerprint([{ type: "text", text: `mount failed: ${r} rc=32` }], false)! },
|
|
855
|
+
]);
|
|
856
|
+
});
|
|
857
|
+
const diffFailDet = detectLoops(asState(diffFailMsgs), tcfg);
|
|
858
|
+
out.push(`outcome diff failures → ${diffFailDet.some((d) => d.type === "outcome") ? "outcome" : "silent"} (exp silent — error changed = progress) ${!diffFailDet.some((d) => d.type === "outcome") ? "✅" : "❌"}`);
|
|
859
|
+
|
|
860
|
+
// Task-stream coexistence: a punched_log batch with identical tool results
|
|
861
|
+
// must NOT count toward the outcome run (identical OK = normal batch).
|
|
862
|
+
const batchFail = resultFingerprint([{ type: "text", text: "logged" }], false)!;
|
|
863
|
+
const batchOutMsgs = [1, 2, 3, 4, 5, 6, 7, 8, 9].map((n) => mk(NARR, [{ name: "punched_log", args: noteArgs(String(n)), result: batchFail }]));
|
|
864
|
+
const batchOutDet = detectLoops(asState(batchOutMsgs), tcfg);
|
|
865
|
+
out.push(`outcome batch excluded → ${batchOutDet.some((d) => d.type === "outcome") ? "outcome" : "silent"} (exp silent — identical OKs are batch norm) ${!batchOutDet.some((d) => d.type === "outcome") ? "✅" : "❌"}`);
|
|
866
|
+
|
|
867
|
+
// Failure gate: 9 near-identical attempts that all SUCCEED identically (e.g.
|
|
868
|
+
// re-verifying a working setup, or a file-write batch) must stay silent —
|
|
869
|
+
// only repeated FAILURES prove no progress.
|
|
870
|
+
const okMsgs = [..."ABCDEFGHI"].map((l, idx) => mk("", [{ name: "bash", args: nfsCmd(l, targets[idx % 2]), result: nfsOk }]));
|
|
871
|
+
const okDet = detectLoops(asState(okMsgs), tcfg);
|
|
872
|
+
out.push(`outcome identical OKs → ${okDet.some((d) => d.type === "outcome") ? "outcome" : "silent"} (exp silent — success repeats ≠ loop) ${!okDet.some((d) => d.type === "outcome") ? "✅" : "❌"}`);
|
|
873
|
+
out.push(`isFailResult gate → err| → ${isFailResult("err|boom") ? "fail" : "ok"}, rc32 → ${isFailResult("ok|rc32 denied") ? "fail" : "ok"}, rc0/ok → ${isFailResult("ok|rc 0 12 passed") ? "fail" : "ok"} (exp fail, fail, ok) ${isFailResult("err|boom") && isFailResult("ok|rc32 denied") && !isFailResult("ok|rc 0 12 passed") ? "✅" : "❌"}`);
|
|
874
|
+
|
|
446
875
|
return out;
|
|
447
876
|
}
|
package/src/index.ts
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
/**
|
|
2
2
|
* antiloop — detect reasoning loops and intervene.
|
|
3
|
-
* Hooks: message_end, input, turn_end, session_start, session_shutdown.
|
|
3
|
+
* Hooks: message_end, tool_call, input, turn_end, session_start, session_shutdown.
|
|
4
4
|
* Commands: /antiloop [enable|disable|status|config|log|reset|test]
|
|
5
5
|
*
|
|
6
6
|
* Intervention model (v1.5):
|
|
@@ -15,9 +15,19 @@
|
|
|
15
15
|
* (≥98% similar / identical tool loop) ignoredSteerLimit times, antiloop
|
|
16
16
|
* hard-stops the run (ctx.abort).
|
|
17
17
|
* abort (level 3, opt-in via abortThreshold) — stops the run outright.
|
|
18
|
+
*
|
|
19
|
+
* v1.6 adds the intra-message degenerate detector: a model whose decoder
|
|
20
|
+
* anchors on a token repeats it hundreds of times INSIDE one message / tool
|
|
21
|
+
* call (the real 46 KB bash "noguerol ×5145" meltdown). That needs no peer
|
|
22
|
+
* message, so it is caught at message_end — BEFORE the tool calls execute —
|
|
23
|
+
* and degenerate bash commands are additionally blocked in the tool_call
|
|
24
|
+
* hook (blockDegenerateBash). One degenerate turn counts degenerateTurnWeight
|
|
25
|
+
* (2) consecutive points: warning on first sight, force-break steer on the
|
|
26
|
+
* second consecutive meltdown, hard stop shortly after if it keeps repeating.
|
|
18
27
|
*/
|
|
19
28
|
|
|
20
29
|
import type { ExtensionAPI, ExtensionContext } from "@earendil-works/pi-coding-agent";
|
|
30
|
+
import { isToolCallEventType } from "@earendil-works/pi-coding-agent";
|
|
21
31
|
import { truncateToWidth } from "@earendil-works/pi-tui";
|
|
22
32
|
import { loadConfig, saveConfig } from "./config.ts";
|
|
23
33
|
import type { AntiloopState, LoopDetection, Runtime, TrackedToolCall } from "./types.ts";
|
|
@@ -38,6 +48,7 @@ function newState(): AntiloopState {
|
|
|
38
48
|
lastDetectedTurnIndex: -1,
|
|
39
49
|
steerDelivered: false,
|
|
40
50
|
ignoredSteerCount: 0,
|
|
51
|
+
turnSeq: 0,
|
|
41
52
|
};
|
|
42
53
|
}
|
|
43
54
|
|
|
@@ -235,7 +246,7 @@ export default function antiloopExtension(pi: ExtensionAPI) {
|
|
|
235
246
|
});
|
|
236
247
|
}
|
|
237
248
|
|
|
238
|
-
pi.on("message_end", async (event) => {
|
|
249
|
+
pi.on("message_end", async (event, ctx) => {
|
|
239
250
|
if (!config.enabled) return;
|
|
240
251
|
const msg = event.message;
|
|
241
252
|
if (msg.role !== "assistant") return;
|
|
@@ -263,12 +274,79 @@ export default function antiloopExtension(pi: ExtensionAPI) {
|
|
|
263
274
|
thinking: thinking || undefined,
|
|
264
275
|
toolCalls: toolCalls.length ? toolCalls : undefined,
|
|
265
276
|
timestamp: Date.now(),
|
|
266
|
-
|
|
277
|
+
// Monotonic: the array is trimmed below (detectionWindow + 5), so
|
|
278
|
+
// recentMessages.length would repeat after the first trim and the
|
|
279
|
+
// turn_end dedupe guard (turnIndex === lastDetectedTurnIndex) would
|
|
280
|
+
// skip every later message — antiloop going blind mid-session.
|
|
281
|
+
turnIndex: state.turnSeq++,
|
|
267
282
|
});
|
|
268
283
|
}
|
|
269
284
|
if (state.recentMessages.length > config.detectionWindow + 5) {
|
|
270
285
|
state.recentMessages = state.recentMessages.slice(-(config.detectionWindow + 5));
|
|
271
286
|
}
|
|
287
|
+
|
|
288
|
+
// v1.6 — degenerate meltdowns are handled HERE, at message_end, because
|
|
289
|
+
// turn_end only fires after tool execution (too late to stop the 46 KB
|
|
290
|
+
// "noguerol ×5000" bash from running) and an aborted generation may never
|
|
291
|
+
// reach turn_end at all. The signal is self-contained (one pathological
|
|
292
|
+
// payload, no peer message) so the full escalation ladder runs right now;
|
|
293
|
+
// the turn-guard makes the later turn_end pass skip this message and no
|
|
294
|
+
// detection is double-counted.
|
|
295
|
+
if (!config.detectDegenerate) return;
|
|
296
|
+
const { scanMessageDegenerate, degenerateDescription, nextLevel, isVerbatimRepeat } = await import("./detect.ts");
|
|
297
|
+
const hit = scanMessageDegenerate(content, toolCalls, config);
|
|
298
|
+
if (!hit) return;
|
|
299
|
+
const tracked = state.recentMessages[state.recentMessages.length - 1];
|
|
300
|
+
if (tracked) state.lastDetectedTurnIndex = tracked.turnIndex;
|
|
301
|
+
const prevLevel = state.currentLevel;
|
|
302
|
+
state.consecutiveDetections += Math.max(1, config.degenerateTurnWeight);
|
|
303
|
+
state.totalDetections++;
|
|
304
|
+
const det: LoopDetection = {
|
|
305
|
+
type: "degenerate",
|
|
306
|
+
similarity: 1,
|
|
307
|
+
messageIndices: [state.recentMessages.length - 1],
|
|
308
|
+
description: degenerateDescription(hit),
|
|
309
|
+
timestamp: Date.now(),
|
|
310
|
+
};
|
|
311
|
+
state.detections.push(det);
|
|
312
|
+
if (state.detections.length > config.maxHistoryEntries) {
|
|
313
|
+
state.detections = state.detections.slice(-config.maxHistoryEntries);
|
|
314
|
+
}
|
|
315
|
+
applyLevel(nextLevel(state.consecutiveDetections, config));
|
|
316
|
+
const level = state.currentLevel;
|
|
317
|
+
|
|
318
|
+
if (level === 1 && prevLevel < 1) {
|
|
319
|
+
// Warning: informational only (never injects — a warning must not stall).
|
|
320
|
+
if (config.notifyOnDetection) {
|
|
321
|
+
ctx.ui.notify(`antiloop: warning — ${det.description}`, "warning");
|
|
322
|
+
}
|
|
323
|
+
} else if (level === 2) {
|
|
324
|
+
if (!state.steerDelivered) {
|
|
325
|
+
// Steer the break message before the next LLM call. The degenerate bash
|
|
326
|
+
// itself is blocked by the tool_call gate, so the model sees the block
|
|
327
|
+
// reason + the steer together and can still change approach.
|
|
328
|
+
if (ctx.signal !== undefined) {
|
|
329
|
+
deliverForceBreak(ctx, [det]);
|
|
330
|
+
} else if (prevLevel < 2 && config.notifyOnDetection) {
|
|
331
|
+
ctx.ui.notify(`antiloop: force break — ${det.description}`, "error");
|
|
332
|
+
}
|
|
333
|
+
} else if (isVerbatimRepeat([det])) {
|
|
334
|
+
// The model produced degenerate output AGAIN after the break message:
|
|
335
|
+
// count it; once the ignore limit is hit the run is cut for good.
|
|
336
|
+
state.ignoredSteerCount++;
|
|
337
|
+
if (state.ignoredSteerCount >= config.ignoredSteerLimit) {
|
|
338
|
+
hardStop(
|
|
339
|
+
ctx,
|
|
340
|
+
`antiloop: abort — degenerate output repeated ${state.ignoredSteerCount}× after the force break — run stopped; provide new instructions`,
|
|
341
|
+
);
|
|
342
|
+
}
|
|
343
|
+
}
|
|
344
|
+
} else if (level === 3) {
|
|
345
|
+
// abortThreshold configured and reached: stop the run BEFORE the degenerate
|
|
346
|
+
// tool calls can execute (ctx.abort kills the pending tool batch).
|
|
347
|
+
hardStop(ctx, `antiloop: abort — ${det.description} — run stopped; provide new instructions`);
|
|
348
|
+
}
|
|
349
|
+
updateStatus(ctx);
|
|
272
350
|
});
|
|
273
351
|
|
|
274
352
|
pi.on("input", async (event) => {
|
|
@@ -291,9 +369,30 @@ export default function antiloopExtension(pi: ExtensionAPI) {
|
|
|
291
369
|
return { action: "continue" };
|
|
292
370
|
});
|
|
293
371
|
|
|
372
|
+
/**
|
|
373
|
+
* v1.6 — degenerate bash gate. A meltdown message whose command repeats one
|
|
374
|
+
* word hundreds of times (the 46 KB "noguerol ×5145" SSH-wordlist brute
|
|
375
|
+
* force) must NEVER execute: it is pure context burn at best, and a real
|
|
376
|
+
* brute-force / destructive repetition at worst. message_end already
|
|
377
|
+
* escalated it; here we block the actual call before it runs. The block
|
|
378
|
+
* reason is fed back to the model as the tool error, so the next LLM call
|
|
379
|
+
* sees why the command was refused and can change approach.
|
|
380
|
+
*/
|
|
381
|
+
pi.on("tool_call", async (event, ctx) => {
|
|
382
|
+
if (!config.enabled || !config.detectDegenerate || !config.blockDegenerateBash) return;
|
|
383
|
+
if (!isToolCallEventType("bash", event)) return;
|
|
384
|
+
const { findDegenerateRepetition, degenerateDescription } = await import("./detect.ts");
|
|
385
|
+
const hit = findDegenerateRepetition(event.input.command ?? "", config);
|
|
386
|
+
if (!hit) return;
|
|
387
|
+
return {
|
|
388
|
+
block: true,
|
|
389
|
+
reason: `[antiloop] blocked: ${degenerateDescription({ ...hit, where: "bash command" })} — this is stuck generation, not a real command. Do NOT retry it: stop and take one small, concrete step instead.`,
|
|
390
|
+
};
|
|
391
|
+
});
|
|
392
|
+
|
|
294
393
|
pi.on("turn_end", async (event, ctx) => {
|
|
295
394
|
if (!config.enabled) return;
|
|
296
|
-
const { detectLoops, detectTaskStreams, isVerbatimRepeat, nextLevel, resultFingerprint } = await import("./detect.ts");
|
|
395
|
+
const { detectLoops, detectTaskStreams, detectionTurnWeight, isVerbatimRepeat, nextLevel, resultFingerprint } = await import("./detect.ts");
|
|
297
396
|
|
|
298
397
|
const last = state.recentMessages[state.recentMessages.length - 1];
|
|
299
398
|
if (!last || last.turnIndex === state.lastDetectedTurnIndex) {
|
|
@@ -337,7 +436,9 @@ export default function antiloopExtension(pi: ExtensionAPI) {
|
|
|
337
436
|
}
|
|
338
437
|
|
|
339
438
|
// ---- detection: escalate ------------------------------------------
|
|
340
|
-
|
|
439
|
+
// Degenerate turns count degenerateTurnWeight (default 2) points so a
|
|
440
|
+
// single intra-message meltdown already reaches the warning level.
|
|
441
|
+
state.consecutiveDetections += detectionTurnWeight(detections, config);
|
|
341
442
|
state.totalDetections++;
|
|
342
443
|
state.detections.push(...detections);
|
|
343
444
|
if (state.detections.length > config.maxHistoryEntries) {
|
package/src/types.ts
CHANGED
|
@@ -16,6 +16,46 @@ export interface AntiloopConfig {
|
|
|
16
16
|
* its output (even while still similar) gets room to escape on its own.
|
|
17
17
|
*/
|
|
18
18
|
ignoredSteerLimit: number;
|
|
19
|
+
/**
|
|
20
|
+
* Intra-message degenerate repetition (the "noguerol \u00d75145" meltdown class): a model
|
|
21
|
+
* stuck emitting the same token hundreds of times INSIDE one message / tool call.
|
|
22
|
+
* Unlike the other detectors it needs no peer message: one pathological payload is
|
|
23
|
+
* already conclusive. Fires on the first occurrence; each degenerate turn is weighted
|
|
24
|
+
* (degenerateTurnWeight, default 2) so a single meltdown reaches the warning level.
|
|
25
|
+
*/
|
|
26
|
+
detectDegenerate: boolean;
|
|
27
|
+
/** Minimum normalized tokens in a payload before it is scanned (shorter = not conclusive). */
|
|
28
|
+
degenerateMinTokens: number;
|
|
29
|
+
/** Longest run of ONE identical word that flags a payload as degenerate. */
|
|
30
|
+
degenerateMaxRun: number;
|
|
31
|
+
/** Total occurrences of one word (anywhere, interleaved) that flags when combined with share. */
|
|
32
|
+
degenerateMaxFreq: number;
|
|
33
|
+
/** Word frequency share (freq/total) needed together with degenerateMaxFreq. */
|
|
34
|
+
degenerateMaxShare: number;
|
|
35
|
+
/** How many consecutive-detection points ONE degenerate turn adds (2 = warn on first sight). */
|
|
36
|
+
degenerateTurnWeight: number;
|
|
37
|
+
/** Block a bash tool call whose command shows degenerate repetition BEFORE it executes. */
|
|
38
|
+
blockDegenerateBash: boolean;
|
|
39
|
+
/**
|
|
40
|
+
* No-progress outcome runs (v1.6.1): a model that keeps re-running the SAME
|
|
41
|
+
* experiment with cosmetic mutations (labels/permutations) while the outcome
|
|
42
|
+
* stays the SAME FAILURE — the real NFS session where ~90 near-identical
|
|
43
|
+
* ssh exportfs/mount tests (test A … test QQQ) all failed rc=32. The
|
|
44
|
+
* tool-loop detector can't see it: args mutate every turn so the same call
|
|
45
|
+
* never recurs (labels differ), and results only VETO tool loops today.
|
|
46
|
+
* Signal: >= outcomeMinRepeats PRIOR attempts inside the window whose args
|
|
47
|
+
* are >= outcomeArgSimilarity similar AND whose captured result is the same
|
|
48
|
+
* outcome (>= resultSimilarityThreshold), all after the last real user input.
|
|
49
|
+
*/
|
|
50
|
+
detectOutcomeLoops: boolean;
|
|
51
|
+
outcomeMinRepeats: number;
|
|
52
|
+
outcomeArgSimilarity: number;
|
|
53
|
+
/** Same-FAILURE gate for the outcome detector: minimum similarity between the
|
|
54
|
+
* digit-stripped error signatures of two failing attempts to count as the
|
|
55
|
+
* SAME failure. Looser than the veto threshold on purpose: the signature
|
|
56
|
+
* repeats across attempts of the same wall even when legitimately varying
|
|
57
|
+
* words (mount targets) sit inside it. */
|
|
58
|
+
outcomeSigThreshold: number;
|
|
19
59
|
/**
|
|
20
60
|
* How close tool-call arguments must be (0..1) to count as the SAME call.
|
|
21
61
|
* High by default: long bash commands share scaffolding (env setup, flags,
|
|
@@ -66,7 +106,7 @@ export interface AntiloopConfig {
|
|
|
66
106
|
toggleShortcut: string;
|
|
67
107
|
}
|
|
68
108
|
|
|
69
|
-
export type LoopKind = "text" | "tool" | "thinking" | "structural";
|
|
109
|
+
export type LoopKind = "text" | "tool" | "thinking" | "structural" | "degenerate" | "outcome";
|
|
70
110
|
|
|
71
111
|
export interface LoopDetection {
|
|
72
112
|
type: LoopKind;
|
|
@@ -94,6 +134,10 @@ export interface TrackedMessage {
|
|
|
94
134
|
thinking?: string;
|
|
95
135
|
toolCalls?: TrackedToolCall[];
|
|
96
136
|
timestamp: number;
|
|
137
|
+
/** Monotonic push sequence (state.turnSeq++), NOT the window index: the
|
|
138
|
+
* recent-messages array is trimmed to detectionWindow+5, so an array-length
|
|
139
|
+
* based index would collide after trimming and make the turn_end dedupe
|
|
140
|
+
* guard skip every later message (antiloop going blind mid-session). */
|
|
97
141
|
turnIndex: number;
|
|
98
142
|
}
|
|
99
143
|
|
|
@@ -116,6 +160,10 @@ export interface AntiloopState {
|
|
|
116
160
|
lastUserMessageTime: number;
|
|
117
161
|
/** turnIndex of the last tracked message detection already ran on. */
|
|
118
162
|
lastDetectedTurnIndex: number;
|
|
163
|
+
/** Monotonic sequence for the next tracked message's turnIndex (see
|
|
164
|
+
* TrackedMessage.turnIndex). Persists across trims; reset on /reset and at
|
|
165
|
+
* session start. */
|
|
166
|
+
turnSeq: number;
|
|
119
167
|
/** True once the force-break user message was steered into the current episode.
|
|
120
168
|
* One steer per episode: repeated steering would spam the conversation. Cleared
|
|
121
169
|
* when the episode decays (currentLevel back to 0) or on real user input. */
|