mimo-stable 1.1.0__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,299 @@
1
+ Metadata-Version: 2.4
2
+ Name: mimo-stable
3
+ Version: 1.1.0
4
+ Summary: Record, detect, and apply engineering guardrails for degenerate loops in LLM output and tool calls
5
+ Author: xli498
6
+ License: MIT License
7
+
8
+ Copyright (c) 2026 xli498
9
+
10
+ Permission is hereby granted, free of charge, to any person obtaining a copy
11
+ of this software and associated documentation files (the "Software"), to deal
12
+ in the Software without restriction, including without limitation the rights
13
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
14
+ copies of the Software, and to permit persons to whom the Software is
15
+ furnished to do so, subject to the following conditions:
16
+
17
+ The above copyright notice and this permission notice shall be included in all
18
+ copies or substantial portions of the Software.
19
+
20
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
21
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
22
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
23
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
24
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
25
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
26
+ SOFTWARE.
27
+
28
+ Requires-Python: >=3.10
29
+ Description-Content-Type: text/markdown
30
+ License-File: LICENSE
31
+ Dynamic: license-file
32
+
33
+ # LLM Degenerate Loop Guardrails
34
+
35
+ [English README](README.en.md)
36
+
37
+ [![Quality Gate](https://github.com/xli498/mimo-stable/actions/workflows/quality.yml/badge.svg)](https://github.com/xli498/mimo-stable/actions/workflows/quality.yml)
38
+ [![Release](https://img.shields.io/github/v/release/xli498/mimo-stable)](https://github.com/xli498/mimo-stable/releases)
39
+ [![License](https://img.shields.io/github/license/xli498/mimo-stable)](LICENSE)
40
+ [![Python](https://img.shields.io/badge/python-3.10%2B-blue)](pyproject.toml)
41
+
42
+ 面向国产及其他 LLM 的输出流和工具调用退化循环经验记录与检测工具。
43
+ 项目最初来自 MiMo `reasoning=True` 场景的重复输出观察,后来发现类似现象
44
+ 也可能出现在 GLM 等其他模型中。这里分享的是识别、止损和复盘方法,不是
45
+ 针对任何模型的根治方案。
46
+
47
+ > [!IMPORTANT]
48
+ > 当前项目的核心能力是**检测与止损**,不是自动修复模型。检测到循环后,
49
+ > 由上层运行器决定停止、切换模型、重试或人工复核。
50
+
51
+ ## 目录
52
+
53
+ - [30 秒开始](#30-秒开始)
54
+ - [问题描述](#问题描述)
55
+ - [三层防御体系](#三层防御体系)
56
+ - [检测策略](#检测策略)
57
+ - [可复现证据](#可复现证据)
58
+ - [如何记录和分享新案例](#如何记录和分享新案例)
59
+ - [工程侧缓解措施](#工程侧缓解措施不是模型修复)
60
+ - [示例](#示例)
61
+ - [文件结构](#文件结构)
62
+ - [测试](#测试)
63
+ - [安装与集成](#安装与集成)
64
+ - [许可](#许可)
65
+ - [行为契约](#行为契约)
66
+
67
+ ## 30 秒开始
68
+
69
+ ![Guardrails architecture](docs/architecture.svg)
70
+
71
+ > **核心流程:** 模型输出 → 检测信号 → 保守决策摘要 → 上层运行器处理。
72
+ > 检测器和恢复层都不执行重试、模型切换或工具调用。
73
+
74
+ ```bash
75
+ python3 scripts/detect_loop.py --log fixtures/loop_detected.log
76
+ python3 tests/test_detector.py
77
+ python3 scripts/benchmark_fixtures.py
78
+ ```
79
+
80
+ 机器集成使用单份 JSON 摘要和退出码:
81
+
82
+ ```bash
83
+ python3 scripts/detect_loop.py --json --timeout 60 --log fixtures/loop_detected.log
84
+ # 退出码 0 = 未检测到;1 = 检测到;2 = 参数/输入错误
85
+
86
+ # 将检测摘要转成保守的恢复决策(只输出决策,不执行重试)
87
+ python3 scripts/detect_loop.py --json --timeout 60 --log fixtures/loop_detected.log | \
88
+ python3 scripts/recovery_policy.py --retryable
89
+ ```
90
+
91
+ 典型检测摘要:
92
+
93
+ ```json
94
+ {
95
+ "loop_detected": true,
96
+ "details": {
97
+ "type": "consecutive_identical_output"
98
+ }
99
+ }
100
+ ```
101
+
102
+ | 退出码 | 含义 |
103
+ | :---: | :--- |
104
+ | `0` | 未检测到循环 |
105
+ | `1` | 检测到循环 |
106
+ | `2` | 参数或输入错误 |
107
+
108
+ ## 问题描述
109
+
110
+ 在特定模型、参数、任务和上游服务条件下,LLM 可能进入退化循环状态:
111
+
112
+ - **症状**:同一段输出重复,持续时间异常增长
113
+ - **语言切换**:特定中文任务中切换为英文,且无视语言约束
114
+ - **功能停滞**:不执行有进展的工具调用,仅重复输出文本
115
+ - **根因尚未确定**:`reasoning=True`、上下文长度、工具链状态、服务端实现等
116
+ 都可能是相关变量;本项目不把相关性写成因果结论。
117
+
118
+ 类似表现并不等于同一个 bug。MiMo、GLM 或其他国产模型的案例必须分别记录
119
+ 模型版本、端点、参数、任务和时间,不能把一个模型的触发概率或规避经验直接
120
+ 外推到另一个模型。
121
+
122
+ ## 三层防御体系
123
+
124
+ ### 第一层:工程侧限制(止损,不是修复)
125
+
126
+ 在运行器中设置硬性超时和 Token 限制(具体字段以所用框架官方文档为准):
127
+
128
+ ```yaml
129
+ # 示例结构;字段名和数值必须按实际框架、模型与任务校准。
130
+ provider:
131
+ model: <your-model>
132
+ timeout: <task-specific-limit>
133
+ max_tokens: <task-specific-limit>
134
+ ```
135
+
136
+ > [!WARNING]
137
+ > **历史案例,不是通用推荐:** 最初 MiMo 案例曾使用
138
+ > `timeoutSeconds=180` 和 `maxTokens=8000` 限制单次资源占用。这些值不是
139
+ > 跨模型配置建议,也不表示能够改变模型进入循环的概率。
140
+
141
+ 作用是限制单次故障的最长资源占用,不改变模型本身的循环概率。
142
+
143
+ ### 第二层:行为检测
144
+
145
+ 运行 `scripts/detect_loop.py` 实时监控模型输出,检测连续重复。
146
+
147
+ 管道输入以空行切分输出块;`--timeout` 的持续时间只在流式输入期间有实际意义,
148
+ 离线日志使用日志中的块时间戳。
149
+
150
+ ```bash
151
+ python3 scripts/detect_loop.py --log logs/sample_degenerate_loop.log
152
+ model_output 2>&1 | python3 scripts/detect_loop.py
153
+ python3 scripts/detect_loop.py --json --log logs/sample_degenerate_loop.log
154
+ ```
155
+
156
+ 检测规则:
157
+
158
+ - 连续 3+ 次输出块完全相同或高度相似
159
+ - 连续 3+ 次工具调用参数完全相同
160
+ - 已知副作用工具的重复调用
161
+ - 显式声明中文任务后的英文漂移
162
+ - 可选的持续时间门控
163
+
164
+ ### 第三层:行为规则(AGENTS.md)
165
+
166
+ 在 AGENTS.md 或对应运行器规则中加入检测和处置约束。详见
167
+ [SKILL.md](SKILL.md)。该层是提示和流程规则,不替代运行时检测。
168
+
169
+ ## 检测策略
170
+
171
+ 默认文本重复采用**持续时间门控**:需要达到 `--threshold` 次相似输出,且
172
+ 重复窗口达到 `--timeout` 秒。这适合日志后处理,减少短暂重复的误报。
173
+
174
+ 如果上层需要在达到重复次数后立即得到信号,可显式使用:
175
+
176
+ ```bash
177
+ python3 scripts/detect_loop.py --text-mode instant --log fixtures/repeated_but_short.log
178
+ ```
179
+
180
+ 工具调用重复和已知副作用工具的重复调用不受文本持续时间门控影响;生产集成仍应
181
+ 结合幂等键、调用结果和重试原因做二次判断。
182
+
183
+ ## 可复现证据
184
+
185
+ > [!NOTE]
186
+ > Fixture 用于稳定回归,历史日志用于记录具体观察;两者的证据性质不同。
187
+
188
+ `logs/sample_degenerate_loop.log` 是历史观察日志;`logs/fixed_normal_run.log`
189
+ 是一次正常运行日志。它们只说明具体案例,不构成模型故障率统计,也不证明
190
+ 任何参数能“修复”模型。
191
+
192
+ 当前仓库没有足够实验次数估计任何模型的通用触发概率,因此不提供“某模型
193
+ 有 X% 概率出问题”之类的结论。
194
+
195
+ ## 如何记录和分享新案例
196
+
197
+ 建议至少记录以下信息,并在发布前脱敏:
198
+
199
+ - 模型与精确版本、供应商/端点、调用时间段
200
+ - `reasoning`、temperature、max tokens、上下文规模等实际参数
201
+ - 任务类型、是否多轮、是否涉及工具调用
202
+ - 重复发生前后的输出块摘要或哈希,不公开密钥、隐私和完整敏感工具参数
203
+ - 是否真正发生资源浪费/副作用,检测器是否报警,检测延迟
204
+ - 重试、切换模型或改变参数后的结果
205
+
206
+ 案例记录用于复盘和横向比较,不应写成因果证明。没有原始证据时,使用
207
+ “观察到”“可能相关”“尚未复现”,不要使用“必然”“已证明”“彻底解决”。
208
+
209
+ ## 工程侧缓解措施(不是模型修复)
210
+
211
+ | 措施 | 作用 | 边界 |
212
+ | :--- | :--- | :--- |
213
+ | `timeoutSeconds=180` | 限制单次资源损失 | 不改变模型行为 |
214
+ | `maxTokens=8000` | 限制输出上限 | 不等于不再循环 |
215
+ | 行为层检测 | 提供停止/切换信号 | 需要上层执行恢复动作 |
216
+
217
+ ## 示例
218
+
219
+ - [基础文本检测](examples/basic-text-detection.md)
220
+ - [工具调用检测](examples/tool-call-detection.md)
221
+ - [恢复决策接入](examples/recovery-policy-integration.md)
222
+
223
+ 每个示例都只展示输入、检测信号或决策输出;实际停止、重试、切换模型和人工复核
224
+ 仍由上层运行器负责。
225
+
226
+ ## 文件结构
227
+
228
+ ```
229
+ mimo-stable/
230
+ ├── README.md
231
+ ├── SKILL.md
232
+ ├── pyproject.toml
233
+ ├── CHANGELOG.md
234
+ ├── scripts/
235
+ │ ├── detect_loop.py # 循环检测脚本
236
+ │ ├── benchmark_fixtures.py # 可复现 fixture 基准测试
237
+ │ └── recovery_policy.py # 保守恢复决策层(不执行副作用)
238
+ ├── tests/test_detector.py # 行为契约测试
239
+ ├── examples/ # 最小集成示例
240
+ ├── fixtures/ # 规范化回归样例
241
+ ├── logs/ # 历史观察日志
242
+ └── references/parameters.md
243
+ ```
244
+
245
+ ## 测试
246
+
247
+ ```bash
248
+ python3 -m py_compile scripts/*.py
249
+ bash -n scripts/*.sh
250
+ python3 tests/test_detector.py
251
+ python3 scripts/benchmark_fixtures.py
252
+ # 或使用仓库提供的入口:
253
+ bash scripts/test_short.sh
254
+ bash scripts/test_long.sh
255
+ ```
256
+
257
+ 当前 benchmark 覆盖 9 个规范化案例:循环、正常输出、短时重复、近似文本、
258
+ 变化参数重试、非连续工具调用、工具 key 顺序、重复副作用工具和中文任务语言漂移。
259
+
260
+ 历史日志在默认 180 秒阈值下可能不报警;复核时使用 `--timeout 60`。生产阈值
261
+ 应按业务容忍度评估,不能把测试阈值直接当作生产配置。
262
+
263
+ ## 安装与集成
264
+
265
+ 项目保持零运行时依赖。当前支持源码直接运行和本地 CLI 安装,尚未发布到 PyPI。
266
+
267
+ ### 方式一:直接运行源码
268
+
269
+ ```bash
270
+ git clone https://github.com/xli498/mimo-stable.git
271
+ cd mimo-stable
272
+ python3 scripts/detect_loop.py --log fixtures/loop_detected.log
273
+ ```
274
+
275
+ ### 方式二:安装本地 CLI
276
+
277
+ ```bash
278
+ python3 -m pip install --no-deps .
279
+ mimo-loop-detect --json --timeout 60 --log fixtures/loop_detected.log
280
+ ```
281
+
282
+ > [!NOTE]
283
+ > 当前尚未发布公共 PyPI 包,因此不要使用 `pip install mimo-stable` 获取发行包。
284
+
285
+ 上层运行器不应在检测到循环后盲目重试:
286
+
287
+ 1. 保存脱敏后的事件摘要;
288
+ 2. 停止当前生成或工具链;
289
+ 3. 根据任务是否幂等决定重试;
290
+ 4. 必要时切换模型或请求人工复核。
291
+
292
+ ## 许可
293
+
294
+ MIT
295
+
296
+ ## 行为契约
297
+
298
+ `fixtures/` 中的规范化样例用于稳定回归,历史日志用于说明观察事实;二者不互相
299
+ 替代。执行 `python3 tests/test_detector.py` 可验证检测器行为。
@@ -0,0 +1,11 @@
1
+ mimo_stable-1.1.0.dist-info/licenses/LICENSE,sha256=Ie3LloKoBYMu1ugx4D6aUTHmzfmA7R-V23Th1L5U5Jo,1063
2
+ scripts/__init__.py,sha256=FvX-LylRYHIfvaSLKqwEnq9GkV3caaJgpHMmc0_czkk,42
3
+ scripts/benchmark_fixtures.py,sha256=wfbATtAK9Y8iMHEZDez7GIjqv7BnZogWY9CWdT-_11M,1666
4
+ scripts/check_version.py,sha256=NZ-Rkl_dBx4YI5uC-v0BMDyOixMajBOO4qamTM8Wfzo,1050
5
+ scripts/detect_loop.py,sha256=AT4Ex7tLWDF4anDPj7DsHMb68gqFRC-6PW1uYrpwC5U,16994
6
+ scripts/recovery_policy.py,sha256=cm-ktDHDbOv980bvM1dOnlBDuGL8heMqceUQdm7x1OE,3044
7
+ mimo_stable-1.1.0.dist-info/METADATA,sha256=g6wV4EUxPmlga02Kk7TgjKj1mYgHaFZz0hbxXjrbWdY,11325
8
+ mimo_stable-1.1.0.dist-info/WHEEL,sha256=YVMoNqKzERt-wjUZwJ33xBGAwnFl-4cqbYkTtWa4itE,91
9
+ mimo_stable-1.1.0.dist-info/entry_points.txt,sha256=FMfdVPdqCC1m9WkgWZx34OEPN6AGRtM7iQS7mcf_Bx4,62
10
+ mimo_stable-1.1.0.dist-info/top_level.txt,sha256=rmzd5mewlrJy4sT608KPib7sM7edoY75AeqJeY3SPB4,8
11
+ mimo_stable-1.1.0.dist-info/RECORD,,
@@ -0,0 +1,5 @@
1
+ Wheel-Version: 1.0
2
+ Generator: setuptools (84.0.0)
3
+ Root-Is-Purelib: true
4
+ Tag: py3-none-any
5
+
@@ -0,0 +1,2 @@
1
+ [console_scripts]
2
+ mimo-loop-detect = scripts.detect_loop:main
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 xli498
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1 @@
1
+ scripts
scripts/__init__.py ADDED
@@ -0,0 +1 @@
1
+ """Executable modules for mimo-stable."""
@@ -0,0 +1,51 @@
1
+ #!/usr/bin/env python3
2
+ """Run the checked-in fixtures and print a small reproducible benchmark."""
3
+ from __future__ import annotations
4
+
5
+ import json
6
+ import subprocess
7
+ import sys
8
+ from pathlib import Path
9
+
10
+ ROOT = Path(__file__).resolve().parents[1]
11
+ DETECTOR = ROOT / "scripts" / "detect_loop.py"
12
+
13
+ CASES = [
14
+ ("loop_detected.log", True),
15
+ ("normal_output.log", False),
16
+ ("repeated_but_short.log", False),
17
+ ("near_duplicate_below_threshold.log", False),
18
+ ("tool_retry_changed_params.log", False),
19
+ ("nonconsecutive_tool_calls.log", False),
20
+ ("tool_key_order_repeat.log", True),
21
+ ("side_effect_repeat.log", True),
22
+ ("language_drift_zh.log", True),
23
+ ]
24
+
25
+
26
+ def run(path: str, expected: bool) -> tuple[bool, str]:
27
+ args = [sys.executable, str(DETECTOR), "--json", "--timeout", "60"]
28
+ if path == "language_drift_zh.log":
29
+ args += ["--expect-language", "zh"]
30
+ args += ["--log", str(ROOT / "fixtures" / path)]
31
+ result = subprocess.run(args, capture_output=True, text=True)
32
+ try:
33
+ data = json.loads(result.stdout)
34
+ except json.JSONDecodeError:
35
+ return False, f"invalid JSON: {result.stdout!r}"
36
+ actual = bool(data.get("loop_detected"))
37
+ return actual == expected, f"expected={expected} actual={actual} type={data.get('details', {}).get('type')}"
38
+
39
+
40
+ def main() -> int:
41
+ passed = 0
42
+ for path, expected in CASES:
43
+ ok, detail = run(path, expected)
44
+ print(f"{'PASS' if ok else 'FAIL'} {path}: {detail}")
45
+ passed += ok
46
+ print(f"\nfixture benchmark: {passed}/{len(CASES)} cases passed")
47
+ return 0 if passed == len(CASES) else 1
48
+
49
+
50
+ if __name__ == "__main__":
51
+ raise SystemExit(main())
@@ -0,0 +1,35 @@
1
+ #!/usr/bin/env python3
2
+ """Verify that the package version is represented in the changelog."""
3
+
4
+ from __future__ import annotations
5
+
6
+ import re
7
+ import sys
8
+ try:
9
+ import tomllib
10
+ except ModuleNotFoundError: # Python 3.10 compatibility
11
+ try:
12
+ import tomli as tomllib
13
+ except ModuleNotFoundError as exc:
14
+ raise SystemExit(
15
+ "Python 3.10 requires the optional 'tomli' package to run check_version.py"
16
+ ) from exc
17
+ from pathlib import Path
18
+
19
+
20
+ ROOT = Path(__file__).resolve().parents[1]
21
+
22
+
23
+ def main() -> int:
24
+ pyproject = tomllib.loads((ROOT / "pyproject.toml").read_text(encoding="utf-8"))
25
+ version = pyproject["project"]["version"]
26
+ changelog = (ROOT / "CHANGELOG.md").read_text(encoding="utf-8")
27
+ if not re.search(rf"^## \[{re.escape(version)}\]", changelog, re.MULTILINE):
28
+ print(f"CHANGELOG.md has no release heading for version {version}", file=sys.stderr)
29
+ return 1
30
+ print(f"version consistency passed: {version}")
31
+ return 0
32
+
33
+
34
+ if __name__ == "__main__":
35
+ raise SystemExit(main())
scripts/detect_loop.py ADDED
@@ -0,0 +1,465 @@
1
+ #!/usr/bin/env python3
2
+ """
3
+ LLM Degenerate Loop Guardrails — Detection Script
4
+
5
+ Detects degenerate loops in model output streams where the model
6
+ repeatedly emits identical or near-identical text blocks.
7
+
8
+ Detection rules:
9
+ 1. Consecutive identical output blocks (3+ repeats)
10
+ 2. Same tool called 3+ times with identical parameters
11
+ 3. Output duration exceeding threshold without meaningful change
12
+
13
+ Usage:
14
+ # From stdin (pipe model output):
15
+ model_output 2>&1 | python3 detect_loop.py
16
+
17
+ # From log file:
18
+ python3 detect_loop.py --log sample_degenerate_loop.log
19
+
20
+ # With custom thresholds:
21
+ python3 detect_loop.py --threshold 4 --timeout 300 --log sample.log
22
+
23
+ # JSON output for integration:
24
+ python3 detect_loop.py --json --log sample.log
25
+
26
+ Exit codes:
27
+ 0 — No loop detected
28
+ 1 — Loop detected
29
+ 2 — Invalid arguments
30
+ """
31
+
32
+ import argparse
33
+ import hashlib
34
+ import json
35
+ import re
36
+ import sys
37
+ import time
38
+ from collections import deque
39
+ from datetime import datetime
40
+ from difflib import SequenceMatcher
41
+ from pathlib import Path
42
+
43
+
44
+ # Repeating any external action is riskier than repeating a read-only action.
45
+ # This set is intentionally conservative: unknown tools retain the normal
46
+ # three-repeat rule rather than being guessed as side-effecting.
47
+ SIDE_EFFECT_TOOLS = {
48
+ "send", "write", "edit", "delete", "remove", "payment", "pay",
49
+ "purchase", "post", "publish", "create", "submit",
50
+ }
51
+
52
+
53
+ class LoopDetector:
54
+ """Detects degenerate loops in model output streams."""
55
+
56
+ def __init__(
57
+ self,
58
+ repeat_threshold: int = 3,
59
+ time_threshold: int = 180,
60
+ similarity_threshold: float = 0.95,
61
+ text_mode: str = "duration",
62
+ json_output: bool = False,
63
+ expected_language: str | None = None,
64
+ ):
65
+ self.repeat_threshold = repeat_threshold
66
+ self.time_threshold = time_threshold
67
+ self.similarity_threshold = similarity_threshold
68
+ self.text_mode = text_mode
69
+ self.json_output = json_output
70
+ self.expected_language = expected_language
71
+
72
+ # State
73
+ window_size = max(repeat_threshold + 5, 20)
74
+ self.blocks: deque = deque(maxlen=window_size)
75
+ self.tool_calls: deque = deque(maxlen=window_size)
76
+ self.block_timestamps: deque = deque(maxlen=window_size)
77
+ self.loop_detected = False
78
+ self.loop_reason = ""
79
+ self.loop_details: dict = {}
80
+
81
+ def _similarity(self, a: str, b: str) -> float:
82
+ """Compute similarity ratio between two strings."""
83
+ return SequenceMatcher(None, a, b).ratio()
84
+
85
+ @staticmethod
86
+ def _params_hash(params: str) -> str:
87
+ """Return a stable, non-reversible identifier for logging only."""
88
+ return hashlib.sha256(params.encode("utf-8")).hexdigest()[:16]
89
+
90
+ @staticmethod
91
+ def _looks_english(text: str) -> bool:
92
+ """Conservative language-drift heuristic for an explicitly Chinese task."""
93
+ latin = len(re.findall(r"[A-Za-z]", text))
94
+ cjk = len(re.findall(r"[\u4e00-\u9fff]", text))
95
+ return latin >= 10 and cjk == 0
96
+
97
+ def _extract_tool_calls(self, text: str) -> list[dict]:
98
+ """Extract tool call signatures from model output text.
99
+
100
+ Handles common formats:
101
+ - JSON tool_call blocks
102
+ - Function call patterns
103
+ - Tool use markers
104
+ """
105
+ calls = []
106
+
107
+ # Pattern: JSON-style tool calls
108
+ # Match blocks like: {"name": "read", "parameters": {"path": "foo"}}
109
+ json_tool_pattern = r'\{\s*"name"\s*:\s*"([^"]+)"\s*,\s*"parameters"\s*:\s*(\{[^}]+\})\s*\}'
110
+ for match in re.finditer(json_tool_pattern, text, re.DOTALL):
111
+ try:
112
+ params_str = match.group(2)
113
+ params = json.loads(params_str)
114
+ calls.append({
115
+ "name": match.group(1),
116
+ "params": json.dumps(params, sort_keys=True),
117
+ })
118
+ except json.JSONDecodeError:
119
+ pass
120
+
121
+ # Pattern: function-like calls
122
+ func_pattern = r'(read|write|exec|edit|browser|web_fetch)\s*\(([^)]*)\)'
123
+ for match in re.finditer(func_pattern, text, re.IGNORECASE):
124
+ tool = match.group(1).lower()
125
+ args = match.group(2).strip()
126
+ if args:
127
+ calls.append({"name": tool, "params": args})
128
+
129
+ return calls
130
+
131
+ def _log(self, level: str, message: str):
132
+ """Output a log message."""
133
+ timestamp = datetime.now().isoformat()
134
+ # --json is a machine-readable contract: emit exactly one final summary
135
+ # document from main(), never interleave event records with it.
136
+ if self.json_output:
137
+ return
138
+ print(f"[{timestamp}] [{level}] {message}", flush=True)
139
+
140
+ def process_block(self, text: str, block_time: float | None = None):
141
+ """Process a single output block from the model.
142
+
143
+ Args:
144
+ text: The model output text for this block
145
+ block_time: Timestamp when this block was emitted (unix epoch)
146
+ """
147
+ if block_time is None:
148
+ block_time = time.time()
149
+
150
+ text = text.strip()
151
+ if not text:
152
+ return
153
+
154
+ self.blocks.append(text)
155
+ self.block_timestamps.append(block_time)
156
+
157
+ # Tool calls must be consecutive output events. A normal model/result
158
+ # block between calls is evidence of progress, so do not carry an old
159
+ # call across it when evaluating a call-loop.
160
+ tools = self._extract_tool_calls(text)
161
+ if not tools:
162
+ self.tool_calls.clear()
163
+ for t in tools:
164
+ self.tool_calls.append(t)
165
+
166
+ # A configured Chinese task that repeatedly produces English-only
167
+ # output is an actionable drift signal. It is opt-in because language
168
+ # cannot be inferred safely from an arbitrary log.
169
+ if self.expected_language == "zh" and len(self.blocks) >= self.repeat_threshold:
170
+ recent_blocks = list(self.blocks)[-self.repeat_threshold:]
171
+ if all(self._looks_english(block) for block in recent_blocks):
172
+ self.loop_detected = True
173
+ self.loop_reason = "Detected repeated English-only output in a Chinese task"
174
+ self.loop_details = {
175
+ "type": "language_drift",
176
+ "expected_language": "zh",
177
+ "repeats": len(recent_blocks),
178
+ }
179
+ self._log("LOOP_DETECTED", self.loop_reason)
180
+ return
181
+
182
+ # --- Rule 1: Consecutive identical output blocks ---
183
+ if len(self.blocks) >= self.repeat_threshold:
184
+ recent = list(self.blocks)[-self.repeat_threshold:]
185
+ # Check if all recent blocks are identical or highly similar
186
+ base = recent[0]
187
+ identical = all(
188
+ self._similarity(base, b) >= self.similarity_threshold
189
+ for b in recent[1:]
190
+ )
191
+
192
+ if identical:
193
+ duration = block_time - self.block_timestamps[-self.repeat_threshold]
194
+ # Also check duration threshold (default 3 min)
195
+ if self.text_mode == "instant" or duration >= self.time_threshold:
196
+ self.loop_detected = True
197
+ self.loop_reason = (
198
+ f"Detected {self.repeat_threshold}+ consecutive identical "
199
+ f"output blocks over {duration:.0f}s"
200
+ )
201
+ self.loop_details = {
202
+ "type": "consecutive_identical_output",
203
+ "signal": "instant" if self.text_mode == "instant" else "duration_gated",
204
+ "repeats": len(recent),
205
+ "duration_seconds": duration,
206
+ "sample": base[:200] + ("..." if len(base) > 200 else ""),
207
+ "block_sizes": [len(b) for b in recent],
208
+ }
209
+ self._log(
210
+ "LOOP_DETECTED",
211
+ f"{self.loop_reason}\n Sample: {self.loop_details['sample']}",
212
+ )
213
+
214
+ # --- Rule 2: Same tool called 3+ times with identical params ---
215
+ if len(self.tool_calls) >= self.repeat_threshold:
216
+ recent_tools = list(self.tool_calls)[-self.repeat_threshold:]
217
+ tool_sigs = [(t["name"], t["params"]) for t in recent_tools]
218
+ if len(set(tool_sigs)) == 1:
219
+ self.loop_detected = True
220
+ name, params = tool_sigs[0]
221
+ self.loop_reason = (
222
+ f"Detected {len(recent_tools)} consecutive identical "
223
+ f"tool calls: {name}"
224
+ )
225
+ self.loop_details = {
226
+ "type": "identical_tool_calls",
227
+ "tool": name,
228
+ "params_hash": self._params_hash(params),
229
+ "repeats": len(recent_tools),
230
+ }
231
+ self._log(
232
+ "LOOP_DETECTED",
233
+ f"{self.loop_reason}\n Params hash: {self.loop_details['params_hash']}",
234
+ )
235
+
236
+ # External side effects must not be retried blindly. Two identical
237
+ # attempts are enough to stop and require a fresh safety review.
238
+ if len(self.tool_calls) >= 2:
239
+ previous, current = list(self.tool_calls)[-2:]
240
+ if (
241
+ previous["name"] == current["name"]
242
+ and previous["params"] == current["params"]
243
+ and current["name"] in SIDE_EFFECT_TOOLS
244
+ ):
245
+ self.loop_detected = True
246
+ self.loop_reason = f"Detected repeated side-effecting tool call: {current['name']}"
247
+ self.loop_details = {
248
+ "type": "repeated_side_effect_tool_call",
249
+ "tool": current["name"],
250
+ "params_hash": self._params_hash(current["params"]),
251
+ "repeats": 2,
252
+ }
253
+ self._log("LOOP_DETECTED", self.loop_reason)
254
+
255
+ def reset(self):
256
+ """Reset detector state."""
257
+ self.blocks.clear()
258
+ self.tool_calls.clear()
259
+ self.block_timestamps.clear()
260
+ self.loop_detected = False
261
+ self.loop_reason = ""
262
+ self.loop_details = {}
263
+
264
+ def summary(self) -> dict:
265
+ """Return detection summary."""
266
+ return {
267
+ "loop_detected": self.loop_detected,
268
+ "reason": self.loop_reason,
269
+ "details": self.loop_details,
270
+ "blocks_processed": len(self.blocks),
271
+ "tool_calls_tracked": len(self.tool_calls),
272
+ }
273
+
274
+
275
+ def read_from_file(filepath: str) -> list[tuple[str, float]]:
276
+ """Read blocks from a log file.
277
+
278
+ Each block is separated by a delimiter line like:
279
+ --- BLOCK N at TIMESTAMP ---
280
+
281
+ Returns list of (text, timestamp) tuples.
282
+ """
283
+ path = Path(filepath)
284
+ if not path.exists():
285
+ print(f"Error: File not found: {filepath}", file=sys.stderr)
286
+ sys.exit(2)
287
+
288
+ content = path.read_text(encoding="utf-8", errors="replace")
289
+ blocks = []
290
+ current_block: list[str] = []
291
+ current_time = time.time()
292
+
293
+ for line in content.splitlines():
294
+ # Match block delimiter
295
+ match = re.match(r"^--- BLOCK (\d+) at (.+) ---$", line)
296
+ if match:
297
+ if current_block:
298
+ blocks.append(("\n".join(current_block), current_time))
299
+ current_block = []
300
+ try:
301
+ ts = datetime.fromisoformat(match.group(2))
302
+ current_time = ts.timestamp()
303
+ except ValueError:
304
+ pass
305
+ else:
306
+ current_block.append(line)
307
+
308
+ # Don't forget the last block
309
+ if current_block:
310
+ blocks.append(("\n".join(current_block), current_time))
311
+
312
+ return blocks
313
+
314
+
315
+ def iter_from_stdin():
316
+ """Yield stdin blocks as they close, preserving their arrival time.
317
+
318
+ Processing must happen during iteration. Collecting all of stdin before
319
+ evaluating blocks would assign nearly identical timestamps and invalidate
320
+ duration-gated detection for real streamed output.
321
+ """
322
+ current_block: list[str] = []
323
+
324
+ for line in sys.stdin:
325
+ if line.strip() == "":
326
+ if current_block:
327
+ yield "\n".join(current_block), time.time()
328
+ current_block = []
329
+ else:
330
+ current_block.append(line.rstrip("\n"))
331
+
332
+ if current_block:
333
+ yield "\n".join(current_block), time.time()
334
+
335
+
336
+ def main():
337
+ parser = argparse.ArgumentParser(
338
+ description="LLM Degenerate Loop Guardrails — Detection",
339
+ formatter_class=argparse.RawDescriptionHelpFormatter,
340
+ epilog="""
341
+ Examples:
342
+ model_output 2>&1 | python3 detect_loop.py
343
+ python3 detect_loop.py --log sample.log
344
+ python3 detect_loop.py --threshold 4 --timeout 300 --log sample.log
345
+ python3 detect_loop.py --json --log sample.log --timeout 180
346
+ """,
347
+ )
348
+ parser.add_argument(
349
+ "--log",
350
+ type=str,
351
+ help="Path to log file (reads from stdin if not provided)",
352
+ )
353
+ parser.add_argument(
354
+ "--threshold",
355
+ type=int,
356
+ default=3,
357
+ help="Number of consecutive identical blocks to trigger detection (default: 3)",
358
+ )
359
+ parser.add_argument(
360
+ "--timeout",
361
+ type=int,
362
+ default=180,
363
+ help="Minimum duration in seconds for loop detection (default: 180, i.e. 3 min)",
364
+ )
365
+ parser.add_argument(
366
+ "--text-mode",
367
+ choices=["duration", "instant"],
368
+ default="duration",
369
+ help=(
370
+ "Text-repeat policy: duration requires --timeout; instant triggers "
371
+ "after the repeat threshold (default: duration)"
372
+ ),
373
+ )
374
+ parser.add_argument(
375
+ "--similarity",
376
+ type=float,
377
+ default=0.95,
378
+ help="Similarity threshold for text comparison (default: 0.95)",
379
+ )
380
+ parser.add_argument(
381
+ "--json",
382
+ action="store_true",
383
+ help="Output results in JSON format",
384
+ )
385
+ parser.add_argument(
386
+ "--expect-language",
387
+ choices=["zh"],
388
+ help="Enable language-drift detection for an explicitly Chinese task",
389
+ )
390
+ args = parser.parse_args()
391
+
392
+ if args.threshold < 2:
393
+ parser.error("--threshold must be at least 2")
394
+ if args.timeout < 0:
395
+ parser.error("--timeout must be non-negative")
396
+ if not 0.0 <= args.similarity <= 1.0:
397
+ parser.error("--similarity must be between 0 and 1")
398
+
399
+ # Create detector
400
+ detector = LoopDetector(
401
+ repeat_threshold=args.threshold,
402
+ time_threshold=args.timeout,
403
+ similarity_threshold=args.similarity,
404
+ text_mode=args.text_mode,
405
+ json_output=args.json,
406
+ expected_language=args.expect_language,
407
+ )
408
+
409
+ # Read input
410
+ start_time = time.time()
411
+ total_blocks = 0
412
+ if args.log:
413
+ blocks = read_from_file(args.log)
414
+ for text, ts in blocks:
415
+ total_blocks += 1
416
+ detector.process_block(text, ts)
417
+ if detector.loop_detected:
418
+ break
419
+ else:
420
+ for text, ts in iter_from_stdin():
421
+ total_blocks += 1
422
+ detector.process_block(text, ts)
423
+ if detector.loop_detected:
424
+ break
425
+
426
+ if total_blocks == 0:
427
+ print("Warning: No input blocks found", file=sys.stderr)
428
+ if args.json:
429
+ print(json.dumps({"error": "no_input"}))
430
+ sys.exit(2)
431
+
432
+ # Output summary
433
+ summary = detector.summary()
434
+ summary["elapsed_seconds"] = time.time() - start_time
435
+ summary["total_blocks"] = total_blocks
436
+
437
+ if args.json:
438
+ print(json.dumps(summary, ensure_ascii=False, indent=2))
439
+ else:
440
+ print(f"\n{'=' * 60}")
441
+ print(f"Loop Detection Summary")
442
+ print(f"{'=' * 60}")
443
+ print(f" Blocks processed: {total_blocks}")
444
+ print(f" Loop detected: {'YES ⚠️' if summary['loop_detected'] else 'NO ✅'}")
445
+ if summary["loop_detected"]:
446
+ print(f" Reason: {summary['reason']}")
447
+ details = summary.get("details", {})
448
+ if details.get("type") == "consecutive_identical_output":
449
+ print(f" Repeats: {details.get('repeats')}")
450
+ print(f" Duration: {details.get('duration_seconds', 0):.0f}s")
451
+ print(f" Sample: {details.get('sample', 'N/A')}")
452
+ elif details.get("type") == "identical_tool_calls":
453
+ print(f" Tool: {details.get('tool')}")
454
+ print(f" Repeats: {details.get('repeats')}")
455
+ print(f" Params hash: {details.get('params_hash', 'N/A')}")
456
+ elif details.get("type") == "repeated_side_effect_tool_call":
457
+ print(f" Tool: {details.get('tool', 'N/A')}")
458
+ print(" Action: pause and re-check before retrying")
459
+ print(f"{'=' * 60}")
460
+
461
+ sys.exit(1 if summary["loop_detected"] else 0)
462
+
463
+
464
+ if __name__ == "__main__":
465
+ main()
@@ -0,0 +1,78 @@
1
+ #!/usr/bin/env python3
2
+ """Turn detector output into a conservative, machine-readable next action.
3
+
4
+ This module deliberately does not execute tools, retry requests, or switch models.
5
+ It only produces a decision so the caller can apply its own permissions and policy.
6
+ """
7
+ from __future__ import annotations
8
+
9
+ import argparse
10
+ import json
11
+ import sys
12
+ from typing import Any
13
+
14
+
15
+ def decide(summary: dict[str, Any], *, retryable: bool = False, retry_count: int = 0) -> dict[str, Any]:
16
+ raw_detected = summary.get("loop_detected", False)
17
+ if not isinstance(raw_detected, bool):
18
+ raise ValueError("loop_detected must be a JSON boolean")
19
+ detected = raw_detected
20
+ details = summary.get("details") or {}
21
+ kind = details.get("type")
22
+
23
+ if not detected:
24
+ action = "continue"
25
+ rationale = "No degenerate-loop signal was detected."
26
+ elif kind == "repeated_side_effect_tool_call":
27
+ action = "pause_and_review"
28
+ rationale = "A repeated side-effecting call requires an idempotency and outcome check before retry."
29
+ elif retryable and retry_count == 0:
30
+ action = "stop_and_retry_once"
31
+ rationale = "Stop the current generation, then allow one controlled retry with a fresh context or strategy."
32
+ else:
33
+ action = "stop_and_escalate"
34
+ rationale = "Stop the current generation; do not blindly retry after a loop or exhausted retry budget."
35
+
36
+ return {
37
+ "action": action,
38
+ "rationale": rationale,
39
+ "detector_reason": summary.get("reason", ""),
40
+ "detector_type": kind,
41
+ "retryable": retryable,
42
+ "retry_count": retry_count,
43
+ }
44
+
45
+
46
+ def main() -> int:
47
+ parser = argparse.ArgumentParser(description="Create a conservative recovery decision from detector JSON.")
48
+ parser.add_argument("--summary", help="Detector JSON file; otherwise read one JSON document from stdin.")
49
+ parser.add_argument("--retryable", action="store_true", help="Allow one controlled retry when no retry has happened.")
50
+ parser.add_argument("--retry-count", type=int, default=0)
51
+ args = parser.parse_args()
52
+
53
+ try:
54
+ if args.summary:
55
+ with open(args.summary, encoding="utf-8") as handle:
56
+ raw = handle.read()
57
+ else:
58
+ raw = sys.stdin.read()
59
+ summary = json.loads(raw)
60
+ if not isinstance(summary, dict):
61
+ raise ValueError("summary must be a JSON object")
62
+ if args.retry_count < 0:
63
+ raise ValueError("retry-count must be non-negative")
64
+ except (OSError, json.JSONDecodeError, ValueError) as exc:
65
+ print(json.dumps({"error": str(exc)}, ensure_ascii=False), file=sys.stderr)
66
+ return 2
67
+
68
+ try:
69
+ decision = decide(summary, retryable=args.retryable, retry_count=args.retry_count)
70
+ except ValueError as exc:
71
+ print(json.dumps({"error": str(exc)}, ensure_ascii=False), file=sys.stderr)
72
+ return 2
73
+ print(json.dumps(decision, ensure_ascii=False, indent=2))
74
+ return 0
75
+
76
+
77
+ if __name__ == "__main__":
78
+ raise SystemExit(main())