@hicaru/pi-rlm 0.3.14 → 0.3.16

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,6 +1,6 @@
1
- <p align="center">
2
- <img src="https://github.com/openzebra/rlm.pi/blob/master/assets/plugin-cover.png?raw=true" width="100%" alt="pi-rlm — Recursive Language Model plugin for Pi">
3
- </p>
1
+ # rlm.pi PI plugin
2
+
3
+ > pi-rlm — Large contexts on cheap models: Recursive Language Model (RLM) for Pi
4
4
 
5
5
  <p align="center">
6
6
  <a href="https://www.npmjs.com/package/@hicaru/pi-rlm"><img src="https://img.shields.io/npm/v/@hicaru/pi-rlm?color=cb3837&logo=npm" alt="npm version"></a>
@@ -14,45 +14,78 @@
14
14
  <a href="https://www.npmjs.com/package/@hicaru/pi-rlm">📦 npm</a>
15
15
  </p>
16
16
 
17
- ---
17
+ ## Install
18
+
19
+ ```bash
20
+ pi install npm:@hicaru/pi-rlm
21
+ ```
22
+
23
+ Run `/reload` in Pi — `/rlm`, `/rlm-config`, `/rlm-stop` appear under **[Extensions]**.
24
+ Toggle with `Ctrl+Shift+R` or `/rlm`.
25
+
26
+ To remove it later:
27
+
28
+ ```bash
29
+ pi uninstall npm:@hicaru/pi-rlm
30
+ ```
31
+
32
+ <p align="center">
33
+ <img src="https://github.com/openzebra/rlm.pi/blob/master/assets/hero.png?raw=true" width="100%" alt="rlm.pi — OOLONG benchmark results">
34
+ </p>
35
+
36
+ ## What is pi-rlm?
18
37
 
19
- **The ONLY Recursive Language Model plugin for Pi.** No new agent to learn, no
20
- separate CLI, no YAML workflows just `/rlm` and your existing Pi session becomes a
21
- recursive orchestration engineyour best model orchestrates, cheap worker models
22
- do the reading.
38
+ A Pi plugin that turns your session into a **Recursive Language Model (RLM)**: instead of
39
+ stuffing a huge document into the prompt, the context lives in a Python REPL and your best
40
+ model orchestrates itsearching, decomposing, and delegating leaf reads to cheap worker
41
+ models, recursively. Same Pi session, same tools, same keys: `/rlm` and go. Reads
42
+ `.pdf` `.docx` `.xlsx` `.epub` and more, works with any OpenRouter model, 100% local.
23
43
 
24
- > **One install. One toggle. Infinite context.**
44
+ ## Benchmarks
25
45
 
26
- ## Why pi-rlm?
46
+ <p align="center">
47
+ <img src="https://github.com/openzebra/rlm.pi/blob/master/assets/hero.png?raw=true" width="100%" alt="OOLONG benchmark — latest results">
48
+ </p>
27
49
 
28
- | Advantage | What it means |
29
- |-----------|---------------|
30
- | 🔌 **Plugin, not a new agent** | Stays inside Pi. You keep your keybindings, your theme, your tools, your muscle memory. |
31
- | 📄 **Reads ANY document** | `.pdf` `.docx` `.pptx` `.xlsx` `.epub` `.rtf` `.odt` `.csv` `.html` `.xml` — drop them in, they become Markdown in `context`. |
32
- | 🪶 **Unix-style — tiny & composable** | Does ONE thing (RLM orchestration). Pair it with any other Pi plugin. No lock-in. |
33
- | 🧠 **Smartest model orchestrates, cheapest model researches** | Root uses your best model; workers auto-pick the cheapest. Recursive children inherit the full `context` for free. |
34
- | ⏳ **Long-running with goals** | Toggle `/rlm` on, set a goal, let it loop. Runs survive across chat turns — go make coffee. |
35
- | 🔒 **100% local, 100% private** | No servers. Your API keys never leave your machine. One `python3` subprocess — that's it. |
50
+ **OOLONG (oolong-synth)** paper-tier long-context suite; latest journal per model,
51
+ cost per task from real `costUsd` (older journals estimated at OpenRouter list prices):
36
52
 
37
- ## See it in action
53
+ | Model | Score | Avg. cost/task |
54
+ |-------|-------|----------------|
55
+ | `qwen/qwen3.8-27b` | **100%** | $0.0127 |
56
+ | `google/gemma-3-27b-it` | 83.3% | $0.0013 |
57
+ | `qwen/qwen3-30b-a3b-instruct-2507` | 66.7% | $0.0009 |
58
+ | `mistralai/mistral-small-3.2-24b-instruct` | 66.7% | $0.0025 |
38
59
 
39
- <div align="center">
60
+ Lite suite — `needle` multi-needle recall, `codeqa` repo-QA, `coding` fix task
61
+ (7 tasks × 2 passes per model, deterministic graders, no LLM-as-judge):
40
62
 
41
- <video src="https://github.com/openzebra/rlm.pi/raw/refs/heads/master/animation/rlm_pi_explainer.mp4" controls width="854" poster="https://raw.githubusercontent.com/openzebra/rlm.pi/master/assets/hero.png"></video>
63
+ | Model | Score | Accuracy |
64
+ |-------|-------|----------|
65
+ | `qwen/qwen3-30b-a3b-instruct-2507` | **14/14** | **100%** |
66
+ | `google/gemma-3-27b-it` | 12/14 | 86% |
67
+ | `mistralai/mistral-small-3.2-24b-instruct` | 12/14 | 86% |
42
68
 
43
- </div>
69
+ Raw per-task rows (correct, recall, latency, tokens, cost) live in
70
+ `bench/runs/*.jsonl` — one JSONL row per task, committed as history.
44
71
 
45
- ## Install (30 seconds)
72
+ ### Run the benchmarks
46
73
 
47
74
  ```bash
48
- pi install npm:@hicaru/pi-rlm
49
- ```
75
+ export OPENROUTER_API_KEY=sk-or-... # required — env vars are the only key transport
50
76
 
51
- Run `/reload` in Pi. Done. `/rlm`, `/rlm-config`, `/rlm-stop` appear under **[Extensions]**.
77
+ bun run bench # lite suite: needle + codeqa + coding
78
+ bun run bench --suite needle --limit 1 # one suite, first task only
79
+ bun run bench --model openrouter/qwen/qwen3-30b-a3b-instruct-2507
80
+ bun run bench --list # print tasks, no engine / no key
81
+ bun run bench --suite paper # paper tier: s_niah, oolong, browsecomp, codeqa_lb (downloads datasets)
82
+ ```
52
83
 
53
- Toggle with `Ctrl+Shift+R` or `/rlm` plain prompts now route through the RLM engine.
84
+ Suites: `all` (lite, default) · `needle` · `codeqa` · `coding` · `paper` · `s_niah` ·
85
+ `oolong` · `browsecomp` · `codeqa_lb`. Regenerate the hero chart:
86
+ `python3 bench/hero.py` (needs `matplotlib`).
54
87
 
55
- ## What you get
88
+ ## How it works
56
89
 
57
90
  ```
58
91
  ┌─────────────────────────┐
@@ -106,10 +139,6 @@ A **Recursive Language Model (RLM)** replaces `llm.completion(prompt)` with
106
139
  launch sub-LLM and sub-RLM calls as ordinary Python functions — decomposing,
107
140
  delegating, and synthesizing across a tree of models, not a single context window.
108
141
 
109
- **This is the only plugin that brings true RLM recursion to Pi.** Prime Agent and
110
- the reference Python library are separate agents you must switch to. pi-rlm lives
111
- inside Pi — same session, same tools, same everything.
112
-
113
142
  ## Commands
114
143
 
115
144
  | Command | Shortcut | What it does |
@@ -129,6 +158,45 @@ inside Pi — same session, same tools, same everything.
129
158
  | REPL timeout | `120`s | Bump for slow computations |
130
159
  | Max concurrent subs | `16` | More parallelism (costs RAM) |
131
160
 
161
+ ## Sampling & reproducibility
162
+
163
+ The r3 bench showed the biggest capability lever is not the model — it is the sampling:
164
+ temperature 0 took OOLONG from 71% pooled / 40% flips to 91.7–100% all-stable for $1.74
165
+ total. Those knobs are first-class in `rlm.json` (`~/.pi/agent/rlm.json`) and on the
166
+ `/rlm-config` panel:
167
+
168
+ | Field | Where | Default | What it governs |
169
+ |-------|-------|---------|-----------------|
170
+ | `rootSampling.maxTokens` | rlm.json, panel | `16384` | Output cap per root-model turn (finalize included) |
171
+ | `rootSampling.temperature` | rlm.json, panel | provider default | Root + finalize sampling temperature; `0` = deterministic |
172
+ | `smartReasoning` | rlm.json, panel | none | Thinking effort for the root model |
173
+ | `subSampling.maxTokens` | rlm.json, panel | `8192` | Output cap per leaf sub-call (`llm_query`, `llm_batch`, `map_files`) |
174
+ | `subSampling.temperature` | rlm.json, panel | provider default | Leaf sampling temperature |
175
+ | `enableVerificationNudge` | rlm.json | off | One coached redo when the root finalizes early with a bare number / short label |
176
+
177
+ **Reproducibility recipe (validated by r3):**
178
+
179
+ ```json
180
+ {
181
+ "config": {
182
+ "rootSampling": { "maxTokens": 8192, "temperature": 0, "reasoning": "high" }
183
+ }
184
+ }
185
+ ```
186
+
187
+ Reasoning tokens share the completion budget with the answer — with thinking on, keep
188
+ `maxTokens` generous (the bench doubles it to 8192; the engine warns once on turn 0 when it
189
+ is tight).
190
+
191
+ **Scope boundary:** `rlm.json` sampling applies to RLM-mode runs, `rlm()` delegation, and
192
+ child recursion at any depth (same engine function). The native Pi agent loop follows Pi's
193
+ own session settings — rlm.json never touches it.
194
+
195
+ **Model capability:** reasoning requires a model whose registry entry has `reasoning: true`.
196
+ Anything else has the level dropped before it reaches the provider (pi-ai clamps unsupported
197
+ levels to off); capability comes from the registry, so OpenRouter hybrids like
198
+ `qwen/qwen3.8-27b` just work.
199
+
132
200
  ## Prompt Architecture
133
201
 
134
202
  The system prompt follows a **contract / routing / examples / rules** pattern
@@ -142,22 +210,6 @@ The system prompt follows a **contract / routing / examples / rules** pattern
142
210
  **Key insight:** children see `Recursion depth: N` and calibrate ambition —
143
211
  delegating only when their task genuinely decomposes further.
144
212
 
145
- ## Benchmarks
146
-
147
- Tested against `rlm-lab` prompt bake-off and full dual-mode RLM runtime benchmarks
148
- on `poolside/laguna-xs-2.1:free` (a free ~32B model):
149
-
150
- | Benchmark | Mode | Result |
151
- |-----------|------|--------|
152
- | Main orchestrator (7 scenarios) | prompt bake-off | **0.958** mean score (v3 fewshot arm) |
153
- | RLM worker (4 scenarios) | prompt bake-off | **0.94** mean score (v2 contract arm) |
154
- | Needle-in-haystack (3 needles) | classic RLM | **recall 1.0** |
155
- | CodeQA timeout | classic RLM | **correct** (~3.7k tokens) |
156
- | Coding (retry fix) | orchestrator | **correct** (file edited) |
157
- | Live smoke needle | classic RLM | **hit** (~5k tokens) |
158
-
159
- > On a *free* model. Frontier models do even better.
160
-
161
213
  ## Security
162
214
 
163
215
  - **Key isolation** — provider keys live in TypeScript only; sandbox receives prompts, returns text.
@@ -166,12 +218,6 @@ on `poolside/laguna-xs-2.1:free` (a free ~32B model):
166
218
  - **Per-block timeout** — SIGALRM + parent watchdog (SIGKILL on hang).
167
219
  - **Trust** — project-local install requires Pi project trust.
168
220
 
169
- ## Uninstall
170
-
171
- ```bash
172
- pi uninstall npm:@hicaru/pi-rlm
173
- ```
174
-
175
221
  ## License
176
222
 
177
223
  MIT — see [LICENSE](./LICENSE).
package/README.ru.md CHANGED
@@ -132,9 +132,78 @@ rm -rf ~/.pi/agent/extensions/rlm
132
132
 
133
133
  > **Примечание по параллелизму:** каждый дочерний `rlm_query` запускает собственного worker `python3` (~50–150 мс «холодного старта»). В худшем случае количество параллельных интерпретаторов ≈ `maxConcurrentSubcalls`^(depth−1); при настройках по умолчанию (глубина 4, параллелизм 4) это 4³ = 64 в патологическом случае. Лимиты бюджета и ошибок (см. выше) ограничивают общие затраты независимо от степени разветвления.
134
134
 
135
+ ## Сэмплинг и воспроизводимость
136
+
137
+ Бенчмарк r3 показал: самый большой рычаг качества — не модель, а сэмплинг. Temperature 0
138
+ перевела OOLONG с 71% (pooled) / 40% (flips) на 91.7–100% all-stable за $1.74 суммарно.
139
+ Эти ручки — полноправные поля `rlm.json` (`~/.pi/agent/rlm.json`) и панели `/rlm-config`:
140
+
141
+ | Поле | Где | По умолчанию | За что отвечает |
142
+ |---|---|---|---|
143
+ | `rootSampling.maxTokens` | rlm.json, панель | `16384` | Лимит вывода на ход корневой модели (включая finalize) |
144
+ | `rootSampling.temperature` | rlm.json, панель | провайдер | Температура корневых ходов и finalize; `0` = детерминированно |
145
+ | `smartReasoning` | rlm.json, панель | нет | Усилие «размышления» корневой модели |
146
+ | `subSampling.maxTokens` | rlm.json, панель | `8192` | Лимит вывода одного листового вызова (`llm_query`, `llm_batch`, `map_files`) |
147
+ | `subSampling.temperature` | rlm.json, панель | провайдер | Температура листовых вызовов |
148
+ | `enableVerificationNudge` | rlm.json | выкл | Один повтор с подсказкой, если корень рано финализирует «голым числом» |
149
+
150
+ **Рецепт воспроизводимости (проверен r3):**
151
+
152
+ ```json
153
+ {
154
+ "config": {
155
+ "rootSampling": { "maxTokens": 8192, "temperature": 0, "reasoning": "high" }
156
+ }
157
+ }
158
+ ```
159
+
160
+ Токены размышлений делят бюджет завершения с ответом — при включённом thinking держите
161
+ `maxTokens` щедрым (бенчмарк удваивает его до 8192; движок один раз предупредит на ходе 0,
162
+ если лимит тесный).
163
+
164
+ **Граница действия:** сэмплинг из `rlm.json` управляет запусками RLM-режима, делегированием
165
+ `rlm()` и рекурсией на любой глубине (тот же движок). Собственный цикл агента Pi следует
166
+ настройкам сессии Pi — rlm.json его не касается.
167
+
168
+ **Возможности модели:** reasoning требует модель с `reasoning: true` в записи реестра.
169
+ Для остальных уровень отбрасывается до отправки провайдеру (pi-ai приводит его к off);
170
+ возможность берётся из реестра, а не из конфига.
171
+
135
172
  ## Логи запусков
136
173
 
137
174
 
175
+ ## Бенчмарки
176
+
177
+ E2E-прогоны реального движка против моделей OpenRouter — lite-сьют (`needle` поиск игл в
178
+ куче, `codeqa` вопросы по коду, `coding` задача-фикс; 7 задач × 2 прохода на модель).
179
+ Детерминированные грейдеры (recall / вхождение эталона / regex), без LLM-судьи.
180
+
181
+ Актуальные результаты — маленькие модели (≤32B параметров, платный тариф):
182
+
183
+ | Модель | Параметры | Счёт | Точность | Latency/задача |
184
+ |--------|-----------|------|----------|----------------|
185
+ | `qwen/qwen3-30b-a3b-instruct-2507` | MoE 30B / 3B активных | **14/14** | **100%** | ~15s |
186
+ | `google/gemma-3-27b-it` | dense 27B | 12/14 | 86% | ~26s |
187
+ | `mistralai/mistral-small-3.2-24b-instruct` | dense 24B | 12/14 | 86% | ~28s |
188
+
189
+ Построчные результаты (correct, recall, latency, токены, цена) — в
190
+ `bench/runs/bench-<ts>.jsonl`, одна JSONL-строка на задачу.
191
+
192
+ ### Как запустить
193
+
194
+ ```bash
195
+ export OPENROUTER_API_KEY=sk-or-... # обязателен — ключи ходят только через env
196
+
197
+ bun run bench # lite-сьют: needle + codeqa + coding
198
+ bun run bench --suite needle --limit 1 # один сьют, первая задача
199
+ bun run bench --model openrouter/qwen/qwen3-30b-a3b-instruct-2507
200
+ bun run bench --list # показать задачи, без движка и ключа
201
+ bun run bench --suite paper # paper-сьют: s_niah, oolong, browsecomp, codeqa_lb (скачивает датасеты)
202
+ ```
203
+
204
+ Сьюты: `all` (lite, по умолчанию) · `needle` · `codeqa` · `coding` · `paper` · `s_niah` ·
205
+ `oolong` · `browsecomp` · `codeqa_lb`.
206
+
138
207
  ## Безопасность
139
208
 
140
209
  - **Изоляция ключей**: ключи провайдеров хранятся только в TypeScript (`AuthStorage`); песочница получает промпты и возвращает текст, но никогда не получает ключи.
package/README.zh-CN.md CHANGED
@@ -141,6 +141,40 @@ rm -rf ~/.pi/agent/extensions/rlm
141
141
  > 默认设置下 (深度 4, 并发 4),极端情况下为 4³ = 64。预算和错误
142
142
  > 上限 (见上文) 无论扇出 (fan-out) 如何都会限制总支出。
143
143
 
144
+ ## 采样与可复现性
145
+
146
+ r3 基准测试表明:最大的能力杠杆不是模型,而是采样。仅 temperature 0 就让 OOLONG 从
147
+ 71% (pooled) / 40% (flips) 提升到 91.7–100% all-stable,总计仅花费 $1.74。这些旋钮在
148
+ `rlm.json` (`~/.pi/agent/rlm.json`) 和 `/rlm-config` 面板中都是一等公民:
149
+
150
+ | 字段 | 位置 | 默认值 | 作用范围 |
151
+ |---|---|---|---|
152
+ | `rootSampling.maxTokens` | rlm.json, 面板 | `16384` | 根模型每轮的输出上限 (包括 finalize) |
153
+ | `rootSampling.temperature` | rlm.json, 面板 | 提供商默认 | 根轮次与 finalize 的采样温度;`0` = 确定性 |
154
+ | `smartReasoning` | rlm.json, 面板 | 无 | 根模型的思考强度 |
155
+ | `subSampling.maxTokens` | rlm.json, 面板 | `8192` | 每个叶子子调用 (`llm_query`, `llm_batch`, `map_files`) 的输出上限 |
156
+ | `subSampling.temperature` | rlm.json, 面板 | 提供商默认 | 叶子调用的采样温度 |
157
+ | `enableVerificationNudge` | rlm.json | 关 | 当根节点过早以“裸数字”定稿时,给予一次带提示的重做 |
158
+
159
+ **可复现性配方 (经 r3 验证):**
160
+
161
+ ```json
162
+ {
163
+ "config": {
164
+ "rootSampling": { "maxTokens": 8192, "temperature": 0, "reasoning": "high" }
165
+ }
166
+ }
167
+ ```
168
+
169
+ 思考 token 与答案共享补全预算——开启 thinking 时请保持充裕的 `maxTokens`
170
+ (基准测试将其翻倍至 8192;若过紧,引擎会在第 0 轮警告一次)。
171
+
172
+ **作用边界:** `rlm.json` 的采样适用于 RLM 模式运行、`rlm()` 委托以及任意深度的子递归
173
+ (同一个引擎函数)。Pi 原生代理循环遵循 Pi 自身的会话设置——rlm.json 不会触及它。
174
+
175
+ **模型能力:** reasoning 要求模型的注册表条目为 `reasoning: true`。其他情况会在发送给
176
+ 提供商之前丢弃该级别 (pi-ai 会将不支持的级别钳制为 off);能力来自注册表,而非配置。
177
+
144
178
  ## 运行日志
145
179
 
146
180
  - **密钥隔离**:供应商密钥仅存在于 TypeScript (`AuthStorage`) 中;沙箱
@@ -157,6 +191,38 @@ rm -rf ~/.pi/agent/extensions/rlm
157
191
  连续错误上限。
158
192
  - **信任**:本地安装需要 Pi 项目信任。
159
193
 
194
+ ## 基准测试
195
+
196
+ 针对 OpenRouter 聊天模型对真实引擎进行端到端测试 —— 精简套件(`needle` 多针召回、
197
+ `codeqa` 代码库问答、`coding` 修复任务;每个模型 7 个任务 × 2 轮)。确定性评分
198
+ (召回率 / 标准答案包含 / 正则),无 LLM 评审。
199
+
200
+ 最新结果 —— 小型模型(≤32B 参数,付费额度):
201
+
202
+ | 模型 | 参数 | 得分 | 准确率 | 每任务延迟 |
203
+ |------|------|------|--------|------------|
204
+ | `qwen/qwen3-30b-a3b-instruct-2507` | MoE 30B / 3B 激活 | **14/14** | **100%** | ~15s |
205
+ | `google/gemma-3-27b-it` | dense 27B | 12/14 | 86% | ~26s |
206
+ | `mistralai/mistral-small-3.2-24b-instruct` | dense 24B | 12/14 | 86% | ~28s |
207
+
208
+ 逐任务原始数据(正确性、召回率、延迟、token、成本)位于 `bench/runs/bench-<ts>.jsonl`
209
+ —— 每个任务一行 JSONL,作为历史记录提交。
210
+
211
+ ### 运行基准测试
212
+
213
+ ```bash
214
+ export OPENROUTER_API_KEY=sk-or-... # 必需 —— 密钥仅通过环境变量传递
215
+
216
+ bun run bench # 精简套件:needle + codeqa + coding
217
+ bun run bench --suite needle --limit 1 # 单个套件,仅第一个任务
218
+ bun run bench --model openrouter/qwen/qwen3-30b-a3b-instruct-2507
219
+ bun run bench --list # 仅列出任务,无需引擎和密钥
220
+ bun run bench --suite paper # paper 套件:s_niah, oolong, browsecomp, codeqa_lb(需下载数据集)
221
+ ```
222
+
223
+ 套件:`all`(精简版,默认) · `needle` · `codeqa` · `coding` · `paper` · `s_niah` ·
224
+ `oolong` · `browsecomp` · `codeqa_lb`。
225
+
160
226
  ## 项目布局
161
227
 
162
228
  ```
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@hicaru/pi-rlm",
3
- "version": "0.3.14",
3
+ "version": "0.3.16",
4
4
  "author": "hicaru",
5
5
  "repository": {
6
6
  "type": "git",
@@ -56,9 +56,13 @@ export async function emitting<T>(
56
56
 
57
57
  let costUsd = 0;
58
58
  let tokens = 0;
59
+ let tokensIn = 0;
60
+ let tokensOut = 0;
59
61
  const track = (u: Usage): void => {
60
62
  costUsd += u.cost.total;
61
63
  tokens += u.totalTokens;
64
+ tokensIn += u.input;
65
+ tokensOut += u.output;
62
66
  };
63
67
 
64
68
  try {
@@ -70,6 +74,8 @@ export async function emitting<T>(
70
74
  resultPreview: summary.preview,
71
75
  costUsd,
72
76
  tokens,
77
+ tokensIn,
78
+ tokensOut,
73
79
  detail: summary.error,
74
80
  failedCount: summary.failed,
75
81
  totalCount: summary.total,
@@ -83,6 +89,8 @@ export async function emitting<T>(
83
89
  resultPreview: msg,
84
90
  costUsd,
85
91
  tokens,
92
+ tokensIn,
93
+ tokensOut,
86
94
  detail: msg,
87
95
  });
88
96
  throw err;
@@ -206,6 +206,8 @@ async function childRun(
206
206
  id: subId,
207
207
  status: "done",
208
208
  resultPreview: res.answer.slice(0, 200),
209
+ tokensIn: res.inputTokens,
210
+ tokensOut: res.outputTokens,
209
211
  });
210
212
  return res;
211
213
  } catch (err: unknown) {
@@ -9,6 +9,7 @@
9
9
  * RLM root (full history).
10
10
  */
11
11
 
12
+ import { clampThinkingLevel } from "@earendil-works/pi-ai";
12
13
  import { type Api, completeSimple, type Message, type Model, type ThinkingLevel, type Usage } from "@earendil-works/pi-ai/compat";
13
14
  import type { ModelRegistry } from "@earendil-works/pi-coding-agent";
14
15
  import { completeWithRetry, DEFAULT_RETRY_POLICY, type RetryPolicy } from "../util/retry.ts";
@@ -68,6 +69,19 @@ function toPiMessages(messages: readonly ChatMsg[], model: Model<Api>): { readon
68
69
  return { systemPrompt, messages: out };
69
70
  }
70
71
 
72
+ /**
73
+ * Explicit reasoning gate. pi-ai clamps unsupported levels too (a model with `reasoning:
74
+ * false` supports only "off", so any requested level clamps to it and nothing goes out on
75
+ * the wire) — owning the gate here makes the behavior engine policy, testable without a
76
+ * provider, and drops the option outright instead of forwarding a level the model cannot
77
+ * honor. Capability comes from the registry entry, never from config.
78
+ */
79
+ export function effectiveReasoning(model: Model<Api>, level: ThinkingLevel | undefined): ThinkingLevel | undefined {
80
+ if (level === undefined) return undefined;
81
+ const clamped = clampThinkingLevel(model, level);
82
+ return clamped === "off" ? undefined : clamped;
83
+ }
84
+
71
85
  /** Extract the assistant's plain text from a completion. */
72
86
  function extractText(content: readonly { readonly type: string; readonly text?: string }[]): string {
73
87
  return content
@@ -97,7 +111,7 @@ export async function modelComplete(messages: readonly ChatMsg[], opts: Complete
97
111
  headers: auth.headers,
98
112
  maxTokens: opts.maxTokens,
99
113
  temperature: opts.temperature,
100
- reasoning: opts.reasoning,
114
+ reasoning: effectiveReasoning(opts.model, opts.reasoning),
101
115
  signal: opts.signal,
102
116
  onResponse: (res) => { note(res.status, res.headers); },
103
117
  },
@@ -58,4 +58,7 @@ export const DEFAULT_CONFIG: Readonly<RlmConfig> = Object.freeze({
58
58
  memoryDir: null,
59
59
  // v5 role separation: children delegate (llm + memory/ledger); "legacy" = full child surface.
60
60
  childSurface: "delegation",
61
+ // Verification-discipline nudge — deliberately OFF (plan guardrail): when on, an early
62
+ // bare-number finalize gets one coached redo instead of being accepted. Opt-in via rlm.json.
63
+ enableVerificationNudge: false,
61
64
  });
@@ -38,9 +38,10 @@ function validateString(v: unknown): string | undefined {
38
38
  /**
39
39
  * Every value pi-ai accepts for `reasoning`. Keyed by the union so a new level added upstream
40
40
  * is a compile error here rather than a silently-rejected setting. Note `off` is NOT a
41
- * ThinkingLevel — a hand-edited rlm.json carrying one is dropped, not forwarded.
41
+ * ThinkingLevel — a hand-edited rlm.json carrying one is dropped, not forwarded. Exported so
42
+ * the config panel derives its choices from the same source of truth.
42
43
  */
43
- const THINKING_LEVELS: Readonly<Record<ThinkingLevel, true>> = Object.freeze({
44
+ export const THINKING_LEVELS: Readonly<Record<ThinkingLevel, true>> = Object.freeze({
44
45
  minimal: true, low: true, medium: true, high: true, xhigh: true, max: true,
45
46
  });
46
47
 
@@ -149,6 +150,9 @@ export function validateConfig(raw: unknown): Partial<RlmConfig> {
149
150
  }
150
151
  // v5 child surface doctrine
151
152
  if (r.childSurface === "delegation" || r.childSurface === "legacy") out.childSurface = r.childSurface;
153
+ // Verification-discipline nudge (default OFF).
154
+ const enableVerificationNudge = validateBoolean(r.enableVerificationNudge);
155
+ if (enableVerificationNudge !== undefined) out.enableVerificationNudge = enableVerificationNudge;
152
156
  if (typeof r.subSampling === "object" && r.subSampling !== null) {
153
157
  const ss = r.subSampling as Record<string, unknown>;
154
158
  const sampling: { maxTokens?: number; temperature?: number; reasoning?: ThinkingLevel } = {};
@@ -3,9 +3,12 @@
3
3
  import type { ReplResult } from "../sandbox/protocol.ts";
4
4
  import { formatReplStderr } from "../text/repl-output.ts";
5
5
 
6
- /** First non-null final answer across a turn's executed blocks, or null. */
6
+ /** First non-blank final answer across a turn's executed blocks, or null. A blank capture counts
7
+ * as absent (H2): an empty `answer.ready` flip must never terminate a run with "". */
7
8
  export function finalAnswerOf(results: readonly ReplResult[]): string | null {
8
- for (const r of results) if (r.finalAnswer != null) return r.finalAnswer;
9
+ for (const r of results) {
10
+ if (r.finalAnswer != null && r.finalAnswer.trim() !== "") return r.finalAnswer;
11
+ }
9
12
  return null;
10
13
  }
11
14
 
@@ -107,8 +107,30 @@ export class TokenBudget {
107
107
  }
108
108
 
109
109
  /** Cap derivation (v5 `resolve_budget`): share × context window, clamped by the task cap. */
110
+ /**
111
+ * Minimum context window (tokens) for the token-budget cascade to engage at all.
112
+ *
113
+ * The formula (window × budgetShare) assumes the window is large enough that a fraction of it
114
+ * is a meaningful working budget. Below this floor the derived cap shrinks below a task's FIXED
115
+ * overhead (system prompt + per-turn history re-send + sub-LLM calls) and strangles the run —
116
+ * a 32k window would cap a task at 8k tokens, less than the protocol scaffolding alone.
117
+ * So for smaller windows the rule does not apply: the budget is effectively unbounded and runs
118
+ * stay bounded by maxIterations / maxErrors / wall-clock instead.
119
+ */
120
+ export const BUDGET_WINDOW_FLOOR = 250_000;
121
+
122
+ /** An effective budget that can never trigger — the cascade "switched off" without changing
123
+ * any call-site types (budget: TokenBudget | undefined). */
124
+ function unboundedBudget(config: RlmConfig): TokenBudget {
125
+ return new TokenBudget(Number.MAX_SAFE_INTEGER, {
126
+ softFrac: config.budgetSoftFrac,
127
+ maxContinuations: config.budgetMaxContinuations,
128
+ });
129
+ }
130
+
110
131
  export function resolveBudget(contextWindow: number | undefined, config: RlmConfig): TokenBudget {
111
132
  const ctx = contextWindow !== undefined && contextWindow > 0 ? contextWindow : 32_000;
133
+ if (ctx < BUDGET_WINDOW_FLOOR) return unboundedBudget(config);
112
134
  const shareCap = Math.floor(ctx * config.budgetShare);
113
135
  const cap = config.budgetTaskCap > 0 ? Math.min(shareCap, config.budgetTaskCap) : shareCap;
114
136
  return new TokenBudget(Math.max(cap, 1), {