billion-context 0.1.39 → 0.1.41

package/README.md

@@ -431,7 +431,7 @@ The config file is a single JSON object. Example:
 | `debug` | `false` | Verbose logging (same as `ACP_DEBUG=1`) |
 | `passthrough` | `false` | Forward without compression (same as `ACP_PASSTHROUGH=1`) |
 | `providers` | *(none)* | Per-URL context overrides — see below |
-| `compress` | *(see defaults)* | `{ injectTool, injectNudge }` |
+| `compress` | *(see defaults)* | Global compression block: `{ injectTool, injectNudge }` injection toggles **plus** engine tuning (`nudgeGrowthTokens`, `modelContextLimit`, …) — see [Compression tuning](#compression-tuning-the-compress-block) |
 | `proxy` | *(none)* | Upstream HTTP proxy for the proxy's OWN outbound connections to model providers (`http://host:port`). Per-URL `proxy` overrides this. See [Upstream proxy](#upstream-proxy-firewall--gfw). |
 
 > **Choosing a `host`** (IPv6 / containers): the default `127.0.0.1` is
@@ -475,6 +475,61 @@ registry, then the built-in prefix table.
 > instead of 1M) causes spurious frequent compression. Declaring it per
 > URL + model makes the proxy match the registry the client itself uses.
 
+### Compression tuning (the `compress` block)
+
+The `compress` object tunes the compression engine itself — when to nudge, how much
+to compress per step, how many recent messages to protect. It is configurable
+at **three levels** that merge **per field, deepest wins** (child covers
+parent; an unset field at a deeper level never clears a value set higher up):
+
+1. **Global** — top-level `"compress": { … }` (applies to every request). This is
+   also where the `injectTool` / `injectNudge` toggles live (honored globally).
+2. **Per-provider** — `"compress": { … }` inside a `providers[url]` entry.
+3. **Per-model** — `"compress": { … }` inside a `providers[url].models[model]` entry.
+
+```jsonc
+{
+  // Level 1: global default for all providers/models
+  "compress": { "nudgeGrowthTokens": 50000, "maxContextLimit": "70%" },
+  "providers": {
+    "https://api.anthropic.com": {
+      // Level 2: override for this provider only
+      "compress": { "nudgeGrowthTokens": 30000, "preserveRecentMessages": 6 },
+      "models": {
+        "claude-opus-4": {
+          // Level 3: override for this one model (wins per-field)
+          "compress": { "nudgeGrowthTokens": 20000, "tiers": false }
+        }
+      }
+    }
+  }
+}
+```
+
+Fields (all optional; unset fields inherit the kernel default):
+
+| Field | Description |
+|-------|-------------|
+| `modelContextLimit` | The model's context **window size** — the denominator the kernel uses for its usage ratio (`usage = tokens / contextLimit`). **Not** a truncation cap. Accepts an absolute number (`200000`) or a percentage of the native window (`"70%"` → 140000 on a 200K model). When unset, defaults to the native window (built-in table / models.dev registry). ⚠️ Shrinking it pulls *every* ratio threshold (`emergencyThresholdPercent`, truncate) down with it — to leave headroom, set `emergencyThresholdPercent` instead. Highest-priority source for the model limit — overrides the built-in table, the models.dev registry, the legacy `modelContextLimit`, and per-model `context`. |
+| `maxContextLimit` | Usage ratio `0`–`1` at which compression is **forced** (nudge injected regardless of growth). Defaults to `0.75` (75% of the context window). Accepts a number (`0.75`) or percentage string (`"75%"`). Lower = compress earlier / more aggressively. |
+| `nudgeGrowthTokens` | Nudge growth step in tokens. A compression nudge fires roughly every time this many tokens become compressible. Flattens the adaptive band to a fixed step (default 50000 at 1M context). |
+| `emergencyThresholdPercent` | Usage ratio `0`–`1` at which compression becomes an **emergency** and tool outputs are hard-truncated to keep the session alive (default `0.95`). Accepts a number (`0.95`) or percentage string (`"95%"`). Must be ≥ `maxContextLimit`. |
+| `preserveRecentMessages` | Number of trailing messages never offered for compression. |
+| `preserveRecentTokens` | Token budget reserved for recent messages. |
+| `minCompressRange` | Minimum compressible range size in tokens; smaller ranges are skipped. |
+| `tiers` | Enable multi-tier (T2/T3) distillation (`true`/`false`). Promotion across tiers is driven by `nudgeGrowthTokens` (token accumulation), not a separate block-count knob. |
+
+The most common knobs are **`nudgeGrowthTokens`** (raise it to compress less often /
+delay compression) and **`modelContextLimit`** (pin an exact window the registry
+doesn't know). Everything else is for advanced tuning.
+
+**Injection toggles** (`injectTool`, `injectNudge` — global only, not per-level):
+
+| Toggle | Default | Effect |
+|--------|---------|--------|
+| `injectTool` | `true` | Injects the compress/decompress/search **tools** + the compress system prompt, so the model can trigger compression via a tool call. Disable to make compression fully automatic (no manual tool). Env `ACP_COMPRESS_TOOL=0`. |
+| `injectNudge` | `true` | Injects automatic **nudge** messages that prompt the model to compress when the context grows. Disable for tool-only / silent operation. Env `ACP_COMPRESS_NUDGE=0`. |
+
 ### URL key matching rules
 
 - A request matches a key when the client's embedded URL **equals the key or

package/README.zh-CN.md

@@ -310,7 +310,7 @@ bili --no-auto-update        # 本次启动禁用自动更新
 | `debug` | `false` | 详细日志(等同 `ACP_DEBUG=1`) |
 | `passthrough` | `false` | 不压缩直接转发(等同 `ACP_PASSTHROUGH=1`) |
 | `providers` | *(无)* | 按 URL 的 context 覆盖 —— 见下文 |
-| `compress` | *(见默认值)* | `{ injectTool, injectNudge }` |
+| `compress` | *(见默认值)* | 全局压缩块:`{ injectTool, injectNudge }` 注入开关 **外加** 引擎调参(`nudgeGrowthTokens`、`modelContextLimit` …)—— 见[压缩调参](#压缩调参compress-块) |
 | `proxy` | *(无)* | 代理自身访问模型提供商时走的上游 HTTP 代理(`http://host:port`)。按 URL 的 `proxy` 会覆盖它。见[上游代理](#上游代理防火墙gfw)。 |
 
 > **选择 `host`**(IPv6 / 容器):默认 `127.0.0.1` 只听 IPv4 且仅
@@ -345,6 +345,55 @@ key 就是客户端写在 `/bili/` 后面的那个字符串:
 
 > **为什么要声明 context?** LLM 的 `/models` API **不返回** context 窗口(已跨 OpenAI、Anthropic、智谱、comfly 验证)。它们是文档级信息。值错了(例如把 GLM-5.2 猜成 128K 而非 1M)会导致频繁误触发压缩。按 URL + 模型声明能让代理匹配客户端自己用的注册表。
 
+### 压缩调参(`compress` 块)
+
+`compress` 对象调的是压缩引擎本身 —— 何时 nudge、每步压多少、保护多少条最近消息。可在**三个层级**配置,**按字段深度优先合并**(子覆盖父;深层未设的字段不会清掉高层设的值):
+
+1. **全局** —— 顶层 `"compress": { … }`(对每个请求生效)。`injectTool` / `injectNudge` 开关也放这里(仅全局生效)。
+2. **按 provider** —— `providers[url]` 里的 `"compress": { … }`。
+3. **按模型** —— `providers[url].models[model]` 里的 `"compress": { … }`。
+
+```jsonc
+{
+  // 层级 1:所有 provider/模型的全局默认
+  "compress": { "nudgeGrowthTokens": 50000, "maxContextLimit": "70%" },
+  "providers": {
+    "https://api.anthropic.com": {
+      // 层级 2:仅此 provider 覆盖
+      "compress": { "nudgeGrowthTokens": 30000, "preserveRecentMessages": 6 },
+      "models": {
+        "claude-opus-4": {
+          // 层级 3:仅此模型覆盖(按字段胜出)
+          "compress": { "nudgeGrowthTokens": 20000, "tiers": false }
+        }
+      }
+    }
+  }
+}
+```
+
+字段(均可选;未设的字段继承内核默认值):
+
+| 字段 | 说明 |
+|-------|-------------|
+| `modelContextLimit` | 模型的 context **窗口大小** —— 内核算使用率时的分母(`usage = tokens / contextLimit`)。**不是**截断阈值。支持绝对值(`200000`)或相对原生窗口的百分比(`"70%"` → 200K 模型上 = 140000)。未设置时默认取原生窗口(内置表 / models.dev registry)。⚠️ 调小它会把**所有**按比例的阈值(`emergencyThresholdPercent`、截断)一起拉低 —— 想留余量请调 `emergencyThresholdPercent`。模型上限的最高优先级来源 —— 覆盖内置表、models.dev registry、旧版 `modelContextLimit` 以及按模型的 `context`。 |
+| `maxContextLimit` | 使用率 `0`–`1`,达到则**强制**压缩(无视增长,直接注入 nudge)。默认 `0.75`(窗口的 75%)。支持数字(`0.75`)或百分比字符串(`"75%"`)。调小 → 更早/更激进压缩。 |
+| `nudgeGrowthTokens` | nudge 增长步长(token)。大约每积累这么多可压缩 token 就触发一次压缩 nudge。把自适应区间拍平成固定步长(1M context 默认 50000)。 |
+| `emergencyThresholdPercent` | 使用率 `0`–`1`,达到则进入**紧急**模式,工具输出被硬截断以保住会话(默认 `0.95`)。支持数字(`0.95`)或百分比字符串(`"95%"`)。必须 ≥ `maxContextLimit`。 |
+| `preserveRecentMessages` | 永不纳入压缩的尾部消息条数。 |
+| `preserveRecentTokens` | 为最近消息预留的 token 预算。 |
+| `minCompressRange` | 可压缩范围的最小 token 数;更小的范围会被跳过。 |
+| `tiers` | 是否启用多层(T2/T3)蒸馏(`true`/`false`)。分层晋级由 `nudgeGrowthTokens`(token 累积)驱动,没有单独的块数门槛。 |
+
+最常用的两个旋钮是 **`nudgeGrowthTokens`**(调大 → 压得更少/更晚)和 **`modelContextLimit`**(钉死 registry 不认识的精确窗口)。其余皆为高级调参。
+
+**注入开关**(`injectTool`、`injectNudge` —— 仅全局,不分级):
+
+| 开关 | 默认 | 作用 |
+|--------|---------|--------|
+| `injectTool` | `true` | 注入 compress/decompress/search **工具** + 压缩系统提示,让模型能通过工具调用主动触发压缩。关闭后压缩完全自动化(无手动工具)。环境变量 `ACP_COMPRESS_TOOL=0`。 |
+| `injectNudge` | `true` | 注入自动 **nudge** 提示消息,在 context 增长时提醒模型压缩。关闭后只靠工具/静默运行。环境变量 `ACP_COMPRESS_NUDGE=0`。 |
+
 ### URL key 匹配规则
 
 - 一个请求在客户端嵌入的 URL **等于 key 或以 key 开头**时匹配(最长 key 优先)。