dsh-layered-memory 0.8.3 → 0.8.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.en.md +19 -15
- package/README.md +19 -15
- package/assets/readme/bench-dialog.svg +38 -49
- package/assets/readme/bench-workflow.svg +29 -29
- package/dist/client.js +32 -29
- package/dist/config.d.ts +7 -0
- package/dist/config.js +8 -1
- package/dist/llm.d.ts +8 -2
- package/dist/llm.js +14 -6
- package/dist/settings.d.ts +3 -3
- package/dist/settings.js +3 -3
- package/dist/stats.js +5 -3
- package/package.json +17 -17
package/README.en.md
CHANGED
|
@@ -10,7 +10,7 @@
|
|
|
10
10
|
[简体中文](README.md) · [Latest release](https://github.com/JunNanLYS/dsh-layered-memory/releases/latest) · [Report issues](https://github.com/JunNanLYS/dsh-layered-memory/issues)
|
|
11
11
|
|
|
12
12
|
[](https://www.npmjs.com/package/dsh-layered-memory)
|
|
13
|
-
[](https://github.com/deepseek-ai/deepseek-harness)
|
|
14
14
|
[](LICENSE)
|
|
15
15
|
|
|
16
16
|
</div>
|
|
@@ -144,39 +144,43 @@ trajectory view):
|
|
|
144
144
|
|
|
145
145
|
## Measured Comparison (DSH-MemBench: Automated Benchmark)
|
|
146
146
|
|
|
147
|
-
Screenshots show what the plugin looks like — this section answers "**what does enabling it actually buy you?**" with measured numbers from an **automated benchmark** ([`bench/`](./bench/), one command to reproduce). Method: the same scenario bank with verbatim-identical inputs runs in **Group A (memory on)** and **Group B (memory off)
|
|
147
|
+
Screenshots show what the plugin looks like — this section answers "**what does enabling it actually buy you?**" with measured numbers from an **automated benchmark** ([`bench/`](./bench/), one command to reproduce). Method: the same scenario bank with verbatim-identical inputs runs in **Group A (memory on)** with 3 merged repetitions and **Group B (memory off)** with 1 repetition (a memory-off long task burns multiples of the tokens per scenario — a cost guardrail); the dialog track now runs Group A only (memory-off probes in independent sessions cannot succeed, so the control carries no information — retired). Workflow-track environment: DeepSeek official `deepseek-v4-flash` (reasoning effort high), judge `glm-5.3`, plugin 0.8.3, Windows; taxonomy adapted from [LongMemEval](https://github.com/xiaowu0162/longmemeval) / [LoCoMo](https://snap-research.github.io/locomo/) / [AMB](https://github.com/vectorize-io/agent-memory-benchmark).
|
|
148
148
|
|
|
149
|
-
### Dialog track (15 scenarios × 6 probe types × 3 reps = 270 questions
|
|
149
|
+
### Dialog track (15 scenarios × 6 probe types × 3 reps = 270 questions): does it remember correctly
|
|
150
|
+
|
|
151
|
+
> Archived 0.8.0 baseline (Group A data; the dialog-track B arm has since been retired — Group A only).
|
|
150
152
|
|
|
151
153
|
<p align="center">
|
|
152
154
|
<img src="./assets/readme/bench-dialog.svg" width="100%"
|
|
153
|
-
alt="DSH-MemBench dialog track
|
|
155
|
+
alt="DSH-MemBench dialog track accuracy chart (Group A, memory on): overall accuracy 92.6% (250/270); per probe type, 45 each — extraction 45/45, multi-hop 45/45, temporal 43/45, knowledge updates 31/45, scene recall 41/45, abstention 45/45 with 0 fabricated; the dialog-track B arm is retired (memory-off probes in independent sessions cannot succeed)">
|
|
154
156
|
</p>
|
|
155
157
|
|
|
156
158
|
**Dual-channel recall** (Group A): passive injection hit rate **75.1%** (the answer's key points appear in the recall injection, 169/225); most of the rest the model recovered by **actively calling the memory tools** — 84 questions with active queries, **60 rescued by tools**. The end-to-end 92.6% is the composite of both channels plus model utilization. With the memory store accumulating across scenarios for the whole run, 144 probe injections carried other scenarios' memories (honestly counted) — yet overall accuracy held at 92.6%: interference resistance under a growing store, measured.
|
|
157
159
|
|
|
158
|
-
### Workflow track (
|
|
160
|
+
### Workflow track (7 scenarios · Group A ×3 / Group B ×1, real tool sandbox): does it do it right, and cheaper
|
|
159
161
|
|
|
160
162
|
<p align="center">
|
|
161
163
|
<img src="./assets/readme/bench-workflow.svg" width="100%"
|
|
162
|
-
alt="DSH-MemBench workflow track, Group A vs Group B:
|
|
164
|
+
alt="DSH-MemBench workflow track, Group A vs Group B: probe-phase completion A 59/69 (85.5%) vs B 10/23 (43.5%); cost comparison (Group B as the full-bar baseline, per-scenario means) — steps 24.3 vs 41.4 (B +70%), tool calls 37.7 vs 62.1 (B +65%), input tokens 266k vs 1.81M (B 6.8×); style-convention scenario probes A 12/12 vs B 0/4; long-task input tokens per scenario A 266k vs B 1.81M">
|
|
163
165
|
</p>
|
|
164
166
|
|
|
165
|
-
**
|
|
167
|
+
**Probe-phase completion 85.5% vs 43.5% (+42pp)**: both groups have live context during teach/change phases — the probe phase (continuation task in a fresh session) is the pure memory window. Group A scored a perfect 12/12 on all three new probe archetypes (workflow knowledge update / twin-runbook disambiguation / style-convention continuity), consistent across all three reps; Group B scored **0/4** on style-convention probes (naming/structure/thousands-separator/footer conventions exist only in memory — they cannot be explored out of the sandbox), while on the workflow-update scenario it can reverse-engineer the procedure by reading the script (discrimination limited by sandbox affordances, honestly noted).
|
|
168
|
+
|
|
169
|
+
**Long-task cost: Group B burns 6.8× Group A's input tokens per scenario** (1.81M vs 266k) — without memory the agent advances by re-exploring, and under a high reasoning effort it even builds its own projects to probe what a one-line script convention would have done; output tokens 3× (46.2k vs 15.4k), steps +70%. This is memory's core value: **what it saves is not task difficulty, but pointless round-trips and re-exploration**.
|
|
166
170
|
|
|
167
171
|
### Methodology & reproduction
|
|
168
172
|
|
|
169
173
|
```bash
|
|
170
|
-
node bench/harness/run.mjs --arm A --repeats 3 --provider deepseek-official --model deepseek-v4-flash
|
|
171
|
-
node bench/harness/run.mjs --arm
|
|
172
|
-
node bench/harness/run.mjs --track workflow --arm A/B --repeats 3 ... # workflow track
|
|
174
|
+
node bench/harness/run.mjs --arm A --repeats 3 --provider deepseek-official --model deepseek-v4-flash # dialog track (Group A only)
|
|
175
|
+
node bench/harness/run.mjs --track workflow --arm AB --repeats 3 ... # workflow track (A/B arms in parallel)
|
|
173
176
|
node bench/harness/report.mjs --latest [dialog|workflow] # aggregate report
|
|
174
177
|
```
|
|
175
178
|
|
|
176
|
-
- Scoring: programmatic `contains-all` plus an LLM judge against key points (every answer and verdict is preserved in `result.json` for human audit); workflow completion is verified programmatically from produced files and their contents;
|
|
177
|
-
-
|
|
178
|
-
-
|
|
179
|
-
-
|
|
179
|
+
- Scoring: programmatic `contains-all` plus an LLM judge against key points (every answer and verdict is preserved in `result.json` for human audit); workflow completion is verified programmatically from produced files and their contents (four check kinds: positive / forbidden-word / must-not-exist / exists);
|
|
180
|
+
- Live progress: running the benchmark auto-starts a local progress panel and opens the browser (`--no-panel` to disable) — per-arm scenario/phase/message-level progress, heartbeat & activity freshness (distinguishes "stuck" from "process died"), and cumulative cost as it accrues;
|
|
181
|
+
- Metrics come from provider-reported usage (input with cache-hit split) and session-event folding; the steady-state cache rate excludes each session's first request (archived 0.8.0 baseline: A 88.7% vs B 85.4% — memory injection does not hurt caching);
|
|
182
|
+
- Regression use: run before/after a plugin change and diff with `compare.mjs` (environment header check including git SHA + Group-B control-drift warning);
|
|
183
|
+
- Limitations (stated honestly): single machine; Group A ×3 merged, Group B ×1 (cost guardrail — noisier); judge vs tested model: same model in the archived dialog baseline, heterogeneous in the new workflow run (glm-5.3 judging v4-flash); the scenario bank is author-built (biased toward memory-advantage scenarios — reproduce it yourself); sandbox-file affordances partially leak procedures (Group B can reverse-engineer by reading scripts — discrimination limits honestly noted); dual-tier tool audit (strict violation voids the scenario / loose heuristic flags only), with 0 violations measured on both sides.
|
|
180
184
|
|
|
181
185
|
Full reports and per-question data: [`bench/baseline/`](./bench/baseline/).
|
|
182
186
|
|
|
@@ -235,7 +239,7 @@ the bundle layer appends and causes `duplicate loader entry id` startup failure)
|
|
|
235
239
|
| `embedding.proxy` | `''` | Three-state download proxy: `''` (default) = auto-detect proxy env vars (`HTTPS_PROXY`/`ALL_PROXY` etc., honoring `NO_PROXY`); `none` = disable, always direct; any other value = proxy URL (e.g. `http://127.0.0.1:7890`). Direct connections to the mirror are intermittently unreachable on some networks (connect timeouts and poisoned bytes have both been observed) — keep the default auto-detection on machines with a proxy |
|
|
236
240
|
| `llm.provider/model` | empty | Static distillation route (deployment pin): when **both** fields are set the route is locked, outranking the settings-page selection and the default model (deployments can force distillation onto a specific route); when empty the route follows "settings-page selection → default model". At runtime, switch among **configured providers** (including custom ones added in dsh Settings → Models) via the "distillation model" picker in Settings → Memory → Overview — effective immediately, no restart needed |
|
|
237
241
|
| `llm.maxTokens` | `65536` | Fallback output cap for non-layered calls. Each distillation stage has its own budget (extraction 16k / dedup 8k / L2 32k / L3 16k; auto ×4 when the reasoning effort is high/xhigh/max, so thinking can't starve the text budget); the per-layer budgets are runtime-adjustable in Settings → Memory → Overview → distillation parameters (empty/0 = built-in defaults) |
|
|
238
|
-
| `llm.reasoningEffort` | empty | Distillation reasoning effort: empty = **auto** (resolved from model capability: the model's default tier, else `high`); an explicit value (`off`/`none`/`low`/`medium`/`high`/`xhigh`/`max`) is only sent when the model declares support — effort vocabularies differ across providers (deepseek accepts `off`, OpenAI-style APIs use `none`, models that declare no tiers get nothing), and unsupported tiers degrade to not-sending with a one-time warning; output budgets auto-×4 at high/xhigh/max. Switchable at runtime in Settings → Memory → Overview; the selectable tiers follow the current model live |
|
|
242
|
+
| `llm.reasoningEffort` | empty | Distillation reasoning effort: empty = **auto** (resolved from model capability: the model's default tier, else `high`); an explicit value (`off`/`none`/`minimal`/`low`/`medium`/`high`/`xhigh`/`max`) is only sent when the model declares support — effort vocabularies differ across providers (deepseek accepts `off`, OpenAI-style APIs use `none`, models that declare no tiers get nothing), and unsupported tiers degrade to not-sending with a one-time warning; output budgets auto-×4 at high/xhigh/max. Switchable at runtime in Settings → Memory → Overview; the selectable tiers follow the current model live |
|
|
239
243
|
| `llm.temperature` | `0.3` | Distillation temperature |
|
|
240
244
|
| `llm.maxInputChars` | `700000` | Input character budget per distillation call (over-budget L1 inputs are chunked automatically); runtime-adjustable in Settings → distillation parameters → input budget (empty/0 = follow this value) |
|
|
241
245
|
| `llm.timeoutMs` | `120000` | Per-call distillation timeout (ms) |
|
package/README.md
CHANGED
|
@@ -10,7 +10,7 @@
|
|
|
10
10
|
[English](README.en.md) · [最新发行版](https://github.com/JunNanLYS/dsh-layered-memory/releases/latest) · [反馈问题](https://github.com/JunNanLYS/dsh-layered-memory/issues)
|
|
11
11
|
|
|
12
12
|
[](https://www.npmjs.com/package/dsh-layered-memory)
|
|
13
|
-
[](https://github.com/deepseek-ai/deepseek-harness)
|
|
14
14
|
[](LICENSE)
|
|
15
15
|
|
|
16
16
|
</div>
|
|
@@ -124,39 +124,43 @@ npx tsc src/smoke.ts --outDir dist-smoke --module nodenext --moduleResolution no
|
|
|
124
124
|
|
|
125
125
|
## 实测对比(DSH-MemBench:自动化基准)
|
|
126
126
|
|
|
127
|
-
图文回答"长什么样",这一节用**自动化基准**的实测数字回答"**开了到底有什么用**"([`bench/`](./bench/),一条命令可复现)。方法:同场景库、逐字相同输入,**A
|
|
127
|
+
图文回答"长什么样",这一节用**自动化基准**的实测数字回答"**开了到底有什么用**"([`bench/`](./bench/),一条命令可复现)。方法:同场景库、逐字相同输入,**A 组(记忆开)跑 3 次取合并值,B 组(记忆关)跑 1 次**(无记忆的长任务每场景要吞数倍 token,成本护栏);对话赛道只跑 A 组(B 组会话独立无记忆必然失败,对照无信息量,已下线)。工作流赛道环境:DeepSeek 官方 `deepseek-v4-flash`(思考档 high)、判卷 `glm-5.3`、插件 0.8.3、Windows;题型设计借鉴 [LongMemEval](https://github.com/xiaowu0162/longmemeval) / [LoCoMo](https://snap-research.github.io/locomo/) / [AMB](https://github.com/vectorize-io/agent-memory-benchmark)。
|
|
128
128
|
|
|
129
|
-
### 对话赛道(15 场景 × 6 题型 × 3 次 = 270
|
|
129
|
+
### 对话赛道(15 场景 × 6 题型 × 3 次 = 270 题):答得准吗
|
|
130
|
+
|
|
131
|
+
> 0.8.0 留档基线(A 组数据;此后对话赛道 B 组下线,只跑 A 组)。
|
|
130
132
|
|
|
131
133
|
<p align="center">
|
|
132
134
|
<img src="./assets/readme/bench-dialog.svg" width="100%"
|
|
133
|
-
alt="DSH-MemBench
|
|
135
|
+
alt="DSH-MemBench 对话赛道准确率图(A 组·记忆开):总准确率 92.6%(250/270);分题型各 45 题——抽取 45/45、多跳 45/45、时序 43/45、知识更新 31/45、场景回忆 41/45、拒答 45/45 且 0 编造;对话赛道 B 组已下线(会话独立无记忆必然失败)">
|
|
134
136
|
</p>
|
|
135
137
|
|
|
136
138
|
**召回双通道**(A 组):被动注入召回率 **75.1%**(该题要点出现在召回注入中,169/225),其余多数由模型**主动调用记忆工具**查回——84 题主动查询、**60 题靠工具兜底答对**;端到端 92.6% 是两通道 + 模型利用的合成结果。记忆库跨场景全程累积下,探针召回注入混入其他场景记忆 144 次(已如实计数),总准确率仍稳在 92.6%——抗干扰能力经受住了膨胀记忆库的考验。
|
|
137
139
|
|
|
138
|
-
### 工作流赛道(
|
|
140
|
+
### 工作流赛道(7 场景 · A 组 3 次 / B 组 1 次,真实工具沙箱):做得对、做得省吗
|
|
139
141
|
|
|
140
142
|
<p align="center">
|
|
141
143
|
<img src="./assets/readme/bench-workflow.svg" width="100%"
|
|
142
|
-
alt="DSH-MemBench 工作流赛道 A/B
|
|
144
|
+
alt="DSH-MemBench 工作流赛道 A/B 对照图:探针段完成度 A 组 59/69(85.5%)对 B 组 10/23(43.5%);成本对比(B 组为满格基准,每场景均值)——步骤 24.3 对 41.4(B +70%)、工具调用 37.7 对 62.1(B +65%)、输入 token 266k 对 1.81M(B 6.8 倍);风格规范场景探针 A 12/12 对 B 0/4;长任务每场景输入 token A 266k 对 B 1.81M">
|
|
143
145
|
</p>
|
|
144
146
|
|
|
145
|
-
|
|
147
|
+
**探针段完成度 85.5% vs 43.5%(+42pp)**:教学/变更段两组都有现场上下文,探针段(新会话延续任务)才是纯记忆窗口——A 组三个新考法场景(流程知识更新 / 双胞胎消歧 / 风格规范延续)全部 12/12 满分且三轮一致;B 组在风格规范场景探针 **0/4**(命名/结构/千分位/页脚约定只存记忆,沙箱探不出来),在流程更新场景则能靠读脚本逆向(判别力受沙箱可供性限制,已如实标注)。
|
|
148
|
+
|
|
149
|
+
**长任务成本:B 组每场景输入 token 是 A 组的 6.8 倍**(1.81M vs 266k)——无记忆时 agent 靠重新探索前进,high 思考档下甚至会自建工程去探测本可用一条脚本约定完成的流程;输出 token 3 倍(46.2k vs 15.4k)、步骤 +70%。这正是记忆的核心价值:**省掉的不是任务难度,是无谓的往返与重复探索**。
|
|
146
150
|
|
|
147
151
|
### 方法论与复现
|
|
148
152
|
|
|
149
153
|
```bash
|
|
150
|
-
node bench/harness/run.mjs --arm A --repeats 3 --provider deepseek-official --model deepseek-v4-flash
|
|
151
|
-
node bench/harness/run.mjs --arm
|
|
152
|
-
node bench/harness/run.mjs --track workflow --arm A/B --repeats 3 ... # 工作流赛道
|
|
154
|
+
node bench/harness/run.mjs --arm A --repeats 3 --provider deepseek-official --model deepseek-v4-flash # 对话赛道(只跑 A 组)
|
|
155
|
+
node bench/harness/run.mjs --track workflow --arm AB --repeats 3 ... # 工作流赛道(A/B 双组并行)
|
|
153
156
|
node bench/harness/report.mjs --latest [dialog|workflow] # 汇总报告
|
|
154
157
|
```
|
|
155
158
|
|
|
156
|
-
- 判分:`contains-all` 程序判 + 判卷模型按要点判(答案原文与判分理由全部留痕 `result.json` 可人工复核);工作流完成度为产物文件 +
|
|
157
|
-
-
|
|
158
|
-
-
|
|
159
|
-
-
|
|
159
|
+
- 判分:`contains-all` 程序判 + 判卷模型按要点判(答案原文与判分理由全部留痕 `result.json` 可人工复核);工作流完成度为产物文件 + 关键内容程序化校验(四型判据:正检查/禁词/产物缺席/存在性);
|
|
160
|
+
- 实时进度:跑基准时自动拉起本地进度面板并打开浏览器(`--no-panel` 关闭)——A/B 双臂场景/阶段/消息粒度进度、心跳与活动新鲜度(直判"卡住 vs 进程挂了")、累计成本随跑随涨;
|
|
161
|
+
- 指标全部来自供应商上报 usage(输入含缓存命中拆分)与会话事件折叠;稳态缓存率剔除每会话首请求(0.8.0 留档基线:A 88.7% vs B 85.4%——记忆注入不伤缓存);
|
|
162
|
+
- 回归用途:改插件前后各跑一遍,`compare.mjs` 出对比表(环境头校验含 gitSha + B 组对照组漂移告警);
|
|
163
|
+
- 局限(诚实声明):单机;A 组 ×3 合并、B 组 ×1(成本护栏,噪声更大);判卷与被测模型:对话留档基线同源、工作流新跑为异构(glm-5.3 判 v4-flash);作者自建场景库(倾向记忆优势场景,欢迎自行复现);沙箱文件的可供性会部分泄露流程(B 组可读脚本逆向,判别力受限处已如实标注);工具审计双档(严格违规判负/宽松提示),实测双方 0 违规。
|
|
160
164
|
|
|
161
165
|
完整报告与逐题数据:[`bench/baseline/`](./bench/baseline/)。
|
|
162
166
|
|
|
@@ -252,7 +256,7 @@ ONNX 量化 **CPU 推理**——无需 API Key,数据不出本机)。本地
|
|
|
252
256
|
| `embedding.proxy` | `''` | 模型下载代理三态:`''`(默认)= 自动探测代理环境变量(`HTTPS_PROXY`/`ALL_PROXY` 等,尊重 `NO_PROXY`);`none` = 禁用强制直连;其他值 = 代理 URL(如 `http://127.0.0.1:7890`)。镜像直连在国内网络间歇不可达(直连超时与污染字节交替出现过),开代理的机器建议保持默认自动探测 |
|
|
253
257
|
| `llm.provider/model` | 空 | 蒸馏模型静态路由(部署 pin):provider 与 model **双字段齐**时锁定蒸馏路由,优先于设置页的运行时选择与默认模型(部署可强制蒸馏走指定路由);留空则跟随"设置页选择 → 默认模型"。运行时可在设置页 → 记忆 → 概览的"蒸馏模型"选择器从**已配置的供应商**(含 dsh 设置 → 模型里添加的自定义供应商)中切换,即时生效无需重启 |
|
|
254
258
|
| `llm.maxTokens` | `65536` | 未分层调用的兜底输出总闸。各蒸馏层有独立预算(抽取 16k / 去重 8k / L2 32k / L3 16k;思考档 high/xhigh/max 时自动 ×4,防 reasoning 吃光预算),分层预算可在设置页 → 记忆 → 概览 → 蒸馏参数运行时调整(留空/0 = 跟随内置默认) |
|
|
255
|
-
| `llm.reasoningEffort` | 空 | 蒸馏思考档位:空串 = **自动**(按模型能力解析:模型默认档 → `high`);显式值(`off`/`none`/`low`/`medium`/`high`/`xhigh`/`max`)仅在该模型声明支持时发送——跨供应商 effort 词汇表不同(deepseek 认 `off`,OpenAI 系是 `none`,未声明档位的模型不传),不支持的档位自动降级为不传并告警一次;思考档 high/xhigh/max 时输出预算自动 ×4。运行时在设置页 → 记忆 → 概览切换,可选档位表跟随当前模型实时显示 |
|
|
259
|
+
| `llm.reasoningEffort` | 空 | 蒸馏思考档位:空串 = **自动**(按模型能力解析:模型默认档 → `high`);显式值(`off`/`none`/`minimal`/`low`/`medium`/`high`/`xhigh`/`max`)仅在该模型声明支持时发送——跨供应商 effort 词汇表不同(deepseek 认 `off`,OpenAI 系是 `none`,未声明档位的模型不传),不支持的档位自动降级为不传并告警一次;思考档 high/xhigh/max 时输出预算自动 ×4。运行时在设置页 → 记忆 → 概览切换,可选档位表跟随当前模型实时显示 |
|
|
256
260
|
| `llm.temperature` | `0.3` | 蒸馏温度 |
|
|
257
261
|
| `llm.maxInputChars` | `700000` | 单次蒸馏输入字符预算(超限的 L1 输入自动分块抽取);运行时可在设置页 → 蒸馏参数 → 输入预算调整(留空/0 = 跟随本值) |
|
|
258
262
|
| `llm.timeoutMs` | `120000` | 单次蒸馏调用超时(ms) |
|
|
@@ -1,87 +1,76 @@
|
|
|
1
1
|
<svg xmlns="http://www.w3.org/2000/svg"
|
|
2
|
-
width="1200" height="
|
|
2
|
+
width="1200" height="630" viewBox="0 0 1200 630"
|
|
3
3
|
role="img" aria-labelledby="benchDialogTitle benchDialogDesc">
|
|
4
|
-
<title id="benchDialogTitle">DSH-MemBench 对话赛道:A
|
|
5
|
-
<desc id="benchDialogDesc"
|
|
4
|
+
<title id="benchDialogTitle">DSH-MemBench 对话赛道:A 组(记忆开)准确率</title>
|
|
5
|
+
<desc id="benchDialogDesc">A 组(记忆开)总准确率 92.6%(250/270,15 场景 × 6 题型 × 3 次)。分题型(每题型 45 题):抽取 45/45;多跳 45/45;时序 43/45;知识更新 31/45;场景回忆 41/45;拒答 45/45、0 编造。对话赛道 B 组已下线(会话独立、无记忆必然失败,对照无信息量)。</desc>
|
|
6
6
|
|
|
7
|
-
<rect width="1200" height="
|
|
7
|
+
<rect width="1200" height="630" rx="26" fill="#0D1526"/>
|
|
8
8
|
|
|
9
9
|
<g font-family="-apple-system,BlinkMacSystemFont,'Segoe UI','PingFang SC','Microsoft YaHei',sans-serif">
|
|
10
10
|
|
|
11
11
|
<!-- ── 标题与图例 ── -->
|
|
12
|
-
<text x="64" y="66" font-size="26" font-weight="700" fill="#EFF3FA"
|
|
13
|
-
<text x="64" y="94" font-size="18" fill="#8296B3">15 场景 × 6 题型 × 3 次 = 270
|
|
12
|
+
<text x="64" y="66" font-size="26" font-weight="700" fill="#EFF3FA">对话赛道:记忆开 · 准确率</text>
|
|
13
|
+
<text x="64" y="94" font-size="18" fill="#8296B3">15 场景 × 6 题型 × 3 次 = 270 题(每题型 45 题)· 条长 = 答对率,数值 = 答对题数</text>
|
|
14
14
|
|
|
15
15
|
<rect x="842" y="50" width="18" height="18" rx="5" fill="#FFD15D"/>
|
|
16
16
|
<text x="868" y="65" font-size="18" fill="#C2CDDE">A 组 · 记忆开</text>
|
|
17
|
-
<rect x="842" y="76" width="18" height="18" rx="5" fill="#4E6DA8"/>
|
|
18
|
-
<text x="868" y="91" font-size="18" fill="#C2CDDE">B 组 · 记忆关</text>
|
|
19
17
|
|
|
20
18
|
<!-- ── 网格线(0/25/50/75/100%)── -->
|
|
21
19
|
<g stroke="#1C2A47" stroke-width="1">
|
|
22
|
-
<line x1="260" y1="122" x2="260" y2="
|
|
23
|
-
<line x1="435" y1="122" x2="435" y2="
|
|
24
|
-
<line x1="610" y1="122" x2="610" y2="
|
|
25
|
-
<line x1="785" y1="122" x2="785" y2="
|
|
26
|
-
<line x1="960" y1="122" x2="960" y2="
|
|
20
|
+
<line x1="260" y1="122" x2="260" y2="530"/>
|
|
21
|
+
<line x1="435" y1="122" x2="435" y2="530"/>
|
|
22
|
+
<line x1="610" y1="122" x2="610" y2="530"/>
|
|
23
|
+
<line x1="785" y1="122" x2="785" y2="530"/>
|
|
24
|
+
<line x1="960" y1="122" x2="960" y2="530"/>
|
|
27
25
|
</g>
|
|
28
26
|
<g font-size="17" fill="#8296B3" text-anchor="middle">
|
|
29
|
-
<text x="260" y="
|
|
30
|
-
<text x="435" y="
|
|
31
|
-
<text x="610" y="
|
|
32
|
-
<text x="785" y="
|
|
33
|
-
<text x="960" y="
|
|
27
|
+
<text x="260" y="558">0%</text>
|
|
28
|
+
<text x="435" y="558">25%</text>
|
|
29
|
+
<text x="610" y="558">50%</text>
|
|
30
|
+
<text x="785" y="558">75%</text>
|
|
31
|
+
<text x="960" y="558">100%</text>
|
|
34
32
|
</g>
|
|
35
33
|
|
|
36
34
|
<!-- ── 总准确率(强调行)── -->
|
|
37
|
-
<text x="240" y="
|
|
35
|
+
<text x="240" y="169" font-size="21" font-weight="700" fill="#EFF3FA" text-anchor="end">总准确率</text>
|
|
38
36
|
<rect x="260" y="148" width="648" height="26" rx="6" fill="#FFD15D"/>
|
|
39
37
|
<text x="922" y="169" font-size="22" font-weight="700" fill="#FFD15D">92.6%(250/270)</text>
|
|
40
|
-
<rect x="260" y="182" width="124" height="26" rx="6" fill="#4E6DA8"/>
|
|
41
|
-
<text x="396" y="203" font-size="22" font-weight="700" fill="#8FA6C6">17.8%(48/270)</text>
|
|
42
38
|
|
|
43
|
-
<line x1="64" y1="
|
|
39
|
+
<line x1="64" y1="200" x2="1136" y2="200" stroke="#263A5C" stroke-width="1.5"/>
|
|
44
40
|
|
|
45
41
|
<!-- ── 分题型行 ── -->
|
|
46
42
|
<!-- 抽取 -->
|
|
47
|
-
<text x="240" y="
|
|
48
|
-
<rect x="260" y="
|
|
49
|
-
<text x="974" y="
|
|
50
|
-
<rect x="260" y="278" width="47" height="18" rx="5" fill="#4E6DA8"/>
|
|
51
|
-
<text x="319" y="293" font-size="18" font-weight="700" fill="#8FA6C6" font-family="ui-monospace,SFMono-Regular,Menlo,Consolas,monospace">3/45</text>
|
|
43
|
+
<text x="240" y="249" font-size="19" fill="#C2CDDE" text-anchor="end">抽取</text>
|
|
44
|
+
<rect x="260" y="234" width="700" height="18" rx="5" fill="#FFD15D"/>
|
|
45
|
+
<text x="974" y="249" font-size="18" font-weight="700" fill="#FFD15D" font-family="ui-monospace,SFMono-Regular,Menlo,Consolas,monospace">45/45</text>
|
|
52
46
|
|
|
53
47
|
<!-- 多跳 -->
|
|
54
|
-
<text x="240" y="
|
|
55
|
-
<rect x="260" y="
|
|
56
|
-
<text x="974" y="
|
|
57
|
-
<text x="272" y="355" font-size="18" font-weight="700" fill="#8FA6C6" font-family="ui-monospace,SFMono-Regular,Menlo,Consolas,monospace">0/45</text>
|
|
48
|
+
<text x="240" y="303" font-size="19" fill="#C2CDDE" text-anchor="end">多跳</text>
|
|
49
|
+
<rect x="260" y="288" width="700" height="18" rx="5" fill="#FFD15D"/>
|
|
50
|
+
<text x="974" y="303" font-size="18" font-weight="700" fill="#FFD15D" font-family="ui-monospace,SFMono-Regular,Menlo,Consolas,monospace">45/45</text>
|
|
58
51
|
|
|
59
52
|
<!-- 时序 -->
|
|
60
|
-
<text x="240" y="
|
|
61
|
-
<rect x="260" y="
|
|
62
|
-
<text x="943" y="
|
|
63
|
-
<text x="272" y="417" font-size="18" font-weight="700" fill="#8FA6C6" font-family="ui-monospace,SFMono-Regular,Menlo,Consolas,monospace">0/45</text>
|
|
53
|
+
<text x="240" y="357" font-size="19" fill="#C2CDDE" text-anchor="end">时序</text>
|
|
54
|
+
<rect x="260" y="342" width="669" height="18" rx="5" fill="#FFD15D"/>
|
|
55
|
+
<text x="943" y="357" font-size="18" font-weight="700" fill="#FFD15D" font-family="ui-monospace,SFMono-Regular,Menlo,Consolas,monospace">43/45</text>
|
|
64
56
|
|
|
65
57
|
<!-- 知识更新 -->
|
|
66
|
-
<text x="240" y="
|
|
67
|
-
<rect x="260" y="
|
|
68
|
-
<text x="756" y="
|
|
69
|
-
<text x="272" y="479" font-size="18" font-weight="700" fill="#8FA6C6" font-family="ui-monospace,SFMono-Regular,Menlo,Consolas,monospace">0/45</text>
|
|
58
|
+
<text x="240" y="411" font-size="19" fill="#C2CDDE" text-anchor="end">知识更新 *</text>
|
|
59
|
+
<rect x="260" y="396" width="482" height="18" rx="5" fill="#FFD15D"/>
|
|
60
|
+
<text x="756" y="411" font-size="18" font-weight="700" fill="#FFD15D" font-family="ui-monospace,SFMono-Regular,Menlo,Consolas,monospace">31/45</text>
|
|
70
61
|
|
|
71
62
|
<!-- 场景回忆 -->
|
|
72
|
-
<text x="240" y="
|
|
73
|
-
<rect x="260" y="
|
|
74
|
-
<text x="912" y="
|
|
75
|
-
<text x="272" y="541" font-size="18" font-weight="700" fill="#8FA6C6" font-family="ui-monospace,SFMono-Regular,Menlo,Consolas,monospace">0/45</text>
|
|
63
|
+
<text x="240" y="465" font-size="19" fill="#C2CDDE" text-anchor="end">场景回忆</text>
|
|
64
|
+
<rect x="260" y="450" width="638" height="18" rx="5" fill="#FFD15D"/>
|
|
65
|
+
<text x="912" y="465" font-size="18" font-weight="700" fill="#FFD15D" font-family="ui-monospace,SFMono-Regular,Menlo,Consolas,monospace">41/45</text>
|
|
76
66
|
|
|
77
67
|
<!-- 拒答 -->
|
|
78
|
-
<text x="240" y="
|
|
79
|
-
<rect x="260" y="
|
|
80
|
-
<text x="974" y="
|
|
81
|
-
<rect x="260" y="588" width="700" height="18" rx="5" fill="#4E6DA8"/>
|
|
82
|
-
<text x="974" y="603" font-size="18" font-weight="700" fill="#8FA6C6" font-family="ui-monospace,SFMono-Regular,Menlo,Consolas,monospace">45/45</text>
|
|
68
|
+
<text x="240" y="519" font-size="19" fill="#C2CDDE" text-anchor="end">拒答 *</text>
|
|
69
|
+
<rect x="260" y="504" width="700" height="18" rx="5" fill="#FFD15D"/>
|
|
70
|
+
<text x="974" y="519" font-size="18" font-weight="700" fill="#FFD15D" font-family="ui-monospace,SFMono-Regular,Menlo,Consolas,monospace">45/45</text>
|
|
83
71
|
|
|
84
72
|
<!-- ── 脚注 ── -->
|
|
85
|
-
<text x="64" y="
|
|
73
|
+
<text x="64" y="600" font-size="16" fill="#8296B3">* 知识更新:改口后仍答旧值记 0 分(直接考核 L1 去重更新);拒答:没发生过的事编造即 0 分(0 编造)。</text>
|
|
74
|
+
<text x="64" y="622" font-size="16" fill="#8296B3">对话赛道 B 组已下线:Harness 会话彼此独立,无记忆的探针必然失败,对照无信息量。</text>
|
|
86
75
|
</g>
|
|
87
76
|
</svg>
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
width="1200" height="660" viewBox="0 0 1200 660"
|
|
3
3
|
role="img" aria-labelledby="benchWorkflowTitle benchWorkflowDesc">
|
|
4
4
|
<title id="benchWorkflowTitle">DSH-MemBench 工作流赛道:A/B 两组完成度与成本对照</title>
|
|
5
|
-
<desc id="benchWorkflowDesc">真实工具沙箱(
|
|
5
|
+
<desc id="benchWorkflowDesc">真实工具沙箱(7 场景,A 组 3 轮 / B 组 1 轮)探针段合计:任务完成度 A 组 59/69(85.5%)对 B 组 10/23(43.5%);成本对比以 B 组为满格基准(每场景均值),A 组相对更省——步骤 24.3 对 41.4(B +70%)、工具调用 37.7 对 62.1(B +65%)、输入 token 266k 对 1.81M(B 6.8 倍);风格规范场景探针 A 12/12 对 B 0/4;向用户求助 A 组 3 轮共 1 次、B 组 1 轮 1 次。</desc>
|
|
6
6
|
|
|
7
7
|
<rect width="1200" height="660" rx="26" fill="#0D1526"/>
|
|
8
8
|
|
|
@@ -10,70 +10,70 @@
|
|
|
10
10
|
|
|
11
11
|
<!-- ── 标题与图例 ── -->
|
|
12
12
|
<text x="64" y="66" font-size="26" font-weight="700" fill="#EFF3FA">工作流赛道:真实工具沙箱 · 做得对,也更省</text>
|
|
13
|
-
<text x="64" y="94" font-size="18" fill="#8296B3">
|
|
13
|
+
<text x="64" y="94" font-size="18" fill="#8296B3">7 场景 · A 组 3 轮 / B 组 1 轮(成本护栏)· 探针段合计 · 成本行以 B 组为满格基准</text>
|
|
14
14
|
|
|
15
15
|
<rect x="842" y="50" width="18" height="18" rx="5" fill="#FFD15D"/>
|
|
16
16
|
<text x="868" y="65" font-size="18" fill="#C2CDDE">A 组 · 记忆开</text>
|
|
17
17
|
<rect x="842" y="76" width="18" height="18" rx="5" fill="#4E6DA8"/>
|
|
18
18
|
<text x="868" y="91" font-size="18" fill="#C2CDDE">B 组 · 记忆关</text>
|
|
19
19
|
|
|
20
|
-
<!-- ──
|
|
21
|
-
<text x="64" y="152" font-size="20" font-weight="700" fill="#EFF3FA"
|
|
22
|
-
<text x="
|
|
20
|
+
<!-- ── 探针段完成度 ── -->
|
|
21
|
+
<text x="64" y="152" font-size="20" font-weight="700" fill="#EFF3FA">探针段完成度</text>
|
|
22
|
+
<text x="216" y="152" font-size="17" fill="#8296B3">(新会话延续任务,产物程序化校验)</text>
|
|
23
23
|
|
|
24
|
-
<rect x="260" y="170" width="
|
|
25
|
-
<text x="
|
|
26
|
-
<rect x="260" y="206" width="
|
|
27
|
-
<text x="
|
|
24
|
+
<rect x="260" y="170" width="598" height="26" rx="6" fill="#FFD15D"/>
|
|
25
|
+
<text x="872" y="191" font-size="22" font-weight="700" fill="#FFD15D">59/69 · 85.5%</text>
|
|
26
|
+
<rect x="260" y="206" width="304" height="26" rx="6" fill="#4E6DA8"/>
|
|
27
|
+
<text x="578" y="227" font-size="22" font-weight="700" fill="#8FA6C6">10/23 · 43.5%</text>
|
|
28
28
|
|
|
29
29
|
<line x1="64" y1="258" x2="1136" y2="258" stroke="#263A5C" stroke-width="1.5"/>
|
|
30
30
|
|
|
31
31
|
<!-- ── 成本对比(B = 满格基准)── -->
|
|
32
32
|
<text x="64" y="290" font-size="20" font-weight="700" fill="#EFF3FA">成本对比</text>
|
|
33
|
-
<text x="166" y="290" font-size="17" fill="#8296B3"
|
|
33
|
+
<text x="166" y="290" font-size="17" fill="#8296B3">(每场景均值,B 组 = 满格基准,条越短越省)</text>
|
|
34
34
|
|
|
35
35
|
<!-- 步骤 -->
|
|
36
36
|
<text x="240" y="348" font-size="19" fill="#C2CDDE" text-anchor="end">步骤</text>
|
|
37
|
-
<rect x="260" y="318" width="
|
|
38
|
-
<text x="
|
|
37
|
+
<rect x="260" y="318" width="329" height="18" rx="5" fill="#FFD15D"/>
|
|
38
|
+
<text x="603" y="333" font-size="18" font-weight="700" fill="#FFD15D" font-family="ui-monospace,SFMono-Regular,Menlo,Consolas,monospace">24.3</text>
|
|
39
39
|
<rect x="260" y="342" width="560" height="18" rx="5" fill="#4E6DA8"/>
|
|
40
|
-
<text x="834" y="357" font-size="18" font-weight="700" fill="#8FA6C6" font-family="ui-monospace,SFMono-Regular,Menlo,Consolas,monospace">
|
|
40
|
+
<text x="834" y="357" font-size="18" font-weight="700" fill="#8FA6C6" font-family="ui-monospace,SFMono-Regular,Menlo,Consolas,monospace">41.4</text>
|
|
41
41
|
<rect x="980" y="321" width="136" height="36" rx="10" fill="#315DA6"/>
|
|
42
|
-
<text x="1048" y="345" font-size="19" font-weight="700" fill="#FFD15D" text-anchor="middle">B +
|
|
42
|
+
<text x="1048" y="345" font-size="19" font-weight="700" fill="#FFD15D" text-anchor="middle">B +70%</text>
|
|
43
43
|
|
|
44
44
|
<!-- 工具调用 -->
|
|
45
45
|
<text x="240" y="430" font-size="19" fill="#C2CDDE" text-anchor="end">工具调用</text>
|
|
46
|
-
<rect x="260" y="400" width="
|
|
47
|
-
<text x="
|
|
46
|
+
<rect x="260" y="400" width="340" height="18" rx="5" fill="#FFD15D"/>
|
|
47
|
+
<text x="614" y="415" font-size="18" font-weight="700" fill="#FFD15D" font-family="ui-monospace,SFMono-Regular,Menlo,Consolas,monospace">37.7</text>
|
|
48
48
|
<rect x="260" y="424" width="560" height="18" rx="5" fill="#4E6DA8"/>
|
|
49
|
-
<text x="834" y="439" font-size="18" font-weight="700" fill="#8FA6C6" font-family="ui-monospace,SFMono-Regular,Menlo,Consolas,monospace">
|
|
49
|
+
<text x="834" y="439" font-size="18" font-weight="700" fill="#8FA6C6" font-family="ui-monospace,SFMono-Regular,Menlo,Consolas,monospace">62.1</text>
|
|
50
50
|
<rect x="980" y="403" width="136" height="36" rx="10" fill="#315DA6"/>
|
|
51
|
-
<text x="1048" y="427" font-size="19" font-weight="700" fill="#FFD15D" text-anchor="middle">B +
|
|
51
|
+
<text x="1048" y="427" font-size="19" font-weight="700" fill="#FFD15D" text-anchor="middle">B +65%</text>
|
|
52
52
|
|
|
53
53
|
<!-- 输入 token -->
|
|
54
54
|
<text x="240" y="512" font-size="19" fill="#C2CDDE" text-anchor="end">输入 token</text>
|
|
55
|
-
<rect x="260" y="482" width="
|
|
56
|
-
<text x="
|
|
55
|
+
<rect x="260" y="482" width="82" height="18" rx="5" fill="#FFD15D"/>
|
|
56
|
+
<text x="356" y="497" font-size="18" font-weight="700" fill="#FFD15D" font-family="ui-monospace,SFMono-Regular,Menlo,Consolas,monospace">266k</text>
|
|
57
57
|
<rect x="260" y="506" width="560" height="18" rx="5" fill="#4E6DA8"/>
|
|
58
|
-
<text x="834" y="521" font-size="18" font-weight="700" fill="#8FA6C6" font-family="ui-monospace,SFMono-Regular,Menlo,Consolas,monospace">1.
|
|
58
|
+
<text x="834" y="521" font-size="18" font-weight="700" fill="#8FA6C6" font-family="ui-monospace,SFMono-Regular,Menlo,Consolas,monospace">1.81M</text>
|
|
59
59
|
<rect x="980" y="485" width="136" height="36" rx="10" fill="#315DA6"/>
|
|
60
|
-
<text x="1048" y="509" font-size="19" font-weight="700" fill="#FFD15D" text-anchor="middle">B +
|
|
60
|
+
<text x="1048" y="509" font-size="19" font-weight="700" fill="#FFD15D" text-anchor="middle">B +580%</text>
|
|
61
61
|
|
|
62
62
|
<!-- ── 底部特写卡片 ── -->
|
|
63
63
|
<rect x="64" y="544" width="512" height="72" rx="14" fill="#15223A" stroke="#263A5C" stroke-width="1.5"/>
|
|
64
|
-
<text x="88" y="574" font-size="17" fill="#8296B3"
|
|
64
|
+
<text x="88" y="574" font-size="17" fill="#8296B3">风格规范场景 · 探针段(约定只存记忆,沙箱探不出来)</text>
|
|
65
65
|
<text x="88" y="600" font-size="20" font-weight="700">
|
|
66
|
-
<tspan fill="#FFD15D">A
|
|
67
|
-
<tspan fill="#8FA6C6"> · B
|
|
66
|
+
<tspan fill="#FFD15D">A 12/12</tspan>
|
|
67
|
+
<tspan fill="#8FA6C6"> · B 0/4</tspan>
|
|
68
68
|
</text>
|
|
69
69
|
|
|
70
70
|
<rect x="624" y="544" width="512" height="72" rx="14" fill="#15223A" stroke="#263A5C" stroke-width="1.5"/>
|
|
71
|
-
<text x="648" y="574" font-size="17" fill="#8296B3"
|
|
71
|
+
<text x="648" y="574" font-size="17" fill="#8296B3">长任务成本 · 每场景输入 token(B 无记忆重新探索)</text>
|
|
72
72
|
<text x="648" y="600" font-size="20" font-weight="700">
|
|
73
|
-
<tspan fill="#FFD15D">A
|
|
74
|
-
<tspan fill="#8FA6C6"> · B
|
|
73
|
+
<tspan fill="#FFD15D">A 266k</tspan>
|
|
74
|
+
<tspan fill="#8FA6C6"> · B 1.81M</tspan>
|
|
75
75
|
</text>
|
|
76
76
|
<rect x="1040" y="578" width="72" height="26" rx="8" fill="#315DA6"/>
|
|
77
|
-
<text x="1076" y="597" font-size="18" font-weight="700" fill="#FFD15D" text-anchor="middle"
|
|
77
|
+
<text x="1076" y="597" font-size="18" font-weight="700" fill="#FFD15D" text-anchor="middle">6.8×</text>
|
|
78
78
|
</g>
|
|
79
79
|
</svg>
|
package/dist/client.js
CHANGED
|
@@ -2044,7 +2044,7 @@ window.__ModuleLoader__.load({
|
|
|
2044
2044
|
"div", { style: S.switchDesc },
|
|
2045
2045
|
"各蒸馏层单次输出的 token 上限(抽取/去重/L2/L3);留空或 0 = 跟随默认(当前生效 " +
|
|
2046
2046
|
layers.map(function (l) { return eff[l[0]] || "?"; }).join(" / ") +
|
|
2047
|
-
");思考档 high/max 时实际限额自动 ×4",
|
|
2047
|
+
");思考档 high/xhigh/max 时实际限额自动 ×4",
|
|
2048
2048
|
)),
|
|
2049
2049
|
),
|
|
2050
2050
|
react.createElement(
|
|
@@ -2197,34 +2197,37 @@ window.__ModuleLoader__.load({
|
|
|
2197
2197
|
// 分组:蒸馏参数(模型路由 / 思考档位 / 输出预算)
|
|
2198
2198
|
react.createElement("div", { style: S.panelLabel }, "蒸馏参数"),
|
|
2199
2199
|
settingsData.effort
|
|
2200
|
-
?
|
|
2201
|
-
|
|
2202
|
-
|
|
2203
|
-
|
|
2204
|
-
|
|
2205
|
-
|
|
2206
|
-
|
|
2207
|
-
|
|
2208
|
-
|
|
2209
|
-
|
|
2210
|
-
|
|
2211
|
-
|
|
2212
|
-
|
|
2213
|
-
|
|
2214
|
-
|
|
2215
|
-
|
|
2216
|
-
|
|
2217
|
-
|
|
2218
|
-
|
|
2219
|
-
|
|
2220
|
-
|
|
2221
|
-
react.createElement(
|
|
2222
|
-
"div",
|
|
2223
|
-
|
|
2224
|
-
|
|
2225
|
-
|
|
2226
|
-
|
|
2227
|
-
|
|
2200
|
+
? (function () {
|
|
2201
|
+
// 档位表来自当前生效模型的能力声明(settings-get 的 effort.options;
|
|
2202
|
+
// 空声明 → 只显示 high)。首项固定「自动」(key='',点击回写空串=
|
|
2203
|
+
// 按模型能力解析),选过显式档位后仍可回到自动;失效档位高亮实际生效值。
|
|
2204
|
+
var f = settingsData.effort;
|
|
2205
|
+
var opts = f.options && f.options.length ? f.options : ["high"];
|
|
2206
|
+
var segOptions = [{ key: "", label: "自动" }].concat(
|
|
2207
|
+
opts.map(function (k) { return { key: k, label: k }; })
|
|
2208
|
+
);
|
|
2209
|
+
var segValue = f.current === "" || opts.indexOf(f.current) >= 0
|
|
2210
|
+
? f.current
|
|
2211
|
+
: opts.indexOf(f.effective) >= 0 ? f.effective : "high";
|
|
2212
|
+
return react.createElement(
|
|
2213
|
+
"div",
|
|
2214
|
+
{ style: S.switchRow },
|
|
2215
|
+
react.createElement(Segmented, {
|
|
2216
|
+
value: segValue,
|
|
2217
|
+
options: segOptions,
|
|
2218
|
+
disabled: !master,
|
|
2219
|
+
onChange: function (v) { toggle("reasoningEffort", v); },
|
|
2220
|
+
}),
|
|
2221
|
+
react.createElement("div", null,
|
|
2222
|
+
react.createElement("div", { style: S.switchLabel }, "蒸馏思考"),
|
|
2223
|
+
react.createElement(
|
|
2224
|
+
"div",
|
|
2225
|
+
{ style: S.switchDesc },
|
|
2226
|
+
"当前生效 " + (f.effective || "(不传,跟随模型默认)") +
|
|
2227
|
+
(f.current ? "" : "(自动)"),
|
|
2228
|
+
)),
|
|
2229
|
+
);
|
|
2230
|
+
})()
|
|
2228
2231
|
: null,
|
|
2229
2232
|
react.createElement(LlmModelRow, { rpc: rpc, disabled: !master }),
|
|
2230
2233
|
react.createElement(BudgetInputs, {
|
package/dist/config.d.ts
CHANGED
|
@@ -5,6 +5,13 @@
|
|
|
5
5
|
*/
|
|
6
6
|
import Schema from '@deepseek-ai/schemastery';
|
|
7
7
|
import type { ExtractMode } from './types.js';
|
|
8
|
+
/**
|
|
9
|
+
* 蒸馏思考档位全词汇表(唯一事实源):'' = 自动(模型默认档 → high),
|
|
10
|
+
* 其余为各适配器通用档位词汇表(deepseek 认 'off',OpenAI 系是 'none')。
|
|
11
|
+
* schema(config/settings)、运行时解析(settings.resolveSettings)与 RPC
|
|
12
|
+
* 写入门(stats.settings-set)共用,勿在别处再抄字面量表。
|
|
13
|
+
*/
|
|
14
|
+
export declare const EFFORT_CHOICES: readonly ["", "off", "none", "minimal", "low", "medium", "high", "xhigh", "max"];
|
|
8
15
|
export interface MemoryConfig {
|
|
9
16
|
/** 数据目录;留空则用 $DSH_HOME/memory。 */
|
|
10
17
|
dataDir: string;
|
package/dist/config.js
CHANGED
|
@@ -5,6 +5,13 @@
|
|
|
5
5
|
*/
|
|
6
6
|
import Schema from '@deepseek-ai/schemastery';
|
|
7
7
|
import { dshHomePath } from '@deepseek-ai/dsh-home-paths';
|
|
8
|
+
/**
|
|
9
|
+
* 蒸馏思考档位全词汇表(唯一事实源):'' = 自动(模型默认档 → high),
|
|
10
|
+
* 其余为各适配器通用档位词汇表(deepseek 认 'off',OpenAI 系是 'none')。
|
|
11
|
+
* schema(config/settings)、运行时解析(settings.resolveSettings)与 RPC
|
|
12
|
+
* 写入门(stats.settings-set)共用,勿在别处再抄字面量表。
|
|
13
|
+
*/
|
|
14
|
+
export const EFFORT_CHOICES = ['', 'off', 'none', 'minimal', 'low', 'medium', 'high', 'xhigh', 'max'];
|
|
8
15
|
export const memorySchema = Schema.object({
|
|
9
16
|
dataDir: Schema.string().default(''),
|
|
10
17
|
family: Schema.union(['auto', 'chat', 'work']).default('auto'),
|
|
@@ -64,7 +71,7 @@ export const memorySchema = Schema.object({
|
|
|
64
71
|
// 蒸馏思考档位:'' = 自动(按模型能力解析:模型默认档 → high,见 llm.ts decideSendableEffort);
|
|
65
72
|
// 显式值仅在该模型声明支持时发送(跨供应商 effort 词汇表不同:deepseek 认 'off',
|
|
66
73
|
// openai 系是 'none',未声明档位的模型不传)。旧默认 'off' 在非 deepseek 模型上必炸(400/本地拒绝)
|
|
67
|
-
reasoningEffort: Schema.union([
|
|
74
|
+
reasoningEffort: Schema.union([...EFFORT_CHOICES]).default(''),
|
|
68
75
|
temperature: Schema.number().min(0).max(2).default(0.3),
|
|
69
76
|
// 模型上下文 1M token,日常压到 ~700k 使用(中文按 1 字≈1 token 保守折算)
|
|
70
77
|
maxInputChars: Schema.number().min(1000).max(1_000_000).default(700_000),
|
package/dist/llm.d.ts
CHANGED
|
@@ -28,7 +28,7 @@ export declare const LAYER_DEFAULT_BUDGETS: Record<DistillBudgetLayer, number>;
|
|
|
28
28
|
/**
|
|
29
29
|
* 解析某蒸馏层的生效输出预算:运行时覆盖(cfg.llm.budgets,由 effectiveCfg 从
|
|
30
30
|
* 设置页 distillBudgets 注入,0/缺省 = 跟随)→ 内置默认 → 思考档放大
|
|
31
|
-
* (high/max ×4,reasoning 计入输出预算的历史事故防线)。
|
|
31
|
+
* (high/xhigh/max ×4,reasoning 计入输出预算的历史事故防线)。
|
|
32
32
|
*/
|
|
33
33
|
export declare function resolveLayerTokens(cfg: {
|
|
34
34
|
llm: {
|
|
@@ -36,9 +36,15 @@ export declare function resolveLayerTokens(cfg: {
|
|
|
36
36
|
budgets?: Partial<Record<DistillBudgetLayer, number>>;
|
|
37
37
|
};
|
|
38
38
|
}, layer: DistillBudgetLayer): number;
|
|
39
|
+
/**
|
|
40
|
+
* 高思考档集合(输出预算 ×4 的档位):阶段侧 layerMaxTokens 与 callLLM 的
|
|
41
|
+
* 自动档防线共用同一张表——此前两处字面量表分叉(防线漏 xhigh),显式 xhigh
|
|
42
|
+
* 配置被双重放大 ×16。勿再在别处抄写该列表。
|
|
43
|
+
*/
|
|
44
|
+
export declare const HIGH_EFFORT_TIERS: readonly ["high", "xhigh", "max"];
|
|
39
45
|
/**
|
|
40
46
|
* 思考档预算放大:reasoning 计入输出预算(v4-flash 事故:high 思考可吃光全部
|
|
41
|
-
* 预算致正文 0 字符)——effort 为 high/max 时分层预算 ×4。
|
|
47
|
+
* 预算致正文 0 字符)——effort 为 high/xhigh/max 时分层预算 ×4。
|
|
42
48
|
*/
|
|
43
49
|
export declare function layerMaxTokens(base: number, reasoningEffort: string): number;
|
|
44
50
|
/** 解析蒸馏用的 provider/model:配置优先,其次当前默认选择。 */
|
package/dist/llm.js
CHANGED
|
@@ -20,18 +20,24 @@ export const LAYER_DEFAULT_BUDGETS = {
|
|
|
20
20
|
/**
|
|
21
21
|
* 解析某蒸馏层的生效输出预算:运行时覆盖(cfg.llm.budgets,由 effectiveCfg 从
|
|
22
22
|
* 设置页 distillBudgets 注入,0/缺省 = 跟随)→ 内置默认 → 思考档放大
|
|
23
|
-
* (high/max ×4,reasoning 计入输出预算的历史事故防线)。
|
|
23
|
+
* (high/xhigh/max ×4,reasoning 计入输出预算的历史事故防线)。
|
|
24
24
|
*/
|
|
25
25
|
export function resolveLayerTokens(cfg, layer) {
|
|
26
26
|
const override = cfg.llm.budgets?.[layer];
|
|
27
27
|
return layerMaxTokens(override && override > 0 ? override : LAYER_DEFAULT_BUDGETS[layer], cfg.llm.reasoningEffort);
|
|
28
28
|
}
|
|
29
|
+
/**
|
|
30
|
+
* 高思考档集合(输出预算 ×4 的档位):阶段侧 layerMaxTokens 与 callLLM 的
|
|
31
|
+
* 自动档防线共用同一张表——此前两处字面量表分叉(防线漏 xhigh),显式 xhigh
|
|
32
|
+
* 配置被双重放大 ×16。勿再在别处抄写该列表。
|
|
33
|
+
*/
|
|
34
|
+
export const HIGH_EFFORT_TIERS = ['high', 'xhigh', 'max'];
|
|
29
35
|
/**
|
|
30
36
|
* 思考档预算放大:reasoning 计入输出预算(v4-flash 事故:high 思考可吃光全部
|
|
31
|
-
* 预算致正文 0 字符)——effort 为 high/max 时分层预算 ×4。
|
|
37
|
+
* 预算致正文 0 字符)——effort 为 high/xhigh/max 时分层预算 ×4。
|
|
32
38
|
*/
|
|
33
39
|
export function layerMaxTokens(base, reasoningEffort) {
|
|
34
|
-
return reasoningEffort
|
|
40
|
+
return HIGH_EFFORT_TIERS.includes(reasoningEffort) ? base * 4 : base;
|
|
35
41
|
}
|
|
36
42
|
/** 解析蒸馏用的 provider/model:配置优先,其次当前默认选择。 */
|
|
37
43
|
export async function resolveModelRoute(ctx, cfg) {
|
|
@@ -133,10 +139,12 @@ export async function callLLM(ctx, cfg, opts) {
|
|
|
133
139
|
const user = opts.user.length > cfg.llm.maxInputChars
|
|
134
140
|
? `${opts.user.slice(0, cfg.llm.maxInputChars)}\n\n[输入超出 ${cfg.llm.maxInputChars} 字符预算,已截断]`
|
|
135
141
|
: opts.user;
|
|
136
|
-
// 输出预算 ×4
|
|
137
|
-
//
|
|
142
|
+
// 输出预算 ×4 防线跟随【实际发送】的档位:阶段侧已按原始配置的高档位
|
|
143
|
+
// (HIGH_EFFORT_TIERS)放大过,这里只补自动档('' → 模型默认/高档)解析出
|
|
144
|
+
// 高档时的欠放大缺口——两侧共用一张表,配置本身就是高档时不再放大(防 ×16 双乘)
|
|
138
145
|
const baseMaxTokens = opts.maxTokens ?? cfg.llm.maxTokens;
|
|
139
|
-
const
|
|
146
|
+
const highTiers = HIGH_EFFORT_TIERS;
|
|
147
|
+
const maxTokens = highTiers.includes(effort.effort) && !highTiers.includes(cfg.llm.reasoningEffort)
|
|
140
148
|
? layerMaxTokens(baseMaxTokens, 'high')
|
|
141
149
|
: baseMaxTokens;
|
|
142
150
|
const stream = ctx.llm.stream({
|
package/dist/settings.d.ts
CHANGED
|
@@ -5,11 +5,11 @@
|
|
|
5
5
|
*/
|
|
6
6
|
import type { Context } from '@deepseek-ai/cordis';
|
|
7
7
|
import Schema from '@deepseek-ai/schemastery';
|
|
8
|
+
import { EFFORT_CHOICES } from './config.js';
|
|
8
9
|
import type { DistillBudgetLayer } from './llm.js';
|
|
9
10
|
import type { MemoryLogger } from './types.js';
|
|
10
|
-
/**
|
|
11
|
-
|
|
12
|
-
export type EffortChoice = '' | 'off' | 'none' | 'minimal' | 'low' | 'medium' | 'high' | 'xhigh' | 'max';
|
|
11
|
+
/** 蒸馏思考档位:'' = 自动(模型默认档 → high);词汇表单源于 config.ts 的 EFFORT_CHOICES。 */
|
|
12
|
+
export type EffortChoice = (typeof EFFORT_CHOICES)[number];
|
|
13
13
|
/** 分层输出预算(与 llm.ts 的 DistillBudgetLayer 同键;0 = 跟随内置默认)。 */
|
|
14
14
|
export type DistillBudgets = Record<DistillBudgetLayer, number>;
|
|
15
15
|
export interface MemoryLiveSettings {
|
package/dist/settings.js
CHANGED
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
import Schema from '@deepseek-ai/schemastery';
|
|
2
2
|
import { settingsNamespace } from '@deepseek-ai/dsh-settings';
|
|
3
|
+
import { EFFORT_CHOICES } from './config.js';
|
|
3
4
|
const NS = settingsNamespace('dsh-memory');
|
|
4
5
|
const ALWAYS_ON = {
|
|
5
6
|
enabled: true,
|
|
@@ -36,7 +37,7 @@ export function liveSettingsSchema() {
|
|
|
36
37
|
capture: Schema.boolean().default(true),
|
|
37
38
|
distill: Schema.boolean().default(true),
|
|
38
39
|
recall: Schema.boolean().default(true),
|
|
39
|
-
reasoningEffort: Schema.union([
|
|
40
|
+
reasoningEffort: Schema.union([...EFFORT_CHOICES]).default(''),
|
|
40
41
|
distillProvider: Schema.string().default(''),
|
|
41
42
|
distillModel: Schema.string().default(''),
|
|
42
43
|
distillBudgets: Schema.object({
|
|
@@ -154,7 +155,6 @@ function resolveSettings(value) {
|
|
|
154
155
|
if (!value || typeof value !== 'object')
|
|
155
156
|
return { ...ALWAYS_ON };
|
|
156
157
|
const v = value;
|
|
157
|
-
const efforts = ['', 'off', 'none', 'minimal', 'low', 'medium', 'high', 'xhigh', 'max'];
|
|
158
158
|
const num = (x) => (typeof x === 'number' && Number.isFinite(x) && x >= 0 ? Math.floor(x) : 0);
|
|
159
159
|
const rawBudgets = (v.distillBudgets ?? {});
|
|
160
160
|
return {
|
|
@@ -162,7 +162,7 @@ function resolveSettings(value) {
|
|
|
162
162
|
capture: v.capture !== false,
|
|
163
163
|
distill: v.distill !== false,
|
|
164
164
|
recall: v.recall !== false,
|
|
165
|
-
reasoningEffort: typeof v.reasoningEffort === 'string' &&
|
|
165
|
+
reasoningEffort: typeof v.reasoningEffort === 'string' && EFFORT_CHOICES.includes(v.reasoningEffort)
|
|
166
166
|
? v.reasoningEffort
|
|
167
167
|
: '',
|
|
168
168
|
distillProvider: typeof v.distillProvider === 'string' ? v.distillProvider : '',
|
package/dist/stats.js
CHANGED
|
@@ -9,7 +9,7 @@
|
|
|
9
9
|
import { createRequire } from 'node:module';
|
|
10
10
|
import { closeSync, openSync, readSync, statSync } from 'node:fs';
|
|
11
11
|
import { join } from 'node:path';
|
|
12
|
-
import { resolveDataDir } from './config.js';
|
|
12
|
+
import { EFFORT_CHOICES, resolveDataDir } from './config.js';
|
|
13
13
|
import { effectiveCfg } from './pipeline/runner.js';
|
|
14
14
|
import { decideSendableEffort, LAYER_DEFAULT_BUDGETS, resolveModelEfforts, resolveModelRoute } from './llm.js';
|
|
15
15
|
import { errDetail } from './util/filelog.js';
|
|
@@ -234,8 +234,10 @@ async function handleEndpoint(endpoint, payload, deps) {
|
|
|
234
234
|
}
|
|
235
235
|
if (patch.reasoningEffort !== undefined) {
|
|
236
236
|
const v = String(patch.reasoningEffort);
|
|
237
|
-
|
|
238
|
-
|
|
237
|
+
// 白名单与 schema/settings 同源(config.ts EFFORT_CHOICES)——此前此处漏扩词表,
|
|
238
|
+
// 设置页新词汇(none/minimal/low/medium/xhigh)被拒并回滚
|
|
239
|
+
if (!EFFORT_CHOICES.includes(v)) {
|
|
240
|
+
throw new Error(`非法思考档位: ${v}(允许 '' 或 ${EFFORT_CHOICES.filter((x) => x !== '').join('/')})`);
|
|
239
241
|
}
|
|
240
242
|
clean.reasoningEffort = v;
|
|
241
243
|
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "dsh-layered-memory",
|
|
3
|
-
"version": "0.8.
|
|
3
|
+
"version": "0.8.4",
|
|
4
4
|
"description": "L0~L3 分层蒸馏记忆插件 for DeepSeek Harness:自动捕获对话(L0)、抽取原子记忆(L1)、整合场景块(L2)、蒸馏核心画像/团队方法论(L3),并在模型步骤前自动召回注入。移植自 MemoryCore (TencentDB Agent Memory) 的管线设计。",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "dist/index.js",
|
|
@@ -58,25 +58,25 @@
|
|
|
58
58
|
},
|
|
59
59
|
"peerDependencies": {
|
|
60
60
|
"@deepseek-ai/cordis": "^4.0.1",
|
|
61
|
-
"@deepseek-ai/dsh-agent": "^0.1.
|
|
62
|
-
"@deepseek-ai/dsh-home-paths": "^0.1.
|
|
63
|
-
"@deepseek-ai/dsh-llm": "^0.1.
|
|
64
|
-
"@deepseek-ai/dsh-session": "^0.1.
|
|
65
|
-
"@deepseek-ai/dsh-settings": "^0.1.
|
|
66
|
-
"@deepseek-ai/dsh-system-prompt": "^0.1.
|
|
67
|
-
"@deepseek-ai/dsh-tools": "^0.1.
|
|
61
|
+
"@deepseek-ai/dsh-agent": "^0.1.1-rc.2",
|
|
62
|
+
"@deepseek-ai/dsh-home-paths": "^0.1.1-rc.2",
|
|
63
|
+
"@deepseek-ai/dsh-llm": "^0.1.1-rc.2",
|
|
64
|
+
"@deepseek-ai/dsh-session": "^0.1.1-rc.2",
|
|
65
|
+
"@deepseek-ai/dsh-settings": "^0.1.1-rc.2",
|
|
66
|
+
"@deepseek-ai/dsh-system-prompt": "^0.1.1-rc.2",
|
|
67
|
+
"@deepseek-ai/dsh-tools": "^0.1.1-rc.2"
|
|
68
68
|
},
|
|
69
69
|
"devDependencies": {
|
|
70
70
|
"@deepseek-ai/cordis": "4.0.1",
|
|
71
|
-
"@deepseek-ai/dsh-agent": "0.1.
|
|
72
|
-
"@deepseek-ai/dsh-agent-default-model": "0.1.
|
|
73
|
-
"@deepseek-ai/dsh-client-connection": "0.1.
|
|
74
|
-
"@deepseek-ai/dsh-home-paths": "0.1.
|
|
75
|
-
"@deepseek-ai/dsh-llm": "0.1.
|
|
76
|
-
"@deepseek-ai/dsh-session": "0.1.
|
|
77
|
-
"@deepseek-ai/dsh-settings": "0.1.
|
|
78
|
-
"@deepseek-ai/dsh-system-prompt": "0.1.
|
|
79
|
-
"@deepseek-ai/dsh-tools": "0.1.
|
|
71
|
+
"@deepseek-ai/dsh-agent": "0.1.1-rc.2",
|
|
72
|
+
"@deepseek-ai/dsh-agent-default-model": "0.1.1-rc.2",
|
|
73
|
+
"@deepseek-ai/dsh-client-connection": "0.1.1-rc.2",
|
|
74
|
+
"@deepseek-ai/dsh-home-paths": "0.1.1-rc.2",
|
|
75
|
+
"@deepseek-ai/dsh-llm": "0.1.1-rc.2",
|
|
76
|
+
"@deepseek-ai/dsh-session": "0.1.1-rc.2",
|
|
77
|
+
"@deepseek-ai/dsh-settings": "0.1.1-rc.2",
|
|
78
|
+
"@deepseek-ai/dsh-system-prompt": "0.1.1-rc.2",
|
|
79
|
+
"@deepseek-ai/dsh-tools": "0.1.1-rc.2",
|
|
80
80
|
"@types/node": "^22.0.0",
|
|
81
81
|
"typescript": "^5.6.0"
|
|
82
82
|
}
|