dsh-plugin-bridge 0.2.10
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +146 -0
- package/README.zh.md +146 -0
- package/cordis.patch.yml +18 -0
- package/docs/articles/agent-session-handoff.md +162 -0
- package/docs/benchmark.md +165 -0
- package/docs/design.md +82 -0
- package/docs/guide.zh.md +152 -0
- package/docs/native-webui-feasibility.md +44 -0
- package/docs/plan.md +65 -0
- package/lib/api-rpc.d.ts +64 -0
- package/lib/api-rpc.js +76 -0
- package/lib/cli.d.ts +2 -0
- package/lib/cli.js +368 -0
- package/lib/command.d.ts +92 -0
- package/lib/command.js +377 -0
- package/lib/compression.d.ts +90 -0
- package/lib/compression.js +369 -0
- package/lib/fold.d.ts +39 -0
- package/lib/fold.js +406 -0
- package/lib/index.d.ts +55 -0
- package/lib/index.js +76 -0
- package/lib/migrate.d.ts +176 -0
- package/lib/migrate.js +449 -0
- package/lib/rpc.d.ts +34 -0
- package/lib/rpc.js +76 -0
- package/lib/types.d.ts +64 -0
- package/lib/types.js +9 -0
- package/package.json +106 -0
- package/reports/v0.2.3-e2e-2026-08-20T13-19-13-924Z.raw.json +2727 -0
- package/reports/v0.2.3-e2e-report.md +116 -0
- package/reports/v0.2.6-rc11-vision-report.md +65 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 dsh-plugin-bridge contributors
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
ADDED
|
@@ -0,0 +1,146 @@
|
|
|
1
|
+
# dsh-plugin-bridge
|
|
2
|
+
|
|
3
|
+
<p align="center">
|
|
4
|
+
<img src="./assets/cover/cover-en.png" width="100%" alt="dsh-plugin-bridge moves a locked session to a new preset through a previewable five-part handoff">
|
|
5
|
+
</p>
|
|
6
|
+
|
|
7
|
+
[](https://github.com/deepseek-ai/deepseek-harness)
|
|
8
|
+
[](https://github.com/Totoro-qaq/dsh-plugin-bridge/actions/workflows/ci.yml)
|
|
9
|
+
[](LICENSE)
|
|
10
|
+
[](package.json)
|
|
11
|
+
[](https://github.com/deepseek-ai/deepseek-harness)
|
|
12
|
+
[](https://github.com/awesome-dsh-plugin/awesome-dsh-plugin)
|
|
13
|
+
|
|
14
|
+
English | [中文](README.zh.md)
|
|
15
|
+
|
|
16
|
+
Halfway through a task and need another tool preset? Switching the produced session in place would leave tool history that belongs to the old assembly. Bridge previews a bounded five-part handoff, opens a clean target, and leaves the original session untouched.
|
|
17
|
+
|
|
18
|
+
<p align="center">
|
|
19
|
+
<img src="https://raw.githubusercontent.com/Totoro-qaq/dsh-plugin-bridge/main/assets/bridge-demo.en.gif" width="880" alt="A real Bridge migration in the official DeepSeek Harness WebUI">
|
|
20
|
+
</p>
|
|
21
|
+
|
|
22
|
+
[Quick start](#quick-start) · [Why Bridge](#why-bridge) · [Evidence](#evidence-at-a-glance) · [Decisions](#migration-decisions) · [Compatibility](#compatibility)
|
|
23
|
+
|
|
24
|
+
## Quick start
|
|
25
|
+
|
|
26
|
+
Install from npm:
|
|
27
|
+
|
|
28
|
+
```bash
|
|
29
|
+
dsh plugin --profile web add dsh-plugin-bridge
|
|
30
|
+
# restart dsh web once
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
Pinned GitHub fallback:
|
|
34
|
+
|
|
35
|
+
```bash
|
|
36
|
+
dsh plugin --profile web add github:Totoro-qaq/dsh-plugin-bridge#v0.2.10
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
Then type in the official WebUI:
|
|
40
|
+
|
|
41
|
+
```text
|
|
42
|
+
/bridge list target presets
|
|
43
|
+
/bridge --doctor check the host contract after a DSH upgrade
|
|
44
|
+
/bridge code preview the handoff; change nothing
|
|
45
|
+
/bridge code --go migrate, restate, then wait
|
|
46
|
+
/bridge code --go --continue restate and start work in the same target request
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
The preview is editable. Correct a number or path in the printed summary file, then run:
|
|
50
|
+
|
|
51
|
+
```text
|
|
52
|
+
/bridge code --go --file <path>
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
Uninstall with `dsh plugin --profile web remove dsh-plugin-bridge`, then restart `dsh web`.
|
|
56
|
+
|
|
57
|
+
## Why Bridge
|
|
58
|
+
|
|
59
|
+
| Promise | What it means |
|
|
60
|
+
|---|---|
|
|
61
|
+
| **Preview before execution** | `/bridge <preset>` creates no target and changes no source session. Review or edit the five-section handoff first. |
|
|
62
|
+
| **Move state, not tool traces** | Decisions, paths, current state, and next steps move to a clean preset. Incompatible calls from the old tool assembly do not. |
|
|
63
|
+
| **Fail closed** | The target goal is paused before kickoff. If that cannot be guaranteed, Bridge clears/cancels the target and sends no model request. |
|
|
64
|
+
|
|
65
|
+
Installing Bridge adds **zero prompt tokens** to ordinary sessions. It is a host slash command, not a model tool or skill.
|
|
66
|
+
|
|
67
|
+
## Evidence at a glance
|
|
68
|
+
|
|
69
|
+
The release gate is intentionally small and reproducible; these are regression results, not population guarantees.
|
|
70
|
+
|
|
71
|
+
| Gate | Result |
|
|
72
|
+
|---|---:|
|
|
73
|
+
| Five-part summary facts | **30/30** |
|
|
74
|
+
| Target restatement / first useful work facts | **60/60 · 60/60** |
|
|
75
|
+
| Critical facts / obsolete-value resurrection | **90/90 · 0** |
|
|
76
|
+
| Existing image evidence / unresolved raw image | **5/5 · 5/5** |
|
|
77
|
+
| Confirm / `--continue` target request shape | **2 · 1** to first useful work |
|
|
78
|
+
| Confirm extra, paired nominal median | **+8.1%** vs `--continue` |
|
|
79
|
+
| Summary worker share of clean acceptance components | **20.74% nominal** |
|
|
80
|
+
|
|
81
|
+
The token percentage varies widely with preset, response length, and cache state. The worker share is composition, not causal overhead versus no Bridge; the stable product claim is one additional confirmation request. Read the [design and evidence boundaries](docs/design.md), [full release report](reports/v0.2.3-e2e-report.md), and [vision report](reports/v0.2.6-rc11-vision-report.md).
|
|
82
|
+
|
|
83
|
+
## How it works
|
|
84
|
+
|
|
85
|
+
```text
|
|
86
|
+
fold history -> five-part handoff -> preview/edit -> clean target session
|
|
87
|
+
-> pause stored goal -> inject -> restate -> wait or continue
|
|
88
|
+
image history -> verbatim assistant evidence; unresolved originals use the attachment gateway
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
The five sections are Goal, Current state, Key decisions and conventions, Key files, and Next step. The original session is never rewritten; archive the target and return to the source if the handoff is unsatisfactory.
|
|
92
|
+
|
|
93
|
+
## Migration decisions
|
|
94
|
+
|
|
95
|
+
| Situation | Bridge behavior | Cost / fidelity effect |
|
|
96
|
+
|---|---|---|
|
|
97
|
+
| Plugin installed, no `/bridge` call | No prompt injection or model tool | **0 Bridge prompt tokens** |
|
|
98
|
+
| `/bridge code` | One bounded summary worker; preview only | No target session is created |
|
|
99
|
+
| Default `--go` | Target restates and waits | One explicit confirmation request before useful work |
|
|
100
|
+
| `--go --continue` | Restate and work in one target request | Lower request count; no background goal round |
|
|
101
|
+
| Image already has assistant analysis | Copy that response verbatim | No raw image is resent by default |
|
|
102
|
+
| Image is unresolved and target accepts images | Copy the original attachment and preserve the source VLM | Vision pricing comes from the selected provider |
|
|
103
|
+
| Image is unresolved and target is text-only | Prompt admission rejects the image; Bridge sends a visible text fallback | No hidden local VLM and no silent claim of visual understanding |
|
|
104
|
+
|
|
105
|
+
## Compatibility
|
|
106
|
+
|
|
107
|
+
| DSH baseline | Text handoff | Raw unresolved image | Verification boundary |
|
|
108
|
+
|---|---:|---:|---|
|
|
109
|
+
| 0.1.0-rc.6 / rc.7 | Yes | Optional gateway unavailable | Narrow RPC contract and text compatibility tests |
|
|
110
|
+
| 0.1.0-rc.8 | Yes | Host-dependent | Real install, restart, command lifecycle, and migration baseline |
|
|
111
|
+
| 0.1.1-rc.2 | Yes | Yes | Official WebUI migration with `deepseek-v4-flash-vision-exp`, 13/13 doctor methods, and 5/5 vision gates |
|
|
112
|
+
|
|
113
|
+
CI covers Node.js 22 and 24. Run `/bridge --doctor` after every Harness upgrade; it names missing required gateway methods instead of failing vaguely.
|
|
114
|
+
|
|
115
|
+
Current limits:
|
|
116
|
+
|
|
117
|
+
- installation needs one WebUI restart;
|
|
118
|
+
- Bridge prints the created title and session ID because stable plugin-driven session navigation is not yet available;
|
|
119
|
+
- preview normally takes 20–60 seconds and is bounded by `previewTimeoutMs`;
|
|
120
|
+
- text-only models cannot inspect unresolved images;
|
|
121
|
+
- each release-acceptance cell currently has one run, so the tables are release evidence rather than statistical guarantees.
|
|
122
|
+
|
|
123
|
+
The server command stays the compatibility core. A native migration card is technically possible through official client modules and slots, but is intentionally deferred until that prerelease contract is stable; see the [feasibility note](docs/native-webui-feasibility.md).
|
|
124
|
+
|
|
125
|
+
## Documentation
|
|
126
|
+
|
|
127
|
+
- [Design, safety, image policy, cost, and evidence](docs/design.md)
|
|
128
|
+
- [Chinese install, configuration, rollback, and FAQ](docs/guide.zh.md)
|
|
129
|
+
- [Release acceptance report](reports/v0.2.3-e2e-report.md)
|
|
130
|
+
- [Vision migration report](reports/v0.2.6-rc11-vision-report.md)
|
|
131
|
+
- [Historical compression benchmark](docs/benchmark.md)
|
|
132
|
+
|
|
133
|
+
## Development
|
|
134
|
+
|
|
135
|
+
```bash
|
|
136
|
+
npm ci
|
|
137
|
+
npm run verify
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
`verify` builds and type-checks the project, runs 125 fake-host tests, checks generated `lib/` and datasets, then packs, installs, and imports the actual npm tarball. Tests spend no model tokens. `prepublishOnly` runs the same gate; GitHub releases also require the tag to match `package.json` before trusted npm publishing.
|
|
141
|
+
|
|
142
|
+
Community listings: [Awesome DSH Plugin](https://github.com/awesome-dsh-plugin/awesome-dsh-plugin) · [Awesome DeepSeek Harness](https://github.com/Dominic789654/awesome-deepseek-harness)
|
|
143
|
+
|
|
144
|
+
## License
|
|
145
|
+
|
|
146
|
+
MIT
|
package/README.zh.md
ADDED
|
@@ -0,0 +1,146 @@
|
|
|
1
|
+
# dsh-plugin-bridge
|
|
2
|
+
|
|
3
|
+
<p align="center">
|
|
4
|
+
<img src="./assets/cover/cover-zh.png" width="100%" alt="dsh-plugin-bridge 通过可预览的固定五段交接,把锁定会话迁移到新的 preset">
|
|
5
|
+
</p>
|
|
6
|
+
|
|
7
|
+
[](https://github.com/deepseek-ai/deepseek-harness)
|
|
8
|
+
[](https://github.com/Totoro-qaq/dsh-plugin-bridge/actions/workflows/ci.yml)
|
|
9
|
+
[](LICENSE)
|
|
10
|
+
[](package.json)
|
|
11
|
+
[](https://github.com/deepseek-ai/deepseek-harness)
|
|
12
|
+
[](https://github.com/awesome-dsh-plugin/awesome-dsh-plugin)
|
|
13
|
+
|
|
14
|
+
[English](README.md) | 中文
|
|
15
|
+
|
|
16
|
+
在 Web preset 做到一半,想换 Code preset 继续?直接切换会让旧工具组的调用历史留在新组合里。Bridge 先生成一份有界、可编辑的五段交接,再建立干净目标会话;原会话始终不动。
|
|
17
|
+
|
|
18
|
+
<p align="center">
|
|
19
|
+
<img src="https://raw.githubusercontent.com/Totoro-qaq/dsh-plugin-bridge/main/assets/bridge-demo.zh.gif" width="880" alt="Bridge 在 DeepSeek Harness 官方 WebUI 中完成一次真实会话迁移">
|
|
20
|
+
</p>
|
|
21
|
+
|
|
22
|
+
[快速开始](#快速开始) · [为什么是-bridge](#为什么是-bridge) · [实测证据](#实测证据) · [迁移决策](#迁移决策) · [兼容性](#兼容性)
|
|
23
|
+
|
|
24
|
+
## 快速开始
|
|
25
|
+
|
|
26
|
+
从 npm 安装:
|
|
27
|
+
|
|
28
|
+
```bash
|
|
29
|
+
dsh plugin --profile web add dsh-plugin-bridge
|
|
30
|
+
# 重启一次 dsh web
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
GitHub 固定版本备用路径:
|
|
34
|
+
|
|
35
|
+
```bash
|
|
36
|
+
dsh plugin --profile web add github:Totoro-qaq/dsh-plugin-bridge#v0.2.10
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
然后在官方 WebUI 输入:
|
|
40
|
+
|
|
41
|
+
```text
|
|
42
|
+
/bridge 列出目标 preset
|
|
43
|
+
/bridge --doctor DSH 升级后检查 host 契约
|
|
44
|
+
/bridge code 只预览交接,什么都不改
|
|
45
|
+
/bridge code --go 迁移、复述,然后等待
|
|
46
|
+
/bridge code --go --continue 在同一次目标请求里复述并开始工作
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
预览可以编辑。数字或路径不对时,修改输出里打印的摘要文件,再执行:
|
|
50
|
+
|
|
51
|
+
```text
|
|
52
|
+
/bridge code --go --file <路径>
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
卸载:`dsh plugin --profile web remove dsh-plugin-bridge`,然后重启 `dsh web`。
|
|
56
|
+
|
|
57
|
+
## 为什么是 Bridge
|
|
58
|
+
|
|
59
|
+
| 承诺 | 具体含义 |
|
|
60
|
+
|---|---|
|
|
61
|
+
| **执行前预览** | `/bridge <preset>` 不创建目标、不改源会话;先检查或修改固定五段交接。 |
|
|
62
|
+
| **迁状态,不迁工具痕迹** | 决策、路径、当前状态和下一步进入干净 preset;旧工具组的不兼容调用不会跟过去。 |
|
|
63
|
+
| **失败时关闭迁移** | kickoff 前暂停目标;无法保证时清除/取消目标,不发送模型请求。 |
|
|
64
|
+
|
|
65
|
+
只安装、不调用时,普通会话增加 **0 Bridge prompt token**。它是 host slash command,不是模型工具或 skill。
|
|
66
|
+
|
|
67
|
+
## 实测证据
|
|
68
|
+
|
|
69
|
+
这是小样本、可复现的回归门禁,不是总体准确率保证。
|
|
70
|
+
|
|
71
|
+
| Gate | 结果 |
|
|
72
|
+
|---|---:|
|
|
73
|
+
| 五段摘要事实 | **30/30** |
|
|
74
|
+
| 目标复述 / 首次有效工作事实 | **60/60 · 60/60** |
|
|
75
|
+
| 关键事实 / 旧值复活 | **90/90 · 0** |
|
|
76
|
+
| 已有图片证据 / 未解析原图路径 | **5/5 · 5/5** |
|
|
77
|
+
| Confirm / `--continue` 到首次有效工作的目标请求数 | **2 · 1** |
|
|
78
|
+
| Confirm 相对 `--continue` 的 nominal 配对中位额外成本 | **+8.1%** |
|
|
79
|
+
| 摘要 worker 在干净验收组件中的 nominal 占比 | **20.74%** |
|
|
80
|
+
|
|
81
|
+
token 百分比会随 preset、回复长度和缓存状态大幅波动;worker 占比是组成,不是相对“无 Bridge”的因果开销。稳定结论是默认确认多一个请求。边界和原始证据见[设计与证据说明](docs/design.md)、[完整 release report](reports/v0.2.3-e2e-report.md)和[视觉迁移报告](reports/v0.2.6-rc11-vision-report.md)。
|
|
82
|
+
|
|
83
|
+
## 工作原理
|
|
84
|
+
|
|
85
|
+
```text
|
|
86
|
+
折叠历史 -> 五段交接 -> 预览/编辑 -> 干净目标会话
|
|
87
|
+
-> 暂停存储目标 -> 注入 -> 复述 -> 等待或同轮继续
|
|
88
|
+
图片历史 -> 原样搬运助手证据;未解析原图走附件网关
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
五段分别是:目标、当前状态、关键决策与约定、关键文件、下一步。原会话不会被改写;接得不好时归档目标,直接回源会话。
|
|
92
|
+
|
|
93
|
+
## 迁移决策
|
|
94
|
+
|
|
95
|
+
| 场景 | Bridge 行为 | 成本 / 保真影响 |
|
|
96
|
+
|---|---|---|
|
|
97
|
+
| 只安装,不调用 `/bridge` | 不注入提示,不注册模型工具 | **0 Bridge prompt token** |
|
|
98
|
+
| `/bridge code` | 起一个有界摘要 worker,只输出预览 | 不创建目标会话 |
|
|
99
|
+
| 默认 `--go` | 目标先复述再等待 | 首次有效工作前多一个显式确认请求 |
|
|
100
|
+
| `--go --continue` | 同一次目标请求里复述并开工 | 请求数更低,没有后台 goal 轮次 |
|
|
101
|
+
| 图片已有助手分析 | 逐字搬运对应回答 | 默认不重发原图 |
|
|
102
|
+
| 图片未解析,目标可接图 | 搬原附件并保留源 VLM | 视觉费用由所选 provider 计算 |
|
|
103
|
+
| 图片未解析,目标是纯文本模型 | prompt 准入拒图,Bridge 显式发送文字降级 | 不暗启本地 VLM,也不假装看懂图片 |
|
|
104
|
+
|
|
105
|
+
## 兼容性
|
|
106
|
+
|
|
107
|
+
| DSH 基线 | 文本交接 | 未解析原图 | 验证边界 |
|
|
108
|
+
|---|---:|---:|---|
|
|
109
|
+
| 0.1.0-rc.6 / rc.7 | 支持 | 无可选附件网关 | 窄 RPC 契约与文本兼容测试 |
|
|
110
|
+
| 0.1.0-rc.8 | 支持 | 取决于 host | 真实安装、重启、命令生命周期和迁移基线 |
|
|
111
|
+
| 0.1.1-rc.2 | 支持 | 支持 | 官方 WebUI + `deepseek-v4-flash-vision-exp`,doctor 13/13,视觉 gate 5/5 |
|
|
112
|
+
|
|
113
|
+
CI 覆盖 Node.js 22/24。每次升级 Harness 后先跑 `/bridge --doctor`;缺哪个必要网关方法会被直接点名。
|
|
114
|
+
|
|
115
|
+
当前边界:
|
|
116
|
+
|
|
117
|
+
- 安装后需要重启一次 WebUI;
|
|
118
|
+
- 官方尚无稳定的插件跳转目标会话接口,Bridge 会打印新会话标题和 ID;
|
|
119
|
+
- 预览通常 20–60 秒,并受 `previewTimeoutMs` 限制;
|
|
120
|
+
- 纯文本模型无法读取未解析原图;
|
|
121
|
+
- release acceptance 每个 cell 目前只有一次运行,表格是发布证据,不是统计保证。
|
|
122
|
+
|
|
123
|
+
服务端命令仍是兼容核心。官方 client module 与 slot 已证明原生迁移卡片可行,但其 prerelease 契约还不稳定,因此暂不把它塞进 v0.2.10;详见[可行性记录](docs/native-webui-feasibility.md)。
|
|
124
|
+
|
|
125
|
+
## 文档
|
|
126
|
+
|
|
127
|
+
- [设计、安全、图片策略、成本与证据](docs/design.md)
|
|
128
|
+
- [中文安装、配置、回退与 FAQ](docs/guide.zh.md)
|
|
129
|
+
- [完整 release acceptance](reports/v0.2.3-e2e-report.md)
|
|
130
|
+
- [视觉迁移报告](reports/v0.2.6-rc11-vision-report.md)
|
|
131
|
+
- [历史压缩档位 benchmark](docs/benchmark.md)
|
|
132
|
+
|
|
133
|
+
## 开发验证
|
|
134
|
+
|
|
135
|
+
```bash
|
|
136
|
+
npm ci
|
|
137
|
+
npm run verify
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
`verify` 会构建、类型检查、运行 125 项 fake-host 测试、核对 `lib/` 与数据集,再把真实 npm tarball 打包、安装并导入。测试不消耗模型 token。`prepublishOnly` 使用同一个 gate;GitHub Release 还会先检查 tag 与 `package.json` 版本一致,再走可信 npm 发布。
|
|
141
|
+
|
|
142
|
+
社区收录:[Awesome DSH Plugin](https://github.com/awesome-dsh-plugin/awesome-dsh-plugin) · [Awesome DeepSeek Harness](https://github.com/Dominic789654/awesome-deepseek-harness)
|
|
143
|
+
|
|
144
|
+
## License
|
|
145
|
+
|
|
146
|
+
MIT
|
package/cordis.patch.yml
ADDED
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
# dsh-plugin-bridge 的 profile 层:把 /bridge 命令挂进 dsh.profile.bundles。
|
|
2
|
+
#
|
|
3
|
+
# 每个键都被 apply() 真正消费(0.1 里这些值是死的,配了不生效)。
|
|
4
|
+
- insert:
|
|
5
|
+
- id: bridge
|
|
6
|
+
name: dsh-plugin-bridge
|
|
7
|
+
config:
|
|
8
|
+
modelTier: !!js process.env.DSH_BRIDGE_TIER ?? 'pro'
|
|
9
|
+
sourceCharBudget: !!js Number(process.env.DSH_BRIDGE_SOURCE_BUDGET ?? 60000)
|
|
10
|
+
summaryCharBudget: !!js Number(process.env.DSH_BRIDGE_SUMMARY_BUDGET ?? 2400)
|
|
11
|
+
# 上游 goal.create 的部署默认是 256 轮自主循环;交接只需要一轮。
|
|
12
|
+
goalRounds: !!js Number(process.env.DSH_BRIDGE_GOAL_ROUNDS ?? 1)
|
|
13
|
+
inject: !!js process.env.DSH_BRIDGE_INJECT ?? 'both'
|
|
14
|
+
lang: !!js process.env.DSH_BRIDGE_LANG ?? 'auto'
|
|
15
|
+
workerProvider: !!js process.env.DSH_BRIDGE_PROVIDER ?? ''
|
|
16
|
+
workerModel: !!js process.env.DSH_BRIDGE_MODEL ?? ''
|
|
17
|
+
# /bridge 是同步返回的,等压缩工人不能等太久。
|
|
18
|
+
previewTimeoutMs: !!js Number(process.env.DSH_BRIDGE_PREVIEW_TIMEOUT ?? 180000)
|
|
@@ -0,0 +1,162 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Moving an Agent Session Without Moving Its Tool History
|
|
3
|
+
published: false
|
|
4
|
+
description: A previewable, fixed-schema, fail-closed protocol for switching an agent to a different tool environment.
|
|
5
|
+
tags: ai, agents, opensource, typescript
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
I wanted to move a half-finished coding task from one agent preset to another without asking the new session to rediscover the project.
|
|
9
|
+
|
|
10
|
+
This is the result in the official DeepSeek Harness WebUI:
|
|
11
|
+
|
|
12
|
+

|
|
13
|
+
|
|
14
|
+
The source session knows five facts: port `43179`, SQLite, no Redis, a target file, and the next step. Bridge produces a reviewable handoff, opens a clean session under the `code` preset, and asks that session to restate the facts. It does not start implementation until the user confirms.
|
|
15
|
+
|
|
16
|
+
The interesting part is not the slash command. It is the boundary the command preserves.
|
|
17
|
+
|
|
18
|
+
## A preset is more than a personality
|
|
19
|
+
|
|
20
|
+
An agent preset usually combines a system prompt, tools, plugins, permissions, and execution policy. Once a session has used that composition, its history contains tool calls and results that only make sense under that exact environment.
|
|
21
|
+
|
|
22
|
+
Changing the preset in place looks cheap, but it creates an ambiguous history:
|
|
23
|
+
|
|
24
|
+
- a tool call may name a tool the new preset does not have;
|
|
25
|
+
- the same tool name may now have different semantics;
|
|
26
|
+
- old permission assumptions may no longer hold;
|
|
27
|
+
- the model sees a continuous transcript even though the executable environment changed underneath it.
|
|
28
|
+
|
|
29
|
+
DeepSeek Harness avoids this by locking the preset after a session starts. That constraint is useful. The safer migration primitive is not “unlock the preset.” It is “open a clean destination and carry only the state that is still actionable.”
|
|
30
|
+
|
|
31
|
+
History is environment-specific. State can be portable.
|
|
32
|
+
|
|
33
|
+
## Move state, not tool traces
|
|
34
|
+
|
|
35
|
+
Bridge turns the source conversation into a five-part contract:
|
|
36
|
+
|
|
37
|
+
```text
|
|
38
|
+
Goal
|
|
39
|
+
Current state
|
|
40
|
+
Key decisions and conventions
|
|
41
|
+
Key files
|
|
42
|
+
Next step
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
That schema is intentionally boring. It forces the handoff to answer the questions the destination needs before acting:
|
|
46
|
+
|
|
47
|
+
- What outcome are we pursuing?
|
|
48
|
+
- What has already happened?
|
|
49
|
+
- Which constraints are still in force?
|
|
50
|
+
- Which files carry the work?
|
|
51
|
+
- What is the next executable step?
|
|
52
|
+
|
|
53
|
+
Raw tool traces are omitted. Obsolete, revoked, or superseded concrete values are omitted too; only the currently effective replacement remains actionable.
|
|
54
|
+
|
|
55
|
+
This last rule came from a real failure. An earlier prompt preserved both an old port and the reason it had been replaced. The summary correctly called the old value obsolete, but it still exposed the literal number to future context. That was unnecessary surface area for a later model to misuse. The release gate now checks that obsolete values do not reappear.
|
|
56
|
+
|
|
57
|
+
## Preview is part of the protocol
|
|
58
|
+
|
|
59
|
+
Summarization is probabilistic, so a generated handoff should not immediately become an execution instruction.
|
|
60
|
+
|
|
61
|
+
Bridge separates the operation into two phases:
|
|
62
|
+
|
|
63
|
+
```text
|
|
64
|
+
/bridge code # generate and preview; change nothing
|
|
65
|
+
/bridge code --go # create the target from the reviewed preview
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
The preview makes numbers, paths, and negative constraints visible before any target session exists. It is also written to a file, so the user can edit it and execute that exact version:
|
|
69
|
+
|
|
70
|
+
```text
|
|
71
|
+
/bridge code --go --file <reviewed-summary-path>
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
This is more than a nicer confirmation dialog. It converts one opaque model step into a human-checkable interface. Ports, IDs, file paths, and “do not introduce X” constraints deserve that treatment because a fluent but slightly wrong summary is often more dangerous than an obvious failure.
|
|
75
|
+
|
|
76
|
+
## The destination must fail closed
|
|
77
|
+
|
|
78
|
+
Creating a clean target is not enough. Agent runtimes may have goal drivers that schedule work as soon as a goal changes.
|
|
79
|
+
|
|
80
|
+
The safe sequence is:
|
|
81
|
+
|
|
82
|
+
1. Create a blank target under the requested preset.
|
|
83
|
+
2. Store the reviewed handoff.
|
|
84
|
+
3. Pause the stored goal before sending the kickoff.
|
|
85
|
+
4. Send one prompt that asks the target to restate the current state.
|
|
86
|
+
5. Wait for user confirmation.
|
|
87
|
+
|
|
88
|
+
If the pause step fails, Bridge clears the goal when that route is available, cancels the target session, and sends no kickoff. A partially prepared migration must not turn into an autonomous run.
|
|
89
|
+
|
|
90
|
+
The default path deliberately spends one target turn on restatement. For lower latency, `--continue` combines restatement and the first useful work in the same target request:
|
|
91
|
+
|
|
92
|
+
```text
|
|
93
|
+
/bridge code --go --continue
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
Both paths pause the stored goal. “Continue” changes request shape; it does not enable a background loop.
|
|
97
|
+
|
|
98
|
+
## Images need a separate evidence path
|
|
99
|
+
|
|
100
|
+
Image history should not be treated like ordinary summary text.
|
|
101
|
+
|
|
102
|
+
If an assistant has already analyzed an image, Bridge copies the associated assistant response verbatim outside the compressed five-part summary. The summary worker cannot rewrite that evidence, and the text is admitted as a whole block rather than cut mid-description.
|
|
103
|
+
|
|
104
|
+
If the image has no associated analysis, it remains unresolved. On a host with durable attachment recovery, Bridge tries to move the original attachment and preserves the source model selection before kickoff. An image-capable destination can inspect it; a text-only destination receives an explicit fallback warning instead of silently pretending the image was understood.
|
|
105
|
+
|
|
106
|
+
This policy avoids two bad defaults:
|
|
107
|
+
|
|
108
|
+
- recompressing an existing visual description until details drift;
|
|
109
|
+
- resending every image and paying vision cost when authoritative text already exists.
|
|
110
|
+
|
|
111
|
+
It also avoids a hidden local vision model. The route is visible: either preserved text is reused, or the selected VLM receives the unresolved image.
|
|
112
|
+
|
|
113
|
+
## Cost is a request-shape decision, not a universal percentage
|
|
114
|
+
|
|
115
|
+
Installing Bridge adds zero prompt tokens to ordinary sessions. `/bridge` is handled by the host and only invokes a summary worker when migration is requested.
|
|
116
|
+
|
|
117
|
+
In the fixed 12-cell release acceptance, the default confirmation path always used two target requests to reach first useful work. `--continue` always used one. That request-count difference is stable.
|
|
118
|
+
|
|
119
|
+
Token percentages were not stable. Across six paired fixtures, the confirmation path's nominal extra cost had a median of `+8.1%`, but the range was `-47.9%` to `+206.7%`. Preset system prompts, output length, and cache state dominated individual totals. A single “Bridge costs X%” claim would be false precision.
|
|
120
|
+
|
|
121
|
+
The useful product choice is therefore explicit:
|
|
122
|
+
|
|
123
|
+
- choose the default when a separate accuracy checkpoint is worth one request;
|
|
124
|
+
- choose `--continue` when you accept a combined confirmation-and-work turn.
|
|
125
|
+
|
|
126
|
+
## What the release gate actually proves
|
|
127
|
+
|
|
128
|
+
The current text migration gate covers six frozen fixtures and 12 target sessions across `minimal`, `standard`, and `code` presets:
|
|
129
|
+
|
|
130
|
+
| Measure | Result |
|
|
131
|
+
|---|---:|
|
|
132
|
+
| Summary facts | 30/30 |
|
|
133
|
+
| Target restatement facts | 60/60 |
|
|
134
|
+
| First useful work facts | 60/60 |
|
|
135
|
+
| Obsolete-value resurrection | 0 |
|
|
136
|
+
| Confirm request count | 6/6 exact |
|
|
137
|
+
| Continue request count | 6/6 exact |
|
|
138
|
+
|
|
139
|
+
Three source fixtures contain 21 user messages and reuse real compaction output. A separate vision gate uses five non-guessable facts visible only in a PNG; both the verbatim-evidence path and unresolved-image path recovered 5/5 facts.
|
|
140
|
+
|
|
141
|
+
These are release gates, not population accuracy estimates. Each cell ran once, the suite was repair-driven, and the vision gate covers one PNG fixture and one experimental VLM route. The repository keeps the raw data and limitations next to the headline numbers rather than turning them into a universal reliability claim.
|
|
142
|
+
|
|
143
|
+
## Try the implementation
|
|
144
|
+
|
|
145
|
+
[dsh-plugin-bridge](https://github.com/Totoro-qaq/dsh-plugin-bridge) is open source and works directly in the official DeepSeek Harness WebUI.
|
|
146
|
+
|
|
147
|
+
```bash
|
|
148
|
+
dsh plugin --profile web add github:Totoro-qaq/dsh-plugin-bridge#v0.2.9
|
|
149
|
+
# restart dsh web once
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
Then run:
|
|
153
|
+
|
|
154
|
+
```text
|
|
155
|
+
/bridge --doctor
|
|
156
|
+
/bridge code
|
|
157
|
+
/bridge code --go
|
|
158
|
+
```
|
|
159
|
+
|
|
160
|
+
The repository includes the [full text migration report](https://github.com/Totoro-qaq/dsh-plugin-bridge/blob/main/reports/v0.2.3-e2e-report.md), the [vision migration report](https://github.com/Totoro-qaq/dsh-plugin-bridge/blob/main/reports/v0.2.6-rc11-vision-report.md), and raw evaluation artifacts.
|
|
161
|
+
|
|
162
|
+
The implementation is specific to DeepSeek Harness. The protocol is not: open a clean destination, carry a bounded state contract, expose it for review, stop on partial setup, and make the destination prove what it understood before it acts.
|
|
@@ -0,0 +1,165 @@
|
|
|
1
|
+
# Bridge 跨模式迁移 · 准确率实验报告
|
|
2
|
+
|
|
3
|
+
日期:2026-08-17 | 宿主:本地 `dsh web`(默认 127.0.0.1:3080) | 模型:deepseek-v4-flash / deepseek-v4-pro
|
|
4
|
+
脚本:`eval/run.mjs` | 原始数据:`reports/benchmark-2026-08-17.raw.json`(含逐 run token 记账)
|
|
5
|
+
|
|
6
|
+
## 1. 实验设计
|
|
7
|
+
|
|
8
|
+
**被测对象**:TotoroPilot 的 Bridge 迁移全链路(`buildBridgeSource` 取材 → 工人模型压缩成交接摘要 → 目标 preset 新会话挂 goal + kickoff → 继续执行)。
|
|
9
|
+
|
|
10
|
+
**因子与水平**
|
|
11
|
+
|
|
12
|
+
| 因子 | 水平 | 备注 |
|
|
13
|
+
|---|---|---|
|
|
14
|
+
| 压缩档位 tier | flash / pro | 工人会话显式 selectModel |
|
|
15
|
+
| 目标 preset | standard / code / minimal / cordis | 迁移目的地 |
|
|
16
|
+
| 源 preset | standard / code / minimal / cordis | 仅 C 组变化 |
|
|
17
|
+
|
|
18
|
+
**控制变量**:同一真实工作区、同一埋点模板、每 run 恰好 5 个事实点(端口/数据库/禁令/路径/提交语言)、同一探针问题集、同一漂移任务、单轮超时统一(埋点 150s / 工人 360s / 目标 240s,超时即 `session.cancel` 记 caps)。
|
|
19
|
+
|
|
20
|
+
**数据集分离**
|
|
21
|
+
- 测试集 T16:tier(2) × 目标(4) × 固定题材(2:电商后端/数据管道),源统一 minimal(排除源干扰)
|
|
22
|
+
- 源对照 C4:同题材同档位(pro),目标统一 code,只变源 preset
|
|
23
|
+
- 验证集 V6:3 个 held-out 题材(物联网采集/游戏存档/日志分析),方向与档位抽样,含 1 条真实 cordis 源
|
|
24
|
+
|
|
25
|
+
**四层指标**(每层按 5 事实点计命中率)
|
|
26
|
+
1. 摘要层:工人摘要本身含多少事实(压缩保真上限)
|
|
27
|
+
2. 复述层:kickoff 后新会话自述理解含多少事实
|
|
28
|
+
3. 探针层:对新会话提问能答出多少事实(**用户真实可用性**)
|
|
29
|
+
4. 漂移层:让新会话"执行下一步"(写启动命令+首行代码),输出携带多少约定
|
|
30
|
+
|
|
31
|
+
26 条 run 全部成功(无 error,2 条触发过重试)。本轮实验总消耗 ≈ **13.6M 输入 / 0.42M 输出 tokens**(作为对照,早期无限速试跑 6 条即烧 ~70M)。
|
|
32
|
+
|
|
33
|
+
## 2. 总体结果
|
|
34
|
+
|
|
35
|
+
| 层 | 测试集 T | 验证集 V |
|
|
36
|
+
|---|---|---|
|
|
37
|
+
| 摘要保真 | 97.5% | 96.7% |
|
|
38
|
+
| 复述 | 38.8% | 33.3% |
|
|
39
|
+
| **探针(可用性)** | **87.5%** | **83.3%** |
|
|
40
|
+
| 漂移 | 45.0% | 50.0% |
|
|
41
|
+
| 摘要结构合规(5 段标题) | 100% | 100% |
|
|
42
|
+
|
|
43
|
+
验证集与测试集差距 ≤4.2pp,结论可外推。
|
|
44
|
+
|
|
45
|
+
## 3. flash vs pro 压缩(测试集 T,核心对比)
|
|
46
|
+
|
|
47
|
+
| 档位 | 摘要保真 | 复述 | 探针 | 漂移 | 工人成本(均值) | 单 run 耗时 |
|
|
48
|
+
|---|---|---|---|---|---|---|
|
|
49
|
+
| flash | 95.0% | 22.5% | 80.0% | 40.0% | 1,626 in / 543 out | 348s |
|
|
50
|
+
| pro | **100%** | **55.0%** | **95.0%** | 50.0% | 1,605 in / 707 out | 353s |
|
|
51
|
+
|
|
52
|
+
**结论:这是一个方差结论,不是均值结论。**
|
|
53
|
+
|
|
54
|
+
两者输入相同(同一份取材),pro 只多输出 ~30% tokens(数百个),耗时持平。但那 15pp 的均值差距**全部来自一次全灭**——把 8 个 run 拆开看:
|
|
55
|
+
|
|
56
|
+
| 档位 | run 级探针命中率 | 均值 | 标准差 |
|
|
57
|
+
|---|---|---|---|
|
|
58
|
+
| flash | 1.0, 1.0, 0.8, 0.8, **0.0**, 0.8, 1.0, 1.0 | 0.80 | 0.32 |
|
|
59
|
+
| pro | 0.8, 1.0, 1.0, 1.0, 1.0, 0.8, 1.0, 1.0 | 0.95 | 0.09 |
|
|
60
|
+
|
|
61
|
+
去掉那条 0/5,flash 是 0.91。事实级 Fisher 精确检验 p = 0.087(不显著),而且事实级计数本身高估了精度——一个 run 里的 5 个事实并不独立,全灭是 5 个一起丢的。
|
|
62
|
+
|
|
63
|
+
所以数据支持的说法不是「pro 更准」,而是:**同样的价格下 flash 的离散度大一个数量级,且它的失败是整条全灭而不是少一两个事实**。压缩环节本身极便宜(约 2K tokens/run),花同样的钱买掉这条尾巴是划算的——这才是「默认 pro」的理由。
|
|
64
|
+
|
|
65
|
+
## 4. 目标 preset 影响(测试集 T)
|
|
66
|
+
|
|
67
|
+
| 目标 | 摘要保真 | 复述 | 探针 | 漂移 |
|
|
68
|
+
|---|---|---|---|---|
|
|
69
|
+
| cordis | 100% | 45.0% | **100%** | 55.0% |
|
|
70
|
+
| code | 100% | 60.0% | 95.0% | 55.0% |
|
|
71
|
+
| standard | 95.0% | 45.0% | 90.0% | 40.0% |
|
|
72
|
+
| minimal | 95.0% | **5.0%** | **65.0%** | 30.0% |
|
|
73
|
+
|
|
74
|
+
迁入 minimal 是最弱环节;且失败有组合特征:**flash→minimal 三次 run 两次全灭**(探针命中 4/15),pro→minimal 正常(9/10)。推测 flash 摘要信息密度低 + minimal 目标上下文引导弱,叠加后目标会话"接不住"。
|
|
75
|
+
|
|
76
|
+
## 5. 源 preset 对照(C 组,控制变量验证)
|
|
77
|
+
|
|
78
|
+
| 源 | 摘要 | 复述 | 探针 | 漂移 |
|
|
79
|
+
|---|---|---|---|---|
|
|
80
|
+
| standard / code / minimal / cordis | 5/5 | 5/5 | 5/5 | 2-3/5 |
|
|
81
|
+
|
|
82
|
+
四种源 preset 下保真度完全一致——**迁移准确率与源模式无关**(摘要取材自消息文本,与源工具集无关),源 preset 无需作为风险因子。执行漂移(2-3/5)也与源无关,见 §6。
|
|
83
|
+
|
|
84
|
+
## 6. 执行偏移分析
|
|
85
|
+
|
|
86
|
+
漂移任务 = "写出启动命令(含端口)+ 核心文件首行"。漂移层 45-50% 的口径偏严:答案天然只携带端口/路径两类事实,数据库/禁令/提交语言不出现是正常的。按此口径,**端口与路径两个关键约定的实际漂移率约 20-35%**,主要形态:
|
|
87
|
+
- 路径对但端口"合理化"成 3000/8080 等常见值(摘要里有正确端口但模型补全习惯压过事实)
|
|
88
|
+
- 首行代码语言/框架与约定文件后缀不符(`.ts` 写成 `.py` 风格)
|
|
89
|
+
|
|
90
|
+
复述层普遍低(39%)要区分解读:kickoff 只要求"一段复述",模型倾向概括目标而非罗列参数;探针层(87.5%)证明事实在上下文中可用。复述层低分还受 240s 限速截断影响(25/26 的 kickoff 轮触顶,长思考被 cancel 后按已产出文本计分)。
|
|
91
|
+
|
|
92
|
+
## 7. 失败模式清单(开源 FAQ 素材)
|
|
93
|
+
|
|
94
|
+
| 模式 | 频次 | 表现 | 缓解 |
|
|
95
|
+
|---|---|---|---|
|
|
96
|
+
| flash→minimal 全灭 | 2/3 run | 探针 0/5,goal 形同未注入 | 默认 pro 压缩;GUI 可对该组合告警 |
|
|
97
|
+
| 端口合理化漂移 | 多次 | 7101→3000 等 | 摘要中数字类事实可加粗/单列 |
|
|
98
|
+
| 单轮限速截断 | 25/26 kickoff | cordis/standard 目标 kickoff 超 240s | GUI 已在跑批外,不影响结果但见 §8 |
|
|
99
|
+
| 摘要丢 1 事实 | 3/26(全 flash) | 禁令类事实最易丢 | pro 压缩 |
|
|
100
|
+
|
|
101
|
+
## 8. 附带的工程发现(超出准确率本身)
|
|
102
|
+
|
|
103
|
+
1. **cordis 会话单轮可跑飞**:无约束提示下 cordis 埋点轮会进入工具循环,单轮 >100K 事件、>10 分钟;杀掉客户端进程不会终止 host 侧轮次(本次烧掉的 ~70M tokens 的主因)。建议 GUI 侧加"单轮看门狗"。
|
|
104
|
+
2. **host 无会话删除 RPC**:只有归档;物理删除需停 host 后移出 `~/.dsh/sessions/<workspace>/` 目录。本轮 测试会话已归档并隔离,host 重启后彻底清除。
|
|
105
|
+
3. 压缩成本极低(~2K tokens/run),**token 消耗的大头永远是 agentic 会话本身**,"用 flash 省钱"在 bridge 场景不成立。
|
|
106
|
+
|
|
107
|
+
## 9. 结论
|
|
108
|
+
|
|
109
|
+
Bridge 迁移在 **pro 压缩 + 任意目标** 下达到 95-100% 探针可用性,可以发布;默认档位应保持 pro——理由是尾部风险而非均值(见 §3)。执行偏移存在但集中在"数字合理化"一类;0.2 已在压缩指令里加了「端口、版本号、数量上限必须原样单独成行抄写,不得改写成常见值」的规则,但**尚未重测**。
|
|
110
|
+
|
|
111
|
+
读这份报告前请先读 §10:这批数据有几处口径上的局限,其中两处会系统性地抬高所有数字。
|
|
112
|
+
|
|
113
|
+
*可复现:`node eval/run.mjs 3`(需 host 在线;BRIDGE_ONLY 可筛子集)。*
|
|
114
|
+
|
|
115
|
+
## 10. 方法学局限(以及 0.2 做了什么)
|
|
116
|
+
|
|
117
|
+
这批数据是在 0.1 上跑的。下面几条是它已知的局限,按「会不会影响结论」排序。0.2 修掉了工具层面的部分,但**没有重测**——所以上面的数字仍然是 0.1 的数字。
|
|
118
|
+
|
|
119
|
+
### 10.1 打分口径有非零下限(会抬高所有臂)
|
|
120
|
+
|
|
121
|
+
`score()` 是大小写不敏感的**子串匹配**,而 5 个事实里至少有 1-2 个是语义可猜的:`lang` 的期望值是字面量「中文」/「English」,而探针本身就是中文提问,回答里出现「中文」二字几乎必然;`db = PostgreSQL` 对一个电商项目、`ban = Kafka` 对一个数据管道,都是模型的默认猜测。
|
|
122
|
+
|
|
123
|
+
证据:裸重开进 minimal(无工具、无历史、真失忆)的两条 run **都恰好得 1/5**,不是 0/5。
|
|
124
|
+
|
|
125
|
+
**0.2 的做法**:评测新增 `guess` 臂(`BRIDGE_ARM=guess`)——不埋点,直接问探针,测出这个下限。所有命中率都应减去它再读。新题材进 `datasets/` 前必须报告猜测基线得分(见 CONTRIBUTING)。
|
|
126
|
+
|
|
127
|
+
### 10.2 A/B 对照臂被工作区污染(会抬高对照臂)
|
|
128
|
+
|
|
129
|
+
0.1 的所有 run 共用 `workspace.list()` 的第一个工作区,源会话的日志、甚至本仓库的 `datasets/test.json`(里面明文写着 7101 / PostgreSQL / `src/shop/orders.ts`)都在 agent 的可读范围内。这正是 bare→code 能拿到 9/10 的机制(`eval/inspect-bare.mjs` 复现了它)。
|
|
130
|
+
|
|
131
|
+
**0.2 的做法**:每个 run 建一个空的临时工作区(`workspace.create` / `workspace.delete`),跑完即销毁注册。
|
|
132
|
+
|
|
133
|
+
### 10.3 样本量只够给方向
|
|
134
|
+
|
|
135
|
+
- flash vs pro:每臂 8 个 run,结论已按方差重述(§3)。
|
|
136
|
+
- A/B:4 对。按对拆开后,**迁进 code 时两臂基本打平(5-5 / 5-4),全部差异来自 minimal 目标**——把两种目标平均成一个数会盖住机制。README 的 A/B 表已按目标 preset 拆开。
|
|
137
|
+
|
|
138
|
+
统计口径建议:run 内的 5 个事实不独立,不要把 26×5 当成 130 个独立观测;按 run 级算均值与方差,或给出置信区间(探针可用性 T 组 87.5%,95% CI 78.5–93.1)。
|
|
139
|
+
|
|
140
|
+
### 10.4 外部效度:测的是最好情况
|
|
141
|
+
|
|
142
|
+
每个 run 的埋点只有**一条**用户消息,agent 回一句确认,然后立刻迁移。于是:
|
|
143
|
+
|
|
144
|
+
- 26 组 run 的 `truncated` **全部为 false**——预算与截断逻辑一次都没被触发;
|
|
145
|
+
- `reusedCompaction` **从未为 true**——复用 compaction 底稿这条核心设计路径零覆盖;
|
|
146
|
+
- 摘要长度均值 635 字符,离 2400 字符 / 900 tokens 的预算都远,约束**从未生效**。
|
|
147
|
+
|
|
148
|
+
而插件的目标用户是「聊了很久、已经压缩过、上下文很杂」的会话。**0.2 的做法**:预算路径补了语义单元测试(超预算时最新的用户消息与最近助手结论必须还在),但**长会话数据集仍是待办**——真跑起来的分数大概率低于 87.5%。
|
|
149
|
+
|
|
150
|
+
### 10.5 复述层这个指标目前不可解释
|
|
151
|
+
|
|
152
|
+
26 条 run 里 **25 条的 kickoff 轮触到 240s 限速被 cancel**,按已产出文本计分。这个数字衡量的是「240 秒内说没说完」,不是「记不记得」。README 已不再展示它;本报告 §2 的总表保留是为了留痕。
|
|
153
|
+
|
|
154
|
+
一个后来才查清的原因:0.1 的 `goal.create` 没有传 `maxGoalRounds`,用的是上游部署默认的 **256**,而 `dsh-goal-round-driver` 会在 agent 空闲时把目标渲染成 `<goal_round>` 提示反复跑。目标会话因此进入自主循环——这也解释了 §8.1 的「cordis 会话单轮可跑飞」与目标会话累计均值 53 万 tokens。**0.2 默认只给 1 轮。**
|
|
155
|
+
|
|
156
|
+
### 10.6 可复现性
|
|
157
|
+
|
|
158
|
+
0.1 的 run id 由全局下标生成(`T01`…`V26`),往 `datasets/` 加一个题材会让所有 id 重编号,README 里的复现命令就会选中另一批用例。**0.2 改为由配置派生**(`T-pro-minimal→code-电商后端`)。历史报告里的旧 id 与新 id 不对应,这是一次性断裂。
|
|
159
|
+
|
|
160
|
+
### 10.7 下一批实验应该跑什么
|
|
161
|
+
|
|
162
|
+
1. `guess` 三臂重跑 A/B(隔离工作区 + 猜测基线),n ≥ 12 对;
|
|
163
|
+
2. 一个 `long` 数据集:20-30 轮真实工具调用、至少触发一次 `/compact` 再迁移;
|
|
164
|
+
3. 0.2 的注入改动(摘要进首轮提示)与数字防四舍五入规则的 A/B;
|
|
165
|
+
4. 复述层把 cap 提到 600s 重测,或彻底删掉这一层。
|