dsh-skill-eval 1.20.2 → 1.20.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,48 @@
1
1
  # Changelog
2
2
 
3
+ ## 1.20.3 — review hardening
4
+
5
+ A full independent review (host, client, tests/packaging, plus a second-model
6
+ pass) reproduced the issues below. Every fix has a regression test.
7
+
8
+ - **The per-turn strip counts real errors.** It read `data.isError`, but DSH's
9
+ tool-call payload is `ToolChatData { root }`; errors now come from
10
+ `root.isError`, and the strip retries the report when the 10 s cache predates
11
+ the turn (rapid consecutive turns used to render nothing).
12
+ - **Settings writes cannot lose a field.** Both HTTP paths now merge inside the
13
+ serialized write queue (`updateSettings`); two overlapping partial saves keep
14
+ both keys. `windowTtlMs`, `reportOnTurnEnd`, `turnReports` and `promptHint`
15
+ are read live, so "saved values apply immediately" is true.
16
+ - **A failed `skill` load opens no evidence window** and no longer supersedes the
17
+ previous skill's window; the test now asserts after turn-stop/dispose.
18
+ - **Corrupt window records cannot crash reporting.** Nested `mcp` buckets and
19
+ tool entries are normalised on read (plus defence-in-depth guards in the
20
+ score/report merge), and finite-but-invalid timestamps render as `—`.
21
+ - **The release checks can fail.** `canonical-route` fails on restart skew,
22
+ `turn-boards` fails with zero boards, manual rows are INFO and excluded from
23
+ the PASS count, and `weightsPercent` uses largest-remainder so it always sums
24
+ to 100 (2/2/2/1 used to show 101 and fail verify-live).
25
+ - **The accept CLI is honest.** It reads `dataDir` from the served report
26
+ (previous `JSON.parse` of an object was silently swallowed), and flag values
27
+ no longer become the base URL.
28
+ - **The packaging doctor passes in an installed tarball** (no `client/` sources
29
+ to compare), requires locale/scripts/README/CHANGELOG/LICENSE in `files`, and
30
+ `--release` opens the tarball instead of only checking that it exists.
31
+ - **Client degradation works.** Versioned data/settings routes that answer
32
+ 200 HTML (DSH's prefix route) now fall back to the canonical JSON routes
33
+ instead of showing a parse error; the stale banner is reachable again.
34
+ - **Per-turn boards carry the settled turn** (session+turn passed from
35
+ `agent/turn-stopping`), coalesced writes emit one board per turn, snapshots
36
+ are written atomically, and history/impact use the shared timestamp guard.
37
+ - **Security/hygiene:** the settings fence also rejects rebinding Hosts (not
38
+ only a mismatched Origin), 403/413 paths and DNS rebinding have tests,
39
+ analyzer/listener disposers are released on teardown, the scheduler guards
40
+ against writes after dispose, duplicate `/skill-name` invocations collapse on
41
+ the durable read path, the dev tab renders the settings file/warning count,
42
+ the settings form keeps in-flight edits and labels its controls, and the
43
+ score colours follow the configured grade floors.
44
+ - Tests: **258 passing** (`npm test`, jsdom installed).
45
+
3
46
  ## 1.20.2 — MCP board and settings readability
4
47
 
5
48
  - **MCP tab rebuilt.** The old tab was one SVG chart with two thin bars plus a
package/README.md CHANGED
@@ -1,5 +1,6 @@
1
1
  # dsh-skill-eval
2
2
 
3
+ [![npm](https://img.shields.io/npm/v/dsh-skill-eval)](https://www.npmjs.com/package/dsh-skill-eval)
3
4
  [![license: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
4
5
  [![latest release](https://img.shields.io/github/v/release/carlslin/dsh-skill-eval?display_name=tag)](https://github.com/carlslin/dsh-skill-eval/releases)
5
6
 
@@ -32,8 +33,8 @@ is a DSH plugin rather than a CLI.
32
33
 
33
34
  ## Screenshots
34
35
 
35
- | Skills board | MCP tab | Scoring settings |
36
- | --- | --- | --- |
36
+ | Skills board | MCP tab | Scoring settings |
37
+ | ----------------------------------------- | ---------------------------------------- | -------------------------------------------------- |
37
38
  | ![skills board](docs/ui/board-skills.png) | ![MCP tab](docs/ui/board-mcp-1.20.2.png) | ![config dialog](docs/ui/config-dialog-1.20.2.png) |
38
39
 
39
40
  More real-window screenshots and the per-screen notes:
@@ -80,7 +81,7 @@ switches: [docs/ENTRYPOINTS.md](docs/ENTRYPOINTS.md).
80
81
  ```bash
81
82
  dsh-skill-eval-doctor # packaging pre-flight (release gate with --release)
82
83
  dsh-skill-eval-accept # generation & data contract (17 checks)
83
- dsh-skill-eval-verify # product surfaces (9 checks, after a reload)
84
+ dsh-skill-eval-verify # product surfaces (7 checks + 2 manual, after a reload)
84
85
  ```
85
86
 
86
87
  ## Install
@@ -88,6 +89,12 @@ dsh-skill-eval-verify # product surfaces (9 checks, after a reload)
88
89
  Requirements: DeepSeek Harness with plugin bundles (the package declares
89
90
  `dsh.engines.dsh >= 0.2.0`) and Node.js ≥ 20 for the CLI helpers.
90
91
 
92
+ From npm:
93
+
94
+ ```bash
95
+ dsh plugin --profile desktop add dsh-skill-eval
96
+ ```
97
+
91
98
  From GitHub source:
92
99
 
93
100
  ```bash
@@ -159,7 +166,7 @@ the matching `dist/` artifact.
159
166
  Editing `client/*.js` requires rebuilding the single-file browser bundle:
160
167
 
161
168
  ```bash
162
- npm run build # format check → lint → client build → doctor → 241 tests
169
+ npm run build # format check → lint → client build → doctor → 258 tests
163
170
  npm run build:fast # client build + doctor only
164
171
  npm run build:pack # full build + dist/dsh-skill-eval-<version>.tgz
165
172
  npm run build:install # pack + install into the desktop profile
@@ -174,7 +181,7 @@ npm run build:install # pack + install into the desktop profile
174
181
  node --test test/
175
182
  ```
176
183
 
177
- 241 tests (`npm test` rebuilds the browser bundle first): scoring bands, static checks, correction detection,
184
+ 258 tests (`npm test` rebuilds the browser bundle first): scoring bands, static checks, correction detection,
178
185
  window lifecycle (turn-end snapshot → correction → close), store round-trip with
179
186
  a torn line, analyzer join, markdown/HTML rendering incl. escaping and bilingual
180
187
  advice copy, host route registration under duplicate-route pressure, and the
@@ -198,4 +205,4 @@ client surfaces mounted in jsdom.
198
205
  - Loads are recorded from `skill` tool calls and user-typed `/skill-name` invocations; a skill the harness activates without either is invisible.
199
206
  - Effectiveness is inferred from the next user message, not from a task verdict.
200
207
  - Advice evidence keeps measured ids/numbers in English even in the Chinese UI.
201
- - Telemetry grows without pruning (one small record pair per load).
208
+ - Telemetry grows without pruning: each load writes a skill/window pair, each resolved skill also a `static` profile, and every agent step one `step` row (the dominant growth in long sessions).
package/README.zh.md CHANGED
@@ -1,5 +1,6 @@
1
1
  # dsh-skill-eval(技能评测)
2
2
 
3
+ [![npm](https://img.shields.io/npm/v/dsh-skill-eval)](https://www.npmjs.com/package/dsh-skill-eval)
3
4
  [![license: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
4
5
  [![latest release](https://img.shields.io/github/v/release/carlslin/dsh-skill-eval?display_name=tag)](https://github.com/carlslin/dsh-skill-eval/releases)
5
6
 
@@ -30,8 +31,8 @@
30
31
 
31
32
  ## 截图
32
33
 
33
- | 技能看板 | MCP 标签 | 评分配置 |
34
- | --- | --- | --- |
34
+ | 技能看板 | MCP 标签 | 评分配置 |
35
+ | ------------------------------------- | ----------------------------------------- | --------------------------------------------- |
35
36
  | ![技能看板](docs/ui/board-skills.png) | ![MCP 标签](docs/ui/board-mcp-1.20.2.png) | ![评分配置](docs/ui/config-dialog-1.20.2.png) |
36
37
 
37
38
  更多真实窗口截图与逐屏记录:[docs/UI-REVIEW.md](docs/UI-REVIEW.md)。
@@ -68,7 +69,7 @@
68
69
  ```bash
69
70
  dsh-skill-eval-doctor # 打包预检(--release 为发布门禁)
70
71
  dsh-skill-eval-accept # 数据契约与生成检查(17 项)
71
- dsh-skill-eval-verify # 产品面可达(9 项,重载后)
72
+ dsh-skill-eval-verify # 产品面可达(7 项 + 2 项手动,重载后)
72
73
  ```
73
74
 
74
75
  ## 安装
@@ -76,6 +77,12 @@ dsh-skill-eval-verify # 产品面可达(9 项,重载后)
76
77
  要求:带插件 bundle 能力的 DeepSeek Harness(包内声明 `dsh.engines.dsh >= 0.2.0`),
77
78
  CLI 辅助脚本需要 Node.js ≥ 20。
78
79
 
80
+ 从 npm 安装:
81
+
82
+ ```bash
83
+ dsh plugin --profile desktop add dsh-skill-eval
84
+ ```
85
+
79
86
  从 GitHub 源码安装:
80
87
 
81
88
  ```bash
@@ -132,7 +139,7 @@ registerReviewTool: true # skill_eval_review:泄漏审查 + 审计账本
132
139
  node --test test/
133
140
  ```
134
141
 
135
- 241 项(`npm test` 会先重建客户端 bundle):评分分档、静态检查、纠正识别、窗口生命周期(回合结束快照 →
142
+ 258 项(`npm test` 会先重建客户端 bundle):评分分档、静态检查、纠正识别、窗口生命周期(回合结束快照 →
136
143
  纠正 → 关闭)、带残缺行的存储往返、分析器 join、Markdown/HTML 渲染(含转义与中文
137
144
  建议文案)、重复路由压力下的宿主路由注册,以及客户端的 jsdom 挂载测试。
138
145