dsh-skill-eval 0.0.0-stage → 1.20.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +675 -0
- package/LICENSE +21 -0
- package/README.md +200 -2
- package/README.zh.md +157 -0
- package/client.js +2637 -0
- package/cordis.patch.yml +9 -0
- package/docs/ACCEPTANCE.md +71 -0
- package/docs/AUDIT.md +64 -0
- package/docs/CODE-REVIEW.md +49 -0
- package/docs/COMPARISON.md +33 -0
- package/docs/DETAIL.md +463 -0
- package/docs/DEV-RELOAD.md +114 -0
- package/docs/ENTRYPOINTS.md +65 -0
- package/docs/RELEASE.md +44 -0
- package/docs/RRSI-NOTES.md +129 -0
- package/docs/SCENARIOS.md +91 -0
- package/docs/UI-REVIEW.md +115 -0
- package/docs/WORKFLOW.md +108 -0
- package/icon.svg +9 -0
- package/index.js +1111 -0
- package/lib/accept.js +350 -0
- package/lib/advice.js +547 -0
- package/lib/critic.js +101 -0
- package/lib/dev.js +148 -0
- package/lib/history.js +140 -0
- package/lib/html.js +8 -0
- package/lib/inspect.js +142 -0
- package/lib/integrity.js +46 -0
- package/lib/loop.js +195 -0
- package/lib/random.js +29 -0
- package/lib/recorder.js +634 -0
- package/lib/render.js +582 -0
- package/lib/report.js +646 -0
- package/lib/score.js +525 -0
- package/lib/settings.js +396 -0
- package/lib/snapshot.js +166 -0
- package/lib/steps.js +141 -0
- package/lib/store.js +186 -0
- package/lib/validity.js +115 -0
- package/locale/en.json +6 -0
- package/locale/zh.json +6 -0
- package/package.json +96 -4
- package/scripts/accept.mjs +226 -0
- package/scripts/build-client.mjs +59 -0
- package/scripts/build.mjs +81 -0
- package/scripts/doctor.mjs +199 -0
- package/scripts/lib.mjs +52 -0
- package/scripts/verify-live.mjs +168 -0
package/docs/DETAIL.md
ADDED
|
@@ -0,0 +1,463 @@
|
|
|
1
|
+
# dsh-skill-eval — design detail
|
|
2
|
+
|
|
3
|
+
## Why this exists
|
|
4
|
+
|
|
5
|
+
Every published skill-evaluation tool we looked at is a **static, offline** checker:
|
|
6
|
+
|
|
7
|
+
| Project | What it does | What it cannot see |
|
|
8
|
+
| --------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------- |
|
|
9
|
+
| `aws-samples/sample-agent-skill-eval` | audit (100 − deductions), functional/trigger runs through Claude CLI, weighted report 40/40/20, `.skilleval.yaml` | nothing about the skill as it is actually used in this harness |
|
|
10
|
+
| `NVIDIA/SkillEvaluator` | T1 static validation (SkillSpector, Gitleaks, Semgrep), T2 embedding dedup, T3 Docker-sandboxed live eval with an LLM rubric | needs a sandboxed re-run, not the real session |
|
|
11
|
+
| `darkrishabh/agent-skills-eval`, `alibaba/skill-up`, `adewale/skill-eval-harness` | test runners / evolution loops over fixtures | no runtime attachment to a live agent |
|
|
12
|
+
|
|
13
|
+
They all answer "is this skill file well-formed?". None of them can answer
|
|
14
|
+
"when this agent loaded this skill in this session, did it actually help?".
|
|
15
|
+
That second question is what this plugin measures, because inside DSH it can sit
|
|
16
|
+
on the real event stream.
|
|
17
|
+
|
|
18
|
+
The weighting and banding ideas (multi-dimensional rubric, A–F bands, a ~70
|
|
19
|
+
pass line) are borrowed from those projects. No code is copied.
|
|
20
|
+
|
|
21
|
+
## What is recorded
|
|
22
|
+
|
|
23
|
+
Two append-only JSONL record types live in `$DSH_HOME/skill-eval/$DSH_PROFILE/events.jsonl`
|
|
24
|
+
(override with `dataDir` or `DSH_SKILL_EVAL_DIR`). The session log is **never**
|
|
25
|
+
written to — plugins must not append foreign event types to it.
|
|
26
|
+
|
|
27
|
+
```jsonc
|
|
28
|
+
// written when a `skill` tool call completes
|
|
29
|
+
{ "t": "skill", "v": 1, "at": 1763, "sessionId": "…", "callId": "…",
|
|
30
|
+
"skill": "pdf-tools", "ok": true, "loadMs": 37, "windowId": "…" }
|
|
31
|
+
|
|
32
|
+
// written at turn end and again when the window closes
|
|
33
|
+
{ "t": "window", "v": 1, "at": 1763, "windowId": "…", "sessionId": "…",
|
|
34
|
+
"skill": "pdf-tools", "toolCalls": 4, "errors": 1, "errorNames": ["ToolError"],
|
|
35
|
+
"corrections": ["不对,页眉没删掉"], "otherSkills": ["docx"], "turnEnded": true,
|
|
36
|
+
"truncated": false, "closeReason": "user-turn" }
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
A **window** is the stretch of work that follows one skill load. It is the unit
|
|
40
|
+
of evidence. Readers keep the _last_ record per `windowId`, so the turn-end write
|
|
41
|
+
is an update and the close-time write is a second update — durability without
|
|
42
|
+
duplicates.
|
|
43
|
+
|
|
44
|
+
Window lifetime, deliberately unusual:
|
|
45
|
+
|
|
46
|
+
- opens on a successful `skill` load
|
|
47
|
+
- **stays open across the turn boundary** — the evidence that a skill failed is
|
|
48
|
+
usually the next user message ("no, that's wrong"), so closing at turn end
|
|
49
|
+
would throw away the strongest signal. The turn-end write only persists a
|
|
50
|
+
snapshot.
|
|
51
|
+
- closes on: the next user message (correction recorded first), a new skill load
|
|
52
|
+
in the same session (the previous skill records the new one as a sibling), TTL
|
|
53
|
+
expiry (`windowTtlMs`, default 30 min), session eviction, or plugin dispose.
|
|
54
|
+
|
|
55
|
+
## Scoring
|
|
56
|
+
|
|
57
|
+
```
|
|
58
|
+
total = Σ(weightᵢ × dimensionᵢ) / Σ(weightᵢ over dimensions with evidence)
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
| dimension | weight | source |
|
|
62
|
+
| ------------- | ------ | --------------------------------------------------------------- |
|
|
63
|
+
| static | 0.30 | the live skill catalog (`skills.list` + `skills.get`) |
|
|
64
|
+
| reliability | 0.25 | load success × post-load tool error rate |
|
|
65
|
+
| effectiveness | 0.30 | user corrections, ignored loads, sibling collisions, error rate |
|
|
66
|
+
| efficiency | 0.15 | average tool calls per run, median load latency |
|
|
67
|
+
|
|
68
|
+
A dimension with no evidence is dropped and its weight redistributed, so a
|
|
69
|
+
brand-new skill is not punished for having no history; `provisional` marks
|
|
70
|
+
scores below `minSamples` loads.
|
|
71
|
+
|
|
72
|
+
### static checks (117 points, normalised)
|
|
73
|
+
|
|
74
|
+
Twelve weighted checks; checks that cannot be evaluated (no filesystem view,
|
|
75
|
+
unresolved references) are marked skipped and dropped from both earned and
|
|
76
|
+
total weight, so the static score is always a percentage of what was checkable.
|
|
77
|
+
|
|
78
|
+
| check | weight | rule |
|
|
79
|
+
| ---------------------- | ------ | ------------------------------------------------------- |
|
|
80
|
+
| desc-length | 12 | description length within the useful band |
|
|
81
|
+
| desc-trigger | 12 | description says _when_ to use the skill |
|
|
82
|
+
| when-to-use | 8 | `whenToUse` present |
|
|
83
|
+
| body-present | 10 | SKILL.md body resolves and is non-trivial |
|
|
84
|
+
| body-compact | 18 | ≤ 500 body lines (`1 − (lines − 500)/1000` beyond that) |
|
|
85
|
+
| sections | 10 | ≥ 2 `##` headings |
|
|
86
|
+
| progressive-disclosure | 15 | delegates to `references/`, `scripts/` or `assets/` |
|
|
87
|
+
| no-placeholder | 10 | no TODO/TBD/FIXME/XXX |
|
|
88
|
+
| has-example | 5 | at least one fenced code block |
|
|
89
|
+
| anti-trigger | 5 | states when _not_ to use it |
|
|
90
|
+
| skill-name | 4 | kebab-case skill id |
|
|
91
|
+
| reference-integrity | 8 | every referenced file exists |
|
|
92
|
+
|
|
93
|
+
### runtime dimensions
|
|
94
|
+
|
|
95
|
+
```
|
|
96
|
+
reliability = 100 · loadOkRate · (0.6 + 0.4 · (1 − min(1, 2·errRate)))
|
|
97
|
+
effectiveness = 100 − 45·correctionRate − 25·abandonRate − 20·siblingRate − 10·errRate
|
|
98
|
+
efficiency = 100 − (avgCalls > 8 ? min(40, (avgCalls−8)·6) : 0) − (avgCalls < 1 ? 15 : 0)
|
|
99
|
+
− (p50LoadMs > 2000 ? min(30, (p50LoadMs−2000)/100) : 0)
|
|
100
|
+
− (tokensPerLoad > budget ? min(35, (tokensPerLoad−budget)/budget·30) : 0)
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
A failed load multiplies reliability instead of only halving it: a skill that
|
|
104
|
+
cannot be loaded has zero reliability, not 40/100.
|
|
105
|
+
|
|
106
|
+
Bands: `A ≥ 90`, `B ≥ 80`, `C ≥ 70`, `D ≥ 60`, `F < 60` (configurable via
|
|
107
|
+
`thresholds`).
|
|
108
|
+
|
|
109
|
+
## Advice
|
|
110
|
+
|
|
111
|
+
`lib/advice.js` is a rule engine over the same numbers. Every rule emits
|
|
112
|
+
`{id, severity, dimension, title, evidence, fix}` where `evidence` is the measured
|
|
113
|
+
value (or the user's own words) and `fix` is an instruction the agent can execute
|
|
114
|
+
against the skill file. Rules:
|
|
115
|
+
|
|
116
|
+
| id | severity | trigger |
|
|
117
|
+
| ------------------------------------------------------------------------- | -------- | ---------------------------- |
|
|
118
|
+
| load-failed | critical | ≥ 1 failed load |
|
|
119
|
+
| post-load-errors | critical | errRate > 25 % |
|
|
120
|
+
| user-corrections | critical | correctionRate > 30 % |
|
|
121
|
+
| dead-load | warning | abandonRate > 30 % |
|
|
122
|
+
| trigger-collision | warning | siblingRate > 30 % |
|
|
123
|
+
| verbose-execution | info | avgCalls > 8 |
|
|
124
|
+
| slow-load | info | p50LoadMs > 2000 |
|
|
125
|
+
| body-too-long / weak-description / no-trigger-language / placeholder-text | warning | matching static check |
|
|
126
|
+
| no-progressive-disclosure / no-example | info | matching static check |
|
|
127
|
+
| unresolved-skill / low-sample | info | catalog miss / too few loads |
|
|
128
|
+
|
|
129
|
+
## Surfaces
|
|
130
|
+
|
|
131
|
+
- **Client seats** (`client.js`): the result strip under a completed turn
|
|
132
|
+
(`conversation.chat.turnTail`), the sidebar panel icon plus its main board
|
|
133
|
+
panel (`sidebar.panellist` + `main`), and the native settings form on the
|
|
134
|
+
bundle page (`plugins.bundle.config`). An opt-in debug popup
|
|
135
|
+
(`window.__SEV_ENABLE_POPUP__ = true`) additionally registers
|
|
136
|
+
`sidebar.footer.action` + `shell.overlay`. All surfaces read the same host
|
|
137
|
+
route, so the tool, the board and the popup show the same numbers.
|
|
138
|
+
- `skill_eval_report` tool — `{skill?, format?: 'markdown'|'json'}`. Registered
|
|
139
|
+
so the agent can read its own numbers and cite them; a system-prompt section
|
|
140
|
+
tells it to.
|
|
141
|
+
- `/skill-eval [skill]` command — the same report in the composer.
|
|
142
|
+
- **Turn snapshot** — after a turn settles, a debounced write puts the whole
|
|
143
|
+
aggregated report at `<dataDir>/reports/latest.md` and `latest.json`. Zero
|
|
144
|
+
tokens: it never enters the model context, it is just a file you (or the
|
|
145
|
+
agent, on request) can read.
|
|
146
|
+
- `GET /skill-eval` — HTML dashboard served through `webServer` (no build step,
|
|
147
|
+
self-contained, light/dark). `GET /skill-eval/data` returns the raw report
|
|
148
|
+
JSON, and is registered as its own **exact** route so it survives the page
|
|
149
|
+
route failing to re-register after a reload.
|
|
150
|
+
|
|
151
|
+
A Client half loads only when the host scans the package at boot, so installing
|
|
152
|
+
or changing `client.js` needs one DSH restart before the button appears.
|
|
153
|
+
|
|
154
|
+
## Upgrading without breaking the dashboard
|
|
155
|
+
|
|
156
|
+
The host keeps `webServer` routes registered by a disposed plugin generation, so
|
|
157
|
+
after a hot reload the canonical `/skill-eval/data` can still be answered by the
|
|
158
|
+
old handler while the new module is running. Two measures keep that from
|
|
159
|
+
silently serving stale numbers:
|
|
160
|
+
|
|
161
|
+
1. the data API is registered both canonically and under the package version
|
|
162
|
+
(`/skill-eval/data/<version>`), and the versioned path is new on every
|
|
163
|
+
release, so a reload can always serve current data;
|
|
164
|
+
2. the popup asks the versioned path first and falls back to the canonical one
|
|
165
|
+
on 404, and a test pins the client constant to `package.json.version`.
|
|
166
|
+
|
|
167
|
+
`/skill-eval accept` runs the in-process half of the same checks from the
|
|
168
|
+
composer (route shape, boards, intervals, snapshot generation, telemetry), and
|
|
169
|
+
`node scripts/accept.mjs` checks both routes and reports a generation mismatch
|
|
170
|
+
instead of leaving you to guess. The popup does the same from inside: when the
|
|
171
|
+
payload it receives has no `scoring`/`ranking` fields it shows a red banner
|
|
172
|
+
("the running host serves an older plugin generation … restart DeepSeek
|
|
173
|
+
Harness"), so a stale process is visible where the user actually looks.
|
|
174
|
+
|
|
175
|
+
## Agent-loop wiring
|
|
176
|
+
|
|
177
|
+
Which loop stages feed the evidence, and which do not:
|
|
178
|
+
|
|
179
|
+
| loop stage | event | what the plugin does |
|
|
180
|
+
| --------------------- | --------------------------------- | -------------------------------------------------------------------------------------------------------------- |
|
|
181
|
+
| loop step | `agent/pre-step` (waterfall) | records the step number (parallel vs switched loads), then `next()` untouched |
|
|
182
|
+
| tool dispatch start | `tools/pre-execute` (waterfall) | timestamps `skill` loads only, then `next()` untouched |
|
|
183
|
+
| tool dispatch end | `tools/result` (emit) | call/error counting per open window, skill load records, static capture |
|
|
184
|
+
| user turn opens | `agent/inbox/claimed` (emit) | reads the message for correction language, then closes the windows |
|
|
185
|
+
| turn about to close | `agent/turn-stopping` (serial) | persists a snapshot, keeps the window open for the follow-up message |
|
|
186
|
+
| step/turn error | `agent/error` (emit) | `loopErrors` + `lastLoopError` on every open window; costs up to 40% of reliability and raises a critical rule |
|
|
187
|
+
| model request failure | `agent/request-error` (waterfall) | `llmRetries` per window, then `next()` untouched (the retry policy stays the host's) |
|
|
188
|
+
|
|
189
|
+
Token accounting **is** wired (`countTokens`): provider usage is attributed to
|
|
190
|
+
the open windows and `tokensPerLoad` feeds the efficiency dimension and the
|
|
191
|
+
token-cost advice. Deliberately **not** wired yet:
|
|
192
|
+
|
|
193
|
+
- **compaction** — a mid-window compaction does not break attribution, but the
|
|
194
|
+
window does not record that the model's context was rewritten.
|
|
195
|
+
- **turn truncation from max tokens** — a turn that dies on the output limit is
|
|
196
|
+
currently only visible through the following correction or error path.
|
|
197
|
+
|
|
198
|
+
Every listener above is read-only except the two waterfalls, which always call
|
|
199
|
+
`next()`, and every body is wrapped so telemetry can never fail a turn.
|
|
200
|
+
|
|
201
|
+
## Concurrency and multi-agent work
|
|
202
|
+
|
|
203
|
+
Measured on the live host with the `workflow` tool (two `agent()` children in one
|
|
204
|
+
parallel batch, each loading the same skill):
|
|
205
|
+
|
|
206
|
+
- every child session produced its own `skill` record (distinct `sessionId`,
|
|
207
|
+
its own `loadMs`) plus its own `window` and `static` record — a root-scope
|
|
208
|
+
`tools/result` listener does receive child-agent dispatches;
|
|
209
|
+
- the two loads happened ~8 s apart and did not interfere, because recorder
|
|
210
|
+
state is keyed by session id;
|
|
211
|
+
- static authoring quality resolved inside the child agent's scope
|
|
212
|
+
(`skills.list({ scope: agent })`), so subagent loads are scored on the same
|
|
213
|
+
four dimensions as root loads.
|
|
214
|
+
|
|
215
|
+
Model rules that follow from that:
|
|
216
|
+
|
|
217
|
+
- **One window per load, several per session.** A load in a _later_ loop step
|
|
218
|
+
closes the previous window as `superseded` and records the newcomer in
|
|
219
|
+
`otherSkills` (that is the trigger-collision signal). Two loads inside the
|
|
220
|
+
_same_ loop step are parallel use: both windows stay open, and the follow-up
|
|
221
|
+
tool calls count for each of them, because one batch of work is not one
|
|
222
|
+
winner. The step comes from `agent/pre-step`.
|
|
223
|
+
- **Aggregation is per skill, across every session.** A skill loaded by a
|
|
224
|
+
subagent contributes to the same skill entry as a root load. The raw JSONL
|
|
225
|
+
keeps `sessionId`/`step` on every record, so a per-agent breakdown is a report
|
|
226
|
+
projection away, not a recording gap.
|
|
227
|
+
- **Correction evidence is weaker in child sessions.** The next user message in
|
|
228
|
+
a child is usually the parent agent's instruction, not human pushback, so the
|
|
229
|
+
correction rate mostly stays at zero there; effectiveness for subagent runs
|
|
230
|
+
leans on tool-error and abandoned-load rates instead.
|
|
231
|
+
- An interrupted child can leave its window open until the TTL sweep, which
|
|
232
|
+
shows up as an open window rather than a closed one. It is never attributed to
|
|
233
|
+
unrelated later work.
|
|
234
|
+
|
|
235
|
+
### MCP thresholds are configuration
|
|
236
|
+
|
|
237
|
+
The three MCP rules use configurable limits so a deployment can be stricter than
|
|
238
|
+
the defaults (`20%` server error rate, `3000 ms` p95, `30%` single-tool error
|
|
239
|
+
rate):
|
|
240
|
+
|
|
241
|
+
```yaml
|
|
242
|
+
config:
|
|
243
|
+
mcpErrorRatePercent: 20
|
|
244
|
+
mcpToolErrorRatePercent: 30
|
|
245
|
+
mcpP95Ms: 3000
|
|
246
|
+
```
|
|
247
|
+
|
|
248
|
+
### Validating with real MCP traffic
|
|
249
|
+
|
|
250
|
+
1. Restart DSH once so the current plugin generation loads (profile plugins are
|
|
251
|
+
imported at boot), then run `node scripts/accept.mjs [baseUrl]`: it checks the
|
|
252
|
+
JSON route shape, the scoring block, the MCP/runs boards, the snapshot files
|
|
253
|
+
and the recorded telemetry, and prints the restart hint when the host is still
|
|
254
|
+
serving an older generation.
|
|
255
|
+
2. Run a session that uses an MCP tool, e.g. "use context7 to look up the Rails
|
|
256
|
+
routing API".
|
|
257
|
+
3. Open the popup → **MCP** tab: the server should appear with calls, error rate,
|
|
258
|
+
p50/p95, health and any per-tool failures.
|
|
259
|
+
4. `cordis_inspect_query { provider: "skill-eval", method: "mcp" }` returns the
|
|
260
|
+
same board for scripting/agents.
|
|
261
|
+
|
|
262
|
+
A headless run (`dsh --profile <p> "task"`) can produce the same evidence, but it
|
|
263
|
+
needs model credentials in that profile (`DEEPSEEK_API_KEY`), which is why the
|
|
264
|
+
in-session validation above is done on the desktop host.
|
|
265
|
+
|
|
266
|
+
## Run scores (every execution gets a grade)
|
|
267
|
+
|
|
268
|
+
The skill score grades an artifact; a **run score** grades one execution (one
|
|
269
|
+
session) so "how did this run go?" has an answer too. It is built from what the
|
|
270
|
+
session actually did, not from a re-run:
|
|
271
|
+
|
|
272
|
+
| dimension | weight | formula |
|
|
273
|
+
| ---------- | ------ | ------------------------------------------------------------------------ |
|
|
274
|
+
| quality | 0.40 | mean score of the skills this run loaded |
|
|
275
|
+
| stability | 0.30 | `100 − 60·toolErrorRate − 40·loopErrorRate` |
|
|
276
|
+
| outcome | 0.20 | `100 − 45·correctionRate` |
|
|
277
|
+
| efficiency | 0.10 | `100 − max(0, callsPerLoad − 8)·6`, capped at 45 when `callsPerLoad < 1` |
|
|
278
|
+
|
|
279
|
+
Weights differ from the skill score on purpose: a run is a sample of work, so
|
|
280
|
+
outcome and stability carry more than authoring quality. Each run score carries
|
|
281
|
+
`confidence` and `provisional` (fewer loads than `minSamples`), and the
|
|
282
|
+
`efficiency` cap stops "loaded a skill and did nothing" from looking perfect.
|
|
283
|
+
Run scores appear in the **Runs** board (popup), the markdown/HTML reports, the
|
|
284
|
+
`runs` inspect method and `/skill-eval runs`.
|
|
285
|
+
|
|
286
|
+
## Developer mode: Cordis Inspect provider
|
|
287
|
+
|
|
288
|
+
`cordis_inspect_list` / `cordis_inspect_query` are the harness's read-only
|
|
289
|
+
developer capabilities. The plugin registers its own Host provider (default on,
|
|
290
|
+
`inspectProvider: false` to disable), so eval data is queryable next to the
|
|
291
|
+
harness's own `Service`/`Event`/`Config` providers:
|
|
292
|
+
|
|
293
|
+
```jsonc
|
|
294
|
+
// cordis_inspect_query { platform: "host", provider: "skill-eval", method: "summary" }
|
|
295
|
+
{ "totals": {...}, "scoring": {...},
|
|
296
|
+
"skills": [{ "skill": "pdf-tools", "score": 88, "grade": "B", "confidence": "medium",
|
|
297
|
+
"dimensions": {...}, "advice": [{ "id": "user-corrections", "severity": "critical" }] }] }
|
|
298
|
+
```
|
|
299
|
+
|
|
300
|
+
| method | returns |
|
|
301
|
+
| ---------- | ---------------------------------------------------------------------------------------------- |
|
|
302
|
+
| `summary` | every skill: score, grade, confidence, four dimensions, advice identities (no prose — bounded) |
|
|
303
|
+
| `skill` | one skill in full, including advice text; errors when nothing is recorded |
|
|
304
|
+
| `mcp` | per-server calls, errors, error rate, p50/p95 latency, health, per-tool breakdown, MCP fixes |
|
|
305
|
+
| `edits` | screened skill edits from the review ledger (hypothesis, dimension, verdict, diff hash) |
|
|
306
|
+
| `runs` | recent per-session runs |
|
|
307
|
+
| `standard` | active weights, grade floors, minimum samples |
|
|
308
|
+
|
|
309
|
+
The registration is a real `cordisInspect.register({ manifest, query })` call:
|
|
310
|
+
the manifest carries `id`, `description` and methods with descriptions and JSON
|
|
311
|
+
schemas, and the query handler is read-only (it reuses the same analyzer as the
|
|
312
|
+
tool, the popup and the HTTP route, so every surface agrees by construction).
|
|
313
|
+
|
|
314
|
+
Text surfaces got the same views without leaving the conversation:
|
|
315
|
+
`skill_eval_report { view: "mcp" | "runs" | "standard" }` and
|
|
316
|
+
`/skill-eval mcp|runs|standard`.
|
|
317
|
+
|
|
318
|
+
## Changing the grading standard
|
|
319
|
+
|
|
320
|
+
The standard is configuration, not code — a user can re-weight the four
|
|
321
|
+
dimensions and move the grade bands without touching the plugin:
|
|
322
|
+
|
|
323
|
+
```yaml
|
|
324
|
+
config:
|
|
325
|
+
weights: { static: 30, reliability: 25, effectiveness: 30, efficiency: 15 }
|
|
326
|
+
thresholds: { excellent: 90, good: 80, pass: 70 }
|
|
327
|
+
minSamples: 3
|
|
328
|
+
```
|
|
329
|
+
|
|
330
|
+
Rules the analyzer enforces so a hand-typed number cannot corrupt a score:
|
|
331
|
+
|
|
332
|
+
- weights are clamped to positive numbers and **normalised to 100%**; the
|
|
333
|
+
report shows the normalised values under `scoring.weightsPercent`;
|
|
334
|
+
- a weight of `0` drops that dimension entirely (weight redistribution already
|
|
335
|
+
handled it) — e.g. `static: 0` grades pure runtime behaviour;
|
|
336
|
+
- all-zero weights fall back to the defaults instead of dividing by zero;
|
|
337
|
+
- thresholds are re-ordered (`excellent ≥ good ≥ pass`), D stays 60, and any
|
|
338
|
+
value outside `1..100` falls back to the default.
|
|
339
|
+
|
|
340
|
+
Optional external basis: `normalizeWeights(input)` accepts either fractions
|
|
341
|
+
(`0.3`) or percentages (`30`), so a workflow can compute weights elsewhere.
|
|
342
|
+
|
|
343
|
+
## Is the score valid?
|
|
344
|
+
|
|
345
|
+
The score is a _diagnostic index_, not a benchmark. Each dimension is anchored to
|
|
346
|
+
one observable signal, and the suite pins the direction of every dependency:
|
|
347
|
+
|
|
348
|
+
| dimension | anchored to | deliberately not |
|
|
349
|
+
| ----------------- | ------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------- |
|
|
350
|
+
| static 30% | the skill definition itself (description, trigger language, sections, size, examples, progressive disclosure) | whether the content is _correct_ |
|
|
351
|
+
| reliability 25% | load success × post-load tool error rate × loop-error rate | error severity (a refused write and a flaky network count the same) |
|
|
352
|
+
| effectiveness 30% | user corrections, ignored loads, sibling collisions | task outcome (there is no ground truth per run) |
|
|
353
|
+
| efficiency 15% | tool calls per run, load latency, billable tokens per load | exact provider billing reconciliation |
|
|
354
|
+
|
|
355
|
+
Every skill score also carries a **95% bootstrap interval** over its recorded
|
|
356
|
+
loads (`score.ci = {low, high, resamples}`): load-level samples are resampled
|
|
357
|
+
with replacement 200 times, each draw is re-aggregated and re-scored, and the
|
|
358
|
+
2.5/97.5 percentiles become the interval. That is why the report can say
|
|
359
|
+
"88.0 (79-94)" instead of pretending a 5-load point estimate is exact. The
|
|
360
|
+
resample count is configurable (`ciResamples`, 0 disables it), the RNG is seeded
|
|
361
|
+
per skill so the same data always yields the same interval, and fewer than two
|
|
362
|
+
loads yields no interval at all.
|
|
363
|
+
|
|
364
|
+
Intervals also drive a **ranking caveat**: when two adjacent skills' intervals
|
|
365
|
+
overlap and their totals are within 10 points, the report says so
|
|
366
|
+
(`ranking.note`) instead of letting a "worst skill" list imply an order the data
|
|
367
|
+
cannot support. Run scores get the same treatment: each run bootstraps its own
|
|
368
|
+
windows, so `62.5 (41.2–88.3)` shows how little a two-window run can prove.
|
|
369
|
+
|
|
370
|
+
Beyond the sensitivity tests, a property suite generates random windows with a
|
|
371
|
+
seeded RNG and asserts invariants over the real scoring pipeline: scores stay in
|
|
372
|
+
0-100, the interval always contains the point estimate, one more tool error never
|
|
373
|
+
raises reliability, one more correction never raises effectiveness, custom
|
|
374
|
+
weights always normalise, the ranking caveat appears exactly when intervals
|
|
375
|
+
overlap closely, MCP advice appears exactly past its threshold, and the same
|
|
376
|
+
evidence always yields the same score and advice (reproducibility).
|
|
377
|
+
|
|
378
|
+
`node --test` contains sensitivity tests that inject exactly one bad signal at a
|
|
379
|
+
time and assert that _only_ the matching dimension moves:
|
|
380
|
+
|
|
381
|
+
- four tool errors out of eight calls → reliability drops, static unchanged
|
|
382
|
+
- three correcting messages → effectiveness drops, reliability unchanged
|
|
383
|
+
- twenty calls and a 9 s load → efficiency drops, effectiveness roughly stable
|
|
384
|
+
- three failed loads with a perfect body → reliability is 0 and the total falls
|
|
385
|
+
below 60, so good writing cannot rescue an unusable skill
|
|
386
|
+
|
|
387
|
+
Known validity limits, stated as limits rather than fixed:
|
|
388
|
+
|
|
389
|
+
1. **Short sessions are now judged separately.** A subagent session that loads a
|
|
390
|
+
skill and immediately returns used to count as "ignored after load" — literally
|
|
391
|
+
true, but a 2-step child is not the same failure mode as a root session
|
|
392
|
+
dropping a skill. Windows now record `origin` (`subagent` or null) and the
|
|
393
|
+
abandonment rate is computed over root runs only; the denominator is exposed
|
|
394
|
+
as `abandonBase` so the report shows how much evidence it used.
|
|
395
|
+
2. **Corrections are language-shaped.** The detector covers common Chinese and
|
|
396
|
+
English pushback; sarcasm, silent dissatisfaction, or a correction phrased as
|
|
397
|
+
a new request are missed.
|
|
398
|
+
3. **No outcome ground truth.** Effectiveness is a proxy built from what the user
|
|
399
|
+
did next, not from a verified task verdict.
|
|
400
|
+
4. **Provisional scores.** Below `minSamples` loads the number is marked
|
|
401
|
+
provisional on purpose; treat grades on 1–2 loads as anecdote.
|
|
402
|
+
|
|
403
|
+
## Known blind spots
|
|
404
|
+
|
|
405
|
+
- Two load paths are recorded: the `skill` tool call (`via` absent, timed) and a
|
|
406
|
+
user-typed `/skill-name`, which the harness resolves without a tool call and
|
|
407
|
+
injects as a user message carrying `source.kind === 'skill-invocation'`
|
|
408
|
+
(`via: 'invocation'`, no `loadMs`). The message repeats on later steps, so it
|
|
409
|
+
is deduped by message id. Still invisible: content injected by another plugin
|
|
410
|
+
(`agent.inject()`), instructions baked into a preset's own prompt (not a skill
|
|
411
|
+
load at all), and any other profile or host, whose telemetry lives in its own
|
|
412
|
+
data directory.
|
|
413
|
+
- "Effectiveness" is inferred from the follow-up message, not from a task
|
|
414
|
+
outcome. A correct result followed by an unrelated complaint counts as a
|
|
415
|
+
correction; the TTL keeps that window narrow but cannot make it exact.
|
|
416
|
+
- Static checks read the _resolved_ catalog entry for the global layer. Skills
|
|
417
|
+
that only exist inside an agent-preset layer may resolve to `null` static
|
|
418
|
+
evidence (the runtime dimensions still work).
|
|
419
|
+
- The store grows without pruning. One record pair per skill load; a very busy
|
|
420
|
+
profile should eventually get a compaction pass.
|
|
421
|
+
- The dashboard route is served by `webServer` and cannot be reclaimed during a
|
|
422
|
+
hot reload in the same host process: after a disable/enable cycle the earlier
|
|
423
|
+
instance keeps answering `dashboardPath` (old handler, old state), and the new
|
|
424
|
+
registration is refused as a duplicate. Registration is therefore best-effort
|
|
425
|
+
and logged instead of thrown, and `dashboardPath` is a config knob so a
|
|
426
|
+
source edit can be previewed on a fresh path. A host restart re-registers the
|
|
427
|
+
canonical path cleanly.
|
|
428
|
+
|
|
429
|
+
## Configuration keys
|
|
430
|
+
|
|
431
|
+
All 25 settings, in the order the native form renders them. The form persists to
|
|
432
|
+
`settings.json`; `dataDir`/`DSH_SKILL_EVAL_DIR` still wins for the data root.
|
|
433
|
+
|
|
434
|
+
```yaml
|
|
435
|
+
enabled: true
|
|
436
|
+
windowToolCalls: 12 # tool calls that count as evidence for one load
|
|
437
|
+
minSamples: 3 # loads before a score stops being provisional
|
|
438
|
+
maxAdvice: 3 # advice items per skill in the summary
|
|
439
|
+
weights: { static: 30, reliability: 25, effectiveness: 30, efficiency: 15 } # normalised to 100%
|
|
440
|
+
thresholds: { excellent: 90, good: 80, pass: 70 } # A/B/C floors
|
|
441
|
+
windowTtlMs: 1800000 # a run older than this is not blamed on a later message
|
|
442
|
+
dataDir: ~/.dsh/skill-eval # optional override
|
|
443
|
+
turnReports: true # one board per settled turn under reports/turns/
|
|
444
|
+
turnReportsKeep: 50 # how many per-turn boards to retain
|
|
445
|
+
reportOnTurnEnd: true # write reports/latest.md + latest.json after each turn (no tokens)
|
|
446
|
+
tokenBudgetPerLoad: 20000 # billable tokens per load before efficiency/advice react
|
|
447
|
+
cacheReadWeight: 0 # weight of cached input tokens (near-free by default)
|
|
448
|
+
countTokens: true # attribute real provider usage to skill windows
|
|
449
|
+
adviceNoiseFloor: true # advice must clear its 95% interval to stay critical
|
|
450
|
+
registerReviewTool: true # skill_eval_review: leakage critic + audit ledger
|
|
451
|
+
editBudgetMax: 4 # concurrent edits a first-round skill may carry
|
|
452
|
+
editBudgetMin: 1 # annealed minimum
|
|
453
|
+
editBudgetHorizon: 6 # accepted edits over which the budget anneals
|
|
454
|
+
noveltyWindow: 2 # same-dimension edits that trigger exploration
|
|
455
|
+
pruneMinLoads: 6 # loads before a prune recommendation is allowed
|
|
456
|
+
pruneMaxScore: 60 # low score + low interval -> prune candidate
|
|
457
|
+
promptHint: true # the one-line system-prompt hint (~60 tokens/turn); false = zero fixed cost
|
|
458
|
+
inspectProvider: true # register the eval data as a Cordis Inspect provider
|
|
459
|
+
registerTool: true
|
|
460
|
+
registerCommand: true
|
|
461
|
+
webDashboard: true # HTML page + the JSON route the client reads (versioned alias included)
|
|
462
|
+
dashboardPath: /skill-eval # change it if a reload could not reclaim the old route
|
|
463
|
+
```
|
|
@@ -0,0 +1,114 @@
|
|
|
1
|
+
# 开发模式与热重载(给改插件的人看)
|
|
2
|
+
|
|
3
|
+
这份文档解决一个反复踩的坑:**改了插件代码,界面上却看不到**。
|
|
4
|
+
|
|
5
|
+
## 为什么改了没生效
|
|
6
|
+
|
|
7
|
+
DSH 的插件在启动时作为 ES module 加载。之后:
|
|
8
|
+
|
|
9
|
+
- **disable / enable 只会重跑 `apply`,不会重新 import 文件**——模块实例仍在 ESM 缓存里
|
|
10
|
+
- 因此 `PLUGIN_VERSION`、路由注册表、客户端模块清单都停留在启动那一刻
|
|
11
|
+
- 客户端(`client.js`)更严格:模块清单来自 boot 扫描,**刷新页面也不重新扫描**
|
|
12
|
+
|
|
13
|
+
所以判断"是不是新代次",不要看文件 mtime,要探测**只有新一代才有的路由**:
|
|
14
|
+
|
|
15
|
+
```bash
|
|
16
|
+
curl -s -o /dev/null -w '%{http_code} %{content_type}\n' \
|
|
17
|
+
http://127.0.0.1:<port>/skill-eval/data/<当前版本>
|
|
18
|
+
# application/json → 宿主就是这一代
|
|
19
|
+
# text/html → 命中旧前缀兜底路由,宿主还是旧代次
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
`dsh-skill-eval-accept` 的 `versioned-route` 与 `settings-api` 两项就是在做这件事。
|
|
23
|
+
更快的办法是在对话里跑 `/skill-eval dev`:它会直接对比**已加载模块版本**与
|
|
24
|
+
**磁盘 package.json 版本**,不一致就给出 `module-skew` 与重启提示。
|
|
25
|
+
|
|
26
|
+
## 让热重载真正生效(已在本机验证)
|
|
27
|
+
|
|
28
|
+
根因:插件模块是 ES module,禁用/启用只会重跑 `apply`,不会重新 `import` 文件;
|
|
29
|
+
只有当 **Cordis HMR 监视到源文件变化**时,它才会让 loader 重新导入该模块。
|
|
30
|
+
|
|
31
|
+
启用监视(一次性,写入 profile patch):
|
|
32
|
+
|
|
33
|
+
```yaml
|
|
34
|
+
# ~/.dsh/profiles/desktop/cordis.patch.yml
|
|
35
|
+
- id: hmr
|
|
36
|
+
name: '@deepseek-ai/dsh-hmr'
|
|
37
|
+
config:
|
|
38
|
+
root:
|
|
39
|
+
- /Users/adams/Documents/dsh-plugins/dsh-skill-eval
|
|
40
|
+
ignored:
|
|
41
|
+
- '**/node_modules/**'
|
|
42
|
+
- '**/.git/**'
|
|
43
|
+
- '**/dist/**'
|
|
44
|
+
- 'cache'
|
|
45
|
+
- 'data'
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
之后每次改完源码:
|
|
49
|
+
|
|
50
|
+
```bash
|
|
51
|
+
touch index.js lib/*.js client.js # 或任何真实改动
|
|
52
|
+
sleep 3 # HMR 默认 debounce + awaitWriteFinish
|
|
53
|
+
curl -s -o /dev/null -w '%{content_type}\n' \
|
|
54
|
+
http://127.0.0.1:<port>/skill-eval/data/<新版本>
|
|
55
|
+
# application/json → 新代次已生效(改版本号时用新版本号探;不改版本用旧版本号)
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
实测:1.15.2 → 1.15.3 → 1.15.4 都靠 HMR 完成,没有重启;客户端半个由
|
|
59
|
+
client HMR receiver 替换已构建的 bundle(改的是 `client/*.js`,先
|
|
60
|
+
`node scripts/build-client.mjs` 再 `touch client.js`;未替换时刷新一次页面)。
|
|
61
|
+
|
|
62
|
+
注意两点:
|
|
63
|
+
|
|
64
|
+
- **canonical 路由可能仍由旧实例应答**(`/skill-eval/data`、`/skill-eval/settings`)。
|
|
65
|
+
热重载后请用**版本化路由**(`/data/<version>`、`/settings/<version>`)做判断,
|
|
66
|
+
客户端和 verify 脚本用的都是版本化路由。
|
|
67
|
+
- profile patch 改动本身也会被 HMR 监视并重排,所以启用监视不需要重启;
|
|
68
|
+
但把 patch 写坏会导致下次启动失败,改前先备份(本仓库操作时已生成
|
|
69
|
+
`cordis.patch.yml.bak-skill-eval-<时间>`)。
|
|
70
|
+
|
|
71
|
+
## 什么时候必须重启
|
|
72
|
+
|
|
73
|
+
| 改了什么 | 生效方式 |
|
|
74
|
+
| --------------------------------------- | -------------------------------------------------------------------------- |
|
|
75
|
+
| `index.js` / `lib/*`(host 半) | 正常情况:重启;极少数情况下 disable/enable 会重新 import,不要依赖 |
|
|
76
|
+
| `client/*.js`(client 源) | `node scripts/build-client.mjs` → `touch client.js`,HMR 替换已构建 bundle |
|
|
77
|
+
| `client.js`(生成产物) | 不要手改;由 `client/*.js` 构建,doctor 检查两者一致 |
|
|
78
|
+
| `package.json` 版本号 | 重启(版本化路由前缀由模块加载时的版本决定) |
|
|
79
|
+
| 数据文件(events.jsonl、settings.json) | 立即(每次请求重新读) |
|
|
80
|
+
| 由 `settings.json` 覆盖的配置 | 立即(保存即改内存中的 cfg) |
|
|
81
|
+
|
|
82
|
+
## 给调试用的开关
|
|
83
|
+
|
|
84
|
+
| 开关 | 作用 |
|
|
85
|
+
| ------------------------------------ | --------------------------------------------------- |
|
|
86
|
+
| `window.__SEV_ENABLE_POPUP__ = true` | 恢复侧边栏弹窗模态框(默认只有常驻按钮打开看板) |
|
|
87
|
+
| `DSH_SKILL_EVAL_DEBUG=1` | host 侧把警告同时写 stderr |
|
|
88
|
+
| `DSH_SKILL_EVAL_DIR=<path>` | 覆盖数据目录(隔离实验数据) |
|
|
89
|
+
| `DSH_SKILL_EVAL_BASE=<path>` | 验收脚本使用的看板路径前缀(默认 `/skill-eval`) |
|
|
90
|
+
| `DSH_SKILL_EVAL_VERSION=<v>` | 验收脚本探测的版本化路由版本(默认取 package.json) |
|
|
91
|
+
|
|
92
|
+
## 客户端与旧宿主兼容
|
|
93
|
+
|
|
94
|
+
新客户端必须能跑在旧宿主返回的数据上(1.10.x 没有 `edits` / `validity`)。测试用
|
|
95
|
+
`test/fixtures/live-1.10.1.json`(真实宿主抓下来的载荷)锁定这一点;原生设置页在
|
|
96
|
+
旧宿主上会显示"重启 DeepSeek Harness 后可用"并给出网页配置页链接,而不是白屏。
|
|
97
|
+
|
|
98
|
+
## 一个可复制的验证流程
|
|
99
|
+
|
|
100
|
+
```bash
|
|
101
|
+
dsh-skill-eval-verify # 重启后一键验证产品面(10 项)
|
|
102
|
+
dsh-skill-eval-accept # 探当前代次(端口自动发现)
|
|
103
|
+
curl -s http://127.0.0.1:<port>/skill-eval/data | head -c 200
|
|
104
|
+
# host 需重启时会出现:/data/<new-version> 返回 text/html
|
|
105
|
+
|
|
106
|
+
# 想在不动 GUI 进程的前提下验证 host 代码:起一个一次性 profile
|
|
107
|
+
dsh sevcheck --from-default-profile web --help
|
|
108
|
+
dsh plugin --profile sevcheck add "$PWD"
|
|
109
|
+
dsh --profile sevcheck --no-open --port 59231 &
|
|
110
|
+
node scripts/accept.mjs http://127.0.0.1:59231
|
|
111
|
+
kill %1 && rm -rf ~/.dsh/profiles/sevcheck
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
这条流程在 1.12.0/1.12.1 上实际用过:17 项 PASS,设置 API 的保存/归一化/400 都验证过。
|
|
@@ -0,0 +1,65 @@
|
|
|
1
|
+
# 入口 · Entry points
|
|
2
|
+
|
|
3
|
+
三个常用入口,其余都是给 agent / 终端 / 调试用的。日常只需要记住前三行。
|
|
4
|
+
Three everyday entry points; everything else is for agents, terminals or debugging.
|
|
5
|
+
|
|
6
|
+
## 日常 · Everyday
|
|
7
|
+
|
|
8
|
+
| 要做什么 · Goal | 中文入口 · Chinese | English entry | 说明 · Notes |
|
|
9
|
+
| --------------- | -------------------------------------------------- | --------------------------------------------------------------------- | ------------------------------------------------------------------- |
|
|
10
|
+
| 看这一轮的结果 | 每个完成回合下方的**结果条** | the **result strip** under a completed turn | 分数、调用/报错、下一步修复、改动记录;点「本轮详情」展开 |
|
|
11
|
+
| 打开看板 | 侧边栏底部的**面板图标「技能评测」** → 主面板 | the **panel icon "Skill eval"** in the sidebar → main panel | 与「上下文」插件同位置;面板内含 技能/轨迹/运行/MCP/开发者 五个标签 |
|
|
12
|
+
| 改评分标准 | **设置 → 插件 → `dsh-skill-eval` 的包页面 → 配置** | **Settings → Plugins → the `dsh-skill-eval` bundle page → Configure** | 与上下文插件同一位置(bundle config);权重、阈值、预算等保存即生效 |
|
|
13
|
+
|
|
14
|
+
## 网页 · Web
|
|
15
|
+
|
|
16
|
+
| 用途 · Purpose | 地址 · URL |
|
|
17
|
+
| ----------------------------------------- | ----------------------------------------------------------------------------------- |
|
|
18
|
+
| 汇报看板 · Board | `http://127.0.0.1:<当前端口>/skill-eval`(热重载期间用 `/skill-eval/board/<版本>`) |
|
|
19
|
+
| 网页配置页(与设置页同源) · Web settings | `…/skill-eval/config` |
|
|
20
|
+
| JSON 数据 · JSON report | `…/skill-eval/data/<version>` |
|
|
21
|
+
|
|
22
|
+
端口每次启动会变;`dsh-skill-eval-verify` 会自动发现。Port changes per launch; the verify script discovers it.
|
|
23
|
+
|
|
24
|
+
看板与设置页支持中英双语:按浏览器 `Accept-Language` 自动选择,也可用 `?lang=zh|en` 强转;页面右上角有切换链接。建议正文(advice)目前仍是引擎输出的英文。
|
|
25
|
+
|
|
26
|
+
The board and settings page are bilingual: `Accept-Language` picks the language, `?lang=zh|en` overrides it, and a toggle link is on the page. Rule-generated advice text stays English for now.
|
|
27
|
+
|
|
28
|
+
## 对话内 · In chat
|
|
29
|
+
|
|
30
|
+
| 命令 · Command | 作用 · What |
|
|
31
|
+
| ------------------------------------------------------- | ------------------------------------------- |
|
|
32
|
+
| `/skill-eval` | 总览 · cross-skill summary |
|
|
33
|
+
| `/skill-eval <skill>` | 单个技能 · one skill |
|
|
34
|
+
| `/skill-eval loop [all]` | 下一步改什么 · next optimization step |
|
|
35
|
+
| `/skill-eval validity` | 证据能支持什么 · what the evidence supports |
|
|
36
|
+
| `/skill-eval history <skill>` / `impact <skill>` | 趋势 / 改动前后 · trend / edit impact |
|
|
37
|
+
| `/skill-eval mcp` / `runs` / `turns` / `dev` / `config` | MCP / 运行 / 每次执行的板 / 诊断 / 生效配置 |
|
|
38
|
+
|
|
39
|
+
## 给 agent · For agents
|
|
40
|
+
|
|
41
|
+
| 工具 · Tool | 作用 · What |
|
|
42
|
+
| ------------------- | --------------------------------------------------------------------------------- |
|
|
43
|
+
| `skill_eval_report` | 同一份报告(`view` 可选 summary/mcp/runs/loop/validity/history/impact/turns/dev) |
|
|
44
|
+
| `skill_eval_review` | 改技能前做泄漏审查并写 ledger · leak screen before editing a skill |
|
|
45
|
+
|
|
46
|
+
## 终端 · Terminal
|
|
47
|
+
|
|
48
|
+
```bash
|
|
49
|
+
dsh-skill-eval-doctor # 打包预检 19 项(--release 加产物检查)· packaging pre-flight
|
|
50
|
+
dsh-skill-eval-accept # 代次/数据契约 17 项 · generation & data contract
|
|
51
|
+
dsh-skill-eval-verify # 产品面可达 9 项 · product surfaces (after a reload)
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
## 调试(可选) · Debug (opt-in)
|
|
55
|
+
|
|
56
|
+
| 开关 · Switch | 作用 · What |
|
|
57
|
+
| --------------------------------------------- | ---------------------------------------------------------------------- |
|
|
58
|
+
| 页面里设 `window.__SEV_ENABLE_POPUP__ = true` | 弹窗 + 侧边栏按钮回来 · popup + sidebar button return |
|
|
59
|
+
| `/skill-eval dev` | 代次漂移、存储健康、已注册路由 · generation skew, store health, routes |
|
|
60
|
+
|
|
61
|
+
## 刻意不做的入口 · Deliberately removed
|
|
62
|
+
|
|
63
|
+
- 插件详情页里再放一个区块(与看板重复)· a second section on the plugin detail page
|
|
64
|
+
- 侧边栏常驻按钮(默认不注册,避免和面板图标重复)· an always-on sidebar button
|
|
65
|
+
- 「更多」菜单里的看板行(看板已升为侧边栏主面板)· the More-menu board row (the board is now a main panel)
|