dsh-report-card 0.0.0-stage → 1.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 DSH Report Card contributors
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
package/README.md CHANGED
@@ -1,3 +1,291 @@
1
- # Temporary Holding Version
1
+ # DSH Report Card
2
2
 
3
- This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
3
+ An independent, **dynamic** benchmark for DeepSeek Harness plugins. It installs
4
+ one plugin into a throwaway profile, boots the real harness with it, runs one
5
+ scripted turn against a local fixture endpoint, and reports what the plugin
6
+ costs: boot time, per-turn hook overhead, the tokens it adds to the assembled
7
+ prompt, the capability surface it contributes, and the static risk flags its
8
+ package carries.
9
+
10
+ Every number is a reading from a real harness process — wall clock, process CPU,
11
+ or the exact bytes of the request it sent — none of it estimated from source
12
+ code. Every card stores its samples, its digest, and the command that reproduces
13
+ it.
14
+
15
+ ## What already exists, and why this is not a duplicate
16
+
17
+ Searched 2026-11-10: the `dsh-plugin-catalog` 2026.1110.18259 (9,959 plugins)
18
+ and npm. The adjacent tools are all **static scores or in-profile meters**:
19
+
20
+ | Existing | What it actually does | Why it is a different thing |
21
+ |---|---|---|
22
+ | `dsh-plugin-scorecard` 0.3.2 | Static quality/security score, rankings, history across the whole catalog | Scores package *files*; never boots the plugin |
23
+ | `dsh-quality-score` 0.1.4 | Static 0-100 from manifest, peers, dist-tags, docs | Same: a metadata rubric, no runtime |
24
+ | `dsh-trust-check` 0.2.6 | Static capability disclosure with `file:line` evidence | Facts about source, not behaviour under load |
25
+ | `dsh-xray` 0.11.0 | Per-plugin context-tax attribution, per-request token ledger | Prices what is **already loaded in your profile**; cannot evaluate a candidate |
26
+ | `dsh-context-doctor` | Token costs of instruction chains, skill catalogs, tool schemas | Static accounting of the current context |
27
+ | `dsh-agent-bench` 0.1.1 | Replays fixed agent cases and diffs scorecards | Measures *model behaviour*; the plugin is not the subject |
28
+ | `dsh-workspace-switch-cost` | Cold-start cost of switching workspaces | Per-workspace, not per candidate plugin |
29
+
30
+ **One line:** existing scorecards read a plugin's files, and the closest dynamic
31
+ instruments price what is already in your profile — nothing installs a candidate
32
+ into a disposable profile and times its boot, a scripted turn, and the exact
33
+ request it produces; that is what this does, so it complements them.
34
+
35
+ ## What a card contains
36
+
37
+ | Dimension | How it is measured |
38
+ |---|---|
39
+ | **Boot wall** | Process start (`performance.timeOrigin`) → the measurement probe reporting the plugin tree composed, with and without the plugin |
40
+ | **Boot CPU** | The child's own CPU time (user + system) at that same dump: what composing the tree cost, independent of what else the machine was doing. This is the graded boot figure |
41
+ | **Launch** | The same point measured from process spawn, including OS process startup |
42
+ | **Turn / step** | `turn_start → turn_end` and `step_start → step_end` on one fixed scripted turn |
43
+ | **Assembly** | `step/start` (from the child's own session log inside the captured request) to the request reaching the fixture endpoint: prompt assembly and request serialization — the phase plugin hooks run in. This is the graded per-turn figure |
44
+ | **Context tax** | Model-visible bytes of the request the harness actually sent (`system` + `tools` + `messages`), in tokens at the documented 4-chars/token heuristic; harness extension fields are counted separately |
45
+ | **Capability surface** | Tools and schemas the model is offered, prompt sections and their sizes, event listeners per event, loader entries, prompt variables — all as a diff against the same profile without the plugin |
46
+ | **Static risk flags** | Install scripts, shipped binaries, network hosts, subprocess use, dynamic code, filesystem writes, `process.exit` — with file and line evidence. Nothing is executed to produce them |
47
+ | **Health** | `ok`, `install-rejected`, `install-failed`, `boot-failed`, `turn-failed`, `entry-failed`, `entry-absent`. A plugin that fails to activate is a failed card, never a passing one |
48
+
49
+ See [`docs/example-card.md`](docs/example-card.md) for a complete card of the
50
+ test fixture.
51
+
52
+ ## How it measures
53
+
54
+ ```
55
+ ┌──────────────── throwaway DSH_HOME ────────────────┐
56
+ subject package ► │ profile "rc-measured" ← shipped "headless" template │
57
+ │ subject plugin: an insert row, or the bundle list │
58
+ │ probe: inserted last, dumps the surface, exits │
59
+ └────────────┬───────────────────────────┬────────────┘
60
+ │ surface runs │ turn runs
61
+ ▼ ▼
62
+ spawn → probe dump → exit spawn → session → turn → exit
63
+ boot wall/CPU + surface assembly/step + exact request
64
+ DEEPSEEK_BASE_URL → local fixture
65
+ ```
66
+
67
+ * **Fresh state.** Each report gets a new `DSH_HOME`, a new profile copied from
68
+ the shipped `headless` template, and an empty working directory. Your real
69
+ profile is never opened, so a measured plugin cannot disturb it.
70
+ * **No model calls.** `DEEPSEEK_BASE_URL` points at a loopback endpoint that
71
+ speaks the Anthropic-style Messages SSE stream the shipped adapter parses. The
72
+ reply is a fixed one-line text block, so the turn is deterministic — and the
73
+ *request* is captured verbatim, which is the context-tax measurement.
74
+ * **No background noise.** The fixture overlay disables session-title generation
75
+ and telemetry, and a discarded warm-up pair primes the file cache before any
76
+ sample is recorded.
77
+ * **Round-paired deltas.** Each subject run is paired with the baseline run
78
+ immediately before it; the delta is the median of those paired differences,
79
+ which cancels slow machine drift. Grades charge only the part of a delta that
80
+ clears the machine's measured noise (one standard error of the paired
81
+ differences), so a loaded machine cannot turn a free plugin into a bad grade —
82
+ and cannot hide real overhead either, because the raw numbers stay on the
83
+ card next to their floor.
84
+ * **Integrity.** The card is hashed (`sha256` over canonical JSON, digest
85
+ excluded). Re-running the same subject compares against the previous card in
86
+ the same directory and prints drift.
87
+
88
+ ## Install
89
+
90
+ Published on npm as **`dsh-report-card@1.0.0`**. It ships as a bundle with
91
+ metadata, `locale/en.json`, `locale/zh.json` and an icon. The plugin manager adds
92
+ the dependency *and* enables the bundle in one step; removing it undoes both:
93
+
94
+ ```
95
+ plugin_manager install_bundle target: "dsh-report-card@1.0.0"
96
+ plugin_manager remove_bundle target: "dsh-report-card"
97
+ ```
98
+
99
+ or from a terminal:
100
+
101
+ ```powershell
102
+ dsh plugin --profile web add dsh-report-card
103
+ ```
104
+
105
+ For development from a checkout, pass a directory spec instead
106
+ (`file:C:/path/to/dsh-report-card`). `install_bundle` is the sanctioned path
107
+ because a bundle is composed through the profile's bundle list, not just
108
+ `node_modules`; the bundle inserts one entry, `dsh-report-card`, which
109
+ contributes one tool.
110
+
111
+ ## Use
112
+
113
+ From a session (the tool is model-facing):
114
+
115
+ ```
116
+ report_card(pluginSpec: "dsh-trust-check@0.2.6", samples: 3)
117
+ report_cards_recent(limit: 10) # the last cards on disk, including automatic ones
118
+ ```
119
+
120
+ `report_card` measures on demand; `report_cards_recent` reads the card index so
121
+ you can see what the background pass produced.
122
+
123
+ From a terminal — the same code path, printable output:
124
+
125
+ ```powershell
126
+ node harness/cli.mjs ./test/fixtures/fake-plugin # local directory
127
+ node harness/cli.mjs dsh-trust-check@0.2.6 --samples 3 # npm package
128
+ node harness/cli.mjs dsh-quality-score --json # raw card
129
+ ```
130
+
131
+ Cards are written to `./report-cards/<name>@<version>/<timestamp>.{json,md}` and
132
+ indexed in `./report-cards/index.json`; set `DSH_REPORT_CARD_DIR` to move that.
133
+
134
+ The tool takes **~60–90 s** for the default five samples: four harness boots per
135
+ sample round (surface × 2, turn × 2) plus one discarded warm-up round. Pass
136
+ `samples: 3` for a quick look and `samples: 8..10` when you need to resolve a
137
+ few milliseconds.
138
+
139
+ ## Automatic measurement
140
+
141
+ Since 1.1.0 the card is produced **after a plugin is installed**, without
142
+ anyone asking. The plugin listens to the plugin manager's
143
+ `plugin-manager/operation` event and watches the profile manifest, queues the
144
+ new package, and measures it in the background — the same throwaway profile,
145
+ three samples, one measurement at a time.
146
+
147
+ ```
148
+ install a plugin
149
+ │ manager event (Plugins page, plugin_manager tool)
150
+ │ or profile package.json change (dsh plugin add)
151
+ ▼
152
+ debounce 2.5 s → list installed third-party bundles → skip ones already carded
153
+ ▼
154
+ queue → measure in a throwaway profile → save card under $DSH_HOME/report-cards
155
+ ▼
156
+ report_cards_recent lists it
157
+ ```
158
+
159
+ What it skips: shipped/optional/official bundles, bundles that failed to load,
160
+ its own package, and any `name@version` that already has a card. What it does
161
+ **not** do: measure plugins that were already installed before this version
162
+ started, unless `autoBackfill` is on.
163
+
164
+ Automatic cards take roughly 45 s each (three samples) and run serially; a
165
+ manual `report_card` call queues behind a running automatic card so the two
166
+ never compete for the machine.
167
+
168
+ Config, in the profile patch (or the plugin's config in a preset):
169
+
170
+ ```yaml
171
+ - id: dsh-report-card
172
+ config:
173
+ autoMeasure: true # false turns the background behavior off
174
+ autoSamples: 3 # 3..10; steadier numbers cost more time
175
+ autoBackfill: false # true: card already-installed plugins at startup
176
+ autoReportsDir: null # default: <DSH_HOME>/report-cards
177
+ ```
178
+
179
+ `DSH_REPORT_CARD_AUTO=0` disables it for one process without touching config.
180
+
181
+ ## Reading a card
182
+
183
+ ```
184
+ **Grade D** · graded boot +6.3 ms CPU · graded turn 0 ms · context +76 tok · risk D
185
+ ```
186
+
187
+ * The headline shows the two **graded** deltas plus the exact context delta and
188
+ the risk grade. Every raw number — wall boot, process CPU, turn, assembly,
189
+ step, launch — is in the tables below with its samples and noise floor.
190
+ * Wall time is presented but never graded: it is hostage to whatever else the
191
+ machine is doing. The boot grade uses noise-adjusted **process CPU** at the
192
+ moment the plugin tree is ready; the turn grade uses the noise-adjusted
193
+ **assembly** time, the phase plugin hooks run in.
194
+ * `context` is the median of the captured requests; it is exactly reproducible,
195
+ since the prompt is a function of the profile.
196
+ * `risk` is disclosure, not a verdict: `A` none, `B` informational only,
197
+ `C` medium flags, `D` high flags. A postinstall script is a high flag.
198
+ * The overall grade is the worst dimension. `F` means the card could not measure
199
+ a running plugin.
200
+
201
+ | Dimension | A | B | C | D | F |
202
+ |---|---|---|---|---|---|
203
+ | Boot CPU (noise-adjusted) | ≤ 10 ms | ≤ 30 ms | ≤ 80 ms | ≤ 250 ms | > 250 ms |
204
+ | Assembly (noise-adjusted) | ≤ 10 ms | ≤ 30 ms | ≤ 80 ms | ≤ 200 ms | > 200 ms |
205
+ | Context token delta | ≤ 100 | ≤ 500 | ≤ 2000 | ≤ 8000 | > 8000 |
206
+ | Risk | clean | info | medium | high | — |
207
+
208
+ Numbers are comparable **only on the same machine identity** (platform, CPU
209
+ count, dsh version) — the card prints it. A delta inside the printed noise floor
210
+ is noise; the card says so in its notes.
211
+
212
+ ## Reproduce and verify
213
+
214
+ Every card prints its own reproduce command. The harness's own repeatability
215
+ check runs one subject several times and reports the spread:
216
+
217
+ ```powershell
218
+ node harness/repeat.mjs ./test/fixtures/fake-plugin --runs 2 --samples 3
219
+ ```
220
+
221
+ Measured while a harness session was actively running on the same machine (full
222
+ output in [`docs/acceptance.md`](docs/acceptance.md)): the **context delta
223
+ repeated at 0.00%**, the **graded boot CPU and assembly deltas at 0.00%**, and
224
+ the raw round-paired wall deltas within 1.9% (boot) and 2.4% (assembly). Earlier
225
+ in the same session, with a busier machine, raw wall deltas moved by up to 19%.
226
+ The card publishes its own jitter and noise floor for exactly this reason.
227
+
228
+ Tests:
229
+
230
+ ```powershell
231
+ node test/unit.test.mjs # units, ~1 s
232
+ node test/auto.test.mjs # install trigger and queue, ~1 s
233
+ node test/harness.test.mjs # real boots against fixtures, ~80 s
234
+ ```
235
+
236
+ `test/harness.test.mjs` covers the whole harness with fake plugins: a healthy
237
+ plugin (surface, context tax, risk flags, digest), a plugin that throws during
238
+ activation (must be a failed card), a plugin that calls `process.exit` (must be
239
+ a failed boot that still returns), and a second card that must show drift
240
+ against the first.
241
+
242
+ ## Publishing
243
+
244
+ The package is publish-ready: `npm publish` and the catalog PR are documented in
245
+ [`docs/publishing.md`](docs/publishing.md), including the exact
246
+ `data/plugins/<owner>__<repo>.yml` entry the awesome-dsh-plugin list expects, the
247
+ `repository` field the catalog's npm mapping depends on, and the tarball-install
248
+ check that was run against the packed artifact.
249
+ ## Safety and limits
250
+
251
+ * **The subject executes.** It runs in a disposable profile with a read-only
252
+ file policy and a fresh `DSH_HOME`, as your user — that is enough to keep it
253
+ away from your real profile, but it is **not** a security sandbox. Read the
254
+ risk flags before measuring something you do not trust.
255
+ * **No `app.asar` edits.** The harness only spawns the shipped CLI and writes
256
+ into a temp directory.
257
+ * **Fail closed.** A package the runtime refuses to install (for example a peer
258
+ version gate) is reported as `install-rejected` or `install-failed`; the harness never grants a
259
+ version exemption and never bypasses the gate. A plugin that crashes the host
260
+ or hangs is bounded by a per-child timeout.
261
+ * **The heuristic is not billing.** Context tokens use four characters per
262
+ token, the same heuristic the shipped token meter documents; CJK and JSON
263
+ schemas are underpriced by it. Extension fields (`dsh_session_log`,
264
+ `dsh_plugin_packages`) are measured separately and excluded.
265
+ * **One turn, one profile shape.** The card does not measure multi-turn
266
+ behaviour, compaction, or client-side (browser) cost. Client slot
267
+ contributions are not observable host-side; only host-side surface is
268
+ reported.
269
+
270
+ ## Layout
271
+
272
+ | Path | Role |
273
+ |---|---|
274
+ | `index.js` | The bundle entry: registers the `report_card` tool |
275
+ | `harness/cli.mjs` | Command line entry |
276
+ | `harness/generate.mjs` | measure → diff → grade → render → store |
277
+ | `harness/measure.mjs` | The four-run loop, event parsing, request pricing |
278
+ | `harness/mock-llm.mjs` | The local Messages endpoint and its SSE fixture |
279
+ | `harness/probe/` | The fixture plugin that dumps a profile's surface |
280
+ | `harness/risk.mjs` | Static disclosure scan |
281
+ | `harness/report.mjs` | Grades, digest, drift, card assembly |
282
+ | `harness/render.mjs` | Markdown card |
283
+ | `harness/auto.mjs` | Install trigger, eligibility, and the background measurement queue |
284
+ | `harness/repeat.mjs` | Repeatability check for the harness itself |
285
+ | `docs/` | An example card and the acceptance evidence |
286
+ | `locale/`, `icon.svg`, `cordis.patch.yml` | Bundle metadata and its one insert row |
287
+ | `test/` | Unit tests, end-to-end tests, fake plugins |
288
+
289
+ ## License
290
+
291
+ MIT.
@@ -0,0 +1,4 @@
1
+ - insert:
2
+ - id: dsh-report-card
3
+ name: 'dsh-report-card'
4
+ config: {}
@@ -0,0 +1,127 @@
1
+ # Acceptance evidence
2
+
3
+ This file records what was actually measured when the v1 was built, on this
4
+ machine, so the claims in the README can be checked rather than trusted.
5
+
6
+ Machine: win32-x64, 8 logical CPUs, dsh 0.2.2-rc.1, Node v24.19.0 for the
7
+ harness and Electron 44.7.0's Node v24.21.0 for the measured children.
8
+
9
+ **The machine was not quiet.** A DeepSeek Harness Web session was running and
10
+ actively issuing commands throughout, and several other builds shared the
11
+ machine. That matters for the numbers below and is stated again where it
12
+ changes their meaning.
13
+
14
+ ## 1. Same plugin, two runs
15
+
16
+ Command:
17
+
18
+ ```powershell
19
+ node harness/repeat.mjs ./test/fixtures/fake-plugin --runs 2 --samples 3
20
+ ```
21
+
22
+ Result (raw output, 2026-11-10, final code):
23
+
24
+ ```
25
+ metric run 1 run 2 spread % of baseline
26
+ boot CPU graded delta 0 0 0.0 0.00%
27
+ boot CPU raw delta -62 -172 110.0 4.24%
28
+ boot wall graded delta 11.8 0 11.8 0.54%
29
+ boot paired median delta 50.6 31.8 18.8 0.86%
30
+ boot best-case delta 50.6 150.8 100.1 4.58%
31
+ assembly graded delta 0.6 0 0.6 1.90%
32
+ assembly paired median delta 5 -3 8.0 24.24%
33
+ assembly best-case delta 3 -5 8.0 24.24%
34
+ step best-case delta 7 -7 14.0 29.79%
35
+ context token delta 76 76 0.0 0.00%
36
+ baseline boot (median) 2186.1 2267.2 81.1 3.71%
37
+ baseline assembly (median) 33 41 8.0 24.24%
38
+
39
+ worst spread on a graded delta: 1.90% of its baseline (target: under 5%)
40
+ PASS
41
+ ```
42
+
43
+ Reading:
44
+
45
+ * The **context delta is exact** (identical bytes, identical tokens) because the
46
+ prompt is a function of the profile, not of the machine.
47
+ * The **graded deltas repeat inside 1.9% of their baseline**: boot CPU 0.00%,
48
+ assembly 1.90%. Both grades are noise-adjusted, so the fixture plugin (which
49
+ contributes one tool, one prompt section and two listeners) grades A in both
50
+ runs.
51
+ * The **raw wall-clock deltas** also repeated well in this run — the round-paired
52
+ boot median moved 0.86%, the boot best case 4.58% — but that is luck, not a
53
+ guarantee: earlier in the same session, on a busier machine, baseline boot
54
+ medians moved by 19% between cards. The card prints its jitter and noise floor
55
+ so a reader can tell which situation they are in.
56
+ * `step` (the whole `step_start → step_end` window) is the noisiest figure at
57
+ 30% of a ~45 ms baseline; that is why it is reported but not graded. `assembly`
58
+ — the phase plugin hooks actually run in — is five times steadier.
59
+
60
+ ## 2. A measured plugin cannot touch the real profile
61
+
62
+ Every report runs in a fresh tree and points the child at it:
63
+
64
+ * `DSH_HOME` is a new `%TEMP%\dsh-report-card-*` directory, removed after the
65
+ report unless `--keep` is passed.
66
+ * The profile is a fresh copy of the shipped `headless` template under that
67
+ home; the subject is copied or installed into *that* profile's
68
+ `node_modules`.
69
+ * The working directory is an empty `workspace` directory under the same temp
70
+ tree.
71
+ * The probe records the state it actually saw. A real card contains:
72
+
73
+ ```json
74
+ "measuredEnvironment": {
75
+ "home": "C:\\Users\\conta\\AppData\\Local\\Temp\\dsh-report-card-XXXXXXXX\\home",
76
+ "cwd": "C:\\Users\\conta\\AppData\\Local\\Temp\\dsh-report-card-XXXXXXXX\\workspace"
77
+ }
78
+ ```
79
+
80
+ so the isolation is part of the evidence, not a promise. `~/.dsh` is never
81
+ passed to the child; the parent harness process never imports the subject.
82
+
83
+ The one thing this does **not** do is sandbox the subject from the operating
84
+ system: it runs as your user inside that disposable process. The README says so
85
+ and the card lists the static risk flags before you install anything.
86
+
87
+ ## 3. Every number is reproducible from a command
88
+
89
+ * The card prints its own command in the **Reproduce** section, e.g.
90
+ `node harness/cli.mjs "./test/fixtures/fake-plugin" --samples 3`.
91
+ * `harness/cli.mjs --json` prints the same card as JSON, including every sample
92
+ and the digest.
93
+ * `harness/repeat.mjs` runs a card N times and prints the spread (section 1).
94
+ * A card's `digest` is the SHA-256 of the canonical JSON with the digest field
95
+ removed; `digestReport()` in `harness/report.mjs` recomputes it, and
96
+ `test/unit.test.mjs` proves the round-trip.
97
+
98
+ ## 4. `node test/*.mjs` covers the harness with fake plugins
99
+
100
+ ```
101
+ $ node test/unit.test.mjs
102
+ 12 passed, 0 failed
103
+
104
+ $ node test/harness.test.mjs
105
+ ok a fake plugin gets a complete, measured card
106
+ ok a plugin that fails to start is a failed card, not a crash
107
+ ok a plugin that kills the host is a failed boot, bounded
108
+ ok the second card compares with the first one for drift
109
+
110
+ 4 passed, 0 failed
111
+ ```
112
+
113
+ The end-to-end test asserts the fixture's exact capability surface
114
+ (`fixture_echo`, `tool:fixture_echo`, the `tools/pre-execute` listener, the
115
+ loader entry and its `active` phase), the risk flags (install script, network
116
+ host), the digest, the saved files, the failed-card path for a plugin that
117
+ throws in `apply()`, the failed-boot path for a plugin that calls
118
+ `process.exit()`, and drift on the second card.
119
+
120
+ ## 5. What v1 does not claim
121
+
122
+ * Cross-machine comparability. Cards record the machine identity and are only
123
+ comparable within it.
124
+ * Multi-turn, compaction, or client-side cost. One scripted turn, host side.
125
+ * Sandboxing the subject. See section 2.
126
+ * Billing-grade tokens. The estimate is the shipped token meter's documented
127
+ 4-characters-per-token heuristic; CJK and JSON schemas are underpriced.
@@ -0,0 +1,106 @@
1
+ # Report card — @local/rc-fake-plugin@0.1.0
2
+
3
+ **Grade D** · graded boot +6.3 ms CPU · graded turn 0 ms · context +76 tok · risk D
4
+
5
+ > Fake plugin used by the report-card harness tests: one tool, one prompt section, one listener.
6
+
7
+ ## At a glance
8
+
9
+ | Dimension | Baseline | Subject | Delta | Grade |
10
+ | --- | --- | --- | --- | --- |
11
+ | Boot wall (process → plugin tree) | 2397 / 2619 / 2734 ms | 2465 / 2625 / 2805 ms | +67.9 ms | — |
12
+ | Boot CPU (process at ready) | 2780 / 2952 / 2953 ms | 2827 / 2921 / 2983 ms | +30 ms | A |
13
+ | Turn (turn_start → turn_end) | 142 / 101 / 154 ms | 114 / 111 / 114 ms | -28 ms | — |
14
+ | Assembly (step/start → request) | 88 / 38 / 60 ms | 48 / 46 / 45 ms | -15 ms | A |
15
+ | Step (prompt → model answered) | 105 / 52 / 79 ms | 65 / 65 / 61 ms | -18 ms | — |
16
+ | Context (model-visible tokens) | 6237 | 6313 | +76 tok | A |
17
+ | Static risk flags | — | 3 | — | D |
18
+
19
+ Samples: 3 per variant, medians shown. Boot jitter +2.5%; step jitter +67.1%.
20
+
21
+ Grades charge only overhead above this machine's noise floor: boot CPU +6.3 ms graded from +30 ms raw (floor +23.7 ms), turn assembly 0 ms graded from -15 ms raw (floor +13.9 ms). Wall time is reported but never graded: it is hostage to whatever else the machine is doing.
22
+
23
+ ## Boot
24
+
25
+ | Sample | Baseline | Subject |
26
+ | --- | --- | --- |
27
+ | 1 | 2397 ms | 2465 ms |
28
+ | 2 | 2619 ms | 2625 ms |
29
+ | 3 | 2734 ms | 2805 ms |
30
+
31
+ Measured as process start (performance.timeOrigin) to the probe writing its dump (plugin tree composed). Deltas are round-paired medians (each subject run is paired with the baseline run before it), which cancels slow machine drift; the unpaired median difference was +6.3 ms. A negative delta means the subject booted sooner.
32
+
33
+ Launch (spawn → the same dump, including process startup): baseline 2724 ms, subject 2734 ms, delta +36 ms.
34
+
35
+ Boot CPU (the graded figure, total process CPU when the plugin tree is ready): baseline 2952 ms, subject 2921 ms, delta +30 ms raw, +6.3 ms after the noise floor. CPU time moves only when the plugin actually does work, so it stays stable on a loaded machine.
36
+
37
+ ## One scripted turn
38
+
39
+ | Metric | Baseline | Subject | Delta |
40
+ | --- | --- | --- | --- |
41
+ | Turn | 142 / 101 / 154 ms | 114 / 111 / 114 ms | -28 ms |
42
+ | Assembly (hook phase) | 88 / 38 / 60 ms | 48 / 46 / 45 ms | -15 ms |
43
+ | Step (whole) | 105 / 52 / 79 ms | 65 / 65 / 61 ms | -18 ms |
44
+
45
+ Best-case assembly delta +7 ms; graded assembly delta 0 ms — the per-turn figure that sets the turn grade.
46
+
47
+ Measured as step_start to the first step_end on that turn, against local fixture endpoint, no network call.
48
+
49
+ ## Context tax
50
+
51
+ | Field | Baseline | Subject | Delta |
52
+ | --- | --- | --- | --- |
53
+ | Model-visible chars | 24945 | 25249 | +304 |
54
+ | · system prompt | 2758 | 2826 | +68 |
55
+ | · tool schemas | 20369 | 20605 | +236 |
56
+ | · conversation | 1818 | 1818 | 0 |
57
+ | Estimated tokens | 6237 | 6313 | +76 |
58
+ | Harness extension fields | 34142 | 34497 | +355 |
59
+
60
+ Measured as model-visible bytes of the request the harness sent (system + tools + messages), tokens at 4 chars/token. Extension fields are request metadata the harness sends beside the model-visible prompt; they are listed separately and excluded from the token estimate.
61
+
62
+ ## Capability surface
63
+
64
+ | Added | Detail |
65
+ | --- | --- |
66
+ | Tools | `fixture_echo` (235 schema chars) |
67
+ | Prompt sections | `tool:fixture_echo` (66 chars) |
68
+ | Event listeners | `tools/pre-execute` +1, `session/start` +1 |
69
+ | Loader entries | `@local/rc-fake-plugin` |
70
+ | Subject entry phase | active |
71
+ | Prompt variables | none |
72
+ | Profile size | 101 → 102 entries |
73
+
74
+ ## Static risk flags
75
+
76
+ | Level | Flag | Detail |
77
+ | --- | --- | --- |
78
+ | high | `install-scripts` | runs code during install: postinstall |
79
+ | medium | `file-write` | writes to the filesystem |
80
+ | info | `network-hosts` | 1 host referenced: collector.example.test |
81
+
82
+ Evidence:
83
+ - `file-write` — index.js: `const writeReport = (file) => fs.writeFileSync(file, uploadUrl)`
84
+
85
+ ## Notes
86
+
87
+ - Step samples vary by 67.1%; treat deltas below that as noise.
88
+
89
+ ## Method and limits
90
+
91
+ Measured on win32-x64, 8 CPUs, dsh 0.2.2-rc.1, Node v24.19.0.
92
+
93
+ - Profile: `headless` template, fresh profile copied from a shipped template, fresh DSH_HOME per report.
94
+ - Isolation: empty workspace directory, read-only file policy; the subject never touches your real profile.
95
+ - The subject still runs with your user privileges inside that disposable process. Read the risk flags before installing anything you did not write.
96
+ - Numbers are comparable only on the same machine identity; the digest changes whenever any number does.
97
+
98
+ ## Reproduce
99
+
100
+ ```
101
+ node harness/cli.mjs "./test/fixtures/fake-plugin" --samples 3
102
+ ```
103
+
104
+ Digest: `sha256:55ffdbf5e230a5fbc258445d58a126290e97e0e2c47fe8cfbaae36bd122918c3`
105
+
106
+ Generated 2026-11-10T16:27:44.348Z by `dsh-report-card` (dsh-report-card/1).