dsh-test-drive 0.3.24 → 0.3.26

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (46) hide show
  1. package/CHANGELOG.md +12 -0
  2. package/LICENSE +0 -0
  3. package/README.en.md +548 -0
  4. package/README.es.md +548 -0
  5. package/README.hi.md +548 -0
  6. package/README.md +337 -229
  7. package/README.pt.md +548 -0
  8. package/SECURITY.md +0 -0
  9. package/THIRD_PARTY_NOTICES.md +0 -0
  10. package/cordis.patch.yml +0 -0
  11. package/lib/index.js +16 -31
  12. package/lib/types/batch.d.ts +0 -0
  13. package/lib/types/batch.d.ts.map +0 -0
  14. package/lib/types/capability.d.ts +0 -0
  15. package/lib/types/capability.d.ts.map +0 -0
  16. package/lib/types/command.d.ts +0 -0
  17. package/lib/types/command.d.ts.map +0 -0
  18. package/lib/types/config.d.ts +0 -0
  19. package/lib/types/config.d.ts.map +0 -0
  20. package/lib/types/domain.d.ts +0 -0
  21. package/lib/types/domain.d.ts.map +0 -0
  22. package/lib/types/drive.d.ts +0 -0
  23. package/lib/types/drive.d.ts.map +0 -0
  24. package/lib/types/driver.d.ts +0 -0
  25. package/lib/types/driver.d.ts.map +0 -0
  26. package/lib/types/index.d.ts +0 -0
  27. package/lib/types/index.d.ts.map +0 -0
  28. package/lib/types/report.d.ts +0 -0
  29. package/lib/types/report.d.ts.map +0 -0
  30. package/lib/types/result.d.ts +0 -0
  31. package/lib/types/result.d.ts.map +0 -0
  32. package/lib/types/sanitize.d.ts +0 -0
  33. package/lib/types/sanitize.d.ts.map +0 -0
  34. package/lib/types/session-log.d.ts +0 -0
  35. package/lib/types/session-log.d.ts.map +0 -0
  36. package/lib/types/tools.d.ts +0 -0
  37. package/lib/types/tools.d.ts.map +0 -0
  38. package/lib/types/version.d.ts +1 -1
  39. package/lib/types/version.d.ts.map +0 -0
  40. package/lib/types/workspace.d.ts +0 -0
  41. package/lib/types/workspace.d.ts.map +0 -0
  42. package/package.json +64 -38
  43. package/README-es.md +0 -418
  44. package/README-hi.md +0 -418
  45. package/README-pt.md +0 -418
  46. package/README-zh.md +0 -417
package/CHANGELOG.md CHANGED
@@ -1,5 +1,17 @@
1
1
  # Changelog
2
2
 
3
+ ## [0.3.25] - 2026-11-05
4
+
5
+ ### Changed
6
+
7
+ - **Host 0.2.2 line support.** devDependencies move from 0.2.1-rc.2 to 0.2.2-rc.1, cordis to 4.1.0 / include 1.1.0 / loader 1.1.0, and schemastery to ~3.19.0 so a single copy resolves against `@deepseek-ai/dsh-llm` 0.2.2. `engines.dsh` appends `>=0.2.2-0 <0.3.0` (append-only), `engines.node` becomes `^22.19.0 || >=24.9.0`, and `dshWorkshop.compatibility.dshVersions` lists `0.2.2-rc.1`.
8
+ - **The tests drive the 0.2.2 batch tool runtime.** The harness mounts `@deepseek-ai/dsh-tool-executor-native` after `ToolRuntime` (0.2.2's `ToolRuntime` delegates a raw `ToolCallBlock` batch to a selected executor) and `SessionProjectionRegistry` between the session store and `AgentRegistry` (0.2.2's `AgentRegistry` injects `sessionProjections`). The tool-call helpers now send `{ calls: [{ type: 'tool-call', id, name, arguments: JSON.stringify(args) }] }` and read `results[0].result`.
9
+
10
+ ### Fixed
11
+
12
+ - Add `@deepseek-ai/dsh-package-manifest` (0.2.2 `dsh-llm` types import it without declaring it), plus `@deepseek-ai/dsh-tool-executor-native` and `@deepseek-ai/dsh-session-projection` as devDependencies the tests need to publish an executor and the projection registry.
13
+ - `tests/helpers/memory-backend.ts` implements `KvUnit.deleteRecords`: the 0.2.2 storage backend contract gained the batch delete, so the in-memory double must satisfy it.
14
+
3
15
  ## [0.3.24] - 2026-10-24
4
16
 
5
17
  ### Fixed
package/LICENSE CHANGED
File without changes
package/README.en.md ADDED
@@ -0,0 +1,548 @@
1
+ <div align="center">
2
+
3
+ <picture>
4
+ <source media="(prefers-color-scheme: dark)" srcset="docs/assets/dsh-test-drive-hero-en-dark.webp">
5
+ <img src="docs/assets/dsh-test-drive-hero-en-light.webp" width="100%" alt="dsh-test-drive: the plugin name and two value lines on the left; on the right the receipt of one real drive — target dsh-click, run id tdr_0b9b…, the six stages with their status words and durations, an isolation strip naming the throwaway DSH_HOME, working directory and pnpm store while the host profile is never read or written, and the verdict line plus the dsh-test-drive/v1 record outlet">
6
+ </picture>
7
+
8
+ # dsh-test-drive
9
+
10
+ **Install, boot, assert, uninstall, clean up inside a throwaway profile — turn "does it even run?" into a reproducible pipeline.**
11
+
12
+ Isolated install-and-smoke test drives for DeepSeek Harness plugins: every target gets its own `mkdtemp` root (a throwaway `DSH_HOME`, a throwaway working directory, a redirected pnpm store), your real `~/.dsh` is never read or written, and the result is a structured record downstream tools can consume.
13
+
14
+ [Install](#three-step-setup) · [Configuration](#configuration) · [FAQ](#faq) · [简体中文](README.md) · [English](README.en.md) · [Español](README.es.md) · [Português](README.pt.md) · [हिन्दी](README.hi.md)
15
+
16
+ [![License](https://img.shields.io/badge/license-Apache--2.0-3366CC?style=flat-square&labelColor=2B3245)](LICENSE)
17
+ [![DSH](https://img.shields.io/badge/DSH-%E2%89%A50.2.1%20%3C0.3.0-4B67B4?style=flat-square&labelColor=2B3245)](#compatibility)
18
+ [![npm](https://img.shields.io/npm/v/dsh-test-drive?style=flat-square&labelColor=2B3245&color=CB3837)](https://www.npmjs.com/package/dsh-test-drive)
19
+ [![CI](https://img.shields.io/github/actions/workflow/status/PerryLink/dsh-test-drive/ci.yml?branch=main&label=CI&style=flat-square&labelColor=2B3245)](https://github.com/PerryLink/dsh-test-drive/actions/workflows/ci.yml)
20
+ [![Node](https://img.shields.io/badge/node-%5E22.19%20%7C%7C%20%3E%3D24.9-339933?style=flat-square&labelColor=2B3245)](#compatibility)
21
+
22
+ [![Stars](https://img.shields.io/github/stars/PerryLink/dsh-test-drive?style=flat-square&labelColor=2B3245)](https://github.com/PerryLink/dsh-test-drive/stargazers)
23
+ [![npm downloads](https://img.shields.io/npm/dm/dsh-test-drive?style=flat-square&labelColor=2B3245&color=CB3837)](https://www.npmjs.com/package/dsh-test-drive)
24
+ [![dshfind](https://dshfind.com/api/badge/PerryLink/dsh-test-drive?metric=downloads)](https://dshfind.com/plugins/PerryLink/dsh-test-drive?ref=badge)
25
+ [![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/PerryLink/dsh-test-drive/badge)](https://securityscorecards.dev/viewer/?uri=github.com/PerryLink/dsh-test-drive)
26
+ [![DSH Market](https://img.shields.io/badge/DSH_Market-listed-4B67B4?style=flat-square&labelColor=2B3245)](https://dsh.market/)
27
+ [![Gitee](https://img.shields.io/badge/Gitee-mirror-C71D23?style=flat-square&labelColor=2B3245&logo=gitee)](https://gitee.com/perrylink/dsh-test-drive)
28
+
29
+ ```sh
30
+ dsh plugin --profile <name> add dsh-test-drive
31
+ ```
32
+
33
+ </div>
34
+
35
+ <sub>Community plugin, not affiliated with DeepSeek. Canonical repository: <https://github.com/PerryLink/dsh-test-drive> (Gitee is a mirror).</sub>
36
+
37
+ ## What it is
38
+
39
+ `dsh-test-drive` turns "can this plugin even install, and does it boot?" into a six-stage pipeline: `dsh plugin add` into a throwaway `DSH_HOME` → `--dump-config` to prove the bundle patch layer really took effect → a headless boot that scans for loader failure markers → an optional capability assertion (drive the agent to call a named tool or command, then check that the durable session log recorded the invocation and the observed output contains the expected literal) → `dsh plugin remove` → cleanup of the temp roots this plugin owns through the dry-run → quarantine-rename → delete ladder. Every stage carries a status, exit code, duration and a sanitized output tail, and the whole run is one `schema: "dsh-test-drive/v1"` record: `test_drive` for a single target, `/testdrive` for a batch, `drive_report` to read results back.
40
+
41
+ <picture>
42
+ <source media="(prefers-color-scheme: dark)" srcset="docs/assets/dsh-test-drive-flow-v5-dark.svg">
43
+ <img src="docs/assets/dsh-test-drive-flow-v5-light.svg" width="100%" alt="dsh-test-drive structure: one owned temp root drawn as a clean-room envelope — inside are the three throwaway sub-roots (home/, workspace/, pnpm-store/) and the vertical five-stage spine install → config → smoke → capability → uninstall; at the floor is the finally drain (dry-run plan → quarantine rename → delete retries) that pass, fail, timeout and abort all go through. Outside on the left are the three entry surfaces, the target kinds and the host profile that is never opened; outside on the right are the dsh-test-drive/v1 record, the ordered verdict rules and the /testdrive batch matrix">
44
+ </picture>
45
+
46
+ ## Three-step setup
47
+
48
+ 1. **Install** (two channels):
49
+
50
+ ```sh
51
+ # npm channel (published releases; prebuilt packages need no build allowance)
52
+ dsh plugin --profile web add dsh-test-drive
53
+ # git channel (latest main; same as git+https://github.com/PerryLink/dsh-test-drive.git)
54
+ dsh plugin --profile web add github:PerryLink/dsh-test-drive#<commit-sha>
55
+ ```
56
+
57
+ The first `add` on the git channel fails because pnpm blocks the package's `prepare` build; copy the exact key pnpm printed into the profile's `pnpm-workspace.yaml` and re-run:
58
+
59
+ ```yaml
60
+ allowBuilds:
61
+ 'dsh-test-drive': true
62
+ ```
63
+
64
+ Prebuilt npm packages need no build allowance. Restart the profile, then use `test_drive` / `/testdrive` from a session.
65
+
66
+ 2. **Minimal config** (the defaults already work; only the capability stage needs `capability.*` and `forwardEnv`):
67
+
68
+ ```yaml
69
+ - insert:
70
+ - id: test-drive
71
+ name: dsh-test-drive
72
+ config:
73
+ headlessTask: "Reply with exactly: ok"
74
+ forwardEnv: [DEEPSEEK_API_KEY]
75
+ capability:
76
+ enabled: true
77
+ kind: tool
78
+ name: plugin_vet
79
+ args: '{}'
80
+ expect: pass
81
+ ```
82
+
83
+ 3. **Verify it took effect** (after a restart):
84
+
85
+ ```sh
86
+ dsh --profile web --dump-config | grep -A3 'id: test-drive'
87
+ ```
88
+
89
+ Then drive a known target for the first time:
90
+
91
+ ```text
92
+ test_drive({ target: "dsh-click" })
93
+ → verdict pass · tdr_0b9b6e6a61e3e80cc4620f2e · full record in "Examples"
94
+ ```
95
+
96
+ **Uninstall**: `dsh plugin --profile web remove dsh-test-drive` (or delete the line from the profile patch).
97
+
98
+ > `test_drive` needs a `dsh` CLI that can boot a **second** headless DSH process. The DSH Desktop launcher (`dsh.CMD`) only permits `dsh plugin …`, `--dump-config` and `--version`, and rejects a second headless boot by policy — the error now says so explicitly instead of "cannot parse npm shim". Point `config.dshBin` at an npm/pnpm-installed `dsh` CLI, or a plain Node + dsh JS entry.
99
+
100
+ <details>
101
+ <summary>Install from the DSH Desktop marketplace</summary>
102
+
103
+ Every PerryLink plugin is listed in the DSH Desktop built-in marketplace: **Market → Sources → Add source → paste** `https://perrylink-dsh-catalog.perrylink.workers.dev/catalog-source.json` **→ select**. Installation still goes through the marketplace's npm identity check and your confirmation.
104
+
105
+ </details>
106
+
107
+ ## Core capabilities
108
+
109
+ `dsh-test-drive` splits "verify this install" into stages you can watch and replay:
110
+
111
+ - **`test_drive` tool** — runs the full pipeline for one target and returns the structured record synchronously; `background: true` starts a `drive-batch` job and returns `{ kind: 'background', jobId }`.
112
+ - **`/testdrive <targets...>` command** — takes a whitespace/comma-separated target list, runs it as one `drive-batch` background job, streams progress through the job output, and finishes with a matrix report (JSON + Markdown) whose last line names the matrix id.
113
+ - **`drive_report` tool** — fetches any stored run (`tdr_...`), matrix (`tdm_...`), or the latest matrix when no id is given, rendered as Markdown.
114
+ - **Capability assertion** — beyond "booted and exited": the optional `capability` stage drives the agent to call a named tool (`kind: tool`) or run `/command` (`kind: command`) with `args`, then verifies the durable session log recorded the invocation and the observed output contains `expect`. A clean boot is only a smoke test; `observed` proves a named capability really works.
115
+ - **Structured results** — every record carries the discriminator `schema: "dsh-test-drive/v1"` with first-class fields: `stages.install.status`, `stages.smoke.status`, `stages.capability.status`, per-stage `durationMs`, sanitized `summary` / `outputTail`, and an overall `verdict` (`pass` / `fail` / `partial` / `unknown`). This is the machine-readable contract downstream scorers (dsh-score) consume.
116
+ - **Safety by construction** — every temp directory is created by this plugin under a dedicated `dsh-test-drive-` prefix, tracked in a live ownership registry, and removed only through the dry-run → quarantine-rename → delete ladder. The host profile is never read or written.
117
+
118
+ ```text
119
+ target ──dsh plugin add──▶ throwaway DSH_HOME ──--dump-config──▶ patch layer
120
+ │ │
121
+ ├──headless boot──▶ FAILED-marker scan
122
+ ├──capability────▶ tool/command + session log
123
+ └──dsh plugin remove──▶ quarantine → delete
124
+ ```
125
+
126
+ ### Tool surface
127
+
128
+ | Surface | Form | Returns / notes |
129
+ |---|---|---|
130
+ | `test_drive` | tool | the full structured record (synchronous), or `{ kind: 'background', jobId }` |
131
+ | `/testdrive` | command | one `drive-batch` job; the last line names the matrix id for `drive_report` |
132
+ | `drive_report` | tool | a `tdr_...` run, a `tdm_...` matrix, or the latest matrix, rendered as Markdown |
133
+
134
+ `test_drive`'s `target` accepts a git spec (`github:owner/repo#sha`, `git+https://...`), an npm package name, a local path, or a `.tgz` tarball; `capability` looks like `{ kind: 'tool' | 'command', name, args, expect }` and needs `DEEPSEEK_API_KEY` (host environment or `forwardEnv`) — without it the stage is `skipped`, never failed.
135
+
136
+ ### Structured result
137
+
138
+ Verdict rules: an install failure, a boot failure (`smoke.fail`), or a capability stage that reached `not-registered` / `failed` ⇒ `fail`; install pass + patch effective + clean boot (`pass` / `boot-ok`) + uninstall pass ⇒ `pass` (with a note when the capability stage reached `observed`); anything installed but missing a later assurance ⇒ `partial`; otherwise ⇒ `unknown`.
139
+
140
+ <details>
141
+ <summary>Full record sample (dsh-test-drive/v1)</summary>
142
+
143
+ ```json
144
+ {
145
+ "schema": "dsh-test-drive/v1",
146
+ "run": { "runId": "tdr_9f2c...", "startedAt": "2026-08-16T00:00:00.000Z",
147
+ "finishedAt": "2026-08-16T00:00:45.120Z", "durationMs": 45120,
148
+ "harnessVersion": "0.1.7-rc.2", "pluginVersion": "0.3.13",
149
+ "platform": "win32", "node": "v22.22.3" },
150
+ "target": { "kind": "repo", "spec": "github:owner/dsh-click#abc123",
151
+ "resolved": { "packageName": "dsh-click", "packageVersion": "0.1.0",
152
+ "hasBundleManifest": true } },
153
+ "isolation": { "tempDshHome": true, "tempWorkspace": true, "tempStore": true,
154
+ "hostHomeTouched": false },
155
+ "stages": {
156
+ "install": { "status": "pass", "exitCode": 0, "durationMs": 30412, "attempts": 2,
157
+ "summary": "install ok after allowBuilds allowance", "outputTail": "",
158
+ "allowBuildsNeeded": true },
159
+ "config": { "status": "pass", "exitCode": 0, "durationMs": 2310, "attempts": 1,
160
+ "summary": "dump ok (exit 0)", "outputTail": "",
161
+ "patchEffective": true, "layers": ["dsh-click"] },
162
+ "smoke": { "status": "boot-ok", "exitCode": 1, "durationMs": 4123, "attempts": 1,
163
+ "summary": "booted without loader failures; headless task did not complete (credentials/model unreachable)",
164
+ "outputTail": "", "bootFailed": false, "taskCompleted": false },
165
+ "capability": { "status": "observed", "exitCode": 0, "durationMs": 8123, "attempts": 1,
166
+ "summary": "tool \"plugin_vet\" called and its result contains the expectation",
167
+ "outputTail": "", "capabilityKind": "tool", "name": "plugin_vet",
168
+ "expectMatched": true,
169
+ "detail": "tool \"plugin_vet\" called and its result contains the expectation" },
170
+ "uninstall": { "status": "pass", "exitCode": 0, "durationMs": 5123, "attempts": 1,
171
+ "summary": "remove ok (exit 0)", "outputTail": "" },
172
+ "cleanup": { "status": "pass", "quarantined": true, "removed": true,
173
+ "summary": "owned temp root quarantined and removed" }
174
+ },
175
+ "verdict": "pass",
176
+ "verdictReason": "install, patch, boot, and uninstall verified; headless task inconclusive (see smoke.summary)"
177
+ }
178
+ ```
179
+
180
+ </details>
181
+
182
+ ### Examples
183
+
184
+ These are the plugin's real entry points; install once, then call one.
185
+
186
+ **Detailed example** — command `test_drive`, input:
187
+
188
+ ```sh
189
+ test_drive({ target: "dsh-click" })
190
+ ```
191
+
192
+ Output:
193
+
194
+ ```text
195
+ # Test drive tdr_0b9b6e6a61e3e80cc4620f2e
196
+
197
+ - Target: `dsh-click` (npm)
198
+ - Verdict: ✅ pass — install, patch, boot, task, and uninstall all verified
199
+ - Installed: dsh-click@0.2.0
200
+ - Isolation: throwaway DSH_HOME + workspace + pnpm store; host profile untouched
201
+
202
+ | Stage | Result | Exit | Duration | Summary |
203
+ |---|---|---|---|---|
204
+ | install | ✅ pass | 0 | 1 ms | install ok (exit 0) |
205
+ | config (--dump-config) | ✅ pass | 0 | 0 ms | patch effective: dsh-click |
206
+ | smoke (headless boot) | ✅ pass | 0 | 0 ms | booted cleanly and the headless task completed |
207
+ | capability | ⏭️ skipped | null | 0 ms | capability assertion disabled (config.capability.enabled is false and no per-drive capability given) |
208
+ | uninstall | ✅ pass | 0 | 0 ms | remove ok (exit 0) |
209
+ | cleanup | ✅ pass | — | — | owned temp root quarantined and removed |
210
+ ```
211
+
212
+ *Observed from `README.md:153-163,174-211; tests/drive.spec.ts:73-101; src/report.ts:34-55`*
213
+
214
+ **Failure branch** — input:
215
+
216
+ ```text
217
+ {"target":" "}
218
+ ```
219
+
220
+ Output:
221
+
222
+ ```text
223
+ isError=true; error.message=dsh-test-drive: target must be a non-empty string
224
+ ```
225
+
226
+ *Observed from `plugins/dsh-test-drive/src/tools.ts:297-299 (blank target check); exercised through plugins/dsh-test-drive/tests/harness.ts`*
227
+
228
+ **Brief examples**: `test_drive` · `/testdrive` · `drive_report`
229
+
230
+ ## Configuration
231
+
232
+ All keys are optional (defaults shown); invalid values fail loudly at load. `cordis.patch.yml` documents every key inline.
233
+
234
+ Most-used keys:
235
+
236
+ | Key | Default | Description |
237
+ |---|---|---|
238
+ | `profileName` | `headless` | Profile template initialized inside each throwaway DSH_HOME (base + headless bundles). |
239
+ | `dshBin` | `""` | Absolute `dsh` executable override; empty auto-detects PATH (on Windows the npm `.cmd`/`.bat` shim is parsed automatically). |
240
+ | `headlessTask` | `"Reply with exactly: ok"` | One-shot task for the boot-smoke stage; empty skips the stage. |
241
+ | `forwardEnv` | `[]` | Environment VARIABLE NAMES (never values) forwarded into test-profile child processes; values are never logged. |
242
+ | `allowBuilds` | `true` | Allowlist a blocked git `prepare` build in the test profile and retry the install once. |
243
+ | `capability.enabled` | `false` | Run the capability-assertion stage (registered → invoked → observed). |
244
+ | `capability.expect` | `""` | Literal expected in the observed output (case-insensitive substring). |
245
+ | `keepTempDirs` | `false` | Keep temp dirs on failure for forensics (ownership is dropped; you clean up). |
246
+
247
+ <details>
248
+ <summary>Full configuration table (19 keys)</summary>
249
+
250
+ | Key | Default | Description |
251
+ |---|---|---|
252
+ | `profileName` | `headless` | Profile template initialized inside each throwaway DSH_HOME (base + headless bundles). |
253
+ | `dshBin` | `""` | Absolute `dsh` executable override; empty auto-detects PATH (on Windows the npm `.cmd`/`.bat` shim is parsed automatically). |
254
+ | `headlessTask` | `"Reply with exactly: ok"` | One-shot task for the boot-smoke stage; empty skips the stage. |
255
+ | `forwardEnv` | `[]` | Environment VARIABLE NAMES (never values) forwarded into test-profile child processes; values are never logged. |
256
+ | `allowBuilds` | `true` | Allowlist a blocked git `prepare` build in the test profile and retry the install once. |
257
+ | `installTimeoutMs` | `600000` | `dsh plugin add` stage deadline (ms). |
258
+ | `configTimeoutMs` | `60000` | `--dump-config` stage deadline (ms). |
259
+ | `smokeTimeoutMs` | `300000` | Headless boot-smoke stage deadline (ms). |
260
+ | `capabilityTimeoutMs` | `300000` | Capability-assertion task deadline (ms). |
261
+ | `capability.enabled` | `false` | Run the capability-assertion stage (registered → invoked → observed). |
262
+ | `capability.kind` | `tool` | What to assert: `tool` or `command`. |
263
+ | `capability.name` | `""` | Tool or command name (no leading `/`). |
264
+ | `capability.args` | `""` | Invocation text: tool arguments (JSON-ish) or command words. |
265
+ | `capability.expect` | `""` | Literal expected in the observed output (case-insensitive substring). |
266
+ | `uninstallTimeoutMs` | `120000` | `dsh plugin remove` stage deadline (ms). |
267
+ | `outputTailBytes` | `8000` | Cap on the sanitized output tail recorded per stage (bytes). |
268
+ | `keepTempDirs` | `false` | Keep temp dirs on failure for forensics (ownership is dropped; you clean up). |
269
+ | `maxBatchTargets` | `20` | `/testdrive` batch cap. |
270
+ | `batchConcurrency` | `1` | Batch concurrency (serial avoids pnpm-store contention). |
271
+
272
+ </details>
273
+
274
+ Override example in a profile patch:
275
+
276
+ ```yaml
277
+ - insert:
278
+ - id: test-drive
279
+ name: dsh-test-drive
280
+ config:
281
+ headlessTask: "Reply with exactly: ok"
282
+ forwardEnv: [DEEPSEEK_API_KEY]
283
+ capability:
284
+ enabled: true
285
+ kind: tool
286
+ name: plugin_vet
287
+ args: '{}'
288
+ expect: pass
289
+ ```
290
+
291
+ ## FAQ
292
+
293
+ <details>
294
+ <summary>Does it touch my real <code>~/.dsh</code>?</summary>
295
+
296
+ No. Every drive runs inside a fresh `mkdtemp` root under the OS temp directory: a throwaway `DSH_HOME`, a throwaway working directory, and a redirected pnpm store. The tested plugin's code only ever runs in that profile; the host profile is neither read nor written, and the structured record states `isolation.hostHomeTouched: false`. Cleanup only accepts directories this plugin registered — no `%TEMP%` sweeps, no foreign prefixes, no real-home paths.
297
+
298
+ </details>
299
+
300
+ <details>
301
+ <summary>What happens without a <code>storageDomain</code> in the composition?</summary>
302
+
303
+ The tools still work, but reports are not persisted: the plugin logs a reason and disables persistence, and `drive_report` fails honestly instead of pretending. Conversely, compositions that do have a `storageDomain` store the records in the `test_drive` domain (tables `runs`, `matrices`, plus the latest-matrix pointer).
304
+
305
+ </details>
306
+
307
+ <details>
308
+ <summary>Do tested child processes get my credentials?</summary>
309
+
310
+ Not by default. Child processes inherit a credential-scrubbed environment; unless you explicitly name a variable in `forwardEnv` (names only), host secrets never reach a tested profile, and those values are never logged. Every report and log string also passes through pure sanitizers: token literals, URL credentials and bearer headers are redacted, temp-root paths become `<testdrive-temp>`, and output tails are byte-capped.
311
+
312
+ </details>
313
+
314
+ <details>
315
+ <summary>Why does the smoke stage often stop at <code>boot-ok</code>?</summary>
316
+
317
+ Because the boot check itself needs no credentials, while actually completing the one-shot task does. Without credentials the plugin reports the honest `boot-ok` — it saw no loader failure and did not finish the task — instead of recording something unverified as `pass`. To reach `pass`, forward credentials explicitly with `forwardEnv` (their values still never enter the log).
318
+
319
+ </details>
320
+
321
+ <details>
322
+ <summary>Do temp directories leak when a drive fails?</summary>
323
+
324
+ Not silently. Cleanup runs in a `finally` across success, failure, timeout and abort, and once more on plugin teardown; removal first renames the root into a `dsh-test-drive-quarantine-<ts>` directory, verifies it, and only then deletes. A failed removal leaves the directory quarantined and reported. To keep the crime scene for forensics, set `keepTempDirs: true` — ownership is then dropped and cleanup becomes your job.
325
+
326
+ </details>
327
+
328
+ ## Security boundaries
329
+
330
+ - **Isolation.** Each drive runs inside a fresh `mkdtemp` root under the OS temp dir: a throwaway `DSH_HOME`, a throwaway working directory, and a redirected pnpm store. The tested plugin's code only ever runs in that profile; your host profile is untouched.
331
+ - **Ownership.** A live registry records every root this plugin instance creates. Cleanup refuses anything that is not a registered direct child of the OS temp dir carrying the `dsh-test-drive-` prefix — no `%TEMP%` sweeps, no foreign prefixes, no real-home paths.
332
+ - **Cleanup ladder.** Before any mutation the full dry-run plan is logged (absolute paths). Removal renames the root into a `dsh-test-drive-quarantine-<ts>` directory first, verifies, then deletes; failures leave the directory quarantined and reported, never silently dropped. Cleanup runs in a `finally` on success, failure, timeout, and abort, and again on plugin teardown.
333
+ - **`allowBuilds` is a real permission.** Allowing a git package's `prepare` build executes that package's code at install time. The allowance is scoped to the throwaway profile only, but only test targets you trust, and pin commits.
334
+ - **Headless smoke is keyless by default.** The boot check needs no credentials; completing the one-shot task does. Forward credentials explicitly (`forwardEnv`) and never log them.
335
+
336
+ ## Permissions & data
337
+
338
+ - **Service surface.** Only public services are consumed: `ctx.subprocess`, `ctx.jobs`, `ctx.storageDomain`, `ctx.tools`, `ctx.commands`. `storageDomain` is optional — without it both boot and drives keep working.
339
+ - **Permissions.** The `dshWorkshop` manifest declares `fs:read`, `fs:write`, `subprocess:spawn`, `session:append`, `network:inherit`, `native-code:none` — write access covers only the temp roots it creates and report persistence, and `network:inherit` just inherits the host (registry/git installs for the target need network anyway).
340
+ - **Data.** Reports live in the `test_drive` storage domain (tables `runs`, `matrices`, plus the latest-matrix pointer); without that domain reports are not persisted and the reason is logged.
341
+ - **Session log.** Child processes get a credential-scrubbed environment and `forwardEnv` forwards names only; every report/log string is sanitized — token literals, URL credentials and bearer headers redacted, temp-root paths replaced with `<testdrive-temp>`, output tails byte-capped.
342
+
343
+ ## Known limitations
344
+
345
+ - **DSH Desktop is not a supported test host.** `test_drive` needs a `dsh` CLI that can boot a second headless process, while the `dsh.CMD` launcher only permits `dsh plugin …`, `--dump-config` and `--version`; point `dshBin` at an npm/pnpm-installed `dsh`.
346
+ - Installing registry/git targets requires network access from the child `dsh`/pnpm processes.
347
+ - The smoke task needs model credentials to reach `pass`; without them it reports the honest `boot-ok`.
348
+ - In compositions without `storageDomain`, reports are not persisted (`drive_report` fails honestly).
349
+ - `dsh` must be locatable on PATH (or set `dshBin`); on Windows the npm `.cmd`/`.bat` shim is parsed automatically, a bare `.ps1` resolution asks for `dshBin`.
350
+ - Batches default to serial execution; raising `batchConcurrency` shares the pnpm-store disk, not correctness.
351
+
352
+ ## Interoperability with other DSH plugins
353
+
354
+ `dsh-test-drive` exposes two tools and one command, and does not join other plugins' seams:
355
+
356
+ - **Namespace isolation.** It registers only `test_drive` and `drive_report`, plus the `/testdrive` command — no bare names held by built-in tools or other plugins.
357
+ - **No client or HTTP resources.** This package has no `client/` half: no client slot keys, no `webServer` prefixes, no `agent/pre-step` or `tools/post-execute` waterfall listeners (`AGENTS.md` records that this plugin registers no waterfall listeners today), and no global patching.
358
+ - **No service-key conflicts.** It provides no service keys and consumes only `subprocess`, `jobs`, `tools`, `commands`, and the optional `storageDomain`.
359
+ - **The patch layer inserts its own row only.** The bundle patch `insert`s its single `test-drive` row and never overwrites a built-in row's `config`.
360
+ - **Registration is an effect.** Tools, the command and the domain closure all register inside `ctx.effect`; stop or hot-reload revokes only what this plugin registered, and the temp-root registry sweeps leftovers through its disposer.
361
+
362
+ ## Compatibility
363
+
364
+ | Surface | Status |
365
+ |---|---|
366
+ | Harness | DeepSeek Harness `dsh-v0.2.1-alpha.1` (GitHub tag, verified 2026-09-11: full gate chain + profile install smoke). npm dependency lines `0.1.2-rc.1` and `0.1.7-rc.2`; `engines.dsh`: `>=0.2.1-0 <0.3.0 \|\| >=0.2.2-0 <0.3.0` |
367
+ | Node | `^22.19.0 \|\| >=24.9.0` |
368
+ | Package manager | `pnpm@11.7.0` |
369
+ | Platform | Windows / macOS / Linux (pure host plugin, no UI half) |
370
+ | External tools | `dsh` CLI on PATH (auto-detected, npm shims parsed), `pnpm` on PATH |
371
+
372
+ ## Ecosystem & quality
373
+
374
+ - [Gitee mirror](https://gitee.com/perrylink/dsh-test-drive) (sync mirror; GitHub is canonical)
375
+ - [dshfind plugin page](https://dshfind.com/plugins/PerryLink/dsh-test-drive) (installs and ranking)
376
+ - [OpenSSF Scorecard report](https://securityscorecards.dev/viewer/?uri=github.com/PerryLink/dsh-test-drive) (driven by [.github/workflows/scorecard.yml](.github/workflows/scorecard.yml))
377
+ - [dsh-plugin-doctor verification report](https://github.com/PerryLink/dsh-plugin-doctor#verified-徽章) (driven by this repo's [.github/workflows/plugin-doctor.yml](.github/workflows/plugin-doctor.yml))
378
+ - [DSH Market](https://dsh.market/) (listed)
379
+ - **1024 store channel**: `npm i -g dsh1024` once, then `dsh1024 plugin --profile web add dsh-test-drive` (counts toward the [deepseek1024.com](https://deepseek1024.com) install ranking)
380
+ - [Ecosystem knowledge base](https://perrylink.github.io/dsh-plugin-guide/) — measured data, not marketing: plugin development guide · plugin-selection data · maintenance criteria
381
+
382
+ ## Development
383
+
384
+ ```sh
385
+ pnpm install # node ^22.19 || >=24.9
386
+ pnpm run typecheck # tsc: src + tests, resolving @deepseek-ai/* through the local harness checkout
387
+ pnpm run typecheck:ci # tsc: against the published host types
388
+ pnpm test # vitest
389
+ pnpm run build # node scripts/prepare.mjs (tsdown bundle + declarations)
390
+ pnpm run verify:self-contained # dependency specs resolve from the registry
391
+ pnpm run verify:artifacts # shipped files present + lib/index.js importable
392
+ pnpm run check:readmes # five-language README consistency
393
+ pnpm pack # the published tarball
394
+ ```
395
+
396
+ - `typecheck` resolves `@deepseek-ai/*` through the local harness checkout; `typecheck:ci` checks against the published host types.
397
+ - Tests use the real `Context`/`Session`/`ToolRuntime`/`LocalJobRegistry`/storage stack with a scripted subprocess provider.
398
+ - Real-CLI end-to-end (requires network + `dsh` on PATH): `DSH_TESTDRIVE_E2E=1 pnpm run test:e2e` — drives this package's own checkout through the real install-smoke loop.
399
+ - Release: `node scripts/release.mjs <x.y.z>` (bumps, stamps CHANGELOG, re-runs the gate, commits + tags; never pushes).
400
+
401
+ ### GitHub Action (action.yml)
402
+
403
+ The repository ships a composite [`action.yml`](action.yml) that reuses this plugin in any plugin repo: it drives a target in an isolated throwaway profile and emits the report pair CI consumes — Markdown (PR comment) and JUnit XML (test reporter / status check).
404
+
405
+ ```yaml
406
+ # .github/workflows/test-drive.yml
407
+ name: test-drive
408
+ on: [pull_request, workflow_dispatch]
409
+ jobs:
410
+ smoke:
411
+ runs-on: ubuntu-latest
412
+ steps:
413
+ - uses: actions/checkout@v4
414
+ - name: Drive this plugin
415
+ id: drive
416
+ uses: PerryLink/dsh-test-drive@v0.2.4
417
+ with:
418
+ target: github:${{ github.repository }}#${{ github.sha }}
419
+ - name: Publish JUnit
420
+ uses: EnricoMi/publish-unit-test-result-action@v2
421
+ with:
422
+ files: ${{ steps.drive.outputs.junit }}
423
+ - name: Comment the report
424
+ run: cat "${{ steps.drive.outputs.markdown }}"
425
+ ```
426
+
427
+ `action.yml` inputs: `target` (required), `headless-task` (optional smoke task), `dsh-version` (the `dsh` CLI spec). Outputs: `markdown`, `junit` (report paths), and `verdict` (`pass`/`fail`/`partial`/`unknown`). The drive itself stays keyless (install → patch check → boot smoke → uninstall → cleanup); a capability assertion is the only stage that needs `DEEPSEEK_API_KEY`, and it is `skipped`, never failed, without one.
428
+
429
+ The same report pair is available programmatically: `renderDriveResult` / `renderMatrix` (Markdown) and `renderDriveJUnitXml` / `renderMatrixJUnitXml` (JUnit) are exported from the package root over a settled `DriveResult` or `MatrixRecord`.
430
+
431
+ The README images are generated programmatically by [`scripts/gen-readme-assets.py`](scripts/gen-readme-assets.py) (Python + Pillow) and are safe to re-run: `python scripts/gen-readme-assets.py`.
432
+
433
+ Topics: `dsh`, `dsh-plugin`, `deepseek-harness`, `deepseek`, `cordis`, `plugin-testing`, `install-smoke`, `compatibility-matrix`, `ci`
434
+
435
+ ## Family & thanks
436
+
437
+ This project is part of the DeepSeek Harness plugin family from [PerryLink](https://github.com/PerryLink) (all Apache-2.0). If it is useful, **a star helps the next person find it** — nothing is gated behind it.
438
+
439
+ - [@PerryLink](https://github.com/PerryLink) — creator and maintainer: the single-target pipeline, the capability-assertion stage, temp-root ownership and the isolation cleanup ladder, the structured result contract, and the five-language docs.
440
+
441
+ <details>
442
+ <summary>Expand all plugins</summary>
443
+
444
+ | Plugin | Status |
445
+ |---|---|
446
+ | **[dsh-aqua-input-check](https://github.com/PerryLink/dsh-aqua-input-check)** | |
447
+ | **[dsh-archive-check](https://github.com/PerryLink/dsh-archive-check)** | |
448
+ | **[dsh-auto-review](https://github.com/PerryLink/dsh-auto-review)** | |
449
+ | **[dsh-autotier](https://github.com/PerryLink/dsh-autotier)** | 🧊 FROZEN — see the repo README |
450
+ | **[dsh-bid-ca-precheck](https://github.com/PerryLink/dsh-bid-ca-precheck)** | |
451
+ | **[dsh-bid-qual-check](https://github.com/PerryLink/dsh-bid-qual-check)** | |
452
+ | **[dsh-cert-mcp](https://github.com/PerryLink/dsh-cert-mcp)** | 🧊 FROZEN — see the repo README |
453
+ | **[dsh-checkpoint-rewind](https://github.com/PerryLink/dsh-checkpoint-rewind)** | |
454
+ | **[dsh-click](https://github.com/PerryLink/dsh-click)** | |
455
+ | **[dsh-composer-history](https://github.com/PerryLink/dsh-composer-history)** | |
456
+ | **[dsh-contract-stance](https://github.com/PerryLink/dsh-contract-stance)** | |
457
+ | **[dsh-customs-doc-check](https://github.com/PerryLink/dsh-customs-doc-check)** | |
458
+ | **[dsh-data-quality](https://github.com/PerryLink/dsh-data-quality)** | 🧊 FROZEN — see the repo README |
459
+ | **[dsh-demurrage-ledger](https://github.com/PerryLink/dsh-demurrage-ledger)** | |
460
+ | **[dsh-doublecheck](https://github.com/PerryLink/dsh-doublecheck)** | |
461
+ | **[dsh-drill-script-check](https://github.com/PerryLink/dsh-drill-script-check)** | |
462
+ | **[dsh-eia-guide-check](https://github.com/PerryLink/dsh-eia-guide-check)** | |
463
+ | **[dsh-emergency-plan](https://github.com/PerryLink/dsh-emergency-plan)** | |
464
+ | **[dsh-essay-rubric-check](https://github.com/PerryLink/dsh-essay-rubric-check)** | |
465
+ | **[dsh-evidence-check](https://github.com/PerryLink/dsh-evidence-check)** | |
466
+ | **[dsh-export-ctl-check](https://github.com/PerryLink/dsh-export-ctl-check)** | |
467
+ | **[dsh-fast](https://github.com/PerryLink/dsh-fast)** | |
468
+ | **[dsh-fmea-table-check](https://github.com/PerryLink/dsh-fmea-table-check)** | |
469
+ | **[dsh-forecast-penalty](https://github.com/PerryLink/dsh-forecast-penalty)** | |
470
+ | **[dsh-fund-research](https://github.com/PerryLink/dsh-fund-research)** | |
471
+ | **[dsh-github](https://github.com/PerryLink/dsh-github)** | |
472
+ | **[dsh-gongwen-flow-check](https://github.com/PerryLink/dsh-gongwen-flow-check)** | |
473
+ | **[dsh-gongwen-word-check](https://github.com/PerryLink/dsh-gongwen-word-check)** | |
474
+ | **[dsh-guard-plan-qc](https://github.com/PerryLink/dsh-guard-plan-qc)** | |
475
+ | **[dsh-hazchem-check](https://github.com/PerryLink/dsh-hazchem-check)** | |
476
+ | **[dsh-hazplan-check](https://github.com/PerryLink/dsh-hazplan-check)** | |
477
+ | **[dsh-hidden-risk-map](https://github.com/PerryLink/dsh-hidden-risk-map)** | |
478
+ | **[dsh-hs-classify](https://github.com/PerryLink/dsh-hs-classify)** | |
479
+ | **[dsh-icd-rule-check](https://github.com/PerryLink/dsh-icd-rule-check)** | |
480
+ | **[dsh-industry-research](https://github.com/PerryLink/dsh-industry-research)** | |
481
+ | **[dsh-lawcite-adapter](https://github.com/PerryLink/dsh-lawcite-adapter)** | |
482
+ | **[dsh-laya](https://github.com/PerryLink/dsh-laya)** | |
483
+ | **[dsh-lc-doc-check](https://github.com/PerryLink/dsh-lc-doc-check)** | |
484
+ | **[dsh-learning-gap-check](https://github.com/PerryLink/dsh-learning-gap-check)** | |
485
+ | **[dsh-library](https://github.com/PerryLink/dsh-library)** | |
486
+ | **[dsh-local-ai](https://github.com/PerryLink/dsh-local-ai)** | |
487
+ | **[dsh-lsp-actions](https://github.com/PerryLink/dsh-lsp-actions)** | |
488
+ | **[dsh-mask](https://github.com/PerryLink/dsh-mask)** | |
489
+ | **[dsh-mcp-panel](https://github.com/PerryLink/dsh-mcp-panel)** | |
490
+ | **[dsh-medrec-qc](https://github.com/PerryLink/dsh-medrec-qc)** | |
491
+ | **[dsh-memento](https://github.com/PerryLink/dsh-memento)** | 🧊 FROZEN — see the repo README |
492
+ | **[dsh-nsfc-form-check](https://github.com/PerryLink/dsh-nsfc-form-check)** | |
493
+ | **[dsh-nurse-record-check](https://github.com/PerryLink/dsh-nurse-record-check)** | |
494
+ | **[dsh-observe](https://github.com/PerryLink/dsh-observe)** | |
495
+ | **[dsh-origin-rvc-check](https://github.com/PerryLink/dsh-origin-rvc-check)** | |
496
+ | **[dsh-ota-review-check](https://github.com/PerryLink/dsh-ota-review-check)** | |
497
+ | **[dsh-output-styles](https://github.com/PerryLink/dsh-output-styles)** | |
498
+ | **[dsh-paper-doc-adapter](https://github.com/PerryLink/dsh-paper-doc-adapter)** | |
499
+ | **[dsh-permission-rules](https://github.com/PerryLink/dsh-permission-rules)** | |
500
+ | **[dsh-permit-report-check](https://github.com/PerryLink/dsh-permit-report-check)** | |
501
+ | **[dsh-pipeline-check](https://github.com/PerryLink/dsh-pipeline-check)** | |
502
+ | **[dsh-pleading-draft](https://github.com/PerryLink/dsh-pleading-draft)** | |
503
+ | **[dsh-plugin-doctor](https://github.com/PerryLink/dsh-plugin-doctor)** | 🚫 RETIRED (repo archived) |
504
+ | **[dsh-plugin-guide](https://github.com/PerryLink/dsh-plugin-guide)** | |
505
+ | **[dsh-plugin-kit](https://github.com/PerryLink/dsh-plugin-kit)** | 🚫 RETIRED (repo archived) |
506
+ | **[dsh-plugin-upgrade](https://github.com/PerryLink/dsh-plugin-upgrade)** | 🚫 RETIRED (repo archived) |
507
+ | **[dsh-policy-brief-draft](https://github.com/PerryLink/dsh-policy-brief-draft)** | |
508
+ | **[dsh-power-loss-split](https://github.com/PerryLink/dsh-power-loss-split)** | |
509
+ | **[dsh-power-ticket-check](https://github.com/PerryLink/dsh-power-ticket-check)** | |
510
+ | **[dsh-ppap-check](https://github.com/PerryLink/dsh-ppap-check)** | |
511
+ | **[dsh-protest-deadline](https://github.com/PerryLink/dsh-protest-deadline)** | |
512
+ | **[dsh-railway-window](https://github.com/PerryLink/dsh-railway-window)** | |
513
+ | **[dsh-repair-order-qc](https://github.com/PerryLink/dsh-repair-order-qc)** | |
514
+ | **[dsh-research-report](https://github.com/PerryLink/dsh-research-report)** | |
515
+ | **[dsh-review-reply-check](https://github.com/PerryLink/dsh-review-reply-check)** | |
516
+ | **[dsh-rulefile-check](https://github.com/PerryLink/dsh-rulefile-check)** | |
517
+ | **[dsh-safety-brief-check](https://github.com/PerryLink/dsh-safety-brief-check)** | |
518
+ | **[dsh-score](https://github.com/PerryLink/dsh-score)** | 🧊 FROZEN — see the repo README |
519
+ | **[dsh-session-sync](https://github.com/PerryLink/dsh-session-sync)** | |
520
+ | **[dsh-site-log-check](https://github.com/PerryLink/dsh-site-log-check)** | |
521
+ | **[dsh-skill-pack-security](https://github.com/PerryLink/dsh-skill-pack-security)** | |
522
+ | **[dsh-soe-decision-check](https://github.com/PerryLink/dsh-soe-decision-check)** | |
523
+ | **[dsh-soilwater-check](https://github.com/PerryLink/dsh-soilwater-check)** | |
524
+ | **[dsh-sop-sync-check](https://github.com/PerryLink/dsh-sop-sync-check)** | |
525
+ | **[dsh-spc-gbt-adapter](https://github.com/PerryLink/dsh-spc-gbt-adapter)** | |
526
+ | **[dsh-talk](https://github.com/PerryLink/dsh-talk)** | |
527
+ | **[dsh-tender-extract](https://github.com/PerryLink/dsh-tender-extract)** | |
528
+ | **[dsh-tender-matrix](https://github.com/PerryLink/dsh-tender-matrix)** | |
529
+ | **[dsh-test-drive](https://github.com/PerryLink/dsh-test-drive)** | |
530
+ | **[dsh-ticktick](https://github.com/PerryLink/dsh-ticktick)** | 🚫 RETIRED (repo archived) |
531
+ | **[dsh-translate](https://github.com/PerryLink/dsh-translate)** | |
532
+ | **[dsh-warranty-calc](https://github.com/PerryLink/dsh-warranty-calc)** | |
533
+ | **[dsh-budget](https://github.com/PerryLink/dsh-budget)** | 🧊 FROZEN — see the repo README |
534
+ | **[dsh-claude-move](https://github.com/PerryLink/dsh-claude-move)** | 🧊 FROZEN — see the repo README |
535
+ | **[dsh-defend](https://github.com/PerryLink/dsh-defend)** | 🧊 FROZEN — see the repo README |
536
+ | **[dsh-draw](https://github.com/PerryLink/dsh-draw)** | 🧊 FROZEN — see the repo README |
537
+ | **[dsh-reach](https://github.com/PerryLink/dsh-reach)** | 🧊 FROZEN — see the repo README |
538
+ | **[dsh-background-agents](https://github.com/PerryLink/dsh-background-agents)** | 🚫 RETIRED (repo archived) |
539
+ | **[dsh-session-pin](https://github.com/PerryLink/dsh-session-pin)** | 🚫 RETIRED (repo archived) |
540
+ | **[dsh-team-rooms](https://github.com/PerryLink/dsh-team-rooms)** | 🚫 RETIRED (repo archived) |
541
+
542
+ </details>
543
+
544
+ ## License
545
+
546
+ [LICENSE](LICENSE) (Apache License 2.0) © 2026 dsh-test-drive contributors
547
+
548
+ <p align="center"><sub>Apache-2.0 · Community plugin, not affiliated with DeepSeek.</sub></p>