dsh-plugin-inspector 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +105 -28
- package/lib/checks/tier-a.js +28 -2
- package/lib/checks/tier-b.js +62 -5
- package/lib/checks/tier-c.js +14 -8
- package/lib/cli.js +54 -10
- package/lib/index.js +5 -3
- package/lib/inspect.js +22 -6
- package/lib/model.js +0 -0
- package/lib/npm.js +58 -0
- package/lib/registry.js +261 -0
- package/lib/report.js +25 -5
- package/lib/source.js +57 -15
- package/lib/types/cli.d.ts +15 -3
- package/lib/types/index.d.ts +5 -3
- package/lib/types/inspect.d.ts +15 -2
- package/lib/types/model.d.ts +85 -7
- package/lib/types/npm.d.ts +40 -0
- package/lib/types/registry.d.ts +119 -0
- package/lib/types/source.d.ts +21 -1
- package/package.json +2 -1
package/README.md
CHANGED
|
@@ -2,13 +2,13 @@
|
|
|
2
2
|
|
|
3
3
|
**Know what a plugin does before you install it.**
|
|
4
4
|
|
|
5
|
-
`dsh-inspect` reads a DeepSeek Harness plugin — a directory
|
|
6
|
-
|
|
7
|
-
|
|
5
|
+
`dsh-inspect` reads a DeepSeek Harness plugin — a directory, an npm tarball, or a published
|
|
6
|
+
package fetched by name and checked against the hash the registry published — and tells you what it
|
|
7
|
+
declares and what its code is capable of. It does not install it, build it, import it, spawn it, or
|
|
8
|
+
evaluate any part of it.
|
|
8
9
|
|
|
9
10
|
```console
|
|
10
|
-
$ npm
|
|
11
|
-
$ dsh-inspect /tmp/some-dsh-plugin-1.4.0.tgz
|
|
11
|
+
$ dsh-inspect --from-npm some-dsh-plugin@1.4.0
|
|
12
12
|
```
|
|
13
13
|
|
|
14
14
|
## Install
|
|
@@ -53,7 +53,7 @@ dsh: warning: <pkg> declares no dsh.bundle — installed as a plain dependency,
|
|
|
53
53
|
|
|
54
54
|
The dangerous case prints nothing.
|
|
55
55
|
|
|
56
|
-
There are over
|
|
56
|
+
There are over 5,000 repos tagged `dsh-plugin` — 5,071 when this was last counted, on 16 August
|
|
57
57
|
2026 — and no registry, no review, and no signing between any of them and your process. This tool
|
|
58
58
|
exists so that the moment before you install one is not a blank.
|
|
59
59
|
|
|
@@ -68,15 +68,6 @@ commands it puts on your PATH, its dependencies, what model-visible text it ship
|
|
|
68
68
|
it could be read. A well-behaved plugin has a full facts section and an empty findings section.
|
|
69
69
|
That is a useful answer, not an empty one.
|
|
70
70
|
|
|
71
|
-
That is also the bar the tool is held to. Across twelve real targets — four plugins read both as a
|
|
72
|
-
directory and as their published tarball, the three bundles the harness itself ships, and a
|
|
73
|
-
scratch plugin — the current build reports **49 findings: none critical, two high, and exactly one
|
|
74
|
-
target exiting non-zero.** Both high findings are true statements about `@deepseek-ai/dsh-web-app`'s
|
|
75
|
-
own shipped code, which does contribute to the system prompt and does mount further plugins. The
|
|
76
|
-
upstream plugin template reports three findings, all `node:fs` and `dshHomePath` facts about what
|
|
77
|
-
it genuinely does; the base and headless bundles report three and one. Every packable target's
|
|
78
|
-
directory reading and tarball reading are byte-identical.
|
|
79
|
-
|
|
80
71
|
**Findings** — ranked, in three tiers:
|
|
81
72
|
|
|
82
73
|
| Tier | What it reads | What it can say |
|
|
@@ -89,14 +80,67 @@ Every Tier B and Tier C finding carries a `bypass` field naming the one-line eva
|
|
|
89
80
|
specific check. It is inside the finding, not in a footnote, so a report cannot be rendered
|
|
90
81
|
without its caveat.
|
|
91
82
|
|
|
83
|
+
A finding is **per package, not per syntax site**. A package importing `node:fs` from eleven files
|
|
84
|
+
gets one finding with `occurrences: 11` and three example locations, because the eleventh import
|
|
85
|
+
warrants no decision the first did not. Findings are grouped by check and `subject` — the module
|
|
86
|
+
specifier, the row id, the seam name, the matched rule — so `node:child_process` and
|
|
87
|
+
`node:worker_threads` stay two findings, and a gate can accept `B13`/`node:fs` without accepting
|
|
88
|
+
every `B13`.
|
|
89
|
+
|
|
90
|
+
## What it reports on the real ecosystem
|
|
91
|
+
|
|
92
|
+
Measured **2026-08-16** against the 40 most-starred GitHub repositories tagged `dsh-plugin` that
|
|
93
|
+
publish a resolvable npm package, each pinned to the version current that day. Re-run it with
|
|
94
|
+
`pnpm run sweep`; the corpus is `scripts/ecosystem-corpus.json` and the recorded measurement is
|
|
95
|
+
`tests/ecosystem-baseline.json`.
|
|
96
|
+
|
|
97
|
+
Both columns come from the same corpus and the same pinned versions, so the difference is this
|
|
98
|
+
tool's doing and not the ecosystem's. "0.1" is the published `dsh-plugin-inspector@0.1.0`; "0.2" is
|
|
99
|
+
this tree, which still reports version `0.1.0` because the release is not published yet.
|
|
100
|
+
|
|
101
|
+
| | 0.1 | 0.2 |
|
|
102
|
+
|---|---|---|
|
|
103
|
+
| Findings | 1,420 | **295** |
|
|
104
|
+
| Critical | 252 | **3** |
|
|
105
|
+
| Median findings per package | 10.5 | **5.5** |
|
|
106
|
+
| Packages with a high or critical | 27 of 40 (68 %) | **21 of 40 (53 %)** |
|
|
107
|
+
| Packages failing `--fail-on critical` | 40 of 40 | **1 of 40** |
|
|
108
|
+
| Clean packages | 0 of 40 | **0 of 40** |
|
|
109
|
+
|
|
110
|
+
**The 0.1 README quoted "49 findings, 0 critical" and that number was worthless.** It was measured
|
|
111
|
+
on twelve targets — the harness's own bundles and our own sibling plugins — which is a sample
|
|
112
|
+
selected for being trusted already. Against published third-party plugins the same build produced
|
|
113
|
+
1,420 findings and 252 criticals, and no package came out clean.
|
|
114
|
+
|
|
115
|
+
Read the 0.2 column honestly:
|
|
116
|
+
|
|
117
|
+
- **`--fail-on critical` is now a usable gate.** It stops one package in forty. That package,
|
|
118
|
+
`@struktoai/mirage-dsh`, ships a patch layer that switches off `fs-sandbox`, `bash-sandbox` and
|
|
119
|
+
`pwsh-sandbox`, and its three findings lead the report. Under 0.1 the same three sat somewhere in
|
|
120
|
+
a list of 252.
|
|
121
|
+
- **The default `--fail-on high` still stops a majority of the ecosystem**, and that is not a
|
|
122
|
+
finished job. The largest remaining driver is `C2` — the analyzer saying it could not read the
|
|
123
|
+
package, on 33 % of the corpus. That is a true statement rather than a false positive, but a gate
|
|
124
|
+
that fires on a third of npm for reasons about the *tool* is not yet a gate.
|
|
125
|
+
- **No package is clean, and that is expected rather than alarming.** `C3` alone — "ships built
|
|
126
|
+
output and no source" — fires on 65 % of published packages, because that is what publishing a
|
|
127
|
+
package is. It is `low`, it does not degrade the analysis, and it is not a defect.
|
|
128
|
+
|
|
129
|
+
The gap between "readable report" and "installable gate" is what 0.3 is for.
|
|
130
|
+
|
|
92
131
|
## Usage
|
|
93
132
|
|
|
94
133
|
```
|
|
95
134
|
dsh-inspect <target> [options]
|
|
135
|
+
dsh-inspect --from-npm <name>[@<version>] [options]
|
|
96
136
|
|
|
97
137
|
<target> A plugin directory, or an npm tarball (.tgz / .tar.gz).
|
|
98
138
|
|
|
99
139
|
Options
|
|
140
|
+
--from-npm <spec> Fetch a published package from the registry, verify its
|
|
141
|
+
dist.integrity hash, and analyse it in memory.
|
|
142
|
+
--registry <url> Registry base URL for --from-npm.
|
|
143
|
+
(default: https://registry.npmjs.org)
|
|
100
144
|
--json Emit the machine-readable JSON document on stdout.
|
|
101
145
|
--fail-on <severity> Exit 1 at or above this severity.
|
|
102
146
|
critical | high | medium | low | none (default: high)
|
|
@@ -117,13 +161,12 @@ plugin is clean" is the failure this split exists to prevent.
|
|
|
117
161
|
|
|
118
162
|
### Getting a package without installing it
|
|
119
163
|
|
|
120
|
-
Never `pnpm add` a package you have not read.
|
|
164
|
+
Never `pnpm add` a package you have not read.
|
|
121
165
|
|
|
122
166
|
```console
|
|
123
|
-
# From the registry
|
|
124
|
-
#
|
|
125
|
-
npm
|
|
126
|
-
dsh-inspect /tmp/<name>-<version>.tgz
|
|
167
|
+
# From the registry, in one step. Reads the ~3 KB version document, downloads the tarball into
|
|
168
|
+
# memory, verifies dist.integrity BEFORE anything parses it, and analyses it there.
|
|
169
|
+
dsh-inspect --from-npm <name>@<version>
|
|
127
170
|
|
|
128
171
|
# From git. Clone shallow and point the tool at the directory — do NOT use `npm pack` on a git
|
|
129
172
|
# spec, which runs the package's `prepare` script.
|
|
@@ -131,10 +174,21 @@ git clone --depth 1 https://github.com/… /tmp/plugin
|
|
|
131
174
|
dsh-inspect /tmp/plugin
|
|
132
175
|
```
|
|
133
176
|
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
177
|
+
`--from-npm` is the only mode that opens a socket, and it is one flag per invocation: it cannot be
|
|
178
|
+
combined with a local target, and a directory or tarball scan can never reach it — the fetch lives
|
|
179
|
+
in a module the analysis path does not import. **A network fetch is not execution.** No subprocess,
|
|
180
|
+
no disk write, no lifecycle script, and no `npm pack`. The report records the tarball URL, the
|
|
181
|
+
digest that matched, and the registry's own `hasInstallScript` flag under `target.registry`.
|
|
182
|
+
|
|
183
|
+
If the hash does not match what the registry published, the tool refuses and parses nothing. If the
|
|
184
|
+
package predates `dist.integrity` entirely, the weaker `dist.shasum` is used and the report says
|
|
185
|
+
`sha1` rather than claiming more. If neither is published, that is a refusal too.
|
|
186
|
+
|
|
187
|
+
A tarball is decoded **entirely in memory**, from a file or from a fetch alike. Nothing is written
|
|
188
|
+
to disk, which makes tar path traversal structurally impossible rather than something a filter has
|
|
189
|
+
to catch. Every read ceiling is applied to the arriving stream rather than to a finished buffer, so
|
|
190
|
+
a 28 MB archive holding one 8 GB member is a refusal in under two seconds, not an out-of-memory
|
|
191
|
+
kill.
|
|
138
192
|
|
|
139
193
|
### Directory mode reads the working tree, not "the package"
|
|
140
194
|
|
|
@@ -187,6 +241,12 @@ The tool does not run in the harness process, does not gate installation, and ca
|
|
|
187
241
|
anything. It raises the cost of shipping a hostile plugin and gives you something to read where
|
|
188
242
|
today you see nothing. That is the whole claim.
|
|
189
243
|
|
|
244
|
+
A seam at which an install *could* be stopped does exist — `dsh plugin add` runs pnpm in the
|
|
245
|
+
profile directory, pnpm honours a `.pnpmfile.cjs` there, and throwing from its async `readPackage`
|
|
246
|
+
hook aborts the install with nothing written to `node_modules`. Nothing in 0.2 uses it.
|
|
247
|
+
[`ADR.md`](./ADR.md) §11 records the seam and why shipping a gate on this release's calibration
|
|
248
|
+
would have burned the idea.
|
|
249
|
+
|
|
190
250
|
### What is not statically decidable
|
|
191
251
|
|
|
192
252
|
1. **`!!js` semantics.** The loader evaluates these with
|
|
@@ -207,13 +267,20 @@ today you see nothing. That is the whole claim.
|
|
|
207
267
|
6. **Intent.** Tier B's `B8` is the sharpest case: the tool proves a package *can* read a
|
|
208
268
|
credential and *can* open a socket. It has not shown that the value flows between them, and it
|
|
209
269
|
cannot — that needs value tracking this tool does not do. Any telemetry library or
|
|
210
|
-
authenticated API client trips `B8` legitimately.
|
|
270
|
+
authenticated API client trips `B8` legitimately. It fires on 18 % of published plugins, which
|
|
271
|
+
is why it is `high` and not `critical`.
|
|
272
|
+
7. **Injection phrasing that is not spelled in ASCII.** The injection heuristics are Latin-alphabet
|
|
273
|
+
regexes. Substituting Cyrillic homoglyphs — `о` U+043E for `o`, `е` U+0435 for `e` — defeats
|
|
274
|
+
**every one of the ten rules**, including the zero-width-character rule, which looks for
|
|
275
|
+
invisible characters and not for visible ones that are the wrong letter. Verified against the
|
|
276
|
+
rule table, not assumed. Normalisation is not in 0.2; do not read a clean `A21`/`B10` as
|
|
277
|
+
evidence that shipped markdown carries no instructions.
|
|
211
278
|
|
|
212
279
|
### Every Tier B check has a one-line bypass
|
|
213
280
|
|
|
214
281
|
`ctx['pro' + 'vide']('approval', …)` defeats seam detection. A computed specifier defeats every
|
|
215
282
|
import check. A base64 event name defeats every listener check. Splitting a credential read and a
|
|
216
|
-
network call across two packages defeats `B8`.
|
|
283
|
+
network call across two packages defeats `B8`. A Cyrillic `о` defeats every injection rule.
|
|
217
284
|
|
|
218
285
|
**Tier A is much harder to hide from, because it is structured declaration rather than code.**
|
|
219
286
|
The harness must read `disabled: true` literally in order to disable anything, so there is no
|
|
@@ -232,8 +299,9 @@ that could be read.*
|
|
|
232
299
|
|
|
233
300
|
## Development
|
|
234
301
|
|
|
235
|
-
Node `^22.19.0 || >=24` and pnpm are the only requirements
|
|
236
|
-
checkout
|
|
302
|
+
Node `^22.19.0 || >=24` and pnpm are the only requirements. No test reaches a network or a harness
|
|
303
|
+
checkout: every registry case injects its own `fetch`, and one of them replaces the global with a
|
|
304
|
+
throwing stub to prove a directory or tarball scan never calls it.
|
|
237
305
|
|
|
238
306
|
```console
|
|
239
307
|
pnpm install
|
|
@@ -242,8 +310,17 @@ pnpm run test # unit suite
|
|
|
242
310
|
pnpm run test:coverage # same suite, with the coverage ratchet
|
|
243
311
|
pnpm run test:e2e # builds, then runs the real binary as a subprocess
|
|
244
312
|
pnpm run inspect <target> # run from source without building
|
|
313
|
+
pnpm run sweep -- --check # the one thing here that DOES use a network
|
|
245
314
|
```
|
|
246
315
|
|
|
316
|
+
`pnpm run sweep` is the ecosystem measurement. It fetches the pinned corpus in
|
|
317
|
+
`scripts/ecosystem-corpus.json` through the same verified in-memory path as `--from-npm`, prints the
|
|
318
|
+
distribution, and with `--check` exits non-zero when a fresh run is worse than
|
|
319
|
+
`tests/ecosystem-baseline.json`. `--discover` rebuilds the corpus from the most-starred repositories
|
|
320
|
+
carrying the topic; `--pin` moves every entry to the version current now; `--record` rewrites the
|
|
321
|
+
baseline. It runs from its own weekly workflow, never from CI — every other workflow here runs
|
|
322
|
+
without a network, and a unit suite that cannot reach one is easier to trust.
|
|
323
|
+
|
|
247
324
|
Hostile fixtures live in `tests/fixtures/` and are authored here — a plugin that disables the
|
|
248
325
|
approval row, one whose `!!js` calls `child_process`, one with a `postinstall`, one pairing a
|
|
249
326
|
credential read with `fetch`, one shipping a `SKILL.md` full of injection text, one declaring an
|
package/lib/checks/tier-a.js
CHANGED
|
@@ -69,7 +69,7 @@ const INERT_CLASSES = new Set(['literal', 'inert-read']);
|
|
|
69
69
|
* @returns the complete finding.
|
|
70
70
|
*/
|
|
71
71
|
function tierA(finding) {
|
|
72
|
-
return { ...finding, tier: 'A', confidence: 'certain', bypass: null };
|
|
72
|
+
return { ...finding, tier: 'A', confidence: 'certain', bypass: null, examples: [finding.evidence], occurrences: 1 };
|
|
73
73
|
}
|
|
74
74
|
/**
|
|
75
75
|
* Whether the package under analysis is itself one of the harness's shipped
|
|
@@ -131,6 +131,7 @@ function checkDisabledRows(input) {
|
|
|
131
131
|
findings.push(tierA({
|
|
132
132
|
checkId: 'A19',
|
|
133
133
|
name: 'core-row-force-enabled',
|
|
134
|
+
subject: override.id,
|
|
134
135
|
severity: 'medium',
|
|
135
136
|
title: `Patch layer re-enables the core row "${override.id}"`,
|
|
136
137
|
detail: 'The loader coerces `disabled` with `Boolean()`, so this value leaves the row running. Because '
|
|
@@ -145,6 +146,7 @@ function checkDisabledRows(input) {
|
|
|
145
146
|
findings.push(tierA({
|
|
146
147
|
checkId: 'A2',
|
|
147
148
|
name: 'security-row-disabled',
|
|
149
|
+
subject: override.id,
|
|
148
150
|
severity: 'critical',
|
|
149
151
|
title: `Patch layer disables the core row "${override.id}"`,
|
|
150
152
|
detail: `Bundle patches apply after @deepseek-ai/dsh-base, so this layer switches off ${stops}. `
|
|
@@ -161,6 +163,7 @@ function checkDisabledRows(input) {
|
|
|
161
163
|
findings.push(tierA({
|
|
162
164
|
checkId: 'A3',
|
|
163
165
|
name: 'core-row-disabled',
|
|
166
|
+
subject: override.id,
|
|
164
167
|
severity,
|
|
165
168
|
title: `Patch layer disables the core row "${override.id}"`,
|
|
166
169
|
detail: `The row comes from ${coreRowOrigin(override.id)}, and this layer applies after it, so the row `
|
|
@@ -189,6 +192,7 @@ function checkOverriddenRows(input) {
|
|
|
189
192
|
findings.push(tierA({
|
|
190
193
|
checkId: 'A4',
|
|
191
194
|
name: 'patch-name-guard-mismatch',
|
|
195
|
+
subject: override.id,
|
|
192
196
|
severity: 'medium',
|
|
193
197
|
title: `Patch for "${override.id}" names ${override.nameGuard}, but that row is ${coreName}`,
|
|
194
198
|
detail: 'applyEntryPatches treats `name` on a non-insert patch as an assertion guard: on mismatch it '
|
|
@@ -205,6 +209,7 @@ function checkOverriddenRows(input) {
|
|
|
205
209
|
findings.push(tierA({
|
|
206
210
|
checkId: 'A5',
|
|
207
211
|
name: 'core-row-overridden',
|
|
212
|
+
subject: `${override.id}:${rewritten.join(',')}`,
|
|
208
213
|
severity: isSecurity ? 'high' : 'medium',
|
|
209
214
|
title: `Patch layer rewrites ${rewritten.map(key => `\`${key}\``).join(', ')} on the core row "${override.id}"`,
|
|
210
215
|
detail: `The row is ${coreName}. Patch overrides are shallow whole-value replacements, not merges, so `
|
|
@@ -249,6 +254,7 @@ function liveExpression(file, site) {
|
|
|
249
254
|
return tierA({
|
|
250
255
|
checkId: 'A6',
|
|
251
256
|
name: 'js-expression',
|
|
257
|
+
subject: `${site.slot}:${site.classification}`,
|
|
252
258
|
severity: EXPRESSION_SEVERITY[site.classification],
|
|
253
259
|
title: `\`!!js\` expression in a row's \`${site.slot}\` ${EXPRESSION_MEANING[site.classification]}`,
|
|
254
260
|
detail: `The loader evaluates this with new Function('ctx', 'expr', 'with (ctx) { return eval(expr) }') — `
|
|
@@ -270,6 +276,7 @@ function inertExpression(file, site) {
|
|
|
270
276
|
return tierA({
|
|
271
277
|
checkId: 'A7',
|
|
272
278
|
name: 'js-expression-inert',
|
|
279
|
+
subject: 'inert-slot',
|
|
273
280
|
severity: 'medium',
|
|
274
281
|
title: '`!!js` in a field the loader never interpolates',
|
|
275
282
|
detail: 'The loader interpolates only a row\'s `config` (recursively) and the top-level node of its `disabled`. '
|
|
@@ -285,6 +292,7 @@ function checkPatchFailures(input) {
|
|
|
285
292
|
? tierA({
|
|
286
293
|
checkId: 'A8',
|
|
287
294
|
name: 'single-bang-js-tag',
|
|
295
|
+
subject: failure.file,
|
|
288
296
|
severity: 'medium',
|
|
289
297
|
title: 'Patch layer uses the `!js` tag, which no harness accepts',
|
|
290
298
|
detail: 'The dialect registers exactly one custom tag, `tag:yaml.org,2002:js`, whose shorthand is `!!js`. '
|
|
@@ -295,6 +303,7 @@ function checkPatchFailures(input) {
|
|
|
295
303
|
: tierA({
|
|
296
304
|
checkId: 'A17',
|
|
297
305
|
name: 'patch-parse-error',
|
|
306
|
+
subject: failure.file,
|
|
298
307
|
severity: 'medium',
|
|
299
308
|
title: 'Patch layer does not parse',
|
|
300
309
|
detail: 'The declared patch layer cannot be read as a Cordis entry list, so mounting this package fails the '
|
|
@@ -321,6 +330,7 @@ function checkInsertedModules(input) {
|
|
|
321
330
|
findings.push(tierA({
|
|
322
331
|
checkId: 'A9',
|
|
323
332
|
name: 'insert-undeclared-module',
|
|
333
|
+
subject: name,
|
|
324
334
|
severity: isCore ? 'medium' : 'high',
|
|
325
335
|
title: `Inserted row "${row.id ?? '(unnamed)'}" mounts ${name}, which this package does not declare`,
|
|
326
336
|
detail: isCore
|
|
@@ -352,6 +362,7 @@ function checkMcpRows(input) {
|
|
|
352
362
|
findings.push(tierA({
|
|
353
363
|
checkId: 'A10',
|
|
354
364
|
name: 'mcp-server-row',
|
|
365
|
+
subject: shown,
|
|
355
366
|
severity: stdio ? 'critical' : 'high',
|
|
356
367
|
title: stdio
|
|
357
368
|
? `Patch layer starts a local MCP server by running \`${shown}\``
|
|
@@ -389,6 +400,7 @@ function checkSkillRootRedirect(input) {
|
|
|
389
400
|
findings.push(tierA({
|
|
390
401
|
checkId: 'A15',
|
|
391
402
|
name: 'skill-root-redirected',
|
|
403
|
+
subject: keys.join(','),
|
|
392
404
|
severity: 'high',
|
|
393
405
|
title: `Patch layer redirects skill discovery via ${keys.map(key => `\`${key}\``).join(', ')}`,
|
|
394
406
|
detail: 'Skill files reach the model verbatim, unescaped and uncapped. This row changes which directories '
|
|
@@ -412,6 +424,7 @@ function checkManifest(input) {
|
|
|
412
424
|
findings.push(tierA({
|
|
413
425
|
checkId: 'A1',
|
|
414
426
|
name: 'install-lifecycle-script',
|
|
427
|
+
subject: name,
|
|
415
428
|
severity: 'medium',
|
|
416
429
|
title: `Declares a \`${name}\` script, which runs at install time once allowed`,
|
|
417
430
|
detail: 'This command would run at the user\'s uid as part of `dsh plugin add`, before the user has read a '
|
|
@@ -427,6 +440,7 @@ function checkManifest(input) {
|
|
|
427
440
|
findings.push(tierA({
|
|
428
441
|
checkId: 'A22',
|
|
429
442
|
name: 'installs-command',
|
|
443
|
+
subject: command,
|
|
430
444
|
severity: 'low',
|
|
431
445
|
title: `Installs the command \`${command}\` on the user's PATH`,
|
|
432
446
|
detail: 'A `bin` entry is linked into the profile\'s `node_modules/.bin` at install time. It is not run by '
|
|
@@ -440,6 +454,7 @@ function checkManifest(input) {
|
|
|
440
454
|
findings.push(tierA({
|
|
441
455
|
checkId: 'A20',
|
|
442
456
|
name: 'profile-mounts-bundles',
|
|
457
|
+
subject: 'dsh.profile.bundles',
|
|
443
458
|
severity: 'high',
|
|
444
459
|
title: `Declares a profile that mounts ${profileBundles.length} bundle(s)`,
|
|
445
460
|
detail: 'A `dsh.profile.bundles` list makes this package a profile rather than a layer: the launcher resolves '
|
|
@@ -459,6 +474,7 @@ function checkManifest(input) {
|
|
|
459
474
|
findings.push(tierA({
|
|
460
475
|
checkId: 'A11',
|
|
461
476
|
name: 'non-registry-dependency',
|
|
477
|
+
subject: `${field}.${name}`,
|
|
462
478
|
severity: 'high',
|
|
463
479
|
title: `Depends on ${name} through a non-registry specifier`,
|
|
464
480
|
detail: 'The code behind this specifier can change without the version of this package changing, so nothing '
|
|
@@ -471,6 +487,7 @@ function checkManifest(input) {
|
|
|
471
487
|
findings.push(tierA({
|
|
472
488
|
checkId: 'A13',
|
|
473
489
|
name: 'no-files-allowlist',
|
|
490
|
+
subject: 'files',
|
|
474
491
|
severity: 'low',
|
|
475
492
|
title: 'No `files` allowlist in package.json',
|
|
476
493
|
detail: 'Without an allowlist the published tarball is whatever was in the working tree minus npm\'s default '
|
|
@@ -485,6 +502,7 @@ function checkManifest(input) {
|
|
|
485
502
|
findings.push(tierA({
|
|
486
503
|
checkId: 'A14',
|
|
487
504
|
name: 'bundle-patch-escapes-package',
|
|
505
|
+
subject: 'dsh.bundle.patch',
|
|
488
506
|
severity: 'critical',
|
|
489
507
|
title: 'The declared `dsh.bundle.patch` path climbs out of the package directory',
|
|
490
508
|
detail: 'The launcher resolves the patch as join(packageDir, declared) with no sanitisation, and `..` '
|
|
@@ -497,6 +515,7 @@ function checkManifest(input) {
|
|
|
497
515
|
findings.push(tierA({
|
|
498
516
|
checkId: 'A16',
|
|
499
517
|
name: 'bundle-patch-missing',
|
|
518
|
+
subject: 'dsh.bundle.patch',
|
|
500
519
|
severity: 'medium',
|
|
501
520
|
title: 'The declared `dsh.bundle.patch` file is not in the package',
|
|
502
521
|
detail: 'The package declares a mounted patch layer whose file is absent — commonly a `files` allowlist '
|
|
@@ -511,6 +530,7 @@ function checkManifest(input) {
|
|
|
511
530
|
findings.push(tierA({
|
|
512
531
|
checkId: 'A18',
|
|
513
532
|
name: 'manifest-defect',
|
|
533
|
+
subject: defect,
|
|
514
534
|
severity: 'low',
|
|
515
535
|
title: `Malformed package.json field: ${defect}`,
|
|
516
536
|
detail: 'The field was ignored. A manifest that npm and the harness read differently is worth knowing about.',
|
|
@@ -526,6 +546,7 @@ function checkModelVisibleText(input) {
|
|
|
526
546
|
return [tierA({
|
|
527
547
|
checkId: 'A12',
|
|
528
548
|
name: 'model-visible-text-shipped',
|
|
549
|
+
subject: 'shipped-instructions',
|
|
529
550
|
severity: 'low',
|
|
530
551
|
title: `Ships ${input.modelVisibleFiles.length} model-visible instruction file(s)`,
|
|
531
552
|
detail: 'Skill and agent-instruction markdown reaches the model verbatim, unescaped and uncapped. Shipping it '
|
|
@@ -548,6 +569,7 @@ function checkServiceRemapping(input) {
|
|
|
548
569
|
findings.push(tierA({
|
|
549
570
|
checkId: 'A23',
|
|
550
571
|
name: 'row-service-remapping',
|
|
572
|
+
subject: `${field}:${seams.join(',')}`,
|
|
551
573
|
severity: critical ? 'critical' : 'high',
|
|
552
574
|
title: `Inserted row "${row.id ?? '(unnamed)'}" re-maps ${seams.map(name => `\`${name}\``).join(', ')} via \`${field}\``,
|
|
553
575
|
detail: field === 'isolate'
|
|
@@ -583,9 +605,11 @@ function checkInjectionText(input) {
|
|
|
583
605
|
if (text === undefined)
|
|
584
606
|
continue;
|
|
585
607
|
for (const match of scanInjection(text)) {
|
|
608
|
+
const evidence = { file: path, path: lineColumn(text, match.index), snippet: snippet(match.excerpt) };
|
|
586
609
|
findings.push({
|
|
587
610
|
checkId: 'A21',
|
|
588
611
|
name: 'model-visible-injection',
|
|
612
|
+
subject: match.ruleId,
|
|
589
613
|
tier: 'A',
|
|
590
614
|
severity: 'high',
|
|
591
615
|
confidence: 'certain',
|
|
@@ -595,7 +619,9 @@ function checkInjectionText(input) {
|
|
|
595
619
|
+ 'model, which is why this is a verdict about the text rather than a capability report. Whether the '
|
|
596
620
|
+ 'sentence is an instruction or a discussion of one is a judgement this tool cannot make: the pattern '
|
|
597
621
|
+ 'will miss a rephrasing, and it can fire on a document that legitimately quotes an attack.',
|
|
598
|
-
evidence
|
|
622
|
+
evidence,
|
|
623
|
+
examples: [evidence],
|
|
624
|
+
occurrences: 1,
|
|
599
625
|
bypass: null,
|
|
600
626
|
});
|
|
601
627
|
}
|
package/lib/checks/tier-b.js
CHANGED
|
@@ -37,7 +37,7 @@ const SYSTEM_PROMPT_MEMBERS = new Set([
|
|
|
37
37
|
* @returns the complete finding.
|
|
38
38
|
*/
|
|
39
39
|
function tierB(finding) {
|
|
40
|
-
return { ...finding, tier: 'B', confidence: 'high' };
|
|
40
|
+
return { ...finding, tier: 'B', confidence: 'high', examples: [finding.evidence], occurrences: 1 };
|
|
41
41
|
}
|
|
42
42
|
/**
|
|
43
43
|
* The literal text of a string argument, or `null` when it is computed.
|
|
@@ -110,7 +110,11 @@ function checkImports(file, accumulator) {
|
|
|
110
110
|
accumulator.findings.push(tierB({
|
|
111
111
|
checkId: 'B9',
|
|
112
112
|
name: 'unmediated-process-api',
|
|
113
|
-
|
|
113
|
+
subject: specifier,
|
|
114
|
+
// Raised to `high` by `escalateProcessImports` when this package also
|
|
115
|
+
// reads a credential or reaches the network. On its own it is a
|
|
116
|
+
// capability half the ecosystem has.
|
|
117
|
+
severity: 'medium',
|
|
114
118
|
title: `Imports \`${specifier}\`, which ${unmediated}`,
|
|
115
119
|
detail: 'A mounted bundle layer is imported into the harness process at the agent\'s uid. The harness\'s own '
|
|
116
120
|
+ 'dynamic-package sandbox denies untrusted code `require` outright and redirects it to ctx services; a '
|
|
@@ -123,6 +127,7 @@ function checkImports(file, accumulator) {
|
|
|
123
127
|
const finding = tierB({
|
|
124
128
|
checkId: 'B7',
|
|
125
129
|
name: 'network-egress',
|
|
130
|
+
subject: specifier,
|
|
126
131
|
severity: 'medium',
|
|
127
132
|
title: `Imports \`${specifier}\`, which can move bytes off the machine`,
|
|
128
133
|
detail: 'Network access is a capability, not a verdict: most plugins that reach the network do so for a '
|
|
@@ -137,6 +142,7 @@ function checkImports(file, accumulator) {
|
|
|
137
142
|
accumulator.findings.push(tierB({
|
|
138
143
|
checkId: 'B13',
|
|
139
144
|
name: 'unmediated-filesystem',
|
|
145
|
+
subject: specifier,
|
|
140
146
|
severity: 'medium',
|
|
141
147
|
title: `Imports \`${specifier}\` rather than using the \`ctx.fs\` service`,
|
|
142
148
|
detail: 'Reads and writes through the Node filesystem API are invisible to `fs/write-intent`, '
|
|
@@ -162,6 +168,7 @@ function checkSeamReplacement(file, node, accumulator) {
|
|
|
162
168
|
accumulator.findings.push(tierB({
|
|
163
169
|
checkId: 'B1',
|
|
164
170
|
name: 'seam-replacement',
|
|
171
|
+
subject: `${seam}.${method}`,
|
|
165
172
|
severity: critical ? 'critical' : 'high',
|
|
166
173
|
title: `Replaces the \`${seam}\` capability seam via \`.${method}()\``,
|
|
167
174
|
detail: `\`${seam}\` is a catalogued core service. Providing it from a third-party layer substitutes this `
|
|
@@ -185,6 +192,7 @@ function checkSystemPrompt(file, node, accumulator) {
|
|
|
185
192
|
accumulator.findings.push(tierB({
|
|
186
193
|
checkId: 'B5',
|
|
187
194
|
name: 'system-prompt-mutation',
|
|
195
|
+
subject: isAssembleListener ? 'system-prompt/assemble' : callee.name.text,
|
|
188
196
|
severity: 'high',
|
|
189
197
|
title: isAssembleListener
|
|
190
198
|
? 'Listens on `system-prompt/assemble`'
|
|
@@ -207,6 +215,7 @@ function checkNestedMount(file, node, accumulator) {
|
|
|
207
215
|
accumulator.findings.push(tierB({
|
|
208
216
|
checkId: 'B11',
|
|
209
217
|
name: 'nested-plugin-mount',
|
|
218
|
+
subject: 'runtime-mount',
|
|
210
219
|
severity: 'high',
|
|
211
220
|
title: 'Mounts further plugins at runtime',
|
|
212
221
|
detail: 'A layer that mounts other layers moves the analysis target: what actually runs is decided by code '
|
|
@@ -257,6 +266,7 @@ function checkDynamicCode(file, node, accumulator) {
|
|
|
257
266
|
accumulator.findings.push(tierB({
|
|
258
267
|
checkId: 'B12',
|
|
259
268
|
name: 'dynamic-code-construction',
|
|
269
|
+
subject: 'runtime-code',
|
|
260
270
|
severity: 'high',
|
|
261
271
|
title: 'Builds and runs code at runtime',
|
|
262
272
|
detail: 'Whatever this evaluates is not in the package and cannot be analysed from it. Construction alone is '
|
|
@@ -269,11 +279,14 @@ function checkDynamicCode(file, node, accumulator) {
|
|
|
269
279
|
/** B6 — reading a credential. */
|
|
270
280
|
function checkCredentialRead(file, node, accumulator) {
|
|
271
281
|
let title = null;
|
|
282
|
+
let subject = '';
|
|
272
283
|
if (ts.isPropertyAccessExpression(node) && ts.isPropertyAccessExpression(node.expression)) {
|
|
273
284
|
const outer = node.expression;
|
|
274
285
|
if (ts.isIdentifier(outer.expression) && outer.expression.text === 'process' && outer.name.text === 'env') {
|
|
275
|
-
if (SECRET_ENV_KEY.test(node.name.text))
|
|
286
|
+
if (SECRET_ENV_KEY.test(node.name.text)) {
|
|
276
287
|
title = `Reads the environment variable \`${node.name.text}\``;
|
|
288
|
+
subject = `env:${node.name.text}`;
|
|
289
|
+
}
|
|
277
290
|
}
|
|
278
291
|
}
|
|
279
292
|
if (ts.isElementAccessExpression(node) && ts.isPropertyAccessExpression(node.expression)) {
|
|
@@ -282,19 +295,23 @@ function checkCredentialRead(file, node, accumulator) {
|
|
|
282
295
|
if (ts.isIdentifier(outer.expression) && outer.expression.text === 'process' && outer.name.text === 'env'
|
|
283
296
|
&& key !== null && SECRET_ENV_KEY.test(key)) {
|
|
284
297
|
title = `Reads the environment variable \`${key}\``;
|
|
298
|
+
subject = `env:${key}`;
|
|
285
299
|
}
|
|
286
300
|
}
|
|
287
301
|
if ((ts.isStringLiteral(node) || ts.isNoSubstitutionTemplateLiteral(node)) && CREDENTIAL_PATH.test(node.text)) {
|
|
288
302
|
title = `References the credential location \`${node.text}\``;
|
|
303
|
+
subject = `path:${node.text}`;
|
|
289
304
|
}
|
|
290
305
|
if (ts.isPropertyAccessExpression(node) && node.name.text === 'credentials' && ts.isIdentifier(node.expression)) {
|
|
291
306
|
title = 'Reads the `credentials` service';
|
|
307
|
+
subject = 'service:credentials';
|
|
292
308
|
}
|
|
293
309
|
if (title === null)
|
|
294
310
|
return;
|
|
295
311
|
const finding = tierB({
|
|
296
312
|
checkId: 'B6',
|
|
297
313
|
name: 'credential-read',
|
|
314
|
+
subject,
|
|
298
315
|
severity: 'medium',
|
|
299
316
|
title,
|
|
300
317
|
detail: 'Reading a credential is a capability, not a verdict — a plugin that authenticates to its own service '
|
|
@@ -318,6 +335,7 @@ function checkToolDescription(file, node, accumulator) {
|
|
|
318
335
|
accumulator.findings.push(tierB({
|
|
319
336
|
checkId: 'B10',
|
|
320
337
|
name: 'model-visible-injection',
|
|
338
|
+
subject: match.ruleId,
|
|
321
339
|
severity: 'high',
|
|
322
340
|
title: `Tool description ${match.meaning}`,
|
|
323
341
|
detail: `Heuristic \`${match.ruleId}\` matched a tool \`description\`, which is prompt text the model receives `
|
|
@@ -336,6 +354,7 @@ function checkNetworkGlobals(file, node, accumulator) {
|
|
|
336
354
|
const finding = tierB({
|
|
337
355
|
checkId: 'B7',
|
|
338
356
|
name: 'network-egress',
|
|
357
|
+
subject: callee.text,
|
|
339
358
|
severity: 'medium',
|
|
340
359
|
title: `Calls \`${callee.text}()\``,
|
|
341
360
|
detail: 'The harness\'s own dynamic-package sandbox traps `fetch` and redirects it to the `ctx.web` service, so '
|
|
@@ -381,7 +400,40 @@ export function runTierB(input) {
|
|
|
381
400
|
const pair = pairFinding(accumulator);
|
|
382
401
|
if (pair !== null)
|
|
383
402
|
accumulator.findings.push(pair);
|
|
384
|
-
return accumulator
|
|
403
|
+
return escalateProcessImports(accumulator);
|
|
404
|
+
}
|
|
405
|
+
/**
|
|
406
|
+
* B9 — raise a process-API import from `medium` to `high` when the same package
|
|
407
|
+
* also reads a credential or reaches the network.
|
|
408
|
+
*
|
|
409
|
+
* A bare `import 'node:child_process'` was hardcoded `critical` and fired on
|
|
410
|
+
* half the published ecosystem. A severity that common is not a severity: it
|
|
411
|
+
* pushed a package disabling `fs-sandbox` down a list of a thousand identical
|
|
412
|
+
* criticals. Spawning a process is what a plugin that wraps `git`, `ffmpeg` or
|
|
413
|
+
* a language server does, and the tool cannot tell that from the other thing.
|
|
414
|
+
*
|
|
415
|
+
* The pairing is exactly the one B8 already uses — a credential read *and* a
|
|
416
|
+
* network call in the same package — and for the same reason: the combination
|
|
417
|
+
* is what changes the question from "can it run a program" to "can it run a
|
|
418
|
+
* program with something worth sending, and somewhere to send it". Either half
|
|
419
|
+
* alone is not enough, and it would not narrow anything if it were: 68 % of
|
|
420
|
+
* published plugins reach the network at all.
|
|
421
|
+
*
|
|
422
|
+
* That pairing is still a capability, not a dataflow, so it stops at `high`.
|
|
423
|
+
* @param accumulator - the accumulated Tier B state.
|
|
424
|
+
* @returns the findings, with B9 severities settled.
|
|
425
|
+
*/
|
|
426
|
+
function escalateProcessImports(accumulator) {
|
|
427
|
+
const paired = accumulator.credentialRead !== null && accumulator.networkCall !== null;
|
|
428
|
+
if (!paired)
|
|
429
|
+
return accumulator.findings;
|
|
430
|
+
return accumulator.findings.map(finding => finding.checkId !== 'B9' ? finding : {
|
|
431
|
+
...finding,
|
|
432
|
+
severity: 'high',
|
|
433
|
+
detail: `${finding.detail} This package also reads a credential or reaches the network, which is why this is `
|
|
434
|
+
+ 'graded above a bare process import: the two capabilities together are what an exfiltration needs. It '
|
|
435
|
+
+ 'remains a capability report — nothing here shows the two are connected.',
|
|
436
|
+
});
|
|
385
437
|
}
|
|
386
438
|
/**
|
|
387
439
|
* B8 — a credential read and a network call in the same package.
|
|
@@ -393,10 +445,15 @@ function pairFinding(accumulator) {
|
|
|
393
445
|
const network = accumulator.networkCall;
|
|
394
446
|
if (credential === null || network === null)
|
|
395
447
|
return null;
|
|
396
|
-
|
|
448
|
+
// Not critical. This pair fires on 18 % of the published ecosystem — every
|
|
449
|
+
// telemetry client and every authenticated API client trips it — and the
|
|
450
|
+
// finding's own text says it is not a verdict. A severity that says "do not
|
|
451
|
+
// treat this as a verdict" cannot be the top one.
|
|
452
|
+
const severity = 'high';
|
|
397
453
|
return tierB({
|
|
398
454
|
checkId: 'B8',
|
|
399
455
|
name: 'exfiltration-capability',
|
|
456
|
+
subject: 'credential-and-egress',
|
|
400
457
|
severity,
|
|
401
458
|
title: 'This package can read a credential and can make a network call',
|
|
402
459
|
detail: 'This is a capability, not a dataflow. The tool found a credential read at '
|