dsh-vet 0.2.6 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +45 -11
- package/docs/adopt-marketplace.md +83 -0
- package/docs/dsh-vet-v1.md +40 -5
- package/docs/emitters.md +57 -0
- package/docs/outreach/README.md +15 -0
- package/docs/outreach/awesome-list-pr.md +39 -0
- package/docs/outreach/deepseek-harness-1115-reply.md +38 -0
- package/docs/outreach/dsh-plugin-audit-collab.md +37 -0
- package/docs/outreach/release-risk-1115-followup.md +37 -0
- package/docs/outreach/release-risk-pilot-ask.md +71 -0
- package/docs/outreach/show-your-plugins-post.md +41 -0
- package/docs/release-risk-pilot.md +109 -0
- package/docs/report-diff-v1.md +80 -0
- package/docs/scan-context-v1.md +218 -0
- package/docs/superpowers/plans/2026-09-10-release-risk-development-plan.md +397 -0
- package/lib/index.d.mts +344 -13
- package/lib/index.mjs +1417 -51
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -6,10 +6,12 @@ Security vetting for DeepSeek Harness (DSH) plugins: permission & supply-chain
|
|
|
6
6
|
audits before install, graded via the open [`dsh-vet/v1`](docs/dsh-vet-v1.md)
|
|
7
7
|
report standard.
|
|
8
8
|
|
|
9
|
-
> **Status: v0.
|
|
10
|
-
>
|
|
11
|
-
>
|
|
12
|
-
>
|
|
9
|
+
> **Status: v0.3 underway.** v0.2 shipped the author side — reference
|
|
10
|
+
> scanner ([npm](https://www.npmjs.com/package/dsh-vet), 16 calibrated rules
|
|
11
|
+
> with public rationales), CI Action, and auditable grade badges live in two
|
|
12
|
+
> repos. v0.3 is the ecosystem round: report validation for consumers, the
|
|
13
|
+
> verified-emitter program, and marketplace adoption before the contract's
|
|
14
|
+
> formal freeze ([roadmap](ROADMAP.md)).
|
|
13
15
|
|
|
14
16
|
## Install
|
|
15
17
|
|
|
@@ -31,12 +33,33 @@ npx dsh-vet <specifier> # npm package, git URL, or local path
|
|
|
31
33
|
npx dsh-vet --json <specifier> # dsh-vet/v1 report on stdout
|
|
32
34
|
npx dsh-vet --strict <specifier> # exit 1 on findings >= high (confidence >= medium)
|
|
33
35
|
npx dsh-vet --rules dep.install-scripts <specifier>
|
|
36
|
+
npx dsh-vet validate <report.json> # check a report against the contract
|
|
37
|
+
npx dsh-vet diff base.report.json head.report.json --json
|
|
34
38
|
```
|
|
35
39
|
|
|
36
40
|
Any completed report exits `0` — grades describe findings, they do not gate.
|
|
37
41
|
Scanner failures exit non-zero. The scanner runs locally, reads the npm
|
|
38
42
|
registry for dependency metadata only, and never transmits audited code.
|
|
39
43
|
|
|
44
|
+
Every report records what the scan actually covered (`x-dsh-vet` scan
|
|
45
|
+
context: rule profile, coverage, content digests, stable finding
|
|
46
|
+
identities); human output and badges show coverage alongside the grade,
|
|
47
|
+
and a missing context reads as `unknown`, never complete.
|
|
48
|
+
|
|
49
|
+
### Comparing two reports (release review)
|
|
50
|
+
|
|
51
|
+
`dsh-vet diff` compares two reports produced by the same scanner version
|
|
52
|
+
and rule profile over complete scans, and reports what changed:
|
|
53
|
+
[`dsh-vet/diff/v1`](docs/report-diff-v1.md) — added/removed/changed
|
|
54
|
+
findings by stable identity, behavior-observation deltas, both sides of
|
|
55
|
+
every transition. Exit `0` for a comparable result regardless of risk
|
|
56
|
+
changes, `1` when the pair cannot be trusted to describe the same
|
|
57
|
+
subject under the same checks (with explicit reasons), `2` on usage or
|
|
58
|
+
invalid reports. Two local-directory scans additionally need
|
|
59
|
+
`--subject <label>`. An incomparable pair is a prompt to rescan both
|
|
60
|
+
artifacts with the same configuration — never a claim that nothing
|
|
61
|
+
changed.
|
|
62
|
+
|
|
40
63
|
Shipped rules (each with a public rationale under
|
|
41
64
|
[`docs/rules/`](docs/rules)):
|
|
42
65
|
|
|
@@ -61,13 +84,19 @@ from your repo, so its value is auditable through git history and no badge
|
|
|
61
84
|
service is involved:
|
|
62
85
|
|
|
63
86
|
```yaml
|
|
64
|
-
- uses: rogerdigital/dsh-vet/action@v0.
|
|
87
|
+
- uses: rogerdigital/dsh-vet/action@v0.4.0
|
|
65
88
|
with:
|
|
66
89
|
specifier: '.'
|
|
67
90
|
commit-report: true
|
|
68
91
|
```
|
|
69
92
|
|
|
70
93
|
Every run uploads the full report as an artifact; PRs get a single
|
|
94
|
+
comment, edited in place. Set `baseline-report` to a report scanned from
|
|
95
|
+
the merge base and PRs additionally show what changed since it — see
|
|
96
|
+
[comparing releases](action/README.md#comparing-releases) for the
|
|
97
|
+
trusted-baseline recipe, and the
|
|
98
|
+
[pilot record](docs/release-risk-pilot.md) for how the comparison
|
|
99
|
+
behaved on real release pairs.
|
|
71
100
|
edited-in-place findings comment. Badge snippet and all inputs:
|
|
72
101
|
[`action/README.md`](action/README.md). The `dsh-vet badge <report.json>`
|
|
73
102
|
command renders the shields endpoint JSON if you wire CI yourself.
|
|
@@ -104,8 +133,12 @@ The differentiating piece is not another scanner — it is
|
|
|
104
133
|
deterministic JSON report contract (findings with severity **and confidence**,
|
|
105
134
|
derived A–F grades) that any scanner may emit and any marketplace, CI job, or
|
|
106
135
|
UI may consume, in the spirit of the community's `dsh-doctor/v1` contract.
|
|
107
|
-
The TypeScript reference types
|
|
108
|
-
|
|
136
|
+
The TypeScript reference types and the reference markdown renderer ship from
|
|
137
|
+
this package; `dsh-vet validate` checks any report against the contract —
|
|
138
|
+
including the derived grade, so a report from an emitter you don't know can't
|
|
139
|
+
forge one. Third-party emitters are welcome and
|
|
140
|
+
[listed once verified](docs/emitters.md); marketplaces can start from
|
|
141
|
+
[docs/adopt-marketplace.md](docs/adopt-marketplace.md).
|
|
109
142
|
|
|
110
143
|
## How it differs
|
|
111
144
|
|
|
@@ -127,12 +160,13 @@ rules get re-examined and the rule set gets corrected in public.
|
|
|
127
160
|
|
|
128
161
|
## Roadmap
|
|
129
162
|
|
|
130
|
-
- **v0.1** — contract
|
|
131
|
-
check families above
|
|
163
|
+
- **v0.1** — contract shipped (stable, additive-only since 0.1.0) + reference
|
|
164
|
+
CLI (`dsh-vet <pkg>`) with the four check families above
|
|
132
165
|
- **v0.2** — GitHub Action + badge so plugin authors self-audit and publish
|
|
133
166
|
their grade
|
|
134
|
-
- **v0.3** —
|
|
135
|
-
|
|
167
|
+
- **v0.3** — ecosystem round: `dsh-vet validate` + the verified-emitter
|
|
168
|
+
program, marketplace integrations rendering `dsh-vet/v1` reports, and the
|
|
169
|
+
contract's formal freeze after the feedback round
|
|
136
170
|
|
|
137
171
|
The detailed, trackable plan — task breakdowns, recorded decisions,
|
|
138
172
|
definitions of done, risks, and kill criteria — lives in
|
|
@@ -0,0 +1,83 @@
|
|
|
1
|
+
# Rendering `dsh-vet/v1` reports in your marketplace
|
|
2
|
+
|
|
3
|
+
A one-pager for marketplace and catalog maintainers. Short version: your
|
|
4
|
+
users currently judge DSH plugins by vibes at install time; a committed
|
|
5
|
+
`dsh-vet/v1` report gives you a grade, a findings table, and filters from
|
|
6
|
+
one JSON file — with zero servers to run and no endorsement implied.
|
|
7
|
+
|
|
8
|
+
## Why render reports
|
|
9
|
+
|
|
10
|
+
- **Demand already exists.** The community's most-upvoted feature request
|
|
11
|
+
([deepseek-harness#1115](https://github.com/deepseek-ai/deepseek-harness/discussions/1115))
|
|
12
|
+
asks for marketplace standards and review mechanisms. The official
|
|
13
|
+
marketplace will take time.
|
|
14
|
+
- **Authors publish reports already.** The
|
|
15
|
+
[GitHub Action](https://github.com/rogerdigital/dsh-vet/tree/main/action)
|
|
16
|
+
audits a plugin on every push and publishes the report + badge to a
|
|
17
|
+
`dsh-vet/report` branch, so for adopting repos the data is a raw-file URL
|
|
18
|
+
away — nothing for you to run.
|
|
19
|
+
- **Zero lock-in.** The contract is additive-only, consumers must ignore
|
|
20
|
+
unknown fields, and the grade ships inside the report — you render it, you
|
|
21
|
+
never recompute it. Dropping the integration later loses a column, not
|
|
22
|
+
your site.
|
|
23
|
+
- **Display is not endorsement.** Reports are signals, not verdicts; the
|
|
24
|
+
spec says so, and the badge says so. You surface what the audit found.
|
|
25
|
+
|
|
26
|
+
## Three integration paths
|
|
27
|
+
|
|
28
|
+
**1. The badge (minutes).** Render a shields.io badge from a report:
|
|
29
|
+
|
|
30
|
+
```sh
|
|
31
|
+
npx dsh-vet badge <report.json> # → shields endpoint JSON
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
**2. The findings table (an hour).** Import the reference renderer — the
|
|
35
|
+
same one the GitHub Action uses for PR comments:
|
|
36
|
+
|
|
37
|
+
```ts
|
|
38
|
+
import { renderMarkdown, validateReport } from 'dsh-vet'
|
|
39
|
+
|
|
40
|
+
const report = await fetch(reportUrl).then((r) => r.json())
|
|
41
|
+
const { ok } = validateReport(report) // never trust an unverified emitter
|
|
42
|
+
if (ok) page.add(renderMarkdown(report, { runUrl: reportUrl }))
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
**3. Ingestion-time validation (for pipelines).** Reject malformed or
|
|
46
|
+
forged reports before they reach your UI — the validator recomputes the
|
|
47
|
+
grade from the findings, so an emitter cannot assert a grade its evidence
|
|
48
|
+
does not support:
|
|
49
|
+
|
|
50
|
+
```sh
|
|
51
|
+
npx dsh-vet validate report.json && echo structurally-conformant
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
`validate` proves structural conformance only. It does not prove
|
|
55
|
+
completeness (that the emitter ran every check and omitted nothing),
|
|
56
|
+
artifact identity (that the report describes the plugin version you are
|
|
57
|
+
serving), or provenance (who produced the report). For those guarantees,
|
|
58
|
+
fetch reports through channels you control — the Action's report branch in
|
|
59
|
+
the author's own repo, or a scan you run yourself — and treat reports from
|
|
60
|
+
unverified emitters as unreviewed claims.
|
|
61
|
+
|
|
62
|
+
## Consumer rules (from the spec)
|
|
63
|
+
|
|
64
|
+
- **Ignore unknown fields** — emitters may add `x-`-prefixed extras; tolerate
|
|
65
|
+
them.
|
|
66
|
+
- **Read `summary.grade`; never recompute it.** Grades are derived at emit
|
|
67
|
+
time by contract.
|
|
68
|
+
- **Never present grade `X`** as a plugin's grade — it marks an incomplete
|
|
69
|
+
scan.
|
|
70
|
+
- **List findings, don't re-score them.** Severity and confidence are part
|
|
71
|
+
of the data; a low-confidence finding never lowers a grade, by contract.
|
|
72
|
+
|
|
73
|
+
## Where reports come from
|
|
74
|
+
|
|
75
|
+
Plugin authors commit them via the [Action](https://github.com/rogerdigital/dsh-vet/tree/main/action)
|
|
76
|
+
(`dsh-vet/report` branch in their repo), or you can run the scanner yourself
|
|
77
|
+
on any npm-installable plugin: `npx dsh-vet --json <specifier>`. Emitters
|
|
78
|
+
you didn't write must pass `dsh-vet validate` first; verified emitters are
|
|
79
|
+
listed in [emitters.md](emitters.md).
|
|
80
|
+
|
|
81
|
+
The contract itself: [dsh-vet-v1.md](dsh-vet-v1.md) — stable, additive-only
|
|
82
|
+
since 0.1.0, freezing after this adoption round. Feedback on it is exactly
|
|
83
|
+
what the freeze round is for: [discussions](https://github.com/rogerdigital/dsh-vet/discussions).
|
package/docs/dsh-vet-v1.md
CHANGED
|
@@ -1,7 +1,11 @@
|
|
|
1
1
|
# The `dsh-vet/v1` report contract
|
|
2
2
|
|
|
3
|
-
Status: **
|
|
3
|
+
Status: **stable — additive-only since v0.1.0.** Every report the 0.1.0
|
|
4
|
+
reference scanner emitted still validates today. The formal freeze is
|
|
5
|
+
announced with v0.3, after the marketplace-feedback round; until then,
|
|
6
|
+
changes are limited to new optional fields and new rule ids.
|
|
4
7
|
Reference TypeScript types: `src/contract.ts` (shipped from this package).
|
|
8
|
+
Conformance checking: `dsh-vet validate <report.json>`.
|
|
5
9
|
|
|
6
10
|
`dsh-vet/v1` defines a machine-readable audit report for a DeepSeek Harness
|
|
7
11
|
(DSH) plugin. It is implementation-agnostic: any scanner may emit it, and any
|
|
@@ -97,13 +101,43 @@ initially broke vendor-prefixed check ids — do not repeat that.
|
|
|
97
101
|
|
|
98
102
|
| Grade | Condition (over findings with confidence ≥ `medium`) |
|
|
99
103
|
|---|---|
|
|
100
|
-
| `A` |
|
|
104
|
+
| `A` | no graded findings — the report is empty, or every finding is `info`-severity or `low`-confidence |
|
|
101
105
|
| `B` | worst graded finding is `low` |
|
|
102
106
|
| `C` | worst graded finding is `medium` |
|
|
103
107
|
| `D` | worst graded finding is `high` |
|
|
104
108
|
| `F` | at least one `critical` |
|
|
105
109
|
| `X` | scan incomplete or errored — never presented as the plugin's grade |
|
|
106
110
|
|
|
111
|
+
## What validation proves — and what it does not
|
|
112
|
+
|
|
113
|
+
`dsh-vet validate` checks **structural conformance** with this contract:
|
|
114
|
+
field types and enums, rule-id shape, evidence presence, the deterministic
|
|
115
|
+
sort, and — the load-bearing check — that `summary` is exactly what the
|
|
116
|
+
`findings` derive. A conformant report cannot assert a grade its evidence
|
|
117
|
+
does not support.
|
|
118
|
+
|
|
119
|
+
It does not, and cannot, verify:
|
|
120
|
+
|
|
121
|
+
- **Emitter honesty** — whether the emitter omitted findings or fabricated
|
|
122
|
+
evidence. Structural consistency is not evidence that the audit was
|
|
123
|
+
thorough.
|
|
124
|
+
- **Completeness** — v1 records nothing about which files or rules a scan
|
|
125
|
+
covered. A subset scan's report is structurally indistinguishable from a
|
|
126
|
+
full one.
|
|
127
|
+
- **Artifact identity** — nothing ties a report to the artifact you are
|
|
128
|
+
holding. `target.resolved.integrity` is recorded by the emitter, not
|
|
129
|
+
checked against your copy.
|
|
130
|
+
- **Provenance** — who actually produced the report, and whether it changed
|
|
131
|
+
in transit.
|
|
132
|
+
|
|
133
|
+
Those guarantees come from delivery channels, not from the report's shape:
|
|
134
|
+
see [emitters.md](emitters.md) for how emitters are verified and
|
|
135
|
+
[adopt-marketplace.md](adopt-marketplace.md) for consumer guidance. The
|
|
136
|
+
optional scan-context extension ([scan-context-v1.md](scan-context-v1.md))
|
|
137
|
+
records coverage and content identity; the reference scanner begins
|
|
138
|
+
emitting it in a later release, and until a report carries it, absent
|
|
139
|
+
coverage data means **unknown**, never complete.
|
|
140
|
+
|
|
107
141
|
## CLI recommendations (non-normative)
|
|
108
142
|
|
|
109
143
|
Emitters that ship a CLI should exit `0` whenever a report was produced —
|
|
@@ -152,8 +186,9 @@ confidence ≥ `medium` exist, for CI gating.
|
|
|
152
186
|
|
|
153
187
|
## Versioning
|
|
154
188
|
|
|
155
|
-
`/v1`
|
|
156
|
-
|
|
157
|
-
|
|
189
|
+
`/v1` has been additive-only since the v0.1 release of this package — new
|
|
190
|
+
optional fields and new rule ids stay in `/v1`; semantic changes get `/v2`
|
|
191
|
+
with a migration note. The formal freeze follows the v0.3 marketplace-feedback
|
|
192
|
+
round: afterwards even additive changes land only after public discussion in
|
|
158
193
|
[GitHub Discussions](https://github.com/rogerdigital/dsh-vet/discussions)
|
|
159
194
|
and the [dsh-plugin topic](https://github.com/topics/dsh-plugin).
|
package/docs/emitters.md
ADDED
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
# Verified emitters of `dsh-vet/v1`
|
|
2
|
+
|
|
3
|
+
An **emitter** is any tool that produces `dsh-vet/v1` reports — a scanner
|
|
4
|
+
like this one, a CI integration, a marketplace's own analysis pipeline. The
|
|
5
|
+
contract is only useful if consumers can trust reports from emitters they
|
|
6
|
+
did not write, so this page defines what *verified* means and lists the
|
|
7
|
+
emitters that made it.
|
|
8
|
+
|
|
9
|
+
## The checklist
|
|
10
|
+
|
|
11
|
+
An emitter is listed as verified when it meets every item below. The list is
|
|
12
|
+
deliberately short: the contract carries the structure, this carries the
|
|
13
|
+
honesty.
|
|
14
|
+
|
|
15
|
+
1. **Structure.** Reports are built through `createReport()` from this
|
|
16
|
+
package, or — for non-TypeScript emitters — every published report passes
|
|
17
|
+
`dsh-vet validate` (`npx dsh-vet validate <report.json>`). Either path
|
|
18
|
+
guarantees the derived summary, the deterministic sort, and well-formed
|
|
19
|
+
rule ids; an emitter that hand-assembles reports and skips validation is
|
|
20
|
+
not verified. Validation proves structure, not honesty — it cannot
|
|
21
|
+
detect omitted findings, which is why the remaining items on this list
|
|
22
|
+
exist.
|
|
23
|
+
2. **Determinism.** Two runs over the same artifact with the same emitter
|
|
24
|
+
version produce identical reports, `scanner.ranAt` aside.
|
|
25
|
+
3. **Conservative severity.** Findings follow the severity ladder in the
|
|
26
|
+
[spec](dsh-vet-v1.md#severity-definitions); anything that depends on
|
|
27
|
+
runtime values is emitted at `low` confidence (or reduced severity), never
|
|
28
|
+
the reverse. The cost of a false positive is paid by a plugin author.
|
|
29
|
+
4. **Evidence.** Every finding carries at least one evidence item (`file`,
|
|
30
|
+
and `line` when the emitter has it); snippets are minimal and never
|
|
31
|
+
include secrets.
|
|
32
|
+
5. **Vendor-prefixed, documented rules.** Third-party rule ids start with
|
|
33
|
+
the vendor's own segment (`acme.eval-detect`) and each rule has a public
|
|
34
|
+
rationale page — a rule nobody can dispute in public is a rule nobody
|
|
35
|
+
should trust.
|
|
36
|
+
6. **A public dispute channel.** A place authors can contest findings, with
|
|
37
|
+
a visible record of corrections.
|
|
38
|
+
7. **Honest `scanner` fields.** `name` and `version` identify the emitting
|
|
39
|
+
tool as it actually ran.
|
|
40
|
+
|
|
41
|
+
## Verification process
|
|
42
|
+
|
|
43
|
+
1. The emitter's maintainer opens an issue here linking to the emitter and
|
|
44
|
+
2–3 sample reports against real plugins.
|
|
45
|
+
2. We run `dsh-vet validate` on the samples and read the vendor rule docs,
|
|
46
|
+
checking severity calibration against the spec ladder (item 3).
|
|
47
|
+
3. Both sides record the verified version range; listings link to the
|
|
48
|
+
emitter's repo and rule docs. Breaking the checklist later removes the
|
|
49
|
+
listing, with the reason stated in the issue.
|
|
50
|
+
|
|
51
|
+
## Registry
|
|
52
|
+
|
|
53
|
+
| Emitter | Verified versions | Rules | Notes |
|
|
54
|
+
|---|---|---|---|
|
|
55
|
+
| [`dsh-vet`](https://github.com/rogerdigital/dsh-vet) | 0.1.0 – | [`docs/rules/`](rules/) | the reference emitter; self-audited in [`examples/`](../examples/) |
|
|
56
|
+
|
|
57
|
+
Third-party emitters: none yet — the slot is open.
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
# Outreach drafts
|
|
2
|
+
|
|
3
|
+
Ready-to-send drafts. Each file names its destination. The v0.3
|
|
4
|
+
announcement round went live on 2026-09-02 (thread links in ROADMAP.md);
|
|
5
|
+
the release-risk round below is pending. Send from the maintainer's
|
|
6
|
+
account, then record the thread link so feedback has a traceable home.
|
|
7
|
+
|
|
8
|
+
| File | Destination | Depends on |
|
|
9
|
+
|---|---|---|
|
|
10
|
+
| [deepseek-harness-1115-reply.md](deepseek-harness-1115-reply.md) | reply in [deepseek-harness#1115](https://github.com/deepseek-ai/deepseek-harness/discussions/1115) | npm 0.3.0 published — **sent 2026-09-02** |
|
|
11
|
+
| [show-your-plugins-post.md](show-your-plugins-post.md) | community "Show Your Plugins" thread | 0.3.0 published — **sent 2026-09-02** |
|
|
12
|
+
| [awesome-list-pr.md](awesome-list-pr.md) | PR against a dsh/agent awesome-list | — **merged 2026-09-03** |
|
|
13
|
+
| [dsh-plugin-audit-collab.md](dsh-plugin-audit-collab.md) | issue/DM to dsh-plugin-audit maintainers | — **sent 2026-09-02** |
|
|
14
|
+
| [release-risk-1115-followup.md](release-risk-1115-followup.md) | follow-up in [deepseek-harness#1115](https://github.com/deepseek-ai/deepseek-harness/discussions/1115) | next npm release ships `dsh-vet diff` |
|
|
15
|
+
| [release-risk-pilot-ask.md](release-risk-pilot-ask.md) | issue/DM per target plugin author (template with pre-run results) | — |
|
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
<!-- Destination: PR against a dsh / agent-skills / security-tools awesome list.
|
|
2
|
+
Swap in the list's entry format (table vs bullet) and category. -->
|
|
3
|
+
|
|
4
|
+
PR title: `Add dsh-vet — pre-install security audits for DSH plugins`
|
|
5
|
+
|
|
6
|
+
## What
|
|
7
|
+
|
|
8
|
+
[dsh-vet](https://github.com/rogerdigital/dsh-vet) — static security vetting
|
|
9
|
+
for DeepSeek Harness plugins, built around an open report contract:
|
|
10
|
+
|
|
11
|
+
```markdown
|
|
12
|
+
- [dsh-vet](https://github.com/rogerdigital/dsh-vet) — pre-install permission & supply-chain audits
|
|
13
|
+
for DSH plugins; emits the open `dsh-vet/v1` report (severity + confidence, derived grades),
|
|
14
|
+
ships a CI Action + auditable badge, and validates/renders reports from any conforming emitter.
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
or, for table-format lists:
|
|
18
|
+
|
|
19
|
+
```markdown
|
|
20
|
+
| [dsh-vet](https://github.com/rogerdigital/dsh-vet) | CLI · library · CI Action | Pre-install static audits for DSH plugins; emits and verifies the open `dsh-vet/v1` report |
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
## Why it fits
|
|
24
|
+
|
|
25
|
+
- Solves a live pain: the community's top feature request is marketplace
|
|
26
|
+
review standards (deepseek-harness#1115); incidents like the Full Access
|
|
27
|
+
home-directory wipe (#461) show the stakes.
|
|
28
|
+
- Not another closed tool: the differentiating artifact is the
|
|
29
|
+
implementation-agnostic report contract — any scanner may emit it, any
|
|
30
|
+
marketplace/UI may consume it, `dsh-vet validate` verifies conformance.
|
|
31
|
+
- Maintained in the open: public per-rule rationales, public
|
|
32
|
+
false-positive dispute process, calibration record against 11 real
|
|
33
|
+
plugins, self-audited with its own scanner, MIT.
|
|
34
|
+
|
|
35
|
+
## Checks
|
|
36
|
+
|
|
37
|
+
- [ ] MIT licensed, docs and tests present, CI green
|
|
38
|
+
- [x] Installable via npm (`dsh-vet`), runs via `npx`, zero runtime deps
|
|
39
|
+
beyond the parser
|
|
@@ -0,0 +1,38 @@
|
|
|
1
|
+
<!-- Destination: reply in deepseek-ai/deepseek-harness discussion #1115
|
|
2
|
+
(marketplace standards / review mechanisms). Send after 0.3.0 is on npm. -->
|
|
3
|
+
|
|
4
|
+
Sharing what we built against exactly this problem, in case it's useful
|
|
5
|
+
before an official marketplace lands.
|
|
6
|
+
|
|
7
|
+
The core idea: "should I install this plugin?" needs a **shared,
|
|
8
|
+
machine-readable answer**, not per-tool vibes. So the contract came first —
|
|
9
|
+
[`dsh-vet/v1`](https://github.com/rogerdigital/dsh-vet/blob/main/docs/dsh-vet-v1.md),
|
|
10
|
+
an implementation-agnostic audit report (findings with severity *and*
|
|
11
|
+
confidence, derived A–F grades), following the pattern `dsh-doctor/v1`
|
|
12
|
+
proved: freeze a small boring contract, compete on implementations.
|
|
13
|
+
|
|
14
|
+
On top of it:
|
|
15
|
+
|
|
16
|
+
- **A reference scanner** ([dsh-vet on npm](https://www.npmjs.com/package/dsh-vet)) —
|
|
17
|
+
static, deterministic, never installs or transmits the audited plugin.
|
|
18
|
+
16 rules across capability seams, supply chain, obfuscation, and data
|
|
19
|
+
egress; every rule has a public rationale page and a public dispute
|
|
20
|
+
template. Calibrated against 11 real ecosystem plugins
|
|
21
|
+
([sweep record](https://github.com/rogerdigital/dsh-vet/blob/main/docs/calibration-v0.1.md)).
|
|
22
|
+
- **Author-side CI** — a GitHub Action that audits on every push and
|
|
23
|
+
publishes the report + grade badge from the plugin's own repo (zero
|
|
24
|
+
server; the badge value is auditable through git history). Live in two
|
|
25
|
+
repos already, including the scanner's own self-audit.
|
|
26
|
+
- **A consumer story** — `dsh-vet validate` checks any report against the
|
|
27
|
+
contract (grade is recomputed from findings, so an emitter can't forge
|
|
28
|
+
one), and `renderMarkdown` gives marketplaces/UIs the same rendering the
|
|
29
|
+
Action uses: [adopting it](https://github.com/rogerdigital/dsh-vet/blob/main/docs/adopt-marketplace.md).
|
|
30
|
+
|
|
31
|
+
The contract has been additive-only since 0.1.0 — every report the first
|
|
32
|
+
release emitted still validates today. **We're actively seeking feedback
|
|
33
|
+
from marketplace and tool maintainers before formally freezing `/v1`.** If
|
|
34
|
+
you're building a catalog or review layer, this round is exactly for you:
|
|
35
|
+
what would you need the report to carry?
|
|
36
|
+
|
|
37
|
+
Happy to go deeper on any piece — severity calibration, false-positive
|
|
38
|
+
handling, or the freeze criteria.
|
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
<!-- Destination: issue on dsh-plugin-audit's repo (or DM to its maintainers).
|
|
2
|
+
Tone: peer-to-peer, one concrete ask (a conversation), no commitment demanded. -->
|
|
3
|
+
|
|
4
|
+
Hi — maintainer of [dsh-vet](https://github.com/rogerdigital/dsh-vet) here.
|
|
5
|
+
Your runtime-sentinel work is in our README's comparison table as the
|
|
6
|
+
complementary half of this problem, and I'd like to make that explicit
|
|
7
|
+
instead of just documented on our side.
|
|
8
|
+
|
|
9
|
+
The shape of it:
|
|
10
|
+
|
|
11
|
+
- **dsh-vet** answers *"what does this plugin do?"* **before install** —
|
|
12
|
+
static analysis, emitted as the open
|
|
13
|
+
[`dsh-vet/v1`](https://github.com/rogerdigital/dsh-vet/blob/main/docs/dsh-vet-v1.md)
|
|
14
|
+
report (findings with severity + confidence, derived grades, evidence per
|
|
15
|
+
finding).
|
|
16
|
+
- **dsh-plugin-audit** answers *"what is it doing right now?"* at runtime —
|
|
17
|
+
permission profiling and sentinels.
|
|
18
|
+
|
|
19
|
+
Static-before-install and runtime-during-use cover different failure modes
|
|
20
|
+
(obfuscated payloads vs. behavior that only emerges live), which is why we
|
|
21
|
+
list you as complementary rather than competing — and why we'd rather
|
|
22
|
+
coordinate than FUD.
|
|
23
|
+
|
|
24
|
+
Two things that might be cheap and useful, if you're interested:
|
|
25
|
+
|
|
26
|
+
1. **Shared vocabulary.** Our report contract deliberately leaves rule ids
|
|
27
|
+
open-ended (`acme.eval-detect` style vendor prefixes). If your runtime
|
|
28
|
+
findings ever want a common shape — severity/confidence semantics,
|
|
29
|
+
evidence, dispute channels — the contract is additive-only and open for
|
|
30
|
+
feedback before its formal freeze. You'd be the most natural co-author
|
|
31
|
+
of whatever `/v1` learns from runtime auditing.
|
|
32
|
+
2. **Cross-linking.** We already point to you as the runtime half; a link
|
|
33
|
+
back from your side (if you find the pre-install half useful) would let
|
|
34
|
+
users find the full stack.
|
|
35
|
+
|
|
36
|
+
No ask beyond a conversation — if either half sounds useful, I'm happy to
|
|
37
|
+
open a discussion with concrete details.
|
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
<!-- Destination: follow-up comment in deepseek-ai/deepseek-harness discussion #1115
|
|
2
|
+
(marketplace standards / review mechanisms), under our 2026-09-02 reply.
|
|
3
|
+
Depends on: an npm release that ships `dsh-vet diff` (next version) —
|
|
4
|
+
send from the maintainer's account, then record the thread link in ROADMAP. -->
|
|
5
|
+
|
|
6
|
+
Following up with the piece the contract was missing: **what changed since
|
|
7
|
+
the last release**.
|
|
8
|
+
|
|
9
|
+
"Should I install this plugin?" got a machine-readable answer with
|
|
10
|
+
`dsh-vet/v1`. "Should I *upgrade*?" now has one too:
|
|
11
|
+
|
|
12
|
+
- Every report records **what the scan actually covered** — rule profile,
|
|
13
|
+
coverage, content digests, stable finding identities — as an optional
|
|
14
|
+
`x-dsh-vet` extension ([spec](https://github.com/rogerdigital/dsh-vet/blob/main/docs/scan-context-v1.md)).
|
|
15
|
+
A grade no longer silently means "whatever subset happened to run":
|
|
16
|
+
partial coverage says so, in the CLI, the PR comment, and the badge.
|
|
17
|
+
- `dsh-vet diff` compares two reports from the same scanner and profile
|
|
18
|
+
and reports added / removed / changed risk by **stable identity** —
|
|
19
|
+
line moves and reformatting are not risk additions, a second endpoint
|
|
20
|
+
is ([diff spec](https://github.com/rogerdigital/dsh-vet/blob/main/docs/report-diff-v1.md)).
|
|
21
|
+
Two reports that can't be honestly compared say so with explicit
|
|
22
|
+
reasons instead of a fake "no changes".
|
|
23
|
+
- The GitHub Action takes an optional `baseline-report` scanned from the
|
|
24
|
+
merge base, so PRs show exactly what risk-relevant behavior the PR
|
|
25
|
+
introduces — with a recipe that keeps the baseline out of the PR's
|
|
26
|
+
control ([action docs](https://github.com/rogerdigital/dsh-vet/blob/main/action/README.md#comparing-releases)).
|
|
27
|
+
|
|
28
|
+
We piloted it on real releases —
|
|
29
|
+
[left-pad and chalk, every reported addition reconciled against the
|
|
30
|
+
code](https://github.com/rogerdigital/dsh-vet/blob/main/docs/release-risk-pilot.md).
|
|
31
|
+
The chalk case is the interesting one: 5.x vendored its dependencies and
|
|
32
|
+
the diff flagged exactly the two vendored files unreachable through the
|
|
33
|
+
`#`-imports map — nothing else moved.
|
|
34
|
+
|
|
35
|
+
If you maintain a plugin and want the same read on your last two
|
|
36
|
+
releases, reply here or open an issue — we'll run the comparison and
|
|
37
|
+
post the result for you to check against what you actually changed.
|
|
@@ -0,0 +1,71 @@
|
|
|
1
|
+
<!-- Destination: issue (or DM) to the author of each target plugin, one
|
|
2
|
+
thread per plugin. The template below is filled with real pre-run
|
|
3
|
+
results — send as-is; the recipient installs nothing and answers from
|
|
4
|
+
their own knowledge of the release. Send from the maintainer's account,
|
|
5
|
+
then record each thread link in docs/release-risk-pilot.md. -->
|
|
6
|
+
|
|
7
|
+
# Release-risk pilot ask (template)
|
|
8
|
+
|
|
9
|
+
Subject: `Did your last release change what your plugin can do? (2-minute check)`
|
|
10
|
+
|
|
11
|
+
Hi — I maintain [`dsh-vet`](https://github.com/rogerdigital/dsh-vet), the
|
|
12
|
+
static audit tool for DSH plugins. It just learned to **compare two
|
|
13
|
+
releases of the same plugin and report exactly what risk-relevant
|
|
14
|
+
behavior changed** — new endpoints, new capabilities, new install
|
|
15
|
+
scripts, changed confidence — while ignoring line moves and reformatting.
|
|
16
|
+
|
|
17
|
+
I ran it on your last two releases so you don't have to install anything.
|
|
18
|
+
The results are below; three questions at the end.
|
|
19
|
+
|
|
20
|
+
---
|
|
21
|
+
|
|
22
|
+
## Filled example — dsh-doctor 0.4.2 → 0.4.3
|
|
23
|
+
|
|
24
|
+
Both scans: same scanner, full rule profile, complete coverage. Grades
|
|
25
|
+
stayed **D → D**. What moved:
|
|
26
|
+
|
|
27
|
+
- **New network client call** in `lib/client.js` — the destination is
|
|
28
|
+
computed at runtime, so the scanner can't see where it sends
|
|
29
|
+
(`perm.network-client`, subject `runtime-target`).
|
|
30
|
+
- **Subprocess spawns went from 1 to 2** in `lib/index.js`
|
|
31
|
+
(`child_process.spawn`).
|
|
32
|
+
|
|
33
|
+
Nothing else changed (13 identities unchanged).
|
|
34
|
+
|
|
35
|
+
## Filled example — dsh-searxng 0.2.1 → 0.3.0
|
|
36
|
+
|
|
37
|
+
Grades stayed **A → A**. One transition: **write/delete calls with
|
|
38
|
+
runtime-computed targets went from 1 to 17** in `lib/cli.mjs`
|
|
39
|
+
(`perm.undeclared-fs-write`, dynamic variant). These are low-severity /
|
|
40
|
+
low-confidence — they never affect the grade — but the count jump is the
|
|
41
|
+
kind of thing worth a look during a minor bump.
|
|
42
|
+
|
|
43
|
+
## Filled example — dsh-wechat 0.9.5 → 0.9.6
|
|
44
|
+
|
|
45
|
+
**Zero risk-relevant changes.** All 7 audited behaviors identical, grade
|
|
46
|
+
C → C. If that matches your intent for the release, the diff just saved
|
|
47
|
+
you the re-review.
|
|
48
|
+
|
|
49
|
+
---
|
|
50
|
+
|
|
51
|
+
## The three questions
|
|
52
|
+
|
|
53
|
+
1. Does the reported delta match what you intended to change in that
|
|
54
|
+
release? Anything you'd flag that the scanner missed or got wrong?
|
|
55
|
+
2. Any entries that felt like noise (didn't help you understand the
|
|
56
|
+
release)?
|
|
57
|
+
3. If this ran automatically on your PRs against the base revision —
|
|
58
|
+
[one Action input](https://github.com/rogerdigital/dsh-vet/blob/main/action/README.md#comparing-releases) —
|
|
59
|
+
would you read it before merging?
|
|
60
|
+
|
|
61
|
+
Answers in any form are useful; I'll record anonymized takeaways in the
|
|
62
|
+
[pilot record](https://github.com/rogerdigital/dsh-vet/blob/main/docs/release-risk-pilot.md).
|
|
63
|
+
This is the last feedback loop before deciding whether to build CI
|
|
64
|
+
gating on top (opt-in "fail on new risk"), so negative answers are as
|
|
65
|
+
valuable as positive ones.
|
|
66
|
+
|
|
67
|
+
<!-- Per-send checklist:
|
|
68
|
+
- re-run the pair with the latest main build the day of sending
|
|
69
|
+
- paste the actual result block for THIS author's package
|
|
70
|
+
- keep grades/identities verbatim from the diff, don't paraphrase
|
|
71
|
+
- one package per thread; don't batch multiple authors -->
|
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
<!-- Destination: a community "Show Your Plugins" / project showcase thread.
|
|
2
|
+
Adjust the opening line to the thread's framing. Send after 0.3.0 is on npm. -->
|
|
3
|
+
|
|
4
|
+
**dsh-vet** — know what a plugin does *before* you install it.
|
|
5
|
+
|
|
6
|
+
DSH's everything-is-a-plugin model is its superpower and its attack surface:
|
|
7
|
+
an installed plugin can touch your filesystem, spawn processes, and open
|
|
8
|
+
network connections. After the incident where a Full Access session wiped a
|
|
9
|
+
user's home directory, "should I install this?" deserved better than vibes.
|
|
10
|
+
|
|
11
|
+
dsh-vet audits any npm-installable DSH plugin — statically, locally, without
|
|
12
|
+
installing or executing it — and emits an open, deterministic
|
|
13
|
+
[`dsh-vet/v1`](https://github.com/rogerdigital/dsh-vet/blob/main/docs/dsh-vet-v1.md)
|
|
14
|
+
report: findings with severity **and** confidence, and a derived A–F grade
|
|
15
|
+
where a low-confidence finding never counts against you.
|
|
16
|
+
|
|
17
|
+
```sh
|
|
18
|
+
npx dsh-vet <plugin> # human summary
|
|
19
|
+
npx dsh-vet --json <plugin> # full dsh-vet/v1 report
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
What makes it defensible rather than noisy:
|
|
23
|
+
|
|
24
|
+
- **Every rule has a public rationale** and a public false-positive dispute
|
|
25
|
+
template; disputed rules get corrected in the open (it has happened, and
|
|
26
|
+
the changelog shows it).
|
|
27
|
+
- **Calibrated on real plugins** — an 11-package sweep record is published,
|
|
28
|
+
including grades and the two tunings the sweep forced.
|
|
29
|
+
- **It audits itself** — the repo's own grade badge is generated from the
|
|
30
|
+
exact tarball that ships to npm, and every signal the scanner finds in
|
|
31
|
+
itself is in the published report.
|
|
32
|
+
|
|
33
|
+
For plugin authors: a [GitHub Action](https://github.com/rogerdigital/dsh-vet/tree/main/action)
|
|
34
|
+
audits on every push and publishes your grade badge from your own repo —
|
|
35
|
+
no badge service, the value is auditable through your git history. For
|
|
36
|
+
catalog/UI builders: [`dsh-vet validate`](https://github.com/rogerdigital/dsh-vet/blob/main/docs/adopt-marketplace.md)
|
|
37
|
+
ingests and verifies reports from any conforming emitter.
|
|
38
|
+
|
|
39
|
+
The contract is additive-only since 0.1.0 and open for feedback before the
|
|
40
|
+
formal freeze. Repo: [rogerdigital/dsh-vet](https://github.com/rogerdigital/dsh-vet) —
|
|
41
|
+
criticism on severity calibration is genuinely welcome.
|