promptwheel 0.7.10 → 0.7.11
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +14 -2
- package/bin/promptwheel.d.ts +12 -0
- package/bin/promptwheel.mjs +84 -9
- package/package.json +1 -1
- package/packs/supply-chain/README.md +103 -0
- package/packs/supply-chain/lock.mjs +154 -0
- package/packs/supply-chain/pack.json +76 -0
- package/packs/supply-chain/scan.mjs +221 -0
package/README.md
CHANGED
|
@@ -54,7 +54,7 @@ npx promptwheel run --working # gate UNCOMMITTED changes (incl. newl
|
|
|
54
54
|
npx promptwheel run --repeat 5 --json # measure 5× to establish a noise band, emit JSON
|
|
55
55
|
|
|
56
56
|
npx promptwheel init # optional: write promptwheel.config.json (guarded tests + antihack tripwires, + lint if eslint is set up)
|
|
57
|
-
npx promptwheel init --list # presets: tests-pass · lint · bundle-size · security · llm-eval · antihack · no-ai-tells · security-antihack
|
|
57
|
+
npx promptwheel init --list # presets: tests-pass · lint · bundle-size · security · llm-eval · antihack · no-ai-tells · security-antihack · supply-chain
|
|
58
58
|
|
|
59
59
|
# the loop: run any agent/script, keep the change ONLY if a metric improved, else revert
|
|
60
60
|
npx promptwheel improve --attempt "claude -p 'reduce lint errors'"
|
|
@@ -99,6 +99,8 @@ Gaming detection is on by default in `run` and `improve`; `--no-detect-gaming` g
|
|
|
99
99
|
- **gamingThreshold**, the fraction of a win that must survive the source-only re-run to count as earned (default `0.5`). Config-level scalar, overridable per metric; inherited through `extends` like `repeat`.
|
|
100
100
|
- **evaluatorPaths**, globs for files that *define* your gate (a threshold file, a budget, a CI limit). Editing one in the same diff that improves a metric is `GAMED` by declaration, deterministically. Opt-in, because only you know which files are the goalposts.
|
|
101
101
|
|
|
102
|
+
- **allowable**, `true` lets a declared reason (`--allow "<reason>"`, a `Promptwheel-Allow:` commit trailer, `check({ allow })`) clear *this* metric's guarded regression for that run, recorded in `report.allowed` and on the metric as `allowed: true`. For counts where a rise is sometimes the job (the `supply-chain` preset: a new dependency), so the change passes with the reason on the record instead of the guard being loosened. Without a reason the guard fails as usual.
|
|
103
|
+
|
|
102
104
|
Dependency and environment keys (config-level; `init` writes stack defaults):
|
|
103
105
|
|
|
104
106
|
- **linkDirs**, dirs to symlink from your checkout into each measuring worktree, for deps that live outside git. Default `["node_modules"]`; `init` writes `[".venv"]` for Python and `[]` for Rust (`cargo test` writes to `target/`, so it is built fresh in the worktree rather than shared across refs). `"linkNodeModules": false` still works as shorthand for "link nothing".
|
|
@@ -133,6 +135,16 @@ Some rules are not a metric at all, they are a hard line: *never touch `src/auth
|
|
|
133
135
|
|
|
134
136
|
A mandate is deterministic (a path match on the diff, zero-LLM), rides at the top of the report, and **inherits through `extends`** so a parent scope's hard rule holds for every repo beneath it. It is folded into the gate fingerprint at both refs, so quietly deleting a mandate changes the fingerprint: the report records `gate.changed: true` and prints a "gate definition changed" line, a tracked, tamper-evident change. And deleting it does not clear it: the mandates in force are the union of the base ref's and the head's, so a change that removes a mandate and touches its path in the same diff still fails (reported with `removed: true`). Retire a mandate in its own change, with the protected path untouched. (Before 0.7.5 the base-ref fingerprint did not include mandates, so this was recorded backwards: an unchanged mandate read as a changed gate and a deleted one did not, and the deletion cleared the rule.) This is the line between a contract and a request: `AGENTS.md` can *ask* an agent not to touch auth; a mandate will not let the change past the gate.
|
|
135
137
|
|
|
138
|
+
### Supply chain (preset `supply-chain`)
|
|
139
|
+
|
|
140
|
+
The step every npm supply-chain attack needs inside a target repo is a supply-chain edit: a lifecycle script, a new dependency, a lockfile entry that resolves somewhere other than the registry, a workflow or publish config. Those are paths and counts, so the gate can hold them deterministically, whatever steered the agent into the edit.
|
|
141
|
+
|
|
142
|
+
```bash
|
|
143
|
+
npx promptwheel init --preset supply-chain
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
writes `promptwheel.lock.json` (the shape of the repo, from `promptwheel lock`) and seven guarded counts that must not rise (`lifecycle_scripts`, `dependency_entries`, `lock_nonregistry_sources`, `unpinned_execution`, `install_policy_gaps`, `lock_drift`, `pins_changed`, measured by a zero-dep pack copied into `.promptwheel/packs/supply-chain/`), mandates that freeze `.github/workflows/**`, `.npmrc` and the yarn rc files, `Dockerfile*` and `docker-compose*`, `scripts/release*` and `scripts/publish*`, `CODEOWNERS`, and the Claude Code hook files, and lists the contract itself (`promptwheel.config.json`, `sawbuck.config.json`) under `evaluatorPaths`, so a change that edits it alongside a win is gamed by declaration while a change that only edits the contract still passes. When the rise is the job, say so and it is recorded: `promptwheel run --allow "needed for the CSV export (papaparse)"`. It is not a dependency firewall and not an injection detector: it does nothing about a package already compromised on the registry, and it does not decide where an instruction came from. Details in [packs/supply-chain/README.md](packs/supply-chain/README.md).
|
|
147
|
+
|
|
136
148
|
## In CI and in Claude Code
|
|
137
149
|
|
|
138
150
|
**GitHub Action.** Drop this in your repo; it posts a verdict comment on every PR and fails the check on a guarded regression, a gamed win, or an inconclusive gate:
|
|
@@ -169,7 +181,7 @@ Then `/promptwheel:setup` (write a config) · `/promptwheel:gate` (gate uncommit
|
|
|
169
181
|
|
|
170
182
|
- **Tripwire guards** (`test_count`, `skipped_tests`, `suppressions`, `assertions`, written by the plain `init` default) catch the quieter shapes: an inline suppression (`@ts-nocheck`, `eslint-disable`, `# noqa`) or a suite gutted while the metric stays flat.
|
|
171
183
|
- **Gate integrity.** The gate is resolved as of the base commit too. A change that disables a guard or deletes a guarded metric in the same diff it passes is re-measured under the base definition and reads `GAMED` if that guard would have failed. The gate definition (metrics, guards, `evaluatorPaths`, `gamingThreshold`, `mandates`) is fingerprinted at both refs; a difference is recorded as `gate.changed` and printed.
|
|
172
|
-
- **A sanctioned override.** When you *do* mean to change a test, config, or grader, declare it: a `Promptwheel-Allow: <reason>` trailer on the commit (`Sawbuck-Allow:` is accepted as an alias), `--allow "<reason>"` on the CLI, or the `allow` option on `check()`, turns what would read `GAMED` into a `pass` with the reason recorded in `report.allowed`. It covers a win whose source-only re-run could not be applied (`unprovable`, otherwise `inconclusive`) the same way. An undeclared test-only win is still `gamed`, and a reasonless declaration does not override. The check never judges intent; it forces intent to be explicit and auditable.
|
|
184
|
+
- **A sanctioned override.** A commit trailer can be written by whoever writes the commit, including an agent, so `"allowTrailer": false` in the contract turns trailers off and only the `--allow` flag counts (the supply-chain preset sets this; the contract at the base ref decides, so a change cannot turn trailers back on for itself). When you *do* mean to change a test, config, or grader, declare it: a `Promptwheel-Allow: <reason>` trailer on the commit (`Sawbuck-Allow:` is accepted as an alias), `--allow "<reason>"` on the CLI, or the `allow` option on `check()`, turns what would read `GAMED` into a `pass` with the reason recorded in `report.allowed`. It covers a win whose source-only re-run could not be applied (`unprovable`, otherwise `inconclusive`) the same way. An undeclared test-only win is still `gamed`, and a reasonless declaration does not override. The check never judges intent; it forces intent to be explicit and auditable.
|
|
173
185
|
- **A provenance receipt** on every verdict (`report.provenance`: base and head SHAs, a config hash, a verdict hash), so a skeptic can re-run `(base, head, config)` and get the same hash.
|
|
174
186
|
|
|
175
187
|
**The honest limit.** This is one layer, not a silver bullet. It catches the evaluator-tampering class (test, grader, golden and config edits) deterministically and for free. It is blind, by construction, to a cheat that lives in the source file itself (a hardcoded answer, behavior that differs under test): that needs held-out tests, an LLM judge, or a human, and those expensive layers should be reserved for the calls only they can make. The coverage matrix and the in/out-of-scope boundary are in [docs/DETECTION-LAYERS.md](docs/DETECTION-LAYERS.md); the measured numbers are in [bench/RESULTS.md](bench/RESULTS.md) (`node bench/gaming-bench.mjs` to reproduce). Another gameable thing, an LLM judge asking "did you cheat?", is deliberately not part of the check.
|
package/bin/promptwheel.d.ts
CHANGED
|
@@ -40,6 +40,8 @@ export interface MetricConfig {
|
|
|
40
40
|
extract?: ExtractMode;
|
|
41
41
|
/** Default `'up'`. */
|
|
42
42
|
direction?: Direction;
|
|
43
|
+
/** `true`: the value is a fingerprint, so any change at all regresses, whichever way the number moves. */
|
|
44
|
+
exact?: boolean;
|
|
43
45
|
/** A guarded metric fails the gate on a trusted regression; an unguarded one is tracked only. */
|
|
44
46
|
guard?: boolean;
|
|
45
47
|
/** Read a value even when `cmd` exits nonzero (never needed for `extract: 'exit'`). */
|
|
@@ -50,6 +52,14 @@ export interface MetricConfig {
|
|
|
50
52
|
gamingCheck?: boolean;
|
|
51
53
|
/** Per-metric override of `Config.gamingThreshold`. */
|
|
52
54
|
gamingThreshold?: number;
|
|
55
|
+
/**
|
|
56
|
+
* `true` lets a declared reason (`--allow`, a `Promptwheel-Allow:` trailer, `check({ allow })`) clear
|
|
57
|
+
* THIS metric's guarded regression for that run, recorded in `report.allowed` and on the metric as
|
|
58
|
+
* `allowed: true`. The supply-chain preset's counters use it: a new dependency is sometimes the job,
|
|
59
|
+
* so it passes with the reason on the record instead of the guard being loosened. Without a reason
|
|
60
|
+
* the guard fails as usual.
|
|
61
|
+
*/
|
|
62
|
+
allowable?: boolean;
|
|
53
63
|
}
|
|
54
64
|
|
|
55
65
|
/** A named hard rule: any change touching one of `paths` fails the gate before any metric is read. */
|
|
@@ -149,6 +159,8 @@ export interface MetricResult extends Evaluation {
|
|
|
149
159
|
gamingReason?: string;
|
|
150
160
|
/** Set when a declared override cleared a `gamed` or an `unprovable` on this metric. */
|
|
151
161
|
allowed?: true;
|
|
162
|
+
/** Present (`true`) only when this guarded metric regressed, no reason was declared, and `allowable: true` means a `--allow` / trailer reason would clear it. */
|
|
163
|
+
allowable?: true;
|
|
152
164
|
}
|
|
153
165
|
|
|
154
166
|
export interface MandateViolation {
|
package/bin/promptwheel.mjs
CHANGED
|
@@ -83,13 +83,13 @@ function resolveConfig(p, repo, label, seen) {
|
|
|
83
83
|
const base = resolveConfig(bp, repo, relOf(repo, bp), seen);
|
|
84
84
|
for (const m of base.metrics) byName.set(m.name, m);
|
|
85
85
|
for (const md of (base.mandates || [])) if (md && md.name) mandatesByName.set(md.name, md);
|
|
86
|
-
for (const k of ['repeat', 'linkNodeModules', 'linkDirs', 'env', 'setup', 'record', 'gamingThreshold', 'evaluatorPaths']) if (base[k] !== undefined) scalars[k] = base[k];
|
|
86
|
+
for (const k of ['repeat', 'linkNodeModules', 'linkDirs', 'env', 'setup', 'record', 'gamingThreshold', 'evaluatorPaths', 'allowTrailer']) if (base[k] !== undefined) scalars[k] = base[k];
|
|
87
87
|
}
|
|
88
88
|
for (const m of (cfg.metrics || [])) {
|
|
89
89
|
byName.set(m.name, { ...m, __src: label, __override: byName.has(m.name) });
|
|
90
90
|
}
|
|
91
91
|
for (const md of (cfg.mandates || [])) if (md && md.name) mandatesByName.set(md.name, { ...md, __src: label });
|
|
92
|
-
for (const k of ['repeat', 'linkNodeModules', 'linkDirs', 'env', 'setup', 'record', 'gamingThreshold', 'evaluatorPaths']) if (cfg[k] !== undefined) scalars[k] = cfg[k];
|
|
92
|
+
for (const k of ['repeat', 'linkNodeModules', 'linkDirs', 'env', 'setup', 'record', 'gamingThreshold', 'evaluatorPaths', 'allowTrailer']) if (cfg[k] !== undefined) scalars[k] = cfg[k];
|
|
93
93
|
return { ...scalars, metrics: [...byName.values()], mandates: [...mandatesByName.values()] };
|
|
94
94
|
} finally {
|
|
95
95
|
seen.delete(abs); // only ACTIVE ancestors are a cycle, a diamond (a base reached two ways) is fine
|
|
@@ -228,6 +228,7 @@ function evaluate(m, beforeS, afterS, repeat) {
|
|
|
228
228
|
if (dir === 'up') { improved = delta > 0; regressed = delta < 0; }
|
|
229
229
|
else if (dir === 'down') { improved = delta < 0; regressed = delta > 0; }
|
|
230
230
|
else if (dir === 'pass') { improved = after === 1 && before !== 1; regressed = after < before; }
|
|
231
|
+
if (m.exact === true) { improved = false; regressed = delta !== 0; } // a fingerprint: any change at all needs a reason
|
|
231
232
|
|
|
232
233
|
let status;
|
|
233
234
|
if (withinNoise && delta !== 0) { status = 'inconclusive'; improved = regressed = false; } // within measured noise → don't call it
|
|
@@ -333,7 +334,7 @@ function checkMandates(repo, base, head, cfg, baseMandates) {
|
|
|
333
334
|
if (!pats.length) continue;
|
|
334
335
|
const res = pats.map(globToRe);
|
|
335
336
|
const hit = changed.filter((f) => res.some((re) => re.test(f)));
|
|
336
|
-
if (hit.length) out.push({ name: String(m.name || 'mandate'), reason: m.reason ? String(m.reason).slice(0, 300) : '', files: hit.slice(0, 20), ...(m.__removed ? { removed: true } : {}), ...(m.__edited ? { edited: true } : {}) });
|
|
337
|
+
if (hit.length) out.push({ name: String(m.name || 'mandate'), reason: m.reason ? String(m.reason).slice(0, 300) : '', files: hit.slice(0, 20), ...(m.allowable ? { allowable: m.allowable } : {}), ...(m.__removed ? { removed: true } : {}), ...(m.__edited ? { edited: true } : {}) });
|
|
337
338
|
}
|
|
338
339
|
// one entry per name: the head version and the edited base version of a rule merge (files unioned)
|
|
339
340
|
const merged = [];
|
|
@@ -501,7 +502,7 @@ function warnEditableInstall(repo) {
|
|
|
501
502
|
// guarded metric in the very diff it "passes". We resolve the gate as of BASE too and, if a
|
|
502
503
|
// guard was dropped, re-measure it under its BASE definition; the pass only reads GAMED when
|
|
503
504
|
// that guard WOULD HAVE FAILED at head (a real regression hidden by moving the goalposts).
|
|
504
|
-
const GATE_SCALARS = ['gamingThreshold', 'evaluatorPaths'];
|
|
505
|
+
const GATE_SCALARS = ['gamingThreshold', 'evaluatorPaths', 'allowTrailer'];
|
|
505
506
|
|
|
506
507
|
// Resolve the gate definition as of `ref` (metrics + mandates + scalars), following `extends` by
|
|
507
508
|
// reading every file from that ref via `git show`, mirroring resolveConfig (a parent's mandates
|
|
@@ -731,7 +732,12 @@ function gate(repo, opts) {
|
|
|
731
732
|
// It covers an `unprovable` win too (the source-only re-run could not be applied): the declared
|
|
732
733
|
// intent is exactly the uncertainty that verdict expresses, and without this a declared change
|
|
733
734
|
// whose source slice will not re-apply had no override short of turning the check off.
|
|
734
|
-
|
|
735
|
+
// A trailer is off when the committed contract says `allowTrailer: false` or the caller set
|
|
736
|
+
// PROMPTWHEEL_NO_ALLOW_TRAILER (Sawbuck does, for a commit an agent made): an agent can write a
|
|
737
|
+
// commit message, so where the threat is the agent itself only the --allow flag counts. The base
|
|
738
|
+
// ref's contract is read too, so an agent cannot turn trailers back on in the change it is making.
|
|
739
|
+
const trailersOff = () => !!process.env.PROMPTWHEEL_NO_ALLOW_TRAILER || cfg.allowTrailer === false || ((gateDefAtRef(repo, base) || {}).scalars || {}).allowTrailer === false;
|
|
740
|
+
const allow = opts.allow ? { reason: String(opts.allow).slice(0, 300), by: 'flag' } : (opts.working || trailersOff() ? null : readAllowTrailer(repo, head));
|
|
735
741
|
let allowed = null;
|
|
736
742
|
if (opts.detectGaming && allow) {
|
|
737
743
|
const gm = metrics.filter((m) => m.gamed === true);
|
|
@@ -745,11 +751,37 @@ function gate(repo, opts) {
|
|
|
745
751
|
if (gw) gateInfo.allowed = true;
|
|
746
752
|
}
|
|
747
753
|
}
|
|
754
|
+
// `allowable: true` on a guarded metric (the supply-chain counters: a new dependency or lifecycle
|
|
755
|
+
// script is sometimes the job) lets the same declared reason clear THAT metric's regression, so the
|
|
756
|
+
// change passes with the reason on the record instead of the guard being loosened to let it through.
|
|
757
|
+
// Never implicit: without a reason an allowable guard fails like any other, and the reason is recorded.
|
|
758
|
+
{
|
|
759
|
+
// `allowable: 'flag'` takes the reason from the --allow flag only, never from a commit trailer: an
|
|
760
|
+
// agent can write a trailer, so a guard meant to stop a compromised agent cannot be cleared by one.
|
|
761
|
+
const mode = new Map((cfg.metrics || []).filter((c) => c && c.allowable && c.name).map((c) => [c.name, c.allowable]));
|
|
762
|
+
const ga = metrics.filter((m) => m.guard && !m.ok && mode.has(m.name) && (mode.get(m.name) === true || !allow || allow.by === 'flag'));
|
|
763
|
+
const held = metrics.filter((m) => m.guard && !m.ok && mode.has(m.name) && !ga.includes(m));
|
|
764
|
+
for (const m of held) m.allowable = true;
|
|
765
|
+
if (ga.length && allow) {
|
|
766
|
+
allowed = allowed || { reason: allow.reason, cleared: [] };
|
|
767
|
+
for (const m of ga) { m.ok = true; m.allowed = true; allowed.cleared.push(m.name); }
|
|
768
|
+
} else for (const m of ga) m.allowable = true; // failed, and a declared reason would clear it: say so in the report
|
|
769
|
+
}
|
|
748
770
|
// mandates run first: a protected-path hard rule fails the gate no matter what the metrics say.
|
|
749
771
|
// The base ref's mandates count too (resolved through `extends` by gateDefAtRef), so a change
|
|
750
772
|
// cannot clear a mandate by deleting it in the same diff that violates it.
|
|
751
773
|
const baseMandates = (gateInfo && gateInfo.baseMandates) || (!opts.detectGaming ? ((gateDefAtRef(repo, base) || {}).scalars || {}).mandates : null);
|
|
752
|
-
|
|
774
|
+
let mandateViolations = checkMandates(repo, base, head, cfg, baseMandates);
|
|
775
|
+
// an allowable mandate (the lock file) changes only with a reason from the --allow flag, recorded
|
|
776
|
+
// like any other override; a commit trailer never clears one.
|
|
777
|
+
if (allow && allow.by === 'flag') {
|
|
778
|
+
const clear = mandateViolations.filter((v) => v.allowable);
|
|
779
|
+
if (clear.length) {
|
|
780
|
+
allowed = allowed || { reason: allow.reason, cleared: [] };
|
|
781
|
+
for (const v of clear) allowed.cleared.push(`mandate:${v.name}`);
|
|
782
|
+
mandateViolations = mandateViolations.filter((v) => !v.allowable);
|
|
783
|
+
}
|
|
784
|
+
}
|
|
753
785
|
const mandated = mandateViolations.length > 0;
|
|
754
786
|
const verdict = deriveVerdict(metrics, gateInfo, mandated);
|
|
755
787
|
const report = {
|
|
@@ -900,6 +932,32 @@ const PRESETS = {
|
|
|
900
932
|
metrics: [
|
|
901
933
|
{ name: 'em_dashes', cmd: 'grep -rIoe "—" --exclude-dir=node_modules --exclude-dir=.git --exclude-dir=dist --exclude-dir=build --exclude-dir=coverage --exclude-dir=.next --exclude-dir=out --exclude-dir=lib-cov --exclude-dir=__pycache__ --exclude-dir=.pytest_cache --exclude-dir=target --exclude-dir=.venv --exclude-dir=venv --exclude=promptwheel.config.json . 2>/dev/null | wc -l | tr -d " "', extract: 'number', direction: 'down', guard: true, gamingCheck: false },
|
|
902
934
|
] },
|
|
935
|
+
// supply-chain: the step every npm supply-chain attack needs inside a target repo is a supply-chain
|
|
936
|
+
// edit (a lifecycle script, a new dependency, a non-registry lock source, a workflow or publish
|
|
937
|
+
// config). Paths and counts, checked deterministically: three guarded counts that must not rise
|
|
938
|
+
// without a declared reason (`allowable`), plus mandates that freeze the paths. The contract itself is
|
|
939
|
+
// an evaluatorPath, not a mandate, so a person can still retire a rule in a change that only edits it.
|
|
940
|
+
'supply-chain': { allowTrailer: false, desc: 'gate: no new lifecycle scripts, dependencies, or non-registry lock sources without a recorded reason; the agent cannot touch workflows, registries, publish scripts, or the contract',
|
|
941
|
+
evaluatorPaths: ['promptwheel.config.json', 'sawbuck.config.json'],
|
|
942
|
+
mandates: [
|
|
943
|
+
{ name: 'workflows-frozen', paths: ['.github/workflows/**'], reason: 'workflows run with the repository secrets; change them in a separate, reviewed change' },
|
|
944
|
+
{ name: 'registry-config-frozen', paths: ['.npmrc', '.yarnrc', '.yarnrc.yml'], reason: 'registry and token configuration decides where packages come from and where credentials go' },
|
|
945
|
+
{ name: 'build-images-frozen', paths: ['Dockerfile*', 'docker-compose*'], reason: 'what runs at build and deploy time is not an agent decision' },
|
|
946
|
+
{ name: 'publish-scripts-frozen', paths: ['scripts/release*', 'scripts/publish*'], reason: 'the publish path is how a compromise spreads' },
|
|
947
|
+
{ name: 'codeowners-frozen', paths: ['CODEOWNERS'], reason: 'review ownership is set by people' },
|
|
948
|
+
{ name: 'scanner-frozen', paths: ['.promptwheel/packs/**'], reason: 'the scanners that read the contract are not edited by the change they read' },
|
|
949
|
+
{ name: 'lock-frozen', paths: ['promptwheel.lock.json'], allowable: 'flag', reason: 'the architecture lock changes with a reason from a person (--allow), never as a side effect of a change' },
|
|
950
|
+
{ name: 'hooks-frozen', paths: ['.claude/settings.json', '.claude/settings.local.json', '.sawbuck/**'], reason: 'the hooks that enforce the contract are wired by a person, not detached by the agent' },
|
|
951
|
+
],
|
|
952
|
+
metrics: [
|
|
953
|
+
{ name: 'lifecycle_scripts', cmd: 'node packs/supply-chain/scan.mjs lifecycle .', extract: 'number', direction: 'down', guard: true, gamingCheck: false, allowable: 'flag' },
|
|
954
|
+
{ name: 'dependency_entries', cmd: 'node packs/supply-chain/scan.mjs deps .', extract: 'number', direction: 'down', guard: true, gamingCheck: false, allowable: 'flag' },
|
|
955
|
+
{ name: 'lock_nonregistry_sources', cmd: 'node packs/supply-chain/scan.mjs lock-sources .', extract: 'number', direction: 'down', guard: true, gamingCheck: false, allowable: 'flag' },
|
|
956
|
+
{ name: 'unpinned_execution', cmd: 'node packs/supply-chain/scan.mjs unpinned .', extract: 'number', direction: 'down', guard: true, gamingCheck: false, allowable: 'flag' },
|
|
957
|
+
{ name: 'install_policy_gaps', cmd: 'node packs/supply-chain/scan.mjs policy .', extract: 'number', direction: 'down', guard: true, gamingCheck: false, allowable: 'flag' },
|
|
958
|
+
{ name: 'lock_drift', cmd: 'node packs/supply-chain/lock.mjs drift .', extract: 'number', direction: 'down', guard: true, gamingCheck: false, allowable: 'flag' },
|
|
959
|
+
{ name: 'pins_changed', cmd: 'node packs/supply-chain/lock.mjs pins .', extract: 'number', direction: 'down', exact: true, guard: true, gamingCheck: false, allowable: 'flag' },
|
|
960
|
+
] },
|
|
903
961
|
};
|
|
904
962
|
|
|
905
963
|
// security-antihack = the security-invariant findings gate + the SAME antihack tripwires
|
|
@@ -985,19 +1043,23 @@ function init(argv) {
|
|
|
985
1043
|
if (existsSync(out) && !argv.includes('--force')) die('promptwheel.config.json already exists, pass --force to overwrite');
|
|
986
1044
|
const pi = argv.indexOf('--preset');
|
|
987
1045
|
const presetName = pi >= 0 ? argv[pi + 1] : null;
|
|
988
|
-
let metrics, note, evaluatorPaths;
|
|
1046
|
+
let allowTrailer, metrics, note, evaluatorPaths, mandates;
|
|
989
1047
|
if (presetName) {
|
|
990
1048
|
const p = PRESETS[presetName];
|
|
991
1049
|
if (!p) die(`unknown preset "${presetName}", try: promptwheel init --list`);
|
|
992
1050
|
evaluatorPaths = p.evaluatorPaths; // a preset may ship a starter evaluator allowlist (e.g. security-antihack)
|
|
1051
|
+
mandates = p.mandates; allowTrailer = p.allowTrailer; // a preset may ship hard rules (supply-chain: workflows, registries, publish scripts, hooks)
|
|
993
1052
|
metrics = p.metrics || [{ name: 'tests_pass', cmd: detectTestCmd(repo), extract: 'exit', direction: 'pass', guard: true }];
|
|
994
1053
|
metrics = metrics.map((m) => m.cmd === '__TESTCMD__' ? { ...m, cmd: detectTestCmd(repo) } : m); // fill the preset's target
|
|
995
1054
|
// pack-backed presets: the referenced scripts live in THIS package, not
|
|
996
1055
|
// the user's repo, copy them in so the generated config is
|
|
997
1056
|
// self-contained and committable (a bare reference silently 404'd)
|
|
1057
|
+
const installed = new Set(); // a pack several metrics run (supply-chain) is copied and announced once
|
|
998
1058
|
metrics = metrics.map((m) => {
|
|
999
1059
|
const pk = /(^|\s)(packs\/[\w-]+)\//.exec(m.cmd);
|
|
1000
1060
|
if (!pk) return m;
|
|
1061
|
+
if (installed.has(pk[2])) return { ...m, cmd: m.cmd.replaceAll(pk[2] + '/', `.promptwheel/${pk[2]}/`) };
|
|
1062
|
+
installed.add(pk[2]);
|
|
1001
1063
|
const src = fileURLToPath(new URL(`../${pk[2]}/`, import.meta.url));
|
|
1002
1064
|
const dst = join(repo, '.promptwheel', pk[2]);
|
|
1003
1065
|
mkdirSync(dst, { recursive: true });
|
|
@@ -1025,8 +1087,10 @@ function init(argv) {
|
|
|
1025
1087
|
const setup = presetName ? undefined : detectSetup(repo);
|
|
1026
1088
|
const envCfg = presetName ? {} : { ...detectEnv(repo), ...(setup ? { setup } : {}) };
|
|
1027
1089
|
if (envCfg.linkDirs) note += ` · linking ${envCfg.linkDirs.join(', ')}`;
|
|
1028
|
-
writeFileSync(out, JSON.stringify({ repeat: 1, ...envCfg, ...(evaluatorPaths ? { evaluatorPaths } : {}), metrics }, null, 2) + '\n');
|
|
1090
|
+
writeFileSync(out, JSON.stringify({ repeat: 1, ...envCfg, ...(evaluatorPaths ? { evaluatorPaths } : {}), ...(mandates ? { mandates } : {}), ...(allowTrailer === false ? { allowTrailer } : {}), metrics }, null, 2) + '\n');
|
|
1029
1091
|
console.log(`✓ wrote promptwheel.config.json, ${note}`);
|
|
1092
|
+
const lockTool = join(repo, '.promptwheel', 'packs', 'supply-chain', 'lock.mjs');
|
|
1093
|
+
if (existsSync(lockTool)) console.log(`✓ ${execFileSync(process.execPath, [lockTool, 'write', repo], { encoding: 'utf8' }).trim()} (commit it with the config)`);
|
|
1030
1094
|
warnEditableInstall(repo); // one-time advice at setup, never on the per-turn path
|
|
1031
1095
|
if (!argv.includes('--no-agent-block')) {
|
|
1032
1096
|
const wrote = writeAgentBlock(repo);
|
|
@@ -1102,6 +1166,7 @@ function printHuman(r) {
|
|
|
1102
1166
|
const failReason = r.mandates ? ', a mandate was violated (a protected path changed)' : ', a guarded metric regressed (beyond noise)';
|
|
1103
1167
|
console.log(`\n VERDICT: ${r.verdict.toUpperCase()}${r.verdict === 'fail' ? failReason : r.verdict === 'gamed' ? ', a metric "improved" by editing the goalposts, not the source' : r.verdict === 'inconclusive' ? ', a guard is inert (measured nothing); a pass cannot be certified' : ''}`);
|
|
1104
1168
|
if (r.verdict === 'fail' && r.mandates) console.log(' a mandate is a hard rule; if the change is truly intended, edit the mandate in promptwheel.config.json (a tracked, tamper-evident change), do not work around it');
|
|
1169
|
+
else if (r.verdict === 'fail' && r.metrics.some((m) => m.allowable)) console.log(` intentional? this guard accepts a person's declared reason: promptwheel run --allow "<reason>" passes with the reason on the record (${r.metrics.filter((m) => m.allowable).map((m) => m.name).join(', ')})`);
|
|
1105
1170
|
else if (r.verdict === 'fail') console.log(' intentional? loosen that guard locally in promptwheel.config.json (guard:false, or override the inherited metric by name), `promptwheel guards` shows the effective set');
|
|
1106
1171
|
if (r.allowed) console.log(` ✓ allowed: ${r.allowed.cleared.join(', ')} acknowledged as an intentional change, recorded (Promptwheel-Allow: ${r.allowed.reason})`);
|
|
1107
1172
|
console.log('');
|
|
@@ -1368,7 +1433,7 @@ function conformance(argv) {
|
|
|
1368
1433
|
const GATE_FLAGS = ['--base', '--head', '--repeat', '--working', '--no-record', '--no-auto', '--allow', '--detect-gaming', '--no-detect-gaming', '--json', '--markdown'];
|
|
1369
1434
|
const COMMAND_FLAGS = {
|
|
1370
1435
|
run: GATE_FLAGS, improve: [...GATE_FLAGS, '--attempt'], insights: ['--json'], guards: ['--json'],
|
|
1371
|
-
init: ['--preset', '--list', '--force', '--no-agent-block'], conformance: ['--cmd'],
|
|
1436
|
+
init: ['--preset', '--list', '--force', '--no-agent-block'], conformance: ['--cmd'], lock: [],
|
|
1372
1437
|
};
|
|
1373
1438
|
const VALUED_FLAGS = new Set(['--base', '--head', '--repeat', '--allow', '--attempt', '--preset', '--cmd']);
|
|
1374
1439
|
function checkFlags(cmd, argv) {
|
|
@@ -1381,6 +1446,14 @@ function checkFlags(cmd, argv) {
|
|
|
1381
1446
|
}
|
|
1382
1447
|
}
|
|
1383
1448
|
|
|
1449
|
+
// write or refresh the architecture lock from the tree as it stands. A person runs this: the lock file is
|
|
1450
|
+
// a mandate, so the refreshed file passes the gate only with --allow "<reason>".
|
|
1451
|
+
async function lock() {
|
|
1452
|
+
const repo = repoRoot(process.cwd());
|
|
1453
|
+
const { writeLock, LOCK_FILE } = await import(new URL('../packs/supply-chain/lock.mjs', import.meta.url));
|
|
1454
|
+
console.log(`✓ ${writeLock(repo)} items locked in ${LOCK_FILE} (commit it; a later change to it needs promptwheel run --allow "<reason>")`);
|
|
1455
|
+
}
|
|
1456
|
+
|
|
1384
1457
|
function dispatch() {
|
|
1385
1458
|
const [first, ...rest] = process.argv.slice(2);
|
|
1386
1459
|
if (first === '--version' || first === '-v') { console.log(TOOL_VERSION ?? 'unknown'); process.exit(0); }
|
|
@@ -1392,6 +1465,7 @@ function dispatch() {
|
|
|
1392
1465
|
else if (cmd === 'init') init(rest);
|
|
1393
1466
|
else if (cmd === 'guards') guards(rest);
|
|
1394
1467
|
else if (cmd === 'conformance') conformance(rest);
|
|
1468
|
+
else if (cmd === 'lock') lock(rest);
|
|
1395
1469
|
else {
|
|
1396
1470
|
const help = !cmd || cmd === '--help' || cmd === '-h' || cmd === 'help';
|
|
1397
1471
|
if (!help) console.error(`unknown command: ${cmd}`);
|
|
@@ -1400,6 +1474,7 @@ function dispatch() {
|
|
|
1400
1474
|
'',
|
|
1401
1475
|
' promptwheel --version | -v print the version',
|
|
1402
1476
|
' promptwheel --help | -h this banner',
|
|
1477
|
+
' promptwheel lock write promptwheel.lock.json, the shape of this repo (dependencies, install scripts, workflows, modules, import edges, routes, entry points, hosts); the supply-chain preset holds it',
|
|
1403
1478
|
' promptwheel init [--preset <name> | --list] [--no-agent-block] write a starter config + an AGENTS.md guide so your agent runs the gate (OPTIONAL: run works zero-config, auto-detecting your test command)',
|
|
1404
1479
|
' promptwheel run [--base R] [--head R] [--repeat N] [--json|--markdown]',
|
|
1405
1480
|
' catches reward-hacking BY DEFAULT, re-proves each win from SOURCE edits alone (verdict GAMED,',
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "promptwheel",
|
|
3
|
-
"version": "0.7.
|
|
3
|
+
"version": "0.7.11",
|
|
4
4
|
"description": "A deterministic, zero-LLM check for your coding loop. It proves every change earned its win by re-running the result from the agent's source edits alone, so a green that came from editing the test reads GAMED. Runs in CI or a Claude Code hook, or import { check } in process.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
|
@@ -0,0 +1,103 @@
|
|
|
1
|
+
# PromptWheel supply-chain guard-pack
|
|
2
|
+
|
|
3
|
+
The step every npm supply-chain attack needs inside a target repo is a supply-chain edit: a
|
|
4
|
+
lifecycle script, a new dependency, a lockfile entry that resolves somewhere other than the
|
|
5
|
+
registry, a workflow or publish config. Those are paths and counts, and paths and counts are what
|
|
6
|
+
the gate checks deterministically. This pack holds the counts; the preset that installs it also
|
|
7
|
+
writes the mandates that freeze the paths.
|
|
8
|
+
|
|
9
|
+
```bash
|
|
10
|
+
promptwheel init --preset supply-chain
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
`init` copies the pack into `.promptwheel/packs/supply-chain/`, writes seven guarded metrics that run
|
|
14
|
+
it from there, and writes the mandates below. Commit the pack with the config: the gate measures in
|
|
15
|
+
clean git worktrees, so a pack that is not in git is absent at both refs and the metrics read
|
|
16
|
+
`unmeasurable` (init warns when `.promptwheel/` is gitignored).
|
|
17
|
+
|
|
18
|
+
## The seven counts
|
|
19
|
+
|
|
20
|
+
- **lifecycle_scripts**: install-time script entries (`preinstall`, `install`, `postinstall`,
|
|
21
|
+
`prepare`, `prepublish`, `prepublishOnly`, `postpack`) across every `package.json` in the repo.
|
|
22
|
+
- **dependency_entries**: dependency entries across every `package.json` (`dependencies`,
|
|
23
|
+
`devDependencies`, `optionalDependencies`, `peerDependencies`).
|
|
24
|
+
- **lock_nonregistry_sources**: lockfile entries (`package-lock.json`, `pnpm-lock.yaml`, `yarn.lock`)
|
|
25
|
+
resolved from somewhere other than the npm registry (git, ssh, http, a tarball on another host,
|
|
26
|
+
`file:`) plus registry entries that carry no integrity hash. Workspace links inside the repo do not
|
|
27
|
+
count. A private registry mirror makes this baseline large, not failing.
|
|
28
|
+
- **unpinned_execution**: code run from a reference that can move. A GitHub Action not pinned to a full commit SHA (a local `./action` does not count), `npx`, `bunx` or `dlx` with no exact version, `npm install -g` with no exact version, and a remote script piped into a shell. Read from workflows, `action.yml`, package.json scripts, `scripts/*.sh` and Dockerfiles. In package.json scripts, a tool the manifest declares does not count: the lockfile already pins it.
|
|
29
|
+
- **install_policy_gaps**: 0, 1 or 2. Are install scripts disabled, and is a release-age cooldown set (`ignore-scripts=true` and `min-release-age` in `.npmrc`, or the pnpm or yarn equivalents)? The cooldown is enforced by the package manager, not by this pack: a registry lookup here would make the verdict depend on the clock.
|
|
30
|
+
- **lock_drift**: structural items the tree has that `promptwheel.lock.json` does not. `promptwheel lock` reads the repo once and writes its shape: dependencies, install scripts, workflows, modules, import edges between modules, routes, public entry points, and the network hosts the source names. It also records each script's body, native binaries, and what a module can do (network, process spawning, eval), so a new capability shows even when the host is built at runtime. Ordinary edits never touch it. A new route is reported but not held by default; add `--routes` to hold it. On 1,403 commits across eight repositories this guard would have stopped 5.1%, about one in 20. JS and TS only for edges, routes, hosts and capabilities today.
|
|
31
|
+
- **pins_changed**: a fingerprint of the resolved dependency tree, compared base to head, because a new version under an old name is how a worm arrives. It covers every package, version, source and integrity, not the file's bytes, so a release of your own project does not count. Any change needs a reason. Nothing is stored, so the lock does not change across updates and an approved update does not blind the next check. On the same commits this stopped 8.7%, all real dependency changes; use the CI path below for those.
|
|
32
|
+
|
|
33
|
+
**Dependency updates in CI.** Give the reason from a step the agent cannot reach. Workflows are a frozen path, so an agent cannot edit this:
|
|
34
|
+
|
|
35
|
+
```yaml
|
|
36
|
+
- uses: promptwheel-ai/promptwheel@<full commit sha>
|
|
37
|
+
with:
|
|
38
|
+
allow: ${{ github.event.pull_request.user.login == 'dependabot[bot]' && 'dependabot update, reviewed in this PR' || '' }}
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
Check the PR author, not `github.actor`, which is whoever last triggered the run. Everything else still fails without a person's reason.
|
|
42
|
+
|
|
43
|
+
**What this does not catch.** A new destination reached through a module that already uses the network, when the address is assembled at runtime, reads as nothing new. The lock holds structure; it cannot follow a value.
|
|
44
|
+
|
|
45
|
+
A reason clears these guards only through the `--allow` flag, never through a commit trailer: an agent can write a trailer, so a guard meant to stop a compromised agent cannot be cleared by one. The lock file itself is a mandate that a person's `--allow` can clear, so an agent cannot launder a change by updating the lock in the same diff.
|
|
46
|
+
|
|
47
|
+
Each is a count, not a finding. The gate holds the number: a change that makes it rise fails,
|
|
48
|
+
whatever else the change did right. The count does not have to be zero.
|
|
49
|
+
|
|
50
|
+
## When the rise is the job
|
|
51
|
+
|
|
52
|
+
Sometimes adding a dependency is exactly what was asked. Say so, and the gate records it:
|
|
53
|
+
|
|
54
|
+
```bash
|
|
55
|
+
promptwheel run --working --allow "needed for the CSV export (papaparse)"
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
or a `Promptwheel-Allow: <reason>` trailer on the commit. The seven metrics carry `allowable: true`,
|
|
59
|
+
which means a declared reason clears their guarded regression for that one run, with the reason
|
|
60
|
+
written into `report.allowed` and the outcome record. Nothing is loosened in the config, and a rise
|
|
61
|
+
without a reason still fails. That is the point: every added package, script, or non-registry
|
|
62
|
+
source leaves an audit line a reviewer can read.
|
|
63
|
+
|
|
64
|
+
`gamingCheck` is `false` on all seven because these numbers legitimately live in config files
|
|
65
|
+
(`package.json`, lockfiles), which the source-only re-run reverts; re-proving them from source would
|
|
66
|
+
flag every honest dependency bump.
|
|
67
|
+
|
|
68
|
+
## The mandates the preset writes
|
|
69
|
+
|
|
70
|
+
A mandate is a path the agent must not touch in a change; the gate fails before any metric is read,
|
|
71
|
+
and a mandate deleted or narrowed in the same change that violates it stays in force.
|
|
72
|
+
|
|
73
|
+
| mandate | paths | why |
|
|
74
|
+
| --- | --- | --- |
|
|
75
|
+
| workflows-frozen | `.github/workflows/**` | workflows run with the repository's secrets |
|
|
76
|
+
| registry-config-frozen | `.npmrc`, `.yarnrc`, `.yarnrc.yml` | where packages come from and where tokens go |
|
|
77
|
+
| build-images-frozen | `Dockerfile*`, `docker-compose*` | what runs at build and deploy time |
|
|
78
|
+
| publish-scripts-frozen | `scripts/release*`, `scripts/publish*` | the publish path is how a compromise spreads |
|
|
79
|
+
| codeowners-frozen | `CODEOWNERS` | review ownership is set by people |
|
|
80
|
+
| hooks-frozen | `.claude/settings.json`, `.claude/settings.local.json`, `.sawbuck/**` | the hooks that enforce the contract are wired by a person |
|
|
81
|
+
|
|
82
|
+
The contract itself (`promptwheel.config.json`, `sawbuck.config.json`) is protected as an
|
|
83
|
+
`evaluatorPaths` entry rather than a mandate: a change that edits it alongside a win is `GAMED` by
|
|
84
|
+
declaration unless the change carries a declared reason, while a change that only edits the
|
|
85
|
+
contract (a person retiring a rule) still passes. A mandate there would make the contract
|
|
86
|
+
un-editable through the gate.
|
|
87
|
+
|
|
88
|
+
A person who needs to change a frozen path does it in a separate change that only touches that
|
|
89
|
+
path, reviewed as such.
|
|
90
|
+
|
|
91
|
+
## What this is not
|
|
92
|
+
|
|
93
|
+
- **Not a dependency firewall.** It does nothing about installing a package that is already
|
|
94
|
+
compromised on the registry. Use a registry policy, a cooldown on new versions, and
|
|
95
|
+
`--ignore-scripts` for that.
|
|
96
|
+
- **Not an injection detector.** It does not decide whether an instruction came from you or from
|
|
97
|
+
content the agent read. Whatever steered the agent, a supply-chain edit it was steered into fails
|
|
98
|
+
here, deterministically.
|
|
99
|
+
- **Not a scanner.** No matchers, no findings, no false positives to triage: a path, a count, or a
|
|
100
|
+
hash.
|
|
101
|
+
|
|
102
|
+
Pair it with the `security` preset for the eight code-level invariants, and with the GitHub Action so
|
|
103
|
+
the gate runs where the agent cannot switch it off.
|
|
@@ -0,0 +1,154 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
// PromptWheel architecture lock, zero-dep and zero-LLM. It reads a repository once and writes the shape
|
|
3
|
+
// of it to promptwheel.lock.json: dependencies, install scripts, workflows, modules, import edges between
|
|
4
|
+
// modules, routes, public entry points, and the network hosts the source names. The gate then holds that
|
|
5
|
+
// shape: anything the lock has not seen is drift.
|
|
6
|
+
// node lock.mjs write [path] write or refresh promptwheel.lock.json
|
|
7
|
+
// node lock.mjs drift [path] [--list] [--routes] print how many structural items the tree has that the lock does not
|
|
8
|
+
// node lock.mjs pins [path] print a fingerprint of the resolved dependency tree (the gate compares base to head)
|
|
9
|
+
// (routes are locked and listed, and counted only with --routes)
|
|
10
|
+
// The lock lists structure, not content, so ordinary edits never touch it. There is no timestamp in it:
|
|
11
|
+
// the same tree always writes the same bytes. JS and TS only for edges, routes and hosts today.
|
|
12
|
+
import { readFileSync, writeFileSync, existsSync, statSync } from 'node:fs';
|
|
13
|
+
import { join, dirname, normalize } from 'node:path';
|
|
14
|
+
import { execFileSync } from 'node:child_process';
|
|
15
|
+
import { createHash } from 'node:crypto';
|
|
16
|
+
import { fileURLToPath } from 'node:url';
|
|
17
|
+
|
|
18
|
+
export const LOCK_FILE = 'promptwheel.lock.json';
|
|
19
|
+
const SKIP = /(^|\/)(node_modules|\.git|dist|build|\.next|\.astro|coverage|vendor|out|target|\.venv|venv|__pycache__|\.promptwheel|\.sawbuck)(\/|$)/;
|
|
20
|
+
const TESTISH = /(^|\/)(test|tests|__tests__|__mocks__|e2e|spec|fixtures?|examples?|docs?|bench)\/|\.(test|spec|stories)\.[^/]+$/;
|
|
21
|
+
const CODE = /\.(mjs|cjs|js|jsx|ts|tsx|astro|svelte|vue)$/;
|
|
22
|
+
const DEP_KEYS = ['dependencies', 'devDependencies', 'optionalDependencies', 'peerDependencies'];
|
|
23
|
+
const LIFECYCLE = new Set(['preinstall', 'install', 'postinstall', 'prepare', 'prepublish', 'prepublishOnly', 'postpack']);
|
|
24
|
+
const IMPORT = /(?:\bfrom\s+|\brequire\(\s*|\bimport\s*\(\s*|^\s*import\s+)['"](\.{1,2}\/[^'"]+|@\/[^'"]+)['"]/gm;
|
|
25
|
+
const ROUTE_CALL = /\b(?:app|router|server|fastify)\.(get|post|put|patch|delete|all)\(\s*['"`](\/[^'"`]*)['"`]/g;
|
|
26
|
+
const HOST = /\bhttps?:\/\/([a-z0-9][a-z0-9.-]*\.[a-z]{2,}|\d{1,3}(?:\.\d{1,3}){3})(?=[:/'"`\s)]|$)/gi;
|
|
27
|
+
const NOT_A_HOST = /(^|\.)(example\.(com|org|net)|localhost|w3\.org|schema\.org|json-schema\.org|npmjs\.(com|org)|github\.com)$|^(127\.|0\.0\.0\.0)/i;
|
|
28
|
+
// lockfiles are fingerprinted, binaries hashed: any change to either needs a reason, whatever the name
|
|
29
|
+
const LOCKFILE = /(^|\/)(package-lock\.json|npm-shrinkwrap\.json|pnpm-lock\.yaml|yarn\.lock|bun\.lock)$/;
|
|
30
|
+
const BINARY = /\.(node|wasm)$/;
|
|
31
|
+
const sha = (t) => createHash('sha256').update(t).digest('hex').slice(0, 16);
|
|
32
|
+
// the resolved tree a lockfile pins, not its bytes: an npm lock repeats the project's own version, so a
|
|
33
|
+
// release would otherwise read as a dependency change. Every package, its version, source and hash stay in.
|
|
34
|
+
function resolvedTree(path, text) {
|
|
35
|
+
if (!/(package-lock|npm-shrinkwrap)\.json$/.test(path)) return text;
|
|
36
|
+
let j; try { j = JSON.parse(text); } catch { return text; }
|
|
37
|
+
const rows = [];
|
|
38
|
+
for (const [k, v] of Object.entries(j.packages || {})) if (k && v && !v.link) rows.push([k, v.version, v.resolved, v.integrity].join(' '));
|
|
39
|
+
const walk = (deps, pre) => { for (const [k, v] of Object.entries(deps || {})) { rows.push([pre + k, v.version, v.resolved, v.integrity].join(' ')); walk(v.dependencies, pre + k + '/'); } };
|
|
40
|
+
if (!j.packages) walk(j.dependencies, '');
|
|
41
|
+
return rows.sort().join('\n');
|
|
42
|
+
}
|
|
43
|
+
// what a module is able to do. First use in a module is structural; this is what catches an IP literal or a
|
|
44
|
+
// host assembled at runtime, which no URL pattern can see.
|
|
45
|
+
const CAPS = [
|
|
46
|
+
['exec', /\b(?:from\s+|require\(\s*|import\s*\(\s*)['"](?:node:)?(?:child_process|worker_threads|vm)['"]/],
|
|
47
|
+
['eval', /(?:^|[^.\w])eval\s*\(|\bnew\s+Function\s*\(/],
|
|
48
|
+
['net', /\bfetch\s*\(|\bXMLHttpRequest\b|\bWebSocket\s*\(|\b(?:from\s+|require\(\s*|import\s*\(\s*)['"](?:node:)?(?:net|dgram|http|https|tls|http2|dns)['"]/],
|
|
49
|
+
];
|
|
50
|
+
|
|
51
|
+
// the top-level module a path belongs to: packages/x, apps/x, src/x, lib/x, or the first directory
|
|
52
|
+
export function moduleOf(path) {
|
|
53
|
+
const p = path.split('/');
|
|
54
|
+
for (const pre of [['src', 'lib'], ['apps'], ['packages'], ['src'], ['lib']])
|
|
55
|
+
if (p.length > pre.length + 1 && pre.every((s, i) => p[i] === s)) return p.slice(0, pre.length + 1).join('/'); // src/app stays one module: a new page is a route, not a new module
|
|
56
|
+
return p.length > 1 ? p[0] : '.';
|
|
57
|
+
}
|
|
58
|
+
function routeOf(path) {
|
|
59
|
+
let m = path.match(/(?:^|\/)app\/(.*)\/(?:route|page)\.[jt]sx?$/);
|
|
60
|
+
if (m) return '/' + m[1].split('/').filter((s) => !/^\(.*\)$/.test(s)).join('/');
|
|
61
|
+
if (/(?:^|\/)app\/(?:route|page)\.[jt]sx?$/.test(path)) return '/';
|
|
62
|
+
m = path.match(/(?:^|\/)pages\/(.+)\.(?:astro|mdx?|[jt]sx?)$/);
|
|
63
|
+
if (m && !/\/_|^_/.test(m[1])) return '/' + m[1].replace(/(^|\/)index$/, '');
|
|
64
|
+
return null;
|
|
65
|
+
}
|
|
66
|
+
// every structural item one file contributes; pure, so a replay over history uses the same code
|
|
67
|
+
export function itemsOf(path, text) {
|
|
68
|
+
const out = new Set();
|
|
69
|
+
if (SKIP.test(path)) return out;
|
|
70
|
+
if (path.startsWith('.github/workflows/')) { out.add(`workflow:${path}`); return out; }
|
|
71
|
+
if (BINARY.test(path)) { out.add(`binary:${path}:${sha(text)}`); return out; }
|
|
72
|
+
if (path === 'package.json' || path.endsWith('/package.json')) {
|
|
73
|
+
let j; try { j = JSON.parse(text); } catch { return out; }
|
|
74
|
+
for (const k of DEP_KEYS) for (const n of Object.keys(j[k] || {})) out.add(`dep:${n}`);
|
|
75
|
+
for (const [s, v] of Object.entries(j.scripts || {})) { if (LIFECYCLE.has(s)) out.add(`script:${path}:${s}`); out.add(`scripthash:${path}:${s}:${sha(String(v))}`); }
|
|
76
|
+
if (path === 'package.json') {
|
|
77
|
+
if (typeof j.bin === 'string') out.add(`entry:bin:${j.name}`); else for (const b of Object.keys(j.bin || {})) out.add(`entry:bin:${b}`);
|
|
78
|
+
if (j.exports && typeof j.exports === 'object') for (const e of Object.keys(j.exports)) if (e.startsWith('.')) out.add(`entry:export:${e}`);
|
|
79
|
+
}
|
|
80
|
+
return out;
|
|
81
|
+
}
|
|
82
|
+
if (!CODE.test(path)) return out;
|
|
83
|
+
const route = routeOf(path);
|
|
84
|
+
if (route) out.add(`route:${route}`);
|
|
85
|
+
if (TESTISH.test(path)) return out;
|
|
86
|
+
const from = moduleOf(path);
|
|
87
|
+
out.add(`module:${from}`);
|
|
88
|
+
for (const m of text.matchAll(IMPORT)) {
|
|
89
|
+
const t = m[1];
|
|
90
|
+
const to = normalize(t.startsWith('@/') ? 'src/' + t.slice(2) : join(dirname(path), t));
|
|
91
|
+
if (to.startsWith('..')) continue;
|
|
92
|
+
const dest = moduleOf(to);
|
|
93
|
+
if (dest !== from) out.add(`edge:${from}>${dest}`);
|
|
94
|
+
}
|
|
95
|
+
for (const m of text.matchAll(ROUTE_CALL)) out.add(`route:${m[1].toUpperCase()} ${m[2]}`);
|
|
96
|
+
for (const line of text.split('\n')) {
|
|
97
|
+
const t = line.trim();
|
|
98
|
+
if (t.startsWith('//') || t.startsWith('*') || t.startsWith('/*')) continue;
|
|
99
|
+
for (const m of t.matchAll(HOST)) if (!NOT_A_HOST.test(m[1])) out.add(`host:${m[1].toLowerCase()}`);
|
|
100
|
+
for (const [cap, re] of CAPS) if (re.test(t)) out.add(`cap:${from}:${cap}`);
|
|
101
|
+
}
|
|
102
|
+
return out;
|
|
103
|
+
}
|
|
104
|
+
function tracked(root) {
|
|
105
|
+
try { return execFileSync('git', ['-C', root, 'ls-files', '-z', '--cached', '--others', '--exclude-standard'], { encoding: 'utf8', maxBuffer: 256 * 1024 * 1024, stdio: ['ignore', 'pipe', 'ignore'] }).split('\0').filter(Boolean); }
|
|
106
|
+
catch { return []; }
|
|
107
|
+
}
|
|
108
|
+
export function shapeOf(root) {
|
|
109
|
+
const all = new Set();
|
|
110
|
+
for (const f of tracked(root)) {
|
|
111
|
+
if (f === LOCK_FILE) continue;
|
|
112
|
+
let text = '';
|
|
113
|
+
const held = LOCKFILE.test(f) || BINARY.test(f); // never skipped for size: a large lockfile is the one that matters
|
|
114
|
+
try { if (!held && statSync(join(root, f)).size > 2_000_000) continue; text = readFileSync(join(root, f), held && BINARY.test(f) ? 'latin1' : 'utf8'); } catch { continue; }
|
|
115
|
+
for (const i of itemsOf(f, text)) all.add(i);
|
|
116
|
+
}
|
|
117
|
+
return [...all].sort();
|
|
118
|
+
}
|
|
119
|
+
export function writeLock(root) {
|
|
120
|
+
const items = shapeOf(root);
|
|
121
|
+
writeFileSync(join(root, LOCK_FILE), JSON.stringify({ version: 1, items }, null, 2) + '\n');
|
|
122
|
+
return items.length;
|
|
123
|
+
}
|
|
124
|
+
// a number that changes whenever the resolved dependency tree does. Compared base to head by the gate, so it
|
|
125
|
+
// needs no stored value: the lock stays the same across updates, and an approved update never blinds the next check.
|
|
126
|
+
export function pinsOf(root) {
|
|
127
|
+
const rows = [];
|
|
128
|
+
for (const f of tracked(root).filter((p) => LOCKFILE.test(p) && !SKIP.test(p)).sort()) {
|
|
129
|
+
try { rows.push(f + '\n' + resolvedTree(f, readFileSync(join(root, f), 'utf8'))); } catch { /* deleted but still in the index */ }
|
|
130
|
+
}
|
|
131
|
+
return rows.length ? parseInt(sha(rows.join('\n')).slice(0, 12), 16) : 0;
|
|
132
|
+
}
|
|
133
|
+
export function driftOf(root) {
|
|
134
|
+
const p = join(root, LOCK_FILE);
|
|
135
|
+
if (!existsSync(p)) return null;
|
|
136
|
+
let lock; try { lock = JSON.parse(readFileSync(p, 'utf8')); } catch { return null; }
|
|
137
|
+
if (!lock || !Array.isArray(lock.items)) return null;
|
|
138
|
+
const known = new Set(lock.items);
|
|
139
|
+
return shapeOf(root).filter((i) => !known.has(i));
|
|
140
|
+
}
|
|
141
|
+
if (process.argv[1] && fileURLToPath(import.meta.url) === process.argv[1]) {
|
|
142
|
+
const args = process.argv.slice(2).filter((a) => !a.startsWith('--'));
|
|
143
|
+
const root = args[1] || '.';
|
|
144
|
+
if (args[0] === 'write') { process.stdout.write(`${writeLock(root)} items locked in ${LOCK_FILE}\n`); }
|
|
145
|
+
else if (args[0] === 'drift') {
|
|
146
|
+
const all = driftOf(root);
|
|
147
|
+
// a new route is ordinary feature work, so by default it is reported, not held; --routes holds it too
|
|
148
|
+
const d = all && all.filter((i) => process.argv.includes('--routes') || !i.startsWith('route:'));
|
|
149
|
+
if (d === null) { process.stderr.write(`no readable ${LOCK_FILE}: run promptwheel lock\n`); process.exit(1); } // unmeasurable, never a clean 0
|
|
150
|
+
if (process.argv.includes('--list')) for (const i of all) process.stderr.write(` ${i}\n`);
|
|
151
|
+
process.stdout.write(process.argv.includes('--json') ? JSON.stringify(d) + '\n' : `${d.length}\n`);
|
|
152
|
+
} else if (args[0] === 'pins') { process.stdout.write(`${pinsOf(root)}\n`);
|
|
153
|
+
} else { process.stderr.write('usage: node lock.mjs <write|drift|pins> [path] [--list] [--json]\n'); process.exit(64); }
|
|
154
|
+
}
|
|
@@ -0,0 +1,76 @@
|
|
|
1
|
+
{
|
|
2
|
+
"$comment": "PromptWheel supply-chain guard-pack. Use via { \"extends\": [\"packs/supply-chain/pack.json\"] } in your promptwheel.config.json, or install it with `promptwheel init --preset supply-chain` (which also writes the mandates). Three counts the gate holds: a rise fails the gate unless the change carries a declared reason (`--allow \"<reason>\"` or a `Promptwheel-Allow:` trailer), which is recorded in the outcome record. gamingCheck is false because these numbers legitimately live in config files (package.json, lockfiles), which the source-only re-run reverts.",
|
|
3
|
+
"metrics": [
|
|
4
|
+
{
|
|
5
|
+
"$comment": "install-time script entries (preinstall, install, postinstall, prepare, prepublish, prepublishOnly, postpack) across every package.json: the step a supply-chain attack needs inside your repo",
|
|
6
|
+
"name": "lifecycle_scripts",
|
|
7
|
+
"cmd": "node packs/supply-chain/scan.mjs lifecycle .",
|
|
8
|
+
"extract": "number",
|
|
9
|
+
"direction": "down",
|
|
10
|
+
"guard": true,
|
|
11
|
+
"gamingCheck": false,
|
|
12
|
+
"allowable": "flag"
|
|
13
|
+
},
|
|
14
|
+
{
|
|
15
|
+
"$comment": "dependency entries across every package.json: a new package needs a recorded reason, so every addition leaves an audit line",
|
|
16
|
+
"name": "dependency_entries",
|
|
17
|
+
"cmd": "node packs/supply-chain/scan.mjs deps .",
|
|
18
|
+
"extract": "number",
|
|
19
|
+
"direction": "down",
|
|
20
|
+
"guard": true,
|
|
21
|
+
"gamingCheck": false,
|
|
22
|
+
"allowable": "flag"
|
|
23
|
+
},
|
|
24
|
+
{
|
|
25
|
+
"$comment": "lockfile entries resolved from outside the npm registry (git, ssh, http, another host's tarball, file:) plus registry entries with no integrity hash: where a package really comes from",
|
|
26
|
+
"name": "lock_nonregistry_sources",
|
|
27
|
+
"cmd": "node packs/supply-chain/scan.mjs lock-sources .",
|
|
28
|
+
"extract": "number",
|
|
29
|
+
"direction": "down",
|
|
30
|
+
"guard": true,
|
|
31
|
+
"gamingCheck": false,
|
|
32
|
+
"allowable": "flag"
|
|
33
|
+
},
|
|
34
|
+
{
|
|
35
|
+
"$comment": "code run from a floating reference: an action not pinned by SHA, npx or dlx with no exact version, a global install with no exact version, a remote script piped to a shell. Must not rise.",
|
|
36
|
+
"name": "unpinned_execution",
|
|
37
|
+
"cmd": "node packs/supply-chain/scan.mjs unpinned .",
|
|
38
|
+
"extract": "number",
|
|
39
|
+
"direction": "down",
|
|
40
|
+
"guard": true,
|
|
41
|
+
"gamingCheck": false,
|
|
42
|
+
"allowable": "flag"
|
|
43
|
+
},
|
|
44
|
+
{
|
|
45
|
+
"$comment": "install protections missing at the repo root: install scripts not disabled, no release-age cooldown. Must not rise.",
|
|
46
|
+
"name": "install_policy_gaps",
|
|
47
|
+
"cmd": "node packs/supply-chain/scan.mjs policy .",
|
|
48
|
+
"extract": "number",
|
|
49
|
+
"direction": "down",
|
|
50
|
+
"guard": true,
|
|
51
|
+
"gamingCheck": false,
|
|
52
|
+
"allowable": "flag"
|
|
53
|
+
},
|
|
54
|
+
{
|
|
55
|
+
"$comment": "structural items the tree has that promptwheel.lock.json does not. Must not rise.",
|
|
56
|
+
"name": "lock_drift",
|
|
57
|
+
"cmd": "node packs/supply-chain/lock.mjs drift .",
|
|
58
|
+
"extract": "number",
|
|
59
|
+
"direction": "down",
|
|
60
|
+
"guard": true,
|
|
61
|
+
"gamingCheck": false,
|
|
62
|
+
"allowable": "flag"
|
|
63
|
+
},
|
|
64
|
+
{
|
|
65
|
+
"$comment": "a fingerprint of the resolved dependency tree, compared base to head: a new version under an old name is how a worm arrives. Any change needs a reason.",
|
|
66
|
+
"name": "pins_changed",
|
|
67
|
+
"cmd": "node packs/supply-chain/lock.mjs pins .",
|
|
68
|
+
"extract": "number",
|
|
69
|
+
"direction": "down",
|
|
70
|
+
"guard": true,
|
|
71
|
+
"gamingCheck": false,
|
|
72
|
+
"allowable": "flag",
|
|
73
|
+
"exact": true
|
|
74
|
+
}
|
|
75
|
+
]
|
|
76
|
+
}
|
|
@@ -0,0 +1,221 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
// PromptWheel supply-chain guard-pack scanner, zero-dep. Counters over the repo's package
|
|
3
|
+
// manifests and lockfiles; each is a number the gate holds (it must not rise):
|
|
4
|
+
// lifecycle install-time script entries across every package.json
|
|
5
|
+
// (preinstall, install, postinstall, prepare, prepublish, prepublishOnly, postpack)
|
|
6
|
+
// deps dependency entries across every package.json
|
|
7
|
+
// (dependencies, devDependencies, optionalDependencies, peerDependencies)
|
|
8
|
+
// lock-sources lockfile entries resolved from somewhere other than the npm registry (git, ssh,
|
|
9
|
+
// http, a tarball on another host, file:) plus registry entries with no integrity hash
|
|
10
|
+
// These are counts, not findings. A private registry makes the baseline large, not failing: the
|
|
11
|
+
// guard only fails when a count rises, and a rise with a reason is recorded with `--allow`.
|
|
12
|
+
//
|
|
13
|
+
// Usage: node scan.mjs <lifecycle|deps|lock-sources> [path] prints one integer
|
|
14
|
+
// node scan.mjs <what> [path] --list also lists each hit on stderr
|
|
15
|
+
import { readFileSync, readdirSync, statSync } from 'node:fs';
|
|
16
|
+
import { join, basename } from 'node:path';
|
|
17
|
+
|
|
18
|
+
const positional = process.argv.slice(2).filter((a) => !a.startsWith('--'));
|
|
19
|
+
const WHAT = positional[0] || '';
|
|
20
|
+
const ROOT = positional[1] || '.';
|
|
21
|
+
const LIST = process.argv.includes('--list');
|
|
22
|
+
|
|
23
|
+
const SKIP = /(^|\/)(node_modules|\.git|dist|build|\.next|coverage|vendor|out|target|\.venv|venv|__pycache__)(\/|$)/;
|
|
24
|
+
const LIFECYCLE = ['preinstall', 'install', 'postinstall', 'prepare', 'prepublish', 'prepublishOnly', 'postpack'];
|
|
25
|
+
const DEP_KEYS = ['dependencies', 'devDependencies', 'optionalDependencies', 'peerDependencies'];
|
|
26
|
+
const REGISTRY = /^https:\/\/(registry\.npmjs\.org|registry\.yarnpkg\.com)\//;
|
|
27
|
+
|
|
28
|
+
const hits = [];
|
|
29
|
+
const hit = (where, what) => hits.push(`${where}: ${what}`);
|
|
30
|
+
|
|
31
|
+
function* walk(dir, names) {
|
|
32
|
+
let entries;
|
|
33
|
+
try { entries = readdirSync(dir); } catch { return; }
|
|
34
|
+
for (const e of entries) {
|
|
35
|
+
const p = join(dir, e);
|
|
36
|
+
if (SKIP.test(p)) continue;
|
|
37
|
+
let st;
|
|
38
|
+
try { st = statSync(p); } catch { continue; }
|
|
39
|
+
if (st.isDirectory()) yield* walk(p, names);
|
|
40
|
+
else if (names.has(basename(p))) yield p;
|
|
41
|
+
}
|
|
42
|
+
}
|
|
43
|
+
function readJson(p) { try { return JSON.parse(readFileSync(p, 'utf8')); } catch { return null; } }
|
|
44
|
+
function readText(p) { try { return readFileSync(p, 'utf8'); } catch { return ''; } }
|
|
45
|
+
const isObj = (v) => !!v && typeof v === 'object' && !Array.isArray(v);
|
|
46
|
+
|
|
47
|
+
// --- lifecycle: install-time script entries ---------------------------------------------------
|
|
48
|
+
function lifecycle() {
|
|
49
|
+
for (const file of walk(ROOT, new Set(['package.json']))) {
|
|
50
|
+
const pkg = readJson(file);
|
|
51
|
+
if (!isObj(pkg) || !isObj(pkg.scripts)) continue;
|
|
52
|
+
for (const k of LIFECYCLE) if (typeof pkg.scripts[k] === 'string' && pkg.scripts[k].trim()) hit(file, `${k} = ${pkg.scripts[k].slice(0, 120)}`); // an empty script runs nothing
|
|
53
|
+
}
|
|
54
|
+
}
|
|
55
|
+
|
|
56
|
+
// --- deps: dependency entries -------------------------------------------------------------------
|
|
57
|
+
function deps() {
|
|
58
|
+
for (const file of walk(ROOT, new Set(['package.json']))) {
|
|
59
|
+
const pkg = readJson(file);
|
|
60
|
+
if (!isObj(pkg)) continue;
|
|
61
|
+
for (const k of DEP_KEYS) if (isObj(pkg[k])) for (const name of Object.keys(pkg[k])) hit(file, `${k}: ${name}`);
|
|
62
|
+
}
|
|
63
|
+
}
|
|
64
|
+
|
|
65
|
+
// --- lock-sources: where lockfile entries come from ---------------------------------------------
|
|
66
|
+
// npm: lockfileVersion 2/3 carry a `packages` map; 1 carries nested `dependencies`. A workspace
|
|
67
|
+
// link (`link: true`) is a path inside the repo, not a source. Bundled entries carry no `resolved`
|
|
68
|
+
// and are skipped. Any entry with a `resolved` that is not the registry counts once; a registry
|
|
69
|
+
// entry with no `integrity` counts once.
|
|
70
|
+
function npmLock(file) {
|
|
71
|
+
const lock = readJson(file);
|
|
72
|
+
if (!isObj(lock)) return;
|
|
73
|
+
const judge = (entry, label) => {
|
|
74
|
+
if (!isObj(entry) || entry.link === true) return;
|
|
75
|
+
const r = entry.resolved;
|
|
76
|
+
if (typeof r !== 'string' || !r) return;
|
|
77
|
+
if (!REGISTRY.test(r)) hit(`${file} ${label}`, r.slice(0, 160));
|
|
78
|
+
else if (typeof entry.integrity !== 'string' || !entry.integrity) hit(`${file} ${label}`, 'no integrity');
|
|
79
|
+
};
|
|
80
|
+
if (isObj(lock.packages)) {
|
|
81
|
+
for (const [k, v] of Object.entries(lock.packages)) { if (k === '') continue; judge(v, k); }
|
|
82
|
+
} else if (isObj(lock.dependencies)) {
|
|
83
|
+
const rec = (d, prefix) => { for (const [k, v] of Object.entries(d)) { judge(v, prefix + k); if (isObj(v) && isObj(v.dependencies)) rec(v.dependencies, prefix + k + '>'); } };
|
|
84
|
+
rec(lock.dependencies, '');
|
|
85
|
+
}
|
|
86
|
+
}
|
|
87
|
+
// pnpm: a tolerant line scan of `resolution:` lines (no YAML parser). A registry resolution is
|
|
88
|
+
// `{integrity: ...}`; a tarball on another host, a git commit, or a tarball with no integrity counts.
|
|
89
|
+
// `directory:` resolutions are links inside the repo. Lines of a shape this scanner does not know
|
|
90
|
+
// count nothing.
|
|
91
|
+
function pnpmLock(file) {
|
|
92
|
+
const text = readText(file);
|
|
93
|
+
if (!text) return;
|
|
94
|
+
const lines = text.split('\n');
|
|
95
|
+
for (let i = 0; i < lines.length; i++) {
|
|
96
|
+
const m = /^\s*resolution:\s*(.*)$/.exec(lines[i]);
|
|
97
|
+
if (!m) continue;
|
|
98
|
+
const rest = m[1];
|
|
99
|
+
const tb = /\btarball:\s*['"]?([^'",}\s]+)/.exec(rest);
|
|
100
|
+
if (tb) {
|
|
101
|
+
if (!REGISTRY.test(tb[1])) hit(`${file}:${i + 1}`, tb[1].slice(0, 160));
|
|
102
|
+
else if (!/\bintegrity:/.test(rest)) hit(`${file}:${i + 1}`, 'no integrity');
|
|
103
|
+
} else if (/\b(commit|repo):/.test(rest) || /\btype:\s*git\b/.test(rest)) {
|
|
104
|
+
hit(`${file}:${i + 1}`, rest.slice(0, 160));
|
|
105
|
+
}
|
|
106
|
+
}
|
|
107
|
+
}
|
|
108
|
+
// yarn classic: blank-line separated blocks with `resolved "<url>#<hash>"` and an `integrity` line;
|
|
109
|
+
// a non-registry host counts, a registry entry with no integrity counts. yarn berry (`__metadata:`):
|
|
110
|
+
// `resolution: "<name>@<protocol>:..."`; anything but npm:, workspace:, patch:, link:, portal:
|
|
111
|
+
// (a git url, an https tarball, file:, exec:) counts. Berry always writes a checksum for npm
|
|
112
|
+
// resolutions, so no integrity check is applied there.
|
|
113
|
+
function yarnLock(file) {
|
|
114
|
+
const text = readText(file);
|
|
115
|
+
if (!text) return;
|
|
116
|
+
if (/^__metadata:/m.test(text)) {
|
|
117
|
+
const lines = text.split('\n');
|
|
118
|
+
for (let i = 0; i < lines.length; i++) {
|
|
119
|
+
const m = /^\s*resolution:\s*"?([^"]+)"?\s*$/.exec(lines[i]);
|
|
120
|
+
if (!m) continue;
|
|
121
|
+
const res = m[1];
|
|
122
|
+
const at = res.startsWith('@') ? res.indexOf('@', 1) : res.indexOf('@');
|
|
123
|
+
if (at < 0) continue;
|
|
124
|
+
const desc = res.slice(at + 1);
|
|
125
|
+
const proto = (/^([a-z][a-z0-9+]*):/i.exec(desc) || [])[1];
|
|
126
|
+
if (proto && ['npm', 'workspace', 'patch', 'link', 'portal'].includes(proto.toLowerCase())) continue;
|
|
127
|
+
if (REGISTRY.test(desc)) continue;
|
|
128
|
+
hit(`${file}:${i + 1}`, res.slice(0, 160));
|
|
129
|
+
}
|
|
130
|
+
return;
|
|
131
|
+
}
|
|
132
|
+
for (const block of text.split(/\n\s*\n/)) {
|
|
133
|
+
const r = /^\s*resolved\s+"?([^"\s]+)"?/m.exec(block);
|
|
134
|
+
if (!r) continue;
|
|
135
|
+
const url = r[1].replace(/#.*$/, '');
|
|
136
|
+
const label = (block.split('\n').find((l) => l.trim() && !l.startsWith('#')) || '').trim().slice(0, 80);
|
|
137
|
+
if (!REGISTRY.test(url)) hit(`${file} ${label}`, url.slice(0, 160));
|
|
138
|
+
else if (!/^\s*integrity\s/m.test(block)) hit(`${file} ${label}`, 'no integrity');
|
|
139
|
+
}
|
|
140
|
+
}
|
|
141
|
+
function lockSources() {
|
|
142
|
+
for (const file of walk(ROOT, new Set(['package-lock.json', 'pnpm-lock.yaml', 'yarn.lock']))) {
|
|
143
|
+
const name = basename(file);
|
|
144
|
+
try {
|
|
145
|
+
if (name === 'package-lock.json') npmLock(file);
|
|
146
|
+
else if (name === 'pnpm-lock.yaml') pnpmLock(file);
|
|
147
|
+
else yarnLock(file);
|
|
148
|
+
} catch { /* an unknown shape counts nothing, it never throws */ }
|
|
149
|
+
}
|
|
150
|
+
}
|
|
151
|
+
|
|
152
|
+
// --- unpinned: code that runs from a floating reference ----------------------------------------
|
|
153
|
+
// The step a worm needs is a reference that moves. Counted wherever a repo executes something it
|
|
154
|
+
// did not pin: a GitHub Action not pinned to a full commit SHA, npx/dlx with no exact version, a
|
|
155
|
+
// global install with no exact version, and a remote script piped into a shell. Local actions
|
|
156
|
+
// (./path) are not counted, and in package.json scripts neither is a tool the manifest declares,
|
|
157
|
+
// because the lockfile already pins that one.
|
|
158
|
+
const EXACT = /@\d+\.\d+\.\d+([-+][\w.-]+)?$/;
|
|
159
|
+
const pkgToken = (t) => t.replace(/^['"]|['"]$/g, '');
|
|
160
|
+
function execLines(text, where, declared) {
|
|
161
|
+
for (const raw of text.split('\n')) {
|
|
162
|
+
const line = raw.trim();
|
|
163
|
+
if (!line || line.startsWith('#') || line.startsWith('//')) continue;
|
|
164
|
+
const u = line.match(/^(?:-\s*)?uses:\s*['"]?([^'"\s#]+)/);
|
|
165
|
+
if (u) {
|
|
166
|
+
const ref = u[1];
|
|
167
|
+
if (ref.startsWith('./') || ref.startsWith('../')) continue;
|
|
168
|
+
if (ref.startsWith('docker://') ? !/@sha256:[0-9a-f]{64}$/.test(ref) : !/@[0-9a-f]{40}$/.test(ref)) hit(where, `action not pinned by SHA: ${ref}`);
|
|
169
|
+
continue;
|
|
170
|
+
}
|
|
171
|
+
for (const m of line.matchAll(/\b(?:npx|bunx|pnpx|pnpm\s+dlx|yarn\s+dlx)\s+((?:-{1,2}[\w-]+\s+)*)([^\s;&|)]+)/g)) {
|
|
172
|
+
const t = pkgToken(m[2]);
|
|
173
|
+
if (t.startsWith('-') || t.startsWith('$') || t.startsWith('.') || t.startsWith('/')) continue;
|
|
174
|
+
const name = t.replace(/@[^@/]*$/, '') || t;
|
|
175
|
+
if (declared && (declared.has(name) || declared.has(t))) continue;
|
|
176
|
+
if (!EXACT.test(t)) hit(where, `no exact version: ${m[0].trim()}`);
|
|
177
|
+
}
|
|
178
|
+
for (const m of line.matchAll(/\bnpm\s+(?:i|install)\s+(?:-g|--global)\s+((?:[^\s;&|)]+\s*)+)/g)) {
|
|
179
|
+
for (const t of m[1].trim().split(/\s+/).map(pkgToken)) if (t && !t.startsWith('-') && !t.startsWith('.') && !EXACT.test(t)) hit(where, `no exact version: npm install -g ${t}`);
|
|
180
|
+
}
|
|
181
|
+
if (/\b(curl|wget)\b[^|;&]*\|\s*(sudo\s+)?(ba|z|da)?sh\b/.test(line)) hit(where, `remote script piped to a shell: ${line.slice(0, 80)}`);
|
|
182
|
+
}
|
|
183
|
+
}
|
|
184
|
+
function unpinned() {
|
|
185
|
+
const yml = /\.ya?ml$/;
|
|
186
|
+
for (const dir of [join(ROOT, '.github', 'workflows')]) {
|
|
187
|
+
let names = []; try { names = readdirSync(dir); } catch { /* no workflows */ }
|
|
188
|
+
for (const n of names) if (yml.test(n)) execLines(readText(join(dir, n)), join(dir, n));
|
|
189
|
+
}
|
|
190
|
+
for (const f of walk(ROOT, new Set(['action.yml', 'action.yaml']))) execLines(readText(f), f);
|
|
191
|
+
for (const f of walk(ROOT, new Set(['package.json']))) {
|
|
192
|
+
const j = readJson(f); if (!isObj(j) || !isObj(j.scripts)) continue;
|
|
193
|
+
const declared = new Set(DEP_KEYS.flatMap((k) => (isObj(j[k]) ? Object.keys(j[k]) : [])));
|
|
194
|
+
for (const [k, v] of Object.entries(j.scripts)) if (typeof v === 'string') execLines(v, `${f} scripts.${k}`, declared);
|
|
195
|
+
}
|
|
196
|
+
for (const n of (() => { try { return readdirSync(join(ROOT, 'scripts')); } catch { return []; } })()) if (n.endsWith('.sh')) execLines(readText(join(ROOT, 'scripts', n)), join(ROOT, 'scripts', n));
|
|
197
|
+
for (const n of (() => { try { return readdirSync(ROOT); } catch { return []; } })()) if (/^Dockerfile/.test(n)) execLines(readText(join(ROOT, n)), join(ROOT, n));
|
|
198
|
+
}
|
|
199
|
+
|
|
200
|
+
// --- policy: install protections the repo declares ----------------------------------------------
|
|
201
|
+
// Two gaps, counted at the repo root. Release age itself stays with the package manager: a registry
|
|
202
|
+
// lookup here would make the verdict depend on the clock.
|
|
203
|
+
function policy() {
|
|
204
|
+
if (!readJson(join(ROOT, 'package.json'))) return;
|
|
205
|
+
const text = ['.npmrc', 'pnpm-workspace.yaml', '.yarnrc.yml'].map((f) => readText(join(ROOT, f))).join('\n');
|
|
206
|
+
if (!/^\s*ignore-scripts\s*=\s*true/m.test(text) && !/^\s*(strictDepBuilds|enableScripts)\s*:\s*(true|false)/m.test(text)) hit('.npmrc', 'install scripts are not disabled (ignore-scripts=true)');
|
|
207
|
+
if (!/^\s*min-release-age\s*=\s*[1-9]/m.test(text) && !/^\s*minimumReleaseAge\s*:\s*[1-9]/m.test(text) && !/^\s*npmMinimalAgeGate\s*:\s*[1-9"']/m.test(text)) hit('.npmrc', 'no release-age cooldown (min-release-age)');
|
|
208
|
+
}
|
|
209
|
+
|
|
210
|
+
const COUNTERS = { lifecycle, deps, 'lock-sources': lockSources, unpinned, policy };
|
|
211
|
+
if (!COUNTERS[WHAT]) {
|
|
212
|
+
process.stderr.write('usage: node scan.mjs <lifecycle|deps|lock-sources|unpinned|policy> [path] [--list]\n');
|
|
213
|
+
process.exit(64); // a typo in the config must read as unmeasurable, never as a clean 0
|
|
214
|
+
}
|
|
215
|
+
COUNTERS[WHAT]();
|
|
216
|
+
if (LIST) {
|
|
217
|
+
for (const h of hits) process.stderr.write(` ${h}\n`);
|
|
218
|
+
process.stderr.write(`\n${hits.length} ${WHAT} entr${hits.length === 1 ? 'y' : 'ies'}\n`);
|
|
219
|
+
}
|
|
220
|
+
// the metric value: lower is better; the gate holds the count, it does not require 0
|
|
221
|
+
process.stdout.write(String(hits.length) + '\n');
|