promptwheel 0.7.16 → 0.8.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "promptwheel",
3
- "version": "0.7.16",
3
+ "version": "0.8.1",
4
4
  "description": "A deterministic, zero-LLM check for your coding loop. A win must survive a re-run from the agent's source edits alone, so a test that went green by editing the test reads GAMED. Runs in CI or a Claude Code plugin.",
5
5
  "type": "module",
6
6
  "bin": {
@@ -27,7 +27,7 @@ clean git worktrees, so a pack that is not in git is absent at both refs and the
27
27
  count. A private registry mirror makes this baseline large, not failing.
28
28
  - **unpinned_execution**: code run from a reference that can move. A GitHub Action not pinned to a full commit SHA (a local `./action` does not count), `npx`, `bunx` or `dlx` with no exact version, `npm install -g` with no exact version, and a remote script piped into a shell. Read from workflows, `action.yml`, package.json scripts, `scripts/*.sh` and Dockerfiles. In package.json scripts, a tool the manifest declares does not count: the lockfile already pins it.
29
29
  - **install_policy_gaps**: 0, 1 or 2. Are install scripts disabled, and is a release-age cooldown set (`ignore-scripts=true` and `min-release-age` in `.npmrc`, or the pnpm or yarn equivalents)? The cooldown is enforced by the package manager, not by this pack: a registry lookup here would make the verdict depend on the clock.
30
- - **lock_drift**: structural items the tree has that `promptwheel.lock.json` does not. `promptwheel lock` reads the repo once and writes its shape: dependencies, install scripts, workflows, modules, import edges between modules, routes, public entry points, and the network hosts the source names. It also records each script's body, native binaries, and what a module can do (network, process spawning, eval), so a new capability shows even when the host is built at runtime. Ordinary edits never touch it. A new route is reported but not held by default; add `--routes` to hold it. On 1,403 commits across eight repositories this guard would have stopped 5.1%, about one in 20. JS and TS only for edges, routes, hosts and capabilities today.
30
+ - **lock_drift**: structural items the tree has that `promptwheel.lock.json` does not. `promptwheel lock` reads the repo once and writes its shape: dependencies, install scripts, workflows, modules, import edges between modules, routes, public entry points, and the network hosts the source names. It also records each script's body, native binaries, and what a module can do (network, process spawning, eval), so a new capability shows even when the host is built at runtime. Ordinary edits never touch it. A new route is locked but not counted by default (`node .promptwheel/packs/supply-chain/lock.mjs drift . --list` shows it). To hold new routes too, add `--routes` to the end of the `lock_drift` metric's `cmd` in `promptwheel.config.json`, so it reads `node .promptwheel/packs/supply-chain/lock.mjs drift . --routes`. The flag belongs to that script: `promptwheel lock` takes no flags and always writes routes into the lock. The config is a frozen path here, so that edit passes with `promptwheel run --allow "<reason>"`. On 1,403 commits across eight repositories this guard would have stopped 5.1%, about one in 20. JS and TS only for edges, routes, hosts and capabilities today.
31
31
  - **pins_changed**: a fingerprint of the resolved dependency tree, compared base to head, because a new version under an old name is how a worm arrives. It covers every package, version, source and integrity, not the file's bytes, so a release of your own project does not count. Any change needs a reason. Nothing is stored, so the lock does not change across updates and an approved update does not blind the next check. On the same commits this stopped 8.7%, dependency changes; use the CI path below for those.
32
32
 
33
33
  **On a pull request.** Someone with write access comments `sawbuck-allow: <reason>` and re-runs the check. `sawbuck contract` marks everyday guards `report: true`, so they note a change without failing until you tighten them; this preset on its own holds them all.
@@ -57,10 +57,10 @@ Sometimes adding a dependency is exactly what was asked. Say so, and the gate re
57
57
  promptwheel run --working --allow "needed for the CSV export (papaparse)"
58
58
  ```
59
59
 
60
- or a `Promptwheel-Allow: <reason>` trailer on the commit. The seven metrics carry `allowable: true`,
61
- which means a declared reason clears their guarded regression for that one run, with the reason
62
- written into `report.allowed` and the outcome record. Nothing is loosened in the config, and a rise
63
- without a reason still fails. That is the point: every added package, script, or non-registry
60
+ A commit trailer does not count here. The seven metrics carry `allowable: "flag"`, which means a
61
+ reason from the `--allow` flag clears their guarded regression for that one run, with the reason
62
+ written into `report.allowed`. Nothing is loosened in the config, and a rise without a reason still
63
+ fails. That is the point: every added package, script, or non-registry
64
64
  source leaves an audit line a reviewer can read.
65
65
 
66
66
  `gamingCheck` is `false` on all seven because these numbers legitimately live in config files
@@ -69,8 +69,8 @@ flag every honest dependency bump.
69
69
 
70
70
  ## The mandates the preset writes
71
71
 
72
- A mandate is a path the agent must not touch in a change; the gate fails before any metric is read,
73
- and a mandate deleted or narrowed in the same change that violates it stays in force.
72
+ A mandate is a path the agent must not touch in a change. Touching one fails the gate whatever the
73
+ metrics say, and a mandate deleted or narrowed in the same change that violates it stays in force.
74
74
 
75
75
  | mandate | paths | why |
76
76
  | --- | --- | --- |
@@ -79,15 +79,20 @@ and a mandate deleted or narrowed in the same change that violates it stays in f
79
79
  | build-images-frozen | `Dockerfile*`, `docker-compose*` | what runs at build and deploy time |
80
80
  | publish-scripts-frozen | `scripts/release*`, `scripts/publish*` | the publish path is how a compromise spreads |
81
81
  | codeowners-frozen | `CODEOWNERS` | review ownership is set by people |
82
+ | scanner-frozen | `.promptwheel/packs/**` | the scanners are not edited by the change they read |
82
83
  | contract-frozen | `promptwheel.config.json`, `sawbuck.config.json` | the contract changes with a reason from a person (`--allow`) |
83
- | hooks-frozen | `.claude/settings.json`, `.claude/settings.local.json`, `.sawbuck/**` | the hooks that enforce the contract are wired by a person |
84
+ | lock-frozen | `promptwheel.lock.json` | the lock changes with a reason from a person (`--allow`) |
85
+ | hooks-frozen | `.claude/settings.json`, `.claude/settings.local.json`, `.sawbuck/**` | the hooks change with a reason from a person (`--allow`) |
84
86
 
85
87
  The contract itself (`promptwheel.config.json`, `sawbuck.config.json`) is frozen too, and it is also an
86
88
  `evaluatorPaths` entry. A change to it needs `--allow "<reason>"` from a person. Without that, a change that
87
89
  only weakens a rule would pass, and the next change would be judged by the weakened contract.
88
90
 
89
- A person who needs to change a frozen path does it in a separate change that only touches that
90
- path, reviewed as such.
91
+ `--allow` clears only three of these: `contract-frozen`, `lock-frozen` and `hooks-frozen` (so a team can commit
92
+ shared hooks to a repo that already has its rules). A commit trailer clears none. Every other frozen path stays
93
+ frozen, even with `--allow`. To change one, a person first retires its mandate in a change of its own
94
+ (it edits the contract, so it needs `--allow`), then makes the change, then puts the mandate back the
95
+ same way.
91
96
 
92
97
  ## What this is not
93
98
 
@@ -100,5 +105,4 @@ path, reviewed as such.
100
105
  - **Not a scanner.** No matchers, no findings, no false positives to triage: a path, a count, or a
101
106
  hash.
102
107
 
103
- Pair it with the `security` preset for the eight code-level invariants, and with the GitHub Action so
104
- the gate runs where the agent cannot switch it off.
108
+ Pair it with the GitHub Action so the gate runs where the agent cannot switch it off.
@@ -1,5 +1,5 @@
1
1
  {
2
- "$comment": "PromptWheel supply-chain guard-pack. Use via { \"extends\": [\"packs/supply-chain/pack.json\"] } in your promptwheel.config.json, or install it with `promptwheel init --preset supply-chain` (which also writes the mandates). Three counts the gate holds: a rise fails the gate unless the change carries a declared reason (`--allow \"<reason>\"` or a `Promptwheel-Allow:` trailer), which is recorded in the outcome record. gamingCheck is false because these numbers legitimately live in config files (package.json, lockfiles), which the source-only re-run reverts.",
2
+ "$comment": "PromptWheel supply-chain guard-pack. Install it with `promptwheel init --preset supply-chain`: that copies the pack to .promptwheel/packs/supply-chain/, writes these metrics into promptwheel.config.json with their commands pointed at the copy, and writes the mandates. The commands in this file run packs/supply-chain/..., which exists only in the PromptWheel repository itself, so `extends` on this file (here or on the copy under .promptwheel/packs/) reads every metric as unmeasurable, and the gate fails, in any other repository. Seven counts the gate holds: a rise fails the gate unless a person gives a reason with `--allow \"<reason>\"` (a commit trailer does not count), which is recorded in report.allowed. gamingCheck is false because these numbers legitimately live in config files (package.json, lockfiles), which the source-only re-run reverts.",
3
3
  "metrics": [
4
4
  {
5
5
  "$comment": "install-time script entries (preinstall, install, postinstall, prepare, prepublish, prepublishOnly, postpack) across every package.json: the step a supply-chain attack needs inside your repo",
@@ -1,62 +0,0 @@
1
- # PromptWheel security guard-pack
2
-
3
- Gate your AI coding loop on **security invariants**, not just tests. When an agent
4
- writes backend code, "tests pass" says nothing about whether it shipped a CORS
5
- wildcard, a client-controlled payment amount, or a hardcoded secret. This pack
6
- adds a `security_findings` metric that must not climb - and because it's a
7
- PromptWheel metric, the win has to survive `--detect-gaming`: an agent can't
8
- lower the count by editing the pattern file or a config, only by fixing the code.
9
-
10
- ## Use it
11
-
12
- The pack ships inside the `promptwheel` package, not in your repo, so install it with the
13
- preset:
14
-
15
- ```bash
16
- promptwheel init --preset security # or --preset security-antihack, which adds the tripwires
17
- ```
18
-
19
- `init` copies the pack into `.promptwheel/packs/security/` and writes a `security_findings`
20
- metric that runs it from there. Commit the pack with the config: the gate measures in clean
21
- git worktrees, so a pack that is not in git is absent at both refs and the metric reads
22
- `unmeasurable` (init warns when `.promptwheel/` is gitignored; change that rule to
23
- `.promptwheel/*` plus `!.promptwheel/packs/`, since git cannot re-include a file under an
24
- ignored directory). Then gate as usual:
25
-
26
- ```bash
27
- promptwheel run --working # fails if security_findings increased
28
- ```
29
-
30
- `pack.json` is the same metric as a config fragment for `extends`
31
- (`{ "extends": ["packs/security/pack.json"] }`); its command is relative to a repo that
32
- holds the pack at `packs/security/`, such as a vendored copy or this repository.
33
-
34
- A finding-count that drops because you fixed the code is a `PASS`; a "fix" that
35
- only survives with test/config/pattern edits reverted is `GAMED`.
36
-
37
- ## What it checks (v0.1 - 8 high-precision invariants)
38
-
39
- Decanted from [securitychecks](https://securitychecks.ai)' "recommended" tier -
40
- the regex-portable subset, chosen for near-zero false positives:
41
-
42
- | Invariant | Severity |
43
- | --- | --- |
44
- | CORS wildcard origin | P0 |
45
- | CORS origin reflection | P0 |
46
- | Insecure session cookie (`secure:false`) | P0 |
47
- | Payment amount from client input | P0 |
48
- | Hardcoded secret in a sensitive variable | P0 |
49
- | NPM token committed in `.npmrc` | P0 |
50
- | Template autoescape disabled (XSS) | P0 |
51
- | S3 presigned upload without content validation | P2 |
52
-
53
- Each pattern ships with its own false-positive guards (the `exclude` regexes).
54
-
55
- ## Scope, honestly
56
-
57
- This is the **regex-portable** slice. The other ~75 recommended invariants
58
- (auth-enforcement, authz-revocation, tenant-isolation, dataflow-tainted sinks)
59
- need an AST + dataflow engine and are a possible follow-up, not part of this
60
- pack. The point of v0.1 is the architecture: security invariants are
61
- now a first-class, un-gameable PromptWheel metric. Zero dependencies; the scanner
62
- is one file (`scan.mjs`).
@@ -1,12 +0,0 @@
1
- {
2
- "$comment": "PromptWheel security guard-pack. Decanted from securitychecks (recommended tier). Use via { \"extends\": [\"packs/security/pack.json\"] } in your promptwheel.config.json. Gates on the count of security-invariant violations (direction: down = lower is better; guard: true = fail if it climbs). Because it's a promptwheel metric, the win must survive --detect-gaming: an agent can't lower the count by editing the pattern file or a config, only by fixing the code.",
3
- "metrics": [
4
- {
5
- "name": "security_findings",
6
- "cmd": "node packs/security/scan.mjs .",
7
- "extract": "number",
8
- "direction": "down",
9
- "guard": true
10
- }
11
- ]
12
- }
@@ -1,209 +0,0 @@
1
- {
2
- "pack": "security",
3
- "version": "0.1.0",
4
- "source": "securitychecks recommended tier (regex-portable subset)",
5
- "patterns": [
6
- {
7
- "id": "DATAFLOW.UNTRUSTED.XSS",
8
- "name": "Template Engine Autoescape Disabled",
9
- "severity": "P0",
10
- "match": "autoescape\\s*:\\s*false",
11
- "exclude": [],
12
- "message": "Template engine autoescape disabled \u2014 enables stored XSS attacks",
13
- "fix": "Enable autoescape or sanitize all user input before rendering",
14
- "files": [
15
- "**/*.ts",
16
- "**/*.js",
17
- "!**/__tests__/**",
18
- "!**/*.test.*",
19
- "!**/*.spec.*",
20
- "!**/test/**",
21
- "!**/tests/**",
22
- "!**/__mocks__/**",
23
- "!**/fixtures/**",
24
- "!**/www/**",
25
- "!**/docs/**",
26
- "!**/documentation/**"
27
- ]
28
- },
29
- {
30
- "id": "SESSION.INSECURE_COOKIE",
31
- "name": "Insecure Session Cookie",
32
- "severity": "P0",
33
- "match": "secure\\s*:\\s*false",
34
- "exclude": [],
35
- "message": "Session cookies must have secure and httpOnly flags",
36
- "fix": "Verify this is not a false positive",
37
- "files": [
38
- "**/*.ts",
39
- "**/*.js",
40
- "**/*.tsx",
41
- "**/*.jsx",
42
- "!**/__tests__/**",
43
- "!**/*.test.*",
44
- "!**/*.spec.*",
45
- "!**/test/**",
46
- "!**/tests/**",
47
- "!**/__mocks__/**",
48
- "!**/fixtures/**",
49
- "!**/e2e/**",
50
- "!**/playwright/**",
51
- "!**/cypress/**",
52
- "!**/test-utils/**",
53
- "!**/*.e2e.*",
54
- "!**/e2e-*/**"
55
- ]
56
- },
57
- {
58
- "id": "AUTHZ.CORS_WILDCARD",
59
- "name": "CORS Wildcard Origin",
60
- "severity": "P0",
61
- "match": "Access-Control-Allow-Origin.*\\*|origin\\s*:\\s*\\*|origin:\\s*true",
62
- "exclude": [
63
- "!credentials"
64
- ],
65
- "message": "Wildcard origin with credentials exposes user data",
66
- "fix": "Verify this is not a false positive",
67
- "files": [
68
- "**/*.ts",
69
- "**/*.js",
70
- "**/*.tsx",
71
- "**/*.jsx",
72
- "!**/__tests__/**",
73
- "!**/*.test.*",
74
- "!**/*.spec.*",
75
- "!**/test/**",
76
- "!**/tests/**",
77
- "!**/__mocks__/**",
78
- "!**/fixtures/**",
79
- "!**/*.tsx",
80
- "!**/*.jsx",
81
- "!**/components/**"
82
- ]
83
- },
84
- {
85
- "id": "AUTHZ.CORS_REFLECT",
86
- "name": "CORS Origin Reflection",
87
- "severity": "P0",
88
- "match": "(?:Access-Control-Allow-Origin|res\\.(?:set|setHeader|header)\\(['\"]Access-Control).*(?:req\\.headers\\.origin|headers\\.get\\(['\"]origin['\"]\\))",
89
- "exclude": [],
90
- "message": "Reflecting origin is equivalent to wildcard \u2014 validate against allowlist",
91
- "fix": "Verify this is not a false positive",
92
- "files": [
93
- "**/*.ts",
94
- "**/*.js",
95
- "**/*.tsx",
96
- "**/*.jsx",
97
- "!**/__tests__/**",
98
- "!**/*.test.*",
99
- "!**/*.spec.*",
100
- "!**/test/**",
101
- "!**/tests/**",
102
- "!**/__mocks__/**",
103
- "!**/fixtures/**"
104
- ]
105
- },
106
- {
107
- "id": "PAY.CLIENT_AMOUNT",
108
- "name": "Payment Amount From Client",
109
- "severity": "P0",
110
- "match": "amount.*req\\.body|price.*req\\.body|total.*body\\.amount",
111
- "exclude": [
112
- "server|database|catalog"
113
- ],
114
- "message": "Calculate payment amounts server-side, not from client input",
115
- "fix": "Verify this is not a false positive",
116
- "files": [
117
- "**/*.ts",
118
- "**/*.js",
119
- "**/*.tsx",
120
- "**/*.jsx",
121
- "!**/__tests__/**",
122
- "!**/*.test.*",
123
- "!**/*.spec.*",
124
- "!**/test/**",
125
- "!**/tests/**",
126
- "!**/__mocks__/**",
127
- "!**/fixtures/**"
128
- ]
129
- },
130
- {
131
- "id": "CONFIG.NPMRC_TOKEN",
132
- "name": "NPM Token in .npmrc",
133
- "severity": "P0",
134
- "match": "_authToken=|//registry.*:_auth",
135
- "exclude": [
136
- "\\$\\{NPM_TOKEN\\}|process\\.env"
137
- ],
138
- "message": "NPM tokens should come from environment variables",
139
- "fix": "Verify this is not a false positive",
140
- "files": [
141
- "**/*",
142
- "!**/__tests__/**",
143
- "!**/*.test.*",
144
- "!**/*.spec.*",
145
- "!**/test/**",
146
- "!**/tests/**",
147
- "!**/__mocks__/**",
148
- "!**/fixtures/**"
149
- ]
150
- },
151
- {
152
- "id": "VALID.UPLOAD_NOCHECK",
153
- "name": "S3 Presigned Upload Without Content Validation",
154
- "severity": "P2",
155
- "match": "(?:createPresignedPost|getSignedUrl|PutObjectCommand)\\s*\\(",
156
- "exclude": [
157
- "ContentType|content-type|Content-Length|content-length|Conditions|conditions",
158
- "allowedTypes|fileFilter|mimetype|maxSize|maxFileSize",
159
- "@example|Example:|```|description:|guidance",
160
- "GetObjectCommand|getObject|HeadObjectCommand|DeleteObjectCommand|download|Download|read|Read",
161
- "generateSignedUrl.*get|getSignedUrl.*getObject"
162
- ],
163
- "message": "S3 presigned upload URL has no content type or size conditions \u2014 any file type/size can be uploaded",
164
- "fix": "Add Conditions to presigned POST or ContentType/ContentLength constraints",
165
- "files": [
166
- "**/*.ts",
167
- "**/*.js",
168
- "!**/*.tsx",
169
- "!**/*.jsx",
170
- "!**/components/**",
171
- "!**/frontend/**",
172
- "!**/public/**",
173
- "!**/__tests__/**",
174
- "!**/*.test.*",
175
- "!**/*.spec.*",
176
- "!**/test/**",
177
- "!**/tests/**",
178
- "!**/__mocks__/**",
179
- "!**/fixtures/**"
180
- ]
181
- },
182
- {
183
- "id": "SECRETS.HARDCODED",
184
- "name": "Hardcoded Secret in Sensitive Variable",
185
- "severity": "P0",
186
- "match": "(?:const|let|var)\\s+(?:[A-Z_]*(?:SECRET|JWT_SECRET|API_KEY|AUTH_TOKEN|PRIVATE_KEY|SIGNING_KEY|ENCRYPTION_KEY)[A-Z_]*)\\s*=\\s*['\"][^'\"]{8,}['\"]",
187
- "exclude": [
188
- "process\\.env|import\\.meta\\.env|getenv|\\$\\{|placeholder|changeme|YOUR_|TODO|FIXME|example|EXAMPLE"
189
- ],
190
- "message": "Secret value hardcoded in source code \u2014 use environment variables",
191
- "fix": "Verify this is not a placeholder or example value",
192
- "files": [
193
- "**/*.ts",
194
- "**/*.js",
195
- "!**/__tests__/**",
196
- "!**/*.test.*",
197
- "!**/*.spec.*",
198
- "!**/test/**",
199
- "!**/tests/**",
200
- "!**/__mocks__/**",
201
- "!**/fixtures/**",
202
- "!**/sample/**",
203
- "!**/samples/**",
204
- "!**/example/**",
205
- "!**/examples/**"
206
- ]
207
- }
208
- ]
209
- }
@@ -1,57 +0,0 @@
1
- #!/usr/bin/env node
2
- // PromptWheel security guard-pack scanner — zero-dep.
3
- // Decanted from securitychecks' "recommended" tier (the regex-portable subset).
4
- // Emits the count of security-invariant violations to stdout (the metric value).
5
- // The full AST+dataflow engine (the other 75 recommended patterns) is the follow-up.
6
- //
7
- // Usage: node scan.mjs [path] → prints the finding count (for `promptwheel` metrics)
8
- // node scan.mjs [path] --list → prints each finding (file:line · id · message)
9
- import { readFileSync, readdirSync, statSync } from 'node:fs';
10
- import { join, extname } from 'node:path';
11
- import { fileURLToPath } from 'node:url';
12
-
13
- const ROOT = process.argv[2] && !process.argv[2].startsWith('--') ? process.argv[2] : '.';
14
- const LIST = process.argv.includes('--list');
15
- const PACK = JSON.parse(readFileSync(join(fileURLToPath(new URL('.', import.meta.url)), 'patterns.json'), 'utf8'));
16
-
17
- const SKIP = /(^|\/)(node_modules|\.git|dist|build|\.next|coverage|vendor)(\/|$)/;
18
- const CODE = new Set(['.ts', '.tsx', '.js', '.jsx', '.mjs', '.cjs']);
19
- const CONFIG_FILES = /(^|\/)\.npmrc$/;
20
-
21
- function* walk(dir) {
22
- let entries;
23
- try { entries = readdirSync(dir); } catch { return; }
24
- for (const e of entries) {
25
- const p = join(dir, e);
26
- if (SKIP.test(p)) continue;
27
- let st;
28
- try { st = statSync(p); } catch { continue; }
29
- if (st.isDirectory()) yield* walk(p);
30
- else if (CODE.has(extname(p)) || CONFIG_FILES.test(p)) yield p;
31
- }
32
- }
33
-
34
- const findings = [];
35
- for (const file of walk(ROOT)) {
36
- let text;
37
- try { text = readFileSync(file, 'utf8'); } catch { continue; }
38
- const lines = text.split('\n');
39
- for (const pat of PACK.patterns) {
40
- let re;
41
- try { re = new RegExp(pat.match, 'i'); } catch { continue; }
42
- const excludes = (pat.exclude || []).map((x) => { try { return new RegExp(x, 'i'); } catch { return null; } }).filter(Boolean);
43
- for (let i = 0; i < lines.length; i++) {
44
- const line = lines[i];
45
- if (!re.test(line)) continue;
46
- if (excludes.some((ex) => ex.test(line))) continue; // pattern's own false-positive guards
47
- findings.push({ file, line: i + 1, id: pat.id, sev: pat.severity, msg: pat.message });
48
- }
49
- }
50
- }
51
-
52
- if (LIST) {
53
- for (const f of findings) console.error(` [${f.sev}] ${f.file}:${f.line} · ${f.id} · ${f.msg}`);
54
- console.error(`\n${findings.length} security-invariant violation(s)`);
55
- }
56
- // the metric value: lower is better, 0 is clean
57
- process.stdout.write(String(findings.length) + '\n');