shipclear 1.0.0 → 1.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -7,6 +7,7 @@
7
7
  Your AI writes code fast. ShipClear makes sure it's safe to ship —
8
8
  free, open source, runs 100% locally, works with every AI coding tool.
9
9
 
10
+ [![ShipClear](https://github.com/adamtimmerberg/ShipClear/actions/workflows/shipclear.yml/badge.svg?branch=main&event=push)](https://github.com/adamtimmerberg/ShipClear/actions/workflows/shipclear.yml)
10
11
  [![CI](https://github.com/adamtimmerberg/ShipClear/actions/workflows/ci.yml/badge.svg)](https://github.com/adamtimmerberg/ShipClear/actions/workflows/ci.yml)
11
12
  [![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
12
13
  [![Node >= 18](https://img.shields.io/badge/node-%3E%3D18-brightgreen.svg)](package.json)
@@ -32,7 +33,7 @@ npx shipclear ship
32
33
  That runs the full gate and gives you a verdict:
33
34
 
34
35
  ```
35
- ShipClear v1.0.0 — ship report
36
+ ShipClear v1.1.0 — ship report
36
37
 
37
38
  🛑 CRITICAL A secret key is written directly in your code
38
39
  where: server.js:7 — OpenAI API key: `sk-FAKE0…`
@@ -58,7 +59,7 @@ Most tools scan your app *after* you've made the mistake. ShipClear rides along
58
59
  |---|---|
59
60
  | 🧠 **Prevention** | Rules for your AI assistant so secrets never get hardcoded in the first place — `.env` from the first line of code, no test accounts, no real data in the repo. |
60
61
  | 🚫 **Enforcement** | A git commit guard (`shipclear setup`) that blocks any commit containing secrets or a `.env` file. Deterministic — it isn't the AI deciding to be careful. (If the scanner itself can't run — you're offline, say — it lets the commit through with a warning rather than blocking your work; it fails safe, not stuck.) |
61
- | ✅ **The gate** | `shipclear ship`: 13 deterministic checks + a semantic checklist for your AI, ending in one verdict — cleared to ship, or not. Found a secret in old commits? `shipclear ship --fix-history` erases that history safely (never-pushed repos only; it refuses if it would lose anything). |
62
+ | ✅ **The gate** | `shipclear ship`: 22 deterministic checks — across your code, your git history, *and* the config you deploy with (`.env`, docker-compose, proxy routes, bucket policies) — plus a semantic checklist for your AI, ending in one verdict — cleared to ship, or not. Found a secret in old commits? `shipclear ship --fix-history` erases that history safely (never-pushed repos only; it refuses if it would lose anything). |
62
63
 
63
64
  ## Works with every AI coding tool
64
65
 
@@ -85,15 +86,47 @@ Detection is tuned against real-world starter repos, with a false-positive regre
85
86
 
86
87
  Plus a ten-point semantic checklist your AI assistant runs — the checks a regex can't do: auth **and ownership** on every route, database rules, client-side leakage, injection/XSS/uploads, server-side trust (prices, roles, webhook signatures), session hygiene and rate limiting, debug surface, and an AI-surface review for apps that call LLMs themselves (prompt injection, unvalidated model output, overprivileged tools).
87
88
 
88
- Pass the gate and you'll get a badge for your README: ![ShipClear: cleared to ship](https://img.shields.io/badge/ShipClear-cleared%20to%20ship-brightgreen)
89
+ ## The badge can't lie
90
+
91
+ Pass the gate and `shipclear badge` adds a **cleared to ship** badge to your README — one command, and it maintains itself from then on:
92
+
93
+ ```bash
94
+ npx shipclear badge
95
+ ```
96
+
97
+ Most badges are a static green image: paste it once and it keeps vouching for every commit you make afterwards, checked or not. That's a security claim nobody verified, which is exactly what this tool exists to prevent — so ShipClear doesn't hand you one. Instead `shipclear badge` installs a GitHub Actions workflow that re-runs this gate on GitHub for **every push**, and a badge that shows that workflow's real result:
98
+
99
+ | | means |
100
+ |---|---|
101
+ | 🟢 **passing** | every commit pushed to this branch cleared the gate — including its git history, so an old leaked secret keeps it red until the history is actually cleaned |
102
+ | 🔴 **failing** | a commit landed that ShipClear would not clear |
103
+ | ⚪ **no status** | nothing has been checked, so nothing is claimed |
104
+
105
+ The workflow runs `shipclear ship --no-fix --strict`: `--no-fix` so the verdict describes the commit as pushed rather than a copy the job quietly repaired, and `--strict` so only a full CLEARED counts as green. Skipping the gate locally can't produce a green badge — only a real passing run on GitHub can. (`--strict` is useful in your own CI for the same reason.)
106
+
107
+ Deleting the workflow or turning off Actions doesn't fake a green badge either — the badge drops to "no status". Nothing stops someone from deliberately pointing the badge at a different workflow; it defends against forgetting, not fraud.
108
+
109
+ ## Checking the site you actually deployed
110
+
111
+ Some problems don't exist in your files. A storage bucket's permissions are applied at runtime; a missing response header is an absence; a proxy route's real behaviour is a behaviour. So the gate can also ask your live site:
112
+
113
+ ```bash
114
+ npx shipclear ship --probe https://your-app.com
115
+ ```
116
+
117
+ It reports what is genuinely reachable — an open `/metrics` or `/api/docs`, a storage folder that hands out a list of every file in it, a home page served without the headers that protect pages — and confirms (or quietly drops) the findings the file scan could only suspect.
118
+
119
+ The rules it works under: **`GET` only**, so nothing is written, changed or deleted; no logins, passwords, or attempts to get past anything; only the one address you type and paths beneath it; and it tells you all of that and waits for a yes before the first request. It's off unless you ask for it, and the AI adapters are instructed never to run it on your behalf. Only point it at a site you own.
89
120
 
90
121
  ## Privacy, by architecture
91
122
 
92
123
  ShipClear runs entirely on your machine. **No account. No server. No telemetry. Nothing is uploaded, ever.** It has zero runtime dependencies — what you install is what's in this repo, and CI enforces that by not having an install step.
93
124
 
125
+ The single exception is opt-in and explicit: `ship --probe <url>` makes read-only requests to the address you give it, after asking. It still sends nothing anywhere else, and no part of your code or report leaves your machine.
126
+
94
127
  ## What ShipClear can't do
95
128
 
96
- Honesty is a feature in a security tool. ShipClear does **not**: audit your business logic, test your deployed infrastructure, protect against DDoS, review your cloud console settings, or replace a professional security audit for high-stakes products. It also can't catch a secret format nobody has taught it — if you find one it misses, [that's a security bug and we treat it like one](SECURITY.md).
129
+ Honesty is a feature in a security tool. ShipClear does **not**: audit your business logic, protect against DDoS, review your cloud console settings, or replace a professional security audit for high-stakes products. It doesn't pen-test your deployment either — `--probe` makes a handful of read-only requests to pages that shouldn't be public, which is a long way from a penetration test. It also can't catch a secret format nobody has taught it — if you find one it misses, [that's a security bug and we treat it like one](SECURITY.md).
97
130
 
98
131
  ## Contributing
99
132
 
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "shipclear",
3
- "version": "1.0.0",
3
+ "version": "1.1.0",
4
4
  "description": "Security companion for AI-assisted coding: prevention rules applied to every task, a commit guard that blocks leaked secrets, and the /safe-to-ship gate.",
5
5
  "author": { "name": "Adam Timmerberg" },
6
6
  "homepage": "https://github.com/adamtimmerberg/ShipClear"
@@ -13,6 +13,7 @@ The user wants to know whether this project is safe to deploy or share publicly.
13
13
  1. Run the deterministic scan: `npx --yes shipclear ship` (it writes `SHIP-REPORT.md` and applies safe fixes automatically).
14
14
  2. Read `SHIP-REPORT.md` and walk the user through each finding in plain English.
15
15
  - **Never run `shipclear ship --fix-history` on the user's behalf.** It erases git history. If a finding recommends it, explain what it does, confirm the other findings are fixed first, and let the user run it — or run it only after they explicitly say to.
16
+ - **Never run `shipclear ship --probe <url>` unless the user asks for it and names the address.** It makes real (read-only) requests to a live site. Deciding which site may be contacted is the user's, not yours. You may *tell* them it exists — it confirms findings a file-based scan cannot see, like a storage folder anyone can list — and let them run it.
16
17
  3. Perform the semantic checks below yourself — these need code understanding and are your responsibility:
17
18
 
18
19
  1. **Hidden accounts:** search for any route, seed, migration, or conditional that grants access via a fixed credential, magic string, or special email — including ones the AI itself created during this project. Check `.shipclear/test-accounts.json` and confirm every listed credential is absent from code and the production database plan.
@@ -24,7 +25,12 @@ The user wants to know whether this project is safe to deploy or share publicly.
24
25
  7. **Session and abuse surface:** session tokens in httpOnly cookies rather than localStorage; rate limiting (or provider-equivalent) on login, signup, and password-reset endpoints.
25
26
  8. **Debug surface:** look for debug endpoints, verbose stack traces returned to users, `console.log` of sensitive values, and permissive CORS on authenticated APIs.
26
27
  9. **AI surface** (when the app itself calls LLMs or runs agents/MCP tools): treat user input and any external content entering prompts as untrusted; never execute, render as HTML, or write to the database unvalidated model output; give agent/MCP tools the narrowest permissions that work, and keep agents away from production resources during testing.
27
- 10. **Report merge:** append semantic findings to `SHIP-REPORT.md` in the same *What / Why / Fix* format, then restate the final verdict — a semantic critical downgrades the overall verdict to DO NOT SHIP.
28
+ 10. **Credentials actually in use:** every password, key and token in the live config differs from the value in the example/sample config and from the service's documented default. The app refuses to start in production if a placeholder remains.
29
+ 11. **File and object storage:** no bucket or container is publicly readable; nothing anonymous can **list** its contents. Confirm with a request to the bucket root, not just by reading config.
30
+ 12. **Proxy and ingress:** no public route forwards wholesale into a backing service (database, object store, queue, admin UI); routes are narrowed to the paths actually needed.
31
+ 13. **Outbound fetches:** if the server fetches URLs supplied by users (import-from-URL, link previews, webhooks, avatars), internal and cloud-metadata addresses are refused — and re-checked after every redirect.
32
+ 14. **Header coverage:** security headers apply to the responses that serve HTML, not only to the API.
33
+ 15. **Report merge:** append semantic findings to `SHIP-REPORT.md` in the same *What / Why / Fix* format, then restate the final verdict — a semantic critical downgrades the overall verdict to DO NOT SHIP.
28
34
 
29
35
  ## Verdict rules
30
36
 
@@ -42,6 +42,7 @@ These rules are for AI coding agents and assistants. Follow them on every task,
42
42
  1. Run the deterministic scan: `npx --yes shipclear ship` (it writes `SHIP-REPORT.md` and applies safe fixes automatically).
43
43
  2. Read `SHIP-REPORT.md` and walk the user through each finding in plain English.
44
44
  - **Never run `shipclear ship --fix-history` on the user's behalf.** It erases git history. If a finding recommends it, explain what it does, confirm the other findings are fixed first, and let the user run it — or run it only after they explicitly say to.
45
+ - **Never run `shipclear ship --probe <url>` unless the user asks for it and names the address.** It makes real (read-only) requests to a live site. Deciding which site may be contacted is the user's, not yours. You may *tell* them it exists — it confirms findings a file-based scan cannot see, like a storage folder anyone can list — and let them run it.
45
46
  3. Perform the semantic checks below yourself — these need code understanding and are your responsibility:
46
47
 
47
48
  1. **Hidden accounts:** search for any route, seed, migration, or conditional that grants access via a fixed credential, magic string, or special email — including ones the AI itself created during this project. Check `.shipclear/test-accounts.json` and confirm every listed credential is absent from code and the production database plan.
@@ -53,7 +54,12 @@ These rules are for AI coding agents and assistants. Follow them on every task,
53
54
  7. **Session and abuse surface:** session tokens in httpOnly cookies rather than localStorage; rate limiting (or provider-equivalent) on login, signup, and password-reset endpoints.
54
55
  8. **Debug surface:** look for debug endpoints, verbose stack traces returned to users, `console.log` of sensitive values, and permissive CORS on authenticated APIs.
55
56
  9. **AI surface** (when the app itself calls LLMs or runs agents/MCP tools): treat user input and any external content entering prompts as untrusted; never execute, render as HTML, or write to the database unvalidated model output; give agent/MCP tools the narrowest permissions that work, and keep agents away from production resources during testing.
56
- 10. **Report merge:** append semantic findings to `SHIP-REPORT.md` in the same *What / Why / Fix* format, then restate the final verdict — a semantic critical downgrades the overall verdict to DO NOT SHIP.
57
+ 10. **Credentials actually in use:** every password, key and token in the live config differs from the value in the example/sample config and from the service's documented default. The app refuses to start in production if a placeholder remains.
58
+ 11. **File and object storage:** no bucket or container is publicly readable; nothing anonymous can **list** its contents. Confirm with a request to the bucket root, not just by reading config.
59
+ 12. **Proxy and ingress:** no public route forwards wholesale into a backing service (database, object store, queue, admin UI); routes are narrowed to the paths actually needed.
60
+ 13. **Outbound fetches:** if the server fetches URLs supplied by users (import-from-URL, link previews, webhooks, avatars), internal and cloud-metadata addresses are refused — and re-checked after every redirect.
61
+ 14. **Header coverage:** security headers apply to the responses that serve HTML, not only to the API.
62
+ 15. **Report merge:** append semantic findings to `SHIP-REPORT.md` in the same *What / Why / Fix* format, then restate the final verdict — a semantic critical downgrades the overall verdict to DO NOT SHIP.
57
63
 
58
64
  ## Verdict rules
59
65
 
@@ -1,12 +1,15 @@
1
1
  #!/usr/bin/env node
2
2
  import fs from 'node:fs';
3
3
  import path from 'node:path';
4
+ import readline from 'node:readline/promises';
4
5
  import { fileURLToPath } from 'node:url';
5
6
  import { runGate } from '../lib/checks.js';
6
7
  import { applyFixes, resetHistory, installHook, envExampleFrom, GITIGNORE_BLOCK } from '../lib/fix.js';
7
8
  import { renderTerminal, renderMarkdown, verdictOf, VERDICT_TEXT } from '../lib/report.js';
8
9
  import { findSecrets, maskSecret, lineOfIndex } from '../lib/patterns.js';
9
10
  import { isGitRepo, stagedFiles, stagedContent } from '../lib/git.js';
11
+ import { installBadge } from '../lib/badge.js';
12
+ import { runProbe, normalizeBase } from '../lib/probe.js';
10
13
  import { loadIgnore } from '../lib/ignore.js';
11
14
 
12
15
  const pkg = JSON.parse(
@@ -22,12 +25,26 @@ Usage:
22
25
  shipclear scan --staged Scan only what's about to be committed (used by the guard)
23
26
  shipclear ship Full safe-to-ship gate → SHIP-REPORT.md + verdict
24
27
  shipclear ship --no-fix Same, but don't apply automatic fixes
28
+ shipclear ship --strict Same, but treat ANY finding as a failure (exit 1 unless
29
+ the verdict is CLEARED) — for CI and the README badge
30
+ shipclear ship --probe https://your-app.com
31
+ Also ask your deployed site what is actually
32
+ reachable: open debug pages, a storage folder anyone
33
+ can list, missing page headers. Read-only (GET
34
+ only), asks you to confirm first, and only ever
35
+ touches the address you type. Add --yes to skip the
36
+ confirmation (for scripts).
25
37
  shipclear ship --fix-history
26
38
  Also erase git history to remove a leaked secret from
27
39
  old commits (only when nothing was ever pushed; refuses
28
40
  if it would lose branches or stashes)
41
+ shipclear badge Add a self-updating "cleared to ship" badge to your
42
+ README, backed by a GitHub Actions run of this gate on
43
+ every push — so it turns red if an unchecked or failing
44
+ commit lands
29
45
 
30
- Everything runs locally. Nothing is uploaded, ever.
46
+ Everything runs locally and nothing is ever uploaded. The one exception is
47
+ opt-in: --probe makes read-only requests to the single address you give it.
31
48
  Docs: https://github.com/adamtimmerberg/ShipClear
32
49
  `;
33
50
 
@@ -114,7 +131,34 @@ function cmdScan(root) {
114
131
  return findings.some((f) => f.severity === 'critical') ? 1 : 0;
115
132
  }
116
133
 
117
- function cmdShip(root, { fix = true, fixHistory = false } = {}) {
134
+ // The probe reaches out to the network, which nothing else in ShipClear does.
135
+ // So it is opt-in per run, it says exactly what it will do, and it asks —
136
+ // unless the user passed --yes, or there is no terminal to ask in (a CI job),
137
+ // where it declines rather than guessing.
138
+ async function confirmProbe(url) {
139
+ const { base, error } = normalizeBase(url);
140
+ if (error) {
141
+ console.error(`\n ${error}\n`);
142
+ return null;
143
+ }
144
+ if (!process.stdin.isTTY) {
145
+ return { base, confirmed: false, why: 'there is no terminal here to confirm in — re-run it yourself, or add --yes if you are scripting this' };
146
+ }
147
+ console.log('');
148
+ console.log(` --probe will make read-only requests to ${base}:`);
149
+ console.log(' • GET only. Nothing is written, changed or deleted.');
150
+ console.log(' • No logins, no passwords, no attempts to get past anything.');
151
+ console.log(' • Only that address and paths under it — nowhere else.');
152
+ console.log(' • It checks for open debug pages, a storage folder anyone can list,');
153
+ console.log(' and whether your home page sends the headers that protect pages.');
154
+ console.log(' Only do this against a site you own.');
155
+ const rl = readline.createInterface({ input: process.stdin, output: process.stdout });
156
+ const answer = await rl.question(`\n Probe ${base}? (y/N) `);
157
+ rl.close();
158
+ return { base, confirmed: /^y(es)?$/i.test(answer.trim()), why: 'you answered no' };
159
+ }
160
+
161
+ async function cmdShip(root, { fix = true, fixHistory = false, strict = false, probeUrl = null, assumeYes = false } = {}) {
118
162
  let { findings, notes } = runGate(root);
119
163
  const fixes = fix ? applyFixes(root, findings) : [];
120
164
 
@@ -130,13 +174,68 @@ function cmdShip(root, { fix = true, fixHistory = false } = {}) {
130
174
  }
131
175
  }
132
176
 
177
+ if (probeUrl) {
178
+ const decision = await confirmProbe(probeUrl);
179
+ if (decision === null) {
180
+ notes.push('--probe was not run: that address could not be read.');
181
+ } else if (!decision.confirmed && !assumeYes) {
182
+ notes.push(`--probe was not run: ${decision.why}. Everything above is from your files alone.`);
183
+ } else {
184
+ const probed = await runProbe(decision.base, findings);
185
+ findings.push(...probed.findings);
186
+ notes.push(...probed.notes);
187
+ }
188
+ }
189
+
133
190
  const verdict = verdictOf(findings);
191
+ // Strict mode exists for the README badge and CI: a badge may only be
192
+ // green on a full CLEARED, so "SHIP WITH FIXES" — which exits 0 normally,
193
+ // because a human can read it and decide — has to fail here.
194
+ if (strict && verdict !== 'CLEARED') {
195
+ notes.push('--strict: failing because the verdict is not CLEARED. Fix the findings above; a ShipClear badge stays red until they are gone.');
196
+ }
134
197
  const result = { findings, notes, verdict, fixes, version: pkg.version };
135
198
  fs.writeFileSync(path.join(root, 'SHIP-REPORT.md'), renderMarkdown(result));
136
199
  console.log(renderTerminal(result));
200
+ if (strict) return verdict === 'CLEARED' ? 0 : 1;
137
201
  return verdict === 'DO_NOT_SHIP' ? 1 : 0;
138
202
  }
139
203
 
204
+ function cmdBadge(root) {
205
+ if (!isGitRepo(root)) {
206
+ console.error("\n This folder isn't set up with git yet, so there's no GitHub repo for a badge to report on.");
207
+ console.error(' Run `git init` here, push the project to GitHub, then run `shipclear badge` again.\n');
208
+ return 1;
209
+ }
210
+ const result = installBadge(root);
211
+ if (!result.ok) {
212
+ console.error(`\n ShipClear can't add the badge yet: ${result.reason}\n`);
213
+ return 1;
214
+ }
215
+ console.log('\n ShipClear badge installed.');
216
+ for (const d of result.done) console.log(` ✔ ${d}`);
217
+ for (const w of result.warnings) console.log(` ⚠ ${w}`);
218
+ console.log('\n Commit and push both files to switch it on:');
219
+ console.log(' git add -A && git commit -m "add ShipClear badge" && git push');
220
+ // The badge reads "no status" until the workflow has run once. Saying so
221
+ // here stops that grey badge from looking like a broken install.
222
+ console.log(`\n It shows "no status" until that first push runs the workflow — grey because`);
223
+ console.log(' nothing has been checked yet, which is the honest answer at that point.');
224
+ console.log(` Then it tracks the \`${result.branch}\` branch: green while every pushed commit`);
225
+ console.log(' clears this gate, red the moment one does not.\n');
226
+ return 0;
227
+ }
228
+
229
+ /** Value of a `--flag value` or `--flag=value` argument, or null. */
230
+ function flagValue(argv, flag) {
231
+ const inline = argv.find((a) => a.startsWith(`${flag}=`));
232
+ if (inline) return inline.slice(flag.length + 1) || null;
233
+ const i = argv.indexOf(flag);
234
+ if (i === -1) return null;
235
+ const next = argv[i + 1];
236
+ return next && !next.startsWith('-') ? next : null;
237
+ }
238
+
140
239
  const args = process.argv.slice(2);
141
240
  const cmd = args[0];
142
241
  const root = process.cwd();
@@ -156,9 +255,18 @@ try {
156
255
  const fixHistory = args.includes('--fix-history');
157
256
  // A history reset needs .gitignore in place first so the fresh commit
158
257
  // doesn't re-add .env — so --fix-history implies the additive fixes.
159
- exitCode = cmdShip(root, { fix: !args.includes('--no-fix') || fixHistory, fixHistory });
258
+ exitCode = await cmdShip(root, {
259
+ fix: !args.includes('--no-fix') || fixHistory,
260
+ fixHistory,
261
+ strict: args.includes('--strict'),
262
+ probeUrl: flagValue(args, '--probe'),
263
+ assumeYes: args.includes('--yes'),
264
+ });
160
265
  break;
161
266
  }
267
+ case 'badge':
268
+ exitCode = cmdBadge(root);
269
+ break;
162
270
  case '--version':
163
271
  case '-v':
164
272
  console.log(pkg.version);
@@ -0,0 +1,216 @@
1
+ // The README badge.
2
+ //
3
+ // A badge is a security claim, so it is held to the same standard as a
4
+ // verdict: it must never say "cleared" for code nobody checked. That rules
5
+ // out the obvious implementation. A static image —
6
+ // `shields.io/badge/ShipClear-cleared%20to%20ship-brightgreen` — is a
7
+ // hardcoded green picture: paste it once and it keeps making the claim on
8
+ // every later commit, whether or not the gate ever ran again. That is a
9
+ // false GO, the one failure mode this product exists to prevent
10
+ // (CONTRIBUTING.md), and ShipClear used to hand it to users itself.
11
+ //
12
+ // The honest version has to be computed by something that runs on every
13
+ // push and that the author cannot skip by forgetting. On GitHub that is
14
+ // GitHub Actions. So `shipclear badge` installs a workflow whose only job
15
+ // is the verdict, and a badge that renders that workflow's real status:
16
+ //
17
+ // green — the latest push to this branch cleared the gate
18
+ // red — it did not clear
19
+ // grey — nothing ran ("no status"), so nothing is claimed
20
+ //
21
+ // Two flags in the workflow carry the whole guarantee:
22
+ // --no-fix the verdict must describe the commit as pushed, not a copy
23
+ // the job quietly repaired on its way past (plain `ship` would
24
+ // add the missing .gitignore line itself and then report clean)
25
+ // --strict only a CLEARED verdict may show green (plain `ship` exits 0
26
+ // on SHIP WITH FIXES, which would paint HIGH findings green)
27
+ //
28
+ // "Every commit was cleared" holds rather than just "the last one did",
29
+ // because the gate reads git history: a secret committed at any point keeps
30
+ // failing at HEAD until the history is actually cleaned.
31
+ import fs from 'node:fs';
32
+ import path from 'node:path';
33
+ import { run } from './git.js';
34
+
35
+ export const WORKFLOW_REL = '.github/workflows/shipclear.yml';
36
+
37
+ // The release that introduced `--strict`. The workflow must pin at least
38
+ // this, because unknown flags are silently ignored: run `ship --no-fix
39
+ // --strict` on 1.0.0 and you get plain `ship --no-fix`, which exits 0 on
40
+ // SHIP WITH FIXES — a green badge for a repo with HIGH findings. Verified
41
+ // against the published 1.0.0, not assumed. A caret range still picks up
42
+ // later 1.x releases, so the badge keeps reflecting current knowledge.
43
+ export const MIN_CLI_VERSION = '1.1.0';
44
+
45
+ /** owner/repo from the origin remote, or null if it isn't a GitHub remote. */
46
+ export function githubSlug(root) {
47
+ let url;
48
+ try {
49
+ url = run(root, ['remote', 'get-url', 'origin']).trim();
50
+ } catch {
51
+ return null;
52
+ }
53
+ // https://github.com/o/r(.git) · git@github.com:o/r(.git) · ssh://git@github.com/o/r
54
+ const m = url.match(/github\.com[:/]+([^/]+)\/([^/]+?)(?:\.git)?\/?$/i);
55
+ if (!m) return null;
56
+ return { owner: m[1], repo: m[2] };
57
+ }
58
+
59
+ export function currentBranch(root) {
60
+ try {
61
+ const b = run(root, ['rev-parse', '--abbrev-ref', 'HEAD']).trim();
62
+ return b && b !== 'HEAD' ? b : 'main';
63
+ } catch {
64
+ return 'main';
65
+ }
66
+ }
67
+
68
+ // Served by GitHub itself rather than a third-party image host, so the
69
+ // authority on whether the workflow passed is also the thing drawing the
70
+ // picture. `event=push` pins it to pushed code, so a green pull-request run
71
+ // can't speak for the branch; the label is the workflow's own name.
72
+ export function badgeMarkdown({ owner, repo, branch }) {
73
+ const wf = `https://github.com/${owner}/${repo}/actions/workflows/shipclear.yml`;
74
+ return `[![ShipClear](${wf}/badge.svg?branch=${encodeURIComponent(branch)}&event=push)](${wf})`;
75
+ }
76
+
77
+ export function workflowYaml() {
78
+ return `# Keeps the ShipClear badge in this README honest.
79
+ #
80
+ # Re-runs the gate on GitHub for every push, so the badge reflects the code
81
+ # that is actually on GitHub — not whatever was on someone's laptop the last
82
+ # time they remembered to check. A commit ShipClear won't clear fails this
83
+ # job, and the badge turns red.
84
+ #
85
+ # --no-fix: judge the commit as pushed. Without it the job would apply the
86
+ # safe fixes to its own checkout and report the repaired copy.
87
+ # --strict: only a clean CLEARED verdict passes. Without it "SHIP WITH
88
+ # FIXES" would exit 0 and show green.
89
+ name: ShipClear
90
+
91
+ on:
92
+ push:
93
+ pull_request:
94
+ workflow_dispatch:
95
+
96
+ permissions:
97
+ contents: read
98
+
99
+ jobs:
100
+ gate:
101
+ runs-on: ubuntu-latest
102
+ steps:
103
+ - uses: actions/checkout@v4
104
+ with:
105
+ fetch-depth: 0 # the gate scans your whole git history
106
+ - uses: actions/setup-node@v4
107
+ with:
108
+ node-version: 20
109
+ # Pinned at or above the release that added --strict: an older CLI
110
+ # ignores the flag silently and would exit 0 on findings. The caret
111
+ # still takes later 1.x releases, so a security badge keeps
112
+ # reflecting what is known to be unsafe today.
113
+ - run: npx --yes shipclear@^${MIN_CLI_VERSION} ship --no-fix --strict
114
+ `;
115
+ }
116
+
117
+ const IMAGE_MD = /\[?!\[([^\]]*)\]\(([^)]*)\)(?:\]\([^)]*\))?/g;
118
+
119
+ // Which images are *our* badge. This has to be exact, not a "shipclear"
120
+ // substring search: in this very repo every badge URL contains
121
+ // github.com/adamtimmerberg/ShipClear, so a loose match replaced the CI
122
+ // badge instead (caught by running this command on its own README). Any
123
+ // user whose repo is named shipclear-something would have hit the same
124
+ // thing. Match the badge endpoint or the retired static badge, or an alt
125
+ // text that opens with the product name.
126
+ function isOurBadge(whole, alt, url) {
127
+ return /\/actions\/workflows\/shipclear\.yml\/badge\.svg/i.test(url)
128
+ || /shields\.io\/badge\/ShipClear/i.test(url)
129
+ || /^\s*ShipClear\b/i.test(alt);
130
+ }
131
+
132
+ /** A line that is nothing but badges — the row to join if one exists. */
133
+ function isBadgeRow(line) {
134
+ const stripped = line.replace(IMAGE_MD, '').trim();
135
+ return stripped === '' && line.trim() !== '';
136
+ }
137
+
138
+ /**
139
+ * Put `badge` in the README: replacing any badge already there (including
140
+ * the old static image), else joining the badge row, else after the H1.
141
+ * Pure string work so the placement is testable without a repo.
142
+ */
143
+ export function withBadge(readme, badge) {
144
+ for (const m of readme.matchAll(IMAGE_MD)) {
145
+ if (!isOurBadge(m[0], m[1], m[2])) continue;
146
+ if (m[0] === badge) return { text: readme, changed: false, how: 'already current' };
147
+ return {
148
+ text: readme.slice(0, m.index) + badge + readme.slice(m.index + m[0].length),
149
+ changed: true,
150
+ how: 'replaced the old badge',
151
+ };
152
+ }
153
+ const lines = readme.split('\n');
154
+ const row = lines.findIndex(isBadgeRow);
155
+ if (row !== -1) {
156
+ lines[row] = `${lines[row].trimEnd()} ${badge}`;
157
+ return { text: lines.join('\n'), changed: true, how: 'added it to your badge row' };
158
+ }
159
+ const h1 = lines.findIndex((l) => /^#\s+\S/.test(l));
160
+ if (h1 !== -1) {
161
+ lines.splice(h1 + 1, 0, '', badge);
162
+ return { text: lines.join('\n'), changed: true, how: 'added it under your title' };
163
+ }
164
+ return { text: [badge, '', readme].join('\n'), changed: true, how: 'added it at the top' };
165
+ }
166
+
167
+ /**
168
+ * Install the workflow and the badge. Returns { ok:false, reason } when the
169
+ * repo has nowhere for a live badge to point.
170
+ */
171
+ export function installBadge(root) {
172
+ const slug = githubSlug(root);
173
+ if (!slug) {
174
+ return {
175
+ ok: false,
176
+ reason: "this project has no GitHub remote, so there's nothing to publish a live badge from. Push it to GitHub first (create an empty repo there, then `git remote add origin <its URL>` and `git push -u origin HEAD`), then run `shipclear badge` again.",
177
+ };
178
+ }
179
+ const branch = currentBranch(root);
180
+ const badge = badgeMarkdown({ ...slug, branch });
181
+ const done = [];
182
+ const warnings = [];
183
+
184
+ const wfPath = path.join(root, WORKFLOW_REL);
185
+ if (!fs.existsSync(wfPath)) {
186
+ fs.mkdirSync(path.dirname(wfPath), { recursive: true });
187
+ fs.writeFileSync(wfPath, workflowYaml());
188
+ done.push(`created ${WORKFLOW_REL} — it re-runs the gate on GitHub for every push, which is what keeps the badge honest`);
189
+ } else {
190
+ const existing = fs.readFileSync(wfPath, 'utf8');
191
+ // Never silently overwrite someone's workflow — but a badge pointing at
192
+ // a workflow missing these flags can show green for a repo that didn't
193
+ // clear, so say so rather than leaving a quiet lie in place.
194
+ if (!/--strict/.test(existing) || !/--no-fix/.test(existing)) {
195
+ warnings.push(`${WORKFLOW_REL} already exists, so ShipClear left it alone — but it doesn't run \`shipclear ship --no-fix --strict\`, so the badge can show green for a repo that only partly cleared. Add those flags to that file.`);
196
+ } else {
197
+ done.push(`${WORKFLOW_REL} was already in place`);
198
+ }
199
+ }
200
+
201
+ const readmePath = ['README.md', 'readme.md', 'README.markdown']
202
+ .map((n) => path.join(root, n))
203
+ .find((p) => fs.existsSync(p));
204
+ if (!readmePath) {
205
+ warnings.push(`no README.md found in this folder, so there was nothing to add the badge to. Create one and paste this line into it:\n${badge}`);
206
+ } else {
207
+ const before = fs.readFileSync(readmePath, 'utf8');
208
+ const { text, changed, how } = withBadge(before, badge);
209
+ if (changed) fs.writeFileSync(readmePath, text);
210
+ done.push(changed
211
+ ? `${path.basename(readmePath)} — ${how}`
212
+ : `${path.basename(readmePath)} already has the current badge`);
213
+ }
214
+
215
+ return { ok: true, done, warnings, badge, branch, slug };
216
+ }
package/cli/lib/checks.js CHANGED
@@ -8,6 +8,7 @@ import {
8
8
  personalPaths, TLS_VERIFICATION_OFF, DEBUG_MODE_ON,
9
9
  } from './patterns.js';
10
10
  import { loadIgnore } from './ignore.js';
11
+ import { runInfraChecks } from './infra.js';
11
12
  import { walkFiles, isTextCandidate, readFileSafe } from './scan.js';
12
13
  import { isGitRepo, hasCommits, hasRemote, branchNames, trackedFiles, historyAddedLines } from './git.js';
13
14
 
@@ -28,6 +29,9 @@ const isEnvTemplate = (rel) => ENV_TEMPLATE.test(path.posix.basename(rel));
28
29
  const isKeyFile = (rel) => KEY_FILE.test(rel);
29
30
  const isDataFile = (rel) => DATA_FILE_EXT.has(path.extname(rel).toLowerCase());
30
31
  const isDocFile = (rel) => DOC_EXT.has(path.extname(rel).toLowerCase());
32
+ // A database password in a compose/k8s file needs an extra rotation step that
33
+ // fails silently if missed — see containerDbNote in report.js.
34
+ const isContainerConfig = (rel) => /^(?:docker-)?compose(?:[.-].+)?\.ya?ml$|\.ya?ml$/i.test(path.posix.basename(rel));
31
35
 
32
36
  /**
33
37
  * Run the deterministic layer of the gate. Returns { findings, notes, inGit }.
@@ -56,7 +60,7 @@ export function runGate(root, { quick = false } = {}) {
56
60
  const content = read(rel);
57
61
  if (content === null) continue;
58
62
 
59
- for (const hit of findSecrets(content, { includeGeneric: !isDocFile(rel) })) {
63
+ for (const hit of findSecrets(content, { includeGeneric: !isDocFile(rel), docFile: isDocFile(rel) })) {
60
64
  seenSecretValues.add(hit.match);
61
65
  findings.push({
62
66
  id: hit.patternId === 'generic-secret' ? 'generic-secret' : 'secret-in-code',
@@ -64,6 +68,8 @@ export function runGate(root, { quick = false } = {}) {
64
68
  file: rel,
65
69
  line: lineOfIndex(content, hit.index),
66
70
  detail: `${hit.name}: \`${maskSecret(hit.value ?? hit.match)}\``,
71
+ dbInContainer: isContainerConfig(rel) && /volumes\s*:/.test(content)
72
+ && /postgres|mysql|mariadb|mongo|redis/i.test(hit.name + hit.match),
67
73
  });
68
74
  }
69
75
 
@@ -226,7 +232,7 @@ export function runGate(root, { quick = false } = {}) {
226
232
  const byLocation = new Map();
227
233
  for (const { commit, file, text } of historyAddedLines(root)) {
228
234
  if (ignored(file)) continue;
229
- for (const hit of findSecrets(text, { includeGeneric: false })) {
235
+ for (const hit of findSecrets(text, { includeGeneric: false, docFile: isDocFile(file) })) {
230
236
  if (seen.has(hit.match)) continue;
231
237
  seen.add(hit.match);
232
238
  // Already reported from the working tree — that finding's advice
@@ -257,6 +263,11 @@ export function runGate(root, { quick = false } = {}) {
257
263
  }
258
264
  }
259
265
 
266
+ // --- how this project is deployed: live credentials, storage, proxy routes ---
267
+ // Walks the working tree rather than git's index on purpose — the live
268
+ // `.env` is the subject of the first of these checks and is never tracked.
269
+ findings.push(...runInfraChecks(root, { ignored }));
270
+
260
271
  // --- generated test logins exist; the rules promise a launch-time reminder ---
261
272
  if (fs.existsSync(path.join(root, '.shipclear', 'test-accounts.json'))) {
262
273
  findings.push({ id: 'test-registry-present', severity: 'info', file: '.shipclear/test-accounts.json' });
package/cli/lib/git.js CHANGED
@@ -86,7 +86,15 @@ export function stagedContent(root, file) {
86
86
 
87
87
  /**
88
88
  * Every line ever *added* across the full history of all branches, so a
89
- * secret that was committed and later deleted is still found.
89
+ * secret that was committed and later deleted is still found. Added lines are
90
+ * grouped per commit+file, which is also how findings are reported.
91
+ *
92
+ * The grouping is load-bearing, not tidiness: handing detectors one line at a
93
+ * time makes any pattern that needs more than one line silently blind in
94
+ * history. Private key material is exactly that shape — a PEM header is only
95
+ * a leak when its base64 body follows — so a per-line feed would have found
96
+ * every leaked key file in the working tree and none in the history, which is
97
+ * the half that matters once a secret has been deleted.
90
98
  * Returns [{ commit, file, text }].
91
99
  */
92
100
  export function historyAddedLines(root) {
@@ -94,7 +102,8 @@ export function historyAddedLines(root) {
94
102
  'log', '--all', '-p', '--unified=0', '--no-color',
95
103
  '--pretty=format:@@shipclear-commit@@ %h',
96
104
  ]);
97
- const added = [];
105
+ const blocks = new Map();
106
+ const order = [];
98
107
  let commit = null;
99
108
  let file = null;
100
109
  for (const line of out.split('\n')) {
@@ -105,8 +114,16 @@ export function historyAddedLines(root) {
105
114
  } else if (line.startsWith('+++ /dev/null')) {
106
115
  file = null;
107
116
  } else if (line.startsWith('+') && !line.startsWith('+++') && file) {
108
- added.push({ commit, file, text: line.slice(1) });
117
+ const key = `${commit}\u0000${file}`;
118
+ if (!blocks.has(key)) {
119
+ blocks.set(key, { commit, file, lines: [] });
120
+ order.push(key);
121
+ }
122
+ blocks.get(key).lines.push(line.slice(1));
109
123
  }
110
124
  }
111
- return added;
125
+ return order.map((key) => {
126
+ const b = blocks.get(key);
127
+ return { commit: b.commit, file: b.file, text: b.lines.join('\n') };
128
+ });
112
129
  }