shipclear 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,316 @@
1
+ // Every finding gets three plain-English answers: what is this, why does it
2
+ // matter, what do I do. If an explanation needs jargon, it gets rewritten.
3
+ //
4
+ // Fix texts that name a file are functions of the finding, so the reader
5
+ // gets `git rm --cached signups.csv`, never `git rm --cached <file>` — an
6
+ // independent test showed that placeholder alone is enough to stall a
7
+ // first-time user. Every git step is a literal command: "commit" is not an
8
+ // instruction to someone who has never run git.
9
+ const untrackThenCommit = (file, what) =>
10
+ `Run \`git rm --cached ${file}\` — that only stops git tracking it; the file stays on your machine (git will print \`rm '${file}'\`, which looks like a deletion but isn't). Then save that change: \`git commit -m "stop tracking ${what}"\`.`;
11
+
12
+ const dbUrlNote = ' (For a database URL, "the service" is wherever your database is hosted — change its password there.)';
13
+
14
+ const EXPLAIN = {
15
+ 'secret-in-code': {
16
+ title: 'A secret key is written directly in your code',
17
+ why: 'Anyone who sees this code — on GitHub, in a screenshot, in your deployed app — can use this key as you, and you get the bill.',
18
+ fix: 'Move the value into your .env file and read it from an environment variable (`process.env.YOUR_KEY_NAME` in JavaScript, `os.environ["YOUR_KEY_NAME"]` in Python). Most frameworks (Next.js, Create React App, Django, Rails, and others) load .env automatically; plain Node.js does not — add `require(\'dotenv\').config()` at the top of your entry file (`npm install dotenv` first), or on Node 20+ run your app with `node --env-file=.env yourapp.js`. Then rotate the key: log in to the provider and generate a new one, because this one is burned.' + dbUrlNote,
19
+ },
20
+ 'secret-in-history': {
21
+ title: 'A secret exists in your git history',
22
+ why: 'Deleting a key from your code does not delete it from git — every past version stays downloadable. If this repo goes public, the old key goes public with it.',
23
+ // Varies by repo state. A static "use filter-repo or BFG" pointer was
24
+ // the single thing that stopped the first independent test from ever
25
+ // reaching a clean verdict; the second found the gaps in the hand-typed
26
+ // path; the third found that running the reset *first* bakes every
27
+ // other open finding into the new root commit — hence "do this last".
28
+ fix: (f) => {
29
+ const seeIt = `(Want to see exactly what leaked? Run \`git show ${f.commit || '<commit>'} -- ${f.file || '<file>'}\`.)`;
30
+ const rotate = `Step 1 (does the actual protecting): rotate the key now — log in to whichever service issued it and generate a new one.${dbUrlNote}`;
31
+ if (f.hasRemote) {
32
+ const file = f.file || '<file>';
33
+ // A whole-secrets file (.env, a key file) should vanish from history.
34
+ // A secret *inside* a code file must be scrubbed while the file
35
+ // stays — `--invert-paths` on bot.js would delete the user's program.
36
+ const wholeFile = /(^|\/)\.env(\..+)?$|\.(pem|p12|pfx|ppk)$|(^|\/)id_(rsa|ed25519|ecdsa|dsa)(\.pub)?$/.test(file);
37
+ const askAi = wholeFile
38
+ ? `"My git history contains a leaked secrets file, ${file}; remove that file from every commit with git filter-repo and force-push the cleaned history to origin."`
39
+ : `"My git history contains a leaked API key inside ${file}; scrub that value from every commit with git filter-repo --replace-text (keep the file), then force-push the cleaned history to origin."`;
40
+ const scrub = wholeFile
41
+ ? `run \`git filter-repo --path ${file} --invert-paths --force\` (removes that file from every commit — it's git-ignored now, so it won't come back)`
42
+ : `make a plain text file called \`secrets-to-remove.txt\` containing just the leaked value on one line (copy it from the \`git show\` command below), run \`git filter-repo --replace-text secrets-to-remove.txt --force\` (replaces that value with \`***REMOVED***\` in every commit; the file itself stays), then delete \`secrets-to-remove.txt\``;
43
+ const branch = f.branch || 'main';
44
+ // Multi-line: rendered as numbered steps (see the renderers). The
45
+ // single-paragraph version of this was the #1 readability complaint
46
+ // from two independent tests.
47
+ return [
48
+ rotate,
49
+ "Once the key is rotated, the copy in history is useless to anyone — so if the next part gets hard, you are already safe; it's cleanup. (GitHub can keep an old commit reachable by its ID for a while, which is exactly why rotating is the step that matters.)",
50
+ `Step 2 (cleanup): the old commit is also on the remote (your copy on GitHub), so erasing local history isn't enough. Two routes — pick one:`,
51
+ `A) Easiest: ask the AI assistant you built this with. Paste it: ${askAi}`,
52
+ `B) Do it yourself: install git filter-repo (it isn't part of git — try \`pipx install git-filter-repo\`, or \`pip install --user git-filter-repo\`, or \`brew install git-filter-repo\` on a Mac; if pip says "externally-managed-environment", use pipx, or just use route A — it needs none of this). Confirm with \`git filter-repo --version\` (any version number or short code means it's installed).`,
53
+ ` Then: ${scrub}.`,
54
+ ` Then re-add your remote (filter-repo removes it as a safety measure): \`git remote add origin <your repo url>\`, and \`git push --force origin ${branch}\` (this overwrites the copy on GitHub — fine for a repo only you work on; if anyone else cloned it, tell them first).`,
55
+ ` Finally: \`shipclear ship\` again to confirm.`,
56
+ `(BFG Repo-Cleaner is an alternative — needs Java, works on a separate "mirror" copy; if it prints a wall of text, the useful line starts with "Caused by:". It cleans GitHub but not your own folder, so finish with \`git fetch && git reset --hard origin/${branch}\` in your project, then \`shipclear ship\` again.)`,
57
+ seeIt,
58
+ ].join('\n');
59
+ }
60
+ return [
61
+ rotate,
62
+ "Step 2 (cleanup, and the easy one): this repo has never been pushed anywhere (no remote), so the fix is to erase git's memory and start fresh — every file stays exactly as it is, only old commits are forgotten.",
63
+ 'Do this LAST, after every other finding above is fixed — a fresh start saves your project exactly as it is right now, problems included.',
64
+ 'Easiest and safest: run `shipclear ship --fix-history` and ShipClear does it for you, in this folder, after checking it won\'t lose any other saved work.',
65
+ 'To do it by hand instead: `rm -rf .git && git init && git add -A && git commit -m "start"` (Mac/Linux; on Windows delete the `.git` folder first, then run the rest) — if git says "Please tell me who you are," run the two `git config` lines it shows you and repeat the commit.',
66
+ seeIt,
67
+ ].join('\n');
68
+ },
69
+ },
70
+ 'env-tracked': {
71
+ title: 'Your .env file is checked into git',
72
+ why: 'The .env file is your box of secrets. Committing it means every secret inside ships with the repo, forever, to everyone who can see it.',
73
+ fix: (f) => `This was saved into git before ShipClear's protection was in place — setup stops it happening again, but can't undo the past. ${untrackThenCommit(f.file || '.env', 'secrets file')} Then rotate every key inside it — log in to each service and generate new ones.${dbUrlNote} One more thing: because it was committed, its contents are also in your git history — that's the separate "secret exists in your git history" finding, and \`shipclear ship --fix-history\` clears it once everything else is fixed.`,
74
+ },
75
+ 'key-file-tracked': {
76
+ title: 'A private key file is checked into git',
77
+ why: 'Private key files (.pem, id_rsa and friends) are the literal keys to servers and services. Anyone with the repo can log in as you.',
78
+ fix: (f) => `${untrackThenCommit(f.file || '<file>', 'key file')} Add its name (\`${f.file || '<file>'}\`) on its own line in .gitignore so it can't be re-added. Then replace the key itself: go back to wherever it came from (AWS, Firebase, your hosting provider, an SSH keygen) and create a new one — the committed one can never be trusted again, even after it's untracked.`,
79
+ },
80
+ 'env-example-leak': {
81
+ title: '.env.example contains a real-looking value',
82
+ why: '.env.example is meant to be shared — it shows which settings exist, with the values blank. A real value here is a secret published on purpose.',
83
+ fix: 'Replace the value with nothing (KEY=) or an obvious placeholder. If the value was real, rotate it — log in to whichever service issued it and generate a new one.',
84
+ },
85
+ 'gitignore-incomplete': {
86
+ title: '.env files are not protected by .gitignore',
87
+ why: 'Without this, one ordinary `git add .` commits your secrets. This is the single most common way keys leak.',
88
+ fix: 'Add a line with just `.env` (and `.env.*`) to your .gitignore file — or run `shipclear ship` again without `--no-fix` and ShipClear will add it for you.',
89
+ },
90
+ 'pii-data-file': {
91
+ title: 'A data file with real-looking personal information is in the repo',
92
+ why: 'Emails, SSNs, and customer records in a repo become public the moment the repo does. That is a privacy breach, and in many places a legal one.',
93
+ fix: (f) => `${untrackThenCommit(f.file || '<file>', 'data file')} Add its name (\`${f.file || '<file>'}\`) on its own line in .gitignore so it can't be re-added. If this is real data your app needs, that's fine — it stays on the machine where the app runs, just not in the repo (and if it's a list of people, it probably belongs in a database with proper access controls rather than a file). For testing and demos, use made-up data.`,
94
+ },
95
+ 'test-account': {
96
+ title: 'A login is hardcoded in the app',
97
+ why: 'AI assistants often create accounts like admin@… "just for testing". If it ships, anyone who reads the code can log in to your live app with it.',
98
+ fix: 'Delete the lines that define the credentials AND the lines that check against them — leaving the check behind makes login crash instead of merely not working. Then ask the AI assistant you built this with to replace it: tell it "replace the hardcoded login with a real login system." (If you want the names: Supabase Auth, Clerk, Auth0, and NextAuth are common choices — but you don\'t need to pick; your AI can.) If tests need an account, create it with random values at test time and keep them out of git.',
99
+ },
100
+ 'dependency-vulns': {
101
+ title: 'Dependencies have known security holes',
102
+ why: 'These are published vulnerabilities with public write-ups — attackers scan for them automatically.',
103
+ fix: 'Run `npm audit fix` and re-test the app. For anything it can\'t fix, `npm audit` names the package so you can update or replace it.',
104
+ },
105
+ 'generic-secret': {
106
+ title: 'A value that looks like a credential is hardcoded',
107
+ why: 'Something named like a key, token, or password is assigned directly in the code. If it\'s real, it leaks with the repo.',
108
+ fix: 'If it\'s a real credential: move it to .env and rotate it (log in to whichever service issued it and generate a new one). If it\'s a harmless placeholder, make it look like one (empty, or "REPLACE_ME") so scanners stay calm — or add the file to .shipclearignore.',
109
+ },
110
+ 'debug-mode-on': {
111
+ title: 'Debug mode is switched on',
112
+ why: 'With debug mode on, anyone who triggers an error in your live app gets an interactive Python console in their browser — which is full control of your server. It is meant only for your own machine while building.',
113
+ fix: 'This is fine while you build locally, but it must be off in the version that runs for real. Remove `debug=True` (or read it from an environment variable that is off in production). If your host runs the app for you, make sure it is not started in development mode.',
114
+ },
115
+ 'tls-verification-off': {
116
+ title: 'Certificate checking is switched off',
117
+ why: 'This line tells your app to trust any server, even a fake one — so anyone on the same network (a café, a hotel) can sit between your app and the real service and read or change what passes through, including your API keys.',
118
+ fix: 'Delete this line. It was almost certainly added to make a certificate error go away; the real fix is to solve that error (usually a wrong URL, a missing certificate on your own server, or a corporate proxy). If it was only for local testing, keep it out of the code that ships.',
119
+ },
120
+ 'personal-path': {
121
+ title: 'Your local file path (with your username) is in the code',
122
+ why: 'Purely a privacy footprint: it reveals your computer\'s username and folder layout. Harmless to the app, but not something to publish.',
123
+ fix: 'Use a relative path or an environment variable instead.',
124
+ },
125
+ 'agent-artifact': {
126
+ title: 'AI-tool working files are checked into git',
127
+ why: 'Local agent settings and session files can contain machine details, and occasionally pasted secrets. They are your workspace, not your product.',
128
+ fix: (f) => `Run \`git rm -r --cached ${f.file || '<path>'}\` (stops tracking it; the file stays on your machine), add its folder to .gitignore, then \`git commit -m "stop tracking AI tool files"\`.`,
129
+ },
130
+ 'test-registry-present': {
131
+ title: 'Generated test logins exist on this machine',
132
+ why: 'Your AI kept the logins it made for testing in .shipclear/test-accounts.json (git-ignored, so not in the repo). If any of them were ever added to a real database, they are working logins to your live app.',
133
+ fix: 'Before launch: make sure none of these accounts exist in your production database, then delete .shipclear/test-accounts.json. Nothing to do in git — this is about your live app, not the code.',
134
+ },
135
+ 'env-example-missing': {
136
+ title: 'No .env.example template exists',
137
+ why: 'Not a leak — just a courtesy: a template tells collaborators (and AI agents) which settings exist without revealing any values.',
138
+ fix: 'Copy .env to .env.example and delete all the values, keeping just the variable names — or run `shipclear ship` again without `--no-fix` and ShipClear will create it for you.',
139
+ },
140
+ };
141
+
142
+ const SEVERITY_ORDER = { critical: 0, high: 1, medium: 2, low: 3, info: 4 };
143
+ const SEVERITY_LABEL = {
144
+ critical: '🛑 CRITICAL', high: '🔴 HIGH', medium: '🟠 MEDIUM', low: '🟡 LOW', info: 'ℹ️ INFO',
145
+ };
146
+
147
+ export function verdictOf(findings) {
148
+ const open = findings.filter((f) => !f.resolved);
149
+ if (open.some((f) => f.severity === 'critical')) return 'DO_NOT_SHIP';
150
+ if (open.some((f) => f.severity === 'high' || f.severity === 'medium')) return 'SHIP_WITH_FIXES';
151
+ return 'CLEARED';
152
+ }
153
+
154
+ export const VERDICT_TEXT = {
155
+ DO_NOT_SHIP: '🛑 DO NOT SHIP — fix the critical findings first.',
156
+ SHIP_WITH_FIXES: '🟡 SHIP WITH FIXES — nothing catastrophic, but close the items listed above first.',
157
+ CLEARED: '✅ CLEARED TO SHIP — no blocking findings.',
158
+ };
159
+
160
+ // `scan` deliberately skips slower checks (full git history, dependency
161
+ // audits — see checks.js's `quick` mode) so it can run instantly. Its
162
+ // verdict must never be visually or textually confusable with the real
163
+ // ship/no-ship call from the full gate — a quick scan saying "CLEARED TO
164
+ // SHIP" is exactly the false-GO failure mode this project treats as its
165
+ // worst case (see CONTRIBUTING.md). Found by an independent fresh-eyes
166
+ // test that ran `scan`, saw this exact banner, and nearly stopped there.
167
+ const QUICK_VERDICT_TEXT = {
168
+ DO_NOT_SHIP: '🛑 Secrets found in this quick check — do not ship yet.',
169
+ SHIP_WITH_FIXES: '🟡 Issues found in this quick check.',
170
+ CLEARED: '✅ No secrets found in this quick check — but this is NOT the full gate (it looks at your current files only, not git history or dependencies).',
171
+ };
172
+
173
+ export const BADGE_MARKDOWN =
174
+ '[![ShipClear: cleared to ship](https://img.shields.io/badge/ShipClear-cleared%20to%20ship-brightgreen)](https://github.com/adamtimmerberg/ShipClear)';
175
+
176
+ function sorted(findings) {
177
+ return [...findings].sort(
178
+ (a, b) => SEVERITY_ORDER[a.severity] - SEVERITY_ORDER[b.severity]
179
+ || (a.file || '').localeCompare(b.file || '')
180
+ || (a.line || 0) - (b.line || 0)
181
+ );
182
+ }
183
+
184
+ function location(f) {
185
+ if (!f.file) return '';
186
+ return f.line ? `${f.file}:${f.line}` : f.file;
187
+ }
188
+
189
+ const color = (code, s) =>
190
+ process.stdout.isTTY && !process.env.NO_COLOR ? `\x1b[${code}m${s}\x1b[0m` : s;
191
+
192
+ // Fix texts are either a plain string or a function of the finding (see
193
+ // EXPLAIN above) — so the reader gets real file names and repo-specific
194
+ // steps, never a `<file>` placeholder.
195
+ const fixTextFor = (ex, f) => (typeof ex.fix === 'function' ? ex.fix(f) : ex.fix);
196
+
197
+ export function renderTerminal({ findings, notes, verdict, fixes = [], version, quick = false }) {
198
+ const lines = [];
199
+ lines.push('');
200
+ lines.push(color('1', ` ShipClear v${version} — ${quick ? 'quick scan' : 'ship report'}`));
201
+ lines.push('');
202
+
203
+ const open = sorted(findings.filter((f) => !f.resolved));
204
+ for (const f of open) {
205
+ const ex = EXPLAIN[f.id];
206
+ lines.push(` ${SEVERITY_LABEL[f.severity]} ${color('1', ex.title)}`);
207
+ if (location(f)) lines.push(` where: ${location(f)}${f.detail ? ` — ${f.detail}` : ''}`);
208
+ else if (f.detail) lines.push(` ${f.detail}`);
209
+ lines.push(` why it matters: ${ex.why}`);
210
+ // Fix text may be multi-line (numbered steps for the involved fixes);
211
+ // keep the "what to do:" label on the first line and indent the rest so
212
+ // steps read as a list, not a wall of prose.
213
+ const fixLines = fixTextFor(ex, f).split('\n');
214
+ lines.push(` what to do: ${fixLines[0]}`);
215
+ for (const extra of fixLines.slice(1)) lines.push(` ${extra}`);
216
+ lines.push('');
217
+ }
218
+ if (open.length === 0) lines.push(' No findings. Nice.');
219
+ if (fixes.length) {
220
+ lines.push('');
221
+ lines.push(color('1', ' Fixed for you:'));
222
+ for (const fix of fixes) lines.push(` ✔ ${fix}`);
223
+ }
224
+ for (const note of notes) lines.push(` note: ${note}`);
225
+ lines.push('');
226
+ const v = quick ? QUICK_VERDICT_TEXT[verdict] : VERDICT_TEXT[verdict];
227
+ lines.push(' ' + color(verdict === 'CLEARED' ? '32;1' : verdict === 'DO_NOT_SHIP' ? '31;1' : '33;1', v));
228
+ // The badge claims "cleared to ship" — only the full gate has earned
229
+ // that claim. Never offer it off the back of a quick, partial check.
230
+ if (verdict === 'CLEARED' && !quick) {
231
+ lines.push('');
232
+ lines.push(' Show it off — add the badge to your README:');
233
+ lines.push(' ' + BADGE_MARKDOWN);
234
+ } else if (open.length) {
235
+ // A stuck first-timer needs the pointer exactly here, at the verdict.
236
+ lines.push(' Stuck on any of the above? Plain-English help: https://github.com/adamtimmerberg/ShipClear/blob/main/docs/troubleshooting.md');
237
+ }
238
+ lines.push('');
239
+ lines.push(quick ? ' This was a quick check only. For the full gate: shipclear ship' : ' Full report written to SHIP-REPORT.md');
240
+ lines.push('');
241
+ return lines.join('\n');
242
+ }
243
+
244
+ export function renderMarkdown({ findings, notes, verdict, fixes = [], version, semanticSection = true }) {
245
+ const open = sorted(findings.filter((f) => !f.resolved));
246
+ const out = [];
247
+ out.push('# Ship Report');
248
+ out.push('');
249
+ out.push(`> Generated by [ShipClear](https://github.com/adamtimmerberg/ShipClear) v${version} on ${new Date().toISOString().slice(0, 10)}`);
250
+ out.push('');
251
+ out.push(`## Verdict: ${VERDICT_TEXT[verdict]}`);
252
+ out.push('');
253
+ if (verdict === 'CLEARED') {
254
+ out.push('Cleared! Add the badge to your README to show your app passed:');
255
+ out.push('');
256
+ out.push('```markdown');
257
+ out.push(BADGE_MARKDOWN);
258
+ out.push('```');
259
+ out.push('');
260
+ }
261
+ if (open.length === 0) {
262
+ out.push('No findings from the deterministic scan.');
263
+ } else {
264
+ out.push(`## Findings (${open.length})`);
265
+ for (const f of open) {
266
+ const ex = EXPLAIN[f.id];
267
+ out.push('');
268
+ out.push(`### ${SEVERITY_LABEL[f.severity]} — ${ex.title}`);
269
+ if (location(f)) out.push(`- **Where:** \`${location(f)}\`${f.detail ? ` — ${f.detail}` : ''}`);
270
+ else if (f.detail) out.push(`- **Detail:** ${f.detail}`);
271
+ out.push(`- **Why it matters:** ${ex.why}`);
272
+ // Multi-line fix text becomes a nested list so numbered steps render
273
+ // as steps in the markdown report, not one run-on bullet.
274
+ const fixLines = fixTextFor(ex, f).split('\n');
275
+ if (fixLines.length === 1) {
276
+ out.push(`- **What to do:** ${fixLines[0]}`);
277
+ } else {
278
+ out.push('- **What to do:**');
279
+ for (const step of fixLines) out.push(` - ${step}`);
280
+ }
281
+ }
282
+ }
283
+ if (fixes.length) {
284
+ out.push('');
285
+ out.push('## Fixed automatically');
286
+ for (const fix of fixes) out.push(`- ✔ ${fix}`);
287
+ }
288
+ if (notes.length) {
289
+ out.push('');
290
+ out.push('## Notes');
291
+ for (const note of notes) out.push(`- ${note}`);
292
+ }
293
+ if (semanticSection) {
294
+ out.push('');
295
+ out.push('## Semantic checks — for your AI assistant');
296
+ out.push('');
297
+ out.push('The scan above catches everything a pattern can catch. These need code understanding — ask your AI assistant to work through them (the ShipClear adapters do this automatically):');
298
+ out.push('');
299
+ out.push('- [ ] **Hidden accounts:** no route, seed, or conditional grants access via a fixed credential or magic string — including ones the AI created during development.');
300
+ out.push('- [ ] **Auth and ownership:** every endpoint either requires authentication or is intentionally public; every resource access verifies the resource belongs to the requester (an ID alone is never enough).');
301
+ out.push('- [ ] **Database exposure:** no world-readable/world-writable rules (Firebase rules, Supabase RLS, etc.).');
302
+ out.push('- [ ] **Client-side leakage:** no server secret reaches anything delivered to the browser.');
303
+ out.push('- [ ] **Input handling:** database/shell input parameterized, rendered HTML escaped, file uploads validated with size limits.');
304
+ out.push('- [ ] **Server-side trust:** prices, roles, and privileged fields computed or verified server-side; webhook signatures verified before processing.');
305
+ out.push('- [ ] **Session and abuse surface:** tokens in httpOnly cookies (not localStorage); rate limiting on login/signup/reset.');
306
+ out.push('- [ ] **Debug surface:** no debug endpoints, stack traces shown to users, or `*` CORS on authenticated APIs.');
307
+ out.push('- [ ] **AI surface** (if the app calls LLMs/agents): prompt inputs treated as untrusted, model output validated before executing/rendering/storing, agent and MCP tools least-privilege.');
308
+ out.push('- [ ] **Verdict update:** append any semantic findings here and restate the verdict (a critical one means DO NOT SHIP).');
309
+ out.push('');
310
+ out.push("Running ShipClear without an AI coding tool? Paste this checklist and your code into any AI chat (ChatGPT, Claude.ai, Gemini — even a free one) and ask it to work through the list — or ask a developer friend. These checks matter, but they need something that can read your code to answer them.");
311
+ }
312
+ out.push('');
313
+ return out.join('\n');
314
+ }
315
+
316
+ export { EXPLAIN };
@@ -0,0 +1,51 @@
1
+ import fs from 'node:fs';
2
+ import path from 'node:path';
3
+
4
+ const SKIP_DIRS = new Set([
5
+ 'node_modules', '.git', 'dist', 'build', 'out', '.next', '.nuxt',
6
+ 'vendor', '__pycache__', 'venv', '.venv', 'coverage', 'target',
7
+ ]);
8
+
9
+ const BINARY_EXT = new Set([
10
+ '.png', '.jpg', '.jpeg', '.gif', '.webp', '.ico', '.svgz',
11
+ '.woff', '.woff2', '.ttf', '.otf', '.eot',
12
+ '.zip', '.gz', '.tar', '.rar', '.7z',
13
+ '.pdf', '.mp3', '.mp4', '.mov', '.avi',
14
+ '.sqlite', '.db', '.bin', '.exe', '.dll', '.so', '.dylib',
15
+ '.pyc', '.class', '.jar', '.wasm',
16
+ ]);
17
+
18
+ const MAX_FILE_BYTES = 2 * 1024 * 1024;
19
+
20
+ /** Walk a directory tree, returning relative paths of candidate files. */
21
+ export function walkFiles(root) {
22
+ const results = [];
23
+ const walk = (dir) => {
24
+ for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
25
+ if (entry.isSymbolicLink()) continue;
26
+ const abs = path.join(dir, entry.name);
27
+ if (entry.isDirectory()) {
28
+ if (!SKIP_DIRS.has(entry.name)) walk(abs);
29
+ } else if (entry.isFile()) {
30
+ results.push(path.relative(root, abs).split(path.sep).join('/'));
31
+ }
32
+ }
33
+ };
34
+ walk(root);
35
+ return results;
36
+ }
37
+
38
+ export function isTextCandidate(rel) {
39
+ return !BINARY_EXT.has(path.extname(rel).toLowerCase());
40
+ }
41
+
42
+ /** Read a file as UTF-8, or null if missing, too large, or unreadable. */
43
+ export function readFileSafe(absPath) {
44
+ try {
45
+ const stat = fs.statSync(absPath);
46
+ if (!stat.isFile() || stat.size > MAX_FILE_BYTES) return null;
47
+ return fs.readFileSync(absPath, 'utf8');
48
+ } catch {
49
+ return null;
50
+ }
51
+ }
@@ -0,0 +1,44 @@
1
+ # The ShipClear gate — canonical checklist
2
+
3
+ This is the full specification of what `shipclear ship` verifies (deterministic layer) and what the AI adapter layer verifies on top (semantic layer). The CLI implements every deterministic check; adapters instruct the host agent through every semantic check. A project passes the gate only when both layers are clean.
4
+
5
+ ## Deterministic layer (implemented in the CLI)
6
+
7
+ | ID | Check | Severity when found |
8
+ |---|---|---|
9
+ | `secret-in-code` | Known secret formats (API keys, tokens, private keys, passwords in connection strings) in tracked files | critical |
10
+ | `secret-in-history` | Secrets present in any past commit, even if deleted since | critical |
11
+ | `env-tracked` | `.env` or other secrets files tracked by git | critical |
12
+ | `key-file-tracked` | Private key material (`.pem`, `id_rsa`, etc.) tracked by git | critical |
13
+ | `env-example-leak` | Real-looking values in `.env.example` | high |
14
+ | `gitignore-incomplete` | `.gitignore` missing `.env` coverage (or missing entirely) | high |
15
+ | `pii-data-file` | Tracked data files (CSV/TSV/SQL/JSONL) containing real-looking emails or SSNs | high |
16
+ | `test-account` | Hardcoded credential pairs (email + password literals) in source | high |
17
+ | `dependency-vulns` | Known CVEs in dependencies (`npm audit`, high and critical) | high |
18
+ | `generic-secret` | Suspicious hardcoded values assigned to names like `apiKey`, `password`, `token` | medium |
19
+ | `tls-verification-off` | Certificate checking disabled (`rejectUnauthorized: false`, `NODE_TLS_REJECT_UNAUTHORIZED=0`, `verify=False`) | medium |
20
+ | `debug-mode-on` | Framework debug mode left on in shipped code (`app.run(debug=True)`, `FLASK_DEBUG=1`) | medium |
21
+ | `personal-path` | Absolute paths exposing local usernames (`/home/adam/...`, `C:\Users\...`) | low |
22
+ | `agent-artifact` | AI-session artifacts (local agent settings, transcripts) tracked by git | low |
23
+ | `env-example-missing` | `.env` exists but `.env.example` doesn't | info |
24
+
25
+ ## Semantic layer (performed by the host AI via adapters)
26
+
27
+ 1. **Hidden accounts:** search for any route, seed, migration, or conditional that grants access via a fixed credential, magic string, or special email — including ones the AI itself created during this project. Check `.shipclear/test-accounts.json` and confirm every listed credential is absent from code and the production database plan.
28
+ 2. **Auth and ownership:** enumerate every route/endpoint; confirm each one either requires authentication or is intentionally public (list the public ones). For every authenticated resource access, confirm the code checks the resource *belongs to the requester* — an ID in a URL or request body must never be enough on its own.
29
+ 3. **Database exposure:** review Firebase rules / Supabase RLS / equivalent for world-readable or world-writable access. Deny-by-default expected.
30
+ 4. **Client-side leakage:** confirm no server secret is imported, bundled, or rendered into anything delivered to the browser.
31
+ 5. **Input handling:** user input reaching SQL/NoSQL/shell must be parameterized; user content rendered into HTML must be escaped or sanitized; file uploads must validate type and enforce a size limit.
32
+ 6. **Server-side trust:** anything with consequences must be computed or verified server-side — prices and entitlements (never trust totals from the client), roles and privileged fields (no mass-assignment of `role`/`isAdmin` from request bodies), and incoming webhooks (verify the signature before processing).
33
+ 7. **Session and abuse surface:** session tokens in httpOnly cookies rather than localStorage; rate limiting (or provider-equivalent) on login, signup, and password-reset endpoints.
34
+ 8. **Debug surface:** look for debug endpoints, verbose stack traces returned to users, `console.log` of sensitive values, and permissive CORS on authenticated APIs.
35
+ 9. **AI surface** (when the app itself calls LLMs or runs agents/MCP tools): treat user input and any external content entering prompts as untrusted; never execute, render as HTML, or write to the database unvalidated model output; give agent/MCP tools the narrowest permissions that work, and keep agents away from production resources during testing.
36
+ 10. **Report merge:** append semantic findings to `SHIP-REPORT.md` in the same *What / Why / Fix* format, then restate the final verdict — a semantic critical downgrades the overall verdict to DO NOT SHIP.
37
+
38
+ ## Verdict rules
39
+
40
+ - Any **critical** finding → 🛑 **DO NOT SHIP**
41
+ - Any **high** or **medium** finding (no criticals) → 🟡 **SHIP WITH FIXES**
42
+ - Only low/info notes → ✅ **CLEARED TO SHIP**
43
+
44
+ A false ✅ is the worst possible outcome and always takes priority over false-positive comfort.
@@ -0,0 +1,33 @@
1
+ # ShipClear prevention rules
2
+
3
+ These rules are for AI coding agents and assistants. Follow them on every task, in every project, from the first line of code. They exist so the human you're helping never ships a leaked secret, an exposed database, or a forgotten test account. When a rule conflicts with speed, the rule wins.
4
+
5
+ ## 1. Secrets never touch source code
6
+
7
+ - Before writing the first line of application code in a new project, create three files: `.env` (real values, never committed), `.env.example` (same keys, empty or placeholder values), and `.gitignore` containing at minimum `.env`, `.env.*` (with an exception for `.env.example`), and your dependency directory.
8
+ - Never write an API key, password, token, connection string, or any credential directly in source code, config files, notebooks, HTML, or client-side JavaScript — not even "temporarily", not even in comments. Use environment variables (`process.env.X`, `os.environ["X"]`) from the very first draft.
9
+ - Server-side secrets must never reach frontend code. Anything shipped to a browser is public. If the frontend needs data that requires a secret, put the secret behind a server endpoint.
10
+ - If a secret was ever written into a tracked file — even if deleted afterwards — tell the user plainly: the key must be **rotated** (regenerated at the provider), because git history preserves it.
11
+
12
+ ## 2. No hardcoded accounts, ever
13
+
14
+ - Never create hardcoded login credentials, "temporary" admin accounts, magic bypass strings, or backdoor routes while building or testing — not in code, not in seed scripts, not in database migrations.
15
+ - If testing requires an account, generate random credentials at seed time, store them only in a git-ignored file (`.shipclear/test-accounts.json`), and tell the user it exists so it can be deleted before launch.
16
+ - Never weaken authentication to make a test pass. Fix the test.
17
+
18
+ ## 3. Real data stays out of the repository
19
+
20
+ - Never commit data files (CSV, JSON, SQL dumps, database files, exports) containing real names, emails, phone numbers, or any personal information. Generate synthetic data for development and seeding.
21
+ - If the user provides a real data file, keep it git-ignored and say so explicitly.
22
+
23
+ ## 4. Secure defaults for common stacks
24
+
25
+ - Every non-public API route gets an authentication check — deny by default. Do not defer auth "until later".
26
+ - Database access rules (Firebase rules, Supabase RLS, and similar) start locked: no world-readable or world-writable tables. Enable row-level security before inserting the first row.
27
+ - Validate and parameterize all user input touching a database (no string-built SQL). Escape output rendered into HTML.
28
+ - Don't disable security to fix an error (CORS `*` on authenticated APIs, `rejectUnauthorized: false`, disabled CSRF) — solve the actual problem, and say what it was.
29
+
30
+ ## 5. Be honest about security state
31
+
32
+ - When you take a shortcut or leave something unfinished that has security implications, say so out loud and record it — never let the user believe the project is more secure than it is.
33
+ - Before the user shares the repository publicly or deploys, recommend running the ShipClear gate: `npx shipclear ship`.
@@ -0,0 +1,87 @@
1
+ # Quickstart
2
+
3
+ Every path below ends the same way: run `npx shipclear ship` before you deploy or make the repo public, and get a plain-English verdict.
4
+
5
+ ## First time typing a command? Start here.
6
+
7
+ Every instruction below happens in a **terminal** — a text window where you type commands instead of clicking buttons. If you've only ever built with AI tools in the browser, here's where to find one:
8
+
9
+ - **Bolt.new / Lovable / Replit / v0:** look for a "Terminal" or "Console" tab or panel, usually near the bottom of the screen or behind a `>_` icon. Once you find it, skip straight to **["Use the standalone CLI"](#use-the-standalone-cli-works-with-any-ai-tool-or-none)** below — these tools' AI runs in your browser, not as a terminal agent ShipClear plugs into, so that's the section for you.
10
+ - **Cursor / VS Code:** the menu bar → **Terminal → New Terminal**.
11
+ - **Mac:** open the **Terminal** app (search for it with Spotlight, ⌘+Space).
12
+ - **Windows:** open **PowerShell** or **Windows Terminal** from the Start menu.
13
+
14
+ Once it's open, make sure you're inside your project's folder: type `cd ` (with a space), then the folder's path — e.g. `cd my-app` if it's right there, or on a Mac just drag the folder onto the terminal window to paste its path. Then type the commands below and press Enter.
15
+
16
+ **If an `npx` command says `command not found`:** you'll also need [Node.js](https://nodejs.org) installed — it's what runs `npx`. Download the "LTS" version, install it like any other app, then try again. (If a *different* command says this — one that doesn't start with `npx` — that specific tool isn't installed; Node.js won't fix it.)
17
+
18
+ ## Which section below is yours?
19
+
20
+ - You talk to your AI **in a browser tab** (Bolt, Lovable, Replit, v0, ChatGPT, Claude.ai) → [Use the standalone CLI](#use-the-standalone-cli-works-with-any-ai-tool-or-none).
21
+ - You talk to your AI **inside your code editor or terminal** (Claude Code, Codex, Cursor, Gemini CLI) → find it by name below; you get the extra layers (prevention rules, commit guard).
22
+ - Not sure, or none of the above → the standalone CLI works for everyone.
23
+
24
+ ## Claude Code (recommended: the full plugin)
25
+
26
+ ```
27
+ /plugin marketplace add adamtimmerberg/ShipClear
28
+ /plugin install shipclear@shipclear
29
+ ```
30
+
31
+ You now have all three layers:
32
+ - the **prevention skill** — Claude builds with secure defaults on every task;
33
+ - the **commit guard** — a hook blocks `git commit` / `git push` while staged changes contain secrets;
34
+ - the **`/safe-to-ship` command** — the full gate plus Claude's semantic review, ending in a verdict.
35
+
36
+ ## Codex CLI
37
+
38
+ 1. Copy the contents of [`adapters/codex/AGENTS.md`](../adapters/codex/AGENTS.md) into your project's `AGENTS.md` (create it if needed).
39
+ 2. In your project: `npx shipclear setup` (installs the git commit guard).
40
+ 3. Before launch, ask Codex to run the gate, or run `npx shipclear ship` yourself.
41
+
42
+ ## Cursor
43
+
44
+ 1. Copy [`adapters/cursor/shipclear.mdc`](../adapters/cursor/shipclear.mdc) to `.cursor/rules/shipclear.mdc` in your project.
45
+ 2. `npx shipclear setup`
46
+ 3. Before launch: `npx shipclear ship`, then ask Cursor to complete the semantic checklist in `SHIP-REPORT.md`.
47
+
48
+ ## Gemini CLI
49
+
50
+ 1. Append [`adapters/gemini/GEMINI.md`](../adapters/gemini/GEMINI.md) to your project's `GEMINI.md`.
51
+ 2. `npx shipclear setup`
52
+ 3. Before launch: `npx shipclear ship`.
53
+
54
+ ## Any other tool, or a local model
55
+
56
+ 1. Paste [`adapters/generic/INSTRUCTIONS.md`](../adapters/generic/INSTRUCTIONS.md) into your system prompt or project context.
57
+ 2. `npx shipclear setup`
58
+ 3. Before launch: `npx shipclear ship`.
59
+
60
+ ## Use the standalone CLI (works with any AI tool, or none)
61
+
62
+ This is the path for **Bolt.new, Lovable, Replit, v0**, or anyone without a terminal-integrated AI coding tool — including if you're not using AI at all. ShipClear is a complete standalone CLI:
63
+
64
+ ```bash
65
+ npx shipclear setup # .env/.env.example/.gitignore + commit guard
66
+ npx shipclear scan # quick secrets check any time
67
+ npx shipclear ship # the full gate → SHIP-REPORT.md + verdict
68
+ npx shipclear ship --fix-history
69
+ # after fixing everything else: erase old commits that held a secret
70
+ # (only for repos never pushed anywhere; refuses if it would lose any work)
71
+ ```
72
+
73
+ The full report (`SHIP-REPORT.md`) ends with a checklist meant for an AI to work through with you — if you don't have one built into your terminal, paste that checklist and your code into any AI chat (ChatGPT, Claude.ai, Gemini — even a free one) and ask it to go through the list, or ask a developer friend.
74
+
75
+ ## Stuck?
76
+
77
+ [troubleshooting.md](troubleshooting.md) answers the questions first-timers actually hit — "command not found", "it told me to commit but I don't know how", "pip install failed", "I cleaned up but the finding is still there" — plus a plain-English glossary (terminal, commit, remote, rotate, .env…).
78
+
79
+ ## Reading the verdict
80
+
81
+ - ✅ **CLEARED TO SHIP** — no blocking findings.
82
+ - 🟡 **SHIP WITH FIXES** — nothing catastrophic; close the listed items first.
83
+ - 🛑 **DO NOT SHIP** — a critical finding (a leaked secret, a committed `.env`) needs fixing *and* usually a key rotation. The report tells you exactly what to do, step by step.
84
+
85
+ ## False positives
86
+
87
+ Real-world code sometimes trips a pattern. Add the file path to a `.shipclearignore` in your project root (one path or glob per line) — and please [tell us](https://github.com/adamtimmerberg/ShipClear/issues/new?template=false_positive.md) so the pattern gets fixed for everyone.
@@ -0,0 +1,75 @@
1
+ # Troubleshooting & plain-English glossary
2
+
3
+ Stuck on something ShipClear said? This page answers the questions real first-time users actually hit, in plain language. Nothing here assumes you're a programmer.
4
+
5
+ If your question isn't here, paste ShipClear's output (it never includes your secret values — those are masked) into any AI chat like ChatGPT, Claude.ai, or Gemini and ask "what does this mean and what do I do?"
6
+
7
+ ---
8
+
9
+ ## "It says `command not found`"
10
+
11
+ - If the command started with **`npx`**: you need [Node.js](https://nodejs.org) installed — it's the free program that runs `npx`. Download the "LTS" version, install it like any app, restart your terminal, and try again.
12
+ - If it was a **different** command (like `pip`, `bfg`, or `git`): that specific program isn't installed. Node.js won't fix it — installing that program (or using the "ask your AI assistant" route ShipClear offers) will.
13
+
14
+ ## "It told me to `commit`, but I don't know how"
15
+
16
+ "Commit" means "save this change into git's history." The command is:
17
+
18
+ ```
19
+ git add -A && git commit -m "describe what you changed"
20
+ ```
21
+
22
+ If git then says **"Please tell me who you are"**, it's a one-time setup — run the two lines it prints (they start with `git config`), then run the commit again.
23
+
24
+ ## "`pip install git-filter-repo` failed with `externally-managed-environment`"
25
+
26
+ This is common on newer Macs and Linux. Two easy ways around it:
27
+
28
+ - Try `pipx install git-filter-repo` instead, or `pip install --user git-filter-repo`.
29
+ - Or don't bother — use the **"ask your AI assistant"** route ShipClear offers for that finding. It doesn't need any of this installed.
30
+
31
+ ## "I ran `shipclear ship` after cleaning up, and it STILL shows the same finding"
32
+
33
+ If the finding was **a secret in git history** and you cleaned it using a tool like BFG or git-filter-repo: those tools clean the copy on GitHub, but your **own folder still has the old history**. Sync your folder to the cleaned version:
34
+
35
+ ```
36
+ git fetch && git reset --hard origin/main
37
+ ```
38
+
39
+ (Replace `main` with `master` if that's your branch.) Then run `shipclear ship` again.
40
+
41
+ ## "git printed `rm '.env'` / `delete mode ...` — did I just delete my file?"
42
+
43
+ No. `git rm --cached` only tells git to **stop tracking** the file — the file stays safely on your computer. The scary-looking "rm" and "delete" words refer to git's records, not your actual files.
44
+
45
+ ## "The commit guard let my commit through with a warning about being offline"
46
+
47
+ The guard blocks commits that contain secrets. But if it can't run its check at all (you're offline, or the tool isn't fully installed), it lets the commit through with a warning rather than blocking your work forever. It fails *safe*, not *stuck*. When you're back online, run `npx shipclear scan --staged` yourself to check.
48
+
49
+ ## "The report has a checklist full of words I don't understand"
50
+
51
+ That's the "Semantic checks" section at the bottom of `SHIP-REPORT.md`, and it's meant for an AI, not for you to decode. Paste it and your code into any AI chat (ChatGPT, Claude.ai, Gemini — a free one is fine) and ask it to work through the list. Or ask a developer friend.
52
+
53
+ ## "How do I put a secret in `.env` and read it in my code?"
54
+
55
+ 1. Put the value in the `.env` file, one per line: `MY_KEY=the-actual-value`
56
+ 2. In your code, read it with `process.env.MY_KEY` (JavaScript) or `os.environ["MY_KEY"]` (Python).
57
+ 3. Most frameworks load `.env` automatically. **Plain Node.js does not** — either run your app with `node --env-file=.env yourapp.js`, or add `require('dotenv').config()` at the top (after `npm install dotenv`).
58
+
59
+ ## "It found a hardcoded login and I deleted it — now my login page is broken"
60
+
61
+ Expected. Deleting the fake account also removed what your login checked against, so you need a *real* login system now. The easiest path: ask the AI assistant you built the app with to "replace the hardcoded login with a real login system." (Common choices it might use: Supabase Auth, Clerk, Auth0, NextAuth — but you don't have to choose; it can.)
62
+
63
+ ---
64
+
65
+ ## Mini-glossary
66
+
67
+ - **Terminal** — the text window where you type commands instead of clicking. (In Bolt/Lovable/Replit, look for a "Terminal" or "Console" tab; on Mac, the Terminal app; on Windows, PowerShell.)
68
+ - **git** — the tool that tracks the history of your project's files. "Committing" saves a version; "pushing" sends it to GitHub.
69
+ - **commit** — one saved version of your project in git's history.
70
+ - **remote / origin** — the copy of your project on GitHub (or similar). "Already pushed" means a copy exists there.
71
+ - **`.gitignore`** — a list of files git should never save/share (like `.env`).
72
+ - **`.env`** — a private file holding your real secret values. It stays on your machine and should never be committed.
73
+ - **`.env.example`** — a safe-to-share template: the same setting *names* as `.env`, but with the values blank.
74
+ - **rotate a key** — get a brand-new key from whoever issued it (e.g. log into OpenAI and generate a new one) and stop using the old one. This is what actually protects you after a leak.
75
+ - **force-push** — overwrite the copy of history on GitHub with your cleaned-up local version. Fine for a repo only you work on.