shipped-secrets 0.0.0-stage → 1.0.1

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Fuad Laguda
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,140 @@
-# Temporary Holding Version
-
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+# shipped-secrets
+
+Find secrets that ship to the browser, or sit in your git history, and check whether they still work.
+
+Every frontend framework has a prefix that puts a setting into the JavaScript every visitor downloads: `VITE_`, `NEXT_PUBLIC_`, `REACT_APP_`, `EXPO_PUBLIC_`, `PUBLIC_`. Give a server key that prefix by mistake, or paste a key into client code, and it is public. The build works, the site works, and nobody notices until a bill or a breach. One Vite user lost about €800 when bots scraped a `VITE_GEMINI_API_KEY` from the bundle.
+
+`shipped-secrets` looks at what your build actually ships and tells you, before you deploy.
+
+```sh
+npx shipped-secrets
+```
+
+```
+ERROR .next/static/chunks/app/page-5fb8.js:1  NEXT_PUBLIC_SUPABASE_SERVICE_ROLE_KEY from .env.local  [eyJh…sM (157 chars)]
+ERROR .next/server/app/index.html:1  STRIPE_SECRET_KEY from .env.local  [sk_t…H5 (38 chars)]  (2 times)
+warn  .env.local:1  NEXT_PUBLIC_SUPABASE_SERVICE_ROLE_KEY has a public prefix, so its value goes to every visitor. ...
+
+2 secrets shipped to the browser, in 3 places. Rotate them: deleting a key from the code does not take it back from anyone who already downloaded it.
+```
+
+Secrets are always masked in the output. Nothing is sent anywhere unless you ask for `--verify`.
+
+## What it checks
+
+1. **Your real secret values.** It reads your `.env*` files (not `.env.example`), Cloudflare's `.dev.vars`, and in CI the environment variables whose names look secret. Then it searches every file the browser gets, source maps included, for each server value. This catches any secret, whatever it looks like, including your own tokens that no pattern knows. It also looks for each value in base64 (standard and URL-safe, with or without padding), so a key put through `btoa()` on its way into the bundle is still found.
+2. **Known key shapes**, for keys that are not in your local files: Stripe secret and restricted keys, Supabase service_role keys and `sb_secret_` keys, OpenAI, Anthropic, AWS, GitHub, SendGrid, Mailgun, Resend, Twilio, Slack, private keys, and database URLs with a password. Google API keys are a warning, because Maps and Firebase browser keys are meant to be public.
+3. **Supabase keys are decoded.** The anon key and the service_role key look the same. `shipped-secrets` reads the role inside the key, so the anon key is never reported and the service_role key always is.
+4. **Risky names.** A public prefix on a name that sounds secret (`VITE_STRIPE_SECRET`, `NEXT_PUBLIC_SUPABASE_SERVICE_ROLE_KEY`) is a warning. Names meant for browsers (anon key, publishable key, client ID, client token, site key) are not.
+5. **Source maps left public.** A warning, because anyone can read your original code from them, and a key pasted into the code is in there too.
+
+## Where it looks
+
+It finds the browser output by itself and never reads server code:
+
+| framework | folder |
+|---|---|
+| Next.js | `.next/static`, plus prerendered pages in `.next/server/app` (`.html`, `.rsc`), which go straight to visitors |
+| Next.js static export | `out` |
+| Vite, Astro | `dist`, or `dist/client` when there is a server build |
+| Create React App, React Router, Remix | `build`, or `build/client` |
+| SvelteKit | `.svelte-kit/output/client` |
+| Nuxt, Nitro | `.output/public` |
+| Vercel output | `.vercel/output/static` |
+| Storybook | `storybook-static` |
+
+Folders named `server` or `ssr` inside them are skipped. Point it anywhere else with `--dir`.
+
+## A live site
+
+```sh
+npx shipped-secrets https://your.site
+```
+
+It loads the page the way a stranger would, follows its scripts, the chunks they load and their source maps, and looks for key shapes. Run it from your project folder and it also looks for your own `.env` values on the live site. Only files on the same site are fetched.
+
+## In your build
+
+Vite plugin, which fails the build when a secret ships:
+
+```js
+// vite.config.js
+import shippedSecrets from 'shipped-secrets/vite'
+
+export default { plugins: [shippedSecrets()] }
+```
+
+Any framework, after the build in CI:
+
+```sh
+npm run build && npx shipped-secrets
+```
+
+It exits with code 1 when a secret is found, so CI stops. Warnings do not fail it unless you add `--strict`.
+
+## When something is meant to be public
+
+Add `# shipped-ok` to the line in your `.env` file:
+
+```
+NEXT_PUBLIC_MAPS_KEY=AIza...   # shipped-ok, restricted to our domain
+```
+
+or pass `--allow NAME`. `--json` gives the findings as JSON, masked like the text output.
+
+## Git history
+
+```sh
+npx shipped-secrets --history
+```
+
+A key that was committed once is in every clone of the repository, even after you delete it. `--history` reads every commit on every branch and reports:
+
+- known key shapes in any line ever added
+- your current `.env` secret values in any file ever committed, so a key you moved out of the code is still caught where it used to be
+- any `.env` file ever committed with a server secret in it. A public build switch like `VITE_EDITION=web` is not reported.
+
+Each finding names the file, line, commit and date. Run it without a build and it searches history only.
+
+## Is the key still live?
+
+```sh
+npx shipped-secrets --verify
+```
+
+For each key it finds, it asks that key's own provider, with a read-only call, whether the key still works:
+
+```
+ERROR dist/assets/index-3f9a.js:1  Stripe secret or restricted key  [sk_l…x2 (107 chars)]  (LIVE, rotate now)
+ERROR dist/assets/index-3f9a.js:1  GitHub token  [ghp_…9Q (40 chars)]  (dead)
+```
+
+Checked: Stripe (`/v1/balance`), OpenAI and Anthropic (list models), GitHub (`/user`), SendGrid, Mailgun, Resend, Slack (`auth.test`), and Supabase service_role keys (your project's own API, read from the key). Others are marked "not checked". The key goes to its own provider over HTTPS and nowhere else, and only when you pass `--verify`. Live keys are listed first.
+
+`--all` is `--history` and `--verify` together.
+
+## What it does not do
+
+- It does not find a secret that was split into pieces, encrypted, or encoded some other way before it reached the bundle. It matches the exact value, its base64 forms and the known key shapes.
+- It does not fix anything. A shipped or committed secret must be rotated at the provider: deleting it from the code, or rewriting history, does not take it back from anyone who already has it.
+- `--verify` does not check AWS, Twilio, private keys or database URLs: checking those safely needs more than one harmless call.
+
+## Tested
+
+`npm test` runs the unit tests. `npm run test:apps` builds a real Vite app and a real Next.js app with planted fake keys, and checks that:
+
+- a Stripe key behind `VITE_` is found by its real value, and a key pasted into code is found in the source map too
+- the Supabase anon key and a server-only token are not reported
+- a Supabase service_role key behind `NEXT_PUBLIC_` is found in the Next.js client bundle
+- a secret printed into a page by a server component is found in the prerendered HTML and RSC payload
+- a server-only secret in Next.js server code is not reported
+- the same build served locally is caught by URL, with and without local `.env` values
+- the Vite plugin stops a build with a secret and passes a clean one
+
+The unit tests also build a git repository with a key committed and then deleted and check `--history` finds it, and check `--verify` against a stand-in provider (live, dead and unreachable). Full keys are never in the JSON output.
+
+Fake keys are made while the tests run, so nothing secret-shaped is ever committed.
+
+## Licence
+
+MIT. By Fuad Laguda.

package/bin/shipped-secrets.js

@@ -0,0 +1,71 @@
+#!/usr/bin/env node
+// shipped-secrets [folder or URL] [--dir <path>] [--allow NAME] [--history] [--verify] [--all] [--strict] [--json]
+//
+// With no argument: finds the browser output in this project (dist, .next/static,
+// build, out, ...), reads this project's .env files, and looks for your real
+// secret values and known key shapes in what ships. With a URL: scans the live
+// site the way a stranger would, using this folder's .env values too.
+// --history searches every commit, --verify asks each key's provider if it still works.
+// Exit code 1 when a secret is found, or a warning with --strict.
+
+import { resolve } from 'node:path'
+import { loadSecrets, report, scanHistory, scanProject, scanUrl, verifyFindings } from '../src/index.js'
+
+const args = process.argv.slice(2)
+const flag = (name) => args.includes(name)
+const values = (name) => args.flatMap((a, i) => (a === name && args[i + 1] ? [args[i + 1]] : []))
+const positional = args.filter((a, i) => !a.startsWith('--') && !['--dir', '--allow'].includes(args[i - 1]))
+
+if (flag('--help') || flag('-h')) {
+  console.log(`Usage: shipped-secrets [folder | https://your.site] [options]
+
+Finds secrets that ship to the browser: your real .env values, known key shapes
+(Stripe, Supabase service_role, OpenAI, Anthropic, AWS, GitHub, ...), public-prefixed
+names that look secret, and source maps left public.
+
+  --history       also search every commit in git history, and flag any .env file ever committed
+  --verify        ask each found key's own provider whether it still works (sends the key to that provider only)
+  --all           --history and --verify
+  --dir <path>    scan this folder instead of finding the build output (repeatable)
+  --allow NAME    a variable that is fine to ship (repeatable). Or add "# shipped-ok" to its .env line
+  --strict        warnings fail too
+  --json          machine-readable output (full keys are never included)`)
+  process.exit(0)
+}
+
+const target = positional[0] ?? '.'
+const allow = values('--allow')
+const history = flag('--history') || flag('--all')
+const verify = flag('--verify') || flag('--all')
+let result
+if (/^https?:\/\//.test(target)) {
+  const { secrets } = loadSecrets(process.cwd(), { env: process.env, allow })
+  result = await scanUrl(target, secrets)
+} else {
+  const root = resolve(target)
+  const dirs = values('--dir')
+  result = scanProject(root, { dirs: dirs.length ? dirs.map((d) => resolve(root, d)) : undefined, allow })
+  if (!dirs.length && !result.dirs.length && !history) {
+    console.error('No build output found (dist, build, out, .next/static, ...). Build first, pass --dir <path>, or use --history.')
+    process.exit(2)
+  }
+  if (history) {
+    const { secrets } = loadSecrets(root, { env: {}, allow })
+    const past = await scanHistory(root, secrets)
+    if (!past) console.error(`${root} is not in a git repository, so history was not searched.`)
+    else {
+      result.findings.push(...past.findings)
+      result.commits = past.commits
+    }
+  }
+}
+
+if (verify) {
+  await verifyFindings(result.findings)
+  result.verified = true
+}
+
+if (flag('--json')) console.log(JSON.stringify(result, null, 2))
+else console.log(report(result))
+const errors = result.findings.some((f) => f.level === 'error')
+process.exit(errors || (flag('--strict') && result.findings.length) ? 1 : 0)

package/package.json

@@ -1,6 +1,22 @@
 {
   "name": "shipped-secrets",
-  "version": "0.0.0-stage",
-  "stub": true,
-  "description": "Temporary package placeholder for staged publishing"
-}
+  "version": "1.0.1",
+  "description": "Find secrets that ship to the browser or sit in git history, and check whether they still work: your real .env values, known key shapes and risky public env names, in any framework's build output, source maps included.",
+  "type": "module",
+  "exports": {
+    ".": { "types": "./src/index.d.ts", "default": "./src/index.js" },
+    "./vite": { "types": "./src/vite.d.ts", "default": "./src/vite.js" }
+  },
+  "types": "./src/index.d.ts",
+  "bin": { "shipped-secrets": "bin/shipped-secrets.js" },
+  "files": ["src", "bin"],
+  "scripts": {
+    "test": "node --test test/*.test.js",
+    "test:apps": "node test/apps/run.mjs"
+  },
+  "keywords": ["secrets", "security", "vite", "vite-plugin", "nextjs", "env", "bundle", "leak", "supabase", "stripe", "ci", "git-history", "verify"],
+  "author": "Fuad Laguda",
+  "license": "MIT",
+  "repository": { "type": "git", "url": "git+https://github.com/lagudafuadtosin/shipped-secrets.git" },
+  "engines": { "node": ">=18" }
+}

package/src/env.js

@@ -0,0 +1,111 @@
+// Reads the secret values the project actually has, so the scan can look for the
+// real thing in what ships, not just for shapes.
+//
+// Sources: every .env* file in the project folder (not the .example / .sample /
+// .template ones) and Cloudflare's .dev.vars, plus, for CI, environment
+// variables whose names look secret. A value counts as a secret only if its
+// name has no public prefix. Values that are also given to a public-prefixed
+// name are public on purpose and left out.
+
+import { existsSync, readdirSync, readFileSync } from 'node:fs'
+import { join } from 'node:path'
+
+export const PUBLIC_PREFIXES = ['VITE_', 'NEXT_PUBLIC_', 'REACT_APP_', 'EXPO_PUBLIC_', 'PUBLIC_', 'NUXT_PUBLIC_', 'GATSBY_', 'STORYBOOK_']
+export const isPublicName = (name) => PUBLIC_PREFIXES.some((p) => name.startsWith(p))
+const stripPrefix = (name) => {
+  const p = PUBLIC_PREFIXES.find((x) => name.startsWith(x))
+  return p ? name.slice(p.length) : name
+}
+
+const SECRET_WORDS = /(SECRET|TOKEN|PASSWORD|PASSWD|PRIVATE|API_?KEY|ACCESS_?KEY|SERVICE_?ROLE|DATABASE_URL|DB_URL|CONNECTION_STRING|CREDENTIAL|AUTH_KEY|WEBHOOK)/
+// Names that look secret but are meant for browsers.
+const SAFE_WORDS = /(ANON_?KEY|PUBLISHABLE|PUBLIC_?KEY|SITE_?KEY|MEASUREMENT_ID|CLIENT_?ID|CLIENT_?TOKEN|APP_?ID)/
+
+export const looksSecretName = (name) => SECRET_WORDS.test(name) && !SAFE_WORDS.test(name)
+export const looksSafeName = (name) => SAFE_WORDS.test(name)
+
+/** Parses dotenv text. Returns [{ name, value, line, ok }] where ok = a "shipped-ok" comment on the line. */
+export function parseDotenv(text) {
+  const out = []
+  text.split(/\r?\n/).forEach((raw, i) => {
+    const line = raw.trim()
+    if (!line || line.startsWith('#')) return
+    const m = /^(?:export\s+)?([A-Za-z_][A-Za-z0-9_]*)\s*=\s*(.*)$/.exec(line)
+    if (!m) return
+    let value = m[2]
+    let ok = /#.*shipped-ok/.test(value)
+    if (/^"/.test(value)) value = (/^"((?:[^"\\]|\\.)*)"/.exec(value) || [, ''])[1].replace(/\\n/g, '\n')
+    else if (/^'/.test(value)) value = (/^'([^']*)'/.exec(value) || [, ''])[1]
+    else value = value.replace(/\s+#.*$/, '').trim()
+    out.push({ name: m[1], value, line: i + 1, ok })
+  })
+  return out
+}
+
+const SKIP_FILES = /\.(example|sample|template|dist|defaults)$/i
+
+/** Every env file in dir that holds real values. */
+export function envFiles(dir) {
+  const names = existsSync(dir) ? readdirSync(dir) : []
+  return names.filter((n) => (n === '.dev.vars' || n === '.env' || n.startsWith('.env.')) && !SKIP_FILES.test(n)).map((n) => join(dir, n))
+}
+
+// Too short, or a value that is fine to be public whatever its name.
+function trivial(value) {
+  if (value.length < 12) return true
+  if (/^(true|false|null|undefined|production|development|test)$/i.test(value)) return true
+  if (/^\d+(\.\d+)?$/.test(value)) return true
+  // A URL is only a secret if it carries a password or a token in it.
+  if (/^https?:\/\/[^\s]+$/i.test(value) && !/:\/\/[^/@\s]+:[^/@\s]+@/.test(value) && !/[?&](key|token|secret|sig|signature|password)=/i.test(value)) return true
+  return false
+}
+
+/**
+ * The secrets to look for: { name, value, source, ok }.
+ * @param {string} dir the project folder
+ * @param {{ env?: Record<string, string | undefined>, files?: string[], allow?: string[] }} [opts]
+ */
+export function loadSecrets(dir, opts = {}) {
+  const allow = new Set(opts.allow ?? [])
+  const files = opts.files ?? envFiles(dir)
+  const all = []
+  for (const file of files) for (const v of parseDotenv(readFileSync(file, 'utf8'))) all.push({ ...v, source: file })
+  // CI: secrets usually arrive as environment variables, not files. Only take
+  // the ones whose names look secret, so PATH and friends are never searched for.
+  for (const [name, value] of Object.entries(opts.env ?? {})) {
+    if (value && looksSecretName(name) && !isPublicName(name)) all.push({ name, value, line: 0, ok: false, source: 'environment' })
+  }
+  // A value given to a public name is public on purpose, unless that public name
+  // itself sounds secret (VITE_STRIPE_SECRET): then it is the leak, not the proof.
+  const risky = (v) => isPublicName(v.name) && looksSecretName(stripPrefix(v.name))
+  const publicValues = new Set(all.filter((v) => isPublicName(v.name) && (v.ok || !risky(v))).map((v) => v.value))
+  const seen = new Set()
+  const secrets = []
+  for (const v of all) {
+    if (v.ok || allow.has(v.name)) continue
+    if (isPublicName(v.name) ? !risky(v) : looksSafeName(v.name)) continue
+    if (trivial(v.value) || publicValues.has(v.value) || seen.has(v.value)) continue
+    seen.add(v.value)
+    secrets.push(v)
+  }
+  return { secrets, all }
+}
+
+/** Public-prefixed names that look secret: { name, source, line }. */
+export function riskyNames(all, allow = []) {
+  const allowed = new Set(allow)
+  return all.filter((v) => isPublicName(v.name) && !v.ok && !allowed.has(v.name) && looksSecretName(stripPrefix(v.name)))
+}
+
+/** Shows enough of a secret to recognise it, never the whole thing. */
+export function mask(value) {
+  return value.length <= 8 ? '****' : `${value.slice(0, 4)}…${value.slice(-2)} (${value.length} chars)`
+}
+
+/** One .env line that holds a server secret: not public, not a known browser key, not trivial, not marked shipped-ok. */
+export function isSecretLine(v) {
+  if (v.ok) return false
+  const risky = isPublicName(v.name) && looksSecretName(stripPrefix(v.name))
+  if (isPublicName(v.name) ? !risky : looksSafeName(v.name)) return false
+  return !trivial(v.value)
+}

package/src/history.js

@@ -0,0 +1,87 @@
+// Searches every commit in the repository: lines that were ever added, for known
+// key shapes and for the project's current secret values, and every .env file
+// that was ever committed with a server secret in it, even if it was deleted later. A secret that was
+// committed once is in every clone of the repository, deleted or not.
+
+import { spawn } from 'node:child_process'
+import { createInterface } from 'node:readline'
+import { findShapes, withKey } from './shapes.js'
+import { isSecretLine, mask, parseDotenv } from './env.js'
+import { forms, keyOf } from './scan.js'
+
+const ENV_FILE = /(^|\/)(\.env(\.[^/]*)?|\.dev\.vars)$/
+const SAFE_ENV = /\.(example|sample|template|dist|defaults)$/i
+
+/**
+ * @param {string} root a folder inside the repository
+ * @param {{ name: string, value: string }[]} secrets current values to look for
+ * @returns {Promise<{ findings: object[], commits: number } | null>} null when root is not a git repository
+ */
+export function scanHistory(root, secrets = []) {
+  return new Promise((resolve) => {
+    // Every commit on every branch, oldest last, with the added lines of each diff.
+    const git = spawn('git', ['log', '--all', '-p', '--no-color', '--no-ext-diff', '--unified=0', '--format=@@commit %H %ad', '--date=short'], { cwd: root })
+    const findings = []
+    const reported = new Set()
+    let commits = 0
+    let commit = ''
+    let date = ''
+    let file = ''
+    let line = 0
+    let envFile = false
+    let failed = false
+    git.on('error', () => {
+      failed = true
+    })
+    git.stderr.on('data', (d) => {
+      if (/not a git repository/i.test(String(d))) failed = true
+    })
+    const add = (f) => {
+      const key = `${f.kind}|${f.sample}|${f.file}`
+      if (reported.has(key)) return // the same secret in the same file: report the newest commit only
+      reported.add(key)
+      findings.push(f)
+    }
+    createInterface({ input: git.stdout, crlfDelay: Infinity }).on('line', (text) => {
+      if (text.startsWith('@@commit ')) {
+        ;[, commit, date] = text.split(' ')
+        commits++
+        return
+      }
+      if (text.startsWith('+++ b/')) {
+        file = text.slice(6)
+        envFile = ENV_FILE.test(file) && !SAFE_ENV.test(file)
+        return
+      }
+      const hunk = /^@@ -\d+(?:,\d+)? \+(\d+)/.exec(text)
+      if (hunk) {
+        line = Number(hunk[1])
+        return
+      }
+      if (!text.startsWith('+') || text.startsWith('+++')) return
+      const added = text.slice(1)
+      // A committed .env file counts only for the server secrets it held, not for VITE_EDITION=web.
+      if (envFile) {
+        for (const v of parseDotenv(added)) {
+          if (!isSecretLine(v)) continue
+          add(keyOf({ level: 'error', kind: 'history-env', file, line, commit: commit.slice(0, 8), date, what: `${v.name} was committed in ${file} (${commit.slice(0, 8)}, ${date}). Every clone has it, even if the file was deleted later.`, sample: mask(v.value) }, v.value))
+        }
+        line++
+        return
+      }
+      for (const s of secrets) {
+        for (const { text: form, how } of forms(s.value)) {
+          if (added.includes(form)) {
+            add(keyOf({ level: 'error', kind: 'history-value', file, line, commit: commit.slice(0, 8), date, what: `${s.name}${how ? ` (${how.slice(2)})` : ''} was committed in ${file} (${commit.slice(0, 8)}, ${date})`, sample: mask(s.value) }, s.value))
+          }
+        }
+      }
+      for (const f of findShapes(added)) {
+        if (secrets.some((s) => f.match.includes(s.value) || s.value.includes(f.match))) continue
+        add(withKey({ level: f.shape.level, kind: 'history-shape', file, line, commit: commit.slice(0, 8), date, what: `${f.shape.name} was committed in ${file} (${commit.slice(0, 8)}, ${date})`, sample: mask(f.match), shapeId: f.shape.id }, f.match))
+      }
+      line++
+    })
+    git.on('close', () => resolve(failed ? null : { findings, commits }))
+  })
+}

package/src/index.d.ts

@@ -0,0 +1,29 @@
+export type Finding = {
+  level: 'error' | 'warn'
+  kind: 'value' | 'shape' | 'name' | 'sourcemap' | 'history-value' | 'history-shape' | 'history-env'
+  file: string
+  line: number
+  what: string
+  sample?: string
+  shapeId?: string
+  /** History findings: the short commit and its date. */
+  commit?: string
+  date?: string
+  /** Set by verifyFindings: did the provider accept the key? */
+  verified?: 'live' | 'dead' | 'unknown'
+}
+export type Result = { findings: Finding[]; files: number; dirs?: string[]; secretsChecked?: number; commits?: number; verified?: boolean }
+export type Secret = { name: string; value: string; source: string; line: number; ok: boolean }
+
+export declare function scanProject(root: string, opts?: { dirs?: string[]; allow?: string[]; env?: Record<string, string | undefined>; envFiles?: string[] }): Result
+export declare function scanUrl(url: string, secrets?: Secret[]): Promise<Result>
+export declare function scanHistory(root: string, secrets?: Secret[]): Promise<{ findings: Finding[]; commits: number } | null>
+export declare function verifyKey(shapeId: string, key: string, fetch?: typeof globalThis.fetch): Promise<'live' | 'dead' | 'unknown'>
+export declare function verifyFindings(findings: Finding[], fetch?: typeof globalThis.fetch): Promise<Finding[]>
+export declare const VERIFIABLE: string[]
+export declare function scanText(text: string, secrets: Secret[]): Omit<Finding, 'file'>[]
+export declare function findOutputDirs(root: string): string[]
+export declare function loadSecrets(dir: string, opts?: { env?: Record<string, string | undefined>; files?: string[]; allow?: string[] }): { secrets: Secret[]; all: Secret[] }
+export declare function report(result: Result): string
+export declare function mask(value: string): string
+export declare const PUBLIC_PREFIXES: string[]

package/src/index.js

@@ -0,0 +1,47 @@
+// shipped-secrets: find secrets that ship to the browser, or sit in git history.
+export { scanProject, scanText, findOutputDirs } from './scan.js'
+export { scanUrl } from './url.js'
+export { scanHistory } from './history.js'
+export { verifyKey, verifyFindings, VERIFIABLE } from './verify.js'
+export { loadSecrets, riskyNames, parseDotenv, envFiles, mask, PUBLIC_PREFIXES } from './env.js'
+export { SHAPES, findShapes } from './shapes.js'
+
+const ORDER = { error: 0, warn: 1 }
+const VERDICT = { live: 'LIVE, rotate now', dead: 'dead', unknown: 'not checked' }
+
+/** Plain-text report, worst first. Secrets are always masked. */
+export function report(result) {
+  const lines = []
+  const live = (f) => (f.verified === 'live' ? 0 : 1)
+  const findings = [...result.findings].sort((a, b) => ORDER[a.level] - ORDER[b.level] || live(a) - live(b) || String(a.file).localeCompare(String(b.file)) || a.line - b.line)
+  // The same secret twice on one line is one line of output.
+  const counts = new Map()
+  for (const f of findings) {
+    const key = `${f.level}|${f.file}|${f.line}|${f.what}|${f.sample ?? ''}|${f.verified ?? ''}`
+    counts.set(key, (counts.get(key) ?? 0) + 1)
+  }
+  for (const [key, n] of counts) {
+    const [level, file, line, what, sample, verified] = key.split('|')
+    const where = line !== '0' ? `${file}:${line}` : file
+    lines.push(`${level === 'error' ? 'ERROR' : 'warn '} ${where}  ${what}${sample ? `  [${sample}]` : ''}${verified ? `  (${VERDICT[verified]})` : ''}${n > 1 ? `  (${n} times)` : ''}`)
+  }
+  const errors = findings.filter((f) => f.level === 'error')
+  const history = errors.filter((f) => f.kind.startsWith('history'))
+  const shipped = errors.filter((f) => !f.kind.startsWith('history'))
+  const distinct = (list) => new Set(list.map((f) => `${f.what}|${f.sample}`)).size
+  const plural = (n, word) => `${n} ${word}${n === 1 ? '' : 's'}`
+  const warns = findings.length - errors.length
+  lines.push('')
+  if (shipped.length) {
+    const n = distinct(shipped)
+    const places = new Set(shipped.map((f) => `${f.file}:${f.line}`)).size
+    lines.push(`${plural(n, 'secret')} shipped to the browser, in ${plural(places, 'place')}. Rotate ${n === 1 ? 'it' : 'them'}: deleting a key from the code does not take it back from anyone who already downloaded it.`)
+  } else if (result.dirs?.length || result.files) lines.push('No secrets found in what ships to the browser.')
+  if (history.length) {
+    lines.push(`${plural(new Set(history.map((f) => f.sample ?? f.file)).size, 'secret')} in git history. Rotate ${history.length === 1 ? 'it' : 'them'}: every clone of the repository has ${history.length === 1 ? 'it' : 'them'}, even after a delete.`)
+  } else if (result.commits != null) lines.push(`No secrets found in ${plural(result.commits, 'commit')} of git history.`)
+  const liveCount = new Set(errors.filter((f) => f.verified === 'live').map((f) => f.sample)).size
+  if (result.verified && errors.some((f) => f.verified)) lines.push(liveCount ? `${plural(liveCount, 'key')} still work${liveCount === 1 ? 's' : ''}: rotate ${liveCount === 1 ? 'it' : 'them'} first.` : 'None of the keys that could be checked still work.')
+  if (warns) lines.push(`${plural(warns, 'warning')} to check.`)
+  return lines.join('\n')
+}

package/src/scan.js

@@ -0,0 +1,142 @@
+// Finds what a site ships to the browser and looks through it for secrets.
+
+import { existsSync, readdirSync, readFileSync, statSync } from 'node:fs'
+import { extname, join, relative, sep } from 'node:path'
+import { findShapes, withKey } from './shapes.js'
+import { loadSecrets, mask, riskyNames } from './env.js'
+
+// Where each framework puts what the browser downloads, most specific first.
+const OUTPUT_DIRS = [
+  '.next/static', // Next.js (.next/server is server code and is never scanned)
+  'out', // Next.js static export
+  '.output/public', // Nuxt, Nitro
+  '.svelte-kit/output/client', // SvelteKit
+  '.vercel/output/static', // Vercel build output
+  'dist/client', // Vite and Astro with a server build next to it
+  'dist',
+  'build/client', // React Router, Remix
+  'build',
+  'storybook-static',
+]
+// Server-side output inside an otherwise browser folder.
+const SERVER_PARTS = new Set(['server', 'ssr', '.server'])
+const TEXT = new Set(['.js', '.mjs', '.cjs', '.css', '.html', '.htm', '.map', '.json', '.txt', '.xml', '.webmanifest', '.svg'])
+
+/** The folders of browser output found under the project folder. */
+export function findOutputDirs(root) {
+  const found = []
+  for (const d of OUTPUT_DIRS) {
+    const p = join(root, d)
+    if (!existsSync(p) || !statSync(p).isDirectory()) continue
+    // dist and build are skipped when their client half was already taken.
+    if (found.some((f) => p.startsWith(f + sep) || f.startsWith(p + sep))) continue
+    found.push(p)
+  }
+  return found
+}
+
+function walk(dir, out = [], exts = TEXT) {
+  for (const name of readdirSync(dir)) {
+    const p = join(dir, name)
+    const st = statSync(p)
+    if (st.isDirectory()) {
+      if (!SERVER_PARTS.has(name) && name !== 'node_modules') walk(p, out, exts)
+    } else if (exts.has(extname(name).toLowerCase()) && st.size < 20 * 1024 * 1024) out.push(p)
+  }
+  return out
+}
+
+// Next.js keeps prerendered pages under .next/server, and they go straight to
+// visitors: the HTML and the RSC payload. Only those are read there, never the
+// server's own JavaScript.
+const NEXT_PAGES = ['.next/server/app', '.next/server/pages']
+const PAGE_FILES = new Set(['.html', '.rsc', '.body', '.meta'])
+function nextPages(root) {
+  const out = []
+  for (const d of NEXT_PAGES) {
+    const p = join(root, d)
+    if (existsSync(p)) walk(p, out, PAGE_FILES)
+  }
+  return out
+}
+
+const lineAt = (text, index) => text.slice(0, index).split('\n').length
+
+/**
+ * Scans text that a browser receives. Returns findings without the file name.
+ * @param {string} text
+ * @param {{ name: string, value: string, source: string }[]} secrets
+ */
+/** A secret value that has a known shape can be checked by --verify too. */
+export function keyOf(finding, value) {
+  const shape = findShapes(value)[0]
+  return shape && shape.match === value ? withKey(Object.assign(finding, { shapeId: shape.shape.id }), value) : finding
+}
+
+/**
+ * The value as written, and base64 of it, so a key put through btoa() or
+ * Buffer.toString('base64') on its way into the bundle is still found. The
+ * padding is dropped, so it matches with or without the trailing "=".
+ */
+export function forms(value) {
+  const b64 = Buffer.from(value, 'utf8').toString('base64').replace(/=+$/, '')
+  const url = b64.replace(/\+/g, '-').replace(/\//g, '_')
+  return [{ text: value, how: '' }, { text: b64, how: ', base64' }, ...(url !== b64 ? [{ text: url, how: ', base64url' }] : [])]
+}
+
+export function scanText(text, secrets) {
+  const found = []
+  for (const s of secrets) {
+    for (const { text: form, how } of forms(s.value)) {
+      let at = text.indexOf(form)
+      while (at !== -1) {
+        found.push(keyOf({ level: 'error', kind: 'value', what: `${s.name} from ${s.source.split(/[\\/]/).pop()}${how}`, sample: mask(s.value), line: lineAt(text, at) }, s.value))
+        at = text.indexOf(form, at + form.length)
+        if (found.length > 50) break
+      }
+    }
+  }
+  for (const f of findShapes(text)) {
+    // A value already reported by name is not reported twice as a shape.
+    if (secrets.some((s) => f.match.includes(s.value) || s.value.includes(f.match))) continue
+    found.push(withKey({ level: f.shape.level, kind: 'shape', what: f.shape.name, sample: mask(f.match), line: lineAt(text, f.index), shapeId: f.shape.id }, f.match))
+  }
+  return found
+}
+
+/**
+ * Scans a project folder: its env files for risky names, and its browser output
+ * for real secret values and known key shapes.
+ * @param {string} root
+ * @param {{ dirs?: string[], allow?: string[], env?: Record<string, string | undefined>, envFiles?: string[] }} [opts]
+ */
+export function scanProject(root, opts = {}) {
+  const { secrets, all } = loadSecrets(root, { env: opts.env ?? process.env, files: opts.envFiles, allow: opts.allow })
+  const findings = []
+  for (const v of riskyNames(all, opts.allow)) {
+    findings.push({
+      level: 'warn', kind: 'name', file: relative(root, v.source) || v.source, line: v.line,
+      what: `${v.name} has a public prefix, so its value goes to every visitor. If it is a secret, drop the prefix and use it on the server only. If it is meant to be public, add "# shipped-ok" to the line.`,
+    })
+  }
+  const dirs = (opts.dirs ?? findOutputDirs(root)).map((d) => (d.startsWith(root) ? d : join(root, d)))
+  let files = 0
+  let maps = 0
+  const lists = dirs.map((d) => walk(d))
+  if (!opts.dirs) lists.push(nextPages(root))
+  for (const list of lists) {
+    for (const file of list) {
+      files++
+      if (file.endsWith('.map')) maps++
+      const text = readFileSync(file, 'utf8')
+      for (const f of scanText(text, secrets)) findings.push(Object.assign(f, { file: relative(root, file) }))
+    }
+  }
+  if (maps) {
+    findings.push({
+      level: 'warn', kind: 'sourcemap', file: dirs.map((d) => relative(root, d)).join(', '), line: 0,
+      what: `${maps} source map${maps === 1 ? '' : 's'} ship to the public. Anyone can read your original source code from them. Turn them off for production or keep them off the public server.`,
+    })
+  }
+  return { findings, dirs: dirs.map((d) => relative(root, d) || '.'), files, secretsChecked: secrets.length }
+}

package/src/shapes.js

@@ -0,0 +1,55 @@
+// What known secret keys look like. Publishable keys that are meant for browsers
+// (Stripe pk_, Supabase anon and sb_publishable_) are never reported.
+// level: error = a server secret, warn = often fine in a browser, check it.
+
+const b64url = (s) => Buffer.from(s.replace(/-/g, '+').replace(/_/g, '/'), 'base64').toString('utf8')
+
+export const SHAPES = [
+  { id: 'stripe-secret', name: 'Stripe secret or restricted key', level: 'error', re: /\b(?:sk|rk)_(?:live|test)_[0-9A-Za-z]{20,}\b/g },
+  { id: 'supabase-secret', name: 'Supabase secret key', level: 'error', re: /\bsb_secret_[0-9A-Za-z_-]{20,}\b/g },
+  {
+    id: 'supabase-service-role', name: 'Supabase service_role key (bypasses row level security)', level: 'error',
+    re: /\beyJ[A-Za-z0-9_-]{10,}\.eyJ[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\b/g,
+    // A Supabase key is a JWT. Only the service_role one is a secret; the anon one is meant for browsers.
+    check(match) {
+      try {
+        const payload = JSON.parse(b64url(match.split('.')[1]))
+        return payload && payload.role === 'service_role'
+      } catch {
+        return false
+      }
+    },
+  },
+  { id: 'anthropic', name: 'Anthropic API key', level: 'error', re: /\bsk-ant-(?:api|admin)\d{2}-[0-9A-Za-z_-]{20,}\b/g },
+  { id: 'openai', name: 'OpenAI API key', level: 'error', re: /\bsk-(?!ant-)(?:proj-|svcacct-|admin-)?[0-9A-Za-z_-]{32,}\b/g },
+  { id: 'aws-access-key', name: 'AWS access key ID', level: 'error', re: /\b(?:AKIA|ASIA)[0-9A-Z]{16}\b/g },
+  { id: 'github-token', name: 'GitHub token', level: 'error', re: /\b(?:gh[pousr]_[0-9A-Za-z]{36}|github_pat_[0-9A-Za-z_]{60,})\b/g },
+  { id: 'sendgrid', name: 'SendGrid API key', level: 'error', re: /\bSG\.[0-9A-Za-z_-]{22}\.[0-9A-Za-z_-]{43}\b/g },
+  { id: 'mailgun', name: 'Mailgun API key', level: 'error', re: /\bkey-[0-9a-f]{32}\b/g },
+  { id: 'resend', name: 'Resend API key', level: 'error', re: /\bre_[0-9A-Za-z]{8}_[0-9A-Za-z]{20,}\b/g },
+  { id: 'twilio-api-key', name: 'Twilio API key', level: 'error', re: /\bSK[0-9a-f]{32}\b/g },
+  { id: 'slack-token', name: 'Slack token', level: 'error', re: /\bxox[abprs]-[0-9A-Za-z-]{10,}\b/g },
+  { id: 'private-key', name: 'Private key', level: 'error', re: /-----BEGIN (?:RSA |EC |DSA |OPENSSH |ENCRYPTED )?PRIVATE KEY-----/g },
+  { id: 'database-url', name: 'Database URL with a password', level: 'error', re: /\b(?:postgres(?:ql)?|mysql|mariadb|mongodb(?:\+srv)?|redis|rediss):\/\/[^\s:'"`/@]+:[^\s'"`@]+@[^\s'"`]+/g },
+  // Google browser keys (Maps, Firebase) are usually meant to be public and locked by referrer.
+  { id: 'google-api-key', name: 'Google API key (often public by design: check it is restricted)', level: 'warn', re: /\bAIza[0-9A-Za-z_-]{35}\b/g },
+]
+
+/** Every known secret shape in text: [{ shape, match, index }]. */
+export function findShapes(text) {
+  const out = []
+  for (const shape of SHAPES) {
+    shape.re.lastIndex = 0
+    let m
+    while ((m = shape.re.exec(text))) {
+      if (shape.check && !shape.check(m[0])) continue
+      out.push({ shape, match: m[0], index: m.index })
+    }
+  }
+  return out
+}
+
+/** Keeps the full key on a finding for --verify, hidden from JSON and from printing. */
+export function withKey(finding, key) {
+  return Object.defineProperty(finding, 'key', { value: key, enumerable: false })
+}

package/src/url.js

@@ -0,0 +1,67 @@
+// Scans a live site the way a stranger would: the page, the scripts it loads,
+// the chunks those scripts load, and any source maps they point to.
+
+import { scanText } from './scan.js'
+
+const MAX_FILES = 200
+
+async function get(url) {
+  try {
+    const res = await fetch(url, { redirect: 'follow', headers: { 'user-agent': 'shipped-secrets' } })
+    if (!res.ok) return null
+    const type = res.headers.get('content-type') || ''
+    if (!/text|javascript|json|html|xml|svg/.test(type) && !/\.(m?js|map|json|css|html?)(\?|$)/.test(url)) return null
+    return await res.text()
+  } catch {
+    return null
+  }
+}
+
+function assetsIn(text, base, isHtml) {
+  const urls = new Set()
+  const add = (u) => {
+    try {
+      const abs = new URL(u, base)
+      if (abs.origin === new URL(base).origin) urls.add(abs.href.split('#')[0])
+    } catch {
+      /* not a URL */
+    }
+  }
+  if (isHtml) {
+    for (const m of text.matchAll(/<script[^>]*\bsrc=["']([^"']+)["']/gi)) add(m[1])
+    for (const m of text.matchAll(/<link[^>]*\brel=["'](?:modulepreload|preload|stylesheet)["'][^>]*\bhref=["']([^"']+)["']/gi)) add(m[1])
+    for (const m of text.matchAll(/<link[^>]*\bhref=["']([^"']+)["'][^>]*\brel=["'](?:modulepreload|preload|stylesheet)["']/gi)) add(m[1])
+  } else {
+    // Chunks a bundle loads later, and its source map.
+    for (const m of text.matchAll(/["'`]((?:\.{0,2}\/)?[\w./-]+\.(?:m?js|css))["'`]/g)) add(m[1])
+    for (const m of text.matchAll(/\/[/*]# sourceMappingURL=([^\s*]+)/g)) if (!m[1].startsWith('data:')) add(m[1])
+  }
+  return urls
+}
+
+/**
+ * Scans a deployed site by URL. Only files on the same site are fetched.
+ * @param {string} url
+ * @param {{ name: string, value: string, source: string }[]} [secrets] local values to look for too
+ */
+export async function scanUrl(url, secrets = []) {
+  const findings = []
+  const seen = new Set()
+  const queue = [url]
+  let maps = 0
+  while (queue.length && seen.size < MAX_FILES) {
+    const u = queue.shift()
+    if (seen.has(u)) continue
+    seen.add(u)
+    const text = await get(u)
+    if (text === null) continue
+    const isHtml = u === url || /<html|<!doctype/i.test(text.slice(0, 500))
+    if (/\.map(\?|$)/.test(u)) maps++
+    for (const f of scanText(text, secrets)) findings.push(Object.assign(f, { file: u }))
+    for (const next of assetsIn(text, u, isHtml)) if (!seen.has(next)) queue.push(next)
+  }
+  if (maps) {
+    findings.push({ level: 'warn', kind: 'sourcemap', file: url, line: 0, what: `${maps} source map${maps === 1 ? '' : 's'} can be downloaded from this site. Anyone can read your original source code from them.` })
+  }
+  return { findings, files: seen.size }
+}

package/src/verify.js

@@ -0,0 +1,70 @@
+// Checks whether a found key still works, by asking its own provider with a
+// harmless read-only call, the way TruffleHog does. Opt-in (--verify): the key
+// goes only to the provider that issued it, over HTTPS, and nowhere else.
+//
+// Answers: 'live' (the provider accepted it: rotate it now), 'dead' (refused:
+// revoked or never real), 'unknown' (no safe way to check, or the provider did
+// not answer).
+
+const b64url = (s) => Buffer.from(s.replace(/-/g, '+').replace(/_/g, '/'), 'base64').toString('utf8')
+
+// shape id -> how to ask. Each returns the Response, or null when it cannot ask.
+const CHECKS = {
+  'stripe-secret': (key, f) => f('https://api.stripe.com/v1/balance', { headers: { authorization: `Bearer ${key}` } }),
+  openai: (key, f) => f('https://api.openai.com/v1/models', { headers: { authorization: `Bearer ${key}` } }),
+  anthropic: (key, f) => f('https://api.anthropic.com/v1/models', { headers: { 'x-api-key': key, 'anthropic-version': '2023-06-01' } }),
+  'github-token': (key, f) => f('https://api.github.com/user', { headers: { authorization: `Bearer ${key}`, 'user-agent': 'shipped-secrets' } }),
+  sendgrid: (key, f) => f('https://api.sendgrid.com/v3/scopes', { headers: { authorization: `Bearer ${key}` } }),
+  resend: (key, f) => f('https://api.resend.com/domains', { headers: { authorization: `Bearer ${key}` } }),
+  mailgun: (key, f) => f('https://api.mailgun.net/v3/domains', { headers: { authorization: `Basic ${Buffer.from(`api:${key}`).toString('base64')}` } }),
+  'slack-token': (key, f) => f('https://slack.com/api/auth.test', { method: 'POST', headers: { authorization: `Bearer ${key}` } }),
+  // A Supabase key carries its project ref, so the project's own API can be asked.
+  'supabase-service-role': (key, f) => {
+    try {
+      const ref = JSON.parse(b64url(key.split('.')[1])).ref
+      if (!/^[a-z0-9]{8,40}$/.test(ref ?? '')) return null
+      return f(`https://${ref}.supabase.co/rest/v1/`, { headers: { apikey: key, authorization: `Bearer ${key}` } })
+    } catch {
+      return null
+    }
+  },
+}
+
+/** Which shape ids can be verified. */
+export const VERIFIABLE = Object.keys(CHECKS)
+
+/**
+ * @param {string} shapeId
+ * @param {string} key
+ * @param {typeof fetch} [f]
+ * @returns {Promise<'live' | 'dead' | 'unknown'>}
+ */
+export async function verifyKey(shapeId, key, f = globalThis.fetch) {
+  const ask = CHECKS[shapeId]
+  if (!ask) return 'unknown'
+  try {
+    const res = await ask(key, (url, init) => f(url, { ...init, signal: AbortSignal.timeout(10_000) }))
+    if (!res) return 'unknown'
+    if (shapeId === 'slack-token') return (await res.json()).ok ? 'live' : 'dead'
+    if (res.status === 200) return 'live'
+    if (res.status === 401 || res.status === 403) return 'dead'
+    return 'unknown'
+  } catch {
+    return 'unknown'
+  }
+}
+
+/**
+ * Adds a verdict to every shape finding that carries its full key.
+ * @param {{ shapeId?: string, key?: string, what: string }[]} findings
+ */
+export async function verifyFindings(findings, f = globalThis.fetch) {
+  const asked = new Map()
+  for (const finding of findings) {
+    if (!finding.shapeId || !finding.key) continue
+    const id = `${finding.shapeId}|${finding.key}`
+    if (!asked.has(id)) asked.set(id, verifyKey(finding.shapeId, finding.key, f))
+    finding.verified = await asked.get(id)
+  }
+  return findings
+}

package/src/vite.d.ts

@@ -0,0 +1 @@
+export default function shippedSecrets(opts?: { allow?: string[]; failOnWarn?: boolean }): { name: string; apply: 'build' }

package/src/vite.js

@@ -0,0 +1,34 @@
+// Vite plugin: after every production build, scan what was built and fail the
+// build if a secret is in it. Warnings are printed and do not fail the build.
+//
+//   import shippedSecrets from 'shipped-secrets/vite'
+//   export default { plugins: [shippedSecrets()] }
+
+import { report, scanProject } from './index.js'
+
+/**
+ * @param {{ allow?: string[], failOnWarn?: boolean }} [opts]
+ */
+export default function shippedSecrets(opts = {}) {
+  let root = process.cwd()
+  let outDir = 'dist'
+  let isBuild = false
+  return {
+    name: 'shipped-secrets',
+    apply: 'build',
+    configResolved(config) {
+      root = config.root
+      outDir = config.build.outDir
+      isBuild = config.command === 'build' && !config.build.ssr
+    },
+    closeBundle() {
+      if (!isBuild) return
+      const result = scanProject(root, { dirs: [outDir], allow: opts.allow })
+      if (!result.findings.length) return
+      const text = report(result)
+      const fail = result.findings.some((f) => f.level === 'error') || (opts.failOnWarn && result.findings.length)
+      if (fail) throw new Error(`shipped-secrets stopped the build:\n${text}`)
+      console.warn(`shipped-secrets:\n${text}`)
+    },
+  }
+}