rvlib-shipkit 0.36.0 → 0.37.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,12 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.37.0
4
+
5
+ - **The leak guard grew an allowlist, because it had no way to be wrong.** Every keyword matched as a case-insensitive substring over the staged added lines, so a four letter row landed inside a real npm dependency name and inside a base64 integrity hash in `bun.lock`, and that lockfile became uncommittable forever with no per repo override of any kind. The only way past was `--no-verify`, which disarms every other gate too, so the guard was training people to switch it off. Two answers ship, in this order. A `w:<word>` keyword row now matches on word boundaries spelled in unicode rather than with `\b`, which is ascii only and would let an accented name match inside a longer accented word: that makes the PATTERN smarter rather than the guard looser, and it is opt in, so a keyword file that carries no `w:` row behaves exactly as before. Then `repo.config.ts hooks.leakAllow` (with the machine-global twin `~/.rv-private/leak-allow.json`) declares a known-good match: `paths` names the files, `patterns` names the keywords, and `inHashes` narrows to a match that falls inside a content hash, where the bytes are an address rather than prose. `paths` plus `patterns` is the shape to reach for, because it waives one keyword in named files while the same keyword still blocks everywhere else, and a single occurrence outside a hash refuses an `inHashes` entry outright.
6
+ - **A waiver is never silent.** Each entry carries a required `reason`, every waiver that fires PRINTS with that reason and the entry that owns it (on the FAIL path too, so a partial waive is visible beside what still blocks), and `hooks list` counts the declared waivers. An entry with no reason, with neither `paths` nor `patterns`, or with an unknown key dies at config load through `defineRepo`, and an allowlist file that will not parse waives nothing and WARNs, so a broken waiver leaves the guard stricter rather than looser.
7
+ - **A tracked allowlist never becomes a copy of the private keyword list.** An entry names a keyword by FINGERPRINT (`fp:` plus 12 hex of its sha256), printed beside every hit, which is how a waiver gets written without anyone typing the private word into git. `repo.config.ts` takes the fingerprint form alone and refuses a literal by name. The private machine-global file may use either. The honest limit is stated in the source: a fingerprint stops the list being read off a tracked file and is no defence against a targeted guess.
8
+ - The staged diff is now parsed per FILE (`addedLinesByFile`), so a hit says which file it came from and a waiver can name one. Which file a line belongs to is decided by COUNTING each hunk body from the size its header declares, never by trusting a `+++ …` line wherever it appears: an added line whose own text starts with `++` reaches the parser looking exactly like a file header, and believing one inside a hunk would let ordinary looking text drop every added line after it (`++ /dev/null`) or file the rest under a name a waiver covers (`++ b/bun.lock`). Substring rows match through a case-insensitive regex instead of `indexOf` over a lowercased copy, because one unicode character lowercases into two and slides every offset after it, and those offsets are what decide whether a match sits inside a content hash. A content-hash span is consulted ONLY by an entry that asked for `inHashes`: an api token is also a long base64 run, so skipping those spans by default would silence the guard on the exact shape it exists to catch.
9
+
3
10
  ## 0.36.0
4
11
 
5
12
  - **`shipkit smoke` — verify the way the end user does, after a deploy.** A read only Playwright pass: it logs in through the app's OWN login page, visits every route `repo.config.ts smoke` declares, samples the body text until it stops growing (a `waitFor` selector first, because a spinner holds the length constant), counts the selector's matches, measures whether the PAGE scrolls sideways, screenshots each route under `tmp/smoke/`, and collects console errors, page errors, failed requests and 4xx/5xx responses. The exit code is the number of failed routes. The password is a secret NAME, read through the repo's new `credsReadCommand` template (the READ twin of `credsCommand`, `{name}` and `{field}` substituted, no shell), never argv, env or config, and `defineRepo` refuses a value typed where the name belongs and a login declared without the reader. Playwright is resolved from the CONSUMER repo at run time, never carried by this package: a repo without it gets one line naming `bun add -d playwright && npx playwright install chromium`. When the host sends `x-pockethost-ratelimit-ip-hourly-*` headers, the report prints the budget and labels a CORS-shaped failure as the backend refusing rather than the app failing, telling the hourly quota from the concurrent cap by the remaining count. Between routes it pauses `paceMs` (default 3000, measured against a 15-concurrent cap: 0 reported 8 of 12 routes, 3000 reports 12 of 12). A route with `open` (a link selector) follows the first match's href after it and judges the landed page as its own row, which is how a detail page with a real id gets visited without the config knowing one. The login form lives at `login.path`, default `/login`. A problem collected during the login is a failed row of its own, and the pause runs before the row label moves, so a previous page's late failure lands on the page that caused it. `emailSecret` is refused when the reader template carries no `{field}`: the account read would answer the value, and the run would type the password into the identity box, where a failed-login journal keeps it.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "rvlib-shipkit",
3
- "version": "0.36.0",
3
+ "version": "0.37.0",
4
4
  "funding": "https://github.com/sponsors/rvion",
5
5
  "description": "All-in-one repo companion: ci gate (task DAG), auto-discovered health checks, release-contract fulfillness, asks queue, md-wiki tools, asset atelier, live tui",
6
6
  "license": "MIT",
@@ -4,10 +4,23 @@
4
4
  //
5
5
  // the keyword file is deliberately untracked (a list of things to grep for
6
6
  // is itself private). absent file → loud skip, never a silent pass.
7
+ //
8
+ // a keyword matches as a SUBSTRING, which is what catches a private name
9
+ // glued into a hostname or an identifier. The cost is a false positive a repo
10
+ // cannot escape: a four letter row lands inside a dependency name or inside a
11
+ // base64 integrity hash in a lockfile, and that lockfile is uncommittable
12
+ // forever. A guard that cries wolf is a guard people answer with --no-verify,
13
+ // which protects nothing. Two answers, in this order. `w:<word>` rows match on
14
+ // word boundaries, which makes the PATTERN smarter rather than the guard
15
+ // looser. An ALLOWLIST waives a known-good match in a named context, and every
16
+ // waiver that fires is PRINTED with its reason, so a reader sees what was let
17
+ // through and why.
18
+ import { createHash } from 'crypto'
7
19
  import { existsSync, readFileSync } from 'fs'
8
- import { join } from 'path'
20
+ import { basename, join } from 'path'
9
21
 
10
- export type LeakHit = { where: string; pattern: string }
22
+ export type LeakHit = { where: string; pattern: string; file?: string }
23
+ export type LeakWaiver = LeakHit & { reason: string; source: string }
11
24
 
12
25
  /** what a hit REPORTS. The matched string is private by construction, and a
13
26
  * gate that prints it verbatim puts it in a terminal, a CI log and any paste
@@ -15,16 +28,66 @@ export type LeakHit = { where: string; pattern: string }
15
28
  export const maskPattern = (p: string): string =>
16
29
  p.startsWith('re:') || p.length <= 6 ? p : `${p.slice(0, 3)}…${p.slice(-2)} (${p.length} chars)`
17
30
 
31
+ /** a stable name for a pattern that does not CARRY it, so a tracked allowlist
32
+ * points at a private keyword without becoming a copy of the private keyword
33
+ * list. Printed beside every hit, which is how a waiver gets written without
34
+ * anyone typing the word.
35
+ *
36
+ * Honest scope: this stops the list being READ off a tracked file. It is no
37
+ * defence against a targeted guess, because a short keyword has a small
38
+ * search space and whoever already suspects the word can confirm it. */
39
+ export const patternFingerprint = (pattern: string): string =>
40
+ 'fp:' + createHash('sha256').update(pattern.toLowerCase()).digest('hex').slice(0, 12)
41
+
42
+ /** one known-good match a repo (or this machine) waives.
43
+ *
44
+ * `paths` alone waives every pattern in those files. `patterns` alone waives
45
+ * those patterns everywhere. The PAIR is the shape to reach for: this
46
+ * pattern, in these files, while the same pattern still blocks anywhere else.
47
+ * `inHashes` narrows further, to a match that lands inside a content hash. */
48
+ export type LeakAllowEntry = {
49
+ /** why this match is known good. Printed every time the waiver fires, so
50
+ * a reader months later can judge it. Required. */
51
+ reason: string
52
+ /** staged paths this entry covers: a repo-relative path, a bare basename
53
+ * ('bun.lock'), or a 'dir/' prefix. Absent = every target, the commit
54
+ * message included. */
55
+ paths?: string[]
56
+ /** the patterns this entry waives, named by FINGERPRINT (`fp:` + 12 hex,
57
+ * printed beside every hit). Absent = every pattern, on `paths` alone.
58
+ * A TRACKED source (repo.config.ts) takes the fingerprint form only: a
59
+ * literal there puts the private keyword into git. */
60
+ patterns?: string[]
61
+ /** waive only a match that falls inside a content hash (an npm `sha512-…`
62
+ * integrity field, a 40+ char base64 run). The bytes of a content address
63
+ * are arithmetic, not prose, so a keyword inside one is noise. The same
64
+ * keyword on a normal line of the same file still blocks. */
65
+ inHashes?: boolean
66
+ }
67
+
18
68
  export type LeakGuardResult =
19
69
  | { status: 'skipped'; reason: string }
20
- | { status: 'clean'; patterns: number }
21
- | { status: 'hit'; hits: LeakHit[]; patterns: number }
70
+ | { status: 'clean'; patterns: number; waived: LeakWaiver[]; notes: string[] }
71
+ | { status: 'hit'; hits: LeakHit[]; patterns: number; waived: LeakWaiver[]; notes: string[] }
72
+
73
+ export type ParsedPatterns = {
74
+ substrings: string[]
75
+ words: { word: string; re: RegExp }[]
76
+ regexes: { row: string; re: RegExp }[]
77
+ }
78
+
79
+ const escapeRegex = (s: string): string => s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')
22
80
 
23
81
  /** Parse the keyword file: blank lines and `#` comments dropped, `re:<pattern>`
24
- * rows become case-insensitive regexes, everything else a case-insensitive
25
- * substring. */
26
- export function parsePatterns(text: string): { substrings: string[]; regexes: { row: string; re: RegExp }[] } {
82
+ * rows become case-insensitive regexes, `w:<word>` rows match on word
83
+ * boundaries, everything else a case-insensitive substring.
84
+ *
85
+ * A `w:` row REPORTS as the bare word, so its mask and its fingerprint are
86
+ * the same with or without the prefix: adding the prefix to a row never
87
+ * invalidates a waiver already written against it. */
88
+ export function parsePatterns(text: string): ParsedPatterns {
27
89
  const substrings: string[] = []
90
+ const words: { word: string; re: RegExp }[] = []
28
91
  const regexes: { row: string; re: RegExp }[] = []
29
92
  for (const raw of text.split('\n')) {
30
93
  const row = raw.trim()
@@ -32,24 +95,134 @@ export function parsePatterns(text: string): { substrings: string[]; regexes: {
32
95
  if (row.startsWith('re:')) {
33
96
  try { regexes.push({ row, re: new RegExp(row.slice(3), 'i') }) }
34
97
  catch { /* a malformed row must not disable the whole guard */ }
98
+ } else if (row.startsWith('w:')) {
99
+ const word = row.slice(2).trim().toLowerCase()
100
+ // \b is ascii-only in JS, which lets an accented name match inside
101
+ // a longer accented word: the boundary is spelled in unicode
102
+ if (word !== '') words.push({ word, re: new RegExp(`(?<![\\p{L}\\p{N}_])${escapeRegex(word)}(?![\\p{L}\\p{N}_])`, 'giu') })
35
103
  } else substrings.push(row.toLowerCase())
36
104
  }
37
- return { substrings, regexes }
105
+ return { substrings, words, regexes }
38
106
  }
39
107
 
40
- function scan(haystack: string, where: string, p: ReturnType<typeof parsePatterns>, hits: LeakHit[]): void {
41
- const lower = haystack.toLowerCase()
42
- for (const s of p.substrings) if (lower.includes(s)) hits.push({ where, pattern: s })
43
- for (const r of p.regexes) if (r.re.test(haystack)) hits.push({ where, pattern: r.row })
108
+ type Span = [start: number, end: number]
109
+
110
+ // a substring row is matched by a case-insensitive REGEX rather than by
111
+ // indexOf over a lowercased copy: one unicode character (İ) lowercases into
112
+ // two, which slides every offset after it, and the offsets are what decide
113
+ // whether a match sits inside a content hash. Compiled once per keyword.
114
+ const subRe = new Map<string, RegExp>()
115
+ const substringRegex = (s: string): RegExp => {
116
+ const hit = subRe.get(s)
117
+ if (hit != null) return hit
118
+ const re = new RegExp(escapeRegex(s), 'gi')
119
+ subRe.set(s, re)
120
+ return re
44
121
  }
45
122
 
46
- export type LeakGuardOpts = {
47
- /** path to the keyword file, relative to root (default '.rv-private/banned-keywords.txt') */
48
- keywordFile?: string
49
- /** extra text to scan (a commit message, a release note) */
50
- text?: { label: string; content: string }[]
51
- /** scan the staged diff + staged paths (default true) */
52
- staged?: boolean
123
+ /** every occurrence of every pattern, keyed by the id a hit reports. */
124
+ function occurrences(haystack: string, p: ParsedPatterns): Map<string, Span[]> {
125
+ const found = new Map<string, Span[]>()
126
+ const push = (id: string, span: Span): void => {
127
+ const prev = found.get(id)
128
+ if (prev == null) found.set(id, [span])
129
+ else prev.push(span)
130
+ }
131
+ for (const s of p.substrings) for (const m of haystack.matchAll(substringRegex(s))) push(s, [m.index, m.index + m[0].length])
132
+ for (const w of p.words) for (const m of haystack.matchAll(w.re)) push(w.word, [m.index, m.index + m[0].length])
133
+ for (const r of p.regexes) {
134
+ const g = new RegExp(r.re.source, r.re.flags.includes('g') ? r.re.flags : r.re.flags + 'g')
135
+ for (const m of haystack.matchAll(g)) push(r.row, [m.index, m.index + m[0].length])
136
+ }
137
+ return found
138
+ }
139
+
140
+ // an npm/yarn integrity field, and any long base64 run. ONLY an entry that
141
+ // asked for `inHashes` consults this: an api token is also a long base64 run,
142
+ // so skipping these by default would be the guard going quiet on the exact
143
+ // shape it exists to catch.
144
+ const HASH_SPAN = /(?:sha\d{1,3}|md5)-[A-Za-z0-9+/=]{16,}|[A-Za-z0-9+/]{40,}={0,2}/gi
145
+
146
+ const hashSpans = (text: string): Span[] => [...text.matchAll(HASH_SPAN)].map((m): Span => [m.index, m.index + m[0].length])
147
+
148
+ const inside = (span: Span, spans: Span[]): boolean => spans.some(([a, b]) => span[0] >= a && span[1] <= b)
149
+
150
+ /** a path selector: a repo-relative path, a bare basename, or a 'dir/' prefix. */
151
+ function pathCovered(file: string | undefined, selectors: string[]): boolean {
152
+ if (file == null) return false
153
+ return selectors.some((s) => s === file || (s.endsWith('/') && file.startsWith(s)) || (!s.includes('/') && basename(file) === s))
154
+ }
155
+
156
+ export type LoadedAllow = LeakAllowEntry & { source: string }
157
+
158
+ /** what the guard reads: one file's added lines, one staged path, or a text
159
+ * the caller handed in (a commit message). */
160
+ export type ScanTarget = { where: string; content: string; file?: string }
161
+
162
+ /** the whole decision, with no git and no disk in it: which matches block,
163
+ * which are waived, by whom, and why. */
164
+ export function evaluateLeak(targets: ScanTarget[], p: ParsedPatterns, allow: LoadedAllow[]): { hits: LeakHit[]; waived: LeakWaiver[] } {
165
+ const hits: LeakHit[] = []
166
+ const waived: LeakWaiver[] = []
167
+ // one row per distinct (where, file, pattern): a keyword hit 40 times in
168
+ // one file is one finding
169
+ const seen = new Set<string>()
170
+ for (const t of targets) {
171
+ const found = occurrences(t.content, p)
172
+ if (found.size === 0) continue
173
+ let hashes: Span[] | null = null
174
+ for (const [id, spans] of found) {
175
+ const key = `${t.where} ${t.file ?? ''} ${id}`
176
+ if (seen.has(key)) continue
177
+ seen.add(key)
178
+ const fp = patternFingerprint(id)
179
+ const entry = allow.find((a) => {
180
+ if (a.paths != null && !pathCovered(t.file, a.paths)) return false
181
+ if (a.patterns != null && !a.patterns.some((x) => x === fp || x.toLowerCase() === id)) return false
182
+ if (a.inHashes === true) {
183
+ hashes ??= hashSpans(t.content)
184
+ if (!spans.every((s) => inside(s, hashes!))) return false
185
+ }
186
+ return true
187
+ })
188
+ if (entry == null) hits.push({ where: t.where, pattern: id, file: t.file })
189
+ else waived.push({ where: t.where, pattern: id, file: t.file, reason: entry.reason, source: entry.source })
190
+ }
191
+ }
192
+ return { hits, waived }
193
+ }
194
+
195
+ const ALLOW_KEYS = ['reason', 'paths', 'patterns', 'inHashes']
196
+ const FINGERPRINT = /^fp:[0-9a-f]{12}$/
197
+
198
+ /** why an entry cannot be used, or null. `tracked` = the source is committed
199
+ * (repo.config.ts), where a literal pattern publishes the private word the
200
+ * guard exists to keep out of git. */
201
+ export function leakAllowProblem(entry: unknown, where: string, tracked: boolean): string | null {
202
+ if (entry == null || typeof entry !== 'object' || Array.isArray(entry)) return `${where}: every entry is an object`
203
+ const e = entry as Record<string, unknown>
204
+ for (const k of Object.keys(e)) if (!ALLOW_KEYS.includes(k)) return `${where}: unknown key '${k}' (known: ${ALLOW_KEYS.join(', ')})`
205
+ if (typeof e['reason'] !== 'string' || e['reason'].trim() === '') return `${where}: needs a 'reason' — a waiver nobody can judge later is a hole`
206
+ if (e['paths'] == null && e['patterns'] == null)
207
+ return `${where}: needs 'paths' or 'patterns' — an entry with neither waives every pattern in every file, which is the guard switched off`
208
+ for (const k of ['paths', 'patterns'] as const) {
209
+ const v = e[k]
210
+ if (v == null) continue
211
+ if (!Array.isArray(v) || v.length === 0 || v.some((x) => typeof x !== 'string' || x.trim() === ''))
212
+ return `${where}: '${k}' is a non-empty array of non-empty strings`
213
+ }
214
+ if (e['inHashes'] != null && typeof e['inHashes'] !== 'boolean') return `${where}: 'inHashes' is a boolean`
215
+ if (tracked)
216
+ for (const x of (e['patterns'] as string[] | undefined) ?? [])
217
+ if (!FINGERPRINT.test(x))
218
+ return `${where}: '${x}' — a tracked allowlist names a pattern by FINGERPRINT (fp:…, printed beside every hit), never by value, or the private keyword lands in git`
219
+ return null
220
+ }
221
+
222
+ /** every problem in a declared allowlist. `defineRepo` throws on the first. */
223
+ export function leakAllowProblems(entries: unknown, where: string, tracked: boolean): string[] {
224
+ if (!Array.isArray(entries)) return [`${where}: leakAllow is an array of entries`]
225
+ return entries.map((e, i) => leakAllowProblem(e, `${where}[${i}]`, tracked)).filter((x): x is string => x != null)
53
226
  }
54
227
 
55
228
  /** the machine-global list every repo's guard also reads: one private list
@@ -62,36 +235,138 @@ export function globalKeywordFile(): string | null {
62
235
  return existsSync(p) ? p : null
63
236
  }
64
237
 
238
+ /** the machine-global allowlist, the twin of the keyword file. It is private,
239
+ * so it may name a pattern by value as well as by fingerprint. */
240
+ export function globalAllowFile(): string | null {
241
+ const home = process.env.HOME ?? ''
242
+ if (home === '') return null
243
+ const p = join(home, '.rv-private', 'leak-allow.json')
244
+ return existsSync(p) ? p : null
245
+ }
246
+
247
+ /** read an allowlist file (`LeakAllowEntry[]` as JSON). A file that will not
248
+ * parse, or an entry that cannot be used, is a NOTE and never a failure: a
249
+ * waiver that does not load leaves the guard stricter, never looser. */
250
+ export function loadAllowFile(path: string, tracked: boolean): { entries: LoadedAllow[]; notes: string[] } {
251
+ let parsed: unknown
252
+ try { parsed = JSON.parse(readFileSync(path, 'utf8')) }
253
+ catch (e) { return { entries: [], notes: [`${path} is not readable JSON (${e instanceof Error ? e.message.split('\n')[0] : String(e)}) — its waivers are IGNORED`] } }
254
+ if (!Array.isArray(parsed)) return { entries: [], notes: [`${path} holds an array of allowlist entries — its waivers are IGNORED`] }
255
+ const entries: LoadedAllow[] = []
256
+ const notes: string[] = []
257
+ parsed.forEach((raw, i) => {
258
+ const problem = leakAllowProblem(raw, `${path}[${i}]`, tracked)
259
+ if (problem != null) notes.push(`${problem} — entry IGNORED`)
260
+ else entries.push({ ...(raw as LeakAllowEntry), source: `${basename(path)}[${i}]` })
261
+ })
262
+ return { entries, notes }
263
+ }
264
+
265
+ export type LeakGuardOpts = {
266
+ /** path to the keyword file, relative to root (default '.rv-private/banned-keywords.txt') */
267
+ keywordFile?: string
268
+ /** extra text to scan (a commit message, a release note) */
269
+ text?: { label: string; content: string }[]
270
+ /** scan the staged diff + staged paths (default true) */
271
+ staged?: boolean
272
+ /** the repo's declared waivers (repo.config.ts hooks.leakAllow). Tracked,
273
+ * so fingerprints only. */
274
+ allow?: LeakAllowEntry[]
275
+ /** override the machine-global allowlist path; null reads none */
276
+ allowFile?: string | null
277
+ }
278
+
279
+ /** git c-quotes a path that carries a space or a non-ascii byte. */
280
+ function unquotePath(p: string): string {
281
+ if (!p.startsWith('"')) return p
282
+ try { return JSON.parse(p) as string } catch { return p.slice(1, -1) }
283
+ }
284
+
285
+ /** `@@ -old,n +new,m @@`, the only line that says how long a hunk body is. */
286
+ const HUNK_HEADER = /^@@ -\d+(?:,(\d+))? \+\d+(?:,(\d+))? @@/
287
+
288
+ /** ADDED lines only, grouped by the file that receives them: what this commit
289
+ * puts INTO the object store. Removed and context lines are already in
290
+ * history, and a commit that DELETES private content must never be blocked by
291
+ * that content. The grouping is what lets a waiver name a file.
292
+ *
293
+ * A `+++ x` line is a file header only OUTSIDE a hunk body. Inside one it is
294
+ * an added line whose own text starts with `++`, and reading it as a header
295
+ * is a hole in both directions: `++ /dev/null` would drop every added line
296
+ * after it in that file, and `++ b/bun.lock` would file them under a name a
297
+ * waiver covers. So the hunk body is COUNTED, from the size its header
298
+ * declares. A header this does not recognise (a combined `@@@` diff) opens an
299
+ * unbounded body that only the next file closes, which is the safe direction:
300
+ * content is still scanned, and no line inside it can name a file. */
301
+ export function addedLinesByFile(diff: string): { file: string; content: string }[] {
302
+ const out: { file: string; content: string }[] = []
303
+ let current: { file: string; lines: string[] } | null = null
304
+ let oldLeft = 0
305
+ let newLeft = 0
306
+ const flush = (): void => {
307
+ if (current != null && current.lines.length > 0) out.push({ file: current.file, content: current.lines.join('\n') })
308
+ current = null
309
+ }
310
+ const add = (line: string): void => { if (current != null) current.lines.push(line.slice(1)) }
311
+ for (const line of diff.split('\n')) {
312
+ // a `diff --git ` at column 0 can never be diff CONTENT: a content line
313
+ // always carries its ' ', '+' or '-' prefix first
314
+ if (line.startsWith('diff --git ')) {
315
+ flush()
316
+ oldLeft = 0
317
+ newLeft = 0
318
+ } else if (oldLeft > 0 || newLeft > 0) {
319
+ if (line.startsWith('+')) { newLeft--; add(line) }
320
+ else if (line.startsWith('-')) oldLeft--
321
+ else if (!line.startsWith('\\')) { oldLeft--; newLeft-- } // '' counts for neither
322
+ } else if (line.startsWith('@@')) {
323
+ const h = HUNK_HEADER.exec(line)
324
+ oldLeft = h == null ? Infinity : h[1] == null ? 1 : Number(h[1])
325
+ newLeft = h == null ? Infinity : h[2] == null ? 1 : Number(h[2])
326
+ } else if (line.startsWith('+++ ')) {
327
+ flush()
328
+ const p = unquotePath(line.slice(4).trim())
329
+ current = p === '/dev/null' ? null : { file: p.startsWith('b/') ? p.slice(2) : p, lines: [] }
330
+ } else if (line.startsWith('+')) add(line) // a stray added line outside any hunk is still content
331
+ }
332
+ flush()
333
+ return out
334
+ }
335
+
65
336
  export function runLeakGuard(root: string, o: LeakGuardOpts = {}): LeakGuardResult {
66
337
  const rel = o.keywordFile ?? '.rv-private/banned-keywords.txt'
67
338
  const file = join(root, rel)
68
339
  const sources = [existsSync(file) ? file : null, globalKeywordFile()].filter((f): f is string => f != null)
69
340
  if (sources.length === 0) return { status: 'skipped', reason: `${rel} not found and no ~/.rv-private/banned-keywords.txt — no keyword list, no leak guard` }
70
341
  const patterns = parsePatterns(sources.map((f) => readFileSync(f, 'utf8')).join('\n'))
71
- const count = patterns.substrings.length + patterns.regexes.length
342
+ const count = patterns.substrings.length + patterns.words.length + patterns.regexes.length
72
343
  if (count === 0) return { status: 'skipped', reason: `keyword file(s) have no patterns (${sources.join(', ')})` }
73
344
 
74
- const hits: LeakHit[] = []
345
+ const notes: string[] = []
346
+ const allow: LoadedAllow[] = []
347
+ ;(o.allow ?? []).forEach((e, i) => {
348
+ const problem = leakAllowProblem(e, `repo.config.ts hooks.leakAllow[${i}]`, true)
349
+ if (problem != null) notes.push(`${problem} — entry IGNORED`)
350
+ else allow.push({ ...e, source: `repo.config.ts hooks.leakAllow[${i}]` })
351
+ })
352
+ const globalAllow = o.allowFile === undefined ? globalAllowFile() : o.allowFile
353
+ if (globalAllow != null && existsSync(globalAllow)) {
354
+ const g = loadAllowFile(globalAllow, false)
355
+ allow.push(...g.entries)
356
+ notes.push(...g.notes)
357
+ }
358
+
359
+ const targets: ScanTarget[] = []
75
360
  if (o.staged !== false) {
76
- // staged CONTENT (what lands in the object store) and staged PATHS
77
- // (a filename can leak as loudly as a line of code)
78
361
  const diff = Bun.spawnSync(['git', 'diff', '--cached'], { cwd: root, maxBuffer: 256 * 1024 * 1024 })
79
- if (diff.exitCode === 0) {
80
- // ADDED lines only: what this commit puts INTO the object store.
81
- // removed/context lines are already in history, and a commit that
82
- // deletes private content must never be blocked by that content
83
- const added = diff.stdout.toString().split('\n')
84
- .filter((l) => l.startsWith('+') && !l.startsWith('+++'))
85
- .join('\n')
86
- scan(added, 'staged content', patterns, hits)
87
- }
362
+ if (diff.exitCode === 0)
363
+ for (const f of addedLinesByFile(diff.stdout.toString())) targets.push({ where: 'staged content', file: f.file, content: f.content })
364
+ // a filename can leak as loudly as a line of code
88
365
  const names = Bun.spawnSync(['git', 'diff', '--cached', '--name-only'], { cwd: root })
89
- if (names.exitCode === 0) scan(names.stdout.toString(), 'staged path', patterns, hits)
366
+ if (names.exitCode === 0) for (const n of names.stdout.toString().split('\n')) if (n.trim() !== '') targets.push({ where: 'staged path', file: n, content: n })
90
367
  }
91
- for (const t of o.text ?? []) scan(t.content, t.label, patterns, hits)
368
+ for (const t of o.text ?? []) targets.push({ where: t.label, content: t.content })
92
369
 
93
- // one row per distinct (where, pattern) — a keyword hit 40 times is one finding
94
- const seen = new Set<string>()
95
- const unique = hits.filter((h) => { const k = h.where + ' ' + h.pattern; if (seen.has(k)) return false; seen.add(k); return true })
96
- return unique.length > 0 ? { status: 'hit', hits: unique, patterns: count } : { status: 'clean', patterns: count }
370
+ const { hits, waived } = evaluateLeak(targets, patterns, allow)
371
+ return hits.length > 0 ? { status: 'hit', hits, patterns: count, waived, notes } : { status: 'clean', patterns: count, waived, notes }
97
372
  }
@@ -7,7 +7,7 @@ import { existsSync, mkdirSync, readFileSync, realpathSync, writeFileSync } from
7
7
  import { dirname, join } from 'path'
8
8
  import { fileURLToPath } from 'url'
9
9
  import { commitMessageOffenses } from './commit-message.ts'
10
- import { globalKeywordFile, maskPattern, runLeakGuard } from './leak-guard.ts'
10
+ import { globalAllowFile, globalKeywordFile, maskPattern, patternFingerprint, runLeakGuard, type LeakAllowEntry, type LeakHit, type LeakWaiver } from './leak-guard.ts'
11
11
  import { runRootIdentity } from './root-identity.ts'
12
12
  import { spawnWithTimeout } from '../core/spawn-timeout.ts'
13
13
  import type { RepoConfig } from '../repo-config.ts'
@@ -21,6 +21,14 @@ export type HooksConfig = {
21
21
  commitMsg?: { id: string; cmd: string[] }[]
22
22
  /** keyword file for the leak guard (default '.rv-private/banned-keywords.txt') */
23
23
  keywordFile?: string
24
+ /** known-good leak-guard matches this repo waives: a lockfile whose
25
+ * dependency names or integrity hashes collide with a keyword is the
26
+ * motivating case. Each entry carries a reason, and a waiver that fires
27
+ * PRINTS, so nothing passes silently. This file is tracked, so an entry
28
+ * names a pattern by fingerprint (`fp:…`, printed beside every hit) and
29
+ * never by value. The machine-global twin is ~/.rv-private/leak-allow.json.
30
+ * hooks/leak-guard.ts */
31
+ leakAllow?: LeakAllowEntry[]
24
32
  /** staged-path prefixes that may never enter git (local state like
25
33
  * '.comfy-ts/'). gitignore is advisory and editable; this gate is not. */
26
34
  blockedPaths?: string[]
@@ -148,6 +156,29 @@ function seedKeywordFile(root: string, keywordFile?: string): string | null {
148
156
 
149
157
  export type GateOutcome = { id: string; ok: boolean; lines: string[] }
150
158
 
159
+ /** a hit line: the masked pattern plus its FINGERPRINT, which is the token a
160
+ * waiver is written against, so nobody has to type the private word to
161
+ * exempt it. The file rides along for a content hit; a staged-PATH hit adds
162
+ * nothing, because there the path IS what matched. */
163
+ const hitLine = (x: LeakHit): string =>
164
+ ` ${x.where}${x.where === 'staged content' && x.file != null ? ` ${x.file}` : ''}: ${maskPattern(x.pattern)} [${patternFingerprint(x.pattern)}]`
165
+
166
+ /** what the allowlist let through, one line per waiver. An allowlist that
167
+ * fires invisibly is a silent weakening, so a waived match is as loud as a
168
+ * blocked one and carries the reason its author wrote. */
169
+ function waivedLines(waived: LeakWaiver[]): string[] {
170
+ const rows = new Map<string, { pattern: string; reason: string; source: string; targets: Set<string> }>()
171
+ for (const w of waived) {
172
+ const key = `${w.pattern} ${w.source}`
173
+ const row = rows.get(key) ?? { pattern: w.pattern, reason: w.reason, source: w.source, targets: new Set<string>() }
174
+ row.targets.add(w.file ?? w.where)
175
+ rows.set(key, row)
176
+ }
177
+ return [...rows.values()].map(
178
+ (r) => ` waived ${maskPattern(r.pattern)} [${patternFingerprint(r.pattern)}] in ${r.targets.size} target(s) — ${r.reason} (${r.source})`,
179
+ )
180
+ }
181
+
151
182
  /** the pre-commit gate: built-in leak guard, then the repo's own commands.
152
183
  * A commit takes ~1s (build/shipkit.md): the gate times itself, and going
153
184
  * over budget WARNs naming the fix, so drift back to slow commits is loud. */
@@ -156,16 +187,27 @@ export async function runPreCommit(root: string, config: RepoConfig): Promise<Ga
156
187
  const h = config.hooks ?? {}
157
188
  const out: GateOutcome[] = []
158
189
  if (!(h.disable ?? []).includes('leak-guard')) {
159
- const r = runLeakGuard(root, { keywordFile: h.keywordFile })
190
+ const r = runLeakGuard(root, { keywordFile: h.keywordFile, allow: h.leakAllow })
160
191
  if (r.status === 'skipped') out.push({ id: 'leak-guard', ok: true, lines: [`skip: leak-guard (${r.reason})`] })
161
- else if (r.status === 'clean') out.push({ id: 'leak-guard', ok: true, lines: [`ok: leak-guard clean (${r.patterns} pattern(s))`] })
192
+ else if (r.status === 'clean') out.push({
193
+ id: 'leak-guard',
194
+ ok: true,
195
+ lines: [
196
+ `ok: leak-guard clean (${r.patterns} pattern(s)${r.waived.length === 0 ? '' : `, ${r.waived.length} waived`})`,
197
+ ...waivedLines(r.waived),
198
+ ...r.notes.map((n) => ` WARN ${n}`),
199
+ ],
200
+ })
162
201
  else out.push({
163
202
  id: 'leak-guard',
164
203
  ok: false,
165
204
  lines: [
166
205
  `FAIL: leak-guard matched ${r.hits.length} banned pattern(s) — this commit would leak private content`,
167
- ...r.hits.map((x) => ` ${x.where}: ${maskPattern(x.pattern)}`),
206
+ ...r.hits.map(hitLine),
207
+ ...waivedLines(r.waived),
208
+ ...r.notes.map((n) => ` WARN ${n}`),
168
209
  ` fix: remove the content, or edit the keyword file if the pattern is wrong`,
210
+ ` a known-good match: repo.config.ts hooks.leakAllow += { reason, paths, patterns: ['<the fp: above>'] }`,
169
211
  ],
170
212
  })
171
213
  }
@@ -242,9 +284,11 @@ export async function runCommitMsg(root: string, config: RepoConfig, messageFile
242
284
  // the leak guard also reads the MESSAGE, a private name pasted into a
243
285
  // commit body is as public as one pasted into a file
244
286
  if (!(h.disable ?? []).includes('leak-guard')) {
245
- const r = runLeakGuard(root, { keywordFile: h.keywordFile, staged: false, text: [{ label: 'commit message', content: message }] })
287
+ const r = runLeakGuard(root, { keywordFile: h.keywordFile, staged: false, allow: h.leakAllow, text: [{ label: 'commit message', content: message }] })
246
288
  if (r.status === 'hit')
247
- out.push({ id: 'leak-guard', ok: false, lines: [`FAIL: leak-guard matched in the commit message`, ...r.hits.map((x) => ` ${x.where}: ${maskPattern(x.pattern)}`)] })
289
+ out.push({ id: 'leak-guard', ok: false, lines: [`FAIL: leak-guard matched in the commit message`, ...r.hits.map(hitLine), ...waivedLines(r.waived)] })
290
+ else if (r.status === 'clean' && r.waived.length > 0)
291
+ out.push({ id: 'leak-guard', ok: true, lines: [`ok: leak-guard clean in the commit message (${r.waived.length} waived)`, ...waivedLines(r.waived)] })
248
292
  }
249
293
  for (const c of h.commitMsg ?? []) {
250
294
  const r = await spawnWithTimeout([...c.cmd, messageFile], { cwd: root, timeoutMs: 60_000 })
@@ -265,8 +309,12 @@ export function listGates(config: RepoConfig, root = process.cwd()): string[] {
265
309
  // a gate listed as active but unable to fire is worse than no listing:
266
310
  // this surface exists so nobody has to guess
267
311
  const leakState = off.has('leak-guard') ? ' (DISABLED by config)' : existsSync(join(root, kw)) || globalKeywordFile() != null ? '' : ` (INACTIVE, ${kw} absent and no ~/.rv-private list)`
312
+ // a waiver nobody can enumerate is a hole nobody can audit: the count of
313
+ // declared exemptions belongs on the same surface as the gate itself
314
+ const waivers = (h.leakAllow ?? []).length + (globalAllowFile() != null ? 1 : 0)
315
+ const leakAllowState = waivers === 0 ? '' : `, ${(h.leakAllow ?? []).length} repo waiver(s)${globalAllowFile() == null ? '' : ' + ~/.rv-private/leak-allow.json'}`
268
316
  return [
269
- `pre-commit leak-guard${leakState} — staged added lines + paths vs ${kw} + ~/.rv-private/banned-keywords.txt`,
317
+ `pre-commit leak-guard${leakState} — staged added lines + paths vs ${kw} + ~/.rv-private/banned-keywords.txt${leakAllowState}`,
270
318
  ...((h.blockedPaths ?? []).length > 0 ? [`pre-commit blocked-paths — nothing under ${h.blockedPaths!.join(', ')} enters git`] : []),
271
319
  `pre-commit root-identity${off.has('root-identity') ? ' (DISABLED by config)' : ''} — the repo's own name in package.json + repo.config.ts, and scratch files`,
272
320
  ...(h.preCommit ?? []).map((c) => `pre-commit ${c.id} — ${c.cmd.join(' ')}`),
@@ -22,6 +22,7 @@ import { validateSmokeConfig, type SmokeConfig } from './smoke/run-smoke.ts'
22
22
  import { contractsFulfillness } from './rules/SK002-contracts-fulfillness.ts'
23
23
  import { BUILTIN_RULE_IDS, validateRules } from './rules/rule-registry.ts'
24
24
  import type { Rule } from './rules/rule.ts'
25
+ import { leakAllowProblems } from './hooks/leak-guard.ts'
25
26
  import type { HooksConfig } from './hooks/run-hooks.ts'
26
27
  import type { ReleaseConfig } from './release/run-release.ts'
27
28
  import { UNIT_DECL_KEYS, type ReleaseUnitDecl } from './release/units.ts'
@@ -202,6 +203,12 @@ export function defineRepo(c: RepoConfig): RepoConfig {
202
203
  if (c.rules != null) validateRules(c.rules, { label: 'defineRepo: rules', taken: BUILTIN_RULE_IDS })
203
204
  // a smoke typo would land as a run that visits nothing and exits 0
204
205
  if (c.smoke != null) validateSmokeConfig(c.smoke)
206
+ // a leak-guard waiver is the one config row that can make a gate quieter,
207
+ // so a malformed one dies here rather than being dropped at commit time
208
+ if (c.hooks?.leakAllow != null) {
209
+ const problems = leakAllowProblems(c.hooks.leakAllow, 'defineRepo: hooks.leakAllow', true)
210
+ if (problems.length > 0) throw new Error(problems[0])
211
+ }
205
212
  if (c.credsReadCommand != null && !/\{name\}/.test(c.credsReadCommand))
206
213
  throw new Error(`defineRepo: credsReadCommand must carry {name} (and {field} for get | account), e.g. 'my-secrets {field} {name}'`)
207
214
  if (c.smoke?.login != null && (c.credsReadCommand ?? '') === '')