@cosyte/script-utils 0.0.1 → 0.0.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/phi-scan.js ADDED
@@ -0,0 +1,1543 @@
1
+ // @cosyte/script-utils/phi-scan
2
+ //
3
+ // THE SHARED MACHINERY OF THE `@cosyte/*` PHI COMMIT-GATE, PARAMETERISED.
4
+ //
5
+ // WHY THIS FILE EXISTS, IN ONE SENTENCE: `scripts/parser-template/` is a SCAFFOLD, not a
6
+ // dependency, so `scripts/phi-scan.ts` was COPIED into every parser repo, and a newly-found escape
7
+ // therefore cost one pull request and one adversarial review PER REPO. Three escape classes have
8
+ // been closed that way already (the index union, the completeness rule, and the read filters), and
9
+ // each one was paid for thirteen times. The machinery lives here so the next one is paid for once.
10
+ //
11
+ // WHAT IS SHARED AND WHAT IS NOT. This module owns the ENGINE: argument parsing, the allow-list and
12
+ // override log, target enumeration on all three routes, the union of the working-tree walk with the
13
+ // bytes git carries, content deduplication under git's own `blob <len>\0` framing, the completeness
14
+ // rule, the refusals, and the cross-cutting SSN/email FLOOR. It does NOT own the per-standard,
15
+ // field-level detectors (names, DOB, MRN / member id, address, phone), which genuinely differ per
16
+ // healthcare standard and are supplied by the caller through `detect`.
17
+ //
18
+ // THE FIVE PER-REPO AXES ARE PARAMETERS, NOT FORKS. They differ between repos, which is an argument
19
+ // for parameters and not against sharing:
20
+ //
21
+ // 1. EXIT CODES `exitCodes`, REQUIRED. The siblings deliberately disagree; a default here
22
+ // would be the porting mistake this whole campaign exists to catch.
23
+ // 2. ROOTS+EXCLUSIONS `scanRoots` (REQUIRED), `excludedPaths` and `isWalkReadable` (both
24
+ // DEFAULTED, so a change to the shared default reaches every consumer
25
+ // through a version bump instead of thirteen edits).
26
+ // 3. `--staged` SCOPE `isStagedReadable`, REQUIRED. Widening it changes what a COMMIT is blocked
27
+ // on, which is a hook decision each repo takes for itself.
28
+ // 4. GITLINKS `regularBlobModes`, DEFAULTED to git's two regular-blob modes.
29
+ // 5. EOL NORMALIZATION No parameter. The walk/index deduplication is BY CONTENT, so where a `text`
30
+ // attribute or `core.autocrlf` makes the index carry LF and the working tree
31
+ // CRLF, BOTH forms are scanned rather than one being assumed to stand for the
32
+ // other. It is listed because a port must CHECK it, not skip it.
33
+ //
34
+ // ZERO DEPENDENCIES AND NO BUILD STEP, the same constraint the rest of this package carries. `git`
35
+ // is the only subprocess, always through `execFileSync` with array args, never shell-form.
36
+ //
37
+ // PHI DISCIPLINE INSIDE THIS FILE ITSELF: no literal identifier-shaped value appears anywhere in
38
+ // this source, not even as an example in a comment. A repo that widens its scan roots to the whole
39
+ // repository reads its own `node_modules` copy of this file only if it is not gitignored, and a
40
+ // diagnostic ABOUT a PHI leak is itself a PHI surface. Describe the shape; never write one down.
41
+
42
+ import { readFileSync, statSync, lstatSync, existsSync, readdirSync } from "node:fs";
43
+ import { execFileSync } from "node:child_process";
44
+ import { createHash } from "node:crypto";
45
+ import { join, resolve, relative, sep, isAbsolute } from "node:path";
46
+
47
+ /**
48
+ * Every state in which the scan cannot account for something. Raised throughout, caught in `run`,
49
+ * and turned into the caller's REFUSE code there.
50
+ *
51
+ * IT IS NOT EXPORTED, DELIBERATELY. A consumer that could catch it could also swallow it, and every
52
+ * one of these states must reach a non-zero exit.
53
+ */
54
+ class InvocationError extends Error {
55
+ /** @param {string} message */
56
+ constructor(message) {
57
+ super(message);
58
+ this.name = "InvocationError";
59
+ }
60
+ }
61
+
62
+ /** git's file modes for a regular blob. Every other mode names something with no bytes to read. */
63
+ const DEFAULT_REGULAR_BLOB_MODES = new Set(["100644", "100755"]);
64
+
65
+ /**
66
+ * AXIS 2's shared default read filter for the two SWEEPING routes: Markdown is documentation rather
67
+ * than fixture data and may legitimately describe a violator value.
68
+ *
69
+ * IT IS A DEFAULT RATHER THAN A CONSTANT SO THAT MOVING IT IS ONE CHANGE. Every `@cosyte/*` scanner
70
+ * carries this exemption today, and a tracked `.md` is therefore read by NEITHER sweeping route
71
+ * (the union inherits this filter on purpose, so `all` mode's verdict cannot depend on which copy of
72
+ * a file it reached). Whether that boundary should move is a real question with a real cost on both
73
+ * sides: `README.md` and `CHANGELOG.md` ship inside the npm tarball, and a `.md` named explicitly on
74
+ * argv IS scanned today, so the escape is route-dependent rather than file-dependent. Moving it is a
75
+ * deliberate change to THIS line, with its own measurement, and it then reaches every consumer
76
+ * through a version bump.
77
+ *
78
+ * @param {string} relPath A repo-relative, forward-slashed path.
79
+ * @returns {boolean} `true` when the sweeping routes may read this path's bytes.
80
+ */
81
+ export function exemptsMarkdown(relPath) {
82
+ return !relPath.toLowerCase().endsWith(".md");
83
+ }
84
+
85
+ /**
86
+ * Normalise and CHECK the caller's configuration.
87
+ *
88
+ * IT THROWS A `TypeError` RATHER THAN RETURNING A CODE, because at the point a required axis is
89
+ * missing there is no trustworthy code to return: `exitCodes` is itself the thing that was not
90
+ * supplied. That escapes to node's own exit 1 with a stack, which is loud and lands on the author's
91
+ * very first run of their own scanner.
92
+ *
93
+ * WHAT IT CHECKS AND WHAT IT DOES NOT, SCOPED TO WHAT IS ENFORCED HERE RATHER THAN TO AN INTENT.
94
+ * An earlier draft of this docblock said a misconfigured scanner "must not be able to report clean",
95
+ * and a reviewer falsified that sentence twice in four lines. Both reproductions were the same
96
+ * shape: a configuration this function ACCEPTED, whose roots then matched nothing, so every
97
+ * index-keyed rule went silently empty and the run printed `OK: no hits`. So the two are named:
98
+ *
99
+ * 1. A ROOT IS NORMALISED THE SAME WAY A PATH IS. `"./src"` is a spelling the type describes as
100
+ * valid, and it walked correctly while `isUnderScanRoot` compared it against the NORMALIZED
101
+ * index path `src/...` and never matched. Measured: `["src"]` refused a tracked mode-120000
102
+ * entry at exit 2 while `["./src"]` reported clean at exit 0 over the same repository.
103
+ * 2. A ROOT THAT RESOLVES OUTSIDE THE REPOSITORY IS REFUSED, because no index path can ever be
104
+ * under it, which is the same silently-empty state arrived at by another spelling.
105
+ *
106
+ * IT IS STILL NOT AN EXHAUSTIVE GUARANTEE, and the residual is named rather than implied: the
107
+ * CONTENT of a caller's predicates is not checkable here. `isStagedReadable` admitting a path
108
+ * outside every scan root is the one containment that matters, and it is enforced where it can be
109
+ * observed, in `buildTargetsForStaged`, not here.
110
+ *
111
+ * @param {import("./phi-scan.js").PhiScanConfig} config
112
+ * @returns {Required<Omit<import("./phi-scan.js").PhiScanConfig, "detect">> & { detect?: import("./phi-scan.js").DetectFn }}
113
+ */
114
+ function normalizeConfig(config) {
115
+ if (typeof config !== "object" || config === null) {
116
+ throw new TypeError("runPhiScan(config) expects a configuration object.");
117
+ }
118
+
119
+ const repoRoot = config.repoRoot ?? process.cwd();
120
+ if (typeof repoRoot !== "string" || repoRoot === "") {
121
+ throw new TypeError("runPhiScan: `repoRoot` must be a non-empty string when given.");
122
+ }
123
+
124
+ const codes = config.exitCodes;
125
+ if (typeof codes !== "object" || codes === null) {
126
+ throw new TypeError(
127
+ "runPhiScan: `exitCodes` is REQUIRED ({ clean, hits, refuse }). There is deliberately no " +
128
+ "default: the sibling @cosyte/* scanners do not agree on these numbers, and a caller that " +
129
+ "branches on the code (CI does) must read this repo's own contract, never an inherited one.",
130
+ );
131
+ }
132
+ for (const key of /** @type {const} */ (["clean", "hits", "refuse"])) {
133
+ const value = codes[key];
134
+ if (!Number.isInteger(value) || value < 0 || value > 125) {
135
+ throw new TypeError(
136
+ `runPhiScan: exitCodes.${key} must be an integer in 0..125, got ${String(value)}.`,
137
+ );
138
+ }
139
+ }
140
+ if (codes.clean === codes.hits || codes.clean === codes.refuse || codes.hits === codes.refuse) {
141
+ throw new TypeError(
142
+ "runPhiScan: exitCodes.clean, .hits and .refuse must be three DIFFERENT numbers. A caller " +
143
+ "has to be able to tell `this corpus contains something that looks like PHI` from `this " +
144
+ "scan is not trustworthy`: those need different human responses, and collapsing them makes " +
145
+ "the second read as the first.",
146
+ );
147
+ }
148
+
149
+ const rawRoots = config.scanRoots;
150
+ if (!Array.isArray(rawRoots) || rawRoots.length === 0) {
151
+ throw new TypeError(
152
+ 'runPhiScan: `scanRoots` is REQUIRED and must name at least one root. Use `["."]` for the ' +
153
+ "whole repository.",
154
+ );
155
+ }
156
+ /** @type {Set<string>} */
157
+ const scanRootSet = new Set();
158
+ for (const root of rawRoots) {
159
+ if (typeof root !== "string" || root === "") {
160
+ throw new TypeError("runPhiScan: every entry in `scanRoots` must be a non-empty string.");
161
+ }
162
+ // THE SAME NORMALIZATION EVERY OTHER PATH GETS, and it is the fix for reproduction 1 above:
163
+ // `isUnderScanRoot` compares against repo-relative, forward-slashed paths, so a root must be one
164
+ // too or it matches nothing while still walking correctly. This maps `./src`, `src/` and
165
+ // `<repoRoot>/src` onto `src`, and the repository root itself onto `.`.
166
+ const rel = relative(repoRoot, isAbsolute(root) ? root : resolve(repoRoot, root))
167
+ .split(sep)
168
+ .join("/");
169
+ if (rel === ".." || rel.startsWith("../")) {
170
+ throw new TypeError(
171
+ `runPhiScan: scanRoots entry ${JSON.stringify(root)} resolves outside the repository ` +
172
+ `(${rel}). No path git can name is under it, so every index-keyed rule would go silently ` +
173
+ `empty and the sweep would report clean over a corpus it never had in scope.`,
174
+ );
175
+ }
176
+ scanRootSet.add(rel === "" ? "." : rel);
177
+ }
178
+ // Deduped, so two spellings of one root cannot walk the same tree twice.
179
+ const scanRoots = [...scanRootSet];
180
+
181
+ if (typeof config.isStagedReadable !== "function") {
182
+ throw new TypeError(
183
+ "runPhiScan: `isStagedReadable` is REQUIRED. It decides which staged blobs a COMMIT is " +
184
+ "blocked on, which is a per-repo hook decision and must not be inherited silently.",
185
+ );
186
+ }
187
+
188
+ // THE OPTIONAL AXES ARE SHAPE-CHECKED TOO, AND THAT IS NOT TIDINESS. A reviewer measured
189
+ // `excludedPaths: ["a"]`, which is a plausible reading of "repo-relative paths": it survived
190
+ // normalization, reached `.has(...)` inside enumeration, threw an uncaught `TypeError` from there,
191
+ // and the run took node's exit 1, which the exit contract reserves for HITS FOUND. Checking the
192
+ // shape here turns that into a throw that says what to fix, before any scan begins.
193
+ if (config.excludedPaths !== undefined && typeof config.excludedPaths.has !== "function") {
194
+ throw new TypeError(
195
+ "runPhiScan: `excludedPaths` must be a Set (or anything with `.has(path)`), not an array.",
196
+ );
197
+ }
198
+ if (config.regularBlobModes !== undefined && typeof config.regularBlobModes.has !== "function") {
199
+ throw new TypeError(
200
+ "runPhiScan: `regularBlobModes` must be a Set (or anything with `.has(mode)`).",
201
+ );
202
+ }
203
+ for (const key of /** @type {const} */ (["isWalkReadable", "detect"])) {
204
+ if (config[key] !== undefined && typeof config[key] !== "function") {
205
+ throw new TypeError(`runPhiScan: \`${key}\` must be a function when given.`);
206
+ }
207
+ }
208
+ for (const key of /** @type {const} */ (["allowListPath", "overrideLogPath"])) {
209
+ const value = config[key];
210
+ if (value !== undefined && (typeof value !== "string" || value === "")) {
211
+ throw new TypeError(`runPhiScan: \`${key}\` must be a non-empty string when given.`);
212
+ }
213
+ }
214
+ if (config.argv !== undefined && !Array.isArray(config.argv)) {
215
+ throw new TypeError("runPhiScan: `argv` must be an array of strings when given.");
216
+ }
217
+
218
+ return {
219
+ repoRoot,
220
+ argv: config.argv ?? process.argv.slice(2),
221
+ exitCodes: { clean: codes.clean, hits: codes.hits, refuse: codes.refuse },
222
+ scanRoots,
223
+ excludedPaths: config.excludedPaths ?? new Set(),
224
+ isWalkReadable: config.isWalkReadable ?? exemptsMarkdown,
225
+ isStagedReadable: config.isStagedReadable,
226
+ regularBlobModes: config.regularBlobModes ?? DEFAULT_REGULAR_BLOB_MODES,
227
+ allowListPath: config.allowListPath ?? join(repoRoot, "scripts", "phi-allow-list.txt"),
228
+ overrideLogPath: config.overrideLogPath ?? join(repoRoot, "phi-scan-overrides.md"),
229
+ detect: config.detect,
230
+ };
231
+ }
232
+
233
+ /**
234
+ * Run the PHI scan and RETURN an exit code. Nothing here calls `process.exit`, so a test can drive
235
+ * the engine in-process; the scanner script is the one that exits.
236
+ *
237
+ * ===========================================================================================
238
+ * MODES
239
+ * `--staged` scan only the blobs `git diff --cached` names.
240
+ * `<path> [<path>...]` scan specific paths.
241
+ * (no args) `all` mode: sweep the scan roots, as a union with the bytes git carries.
242
+ *
243
+ * `--allow-fixture <path>` IS A MODIFIER, NOT A MODE. A bypass is subtractive, so it must not also
244
+ * decide what gets scanned: a lone `--allow-fixture X` used to select `paths` mode over exactly
245
+ * `X`, then withdraw `X`, then report a clean whole run having opened nothing. The mode is chosen by
246
+ * positional paths alone, and in `paths` mode the flag is UNCONDITIONALLY UNIONED into the target
247
+ * list, deduped by repo-relative path, so it means the same thing in every argv.
248
+ * ===========================================================================================
249
+ * THE COMPLETENESS RULE: a target this run ENUMERATED and NEVER READ refuses, IN EVERY MODE, NAMING
250
+ * THE PATHS. The comparison is a SET DIFFERENCE, never a size: a count counts the targets that DID
251
+ * get read, so a plausible-looking total hides exactly the paths that did not.
252
+ *
253
+ * ENUMERATION IS THE RUN'S OWN DECLARATION OF WHAT IT WILL READ, so the read filters upstream of it
254
+ * are not weakened by the rule: a path the read filter dropped, a gitignored entry, an excluded
255
+ * path and a staged path outside `isStagedReadable` never become targets at all. What the rule
256
+ * catches is a path that BECAME a target and then did not get opened.
257
+ *
258
+ * A BYPASS NAMING A PATH THIS RUN DOES NOT ENUMERATE ALSO REFUSES: such a flag subtracts nothing, so
259
+ * honouring it silently would let a developer believe a file was acknowledged when the run never had
260
+ * it in scope.
261
+ *
262
+ * WHAT IT COSTS, STATED RATHER THAN LEFT TO BE DISCOVERED: `--allow-fixture` CANNOT REACH THE CLEAN
263
+ * CODE IN ANY MODE. The flag, the override log and the rejection gate are all kept, so an attempt is
264
+ * RECORDED AND REFUSED rather than silently honoured.
265
+ * ===========================================================================================
266
+ * `all` MODE READS THE BYTES GIT CARRIES AS A UNION WITH THE WALK.
267
+ *
268
+ * The walk answers "what is on disk under the scan roots", which is not the question "what does this
269
+ * repository carry". Three states were measured in which the walk alone reported clean at exit 0
270
+ * over a TRACKED file carrying a live, detectable hit: the path OCCUPIED BY A DIRECTORY (a path-SET
271
+ * reconciliation cannot see this one, because the path IS present: only reading the OBJECT closes
272
+ * it), the working tree SHORT of a tracked file, and the two copies simply DIFFERING.
273
+ *
274
+ * So `git ls-files -s -z` is read for the whole index, and every in-scope tracked path whose bytes
275
+ * the walk did not already read VERBATIM is scanned through `git cat-file blob <sha>`. It is a union
276
+ * and never a replacement: the walk still runs first and still reaches UNTRACKED files, which git
277
+ * cannot name at all.
278
+ *
279
+ * DEDUPLICATION IS BY CONTENT, NOT BY PATH, AND THAT IS THE EOL AXIS. A walk target is skipped by
280
+ * the union only when the bytes it read hash to the index entry's own object id, so on a clean
281
+ * checkout the union adds ZERO reads and NEVER invokes `git cat-file`. The exact fixed cost is ONE
282
+ * `git rev-parse --show-object-format` per `all`-mode run, because the deduplication needs the
283
+ * algorithm before it can compare anything. Where the two copies of a path DIFFER, BOTH are scanned.
284
+ *
285
+ * A HIT FROM THE UNION IS LABELLED `(as git carries it)`, ON THE REPORTED LOCUS ONLY. A hit naming
286
+ * the bare path would send a developer to open a file that is clean, or not there at all. The
287
+ * target's `path` stays undecorated because every filter, exclusion and completeness tier keys on it.
288
+ * The `detect` callback is handed the LOCUS as `path`, so a caller cannot get this wrong.
289
+ *
290
+ * `all` MODE REFUSES WHEN GIT CANNOT NAME THE INDEX, OR NAMES IT EMPTY, and the two arrive through
291
+ * DIFFERENT branches. Measured on git 2.39.5: a directory that is no repository at all FATALS (exit
292
+ * 128), so the `catch` is what turns it into a refusal, and WITHOUT that catch the throw escapes and
293
+ * the run takes node's own exit 1, which this contract reserves for HITS FOUND. A repository whose
294
+ * index is empty, and a directory inside a repository with nothing tracked under it, both print
295
+ * nothing and exit 0, which is what the size check is for. A scaffolded repo must therefore
296
+ * `git init` and commit before an `all`-mode run means anything.
297
+ * ===========================================================================================
298
+ *
299
+ * @param {import("./phi-scan.js").PhiScanConfig} config
300
+ * @returns {number} The exit code, drawn from `config.exitCodes`.
301
+ * @example
302
+ * process.exit(
303
+ * runPhiScan({
304
+ * exitCodes: { clean: 0, hits: 1, refuse: 2 },
305
+ * scanRoots: ["."],
306
+ * isStagedReadable: (p) => p.startsWith("test/fixtures/"),
307
+ * }),
308
+ * );
309
+ */
310
+ export function runPhiScan(config) {
311
+ return new PhiScan(normalizeConfig(config)).run();
312
+ }
313
+
314
+ class PhiScan {
315
+ /** @param {ReturnType<typeof normalizeConfig>} cfg */
316
+ constructor(cfg) {
317
+ this.cfg = cfg;
318
+ /** Does any scan root name the repository root itself? Then everything is in scope. */
319
+ this.wholeRepo = cfg.scanRoots.includes(".");
320
+ }
321
+
322
+ // -------------------------------------------------------------------------
323
+ // Paths
324
+ // -------------------------------------------------------------------------
325
+
326
+ /**
327
+ * A repo-relative, forward-slashed path: the key every filter, exclusion, dedupe and completeness
328
+ * tier uses, and the only spelling that appears in a diagnostic.
329
+ *
330
+ * @param {string} p
331
+ * @returns {string}
332
+ */
333
+ normalizePath(p) {
334
+ const abs = isAbsolute(p) ? p : resolve(this.cfg.repoRoot, p);
335
+ return relative(this.cfg.repoRoot, abs).split(sep).join("/");
336
+ }
337
+
338
+ /**
339
+ * AXIS 2, the ROOT half of scope: is this entry the scan's BUSINESS at all?
340
+ *
341
+ * THERE ARE TWO SCOPE PREDICATES AND COLLAPSING THEM REOPENS A MEASURED HOLE. This one decides
342
+ * whether an entry is in scope; the READ filters (`isWalkReadable`, `isStagedReadable`) decide
343
+ * whether a REGULAR FILE's bytes get read. Every non-regular and non-blob check keys on THIS one.
344
+ * Two sibling ports independently shipped a single shared predicate and both had the routes
345
+ * disagree about the same entry: a `.md`-named link fell out through the read filter on one route
346
+ * while the other refused it. A link's NAME is no evidence at all about what is on the other side
347
+ * of it, which is exactly what a read filter is entitled to assume about a file and is not
348
+ * entitled to assume about a link.
349
+ *
350
+ * A bare root name is in scope because git records no index entry for a directory. A scan root
351
+ * appearing as an index entry therefore means it is not a directory, and AT LEAST these readings
352
+ * exist: the root has been replaced by a blob or a link; it is a FILE ROOT, which this engine
353
+ * admits, since `scanRoots` derives a root's kind from the filesystem; or it is a gitlink, a
354
+ * nested repository, which the index refusal names in those words. No enumeration is attempted.
355
+ * An earlier draft predated file roots and said the first reading was the only one.
356
+ *
357
+ * @param {string} relPath
358
+ * @returns {boolean}
359
+ */
360
+ isUnderScanRoot(relPath) {
361
+ if (this.wholeRepo) return true;
362
+ return this.cfg.scanRoots.some((root) => relPath === root || relPath.startsWith(`${root}/`));
363
+ }
364
+
365
+ // -------------------------------------------------------------------------
366
+ // Argument parsing
367
+ // -------------------------------------------------------------------------
368
+
369
+ /**
370
+ * @param {string[]} argv
371
+ * @returns {{ mode: "all" | "staged" | "paths", paths: string[], allowFixtures: string[] }}
372
+ */
373
+ parseArgs(argv) {
374
+ let staged = false;
375
+ /** @type {string[]} */
376
+ const paths = [];
377
+ /** @type {string[]} */
378
+ const allowFixtures = [];
379
+ let i = 0;
380
+ while (i < argv.length) {
381
+ const a = argv[i];
382
+ if (a === "--") {
383
+ for (let j = i + 1; j < argv.length; j += 1) {
384
+ const v = argv[j];
385
+ if (v !== undefined) paths.push(v);
386
+ }
387
+ break;
388
+ } else if (a === "--staged") {
389
+ staged = true;
390
+ i += 1;
391
+ } else if (a === "--allow-fixture") {
392
+ const next = argv[i + 1];
393
+ if (next === undefined) {
394
+ throw new InvocationError("--allow-fixture requires a path argument");
395
+ }
396
+ allowFixtures.push(next);
397
+ i += 2;
398
+ } else if (a !== undefined && a.startsWith("--")) {
399
+ throw new InvocationError(`Unknown flag: ${a}`);
400
+ } else if (a !== undefined) {
401
+ paths.push(a);
402
+ i += 1;
403
+ } else {
404
+ i += 1;
405
+ }
406
+ }
407
+
408
+ if (staged && paths.length > 0) {
409
+ throw new InvocationError("--staged cannot be combined with positional paths");
410
+ }
411
+
412
+ // THE MODE IS CHOSEN BY POSITIONAL PATHS ALONE. Letting a bypass select the mode made
413
+ // `--allow-fixture X` scan exactly `X`, then withdraw it, then report a clean whole run over a
414
+ // corpus it never touched.
415
+ /** @type {"all" | "staged" | "paths"} */
416
+ let mode;
417
+ if (staged) mode = "staged";
418
+ else if (paths.length > 0) mode = "paths";
419
+ else mode = "all";
420
+
421
+ // UNCONDITIONAL, DEDUPED SEEDING, so the flag has ONE meaning in every argv. The old form was
422
+ // `paths.length > 0 ? paths : [...allowFixtures]`, which seeded the target list ONLY when no
423
+ // positional path was given: with one present the bypass was a silent no-op and the named file
424
+ // was never ADMITTED to the run rather than withdrawn from it. Unioning admits it in every case,
425
+ // so the withdrawal below is always a withdrawal of something enumerated and is therefore always
426
+ // caught by the completeness rule. Dedupe is by repo-relative path, so a file named both as a
427
+ // positional and as a bypass is one target, not two.
428
+ const seed = [...paths, ...allowFixtures];
429
+ const scanPaths = mode === "paths" ? this.dedupeByRepoPath(seed) : paths;
430
+
431
+ return { mode, paths: scanPaths, allowFixtures };
432
+ }
433
+
434
+ /**
435
+ * Dedupe argument paths by the repo-relative path each resolves to, keeping the caller's original
436
+ * spelling for the first occurrence: that spelling is what gets resolved and echoed back.
437
+ *
438
+ * @param {string[]} paths
439
+ * @returns {string[]}
440
+ */
441
+ dedupeByRepoPath(paths) {
442
+ const seen = new Set();
443
+ /** @type {string[]} */
444
+ const out = [];
445
+ for (const p of paths) {
446
+ const key = this.normalizePath(p);
447
+ if (seen.has(key)) continue;
448
+ seen.add(key);
449
+ out.push(p);
450
+ }
451
+ return out;
452
+ }
453
+
454
+ // -------------------------------------------------------------------------
455
+ // Allow-list + override log
456
+ // -------------------------------------------------------------------------
457
+
458
+ /**
459
+ * The positive declaration that specific identifiers are synthetic.
460
+ *
461
+ * ONE ESCAPE IS DISCLOSED RATHER THAN CLOSED: an allow-list that EXISTS but cannot be READ (a
462
+ * directory at that path, or mode 000) makes `readFileSync` throw a plain `Error`, which is not an
463
+ * `InvocationError` and is therefore rethrown, and the run takes node's own exit 1 with a stack. A
464
+ * caller reads that as "hits found". `loadOverrideLog` has the identical shape. Both are left
465
+ * alone deliberately rather than papered over by widening a catch or enumerating `EACCES`/
466
+ * `EISDIR`: a deny-list of spellings buys exactly one more evasion per round, and this repo
467
+ * retired that shape once already on the `attw` gate. The exit contract below therefore says
468
+ * MISSING rather than "missing or unreadable", and says 1 is reserved but NOT exclusive.
469
+ *
470
+ * @returns {import("./phi-scan.js").AllowList}
471
+ */
472
+ loadAllowList() {
473
+ if (!existsSync(this.cfg.allowListPath)) {
474
+ throw new InvocationError(`allow-list not found at ${this.cfg.allowListPath}`);
475
+ }
476
+ const raw = readFileSync(this.cfg.allowListPath, "utf8");
477
+ /** @type {Set<string>} */ const names = new Set();
478
+ /** @type {Set<string>} */ const dobs = new Set();
479
+ /** @type {Set<string>} */ const ids = new Set();
480
+ /** @type {Set<string>} */ const emailDomains = new Set();
481
+ for (const lineRaw of raw.split(/\r?\n/)) {
482
+ const line = lineRaw.trim();
483
+ if (line.length === 0 || line.startsWith("#")) continue;
484
+ const sp = line.indexOf(" ");
485
+ if (sp < 0) continue;
486
+ const tag = line.slice(0, sp);
487
+ const value = line.slice(sp + 1).trim();
488
+ if (value.length === 0) continue;
489
+ switch (tag) {
490
+ case "NAME":
491
+ names.add(value.toUpperCase());
492
+ break;
493
+ case "DOB":
494
+ dobs.add(value);
495
+ break;
496
+ case "ID":
497
+ ids.add(value.toUpperCase());
498
+ break;
499
+ case "EMAILDOMAIN":
500
+ emailDomains.add(value.toLowerCase());
501
+ break;
502
+ default:
503
+ break;
504
+ }
505
+ }
506
+ return { names, dobs, ids, emailDomains };
507
+ }
508
+
509
+ /** @returns {Set<string>} Every path the override log records, repo-relative. */
510
+ loadOverrideLog() {
511
+ if (!existsSync(this.cfg.overrideLogPath)) return new Set();
512
+ const raw = readFileSync(this.cfg.overrideLogPath, "utf8");
513
+ /** @type {Set<string>} */
514
+ const out = new Set();
515
+ for (const lineRaw of raw.split(/\r?\n/)) {
516
+ const m = /^###\s+(.+?)\s*$/.exec(lineRaw);
517
+ if (m && m[1] !== undefined) out.add(this.normalizePath(m[1]));
518
+ }
519
+ return out;
520
+ }
521
+
522
+ /** @param {string[]} allowFixtures */
523
+ validateAllowFixtures(allowFixtures) {
524
+ if (allowFixtures.length === 0) return;
525
+ const overrides = this.loadOverrideLog();
526
+ const missing = allowFixtures
527
+ .map((p) => this.normalizePath(p))
528
+ .filter((p) => !overrides.has(p));
529
+ if (missing.length > 0) {
530
+ const lines = missing.map((p) => ` - ${p}`).join("\n");
531
+ throw new InvocationError(
532
+ `--allow-fixture rejected: no matching entry in ${this.relOverrideLog()} for:\n${lines}\n` +
533
+ `Add a "### <path>" subsection to ${this.relOverrideLog()} and commit it.`,
534
+ );
535
+ }
536
+ }
537
+
538
+ /** @returns {string} The override log's repo-relative path, for a diagnostic. */
539
+ relOverrideLog() {
540
+ return this.normalizePath(this.cfg.overrideLogPath);
541
+ }
542
+
543
+ /** @returns {string} The allow-list's repo-relative path, for a diagnostic. */
544
+ relAllowList() {
545
+ return this.normalizePath(this.cfg.allowListPath);
546
+ }
547
+
548
+ // -------------------------------------------------------------------------
549
+ // git
550
+ // -------------------------------------------------------------------------
551
+
552
+ /**
553
+ * Which of these paths git considers ignored.
554
+ *
555
+ * ONE BOUNDARY, NOT TWO: an ignored entry is already out of scope for the file route, so applying
556
+ * the same rule to a link keeps a single boundary rather than inventing a second, stricter one for
557
+ * links alone. `git check-ignore` is INDEX-AWARE, so `git add -f` on an ignored link does not buy
558
+ * a bypass: once tracked it is no longer reported ignored.
559
+ *
560
+ * @param {string[]} paths Repo-relative or absolute paths.
561
+ * @returns {Set<string>} The repo-relative paths git reports ignored.
562
+ */
563
+ gitIgnored(paths) {
564
+ /** @type {Set<string>} */
565
+ const ignored = new Set();
566
+ if (paths.length === 0) return ignored;
567
+ try {
568
+ // SECURITY: array-form execFileSync, no shell. Default (Buffer) encoding, because
569
+ // `encoding: "buffer"` together with `input` is rejected by Node.
570
+ const out = execFileSync("git", ["check-ignore", "--stdin", "-z"], {
571
+ cwd: this.cfg.repoRoot,
572
+ input: paths.map((p) => this.normalizePath(p)).join("\0"),
573
+ stdio: ["pipe", "pipe", "ignore"],
574
+ maxBuffer: 64 * 1024 * 1024,
575
+ });
576
+ for (const p of out.toString("utf8").split("\0")) {
577
+ if (p.length > 0) ignored.add(p);
578
+ }
579
+ } catch {
580
+ // `git check-ignore` exits 1 when nothing matches, and fatals outside a repository. Treat both
581
+ // as "none ignored": in `all` mode the missing index is refused separately and loudly, and
582
+ // pruning nothing can only ever make the sweep read MORE.
583
+ }
584
+ return ignored;
585
+ }
586
+
587
+ /**
588
+ * The repository's object format as a Node hash name, or `null` when git says something we do not
589
+ * recognise. `null` disables the union's content deduplication, which scans MORE, never less.
590
+ *
591
+ * WHEN GIT WILL NOT SAY AT ALL THE ANSWER IS `sha1`, NOT `null`, and the two cases are stated
592
+ * apart. A git too old to know `--show-object-format` predates sha256 repositories entirely, so
593
+ * the fallback is a derivation rather than a guess; an answer we do not recognise is a git NEWER
594
+ * than this file, and there the honest move is to stop deduplicating.
595
+ *
596
+ * @returns {string | null}
597
+ */
598
+ gitObjectHash() {
599
+ let answer;
600
+ try {
601
+ // SECURITY: array-form execFileSync, no shell.
602
+ answer = execFileSync("git", ["rev-parse", "--show-object-format"], {
603
+ cwd: this.cfg.repoRoot,
604
+ stdio: ["ignore", "pipe", "ignore"],
605
+ })
606
+ .toString("utf8")
607
+ .trim();
608
+ } catch {
609
+ return "sha1";
610
+ }
611
+ if (answer === "sha1") return "sha1";
612
+ if (answer === "sha256") return "sha256";
613
+ return null;
614
+ }
615
+
616
+ /**
617
+ * The object id git would record for these bytes, under its own `blob <len>\0` framing.
618
+ *
619
+ * Used only to answer "did the walk already read EXACTLY the bytes the index carries here", so a
620
+ * wrong answer can only ever cost a second scan of the same content. THIS IS THE EOL AXIS: where a
621
+ * `text` attribute or `core.autocrlf` makes the index carry LF and the working tree CRLF, the two
622
+ * ids differ and BOTH copies are scanned.
623
+ *
624
+ * @param {string} algorithm
625
+ * @param {Buffer} bytes
626
+ * @returns {string | null}
627
+ */
628
+ blobOid(algorithm, bytes) {
629
+ try {
630
+ return createHash(algorithm)
631
+ .update(`blob ${String(bytes.length)}\0`)
632
+ .update(bytes)
633
+ .digest("hex");
634
+ } catch {
635
+ return null;
636
+ }
637
+ }
638
+
639
+ /**
640
+ * Every stage-0 index entry keyed by repo-relative path, plus the paths that have a record but NO
641
+ * stage-0 record, or `null` when git could not answer.
642
+ *
643
+ * AN EMPTY ANSWER COUNTS AS NO ANSWER: an empty map would make every tracked path untracked, which
644
+ * is the one state in which the union silently stops existing.
645
+ *
646
+ * 🛑 THE STAGE DIGIT IS READ, AND KEYING ON IT IS NOT OPTIONAL. THE RULE IS THE ABSENCE OF STAGE 0.
647
+ * Do NOT re-derive it from a record count or from a mode, and do NOT port it from the `--staged`
648
+ * route: that route spots an unmerged path from `--raw`'s status `U` and a destination mode of
649
+ * `000000`, and NOTHING IN `ls-files -s` LOOKS LIKE THAT. An unmerged path is reported here only at
650
+ * stages 1, 2 and/or 3, with ORDINARY BLOB MODES, so the mode rule cannot see it. A sibling's draft
651
+ * took the FIRST record per path and never looked at the stage: it scanned STAGE 1, THE MERGE BASE,
652
+ * labelled it as the bytes git carries, and printed a clean line over a marker living only in
653
+ * stage 3.
654
+ *
655
+ * @returns {{ entries: Map<string, import("./phi-scan.js").IndexEntry>, unmerged: string[] } | null}
656
+ */
657
+ gitIndexEntries() {
658
+ let out;
659
+ try {
660
+ // SECURITY: array-form execFileSync, no shell. `-z` is NUL-separated and unquoted, so it
661
+ // matches the walk's forward-slash relative paths exactly. `maxBuffer` is raised because a
662
+ // TRUNCATED list is a SHORT list, and a short list is the unscanned corpus this whole rule is
663
+ // about; Node throws `ENOBUFS` rather than truncating, so the bound refuses either way.
664
+ out = execFileSync("git", ["ls-files", "-s", "-z"], {
665
+ cwd: this.cfg.repoRoot,
666
+ stdio: ["ignore", "pipe", "ignore"],
667
+ maxBuffer: 64 * 1024 * 1024,
668
+ });
669
+ } catch {
670
+ // LOAD-BEARING. A directory that is no repository FATALS at 128; without this catch the throw
671
+ // escapes and the run takes node's exit 1, which this contract reserves for HITS FOUND.
672
+ return null;
673
+ }
674
+ /** @type {Map<string, import("./phi-scan.js").IndexEntry>} */
675
+ const entries = new Map();
676
+ /** @type {Set<string>} */
677
+ const higherStages = new Set();
678
+ for (const rec of out.toString("utf8").split("\0")) {
679
+ if (rec.length === 0) continue;
680
+ // `<mode> <oid> <stage>\t<path>`; a path may contain anything but NUL.
681
+ const m = /^(\d{6}) ([0-9a-f]+) (\d)\t([\s\S]+)$/.exec(rec);
682
+ const mode = m?.[1];
683
+ const oid = m?.[2];
684
+ const stage = m?.[3];
685
+ const path = m?.[4];
686
+ if (mode === undefined || oid === undefined || stage === undefined || path === undefined) {
687
+ // An unparseable record means the list may be SHORT in a way we cannot see, which is the one
688
+ // thing this sweep must never scan past.
689
+ return null;
690
+ }
691
+ if (stage === "0") entries.set(path, { mode, oid });
692
+ else higherStages.add(path);
693
+ }
694
+ // A path is unmerged when it has a record and none of them is stage 0. The set difference is
695
+ // taken rather than assuming the two are disjoint.
696
+ const unmerged = [...higherStages].filter((p) => !entries.has(p));
697
+ if (entries.size === 0 && unmerged.length === 0) return null;
698
+ return { entries, unmerged };
699
+ }
700
+
701
+ // -------------------------------------------------------------------------
702
+ // Enumeration: the walk
703
+ // -------------------------------------------------------------------------
704
+
705
+ /**
706
+ * Enumerate the scan roots.
707
+ *
708
+ * `Dirent`'s predicates are lstat answers and are not exhaustive: an entry that is neither a
709
+ * directory nor a regular file is collected into `unscannable` rather than dropped, so the caller
710
+ * can refuse instead of reporting clean over it.
711
+ *
712
+ * IGNORED DIRECTORIES ARE PRUNED DURING DESCENT, ONE `git check-ignore` PER LEVEL. That is not an
713
+ * optimisation bolted onto a filter: `scanRoots: ["."]` is the only honest default for a freshly
714
+ * scaffolded repo, and without pruning such a sweep descends into `node_modules`.
715
+ *
716
+ * WHY IT IS EQUIVALENT TO FILTERING AFTERWARDS, NAMED AS THE RULE THAT ACTUALLY CARRIES IT. An
717
+ * earlier draft said "git cannot re-include a path under an excluded directory", which is a
718
+ * gitignore-PATTERN rule and does not settle this on its own, because the filter it replaces
719
+ * asked `check-ignore` too. The load-bearing property is that `git check-ignore` IS INDEX-AWARE AT
720
+ * DIRECTORY GRANULARITY. Measured on git 2.39.5, with `node_modules/` in `.gitignore`:
721
+ *
722
+ * - nothing tracked underneath -> `check-ignore node_modules` exits 0 and the directory is
723
+ * pruned, which is right, because no file under it could have survived the file-level filter
724
+ * either;
725
+ * - one file force-added underneath -> `check-ignore node_modules` exits 1, the directory is NOT
726
+ * pruned, and the walk descends and reads the tracked file exactly as before.
727
+ *
728
+ * The file-level filter still runs below, so pruning only ever removes work the filter would have
729
+ * removed anyway. `test/phi-scan-engine.test.ts` pins BOTH directions, because a test that only
730
+ * plants a violator in an ignored directory measures that pruning happens, not that it is safe.
731
+ *
732
+ * A DIRECTORY NAMED `.git` IS SKIPPED BY NAME, at any depth. It is git's own object store rather
733
+ * than the corpus, git does not report it ignored, and the union already reads what the repository
734
+ * carries. This is a literal name, never a predicate over content.
735
+ *
736
+ * A SCAN ROOT MAY NAME A REGULAR FILE, AND THE KIND IS DERIVED FROM THE FILESYSTEM RATHER THAN
737
+ * DECLARED. That is a deliberate API decision, and it was made against a measurement rather than a
738
+ * preference: one sibling declares its roots as `{ rel, shape: "directory" | "file" }` and lists a
739
+ * single file (`README.md`) among them, and an earlier draft of this walk fed every root straight
740
+ * to `readdirSync`, so such a root threw `ENOTDIR` from here, uncaught, and the run took node's own
741
+ * exit 1, which this contract reserves for HITS FOUND. Deriving the kind keeps the parameter a
742
+ * plain `string[]` and still expresses that shape.
743
+ *
744
+ * THAT PARTICULAR ROOT READS NOTHING UNDER THE DEFAULT `isWalkReadable`, and the example is
745
+ * caveated rather than left to be copied. A file root goes through the same read filter every
746
+ * other file does, so with the shared Markdown exemption in place `scanRoots: ["README.md"]` was
747
+ * measured returning `OK: no hits` at exit 0 over a live dashed identifier. A repo porting a `.md`
748
+ * root has to override that filter as well.
749
+ *
750
+ * WHAT DERIVING GIVES UP, STATED RATHER THAN LEFT TO BE FOUND: a declaration can notice that a root
751
+ * is not the KIND it was meant to be, and derivation cannot. A root that was a file and became a
752
+ * directory is descended, and one that was a directory and became a file is read as a file, in both
753
+ * cases silently. What is NOT given up is the non-regular case: a root that is neither is collected
754
+ * as unscannable and refused, exactly as an entry inside one would be.
755
+ *
756
+ * A MISSING ROOT IS SKIPPED, AND THAT IS UNCHANGED FROM THE COPIED SCANNERS rather than chosen
757
+ * here. Two root states are refused or normalised away in `normalizeConfig` (a spelling that
758
+ * matches no index path is normalised away and the run proceeds; a root outside the repository
759
+ * throws), and this one is neither.
760
+ *
761
+ * NO CLAIM IS MADE THAT IT IS THE LAST SUCH STATE, and a draft of this paragraph said it was. At
762
+ * least two others are known: a file root the read filter drops, described above and created by
763
+ * the same change that added file roots; and an UNREADABLE root, which `lstatOrNull`'s bare catch
764
+ * reports the same way it reports a missing one, so the word MISSING there covers a state that is
765
+ * not missing (an `EACCES` on a parent directory, or an `ELOOP`). Both contribute nothing without
766
+ * saying so. The set is not enumerated because nothing here has measured it to be complete.
767
+ *
768
+ * The result is SORTED by repo-relative path, so a report and a refusal read the same way twice.
769
+ *
770
+ * @returns {{ files: string[], unscannable: import("./phi-scan.js").Unscannable[] }}
771
+ */
772
+ walkRoots() {
773
+ /** @type {string[]} */
774
+ const files = [];
775
+ /** @type {import("./phi-scan.js").Unscannable[]} */
776
+ const unscannable = [];
777
+
778
+ // Split the roots by what is actually THERE, before the descent. A root naming a regular file is
779
+ // one target; a root naming a directory is a frontier; a root naming anything else is refused
780
+ // through the same unscannable path an entry inside one would take.
781
+ /** @type {string[]} */
782
+ const rootDirs = [];
783
+ for (const root of this.cfg.scanRoots) {
784
+ const abs = root === "." ? this.cfg.repoRoot : join(this.cfg.repoRoot, ...root.split("/"));
785
+ const stats = lstatOrNull(abs);
786
+ if (stats === null) continue; // A missing root, skipped. See the docblock above.
787
+ if (stats.isDirectory()) rootDirs.push(abs);
788
+ else if (stats.isFile()) {
789
+ if (this.cfg.isWalkReadable(this.normalizePath(abs))) files.push(abs);
790
+ } else {
791
+ unscannable.push({ path: this.normalizePath(abs), kind: statsKind(stats) });
792
+ }
793
+ }
794
+
795
+ let frontier = rootDirs;
796
+ const visited = new Set(frontier);
797
+
798
+ while (frontier.length > 0) {
799
+ /** @type {string[]} */
800
+ const nextDirs = [];
801
+ for (const dir of frontier) {
802
+ if (!existsSync(dir)) continue;
803
+ for (const e of readdirSync(dir, { withFileTypes: true })) {
804
+ const full = join(dir, e.name);
805
+ if (e.isDirectory()) {
806
+ if (e.name === ".git") continue;
807
+ if (visited.has(full)) continue;
808
+ visited.add(full);
809
+ nextDirs.push(full);
810
+ } else if (e.isFile()) {
811
+ // A READ filter. The branch below is deliberately NOT subject to it: that exemption is a
812
+ // judgement about a file whose bytes the walk could have read, and a link's name is no
813
+ // evidence at all about what is on the other side.
814
+ if (!this.cfg.isWalkReadable(this.normalizePath(full))) continue;
815
+ files.push(full);
816
+ } else {
817
+ unscannable.push({ path: this.normalizePath(full), kind: direntKind(e) });
818
+ }
819
+ }
820
+ }
821
+ frontier = this.pruneIgnoredDirs(nextDirs);
822
+ }
823
+
824
+ files.sort((a, b) => (this.normalizePath(a) < this.normalizePath(b) ? -1 : 1));
825
+ unscannable.sort((a, b) => (a.path < b.path ? -1 : 1));
826
+ return { files, unscannable };
827
+ }
828
+
829
+ /**
830
+ * @param {string[]} dirs
831
+ * @returns {string[]} The subset git does not report ignored.
832
+ */
833
+ pruneIgnoredDirs(dirs) {
834
+ if (dirs.length === 0) return dirs;
835
+ const ignored = this.gitIgnored(dirs);
836
+ return dirs.filter((d) => !ignored.has(this.normalizePath(d)));
837
+ }
838
+
839
+ // -------------------------------------------------------------------------
840
+ // Enumeration: the three routes
841
+ // -------------------------------------------------------------------------
842
+
843
+ /**
844
+ * `all` mode's enumeration: the walk, PLUS the in-scope index the union half reads.
845
+ *
846
+ * @returns {{ targets: import("./phi-scan.js").Target[], index: Map<string, import("./phi-scan.js").IndexEntry> }}
847
+ */
848
+ buildTargetsForAll() {
849
+ const { files, unscannable } = this.walkRoots();
850
+
851
+ // One `git check-ignore` over both lists, so a link and a file get the same boundary.
852
+ const ignored = this.gitIgnored([
853
+ ...files.map((f) => this.normalizePath(f)),
854
+ ...unscannable.map((u) => u.path),
855
+ ]);
856
+
857
+ this.refuseUnscannable(
858
+ unscannable.filter((u) => !ignored.has(u.path) && !this.cfg.excludedPaths.has(u.path)),
859
+ "The walk can neither read such an entry nor vouch for what is on the other side of it.",
860
+ "Remove it, replace it with a regular file, or (if it is genuinely not part of the corpus) " +
861
+ "untrack it and add it to .gitignore.",
862
+ );
863
+
864
+ const listed = this.gitIndexEntries();
865
+ if (listed === null) {
866
+ throw new InvocationError(
867
+ "refusing the sweep: git could not name this repository's index, or named it empty, so the " +
868
+ "sweep would be the working-tree walk's word alone and could report clean over tracked " +
869
+ "bytes it never opened. Run it inside a git repository with a readable index.",
870
+ );
871
+ }
872
+
873
+ // Unmerged first, and under its OWN sentence: an unmerged path is not a link and not a gitlink,
874
+ // and reporting it as one sends a developer looking for something that is not there.
875
+ this.refuseUnscannable(
876
+ listed.unmerged
877
+ .filter((p) => this.isUnderScanRoot(p) && !this.cfg.excludedPaths.has(p))
878
+ .map((p) => ({ path: p, kind: "no stage-0 blob" })),
879
+ "An unmerged path has no single merged blob, so there is no one set of bytes git carries here " +
880
+ "for the sweep to read, only the conflicting sides and, when there is one, their base.",
881
+ "Resolve the conflict and stage the result, then re-run.",
882
+ { one: "path is unmerged", many: "paths are unmerged" },
883
+ );
884
+
885
+ // The index's own non-blob entries, refused BEFORE anything is read so a developer is not made
886
+ // to wait out a whole sweep for it. `120000` is a symbolic link, whose blob is its TARGET PATH
887
+ // and not any content; `160000` is a gitlink, which carries a commit id and no bytes at this
888
+ // path at all.
889
+ this.refuseUnscannable(
890
+ [...listed.entries]
891
+ .filter(
892
+ ([p, e]) =>
893
+ this.isUnderScanRoot(p) &&
894
+ !this.cfg.regularBlobModes.has(e.mode) &&
895
+ !this.cfg.excludedPaths.has(p),
896
+ )
897
+ .map(([p, e]) => ({ path: p, kind: gitModeKind(e.mode) })),
898
+ "Git records no readable content at such a path, so scanning it would prove nothing about " +
899
+ "what it stands for.",
900
+ "Untrack it, or replace it with a regular file.",
901
+ // Its own noun: the offender is an INDEX RECORD, and a gitlink's working tree may not exist at
902
+ // all, so "not a regular file" would send a developer to a path where there is nothing to see.
903
+ { one: "index entry is not a regular blob", many: "index entries are not regular blobs" },
904
+ );
905
+
906
+ const targets = files
907
+ .map((abs) => ({ abs, rel: this.normalizePath(abs) }))
908
+ .filter(({ rel }) => !ignored.has(rel) && !this.cfg.excludedPaths.has(rel))
909
+ .map(({ abs, rel }) => ({ path: rel, read: () => readFileSync(abs) }));
910
+ return { targets, index: listed.entries };
911
+ }
912
+
913
+ /**
914
+ * The in-scope tracked paths the union half is entitled to read.
915
+ *
916
+ * IT IS COMPUTED BEFORE THE FIRST BYTE IS READ, AND THAT IS LOAD-BEARING. This set is part of what
917
+ * `all` mode ENUMERATES, so both completeness tiers see it: a bypass naming a tracked-but-absent
918
+ * path subtracts something real rather than being refused as naming nothing, and a target that
919
+ * ends up unread is named by the unread refusal whichever route would have read it.
920
+ *
921
+ * @param {Map<string, import("./phi-scan.js").IndexEntry>} index
922
+ * @returns {string[]}
923
+ */
924
+ unionCandidatePaths(index) {
925
+ return [...index]
926
+ .filter(
927
+ ([p, e]) =>
928
+ this.cfg.regularBlobModes.has(e.mode) &&
929
+ this.isUnderScanRoot(p) &&
930
+ this.cfg.isWalkReadable(p),
931
+ )
932
+ .filter(([p]) => !this.cfg.excludedPaths.has(p))
933
+ .map(([p]) => p);
934
+ }
935
+
936
+ /**
937
+ * THE UNION HALF of `all` mode: the bytes git carries at every in-scope tracked path whose bytes
938
+ * the walk did not already read VERBATIM.
939
+ *
940
+ * `readOids` maps a path the walk actually READ to the object id of what it read. A path absent
941
+ * from it was never opened, whatever the reason, so its blob is scanned; a path present with a
942
+ * DIFFERENT id had a different copy read, so its blob is scanned too. That second case is the EOL
943
+ * axis.
944
+ *
945
+ * @param {Map<string, import("./phi-scan.js").IndexEntry>} index
946
+ * @param {Map<string, string>} readOids
947
+ * @returns {import("./phi-scan.js").Target[]}
948
+ */
949
+ buildTargetsForGitIndex(index, readOids) {
950
+ /** @type {import("./phi-scan.js").Target[]} */
951
+ const targets = [];
952
+ for (const path of this.unionCandidatePaths(index)) {
953
+ const entry = index.get(path);
954
+ if (entry === undefined) continue;
955
+ if (readOids.get(path) === entry.oid) continue;
956
+ targets.push({
957
+ path,
958
+ origin: "as git carries it",
959
+ // SECURITY: array-form execFileSync, no shell. The object id is git's own output, and naming
960
+ // the OBJECT rather than the path is the whole point: it cannot be redirected by whatever
961
+ // the working tree currently holds. `maxBuffer` defaults to 1 MiB, so a larger tracked blob
962
+ // fails the read and REFUSES rather than reporting a truncated scan clean.
963
+ read: () =>
964
+ execFileSync("git", ["cat-file", "blob", entry.oid], {
965
+ cwd: this.cfg.repoRoot,
966
+ encoding: "buffer",
967
+ stdio: ["ignore", "pipe", "pipe"],
968
+ }),
969
+ });
970
+ }
971
+ return targets;
972
+ }
973
+
974
+ /**
975
+ * @param {string[]} paths
976
+ * @returns {import("./phi-scan.js").Target[]}
977
+ */
978
+ buildTargetsForPaths(paths) {
979
+ return paths.map((p) => {
980
+ const abs = isAbsolute(p) ? p : resolve(this.cfg.repoRoot, p);
981
+ if (!existsSync(abs)) throw new InvocationError(`File not found: ${p}`);
982
+ if (!statSync(abs).isFile()) throw new InvocationError(`Not a regular file: ${p}`);
983
+ return { path: this.normalizePath(abs), read: () => readFileSync(abs) };
984
+ });
985
+ }
986
+
987
+ /**
988
+ * `--staged`: exactly the blobs a commit would carry.
989
+ *
990
+ * `--raw` rather than `--name-only` because the DESTINATION MODE is the only thing that
991
+ * distinguishes a staged regular file from a staged symlink or gitlink. `git show :<path>` does not
992
+ * stand in for it: for a symbolic link it hands back the target path as if it were content, and it
993
+ * is the mode, not the answer, that says so.
994
+ *
995
+ * `--diff-filter=d` IS AN EXCLUSION ("everything EXCEPT deletions"), NOT AN ALLOW-LIST OF STATUS
996
+ * LETTERS, AND THE POLARITY IS THE WHOLE POINT. An allow-list drops every letter it does not name,
997
+ * silently; that polarity is what made sibling scanners miss `R` (rename) and then `T`
998
+ * (typechange), each found by a separate refuter pass one round apart. An unfamiliar or future
999
+ * status letter can now only ever cost a wasted scan or a loud refusal, never a missed file.
1000
+ * Against an `AMT` allow-list, what `d` newly enumerates is `U` (unmerged), `X` (unknown) and `B`
1001
+ * (broken pairing). Measured on git 2.39.5, a conflicted path lists as one record with destination
1002
+ * mode `000000`, so the stride is unaffected and the mode is not a regular blob, which puts it
1003
+ * through the refusal below instead of past it.
1004
+ *
1005
+ * `T` (TYPECHANGE) IS ENUMERATED, AND LEAVING IT OUT MAKES THE MODE CHECK UNREACHABLE WHENEVER THE
1006
+ * FILE IS ALREADY TRACKED: replacing a TRACKED regular file with a link is not an add and not a
1007
+ * modify, so an `AM` allow-list deletes the record before any mode can be read.
1008
+ *
1009
+ * `--no-renames` IS SEPARATELY LOAD-BEARING. Rename detection is on by default, so `git mv <link>
1010
+ * <scan root>/<name>` stages as one record carrying TWO paths, which desyncs the two-field stride.
1011
+ * Turning detection off makes the destination arrive as an ordinary single-path `A` and the source
1012
+ * a `D` the filter drops, which makes the stride STRUCTURAL rather than conditional on the
1013
+ * caller's `diff.renames` setting.
1014
+ *
1015
+ * @returns {import("./phi-scan.js").Target[]}
1016
+ */
1017
+ buildTargetsForStaged() {
1018
+ let listBuf;
1019
+ try {
1020
+ // SECURITY: array-form execFileSync, no shell.
1021
+ listBuf = execFileSync(
1022
+ "git",
1023
+ ["diff", "--cached", "--raw", "-z", "--no-renames", "--diff-filter=d"],
1024
+ {
1025
+ cwd: this.cfg.repoRoot,
1026
+ encoding: "buffer",
1027
+ stdio: ["ignore", "pipe", "pipe"],
1028
+ maxBuffer: 64 * 1024 * 1024,
1029
+ },
1030
+ );
1031
+ } catch (err) {
1032
+ throw new InvocationError(
1033
+ `git diff --cached failed: ${err instanceof Error ? err.message : String(err)}`,
1034
+ );
1035
+ }
1036
+
1037
+ // `--raw -z` emits `<info>\0<path>\0` per record. `R` and `C` are the only statuses carrying a
1038
+ // SECOND path and `--no-renames` means git cannot emit either, so the stride is two fields. A
1039
+ // record that does not parse REFUSES rather than being skipped: a silently shortened list is
1040
+ // exactly the shape this scan must never report clean over.
1041
+ const fields = listBuf.toString("utf8").split("\0");
1042
+ /** @type {{ path: string, mode: string }[]} */
1043
+ const staged = [];
1044
+ let i = 0;
1045
+ while (i < fields.length) {
1046
+ const info = fields[i];
1047
+ if (info === undefined || info.length === 0) {
1048
+ i += 1;
1049
+ continue;
1050
+ }
1051
+ const m = RAW_RECORD.exec(info);
1052
+ const mode = m?.[1];
1053
+ const path = fields[i + 1];
1054
+ if (mode === undefined || path === undefined || path.length === 0) {
1055
+ throw new InvocationError(
1056
+ "could not read the output of `git diff --cached --raw -z`: unrecognized record. " +
1057
+ "Refusing rather than scanning a list that may be short.",
1058
+ );
1059
+ }
1060
+ staged.push({ path, mode });
1061
+ i += 2;
1062
+ }
1063
+
1064
+ // THE REFUSAL KEYS ON THE ROOT HALF OF SCOPE, NOT ON THE READ FILTER. Running `isStagedReadable`
1065
+ // first would let a link whose NAME the read filter drops fall out through a filter that exists
1066
+ // to judge a file's BYTES, and this route would then disagree with the walk about the same entry.
1067
+ this.refuseUnscannable(
1068
+ staged
1069
+ .filter(
1070
+ (s) =>
1071
+ this.isUnderScanRoot(s.path) &&
1072
+ !this.cfg.regularBlobModes.has(s.mode) &&
1073
+ !this.cfg.excludedPaths.has(s.path),
1074
+ )
1075
+ .map((s) => ({ path: s.path, kind: gitModeKind(s.mode) })),
1076
+ "The index holds no file content for such an entry, so scanning it would prove nothing about " +
1077
+ "what it refers to.",
1078
+ "Unstage it, or replace it with a regular file.",
1079
+ );
1080
+
1081
+ // THE CONTAINMENT IS ENFORCED, NOT ASSUMED, AND IT USED TO BE THE WORD "by construction".
1082
+ // `isStagedReadable` and `scanRoots` are two independent keys a repo fills in, and nothing
1083
+ // relates them. A reviewer measured what that costs: with roots narrowed to `src` and this
1084
+ // filter left at the shared Markdown exemption, a STAGED mode-120000 entry under
1085
+ // `test/fixtures/` was outside every scan root, so the non-regular refusal above never saw it,
1086
+ // and the route then ENUMERATED it, READ it, handed the link's TARGET PATH to the detector as
1087
+ // if it were content, counted the scan complete, and printed `OK: no hits` at exit 0. That is
1088
+ // exactly the hole the two-predicate rule exists to close, arriving through configuration
1089
+ // instead of through code.
1090
+ //
1091
+ // So a staged path this filter admits and `isUnderScanRoot` does not is REFUSED. Narrowing it
1092
+ // silently to the intersection would be the wrong repair: it would hide a misconfiguration that
1093
+ // the author needs to see, in the one place the gate is a commit blocker.
1094
+ this.refuseUnscannable(
1095
+ staged
1096
+ .filter(
1097
+ (s) =>
1098
+ this.cfg.isStagedReadable(s.path) &&
1099
+ !this.isUnderScanRoot(s.path) &&
1100
+ !this.cfg.excludedPaths.has(s.path),
1101
+ )
1102
+ .map((s) => ({ path: s.path, kind: "outside every scan root" })),
1103
+ "`isStagedReadable` admits a path no scan root covers, so the checks that key on the ROOT " +
1104
+ "half of scope never ran for it: its bytes would be read without anything having " +
1105
+ "established that there are bytes there to read.",
1106
+ "Widen `scanRoots` to cover it, or narrow `isStagedReadable` to stay inside them.",
1107
+ {
1108
+ one: "staged path is readable but outside every scan root",
1109
+ many: "staged paths are readable but outside every scan root",
1110
+ },
1111
+ );
1112
+
1113
+ // Every remaining readable record is a regular blob under a scan root: anything non-regular was
1114
+ // refused above, and anything outside the roots was refused just now.
1115
+ return staged
1116
+ .filter((s) => this.cfg.isStagedReadable(s.path) && !this.cfg.excludedPaths.has(s.path))
1117
+ .map((s) => s.path)
1118
+ .map((relPath) => ({
1119
+ path: relPath,
1120
+ // SECURITY: array-form execFileSync, no shell. `:<path>` is a git pathspec.
1121
+ read: () =>
1122
+ execFileSync("git", ["show", `:${relPath}`], {
1123
+ cwd: this.cfg.repoRoot,
1124
+ encoding: "buffer",
1125
+ stdio: ["ignore", "pipe", "pipe"],
1126
+ }),
1127
+ }));
1128
+ }
1129
+
1130
+ /**
1131
+ * Refuse over entries the enumeration reached and cannot scan. EVERY offender IN THE GROUP is
1132
+ * named, not just the first: a developer who has to re-run the gate once per link learns to
1133
+ * distrust it.
1134
+ *
1135
+ * A refusal names the entry's own repo-relative path and an engine-owned token for its kind. IT
1136
+ * NEVER REPORTS A LINK TARGET, which is text off the working tree and can itself carry PHI.
1137
+ *
1138
+ * `noun` is overridable because the refusal must say something TRUE about what it refused: an
1139
+ * unmerged path is not a non-regular file, it is a path with no single blob.
1140
+ *
1141
+ * @param {import("./phi-scan.js").Unscannable[]} entries
1142
+ * @param {string} why
1143
+ * @param {string} remedy
1144
+ * @param {{ one: string, many: string }} [noun]
1145
+ */
1146
+ refuseUnscannable(
1147
+ entries,
1148
+ why,
1149
+ remedy,
1150
+ noun = { one: "entry is not a regular file", many: "entries are not regular files" },
1151
+ ) {
1152
+ if (entries.length === 0) return;
1153
+ const lines = entries.map((u) => ` - ${u.path} (${u.kind})`).join("\n");
1154
+ const phrase = entries.length === 1 ? noun.one : noun.many;
1155
+ throw new InvocationError(
1156
+ `refusing the scan: ${String(entries.length)} ${phrase}:\n${lines}\n${why} ${remedy}`,
1157
+ );
1158
+ }
1159
+
1160
+ // -------------------------------------------------------------------------
1161
+ // Detection
1162
+ // -------------------------------------------------------------------------
1163
+
1164
+ /**
1165
+ * The format-agnostic FLOOR: a dashed Social Security Number shape, and an email at a domain the
1166
+ * allow-list does not declare.
1167
+ *
1168
+ * BOTH BRANCHES CONSULT THE ALLOW-LIST, AND THE SSN BRANCH DOING SO IS A CORRECTION. A sibling's
1169
+ * refuter measured that its dashed-SSN branch consulted nothing, so the footer's claim that the
1170
+ * token allow-list "is the only remedy that reaches a clean run" was FALSE for exactly that
1171
+ * branch: a developer meeting it had no remedy at all, the bypass having been closed. Declaring an
1172
+ * identifier is a reviewed, committed act, which is the mechanism this gate is built on; a
1173
+ * detector that cannot be answered is a detector people route around.
1174
+ *
1175
+ * The declared form is matched BOTH as written and with its separators removed, so one `ID` entry
1176
+ * covers both renderings rather than requiring a repo to guess which one a fixture uses.
1177
+ *
1178
+ * @param {string} path The reported LOCUS.
1179
+ * @param {string} content
1180
+ * @param {import("./phi-scan.js").AllowList} allow
1181
+ * @param {import("./phi-scan.js").Hit[]} hits
1182
+ */
1183
+ scanCommonShapes(path, content, allow, hits) {
1184
+ for (const m of content.matchAll(/\b\d{3}-\d{2}-\d{4}\b/g)) {
1185
+ const value = m[0];
1186
+ if (allow.ids.has(value.toUpperCase())) continue;
1187
+ if (allow.ids.has(value.replace(/\D/g, ""))) continue;
1188
+ hits.push({ path, segment: "(ssn)", value, reason: "dashed SSN pattern" });
1189
+ }
1190
+ for (const m of content.matchAll(/\b[A-Za-z0-9._%+-]+@([A-Za-z0-9.-]+\.[A-Za-z]{2,})\b/g)) {
1191
+ const domain = (m[1] ?? "").toLowerCase();
1192
+ if (!allow.emailDomains.has(domain)) {
1193
+ hits.push({ path, segment: "(email)", value: m[0], reason: "email with non-test domain" });
1194
+ }
1195
+ }
1196
+ }
1197
+
1198
+ /**
1199
+ * Scan one target and RETURN THE BYTES IT OBSERVED. The bytes are returned rather than a boolean
1200
+ * so `all` mode can ask whether the walk already read exactly what the index carries at this path.
1201
+ *
1202
+ * THE CALLER'S DETECTOR IS HANDED THE LOCUS, NOT THE TARGET PATH, so a hit from the union half
1203
+ * cannot be reported against a bare path a developer would open and find clean. That used to be a
1204
+ * sentence in a comment; here it is the only path a caller can reach.
1205
+ *
1206
+ * A DETECTOR THAT THROWS REFUSES THE SCAN rather than escaping to node's own exit code. A
1207
+ * per-standard parser meeting input it cannot handle is an ordinary event, and the code node would
1208
+ * pick is the one this contract reserves for HITS FOUND.
1209
+ *
1210
+ * 🛑 THE REFUSAL PRINTS THE DETECTOR'S OWN MESSAGE VERBATIM, AND THAT IS A DISCLOSED RESIDUAL
1211
+ * RATHER THAN A CLOSED ONE. Everywhere else this engine prints only a repo-relative path and a
1212
+ * token from a closed set, precisely because a diagnostic ABOUT a PHI leak is itself a PHI
1213
+ * surface. A caller's error message is text this engine cannot vouch for, and a parser that
1214
+ * interpolates the record it choked on will put that record on stderr, which is CI logs. It is not
1215
+ * suppressed, because a refusal nobody can diagnose is its own defect, and it is not new: before
1216
+ * this engine existed the same string reached the same place through node's default handler. What
1217
+ * is new is that it is written down, here and on `DetectFn`. Throw a message that names the
1218
+ * position, never the content.
1219
+ *
1220
+ * @param {import("./phi-scan.js").Target} target
1221
+ * @param {import("./phi-scan.js").AllowList} allow
1222
+ * @param {import("./phi-scan.js").Hit[]} hits
1223
+ * @returns {Buffer}
1224
+ */
1225
+ scanTarget(target, allow, hits) {
1226
+ let buf;
1227
+ try {
1228
+ buf = target.read();
1229
+ } catch (err) {
1230
+ throw new InvocationError(
1231
+ `could not read ${target.path}: ${err instanceof Error ? err.message : String(err)}`,
1232
+ );
1233
+ }
1234
+ const text = buf.toString("utf8");
1235
+ // Scope is decided on the target's own path; only the REPORTED locus carries the origin label,
1236
+ // so a labelled target is never a differently-scoped one.
1237
+ const locus = target.origin === undefined ? target.path : `${target.path} (${target.origin})`;
1238
+
1239
+ this.scanCommonShapes(locus, text, allow, hits);
1240
+
1241
+ const detect = this.cfg.detect;
1242
+ if (detect !== undefined) {
1243
+ try {
1244
+ detect({
1245
+ path: locus,
1246
+ text,
1247
+ bytes: buf,
1248
+ allow,
1249
+ hit: (h) => {
1250
+ hits.push({ path: locus, segment: h.segment, value: h.value, reason: h.reason });
1251
+ },
1252
+ });
1253
+ } catch (err) {
1254
+ throw new InvocationError(
1255
+ `the field detector threw on ${locus}: ${err instanceof Error ? err.message : String(err)}`,
1256
+ );
1257
+ }
1258
+ }
1259
+
1260
+ return buf;
1261
+ }
1262
+
1263
+ // -------------------------------------------------------------------------
1264
+ // Reporting
1265
+ // -------------------------------------------------------------------------
1266
+
1267
+ /**
1268
+ * Print the hits.
1269
+ *
1270
+ * SPLIT FROM THE CLEAN LINE ON PURPOSE. Every refusal that can follow a read prints the hits
1271
+ * FIRST, so a run that is both incomplete and carrying hits prints both. The clean line is printed
1272
+ * only once every tier has passed, so it can never appear beside a refusal.
1273
+ *
1274
+ * THE FOOTER IS SCOPED TO WHAT THIS ENGINE KNOWS. It does not claim the allow-list reaches a clean
1275
+ * run for every hit, because a per-repo detector supplied through `detect` may raise one without
1276
+ * consulting the allow-list at all, and a sibling shipped exactly that claim and had it refuted.
1277
+ * What is true, and all that is said: the two branches of the cross-cutting floor consult it, a
1278
+ * whole-file bypass is recorded and then refused, and a detector that consults nothing has to be
1279
+ * changed rather than argued with.
1280
+ *
1281
+ * @param {import("./phi-scan.js").Hit[]} hits
1282
+ */
1283
+ reportHits(hits) {
1284
+ if (hits.length === 0) return;
1285
+ /** @type {Map<string, import("./phi-scan.js").Hit[]>} */
1286
+ const byPath = new Map();
1287
+ for (const h of hits) {
1288
+ const arr = byPath.get(h.path);
1289
+ if (arr) arr.push(h);
1290
+ else byPath.set(h.path, [h]);
1291
+ }
1292
+ for (const [path, group] of byPath) {
1293
+ process.stderr.write(`[phi-scan] HIT: ${path}\n`);
1294
+ for (const h of group) {
1295
+ process.stderr.write(
1296
+ ` segment=${h.segment} value=${JSON.stringify(h.value)} (${h.reason})\n`,
1297
+ );
1298
+ }
1299
+ }
1300
+ process.stderr.write(
1301
+ `[phi-scan] ${String(hits.length)} hit(s) across ${String(byPath.size)} file(s). ` +
1302
+ `The cross-cutting floor (SSN and email shapes) consults ${this.relAllowList()}, so a ` +
1303
+ `genuinely synthetic identifier is declared there: a token-level, reviewed declaration. A ` +
1304
+ `hit raised by this repo's own field detectors is answerable that way only if that detector ` +
1305
+ `consults the allow-list; one that does not has to be changed rather than declared around. ` +
1306
+ `A whole-file --allow-fixture bypass is recorded and then REFUSED, because a scan that ` +
1307
+ `never opened a file has no clean verdict to give about it.\n`,
1308
+ );
1309
+ }
1310
+
1311
+ // -------------------------------------------------------------------------
1312
+ // The run
1313
+ // -------------------------------------------------------------------------
1314
+
1315
+ /**
1316
+ * ===========================================================================================
1317
+ * THE EXIT CONTRACT IS THE CALLER'S, NOT THIS FILE'S. The three codes come from `exitCodes`, and
1318
+ * their meanings are:
1319
+ *
1320
+ * clean the scan ran, READ EVERY TARGET IT ENUMERATED, and found nothing.
1321
+ * hits this corpus contains something that looks like PHI. Nothing this engine RAISES ever
1322
+ * takes it. It is NOT exclusive, and the escapes are named rather than left to be
1323
+ * discovered: an allow-list or an override log that EXISTS but cannot be READ throws a
1324
+ * plain `Error`, which is rethrown, and the run takes node's own exit 1.
1325
+ * refuse every state this engine RAISES in which the scan cannot account for something: a bad
1326
+ * argument, a MISSING allow-list, an unlogged bypass, a bypass naming a path this run
1327
+ * does not enumerate, an in-scope entry that is not a regular file, an unparseable
1328
+ * `git diff --cached` record, an index git cannot name or names empty, an in-scope index
1329
+ * entry that is not a regular blob, an in-scope path with no stage-0 blob (unmerged), a
1330
+ * target whose bytes cannot be read, a field detector that threw, and a target
1331
+ * enumerated but never read.
1332
+ * ===========================================================================================
1333
+ *
1334
+ * @returns {number}
1335
+ */
1336
+ run() {
1337
+ const { clean: EXIT_CLEAN, hits: EXIT_HITS, refuse: EXIT_REFUSE } = this.cfg.exitCodes;
1338
+
1339
+ /** @type {{ mode: "all" | "staged" | "paths", paths: string[], allowFixtures: string[] }} */
1340
+ let args;
1341
+ try {
1342
+ args = this.parseArgs(this.cfg.argv);
1343
+ this.validateAllowFixtures(args.allowFixtures);
1344
+ } catch (err) {
1345
+ if (err instanceof InvocationError) {
1346
+ process.stderr.write(`[phi-scan] ${err.message}\n`);
1347
+ return EXIT_REFUSE;
1348
+ }
1349
+ throw err;
1350
+ }
1351
+
1352
+ const allowed = new Set(args.allowFixtures.map((p) => this.normalizePath(p)));
1353
+
1354
+ /** @type {import("./phi-scan.js").AllowList} */
1355
+ let allow;
1356
+ /** @type {import("./phi-scan.js").Target[]} */
1357
+ let targets;
1358
+ /** `all` mode's index, read once: it is the union half's whole enumeration. */
1359
+ /** @type {Map<string, import("./phi-scan.js").IndexEntry> | null} */
1360
+ let index = null;
1361
+ try {
1362
+ // `loadAllowList()` IS INSIDE THIS HANDLER, AND THAT PLACEMENT IS THE POINT. Outside it, a
1363
+ // missing allow-list escaped as an uncaught throw and took node's exit 1, which this contract
1364
+ // reserves for "hits found".
1365
+ allow = this.loadAllowList();
1366
+ if (args.mode === "staged") targets = this.buildTargetsForStaged();
1367
+ else if (args.mode === "paths") targets = this.buildTargetsForPaths(args.paths);
1368
+ else {
1369
+ const built = this.buildTargetsForAll();
1370
+ targets = built.targets;
1371
+ index = built.index;
1372
+ }
1373
+ } catch (err) {
1374
+ if (err instanceof InvocationError) {
1375
+ process.stderr.write(`[phi-scan] ${err.message}\n`);
1376
+ return EXIT_REFUSE;
1377
+ }
1378
+ throw err;
1379
+ }
1380
+
1381
+ // ENUMERATED: the set of paths this run declared it would read. Everything the read filters
1382
+ // dropped upstream never became a target and is not in here, which is why the completeness rule
1383
+ // does not fire on them. In `all` mode it is the walk's targets UNION the in-scope tracked paths.
1384
+ const enumerated = new Set(targets.map((t) => t.path));
1385
+ if (index !== null) for (const p of this.unionCandidatePaths(index)) enumerated.add(p);
1386
+
1387
+ // TIER: A BYPASS MUST NAME A PATH THIS RUN ENUMERATES. Otherwise it subtracts nothing, and a
1388
+ // flag that subtracts nothing lets a developer believe a file was acknowledged when the run
1389
+ // never had it in scope. Compared by DIFFERENCE, and every offender is named.
1390
+ //
1391
+ // THIS TIER FIRES BEFORE ANY TARGET IS READ, so no hit exists for it to swallow. That is a
1392
+ // narrower guarantee than the unread tier's and is stated as such rather than generalised.
1393
+ const unmatched = [...allowed].filter((p) => !enumerated.has(p));
1394
+ if (unmatched.length > 0) {
1395
+ process.stderr.write(
1396
+ `[phi-scan] --allow-fixture names ${String(unmatched.length)} path(s) this run does not ` +
1397
+ `enumerate, so the flag subtracts nothing:\n${unmatched.map((p) => ` - ${p}`).join("\n")}\n` +
1398
+ `Scan a corpus that contains the path, or drop the flag.\n`,
1399
+ );
1400
+ return EXIT_REFUSE;
1401
+ }
1402
+
1403
+ /** @type {import("./phi-scan.js").Hit[]} */
1404
+ const hits = [];
1405
+ // READ: filled in only after a target's bytes have actually been through `scanTarget`. This is
1406
+ // evidence of observation, never a plan to observe.
1407
+ /** @type {Set<string>} */
1408
+ const read = new Set();
1409
+ /** Path -> object id of the bytes the walk actually read, so the union can skip a re-scan. */
1410
+ /** @type {Map<string, string>} */
1411
+ const readOids = new Map();
1412
+ const objectHash = index === null ? null : this.gitObjectHash();
1413
+
1414
+ /**
1415
+ * @param {import("./phi-scan.js").Target[]} batch
1416
+ * @returns {number | null} A refusal code, or `null` when the batch completed.
1417
+ */
1418
+ const sweep = (batch) => {
1419
+ for (const t of batch) {
1420
+ if (allowed.has(t.path)) continue;
1421
+ let bytes;
1422
+ try {
1423
+ bytes = this.scanTarget(t, allow, hits);
1424
+ } catch (err) {
1425
+ if (err instanceof InvocationError) {
1426
+ // HITS FOUND SO FAR ARE PRINTED BEFORE THIS REFUSAL, DELIBERATELY, AND THIS IS A CHANGE
1427
+ // FROM THE COPIED SCANNERS. A refuter measured the old ordering: a fatal partway through
1428
+ // the sweep discarded every hit found before it, so a consumer saw a refusal with no
1429
+ // indication that PHI had already been found. The refusal still wins the exit code, and
1430
+ // the clean line is still unreachable from here, so nothing is reported as accounted for
1431
+ // that is not; what changes is that a finding already made is not thrown away.
1432
+ this.reportHits(hits);
1433
+ process.stderr.write(`[phi-scan] ${err.message}\n`);
1434
+ return EXIT_REFUSE;
1435
+ }
1436
+ throw err;
1437
+ }
1438
+ read.add(t.path);
1439
+ if (objectHash !== null && t.origin === undefined) {
1440
+ const oid = this.blobOid(objectHash, bytes);
1441
+ if (oid !== null) readOids.set(t.path, oid);
1442
+ }
1443
+ }
1444
+ return null;
1445
+ };
1446
+
1447
+ const walkFailure = sweep(targets);
1448
+ if (walkFailure !== null) return walkFailure;
1449
+
1450
+ // THE UNION. It runs AFTER the walk, not instead of it, and only over the paths the walk did not
1451
+ // already read verbatim.
1452
+ if (index !== null) {
1453
+ const unionFailure = sweep(this.buildTargetsForGitIndex(index, readOids));
1454
+ if (unionFailure !== null) return unionFailure;
1455
+ }
1456
+
1457
+ // THE COMPLETENESS RULE. A SET DIFFERENCE, NEVER A SIZE COMPARISON: a count counts the targets
1458
+ // that DID get read, so `n read of n targets` is exactly the arithmetic that hides which ones did
1459
+ // not. Names every offender.
1460
+ const unread = [...enumerated].filter((p) => !read.has(p));
1461
+
1462
+ // Hits FIRST, so the refusal below can never swallow one.
1463
+ this.reportHits(hits);
1464
+
1465
+ if (unread.length > 0) {
1466
+ process.stderr.write(
1467
+ `[phi-scan] refusing the scan: ${String(unread.length)} target(s) were enumerated and ` +
1468
+ `never read:\n${unread.map((p) => ` - ${p}`).join("\n")}\n` +
1469
+ `A scan that did not open a file has no clean verdict to give about it. If the file is ` +
1470
+ `genuinely synthetic, declare its identifiers in ${this.relAllowList()} rather than ` +
1471
+ `withdrawing the file from the scan.\n`,
1472
+ );
1473
+ return EXIT_REFUSE;
1474
+ }
1475
+
1476
+ if (hits.length > 0) return EXIT_HITS;
1477
+ process.stdout.write("[phi-scan] OK: no hits\n");
1478
+ return EXIT_CLEAN;
1479
+ }
1480
+ }
1481
+
1482
+ /** `:<srcmode> <dstmode> <srcsha> <dstsha> <status>`: the info half of a `--raw -z` record. */
1483
+ const RAW_RECORD = /^:(?:\d{6}) (\d{6}) [0-9a-f]+ [0-9a-f]+ [A-Z]\d*$/;
1484
+
1485
+ /**
1486
+ * `lstatSync` that reports a missing path as `null` rather than throwing. LSTAT, not stat, so a
1487
+ * symbolic link named as a scan root is seen as a link rather than as whatever it points at: the
1488
+ * walk refuses such an entry everywhere else and a root must not be the one place it follows one.
1489
+ *
1490
+ * @param {string} path
1491
+ * @returns {import("node:fs").Stats | null}
1492
+ */
1493
+ function lstatOrNull(path) {
1494
+ try {
1495
+ return lstatSync(path);
1496
+ } catch {
1497
+ return null;
1498
+ }
1499
+ }
1500
+
1501
+ /**
1502
+ * Closed-set, engine-owned description of a `Stats` kind, for a scan root that is neither a
1503
+ * directory nor a regular file. Same closed set as `direntKind`, off the same predicates.
1504
+ *
1505
+ * @param {import("node:fs").Stats} s
1506
+ * @returns {string}
1507
+ */
1508
+ function statsKind(s) {
1509
+ if (s.isSymbolicLink()) return "a symbolic link";
1510
+ if (s.isFIFO()) return "a FIFO";
1511
+ if (s.isSocket()) return "a socket";
1512
+ if (s.isBlockDevice()) return "a block device";
1513
+ if (s.isCharacterDevice()) return "a character device";
1514
+ return "not a regular file";
1515
+ }
1516
+
1517
+ /**
1518
+ * Closed-set, engine-owned description of a directory entry's kind. Nothing off the other side of a
1519
+ * link is ever recorded.
1520
+ *
1521
+ * @param {import("node:fs").Dirent} e
1522
+ * @returns {string}
1523
+ */
1524
+ function direntKind(e) {
1525
+ if (e.isSymbolicLink()) return "a symbolic link";
1526
+ if (e.isFIFO()) return "a FIFO";
1527
+ if (e.isSocket()) return "a socket";
1528
+ if (e.isBlockDevice()) return "a block device";
1529
+ if (e.isCharacterDevice()) return "a character device";
1530
+ return "not a regular file";
1531
+ }
1532
+
1533
+ /**
1534
+ * Closed-set, engine-owned description of a git file mode.
1535
+ *
1536
+ * @param {string} mode
1537
+ * @returns {string}
1538
+ */
1539
+ function gitModeKind(mode) {
1540
+ if (mode === "120000") return "a symbolic link";
1541
+ if (mode === "160000") return "a gitlink (a nested repository)";
1542
+ return `a git mode-${mode} entry`;
1543
+ }