@clear-capabilities/agentic-security-scanner 0.184.0 → 0.186.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (228) hide show
  1. package/CHANGELOG.md +37 -0
  2. package/bin/agentic-security.js +111 -1
  3. package/dist/1128.index.js +912 -0
  4. package/dist/1665.index.js +828 -0
  5. package/dist/2584.index.js +1 -1
  6. package/dist/4171.index.js +2 -2
  7. package/dist/5752.index.js +466 -0
  8. package/dist/7405.index.js +1 -1
  9. package/dist/8158.index.js +1713 -0
  10. package/dist/8798.index.js +1 -1
  11. package/dist/9382.index.js +2 -2
  12. package/dist/9579.index.js +1 -1
  13. package/dist/agentic-security.mjs +29 -29
  14. package/dist/agentic-security.mjs.sha256 +1 -1
  15. package/dist/{dad58d1113a12c23a142.json → ca62ee93079be4c24eee.json} +29 -12
  16. package/dist/frontend/src/shell.js +12 -2
  17. package/package.json +29 -12
  18. package/src/dataflow/CLAUDE.md +46 -1
  19. package/src/dataflow/access-paths.js +12 -2
  20. package/src/dataflow/backward.js +42 -7
  21. package/src/dataflow/cross-service-taint.js +83 -1
  22. package/src/dataflow/demand-query.js +1168 -0
  23. package/src/dataflow/dep-fingerprint.js +718 -0
  24. package/src/dataflow/engine.js +48 -6
  25. package/src/dataflow/exploit-prover.js +6 -2
  26. package/src/dataflow/ifds-precise.js +22 -0
  27. package/src/dataflow/incremental-replay.js +118 -0
  28. package/src/dataflow/incremental-session.js +257 -0
  29. package/src/dataflow/incremental.js +237 -35
  30. package/src/dataflow/index.js +25 -94
  31. package/src/dataflow/ir-cache.js +268 -0
  32. package/src/dataflow/privacy-taint.js +7 -0
  33. package/src/dataflow/summaries.js +72 -6
  34. package/src/dataflow/tabulation.js +80 -24
  35. package/src/engine.js +113 -8
  36. package/src/graph/CLAUDE.md +31 -0
  37. package/src/graph/adapters/artifact.js +81 -0
  38. package/src/graph/adapters/crossrepo.js +75 -0
  39. package/src/graph/adapters/dataflow.js +76 -0
  40. package/src/graph/adapters/provenance.js +92 -0
  41. package/src/graph/adapters/semantic.js +89 -0
  42. package/src/graph/crosswalk-builders.js +152 -0
  43. package/src/graph/crosswalk.js +146 -0
  44. package/src/graph/index.js +15 -0
  45. package/src/graph/observation.js +57 -0
  46. package/src/graph/refs.js +105 -0
  47. package/src/graph/registry.js +129 -0
  48. package/src/graph/relations.js +81 -0
  49. package/src/graph/scc.js +241 -0
  50. package/src/graph/view.js +254 -0
  51. package/src/ir/CLAUDE.md +4 -0
  52. package/src/ir/callgraph.js +36 -8
  53. package/src/ir/index.js +18 -4
  54. package/src/ir/parser-js.js +10 -0
  55. package/src/language/CLAUDE.md +13 -0
  56. package/src/language/nix-option-types.js +170 -0
  57. package/src/language/nix-script-taint.js +46 -3
  58. package/src/language/nix-secrets.js +20 -2
  59. package/src/lineage/CLAUDE.md +72 -8
  60. package/src/lineage/DESIGN_GRAPH_BUILDER.md +1 -1
  61. package/src/lineage/DESIGN_PATH_PROVENANCE.md +6 -0
  62. package/src/lineage/boundary-contract.js +273 -0
  63. package/src/lineage/completion.js +205 -0
  64. package/src/lineage/continuation.js +140 -0
  65. package/src/lineage/control-credit.js +313 -0
  66. package/src/lineage/correlation.js +199 -0
  67. package/src/lineage/coverage.js +8 -1
  68. package/src/lineage/cross-repo-link.js +22 -0
  69. package/src/lineage/cut-planner.js +548 -0
  70. package/src/lineage/driver.js +219 -66
  71. package/src/lineage/engine.js +166 -21
  72. package/src/lineage/export-json.js +5 -0
  73. package/src/lineage/graph-builder.js +84 -7
  74. package/src/lineage/haskell-view.js +59 -0
  75. package/src/lineage/index.js +3 -0
  76. package/src/lineage/nix-view.js +35 -0
  77. package/src/lineage/owned-build.js +77 -0
  78. package/src/lineage/owned-worker-entry.js +22 -0
  79. package/src/lineage/owned-worker.js +95 -0
  80. package/src/lineage/path-evidence.js +123 -0
  81. package/src/lineage/path-predicates.js +529 -0
  82. package/src/lineage/path-query.js +189 -20
  83. package/src/lineage/path-store.js +191 -17
  84. package/src/lineage/scc-solver.js +186 -0
  85. package/src/lineage/smt-orchestrator.js +174 -0
  86. package/src/lineage/smt-z3-backend.js +294 -0
  87. package/src/lineage/source-seeding.js +36 -2
  88. package/src/lineage/summaries.js +66 -4
  89. package/src/lineage/value-versions.js +130 -0
  90. package/src/lineage/witness-selection.js +95 -0
  91. package/src/pipeline/scan-health.js +18 -0
  92. package/src/posture/CLAUDE.md +17 -0
  93. package/src/posture/business-logic.js +16 -7
  94. package/src/posture/dag-commands.js +442 -0
  95. package/src/posture/evidence-bundle.js +1 -1
  96. package/src/posture/finding-defaults.js +19 -0
  97. package/src/posture/git-history.js +91 -1
  98. package/src/posture/witness-bundle.js +573 -0
  99. package/src/posture/witness-runtime.js +276 -0
  100. package/src/report/index.js +2 -0
  101. package/src/sast/CLAUDE.md +2 -0
  102. package/src/sast/_call-args.js +70 -0
  103. package/src/sast/init-scope.js +87 -0
  104. package/src/sast/listen-bind-address.js +49 -0
  105. package/src/sast/redirect-target.js +106 -0
  106. package/src/sast/reflected-input.js +76 -0
  107. package/src/sast/send-file-root.js +27 -0
  108. package/src/sast/sensitive-data-logging.js +38 -1
  109. package/dist/119.index.js +0 -197
  110. package/dist/128.index.js +0 -129
  111. package/dist/1293.index.js +0 -5689
  112. package/dist/1336.index.js +0 -16182
  113. package/dist/1351.index.js +0 -133
  114. package/dist/1387.index.js +0 -271
  115. package/dist/1437.index.js +0 -23
  116. package/dist/1445.index.js +0 -110
  117. package/dist/1505.index.js +0 -151
  118. package/dist/1557.index.js +0 -293
  119. package/dist/1621.index.js +0 -311
  120. package/dist/1751.index.js +0 -160
  121. package/dist/1765.index.js +0 -727
  122. package/dist/1861.index.js +0 -690
  123. package/dist/1906.index.js +0 -433
  124. package/dist/208.index.js +0 -1631
  125. package/dist/2223.index.js +0 -159
  126. package/dist/2238.index.js +0 -295
  127. package/dist/2530.index.js +0 -152
  128. package/dist/2565.index.js +0 -116
  129. package/dist/2574.index.js +0 -127
  130. package/dist/2620.index.js +0 -345
  131. package/dist/2773.index.js +0 -98
  132. package/dist/2775.index.js +0 -32
  133. package/dist/2966.index.js +0 -171
  134. package/dist/2972.index.js +0 -87
  135. package/dist/2991.index.js +0 -131
  136. package/dist/3063.index.js +0 -833
  137. package/dist/3075.index.js +0 -227
  138. package/dist/3121.index.js +0 -297
  139. package/dist/3314.index.js +0 -263
  140. package/dist/3480.index.js +0 -1822
  141. package/dist/3481.index.js +0 -24
  142. package/dist/3506.index.js +0 -268
  143. package/dist/3533.index.js +0 -591
  144. package/dist/3653.index.js +0 -450
  145. package/dist/3740.index.js +0 -722
  146. package/dist/3822.index.js +0 -131
  147. package/dist/3868.index.js +0 -292
  148. package/dist/3918.index.js +0 -192
  149. package/dist/4090.index.js +0 -288
  150. package/dist/4113.index.js +0 -1286
  151. package/dist/4121.index.js +0 -1190
  152. package/dist/4142.index.js +0 -690
  153. package/dist/4168.index.js +0 -509
  154. package/dist/4281.index.js +0 -993
  155. package/dist/4336.index.js +0 -436
  156. package/dist/4387.index.js +0 -532
  157. package/dist/4413.index.js +0 -302
  158. package/dist/4476.index.js +0 -4510
  159. package/dist/4491.index.js +0 -133
  160. package/dist/4538.index.js +0 -185
  161. package/dist/4684.index.js +0 -319
  162. package/dist/4740.index.js +0 -165
  163. package/dist/4780.index.js +0 -209
  164. package/dist/4802.index.js +0 -225
  165. package/dist/4812.index.js +0 -379
  166. package/dist/4835.index.js +0 -126
  167. package/dist/494.index.js +0 -226
  168. package/dist/4991.index.js +0 -549
  169. package/dist/5199.index.js +0 -305
  170. package/dist/5216.index.js +0 -109
  171. package/dist/5439.index.js +0 -301
  172. package/dist/5443.index.js +0 -97
  173. package/dist/5496.index.js +0 -377
  174. package/dist/5738.index.js +0 -240
  175. package/dist/5801.index.js +0 -588
  176. package/dist/5817.index.js +0 -714
  177. package/dist/5858.index.js +0 -223
  178. package/dist/5933.index.js +0 -143
  179. package/dist/601.index.js +0 -23
  180. package/dist/6173.index.js +0 -273
  181. package/dist/6291.index.js +0 -1871
  182. package/dist/6312.index.js +0 -666
  183. package/dist/634.index.js +0 -16182
  184. package/dist/644.index.js +0 -87
  185. package/dist/6508.index.js +0 -283
  186. package/dist/6560.index.js +0 -645
  187. package/dist/6576.index.js +0 -422
  188. package/dist/6579.index.js +0 -271
  189. package/dist/6655.index.js +0 -3221
  190. package/dist/6745.index.js +0 -305
  191. package/dist/6818.index.js +0 -21642
  192. package/dist/6932.index.js +0 -1190
  193. package/dist/6972.index.js +0 -167
  194. package/dist/6995.index.js +0 -163
  195. package/dist/7249.index.js +0 -191
  196. package/dist/7372.index.js +0 -1197
  197. package/dist/7546.index.js +0 -541
  198. package/dist/76.index.js +0 -448
  199. package/dist/7879.index.js +0 -163
  200. package/dist/8025.index.js +0 -130
  201. package/dist/8054.index.js +0 -248
  202. package/dist/808.index.js +0 -1330
  203. package/dist/8097.index.js +0 -201
  204. package/dist/8126.index.js +0 -191
  205. package/dist/8178.index.js +0 -78
  206. package/dist/8382.index.js +0 -93
  207. package/dist/8408.index.js +0 -261
  208. package/dist/8540.index.js +0 -174
  209. package/dist/8635.index.js +0 -802
  210. package/dist/8688.index.js +0 -410
  211. package/dist/8699.index.js +0 -779
  212. package/dist/8891.index.js +0 -457
  213. package/dist/8916.index.js +0 -885
  214. package/dist/9146.index.js +0 -170
  215. package/dist/9168.index.js +0 -841
  216. package/dist/9292.index.js +0 -369
  217. package/dist/9422.index.js +0 -866
  218. package/dist/9440.index.js +0 -453
  219. package/dist/9479.index.js +0 -152
  220. package/dist/9481.index.js +0 -220
  221. package/dist/9539.index.js +0 -100
  222. package/dist/9547.index.js +0 -103
  223. package/dist/9692.index.js +0 -90
  224. package/dist/9800.index.js +0 -462
  225. package/dist/9899.index.js +0 -138
  226. package/dist/9981.index.js +0 -206
  227. package/dist/999.index.js +0 -117
  228. package/dist/9999.index.js +0 -1698
@@ -22,6 +22,7 @@
22
22
  import { parseNix, makeLocator } from './nix-parser.js';
23
23
  import { resolveImportPath } from './nix-ir.js';
24
24
  import { isLanguageExcludedPath } from './discovery.js';
25
+ import { buildOptionTypeIndex } from './nix-option-types.js';
25
26
 
26
27
  export const NIX_SCRIPT_VERSION = 'nix-script-taint/1';
27
28
  const PH = '';
@@ -44,7 +45,7 @@ const TRUSTED_PARAMS = new Set(['pkgs', 'lib', 'stdenv', 'self', 'inputs', 'syst
44
45
  const CONFIG_PARAMS = new Set(['config', 'options', 'osConfig']);
45
46
  const RUNTIME_SECRET_DIRS = /^\/(?:run\/(?:secrets|agenix|credentials|keys)|var\/lib\/(?:secrets|credentials)|etc\/secrets)\b/;
46
47
  const SECRET_NAME = /(?:pass(?:word|wd|phrase)?|psk|secret|token|api_?key|private_?key|credential|auth_?key|client_?secret|bearer|cookie|session_?key)/i;
47
- const PATHISH_TAIL = /(?:file|path|dir|directory|package|pkg|enable|port|user|group|name)$/i;
48
+ const PATHISH_TAIL = /(?:file|path|dir|directory|package|pkg|enable|port|user|group|name|scripts?|commands?|cmds?)$/i;
48
49
  const FETCHERS = new Set(['fetchurl', 'fetchTarball', 'fetchGit', 'fetchzip', 'fetchFromGitHub', 'fetchgit', 'fetchTree', 'fetchClosure', 'fetchMercurial', 'fetchpatch']);
49
50
  const FS_CMDS = new Set(['rm', 'rmdir', 'mv', 'cp', 'chmod', 'chown', 'chgrp', 'mkdir', 'ln', 'touch', 'tee', 'dd', 'install', 'truncate', 'shred', 'unlink', 'rsync', 'tar', 'unzip', 'cat', 'sed', 'find', 'mount', 'umount']);
50
51
  const NET_CMDS = new Set(['curl', 'wget', 'nc', 'ncat', 'netcat', 'ssh', 'scp', 'sftp', 'rsync', 'ftp', 'telnet', 'socat', 'git', 'psql', 'mysql', 'redis-cli', 'mongo', 'aws', 'gsutil']);
@@ -58,7 +59,16 @@ const spanOf = (n) => (n && n.span ? n.span : null);
58
59
  // ── files ────────────────────────────────────────────────────────────────────
59
60
  class Workspace {
60
61
  constructor(files) {
61
- this.files = files; this.parsed = new Map();
62
+ this.files = files; this.parsed = new Map(); this._optionTypes = null;
63
+ }
64
+ /** Declared option types across every file in scope (see nix-option-types.js). Built once. */
65
+ optionTypes() {
66
+ if (!this._optionTypes) {
67
+ const asts = [];
68
+ for (const f of Object.keys(this.files)) { const r = this.parse(f); if (r && r.ast) asts.push(r.ast); }
69
+ this._optionTypes = buildOptionTypeIndex(asts);
70
+ }
71
+ return this._optionTypes;
62
72
  }
63
73
  parse(file) {
64
74
  if (this.parsed.has(file)) return this.parsed.get(file);
@@ -227,7 +237,38 @@ class Evaluator {
227
237
  return { origins: [mkOrigin('argument', `function argument "${name}"`, hops)], protection: null };
228
238
  }
229
239
 
230
- eval(expr, env, depth = 0) {
240
+ /**
241
+ * An option reference whose DECLARED type bounds its value (bool, int, port, package, a listed enum) cannot carry
242
+ * a shell metacharacter or a credential, whatever its name suggests, so it is a constant. An option declared out of
243
+ * scope keeps its origin, except that the module-system convention for a package option (`package`, `finalPackage`,
244
+ * `fooPackage`) is a store path.
245
+ */
246
+ refine(r) {
247
+ if (!r || !r.origins.some((o) => o.kind === 'configuration' || o.kind === 'secret')) return r;
248
+ const idx = this.res.ws.optionTypes();
249
+ let changed = false;
250
+ const origins = r.origins.map((o) => {
251
+ if (o.kind !== 'configuration' && o.kind !== 'secret') return o;
252
+ const m = /^(?:option|module parameter) (?:config|osConfig)\.(.+)$/.exec(o.detail);
253
+ if (!m) return o;
254
+ const decl = idx.get(m[1]);
255
+ if (decl) {
256
+ if (!decl.bounded) return o;
257
+ changed = true;
258
+ return mkOrigin('constant', `${o.detail} (declared ${decl.kind})`, o.chain);
259
+ }
260
+ if (/^(?:package|finalPackage|[a-z][A-Za-z0-9]*Package)$/.test(m[1].split('.').pop())) {
261
+ changed = true;
262
+ return mkOrigin('store', `${o.detail} (a package option)`, o.chain);
263
+ }
264
+ return o;
265
+ });
266
+ return changed ? { ...r, origins: dedupe(origins) } : r;
267
+ }
268
+
269
+ eval(expr, env, depth = 0) { return this.refine(this.evalRaw(expr, env, depth)); }
270
+
271
+ evalRaw(expr, env, depth = 0) {
231
272
  const n = unparen(expr);
232
273
  if (!n || depth > MAX_DEPTH) return this.unknown('depth limit');
233
274
  switch (n.type) {
@@ -364,6 +405,8 @@ class Evaluator {
364
405
  const r = rest.length ? this.combine(rest) : this.constant('empty');
365
406
  return { ...r, origins: dedupe([...(sep ? sep.origins : []), ...r.origins]) };
366
407
  }
408
+ // a hex digest, whatever it was computed from
409
+ if (name === 'hashString' || name === 'hashFile') return this.constant(`${name} (hex digest)`);
367
410
  if (name === 'optionalString' || name === 'optional' || name === 'optionals') return args[1] ? this.eval(args[1], env, depth + 1) : this.constant('empty');
368
411
  if (name === 'getExe' || name === 'getBin' || name === 'getLib' || name === 'getDev' || name === 'getExe\'') return { origins: [mkOrigin('store', `${name} (store path)`)], protection: null };
369
412
  if (name === 'writeText' || name === 'writeTextFile' || name === 'writeShellScript' || name === 'writeShellScriptBin' || name === 'writeScript' || name === 'writeScriptBin' || name === 'runCommand' || name === 'mkDerivation') return { origins: [mkOrigin('store', `${name} output (store path)`)], protection: null };
@@ -217,7 +217,8 @@ class Walker {
217
217
  }
218
218
  case 'app': {
219
219
  if (!asFn) this.inspectApp(file, node, env, path);
220
- this.visit(file, node.fn, env, path, depth + 1, true); this.visit(file, node.arg, env, path, depth + 1, false); return;
220
+ this.visit(file, node.fn, env, path, depth + 1, true);
221
+ this.visit(file, this.withoutOptionDocs(node), env, path, depth + 1, false); return;
221
222
  }
222
223
  case 'string': {
223
224
  if (!node.interpolated) { for (const [label, re, sev] of TOKEN_FORMATS) if (re.test(node.literal || '')) { /* reported through its binding when it has one; bare strings: */ if (!this.stringSeen(file, node)) this.emit('nix-secret-plaintext', { ...this.at(file, node.span), file, severity: sev, exposure: 'plaintext-source', description: `A ${label} is written as a literal (value redacted).`, source: { kind: 'literal', file, line: node.span.startLine }, destination: { kind: 'source', label: 'Nix source' }, evidence: { ...redact(node.literal), format: label }, originalLocation: node.span, chain: [hop(file, node.span, `literal ${label}`)].filter(Boolean), origins: [mkOrigin('secret', `literal ${label}`)] }); } }
@@ -229,10 +230,27 @@ class Walker {
229
230
  if (node.items) for (const it of node.items) this.visit(file, it, env, path, depth + 1, false);
230
231
  }
231
232
  }
233
+ /**
234
+ * The argument of `mkOption { ... }` without its documentation attributes. `example`, `description` and
235
+ * `defaultText` are rendered into the manual, never into the configuration, so a credential-shaped literal
236
+ * there is an illustration. `default` is NOT documentation: it is the value the option takes.
237
+ */
238
+ withoutOptionDocs(appNode) {
239
+ const arg = appNode.arg;
240
+ const { fn } = this.ev.flatten(appNode);
241
+ const name = this.ev.calleeOf(fn).name;
242
+ const a = unparen(arg);
243
+ if (name !== 'mkOption' || !a || a.type !== 'attrset') return arg;
244
+ const DOC = new Set(['example', 'description', 'defaultText', 'longDescription', 'relatedPackages']);
245
+ return { ...a, bindings: a.bindings.filter((b) => !(b.kind === 'attr' && DOC.has(attrKey(b.path)[0]))) };
246
+ }
232
247
  stringSeen(file, node) { const k = `${file}:${node.start}`; if (this.scriptKeys.has(k)) return true; this.scriptKeys.add(k); return false; }
233
248
 
234
249
  inspectBinding(file, full, value, bspan, env) {
235
- const v = unparen(value);
250
+ let v = unparen(value);
251
+ // `x = let ... in { ... }` / `with ...; { ... }` is an attribute SET, not a scalar: a credential name on the
252
+ // binding (a service called pass-secret-service) says nothing about a set's members, which are visited on their own
253
+ for (let guard = 0; v && (v.type === 'let' || v.type === 'with' || v.type === 'assert') && guard < 20; guard++) v = unparen(v.body);
236
254
  const last = full[full.length - 1];
237
255
  const dest = destOf(full);
238
256
  const sname = secretName(last);
@@ -35,7 +35,7 @@ never `engine.js`'s live taint state).
35
35
  - *Selection* (reading a field off an already-produced structured value, round 4) — `member`'s base isn't always a pure ident/member chain (`accessPathOf` returns `null` for a ternary, a logical expression, an object literal, an assign-expr, a call). Before round 4, that fell straight to `noIdentity()`, silently dropping the identity: `(user ?? other).email` returned `[]` even though `const c = user ?? other; return c.email;` correctly returned `[email]` — same semantics, two forms, two different answers, the exact tell every prior round used to find its bug. Fixed by giving `member` a second path: resolve the base recursively and select `prop` out of the base's `byPath` (plus its residual), mirroring how `object`'s construction attributes a property to its own key. Round 5 found a further, narrower selection gap: `accessPathOf` maps a computed member access with a statically-unknown key (`user[k]`) to a path ending in the literal segment `'*'` (e.g. `'user.*'`) — an existing convention in the shared access-path layer that `member`'s path-succeeds branch never interpreted specially, so it queried `identitiesAt(state, 'user.*')`, which matches nothing, and silently returned `[]` instead of a widened flow. Fixed (both the path-succeeds branch and the non-path fallback, for e.g. `(user ?? other)[k]`) by recognizing a `'*'`-ending path/prop and conservatively resolving the CONTAINER's full aggregate, flagged `widened: true`. **Round 6 found round 5's own guard here (`path === '*' || path.endsWith('.*')`) only recognized a TRAILING wildcard, not an INTERIOR one**: `store[k1].name`/`store[k2].name` both lower to the identical access path `'store.*.name'`, which is neither exactly `'*'` nor ending in `.*'`, so the read side silently returned `[]` for `store[k].name = user.ssn; return store.a.name;` instead of conservatively widening. Fixed with a position-independent `pathHasWildcard`/`definitePrefixBeforeWildcard(path)` helper pair (near the top of the file, after `unionOfByPath`/`residualFlat`) that finds the wildcard at ANY position — `definitePrefixBeforeWildcard('bag.*')` still gives round 5's own `'bag'` (no behavior change for the trailing case), `definitePrefixBeforeWildcard('store.*.name')` now correctly gives `'store'`. `member`'s non-path fallback branch (the paragraph above) needed no change: it never parses a dotted path string at all — it resolves the base recursively via the expression tree, so an interior wildcard there is just a nested `member` node whose own `expr.prop === '*'` check fires one recursion level in.
36
36
  - *Write-out* (writing a resolved value to a target path that must itself be a valid, non-fabricated path, round 4) — `step()`'s `assign` case passed `node.target` straight through to `removeIdentitiesAt`/`addIdentity` with no guard it was a string. Assignment-expression-form destructuring (`({a} = obj)`, as opposed to declaration-form `const {a} = obj`) is lowered by the real parser into one `assign` CFG node whose `target` is the raw pattern object, not a string — this implicitly stringified to the literal `"[object Object]"`, so every such destructuring anywhere in a function collided onto one fabricated key, silently merging unrelated statements' fields. Fixed by guarding `node.target` to be a string before writing anything (skip rather than fabricate a key), matching a precedent `scanner/src/dataflow/engine.js`'s own `assign` case already established that this package hadn't inherited. Round 5 found a second write-out bug in the same case, once the target IS a string: EVERY computed-key write on the SAME container lowers to the SAME literal target string (`'bag.*'`, since the real runtime key is statically unknown), so a strong update (`removeIdentitiesAt` before writing) on a second computed-key write to the same container silently deleted the FIRST write's identity — `bag[k1] = user.email; bag[k2] = user.ssn; return bag;` returned only `data:ssn`. `scanner/src/dataflow/engine.js`'s `_addPathAliasAware` had already solved exactly this for the sibling taint engine (a trailing `.*` write must be a WEAK update — add to the container, never clear it first); this package had not inherited that precedent either, the same pattern as the destructuring fix one paragraph up. Fixed by recognizing a `'*'`-ending target and performing a weak `addIdentity` onto the container path (never `removeIdentitiesAt` first), flagged as a `dynamic-property-key` widening event. **Round 6 generalized this same guard to be position-independent too** (via the same shared `pathHasWildcard`/`definitePrefixBeforeWildcard` helper pair described in the *Selection* bullet above): `store[k1].name = user.email; store[k2].name = user.ssn;` both lowered to the identical target `'store.*.name'` (an INTERIOR wildcard, not trailing), so the round-5 trailing-only check let the second write's strong update silently delete the first write's identity — round 5's own bug recurring one path segment deeper. **Deferred, not fixed this round (Finding 3, Minor, soundness-unaffected):** both this branch's `assign`-case widening push and `return`'s equivalent still label EVERY widening event with the hardcoded reason string `'unresolved-call'` when they fire via the general `resolved.widened` path (as opposed to this `'*'`-target branch's own correctly-labeled `dynamic-property-key` push) — so a widening actually caused by a dynamic property key on the READ side can be mislabeled `'unresolved-call'` in the ledger. The identity SETS themselves are unaffected either way; only the stated cause can be wrong. A real fix needs `resolveExprIdentities`'s return shape to thread a reason string through more broadly than this round's scope covers.
37
37
 
38
- See the ADR §3's "The residual principle" and "The structure-preserving vs. structure-flattening invariant (round 3)" notes, the ADR's "The three hop types the invariant covers (round 4)" note, the ADR's "Statically-unknown path components (round 5)" note, and the ADR's "Two under-enforcements of the round-5 invariant (round 6)" note, for the full worked examples. `analyzeFunctionFieldIdentity(fn, entryState, ctx?)`: a forward-worklist CFG analysis over a single function, structurally mirroring `dataflow/engine.js`'s `analyzeFunction` (same fixed-point/join algorithm, entirely different state type — never imports or touches that file's taint state) — returns `{exitState, returnFacts, mutatedParams, widenings}`. `returnFacts` accumulates per-node in a `Map` during the worklist and unions on revisit before materializing to an array, so a `return` node revisited before its incoming join settles gets exactly one, fully-unioned entry (a final whole-branch review found and fixed a real duplicate-stale-entry bug here). `mutatedParams` is computed via a direct `identitiesAt` call per param, relying on `identitiesAt`'s bidirectional aggregation to see both the param's own path and any descendant a mutation wrote to. **Interprocedural resolution (Sub-project B, increment 2):** the optional `ctx` param is threaded through `step()` into every recursive `resolveExprIdentities` call it makes (`assign`/`return`/bare-call CFG nodes); when `ctx.resolveCallSummary` is present, a `call` expression consults it instead of falling back to the flat/`widened:true` unresolved-call case — see `summaries.js`'s `createCallSummaryResolver` below for how that resolver is built and how it threads itself down through multi-hop call chains. No path DAG, no registry/graph-output wiring yet — see the ADR §5 for the explicit scope line. Unit-tested against hand-built fixtures in `test/lineage/engine-expr-resolver.test.js` and `test/lineage/engine-walker.test.js`, and against real parsed JS/TS source (via `scanner/src/ir/parser-js.js`) in `test/lineage/engine-integration.test.js`. **Object spread/rest fix:** a post-implementation audit (Tasks 1–2) found and closed a long-standing parser-level gap where object spread (`{...user}`) and destructuring rest (`const {...rest} = user`) were silently dropped — see `DESIGN_INTRAPROCEDURAL.md`'s object-spread bullet for the full fix description and the array-rest cross-reference (distinct, unrelated issue, still unfixed). |
38
+ See the ADR §3's "The residual principle" and "The structure-preserving vs. structure-flattening invariant (round 3)" notes, the ADR's "The three hop types the invariant covers (round 4)" note, the ADR's "Statically-unknown path components (round 5)" note, and the ADR's "Two under-enforcements of the round-5 invariant (round 6)" note, for the full worked examples. `analyzeFunctionFieldIdentity(fn, entryState, ctx?)`: a forward-worklist CFG analysis over a single function, structurally mirroring `dataflow/engine.js`'s `analyzeFunction` (same fixed-point/join algorithm, entirely different state type — never imports or touches that file's taint state) — returns `{exitState, returnFacts, mutatedParams, widenings}`. `returnFacts` accumulates per-node in a `Map` during the worklist and unions on revisit before materializing to an array, so a `return` node revisited before its incoming join settles gets exactly one, fully-unioned entry (a final whole-branch review found and fixed a real duplicate-stale-entry bug here). `mutatedParams` is computed via a direct `identitiesAt` call per param, relying on `identitiesAt`'s bidirectional aggregation to see both the param's own path and any descendant a mutation wrote to. **Interprocedural resolution (Sub-project B, increment 2):** the optional `ctx` param is threaded through `step()` into every recursive `resolveExprIdentities` call it makes (`assign`/`return`/bare-call CFG nodes); when `ctx.resolveCallSummary` is present, a `call` expression consults it instead of falling back to the flat/`widened:true` unresolved-call case — see `summaries.js`'s `createCallSummaryResolver` below for how that resolver is built and how it threads itself down through multi-hop call chains. No path provenance graph, no registry/graph-output wiring yet — see the ADR §5 for the explicit scope line. Unit-tested against hand-built fixtures in `test/lineage/engine-expr-resolver.test.js` and `test/lineage/engine-walker.test.js`, and against real parsed JS/TS source (via `scanner/src/ir/parser-js.js`) in `test/lineage/engine-integration.test.js`. **Object spread/rest fix:** a post-implementation audit (Tasks 1–2) found and closed a long-standing parser-level gap where object spread (`{...user}`) and destructuring rest (`const {...rest} = user`) were silently dropped — see `DESIGN_INTRAPROCEDURAL.md`'s object-spread bullet for the full fix description and the array-rest cross-reference (distinct, unrelated issue, still unfixed). |
39
39
 
40
40
  **Milestone 1, Sub-project B, increments 1-6 — COMPLETE (interprocedural summaries — cache core, real call-case wiring, real call-graph integration, project-wide driver, recursion refinement, context-sensitivity tuning):**
41
41
 
@@ -48,20 +48,20 @@ never `engine.js`'s live taint state).
48
48
 
49
49
  | Module | Responsibility |
50
50
  |---|---|
51
- | `DESIGN_PATH_PROVENANCE.md` | Design record (binding on every later Sub-project C increment, the same way `DESIGN_INTRAPROCEDURAL.md` binds Sub-project A/B): resolves HOW provenance/predecessor information is recorded as data flows through the engine, WITHOUT touching `field-identity.js`'s core `Map<path, Set<id>>` state shape. **Decision 1**: provenance is an emitted, write-only RECORD STREAM via an optional `ctx.recordHop` callback (mirroring `ctx.resolveCallSummary`'s exact precedent from Sub-project B, increment 2) — the engine never accumulates, stores, or reads back provenance of its own; with no recorder supplied, every observable output stays byte-identical to pre-C1 behavior (proven, not assumed — see below). **Decision 2**: the DAG's node granularity is `(scope, accessPath, dataElementId)` — expression-internal constructs (object literals, ternaries, resolved calls) annotate edges, they never create nodes, which is what keeps the structure compact per FR-303. **Records are HALF-EDGES** (`production`/`selection`/`write-out`, exactly `DESIGN_INTRAPROCEDURAL.md`'s already-hardened three-hop-type taxonomy), joined later by a consumer on `(scope, nodeId, dataElementId)` — the resolver knows a hop's source but never its destination, and `step()` knows the destination but never the source, so no single site can ever emit a complete edge. **Decision 4**: one record per `dataElementId`, never a Set-valued record — keeps DAG edges atomic and independently gradeable per FR-306, and avoids deferring an explosion to reconstruction time that recording can resolve once, up front. **Decision 5**: a hop's `fromPath`/`toPath` is never a syntactic/wildcard-containing path (`store.*.name`) — `step()`'s own wildcard-write branch already resolves to the definite prefix before this increment even runs (`definitePrefixBeforeWildcard`), and recording the wildcard form instead would create DAG nodes no read hop could ever reach, silently reading as "no path" rather than "path exists, not representable" (exactly the §18.4 failure mode this whole sub-project exists to prevent). **Decision 6, the design's most consequential correction to its own first draft**: `fromPath` must be the state key that actually CONTRIBUTED the identity, not the path that was QUERIED — `identitiesAt`'s bidirectional aggregation (an ancestor's identity is visible from a descendant query, and a container query aggregates every identity recorded under it) means the queried path is frequently NOT itself a key in `state`; recording it instead of the real contributing key(s) would silently disconnect the DAG at exactly the points FR-301's own multi-identity distinctness matters most. Verified empirically (16,000 randomized checks against a query's own independently-derived oracle in the final task review, 651 of which had a genuinely different contributing key than the queried path — not a vacuous edge case). A structural test (not just a comment) pins `contributingKeys` against `identitiesAt` directly, so the two can never silently drift apart. **Known, disclosed, load-bearing gap named for C3 (not C1/C2's job)**: the join key `(scope, nodeId, dataElementId)` does not distinguish entry CONTEXTS — Sub-project B (B1/B6) already computes up to 16 distinct entry contexts per function, and two contexts of the same function emit hops sharing this join key but describing different, mutually-inconsistent endpoints. Cannot manifest yet (C1/C2 only ever run one context at a time) but is a precondition C3 must close (an additive `context` field, e.g. `hashState(entryState)`) before its own hops are correct — see the design doc's §9.4 for the worked example. Also disclosed: §9.1's cross-join ambiguity (same id, ≥2 read paths AND ≥2 write paths at one node — detect-and-mark via FR-306, not prevented; confirmed NOT rare, reproduces with a plain multi-field-object alias), §9.2's flow-insensitivity at reconstruction time, §9.3's cyclic-DAG possibility (C5 must be cycle-safe by construction, never assume acyclicity), and §9.5's unrepresented `ITER_BUDGET` truncation (the one remaining un-flagged instance of the exact §18.4 constraint this whole document treats as load-bearing). §10 is a full, explicit instrumentation checklist for every `resolveExprIdentities` case and every `step()` CFG-node kind — increment C1's own proof-of-concept implemented four of them; increment C2 (below) implemented every remaining one, closing the checklist in full. One correction made DURING C2's own implementation, per this document's own stated policy of fixing rather than silently diverging: §10.1's `assign-expr` row originally specified forwarding a `widenReason` value, but `resolveExprIdentities`'s actual return shape carries only a boolean `widened`, not a reason string — resolved by matching the SAME already-documented-as-approximate convention `step()`'s pre-existing `assign`/`return` sites use (a hardcoded `'unresolved-call'` reason when widened), disclosed as inheriting that convention's known imprecision rather than claiming new precision. §10.3 (cross-file sites, the context-join fix) remains C3's job, untouched by C2. **§13 (added by increment C3's design task, then IMPLEMENTED by C3's two follow-up implementation tasks — see below) resolves all of it**: the record shape gains exactly three fields (`context` = `hashState(entryState)`, plus `peerScope`/`peerContext` naming the function on the far side of a cross-function hop, direction read off the existing `kind` rather than a fourth field); the join key becomes `(scope, nodeId, dataElementId, context)`; `resolveCallSummary` and `entryStateFromCall` each gain an optional 4th `ctx` parameter, closing all three §7.4 holes; the argument→parameter binding becomes a `write-out/call-arg-bind` hop stamped with the CALLER's scope/nodeId (§10.3's original "in the callee's scope" instruction is corrected in place — the caller's stamps are what let it join with the argument's own in-halves). §13 also records three MEASURED findings: a cache HIT records the callee's body once per entry context rather than once per call site (accepted and disclosed as §9.6 — a sharing property, not a loss, precisely because `peerContext` points every call site at that one body, so the much larger cache-replay fix is rejected); a B6 context-cap degradation is today entirely SILENT in the hop stream (no hop is emitted at all, since the degraded summary's `returnFlat` is empty — closed by a new permanent `degradedReason` field and a `lossReason: 'context-cap-degraded'` hop, and note the cap fallback is the SAME OBJECT as the empty-entry summary, so it must be shallow-copied, never marked in place); and a B5 recursion bottom stub needs no marking at all, being self-effacing for the same empty-`returnFlat` reason. §13.5 rejects §7.4's proposed `hopSite` field as redundant once the whole stamped ctx is threaded. §13.7 is the follow-up implementation task's exact file/line/signature checklist. **The single most important thing an implementer must not get wrong (§13.2a, found by this increment's own task review after §13's first draft got it wrong): `entryStateFromCall` must forward a RECORDER-ONLY derivation — `ctx?.recordHop ? { recordHop: ctx.recordHop } : undefined` — never the ctx itself.** Forwarding the full ctx hands `resolveExprIdentities` a live `resolveCallSummary`, so an argument that is itself a resolvable call (`sink(scrub(user))`) starts resolving interprocedurally where the shipped engine takes the unresolved fallback — changing the ANALYSIS RESULT with no recorder attached anywhere, and doing so in the UNSOUND direction under a tight B6 cap (the extra nested resolve eats the callee's only context slot, so a later, unrelated call degrades to an empty summary and loses an identity the shipped engine keeps). This is the same bug class C2 already shipped once and fixed in §10.2's `assign`/unsupported-target row. Note that the usual with-recorder/without-recorder guard is structurally BLIND to it (the divergence moves both arms identically) — **do NOT compare against "the shipped resolver"**: once this wiring is itself the shipped resolver, that comparison degenerates into `assert.deepEqual(result, result)`, a vacuous, always-passing test (this exact wording was in an earlier draft of this sentence and was corrected once already — see §13.2a's boxed note and §13.7 item 15b in the design doc for the full reasoning). The real guard is a hardcoded **pre-C3 golden literal**, shipped as `engine-provenance-interprocedural.test.js`'s two golden-baseline tests. Every claim in §13 was produced by running real code — via a deliberately throwaway-named design-task PoC file, engine-provenance-interprocedural-poc.test.js (not backtick-quoted here on purpose — it no longer exists in the tree, see the next sentence, and this document's own doc-drift checker flags a dangling backtick-quoted path), which prototyped the new signatures locally (shipped source was unmodified by the design task). Per §13.7 item 15, the two follow-up implementation tasks absorbed everything the PoC proved into the permanent suite (`test/lineage/engine-provenance-interprocedural.test.js`, `engine-provenance.test.js`, `driver.test.js`) and deleted the PoC file in the same commit that closed its absorption. |
51
+ | `DESIGN_PATH_PROVENANCE.md` | Design record (binding on every later Sub-project C increment, the same way `DESIGN_INTRAPROCEDURAL.md` binds Sub-project A/B): resolves HOW provenance/predecessor information is recorded as data flows through the engine, WITHOUT touching `field-identity.js`'s core `Map<path, Set<id>>` state shape. **Decision 1**: provenance is an emitted, write-only RECORD STREAM via an optional `ctx.recordHop` callback (mirroring `ctx.resolveCallSummary`'s exact precedent from Sub-project B, increment 2) — the engine never accumulates, stores, or reads back provenance of its own; with no recorder supplied, every observable output stays byte-identical to pre-C1 behavior (proven, not assumed — see below). **Decision 2**: the provenance graph's node granularity is `(scope, accessPath, dataElementId)` — expression-internal constructs (object literals, ternaries, resolved calls) annotate edges, they never create nodes, which is what keeps the structure compact per FR-303. **Records are HALF-EDGES** (`production`/`selection`/`write-out`, exactly `DESIGN_INTRAPROCEDURAL.md`'s already-hardened three-hop-type taxonomy), joined later by a consumer on `(scope, nodeId, dataElementId)` — the resolver knows a hop's source but never its destination, and `step()` knows the destination but never the source, so no single site can ever emit a complete edge. **Decision 4**: one record per `dataElementId`, never a Set-valued record — keeps provenance-graph edges atomic and independently gradeable per FR-306, and avoids deferring an explosion to reconstruction time that recording can resolve once, up front. **Decision 5**: a hop's `fromPath`/`toPath` is never a syntactic/wildcard-containing path (`store.*.name`) — `step()`'s own wildcard-write branch already resolves to the definite prefix before this increment even runs (`definitePrefixBeforeWildcard`), and recording the wildcard form instead would create provenance-graph nodes no read hop could ever reach, silently reading as "no path" rather than "path exists, not representable" (exactly the §18.4 failure mode this whole sub-project exists to prevent). **Decision 6, the design's most consequential correction to its own first draft**: `fromPath` must be the state key that actually CONTRIBUTED the identity, not the path that was QUERIED — `identitiesAt`'s bidirectional aggregation (an ancestor's identity is visible from a descendant query, and a container query aggregates every identity recorded under it) means the queried path is frequently NOT itself a key in `state`; recording it instead of the real contributing key(s) would silently disconnect the provenance graph at exactly the points FR-301's own multi-identity distinctness matters most. Verified empirically (16,000 randomized checks against a query's own independently-derived oracle in the final task review, 651 of which had a genuinely different contributing key than the queried path — not a vacuous edge case). A structural test (not just a comment) pins `contributingKeys` against `identitiesAt` directly, so the two can never silently drift apart. **Known, disclosed, load-bearing gap named for C3 (not C1/C2's job)**: the join key `(scope, nodeId, dataElementId)` does not distinguish entry CONTEXTS — Sub-project B (B1/B6) already computes up to 16 distinct entry contexts per function, and two contexts of the same function emit hops sharing this join key but describing different, mutually-inconsistent endpoints. Cannot manifest yet (C1/C2 only ever run one context at a time) but is a precondition C3 must close (an additive `context` field, e.g. `hashState(entryState)`) before its own hops are correct — see the design doc's §9.4 for the worked example. Also disclosed: §9.1's cross-join ambiguity (same id, ≥2 read paths AND ≥2 write paths at one node — detect-and-mark via FR-306, not prevented; confirmed NOT rare, reproduces with a plain multi-field-object alias), §9.2's flow-insensitivity at reconstruction time, §9.3's cyclic-graph possibility (C5 must be cycle-safe by construction, never assume acyclicity), and §9.5's unrepresented `ITER_BUDGET` truncation (the one remaining un-flagged instance of the exact §18.4 constraint this whole document treats as load-bearing). §10 is a full, explicit instrumentation checklist for every `resolveExprIdentities` case and every `step()` CFG-node kind — increment C1's own proof-of-concept implemented four of them; increment C2 (below) implemented every remaining one, closing the checklist in full. One correction made DURING C2's own implementation, per this document's own stated policy of fixing rather than silently diverging: §10.1's `assign-expr` row originally specified forwarding a `widenReason` value, but `resolveExprIdentities`'s actual return shape carries only a boolean `widened`, not a reason string — resolved by matching the SAME already-documented-as-approximate convention `step()`'s pre-existing `assign`/`return` sites use (a hardcoded `'unresolved-call'` reason when widened), disclosed as inheriting that convention's known imprecision rather than claiming new precision. §10.3 (cross-file sites, the context-join fix) remains C3's job, untouched by C2. **§13 (added by increment C3's design task, then IMPLEMENTED by C3's two follow-up implementation tasks — see below) resolves all of it**: the record shape gains exactly three fields (`context` = `hashState(entryState)`, plus `peerScope`/`peerContext` naming the function on the far side of a cross-function hop, direction read off the existing `kind` rather than a fourth field); the join key becomes `(scope, nodeId, dataElementId, context)`; `resolveCallSummary` and `entryStateFromCall` each gain an optional 4th `ctx` parameter, closing all three §7.4 holes; the argument→parameter binding becomes a `write-out/call-arg-bind` hop stamped with the CALLER's scope/nodeId (§10.3's original "in the callee's scope" instruction is corrected in place — the caller's stamps are what let it join with the argument's own in-halves). §13 also records three MEASURED findings: a cache HIT records the callee's body once per entry context rather than once per call site (accepted and disclosed as §9.6 — a sharing property, not a loss, precisely because `peerContext` points every call site at that one body, so the much larger cache-replay fix is rejected); a B6 context-cap degradation is today entirely SILENT in the hop stream (no hop is emitted at all, since the degraded summary's `returnFlat` is empty — closed by a new permanent `degradedReason` field and a `lossReason: 'context-cap-degraded'` hop, and note the cap fallback is the SAME OBJECT as the empty-entry summary, so it must be shallow-copied, never marked in place); and a B5 recursion bottom stub needs no marking at all, being self-effacing for the same empty-`returnFlat` reason. §13.5 rejects §7.4's proposed `hopSite` field as redundant once the whole stamped ctx is threaded. §13.7 is the follow-up implementation task's exact file/line/signature checklist. **The single most important thing an implementer must not get wrong (§13.2a, found by this increment's own task review after §13's first draft got it wrong): `entryStateFromCall` must forward a RECORDER-ONLY derivation — `ctx?.recordHop ? { recordHop: ctx.recordHop } : undefined` — never the ctx itself.** Forwarding the full ctx hands `resolveExprIdentities` a live `resolveCallSummary`, so an argument that is itself a resolvable call (`sink(scrub(user))`) starts resolving interprocedurally where the shipped engine takes the unresolved fallback — changing the ANALYSIS RESULT with no recorder attached anywhere, and doing so in the UNSOUND direction under a tight B6 cap (the extra nested resolve eats the callee's only context slot, so a later, unrelated call degrades to an empty summary and loses an identity the shipped engine keeps). This is the same bug class C2 already shipped once and fixed in §10.2's `assign`/unsupported-target row. Note that the usual with-recorder/without-recorder guard is structurally BLIND to it (the divergence moves both arms identically) — **do NOT compare against "the shipped resolver"**: once this wiring is itself the shipped resolver, that comparison degenerates into `assert.deepEqual(result, result)`, a vacuous, always-passing test (this exact wording was in an earlier draft of this sentence and was corrected once already — see §13.2a's boxed note and §13.7 item 15b in the design doc for the full reasoning). The real guard is a hardcoded **pre-C3 golden literal**, shipped as `engine-provenance-interprocedural.test.js`'s two golden-baseline tests. Every claim in §13 was produced by running real code — via a deliberately throwaway-named design-task PoC file, engine-provenance-interprocedural-poc.test.js (not backtick-quoted here on purpose — it no longer exists in the tree, see the next sentence, and this document's own doc-drift checker flags a dangling backtick-quoted path), which prototyped the new signatures locally (shipped source was unmodified by the design task). Per §13.7 item 15, the two follow-up implementation tasks absorbed everything the PoC proved into the permanent suite (`test/lineage/engine-provenance-interprocedural.test.js`, `engine-provenance.test.js`, `driver.test.js`) and deleted the PoC file in the same commit that closed its absorption. |
52
52
  | `engine.js` (extended, additively) | **Increment C1** built the mechanism and proved it against four representative sites (`resolveExprIdentities`'s `ident`/`object` production cases, `step()`'s non-wildcard `assign`/`return` write-out cases). **Increment C2 completed FULL intraprocedural coverage** — every remaining case in both switch statements now records hops: `resolveExprIdentities`'s all four `member` sub-cases (path/no-wildcard, path/wildcard via `definitePrefixBeforeWildcard` per Decision 5, non-path-base × known-prop, non-path-base × `prop==='*'`), `array`/`tpl`/`binary` (structure-flattening, no `widenReason`), `logical`/`union` (structure-preserving), both `call` branches (unresolved → `widenReason: 'unresolved-call'`; resolved via `ctx.resolveCallSummary` → `subKind: 'call-resolved'`), `assign-expr`, and explicit "emits nothing" verdicts for `literal`/`unknown`/`default`; `step()`'s `assign` target-not-a-string (a genuine **loss** site, `lossReason: 'unsupported-target'` — resolving `node.source` purely to learn its ids genuinely emits real in-half hops when a recorder is present, not suppressed computation), `assign` wildcard-target (`subKind: 'assign-weak'`, `toPath` = the definite prefix, never the raw `'*'`-path, same Decision 5 discipline), and bare `call` statements (`subKind: 'call-arg'`). `assign`'s kill (`removeIdentitiesAt`) deliberately gets NO instrumentation — §10.2's own table gives this an explicit "no row of its own, correctly" verdict, not an oversight. A single comprehensive real-parser test (`test/lineage/engine-provenance.test.js`) exercises all 15 hop-emitting `(kind, subKind)` shapes from §10.1+§10.2 together in one realistic function, asserting closed-set coverage (every expected shape present, no stray ones) — proven, not merely claimed. `ctx.recordHop`, when present, is progressively stamped with `scope`/`nodeId`/`line` inside `analyzeFunctionFieldIdentity`'s own worklist (`nodeId` sourced from the worklist's Map key, NOT `node.id` — hand-built fixtures across every prior increment's own tests set no `id` field, a documented likely mistake this increment's own tests pin against) before being threaded down through the SAME `ctx` parameter every recursive call site already carries unconditionally since Sub-project B, increment 2 — zero new plumbing beyond what already existed. Two internal helpers implement Decision 6: `contributingKeysAllIds` (not exported) does the real work as a single O(|state|)-total pass over `state`; `contributingKeys` (exported) is a thin per-id wrapper over it, so a caller asking for ONE id's contributing keys still pays that O(|state|) scan — union it across every id at a path and the total is the O(|state| × |ids|) shape the single-pass helper exists specifically to avoid internally. `contributingKeys` is exported ONLY so a structural test can pin it against `field-identity.js`'s own `identitiesAt` rather than relying on a comment to catch drift — it is not the performance-sensitive entry point production code should call per-id in a loop; that's what `contributingKeysAllIds` is for. **Backward compatibility, proven not assumed**: a differential harness (in the final task review, not shipped as a permanent fixture) ran 15 real-parser fixtures × 4 entry states × 3 `ctx` configurations (no ctx; `ctx.resolveCallSummary` only; `ctx` with a recorder attached) against the pre-C1 engine — zero behavioral mismatches across all 120 cases. The SHIPPED test suite's own "write-only invariant" test (added in a C1 fix round after a review caught the gap with a real injected mutant that passed all 217 pre-fix-round lineage tests; extended again by C2 with fixtures for every case it added, including the full comprehensive-coverage fixture) runs its whole fixture list twice each — with and without a recorder — and `deepEqual`s a canonicalized `{exitState, returnFacts, mutatedParams, widenings}` between the two runs, closing exactly the property the differential harness proved but the original shipped tests didn't actually check. `field-identity.js` is untouched, byte-for-byte, confirmed via `git diff` at every review stage — this increment adds capability entirely alongside the existing state shape, never inside it. See `docs/superpowers/plans/2026-08-30-data-flow-explorer-m1-subproject-c1.md` and `docs/superpowers/plans/2026-08-30-data-flow-explorer-m1-subproject-c-scoping.md` for the full plan and increment breakdown. **Increment C3 (Task 2) extended this file in five places, all additive**: the `field-identity.js` import gained `hashState`; `analyzeFunctionFieldIdentity` computes `context = ctx?.recordHop ? hashState(entryState) : null` once per analysis run, alongside the pre-existing `scope`; the `stepCtx` wrapper stamps `context`/`peerScope: null`/`peerContext: null` onto every hop before the per-hop `...h` spread (so a hop that needs non-null peer fields can still override the default — only the two cross-function hop sites below ever do); `resolveExprIdentities`'s `case 'call'` now passes `ctx` as `resolveCallSummary`'s 4th argument (the **only** place the full `ctx` crosses from `engine.js` into `summaries.js` — `summaries.js`'s own `entryStateFromCall` deliberately strips it back down to a recorder-only derivation before going any further, see that file's own row below); and the resolved-call branch's `call-resolved` hop gained `peerScope: summary.resolvedQid ?? null, peerContext: summary.resolvedContext ?? null`, one of exactly two hop sites in the whole package that ever set these two fields (the other is `summaries.js`'s `call-arg-bind` hop). |
53
53
 
54
- **Milestone 1, Sub-project C, increment 4 (`path-store.js` — the compact provenance DAG) — COMPLETE:**
54
+ **Milestone 1, Sub-project C, increment 4 (`path-store.js` — the compact provenance graph) — COMPLETE:**
55
55
 
56
56
  | Module | Responsibility |
57
57
  |---|---|
58
- | `path-store.js` | Implements `DESIGN_PATH_PROVENANCE.md` §14 exactly (design + PoC landed as Task 1; this is Task 2, the mechanical implementation, absorbing the design task's PoC into the permanent suite). A **pure consumer** of the 14-field hop-record stream C1-C3 ship — never imports `engine.js`/`summaries.js`/`driver.js` (§14.1), enforced by a self-checking test that reads the module's own import list, not just documented. `PathStore.addHop(hop)`/`addHops(hops)` accumulate hops into join-key groups (`(scope, nodeId, dataElementId, context)`, §13.3); nodes/edges materialize lazily on first read and are cached until the next `addHop` (§14.6's two-phase construction). Five node **kinds**, each with its own keying rule (§14.2): `path` (`(scope, context, path, id)`, the ordinary state location), `return` (`(scope, context, id)` — per function-CONTEXT, not per CFG node, since a `call-resolved` hop addresses it with only `(peerScope, peerContext)`), `escape`/`loss` (`(scope, context, siteNodeId, id)`, per CFG node, terminal), `origin` (`(scope, context, siteNodeId, id)`, a group whose only in-halves are annotations — "a value with no prior aliasing source", hand-fixture-only today, no real-parser fixture reaches it yet). `classifyIn`/`classifyOut` (both exported for direct unit-testing) implement §14.3's rules verbatim, including §14.4's corrected join rule — the single most load-bearing rule in the module: a `null`-`fromPath` in-half with a non-null `peerScope` is **peer-sourced** (a real cross-scope edge to the callee's own function-exit node) UNLESS `lossReason` is also non-null (a §13.6 context-cap-degraded hop, which names a callee whose body was never analyzed and must fall through to *annotation* rather than fabricate an origin — this is what `C4/Q2b` pins against a future refactor silently undoing). Edge construction is a per-group cross product with the peer×peer exclusion (§14.3 — a pair whose both endpoints are peer-addressed describes a transition entirely inside the callee and would manufacture a callee-return→callee-parameter cycle no program executed) and the per-pairing (not per-group) ambiguity measure (§14.7 — the group-level form measured 3-of-5 marked at a plain resolved call, including both correct call-boundary edges; per-pairing correctly narrows to the real artifact, the caller-side bypass that skips the callee). **Both** §14.6 dedup boundaries are shipped, not just one: raw-hop ingest dedup via a `Set` keyed on the explicit, FIXED-order `HOP_FIELDS` list (never `Object.keys(h)`, which is order-dependent) — collapses worklist re-emission (§8) and flags a hop missing a required field into `diagnostics().malformed` rather than silently mis-keying it; node/edge dedup via the content-hash ids from `ids.js` at materialization — collapses two non-byte-identical hops (e.g. differing only in `syntacticPath`) that describe the same logical edge, which ingest dedup alone cannot catch. `diagnostics()` returns **four** buckets, all "recorded, never thrown, never silently dropped": `malformed`, `unclassified` (an out-half matching none of §14.3's rules — empty on every fixture tried, the closed-set proof that the rules cover today's whole vocabulary), `truncations` (§9.5's reserved out-of-band channel — `markTruncated(scope, context, reason)` marks every node/edge in that `(scope, context)` `truncated: true`, since an `ITER_BUDGET` break is a whole-analysis-run truncation, not a per-hop one, and §2.2's three-kind hop taxonomy is deliberately not extended to a fourth kind for it), and `orphanedPeerSources` (added during Task 1's own review: a peer-sourced hop, `lossReason: null`, whose named `(peerScope, peerContext, ⟨return⟩, dataElementId)` node has zero real in-edges once the whole stream is ingested — detectable via the store's own `inIndex` at build-finalize time, no new input needed; reachable via a cache warmed by a no-recorder run and reused by a later recorder-attached run, exactly `driver.js`'s own returned-and-reused cache pattern — never fabricated into an origin, never dropped, proven live by a dedicated test reproducing that exact cache-reuse scenario against the real resolver). Construction is cycle-safe **by construction**, never by discipline (§9.3/§14.6): one linear pass over the offered hops plus a per-group cross product, no recursion anywhere in the file; the read API (`nodes`/`edges`/`getNode`/`getEdge`/`edgesFrom`/`edgesTo`/`hasEdge`/`nodeIdFor`/`stats`/`diagnostics`) is deliberately traversal-free, every method an O(1)/O(degree) index lookup — bounded backward reconstruction is C5's job and C5's alone. Task 2's own review also closed two carried-forward findings from Task 1's review: a real fixture (`C4/leg`) for §14.7's rejected leg-based-pruning counter-example (`const o = { r: helper(a), s: b.email }` — a group containing a peer half also contains a genuine, purely-intraprocedural non-peer×non-peer pair the rejected rule would have deleted alongside the bypass it targeted; verified against the real engine that BOTH survive, and both are in fact marked `ambiguousCorrelation: true` — a genuine §9.1-style 2×2 cross-join independent of the call boundary, matching §14.7's own "keeping a marked extra one" phrasing more precisely than a narrower reading might suggest), and explicit count assertions pinning §14.11's headline compactness numbers (§6's worked example: 14 deduplicated records → 8 nodes / 6 edges, the FR-303 proof; the mutual-recursion fixture: 34 raw → 19 dedup → 8 nodes / 11 edges) so a future refactor that silently changes them fails a test rather than leaving this file/the design doc stale. Every assertion from the design task's PoC — path-store-poc.test.js, not backtick-quoted here on purpose (it no longer exists in the tree, and this document's own doc-drift checker flags a dangling backtick-quoted path) — was kept and re-pointed at this shipped module in `test/lineage/path-store.test.js`, including `C4/Q2b` (the literal-§2.2 store with a dead-end callee exit — the only guard against §14.4's correction being silently undone). `ids.js` gained the two stable-ID functions this module needs: `provenanceNodeId`/`provenanceEdgeId`, `pnode:`/`pedge:`-prefixed (deliberately distinct from `node:`/`edge:` — a provenance node/edge is never a `DataFlowGraph v1` entity, confirmed by `json-schema-parity.test.js`/`validate.js` needing zero changes), object-argument signatures (not this file's usual positional form — `provenanceEdgeId`'s discriminator is twelve fields wide, `provenanceNodeId`'s is six, exactly the shape a positional array silently drops a field from; a final whole-branch review corrected an earlier claim here that `path-store.js` calls each "from exactly one site" — `provenanceEdgeId` does, but `provenanceNodeId` has five call sites, which if anything strengthens the case for object arguments rather than weakening it). No `DataFlowGraph v1` output, no backward-walk/reconstruction query, no FR-306 grading, no driver wiring, no library/framework node collapsing — all deliberately out of scope, per §14.9. **The final whole-branch review** independently re-derived the §2.2 correction on a fixture of its own (not the shipped test's), recomputed all seven of §14.11's measured-numbers rows against the shipped module (zero drift), and mutation-proved 5 of 6 load-bearing properties directly — the one that survived (`HOP_FIELDS`'s fixed-order key vs. a naive `Object.keys(h)` mutant) is now closed by `C4/1e`. It also found and fixed a genuine, currently-unreachable order-dependency in the `origin` branch: picking `g.annotations[0]` as the sole representative for an originating group's `kind`/`subKind`/reasons made the resulting edge's id AND its `widenReasons`/`lossReasons` depend on hop delivery order, contradicting §14.6's own "the same hops delivered in any order produce the same DAG" guarantee — fixed to union reasons across every annotation and pick the representative by content, not position (`C4/6g`, mutation-verified). |
58
+ | `path-store.js` | Implements `DESIGN_PATH_PROVENANCE.md` §14 exactly (design + PoC landed as Task 1; this is Task 2, the mechanical implementation, absorbing the design task's PoC into the permanent suite). A **pure consumer** of the 14-field hop-record stream C1-C3 ship — never imports `engine.js`/`summaries.js`/`driver.js` (§14.1), enforced by a self-checking test that reads the module's own import list, not just documented. `PathStore.addHop(hop)`/`addHops(hops)` accumulate hops into join-key groups (`(scope, nodeId, dataElementId, context)`, §13.3); nodes/edges materialize lazily on first read and are cached until the next `addHop` (§14.6's two-phase construction). Five node **kinds**, each with its own keying rule (§14.2): `path` (`(scope, context, path, id)`, the ordinary state location), `return` (`(scope, context, id)` — per function-CONTEXT, not per CFG node, since a `call-resolved` hop addresses it with only `(peerScope, peerContext)`), `escape`/`loss` (`(scope, context, siteNodeId, id)`, per CFG node, terminal), `origin` (`(scope, context, siteNodeId, id)`, a group whose only in-halves are annotations — "a value with no prior aliasing source", hand-fixture-only today, no real-parser fixture reaches it yet). `classifyIn`/`classifyOut` (both exported for direct unit-testing) implement §14.3's rules verbatim, including §14.4's corrected join rule — the single most load-bearing rule in the module: a `null`-`fromPath` in-half with a non-null `peerScope` is **peer-sourced** (a real cross-scope edge to the callee's own function-exit node) UNLESS `lossReason` is also non-null (a §13.6 context-cap-degraded hop, which names a callee whose body was never analyzed and must fall through to *annotation* rather than fabricate an origin — this is what `C4/Q2b` pins against a future refactor silently undoing). Edge construction is a per-group cross product with the peer×peer exclusion (§14.3 — a pair whose both endpoints are peer-addressed describes a transition entirely inside the callee and would manufacture a callee-return→callee-parameter cycle no program executed) and the per-pairing (not per-group) ambiguity measure (§14.7 — the group-level form measured 3-of-5 marked at a plain resolved call, including both correct call-boundary edges; per-pairing correctly narrows to the real artifact, the caller-side bypass that skips the callee). **Both** §14.6 dedup boundaries are shipped, not just one: raw-hop ingest dedup via a `Set` keyed on the explicit, FIXED-order `HOP_FIELDS` list (never `Object.keys(h)`, which is order-dependent) — collapses worklist re-emission (§8) and flags a hop missing a required field into `diagnostics().malformed` rather than silently mis-keying it; node/edge dedup via the content-hash ids from `ids.js` at materialization — collapses two non-byte-identical hops (e.g. differing only in `syntacticPath`) that describe the same logical edge, which ingest dedup alone cannot catch. `diagnostics()` returns **four** buckets, all "recorded, never thrown, never silently dropped": `malformed`, `unclassified` (an out-half matching none of §14.3's rules — empty on every fixture tried, the closed-set proof that the rules cover today's whole vocabulary), `truncations` (§9.5's reserved out-of-band channel — `markTruncated(scope, context, reason)` marks every node/edge in that `(scope, context)` `truncated: true`, since an `ITER_BUDGET` break is a whole-analysis-run truncation, not a per-hop one, and §2.2's three-kind hop taxonomy is deliberately not extended to a fourth kind for it), and `orphanedPeerSources` (added during Task 1's own review: a peer-sourced hop, `lossReason: null`, whose named `(peerScope, peerContext, ⟨return⟩, dataElementId)` node has zero real in-edges once the whole stream is ingested — detectable via the store's own `inIndex` at build-finalize time, no new input needed; reachable via a cache warmed by a no-recorder run and reused by a later recorder-attached run, exactly `driver.js`'s own returned-and-reused cache pattern — never fabricated into an origin, never dropped, proven live by a dedicated test reproducing that exact cache-reuse scenario against the real resolver). Construction is cycle-safe **by construction**, never by discipline (§9.3/§14.6): one linear pass over the offered hops plus a per-group cross product, no recursion anywhere in the file; the read API (`nodes`/`edges`/`getNode`/`getEdge`/`edgesFrom`/`edgesTo`/`hasEdge`/`nodeIdFor`/`stats`/`diagnostics`) is deliberately traversal-free, every method an O(1)/O(degree) index lookup — bounded backward reconstruction is C5's job and C5's alone. Task 2's own review also closed two carried-forward findings from Task 1's review: a real fixture (`C4/leg`) for §14.7's rejected leg-based-pruning counter-example (`const o = { r: helper(a), s: b.email }` — a group containing a peer half also contains a genuine, purely-intraprocedural non-peer×non-peer pair the rejected rule would have deleted alongside the bypass it targeted; verified against the real engine that BOTH survive, and both are in fact marked `ambiguousCorrelation: true` — a genuine §9.1-style 2×2 cross-join independent of the call boundary, matching §14.7's own "keeping a marked extra one" phrasing more precisely than a narrower reading might suggest), and explicit count assertions pinning §14.11's headline compactness numbers (§6's worked example: 14 deduplicated records → 8 nodes / 6 edges, the FR-303 proof; the mutual-recursion fixture: 34 raw → 19 dedup → 8 nodes / 11 edges) so a future refactor that silently changes them fails a test rather than leaving this file/the design doc stale. Every assertion from the design task's PoC — path-store-poc.test.js, not backtick-quoted here on purpose (it no longer exists in the tree, and this document's own doc-drift checker flags a dangling backtick-quoted path) — was kept and re-pointed at this shipped module in `test/lineage/path-store.test.js`, including `C4/Q2b` (the literal-§2.2 store with a dead-end callee exit — the only guard against §14.4's correction being silently undone). `ids.js` gained the two stable-ID functions this module needs: `provenanceNodeId`/`provenanceEdgeId`, `pnode:`/`pedge:`-prefixed (deliberately distinct from `node:`/`edge:` — a provenance node/edge is never a `DataFlowGraph v1` entity, confirmed by `json-schema-parity.test.js`/`validate.js` needing zero changes), object-argument signatures (not this file's usual positional form — `provenanceEdgeId`'s discriminator is twelve fields wide, `provenanceNodeId`'s is six, exactly the shape a positional array silently drops a field from; a final whole-branch review corrected an earlier claim here that `path-store.js` calls each "from exactly one site" — `provenanceEdgeId` does, but `provenanceNodeId` has five call sites, which if anything strengthens the case for object arguments rather than weakening it). No `DataFlowGraph v1` output, no backward-walk/reconstruction query, no FR-306 grading, no driver wiring, no library/framework node collapsing — all deliberately out of scope, per §14.9. **The final whole-branch review** independently re-derived the §2.2 correction on a fixture of its own (not the shipped test's), recomputed all seven of §14.11's measured-numbers rows against the shipped module (zero drift), and mutation-proved 5 of 6 load-bearing properties directly — the one that survived (`HOP_FIELDS`'s fixed-order key vs. a naive `Object.keys(h)` mutant) is now closed by `C4/1e`. It also found and fixed a genuine, currently-unreachable order-dependency in the `origin` branch: picking `g.annotations[0]` as the sole representative for an originating group's `kind`/`subKind`/reasons made the resulting edge's id AND its `widenReasons`/`lossReasons` depend on hop delivery order, contradicting §14.6's own "the same hops delivered in any order produce the same graph" guarantee — fixed to union reasons across every annotation and pick the representative by content, not position (`C4/6g`, mutation-verified). |
59
59
 
60
60
  **Milestone 1, Sub-project C, increment 5 (`path-query.js` — bounded backward reconstruction + truncation semantics) — COMPLETE:**
61
61
 
62
62
  | Module | Responsibility |
63
63
  |---|---|
64
- | `path-query.js` | Implements `DESIGN_PATH_PROVENANCE.md` §15 exactly (design + PoC landed as Task 1, including a fix round closing one BLOCKING finding plus 8 non-blocking ones, a scoped re-review, and a final round closing 3 more nitpicks; this is Task 2, the mechanical implementation, absorbing the design task's PoC into the permanent suite). A **pure consumer** of a built `PathStore` — never sees a hop record, never imports `engine.js`/`summaries.js`/`driver.js`, and its only import is `ids.js` (§15.1), enforced by a self-checking boundary test that also asserts the import list is EXACTLY `['./ids.js']`, one step stricter than `path-store.js`'s own equivalent test. One entry point plus two helpers, deliberately named `startNodeId` rather than `sinkNodeId` (there is no sink registry yet — Sub-project D): `reconstructPaths(store, startNodeId, opts)`, `sinkCandidates(store)` (§15.9's registry stand-in — every `return`/`escape`/`loss`-kind node, no security opinion), and `isIncompleteAnswer(result)` (AC-10's five-term banner predicate, so no caller re-derives it). `comparePaths(a, b)` is also exported, not merely internal (fix round 1's own correction to the original design's export list) — the shipped tests call it directly to build the naive-global-cap contrast fixture (`C5/5f`). The walk (`reconstructPaths`) is an **iterative DFS over an explicit stack, never recursion** — the DAG can genuinely be cyclic (§9.3, `path-store.test.js`'s `C4/4` mutual-recursion fixture) — terminated by an explicit **per-path** visited set (never global: the same node can legitimately appear on two different candidate paths), not by any budget; `edgesTo` results are sorted by edge id before expansion for determinism, since the store's own indexes are `Set`-backed and carry no inherent order. **The zero-in-edges check runs BEFORE the depth check** (§15.3's exact ordering constraint) — get this backwards and a genuine, well-formed dead end gets mislabeled as depth-truncated instead of a real origin. Three independent in-walk budgets (`maxExpansions`, `maxDepth`, `maxCandidatePaths`) plus the two post-hoc `maxPaths`/`maxPathsPerTerminal` caps below — five knobs total, all in `DEFAULTS` (exported), all `opts`-overridable, documented as uncalibrated — every fixture measured against them so far is tiny (§15.11's largest row uses 35 expansions); nothing bigger is measurable until a driver run emits real hops (Sub-projects D/E). Terminal classification (`terminal.reason`, per path): `'origin'` (a genuine `origin`-kind node), `'incomplete-record'` (the node appears in `store.diagnostics().orphanedPeerSources` — §14.4's disclosed stream-completeness gap surfacing here, never silently presented as a real origin), `'cycle'` (every continuation from this node was already visited on this path), `'depth-limit'` (the per-path budget hit `maxDepth`). `'expansion-budget'`/`'candidate-cap'` are **result-level truncation reasons only** — those two branches are abandoned when hit, never emitted as a marked partial path, since a branch cut at an arbitrary global point has a meaningless prefix and emitting it would manufacture a "path" the graph does not contain; `'depth-limit'` is different, because that branch genuinely reached a stated ceiling, so it IS emitted, marked `complete: false`. The cap: `maxPathsPerTerminal` applied per `terminal.nodeId` FIRST (ordered by `comparePaths`, truncated there), then a **diversity-first round-robin** over `(terminal.nodeId, shape)` buckets for the global `maxPaths` cap (§15.7's exact prioritization rule, using only signals that exist today: `crossScopeCount`, presence of `ambiguousCorrelation`/`widenReasons`/`lossReasons`, and `complete` as `comparePaths`' first key — transformation-kind and protection-state are honestly deferred to Sub-project D / Milestone 2, never faked). **`result.terminals[].truncated`/`.droppedPathCount` are computed AFTER the global round-robin runs, never from the per-terminal cap alone — this is the single blocking finding from Task 1's own review, and the single most load-bearing correctness property in this module.** The formula is `enumeratedPathCount > returnedPathCount` (equivalently `droppedPathCount > 0`); computing it from the per-terminal cap alone would report `truncated: false` for a terminal the GLOBAL cap later starves to zero returned paths — reproduced concretely pre-fix on the §9.1 cross-join fixture (`enumerated=2 kept=2 returned=0 truncated=false`, §18.4's exact failure mode at *pair* granularity, inside the very field designed to prevent it at *call* granularity) and pinned verbatim, both directions, by `C5/5e`. A terminal's `terminalReasons` is likewise a sorted UNION over its own paths' reasons, never a positional pick (`group[0].terminal.reason`) — the same representative-picking bug class C4's own final whole-branch review found and fixed in `path-store.js`'s `origin` branch; a terminal genuinely CAN carry mixed reasons (e.g. one sink reached by both a `cycle` clip and a `depth-limit` stop at `maxDepth: 3` on the mutual-recursion fixture), pinned by `C5/5h`. `result.noPathReason` is computed **only when `truncated === false`** — this ordering IS §18.4's "never translate budget exhausted into no path" constraint expressed as a code path, not a stylistic choice: a truncated result can never acquire a `noPathReason`. §15.4's five pairwise-JSON-distinguishable answers (complete-with-paths / genuinely-empty / recording-gap / unknown-start-node / budget-truncated) are pinned as literal JSON by `C5/4b`, and the recording-gap row against a real cache-warming fixture by `C5/4c` — together the only guard against §18.4's constraint being silently undone by a future refactor. A path never changes data element (every edge joins two nodes sharing one `dataElementId` by construction, §14.3) so no filtering is needed for FR-301 field-precision to carry into FR-303 structure; a hop is a **denormalized copy** of its edge (not just the edge id) so a path can be graded (FR-306) without also carrying the store back to the caller. `ids.js` gained `pathId({ startNodeId, edgeIds }, discriminatorParts)` → `ppath:<12 hex>` (§15.6/§15.10 item 1) — the discriminator is the **edge id SEQUENCE**, never the node id sequence or a sorted set: order matters for a path (two assignments at two different program points joining the same node pair are two materially different edges, per `provenanceEdgeId`'s own `siteNodeId` discriminator, and FR-305 forbids hiding that difference by node-keyed dedup — proven on a real fixture by `C5/5`, ids pinned idempotent/order-sensitive by `C5/id-1` through `C5/id-4`). `validate.js` needed no change — `ppath:` is not a `DataFlowGraph v1` entity kind, confirmed by `json-schema-parity.test.js` staying green untouched. Every assertion from the design task's PoC — path-query-poc.test.js, not backtick-quoted here on purpose (it no longer exists in the tree, and this document's own doc-drift checker flags a dangling backtick-quoted path) — was kept and re-pointed at this shipped module in `test/lineage/path-query.test.js`, including `C5/4b`/`C5/4c` (the truncation-honesty guard named above) and `C5/5e`/`C5/5f`/`C5/5d` (the fix round's own load-bearing regression guards for the blocking finding and the two "prose stronger than proof" measurement fixes). The old PoC's hand-written function list for the private-field source scan (`C5/6`) was replaced with programmatic export enumeration (`Object.values(await import(...)).filter(v => typeof v === 'function')`) per the PoC's own fix-round-1 note, so an eighth exported function is swept in automatically rather than requiring a remembered list. No `DataFlowGraph v1` output, no FR-306 grade computation (counts only — `widenedHopCount`/`lossHopCount`/`ambiguousHopCount`/`crossScopeCount` — turning them into a grade is **C6**), no source/sink registry, no flow-sensitivity filter, no change to `path-store.js`/`engine.js`/`summaries.js`/`driver.js`, no driver wiring (a driver run still emits zero hops today, no source registry — Sub-projects D/E) — all deliberately out of scope, per §15.8. |
64
+ | `path-query.js` | Implements `DESIGN_PATH_PROVENANCE.md` §15 exactly (design + PoC landed as Task 1, including a fix round closing one BLOCKING finding plus 8 non-blocking ones, a scoped re-review, and a final round closing 3 more nitpicks; this is Task 2, the mechanical implementation, absorbing the design task's PoC into the permanent suite). A **pure consumer** of a built `PathStore` — never sees a hop record, never imports `engine.js`/`summaries.js`/`driver.js`, and its only import is `ids.js` (§15.1), enforced by a self-checking boundary test that also asserts the import list is EXACTLY `['./ids.js']`, one step stricter than `path-store.js`'s own equivalent test. One entry point plus two helpers, deliberately named `startNodeId` rather than `sinkNodeId` (there is no sink registry yet — Sub-project D): `reconstructPaths(store, startNodeId, opts)`, `sinkCandidates(store)` (§15.9's registry stand-in — every `return`/`escape`/`loss`-kind node, no security opinion), and `isIncompleteAnswer(result)` (AC-10's five-term banner predicate, so no caller re-derives it). `comparePaths(a, b)` is also exported, not merely internal (fix round 1's own correction to the original design's export list) — the shipped tests call it directly to build the naive-global-cap contrast fixture (`C5/5f`). The walk (`reconstructPaths`) is an **iterative DFS over an explicit stack, never recursion** — the provenance graph can genuinely be cyclic (§9.3, `path-store.test.js`'s `C4/4` mutual-recursion fixture) — terminated by an explicit **per-path** visited set (never global: the same node can legitimately appear on two different candidate paths), not by any budget; `edgesTo` results are sorted by edge id before expansion for determinism, since the store's own indexes are `Set`-backed and carry no inherent order. **The zero-in-edges check runs BEFORE the depth check** (§15.3's exact ordering constraint) — get this backwards and a genuine, well-formed dead end gets mislabeled as depth-truncated instead of a real origin. Three independent in-walk budgets (`maxExpansions`, `maxDepth`, `maxCandidatePaths`) plus the two post-hoc `maxPaths`/`maxPathsPerTerminal` caps below — five knobs total, all in `DEFAULTS` (exported), all `opts`-overridable, documented as uncalibrated — every fixture measured against them so far is tiny (§15.11's largest row uses 35 expansions); nothing bigger is measurable until a driver run emits real hops (Sub-projects D/E). Terminal classification (`terminal.reason`, per path): `'origin'` (a genuine `origin`-kind node), `'incomplete-record'` (the node appears in `store.diagnostics().orphanedPeerSources` — §14.4's disclosed stream-completeness gap surfacing here, never silently presented as a real origin), `'cycle'` (every continuation from this node was already visited on this path), `'depth-limit'` (the per-path budget hit `maxDepth`). `'expansion-budget'`/`'candidate-cap'` are **result-level truncation reasons only** — those two branches are abandoned when hit, never emitted as a marked partial path, since a branch cut at an arbitrary global point has a meaningless prefix and emitting it would manufacture a "path" the graph does not contain; `'depth-limit'` is different, because that branch genuinely reached a stated ceiling, so it IS emitted, marked `complete: false`. The cap: `maxPathsPerTerminal` applied per `terminal.nodeId` FIRST (ordered by `comparePaths`, truncated there), then a **diversity-first round-robin** over `(terminal.nodeId, shape)` buckets for the global `maxPaths` cap (§15.7's exact prioritization rule, using only signals that exist today: `crossScopeCount`, presence of `ambiguousCorrelation`/`widenReasons`/`lossReasons`, and `complete` as `comparePaths`' first key — transformation-kind and protection-state are honestly deferred to Sub-project D / Milestone 2, never faked). **`result.terminals[].truncated`/`.droppedPathCount` are computed AFTER the global round-robin runs, never from the per-terminal cap alone — this is the single blocking finding from Task 1's own review, and the single most load-bearing correctness property in this module.** The formula is `enumeratedPathCount > returnedPathCount` (equivalently `droppedPathCount > 0`); computing it from the per-terminal cap alone would report `truncated: false` for a terminal the GLOBAL cap later starves to zero returned paths — reproduced concretely pre-fix on the §9.1 cross-join fixture (`enumerated=2 kept=2 returned=0 truncated=false`, §18.4's exact failure mode at *pair* granularity, inside the very field designed to prevent it at *call* granularity) and pinned verbatim, both directions, by `C5/5e`. A terminal's `terminalReasons` is likewise a sorted UNION over its own paths' reasons, never a positional pick (`group[0].terminal.reason`) — the same representative-picking bug class C4's own final whole-branch review found and fixed in `path-store.js`'s `origin` branch; a terminal genuinely CAN carry mixed reasons (e.g. one sink reached by both a `cycle` clip and a `depth-limit` stop at `maxDepth: 3` on the mutual-recursion fixture), pinned by `C5/5h`. `result.noPathReason` is computed **only when `truncated === false`** — this ordering IS §18.4's "never translate budget exhausted into no path" constraint expressed as a code path, not a stylistic choice: a truncated result can never acquire a `noPathReason`. §15.4's five pairwise-JSON-distinguishable answers (complete-with-paths / genuinely-empty / recording-gap / unknown-start-node / budget-truncated) are pinned as literal JSON by `C5/4b`, and the recording-gap row against a real cache-warming fixture by `C5/4c` — together the only guard against §18.4's constraint being silently undone by a future refactor. A path never changes data element (every edge joins two nodes sharing one `dataElementId` by construction, §14.3) so no filtering is needed for FR-301 field-precision to carry into FR-303 structure; a hop is a **denormalized copy** of its edge (not just the edge id) so a path can be graded (FR-306) without also carrying the store back to the caller. `ids.js` gained `pathId({ startNodeId, edgeIds }, discriminatorParts)` → `ppath:<12 hex>` (§15.6/§15.10 item 1) — the discriminator is the **edge id SEQUENCE**, never the node id sequence or a sorted set: order matters for a path (two assignments at two different program points joining the same node pair are two materially different edges, per `provenanceEdgeId`'s own `siteNodeId` discriminator, and FR-305 forbids hiding that difference by node-keyed dedup — proven on a real fixture by `C5/5`, ids pinned idempotent/order-sensitive by `C5/id-1` through `C5/id-4`). `validate.js` needed no change — `ppath:` is not a `DataFlowGraph v1` entity kind, confirmed by `json-schema-parity.test.js` staying green untouched. Every assertion from the design task's PoC — path-query-poc.test.js, not backtick-quoted here on purpose (it no longer exists in the tree, and this document's own doc-drift checker flags a dangling backtick-quoted path) — was kept and re-pointed at this shipped module in `test/lineage/path-query.test.js`, including `C5/4b`/`C5/4c` (the truncation-honesty guard named above) and `C5/5e`/`C5/5f`/`C5/5d` (the fix round's own load-bearing regression guards for the blocking finding and the two "prose stronger than proof" measurement fixes). The old PoC's hand-written function list for the private-field source scan (`C5/6`) was replaced with programmatic export enumeration (`Object.values(await import(...)).filter(v => typeof v === 'function')`) per the PoC's own fix-round-1 note, so an eighth exported function is swept in automatically rather than requiring a remembered list. No `DataFlowGraph v1` output, no FR-306 grade computation (counts only — `widenedHopCount`/`lossHopCount`/`ambiguousHopCount`/`crossScopeCount` — turning them into a grade is **C6**), no source/sink registry, no flow-sensitivity filter, no change to `path-store.js`/`engine.js`/`summaries.js`/`driver.js`, no driver wiring (a driver run still emits zero hops today, no source registry — Sub-projects D/E) — all deliberately out of scope, per §15.8. |
65
65
 
66
66
  **Milestone 1, Sub-project C, increment 6 (`flow-grade.js` — FR-306 edge grading) — COMPLETE, closes out Sub-project C:**
67
67
 
@@ -107,7 +107,7 @@ Two Step-1 empirical findings worth a future reader's attention, since they cont
107
107
 
108
108
  | Module | Responsibility |
109
109
  |---|---|
110
- | `DESIGN_GRAPH_BUILDER.md` | Design record, binding on every later Sub-project E increment (same role `DESIGN_INTRAPROCEDURAL.md` plays for A/B, `DESIGN_PATH_PROVENANCE.md` for C, `DESIGN_REGISTRIES.md` for D). Resolves the two mechanisms E2-E5 could not be written without, both proven by running real code: **SEEDING** (§3 — how a matched call site becomes a field-identity entry state) and **PROJECTION** (§6 — how a variable-granular provenance DAG becomes a system-granular `DataFlowGraph v1`). **The four things a reader must not miss.** (1) **The projection rule is "a graph node is a REGISTRY DECISION, not a provenance node and not a call site"** (§6.1): node identity is `(kind, subtypeKey, coverageStatus, externality, destination)`, which makes the node count bounded by the TAXONOMY rather than by repository size — measured at a constant **9 nodes** while replicating `test/fixtures/vulnerable-js` 1x -> 10x -> 50x -> 200x (8 -> 1600 functions), with edges/flows/dataElements scaling linearly over the same range. Per-call-site node minting was measured and REJECTED: it gives a 42-line fixture the same node count as the whole flagship platform. (2) **Seeding lands on the FIELD, not the container** (§3.2): `matchSource` matches `req.body`, and the seed path is extended outward through every enclosing pure-member access to `req.body.password`, which is the only reason `classifyDataElementName` can return `CREDENTIALS` rather than nothing. (3) **`driver.js`'s disclosed cache hazard had a second, worse half nobody had named — the KEY** (§3.6): writing a summary under `emptyState()` claims "this is what `fn` does when nothing flows in", which a seed makes false, so a clean call site was handed the DRIVER's own seed. Measured on a two-function real-parser fixture: pre-fix the caller's local carries `data:SEEDED`, post-fix it correctly carries nothing. (4) **§16.7 Finding 2's enumerator is CONFIRMED computable from `path-store.js`'s public read API alone** (§5) — `edgesFrom(id).length === 0` plus a `context-cap-degraded` check over `edgesTo(id)`'s `lossReasons` AND `annotations[]` — so the union mechanism was chosen over adding a sixth node kind, and `path-store.js` is untouched. **§11 escalates five things E cannot fix**, of which one is load-bearing for Sub-project F: **`engine.js`'s `case 'call'` unresolved branch unions only `expr.args`, never the RECEIVER**, so `pan.slice(0, 4)` loses the identity that `pan + 'x'` and `String(pan)` keep — which produces **zero flows** for the lineage corpus's own mask-then-log fixture, AC-02's worked example. `dataflow/engine.js` solved this with `_calleeReceiverTainted`; this package never inherited it and no design doc disclosed it. |
110
+ | `DESIGN_GRAPH_BUILDER.md` | Design record, binding on every later Sub-project E increment (same role `DESIGN_INTRAPROCEDURAL.md` plays for A/B, `DESIGN_PATH_PROVENANCE.md` for C, `DESIGN_REGISTRIES.md` for D). Resolves the two mechanisms E2-E5 could not be written without, both proven by running real code: **SEEDING** (§3 — how a matched call site becomes a field-identity entry state) and **PROJECTION** (§6 — how a variable-granular provenance graph becomes a system-granular `DataFlowGraph v1`). **The four things a reader must not miss.** (1) **The projection rule is "a graph node is a REGISTRY DECISION, not a provenance node and not a call site"** (§6.1): node identity is `(kind, subtypeKey, coverageStatus, externality, destination)`, which makes the node count bounded by the TAXONOMY rather than by repository size — measured at a constant **9 nodes** while replicating `test/fixtures/vulnerable-js` 1x -> 10x -> 50x -> 200x (8 -> 1600 functions), with edges/flows/dataElements scaling linearly over the same range. Per-call-site node minting was measured and REJECTED: it gives a 42-line fixture the same node count as the whole flagship platform. (2) **Seeding lands on the FIELD, not the container** (§3.2): `matchSource` matches `req.body`, and the seed path is extended outward through every enclosing pure-member access to `req.body.password`, which is the only reason `classifyDataElementName` can return `CREDENTIALS` rather than nothing. (3) **`driver.js`'s disclosed cache hazard had a second, worse half nobody had named — the KEY** (§3.6): writing a summary under `emptyState()` claims "this is what `fn` does when nothing flows in", which a seed makes false, so a clean call site was handed the DRIVER's own seed. Measured on a two-function real-parser fixture: pre-fix the caller's local carries `data:SEEDED`, post-fix it correctly carries nothing. (4) **§16.7 Finding 2's enumerator is CONFIRMED computable from `path-store.js`'s public read API alone** (§5) — `edgesFrom(id).length === 0` plus a `context-cap-degraded` check over `edgesTo(id)`'s `lossReasons` AND `annotations[]` — so the union mechanism was chosen over adding a sixth node kind, and `path-store.js` is untouched. **§11 escalates five things E cannot fix**, of which one is load-bearing for Sub-project F: **`engine.js`'s `case 'call'` unresolved branch unions only `expr.args`, never the RECEIVER**, so `pan.slice(0, 4)` loses the identity that `pan + 'x'` and `String(pan)` keep — which produces **zero flows** for the lineage corpus's own mask-then-log fixture, AC-02's worked example. `dataflow/engine.js` solved this with `_calleeReceiverTainted`; this package never inherited it and no design doc disclosed it. |
111
111
  | `driver.js` (extended, additively) | Gained `opts.seedEntryState(fn) -> state \| falsy`, the source-seeding hook, plus the one-identifier cache-key fix that hook makes load-bearing (`cache.set(fn.qid, entryState, ...)` instead of `cache.set(fn.qid, emptyState(), ...)`). Both are no-ops when the hook is omitted — `entryState` IS `emptyState()` then — and `test/lineage/driver.test.js`'s `E1/driver-1` pins results AND cache keys against a **hardcoded golden literal captured from the pre-hook file**, never against the shipped implementation (C3 §13.2a's vacuous-test trap). `E1/driver-3` proves the cache-key fix in BOTH directions, using a local replica of the pre-fix loop as the mutant. Nothing else in this file changed at E1. **Sub-project E, increment 2 (task review M-2): `source-seeding.js` is now this hook's first real consumer**, and **increment 3's `graph-builder.js` (below) is now this package's real end-to-end caller** — see those modules' own rows below. **Increment 5** ships `lineage/index.js` and wires it into `runFullScan` (`engine.js`) — see that row below for the full wiring story. |
112
112
 
113
113
  **Milestone 1, Sub-project E, increment 2 (`source-seeding.js` — shipping the seeding mechanism proven by E1; closes the "0 hops on real code" gap for real) — COMPLETE. E3-E5 are NOT started.**
@@ -273,6 +273,41 @@ Two Step-1 empirical findings worth a future reader's attention, since they cont
273
273
 
274
274
  **Milestone 2 status note**: Sub-project D (FR-403 taxonomy + FR-307 control-credit) is fully complete. Sub-project E is fully complete (increments 1-3). Sub-project B (transit protection) is functionally complete for its exit-gate scope — closes AC-03/AC-04 for real. **Sub-project C (at-rest protection) has its application-layer evidence source done** (`edge.protection.atRest` now carries real `protected` verdicts for encrypted fields reaching a store) — C2 (storage/IaC encryption config detection) and C3 (database column encryption config) remain genuinely new, unbuilt detection, each needing its own dedicated scoping pass. **Sub-project G (scoped policy verdict) is fully complete** — closes AC-09 for real. **Sub-project F increment 1 (`edge.provenance`) is complete** — closes only FR-304's first clause ("identify whether an edge is code-derived...") — F2 (bridging `cross-lang-*`'s schema parsers into real schema-derived edges) and F3 (declared-service-graph ingestion) remain genuinely new, unbuilt work, each needing its own future scoping pass; no exit-gate AC depends on either. **Sub-project H increment 1 (the false-protected release gate for `transit`/`atRest`) is complete** — `handling`'s own dimension remains genuinely open (the scoping doc's own (a)/(b) fork: wait for a real `edge.protection.handling` producer, or reframe around `HANDLING_VALUES`' own vocabulary — neither decided). **Sub-project I (exit-gate closure) is fully complete** — every AC-03 through AC-09 plus AC-12 named in the Milestone 2 exit gate's own wording has a real, verified proof, and the false-protected release gate passes; see the exit-gate status table above. External destination resolution (Sub-project A) has only increment 1 done (literal/dynamic resolution only — hostname/port/route extraction, constant-folding, config-chain resolution, schema correlation, and AI-provider/model resolution are all still open, per `DESIGN_DESTINATION_RESOLVER.md`'s own deferred list). **Milestone 2's own exit gate (PRD line 1809) is satisfied as literally worded** — C2/C3, F2/F3, H's own `handling` dimension, and Sub-project A's remaining increments are real, disclosed, non-blocking future work, not part of the exit gate's own text.
275
275
 
276
+ ## Provenance graph vs. its DAG view, and SCC scheduling (DAG-01)
277
+
278
+ Terminology, binding for active docs: the raw hop structure (`path-store.js`, and the call graph the driver schedules over) is a **provenance graph**. It may contain cycles (mutual recursion, `path-store.test.js`'s 8-node, 11-edge fixture). It is never called a DAG. The **DAG** is the *derived* condensation: `graph/scc.js` collapses each strongly connected component to one node, and the resulting component graph is acyclic by construction. The raw graph is never modified. The `DESIGN_*.md` records still say "DAG" in places; read those as "provenance graph" (see the note at the top of `DESIGN_PATH_PROVENANCE.md`).
279
+
280
+ | Module | Responsibility |
281
+ |---|---|
282
+ | `../graph/scc.js` | Pure, iterative (no recursion) Tarjan SCC. `computeScc({nodes, edges})` returns `components` (stable ids by smallest member, sorted `members`, sorted `internalEdges`, `recursive`), `componentOf`, sorted `condensationEdges`, and a deterministic topological `order` (ties by component id). Output is independent of input order. `verifyScc(graph, result)` checks every edge is internal or forward in `order`. Tested at 100,000 nodes. SCC membership says nodes can reach each other, not that every pair has a feasible taint path in every context. |
283
+ | `scc-solver.js` | `solveRecursiveComponent` and the fact domain below. Pure; budgets are injected callbacks. |
284
+ | `driver.js` (extended) | Schedules `runFieldIdentityAnalysis` over the call-graph condensation, callees before callers (edges from `callGraph.edges` or `opts.calleesOf`). A singleton with no self-call runs the unchanged single-pass step, so non-recursive results are byte-identical (pinned by a differential test against the legacy loop). A recursive component is re-analyzed round by round. Returns `convergence`: `converged`, `status` (`complete` or `partial`), `rounds`, `changedFacts` (capped sample plus `changedFactsTotal`), `wideningReasons`, `remainingFrontier`, `stoppedBy`, `components[]`. Budgets: `opts.maxRounds`, `maxPathDepth`, `maxFunctionAnalyses`, `deadlineMs` with `now()`, `signal`. |
285
+
286
+ **Bounded fact domain (R01.4).** A function summary is a finite set of facts: `ret:<dataElementId>` and `mut:<paramPath>:<dataElementId>`. Ids come from the finite set of entry-state and program identities; parameter paths are syntactic and additionally capped at `maxPathDepth` (default 6 segments). A deeper path folds to its capped prefix (an over-approximation) and records the widening reason `path-depth-cap`. Join is set union per fact family, and the solver stores `previous JOIN analyze(previous)`, so each summary sequence is an ascending chain in a finite lattice and must stabilize even if the underlying analysis were not monotone. `maxRounds` (default 32) is an independent bound: hitting it records `round-cap` and the component is reported partial. An SCC solver over an unbounded path domain would not gain termination from SCCs alone; this bound is what supplies it.
287
+
288
+ **What `converged` means.** A full round changed no member summary and at least two rounds ran (round 1 compares against the bottom seed, which is not an analysis result). It does not mean the code is safe. A budget stop, a round cap, or an unreached function yields `converged: false`, `status: 'partial'` and the unfinished qids in `remainingFrontier`; a last-iteration under-approximation is never a conservative proof.
289
+
290
+ **Disclosed limit.** With empty entry states (no `seedEntryState`) the real engine produces bottom summaries for recursive members, so a real-code recursive component converges in 2 rounds. Multi-round propagation is exercised by the six-function ring in `test/lineage/scc-solver.test.js` against a slow exhaustive reference. Contexts reached through non-empty call-site argument states are dropped between rounds (`FieldIdentitySummaryCache.dropContexts`) and recomputed lazily, so they are not themselves part of the fixed point.
291
+
292
+ Tests: `test/lineage/scc.test.js`, `test/lineage/scc-solver.test.js`, plus the existing `driver.test.js` and `summaries.test.js`.
293
+
294
+ ## Provenance correlation version 2 (DAG-03)
295
+
296
+ Version 1 pairs the two halves of a hop at the join key `(scope, nodeId, dataElementId, context)` and marks what it cannot tell apart as `ambiguousCorrelation`. Version 2 adds evidence the engine had when it emitted each half, so halves pair only when that evidence proves them compatible. **It is OFF by default.** Enable with the `correlationVersion: 2` option (`buildDataFlowGraph`, `runFieldIdentityAnalysis`, `new PathStore({}, {correlationVersion: 2})`) or `AGENTIC_SECURITY_LINEAGE_CORRELATION=2`; an explicit option wins over the environment. With it off, no hop, id, digest or output field changes (pinned against digests captured before the change, from a fresh process, in `test/lineage/correlation-v2.test.js`).
297
+
298
+ | Module | Responsibility |
299
+ |---|---|
300
+ | `correlation.js` | Pure vocabulary: `V2_HOP_FIELDS`, `resolveCorrelationVersion`, derivation helpers (`deriveChild`, `derivationCovers`, `scopeForSubPath`, `relPathOf`), `invocationIdOf`, `pairCompatibility` (the one pairing rule) and `buildCrosswalk`. |
301
+ | `value-versions.js` | Reaching definitions over one function's CFG. A definition id is the writing `assign` node, or `entry`. A merge keeps both alternatives, a strong write kills the path and everything under it even when the written value is clean, a computed-key write is weak. Purely structural: it labels hops and never touches the identity state. |
302
+ | `engine.js` (extended) | When `ctx.correlationVersion === 2` and a recorder is attached, every sub-expression gets a derivation segment (`sub`), in-halves carry `derivationId`/`relPath`/`fromVersions`, out-halves carry `derivationScope`/`relPath`/`toVersion`, a resolved call carries `invocationId` and `callRole: 'return'`. `rawRecordHop` is the recorder below the derivation wrappers, which is what a callee's analysis is handed so it does not inherit the caller's expression stamps. |
303
+ | `summaries.js` (extended) | Argument reads are stamped `callRole: 'arg'` with the invocation and actual position (`argumentContext`); the parameter binding is `callRole: 'bind'` with the formal position and `toVersion: 'entry'`. |
304
+ | `path-store.js` (extended) | Version 2 builds value-versioned path nodes (one source node per reaching definition), pairs through `pairCompatibility`, and gives every edge `correlation` (`proven` or `legacy-unknown`), `invocationId` and `callBoundary` (`call` or `return`). A source node whose definition never wrote the data element is a phantom alternative and is dropped and counted (`diagnostics().versionPruned`). `crosswalk()` returns the v1 to v2 id map. Annotations are sorted so graph content does not depend on hop delivery order. |
305
+ | `path-query.js` (extended) | Balanced call/return traversal: walking back through a return edge pushes its invocation, the matching call edge pops it, a mismatched call edge is not followed and is counted (`result.balancing.unbalancedPruned`, with edge ids). Past `maxCallDepth` (default 8) the stack is abstracted: the path says so in `callContext.widened`. If every way out of a node was a mismatched pairing the branch ends as a partial path with terminal `unbalanced-call`, never as an empty "no flow". |
306
+
307
+ **What is never pruned.** A hop without version 2 evidence (hand-built streams, an expression kind that carries none) pairs exactly as in version 1 and the edge says `legacy-unknown`; an edge marked ambiguous stays marked. Evidence only removes pairings it proves incompatible. A container-level field path (`''`) never disagrees with a field path, because pruning on it would drop a real edge.
308
+
309
+ **Limits, stated.** The engine copies values on assignment, so an alias write after `o2 = o1` does not reach `o1`; version 2 labels that faithfully and does not model aliasing the engine does not. A write through a callee is not applied to the caller's state (`applyAtCallSite` is not wired into the engine). The receiver of an unresolved method call is stamped `receiver: true` and still pairs. Reaching definitions treat `entry` as an alternative only when the entry state carried that identity at that path. Recursion is handled by the per-path visited set plus the call-depth bound, not by a context-sensitive summary. Flow ids, node ids and edge ids of `DataFlowGraph v1` are unchanged in both modes; only `pnode:`/`pedge:` ids differ. Suite: `test/lineage/correlation-v2.test.js`.
310
+
276
311
  ## Milestone 1 exit-gate status
277
312
 
278
313
  The PRD's own Milestone 1 exit gate (§26, line 1796): *"AC-01, AC-02, AC-07,
@@ -357,7 +392,7 @@ fabricated verdict.
357
392
  integration, the project-wide two-phase driver, recursion
358
393
  refinement, context-sensitivity tuning) are done, per
359
394
  `docs/superpowers/plans/2026-08-30-data-flow-explorer-m1-subproject-b-scoping.md`.
360
- The path DAG (Sub-project C) increments 1-3 are now **complete** —
395
+ The path provenance graph (Sub-project C) increments 1-3 are now **complete** —
361
396
  increment C1 (design spike + a 4-site proof-of-concept), C2 (full
362
397
  intraprocedural instrumentation, every case in both
363
398
  `resolveExprIdentities` and `step()`, per `DESIGN_PATH_PROVENANCE.md`
@@ -367,7 +402,7 @@ fabricated verdict.
367
402
  and B5/B6 degradation marking, per `DESIGN_PATH_PROVENANCE.md` §13/§13.7)
368
403
  are all done, above; a cross-function hop is now recorded by shipped code
369
404
  whenever a recorder is attached. C4 (`path-store.js`, the compact
370
- provenance DAG structure itself, per `DESIGN_PATH_PROVENANCE.md` §14) is
405
+ provenance graph structure itself, per `DESIGN_PATH_PROVENANCE.md` §14) is
371
406
  now also **complete** — see the module table above. C5 (`path-query.js`,
372
407
  bounded backward reconstruction + truncation semantics, per
373
408
  `DESIGN_PATH_PROVENANCE.md` §15) is now also **complete** — see the
@@ -1232,6 +1267,35 @@ The digest of an ingested file is over the UTF-8 text that was parsed. Observati
1232
1267
  `agentic-security boundaries` (`boundaries-run.js`, explicit, read-only, off by default), so default output is unchanged. The boundary graph attribute grammar admits credential-shaped values such as `password=x` (it only
1233
1268
  restricts characters); the projection refuses them, but the graph contract itself does not.
1234
1269
 
1270
+ ## Path predicates and control effectiveness (DAG-04)
1271
+
1272
+ **OFF by default.** Enable with `buildDataFlowGraph(callGraph, { pathEvidence: true })` or `AGENTIC_SECURITY_LINEAGE_PATH_PREDICATES=1` (an explicit option wins). Enabled, a flow gains `pathEvidence` and a graph edge may gain `predicateRefs`; with it off no field, id or digest changes (pinned by stripping the added fields and comparing the graph and its digest).
1273
+
1274
+ | Module | Responsibility |
1275
+ |---|---|
1276
+ | `path-predicates.js` | Branch conditions for a witness. A node is guarded by (branch, polarity) only when that arm dominates it and has the branch as its only predecessor, so a node after a join is guarded by neither arm. Conditions are kept as bounded DNF (`a || b` is two distinct alternatives), variables are keyed by (call frame, name, reaching-definition versions), and a callee parameter is tied to the caller argument only through an explicit call binding. Both arms of one branch on one witness are reported (`mutually-exclusive-branches`), never conjoined; inside a loop with a per-iteration definition the answer is `unknown`. Evaluation reuses `numeric-domain.js` intervals through `symbolic-exec.js` `assume`, plus equality on constants and boolean atoms. Results: `feasible-model`, `infeasible-under-model`, `unknown`, `unsupported` (reflective dispatch anywhere in a frame's function, or no CFG), `timeout`; each with semantics version, reasons, limitations. Satisfiability never implies exploitability, and `infeasible-under-model` is relative to the stated limits. |
1277
+ | `smt-orchestrator.js` | The optional solver path. `checkWithSolver` is awaited, deadline-bounded, cancellable (caller `AbortSignal` plus its own deadline abort the solver's signal) and never throws; a late solver settlement is swallowed. `noSolver` is the default stub (`unsupported`). A solver is consulted when the abstract answer is `unknown`, or `feasible-model` with `consultOnFeasible` (default: on only when `AGENTIC_SECURITY_LINEAGE_SMT=z3`); the abstract answer stays on the result, and on a feasible answer only a solver `unsat` changes it. `resolveSolver` picks an explicit solver, else the z3 backend when the env switch is `z3`, else `noSolver`. An `unsat` over branch arms in a loop is not trusted. |
1278
+ | `smt-z3-backend.js` | The real backend on the optional `z3-solver`. Lazy dynamic import (variable specifier, build flag `-e z3-solver`), so a default scan never loads it; a missing or failing package, or a context that ignores its timeout, is `unsupported`, never a crash and never `unsat`. `encodeProblem` is the verified encoding (Real sort, `v0..vN` renaming, constants asserted, opaque predicate = no constraint); an inexpressible problem is `unknown` with an `encoding-unverified:*` reason, and a `sat` that used an opaque predicate is `unknown`. Per-query timeout, Z3 memory cap, fixed seed, one query at a time, definitive answers cached by SMT-LIB text. Reached only from the async path-evidence entry points; the synchronous graph build never calls it. Suite: `test/lineage/smt-z3-backend.test.js` (real-solver tests skip loudly where z3 is absent). |
1279
+ | `control-credit.js` | A control earns credit on a witness only for the matching sink family (catalog `appliesTo` vs the sink CWE family), on the value version the sink reads (reassignment or a merge with another definition refuses), and when nothing later on the witness reverses it (`_reversedOnPath`). A parameterized SQL call earns SQL credit only for values bound after a literal placeholder query text. A bare callee name shared by entries that disagree on families is ambiguous and not credited. An unrecognized call that widened the value is opaque: `unknown`. Credits say `semanticsProof: false`: dominance and catalog family are evidence, not proof of sanitizer semantics. `assessAllPathsProtectedWithControls` feeds real control states into `witness-selection.js` `assessAllPathsProtected`, lists `infeasible-under-model` paths it excluded, keeps `unknown`/`timeout` paths relevant, and accepts an `abstractControlProof` (dominating, single-reaching-definition control, intraprocedural) only when no enumerated path refutes. |
1280
+ | `path-evidence.js` | Assembles per-path evidence, summarizes a flow group (alternatives stay distinct, aggregate control state is the worst over relevant paths) and `buildControlEvidence`, which fills the witness bundle `controlEvidence` slot (`feasible`/`infeasible`/`unknown` plus the precise result in `detail`). |
1281
+
1282
+ Limits: value versions come from per-function reaching definitions; mutation through a callee, alias or closure is not modeled; control matching is by callee name; the control credit read of later reads is checked within the control's own function frame only. Suite: `test/lineage/path-control-evidence.test.js`.
1283
+
1284
+ ## Completeness and bounded execution, part 2 (DAG-07)
1285
+
1286
+ | Module | Responsibility |
1287
+ |---|---|
1288
+ | `completion.js` | The shared completion envelope (status `complete`/`partial`/`cancelled`/`unavailable`/`invalid`, `queryExhausted`, `modelCoverage`, `negativeClaimAllowed`, `reasons`, `limits`, `unexploredFrontierCount`). Pure. `negativeClaimAllowed` is always derived, never taken from a caller: it needs a complete status, an exhausted query, an empty frontier and no reasons, and is scoped to the modeled graph. Adapters: `fromAnalysisCompletion`, `fromQueryResult`, `completionOfGraph` (adds `zero-flows-with-incomplete-analysis`). `exportGraphJSON` carries the envelope top-level (additive; the digests cover `graph` only). The CLI prints it for `explore` and `dataflow export`, scan health lists it under `lineageAnalysis.completion`, and the frontend coverage banner states it. |
1289
+ | `owned-worker.js` + `owned-worker-entry.js` | Worker-thread run with an AbortSignal and a deadline. Cancellation is a shared flag the work polls (works inside a synchronous loop); a worker that does not acknowledge within `ackGraceMs` (500 ms) is terminated by the watchdog, so the caller is released inside one second either way. Same `worker_threads`/`terminate()` convention as `pipeline/analyzer-supervisor.js`. |
1290
+ | `owned-build.js` | Wires the owned worker into the real lineage build, OFF by default. `AGENTIC_SECURITY_LINEAGE_DEADLINE_MS` (or `deadlineMs`) set: `runFullScan` sends the file contents to a worker that rebuilds the IR and call graph and runs `buildLineageGraph` with a cooperative stop signal (`index.js` forwards `signal`/`deadline` only when given). The analysis stops at the next function boundary and returns a graph whose `scanHealth.status` is `cancelled`; a worker that never polls, or is still parsing, is terminated by the watchdog and the result is `{status: 'cancelled', graph: null, terminatedByWatchdog: true}` (the scan records it as a lineage failure). Unset: the in-process call, unchanged. Limits: the IR is rebuilt in the worker (twice the parse cost), `onProgress` and `perFile` cannot cross the thread boundary, and the worker entry is resolved from the source tree, so in the single-file `dist` bundle the worker entry is expected not to be found (not verified against a build) and the result would be a `failed` status, not a crash. Suite: `test/lineage/owned-build.test.js`. |
1291
+ | `witness-selection.js` | Display-only diversity ranking (source, sink, data class, trust boundary, control state, ambiguity) and `assessAllPathsProtected`, which looks at every enumerated path (a hidden unprotected path refutes the claim) and refuses outright on a truncated, incomplete or resumed query. |
1292
+ | `continuation.js` | Resumable path queries. A token is bound to the exact `PathStore.digest()` and a digest of the query configuration, is one-shot, expires (TTL) and the store is bounded in entries and bytes. A changed graph or configuration is rejected (`invalid`, never a silent restart). A resumed page never licenses a negative claim. |
1293
+
1294
+ `path-store.js`: an explicit join above `maxPairingsPerGroup` becomes a typed ambiguous-summary relation (S+O edges through one `summary` node, all marked ambiguous) and the group stays reported in `limitStatus()`. `path-query.js`: `maxMaterializedHops` bounds path output; the result carries `unexploredFrontierCount` and, when a budget stopped the walk, `resumeState`. `source-seeding.js`: `planSeeds` also returns `completion`. Outside this package: `dataflow/tabulation.js` (`reachabilitySliceWithCompletion`, `completion` on `rhsReachableFunctions`), `dataflow/backward.js` (`sliceBackwardWithCompletion`), and `dataflow/incremental.js` (only the empty-entry summary is persisted, keyed by the real qid; a restored summary standing in for a tainted context is reported `incremental-empty-entry-reuse`). Suite: `test/lineage/dag07-completion.test.js`.
1295
+ ## Boundary contracts (DAG-08)
1296
+
1297
+ `boundary-contract.js` is a pure module (imports only `node:crypto`, `schema.js`, `graph/scc.js`). A versioned contract declares an HTTP, queue or storage edge: producer and consumer sites, schema name/version, field mappings (renames), persistence class (`immediate` vs `persisted`), provenance and a configuration digest. A declaration establishes a MODELED untrusted input at the consumer, never an observation: every attribution carries `modeledInput: true, observed: false, exploited: false`. `bindBoundaryContracts` binds sites to parser-resolved functions and parameters; an unbound site (file/function/param not found, ambiguous name, field outside a known Haskell record) is a typed `unbound-boundary` gap, never a finding. The consumer is seeded only at the mapped consumer fields (consumer projection), so a sibling field is not attributed. `correlateStorage` joins a write and a read only under a declared storage contract naming the same store, table and key/schema version. `analyzeServiceBoundaries` condenses the service graph with `graph/scc.js` and solves recursive components with a round and fact budget (a budget hit is `truncated`, never convergence). Evidence kinds (`foreign-graph-validation`, `digest-check`, `manual-assertion`, `code-mapping`, `observed-connection`) stay separate; only `observed-connection` can set `observed`, and nothing here creates one. The seeding step that needs the engine is `dataflow/cross-service-taint.js` (`runBoundaryTaint`), which hands seeds to the engine through `fn.boundarySeeds`. `haskell-view.js` adds `describeHaskellBoundaries` (unresolved higher-order constructs) and `haskellConsumerLocations` (original and view locations); `nix-view.js` adds `nixGeneratedScriptScopes` and `bindNixValueToApplication` (static-configuration scope, runtime entry only through an explicit verified binding); `cross-repo-link.js` adds `linkEvidence` (a declared link is a `manual-assertion`). Limits: Haskell record fields are checked only where the IR recorded the record parameter; the seeded run is one engine pass per seed (budget `maxRuns`, default 64); no storage or queue site is discovered automatically (sites come from contracts).
1298
+
1235
1299
  ## Conventions
1236
1300
 
1237
1301
  - Every enum here is a single source of truth for its concept. If you add
@@ -32,7 +32,7 @@ both are settled here:
32
32
  1. **Seeding** (§3) — how a real call site becomes a field-identity entry
33
33
  state. Today the shipped driver hardcodes `emptyState()` and produces
34
34
  **exactly zero hops** on real code. Measured, twice, in `E1/1`.
35
- 2. **Projection** (§6) — how a provenance DAG that is *variable*-granular
35
+ 2. **Projection** (§6) — how a provenance graph that is *variable*-granular
36
36
  (494 pnodes for a 33-file project) becomes a `DataFlowGraph v1` document
37
37
  that is *system*-granular (14 nodes for a whole synthetic platform).
38
38
 
@@ -1,5 +1,11 @@
1
1
  # Path Provenance Recording — Design Record
2
2
 
3
+ > Terminology note (DAG-01): this record predates the SCC work and says "DAG"
4
+ > for the recorded provenance structure. That structure is a **provenance
5
+ > graph** and can be cyclic (see section 9.3). The acyclic "DAG" is the derived
6
+ > component view built by `scanner/src/graph/scc.js`. Read every "DAG" below as
7
+ > "provenance graph". Decisions here are unchanged.
8
+
3
9
  Scope: Sub-project C of Milestone 1, increment C1 (see
4
10
  `docs/superpowers/plans/2026-08-30-data-flow-explorer-m1-subproject-c-scoping.md`).
5
11
  This document decides **how provenance is recorded** as data flows through