@clear-capabilities/agentic-security-scanner 0.186.1 → 0.187.0

package/CHANGELOG.md

@@ -9,6 +9,29 @@
 > make the history less accurate, not more.
 
 
+## 0.187.0 - Haskell parity work: wider exploit proof, broader SAST and taint, new unseen measurement sets
+
+- **Haskell SAST and taint, generalised beyond the shapes they were written against.** New sources and sinks (Scotty, Yesod, WAI, Servant, Req, conduit, http-client), guards recognised by what they do rather than by name (host allow-lists, hand-written HTML escapers, `sh -c` scripts), SSRF reported at the send instead of the parse, and Haskell-scoped engine fixes for branches handed to combinators, `words`, `>>=` and common lowering forms. On unseen-v3, now a development set, SAST and taint reach 100%; that is a regression result, not a generalisation claim.
+- **Honest unseen measurement.** A new set, unseen-v4 (78 forms, 13 families, measured once before this release's engine work), gives SAST recall 45.8% to 54.2% and taint precision 73.3% to 84.6%; taint recall is unchanged at 73.3%. Several families remain at 0 to 33% recall.
+- **Real-code precision did not move.** On four pinned real Haskell projects (`bench/dag/independent-haskell/`, one LLM labeler, 60 of the flagged sites), default-scan precision is 2.3% (95% interval 0.4% to 12.1%), against 2.0% before. The new engine flags additional sites that are unlabeled and are not counted.
+- **Alias fix.** A point-free alias of a library function (`tidy = takeFileName`, `run = callCommand`) was treated as an unresolved call, so a sanitizer alias left a flow unsanitized and a sink alias went undetected. Aliases of library functions are now expanded; a local binding with the same name disables the expansion.
+- **Exploit proof.** Widened Haskell reproducer coverage and oracles.
+- **Gates.** 56 Haskell mutation cases and 8 synthetic corpus entries (verdict-flip 460 of 460, corpus 228 of 228); layer-recall maps `.hs` and `.lhs`, taint attribution 126 to 134.
+- **Parity outside the engine.** Haskell on the MCP `query_taint`, context, compliance mapping, capability decisions, attack taxonomy, entrypoint inventory and reports, with `docs/language-parity-haskell.md` listing what remains open.
+- **Not done.** The container proof path is untested, `prove_finding` and `prove_fix` have no `.hs` harness builder, and `haskell-web-auth` route fixtures need a host with scotty, wai, servant and yesod to compile.
+
+## 0.186.2 - Haskell exploit proof, a default-scan drift gate, and the measurements behind earlier claims
+
+- **Haskell in the exploit proof fabric (GA capability, new language).** `proof doctor` now lists `haskell` and `proof generate` builds reproducers for Haskell findings. Reproducers run in GHC's interpreter (`ghc -e`), so there is no build step. Command injection and path-traversal writes are proven; server-side request forgery is refused because no HTTP client ships with GHC; SQL, argument injection and deserialization are refused by name. Only `base` and the packages that ship with GHC are allowed (read from the installed GHC); a handler that needs a web framework is refused with the package named. Template Haskell, quasi-quotes, CPP, the foreign function interface and plugin options are refused as load-time code execution. 45 of 45 fixture reproductions across 15 classes, 0 false confirmations. Limits: exercised under local confinement on macOS with GHC 9.14.1 only; the container image is wired but was not run against a real engine, and Linux was not exercised.
+- **`fix --apply --require-proof` now holds for Haskell and Nix fixes.** Their fixer returned before the proof gate, so the flag was silently ignored. The gate is shared now; the patch is previewed, proven, then written, and aborted if the written patch differs from the proven one.
+- **Correction to 0.186.0: default scan output did change.** 0.186.0 said a default scan's output was unchanged. It was not. Findings from flow-pair and structural detectors now carry `line` (taken from the sink, then the source) where they had none; on the fixtures used by the new gate, 33 of 84 findings gained a line, and one command-injection finding is now reported by the structural detector under the title "Command Injection (User-Controlled Input)". The stable finding id does not use the line. New detectors (redirect target, reflected input, send-file root, init scope, listen bind address) and general precision rules for open redirect, reflected input, blocking I/O, file serving, logging by name and Nix option types changed what some projects report. Line pragmas now match findings that had no line.
+- **New gate: default-scan drift (`bench/default-scan-drift`).** A default scan over eight committed fixtures is compared to a committed baseline; any added, removed or changed finding fails `npm test` and prints the difference. Re-baselining is explicit and belongs with a changelog entry naming the rule.
+- **Independent precision measurement (`bench/dag/independent`).** On projects the rule work never touched (koa, hapi, nest samples and strapi server packages; five Nix configuration collections), a seeded random sample of 60 flagged sites per language was labeled from source by one LLM reader. Precision of the whole default scan: JavaScript and TypeScript 7 of 60, 11.7% (95% interval 5.8% to 22.2%); Nix 4 of 59 decided sites, 6.8% (2.7% to 16.2%). This is the whole scan, not the narrower classes the 0.186.0 precision work targeted, so it does not contradict those figures; it shows they do not describe a typical project. The sample is for measurement only.
+- **Speed gate now matches the hosted runner.** The time-to-first-finding budget on the hosted runner is the median of ten passing release runs (11.1 s) with a 1.35x limit (15.0 s), and a new `ttff` job runs it on every pull request, so a slowdown like 0.186.0's is found before a tag.
+- **Unmet targets are stated.** The capability registry, README and graph-precision page list the targets 0.186.x missed: incremental analysis 2.5x and 2.4x on the synthetic projects against 3x, 75 of 982 sink sites still on the full path, 3 of 8 on the multi-service fixture, the z3 backend not used by a default scan.
+- **S7 test failures explained.** They ran the built bundle while it crashed at start-up on a statically imported optional package (the `z3-solver` import fixed in 0.186.0's follow-up). A new test fails if the bundle imports any optional package statically.
+- **The `v0.186.0` tag was removed.** That release was never published to npm.
+
 ## 0.186.1 - Git history annotation blames once per file, which restores scan speed (fix)
 
 0.186.0 was tagged but never published: its release workflow stopped at the time-to-first-finding gate (15 to 18 s against a 13 s limit), so the first npm release carrying that work is 0.186.1. Nothing in 0.186.0's content changed apart from the fix below.
@@ -19,7 +42,7 @@
 
 ## 0.186.0 - The graph-precision capabilities are reachable from the CLI, a z3 backend checks path feasibility, and demand queries and incremental analysis got faster (Preview)
 
-Everything new is Preview, opt-in and off by default. A default scan's output is unchanged by the new commands and by the solver. No GA capability changed its contract.
+Everything new is Preview, opt-in and off by default. The new commands and the solver do not run in a default scan. **Correction (added in 0.186.2):** this entry originally said a default scan's output was unchanged. That was wrong for the release as a whole: the precision work, new detectors and a line-number backfill changed default output. See 0.186.2 for the list and the check that now catches this. No GA capability changed its contract.
 
 - **Reachable from the CLI.** `dataflow view`, `dataflow contracts`, `dataflow sink-query`, `dataflow plan-cuts`, `witness build` and `witness verify` expose the typed graph views, boundary contracts, demand reverse taint, min-cut control planner and signed witness bundles. Every report carries a completion status (`complete`, `partial`, `cancelled`, `unavailable`, `invalid`) and `negativeClaimAllowed`; "no modeled path" is never reported as "no vulnerability". `witness build` exits 1 and writes nothing when no modeled path exists. The graph views other than the dataflow view and a tool in the MCP server remain library-only.
 - **A real z3 feasibility backend (`AGENTIC_SECURITY_LINEAGE_SMT=z3`).** Path conditions are translated to SMT-LIB and checked with a per-query timeout and memory cap. Only a proven `unsat` over a verified encoding can reduce a path's exploitability; a timeout, an unknown result, a missing solver or a condition the encoding cannot express changes nothing. The synchronous graph build does not call it: it is reached through the async path-evidence entry points. NaN and loose-equality coercion are not modeled.

package/bin/agentic-security.js

@@ -3587,6 +3587,32 @@ async function cmdDigest(args) {
   console.error('digest --slack <url> OR digest --discord <url>'); return 4;
 }
 
+/**
+ * --require-proof gate shared by the generic fix path and the Haskell/Nix language fixer: the patch (`files`: path -> patched text) is
+ * allowed only if the original exploit is confirmed beforehand and demonstrably blocked afterwards. Returns { ok, code }; nothing is written here.
+ */
+async function requireProofGate(args, scanRoot, f, last, files) {
+  // --require-proof: the patch is applied ONLY if the original exploit is confirmed beforehand and demonstrably blocked
+  // afterwards (posture/proof-fabric/fix-proof.js). Missing reproducer, unavailable environment or an incomplete replay all refuse.
+  const { resolveReproducer } = await import('../src/posture/proof-fabric/cli.js');
+  const { findSiblings } = await import('../src/posture/proof-fabric/siblings.js');
+  const { verifyFixWithProof, formatFixProof } = await import('../src/posture/proof-fabric/fix-proof.js');
+  const { verifyFix } = await import('../src/posture/fix-verify.js');
+  const loaded = resolveReproducer(scanRoot, f, args.flags.reproducer);
+  if (!loaded.reproducer) { console.error(`--require-proof: no reproducer for ${f.id}, and none could be generated: ${loaded.reason}. Refusing to apply; nothing was executed. Supply --reproducer <file>.`); return { ok: false, code: 4 }; }
+  if (loaded.generated) console.error(`--require-proof: using a reproducer generated from the finding (${loaded.generated.shape}, ${loaded.generated.class}); it counts only if executing it confirms the exploit.`);
+  const staticVerification = await verifyFix({ scanRoot, originalFindingStableId: f.stableId || null, files, recordMetrics: false });
+  // equivalent sibling exposures: the engine's own other findings of this kind, plus other places with the same code shape
+  const sib = findSiblings({ root: scanRoot, finding: f, scanFindings: last.findings || [] });
+  const siblings = sib.siblings.map((x) => ({ id: x.id ?? null, file: x.file, line: x.line, source: x.source }));
+  if (sib.truncated.length) console.error(`--require-proof: the sibling search stopped early (${sib.truncated.join(', ')}); absence of siblings is not shown.`);
+  const fp = await verifyFixWithProof({ finding: f, reproducer: loaded.reproducer, files, scanRoot, staticVerification, siblings, siblingSearchTruncated: sib.truncated });
+  console.log(formatFixProof(fp));
+  try { (await import('../src/posture/dnr/store.js')).saveRecord(scanRoot, f.stableId || f.id, 'fix', fp); } catch { /* recording is best effort; the refusal below does not depend on it */ }
+  if (!fp.verified) { console.error('\nRefusing to apply: the fix is not proof-verified (--require-proof).'); return { ok: false, code: 4 }; }
+  return { ok: true, code: 0 };
+}
+
 async function cmdFix(args) {
   const id = args.flags.finding;
   const isPreview = !!args.flags.preview;
@@ -3636,7 +3662,21 @@ async function cmdFix(args) {
         return 4;
       }
       const proj = loadLanguageProject(scanRoot);
+      // --require-proof applies to a Haskell fix exactly as to any other: the verified patch is previewed first, the exploit is
+      // confirmed against the original and blocked against the patch, and only then is the same patch written.
+      let provenFiles = null;
+      if (isApply && args.flags['require-proof']) {
+        const prev = await languageFixPreview(f, proj.files, { apply: false, root: scanRoot });
+        if (!prev.ok) { console.error(`No verified fix for this finding (${prev.status}${prev.tier ? `, ${prev.tier}` : ''}): ${prev.reason || 'unsupported'}`); return 4; }
+        provenFiles = prev.edits ? Object.fromEntries(prev.edits.map((e) => [e.file, e.after])) : { [prev.file]: prev.after };
+        const gate = await requireProofGate(args, scanRoot, f, last, provenFiles);
+        if (!gate.ok) return gate.code;
+      }
       const res = await languageFixPreview(f, proj.files, { apply: isApply, root: scanRoot });
+      if (provenFiles && res.ok) {
+        const applied = res.edits ? Object.fromEntries(res.edits.map((e) => [e.file, e.after])) : { [res.file]: res.after };
+        if (JSON.stringify(Object.entries(applied).sort()) !== JSON.stringify(Object.entries(provenFiles).sort())) { console.error('The patch written differs from the patch that was proof-verified. Run `agentic-security undo` and report this.'); return 1; }
+      }
       if (!res.ok) { console.error(`No verified fix for this finding (${res.status}${res.tier ? `, ${res.tier}` : ''}): ${res.reason || 'unsupported'}`); return 4; }
       console.log(`${res.label || 'FIX'}${res.tier ? ` (${res.tier})` : ''}: ${res.explanation || ''}`.trim());
       console.log(res.diff);
@@ -3740,23 +3780,8 @@ async function cmdFix(args) {
   // --require-proof: the patch is applied ONLY if the original exploit is confirmed beforehand and demonstrably blocked
   // afterwards (posture/proof-fabric/fix-proof.js). Missing reproducer, unavailable environment or an incomplete replay all refuse.
   if (args.flags['require-proof']) {
-    const { resolveReproducer } = await import('../src/posture/proof-fabric/cli.js');
-    const { findSiblings } = await import('../src/posture/proof-fabric/siblings.js');
-    const { verifyFixWithProof, formatFixProof } = await import('../src/posture/proof-fabric/fix-proof.js');
-    const { verifyFix } = await import('../src/posture/fix-verify.js');
-    const loaded = resolveReproducer(scanRoot, f, args.flags.reproducer);
-    if (!loaded.reproducer) { console.error(`--require-proof: no reproducer for ${f.id}, and none could be generated: ${loaded.reason}. Refusing to apply; nothing was executed. Supply --reproducer <file>.`); return 4; }
-    if (loaded.generated) console.error(`--require-proof: using a reproducer generated from the finding (${loaded.generated.shape}, ${loaded.generated.class}); it counts only if executing it confirms the exploit.`);
-    const files = { [f.file]: newContent };
-    const staticVerification = await verifyFix({ scanRoot, originalFindingStableId: f.stableId || null, files, recordMetrics: false });
-    // equivalent sibling exposures: the engine's own other findings of this kind, plus other places with the same code shape
-    const sib = findSiblings({ root: scanRoot, finding: f, scanFindings: last.findings || [] });
-    const siblings = sib.siblings.map((x) => ({ id: x.id ?? null, file: x.file, line: x.line, source: x.source }));
-    if (sib.truncated.length) console.error(`--require-proof: the sibling search stopped early (${sib.truncated.join(', ')}); absence of siblings is not shown.`);
-    const fp = await verifyFixWithProof({ finding: f, reproducer: loaded.reproducer, files, scanRoot, staticVerification, siblings, siblingSearchTruncated: sib.truncated });
-    console.log(formatFixProof(fp));
-    try { (await import('../src/posture/dnr/store.js')).saveRecord(scanRoot, f.stableId || f.id, 'fix', fp); } catch { /* recording is best effort; the refusal below does not depend on it */ }
-    if (!fp.verified) { console.error('\nRefusing to apply: the fix is not proof-verified (--require-proof).'); return 4; }
+    const gate = await requireProofGate(args, scanRoot, f, last, { [f.file]: newContent });
+    if (!gate.ok) return gate.code;
   }
   let result = await applyVerifiedFix({
     scanRoot,