@clear-capabilities/agentic-security-scanner 0.184.0 → 0.186.1

package/CHANGELOG.md

@@ -9,6 +9,43 @@
 > make the history less accurate, not more.
 
 
+## 0.186.1 - Git history annotation blames once per file, which restores scan speed (fix)
+
+0.186.0 was tagged but never published: its release workflow stopped at the time-to-first-finding gate (15 to 18 s against a 13 s limit), so the first npm release carrying that work is 0.186.1. Nothing in 0.186.0's content changed apart from the fix below.
+
+- **Why it got slower.** Findings that used to carry no line number now carry one, so the git history step, which asks git who introduced each finding, ran for about twice as many findings. It ran one `git blame` per finding and one `git show` per commit, about 14 ms each. Measured on this repository's own source: 7.9 s on 0.185.0, 10.9 s on 0.186.0.
+- **The fix.** One `git blame` per file for all of its lines and one `git log --no-walk` for all commit messages. A file whose batch fails is blamed line by line, so a bad line number cannot lose the others. The annotations are the same; `test/git-history-batch.test.js` checks the batched answers against per-line `git blame`. Measured after the change: 8.3 s locally.
+- **Also in this release.** Two test fixes found by hosted CI: the artifact CLI tests now warm the OSV cache once, because a cold runner has none, and `owned-build/6` accepts a watchdog stop on a slow runner.
+
+## 0.186.0 - The graph-precision capabilities are reachable from the CLI, a z3 backend checks path feasibility, and demand queries and incremental analysis got faster (Preview)
+
+Everything new is Preview, opt-in and off by default. A default scan's output is unchanged by the new commands and by the solver. No GA capability changed its contract.
+
+- **Reachable from the CLI.** `dataflow view`, `dataflow contracts`, `dataflow sink-query`, `dataflow plan-cuts`, `witness build` and `witness verify` expose the typed graph views, boundary contracts, demand reverse taint, min-cut control planner and signed witness bundles. Every report carries a completion status (`complete`, `partial`, `cancelled`, `unavailable`, `invalid`) and `negativeClaimAllowed`; "no modeled path" is never reported as "no vulnerability". `witness build` exits 1 and writes nothing when no modeled path exists. The graph views other than the dataflow view and a tool in the MCP server remain library-only.
+- **A real z3 feasibility backend (`AGENTIC_SECURITY_LINEAGE_SMT=z3`).** Path conditions are translated to SMT-LIB and checked with a per-query timeout and memory cap. Only a proven `unsat` over a verified encoding can reduce a path's exploitability; a timeout, an unknown result, a missing solver or a condition the encoding cannot express changes nothing. The synchronous graph build does not call it: it is reached through the async path-evidence entry points. NaN and loose-equality coercion are not modeled.
+- **Demand queries are faster.** A query with the index built per query is 23.4x faster than full analysis on this repository (was 1.9x), and 3.7x averaged over a 30-query sample that includes fallbacks. Instance-field reads in JS and TS no longer force a full fallback; 75 of 982 sink sites still use the full path, all at the caller-depth limit. Findings equal full analysis on all 982 sites. Call-graph name resolution got a name index that also speeds up the cold engine.
+- **Incremental analysis is faster.** A per-file IR cache (JS and TS, under `AGENTIC_SECURITY_INCREMENTAL`) removes re-parsing. End to end it is now 4.3x on `scanner/src`; on the two synthetic projects it is 2.5x and 2.4x, still short of the 3x target. Projects above about 5,000 functions still run cold at the default summary budget.
+- **Fewer false positives on the labeled samples.** Flagged sites that were true rose from 5 of 25 to 5 of 8 for JavaScript and from 6 of 17 to 6 of 8 for Nix, with recall unchanged (5 of 5 and 6 of 6). Part of the JavaScript gain is a measurement correction: line-less findings were credited to every same-class label in their file. The rules are general (a literal origin prefix on a redirect, request-input-only reflection, module-load blocking calls, confined file serving, bounded Nix option types) and each has a metamorphic test; the held-out split was not independent, which the benchmark summary now says. The multi-service fixture still finds 3 of 8: the misses need the cross-service join in a normal scan and a queue-consumer source model.
+- **The state-machine heuristic is narrowed.** A `.status` write is compared only with status sets declared in the same file, and test files are skipped. Before, the first set found anywhere in a project was applied to every write.
+- **Fixed.** The lineage deadline tests no longer assume which of two valid cancellations wins on a slow runner, and the regression report no longer embeds the exact bundle size, so a rebuild cannot make it stale. `@babel/core` and `z3-solver` moved to their latest releases and `@azure/msal-node` is pinned past an advisory in the extension's dev tooling.
+- **Self-scan baseline.** The scanner's own source falls to 688 findings; the only increases are three low advisories in the two new files that read files from disk.
+- **Not claimed.** Reachability, satisfiability and runtime observation are not exploit proof. A signature on a witness bundle proves integrity, not that the transfer models are right.
+
+## 0.185.0 - Graph analysis says when it did not finish, and gains recursion handling, call-site correlation, demand queries, incremental invalidation, boundary contracts, control planning and replayable witnesses (Preview)
+
+All new behavior is Preview and is off by default or purely additive. No GA capability changed its contract.
+
+- **A graph analysis now says whether it finished.** The field analysis, the call-graph driver, source seeding and reverse selection return a completion record (`complete`, `partial` or `cancelled`) with reasons. A stopped analysis, a coarser fallback context, a construction limit or a cancellation shows in `graph.scanHealth`, `coverage.analysisCompletion`, the limitations list, the signed export and scan health, even when zero flows were projected. A query that did not finish never says "no path" (`negativeClaimAllowed: false`). Invalid query budgets (NaN, Infinity, negative, fractional, over the bound) return `status: "invalid"`. `AGENTIC_SECURITY_LINEAGE_DEADLINE_MS` and the `signal` and `deadline` options cancel a build; from the bundle the cooperative deadline applies and the watchdog for a worker that never stops is source-only.
+- **Recursion.** The call graph is condensed into strongly connected components, callees are solved first, and recursive groups iterate to a fixed point or report `partial`. Reconstruction walks one shared path, so a 20,000-hop chain went from 8.9 s to 22 ms and a 400,000-hop chain completes without a stack overflow.
+- **Call-site correlation (`correlationVersion: 2`).** Value versions and invocation identity keep two calls to one helper, sibling fields and overwritten values apart, with balanced call and return witnesses. Version 1 output is unchanged.
+- **Demand-driven reverse queries (`AGENTIC_SECURITY_DEMAND=1`).** A selector narrows analysis to the functions that can feed one sink argument and the forward engine corroborates. Parity with full analysis holds on 21 programs. The 2x target was missed on this repository's own source when the index is built per query (1.8x); with a warm index a query is about 277x faster. 10 of 30 own-tree queries fell back to full analysis.
+- **Path predicates and control credit (`pathEvidence: true`).** Branch conditions travel with each witness. A control earns credit only for its own sink family and value version, and an all-path claim is refused on any truncated sample.
+- **Dependency-complete incremental analysis (`AGENTIC_SECURITY_INCREMENTAL=1`).** Cached summaries are invalidated through the reverse-dependency closure, including deleted and renamed functions. Corrupt or interrupted caches fall back to a cold scan and say why. A key bug is fixed: summaries for qualified names (those containing `::`) were persisted under the file path, so reuse was effectively inert. The deep stage meets 3x on every project measured; end to end the median is 1.8x to 2.9x, and projects of about 5,000 functions or more run cold at the default summary budget.
+- **Boundary contracts, graph views, control planning and witness bundles.** HTTP, queue and storage contracts seed a consumer's parameter or record field (library entry point). Read-only graph views join findings, call sites, witnesses and flows. `dataflow plan-cuts` computes minimum cuts over the modeled flow network. Signed witness bundles replay offline, with four evidence statuses that are never merged. Witness replay succeeded on 64 of 64 bundles built from real hop streams, and all four tamper kinds tried were rejected 64 of 64.
+- **Measured, with the limits stated.** `bench/dag/` pins six real projects and a labeled multi-service fixture. Precision on the labeled JS and Nix samples is low (5 of 25 and 6 of 17 flagged sites were true), and the fixture finds 3 of 8 labeled positives. Impossible call and return witnesses: 6 of 65 under version 1 and 0 of 64 under version 2 on the fixtures. Labels were written after seeing scanner output and are not independent.
+- **Not claimed.** Reachability, satisfiability and runtime observation are not exploit proof. A signature on a witness bundle proves integrity, not that the transfer models are right. No real SMT solver ships. Boundary contracts, witness building, graph views and demand queries are not wired into a normal scan.
+- **Self-scan baseline.** The scanner's own source gained 18 findings from existing heuristics (14 from the state-machine heuristic, which applies the first declared status set in a project to every `.status` write). The baseline was re-recorded with that reason. Narrowing that heuristic is a follow-up.
+
 ## 0.184.0 - Runtime corroboration reaches GA on real exports, dependency and taint findings get contracts, and a scan says when it could not consult advisories
 
 Runtime corroboration is the first Observe capability to earn GA. The other promotions of the programme's remaining items are contracts and corrections, not new maturity labels, and the ones that stay held say why.

package/bin/agentic-security.js

@@ -203,6 +203,25 @@ Commands:
                                --filter <path.json>   {nodeIds,edgeIds} to scope the export (no-op + warning for csv/coverage)
                                --audience board|ciso|privacy|compliance|regulator|technical
                                                       (default: technical; briefing only — wording/verbosity, never facts)
+  dataflow plan-cuts [path] --candidates <file.json> --output <file> [--query <file.json>] [--format json|markdown]
+                               Plan a minimum-cost, hypothetical control placement (graph cut) over the
+                               already-scanned lineage graph. Never verifies a remediation.
+                               The plan carries a completion contract (never a negative claim).
+  dataflow view [path] [--node <id> [--direction in|out|both] [--relations a,b] [--origins a,b]] [--output <file>]
+                               Opt-in typed graph view over the already-scanned lineage graph: identity,
+                               completion, SCC condensation of the data-flow relation, or one node's neighbours.
+  dataflow contracts [path] [--contracts <file>] [--output <file>]
+                               Opt-in boundary contracts: bind declared HTTP/queue/storage edges to parsed
+                               consumer sites and seed taint there. A declaration is a modeled input only.
+  dataflow sink-query [path] (--list | --file <rel> --line <n>) [--mode demand|full|compare] [--arg-index <n>]
+                               [--sink-id <id>] [--access-path <p>] [--rule-family <f>] [--max-work <n>]
+                               [--max-caller-depth <n>] [--on-limit full|partial] [--output <file>]
+                               Opt-in demand-driven reverse taint for one sink site. Verdicts: flow-found,
+                               no-flow-in-modeled-scope, unknown. There is no "safe".
+  witness build [path] --out <dir> [--sink <id>] | --list-sinks
+  witness verify <bundle.json> [--public-key <path>] [--artifacts <dir> --root <project>] [--json]
+                               Opt-in signed, replayable path witness bundles (Ed25519, same key as attest).
+                               A valid signature proves integrity only, never that the finding is real.
   dataflow scenario apply [path] --operations <file.json> --output <file> [--format json|markdown]
                                Simulate a hypothetical architecture change against the
                                already-scanned lineage graph — never mutates the real scan.
@@ -3385,6 +3404,34 @@ async function cmdVerifyAttestation(args) {
     return 0;
   }
 
+  // DAG-10: a replayable path witness (schema: agentic-security/path-witness@1). The signature alone proves
+  // integrity; `--witness-artifacts <file.json>` ({graph, provenance, sources?, config?}) additionally replays
+  // the recorded path offline against the artifacts it names. Building a witness is a module API
+  // (posture/witness-bundle.js buildWitnessBundle): the hop stream it needs is not persisted by a scan.
+  const { WITNESS_SCHEMA, verifyWitness } = await import('../src/posture/witness-bundle.js');
+  if (bundle.schema === WITNESS_SCHEMA) {
+    let artifacts = null;
+    if (args.flags['witness-artifacts']) {
+      try { artifacts = JSON.parse(fs.readFileSync(path.resolve(args.flags['witness-artifacts']), 'utf8')); }
+      catch (e) { console.error(`Could not read witness artifacts: ${e.message}`); return 2; }
+    }
+    const w = verifyWitness(bundle, publicKeyPem, artifacts);
+    if (!w.signature.ok) { console.error(`✗ INVALID — ${w.signature.reason}`); return 1; }
+    if (w.replay.performed && !w.replay.ok) {
+      console.error('✗ INVALID — the signature holds but replay against the supplied artifacts failed:');
+      for (const x of w.replay.failures) console.error(`  - ${x.code}: ${x.detail}`);
+      return 1;
+    }
+    console.log(w.replay.performed
+      ? '✓ VALID — signature holds and the recorded path replays against the supplied artifacts.'
+      : '✓ SIGNATURE VALID (integrity only) — replay was not attempted; pass --witness-artifacts <file.json> to replay.');
+    console.log(`  completion: ${bundle.completion?.overall}   evidence: ${Object.entries(bundle.evidence || {}).map(([k, v]) => `${k}=${v.status}`).join('  ')}`);
+    console.log(`  proves:        ${w.statement.proves}`);
+    console.log(`  does NOT prove: ${w.statement.doesNotProve}`);
+    console.log(`  trust basis:   ${w.signature.trustBasis}`);
+    return 0;
+  }
+
   const r = verifyEvidenceBundle(bundle, publicKeyPem);
   if (!r.ok) {
     console.error(`✗ INVALID — ${r.reason}`);
@@ -4083,6 +4130,16 @@ async function cmdBoundaries(args) {
 // itself closes (idle-timeout auto-stop, or an external kill) — matching
 // every other cmdX(args) function's "resolves to an exit code" contract
 // while keeping the process alive for as long as the server is listening.
+// DAG-07 R07.2: show the graph's completion state wherever a graph is
+// loaded for display or export. Silent for a complete graph; for anything
+// else it says so on stderr, including when zero flows were projected.
+async function _printGraphCompletion(graph, who) {
+  try {
+    const { completionOfGraph, describeCompletion } = await import('../src/lineage/completion.js');
+    for (const line of describeCompletion(completionOfGraph(graph))) process.stderr.write(`${who}: ${line}\n`);
+  } catch { /* disclosure must never break the command */ }
+}
+
 async function cmdExplore(args) {
   // `explore . --runtime`: the runtime views are one static, script-free page, not a route on the hardened local server.
   if (args.flags.runtime) return cmdRuntimeExplorer(args, args._[1] || '.');
@@ -4095,6 +4152,7 @@ async function cmdExplore(args) {
     process.stderr.write(`agentic-security explore: ${loaded.message}\n`);
     return 1;
   }
+  await _printGraphCompletion(loaded.graph, 'agentic-security explore');
 
   let port = 0;
   if (args.flags.port !== undefined) {
@@ -4355,6 +4413,7 @@ async function cmdDataflowExport(args) {
     return 1;
   }
   const graph = loaded.graph;
+  await _printGraphCompletion(graph, 'agentic-security dataflow export');
   // Task-3 review finding (non-blocking, fixed): emitGraphDpiaArtifact's
   // own opts.generatedAt fallback is `new Date()` — omitting it here made
   // dpia/ropa the only two formats whose content reflects EXPORT-time
@@ -5094,6 +5153,52 @@ async function cmdDataflowScenarioApply(args) {
   return 0;
 }
 
+// agentic-security dataflow plan-cuts [path] --candidates <file.json>
+// --output <file> [--query <file.json>] [--format json|markdown] — DAG-09.
+// Plans a minimum-cost control placement over the already-scanned,
+// already-signed graph. HYPOTHETICAL: never writes remediation state or
+// the scan artifact. Exit codes: 0 success, 1 graph-load failure, 2
+// argument problem.
+async function cmdDataflowPlanCuts(args) {
+  const targetAbs = path.resolve(args._[2] || '.');
+  const candFlag = args.flags.candidates;
+  const outputPath = args.flags.output;
+  const format = args.flags.format || 'json';
+  if (!candFlag || typeof candFlag !== 'string') { process.stderr.write('agentic-security dataflow plan-cuts: --candidates <file> is required.\n'); return 2; }
+  if (!outputPath || typeof outputPath !== 'string') { process.stderr.write('agentic-security dataflow plan-cuts: --output <file> is required.\n'); return 2; }
+  if (format !== 'json' && format !== 'markdown') { process.stderr.write('agentic-security dataflow plan-cuts: --format must be one of json|markdown.\n'); return 2; }
+  let candidates; let query = {};
+  try {
+    candidates = JSON.parse(await fsp.readFile(path.resolve(candFlag), 'utf8'));
+    if (typeof args.flags.query === 'string') query = JSON.parse(await fsp.readFile(path.resolve(args.flags.query), 'utf8'));
+  } catch (e) { process.stderr.write(`agentic-security dataflow plan-cuts: could not read/parse input: ${e.message}\n`); return 2; }
+  if (!Array.isArray(candidates)) { process.stderr.write('agentic-security dataflow plan-cuts: --candidates must be a JSON array.\n'); return 2; }
+  const { loadSignedGraph } = await import('../src/server/graph-loader.js');
+  const loaded = loadSignedGraph(targetAbs);
+  if (!loaded.ok) { process.stderr.write(`agentic-security dataflow plan-cuts: ${loaded.message}\n`); return 1; }
+  const { planCuts } = await import('../src/lineage/cut-planner.js');
+  const plan = planCuts(loaded.graph, query, candidates);
+  // Completion contract: a plan is `complete` only for a finite cut over a graph known complete; never a negative claim.
+  plan.contract = (await import('../src/posture/dag-commands.js')).planContract(plan);
+  let data;
+  if (format === 'json') data = JSON.stringify(plan, null, 2);
+  else {
+    const lines = ['# Control placement plan (hypothetical)', '', `Status: ${plan.status}. Total coverage: ${plan.completeness.totalCoverage}. Not verified.`, '', '## Proposed controls', ''];
+    for (const c of plan.cut.controls) lines.push(`- ${c.location.type} \`${c.location.id}\` (${c.semantics}, owner ${c.owner}, cost ${c.cost})`);
+    lines.push('', '## Residual bypass paths', '');
+    for (const r of plan.validation?.residualPaths ?? []) lines.push(`- to \`${r.sink}\` via ${r.nodes.map((n) => `\`${n}\``).join(' -> ')}`);
+    lines.push('', '## Limitations', '');
+    for (const l of plan.limitations) lines.push(`- ${l}`);
+    lines.push('', `Completion: ${plan.contract.status}. A negative claim is never allowed for a hypothetical plan.`);
+    data = lines.join('\n') + '\n';
+  }
+  try {
+    await fsp.mkdir(path.dirname(path.resolve(outputPath)), { recursive: true });
+    await fsp.writeFile(path.resolve(outputPath), data);
+  } catch (e) { process.stderr.write(`agentic-security dataflow plan-cuts: could not write --output: ${e.message}\n`); return 2; }
+  return 0;
+}
+
 // agentic-security dataflow impact assess [path] --target <canonical-id>
 // --output <file> [--format json|markdown] — M5 deliverable #4
 // (FR-507). Loads the already-scanned, already-signed graph via
@@ -7926,6 +8031,7 @@ async function main() {
       }
       case 'boundaries': process.exit(await cmdBoundaries(args));
       case 'explore':  process.exit(await cmdExplore(args));
+      case 'witness':  process.exit(await (await import('../src/posture/dag-commands.js')).cmdWitness(args));
       case 'dataflow': {
         const sub = args._[1];
         if (sub === 'export') { process.exit(await cmdDataflowExport(args)); }
@@ -7966,6 +8072,10 @@ async function main() {
           }
           break;
         }
+        else if (sub === 'plan-cuts') { process.exit(await cmdDataflowPlanCuts(args)); }
+        else if (sub === 'view') { process.exit(await (await import('../src/posture/dag-commands.js')).cmdDataflowView(args)); }
+        else if (sub === 'contracts') { process.exit(await (await import('../src/posture/dag-commands.js')).cmdDataflowContracts(args)); }
+        else if (sub === 'sink-query') { process.exit(await (await import('../src/posture/dag-commands.js')).cmdDataflowSinkQuery(args)); }
         else if (sub === 'impact') {
           const impactSub = args._[2];
           if (impactSub === 'assess') { process.exit(await cmdDataflowImpactAssess(args)); }
@@ -7986,7 +8096,7 @@ async function main() {
           break;
         }
         else if (sub === 'twin') { process.exit(await cmdDataflowTwin(args)); }
-        process.stderr.write(`agentic-security dataflow: unknown subcommand "${sub}" — only "export", "diff", "watch", "scenario", "impact", "observations", and "twin" are supported.\n`);
+        process.stderr.write(`agentic-security dataflow: unknown subcommand "${sub}" — only "export", "diff", "watch", "scenario", "plan-cuts", "view", "contracts", "sink-query", "impact", "observations", and "twin" are supported.\n`);
         process.exit(2);
       }
       case 'governance': {