@clear-capabilities/agentic-security-scanner 0.128.1 → 0.132.0

package/CHANGELOG.md

@@ -1,5 +1,228 @@
 # Changelog
 
+## 0.132.0 — a proven exploit becomes a permanent regression test
+
+The corpus stops being only a regression net and starts being fed by the engine
+itself. Every claim below was verified by a command in the session that made it.
+
+- **Execution-proven findings auto-enrol as corpus entries.** This was R2's
+  differentiator and the last missing piece of it. A finding whose
+  proof-of-concept RAN in the sandbox and produced the predicted effect can now
+  be turned into a permanent `pre:TP post:TN` corpus entry, so every exploit
+  proved once is defended against its own regression forever. Demonstrated end
+  to end rather than in a unit test: a real command-injection finding was proved
+  in the sandbox, its fix supplied the `post/` tree, and the entry was enrolled
+  and scored by the real gate — **the baseline moved 199 → 200**. Proven in the
+  other direction too: with the entry's `pre/` neutered the gate reports
+  `REGRESSED (1)` and exits non-zero.
+- **Nothing reaches the corpus unscored.** The entry is built in a temporary
+  directory, `pre/` and `post/` are scanned, and it is moved into the corpus
+  only on `pre:TP post:TN`. There is no force flag, and the scoring function is
+  unexported so no caller can score by one route and write by another — the
+  v0.106.0 mistake (fixtures committed without verifying they score) is the one
+  thing an automated writer must never industrialise. Scoring itself now lives
+  in one module shared by the enroller and the corpus runner, so the two cannot
+  drift apart; the refactor was proven behaviour-preserving at 199/199, no drift.
+  New entries land in `capability/`, never the CI-gated `regression/` tier — a
+  machine must not decide what blocks everyone's build.
+- **`verify_fix` now runs the proof-of-concept against the candidate patch.** A
+  re-scan only proves the DETECTOR stopped firing, which a cosmetic edit
+  achieves; re-running the exploit proves the hole is shut. Still-exploitable
+  after the patch is a hard failure. Deliberately asymmetric: a PoC that could
+  not run is recorded `inconclusive` and excluded from the verdict, because
+  reading "could not prove it" as "fixed" is exactly the false confidence this
+  leg exists to prevent. Both directions were executed against the real sandbox.
+- **Time-to-validated-fix is now measured and reported**, closing R5. Every
+  verification stage is timed and appended to a per-project log, and the
+  reported distribution is deliberately hard to flatter: failed attempts never
+  enter the validated median (they short-circuit, so blending them makes a worse
+  pipeline look faster), "no test suite to run" is bucketed apart from "tests
+  passed", and per-stage timings come from validated runs only. Percentiles are
+  nearest-rank — every figure shown is a duration some run actually took — and
+  are flagged unreliable below n=10 rather than quoted as settled.
+- **Cross-machine determinism now has a gate behind it.** A dependency-free
+  fixture is scanned on two operating systems and the run-attestation digests
+  must match. The comparator refuses every route to a meaningless pass: fewer
+  than two attestations, two runs from the same platform, a zero-finding digest,
+  mismatched canonicalisations, unparseable input. Each refusal was fired
+  deliberately and confirmed to exit non-zero.
+
+Honest limits, stated rather than implied:
+
+- The enrolment loop is **not yet automatic end to end**. Nothing in the scan
+  pipeline attaches a proof-of-concept to a finding or promotes proof tiers, so
+  a scan never produces an `execution-proven` finding on its own; PoCs come from
+  the generator and are proved at enrol time. Automatic attachment during a scan
+  is the remaining work.
+- The cross-machine determinism jobs have **not yet run in CI**, so no second
+  machine has actually been compared. Same-machine repeatability is verified.
+  The attestation's own "does not prove" statement is unchanged and stays
+  correct either way: one attestation is one run on one machine.
+- The sandbox wall-clock timeout stops the **direct child, not the process
+  tree**, on both backends. No test asserts a backgrounded grandchild dies.
+
+Also in this release:
+
+- **Documentation corrected against evidence.** The roadmap and the sandbox and
+  posture guides still described the kernel-namespace backend as "implemented,
+  unverified" after CI had already proved otherwise; they now cite the run
+  (Ubuntu 24.04, kernel 6.17.0-1020-azure, 41 assertions, 0 failures, all eight
+  escape cases). The corpus entry count was three tiers and 14 entries out of
+  date.
+- **Dependency advisories cleared** in the extension tree and `@types/node`
+  brought current in both trees.
+
+## 0.131.0 — decorated files stop vanishing; the Linux sandbox is verified
+
+Two correctness fixes, both found by pushing measurement further than the last
+release did.
+
+- **Decorator syntax is now accepted by the IR frontend.** The parser was told
+  about TypeScript and JSX but not decorators, so it rejected the ENTIRE file on
+  the first `@tracked` it met — no error, no warning, just findings that never
+  existed. Measured on a live third-party target: **4,023 of 4,271 JavaScript
+  files parsed before, 4,262 after** — 239 files, roughly 6% of that project,
+  were invisible to the scanner and are now analysed. `decorators-legacy` plus
+  `decoratorAutoAccessors` was chosen over the modern `decorators` variant
+  because the modern one cannot parse TypeScript parameter decorators; it would
+  have swapped one blind spot for another. No new dependency. Guarded by a
+  regression test proven in both directions — 5/5 pass with the fix, 4 fail
+  without it.
+- **The kernel-namespace sandbox now confines writes, and is verified.** It
+  previously confined network only and had never been executed. It now enters a
+  private mount namespace, rebinds every mount read-only except the sandbox
+  root, and drops the entire capability set before exec so a payload cannot
+  rebind the tree writable. A CI job relaxes the host restriction on
+  unprivileged user namespaces and runs the real escape suite: a write outside
+  the root is blocked with no file created, and outbound egress is blocked. The
+  verifier exits non-zero unless that suite actually RAN, so a skip can never be
+  read as a pass. Execution-proof on Linux no longer rests on an unexercised
+  backend.
+
+Also in this release:
+
+- **A quadratic blowup in the root-cause sweep**, which ran on every full scan:
+  it re-split the whole corpus into lines once per finding and kept a record per
+  finding x matching line. The largest benchmark corpus went from dying after
+  55+ minutes to **exit 0, 933 MB peak, 330s** — smaller and roughly ten times
+  faster, with metrics identical before and after.
+- **Benchmark corpora are fetched by pinned commit** instead of guessing a clone
+  depth; six corpora had never been scoreable because their pin sat more than
+  100 commits back.
+- **A pre-push gate** runs bundle integrity, the full suite, the corpus baseline
+  and the precision baseline before anything leaves the machine, plus branch
+  protection requiring green checks to merge.
+- **A dependency-currency release gate**: any advisory at moderate or above
+  fails with no opt-out; anything behind latest fails unless explicitly held
+  with a stated reason and a review date that expires.
+- **The glob dependency was replaced by the platform built-in** — 92 to 73
+  production packages — after a differential over 320 trees and 1,419,229 paths
+  showed zero differences.
+
+`npm test` 2072/0; cve-replay 199/199; self-scan no drift; proof corpus
+ghost/superset/godot all 100% parse coverage.
+
+## 0.130.0 — the roadmap's first ten: provable security over orchestration parity
+
+A capability roadmap (`docs/ROADMAP.md`) plus its first ten items, derived from a
+survey of the current agentic security-review field. The strategic call: that
+field cannot state a false-positive rate or prove it did not regress, because its
+core is a model call. This project's core is a deterministic engine behind three
+gates, so the work doubles down on provable, measurable, reproducible.
+
+- **Execution sandbox** (`src/sandbox/`) — fail-closed confined execution. Verified
+  by execution: writes outside the sandbox root blocked, network blocked, wall-clock
+  overrun terminated, benign work still succeeds. With no confinement primitive
+  available, execution is REFUSED, never run unconfined.
+- **Execution-verified findings** (`posture/execution-proof.js`) — a finding can be
+  promoted to `execution-proven` by running its proof-of-concept in the sandbox.
+  Proof is a marker file, never an exit code, because the sandbox cannot reliably
+  distinguish a denied run from a clean exit. `proof-failed` is a triage signal,
+  NOT a false-positive verdict.
+- **Published accuracy scorecard** (`npm run scorecard`) — detection and correct-
+  silence rates sliced by language and CWE, every figure with its denominator,
+  regeneration-stable. F1 deliberately omitted: no labelled real-world population
+  exists to measure precision over, and the document says so.
+- **Determinism attestation** (`posture/attestation.js`) — an order-independent
+  signed digest. Cross-machine reproducibility is explicitly NOT claimed.
+- **Fixes must pass your tests** (`posture/test-runner.js`) — `verifyFix` previously
+  proved only that the finding disappeared, which deleting the feature also
+  achieves. It now runs the project's own suite, and says so when the suite ran
+  against unpatched code.
+- **Relevance ranking** (`posture/relevance.js`) — re-ranks by entry-point
+  reachability. Recall-preserving: nothing deleted, severity never touched,
+  `unreachable` only on positive evidence.
+- **Enforced verification separation** (`posture/verification-separation.js`) — a
+  verifier cannot rubber-stamp its own finding.
+- **Resumable scans** (`posture/scan-checkpoint.js`) — opt-in via
+  `AGENTIC_SECURITY_RESUME=1`, crash-safe, conservatively invalidated.
+- **Secret redaction** (`llm-validator/redact.js`) — credentials removed before any
+  code leaves the machine; ordinary code passes through byte-unchanged.
+
+Also: dependencies updated to latest (Babel 7→8 with the removed preset options
+migrated, js-yaml 4→5 with an empty-input shim). One dependency deliberately held
+back: bumping the grammar runtime silently drops all six long-tail language
+grammars, so it stays pinned and the reason is documented.
+
+`npm test` 1989/0; cve-replay 199/199; self-scan no drift.
+
+## 0.129.0 — closing two taint-engine recall gaps
+
+Two defects were found by execution on the merged tree, each silencing real findings across
+every supported language. Both are now fixed, measured, and gated.
+
+- **Sinks are matched on assignment right-hand sides** (`dataflow/engine.js`). The engine only
+  ever sink-matched in statement position, so `db.query(tainted)` was reported while
+  `const rows = db.query(tainted)` was silent — in every language. The sink-matching logic is
+  now extracted into shared helpers called from both `case 'call'` and `case 'assign'` rather
+  than duplicated, and the pre-existing statement-position path is unchanged (measured control:
+  `1/1` before and after; assignment position `0/0 -> 1/1`).
+- **`match.type:'global'` catalog entries are indexed and reachable** (`dataflow/catalog.js`).
+  All 10 global entries were unreachable from `matchSource()` — including `$_GET`/`$_POST`/
+  `$_REQUEST`, the canonical PHP taint sources, in a language that already had interprocedural
+  analysis. A new `GLOBAL_INDEX`, plus lookup-side sigil normalization (`_globalKey()`) so PHP's
+  `$` prefix matches sigil-free catalog keys, takes catalog reachability `0/10 -> 10/10` with
+  language scoping preserved.
+- **Seven self-scan false positives eliminated at source.** Raising sink recall exposed
+  pre-existing catalog imprecision: `py-yaml-load`/`py-pickle-load` matched bare callee `load`
+  with no receiver constraint, so ordinary `json.load(fh)` was flagged as unsafe deserialization.
+  Each finding was inspected individually and all seven were false positives; the entries are now
+  pinned to their receiver. **Nothing was baselined** — `bench/self-scan/BASELINE.json` is
+  unchanged and the gate is green on the source fix.
+- **New**: `bench/engine-recall` before/after harness (`npm run bench:engine-recall`) and
+  `bench/engine-recall/RESULTS.md`, the full measurement record including what the fixes cost.
+- **Corpus 197 -> 199**: two deep-tier entries, each verified missed-before / found-after against
+  its specific fix.
+
+Known trade, recorded rather than papered over: pinning the receiver drops `import yaml as y;
+y.load(f)` and `from yaml import load; load(x)`, which are now covered at no layer. A corpus guard
+was attempted and deliberately withheld because it would score `pre:TN`; closing it needs
+import-alias resolution in the Python IR. Separately, `10/10` is catalog reachability, not
+end-to-end recall — only PHP is proven end to end; Ruby's deep engine does not complete those
+flows (pre-existing). Both are documented in `RESULTS.md` §3 and §8.
+
+`npm test` 1854/0; cve-replay 199/199; self-scan no drift.
+
+## 0.128.2 — compliance attestation accuracy + quieter self-scans
+
+Two fixes surfaced while dogfooding the compliance flow on this repo:
+
+- **Fixed a compliance-attestation path bug** (`posture/auditor-walkthrough.js`). Three
+  evidence checks (`mcp-tools`, `security-fixer`, `pre-edit-bodyguard`) carried a literal
+  `.../` placeholder path that `path.join(scanRoot, STATE, '.../x')` could never resolve, so
+  they read **"not present" for every project** — falsely dragging OWASP LLM08/LLM09 (and any
+  framework mapping to those modules) to "manual/not-present". A `.../` sentinel now resolves
+  against the scan root itself. On a self-attestation this flips LLM09 → satisfied and makes
+  LLM08 honestly partial.
+- **Repo `ignorePaths` for meaningful self-scans** (`.agentic-security/rules.yml`). Added
+  `bench/**` and `scanner/test/fixtures/**` so a repo-root `/scan` no longer counts the ~600
+  intentionally-vulnerable benchmark corpora and test fixtures as findings. Safe: `rules.yml`
+  is loaded from the exact scan root only, so the cve-replay runner (which scans each
+  pre/post fixture as its own root) and the unit tests are unaffected — corpus gate stays
+  185/185.
+
+`npm test` 1695/0; cve-replay 185/185.
+
 ## 0.128.1 — patch dependency vulnerabilities (11 Dependabot alerts → 0)
 
 Security maintenance. Cleared all 11 open Dependabot alerts by updating the two lockfiles to

package/bin/agentic-security.js

@@ -28,7 +28,7 @@ import { syncTickets } from '../src/integrations/tickets.js';
 import { decide as decideNextAction, explain as explainDecision } from '../src/posture/router.js';
 import * as triage from '../src/posture/triage.js';
 import { buildSlackDigest, buildDiscordDigest, postWebhook, buildJiraIssue, buildPrComment, buildSiemEvent, loadIntegrationConfig } from '../src/integrations/index.js';
-import fg from 'fast-glob';
+import { globFiles } from '../src/util/glob.js';
 
 // last-scan.json integrity helpers — implementation in posture/integrity.js
 // so the MCP server tools can share verification.
@@ -274,6 +274,21 @@ function renderV3Blocks(scan, flags) {
 }
 
 // Always-on machine output (R2). Vibecoder gets JSON only; pro gets JSON+SARIF+CSV.
+// SHA-256 of the running bundle, read from the sidecar `npm run build` emits
+// NEXT TO the bundle. Running from source (bin/ + src/) yields 'unavailable'
+// rather than the checkout's dist hash: src and a previously-built dist can
+// disagree, and attesting a bundle that did not produce this run would be a
+// false claim.
+function _bundleSha() {
+  const here = path.dirname(new URL(import.meta.url).pathname);
+  try {
+    const raw = fs.readFileSync(path.join(here, 'agentic-security.mjs.sha256'), 'utf8').trim();
+    const m = /^([0-9a-f]{64})\b/.exec(raw);
+    if (m) return m[1];
+  } catch { /* not running from the bundle */ }
+  return 'unavailable';
+}
+
 async function writeMachineOutput(targetAbs, scan, meta, profile) {
   const stateDir = path.join(targetAbs, '.agentic-security');
   const { isSafeStateDir: _isSafe } = await import('../src/posture/state-dir.js');
@@ -468,7 +483,7 @@ async function cmdScan(args) {
   // fingerprint and tracking provenance via sources[].
   if (args.flags['ingest-sarif']) {
     const glob = args.flags['ingest-sarif'];
-    const paths = await fg(glob, { dot: false, onlyFiles: true });
+    const paths = await globFiles(glob);
     if (paths.length) {
       const r = ingestAndMerge(scan, paths);
       if (process.stderr.isTTY) process.stderr.write(`[ingest] merged ${r.merged} / added ${r.added} findings from ${paths.length} SARIF file(s)\n`);
@@ -553,6 +568,24 @@ async function cmdScan(args) {
   // Deterministic post-process: stable-sort findings + zero out timing.
   if (isDeterministic()) makeDeterministic(scan, meta);
 
+  // R4 — determinism as a contract. Bind the PUBLISHED finding set (the same
+  // normalization every report format emits) to the engine version, ruleset
+  // version and bundle hash that produced it, via an order-independent digest.
+  // Runs after every filter above so it attests what actually ships. Metadata
+  // only — a failure here must never fail a scan.
+  try {
+    const { computeRunAttestation } = await import('../src/posture/attestation.js');
+    const { effectiveVersion } = await import('../src/posture/ruleset-version.js');
+    scan.attestation = computeRunAttestation({
+      findings: normalizeFindings(scan),
+      engineVersion: PKG_VERSION,
+      rulesetVersion: effectiveVersion(targetAbs).version,
+      bundleSha: _bundleSha(),
+      root: targetAbs,
+      sign: true,
+    });
+  } catch { /* attestation is metadata; never fail a scan over it */ }
+
   // R2: Always emit machine-readable artifacts to .agentic-security/.
   await writeMachineOutput(targetAbs, scan, meta, profile);
 
@@ -622,6 +655,23 @@ async function cmdScan(args) {
           if (sla) process.stderr.write(`⏰ agentic-security: ${sla}\n`);
         }
       } catch { /* MTTR is best-effort — never block a scan write */ }
+
+      // R5 — report the observed time-to-validated-fix distribution from the
+      // fix attempts recorded by `verifyFix`. This is measurement, not
+      // estimation: it says nothing until fixes have actually been verified in
+      // this project, and it prints nothing when there is nothing measured.
+      // Skipped under --deterministic for the same reason MTTR is: the
+      // durations are wall-clock and would break byte-identical state.
+      try {
+        const { fixDurationReport, renderFixDurationSummary } = await import('../src/posture/fix-metrics.js');
+        const fixMetrics = fixDurationReport(path.resolve(target));
+        if (fixMetrics.attempts > 0) {
+          persistedScan.fixMetrics = fixMetrics;
+          const isJsonFmt = format === 'json' || format === 'sarif' || format === 'cyclonedx' || format === 'sbom' || format === 'spdx' || format === 'vex' || format === 'openvex' || format === 'pbom' || format === 'aibom';
+          const line = renderFixDurationSummary(fixMetrics);
+          if (!isJsonFmt && line) process.stderr.write(`🔧 agentic-security: ${line}\n`);
+        }
+      } catch { /* fix metrics are best-effort — never block a scan write */ }
     }
     // #22 — live-secret validation (opt-in, offline-degrading). Label each
     // detected secret live | dead | unknown via a read-only provider "whoami".

package/dist/11.index.js

@@ -21,8 +21,8 @@ var external_node_child_process_ = __webpack_require__(1421);
 var external_node_fs_ = __webpack_require__(3024);
 // EXTERNAL MODULE: external "node:path"
 var external_node_path_ = __webpack_require__(6760);
-// EXTERNAL MODULE: ./src/engine.js + 524 modules
-var engine = __webpack_require__(8215);
+// EXTERNAL MODULE: ./src/engine.js + 592 modules
+var engine = __webpack_require__(9408);
 ;// CONCATENATED MODULE: ./src/posture/fix-honesty-gate.js
 // Deterministic honesty gates on fix / finding output (#7).
 //