@clear-capabilities/agentic-security-scanner 0.143.0 → 0.144.0

package/CHANGELOG.md

@@ -11,6 +11,310 @@
 
 
 
+## 0.144.0 — Assurance hardening closes Epic E2, and an independent audit finds what "verified" missed
+
+The assurance-hardening PRD (`docs/implementation/assurance-hardening-*`)
+closes 72 of its 73 requirements this release. Epic E2 ("analyzer supervision
+and scan health" — the claim that a scan can say, honestly, whether it
+actually completed) is now fully closed, 8 of 8. The one requirement left
+open (FR-401's "types" element — real type inference over the Layer-1 IR) is
+a deliberate, documented scope decision, not an oversight: see its own
+section below.
+
+### An independent 16-agent audit found 4 real gaps in requirements this project had already marked "verified"
+
+Before continuing forward implementation, the session was asked to verify the
+PRD was truly complete. Sixteen independent agents, briefed only with each
+requirement's literal PRD text (not this project's own tracking files), each
+re-checked one previously-"verified" requirement against the real code. 12 of
+16 held up. 4 did not — all fixed and re-verified in the same pass:
+
+- **A broken scan could still say "Safe to deploy."** `scanHealth.status`
+  (added by an earlier requirement) was computed correctly but never read by
+  the actual human-facing verdict — a scan that hit an annotator exception or
+  timed out on files, with zero findings, still printed a clean bill of
+  health. `toShipVerdict` now checks `scanHealth.status` before it says
+  anything is safe.
+- **"Approvals, exceptions, and suppressions" only had identity verification
+  wired for one of the three nouns.** `posture/suppressions.js` never checked
+  a suppression's `justification_signed_by` against the approver registry a
+  sibling requirement had already built — a suppression could be filed under
+  anyone's name with no verification at all. Fixed by threading the same
+  registry through.
+- **A "privacy-preserving" feedback module persisted the exact thing it
+  promised not to.** A caller-supplied finding id — which, for a real
+  finding, embeds its file path — was stored verbatim instead of hashed.
+  Fixed by substituting the finding's stable id or a genuine SHA-256 hash.
+- **Two detectors were silently exempt from the fault-isolation work a prior
+  requirement claimed was complete for "every detector."** `scanWeb3Advanced`
+  and `scanK8sAdmission` were called directly instead of through the
+  isolating wrapper, so an exception thrown by either would silently discard
+  every OTHER detector's findings for that file. The original verification
+  had grepped for a call-site count that could not distinguish a wrapped call
+  from an unwrapped one sharing the same substring.
+
+Three of these four were the same failure class repeating: a mechanism proven
+correct in a unit test, never checked against the real, human-facing (or
+caller-facing) entry point it was supposed to protect. A fifth requirement
+(the privacy IR adapter, below) was found to be a genuine partial — 2 of its
+8 named elements were real, the rest were stubs or silently merged — and was
+honestly downgraded rather than left mismarked.
+
+### Coverage ledgers: every analyzer now has one real, computed status per file
+
+`pipeline/coverage-ledger.js` is a new, drift-guarded registry of all 121
+real per-file analyzer call sites (extracted from the engine's own cascade
+code, not hand-maintained), each tagged always-applicable,
+extension-gated, or policy-gated. Combined with the fault-isolation and
+per-file-timeout work from earlier requirements, this computes exactly one
+terminal status — completed, failed, timed out, or skipped by policy — for
+every (file, analyzer) pair a scan actually ran. `scanHealth.analyzers`,
+previously a hardcoded `null`, is now this real summary; a detector that
+throws on even one file now demotes scan status, a gap that had no signal
+before.
+
+### `--assurance advisory|standard|strict`
+
+A new `agentic-security ci` flag, running alongside the existing `--fail-on`
+and `--policy` gates rather than replacing either. `strict` fails the build
+outright when `scanHealth.status` is not `complete` — a failed, timed-out, or
+silently policy-skipped analyzer; an annotator exception; a stale
+vulnerability feed (see below); or a CI environment that silently downgraded
+deep analysis (see below) all now have one build-failing consequence, in
+addition to whatever `--fail-on`'s severity threshold already does.
+`advisory` and `standard` report the same signal but never gate — the PRD's
+own acceptance criterion only specifies strict mode's behavior in full.
+
+### CI can no longer silently downgrade deep analysis without saying so
+
+Requesting `--deep` in a CI environment without the explicit
+`AGENTIC_SECURITY_DEEP_IN_CI=1` override already fell back to pattern-only
+analysis; what was missing was that decision showing up anywhere a human or
+a build gate would see it. It's now a real `scanHealth` condition, visible in
+the human headline (never "Safe to deploy" under a silent downgrade), in
+`ci`'s own stderr, and failable under `--assurance strict`.
+
+### Stale vulnerability feeds, calibration data, and rulesets are now visible — and can fail strict policy
+
+Four independent freshness signals, each reusing an already-computed
+staleness check rather than inventing a new one, now feed `scanHealth`:
+
+- The CISA KEV catalog's existing staleness tracking is wired through for
+  the first time.
+- The engine's own EPSS enrichment path (a separate, duplicate
+  implementation from `posture/epss.js`'s — a pre-existing duplication, not
+  introduced here) gained the same kind of per-entry age tracking KEV
+  already had.
+- The seed calibration table (`calibration-seed.json`) now carries a real,
+  git-derived generation timestamp and is checked against a 180-day
+  threshold.
+- Custom rule packs (`.agentic-security/rules/*.yml`) can now opt into a
+  `review-interval-days` / `reviewed-at` freshness check — deliberately not
+  based on file mtime, since a CI checkout resets mtimes on every run,
+  which would make an mtime-based check silently blind in CI, the one place
+  a strict-mode failure matters most.
+
+Found while wiring this: this development machine's own, real, previously
+invisible EPSS disk cache had a genuine stale entry — a live demonstration
+of exactly the gap this requirement closes, not a synthetic test case.
+
+### Checkpointed scans now invalidate only what actually changed
+
+Interrupted-scan resume (`AGENTIC_SECURITY_RESUME=1`) previously discarded an
+entire checkpoint — every already-completed file's work — if a single
+scanned file changed, was added, or was removed. Run identity is now split
+into a GLOBAL component (engine version, ruleset version, running bundle,
+dependency manifests, environment switches — a change to any of these still
+discards everything, because those affect how every file would be analyzed)
+and a PER-FILE content hash on each checkpoint record. Changing one file now
+re-analyzes only that file; every other already-checkpointed file still
+resumes. A global discard now names what changed
+(`"engine version changed (0.143.0 -> 0.144.0)"`); a per-file invalidation
+names why (`"content changed since it was checkpointed"`).
+
+### The privacy IR adapter now supplies 7 of its 8 named elements — "types" is a deliberate exception, not a gap
+
+An earlier requirement claimed the privacy taint engine's IR adapter
+supplied "declarations, types, assignments, calls, parameters, returns,
+storage, and sinks." An independent check found only 2–3 of the 8 were
+real: types was a hardcoded `null`, assignments and parameters were merged
+into one undifferentiated bucket, returns and storage were absent
+entirely. That requirement was honestly downgraded rather than left
+mismarked. This release closes parameters, assignments (now distinctly
+tagged), returns, and storage — the last of these by reindexing a
+stored-taint registry the engine already computes for an unrelated
+correlation, not by adding new detection logic.
+
+"Types" remains unimplemented, on purpose. A narrow, TypeScript-only signal
+is technically possible — type-annotation AST nodes survive into this
+codebase's own Babel visitor before TypeScript's own preset strips them —
+but supplying it means modifying the shared JS/TS parser every SAST, taint,
+and privacy consumer depends on, for a benefit that would only ever cover
+one of roughly ten supported languages. That trade was judged not worth the
+risk to already-correct, heavily-relied-on infrastructure, and is recorded
+as a deliberate scope decision rather than attempted as an unproven partial
+fix.
+
+### A real CI-only test failure, caught by hosted CI rather than the local gate
+
+Two `privacy-ir-adapter.test.js` tests requested deep analysis via the
+`{deep: true}` runScan option and asserted `scanHealth.deepAnalysis.enabled
+=== true`. That held locally but failed on the hosted CI runner: the engine
+downgrades deep mode under `CI=true` unless a second opt-in
+(`AGENTIC_SECURITY_DEEP_IN_CI`) is also set, and this codebase's own
+`test:ci-parity` static checker — built specifically to catch this class of
+bug after a near-identical 2026-08-19 incident — incorrectly treats the
+`{deep:true}` OPTION shape as exempt from needing that opt-in ("does not go
+through the env gate," per its own comment). That reasoning is wrong: the
+option and the environment variable are two different ways to set the same
+internal `_deepRequested` flag, and both go through the identical
+`_inCi`/`_deepInCiAllowed` gate afterward. Fixed by passing `deepInCi: true`
+explicitly on both tests. The checker's incorrect exemption for the
+`{deep:true}` shape is a known, separately-tracked gap — roughly forty other
+test files use that same option, and this incident does not establish which
+of them are actually exposed to it; auditing that is future work, not done
+inside this release.
+
+### npm publish no longer pays for the release gate twice
+
+`.github/workflows/release.yml`'s publish job ran the full uncached release
+gate as an explicit step, then called `npm publish` — which triggers
+scanner/package.json's `prepublishOnly`, itself `build && sync-changelog &&
+release-check.mjs`. Read from v0.143.0's actual workflow log: the explicit
+step took **280s**, and the one inside `npm publish` took **318s**, ten
+seconds later, on the same commit, same checkout, same runner. Caching could
+not have closed that gap by design — this workflow always passes `--no-cache`
+(a cross-machine cache would reintroduce the reproducibility claim
+`posture/attestation.js` explicitly declines to make), so the first run never
+wrote anything for the second to reuse. That is ~600s of a release entirely
+spent proving the same thing twice.
+
+`npm publish` now runs with `--ignore-scripts`, so `prepublishOnly` does not
+fire on the publish step. This is safe here specifically because it would not
+be safe in general: the three explicit steps immediately above it (build,
+changelog sync, gate) already did everything `prepublishOnly` would, so
+nothing it produces is missing from disk by the time `npm publish` runs. The
+local publish path is untouched — `scanner/package.json`'s `prepublishOnly`
+stays fully wired, because a local `npm publish` has no preceding gate step to
+make it redundant (root `CLAUDE.md`'s "Two publish paths" section still
+applies there unchanged). `test/release-workflow.test.js` pins the flag, pins
+that the three steps it stands in for still run first, and adds a tripwire: if
+`scanner/package.json` ever gains a `prepack`/`postpack` script,
+`--ignore-scripts` would silently skip it too, and that test fails until its
+effect is reproduced as an explicit step. Both new checks confirmed to fail
+when the fix is reverted, and the file is restored byte-identical after.
+
+### `npm test`: one `node --test` invocation instead of eleven
+
+`npm test` chained eleven separate `npm run test:<scope>` processes in series
+— `node --test` already runs a multi-file invocation's files CONCURRENTLY
+against the same cores, so eleven separate processes were paying eleven
+startup costs while getting zero overlap ACROSS scopes: `test:posture` could
+not start until `test:sast` had entirely finished. Repeated standalone timings
+on this machine put the old chain at roughly 258s and a single combined
+invocation over the same 397 files at 111-154s, run five times with an
+identical 3955/3955/0 (later 3964/3964/0, once this section's own tests were
+added) result every time — this machine has been under sustained load all
+session (`uptime` reports a load average over 5 on 8 cores while this was
+written), so the absolute local number is noisy; the authoritative figure is
+whatever the next hosted-CI run reports, on a dedicated runner. The mechanism
+is not in question: it is the same tests, run once instead of eleven times.
+
+`scripts/run-unit-tests.mjs` derives its file list from the existing
+`test:<scope>` scripts rather than hand-maintaining a second list — a
+duplicated list is exactly the shape that silently drifts (add a file to
+`test:sast`, forget to add it here, and the combined run quietly covers less
+than `npm run test:sast` alone does). It refuses to run if a `test:*` script
+using `node --test` is not accounted for in its `SCOPES` list, both as a hard
+failure from the script itself and as a named test
+(`test/run-unit-tests.test.js`) so the drift guard is visible in the suite,
+not just as a side effect nobody reads. `test/cpp-dataflow.test.js` and
+`test:python` are deliberately NOT folded in: cpp-dataflow sets its feature
+flag at module load rather than inside a test, and when included in the
+combined invocation its 26 tests silently contributed **zero** results to the
+totals — not a failure, not a skip, just absent — for a reason not chased to
+ground; python is a different runtime entirely. Both still run, as their own
+separate steps, exactly as before.
+
+`test/discovery-wiring.test.js`'s "a scope is wired into the full gate" check
+previously grepped the `test` script's text for the literal substring
+`test:discovery` — true when `npm test` was a literal chain, meaningless once
+it calls a derivation script instead (the substring never appears in the new
+`test` script whether or not discovery's files are covered). Rewritten to
+assert discovery's files are actually present in the union the runner
+computes, which is a stronger claim than the substring match ever was: it
+would have caught the runner's own extraction regex silently missing this
+scope's files, which the substring match could not have.
+
+### The JetBrains plugin builds again — and its support floor moved
+
+`jetbrains-plugin` had been red in CI, classified INFORMATIONAL, and treated as
+a known toolchain gap. It was not a toolchain gap. LSP4IJ dropped IntelliJ 233
+support at its 0.18.0 release, so the pinned `com.redhat.devtools.lsp4ij:0.19.4`
+could never resolve against the pinned IDE 2023.3.6:
+
+```
+Plugin 'com.redhat.devtools.lsp4ij:0.19.4' is not compatible to: IC-233.15026.9
+```
+
+There is no configuration that supports IntelliJ 2023.3 *and* a maintained
+LSP4IJ. **`sinceBuild` moves 233 → 242**: IntelliJ 2023.3 and 2024.1 are no
+longer supported by this plugin. That is LSP4IJ's floor, not a preference. The
+build also moves off `org.jetbrains.intellij` 1.17.4 — the superseded major,
+which Gradle 9 cannot apply at all — onto the IntelliJ Platform Gradle Plugin
+2.18.1, which removes the reason CI had to pin Gradle 8.10.
+
+`untilBuild` is now open rather than `251.*`. The old cap had already gone stale
+(2025.2 exists), and a stale cap reaches the user as "plugin incompatible" on an
+IDE that would have worked.
+
+**Two defects the passing build was hiding.** Once it compiled, `buildPlugin`
+exited 0 and produced a zip — and `verifyPluginProjectConfiguration` reported,
+in text nothing was reading, that (1) the plugin was compiled for Java 17
+against a platform requiring Java 21, and (2) the Kotlin stdlib was being
+double-bundled, putting a 1.7 MB `kotlin-stdlib-2.1.0.jar` in the distribution
+next to 5 KB of plugin code and leaving the platform's class loader free to
+resolve stdlib classes from either. Both are fixed; the distribution zip went
+from 1.59 MB to 4 KB. The CI job now **fails on that verifier's output**, since a
+warning nothing fails on is a warning nobody reads, and it unpacks the zip to
+confirm the `factoryClass` named in `plugin.xml` is actually in the jar — an
+empty zip passed the old check.
+
+**A defect introduced by this fix, caught before it shipped.** The new CI step
+pipes Gradle into `tee` so a later step can grep the verifier's output. A `run:`
+step's default shell is `bash -e {0}` — no `pipefail` — so the pipeline's exit
+status is *tee's*, and a failed build would have reported success. That is the
+same "green gate that verifies nothing" this entire change is about, reintroduced
+by the change itself. Both pipe-to-tee steps in the workflow now set
+`set -o pipefail`, including the pre-existing one in `determinism-attest`: that
+job is BLOCKING, and without pipefail it could upload an empty attestation —
+including on `attest-fixture.mjs`'s own zero-findings refusal — for
+`determinism-compare` to compare against.
+
+The guard added for it was itself broken twice, in opposite directions, and only
+running the negative control in both found them: it first matched the word
+`pipefail` inside the comment explaining the fix (so deleting the real line still
+passed), then matched `| tee` inside a comment (so a correct workflow failed). A
+guard fooled by its own documentation is worse than no guard, because it reads as
+coverage.
+
+**A committed Gradle wrapper.** `ide/jetbrains/README.md` told contributors to
+run `./gradlew` for a long time while no wrapper existed; CI pinned a Gradle
+version in the workflow instead, so the two could drift. The wrapper is now
+committed and pins the Gradle distribution by SHA-256 — verified to reject a
+tampered checksum, which `gradle wrapper` does not configure by default and
+which is not optional in this repository.
+
+**The classification stays INFORMATIONAL, and why.** Making a job that downloads
+a full IntelliJ distribution into a release blocker trades one failure mode for a
+worse one. The lesson taken instead: everything checkable without the network
+moves into the blocking offline gate. `test/ide-surfaces.test.js` now asserts
+that the JDK CI provisions equals the `jvmToolchain` the build asks for, that the
+committed wrapper exists and pins its distribution by checksum, that the README's
+stated support floor is the `sinceBuild` the artifact declares, and that an
+LSP4IJ ≥ 0.18 is never paired with a `sinceBuild` below 242. Each was confirmed
+to fail when its subject is broken.
+
 ## 0.143.0 — OSCAL output, and the finding an OSCAL document must refuse to make
 
 `--format oscal` was documented in `commands/compliance.md` long before anything