@clear-capabilities/agentic-security-scanner 0.151.3 → 0.153.0

package/CHANGELOG.md

@@ -9,6 +9,131 @@
 > make the history less accurate, not more.
 
 
+## 0.153.0 - Taint-engine points-to precision, and two default-adjacent capabilities rebuilt from broken to sound
+
+Four taint-engine capabilities landed since 0.152.0, continuing the "next-generation taint analysis
+capabilities" initiative: two narrow points-to-precision fixes for JS/TS class-hierarchy resolution,
+and two substantial rebuilds of capabilities found to be structurally non-functional on real data —
+one opt-in (SMT path feasibility), one running by default (cross-service schema-driven taint).
+
+- **Cross-call-site parameter type inference for CHA** (`dataflow/class-hierarchy.js`): JS/TS gains
+  the same "driver constructs a concrete instance, passes it as a parameter to a helper that invokes
+  a virtual method" resolution Java/C# already had via declared parameter types — a narrow,
+  well-scoped slice of Andersen-style points-to analysis, not the full thing. A call site's literal
+  `new ClassName(...)` argument, or an identifier whose type was already established, now propagates
+  onto the callee's parameter; two call sites disagreeing on the class permanently refuse to resolve
+  it rather than guess.
+- **Factory-function return-type inference for CHA**: the sibling gap on the RETURN side —
+  `function makeHelper() { return new Helper(); } const h = makeHelper();` — closed with the same
+  "never guess" discipline: a function only qualifies when every reachable `return` is a `new
+  ClassName(...)` of the same class.
+- **SMT/exploit-proof path feasibility, rebuilt from the ground up** (`dataflow/exploit-prover.js`,
+  `dataflow/backward.js`): the pre-existing SMT-lite infeasibility check was fed a flat list of
+  taint sources reaching a sink, never a real sequential source-to-sink path, so its own
+  (correctly-written) metacharacter-exclusion logic had nothing genuine to walk — confirmed by
+  direct code reading, and by its own tests only ever exercising hand-fabricated fixtures the real
+  engine never produces. A second, parallel, independently-broken module (`smt-feasibility.js`, also
+  opt-in) had the identical defect plus a direct `severity` mutation that violated this codebase's
+  recall-preserving-demotion convention; retired outright rather than fixed twice. Three real engine
+  bugs were found and fixed to make the rebuild work on real data: `sliceBackward` couldn't trace
+  through a call-shaped sanitizer RHS, `engine.js`'s own finding-normalization allowlist silently
+  dropped `argIndex`, and a compound (string-concatenation) sink argument couldn't be resolved. The
+  rebuilt check now refuses to conclude safety the moment an opaque, unmodeled call sits between a
+  trusted sanitizer and a sink, verified via direct adversarial reproduction. `z3-solver` is added as
+  a genuine `optionalDependencies` entry, with its current (load-only, not yet query-discharging)
+  scope documented honestly rather than overclaimed. Opt-in via `AGENTIC_SECURITY_SYMEXEC=1`.
+- **Cross-service schema-driven taint, rebuilt from a complete no-op** (`dataflow/cross-service-taint.js`):
+  this module runs BY DEFAULT (opt-out via `AGENTIC_SECURITY_NO_CROSS_SERVICE=1`) but was found, via
+  direct reproduction, to have never worked at all for real findings — an HTTP path-template matcher
+  that failed on the module's own documented example (a `:param` vs. `{param}` placeholder-syntax
+  mismatch), a field-name correlation check that read finding properties (`source.snippet`/
+  `source.expr`) that do not exist on any real finding, and a report-layer allowlist that silently
+  dropped the annotation even when it did fire. Rebuilt with structural path-template matching and
+  provenance-based correlation for HTTP transport (a real, catalog-backed signal, since a real
+  finding never retains the specific field name that reached a sink) with a best-effort text-match
+  fallback for other transports; the report-layer allowlist fix means the annotation now actually
+  reaches SARIF/JSON/HTML output. A genuine remaining scope limitation — no mechanism seeds a NEW
+  taint source from a `consumes` declaration — is documented in the module's own header rather than
+  left implicit.
+- **Measured**: both rebuilds verified via full real-corpus TEST-split runs for all three
+  SARD/Juliet languages, byte-identical to the 0.152.0 baseline in every case (Java macroF1=59.1%,
+  C# macroF1=68.6%, PHP macroF1=48.2%) — neither the SARD fixtures nor the CVE-replay/self-scan/
+  mutation/layer-recall corpora exercise these specific capabilities, so this release ships real,
+  adversarially-tested engine correctness with no corpus-visible movement, consistent with this
+  project's own established precedent for capability work independent of any one benchmark.
+
+
+## 0.152.0 - Rust: first-class language, and the SARD 80% F1 push's taint-engine architecture work
+
+Rust joins the first-class language set (JS/TS, Python, Java, Kotlin, Go, Ruby, PHP, C#) with both
+a hand-rolled IR frontend (`scanner/src/ir/parser-rust.js`, the `parser-go.js`/`parser-cs.js`
+pattern) feeding the Layer-2 taint engine, and dedicated SAST rules — Rust previously had only the
+opt-in, WASM-based tree-sitter long-tail path, which nothing in the taint engine ever consumed.
+
+- **Taint coverage** (`dataflow/catalog.js`, ~90 new source/sink/sanitizer entries): SQL injection
+  (sqlx/diesel/rusqlite/postgres, distinguishing the compile-checked `query!` macro and `.bind()`
+  parameterization from `format!`-built SQL text), command injection (`Command::new("sh").arg("-c")`
+  vs. safe argv-array execution), path traversal (`std::fs`/`tokio::fs`, recognizing a
+  canonicalize+`starts_with` containment guard), SSRF (reqwest/ureq), reflected XSS
+  (actix/axum/rocket/warp response bodies), open redirect, log injection, insecure deserialization,
+  and uncontrolled resource consumption. Sources include axum/actix/rocket extractor-typed
+  parameters (`Query`/`Path`/`Json`/`Form`/`HeaderMap`) via a new `paramAnnotations` side-channel
+  and route-attributed handlers (`#[get("/<x>")]`).
+- **SAST rules** (`sast/rust.js`): TLS verification disabled, weak hash/cipher (MD5/SHA-1/DES/RC4),
+  weak randomness feeding a token/secret identifier, and unsafe raw-memory operations
+  (`transmute`/`from_raw_parts`/`get_unchecked`) — alongside the pre-existing sqlx-format/shell-form
+  command-injection/zero-seed-RNG/unsafe-block regex rules.
+- **A real parser design lesson, not specific to Rust's own correctness**: `.unwrap()`/`.expect(msg)`
+  chained onto a sink call shift the chain's terminal segment away from the sink's own name (the same
+  "terminal segment shift" defect class already documented for Go/Java/Kotlin) — nearly every
+  real-world async Rust call site ends this way, so they are popped from the parsed call chain
+  rather than merely treated as taint-transparent.
+- **Measured**: `bench/layer-recall` — 5/5 (100%) whole-corpus, 3/3 (100%) deep-tier on first
+  measurement (see `docs/METRICS.md`; the corpus is small today, so this reflects the corpus's own
+  composition, not general ecosystem coverage). `bench/cve-replay` — 5 new entries (2 capability,
+  3 deep), all `pre:TP post:TN`. `bench/mutation` — 1 new detection-dimension baseline/adversarial
+  pair (`format!`-built SQL fires; the parameterized `.bind()` form does not).
+
+### SARD/Juliet 80% macro-F1 push (`bench/sard/EXECUTION_STATUS.md`) — substantial progress, DoD not yet met
+
+A long-running execution PRD targeting macro-F1 >= 80% on the held-out TEST split for Java, C#, and
+PHP against the SARD/Juliet benchmark corpus. **Honest status: real, verified, substantial gains for
+all three languages, but the 80% target is not yet reached** — Java macroF1=59.1%, C#
+macroF1=68.6% (clears the 50% M2 gate), PHP macroF1=48.2% (all measured on TEST split, never on
+gold/answer-key signals — see the PRD's own integrity rules). This release ships the engine and
+scoring work; the PRD stays open.
+
+- **Measurement integrity first**: fixed truncation-masking in the benchmark harness (a truncated
+  scan was silently scoring as if complete), added per-CWE batched scanning to eliminate it in
+  practice, and made the scan surface match the gold surface exactly.
+- **Structural class resolution**: real `ir.classes` emission and class-hierarchy-based method
+  resolution replacing regex heuristics, across the languages that lacked it.
+- **Interprocedural completeness**: cross-file field/chain taint for C# (`csharp-cross-file.js`) and
+  Java, collection-element taint (typed reads, for-each binding), abstract/interface dispatch via
+  declared base type, and real branch+join CFGs for Java `if`/`switch` (previously linear
+  fall-through — a real correctness bug, not just a recall gap).
+- **Precision**: guard-predicate narrowing (`is_numeric`/`ctype_digit`/`int.TryParse`/etc. on an
+  if-statement's TRUE branch only), a proven-clean ledger, and dozens of per-CWE false-positive
+  fixes across Java/C# (many rooted in Juliet's own "dead code by construction" idioms —
+  `if(true)`/`if(5==5)`/constant-folded fields/methods — defeating naive literal-tracking heuristics).
+- **Call-string context-sensitivity, built from scratch this push**: `dataflow/engine.js` now
+  narrows a shared static/class field's taint to only the callers that can actually reach it
+  (previously any writer anywhere in the class tainted every reader), closing a real
+  context-sensitivity gap independently identified across all three languages. A second,
+  previously-built-but-never-validated k=1 call-string mechanism (`AGENTIC_SECURITY_KCFA_CALLSTRING`,
+  opt-in) was found broken, root-caused, and repaired.
+- **Two new "next-generation taint analysis" capabilities**, requested and scoped independently of
+  the SARD benchmark itself: (1) sound regex-validation-guard narrowing for JS/TS — a
+  fully-anchored pattern like `/^[a-zA-Z0-9]+$/` used as `if (pattern.test(x))` now provably proves
+  `x` metacharacter-free on the true branch, via a new conservative character-class parser
+  (`dataflow/string-domain.js`'s `isSafeValidationPattern`); (2) object-instance field taint via
+  method calls, a general (not JS-specific) engine capability — `bad.setData(tainted)` now taints
+  `bad`'s own field, read back correctly through `bad.getData()`, while a **different** instance of
+  the same class stays untainted (real object-instance sensitivity, without a full
+  allocation-site-keyed heap model).
+- PHP: closed total blackouts for LFI/RFI (`include`/`require` result assignment), several IDOR
+  sub-shapes, and a cross-file variable-scope-merging capability for split include chains.
+
 ## 0.151.3 - Adversarial premortem on the SARD benchmarking PROGRAM itself: full remediation, first genuine held-out numbers
 
 A structured adversarial premortem ("assume this has completely failed six months from now — work