@clear-capabilities/agentic-security-scanner 0.137.1 → 0.139.0

package/src/ir/CLAUDE.md

@@ -7,15 +7,28 @@ consumed by `scanner/src/dataflow/` for taint analysis.
 
 | Language | Module                | Backend                                          |
 |----------|-----------------------|--------------------------------------------------|
-| JS / TS  | `parser-js.js`        | `@babel/parser`                                  |
+| JS / TS  | `parser-js.js`        | `@babel/parser`. **Taint-recall PRD (80%) Tier 3: JSX had ZERO IR modeling** — every `JSXElement` fell through `exprOf`'s switch to `{kind:'unknown'}`, so `return <div dangerouslySetInnerHTML={{__html: html}} />` (React's canonical XSS sink, and a real corpus miss) silently dropped `html`'s taint entirely, even though the plain-JS member-write form (`x.dangerouslySetInnerHTML = {...}`) was already cataloged. Deliberately narrow, not general JSX modeling: `exprOf`'s new `JSXElement` case (via `_findDangerouslySetInnerHTML`) extracts ONLY the `dangerouslySetInnerHTML` attribute's `__html` property value — found on the element itself or, recursively, any descendant (the attribute can sit on a nested element, not just the one directly returned) — and lowers it to a synthetic call (`__jsx_dangerously_set_inner_html__`), targeted by a sink of the same name (`react-jsx-dangerouslySetInnerHTML`) sibling to the existing member-write entry. Ordinary JSX children (`<div>{unsafeText}</div>`) are deliberately NOT modeled — React auto-escapes children by default, so that shape isn't itself a vulnerability the way `dangerouslySetInnerHTML` is. Guarded by `test/catalog-xss-p4.test.js`. |
 | Python   | `parser-py-cst.js`    | Python 3.8+ stdlib `ast` via subprocess (default when available) |
 | Python   | `parser-py.js`        | Hand-rolled regex parser (fallback when python3 missing) |
 | Java     | `parser-java.js`      | `java-parser` npm package (**async only** — the deep path in `engine.js` therefore awaits `buildProjectIRAsync` when any `.java` file is present, and uses the sync builder otherwise).
-⚠ Three defects made Java taint impossible until v0.136.3+: the sync-only call site; a CST walk looking for `blockStatement` on a `block` (java-parser nests `block → blockStatements → blockStatement`), which emptied every method CFG; and `exprFromCst` missing the `primary → primaryPrefix + primarySuffix` form that models **every** method call. Guarded by `test/java-taint-flow.test.js`. Real parameter names (previously always `params: []`, marked "deferred") plus Spring `@RequestParam`/`@PathVariable`/`@RequestBody`/`@RequestHeader` param annotations (`fn.paramAnnotations`) are extracted in one CST walk over `formalParameterList` (PRD R14(a) Task 5). Varargs parameters (`String... args`) live under a distinct `variableArityParameter` node this walk doesn't extract from — they're gracefully dropped, not corrupted or crashed on. **PRD R9 (partial): `fn.calls` is now populated** via the shared `call-sites.js#callSitesFromCfg` (the same language-agnostic helper `parser-py-cst.js` uses) — Java's CFG nodes (`call`, `assign`, `return`, `if`) already matched the documented contract. This creates real cross-file call-graph edges for Java (`callgraph.js`'s `edges`/`callersOf`/`resolveKnownCallee`, previously always empty for Java) — it does NOT change the generic tainted-call-argument fallback in `engine.js`'s `exprTaint`, which reads CFG expression args directly off `expr.args` and already worked for Java independent of `fn.calls`. However, **same-class (intra-class) method calls do not resolve, in both the bare and `this.`-qualified forms**: `parser-java.js` names functions `"App.buildCmd"` (class-qualified), but a bare call extracts `"buildCmd"` (unqualified) and `callgraph.js`'s name-based resolution cannot match them — this is the most idiomatic Java call shape (private helpers, intra-class delegation) and remains a real gap, documented here as a candidate follow-up PRD item (a per-file bare-tail fallback in `callgraph.js`, mirroring the existing `~bare~`-key collision-refusal pattern, would plausibly fix it without touching `parser-java.js`). `this.buildCmd(id)` is not merely unresolved, it's worse: `parser-java.js:76` lowers any `this.`-qualified call whose prefix isn't a plain FQN to the literal callee string `"unknown"`, so it doesn't fail closed, it fails to a fabricated name. Guarded by `test/parser-java-calls.test.js` (the test's honest caveat discloses this; the fixture shows that edges exist but unresolved) (mirrors `test/parser-rb-calls.test.js`, the identical `fn.calls` wiring, though Ruby's unqualified names make bare-call resolution work there). **PRD R8: `walkStmts` now recurses into `for`/`try`/`switch`/`do`/bare-block bodies** (previously only `if`/`while` were walked — every other braced statement kind silently dropped its body from the CFG, including try-with-resources, the single most idiomatic JDBC shape). A fix round closed two further gaps the initial review found: enhanced-for (`for (x : xs)`) now synthesizes an assign binding the loop variable to the iterated expression, so the variable itself carries taint provenance (mirroring `parser-js.js`'s `ForOfStatement` pattern); and Java 14+ arrow-form `switch` (`case 1 -> …`) is now recognized via a second CST-shape branch. Deferred, not fixed: `forInit`/`forUpdate` clauses of a basic 3-clause `for` loop are still not walked (only the loop body is); `synchronized` blocks and labeled statements are still fully dropped. Guarded by `test/parser-java-control-flow.test.js`. **Measured `bench/layer-recall` impact: unchanged, 1/25 before and after** — the fix is real and directly proven by the dedicated unit tests above, but this corpus's existing 25 Java fixtures happen not to place a sink genuinely inside a braced control-flow body (the ones with `if`/`try` syntax use it as a single-line guard clause ahead of a flat-level sink, not a nested one) — see the PRD R8 status entry for the full explanation and the same finding for C#. |
-| Ruby     | `parser-rb.js`        | Hand-rolled. **`DEF_RE` must not let `\s*` cross a newline** — it did, and the body slice then started after the method's first statement, silently dropping it from every method (a one-statement body became empty). Measured as Ruby 0/20 IR-TAINT recall in `bench/layer-recall`. Guarded by `test/parser-php-rb.test.js`. ⚠ Also emitted no `fn.calls` at all (every OTHER parser does) — `callgraph.js`'s edges/callersOf/resolveKnownCallee are built entirely from `fn.calls`, so this left dead-code demotion and any interprocedural signal that depends on real call-graph resolution (rather than engine.js's generic tainted-call-argument fallback) permanently blind to Ruby. Fixed by deriving `fn.calls` from the CFG via the shared `call-sites.js#callSitesFromCfg` (the same helper `parser-py-cst.js` uses) — Ruby's node shapes already matched its documented contract. Guarded by `test/parser-rb-calls.test.js`. |
-| PHP      | `parser-php.js`       | Regex-based, hand-rolled. ⚠ **PRD R8 was this codebase's hardest single task — a genuine 3-fix-round debugging saga, all substantially about line-number precision, not detection shape.** The core fix flushes the statement splitter on a closing `}` (previously only on `;`), and adds `try`/`switch` recognizers plus a recursion guard, so statements inside `if`/`while`/`foreach`/`try`/`switch` bodies are now real CFG nodes instead of being dropped or folded into a bogus call node — this alone also resolved a pre-existing bug where `if`/`while`/`foreach` bodies were already being mis-split even before R8 touched them. Round 1 fixed the naive `}`-flush breaking `if`/`else` and multi-clause `try` (via a continuation-keyword lookahead) and switch/case's first-statement drop (via a `:`-based flush, careful to exclude `::` so PHP 8.1 enum cases and `case Foo::BAR:` class-constant labels aren't false-positived) — but round 1's own line-tracking approach was then found wrong for comment-bearing bodies and multi-line headers, a regression the round itself introduced. Round 2 fixed that (comment-skip handlers were discarding newlines uncounted; use exact `_countNewlines`-based computation everywhere) but its own re-review found the overall "exact line" property still failed, due to a *different*, genuinely pre-existing bug in the function-body's own base-line computation (wrong for Allman-brace style, multi-line signatures, blank-line-preceded functions) plus a genuine new regression from round 1 (a comment between `}` and `else`/`catch`/`finally` dropped that continuation's body entirely). Round 3 fixed both and was confirmed clean by an 8-shape holistic sweep. Also fixed along the way: dead `finally` support (greedy regex capture bug), unrecognized `try{}finally{}` with no `catch`, and the `::` case-label false-positive — via a hand-rolled balanced-brace scanner replacing the fragile regex approach. **Known, deliberately deferred gaps** (full list + grouping in the PRD R8 status entry): the PHP 8 `match` expression is unmodeled (same class as Java's arrow-switch); `if`/`else`'s pre-existing greedy-capture bug still drops the else-body's first statement; a heredoc containing a bare `}` loses its sink entirely (a real regression from R8's original commit, not the fix rounds); `#`-style comments are invisible to the splitter (pre-existing, more exposed now that control-flow bodies are reachable); `_extractBody` is not comment-aware, so an apostrophe inside *any* `//` comment (e.g. "don't" — extremely common in real PHP) silently drops the **entire file's** IR (pre-existing, large, real — see the Kotlin row below for the same defect class); `elseif`/`else if` chains are still fully unsupported (pre-existing). Guarded by `test/parser-php-control-flow.test.js`. **Measured `bench/layer-recall` impact: unchanged, 1/23 before and after — the corpus's own `1/23 → 2/23` movement is real but belongs to R14(b) (PHP `<module>` top-level lowering), not this task**, confirmed by commit-swap A/B testing (the pre-R8 parser still reproduces 2/23; the pre-R14(b) parser reproduces only 1/23). None of this corpus's `pre/` PHP fixtures place a sink genuinely inside a braced control-flow body — same explanation as the other three languages' rows. |
+⚠ Three defects made Java taint impossible until v0.136.3+: the sync-only call site; a CST walk looking for `blockStatement` on a `block` (java-parser nests `block → blockStatements → blockStatement`), which emptied every method CFG; and `exprFromCst` missing the `primary → primaryPrefix + primarySuffix` form that models **every** method call. Guarded by `test/java-taint-flow.test.js`. Real parameter names (previously always `params: []`, marked "deferred") plus Spring `@RequestParam`/`@PathVariable`/`@RequestBody`/`@RequestHeader` param annotations (`fn.paramAnnotations`) are extracted in one CST walk over `formalParameterList` (PRD R14(a) Task 5). Varargs parameters (`String... args`) live under a distinct `variableArityParameter` node this walk doesn't extract from — they're gracefully dropped, not corrupted or crashed on. **PRD R9 (partial): `fn.calls` is now populated** via the shared `call-sites.js#callSitesFromCfg` (the same language-agnostic helper `parser-py-cst.js` uses) — Java's CFG nodes (`call`, `assign`, `return`, `if`) already matched the documented contract. This creates real cross-file call-graph edges for Java (`callgraph.js`'s `edges`/`callersOf`/`resolveKnownCallee`, previously always empty for Java) — it does NOT change the generic tainted-call-argument fallback in `engine.js`'s `exprTaint`, which reads CFG expression args directly off `expr.args` and already worked for Java independent of `fn.calls`. However, **same-class (intra-class) method calls do not resolve, in both the bare and `this.`-qualified forms**: `parser-java.js` names functions `"App.buildCmd"` (class-qualified), but a bare call extracts `"buildCmd"` (unqualified) and `callgraph.js`'s name-based resolution cannot match them — this is the most idiomatic Java call shape (private helpers, intra-class delegation) and remains a real gap, documented here as a candidate follow-up PRD item (a per-file bare-tail fallback in `callgraph.js`, mirroring the existing `~bare~`-key collision-refusal pattern, would plausibly fix it without touching `parser-java.js`). `this.buildCmd(id)` is not merely unresolved, it's worse: `parser-java.js:76` lowers any `this.`-qualified call whose prefix isn't a plain FQN to the literal callee string `"unknown"`, so it doesn't fail closed, it fails to a fabricated name. Guarded by `test/parser-java-calls.test.js` (the test's honest caveat discloses this; the fixture shows that edges exist but unresolved) (mirrors `test/parser-rb-calls.test.js`, the identical `fn.calls` wiring, though Ruby's unqualified names make bare-call resolution work there). **PRD R8: `walkStmts` now recurses into `for`/`try`/`switch`/`do`/bare-block bodies** (previously only `if`/`while` were walked — every other braced statement kind silently dropped its body from the CFG, including try-with-resources, the single most idiomatic JDBC shape). A fix round closed two further gaps the initial review found: enhanced-for (`for (x : xs)`) now synthesizes an assign binding the loop variable to the iterated expression, so the variable itself carries taint provenance (mirroring `parser-js.js`'s `ForOfStatement` pattern); and Java 14+ arrow-form `switch` (`case 1 -> …`) is now recognized via a second CST-shape branch. Deferred, not fixed: `forInit`/`forUpdate` clauses of a basic 3-clause `for` loop are still not walked (only the loop body is); `synchronized` blocks and labeled statements are still fully dropped. Guarded by `test/parser-java-control-flow.test.js`. **Measured `bench/layer-recall` impact: unchanged, 1/25 before and after** — the fix is real and directly proven by the dedicated unit tests above, but this corpus's existing 25 Java fixtures happen not to place a sink genuinely inside a braced control-flow body (the ones with `if`/`try` syntax use it as a single-line guard clause ahead of a flat-level sink, not a nested one) — see the PRD R8 status entry for the full explanation and the same finding for C#. **Taint-recall PRD (80%) Tier 4: chained-call CST fix.** `exprFromCst`'s `primaryPrefix` handling previously used `.find(Boolean)` to grab the FIRST `methodInvocationSuffix` in a `primary` node's `primarySuffix[]` array and discarded the rest — so any 2+-level fluent chain (`DocumentBuilderFactory.newInstance().newDocumentBuilder().parse(xml)`, `new SpelExpressionParser().parseExpression(expr).getValue()`) resolved to the wrong (inner, usually untainted) call, and the real sink call at the end of the chain was invisible to the taint walker. Rewritten to walk the full `primarySuffix[]` array in source order, dot-joining bare member-access segments (`{Dot, Identifier}` suffixes) into the callee name and accumulating args from every `methodInvocationSuffix` encountered — same outermost-first accumulation convention (`args.concat(soFar)`) as the `_followChain` helper shared by the other five hand-rolled parsers (this PRD's earlier, language-general fix for the identical class of bug: a trailing call/member continuation being silently dropped rather than followed). A companion catalog entry (`java-spel-getvalue`, receiver-scoped to `^parseExpression`, `argIndex: 'all'`) was added because the chain fix shifts the terminal callee segment from `parseExpression` to `getValue` — same "terminal segment shift" pattern documented for Kotlin/Go/Java elsewhere in this PRD. Fixes `CVE-2018-1000632-java-xxe` (DocumentBuilderFactory chain) and `CVE-2018-1273-java-code-injection` (SpelExpressionParser chain). Guarded by two new cases in `test/java-taint-flow.test.js`. **UPDATE (Tier 3 command-injection audit): the `_SHELL_META` gap this note originally flagged is fixed.** `CVE-2016-10033-java-cmdi`'s real-world shape `Runtime.getRuntime().exec("ping " + host)` was blocked by a SEPARATE, language-agnostic core-engine bug (`engine.js`'s `literalSkeletonMatchesFamily`/`_SHELL_META`, requiring a shell metacharacter in the STATIC portion of a CWE-78 concat — backwards for command injection, see `dataflow/CLAUDE.md`), now removed entirely. The corpus fixture also needed a Tier 2 fixture enrichment (`@RequestParam` on the `host` parameter — the committed fixture had no cataloged source at all, a second, independent blocker this note's original git-stash bisection had isolated away from before concluding `_SHELL_META` was the (sole, at the time) remaining issue). Both fixed together; `CVE-2016-10033-java-cmdi` now scores `pre:TP post:TN`. **Taint-recall PRD (80%) Tier 5: a THIRD, independent chain-CST bug — a constructor STARTING a chain.** Found via `CVE-2019-3799-spring-ssrf-shape`'s real-world shape `new URL(url).openStream()`: the chained-call fix above (the `primarySuffix[]` walk) only handles a chain that starts from an FQN/identifier prefix (`prefix?.children?.fqnOrRefType?.[0]`) — a chain whose FIRST segment is itself `new X(args)` takes a completely different CST shape and was never checked, so both the class name and the constructor's own args were silently dropped, leaving only the trailing `.openStream()` call with no receiver. Root-caused via temporary `console.error(Object.keys(prefix?.children))` tracing (the first guess, matching the STANDALONE `new X(args)` handling elsewhere in this file at `unqualifiedClassInstanceCreationExpression` directly off `node`, was wrong — a chain-starting constructor sits one level deeper, under `prefix?.children?.newExpression?.[0]?.children?.unqualifiedClassInstanceCreationExpression?.[0]`). Fixed by extending the same `primaryPrefix` chain-seed logic to also recognize this shape (`ctorPrefix`), extracting the class name and constructor args exactly like the FQN case, so the chain fix now covers both "chain starts on an identifier" and "chain starts on a constructor". Companion catalog fix: `java-url-openstream` was previously keyed to the (now-unreachable-via-chain) bare `callee: 'URL'`; corrected to `callee: 'openStream', receiver: '^URL'` — the same "terminal segment shift" pattern this PRD documents repeatedly elsewhere (Kotlin/Go). Guarded by `test/catalog-path-ssrf-p3.test.js`. |
+| Go       | `parser-go.js`        | Hand-rolled. **Taint-recall PRD (80%) Tier 4: inline anonymous closures.** Go's dominant HTTP-handler-registration idiom across every framework this file targets (net/http, gin, echo, fiber, chi) is an inline closure passed as the LAST argument to a registration call — `app.Get(path, func(c *fiber.Ctx) error { ... })`, `http.HandleFunc(path, func(w, r) { ... })`. This parser previously had ZERO support for anonymous functions at all: `FUNC_RE` requires a NAME between `func` and `(`, so an inline closure never matched it, and `_lowerExpr`'s generic call-matching regex mis-parsed `func(params) rtype { body }` as a call to something literally named "func" — `matchBalancedCall` correctly captured only `(params)` as that "call"'s own args, and the return-type token plus the ENTIRE closure body were silently discarded (no `.method(...)` continuation follows a return type, so `_followChain` found nothing to recover). This was very likely this PRD's single most consequential Go gap — it made every framework's route-handler BODY invisible to taint analysis regardless of what it did — found via this PRD's Tier 3 command-injection audit, not the parent PRD's original per-language sweep. Fixed by `_extractTrailingClosureArg` detecting a statement's LAST top-level call argument as an inline closure literal and INLINING its body directly into the enclosing function's CFG right after the registration call — a permissive, recall-preserving approximation (the closure genuinely runs later, but what matters for taint is that its statements become real CFG nodes at all), mirroring the Kotlin trailing-lambda / Ruby trailing-block precedent elsewhere in this PRD. The closure's OWN parameter (`c`, `w`, `r`) needs no synthetic taint binding, unlike Kotlin's `.forEach { x -> ... }`: a framework context object isn't itself a source — `c.Query(...)` is recognized by the EXISTING member/call-source catalog matching regardless of which function scope `c` was declared in. Chained calls before the closure arg (`app.Group("/api").Get(path, func(){...})`) are not specially handled — out of scope, matches this file's "handle the dominant shape" precedent elsewhere. Guarded by `test/parser-go.test.js`. Two catalog/engine precision bugs surfaced in the SAME audit, both fixed alongside: `go-os-exec-command`'s `argIndex: 0` checked the WRONG argument (the real dangerous shape is `exec.Command("/bin/sh", "-c", tainted)`, where arg 0 is always the literal interpreter and the tainted content sits later) — widened to `argIndex: 'all'`, gated by a new `match.requireLiteralArg` precision primitive (`catalog.js`/`engine.js`) requiring arg 0 to literally be a shell interpreter, so the safe array-execve form (`exec.Command("ping", "-c", "1", host)`) does not spuriously fire; and a "terminal segment shift" (same pattern as `kt-xpath-evaluate`/`java-spel-getvalue`/`go-r-uquery-get` elsewhere in this PRD) for `exec.Command(...).Output()`/`.Run()`/`.CombinedOutput()`/`.Start()` — chaining a Cmd-execution method directly onto `exec.Command(...)`, the dominant idiom for actually RUNNING the command, collapses the chain into one dotted string whose last segment is no longer "Command". Guarded by `test/catalog-command-injection-p4.test.js`. **Measured `bench/layer-recall` impact: real movement — command-injection (this corpus's dominant Go family) moved from 5/23 → 18/23 across this fix plus the sibling `literalSkeletonMatchesFamily`/CWE-78 engine fix (`dataflow/CLAUDE.md`).** |
+| Ruby     | `parser-rb.js`        | Hand-rolled. **`DEF_RE` must not let `\s*` cross a newline** — it did, and the body slice then started after the method's first statement, silently dropping it from every method (a one-statement body became empty). Guarded by `test/parser-php-rb.test.js`. ⚠ Also emitted no `fn.calls` at all (every OTHER parser does) — `callgraph.js`'s edges/callersOf/resolveKnownCallee are built entirely from `fn.calls`, so this left dead-code demotion and any interprocedural signal that depends on real call-graph resolution (rather than engine.js's generic tainted-call-argument fallback) permanently blind to Ruby. Fixed by deriving `fn.calls` from the CFG via the shared `call-sites.js#callSitesFromCfg` (the same helper `parser-py-cst.js` uses) — Ruby's node shapes already matched its documented contract. Guarded by `test/parser-rb-calls.test.js`. **Taint-recall PRD (80%): full CFG rebuild.** `_buildCfg` previously only recursed into `if`/`unless`/`while`/`until` bodies — `for`, `case`/`when`/`else`, `begin`/`rescue`/`ensure`, and any trailing block attached to a call (`xs.each do |x| … end`, the dominant Rails/ActiveRecord idiom) were silently dropped entirely, with no CFG node at all. `case`/`when` and `begin`/`rescue`/`ensure` use a recall-preserving model (every arm/clause reachable directly from the construct's own entry point, not modeled as mutually exclusive — same tradeoff `parser-kt.js`'s `_buildWhenArms` already established). A `do` block's opener also had to sit at the START of a line to be recognized at all — `_splitStatements`' depth-tracking now scans the WHOLE line (via `_rbLineDepthDelta`) rather than gating on `_RB_OPENERS.test(line)`, so a trailing `do` (mid-line, the common case) is now correctly consolidated instead of splitting its own body into independent nonsense statements. Trailing blocks recurse unconditionally and bind every named block parameter to the call's receiver (permissive by design — Ruby has no equivalent of Kotlin's implicit-this `apply`/`run` that would need to NOT bind). A recursion-depth guard (60) was added, matching every other R8-style rebuild in this codebase. Three further, independent bugs surfaced during the rebuild's own corpus-fixture debugging, all fixed: (1) `::` (Ruby's module-scope call operator, e.g. `Nokogiri::XML(x)`) was entirely invisible to `matchBalancedCall`'s `[\w.]+` callee regex — normalized to `.` on lowering; (2) a subscript-assignment on a member chain (`response.headers[key] = value`) had no assign-target branch at all and was silently dropped — lowered as a synthetic `<receiver>.[]=(key, value)` call, same shape as `parser-py.helper.py`'s `__setitem__` synthesis; (3) `_lowerExpr` checked "does this start with a quote" BEFORE checking for top-level `+` concatenation, so `"/var/data/" + name` was swallowed whole as one opaque literal — reordered, with a new `_splitTopLevelPlus` helper (string/paren/bracket-aware, so `"a+b"` is not mis-split) fixing it the same way `parser-go.js`'s R3 fix did for the identical ordering bug. Also inherits this PRD's shared chained-call fix (`_followChain`, `matchBalancedCall`-based) and keyword-argument fix from earlier in the same PRD. **Known, deliberately deferred gap:** the brace form of a trailing block (`Nokogiri::XML(xml) { |c| … }`) has its TRIGGER call correctly recognized (unlike before, where the trailing `{ }` corrupted or blocked the match), but the block BODY itself is not recursed into — only the `do...end` form got full body recursion, since it is the dominant multi-line Rails idiom and the single corpus fixture needing brace-form support only needed the trigger call's own argument, not its body. **Also known, NOT Ruby-specific:** a sink nested inside ANOTHER call's own argument (e.g. `render plain: URI.open(tainted).read`) is not independently checkable — `engine.js`'s sink-matching operates at CFG-node granularity (the outer `render` call), not on arbitrary nested sub-expressions; the SAME limitation was found via C#/Go/Kotlin corpus work earlier in this PRD, not something this task introduced or fixed. Guarded by `test/parser-rb-control-flow.test.js`. Two genuine ReDoS regexes were caught and fixed by `bench:self-scan:check` during this task — the trailing-block detector's first version had the `(.+?)\s+do` shape (an unbounded lazy prefix, NOT the "optional group between `\s*`" class every other ReDoS fix in this codebase has been; splitting into alternatives alone did not fix it — the leading capturing group itself had to go, replaced with a direct anchored search for `\bdo\b`), and the SAME detector's heuristic still flagged the replacement's residual "optional group between `\s*`" shape even though it measured linear on its own — split into two alternatives to satisfy the detector too. Measured `bench/layer-recall` impact on dataflow-shaped-subset taint recall: real, substantial movement (see `docs/TAINT_RECALL_80PCT_PRD.md`) — Ruby's own family entries moved from mostly-0% to 8/10 taint-detected. **Taint-recall PRD (80%) Tier 3: backtick shell-execution operator.** Found in the command-injection audit: `` `finger #{user}` `` (Kernel#`, equivalent to `%x{...}`) is a completely distinct syntax from a double-quoted string, but had NO recognizer at all — it fell through every `_lowerExpr` branch to `{kind:'unknown'}`, silently dropping the shell command (and any interpolated taint inside it) entirely. Lowered to a synthetic call (`__ruby_backtick_exec__`, an identifier real Ruby code can never actually name a dotted method) carrying the interpolated command as its sole argument, so a normal callee-keyed catalog sink (`rb-backtick-exec`) targets it exactly like any other call-shaped sink. A SEPARATE gap surfaced testing this: a bare backtick expression as its OWN statement (not assigned to a variable — the LAST expression of a `do...end` block, which Ruby implicitly returns, is exactly this shape) matched none of `_lowerStmt`'s branches and was dropped even after the `_lowerExpr` fix; `_lowerStmt` now delegates to `_lowerExpr` directly for this shape. Guarded by `test/catalog-command-injection-p4.test.js`. |
+| PHP      | `parser-php.js`       | Regex-based, hand-rolled. ⚠ **PRD R8 was this codebase's hardest single task — a genuine 3-fix-round debugging saga, all substantially about line-number precision, not detection shape.** The core fix flushes the statement splitter on a closing `}` (previously only on `;`), and adds `try`/`switch` recognizers plus a recursion guard, so statements inside `if`/`while`/`foreach`/`try`/`switch` bodies are now real CFG nodes instead of being dropped or folded into a bogus call node — this alone also resolved a pre-existing bug where `if`/`while`/`foreach` bodies were already being mis-split even before R8 touched them. Round 1 fixed the naive `}`-flush breaking `if`/`else` and multi-clause `try` (via a continuation-keyword lookahead) and switch/case's first-statement drop (via a `:`-based flush, careful to exclude `::` so PHP 8.1 enum cases and `case Foo::BAR:` class-constant labels aren't false-positived) — but round 1's own line-tracking approach was then found wrong for comment-bearing bodies and multi-line headers, a regression the round itself introduced. Round 2 fixed that (comment-skip handlers were discarding newlines uncounted; use exact `_countNewlines`-based computation everywhere) but its own re-review found the overall "exact line" property still failed, due to a *different*, genuinely pre-existing bug in the function-body's own base-line computation (wrong for Allman-brace style, multi-line signatures, blank-line-preceded functions) plus a genuine new regression from round 1 (a comment between `}` and `else`/`catch`/`finally` dropped that continuation's body entirely). Round 3 fixed both and was confirmed clean by an 8-shape holistic sweep. Also fixed along the way: dead `finally` support (greedy regex capture bug), unrecognized `try{}finally{}` with no `catch`, and the `::` case-label false-positive — via a hand-rolled balanced-brace scanner replacing the fragile regex approach. **Known, deliberately deferred gaps** (full list + grouping in the PRD R8 status entry): the PHP 8 `match` expression is unmodeled (same class as Java's arrow-switch); `if`/`else`'s pre-existing greedy-capture bug still drops the else-body's first statement; a heredoc containing a bare `}` loses its sink entirely (a real regression from R8's original commit, not the fix rounds); `elseif`/`else if` chains are still fully unsupported (pre-existing). Guarded by `test/parser-php-control-flow.test.js`. **Measured `bench/layer-recall` impact: unchanged, 1/23 before and after — the corpus's own `1/23 → 2/23` movement is real but belongs to R14(b) (PHP `<module>` top-level lowering), not this task**, confirmed by commit-swap A/B testing (the pre-R8 parser still reproduces 2/23; the pre-R14(b) parser reproduces only 1/23). None of this corpus's `pre/` PHP fixtures place a sink genuinely inside a braced control-flow body — same explanation as the other three languages' rows.
+
+**Taint-recall PRD (80%) Tier 4: comment-unawareness + `"literal" . $var` concat mis-parse.** Two PREVIOUSLY-DEFERRED gaps from the R8 list above, closed together (found while working the same gap class for Kotlin, below): (1) `_extractBody` (the function-BODY brace-matcher, a SEPARATE code path from `_splitStatements`'s own comment handling) had ZERO comment awareness — an apostrophe inside ANY comment ("don't", "it's") toggled its string-tracking state exactly like a real string literal, corrupting brace-depth for everything after it; depending on what followed, this either made `_extractBody` return `null` (silently dropping the **entire file's** IR — a single failed top-level function match corrupts every span downstream) or extracted the wrong body. Fixed by making `_extractBody` skip all three PHP comment forms (`//`, `#`, `/* */`) — the same fix `_splitStatements` already had for two of the three; `#`-comments (PHP's third form, previously invisible to `_splitStatements` too) are now handled there as well, careful to exclude `#[...]` PHP 8 attribute syntax from being mistaken for a comment. (2) `"literal" . $var` — arguably the single most common real-world PHP SQL-injection shape — was swallowed whole into one opaque `literal` node by TWO stacked bugs in `_lowerExpr`, each the same "unanchored prefix" defect class: the double-quoted-interpolation branch (`/^"/.test(s) && s.includes('$')`) matched on the WHOLE concat expression merely because it started with `"` and contained a `$` *anywhere* (in `$var`, outside the string, after the `.`), slicing off the wrong first/last characters as if the entire expression were one interpolated string; once that was fixed, the plain string-literal fallback (`/^"/.test(s)`) did the identical unanchored check one layer down and caught what the first fix now let through. Both are now anchored (`/^"(?:[^"\\]|\\.)*"$/`, full match required) so a concat correctly falls through to the `.`-splitting branch instead of either misreading the trailing ` . $var` as string content or swallowing the whole expression as an opaque literal. Guarded by 6 new cases in `test/parser-php-control-flow.test.js`, including an end-to-end `runScan` test proving real `$_GET`-to-PDO-sink taint flow through this exact concat shape. **Measured `bench/layer-recall` impact: real movement, PHP taint recall 2/23 (9%) → 7/23 (30%)** — the largest single per-language jump measured in this PRD, consistent with how common the `"literal" . $var` idiom is in real PHP code. **Taint-recall PRD (80%) Tier 3: PHP carried zero XSS sink entries.** `echo`/`print` are PHP LANGUAGE CONSTRUCTS, not function calls (`echo "<div>" . $_GET['q'] . "</div>";` has no `(` immediately after the keyword), so the statement-form call regex never matched them — the entire echoed expression, including any reflected taint, was silently dropped. `_lowerStmt` now recognizes `echo`/`print` directly and lowers to a synthetic call (`__php_echo__`, args split on top-level commas since `echo` accepts multiple comma-separated expressions), targeted by a new `php-echo-xss` sink — same synthetic-callee convention as Ruby's `__ruby_backtick_exec__` below. Guarded by `test/catalog-xss-p4.test.js`. |
 | C#       | `parser-cs.js`        | Hand-rolled. ⚠ `_lowerExpr`'s string-concat branch **must** guard on `_splitTopLevelPlus` returning more than one part — when the `+` is nested inside parens the splitter returns the input unchanged and the branch recurses on the identical string (stack overflow, swallowed by `buildProjectIR`'s per-file catch, surfacing only as "no IR"). `new Type(args)` is lowered to a call so taint reaches constructor sinks such as `new SqlCommand`. Guarded by `test/parser-cs-kt.test.js`. **PRD R8: `_buildCfg` was rewritten from a flat linear CFG loop into a recursive builder** (ported from `parser-cpp.js`'s proven pattern) that recurses into `if`/`else`/`while`/`for`/`foreach`/`switch`/`do`/`try`/`catch`/`finally` bodies, with exact character-offset line computation and a `foreach`/`for`-init loop-variable assign (same lesson Java's for-each fix needed). Landed with one fix round: `using (...) { }` and `lock (...) { }` bodies were completely invisible (not in the recognized-keyword regex) — `using` is THE canonical ADO.NET wrapper around exactly the sinks this task targets, so this was a real, significant gap, fixed with a one-word regex addition. The task also deliberately deviated from its own brief in one place: it added a guarded `}`-flush to the statement splitter (the brief said not to; the reviewer confirmed the brief's own illustrative code would have dropped every statement following a control-flow block, and independently verified the deviation safe against collection/object initializers and lambda arguments). Deferred, not fixed: a collection-initializer-then-chained-call shape mis-splits (not a regression); the pre-existing `@"C:\"` verbatim-string escape bug (confirmed unchanged). Guarded by `test/parser-cs-control-flow.test.js`. **Measured `bench/layer-recall` impact: unchanged, 1/21 before and after** — same explanation as Java's row: this corpus's existing C# fixtures with `if`/`try`/`using` syntax use it as a guard clause ahead of a flat-level sink rather than nesting the sink inside the body, so they don't exercise the exact shape this fix targets. |
-| Kotlin   | `parser-kt.js`        | Hand-rolled, parallel approach to `parser-cs.js`. **PRD R8: `_buildCfg` was rewritten from a flat linear CFG loop into a new recursive builder mirroring C#'s**, with Kotlin-specific adaptations — a `_consumeChunk` chain-consumer (Kotlin's splitter doesn't flush on `}` the way C#'s does) and dedicated `_buildWhenArms` parsing (`when`'s `else` arm would otherwise collide with `if`/`else` chaining in a generic keyword scan). This was the cleanest of the four R8 tasks — zero fix rounds — in part because the implementer was briefed on the other three tasks' hard-won lessons up front: it self-caught and fixed, before it could become a fix round, the exact same function-body-base-line-anchor bug PHP's hardest round found; it correctly identified and fixed a real gap (trailing-lambda calls like `xs.forEach { x -> ... }` genuinely fell through to `{kind:'unknown'}` before the fix); and it made a deliberate, correctly-scoped decision **not** to special-case `synchronized(lock) { }` — confirmed to be an ordinary Kotlin stdlib `inline fun`, not real keyword grammar, unlike C#'s `lock`. Statement-position control flow (`if`/`while`/`for`/`when`/`do`/`try`/`catch`/`finally`) is now covered; was previously the documented 0% Kotlin taint recall in `bench/layer-recall` (0/20). **Measured `bench/layer-recall` impact: unchanged, 0/20 before and after** — the dedicated unit tests below directly prove the fix works for a sink genuinely nested inside a control-flow body, but none of this corpus's 20 existing Kotlin fixtures happen to place a sink that way (same explanation as Java's and C#'s rows) — a candidate future item is enrolling a Kotlin corpus fixture that actually exercises this shape. **Two important, NOT-a-regression scope boundaries, worth reading before assuming Kotlin's control-flow support is complete:** (1) control flow *inside* any trailing lambda (`.forEach{}`, `.use{}` — the idiomatic Kotlin JDBC/resource-management shape — `.run{}`/`.apply{}`/`.let{}`/`.also{}`) is still invisible; the lambda's call SITE is captured, but its body is not parsed, so a sink one level inside a trailing lambda is exactly as invisible as before this task — and this is plausibly where a lot of real Kotlin taint-relevant code actually lives. (2) expression-position `if` (`val r = if (...) {...} else {...}`, Kotlin's ternary-replacement idiom) still drops both branch bodies. Also deferred: `_extractBody`'s comment-unawareness — the same defect class as PHP's (see the PHP row), an apostrophe inside any `//` comment silently drops the whole function, confirmed possibly larger real-world impact than the CFG gap this task closed. Guarded by `test/parser-kt-control-flow.test.js`. ⚠ **`bench:self-scan:check`, run during this PRD's own Task 5 (full-gate verification), caught a genuine ReDoS this task's new trailing-lambda regex introduced** (`/^([\w.]+)\s*(\([^()]*\))?\s*\{[\s\S]*\}\s*$/` — an optional paren group sandwiched between two `\s*` quantifiers, the identical defect class R14(a)'s C# `attrRegex` ReDoS; confirmed genuinely quadratic by direct timing, not a detector false alarm). Fixed the same way that precedent was: restructured into two mutually-exclusive alternatives (no-parens / with-parens) rather than one optional group, re-verified linear and byte-identical across a 15-shape sweep; no `bench/self-scan/BASELINE.json` bump was needed once fixed. **Separately, and NOT fixed here** (pre-existing since commit `99c2b6a`, 2026-05-20 — predates this PRD entirely, already counted in the pre-R8 self-scan baseline): the variable-declaration regex a few lines above (`decl`, matching `val`/`var … : Type = expr`) has the same adjacent-`\s*`-around-ambiguous-content shape and is also confirmed genuinely quadratic — logged as a candidate future item, matching this PRD's own precedent for pre-existing bugs found incidentally (e.g. the R14(a) C# parenthesized-attribute-argument note). |
+| Kotlin   | `parser-kt.js`        | Hand-rolled, parallel approach to `parser-cs.js`. **PRD R8: `_buildCfg` was rewritten from a flat linear CFG loop into a new recursive builder mirroring C#'s**, with Kotlin-specific adaptations — a `_consumeChunk` chain-consumer (Kotlin's splitter doesn't flush on `}` the way C#'s does) and dedicated `_buildWhenArms` parsing (`when`'s `else` arm would otherwise collide with `if`/`else` chaining in a generic keyword scan). This was the cleanest of the four R8 tasks — zero fix rounds — in part because the implementer was briefed on the other three tasks' hard-won lessons up front: it self-caught and fixed, before it could become a fix round, the exact same function-body-base-line-anchor bug PHP's hardest round found; it correctly identified and fixed a real gap (trailing-lambda calls like `xs.forEach { x -> ... }` genuinely fell through to `{kind:'unknown'}` before the fix); and it made a deliberate, correctly-scoped decision **not** to special-case `synchronized(lock) { }` — confirmed to be an ordinary Kotlin stdlib `inline fun`, not real keyword grammar, unlike C#'s `lock`. Statement-position control flow (`if`/`while`/`for`/`when`/`do`/`try`/`catch`/`finally`) is now covered; was previously the documented 0% Kotlin taint recall in `bench/layer-recall` (0/20). **Measured `bench/layer-recall` impact: unchanged, 0/20 before and after** — the dedicated unit tests below directly prove the fix works for a sink genuinely nested inside a control-flow body, but none of this corpus's 20 existing Kotlin fixtures happen to place a sink that way (same explanation as Java's and C#'s rows) — a candidate future item is enrolling a Kotlin corpus fixture that actually exercises this shape. **Two important, NOT-a-regression scope boundaries, worth reading before assuming Kotlin's control-flow support is complete:** (1) ~~control flow *inside* any trailing lambda... is still invisible~~ **fixed by taint-engine PRD P1, see below.** (2) expression-position `if` (`val r = if (...) {...} else {...}`, Kotlin's ternary-replacement idiom) still drops both branch bodies. Also deferred: `_extractBody`'s comment-unawareness — the same defect class as PHP's (see the PHP row), an apostrophe inside any `//` comment silently drops the whole function, confirmed possibly larger real-world impact than the CFG gap this task closed. Guarded by `test/parser-kt-control-flow.test.js`. ⚠ **`bench:self-scan:check`, run during this PRD's own Task 5 (full-gate verification), caught a genuine ReDoS this task's new trailing-lambda regex introduced** (`/^([\w.]+)\s*(\([^()]*\))?\s*\{[\s\S]*\}\s*$/` — an optional paren group sandwiched between two `\s*` quantifiers, the identical defect class R14(a)'s C# `attrRegex` ReDoS; confirmed genuinely quadratic by direct timing, not a detector false alarm). Fixed the same way that precedent was: restructured into two mutually-exclusive alternatives (no-parens / with-parens) rather than one optional group, re-verified linear and byte-identical across a 15-shape sweep; no `bench/self-scan/BASELINE.json` bump was needed once fixed. **Separately, and NOT fixed here** (pre-existing since commit `99c2b6a`, 2026-05-20 — predates this PRD entirely, already counted in the pre-R8 self-scan baseline): the variable-declaration regex a few lines above (`decl`, matching `val`/`var … : Type = expr`) has the same adjacent-`\s*`-around-ambiguous-content shape and is also confirmed genuinely quadratic — logged as a candidate future item, matching this PRD's own precedent for pre-existing bugs found incidentally (e.g. the R14(a) C# parenthesized-attribute-argument note).
+
+**Taint-engine PRD P1: trailing-lambda BODY recursion (closing the scope boundary (1) above).** `_consumeChunk` now detects a `recv.method(args)? { … }` trailing-lambda call site via `TRAILING_LAMBDA_TRIGGER_RE`, finds the real matching `}` with `_matchDelim` (not the old `_lowerStmt` fallback's greedy `[\s\S]*\}\s*$`, which mis-captured a chained `xs.filter{}.forEach{}` as one opaque lambda), and recurses `_buildCfg` into the body — so a sink nested inside `.forEach{}`/`.use{}`/`.apply{}`/`.run{}`/etc. is now a real CFG node, not dropped. A fixed `LAMBDA_BINDABLE_METHODS` set (`forEach`/`map`/`filter`/`reduce`/`fold`/`use`/`let`/`also`) gets a synthesized taint-binding assign for the lambda parameter (implicit `it` or a named param, both accumulator+element for `reduce`/`fold`) before the body is recursed into — mirroring the for-loop's existing loop-variable binding. `.apply`/`.run` (implicit-`this`, no parameter) correctly recurse WITHOUT a binding. Chained lambdas (`.filter{}.forEach{}`) are not mis-captured; both bodies are reached. Verified via direct CFG inspection AND an end-to-end `runScan` test proving real taint flow through the binding into a sink. Guarded by `test/parser-kt-control-flow.test.js` (10 new cases) and `bench/cve-replay/deep/kt-trailing-lambda-pathtraversal-shape/` (added because, same as Java's/C#'s R8 rows, none of the *existing* 20 Kotlin corpus fixtures place a sink inside a trailing-lambda body — this new entry does, and is confirmed by direct env-var toggling to fire ONLY with `AGENTIC_SECURITY_DEEP=1`). **Measured `bench/layer-recall` impact: real movement, Kotlin taint recall 0/20 (0%) → 1/21 (5%).**
+
+**Taint-recall PRD (80%) Tier 4: the `?.` safe-call operator gap.** Every regex-based matcher in this file — the callee-matching regexes (`[\w.]+`-style), the plain-dotted-ident check, the trailing-lambda trigger, `_followChain`'s continuation regex — keys off a character class that excludes `?`, so `str?.trim()`, `xs?.forEach { … }`, and even a bare property read `x?.y` all fell through to `{kind:'unknown'}` entirely. Unlike `::` (Ruby) or a chain continuation, this is NOT limited to a later segment of a chain — the safe-call operator can appear on the very FIRST segment of an expression — so patching individual regexes one at a time would have missed call sites this file doesn't enumerate as a list. Fixed with a single global normalization pass (`_stripSafeCallOperator`) at the very top of `parseKotlinFile`, applied to the whole file's source text before any other parsing runs: string-literal-aware (so `"a?.b"` inside a literal is untouched) and strips only a `?` immediately before a `.` (so the elvis operator `?:` and a bare nullable-type marker `String?` are both unaffected — neither is followed by `.`). Dropping the `?` outright (not replacing it with a same-length filler) is safe specifically because `_lineStarts`/`_lineAt`/`_lineForOffset` are purely newline-position-based — removing a non-newline character can shift a later character's column but never its line, and this file never relies on column offsets. Guarded by two new cases in `test/kt-taint-flow.test.js` (one proving `?.`-qualified calls now flow taint into a sink, one proving the elvis operator and nullable-type declarations are unaffected). **Measured `bench/layer-recall` impact: unchanged, 6/21 before and after** — the fix is real and directly proven by the dedicated unit tests, but none of this corpus's 21 existing Kotlin fixtures happen to use the `?.` operator in their vulnerable shape (same "corpus doesn't exercise this exact syntax" explanation as every other R8-class fix in this PRD's Java/C#/PHP rows above).
+
+**Taint-recall PRD (80%) Tier 3: no subscript/bracket-access support at all.** Found via the XSS audit's `CVE-2021-29447-kotlin-xss` (`call.parameters["q"]` inside a string interpolation), but the gap is general, not XSS-specific: Kotlin's `operator fun get(key)` bracket syntax — used pervasively for Maps, arrays, and framework accessors like Ktor's `call.parameters[...]` — had NO recognizer in `_lowerExpr` at all; `map[key]` fell through every branch to `{kind:'unknown'}`, silently dropping the value (and any taint on it). Lowered to the same synthetic `'[]'` prop convention `parser-go.js`'s own Indexing branch already uses: the base expression becomes a real member chain, wrapped in one more member layer with `prop: '[]'`. This is what lets a cataloged MEMBER source on the base (`kt-ktor-parameters` — `call.parameters`) still taint the subscripted read: `exprIsSource` recurses into `expr.object` when the outer member itself doesn't match, landing on the base member it already knows — no new source-matching logic needed. Guarded by 2 new cases in `test/parser-cs-kt.test.js`. Also added this same audit: `kt-ktor-respondtext` (Ktor's `call.respondText(...)` had no XSS sink at all) and `kt-html-utils` (Spring's `HtmlUtils.htmlEscape` sanitizer was scoped `language: 'java'` only — JVM interop means the identical call is equally valid, idiomatic Kotlin, and the java-only entry was silently invisible to a `.kt` file). Guarded by `test/catalog-xss-p4.test.js`.
+
+⚠ **A second, fresh ReDoS, same defect class, this time in this task's OWN new code:** the first-draft `TRAILING_LAMBDA_TRIGGER_RE` (`/^([\w.]+)\s*(\([^()]*\))?\s*\{/`) had the identical optional-group-between-two-`\s*` shape as the R8-era regex documented in the paragraph above — `bench:self-scan:check` caught it immediately (a new self-finding in this very file) and direct timing confirmed genuinely quadratic (40 000 non-matching whitespace chars: ~1 s). Fixed identically: split into two alternatives (no-args / with-args), each with its own capture group, re-verified linear (200 000 chars: 0 ms) and correctness-preserving across 6 shapes. The lesson repeats a third time in this codebase (R14(a) C#, R8 Kotlin `decl`, now this) — the "optional group flanked by two `\s*`" shape should be treated as an automatic red flag whenever writing a new statement-recognizer regex here.
+
+**Two further gaps found while building the corpus entry above, NOT fixed (out of scope for trailing-lambda body recursion, routed around instead — same pattern as Java's chained-call CFG bug):** (a) a call with NO args made on a tainted receiver via a dotted callee string — e.g. `it.toString()`, `tainted.trim()` — does not inherit the receiver's taint. `engine.js`'s `exprTaint` for `case 'call'` checks only `expr.args` and the resolved callee's own return-taint summary (`_nestedCallReturnTainted`); a callee string like `"it.toString"` is never parsed back into a member expression to ask whether its OWN receiver (`it`) is tainted. This is not Kotlin-specific (any language whose IR lowers a method call to a bare dotted-string callee has the same exposure), so a real fix belongs in `engine.js`, not here. (b) `call.parameters` (Ktor's own cataloged `member`-type source) does not propagate taint through a subsequent `.getAll(...)` call made on top of it — `call.parameters.getAll("id")` reads as untainted, because the catalog entry matches a bare property READ, not a call whose receiver is that property. Both gaps were discovered via a fixture that initially used these exact shapes and silently produced zero findings; each was routed around (by using a no-transform binding and a `call`-type source instead) rather than fixed, to keep this task scoped to CFG body recursion. **Taint-recall PRD (80%) Tier 5: `kt-file-readtext`/`kt-url-readtext` catalog cross-collision.** Found via `CVE-2022-22965-kt-ssrf`: both entries were unscoped bare `callee: 'readText'` (path-traversal and SSRF sinks sharing the identical last-segment name), so a single `URL(url).readText()` call fired BOTH — the correct SSRF finding and a spurious Path Traversal finding on the same line. Fixed with receiver scoping (`^File` / `^URL`), the same precision pattern `match.receiver` uses throughout this catalog. Guarded by `test/catalog-path-ssrf-p3.test.js`; required updating `test/phase2-scoping.test.js`'s existing `matchSinkOrSanitizer('readText', 'A.java')` call (a bare string with no receiver context, which the new requirement correctly rejects) to a realistic dotted-string shape (`'File.readText'`). |
 | C / C++  | `parser-cpp.js`       | Hand-rolled parser (functions, qualified names, CFG lowering). Dispatched by extension (`c/cc/cpp/cxx/h/hh/hpp/hxx`) in both `buildProjectIR` and `buildProjectIRAsync`. |
 | Long-tail (rust/solidity/go/swift/dart) | `tree-sitter-loader.js` | **Optional** `web-tree-sitter` + `tree-sitter-wasms` (ABI-pinned 0.20.8 ↔ 0.1.13), lazy + degrades when absent. Powers `sast/tree-sitter-sinks.js` (opt-in via `AGENTIC_SECURITY_TREE_SITTER=1`). Marked `--external` in the build so the committed bundle never embeds WASM. |
 
@@ -138,6 +151,8 @@ of the constructs this section once listed as unmodeled. Verified end-to-end in
 - destructuring assignment (`a, b = expr`) — one assign per target, sourced from
   the element (`member[]`) of the RHS.
 
+**Taint-recall PRD (80%) Tier 5: `_lower_expr`'s `ast.Call` branch silently discarded an entire inner call in a 2+-level chain.** Found via `CVE-2019-10097-python-path-traversal`'s real-world shape `open(tainted).read()`: the old `_flatten_callee` "mixed shape" fallback (`return parts[-1] if parts else None`) collapsed a chain to its LAST callee name only and threw away every earlier `Call` node's own arguments in the process — not misattribution, genuine absence from the IR, so `tainted` never appeared anywhere in `open(tainted).read()`'s lowered form. Replaced with `_lower_call_chain(node)`, which walks the chain outer-to-inner, reconstructs the dot-joined callee name in source order, and accumulates each level's args outermost-first — the same accumulation convention `_followChain` uses in every hand-rolled JS-family parser in this directory. New sink `py-open-read-chained` (`callee: 'read', receiver: '^open'`, `argIndex: 'all'`) targets the now-visible chain. Guarded by `test/catalog-path-ssrf-p3.test.js`; regression-checked via full `npm run test:dataflow` (no drift). ⚠ The function's own docstring is Python source text inside a triple-quoted string LITERAL, not a `#`-comment — `blankComments()` does not strip it, so an illustrative code EXAMPLE written literally in the docstring (`open('/var/data/' + name).read()`) was matched as real vulnerable code by this project's own self-scan. Reworded to describe the shape in prose (referencing the corpus fixture by path) rather than literal syntax — the general lesson (illustrative code inside a Python docstring is live source text to every detector, unlike a `#` comment) applies to any future docstring in this file or `parser-py.js`.
+
 **Remaining limit (a deep-engine collection-element trait, NOT a dropped CFG
 node):** taint carried through the *element* of a destructured tuple or a
 comprehension result — `a, b = src1, src2; sink(a)` or `xs = [src…]; sink(xs[0])`

package/src/ir/balanced-call.js

@@ -51,5 +51,15 @@ export function matchBalancedCall(s, calleeRe) {
   }
   if (depth !== 0 || s[i] !== ')') return null; // unbalanced — refuse to guess
   const callee = m[1] !== undefined ? m[1] : m[0];
-  return { callee, argsText: s.slice(openIdx + 1, i) };
+  // Taint-recall PRD (80%): `endIdx` (the index right after the matched
+  // closing paren) lets a caller detect and recurse into a CHAINED
+  // continuation (`X(args).Y(args2)`) instead of silently leaving it
+  // unconsumed. Additive — existing callers that only destructure
+  // {callee, argsText} are unaffected. Confirmed via real corpus fixtures
+  // that the outer call in a chain is frequently the one carrying the
+  // actual sink and its tainted argument (`new DataTable().Compute(expr)`,
+  // `template.New("page").Parse(userTemplate)`,
+  // `w.Header().Set("X", tainted)`) — silently dropping it, not just
+  // leaving it unconsumed, is what this field exists to let callers fix.
+  return { callee, argsText: s.slice(openIdx + 1, i), endIdx: i + 1 };
 }

package/src/ir/callgraph.js

@@ -48,15 +48,37 @@ export function buildCallGraph(perFileIR, fileContents) {
     }
   }
 
+  // Taint-engine PRD P1: per-file bare-tail index. Java's fn.name is
+  // class-qualified ("App.buildCmd"), so byNameInFile's exact-match lookup
+  // above never matches a bare call site's callee ("buildCmd") — the most
+  // idiomatic Java call shape (private-helper delegation) was permanently
+  // unresolved. Scoped per-file (not project-wide) and refuses to guess on
+  // a same-file collision, mirroring the existing `~bare~`-key
+  // ambiguity-refusal pattern the C++ qualified-name index already uses —
+  // a wrong edge invents a data-flow path that doesn't exist, worse than a
+  // missing one.
+  const bareTailInFile = new Map(); // file -> Map<bareTail, qid|null>
+
   for (const file of Object.keys(perFileIR || {})) {
     const ir = perFileIR[file];
     if (!ir || !ir.functions) continue;
     byNameInFile.set(file, new Map());
+    bareTailInFile.set(file, new Map());
     for (const fn of ir.functions) {
       functions.set(fn.qid, fn);
       byNameInFile.get(file).set(fn.name, fn.qid);
       const m = fn.qid.match(/::([A-Z]\w*)::(\w+)@/);
       if (m) classMethods.set(`${m[1]}.${m[2]}`, fn.qid);
+      if (fn.name && fn.name.includes('.')) {
+        const tail = fn.name.split('.').pop();
+        const fileMap = bareTailInFile.get(file);
+        if (fileMap.has(tail)) {
+          const cur = fileMap.get(tail);
+          if (cur !== null && cur !== fn.qid) fileMap.set(tail, null); // ambiguous — refuse to guess
+        } else {
+          fileMap.set(tail, fn.qid);
+        }
+      }
     }
   }
 
@@ -151,6 +173,9 @@ export function buildCallGraph(perFileIR, fileContents) {
                        classMethods.get(c.callee) ||
                        // 2. ClassName.method form
                        (c.callee.includes('.') ? classMethods.get(c.callee) : null) ||
+                       // 2b. Bare-tail fallback, same file only, refuses to
+                       // guess on ambiguity (see bareTailInFile above).
+                       (!c.callee.includes('.') ? (bareTailInFile.get(fn.file)?.get(c.callee) || null) : null) ||
                        // 3. Cross-TU qualified-name index (C++ header/source
                        // pairing) — gated on the CALLER carrying a `qname`
                        // (only parser-cpp.js emits one) so a same-named call
@@ -220,6 +245,15 @@ export function buildCallGraph(perFileIR, fileContents) {
     if (callerFile) {
       const local = byNameInFile.get(callerFile);
       if (local && local.has(name)) return local.get(name);
+      // Bare-tail fallback, same file only (see bareTailInFile above) — an
+      // exact or intentionally-qualified match, not a guess: it is scoped
+      // to the caller's own file and already refuses ambiguity at index-
+      // build time, so it runs unconditionally like the other exact-match
+      // branches, not gated behind allowTailGuess.
+      if (!name.includes('.')) {
+        const tail = bareTailInFile.get(callerFile)?.get(name);
+        if (tail) return tail;
+      }
     }
     for (const m of byNameInFile.values()) {
       if (m.has(name) && !isCrossLanguageUnsafe(m.get(name), callerFile)) return m.get(name);

package/src/ir/parser-cs.js

@@ -49,10 +49,21 @@ import * as crypto from 'node:crypto';
 import { callSitesFromCfg } from './call-sites.js';
 import { matchBalancedCall } from './balanced-call.js';
 
+// Taint-engine PRD P1: the modifier group used to be MANDATORY (at least
+// one of public/private/.../partial required before the return type), so a
+// bare, implicitly-private method — legal and common for private helpers,
+// e.g. `void Render() { ... }` — never matched at all: the whole method,
+// and any sink inside it, was invisible to the IR. Each modifier now
+// consumes its own trailing whitespace and the whole group is zero-or-more,
+// so zero modifiers is a valid match. Safe against false positives:
+// control-flow keywords (if/for/while/using/catch/...) have only ONE token
+// before their parens, never this pattern's "type name(args)" two-token
+// shape, so they cannot start matching just because the modifier
+// requirement was dropped — pinned by a dedicated precision test.
 const METHOD_RE = new RegExp(
-  '(?:^|[\\s;{}])(?:public|private|protected|internal|static|virtual|override|async|sealed|abstract|new|readonly|partial)' +
-  '(?:\\s+(?:public|private|protected|internal|static|virtual|override|async|sealed|abstract|new|readonly|partial))*' +
-  '\\s+([A-Za-z_][A-Za-z0-9_<>?\\[\\],\\s]*?)' +    // return type (group 1)
+  '(?:^|[\\s;{}])' +
+  '(?:(?:public|private|protected|internal|static|virtual|override|async|sealed|abstract|new|readonly|partial)\\s+)*' +
+  '([A-Za-z_][A-Za-z0-9_<>?\\[\\],\\s]*?)' +    // return type (group 1)
   '\\s+([A-Za-z_][A-Za-z0-9_]*)' +                  // method name (group 2)
   '\\s*\\(([^)]*)\\)' +                             // params (group 3)
   '\\s*\\{', 'g');
@@ -249,6 +260,43 @@ function _splitTopLevelSemi(s) {
   return out;
 }
 
+// Taint-recall PRD (80%): a chained call (`new DataTable().Compute(expr)`,
+// `Response.Headers.Add(...)` chains further, etc.) previously stopped at
+// the FIRST balanced call and left any `.Method(args)` continuation
+// unconsumed (by matchBalancedCall's own design — see its header) —
+// correct for not corrupting the parse, but it meant the OUTER call, which
+// is frequently the one actually carrying the sink and its tainted
+// argument, was silently absent from the CFG entirely. Confirmed via a
+// real corpus fixture: `new DataTable().Compute(expr, "")` collapsed to
+// just the constructor call, args: [], dropping `.Compute(expr, "")`
+// completely.
+//
+// Walks forward from `endIdx` (the position right after the just-matched
+// call's closing paren) following every `.Method(args)` continuation,
+// dot-joining each level's name into one callee string (`DataTable.Compute`,
+// `template.New.Parse`) so both bare-name matching (last segment) and
+// receiver-pattern matching (earlier segments) keep working unchanged.
+//
+// Args from EVERY level are kept, outermost-first (`outerArgs.concat(prior)`
+// at each step) — NOT just the outermost. A first version kept only the
+// outermost call's args, which broke a real chain shape
+// (`xp.compile(taintedExpr).evaluate(doc, XPathConstants.NODESET)`-style,
+// found in Kotlin but the identical defect class applies here) where the
+// tainted value sits on an INNER call, not the final one — the outer call's
+// own args (if any) correctly stay at the front so an existing
+// `argIndex: 0` catalog entry keyed to the outermost call is unaffected;
+// inner levels' args are appended after so an `argIndex: 'all'` entry can
+// still find taint that only an inner call actually carried.
+function _followChain(s, endIdx, calleeSoFar, argsSoFar, isNew) {
+  const rest = s.slice(endIdx);
+  const m = rest.match(/^\.(\w+)/);
+  if (!m) return { kind: 'call', callee: calleeSoFar, args: argsSoFar, isNew };
+  const outer = matchBalancedCall(rest, /^\.(\w+)/);
+  if (!outer) return { kind: 'call', callee: calleeSoFar, args: argsSoFar, isNew };
+  const outerArgs = _splitTopLevelCommas(outer.argsText).map(_lowerExpr);
+  return _followChain(rest, outer.endIdx, `${calleeSoFar}.${outer.callee}`, outerArgs.concat(argsSoFar), false);
+}
+
 function _lowerExpr(text) {
   const s = String(text || '').trim();
   if (!s) return { kind: 'unknown' };
@@ -278,7 +326,7 @@ function _lowerExpr(text) {
   if (newMatch) {
     const callee = newMatch.callee.split('.').pop();
     const args = _splitTopLevelCommas(newMatch.argsText).map(_lowerExpr);
-    return { kind: 'call', callee, args, isNew: true };
+    return _followChain(s, newMatch.endIdx, callee, args, true);
   }
   // Call: foo.bar(args) or Bar(args). matchBalancedCall finds the paren
   // that actually balances the FIRST '(' — not the greedy-to-end-of-string
@@ -288,7 +336,7 @@ function _lowerExpr(text) {
   const callMatch = matchBalancedCall(s, /^([\w.]+)/);
   if (callMatch) {
     const args = _splitTopLevelCommas(callMatch.argsText).map(_lowerExpr);
-    return { kind: 'call', callee: callMatch.callee, args };
+    return _followChain(s, callMatch.endIdx, callMatch.callee, args, false);
   }
   // String concat / interpolation — heuristic.
   //
@@ -409,7 +457,8 @@ function _lowerStmt(stmt, line) {
   // statement-form call
   const cm = matchBalancedCall(s, /^([A-Za-z_][\w.]*)/);
   if (cm) {
-    return { kind: 'call', line, callee: cm.callee, args: _splitTopLevelCommas(cm.argsText).map(_lowerExpr) };
+    const chained = _followChain(s, cm.endIdx, cm.callee, _splitTopLevelCommas(cm.argsText).map(_lowerExpr), false);
+    return { kind: 'call', line, callee: chained.callee, args: chained.args };
   }
   return { kind: 'unknown', line, text: s };
 }

package/src/ir/parser-go.js

@@ -73,6 +73,33 @@ function _splitStatements(body) {
   return out;
 }
 
+// Taint-recall PRD (80%): a chained call (`w.Header().Set("X", tainted)`,
+// `template.New("page").Parse(userTemplate)`) previously stopped at the
+// FIRST balanced call, leaving any `.Method(args)` continuation
+// unconsumed — matchBalancedCall's own deliberate design, correct for not
+// corrupting the parse, but it meant the OUTER call — frequently the one
+// actually carrying the sink and its tainted argument — was silently
+// absent from the CFG. Confirmed via two real corpus fixtures (Go's
+// response-splitting and code-injection entries both hit this). Same
+// dot-joining approach as parser-cs.js's twin fix.
+//
+// Args from EVERY level are kept, outermost-first (`outerArgs.concat(prior)`
+// at each step) — NOT just the outermost. A first version kept only the
+// outermost call's args, which broke a real 3-level chain
+// (`X().getEngineByName("js").eval(userCode)`-shaped) where the tainted
+// value sits on an INNER call, not the final one — the outer call's own
+// args (if any) correctly stay at the front so an existing `argIndex: 0`
+// catalog entry keyed to the outermost call is unaffected; the inner
+// levels' args are appended after so an `argIndex: 'all'` entry can still
+// find taint that only an inner call actually carried.
+function _followChain(s, endIdx, calleeSoFar, argsSoFar) {
+  const rest = s.slice(endIdx);
+  const outer = matchBalancedCall(rest, /^\.(\w+)/);
+  if (!outer) return { kind: 'call', callee: calleeSoFar, args: argsSoFar };
+  const outerArgs = _splitTopLevelCommas(outer.argsText).map(_lowerExpr);
+  return _followChain(rest, outer.endIdx, `${calleeSoFar}.${outer.callee}`, outerArgs.concat(argsSoFar));
+}
+
 function _lowerExpr(text) {
   const s = String(text || '').trim();
   if (!s) return { kind: 'unknown' };
@@ -104,7 +131,7 @@ function _lowerExpr(text) {
   const callMatch = matchBalancedCall(s, /^([\w.]+)/);
   if (callMatch) {
     const args = _splitTopLevelCommas(callMatch.argsText).map(_lowerExpr);
-    return { kind: 'call', callee: callMatch.callee, args };
+    return _followChain(s, callMatch.endIdx, callMatch.callee, args);
   }
   // String concat with + — a SECOND occurrence of the same check as above
   // (line ~92), reached when the expression didn't match any branch in
@@ -240,11 +267,58 @@ function _lowerStmt(stmt, line) {
   // Statement-form call: obj.Method(args) or Method(args)
   const cm = matchBalancedCall(s, /^([\w.]+)/);
   if (cm) {
-    return { kind: 'call', line, callee: cm.callee, args: _splitTopLevelCommas(cm.argsText).map(_lowerExpr) };
+    const chained = _followChain(s, cm.endIdx, cm.callee, _splitTopLevelCommas(cm.argsText).map(_lowerExpr));
+    return { kind: 'call', line, callee: chained.callee, args: chained.args };
   }
   return null;
 }
 
+// Taint-recall PRD (80%): Go's dominant HTTP-handler-registration idiom
+// across every framework this file targets (net/http, gin, echo, fiber,
+// chi) is an inline anonymous closure passed as the LAST argument to a
+// registration call — `app.Get("/p", func(c *fiber.Ctx) error { ... })`,
+// `http.HandleFunc(path, func(w http.ResponseWriter, r *http.Request) {
+// ... })`. This parser previously had ZERO support for anonymous
+// functions: `FUNC_RE` requires a NAME between `func` and `(`, so an
+// inline closure never matched it at all, and `_lowerExpr`'s generic
+// call-matching regex mis-parsed `func(params) rtype { body }` as a call
+// to something literally named "func" — `matchBalancedCall` correctly
+// captured only `(params)` as that "call"'s own args, and the return-type
+// token plus the ENTIRE closure body were silently discarded (no
+// `.method(...)` continuation follows a return type, so `_followChain`
+// found nothing to recover). This was very likely this PRD's single most
+// consequential Go gap: it made every framework's route-handler BODY
+// invisible to taint analysis regardless of what it did — found via this
+// PRD's Tier 3 audit, not the parent PRD's original per-language sweep.
+const CLOSURE_ARG_RE = /^func\s*\(([^)]*)\)\s*(?:\([^)]*\)|[\w*[\].\s]*)?\s*\{/;
+
+// If the LAST top-level comma-separated part of `argsText` is an inline
+// closure literal, extracts its body text and returns `{ body, closureText,
+// prefixParts }` — `prefixParts` are the OTHER top-level args (unchanged,
+// still to be lowered normally), `closureText` is the exact substring
+// matched (used by the caller to locate its start line within the
+// enclosing statement). Returns null when there is no trailing closure —
+// every existing call site is unaffected.
+function _extractTrailingClosureArg(argsText) {
+  const parts = _splitTopLevelCommas(argsText);
+  if (!parts.length) return null;
+  const last = parts[parts.length - 1];
+  const m = last.match(CLOSURE_ARG_RE);
+  if (!m) return null;
+  const openBrace = last.indexOf('{', m[0].length - 1);
+  if (openBrace < 0) return null;
+  const extracted = _extractBody(last, openBrace);
+  if (!extracted) return null;
+  return { body: extracted.body, closureText: last, prefixParts: parts.slice(0, -1) };
+}
+
+function _countNewlinesUpTo(s, upTo) {
+  let n = 0;
+  const end = Math.min(upTo, s.length);
+  for (let i = 0; i < end; i++) if (s[i] === '\n') n++;
+  return n;
+}
+
 function _extractBody(src, openBrace) {
   let depth = 1;
   let i = openBrace + 1;
@@ -381,6 +455,36 @@ function _buildCfg(bodyText, nodes, prevId, startLine) {
       continue;
     }
 
+    // Statement-form call whose LAST top-level argument is an inline
+    // closure literal — see `_extractTrailingClosureArg` above. Recall-
+    // preserving inlining: the closure genuinely runs later (often
+    // asynchronously, on a request), but for taint analysis what matters
+    // is that its statements become real CFG nodes at all. The closure's
+    // OWN parameter (`c`, `w`, `r`) needs no synthetic taint binding, unlike
+    // e.g. Kotlin's `.forEach { x -> ... }`: a framework context object
+    // isn't itself a taint source — `c.Query(...)` is recognized by the
+    // EXISTING member/call-source catalog matching regardless of which
+    // function scope `c` was declared in. Chained calls before the closure
+    // arg (`app.Group("/api").Get(path, func(){...})`) are not specially
+    // handled — out of scope for this fix, matches this file's existing
+    // "handle the dominant shape" precedent elsewhere.
+    const closureCall = matchBalancedCall(s, /^([\w.]+)/);
+    if (closureCall) {
+      const closure = _extractTrailingClosureArg(closureCall.argsText);
+      if (closure) {
+        const outerArgs = closure.prefixParts.map(_lowerExpr);
+        const callId = _addNode(nodes, { kind: 'call', line, callee: closureCall.callee, args: outerArgs });
+        _link(nodes, prev, callId);
+        const closureOffset = s.indexOf(closure.closureText);
+        const bodyStartLine = closureOffset >= 0
+          ? line + _countNewlinesUpTo(s, closureOffset)
+          : line;
+        prev = _buildCfg(closure.body, nodes, callId, bodyStartLine);
+        line += (s.match(/\n/g) || []).length + 1;
+        continue;
+      }
+    }
+
     // Regular statement
     const node = _lowerStmt(s, line);
     if (!node) { line++; continue; }