@clear-capabilities/agentic-security-scanner 0.195.7 → 0.195.9

package/CHANGELOG.md

@@ -9,6 +9,32 @@
 > make the history less accurate, not more.
 
 
+## 0.195.9 - Haskell real-code precision, a blind unseen-v6 and the first blind Nix split, the unseen-v5 and Nix development shapes
+
+Four pieces of work, in the order they were done, each measured where it could be measured. Every number below comes from a run recorded in this repository on 2026-11-01.
+
+**Real-code precision (`bench/dag/dev-haskell`, `bench/haskell-precision`).** The 28 flagged false positives of the fresh set and the 5 of the development remainder were read by cause, live, with the current engine. Fourteen were a command-line program reading a FILE NAME from its own standard input (dhall's expression input, summoner's prompts); the trust-boundary module's own header said standard input in a command-line program crosses no boundary, but the code excluded it, so for a path operation it now counts (a command built from standard input, `getLine >>= callCommand`, stays reported, as before). Five were a path parameter handed on whole by a library function (`renderFile path = responseFile .. path`, `store path = readFile (cs path)`): the whole-handoff rule the process wrappers already had (`api-contract`, lowered to information) now covers path operations too, and a parameter spliced under a fixed directory is still a finding. The decision goes against three synthetic development splits, which label `load name = readFile name` vulnerable, and with the real code, where no true positive had that shape; the synthetic numbers that moved are recorded here, not hidden. Three were scotty's own cookie helpers (`setSimpleCookie n v`: a name that is the function's own parameter is decided by the caller), one a hashing tool printing the hash it made (`hashedPassword`: a binding whose name says digest is the stored form). Fresh set: 45 flagged, 17 true positives, 37.8% before; 26 flagged, 17 true positives, 65.4% after; no fresh true positive moved. Development tuning half: 10 flagged, 4 true positives before; 5 flagged, 2 true positives after, because two labelled sites are call-site copies of one fast-hash helper the labeller recorded as "three sites of one weakness", now carried as `alsoAt` on the helper's finding (still reported). The remaining false positives are named in `bench/dag/dev-haskell/README.md`; the sealed set was not run. Baseline re-recorded with `bench:haskell-precision:update`, every changed site checked against its label.
+
+**unseen-v6 (Haskell) and unseen-nix-v1 (Nix), both written blind.** Two separate agent sessions that could not see the repository wrote 78 Haskell forms and 72 NixOS forms from the family names, their CWE meanings and the authoring format, and reported counts only (the unseen-v5 author's per-family library list was a disclosed leak; this time the brief forbade it). The shared builder (`corpora/frozen-split.mjs`) takes an ecosystem, a split's manifest records it, a Nix form's marker is `#@@`, and the detection suite scores a frozen split under its own ecosystem; the two Haskell splits frozen before the field existed reproduce their digests unchanged. Baselines, taken before any engine change motivated by a split: unseen-v6 precision 71.0%, recall 56.4%, F1 62.9% (after the real-code work above, before unseen-v5 was read); unseen-nix-v1 precision 84.6%, recall 61.1%, F1 71.0%, the first Nix generalisation number this project has had.
+
+**unseen-v5 opened (a development set from 2026-11-01), the Nix unseen-v3 development set finished.** Haskell, pinned both ways in `test/haskell/haskell-unseen-v5-shapes.test.js`: a Servant handler pairs the API's reads with its LAST parameters (`orderServer conn ref` was annotated on `conn`); a `Set-Cookie` header handed to `setHeader`; a helper the IR inlined at its call site is reported once, in the helper; mwc-random and splitmix are predictable; a function that makes a key is security-shaped and a `code` is a one-time code where the function talks about a reset; an authentication endpoint is never missing authentication; `X-Session` and similar project header names are credentials; a yes/no answer about a credential, a record logged through a non-credential field and a `lastFour` helper are not a leak, the record logged whole is; fast-logger is a log sink; a URL that starts from a literal naming the host (or a request built over a fixed base by http-conduit's modifiers) is information with the reason `fixed-host-request`; a guard in the helper that holds the sink refutes the flow found from its caller; a blaze helper that escapes its parameter is an escaper for the caller's sink. The detection harness now scores `actionable` findings (not `info`, not proven clean or infeasible), the rule the other benches already used. Nix, pinned in `test/nix/nix-development-shapes.test.js`: `AmbientCapabilities` as one string, `Defaults !authenticate`, a `*File` option handed a `writeText` of a literal, a TLS key path on any enabled service, one script-taint finding per rule, line and origin class. Results: unseen-v5 (development) F1 65.7% to 84.1% (precision 96.7%, recall 74.4%); unseen-v6 (generalisation, aggregate only) F1 62.9% to 72.5% (precision 83.3%, recall 64.1%), so part of it generalised this time; Nix unseen-v3 (development) to 100% on both layers; unseen-nix-v1 (generalisation) 71.0% to 71.0%, none of it generalised. Left in unseen-v5 on purpose or for want of a model: a request count as a SQL LIMIT, a fast hash compared with a stored digest, a delete in a helper behind an `AuthProtect` principal, a Yesod form-field object id, a gzip decompress of a request body, a Snap CSV export, stored XSS through an `IORef`, a `ReqBody` record's host field, and two policy shapes (command-line arguments to SQL, a command-line file's content to a partial pattern), listed in its USE-LOG. **Disclosure:** two frozen-holdout path-traversal sources were printed and read during the precision investigation (both still findings; no rule was changed to them); `scanner/src/language/CLAUDE.md` records it. The support registry was re-measured once on the frozen holdout with GHC 9.4.8 on PATH (the machine's default moved to 9.14.1 and records a false `auth` failure without it): every Haskell and Nix row stays supported at the same holdout numbers; the unseen-v3 development column reads Haskell taint 28 of 30 (the whole-handoff path shape, now information) and Nix 100% on both layers; `nix-eval` and `nixos-host` stay blocked on a host with no `nix`.
+
+**Not done, and why.** Installing `nix` on the maintainer's machine (the two blocked registry rows, and the fetch-hash remediation that needs it) requires an administrator password this session did not have; it is the one item of the plan left for the user. The CWE-916 migration fixer and the registry's own GHC version check were not started: the engine work above took the session. `test:haskell` 426/426, `test:nix` 305/305, `test:language` 257/257 (one pin updated: the fixed-host URL is now information), `test:lineage` 1554/1554, mutation 460/460, layer-recall unchanged. Two `package.json` test scripts were briefly wired with a lost space (two names glued) and ran two files short without failing; found and fixed before this entry, and recorded in memory.
+
+## 0.195.8 - Haskell: the unseen-v4 shapes are found, a blind unseen-v5 says most of that did not generalise; the NixOS job's status is stated honestly
+
+**Measured, 0.195.7:** commit to published GitHub release 9 min 44 s (`TIMINGS push=6s pr=3s checks=355s merge=4s verify=8s tag=4s release=192s npm=90s total=665s release-clock=208s`); the registry listed the version 4 min 14 s after the publish step. First run under the 10 minute requirement, with about 16 s of margin, on one measurement.
+
+Why: unseen-v4, the Haskell set written blind on 2026-10-26, reported F1 64.7% (recall 56.4%) against 100% on the frozen holdout, with `object-authorization`, `parser-safety` and `route-authentication` at zero recall, unchanged across three releases. The discipline for reading it is to write the next blind set first.
+
+- **unseen-v5 (78 Haskell forms, 13 families) was written by a separate agent session that could not see this repository**, frozen and measured once before any engine change: SAST 11 tp / 4 fp / 13 fn, taint 11 / 4 / 4, F1 63.8%, the same figures as unseen-v4. The builder, loader and summariser every frozen split uses are one shared module (`bench/language-support/corpora/frozen-split.mjs`; unseen-v4's files were rewritten over it and reproduce their 2026-10-26 digests), `measure.mjs` reads each split's role from its own manifest, and `test/language/frozen-splits.test.js` checks the digests, the per-family counts and the role of every split on every test run.
+- **unseen-v4 became a development set** (its USE-LOG records the use; `role: development` since 2026-11-01) and its misses were read. On it the engine went from 24 found of 39 to 37 of 39 (SAST 23 tp / 1 fp / 1 fn, taint 14 / 1 / 1, F1 94.9%). The rules are listed in `scanner/src/language/CLAUDE.md` and pinned both ways in `test/haskell/haskell-unseen-v4-shapes.test.js`: request reads inside a function are caller text for the partial-function and allocation rules; a presence test does not cover a split of the tested value; a count with a relational test is bounded; randomness reaching a token function's result; a record with a card field logged whole; wai-extra's size check; Servant closure arguments paired from the end; Map lookups and owner evidence (a column outside the WHERE clause, a record field never read) for object authorization, with or without an authenticated principal; framework-typed Scotty handlers registered elsewhere; Yesod `isAuthorized` clauses with a rejection path; `Auth '[JWT] Admin` as a role; the uploaded file name as a source; raw WAI responses as HTML sinks judged by their content type; escapers behind local bindings; and hamlet interpolations, which the parser now rewrites in place so `#{preEscapedToHtml body}` is analysed instead of being an opaque boundary. Two v4 labels are left as they were on purpose (`getArgs` to a shell stays `operator-controlled`; a URL wrapper stays a finding) and one v4 form is not found by decision (a POST whose only evidence of privilege was its own log text).
+- **unseen-v5 after that work: F1 65.7%** (SAST 12 / 4 / 12, taint 11 / 4 / 4; precision 74.2%, recall 59.0%), one more form found than its baseline. Most of the v4 fixes answered v4's shapes. The per-case output was not read; the next engine work on Haskell starts by writing unseen-v6. The author's completion report, which named per family the libraries and shapes it used, was read before the baseline; the USE-LOG says so.
+- **Two rules were tightened after a check on real code**, before anything was recorded: a state-changing route with no visible database, file or process call was reported when its handler called into unseen code, and on four pinned real projects that was 23 echo, upload-demo and health-check routes, so it now needs a privileged path; and owner-field evidence needed only a record with an owner field somewhere in the module, which flagged three stackage-server handlers beside an unrelated `Post` type, so the handler must read another field of that record. What remains new on those projects is the upload-name traversal in scotty's own upload example and three raw `responseLBS` HTML responses; `bench/haskell-precision` was re-recorded deliberately with those reviewed, the labelled precision rows unchanged.
+- **The support registry was re-measured** (`bench/language-support/promote.mjs`) after the engine work, as an engine change requires. The first run reported the Haskell `auth` row failed: `haskell-web-auth.test.js`'s compile criterion could not find `Web.Scotty`, because the machine's default `ghc` had moved to 9.14.1 since 2026-10-26 while the fixtures' packages live under GHC 9.4.8 (the recorded toolchain). With 9.4.8 first on the path the row is supported again and the registry recorded here is that run; the lineage registries' pinned counts moved by the three upload sources (672 to 678) and the two WAI response sinks (597 to 599).
+- **The first push of this release was stopped by the hosted checks at 427 s**, by the hosted tier working as described: `test/phase7-extensions.test.js` pins the set of catalog provenance values and did not know `upload`. Nothing was merged, tagged or published; the set gained the value, and `cross-service-taint.js` counts it among the HTTP provenances.
+- **`nixos-runtime` stays informational, for a stated reason.** It had been "informational until a recorded green run"; both legs were green on 12 consecutive main runs, so the reason is now the 10 minute release requirement: the x86_64 leg takes 4 to 8 minutes and starts with a network install. The job is no longer `continue-on-error`, so a failed step fails the job and the release gate's informational warning can see it. The arm leg never boots a guest (no KVM) and runs the real-evaluator harness, the flake build and the example scan on aarch64, which is by design and now written where the job is classified; the aarch64 guest runs emulated on the x86_64 leg.
+
 ## 0.195.7 - The two remaining release-path costs: the tag push stops repeating four pull-request checks, and language-slow is two jobs
 
 Why: 0.195.6 measured 11 min 32 s from the commit to the published GitHub release (push 8 s, hosted checks 391 s, merge to tag 17 s, release workflow 284 s). Two things held it over 10 minutes, and both are the same shape: work done twice.

package/dist/{966315fcdef8a956d535.json → 851efefbfcfa2a9aa691.json}

@@ -1,6 +1,6 @@
 {
   "name": "@clear-capabilities/agentic-security-scanner",
-  "version": "0.195.7",
+  "version": "0.195.9",
   "description": "Scanner engine for the agentic-security Claude Code plugin: SAST, SCA with reachability and CISA KEV, secrets, IaC, Haskell and Nix/NixOS, prompt-injection, MCP/agent-tool audit, authZ analysis, SBOM, compliance attestation, and more.",
   "type": "module",
   "main": "src/index.js",
@@ -93,9 +93,9 @@
     "test:lineage": "node --test test/lineage/analysis-completion.test.js test/lineage/schema.test.js test/lineage/ids.test.js test/lineage/protection.test.js test/lineage/classification.test.js test/lineage/validate.test.js test/lineage/json-schema-parity.test.js test/lineage/flagship-fixture.test.js test/lineage/flagship-fixture-semantics.test.js test/lineage/field-identity.test.js test/lineage/engine-expr-resolver.test.js test/lineage/engine-walker.test.js test/lineage/engine-integration.test.js test/lineage/engine-receiver-identity.test.js test/lineage/summaries.test.js test/lineage/driver.test.js test/lineage/engine-provenance.test.js test/lineage/engine-provenance-interprocedural.test.js test/lineage/path-store.test.js test/lineage/path-query.test.js test/lineage/flow-grade.test.js test/lineage/source-registry.test.js test/lineage/sink-registry.test.js test/lineage/transform-catalog.test.js test/lineage/handling-analyzer.test.js test/lineage/registry-real-code.test.js test/lineage/source-seeding.test.js test/lineage/graph-builder.test.js test/lineage/coverage.test.js test/lineage/resolve-destination.test.js test/lineage/queue-detail.test.js test/lineage/index.test.js test/lineage/obligation-mapping.test.js test/lineage/obligation-predicates.test.js test/lineage/obligation-predicates-walkthrough.test.js test/lineage/transit-protection.test.js test/lineage/at-rest-protection.test.js test/lineage/protection-summary.test.js test/lineage/ac01-multi-sink.test.js test/lineage/policy-verdict.test.js test/lineage/governance-refs.test.js test/lineage/export-privacy.test.js test/lineage/export-json.test.js test/lineage/export-csv.test.js test/lineage/decision-story.test.js test/lineage/export-briefing.test.js test/lineage/graph-snapshot.test.js test/lineage/graph-diff.test.js test/lineage/drift-policy.test.js test/lineage/recipient-profile.test.js test/lineage/recipient-registry.test.js test/lineage/recipient-wiring.test.js test/lineage/redact-graph.test.js test/lineage-scan-wiring.test.js test/lineage-artifact-write.test.js test/bench-data-lineage-runner.test.js test/bundle-frontend.test.js test/bundle-frontend-golden.test.js test/generate-html-report.test.js test/export-image.test.js test/lineage/scenario.test.js test/lineage/scenario-engine.test.js test/lineage/scenario-diff.test.js test/lineage/scenario-no-obligation-wiring.test.js test/lineage/impact-assessment.test.js test/lineage/impact-engine.test.js test/lineage/impact-no-obligation-wiring.test.js test/lineage/governance-edit.test.js test/lineage/remediation.test.js test/lineage/runtime-observation.test.js test/lineage/observation-correlation.test.js test/lineage/observation-adapters.test.js test/lineage/observation-store.test.js test/lineage/runtime-corroboration-wiring.test.js test/lineage/cross-repo-link.test.js test/lineage/federation-loader.test.js test/lineage/cross-repo-link-wiring.test.js test/lineage/runtime-assurance.test.js test/lineage/runtime-edge-states.test.js test/lineage/runtime-explorer-view.test.js test/cli/runtime-explorer-cli.test.js test/cli/runtime-identity-cli.test.js test/cli/runtime-twin-cli.test.js test/lineage/cut-planner.test.js test/lineage/scc.test.js test/lineage/scc-solver.test.js test/graph/graph-views.test.js test/lineage/dag07-completion.test.js test/lineage/owned-build.test.js test/dataflow/boundary-propagation.test.js test/lineage/boundary-contract.test.js test/lineage/correlation-v2.test.js test/lineage/path-control-evidence.test.js test/lineage/smt-z3-backend.test.js test/cli/dag-commands.test.js",
     "test:bench-modules": "node --test test/phase4-harness.test.js test/pipeline.test.js test/proof-corpus-lib.test.js test/proof-corpus-runner.test.js",
     "test:server": "node --test test/server/graph-loader.test.js test/server/security.test.js test/server/routes.test.js test/server/http-server.test.js test/server/cmd-explore.test.js test/server/cmd-dataflow-export.test.js test/server/static-assets.test.js test/server/dataflow-scenario-cli.test.js test/server/dataflow-impact-cli.test.js",
-    "test:haskell": "node --test test/haskell/haskell-reachability.test.js test/haskell/haskell-alias-taint.test.js test/haskell/haskell-weak-hash-names.test.js test/haskell/haskell-models-extended.test.js test/haskell/haskell-rules-extended.test.js test/haskell/haskell-injection.test.js test/haskell/haskell-ir.test.js test/haskell/haskell-manifests.test.js test/haskell/haskell-parser.test.js test/haskell/haskell-remediation.test.js test/haskell/haskell-remediation-0190.test.js test/haskell/haskell-resolved-graph.test.js test/haskell/haskell-sca.test.js test/haskell/haskell-sca-severity.test.js test/haskell/haskell-web-guards.test.js test/haskell/haskell-advisory-feed.test.js test/haskell/haskell-security-rules.test.js test/haskell/haskell-taint.test.js test/haskell/haskell-boundaries.test.js test/haskell/haskell-live-feed-scan.test.js test/haskell/haskell-supply-bounds.test.js test/haskell/haskell-ir-scaling.test.js test/haskell/haskell-boundary-propagation.test.js test/haskell/haskell-generalization.test.js test/haskell/haskell-trust-boundary.test.js test/haskell/haskell-comment-invariance.test.js test/haskell/haskell-remediation-classes.test.js test/haskell/haskell-trust-boundary-frameworks.test.js test/haskell/haskell-generic-detectors.test.js",
-    "test:nix": "node --test test/nix/nix-option-types.test.js test/nix/nixos-hardening-daemon.test.js test/nix/nixos-hardening-extended.test.js test/nix/nix-build-trust.test.js test/nix/nix-eval-isolation.test.js test/nix/nix-input-inventory.test.js test/nix/nix-parser-ir.test.js test/nix/nix-remediation.test.js test/nix/nix-resolved-closure.test.js test/nix/nix-sca-patches.test.js test/nix/nix-sca-severity.test.js test/nix/nix-haskell-closure-names.test.js test/nix/nix-script-taint.test.js test/nix/nix-secrets-store.test.js test/nix/nixos-hardening.test.js test/nix/nixos-module-resolution.test.js test/nix/nix-fix-multifile.test.js test/nix/nixos-evaluator-forms.test.js test/nix/nix-advisory-feed.test.js test/nix/nix-fix-prewrite.test.js test/nix/nix-boundary-scope.test.js",
-    "test:language": "node --test test/bench-haskell-precision.test.js test/bench-independent-haskell-panel.test.js test/language/capability-ledger.test.js test/language/language-accuracy-regression.test.js test/language/language-aibom.test.js test/language/language-assurance.test.js test/language/language-bom.test.js test/language/language-compliance.test.js test/language/language-contracts.test.js test/language/language-corpora-integrity.test.js test/language/language-discovery.test.js test/language/language-doc-coverage.test.js test/language/language-evidence-posture.test.js test/language/language-explorer.test.js test/language/language-gap-fixes.test.js test/language/language-gap-fixes-2.test.js test/language/language-governance.test.js test/language/language-integrations.test.js test/language/language-llm-agent.test.js test/language/language-local-ai.test.js test/language/language-parity-haskell.test.js test/language/language-polyglot-bridges.test.js test/language/language-privacy-lineage.test.js test/language/language-proof-remediation.test.js test/language/language-report-formats.test.js test/language/language-secrets.test.js test/language/language-trust-boundary.test.js",
+    "test:haskell": "node --test test/haskell/haskell-reachability.test.js test/haskell/haskell-real-code-shapes.test.js test/haskell/haskell-unseen-v5-shapes.test.js test/haskell/haskell-unseen-v4-shapes.test.js test/haskell/haskell-alias-taint.test.js test/haskell/haskell-weak-hash-names.test.js test/haskell/haskell-models-extended.test.js test/haskell/haskell-rules-extended.test.js test/haskell/haskell-injection.test.js test/haskell/haskell-ir.test.js test/haskell/haskell-manifests.test.js test/haskell/haskell-parser.test.js test/haskell/haskell-remediation.test.js test/haskell/haskell-remediation-0190.test.js test/haskell/haskell-resolved-graph.test.js test/haskell/haskell-sca.test.js test/haskell/haskell-sca-severity.test.js test/haskell/haskell-web-guards.test.js test/haskell/haskell-advisory-feed.test.js test/haskell/haskell-security-rules.test.js test/haskell/haskell-taint.test.js test/haskell/haskell-boundaries.test.js test/haskell/haskell-live-feed-scan.test.js test/haskell/haskell-supply-bounds.test.js test/haskell/haskell-ir-scaling.test.js test/haskell/haskell-boundary-propagation.test.js test/haskell/haskell-generalization.test.js test/haskell/haskell-trust-boundary.test.js test/haskell/haskell-comment-invariance.test.js test/haskell/haskell-remediation-classes.test.js test/haskell/haskell-trust-boundary-frameworks.test.js test/haskell/haskell-generic-detectors.test.js",
+    "test:nix": "node --test test/nix/nix-option-types.test.js test/nix/nix-development-shapes.test.js test/nix/nixos-hardening-daemon.test.js test/nix/nixos-hardening-extended.test.js test/nix/nix-build-trust.test.js test/nix/nix-eval-isolation.test.js test/nix/nix-input-inventory.test.js test/nix/nix-parser-ir.test.js test/nix/nix-remediation.test.js test/nix/nix-resolved-closure.test.js test/nix/nix-sca-patches.test.js test/nix/nix-sca-severity.test.js test/nix/nix-haskell-closure-names.test.js test/nix/nix-script-taint.test.js test/nix/nix-secrets-store.test.js test/nix/nixos-hardening.test.js test/nix/nixos-module-resolution.test.js test/nix/nix-fix-multifile.test.js test/nix/nixos-evaluator-forms.test.js test/nix/nix-advisory-feed.test.js test/nix/nix-fix-prewrite.test.js test/nix/nix-boundary-scope.test.js",
+    "test:language": "node --test test/bench-haskell-precision.test.js test/bench-independent-haskell-panel.test.js test/language/capability-ledger.test.js test/language/language-accuracy-regression.test.js test/language/language-aibom.test.js test/language/language-assurance.test.js test/language/language-bom.test.js test/language/language-compliance.test.js test/language/language-contracts.test.js test/language/language-corpora-integrity.test.js test/language/language-discovery.test.js test/language/language-doc-coverage.test.js test/language/language-evidence-posture.test.js test/language/language-explorer.test.js test/language/frozen-splits.test.js test/language/language-gap-fixes.test.js test/language/language-gap-fixes-2.test.js test/language/language-governance.test.js test/language/language-integrations.test.js test/language/language-llm-agent.test.js test/language/language-local-ai.test.js test/language/language-parity-haskell.test.js test/language/language-polyglot-bridges.test.js test/language/language-privacy-lineage.test.js test/language/language-proof-remediation.test.js test/language/language-report-formats.test.js test/language/language-secrets.test.js test/language/language-trust-boundary.test.js",
     "test:language-stress": "node --test --test-concurrency=1 test/language/language-stress-offline.test.js",
     "test:language-tools": "node --test test/haskell/haskell-web-auth.test.js test/haskell/haskell-support-gate.test.js",
     "bench:language-support:check": "node ../bench/language-support/check.mjs",