@clear-capabilities/agentic-security-scanner 0.155.0 → 0.157.0

package/CHANGELOG.md

@@ -9,6 +9,36 @@
 > make the history less accurate, not more.
 
 
+## 0.157.0 - hosted verification for the suites that need Nix or NixOS
+
+NIX-011 needs `nix` and NIX-012 needs a booted NixOS, which a developer's laptop often lacks, so the loop controller could only ever report them as blocked. This release lets the controller run them on a GitHub-hosted runner and record the outcome as evidence, bound to the exact commit.
+
+- **Hosted-CI executor** (`scripts/loop-engineering/lib/remote.mjs`, `.github/workflows/verify-remote.yml`). A suite whose profile entry has a `remote` block runs on a hosted runner when the required tool is missing locally. The controller refuses unless the tree is clean and HEAD is the tip of a pushed branch, dispatches the committed workflow with the commit, the requirement, the watch globs and a nonce, finds ITS run by the nonce, requires the runner's digest of the watched files to equal its own (the runner uses `scripts/loop-engineering/digest.mjs`, the controller's own code), reads per-leg TAP, and requires every leg to pass every criterion. Evidence is labelled `controller+hosted-ci` with the run id, artifact digest and a limitation line; it is never presented as a local run. Dispatch and download are retried a few times on a reset connection (GitHub resets one now and then); a real error is not retried. A failed run, a wrong commit or digest, a missing leg or file, a hash mismatch or a skipped test fails closed; if the preflight cannot be met the requirement is blocked, not passed.
+- **NixOS guests.** The NIX-012 suite runs under a TAP reporter inside each guest and the output is copied out. NIX-012 has two legs: an x86_64 guest on KVM, and an aarch64 guest emulated in software on the x86_64 host (the criterion allows "actual tested emulation"; arm runners have no KVM). The test driver's fixed five-minute shell wait is stretched for the emulated guest, and the guest test asserts `uname -m`, so a leg's name cannot be a false label.
+- **Docs.** `docs/guides/loop-engineering.md` has a "Hosted verification" section stating what the mode requires and what it trusts (GitHub's account of what ran, and the committed workflow).
+
+- **The executor never blocks the controller and survives sleep.** Its `gh` and `git` calls are asynchronous (a synchronous call froze the controller's heartbeat for as long as GitHub took to answer), and every deadline uses a monotonic clock, so a machine that sleeps cannot expire a ten-minute wait before the process has polled once (a real failure seen during this release's own verification).
+- **The status says which requirements were verified remotely.** `docs/completion-status.json` now records the hosted-verified requirements (run id, URL, commit, legs) and the README status block says so, so "all verified" is never read as "all verified on this machine".
+- **The model-spend cap gates model work only.** The controller checked its $50 model budget at the top of its loop, so a run that had spent it could not even re-validate stale evidence or run the final phase, neither of which uses the model. The cap is unchanged and still refuses any further worker attempt; the wall-clock and attempt budgets still gate everything.
+
+This entry deliberately says nothing about how many requirements are verified: that number is derived from the controller's evidence and is in `docs/completion-status.json` and the README status block.
+
+## 0.156.0 - the three below-target rows fixed, measured on a fresh unseen set
+
+0.155.0 reported Haskell SAST recall (79.2%), Haskell taint precision (77.8%) and Nix config-sast precision (86.1%) below target on shapes the templates did not contain, while the frozen holdout said 100%. Each gap was read against the code and fixed in the engine, not in the scoring.
+
+- **Haskell, structural rules.** A function named for a PIN or password is a security value for the randomness rule (matched as a whole word, so `spinWheel` is not); `replicateM`/`replicateM_` sized by parsed caller text is an unbounded allocation; `xs !! n` on a caller-supplied list is a partial function; a log argument labelled `"token=" ++ x` or `"password: " ++ x` is a credential, unless it is redacted.
+- **Haskell, route authentication.** A wrapper guard (`guarded $ do ...`) now dominates the handler it encloses: program order is source order, not control-flow node order. A guard that runs AFTER the write is still reported.
+- **Haskell, taint.** `lookup key table` returns an element of the table, so only the table flows to the result (the key stays visible to rules as `hs.selector`, so `fromJust (lookup key t)` is still a partial-function finding); `length`, `null`, `ord` and similar return numbers and carry no caller text; a host allow-list (`uriRegName a `elem` [...]`) over a URL parsed from the checked value is a URL guard.
+- **Nix.** `security.sudo.extraRules` with `NOPASSWD` for `ALL` is a finding (one named command is not); a TLS key produced by `writeText`/`toFile` is in the store like a path literal; wpa_supplicant's `ext:NAME` is a reference to a runtime variable, not a credential.
+- **The measurement.** The unseen-v1 shapes were used to make these fixes, so they are now the `shape-dev` regression set. A fresh unseen-v2 set (3 vulnerable and 3 safe NEW single-flaw shapes per family, 2 nouns each, written after the fixes) was measured ONCE with the fixed engine: Haskell SAST P 91.7% / R 91.7%, Haskell taint P 93.3% / R 93.3%, Nix config-sast P 100% / R 97.0%; all three meet the targets (precision 90%, recall 85%). The frozen holdout is unchanged at 100% (re-measured after the engine change, which counts as repeated holdout use; nothing was tuned against it). Intervals on the unseen numbers are now over shapes, not near-identical cases.
+- **What the fresh set still misses (not fixed, because fixing them would consume it).** Haskell: `strictRequestBody` read through an explicit import list, `fromJust (lookup k t)` (measured before the key-visibility fix above), `when (isNothing h) (... >> finish)` and `X-Api-Key` header checks reported as missing authentication, `getWith opts url` (the URL is the second argument), and duplicate findings for `readCreateProcess (shell ...)`. Nix: a firewall opened by a full-range `allowedTCPPortRanges`. The causes were not investigated (that would consume the set); these are the next known gaps. One engine change, the `fromJust (lookup ...)` regression fix, was made after the unseen measurement because the `lookup` change itself had caused it; the stored numbers were not re-measured, so they are conservative, and the set should be treated as consumed for anything after this release (an unseen-v3 is needed for the next round).
+- **Author errors in the retired set, named.** A cookie called `usersses` (nothing in the code says it is a session), an HTML shape that prints to stdout, vulnerable Nix shapes with a second real flaw, and a path shape with two sinks. They stay in `shape-dev` with a note; they were not "fixed" in the engine.
+
+- **A flaky probe, fixed.** The Haskell witness runner's `runghc --version` probe gave up after 10 s, which a cold CI runner exceeded once: the toolchain then read as absent in one call and present in the next, failing a test that asked twice. The probe now allows 60 s, says so when it times out, and the test decides availability once.
+
+Partial release: the loop controller still lists NIX-011, NIX-012 and REL-001 as not verified. NIX-011 and NIX-012 passed on hosted CI (the NIX-012 suite inside a real NixOS guest on x86_64; no aarch64 guest was run), which is CI evidence, not controller-signed evidence. REL-001 needs those to be controller-verified first.
+
 ## 0.155.0 - premortem remediation (trust boundary, unseen-shape measurement), NixOS VM check
 
 An adversarial premortem of 0.154.0 found the following; each is fixed here and covered by `test/language/language-trust-boundary.test.js`.

package/dist/8798.index.js

@@ -174,8 +174,10 @@ function judgeFixWitness(before, after) {
 function haskellRunner() {
   const p = which('runghc') || which('runhaskell');
   if (!p) return { available: false, reason: 'no Haskell toolchain (runghc) on PATH; witnesses for Haskell findings are unavailable here' };
-  const v = (0,node_child_process__WEBPACK_IMPORTED_MODULE_3__.spawnSync)(p, ['--version'], { encoding: 'utf8', timeout: 10_000 });
-  return v.status === 0 ? { available: true, path: p, version: String(v.stdout || v.stderr).trim() } : { available: false, reason: 'the Haskell runner did not start' };
+  // GHC's first start on a cold, loaded machine (a CI runner) has taken more than 10 s; a probe that gives up there reports a toolchain
+  // that is installed as absent, and the answer then differs between two calls in one run.
+  const v = (0,node_child_process__WEBPACK_IMPORTED_MODULE_3__.spawnSync)(p, ['--version'], { encoding: 'utf8', timeout: 60_000 });
+  return v.status === 0 ? { available: true, path: p, version: String(v.stdout || v.stderr).trim() } : { available: false, reason: v.error && v.error.code === 'ETIMEDOUT' ? 'the Haskell runner did not start within 60 s' : 'the Haskell runner did not start' };
 }
 
 /**