@clear-capabilities/agentic-security-scanner 0.149.4 → 0.150.2

package/CHANGELOG.md

@@ -10,6 +10,144 @@
 
 
 
+## 0.150.2 - Two adversarial-review passes on 0.150.0's Ollama support: redaction, disclosure, and robustness fixes
+
+0.150.0's offline-Ollama release was put through two rounds of adversarial premortem review after
+shipping. Both surfaced real gaps; this release closes them.
+
+**Security — secret redaction (the most important fix here).** The general-purpose `redactSecrets`
+function every Ollama-backed role (`fix`/`explain`/`poc`, the `ask` agent loop's `read_file`/
+`search_code` tools) routes untrusted file content through before it reaches a model had several
+real gaps, found and closed in order as review went deeper:
+
+1. Unquoted `.env`/shell-export syntax (`DB_PASSWORD=x`, no quotes) evaded the existing
+   quote-requiring pattern entirely — this was a previously-documented, never-fixed-at-the-general-
+   level gap (`secret-redaction.test.js`'s own long-standing comment named it).
+2. Compound identifiers joined by `_`/`-` (`DB_PASSWORD`, `STRIPE_API_KEY`) didn't match even in the
+   already-supported quoted form, because a plain `\b` treats `_` as a word character.
+3. A second review pass found the above fix was narrower than the real threat surface: **camelCase**
+   compounds (`authToken`, `apiSecret`), a **JSON-quoted key** (`"password": "value"` — the closing
+   quote wasn't consumed before the operator), **YAML** `key: value` syntax (added, deliberately
+   scoped to `.yml`/`.yaml` files only, to avoid colliding with the already-tested TypeScript
+   type-annotation and JS object-literal-key exclusions), and a secret **split across a string
+   concatenation** (`"Super" + "Secret123"` — only the first segment was redacted, leaking the tail)
+   all still leaked. All four are fixed, each with an explicit false-positive regression test
+   (`primaryKey`/`cacheKey` must not redact; `password: string;` in a `.ts` file must not redact;
+   ordinary string-building with no secret-shaped key name must not redact).
+
+**Governance — the "AI Assistance" report's role-scope disclosure had a blind spot.** 0.150.0 added
+a report block scoped to the `validate` role, with a warning when another role resolves to a remote
+provider — but `hunt` (the heaviest LLM-driven discovery work) is deliberately not one of
+`providers.js`'s per-role-overridable roles, so it could never appear in that warning, by
+construction. `hunt` is now checked explicitly.
+
+**Robustness.**
+- The wall-clock/per-call timeout mismatch in the `ask` agent loop (a slow call could exceed the
+  loop's own overall budget, and raising the documented remedy for slow models did nothing to help)
+  is fixed and was verified against two real installed Ollama models on real hardware
+  (`gemma4:e4b`, `qwen3.5:9b`) — both complete a live tool-calling round trip within the new budget.
+- The capability-probe cache now has a 30-day safety-net TTL and a `--force` flag
+  (`agentic-security models test <model> --force`), on top of its existing version+digest+name key.
+- Added `AGENTIC_SECURITY_OLLAMA_DISABLED=1`, a kill switch checked in the one function every Ollama
+  call path resolves through — closes a gap where a per-role override could survive an operator
+  unsetting only the global preset during an incident.
+- `agentic-security fix`'s Ollama-proposed-patch path now retries exactly once with the
+  deterministic gate's own rejection reason fed back into the prompt, instead of asking an
+  identical (temperature-0) question and getting an identical bad patch back. Automated test
+  coverage added (`test/cli/fix-retry.test.js`); previously this had only ever been verified by a
+  single manual run.
+- A real, observed out-of-memory failure is now recorded per-model and surfaced as a warning
+  wherever that model is used again — including proactively in the `ask` loop itself, not only in
+  `models doctor` (a user who never runs `doctor` used to get no warning at all before repeating the
+  same failure).
+- Built the PRD's own required opt-in live-model contract tier (`AGENTIC_SECURITY_OLLAMA_E2E=1 npm
+  run test:ollama-e2e`), which did not exist at 0.150.0 despite being an explicit requirement — run
+  against two real installed models, all 7 required contract checks pass. This tests the WIRE
+  CONTRACT (valid structured output, a real tool-call round trip, prompt-injection resistance,
+  graceful context-overflow handling), not output QUALITY — a model-quality benchmark suite remains
+  future work, and `docs/guides/ollama.md` says so explicitly rather than letting contract-passing
+  read as quality-proven.
+
+**Documentation.** `docs/guides/ollama.md` gained an honest statement of the Ollama supply-chain
+trust boundary (no Sigstore-equivalent verification exists for the `ollama` binary or pulled model
+weights, unlike this project's own npm dependencies) and a doc-drift test
+(`test/ollama-doc-drift.test.js`) that fails if the guide's stated numeric defaults (timeouts,
+context sizes, cache TTL, the tool-loop iteration ceiling) ever diverge from the real source
+constants — verified by deliberately breaking one and confirming the test catches it.
+
+**Clarification, not a correction, to 0.150.0's own wording** (this project's changelog convention
+is to never rewrite a past entry): "the same offline guarantee the existing `local` preset already
+made — nothing leaves the machine" describes LLM inference traffic only. The same scan still makes
+separate, deterministic OSV/KEV/EPSS network calls unless `AGENTIC_SECURITY_OFFLINE=1` is also set —
+`docs/guides/ollama.md` already said this; this entry says it here too, next to the original claim.
+Separately, PRD §26 asked for a combined validate/verify/explain/fix/hunt "AI Assistance" summary;
+what shipped in 0.150.0 measures the `validate` role only (the one role that runs automatically
+during a plain `scan`) — `docs/guides/ollama.md` and this entry disclose the scope-down; the other
+five roles are each single-finding or discovery-run commands with no natural combined-scan moment to
+summarize.
+
+## 0.150.1 - Fix 0.150.0's release-gate failure: @babel/core dependency currency
+
+0.150.0's tag push failed `release.yml`'s gate on its one network-dependent, deliberately-excluded-
+from-local-pre-push check: `scanner`'s `@babel/core` had drifted to 8.0.1 against a since-published
+8.0.5 on the registry — nothing to do with the Ollama work itself, just registry drift between when
+local verification ran and when the tag reached the clean CI runner. `npm install @babel/core@8.0.5`
+(already within the existing `^8.0.1` semver range) resolves it; `npm run test:smoke` and
+`npm run test:sast` (731 tests) both pass unchanged against the bump. No functional change beyond
+the dependency bump — see 0.150.0's entry for the actual release content.
+
+## 0.150.0 - Offline Ollama support: local models as a first-class provider, not a workaround
+
+Every LLM-backed role in this project (`validate`, `verify`, `explain`, `fix`, `poc`, `logic`, `hunt`)
+can now run entirely against a local Ollama install, with the same offline guarantee the existing
+`local` preset already made — nothing leaves the machine unless an operator explicitly opts into a
+remote host — and with zero change to `local`/BYO/`anthropic`/`openai`/`gemini` behavior.
+
+1. **New `ollama` provider** (`src/llm-validator/ollama-provider.js`), a distinct preset from `local`:
+   speaks Ollama's native `/api/chat` (messages array, JSON-schema-constrained structured output,
+   native tool calling, `think`, `keep_alive`), not the generic `{prompt, model}` shape `local` keeps
+   forever for its own existing consumers. Loopback enforced by default, with a named escape hatch
+   (`AGENTIC_SECURITY_OLLAMA_ALLOW_REMOTE=1`) for an explicitly-configured remote Ollama server —
+   never an accidental one. `AGENTIC_SECURITY_LLM_PRESET=ollama` plus per-role `_MODEL_<ROLE>`
+   overrides work exactly like every existing provider.
+2. **Every role now has a real call site.** `fix`/`explain`/`poc` had reserved env vars and nothing
+   that ever invoked them; each is real now: `agentic-security fix` falls back to an Ollama-proposed
+   patch when no deterministic one exists (verified through the exact same `applyVerifiedFix`
+   rescan/lint/test gate a deterministic patch already goes through — no new bypass path),
+   `agentic-security triage --explain <id>` and `--poc <id>` add read-only, narrative-only AI
+   assistance to triage, visually separated from deterministic evidence and forbidden from
+   fabricating exploit confirmation, cost, or compliance claims.
+3. **Model capability detection**, three layers, cheapest first: Ollama's own `/api/show` metadata
+   (authoritative where present), a non-authoritative family hint (Qwen 3.5, Qwen 3 Coder, Gemma 4,
+   FunctionGemma, and more — read from what's installed, never a hardcoded allowlist), and an opt-in
+   runtime probe (a real structured-output request, a real one-tool-call request) cached to disk by
+   Ollama version + model digest + model name so it never repeats needlessly. `agentic-security
+   models test <model>` runs it on demand; `models doctor`/`models inspect --probe` surface it.
+4. **RAM-aware model admission** — 8GB and 16GB (Qwen/Gemma) memory profiles, each trying its target
+   context, then a reduced context, then a smaller model, before ever falling back to
+   deterministic-only. Never falls back to a cloud provider under any circumstance.
+5. **A bounded local agent loop** (`agentic-security ask "<question>" [target]`): free-form Q&A with
+   read-only tool access (`read_file`, `list_files`, `search_code`, `read_finding`) confined to the
+   scan root, symlink-safe, secret-redacted, every tool result framed as untrusted data before it
+   re-enters the model's context. No write or execute tool exists in this cut. Hard-terminates after
+   12 tool-call iterations or a 5-minute wall clock, whichever comes first; a model requesting an
+   unregistered tool ends the loop immediately as a policy violation, not a retry.
+6. **`agentic-security setup --llm ollama [--model <name>]`** discovers the local server, lists
+   installed models, and picks a sane default for the detected RAM tier (or uses the one you name) —
+   noninteractive throughout, matching every other command in this CLI, rather than introducing this
+   codebase's first interactive prompt.
+7. **Per-scan AI Assistance reporting.** Fixed a real, pre-existing gap along the way: the validator's
+   own provider matrix and per-finding model-status summary were computed on every scan and then
+   silently discarded before reaching any report. `agentic-security scan` now prints provider, model,
+   egress mode, and validate-call success/refusal/failure counts when a model tier actually ran —
+   worded as "LLM inference was loopback-only," never as "this scan was fully offline," which is a
+   separate claim about the deterministic OSV/KEV/EPSS network path this line does not describe.
+
+Full docs at `docs/guides/ollama.md`. See `agentic-security-ollama-offline-prd.md`'s own Definition of
+Done section for the complete requirements matrix this release was built against; local benchmark
+suite / model-quality-history-by-digest (the PRD's own explicitly lowest-priority stretch scope) are
+the one tracked remainder.
+
 ## 0.149.4 - Fix 0.149.3's vendor/ gap: populate it in `prepare`, not only `build`
 
 0.149.3's `vendor/java-parser/` was only ever populated by `npm run build` (`scripts/vendor-java-parser.mjs`