fullshield 0.2.6 → 0.4.0

package/CHANGELOG.md

@@ -2,6 +2,105 @@
 
 All notable changes to FullShield are documented here.
 
+## 0.4.0 — 2026-10-18
+
+Phase 46 still enforces Node >= 22.13 in `engines`, `doctor`, and CI. Phase 47 still owns the console view.
+
+### New/changed stable findings — may change CI results; use --update-baseline
+
+Same-function Mongo filters are reported by the existing `SAST-NOSQLI-001` rule (approved under D7). vulnerable-node gains two keys: `routes/shop.js` lines 41 and 58, both high. Every other tuning app is unchanged. Ground truth was not edited.
+
+### Upgrade plan, evidence, explain, SBOM, supply heuristics, lockfile diff, modern lockfiles
+
+Phases 49, 61, 60, 66, 70, 62, and 63. The upgrade plan is a sentence built from OSV `fixed` versions. The other six are off unless their `FULLSHIELD_EXPERIMENTAL_*` flag is set. Default `(ruleId, file, line, severity)` keys are unchanged.
+
+### Validation, workspaces, custom taint, summaries, SARIF
+
+Phases 53, 54, 64, 65, and 74. Each stays off unless its flag is set. Phase 54 reads `package.json` workspaces. Phase 65 caches taint summaries; the 5-second nest and undici target was not met. Default scan keys stay the same when the flags are off.
+
+### Unused direct dependencies
+
+A scan names direct `dependencies` that no project file imports and that no imported package requires. Imported and imported-not-called packages stay out. `fix` prints the removal plan and writes nothing. `fix --unsafe` deletes those names from `package.json` only, leaves the lockfile alone, and prints `run npm install`. The finding list is unchanged, including a `DEPS-MALICIOUS-001` on an unused package, which stays critical.
+
+### Console
+
+Default console prints the human-decision shown rows (2026-10-18), grouped one row per package name. JSON, SARIF, Markdown, HTML, the baseline, and `--fail-on` stay the full list. `--all` prints every finding, ungrouped. This is not a stable-output change.
+
+Recount on `next` after restore-0.4.0, from `scan --json --offline-safe`, unreviewed. The counts match the 2026-10-18 decision (D4). There is no row-count threshold. The shown rule, the 20-row cap, and severities were not changed.
+
+| app | shown rows | shown findings | folded findings | shown findings by reachability |
+|---|---|---|---|---|
+| nodegoat | 36 | 62 | 194 | reachable-via-dependency 41, reachable 12, none 8, not-imported 1 |
+| juice-shop | 49 | 59 | 46 | none 46, reachable 13 |
+| dvna | 14 | 16 | 7 | none 12, reachable 4 |
+| goof | 63 | 157 | 114 | reachable-via-dependency 86, reachable 60, none 11 |
+| vulnerable-node | 41 | 43 | 20 | none 38, reachable-via-dependency 4, reachable 1 |
+
+## 0.3.0 — 2026-10-16
+
+### Default scan changes
+
+Stable finding keys changed. CI baselines may need `--update-baseline`. This is not a recall claim. Ground truth was not edited and nothing was marked reviewed.
+
+Before (`bench/snapshots/0.1.3.json` / previous `HEAD.json`, `scan --json --offline-safe`):
+
+| app | findings |
+|---|---|
+| vulnerable-app | 35 |
+| clean-app | 0 |
+| nodegoat | 379 |
+| juice-shop | 105 |
+| dvna | 23 |
+| goof | 354 |
+| vulnerable-node | 61 |
+
+After (`bench/snapshots/HEAD.json`, same flags):
+
+| app | findings |
+|---|---|
+| vulnerable-app | 35 |
+| clean-app | 0 |
+| nodegoat | 256 |
+| juice-shop | 105 |
+| dvna | 23 |
+| goof | 271 |
+| vulnerable-node | 61 |
+
+Unique `(ruleId, file, line, severity)` keys did not change. The count drop is duplicate keys: several versions of one package and advisory shared one package.json line and one severity. In-scope matches on the unreviewed ground truth stayed at NodeGoat 5/7 (plus 1 line-only), Juice Shop 10/12, DVNA 9/10, goof 11/11, vulnerable-node 9/12. See `bench/snapshots/DIFF.txt`.
+
+- Dependency advisories are one finding per package name + advisory. Every retained dependency path is listed on that finding. Severity is the highest of the grouped versions. Rules were not removed and severities were not lowered to cut the count.
+- SAST findings with the same rule id and sink line collapse to one.
+- Default walk skips `node_modules`, `dist`, `build`, `.next`, `coverage`, `vendor`, `vendored`, `third_party`, `third-party`, `compiled`, and `*.min.js`. Set `"defaultExcludes": false` or replace the list with `"excludes"`.
+- Files over `sast.maxFileBytes` (default 1 MB), and files that contain NUL bytes, are reported as `SAST-SKIP-001` (info) instead of being skipped silently.
+- Repos with more than 4000 source files are parsed one file at a time so the AST is released before the next file. Cross-file taint on those repos runs in batches of 150. Smaller repos, including the in-scope bench apps, keep the previous all-files parse.
+- The parse pool keeps an LRU of 64 ASTs, caps workers (default 2, explicit max 4), and caps each worker heap (`FULLSHIELD_PARSE_WORKER_MB`, default 512).
+
+### Experimental CI reports
+
+Off unless `FULLSHIELD_EXPERIMENTAL_CI_REPORTS=1` or `scan --experimental-ci-reports`. Default `scan` and `--json` are unchanged.
+
+- `--github-annotations` writes GitHub Actions workflow commands. Critical and high are `error`, medium is `warning`, low and info are `notice`.
+- `--junit` writes JUnit XML. Medium and above are failures. Low and info are skipped. An empty scan is one passing testcase.
+- `--gitlab` writes a GitLab Code Quality report. Severities map to info, minor, major, critical, and blocker.
+
+### Experimental scan diff
+
+Off unless `FULLSHIELD_EXPERIMENTAL_DIFF=1` or `fullshield diff --experimental-diff`. It compares two `scan --json` files. It does not scan and does not use the network.
+
+- Match key is rule id, file, severity, and the snippet or message. The line number is not part of the key. Two findings with that same text stay two results. A line shift is a move. One shared finding id with a different message is a move. Duplicate ids are not paired.
+- Added findings at or above `--fail-on` (default `high`) exit 1. AI findings, moves, and removals do not. The human report lists at most 20 added findings.
+- Diff matching and the GitLab fingerprint ignore the line number. A changed message with the same id is a move.
+- Console, JSON, SARIF, Markdown, HTML, GitHub, JUnit, and GitLab reports mask AWS, GitHub, Stripe, and Slack token shapes before writing them.
+- `.fullshield-baseline.json` may list `suppressions` with `ruleId`, `owner`, `reason`, and ISO `expires`. Expired or incomplete rows do not hide findings and fail the scan.
+- `FULLSHIELD_EXPERIMENTAL_IAC=1` also flags `github.event` inside `run:` steps and a `pull_request_target` job that checks out pull-request code.
+- Lockfile helpers flag `http://` or a non-registry `resolved` URL, and a resolved entry with no integrity hash.
+- `scan --history` stops after 500 commits (`FULLSHIELD_HISTORY_MAX`, max 5000). When older commits are left out, the console and the JSON metadata say `scanned N of M commits`. History snippets are masked.
+- `FULLSHIELD_EXPERIMENTAL_LICENSE=1` flags an installed direct dependency whose package.json says GPL or AGPL.
+
+### Node 20
+
+- `engines` is `>=20.11`. `node:sqlite` is the only runtime API that needs Node 22.13. History storage prints that requirement. Confirm does not import `node:sqlite`. RASP sqlite hooks no-op when the module is missing. CI runs Node 20, 22, and 24.
+
 ## 0.2.6 — 2026-10-16
 
 - v0.2.5 reached staging with a fresh token, then sigstore threw `TLOG_CREATE_ENTRY_ERROR` because Rekor already had that transparency-log entry (HTTP 409). npm 12.2.0 leaves `fetchOnConflict` off, so the publish stopped. Staging now turns that on before loading the publisher, and gives the Rekor request 120 seconds. It still refuses to publish without provenance.

package/README.md

@@ -1,6 +1,6 @@
 # FullShield
 
-**v0.2.6 / alpha.** One install for Node.js/TypeScript backends: detect → classify → report → notify → fix → gate (CI `--fail-on` + runtime monitor). It does not stop attacks by itself.
+**v0.4.0 / alpha.** One install for Node.js/TypeScript backends: detect → classify → report → notify → fix → gate (CI `--fail-on` + runtime monitor). It does not stop attacks by itself.
 
 **Headline: dependency reachability.** A lockfile CVE is not the same as an unused package. FullShield classifies each OSV hit as `reachable` (imported and called), `imported-not-called` (−1), `reachable-via-dependency` (lockfile path from an imported direct dep — **severity kept**, path shown, e.g. `express > body-parser > qs`), `not-imported` (−2, only when no such path exists), or `reachability-unknown` (severity kept) when the lockfile format has no parsed dependency edges. DevDependency-only vulns are `info` with a `dev-only` note unless production sources import the package. Pair it with `fullshield sbom --format cyclonedx`.
 
@@ -12,7 +12,7 @@ npx fullshield scan --sarif fullshield.sarif
 
 This is a `0.x` release. APIs may change. Do not treat it as a penetration test.
 
-**Requires Node.js >= 22.13.** `node:sqlite` runs without `--experimental-sqlite` from 22.13.0 (the module is still experimental and is loaded only by history/confirm, not by `--import` register).
+**Supported runtime is Node.js >= 22.13.** `package.json` `engines` and `doctor` enforce that floor. `node:sqlite` runs without `--experimental-sqlite` from 22.13.0 (the module is still experimental and is loaded only by history/confirm, not by `--import` register).
 
 ## Coverage map
 
@@ -68,7 +68,7 @@ This is a `0.x` release. APIs may change. Do not treat it as a penetration test.
 
 ## Known limitations
 
-- Platforms: Linux, macOS, and Windows on Node 22.13+. Quoted filenames in dirty-tree tests are POSIX-only (Windows forbids `"` in names).
+- Platforms: Linux, macOS, and Windows on Node >= 22.13. Quoted filenames in dirty-tree tests are POSIX-only (Windows forbids `"` in names).
 - `node:sqlite` is loaded only by history/confirm, not by `--import fullshield/runtime/register`. Dashboard closes every handle in `try/finally` (Windows cannot delete an open `history.db`).
 - FullShield does not stop OS or SSH exploits (including regreSSHion). It scans Node/TS apps you own.
 - Extra classes (ReDoS, deserialization, XXE, upload, cookie flags, CSRF, mass assignment, timing compare, hash cost, GraphQL limits, `trust proxy` + IP rate limit) run only with `FULLSHIELD_EXPERIMENTAL_CLASSES=1` or `scan --experimental-classes`. ReDoS is a same-file static quantifier check. CSRF looks for `csurf` or `sameSite: 'strict'|'lax'` in that file.
@@ -92,7 +92,7 @@ In-scope public-bench patterns we still do not claim (one line each):
 - Juice Shop `data/static/codefixes/*` — six `SAST-NOSQLI-001` hits are copies of the challenge fixes. They stay. There is no special-case exclusion for that directory.
 - `DEPS-VULN-001` — when an OSV hit exists, the finding is attributed to the `package.json` name line; `--offline-safe` on a cache-less clone still emits no OSV hits (`package.json:N` will not match).
 - DVNA calculator “stack leak” — that line is `mathjs.eval` (`SAST-EVAL-001`), not `err.stack`.
-- Worker-pool timings from `node bench/perf.mjs --measure` on 2026-10-16 (`bench/perf-baseline.json`): nest cold p50 9.2s, warm 9.4s, peak 938 MB; undici cold p50 7.9s, warm 8.1s, peak 609 MB. Both missed the 5s incremental target (nest 8.8s, undici 8.0s). next.js aborted with SIGABRT on all five samples (JavaScript heap out of memory); the recorded times are time-to-abort (cold p50 55.4s, warm 56.7s, incremental 51.9s, peak 1705 MB), not finished scans.
+- Worker-pool timings from `node bench/perf.mjs --measure` on 2026-10-16 (`bench/perf-baseline.json`): next.js finished all 5 samples (cold p50 71.7s, warm 71.5s, incremental 70.6s, peak RSS 788 MB, every sample under 1 GB). nest cold p50 9.1s, warm 9.3s, peak 815 MB; undici cold p50 8.4s, warm 8.4s, peak 563 MB. nest and undici still miss the 5s incremental target (nest 9.4s, undici 8.2s). These are local timings, not a recall claim.
 - Real-world precision is not sampled. Nine of the ten pins in `bench/realworld.json` have finding counts in `BENCHMARK.md`. next.js produced no JSON. No false-positive rate is claimed.
 
 Public-app in-scope detection on **unreviewed** ground truth (`"reviewed": false` in `bench/*/ground-truth.json`). These are not product recall claims. Matching is class + file + line ±3. Helmet comment-only line attribution is not counted.
@@ -130,6 +130,10 @@ app.use(shieldGuard({ mode: 'monitor', authRoutes: ['/login'] }));
 
 **Experimental Frontier (off by default):** `FULLSHIELD_EXPERIMENTAL_VPATCH=1` + `scan --emit-virtual-patches`; `FULLSHIELD_EXPERIMENTAL_HONEY=1` + `fullshield honey init`.
 
+**Experimental CI reports (off by default):** `FULLSHIELD_EXPERIMENTAL_CI_REPORTS=1` or `scan --experimental-ci-reports`, then `--github-annotations`, `--junit`, and `--gitlab`. Default console and `--json` output stay the same. Critical and high become GitHub `error` commands and JUnit failures. Medium is a GitHub warning and a JUnit failure. Low and info are GitHub notices and JUnit skips. GitLab severities are info, minor, major, critical, and blocker.
+
+**Experimental scan diff (off by default):** `FULLSHIELD_EXPERIMENTAL_DIFF=1` or `fullshield diff --experimental-diff <before.json> <after.json>`. Compares two `scan --json` files. The match ignores the line number. It does not scan and does not decide which findings are real. Added findings at or above `--fail-on` (default high) exit 1. Baseline `suppressions` need an owner, a reason, and an expiry. The spec is [`docs/ADVANCED-SPEC.md`](docs/ADVANCED-SPEC.md).
+
 **Experimental RASP** (Frontier, off by default): structure-change sink hooks via `node --import fullshield/runtime/register app.js` or `import 'fullshield/runtime/auto'` as the first import. Enable only with `FULLSHIELD_EXPERIMENTAL_RASP=1`. Default mode is `monitor` (events only). `FULLSHIELD_RASP_MODE=block` throws on structure-changing SQL/shell/path/NoSQL/SSRF/prototype input. Hooks **fail open**: a hook error logs `RASP-FAIL-OPEN-001` and the original call proceeds. Do not use block mode in production until you have measured the benign corpus on your app.
 
 `npm run bench` reports `inspect()` p50/p95 (must stay under 1ms) and full Express request overhead p50/p95 with the middleware on the stack.
@@ -141,6 +145,7 @@ app.use(shieldGuard({ mode: 'monitor', authRoutes: ['/login'] }));
 | Command | Purpose |
 |---|---|
 | `scan` | Run deps, secrets, sast, config. `--json`, `--sarif`, `--markdown`, `--html`, `--fail-on`, `--offline`, `--staged`, `--update-baseline`. `--ai explain` is **experimental** (off by default; explanation text only, never a verdict). `--pr-summary` is **experimental** (`FULLSHIELD_EXPERIMENTAL_POLICY=1`): deterministic Markdown of new findings versus baseline for GitHub/GitLab PR comments (no AI; links to rule docs). |
+| `diff` | **Experimental** (`FULLSHIELD_EXPERIMENTAL_DIFF=1`). Compare two `scan --json` files. `--fail-on` applies to added findings only. |
 | `fullshield.policy.json` | **Experimental** policy-as-code (`FULLSHIELD_EXPERIMENTAL_POLICY=1`): `requiredModules`, per-rule `severityOverrides`, `failOn` per branch, and `exceptions` that require `owner`, `reason`, and ISO `expires`. Expired exceptions fail CI. Exceptions without owner or expiry are rejected. |
 | `fix` | Dry-run patches (unsafe suggestions included). `--apply` writes only `safe: true` patches on a clean git tree. `--unsafe` is required for `helmet()` insertion and secret → `process.env` replacement. `--force` skips files with uncommitted changes. `--ai` is **experimental**: drafts a verified, reviewable diff for findings without a rule-based fix (never auto-merged). |
 | `init` | Config + GitHub/GitLab CI + pre-commit hook |
@@ -203,7 +208,7 @@ npx fullshield scan --debug
 
 `doctor` prints the Node version, lockfile, cache directory, experimental flags that are set to `1`, whether `fullshield.config.json` parses, and whether OSV is allowed to use the network. `--offline` skips that network. CLI failures use `FS-E###` codes. The message stays the same. A stack is printed only with `--debug`.
 
-Console output shows the highest 10 findings, then `N more — use --json`. Each line has the file, the finding message, a fix line, and `rules/<rule-id>.md`.
+Default console prints shown rows only, capped at 20. A dependency advisory is shown when it is `DEPS-MALICIOUS-001`, or when it is `DEPS-VULN-001`, not dev-only, and reachability is reachable, symbol-reachable, via a dependency, or unknown. Severity and confidence do not show a dependency row. SAST, secrets, and config are shown when severity is high or critical, or confidence is high. Shown dependency findings are one row per package. `--all` prints every finding. JSON, SARIF, Markdown, HTML, the baseline, and `--fail-on` stay the full list. On the 2026-10-18 offline-safe recount, grouped rows were NodeGoat 36, goof 63, juice-shop 49, dvna 14, and vulnerable-node 41. Ground truth is still `"reviewed": false`.
 
 Experimental taint propagation (`FULLSHIELD_EXPERIMENTAL_TAINT_V2=1`) and the parse pool (`FULLSHIELD_PARSE_POOL=1`) are off by default. Symbol reachability runs only when an OSV advisory lists `affected_functions`. Advisories without that list keep the import-level class.