fullshield 0.1.3 → 0.3.0

package/CHANGELOG.md

@@ -2,7 +2,118 @@
 
 All notable changes to FullShield are documented here.
 
-## Unreleased
+## 0.3.0 — 2026-10-16
+
+### Default scan changes
+
+Stable finding keys changed. CI baselines may need `--update-baseline`. This is not a recall claim. Ground truth was not edited and nothing was marked reviewed.
+
+Before (`bench/snapshots/0.1.3.json` / previous `HEAD.json`, `scan --json --offline-safe`):
+
+| app | findings |
+|---|---|
+| vulnerable-app | 35 |
+| clean-app | 0 |
+| nodegoat | 379 |
+| juice-shop | 105 |
+| dvna | 23 |
+| goof | 354 |
+| vulnerable-node | 61 |
+
+After (`bench/snapshots/HEAD.json`, same flags):
+
+| app | findings |
+|---|---|
+| vulnerable-app | 35 |
+| clean-app | 0 |
+| nodegoat | 256 |
+| juice-shop | 105 |
+| dvna | 23 |
+| goof | 271 |
+| vulnerable-node | 61 |
+
+Unique `(ruleId, file, line, severity)` keys did not change. The count drop is duplicate keys: several versions of one package and advisory shared one package.json line and one severity. In-scope matches on the unreviewed ground truth stayed at NodeGoat 5/7 (plus 1 line-only), Juice Shop 10/12, DVNA 9/10, goof 11/11, vulnerable-node 9/12. See `bench/snapshots/DIFF.txt`.
+
+- Dependency advisories are one finding per package name + advisory. Every retained dependency path is listed on that finding. Severity is the highest of the grouped versions. Rules were not removed and severities were not lowered to cut the count.
+- SAST findings with the same rule id and sink line collapse to one.
+- Default walk skips `node_modules`, `dist`, `build`, `.next`, `coverage`, `vendor`, `vendored`, `third_party`, `third-party`, `compiled`, and `*.min.js`. Set `"defaultExcludes": false` or replace the list with `"excludes"`.
+- Files over `sast.maxFileBytes` (default 1 MB), and files that contain NUL bytes, are reported as `SAST-SKIP-001` (info) instead of being skipped silently.
+- Repos with more than 4000 source files are parsed one file at a time so the AST is released before the next file. Cross-file taint on those repos runs in batches of 150. Smaller repos, including the in-scope bench apps, keep the previous all-files parse.
+- The parse pool keeps an LRU of 64 ASTs, caps workers (default 2, explicit max 4), and caps each worker heap (`FULLSHIELD_PARSE_WORKER_MB`, default 512).
+
+### Experimental CI reports
+
+Off unless `FULLSHIELD_EXPERIMENTAL_CI_REPORTS=1` or `scan --experimental-ci-reports`. Default `scan` and `--json` are unchanged.
+
+- `--github-annotations` writes GitHub Actions workflow commands. Critical and high are `error`, medium is `warning`, low and info are `notice`.
+- `--junit` writes JUnit XML. Medium and above are failures. Low and info are skipped. An empty scan is one passing testcase.
+- `--gitlab` writes a GitLab Code Quality report. Severities map to info, minor, major, critical, and blocker.
+
+### Experimental scan diff
+
+Off unless `FULLSHIELD_EXPERIMENTAL_DIFF=1` or `fullshield diff --experimental-diff`. It compares two `scan --json` files. It does not scan and does not use the network.
+
+- Match key is rule id, file, severity, and the snippet or message. The line number is not part of the key. Two findings with that same text stay two results. A line shift is a move. One shared finding id with a different message is a move. Duplicate ids are not paired.
+- Added findings at or above `--fail-on` (default `high`) exit 1. AI findings, moves, and removals do not. The human report lists at most 20 added findings.
+- Diff matching and the GitLab fingerprint ignore the line number. A changed message with the same id is a move.
+- Console, JSON, SARIF, Markdown, HTML, GitHub, JUnit, and GitLab reports mask AWS, GitHub, Stripe, and Slack token shapes before writing them.
+- `.fullshield-baseline.json` may list `suppressions` with `ruleId`, `owner`, `reason`, and ISO `expires`. Expired or incomplete rows do not hide findings and fail the scan.
+- `FULLSHIELD_EXPERIMENTAL_IAC=1` also flags `github.event` inside `run:` steps and a `pull_request_target` job that checks out pull-request code.
+- Lockfile helpers flag `http://` or a non-registry `resolved` URL, and a resolved entry with no integrity hash.
+- `scan --history` stops after 500 commits (`FULLSHIELD_HISTORY_MAX`, max 5000). When older commits are left out, the console and the JSON metadata say `scanned N of M commits`. History snippets are masked.
+- `FULLSHIELD_EXPERIMENTAL_LICENSE=1` flags an installed direct dependency whose package.json says GPL or AGPL.
+
+### Node 20
+
+- `engines` is `>=20.11`. `node:sqlite` is the only runtime API that needs Node 22.13. History storage prints that requirement. Confirm does not import `node:sqlite`. RASP sqlite hooks no-op when the module is missing. CI runs Node 20, 22, and 24.
+
+## 0.2.6 — 2026-10-16
+
+- v0.2.5 reached staging with a fresh token, then sigstore threw `TLOG_CREATE_ENTRY_ERROR` because Rekor already had that transparency-log entry (HTTP 409). npm 12.2.0 leaves `fetchOnConflict` off, so the publish stopped. Staging now turns that on before loading the publisher, and gives the Rekor request 120 seconds. It still refuses to publish without provenance.
+
+## 0.2.5 — 2026-10-16
+
+- The publish probe saved the GitHub OIDC token, then verify ran for 24 minutes. npm rejected that expired token with HTTP 401 when staging started. Staging now requests a new token immediately before the exchange, and the probe does not save one. The job pins npm 12.2.0.
+
+## 0.2.4 — 2026-10-16
+
+- Publish installs npm 12. `npm pack --json` is now an object, and the pack budget was parsing the inner `files` array. It reads the pack entry from either shape.
+
+## 0.2.3 — 2026-10-16
+
+- The publish checkout was shallow, so `semver-check` could see only the tag being released. That job now fetches full history and tags before comparing versions.
+
+## 0.2.2 — 2026-10-16
+
+- `v0.2.1` failed typecheck before staging. The baseline-tag test now types its tag argument.
+
+## 0.2.1 — 2026-10-16
+
+- `v0.2.0` was tagged and the publish job failed before staging. `semver-check` treated that tag as the previous release and compared `0.2.0` to itself. The check now skips a tag that points at HEAD.
+
+## 0.2.0 — 2026-10-16
+
+Phases 36–45 ship in this version. `0.3.0` and `0.4.0` were not tagged.
+
+### Phase 36 — provenance and semver
+
+- Publish refuses to stage without provenance. `0.1.3` has no attestation because the stage retried after a sigstore 409. The publish job now polls `dist.attestations` and fails if it stays empty.
+- `scripts/semver-check.mjs`, `scripts/pack-budget.mjs`, and `scripts/provenance-gate.mjs`.
+- TypeORM `class User extends BaseEntity` plus `User.find` is not `SAST-NOSQLI-001` when the file does not import `typeorm`. A bare `User.find(req.body)` is unchanged.
+- Default-scan snapshots are `bench/snapshots/0.1.1.json`, `bench/snapshots/0.1.3.json`, and `bench/snapshots/HEAD.json`. Finding keys are `ruleId`, `file`, `line`, `severity`. Published `0.1.3` and this tree match: vulnerable-app 35, clean-app 0, nodegoat 379, juice-shop 105, dvna 23, goof 354, vulnerable-node 61. `0.1.1` has no `--offline-safe` flag, so that snapshot is `scan --json` only. It differs. The printed diff is `bench/snapshots/DIFF.txt`. That file is not a human approval.
+
+### Phases 37–45
+
+- Holdout registry, three untouched apps, and review sheets. Those apps were not scanned.
+- Comparison harness. One local run is in `COMPARISON.md`: semgrep 1.180.0, osv-scanner 2.7.0, npm 11.19.0. Mapped class buckets stayed empty. osv-scanner returned no JSON for juice-shop and dvna (no lockfile at those pins). Commercial scanners were not executed. This is not a recall claim.
+- Symbol reachability when an advisory's `affected_functions` list is present. Advisories without that list stay on the import-level class. `DEPS-MALICIOUS-001` is not downgraded.
+- Taint propagation for callbacks, `call`/`apply`, `Object.assign`, spread, `this` fields, and closures behind `FULLSHIELD_EXPERIMENTAL_TAINT_V2=1`. Default scan does not enable it.
+- Ten real-world repos are pinned. Nine produced finding counts in `BENCHMARK.md`. next.js aborted with a JavaScript heap out of memory and has no count. Nothing is labeled, so precision is not sampled.
+- Parse pool behind `FULLSHIELD_PARSE_POOL=1`. Measured times are in `bench/perf-baseline.json`. next.js aborted (SIGABRT) on every sample. nest and undici finished under the cold and warm targets and missed the 5s incremental target.
+- Promotion report lists every experimental flag as unmet. Nothing was promoted.
+- `fullshield doctor`, `FS-E###` error codes, and console output capped at 10 findings.
+- GitHub Actions in the project workflows are pinned by commit SHA. Scorecard and CodeQL workflows are added. Scorecard is v2.5.0 from `ghcr.io` because `gcr.io` refused the pull. A tag workflow attaches a CycloneDX SBOM to the GitHub Release.
+- `node scripts/repro-build.mjs --check` on 2026-10-16: 26 `dist` files hashed the same across two builds.
 
 ## 0.1.3 — 2026-10-15
 

package/README.md

@@ -1,6 +1,6 @@
 # FullShield
 
-**v0.1.3 / alpha.** One install for Node.js/TypeScript backends: detect → classify → report → notify → fix → gate (CI `--fail-on` + runtime monitor). It does not stop attacks by itself.
+**v0.2.6 / alpha.** One install for Node.js/TypeScript backends: detect → classify → report → notify → fix → gate (CI `--fail-on` + runtime monitor). It does not stop attacks by itself.
 
 **Headline: dependency reachability.** A lockfile CVE is not the same as an unused package. FullShield classifies each OSV hit as `reachable` (imported and called), `imported-not-called` (−1), `reachable-via-dependency` (lockfile path from an imported direct dep — **severity kept**, path shown, e.g. `express > body-parser > qs`), `not-imported` (−2, only when no such path exists), or `reachability-unknown` (severity kept) when the lockfile format has no parsed dependency edges. DevDependency-only vulns are `info` with a `dev-only` note unless production sources import the package. Pair it with `fullshield sbom --format cyclonedx`.
 
@@ -12,7 +12,7 @@ npx fullshield scan --sarif fullshield.sarif
 
 This is a `0.x` release. APIs may change. Do not treat it as a penetration test.
 
-**Requires Node.js >= 22.13.** `node:sqlite` runs without `--experimental-sqlite` from 22.13.0 (the module is still experimental and is loaded only by history/confirm, not by `--import` register).
+**Requires Node.js >= 20.11.** Scan, fix, and init run on Node 20.11+. History storage (`report --serve`) needs `node:sqlite` and prints `requires Node 22.13+` on older Node instead of failing at import. `node:sqlite` is still experimental and is not loaded by `--import` register.
 
 ## Coverage map
 
@@ -68,7 +68,7 @@ This is a `0.x` release. APIs may change. Do not treat it as a penetration test.
 
 ## Known limitations
 
-- Platforms: Linux, macOS, and Windows on Node 22.13+. Quoted filenames in dirty-tree tests are POSIX-only (Windows forbids `"` in names).
+- Platforms: Linux, macOS, and Windows on Node 20.11, 22, and 24. Quoted filenames in dirty-tree tests are POSIX-only (Windows forbids `"` in names). History storage needs Node 22.13+.
 - `node:sqlite` is loaded only by history/confirm, not by `--import fullshield/runtime/register`. Dashboard closes every handle in `try/finally` (Windows cannot delete an open `history.db`).
 - FullShield does not stop OS or SSH exploits (including regreSSHion). It scans Node/TS apps you own.
 - Extra classes (ReDoS, deserialization, XXE, upload, cookie flags, CSRF, mass assignment, timing compare, hash cost, GraphQL limits, `trust proxy` + IP rate limit) run only with `FULLSHIELD_EXPERIMENTAL_CLASSES=1` or `scan --experimental-classes`. ReDoS is a same-file static quantifier check. CSRF looks for `csurf` or `sameSite: 'strict'|'lax'` in that file.
@@ -89,8 +89,11 @@ In-scope public-bench patterns we still do not claim (one line each):
 - Juice Shop track-order XSS — server returns `res.json`; we do not treat JSON fields as HTML sinks.
 - Juice Shop `url.includes(allowedUrl)` — weak allowlist helper is not a redirect sink; we flag `res.redirect(tainted)`.
 - Juice Shop `{ _id: req.body.id }` — flagged as `SAST-NOSQLI-001` (medium) unless coerced (`String()`/`Number()`/`Boolean()`), schema-validated, or mongo-sanitize is applied. Same for goof `User.find({ password: req.body.password })`.
+- Juice Shop `data/static/codefixes/*` — six `SAST-NOSQLI-001` hits are copies of the challenge fixes. They stay. There is no special-case exclusion for that directory.
 - `DEPS-VULN-001` — when an OSV hit exists, the finding is attributed to the `package.json` name line; `--offline-safe` on a cache-less clone still emits no OSV hits (`package.json:N` will not match).
 - DVNA calculator “stack leak” — that line is `mathjs.eval` (`SAST-EVAL-001`), not `err.stack`.
+- Worker-pool timings from `node bench/perf.mjs --measure` on 2026-10-16 (`bench/perf-baseline.json`): next.js finished all 5 samples (cold p50 71.7s, warm 71.5s, incremental 70.6s, peak RSS 788 MB, every sample under 1 GB). nest cold p50 9.1s, warm 9.3s, peak 815 MB; undici cold p50 8.4s, warm 8.4s, peak 563 MB. nest and undici still miss the 5s incremental target (nest 9.4s, undici 8.2s). These are local timings, not a recall claim.
+- Real-world precision is not sampled. Nine of the ten pins in `bench/realworld.json` have finding counts in `BENCHMARK.md`. next.js produced no JSON. No false-positive rate is claimed.
 
 Public-app in-scope detection on **unreviewed** ground truth (`"reviewed": false` in `bench/*/ground-truth.json`). These are not product recall claims. Matching is class + file + line ±3. Helmet comment-only line attribution is not counted.
 
@@ -127,6 +130,10 @@ app.use(shieldGuard({ mode: 'monitor', authRoutes: ['/login'] }));
 
 **Experimental Frontier (off by default):** `FULLSHIELD_EXPERIMENTAL_VPATCH=1` + `scan --emit-virtual-patches`; `FULLSHIELD_EXPERIMENTAL_HONEY=1` + `fullshield honey init`.
 
+**Experimental CI reports (off by default):** `FULLSHIELD_EXPERIMENTAL_CI_REPORTS=1` or `scan --experimental-ci-reports`, then `--github-annotations`, `--junit`, and `--gitlab`. Default console and `--json` output stay the same. Critical and high become GitHub `error` commands and JUnit failures. Medium is a GitHub warning and a JUnit failure. Low and info are GitHub notices and JUnit skips. GitLab severities are info, minor, major, critical, and blocker.
+
+**Experimental scan diff (off by default):** `FULLSHIELD_EXPERIMENTAL_DIFF=1` or `fullshield diff --experimental-diff <before.json> <after.json>`. Compares two `scan --json` files. The match ignores the line number. It does not scan and does not decide which findings are real. Added findings at or above `--fail-on` (default high) exit 1. Baseline `suppressions` need an owner, a reason, and an expiry. The spec is [`docs/ADVANCED-SPEC.md`](docs/ADVANCED-SPEC.md).
+
 **Experimental RASP** (Frontier, off by default): structure-change sink hooks via `node --import fullshield/runtime/register app.js` or `import 'fullshield/runtime/auto'` as the first import. Enable only with `FULLSHIELD_EXPERIMENTAL_RASP=1`. Default mode is `monitor` (events only). `FULLSHIELD_RASP_MODE=block` throws on structure-changing SQL/shell/path/NoSQL/SSRF/prototype input. Hooks **fail open**: a hook error logs `RASP-FAIL-OPEN-001` and the original call proceeds. Do not use block mode in production until you have measured the benign corpus on your app.
 
 `npm run bench` reports `inspect()` p50/p95 (must stay under 1ms) and full Express request overhead p50/p95 with the middleware on the stack.
@@ -138,6 +145,7 @@ app.use(shieldGuard({ mode: 'monitor', authRoutes: ['/login'] }));
 | Command | Purpose |
 |---|---|
 | `scan` | Run deps, secrets, sast, config. `--json`, `--sarif`, `--markdown`, `--html`, `--fail-on`, `--offline`, `--staged`, `--update-baseline`. `--ai explain` is **experimental** (off by default; explanation text only, never a verdict). `--pr-summary` is **experimental** (`FULLSHIELD_EXPERIMENTAL_POLICY=1`): deterministic Markdown of new findings versus baseline for GitHub/GitLab PR comments (no AI; links to rule docs). |
+| `diff` | **Experimental** (`FULLSHIELD_EXPERIMENTAL_DIFF=1`). Compare two `scan --json` files. `--fail-on` applies to added findings only. |
 | `fullshield.policy.json` | **Experimental** policy-as-code (`FULLSHIELD_EXPERIMENTAL_POLICY=1`): `requiredModules`, per-rule `severityOverrides`, `failOn` per branch, and `exceptions` that require `owner`, `reason`, and ISO `expires`. Expired exceptions fail CI. Exceptions without owner or expiry are rejected. |
 | `fix` | Dry-run patches (unsafe suggestions included). `--apply` writes only `safe: true` patches on a clean git tree. `--unsafe` is required for `helmet()` insertion and secret → `process.env` replacement. `--force` skips files with uncommitted changes. `--ai` is **experimental**: drafts a verified, reviewable diff for findings without a rule-based fix (never auto-merged). |
 | `init` | Config + GitHub/GitLab CI + pre-commit hook |
@@ -190,6 +198,22 @@ Public-app detection results (OWASP NodeGoat, Juice Shop, DVNA, seen holdouts `s
 
 `propose_fix` returns a diff only. It never writes your tree.
 
+## Developer commands
+
+```bash
+npx fullshield doctor
+npx fullshield doctor --offline
+npx fullshield scan --debug
+```
+
+`doctor` prints the Node version, lockfile, cache directory, experimental flags that are set to `1`, whether `fullshield.config.json` parses, and whether OSV is allowed to use the network. `--offline` skips that network. CLI failures use `FS-E###` codes. The message stays the same. A stack is printed only with `--debug`.
+
+Console output shows the highest 10 findings, then `N more — use --json`. Each line has the file, the finding message, a fix line, and `rules/<rule-id>.md`.
+
+Experimental taint propagation (`FULLSHIELD_EXPERIMENTAL_TAINT_V2=1`) and the parse pool (`FULLSHIELD_PARSE_POOL=1`) are off by default. Symbol reachability runs only when an OSV advisory lists `affected_functions`. Advisories without that list keep the import-level class.
+
+`COMPARISON.md` is not linked here. The mapping has not been reviewed. A local semgrep / osv-scanner / npm audit run is in that file, and it is not a recall claim. Benchmark numbers stay in [`BENCHMARK.md`](BENCHMARK.md) with the unreviewed label.
+
 See [`SECURITY.md`](SECURITY.md), [`CONTRIBUTING.md`](CONTRIBUTING.md), and [`RELEASING.md`](RELEASING.md).
 
 ## License