fad-checker 2.0.0 → 2.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (139) hide show
  1. package/CHANGELOG.md +65 -0
  2. package/README.md +147 -25
  3. package/completions/fad-checker.bash +1 -1
  4. package/completions/fad-checker.zsh +14 -1
  5. package/data/eol-mapping.json +5 -5
  6. package/data/license-policy.json +97 -0
  7. package/fad-checker-report/cve-report.doc +630 -0
  8. package/fad-checker-report/cve-report.html +748 -0
  9. package/fad-checker.js +556 -245
  10. package/lib/cache-archive.js +3 -0
  11. package/lib/codecs/codec.interface.js +3 -0
  12. package/lib/codecs/composer/parse.js +3 -0
  13. package/lib/codecs/composer/registry.js +13 -5
  14. package/lib/codecs/composer.codec.js +3 -0
  15. package/lib/codecs/go/parse.js +65 -0
  16. package/lib/codecs/go/registry.js +74 -0
  17. package/lib/codecs/go.codec.js +76 -0
  18. package/lib/codecs/index.js +7 -2
  19. package/lib/codecs/maven/jar-scan.js +199 -0
  20. package/lib/codecs/maven.codec.js +17 -1
  21. package/lib/codecs/npm/collect.js +37 -17
  22. package/lib/codecs/npm/parse.js +117 -10
  23. package/lib/codecs/npm/registry.js +14 -5
  24. package/lib/codecs/npm.codec.js +4 -1
  25. package/lib/codecs/nuget/parse.js +3 -0
  26. package/lib/codecs/nuget/registry.js +11 -4
  27. package/lib/codecs/nuget.codec.js +7 -2
  28. package/lib/codecs/pypi/parse.js +111 -14
  29. package/lib/codecs/pypi/registry.js +24 -5
  30. package/lib/codecs/pypi.codec.js +30 -6
  31. package/lib/codecs/recipes.js +28 -1
  32. package/lib/codecs/ruby/parse.js +42 -0
  33. package/lib/codecs/ruby/registry.js +76 -0
  34. package/lib/codecs/ruby.codec.js +66 -0
  35. package/lib/codecs/select.js +3 -0
  36. package/lib/codecs/yarn.codec.js +3 -0
  37. package/lib/config.js +3 -0
  38. package/lib/core.js +38 -1
  39. package/lib/cpe.js +30 -5
  40. package/lib/csaf-export.js +159 -0
  41. package/lib/cve-download.js +3 -0
  42. package/lib/cve-match.js +27 -6
  43. package/lib/cve-report.js +157 -28
  44. package/lib/dep-record.js +15 -2
  45. package/lib/deps-descriptor.js +113 -0
  46. package/lib/epss.js +115 -0
  47. package/lib/gate.js +45 -0
  48. package/lib/json-export.js +110 -0
  49. package/lib/kev.js +88 -0
  50. package/lib/license-policy.js +110 -0
  51. package/lib/maven-license.js +52 -0
  52. package/lib/maven-repo.js +3 -0
  53. package/lib/maven-version.js +8 -3
  54. package/lib/nvd.js +17 -6
  55. package/lib/osv.js +75 -23
  56. package/lib/outdated.js +10 -2
  57. package/lib/priority.js +90 -0
  58. package/lib/purl.js +77 -0
  59. package/lib/retire.js +80 -13
  60. package/lib/sarif-export.js +134 -0
  61. package/lib/sbom-export.js +153 -0
  62. package/lib/scan-completeness.js +3 -0
  63. package/lib/snyk.js +3 -0
  64. package/lib/suppress.js +113 -0
  65. package/lib/transitive.js +6 -3
  66. package/lib/ui.js +90 -0
  67. package/package.json +49 -2
  68. package/CLAUDE.md +0 -147
  69. package/docs/ARCHITECTURE.md +0 -182
  70. package/docs/USAGE.md +0 -187
  71. package/docs/superpowers/plans/2026-05-29-codec-composer.md +0 -556
  72. package/docs/superpowers/plans/2026-05-29-codec-foundation.md +0 -851
  73. package/docs/superpowers/plans/2026-05-29-codec-nuget.md +0 -432
  74. package/docs/superpowers/plans/2026-05-29-codec-pypi.md +0 -450
  75. package/docs/superpowers/specs/2026-05-29-codecs-multi-ecosystem-design.md +0 -251
  76. package/test/cli-ecosystem.test.js +0 -30
  77. package/test/codec-edge-cases.test.js +0 -131
  78. package/test/codec-integration.test.js +0 -119
  79. package/test/codecs.test.js +0 -92
  80. package/test/composer.test.js +0 -71
  81. package/test/core.test.js +0 -153
  82. package/test/cpe.test.js +0 -166
  83. package/test/cve-download.test.js +0 -39
  84. package/test/cve-match.test.js +0 -217
  85. package/test/cve-report.test.js +0 -180
  86. package/test/dep-record.test.js +0 -31
  87. package/test/fixtures/complex-enterprise/api/pom.xml +0 -32
  88. package/test/fixtures/complex-enterprise/build/pom.xml +0 -46
  89. package/test/fixtures/complex-enterprise/dao/pom.xml +0 -37
  90. package/test/fixtures/complex-enterprise/pom.xml +0 -66
  91. package/test/fixtures/complex-enterprise/web/pom.xml +0 -77
  92. package/test/fixtures/csharp-config/packages.config +0 -4
  93. package/test/fixtures/csharp-csproj/Directory.Packages.props +0 -5
  94. package/test/fixtures/csharp-csproj/app.csproj +0 -7
  95. package/test/fixtures/csharp-lock/packages.lock.json +0 -6
  96. package/test/fixtures/cve-samples/cve-non-java.json +0 -19
  97. package/test/fixtures/cve-samples/cve-product-only.json +0 -31
  98. package/test/fixtures/cve-samples/cve-with-packagename.json +0 -37
  99. package/test/fixtures/cve-samples/nvd-log4shell.json +0 -40
  100. package/test/fixtures/cve-samples/nvd-npm-lodash.json +0 -22
  101. package/test/fixtures/monorepo-mixed/libs/common-bom/pom.xml +0 -26
  102. package/test/fixtures/monorepo-mixed/packages/cli/package.json +0 -14
  103. package/test/fixtures/monorepo-mixed/packages/cli/yarn.lock +0 -41
  104. package/test/fixtures/monorepo-mixed/packages/no-lock/package.json +0 -10
  105. package/test/fixtures/monorepo-mixed/packages/web-app/package-lock.json +0 -71
  106. package/test/fixtures/monorepo-mixed/packages/web-app/package.json +0 -17
  107. package/test/fixtures/monorepo-mixed/pom.xml +0 -29
  108. package/test/fixtures/monorepo-mixed/services/api/pom.xml +0 -27
  109. package/test/fixtures/monorepo-mixed/services/worker/pom.xml +0 -28
  110. package/test/fixtures/php-app/composer.json +0 -5
  111. package/test/fixtures/php-app/composer.lock +0 -10
  112. package/test/fixtures/polyglot/cs/packages.lock.json +0 -1
  113. package/test/fixtures/polyglot/js/package-lock.json +0 -1
  114. package/test/fixtures/polyglot/js/package.json +0 -1
  115. package/test/fixtures/polyglot/mvn/pom.xml +0 -7
  116. package/test/fixtures/polyglot/php/composer.lock +0 -1
  117. package/test/fixtures/polyglot/py/poetry.lock +0 -3
  118. package/test/fixtures/private-lib-detection/core/pom.xml +0 -26
  119. package/test/fixtures/private-lib-detection/plugin/pom.xml +0 -23
  120. package/test/fixtures/private-lib-detection/pom.xml +0 -35
  121. package/test/fixtures/python-pipenv/Pipfile.lock +0 -1
  122. package/test/fixtures/python-poetry/poetry.lock +0 -7
  123. package/test/fixtures/python-reqs/requirements.txt +0 -5
  124. package/test/fixtures/python-uv/uv.lock +0 -3
  125. package/test/fixtures/simple/app/pom.xml +0 -28
  126. package/test/fixtures/simple/lib/pom.xml +0 -18
  127. package/test/fixtures/simple/pom.xml +0 -24
  128. package/test/maven-repo.test.js +0 -111
  129. package/test/maven-version.test.js +0 -57
  130. package/test/monorepo.test.js +0 -132
  131. package/test/npm-registry.test.js +0 -64
  132. package/test/npm.test.js +0 -150
  133. package/test/nuget.test.js +0 -66
  134. package/test/osv.test.js +0 -62
  135. package/test/outdated.test.js +0 -101
  136. package/test/pypi.test.js +0 -72
  137. package/test/snyk.test.js +0 -64
  138. package/test/transitive.test.js +0 -305
  139. package/test/webjar.test.js +0 -33
package/CLAUDE.md DELETED
@@ -1,147 +0,0 @@
1
- # CLAUDE.md
2
-
3
- Code-level orientation for contributors and Claude Code sessions on this repo.
4
-
5
- ## What this is
6
-
7
- `fad-checker` — **Fucking Autonomous Dependency Checker**. Node.js CLI (`fad-checker`, or short alias `fad`) that:
8
-
9
- 1. Walks a multi-module Maven tree, removes private/excluded dependencies (regex on groupId), writes a parallel directory of "cleaned" POMs that can be fed to Snyk.
10
- 2. Walks every JS package (`package.json` + `package-lock.json` v1/v2/v3 or `yarn.lock` v1), every PHP package (`composer.lock`, or `composer.json` best-effort), and every Python project (`poetry.lock`/`Pipfile.lock`/`uv.lock`/`pdm.lock`, or `requirements.txt` best-effort), and every .NET project (`packages.lock.json`, or `*.csproj`+`Directory.Packages.props`/`packages.config` best-effort) in the same source tree. Each ecosystem is a **codec** (`lib/codecs/`): maven, npm, yarn, composer, pypi, nuget. Adding one is adding a codec.
11
- 3. Scans the union against:
12
- - the CVEProject `cvelistV5` Maven-relevant index (built locally),
13
- - OSV.dev (multi-ecosystem),
14
- - NIST NVD (enrichment: CVSS, CPE configurations, references),
15
- - retire.js (vendored JS signatures),
16
- - optionally Snyk (`--snyk`).
17
- 4. Cross-checks every match's NVD CPE configurations against the dep version (`lib/cpe.js`) to filter false positives.
18
- 5. Reports EOL frameworks (endoflife.date — Maven & npm), obsolete libs (curated Maven + npm-registry per-version `deprecated` field — authoritative, skips nothing), outdated libs (Maven Central + npm registry `dist-tags.latest`). **WebJars** (`org.webjars*`) are reduced to their npm coordinate by `webjarToNpm()` and run through the npm EOL/deprecation/outdated paths — so e.g. `org.webjars:angularjs:1.8.3` is flagged EOL.
19
- 6. Produces a self-contained HTML report + Word-compatible `.doc`, organised by ecosystem and by defining manifest, with per-tool fix recipes and an executive summary.
20
-
21
- No build tool (`mvn`, `npm install`, `yarn`) is required on PATH — `pom.xml` / `package-lock.json` / `yarn.lock` are parsed directly.
22
-
23
- ## Running
24
-
25
- ```bash
26
- npm install
27
- npm test # 194 unit tests via node --test
28
-
29
- # basic cleanup workflow
30
- node fad-checker.js -s ./proj # read-only, full report
31
- node fad-checker.js -s ./proj -t ../pom-clean -e "^client\\." # write cleaned tree
32
- node fad-checker.js -s ./proj -t ../pom-clean -e "^client\\." --snyk # also drive snyk
33
-
34
- # read the full usage doc
35
- cat docs/USAGE.md
36
- ```
37
-
38
- Binary builds (requires `bun`):
39
-
40
- ```bash
41
- npm run build:linux # → dist/fad-checker-linux
42
- npm run build:win # → dist/fad-checker.exe
43
- npm run build # both
44
- ```
45
-
46
- Guardrails enforced at startup:
47
- - `--target` is required only when you want a cleaned POM tree. Without it the run is read-only.
48
- - `--target` may not equal or be a subdirectory of `--src`.
49
- - `--target` is `rimraf`'d before being rewritten — never point at anything precious.
50
-
51
- ## Architecture (one-liner per file)
52
-
53
- ```
54
- fad-checker.js Thin CLI: commander parsing, orchestration only (loops over active codecs).
55
- lib/codecs/index.js Codec registry: getCodec / allCodecs / detectCodecs.
56
- lib/codecs/codec.interface.js Codec contract + assertCodecShape() validator.
57
- lib/codecs/maven.codec.js Maven codec (wraps core.js + transitive.js + CVE-index scanner).
58
- lib/codecs/npm.codec.js npm codec (wraps lib/codecs/npm/* + retire.js scanner). yarn.codec.js shares it.
59
- lib/codecs/select.js resolveActiveCodecs(): --ecosystem list + --no-<id> → active codec ids.
60
- lib/codecs/recipes.js Per-ecosystem fix-recipe specs (pin snippet + direct-update wording).
61
- lib/codecs/composer.codec.js Composer (PHP) codec.
62
- lib/codecs/composer/parse.js composer.lock + composer.json parsers.
63
- lib/codecs/composer/registry.js Packagist query → latest stable + `abandoned` flag.
64
- lib/codecs/pypi.codec.js PyPI (Python) codec.
65
- lib/codecs/pypi/parse.js poetry.lock/Pipfile.lock/uv.lock/pdm.lock (smol-toml) + requirements.txt parsers (PEP 503).
66
- lib/codecs/pypi/registry.js PyPI JSON query → latest + yanked + inactive classifier.
67
- lib/codecs/nuget.codec.js NuGet (C#/.NET) codec.
68
- lib/codecs/nuget/parse.js packages.lock.json + *.csproj (+CPM Directory.Packages.props) + packages.config parsers.
69
- lib/codecs/nuget/registry.js NuGet registration query → latest stable + per-version deprecation.
70
- lib/dep-record.js makeDepRecord(): generalized depRecord ({ ecosystem, namespace, name, coordKey, … }).
71
- lib/core.js POM parsing, parent resolution, all-profile merge, rewrite.
72
- lib/maven-version.js Maven version parsing + range comparison (no external deps).
73
- lib/cve-download.js Bulk download of CVEProject/cvelistV5 + Maven-relevant index build.
74
- lib/cve-match.js Resolved-dep collection + 3-tier CVE matching with dedup.
75
- lib/cve-report.js Self-contained HTML and Word-compatible (.doc) report rendering.
76
- lib/cpe.js CPE 2.3 parsing + NVD configurations evaluator (post-match refinement).
77
- lib/outdated.js EOL (endoflife.date), obsolete (curated), outdated (Maven Central).
78
- lib/transitive.js Maven Central POM walker (transitive resolution).
79
- lib/osv.js OSV.dev batched query + per-vuln detail fetch.
80
- lib/nvd.js NIST NVD enrichment (CVSS, references, CPE configurations).
81
- lib/snyk.js `snyk test --all-projects --json` runner + merge.
82
- lib/retire.js retire.js (vendored-JS scanner) wrapper + cache + normaliser.
83
- lib/scan-completeness.js Warnings for deps fad-checker couldn't fully resolve.
84
- lib/codecs/npm/parse.js package.json, package-lock.json (v1/2/3), yarn.lock v1 parsers.
85
- lib/codecs/npm/collect.js Merge across JS manifests → unified resolvedDeps Map.
86
- lib/codecs/npm/registry.js npm registry packument query → per-version deprecation + dist-tags.latest (npm EOL feeds via lib/outdated.js).
87
- lib/cache-archive.js tar.gz / zip export & import of ~/.fad-checker/.
88
- lib/config.js Persistent user config in ~/.fad-checker/config.json (mode 0600).
89
- data/ known-obsolete.json, eol-mapping.json, cpe-coord-map.json, known-public-namespaces.json
90
- completions/ fad-checker.bash, fad-checker.zsh
91
- test/ node:test suite + fixtures (simple, complex-enterprise, monorepo-mixed, …).
92
- ```
93
-
94
- For the deep dive — pipeline stages, the resolved-deps Map shape, report structure — see [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md).
95
-
96
- ## Important conventions
97
-
98
- - **`coord()` always trims**: real-world POMs occasionally contain whitespace around `<artifactId>`. Every coord-derived lookup goes through `coord()` in `lib/core.js`.
99
- - **`byId` keys are never polluted with `undefined`**: only indexed when both `groupId` and `artifactId` are present. Enforced by test.
100
- - **All profiles are merged, never prompted for**: every profile's deps are unioned so Snyk sees every dep any profile could pull in. `activeByDefault` wins only for property value conflicts.
101
- - **No `process.exit(1)` mid-pipeline**: a parse/rewrite failure for one POM logs and continues so the summary still prints.
102
- - **HTML report is self-contained**: inline CSS, no external assets. The `.doc` variant is the same HTML with Office XML namespace meta tags — Word opens it natively.
103
- - **Codec abstraction**: every ecosystem lives behind a codec (`lib/codecs/*`) implementing one interface (`detect`/`collect`/`coordKey`/`formatCoord`/`osvPackageName`/`checkRegistry`/`resolveEolProduct`/`recipe`/`nativeScanners`). The orchestrator loops over the codecs `detectCodecs()` returns. OSV/NVD/CPE and the endoflife.date fetch are **shared, ecosystem-agnostic** services that ask the codec for a package/product name. CVE-index (maven) and retire.js (npm) are `nativeScanners` owned by their codec. Adding an ecosystem = adding a codec, no orchestrator edits.
104
- - **Map keys are ecosystem-namespaced** (`dep.coordKey`): Maven uses a **bare** `g:a` (kept prefix-free so `transitive.js` internals and existing tests are untouched); npm uses `npm:name`; new ecosystems use `nuget:`/`composer:vendor/pkg`/`pypi:`. Bare `g:a` is collision-free against those prefixes. Built by `makeDepRecord()`; `groupId`/`artifactId`/`pomPaths` are kept as real duplicated alias fields (not getters — depRecords are spread in hot paths).
105
- - **Every distinct version is scanned, not just the highest**: when profiles/modules pin the same `g:a` to different versions, the resolved-dep entry keeps `version` = highest (representative for display/EOL/outdated) but `versions` = all distinct concrete versions. CVE matching (`matchOne`) and OSV (`queryOsvForDeps`) iterate `versions` so a vuln affecting only a lower-versioned profile variant isn't missed. Match dedup keys are `g:a:version|cve.id` (version included) to preserve per-version findings.
106
- - **npm no-lockfile = best-effort (not skipped)**: `package.json` without a sibling `package-lock.json`/`yarn.lock` is parsed best-effort — pinned exact versions (`"1.2.3"`) are scanned, ranges (`"^1.0.0"`) are skipped, and a `no-lockfile` warning (chapter 0) flags the partial coverage. (Earlier versions skipped such manifests entirely.)
107
- - **`--ecosystem` is a list**: `auto` (default = `detectCodecs()`) | `all` | comma list `maven,npm,nuget,composer,pypi` (legacy `both`/`maven`/`npm` still parse). Per-codec opt-out via `--no-maven`/`--no-npm`/`--no-yarn`/`--no-nuget`/`--no-composer`/`--no-pypi`; `--no-js` is an alias for `--no-npm`+`--no-yarn`.
108
- - **Source identifiers**: every match carries `source: "fad" | "osv" | "nvd" | "snyk" | "retire"` (or a `+`-joined combination).
109
-
110
- ## Testing
111
-
112
- ```bash
113
- node --test test/*.test.js # full suite (96 tests)
114
- node --test test/core.test.js # one file
115
- ```
116
-
117
- Test fixtures live in `test/fixtures/`:
118
- - `simple/` — 3 POMs with parent inheritance + property substitution
119
- - `complex-enterprise/` — Spring Boot parent (external), local BOM via `scope=import`, three profiles
120
- - `private-lib-detection/` — mixed public/private groupIds, external private parent
121
- - `monorepo-mixed/` — Maven + npm (package-lock v3) + yarn.lock v1 + a no-lockfile package.json
122
- - `cve-samples/` — small CVE / NVD JSON files for the matchers
123
-
124
- ## Gotchas / edge cases worth knowing
125
-
126
- - CVE bundle from CVEProject is ~500 MB unpacked. Shells out to `curl + unzip` (fallback to `fetch()` + `unzip` / `Expand-Archive`). Extracted JSON deleted after index build. Ships as `cves.zip.zip` (nested zip) — `extractZip()` recurses up to 3 levels.
127
- - `endoflife.date` API responses cached 7 days; Maven Central version lookups cached 24 hours. Cache lives in `~/.fad-checker/`.
128
- - **Persistent config**: `~/.fad-checker/config.json` (mode 0600). Set NVD key via `fad-checker --set-nvd-key <KEY>` (free, instant from <https://nvd.nist.gov/developers/request-an-api-key> — bumps rate limit from 5/30s to 50/30s).
129
- - **`--offline` umbrella flag**: skips every network call (CVE/OSV/NVD/Maven Central/endoflife/npm-registry/retire). Falls back to whatever is already cached. Per-source variants (`--cve-offline`, `--no-osv`, `--no-nvd`, `--no-retire`, `--no-transitive`) and per-codec toggles (`--no-maven`/`--no-npm`/`--no-yarn`/`--no-nuget`/`--no-composer`/`--no-pypi`, `--no-js`) still work independently. npm registry deprecation always runs when online; npm (and Maven) outdated is gated by `--no-all-libs`.
130
- - `snyk` is not a hard dep — shells out via `execFile`. `snyk` exits 1 on findings; the JSON is still on stdout.
131
- - The cleaned POM is the union of every profile's deps. Counts will be larger than the source POM. Intentional — don't "fix" that.
132
- - Unresolved `${…}` Maven variables stay verbatim in the rewritten POM. `lib/cve-match.js` resolves them lazily via `resolveDepVersion()` when scanning. Deps that *still* can't be resolved (external BOM not in source tree) surface in chapter 0 as `unresolved-versions` warnings.
133
- - **retire.js** doesn't like `--outputpath /dev/stdout`. We write to a temp file and read it back. Exit code 13 means "vulns found" — expected, not an error.
134
-
135
- ### Per-cache TTLs
136
-
137
- | Cache | Location | TTL |
138
- |---|---|---|
139
- | CVEProject bulk index | `~/.fad-checker/cve-data/maven-cve-index.json` | 24 h |
140
- | OSV per-dep stub list | `~/.fad-checker/osv-cache/<eco>__<g>__<a>__<v>.json` | 12 h |
141
- | OSV vuln details | `~/.fad-checker/osv-cache/vuln_<id>.json` | 12 h |
142
- | NVD CVE record | `~/.fad-checker/nvd-cache/<cveId>.json` | 7 d |
143
- | endoflife.date cycles | `~/.fad-checker/eol-cache.json` | 7 d |
144
- | Maven Central latest | `~/.fad-checker/version-cache.json` | 24 h |
145
- | npm registry (deprecation + latest) | `~/.fad-checker/npm-registry-cache.json` | 24 h |
146
- | Transitive POM | `~/.fad-checker/poms-cache/<g>__<a>__<v>.pom` | ∞ (immutable on Maven Central) |
147
- | retire.js findings | `~/.fad-checker/retire-cache/<md5(src)>.json` | 24 h |
@@ -1,182 +0,0 @@
1
- # Architecture
2
-
3
- This is the deep-dive for anyone modifying `fad-checker`'s internals or wondering why a particular finding shows up the way it does. For day-to-day usage, see [`USAGE.md`](USAGE.md).
4
-
5
- ## Module map
6
-
7
- ```
8
- fad-checker.js Thin CLI: commander parsing + orchestration (loops over active codecs).
9
- lib/codecs/ Per-ecosystem codecs (maven, npm, yarn, composer, pypi, nuget) + registry + select + recipes (see "Codecs" below).
10
- lib/codecs/composer/ composer.lock/composer.json parsers + Packagist registry (PHP codec internals).
11
- lib/codecs/pypi/ poetry/pipfile/uv/pdm/requirements parsers + PyPI registry (Python codec internals).
12
- lib/codecs/nuget/ packages.lock.json/csproj/packages.config parsers + NuGet registry (.NET codec internals).
13
- lib/dep-record.js makeDepRecord(): the generalized depRecord shared by all codecs.
14
- lib/core.js POM parsing, parent resolution, all-profile merge, rewrite.
15
- lib/maven-version.js Maven version parsing + range comparison (no external deps).
16
- lib/cve-download.js Bulk download of CVEProject/cvelistV5 + Maven-relevant index build.
17
- lib/cve-match.js Resolved-dep collection + 3-tier CVE matching with dedup.
18
- lib/cve-report.js Self-contained HTML and Word-compatible (.doc) report rendering.
19
- lib/cpe.js CPE 2.3 parsing + NVD configurations evaluator (post-match refinement).
20
- lib/outdated.js EOL (endoflife.date), obsolete (curated), outdated (Maven Central).
21
- lib/transitive.js Maven Central POM walker (transitive resolution).
22
- lib/osv.js OSV.dev batched query + per-vuln detail fetch.
23
- lib/nvd.js NIST NVD enrichment (CVSS, references, CPE configurations).
24
- lib/snyk.js `snyk test --all-projects --json` runner + merge.
25
- lib/retire.js retire.js (vendored-JS scanner) wrapper + cache + normaliser.
26
- lib/scan-completeness.js Warnings for deps we couldn't fully resolve.
27
- lib/codecs/npm/parse.js package.json, package-lock.json (v1/2/3), yarn.lock v1 parsers.
28
- lib/codecs/npm/collect.js Merge across JS manifests → unified resolvedDeps Map.
29
- lib/codecs/npm/registry.js npm registry packument query → per-version deprecation + dist-tags.latest.
30
- lib/cache-archive.js tar.gz / zip export & import of ~/.fad-checker/.
31
- lib/config.js Persistent user config in ~/.fad-checker/config.json (mode 0600).
32
- data/ Curated JSON: known-obsolete, eol-mapping, cpe-coord-map, known-public-namespaces.
33
- completions/ fad-checker.bash, fad-checker.zsh
34
- test/ node:test suite + fixtures (simple, complex-enterprise, monorepo-mixed, …).
35
- ```
36
-
37
- ## Codecs
38
-
39
- Every ecosystem-specific behaviour lives behind a **codec** (`lib/codecs/*.codec.js`)
40
- implementing one interface (`lib/codecs/codec.interface.js`):
41
-
42
- ```
43
- id, label, osvEcosystem, manifestNames,
44
- detect(dir), collect(dir,opts) → {deps, warnings},
45
- coordKey(dep), formatCoord(dep), osvPackageName(dep),
46
- checkRegistry(deps,opts) → {outdated, deprecated},
47
- resolveEolProduct(dep), recipe, nativeScanners
48
- ```
49
-
50
- - `lib/codecs/index.js` is the registry: `getCodec(id)`, `allCodecs()`, `detectCodecs(dir)`.
51
- - `lib/codecs/select.js` turns `--ecosystem <list>` + `--no-<id>` into the active codec ids.
52
- - The orchestrator collects deps by looping the active codecs, then runs the **shared,
53
- ecosystem-agnostic** services (OSV, NVD, CPE refinement, endoflife.date) which ask the
54
- codec only for a package/product name. `nativeScanners` are extra scanners a codec owns
55
- and the orchestrator runs at their pipeline position by `kind`: `cve` (maven → local
56
- cvelistV5 index, merged into the CVE chapter) and `vendored` (npm → retire.js, its own
57
- chapter). New ecosystems (NuGet/Composer/PyPI) ship as codecs with no native scanners —
58
- OSV + NVD cover them — so no orchestrator changes are needed to add one.
59
-
60
- ## The resolved-deps Map
61
-
62
- The whole pipeline hinges on a single `Map<string, depRecord>` keyed by:
63
- - `groupId:artifactId` for Maven entries
64
- - `npm:<name>` for npm/yarn entries
65
-
66
- Each `depRecord` carries:
67
-
68
- ```js
69
- {
70
- groupId, artifactId, version,
71
- scope, // "compile" | "test" | "import" | "transitive" | "parent" | "prod" | "dev" | "peer" | "optional"
72
- isDev, // Maven test/provided OR npm dev/devOptional/optional
73
- ecosystem, // "maven" | "npm"
74
- ecosystemType, // "maven" | "npm" | "yarn" | "retire"
75
- pomPaths, // absolute paths to manifests declaring this dep
76
- manifestPaths, // same as pomPaths but used by the npm collector
77
- // Transitive-only:
78
- via, viaPaths, depth,
79
- // npm-only:
80
- lockType, resolved, integrity,
81
- }
82
- ```
83
-
84
- The Maven keyspace and npm keyspace never collide — `:lodash` (Maven groupId-less) becomes `npm:lodash` so the same Map can hold both ecosystems without overwrites.
85
-
86
- ## Cleanup pipeline (`lib/core.js`)
87
-
88
- 1. `findPomFiles(src)` — recursive walk, skips known output dirs (`target/`, `node_modules/`, `.git/`, `.idea/`, `dist/`, `build-output/`, `out/`, `.next/`, `.nuxt/`, `coverage/`, `.gradle/`, `.mvn/`, `.vscode/`, `bower_components/`, `jspm_packages/`). Note: `build/` is **not** skipped on the Maven side because some multi-module projects use it for a BOM module.
89
- 2. `parsePom()` — xml2js to JSON. Extracts groupId/artifactId/version, `<parent>`, properties, and indexes every profile (recording which one is `activeByDefault`). Templates with literal `\${…}` are skipped.
90
- 3. `getAllInheritedProps()` — merges `<dependencies>`, `<dependencyManagement>`, `<properties>` from **every** `<profile>` (with `activeByDefault` properties winning for value conflicts), follows `<scope>import</scope>` BOMs to other local POMs, and recurses into resolved parents.
91
- 4. `rewritePoms()` — strips everything outside `nodeToKeep`, runs `cleanDeps()` to apply the `-e` regex, rewrites the `<parent>.relativePath` and `version` to the parent's value (not the child's). Skips disk writes when `readOnly`.
92
-
93
- ## Report pipeline (driven by `fad-checker.js` when `--report` is set)
94
-
95
- 1. **Collect** — `collectResolvedDeps()` dedupes by `groupId:artifactId`, keeps the highest version on conflict, includes external parent POMs as `scope='parent'`. `--ignore-test` honored. For npm, `collectNpmDeps()` walks JS manifests (lockfile-only — `package.json` without sibling lockfile is skipped + warned).
96
- 2. **Transitive expansion** (optional, `--transitive`) — `expandWithTransitives()` walks the Maven Central POM graph honouring exclusions, root depMgmt overrides, nearest-wins on version conflict, `--transitive-depth` cap. Skips test + optional scopes by default.
97
- 3. **CVE index** — `ensureCveIndex()` downloads the daily CVEProject zip (via `curl + unzip`, or falls back to `fetch()` + `unzip` / PowerShell `Expand-Archive`), filters to Maven-relevant entries, caches the compact index to `~/.fad-checker/cve-data/maven-cve-index.json`. Fresh for 24h. `--cve-refresh` forces rebuild, `--cve-offline` uses cache only.
98
- 4. **CVE matching** — `matchDepsAgainstCves()` runs three tiers:
99
- - `exact`: `byPackageName["g:a"]` hit
100
- - `probable`: `byProduct[artifactId]` + vendor matches groupId (`apache` ↔ `org.apache.*`)
101
- - `possible`: product-only match
102
- Dedupes by `(dep, cve.id)` and sorts by severity. npm deps are skipped here — they're scanned by OSV instead.
103
- 5. **OSV** (default on) — `queryOsvForDeps()` POSTs batched queries to `api.osv.dev/v1/querybatch` (Maven ecosystem for Maven deps, npm ecosystem for npm deps). Per-dep stub list cached 12h; per-vuln details cached 12h.
104
- 6. **NVD enrichment** (default on) — for every CVE id matched, fetch the full NVD record (description, CVSS vectors, references categorised by tag, CPE configurations). Rate-limited per NIST policy (5/30s unauthenticated, 50/30s with `NVD_API_KEY`).
105
- 7. **CPE refinement** — `refineMatchesWithCpe()` walks NVD's `configurations[].nodes[]` against each matched dep:
106
- - Confirms the dep version actually falls in the vulnerable range (else `cpeFiltered: true` — likely false positive).
107
- - Upgrades match `confidence` from `possible` → `probable` → `exact` when a curated `cpe-coord-map.json` entry confirms vendor:product → dep coord.
108
- 8. **retire.js** (default on) — shells out to `retire --outputformat json --jspath <src>`. Output normalised to fad-checker match shape, with the vendored file path attached so the report can show where the offending `.js` lives. Cache: `~/.fad-checker/retire-cache/<md5(src)>.json`, 24h TTL.
109
- 9. **EOL / Obsolete / Outdated** — `lib/outdated.js` (Maven) + `lib/codecs/npm/registry.js` (npm):
110
- - **WebJars** (`org.webjars*` — client-side JS shipped as Maven artifacts) are reduced to their npm-equivalent coordinate by `webjarToNpm()` (`lib/codecs/npm/collect.js`): `org.webjars.npm` is a deterministic npm mirror (`angular__core` → `@angular/core`); classic `org.webjars`/bower names pass through. They then flow through the **same npm paths** below — no WebJar-specific data.
111
- - **EOL**: matches dep coord against `data/eol-mapping.json`, fetches the cycle list from endoflife.date (cached 7d), flags cycles past their EOL date. npm packages and WebJars resolve by JS library name via `by_npm_name` / `by_npm_scope` (e.g. npm `angular`/webjar `angularjs` → AngularJS 1.x, `@angular/*` → Angular, `react`/`jquery`/`vue`/`bootstrap`).
112
- - **Obsolete**: Maven via curated `data/known-obsolete.json` (log4j 1.x, jackson-mapper-asl, joda-time, commons-httpclient 3.x, …); npm **and WebJars** via the registry's per-version `deprecated` field (authoritative maintainer data — every dep is checked, nothing curated, nothing skipped).
113
- - **Outdated**: Maven Central Solr query; npm registry `dist-tags.latest` (npm deps and WebJars). Both gated by `--no-all-libs`. Cache 24h. Concurrency 8.
114
- 10. **Snyk** (optional, `--snyk`) — runs `snyk test --all-projects --json` against the cleaned target dir. Normalised + merged. Findings in both sources tagged `source: "both"`.
115
- 11. **Render** — `writeReports()` produces `cve-report.html` (self-contained, inline CSS, no external assets) and `cve-report.doc` (same HTML with Office XML namespace meta tags so Word opens it natively). Default output dir: `./fad-checker-report/`.
116
-
117
- ## Report structure
118
-
119
- ```
120
- <Executive Summary> ← global criticality + key bullet counts
121
- <Summary cards> ← critical / high / medium / low / EOL / obsolete / outdated
122
- <Toolbar> ← expand-all / collapse-all / expand CVE details
123
-
124
- 0. Warnings & scan-completeness ← chapter 0 if any warnings
125
- 1. CVE Vulnerabilities — production (N)
126
- 1.a Maven (n)
127
- 1.a.0 All (n) ← combined direct + transitive
128
- By pom.xml (k files) ← wrapper always present
129
- <relative-path-1> (m) ← direct deps in this pom only
130
- <relative-path-2> (m)
131
-
132
- 1.b npm (package-lock) (n)
133
- 1.b.0 All (n)
134
- By package-lock.json (k files)
135
- web/package-lock.json (m)
136
- 2. CVE in dev dependencies (M) ← same structure as 1
137
- 3. Vendored JS scan — retire.js (R)
138
- 4. End-of-Life Frameworks
139
- 5. Obsolete / Deprecated Libraries
140
- 6. Outdated Libraries
141
- 7. Fix Recommendations ← per-ecosystem snippets
142
- 7.a Maven ← dependencyManagement XML
143
- 7.b npm ← package.json overrides
144
- 7.c yarn ← package.json resolutions
145
- ```
146
-
147
- ## Important conventions
148
-
149
- - **`coord()` always trims**: real-world POMs occasionally contain whitespace around `<artifactId>` (seen in the wild). Every coord-derived lookup goes through `coord()` in `lib/core.js`.
150
- - **`byId` keys are never polluted with `undefined`**: we only index a POM by id when both `groupId` and `artifactId` are present. Test `byId does not get polluted with undefined keys` enforces this.
151
- - **All profiles are merged, never prompted for**: previous versions prompted the user when a POM had multiple profiles. We now union every profile's deps so Snyk sees every dep any profile could pull in. `activeByDefault` wins only for property value conflicts.
152
- - **No `process.exit(1)` mid-pipeline**: a parse/rewrite failure for one POM logs and continues so the summary still prints.
153
- - **HTML report is self-contained**: inline CSS, no external assets. The `.doc` variant is the same HTML with Office XML namespace meta tags — Word opens it natively.
154
- - **Map keys are ecosystem-namespaced**: Maven uses `g:a`, npm uses `npm:name`. They never collide so they can share one resolved-deps Map.
155
- - **Lockfile-only npm**: `package.json` without sibling `package-lock.json`/`yarn.lock` is intentionally skipped (its ranges aren't queryable) and reported in chapter 0. Avoids false negatives on deps that haven't been installed yet.
156
- - **Source identifiers**: every match carries `source: "fad" | "osv" | "nvd" | "snyk" | "retire"` (or a `+`-joined combination like `"fad+osv+nvd"`). The legacy "mbdc" identifier was renamed to "fad" in 3.0.
157
-
158
- ## Gotchas / edge cases worth knowing
159
-
160
- - The CVE bundle from CVEProject is ~500 MB unpacked. We shell out to `curl + unzip` (Node built-in fallback to `fetch()` + system `unzip` / PowerShell `Expand-Archive`). The extracted JSON is deleted after the index is built.
161
- - The bundle ships as `cves.zip.zip` (a zip whose sole content is another zip). `extractZip()` recurses up to 3 levels.
162
- - `endoflife.date` API responses are cached locally for 7 days; Maven Central and npm registry version lookups for 24 hours.
163
- - **Persistent config**: `~/.fad-checker/config.json` (mode 0600) stores per-user state, currently the NVD API key. Set via `fad-checker --set-nvd-key <KEY>`.
164
- - **`--offline` umbrella flag**: skips every network call (CVE index download, OSV queries, NVD enrichment, endoflife.date lookups, Maven Central version queries, npm registry queries, transitive POM fetches, retire.js scans). Falls back to whatever is already cached. Per-source variants (`--cve-offline`, `--no-osv`, `--no-nvd`, `--no-retire`, `--no-js`) still work independently.
165
- - `snyk` is not a dependency — we shell out via `execFile`. `snyk` exits 1 when it finds vulnerabilities, which is expected (the JSON is still on stdout).
166
- - The cleaned POM is the union of every profile's deps. Counts will therefore be larger than the source POM. This is intentional — verify your reasoning before "reducing" them.
167
- - Unresolved `${…}` Maven variables are kept verbatim in the rewritten POM. `lib/cve-match.js` resolves them lazily via `resolveDepVersion()` when collecting deps for the scan. Deps that *still* can't be resolved (external BOM) are surfaced in chapter 0 as `unresolved-versions` warnings.
168
- - **Per-cache TTLs** are documented in the README's "Caching" table.
169
-
170
- ## Testing
171
-
172
- ```bash
173
- npm test # full suite (194 tests)
174
- node --test test/core.test.js # one file
175
- ```
176
-
177
- Test fixtures live in `test/fixtures/`:
178
- - `simple/` — 3 POMs with parent inheritance + property substitution
179
- - `complex-enterprise/` — Spring Boot parent (external), local BOM via `scope=import`, three profiles (two of which inject env-specific JDBC drivers), test-scoped JUnit, jackson-databind via BOM-managed version
180
- - `private-lib-detection/` — mixed public/private groupIds and an externally-hosted private parent — verifies missing-parent tracking
181
- - `monorepo-mixed/` — combined Maven (4 POMs: parent + BOM + 2 modules) + JS (npm package-lock v3 + yarn.lock v1 + a no-lockfile package.json to test the warning)
182
- - `cve-samples/` — small CVE / NVD JSON files to exercise the matchers without the 500 MB real bundle
package/docs/USAGE.md DELETED
@@ -1,187 +0,0 @@
1
- # USAGE
2
-
3
- Every flag, every common workflow, with copy-pasteable commands.
4
-
5
- ## Synopsis
6
-
7
- ```text
8
- fad-checker -s <src> [-t <target>] [-e <regex>] [other options]
9
- ```
10
-
11
- - `-s, --src <src>` — **required**. Root of the source tree to scan. Contains `pom.xml` and/or `package(-lock).json` / `yarn.lock`.
12
- - `-t, --target <dir>` — optional. If given, write a parallel directory of "cleaned" POMs (private/excluded deps stripped) to `<dir>` — useful as Snyk input. Without `-t`, the run is read-only.
13
-
14
- ## Output
15
-
16
- By default the HTML + Word reports land in `./fad-checker-report/`. Override with `--report-output <dir>`.
17
-
18
- ## Ecosystem selection
19
-
20
- ```bash
21
- # Auto-detect (default): scan whatever pom.xml / package(-lock).json / yarn.lock exists
22
- fad-checker -s .
23
-
24
- # Pick ecosystems (codecs). --ecosystem is a list: auto (default) | all | comma list.
25
- fad-checker -s . --ecosystem maven # Maven only
26
- fad-checker -s . --ecosystem maven,npm # both, even if only one is auto-detected
27
- fad-checker -s . --ecosystem all # every supported codec
28
- fad-checker -s . --ecosystem both # legacy alias for maven,npm
29
-
30
- # Opt out of specific codecs (combine freely)
31
- fad-checker -s . --no-npm # skip npm
32
- fad-checker -s . --no-js # alias: skip npm + yarn (Maven-only)
33
- fad-checker -s . --no-pypi --no-nuget # skip Python + C#
34
- ```
35
-
36
- > **npm without a lockfile**: a `package.json` lacking a sibling
37
- > `package-lock.json`/`yarn.lock` is now scanned **best-effort** — pinned exact
38
- > versions are checked, ranges (`^1.0.0`) are skipped, and a `no-lockfile` warning
39
- > flags the partial coverage. Run `npm install`/`yarn install` for full coverage.
40
-
41
- ## Filtering deps
42
-
43
- `-e <regex>` filters out coords whose **groupId** (Maven) or **name** (npm) matches the regex. Useful for private/internal libs that you know aren't on a public registry.
44
-
45
- ```bash
46
- fad-checker -s . -e "^(com\.acme|org\.private)\."
47
- fad-checker -s . -e "^@acme/"
48
- ```
49
-
50
- The excluded coords are listed at the end of the run so you can audit the regex.
51
-
52
- ## Per-source toggles
53
-
54
- Each data source can be disabled independently:
55
-
56
- | Flag | Effect |
57
- | --- | --- |
58
- | `--no-report` | Skip the report flow entirely (just cleanup) |
59
- | `--no-transitive` | Don't fetch transitive Maven deps from Maven Central |
60
- | `--no-all-libs` | Don't query Maven Central for latest versions (skips chapter 6 Outdated and the "missing on Central" check) |
61
- | `--no-osv` | Skip OSV.dev (Google + GitHub aggregated feed) |
62
- | `--no-nvd` | Skip NVD enrichment (no full CVSS, no CPE refinement) |
63
- | `--no-retire` | Skip retire.js vendored-JS scan |
64
- | `--ignore-test` | Drop test-scoped Maven deps and dev npm deps from the scan entirely (chapter 2 will be empty) |
65
-
66
- ## Offline / cache control
67
-
68
- ```bash
69
- # Use cached data only, no network (works for everything)
70
- fad-checker -s . --offline
71
-
72
- # Per-source offline
73
- fad-checker -s . --cve-offline # use cached CVE index only
74
- fad-checker -s . --cve-refresh # force re-download of CVE bundle
75
- fad-checker -s . --retire-refresh # force re-scan with retire.js (ignore cache)
76
-
77
- # Cache export / import (useful for air-gapped boxes)
78
- fad-checker --export-cache fad-cache.tar.gz
79
- fad-checker --export-cache fad-cache.tar.gz --include-config # bundle NVD key too
80
- fad-checker --import-cache fad-cache.tar.gz
81
- fad-checker --import-cache fad-cache.tar.gz --force # replace existing without backup
82
- ```
83
-
84
- ## NVD API key
85
-
86
- NVD's public rate limit is 5 requests / 30s without a key. The free key bumps it to 50 / 30s — **10× faster** for the enrichment step.
87
-
88
- ```bash
89
- # Get a key in 30 seconds: https://nvd.nist.gov/developers/request-an-api-key
90
- fad-checker --set-nvd-key YOUR_KEY # stored in ~/.fad-checker/config.json (mode 0600)
91
- fad-checker --show-config # confirm it's persisted (key masked)
92
- ```
93
-
94
- Or pass it ad-hoc via the `NVD_API_KEY` env var.
95
-
96
- ## Snyk integration
97
-
98
- If you have `snyk` installed and authenticated, `fad-checker` can drive it:
99
-
100
- ```bash
101
- fad-checker -s ./proj -t ../proj-clean -e "^com\.acme\." --snyk
102
- ```
103
-
104
- This:
105
- 1. Generates the cleaned POM tree at `../proj-clean/`.
106
- 2. Runs `snyk test --all-projects --json` against it.
107
- 3. Merges Snyk's findings into the report — entries present in both `fad-checker` and Snyk are tagged `source: "both"`.
108
-
109
- `--snyk` requires `-t` (Snyk needs a real POM tree to scan).
110
-
111
- ## Read-only vs write mode
112
-
113
- | Mode | Trigger | Disk writes |
114
- | --- | --- | --- |
115
- | Read-only | `-t` omitted (default) | Only `~/.fad-checker/` caches and the report dir |
116
- | Write | `-t <dir>` provided | Above + the cleaned POM tree at `<dir>` (and `<dir>` is `rimraf`'d first!) |
117
-
118
- The `--target` guardrails refuse:
119
- - empty `--src`
120
- - `--target` equal to or a subdirectory of `--src`
121
-
122
- ## Verbosity
123
-
124
- ```bash
125
- fad-checker -s . -v # progress per source (OSV batches, NVD pages, retire scan, …)
126
- ```
127
-
128
- ## Shell completion
129
-
130
- ```bash
131
- fad-checker --completion bash > /etc/bash_completion.d/fad-checker
132
- fad-checker --completion zsh > ~/.zsh/completions/_fad-checker
133
- ```
134
-
135
- ## All flags at a glance
136
-
137
- ```bash
138
- fad-checker --help
139
- ```
140
-
141
- ## Recipes
142
-
143
- ### CI gate: fail the build on any CRITICAL prod CVE
144
-
145
- `fad-checker` exits 0 even when CVEs are found (it's a reporter, not a gate). Wire your own:
146
-
147
- ```bash
148
- fad-checker -s . --no-nvd > /dev/null
149
- # Then grep the report or parse the structured output (planned).
150
- ```
151
-
152
- (A `--fail-on critical` flag is a planned addition — track it in issues.)
153
-
154
- ### Diff two runs
155
-
156
- Keep dated copies of the report:
157
-
158
- ```bash
159
- fad-checker -s . --report-output reports/$(date +%F)
160
- diff reports/2026-04-01/cve-report.html reports/2026-05-01/cve-report.html
161
- ```
162
-
163
- ### Air-gapped scan
164
-
165
- On a connected machine:
166
-
167
- ```bash
168
- fad-checker -s ./dummy-empty-dir # populates ~/.fad-checker/ caches
169
- fad-checker --export-cache fad-cache.tar.gz --include-config
170
- ```
171
-
172
- Move `fad-cache.tar.gz` to the air-gapped box, then:
173
-
174
- ```bash
175
- fad-checker --import-cache fad-cache.tar.gz
176
- fad-checker -s ./real-project --offline
177
- ```
178
-
179
- ### Monorepo with Maven + JS + vendored JS
180
-
181
- The melino-style project (Java backend, React frontend in `web/`, vendored jQuery/PDF.js under `web/src/main/webapp/`):
182
-
183
- ```bash
184
- fad-checker -s . -e "^(com\.captcha|org\.voxaly|com\.voxaly)\."
185
- # → finds CVE in Maven deps, in web/package-lock.json deps,
186
- # AND in the vendored .js files under webapp/.
187
- ```