osv-scanner-mcp 0.13.0 → 0.14.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +9 -4
- package/dist/index.js +2 -1
- package/dist/osv/binaryDownloader.js +12 -10
- package/dist/osv/runner.js +6 -2
- package/dist/tools/scanProject.js +51 -3
- package/dist/utils/cargoLock.js +208 -53
- package/dist/utils/dependencyRelations.js +14 -6
- package/dist/utils/manifestDetector.js +20 -12
- package/dist/utils/manifestFormats.js +1 -1
- package/dist/utils/npmLockCopy.js +183 -0
- package/dist/utils/pomParent.js +10 -7
- package/dist/utils/requirementsFile.js +24 -5
- package/dist/utils/scanSnapshot.js +57 -13
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -29,6 +29,7 @@ An MCP server that wraps Google's [OSV-Scanner](https://github.com/google/osv-sc
|
|
|
29
29
|
- A manually installed binary (on `PATH` or set with `OSV_SCANNER_PATH`) is used first
|
|
30
30
|
- Set `OSV_MCP_AUTO_DOWNLOAD=0` to disable the automatic download
|
|
31
31
|
- Set `OSV_MCP_PREFER_DOWNLOAD=1` to always use the verified downloaded binary instead of one on `PATH` (recommended for production)
|
|
32
|
+
- The pinned version is osv-scanner 2.7.0. Its binary is cached in `~/.cache/osv-scanner-mcp/v2.7.0/`; directories of earlier versions (such as `v2.4.0/`) are no longer used and can be deleted
|
|
32
33
|
- Scans, `suggest_fix`, and `explain_vulnerability` access the network. Vulnerabilities are looked up in the OSV database (`api.osv.dev`), and **scanning `pom.xml` or `requirements.txt` also connects to deps.dev (`api.deps.dev`) to resolve transitive dependencies**. See [Network destinations and privacy](#network-destinations-and-privacy) for details and how to turn this off
|
|
33
34
|
|
|
34
35
|
## Setup
|
|
@@ -176,13 +177,13 @@ As an additional layer of defense, you can limit what the server can read and wr
|
|
|
176
177
|
|
|
177
178
|
### Network destinations and privacy
|
|
178
179
|
|
|
179
|
-
Destinations confirmed with osv-scanner v2.4.0 (2026-10-07):
|
|
180
|
+
Destinations confirmed with osv-scanner v2.4.0 (2026-10-07) and v2.7.0, the pinned version (2026-10-15):
|
|
180
181
|
|
|
181
182
|
| Operation | Destinations | Information sent |
|
|
182
183
|
|---|---|---|
|
|
183
184
|
| Scanning `pom.xml` (`scan_project` / `scan_java_project` / `suggest_fix`) | `api.osv.dev`, **`api.deps.dev`** | Package names and versions. To resolve transitive dependencies, the names and versions of the dependencies declared in `pom.xml` (including internal packages) are sent to deps.dev |
|
|
184
185
|
| Scanning `requirements.txt` (`scan_project` / `suggest_fix`) | `api.osv.dev`, **`api.deps.dev`** | As with `pom.xml`, the names and versions of the listed dependencies are sent to deps.dev to resolve transitive dependencies. Package indexes given with `--index-url` and similar options are not contacted |
|
|
185
|
-
| Scanning lockfiles (`gradle.lockfile`, npm lockfiles such as `package-lock.json`, Python lockfiles such as `poetry.lock`, `go.mod`, `Cargo.lock`) | `api.osv.dev` | Package names and versions (lockfiles already list every dependency, so nothing is resolved remotely). For `Cargo.lock`, only crates from crates.io are sent: the names of your own crates, git dependencies, and private-registry crates are not sent (confirmed 2026-10-11) |
|
|
186
|
+
| Scanning lockfiles (`gradle.lockfile`, npm lockfiles such as `package-lock.json`, Python lockfiles such as `poetry.lock`, `go.mod`, `Cargo.lock`) | `api.osv.dev` | Package names and versions (lockfiles already list every dependency, so nothing is resolved remotely). For `Cargo.lock`, only crates from crates.io are sent: the names of your own crates, git dependencies, and private-registry crates are not sent (confirmed 2026-10-11). **Git dependencies in npm lockfiles** (`package-lock.json`, `yarn.lock`, and so on) are looked up by their repository URL (ecosystem `GIT`) and commit, so the host name and path of a private git repository (for example `https://git.example.com/team/repo`) are sent (since osv-scanner 2.6.0) |
|
|
186
187
|
| `scan_java_artifact` / `scan_sbom` | `api.osv.dev` | Names and versions of the identified packages |
|
|
187
188
|
| `suggest_fix` checking recommended versions | `api.osv.dev` | The names of packages with a recommendation (already sent during the scan) and the candidate versions (published fixed versions). Disable with `OSV_MCP_NO_CANDIDATE_CHECK=1` |
|
|
188
189
|
| `explain_vulnerability` | `api.osv.dev` | The vulnerability ID |
|
|
@@ -219,7 +220,11 @@ Detects the lockfiles and manifests in a project and scans the Java / JavaScript
|
|
|
219
220
|
|
|
220
221
|
**`requirements.txt` files are not passed to OSV-Scanner as they are.** The server parses them, rewrites only the dependency lines it can interpret into simple forms such as `name==version`, and scans a private temporary copy (deleted after the scan). OSV-Scanner interprets include directives on its own (it even treats `- r ../x.txt`, with a space, as an include and reads files outside the scan scope), so the copy never contains include directives or options. Includes (`-r` / `--requirement`) are expanded by the server, and only when the included file is inside the project directory.
|
|
221
222
|
|
|
222
|
-
**`Cargo.lock` files are not passed to OSV-Scanner as they are either.** OSV-Scanner (v2.4.0) ignores where each package comes from and looks every package up as a crates.io crate, so a workspace crate, a git fork, or a private-registry crate with the same name and version as a vulnerable crates.io crate is reported as vulnerable, and its name is sent to the OSV API. The server parses `Cargo.lock` and scans a private copy that lists only the packages whose source is crates.io (`registry+https://github.com/rust-lang/crates.io-index` or `sparse+https://index.crates.io/`), like `cargo audit`. Packages replaced with `[replace]` are not scanned, because they are not used (the replacement is classified by its own source). The parser accepts only what Cargo writes; if a `Cargo.lock` contains anything else, the whole file is excluded and listed in `skipped_files` instead of being scanned partially. Cargo is never run.
|
|
223
|
+
**`Cargo.lock` files are not passed to OSV-Scanner as they are either.** OSV-Scanner (v2.4.0 and v2.7.0) ignores where each package comes from and looks every package up as a crates.io crate, so a workspace crate, a git fork, or a private-registry crate with the same name and version as a vulnerable crates.io crate is reported as vulnerable, and its name is sent to the OSV API. The server parses `Cargo.lock` and scans a private copy that lists only the packages whose source is crates.io (`registry+https://github.com/rust-lang/crates.io-index` or `sparse+https://index.crates.io/`), like `cargo audit`. Packages replaced with `[replace]` are not scanned, because they are not used (the replacement is classified by its own source). The parser accepts only what Cargo writes; if a `Cargo.lock` contains anything else, the whole file is excluded and listed in `skipped_files` instead of being scanned partially. Cargo is never run.
|
|
224
|
+
|
|
225
|
+
**`package-lock.json` copies get a `resolved` URL where it is missing.** osv-scanner 2.7.0 treats entries without `resolved` as local packages and does not look them up, which would silently skip bundled dependencies (`inBundle`) and registry packages in lockfiles generated with npm's `omit-lockfile-registry-resolved=true`. The private copy that is scanned gets a public-registry URL for those entries (entries under `node_modules/` that are not links and have a plain version, and aliases such as `npm:lodash@4.17.20` in lockfile version 1), so they are looked up as before; the original file is not changed.
|
|
226
|
+
|
|
227
|
+
**npm JSON limits:** Normalization of `package-lock.json` and `npm-shrinkwrap.json` is limited to 16 MiB per file and 64 MiB per scan, for both input and output. Before parsing, JSON nesting is limited to 128 levels and structural markers (containers, commas, and colons outside strings) to 1,000,000 per file and 2,000,000 per analysis phase. Changed copies use compact JSON, with their size checked before serialization. A lockfile that exceeds these limits is scanned as it is, without filling in `resolved` (OSV-Scanner parses large lockfiles itself; the limits only bound this server's own parsing and rewriting), and it is listed in `coverage.skipped_files` with the reason, so `complete` is `false`: entries without `resolved` in it may not have been looked up. Workspace membership and dependency-relation lookups use the same structural checks and remain unconfirmed or unknown when limits are reached.
|
|
223
228
|
|
|
224
229
|
**Reading the output**
|
|
225
230
|
|
|
@@ -351,7 +356,7 @@ A directory search includes `target` and `build`, and excludes `.git`, `node_mod
|
|
|
351
356
|
The search is limited to a depth of 8, 100 archives, and 10,000 entries. If the search cannot complete within the limits, `artifact_search_limit_exceeded` is returned instead of silently truncating the results; narrow the target and try again.
|
|
352
357
|
`OSV_MCP_ALLOWED_ROOT` applies as well.
|
|
353
358
|
|
|
354
|
-
OSV-Scanner
|
|
359
|
+
OSV-Scanner's `java/archive` plugin is used, and nested JARs are also analyzed by the scanner. No Java code is executed and no build is run.
|
|
355
360
|
Identification relies on metadata inside the archives, so dependencies whose metadata was removed, and dependencies inside shaded/minimized JARs, may be missed.
|
|
356
361
|
|
|
357
362
|
**Reading the output:**
|
package/dist/index.js
CHANGED
|
@@ -61,7 +61,7 @@ if (startupWarning !== null) {
|
|
|
61
61
|
// NOTE: リリース時はpackage.jsonのversionと同じ値に更新すること
|
|
62
62
|
const server = new McpServer({
|
|
63
63
|
name: "osv-scanner-mcp",
|
|
64
|
-
version: "0.
|
|
64
|
+
version: "0.14.0",
|
|
65
65
|
});
|
|
66
66
|
server.registerTool("scan_project", {
|
|
67
67
|
title: "Scan project dependencies for vulnerabilities (Java / JavaScript / Python / Go / Rust)",
|
|
@@ -70,6 +70,7 @@ server.registerTool("scan_project", {
|
|
|
70
70
|
"Python (poetry.lock / uv.lock / Pipfile.lock / pdm.lock / requirements.txt), Go (go.mod), Rust (Cargo.lock). " +
|
|
71
71
|
"Package managers and builds are never run. " +
|
|
72
72
|
"Always check coverage at the top of the response: it lists manifests without a lockfile, requirements lines without a pinned version, and excluded files. " +
|
|
73
|
+
"Git dependencies in npm lockfiles are looked up by repository URL, so private git host names and paths are sent to api.osv.dev. " +
|
|
73
74
|
"For Cargo.lock, only crates from crates.io are scanned: the project's own crates are counted in local_crates, and git or private-registry crates are listed in unscannable_crates. " +
|
|
74
75
|
"Vulnerabilities marked informational (RustSec notices such as unmaintained or unsound crates) are not counted in vulnerability_count; they are counted in informational_count. " +
|
|
75
76
|
"Each package's dependency_relation tells whether it is a direct or transitive dependency (determined for package-lock.json, go.mod, requirements.txt, and pom.xml; unknown otherwise). " +
|
|
@@ -21,20 +21,22 @@ import { readResponseBytes } from "../utils/readResponseBytes.js";
|
|
|
21
21
|
* 範囲外の読み込みを防ぐ検証はこの版の挙動に合わせているため、更新時は
|
|
22
22
|
* docs/DESIGN_TODO.md「B5」の監査と、requirements.txt・親POMの解釈の実機確認をやり直すこと。
|
|
23
23
|
* pom.xmlの直接/推移的依存の判定(dependencyRelations.tsのpomRelations)も、この版が同じpom.xmlを
|
|
24
|
-
* source.type "lockfile"(宣言された依存)と"unknown"(deps.devで解決された推移的依存)
|
|
24
|
+
* source.type "lockfile"(宣言された依存)と"unknown"(deps.devで解決された推移的依存)に分けて報告する挙動に依存するため再確認すること。
|
|
25
|
+
* 2.7.0への更新時の確認結果はdocs/DESIGN_TODO.md「osv-scannerのピン留め更新(2.4.0 → 2.7.0)」(2026-10-15)
|
|
25
26
|
*/
|
|
26
|
-
export const PINNED_OSV_SCANNER_VERSION = "2.
|
|
27
|
+
export const PINNED_OSV_SCANNER_VERSION = "2.7.0";
|
|
27
28
|
/**
|
|
28
|
-
* v2.
|
|
29
|
-
*
|
|
29
|
+
* v2.7.0公式リリースのSHA256(2026-10-15取得)。osv-scanner_SHA256SUMSとGitHub APIのリリース資産のdigestの
|
|
30
|
+
* 両方で一致を確認し、darwin_arm64はダウンロードした実体のハッシュとも照合した。
|
|
31
|
+
* https://github.com/google/osv-scanner/releases/tag/v2.7.0
|
|
30
32
|
*/
|
|
31
33
|
const ASSET_CHECKSUMS = {
|
|
32
|
-
"osv-scanner_darwin_amd64": "
|
|
33
|
-
"osv-scanner_darwin_arm64": "
|
|
34
|
-
"osv-scanner_linux_amd64": "
|
|
35
|
-
"osv-scanner_linux_arm64": "
|
|
36
|
-
"osv-scanner_windows_amd64.exe": "
|
|
37
|
-
"osv-scanner_windows_arm64.exe": "
|
|
34
|
+
"osv-scanner_darwin_amd64": "e82c5df7ab0a42dff29a1cd0660711447b56b4c0483ff66ad8634df10fd81a45",
|
|
35
|
+
"osv-scanner_darwin_arm64": "d4926ef8da134332543be276dff157e4c170f99ad91fc48e89448a8816bdf70b",
|
|
36
|
+
"osv-scanner_linux_amd64": "160c6253e2922e9e0a6a9771c5c4c40a4b3c6090b34a3cdd5e80c96cc2e4218b",
|
|
37
|
+
"osv-scanner_linux_arm64": "4061ef4c021ed3c47f203b818f337e351336cc5cd67c0274fc36bd1dfc728dd7",
|
|
38
|
+
"osv-scanner_windows_amd64.exe": "9964ed64ec9a387391eb4cc006ad5c531154821aef29b793e02c448de4686438",
|
|
39
|
+
"osv-scanner_windows_arm64.exe": "3e630381c1f1fd3c9c4d447e6ff2ef6c737083371722773db8337eead0f97962",
|
|
38
40
|
};
|
|
39
41
|
const DOWNLOAD_TIMEOUT_MS = 300_000;
|
|
40
42
|
/** バイナリは実測50〜55MB。余裕を持たせつつ暴走は防ぐ */
|
package/dist/osv/runner.js
CHANGED
|
@@ -12,7 +12,7 @@
|
|
|
12
12
|
* - タイムアウトと出力サイズ上限を設ける(ハング・巨大出力によるDoS対策)
|
|
13
13
|
* - サーバーの終了時に実行中のプロセスを残さない(processCleanup.tsに登録し、終了時にSIGKILL)
|
|
14
14
|
*
|
|
15
|
-
* 終了コード(2.4.0で実機確認):
|
|
15
|
+
* 終了コード(2.4.0・2.7.0で実機確認):
|
|
16
16
|
* 0 = スキャン成功・脆弱性なし / 1 = スキャン成功・脆弱性あり / 128 = 対象パッケージなし
|
|
17
17
|
*/
|
|
18
18
|
import { spawn } from "node:child_process";
|
|
@@ -56,7 +56,7 @@ export function isRemoteResolutionDisabled(options = {}) {
|
|
|
56
56
|
/**
|
|
57
57
|
* OSV-Scannerに渡す固定引数。ここに無いオプションは一切使わない(ホワイトリスト)
|
|
58
58
|
*
|
|
59
|
-
* pom.xml/requirements.txtの推移的依存はapi.deps.devで解決される(2.4.0で実機確認)。
|
|
59
|
+
* pom.xml/requirements.txtの推移的依存はapi.deps.devで解決される(2.4.0・2.7.0で実機確認)。
|
|
60
60
|
* `--data-source native` はスキャン対象pom.xmlの<repositories>に書かれた任意のURLへ
|
|
61
61
|
* 接続するため使わない。既定値の変更に備えてdeps.devを明示する。
|
|
62
62
|
*/
|
|
@@ -71,6 +71,10 @@ export function buildProjectTargetArgs(targets) {
|
|
|
71
71
|
return ["--lockfile", `${target.format}:${target.path}`];
|
|
72
72
|
});
|
|
73
73
|
}
|
|
74
|
+
/**
|
|
75
|
+
* `--experimental-`のフラグは2.7.0で非推奨(stderrに警告。新名は`--x-`)だが、新名は2.4.0では未定義でexit 127になる。
|
|
76
|
+
* OSV_SCANNER_PATH・PATH上の古い版でも動くよう、両方の版で使える旧名を使う(2026-10-15 実機確認)
|
|
77
|
+
*/
|
|
74
78
|
const FIXED_ARTIFACT_ARGS = [
|
|
75
79
|
"scan", "source", "--format", "json", "--all-packages", "--no-ignore",
|
|
76
80
|
"--experimental-no-default-plugins", "--experimental-plugins", "java/archive",
|
|
@@ -4,6 +4,7 @@
|
|
|
4
4
|
* 誤要約対策(JAR実体スキャンと同じ原則): スキャンできなかった依存を示すcoverageを件数より前に置き、
|
|
5
5
|
* complete=falseの場合は「検出0件でも安全とは言えない」旨の警告を付ける。
|
|
6
6
|
*/
|
|
7
|
+
import { writeFile } from "node:fs/promises";
|
|
7
8
|
import path from "node:path";
|
|
8
9
|
import { ScanToolError } from "../errors.js";
|
|
9
10
|
import { isRemoteResolutionDisabled, runOsvScan } from "../osv/runner.js";
|
|
@@ -13,6 +14,7 @@ import { cargoLockCopyContent } from "../utils/cargoLock.js";
|
|
|
13
14
|
import { sanitizeExternalText } from "../utils/externalText.js";
|
|
14
15
|
import { readRegularFile } from "../utils/safeRead.js";
|
|
15
16
|
import { detectProject } from "../utils/manifestDetector.js";
|
|
17
|
+
import { createNpmLockWorkBudget, fillMissingResolved, MAX_NPM_LOCK_BYTES, MAX_NPM_LOCK_TOTAL_BYTES, parseNpmLock } from "../utils/npmLockCopy.js";
|
|
16
18
|
import { normalizePypiName } from "../utils/requirementsFile.js";
|
|
17
19
|
import { ScanSnapshot, snapshotManifests } from "../utils/scanSnapshot.js";
|
|
18
20
|
import { dependencyResolution, withScopeNotes } from "./scanJavaProject.js";
|
|
@@ -112,7 +114,7 @@ export function markLowerBounds(project, packages) {
|
|
|
112
114
|
* 検証済みの正規化行だけを書いたコピー。コピーできず外したファイルはskippedFilesに記録する。
|
|
113
115
|
* スナップショットは成功・失敗とも削除する。suggest_fixも同じスキャンを使う。
|
|
114
116
|
*/
|
|
115
|
-
/** 直接/推移的依存の判定で読むpackage-lock.json・go.modの合計サイズの上限
|
|
117
|
+
/** 直接/推移的依存の判定で読むpackage-lock.json・go.modの合計サイズの上限 */
|
|
116
118
|
const MAX_RELATION_LOCKFILE_BYTES = 256 * 1024 * 1024;
|
|
117
119
|
const MAX_INTRODUCED_BY = 10;
|
|
118
120
|
/**
|
|
@@ -121,6 +123,7 @@ const MAX_INTRODUCED_BY = 10;
|
|
|
121
123
|
*/
|
|
122
124
|
export async function buildRelationLookups(copies, maxTotalBytes = MAX_RELATION_LOCKFILE_BYTES) {
|
|
123
125
|
const lookups = new Map();
|
|
126
|
+
const npmWorkBudget = createNpmLockWorkBudget();
|
|
124
127
|
let remaining = maxTotalBytes;
|
|
125
128
|
for (const { copy, format, originalRelative, entries, cargoPackages, declaringManifests } of copies) {
|
|
126
129
|
const lockDir = originalRelative === null ? "" : path.dirname(originalRelative);
|
|
@@ -128,11 +131,11 @@ export async function buildRelationLookups(copies, maxTotalBytes = MAX_RELATION_
|
|
|
128
131
|
try {
|
|
129
132
|
if (format === "package-lock.json" || format === "go.mod") {
|
|
130
133
|
// 残りの予算を上限に、読み込みの途中で打ち切る(予算を超えるファイルは全体を読まずにunknownにする)
|
|
131
|
-
const read = await readRegularFile(copy, { maxBytes: remaining });
|
|
134
|
+
const read = await readRegularFile(copy, { maxBytes: format === "package-lock.json" ? Math.min(MAX_NPM_LOCK_BYTES, remaining) : remaining });
|
|
132
135
|
if (read.ok) {
|
|
133
136
|
remaining -= read.bytes.length;
|
|
134
137
|
const text = read.bytes.toString("utf8");
|
|
135
|
-
lookup = format === "go.mod" ? goModRelations(text) : npmLockRelations(
|
|
138
|
+
lookup = format === "go.mod" ? goModRelations(text) : npmLockRelations(parseNpmLock(text, npmWorkBudget));
|
|
136
139
|
}
|
|
137
140
|
}
|
|
138
141
|
else if (format === "pom.xml") {
|
|
@@ -188,6 +191,41 @@ function annotateRelations(packages, lookups) {
|
|
|
188
191
|
};
|
|
189
192
|
});
|
|
190
193
|
}
|
|
194
|
+
export const NPM_LOCK_NOT_NORMALIZED = "Scanned without filling in missing `resolved` fields, because the lockfile exceeds the normalization limits " +
|
|
195
|
+
`(${MAX_NPM_LOCK_BYTES / 1024 / 1024} MiB per file; ${MAX_NPM_LOCK_TOTAL_BYTES / 1024 / 1024} MiB and a JSON structure budget per scan). ` +
|
|
196
|
+
"Entries without `resolved` (such as bundled dependencies) may not have been looked up";
|
|
197
|
+
/**
|
|
198
|
+
* コピーにresolvedを補う。構文エラーの場合はそのまま渡す。
|
|
199
|
+
* 資源の上限を超えた場合もスキャン全体は止めず、コピーをそのまま渡して理由を返す
|
|
200
|
+
* (osv-scannerは大きなlockfileも解析できる。上限は本サーバー自身の解析・再出力を抑えるためのもの)
|
|
201
|
+
*/
|
|
202
|
+
async function fillResolvedInCopy(copyPath, budget) {
|
|
203
|
+
const read = await readRegularFile(copyPath, { maxBytes: Math.min(MAX_NPM_LOCK_BYTES, budget.input) });
|
|
204
|
+
if (!read.ok) {
|
|
205
|
+
if (read.failure === "too_large")
|
|
206
|
+
return NPM_LOCK_NOT_NORMALIZED;
|
|
207
|
+
throw new ScanToolError("scan_failed", "The npm lockfile copy cannot be read safely");
|
|
208
|
+
}
|
|
209
|
+
budget.input -= read.bytes.length;
|
|
210
|
+
const text = read.bytes.toString("utf8");
|
|
211
|
+
let filled;
|
|
212
|
+
try {
|
|
213
|
+
filled = fillMissingResolved(text, Math.min(MAX_NPM_LOCK_BYTES, budget.output), budget.work);
|
|
214
|
+
}
|
|
215
|
+
catch (error) {
|
|
216
|
+
if (error instanceof ScanToolError && error.kind === "scan_input_too_large")
|
|
217
|
+
return NPM_LOCK_NOT_NORMALIZED;
|
|
218
|
+
throw error;
|
|
219
|
+
}
|
|
220
|
+
if (!filled?.changed)
|
|
221
|
+
return null; // 書き換えないコピーは出力の上限に数えない
|
|
222
|
+
const outputBytes = Buffer.byteLength(filled.content, "utf8");
|
|
223
|
+
if (outputBytes > budget.output)
|
|
224
|
+
return NPM_LOCK_NOT_NORMALIZED;
|
|
225
|
+
budget.output -= outputBytes;
|
|
226
|
+
await writeFile(copyPath, filled.content, { mode: 0o600 });
|
|
227
|
+
return null;
|
|
228
|
+
}
|
|
191
229
|
export async function scanFromSnapshot(project, options) {
|
|
192
230
|
const snapshot = await ScanSnapshot.create();
|
|
193
231
|
try {
|
|
@@ -198,6 +236,16 @@ export async function scanFromSnapshot(project, options) {
|
|
|
198
236
|
// コピーと元のファイルの対応(snapshotManifestsは外したもの以外を入力の順に返す)
|
|
199
237
|
const skippedOriginals = new Set(skipped.map((s) => s.path));
|
|
200
238
|
const scannedOriginals = project.targets.filter((t) => !skippedOriginals.has(t.path));
|
|
239
|
+
// osv-scanner 2.7.0はresolvedの無いnpmのエントリを照会しないため、コピーで補う(npmLockCopy.ts)。
|
|
240
|
+
// 上限を超えたlockfileはそのままスキャンし、補えなかったことをcoverageに出す(complete=falseになる)
|
|
241
|
+
const npmBudget = { input: MAX_NPM_LOCK_TOTAL_BYTES, output: MAX_NPM_LOCK_TOTAL_BYTES, work: createNpmLockWorkBudget() };
|
|
242
|
+
for (const [i, copy] of targets.entries()) {
|
|
243
|
+
if (copy.format !== "package-lock.json")
|
|
244
|
+
continue;
|
|
245
|
+
const reason = await fillResolvedInCopy(copy.path, npmBudget);
|
|
246
|
+
if (reason !== null)
|
|
247
|
+
project.skippedFiles.push({ path: path.relative(project.projectDir, scannedOriginals[i].path), reason });
|
|
248
|
+
}
|
|
201
249
|
const relationSources = targets.map((copy, i) => ({ copy: copy.path, format: copy.format, originalRelative: path.relative(project.projectDir, scannedOriginals[i].path) }));
|
|
202
250
|
// 親POMを再現できずにスキャンしたpom.xmlも、欠落の可能性としてcoverageに出す(complete=falseになる)
|
|
203
251
|
project.skippedFiles.push(...incomplete.map((s) => ({ path: path.relative(project.projectDir, s.path), reason: s.reason })));
|
package/dist/utils/cargoLock.js
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
/**
|
|
2
2
|
* Cargo.lockの解析と、osv-scannerに渡すコピーの内容の生成(cargoは実行しない)。
|
|
3
3
|
*
|
|
4
|
-
* osv-scanner v2.4.0はCargo.lockの`source`を見ずに全パッケージをcrates.ioとして照会する(2026-10-11 実機確認)。
|
|
4
|
+
* osv-scanner v2.4.0・v2.7.0はCargo.lockの`source`を見ずに全パッケージをcrates.ioとして照会する(2026-10-11 実機確認)。
|
|
5
5
|
* workspaceのメンバー・pathの依存・gitの依存・別レジストリの依存も、同名・同版のcrates.ioのアドバイザリが付き(誤検知)、
|
|
6
6
|
* 社内crateの名前がapi.osv.devへ送られる。そこで元のファイルは渡さず、ここで解析したcrates.io由来の
|
|
7
7
|
* パッケージだけを書いたコピーをスキャンする(docs/DESIGN_TODO.md「Rust(crates.io)対応」)。
|
|
@@ -16,6 +16,18 @@ const CRATES_IO_SOURCES = new Set([
|
|
|
16
16
|
]);
|
|
17
17
|
/** 1つのCargo.lockで受け付けるパッケージ数の上限 */
|
|
18
18
|
const MAX_PACKAGES = 50_000;
|
|
19
|
+
const MAX_LOCK_STRING_CHARS = 65_536;
|
|
20
|
+
export function createCargoLockParseBudget() {
|
|
21
|
+
return { remainingLines: 1_000_000, remainingDependencies: 200_000, remainingStringChars: 16 * 1024 * 1024 };
|
|
22
|
+
}
|
|
23
|
+
function takeDependency(budget) {
|
|
24
|
+
if (budget.remainingDependencies <= 0) {
|
|
25
|
+
budget.exceeded = "the dependency entry limit was exceeded";
|
|
26
|
+
return false;
|
|
27
|
+
}
|
|
28
|
+
budget.remainingDependencies--;
|
|
29
|
+
return true;
|
|
30
|
+
}
|
|
19
31
|
/** crateの名前(cargoの制約より緩め。英数字・`-`・`_`) */
|
|
20
32
|
const CRATE_NAME = /^[A-Za-z0-9][A-Za-z0-9_-]{0,63}$/;
|
|
21
33
|
/** 版(SemVerの文字の範囲) */
|
|
@@ -34,20 +46,43 @@ const BARE_KEY = /^([A-Za-z0-9_-]+)\s*=\s*(.*)$/;
|
|
|
34
46
|
* TOMLの基本文字列(`"..."`)を先頭から1つ読む。読めなければnull。
|
|
35
47
|
* cargoが書くエスケープ(`\"` `\\` `\n` `\t` `\uXXXX` `\UXXXXXXXX`等)だけを受け付ける
|
|
36
48
|
*/
|
|
37
|
-
function readBasicString(text, start) {
|
|
49
|
+
function readBasicString(text, start, budget) {
|
|
38
50
|
if (text[start] !== '"')
|
|
39
51
|
return null;
|
|
40
|
-
|
|
52
|
+
const parts = [];
|
|
53
|
+
let segmentStart = start + 1;
|
|
54
|
+
let length = 0;
|
|
55
|
+
const takeChars = (count) => {
|
|
56
|
+
if (budget === undefined)
|
|
57
|
+
return true;
|
|
58
|
+
if (length + count > MAX_LOCK_STRING_CHARS) {
|
|
59
|
+
budget.exceeded = "the string length limit was exceeded";
|
|
60
|
+
return false;
|
|
61
|
+
}
|
|
62
|
+
if (count > budget.remainingStringChars) {
|
|
63
|
+
budget.exceeded = "the total string character limit was exceeded";
|
|
64
|
+
return false;
|
|
65
|
+
}
|
|
66
|
+
length += count;
|
|
67
|
+
budget.remainingStringChars -= count;
|
|
68
|
+
return true;
|
|
69
|
+
};
|
|
41
70
|
for (let i = start + 1; i < text.length; i++) {
|
|
42
71
|
const ch = text[i];
|
|
43
|
-
if (ch === '"')
|
|
44
|
-
|
|
72
|
+
if (ch === '"') {
|
|
73
|
+
parts.push(text.slice(segmentStart, i));
|
|
74
|
+
return { value: parts.join(""), end: i + 1 };
|
|
75
|
+
}
|
|
45
76
|
if (ch === "\\") {
|
|
77
|
+
parts.push(text.slice(segmentStart, i));
|
|
46
78
|
const next = text[i + 1];
|
|
47
79
|
const simple = { '"': '"', "\\": "\\", n: "\n", t: "\t", r: "\r", b: "\b", f: "\f" };
|
|
48
80
|
if (next !== undefined && next in simple) {
|
|
49
|
-
|
|
81
|
+
if (!takeChars(1))
|
|
82
|
+
return null;
|
|
83
|
+
parts.push(simple[next]);
|
|
50
84
|
i++;
|
|
85
|
+
segmentStart = i + 1;
|
|
51
86
|
continue;
|
|
52
87
|
}
|
|
53
88
|
const width = next === "u" ? 4 : next === "U" ? 8 : 0;
|
|
@@ -57,13 +92,18 @@ function readBasicString(text, start) {
|
|
|
57
92
|
const codePoint = Number.parseInt(hex, 16);
|
|
58
93
|
if (codePoint > 0x10ffff || (codePoint >= 0xd800 && codePoint <= 0xdfff))
|
|
59
94
|
return null;
|
|
60
|
-
|
|
95
|
+
const decoded = String.fromCodePoint(codePoint);
|
|
96
|
+
if (!takeChars(decoded.length))
|
|
97
|
+
return null;
|
|
98
|
+
parts.push(decoded);
|
|
61
99
|
i += 1 + width;
|
|
100
|
+
segmentStart = i + 1;
|
|
62
101
|
continue;
|
|
63
102
|
}
|
|
64
103
|
if (ch.charCodeAt(0) < 0x20 && ch !== "\t")
|
|
65
104
|
return null;
|
|
66
|
-
|
|
105
|
+
if (!takeChars(1))
|
|
106
|
+
return null;
|
|
67
107
|
}
|
|
68
108
|
return null;
|
|
69
109
|
}
|
|
@@ -72,8 +112,8 @@ function restIsBlank(rest) {
|
|
|
72
112
|
return /^\s*(?:#.*)?$/.test(rest);
|
|
73
113
|
}
|
|
74
114
|
/** 1行全体が文字列1つ(と任意の末尾カンマ・コメント)の場合にその値を返す */
|
|
75
|
-
function lineString(text, allowComma) {
|
|
76
|
-
const parsed = readBasicString(text, 0);
|
|
115
|
+
function lineString(text, allowComma, budget) {
|
|
116
|
+
const parsed = readBasicString(text, 0, budget);
|
|
77
117
|
if (parsed === null)
|
|
78
118
|
return null;
|
|
79
119
|
let rest = text.slice(parsed.end);
|
|
@@ -82,7 +122,7 @@ function lineString(text, allowComma) {
|
|
|
82
122
|
return restIsBlank(rest) ? parsed.value : null;
|
|
83
123
|
}
|
|
84
124
|
/** `["a", "b"]`形式(1行の配列)の要素。読めなければnull */
|
|
85
|
-
function inlineStringArray(text) {
|
|
125
|
+
function inlineStringArray(text, budget) {
|
|
86
126
|
if (!text.startsWith("["))
|
|
87
127
|
return null;
|
|
88
128
|
const items = [];
|
|
@@ -92,7 +132,9 @@ function inlineStringArray(text) {
|
|
|
92
132
|
i++;
|
|
93
133
|
if (text[i] === "]")
|
|
94
134
|
return restIsBlank(text.slice(i + 1)) ? items : null;
|
|
95
|
-
|
|
135
|
+
if (!takeDependency(budget))
|
|
136
|
+
return null;
|
|
137
|
+
const parsed = readBasicString(text, i, budget);
|
|
96
138
|
if (parsed === null)
|
|
97
139
|
return null;
|
|
98
140
|
items.push(parsed.value);
|
|
@@ -110,15 +152,17 @@ function inlineStringArray(text) {
|
|
|
110
152
|
* 受け付ける行: 空行・コメント、先頭の`version = 整数`、`[[package]]`、`[[patch.unused]]`(読み飛ばす)、
|
|
111
153
|
* `[metadata]`(v1のchecksum。読み飛ばす)、`キー = "文字列"`、`dependencies = [` 〜 `]`の各要素
|
|
112
154
|
*/
|
|
113
|
-
export function parseCargoLock(text) {
|
|
155
|
+
export function parseCargoLock(text, budget = createCargoLockParseBudget()) {
|
|
114
156
|
const packages = [];
|
|
115
157
|
let current = null;
|
|
116
158
|
let state = "top";
|
|
117
159
|
let sawTable = false;
|
|
118
160
|
const fail = (line, what) => ({
|
|
119
161
|
ok: false,
|
|
120
|
-
reason: `Cargo.lock could not be interpreted with certainty (${what} at line ${line}), so the file was excluded from the scan`,
|
|
162
|
+
reason: `Cargo.lock could not be interpreted with certainty (${budget.exceeded ?? what} at line ${line}), so the file was excluded from the scan`,
|
|
121
163
|
});
|
|
164
|
+
if (budget.exceeded !== undefined)
|
|
165
|
+
return fail(1, budget.exceeded);
|
|
122
166
|
const finish = (line) => {
|
|
123
167
|
if (current === null)
|
|
124
168
|
return null;
|
|
@@ -135,10 +179,18 @@ export function parseCargoLock(text) {
|
|
|
135
179
|
packages.push({ name, version, source: source ?? null, dependencies, ...(replace !== undefined ? { replace } : {}) });
|
|
136
180
|
return null;
|
|
137
181
|
};
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
182
|
+
let lineNo = 0;
|
|
183
|
+
// Do not materialize a line array: even blank lines can amplify a small input.
|
|
184
|
+
for (let start = 0; start <= text.length;) {
|
|
185
|
+
lineNo++;
|
|
186
|
+
if (budget.remainingLines <= 0) {
|
|
187
|
+
budget.exceeded = "the line limit was exceeded";
|
|
188
|
+
return fail(lineNo, budget.exceeded);
|
|
189
|
+
}
|
|
190
|
+
budget.remainingLines--;
|
|
191
|
+
const end = text.indexOf("\n", start);
|
|
192
|
+
const line = text.slice(start, end === -1 ? text.length : end).trim();
|
|
193
|
+
start = end === -1 ? text.length + 1 : end + 1;
|
|
142
194
|
if (state === "dependencies") {
|
|
143
195
|
if (line === "" || line.startsWith("#"))
|
|
144
196
|
continue;
|
|
@@ -146,7 +198,9 @@ export function parseCargoLock(text) {
|
|
|
146
198
|
state = "package";
|
|
147
199
|
continue;
|
|
148
200
|
}
|
|
149
|
-
|
|
201
|
+
if (!takeDependency(budget))
|
|
202
|
+
return fail(lineNo, budget.exceeded);
|
|
203
|
+
const item = lineString(line, true, budget);
|
|
150
204
|
if (item === null)
|
|
151
205
|
return fail(lineNo, "an unexpected dependency entry");
|
|
152
206
|
current.dependencies.push(item);
|
|
@@ -183,9 +237,9 @@ export function parseCargoLock(text) {
|
|
|
183
237
|
return fail(lineNo, "an unexpected table");
|
|
184
238
|
if (state === "metadata") {
|
|
185
239
|
// v1: "checksum 名前 版 (source)" = "ハッシュ"
|
|
186
|
-
const key = readBasicString(line, 0);
|
|
240
|
+
const key = readBasicString(line, 0, budget);
|
|
187
241
|
const valueMatch = key === null ? null : /^\s*=\s*(.*)$/.exec(line.slice(key.end));
|
|
188
|
-
if (valueMatch === null || lineString(valueMatch[1], false) === null)
|
|
242
|
+
if (valueMatch === null || lineString(valueMatch[1], false, budget) === null)
|
|
189
243
|
return fail(lineNo, "an unexpected metadata entry");
|
|
190
244
|
continue;
|
|
191
245
|
}
|
|
@@ -205,14 +259,15 @@ export function parseCargoLock(text) {
|
|
|
205
259
|
state = "dependencies";
|
|
206
260
|
continue;
|
|
207
261
|
}
|
|
208
|
-
const items = inlineStringArray(valueText);
|
|
262
|
+
const items = inlineStringArray(valueText, budget);
|
|
209
263
|
if (items === null)
|
|
210
264
|
return fail(lineNo, "an unexpected dependencies value");
|
|
211
265
|
if (state === "package")
|
|
212
|
-
|
|
266
|
+
for (const item of items)
|
|
267
|
+
current.dependencies.push(item);
|
|
213
268
|
continue;
|
|
214
269
|
}
|
|
215
|
-
const value = lineString(valueText, false);
|
|
270
|
+
const value = lineString(valueText, false, budget);
|
|
216
271
|
if (value === null)
|
|
217
272
|
return fail(lineNo, `an unexpected value for ${key}`);
|
|
218
273
|
if (state === "package") {
|
|
@@ -225,16 +280,24 @@ export function parseCargoLock(text) {
|
|
|
225
280
|
}
|
|
226
281
|
}
|
|
227
282
|
if (state === "dependencies")
|
|
228
|
-
return fail(
|
|
229
|
-
const failed = finish(
|
|
283
|
+
return fail(lineNo, "an unterminated dependencies array");
|
|
284
|
+
const failed = finish(lineNo);
|
|
230
285
|
if (failed)
|
|
231
286
|
return failed;
|
|
287
|
+
// Count exact identities once; counts distinguish a separate duplicate from self-only replacement.
|
|
288
|
+
const packageCounts = new Map();
|
|
289
|
+
for (const pkg of packages) {
|
|
290
|
+
const id = JSON.stringify([pkg.name, pkg.version, pkg.source]);
|
|
291
|
+
packageCounts.set(id, (packageCounts.get(id) ?? 0) + 1);
|
|
292
|
+
}
|
|
232
293
|
// 置換先が解釈でき、別のパッケージとして記載されていることを確かめる(置換元は使われないためスキャンしない)
|
|
233
294
|
for (const pkg of packages) {
|
|
234
295
|
if (pkg.replace === undefined)
|
|
235
296
|
continue;
|
|
236
297
|
const target = parsePackageId(pkg.replace);
|
|
237
|
-
const found = target !== null &&
|
|
298
|
+
const found = target !== null &&
|
|
299
|
+
(packageCounts.get(JSON.stringify([target.name, target.version, target.source])) ?? 0) >
|
|
300
|
+
(pkg.name === target.name && pkg.version === target.version && pkg.source === target.source ? 1 : 0);
|
|
238
301
|
if (!found) {
|
|
239
302
|
return {
|
|
240
303
|
ok: false,
|
|
@@ -484,50 +547,142 @@ function normalizeWorkspacePath(value) {
|
|
|
484
547
|
function isPathPrefix(prefix, target) {
|
|
485
548
|
return prefix === "" || target === prefix || target.startsWith(`${prefix}/`);
|
|
486
549
|
}
|
|
487
|
-
/**
|
|
488
|
-
|
|
550
|
+
/** 1回の所属の判定(1つのメンバー)で使えるglobの照合の計算量(DPの表のセル数) */
|
|
551
|
+
const MAX_WORKSPACE_GLOB_WORK = 1_000_000;
|
|
552
|
+
/**
|
|
553
|
+
* 1回の検出全体で使えるglobの照合の計算量。メンバーのCargo.tomlごとに上限を戻すと、同名のcrateを名乗る
|
|
554
|
+
* Cargo.tomlを大量に置いたリポジトリで計算量が積み上がる(2,000件で4.6秒の同期処理を確認、2026-10-22)ため、
|
|
555
|
+
* 検出全体でも上限を設ける。使い切った後の判定はnull(未確認)
|
|
556
|
+
*/
|
|
557
|
+
export const MAX_WORKSPACE_GLOB_TOTAL_WORK = 10_000_000;
|
|
558
|
+
export function createGlobWorkBudget(total = MAX_WORKSPACE_GLOB_TOTAL_WORK) {
|
|
559
|
+
return { remaining: total };
|
|
560
|
+
}
|
|
561
|
+
/** `*`・`?`だけのglobの一致。それ以外のglob、計算量の上限超過はnull(判定できない) */
|
|
562
|
+
function globMatches(pattern, target, budget) {
|
|
563
|
+
const preparationWork = pattern.length + target.length + 1;
|
|
564
|
+
if (preparationWork > budget.remaining) {
|
|
565
|
+
// Do not leave an unchargeable remainder that permits unlimited pattern visits.
|
|
566
|
+
budget.remaining = 0;
|
|
567
|
+
return null;
|
|
568
|
+
}
|
|
569
|
+
budget.remaining -= preparationWork;
|
|
489
570
|
const patternParts = pattern.split("/");
|
|
490
571
|
const targetParts = target === "" ? [] : target.split("/");
|
|
491
572
|
if (patternParts.some((part) => part.includes("**") || /[[\]{}]/.test(part)))
|
|
492
573
|
return null;
|
|
493
574
|
if (patternParts.length !== targetParts.length)
|
|
494
575
|
return false;
|
|
495
|
-
|
|
496
|
-
const
|
|
497
|
-
|
|
498
|
-
|
|
576
|
+
for (let i = 0; i < patternParts.length; i++) {
|
|
577
|
+
const part = patternParts[i];
|
|
578
|
+
const value = targetParts[i];
|
|
579
|
+
if (!part.includes("*") && !part.includes("?")) {
|
|
580
|
+
if (part !== value)
|
|
581
|
+
return false;
|
|
582
|
+
continue;
|
|
583
|
+
}
|
|
584
|
+
// Reserve all DP cells before allocating; the budget is shared by every member pattern.
|
|
585
|
+
const work = (part.length + 1) * (value.length + 1);
|
|
586
|
+
if (work > budget.remaining)
|
|
587
|
+
return null;
|
|
588
|
+
budget.remaining -= work;
|
|
589
|
+
let previous = new Uint8Array(value.length + 1);
|
|
590
|
+
let current = new Uint8Array(value.length + 1);
|
|
591
|
+
previous[0] = 1;
|
|
592
|
+
for (let p = 0; p < part.length; p++) {
|
|
593
|
+
const token = part[p];
|
|
594
|
+
current[0] = token === "*" ? previous[0] : 0;
|
|
595
|
+
for (let t = 1; t <= value.length; t++) {
|
|
596
|
+
current[t] = token === "*"
|
|
597
|
+
? previous[t] | current[t - 1]
|
|
598
|
+
: (token === "?" || token === value[t - 1]) ? previous[t - 1] : 0;
|
|
599
|
+
}
|
|
600
|
+
[previous, current] = [current, previous];
|
|
601
|
+
}
|
|
602
|
+
if (previous[value.length] === 0)
|
|
603
|
+
return false;
|
|
604
|
+
}
|
|
605
|
+
return true;
|
|
606
|
+
}
|
|
607
|
+
const compiledWorkspaces = new WeakMap();
|
|
608
|
+
function compileWorkspace(workspace) {
|
|
609
|
+
const cached = compiledWorkspaces.get(workspace);
|
|
610
|
+
if (cached !== undefined)
|
|
611
|
+
return cached;
|
|
612
|
+
const compiled = {
|
|
613
|
+
literalMembers: new Set(), globMembers: [], explicitMembers: new Set(), invalidMember: false, excludes: new Set(), invalidExclude: false,
|
|
614
|
+
};
|
|
615
|
+
for (const member of workspace.members) {
|
|
616
|
+
const pattern = normalizeWorkspacePath(member);
|
|
617
|
+
if (pattern === null) {
|
|
618
|
+
compiled.invalidMember = true;
|
|
619
|
+
continue;
|
|
620
|
+
}
|
|
621
|
+
compiled.explicitMembers.add(pattern);
|
|
622
|
+
if (/[*?[\]{}]/.test(pattern))
|
|
623
|
+
compiled.globMembers.push(pattern);
|
|
624
|
+
else
|
|
625
|
+
compiled.literalMembers.add(pattern);
|
|
626
|
+
}
|
|
627
|
+
for (const ex of workspace.exclude) {
|
|
628
|
+
const prefix = normalizeWorkspacePath(ex);
|
|
629
|
+
if (prefix === null)
|
|
630
|
+
compiled.invalidExclude = true;
|
|
631
|
+
else
|
|
632
|
+
compiled.excludes.add(prefix);
|
|
633
|
+
}
|
|
634
|
+
compiledWorkspaces.set(workspace, compiled);
|
|
635
|
+
return compiled;
|
|
636
|
+
}
|
|
637
|
+
/** targetとその祖先(ルート""を含む)。前方一致の判定を集合の参照にするため */
|
|
638
|
+
function selfAndAncestors(target) {
|
|
639
|
+
const parts = target === "" ? [] : target.split("/");
|
|
640
|
+
const result = [""];
|
|
641
|
+
for (let i = 1; i <= parts.length; i++)
|
|
642
|
+
result.push(parts.slice(0, i).join("/"));
|
|
643
|
+
return result;
|
|
499
644
|
}
|
|
500
645
|
/**
|
|
501
646
|
* memberDir(workspaceのルートからの相対パス)が`members`に含まれ、`exclude`で除かれていないか。
|
|
502
647
|
* Cargoの規則(2026-10-11 cargo 1.98で確認): `members`はglobで展開し、`exclude`はパスの前方一致で除く。
|
|
503
648
|
* ただし`members`に(globでなく)文字どおり書かれたパスの配下は`exclude`より優先する。
|
|
504
|
-
* true=所属 / false=所属しない / null=判定できない(解釈できないパス・glob)
|
|
649
|
+
* true=所属 / false=所属しない / null=判定できない(解釈できないパス・glob・計算量の上限超過)。
|
|
650
|
+
* totalを渡すと、globの照合の計算量を呼び出しをまたいで共有する(検出全体の上限)
|
|
505
651
|
*/
|
|
506
|
-
export function workspaceIncludes(workspace, memberDir) {
|
|
652
|
+
export function workspaceIncludes(workspace, memberDir, total) {
|
|
507
653
|
const target = normalizeWorkspacePath(memberDir);
|
|
508
654
|
if (target === null)
|
|
509
655
|
return null;
|
|
510
|
-
|
|
511
|
-
let matched =
|
|
512
|
-
|
|
513
|
-
|
|
514
|
-
|
|
515
|
-
|
|
516
|
-
|
|
517
|
-
|
|
518
|
-
|
|
656
|
+
const compiled = compileWorkspace(workspace);
|
|
657
|
+
let matched = compiled.literalMembers.has(target);
|
|
658
|
+
let undecided = compiled.invalidMember;
|
|
659
|
+
if (!matched && compiled.globMembers.length > 0) {
|
|
660
|
+
// 1回の判定の上限と、検出全体の残りの小さいほう。使った分は検出全体の残りからも引く
|
|
661
|
+
const budget = { remaining: Math.min(MAX_WORKSPACE_GLOB_WORK, total?.remaining ?? MAX_WORKSPACE_GLOB_WORK) };
|
|
662
|
+
const start = budget.remaining;
|
|
663
|
+
for (const pattern of compiled.globMembers) {
|
|
664
|
+
// 上限を使い切ったら残りのパターンは見ない(パターン数×メンバー数の空回りを避ける)
|
|
665
|
+
if (budget.remaining <= 0) {
|
|
666
|
+
undecided = true;
|
|
667
|
+
break;
|
|
668
|
+
}
|
|
669
|
+
const result = globMatches(pattern, target, budget);
|
|
670
|
+
if (result === null)
|
|
671
|
+
undecided = true;
|
|
672
|
+
else if (result) {
|
|
673
|
+
matched = true;
|
|
674
|
+
break;
|
|
675
|
+
}
|
|
676
|
+
}
|
|
677
|
+
if (total !== undefined)
|
|
678
|
+
total.remaining -= start - budget.remaining;
|
|
519
679
|
}
|
|
520
680
|
if (!matched)
|
|
521
681
|
return undecided ? null : false;
|
|
522
|
-
|
|
523
|
-
const literal = normalizeWorkspacePath(member);
|
|
524
|
-
return literal !== null && isPathPrefix(literal, target);
|
|
525
|
-
});
|
|
526
|
-
const excluded = workspace.exclude.some((ex) => {
|
|
527
|
-
const prefix = normalizeWorkspacePath(ex);
|
|
528
|
-
return prefix !== null && isPathPrefix(prefix, target);
|
|
529
|
-
});
|
|
530
|
-
if (workspace.exclude.some((ex) => normalizeWorkspacePath(ex) === null))
|
|
682
|
+
if (compiled.invalidExclude)
|
|
531
683
|
return null;
|
|
684
|
+
const ancestors = selfAndAncestors(target);
|
|
685
|
+
const explicit = ancestors.some((prefix) => compiled.explicitMembers.has(prefix));
|
|
686
|
+
const excluded = ancestors.some((prefix) => compiled.excludes.has(prefix));
|
|
532
687
|
return !(excluded && !explicit);
|
|
533
688
|
}
|
|
@@ -269,7 +269,7 @@ function stripV(version) {
|
|
|
269
269
|
*/
|
|
270
270
|
export function goModRelations(text) {
|
|
271
271
|
const requires = new Map();
|
|
272
|
-
const replaces =
|
|
272
|
+
const replaces = new Map();
|
|
273
273
|
let block = null;
|
|
274
274
|
for (const raw of text.split(/\r?\n/)) {
|
|
275
275
|
const { body, indirect } = splitGoComment(raw);
|
|
@@ -303,12 +303,20 @@ export function goModRelations(text) {
|
|
|
303
303
|
else if (directive === "replace") {
|
|
304
304
|
const arrow = fields.indexOf("=>");
|
|
305
305
|
if ((arrow === 1 || arrow === 2) && fields[arrow + 1] !== undefined) {
|
|
306
|
-
|
|
306
|
+
const replace = {
|
|
307
307
|
from: fields[0],
|
|
308
308
|
fromVersion: arrow === 2 ? stripV(fields[1]) : null,
|
|
309
309
|
to: fields[arrow + 1],
|
|
310
310
|
toVersion: fields[arrow + 2] !== undefined ? stripV(fields[arrow + 2]) : null,
|
|
311
|
-
}
|
|
311
|
+
};
|
|
312
|
+
let versions = replaces.get(replace.from);
|
|
313
|
+
if (versions === undefined) {
|
|
314
|
+
versions = new Map();
|
|
315
|
+
replaces.set(replace.from, versions);
|
|
316
|
+
}
|
|
317
|
+
// Match the former find calls: retain the first declaration for each version.
|
|
318
|
+
if (!versions.has(replace.fromVersion))
|
|
319
|
+
versions.set(replace.fromVersion, replace);
|
|
312
320
|
}
|
|
313
321
|
}
|
|
314
322
|
}
|
|
@@ -323,8 +331,8 @@ export function goModRelations(text) {
|
|
|
323
331
|
};
|
|
324
332
|
for (const [modulePath, { relation, version }] of requires) {
|
|
325
333
|
setRelation(modulePath, relation);
|
|
326
|
-
const
|
|
327
|
-
|
|
334
|
+
const versions = replaces.get(modulePath);
|
|
335
|
+
const replace = versions?.get(version) ?? versions?.get(null);
|
|
328
336
|
if (replace === undefined)
|
|
329
337
|
continue;
|
|
330
338
|
if (replace.toVersion === null) {
|
|
@@ -344,7 +352,7 @@ export function goModRelations(text) {
|
|
|
344
352
|
}
|
|
345
353
|
/**
|
|
346
354
|
* pom.xmlの依存の関係を、osv-scannerの`source.type`で判定する(pom.xmlは自前で解析しない)。
|
|
347
|
-
* osv-scanner 2.4.0は同じpom.xmlを、宣言された依存(親POM・プロファイル・プロパティ・依存管理をosv-scannerが解釈したもの)を
|
|
355
|
+
* osv-scanner 2.4.0・2.7.0は同じpom.xmlを、宣言された依存(親POM・プロファイル・プロパティ・依存管理をosv-scannerが解釈したもの)を
|
|
348
356
|
* `type: "lockfile"`、deps.devで解決された推移的依存を`type: "unknown"`の結果に分けて報告する(実データで確認)。
|
|
349
357
|
* 文書化された仕様ではないため、想定外のtypeはunknownにする(osv-scannerのピン留め更新時に再確認する)。
|
|
350
358
|
*/
|
|
@@ -13,8 +13,9 @@ import path from "node:path";
|
|
|
13
13
|
import { ScanToolError } from "../errors.js";
|
|
14
14
|
import { assertInsideAllowedRoot, resolveExistingPath, walkProjectFiles, } from "./projectWalk.js";
|
|
15
15
|
import { capitalizeReason, readRegularFile } from "./safeRead.js";
|
|
16
|
+
import { createNpmLockWorkBudget, MAX_NPM_LOCK_BYTES, MAX_NPM_LOCK_TOTAL_BYTES, parseNpmLock } from "./npmLockCopy.js";
|
|
16
17
|
import { analyzeRequirementsFile, createRequirementsReadContext, } from "./requirementsFile.js";
|
|
17
|
-
import { cargoTomlPackageName, cargoWorkspace, workspaceIncludes, classifyCargoPackages, parseCargoLock, UNSCANNABLE_CRATE_REASONS, } from "./cargoLock.js";
|
|
18
|
+
import { cargoTomlPackageName, cargoWorkspace, workspaceIncludes, createGlobWorkBudget, classifyCargoPackages, createCargoLockParseBudget, parseCargoLock, UNSCANNABLE_CRATE_REASONS, } from "./cargoLock.js";
|
|
18
19
|
/** ファイル名→(エコシステム, 解析形式)。npm-shrinkwrap.jsonはpackage-lock.json形式で解析する */
|
|
19
20
|
const LOCKFILES = new Map([
|
|
20
21
|
["pom.xml", { ecosystem: "Maven", format: "pom.xml" }],
|
|
@@ -79,8 +80,7 @@ function satisfiesGroup(manifest, group) {
|
|
|
79
80
|
function isSameOrDescendant(dir, ancestor) {
|
|
80
81
|
return ancestor === "." || dir === ancestor || dir.startsWith(`${ancestor}${path.sep}`);
|
|
81
82
|
}
|
|
82
|
-
/**
|
|
83
|
-
const MAX_LOCKFILE_BYTES = 64 * 1024 * 1024;
|
|
83
|
+
/** Cargo.lockの読み込み合計の上限(npm JSONにはnpmLockCopy.tsの上限を使う) */
|
|
84
84
|
const MAX_LOCKFILE_TOTAL_BYTES = 256 * 1024 * 1024;
|
|
85
85
|
/**
|
|
86
86
|
* workspaceの収録確認用に、package-lock.jsonの`packages`のキー一覧を1回だけ読む。
|
|
@@ -89,8 +89,9 @@ const MAX_LOCKFILE_TOTAL_BYTES = 256 * 1024 * 1024;
|
|
|
89
89
|
*/
|
|
90
90
|
export class LockfileKeyCache {
|
|
91
91
|
keys = new Map();
|
|
92
|
+
workBudget = createNpmLockWorkBudget();
|
|
92
93
|
remaining;
|
|
93
|
-
constructor(maxTotalBytes =
|
|
94
|
+
constructor(maxTotalBytes = MAX_NPM_LOCK_TOTAL_BYTES) {
|
|
94
95
|
this.remaining = maxTotalBytes;
|
|
95
96
|
}
|
|
96
97
|
/** 収録済みのキー。null=確認できない(v1形式・読めない・大きすぎる・上限超過) */
|
|
@@ -99,11 +100,11 @@ export class LockfileKeyCache {
|
|
|
99
100
|
return this.keys.get(lockAbs);
|
|
100
101
|
let keys = null;
|
|
101
102
|
if (this.remaining > 0) {
|
|
102
|
-
const read = await readRegularFile(lockAbs, { maxBytes: Math.min(
|
|
103
|
+
const read = await readRegularFile(lockAbs, { maxBytes: Math.min(MAX_NPM_LOCK_BYTES, this.remaining), boundary: projectDir });
|
|
103
104
|
if (read.ok) {
|
|
104
105
|
this.remaining -= read.bytes.length;
|
|
105
106
|
try {
|
|
106
|
-
const packages =
|
|
107
|
+
const packages = parseNpmLock(read.bytes.toString("utf8"), this.workBudget).packages;
|
|
107
108
|
if (typeof packages === "object" && packages !== null)
|
|
108
109
|
keys = new Set(Object.keys(packages));
|
|
109
110
|
}
|
|
@@ -125,6 +126,7 @@ const MAX_CARGO_TOML_BYTES = 1024 * 1024;
|
|
|
125
126
|
*/
|
|
126
127
|
export class CargoLockCache {
|
|
127
128
|
results = new Map();
|
|
129
|
+
parseBudget = createCargoLockParseBudget();
|
|
128
130
|
remaining;
|
|
129
131
|
tomls = new Map();
|
|
130
132
|
remainingToml = MAX_LOCKFILE_TOTAL_BYTES;
|
|
@@ -157,7 +159,7 @@ export class CargoLockCache {
|
|
|
157
159
|
}
|
|
158
160
|
else {
|
|
159
161
|
this.remaining -= read.bytes.length;
|
|
160
|
-
const parsed = parseCargoLock(read.bytes.toString("utf8"));
|
|
162
|
+
const parsed = parseCargoLock(read.bytes.toString("utf8"), this.parseBudget);
|
|
161
163
|
result = parsed.ok ? classifyCargoPackages(parsed.packages) : { reason: parsed.reason };
|
|
162
164
|
}
|
|
163
165
|
}
|
|
@@ -165,6 +167,8 @@ export class CargoLockCache {
|
|
|
165
167
|
return result;
|
|
166
168
|
}
|
|
167
169
|
workspaces = new Map();
|
|
170
|
+
/** workspaceの所属の判定のglobの照合の計算量(1回の検出全体で共有する) */
|
|
171
|
+
globBudget = createGlobWorkBudget();
|
|
168
172
|
/** Cargo.lockと同じディレクトリのCargo.tomlの`[workspace]`(1回だけ読む)。無い・読めない場合はnull */
|
|
169
173
|
async workspace(lockAbs, projectDir) {
|
|
170
174
|
if (this.workspaces.has(lockAbs))
|
|
@@ -198,7 +202,7 @@ async function cargoLockfileRecords(cache, projectDir, lockfile, memberToml) {
|
|
|
198
202
|
if (workspace === null)
|
|
199
203
|
return null;
|
|
200
204
|
const memberDir = path.relative(path.dirname(lockAbs), path.join(projectDir, path.dirname(memberToml))).split(path.sep).join("/");
|
|
201
|
-
return workspaceIncludes(workspace, memberDir) === true ? true : null;
|
|
205
|
+
return workspaceIncludes(workspace, memberDir, cache.globBudget) === true ? true : null;
|
|
202
206
|
}
|
|
203
207
|
/**
|
|
204
208
|
* package-lock.json(v2以降)の`packages`に、memberDirのエントリがあるか。
|
|
@@ -275,6 +279,7 @@ async function finalize(projectDir, manifests, markers, allowedRootReal) {
|
|
|
275
279
|
skippedFiles: [],
|
|
276
280
|
};
|
|
277
281
|
const readContext = createRequirementsReadContext();
|
|
282
|
+
// A null value permanently marks a crate name with multiple declaration paths.
|
|
278
283
|
const cargoManifests = new Map();
|
|
279
284
|
const cargoMarkers = new Set([
|
|
280
285
|
...markers.filter((m) => m.path.endsWith("Cargo.toml")).map((m) => m.path),
|
|
@@ -284,7 +289,7 @@ async function finalize(projectDir, manifests, markers, allowedRootReal) {
|
|
|
284
289
|
const text = await cargoCache.toml(path.join(projectDir, marker), projectDir);
|
|
285
290
|
const name = text === null ? null : cargoTomlPackageName(text);
|
|
286
291
|
if (name !== null)
|
|
287
|
-
cargoManifests.set(name,
|
|
292
|
+
cargoManifests.set(name, cargoManifests.has(name) ? null : marker);
|
|
288
293
|
}
|
|
289
294
|
for (const manifest of manifests) {
|
|
290
295
|
const absolute = path.join(projectDir, manifest.path);
|
|
@@ -306,9 +311,12 @@ async function finalize(projectDir, manifests, markers, allowedRootReal) {
|
|
|
306
311
|
continue;
|
|
307
312
|
}
|
|
308
313
|
const declaringManifests = new Map();
|
|
309
|
-
|
|
310
|
-
|
|
311
|
-
|
|
314
|
+
// declared_inに使うのは、このCargo.lockのローカルのcrate(依存元)の名前だけ。プロジェクトの全crate名を
|
|
315
|
+
// Cargo.lockごとにたどると、Cargo.lockの数×crate名の数の処理・メモリになる(900件×2万件で約24秒)
|
|
316
|
+
for (const name of lock.localCrates) {
|
|
317
|
+
const marker = cargoManifests.get(name);
|
|
318
|
+
if (typeof marker === "string")
|
|
319
|
+
declaringManifests.set(name, path.relative(path.dirname(manifest.path), marker));
|
|
312
320
|
}
|
|
313
321
|
result.cargoCopies.push({ path: manifest.path, packages: lock.cratesIo, allPackages: lock.allPackages, declaringManifests });
|
|
314
322
|
result.localCrateCount += lock.localCrates.length;
|
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
/**
|
|
2
|
-
* osv-scannerに`--lockfile <形式>:<絶対パス>`で個別に渡せる解析形式(2.4.0で実機確認)。
|
|
2
|
+
* osv-scannerに`--lockfile <形式>:<絶対パス>`で個別に渡せる解析形式(2.4.0・2.7.0で実機確認)。
|
|
3
3
|
* ここに無い形式はosv-scannerに渡さない。npm-shrinkwrap.jsonは"package-lock.json"形式で解析する。
|
|
4
4
|
* Cargo.lockは元のファイルを渡さず、crates.io由来のパッケージだけを書いたコピーを渡す(cargoLock.ts)。
|
|
5
5
|
*/
|
|
@@ -0,0 +1,183 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* package-lock.json(npm-shrinkwrap.jsonを含む)のスナップショットのコピーで、`resolved`の無いエントリに
|
|
3
|
+
* 公開レジストリの形のURLを補う(docs/DESIGN_TODO.md「osv-scannerのピン留め更新」)。
|
|
4
|
+
*
|
|
5
|
+
* osv-scanner 2.7.0(osv-scalibrの`DeterminePackageSource`)は、`resolved`が無いエントリをローカルのパッケージとみなして
|
|
6
|
+
* 照会しない(2026-10-15 実機確認)。バンドルされた依存(`inBundle` / v1の`bundled`)や、npmの
|
|
7
|
+
* `omit-lockfile-registry-resolved=true`で生成したlockfileのレジストリのパッケージが黙って照会されなくなる(偽陰性)。
|
|
8
|
+
* 2.4.0は照会していたため、補うことで同じ範囲を照会させる(送る名前は2.4.0と同じ)。
|
|
9
|
+
*
|
|
10
|
+
* 対象外(ローカルのパッケージ): `link: true`、キーが`node_modules/`を含まないエントリ(ローカルのディレクトリ・workspace)、
|
|
11
|
+
* 版が`file:`・`git+...`・`user/repo`等の参照のもの(v1の別名`npm:<名前>@<版>`は実際の名前と版で補う)。`resolved`があるもの(`file:`のtarballを含む)は変えない。
|
|
12
|
+
*/
|
|
13
|
+
import { ScanToolError } from "../errors.js";
|
|
14
|
+
export const MAX_NPM_LOCK_BYTES = 16 * 1024 * 1024;
|
|
15
|
+
export const MAX_NPM_LOCK_TOTAL_BYTES = 64 * 1024 * 1024;
|
|
16
|
+
const MAX_JSON_DEPTH = 128;
|
|
17
|
+
const MAX_JSON_STRUCTURE = 1_000_000;
|
|
18
|
+
const REGISTRY = "https://registry.npmjs.org";
|
|
19
|
+
export function createNpmLockWorkBudget() {
|
|
20
|
+
return { remaining: 2_000_000 };
|
|
21
|
+
}
|
|
22
|
+
function limitExceeded(limit) {
|
|
23
|
+
throw new ScanToolError("scan_input_too_large", `The npm lockfile exceeds the ${limit}. Narrow the target or use a smaller lockfile`);
|
|
24
|
+
}
|
|
25
|
+
/** Bound allocation before JSON.parse; JSON syntax itself is still validated by the native parser. */
|
|
26
|
+
export function parseNpmLock(text, budget = createNpmLockWorkBudget()) {
|
|
27
|
+
if (text.length > MAX_NPM_LOCK_BYTES || Buffer.byteLength(text, "utf8") > MAX_NPM_LOCK_BYTES) {
|
|
28
|
+
limitExceeded(`input byte limit (${MAX_NPM_LOCK_BYTES} bytes)`);
|
|
29
|
+
}
|
|
30
|
+
let quoted = false;
|
|
31
|
+
let escaped = false;
|
|
32
|
+
let depth = 0;
|
|
33
|
+
let structure = 0;
|
|
34
|
+
for (let i = 0; i < text.length; i++) {
|
|
35
|
+
const char = text[i];
|
|
36
|
+
if (quoted) {
|
|
37
|
+
if (escaped)
|
|
38
|
+
escaped = false;
|
|
39
|
+
else if (char === "\\")
|
|
40
|
+
escaped = true;
|
|
41
|
+
else if (char === '"')
|
|
42
|
+
quoted = false;
|
|
43
|
+
continue;
|
|
44
|
+
}
|
|
45
|
+
if (char === '"')
|
|
46
|
+
quoted = true;
|
|
47
|
+
else if (char === "{" || char === "[") {
|
|
48
|
+
if (++depth > MAX_JSON_DEPTH)
|
|
49
|
+
limitExceeded(`JSON depth limit (${MAX_JSON_DEPTH})`);
|
|
50
|
+
structure++;
|
|
51
|
+
}
|
|
52
|
+
else if (char === "}" || char === "]")
|
|
53
|
+
depth--;
|
|
54
|
+
else if (char === "," || char === ":")
|
|
55
|
+
structure++;
|
|
56
|
+
if (structure > MAX_JSON_STRUCTURE)
|
|
57
|
+
limitExceeded(`JSON structure limit (${MAX_JSON_STRUCTURE})`);
|
|
58
|
+
if (structure > budget.remaining)
|
|
59
|
+
limitExceeded("scan-wide JSON structure limit");
|
|
60
|
+
}
|
|
61
|
+
budget.remaining -= structure;
|
|
62
|
+
return JSON.parse(text);
|
|
63
|
+
}
|
|
64
|
+
/** Measure compact output before materializing it, including JSON escaping and number formatting. */
|
|
65
|
+
function stringifyNpmLock(lock, maxBytes) {
|
|
66
|
+
let remaining = maxBytes;
|
|
67
|
+
const reserve = (bytes) => {
|
|
68
|
+
remaining -= bytes;
|
|
69
|
+
if (remaining < 0)
|
|
70
|
+
limitExceeded(`output byte limit (${maxBytes} bytes)`);
|
|
71
|
+
};
|
|
72
|
+
const measure = (value) => {
|
|
73
|
+
if (typeof value === "object" && value !== null) {
|
|
74
|
+
reserve(2);
|
|
75
|
+
const array = Array.isArray(value);
|
|
76
|
+
const record = value;
|
|
77
|
+
let first = true;
|
|
78
|
+
for (const key of Object.keys(value)) {
|
|
79
|
+
if (!first)
|
|
80
|
+
reserve(1);
|
|
81
|
+
first = false;
|
|
82
|
+
if (!array) {
|
|
83
|
+
reserve(key.length);
|
|
84
|
+
reserve(Buffer.byteLength(JSON.stringify(key), "utf8") - key.length + 1);
|
|
85
|
+
}
|
|
86
|
+
measure(record[key]);
|
|
87
|
+
}
|
|
88
|
+
}
|
|
89
|
+
else {
|
|
90
|
+
if (typeof value === "string" && value.length > remaining)
|
|
91
|
+
limitExceeded(`output byte limit (${maxBytes} bytes)`);
|
|
92
|
+
reserve(Buffer.byteLength(JSON.stringify(value), "utf8"));
|
|
93
|
+
}
|
|
94
|
+
};
|
|
95
|
+
measure(lock);
|
|
96
|
+
return JSON.stringify(lock);
|
|
97
|
+
}
|
|
98
|
+
/** 照会させてよい版(参照でない、ふつうの版の文字) */
|
|
99
|
+
const PLAIN_VERSION = /^[0-9A-Za-z.+-]{1,256}$/;
|
|
100
|
+
const IN_NODE_MODULES = /(?:^|\/)node_modules\//;
|
|
101
|
+
/** v1の別名の依存の版(`npm:<実際の名前>@<版>`。スコープ付きの名前を含む) */
|
|
102
|
+
const NPM_ALIAS = /^npm:((?:@[^/@\s]+\/)?[^/@\s]+)@(.+)$/;
|
|
103
|
+
function isRecord(value) {
|
|
104
|
+
return typeof value === "object" && value !== null && !Array.isArray(value);
|
|
105
|
+
}
|
|
106
|
+
function lacksResolved(entry) {
|
|
107
|
+
return entry.resolved === undefined || entry.resolved === "";
|
|
108
|
+
}
|
|
109
|
+
function registryUrl(name, version) {
|
|
110
|
+
const base = name.split("/").pop();
|
|
111
|
+
return `${REGISTRY}/${name}/-/${base}-${version}.tgz`;
|
|
112
|
+
}
|
|
113
|
+
/** v2以降の`packages`: キーの最後の`node_modules/`より後ろがパッケージ名(別名では`name`が実際の名前) */
|
|
114
|
+
function fillPackages(packages) {
|
|
115
|
+
let filled = 0;
|
|
116
|
+
for (const [key, entry] of Object.entries(packages)) {
|
|
117
|
+
if (!isRecord(entry) || !IN_NODE_MODULES.test(key) || entry.link === true || !lacksResolved(entry))
|
|
118
|
+
continue;
|
|
119
|
+
const version = entry.version;
|
|
120
|
+
if (typeof version !== "string" || !PLAIN_VERSION.test(version))
|
|
121
|
+
continue;
|
|
122
|
+
const name = typeof entry.name === "string" ? entry.name : key.slice(key.lastIndexOf("node_modules/") + "node_modules/".length);
|
|
123
|
+
if (name === "")
|
|
124
|
+
continue;
|
|
125
|
+
entry.resolved = registryUrl(name, version);
|
|
126
|
+
filled++;
|
|
127
|
+
}
|
|
128
|
+
return filled;
|
|
129
|
+
}
|
|
130
|
+
/**
|
|
131
|
+
* v1の依存の実際の名前と版。別名(`"version": "npm:lodash@4.17.20"`)は実際の名前と版を取り出す
|
|
132
|
+
* (osv-scannerもこの表記を実際の名前と版として照会する。2026-10-15 実機確認)。参照(`file:`・git等)はnull
|
|
133
|
+
*/
|
|
134
|
+
function registryPackage(name, version) {
|
|
135
|
+
if (PLAIN_VERSION.test(version))
|
|
136
|
+
return { name, version };
|
|
137
|
+
const alias = NPM_ALIAS.exec(version);
|
|
138
|
+
if (alias !== null && PLAIN_VERSION.test(alias[2]))
|
|
139
|
+
return { name: alias[1], version: alias[2] };
|
|
140
|
+
return null;
|
|
141
|
+
}
|
|
142
|
+
/** v1の`dependencies`(入れ子) */
|
|
143
|
+
function fillDependencies(dependencies, depth) {
|
|
144
|
+
if (depth > 256)
|
|
145
|
+
return 0;
|
|
146
|
+
let filled = 0;
|
|
147
|
+
for (const [name, entry] of Object.entries(dependencies)) {
|
|
148
|
+
if (!isRecord(entry))
|
|
149
|
+
continue;
|
|
150
|
+
const version = entry.version;
|
|
151
|
+
const target = typeof version === "string" ? registryPackage(name, version) : null;
|
|
152
|
+
if (lacksResolved(entry) && target !== null) {
|
|
153
|
+
entry.resolved = registryUrl(target.name, target.version);
|
|
154
|
+
filled++;
|
|
155
|
+
}
|
|
156
|
+
if (isRecord(entry.dependencies))
|
|
157
|
+
filled += fillDependencies(entry.dependencies, depth + 1);
|
|
158
|
+
}
|
|
159
|
+
return filled;
|
|
160
|
+
}
|
|
161
|
+
/**
|
|
162
|
+
* `resolved`を補ったlockfileの内容。補うエントリが無ければ`changed: false`(コピーを書き換えない)。
|
|
163
|
+
* JSONとして解析できなければnull(呼び出し側はコピーをそのまま渡す)。資源上限超過は型付きエラーで停止する。
|
|
164
|
+
*/
|
|
165
|
+
export function fillMissingResolved(text, maxOutputBytes = MAX_NPM_LOCK_BYTES, budget = createNpmLockWorkBudget()) {
|
|
166
|
+
let lock;
|
|
167
|
+
try {
|
|
168
|
+
lock = parseNpmLock(text, budget);
|
|
169
|
+
}
|
|
170
|
+
catch (error) {
|
|
171
|
+
if (error instanceof ScanToolError)
|
|
172
|
+
throw error;
|
|
173
|
+
return null;
|
|
174
|
+
}
|
|
175
|
+
if (!isRecord(lock))
|
|
176
|
+
return null;
|
|
177
|
+
let filled = 0;
|
|
178
|
+
if (isRecord(lock.packages))
|
|
179
|
+
filled += fillPackages(lock.packages);
|
|
180
|
+
if (isRecord(lock.dependencies))
|
|
181
|
+
filled += fillDependencies(lock.dependencies, 0);
|
|
182
|
+
return filled === 0 ? { changed: false, content: text, filled } : { changed: true, content: stringifyNpmLock(lock, maxOutputBytes), filled };
|
|
183
|
+
}
|
package/dist/utils/pomParent.js
CHANGED
|
@@ -1,14 +1,16 @@
|
|
|
1
1
|
/**
|
|
2
2
|
* pom.xmlの親POM(`<parent><relativePath>`)の参照の解釈。連鎖のコピーと検証はscanSnapshot.tsが行う。
|
|
3
3
|
*
|
|
4
|
-
* osv-scanner v2.4.0の実機確認(2026-10-07):
|
|
4
|
+
* osv-scanner v2.4.0の実機確認(2026-10-07。範囲外の親POMを読む挙動はv2.7.0でも同じ、2026-10-15):
|
|
5
5
|
* - `<relativePath>`の指す親POMを読み、その依存を結果に含める(`--no-resolve`でも同じ)。親の親もたどる
|
|
6
6
|
* - `<relativePath>`省略時はMavenの既定値`../pom.xml`を参照する。ディレクトリを指す場合はその中のpom.xml
|
|
7
|
-
* -
|
|
7
|
+
* - 親のGAVが一致しない場合は読まない
|
|
8
|
+
* - `<relativePath/>`・`<relativePath></relativePath>`・空白だけの値も、既定値`../pom.xml`を参照する(v2.4.0・v2.7.0で実機確認、2026-10-22)。
|
|
9
|
+
* Mavenでは空は「ローカルを参照しない」だが、osv-scannerは既定値と同じに扱う。以前は空を「参照しない」と解釈して親をコピーせず、
|
|
10
|
+
* スナップショットでは親から継承する依存が黙って欠け、許可ルートの検証も行われなかった
|
|
8
11
|
*
|
|
9
|
-
*
|
|
10
|
-
* osv-scanner
|
|
11
|
-
* 親から引き継ぐ依存を黙って欠落させないためのもの。
|
|
12
|
+
* コピーしたPOMにも相対参照は残るため、解釈できない参照は除外し、コピー後の連鎖も検証する。
|
|
13
|
+
* 警告だけで続行すると、osv-scannerがスナップショット外の元ファイルを読む可能性がある。
|
|
12
14
|
*/
|
|
13
15
|
const DEFAULT_RELATIVE_PATH = "../pom.xml";
|
|
14
16
|
const NAMED_ENTITIES = { lt: "<", gt: ">", amp: "&", apos: "'", quot: '"' };
|
|
@@ -31,7 +33,7 @@ function decodeEntities(text) {
|
|
|
31
33
|
return unknown ? undefined : decoded;
|
|
32
34
|
}
|
|
33
35
|
/**
|
|
34
|
-
* 親POMのrelativePath。null=親なし /
|
|
36
|
+
* 親POMのrelativePath。null=親なし / undefined=解釈できない(除外する)。空・空白だけの値は既定値`../pom.xml`(osv-scannerと同じ)
|
|
35
37
|
*
|
|
36
38
|
* osv-scannerのXMLデコーダー(Go)の挙動に合わせる(2026-10-07 実機確認):
|
|
37
39
|
* - 要素は名前空間・接頭辞に関係なくローカル名で照合する(`<m:parent>`も親として読む)。大文字小文字は区別する
|
|
@@ -135,7 +137,8 @@ export function parentRelativePath(source) {
|
|
|
135
137
|
// (JSとGoで扱いが異なる)で参照先がずれうるため、解釈できないものとして除外する
|
|
136
138
|
if (/[\p{Cc}\p{Cf}\u0085]|(?! )\p{Z}/u.test(trimmed))
|
|
137
139
|
return undefined;
|
|
138
|
-
|
|
140
|
+
// 空・空白だけの値は、osv-scannerでは省略時と同じく既定値を参照する
|
|
141
|
+
return trimmed === "" ? DEFAULT_RELATIVE_PATH : trimmed;
|
|
139
142
|
}
|
|
140
143
|
/** UTF-8として厳密に読む。不正なバイト列・NULを含む場合はnull(Goと同じ解釈を保証できない) */
|
|
141
144
|
export function decodePomBytes(bytes) {
|
|
@@ -7,7 +7,7 @@
|
|
|
7
7
|
* `名前==版` 等の単純な形に正規化して返し、呼び出し側がそれを専用コピーに書いてスキャンする。
|
|
8
8
|
* コピーには取り込み・オプションを一切含めないため、osv-scannerがたどれる参照は存在しない。
|
|
9
9
|
*
|
|
10
|
-
* osv-scanner v2.4.0の実機確認(2026-10-07):
|
|
10
|
+
* osv-scanner v2.4.0の実機確認(2026-10-07。v2.7.0でも同じ、2026-10-15):
|
|
11
11
|
* - `==` / `===` はその版をスキャンする。`>=X` / `~=X` は下限Xを使用中の版とみなす
|
|
12
12
|
* - `>` `<` `!=` `==1.*` 範囲の組み合わせ・版なしは空の版になり、結果から消える
|
|
13
13
|
* - `-r` はたどるが、`--requirement` / `-c` はたどらない
|
|
@@ -144,8 +144,17 @@ async function include(file, line, content, target, projectDir, depth, out, cont
|
|
|
144
144
|
}
|
|
145
145
|
/** 1回のスキャンで読むrequirements.txt(取り込み先を含む)の合計の上限 */
|
|
146
146
|
const DEFAULT_TOTAL_BYTES = 64 * 1024 * 1024;
|
|
147
|
-
|
|
148
|
-
|
|
147
|
+
const DEFAULT_EXPANDED_BYTES = 64 * 1024 * 1024;
|
|
148
|
+
const DEFAULT_LOGICAL_LINES = 100_000;
|
|
149
|
+
const EXPANSION_BYTE_LIMIT = "Not analyzed because the scan-wide requirements expansion byte limit was reached";
|
|
150
|
+
const LOGICAL_LINE_LIMIT = "Not analyzed because the scan-wide requirements logical-line limit was reached";
|
|
151
|
+
export function createRequirementsReadContext(maxTotalBytes = DEFAULT_TOTAL_BYTES, limits = {}) {
|
|
152
|
+
return {
|
|
153
|
+
remainingBytes: maxTotalBytes,
|
|
154
|
+
remainingExpandedBytes: limits.maxExpandedBytes ?? DEFAULT_EXPANDED_BYTES,
|
|
155
|
+
remainingLogicalLines: limits.maxLogicalLines ?? DEFAULT_LOGICAL_LINES,
|
|
156
|
+
texts: new Map(),
|
|
157
|
+
};
|
|
149
158
|
}
|
|
150
159
|
/** 安全に読む(FIFOで止まらず、差し替え・範囲外を検出する)。同じファイルは1回だけ読む */
|
|
151
160
|
async function readText(file, projectDir, context) {
|
|
@@ -173,13 +182,23 @@ async function readText(file, projectDir, context) {
|
|
|
173
182
|
context.texts.set(file, text);
|
|
174
183
|
return { ok: true, text };
|
|
175
184
|
}
|
|
176
|
-
/** ファイルを解析してout
|
|
185
|
+
/** ファイルを解析してoutに追加する。読めない・上限超過の場合は理由を返す */
|
|
177
186
|
async function analyze(file, projectDir, depth, out, context) {
|
|
187
|
+
if (context.remainingLogicalLines <= 0)
|
|
188
|
+
return LOGICAL_LINE_LIMIT;
|
|
178
189
|
out.visited.add(file);
|
|
179
190
|
const read = await readText(file, projectDir, context);
|
|
180
191
|
if (!read.ok)
|
|
181
192
|
return read.reason;
|
|
193
|
+
// Cached text costs expansion work and output space again for each requirements root.
|
|
194
|
+
const expandedBytes = Buffer.byteLength(read.text, "utf8");
|
|
195
|
+
if (expandedBytes > context.remainingExpandedBytes)
|
|
196
|
+
return EXPANSION_BYTE_LIMIT;
|
|
197
|
+
context.remainingExpandedBytes -= expandedBytes;
|
|
182
198
|
for (const { line, content } of logicalLines(read.text)) {
|
|
199
|
+
if (context.remainingLogicalLines <= 0)
|
|
200
|
+
return LOGICAL_LINE_LIMIT;
|
|
201
|
+
context.remainingLogicalLines--;
|
|
183
202
|
const requirementInclude = INCLUDE_REQUIREMENT.exec(content);
|
|
184
203
|
if (requirementInclude) {
|
|
185
204
|
await include(file, line, content, requirementInclude[1], projectDir, depth, out, context);
|
|
@@ -204,7 +223,7 @@ async function analyze(file, projectDir, depth, out, context) {
|
|
|
204
223
|
return null;
|
|
205
224
|
}
|
|
206
225
|
/**
|
|
207
|
-
* requirements.txt
|
|
226
|
+
* requirements.txtを解析する。元ファイルを読めない・大きすぎる・解析の共有上限を超えた場合は`ok: false`。
|
|
208
227
|
* @param file 検出済みの絶対パス
|
|
209
228
|
* @param projectDir 取り込みを展開する範囲(解決済みの絶対パス)
|
|
210
229
|
*/
|
|
@@ -27,6 +27,7 @@ export class ScanSnapshot {
|
|
|
27
27
|
maxTotalBytes;
|
|
28
28
|
untrack;
|
|
29
29
|
copied = new Map();
|
|
30
|
+
copiedPaths = new Set();
|
|
30
31
|
parentCache = new Map();
|
|
31
32
|
/** 再現した配置のルートごとのディレクトリ → 元のルート("/"等) */
|
|
32
33
|
mirroredRoots = new Map();
|
|
@@ -91,8 +92,13 @@ export class ScanSnapshot {
|
|
|
91
92
|
}
|
|
92
93
|
this.used += result.bytes;
|
|
93
94
|
this.copied.set(absolute, dest);
|
|
95
|
+
this.copiedPaths.add(dest);
|
|
94
96
|
return { ok: true, path: dest };
|
|
95
97
|
}
|
|
98
|
+
/** Only completed private copies can be followed during the final parent-chain check. */
|
|
99
|
+
hasCopy(copyPath) {
|
|
100
|
+
return this.copiedPaths.has(copyPath);
|
|
101
|
+
}
|
|
96
102
|
/** 本サーバーが生成した内容(requirements.txtの正規化コピー等)を書く */
|
|
97
103
|
async writeGenerated(content) {
|
|
98
104
|
const dir = path.join(this.dir, "generated");
|
|
@@ -116,10 +122,10 @@ export class ScanSnapshot {
|
|
|
116
122
|
}
|
|
117
123
|
}
|
|
118
124
|
const UNPARSEABLE = "The parent POM reference cannot be interpreted with certainty (duplicate parent elements, CDATA, DOCTYPE, property references, an encoding other than UTF-8, and so on), " +
|
|
119
|
-
"so
|
|
125
|
+
"so its parent chain could not be confirmed to stay inside the snapshot and the file was excluded from the scan";
|
|
120
126
|
/**
|
|
121
127
|
* pom.xmlと親POMの連鎖をスナップショットへコピーする。
|
|
122
|
-
*
|
|
128
|
+
* 連鎖がスナップショットの外を参照するか解釈できない場合は除外する。許可ルート設定時は元の参照先も制限する。
|
|
123
129
|
* 存在する親POMを再現できなかった場合(サイズ超過・通常のファイルでない等)はスキャンを続けるが、
|
|
124
130
|
* 親から継承する依存が欠ける可能性があるためincompleteに理由を返す(黙って成功扱いにしない)。
|
|
125
131
|
*/
|
|
@@ -129,7 +135,6 @@ async function snapshotPom(snapshot, pomPath, context) {
|
|
|
129
135
|
return { ok: false, reason: capitalizeReason(copied.message), kind: "unreadable" };
|
|
130
136
|
const first = { ok: true, path: copied.path };
|
|
131
137
|
const partial = (reason) => ({ ...first, incomplete: reason });
|
|
132
|
-
const restricted = context.allowedRootReal !== undefined;
|
|
133
138
|
let current = pomPath;
|
|
134
139
|
let currentCopy = first.path;
|
|
135
140
|
for (let depth = 0; depth < MAX_PARENT_DEPTH; depth++) {
|
|
@@ -137,17 +142,13 @@ async function snapshotPom(snapshot, pomPath, context) {
|
|
|
137
142
|
if (relativePath === null || relativePath === "")
|
|
138
143
|
return first;
|
|
139
144
|
if (relativePath === undefined) {
|
|
140
|
-
return
|
|
141
|
-
? { ok: false, reason: UNPARSEABLE, kind: "unreadable" }
|
|
142
|
-
: partial("The parent POM reference cannot be interpreted with certainty, so the file was scanned without its parent POM. Dependencies inherited from the parent may be missing from the results");
|
|
145
|
+
return { ok: false, reason: UNPARSEABLE, kind: "unreadable" };
|
|
143
146
|
}
|
|
144
147
|
// Goのfilepath.Joinと同じ規則で結合する(絶対パスも連結し、..はルートで止まる)
|
|
145
148
|
const inCopy = path.join(path.dirname(currentCopy), relativePath);
|
|
146
149
|
if (!isInsideDir(snapshot.treeRoot, inCopy)) {
|
|
147
150
|
// ..を重ねるとスナップショットの外(本物のファイルシステム)に届くため、コピーではなく元のファイルが読まれる
|
|
148
|
-
return
|
|
149
|
-
? { ok: false, reason: `The parent POM reference (relativePath: ${relativePath}) leads outside the scan scope, so the file was excluded from the scan`, kind: "outside_allowed_root" }
|
|
150
|
-
: first;
|
|
151
|
+
return { ok: false, reason: `The parent POM reference (relativePath: ${relativePath}) leads outside the scan scope, so the file was excluded from the scan`, kind: escapeKind(context) };
|
|
151
152
|
}
|
|
152
153
|
let candidate = path.join(path.dirname(current), relativePath);
|
|
153
154
|
try {
|
|
@@ -180,9 +181,35 @@ async function snapshotPom(snapshot, pomPath, context) {
|
|
|
180
181
|
current = candidate;
|
|
181
182
|
currentCopy = parent.path;
|
|
182
183
|
}
|
|
183
|
-
return
|
|
184
|
-
|
|
185
|
-
|
|
184
|
+
return { ok: false, reason: `The parent POM chain exceeds the limit (${MAX_PARENT_DEPTH} levels)`, kind: "unreadable" };
|
|
185
|
+
}
|
|
186
|
+
/**
|
|
187
|
+
* スナップショットの外に出る参照の除外の種類。許可ルート未設定時は「許可ルートの外」ではないため、
|
|
188
|
+
* 全件が除外されたときのエラーをpath_outside_allowed_rootではなくno_manifest_foundにする
|
|
189
|
+
*/
|
|
190
|
+
function escapeKind(context) {
|
|
191
|
+
return context.allowedRootReal !== undefined ? "outside_allowed_root" : "unreadable";
|
|
192
|
+
}
|
|
193
|
+
/** Check the final scanner-visible graph, independently of the now-mutable source files. */
|
|
194
|
+
async function validateCopiedPom(snapshot, copyPath, context) {
|
|
195
|
+
let current = copyPath;
|
|
196
|
+
for (let depth = 0; depth < MAX_PARENT_DEPTH; depth++) {
|
|
197
|
+
const relativePath = await snapshot.parentOf(current);
|
|
198
|
+
if (relativePath === null || relativePath === "")
|
|
199
|
+
return null;
|
|
200
|
+
if (relativePath === undefined)
|
|
201
|
+
return { reason: UNPARSEABLE, kind: "unreadable" };
|
|
202
|
+
let parent = path.join(path.dirname(current), relativePath);
|
|
203
|
+
if (!isInsideDir(snapshot.treeRoot, parent)) {
|
|
204
|
+
return { reason: "The copied parent POM chain leads outside the scan scope, so the file was excluded from the scan", kind: escapeKind(context) };
|
|
205
|
+
}
|
|
206
|
+
if (!snapshot.hasCopy(parent))
|
|
207
|
+
parent = path.join(parent, "pom.xml");
|
|
208
|
+
if (!snapshot.hasCopy(parent))
|
|
209
|
+
return null;
|
|
210
|
+
current = parent;
|
|
211
|
+
}
|
|
212
|
+
return { reason: `The parent POM chain exceeds the limit (${MAX_PARENT_DEPTH} levels)`, kind: "unreadable" };
|
|
186
213
|
}
|
|
187
214
|
/**
|
|
188
215
|
* 検出済みのマニフェストをスナップショットへコピーし、osv-scannerに渡す対象(コピー)を返す。
|
|
@@ -190,6 +217,7 @@ async function snapshotPom(snapshot, pomPath, context) {
|
|
|
190
217
|
*/
|
|
191
218
|
export async function snapshotManifests(snapshot, targets, context) {
|
|
192
219
|
const copies = [];
|
|
220
|
+
const copiedPoms = [];
|
|
193
221
|
const skipped = [];
|
|
194
222
|
const incomplete = [];
|
|
195
223
|
for (const target of targets) {
|
|
@@ -197,6 +225,7 @@ export async function snapshotManifests(snapshot, targets, context) {
|
|
|
197
225
|
const result = await snapshotPom(snapshot, target.path, context);
|
|
198
226
|
if (result.ok) {
|
|
199
227
|
copies.push({ path: result.path, format: target.format });
|
|
228
|
+
copiedPoms.push({ original: target.path, copy: result.path });
|
|
200
229
|
if (result.incomplete !== undefined)
|
|
201
230
|
incomplete.push({ path: target.path, reason: result.incomplete });
|
|
202
231
|
}
|
|
@@ -211,5 +240,20 @@ export async function snapshotManifests(snapshot, targets, context) {
|
|
|
211
240
|
else
|
|
212
241
|
skipped.push({ path: target.path, reason: capitalizeReason(result.message), kind: "unreadable" });
|
|
213
242
|
}
|
|
214
|
-
|
|
243
|
+
// A later target may populate a parent path that was absent during an earlier check.
|
|
244
|
+
const rejectedCopies = new Set();
|
|
245
|
+
const rejectedOriginals = new Set();
|
|
246
|
+
for (const { original, copy } of copiedPoms) {
|
|
247
|
+
const failure = await validateCopiedPom(snapshot, copy, context);
|
|
248
|
+
if (failure === null)
|
|
249
|
+
continue;
|
|
250
|
+
skipped.push({ path: original, ...failure });
|
|
251
|
+
rejectedCopies.add(copy);
|
|
252
|
+
rejectedOriginals.add(original);
|
|
253
|
+
}
|
|
254
|
+
return {
|
|
255
|
+
targets: copies.filter((target) => !rejectedCopies.has(target.path)),
|
|
256
|
+
skipped,
|
|
257
|
+
incomplete: incomplete.filter((item) => !rejectedOriginals.has(item.path)),
|
|
258
|
+
};
|
|
215
259
|
}
|