muaddib-scanner 2.11.139 → 2.11.141
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md
CHANGED
|
@@ -30,7 +30,7 @@
|
|
|
30
30
|
|
|
31
31
|
npm and PyPI supply-chain attacks are exploding. Shai-Hulud compromised 25K+ repos in 2025. Existing tools detect threats but don't help you respond.
|
|
32
32
|
|
|
33
|
-
MUAD'DIB combines **
|
|
33
|
+
MUAD'DIB combines **21 parallel scanners** (274 detection rules), a **deobfuscation engine**, **inter-module dataflow analysis**, **compound scoring** (20 compound rules), and a gVisor/Docker sandbox to detect known threats and suspicious behavioral patterns in npm and PyPI packages. An XGBoost classifier exists in the codebase but is **currently inactive** (see [Evaluation](#evaluation)).
|
|
34
34
|
|
|
35
35
|
---
|
|
36
36
|
|
|
@@ -45,6 +45,16 @@ MUAD'DIB is an educational tool and a free first line of defense. It detects **k
|
|
|
45
45
|
|
|
46
46
|
---
|
|
47
47
|
|
|
48
|
+
## Scope
|
|
49
|
+
|
|
50
|
+
**Detects** (npm & PyPI): known-malicious packages (name + SHA256 IOC match), typosquats, install-time RCE (lifecycle `preinstall`/`postinstall`, `curl | sh`, Python import-time, `binding.gyp`), credential read then network exfiltration (intra- and cross-file), obfuscated / high-entropy / stub-loader payloads, binary droppers (`chmod +x` + exec/spawn), and anti-analysis evasion markers.
|
|
51
|
+
|
|
52
|
+
**Out of scope**: browser-only attacks (DOM/`window`, no Node.js API), the *contents* of native binaries / WASM (no binary analysis), zero-day unknown packages (the IOC feed is reactive), and non-npm/PyPI ecosystems (RubyGems, Maven, Go). Determined anti-sandbox fingerprinting and multi-stage remote payloads are known false-negative risks. Full detail: [Threat Model](docs/threat-model.md).
|
|
53
|
+
|
|
54
|
+
**No telemetry.** Your code and scan results never leave your machine — MUAD'DIB only *downloads* threat-intel feeds (`muaddib update`) and, for scoring, reads public npm registry metadata. Webhook alerts are opt-in.
|
|
55
|
+
|
|
56
|
+
---
|
|
57
|
+
|
|
48
58
|
## Installation
|
|
49
59
|
|
|
50
60
|
### npm (recommended)
|
|
@@ -176,7 +186,7 @@ muaddib replay # Ground truth validation (90/94 TPR@3, v2.11
|
|
|
176
186
|
|
|
177
187
|
## Features
|
|
178
188
|
|
|
179
|
-
###
|
|
189
|
+
### 21 parallel scanners
|
|
180
190
|
|
|
181
191
|
| Scanner | Detection |
|
|
182
192
|
|---------|-----------|
|
|
@@ -201,10 +211,11 @@ muaddib replay # Ground truth validation (90/94 TPR@3, v2.11
|
|
|
201
211
|
| Trusted-Dep-Diff (opt-in) | Diff against trusted dep tarballs from registry (v2.10.x) |
|
|
202
212
|
| Python Source (PYSRC) | Import-time / install-time RCE patterns in `__init__.py` / `setup.py` (v2.11.41 — closes TrapDoor PyPI gap) |
|
|
203
213
|
| Python AST (PYAST) | Tree-sitter-Python AST with taint-aware detectors (v2.11.42+) |
|
|
214
|
+
| Anti-Scanner Injection (ASI) | Prompt-injection text in comments/strings that coerces an LLM code reviewer into a clean verdict or into skipping an obfuscated payload (ASI-001..004, Hades campaign 2026-06) |
|
|
204
215
|
|
|
205
216
|
### 274 detection rules
|
|
206
217
|
|
|
207
|
-
All rules (269 RULES + 5 PARANOID) are mapped to MITRE ATT&CK techniques. See [SECURITY.md](SECURITY.md#detection-rules-
|
|
218
|
+
All rules (269 RULES + 5 PARANOID) are mapped to MITRE ATT&CK techniques. See [SECURITY.md](SECURITY.md#detection-rules-v211139) for the complete rules reference.
|
|
208
219
|
|
|
209
220
|
### Detected campaigns
|
|
210
221
|
|
|
@@ -278,75 +289,30 @@ With pre-commit framework:
|
|
|
278
289
|
```yaml
|
|
279
290
|
repos:
|
|
280
291
|
- repo: https://github.com/DNSZLSK/muad-dib
|
|
281
|
-
rev: v2.11.
|
|
292
|
+
rev: v2.11.139
|
|
282
293
|
hooks:
|
|
283
294
|
- id: muaddib-scan
|
|
284
295
|
```
|
|
285
296
|
|
|
286
297
|
---
|
|
287
298
|
|
|
288
|
-
## Evaluation
|
|
289
|
-
|
|
290
|
-
Latest measurement: **v2.11.48** (2026-05-26, Track D + PyPI download fix). Ground truth holds 96 samples (94 in-scope, 2 out-of-scope protestware). This run measures the full 94 in-scope set after the 2026-05-25 enrichment (Track C synthetic for the new PYSRC/PYAST/AST-092/AICONF-004/PKG-022 rules, Track A real-world tarballs recovered from VPS archive, Track B reconstructions from the in-house security-review benchmark).
|
|
291
|
-
|
|
292
|
-
### Operational metrics (what an operator actually gets)
|
|
293
|
-
|
|
294
|
-
These are the numbers a user gets when running `muaddib scan` against npm or PyPI packages. The pipeline executes scanners + FP caps only — no ML filter is applied (see ML Classifier note below).
|
|
295
|
-
|
|
296
|
-
| Metric | Result | Details |
|
|
297
|
-
|--------|--------|---------|
|
|
298
|
-
| **Wild TPR** (Datadog 17K) | **92.8%** (13,538/14,587 in-scope) | 17,922 packages. 3,335 skipped (no JS). By category: compromised_lib 97.8%, malicious_intent 92.1% — last measurement v2.9.4, independent of GT. |
|
|
299
|
-
| **TPR@3** (detection rate, v2.11.48) | **95.74%** (90/94 in-scope) | Full GT re-measurement. Threshold=3: any signal. 13 PyPI samples (was 0). 4 misses incl. 3 browser-only (lottie-player, polyfill-io, trojanized-jquery). |
|
|
300
|
-
| **TPR@20** (alert rate, v2.11.48) | **88.30%** (83/94 in-scope) | Operational alert threshold=20. **+3.1pp vs v2.11.47** — Track D `recon_exfil_direct_ip` compound (MUADDIB-COMPOUND-016) closed the GT-095 gap (risk 3→50) and boosted GT-091 byvendors / GT-092 heloo131313 through `linux_fingerprint_exec`. |
|
|
301
|
-
| **FPR rules** (Benign curated, v2.11.48 measure) | **1.10%** (6/545 scanned, 548 total) | **Unchanged after Track D** — the new compound + types created zero new FPs (sameFile gate + public-IP-only filter). Drop from 15.6% (v2.10.95) is attributable to FP caps F1-F14 (v2.10.97 → v2.11.31). 6 remaining FPs are real (meteor, prisma, @prisma/client, drizzle-orm, scrypt, liquid). |
|
|
302
|
-
| **FPR** (Benign random, v2.11.48) | **2.50%** (5/200) | 200 random npm packages, unchanged. |
|
|
303
|
-
| **FPR PyPI** (v2.11.48, first honest measurement) | **9.68%** (12/124 scanned, 132 total) | **Track D fixed the PyPI downloader** — removed `pip --no-binary :all:` flag (forced compile of wheel-only packages, timed out 38% of the time) + added `.whl` extraction via `extractArchive()`. Brought 42 previously-skipped giants (numpy/pandas/django/matplotlib/scikit-learn/...) into scope. All 12 FPs cluster at score 25-35: this is the cap-PyPI-35 artifact, not new rule misfires. Lifting the cap (Track E) would drop FPR PyPI to ≈0%. 8 residual fails are >500MB packages (torch, tensorflow, scipy, opencv-python, ansible…) hitting the 30s `PACK_TIMEOUT_MS`. |
|
|
304
|
-
| **ADR** (Adversarial + Holdout, v2.11.48) | **96.26%** (103/107) | 67 adversarial + 40 holdout, global threshold=20. Stable vs v2.10.95. |
|
|
305
|
-
|
|
306
|
-
**4414 tests** across 141 files. **266 rules** (261 RULES + 5 PARANOID; v2.11.67/70 Phantom Gyp added PKG-023 + COMPOUND-017).
|
|
307
|
-
|
|
308
|
-
**Known issues (v2.11.48):**
|
|
309
|
-
- *Cap PyPI à 35/100*: Python samples plafonnent à `riskScore=35` even when `globalRiskScore=100`. Confirmed empirically — all 12 PyPI FPs at score 25-35 (flask 32, django 35, tornado 35, bottle 30, pandas 25, matplotlib 25, plotly 25, bokeh 25, pymongo 35, coverage 32, fabric 35, websockets 35). Lifting the cap will simultaneously drop FPR PyPI to ≈0% and unblock PyPI MALWARE detection at higher thresholds. Track E target.
|
|
310
|
-
|
|
311
|
-
### Operational coverage (v2.11.67-76)
|
|
312
|
-
|
|
313
|
-
The static ground-truth TPR above is measured offline. Since v2.11.67 the monitor also tracks **operational** coverage on live npm/PyPI ingestion:
|
|
314
|
-
- A per-scan **ledger** (`data/scan-ledger.jsonl`) records every scanned package's outcome; `computeLedgerRollup()` produces a 24h rollup (`alertRate`, per-ecosystem). Note: `alertRate` is a throughput signal, **not** detection TPR.
|
|
315
|
-
- An active **GHSA poller** (~15 min; npm, pypi, crates) builds an authoritative "what should we have caught" denominator (`data/ghsa-malware.jsonl`), plus a **feed-health** alarm that fires when an IOC feed silently goes dark.
|
|
316
|
-
- The Phase 5 **coverage-audit** (`scripts/coverage-audit.js`, daily 05:00 UTC) joins that denominator against ledger outcomes + the tarball archive to compute an honest GHSA-denominated **operational TPR** (`alerted / total`), and surfaces `scannedClean` misses as human-gated ground-truth candidates.
|
|
317
|
-
|
|
318
|
-
This operational TPR is the real production detection rate, distinct from the static GT TPR (which has not been re-measured since v2.11.48).
|
|
319
|
-
|
|
320
|
-
### ML Classifier (offline only)
|
|
321
|
-
|
|
322
|
-
`src/ml/classifier.js` is **not wired into `muaddib scan`**. The XGBoost model is currently exercised only by `muaddib evaluate` (offline metric replay) and `muaddib monitor` (LOG-ONLY since 2026-04-08, model collapsed pending retrain — see `src/monitor/queue.js:628`). The v2.11.48 evaluate-time replay shows the same 1.10% FPR (no additional FPs filtered) — kept as a reference for retrain validation, but the published operational FPR is the rules-only number above.
|
|
323
|
-
|
|
324
|
-
> **Static evaluation caveats:**
|
|
325
|
-
> - TPR measured on the full 94 in-scope samples from the 96-sample ground truth (2 out-of-scope protestware GT-005/GT-009 with `min_threats=0`)
|
|
326
|
-
> - TPR@3 = detection rate (any signal); TPR@20 = operational alert threshold
|
|
327
|
-
> - FPR rules measured on 548 curated popular npm packages (not a random sample)
|
|
328
|
-
> - FPR PyPI: 124/132 scanned (8 download fails on >500MB giants — torch/tensorflow/ansible/…). Smaller N than npm.
|
|
329
|
-
> - ADR measured with global threshold (score >= 20) as of v2.6.5
|
|
330
|
-
|
|
331
|
-
See [Evaluation Methodology](docs/EVALUATION_METHODOLOGY.md) for the full experimental protocol, holdout history, and Datadog benchmark details.
|
|
299
|
+
## Evaluation
|
|
332
300
|
|
|
333
|
-
|
|
301
|
+
Last measured **v2.11.48** (2026-05-26), **rules-only** (the ML classifier is inactive — see below). Ground truth: 94 in-scope real-world attacks + 200 random npm + 124 PyPI + 107 adversarial/holdout.
|
|
334
302
|
|
|
335
|
-
|
|
303
|
+
| Metric | Result |
|
|
304
|
+
|--------|--------|
|
|
305
|
+
| Detection rate (TPR@3) | **95.74%** (90/94) |
|
|
306
|
+
| Alert rate (TPR@20) | **88.30%** (83/94) |
|
|
307
|
+
| FPR — curated npm (548) | **1.10%** (6/545) |
|
|
308
|
+
| FPR — random npm (200) | **2.50%** (5/200) |
|
|
309
|
+
| FPR — PyPI (132) | **9.68%** (12/124) |
|
|
310
|
+
| ADR — adversarial + holdout | **96.26%** (103/107) |
|
|
311
|
+
| Wild TPR (Datadog 17K) | **92.8%** (13,538/14,587) |
|
|
336
312
|
|
|
337
|
-
|
|
338
|
-
|--------|--------|---------|
|
|
339
|
-
| **ML FPR** | **2.85%** (239/8,393 holdout) | XGBoost retrained on 56,564 samples, 64 features, threshold=0.710 |
|
|
340
|
-
| **ML TPR** | **99.93%** (2,918/2,920 holdout) | 377 confirmed_malicious via OSSF/GHSA/npm correlation |
|
|
341
|
-
| **FPR after ML T1** (offline replay, v2.11.48) | **1.10%** (6/545 scanned) | Classifier filters 0/6 raw FPs in this run (filtered 1 at v2.11.47). Not applied during real scans — `muaddib scan` never invokes the classifier. |
|
|
313
|
+
**ML classifier: inactive.** An XGBoost model lives in `src/ml/` but is never wired into `muaddib scan`, and runs LOG-ONLY in the monitor since 2026-04-08 (the trained model collapsed, pending retrain). All numbers above are rules-only.
|
|
342
314
|
|
|
343
|
-
|
|
344
|
-
> - Ground truth: 377 confirmed_malicious via auto-labeler (OSSF malicious-packages, GitHub Advisory Database, npm registry takedown correlation)
|
|
345
|
-
> - Dataset: 56,564 samples (14,602 malicious, 41,962 clean). Stratified 80/20 split
|
|
346
|
-
> - Grid search: depth=4, estimators=300, lr=0.05. AUC-ROC=0.999, F1=0.960
|
|
347
|
-
> - Leaky feature filter: 23 dead/leaky features removed (source-identity proxies)
|
|
348
|
-
>
|
|
349
|
-
> The shadow model continues to log predictions in `muaddib monitor` for retraining validation. When the next model passes shadow validation, the LOG-ONLY guard in `src/monitor/queue.js:660` will be flipped and the metrics above will move back into the operational table.
|
|
315
|
+
Full protocol, per-track history, the PyPI cap-35 caveat, operational (GHSA-denominated) coverage, and the ML retrain methodology: [Evaluation Methodology](docs/EVALUATION_METHODOLOGY.md).
|
|
350
316
|
|
|
351
317
|
---
|
|
352
318
|
|
|
@@ -380,7 +346,7 @@ npm test
|
|
|
380
346
|
|
|
381
347
|
### Testing
|
|
382
348
|
|
|
383
|
-
- **
|
|
349
|
+
- **4500 tests** across 147 modular test files
|
|
384
350
|
- **56 fuzz tests** - Malformed inputs, ReDoS, unicode, binary
|
|
385
351
|
- **Datadog 17K benchmark** - 14,587 confirmed malware samples (in-scope)
|
|
386
352
|
- **Ground truth validation** - 96 real-world attacks (95.74% TPR@3, 88.30% TPR@20 — v2.11.48 full measure on 94 in-scope)
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "muaddib-scanner",
|
|
3
|
-
"version": "2.11.
|
|
3
|
+
"version": "2.11.141",
|
|
4
4
|
"description": "Supply-chain threat detection & response for npm & PyPI/Python",
|
|
5
5
|
"main": "src/index.js",
|
|
6
6
|
"bin": {
|
|
@@ -48,16 +48,16 @@
|
|
|
48
48
|
},
|
|
49
49
|
"dependencies": {
|
|
50
50
|
"@inquirer/prompts": "8.5.2",
|
|
51
|
-
"acorn": "8.
|
|
51
|
+
"acorn": "8.17.0",
|
|
52
52
|
"acorn-walk": "8.3.5",
|
|
53
53
|
"adm-zip": "0.5.17",
|
|
54
|
-
"js-yaml": "
|
|
54
|
+
"js-yaml": "5.1.0",
|
|
55
55
|
"web-tree-sitter": "^0.26.9"
|
|
56
56
|
},
|
|
57
57
|
"devDependencies": {
|
|
58
58
|
"@eslint/js": "10.0.1",
|
|
59
|
-
"eslint": "10.
|
|
59
|
+
"eslint": "10.5.0",
|
|
60
60
|
"eslint-plugin-security": "^4.0.0",
|
|
61
|
-
"globals": "17.
|
|
61
|
+
"globals": "17.7.0"
|
|
62
62
|
}
|
|
63
63
|
}
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"target": "node_modules",
|
|
3
|
-
"timestamp": "2026-07-
|
|
3
|
+
"timestamp": "2026-07-01T21:11:52.228Z",
|
|
4
4
|
"threats": [
|
|
5
5
|
{
|
|
6
6
|
"type": "string_mutation_obfuscation",
|
|
@@ -1068,28 +1068,6 @@
|
|
|
1068
1068
|
"playbook": "SharedArrayBuffer + Worker Thread detectes. Canal IPC memoire partagee qui contourne la surveillance. Verifier si les workers manipulent des donnees sensibles. Isoler si combine avec eval/exec.",
|
|
1069
1069
|
"points": 3
|
|
1070
1070
|
},
|
|
1071
|
-
{
|
|
1072
|
-
"type": "dangerous_exec",
|
|
1073
|
-
"severity": "MEDIUM",
|
|
1074
|
-
"message": "[quick-scan] exec/spawn call detected in overflow file.",
|
|
1075
|
-
"file": "eslint/lib/rules/eol-last.js",
|
|
1076
|
-
"degraded": true,
|
|
1077
|
-
"quickScan": true,
|
|
1078
|
-
"count": 1,
|
|
1079
|
-
"reductions": [],
|
|
1080
|
-
"originalSeverity": "MEDIUM",
|
|
1081
|
-
"confidenceTier": "low",
|
|
1082
|
-
"rule_id": "MUADDIB-AST-007",
|
|
1083
|
-
"rule_name": "Dangerous Shell Command Execution",
|
|
1084
|
-
"confidence": "high",
|
|
1085
|
-
"domain": "malware",
|
|
1086
|
-
"references": [
|
|
1087
|
-
"https://owasp.org/www-community/attacks/Command_Injection"
|
|
1088
|
-
],
|
|
1089
|
-
"mitre": "T1059.004",
|
|
1090
|
-
"playbook": "CRITIQUE: Execution de commande shell dangereuse detectee. Isoler la machine. Verifier si la commande a ete executee.",
|
|
1091
|
-
"points": 3
|
|
1092
|
-
},
|
|
1093
1071
|
{
|
|
1094
1072
|
"type": "dangerous_exec",
|
|
1095
1073
|
"severity": "MEDIUM",
|
|
@@ -1468,7 +1446,29 @@
|
|
|
1468
1446
|
"type": "dangerous_exec",
|
|
1469
1447
|
"severity": "MEDIUM",
|
|
1470
1448
|
"message": "[quick-scan] exec/spawn call detected in overflow file.",
|
|
1471
|
-
"file": "js-yaml/
|
|
1449
|
+
"file": "js-yaml/dist/browser/js-yaml.esm.min.mjs",
|
|
1450
|
+
"degraded": true,
|
|
1451
|
+
"quickScan": true,
|
|
1452
|
+
"count": 1,
|
|
1453
|
+
"reductions": [],
|
|
1454
|
+
"originalSeverity": "MEDIUM",
|
|
1455
|
+
"confidenceTier": "low",
|
|
1456
|
+
"rule_id": "MUADDIB-AST-007",
|
|
1457
|
+
"rule_name": "Dangerous Shell Command Execution",
|
|
1458
|
+
"confidence": "high",
|
|
1459
|
+
"domain": "malware",
|
|
1460
|
+
"references": [
|
|
1461
|
+
"https://owasp.org/www-community/attacks/Command_Injection"
|
|
1462
|
+
],
|
|
1463
|
+
"mitre": "T1059.004",
|
|
1464
|
+
"playbook": "CRITIQUE: Execution de commande shell dangereuse detectee. Isoler la machine. Verifier si la commande a ete executee.",
|
|
1465
|
+
"points": 3
|
|
1466
|
+
},
|
|
1467
|
+
{
|
|
1468
|
+
"type": "dangerous_exec",
|
|
1469
|
+
"severity": "MEDIUM",
|
|
1470
|
+
"message": "[quick-scan] exec/spawn call detected in overflow file.",
|
|
1471
|
+
"file": "js-yaml/dist/browser/js-yaml.umd.min.js",
|
|
1472
1472
|
"degraded": true,
|
|
1473
1473
|
"quickScan": true,
|
|
1474
1474
|
"count": 1,
|
|
@@ -38,6 +38,51 @@ const {
|
|
|
38
38
|
ETHEREUM_PACKAGES,
|
|
39
39
|
SOLANA_PACKAGES
|
|
40
40
|
} = require('./constants.js');
|
|
41
|
+
|
|
42
|
+
// -----------------------------------------------------------------------------
|
|
43
|
+
// SECURITY (E3 — audit 2026-07): ReDoS mitigation for static `.replace()`-chain resolution.
|
|
44
|
+
// The resolver (Blue Team v8, below) statically re-applies `.replace(/regex/, str)` calls
|
|
45
|
+
// taken from the SCANNED package to reconstruct obfuscated strings. On the CLI path this
|
|
46
|
+
// runs synchronously on the main thread, so executing an attacker-authored regex that
|
|
47
|
+
// catastrophically backtracks would wedge it. Note: the subject-length cap does NOT bound
|
|
48
|
+
// this — exponential backtracking blows up on ~30 chars regardless of cap.
|
|
49
|
+
//
|
|
50
|
+
// Defense in depth (this is layer 1 of 2):
|
|
51
|
+
// Layer 1 (here): a BLOCKLIST of common catastrophic shapes + length caps, so the
|
|
52
|
+
// frequent cases fast-fail without running the regex. It is NOT exhaustive — a crafted
|
|
53
|
+
// pattern (e.g. nested groups like `((a+)x)+`) can slip past it.
|
|
54
|
+
// Layer 2 (hard bound): the monitor runs every scan in a Worker with a 45s
|
|
55
|
+
// STATIC_SCAN_TIMEOUT_MS -> worker.terminate() (src/monitor/queue.js). V8 TerminateExecution
|
|
56
|
+
// interrupts even a running regex (verified on Node 24 / V8 13.6: terminate returns ~6ms
|
|
57
|
+
// mid-backtrack) — TO CONFIRM on the VPS Node version: interruptible regexps have been in
|
|
58
|
+
// V8 since ~Node 16, but this was only tested on Node 24. On that basis a layer-1 bypass
|
|
59
|
+
// is bounded to <=45s and the worker is killed.
|
|
60
|
+
// The CLI runs the pipeline inline (src/index.js) and has no such backstop — one-shot,
|
|
61
|
+
// Ctrl-C-able, lower severity.
|
|
62
|
+
//
|
|
63
|
+
// A refused pattern only leaves the chain "unresolved" (still flagged by the depth>=4
|
|
64
|
+
// fallback) — never a false negative by crash.
|
|
65
|
+
// -----------------------------------------------------------------------------
|
|
66
|
+
const MAX_REPLACE_REGEX_LEN = 200; // reject absurdly long attacker regex sources
|
|
67
|
+
const MAX_REPLACE_SUBJECT_LEN = 4000; // caps LINEAR/polynomial blowup on huge subjects (not exponential)
|
|
68
|
+
|
|
69
|
+
// BLOCKLIST (non-exhaustive) of common catastrophic-backtracking regex shapes. Allows any
|
|
70
|
+
// pattern that does not match a known-bad shape — the hard guarantee is layer 2 above, not
|
|
71
|
+
// this heuristic. Its OWN patterns are linear (negated char classes only), so it introduces
|
|
72
|
+
// no ReDoS of its own.
|
|
73
|
+
function isCatastrophicReplaceRegex(src) {
|
|
74
|
+
if (typeof src !== 'string') return true;
|
|
75
|
+
// Quantifier applied to a group whose body already contains a quantifier:
|
|
76
|
+
// (a+)+ (a*)* (a+)* ([ab]+)+ (\w+)*
|
|
77
|
+
if (/\([^()]*[+*][^()]*\)\s*[*+]/.test(src)) return true;
|
|
78
|
+
// Quantified group whose body contains a {n,m} repetition: (\d{1,3})+
|
|
79
|
+
if (/\([^()]*\{\d+(?:,\d*)?\}[^()]*\)\s*[*+]/.test(src)) return true;
|
|
80
|
+
// Quantified group containing alternation (possible overlap): (a|a)* (ab|a)+
|
|
81
|
+
if (/\([^()]*\|[^()]*\)\s*[*+]/.test(src)) return true;
|
|
82
|
+
// Unbounded {n,} repetition applied to a group: (...)+{2,}
|
|
83
|
+
if (/\)[*+?]?\{\d+,\}/.test(src)) return true;
|
|
84
|
+
return false;
|
|
85
|
+
}
|
|
41
86
|
const {
|
|
42
87
|
extractStringValue,
|
|
43
88
|
calculateShannonEntropy,
|
|
@@ -1919,14 +1964,20 @@ function handleCallExpression(node, ctx) {
|
|
|
1919
1964
|
let replacement = null;
|
|
1920
1965
|
// Extract regex or string pattern
|
|
1921
1966
|
if (args[0].type === 'Literal' && args[0].regex) {
|
|
1922
|
-
|
|
1967
|
+
// SECURITY (E3): the pattern is attacker-controlled. Refuse catastrophic
|
|
1968
|
+
// shapes / over-long sources so a ReDoS can't wedge this synchronous scanner.
|
|
1969
|
+
if (args[0].regex.pattern.length <= MAX_REPLACE_REGEX_LEN &&
|
|
1970
|
+
!isCatastrophicReplaceRegex(args[0].regex.pattern)) {
|
|
1971
|
+
try { pattern = new RegExp(args[0].regex.pattern, args[0].regex.flags); } catch { /* skip */ }
|
|
1972
|
+
}
|
|
1923
1973
|
} else if (args[0].type === 'Literal' && typeof args[0].value === 'string') {
|
|
1924
1974
|
pattern = args[0].value;
|
|
1925
1975
|
}
|
|
1926
1976
|
if (args[1].type === 'Literal' && typeof args[1].value === 'string') {
|
|
1927
1977
|
replacement = args[1].value;
|
|
1928
1978
|
}
|
|
1929
|
-
|
|
1979
|
+
// SECURITY (E3): also refuse to run any pattern against an oversized subject.
|
|
1980
|
+
if (pattern !== null && replacement !== null && resolved.length <= MAX_REPLACE_SUBJECT_LEN) {
|
|
1930
1981
|
resolved = resolved.replace(pattern, replacement);
|
|
1931
1982
|
} else {
|
|
1932
1983
|
resolved = null;
|
|
@@ -2101,4 +2152,4 @@ function handleCallExpression(node, ctx) {
|
|
|
2101
2152
|
|
|
2102
2153
|
// _shadowClassifyMcpWrite is shared with handle-post-walk.js (the Wave-4
|
|
2103
2154
|
// keyword-co-occurrence emitter — the third mcp_config_injection site).
|
|
2104
|
-
module.exports = { handleCallExpression, _shadowClassifyMcpWrite };
|
|
2155
|
+
module.exports = { handleCallExpression, _shadowClassifyMcpWrite, _isCatastrophicReplaceRegex: isCatastrophicReplaceRegex };
|
package/src/shared/constants.js
CHANGED
|
@@ -85,6 +85,10 @@ const NPM_PACKAGE_REGEX = /^(@[a-z0-9-~][a-z0-9-._~]*\/)?[a-z0-9-~][a-z0-9-._~]*
|
|
|
85
85
|
// Download/extraction limits
|
|
86
86
|
const MAX_TARBALL_SIZE = 50 * 1024 * 1024; // 50MB
|
|
87
87
|
const DOWNLOAD_TIMEOUT = 30_000; // 30 seconds
|
|
88
|
+
// Decompressed-size cap for tar.gz extraction (M4). PARTIAL tar-bomb mitigation only —
|
|
89
|
+
// bypassable by a bomb forged to a multiple of 4GB + (<1GB), because the gzip ISIZE trailer
|
|
90
|
+
// this is checked against is mod 2^32. A hard cap needs streaming decompression (separate tech debt).
|
|
91
|
+
const MAX_EXTRACTED_SIZE = 1024 * 1024 * 1024; // 1GB
|
|
88
92
|
|
|
89
93
|
// Shared scanner constants
|
|
90
94
|
const MAX_FILE_SIZE = 10 * 1024 * 1024; // 10MB — skip files larger than this to avoid memory issues
|
|
@@ -146,4 +150,4 @@ function clearASTCache() {
|
|
|
146
150
|
_astCache.clear();
|
|
147
151
|
}
|
|
148
152
|
|
|
149
|
-
module.exports = { REHABILITATED_PACKAGES, NPM_PACKAGE_REGEX, MAX_TARBALL_SIZE, DOWNLOAD_TIMEOUT, MAX_FILE_SIZE, ACORN_OPTIONS, safeParse, clearASTCache, getMaxFileSize, setMaxFileSize, resetMaxFileSize };
|
|
153
|
+
module.exports = { REHABILITATED_PACKAGES, NPM_PACKAGE_REGEX, MAX_TARBALL_SIZE, DOWNLOAD_TIMEOUT, MAX_EXTRACTED_SIZE, MAX_FILE_SIZE, ACORN_OPTIONS, safeParse, clearASTCache, getMaxFileSize, setMaxFileSize, resetMaxFileSize };
|
package/src/shared/download.js
CHANGED
|
@@ -3,7 +3,7 @@ const fs = require('fs');
|
|
|
3
3
|
const path = require('path');
|
|
4
4
|
const { execFileSync } = require('child_process');
|
|
5
5
|
const AdmZip = require('adm-zip');
|
|
6
|
-
const { MAX_TARBALL_SIZE, DOWNLOAD_TIMEOUT } = require('./constants.js');
|
|
6
|
+
const { MAX_TARBALL_SIZE, DOWNLOAD_TIMEOUT, MAX_EXTRACTED_SIZE } = require('./constants.js');
|
|
7
7
|
const { registryAuthHeaders } = require('./registry-auth.js');
|
|
8
8
|
|
|
9
9
|
// Allowed redirect domains for tarball downloads (SSRF protection)
|
|
@@ -241,6 +241,34 @@ function detectArchiveFormat(archivePath) {
|
|
|
241
241
|
return 'unknown';
|
|
242
242
|
}
|
|
243
243
|
|
|
244
|
+
// SECURITY (M4 — audit 2026-07): PARTIAL tar-bomb mitigation.
|
|
245
|
+
// Reads the gzip ISIZE trailer (last 4 bytes = uncompressed size mod 2^32) in O(1) memory,
|
|
246
|
+
// synchronously, so _extractTarGzImpl can reject a declared-oversize archive BEFORE spawning
|
|
247
|
+
// `tar` (rejecting pre-spawn matters: worker.terminate() can orphan a running tar child that
|
|
248
|
+
// keeps writing to disk). LIMITATION — this is a PARTIAL mitigation, NOT a hard cap: because
|
|
249
|
+
// ISIZE is mod 2^32, a bomb forged to a multiple of 4GB + (<1GB) wraps under the cap and evades
|
|
250
|
+
// this hint; that narrow residual falls back to the 60s tar timeout. A complete cap requires
|
|
251
|
+
// streaming decompression, which would make extraction async and ripple to all sync callers —
|
|
252
|
+
// tracked as separate tech debt, deliberately not done here.
|
|
253
|
+
function readGzipUncompressedHint(tgzPath) {
|
|
254
|
+
let fd;
|
|
255
|
+
try {
|
|
256
|
+
fd = fs.openSync(tgzPath, 'r');
|
|
257
|
+
const { size } = fs.fstatSync(fd);
|
|
258
|
+
if (size < 18) return null; // too small to be a valid gzip member
|
|
259
|
+
const magic = Buffer.alloc(2);
|
|
260
|
+
fs.readSync(fd, magic, 0, 2, 0);
|
|
261
|
+
if (magic[0] !== 0x1f || magic[1] !== 0x8b) return null; // not gzip — let tar error naturally
|
|
262
|
+
const isize = Buffer.alloc(4);
|
|
263
|
+
fs.readSync(fd, isize, 0, 4, size - 4);
|
|
264
|
+
return isize.readUInt32LE(0);
|
|
265
|
+
} catch {
|
|
266
|
+
return null; // unreadable hint — never block extraction on a hint failure
|
|
267
|
+
} finally {
|
|
268
|
+
if (fd !== undefined) { try { fs.closeSync(fd); } catch { /* ignore */ } }
|
|
269
|
+
}
|
|
270
|
+
}
|
|
271
|
+
|
|
244
272
|
/**
|
|
245
273
|
* Extract a tar.gz tarball with the system `tar` binary. Used for npm
|
|
246
274
|
* tarballs and PyPI sdists. Internal implementation — call extractArchive
|
|
@@ -248,6 +276,14 @@ function detectArchiveFormat(archivePath) {
|
|
|
248
276
|
* scanner/temporal-ast-diff.js callsite.
|
|
249
277
|
*/
|
|
250
278
|
function _extractTarGzImpl(tgzPath, destDir) {
|
|
279
|
+
// SECURITY (M4): reject a declared-oversize gzip before `tar` ever runs. PARTIAL mitigation
|
|
280
|
+
// (bypassable by a 4GB-multiple + <1GB forged bomb) — see readGzipUncompressedHint.
|
|
281
|
+
const uncompressedHint = readGzipUncompressedHint(tgzPath);
|
|
282
|
+
if (uncompressedHint !== null && uncompressedHint > MAX_EXTRACTED_SIZE) {
|
|
283
|
+
throw new Error(
|
|
284
|
+
`Tar extract refused: declared uncompressed size ${uncompressedHint} exceeds ${MAX_EXTRACTED_SIZE} (possible decompression bomb)`
|
|
285
|
+
);
|
|
286
|
+
}
|
|
251
287
|
// Use cwd + relative paths so C: never appears in tar arguments
|
|
252
288
|
// (GNU tar treats C: as remote host, bsdtar doesn't support --force-local)
|
|
253
289
|
const tgzDir = path.dirname(path.resolve(tgzPath));
|
|
@@ -418,6 +454,7 @@ module.exports = {
|
|
|
418
454
|
extractArchive,
|
|
419
455
|
extractArchiveOffThread,
|
|
420
456
|
detectArchiveFormat,
|
|
457
|
+
_readGzipUncompressedHint: readGzipUncompressedHint,
|
|
421
458
|
sanitizePackageName,
|
|
422
459
|
isAllowedDownloadRedirect,
|
|
423
460
|
normalizeHostname,
|