muaddib-scanner 2.2.8 → 2.2.9

package/README.fr.md

@@ -30,7 +30,7 @@
 
 Les attaques supply-chain npm et PyPI explosent. Shai-Hulud a compromis 25K+ repos en 2025. Les outils existants détectent, mais n'aident pas à répondre.
 
-MUAD'DIB combine analyse statique + analyse dynamique (sandbox Docker) + **détection comportementale d'anomalies** (v2.0) + **validation ground truth** (v2.1) pour détecter les menaces ET guider votre réponse — même avant leur apparition dans une base d'IOC.
+MUAD'DIB combine analyse statique + **moteur de désobfuscation** (v2.2.5) + **dataflow inter-module** (v2.2.6) + analyse dynamique (sandbox Docker) + **détection comportementale d'anomalies** (v2.0) + **validation ground truth** (v2.1) pour détecter les menaces ET guider votre réponse — même avant leur apparition dans une base d'IOC.
 
 ---
 
@@ -431,6 +431,10 @@ Détecte les patterns malveillants dans les fichiers YAML `.github/workflows/`,
 | Hooking prototype (fetch, XMLHttpRequest) | T1557 | AST |
 | Injection workflow (.github/workflows) | T1195.002 | AST |
 | Harvest crypto wallets | T1005 | Dataflow |
+| Require cache poisoning | T1574.001 | AST |
+| Staged eval decode (eval+atob/Buffer) | T1140 | AST |
+| Désobfuscation (string concat, charcode, base64, hex) | T1140 | Pré-traitement AST |
+| Dataflow cross-fichiers (exfiltration inter-module) | T1041 | Module graph |
 
 ---
 
@@ -642,7 +646,7 @@ Les alertes apparaissent dans Security > Code scanning alerts.
 ## Architecture
 
 ```
-MUAD'DIB 2.2 Scanner
+MUAD'DIB 2.2.9 Scanner
 |
 +-- IOC Match (225 000+ packages, JSON DB)
 |   +-- OSV.dev npm dump (200K+ entrées MAL-*)
@@ -654,7 +658,17 @@ MUAD'DIB 2.2 Scanner
 |   +-- Snyk Known Malware
 |   +-- Static IOCs (Socket, Phylum)
 |
-+-- 13 Scanners Parallèles (86 règles)
++-- Pré-traitement Désobfuscation (v2.2.5, --no-deobfuscate pour désactiver)
+|   +-- String concat folding, reconstruction CharCode
+|   +-- Décodage Base64, résolution Hex array
+|   +-- Propagation de constantes (Phase 2)
+|
++-- Dataflow Inter-module (v2.2.6, --no-module-graph pour désactiver)
+|   +-- Graphe de dépendances, annotation exports teintés
+|   +-- Chaînes re-export 3 hops, analyse méthodes de classe
+|   +-- Détection credential read → network sink cross-fichiers
+|
++-- 14 Scanners Parallèles (~95 règles)
 |   +-- AST Parse (acorn) — eval/Function, credential CLI theft, binary droppers, prototype hooks
 |   +-- Pattern Matching (shell, scripts)
 |   +-- Typosquat Detection (npm + PyPI, Levenshtein)
@@ -678,6 +692,12 @@ MUAD'DIB 2.2 Scanner
 |   +-- Décomposition Score (scoring explicable par règle)
 |   +-- API Threat Feed (serveur HTTP, flux JSON pour SIEM)
 |
++-- Réduction FP Post-traitement (v2.2.8-v2.2.9)
+|   +-- Downgrade sévérité par comptage (dynamic_require, dataflow, etc.)
+|   +-- Cap scoring prototype_hook MEDIUM
+|   +-- Obfuscation dans dist/build → LOW
+|   +-- Filtrage env vars safe + préfixes
+|
 +-- Paranoid Mode (ultra-strict)
 +-- Docker Sandbox (analyse comportementale, capture réseau, canary tokens, CI-aware)
 +-- Moniteur Zero-Day (interne : polling RSS npm + PyPI, alertes Discord, rapport quotidien)
@@ -696,14 +716,31 @@ Output (CLI, JSON, HTML, SARIF, Webhook, Threat Feed)
 
 ## Metriques d'evaluation
 
-| Version | TPR (Ground Truth) | FPR (Benign) | ADR (Adversarial) | Holdout (pre-tuning) | Date |
-|---------|-------------------|-------------|-------------------|---------------------|------|
-| 2.2.0   | 100.0% (4/4)      | 0.0% (0/98) | 100.0% (35/35)    | 30.0% (3/10)        | 2026-02-20 |
+| Metrique | Resultat | Details |
+|----------|----------|---------|
+| **TPR** (Ground Truth) | **100%** (4/4) | Attaques reelles : event-stream, ua-parser-js, coa, node-ipc |
+| **FPR** (Benign) | **17.5%** (92/527) | 529 packages npm, vrai code source via `npm pack`, seuil > 20 |
+| **ADR** (Adversarial) | **100%** (35/35) | 35 samples evasifs sur 4 vagues red team |
+| **Holdouts** (pre-tuning) | 40/40 pass | Tous les holdouts passent apres corrections |
+
+**Progression FPR** : 0% (invalide, dirs vides, v2.2.0-v2.2.6) → 38% (premiere vraie mesure, v2.2.7) → 19.4% (v2.2.8) → **17.5%** (v2.2.9)
+
+**Progression holdout** (scores pre-tuning, regles gelees) :
+
+| Holdout | Score | Focus |
+|---------|-------|-------|
+| v1 | 30% (3/10) | Patterns generaux |
+| v2 | 40% (4/10) | Env charcode, lifecycle, prototype |
+| v3 | 60% (6/10) | Require cache, DNS TXT, reverse shell |
+| v4 | **80%** (8/10) | Efficacite desobfuscation |
+| v5 | 50% (5/10) | Dataflow inter-module (nouveau scanner) |
+
+- **TPR** (True Positive Rate) : taux de detection sur 4 attaques supply-chain reelles (event-stream, ua-parser-js, coa, node-ipc)
+- **FPR** (False Positive Rate) : packages avec score > 20 sur 529 packages npm reels (code source scanne, pas des dirs vides)
+- **ADR** (Adversarial Detection Rate) : taux de detection sur 35 samples malveillants evasifs sur 4 vagues red team
+- **Holdout** (pre-tuning) : taux de detection sur 10 samples jamais vus avec regles gelees (mesure de generalisation)
 
-- **TPR** (True Positive Rate) : taux de detection sur 5 attaques supply-chain reelles (event-stream, ua-parser-js, coa, node-ipc, colors)
-- **FPR** (False Positive Rate) : packages avec score > 20 sur 98 packages npm populaires
-- **ADR** (Adversarial Detection Rate) : taux de detection sur 35 samples malveillants evasifs (4 vagues red team + holdout promu)
-- **Holdout** (pre-tuning) : taux de detection sur 10 samples jamais vus avant correction des regles (mesure de generalisation)
+Datasets : 529 npm + 132 PyPI packages benins, 35 samples adversariaux, 50 samples holdout (5 batches), 65 packages malveillants documentes.
 
 Voir [Evaluation Methodology](docs/EVALUATION_METHODOLOGY.md) pour le protocole experimental complet.
 
@@ -739,13 +776,13 @@ npm test
 
 ### Tests
 
-- **781 tests unitaires/intégration** sur 18 fichiers modulaires - 74% coverage via [Codecov](https://codecov.io/gh/DNSZLSK/muad-dib)
+- **822 tests unitaires/intégration** sur 20 fichiers modulaires - 74% coverage via [Codecov](https://codecov.io/gh/DNSZLSK/muad-dib)
 - **56 tests de fuzzing** - YAML malformé, JSON invalide, fichiers binaires, ReDoS, unicode, inputs 10MB
 - **35 samples adversariaux** - Packages malveillants évasifs, taux de détection 35/35 (100% ADR)
+- **50 samples holdout** - 5 batches de 10, scores pre-tuning : 30% → 40% → 60% → 80% → 50%
 - **8 tests multi-facteur typosquat** - Cas limites et comportement cache
 - **Validation ground truth** - 5/5 attaques réelles détectées (event-stream, ua-parser-js, coa, node-ipc, colors)
-- **Validation faux positifs** - 0/98 faux positifs sur packages npm populaires (0% FPR)
-- **Validation holdout** - 3/10 détection sur samples jamais vus avant correction (30% pre-tuning)
+- **Validation faux positifs** - 17.5% FPR (92/527) sur vrai code source npm via `npm pack` (mesure honnête)
 - **Audit ESLint sécurité** - `eslint-plugin-security` avec 14 règles activées
 
 ---

package/README.md

@@ -30,7 +30,7 @@
 
 npm and PyPI supply-chain attacks are exploding. Shai-Hulud compromised 25K+ repos in 2025. Existing tools detect threats but don't help you respond.
 
-MUAD'DIB combines static analysis + dynamic analysis (Docker sandbox) + **behavioral anomaly detection** (v2.0) + **ground truth validation** (v2.1) to detect threats AND guide your response — even before they appear in any IOC database.
+MUAD'DIB combines static analysis + **deobfuscation engine** (v2.2.5) + **inter-module dataflow** (v2.2.6) + dynamic analysis (Docker sandbox) + **behavioral anomaly detection** (v2.0) + **ground truth validation** (v2.1) to detect threats AND guide your response — even before they appear in any IOC database.
 
 ---
 
@@ -432,6 +432,10 @@ Detects malicious patterns in `.github/workflows/` YAML files, including Shai-Hu
 | Prototype hooking (fetch, XMLHttpRequest) | T1557 | AST |
 | Workflow injection (.github/workflows) | T1195.002 | AST |
 | Crypto wallet harvesting | T1005 | Dataflow |
+| Require cache poisoning | T1574.001 | AST |
+| Staged eval decode (eval+atob/Buffer) | T1140 | AST |
+| Deobfuscation (string concat, charcode, base64, hex) | T1140 | AST pre-processing |
+| Cross-file dataflow (inter-module exfiltration) | T1041 | Module graph |
 
 ---
 
@@ -643,7 +647,7 @@ Alerts appear in Security > Code scanning alerts.
 ## Architecture
 
 ```
-MUAD'DIB 2.2 Scanner
+MUAD'DIB 2.2.9 Scanner
 |
 +-- IOC Match (225,000+ packages, JSON DB)
 |   +-- OSV.dev npm dump (200K+ MAL-* entries)
@@ -655,7 +659,17 @@ MUAD'DIB 2.2 Scanner
 |   +-- Snyk Known Malware
 |   +-- Static IOCs (Socket, Phylum)
 |
-+-- 13 Parallel Scanners (86 rules)
++-- Deobfuscation Pre-processing (v2.2.5, --no-deobfuscate to disable)
+|   +-- String concat folding, CharCode reconstruction
+|   +-- Base64 decode, Hex array resolution
+|   +-- Const propagation (Phase 2)
+|
++-- Inter-module Dataflow (v2.2.6, --no-module-graph to disable)
+|   +-- Module dependency graph, tainted export annotation
+|   +-- 3-hop re-export chains, class method analysis
+|   +-- Cross-file credential read -> network sink detection
+|
++-- 14 Parallel Scanners (~95 rules)
 |   +-- AST Parse (acorn) — eval/Function, credential CLI theft, binary droppers, prototype hooks
 |   +-- Pattern Matching (shell, scripts)
 |   +-- Obfuscation Detection (skip .min.js, ignore hex/unicode alone)
@@ -681,6 +695,12 @@ MUAD'DIB 2.2 Scanner
 |   +-- Score Breakdown (explainable per-rule scoring)
 |   +-- Threat Feed API (HTTP server, JSON feed for SIEM)
 |
++-- FP Reduction Post-processing (v2.2.8-v2.2.9)
+|   +-- Count-based severity downgrade (dynamic_require, dataflow, etc.)
+|   +-- Framework prototype scoring cap
+|   +-- Obfuscation in dist/build → LOW
+|   +-- Safe env var + prefix filtering
+|
 +-- Paranoid Mode (ultra-strict)
 +-- Docker Sandbox (behavioral analysis, network capture, canary tokens, CI-aware)
 +-- Zero-Day Monitor (internal: npm + PyPI RSS polling, Discord alerts, daily report)
@@ -699,14 +719,31 @@ Output (CLI, JSON, HTML, SARIF, Webhook, Threat Feed)
 
 ## Evaluation Metrics
 
-| Version | TPR (Ground Truth) | FPR (Benign) | ADR (Adversarial) | Holdout (pre-tuning) | Date |
-|---------|-------------------|-------------|-------------------|---------------------|------|
-| 2.2.0   | 100.0% (4/4)      | 0.0% (0/98) | 100.0% (35/35)    | 30.0% (3/10)        | 2026-02-20 |
+| Metric | Result | Details |
+|--------|--------|---------|
+| **TPR** (Ground Truth) | **100%** (4/4) | Real-world attacks: event-stream, ua-parser-js, coa, node-ipc |
+| **FPR** (Benign) | **17.5%** (92/527) | 529 npm packages, real source code via `npm pack`, threshold > 20 |
+| **ADR** (Adversarial) | **100%** (35/35) | 35 evasive samples across 4 red-team waves |
+| **Holdouts** (pre-tuning) | 40/40 pass | All holdout samples pass after corrections |
+
+**FPR progression**: 0% (invalid, empty dirs, v2.2.0-v2.2.6) → 38% (first real measurement, v2.2.7) → 19.4% (v2.2.8) → **17.5%** (v2.2.9)
+
+**Holdout progression** (pre-tuning scores, rules frozen):
+
+| Holdout | Score | Focus |
+|---------|-------|-------|
+| v1 | 30% (3/10) | General patterns |
+| v2 | 40% (4/10) | Env charcode, lifecycle, prototype |
+| v3 | 60% (6/10) | Require cache, DNS TXT, reverse shell |
+| v4 | **80%** (8/10) | Deobfuscation effectiveness |
+| v5 | 50% (5/10) | Inter-module dataflow (new scanner) |
+
+- **TPR** (True Positive Rate): detection rate on 4 real-world supply-chain attacks (event-stream, ua-parser-js, coa, node-ipc)
+- **FPR** (False Positive Rate): packages scoring > 20 out of 529 real npm packages (source code scanned, not empty dirs)
+- **ADR** (Adversarial Detection Rate): detection rate on 35 evasive malicious samples across 4 red-team waves
+- **Holdout** (pre-tuning): detection rate on 10 unseen samples with rules frozen (measures generalization)
 
-- **TPR** (True Positive Rate): detection rate on 5 real-world supply-chain attacks (event-stream, ua-parser-js, coa, node-ipc, colors)
-- **FPR** (False Positive Rate): packages scoring > 20 out of 98 popular npm packages
-- **ADR** (Adversarial Detection Rate): detection rate on 35 evasive malicious samples across 4 red-team waves + promoted holdout
-- **Holdout** (pre-tuning): detection rate on 10 unseen samples before any rule correction (measures generalization)
+Datasets: 529 npm + 132 PyPI benign packages, 35 adversarial samples, 50 holdout samples (5 batches), 65 documented malware packages.
 
 See [Evaluation Methodology](docs/EVALUATION_METHODOLOGY.md) for the full experimental protocol.
 
@@ -742,13 +779,13 @@ npm test
 
 ### Testing
 
-- **781 unit/integration tests** across 18 modular test files - 74% code coverage via [Codecov](https://codecov.io/gh/DNSZLSK/muad-dib)
+- **822 unit/integration tests** across 20 modular test files - 74% code coverage via [Codecov](https://codecov.io/gh/DNSZLSK/muad-dib)
 - **56 fuzz tests** - Malformed YAML, invalid JSON, binary files, ReDoS, unicode, 10MB inputs
 - **35 adversarial samples** - Evasive malicious packages, 35/35 detection rate (100% ADR)
+- **50 holdout samples** - 5 batches of 10, pre-tuning scores: 30% → 40% → 60% → 80% → 50%
 - **8 multi-factor typosquat tests** - Edge cases and cache behavior
 - **Ground truth validation** - 5/5 real-world attacks detected (event-stream, ua-parser-js, coa, node-ipc, colors)
-- **False positive validation** - 0/98 false positives on popular npm packages (0% FPR)
-- **Holdout validation** - 3/10 detection on unseen samples before rule corrections (30% pre-tuning)
+- **False positive validation** - 17.5% FPR (92/527) on real npm source code via `npm pack` (honest measurement)
 - **ESLint security audit** - `eslint-plugin-security` with 14 rules enabled
 
 ---

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "muaddib-scanner",
-  "version": "2.2.8",
+  "version": "2.2.9",
   "description": "Supply-chain threat detection & response for npm & PyPI/Python",
   "main": "src/index.js",
   "bin": {

package/src/index.js

@@ -73,7 +73,9 @@ const MAX_FILE_SIZE = 10 * 1024 * 1024; // 10MB
 const FP_COUNT_THRESHOLDS = {
   dynamic_require: { maxCount: 10, from: 'HIGH', to: 'LOW' },
   dangerous_call_function: { maxCount: 5, from: 'MEDIUM', to: 'LOW' },
-  require_cache_poison: { maxCount: 3, from: 'CRITICAL', to: 'LOW' }
+  require_cache_poison: { maxCount: 3, from: 'CRITICAL', to: 'LOW' },
+  suspicious_dataflow: { maxCount: 5, to: 'LOW' },
+  obfuscation_detected: { maxCount: 3, to: 'LOW' }
 };
 
 // Custom class prototypes that HTTP frameworks legitimately extend.
@@ -94,7 +96,7 @@ function applyFPReductions(threats) {
     // Count-based downgrade: if a threat type appears too many times,
     // it's a framework/plugin system, not malware
     const rule = FP_COUNT_THRESHOLDS[t.type];
-    if (rule && typeCounts[t.type] > rule.maxCount && t.severity === rule.from) {
+    if (rule && typeCounts[t.type] > rule.maxCount && (!rule.from || t.severity === rule.from)) {
       t.severity = rule.to;
     }
 
@@ -638,10 +640,18 @@ async function run(targetPath, options = {}) {
   const mediumCount = deduped.filter(t => t.severity === 'MEDIUM').length;
   const lowCount = deduped.filter(t => t.severity === 'LOW').length;
 
+  // Cap MEDIUM prototype_hook contribution to 15 points max (5 × MEDIUM=3)
+  // Frameworks like Restify have 50+ prototype extensions that are not malicious
+  const mediumProtoHookCount = deduped.filter(t => t.type === 'prototype_hook' && t.severity === 'MEDIUM').length;
+  const PROTO_HOOK_MEDIUM_CAP = 15;
+  const protoHookPoints = Math.min(mediumProtoHookCount * SEVERITY_WEIGHTS.MEDIUM, PROTO_HOOK_MEDIUM_CAP);
+  const otherMediumCount = mediumCount - mediumProtoHookCount;
+
   let riskScore = 0;
   riskScore += criticalCount * SEVERITY_WEIGHTS.CRITICAL;
   riskScore += highCount * SEVERITY_WEIGHTS.HIGH;
-  riskScore += mediumCount * SEVERITY_WEIGHTS.MEDIUM;
+  riskScore += otherMediumCount * SEVERITY_WEIGHTS.MEDIUM;
+  riskScore += protoHookPoints;
   riskScore += lowCount * SEVERITY_WEIGHTS.LOW;
   riskScore = Math.min(MAX_RISK_SCORE, riskScore);
 

package/src/scanner/ast.js

@@ -31,9 +31,15 @@ const SENSITIVE_STRINGS = [
 // Env vars that are safe and should NOT be flagged (common config/runtime vars)
 const SAFE_ENV_VARS = [
   'NODE_ENV', 'PORT', 'HOST', 'HOSTNAME', 'PWD', 'HOME', 'PATH',
-  'LANG', 'TERM', 'CI', 'DEBUG', 'VERBOSE', 'LOG_LEVEL'
+  'LANG', 'TERM', 'CI', 'DEBUG', 'VERBOSE', 'LOG_LEVEL',
+  'SHELL', 'USER', 'LOGNAME', 'EDITOR', 'TZ',
+  'NODE_DEBUG', 'NODE_PATH', 'NODE_OPTIONS',
+  'DISPLAY', 'COLORTERM', 'FORCE_COLOR', 'NO_COLOR', 'TERM_PROGRAM'
 ];
 
+// Env var prefixes that are safe (npm metadata, locale settings)
+const SAFE_ENV_PREFIXES = ['npm_config_', 'npm_lifecycle_', 'npm_package_', 'lc_'];
+
 // Env var keywords to detect sensitive environment access (separate from SENSITIVE_STRINGS)
 const ENV_SENSITIVE_KEYWORDS = [
   'TOKEN', 'SECRET', 'KEY', 'PASSWORD', 'CREDENTIAL', 'AUTH'
@@ -914,6 +920,11 @@ function analyzeFile(content, filePath, basePath) {
           if (SAFE_ENV_VARS.includes(envVar)) {
             return;
           }
+          // Skip safe prefixes (npm_config_*, npm_lifecycle_*, npm_package_*, LC_*)
+          const envLower = envVar.toLowerCase();
+          if (SAFE_ENV_PREFIXES.some(p => envLower.startsWith(p))) {
+            return;
+          }
           // Flag only vars containing sensitive keywords
           if (ENV_SENSITIVE_KEYWORDS.some(s => envVar.toUpperCase().includes(s))) {
             threats.push({

package/src/scanner/obfuscation.js

@@ -29,7 +29,12 @@ function detectObfuscation(targetPath) {
 
     const signals = [];
     let score = 0;
-    const isMinified = path.basename(file).endsWith('.min.js');
+    const basename = path.basename(file);
+    const isMinified = basename.endsWith('.min.js');
+    const isBundled = basename.endsWith('.bundle.js');
+    const pathParts = relativePath.split(path.sep);
+    const isInDistOrBuild = pathParts.some(p => p === 'dist' || p === 'build');
+    const isPackageOutput = isMinified || isBundled || isInDistOrBuild;
 
     // 1. Ratio code sur une seule ligne (skip .min.js — minification, not obfuscation)
     if (!isMinified) {
@@ -86,7 +91,7 @@ function detectObfuscation(targetPath) {
     if (score >= 40) {
       threats.push({
         type: 'obfuscation_detected',
-        severity: score >= 70 ? 'CRITICAL' : 'HIGH',
+        severity: isPackageOutput ? 'LOW' : (score >= 70 ? 'CRITICAL' : 'HIGH'),
         message: `Code obfusque (score: ${score}). Signaux: ${signals.join(', ')}`,
         file: relativePath
       });