progmune-runtime 3.7.30 → 3.7.31

package/CHANGELOG.md

@@ -1,5 +1,29 @@
 # Changelog
 
+## [3.7.31] — 2026-09-18
+
+### 提取器 SSRF 标记正则死循环修复（P0，真实工程可用性故障）
+
+- **现象**：IR 提取阶段对含 `fetch(` 的 TS 文件 100% CPU 挂死——一个 55 行的切片 >480s 未完成，而 108 KB / 5 文件的语料只需 35s。实测 **nuxt-modules/og-image、tinacms/tinacms、zereight/gitlab-mcp 三个独立仓库全部触发**
+- **定位**：对挂住进程做 CPU 采样（`sample`），**2193/2193 采样全部落在 `Builtins_RegExpPrototypeExec`**——确定为正则死循环，而非此前怀疑的 OOM / ts-morph 依赖解析
+- **根因**（`src/extract-ir.ts` SSRF 分支，3.7.28 引入）：`TS_HTTP_FETCH_SINK` 是**无 `g` 标志**的正则，却用于 `while ((m = TS_HTTP_FETCH_SINK.exec(text)) !== null)` 迭代。非全局正则的 `exec()` 忽略 `lastIndex` 并恒返回首个匹配；当首个 `fetch(` 后的 300 字符窗口不含污点时（真实代码里最常见的「良性 fetch」形态），循环无出口。同文件另 7 处 `while-exec` 循环均带 `g`，仅此一处遗漏
+- **修复**：新增 `tsHttpFetchSinkIter()`（同模式 `"g"` 版）专供迭代；布尔探测仍用无标志版本（带 `g` 的 `.test()` 会污染 `lastIndex`）。模式至少匹配 5 字符，不可能零宽匹配
+- **影响面**：任何含 `fetch(` 且函数体内无 SSRF 守卫词汇（`localhost|ssrf|hostname|…`）的 TS 工程都会挂死提取阶段——这是面向真实代码的普遍故障，不是基准内部的边缘情况
+- **验收**：og-image 480s+ → **48s**；tinacms 300s+ → **57s**；gitlab-mcp 切片 480s+ → **80s**；最小复现 70s+ → **18s**
+
+### 修复回归语料扩样至 17 条（fr-012 ~ fr-017）
+
+- **样本结构**：open-webui 占比 75% → **41%**，独立仓库 3 → **10**（gitlab-mcp ×2、mockoon、og-image、redocly-cli、tinacms）
+- **结果**：6 条新语料全部 MISS（pre/post 均 0 违规），全库 **DETECTED 4/17、MISS 13/17、PENDING 0**
+- **正向对照**：为排除「0 违规是哑值」，scan `demo-project/` 报出 1 条真实违规（`SSG_AUTH_STATE_VIOLATION`，`token_lifecycle_flow.ts::main`）——引擎是活的
+- **注册数据修正**：fr-013 的 parent SHA 末 4 位转置（codeload 404）且原 fix_commit 是纯 CHANGELOG 文档提交，经 PR #624 正文 link 定位到真修复 PR #571；fr-012 锚点错位（其「文件读取」半边由更早的 PR #482 修复）
+- **新根因类别**：fr-012（MCP 工具实参 `args.file_path`）、fr-015（`decodeURIComponent` 解码后的对象字段 `font.path`）暴露**污点源词汇表缺口**——与 skyvern/open-webui 已暴露的认证词表缺口同族，共同病灶是「标记管线靠名字识源，不靠数据流语义」
+- 详见 `blind-benchmark/REALWORLD_FIX_REGRESSION_V2.md`
+
+### 回归
+
+- 新增 `src/extract-ir-ssrf-loop.test.ts` 2 green ✓（①良性 fetch 必须正常返回——修复前永不返回；②URL 形参流入 fetch 仍必须注入 `__progmune_ssrf_user_url__`）/ build ✓
+
 ## [3.7.30] — 2026-09-17
 
 ### 提取器性能重构（3.7.28 引入的回归修复）

package/README.md

@@ -109,15 +109,18 @@ Progmune is honest about what it can and cannot verify.
 | **Python** | ✅ Production | Blind benchmark: **recall 100% / precision 100%** (729 gold findings, 90 projects); real-world validation: PyGoat (OWASP vulnerable-by-design Django app) **67 TP / 0 FP, 100% labeled precision**; three well-written apps (django/fastapi realworld, django-unicorn) with 0 false-positive true findings |
 | **C** | ✅ Annotation-driven (Beta) | IR extraction via the registry + SSG state machine; **annotate ~2-3 protocol primitives per protocol to get trusted verification** (gold 5/5 on real modules: redis ACL / libssh client / libssh server / libssh callback-dispatch / uftpd transfer-auth — all 0 FP with precise violation localization; app-level gold v2: **P=91.7% / R=100% / F1=95.7%**). Unannotated auto-detection stays out of scope (0 TP on real corpus — positioning decision recorded in the C Language Status doc); TLS-level coverage still absent. See [C Language Status](https://github.com/shenlian19831109/progmune-runtime/blob/main/docs/c-language-status.md). |
 | **Go** | ✅ Annotation-driven (Beta) | IR extraction via registry + SSG state machine; synthetic gold v1: **P=100% / R=100%** (clean×3 + violations×3); zero external toolchain (pure-TS lexical extractor — works from npm installs) |
-| **Java** | ❌ None | Planned |
+| **Java** | 🔶 Annotation-driven (Beta) | Two layers, verified on real corpora. **Framework layer ✅**: Spring adapter (`spring-detector.ts` — security rule ordering + ant patterns + controller annotation routes) on gothinkster/spring-boot-realworld (1581★): 19 routes / 12 mutations fully parsed, 0 issues, catch-all flip counter-proof ×10; Spring Boot 3 modern dialect re-verified (SecurityFilterChain bean + String[] allowlist expansion): 15 routes, 0 issues. **Core layer**: pure-TS lexical extractor (zero toolchain deps) at **100%/100% recovery vs a tree-sitter AST baseline (1216/1216)**; SSG annotation-driven protocol rows (token / auth-register / resource) engine-locked with a real-corpus closed loop (**1 real TP / 0 FP**). Unannotated auto-detection stays out of scope (same decision as C/Go). Production label pending enterprise PoC. See [Java Language Status](https://github.com/shenlian19831109/progmune-runtime/blob/main/docs/java-language-status.md). |
 
 **Multi-language IR (registry-based).** TypeScript (ts-morph) and Python (AST) extractors are registered entries in one registry (`src/extract-project-ir.ts`); `extractProjectIR` merges every detected language into a single function IR shared by the agent loop, `execute()`, and the MCP server — the agent composes function-protocol chains across both languages. Adding a language (Go, Java, …) = register one extractor entry; callers don't change.
 
-**Framework adapters: 12 dedicated detectors (4 structural + 8 heuristic).** Express ✅, tRPC ✅, FastAPI ✅, Django ✅ (structural route-level auth analysis — every mutation route checked for auth), Flask ✅ (route/before_request auth guard analysis), Fastify ✅ (route preHandler/hook auth analysis) and Next.js ✅ (App Router route-handler auth analysis), NestJS ✅ (decorator route parsing + global APP_GUARD recognition + @Public exemption), Koa ✅, Hapi ✅ (route-config auth strategy analysis), Gin ✅ and Fiber ✅ (Go route/auth-middleware analysis) have dedicated detectors. Next.js version-aware governance also exists. Spring Boot/Go frameworks and a few TS frameworks remain — framework adaptation is the #1 product gap.
+**Framework adapters: 13 dedicated detectors — 4 structural (AST-based) + 8 heuristic (code-string) + Spring Boot (security rule-ordering model); 13/13 now validated on real corpora.** Express ✅, tRPC ✅, FastAPI ✅, Django ✅ (structural route-level auth analysis — every mutation route checked for auth), Flask ✅ (route/before_request auth guard analysis), Fastify ✅ (route preHandler/hook auth analysis) and Next.js ✅ (App Router route-handler auth analysis), NestJS ✅ (decorator route parsing + global APP_GUARD recognition + @Public exemption), Koa ✅, Hapi ✅ (route-config auth strategy analysis), Gin ✅ and Fiber ✅ (Go route/auth-middleware analysis) and Spring Boot ✅ (Spring Security rule ordering; Boot 2 + Boot 3 dialects) have dedicated detectors. Next.js version-aware governance also exists.
+
+**Evidence tiers** (adopted 2026-09-02): 🟡 synthetic → ✅ **real-corpus validated** (≥1 real production corpus at 0 protocol-level FP + guard-removal counter-proof) → 🏭 production (enterprise PoC/adoption — none yet). **All 13/13 are ✅.** Documented boundaries: Express `main.ts` retains a NO_HELMET hardening gap (not protocol-level); the NestJS corpus yields 1 real TP (`DELETE /users/:slug`); jiotv_go's 2 flags are capability-token shapes; gofiber recipes flags are demo-context noise (no auth intent). Remaining gap: a few TS frameworks and Go frameworks beyond Gin/Fiber.
 
 ### What Progmune does NOT cover (honest boundaries)
 
-- **TS-side taint-based injection flaws** — the source-level SQLi/XSS/SSRF detections ship for Python; the TypeScript extractor is name/call-based, so TS injection classes remain uncovered (documented, not hidden).
+- **TS-side taint-based injection flaws — partial.** Python ships source-level SQLi/XSS/SSRF. On the TypeScript side, **path traversal (3.7.26)** and **SSRF (3.7.28)** are covered: the extractor emits markers (request/param taint → fs or HTTP sink, one hop + one cross-function hop) and the engine consumes them at the IR layer — validated against real security fixes at their ground-truth sites (fr-007 openhop, fr-011 mcp-from-openapi). **SQLi and XSS remain uncovered**: the TS extractor is name/call-based, not a dataflow taint engine, so those classes stay out of scope (documented, not hidden).
+- **Missing-check (absence) vulnerabilities** — the dominant real-world fix shape. A protocol state machine proves "these calls happened in the wrong order"; it does not prove "a required check was never called." Authorisation-policy bypasses, allowlist-path omissions, and ownership checks missing on one route are the largest uncovered class — measured, not guessed: see the [real-fix regression corpus](https://github.com/shenlian19831109/progmune-runtime/blob/main/blind-benchmark/fix-regression-corpus.json).
 - **SCA / dependency vulnerabilities** — hallucinated package names, supply-chain issues. Separate tooling exists for this.
 - **Runtime behavior** — Progmune is static analysis only; no DAST/sandbox execution.
 - **Framework internals** — well-known framework dispatch/cache machinery (e.g. django-unicorn internals) can produce a small number of boundary false positives; they are documented per-corpus in the benchmark gold files.

package/dist/extract-ir-ssrf-loop.test.js

@@ -0,0 +1,123 @@
+"use strict";
+/**
+ * SSRF 标记提取的死循环回归（2026-09-18，P0）。
+ *
+ * 根因：computeMarkerCalls 的 SSRF 分支对一个**无 `g` 标志**的正则
+ * TS_HTTP_FETCH_SINK 做 while-exec 迭代。非全局正则的 exec() 忽略
+ * lastIndex 并恒返回首个匹配，因此只要首个 fetch 调用后的 300 字符窗口
+ * 内不含污点——真实代码里最常见的「良性 fetch」形态——循环就没有出口，
+ * 提取阶段 100% CPU 死循环。CPU 采样佐证：2193/2193 采样全部落在
+ * Builtins_RegExpPrototypeExec。
+ *
+ * 影响面：任何含 `fetch(` 且函数体内无 SSRF 守卫词汇的 TS 工程都会挂死
+ * 提取阶段（实测 nuxt og-image、tinacms、gitlab-mcp 三个独立仓库均触发）。
+ *
+ * 本文件锁定两端：
+ *   1. 良性 fetch 必须正常返回（修复前永不返回）；
+ *   2. 真正的 SSRF 污点仍必须注入 __progmune_ssrf_user_url__（修复不得削弱检出）。
+ */
+var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    var desc = Object.getOwnPropertyDescriptor(m, k);
+    if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
+      desc = { enumerable: true, get: function() { return m[k]; } };
+    }
+    Object.defineProperty(o, k2, desc);
+}) : (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    o[k2] = m[k];
+}));
+var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
+    Object.defineProperty(o, "default", { enumerable: true, value: v });
+}) : function(o, v) {
+    o["default"] = v;
+});
+var __importStar = (this && this.__importStar) || (function () {
+    var ownKeys = function(o) {
+        ownKeys = Object.getOwnPropertyNames || function (o) {
+            var ar = [];
+            for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
+            return ar;
+        };
+        return ownKeys(o);
+    };
+    return function (mod) {
+        if (mod && mod.__esModule) return mod;
+        var result = {};
+        if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
+        __setModuleDefault(result, mod);
+        return result;
+    };
+})();
+Object.defineProperty(exports, "__esModule", { value: true });
+const vitest_1 = require("vitest");
+const fs = __importStar(require("fs"));
+const os = __importStar(require("os"));
+const path = __importStar(require("path"));
+const extract_ir_1 = require("./extract-ir");
+const TSCONFIG = JSON.stringify({
+    compilerOptions: {
+        target: "ES2020",
+        module: "commonjs",
+        moduleResolution: "node",
+        strict: false,
+        skipLibCheck: true,
+        noEmit: true,
+    },
+    include: ["**/*.ts"],
+});
+function makeProject(files) {
+    const dir = fs.mkdtempSync(path.join(os.tmpdir(), "pm-ssrf-loop-"));
+    fs.writeFileSync(path.join(dir, "tsconfig.json"), TSCONFIG);
+    fs.writeFileSync(path.join(dir, "package.json"), JSON.stringify({ name: "fixture", version: "0.0.0", private: true }));
+    for (const [rel, body] of Object.entries(files)) {
+        const abs = path.join(dir, rel);
+        fs.mkdirSync(path.dirname(abs), { recursive: true });
+        fs.writeFileSync(abs, body);
+    }
+    return dir;
+}
+(0, vitest_1.describe)("SSRF 标记提取：良性 fetch 不得使提取器死循环", () => {
+    (0, vitest_1.it)("无守卫词汇、无污点的 fetch 调用应正常完成提取", () => {
+        const dir = makeProject({
+            "api.ts": `
+export async function ping(): Promise<any> {
+  const response = await fetch("https://x.example/uploads", { method: "POST" });
+  return response.json();
+}
+
+export async function pong(): Promise<any> {
+  return fetch("https://x.example/other");
+}
+`,
+        });
+        try {
+            const fns = (0, extract_ir_1.extractIR)(dir);
+            const names = fns.map((f) => f.name);
+            (0, vitest_1.expect)(names).toContain("ping");
+            (0, vitest_1.expect)(names).toContain("pong");
+        }
+        finally {
+            fs.rmSync(dir, { recursive: true, force: true });
+        }
+    }, 150000);
+    (0, vitest_1.it)("URL 形参直接流入 fetch 仍应注入 __progmune_ssrf_user_url__", () => {
+        const dir = makeProject({
+            "proxy.ts": `
+export async function proxy(target: string): Promise<any> {
+  const response = await fetch(target);
+  return response;
+}
+`,
+        });
+        try {
+            const fns = (0, extract_ir_1.extractIR)(dir);
+            const proxy = fns.find((f) => f.name === "proxy");
+            (0, vitest_1.expect)(proxy).toBeDefined();
+            (0, vitest_1.expect)(proxy.calls ?? []).toContain("__progmune_ssrf_user_url__");
+        }
+        finally {
+            fs.rmSync(dir, { recursive: true, force: true });
+        }
+    }, 150000);
+});

package/dist/extract-ir.js

@@ -516,6 +516,19 @@ function methodSinkParamMap(project, absRoot) {
 // Web 形态（req.params）共用同一标记；引擎 IR 层消费。
 // ═══════════════════════════════════════════════════════════════
 const TS_HTTP_FETCH_SINK = /\b(fetch|axios\.(?:get|post|put|delete|head|patch)|http\.request|https\.request|ky\.(?:get|post|put|delete|head|patch)|undici\.request|nodeFetch|got)\s*\(/;
+/**
+ * 同模式的**全局**版本，仅用于 while-exec 迭代。
+ *
+ * 2026-09-18 P0 修复：此前 while 循环直接对无 `g` 标志的 TS_HTTP_FETCH_SINK
+ * 反复调用 exec()。非全局正则的 exec() 会忽略 lastIndex 并恒返回首个匹配，
+ * 一旦首个匹配的 300 字符窗口内不含污点（真实代码里最常见的良性 fetch
+ * 形态），循环无出口 → 提取器 100% CPU 死循环。CPU 采样证实：
+ * 2193/2193 采样全部落在 Builtins_RegExpPrototypeExec。
+ * 该缺陷会使任何含 `fetch(` 且无 SSRF 守卫词汇的 TS 工程挂死提取阶段。
+ */
+function tsHttpFetchSinkIter() {
+    return new RegExp(TS_HTTP_FETCH_SINK.source, "g");
+}
 const SSRF_GUARD_EVIDENCE = /127\.0\.0\.1|0\.0\.0\.0|169\.254\.|::1|localhost|isPrivate|isLoopback|hostname|denylist|blocklist|ssrf|validateUrl|isSafeUrl|getAddresses|ipaddress|forbidden_host|private_ip/i;
 const URL_PARAM_NAME = /^(url|target|endpoint|link|href|webUrl|web_url|sourceUrl|source_url|remoteUrl|remote_url|fetchUrl|fetch_url|specUrl|spec_url|apiUrl|api_url|origin|baseUrl|base_url)$/i;
 function collectUrlParamNames(params) {
@@ -569,9 +582,9 @@ function computeMarkerCalls(text, paramNames, sinkParams, onMethodHit) {
             taintParts.push(`\\b(?:${[...reqTainted].join("|")})\\b`);
         taintParts.push(/\b(?:req|request)\.(?:params|query|body|headers|cookies)\b[.[]?/.source);
         const taint = new RegExp(taintParts.join("|"));
-        TS_HTTP_FETCH_SINK.lastIndex = 0;
+        const fetchSinkIter = tsHttpFetchSinkIter();
         let m;
-        while ((m = TS_HTTP_FETCH_SINK.exec(text)) !== null) {
+        while ((m = fetchSinkIter.exec(text)) !== null) {
             const after = text.slice(m.index + m[0].length, m.index + m[0].length + 300);
             if (taint.test(after)) {
                 markers.push("__progmune_ssrf_user_url__");

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "progmune-runtime",
-  "version": "3.7.30",
+  "version": "3.7.31",
   "description": "Progmune — AI Trust Decision Engine. Verify AI-generated code before it reaches production. Outputs APPROVED / NEEDS_REVIEW / BLOCKED with evidence.",
   "files": [
     "dist/",