ai-spec-dev 0.35.0 → 0.37.0

package/RELEASE_LOG.md

@@ -5,6 +5,145 @@
 
 ---
 
+## v0.37.0 — 2026-04-02 — P1 测试覆盖：Mock Server / Types Generator / VCR
+
+### 新增测试（Phase 1 收尾）
+
+**Test 7 — `mock-server-generator.test.ts`（28 tests）**
+
+覆盖 `generateMockAssets`（Express server.js 生成、README.md 端点表格、auth 中间件条件生成、DELETE 204 sendStatus、错误模拟注释、自定义端口、自定义输出目录、列表端点分页 fixture、MSW handlers/browser 生成、proxy 配置生成）、前端框架检测（Vite/Next.js/CRA/Webpack）、fixture 启发式（email/boolean/DateTime 字段类型）、`findLatestDslFile`（不存在目录、无匹配文件、最新文件选择、嵌套目录扫描）、`applyMockProxy`/`restoreMockProxy`（Vite 配置写入+dev:mock 脚本注入+还原、CRA proxy 字段注入+还原、Next.js 手动提示、无 lock 文件容错）。
+
+**Test 8 — `types-generator.test.ts`（28 tests）**
+
+覆盖类型映射（String→string、Int/Float→number、Boolean→boolean、DateTime→string、Json→Record、数组类型、PascalCase 模型引用保留、未知类型回退 string、nullable 标记剥离）、模型接口渲染（export interface、必填/可选字段、模型描述 JSDoc、字段描述 JSDoc）、端点类型（请求体接口、查询参数可选接口、路径参数接口、includeEndpointTypes 开关、无 schema 端点跳过）、端点常量表（API_ENDPOINTS const、method/path/auth 字段、ApiEndpointKey 类型、includeEndpointMap 开关）、自定义 header、前端组件 Props 接口、`saveTypescriptTypes`（默认路径写入、自定义路径写入）。
+
+**Test 9 — `vcr.test.ts`（22 tests）**
+
+覆盖 `VcrRecordingProvider`（透传 generate 调用、元数据记录含 callHash/promptPreview/duration/provider/model、providerName/modelName 代理、无 systemInstruction 时省略字段、promptPreview 截断 200 字符、保存至 .ai-spec-vcr 目录、双 recorder 合并按时间排序+重建索引、多 provider 记录）、`VcrReplayProvider`（按序回放、providerName=vcr-replay、modelName=runId、remaining/consumed 计数、exhausted 抛错、忽略 prompt 内容纯按索引回放）、`loadVcrRecording`（正常加载、不存在返回 null、损坏 JSON 返回 null）、`listVcrRecordings`（不存在目录返回空、逆序排列、跳过损坏文件、忽略非 JSON 文件、summary 字段正确性）。
+
+**测试覆盖率提升：37.5% → 45%（15 → 18 个模块有测试，331 → 409 test cases）**
+
+---
+
+## v0.36.1 — 2026-04-02 — P0 测试覆盖 + 质量硬门禁 + 错误体验优化
+
+### 新增测试（Week 2）
+
+**Test 4 — `context-loader.test.ts`（19 tests）**
+
+覆盖 `isFrontendDeps`（React/Vue/Next/Nuxt/Svelte/纯后端/空数组）、`ContextLoader` 类（Node.js/PHP/Java 三种项目类型的上下文加载、Prisma schema 读取、宪法加载、API 结构扫描、共享配置文件发现、错误模式提取、空目录容错）。
+
+**Test 5 — `openapi-exporter.test.ts`（27 tests）**
+
+覆盖 `dslToOpenApi`（结构完整性、info 元数据、自定义 server URL、路径参数标准化 `:id`→`{id}`、请求体生成、错误响应、认证端点 401 自动注入、安全方案生成、模型 schema 映射、必填字段标记、204 无内容响应、无认证场景）、类型映射（String/Int/Float/Boolean/DateTime/email/password/$ref）、`exportOpenApi`（YAML/JSON 格式、自定义输出路径、自定义 server URL）。
+
+**Test 6 — `spec-versioning.test.ts`（26 tests）**
+
+覆盖 `slugify`（英文转换、特殊字符、CJK 处理、长度限制、空输入回退、连字符折叠）、`computeDiff`（相同/新增/删除/修改/空文本/大文件回退/行类型正确性）、`findLatestVersion`（不存在目录、无匹配文件、单版本、多版本最新、不同 slug 隔离、正则特殊字符）、`nextVersionPath`（无版本/有 v1/跳跃版本号）。
+
+**测试覆盖率提升：30% → 37.5%（12 → 15 个模块有测试，259 → 331 test cases）**
+
+### 质量硬门禁（Week 3）
+
+**Feature 1 — Harness Score 阻断门禁（`cli/commands/create.ts`、`cli/utils.ts`）**
+
+- 新增 `minHarnessScore` 配置项（`.ai-spec.json`，默认 0 = 禁用）
+- 自评阶段（Step 10）后，当 `harnessScore < minHarnessScore` 且未使用 `--force` 时，打印阈值提示并 `exit(1)`
+- 与 `minSpecScore` 同样支持 `--force` 绕过
+
+**Feature 2 — Error Feedback 轮次可配置（`cli/commands/create.ts`、`cli/utils.ts`）**
+
+- 新增 `maxErrorCycles` 配置项（默认 2，TDD 模式默认 3，范围 1-10）
+- 替换原来硬编码的 `maxCycles: opts.tdd ? 3 : 2`，读取 `config.maxErrorCycles`
+
+**Feature 3 — Config 命令增强（`cli/commands/config.ts`）**
+
+- 新增 `--min-harness-score <score>` 和 `--max-error-cycles <n>` CLI 选项
+- 含输入范围校验（0-10 / 1-10）
+
+### 错误体验优化（Week 4）
+
+**Enhancement 1 — Provider 错误消息增强（`core/provider-utils.ts`）**
+
+- **Auth 错误（401/403）**：提示检查 API key 有效性 + 运行 `ai-spec model` 重新配置
+- **Rate Limit（429）**：提示等待或切换 provider + 检查计费面板
+- **网络错误**：提示检查连接和代理设置
+- **模型不存在**：提示运行 `ai-spec model` 查看可用模型
+- **余额/配额不足**：提示检查计费面板 + 切换 provider
+
+**Enhancement 2 — DSL 提取失败诊断增强（`core/dsl-extractor.ts`）**
+
+- JSON 解析失败时，输出 AI 原始响应前 500 字符的预览，方便判断是 prompt 问题还是 model 能力问题
+- Spec 超过 12K 字符截断时，**立即**打印黄色警告（而非静默截断），提醒用户详情可能丢失
+
+**Enhancement 3 — Key Store 读取容错（`core/key-store.ts`）**
+
+- 读取损坏的 key store 文件时，输出具体错误消息（而非静默忽略）
+
+---
+
+## v0.36.0 — 2026-04-01 — 安全修复 + 核心模块测试覆盖
+
+### 安全修复
+
+**Fix 1 — Shell 命令注入防护（`core/code-generator.ts`）**
+
+`execSync` 拼接 shell 字符串传递 prompt 内容时，仅转义了 `"` 字符，未处理 `$`、`;`、`|`、`&` 等 shell 元字符，存在命令注入风险。
+
+- 将 `execSync(\`\${claudeCmd} -p "..."\`)` 替换为 `spawnSync(claudeCmd, ["-p", promptContent], { shell: false })`（共 2 处）
+- `spawnSync` 数组形式绕过 shell 解析，彻底消除注入可能
+- 新增 `spawnSync` 导入（`child_process`）
+
+**Fix 2 — API Key 存储权限时序（`core/key-store.ts`）**
+
+原来先 `writeJson()` 再 `chmod(0o600)`，在写入与权限设置之间存在短暂窗口期，其他进程可能读取到明文 key。
+
+- 改为 `ensureFile()` → `chmod(0o600)` → `writeJson()` 顺序，确保文件权限在写入敏感数据前就已设置
+
+### 新增测试
+
+**Test 1 — `spec-generator.test.ts`（23 tests）**
+
+覆盖 `PROVIDER_CATALOG` 结构完整性、`createProvider` 工厂函数（9 个 provider 分支 + 自定义 model + 未知 provider 异常）、`SpecGenerator` prompt 构建逻辑（architecture decision 注入、constitution 优先级、context 截断限制）。
+
+**Test 2 — `reviewer.test.ts`（19 tests）**
+
+覆盖 `extractComplianceScore`（整数/小数/大小写/空字符串/多行/多匹配）、`extractMissingCount`（正常/大小写/缺失/多行）、`CodeReviewer` 类（空 diff 处理、多 Pass 调用验证、缺失文件容错、大文件截断、历史趋势渲染）。
+
+**Test 3 — `code-generator.test.ts`（23 tests）**
+
+覆盖 `extractBehavioralContract`（interface/enum/type/function/const/class/abstract class/defineStore/return 块/export default/嵌套大括号/throw 捕获上限/无 export 回退）、`printTaskProgress`（百分比计算/run 模式/skip 模式/0 total/未知 layer）。
+
+**测试覆盖率提升：22.5% → 30%（9 → 12 个模块有测试，251 → 259 test cases）**
+
+- `extractBehavioralContract` 从 private 改为 `export`（`core/code-generator.ts`），支持直接单元测试
+
+### DSL 验证增强
+
+**Fix 3 — Endpoint ID 唯一性检查（`core/dsl-validator.ts`）**
+
+AI 经常生成重复的 Endpoint ID（如两个 `EP-001`），导致下游 DSL 消费方（types-generator、mock-server 等）产生覆盖冲突。
+
+- 在 endpoints 验证阶段新增 `Set<string>` 去重检查，重复 ID 报告具体位置（`endpoints[N].id`）
+- 新增 4 个测试用例（唯一 ID 通过、重复 ID 拒绝、路径定位正确、多组重复检测）
+
+**Fix 4 — Model 字段名唯一性检查（`core/dsl-validator.ts`）**
+
+同一 Model 内出现重复字段名（如两个 `id`）会导致 Prisma schema 或 TypeScript interface 生成冲突。
+
+- 在 `validateModel` 内新增 `Set<string>` 去重检查，同一 model 内重复字段报告具体位置
+- 不同 model 之间允许同名字段（如 `User.id` 和 `Post.id`）
+- 新增 4 个测试用例
+
+**Fix 5 — `missing_errors` 误报修复（`core/dsl-feedback.ts`）**
+
+原来的逻辑：只要有任何 endpoint 缺少 errors 且总 endpoint ≥ 2 就标记 gap。这导致当部分 endpoint 已有 errors 时仍然误报。
+
+- 修改为：仅当 **所有** endpoint 都缺少 errors 时才标记 `missing_errors` gap
+- 修复了 `dsl-feedback.test.ts` 中已有的失败测试
+
+---
+
 ## [Unreleased] 2026-04-01 — P1 Task 验证步骤 + P2 设计方案对话
 
 ### 新增 / 增强

package/cli/commands/config.ts

@@ -16,6 +16,8 @@ export function registerConfig(program: Command): void {
     .option("--codegen-provider <name>", "Default provider for code generation")
     .option("--codegen-model <name>", "Default model for code generation")
     .option("--min-spec-score <score>", "Minimum overall spec score (1-10) to pass Approval Gate (0 = disabled)")
+    .option("--min-harness-score <score>", "Minimum harness score (1-10) for pipeline success (0 = disabled)")
+    .option("--max-error-cycles <n>", "Maximum error-feedback fix cycles (1-10, default: 2)")
     .option("--show", "Print current configuration")
     .option("--reset", "Reset configuration to empty")
     .option("--clear-keys", "Delete all saved API keys from ~/.ai-spec-keys.json")
@@ -85,6 +87,22 @@ export function registerConfig(program: Command): void {
         }
         updated.minSpecScore = score;
       }
+      if (opts.minHarnessScore !== undefined) {
+        const score = parseInt(opts.minHarnessScore, 10);
+        if (isNaN(score) || score < 0 || score > 10) {
+          console.error(chalk.red("  --min-harness-score must be a number between 0 and 10"));
+          process.exit(1);
+        }
+        updated.minHarnessScore = score;
+      }
+      if (opts.maxErrorCycles !== undefined) {
+        const cycles = parseInt(opts.maxErrorCycles, 10);
+        if (isNaN(cycles) || cycles < 1 || cycles > 10) {
+          console.error(chalk.red("  --max-error-cycles must be a number between 1 and 10"));
+          process.exit(1);
+        }
+        updated.maxErrorCycles = cycles;
+      }
 
       await fs.writeJson(configPath, updated, { spaces: 2 });
       console.log(chalk.green(`✔ Config saved to ${configPath}`));

package/cli/commands/create.ts

@@ -1076,8 +1076,10 @@ export function registerCreate(program: Command): void {
           console.log(chalk.cyan("[8/9] TDD mode — error feedback loop driving implementation to pass tests..."));
         }
         runLogger.stageStart("error_feedback");
+        const defaultCycles = opts.tdd ? 3 : 2;
+        const maxCycles = config.maxErrorCycles ?? defaultCycles;
         compilePassed = await runErrorFeedback(codegenProvider, workingDir, extractedDsl, {
-          maxCycles: opts.tdd ? 3 : 2,
+          maxCycles,
         });
         runLogger.stageEnd("error_feedback");
       }
@@ -1203,6 +1205,19 @@ export function registerCreate(program: Command): void {
       });
       printSelfEval(selfEvalResult);
 
+      // ── Harness Score Gate ─────────────────────────────────────────────────
+      const minHarness = config.minHarnessScore ?? 0;
+      if (minHarness > 0 && selfEvalResult.harnessScore < minHarness && !opts.force) {
+        console.log(chalk.red(
+          `\n  ✘ Harness score ${selfEvalResult.harnessScore}/10 is below the minimum threshold ${minHarness}/10.`
+        ));
+        console.log(chalk.gray(`  Gate threshold set in .ai-spec.json → "minHarnessScore": ${minHarness}`));
+        console.log(chalk.gray(`  Use --force to bypass, or improve the spec and re-run.`));
+        runLogger.stageEnd("self_eval", { gateBlocked: true, score: selfEvalResult.harnessScore, threshold: minHarness });
+        runLogger.finish();
+        process.exit(1);
+      }
+
       // ── Await async §9 accumulation (fire-and-await pattern) ────────────────
       if (accumulatePromise) await accumulatePromise;
 

package/cli/utils.ts

@@ -16,6 +16,10 @@ export interface AiSpecConfig {
   codegenModel?: string;
   /** Minimum overall spec score (1-10) required to pass Approval Gate. 0 = disabled (default). */
   minSpecScore?: number;
+  /** Minimum harness score (1-10) required for pipeline success. 0 = disabled (default). */
+  minHarnessScore?: number;
+  /** Maximum error-feedback cycles before giving up (default: 2, TDD default: 3). */
+  maxErrorCycles?: number;
 }
 
 export const CONFIG_FILE = ".ai-spec.json";

package/core/code-generator.ts

@@ -1,5 +1,5 @@
 import chalk from "chalk";
-import { execSync } from "child_process";
+import { execSync, spawnSync } from "child_process";
 import * as path from "path";
 import * as fs from "fs-extra";
 import { AIProvider } from "./spec-generator";
@@ -49,7 +49,7 @@ function buildInstalledPackagesSection(context?: ProjectContext): string {
  *
  * Falls back to first 3000 chars for CommonJS files with no explicit exports.
  */
-function extractBehavioralContract(content: string): string {
+export function extractBehavioralContract(content: string): string {
   const lines = content.split("\n");
   const contractLines: string[] = [];
   const throwLines: string[] = [];
@@ -349,9 +349,10 @@ export class CodeGenerator {
       console.log(chalk.cyan(`  🤖 Auto mode: running claude -p (non-interactive)...`));
       console.log(chalk.gray(`  Spec: ${specFilePath}`));
       try {
-        execSync(`${claudeCmd} -p "${promptContent.replace(/"/g, '\\"')}"`, {
+        spawnSync(claudeCmd, ["-p", promptContent], {
           cwd: workingDir,
           stdio: "inherit",
+          shell: false,
         });
         console.log(chalk.green("\n  ✔ Claude Code completed."));
       } catch {
@@ -413,9 +414,10 @@ export class CodeGenerator {
 
       let taskStatus: "done" | "failed" = "done";
       try {
-        execSync(`${claudeCmd} -p "${taskPrompt.replace(/"/g, '\\"').replace(/\n/g, "\\n")}"`, {
+        spawnSync(claudeCmd, ["-p", taskPrompt], {
           cwd: workingDir,
           stdio: "inherit",
+          shell: false,
         });
         completed++;
       } catch {

package/core/dsl-extractor.ts

@@ -128,7 +128,10 @@ export class DslExtractor {
     // Truncate very long specs to avoid token issues
     const specForAI =
       specContent.length > MAX_SPEC_CHARS
-        ? specContent.slice(0, MAX_SPEC_CHARS) + "\n... (truncated for DSL extraction)"
+        ? (() => {
+            console.log(chalk.yellow(`  ⚠ Spec is ${specContent.length} chars — truncating to ${MAX_SPEC_CHARS} for DSL extraction. Details at the end may be lost.`));
+            return specContent.slice(0, MAX_SPEC_CHARS) + "\n... (truncated for DSL extraction)";
+          })()
         : specContent;
 
     let lastRawOutput = "";
@@ -165,6 +168,11 @@ export class DslExtractor {
         parsed = parseJsonFromOutput(rawOutput);
       } catch (parseErr) {
         console.log(chalk.red(`  ✘ Failed to parse JSON from AI output: ${(parseErr as Error).message}`));
+        const preview = rawOutput.slice(0, 500).replace(/\n/g, "\\n");
+        console.log(chalk.gray(`  AI output preview (first 500 chars): ${preview}`));
+        if (rawOutput.length > MAX_SPEC_CHARS) {
+          console.log(chalk.gray(`  Note: spec was truncated to ${MAX_SPEC_CHARS} chars — long specs may lose context`));
+        }
         lastErrors = [{ path: "root", message: "Output is not valid JSON — see raw output above" }];
 
         if (attempt < MAX_RETRIES) continue;

package/core/dsl-feedback.ts

@@ -69,10 +69,16 @@ export function assessDslRichness(dsl: SpecDSL): DslGap[] {
   }
 
   // ── Endpoints with no error definitions (but spec text likely mentions them) ──
+  // Only flag when ALL endpoints lack error definitions — if at least one has
+  // errors, the author is aware of the pattern and the rest may genuinely not
+  // need explicit error cases (e.g. simple GET endpoints).
   const endpointsWithoutErrors = dsl.endpoints.filter(
     (ep) => !ep.errors || ep.errors.length === 0
   );
-  if (endpointsWithoutErrors.length > 0 && dsl.endpoints.length >= 2) {
+  if (
+    endpointsWithoutErrors.length === dsl.endpoints.length &&
+    dsl.endpoints.length >= 2
+  ) {
     gaps.push({
       code: "missing_errors",
       message: `${endpointsWithoutErrors.length}/${dsl.endpoints.length} endpoints have no error definitions`,

package/core/dsl-validator.ts

@@ -74,6 +74,22 @@ export function validateDsl(raw: unknown): DslValidationResult {
     for (let i = 0; i < Math.min(eps.length, MAX_ENDPOINTS); i++) {
       validateEndpoint(eps[i], `endpoints[${i}]`, errors);
     }
+    // ── Endpoint ID uniqueness ──────────────────────────────────────────────
+    const seenEpIds = new Set<string>();
+    for (let i = 0; i < Math.min(eps.length, MAX_ENDPOINTS); i++) {
+      const ep = eps[i] as Record<string, unknown> | null;
+      if (ep && typeof ep === "object" && typeof ep["id"] === "string") {
+        const id = ep["id"] as string;
+        if (seenEpIds.has(id)) {
+          errors.push({
+            path: `endpoints[${i}].id`,
+            message: `Duplicate endpoint id "${id}" — each endpoint must have a unique id`,
+          });
+        } else {
+          seenEpIds.add(id);
+        }
+      }
+    }
   }
 
   // ── behaviors (optional, but must be array if present) ────────────────────
@@ -149,6 +165,22 @@ function validateModel(
     for (let j = 0; j < Math.min(fields.length, MAX_FIELDS_PER_MODEL); j++) {
       validateModelField(fields[j], `${path}.fields[${j}]`, errors);
     }
+    // ── Field name uniqueness within model ──────────────────────────────────
+    const seenFieldNames = new Set<string>();
+    for (let j = 0; j < Math.min(fields.length, MAX_FIELDS_PER_MODEL); j++) {
+      const f = fields[j] as Record<string, unknown> | null;
+      if (f && typeof f === "object" && typeof f["name"] === "string") {
+        const name = f["name"] as string;
+        if (seenFieldNames.has(name)) {
+          errors.push({
+            path: `${path}.fields[${j}].name`,
+            message: `Duplicate field name "${name}" — each field within a model must have a unique name`,
+          });
+        } else {
+          seenFieldNames.add(name);
+        }
+      }
+    }
   }
 
   // relations: optional array of strings

package/core/key-store.ts

@@ -11,16 +11,17 @@ async function readStore(): Promise<KeyStore> {
     if (await fs.pathExists(KEY_STORE_FILE)) {
       return await fs.readJson(KEY_STORE_FILE);
     }
-  } catch {
-    // ignore corrupt file
+  } catch (err) {
+    console.warn(`Warning: Could not read key store at ${KEY_STORE_FILE}: ${(err as Error).message}. Using empty store.`);
   }
   return {};
 }
 
 async function writeStore(store: KeyStore): Promise<void> {
-  await fs.writeJson(KEY_STORE_FILE, store, { spaces: 2 });
-  // Restrict permissions to owner only (600)
+  // Ensure file exists with restricted permissions BEFORE writing sensitive data
+  await fs.ensureFile(KEY_STORE_FILE);
   await fs.chmod(KEY_STORE_FILE, 0o600);
+  await fs.writeJson(KEY_STORE_FILE, store, { spaces: 2 });
 }
 
 export async function getSavedKey(provider: string): Promise<string | undefined> {

package/core/provider-utils.ts

@@ -22,14 +22,49 @@ function classifyError(err: unknown, label: string): ProviderError {
   const status = e.status ?? e.response?.status;
 
   if (status === 401 || status === 403)
-    return new ProviderError(`Auth error — check your API key (${label})`, "auth", err);
+    return new ProviderError(
+      `Auth error (${label}): API key is invalid or expired.\n` +
+      `  → Check that the correct API key is set in your environment or ~/.ai-spec-keys.json\n` +
+      `  → Run "ai-spec model" to reconfigure your provider and key`,
+      "auth", err
+    );
   if (status === 429)
-    return new ProviderError(`Rate limit hit (${label}) — try again later or switch provider`, "rate_limit", err);
+    return new ProviderError(
+      `Rate limit hit (${label}): too many requests.\n` +
+      `  → Wait a few minutes and retry, or switch to a different provider/model\n` +
+      `  → Check your provider's billing dashboard for quota status`,
+      "rate_limit", err
+    );
   if ((e as Error & { _timeout?: boolean })._timeout || e.message?.toLowerCase().includes("timed out"))
     return new ProviderError(`Request timed out (${label})`, "timeout", err);
   if (e.code === "ECONNRESET" || e.code === "ENOTFOUND" || e.code === "ECONNREFUSED")
-    return new ProviderError(`Network error — check connection/proxy (${label}): ${e.message}`, "network", err);
-  return new ProviderError(`Provider error (${label}): ${e.message}`, "provider", err);
+    return new ProviderError(
+      `Network error (${label}): ${e.message}\n` +
+      `  → Check your internet connection and proxy settings (HTTPS_PROXY)\n` +
+      `  → If behind a firewall, ensure the provider's API endpoint is reachable`,
+      "network", err
+    );
+
+  // Check for common model-not-found errors
+  const msg = e.message ?? "";
+  if (status === 404 || msg.includes("model") && (msg.includes("not found") || msg.includes("does not exist")))
+    return new ProviderError(
+      `Model not found (${label}): ${msg}\n` +
+      `  → Run "ai-spec model" to see available models for your provider\n` +
+      `  → The model name may have changed — check your provider's documentation`,
+      "provider", err
+    );
+
+  // Check for insufficient balance / quota exhaustion
+  if (msg.includes("insufficient") || msg.includes("quota") || msg.includes("balance"))
+    return new ProviderError(
+      `Quota/balance error (${label}): ${msg}\n` +
+      `  → Check your provider's billing dashboard\n` +
+      `  → Consider switching to a different provider with "ai-spec model"`,
+      "provider", err
+    );
+
+  return new ProviderError(`Provider error (${label}): ${msg}`, "provider", err);
 }
 
 function isRetryable(err: unknown): boolean {