ai-spec-dev 0.35.0 → 0.36.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/RELEASE_LOG.md +119 -0
- package/cli/commands/config.ts +18 -0
- package/cli/commands/create.ts +16 -1
- package/cli/utils.ts +4 -0
- package/core/code-generator.ts +6 -4
- package/core/dsl-extractor.ts +9 -1
- package/core/dsl-feedback.ts +7 -1
- package/core/dsl-validator.ts +32 -0
- package/core/key-store.ts +5 -4
- package/core/provider-utils.ts +39 -4
- package/dist/cli/index.js +121 -14
- package/dist/cli/index.js.map +1 -1
- package/dist/cli/index.mjs +122 -15
- package/dist/cli/index.mjs.map +1 -1
- package/dist/index.d.mts +16 -1
- package/dist/index.d.ts +16 -1
- package/dist/index.js +77 -8
- package/dist/index.js.map +1 -1
- package/dist/index.mjs +77 -9
- package/dist/index.mjs.map +1 -1
- package/package.json +1 -1
- package/tests/code-generator.test.ts +253 -0
- package/tests/context-loader.test.ts +207 -0
- package/tests/dsl-validator.test.ts +105 -0
- package/tests/openapi-exporter.test.ts +310 -0
- package/tests/reviewer.test.ts +214 -0
- package/tests/spec-generator.test.ts +228 -0
- package/tests/spec-versioning.test.ts +205 -0
package/RELEASE_LOG.md
CHANGED
|
@@ -5,6 +5,125 @@
|
|
|
5
5
|
|
|
6
6
|
---
|
|
7
7
|
|
|
8
|
+
## v0.36.1 — 2026-04-02 — P0 测试覆盖 + 质量硬门禁 + 错误体验优化
|
|
9
|
+
|
|
10
|
+
### 新增测试(Week 2)
|
|
11
|
+
|
|
12
|
+
**Test 4 — `context-loader.test.ts`(19 tests)**
|
|
13
|
+
|
|
14
|
+
覆盖 `isFrontendDeps`(React/Vue/Next/Nuxt/Svelte/纯后端/空数组)、`ContextLoader` 类(Node.js/PHP/Java 三种项目类型的上下文加载、Prisma schema 读取、宪法加载、API 结构扫描、共享配置文件发现、错误模式提取、空目录容错)。
|
|
15
|
+
|
|
16
|
+
**Test 5 — `openapi-exporter.test.ts`(27 tests)**
|
|
17
|
+
|
|
18
|
+
覆盖 `dslToOpenApi`(结构完整性、info 元数据、自定义 server URL、路径参数标准化 `:id`→`{id}`、请求体生成、错误响应、认证端点 401 自动注入、安全方案生成、模型 schema 映射、必填字段标记、204 无内容响应、无认证场景)、类型映射(String/Int/Float/Boolean/DateTime/email/password/$ref)、`exportOpenApi`(YAML/JSON 格式、自定义输出路径、自定义 server URL)。
|
|
19
|
+
|
|
20
|
+
**Test 6 — `spec-versioning.test.ts`(26 tests)**
|
|
21
|
+
|
|
22
|
+
覆盖 `slugify`(英文转换、特殊字符、CJK 处理、长度限制、空输入回退、连字符折叠)、`computeDiff`(相同/新增/删除/修改/空文本/大文件回退/行类型正确性)、`findLatestVersion`(不存在目录、无匹配文件、单版本、多版本最新、不同 slug 隔离、正则特殊字符)、`nextVersionPath`(无版本/有 v1/跳跃版本号)。
|
|
23
|
+
|
|
24
|
+
**测试覆盖率提升:30% → 37.5%(12 → 15 个模块有测试,259 → 331 test cases)**
|
|
25
|
+
|
|
26
|
+
### 质量硬门禁(Week 3)
|
|
27
|
+
|
|
28
|
+
**Feature 1 — Harness Score 阻断门禁(`cli/commands/create.ts`、`cli/utils.ts`)**
|
|
29
|
+
|
|
30
|
+
- 新增 `minHarnessScore` 配置项(`.ai-spec.json`,默认 0 = 禁用)
|
|
31
|
+
- 自评阶段(Step 10)后,当 `harnessScore < minHarnessScore` 且未使用 `--force` 时,打印阈值提示并 `exit(1)`
|
|
32
|
+
- 与 `minSpecScore` 同样支持 `--force` 绕过
|
|
33
|
+
|
|
34
|
+
**Feature 2 — Error Feedback 轮次可配置(`cli/commands/create.ts`、`cli/utils.ts`)**
|
|
35
|
+
|
|
36
|
+
- 新增 `maxErrorCycles` 配置项(默认 2,TDD 模式默认 3,范围 1-10)
|
|
37
|
+
- 替换原来硬编码的 `maxCycles: opts.tdd ? 3 : 2`,读取 `config.maxErrorCycles`
|
|
38
|
+
|
|
39
|
+
**Feature 3 — Config 命令增强(`cli/commands/config.ts`)**
|
|
40
|
+
|
|
41
|
+
- 新增 `--min-harness-score <score>` 和 `--max-error-cycles <n>` CLI 选项
|
|
42
|
+
- 含输入范围校验(0-10 / 1-10)
|
|
43
|
+
|
|
44
|
+
### 错误体验优化(Week 4)
|
|
45
|
+
|
|
46
|
+
**Enhancement 1 — Provider 错误消息增强(`core/provider-utils.ts`)**
|
|
47
|
+
|
|
48
|
+
- **Auth 错误(401/403)**:提示检查 API key 有效性 + 运行 `ai-spec model` 重新配置
|
|
49
|
+
- **Rate Limit(429)**:提示等待或切换 provider + 检查计费面板
|
|
50
|
+
- **网络错误**:提示检查连接和代理设置
|
|
51
|
+
- **模型不存在**:提示运行 `ai-spec model` 查看可用模型
|
|
52
|
+
- **余额/配额不足**:提示检查计费面板 + 切换 provider
|
|
53
|
+
|
|
54
|
+
**Enhancement 2 — DSL 提取失败诊断增强(`core/dsl-extractor.ts`)**
|
|
55
|
+
|
|
56
|
+
- JSON 解析失败时,输出 AI 原始响应前 500 字符的预览,方便判断是 prompt 问题还是 model 能力问题
|
|
57
|
+
- Spec 超过 12K 字符截断时,**立即**打印黄色警告(而非静默截断),提醒用户详情可能丢失
|
|
58
|
+
|
|
59
|
+
**Enhancement 3 — Key Store 读取容错(`core/key-store.ts`)**
|
|
60
|
+
|
|
61
|
+
- 读取损坏的 key store 文件时,输出具体错误消息(而非静默忽略)
|
|
62
|
+
|
|
63
|
+
---
|
|
64
|
+
|
|
65
|
+
## v0.36.0 — 2026-04-01 — 安全修复 + 核心模块测试覆盖
|
|
66
|
+
|
|
67
|
+
### 安全修复
|
|
68
|
+
|
|
69
|
+
**Fix 1 — Shell 命令注入防护(`core/code-generator.ts`)**
|
|
70
|
+
|
|
71
|
+
`execSync` 拼接 shell 字符串传递 prompt 内容时,仅转义了 `"` 字符,未处理 `$`、`;`、`|`、`&` 等 shell 元字符,存在命令注入风险。
|
|
72
|
+
|
|
73
|
+
- 将 `execSync(\`\${claudeCmd} -p "..."\`)` 替换为 `spawnSync(claudeCmd, ["-p", promptContent], { shell: false })`(共 2 处)
|
|
74
|
+
- `spawnSync` 数组形式绕过 shell 解析,彻底消除注入可能
|
|
75
|
+
- 新增 `spawnSync` 导入(`child_process`)
|
|
76
|
+
|
|
77
|
+
**Fix 2 — API Key 存储权限时序(`core/key-store.ts`)**
|
|
78
|
+
|
|
79
|
+
原来先 `writeJson()` 再 `chmod(0o600)`,在写入与权限设置之间存在短暂窗口期,其他进程可能读取到明文 key。
|
|
80
|
+
|
|
81
|
+
- 改为 `ensureFile()` → `chmod(0o600)` → `writeJson()` 顺序,确保文件权限在写入敏感数据前就已设置
|
|
82
|
+
|
|
83
|
+
### 新增测试
|
|
84
|
+
|
|
85
|
+
**Test 1 — `spec-generator.test.ts`(23 tests)**
|
|
86
|
+
|
|
87
|
+
覆盖 `PROVIDER_CATALOG` 结构完整性、`createProvider` 工厂函数(9 个 provider 分支 + 自定义 model + 未知 provider 异常)、`SpecGenerator` prompt 构建逻辑(architecture decision 注入、constitution 优先级、context 截断限制)。
|
|
88
|
+
|
|
89
|
+
**Test 2 — `reviewer.test.ts`(19 tests)**
|
|
90
|
+
|
|
91
|
+
覆盖 `extractComplianceScore`(整数/小数/大小写/空字符串/多行/多匹配)、`extractMissingCount`(正常/大小写/缺失/多行)、`CodeReviewer` 类(空 diff 处理、多 Pass 调用验证、缺失文件容错、大文件截断、历史趋势渲染)。
|
|
92
|
+
|
|
93
|
+
**Test 3 — `code-generator.test.ts`(23 tests)**
|
|
94
|
+
|
|
95
|
+
覆盖 `extractBehavioralContract`(interface/enum/type/function/const/class/abstract class/defineStore/return 块/export default/嵌套大括号/throw 捕获上限/无 export 回退)、`printTaskProgress`(百分比计算/run 模式/skip 模式/0 total/未知 layer)。
|
|
96
|
+
|
|
97
|
+
**测试覆盖率提升:22.5% → 30%(9 → 12 个模块有测试,251 → 259 test cases)**
|
|
98
|
+
|
|
99
|
+
- `extractBehavioralContract` 从 private 改为 `export`(`core/code-generator.ts`),支持直接单元测试
|
|
100
|
+
|
|
101
|
+
### DSL 验证增强
|
|
102
|
+
|
|
103
|
+
**Fix 3 — Endpoint ID 唯一性检查(`core/dsl-validator.ts`)**
|
|
104
|
+
|
|
105
|
+
AI 经常生成重复的 Endpoint ID(如两个 `EP-001`),导致下游 DSL 消费方(types-generator、mock-server 等)产生覆盖冲突。
|
|
106
|
+
|
|
107
|
+
- 在 endpoints 验证阶段新增 `Set<string>` 去重检查,重复 ID 报告具体位置(`endpoints[N].id`)
|
|
108
|
+
- 新增 4 个测试用例(唯一 ID 通过、重复 ID 拒绝、路径定位正确、多组重复检测)
|
|
109
|
+
|
|
110
|
+
**Fix 4 — Model 字段名唯一性检查(`core/dsl-validator.ts`)**
|
|
111
|
+
|
|
112
|
+
同一 Model 内出现重复字段名(如两个 `id`)会导致 Prisma schema 或 TypeScript interface 生成冲突。
|
|
113
|
+
|
|
114
|
+
- 在 `validateModel` 内新增 `Set<string>` 去重检查,同一 model 内重复字段报告具体位置
|
|
115
|
+
- 不同 model 之间允许同名字段(如 `User.id` 和 `Post.id`)
|
|
116
|
+
- 新增 4 个测试用例
|
|
117
|
+
|
|
118
|
+
**Fix 5 — `missing_errors` 误报修复(`core/dsl-feedback.ts`)**
|
|
119
|
+
|
|
120
|
+
原来的逻辑:只要有任何 endpoint 缺少 errors 且总 endpoint ≥ 2 就标记 gap。这导致当部分 endpoint 已有 errors 时仍然误报。
|
|
121
|
+
|
|
122
|
+
- 修改为:仅当 **所有** endpoint 都缺少 errors 时才标记 `missing_errors` gap
|
|
123
|
+
- 修复了 `dsl-feedback.test.ts` 中已有的失败测试
|
|
124
|
+
|
|
125
|
+
---
|
|
126
|
+
|
|
8
127
|
## [Unreleased] 2026-04-01 — P1 Task 验证步骤 + P2 设计方案对话
|
|
9
128
|
|
|
10
129
|
### 新增 / 增强
|
package/cli/commands/config.ts
CHANGED
|
@@ -16,6 +16,8 @@ export function registerConfig(program: Command): void {
|
|
|
16
16
|
.option("--codegen-provider <name>", "Default provider for code generation")
|
|
17
17
|
.option("--codegen-model <name>", "Default model for code generation")
|
|
18
18
|
.option("--min-spec-score <score>", "Minimum overall spec score (1-10) to pass Approval Gate (0 = disabled)")
|
|
19
|
+
.option("--min-harness-score <score>", "Minimum harness score (1-10) for pipeline success (0 = disabled)")
|
|
20
|
+
.option("--max-error-cycles <n>", "Maximum error-feedback fix cycles (1-10, default: 2)")
|
|
19
21
|
.option("--show", "Print current configuration")
|
|
20
22
|
.option("--reset", "Reset configuration to empty")
|
|
21
23
|
.option("--clear-keys", "Delete all saved API keys from ~/.ai-spec-keys.json")
|
|
@@ -85,6 +87,22 @@ export function registerConfig(program: Command): void {
|
|
|
85
87
|
}
|
|
86
88
|
updated.minSpecScore = score;
|
|
87
89
|
}
|
|
90
|
+
if (opts.minHarnessScore !== undefined) {
|
|
91
|
+
const score = parseInt(opts.minHarnessScore, 10);
|
|
92
|
+
if (isNaN(score) || score < 0 || score > 10) {
|
|
93
|
+
console.error(chalk.red(" --min-harness-score must be a number between 0 and 10"));
|
|
94
|
+
process.exit(1);
|
|
95
|
+
}
|
|
96
|
+
updated.minHarnessScore = score;
|
|
97
|
+
}
|
|
98
|
+
if (opts.maxErrorCycles !== undefined) {
|
|
99
|
+
const cycles = parseInt(opts.maxErrorCycles, 10);
|
|
100
|
+
if (isNaN(cycles) || cycles < 1 || cycles > 10) {
|
|
101
|
+
console.error(chalk.red(" --max-error-cycles must be a number between 1 and 10"));
|
|
102
|
+
process.exit(1);
|
|
103
|
+
}
|
|
104
|
+
updated.maxErrorCycles = cycles;
|
|
105
|
+
}
|
|
88
106
|
|
|
89
107
|
await fs.writeJson(configPath, updated, { spaces: 2 });
|
|
90
108
|
console.log(chalk.green(`✔ Config saved to ${configPath}`));
|
package/cli/commands/create.ts
CHANGED
|
@@ -1076,8 +1076,10 @@ export function registerCreate(program: Command): void {
|
|
|
1076
1076
|
console.log(chalk.cyan("[8/9] TDD mode — error feedback loop driving implementation to pass tests..."));
|
|
1077
1077
|
}
|
|
1078
1078
|
runLogger.stageStart("error_feedback");
|
|
1079
|
+
const defaultCycles = opts.tdd ? 3 : 2;
|
|
1080
|
+
const maxCycles = config.maxErrorCycles ?? defaultCycles;
|
|
1079
1081
|
compilePassed = await runErrorFeedback(codegenProvider, workingDir, extractedDsl, {
|
|
1080
|
-
maxCycles
|
|
1082
|
+
maxCycles,
|
|
1081
1083
|
});
|
|
1082
1084
|
runLogger.stageEnd("error_feedback");
|
|
1083
1085
|
}
|
|
@@ -1203,6 +1205,19 @@ export function registerCreate(program: Command): void {
|
|
|
1203
1205
|
});
|
|
1204
1206
|
printSelfEval(selfEvalResult);
|
|
1205
1207
|
|
|
1208
|
+
// ── Harness Score Gate ─────────────────────────────────────────────────
|
|
1209
|
+
const minHarness = config.minHarnessScore ?? 0;
|
|
1210
|
+
if (minHarness > 0 && selfEvalResult.harnessScore < minHarness && !opts.force) {
|
|
1211
|
+
console.log(chalk.red(
|
|
1212
|
+
`\n ✘ Harness score ${selfEvalResult.harnessScore}/10 is below the minimum threshold ${minHarness}/10.`
|
|
1213
|
+
));
|
|
1214
|
+
console.log(chalk.gray(` Gate threshold set in .ai-spec.json → "minHarnessScore": ${minHarness}`));
|
|
1215
|
+
console.log(chalk.gray(` Use --force to bypass, or improve the spec and re-run.`));
|
|
1216
|
+
runLogger.stageEnd("self_eval", { gateBlocked: true, score: selfEvalResult.harnessScore, threshold: minHarness });
|
|
1217
|
+
runLogger.finish();
|
|
1218
|
+
process.exit(1);
|
|
1219
|
+
}
|
|
1220
|
+
|
|
1206
1221
|
// ── Await async §9 accumulation (fire-and-await pattern) ────────────────
|
|
1207
1222
|
if (accumulatePromise) await accumulatePromise;
|
|
1208
1223
|
|
package/cli/utils.ts
CHANGED
|
@@ -16,6 +16,10 @@ export interface AiSpecConfig {
|
|
|
16
16
|
codegenModel?: string;
|
|
17
17
|
/** Minimum overall spec score (1-10) required to pass Approval Gate. 0 = disabled (default). */
|
|
18
18
|
minSpecScore?: number;
|
|
19
|
+
/** Minimum harness score (1-10) required for pipeline success. 0 = disabled (default). */
|
|
20
|
+
minHarnessScore?: number;
|
|
21
|
+
/** Maximum error-feedback cycles before giving up (default: 2, TDD default: 3). */
|
|
22
|
+
maxErrorCycles?: number;
|
|
19
23
|
}
|
|
20
24
|
|
|
21
25
|
export const CONFIG_FILE = ".ai-spec.json";
|
package/core/code-generator.ts
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
import chalk from "chalk";
|
|
2
|
-
import { execSync } from "child_process";
|
|
2
|
+
import { execSync, spawnSync } from "child_process";
|
|
3
3
|
import * as path from "path";
|
|
4
4
|
import * as fs from "fs-extra";
|
|
5
5
|
import { AIProvider } from "./spec-generator";
|
|
@@ -49,7 +49,7 @@ function buildInstalledPackagesSection(context?: ProjectContext): string {
|
|
|
49
49
|
*
|
|
50
50
|
* Falls back to first 3000 chars for CommonJS files with no explicit exports.
|
|
51
51
|
*/
|
|
52
|
-
function extractBehavioralContract(content: string): string {
|
|
52
|
+
export function extractBehavioralContract(content: string): string {
|
|
53
53
|
const lines = content.split("\n");
|
|
54
54
|
const contractLines: string[] = [];
|
|
55
55
|
const throwLines: string[] = [];
|
|
@@ -349,9 +349,10 @@ export class CodeGenerator {
|
|
|
349
349
|
console.log(chalk.cyan(` 🤖 Auto mode: running claude -p (non-interactive)...`));
|
|
350
350
|
console.log(chalk.gray(` Spec: ${specFilePath}`));
|
|
351
351
|
try {
|
|
352
|
-
|
|
352
|
+
spawnSync(claudeCmd, ["-p", promptContent], {
|
|
353
353
|
cwd: workingDir,
|
|
354
354
|
stdio: "inherit",
|
|
355
|
+
shell: false,
|
|
355
356
|
});
|
|
356
357
|
console.log(chalk.green("\n ✔ Claude Code completed."));
|
|
357
358
|
} catch {
|
|
@@ -413,9 +414,10 @@ export class CodeGenerator {
|
|
|
413
414
|
|
|
414
415
|
let taskStatus: "done" | "failed" = "done";
|
|
415
416
|
try {
|
|
416
|
-
|
|
417
|
+
spawnSync(claudeCmd, ["-p", taskPrompt], {
|
|
417
418
|
cwd: workingDir,
|
|
418
419
|
stdio: "inherit",
|
|
420
|
+
shell: false,
|
|
419
421
|
});
|
|
420
422
|
completed++;
|
|
421
423
|
} catch {
|
package/core/dsl-extractor.ts
CHANGED
|
@@ -128,7 +128,10 @@ export class DslExtractor {
|
|
|
128
128
|
// Truncate very long specs to avoid token issues
|
|
129
129
|
const specForAI =
|
|
130
130
|
specContent.length > MAX_SPEC_CHARS
|
|
131
|
-
?
|
|
131
|
+
? (() => {
|
|
132
|
+
console.log(chalk.yellow(` ⚠ Spec is ${specContent.length} chars — truncating to ${MAX_SPEC_CHARS} for DSL extraction. Details at the end may be lost.`));
|
|
133
|
+
return specContent.slice(0, MAX_SPEC_CHARS) + "\n... (truncated for DSL extraction)";
|
|
134
|
+
})()
|
|
132
135
|
: specContent;
|
|
133
136
|
|
|
134
137
|
let lastRawOutput = "";
|
|
@@ -165,6 +168,11 @@ export class DslExtractor {
|
|
|
165
168
|
parsed = parseJsonFromOutput(rawOutput);
|
|
166
169
|
} catch (parseErr) {
|
|
167
170
|
console.log(chalk.red(` ✘ Failed to parse JSON from AI output: ${(parseErr as Error).message}`));
|
|
171
|
+
const preview = rawOutput.slice(0, 500).replace(/\n/g, "\\n");
|
|
172
|
+
console.log(chalk.gray(` AI output preview (first 500 chars): ${preview}`));
|
|
173
|
+
if (rawOutput.length > MAX_SPEC_CHARS) {
|
|
174
|
+
console.log(chalk.gray(` Note: spec was truncated to ${MAX_SPEC_CHARS} chars — long specs may lose context`));
|
|
175
|
+
}
|
|
168
176
|
lastErrors = [{ path: "root", message: "Output is not valid JSON — see raw output above" }];
|
|
169
177
|
|
|
170
178
|
if (attempt < MAX_RETRIES) continue;
|
package/core/dsl-feedback.ts
CHANGED
|
@@ -69,10 +69,16 @@ export function assessDslRichness(dsl: SpecDSL): DslGap[] {
|
|
|
69
69
|
}
|
|
70
70
|
|
|
71
71
|
// ── Endpoints with no error definitions (but spec text likely mentions them) ──
|
|
72
|
+
// Only flag when ALL endpoints lack error definitions — if at least one has
|
|
73
|
+
// errors, the author is aware of the pattern and the rest may genuinely not
|
|
74
|
+
// need explicit error cases (e.g. simple GET endpoints).
|
|
72
75
|
const endpointsWithoutErrors = dsl.endpoints.filter(
|
|
73
76
|
(ep) => !ep.errors || ep.errors.length === 0
|
|
74
77
|
);
|
|
75
|
-
if (
|
|
78
|
+
if (
|
|
79
|
+
endpointsWithoutErrors.length === dsl.endpoints.length &&
|
|
80
|
+
dsl.endpoints.length >= 2
|
|
81
|
+
) {
|
|
76
82
|
gaps.push({
|
|
77
83
|
code: "missing_errors",
|
|
78
84
|
message: `${endpointsWithoutErrors.length}/${dsl.endpoints.length} endpoints have no error definitions`,
|
package/core/dsl-validator.ts
CHANGED
|
@@ -74,6 +74,22 @@ export function validateDsl(raw: unknown): DslValidationResult {
|
|
|
74
74
|
for (let i = 0; i < Math.min(eps.length, MAX_ENDPOINTS); i++) {
|
|
75
75
|
validateEndpoint(eps[i], `endpoints[${i}]`, errors);
|
|
76
76
|
}
|
|
77
|
+
// ── Endpoint ID uniqueness ──────────────────────────────────────────────
|
|
78
|
+
const seenEpIds = new Set<string>();
|
|
79
|
+
for (let i = 0; i < Math.min(eps.length, MAX_ENDPOINTS); i++) {
|
|
80
|
+
const ep = eps[i] as Record<string, unknown> | null;
|
|
81
|
+
if (ep && typeof ep === "object" && typeof ep["id"] === "string") {
|
|
82
|
+
const id = ep["id"] as string;
|
|
83
|
+
if (seenEpIds.has(id)) {
|
|
84
|
+
errors.push({
|
|
85
|
+
path: `endpoints[${i}].id`,
|
|
86
|
+
message: `Duplicate endpoint id "${id}" — each endpoint must have a unique id`,
|
|
87
|
+
});
|
|
88
|
+
} else {
|
|
89
|
+
seenEpIds.add(id);
|
|
90
|
+
}
|
|
91
|
+
}
|
|
92
|
+
}
|
|
77
93
|
}
|
|
78
94
|
|
|
79
95
|
// ── behaviors (optional, but must be array if present) ────────────────────
|
|
@@ -149,6 +165,22 @@ function validateModel(
|
|
|
149
165
|
for (let j = 0; j < Math.min(fields.length, MAX_FIELDS_PER_MODEL); j++) {
|
|
150
166
|
validateModelField(fields[j], `${path}.fields[${j}]`, errors);
|
|
151
167
|
}
|
|
168
|
+
// ── Field name uniqueness within model ──────────────────────────────────
|
|
169
|
+
const seenFieldNames = new Set<string>();
|
|
170
|
+
for (let j = 0; j < Math.min(fields.length, MAX_FIELDS_PER_MODEL); j++) {
|
|
171
|
+
const f = fields[j] as Record<string, unknown> | null;
|
|
172
|
+
if (f && typeof f === "object" && typeof f["name"] === "string") {
|
|
173
|
+
const name = f["name"] as string;
|
|
174
|
+
if (seenFieldNames.has(name)) {
|
|
175
|
+
errors.push({
|
|
176
|
+
path: `${path}.fields[${j}].name`,
|
|
177
|
+
message: `Duplicate field name "${name}" — each field within a model must have a unique name`,
|
|
178
|
+
});
|
|
179
|
+
} else {
|
|
180
|
+
seenFieldNames.add(name);
|
|
181
|
+
}
|
|
182
|
+
}
|
|
183
|
+
}
|
|
152
184
|
}
|
|
153
185
|
|
|
154
186
|
// relations: optional array of strings
|
package/core/key-store.ts
CHANGED
|
@@ -11,16 +11,17 @@ async function readStore(): Promise<KeyStore> {
|
|
|
11
11
|
if (await fs.pathExists(KEY_STORE_FILE)) {
|
|
12
12
|
return await fs.readJson(KEY_STORE_FILE);
|
|
13
13
|
}
|
|
14
|
-
} catch {
|
|
15
|
-
|
|
14
|
+
} catch (err) {
|
|
15
|
+
console.warn(`Warning: Could not read key store at ${KEY_STORE_FILE}: ${(err as Error).message}. Using empty store.`);
|
|
16
16
|
}
|
|
17
17
|
return {};
|
|
18
18
|
}
|
|
19
19
|
|
|
20
20
|
async function writeStore(store: KeyStore): Promise<void> {
|
|
21
|
-
|
|
22
|
-
|
|
21
|
+
// Ensure file exists with restricted permissions BEFORE writing sensitive data
|
|
22
|
+
await fs.ensureFile(KEY_STORE_FILE);
|
|
23
23
|
await fs.chmod(KEY_STORE_FILE, 0o600);
|
|
24
|
+
await fs.writeJson(KEY_STORE_FILE, store, { spaces: 2 });
|
|
24
25
|
}
|
|
25
26
|
|
|
26
27
|
export async function getSavedKey(provider: string): Promise<string | undefined> {
|
package/core/provider-utils.ts
CHANGED
|
@@ -22,14 +22,49 @@ function classifyError(err: unknown, label: string): ProviderError {
|
|
|
22
22
|
const status = e.status ?? e.response?.status;
|
|
23
23
|
|
|
24
24
|
if (status === 401 || status === 403)
|
|
25
|
-
return new ProviderError(
|
|
25
|
+
return new ProviderError(
|
|
26
|
+
`Auth error (${label}): API key is invalid or expired.\n` +
|
|
27
|
+
` → Check that the correct API key is set in your environment or ~/.ai-spec-keys.json\n` +
|
|
28
|
+
` → Run "ai-spec model" to reconfigure your provider and key`,
|
|
29
|
+
"auth", err
|
|
30
|
+
);
|
|
26
31
|
if (status === 429)
|
|
27
|
-
return new ProviderError(
|
|
32
|
+
return new ProviderError(
|
|
33
|
+
`Rate limit hit (${label}): too many requests.\n` +
|
|
34
|
+
` → Wait a few minutes and retry, or switch to a different provider/model\n` +
|
|
35
|
+
` → Check your provider's billing dashboard for quota status`,
|
|
36
|
+
"rate_limit", err
|
|
37
|
+
);
|
|
28
38
|
if ((e as Error & { _timeout?: boolean })._timeout || e.message?.toLowerCase().includes("timed out"))
|
|
29
39
|
return new ProviderError(`Request timed out (${label})`, "timeout", err);
|
|
30
40
|
if (e.code === "ECONNRESET" || e.code === "ENOTFOUND" || e.code === "ECONNREFUSED")
|
|
31
|
-
return new ProviderError(
|
|
32
|
-
|
|
41
|
+
return new ProviderError(
|
|
42
|
+
`Network error (${label}): ${e.message}\n` +
|
|
43
|
+
` → Check your internet connection and proxy settings (HTTPS_PROXY)\n` +
|
|
44
|
+
` → If behind a firewall, ensure the provider's API endpoint is reachable`,
|
|
45
|
+
"network", err
|
|
46
|
+
);
|
|
47
|
+
|
|
48
|
+
// Check for common model-not-found errors
|
|
49
|
+
const msg = e.message ?? "";
|
|
50
|
+
if (status === 404 || msg.includes("model") && (msg.includes("not found") || msg.includes("does not exist")))
|
|
51
|
+
return new ProviderError(
|
|
52
|
+
`Model not found (${label}): ${msg}\n` +
|
|
53
|
+
` → Run "ai-spec model" to see available models for your provider\n` +
|
|
54
|
+
` → The model name may have changed — check your provider's documentation`,
|
|
55
|
+
"provider", err
|
|
56
|
+
);
|
|
57
|
+
|
|
58
|
+
// Check for insufficient balance / quota exhaustion
|
|
59
|
+
if (msg.includes("insufficient") || msg.includes("quota") || msg.includes("balance"))
|
|
60
|
+
return new ProviderError(
|
|
61
|
+
`Quota/balance error (${label}): ${msg}\n` +
|
|
62
|
+
` → Check your provider's billing dashboard\n` +
|
|
63
|
+
` → Consider switching to a different provider with "ai-spec model"`,
|
|
64
|
+
"provider", err
|
|
65
|
+
);
|
|
66
|
+
|
|
67
|
+
return new ProviderError(`Provider error (${label}): ${msg}`, "provider", err);
|
|
33
68
|
}
|
|
34
69
|
|
|
35
70
|
function isRetryable(err: unknown): boolean {
|
package/dist/cli/index.js
CHANGED
|
@@ -152,14 +152,49 @@ function classifyError(err, label) {
|
|
|
152
152
|
const e = err;
|
|
153
153
|
const status = e.status ?? e.response?.status;
|
|
154
154
|
if (status === 401 || status === 403)
|
|
155
|
-
return new ProviderError(
|
|
155
|
+
return new ProviderError(
|
|
156
|
+
`Auth error (${label}): API key is invalid or expired.
|
|
157
|
+
\u2192 Check that the correct API key is set in your environment or ~/.ai-spec-keys.json
|
|
158
|
+
\u2192 Run "ai-spec model" to reconfigure your provider and key`,
|
|
159
|
+
"auth",
|
|
160
|
+
err
|
|
161
|
+
);
|
|
156
162
|
if (status === 429)
|
|
157
|
-
return new ProviderError(
|
|
163
|
+
return new ProviderError(
|
|
164
|
+
`Rate limit hit (${label}): too many requests.
|
|
165
|
+
\u2192 Wait a few minutes and retry, or switch to a different provider/model
|
|
166
|
+
\u2192 Check your provider's billing dashboard for quota status`,
|
|
167
|
+
"rate_limit",
|
|
168
|
+
err
|
|
169
|
+
);
|
|
158
170
|
if (e._timeout || e.message?.toLowerCase().includes("timed out"))
|
|
159
171
|
return new ProviderError(`Request timed out (${label})`, "timeout", err);
|
|
160
172
|
if (e.code === "ECONNRESET" || e.code === "ENOTFOUND" || e.code === "ECONNREFUSED")
|
|
161
|
-
return new ProviderError(
|
|
162
|
-
|
|
173
|
+
return new ProviderError(
|
|
174
|
+
`Network error (${label}): ${e.message}
|
|
175
|
+
\u2192 Check your internet connection and proxy settings (HTTPS_PROXY)
|
|
176
|
+
\u2192 If behind a firewall, ensure the provider's API endpoint is reachable`,
|
|
177
|
+
"network",
|
|
178
|
+
err
|
|
179
|
+
);
|
|
180
|
+
const msg = e.message ?? "";
|
|
181
|
+
if (status === 404 || msg.includes("model") && (msg.includes("not found") || msg.includes("does not exist")))
|
|
182
|
+
return new ProviderError(
|
|
183
|
+
`Model not found (${label}): ${msg}
|
|
184
|
+
\u2192 Run "ai-spec model" to see available models for your provider
|
|
185
|
+
\u2192 The model name may have changed \u2014 check your provider's documentation`,
|
|
186
|
+
"provider",
|
|
187
|
+
err
|
|
188
|
+
);
|
|
189
|
+
if (msg.includes("insufficient") || msg.includes("quota") || msg.includes("balance"))
|
|
190
|
+
return new ProviderError(
|
|
191
|
+
`Quota/balance error (${label}): ${msg}
|
|
192
|
+
\u2192 Check your provider's billing dashboard
|
|
193
|
+
\u2192 Consider switching to a different provider with "ai-spec model"`,
|
|
194
|
+
"provider",
|
|
195
|
+
err
|
|
196
|
+
);
|
|
197
|
+
return new ProviderError(`Provider error (${label}): ${msg}`, "provider", err);
|
|
163
198
|
}
|
|
164
199
|
function isRetryable(err) {
|
|
165
200
|
const e = err;
|
|
@@ -4813,6 +4848,21 @@ function validateDsl(raw) {
|
|
|
4813
4848
|
for (let i = 0; i < Math.min(eps.length, MAX_ENDPOINTS); i++) {
|
|
4814
4849
|
validateEndpoint(eps[i], `endpoints[${i}]`, errors);
|
|
4815
4850
|
}
|
|
4851
|
+
const seenEpIds = /* @__PURE__ */ new Set();
|
|
4852
|
+
for (let i = 0; i < Math.min(eps.length, MAX_ENDPOINTS); i++) {
|
|
4853
|
+
const ep = eps[i];
|
|
4854
|
+
if (ep && typeof ep === "object" && typeof ep["id"] === "string") {
|
|
4855
|
+
const id = ep["id"];
|
|
4856
|
+
if (seenEpIds.has(id)) {
|
|
4857
|
+
errors.push({
|
|
4858
|
+
path: `endpoints[${i}].id`,
|
|
4859
|
+
message: `Duplicate endpoint id "${id}" \u2014 each endpoint must have a unique id`
|
|
4860
|
+
});
|
|
4861
|
+
} else {
|
|
4862
|
+
seenEpIds.add(id);
|
|
4863
|
+
}
|
|
4864
|
+
}
|
|
4865
|
+
}
|
|
4816
4866
|
}
|
|
4817
4867
|
if (obj["behaviors"] !== void 0) {
|
|
4818
4868
|
if (!Array.isArray(obj["behaviors"])) {
|
|
@@ -4869,6 +4919,21 @@ function validateModel(raw, path40, errors) {
|
|
|
4869
4919
|
for (let j2 = 0; j2 < Math.min(fields.length, MAX_FIELDS_PER_MODEL); j2++) {
|
|
4870
4920
|
validateModelField(fields[j2], `${path40}.fields[${j2}]`, errors);
|
|
4871
4921
|
}
|
|
4922
|
+
const seenFieldNames = /* @__PURE__ */ new Set();
|
|
4923
|
+
for (let j2 = 0; j2 < Math.min(fields.length, MAX_FIELDS_PER_MODEL); j2++) {
|
|
4924
|
+
const f = fields[j2];
|
|
4925
|
+
if (f && typeof f === "object" && typeof f["name"] === "string") {
|
|
4926
|
+
const name = f["name"];
|
|
4927
|
+
if (seenFieldNames.has(name)) {
|
|
4928
|
+
errors.push({
|
|
4929
|
+
path: `${path40}.fields[${j2}].name`,
|
|
4930
|
+
message: `Duplicate field name "${name}" \u2014 each field within a model must have a unique name`
|
|
4931
|
+
});
|
|
4932
|
+
} else {
|
|
4933
|
+
seenFieldNames.add(name);
|
|
4934
|
+
}
|
|
4935
|
+
}
|
|
4936
|
+
}
|
|
4872
4937
|
}
|
|
4873
4938
|
if (m["relations"] !== void 0) {
|
|
4874
4939
|
if (!Array.isArray(m["relations"])) {
|
|
@@ -5336,7 +5401,10 @@ var DslExtractor = class {
|
|
|
5336
5401
|
* - throws if user chose to abort
|
|
5337
5402
|
*/
|
|
5338
5403
|
async extract(specContent, opts = {}) {
|
|
5339
|
-
const specForAI = specContent.length > MAX_SPEC_CHARS ?
|
|
5404
|
+
const specForAI = specContent.length > MAX_SPEC_CHARS ? (() => {
|
|
5405
|
+
console.log(import_chalk6.default.yellow(` \u26A0 Spec is ${specContent.length} chars \u2014 truncating to ${MAX_SPEC_CHARS} for DSL extraction. Details at the end may be lost.`));
|
|
5406
|
+
return specContent.slice(0, MAX_SPEC_CHARS) + "\n... (truncated for DSL extraction)";
|
|
5407
|
+
})() : specContent;
|
|
5340
5408
|
let lastRawOutput = "";
|
|
5341
5409
|
let lastErrors = [];
|
|
5342
5410
|
for (let attempt = 1; attempt <= MAX_RETRIES; attempt++) {
|
|
@@ -5360,6 +5428,11 @@ var DslExtractor = class {
|
|
|
5360
5428
|
parsed = parseJsonFromOutput(rawOutput);
|
|
5361
5429
|
} catch (parseErr) {
|
|
5362
5430
|
console.log(import_chalk6.default.red(` \u2718 Failed to parse JSON from AI output: ${parseErr.message}`));
|
|
5431
|
+
const preview = rawOutput.slice(0, 500).replace(/\n/g, "\\n");
|
|
5432
|
+
console.log(import_chalk6.default.gray(` AI output preview (first 500 chars): ${preview}`));
|
|
5433
|
+
if (rawOutput.length > MAX_SPEC_CHARS) {
|
|
5434
|
+
console.log(import_chalk6.default.gray(` Note: spec was truncated to ${MAX_SPEC_CHARS} chars \u2014 long specs may lose context`));
|
|
5435
|
+
}
|
|
5363
5436
|
lastErrors = [{ path: "root", message: "Output is not valid JSON \u2014 see raw output above" }];
|
|
5364
5437
|
if (attempt < MAX_RETRIES) continue;
|
|
5365
5438
|
return this.handleFailure(opts, "AI produced invalid JSON after retries");
|
|
@@ -6468,9 +6541,10 @@ ${tasks.map((t) => `${t.id} [${t.layer}] ${t.title}
|
|
|
6468
6541
|
console.log(import_chalk8.default.cyan(` \u{1F916} Auto mode: running claude -p (non-interactive)...`));
|
|
6469
6542
|
console.log(import_chalk8.default.gray(` Spec: ${specFilePath}`));
|
|
6470
6543
|
try {
|
|
6471
|
-
(0, import_child_process.
|
|
6544
|
+
(0, import_child_process.spawnSync)(claudeCmd, ["-p", promptContent], {
|
|
6472
6545
|
cwd: workingDir,
|
|
6473
|
-
stdio: "inherit"
|
|
6546
|
+
stdio: "inherit",
|
|
6547
|
+
shell: false
|
|
6474
6548
|
});
|
|
6475
6549
|
console.log(import_chalk8.default.green("\n \u2714 Claude Code completed."));
|
|
6476
6550
|
} catch {
|
|
@@ -6522,9 +6596,10 @@ Full spec is at: ${specFilePath}
|
|
|
6522
6596
|
Implement ONLY this task. Do not implement other tasks.`;
|
|
6523
6597
|
let taskStatus = "done";
|
|
6524
6598
|
try {
|
|
6525
|
-
(0, import_child_process.
|
|
6599
|
+
(0, import_child_process.spawnSync)(claudeCmd, ["-p", taskPrompt], {
|
|
6526
6600
|
cwd: workingDir,
|
|
6527
|
-
stdio: "inherit"
|
|
6601
|
+
stdio: "inherit",
|
|
6602
|
+
shell: false
|
|
6528
6603
|
});
|
|
6529
6604
|
completed++;
|
|
6530
6605
|
} catch {
|
|
@@ -9306,7 +9381,7 @@ function assessDslRichness(dsl) {
|
|
|
9306
9381
|
const endpointsWithoutErrors = dsl.endpoints.filter(
|
|
9307
9382
|
(ep) => !ep.errors || ep.errors.length === 0
|
|
9308
9383
|
);
|
|
9309
|
-
if (endpointsWithoutErrors.length
|
|
9384
|
+
if (endpointsWithoutErrors.length === dsl.endpoints.length && dsl.endpoints.length >= 2) {
|
|
9310
9385
|
gaps.push({
|
|
9311
9386
|
code: "missing_errors",
|
|
9312
9387
|
message: `${endpointsWithoutErrors.length}/${dsl.endpoints.length} endpoints have no error definitions`,
|
|
@@ -9892,13 +9967,15 @@ async function readStore() {
|
|
|
9892
9967
|
if (await fs19.pathExists(KEY_STORE_FILE)) {
|
|
9893
9968
|
return await fs19.readJson(KEY_STORE_FILE);
|
|
9894
9969
|
}
|
|
9895
|
-
} catch {
|
|
9970
|
+
} catch (err) {
|
|
9971
|
+
console.warn(`Warning: Could not read key store at ${KEY_STORE_FILE}: ${err.message}. Using empty store.`);
|
|
9896
9972
|
}
|
|
9897
9973
|
return {};
|
|
9898
9974
|
}
|
|
9899
9975
|
async function writeStore(store) {
|
|
9900
|
-
await fs19.
|
|
9976
|
+
await fs19.ensureFile(KEY_STORE_FILE);
|
|
9901
9977
|
await fs19.chmod(KEY_STORE_FILE, 384);
|
|
9978
|
+
await fs19.writeJson(KEY_STORE_FILE, store, { spaces: 2 });
|
|
9902
9979
|
}
|
|
9903
9980
|
async function getSavedKey(provider) {
|
|
9904
9981
|
const store = await readStore();
|
|
@@ -10991,8 +11068,10 @@ function registerCreate(program2) {
|
|
|
10991
11068
|
console.log(import_chalk20.default.cyan("[8/9] TDD mode \u2014 error feedback loop driving implementation to pass tests..."));
|
|
10992
11069
|
}
|
|
10993
11070
|
runLogger.stageStart("error_feedback");
|
|
11071
|
+
const defaultCycles = opts.tdd ? 3 : 2;
|
|
11072
|
+
const maxCycles = config2.maxErrorCycles ?? defaultCycles;
|
|
10994
11073
|
compilePassed = await runErrorFeedback(codegenProvider, workingDir, extractedDsl, {
|
|
10995
|
-
maxCycles
|
|
11074
|
+
maxCycles
|
|
10996
11075
|
});
|
|
10997
11076
|
runLogger.stageEnd("error_feedback");
|
|
10998
11077
|
}
|
|
@@ -11095,6 +11174,18 @@ function registerCreate(program2) {
|
|
|
11095
11174
|
logger: runLogger
|
|
11096
11175
|
});
|
|
11097
11176
|
printSelfEval(selfEvalResult);
|
|
11177
|
+
const minHarness = config2.minHarnessScore ?? 0;
|
|
11178
|
+
if (minHarness > 0 && selfEvalResult.harnessScore < minHarness && !opts.force) {
|
|
11179
|
+
console.log(import_chalk20.default.red(
|
|
11180
|
+
`
|
|
11181
|
+
\u2718 Harness score ${selfEvalResult.harnessScore}/10 is below the minimum threshold ${minHarness}/10.`
|
|
11182
|
+
));
|
|
11183
|
+
console.log(import_chalk20.default.gray(` Gate threshold set in .ai-spec.json \u2192 "minHarnessScore": ${minHarness}`));
|
|
11184
|
+
console.log(import_chalk20.default.gray(` Use --force to bypass, or improve the spec and re-run.`));
|
|
11185
|
+
runLogger.stageEnd("self_eval", { gateBlocked: true, score: selfEvalResult.harnessScore, threshold: minHarness });
|
|
11186
|
+
runLogger.finish();
|
|
11187
|
+
process.exit(1);
|
|
11188
|
+
}
|
|
11098
11189
|
if (accumulatePromise) await accumulatePromise;
|
|
11099
11190
|
if (specVcrRecorder) {
|
|
11100
11191
|
const vcrPath = await specVcrRecorder.save(currentDir, runId, codegenVcrRecorder ?? void 0);
|
|
@@ -11666,7 +11757,7 @@ var path26 = __toESM(require("path"));
|
|
|
11666
11757
|
var fs27 = __toESM(require("fs-extra"));
|
|
11667
11758
|
var import_chalk24 = __toESM(require("chalk"));
|
|
11668
11759
|
function registerConfig(program2) {
|
|
11669
|
-
program2.command("config").description(`Set default configuration for this project (saved to ${CONFIG_FILE})`).option("--provider <name>", "Default AI provider for spec generation").option("--model <name>", "Default model for spec generation").option("--codegen <mode>", "Default code generation mode (claude-code|api|plan)").option("--codegen-provider <name>", "Default provider for code generation").option("--codegen-model <name>", "Default model for code generation").option("--min-spec-score <score>", "Minimum overall spec score (1-10) to pass Approval Gate (0 = disabled)").option("--show", "Print current configuration").option("--reset", "Reset configuration to empty").option("--clear-keys", "Delete all saved API keys from ~/.ai-spec-keys.json").option("--clear-key <provider>", "Delete saved API key for a specific provider").option("--list-keys", "Show which providers have a saved key").action(async (opts) => {
|
|
11760
|
+
program2.command("config").description(`Set default configuration for this project (saved to ${CONFIG_FILE})`).option("--provider <name>", "Default AI provider for spec generation").option("--model <name>", "Default model for spec generation").option("--codegen <mode>", "Default code generation mode (claude-code|api|plan)").option("--codegen-provider <name>", "Default provider for code generation").option("--codegen-model <name>", "Default model for code generation").option("--min-spec-score <score>", "Minimum overall spec score (1-10) to pass Approval Gate (0 = disabled)").option("--min-harness-score <score>", "Minimum harness score (1-10) for pipeline success (0 = disabled)").option("--max-error-cycles <n>", "Maximum error-feedback fix cycles (1-10, default: 2)").option("--show", "Print current configuration").option("--reset", "Reset configuration to empty").option("--clear-keys", "Delete all saved API keys from ~/.ai-spec-keys.json").option("--clear-key <provider>", "Delete saved API key for a specific provider").option("--list-keys", "Show which providers have a saved key").action(async (opts) => {
|
|
11670
11761
|
const currentDir = process.cwd();
|
|
11671
11762
|
const configPath = path26.join(currentDir, CONFIG_FILE);
|
|
11672
11763
|
if (opts.clearKeys) {
|
|
@@ -11724,6 +11815,22 @@ File: ${KEY_STORE_FILE}`));
|
|
|
11724
11815
|
}
|
|
11725
11816
|
updated.minSpecScore = score;
|
|
11726
11817
|
}
|
|
11818
|
+
if (opts.minHarnessScore !== void 0) {
|
|
11819
|
+
const score = parseInt(opts.minHarnessScore, 10);
|
|
11820
|
+
if (isNaN(score) || score < 0 || score > 10) {
|
|
11821
|
+
console.error(import_chalk24.default.red(" --min-harness-score must be a number between 0 and 10"));
|
|
11822
|
+
process.exit(1);
|
|
11823
|
+
}
|
|
11824
|
+
updated.minHarnessScore = score;
|
|
11825
|
+
}
|
|
11826
|
+
if (opts.maxErrorCycles !== void 0) {
|
|
11827
|
+
const cycles = parseInt(opts.maxErrorCycles, 10);
|
|
11828
|
+
if (isNaN(cycles) || cycles < 1 || cycles > 10) {
|
|
11829
|
+
console.error(import_chalk24.default.red(" --max-error-cycles must be a number between 1 and 10"));
|
|
11830
|
+
process.exit(1);
|
|
11831
|
+
}
|
|
11832
|
+
updated.maxErrorCycles = cycles;
|
|
11833
|
+
}
|
|
11727
11834
|
await fs27.writeJson(configPath, updated, { spaces: 2 });
|
|
11728
11835
|
console.log(import_chalk24.default.green(`\u2714 Config saved to ${configPath}`));
|
|
11729
11836
|
console.log(JSON.stringify(updated, null, 2));
|