@tea-agent/loop-agent 0.16.22 → 0.16.24
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +28 -11
- package/dist/executors/shell-presets.js +27 -26
- package/dist/executors/shell-verification.js +2 -0
- package/dist/task/config-types.js +15 -3
- package/dist/workflows/dag/backend-test-analysis-contract.js +21 -8
- package/dist/workflows/dag/backend-test-contract-envelope.js +40 -2
- package/dist/workflows/dag/dynamic-runtime/map.js +129 -22
- package/dist/workflows/dag/init-hybrid.js +328 -259
- package/dist/workflows/dag/runner.js +5 -0
- package/dist/workflows/dag/types.js +7 -0
- package/docs/local-development-environment.md +4 -0
- package/docs/templates/backend-test-dag.json +1 -1
- package/docs/templates/frontend-test-case-checklist.md +25 -0
- package/docs/templates/frontend-test-dag.json +4 -3
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,18 +1,9 @@
|
|
|
1
|
-
# 更新日志
|
|
2
|
-
|
|
3
1
|
## [Unreleased]
|
|
4
2
|
|
|
5
|
-
### 修复
|
|
6
|
-
|
|
7
|
-
- Backend-test 节点 10 `backend-test-semantic-gate-shell` 不再用 Markdown `first-verdict-line` 读取 raw Pi 输出;改为同一 `semantic-initial` pipeline 按 task id 只读 `contracts/backend-test-semantic-review.json`,仅 `verdict=pass` 放行 pytest,`request-revision`/缺失/malformed 清晰 fail-closed;节点 09 仍只负责 materialize+trace,15 节点与单次 pytest 不变。
|
|
8
|
-
- 公开 Case Manifest schema(`docs/templates/backend-test-case-manifest.schema.json`)对齐 materializer dual-scope coverage:`coverageSummary` 在 `additionalProperties:false` 下 required 并声明 `global`/`inScope`(`total`/`covered`/`gapped`/`ratio`)与 `crossDomainGapCount`;回归测试以无依赖轻量 contract 锁定 schema↔canonical 一致性;不改 materializer/15 节点。
|
|
9
|
-
- Backend-test Case Manifest materializer 将模型顶层 `coverageSummary` 视为非权威:secret 扫描后剥离,再 strict parse 其余字段,并始终 `computeCaseManifestCoverageSummary` 写入 canonical summary;my-webapp 自定义/部分 summary 不再导致 invalid-output;prompt 禁止模型输出 coverageSummary;15 节点与单次 pytest 不变。
|
|
10
|
-
- Backend-test 裸 object 路径现消费 scanner 的 `unclosedObject`:当文本含一个完整裸 JSON object,且其前或后仍有未闭合裸 object 时一律 fail-closed,不再因 `spans.length === 1` 错误接受完整对象;Semantic Review / Classification / contracts envelope 共用该行为。
|
|
11
|
-
- Backend-test contracts composite pipeline (`validate-backend-test-contracts-shell`) 使用唯一确定性 envelope 提取:接受 pure JSON、唯一 fenced json、prose+唯一 fence、prose+唯一括号平衡裸 object;对无 JSON、多候选、未闭合、非 object、缺 `analysis`/`execution` fail-closed。提取后仍走 Analysis v2 / Execution v1 schema、source-binding 与 secret 门禁;不引入全局 Pi output rewrite,不将 `invalid-output` 加入自动 retry,不改 15 节点拓扑。
|
|
12
|
-
- Backend-test Semantic Review(08→09)与 Classification(12→13)materializer 复用同一 `extractUniqueJsonObject` scanner:接受 pure / whole-fence / prose+唯一 fence / prose+唯一裸 object;多候选、未闭合、非 object 与 strict schema 错误继续 fail-closed;不改 Case Manifest、Pi SDK/retry 或 15 节点拓扑。
|
|
13
|
-
|
|
14
3
|
### 改进
|
|
15
4
|
|
|
5
|
+
- GitHub Actions CI 降低私有仓分钟消耗:合并为单 job(避免两次 `npm ci`)、同分支/`PR` concurrency 取消旧 run、Draft PR 跳过完整 CI,且仅忽略 `docs/reports/**`、`docs/progress/**`、`docs/design/archive/**`、`docs/exec-plans/completed/**` 等纯运营文档路径;治理文档与代码门禁仍全量跑。
|
|
6
|
+
- 明确 **Compatibility / Operator Assist**:openCode 等主会话只编排已发布 `loop-agent` / `agent-worker` CLI 与只读诊断,不得绕过 CLI 直接改业务实现,失败只走 doctor / reconcile / human gate / CLI 重跑。`loop-agent init` 写入的 `AGENTS.md` managed block、包内 skills 与 website 快速开始/治理说明已对齐;架构决策见 `docs/decisions/0005-governed-operator-surface.md`(accepted)。已有目标项目用 `loop-agent init check-update` / `init update --apply-safe` 或 `init reconcile` 刷新 managed block。
|
|
16
7
|
- 前端测试 DAG 收紧体验:execution preflight 仅硬校验绝对非生产 `baseUrl`;用例 map 缩短为优先用 `playwright-cli` 执行;复盘合并执行证据审查且不依赖 outcome=pass,失败也能出报告。
|
|
17
8
|
- 前端测试用例生成现在会解析明确 `baseUrl`:优先读取任务源 `config.md` 中的前端 URL,缺失时默认 `http://localhost:5173`,并写入 RAG `context.md`;生成命令不得再保留 `<base-url>` 占位符。
|
|
18
9
|
- 前端浏览器测试 DAG 的复盘报告现写入 `testcase/frontend/reports/**`,不再要求 `docs/test-reports/**` 权限;任务执行约束可以安全禁止整个 `docs/**`,同时仍保留可审计的测试资产。
|
|
@@ -24,6 +15,15 @@
|
|
|
24
15
|
|
|
25
16
|
### 修复
|
|
26
17
|
|
|
18
|
+
- Backend-test Analysis v2 intake 现在可确定性归一化模型常见的外部依赖 `type` / `target` 与风险 `type` 字段:dependency `target` 映射为 canonical `name`,扩展字段不落盘;dependency/risk 仍使用类别化白名单,未知字段和 `risks[].target` 继续 fail-closed,避免节点 02 因合法近似形态过早中断。
|
|
19
|
+
|
|
20
|
+
- Backend-test 唯一 JSON object 提取器在扫描裸对象前会等长忽略完整 Markdown 行内代码中的对象形状示例(如 `{error:{code,message,type}}`);唯一真实 envelope 可正常物化,多个真实 JSON、fence 外额外对象与未闭合对象继续 fail-closed。
|
|
21
|
+
- Backend-test 节点 10 `backend-test-semantic-gate-shell` 不再用 Markdown `first-verdict-line` 读取 raw Pi 输出;改为同一 `semantic-initial` pipeline 按 task id 只读 `contracts/backend-test-semantic-review.json`,仅 `verdict=pass` 放行 pytest,`request-revision`/缺失/malformed 清晰 fail-closed;节点 09 仍只负责 materialize+trace,15 节点与单次 pytest 不变。
|
|
22
|
+
- 公开 Case Manifest schema(`docs/templates/backend-test-case-manifest.schema.json`)对齐 materializer dual-scope coverage:`coverageSummary` 在 `additionalProperties:false` 下 required 并声明 `global`/`inScope`(`total`/`covered`/`gapped`/`ratio`)与 `crossDomainGapCount`;回归测试以无依赖轻量 contract 锁定 schema↔canonical 一致性;不改 materializer/15 节点。
|
|
23
|
+
- Backend-test Case Manifest materializer 将模型顶层 `coverageSummary` 视为非权威:secret 扫描后剥离,再 strict parse 其余字段,并始终 `computeCaseManifestCoverageSummary` 写入 canonical summary;my-webapp 自定义/部分 summary 不再导致 invalid-output;prompt 禁止模型输出 coverageSummary;15 节点与单次 pytest 不变。
|
|
24
|
+
- Backend-test 裸 object 路径现消费 scanner 的 `unclosedObject`:当文本含一个完整裸 JSON object,且其前或后仍有未闭合裸 object 时一律 fail-closed,不再因 `spans.length === 1` 错误接受完整对象;Semantic Review / Classification / contracts envelope 共用该行为。
|
|
25
|
+
- Backend-test contracts composite pipeline (`validate-backend-test-contracts-shell`) 使用唯一确定性 envelope 提取:接受 pure JSON、唯一 fenced json、prose+唯一 fence、prose+唯一括号平衡裸 object;对无 JSON、多候选、未闭合、非 object、缺 `analysis`/`execution` fail-closed。提取后仍走 Analysis v2 / Execution v1 schema、source-binding 与 secret 门禁;不引入全局 Pi output rewrite,不将 `invalid-output` 加入自动 retry,不改 15 节点拓扑。
|
|
26
|
+
- Backend-test Semantic Review(08→09)与 Classification(12→13)materializer 复用同一 `extractUniqueJsonObject` scanner:接受 pure / whole-fence / prose+唯一 fence / prose+唯一裸 object;多候选、未闭合、非 object 与 strict schema 错误继续 fail-closed;不改 Case Manifest、Pi SDK/retry 或 15 节点拓扑。
|
|
27
27
|
- 节点检查器「执行过程」中的事件时间戳现在同时显示本地日期与 24 小时制 `HH:mm:ss`(例如 `2026/7/16 08:09:10`),不再被裁掉时间部分;缺少时间戳时仍显示「未记录时间」,事件字段优先级(`timestamp` → `at` → `recordedAt`)保持不变。
|
|
28
28
|
- `frontend-test` 的 case review 现在是 fail-closed browser gate:只有 `VERDICT: pass` 才能物化 manifest 并启动动态 browser map;生成的 case 固定使用默认 browser session,要求每个子场景的 fixture/UI reset 和 fresh snapshot,并由确定性节点校验每个 case 的 `execution.md`、`case-result.json`、`caseId`、`status`、`evidencePaths` 及 blocked `blockedReason`。
|
|
29
29
|
- 前端实现计划/修订节点会注入当前包内权威 `frontend-implementation-contract-v1` Schema 与固定 source binding,避免模型猜测字段导致契约门禁失败。
|
|
@@ -35,6 +35,23 @@
|
|
|
35
35
|
- Pi SDK 执行长推理或大段结构化输出时不再把高频流式增量事件无界累积到内存;同一响应在多个生命周期事件中重复出现的 Token 用量只统计一次,避免 `Invalid string length` 和成本数据虚高。
|
|
36
36
|
- 后端测试复合执行节点继续保持 clean environment、失败分类和 fail-closed outcome,并为唯一 JUnit/initial Result、canonical Result、traceability 与 Observe 投影保留结构化运行证据。
|
|
37
37
|
|
|
38
|
+
## [0.16.24] - 2026-07-21
|
|
39
|
+
|
|
40
|
+
### 改进
|
|
41
|
+
|
|
42
|
+
- frontend-test 默认改为**短链 RAG**:`reviewMode` 默认 `off`(机械 `frontend-case-checklist-shell` 放行,不再默认双轮 LLM review 硬门);`strictOutcomeGate` 默认 `false`(outcome 全绿门 opt-in)。**流程验收=最终 retrospect 报告**,不以 case 100% 通过为条件。map 仅在 `dynamicExpansion.tolerateChildFailures=true`(frontend-test 默认开启)时将 child auth/timeout/executor 错误落 case-level blocked/failed evidence 且不炸 map closeout;其它 map 仍 fail-closed。
|
|
43
|
+
- Shell `verdictGate` 改为纯 `node -e` 单次调用,降低 Windows Git Bash 复合脚本崩溃风险。
|
|
44
|
+
|
|
45
|
+
### 修复
|
|
46
|
+
|
|
47
|
+
- Windows DAG Shell/verification 的 clean environment 现在保留 `APPDATA`,使 Python 3.12/pip 可按系统规则发现用户级 `site-packages` 与其中的 pytest;仍不会继承完整宿主环境或透传未授权 token/credential,backend-test 无需硬编码用户路径。
|
|
48
|
+
|
|
49
|
+
## [0.16.23] - 2026-07-21
|
|
50
|
+
|
|
51
|
+
### 修复
|
|
52
|
+
|
|
53
|
+
- DAG runner 在归档到 `completed/` 前停止 heartbeat 并排空 state 写队列。避免 terminal 节点全 FINISHED 后仍因 completed-facts write guard 拒绝写 `state.json` 导致 `run-dag` 非零退出、Worker 记 `run-error`/Task Pool `Blocked`。
|
|
54
|
+
|
|
38
55
|
## [0.16.22] - 2026-07-21
|
|
39
56
|
|
|
40
57
|
### 修复
|
|
@@ -12,13 +12,6 @@ export function expandShellPreset(preset) {
|
|
|
12
12
|
function escapeShellSingleQuoted(value) {
|
|
13
13
|
return value.replace(/'/g, "'\\''");
|
|
14
14
|
}
|
|
15
|
-
const VERDICT_GATE_NODE_PARSER_FIRST_NON_EMPTY = 'node -e \'const fs=require("fs"); const raw=JSON.parse(fs.readFileSync(process.argv[1], "utf8")); const text=raw.assistantText ?? raw.stdout ?? ""; const line=String(text).split(/\\r?\\n/).find((value) => value.trim().length > 0) ?? ""; process.stdout.write(line);\' "${FILE}" || true';
|
|
16
|
-
const VERDICT_GATE_NODE_PARSER_FIRST_VERDICT_LINE = 'node -e \'const fs=require("fs"); const raw=JSON.parse(fs.readFileSync(process.argv[1], "utf8")); const text=raw.assistantText ?? raw.stdout ?? ""; const normalize=(value) => { const trimmed=String(value).trim(); const emphasized=trimmed.match(/^\\*{1,3}\\s*(VERDICT:[^*]+?)\\s*\\*{1,3}$/); return (emphasized ? emphasized[1] : trimmed).trim(); }; const line=String(text).split(/\\r?\\n/).map(normalize).find((value) => /^VERDICT:/.test(value)) ?? ""; process.stdout.write(line);\' "${FILE}" || true';
|
|
17
|
-
function verdictGateNodeParser(lineMode) {
|
|
18
|
-
return lineMode === "first-verdict-line"
|
|
19
|
-
? VERDICT_GATE_NODE_PARSER_FIRST_VERDICT_LINE
|
|
20
|
-
: VERDICT_GATE_NODE_PARSER_FIRST_NON_EMPTY;
|
|
21
|
-
}
|
|
22
15
|
/**
|
|
23
16
|
* Build a single shell command that reads `$HARNESS_DAG_RUN_DIR/<fromNodeId>.json`
|
|
24
17
|
* (injected by the shell executor from runtime meta), parses a single candidate
|
|
@@ -27,33 +20,41 @@ function verdictGateNodeParser(lineMode) {
|
|
|
27
20
|
* - first-non-empty: first non-empty line
|
|
28
21
|
* - first-verdict-line: first line matching /^VERDICT:/ after trim and optional whole-line Markdown emphasis normalization
|
|
29
22
|
*
|
|
30
|
-
*
|
|
23
|
+
* Implemented as one pure `node -e` invocation (no bash state/subshell chaining) so Windows
|
|
24
|
+
* Git Bash cannot crash the gate with STATUS_DLL_INIT_FAILED on long compound scripts.
|
|
31
25
|
*/
|
|
32
26
|
export function buildVerdictGateShellCommand(gate) {
|
|
33
27
|
const gateLabel = gate.label ?? `${gate.fromNodeId} verdict`;
|
|
34
28
|
const lineMode = gate.lineMode ?? "first-non-empty";
|
|
35
|
-
const parser = verdictGateNodeParser(lineMode);
|
|
36
29
|
const candidateNodeIds = [
|
|
37
30
|
gate.fromNodeId,
|
|
38
31
|
...(gate.fallbackFromNodeIds ?? []).filter((id) => id !== gate.fromNodeId),
|
|
39
32
|
];
|
|
40
|
-
const
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
const
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
const
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
.join("
|
|
56
|
-
|
|
33
|
+
const config = {
|
|
34
|
+
label: gateLabel,
|
|
35
|
+
lineMode,
|
|
36
|
+
accept: gate.accept,
|
|
37
|
+
candidateNodeIds,
|
|
38
|
+
};
|
|
39
|
+
const configLiteral = escapeShellSingleQuoted(JSON.stringify(config));
|
|
40
|
+
const program = [
|
|
41
|
+
'const fs=require("fs");',
|
|
42
|
+
'const path=require("path");',
|
|
43
|
+
'const cfg=JSON.parse(process.argv[1]);',
|
|
44
|
+
'const runDir=process.env.HARNESS_DAG_RUN_DIR;',
|
|
45
|
+
'if(!runDir){console.error("missing HARNESS_DAG_RUN_DIR for "+cfg.label+" gate");process.exit(1);}',
|
|
46
|
+
'let file="";',
|
|
47
|
+
'for(const id of cfg.candidateNodeIds){const candidate=path.join(runDir,id+".json");if(fs.existsSync(candidate)){file=candidate;break;}}',
|
|
48
|
+
'if(!file){console.error("missing "+cfg.label+" JSON output (tried: "+cfg.candidateNodeIds.join(", ")+")");process.exit(1);}',
|
|
49
|
+
'const raw=JSON.parse(fs.readFileSync(file,"utf8"));',
|
|
50
|
+
'const text=String(raw.assistantText ?? raw.stdout ?? "");',
|
|
51
|
+
'const normalize=(value)=>{const trimmed=String(value).trim();const emphasized=trimmed.match(/^\\*{1,3}\\s*(VERDICT:[^*]+?)\\s*\\*{1,3}$/);return (emphasized?emphasized[1]:trimmed).trim();};',
|
|
52
|
+
'const lines=text.split(/\\r?\\n/).map(normalize);',
|
|
53
|
+
'const first=cfg.lineMode==="first-verdict-line"? (lines.find((value)=>/^VERDICT:/.test(value))??"") : (lines.find((value)=>value.length>0)??"");',
|
|
54
|
+
'if(!cfg.accept.includes(first)){console.error(cfg.label+" gate blocked: "+(first||"missing VERDICT line"));process.exit(1);}',
|
|
55
|
+
'process.stdout.write(first+"\\n");',
|
|
56
|
+
].join("");
|
|
57
|
+
return `node -e '${escapeShellSingleQuoted(program)}' '${configLiteral}'`;
|
|
57
58
|
}
|
|
58
59
|
/** Build a deterministic current-run gate over exact REQ-/BR-/AC- ids in node facts. */
|
|
59
60
|
export function buildRequirementCoverageGateShellCommand(gate) {
|
|
@@ -65,12 +65,24 @@ export const frontendMockConfigSchema = z.object({
|
|
|
65
65
|
serviceRoot: z.string().min(1).optional(),
|
|
66
66
|
verifyCommands: z.array(frontendMockVerifyCommandSchema).optional().default([]),
|
|
67
67
|
});
|
|
68
|
-
/** Batch limits for the browser-driven frontend test DAG.
|
|
69
|
-
* stop thresholds, not model-provider hard
|
|
68
|
+
/** Batch limits and gate profile for the browser-driven frontend test DAG.
|
|
69
|
+
* Token fields are post-case stop thresholds, not model-provider hard caps. */
|
|
70
70
|
export const frontendTestConfigSchema = z.object({
|
|
71
|
-
maxCasesPerBatch: z.number().int().min(1).max(50).optional()
|
|
71
|
+
maxCasesPerBatch: z.number().int().min(1).max(50).optional(),
|
|
72
72
|
maxTokensPerCase: z.number().int().positive().optional(),
|
|
73
73
|
maxTotalTokens: z.number().int().positive().optional(),
|
|
74
|
+
/**
|
|
75
|
+
* Case review topology:
|
|
76
|
+
* - off (default): mechanical checklist-shell only; no LLM review hard gate
|
|
77
|
+
* - advisory: reserved (same default path as off for now; LLM notes non-blocking)
|
|
78
|
+
* - blocking: legacy dual-review + final VERDICT: pass gate before materialize
|
|
79
|
+
*/
|
|
80
|
+
reviewMode: z.enum(["off", "advisory", "blocking"]).optional(),
|
|
81
|
+
/**
|
|
82
|
+
* When true, include quality outcome gate (all cases passed, 0 blocked).
|
|
83
|
+
* Default false: pipeline success is result materialize + retrospect report, not full green.
|
|
84
|
+
*/
|
|
85
|
+
strictOutcomeGate: z.boolean().optional(),
|
|
74
86
|
});
|
|
75
87
|
export const convergenceConfigSchema = z.object({
|
|
76
88
|
enabled: z.boolean().optional().default(false),
|
|
@@ -299,7 +299,7 @@ function coerceBoundary(value) {
|
|
|
299
299
|
return null;
|
|
300
300
|
return { field, constraint, sourceRef: firstSourceRef(record) };
|
|
301
301
|
}
|
|
302
|
-
const
|
|
302
|
+
const OPTIONAL_EVIDENCE_BASE_KEYS = [
|
|
303
303
|
"name",
|
|
304
304
|
"id",
|
|
305
305
|
"description",
|
|
@@ -308,21 +308,33 @@ const OPTIONAL_EVIDENCE_KNOWN_KEYS = new Set([
|
|
|
308
308
|
"title",
|
|
309
309
|
"sourceRef",
|
|
310
310
|
"sourceRefs",
|
|
311
|
+
];
|
|
312
|
+
const EXTERNAL_DEPENDENCY_INPUT_KEYS = new Set([
|
|
313
|
+
...OPTIONAL_EVIDENCE_BASE_KEYS,
|
|
314
|
+
"type",
|
|
315
|
+
"target",
|
|
316
|
+
"kind",
|
|
317
|
+
"required",
|
|
318
|
+
]);
|
|
319
|
+
const RISK_INPUT_KEYS = new Set([
|
|
320
|
+
...OPTIONAL_EVIDENCE_BASE_KEYS,
|
|
321
|
+
"type",
|
|
311
322
|
"severity",
|
|
312
323
|
"mitigation",
|
|
313
324
|
"level",
|
|
314
325
|
"impact",
|
|
315
|
-
"kind",
|
|
316
|
-
"required",
|
|
317
326
|
]);
|
|
318
|
-
function coerceOptionalEvidence(value) {
|
|
327
|
+
function coerceOptionalEvidence(value, kind) {
|
|
319
328
|
const record = asRecord(value);
|
|
320
329
|
if (!record)
|
|
321
330
|
return null;
|
|
331
|
+
const knownKeys = kind === "external-dependency"
|
|
332
|
+
? EXTERNAL_DEPENDENCY_INPUT_KEYS
|
|
333
|
+
: RISK_INPUT_KEYS;
|
|
322
334
|
// Fail closed on unknown keys so near-schema payloads cannot strip extras and pass.
|
|
323
335
|
for (const key of Object.keys(record)) {
|
|
324
|
-
if (!
|
|
325
|
-
throw new Error(
|
|
336
|
+
if (!knownKeys.has(key)) {
|
|
337
|
+
throw new Error(`${kind} evidence has unrecognized key: ${key}`);
|
|
326
338
|
}
|
|
327
339
|
}
|
|
328
340
|
const description = (typeof record.description === "string" && record.description.trim()) ||
|
|
@@ -334,6 +346,7 @@ function coerceOptionalEvidence(value) {
|
|
|
334
346
|
return null;
|
|
335
347
|
const out = { description };
|
|
336
348
|
const name = (typeof record.name === "string" && record.name.trim()) ||
|
|
349
|
+
(kind === "external-dependency" && typeof record.target === "string" && record.target.trim()) ||
|
|
337
350
|
(typeof record.id === "string" && record.id.trim()) ||
|
|
338
351
|
"";
|
|
339
352
|
if (name)
|
|
@@ -385,10 +398,10 @@ export function coerceBackendTestAnalysisInput(value) {
|
|
|
385
398
|
.map(coerceBoundary)
|
|
386
399
|
.filter((item) => item !== null);
|
|
387
400
|
next.externalDependencies = asArray(record.externalDependencies)
|
|
388
|
-
.map(coerceOptionalEvidence)
|
|
401
|
+
.map((item) => coerceOptionalEvidence(item, "external-dependency"))
|
|
389
402
|
.filter((item) => item !== null);
|
|
390
403
|
next.risks = asArray(record.risks)
|
|
391
|
-
.map(coerceOptionalEvidence)
|
|
404
|
+
.map((item) => coerceOptionalEvidence(item, "risk"))
|
|
392
405
|
.filter((item) => item !== null);
|
|
393
406
|
next.evidenceGaps = asArray(record.evidenceGaps)
|
|
394
407
|
.map(coerceEvidenceGap)
|
|
@@ -6,6 +6,44 @@
|
|
|
6
6
|
* Semantic review / classification materializers reuse the same scanner.
|
|
7
7
|
*/
|
|
8
8
|
const JSON_FENCE_RE = /```json\s*\r?\n([\s\S]*?)\r?\n```/gi;
|
|
9
|
+
/**
|
|
10
|
+
* Mask complete Markdown inline-code spans without changing source offsets.
|
|
11
|
+
* Object-shaped examples inside backticks are prose, not JSON candidates.
|
|
12
|
+
* Unclosed delimiters stay visible so malformed output cannot hide conflicts.
|
|
13
|
+
*/
|
|
14
|
+
function maskMarkdownInlineCode(text) {
|
|
15
|
+
const chars = text.split("");
|
|
16
|
+
for (let index = 0; index < text.length;) {
|
|
17
|
+
if (text[index] !== "`") {
|
|
18
|
+
index += 1;
|
|
19
|
+
continue;
|
|
20
|
+
}
|
|
21
|
+
let delimiterLength = 1;
|
|
22
|
+
while (text[index + delimiterLength] === "`")
|
|
23
|
+
delimiterLength += 1;
|
|
24
|
+
if (delimiterLength >= 3) {
|
|
25
|
+
index += delimiterLength;
|
|
26
|
+
continue;
|
|
27
|
+
}
|
|
28
|
+
const delimiter = "`".repeat(delimiterLength);
|
|
29
|
+
let close = text.indexOf(delimiter, index + delimiterLength);
|
|
30
|
+
while (close >= 0 &&
|
|
31
|
+
(text[close - 1] === "`" || text[close + delimiterLength] === "`")) {
|
|
32
|
+
close = text.indexOf(delimiter, close + delimiterLength);
|
|
33
|
+
}
|
|
34
|
+
if (close < 0) {
|
|
35
|
+
index += delimiterLength;
|
|
36
|
+
continue;
|
|
37
|
+
}
|
|
38
|
+
const end = close + delimiterLength;
|
|
39
|
+
for (let cursor = index; cursor < end; cursor += 1) {
|
|
40
|
+
if (chars[cursor] !== "\n" && chars[cursor] !== "\r")
|
|
41
|
+
chars[cursor] = " ";
|
|
42
|
+
}
|
|
43
|
+
index = end;
|
|
44
|
+
}
|
|
45
|
+
return chars.join("");
|
|
46
|
+
}
|
|
9
47
|
function isPlainObject(value) {
|
|
10
48
|
return value !== null && typeof value === "object" && !Array.isArray(value);
|
|
11
49
|
}
|
|
@@ -121,7 +159,7 @@ function findUniqueJsonObjectText(text) {
|
|
|
121
159
|
if (fences.length === 1) {
|
|
122
160
|
const fence = fences[0];
|
|
123
161
|
const outsideScans = [trimmed.slice(0, fence.start), trimmed.slice(fence.end)]
|
|
124
|
-
.map((fragment) => scanBalancedObjectSpans(fragment));
|
|
162
|
+
.map((fragment) => scanBalancedObjectSpans(maskMarkdownInlineCode(fragment)));
|
|
125
163
|
if (outsideScans.some((scan) => scan.unclosedObject)) {
|
|
126
164
|
throw new Error("backend-test unique JSON object: unclosed JSON object outside fenced json block");
|
|
127
165
|
}
|
|
@@ -131,7 +169,7 @@ function findUniqueJsonObjectText(text) {
|
|
|
131
169
|
}
|
|
132
170
|
return fence.body;
|
|
133
171
|
}
|
|
134
|
-
const scan = scanBalancedObjectSpans(trimmed);
|
|
172
|
+
const scan = scanBalancedObjectSpans(maskMarkdownInlineCode(trimmed));
|
|
135
173
|
if (scan.unclosedObject) {
|
|
136
174
|
throw new Error("backend-test unique JSON object: unclosed JSON object; expected one unique balanced object");
|
|
137
175
|
}
|
|
@@ -81,27 +81,77 @@ export function resolveRunLocalPath(runDir, ref, label) {
|
|
|
81
81
|
}
|
|
82
82
|
return resolved;
|
|
83
83
|
}
|
|
84
|
+
function resolveCaseIdFromItem(item) {
|
|
85
|
+
if (typeof item === "object" &&
|
|
86
|
+
item !== null &&
|
|
87
|
+
typeof item.caseId === "string") {
|
|
88
|
+
return item.caseId;
|
|
89
|
+
}
|
|
90
|
+
return String(item);
|
|
91
|
+
}
|
|
92
|
+
async function materializeBlockedCaseEvidence(input) {
|
|
93
|
+
if (!input.workspaceRef)
|
|
94
|
+
return;
|
|
95
|
+
const workspace = resolveRunLocalPath(input.cwd, input.workspaceRef, "workspaceRef");
|
|
96
|
+
await mkdir(workspace, { recursive: true });
|
|
97
|
+
const resultPath = path.join(workspace, "case-result.json");
|
|
98
|
+
const caseId = resolveCaseIdFromItem(input.item);
|
|
99
|
+
try {
|
|
100
|
+
const existing = JSON.parse(await readFile(resultPath, "utf-8"));
|
|
101
|
+
if (existing.caseId === caseId &&
|
|
102
|
+
(existing.status === "passed" ||
|
|
103
|
+
existing.status === "failed" ||
|
|
104
|
+
existing.status === "blocked"))
|
|
105
|
+
return;
|
|
106
|
+
}
|
|
107
|
+
catch {
|
|
108
|
+
// Missing or malformed evidence is replaced only for a child that never finished a business result.
|
|
109
|
+
}
|
|
110
|
+
await writeFile(path.join(workspace, "execution.md"), `# ${caseId}\n\nStatus: blocked\n\nReason: ${input.reason}\n`, "utf-8");
|
|
111
|
+
await writeFile(resultPath, `${JSON.stringify({
|
|
112
|
+
caseId,
|
|
113
|
+
status: "blocked",
|
|
114
|
+
blockedReason: input.reason,
|
|
115
|
+
evidencePaths: ["execution.md"],
|
|
116
|
+
}, null, 2)}\n`, "utf-8");
|
|
117
|
+
}
|
|
84
118
|
async function materializeTokenBudgetBlockedEvidence(input) {
|
|
119
|
+
await materializeBlockedCaseEvidence({
|
|
120
|
+
...input,
|
|
121
|
+
reason: "token-budget-exhausted",
|
|
122
|
+
});
|
|
123
|
+
}
|
|
124
|
+
async function materializeFailedCaseEvidence(input) {
|
|
85
125
|
if (!input.workspaceRef)
|
|
86
126
|
return;
|
|
87
127
|
const workspace = resolveRunLocalPath(input.cwd, input.workspaceRef, "workspaceRef");
|
|
88
128
|
await mkdir(workspace, { recursive: true });
|
|
89
129
|
const resultPath = path.join(workspace, "case-result.json");
|
|
90
|
-
const caseId =
|
|
91
|
-
typeof input.item.caseId === "string"
|
|
92
|
-
? input.item.caseId
|
|
93
|
-
: String(input.item);
|
|
130
|
+
const caseId = resolveCaseIdFromItem(input.item);
|
|
94
131
|
try {
|
|
95
132
|
const existing = JSON.parse(await readFile(resultPath, "utf-8"));
|
|
96
133
|
if (existing.caseId === caseId &&
|
|
97
|
-
(existing.status === "passed" ||
|
|
134
|
+
(existing.status === "passed" ||
|
|
135
|
+
existing.status === "failed" ||
|
|
136
|
+
existing.status === "blocked"))
|
|
98
137
|
return;
|
|
99
138
|
}
|
|
100
139
|
catch {
|
|
101
|
-
// Missing or malformed evidence is replaced only
|
|
140
|
+
// Missing or malformed evidence is replaced only when the child never finished a business result.
|
|
102
141
|
}
|
|
103
|
-
await writeFile(path.join(workspace, "execution.md"), `# ${caseId}\n\nStatus:
|
|
104
|
-
await writeFile(resultPath, `${JSON.stringify({
|
|
142
|
+
await writeFile(path.join(workspace, "execution.md"), `# ${caseId}\n\nStatus: failed\n\nReason: ${input.reason}\n`, "utf-8");
|
|
143
|
+
await writeFile(resultPath, `${JSON.stringify({
|
|
144
|
+
caseId,
|
|
145
|
+
status: "failed",
|
|
146
|
+
errorSummary: input.reason,
|
|
147
|
+
evidencePaths: ["execution.md"],
|
|
148
|
+
}, null, 2)}\n`, "utf-8");
|
|
149
|
+
}
|
|
150
|
+
/** Provider auth/timeout on a FE case child is infrastructure, not a product fail; keep the map aggregate. */
|
|
151
|
+
function isInfrastructureCaseChildFailure(record) {
|
|
152
|
+
if (!record || record.status === "FINISHED")
|
|
153
|
+
return false;
|
|
154
|
+
return record.failureCategory === "auth" || record.failureCategory === "timeout";
|
|
105
155
|
}
|
|
106
156
|
export async function executeDynamicMapExpansion(input) {
|
|
107
157
|
const started = Date.now();
|
|
@@ -264,15 +314,64 @@ export async function executeDynamicMapExpansion(input) {
|
|
|
264
314
|
tokenBudgetExhausted = true;
|
|
265
315
|
}
|
|
266
316
|
}
|
|
317
|
+
// Opt-in (frontend-test via tolerateChildFailures): materialize case-level
|
|
318
|
+
// failed/blocked evidence and keep the map barrier green so validate /
|
|
319
|
+
// result / retrospect can still run. Default maps stay fail-closed on child ERROR.
|
|
320
|
+
const tolerateChildFailures = input.expansion.tolerateChildFailures === true;
|
|
321
|
+
const caseOutcomeNotes = [];
|
|
322
|
+
if (tolerateChildFailures) {
|
|
323
|
+
for (let index = 0; index < childNodeIds.length; index += 1) {
|
|
324
|
+
const nodeId = childNodeIds[index];
|
|
325
|
+
const record = input.state.nodes[nodeId];
|
|
326
|
+
if (!record || record.status === "FINISHED")
|
|
327
|
+
continue;
|
|
328
|
+
if (isBudgetBlockedRecord(record))
|
|
329
|
+
continue;
|
|
330
|
+
const infra = isInfrastructureCaseChildFailure(record);
|
|
331
|
+
const reason = infra
|
|
332
|
+
? record?.failureCategory === "timeout"
|
|
333
|
+
? "executor-timeout"
|
|
334
|
+
: "executor-auth-unavailable"
|
|
335
|
+
: `executor-error:${record.failureCategory || record.status || "unknown"}`;
|
|
336
|
+
const outcomeStatus = infra ? "blocked" : "failed";
|
|
337
|
+
try {
|
|
338
|
+
if (outcomeStatus === "blocked") {
|
|
339
|
+
await materializeBlockedCaseEvidence({
|
|
340
|
+
cwd: input.cwd,
|
|
341
|
+
workspaceRef: workspaceRefs[index],
|
|
342
|
+
item: items[index],
|
|
343
|
+
reason,
|
|
344
|
+
});
|
|
345
|
+
}
|
|
346
|
+
else {
|
|
347
|
+
await materializeFailedCaseEvidence({
|
|
348
|
+
cwd: input.cwd,
|
|
349
|
+
workspaceRef: workspaceRefs[index],
|
|
350
|
+
item: items[index],
|
|
351
|
+
reason,
|
|
352
|
+
});
|
|
353
|
+
}
|
|
354
|
+
}
|
|
355
|
+
catch (error) {
|
|
356
|
+
throw new Error(`failed to materialize case outcome evidence for ${nodeId}: ${error instanceof Error ? error.message : String(error)}`);
|
|
357
|
+
}
|
|
358
|
+
caseOutcomeNotes.push({ nodeId, reason, status: outcomeStatus });
|
|
359
|
+
record.status = "FINISHED";
|
|
360
|
+
record.stderr = `${outcomeStatus}: ${reason}`;
|
|
361
|
+
record.failureCategory = "success";
|
|
362
|
+
}
|
|
363
|
+
}
|
|
267
364
|
const budgetBlockedIds = new Set(blockedChildren.map((entry) => entry.nodeId));
|
|
268
|
-
const failedChildren =
|
|
269
|
-
|
|
270
|
-
|
|
271
|
-
|
|
272
|
-
|
|
273
|
-
|
|
274
|
-
|
|
275
|
-
|
|
365
|
+
const failedChildren = tolerateChildFailures
|
|
366
|
+
? []
|
|
367
|
+
: childNodeIds.filter((nodeId) => {
|
|
368
|
+
const record = input.state.nodes[nodeId];
|
|
369
|
+
if (!record || record.status === "FINISHED")
|
|
370
|
+
return false;
|
|
371
|
+
if (budgetBlockedIds.has(nodeId) || isBudgetBlockedRecord(record))
|
|
372
|
+
return false;
|
|
373
|
+
return true;
|
|
374
|
+
});
|
|
276
375
|
const aggregate = {
|
|
277
376
|
workflowNodeId: input.expansion.workflowNodeId,
|
|
278
377
|
itemCount: items.length,
|
|
@@ -285,21 +384,29 @@ export async function executeDynamicMapExpansion(input) {
|
|
|
285
384
|
output: parseJsonFromText(input.state.nodes[nodeId]?.stdout),
|
|
286
385
|
assistantText: input.state.nodes[nodeId]?.assistantText,
|
|
287
386
|
blocked: blockedChildren.find((child) => child.nodeId === nodeId)?.reason ??
|
|
387
|
+
caseOutcomeNotes.find((child) => child.nodeId === nodeId && child.status === "blocked")
|
|
388
|
+
?.reason ??
|
|
288
389
|
(isBudgetBlockedRecord(input.state.nodes[nodeId])
|
|
289
390
|
? "token-budget-exhausted"
|
|
290
391
|
: undefined),
|
|
392
|
+
failed: caseOutcomeNotes.find((child) => child.nodeId === nodeId && child.status === "failed")
|
|
393
|
+
?.reason,
|
|
291
394
|
})),
|
|
292
395
|
tokensUsed: totalTokensUsed,
|
|
396
|
+
caseOutcomeNotes: caseOutcomeNotes.length > 0 ? caseOutcomeNotes : undefined,
|
|
293
397
|
};
|
|
294
398
|
return {
|
|
295
|
-
//
|
|
296
|
-
//
|
|
297
|
-
// Real child failures remain fail-closed even when later cases are budget-blocked.
|
|
399
|
+
// With tolerateChildFailures: barrier always succeeds; product outcomes live in
|
|
400
|
+
// case evidence + result materialization. Without it: real child failures fail-close.
|
|
298
401
|
ok: failedChildren.length === 0,
|
|
299
402
|
stdout: JSON.stringify(aggregate),
|
|
300
|
-
stderr:
|
|
301
|
-
? `
|
|
302
|
-
|
|
403
|
+
stderr: caseOutcomeNotes.length > 0
|
|
404
|
+
? `map children recorded as case outcomes: ${caseOutcomeNotes
|
|
405
|
+
.map((entry) => `${entry.nodeId}=${entry.status}:${entry.reason}`)
|
|
406
|
+
.join(", ")}`
|
|
407
|
+
: failedChildren.length > 0
|
|
408
|
+
? `dynamic map children failed: ${failedChildren.join(", ")}`
|
|
409
|
+
: "",
|
|
303
410
|
failureCategory: failedChildren.length > 0 ? "dynamic-expansion-child-failed" : "success",
|
|
304
411
|
durationMs: Date.now() - started,
|
|
305
412
|
};
|