@tea-agent/loop-agent 0.16.15-beta.0 → 0.16.16
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +58 -0
- package/dist/executors/pi-executor.js +0 -4
- package/dist/executors/shell-executor.js +10 -0
- package/dist/worker/delivery/git-transaction.js +4 -1
- package/dist/worker/materialize/harness-task-materializer.js +14 -1
- package/dist/worker/outcomes/adapters.js +26 -17
- package/dist/worker/outcomes/projector.js +21 -0
- package/dist/worker/run-task/run-task.js +8 -2
- package/dist/workflows/dag/backend-test-execution-contract.js +37 -6
- package/dist/workflows/dag/dynamic-runtime/map.js +33 -1
- package/dist/workflows/dag/dynamic-runtime/shared.js +25 -8
- package/dist/workflows/dag/frontend-test-result-contract.js +165 -0
- package/dist/workflows/dag/init-hybrid.js +153 -21
- package/dist/workflows/dag/node-execution.js +14 -0
- package/dist/workflows/dag/types.js +1 -0
- package/docs/templates/frontend-test-dag.generate-cases.prompt.md +18 -1
- package/docs/templates/frontend-test-dag.json +399 -9
- package/docs/templates/frontend-test-dag.retrospect.prompt.md +1 -1
- package/docs/templates/frontend-test-dag.review-cases.prompt.md +3 -1
- package/docs/templates/frontend-test-dag.review-execution.prompt.md +1 -1
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -4,11 +4,15 @@
|
|
|
4
4
|
|
|
5
5
|
### 改进
|
|
6
6
|
|
|
7
|
+
- 前端浏览器测试 DAG 的复盘报告现写入 `testcase/frontend/reports/**`,不再要求 `docs/test-reports/**` 权限;任务执行约束可以安全禁止整个 `docs/**`,同时仍保留可审计的测试资产。
|
|
8
|
+
- `frontend-test` 结果链新增 run-owned `frontend-test-result-v1`、单次用例修订与终审门禁;只有所有浏览器用例和 AC 覆盖通过且结果合同明确为 real/pass 时,Worker 才会投影真实集成。
|
|
9
|
+
|
|
7
10
|
- 后端测试 DAG 从 38 个收敛为 24 个真实顶层节点;使用 fail-closed `runIf` 和复合 Shell capability 减少调度,同时保留双合同、Manifest、语义评审、JUnit、Result、分类、修复安全、追踪和最终 outcome 证据。三条可选修订/修复分支仍各最多执行一次。
|
|
8
11
|
- Backend Test Analysis、Case Manifest、Semantic Review 与 Classification 的严格 JSON 契约进一步对齐,生成和修订节点明确字段白名单、`sourceBinding` 与 evidence gap 约束,避免模型自定义字段导致确定性门禁失败。
|
|
9
12
|
|
|
10
13
|
### 修复
|
|
11
14
|
|
|
15
|
+
- `frontend-test` 的 case review 现在是 fail-closed browser gate:只有 `VERDICT: pass` 才能物化 manifest 并启动动态 browser map;生成的 case 固定使用默认 browser session,要求每个子场景的 fixture/UI reset 和 fresh snapshot,并由确定性节点校验每个 case 的 `execution.md`、`case-result.json`、`caseId`、`status`、`evidencePaths` 及 blocked `blockedReason`。
|
|
12
16
|
- 前端实现计划/修订节点会注入当前包内权威 `frontend-implementation-contract-v1` Schema 与固定 source binding,避免模型猜测字段导致契约门禁失败。
|
|
13
17
|
- 前端 Mock 策略节点的 canonical 输出会把首条 `MOCK_STRATEGY:` 协议行提升为第一行,避免解释性前言触发 `first-non-empty` 门禁误判。
|
|
14
18
|
- 前端规范回退目录统一为本地 `openspec/`,DAG 能力发现、提示词、Skill 与验证证据检查不再查找大小写不一致的旧目录名。
|
|
@@ -18,6 +22,60 @@
|
|
|
18
22
|
- Pi SDK 执行长推理或大段结构化输出时不再把高频流式增量事件无界累积到内存;同一响应在多个生命周期事件中重复出现的 Token 用量只统计一次,避免 `Invalid string length` 和成本数据虚高。
|
|
19
23
|
- 后端测试复合执行节点继续保持 clean environment、失败分类和 fail-closed outcome,并为 initial/final Result、repair eligibility、traceability 与 Observe 投影保留结构化运行证据。
|
|
20
24
|
|
|
25
|
+
## [0.16.16] - 2026-07-20
|
|
26
|
+
|
|
27
|
+
### 修复
|
|
28
|
+
|
|
29
|
+
- 动态 DAG `itemsFrom` / JSON 选择器从 shell 节点 stdout 解析时,可跳过命令回显行并提取末尾 JSON 对象,避免 frontend-test `execute-frontend-cases-map` 在 materialize 门成功后仍因 `$ node -e ...` 前缀失败。
|
|
30
|
+
|
|
31
|
+
## [0.16.15] - 2026-07-20
|
|
32
|
+
|
|
33
|
+
### 修复
|
|
34
|
+
|
|
35
|
+
- frontend-test case manifest 校验接受 `evidenceDir` 为 `testcase/frontend/evidence/<caseId>` 目录根(不必强制尾斜杠/子路径),避免生成节点写出合法目录后被 materialize 门误杀。
|
|
36
|
+
|
|
37
|
+
## [0.16.14] - 2026-07-20
|
|
38
|
+
|
|
39
|
+
### 修复
|
|
40
|
+
|
|
41
|
+
- `frontend-test` materialize / path audit 自动并入 `testcase/frontend/**` 与 `docs/test-reports/**`,与 hybrid frontend-test DAG 生成约束对齐,避免 TaskSpec 仅声明 `tests/e2e/**` 时 `dag run-task` 直接失败。
|
|
42
|
+
|
|
43
|
+
## [0.16.13] - 2026-07-20
|
|
44
|
+
|
|
45
|
+
### 修复
|
|
46
|
+
|
|
47
|
+
- backend-test 的 `finalize-effective-result` 管道在节点完成后绑定 `contracts/backend-test-result.json` 为 structured artifact,使 Outcome 投影出 `backend-test-result`,解除 FE-TEST Ready Planner 对 BE-TEST 产物门的误阻断。
|
|
48
|
+
|
|
49
|
+
## [0.16.12] - 2026-07-20
|
|
50
|
+
|
|
51
|
+
### 修复
|
|
52
|
+
|
|
53
|
+
- `backend-test` materialize / Git checkpoint 自动并入 `testcase/**`(与 `docs/test-reports/**`)运行时写根,避免 TaskSpec 仍写 `tests/api/**` 时绿跑无法 promote、任务卡在 Running。
|
|
54
|
+
|
|
55
|
+
## [0.16.11] - 2026-07-20
|
|
56
|
+
|
|
57
|
+
### 修复
|
|
58
|
+
|
|
59
|
+
- Worker 报告决策不再把条件分支的 `SKIPPED` 节点当成失败;backend-test 等混合 DAG 在 revise/repair 未选中时,只要 ERROR 不存在且 run finished,即可 `report-completed` 并进入 Outcome/promote。
|
|
60
|
+
|
|
61
|
+
## [0.16.10] - 2026-07-20
|
|
62
|
+
|
|
63
|
+
### 修复
|
|
64
|
+
|
|
65
|
+
- 后端测试执行合同物化会清理 `managedCommand` 中的空字符串(如 `stop: ""`),避免 near-schema 侦察输出在 strict parse 下误拦 in-process BE-TEST。
|
|
66
|
+
|
|
67
|
+
## [0.16.9] - 2026-07-20
|
|
68
|
+
|
|
69
|
+
### 修复
|
|
70
|
+
|
|
71
|
+
- Outcome 结构化产物声明不再绑定节点 mint-time sha256;同一路径若被后续 gate(如 backend-test traceability)就地改写,投影时以磁盘重算 hash 为准,消除 BE-TEST 完成态仍被 `artifact sha256 mismatch` 误拦。
|
|
72
|
+
|
|
73
|
+
## [0.16.8] - 2026-07-20
|
|
74
|
+
|
|
75
|
+
### 修复
|
|
76
|
+
|
|
77
|
+
- Outcome 投影对同一 run-owned 结构化产物路径的多次 rewrite(如 backend-test case manifest initial→final)只保留最后一次 hash 声明,避免 `artifact sha256 mismatch` 误拦已完成的 BE-TEST。
|
|
78
|
+
|
|
21
79
|
## [0.16.7] - 2026-07-20
|
|
22
80
|
|
|
23
81
|
### 修复
|
|
@@ -186,10 +186,6 @@ function resolvePiCliPath() {
|
|
|
186
186
|
path.join(nvmBase, nodeVersion, 'lib', 'node_modules'),
|
|
187
187
|
].flatMap((nodeModulesRoot) => PI_CLI_PACKAGE_NAMES.map((packageName) => piCliPathFromPackageRoot(nodeModulesRoot, packageName)));
|
|
188
188
|
const otherCandidates = [
|
|
189
|
-
// Prefer the project-local Pi dependency. `npm run` adds node_modules/.bin
|
|
190
|
-
// to PATH, but direct invocations from the DAG runner may not, so resolve
|
|
191
|
-
// the package from the current workspace explicitly.
|
|
192
|
-
...PI_CLI_PACKAGE_NAMES.map((packageName) => piCliPathFromPackageRoot(path.resolve(process.cwd(), 'node_modules'), packageName)),
|
|
193
189
|
// Global npm installation (non-nvm)
|
|
194
190
|
...PI_CLI_PACKAGE_NAMES.map((packageName) => piCliPathFromPackageRoot('/usr/local/lib/node_modules', packageName)),
|
|
195
191
|
// Bun installation
|
|
@@ -7,6 +7,7 @@ import { truncateOutput } from "../shared/output-truncation.js";
|
|
|
7
7
|
import { buildRequirementCoverageGateShellCommand, expandShellPreset, buildVerdictGateShellCommand, } from "./shell-presets.js";
|
|
8
8
|
import { materializeBackendTestAnalysisContract } from "../workflows/dag/backend-test-analysis-contract.js";
|
|
9
9
|
import { materializeFrontendImplementationContract } from "../workflows/dag/frontend-implementation-contract.js";
|
|
10
|
+
import { materializeFrontendTestResult, } from "../workflows/dag/frontend-test-result-contract.js";
|
|
10
11
|
import { formatFrontendVerificationTraceStdout, runFrontendVerificationTraceGate, } from "../workflows/dag/frontend-verification-trace.js";
|
|
11
12
|
import { formatFrontendWorktreeDiffStdout, runFrontendWorktreeDiffGate, } from "../workflows/dag/frontend-worktree-diff.js";
|
|
12
13
|
import { formatFrontendFailureAssessStdout, formatFrontendRepairContractStdout, runFrontendFailureAssessGate, runFrontendRepairContractGate, } from "../workflows/dag/frontend-repair.js";
|
|
@@ -544,6 +545,15 @@ export async function executeDagShellNode(input, meta) {
|
|
|
544
545
|
sourceBinding: meta.spec.sourceBinding,
|
|
545
546
|
});
|
|
546
547
|
break;
|
|
548
|
+
case "frontend-test-result-v1":
|
|
549
|
+
artifact = await materializeFrontendTestResult({
|
|
550
|
+
runDir: meta.runDir,
|
|
551
|
+
workspaceRoot: input.cwd,
|
|
552
|
+
artifactName: gate.artifactName,
|
|
553
|
+
outputDir: gate.outputDir,
|
|
554
|
+
sourceBinding: meta.spec.sourceBinding,
|
|
555
|
+
});
|
|
556
|
+
break;
|
|
547
557
|
default:
|
|
548
558
|
throw new Error(`unsupported jsonArtifactGate.schemaId: ${String(gate.schemaId)}`);
|
|
549
559
|
}
|
|
@@ -6,6 +6,8 @@ import path from "node:path";
|
|
|
6
6
|
import { z } from "zod";
|
|
7
7
|
import { getTaskPoolRoot } from "../pool/run-store.js";
|
|
8
8
|
import { assertSafeRuntimeId } from "../follow-up/paths.js";
|
|
9
|
+
import { expandAllowedPathsForWorkflow } from "../materialize/harness-task-materializer.js";
|
|
10
|
+
import { resolveWorkflow } from "../task-spec/workflow-routing.js";
|
|
9
11
|
const execFileAsync = promisify(execFile);
|
|
10
12
|
const checkpointSchema = z.object({
|
|
11
13
|
taskId: z.string().min(1),
|
|
@@ -275,13 +277,14 @@ function auditChangedPaths(changes, taskSpec) {
|
|
|
275
277
|
throw new Error(audit.violations[0]);
|
|
276
278
|
}
|
|
277
279
|
function auditChangedPathsReport(changes, taskSpec) {
|
|
280
|
+
const allowedPaths = expandAllowedPathsForWorkflow(resolveWorkflow(taskSpec).workflow, taskSpec.constraints.allowed_paths);
|
|
278
281
|
const violations = [];
|
|
279
282
|
for (const changed of changes) {
|
|
280
283
|
if (/(^|\/)(\.env(?:\.|$)|[^/]*\.(?:pem|key|p12|pfx))$/i.test(changed))
|
|
281
284
|
violations.push(`changed path may contain sensitive material: ${changed}`);
|
|
282
285
|
else if (taskSpec.constraints.forbidden_paths.some((glob) => matchesGlob(changed, glob)))
|
|
283
286
|
violations.push(`changed path is forbidden for ${taskSpec.id}: ${changed}`);
|
|
284
|
-
else if (!
|
|
287
|
+
else if (!allowedPaths.some((glob) => matchesGlob(changed, glob)))
|
|
285
288
|
violations.push(`changed path is outside allowed_paths for ${taskSpec.id}: ${changed}`);
|
|
286
289
|
}
|
|
287
290
|
return { ok: violations.length === 0, violations };
|
|
@@ -34,13 +34,17 @@ export async function materializeTaskSpec(options) {
|
|
|
34
34
|
}
|
|
35
35
|
const paths = getTaskPaths(options.repoRoot, harnessTaskId);
|
|
36
36
|
const baseConfig = await loadTaskConfig(options.repoRoot, harnessTaskId);
|
|
37
|
+
// backend-test hybrid DAG writes under testcase/** (pytest + md cases + helpers).
|
|
38
|
+
// TaskSpecs often list legacy tests/api/** surfaces; union runtime write roots so
|
|
39
|
+
// git checkpoint / path guard can promote green BE-TEST runs.
|
|
40
|
+
const allowedPaths = expandAllowedPathsForWorkflow(resolvedWorkflow.workflow, options.taskSpec.constraints.allowed_paths);
|
|
37
41
|
const taskConfig = {
|
|
38
42
|
...baseConfig,
|
|
39
43
|
taskId: harnessTaskId,
|
|
40
44
|
title: options.taskSpec.title,
|
|
41
45
|
taskKind: resolvedWorkflow.taskKind,
|
|
42
46
|
featureId: options.taskSpec.feature_id,
|
|
43
|
-
allowedPaths
|
|
47
|
+
allowedPaths,
|
|
44
48
|
forbiddenPaths: options.taskSpec.constraints.forbidden_paths,
|
|
45
49
|
hardConstraints: options.taskSpec.constraints.hard_constraints,
|
|
46
50
|
complexity: mapRiskLevelToComplexity(options.taskSpec.risk_level),
|
|
@@ -312,6 +316,15 @@ function bulletLines(values) {
|
|
|
312
316
|
return ["- None"];
|
|
313
317
|
return values.map((value) => `- ${value}`);
|
|
314
318
|
}
|
|
319
|
+
/** Union TaskSpec allowed_paths with workflow-owned write roots. */
|
|
320
|
+
export function expandAllowedPathsForWorkflow(workflow, allowedPaths) {
|
|
321
|
+
const extras = workflow === "backend-test"
|
|
322
|
+
? ["testcase/**", "docs/test-reports/**"]
|
|
323
|
+
: workflow === "frontend-test"
|
|
324
|
+
? ["testcase/frontend/**", "docs/test-reports/**"]
|
|
325
|
+
: [];
|
|
326
|
+
return Array.from(new Set([...allowedPaths, ...extras]));
|
|
327
|
+
}
|
|
315
328
|
async function loadExistingTaskConfig(repoRoot, taskId) {
|
|
316
329
|
try {
|
|
317
330
|
return await loadTaskConfig(repoRoot, taskId);
|
|
@@ -39,21 +39,29 @@ function toRepoRelativePath(repoRoot, candidate) {
|
|
|
39
39
|
function structuredArtifactsFromReport(run) {
|
|
40
40
|
if (!run?.nodes)
|
|
41
41
|
return [];
|
|
42
|
-
|
|
42
|
+
// Multiple gates may rewrite the same run-owned path (e.g. backend-test
|
|
43
|
+
// case manifest initial → final). Keep the last claim in report node order
|
|
44
|
+
// so projector validates against the file on disk after all rewrites.
|
|
45
|
+
const byPath = new Map();
|
|
43
46
|
for (const node of run.nodes) {
|
|
44
|
-
if (node.structuredArtifactPath
|
|
45
|
-
node.structuredArtifactSha256
|
|
46
|
-
node.structuredArtifactSchemaId) {
|
|
47
|
-
|
|
48
|
-
artifacts.push({
|
|
49
|
-
path: node.structuredArtifactPath,
|
|
50
|
-
sha256: node.structuredArtifactSha256,
|
|
51
|
-
kind: identity.kind,
|
|
52
|
-
schemaId: identity.schemaId,
|
|
53
|
-
});
|
|
47
|
+
if (!node.structuredArtifactPath ||
|
|
48
|
+
!node.structuredArtifactSha256 ||
|
|
49
|
+
!node.structuredArtifactSchemaId) {
|
|
50
|
+
continue;
|
|
54
51
|
}
|
|
52
|
+
const identity = splitStructuredArtifactIdentity(node.structuredArtifactSchemaId);
|
|
53
|
+
const pathKey = node.structuredArtifactPath.replace(/\\/g, "/");
|
|
54
|
+
// Leave sha256 empty: later DAG nodes (e.g. backend-test traceability)
|
|
55
|
+
// may rewrite the same run-owned path in place, invalidating the mint-time
|
|
56
|
+
// structuredArtifactSha256. Projector re-hashes the file on disk.
|
|
57
|
+
byPath.set(pathKey, {
|
|
58
|
+
path: pathKey,
|
|
59
|
+
sha256: "",
|
|
60
|
+
kind: identity.kind,
|
|
61
|
+
schemaId: identity.schemaId,
|
|
62
|
+
});
|
|
55
63
|
}
|
|
56
|
-
return
|
|
64
|
+
return Array.from(byPath.values());
|
|
57
65
|
}
|
|
58
66
|
/**
|
|
59
67
|
* `agent-dag` — the default standard agent DAG. No dedicated test/integration
|
|
@@ -116,12 +124,13 @@ class BackendTestAdapter {
|
|
|
116
124
|
class FrontendTestAdapter {
|
|
117
125
|
workflow = "frontend-test";
|
|
118
126
|
project(input) {
|
|
119
|
-
const
|
|
127
|
+
const structured = structuredArtifactsFromReport(input.dagReportRun);
|
|
128
|
+
const result = structured.find((artifact) => artifact.kind === "frontend-test-result" && artifact.schemaId === "frontend-test-result-v1");
|
|
129
|
+
// The projector validates the bound hash on disk. Missing result evidence is
|
|
130
|
+
// intentionally not upgraded from DAG completion to real integration.
|
|
131
|
+
const real = input.reportDecision.succeeded && Boolean(result);
|
|
120
132
|
return {
|
|
121
|
-
artifacts: [
|
|
122
|
-
...canonicalArtifacts(input),
|
|
123
|
-
...structuredArtifactsFromReport(input.dagReportRun),
|
|
124
|
-
],
|
|
133
|
+
artifacts: [...canonicalArtifacts(input), ...structured],
|
|
125
134
|
acceptanceCoverage: [...input.acceptanceRefs],
|
|
126
135
|
integrationStatus: { mock: false, real },
|
|
127
136
|
};
|
|
@@ -2,6 +2,7 @@ import path from "node:path";
|
|
|
2
2
|
import { readFile, stat } from "node:fs/promises";
|
|
3
3
|
import { sha256File } from "../pool/run-store.js";
|
|
4
4
|
import { taskOutcomeEnvelopeV1Schema } from "./types.js";
|
|
5
|
+
import { frontendTestResultContractSchema } from "../../workflows/dag/frontend-test-result-contract.js";
|
|
5
6
|
import { getOutcomeAdapter } from "./adapters.js";
|
|
6
7
|
import { isPathAllowedForTask, mergeOutcomeArtifacts, projectDeclaredPathArtifacts, } from "./declared-artifacts.js";
|
|
7
8
|
/**
|
|
@@ -117,6 +118,26 @@ export async function projectOutcome(input) {
|
|
|
117
118
|
...(artifact.schemaId ? { schemaId: artifact.schemaId } : {}),
|
|
118
119
|
});
|
|
119
120
|
}
|
|
121
|
+
// A frontend-test success is meaningful only when its finalizer produced a
|
|
122
|
+
// hash-verified real/pass result. This check deliberately happens after the
|
|
123
|
+
// generic rehash so report metadata cannot forge integration evidence.
|
|
124
|
+
if (workflow === "frontend-test") {
|
|
125
|
+
const results = validatedArtifacts.filter((artifact) => artifact.kind === "frontend-test-result" &&
|
|
126
|
+
artifact.schemaId === "frontend-test-result-v1");
|
|
127
|
+
if (results.length !== 1) {
|
|
128
|
+
return contractError("frontend-test requires exactly one hash-bound frontend-test-result-v1 artifact", ["frontend-test-result-v1"]);
|
|
129
|
+
}
|
|
130
|
+
try {
|
|
131
|
+
const resultArtifact = results[0];
|
|
132
|
+
const result = frontendTestResultContractSchema.parse(JSON.parse(await readFile(path.resolve(input.repoRoot, resultArtifact.path), "utf-8")));
|
|
133
|
+
if (result.outcome !== "passed" || result.integrationMode !== "real") {
|
|
134
|
+
return contractError(`frontend-test result is not a real pass: outcome=${result.outcome} integrationMode=${result.integrationMode}`);
|
|
135
|
+
}
|
|
136
|
+
}
|
|
137
|
+
catch (error) {
|
|
138
|
+
return contractError(`invalid frontend-test-result-v1 artifact: ${error instanceof Error ? error.message : String(error)}`);
|
|
139
|
+
}
|
|
140
|
+
}
|
|
120
141
|
// 5. Shell verification from report decision (exit-zero = report succeeded).
|
|
121
142
|
const shellVerification = {
|
|
122
143
|
exitZero: input.reportDecision.succeeded,
|
|
@@ -455,7 +455,8 @@ function isPathInside(root, target) {
|
|
|
455
455
|
const relative = path.relative(root, target);
|
|
456
456
|
return relative !== "" && relative !== ".." && !relative.startsWith(`..${path.sep}`) && !path.isAbsolute(relative);
|
|
457
457
|
}
|
|
458
|
-
|
|
458
|
+
/** Exported for unit tests: map DAG report JSON into Worker report decision. */
|
|
459
|
+
export function decideFromReport(workerRunId, result) {
|
|
459
460
|
if (!result.ok || !result.json) {
|
|
460
461
|
return {
|
|
461
462
|
succeeded: false,
|
|
@@ -473,11 +474,16 @@ function decideFromReport(workerRunId, result) {
|
|
|
473
474
|
}
|
|
474
475
|
const runStatus = readString(run, "status");
|
|
475
476
|
const nodes = readObjectArray(run, "nodes");
|
|
477
|
+
// Condition-branch SKIPPED nodes are expected on green backend-test / hybrid
|
|
478
|
+
// paths (e.g. revise/repair not selected). Do not treat SKIPPED as failure.
|
|
479
|
+
// ERROR and non-success failureCategory on executed nodes still fail closed.
|
|
476
480
|
const failedNode = nodes.find((node) => {
|
|
477
481
|
const status = readString(node, "status");
|
|
478
482
|
const failureCategory = readString(node, "failureCategory");
|
|
483
|
+
if (status === "SKIPPED") {
|
|
484
|
+
return false;
|
|
485
|
+
}
|
|
479
486
|
return (status === "ERROR" ||
|
|
480
|
-
status === "SKIPPED" ||
|
|
481
487
|
(Boolean(failureCategory) && failureCategory !== "success"));
|
|
482
488
|
});
|
|
483
489
|
if (runStatus !== "completed" && runStatus !== "finished") {
|
|
@@ -334,22 +334,50 @@ function asRecord(value) {
|
|
|
334
334
|
* Prefer exact schema payloads; otherwise map common discovery shapes onto the
|
|
335
335
|
* pytest-centric runtime contract without inventing secrets or managed commands.
|
|
336
336
|
*/
|
|
337
|
+
/** Drop empty managedCommand strings; omit block when not managed-command. */
|
|
338
|
+
export function sanitizeBackendTestExecutionInput(value) {
|
|
339
|
+
const record = asRecord(value);
|
|
340
|
+
if (!record)
|
|
341
|
+
return value;
|
|
342
|
+
const next = { ...record };
|
|
343
|
+
const managed = asRecord(record.managedCommand);
|
|
344
|
+
if (!managed)
|
|
345
|
+
return next;
|
|
346
|
+
const cleaned = {};
|
|
347
|
+
for (const key of ["start", "stop", "sourceRef"]) {
|
|
348
|
+
const raw = managed[key];
|
|
349
|
+
if (typeof raw === "string" && raw.trim())
|
|
350
|
+
cleaned[key] = raw.trim();
|
|
351
|
+
}
|
|
352
|
+
if (next.targetMode === "managed-command") {
|
|
353
|
+
next.managedCommand = cleaned;
|
|
354
|
+
}
|
|
355
|
+
else if (Object.keys(cleaned).length === 0) {
|
|
356
|
+
delete next.managedCommand;
|
|
357
|
+
}
|
|
358
|
+
else {
|
|
359
|
+
// in-process / external: optional managedCommand must not carry empty strings
|
|
360
|
+
next.managedCommand = cleaned;
|
|
361
|
+
}
|
|
362
|
+
return next;
|
|
363
|
+
}
|
|
337
364
|
export function coerceBackendTestExecutionInput(value) {
|
|
338
|
-
const
|
|
365
|
+
const sanitized = sanitizeBackendTestExecutionInput(value);
|
|
366
|
+
const direct = backendTestExecutionContractSchema.safeParse(sanitized);
|
|
339
367
|
if (direct.success)
|
|
340
368
|
return direct.data;
|
|
341
|
-
const record = asRecord(
|
|
369
|
+
const record = asRecord(sanitized);
|
|
342
370
|
if (!record)
|
|
343
|
-
return
|
|
371
|
+
return sanitized;
|
|
344
372
|
// Near-schema payloads (string framework + runner + testRoot) must stay fail-closed.
|
|
345
373
|
// Only free-form discovery envelopes are rewritten onto the pytest contract.
|
|
346
374
|
const looksSchemaShaped = typeof record.framework === "string" &&
|
|
347
375
|
asRecord(record.runner) !== null &&
|
|
348
376
|
typeof record.testRoot === "string";
|
|
349
377
|
if (looksSchemaShaped)
|
|
350
|
-
return
|
|
378
|
+
return sanitized;
|
|
351
379
|
if (typeof record.framework === "string" && record.framework !== "pytest") {
|
|
352
|
-
return
|
|
380
|
+
return sanitized;
|
|
353
381
|
}
|
|
354
382
|
const frameworkObj = asRecord(record.framework);
|
|
355
383
|
const discovered = asRecord(record.discoveredFixtures);
|
|
@@ -504,7 +532,10 @@ export async function materializeBackendTestExecutionContract(input) {
|
|
|
504
532
|
catch (error) {
|
|
505
533
|
throw new Error(`invalid-output: ${error instanceof Error ? error.message : String(error)}`);
|
|
506
534
|
}
|
|
507
|
-
const candidates = [
|
|
535
|
+
const candidates = [
|
|
536
|
+
sanitizeBackendTestExecutionInput(parsed),
|
|
537
|
+
coerceBackendTestExecutionInput(parsed),
|
|
538
|
+
];
|
|
508
539
|
let accepted = null;
|
|
509
540
|
let lastSchemaError = "invalid execution contract";
|
|
510
541
|
let lastSecretError = "";
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import { mkdir } from "node:fs/promises";
|
|
1
|
+
import { mkdir, readFile, writeFile } from "node:fs/promises";
|
|
2
2
|
import path from "node:path";
|
|
3
3
|
import { writeDagRunJsonArtifact } from "../../../infrastructure/harness/artifact-store.js";
|
|
4
4
|
import { executeDagNode, } from "../node-execution.js";
|
|
@@ -81,6 +81,28 @@ export function resolveRunLocalPath(runDir, ref, label) {
|
|
|
81
81
|
}
|
|
82
82
|
return resolved;
|
|
83
83
|
}
|
|
84
|
+
async function materializeTokenBudgetBlockedEvidence(input) {
|
|
85
|
+
if (!input.workspaceRef)
|
|
86
|
+
return;
|
|
87
|
+
const workspace = resolveRunLocalPath(input.cwd, input.workspaceRef, "workspaceRef");
|
|
88
|
+
await mkdir(workspace, { recursive: true });
|
|
89
|
+
const resultPath = path.join(workspace, "case-result.json");
|
|
90
|
+
const caseId = typeof input.item === "object" && input.item !== null &&
|
|
91
|
+
typeof input.item.caseId === "string"
|
|
92
|
+
? input.item.caseId
|
|
93
|
+
: String(input.item);
|
|
94
|
+
try {
|
|
95
|
+
const existing = JSON.parse(await readFile(resultPath, "utf-8"));
|
|
96
|
+
if (existing.caseId === caseId &&
|
|
97
|
+
(existing.status === "passed" || existing.status === "failed" || existing.status === "blocked"))
|
|
98
|
+
return;
|
|
99
|
+
}
|
|
100
|
+
catch {
|
|
101
|
+
// Missing or malformed evidence is replaced only for a child that never started.
|
|
102
|
+
}
|
|
103
|
+
await writeFile(path.join(workspace, "execution.md"), `# ${caseId}\n\nStatus: blocked\n\nReason: token-budget-exhausted\n`, "utf-8");
|
|
104
|
+
await writeFile(resultPath, `${JSON.stringify({ caseId, status: "blocked", blockedReason: "token-budget-exhausted", evidencePaths: ["execution.md"] }, null, 2)}\n`, "utf-8");
|
|
105
|
+
}
|
|
84
106
|
export async function executeDynamicMapExpansion(input) {
|
|
85
107
|
const started = Date.now();
|
|
86
108
|
const items = resolveItemsFromSelector(input.expansion.itemsFrom, input.state);
|
|
@@ -195,6 +217,16 @@ export async function executeDynamicMapExpansion(input) {
|
|
|
195
217
|
}
|
|
196
218
|
continue;
|
|
197
219
|
}
|
|
220
|
+
try {
|
|
221
|
+
await materializeTokenBudgetBlockedEvidence({
|
|
222
|
+
cwd: input.cwd,
|
|
223
|
+
workspaceRef: workspaceRefs[children.indexOf(remaining)],
|
|
224
|
+
item: items[children.indexOf(remaining)],
|
|
225
|
+
});
|
|
226
|
+
}
|
|
227
|
+
catch (error) {
|
|
228
|
+
throw new Error(`failed to materialize token-budget blocked evidence for ${remaining.id}: ${error instanceof Error ? error.message : String(error)}`);
|
|
229
|
+
}
|
|
198
230
|
record.status = "SKIPPED";
|
|
199
231
|
record.stderr = "blocked: token-budget-exhausted";
|
|
200
232
|
if (!blockedChildren.some((entry) => entry.nodeId === remaining.id)) {
|
|
@@ -21,12 +21,10 @@ export function getNodeOutputAsJson(state, nodeId) {
|
|
|
21
21
|
if (!raw) {
|
|
22
22
|
throw new Error(`itemsFrom upstream node "${nodeId}" has no JSON output`);
|
|
23
23
|
}
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
throw new Error(`itemsFrom upstream node "${nodeId}" output is not valid JSON: ${error instanceof Error ? error.message : String(error)}`);
|
|
29
|
-
}
|
|
24
|
+
const parsed = parseJsonFromText(raw);
|
|
25
|
+
if (parsed !== undefined)
|
|
26
|
+
return parsed;
|
|
27
|
+
throw new Error(`itemsFrom upstream node "${nodeId}" output is not valid JSON: ${raw.slice(0, 120)}`);
|
|
30
28
|
}
|
|
31
29
|
/** Normalize a line the same way verdict gates do (strip whole-line Markdown emphasis). */
|
|
32
30
|
export function normalizeVerdictCandidateLine(value) {
|
|
@@ -147,10 +145,29 @@ export function parseConditionLiteral(raw) {
|
|
|
147
145
|
return Number.isFinite(numeric) ? numeric : trimmed;
|
|
148
146
|
}
|
|
149
147
|
export function parseJsonFromText(value) {
|
|
150
|
-
|
|
148
|
+
const raw = value?.trim();
|
|
149
|
+
if (!raw)
|
|
151
150
|
return undefined;
|
|
151
|
+
// Pure JSON first.
|
|
152
|
+
try {
|
|
153
|
+
return JSON.parse(raw);
|
|
154
|
+
}
|
|
155
|
+
catch {
|
|
156
|
+
// continue
|
|
157
|
+
}
|
|
158
|
+
// Shell executor stdout prefixes each command with `$ <command>`. Dynamic
|
|
159
|
+
// selectors must consume exactly one trailing JSON payload after those echoes.
|
|
160
|
+
// Reject multi-payload or mixed prose to keep map expansion fail-closed.
|
|
161
|
+
const lines = raw.split(/\r?\n/).filter((line) => line.trim().length > 0);
|
|
162
|
+
if (lines.length < 2)
|
|
163
|
+
return undefined;
|
|
164
|
+
const payload = lines.at(-1)?.trim();
|
|
165
|
+
const prefix = lines.slice(0, -1);
|
|
166
|
+
if (!payload || prefix.some((line) => !line.startsWith("$ "))) {
|
|
167
|
+
return undefined;
|
|
168
|
+
}
|
|
152
169
|
try {
|
|
153
|
-
return JSON.parse(
|
|
170
|
+
return JSON.parse(payload);
|
|
154
171
|
}
|
|
155
172
|
catch {
|
|
156
173
|
return undefined;
|
|
@@ -0,0 +1,165 @@
|
|
|
1
|
+
import { createHash } from "node:crypto";
|
|
2
|
+
import { lstat, readFile, realpath } from "node:fs/promises";
|
|
3
|
+
import path from "node:path";
|
|
4
|
+
import { z } from "zod";
|
|
5
|
+
import { writeDagRunJsonArtifact } from "../../infrastructure/harness/artifact-store.js";
|
|
6
|
+
export const FRONTEND_TEST_RESULT_SCHEMA_ID = "frontend-test-result-v1";
|
|
7
|
+
const safeRelativePathSchema = z.string().min(1).refine((value) => !path.posix.isAbsolute(value) &&
|
|
8
|
+
!path.win32.isAbsolute(value) &&
|
|
9
|
+
!value.includes("\\") &&
|
|
10
|
+
value.split("/").every((part) => part.length > 0 && part !== "." && part !== ".."), "must be a safe repo-relative POSIX path");
|
|
11
|
+
const sha256Schema = z.string().regex(/^[a-f0-9]{64}$/);
|
|
12
|
+
const caseStatusSchema = z.enum(["passed", "failed", "blocked"]);
|
|
13
|
+
export const frontendTestResultContractSchema = z.object({
|
|
14
|
+
schemaVersion: z.literal(1),
|
|
15
|
+
sourceBinding: z.object({
|
|
16
|
+
taskId: z.string().min(1),
|
|
17
|
+
sources: z.array(z.object({
|
|
18
|
+
kind: z.enum(["requirement", "constraint", "reference"]),
|
|
19
|
+
path: safeRelativePathSchema,
|
|
20
|
+
sha256: sha256Schema,
|
|
21
|
+
}).strict()).min(1),
|
|
22
|
+
requirementIds: z.array(z.string().min(1)),
|
|
23
|
+
}).strict(),
|
|
24
|
+
manifest: z.object({
|
|
25
|
+
path: safeRelativePathSchema,
|
|
26
|
+
sha256: sha256Schema,
|
|
27
|
+
}).strict(),
|
|
28
|
+
cases: z.array(z.object({
|
|
29
|
+
caseId: z.string().min(1),
|
|
30
|
+
acIds: z.array(z.string().min(1)).min(1),
|
|
31
|
+
status: caseStatusSchema,
|
|
32
|
+
evidence: z.array(z.object({ path: safeRelativePathSchema, sha256: sha256Schema }).strict()),
|
|
33
|
+
blockedReason: z.string().min(1).optional(),
|
|
34
|
+
}).strict()).min(1),
|
|
35
|
+
totals: z.object({
|
|
36
|
+
cases: z.number().int().positive(),
|
|
37
|
+
passed: z.number().int().min(0),
|
|
38
|
+
failed: z.number().int().min(0),
|
|
39
|
+
blocked: z.number().int().min(0),
|
|
40
|
+
}).strict(),
|
|
41
|
+
acceptanceCoverage: z.object({
|
|
42
|
+
covered: z.array(z.string().min(1)),
|
|
43
|
+
missing: z.array(z.string().min(1)),
|
|
44
|
+
}).strict(),
|
|
45
|
+
integrationMode: z.enum(["real", "none"]),
|
|
46
|
+
outcome: z.enum(["passed", "failed", "incomplete"]),
|
|
47
|
+
}).strict().superRefine((value, ctx) => {
|
|
48
|
+
if (value.totals.cases !== value.cases.length) {
|
|
49
|
+
ctx.addIssue({ code: z.ZodIssueCode.custom, path: ["totals", "cases"], message: "totals.cases must equal cases length" });
|
|
50
|
+
}
|
|
51
|
+
for (const status of ["passed", "failed", "blocked"]) {
|
|
52
|
+
const actual = value.cases.filter((item) => item.status === status).length;
|
|
53
|
+
if (value.totals[status] !== actual) {
|
|
54
|
+
ctx.addIssue({ code: z.ZodIssueCode.custom, path: ["totals", status], message: `totals.${status} must equal case status count` });
|
|
55
|
+
}
|
|
56
|
+
}
|
|
57
|
+
if (value.outcome === "passed" && (value.totals.failed > 0 || value.totals.blocked > 0 || value.acceptanceCoverage.missing.length > 0 || value.integrationMode !== "real")) {
|
|
58
|
+
ctx.addIssue({ code: z.ZodIssueCode.custom, path: ["outcome"], message: "passed requires all cases passed, complete AC coverage, and real integration" });
|
|
59
|
+
}
|
|
60
|
+
if (value.integrationMode === "real" && value.outcome !== "passed") {
|
|
61
|
+
ctx.addIssue({ code: z.ZodIssueCode.custom, path: ["integrationMode"], message: "real integration requires passed outcome" });
|
|
62
|
+
}
|
|
63
|
+
});
|
|
64
|
+
function sha256(content) {
|
|
65
|
+
return createHash("sha256").update(content).digest("hex");
|
|
66
|
+
}
|
|
67
|
+
function assertInside(root, candidate, label) {
|
|
68
|
+
const relative = path.relative(root, candidate);
|
|
69
|
+
if (relative.startsWith("..") || path.isAbsolute(relative)) {
|
|
70
|
+
throw new Error(`${label} escapes its allowed root`);
|
|
71
|
+
}
|
|
72
|
+
}
|
|
73
|
+
async function readEvidenceFile(repoRoot, evidenceRoot, relative) {
|
|
74
|
+
if (!safeRelativePathSchema.safeParse(relative).success) {
|
|
75
|
+
throw new Error(`unsafe evidence path: ${relative}`);
|
|
76
|
+
}
|
|
77
|
+
const absolute = path.resolve(evidenceRoot, relative);
|
|
78
|
+
assertInside(evidenceRoot, absolute, "evidence path");
|
|
79
|
+
const resolved = await realpath(absolute);
|
|
80
|
+
assertInside(repoRoot, resolved, "evidence realpath");
|
|
81
|
+
const info = await lstat(resolved);
|
|
82
|
+
if (!info.isFile())
|
|
83
|
+
throw new Error(`evidence is not a file: ${relative}`);
|
|
84
|
+
return { path: relative, sha256: sha256(await readFile(resolved)) };
|
|
85
|
+
}
|
|
86
|
+
export async function materializeFrontendTestResult(input) {
|
|
87
|
+
if (!/^[a-z0-9][a-z0-9._-]*\.json$/.test(input.artifactName) || !/^[a-z0-9][a-z0-9._-]*$/.test(input.outputDir)) {
|
|
88
|
+
throw new Error("unsafe structured artifact path");
|
|
89
|
+
}
|
|
90
|
+
const sourceBinding = input.sourceBinding;
|
|
91
|
+
if (!sourceBinding?.taskId || !sourceBinding.sources?.length) {
|
|
92
|
+
throw new Error("frontend-test result requires frozen sourceBinding");
|
|
93
|
+
}
|
|
94
|
+
const manifestPath = "testcase/frontend/cases/manifest.json";
|
|
95
|
+
const manifestAbsolute = path.resolve(input.workspaceRoot, manifestPath);
|
|
96
|
+
const manifest = JSON.parse(await readFile(manifestAbsolute, "utf-8"));
|
|
97
|
+
if (manifest.schemaVersion !== 1 || !Array.isArray(manifest.cases) || manifest.cases.length === 0) {
|
|
98
|
+
throw new Error("invalid or empty frontend case manifest");
|
|
99
|
+
}
|
|
100
|
+
const requiredIds = new Set(sourceBinding.requirementIds ?? []);
|
|
101
|
+
const covered = new Set();
|
|
102
|
+
const cases = [];
|
|
103
|
+
for (const raw of manifest.cases) {
|
|
104
|
+
const item = raw;
|
|
105
|
+
if (!item || typeof item.caseId !== "string" || !Array.isArray(item.acIds) || typeof item.evidenceDir !== "string") {
|
|
106
|
+
throw new Error("invalid frontend manifest case");
|
|
107
|
+
}
|
|
108
|
+
if (!item.evidenceDir.startsWith(`testcase/frontend/evidence/${item.caseId}/`)) {
|
|
109
|
+
throw new Error(`unsafe evidenceDir for ${item.caseId}`);
|
|
110
|
+
}
|
|
111
|
+
for (const acId of item.acIds) {
|
|
112
|
+
if (requiredIds.size > 0 && !requiredIds.has(acId))
|
|
113
|
+
throw new Error(`unknown AC in manifest: ${acId}`);
|
|
114
|
+
covered.add(acId);
|
|
115
|
+
}
|
|
116
|
+
const evidenceRoot = path.resolve(input.workspaceRoot, item.evidenceDir);
|
|
117
|
+
assertInside(input.workspaceRoot, evidenceRoot, "evidenceDir");
|
|
118
|
+
const resultRaw = JSON.parse(await readFile(path.join(evidenceRoot, "case-result.json"), "utf-8"));
|
|
119
|
+
const status = caseStatusSchema.parse(resultRaw.status);
|
|
120
|
+
if (resultRaw.caseId !== item.caseId || !Array.isArray(resultRaw.evidencePaths))
|
|
121
|
+
throw new Error(`invalid result identity for ${item.caseId}`);
|
|
122
|
+
if (status === "blocked" && (typeof resultRaw.blockedReason !== "string" || !resultRaw.blockedReason.trim()))
|
|
123
|
+
throw new Error(`blocked result requires reason for ${item.caseId}`);
|
|
124
|
+
const evidence = [await readEvidenceFile(input.workspaceRoot, evidenceRoot, "execution.md")];
|
|
125
|
+
for (const evidencePath of resultRaw.evidencePaths) {
|
|
126
|
+
if (typeof evidencePath !== "string" || evidencePath === "execution.md")
|
|
127
|
+
continue;
|
|
128
|
+
evidence.push(await readEvidenceFile(input.workspaceRoot, evidenceRoot, evidencePath));
|
|
129
|
+
}
|
|
130
|
+
if (status === "passed" && !evidence.some((entry) => /\.(png|jpg|jpeg|webp|zip|har|webm|mp4|ya?ml)$/i.test(entry.path))) {
|
|
131
|
+
throw new Error(`passed case requires browser evidence: ${item.caseId}`);
|
|
132
|
+
}
|
|
133
|
+
cases.push({ caseId: item.caseId, acIds: item.acIds, status, evidence, ...(status === "blocked" ? { blockedReason: resultRaw.blockedReason } : {}) });
|
|
134
|
+
}
|
|
135
|
+
const missing = [...requiredIds].filter((id) => !covered.has(id));
|
|
136
|
+
const totals = {
|
|
137
|
+
cases: cases.length,
|
|
138
|
+
passed: cases.filter((item) => item.status === "passed").length,
|
|
139
|
+
failed: cases.filter((item) => item.status === "failed").length,
|
|
140
|
+
blocked: cases.filter((item) => item.status === "blocked").length,
|
|
141
|
+
};
|
|
142
|
+
const outcome = totals.failed > 0 ? "failed" : totals.blocked > 0 || missing.length > 0 ? "incomplete" : "passed";
|
|
143
|
+
const result = frontendTestResultContractSchema.parse({
|
|
144
|
+
schemaVersion: 1,
|
|
145
|
+
sourceBinding,
|
|
146
|
+
manifest: { path: manifestPath, sha256: sha256(await readFile(manifestAbsolute)) },
|
|
147
|
+
cases,
|
|
148
|
+
totals,
|
|
149
|
+
acceptanceCoverage: { covered: [...covered].sort(), missing: missing.sort() },
|
|
150
|
+
integrationMode: outcome === "passed" ? "real" : "none",
|
|
151
|
+
outcome,
|
|
152
|
+
});
|
|
153
|
+
const relativePath = path.posix.join(input.outputDir, input.artifactName);
|
|
154
|
+
const artifactPath = await writeDagRunJsonArtifact(input.runDir, relativePath, result);
|
|
155
|
+
return { path: artifactPath, sha256: sha256(`${JSON.stringify(result, null, 2)}\n`), schemaId: FRONTEND_TEST_RESULT_SCHEMA_ID };
|
|
156
|
+
}
|
|
157
|
+
export function buildFrontendTestOutcomeGateShellSnippet(options) {
|
|
158
|
+
const resultRelativePath = options?.resultRelativePath ?? "contracts/frontend-test-result.json";
|
|
159
|
+
return [
|
|
160
|
+
'test -n "${HARNESS_DAG_RUN_DIR:-}" || { echo "missing HARNESS_DAG_RUN_DIR for frontend-test outcome gate" >&2; exit 2; }',
|
|
161
|
+
`RESULT="\${HARNESS_DAG_RUN_DIR}/${resultRelativePath}"`,
|
|
162
|
+
'test -f "${RESULT}" || { echo "missing frontend-test result: ${RESULT}" >&2; exit 2; }',
|
|
163
|
+
`node -e 'const fs=require("fs");const r=JSON.parse(fs.readFileSync(process.argv[1],"utf8"));const ok=r.outcome==="passed"&&r.integrationMode==="real"&&Number(r.totals?.failed||0)===0&&Number(r.totals?.blocked||0)===0&&Array.isArray(r.acceptanceCoverage?.missing)&&r.acceptanceCoverage.missing.length===0;console.log("frontend-test outcome="+r.outcome+" integrationMode="+r.integrationMode);if(!ok)process.exit(1);' "\${RESULT}"`,
|
|
164
|
+
].join("; ");
|
|
165
|
+
}
|