@tea-agent/loop-agent 0.16.23 → 0.16.24
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +22 -13
- package/dist/executors/shell-presets.js +27 -26
- package/dist/executors/shell-verification.js +2 -0
- package/dist/task/config-types.js +15 -3
- package/dist/workflows/dag/backend-test-analysis-contract.js +21 -8
- package/dist/workflows/dag/dynamic-runtime/map.js +129 -22
- package/dist/workflows/dag/init-hybrid.js +328 -259
- package/dist/workflows/dag/types.js +7 -0
- package/docs/local-development-environment.md +4 -0
- package/docs/templates/backend-test-dag.json +1 -1
- package/docs/templates/frontend-test-case-checklist.md +25 -0
- package/docs/templates/frontend-test-dag.json +4 -3
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,20 +1,9 @@
|
|
|
1
|
-
# 更新日志
|
|
2
|
-
|
|
3
1
|
## [Unreleased]
|
|
4
2
|
|
|
5
|
-
### 修复
|
|
6
|
-
|
|
7
|
-
- Backend-test 唯一 JSON object 提取器在扫描裸对象前会等长忽略完整 Markdown 行内代码中的对象形状示例(如 `{error:{code,message,type}}`);唯一真实 envelope 可正常物化,多个真实 JSON、fence 外额外对象与未闭合对象继续 fail-closed。
|
|
8
|
-
|
|
9
|
-
- Backend-test 节点 10 `backend-test-semantic-gate-shell` 不再用 Markdown `first-verdict-line` 读取 raw Pi 输出;改为同一 `semantic-initial` pipeline 按 task id 只读 `contracts/backend-test-semantic-review.json`,仅 `verdict=pass` 放行 pytest,`request-revision`/缺失/malformed 清晰 fail-closed;节点 09 仍只负责 materialize+trace,15 节点与单次 pytest 不变。
|
|
10
|
-
- 公开 Case Manifest schema(`docs/templates/backend-test-case-manifest.schema.json`)对齐 materializer dual-scope coverage:`coverageSummary` 在 `additionalProperties:false` 下 required 并声明 `global`/`inScope`(`total`/`covered`/`gapped`/`ratio`)与 `crossDomainGapCount`;回归测试以无依赖轻量 contract 锁定 schema↔canonical 一致性;不改 materializer/15 节点。
|
|
11
|
-
- Backend-test Case Manifest materializer 将模型顶层 `coverageSummary` 视为非权威:secret 扫描后剥离,再 strict parse 其余字段,并始终 `computeCaseManifestCoverageSummary` 写入 canonical summary;my-webapp 自定义/部分 summary 不再导致 invalid-output;prompt 禁止模型输出 coverageSummary;15 节点与单次 pytest 不变。
|
|
12
|
-
- Backend-test 裸 object 路径现消费 scanner 的 `unclosedObject`:当文本含一个完整裸 JSON object,且其前或后仍有未闭合裸 object 时一律 fail-closed,不再因 `spans.length === 1` 错误接受完整对象;Semantic Review / Classification / contracts envelope 共用该行为。
|
|
13
|
-
- Backend-test contracts composite pipeline (`validate-backend-test-contracts-shell`) 使用唯一确定性 envelope 提取:接受 pure JSON、唯一 fenced json、prose+唯一 fence、prose+唯一括号平衡裸 object;对无 JSON、多候选、未闭合、非 object、缺 `analysis`/`execution` fail-closed。提取后仍走 Analysis v2 / Execution v1 schema、source-binding 与 secret 门禁;不引入全局 Pi output rewrite,不将 `invalid-output` 加入自动 retry,不改 15 节点拓扑。
|
|
14
|
-
- Backend-test Semantic Review(08→09)与 Classification(12→13)materializer 复用同一 `extractUniqueJsonObject` scanner:接受 pure / whole-fence / prose+唯一 fence / prose+唯一裸 object;多候选、未闭合、非 object 与 strict schema 错误继续 fail-closed;不改 Case Manifest、Pi SDK/retry 或 15 节点拓扑。
|
|
15
|
-
|
|
16
3
|
### 改进
|
|
17
4
|
|
|
5
|
+
- GitHub Actions CI 降低私有仓分钟消耗:合并为单 job(避免两次 `npm ci`)、同分支/`PR` concurrency 取消旧 run、Draft PR 跳过完整 CI,且仅忽略 `docs/reports/**`、`docs/progress/**`、`docs/design/archive/**`、`docs/exec-plans/completed/**` 等纯运营文档路径;治理文档与代码门禁仍全量跑。
|
|
6
|
+
- 明确 **Compatibility / Operator Assist**:openCode 等主会话只编排已发布 `loop-agent` / `agent-worker` CLI 与只读诊断,不得绕过 CLI 直接改业务实现,失败只走 doctor / reconcile / human gate / CLI 重跑。`loop-agent init` 写入的 `AGENTS.md` managed block、包内 skills 与 website 快速开始/治理说明已对齐;架构决策见 `docs/decisions/0005-governed-operator-surface.md`(accepted)。已有目标项目用 `loop-agent init check-update` / `init update --apply-safe` 或 `init reconcile` 刷新 managed block。
|
|
18
7
|
- 前端测试 DAG 收紧体验:execution preflight 仅硬校验绝对非生产 `baseUrl`;用例 map 缩短为优先用 `playwright-cli` 执行;复盘合并执行证据审查且不依赖 outcome=pass,失败也能出报告。
|
|
19
8
|
- 前端测试用例生成现在会解析明确 `baseUrl`:优先读取任务源 `config.md` 中的前端 URL,缺失时默认 `http://localhost:5173`,并写入 RAG `context.md`;生成命令不得再保留 `<base-url>` 占位符。
|
|
20
9
|
- 前端浏览器测试 DAG 的复盘报告现写入 `testcase/frontend/reports/**`,不再要求 `docs/test-reports/**` 权限;任务执行约束可以安全禁止整个 `docs/**`,同时仍保留可审计的测试资产。
|
|
@@ -26,6 +15,15 @@
|
|
|
26
15
|
|
|
27
16
|
### 修复
|
|
28
17
|
|
|
18
|
+
- Backend-test Analysis v2 intake 现在可确定性归一化模型常见的外部依赖 `type` / `target` 与风险 `type` 字段:dependency `target` 映射为 canonical `name`,扩展字段不落盘;dependency/risk 仍使用类别化白名单,未知字段和 `risks[].target` 继续 fail-closed,避免节点 02 因合法近似形态过早中断。
|
|
19
|
+
|
|
20
|
+
- Backend-test 唯一 JSON object 提取器在扫描裸对象前会等长忽略完整 Markdown 行内代码中的对象形状示例(如 `{error:{code,message,type}}`);唯一真实 envelope 可正常物化,多个真实 JSON、fence 外额外对象与未闭合对象继续 fail-closed。
|
|
21
|
+
- Backend-test 节点 10 `backend-test-semantic-gate-shell` 不再用 Markdown `first-verdict-line` 读取 raw Pi 输出;改为同一 `semantic-initial` pipeline 按 task id 只读 `contracts/backend-test-semantic-review.json`,仅 `verdict=pass` 放行 pytest,`request-revision`/缺失/malformed 清晰 fail-closed;节点 09 仍只负责 materialize+trace,15 节点与单次 pytest 不变。
|
|
22
|
+
- 公开 Case Manifest schema(`docs/templates/backend-test-case-manifest.schema.json`)对齐 materializer dual-scope coverage:`coverageSummary` 在 `additionalProperties:false` 下 required 并声明 `global`/`inScope`(`total`/`covered`/`gapped`/`ratio`)与 `crossDomainGapCount`;回归测试以无依赖轻量 contract 锁定 schema↔canonical 一致性;不改 materializer/15 节点。
|
|
23
|
+
- Backend-test Case Manifest materializer 将模型顶层 `coverageSummary` 视为非权威:secret 扫描后剥离,再 strict parse 其余字段,并始终 `computeCaseManifestCoverageSummary` 写入 canonical summary;my-webapp 自定义/部分 summary 不再导致 invalid-output;prompt 禁止模型输出 coverageSummary;15 节点与单次 pytest 不变。
|
|
24
|
+
- Backend-test 裸 object 路径现消费 scanner 的 `unclosedObject`:当文本含一个完整裸 JSON object,且其前或后仍有未闭合裸 object 时一律 fail-closed,不再因 `spans.length === 1` 错误接受完整对象;Semantic Review / Classification / contracts envelope 共用该行为。
|
|
25
|
+
- Backend-test contracts composite pipeline (`validate-backend-test-contracts-shell`) 使用唯一确定性 envelope 提取:接受 pure JSON、唯一 fenced json、prose+唯一 fence、prose+唯一括号平衡裸 object;对无 JSON、多候选、未闭合、非 object、缺 `analysis`/`execution` fail-closed。提取后仍走 Analysis v2 / Execution v1 schema、source-binding 与 secret 门禁;不引入全局 Pi output rewrite,不将 `invalid-output` 加入自动 retry,不改 15 节点拓扑。
|
|
26
|
+
- Backend-test Semantic Review(08→09)与 Classification(12→13)materializer 复用同一 `extractUniqueJsonObject` scanner:接受 pure / whole-fence / prose+唯一 fence / prose+唯一裸 object;多候选、未闭合、非 object 与 strict schema 错误继续 fail-closed;不改 Case Manifest、Pi SDK/retry 或 15 节点拓扑。
|
|
29
27
|
- 节点检查器「执行过程」中的事件时间戳现在同时显示本地日期与 24 小时制 `HH:mm:ss`(例如 `2026/7/16 08:09:10`),不再被裁掉时间部分;缺少时间戳时仍显示「未记录时间」,事件字段优先级(`timestamp` → `at` → `recordedAt`)保持不变。
|
|
30
28
|
- `frontend-test` 的 case review 现在是 fail-closed browser gate:只有 `VERDICT: pass` 才能物化 manifest 并启动动态 browser map;生成的 case 固定使用默认 browser session,要求每个子场景的 fixture/UI reset 和 fresh snapshot,并由确定性节点校验每个 case 的 `execution.md`、`case-result.json`、`caseId`、`status`、`evidencePaths` 及 blocked `blockedReason`。
|
|
31
29
|
- 前端实现计划/修订节点会注入当前包内权威 `frontend-implementation-contract-v1` Schema 与固定 source binding,避免模型猜测字段导致契约门禁失败。
|
|
@@ -37,6 +35,17 @@
|
|
|
37
35
|
- Pi SDK 执行长推理或大段结构化输出时不再把高频流式增量事件无界累积到内存;同一响应在多个生命周期事件中重复出现的 Token 用量只统计一次,避免 `Invalid string length` 和成本数据虚高。
|
|
38
36
|
- 后端测试复合执行节点继续保持 clean environment、失败分类和 fail-closed outcome,并为唯一 JUnit/initial Result、canonical Result、traceability 与 Observe 投影保留结构化运行证据。
|
|
39
37
|
|
|
38
|
+
## [0.16.24] - 2026-07-21
|
|
39
|
+
|
|
40
|
+
### 改进
|
|
41
|
+
|
|
42
|
+
- frontend-test 默认改为**短链 RAG**:`reviewMode` 默认 `off`(机械 `frontend-case-checklist-shell` 放行,不再默认双轮 LLM review 硬门);`strictOutcomeGate` 默认 `false`(outcome 全绿门 opt-in)。**流程验收=最终 retrospect 报告**,不以 case 100% 通过为条件。map 仅在 `dynamicExpansion.tolerateChildFailures=true`(frontend-test 默认开启)时将 child auth/timeout/executor 错误落 case-level blocked/failed evidence 且不炸 map closeout;其它 map 仍 fail-closed。
|
|
43
|
+
- Shell `verdictGate` 改为纯 `node -e` 单次调用,降低 Windows Git Bash 复合脚本崩溃风险。
|
|
44
|
+
|
|
45
|
+
### 修复
|
|
46
|
+
|
|
47
|
+
- Windows DAG Shell/verification 的 clean environment 现在保留 `APPDATA`,使 Python 3.12/pip 可按系统规则发现用户级 `site-packages` 与其中的 pytest;仍不会继承完整宿主环境或透传未授权 token/credential,backend-test 无需硬编码用户路径。
|
|
48
|
+
|
|
40
49
|
## [0.16.23] - 2026-07-21
|
|
41
50
|
|
|
42
51
|
### 修复
|
|
@@ -12,13 +12,6 @@ export function expandShellPreset(preset) {
|
|
|
12
12
|
function escapeShellSingleQuoted(value) {
|
|
13
13
|
return value.replace(/'/g, "'\\''");
|
|
14
14
|
}
|
|
15
|
-
const VERDICT_GATE_NODE_PARSER_FIRST_NON_EMPTY = 'node -e \'const fs=require("fs"); const raw=JSON.parse(fs.readFileSync(process.argv[1], "utf8")); const text=raw.assistantText ?? raw.stdout ?? ""; const line=String(text).split(/\\r?\\n/).find((value) => value.trim().length > 0) ?? ""; process.stdout.write(line);\' "${FILE}" || true';
|
|
16
|
-
const VERDICT_GATE_NODE_PARSER_FIRST_VERDICT_LINE = 'node -e \'const fs=require("fs"); const raw=JSON.parse(fs.readFileSync(process.argv[1], "utf8")); const text=raw.assistantText ?? raw.stdout ?? ""; const normalize=(value) => { const trimmed=String(value).trim(); const emphasized=trimmed.match(/^\\*{1,3}\\s*(VERDICT:[^*]+?)\\s*\\*{1,3}$/); return (emphasized ? emphasized[1] : trimmed).trim(); }; const line=String(text).split(/\\r?\\n/).map(normalize).find((value) => /^VERDICT:/.test(value)) ?? ""; process.stdout.write(line);\' "${FILE}" || true';
|
|
17
|
-
function verdictGateNodeParser(lineMode) {
|
|
18
|
-
return lineMode === "first-verdict-line"
|
|
19
|
-
? VERDICT_GATE_NODE_PARSER_FIRST_VERDICT_LINE
|
|
20
|
-
: VERDICT_GATE_NODE_PARSER_FIRST_NON_EMPTY;
|
|
21
|
-
}
|
|
22
15
|
/**
|
|
23
16
|
* Build a single shell command that reads `$HARNESS_DAG_RUN_DIR/<fromNodeId>.json`
|
|
24
17
|
* (injected by the shell executor from runtime meta), parses a single candidate
|
|
@@ -27,33 +20,41 @@ function verdictGateNodeParser(lineMode) {
|
|
|
27
20
|
* - first-non-empty: first non-empty line
|
|
28
21
|
* - first-verdict-line: first line matching /^VERDICT:/ after trim and optional whole-line Markdown emphasis normalization
|
|
29
22
|
*
|
|
30
|
-
*
|
|
23
|
+
* Implemented as one pure `node -e` invocation (no bash state/subshell chaining) so Windows
|
|
24
|
+
* Git Bash cannot crash the gate with STATUS_DLL_INIT_FAILED on long compound scripts.
|
|
31
25
|
*/
|
|
32
26
|
export function buildVerdictGateShellCommand(gate) {
|
|
33
27
|
const gateLabel = gate.label ?? `${gate.fromNodeId} verdict`;
|
|
34
28
|
const lineMode = gate.lineMode ?? "first-non-empty";
|
|
35
|
-
const parser = verdictGateNodeParser(lineMode);
|
|
36
29
|
const candidateNodeIds = [
|
|
37
30
|
gate.fromNodeId,
|
|
38
31
|
...(gate.fallbackFromNodeIds ?? []).filter((id) => id !== gate.fromNodeId),
|
|
39
32
|
];
|
|
40
|
-
const
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
const
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
const
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
.join("
|
|
56
|
-
|
|
33
|
+
const config = {
|
|
34
|
+
label: gateLabel,
|
|
35
|
+
lineMode,
|
|
36
|
+
accept: gate.accept,
|
|
37
|
+
candidateNodeIds,
|
|
38
|
+
};
|
|
39
|
+
const configLiteral = escapeShellSingleQuoted(JSON.stringify(config));
|
|
40
|
+
const program = [
|
|
41
|
+
'const fs=require("fs");',
|
|
42
|
+
'const path=require("path");',
|
|
43
|
+
'const cfg=JSON.parse(process.argv[1]);',
|
|
44
|
+
'const runDir=process.env.HARNESS_DAG_RUN_DIR;',
|
|
45
|
+
'if(!runDir){console.error("missing HARNESS_DAG_RUN_DIR for "+cfg.label+" gate");process.exit(1);}',
|
|
46
|
+
'let file="";',
|
|
47
|
+
'for(const id of cfg.candidateNodeIds){const candidate=path.join(runDir,id+".json");if(fs.existsSync(candidate)){file=candidate;break;}}',
|
|
48
|
+
'if(!file){console.error("missing "+cfg.label+" JSON output (tried: "+cfg.candidateNodeIds.join(", ")+")");process.exit(1);}',
|
|
49
|
+
'const raw=JSON.parse(fs.readFileSync(file,"utf8"));',
|
|
50
|
+
'const text=String(raw.assistantText ?? raw.stdout ?? "");',
|
|
51
|
+
'const normalize=(value)=>{const trimmed=String(value).trim();const emphasized=trimmed.match(/^\\*{1,3}\\s*(VERDICT:[^*]+?)\\s*\\*{1,3}$/);return (emphasized?emphasized[1]:trimmed).trim();};',
|
|
52
|
+
'const lines=text.split(/\\r?\\n/).map(normalize);',
|
|
53
|
+
'const first=cfg.lineMode==="first-verdict-line"? (lines.find((value)=>/^VERDICT:/.test(value))??"") : (lines.find((value)=>value.length>0)??"");',
|
|
54
|
+
'if(!cfg.accept.includes(first)){console.error(cfg.label+" gate blocked: "+(first||"missing VERDICT line"));process.exit(1);}',
|
|
55
|
+
'process.stdout.write(first+"\\n");',
|
|
56
|
+
].join("");
|
|
57
|
+
return `node -e '${escapeShellSingleQuoted(program)}' '${configLiteral}'`;
|
|
57
58
|
}
|
|
58
59
|
/** Build a deterministic current-run gate over exact REQ-/BR-/AC- ids in node facts. */
|
|
59
60
|
export function buildRequirementCoverageGateShellCommand(gate) {
|
|
@@ -65,12 +65,24 @@ export const frontendMockConfigSchema = z.object({
|
|
|
65
65
|
serviceRoot: z.string().min(1).optional(),
|
|
66
66
|
verifyCommands: z.array(frontendMockVerifyCommandSchema).optional().default([]),
|
|
67
67
|
});
|
|
68
|
-
/** Batch limits for the browser-driven frontend test DAG.
|
|
69
|
-
* stop thresholds, not model-provider hard
|
|
68
|
+
/** Batch limits and gate profile for the browser-driven frontend test DAG.
|
|
69
|
+
* Token fields are post-case stop thresholds, not model-provider hard caps. */
|
|
70
70
|
export const frontendTestConfigSchema = z.object({
|
|
71
|
-
maxCasesPerBatch: z.number().int().min(1).max(50).optional()
|
|
71
|
+
maxCasesPerBatch: z.number().int().min(1).max(50).optional(),
|
|
72
72
|
maxTokensPerCase: z.number().int().positive().optional(),
|
|
73
73
|
maxTotalTokens: z.number().int().positive().optional(),
|
|
74
|
+
/**
|
|
75
|
+
* Case review topology:
|
|
76
|
+
* - off (default): mechanical checklist-shell only; no LLM review hard gate
|
|
77
|
+
* - advisory: reserved (same default path as off for now; LLM notes non-blocking)
|
|
78
|
+
* - blocking: legacy dual-review + final VERDICT: pass gate before materialize
|
|
79
|
+
*/
|
|
80
|
+
reviewMode: z.enum(["off", "advisory", "blocking"]).optional(),
|
|
81
|
+
/**
|
|
82
|
+
* When true, include quality outcome gate (all cases passed, 0 blocked).
|
|
83
|
+
* Default false: pipeline success is result materialize + retrospect report, not full green.
|
|
84
|
+
*/
|
|
85
|
+
strictOutcomeGate: z.boolean().optional(),
|
|
74
86
|
});
|
|
75
87
|
export const convergenceConfigSchema = z.object({
|
|
76
88
|
enabled: z.boolean().optional().default(false),
|
|
@@ -299,7 +299,7 @@ function coerceBoundary(value) {
|
|
|
299
299
|
return null;
|
|
300
300
|
return { field, constraint, sourceRef: firstSourceRef(record) };
|
|
301
301
|
}
|
|
302
|
-
const
|
|
302
|
+
const OPTIONAL_EVIDENCE_BASE_KEYS = [
|
|
303
303
|
"name",
|
|
304
304
|
"id",
|
|
305
305
|
"description",
|
|
@@ -308,21 +308,33 @@ const OPTIONAL_EVIDENCE_KNOWN_KEYS = new Set([
|
|
|
308
308
|
"title",
|
|
309
309
|
"sourceRef",
|
|
310
310
|
"sourceRefs",
|
|
311
|
+
];
|
|
312
|
+
const EXTERNAL_DEPENDENCY_INPUT_KEYS = new Set([
|
|
313
|
+
...OPTIONAL_EVIDENCE_BASE_KEYS,
|
|
314
|
+
"type",
|
|
315
|
+
"target",
|
|
316
|
+
"kind",
|
|
317
|
+
"required",
|
|
318
|
+
]);
|
|
319
|
+
const RISK_INPUT_KEYS = new Set([
|
|
320
|
+
...OPTIONAL_EVIDENCE_BASE_KEYS,
|
|
321
|
+
"type",
|
|
311
322
|
"severity",
|
|
312
323
|
"mitigation",
|
|
313
324
|
"level",
|
|
314
325
|
"impact",
|
|
315
|
-
"kind",
|
|
316
|
-
"required",
|
|
317
326
|
]);
|
|
318
|
-
function coerceOptionalEvidence(value) {
|
|
327
|
+
function coerceOptionalEvidence(value, kind) {
|
|
319
328
|
const record = asRecord(value);
|
|
320
329
|
if (!record)
|
|
321
330
|
return null;
|
|
331
|
+
const knownKeys = kind === "external-dependency"
|
|
332
|
+
? EXTERNAL_DEPENDENCY_INPUT_KEYS
|
|
333
|
+
: RISK_INPUT_KEYS;
|
|
322
334
|
// Fail closed on unknown keys so near-schema payloads cannot strip extras and pass.
|
|
323
335
|
for (const key of Object.keys(record)) {
|
|
324
|
-
if (!
|
|
325
|
-
throw new Error(
|
|
336
|
+
if (!knownKeys.has(key)) {
|
|
337
|
+
throw new Error(`${kind} evidence has unrecognized key: ${key}`);
|
|
326
338
|
}
|
|
327
339
|
}
|
|
328
340
|
const description = (typeof record.description === "string" && record.description.trim()) ||
|
|
@@ -334,6 +346,7 @@ function coerceOptionalEvidence(value) {
|
|
|
334
346
|
return null;
|
|
335
347
|
const out = { description };
|
|
336
348
|
const name = (typeof record.name === "string" && record.name.trim()) ||
|
|
349
|
+
(kind === "external-dependency" && typeof record.target === "string" && record.target.trim()) ||
|
|
337
350
|
(typeof record.id === "string" && record.id.trim()) ||
|
|
338
351
|
"";
|
|
339
352
|
if (name)
|
|
@@ -385,10 +398,10 @@ export function coerceBackendTestAnalysisInput(value) {
|
|
|
385
398
|
.map(coerceBoundary)
|
|
386
399
|
.filter((item) => item !== null);
|
|
387
400
|
next.externalDependencies = asArray(record.externalDependencies)
|
|
388
|
-
.map(coerceOptionalEvidence)
|
|
401
|
+
.map((item) => coerceOptionalEvidence(item, "external-dependency"))
|
|
389
402
|
.filter((item) => item !== null);
|
|
390
403
|
next.risks = asArray(record.risks)
|
|
391
|
-
.map(coerceOptionalEvidence)
|
|
404
|
+
.map((item) => coerceOptionalEvidence(item, "risk"))
|
|
392
405
|
.filter((item) => item !== null);
|
|
393
406
|
next.evidenceGaps = asArray(record.evidenceGaps)
|
|
394
407
|
.map(coerceEvidenceGap)
|
|
@@ -81,27 +81,77 @@ export function resolveRunLocalPath(runDir, ref, label) {
|
|
|
81
81
|
}
|
|
82
82
|
return resolved;
|
|
83
83
|
}
|
|
84
|
+
function resolveCaseIdFromItem(item) {
|
|
85
|
+
if (typeof item === "object" &&
|
|
86
|
+
item !== null &&
|
|
87
|
+
typeof item.caseId === "string") {
|
|
88
|
+
return item.caseId;
|
|
89
|
+
}
|
|
90
|
+
return String(item);
|
|
91
|
+
}
|
|
92
|
+
async function materializeBlockedCaseEvidence(input) {
|
|
93
|
+
if (!input.workspaceRef)
|
|
94
|
+
return;
|
|
95
|
+
const workspace = resolveRunLocalPath(input.cwd, input.workspaceRef, "workspaceRef");
|
|
96
|
+
await mkdir(workspace, { recursive: true });
|
|
97
|
+
const resultPath = path.join(workspace, "case-result.json");
|
|
98
|
+
const caseId = resolveCaseIdFromItem(input.item);
|
|
99
|
+
try {
|
|
100
|
+
const existing = JSON.parse(await readFile(resultPath, "utf-8"));
|
|
101
|
+
if (existing.caseId === caseId &&
|
|
102
|
+
(existing.status === "passed" ||
|
|
103
|
+
existing.status === "failed" ||
|
|
104
|
+
existing.status === "blocked"))
|
|
105
|
+
return;
|
|
106
|
+
}
|
|
107
|
+
catch {
|
|
108
|
+
// Missing or malformed evidence is replaced only for a child that never finished a business result.
|
|
109
|
+
}
|
|
110
|
+
await writeFile(path.join(workspace, "execution.md"), `# ${caseId}\n\nStatus: blocked\n\nReason: ${input.reason}\n`, "utf-8");
|
|
111
|
+
await writeFile(resultPath, `${JSON.stringify({
|
|
112
|
+
caseId,
|
|
113
|
+
status: "blocked",
|
|
114
|
+
blockedReason: input.reason,
|
|
115
|
+
evidencePaths: ["execution.md"],
|
|
116
|
+
}, null, 2)}\n`, "utf-8");
|
|
117
|
+
}
|
|
84
118
|
async function materializeTokenBudgetBlockedEvidence(input) {
|
|
119
|
+
await materializeBlockedCaseEvidence({
|
|
120
|
+
...input,
|
|
121
|
+
reason: "token-budget-exhausted",
|
|
122
|
+
});
|
|
123
|
+
}
|
|
124
|
+
async function materializeFailedCaseEvidence(input) {
|
|
85
125
|
if (!input.workspaceRef)
|
|
86
126
|
return;
|
|
87
127
|
const workspace = resolveRunLocalPath(input.cwd, input.workspaceRef, "workspaceRef");
|
|
88
128
|
await mkdir(workspace, { recursive: true });
|
|
89
129
|
const resultPath = path.join(workspace, "case-result.json");
|
|
90
|
-
const caseId =
|
|
91
|
-
typeof input.item.caseId === "string"
|
|
92
|
-
? input.item.caseId
|
|
93
|
-
: String(input.item);
|
|
130
|
+
const caseId = resolveCaseIdFromItem(input.item);
|
|
94
131
|
try {
|
|
95
132
|
const existing = JSON.parse(await readFile(resultPath, "utf-8"));
|
|
96
133
|
if (existing.caseId === caseId &&
|
|
97
|
-
(existing.status === "passed" ||
|
|
134
|
+
(existing.status === "passed" ||
|
|
135
|
+
existing.status === "failed" ||
|
|
136
|
+
existing.status === "blocked"))
|
|
98
137
|
return;
|
|
99
138
|
}
|
|
100
139
|
catch {
|
|
101
|
-
// Missing or malformed evidence is replaced only
|
|
140
|
+
// Missing or malformed evidence is replaced only when the child never finished a business result.
|
|
102
141
|
}
|
|
103
|
-
await writeFile(path.join(workspace, "execution.md"), `# ${caseId}\n\nStatus:
|
|
104
|
-
await writeFile(resultPath, `${JSON.stringify({
|
|
142
|
+
await writeFile(path.join(workspace, "execution.md"), `# ${caseId}\n\nStatus: failed\n\nReason: ${input.reason}\n`, "utf-8");
|
|
143
|
+
await writeFile(resultPath, `${JSON.stringify({
|
|
144
|
+
caseId,
|
|
145
|
+
status: "failed",
|
|
146
|
+
errorSummary: input.reason,
|
|
147
|
+
evidencePaths: ["execution.md"],
|
|
148
|
+
}, null, 2)}\n`, "utf-8");
|
|
149
|
+
}
|
|
150
|
+
/** Provider auth/timeout on a FE case child is infrastructure, not a product fail; keep the map aggregate. */
|
|
151
|
+
function isInfrastructureCaseChildFailure(record) {
|
|
152
|
+
if (!record || record.status === "FINISHED")
|
|
153
|
+
return false;
|
|
154
|
+
return record.failureCategory === "auth" || record.failureCategory === "timeout";
|
|
105
155
|
}
|
|
106
156
|
export async function executeDynamicMapExpansion(input) {
|
|
107
157
|
const started = Date.now();
|
|
@@ -264,15 +314,64 @@ export async function executeDynamicMapExpansion(input) {
|
|
|
264
314
|
tokenBudgetExhausted = true;
|
|
265
315
|
}
|
|
266
316
|
}
|
|
317
|
+
// Opt-in (frontend-test via tolerateChildFailures): materialize case-level
|
|
318
|
+
// failed/blocked evidence and keep the map barrier green so validate /
|
|
319
|
+
// result / retrospect can still run. Default maps stay fail-closed on child ERROR.
|
|
320
|
+
const tolerateChildFailures = input.expansion.tolerateChildFailures === true;
|
|
321
|
+
const caseOutcomeNotes = [];
|
|
322
|
+
if (tolerateChildFailures) {
|
|
323
|
+
for (let index = 0; index < childNodeIds.length; index += 1) {
|
|
324
|
+
const nodeId = childNodeIds[index];
|
|
325
|
+
const record = input.state.nodes[nodeId];
|
|
326
|
+
if (!record || record.status === "FINISHED")
|
|
327
|
+
continue;
|
|
328
|
+
if (isBudgetBlockedRecord(record))
|
|
329
|
+
continue;
|
|
330
|
+
const infra = isInfrastructureCaseChildFailure(record);
|
|
331
|
+
const reason = infra
|
|
332
|
+
? record?.failureCategory === "timeout"
|
|
333
|
+
? "executor-timeout"
|
|
334
|
+
: "executor-auth-unavailable"
|
|
335
|
+
: `executor-error:${record.failureCategory || record.status || "unknown"}`;
|
|
336
|
+
const outcomeStatus = infra ? "blocked" : "failed";
|
|
337
|
+
try {
|
|
338
|
+
if (outcomeStatus === "blocked") {
|
|
339
|
+
await materializeBlockedCaseEvidence({
|
|
340
|
+
cwd: input.cwd,
|
|
341
|
+
workspaceRef: workspaceRefs[index],
|
|
342
|
+
item: items[index],
|
|
343
|
+
reason,
|
|
344
|
+
});
|
|
345
|
+
}
|
|
346
|
+
else {
|
|
347
|
+
await materializeFailedCaseEvidence({
|
|
348
|
+
cwd: input.cwd,
|
|
349
|
+
workspaceRef: workspaceRefs[index],
|
|
350
|
+
item: items[index],
|
|
351
|
+
reason,
|
|
352
|
+
});
|
|
353
|
+
}
|
|
354
|
+
}
|
|
355
|
+
catch (error) {
|
|
356
|
+
throw new Error(`failed to materialize case outcome evidence for ${nodeId}: ${error instanceof Error ? error.message : String(error)}`);
|
|
357
|
+
}
|
|
358
|
+
caseOutcomeNotes.push({ nodeId, reason, status: outcomeStatus });
|
|
359
|
+
record.status = "FINISHED";
|
|
360
|
+
record.stderr = `${outcomeStatus}: ${reason}`;
|
|
361
|
+
record.failureCategory = "success";
|
|
362
|
+
}
|
|
363
|
+
}
|
|
267
364
|
const budgetBlockedIds = new Set(blockedChildren.map((entry) => entry.nodeId));
|
|
268
|
-
const failedChildren =
|
|
269
|
-
|
|
270
|
-
|
|
271
|
-
|
|
272
|
-
|
|
273
|
-
|
|
274
|
-
|
|
275
|
-
|
|
365
|
+
const failedChildren = tolerateChildFailures
|
|
366
|
+
? []
|
|
367
|
+
: childNodeIds.filter((nodeId) => {
|
|
368
|
+
const record = input.state.nodes[nodeId];
|
|
369
|
+
if (!record || record.status === "FINISHED")
|
|
370
|
+
return false;
|
|
371
|
+
if (budgetBlockedIds.has(nodeId) || isBudgetBlockedRecord(record))
|
|
372
|
+
return false;
|
|
373
|
+
return true;
|
|
374
|
+
});
|
|
276
375
|
const aggregate = {
|
|
277
376
|
workflowNodeId: input.expansion.workflowNodeId,
|
|
278
377
|
itemCount: items.length,
|
|
@@ -285,21 +384,29 @@ export async function executeDynamicMapExpansion(input) {
|
|
|
285
384
|
output: parseJsonFromText(input.state.nodes[nodeId]?.stdout),
|
|
286
385
|
assistantText: input.state.nodes[nodeId]?.assistantText,
|
|
287
386
|
blocked: blockedChildren.find((child) => child.nodeId === nodeId)?.reason ??
|
|
387
|
+
caseOutcomeNotes.find((child) => child.nodeId === nodeId && child.status === "blocked")
|
|
388
|
+
?.reason ??
|
|
288
389
|
(isBudgetBlockedRecord(input.state.nodes[nodeId])
|
|
289
390
|
? "token-budget-exhausted"
|
|
290
391
|
: undefined),
|
|
392
|
+
failed: caseOutcomeNotes.find((child) => child.nodeId === nodeId && child.status === "failed")
|
|
393
|
+
?.reason,
|
|
291
394
|
})),
|
|
292
395
|
tokensUsed: totalTokensUsed,
|
|
396
|
+
caseOutcomeNotes: caseOutcomeNotes.length > 0 ? caseOutcomeNotes : undefined,
|
|
293
397
|
};
|
|
294
398
|
return {
|
|
295
|
-
//
|
|
296
|
-
//
|
|
297
|
-
// Real child failures remain fail-closed even when later cases are budget-blocked.
|
|
399
|
+
// With tolerateChildFailures: barrier always succeeds; product outcomes live in
|
|
400
|
+
// case evidence + result materialization. Without it: real child failures fail-close.
|
|
298
401
|
ok: failedChildren.length === 0,
|
|
299
402
|
stdout: JSON.stringify(aggregate),
|
|
300
|
-
stderr:
|
|
301
|
-
? `
|
|
302
|
-
|
|
403
|
+
stderr: caseOutcomeNotes.length > 0
|
|
404
|
+
? `map children recorded as case outcomes: ${caseOutcomeNotes
|
|
405
|
+
.map((entry) => `${entry.nodeId}=${entry.status}:${entry.reason}`)
|
|
406
|
+
.join(", ")}`
|
|
407
|
+
: failedChildren.length > 0
|
|
408
|
+
? `dynamic map children failed: ${failedChildren.join(", ")}`
|
|
409
|
+
: "",
|
|
303
410
|
failureCategory: failedChildren.length > 0 ? "dynamic-expansion-child-failed" : "success",
|
|
304
411
|
durationMs: Date.now() - started,
|
|
305
412
|
};
|
|
@@ -2546,6 +2546,7 @@ function buildAnalyzeInputsNode(sources) {
|
|
|
2546
2546
|
"For every endpoint, explicitly set responseBody.kind=array|object|scalar|empty|unknown and ordering=specified|unspecified|not-applicable. Add itemSchemaRef for arrays when documented.",
|
|
2547
2547
|
"For response fields, use comparison=exact|parseable-only|semantic when the source defines assertion semantics; date-time fields whose precision is unspecified should use parseable-only, not string equality.",
|
|
2548
2548
|
"Endpoint sourceRefs and field sourceRefs must cite only requirement/reference evidence actually read. Empty sourceRefs are allowed only when normalizing legacy v1 input; newly generated v2 should cite evidence.",
|
|
2549
|
+
"For externalDependencies and risks, emit canonical items with exactly description plus optional name and sourceRef. For a dependency target, put the target value in name. Do not emit type, target, kind, required, severity, mitigation, level, impact, sourceRefs, or custom keys in newly generated v2 output.",
|
|
2549
2550
|
"Use empty arrays for categories not documented. Never include credentials, tokens, private keys, or secret values.",
|
|
2550
2551
|
"Required top-level keys: schemaVersion=2, sourceBinding, acceptanceCriteria, endpoints, dataModels, businessRules, stateTransitions, boundaryConstraints, externalDependencies, risks, evidenceGaps.",
|
|
2551
2552
|
"Read-only: do not modify code, docs, artifacts, or repository files.",
|
|
@@ -3312,7 +3313,17 @@ function buildBackendTestHybridDag(sources) {
|
|
|
3312
3313
|
// Frontend browser-test RAG DAG template
|
|
3313
3314
|
// ---------------------------------------------------------------------------
|
|
3314
3315
|
function buildFrontendTestHybridDag(sources) {
|
|
3315
|
-
const
|
|
3316
|
+
const rawFrontendTest = sources.taskConfig.frontendTest;
|
|
3317
|
+
const config = {
|
|
3318
|
+
maxCasesPerBatch: rawFrontendTest?.maxCasesPerBatch ?? 20,
|
|
3319
|
+
maxTokensPerCase: rawFrontendTest?.maxTokensPerCase,
|
|
3320
|
+
maxTotalTokens: rawFrontendTest?.maxTotalTokens,
|
|
3321
|
+
reviewMode: rawFrontendTest?.reviewMode ?? "off",
|
|
3322
|
+
strictOutcomeGate: rawFrontendTest?.strictOutcomeGate === true,
|
|
3323
|
+
};
|
|
3324
|
+
const reviewMode = config.reviewMode;
|
|
3325
|
+
const blockingReview = reviewMode === "blocking";
|
|
3326
|
+
const strictOutcomeGate = config.strictOutcomeGate;
|
|
3316
3327
|
const hasFrontendTestWriteScope = sources.taskConfig.allowedPaths.some((pattern) => pattern === "testcase/frontend/**" ||
|
|
3317
3328
|
pattern === "testcase/**" ||
|
|
3318
3329
|
pattern === "**");
|
|
@@ -3323,6 +3334,36 @@ function buildFrontendTestHybridDag(sources) {
|
|
|
3323
3334
|
const ragWriteSet = ["testcase/frontend/rag/**"];
|
|
3324
3335
|
const casesWriteSet = ["testcase/frontend/cases/**"];
|
|
3325
3336
|
const evidenceRoot = "testcase/frontend/evidence";
|
|
3337
|
+
const checklistValidation = [
|
|
3338
|
+
"node -e",
|
|
3339
|
+
JSON.stringify([
|
|
3340
|
+
"const fs=require('fs'),path=require('path');",
|
|
3341
|
+
"const root='testcase/frontend/cases';",
|
|
3342
|
+
"const draft=path.join(root,'manifest.draft.json');",
|
|
3343
|
+
"const final=path.join(root,'manifest.json');",
|
|
3344
|
+
"const manifestPath=fs.existsSync(draft)?draft:(fs.existsSync(final)?final:null);",
|
|
3345
|
+
"if(!manifestPath)throw new Error('checklist: missing manifest.draft.json or manifest.json');",
|
|
3346
|
+
"const manifest=JSON.parse(fs.readFileSync(manifestPath,'utf8'));",
|
|
3347
|
+
"if(!Array.isArray(manifest.cases)||manifest.cases.length===0)throw new Error('checklist: empty cases');",
|
|
3348
|
+
"const issues=[];",
|
|
3349
|
+
"const openRe=/playwright-cli\\s+open\\s+--browser=chrome\\s+--headed\\s+https?:\\/\\/\\S+/i;",
|
|
3350
|
+
"const prodRe=/(?:^|\\/\\/)(?:www\\.)?[^\\s\\/]*(?:prod|production)/i;",
|
|
3351
|
+
"const codeRe=/\\b(pytest|playwright\\.test|@playwright\\/test)\\b/i;",
|
|
3352
|
+
"for(const c of manifest.cases){",
|
|
3353
|
+
" const id=c&&c.caseId||'?';",
|
|
3354
|
+
" const casePath=typeof c.casePath==='string'?c.casePath:null;",
|
|
3355
|
+
" if(!casePath||!fs.existsSync(casePath)){issues.push({ruleId:'case-file-missing',caseId:id,detail:String(casePath)});continue;}",
|
|
3356
|
+
" const body=fs.readFileSync(casePath,'utf8');",
|
|
3357
|
+
" if(!openRe.test(body))issues.push({ruleId:'open-prefix',caseId:id,detail:'missing playwright-cli open --browser=chrome --headed <absolute-url>'});",
|
|
3358
|
+
" const m=body.match(/playwright-cli\\s+open\\s+--browser=chrome\\s+--headed\\s+(https?:\\/\\/\\S+)/i);",
|
|
3359
|
+
" if(m){const url=m[1].replace(/[)\\]},.\"']+$/,''); if(prodRe.test(url))issues.push({ruleId:'production-url',caseId:id,detail:url});}",
|
|
3360
|
+
" if(codeRe.test(body))issues.push({ruleId:'no-test-source',caseId:id,detail:'pytest/playwright test source forbidden'});",
|
|
3361
|
+
" if(!Array.isArray(c.acIds)||c.acIds.length===0)issues.push({ruleId:'ac-mapping',caseId:id,detail:'acIds required'});",
|
|
3362
|
+
"}",
|
|
3363
|
+
"if(issues.length){console.error('frontend-test checklist blocked: '+JSON.stringify(issues)); process.exit(1);}",
|
|
3364
|
+
"console.log('frontend-test checklist ok cases='+manifest.cases.length+' source='+path.basename(manifestPath));",
|
|
3365
|
+
].join("")),
|
|
3366
|
+
].join(" ");
|
|
3326
3367
|
const manifestValidation = [
|
|
3327
3368
|
"node -e",
|
|
3328
3369
|
JSON.stringify([
|
|
@@ -3338,7 +3379,6 @@ function buildFrontendTestHybridDag(sources) {
|
|
|
3338
3379
|
" if(!Array.isArray(c.acIds)||c.acIds.length===0||c.acIds.some(a=>typeof a!=='string'||!a.trim())) throw new Error('invalid acIds');",
|
|
3339
3380
|
" for(const k of ['casePath','evidenceDir']){ const v=c[k]; if(typeof v!=='string'||path.isAbsolute(v)||v.includes('..')) throw new Error('unsafe '+k); }",
|
|
3340
3381
|
" if(c.casePath!=='testcase/frontend/cases/'+c.caseId+'.md') throw new Error('casePath must match caseId');",
|
|
3341
|
-
// Accept evidenceDir as the case root or a nested path under that root.
|
|
3342
3382
|
" { const prefix='testcase/frontend/evidence/'+c.caseId; if(!(c.evidenceDir===prefix||c.evidenceDir.startsWith(prefix+'/'))) throw new Error('case path escapes frontend test roots'); }",
|
|
3343
3383
|
" if(!fs.existsSync(c.casePath)) throw new Error('missing case file '+c.casePath);",
|
|
3344
3384
|
" if(seenCasePath.has(c.casePath)) throw new Error('duplicate casePath'); seenCasePath.add(c.casePath);",
|
|
@@ -3360,6 +3400,290 @@ function buildFrontendTestHybridDag(sources) {
|
|
|
3360
3400
|
].join("")),
|
|
3361
3401
|
].join(" ");
|
|
3362
3402
|
const frontendTestOutcomeGate = buildFrontendTestOutcomeGateShellSnippet();
|
|
3403
|
+
const tasks = [
|
|
3404
|
+
{
|
|
3405
|
+
id: "retrieve-frontend-test-context-pi",
|
|
3406
|
+
depends_on: [],
|
|
3407
|
+
role: "planner",
|
|
3408
|
+
executor: "pi",
|
|
3409
|
+
toolProfile: "write",
|
|
3410
|
+
complexity: "MED",
|
|
3411
|
+
writePolicy: "exclusive",
|
|
3412
|
+
writeSet: ragWriteSet,
|
|
3413
|
+
allowedPaths: [...commonReadOnlyPaths(sources), ...ragWriteSet],
|
|
3414
|
+
forbiddenPaths: forbidden,
|
|
3415
|
+
outputContract: "Write short testcase/frontend/rag/context.md and coverage-map.md with machine-readable baseUrl and capability notes.",
|
|
3416
|
+
subtask_prompt: [
|
|
3417
|
+
"Build the frontend test RAG package (keep it short).",
|
|
3418
|
+
"Read task source, routes/components/API or Mock facts, and execution contract. Write only testcase/frontend/rag/context.md and coverage-map.md.",
|
|
3419
|
+
"Prefer fixed fields: baseUrl, baseUrlSource, AC table, capability matrix (backend real/mock, pagination data, HTTP observation, error injection), risks, forbidden hosts. Do not paste large implementation dumps.",
|
|
3420
|
+
"Base URL resolution (required): (1) Prefer absolute http(s) frontend URL from task source config.md. (2) Else default http://localhost:5173. (3) Never production hosts. (4) Write `baseUrl: <url>` and `baseUrlSource: config.md|<path>|default-localhost-5173`. (5) Include exact start prefix: playwright-cli open --browser=chrome --headed <resolved-base-url>.",
|
|
3421
|
+
buildSourceContextBlock(sources),
|
|
3422
|
+
].join("\n\n"),
|
|
3423
|
+
},
|
|
3424
|
+
{
|
|
3425
|
+
id: "materialize-frontend-test-execution-shell",
|
|
3426
|
+
depends_on: ["retrieve-frontend-test-context-pi"],
|
|
3427
|
+
role: "verifier",
|
|
3428
|
+
executor: "shell",
|
|
3429
|
+
complexity: "LOW",
|
|
3430
|
+
writePolicy: "read-only",
|
|
3431
|
+
allowedPaths: [...ragWriteSet],
|
|
3432
|
+
forbiddenPaths: forbidden,
|
|
3433
|
+
outputContract: "Fail-closed preflight: absolute non-production baseUrl required; fixture/reset not hard-gated.",
|
|
3434
|
+
subtask_prompt: "Hard-validate only an absolute non-production baseUrl in RAG context (from config.md or default http://localhost:5173). Fixture/reset and other isolation details are soft guidance for later nodes, not preflight failures.",
|
|
3435
|
+
shell: {
|
|
3436
|
+
commands: [
|
|
3437
|
+
[
|
|
3438
|
+
"node -e",
|
|
3439
|
+
JSON.stringify("const fs=require('fs'); const p='testcase/frontend/rag/context.md'; if(!fs.existsSync(p))throw new Error('missing '+p); const s=fs.readFileSync(p,'utf8'); const patterns=[ /baseUrl\\s*[:=]\\s*(https?:\\/\\/\\S+)/i, /base[- ]url\\s*[:=]\\s*(https?:\\/\\/\\S+)/i, /playwright-cli open --browser=chrome --headed\\s+(https?:\\/\\/\\S+)/i, /(https?:\\/\\/(?:localhost|127\\.0\\.0\\.1)[^\\s)\\]},\"']*)/i ]; let baseUrl=null; for(const re of patterns){const m=s.match(re); if(m){baseUrl=m[1]; break;}} if(!baseUrl)throw new Error('frontend-test preflight missing absolute baseUrl (prefer config.md; default http://localhost:5173)'); baseUrl=baseUrl.replace(/[)\\]},.\"']+$/,''); if(!/^https?:\\/\\//i.test(baseUrl))throw new Error('baseUrl must be absolute http(s): '+baseUrl); if(/(?:^|\\/\\/)(?:www\\.)?[^\\s/]*(?:prod|production)/i.test(baseUrl))throw new Error('production URL forbidden: '+baseUrl); console.log('frontend-test-execution-v1 validated baseUrl='+baseUrl);"),
|
|
3440
|
+
].join(" "),
|
|
3441
|
+
],
|
|
3442
|
+
cwd: ".",
|
|
3443
|
+
timeoutMs: 60000,
|
|
3444
|
+
},
|
|
3445
|
+
},
|
|
3446
|
+
{
|
|
3447
|
+
id: "generate-frontend-functional-cases-pi",
|
|
3448
|
+
depends_on: ["materialize-frontend-test-execution-shell"],
|
|
3449
|
+
role: "implementer",
|
|
3450
|
+
executor: "pi",
|
|
3451
|
+
toolProfile: "write",
|
|
3452
|
+
complexity: "HIGH",
|
|
3453
|
+
writePolicy: "exclusive",
|
|
3454
|
+
writeSet: casesWriteSet,
|
|
3455
|
+
allowedPaths: [...ragWriteSet, ...casesWriteSet],
|
|
3456
|
+
forbiddenPaths: forbidden,
|
|
3457
|
+
outputContract: "Write executable Markdown frontend cases, index.md, and manifest.draft.json schemaVersion 1; no test source code.",
|
|
3458
|
+
subtask_prompt: [
|
|
3459
|
+
"Use skill playwright-cli-case-generator.",
|
|
3460
|
+
"Read only testcase/frontend/rag/context.md, testcase/frontend/rag/coverage-map.md, and existing testcase/frontend/cases/. Write only testcase/frontend/cases/**.",
|
|
3461
|
+
"Generate Markdown cases, index.md and manifest.draft.json (schemaVersion 1; cases[] with caseId, casePath, dimension, acIds, evidenceDir). IDs use FE-<FEATURE>-<NNN>-<dimension>; dimensions core|boundary|flow|backend.",
|
|
3462
|
+
"Prefer a small smoke suite (default max roughly 4–8 cases unless task frontendTest.maxCasesPerBatch is higher). Never invent unavailable API fields or credentials. Do not create pytest or Playwright source.",
|
|
3463
|
+
"Copy the resolved absolute baseUrl from context.md (baseUrl field; resolved from config.md or default http://localhost:5173). Every browser start command must be: playwright-cli open --browser=chrome --headed <resolved-base-url-from-context.md> with that concrete URL — never leave a <base-url> placeholder. Use default browser session only; never write -s=<case-id>.",
|
|
3464
|
+
"Each case must be independently reproducible with fixture/reset, UI reset, snapshot-before-ref, evidence write point under testcase/frontend/evidence/<case-id>/. If the isolated environment is unavailable, require writing blocked evidence before any browser command.",
|
|
3465
|
+
].join("\n\n"),
|
|
3466
|
+
},
|
|
3467
|
+
];
|
|
3468
|
+
if (blockingReview) {
|
|
3469
|
+
tasks.push({
|
|
3470
|
+
id: "review-frontend-cases-pi",
|
|
3471
|
+
depends_on: ["generate-frontend-functional-cases-pi"],
|
|
3472
|
+
role: "reviewer",
|
|
3473
|
+
executor: "pi",
|
|
3474
|
+
complexity: "HIGH",
|
|
3475
|
+
writePolicy: "read-only",
|
|
3476
|
+
allowedPaths: [...ragWriteSet, ...casesWriteSet],
|
|
3477
|
+
forbiddenPaths: forbidden,
|
|
3478
|
+
outputContract: "First line VERDICT: pass or VERDICT: request-revision, followed by AC-to-case coverage and execution risk findings; no writes. request-revision blocks manifest materialization.",
|
|
3479
|
+
subtask_prompt: "Review only the RAG package, frontend Markdown cases, and manifest.draft.json. Verify traceability, independent execution, safe data/environment handling, manifest correctness, session consistency, fixture/UI reset and fresh snapshot steps, and evidence requirements. Any Important or Critical finding requires VERDICT: request-revision. Browser execution is blocked unless this review passes.",
|
|
3480
|
+
}, {
|
|
3481
|
+
id: "revise-frontend-cases-pi",
|
|
3482
|
+
depends_on: ["review-frontend-cases-pi"],
|
|
3483
|
+
runIf: "$.nodes['review-frontend-cases-pi'].firstVerdictLine == 'VERDICT: request-revision'",
|
|
3484
|
+
role: "implementer",
|
|
3485
|
+
executor: "pi",
|
|
3486
|
+
toolProfile: "write",
|
|
3487
|
+
complexity: "HIGH",
|
|
3488
|
+
writePolicy: "exclusive",
|
|
3489
|
+
writeSet: casesWriteSet,
|
|
3490
|
+
allowedPaths: [...ragWriteSet, ...casesWriteSet],
|
|
3491
|
+
forbiddenPaths: forbidden,
|
|
3492
|
+
outputContract: "Apply the one permitted frontend case revision under testcase/frontend/cases/** only; no browser execution or evidence writes.",
|
|
3493
|
+
subtask_prompt: "This is the only permitted case revision. Read the first review findings and the RAG package. Revise only testcase/frontend/cases/**, preserve traceable AC mappings, and do not execute a browser or write evidence. HARD: Never delete case files; only edit in place or add missing cases. Preserve the full planned suite, index.md, and manifest.draft.json.",
|
|
3494
|
+
}, {
|
|
3495
|
+
id: "review-frontend-cases-final-pi",
|
|
3496
|
+
depends_on: ["revise-frontend-cases-pi"],
|
|
3497
|
+
runIf: "$.nodes['review-frontend-cases-pi'].firstVerdictLine == 'VERDICT: request-revision'",
|
|
3498
|
+
role: "reviewer",
|
|
3499
|
+
executor: "pi",
|
|
3500
|
+
complexity: "HIGH",
|
|
3501
|
+
writePolicy: "read-only",
|
|
3502
|
+
allowedPaths: [...ragWriteSet, ...casesWriteSet],
|
|
3503
|
+
forbiddenPaths: forbidden,
|
|
3504
|
+
outputContract: "First line VERDICT: pass or VERDICT: request-revision after the single allowed case revision; no writes.",
|
|
3505
|
+
subtask_prompt: "Perform the final frontend case review after the sole permitted revision. Apply the same traceability, isolation, manifest, reset, session, snapshot, and evidence checks. First verdict line must be exact; any Important or Critical finding requires request-revision. Do not write files.",
|
|
3506
|
+
}, {
|
|
3507
|
+
id: "final-frontend-case-review-gate-shell",
|
|
3508
|
+
depends_on: ["review-frontend-cases-pi", "review-frontend-cases-final-pi"],
|
|
3509
|
+
dependsPolicy: "all-or-condition-skip",
|
|
3510
|
+
role: "verifier",
|
|
3511
|
+
executor: "shell",
|
|
3512
|
+
complexity: "LOW",
|
|
3513
|
+
writePolicy: "read-only",
|
|
3514
|
+
allowedPaths: [...ragWriteSet, ...casesWriteSet],
|
|
3515
|
+
forbiddenPaths: forbidden,
|
|
3516
|
+
outputContract: "Pass-only effective frontend case review gate; final review takes precedence when the revision branch ran.",
|
|
3517
|
+
subtask_prompt: "Authorize checklist/manifest materialization only after the effective frontend case review passes.",
|
|
3518
|
+
shell: {
|
|
3519
|
+
commands: [],
|
|
3520
|
+
verdictGate: {
|
|
3521
|
+
fromNodeId: "review-frontend-cases-final-pi",
|
|
3522
|
+
fallbackFromNodeIds: ["review-frontend-cases-pi"],
|
|
3523
|
+
accept: ["VERDICT: pass"],
|
|
3524
|
+
label: "effective frontend case review",
|
|
3525
|
+
lineMode: "first-verdict-line",
|
|
3526
|
+
},
|
|
3527
|
+
cwd: ".",
|
|
3528
|
+
timeoutMs: 60000,
|
|
3529
|
+
},
|
|
3530
|
+
});
|
|
3531
|
+
}
|
|
3532
|
+
const checklistDependsOn = blockingReview
|
|
3533
|
+
? ["final-frontend-case-review-gate-shell"]
|
|
3534
|
+
: ["generate-frontend-functional-cases-pi"];
|
|
3535
|
+
tasks.push({
|
|
3536
|
+
id: "frontend-case-checklist-shell",
|
|
3537
|
+
depends_on: checklistDependsOn,
|
|
3538
|
+
role: "verifier",
|
|
3539
|
+
executor: "shell",
|
|
3540
|
+
complexity: "LOW",
|
|
3541
|
+
writePolicy: "read-only",
|
|
3542
|
+
allowedPaths: [...ragWriteSet, ...casesWriteSet],
|
|
3543
|
+
forbiddenPaths: forbidden,
|
|
3544
|
+
outputContract: "Mechanical checklist: open-prefix, non-prod absolute URL, acIds, no pytest/playwright test source; emit structured ruleId issues on failure.",
|
|
3545
|
+
subtask_prompt: "Scan generated cases/manifest against the shared blocking checklist. Do not use free-form LLM verdicts.",
|
|
3546
|
+
shell: { commands: [checklistValidation], cwd: ".", timeoutMs: 120000 },
|
|
3547
|
+
}, {
|
|
3548
|
+
id: "materialize-frontend-case-manifest-shell",
|
|
3549
|
+
depends_on: ["frontend-case-checklist-shell"],
|
|
3550
|
+
role: "verifier",
|
|
3551
|
+
executor: "shell",
|
|
3552
|
+
complexity: "LOW",
|
|
3553
|
+
writePolicy: "exclusive",
|
|
3554
|
+
writeSet: casesWriteSet,
|
|
3555
|
+
allowedPaths: casesWriteSet,
|
|
3556
|
+
forbiddenPaths: forbidden,
|
|
3557
|
+
outputContract: "Validated frontend manifest payload { cases: [...] }; atomically materialize testcase/frontend/cases/manifest.json from manifest.draft.json; shell output may echo only the prefix before exactly one final JSON line.",
|
|
3558
|
+
subtask_prompt: "Validate manifest.draft.json and materialize manifest.json after the mechanical checklist (and optional blocking review) passes.",
|
|
3559
|
+
shell: { commands: [manifestValidation], cwd: ".", timeoutMs: 120000 },
|
|
3560
|
+
}, {
|
|
3561
|
+
id: "execute-frontend-cases-map",
|
|
3562
|
+
depends_on: ["materialize-frontend-case-manifest-shell"],
|
|
3563
|
+
role: "verifier",
|
|
3564
|
+
executor: "static",
|
|
3565
|
+
complexity: "LOW",
|
|
3566
|
+
writePolicy: "none",
|
|
3567
|
+
allowedPaths: [],
|
|
3568
|
+
forbiddenPaths: forbidden,
|
|
3569
|
+
outputContract: "Serial aggregate of case execution summaries, evidence paths, tokens, and token-budget or executor blocked/failed cases.",
|
|
3570
|
+
subtask_prompt: "Expand and execute the validated frontend case manifest serially. Child executor failures become case-level failed/blocked evidence so closeout can still run.",
|
|
3571
|
+
static: { resultMarkdown: "Frontend case map expansion barrier." },
|
|
3572
|
+
dynamicExpansion: {
|
|
3573
|
+
type: "map_agent",
|
|
3574
|
+
workflowNodeId: "execute-frontend-cases-map",
|
|
3575
|
+
itemsFrom: "$.nodes['materialize-frontend-case-manifest-shell'].output.cases",
|
|
3576
|
+
itemName: "case",
|
|
3577
|
+
maxItems: config.maxCasesPerBatch,
|
|
3578
|
+
maxExpandedNodes: config.maxCasesPerBatch,
|
|
3579
|
+
childIdPrefix: "execute-frontend-case",
|
|
3580
|
+
workspaceTemplate: "{{case.evidenceDir}}",
|
|
3581
|
+
tolerateChildFailures: true,
|
|
3582
|
+
tokenBudget: {
|
|
3583
|
+
maxTokensPerCase: config.maxTokensPerCase,
|
|
3584
|
+
maxTotalTokens: config.maxTotalTokens,
|
|
3585
|
+
},
|
|
3586
|
+
childTask: {
|
|
3587
|
+
executor: "pi",
|
|
3588
|
+
role: "implementer",
|
|
3589
|
+
skills: ["playwright-cli"],
|
|
3590
|
+
toolProfile: "write",
|
|
3591
|
+
complexity: "MED",
|
|
3592
|
+
writePolicy: "exclusive",
|
|
3593
|
+
allowedPaths: [
|
|
3594
|
+
"testcase/frontend/cases/{{case.caseId}}.md",
|
|
3595
|
+
"testcase/frontend/rag/context.md",
|
|
3596
|
+
"testcase/frontend/rag/coverage-map.md",
|
|
3597
|
+
`${evidenceRoot}/{{case.caseId}}/**`,
|
|
3598
|
+
],
|
|
3599
|
+
forbiddenPaths: forbidden,
|
|
3600
|
+
writeSet: [`${evidenceRoot}/{{case.caseId}}/**`],
|
|
3601
|
+
outputContract: "Compact JSON <=1200 characters with case status, evidence paths, error summary, and tokens.",
|
|
3602
|
+
subtaskPromptTemplate: [
|
|
3603
|
+
"Primary job: EXECUTE case {{case.caseId}} from {{case.casePath}} with skill playwright-cli (fresh Pi session; do not use /new). Prefer playwright-cli over prose review.",
|
|
3604
|
+
"1) Read baseUrl from testcase/frontend/rag/context.md (config.md preferred, else http://localhost:5173). 2) Start browser: playwright-cli open --browser=chrome --headed <resolved-base-url> (default session only; no -s=). 3) Follow the case steps with snapshot before element refs. 4) If env/CLI/baseUrl is unavailable, write blocked evidence and do not open a browser.",
|
|
3605
|
+
"Always write {{case.evidenceDir}}execution.md and {{case.evidenceDir}}case-result.json (caseId, status passed|failed|blocked, evidencePaths; blocked needs blockedReason). Then validate: node -e \"const fs=require('fs');const p='{{case.evidenceDir}}';const r=JSON.parse(fs.readFileSync(p+'case-result.json','utf8'));if(!fs.existsSync(p+'execution.md')||r.caseId!=='{{case.caseId}}'||!['passed','failed','blocked'].includes(r.status)||!Array.isArray(r.evidencePaths)||(r.status==='blocked'&&!(typeof r.blockedReason==='string'&&r.blockedReason.trim())))process.exit(1)\".",
|
|
3606
|
+
"Business failed/blocked is a recorded result, not a node failure. Close browser. Return compact JSON (<=1200 chars): {caseId,status,evidencePaths,errorSummary,tokens}.",
|
|
3607
|
+
].join("\n\n"),
|
|
3608
|
+
},
|
|
3609
|
+
},
|
|
3610
|
+
}, {
|
|
3611
|
+
id: "validate-frontend-case-evidence-shell",
|
|
3612
|
+
depends_on: ["execute-frontend-cases-map"],
|
|
3613
|
+
role: "verifier",
|
|
3614
|
+
executor: "shell",
|
|
3615
|
+
complexity: "LOW",
|
|
3616
|
+
writePolicy: "read-only",
|
|
3617
|
+
allowedPaths: ["testcase/frontend/cases/**", `${evidenceRoot}/**`],
|
|
3618
|
+
forbiddenPaths: forbidden,
|
|
3619
|
+
outputContract: "Deterministic validation that every manifest case has execution.md and valid matching case-result.json; blocked results require blockedReason.",
|
|
3620
|
+
subtask_prompt: "Validate all frontend case evidence before result materialization; fail closed on missing or malformed records.",
|
|
3621
|
+
shell: { commands: [evidenceValidation], cwd: ".", timeoutMs: 120000 },
|
|
3622
|
+
}, {
|
|
3623
|
+
id: "materialize-frontend-test-result-shell",
|
|
3624
|
+
depends_on: ["validate-frontend-case-evidence-shell"],
|
|
3625
|
+
role: "verifier",
|
|
3626
|
+
executor: "shell",
|
|
3627
|
+
complexity: "LOW",
|
|
3628
|
+
writePolicy: "read-only",
|
|
3629
|
+
allowedPaths: ["testcase/frontend/cases/**", `${evidenceRoot}/**`],
|
|
3630
|
+
forbiddenPaths: forbidden,
|
|
3631
|
+
outputContract: "Run-owned hash-bound frontend-test-result-v1 derived only from the manifest and validated case evidence.",
|
|
3632
|
+
subtask_prompt: "Materialize the authoritative frontend-test-result-v1. Do not use Pi prose or retrospective output as input.",
|
|
3633
|
+
shell: {
|
|
3634
|
+
commands: [],
|
|
3635
|
+
jsonArtifactGate: {
|
|
3636
|
+
fromNodeId: "validate-frontend-case-evidence-shell",
|
|
3637
|
+
schemaId: "frontend-test-result-v1",
|
|
3638
|
+
artifactName: "frontend-test-result.json",
|
|
3639
|
+
outputDir: "contracts",
|
|
3640
|
+
},
|
|
3641
|
+
cwd: ".",
|
|
3642
|
+
timeoutMs: 120000,
|
|
3643
|
+
},
|
|
3644
|
+
});
|
|
3645
|
+
if (strictOutcomeGate) {
|
|
3646
|
+
tasks.push({
|
|
3647
|
+
id: "frontend-test-result-outcome-gate-shell",
|
|
3648
|
+
depends_on: ["materialize-frontend-test-result-shell"],
|
|
3649
|
+
role: "verifier",
|
|
3650
|
+
executor: "shell",
|
|
3651
|
+
complexity: "LOW",
|
|
3652
|
+
writePolicy: "read-only",
|
|
3653
|
+
allowedPaths: [],
|
|
3654
|
+
forbiddenPaths: forbidden,
|
|
3655
|
+
outputContract: "Optional quality gate: pass only when frontend-test-result-v1 is outcome=passed and integrationMode=real with 0 failed/blocked and no missing AC. Does not gate retrospective closeout.",
|
|
3656
|
+
subtask_prompt: "Opt-in Delivery/Worker quality gate (frontendTest.strictOutcomeGate=true). Retrospective does not depend on this node.",
|
|
3657
|
+
shell: { commands: [frontendTestOutcomeGate], cwd: ".", timeoutMs: 60000 },
|
|
3658
|
+
});
|
|
3659
|
+
}
|
|
3660
|
+
tasks.push({
|
|
3661
|
+
id: "frontend-test-retrospect-pi",
|
|
3662
|
+
depends_on: ["materialize-frontend-test-result-shell"],
|
|
3663
|
+
role: "closeout",
|
|
3664
|
+
executor: "pi",
|
|
3665
|
+
toolProfile: "write",
|
|
3666
|
+
complexity: "MED",
|
|
3667
|
+
writePolicy: "exclusive",
|
|
3668
|
+
writeSet: ["testcase/frontend/reports/**"],
|
|
3669
|
+
allowedPaths: ["testcase/frontend/**"],
|
|
3670
|
+
forbiddenPaths: forbidden,
|
|
3671
|
+
outputContract: "Write frontend-test retrospective under testcase/frontend/reports/frontend-test-retrospect-<date>.md with coverage, pass/fail/blocked, execution evidence review, risks, findings, and A/B/C/D rating — even when outcome is failed/incomplete. Pipeline acceptance = this report exists (not case 100% pass).",
|
|
3672
|
+
subtask_prompt: "Write the frontend test retrospective under testcase/frontend/reports/ after result materialization (do not wait for outcome=pass). Combine AC→case→browser-evidence review with the closeout report: coverage, passed/failed/blocked (including token-budget-exhausted / executor-auth-unavailable), evidence gaps, browser anomalies, residual risks, and A/B/C/D rating. Passed cases need assertion plus screenshot or equivalent evidence when available; failed/blocked need explicit reasons. Blocked cases never count as passed. Do not replace browser evidence with model conclusions. Do not write docs/**. Pipeline success is report production, not case full green.",
|
|
3673
|
+
});
|
|
3674
|
+
const globalConstraints = [
|
|
3675
|
+
...sources.taskConfig.hardConstraints,
|
|
3676
|
+
...STANDARD_GLOBAL_CONSTRAINTS,
|
|
3677
|
+
"frontend-test-dag generates Markdown cases and browser evidence only; it must not generate pytest or Playwright test source code.",
|
|
3678
|
+
"Each browser case runs serially in a fresh Pi execution boundary. Persist its evidence before starting the next case.",
|
|
3679
|
+
"Use only the declared isolated test environment. Production URLs, real credentials, and unauthorized data are blocked.",
|
|
3680
|
+
"Browser startup for generated cases must be playwright-cli open --browser=chrome --headed <resolved-base-url>; resolve baseUrl from task source config.md when present, otherwise default http://localhost:5173; generated operations stay in the default browser session and must not use unverified named-session flags.",
|
|
3681
|
+
"Token settings are post-case stop thresholds, never a hard provider token cap. Unstarted cases after a threshold are blocked: token-budget-exhausted.",
|
|
3682
|
+
"Pipeline acceptance for frontend-test is the final retrospect report under testcase/frontend/reports/; case pass rate and outcome=passed are quality signals, not the default pipeline success condition.",
|
|
3683
|
+
blockingReview
|
|
3684
|
+
? "frontendTest.reviewMode=blocking: a frontend case review must emit VERDICT: pass before checklist/manifest materialization; request-revision blocks browser execution."
|
|
3685
|
+
: "frontendTest.reviewMode is off|advisory by default: mechanical checklist-shell gates materialize/execute; LLM review is not a hard browser gate.",
|
|
3686
|
+
];
|
|
3363
3687
|
const spec = {
|
|
3364
3688
|
version: 3,
|
|
3365
3689
|
title: `Frontend test DAG: ${sources.taskConfig.title}`,
|
|
@@ -3367,16 +3691,7 @@ function buildFrontendTestHybridDag(sources) {
|
|
|
3367
3691
|
outputLanguage: sources.outputLanguage ?? DEFAULT_DAG_OUTPUT_LANGUAGE,
|
|
3368
3692
|
objective: extractObjective(sources.requirementMarkdown, sources.taskConfig.title),
|
|
3369
3693
|
successCriteria: extractSuccessCriteria(sources.requirementMarkdown, sources.taskId),
|
|
3370
|
-
globalConstraints
|
|
3371
|
-
...sources.taskConfig.hardConstraints,
|
|
3372
|
-
...STANDARD_GLOBAL_CONSTRAINTS,
|
|
3373
|
-
"frontend-test-dag generates Markdown cases and browser evidence only; it must not generate pytest or Playwright test source code.",
|
|
3374
|
-
"Each browser case runs serially in a fresh Pi execution boundary. Persist its evidence before starting the next case.",
|
|
3375
|
-
"Use only the declared isolated test environment. Production URLs, real credentials, and unauthorized data are blocked.",
|
|
3376
|
-
"Browser startup for generated cases must be playwright-cli open --browser=chrome --headed <resolved-base-url>; resolve baseUrl from task source config.md when present, otherwise default http://localhost:5173; generated operations stay in the default browser session and must not use unverified named-session flags.",
|
|
3377
|
-
"A frontend case review must emit VERDICT: pass before manifest materialization; request-revision blocks browser execution.",
|
|
3378
|
-
"Token settings are post-case stop thresholds, never a hard provider token cap. Unstarted cases after a threshold are blocked: token-budget-exhausted.",
|
|
3379
|
-
],
|
|
3694
|
+
globalConstraints,
|
|
3380
3695
|
defaults: {
|
|
3381
3696
|
...HYBRID_DEFAULTS,
|
|
3382
3697
|
skills: [],
|
|
@@ -3393,253 +3708,7 @@ function buildFrontendTestHybridDag(sources) {
|
|
|
3393
3708
|
closeout: ["verification-before-completion"],
|
|
3394
3709
|
},
|
|
3395
3710
|
executorModels: sources.executorModelMatrix ?? DEFAULT_DAG_EXECUTOR_MODELS,
|
|
3396
|
-
tasks
|
|
3397
|
-
{
|
|
3398
|
-
id: "retrieve-frontend-test-context-pi",
|
|
3399
|
-
depends_on: [],
|
|
3400
|
-
role: "planner",
|
|
3401
|
-
executor: "pi",
|
|
3402
|
-
toolProfile: "write",
|
|
3403
|
-
complexity: "HIGH",
|
|
3404
|
-
writePolicy: "exclusive",
|
|
3405
|
-
writeSet: ragWriteSet,
|
|
3406
|
-
allowedPaths: [...commonReadOnlyPaths(sources), ...ragWriteSet],
|
|
3407
|
-
forbiddenPaths: forbidden,
|
|
3408
|
-
outputContract: "Write testcase/frontend/rag/context.md and coverage-map.md with traceable UI/API/test-environment facts.",
|
|
3409
|
-
subtask_prompt: [
|
|
3410
|
-
"Build the frontend test RAG package.",
|
|
3411
|
-
"Read task source, relevant routes/components/API or Mock facts, existing tests, and execution contract. Write only testcase/frontend/rag/context.md and coverage-map.md.",
|
|
3412
|
-
"Record AC IDs, source paths, routes, states, roles, fixture/data prerequisites, API mapping status, risks, and isolated execution contract. Do not guess unavailable facts.",
|
|
3413
|
-
"Base URL resolution (required): (1) Prefer an absolute http(s) frontend URL from task source config.md (source/references/**/config.md or any attached config.md), including keys baseUrl/base_url/frontendBaseUrl/FRONTEND_BASE_URL/url or labeled frontend base URL text. (2) If config.md has no usable absolute URL, default to http://localhost:5173. (3) Never use production hosts. (4) Write both a human-readable base URL line and machine-readable lines `baseUrl: <url>` and `baseUrlSource: config.md|<path>` or `baseUrlSource: default-localhost-5173`. (5) Include the exact browser start prefix with the resolved URL: playwright-cli open --browser=chrome --headed <resolved-base-url>.",
|
|
3414
|
-
buildSourceContextBlock(sources),
|
|
3415
|
-
].join("\n\n"),
|
|
3416
|
-
},
|
|
3417
|
-
{
|
|
3418
|
-
id: "materialize-frontend-test-execution-shell",
|
|
3419
|
-
depends_on: ["retrieve-frontend-test-context-pi"],
|
|
3420
|
-
role: "verifier",
|
|
3421
|
-
executor: "shell",
|
|
3422
|
-
complexity: "LOW",
|
|
3423
|
-
writePolicy: "read-only",
|
|
3424
|
-
allowedPaths: [...ragWriteSet],
|
|
3425
|
-
forbiddenPaths: forbidden,
|
|
3426
|
-
outputContract: "Fail-closed preflight: absolute non-production baseUrl required; fixture/reset not hard-gated.",
|
|
3427
|
-
subtask_prompt: "Hard-validate only an absolute non-production baseUrl in RAG context (from config.md or default http://localhost:5173). Fixture/reset and other isolation details are soft guidance for later nodes, not preflight failures.",
|
|
3428
|
-
shell: { commands: [["node -e", JSON.stringify("const fs=require('fs'); const p='testcase/frontend/rag/context.md'; if(!fs.existsSync(p))throw new Error('missing '+p); const s=fs.readFileSync(p,'utf8'); const patterns=[ /baseUrl\\s*[:=]\\s*(https?:\\/\\/\\S+)/i, /base[- ]url\\s*[:=]\\s*(https?:\\/\\/\\S+)/i, /playwright-cli open --browser=chrome --headed\\s+(https?:\\/\\/\\S+)/i, /(https?:\\/\\/(?:localhost|127\\.0\\.0\\.1)[^\\s)\\]},\"']*)/i ]; let baseUrl=null; for(const re of patterns){const m=s.match(re); if(m){baseUrl=m[1]; break;}} if(!baseUrl)throw new Error('frontend-test preflight missing absolute baseUrl (prefer config.md; default http://localhost:5173)'); baseUrl=baseUrl.replace(/[)\\]},.\"']+$/,''); if(!/^https?:\\/\\//i.test(baseUrl))throw new Error('baseUrl must be absolute http(s): '+baseUrl); if(/(?:^|\\/\\/)(?:www\\.)?[^\\s/]*(?:prod|production)/i.test(baseUrl))throw new Error('production URL forbidden: '+baseUrl); console.log('frontend-test-execution-v1 validated baseUrl='+baseUrl);")].join(" ")], cwd: ".", timeoutMs: 60000 },
|
|
3429
|
-
},
|
|
3430
|
-
{
|
|
3431
|
-
id: "generate-frontend-functional-cases-pi",
|
|
3432
|
-
depends_on: ["materialize-frontend-test-execution-shell"],
|
|
3433
|
-
role: "implementer",
|
|
3434
|
-
executor: "pi",
|
|
3435
|
-
toolProfile: "write",
|
|
3436
|
-
complexity: "HIGH",
|
|
3437
|
-
writePolicy: "exclusive",
|
|
3438
|
-
writeSet: casesWriteSet,
|
|
3439
|
-
allowedPaths: [...ragWriteSet, ...casesWriteSet],
|
|
3440
|
-
forbiddenPaths: forbidden,
|
|
3441
|
-
outputContract: "Write executable Markdown frontend cases, index.md, and manifest.draft.json schemaVersion 1; no test source code.",
|
|
3442
|
-
subtask_prompt: [
|
|
3443
|
-
"Use skill playwright-cli-case-generator.",
|
|
3444
|
-
"Read only testcase/frontend/rag/context.md, testcase/frontend/rag/coverage-map.md, and existing testcase/frontend/cases/. Write only testcase/frontend/cases/**.",
|
|
3445
|
-
"Generate Markdown cases, index.md and manifest.draft.json (schemaVersion 1; cases[] with caseId, casePath, dimension, acIds, evidenceDir). IDs use FE-<FEATURE>-<NNN>-<dimension>; dimensions core|boundary|flow|backend.",
|
|
3446
|
-
"Never infer API fields, constraints, SLA, credentials, or unrecorded test data. Do not create pytest or Playwright source. Copy the resolved absolute baseUrl from context.md (baseUrl field; resolved from config.md or default http://localhost:5173). Every browser start command must be: playwright-cli open --browser=chrome --headed <resolved-base-url-from-context.md> with that concrete URL — never leave a <base-url> placeholder. Use the same default browser session for every subsequent command; never write -s=<case-id> or assume named-session binding.",
|
|
3447
|
-
"Each case must be independently reproducible: for every executable sub-scenario state fixture/reset, UI reset, a fresh snapshot before references are used, exact evidence write point, preconditions/data cleanup, UI assertions, and evidence paths under testcase/frontend/evidence/<case-id>/. If the isolated environment is unavailable, require writing blocked evidence before any browser command.",
|
|
3448
|
-
].join("\n\n"),
|
|
3449
|
-
},
|
|
3450
|
-
{
|
|
3451
|
-
id: "review-frontend-cases-pi",
|
|
3452
|
-
depends_on: ["generate-frontend-functional-cases-pi"],
|
|
3453
|
-
role: "reviewer",
|
|
3454
|
-
executor: "pi",
|
|
3455
|
-
complexity: "HIGH",
|
|
3456
|
-
writePolicy: "read-only",
|
|
3457
|
-
allowedPaths: [...ragWriteSet, ...casesWriteSet],
|
|
3458
|
-
forbiddenPaths: forbidden,
|
|
3459
|
-
outputContract: "First line VERDICT: pass or VERDICT: request-revision, followed by AC-to-case coverage and execution risk findings; no writes. request-revision blocks manifest materialization.",
|
|
3460
|
-
subtask_prompt: "Review only the RAG package, frontend Markdown cases, and manifest.draft.json. Verify traceability, independent execution, safe data/environment handling, manifest correctness, session consistency, fixture/UI reset and fresh snapshot steps, and evidence requirements. Any Important or Critical finding requires VERDICT: request-revision. Browser execution is blocked unless this review passes.",
|
|
3461
|
-
},
|
|
3462
|
-
{
|
|
3463
|
-
id: "revise-frontend-cases-pi",
|
|
3464
|
-
depends_on: ["review-frontend-cases-pi"],
|
|
3465
|
-
runIf: "$.nodes['review-frontend-cases-pi'].firstVerdictLine == 'VERDICT: request-revision'",
|
|
3466
|
-
role: "implementer",
|
|
3467
|
-
executor: "pi",
|
|
3468
|
-
toolProfile: "write",
|
|
3469
|
-
complexity: "HIGH",
|
|
3470
|
-
writePolicy: "exclusive",
|
|
3471
|
-
writeSet: casesWriteSet,
|
|
3472
|
-
allowedPaths: [...ragWriteSet, ...casesWriteSet],
|
|
3473
|
-
forbiddenPaths: forbidden,
|
|
3474
|
-
outputContract: "Apply the one permitted frontend case revision under testcase/frontend/cases/** only; no browser execution or evidence writes.",
|
|
3475
|
-
subtask_prompt: "This is the only permitted case revision. Read the first review findings and the RAG package. Revise only testcase/frontend/cases/**, preserve traceable AC mappings, and do not execute a browser or write evidence. HARD: Never delete case files; only edit in place or add missing cases. Preserve the full planned suite, index.md, and manifest.draft.json.",
|
|
3476
|
-
},
|
|
3477
|
-
{
|
|
3478
|
-
id: "review-frontend-cases-final-pi",
|
|
3479
|
-
depends_on: ["revise-frontend-cases-pi"],
|
|
3480
|
-
runIf: "$.nodes['review-frontend-cases-pi'].firstVerdictLine == 'VERDICT: request-revision'",
|
|
3481
|
-
role: "reviewer",
|
|
3482
|
-
executor: "pi",
|
|
3483
|
-
complexity: "HIGH",
|
|
3484
|
-
writePolicy: "read-only",
|
|
3485
|
-
allowedPaths: [...ragWriteSet, ...casesWriteSet],
|
|
3486
|
-
forbiddenPaths: forbidden,
|
|
3487
|
-
outputContract: "First line VERDICT: pass or VERDICT: request-revision after the single allowed case revision; no writes.",
|
|
3488
|
-
subtask_prompt: "Perform the final frontend case review after the sole permitted revision. Apply the same traceability, isolation, manifest, reset, session, snapshot, and evidence checks. First verdict line must be exact; any Important or Critical finding requires request-revision. Do not write files.",
|
|
3489
|
-
},
|
|
3490
|
-
{
|
|
3491
|
-
id: "final-frontend-case-review-gate-shell",
|
|
3492
|
-
depends_on: ["review-frontend-cases-pi", "review-frontend-cases-final-pi"],
|
|
3493
|
-
dependsPolicy: "all-or-condition-skip",
|
|
3494
|
-
role: "verifier",
|
|
3495
|
-
executor: "shell",
|
|
3496
|
-
complexity: "LOW",
|
|
3497
|
-
writePolicy: "read-only",
|
|
3498
|
-
allowedPaths: [...ragWriteSet, ...casesWriteSet],
|
|
3499
|
-
forbiddenPaths: forbidden,
|
|
3500
|
-
outputContract: "Pass-only effective frontend case review gate; final review takes precedence when the revision branch ran.",
|
|
3501
|
-
subtask_prompt: "Authorize manifest materialization only after the effective frontend case review passes.",
|
|
3502
|
-
shell: {
|
|
3503
|
-
commands: [],
|
|
3504
|
-
verdictGate: {
|
|
3505
|
-
fromNodeId: "review-frontend-cases-final-pi",
|
|
3506
|
-
fallbackFromNodeIds: ["review-frontend-cases-pi"],
|
|
3507
|
-
accept: ["VERDICT: pass"],
|
|
3508
|
-
label: "effective frontend case review",
|
|
3509
|
-
lineMode: "first-verdict-line",
|
|
3510
|
-
},
|
|
3511
|
-
cwd: ".",
|
|
3512
|
-
timeoutMs: 60000,
|
|
3513
|
-
},
|
|
3514
|
-
},
|
|
3515
|
-
{
|
|
3516
|
-
id: "materialize-frontend-case-manifest-shell",
|
|
3517
|
-
depends_on: ["final-frontend-case-review-gate-shell"],
|
|
3518
|
-
role: "verifier",
|
|
3519
|
-
executor: "shell",
|
|
3520
|
-
complexity: "LOW",
|
|
3521
|
-
writePolicy: "exclusive",
|
|
3522
|
-
writeSet: casesWriteSet,
|
|
3523
|
-
allowedPaths: casesWriteSet,
|
|
3524
|
-
forbiddenPaths: forbidden,
|
|
3525
|
-
outputContract: "Validated frontend manifest payload { cases: [...] }; after the review gate, atomically materialize testcase/frontend/cases/manifest.json from manifest.draft.json; shell output may echo only the prefix before exactly one final JSON line.",
|
|
3526
|
-
subtask_prompt: "Validate manifest.draft.json and materialize manifest.json only after the effective frontend case review has passed.",
|
|
3527
|
-
shell: { commands: [manifestValidation], cwd: ".", timeoutMs: 120000 },
|
|
3528
|
-
},
|
|
3529
|
-
{
|
|
3530
|
-
id: "execute-frontend-cases-map",
|
|
3531
|
-
depends_on: ["materialize-frontend-case-manifest-shell"],
|
|
3532
|
-
role: "verifier",
|
|
3533
|
-
executor: "static",
|
|
3534
|
-
complexity: "LOW",
|
|
3535
|
-
writePolicy: "none",
|
|
3536
|
-
allowedPaths: [],
|
|
3537
|
-
forbiddenPaths: forbidden,
|
|
3538
|
-
outputContract: "Serial aggregate of case execution summaries, evidence paths, tokens, and token-budget blocked cases.",
|
|
3539
|
-
subtask_prompt: "Expand and execute the validated frontend case manifest serially.",
|
|
3540
|
-
static: { resultMarkdown: "Frontend case map expansion barrier." },
|
|
3541
|
-
dynamicExpansion: {
|
|
3542
|
-
type: "map_agent",
|
|
3543
|
-
workflowNodeId: "execute-frontend-cases-map",
|
|
3544
|
-
itemsFrom: "$.nodes['materialize-frontend-case-manifest-shell'].output.cases",
|
|
3545
|
-
itemName: "case",
|
|
3546
|
-
maxItems: config.maxCasesPerBatch,
|
|
3547
|
-
maxExpandedNodes: config.maxCasesPerBatch,
|
|
3548
|
-
childIdPrefix: "execute-frontend-case",
|
|
3549
|
-
workspaceTemplate: "{{case.evidenceDir}}",
|
|
3550
|
-
tokenBudget: {
|
|
3551
|
-
maxTokensPerCase: config.maxTokensPerCase,
|
|
3552
|
-
maxTotalTokens: config.maxTotalTokens,
|
|
3553
|
-
},
|
|
3554
|
-
childTask: {
|
|
3555
|
-
executor: "pi",
|
|
3556
|
-
role: "implementer",
|
|
3557
|
-
skills: ["playwright-cli"],
|
|
3558
|
-
toolProfile: "write",
|
|
3559
|
-
complexity: "MED",
|
|
3560
|
-
writePolicy: "exclusive",
|
|
3561
|
-
allowedPaths: [
|
|
3562
|
-
"testcase/frontend/cases/{{case.caseId}}.md",
|
|
3563
|
-
"testcase/frontend/rag/context.md",
|
|
3564
|
-
"testcase/frontend/rag/coverage-map.md",
|
|
3565
|
-
`${evidenceRoot}/{{case.caseId}}/**`,
|
|
3566
|
-
],
|
|
3567
|
-
forbiddenPaths: forbidden,
|
|
3568
|
-
writeSet: [`${evidenceRoot}/{{case.caseId}}/**`],
|
|
3569
|
-
outputContract: "Compact JSON <=1200 characters with case status, evidence paths, error summary, and tokens.",
|
|
3570
|
-
subtaskPromptTemplate: [
|
|
3571
|
-
"Primary job: EXECUTE case {{case.caseId}} from {{case.casePath}} with skill playwright-cli (fresh Pi session; do not use /new). Prefer playwright-cli over prose review.",
|
|
3572
|
-
"1) Read baseUrl from testcase/frontend/rag/context.md (config.md preferred, else http://localhost:5173). 2) Start browser: playwright-cli open --browser=chrome --headed <resolved-base-url> (default session only; no -s=). 3) Follow the case steps with snapshot before element refs. 4) If env/CLI/baseUrl is unavailable, write blocked evidence and do not open a browser.",
|
|
3573
|
-
"Always write {{case.evidenceDir}}execution.md and {{case.evidenceDir}}case-result.json (caseId, status passed|failed|blocked, evidencePaths; blocked needs blockedReason). Then validate: node -e \"const fs=require('fs');const p='{{case.evidenceDir}}';const r=JSON.parse(fs.readFileSync(p+'case-result.json','utf8'));if(!fs.existsSync(p+'execution.md')||r.caseId!=='{{case.caseId}}'||!['passed','failed','blocked'].includes(r.status)||!Array.isArray(r.evidencePaths)||(r.status==='blocked'&&!(typeof r.blockedReason==='string'&&r.blockedReason.trim())))process.exit(1)\".",
|
|
3574
|
-
"Business failed/blocked is a recorded result, not a node failure. Close browser. Return compact JSON (<=1200 chars): {caseId,status,evidencePaths,errorSummary,tokens}.",
|
|
3575
|
-
].join("\n\n"),
|
|
3576
|
-
},
|
|
3577
|
-
},
|
|
3578
|
-
},
|
|
3579
|
-
{
|
|
3580
|
-
id: "validate-frontend-case-evidence-shell",
|
|
3581
|
-
depends_on: ["execute-frontend-cases-map"],
|
|
3582
|
-
role: "verifier",
|
|
3583
|
-
executor: "shell",
|
|
3584
|
-
complexity: "LOW",
|
|
3585
|
-
writePolicy: "read-only",
|
|
3586
|
-
allowedPaths: ["testcase/frontend/cases/**", `${evidenceRoot}/**`],
|
|
3587
|
-
forbiddenPaths: forbidden,
|
|
3588
|
-
outputContract: "Deterministic validation that every manifest case has execution.md and valid matching case-result.json; blocked results require blockedReason.",
|
|
3589
|
-
subtask_prompt: "Validate all frontend case evidence before evidence review; fail closed on missing or malformed records.",
|
|
3590
|
-
shell: { commands: [evidenceValidation], cwd: ".", timeoutMs: 120000 },
|
|
3591
|
-
},
|
|
3592
|
-
{
|
|
3593
|
-
id: "materialize-frontend-test-result-shell",
|
|
3594
|
-
depends_on: ["validate-frontend-case-evidence-shell"],
|
|
3595
|
-
role: "verifier",
|
|
3596
|
-
executor: "shell",
|
|
3597
|
-
complexity: "LOW",
|
|
3598
|
-
writePolicy: "read-only",
|
|
3599
|
-
allowedPaths: ["testcase/frontend/cases/**", `${evidenceRoot}/**`],
|
|
3600
|
-
forbiddenPaths: forbidden,
|
|
3601
|
-
outputContract: "Run-owned hash-bound frontend-test-result-v1 derived only from the manifest and validated case evidence.",
|
|
3602
|
-
subtask_prompt: "Materialize the authoritative frontend-test-result-v1. Do not use Pi prose or retrospective output as input.",
|
|
3603
|
-
shell: {
|
|
3604
|
-
commands: [],
|
|
3605
|
-
jsonArtifactGate: {
|
|
3606
|
-
fromNodeId: "validate-frontend-case-evidence-shell",
|
|
3607
|
-
schemaId: "frontend-test-result-v1",
|
|
3608
|
-
artifactName: "frontend-test-result.json",
|
|
3609
|
-
outputDir: "contracts",
|
|
3610
|
-
},
|
|
3611
|
-
cwd: ".",
|
|
3612
|
-
timeoutMs: 120000,
|
|
3613
|
-
},
|
|
3614
|
-
},
|
|
3615
|
-
{
|
|
3616
|
-
id: "frontend-test-result-outcome-gate-shell",
|
|
3617
|
-
depends_on: ["materialize-frontend-test-result-shell"],
|
|
3618
|
-
role: "verifier",
|
|
3619
|
-
executor: "shell",
|
|
3620
|
-
complexity: "LOW",
|
|
3621
|
-
writePolicy: "read-only",
|
|
3622
|
-
allowedPaths: [],
|
|
3623
|
-
forbiddenPaths: forbidden,
|
|
3624
|
-
outputContract: "Pass only when the run-owned frontend-test-result-v1 records outcome=passed and integrationMode=real. Does not gate retrospective closeout.",
|
|
3625
|
-
subtask_prompt: "Delivery/Worker gate for authoritative frontend-test result. Retrospective does not depend on this node so failed runs can still write reports.",
|
|
3626
|
-
shell: { commands: [frontendTestOutcomeGate], cwd: ".", timeoutMs: 60000 },
|
|
3627
|
-
},
|
|
3628
|
-
{
|
|
3629
|
-
id: "frontend-test-retrospect-pi",
|
|
3630
|
-
depends_on: ["materialize-frontend-test-result-shell"],
|
|
3631
|
-
role: "closeout",
|
|
3632
|
-
executor: "pi",
|
|
3633
|
-
toolProfile: "write",
|
|
3634
|
-
complexity: "MED",
|
|
3635
|
-
writePolicy: "exclusive",
|
|
3636
|
-
writeSet: ["testcase/frontend/reports/**"],
|
|
3637
|
-
allowedPaths: ["testcase/frontend/**"],
|
|
3638
|
-
forbiddenPaths: forbidden,
|
|
3639
|
-
outputContract: "Write frontend-test retrospective under testcase/frontend/reports/frontend-test-retrospect-<date>.md with coverage, pass/fail/blocked, execution evidence review, risks, findings, and A/B/C/D rating — even when outcome is failed/incomplete.",
|
|
3640
|
-
subtask_prompt: "Write the frontend test retrospective under testcase/frontend/reports/ after result materialization (do not wait for outcome=pass). Combine AC→case→browser-evidence review with the closeout report: coverage, passed/failed/blocked (including token-budget-exhausted), evidence gaps, browser anomalies, residual risks, and A/B/C/D rating. Passed cases need assertion plus screenshot or equivalent evidence when available; failed/blocked need explicit reasons. Blocked cases never count as passed. Do not replace browser evidence with model conclusions. Do not write docs/**.",
|
|
3641
|
-
},
|
|
3642
|
-
],
|
|
3711
|
+
tasks,
|
|
3643
3712
|
};
|
|
3644
3713
|
applyDefaultReadOnlyRetryPolicy(spec);
|
|
3645
3714
|
parseDagSpec(spec);
|
|
@@ -201,6 +201,13 @@ export const dagDynamicExpansionSchema = z.object({
|
|
|
201
201
|
maxTotalTokens: z.number().int().positive().optional(),
|
|
202
202
|
})
|
|
203
203
|
.optional(),
|
|
204
|
+
/**
|
|
205
|
+
* When true, map child ERROR/auth/timeout is recorded as case-level
|
|
206
|
+
* failed/blocked evidence and the map barrier still succeeds (frontend-test).
|
|
207
|
+
* Default false: any non-budget child failure fails the map aggregate
|
|
208
|
+
* (sharded migration and other map_agent workflows).
|
|
209
|
+
*/
|
|
210
|
+
tolerateChildFailures: z.boolean().optional(),
|
|
204
211
|
});
|
|
205
212
|
export const dagDynamicReductionSchema = z.object({
|
|
206
213
|
type: z.literal("verified_findings_report"),
|
|
@@ -45,6 +45,10 @@ git config commit.gpgsign false
|
|
|
45
45
|
|
|
46
46
|
该设置只用于本地开发和测试稳定性,不应写入仓库代码、模板或发布包配置。
|
|
47
47
|
|
|
48
|
+
## Windows clean Shell 与用户级 Python
|
|
49
|
+
|
|
50
|
+
DAG Shell/verification 默认使用安全的 clean environment,不继承完整宿主环境。在 Windows 上,clean baseline 会保留 `APPDATA`,使 Python/pip 能按标准规则定位 `%APPDATA%/Python/PythonXY/site-packages` 中的用户级安装,同时 token、credential 与其他未授权变量仍被过滤。不要在仓库中硬编码用户目录、盘符、Python 版本或 `site-packages` 路径;业务环境变量仍应通过任务/DAG 的显式 `envAllowlist` 授权。
|
|
51
|
+
|
|
48
52
|
## 维护规则
|
|
49
53
|
|
|
50
54
|
- 仅把可复现、会反复影响开发或验证的环境问题写入本文。
|
|
@@ -90,7 +90,7 @@
|
|
|
90
90
|
"artifacts/**"
|
|
91
91
|
],
|
|
92
92
|
"outputContract": "Pure JSON envelope {analysis: Backend Test Analysis v2, execution: Backend Test Execution Contract v1}; no prose or writes.",
|
|
93
|
-
"subtask_prompt": "Read the task source materials and return exactly one JSON object matching Backend Test Analysis v2.\n\nDo not wrap it in explanatory prose. A single fenced json block is tolerated, but pure JSON is preferred.\n\nCopy the sourceBinding object exactly from the JSON block below; do not infer, add, remove, or reclassify source paths.\n\nOnly kind=reference sources belong in referencePaths; kind=constraint sources MUST NOT be included in referencePaths.\n\n## Exact Backend Test Analysis sourceBinding JSON\n\n{\n \"taskId\": \"backend-test-template\",\n \"requirementPath\": \"source/需求.md\",\n \"requirementSha256\": \"e33ab9d1d3d6a785b8f429d8581d015a299adc093b0c9a7c0f9057f968a71837\",\n \"referencePaths\": [],\n \"requirementIds\": [\n \"AC-001\"\n ]\n}\n\nFor every endpoint, explicitly set responseBody.kind=array|object|scalar|empty|unknown and ordering=specified|unspecified|not-applicable. Add itemSchemaRef for arrays when documented.\n\nFor response fields, use comparison=exact|parseable-only|semantic when the source defines assertion semantics; date-time fields whose precision is unspecified should use parseable-only, not string equality.\n\nEndpoint sourceRefs and field sourceRefs must cite only requirement/reference evidence actually read. Empty sourceRefs are allowed only when normalizing legacy v1 input; newly generated v2 should cite evidence.\n\nUse empty arrays for categories not documented. Never include credentials, tokens, private keys, or secret values.\n\nRequired top-level keys: schemaVersion=2, sourceBinding, acceptanceCriteria, endpoints, dataModels, businessRules, stateTransitions, boundaryConstraints, externalDependencies, risks, evidenceGaps.\n\nRead-only: do not modify code, docs, artifacts, or repository files.\n\n## Task source: 需求.md\n\n# Backend test\n- AC-001 proof\n\n## Task config summary\n\n- taskId: backend-test-template\n\n- flow: auto\n\n- complexity: medium\n\n- contextProfile: full\n\n- allowedPaths: testcase/**, docs/test-reports/**\n\n- forbiddenPaths: (none)\n\n- Pi DAG nodes are read-only unless toolProfile=\"write\" is explicitly selected for a bounded writer node.\n\n- Agent DAG read-only nodes must not write root artifacts/**; root artifacts/ is not a per-node scratchpad.\n\n- source/references/* are immutable user/source facts; source/需求.md is the derived execution contract.\n\nAlso perform the read-only environment discovery described by Backend Test Execution Contract v1. Return exactly one JSON envelope with top-level keys analysis and execution; analysis must satisfy v2 and execution must satisfy v1.",
|
|
93
|
+
"subtask_prompt": "Read the task source materials and return exactly one JSON object matching Backend Test Analysis v2.\n\nDo not wrap it in explanatory prose. A single fenced json block is tolerated, but pure JSON is preferred.\n\nCopy the sourceBinding object exactly from the JSON block below; do not infer, add, remove, or reclassify source paths.\n\nOnly kind=reference sources belong in referencePaths; kind=constraint sources MUST NOT be included in referencePaths.\n\n## Exact Backend Test Analysis sourceBinding JSON\n\n{\n \"taskId\": \"backend-test-template\",\n \"requirementPath\": \"source/需求.md\",\n \"requirementSha256\": \"e33ab9d1d3d6a785b8f429d8581d015a299adc093b0c9a7c0f9057f968a71837\",\n \"referencePaths\": [],\n \"requirementIds\": [\n \"AC-001\"\n ]\n}\n\nFor every endpoint, explicitly set responseBody.kind=array|object|scalar|empty|unknown and ordering=specified|unspecified|not-applicable. Add itemSchemaRef for arrays when documented.\n\nFor response fields, use comparison=exact|parseable-only|semantic when the source defines assertion semantics; date-time fields whose precision is unspecified should use parseable-only, not string equality.\n\nEndpoint sourceRefs and field sourceRefs must cite only requirement/reference evidence actually read. Empty sourceRefs are allowed only when normalizing legacy v1 input; newly generated v2 should cite evidence.\n\nFor externalDependencies and risks, emit canonical items with exactly description plus optional name and sourceRef. For a dependency target, put the target value in name. Do not emit type, target, kind, required, severity, mitigation, level, impact, sourceRefs, or custom keys in newly generated v2 output.\n\nUse empty arrays for categories not documented. Never include credentials, tokens, private keys, or secret values.\n\nRequired top-level keys: schemaVersion=2, sourceBinding, acceptanceCriteria, endpoints, dataModels, businessRules, stateTransitions, boundaryConstraints, externalDependencies, risks, evidenceGaps.\n\nRead-only: do not modify code, docs, artifacts, or repository files.\n\n## Task source: 需求.md\n\n# Backend test\n- AC-001 proof\n\n## Task config summary\n\n- taskId: backend-test-template\n\n- flow: auto\n\n- complexity: medium\n\n- contextProfile: full\n\n- allowedPaths: testcase/**, docs/test-reports/**\n\n- forbiddenPaths: (none)\n\n- Pi DAG nodes are read-only unless toolProfile=\"write\" is explicitly selected for a bounded writer node.\n\n- Agent DAG read-only nodes must not write root artifacts/**; root artifacts/ is not a per-node scratchpad.\n\n- source/references/* are immutable user/source facts; source/需求.md is the derived execution contract.\n\nAlso perform the read-only environment discovery described by Backend Test Execution Contract v1. Return exactly one JSON envelope with top-level keys analysis and execution; analysis must satisfy v2 and execution must satisfy v1.",
|
|
94
94
|
"retryPolicy": {
|
|
95
95
|
"maxAttempts": 3,
|
|
96
96
|
"backoff": "exponential",
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
# Frontend-test case blocking checklist
|
|
2
|
+
|
|
3
|
+
Shared mechanical rules for `frontend-case-checklist-shell` and generate prompts.
|
|
4
|
+
LLM review (when `frontendTest.reviewMode=blocking`) must not invent blocking rules outside this list.
|
|
5
|
+
|
|
6
|
+
## Blocking (fail closed)
|
|
7
|
+
|
|
8
|
+
| ruleId | Rule |
|
|
9
|
+
|---|---|
|
|
10
|
+
| `open-prefix` | Each case body includes `playwright-cli open --browser=chrome --headed <absolute-http(s)-url>` |
|
|
11
|
+
| `production-url` | Open URL must not look like a production host |
|
|
12
|
+
| `ac-mapping` | Manifest entry has non-empty `acIds` |
|
|
13
|
+
| `case-file-missing` | `casePath` exists |
|
|
14
|
+
| `no-test-source` | Case text must not introduce pytest / Playwright test source (`pytest`, `playwright.test`, `@playwright/test`) |
|
|
15
|
+
|
|
16
|
+
## Non-blocking (notes only)
|
|
17
|
+
|
|
18
|
+
- Preferred extra evidence filenames not required by generate
|
|
19
|
+
- Style / wording preferences
|
|
20
|
+
- Additional network envelope proofs beyond capability matrix
|
|
21
|
+
|
|
22
|
+
## Pipeline vs quality
|
|
23
|
+
|
|
24
|
+
- **Pipeline acceptance**: final `testcase/frontend/reports/frontend-test-retrospect-*.md` exists after result materialize
|
|
25
|
+
- **Quality**: `frontend-test-result-v1.outcome=passed` with 0 blocked/failed (opt-in via `frontendTest.strictOutcomeGate`)
|
|
@@ -7,14 +7,15 @@
|
|
|
7
7
|
"agentRuntime": "pi-only",
|
|
8
8
|
"repairWriterProtocol": "explicit-node-v1"
|
|
9
9
|
},
|
|
10
|
-
"objective": "Build a frontend test RAG package, generate Markdown cases, execute each case serially through playwright-cli, and
|
|
10
|
+
"objective": "Build a frontend test RAG package, generate Markdown cases, mechanically checklist them (LLM review optional), execute each case serially through playwright-cli, materialize frontend-test-result-v1, and write retrospect report (pipeline acceptance).",
|
|
11
11
|
"globalConstraints": [
|
|
12
12
|
"Do not generate pytest or Playwright source code.",
|
|
13
13
|
"Only use declared isolated test environments; production URLs and real credentials are blocked.",
|
|
14
14
|
"Every generated browser start command is playwright-cli open --browser=chrome --headed <resolved-base-url> (from task source config.md when present, else http://localhost:5173); subsequent commands stay in that default session and must not use unverified named-session flags.",
|
|
15
|
-
"A frontend case review must emit VERDICT: pass before manifest materialization; request-revision blocks browser execution.",
|
|
16
15
|
"Case children execute serially. Persist each case result, logs and browser evidence before the next child starts.",
|
|
17
|
-
"A token threshold is a post-case stop check, not a model hard token cap; unstarted cases must be recorded as blocked: token-budget-exhausted."
|
|
16
|
+
"A token threshold is a post-case stop check, not a model hard token cap; unstarted cases must be recorded as blocked: token-budget-exhausted.",
|
|
17
|
+
"Default pipeline acceptance is the final retrospect report under testcase/frontend/reports/; case full green is optional quality (frontendTest.strictOutcomeGate).",
|
|
18
|
+
"Default frontendTest.reviewMode=off uses mechanical checklist-shell before materialize; set reviewMode=blocking for legacy dual LLM review gate."
|
|
18
19
|
],
|
|
19
20
|
"tasks": [
|
|
20
21
|
{
|