@tea-agent/loop-agent 0.18.0 → 0.18.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md
CHANGED
|
@@ -2,86 +2,21 @@
|
|
|
2
2
|
|
|
3
3
|
## [Unreleased]
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
## [0.18.1] - 2026-07-22
|
|
6
6
|
|
|
7
|
-
###
|
|
8
|
-
|
|
9
|
-
- `backend-test` Markdown 用例生成与独立审查现在默认采用简体中文:用例标题、策略、前置条件、测试数据、步骤、预期结果、自动化说明、表头和证据缺口优先中文表达;Case/AC/REQ/BR ID、固定章节标题、HTTP 方法、路径、字段名、枚举值、命令和代码符号继续保持精确机器契约,不做无意义双语翻译。
|
|
10
|
-
- `backend-test` traceability 现在识别带 `-> None` 等返回类型 annotation 以及可选 `async` 的 pytest 函数,不再把合法 typed test function 全部误报为缺失映射。
|
|
11
|
-
- `backend-test` Markdown gate 只要求 case-level AC;pytest 执行、MD↔pytest 追溯、JUnit/HTML/stdout/stderr 和失败分类等下游证据 AC 交给 traceability、execute 与 report 节点,不再被错误要求写入单条 Markdown case。
|
|
12
|
-
- `backend-test` Markdown gate 的 `Steps` 仍要求编号列表;`Expected Results` 现在接受编号列表或 Markdown bullet 列表,只要保持结构化可断言,不再误拒绝模型生成的合法 bullet 断言;普通散文仍 fail-closed。
|
|
13
|
-
- `backend-test` 的 pre-run Spine 审计不再要求尚未执行的 `artifacts/requirement-coverage.json`:未覆盖需求在该 taskKind 下记为 warning,并交给运行内 Markdown validation 与 traceability gate 确定性阻断;普通实现任务仍保持 error/fail-closed。
|
|
14
|
-
- `backend-test` Markdown 用例门禁现在从 DAG `sourceBinding` 精确识别 `Source References`,支持反引号以及中英文括号/标点,不再将说明文字吞入路径;未绑定引用、路径前缀伪装和绑定文件缺失仍然 fail-closed。
|
|
15
|
-
- `backend-test` Markdown writer/reviewer 的 reference index 现在同时提供 task-relative 引用路径与 repo-relative `readPath`,Pi 会直接只读 `.harness/tasks/<taskId>/source/**` 的绑定文件,不再把 `source/references/**` 误读为仓库根路径,也不会在读取失败后回退到 `docs/**` 替代文件。
|
|
16
|
-
- `backend-test` Markdown 用例门禁会通过 DAG `sourceBinding` 将 `source/需求.md`、`source/执行约束.md` 和 `source/references/**` 解析到对应任务目录,不再错误地相对仓库根查找,也不会接受未绑定的同名仓库文件。
|
|
17
|
-
- `backend-test` traceability 门禁现在优先用 Markdown 中已知 Case ID 匹配 `test_BE_<MODULE>_<NNN>_` 函数前缀,不再把函数描述中的 `100`、`400`、`404` 等三位数字误识别为 Case ID。
|
|
18
|
-
- `backend-test` traceability 门禁允许仅包裹 cleanup 调用的 `except requests.RequestException: pass`,避免 best-effort 资源清理被误报;bare/宽异常、`AssertionError` 及非 cleanup 主体仍然 fail-closed。
|
|
7
|
+
### 重点更新
|
|
19
8
|
|
|
20
|
-
-
|
|
21
|
-
-
|
|
22
|
-
-
|
|
23
|
-
- `frontend-mock-assess-pi` 的 `MOCK_STRATEGY:` 协议行规范化现在可处理列表、引用、inline code、中文冒号、`Mock strategy` 标签和标签/值分行等常见格式漂移;仍只接受明确 Mock strategy 标签和五态白名单,不从普通正文猜默认策略。
|
|
9
|
+
- 放宽后端测试 Markdown 用例的映射与验收标准校验,减少合法用例被误拦截的情况
|
|
10
|
+
- 修复文档审计报错未阻断 CI 的治理漏洞,并全面修正失效链接
|
|
11
|
+
- 清理发布日志中残留的已发布记录,避免将已上线能力误展示为未发布
|
|
24
12
|
|
|
25
|
-
###
|
|
26
|
-
|
|
27
|
-
- `backend-test` 的 source intake 能力已并入 Markdown 用例生成/Review:派生 `需求.md` / `执行约束.md` 有界内联,reference 使用 path/SHA-256/角色线索索引并按需精确读取;pytest writer 不再读取 `source/**`。
|
|
28
|
-
|
|
29
|
-
- **Console Phase 3(Observe 协作)**:
|
|
30
|
-
- Observe `GET /api/health` 升级为 versioned DTO(`schemaVersion: 1`、`repoFingerprint` 复用 Console `repoFingerprintV1`、package name/version、由 ROUTES 派生的 `routeCapabilities`)。
|
|
31
|
-
- Console `observeLink` fail-closed:仅 health **match**(同仓 fingerprint + schema + 目标 kind 所需 capability)时返回深链;offline / mismatch 返回诊断与 `agent-worker observe serve --repo . --port 8787`,不打开错误 URL。
|
|
32
|
-
- Doctor 的 Observe 探测扩展 match / mismatch / unavailable(Observe 仍非阻塞,doctor `ok` 保持)。
|
|
33
|
-
- Recovery CTA 矩阵(report / doctor / decision / resume / reconcile / regenerate / observeLink):**无** Cancel、**无**主 CTA「直接改代码」。
|
|
34
|
-
- Console 与 Observe 仍为独立进程;不 mount、不 proxy、不复制第二套 read model。
|
|
35
|
-
- Phase 1 Console skeleton:`agent-worker console serve|doctor`(默认 loopback `127.0.0.1:8790`、非 loopback fail-closed)、Vite SPA 静态产物 + Node health/static server、boot capability cookie + mutation Host/Origin 门禁、封闭 ResourceLoader 骨架、sibling `loop-agent` identity doctor、`repoFingerprintV1` / Observe 深链 builder、以及 `UPSTREAM.md`(pi-web 源 commit)与预算 report。
|
|
36
|
-
- Phase 0.5 operator surface repair:`buildOperatorCapabilitiesDocument()` 在权威 `src/shared/operator` 上补齐 Console 协商块 `dag`(`generatedSpecVersion: 4`、`requiresTaskContractBinding: true`,与 `dagSpecVersions` 同源常量),并删除冲突的 `src/task/operator` 双实现;CLI `--json` 继续写 `process.stdout`,测试侧改为捕获 stdout。
|
|
37
|
-
- Nightly 自动发布改为双基线:`lastTag`(最新稳定 `vX.Y.Z`)只负责未发布增量、breaking 检测与 release notes;当前 minor 的 `vX.Y.0`(缺失则回退 `lastTag`)只累计严格 Conventional Commit `feat:` 以决定 minor/patch。手动 patch 不再抹掉同 minor 周期的 feature 信号;历史 minor 窗口中的 breaking 不再永久阻断后续自动发布。`Publish npm` 手动入口未改。
|
|
38
|
-
|
|
39
|
-
- `frontend-implementation` 绿色路径由固定全链改为按需分支:删除重复的首轮 design shell gate,初审通过直接复用原计划,只有 `request-revision` 才执行计划修订与终审;只有可修复失败才执行 repair/reverify/retrace。常规全绿路径因此少执行 3 个 Pi 与 4 个 shell 节点,同时保留 pass-only 写入门禁、验证/评审证据和只读 `frontend-closeout-pi`。原计划/修订计划的结构化合同与需求覆盖使用同一有效来源,且只有主分支产物缺失时才回退;已存在但 malformed/schema-invalid 的产物继续 fail-closed。
|
|
40
|
-
- Observe 节点检查器“规范证据”页签新增安全文件预览:已绑定任务源和已读取规范文件条目现以可聚焦按钮呈现,点击后在检查器内下钻查看内容(不新增弹窗),提供“返回规范证据”按钮并恢复触发行焦点。新增只读接口 `GET /api/dag-runs/:runId/nodes/:nodeId/spec-evidence/file?source=binding|read&path=...`,服务端重新校验证据成员身份(binding 必须在 `sourceBinding.sources`,read 必须有成功配对 read),拒 `..`/绝对/目录/二进制/仓库外符号链接;正文上限 64KB 并返回截断标志,统一走 Observe 脱敏规则;binding 返回 `sha256`/`currentSha256`/`hashMatch`,read 返回 session event 中实际读取正文与 `readAt`(不以当前工作区文件替代)。
|
|
41
|
-
- 通用 standard/reviewed/supervised 实现 DAG 现在会在目标仓库存在 `AGENTS.md` 时,按本次 writer 的实际变更解析适用的根级/嵌套指令和仓库内代码规范索引:standard 复用 `verify-pi` 与条件 shell gate,reviewed/supervised 复用现有 `review-pi`/review gate,不新增模型节点;无 `AGENTS.md` 或无适用规范时保持原行为,审查节点只读且不会直接改代码。专用测试、前端和知识类 DAG 不受影响。
|
|
42
|
-
|
|
43
|
-
- GitHub Actions CI 降低私有仓分钟消耗:合并为单 job(避免两次 `npm ci`)、同分支/`PR` concurrency 取消旧 run、Draft PR 跳过完整 CI,且仅忽略 `docs/reports/**`、`docs/progress/**`、`docs/design/archive/**`、`docs/exec-plans/completed/**` 等纯运营文档路径;治理文档与代码门禁仍全量跑。
|
|
44
|
-
- 明确 **Compatibility / Operator Assist**:openCode 等主会话只编排已发布 `loop-agent` / `agent-worker` CLI 与只读诊断,不得绕过 CLI 直接改业务实现,失败只走 doctor / reconcile / human gate / CLI 重跑。`loop-agent init` 写入的 `AGENTS.md` managed block、包内 skills 与 website 快速开始/治理说明已对齐;架构决策见 `docs/decisions/0005-governed-operator-surface.md`(accepted)。已有目标项目用 `loop-agent init check-update` / `init update --apply-safe` 或 `init reconcile` 刷新 managed block。
|
|
45
|
-
- 前端测试 DAG 收紧体验:execution preflight 仅硬校验绝对非生产 `baseUrl`;用例 map 缩短为优先用 `playwright-cli` 执行;复盘合并执行证据审查且不依赖 outcome=pass,失败也能出报告。
|
|
46
|
-
- 前端测试用例生成现在会解析明确 `baseUrl`:优先读取任务源 `config.md` 中的前端 URL,缺失时默认 `http://localhost:5173`,并写入 RAG `context.md`;生成命令不得再保留 `<base-url>` 占位符。
|
|
47
|
-
- 前端浏览器测试 DAG 的复盘报告现写入 `testcase/frontend/reports/**`,不再要求 `docs/test-reports/**` 权限;任务执行约束可以安全禁止整个 `docs/**`,同时仍保留可审计的测试资产。
|
|
48
|
-
- `frontend-test` 结果链新增 run-owned `frontend-test-result-v1`、单次用例修订与终审门禁;只有所有浏览器用例和 AC 覆盖通过且结果合同明确为 real/pass 时,Worker 才会投影真实集成。
|
|
49
|
-
- 后端测试 DAG 从 24 个收敛为 15 个真实顶层节点;用例与语义 `request-revision` 由确定性 gate 直接失败,pytest 只执行一次,TestBug 保留分类并转为独立 follow-up,不再触发同 run repair 或 final rerun。唯一 initial Result 会确定性复制为 canonical `contracts/backend-test-result.json`,最终 outcome 只认该结构化事实。
|
|
50
|
-
- Backend Test Analysis、Case Manifest、Semantic Review 与 Classification 的严格 JSON 契约进一步对齐,生成节点明确字段白名单、`sourceBinding` 与 evidence gap 约束,避免模型自定义字段导致确定性门禁失败。
|
|
51
|
-
- 新增升级后 init surface notifier:普通安全仓库命令结束后,当目标项目存在有效 `.harness/init-surface.json` 且 `controllerVersion` 与当前包版本不同时自动检测。surface 缺失/损坏、源仓库、未初始化目录与版本一致时直接跳过;`checkInitUpdate().ok` 时仅在无活跃运行时静默刷新 state;非 TTY 只向 stderr 输出 `loop-agent init reconcile --repo-root ...` 提示且不写入目标;TTY 且无 human decisions、无活跃 DAG/Worker 时可经明确 `y/yes` 同意后应用 deterministic safe actions;只有 model merge 时直接输出有边界指引,不询问“应用 0 个动作”。notifier 遵循全局 `--repo-root`、不写 stdout、不改变原命令退出码,CI、禁用环境变量、help/version、`--json`/`--markdown`、`init`、`run-dag`、`dag`、`loop`、`delegate`、`pi-prompt`、`cursor-prompt` 均跳过。
|
|
52
|
-
- 新增统一命令 `loop-agent init reconcile --repo-root <target>`:surface 缺失返回 `needs-baseline` 且零写入,存在 human decisions 返回 `needs-human-decision` 且零写入,活跃 DAG/Worker 或 Worker 状态无法确认时返回 `blocked-active-runtime` 且零写入;其余情况复用现有 `applyInitUpdate({ applySafe: true })` 并复查返回 `clean`/`needs-model-merge`/`needs-safe-update`,支持 `--json`/`--markdown`。
|
|
53
|
-
- 测试报告新增 L-5 指标消费与 Python coverage.py JSON、Java JaCoCo XML 的统一 Code Coverage v1 报告入口,覆盖通过率、AC/自动化覆盖率、稳定性、失败原因、缺陷、风险和回归建议;缺失覆盖率证据保持 `unavailable`,不改变 backend-test outcome gate。
|
|
54
|
-
- 现有 backend-test DAG 在 retrospective 之后新增独立的 `l5-metrics-pi` 指标节点;它基于前序测试报告和 run-owned 证据输出 `L-5 ready/not-ready`,不创建独立 L-5 DAG,也不改变 Result v1 outcome gate。
|
|
55
|
-
- 后端测试 DAG 从 38 个收敛为 24 个真实顶层节点;使用 fail-closed `runIf` 和复合 Shell capability 减少调度,同时保留双合同、Manifest、语义评审、JUnit、Result、分类、修复安全、追踪和最终 outcome 证据。三条可选修订/修复分支仍各最多执行一次。
|
|
56
|
-
- Backend Test Analysis、Case Manifest、Semantic Review 与 Classification 的严格 JSON 契约进一步对齐,生成和修订节点明确字段白名单、`sourceBinding` 与 evidence gap 约束,避免模型自定义字段导致确定性门禁失败。
|
|
57
|
-
|
|
58
|
-
### 修复
|
|
59
|
-
|
|
60
|
-
- Init evolution 高影响校验现在覆盖升级后的 init surface notifier、eligibility policy、runtime activity gate 与 CLI entry 接线;这些专用路径单独变化时会触发 `model-review`,不再以 `no trigger matched` 静默跳过审查,同时不扩大到通用 CLI program/command registry 文件。
|
|
13
|
+
### 修复
|
|
61
14
|
|
|
62
|
-
-
|
|
63
|
-
-
|
|
64
|
-
-
|
|
65
|
-
|
|
66
|
-
-
|
|
67
|
-
|
|
68
|
-
- Backend-test 唯一 JSON object 提取器在扫描裸对象前会等长忽略完整 Markdown 行内代码中的对象形状示例(如 `{error:{code,message,type}}`);唯一真实 envelope 可正常物化,多个真实 JSON、fence 外额外对象与未闭合对象继续 fail-closed。
|
|
69
|
-
- Backend-test 节点 10 `backend-test-semantic-gate-shell` 不再用 Markdown `first-verdict-line` 读取 raw Pi 输出;改为同一 `semantic-initial` pipeline 按 task id 只读 `contracts/backend-test-semantic-review.json`,仅 `verdict=pass` 放行 pytest,`request-revision`/缺失/malformed 清晰 fail-closed;节点 09 仍只负责 materialize+trace,15 节点与单次 pytest 不变。
|
|
70
|
-
- 公开 Case Manifest schema(`docs/templates/backend-test-case-manifest.schema.json`)对齐 materializer dual-scope coverage:`coverageSummary` 在 `additionalProperties:false` 下 required 并声明 `global`/`inScope`(`total`/`covered`/`gapped`/`ratio`)与 `crossDomainGapCount`;回归测试以无依赖轻量 contract 锁定 schema↔canonical 一致性;不改 materializer/15 节点。
|
|
71
|
-
- Backend-test Case Manifest materializer 将模型顶层 `coverageSummary` 视为非权威:secret 扫描后剥离,再 strict parse 其余字段,并始终 `computeCaseManifestCoverageSummary` 写入 canonical summary;my-webapp 自定义/部分 summary 不再导致 invalid-output;prompt 禁止模型输出 coverageSummary;15 节点与单次 pytest 不变。
|
|
72
|
-
- Backend-test 裸 object 路径现消费 scanner 的 `unclosedObject`:当文本含一个完整裸 JSON object,且其前或后仍有未闭合裸 object 时一律 fail-closed,不再因 `spans.length === 1` 错误接受完整对象;Semantic Review / Classification / contracts envelope 共用该行为。
|
|
73
|
-
- Backend-test contracts composite pipeline (`validate-backend-test-contracts-shell`) 使用唯一确定性 envelope 提取:接受 pure JSON、唯一 fenced json、prose+唯一 fence、prose+唯一括号平衡裸 object;对无 JSON、多候选、未闭合、非 object、缺 `analysis`/`execution` fail-closed。提取后仍走 Analysis v2 / Execution v1 schema、source-binding 与 secret 门禁;不引入全局 Pi output rewrite,不将 `invalid-output` 加入自动 retry,不改 15 节点拓扑。
|
|
74
|
-
- Backend-test Semantic Review(08→09)与 Classification(12→13)materializer 复用同一 `extractUniqueJsonObject` scanner:接受 pure / whole-fence / prose+唯一 fence / prose+唯一裸 object;多候选、未闭合、非 object 与 strict schema 错误继续 fail-closed;不改 Case Manifest、Pi SDK/retry 或 15 节点拓扑。
|
|
75
|
-
- 节点检查器「执行过程」中的事件时间戳现在同时显示本地日期与 24 小时制 `HH:mm:ss`(例如 `2026/7/16 08:09:10`),不再被裁掉时间部分;缺少时间戳时仍显示「未记录时间」,事件字段优先级(`timestamp` → `at` → `recordedAt`)保持不变。
|
|
76
|
-
- `frontend-test` 的 case review 现在是 fail-closed browser gate:只有 `VERDICT: pass` 才能物化 manifest 并启动动态 browser map;生成的 case 固定使用默认 browser session,要求每个子场景的 fixture/UI reset 和 fresh snapshot,并由确定性节点校验每个 case 的 `execution.md`、`case-result.json`、`caseId`、`status`、`evidencePaths` 及 blocked `blockedReason`。
|
|
77
|
-
- 前端实现计划/修订节点会注入当前包内权威 `frontend-implementation-contract-v1` Schema 与固定 source binding,避免模型猜测字段导致契约门禁失败。
|
|
78
|
-
- 前端 Mock 策略节点的 canonical 输出会把首条 `MOCK_STRATEGY:` 协议行提升为第一行,避免解释性前言触发 `first-non-empty` 门禁误判。
|
|
79
|
-
- 前端规范回退目录统一为本地 `openspec/`,DAG 能力发现、提示词、Skill 与验证证据检查不再查找大小写不一致的旧目录名。
|
|
80
|
-
- 后端测试复合 Shell pipeline 现在与普通 Shell 节点共享 Git write guard;即使命令退出成功,只要越过 `read-only`、`allowedPaths` 或 `forbiddenPaths` 边界,节点仍会 fail-closed。
|
|
81
|
-
- Observe 对新 backend-test run 不再伪造 repair attempt/status,只投影 single-run/canonical Result、分类与 coverage;历史 24 节点 run 的 eligibility、repair、final 和 effective evidence 仍可兼容读取。
|
|
82
|
-
- Pi SDK 对缺少响应 ID 的累计 Token 生命周期事件改为取本次执行最大快照,避免同一响应的匿名 usage 被重复累加。
|
|
83
|
-
- Pi SDK 执行长推理或大段结构化输出时不再把高频流式增量事件无界累积到内存;同一响应在多个生命周期事件中重复出现的 Token 用量只统计一次,避免 `Invalid string length` 和成本数据虚高。
|
|
84
|
-
- 后端测试复合执行节点继续保持 clean environment、失败分类和 fail-closed outcome,并为唯一 JUnit/initial Result、canonical Result、traceability 与 Observe 投影保留结构化运行证据。
|
|
15
|
+
- 放宽后端测试 Markdown 用例的每条补充 Case 必须包含 AC 标识的硬性要求,改为确保任务要求的业务 AC 场景至少被一条最终用例覆盖即可,整体缺失必需 AC 时仍会严格阻断
|
|
16
|
+
- 放宽后端测试 traceability 映射校验,Case ID 可位于测试函数名、装饰器、函数体或文档字符串区域,不再强制首行文档字符串或严格一对一映射,避免重复映射等情况导致的误阻断
|
|
17
|
+
- 修复文档审计在发现错误时未返回非零退出码导致 CI 假绿的问题,并将链接门禁覆盖范围扩展至完整的 docs/ 目录
|
|
18
|
+
- 修正文档报告分类迁移后遗留的 269 条失效链接
|
|
19
|
+
- 修复自动发布创建版本区块时未正确消费已发布内容并清理残留记录的问题,移除了随 0.18.0 版本发布的 59 条残留日志
|
|
85
20
|
|
|
86
21
|
## [0.18.0] - 2026-07-22
|
|
87
22
|
|
|
@@ -326,8 +326,6 @@ export async function validateBackendMarkdownCases(input) {
|
|
|
326
326
|
}
|
|
327
327
|
}
|
|
328
328
|
const acIds = unique(testCase.body.match(AC_ID) ?? []);
|
|
329
|
-
if (acIds.length === 0)
|
|
330
|
-
throw new Error(`${testCase.id} has no AC-* reference`);
|
|
331
329
|
acIds.forEach((id) => coveredAc.add(id));
|
|
332
330
|
const sourceSection = sectionBody(testCase.body, "Source References");
|
|
333
331
|
const sourceRefs = extractSourceReferences({
|
|
@@ -378,12 +376,41 @@ export async function validateBackendMarkdownCases(input) {
|
|
|
378
376
|
];
|
|
379
377
|
return `${report.join("\n")}\n`;
|
|
380
378
|
}
|
|
379
|
+
function collectMarkdownCaseIds(markdown) {
|
|
380
|
+
return splitCases(markdown).map((testCase) => testCase.id);
|
|
381
|
+
}
|
|
382
|
+
function testFunctionRegion(input) {
|
|
383
|
+
const functionLineStart = input.source.lastIndexOf("\n", input.functionIndex - 1) + 1;
|
|
384
|
+
const prefix = input.source.slice(0, functionLineStart);
|
|
385
|
+
const previousBlankLine = Math.max(prefix.lastIndexOf("\n\n"), prefix.lastIndexOf("\r\n\r\n"));
|
|
386
|
+
const decoratorCandidateStart = previousBlankLine >= 0
|
|
387
|
+
? previousBlankLine + (prefix.startsWith("\r\n", previousBlankLine) ? 4 : 2)
|
|
388
|
+
: 0;
|
|
389
|
+
const decoratorCandidate = input.source.slice(decoratorCandidateStart, functionLineStart);
|
|
390
|
+
const regionStart = /^\s*@/m.test(decoratorCandidate)
|
|
391
|
+
? decoratorCandidateStart
|
|
392
|
+
: functionLineStart;
|
|
393
|
+
const nextBoundary = /^(?:(?:async\s+)?def|class)\s+[A-Za-z_][A-Za-z0-9_]*\b/gm;
|
|
394
|
+
nextBoundary.lastIndex = input.functionHeaderEnd;
|
|
395
|
+
const regionEnd = nextBoundary.exec(input.source)?.index ?? input.source.length;
|
|
396
|
+
return input.source
|
|
397
|
+
.slice(regionStart, regionEnd)
|
|
398
|
+
.split(/\r?\n/)
|
|
399
|
+
.filter((line) => !line.trimStart().startsWith("#"))
|
|
400
|
+
.join("\n");
|
|
401
|
+
}
|
|
402
|
+
function symbolCaseId(symbol) {
|
|
403
|
+
const match = symbol.match(/^test_(BE(?:_[A-Z0-9]+)+?_\d{3})(?:_|$)/i);
|
|
404
|
+
return match ? match[1].replaceAll("_", "-").toUpperCase() : undefined;
|
|
405
|
+
}
|
|
381
406
|
export async function validateBackendMarkdownTraceability(workspaceRoot) {
|
|
382
407
|
const files = await markdownFiles(workspaceRoot);
|
|
383
408
|
const markdownIds = new Set();
|
|
384
409
|
for (const file of files) {
|
|
410
|
+
if (path.basename(file).toLowerCase() === "readme.md")
|
|
411
|
+
continue;
|
|
385
412
|
const content = await readFile(file, "utf8");
|
|
386
|
-
for (const id of content
|
|
413
|
+
for (const id of collectMarkdownCaseIds(content))
|
|
387
414
|
markdownIds.add(id);
|
|
388
415
|
}
|
|
389
416
|
const testcaseRoot = path.join(workspaceRoot, "testcase");
|
|
@@ -398,7 +425,8 @@ export async function validateBackendMarkdownTraceability(workspaceRoot) {
|
|
|
398
425
|
}
|
|
399
426
|
}
|
|
400
427
|
await walk(testcaseRoot);
|
|
401
|
-
const
|
|
428
|
+
const mappings = new Map();
|
|
429
|
+
const automationIds = new Set();
|
|
402
430
|
for (const file of pythonFiles) {
|
|
403
431
|
const content = await readFile(file, "utf8");
|
|
404
432
|
if (/pytest\.(?:skip|xfail)\s*\(|@pytest\.mark\.(?:skip|xfail)\b/.test(content)) {
|
|
@@ -410,37 +438,34 @@ export async function validateBackendMarkdownTraceability(workspaceRoot) {
|
|
|
410
438
|
}
|
|
411
439
|
for (const match of content.matchAll(/^(?:async\s+)?def\s+(test_[A-Za-z0-9_]+)\s*\([^)]*\)\s*(?:->\s*[^:\r\n]+)?\s*:/gm)) {
|
|
412
440
|
const symbol = match[1];
|
|
413
|
-
const
|
|
414
|
-
|
|
415
|
-
|
|
441
|
+
const region = testFunctionRegion({
|
|
442
|
+
source: content,
|
|
443
|
+
functionIndex: match.index,
|
|
444
|
+
functionHeaderEnd: match.index + match[0].length,
|
|
416
445
|
});
|
|
417
|
-
|
|
418
|
-
|
|
419
|
-
|
|
420
|
-
|
|
421
|
-
const id
|
|
422
|
-
(
|
|
423
|
-
|
|
424
|
-
|
|
425
|
-
|
|
426
|
-
|
|
427
|
-
|
|
428
|
-
bucket.push(`${path.relative(workspaceRoot, file).replaceAll(path.sep, "/")}#${symbol}`);
|
|
429
|
-
symbols.set(id, bucket);
|
|
430
|
-
const tail = content.slice(match.index + match[0].length, match.index + match[0].length + 500);
|
|
431
|
-
if (!new RegExp(`^[\\s\\r\\n]*(?:[rRuUfF]{0,2})?[\"']{3}[^\n]*${id}`, "m").test(tail)) {
|
|
432
|
-
throw new Error(`${id} pytest symbol is missing a matching first-line docstring: ${bucket.at(-1)}`);
|
|
446
|
+
const associatedIds = new Set(region.match(CASE_ID) ?? []);
|
|
447
|
+
const fromSymbol = symbolCaseId(symbol);
|
|
448
|
+
if (fromSymbol)
|
|
449
|
+
associatedIds.add(fromSymbol);
|
|
450
|
+
for (const id of associatedIds) {
|
|
451
|
+
automationIds.add(id);
|
|
452
|
+
if (!markdownIds.has(id))
|
|
453
|
+
continue;
|
|
454
|
+
const bucket = mappings.get(id) ?? [];
|
|
455
|
+
bucket.push(`${path.relative(workspaceRoot, file).replaceAll(path.sep, "/")}#${symbol}`);
|
|
456
|
+
mappings.set(id, bucket);
|
|
433
457
|
}
|
|
434
458
|
}
|
|
435
459
|
}
|
|
436
|
-
const missing = [...markdownIds].filter((id) => !
|
|
437
|
-
const
|
|
438
|
-
.filter(([, refs]) => refs.length
|
|
460
|
+
const missing = [...markdownIds].filter((id) => !mappings.has(id));
|
|
461
|
+
const multiple = [...mappings]
|
|
462
|
+
.filter(([, refs]) => refs.length > 1)
|
|
439
463
|
.map(([id]) => id);
|
|
440
|
-
const extra = [...
|
|
441
|
-
if (missing.length
|
|
442
|
-
throw new Error(`traceability mismatch: missing=${missing.join(",")
|
|
464
|
+
const extra = [...automationIds].filter((id) => !markdownIds.has(id));
|
|
465
|
+
if (missing.length > 0) {
|
|
466
|
+
throw new Error(`traceability mismatch: missing=${missing.join(",")}; multiple=${multiple.join(",") || "none"}; extra=${extra.join(",") || "none"}`);
|
|
443
467
|
}
|
|
468
|
+
const mappingCount = [...mappings.values()].reduce((total, refs) => total + refs.length, 0);
|
|
444
469
|
return ([
|
|
445
470
|
"# Backend Test Traceability",
|
|
446
471
|
"",
|
|
@@ -449,11 +474,11 @@ export async function validateBackendMarkdownTraceability(workspaceRoot) {
|
|
|
449
474
|
"PASS",
|
|
450
475
|
"",
|
|
451
476
|
`- Markdown Case IDs: ${markdownIds.size}`,
|
|
452
|
-
`-
|
|
453
|
-
`-
|
|
477
|
+
`- Covered Markdown Case IDs: ${mappings.size}`,
|
|
478
|
+
`- Pytest mappings: ${mappingCount}`,
|
|
454
479
|
"- Missing mappings: 0",
|
|
455
|
-
"
|
|
456
|
-
"
|
|
480
|
+
`- Multiple mappings: ${multiple.length}${multiple.length ? ` (${multiple.join(", ")})` : ""}`,
|
|
481
|
+
`- Extra automation Case IDs: ${extra.length}${extra.length ? ` (${extra.join(", ")})` : ""}`,
|
|
457
482
|
"- skip/xfail findings: 0",
|
|
458
483
|
].join("\n") + "\n");
|
|
459
484
|
}
|
|
@@ -3333,14 +3333,14 @@ async function buildBackendTestHybridDag(sources) {
|
|
|
3333
3333
|
executor: "pi", toolProfile: "write", complexity: "HIGH", writePolicy: "exclusive",
|
|
3334
3334
|
writeSet: ["testcase/**/test_*.py", "testcase/**/helpers/**", "testcase/**/factories/**"],
|
|
3335
3335
|
allowedPaths: Array.from(new Set([...ro, "testcase/**"])), forbiddenPaths: forbidden,
|
|
3336
|
-
outputContract: "Convert every final automatable Markdown case
|
|
3336
|
+
outputContract: "Convert every final automatable Markdown case into pytest assets whose actual test function region contains the exact Case ID, preferably in the function name or docstring; no JSON and no pytest execution.",
|
|
3337
3337
|
subtask_prompt: [
|
|
3338
3338
|
"Convert validated testcase/md/** to pytest using upstream environment and validation evidence plus only bounded pytest config/conftest.",
|
|
3339
|
-
"
|
|
3339
|
+
"Ensure every final Markdown Case ID appears in at least one real top-level test_* function region, preferably as `test_BE_<MODULE>_<NNN>_<description>` or in that function's docstring. Multiple test functions may cover one Case ID; assertions come only from Expected Results and setup comes only from Preconditions/Test Data/Automation Notes.",
|
|
3340
3340
|
"Do not read source/**, add cases, reassign ACs, modify conftest/config/production code, use skip/xfail, swallow assertions, execute pytest, or emit JSON.",
|
|
3341
3341
|
].join("\n\n"),
|
|
3342
3342
|
};
|
|
3343
|
-
const traceability = shellNode("backend-test-traceability-gate-shell", [generatePytest.id], "markdown-traceability", "Fail closed
|
|
3343
|
+
const traceability = shellNode("backend-test-traceability-gate-shell", [generatePytest.id], "markdown-traceability", "Fail closed only when a real Markdown case heading has no associated top-level pytest test function. Accept exact Case IDs in the function name or its decorator/body/docstring region; report multiple mappings and extra automation Case IDs without blocking. Continue to reject skip/xfail or swallowed exceptions.", "Run-owned reports/backend-test-traceability.md proving every real Markdown Case ID is covered by at least one pytest test function.");
|
|
3344
3344
|
const pytestCommand = [
|
|
3345
3345
|
'mkdir -p "${HARNESS_DAG_RUN_DIR}/reports"',
|
|
3346
3346
|
'PYTHONUTF8=1 PYTHONIOENCODING=utf-8 PYTHONDONTWRITEBYTECODE=1 python -m pytest testcase/ -v -p no:cacheprovider --junitxml="${HARNESS_DAG_RUN_DIR}/reports/backend-test.junit.xml"',
|
|
@@ -202,8 +202,8 @@
|
|
|
202
202
|
".harness/dag-runs/**",
|
|
203
203
|
"artifacts/**"
|
|
204
204
|
],
|
|
205
|
-
"outputContract": "Convert every final automatable Markdown case
|
|
206
|
-
"subtask_prompt": "Convert validated testcase/md/** to pytest using upstream environment and validation evidence plus only bounded pytest config/conftest.\n\
|
|
205
|
+
"outputContract": "Convert every final automatable Markdown case into pytest assets whose actual test function region contains the exact Case ID, preferably in the function name or docstring; no JSON and no pytest execution.",
|
|
206
|
+
"subtask_prompt": "Convert validated testcase/md/** to pytest using upstream environment and validation evidence plus only bounded pytest config/conftest.\n\nEnsure every final Markdown Case ID appears in at least one real top-level test_* function region, preferably as `test_BE_<MODULE>_<NNN>_<description>` or in that function's docstring. Multiple test functions may cover one Case ID; assertions come only from Expected Results and setup comes only from Preconditions/Test Data/Automation Notes.\n\nDo not read source/**, add cases, reassign ACs, modify conftest/config/production code, use skip/xfail, swallow assertions, execute pytest, or emit JSON."
|
|
207
207
|
},
|
|
208
208
|
{
|
|
209
209
|
"id": "backend-test-traceability-gate-shell",
|
|
@@ -223,8 +223,8 @@
|
|
|
223
223
|
".harness/dag-runs/**",
|
|
224
224
|
"artifacts/**"
|
|
225
225
|
],
|
|
226
|
-
"outputContract": "Run-owned reports/backend-test-traceability.md proving Markdown
|
|
227
|
-
"subtask_prompt": "Fail closed
|
|
226
|
+
"outputContract": "Run-owned reports/backend-test-traceability.md proving every real Markdown Case ID is covered by at least one pytest test function.",
|
|
227
|
+
"subtask_prompt": "Fail closed only when a real Markdown case heading has no associated top-level pytest test function. Accept exact Case IDs in the function name or its decorator/body/docstring region; report multiple mappings and extra automation Case IDs without blocking. Continue to reject skip/xfail or swallowed exceptions.",
|
|
228
228
|
"shell": {
|
|
229
229
|
"commands": [],
|
|
230
230
|
"backendTestPipeline": "markdown-traceability",
|