@tea-agent/loop-agent 0.13.0-alpha.0 → 0.13.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (199) hide show
  1. package/AGENTS.md +4 -0
  2. package/CHANGELOG.md +57 -53
  3. package/README.md +12 -3
  4. package/dist/application/dag/generate-task-dag.js +28 -58
  5. package/dist/application/evaluation/candidate-hash.js +75 -0
  6. package/dist/application/evaluation/candidate.js +52 -0
  7. package/dist/application/evaluation/replay.js +289 -0
  8. package/dist/application/evaluation/types.js +130 -0
  9. package/dist/cli/command-definitions.js +17 -4
  10. package/dist/cli/program.js +8 -4
  11. package/dist/commands/eval.js +235 -0
  12. package/dist/commands/init.js +131 -24
  13. package/dist/executors/pi-sdk-executor.js +38 -24
  14. package/dist/executors/shell-executor.js +226 -15
  15. package/dist/executors/shell-presets.js +20 -0
  16. package/dist/executors/shell-verification.js +7 -0
  17. package/dist/governance/manifest-types.js +1 -0
  18. package/dist/infrastructure/evaluation/candidate-store.js +439 -0
  19. package/dist/infrastructure/evaluation/store.js +40 -0
  20. package/dist/task/config-types.js +23 -0
  21. package/dist/worker/observe/routes.js +18 -3
  22. package/dist/worker/observe/spec-evidence.js +1 -1
  23. package/dist/worker/observe/static/dom.js +160 -1
  24. package/dist/worker/observe/static/state.js +14 -0
  25. package/dist/worker/observe/static/views/dag-inspector.js +35 -4
  26. package/dist/worker/observe/static/views/dag.js +9 -0
  27. package/dist/worker/observe/static/views/dashboard.js +702 -445
  28. package/dist/worker/observe/static/views/session-timeline.js +15 -1
  29. package/dist/workflows/dag/backend-test-analysis-contract.js +120 -0
  30. package/dist/workflows/dag/backend-test-case-manifest.js +503 -0
  31. package/dist/workflows/dag/backend-test-execution-contract.js +353 -0
  32. package/dist/workflows/dag/backend-test-result-contract.js +568 -0
  33. package/dist/workflows/dag/decision-envelope.js +57 -2
  34. package/dist/workflows/dag/dynamic-runtime/map.js +90 -2
  35. package/dist/workflows/dag/frontend-implementation-contract.js +240 -0
  36. package/dist/workflows/dag/frontend-project-capability.js +309 -0
  37. package/dist/workflows/dag/frontend-repair.js +341 -0
  38. package/dist/workflows/dag/frontend-risk.js +161 -0
  39. package/dist/workflows/dag/frontend-verification-trace.js +190 -0
  40. package/dist/workflows/dag/init-hybrid.js +2407 -297
  41. package/dist/workflows/dag/node-execution.js +9 -0
  42. package/dist/workflows/dag/prompt.js +9 -0
  43. package/dist/workflows/dag/repair-artifact.js +43 -3
  44. package/dist/workflows/dag/report.js +35 -1
  45. package/dist/workflows/dag/runner.js +28 -2
  46. package/dist/workflows/dag/skill-instructions.js +4 -2
  47. package/dist/workflows/dag/task-demand-routing.js +383 -0
  48. package/dist/workflows/dag/types.js +71 -13
  49. package/dist/workflows/dag/upstream-artifacts.js +1 -0
  50. package/dist/workflows/dag/validate.js +59 -1
  51. package/docs/README.md +6 -3
  52. package/docs/agent-dag-recovery-playbook.md +5 -3
  53. package/docs/agent-dag-runner.md +3 -3
  54. package/docs/architecture/README.md +3 -3
  55. package/docs/architecture/dag-execution.md +1 -1
  56. package/docs/architecture/evolution.md +13 -13
  57. package/docs/architecture/facts-and-state.md +1 -1
  58. package/docs/architecture/runtime-boundaries.md +7 -7
  59. package/docs/architecture/system-overview.md +3 -3
  60. package/docs/architecture/worker-and-feature.md +3 -3
  61. package/docs/design/README.md +124 -42
  62. package/docs/development-principles.md +4 -4
  63. package/docs/exec-plans/active/README.md +12 -11
  64. package/docs/exec-plans/completed/README.md +33 -0
  65. package/docs/feature-workflow.md +114 -39
  66. package/docs/init-surface.manifest.json +30 -3
  67. package/docs/loop-agent-harness.md +9 -8
  68. package/docs/production-readiness.md +1 -1
  69. package/docs/progress/README.md +23 -1
  70. package/docs/reports/README.md +65 -6
  71. package/docs/skills/vetted-skill-registry.md +2 -0
  72. package/docs/templates/agent-dag.schema.json +29 -1
  73. package/docs/templates/agent-dag.supervised-implementation.json +127 -8
  74. package/docs/templates/backend-test-analysis.schema.json +44 -0
  75. package/docs/templates/backend-test-case-manifest.schema.json +190 -0
  76. package/docs/templates/backend-test-dag.classify.prompt.md +75 -0
  77. package/docs/templates/backend-test-dag.generate-pytest.prompt.md +81 -16
  78. package/docs/templates/backend-test-dag.json +311 -40
  79. package/docs/templates/backend-test-dag.retrospect.prompt.md +44 -30
  80. package/docs/templates/backend-test-dag.review-cases.prompt.md +10 -8
  81. package/docs/templates/backend-test-execution.schema.json +133 -0
  82. package/docs/templates/backend-test-result.schema.json +99 -0
  83. package/docs/templates/branch-merge-report.md +93 -0
  84. package/docs/templates/frontend-design-contract.md +9 -0
  85. package/docs/templates/frontend-eval/fixtures/failures/01-type-build-error.md +17 -0
  86. package/docs/templates/frontend-eval/fixtures/failures/02-unit-component-test-fail.md +16 -0
  87. package/docs/templates/frontend-eval/fixtures/failures/03-fixture-schema-drift.md +16 -0
  88. package/docs/templates/frontend-eval/fixtures/failures/04-missing-loading-empty-error-state.md +16 -0
  89. package/docs/templates/frontend-eval/fixtures/failures/05-forbidden-write-writeset-expansion.md +16 -0
  90. package/docs/templates/frontend-eval/fixtures/failures/06-unapproved-dependency-add.md +16 -0
  91. package/docs/templates/frontend-eval/fixtures/failures/07-mock-production-on.md +21 -0
  92. package/docs/templates/frontend-eval/fixtures/functional/01-simple-component-style.md +29 -0
  93. package/docs/templates/frontend-eval/fixtures/functional/02-form-validation.md +28 -0
  94. package/docs/templates/frontend-eval/fixtures/functional/03-list-detail-page.md +28 -0
  95. package/docs/templates/frontend-eval/fixtures/functional/04-api-mock.md +29 -0
  96. package/docs/templates/frontend-eval/fixtures/functional/05-permission-auth-gated-ui.md +27 -0
  97. package/docs/templates/frontend-eval/fixtures/functional/06-ssr-server-client-boundary.md +28 -0
  98. package/docs/templates/frontend-eval/fixtures/functional/07-shared-public-component-api.md +28 -0
  99. package/docs/templates/frontend-eval/fixtures/functional/08-pure-local-no-remote.md +27 -0
  100. package/docs/templates/frontend-eval/metrics.md +138 -0
  101. package/docs/templates/frontend-eval/smoke-targets.md +53 -0
  102. package/docs/templates/frontend-implementation-contract.schema.json +27 -0
  103. package/docs/templates/frontend-task-constraints.md +10 -0
  104. package/docs/templates/frontend-task-requirement.md +9 -0
  105. package/docs/templates/frontend-test-dag.generate-cases.prompt.md +5 -0
  106. package/docs/templates/frontend-test-dag.json +23 -0
  107. package/docs/templates/frontend-test-dag.retrieve-context.prompt.md +3 -0
  108. package/docs/templates/frontend-test-dag.retrospect.prompt.md +3 -0
  109. package/docs/templates/frontend-test-dag.review-cases.prompt.md +3 -0
  110. package/docs/templates/frontend-test-dag.review-execution.prompt.md +3 -0
  111. package/docs/templates/knowledge-graph-bootstrap-dag.json +1 -1
  112. package/docs/templates/knowledge-sync-dag.json +1 -0
  113. package/docs/verification-matrix.md +4 -1
  114. package/examples/decision-gate-agent-dag.json +4 -4
  115. package/examples/hybrid-loop-agent-dag.json +1 -1
  116. package/package.json +2 -2
  117. package/scripts/kb-bootstrap-init-skeleton.sh +2 -1
  118. package/scripts/kb-graph-incremental-prepare.mjs +19 -5
  119. package/scripts/kb-graph-promote.mjs +12 -1
  120. package/skills/ai-engineering-context/SKILL.md +2 -2
  121. package/skills/analyze-product-dependencies/SKILL.md +67 -0
  122. package/skills/analyze-product-dependencies/agents/openai.yaml +4 -0
  123. package/skills/analyze-product-dependencies/references/api-documentation-schema.md +30 -0
  124. package/skills/analyze-product-dependencies/references/dependency-analysis-schema.md +28 -0
  125. package/skills/analyze-product-dependencies/references/example.md +76 -0
  126. package/skills/analyze-product-dependencies/references/forward-test-cases.md +35 -0
  127. package/skills/analyze-product-dependencies/references/input-contract.md +11 -0
  128. package/skills/analyze-product-dependencies/references/scouting-rules.md +61 -0
  129. package/skills/analyze-product-dependencies/scripts/test-validators.mjs +267 -0
  130. package/skills/analyze-product-dependencies/scripts/validate-api-documentation.mjs +101 -0
  131. package/skills/analyze-product-dependencies/scripts/validate-dependency-analysis.mjs +142 -0
  132. package/skills/analyze-product-dependencies/scripts/validate-product-requirement-input.mjs +76 -0
  133. package/skills/analyze-product-dependencies/scripts/validation-helpers.mjs +146 -0
  134. package/skills/analyze-product-requirements/SKILL.md +90 -0
  135. package/skills/analyze-product-requirements/agents/openai.yaml +4 -0
  136. package/skills/analyze-product-requirements/references/acceptance-criteria.md +91 -0
  137. package/skills/analyze-product-requirements/references/clarification-and-knowledge.md +56 -0
  138. package/skills/analyze-product-requirements/references/example.md +86 -0
  139. package/skills/analyze-product-requirements/references/forward-test-cases.md +66 -0
  140. package/skills/analyze-product-requirements/references/product-analysis-schema.md +32 -0
  141. package/skills/analyze-product-requirements/references/product-requirement-schema.md +33 -0
  142. package/skills/analyze-product-requirements/references/requirement-clarification-schema.md +35 -0
  143. package/skills/analyze-product-requirements/scripts/test-validators.mjs +193 -0
  144. package/skills/analyze-product-requirements/scripts/validate-product-analysis.mjs +69 -0
  145. package/skills/analyze-product-requirements/scripts/validate-product-requirement.mjs +97 -0
  146. package/skills/analyze-product-requirements/scripts/validate-requirement-clarification.mjs +98 -0
  147. package/skills/analyze-product-requirements/scripts/validation-helpers.mjs +156 -0
  148. package/skills/browser-tools/SKILL.md +196 -0
  149. package/skills/browser-tools/browser-content.js +103 -0
  150. package/skills/browser-tools/browser-cookies.js +35 -0
  151. package/skills/browser-tools/browser-eval.js +53 -0
  152. package/skills/browser-tools/browser-hn-scraper.js +108 -0
  153. package/skills/browser-tools/browser-nav.js +44 -0
  154. package/skills/browser-tools/browser-pick.js +162 -0
  155. package/skills/browser-tools/browser-screenshot.js +34 -0
  156. package/skills/browser-tools/browser-start.js +86 -0
  157. package/skills/browser-tools/package-lock.json +2556 -0
  158. package/skills/browser-tools/package.json +19 -0
  159. package/skills/frontend-design-review/SKILL.md +6 -1
  160. package/skills/frontend-design-review/references/review-checklist.md +25 -4
  161. package/skills/frontend-implementation/SKILL.md +25 -30
  162. package/skills/frontend-implementation/references/code-standards.md +20 -22
  163. package/skills/frontend-implementation/references/node-contracts.md +17 -53
  164. package/skills/frontend-review/SKILL.md +10 -4
  165. package/skills/frontend-review/references/review-findings.md +8 -3
  166. package/skills/frontend-verification/SKILL.md +22 -9
  167. package/skills/frontend-verification/references/verification-checklist.md +17 -5
  168. package/skills/grill-with-docs/SKILL.md +5 -5
  169. package/skills/grill-with-docs/adr-format.md +3 -3
  170. package/skills/init-capability-evolution/SKILL.md +5 -5
  171. package/skills/loop-agent/SKILL.md +5 -5
  172. package/skills/loop-agent/references/README.md +3 -3
  173. package/skills/loop-agent/references/command-reference.md +98 -24
  174. package/skills/loop-agent/references/docs-converge.md +15 -15
  175. package/skills/loop-agent/references/harness-policy.md +2 -2
  176. package/skills/loop-agent/references/hybrid-dag.md +32 -22
  177. package/skills/loop-agent/references/multi-worktree.md +1 -1
  178. package/skills/loop-agent/references/orchestrator-and-interventions.md +8 -8
  179. package/skills/loop-agent/references/task-workflow.md +1 -1
  180. package/skills/loop-agent/references/verification-and-failure-handling.md +6 -4
  181. package/skills/playwright-cli/SKILL.md +420 -0
  182. package/skills/playwright-cli/references/element-attributes.md +23 -0
  183. package/skills/playwright-cli/references/playwright-tests.md +39 -0
  184. package/skills/playwright-cli/references/request-mocking.md +87 -0
  185. package/skills/playwright-cli/references/running-code.md +241 -0
  186. package/skills/playwright-cli/references/session-management.md +225 -0
  187. package/skills/playwright-cli/references/storage-state.md +275 -0
  188. package/skills/playwright-cli/references/test-generation.md +433 -0
  189. package/skills/playwright-cli/references/tracing.md +139 -0
  190. package/skills/playwright-cli/references/video-recording.md +143 -0
  191. package/skills/playwright-cli-case-generator/SKILL.md +74 -0
  192. package/skills/requesting-code-review/SKILL.md +1 -1
  193. package/skills/systematic-debugging/CREATION-LOG.md +3 -3
  194. package/skills/systematic-debugging/SKILL.md +1 -1
  195. package/skills/systematic-debugging/test-academic.md +1 -1
  196. package/skills/systematic-debugging/test-pressure-1.md +1 -1
  197. package/skills/systematic-debugging/test-pressure-2.md +1 -1
  198. package/skills/systematic-debugging/test-pressure-3.md +1 -1
  199. package/skills/verification-before-completion/SKILL.md +1 -1
@@ -2,7 +2,7 @@
2
2
 
3
3
  ## Purpose
4
4
 
5
- Use this prompt for a **test retrospective** node: `executor: "pi"`, `role: "closeout"`, `toolProfile: "write"`, `writePolicy: "exclusive"`. The closeout agent reads upstream review reports and pytest execution results, then generates a retrospective report with an objective maturity rating.
5
+ Use this prompt for a **test retrospective** node: `executor: "pi"`, `role: "closeout"`, `toolProfile: "write"`, `writePolicy: "exclusive"`. The closeout agent reads **Backend Test Result v1** (and classification), then generates a retrospective report with an objective maturity rating.
6
6
 
7
7
  Do **not** create a new executor type. This is a standard `executor: pi` writer node.
8
8
 
@@ -11,7 +11,7 @@ Do **not** create a new executor type. This is a standard `executor: pi` writer
11
11
  ```json
12
12
  {
13
13
  "id": "test-retrospect-pi",
14
- "depends_on": ["execute-backend-pytest-shell"],
14
+ "depends_on": ["classify-backend-test-result-pi"],
15
15
  "complexity": "MED",
16
16
  "executor": "pi",
17
17
  "role": "closeout",
@@ -20,7 +20,7 @@ Do **not** create a new executor type. This is a standard `executor: pi` writer
20
20
  "writeSet": ["docs/test-reports/**"],
21
21
  "allowedPaths": ["docs/test-reports/**"],
22
22
  "forbiddenPaths": [".harness/**", "artifacts/**"],
23
- "outputContract": "Markdown retrospective report under docs/test-reports/ with coverage summary, review findings, pytest results, and maturity rating (A/B/C/D).",
23
+ "outputContract": "Markdown retrospective report under docs/test-reports/ with coverage summary, review findings, Result v1 stats, classification, and maturity rating (A/B/C/D).",
24
24
  "subtask_prompt_markdown": "./backend-test-dag.retrospect.prompt.md"
25
25
  }
26
26
  ```
@@ -29,7 +29,9 @@ Do **not** create a new executor type. This is a standard `executor: pi` writer
29
29
 
30
30
  You are the Backend Test DAG **test retrospective** agent.
31
31
 
32
- Your job is to read upstream outputs (review report + pytest results) and generate a retrospective report with a maturity rating. Write the report under `docs/test-reports/` only. Stay within `writeSet`. Do not write root `artifacts/**`.
32
+ Your job is to read upstream Result v1 + classification (+ review report) and generate a retrospective report with a maturity rating. Write the report under `docs/test-reports/` only. Stay within `writeSet`. Do not write root `artifacts/**`.
33
+
34
+ This node runs on **both pass and assertion-fail** paths (after parse + classify). Final task success is decided later by `backend-test-outcome-gate-shell` using Result v1 shell facts only — **never** rewrite a failed result as passed in this report.
33
35
 
34
36
  ### Output Steps (do in order)
35
37
 
@@ -38,28 +40,40 @@ Your job is to read upstream outputs (review report + pytest results) and genera
38
40
 
39
41
  ### Inputs
40
42
 
41
- 1. **Review report** — `review-backend-cases-pi` output (VERDICT, findings, coverage assessment).
42
- 2. **Pytest output** `execute-backend-pytest-shell` stdout/stderr and exit code.
43
- 3. **HTML report** — `reports/backend-test-report.html` (if generated).
43
+ 1. **Result v1 (authoritative stats)** — `$HARNESS_DAG_RUN_DIR/contracts/backend-test-result.json`
44
+ Use `passed` / `failed` / `error` / `skipped` / `outcome` / `failures[]` / `pytestExitCode` only from this artifact.
45
+ 2. **Case Manifest v1 (authoritative AC coverage)** — `$HARNESS_DAG_RUN_DIR/contracts/backend-test-case-manifest.json`
46
+ Use `coverageSummary.acCoverageRatio`, `coveredAcCount`, `explicitAcCount`, case counts only from this artifact.
47
+ 3. **Classification** — `classify-backend-test-result-pi` JSON (`category`, `confidence`, `evidence`). Interpretive only; does not override outcome.
48
+ 4. **Review report** — `review-backend-cases-pi` output (VERDICT, findings, coverage assessment).
49
+ 5. Optional secondary: execute stdout markers / JUnit path (do not re-parse logs for counts when Result v1 exists).
44
50
 
45
51
  Do NOT re-read source documents. Use upstream outputs only.
46
52
 
53
+ ### Stats authority
54
+
55
+ - Pass rate = `passed / (passed + failed + error)` when denominator > 0 (skipped excluded from denominator unless Result documents otherwise) — **Result v1 only**.
56
+ - AC coverage = `coverageSummary.acCoverageRatio` from Case Manifest v1 only (do **not** recompute or invent percentages).
57
+ - Failed case table rows must match `failures[]` from Result v1.
58
+ - If Result v1 `outcome` is not `passed`, the retrospective **must not** claim overall success.
59
+
47
60
  ### Maturity Rating Criteria
48
61
 
49
- | Rating | Coverage | Pass Rate | Review Findings |
50
- |--------|----------|-----------|-----------------|
51
- | **A** | 100% acceptance criteria covered | 100% pytest pass | No Critical or Important findings |
52
- | **B** | ≥80% acceptance criteria covered | ≥90% pytest pass | Only Informational findings |
53
- | **C** | ≥60% acceptance criteria covered | ≥70% pytest pass | No Critical findings (Important allowed) |
62
+ | Rating | Coverage (manifest) | Pass Rate (Result v1) | Review Findings |
63
+ |--------|---------------------|------------------------|-----------------|
64
+ | **A** | `acCoverageRatio` = 1 | 100% pytest pass | No Critical or Important findings |
65
+ | **B** | `acCoverageRatio` 0.8 | ≥90% pytest pass | Only Informational findings |
66
+ | **C** | `acCoverageRatio` 0.6 | ≥70% pytest pass | No Critical findings (Important allowed) |
54
67
  | **D** | Below C thresholds | Below C thresholds | Or any Critical finding unresolved |
55
68
 
56
69
  #### Rating Rules
57
70
 
58
- - **Coverage** = (acceptance criteria with ≥1 covering test case) / (total acceptance criteria) × 100%
59
- - **Pass rate** = (passed pytest functions) / (total non-skipped pytest functions) × 100%
71
+ - **Coverage** from Case Manifest `coverageSummary` only (deterministic gate product).
72
+ - **Pass rate** from Result v1 only (not guessed from logs).
60
73
  - If `review-backend-cases-pi` returned `VERDICT: request-revision` and revision was not completed, cap at **D**.
61
- - If pytest exit code is non-zero and >30% tests failed, cap at **D** regardless of coverage.
62
- - Skipped tests (`@pytest.mark.skip`) count as "not covered" for pass rate but not as failures.
74
+ - If Result v1 shows >30% failed+error among executed tests, cap at **D** regardless of coverage.
75
+ - Skipped tests count as "not covered" for pass rate but not as failures.
76
+ - Collection/command/report errors → cap at **D** and record classification (not ProductBug by default).
63
77
 
64
78
  ### Report Structure
65
79
 
@@ -71,6 +85,8 @@ Write the report as a Markdown file named `backend-test-retrospect-<date>.md` un
71
85
  **Date:** <YYYY-MM-DD>
72
86
  **Task:** <task-id>
73
87
  **Maturity Rating:** <A|B|C|D>
88
+ **Result outcome:** <from Result v1>
89
+ **Classification:** <from classify JSON>
74
90
 
75
91
  ## 1. Test Coverage Summary
76
92
 
@@ -79,43 +95,41 @@ Write the report as a Markdown file named `backend-test-retrospect-<date>.md` un
79
95
  | Total acceptance criteria | N |
80
96
  | Covered by test cases | N (X%) |
81
97
  | Total functional test cases | N |
82
- | Positive path cases | N |
83
- | Negative path cases | N |
84
- | Boundary cases | N |
85
98
 
86
- ## 2. Automation Results
99
+ ## 2. Automation Results (from Result v1)
87
100
 
88
101
  | Metric | Value |
89
102
  |--------|-------|
90
- | Total pytest functions | N |
103
+ | Total tests | N |
91
104
  | Passed | N |
92
105
  | Failed | N |
106
+ | Error | N |
93
107
  | Skipped | N |
94
108
  | Pass rate | X% |
95
109
  | Pytest exit code | N |
110
+ | Outcome | … |
111
+ | Execution status | … |
96
112
 
97
113
  ### Failed Test Analysis
98
114
 
99
- | Test Case ID | Function | Failure Reason | Root Cause |
100
- |--------------|----------|----------------|------------|
101
- | ... | ... | ... | ... |
115
+ | Test Case / Function | Message (truncated) | Classification |
116
+ |----------------------|-------------------|----------------|
117
+ | ... | ... | ... |
102
118
 
103
119
  ## 3. Review Findings
104
120
 
105
121
  | Severity | Finding | Status |
106
122
  |----------|---------|--------|
107
- | Critical | ... | Resolved / Unresolved |
108
- | Important | ... | Resolved / Unresolved |
109
- | Informational | ... | Resolved / Unresolved |
123
+ | | | |
110
124
 
111
125
  ## 4. Maturity Rating Rationale
112
126
 
113
- Explain which threshold was met or missed, and why the specific rating was assigned.
127
+ Explain which threshold was met or missed.
114
128
 
115
129
  ## 5. Recommendations
116
130
 
117
- - Actionable items for improving the rating in the next iteration.
118
- - Specific gaps to close (uncovered criteria, flaky tests, missing negative paths).
131
+ - Actionable items for the next iteration.
132
+ - Do not propose changing production code solely to greenwash tests.
119
133
  ```
120
134
 
121
135
  ### Output Shape (after rating line)
@@ -11,7 +11,7 @@ Do **not** create `executor: reviewer`. Reviewer is a **role** on `executor: pi`
11
11
  ```json
12
12
  {
13
13
  "id": "review-backend-cases-pi",
14
- "depends_on": ["generate-backend-functional-cases-pi"],
14
+ "depends_on": ["backend-test-case-manifest-shell", "backend-test-analysis-contract-shell"],
15
15
  "complexity": "HIGH",
16
16
  "executor": "pi",
17
17
  "role": "reviewer",
@@ -40,10 +40,11 @@ No preamble, heading, or blank lines before the verdict line.
40
40
 
41
41
  ### Inputs to Review
42
42
 
43
- 1. **Acceptance criteria** — from upstream `analyze-inputs-pi` output (AC-001, AC-002, ...).
44
- 2. **Generated test cases** — files under `testcase/md/`.
43
+ 1. **Acceptance criteria / analysis** — from the validated Backend Test Analysis v1 artifact materialized by `backend-test-analysis-contract-shell` (`contracts/backend-test-analysis.json` under the current DAG run). Do not treat free-form Markdown from `analyze-inputs-pi` as the contract.
44
+ 2. **Case Manifest v1** — `contracts/backend-test-case-manifest.json` (schemaId `backend-test-case-manifest-v1`). Prefer `coverageSummary` and caseId↔acIds from this artifact; do not invent coverage percentages.
45
+ 3. **Generated test cases** — files under `testcase/md/`.
45
46
 
46
- Do NOT re-read source documents. Use upstream outputs only.
47
+ Do NOT re-read source documents. Use the validated analysis artifact, case manifest, and generated cases only.
47
48
 
48
49
  ### Review Checklist
49
50
 
@@ -54,15 +55,16 @@ Do NOT re-read source documents. Use upstream outputs only.
54
55
  | **Negative path coverage** | Error/exception scenarios (invalid input, not found, state violations) | Important |
55
56
  | **Boundary conditions** | Edge cases (empty input, max length, edge values) | Important |
56
57
  | **State transitions** | Illegal state changes covered | Important |
57
- | **Requirement traceability** | Each acceptance criterion (AC-xxx) maps to at least one test case ID | Critical |
58
+ | **Requirement traceability** | Each acceptance criterion (AC-xxx) maps to at least one test case ID (manifest coverageSummary or evidenceGaps) | Critical |
59
+ | **Manifest consistency** | Markdown cases align with Case Manifest v1 caseId/acIds | Critical |
58
60
  | **Case structure** | Each case has: ID, Title, Precondition, Steps, Expected Result | Important |
59
61
  | **No duplicate IDs** | All test case IDs are unique across files | Critical |
60
62
 
61
63
  ### Conditional Coverage (check ONLY if mentioned in upstream analysis)
62
64
 
63
- - **Authentication coverage**: check ONLY if `analyze-inputs-pi` mentions auth mechanism (JWT, OAuth2, API Key, etc.)
64
- - **Timeout coverage**: check ONLY if `analyze-inputs-pi` mentions timeout handling or degradation strategy
65
- - If not mentioned in upstream analysis, do NOT flag as missing
65
+ - **Authentication coverage**: check ONLY if the validated analysis artifact mentions auth mechanism (JWT, OAuth2, API Key, etc.)
66
+ - **Timeout coverage**: check ONLY if the validated analysis artifact mentions timeout handling or degradation strategy
67
+ - If not mentioned in the validated analysis artifact, do NOT flag as missing
66
68
 
67
69
  ### Verdict Rules
68
70
 
@@ -0,0 +1,133 @@
1
+ {
2
+ "$schema": "https://json-schema.org/draft/2020-12/schema",
3
+ "$id": "https://tea-agent.dev/schemas/backend-test-execution-v1.json",
4
+ "title": "Backend Test Execution v1",
5
+ "type": "object",
6
+ "additionalProperties": false,
7
+ "required": [
8
+ "schemaVersion",
9
+ "framework",
10
+ "runner",
11
+ "testRoot",
12
+ "workingDirectory",
13
+ "report",
14
+ "targetMode",
15
+ "existingFixtures",
16
+ "authenticationMode",
17
+ "requiredEnvNames",
18
+ "dataIsolation",
19
+ "evidenceGaps",
20
+ "evidenceRefs"
21
+ ],
22
+ "properties": {
23
+ "schemaVersion": { "const": 1 },
24
+ "framework": { "const": "pytest" },
25
+ "runner": {
26
+ "type": "object",
27
+ "additionalProperties": false,
28
+ "properties": {
29
+ "commandParts": {
30
+ "type": "array",
31
+ "items": { "type": "string", "minLength": 1 }
32
+ },
33
+ "frozenCommandHints": {
34
+ "type": "array",
35
+ "items": { "type": "string", "minLength": 1 }
36
+ }
37
+ }
38
+ },
39
+ "testRoot": {
40
+ "type": "string",
41
+ "minLength": 1,
42
+ "description": "Repo-relative posix path; no absolute form or .. segments"
43
+ },
44
+ "workingDirectory": {
45
+ "type": "string",
46
+ "minLength": 1,
47
+ "default": "."
48
+ },
49
+ "report": {
50
+ "type": "object",
51
+ "additionalProperties": false,
52
+ "required": ["format", "relativeHint"],
53
+ "properties": {
54
+ "format": { "const": "junit" },
55
+ "relativeHint": { "type": "string", "minLength": 1 }
56
+ }
57
+ },
58
+ "targetMode": {
59
+ "enum": ["in-process", "external-running-service", "managed-command"]
60
+ },
61
+ "baseUrlEnvName": {
62
+ "type": "string",
63
+ "pattern": "^[A-Z_][A-Z0-9_]*$"
64
+ },
65
+ "readiness": {
66
+ "type": "array",
67
+ "items": {
68
+ "type": "object",
69
+ "additionalProperties": false,
70
+ "required": ["path", "description"],
71
+ "properties": {
72
+ "path": { "type": "string", "minLength": 1 },
73
+ "description": { "type": "string", "minLength": 1 }
74
+ }
75
+ }
76
+ },
77
+ "existingFixtures": {
78
+ "type": "array",
79
+ "items": {
80
+ "type": "object",
81
+ "additionalProperties": false,
82
+ "required": ["name", "sourcePath", "kind"],
83
+ "properties": {
84
+ "name": { "type": "string", "minLength": 1 },
85
+ "sourcePath": { "type": "string", "minLength": 1 },
86
+ "kind": { "type": "string", "minLength": 1 }
87
+ }
88
+ }
89
+ },
90
+ "authenticationMode": { "type": "string", "minLength": 1 },
91
+ "requiredEnvNames": {
92
+ "type": "array",
93
+ "items": {
94
+ "type": "string",
95
+ "pattern": "^[A-Z_][A-Z0-9_]*$"
96
+ }
97
+ },
98
+ "dataIsolation": {
99
+ "type": "object",
100
+ "additionalProperties": false,
101
+ "required": ["mode"],
102
+ "properties": {
103
+ "mode": { "type": "string", "minLength": 1 },
104
+ "evidence": { "type": "string", "minLength": 1 }
105
+ }
106
+ },
107
+ "managedCommand": {
108
+ "type": "object",
109
+ "additionalProperties": false,
110
+ "properties": {
111
+ "start": { "type": "string", "minLength": 1 },
112
+ "stop": { "type": "string", "minLength": 1 },
113
+ "sourceRef": { "type": "string", "minLength": 1 }
114
+ }
115
+ },
116
+ "evidenceGaps": {
117
+ "type": "array",
118
+ "items": {
119
+ "type": "object",
120
+ "additionalProperties": false,
121
+ "required": ["description"],
122
+ "properties": {
123
+ "description": { "type": "string", "minLength": 1 },
124
+ "sourceRef": { "type": "string", "minLength": 1 }
125
+ }
126
+ }
127
+ },
128
+ "evidenceRefs": {
129
+ "type": "array",
130
+ "items": { "type": "string", "minLength": 1 }
131
+ }
132
+ }
133
+ }
@@ -0,0 +1,99 @@
1
+ {
2
+ "$schema": "https://json-schema.org/draft/2020-12/schema",
3
+ "$id": "https://tea-agent.dev/schemas/backend-test-result-v1.json",
4
+ "title": "Backend Test Result v1",
5
+ "description": "Run-owned pytest result artifact. Task Pool may consume outcome, counts, failures[], executionStatus, collectionStatus, pytestExitCode, and junit refs. Auto follow-up is out of scope for M2.",
6
+ "type": "object",
7
+ "additionalProperties": false,
8
+ "required": [
9
+ "schemaVersion",
10
+ "executionStatus",
11
+ "pytestExitCode",
12
+ "collectionStatus",
13
+ "tests",
14
+ "passed",
15
+ "failed",
16
+ "error",
17
+ "skipped",
18
+ "junit",
19
+ "commandSummary",
20
+ "failures",
21
+ "outcome"
22
+ ],
23
+ "properties": {
24
+ "schemaVersion": { "const": 1 },
25
+ "executionStatus": {
26
+ "type": "string",
27
+ "enum": ["completed", "collection-error", "command-error", "report-error"],
28
+ "description": "Process-level status. Task Pool: do not treat collection-error/command-error as ProductBug."
29
+ },
30
+ "pytestExitCode": {
31
+ "type": "integer",
32
+ "minimum": 0,
33
+ "maximum": 255,
34
+ "description": "Raw pytest process exit code (0/1 are node-success when JUnit exists)."
35
+ },
36
+ "collectionStatus": {
37
+ "type": "string",
38
+ "enum": ["ok", "error", "unknown", "skipped"]
39
+ },
40
+ "tests": {
41
+ "type": "integer",
42
+ "minimum": 0,
43
+ "description": "Task Pool: total tests = passed+failed+error+skipped"
44
+ },
45
+ "passed": { "type": "integer", "minimum": 0 },
46
+ "failed": { "type": "integer", "minimum": 0 },
47
+ "error": { "type": "integer", "minimum": 0 },
48
+ "skipped": { "type": "integer", "minimum": 0 },
49
+ "durationMs": { "type": "number", "minimum": 0 },
50
+ "junit": {
51
+ "type": "object",
52
+ "additionalProperties": false,
53
+ "required": ["relativePath", "sha256"],
54
+ "properties": {
55
+ "relativePath": {
56
+ "type": "string",
57
+ "minLength": 1,
58
+ "pattern": "^(?!/)(?!.*(?:^|/)\\.\\.(?:/|$))(?!.*(?:^|/)\\.(?:/|$))(?!.*\\\\)(?!.*//).+$",
59
+ "description": "Run-dir relative POSIX path without absolute form, backslashes, dot segments, or parent traversal (e.g. reports/backend-test-junit.xml)"
60
+ },
61
+ "sha256": {
62
+ "type": "string",
63
+ "pattern": "^[a-f0-9]{64}$"
64
+ }
65
+ }
66
+ },
67
+ "commandSummary": {
68
+ "type": "string",
69
+ "minLength": 1,
70
+ "description": "Non-secret command summary only"
71
+ },
72
+ "failures": {
73
+ "type": "array",
74
+ "description": "Truncated failure/error summaries for triage (Task Pool consumable).",
75
+ "items": {
76
+ "type": "object",
77
+ "additionalProperties": false,
78
+ "required": ["classname", "name", "message"],
79
+ "properties": {
80
+ "classname": { "type": "string", "minLength": 1 },
81
+ "name": { "type": "string", "minLength": 1 },
82
+ "message": { "type": "string", "minLength": 1 },
83
+ "kind": { "type": "string", "enum": ["failure", "error"], "default": "failure" }
84
+ }
85
+ }
86
+ },
87
+ "outcome": {
88
+ "type": "string",
89
+ "enum": [
90
+ "passed",
91
+ "completed-with-failures",
92
+ "collection-error",
93
+ "command-error",
94
+ "report-error"
95
+ ],
96
+ "description": "Authoritative shell-facing outcome for backend-test-outcome-gate-shell. Retrospective must not override."
97
+ }
98
+ }
99
+ }
@@ -0,0 +1,93 @@
1
+ # Branch Merge Report — `<source-ref>` into `<target-branch>` at `<source-short-sha>`
2
+
3
+ Date: `<YYYY-MM-DD>`
4
+ Target branch: `<target-branch>`
5
+ Target before merge: `<full SHA>`
6
+ Source: `<source-ref>@<full SHA>`
7
+ Merge base: `<full SHA>`
8
+ Merge commit: `<full SHA or follow-up commit reference>`
9
+
10
+ ## Naming Convention
11
+
12
+ Use `YYYY-MM-DD-origin-<source>-into-<target>-<source-short-sha>.md` under `docs/reports/`. Replace `/` and other path separators in branch names with `-`. The source short SHA distinguishes repeated merges on the same day.
13
+
14
+ ## Pre-merge Inspection
15
+
16
+ - Divergence: `<target-only count>` target-only commits / `<source-only count>` source-only commits.
17
+ - Dry run: `git merge-tree --write-tree --messages <target> <source>`.
18
+ - Predicted conflicts: `<count and files>`.
19
+ - Baseline verification: `<commands and results>`.
20
+
21
+ ## Incoming Capabilities
22
+
23
+ | Capability | Key files / contracts | Expected user or runtime result | Verification |
24
+ | --- | --- | --- | --- |
25
+ | `<incoming capability>` | `<paths>` | `<behavior>` | `<test/smoke>` |
26
+
27
+ ## Target-branch Capabilities to Preserve
28
+
29
+ | Capability | Key files / contracts | Preservation risk | Verification |
30
+ | --- | --- | --- | --- |
31
+ | `<target capability>` | `<paths>` | `<possible overwrite>` | `<test/smoke>` |
32
+
33
+ ## Conflict Resolution
34
+
35
+ | File | Conflict type | Incoming concern | Target concern | Resolution | Evidence |
36
+ | --- | --- | --- | --- | --- | --- |
37
+ | `<path>` | `content/rename-delete/add-add` | `<incoming behavior>` | `<target behavior>` | `<combined decision>` | `<test/diff>` |
38
+
39
+ If there were no textual conflicts, write `No textual conflicts` and continue with the automatic-merge review below.
40
+
41
+ ## Automatic-merge Semantic Review
42
+
43
+ Record overlapping or high-risk files that Git merged without conflict. These are common places for silent feature loss.
44
+
45
+ | File / area | Why high risk | Incoming behavior retained | Target behavior retained | Evidence |
46
+ | --- | --- | --- | --- | --- |
47
+ | `<path>` | `<shared imports/topology/config>` | `<incoming proof>` | `<target proof>` | `<tests/search>` |
48
+
49
+ ## Functional Impact Matrix
50
+
51
+ | Feature | Before merge | After merge | Deleted or weakened? | Notes |
52
+ | --- | --- | --- | --- | --- |
53
+ | `<feature>` | `<baseline>` | `<result>` | `No/Yes` | `<details>` |
54
+
55
+ ## Init and Update Impact
56
+
57
+ - Changed init/package surface: `<paths or none>`.
58
+ - Fresh full init result: `<result>`.
59
+ - `init check-update` result: `<result>`.
60
+ - `init update --apply-safe` recovery result: `<result>`.
61
+ - Manifest or init implementation updates made: `<details or none>`.
62
+
63
+ ## Package Surface Impact
64
+
65
+ - `npm pack --dry-run --ignore-scripts --json`: `<file count>` files.
66
+ - Added package paths: `<paths>`.
67
+ - Removed package paths: `<paths or none>`.
68
+ - Unexpected files or missing required files: `<details or none>`.
69
+
70
+ ## Verification
71
+
72
+ | Command | Result | Capability covered |
73
+ | --- | --- | --- |
74
+ | `npm run typecheck` | `<pass/fail>` | TypeScript integration |
75
+ | `npm test` | `<pass/fail>` | Full regression |
76
+ | `npm run build` | `<pass/fail>` | Build/package inputs |
77
+ | `bash scripts/check-repo.sh` | `<pass/fail>` | Governance and init surface |
78
+ | `npm run docs:build` | `<pass/fail>` | User documentation |
79
+ | `npm pack --dry-run --ignore-scripts --json` | `<pass/fail>` | Published files |
80
+
81
+ Add focused tests and real init/update/browser/CLI smokes required by the merged capabilities.
82
+
83
+ ## Residual Risk
84
+
85
+ - `<unverified external environment, performance, migration, or compatibility risk>`
86
+
87
+ ## Rollback and Investigation Pointers
88
+
89
+ - Merge commit: `<SHA>`.
90
+ - Conflict files: `<paths>`.
91
+ - First tests to run if a feature appears missing: `<commands>`.
92
+ - Safe rollback strategy: prefer `git revert -m 1 <merge-commit>` after review; do not use destructive reset on shared work.
93
+
@@ -20,6 +20,15 @@ TODO
20
20
 
21
21
  TODO
22
22
 
23
+ ## Mock / API 策略
24
+
25
+ - 接口文档或 schema:TODO
26
+ - 策略:`native | browser-intercept | request-adapter | not-needed | blocked`
27
+ - endpoint / fixture / UI 状态映射:TODO
28
+ - 显式启用方式与 production 默认关闭边界:TODO
29
+ - DAG 已固化的验证入口:TODO
30
+ - Real Integration Gap 与后端就绪后的复验:TODO
31
+
23
32
  ## 样式与设计系统映射
24
33
 
25
34
  TODO
@@ -0,0 +1,17 @@
1
+ # Failure fixture: type / build error
2
+
3
+ - **fixtureId**: `fe-fail-type-build-error`
4
+ - **triggerSignal**: `frontend-static-verify-shell` 非零(tsc 或 build 失败)
5
+ - **expectedGateBehavior**: static 失败阻断 behavior/review 成功路径;不得 closeout pass
6
+ - **repairable (M3 预标注)**: `repairable`(局部类型/导入修复)
7
+ - **Browser**: 不得宣称 Browser 证据
8
+
9
+ ## 场景
10
+
11
+ Writer 引入错误 prop 类型或缺失导出,导致 typecheck/build 失败。
12
+
13
+ ## 期望
14
+
15
+ - 新鲜 stdout/stderr 归档
16
+ - review 不得在 static 失败时 VERDICT: pass
17
+ - M3:可进入 bounded repair;M0 仅记录
@@ -0,0 +1,16 @@
1
+ # Failure fixture: unit/component test fail
2
+
3
+ - **fixtureId**: `fe-fail-unit-component-test`
4
+ - **triggerSignal**: `frontend-behavior-verify-shell` 非零(vitest 断言失败)
5
+ - **expectedGateBehavior**: behavior 失败 → review 不得 pass;closeout 阻断
6
+ - **repairable (M3 预标注)**: `repairable`
7
+ - **Browser**: not-run;不得用「应在浏览器正常」抵消
8
+
9
+ ## 场景
10
+
11
+ 组件测试期望文案/状态与实现不一致。
12
+
13
+ ## 期望
14
+
15
+ - 失败命令为生成期冻结入口
16
+ - 禁止删测 / `.skip` 伪装通过(review 应标 Important)
@@ -0,0 +1,16 @@
1
+ # Failure fixture: fixture schema drift vs API contract
2
+
3
+ - **fixtureId**: `fe-fail-fixture-schema-drift`
4
+ - **triggerSignal**: Mock fixture 字段与接口文档冲突;design gate 或 review 发现;或 mock 专项测试失败
5
+ - **expectedGateBehavior**: design request-revision 或 review request-revision;不得发明字段硬通过
6
+ - **repairable (M3 预标注)**: `non-repairable` 若根因是 spec 冲突未裁决;局部拼写且 contract 已明确时可为 repairable
7
+ - **Browser**: not-run
8
+
9
+ ## 场景
10
+
11
+ Handler 返回 `userName` 而契约为 `name`。
12
+
13
+ ## 期望
14
+
15
+ - Mock assess / design 要求 contract-aligned
16
+ - closeout 不得宣称真实联调成功
@@ -0,0 +1,16 @@
1
+ # Failure fixture: missing loading/empty/error state
2
+
3
+ - **fixtureId**: `fe-fail-missing-ui-states`
4
+ - **triggerSignal**: plan/design 或 review 发现 applicable 异步列表缺少 loading/empty/error
5
+ - **expectedGateBehavior**: first/final design 应 request-revision;若漏到实现则 review request-revision
6
+ - **repairable (M3 预标注)**: `repairable`(补状态 UI + 测试)
7
+ - **Browser**: not-run
8
+
9
+ ## 场景
10
+
11
+ 仅实现 success 列表渲染。
12
+
13
+ ## 期望
14
+
15
+ - design checklist 含 UI States
16
+ - M1 合同将强制 applicable states 映射
@@ -0,0 +1,16 @@
1
+ # Failure fixture: forbidden write / writeSet expansion attempt
2
+
3
+ - **fixtureId**: `fe-fail-forbidden-write`
4
+ - **triggerSignal**: writer 试图修改 writeSet 外路径或 forbiddenPaths(如 `src/**` 任务写入 `package.json`)
5
+ - **expectedGateBehavior**: write-guard 拒绝;run 失败或 NeedsAction;不得当成功 closeout
6
+ - **repairable (M3 预标注)**: `non-repairable`(路径/authority;不得靠 repair 扩大 writeSet)
7
+ - **Browser**: not-run
8
+
9
+ ## 场景
10
+
11
+ 实现中「顺手」改根配置或越界文档。
12
+
13
+ ## 期望
14
+
15
+ - exclusive writeSet 强制
16
+ - M0 eval 计入 `forbidden_write_count`
@@ -0,0 +1,16 @@
1
+ # Failure fixture: unapproved dependency add
2
+
3
+ - **fixtureId**: `fe-fail-unapproved-dependency`
4
+ - **triggerSignal**: diff 出现未授权 `package.json` / lockfile 依赖新增
5
+ - **expectedGateBehavior**: design 应预先拒绝;若发生则 review Important + request-revision;任务 forbidden 含 package 时 write-guard 阻断
6
+ - **repairable (M3 预标注)**: `non-repairable`(依赖策略/人工批准)
7
+ - **Browser**: not-run
8
+
9
+ ## 场景
10
+
11
+ 为方便引入大型 UI 库未在计划 Dependency Policy 批准。
12
+
13
+ ## 期望
14
+
15
+ - 计入 `unapproved_dependency_count`
16
+ - 不得以 build 通过单独放行