@tea-agent/loop-agent 0.13.0 → 0.14.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +157 -157
- package/CHANGELOG.md +73 -301
- package/README.md +338 -334
- package/bin/agent-worker.js +22 -22
- package/bin/loop-agent.js +21 -21
- package/dist/commands/cursor-prompt.js +6 -6
- package/dist/commands/init.js +505 -505
- package/dist/commands/loop-benchmark.js +11 -11
- package/dist/commands/pi-reuse-benchmark.js +16 -16
- package/dist/executors/pi-event-serializer.js +33 -11
- package/dist/sidecars/cursor-prompt/executor.js +1 -1
- package/dist/task/runtime.js +27 -27
- package/dist/worker/observe/spec-evidence.js +19 -10
- package/dist/worker/observe/static/api.js +46 -46
- package/dist/worker/observe/static/app.js +151 -150
- package/dist/worker/observe/static/constants.js +156 -148
- package/dist/worker/observe/static/copy.js +67 -67
- package/dist/worker/observe/static/dag-helpers.js +201 -172
- package/dist/worker/observe/static/dag-layout.d.ts +31 -31
- package/dist/worker/observe/static/dag-layout.js +83 -83
- package/dist/worker/observe/static/dag-model.js +72 -72
- package/dist/worker/observe/static/dom.js +122 -122
- package/dist/worker/observe/static/format-pool.d.ts +71 -0
- package/dist/worker/observe/static/format-pool.js +134 -67
- package/dist/worker/observe/static/format.js +317 -292
- package/dist/worker/observe/static/index.html +350 -308
- package/dist/worker/observe/static/kpi.js +100 -94
- package/dist/worker/observe/static/markdown-render.js +124 -0
- package/dist/worker/observe/static/relations.js +133 -133
- package/dist/worker/observe/static/router.js +93 -93
- package/dist/worker/observe/static/run-processing.js +148 -148
- package/dist/worker/observe/static/shell-chrome.js +74 -68
- package/dist/worker/observe/static/state.js +273 -267
- package/dist/worker/observe/static/styles.css +2504 -1902
- package/dist/worker/observe/static/views/batch.js +227 -227
- package/dist/worker/observe/static/views/dag-graph.js +172 -172
- package/dist/worker/observe/static/views/dag-inspector.js +530 -627
- package/dist/worker/observe/static/views/dag.js +371 -371
- package/dist/worker/observe/static/views/dashboard.js +86 -100
- package/dist/worker/observe/static/views/failures.js +143 -143
- package/dist/worker/observe/static/views/feature.js +492 -492
- package/dist/worker/observe/static/views/pool.js +708 -350
- package/dist/worker/observe/static/views/run.js +453 -453
- package/dist/worker/observe/static/views/session-timeline.js +771 -219
- package/dist/worker/observe/static/views/shell.js +7 -7
- package/dist/worker/observe/static/views/task.js +314 -314
- package/dist/worker/observe/static/views/timeline.js +163 -163
- package/dist/workflows/dag/canvas-observer.js +275 -275
- package/docs/README.md +105 -104
- package/docs/agent-dag-recovery-playbook.md +195 -195
- package/docs/agent-dag-runner.md +67 -67
- package/docs/architecture/README.md +26 -26
- package/docs/architecture/dag-execution.md +140 -140
- package/docs/architecture/evolution.md +54 -54
- package/docs/architecture/facts-and-state.md +71 -71
- package/docs/architecture/runtime-boundaries.md +191 -191
- package/docs/architecture/system-overview.md +93 -93
- package/docs/architecture/worker-and-feature.md +85 -85
- package/docs/cursor-prompt-sidecar.md +36 -36
- package/docs/decisions/README.md +18 -18
- package/docs/design/README.md +167 -167
- package/docs/development-principles.md +73 -73
- package/docs/exec-plans/README.md +6 -6
- package/docs/exec-plans/active/README.md +2 -1
- package/docs/exec-plans/completed/README.md +105 -104
- package/docs/feature-workflow.md +414 -414
- package/docs/harness-methodology-debugging.md +153 -153
- package/docs/harness-methodology-tdd.md +130 -130
- package/docs/harness-methodology-verification.md +27 -27
- package/docs/init-surface.manifest.json +307 -307
- package/docs/loop-agent-harness.md +142 -142
- package/docs/production-readiness.md +96 -96
- package/docs/progress/README.md +59 -58
- package/docs/reports/README.md +123 -119
- package/docs/skills/README.md +7 -7
- package/docs/skills/vetted-skill-registry.md +29 -29
- package/docs/templates/adr.md +60 -60
- package/docs/templates/agent-dag-authority-surface-audit.prompt.md +94 -94
- package/docs/templates/agent-dag-decision-envelope.schema.json +213 -213
- package/docs/templates/agent-dag-decision-gate-dogfood-report.md +117 -117
- package/docs/templates/agent-dag-decision-gate.prompt.md +246 -246
- package/docs/templates/agent-dag-process-supervisor.prompt.md +98 -98
- package/docs/templates/agent-dag-report.schema.json +473 -473
- package/docs/templates/agent-dag-review-verdict.prompt.md +68 -68
- package/docs/templates/agent-dag.base.json +190 -190
- package/docs/templates/agent-dag.final-verification.json +185 -185
- package/docs/templates/agent-dag.schema.json +411 -411
- package/docs/templates/agent-dag.supervised-implementation.json +620 -620
- package/docs/templates/backend-test-analysis.schema.json +44 -44
- package/docs/templates/backend-test-case-manifest.schema.json +190 -190
- package/docs/templates/backend-test-dag.classify.prompt.md +75 -75
- package/docs/templates/backend-test-dag.generate-pytest.prompt.md +204 -204
- package/docs/templates/backend-test-dag.json +559 -559
- package/docs/templates/backend-test-dag.retrospect.prompt.md +139 -139
- package/docs/templates/backend-test-dag.review-cases.prompt.md +83 -83
- package/docs/templates/backend-test-execution.schema.json +133 -133
- package/docs/templates/backend-test-result.schema.json +99 -99
- package/docs/templates/exec-plan.md +64 -64
- package/docs/templates/feature-spec.md +53 -53
- package/docs/templates/frontend-design-contract.md +42 -42
- package/docs/templates/frontend-eval/fixtures/failures/01-type-build-error.md +17 -17
- package/docs/templates/frontend-eval/fixtures/failures/02-unit-component-test-fail.md +16 -16
- package/docs/templates/frontend-eval/fixtures/failures/03-fixture-schema-drift.md +16 -16
- package/docs/templates/frontend-eval/fixtures/failures/04-missing-loading-empty-error-state.md +16 -16
- package/docs/templates/frontend-eval/fixtures/failures/05-forbidden-write-writeset-expansion.md +16 -16
- package/docs/templates/frontend-eval/fixtures/failures/06-unapproved-dependency-add.md +16 -16
- package/docs/templates/frontend-eval/fixtures/failures/07-mock-production-on.md +21 -21
- package/docs/templates/frontend-eval/fixtures/functional/01-simple-component-style.md +29 -29
- package/docs/templates/frontend-eval/fixtures/functional/02-form-validation.md +28 -28
- package/docs/templates/frontend-eval/fixtures/functional/03-list-detail-page.md +28 -28
- package/docs/templates/frontend-eval/fixtures/functional/04-api-mock.md +29 -29
- package/docs/templates/frontend-eval/fixtures/functional/05-permission-auth-gated-ui.md +27 -27
- package/docs/templates/frontend-eval/fixtures/functional/06-ssr-server-client-boundary.md +28 -28
- package/docs/templates/frontend-eval/fixtures/functional/07-shared-public-component-api.md +28 -28
- package/docs/templates/frontend-eval/fixtures/functional/08-pure-local-no-remote.md +27 -27
- package/docs/templates/frontend-eval/metrics.md +138 -138
- package/docs/templates/frontend-eval/smoke-targets.md +53 -53
- package/docs/templates/frontend-implementation-contract.schema.json +27 -27
- package/docs/templates/frontend-task-constraints.md +35 -35
- package/docs/templates/frontend-task-requirement.md +70 -70
- package/docs/templates/frontend-test-dag.generate-cases.prompt.md +5 -5
- package/docs/templates/frontend-test-dag.json +23 -23
- package/docs/templates/frontend-test-dag.retrieve-context.prompt.md +3 -3
- package/docs/templates/frontend-test-dag.retrospect.prompt.md +3 -3
- package/docs/templates/frontend-test-dag.review-cases.prompt.md +3 -3
- package/docs/templates/frontend-test-dag.review-execution.prompt.md +3 -3
- package/docs/templates/harness.schema.json +221 -221
- package/docs/templates/hybrid-dag.json +188 -188
- package/docs/templates/init-evolution-review.md +35 -35
- package/docs/templates/interactive-ui-round2-experiment.md +66 -66
- package/docs/templates/knowledge-graph-bootstrap-dag.json +118 -118
- package/docs/templates/knowledge-sync-dag.json +178 -178
- package/docs/templates/knowledge-sync-draft.schema.json +71 -71
- package/docs/templates/product-line/AGENTS.md +8 -8
- package/docs/templates/product-line/README.md +9 -9
- package/docs/templates/product-line/acceptance.yaml +14 -14
- package/docs/templates/product-line/closeout.yaml +9 -9
- package/docs/templates/product-line/design.md +13 -13
- package/docs/templates/product-line/links.md +10 -10
- package/docs/templates/product-line/requirement.md +17 -17
- package/docs/templates/product-line/task-graph.yaml +15 -15
- package/docs/templates/product-line/task.yaml +64 -64
- package/docs/templates/product-line/test-plan.md +7 -7
- package/docs/templates/production-readiness-checklist.md +57 -57
- package/docs/templates/progress-log.md +17 -17
- package/docs/templates/project-start-checklist.md +9 -9
- package/docs/templates/qa-report.md +48 -48
- package/docs/templates/sprint-contract.md +29 -29
- package/docs/templates/worker-dogfood-evidence.md +80 -80
- package/docs/templates/worker-dogfood-setup.md +68 -68
- package/docs/verification-matrix.md +70 -70
- package/examples/decision-gate-agent-dag.json +173 -173
- package/examples/example-dag.json +46 -46
- package/examples/hybrid-loop-agent-dag.json +188 -188
- package/harness.json +66 -66
- package/package.json +88 -52
- package/scripts/check-product-line-docs.sh +29 -29
- package/scripts/check-task-pool-root.sh +32 -32
- package/scripts/kb-bootstrap-init-skeleton.sh +240 -240
- package/scripts/kb-graph-incremental-prepare.mjs +386 -386
- package/scripts/kb-graph-incremental-prepare.sh +5 -5
- package/scripts/kb-graph-materialize.mjs +105 -105
- package/scripts/kb-graph-materialize.sh +4 -4
- package/scripts/kb-graph-promote.mjs +164 -164
- package/scripts/kb-graph-promote.sh +4 -4
- package/scripts/kb-query.mjs +554 -554
- package/scripts/kb-query.sh +5 -5
- package/skills/agent-worker/SKILL.md +39 -39
- package/skills/agent-worker/references/agent-worker-operator.md +60 -60
- package/skills/ai-engineering-context/SKILL.md +48 -48
- package/skills/analyze-product-dependencies/SKILL.md +67 -67
- package/skills/analyze-product-dependencies/agents/openai.yaml +4 -4
- package/skills/analyze-product-dependencies/references/api-documentation-schema.md +30 -30
- package/skills/analyze-product-dependencies/references/dependency-analysis-schema.md +28 -28
- package/skills/analyze-product-dependencies/references/example.md +76 -76
- package/skills/analyze-product-dependencies/references/forward-test-cases.md +35 -35
- package/skills/analyze-product-dependencies/references/input-contract.md +11 -11
- package/skills/analyze-product-dependencies/references/scouting-rules.md +61 -61
- package/skills/analyze-product-dependencies/scripts/test-validators.mjs +267 -267
- package/skills/analyze-product-dependencies/scripts/validate-api-documentation.mjs +101 -101
- package/skills/analyze-product-dependencies/scripts/validate-dependency-analysis.mjs +142 -142
- package/skills/analyze-product-dependencies/scripts/validate-product-requirement-input.mjs +76 -76
- package/skills/analyze-product-dependencies/scripts/validation-helpers.mjs +146 -146
- package/skills/analyze-product-requirements/SKILL.md +90 -90
- package/skills/analyze-product-requirements/agents/openai.yaml +4 -4
- package/skills/analyze-product-requirements/references/acceptance-criteria.md +91 -91
- package/skills/analyze-product-requirements/references/clarification-and-knowledge.md +56 -56
- package/skills/analyze-product-requirements/references/example.md +86 -86
- package/skills/analyze-product-requirements/references/forward-test-cases.md +66 -66
- package/skills/analyze-product-requirements/references/product-analysis-schema.md +32 -32
- package/skills/analyze-product-requirements/references/product-requirement-schema.md +33 -33
- package/skills/analyze-product-requirements/references/requirement-clarification-schema.md +35 -35
- package/skills/analyze-product-requirements/scripts/test-validators.mjs +193 -193
- package/skills/analyze-product-requirements/scripts/validate-product-analysis.mjs +69 -69
- package/skills/analyze-product-requirements/scripts/validate-product-requirement.mjs +97 -97
- package/skills/analyze-product-requirements/scripts/validate-requirement-clarification.mjs +98 -98
- package/skills/analyze-product-requirements/scripts/validation-helpers.mjs +156 -156
- package/skills/browser-tools/SKILL.md +196 -196
- package/skills/browser-tools/browser-content.js +103 -103
- package/skills/browser-tools/browser-cookies.js +35 -35
- package/skills/browser-tools/browser-eval.js +53 -53
- package/skills/browser-tools/browser-hn-scraper.js +108 -108
- package/skills/browser-tools/browser-nav.js +44 -44
- package/skills/browser-tools/browser-pick.js +162 -162
- package/skills/browser-tools/browser-screenshot.js +34 -34
- package/skills/browser-tools/browser-start.js +86 -86
- package/skills/browser-tools/package-lock.json +2556 -2556
- package/skills/browser-tools/package.json +19 -19
- package/skills/code-review-core/SKILL.md +20 -20
- package/skills/codebase-scout/SKILL.md +19 -19
- package/skills/frontend-design-review/SKILL.md +66 -66
- package/skills/frontend-design-review/references/review-checklist.md +58 -58
- package/skills/frontend-implementation/SKILL.md +49 -49
- package/skills/frontend-implementation/references/code-standards.md +32 -32
- package/skills/frontend-implementation/references/design-spec.md +46 -46
- package/skills/frontend-implementation/references/node-contracts.md +27 -27
- package/skills/frontend-review/SKILL.md +59 -59
- package/skills/frontend-review/references/review-findings.md +47 -47
- package/skills/frontend-verification/SKILL.md +53 -53
- package/skills/frontend-verification/references/verification-checklist.md +68 -68
- package/skills/grill-me/SKILL.md +10 -10
- package/skills/grill-with-docs/SKILL.md +88 -88
- package/skills/grill-with-docs/adr-format.md +47 -47
- package/skills/grill-with-docs/context-format.md +60 -60
- package/skills/init-capability-evolution/SKILL.md +70 -70
- package/skills/loop-agent/SKILL.md +151 -151
- package/skills/loop-agent/references/README.md +67 -67
- package/skills/loop-agent/references/command-reference.md +527 -527
- package/skills/loop-agent/references/docs-converge.md +126 -126
- package/skills/loop-agent/references/harness-policy.md +263 -263
- package/skills/loop-agent/references/hybrid-dag.md +243 -243
- package/skills/loop-agent/references/learned/README.md +21 -21
- package/skills/loop-agent/references/long-running-loop.md +57 -57
- package/skills/loop-agent/references/model-routing.md +36 -36
- package/skills/loop-agent/references/multi-worktree.md +54 -54
- package/skills/loop-agent/references/one-shot-runs.md +85 -85
- package/skills/loop-agent/references/orchestrator-and-interventions.md +169 -169
- package/skills/loop-agent/references/pi-prompt.md +23 -23
- package/skills/loop-agent/references/pi-subagent-assisted-mode.md +84 -84
- package/skills/loop-agent/references/post-implementation-and-patterns.md +44 -44
- package/skills/loop-agent/references/task-workflow.md +89 -89
- package/skills/loop-agent/references/verification-and-failure-handling.md +141 -141
- package/skills/playwright-cli/SKILL.md +420 -420
- package/skills/playwright-cli/references/element-attributes.md +23 -23
- package/skills/playwright-cli/references/playwright-tests.md +39 -39
- package/skills/playwright-cli/references/request-mocking.md +87 -87
- package/skills/playwright-cli/references/running-code.md +241 -241
- package/skills/playwright-cli/references/session-management.md +225 -225
- package/skills/playwright-cli/references/storage-state.md +275 -275
- package/skills/playwright-cli/references/test-generation.md +433 -433
- package/skills/playwright-cli/references/tracing.md +139 -139
- package/skills/playwright-cli/references/video-recording.md +143 -143
- package/skills/playwright-cli-case-generator/SKILL.md +74 -74
- package/skills/requesting-code-review/SKILL.md +101 -101
- package/skills/requesting-code-review/code-reviewer.md +168 -168
- package/skills/systematic-debugging/CREATION-LOG.md +119 -119
- package/skills/systematic-debugging/SKILL.md +296 -296
- package/skills/systematic-debugging/condition-based-waiting-example.ts +158 -158
- package/skills/systematic-debugging/condition-based-waiting.md +115 -115
- package/skills/systematic-debugging/defense-in-depth.md +122 -122
- package/skills/systematic-debugging/find-polluter.sh +63 -63
- package/skills/systematic-debugging/root-cause-tracing.md +169 -169
- package/skills/systematic-debugging/test-academic.md +14 -14
- package/skills/systematic-debugging/test-pressure-1.md +58 -58
- package/skills/systematic-debugging/test-pressure-2.md +68 -68
- package/skills/systematic-debugging/test-pressure-3.md +69 -69
- package/skills/test-driven-development/SKILL.md +20 -20
- package/skills/using-git-worktrees/SKILL.md +215 -215
- package/skills/verification-before-completion/SKILL.md +154 -154
- package/skills/webapp-testing/SKILL.md +19 -19
|
@@ -1,75 +1,75 @@
|
|
|
1
|
-
# Backend Test DAG Classify Prompt Template
|
|
2
|
-
|
|
3
|
-
## Purpose
|
|
4
|
-
|
|
5
|
-
Use this prompt for a **read-only classification** node: `executor: "pi"`, `role: "reviewer"`, `writePolicy: "read-only"`. The agent reads the run-owned Backend Test Result v1 artifact and returns structured failure classification JSON.
|
|
6
|
-
|
|
7
|
-
Do **not** create a new executor type. Do **not** write repository files.
|
|
8
|
-
|
|
9
|
-
## Recommended DAG Node Shape
|
|
10
|
-
|
|
11
|
-
```json
|
|
12
|
-
{
|
|
13
|
-
"id": "classify-backend-test-result-pi",
|
|
14
|
-
"depends_on": ["parse-backend-test-result-shell"],
|
|
15
|
-
"complexity": "MED",
|
|
16
|
-
"executor": "pi",
|
|
17
|
-
"role": "reviewer",
|
|
18
|
-
"writePolicy": "read-only",
|
|
19
|
-
"allowedPaths": ["**"],
|
|
20
|
-
"forbiddenPaths": [".harness/**", "artifacts/**"],
|
|
21
|
-
"outputContract": "Pure JSON classification: category in {ProductBug,TestBug,EnvFailure,ContractMismatch,FlakyTest,Unknown}, evidence[], confidence (capped), notes. No file writes.",
|
|
22
|
-
"subtask_prompt_markdown": "./backend-test-dag.classify.prompt.md"
|
|
23
|
-
}
|
|
24
|
-
```
|
|
25
|
-
|
|
26
|
-
## Prompt Body
|
|
27
|
-
|
|
28
|
-
You are the Backend Test DAG **result classifier** agent.
|
|
29
|
-
|
|
30
|
-
Your job is to classify the structured Backend Test Result v1 produced by `parse-backend-test-result-shell`. Return **exactly one JSON object**. Prefer pure JSON; a single fenced `json` block is tolerated; no trailing prose. Read-only: do not modify code, docs, artifacts, or repository files.
|
|
31
|
-
|
|
32
|
-
### Inputs (authoritative)
|
|
33
|
-
|
|
34
|
-
1. **Result v1** — `$HARNESS_DAG_RUN_DIR/contracts/backend-test-result.json` (schemaId `backend-test-result-v1`).
|
|
35
|
-
2. Optional: execute-node stdout markers (`pytestExitCode=…`, `JUnit report: …`) as secondary evidence only.
|
|
36
|
-
|
|
37
|
-
Do **not** invent pass rates or failure lists from raw logs when Result v1 is present. Counts and `failures[]` come from the result artifact only.
|
|
38
|
-
|
|
39
|
-
### Output JSON shape
|
|
40
|
-
|
|
41
|
-
```json
|
|
42
|
-
{
|
|
43
|
-
"schemaVersion": 1,
|
|
44
|
-
"category": "ProductBug",
|
|
45
|
-
"confidence": 0.0,
|
|
46
|
-
"evidence": ["result.outcome=completed-with-failures", "failures[0].name=…"],
|
|
47
|
-
"notes": "short rationale",
|
|
48
|
-
"forbiddenCategoriesHonored": ["FlakyTest"]
|
|
49
|
-
}
|
|
50
|
-
```
|
|
51
|
-
|
|
52
|
-
### Categories
|
|
53
|
-
|
|
54
|
-
| Category | When |
|
|
55
|
-
|----------|------|
|
|
56
|
-
| **ProductBug** | Assertion failures that indicate implementation/API behavior mismatch (only when collection/command/report are healthy). |
|
|
57
|
-
| **TestBug** | Broken test code, wrong expectations, bad fixtures, or collection/import errors clearly in tests. |
|
|
58
|
-
| **EnvFailure** | Missing env, service down, tooling/runtime failure, command-error. |
|
|
59
|
-
| **ContractMismatch** | Execution/analysis contract assumptions violated (wrong testRoot/mode, missing readiness). |
|
|
60
|
-
| **FlakyTest** | **Only** with multi-run historical evidence of intermittent pass/fail. |
|
|
61
|
-
| **Unknown** | Insufficient evidence. |
|
|
62
|
-
|
|
63
|
-
### Hard constraints (MUST)
|
|
64
|
-
|
|
65
|
-
1. **Single-run failure MUST NOT use `FlakyTest`.** Prefer `Unknown`, `TestBug`, or `ProductBug`.
|
|
66
|
-
2. If `executionStatus` or `outcome` is `collection-error`, `command-error`, or `report-error`, **MUST NOT** use `ProductBug`. Prefer `EnvFailure`, `TestBug`, or `Unknown`.
|
|
67
|
-
3. If `outcome=passed` with `failed=0` and `error=0`, set `category` to `Unknown` (or omit product diagnosis) and note all-pass; do not invent bugs.
|
|
68
|
-
4. `confidence` caps: ≤ `0.75` for assertion failures; ≤ `0.6` for env/collection/command/report errors; `1.0` only for all-pass with no issues.
|
|
69
|
-
5. `evidence[]` must cite concrete result fields (`outcome`, `executionStatus`, `failed`, `failures[].name`, `pytestExitCode`).
|
|
70
|
-
|
|
71
|
-
### Non-goals
|
|
72
|
-
|
|
73
|
-
- Do not rewrite Result v1.
|
|
74
|
-
- Do not decide final DAG success/failure (that is `backend-test-outcome-gate-shell`).
|
|
75
|
-
- Do not implement M3 case manifest / Task Pool auto follow-up.
|
|
1
|
+
# Backend Test DAG Classify Prompt Template
|
|
2
|
+
|
|
3
|
+
## Purpose
|
|
4
|
+
|
|
5
|
+
Use this prompt for a **read-only classification** node: `executor: "pi"`, `role: "reviewer"`, `writePolicy: "read-only"`. The agent reads the run-owned Backend Test Result v1 artifact and returns structured failure classification JSON.
|
|
6
|
+
|
|
7
|
+
Do **not** create a new executor type. Do **not** write repository files.
|
|
8
|
+
|
|
9
|
+
## Recommended DAG Node Shape
|
|
10
|
+
|
|
11
|
+
```json
|
|
12
|
+
{
|
|
13
|
+
"id": "classify-backend-test-result-pi",
|
|
14
|
+
"depends_on": ["parse-backend-test-result-shell"],
|
|
15
|
+
"complexity": "MED",
|
|
16
|
+
"executor": "pi",
|
|
17
|
+
"role": "reviewer",
|
|
18
|
+
"writePolicy": "read-only",
|
|
19
|
+
"allowedPaths": ["**"],
|
|
20
|
+
"forbiddenPaths": [".harness/**", "artifacts/**"],
|
|
21
|
+
"outputContract": "Pure JSON classification: category in {ProductBug,TestBug,EnvFailure,ContractMismatch,FlakyTest,Unknown}, evidence[], confidence (capped), notes. No file writes.",
|
|
22
|
+
"subtask_prompt_markdown": "./backend-test-dag.classify.prompt.md"
|
|
23
|
+
}
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
## Prompt Body
|
|
27
|
+
|
|
28
|
+
You are the Backend Test DAG **result classifier** agent.
|
|
29
|
+
|
|
30
|
+
Your job is to classify the structured Backend Test Result v1 produced by `parse-backend-test-result-shell`. Return **exactly one JSON object**. Prefer pure JSON; a single fenced `json` block is tolerated; no trailing prose. Read-only: do not modify code, docs, artifacts, or repository files.
|
|
31
|
+
|
|
32
|
+
### Inputs (authoritative)
|
|
33
|
+
|
|
34
|
+
1. **Result v1** — `$HARNESS_DAG_RUN_DIR/contracts/backend-test-result.json` (schemaId `backend-test-result-v1`).
|
|
35
|
+
2. Optional: execute-node stdout markers (`pytestExitCode=…`, `JUnit report: …`) as secondary evidence only.
|
|
36
|
+
|
|
37
|
+
Do **not** invent pass rates or failure lists from raw logs when Result v1 is present. Counts and `failures[]` come from the result artifact only.
|
|
38
|
+
|
|
39
|
+
### Output JSON shape
|
|
40
|
+
|
|
41
|
+
```json
|
|
42
|
+
{
|
|
43
|
+
"schemaVersion": 1,
|
|
44
|
+
"category": "ProductBug",
|
|
45
|
+
"confidence": 0.0,
|
|
46
|
+
"evidence": ["result.outcome=completed-with-failures", "failures[0].name=…"],
|
|
47
|
+
"notes": "short rationale",
|
|
48
|
+
"forbiddenCategoriesHonored": ["FlakyTest"]
|
|
49
|
+
}
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
### Categories
|
|
53
|
+
|
|
54
|
+
| Category | When |
|
|
55
|
+
|----------|------|
|
|
56
|
+
| **ProductBug** | Assertion failures that indicate implementation/API behavior mismatch (only when collection/command/report are healthy). |
|
|
57
|
+
| **TestBug** | Broken test code, wrong expectations, bad fixtures, or collection/import errors clearly in tests. |
|
|
58
|
+
| **EnvFailure** | Missing env, service down, tooling/runtime failure, command-error. |
|
|
59
|
+
| **ContractMismatch** | Execution/analysis contract assumptions violated (wrong testRoot/mode, missing readiness). |
|
|
60
|
+
| **FlakyTest** | **Only** with multi-run historical evidence of intermittent pass/fail. |
|
|
61
|
+
| **Unknown** | Insufficient evidence. |
|
|
62
|
+
|
|
63
|
+
### Hard constraints (MUST)
|
|
64
|
+
|
|
65
|
+
1. **Single-run failure MUST NOT use `FlakyTest`.** Prefer `Unknown`, `TestBug`, or `ProductBug`.
|
|
66
|
+
2. If `executionStatus` or `outcome` is `collection-error`, `command-error`, or `report-error`, **MUST NOT** use `ProductBug`. Prefer `EnvFailure`, `TestBug`, or `Unknown`.
|
|
67
|
+
3. If `outcome=passed` with `failed=0` and `error=0`, set `category` to `Unknown` (or omit product diagnosis) and note all-pass; do not invent bugs.
|
|
68
|
+
4. `confidence` caps: ≤ `0.75` for assertion failures; ≤ `0.6` for env/collection/command/report errors; `1.0` only for all-pass with no issues.
|
|
69
|
+
5. `evidence[]` must cite concrete result fields (`outcome`, `executionStatus`, `failed`, `failures[].name`, `pytestExitCode`).
|
|
70
|
+
|
|
71
|
+
### Non-goals
|
|
72
|
+
|
|
73
|
+
- Do not rewrite Result v1.
|
|
74
|
+
- Do not decide final DAG success/failure (that is `backend-test-outcome-gate-shell`).
|
|
75
|
+
- Do not implement M3 case manifest / Task Pool auto follow-up.
|
|
@@ -1,204 +1,204 @@
|
|
|
1
|
-
# Backend Test DAG Generate Pytest Prompt Template
|
|
2
|
-
|
|
3
|
-
## Purpose
|
|
4
|
-
|
|
5
|
-
Use this prompt for a **pytest code generation** node: `executor: "pi"`, `role: "implementer"`, `toolProfile: "write"`, `writePolicy: "exclusive"`. The implementer converts reviewed backend functional test cases into pytest automation code with 1:1 traceability.
|
|
6
|
-
|
|
7
|
-
Do **not** create a new executor type. This is a standard `executor: pi` writer node.
|
|
8
|
-
|
|
9
|
-
## Recommended DAG Node Shape
|
|
10
|
-
|
|
11
|
-
```json
|
|
12
|
-
{
|
|
13
|
-
"id": "generate-backend-pytest-pi",
|
|
14
|
-
"depends_on": ["review-backend-cases-gate-shell", "backend-test-execution-contract-shell"],
|
|
15
|
-
"complexity": "HIGH",
|
|
16
|
-
"executor": "pi",
|
|
17
|
-
"role": "implementer",
|
|
18
|
-
"toolProfile": "write",
|
|
19
|
-
"writePolicy": "exclusive",
|
|
20
|
-
"writeSet": ["testcase/**/test_*.py", "testcase/**/helpers/**", "testcase/**/factories/**"],
|
|
21
|
-
"allowedPaths": ["**"],
|
|
22
|
-
"forbiddenPaths": [".harness/**", "artifacts/**"],
|
|
23
|
-
"outputContract": "Pytest test files under testcase/ with 1:1 mapping to functional test case IDs; optional helpers/factories. Summary lists generated files, test function count, and any skipped cases with reasons.",
|
|
24
|
-
"subtask_prompt_markdown": "./backend-test-dag.generate-pytest.prompt.md"
|
|
25
|
-
}
|
|
26
|
-
```
|
|
27
|
-
|
|
28
|
-
## Prompt Body
|
|
29
|
-
|
|
30
|
-
You are the Backend Test DAG **pytest code generator**.
|
|
31
|
-
|
|
32
|
-
Your job is to convert reviewed test cases under `testcase/md/` into pytest automation code. Write test files under `testcase/` only. Stay within `writeSet`. Do not write root `artifacts/**`.
|
|
33
|
-
|
|
34
|
-
### Output Steps (do in order)
|
|
35
|
-
|
|
36
|
-
1. First, output a brief summary: how many files, how many test functions planned
|
|
37
|
-
2. Then write each test file under `testcase/`
|
|
38
|
-
|
|
39
|
-
### Inputs
|
|
40
|
-
|
|
41
|
-
1. **Reviewed test cases** — files under `testcase/md/` (approved by `review-backend-cases-pi` / `review-backend-cases-gate-shell`).
|
|
42
|
-
2. **Validated Backend Test Analysis v1** — run-owned `contracts/backend-test-analysis.json` from `backend-test-analysis-contract-shell`.
|
|
43
|
-
3. **Validated Backend Test Execution Contract v1** — run-owned `contracts/backend-test-execution.json` from `backend-test-execution-contract-shell` (fixtures, env *names*, `testRoot`, `targetMode`, authenticationMode).
|
|
44
|
-
4. **Target project conventions** — read `conftest.py`, `pytest.ini` / `pyproject.toml` to understand conventions, but do NOT modify them.
|
|
45
|
-
|
|
46
|
-
Do NOT re-read source documents for free-form analysis. Use only reviewed cases and the validated contracts. Use only fixture/env/testRoot facts already present in the execution contract; never invent production credentials or secret values.
|
|
47
|
-
|
|
48
|
-
### Conversion Rules
|
|
49
|
-
|
|
50
|
-
#### File Naming
|
|
51
|
-
|
|
52
|
-
- Every test file must start with `test_` prefix (e.g. `test_order.py`, `test_user_api.py`)
|
|
53
|
-
- pytest collects tests from files matching `test_*.py` or `*_test.py` — use `test_` prefix exclusively
|
|
54
|
-
- Never create test files without the `test_` prefix
|
|
55
|
-
|
|
56
|
-
#### Write Boundary
|
|
57
|
-
|
|
58
|
-
- Only **create new** files under writeSet:
|
|
59
|
-
- `testcase/**/test_*.py`
|
|
60
|
-
- `testcase/**/helpers/**` (optional pure helpers)
|
|
61
|
-
- `testcase/**/factories/**` (optional test data factories)
|
|
62
|
-
- Do NOT modify existing files: `conftest.py`, `pytest.ini`, `pyproject.toml`, `setup.cfg`, `__init__.py`, or any other framework/config file
|
|
63
|
-
- Reuse existing fixtures; if required helpers are missing, create NEW helper/factory modules under the writeSet paths above — never edit root conftest
|
|
64
|
-
- Read existing framework files to understand conventions, but treat them as immutable
|
|
65
|
-
- Do NOT write production code, `.env`, secrets, or credential files
|
|
66
|
-
|
|
67
|
-
#### Naming Conflict Resolution
|
|
68
|
-
|
|
69
|
-
- If a file with the target name already exists under `testcase/`, add a numeric suffix: `test_order.py` → `test_order_01.py` → `test_order_02.py`
|
|
70
|
-
- Never overwrite or append to existing files — each test script must be a standalone file
|
|
71
|
-
- Check for existing files before writing; if `test_<module>.py` exists, use `test_<module>_01.py`
|
|
72
|
-
|
|
73
|
-
#### 1:1 Traceability
|
|
74
|
-
|
|
75
|
-
Every functional test case ID (`BE-<MODULE>-<NNN>`) must map to exactly one pytest function:
|
|
76
|
-
|
|
77
|
-
```python
|
|
78
|
-
# testcase/md/BE-ORDER-001 → testcase/test_order.py
|
|
79
|
-
def test_BE_ORDER_001_create_order_with_valid_data():
|
|
80
|
-
"""BE-ORDER-001: Create order with valid request body."""
|
|
81
|
-
...
|
|
82
|
-
```
|
|
83
|
-
|
|
84
|
-
- Function name: `test_<CASE_ID_with_underscores>` (e.g. `test_BE_ORDER_001_...`)
|
|
85
|
-
- Docstring first line: `<CASE_ID>: <Case Title>`
|
|
86
|
-
|
|
87
|
-
#### File Organization
|
|
88
|
-
|
|
89
|
-
- All test files go under `testcase/` directory in the host project root
|
|
90
|
-
- Group test files by MODULE segment: `BE-ORDER-*` → `testcase/test_order.py`, `BE-USER-*` → `testcase/test_user.py`
|
|
91
|
-
- Follow existing project conventions for import style, fixture scope
|
|
92
|
-
|
|
93
|
-
#### Fixture Strategy
|
|
94
|
-
|
|
95
|
-
- Reuse existing project fixtures from `conftest.py` when available (read-only)
|
|
96
|
-
- Do not create or modify root fixture/configuration files (`conftest.py`, pytest.ini, …)
|
|
97
|
-
- Optional NEW helpers/factories may live under `testcase/**/helpers/**` or `testcase/**/factories/**` only
|
|
98
|
-
- Prefer `@pytest.fixture(scope="function")` for test isolation
|
|
99
|
-
- Use `@pytest.mark.parametrize` for boundary condition cases with multiple inputs
|
|
100
|
-
|
|
101
|
-
#### Test Integrity
|
|
102
|
-
|
|
103
|
-
- Tests verify implementation correctness — if a test fails, the implementation likely has a bug, not the test
|
|
104
|
-
- Do NOT weaken assertions, remove test cases, or modify test logic to make tests pass
|
|
105
|
-
- Do NOT add workarounds, skips, or try/except blocks to hide failures without explicit justification
|
|
106
|
-
- Report all failures honestly in the output; the downstream `execute-backend-pytest-shell` node captures exit codes and stdout/stderr as-is
|
|
107
|
-
|
|
108
|
-
#### Test Data Preparation Rules (MUST follow)
|
|
109
|
-
|
|
110
|
-
**When Setup is Needed**
|
|
111
|
-
|
|
112
|
-
Setup phase is REQUIRED only when test cases need pre-existing data:
|
|
113
|
-
- Query/Read APIs: need data to exist before querying
|
|
114
|
-
- Update/Delete APIs: need data to exist before modifying
|
|
115
|
-
- State transition tests: need data in specific state
|
|
116
|
-
|
|
117
|
-
Setup phase is NOT needed for:
|
|
118
|
-
- Create APIs: testing the creation itself
|
|
119
|
-
- Validation tests: testing input validation with invalid data
|
|
120
|
-
|
|
121
|
-
**Data Setup Strategy**
|
|
122
|
-
|
|
123
|
-
When setup is needed:
|
|
124
|
-
1. Use `@pytest.fixture(scope='module')` or `@pytest.fixture(scope='session')` to prepare shared test data
|
|
125
|
-
2. All test cases in the file share the same pre-constructed data
|
|
126
|
-
|
|
127
|
-
**Data Construction Priority**
|
|
128
|
-
|
|
129
|
-
1. **API-first**: Use documented APIs from `analyze-inputs-pi` / reviewed cases
|
|
130
|
-
2. **Reuse existing conftest fixtures** (read-only)
|
|
131
|
-
3. **Direct DB writes are last resort** and only via safe test-DB fixtures with rollback/isolation
|
|
132
|
-
4. If neither API nor safe DB fixture exists, **skip with an explicit gap note** — do not invent credentials or touch live data
|
|
133
|
-
|
|
134
|
-
**API Data Construction**
|
|
135
|
-
|
|
136
|
-
- Prefer the `analyze-inputs-pi` API Endpoints section and reviewed cases for method/path/fields
|
|
137
|
-
- Chain API calls only when cases document multi-step preconditions
|
|
138
|
-
- Store created resource IDs in fixtures for reuse
|
|
139
|
-
- Do **not** broadly search host route/controller trees for secrets, `.env`, private keys, or production configs
|
|
140
|
-
- Read host API definitions only when needed to resolve a field name already referenced by reviewed cases
|
|
141
|
-
|
|
142
|
-
**Database Data Construction (restricted)**
|
|
143
|
-
|
|
144
|
-
- Allowed only via existing `conftest.py` test-DB fixtures with transaction rollback or equivalent isolation
|
|
145
|
-
- Never hardcode connection strings, passwords, tokens, or cloud credentials
|
|
146
|
-
- Never target production/shared non-test databases
|
|
147
|
-
- If isolation is unclear, report the gap instead of writing DB rows
|
|
148
|
-
|
|
149
|
-
#### Assertion Rules (MUST follow)
|
|
150
|
-
|
|
151
|
-
**Positive Path (成功场景)**
|
|
152
|
-
|
|
153
|
-
MUST assert ALL of the following:
|
|
154
|
-
1. HTTP status code: as defined in API spec (e.g. 200, 201)
|
|
155
|
-
2. Response structure: key fields exist in response body
|
|
156
|
-
3. Specific values: each field equals expected value from test case
|
|
157
|
-
4. Data type: each field is correct type
|
|
158
|
-
|
|
159
|
-
**Negative Path (异常场景)**
|
|
160
|
-
|
|
161
|
-
MUST assert ALL of the following:
|
|
162
|
-
1. HTTP status code: as defined in API spec (e.g. 400, 404, 500)
|
|
163
|
-
2. Error code field: field name from API spec (e.g. code, error_code, errcode, ret)
|
|
164
|
-
3. Error message field: field name from API spec (e.g. message, msg, errmsg, error)
|
|
165
|
-
|
|
166
|
-
**Field Name Resolution**
|
|
167
|
-
|
|
168
|
-
Field names MUST come from the upstream `analyze-inputs-pi` output (API Endpoints section), NOT hardcoded. For example:
|
|
169
|
-
- If API spec defines `{"ret": 0, "msg": "success"}`, assert `response.json()["ret"]` and `response.json()["msg"]`
|
|
170
|
-
- If API spec defines `{"code": 4001, "message": "error"}`, assert `response.json()["code"]` and `response.json()["message"]`
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
#### Conditional Test Implementation (include ONLY if test cases exist)
|
|
174
|
-
|
|
175
|
-
- **Authentication tests**: implement ONLY if `testcase/md/` contains auth-related cases
|
|
176
|
-
- Use `@pytest.mark.auth` marker
|
|
177
|
-
- Test no token, expired token, invalid token, insufficient permissions, cross-user access
|
|
178
|
-
- **Timeout tests**: implement ONLY if `testcase/md/` contains timeout-related cases
|
|
179
|
-
- Use `@pytest.mark.timeout` marker
|
|
180
|
-
- Use `unittest.mock.patch` or `pytest-mock` to simulate slow responses
|
|
181
|
-
- If no such cases exist in the reviewed test cases, do NOT add these tests
|
|
182
|
-
|
|
183
|
-
#### Markers
|
|
184
|
-
|
|
185
|
-
- `@pytest.mark.positive` — happy path cases
|
|
186
|
-
- `@pytest.mark.negative` — error/exception cases
|
|
187
|
-
- `@pytest.mark.boundary` — edge cases
|
|
188
|
-
- `@pytest.mark.<MODULE>` — module-specific marker (e.g. `@pytest.mark.order`)
|
|
189
|
-
|
|
190
|
-
#### Skip Policy
|
|
191
|
-
|
|
192
|
-
If a test case cannot be automated (requires external service not mockable, requires manual verification), add it with `@pytest.mark.skip(reason="...")` and document the reason. Do not omit the function — traceability requires it exists.
|
|
193
|
-
|
|
194
|
-
### Output Shape (after summary line)
|
|
195
|
-
|
|
196
|
-
After the mandatory summary line, provide:
|
|
197
|
-
|
|
198
|
-
1. **Generated Files** — list of files written under `tests/backend/`.
|
|
199
|
-
2. **Function Mapping Table** — `| Test Case ID | Pytest Function | File | Marker |`.
|
|
200
|
-
3. **Skipped Cases** — if any, list with reason.
|
|
201
|
-
4. **Conventions Observed** — note project fixtures/config discovered and followed.
|
|
202
|
-
5. **Residual Risks** — cases that may need manual verification or environment setup.
|
|
203
|
-
|
|
204
|
-
Do not include chain-of-thought. Do not write root `artifacts/**`.
|
|
1
|
+
# Backend Test DAG Generate Pytest Prompt Template
|
|
2
|
+
|
|
3
|
+
## Purpose
|
|
4
|
+
|
|
5
|
+
Use this prompt for a **pytest code generation** node: `executor: "pi"`, `role: "implementer"`, `toolProfile: "write"`, `writePolicy: "exclusive"`. The implementer converts reviewed backend functional test cases into pytest automation code with 1:1 traceability.
|
|
6
|
+
|
|
7
|
+
Do **not** create a new executor type. This is a standard `executor: pi` writer node.
|
|
8
|
+
|
|
9
|
+
## Recommended DAG Node Shape
|
|
10
|
+
|
|
11
|
+
```json
|
|
12
|
+
{
|
|
13
|
+
"id": "generate-backend-pytest-pi",
|
|
14
|
+
"depends_on": ["review-backend-cases-gate-shell", "backend-test-execution-contract-shell"],
|
|
15
|
+
"complexity": "HIGH",
|
|
16
|
+
"executor": "pi",
|
|
17
|
+
"role": "implementer",
|
|
18
|
+
"toolProfile": "write",
|
|
19
|
+
"writePolicy": "exclusive",
|
|
20
|
+
"writeSet": ["testcase/**/test_*.py", "testcase/**/helpers/**", "testcase/**/factories/**"],
|
|
21
|
+
"allowedPaths": ["**"],
|
|
22
|
+
"forbiddenPaths": [".harness/**", "artifacts/**"],
|
|
23
|
+
"outputContract": "Pytest test files under testcase/ with 1:1 mapping to functional test case IDs; optional helpers/factories. Summary lists generated files, test function count, and any skipped cases with reasons.",
|
|
24
|
+
"subtask_prompt_markdown": "./backend-test-dag.generate-pytest.prompt.md"
|
|
25
|
+
}
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
## Prompt Body
|
|
29
|
+
|
|
30
|
+
You are the Backend Test DAG **pytest code generator**.
|
|
31
|
+
|
|
32
|
+
Your job is to convert reviewed test cases under `testcase/md/` into pytest automation code. Write test files under `testcase/` only. Stay within `writeSet`. Do not write root `artifacts/**`.
|
|
33
|
+
|
|
34
|
+
### Output Steps (do in order)
|
|
35
|
+
|
|
36
|
+
1. First, output a brief summary: how many files, how many test functions planned
|
|
37
|
+
2. Then write each test file under `testcase/`
|
|
38
|
+
|
|
39
|
+
### Inputs
|
|
40
|
+
|
|
41
|
+
1. **Reviewed test cases** — files under `testcase/md/` (approved by `review-backend-cases-pi` / `review-backend-cases-gate-shell`).
|
|
42
|
+
2. **Validated Backend Test Analysis v1** — run-owned `contracts/backend-test-analysis.json` from `backend-test-analysis-contract-shell`.
|
|
43
|
+
3. **Validated Backend Test Execution Contract v1** — run-owned `contracts/backend-test-execution.json` from `backend-test-execution-contract-shell` (fixtures, env *names*, `testRoot`, `targetMode`, authenticationMode).
|
|
44
|
+
4. **Target project conventions** — read `conftest.py`, `pytest.ini` / `pyproject.toml` to understand conventions, but do NOT modify them.
|
|
45
|
+
|
|
46
|
+
Do NOT re-read source documents for free-form analysis. Use only reviewed cases and the validated contracts. Use only fixture/env/testRoot facts already present in the execution contract; never invent production credentials or secret values.
|
|
47
|
+
|
|
48
|
+
### Conversion Rules
|
|
49
|
+
|
|
50
|
+
#### File Naming
|
|
51
|
+
|
|
52
|
+
- Every test file must start with `test_` prefix (e.g. `test_order.py`, `test_user_api.py`)
|
|
53
|
+
- pytest collects tests from files matching `test_*.py` or `*_test.py` — use `test_` prefix exclusively
|
|
54
|
+
- Never create test files without the `test_` prefix
|
|
55
|
+
|
|
56
|
+
#### Write Boundary
|
|
57
|
+
|
|
58
|
+
- Only **create new** files under writeSet:
|
|
59
|
+
- `testcase/**/test_*.py`
|
|
60
|
+
- `testcase/**/helpers/**` (optional pure helpers)
|
|
61
|
+
- `testcase/**/factories/**` (optional test data factories)
|
|
62
|
+
- Do NOT modify existing files: `conftest.py`, `pytest.ini`, `pyproject.toml`, `setup.cfg`, `__init__.py`, or any other framework/config file
|
|
63
|
+
- Reuse existing fixtures; if required helpers are missing, create NEW helper/factory modules under the writeSet paths above — never edit root conftest
|
|
64
|
+
- Read existing framework files to understand conventions, but treat them as immutable
|
|
65
|
+
- Do NOT write production code, `.env`, secrets, or credential files
|
|
66
|
+
|
|
67
|
+
#### Naming Conflict Resolution
|
|
68
|
+
|
|
69
|
+
- If a file with the target name already exists under `testcase/`, add a numeric suffix: `test_order.py` → `test_order_01.py` → `test_order_02.py`
|
|
70
|
+
- Never overwrite or append to existing files — each test script must be a standalone file
|
|
71
|
+
- Check for existing files before writing; if `test_<module>.py` exists, use `test_<module>_01.py`
|
|
72
|
+
|
|
73
|
+
#### 1:1 Traceability
|
|
74
|
+
|
|
75
|
+
Every functional test case ID (`BE-<MODULE>-<NNN>`) must map to exactly one pytest function:
|
|
76
|
+
|
|
77
|
+
```python
|
|
78
|
+
# testcase/md/BE-ORDER-001 → testcase/test_order.py
|
|
79
|
+
def test_BE_ORDER_001_create_order_with_valid_data():
|
|
80
|
+
"""BE-ORDER-001: Create order with valid request body."""
|
|
81
|
+
...
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
- Function name: `test_<CASE_ID_with_underscores>` (e.g. `test_BE_ORDER_001_...`)
|
|
85
|
+
- Docstring first line: `<CASE_ID>: <Case Title>`
|
|
86
|
+
|
|
87
|
+
#### File Organization
|
|
88
|
+
|
|
89
|
+
- All test files go under `testcase/` directory in the host project root
|
|
90
|
+
- Group test files by MODULE segment: `BE-ORDER-*` → `testcase/test_order.py`, `BE-USER-*` → `testcase/test_user.py`
|
|
91
|
+
- Follow existing project conventions for import style, fixture scope
|
|
92
|
+
|
|
93
|
+
#### Fixture Strategy
|
|
94
|
+
|
|
95
|
+
- Reuse existing project fixtures from `conftest.py` when available (read-only)
|
|
96
|
+
- Do not create or modify root fixture/configuration files (`conftest.py`, pytest.ini, …)
|
|
97
|
+
- Optional NEW helpers/factories may live under `testcase/**/helpers/**` or `testcase/**/factories/**` only
|
|
98
|
+
- Prefer `@pytest.fixture(scope="function")` for test isolation
|
|
99
|
+
- Use `@pytest.mark.parametrize` for boundary condition cases with multiple inputs
|
|
100
|
+
|
|
101
|
+
#### Test Integrity
|
|
102
|
+
|
|
103
|
+
- Tests verify implementation correctness — if a test fails, the implementation likely has a bug, not the test
|
|
104
|
+
- Do NOT weaken assertions, remove test cases, or modify test logic to make tests pass
|
|
105
|
+
- Do NOT add workarounds, skips, or try/except blocks to hide failures without explicit justification
|
|
106
|
+
- Report all failures honestly in the output; the downstream `execute-backend-pytest-shell` node captures exit codes and stdout/stderr as-is
|
|
107
|
+
|
|
108
|
+
#### Test Data Preparation Rules (MUST follow)
|
|
109
|
+
|
|
110
|
+
**When Setup is Needed**
|
|
111
|
+
|
|
112
|
+
Setup phase is REQUIRED only when test cases need pre-existing data:
|
|
113
|
+
- Query/Read APIs: need data to exist before querying
|
|
114
|
+
- Update/Delete APIs: need data to exist before modifying
|
|
115
|
+
- State transition tests: need data in specific state
|
|
116
|
+
|
|
117
|
+
Setup phase is NOT needed for:
|
|
118
|
+
- Create APIs: testing the creation itself
|
|
119
|
+
- Validation tests: testing input validation with invalid data
|
|
120
|
+
|
|
121
|
+
**Data Setup Strategy**
|
|
122
|
+
|
|
123
|
+
When setup is needed:
|
|
124
|
+
1. Use `@pytest.fixture(scope='module')` or `@pytest.fixture(scope='session')` to prepare shared test data
|
|
125
|
+
2. All test cases in the file share the same pre-constructed data
|
|
126
|
+
|
|
127
|
+
**Data Construction Priority**
|
|
128
|
+
|
|
129
|
+
1. **API-first**: Use documented APIs from `analyze-inputs-pi` / reviewed cases
|
|
130
|
+
2. **Reuse existing conftest fixtures** (read-only)
|
|
131
|
+
3. **Direct DB writes are last resort** and only via safe test-DB fixtures with rollback/isolation
|
|
132
|
+
4. If neither API nor safe DB fixture exists, **skip with an explicit gap note** — do not invent credentials or touch live data
|
|
133
|
+
|
|
134
|
+
**API Data Construction**
|
|
135
|
+
|
|
136
|
+
- Prefer the `analyze-inputs-pi` API Endpoints section and reviewed cases for method/path/fields
|
|
137
|
+
- Chain API calls only when cases document multi-step preconditions
|
|
138
|
+
- Store created resource IDs in fixtures for reuse
|
|
139
|
+
- Do **not** broadly search host route/controller trees for secrets, `.env`, private keys, or production configs
|
|
140
|
+
- Read host API definitions only when needed to resolve a field name already referenced by reviewed cases
|
|
141
|
+
|
|
142
|
+
**Database Data Construction (restricted)**
|
|
143
|
+
|
|
144
|
+
- Allowed only via existing `conftest.py` test-DB fixtures with transaction rollback or equivalent isolation
|
|
145
|
+
- Never hardcode connection strings, passwords, tokens, or cloud credentials
|
|
146
|
+
- Never target production/shared non-test databases
|
|
147
|
+
- If isolation is unclear, report the gap instead of writing DB rows
|
|
148
|
+
|
|
149
|
+
#### Assertion Rules (MUST follow)
|
|
150
|
+
|
|
151
|
+
**Positive Path (成功场景)**
|
|
152
|
+
|
|
153
|
+
MUST assert ALL of the following:
|
|
154
|
+
1. HTTP status code: as defined in API spec (e.g. 200, 201)
|
|
155
|
+
2. Response structure: key fields exist in response body
|
|
156
|
+
3. Specific values: each field equals expected value from test case
|
|
157
|
+
4. Data type: each field is correct type
|
|
158
|
+
|
|
159
|
+
**Negative Path (异常场景)**
|
|
160
|
+
|
|
161
|
+
MUST assert ALL of the following:
|
|
162
|
+
1. HTTP status code: as defined in API spec (e.g. 400, 404, 500)
|
|
163
|
+
2. Error code field: field name from API spec (e.g. code, error_code, errcode, ret)
|
|
164
|
+
3. Error message field: field name from API spec (e.g. message, msg, errmsg, error)
|
|
165
|
+
|
|
166
|
+
**Field Name Resolution**
|
|
167
|
+
|
|
168
|
+
Field names MUST come from the upstream `analyze-inputs-pi` output (API Endpoints section), NOT hardcoded. For example:
|
|
169
|
+
- If API spec defines `{"ret": 0, "msg": "success"}`, assert `response.json()["ret"]` and `response.json()["msg"]`
|
|
170
|
+
- If API spec defines `{"code": 4001, "message": "error"}`, assert `response.json()["code"]` and `response.json()["message"]`
|
|
171
|
+
|
|
172
|
+
|
|
173
|
+
#### Conditional Test Implementation (include ONLY if test cases exist)
|
|
174
|
+
|
|
175
|
+
- **Authentication tests**: implement ONLY if `testcase/md/` contains auth-related cases
|
|
176
|
+
- Use `@pytest.mark.auth` marker
|
|
177
|
+
- Test no token, expired token, invalid token, insufficient permissions, cross-user access
|
|
178
|
+
- **Timeout tests**: implement ONLY if `testcase/md/` contains timeout-related cases
|
|
179
|
+
- Use `@pytest.mark.timeout` marker
|
|
180
|
+
- Use `unittest.mock.patch` or `pytest-mock` to simulate slow responses
|
|
181
|
+
- If no such cases exist in the reviewed test cases, do NOT add these tests
|
|
182
|
+
|
|
183
|
+
#### Markers
|
|
184
|
+
|
|
185
|
+
- `@pytest.mark.positive` — happy path cases
|
|
186
|
+
- `@pytest.mark.negative` — error/exception cases
|
|
187
|
+
- `@pytest.mark.boundary` — edge cases
|
|
188
|
+
- `@pytest.mark.<MODULE>` — module-specific marker (e.g. `@pytest.mark.order`)
|
|
189
|
+
|
|
190
|
+
#### Skip Policy
|
|
191
|
+
|
|
192
|
+
If a test case cannot be automated (requires external service not mockable, requires manual verification), add it with `@pytest.mark.skip(reason="...")` and document the reason. Do not omit the function — traceability requires it exists.
|
|
193
|
+
|
|
194
|
+
### Output Shape (after summary line)
|
|
195
|
+
|
|
196
|
+
After the mandatory summary line, provide:
|
|
197
|
+
|
|
198
|
+
1. **Generated Files** — list of files written under `tests/backend/`.
|
|
199
|
+
2. **Function Mapping Table** — `| Test Case ID | Pytest Function | File | Marker |`.
|
|
200
|
+
3. **Skipped Cases** — if any, list with reason.
|
|
201
|
+
4. **Conventions Observed** — note project fixtures/config discovered and followed.
|
|
202
|
+
5. **Residual Risks** — cases that may need manual verification or environment setup.
|
|
203
|
+
|
|
204
|
+
Do not include chain-of-thought. Do not write root `artifacts/**`.
|