prizmkit 1.1.153 → 1.1.155
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bundled/VERSION.json +3 -3
- package/bundled/adapters/codex/skill-adapter.js +1 -1
- package/bundled/adapters/pi/skill-adapter.js +2 -1
- package/bundled/dev-pipeline/.env.example +2 -6
- package/bundled/dev-pipeline/README.md +120 -113
- package/bundled/dev-pipeline/assets/skill-subagent-integration.md +1 -1
- package/bundled/dev-pipeline/prizmkit_runtime/checkpoint_state.py +355 -15
- package/bundled/dev-pipeline/prizmkit_runtime/cli.py +189 -110
- package/bundled/dev-pipeline/prizmkit_runtime/commands.py +146 -111
- package/bundled/dev-pipeline/prizmkit_runtime/daemon.py +4 -20
- package/bundled/dev-pipeline/prizmkit_runtime/gitops.py +48 -14
- package/bundled/dev-pipeline/prizmkit_runtime/interoperability.py +0 -1
- package/bundled/dev-pipeline/prizmkit_runtime/paths.py +0 -3
- package/bundled/dev-pipeline/prizmkit_runtime/reset.py +122 -15
- package/bundled/dev-pipeline/prizmkit_runtime/reset_preserve.py +1 -1
- package/bundled/dev-pipeline/prizmkit_runtime/runner_classification.py +50 -2
- package/bundled/dev-pipeline/prizmkit_runtime/runner_models.py +1 -10
- package/bundled/dev-pipeline/prizmkit_runtime/runner_prompts.py +3 -3
- package/bundled/dev-pipeline/prizmkit_runtime/runners.py +139 -74
- package/bundled/dev-pipeline/prizmkit_runtime/runtime_commit.py +482 -0
- package/bundled/dev-pipeline/prizmkit_runtime/status.py +106 -1
- package/bundled/dev-pipeline/prizmkit_runtime/test_result.py +64 -11
- package/bundled/dev-pipeline/scripts/generate-bootstrap-prompt.py +145 -367
- package/bundled/dev-pipeline/scripts/generate-bugfix-prompt.py +91 -62
- package/bundled/dev-pipeline/scripts/generate-refactor-prompt.py +105 -69
- package/bundled/dev-pipeline/scripts/init-bugfix-pipeline.py +2 -2
- package/bundled/dev-pipeline/scripts/prompt_framework.py +36 -41
- package/bundled/dev-pipeline/scripts/update-bug-status.py +197 -71
- package/bundled/dev-pipeline/scripts/update-checkpoint.py +77 -16
- package/bundled/dev-pipeline/scripts/update-feature-status.py +46 -121
- package/bundled/dev-pipeline/scripts/update-refactor-status.py +46 -123
- package/bundled/dev-pipeline/scripts/utils.py +119 -0
- package/bundled/dev-pipeline/templates/bootstrap-prompt.md +22 -27
- package/bundled/dev-pipeline/templates/bootstrap-tier3.md +71 -104
- package/bundled/dev-pipeline/templates/bug-fix-list-schema.json +3 -2
- package/bundled/dev-pipeline/templates/bugfix-bootstrap-prompt.md +59 -111
- package/bundled/dev-pipeline/templates/refactor-bootstrap-prompt.md +37 -72
- package/bundled/dev-pipeline/templates/sections/artifact-manifest.md +39 -0
- package/bundled/dev-pipeline/templates/sections/bugfix-critical-paths.md +3 -1
- package/bundled/dev-pipeline/templates/sections/bugfix-phase-commit-report.md +8 -25
- package/bundled/dev-pipeline/templates/sections/bugfix-phase-diagnose-plan.md +3 -3
- package/bundled/dev-pipeline/templates/sections/bugfix-phase-implement.md +2 -2
- package/bundled/dev-pipeline/templates/sections/bugfix-phase-review.md +3 -3
- package/bundled/dev-pipeline/templates/sections/bugfix-phase-test.md +2 -2
- package/bundled/dev-pipeline/templates/sections/bugfix-reminders.md +4 -4
- package/bundled/dev-pipeline/templates/sections/bugfix-session-context.md +0 -1
- package/bundled/dev-pipeline/templates/sections/checkpoint-system.md +7 -4
- package/bundled/dev-pipeline/templates/sections/context-budget-rules.md +14 -27
- package/bundled/dev-pipeline/templates/sections/feature-completion-summary.md +20 -0
- package/bundled/dev-pipeline/templates/sections/phase-commit-full.md +8 -71
- package/bundled/dev-pipeline/templates/sections/phase-implement-full.md +7 -7
- package/bundled/dev-pipeline/templates/sections/phase-prizmkit-test.md +3 -3
- package/bundled/dev-pipeline/templates/sections/phase-review-full.md +4 -4
- package/bundled/dev-pipeline/templates/sections/phase-specify-plan-full.md +5 -17
- package/bundled/dev-pipeline/templates/sections/refactor-critical-paths.md +3 -1
- package/bundled/dev-pipeline/templates/sections/refactor-phase-commit-report.md +10 -23
- package/bundled/dev-pipeline/templates/sections/refactor-phase-implement.md +2 -2
- package/bundled/dev-pipeline/templates/sections/refactor-phase-plan.md +2 -2
- package/bundled/dev-pipeline/templates/sections/refactor-phase-review.md +3 -3
- package/bundled/dev-pipeline/templates/sections/refactor-phase-test.md +2 -2
- package/bundled/dev-pipeline/templates/sections/refactor-reminders.md +7 -7
- package/bundled/dev-pipeline/templates/sections/refactor-session-context.md +0 -1
- package/bundled/dev-pipeline/templates/sections/retrospective.md +12 -0
- package/bundled/dev-pipeline/templates/sections/runtime-commit-handoff.md +35 -0
- package/bundled/dev-pipeline/templates/session-status-schema.json +45 -62
- package/bundled/dev-pipeline/tests/test_auto_skip.py +111 -32
- package/bundled/dev-pipeline/tests/test_checkpoint_state.py +617 -30
- package/bundled/dev-pipeline/tests/test_generate_bootstrap_prompt.py +262 -194
- package/bundled/dev-pipeline/tests/test_generate_bugfix_prompt.py +62 -23
- package/bundled/dev-pipeline/tests/test_generate_refactor_prompt.py +76 -28
- package/bundled/dev-pipeline/tests/test_python_runner_parity.py +281 -206
- package/bundled/dev-pipeline/tests/test_recovery_workflow.py +322 -0
- package/bundled/dev-pipeline/tests/test_reset_modes.py +935 -0
- package/bundled/dev-pipeline/tests/test_reset_preserve.py +5 -7
- package/bundled/dev-pipeline/tests/test_runtime_commit.py +608 -0
- package/bundled/dev-pipeline/tests/test_unified_cli.py +617 -188
- package/bundled/skills/_metadata.json +9 -9
- package/bundled/skills/app-planner/SKILL.md +3 -3
- package/bundled/skills/app-planner/references/generated-plan-review.md +2 -1
- package/bundled/skills/app-planner/references/rules/backend/derivation-rules.md +56 -64
- package/bundled/skills/app-planner/references/rules/backend/fixed-rules.md +85 -240
- package/bundled/skills/app-planner/references/rules/backend/question-bank.md +25 -25
- package/bundled/skills/app-planner/references/rules/backend/question-manifest.json +16 -7
- package/bundled/skills/app-planner/references/rules/backend/template.md +74 -92
- package/bundled/skills/app-planner/references/rules/database/derivation-rules.md +93 -93
- package/bundled/skills/app-planner/references/rules/database/fixed-rules.md +66 -173
- package/bundled/skills/app-planner/references/rules/database/question-bank.md +31 -26
- package/bundled/skills/app-planner/references/rules/database/question-manifest.json +13 -4
- package/bundled/skills/app-planner/references/rules/database/template.md +76 -98
- package/bundled/skills/app-planner/references/rules/frontend/derivation-rules.md +68 -95
- package/bundled/skills/app-planner/references/rules/frontend/fixed-rules.md +93 -166
- package/bundled/skills/app-planner/references/rules/frontend/question-bank.md +46 -36
- package/bundled/skills/app-planner/references/rules/frontend/question-manifest.json +11 -2
- package/bundled/skills/app-planner/references/rules/frontend/template.md +79 -257
- package/bundled/skills/app-planner/references/rules/mobile/derivation-rules.md +91 -99
- package/bundled/skills/app-planner/references/rules/mobile/fixed-rules.md +73 -246
- package/bundled/skills/app-planner/references/rules/mobile/question-bank.md +17 -17
- package/bundled/skills/app-planner/references/rules/mobile/question-manifest.json +17 -2
- package/bundled/skills/app-planner/references/rules/mobile/template.md +73 -85
- package/bundled/skills/app-planner/references/rules-configuration.md +133 -65
- package/bundled/skills/app-planner/scripts/validate-rules-configuration.py +291 -0
- package/bundled/skills/bug-planner/references/schema-validation.md +1 -1
- package/bundled/skills/bug-planner/scripts/validate-bug-list.py +1 -1
- package/bundled/skills/bugfix-pipeline-launcher/SKILL.md +21 -13
- package/bundled/skills/feature-pipeline-launcher/SKILL.md +22 -14
- package/bundled/skills/prizmkit/SKILL.md +17 -18
- package/bundled/skills/prizmkit/references/workflow-state-protocol.md +22 -18
- package/bundled/skills/prizmkit-code-review/SKILL.md +22 -58
- package/bundled/skills/prizmkit-code-review/references/independent-code-review.md +6 -6
- package/bundled/skills/prizmkit-code-review/references/review-report-template.md +2 -2
- package/bundled/skills/prizmkit-committer/SKILL.md +82 -132
- package/bundled/skills/prizmkit-deploy/SKILL.md +27 -11
- package/bundled/skills/prizmkit-deploy/assets/deploy-document-template.md +89 -0
- package/bundled/skills/prizmkit-deploy/references/ci-cd-workflows.md +44 -14
- package/bundled/skills/prizmkit-deploy/references/cloud-platform-deploy.md +19 -3
- package/bundled/skills/prizmkit-deploy/references/database-setup.md +22 -12
- package/bundled/skills/prizmkit-deploy/references/deploy-config-schema.md +3 -1
- package/bundled/skills/prizmkit-deploy/references/deploy-history-schema.md +22 -6
- package/bundled/skills/prizmkit-deploy/references/deploy-metadata-schema.json +49 -0
- package/bundled/skills/prizmkit-deploy/references/live-validation-notes.md +1 -1
- package/bundled/skills/prizmkit-deploy/references/pending-input-schema.json +164 -0
- package/bundled/skills/prizmkit-deploy/references/ssh-adapter-flow.md +9 -8
- package/bundled/skills/prizmkit-deploy/references/ssh-execution-flow.md +6 -6
- package/bundled/skills/prizmkit-implement/SKILL.md +30 -66
- package/bundled/skills/prizmkit-implement/references/implementation-subagent-procedure.md +4 -4
- package/bundled/skills/prizmkit-init/SKILL.md +4 -0
- package/bundled/skills/prizmkit-plan/SKILL.md +21 -64
- package/bundled/skills/prizmkit-plan/references/artifact-identity.md +38 -0
- package/bundled/skills/prizmkit-plan/references/independent-plan-review.md +4 -4
- package/bundled/skills/prizmkit-plan/references/review-plan-spec-loop.md +5 -5
- package/bundled/skills/prizmkit-prizm-docs/SKILL.md +11 -1
- package/bundled/skills/prizmkit-retrospective/SKILL.md +63 -77
- package/bundled/skills/prizmkit-retrospective/references/knowledge-injection-steps.md +3 -3
- package/bundled/skills/prizmkit-retrospective/references/retrospective-result-schema.json +138 -0
- package/bundled/skills/prizmkit-retrospective/references/structural-sync-steps.md +17 -23
- package/bundled/skills/prizmkit-test/SKILL.md +64 -51
- package/bundled/skills/prizmkit-test/references/boundary-contract-and-test-double-guidance.md +206 -0
- package/bundled/skills/prizmkit-test/references/independent-test-review.md +17 -9
- package/bundled/skills/prizmkit-test/references/test-coverage-model.md +45 -21
- package/bundled/skills/prizmkit-test/references/test-report-template.md +20 -13
- package/bundled/skills/prizmkit-workflow/SKILL.md +35 -37
- package/bundled/skills/prizmkit-workflow/references/artifact-identity.md +34 -0
- package/bundled/skills/prizmkit-workflow/references/workflow-state-protocol.md +22 -18
- package/bundled/skills/recovery-workflow/SKILL.md +9 -6
- package/bundled/skills/recovery-workflow/evals/evals.json +2 -2
- package/bundled/skills/recovery-workflow/references/detection.md +7 -6
- package/bundled/skills/recovery-workflow/scripts/detect-recovery-state.py +16 -6
- package/bundled/skills/refactor-pipeline-launcher/SKILL.md +21 -13
- package/bundled/templates/project-memory-template.md +19 -11
- package/package.json +1 -1
- package/src/scaffold.js +33 -22
- package/bundled/dev-pipeline/prizmkit_runtime/runner_recovery.py +0 -228
- package/bundled/dev-pipeline/scripts/generate-recovery-prompt.py +0 -767
- package/bundled/dev-pipeline/templates/sections/bugfix-artifacts.md +0 -16
- package/bundled/dev-pipeline/templates/sections/bugfix-session-status.md +0 -31
- package/bundled/dev-pipeline/templates/sections/critical-paths-agent.md +0 -7
- package/bundled/dev-pipeline/templates/sections/critical-paths-lite.md +0 -7
- package/bundled/dev-pipeline/templates/sections/directory-convention-agent.md +0 -9
- package/bundled/dev-pipeline/templates/sections/directory-convention-lite.md +0 -6
- package/bundled/dev-pipeline/templates/sections/headless-commit-authorization.md +0 -15
- package/bundled/dev-pipeline/templates/sections/phase-commit.md +0 -61
- package/bundled/dev-pipeline/templates/sections/phase-context-snapshot-agent-suffix.md +0 -24
- package/bundled/dev-pipeline/templates/sections/phase-context-snapshot-lite-suffix.md +0 -12
- package/bundled/dev-pipeline/templates/sections/phase-implement-agent.md +0 -44
- package/bundled/dev-pipeline/templates/sections/phase-implement-lite.md +0 -25
- package/bundled/dev-pipeline/templates/sections/phase-plan-agent.md +0 -20
- package/bundled/dev-pipeline/templates/sections/phase-plan-lite.md +0 -20
- package/bundled/dev-pipeline/templates/sections/phase-review-agent.md +0 -20
- package/bundled/dev-pipeline/templates/sections/refactor-artifacts.md +0 -17
- package/bundled/dev-pipeline/templates/sections/refactor-session-status.md +0 -28
- package/bundled/dev-pipeline/templates/sections/test-failure-recovery-agent.md +0 -52
- package/bundled/dev-pipeline/templates/sections/test-failure-recovery-lite.md +0 -52
- package/bundled/skills/prizmkit-code-review/references/workflow-state-protocol.md +0 -174
- package/bundled/skills/prizmkit-committer/references/workflow-state-protocol.md +0 -174
- package/bundled/skills/prizmkit-implement/references/workflow-state-protocol.md +0 -174
- package/bundled/skills/prizmkit-plan/references/workflow-state-protocol.md +0 -174
- package/bundled/skills/prizmkit-retrospective/references/workflow-state-protocol.md +0 -174
- package/bundled/skills/prizmkit-test/references/external-contract-mock-guidance.md +0 -119
|
@@ -1,13 +1,13 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: "prizmkit-test"
|
|
3
|
-
description: "
|
|
3
|
+
description: "Test one supplied current change with project-native affected-module and Regression Ring coverage. Use for contract testing, test-double fidelity, consumer/provider compatibility, browser and Full-stack E2E, persistence re-entry, and cross-boundary integration; returns TEST_PASS, TEST_NEEDS_FIXES, or TEST_BLOCKED with stage-owned report/result artifacts. (project)"
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# PrizmKit Test
|
|
7
7
|
|
|
8
8
|
`/prizmkit-test` is the independent testing stage for the current workspace. It understands the affected business behavior, fills project-native test gaps, executes the tests, repairs valid failures, reviews the final tests, and reports a truthful testing-domain result.
|
|
9
9
|
|
|
10
|
-
It does not build an evidence package, prove historical TDD order, validate a previous checkout
|
|
10
|
+
It does not build an evidence package, prove historical TDD order, or validate a previous checkout.
|
|
11
11
|
|
|
12
12
|
## Atomic Skill Boundary
|
|
13
13
|
|
|
@@ -26,25 +26,21 @@ resolve current change and requirement context
|
|
|
26
26
|
→ return TEST_PASS | TEST_NEEDS_FIXES | TEST_BLOCKED
|
|
27
27
|
```
|
|
28
28
|
|
|
29
|
-
After writing its result, this
|
|
30
|
-
|
|
31
|
-
- invoke implementation, Code Review, retrospective, or committer;
|
|
32
|
-
- update a workflow checkpoint or pipeline runtime state;
|
|
33
|
-
- classify an AI CLI session as success, crash, retry, or continuation;
|
|
34
|
-
- decide whether a workflow terminates or resumes;
|
|
35
|
-
- call a testing-domain non-pass result a runtime failure.
|
|
29
|
+
After writing its result, this Skill stops. It owns only testing-local source/test repairs and its two terminal artifacts. It does not invoke another Skill.
|
|
36
30
|
|
|
37
31
|
## Inputs and Outputs
|
|
38
32
|
|
|
39
33
|
| Parameter | Required | Description |
|
|
40
34
|
|---|---|---|
|
|
41
35
|
| `scope` | No | `full-project`, `module:<name>`, `feature:<slug>`, or `this-change`. Defaults to `this-change` when current change context exists, otherwise `full-project`. |
|
|
42
|
-
| `artifact_dir` |
|
|
36
|
+
| `artifact_dir` | Yes | Exact caller-supplied project-relative directory for the two terminal artifacts and any available specification, plan, and behavior context. Reuse it unchanged. |
|
|
43
37
|
| `changed_files` | No | Explicit current changed paths. Highest-priority scope locator. |
|
|
44
38
|
| `diff_base` | No | Git comparison base when explicit changed paths are absent. |
|
|
45
39
|
| `test_commands` | No | Project-native commands when repository conventions are ambiguous. |
|
|
46
40
|
|
|
47
|
-
|
|
41
|
+
Resolve `artifact_dir` before Test execution. In interactive use, ask the user to confirm an exact project-relative directory when omitted. In non-interactive use, a missing or unsafe directory is a response-only `TEST_BLOCKED` precondition failure: create no fallback directory or artifact and report that the caller must retry with `artifact_dir`. This precondition rejection is the sole case where terminal files cannot be written because their required identity is absent.
|
|
42
|
+
|
|
43
|
+
For every invocation that passes input resolution, write exactly two terminal artifacts in the supplied `artifact_dir`:
|
|
48
44
|
|
|
49
45
|
```text
|
|
50
46
|
{artifact_dir}/
|
|
@@ -52,16 +48,16 @@ Write exactly two terminal artifacts in `artifact_dir` when it is available; oth
|
|
|
52
48
|
└── test-result.json
|
|
53
49
|
```
|
|
54
50
|
|
|
55
|
-
Do not create `.prizmkit/test/evidence/`, manifests, hashes, attestations, package
|
|
51
|
+
Do not create `.prizmkit/test/evidence/`, manifests, hashes, attestations, package state records, or test-internal checkpoints.
|
|
56
52
|
|
|
57
53
|
## Context Loading and Scope Resolution
|
|
58
54
|
|
|
59
55
|
1. Reuse current requirement and workspace context already loaded by the Main Agent.
|
|
60
56
|
2. Load `root.prizm`, the relevant module indexes, and the applicable detail docs before modifying source or tests.
|
|
61
57
|
3. Inspect current staged, unstaged, untracked, deleted, and renamed paths. Changed lines locate impact but never define completeness.
|
|
62
|
-
4. Read the available specification, acceptance criteria, plan, and
|
|
58
|
+
4. Read the available specification, acceptance criteria, plan, and caller-supplied behavior context only as needed to establish expected behavior.
|
|
63
59
|
5. Inspect manifests, runner configuration, CI conventions, existing test assertions, fixtures, fakes, mocks, contracts, and coverage support.
|
|
64
|
-
6. If necessary scope cannot be determined safely, report the precise missing input and return `TEST_BLOCKED
|
|
60
|
+
6. If necessary scope cannot be determined safely, report the precise missing input and return `TEST_BLOCKED` in the resolved artifact pair.
|
|
65
61
|
|
|
66
62
|
Do not infer the active requirement solely from the name of an old artifact directory. Do not use a historical worktree, second checkout, mutation proof, baseline-failure proof, or test overlay.
|
|
67
63
|
|
|
@@ -101,29 +97,29 @@ Use the lowest layer that proves a property without losing relevant composition,
|
|
|
101
97
|
|
|
102
98
|
```text
|
|
103
99
|
focused/unit
|
|
104
|
-
→ module/component
|
|
105
|
-
→ contract
|
|
106
|
-
→
|
|
100
|
+
→ module/component or UI component
|
|
101
|
+
→ consumer/provider contract or integration
|
|
102
|
+
→ Mock Browser or Full-stack E2E according to actual composition
|
|
107
103
|
→ complete affected-module regression
|
|
108
104
|
→ Regression Ring verification
|
|
109
105
|
```
|
|
110
106
|
|
|
111
|
-
Do not mechanically test every private function. Add direct focused tests for critical low-level logic when complexity, boundaries, risk, or failure localization justify them; otherwise prove it through public behavior.
|
|
107
|
+
Browser execution alone is not Full-stack E2E. Do not mechanically test every private function. Add direct focused tests for critical low-level logic when complexity, boundaries, risk, or failure localization justify them; otherwise prove it through public behavior.
|
|
112
108
|
|
|
113
109
|
## Test Construction
|
|
114
110
|
|
|
115
|
-
Read `${SKILL_DIR}/references/test-coverage-model.md` and,
|
|
111
|
+
Read `${SKILL_DIR}/references/test-coverage-model.md` and, whenever serialization, a producer-consumer boundary, or any test double replaces a producer or consumer, `${SKILL_DIR}/references/boundary-contract-and-test-double-guidance.md`.
|
|
116
112
|
|
|
117
|
-
Expected behavior follows
|
|
113
|
+
Expected behavior follows boundary-specific authority:
|
|
118
114
|
|
|
119
|
-
1. confirmed specification;
|
|
120
|
-
2. machine-readable contract;
|
|
121
|
-
3.
|
|
122
|
-
4.
|
|
123
|
-
5.
|
|
124
|
-
6. current implementation.
|
|
115
|
+
1. confirmed specification and acceptance criteria;
|
|
116
|
+
2. machine-readable wire contract;
|
|
117
|
+
3. traceably generated type, client, server stub, or fixture;
|
|
118
|
+
4. provider raw-wire contract evidence or authorized isolated observation;
|
|
119
|
+
5. provider implementation, locked SDK behavior, and matching official documentation;
|
|
120
|
+
6. trusted existing tests, callers and consumers, consumer-local types/fixtures/mocks, and other current implementation evidence.
|
|
125
121
|
|
|
126
|
-
|
|
122
|
+
Lower-authority material cannot override higher-authority truth or prove a property outside its boundary. Conflicts that cannot be resolved are `TEST_BLOCKED`; do not preserve a possible implementation defect as a characterization test.
|
|
127
123
|
|
|
128
124
|
Tests must:
|
|
129
125
|
|
|
@@ -135,6 +131,20 @@ Tests must:
|
|
|
135
131
|
- avoid vacuous assertions and assertions that merely duplicate implementation steps;
|
|
136
132
|
- never delete or weaken a valid existing test just to obtain green output.
|
|
137
133
|
|
|
134
|
+
## Boundary Contract and Composition Gate
|
|
135
|
+
|
|
136
|
+
A test cannot prove a property that depends on production composition it replaces. Before test construction and again before `TEST_PASS`:
|
|
137
|
+
|
|
138
|
+
1. Inventory every verdict-relevant mock, fake, fixture, route interception, mock server, container, emulator, or isolated service that crosses or replaces a producer-consumer boundary.
|
|
139
|
+
2. Record its producer, consumer, boundary/protocol, contract authority, fidelity proof, composition removed, property not proven, composition-preserving test, and remaining risk.
|
|
140
|
+
3. Treat consumer-local types, fixtures, and mocks as expectations, not provider truth, unless they are generated from or validated against the authoritative boundary contract.
|
|
141
|
+
4. Classify tests by actual composition. An API-intercepted browser test is a Mock Browser Test, not Full-stack E2E.
|
|
142
|
+
5. Require risk-relevant Provider Contract Tests to inspect raw wire payload before provider-owned deserialization.
|
|
143
|
+
6. For any verdict-capable serialization or multi-layer risk in removed composition, require provider/consumer contract evidence, a composition-preserving integration test, or a Full-stack path appropriate to the risk and keeping the defect-bearing boundary real.
|
|
144
|
+
7. For state-changing workflows, verify commit point, cancellation, pending/duplicate protection, failure recovery, reload or re-entry, retry/conflict/idempotency, and downstream consumption.
|
|
145
|
+
|
|
146
|
+
If required composition-preserving evidence cannot run in a proven safe isolated environment, return `TEST_BLOCKED`. Never substitute a lower-layer green test or downgrade the proof requirement.
|
|
147
|
+
|
|
138
148
|
Reuse an adequate existing framework. Add only the smallest necessary project-native infrastructure when a required property cannot otherwise be expressed.
|
|
139
149
|
|
|
140
150
|
Minimal behavior-preserving testability seams are allowed, including dependency injection through an existing abstraction, pure-function extraction, an internal adapter, or controllable clock/randomness/state. Classify any production change under the repair boundary below.
|
|
@@ -145,13 +155,13 @@ The target project owns its runner, command, working directory, timeout, concurr
|
|
|
145
155
|
|
|
146
156
|
During construction and repair, run the smallest relevant tests first. Before a pass decision, run all required layers, the complete affected-module regression, the Regression Ring, and project-wide regression when concrete coupling or project convention requires it.
|
|
147
157
|
|
|
148
|
-
A nonzero execution is
|
|
158
|
+
A nonzero execution is testing feedback, not an immediate terminal result. Classify it:
|
|
149
159
|
|
|
150
160
|
| Classification | Action |
|
|
151
161
|
|---|---|
|
|
152
162
|
| `test-defect` | Repair the test, fixture, mock, import, syntax, or test infrastructure. |
|
|
153
163
|
| `local-production-defect` | Repair the proven internal implementation defect when public and cross-module contracts remain unchanged. |
|
|
154
|
-
| `high-risk-production-defect` |
|
|
164
|
+
| `high-risk-production-defect` | Do not repair it in this invocation; record the proven defect and return `TEST_NEEDS_FIXES`. |
|
|
155
165
|
| `environment-unavailable` | Return `TEST_BLOCKED` with the unavailable safe prerequisite. |
|
|
156
166
|
| `truth-unresolved` | Return `TEST_BLOCKED`; never guess the assertion. |
|
|
157
167
|
| `flaky-or-unreliable` | Diagnose within budget; otherwise return `TEST_BLOCKED`. |
|
|
@@ -165,9 +175,9 @@ Never retry until green or weaken scope/assertions to manufacture a pass.
|
|
|
165
175
|
|
|
166
176
|
## Production Repair Boundary
|
|
167
177
|
|
|
168
|
-
A repair may close inside this
|
|
178
|
+
A repair may close inside this Skill only when evidence establishes that it is limited to private/internal implementation, a behavior-preserving algorithm correction, test infrastructure, a behavior-preserving testability seam, or local error handling with unchanged public obligations.
|
|
169
179
|
|
|
170
|
-
|
|
180
|
+
Do not perform the repair in this invocation when it affects or may affect:
|
|
171
181
|
|
|
172
182
|
- public API or observable public behavior;
|
|
173
183
|
- schema, migration, serialization, or generated protocol shape;
|
|
@@ -178,7 +188,7 @@ Set `review_required=true` and `review_scope=delta` when a repair affects or may
|
|
|
178
188
|
- compatibility guarantees;
|
|
179
189
|
- any change whose locality cannot be established confidently.
|
|
180
190
|
|
|
181
|
-
Record
|
|
191
|
+
Record the defect evidence and required correction, return `TEST_NEEDS_FIXES`, and stop. Do not prescribe or execute work outside this testing invocation.
|
|
182
192
|
|
|
183
193
|
## Mandatory Main-Agent Test Review
|
|
184
194
|
|
|
@@ -200,7 +210,10 @@ Every complete round checks:
|
|
|
200
210
|
- critical low-level boundary logic;
|
|
201
211
|
- incorrect truth precedence or expected behavior;
|
|
202
212
|
- weak, vacuous, implementation-only, nondeterministic, or flaky assertions;
|
|
203
|
-
-
|
|
213
|
+
- complete boundary/test-double inventory, boundary-appropriate authority, fidelity proof, and consumer-authored ideal fixtures not being treated as provider truth;
|
|
214
|
+
- Provider raw-wire assertions, test-layer labels matching actual composition, and Browser Tests with intercepted critical APIs not being called Full-stack E2E;
|
|
215
|
+
- any test that would still pass with the target defect, any verdict-capable composition removed without preserving evidence, and missing negative side-effect assertions;
|
|
216
|
+
- state-changing workflow commit, cancellation, failure recovery, reload/re-entry, retry/conflict/idempotency, and repeated-operation coverage;
|
|
204
217
|
- unjustified layer omissions or tests absent from actual native execution;
|
|
205
218
|
- production repairs outside this skill's safe boundary.
|
|
206
219
|
|
|
@@ -244,17 +257,17 @@ Only after Main-Agent review converges, read `${SKILL_DIR}/references/independen
|
|
|
244
257
|
- If response five still contains unresolved truth, input, or safety, return `TEST_BLOCKED`.
|
|
245
258
|
- Never claim `TEST_PASS` for a final mutation that was not independently rechecked when independent review had already begun, unless the reference's strict downgrade rule establishes the permitted Main-Agent fallback.
|
|
246
259
|
|
|
247
|
-
##
|
|
260
|
+
## Boundary Contracts and External Safety
|
|
248
261
|
|
|
249
|
-
Read `${SKILL_DIR}/references/
|
|
262
|
+
Read `${SKILL_DIR}/references/boundary-contract-and-test-double-guidance.md` for internal and external APIs, cross-module endpoints, persistence, queues, filesystems, caches, generated clients/servers, and any test double replacing a producer or consumer.
|
|
250
263
|
|
|
251
|
-
-
|
|
252
|
-
- Use approved retrieval of public official documentation only when project artifacts are insufficient.
|
|
264
|
+
- Select authority for the property at the actual boundary; a database schema does not independently prove HTTP or message serialization.
|
|
265
|
+
- Use approved retrieval of public official documentation only when higher-authority project artifacts are insufficient.
|
|
253
266
|
- Never expose secrets, private source, proprietary payloads, or repository-private data during retrieval.
|
|
254
267
|
- Never use production credentials, production databases/APIs/queues/storage, real user data, or destructive external operations.
|
|
255
268
|
- If an endpoint cannot be proven non-production, do not connect to it.
|
|
256
|
-
- Use contract-backed local
|
|
257
|
-
-
|
|
269
|
+
- Use contract-backed local doubles or project-owned isolated services only within their stated proof boundary.
|
|
270
|
+
- Missing fidelity or composition evidence that affects the verdict produces `TEST_BLOCKED`, not a conditional pass.
|
|
258
271
|
|
|
259
272
|
## Final Verification
|
|
260
273
|
|
|
@@ -262,11 +275,13 @@ Before writing `TEST_PASS`:
|
|
|
262
275
|
|
|
263
276
|
1. Main-Agent review converged on the exact final state.
|
|
264
277
|
2. Independent review converged or was strictly and visibly downgraded under its capability contract.
|
|
265
|
-
3. Every required focused, module, contract
|
|
266
|
-
4.
|
|
267
|
-
5.
|
|
268
|
-
6.
|
|
269
|
-
7. No
|
|
278
|
+
3. Every required focused, module/component, consumer/provider contract, integration, actual-composition E2E, affected-module, and Regression Ring test passes reliably.
|
|
279
|
+
4. Every verdict-relevant test double has boundary-appropriate authority and fidelity proof, and every test-layer claim matches actual composition.
|
|
280
|
+
5. Every verdict-capable property in removed composition has composition-preserving evidence; applicable Provider Contract Tests assert raw wire output and state-changing workflows cover the required lifecycle.
|
|
281
|
+
6. Any required project-wide regression passes.
|
|
282
|
+
7. No accepted correction, unresolved finding, verdict-capable scope edge, missing fidelity/composition proof, or required environment check remains.
|
|
283
|
+
8. No high-risk production defect or correction remains.
|
|
284
|
+
9. No mutation occurred after the final applicable review and execution.
|
|
270
285
|
|
|
271
286
|
## Terminal Results
|
|
272
287
|
|
|
@@ -276,15 +291,15 @@ Only these testing-domain outcomes are valid:
|
|
|
276
291
|
TEST_PASS | TEST_NEEDS_FIXES | TEST_BLOCKED
|
|
277
292
|
```
|
|
278
293
|
|
|
279
|
-
- `TEST_PASS`: final reviewed state passes every required native test and no high-risk
|
|
280
|
-
- `TEST_NEEDS_FIXES`: a known correction remains
|
|
294
|
+
- `TEST_PASS`: final reviewed state passes every required native test and no high-risk correction remains.
|
|
295
|
+
- `TEST_NEEDS_FIXES`: a known correction remains or a testing-local review/repair budget did not converge.
|
|
281
296
|
- `TEST_BLOCKED`: expected truth, safe input, environment, execution reliability, external-target safety, or required review input prevents a safe verdict.
|
|
282
297
|
|
|
283
298
|
No conditional pass exists.
|
|
284
299
|
|
|
285
300
|
## Terminal Report and Result
|
|
286
301
|
|
|
287
|
-
Read `${SKILL_DIR}/references/test-report-template.md` before finalization.
|
|
302
|
+
Read `${SKILL_DIR}/references/test-report-template.md` before finalization. After required input resolution, write `test-report.md` and `test-result.json` together for every terminal outcome. The report must use the exact canonical `## Final Result` section and `- Result: <TEST_*>` marker from that template; do not rename it to `Terminal Result`, `Final Decision`, or another heading.
|
|
288
303
|
|
|
289
304
|
`test-result.json` is a terminal projection, not a checkpoint:
|
|
290
305
|
|
|
@@ -302,12 +317,10 @@ Read `${SKILL_DIR}/references/test-report-template.md` before finalization. Writ
|
|
|
302
317
|
},
|
|
303
318
|
"repair_rounds": 1,
|
|
304
319
|
"production_changed": false,
|
|
305
|
-
"review_required": false,
|
|
306
|
-
"review_scope": null,
|
|
307
320
|
"unresolved_items": []
|
|
308
321
|
}
|
|
309
322
|
```
|
|
310
323
|
|
|
311
|
-
Allowed `independent_review.status` values are `completed`, `downgraded`, and `not_applicable`. `completed` requires one through five responses and `final_state_rechecked=true`; `downgraded` requires zero responses, a non-empty reason, and `final_state_rechecked=false`; `not_applicable` requires zero responses, no reason, and `final_state_rechecked=false`. `
|
|
324
|
+
Allowed `independent_review.status` values are `completed`, `downgraded`, and `not_applicable`. `completed` requires one through five responses and `final_state_rechecked=true`; `downgraded` requires zero responses, a non-empty reason, and `final_state_rechecked=false`; `not_applicable` requires zero responses, no reason, and `final_state_rechecked=false`. `TEST_PASS` requires an empty `unresolved_items`; each non-pass result requires at least one exact unresolved correction or blocker. The report and JSON must agree on result, production-change diagnostics, and unresolved items. If consistency cannot be established, write a truthful `TEST_BLOCKED` result rather than fabricating pass.
|
|
312
325
|
|
|
313
|
-
|
|
326
|
+
Return only the testing result, `test-report.md`/`test-result.json` paths, execution summary, repair diagnostics, and unresolved items.
|
|
@@ -0,0 +1,206 @@
|
|
|
1
|
+
# Boundary Contract and Test Double Guidance
|
|
2
|
+
|
|
3
|
+
## Purpose
|
|
4
|
+
|
|
5
|
+
Apply one contract discipline to every serialized or producer-consumer boundary, whether internal or external: browser/backend APIs, cross-module endpoints, persistence, queues, filesystems, caches, generated clients/servers, SDK providers, and any test double that replaces a producer or consumer.
|
|
6
|
+
|
|
7
|
+
Mocks are test tools, not proof of provider behavior or a deployed environment. Consumer-local types, fixtures, and mocks are expectations, not provider truth. A test cannot prove a property that depends on production composition it replaces.
|
|
8
|
+
|
|
9
|
+
## Boundary-Specific Contract Authority
|
|
10
|
+
|
|
11
|
+
Choose the highest authority that actually defines the property at the tested boundary:
|
|
12
|
+
|
|
13
|
+
1. confirmed specification;
|
|
14
|
+
2. machine-readable wire contract, such as OpenAPI, JSON Schema, Protocol Buffers, GraphQL, or AsyncAPI;
|
|
15
|
+
3. type, client, server stub, or fixture that is traceably generated from that contract;
|
|
16
|
+
4. provider raw-wire contract test or explicitly authorized isolated provider observation;
|
|
17
|
+
5. provider implementation, version-locked SDK behavior, and matching public official service documentation;
|
|
18
|
+
6. consumer-local types, existing tests, historical fixtures, and mocks.
|
|
19
|
+
|
|
20
|
+
Lower-authority material cannot override higher-authority material. A generated artifact must retain traceable provenance. A database schema can define applicable persistence structure and constraints, but a database schema cannot independently prove network serialization. Current provider implementation is evidence, not automatic product truth when it conflicts with a higher-authority contract.
|
|
21
|
+
|
|
22
|
+
Use the current locked/provider version when behavior differs by version. Record the source path, public URL, document/version identifier, generated provenance, raw-wire test, or isolated observation used in `test-report.md`.
|
|
23
|
+
|
|
24
|
+
Never invent fields, statuses, error shapes, retry rules, pagination behavior, ordering, or timing guarantees without a traceable source. If authorities conflict and expected behavior cannot be resolved safely, return `TEST_BLOCKED`.
|
|
25
|
+
|
|
26
|
+
## Safe Documentation Retrieval
|
|
27
|
+
|
|
28
|
+
Public official service documentation retrieval may be used only to read the minimum contract material needed for tests.
|
|
29
|
+
|
|
30
|
+
- Do not send source code, repository-private content, proprietary payloads, secrets, credentials, or user data to a public endpoint.
|
|
31
|
+
- Prefer a project-local or vendored source when it is authoritative.
|
|
32
|
+
- Distinguish official provider documentation from blogs, examples, forum answers, and third-party tutorials.
|
|
33
|
+
- When official sources conflict with the project's pinned generated client or schema, apply the skill's truth precedence and report the conflict.
|
|
34
|
+
- If the required contract cannot be established, return `TEST_BLOCKED` when it prevents a safe verdict.
|
|
35
|
+
|
|
36
|
+
## Boundary and Test-Double Inventory
|
|
37
|
+
|
|
38
|
+
Inventory every test double that crosses or replaces a producer-consumer boundary, including mocks, fakes, fixtures, route interceptions, mock servers, containers, emulators, and isolated services. Record:
|
|
39
|
+
|
|
40
|
+
| Field | Required Content |
|
|
41
|
+
|---|---|
|
|
42
|
+
| Producer | Component that creates the value, message, file, state, or response. |
|
|
43
|
+
| Consumer | Component that parses, reads, maps, or acts on it. |
|
|
44
|
+
| Boundary type | Internal/external API, module, persistence, queue, filesystem, cache, generated client/server, or other boundary. |
|
|
45
|
+
| Serialization or protocol | JSON, HTTP, event/message, database encoding, file format, RPC, or other wire form. |
|
|
46
|
+
| Contract authority | Boundary-appropriate source and provenance from the authority order. |
|
|
47
|
+
| Test Double | Mock, fake, fixture, interception, server, container, emulator, or isolated service. |
|
|
48
|
+
| Fidelity proof | Generation, schema validation, shared raw-wire fixture, contract framework, or authorized isolated observation. |
|
|
49
|
+
| Composition removed | Serializer, adapter, network, provider, persistence, process, or other production component replaced. |
|
|
50
|
+
| Property not proven | Behavior that depends on the removed composition. |
|
|
51
|
+
| Composition-preserving test | Contract, integration, or Full-stack path that keeps the verdict-relevant combination real. |
|
|
52
|
+
| Remaining risk | Unproven behavior and its effect on the verdict. |
|
|
53
|
+
|
|
54
|
+
A boundary with no test double still records its authority and the composition-preserving test when that boundary affects the verdict.
|
|
55
|
+
|
|
56
|
+
## Test-Double Fidelity Proof
|
|
57
|
+
|
|
58
|
+
A cross-boundary test double is fidelity-proven only when at least one of these applies:
|
|
59
|
+
|
|
60
|
+
- generated from the authoritative contract with provenance;
|
|
61
|
+
- validated against the authoritative schema during tests;
|
|
62
|
+
- shared with a provider raw-wire contract test that validates the same fixture;
|
|
63
|
+
- captured through an explicitly authorized isolated provider observation with source/version provenance;
|
|
64
|
+
- checked by the project's consumer/provider contract framework.
|
|
65
|
+
|
|
66
|
+
A consumer-authored ideal fixture without one of these proofs may support local UI or logic tests, but it does not count as provider compatibility evidence. Record its proof boundary explicitly.
|
|
67
|
+
|
|
68
|
+
## Build the Smallest Faithful Double
|
|
69
|
+
|
|
70
|
+
Use the target project's existing fake, fixture, mock library, mock server, container, or isolated service when adequate. Model only behavior relevant to the affected module, but preserve the contract property being tested.
|
|
71
|
+
|
|
72
|
+
Applicable variants include:
|
|
73
|
+
|
|
74
|
+
- normal success;
|
|
75
|
+
- empty success;
|
|
76
|
+
- minimum, maximum, pagination, cursor, or size boundary;
|
|
77
|
+
- malformed or contract-invalid response;
|
|
78
|
+
- timeout or cancellation;
|
|
79
|
+
- rate limiting or quota rejection;
|
|
80
|
+
- authentication or authorization rejection;
|
|
81
|
+
- transient service failure;
|
|
82
|
+
- permanent service failure;
|
|
83
|
+
- partial response or interrupted stream;
|
|
84
|
+
- retry, replay, and idempotency behavior;
|
|
85
|
+
- transaction, constraint, rollback, or conflict behavior for stateful services.
|
|
86
|
+
|
|
87
|
+
Do not require every variant mechanically. Select variants from the actual behavior/risk model and explain material omissions.
|
|
88
|
+
|
|
89
|
+
## Assertions
|
|
90
|
+
|
|
91
|
+
Assert the application's observable handling of the contract:
|
|
92
|
+
|
|
93
|
+
- outbound request shape, required headers or metadata, and serialization;
|
|
94
|
+
- response mapping, nullability, enum and format handling;
|
|
95
|
+
- retry count and retryable versus permanent classification;
|
|
96
|
+
- timeout and cancellation propagation;
|
|
97
|
+
- pagination and cursor behavior;
|
|
98
|
+
- idempotency and side-effect counts;
|
|
99
|
+
- state transition, rollback, cleanup, and absence of partial effects;
|
|
100
|
+
- consumer-visible error type or result.
|
|
101
|
+
|
|
102
|
+
A mock-success-only test is insufficient when dependency failure can change observable behavior.
|
|
103
|
+
|
|
104
|
+
### Provider Raw-Wire Assertions
|
|
105
|
+
|
|
106
|
+
A Provider Contract Test for an HTTP response, message, event, file, cache value, or other serialized output must inspect the raw wire payload before any provider-owned model rehydrates it. Select risk-relevant assertions for:
|
|
107
|
+
|
|
108
|
+
- required, optional, omitted, `null`, empty, and zero values;
|
|
109
|
+
- field names, scalar types, enums, formats, timestamps, and default/omission behavior;
|
|
110
|
+
- status or result code, error envelope, required headers, and metadata;
|
|
111
|
+
- exact bytes or encoding when consumers depend on them.
|
|
112
|
+
|
|
113
|
+
Deserializing output back into the provider's own model is not sufficient because defaults and zero values can conceal missing fields, nullability changes, or serializer-tag drift.
|
|
114
|
+
|
|
115
|
+
## Avoid Over-Mocking
|
|
116
|
+
|
|
117
|
+
Keep production composition real when that composition is the property under test. Mock at a stable external boundary rather than mocking the method being verified or every internal collaborator.
|
|
118
|
+
|
|
119
|
+
Reject a double when it:
|
|
120
|
+
|
|
121
|
+
- repeats the current implementation instead of the external contract;
|
|
122
|
+
- bypasses serialization, mapping, retries, or state transitions under test;
|
|
123
|
+
- returns impossible provider behavior;
|
|
124
|
+
- asserts only that a mock was called without proving the business result;
|
|
125
|
+
- silently drifts from the project or official contract.
|
|
126
|
+
|
|
127
|
+
Use contract/integration or isolated infrastructure tests when a mock cannot preserve the relevant transaction, protocol, or composition property.
|
|
128
|
+
|
|
129
|
+
## Test Layer Taxonomy
|
|
130
|
+
|
|
131
|
+
Classify a test by the production composition it actually executes, not by runner or user-interface presence:
|
|
132
|
+
|
|
133
|
+
| Layer | Critical Provider Replaced? | Proof Capability |
|
|
134
|
+
|---|---:|---|
|
|
135
|
+
| Focused / Unit Test | yes | Local logic, boundaries, and deterministic errors. |
|
|
136
|
+
| UI Component Test | yes | Component rendering and interaction under supplied data. |
|
|
137
|
+
| Mock Browser Test | yes | Browser UI workflow under intercepted or fixture-backed boundaries. |
|
|
138
|
+
| Consumer Contract Test | contract-validated double | Consumer handling of the authoritative contract. |
|
|
139
|
+
| Provider Contract Test | not applicable | Provider raw wire behavior at its serialization boundary. |
|
|
140
|
+
| Integration Test | no for the combination under test | The named modules or infrastructure composed together. |
|
|
141
|
+
| Full-stack E2E | no for critical production boundaries | Complete user path across real application layers in safe isolation. |
|
|
142
|
+
|
|
143
|
+
Browser execution alone does not make a test Full-stack E2E. A browser test that intercepts a critical API is a Mock Browser Test and cannot prove browser/backend compatibility, provider serialization, persistence, or reload behavior.
|
|
144
|
+
|
|
145
|
+
## Composition-Preserving Verdict Rule
|
|
146
|
+
|
|
147
|
+
Locate each material risk in the combination where the defect can occur. Preserve that combination for risks involving consumer/provider serialization, UI/API behavior, API/persistence behavior, generated client/server compatibility, queue producer/consumer behavior, filesystem writer/reader behavior, cache producer/consumer behavior, or transaction/lock/retry/recovery state.
|
|
148
|
+
|
|
149
|
+
Lower layers may diagnose a defect, but they cannot supply verdict evidence for a property that their doubles remove. Before `TEST_PASS`:
|
|
150
|
+
|
|
151
|
+
1. identify every verdict-capable risk located in removed composition;
|
|
152
|
+
2. require a provider contract, consumer contract plus provider raw-wire evidence, composition-preserving integration test, or Full-stack E2E path that proves the property without replacing that composition;
|
|
153
|
+
3. verify the preserving test would fail while the target defect exists;
|
|
154
|
+
4. record the preserving evidence and remaining risk in the report.
|
|
155
|
+
|
|
156
|
+
If required composition-preserving verification cannot run in a proven safe isolated environment, return `TEST_BLOCKED`. Do not substitute a lower-layer green result or relabel a Mock Browser Test as E2E.
|
|
157
|
+
|
|
158
|
+
## Databases and Stateful Services
|
|
159
|
+
|
|
160
|
+
Prefer project-provided isolated infrastructure, a faithful local fake, or a disposable service. Derive structure and constraints from schemas, migrations, models, or official protocol contracts.
|
|
161
|
+
|
|
162
|
+
Any disposable resource must have:
|
|
163
|
+
|
|
164
|
+
- a unique non-production identity;
|
|
165
|
+
- bounded lifetime;
|
|
166
|
+
- deterministic setup;
|
|
167
|
+
- verified cleanup;
|
|
168
|
+
- no dependency on production credentials or data.
|
|
169
|
+
|
|
170
|
+
Test relevant constraints, transactions, rollback, concurrency, idempotency, and failure mapping. Cleanup failure that undermines reliability produces `TEST_BLOCKED`.
|
|
171
|
+
|
|
172
|
+
## Production and Data Prohibitions
|
|
173
|
+
|
|
174
|
+
Never use:
|
|
175
|
+
|
|
176
|
+
- production credentials or tokens;
|
|
177
|
+
- production databases, APIs, queues, object storage, caches, or services;
|
|
178
|
+
- real customer or user data;
|
|
179
|
+
- a target whose non-production status cannot be established;
|
|
180
|
+
- destructive operations against external data.
|
|
181
|
+
|
|
182
|
+
Project configuration may be reused only when it demonstrably selects a safe isolated test target. If safety cannot be proven, do not connect; report the unexecuted check and return `TEST_BLOCKED` when it is necessary for the verdict.
|
|
183
|
+
|
|
184
|
+
## Reporting
|
|
185
|
+
|
|
186
|
+
For every verdict-relevant boundary or test double, complete the Boundary and Test-Double Inventory fields. Distinguish local/unit, consumer-contract, provider-contract, integration, Mock Browser, and Full-stack evidence. State whether a composition-preserving test exists and whether any missing proof changes the verdict.
|
|
187
|
+
|
|
188
|
+
Also record:
|
|
189
|
+
|
|
190
|
+
- variants exercised;
|
|
191
|
+
- production resources used: `no`;
|
|
192
|
+
- unresolved fidelity or deployed-environment risk.
|
|
193
|
+
|
|
194
|
+
Do not claim that a local mock, fake, contract snapshot, container, isolated service, or Mock Browser Test verifies composition it replaces or a deployed production environment.
|
|
195
|
+
|
|
196
|
+
## Generic Composition Example
|
|
197
|
+
|
|
198
|
+
For a browser reading a provider response:
|
|
199
|
+
|
|
200
|
+
1. Treat the machine-readable wire contract as the response-shape authority.
|
|
201
|
+
2. Add a Provider Contract Test that asserts the raw serialized response, including a risk-relevant omitted or null field.
|
|
202
|
+
3. Validate the Consumer Contract Test fixture against the same contract or share the provider-validated fixture.
|
|
203
|
+
4. Classify a route-intercepted browser flow as a Mock Browser Test; it proves UI behavior under that fixture but not live browser/provider compatibility.
|
|
204
|
+
5. Add a safe integration or Full-stack E2E path that keeps browser mapping, provider serialization, and required persistence real when those components contain the verdict-capable risk.
|
|
205
|
+
|
|
206
|
+
The field names and protocol vary by project; the authority, fidelity, proof-boundary, and composition reasoning remain the same.
|
|
@@ -18,7 +18,7 @@ Main Agent builds and executes project-native tests
|
|
|
18
18
|
→ TEST_PASS | TEST_NEEDS_FIXES | TEST_BLOCKED
|
|
19
19
|
```
|
|
20
20
|
|
|
21
|
-
Keep at most one Reviewer active for this invocation and permit at most one replacement. Prefer native continuation and never reuse a Reviewer from another
|
|
21
|
+
Keep at most one Reviewer active for this invocation and permit at most one replacement. Prefer native continuation and never reuse a Reviewer from another invocation. A compliant replacement may continue the same shared response budget only with complete latest state, never only a conversation summary.
|
|
22
22
|
|
|
23
23
|
## Host Capability Gate
|
|
24
24
|
|
|
@@ -50,7 +50,7 @@ Rules:
|
|
|
50
50
|
|
|
51
51
|
## Stage-Specific Review Input
|
|
52
52
|
|
|
53
|
-
|
|
53
|
+
Use the exact caller-supplied testing identity on every response:
|
|
54
54
|
|
|
55
55
|
```text
|
|
56
56
|
artifact_dir: [EXACT_ARTIFACT_DIR]
|
|
@@ -67,9 +67,9 @@ The Main Agent supplies:
|
|
|
67
67
|
- exact artifact, spec, and plan paths plus current contents;
|
|
68
68
|
- complete current production-and-test change;
|
|
69
69
|
- affected business module, observable behaviors, applicable risks, and Regression Ring;
|
|
70
|
-
- existing, added, and modified tests, fixtures, mocks, schemas, contracts, and
|
|
70
|
+
- existing, added, and modified tests, fixtures, mocks, schemas, contracts, selected layers, and the complete boundary/test-double inventory with authority, fidelity, removed composition, proof limits, preserving evidence, and verdict impact;
|
|
71
71
|
- native test commands and results executed by Main Agent;
|
|
72
|
-
-
|
|
72
|
+
- testing-local repairs;
|
|
73
73
|
- response number and total budget;
|
|
74
74
|
- prior adjudication and actual corrections on continuation or replacement.
|
|
75
75
|
|
|
@@ -100,7 +100,7 @@ Execution boundaries:
|
|
|
100
100
|
- Complete this review personally.
|
|
101
101
|
- Do not create, schedule, resume, continue, request, or coordinate another execution unit.
|
|
102
102
|
- Do not ask the Main Agent to create a helper.
|
|
103
|
-
- Do not re-enter
|
|
103
|
+
- Do not re-enter delegation or another review process.
|
|
104
104
|
- Do not modify, create, delete, rename, stage, commit, or otherwise mutate files.
|
|
105
105
|
- Do not execute shell, Git, tests, builds, network calls, external processes, or any operation that can change state.
|
|
106
106
|
- Use read-only checkout access only for concrete module, caller, consumer, schema, type, configuration, fixture, test, or contract coupling.
|
|
@@ -119,13 +119,20 @@ First validate input consistency. Then review:
|
|
|
119
119
|
- selection of focused, module, contract/integration, applicable E2E, module-regression, and Regression Ring layers;
|
|
120
120
|
- assertion strength and observable effect;
|
|
121
121
|
- vacuous or false-positive tests and tests that cannot fail for the intended defect;
|
|
122
|
-
-
|
|
123
|
-
-
|
|
122
|
+
- every producer-consumer boundary and test double having boundary-appropriate contract authority and fidelity proof;
|
|
123
|
+
- any consumer-authored ideal fixture being treated only as consumer expectation unless independently contract-validated;
|
|
124
|
+
- Provider Contract Tests containing raw Provider serialization evidence, including risk-relevant omitted, null, empty, zero, type, and field-name behavior;
|
|
125
|
+
- each Browser Test being classified from actual production composition, with intercepted critical APIs classified as Mock Browser rather than Full-stack E2E;
|
|
126
|
+
- no test claim exceeding the property its test double can prove and no verdict-capable composition being removed without preserving contract, integration, or Full-stack evidence;
|
|
127
|
+
- each relevant test being able to fail while the target defect exists;
|
|
128
|
+
- state-changing workflows covering commit, cancellation, failure recovery, reload or re-entry, retry/conflict/idempotency, and repeated operation semantics;
|
|
124
129
|
- nondeterminism and flakiness risk;
|
|
125
130
|
- native execution results matching the declared scope;
|
|
126
|
-
- production repairs
|
|
131
|
+
- production defects or attempted repairs outside the permitted testing-local repair boundary;
|
|
127
132
|
- remaining risks being stated honestly.
|
|
128
133
|
|
|
134
|
+
Return `CORRECTION_NEEDED` when a verdict-relevant producer-consumer combination is replaced by a test double without authoritative fidelity proof and composition-preserving evidence, or when a test-layer claim exceeds the actual composition executed.
|
|
135
|
+
|
|
129
136
|
Do not rewrite the requirement, modify code, execute tests, or perform broad exploration.
|
|
130
137
|
Return exactly one result using the Reviewer Output Protocol.
|
|
131
138
|
```
|
|
@@ -159,7 +166,7 @@ All initial execution boundaries and the Reviewer Output Protocol remain mandato
|
|
|
159
166
|
|
|
160
167
|
## Reviewer Output Protocol
|
|
161
168
|
|
|
162
|
-
Return exactly one form. Do not add severity, confidence, dimension,
|
|
169
|
+
Return exactly one form. Do not add severity, confidence, dimension, caller routing, or acceptance fields.
|
|
163
170
|
|
|
164
171
|
### No Correction Needed
|
|
165
172
|
|
|
@@ -262,6 +269,7 @@ Record:
|
|
|
262
269
|
- each response number `1..5`, input identity, result, and correction count;
|
|
263
270
|
- every Main-Agent adjudication and its evidence;
|
|
264
271
|
- actual repair and native verification;
|
|
272
|
+
- boundary authority/fidelity, removed-composition, proof-limit, preserving-evidence, layer-classification, and mutation-lifecycle corrections;
|
|
265
273
|
- whether the exact final state was independently rechecked;
|
|
266
274
|
- capability basis, continuation mode (`native`, `replacement`, `mixed`, or `not-applicable`), and replacement count.
|
|
267
275
|
|