prizmkit 1.1.139 → 1.1.141

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (74) hide show
  1. package/bundled/VERSION.json +3 -3
  2. package/bundled/dev-pipeline/prizmkit_runtime/checkpoint_state.py +41 -121
  3. package/bundled/dev-pipeline/prizmkit_runtime/runner_models.py +0 -6
  4. package/bundled/dev-pipeline/prizmkit_runtime/runners.py +1 -2
  5. package/bundled/dev-pipeline/prizmkit_runtime/test_result.py +150 -0
  6. package/bundled/dev-pipeline/scripts/generate-bootstrap-prompt.py +21 -804
  7. package/bundled/dev-pipeline/scripts/generate-bugfix-prompt.py +4 -6
  8. package/bundled/dev-pipeline/scripts/generate-recovery-prompt.py +1 -1
  9. package/bundled/dev-pipeline/scripts/generate-refactor-prompt.py +4 -6
  10. package/bundled/dev-pipeline/scripts/prompt_framework.py +56 -55
  11. package/bundled/dev-pipeline/scripts/update-bug-status.py +1 -2
  12. package/bundled/dev-pipeline/scripts/update-checkpoint.py +14 -96
  13. package/bundled/dev-pipeline/scripts/update-feature-status.py +1 -2
  14. package/bundled/dev-pipeline/scripts/update-refactor-status.py +1 -2
  15. package/bundled/dev-pipeline/templates/bootstrap-prompt.md +3 -3
  16. package/bundled/dev-pipeline/templates/bootstrap-tier3.md +7 -18
  17. package/bundled/dev-pipeline/templates/bugfix-bootstrap-prompt.md +6 -6
  18. package/bundled/dev-pipeline/templates/refactor-bootstrap-prompt.md +5 -5
  19. package/bundled/dev-pipeline/templates/sections/bugfix-phase-commit-report.md +1 -1
  20. package/bundled/dev-pipeline/templates/sections/bugfix-phase-diagnose-plan.md +1 -1
  21. package/bundled/dev-pipeline/templates/sections/bugfix-phase-manual-verification.md +1 -1
  22. package/bundled/dev-pipeline/templates/sections/bugfix-phase-test.md +13 -10
  23. package/bundled/dev-pipeline/templates/sections/bugfix-reminders.md +1 -1
  24. package/bundled/dev-pipeline/templates/sections/checkpoint-system.md +14 -9
  25. package/bundled/dev-pipeline/templates/sections/headless-commit-authorization.md +1 -1
  26. package/bundled/dev-pipeline/templates/sections/phase-commit-full.md +1 -1
  27. package/bundled/dev-pipeline/templates/sections/phase-commit.md +1 -1
  28. package/bundled/dev-pipeline/templates/sections/phase-context-snapshot-agent-suffix.md +2 -2
  29. package/bundled/dev-pipeline/templates/sections/phase-context-snapshot-base.md +1 -1
  30. package/bundled/dev-pipeline/templates/sections/phase-context-snapshot-lite-suffix.md +1 -1
  31. package/bundled/dev-pipeline/templates/sections/phase-implement-lite.md +1 -1
  32. package/bundled/dev-pipeline/templates/sections/phase-prizmkit-test.md +14 -26
  33. package/bundled/dev-pipeline/templates/sections/phase-specify-plan-full.md +3 -3
  34. package/bundled/dev-pipeline/templates/sections/refactor-phase-test.md +13 -10
  35. package/bundled/dev-pipeline/templates/sections/refactor-reminders.md +1 -1
  36. package/bundled/dev-pipeline/tests/test_checkpoint_state.py +287 -384
  37. package/bundled/dev-pipeline/tests/test_generate_bootstrap_prompt.py +119 -684
  38. package/bundled/dev-pipeline/tests/test_generate_bugfix_prompt.py +17 -40
  39. package/bundled/dev-pipeline/tests/test_generate_refactor_prompt.py +20 -23
  40. package/bundled/dev-pipeline/tests/test_python_runner_parity.py +29 -32
  41. package/bundled/skills/_metadata.json +6 -6
  42. package/bundled/skills/prizmkit/SKILL.md +8 -13
  43. package/bundled/skills/prizmkit/references/workflow-state-protocol.md +61 -80
  44. package/bundled/skills/prizmkit-code-review/SKILL.md +3 -3
  45. package/bundled/skills/prizmkit-code-review/references/workflow-state-protocol.md +61 -80
  46. package/bundled/skills/prizmkit-committer/SKILL.md +3 -3
  47. package/bundled/skills/prizmkit-committer/references/workflow-state-protocol.md +61 -80
  48. package/bundled/skills/prizmkit-implement/references/workflow-state-protocol.md +61 -80
  49. package/bundled/skills/prizmkit-plan/references/workflow-state-protocol.md +61 -80
  50. package/bundled/skills/prizmkit-retrospective/SKILL.md +2 -2
  51. package/bundled/skills/prizmkit-retrospective/references/workflow-state-protocol.md +61 -80
  52. package/bundled/skills/prizmkit-test/SKILL.md +217 -187
  53. package/bundled/skills/prizmkit-test/references/external-contract-mock-guidance.md +119 -0
  54. package/bundled/skills/prizmkit-test/references/independent-test-review.md +269 -0
  55. package/bundled/skills/prizmkit-test/references/test-coverage-model.md +193 -0
  56. package/bundled/skills/prizmkit-test/references/test-report-template.md +164 -79
  57. package/bundled/skills/prizmkit-workflow/SKILL.md +15 -23
  58. package/bundled/skills/prizmkit-workflow/references/workflow-state-protocol.md +61 -80
  59. package/package.json +1 -1
  60. package/bundled/skills/prizmkit-test/assets/authoritative-records.schema.json +0 -476
  61. package/bundled/skills/prizmkit-test/assets/behavior-risk-matrix.schema.json +0 -116
  62. package/bundled/skills/prizmkit-test/assets/evidence-manifest.schema.json +0 -111
  63. package/bundled/skills/prizmkit-test/assets/evidence-package-template.json +0 -66
  64. package/bundled/skills/prizmkit-test/references/boundary-coverage-protocol.md +0 -104
  65. package/bundled/skills/prizmkit-test/references/contract-mock-protocol.md +0 -65
  66. package/bundled/skills/prizmkit-test/references/evidence-protocol.md +0 -208
  67. package/bundled/skills/prizmkit-test/references/evidence-request-protocol.md +0 -80
  68. package/bundled/skills/prizmkit-test/references/examples.md +0 -100
  69. package/bundled/skills/prizmkit-test/references/service-boundary-test-catalog.md +0 -228
  70. package/bundled/skills/prizmkit-test/references/test-generation-steps.md +0 -118
  71. package/bundled/skills/prizmkit-test/references/trusted-evidence-execution.md +0 -110
  72. package/bundled/skills/prizmkit-test/references/workflow-state-protocol.md +0 -181
  73. package/bundled/skills/prizmkit-test/scripts/build_test_evidence.py +0 -1851
  74. package/bundled/skills/prizmkit-test/scripts/validate_test_evidence.py +0 -1793
@@ -1,283 +1,313 @@
1
1
  ---
2
2
  name: "prizmkit-test"
3
- description: "Auditable test-evidence protocol for AI-generated behavior changes. Classifies changes, discovers affected modules and regression rings, models observable behavior and risk, builds project-native tests and contract-driven mocks, validates current-workspace test evidence, preserves content-addressed raw evidence, and deterministically returns TEST_PASS, TEST_FAIL, or TEST_BLOCKED. Use after code/config/schema/dependency changes or when users ask to test, verify, add complete module tests, inspect boundaries, or produce auditable evidence. Supports full-project, module, feature, and scope=this-change. (project)"
3
+ description: "AI-led project-native testing for the complete affected business module and its Regression Ring. Adds comprehensive behavior, boundary, error, state, side-effect, permission, concurrency, idempotency, time, dependency, and consumer tests; repairs valid failures in a bounded loop; performs mandatory Main-Agent review plus one optional strict independent Test Reviewer; and returns TEST_PASS, TEST_NEEDS_FIXES, or TEST_BLOCKED with a human report and minimal terminal result. (project)"
4
4
  ---
5
5
 
6
6
  # PrizmKit Test
7
7
 
8
- `/prizmkit-test` converts code changes into auditable testing evidence. Completeness means that the affected module's observable behavior, contracts, boundaries, invariants, errors, state transitions, and coupling risks are represented and proven. It does not mean mechanically reaching a line-coverage threshold.
8
+ `/prizmkit-test` is the independent testing stage for the current workspace. It understands the affected business behavior, fills project-native test gaps, executes the tests, repairs valid failures, reviews the final tests, and reports a truthful testing-domain result.
9
9
 
10
- This skill orchestrates each target project's native runners and conventions. It does not install a cross-language test runtime.
10
+ It does not build an evidence package, prove historical TDD order, validate a previous checkout, or manage pipeline/session state.
11
11
 
12
- ## Model Choice vs Fixed Evidence Mechanics
12
+ ## Atomic Skill Boundary
13
13
 
14
- Choose project-semantic parameters from the target project: framework, commands, cwd, timeout, retry/attempt policy, concurrency, Mock tools, test layers, module boundary, inventory patterns, and environment classification. Do not hard-code ecosystem-specific defaults when the project can decide them.
15
-
16
- Fix only the protocol mechanisms needed for safe, replayable evidence: project/evidence/request locators, schema-shaped requests, path confinement, real process execution, complete raw capture, runner-generated receipts, append-only history, and resume invalidation. Read `${SKILL_DIR}/references/trusted-evidence-execution.md` before inventory, execution, resume, or replay; it defines the trust boundary and builder commands. Read `${SKILL_DIR}/references/evidence-request-protocol.md` when authoring any request or authoritative JSON record; it defines request ownership, structured N/A, and the schema map.
17
-
18
- ## Canonical Evidence Builder Lifecycle
19
-
20
- The Main Agent supplies semantic requests only; it does not author authoritative package records. Initialize and drive one evidence directory through the builder in this order:
14
+ The Main Agent owns this invocation from analysis through final report:
21
15
 
22
16
  ```text
23
- init capture-change inventory → prepare-tests → execute → finalize → render-report → validate --attest → resume
17
+ resolve current change and requirement context
18
+ → discover affected module and Regression Ring
19
+ → model observable behavior and applicable risks
20
+ → inspect and add project-native tests
21
+ → execute and repair valid failures: maximum three rounds
22
+ → mandatory Main-Agent test review: maximum ten completed rounds
23
+ → optional independent Test Reviewer: one unit, maximum five responses
24
+ → final native regression on the exact final state
25
+ → write test-report.md and test-result.json
26
+ → return TEST_PASS | TEST_NEEDS_FIXES | TEST_BLOCKED
24
27
  ```
25
28
 
26
- Use `${SKILL_DIR}/scripts/build_test_evidence.py` for every lifecycle command:
27
-
28
- - `init`: creates the package directories and immutable baseline identity.
29
- - `capture-change`: captures the complete canonical change, including untracked, deleted, renamed, and copied files; never substitute plain `git diff`. `inventory` then requires its changed-file set to match the capture exactly and rejects stale captures or unaccounted live paths.
30
- - `inventory`: hashes current source/tests/contracts/lockfiles, rejects category overlap, excludes runtime-only paths, and validates directory roots plus the complete capture-derived changed-file set.
31
- - `prepare-tests`: discovers repository-native test files from the canonical capture and inventory, generates stable builder-owned test IDs, derives existing/added/modified status, and snapshots only added/modified live test sources.
32
- - `execute`: runs native project commands and owns `executions.json`, `receipts/`, and `raw/`. Each request uses exactly one binding mode: `test_ids`, `test_paths`, or `auto_bind=true`. The builder infers recognized package-wide or explicit file/directory/glob command scope, blocks opaque commands or mismatched declarations, and persists verified canonical `test_ids` only.
33
- - `finalize`: runs the pre-finalize semantic-record and required-layer checks, rechecks canonical capture freshness, blocks every selected unreliable execution, and owns manifest, stage hashes, verdict, generated-test snapshots, and evidence identity. If semantic records are incomplete or malformed, finalize fails before creating an immutable package; do not hand-edit a finalized package or start another package after a protocol budget is exhausted.
34
- - `render-report`: derives `test-report.md`; it is never authoritative.
35
- - `resume`: preserves prior receipts and refuses to rebind an identity-changing package; create a new package after scope/source identity changes.
29
+ After writing its result, this skill returns control to its caller. It must not:
36
30
 
37
- `test-plan.json` is the only source of builder-generated test IDs. An execution request provides exactly one of native `test_paths`, `auto_bind=true`, or canonical `test_ids`; the actual command must expose the same deterministic scope, including all prepared tests for recognized package-wide commands. The builder blocks opaque or mismatched command scope and rewrites verified requests to canonical `test_ids` only before execution. Assess all five layers explicitly, but set `required=true` only when risk requires it; every omitted layer needs structured N/A evidence. The same binding applies test ↔ behavior ↔ execution; current-workspace receipts are the execution evidence.
31
+ - invoke implementation, Code Review, retrospective, or committer;
32
+ - update a workflow checkpoint or pipeline runtime state;
33
+ - classify an AI CLI session as success, crash, retry, or continuation;
34
+ - decide whether a workflow terminates or resumes;
35
+ - call a testing-domain non-pass result a runtime failure.
38
36
 
39
- The builder/validator, not the Main Agent, produces or updates `manifest.json`, `executions.json`, `receipts/*`, `validation.json`, `verdict.json`, `test-report.md`, and `source-change.patch`. Semantic records are authored through schema-shaped requests and must pass the builder preflight before finalization. `validation.json` is produced only by `validate_test_evidence.py --attest`; a package is not a `TEST_PASS` until the second strict validation succeeds.
37
+ ## Inputs and Outputs
40
38
 
39
+ | Parameter | Required | Description |
40
+ |---|---|---|
41
+ | `scope` | No | `full-project`, `module:<name>`, `feature:<slug>`, or `this-change`. Defaults to `this-change` when current change context exists, otherwise `full-project`. |
42
+ | `artifact_dir` | No | Active requirement directory containing available specification, plan, and prior review context. Reuse the caller-provided path. |
43
+ | `changed_files` | No | Explicit current changed paths. Highest-priority scope locator. |
44
+ | `diff_base` | No | Git comparison base when explicit changed paths are absent. |
45
+ | `test_commands` | No | Project-native commands when repository conventions are ambiguous. |
41
46
 
42
- For a formal requirement, `/prizmkit-test` runs after the Main-Agent code-review stage reaches `PASS` and before retrospective:
47
+ Write exactly two terminal artifacts in `artifact_dir` when it is available; otherwise use a clearly reported caller-visible artifact directory selected from the active request context:
43
48
 
44
49
  ```text
45
- plan → implement → code-review → test → retrospective → committer
50
+ {artifact_dir}/
51
+ ├── test-report.md
52
+ └── test-result.json
46
53
  ```
47
54
 
48
- Read `${SKILL_DIR}/references/workflow-state-protocol.md` before reading or updating lifecycle state. Reuse the caller's `artifact_dir`; do not select another recent plan. When an active `orchestrator` is recorded, return the testing result and next-stage metadata to it instead of invoking another skill independently.
49
-
50
- ## Atomic Stage Boundary
55
+ Do not create `.prizmkit/test/evidence/`, manifests, hashes, attestations, package lifecycle records, or test-internal checkpoints.
51
56
 
52
- `prizmkit-test` owns only auditable test evidence generation, validation, and the testing-domain verdict. It writes the authoritative `TEST_*` result and `next_stage`, then returns control. When an external orchestrator is active, it must not invoke retrospective or implementation itself. The active orchestrator owns the next-stage invocation and repair routing.
57
+ ## Context Loading and Scope Resolution
53
58
 
59
+ 1. Reuse current requirement and workspace context already loaded by the Main Agent.
60
+ 2. Load `root.prizm`, the relevant module indexes, and the applicable detail docs before modifying source or tests.
61
+ 3. Inspect current staged, unstaged, untracked, deleted, and renamed paths. Changed lines locate impact but never define completeness.
62
+ 4. Read the available specification, acceptance criteria, plan, and prior review only as needed to establish expected behavior.
63
+ 5. Inspect manifests, runner configuration, CI conventions, existing test assertions, fixtures, fakes, mocks, contracts, and coverage support.
64
+ 6. If necessary scope cannot be determined safely, report the precise missing input and return `TEST_BLOCKED`.
54
65
 
55
- - behavior-changing source code;
56
- - runtime configuration, schema, migration, generated-runtime, lockfile, or dependency changes;
57
- - public interfaces and data contracts;
58
- - frontend, backend, library, CLI, adapter, and multi-module changes;
59
- - legacy modules whose observable test coverage is incomplete;
60
- - explicit test, boundary, evidence, confidence, or audit requests.
66
+ Do not infer the active requirement solely from the name of an old artifact directory. Do not use a historical worktree, second checkout, mutation proof, baseline-failure proof, or test overlay.
61
67
 
62
- Documentation, comment, and formatting-only scopes use deterministic lightweight verification. They do not receive an unjustified full behavior-test claim.
68
+ ## Coverage Model
63
69
 
64
- ## Preconditions and Safety
70
+ Read `${SKILL_DIR}/references/test-coverage-model.md` before designing or reviewing tests.
65
71
 
66
- 1. Load `root.prizm`, relevant L1/L2 docs, project manifests, runner configuration, and changed scope once.
67
- 2. Never use production credentials, production databases, production APIs, production queues/storage, or destructive operations against real data.
68
- 3. Preserve complete command/environment/output values as requested. Mark the package `sensitivity=project-controlled`; the project owns access control, retention, and upload policy.
69
- 4. Treat code-level dependencies as mock-first. Real deployed test-environment validation is a separate authorized activity.
70
- 5. Never delete existing tests.
71
- 6. Evidence-protocol recovery is separately bounded: allow at most two package initializations and two semantic preflight failures for one canonical change identity. After the budget is exhausted, return `TEST_BLOCKED`; do not create another versioned package or hand-edit finalized evidence.
72
- 7. For `TEST_FAIL`, classify and persist an evidence-backed `repair_scope` of `test-infrastructure`, `production`, `runtime`, `schema`, `dependency`, `public-interface`, or `unknown`. Test-infrastructure scope routes to implement then test; production-family scope routes to implement, code-review, then test. If evidence cannot support a safe classification, return `TEST_BLOCKED`.
72
+ ### Affected Business Module
73
73
 
74
- If test infrastructure is missing, the infrastructure preparation state establishes only the necessary project-native infrastructure autonomously. Do not ask the user to install routine test dependencies in a headless run; verify dependency versions before editing manifests.
74
+ Prefer an explicit project module. If none exists, derive one cohesive boundary from files that jointly implement the same observable responsibility. Cover every discoverable observable behavior of that module, including relevant legacy behavior whose tests are incomplete.
75
75
 
76
- ## Input
77
-
78
- | Parameter | Required | Description |
79
- |-----------|----------|-------------|
80
- | `scope` | No | `full-project`, `module:<name>`, `feature:<slug>`, or `this-change`. Defaults to `this-change` with a valid artifact directory, otherwise `full-project`. |
81
- | `artifact_dir` | No | Change artifact containing `spec.md` + `plan.md`; used to resolve acceptance conditions and changed scope. |
82
- | `changed_files` | No | Explicit changed files; highest-priority scope input. |
83
- | `diff_base` | No | Git baseline used when `changed_files` is absent. |
84
- | `evidence_target` | No | Existing evidence ID/directory to validate and resume. |
85
- | `execution_budget` | No | Bounded time/attempt budget. Incomplete necessary work is `TEST_BLOCKED`, never truncated to pass. |
86
- | `test_commands` | No | Explicit project-native commands when discovery is ambiguous. |
76
+ ### Regression Ring
87
77
 
88
- For `scope=this-change`, determine changed files from `changed_files`, `diff_base`, current diff, artifact plan/context manifest, then spec mapping. Unknown scope is `TEST_BLOCKED`.
78
+ Expand only along concrete coupling evidence:
89
79
 
90
- Do not infer a current change artifact from a legacy bug/refactor directory name alone; require explicit `artifact_dir` and changed-file evidence.
80
+ - direct callers;
81
+ - consumers of values, errors, events, files, or ordering;
82
+ - shared schemas, types, protocols, generated assets, and adapters;
83
+ - persistence, cache, lock, queue, transaction, and other shared state dependencies.
91
84
 
92
- ## Authoritative Resources
85
+ Do not perform an unconditional repository-wide sweep.
93
86
 
94
- Load each resource only on its branch:
87
+ ### Behavior and Risks
95
88
 
96
- - Read `${SKILL_DIR}/references/evidence-protocol.md` before a behavior evidence run or validation/resume decision it defines state contracts, identity, proof, and verdict semantics.
97
- - Load `${SKILL_DIR}/assets/evidence-manifest.schema.json` when creating or validating `manifest.json`.
98
- - Load `${SKILL_DIR}/assets/behavior-risk-matrix.schema.json` during `CONTRACT_MODEL` or matrix validation.
99
- - Load `${SKILL_DIR}/assets/authoritative-records.schema.json` when creating or validating any other authoritative request/record.
100
- - Load `${SKILL_DIR}/assets/evidence-package-template.json` when initializing or resuming an evidence directory.
101
- - Read `${SKILL_DIR}/references/contract-mock-protocol.md` only when dependency, contract, or external-boundary risks require doubles.
102
- - Read `${SKILL_DIR}/references/test-generation-steps.md` during `INFRA_READY`, `TEST_BUILD`, or `EXECUTE_PROVE`.
103
- - Read `${SKILL_DIR}/references/service-boundary-test-catalog.md` only when domain boundary signals need risk discovery.
104
- - Read `${SKILL_DIR}/references/boundary-coverage-protocol.md` when defining module roots, exclusions, Regression Ring, or behavior-risk completeness.
89
+ Model outside-in from business capability or acceptance criterion to public entry point, rules, invariants, state, side effects, low-level logic, dependencies, and consumers. For each behavior, assess applicable:
105
90
 
106
- ## Required State Machine
107
-
108
- Execute in this exact order: CHANGE_CLASSIFY, SCOPE_DISCOVER, CONTRACT_MODEL, TEST_PLAN, INFRA_READY, TEST_BUILD, EXECUTE_PROVE, EVIDENCE_PACKAGE, EVIDENCE_VALIDATE. Do not claim a later state complete without valid applicable predecessor outputs.
109
-
110
- ### 1. CHANGE_CLASSIFY
111
-
112
- Classify the scope as `behavior` or `lightweight` and write `change-classification.json`.
91
+ ```text
92
+ functional | boundary | error | state | side-effect | permission
93
+ concurrency | idempotency | time | dependency | consumer
94
+ ```
113
95
 
114
- - `behavior`: code, runtime config, schema, migration, generated runtime, dependency, or mixed/uncertain scope; full protocol required.
115
- - `lightweight`: exclusively docs/comments/formatting with deterministic evidence that runtime behavior cannot change.
96
+ A dimension may be omitted only with a concise behavior-specific reason. Coverage percentages are diagnostic signals, never a completion threshold.
116
97
 
117
- Unknown classification follows the full protocol or becomes `TEST_BLOCKED`.
98
+ ### Test Layers
118
99
 
119
- ### 2. SCOPE_DISCOVER
100
+ Use the lowest layer that proves a property without losing relevant composition, then add higher layers for composition risk:
120
101
 
121
- Write `scope.json`, `target-inventory.json`, and `source-change.patch`. Use a model-authored inventory request with the bundled builder so actual discovered files are hashed. Record:
102
+ ```text
103
+ focused/unit
104
+ → module/component
105
+ → contract/integration
106
+ → code-level E2E when applicable
107
+ → complete affected-module regression
108
+ → Regression Ring verification
109
+ ```
122
110
 
123
- - changed files, all declared module roots, and evidence-backed exclusions;
124
- - affected module boundary (`explicit` or `cohesion-derived`);
125
- - Primary Scope containing all observable behavior of that module;
126
- - Regression Ring containing direct callers, consumers, shared contracts, and state dependencies, each mapped to planned tests;
127
- - contract, lockfile, test, and module-boundary discovery evidence;
128
- - Unresolved Edges for dynamic/unprovable coupling.
111
+ Do not mechanically test every private function. Add direct focused tests for critical low-level logic when complexity, boundaries, risk, or failure localization justify them; otherwise prove it through public behavior.
129
112
 
130
- Require every changed file to appear in inventory or a structured exclusion. Inventory and static discovery constrain obvious omissions but cannot deterministically prove semantic completeness; preserve that residual model judgment. Any verdict-capable unresolved edge blocks `TEST_PASS`.
113
+ ## Test Construction
131
114
 
132
- ### 3. CONTRACT_MODEL
115
+ Read `${SKILL_DIR}/references/test-coverage-model.md` and, when external or cross-module contracts apply, `${SKILL_DIR}/references/external-contract-mock-guidance.md`.
133
116
 
134
- Write `behavior-risk-matrix.json`. Resolve test truth in strict precedence:
117
+ Expected behavior follows this precedence:
135
118
 
136
- 1. specifications;
137
- 2. machine-readable contracts;
138
- 3. acceptance conditions;
119
+ 1. confirmed specification;
120
+ 2. machine-readable contract;
121
+ 3. acceptance criteria;
139
122
  4. trusted existing tests;
140
- 5. callers/consumers;
123
+ 5. callers and consumers;
141
124
  6. current implementation.
142
125
 
143
- Unresolved conflicting truth is `TEST_BLOCKED`, not a characterization test.
126
+ Conflicting higher-precedence truth that cannot be resolved is `TEST_BLOCKED`; do not preserve a possible implementation defect as a characterization test.
144
127
 
145
- For every observable behavior, record contract source, preconditions, input classes/boundaries, outputs, side effects, transitions, errors, and functional/boundary/permission/concurrency/idempotency/time/dependency/consumer risks. Every applicable cell maps to tests and executions or remains explicitly unresolved.
128
+ Tests must:
146
129
 
147
- Coverage metrics are diagnostic signals only and never substitute for this matrix.
130
+ - assert observable outputs, errors, state transitions, side effects, ordering, counts, and absence of partial effects;
131
+ - isolate only dependencies irrelevant to the property being proved;
132
+ - follow project-native naming, location, framework, fixture, and helper conventions;
133
+ - control clocks, randomness, concurrency, synchronization, and fixtures deterministically;
134
+ - verify failure cleanup and useful diagnostics;
135
+ - avoid vacuous assertions and assertions that merely duplicate implementation steps;
136
+ - never delete or weaken a valid existing test just to obtain green output.
148
137
 
149
- ### 4. TEST_PLAN
138
+ Reuse an adequate existing framework. Add only the smallest necessary project-native infrastructure when a required property cannot otherwise be expressed.
150
139
 
151
- Write the semantic preparation request under `requests/` and invoke `prepare-tests`; do not hand-author the resulting `test-plan.json`. Provide only repository-native `project_path` values, behavior mappings, selectors/runner hints, and the five layer decisions. The builder derives test IDs, status, snapshots, and hashes from inventory and canonical Git capture. Assess the ordered execution layers:
140
+ Minimal behavior-preserving testability seams are allowed, including dependency injection through an existing abstraction, pure-function extraction, an internal adapter, or controllable clock/randomness/state. Classify any production change under the repair boundary below.
152
141
 
153
- 1. focused;
154
- 2. module/component;
155
- 3. contract/integration;
156
- 4. complete affected-module regression;
157
- 5. Regression Ring.
142
+ ## Native Execution and Failure Repair
158
143
 
159
- Every omitted layer uses a structured N/A decision containing rationale, typed evidence, considered signals, and explicit explanations for detected conflicts. The model owns semantic judgment; deterministic validation checks shape and obvious conflicts.
144
+ The target project owns its runner, command, working directory, timeout, concurrency, retry behavior already encoded by the project, coverage tooling, and isolated integration environment. Do not impose an ecosystem-specific command policy.
160
145
 
161
- ### 5. INFRA_READY
146
+ During construction and repair, run the smallest relevant tests first. Before a pass decision, run all required layers, the complete affected-module regression, the Regression Ring, and project-wide regression when concrete coupling or project convention requires it.
162
147
 
163
- Discover and reuse native runners and conventions. Install only required dependencies, configure commands, and create fixtures/fakes/contract mock servers/isolated services as needed. Avoid duplicate frameworks and record every infrastructure change and verification in `infrastructure-changes.json`.
148
+ A nonzero execution is development feedback, not an immediate workflow stop. Classify it:
164
149
 
165
- External dependencies use `${SKILL_DIR}/references/contract-mock-protocol.md`. Shared machine-readable contracts are preferred; locally derived contract fixtures must record their source. Independently invented incompatible mocks are rejected.
150
+ | Classification | Action |
151
+ |---|---|
152
+ | `test-defect` | Repair the test, fixture, mock, import, syntax, or test infrastructure. |
153
+ | `local-production-defect` | Repair the proven internal implementation defect when public and cross-module contracts remain unchanged. |
154
+ | `high-risk-production-defect` | Repair only when safe, then require delta Code Review and return `TEST_NEEDS_FIXES`. |
155
+ | `environment-unavailable` | Return `TEST_BLOCKED` with the unavailable safe prerequisite. |
156
+ | `truth-unresolved` | Return `TEST_BLOCKED`; never guess the assertion. |
157
+ | `flaky-or-unreliable` | Diagnose within budget; otherwise return `TEST_BLOCKED`. |
166
158
 
167
- ### 6. TEST_BUILD
159
+ Use at most three execution-failure repair rounds. One round is reproduce → establish truth → repair → focused verification → affected-module and Regression Ring verification. Initial test construction is not a repair round. After the third unsuccessful round:
168
160
 
169
- Fill gaps across the whole affected module's observable behavior, contracts, boundaries, invariants, failure paths, transitions, and coupling risks. Do not test only the changed happy path or unrelated private implementation lines.
161
+ - known remaining correction `TEST_NEEDS_FIXES`;
162
+ - unresolved truth, reliability, safety, or environment → `TEST_BLOCKED`.
170
163
 
171
- Minimal behavior-preserving production refactoring is allowed only for test seams such as dependency injection, pure-function extraction, adapters, or controllable state. If testing requires changing a public production contract, issue `TEST_BLOCKED`.
164
+ Never retry until green or weaken scope/assertions to manufacture a pass.
172
165
 
173
- If a valid test reveals a business/contract defect, do not fix production behavior in this skill. Preserve the evidence for `TEST_FAIL`.
166
+ ## Production Repair Boundary
174
167
 
175
- ### 7. EXECUTE_PROVE
168
+ A repair may close inside this skill only when evidence establishes that it is limited to private/internal implementation, a behavior-preserving algorithm correction, test infrastructure, a behavior-preserving testability seam, or local error handling with unchanged public obligations.
176
169
 
177
- Run required layers in plan order through `${SKILL_DIR}/scripts/build_test_evidence.py execute`. Use exactly one of `test_paths`, `auto_bind=true`, or canonical `test_ids`; each must exactly match a deterministically inferable native command scope. The builder blocks opaque or mismatched commands and persists verified canonical test IDs before the native command runs. The model selects project-native argv, cwd, timeout, attempts, concurrency, layer, environment classification, and tool probes in schema-shaped requests. The runner performs the actual process execution and appends complete environment/tool versions, raw stdout/stderr hashes, request hash, runner hash, and unique receipt to `executions.json`. Do not accept caller-authored execution JSON and do not retry until green.
170
+ Set `review_required=true` and `review_scope=delta` when a repair affects or may affect:
178
171
 
179
- Validate the current implementation directly in the project root:
172
+ - public API or observable public behavior;
173
+ - schema, migration, serialization, or generated protocol shape;
174
+ - dependency or lockfile;
175
+ - authorization, identity, secrets, permissions, or tenant isolation;
176
+ - persistence, transaction, data integrity, or concurrency semantics;
177
+ - cross-module contracts or consumer obligations;
178
+ - compatibility guarantees;
179
+ - any change whose locality cannot be established confidently.
180
180
 
181
- - Run every required layer through the project-native runner; do not create historical checkouts or mutate a second copy of the repository.
182
- - Bind each applicable behavior risk to planned tests and selected, reliable execution receipts. A passing current-state test is valid evidence even when the historical implementation would also have passed it.
183
- - Preserve complete runner receipts, raw outputs, request bindings, and replay links. The builder owns execution provenance; the Main Agent does not author receipt records.
184
- - Required test failures produce `TEST_FAIL`; any selected unreliable execution, missing mapping, unavailable execution, unresolved risk, or budget truncation produces `TEST_BLOCKED`. `TEST_PASS` requires every selected execution to be reliable with exit code zero and every planned test to have a selected reliable zero-exit receipt.
181
+ Record completed repairs and test results, return `TEST_NEEDS_FIXES`, and stop. Do not invoke Code Review or prescribe caller routing from this skill.
185
182
 
186
- TDD history, baseline-failure proof, and mutation sensitivity are outside this skill. This skill validates whether the current implementation satisfies the modeled behavior and risk evidence.
183
+ ## Mandatory Main-Agent Test Review
187
184
 
188
- ### 8. EVIDENCE_PACKAGE
185
+ The Main Agent must personally review the complete current production-and-test state after construction and required execution. This responsibility may not be delegated.
189
186
 
190
- Create `.prizmkit/test/evidence/<evidence-id>/` from the package template with:
187
+ Use at most ten completed rounds. Track report facts:
191
188
 
192
- - `manifest.json`
193
- - `change-classification.json`
194
- - `scope.json`
195
- - `behavior-risk-matrix.json`
196
- - `test-plan.json`
197
- - `infrastructure-changes.json`
198
- - `executions.json`
199
- - `verdict.json`
200
- - `source-change.patch`
201
- - complete `raw/` stdout/stderr
202
- - `generated-tests/` snapshots
203
- - `contracts/` snapshots
204
- - derived `test-report.md`
205
-
206
- Hash baseline, diff, inventoried source/tests/contracts/lockfiles, environment, plan, and every evidence file. Evidence files remain individually content-addressed through `manifest.json`. Set `environment_claim=mocked-code-level-only`.
189
+ ```yaml
190
+ main_review_rounds: 0
191
+ accepted_findings: 0
192
+ fixed_findings: 0
193
+ rejected_findings: 0
194
+ unresolved_findings: 0
195
+ ```
207
196
 
208
- ### 9. EVIDENCE_VALIDATE
197
+ Every complete round checks:
209
198
 
210
- The validator runs in two passes:
199
+ - missing behavior, acceptance criteria, Regression Ring edges, and applicable risk dimensions;
200
+ - critical low-level boundary logic;
201
+ - incorrect truth precedence or expected behavior;
202
+ - weak, vacuous, implementation-only, nondeterministic, or flaky assertions;
203
+ - over-mocking, invented contracts, and missing negative side-effect assertions;
204
+ - unjustified layer omissions or tests absent from actual native execution;
205
+ - production repairs outside this skill's safe boundary.
211
206
 
212
- **First pass (pre-check, no `--attest`):**
207
+ For each candidate finding record exactly one decision:
213
208
 
214
- ```bash
215
- python3 ${SKILL_DIR}/scripts/validate_test_evidence.py \
216
- --evidence-dir .prizmkit/test/evidence/<evidence-id> \
217
- --project-root <target-project-root>
209
+ ```text
210
+ accepted | rejected | unresolved
218
211
  ```
219
212
 
220
- If this pre-check finds errors, fix them only while the package remains mutable and before `finalize` relocates it to its content-addressed immutable directory. Never edit `manifest.json`, `verdict.json`, `validation.json`, or any semantic record after finalization. The validator checks every authoritative record against its shipped schema, live target inventories, patch/diff binding, changed-file/module-root/exclusion/Regression Ring cross-links, generated-test snapshots, planned-test execution links, runner-generated receipts/request/raw-output hashes, successful behavior-risk mappings, production/unknown external-target safety, and package integrity.
213
+ Round behavior:
221
214
 
222
- **Second pass (attestation, with `--attest`):**
215
+ ```text
216
+ accepted = 0 and unresolved = 0
217
+ → Main-Agent review converged
223
218
 
224
- ```bash
225
- python3 ${SKILL_DIR}/scripts/validate_test_evidence.py \
226
- --evidence-dir .prizmkit/test/evidence/<evidence-id> \
227
- --project-root <target-project-root> \
228
- --attest
229
- ```
219
+ accepted > 0 and rounds remain
220
+ repair directly
221
+ targeted native verification
222
+ next complete review round
230
223
 
231
- `--attest` writes an integrity/protocol attestation into `validation.json`, re-derives `test-report.md`, re-hashes final records in `manifest.json`, then runs a **second** strict validation against the refreshed package. A package is not a `TEST_PASS` until this second strict validation succeeds. Use builder replay when feasible to rerun recorded requests and create new linked receipts. Store validation output. Tampering, schema/identity/hash mismatch, source/test/lockfile drift, missing matrix mappings, wrong execution order, unjustified structured N/A, unresolved risk, or a false real-environment claim fails validation.
224
+ known correction remains after round ten
225
+ → TEST_NEEDS_FIXES
232
226
 
233
- ## Resume and Idempotency
227
+ truth, safety, input, or environment remains unresolved
228
+ → TEST_BLOCKED
229
+ ```
234
230
 
235
- When interrupted or re-invoked:
231
+ All evidence-based rejections with no unresolved item are normal convergence. Any mutation invalidates prior review and execution of that content.
236
232
 
237
- 1. validate existing manifest, hashes, and stage outputs;
238
- 2. recompute code/test/contract/dependency/environment/plan input hashes;
239
- 3. preserve prior executions and raw output immutably;
240
- 4. invalidate the first changed stage and every downstream result;
241
- 5. resume from the last valid predecessor.
233
+ ## Optional Independent Test Reviewer
242
234
 
243
- For a large module or bounded budget, persist matrix/checkpoint progress and return `TEST_BLOCKED` until complete evidence exists. Never silently reduce scope.
235
+ Only after Main-Agent review converges, read `${SKILL_DIR}/references/independent-test-review.md` and follow its complete contract.
244
236
 
245
- ## Terminal Verdicts
237
+ - Create at most one Reviewer for this invocation.
238
+ - Permit at most five responses.
239
+ - Use native continuation of the exact same Reviewer after an accepted correction.
240
+ - The Reviewer is optional and available only when every structural capability in the reference is proven.
241
+ - If capability is unavailable or unproven, create no weaker replacement; record a visible strict downgrade and preserve the converged mandatory Main-Agent review.
242
+ - The Main Agent retains adjudication, mutation, execution, final result, and reporting authority.
243
+ - If response five still contains an accepted correction, return `TEST_NEEDS_FIXES`.
244
+ - If response five still contains unresolved truth, input, or safety, return `TEST_BLOCKED`.
245
+ - Never claim `TEST_PASS` for a final mutation that was not independently rechecked when independent review had already begun, unless the reference's strict downgrade rule establishes the permitted Main-Agent fallback.
246
246
 
247
- Only these testing-domain outcomes are valid:
247
+ ## External Services and Safety
248
248
 
249
- - `TEST_PASS`: every required module behavior/risk and Regression Ring check is covered by current-workspace tests, required executions pass reliably, no verdict-capable unresolved edge remains, and deterministic validation passes.
250
- - `TEST_FAIL`: a valid reliable test reproduces an implementation or resolved-contract failure. The authoritative verdict record must include `repair_scope` and enough evidence for the next stage to route safely.
251
- - `TEST_BLOCKED`: unknown scope, conflicting truth, unavailable/unreliable/flaky necessary execution, incomplete evidence, unmapped tests, budget truncation, or deterministic validation failure.
249
+ Read `${SKILL_DIR}/references/external-contract-mock-guidance.md` whenever an external service, persistence boundary, queue, filesystem, generated client, or cross-module endpoint affects the tested property.
252
250
 
253
- No conditional pass exists.
251
+ - Prefer project-owned schemas, generated types, SDK interfaces, fixtures, and documentation.
252
+ - Use approved retrieval of public official documentation only when project artifacts are insufficient.
253
+ - Never expose secrets, private source, proprietary payloads, or repository-private data during retrieval.
254
+ - Never use production credentials, production databases/APIs/queues/storage, real user data, or destructive external operations.
255
+ - If an endpoint cannot be proven non-production, do not connect to it.
256
+ - Use contract-backed local fakes, mocks, fixtures, mock servers, containers, or project-owned isolated services.
257
+ - Record necessary unverified external behavior as a remaining risk or `TEST_BLOCKED` when it prevents the verdict.
254
258
 
255
- ## Workflow State and Handoff
259
+ ## Final Verification
256
260
 
257
- Before writing state, read `${SKILL_DIR}/references/workflow-state-protocol.md`.
261
+ Before writing `TEST_PASS`:
258
262
 
259
- On `TEST_PASS`, update the active workflow state with `status=TEST_PASS`, `stage_result=TEST_PASS`, and `next_stage=retrospective`. On `TEST_FAIL`, persist the evidence-backed `repair_scope`, set `status=TEST_FAIL` and `stage_result=TEST_FAIL`, and return to implementation; test-infrastructure scope selects `next_stage=implement` with a direct test resume, while production-family scope selects `next_stage=implement` and requires code review before testing again. On `TEST_BLOCKED`, preserve the blocker and any validated finalized evidence paths, do not modify production code speculatively, and return a blocked result. A validator `PASS` means only that the evidence package is structurally valid; it does not mean the testing verdict passed. The authoritative verdict is the validated evidence record; the derived Markdown report is only a compatibility view. If an active orchestrator exists, return the terminal result and `next_stage` to it; do not invoke the next skill independently.
263
+ 1. Main-Agent review converged on the exact final state.
264
+ 2. Independent review converged or was strictly and visibly downgraded under its capability contract.
265
+ 3. Every required focused, module, contract/integration, applicable E2E, affected-module, and Regression Ring test passes reliably.
266
+ 4. Any required project-wide regression passes.
267
+ 5. No accepted correction, unresolved finding, verdict-capable scope edge, or required environment check remains.
268
+ 6. No high-risk production repair awaits delta Code Review.
269
+ 7. No mutation occurred after the final applicable review and execution.
260
270
 
261
- ## Derived Report and Handoff
271
+ ## Terminal Results
262
272
 
263
- Generate `test-report.md` from structured evidence using `${SKILL_DIR}/references/test-report-template.md`. It is a replaceable view, not the source of truth. Report only:
264
-
265
- - testing-domain verdict;
266
- - evidence directory and ID;
267
- - validator result and key execution pointers;
268
- - sensitivity and mocked-versus-real warning;
269
- - authoritative evidence pointers for external automation and interactive consumers.
270
-
271
- Do not embed an independent AI reviewer, decide overall code quality, claim broad Spec compliance, repair a validly detected business defect, authorize commit/release, or perform deployed test/production validation. The derived report is a view; `verdict.json`, validation records, and the evidence manifest remain authoritative.
273
+ Only these testing-domain outcomes are valid:
272
274
 
273
- ## Compatibility
275
+ ```text
276
+ TEST_PASS | TEST_NEEDS_FIXES | TEST_BLOCKED
277
+ ```
274
278
 
275
- Consumers must read authoritative `TEST_*` evidence. During migration, a legacy Markdown report may be used only as a compatibility view after the authoritative verdict has passed validation.
279
+ - `TEST_PASS`: final reviewed state passes every required native test and no high-risk repair awaits review.
280
+ - `TEST_NEEDS_FIXES`: a known correction remains, a review/repair budget did not converge, or a completed high-risk repair requires delta Code Review.
281
+ - `TEST_BLOCKED`: expected truth, safe input, environment, execution reliability, external-target safety, or required review input prevents a safe verdict.
276
282
 
277
- Existing Pipeline and Skill consumers are not migrated by this protocol. Temporary post-install incompatibility with callers expecting the legacy report is accepted and must be stated, not hidden.
283
+ No conditional pass exists.
278
284
 
279
- ## Examples
285
+ ## Terminal Report and Result
286
+
287
+ Read `${SKILL_DIR}/references/test-report-template.md` before finalization. Write `test-report.md` and `test-result.json` together for every terminal outcome.
288
+
289
+ `test-result.json` is a terminal projection, not a checkpoint:
290
+
291
+ ```json
292
+ {
293
+ "schema_version": 1,
294
+ "result": "TEST_PASS",
295
+ "report": "test-report.md",
296
+ "main_review_rounds": 3,
297
+ "independent_review": {
298
+ "status": "completed",
299
+ "responses": 2,
300
+ "downgrade_reason": null,
301
+ "final_state_rechecked": true
302
+ },
303
+ "repair_rounds": 1,
304
+ "production_changed": false,
305
+ "review_required": false,
306
+ "review_scope": null,
307
+ "unresolved_items": []
308
+ }
309
+ ```
280
310
 
281
- Read `${SKILL_DIR}/references/examples.md` only when a concrete normal, legacy gap-filling, contract-coupled, failure, blocked, recovery, or tampering example is needed.
311
+ Allowed `independent_review.status` values are `completed`, `downgraded`, and `not_applicable`. `completed` requires one through five responses and `final_state_rechecked=true`; `downgraded` requires zero responses, a non-empty reason, and `final_state_rechecked=false`; `not_applicable` requires zero responses, no reason, and `final_state_rechecked=false`. `review_scope` is `delta` only when `review_required=true`; otherwise it is null. The report and JSON must agree on result, review requirement, and unresolved items. If consistency cannot be established, write a truthful `TEST_BLOCKED` result rather than fabricating pass.
282
312
 
283
- **HANDOFF:** Return the strict testing verdict and structured evidence pointers only.
313
+ **HANDOFF:** Return the testing result plus `test-report.md` and `test-result.json` paths to the caller, then stop.