prizmkit 1.1.140 → 1.1.142

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (83) hide show
  1. package/bundled/VERSION.json +3 -3
  2. package/bundled/dev-pipeline/README.md +1 -1
  3. package/bundled/dev-pipeline/prizmkit_runtime/checkpoint_state.py +165 -125
  4. package/bundled/dev-pipeline/prizmkit_runtime/runner_classification.py +13 -1
  5. package/bundled/dev-pipeline/prizmkit_runtime/runner_models.py +0 -6
  6. package/bundled/dev-pipeline/prizmkit_runtime/runners.py +5 -3
  7. package/bundled/dev-pipeline/prizmkit_runtime/test_result.py +160 -0
  8. package/bundled/dev-pipeline/scripts/check-session-status.py +11 -1
  9. package/bundled/dev-pipeline/scripts/generate-bootstrap-prompt.py +23 -806
  10. package/bundled/dev-pipeline/scripts/generate-bugfix-prompt.py +6 -8
  11. package/bundled/dev-pipeline/scripts/generate-recovery-prompt.py +1 -1
  12. package/bundled/dev-pipeline/scripts/generate-refactor-prompt.py +6 -8
  13. package/bundled/dev-pipeline/scripts/prompt_framework.py +68 -56
  14. package/bundled/dev-pipeline/scripts/update-bug-status.py +21 -2
  15. package/bundled/dev-pipeline/scripts/update-checkpoint.py +52 -131
  16. package/bundled/dev-pipeline/scripts/update-feature-status.py +21 -2
  17. package/bundled/dev-pipeline/scripts/update-refactor-status.py +21 -2
  18. package/bundled/dev-pipeline/templates/bootstrap-prompt.md +3 -3
  19. package/bundled/dev-pipeline/templates/bootstrap-tier3.md +8 -19
  20. package/bundled/dev-pipeline/templates/bugfix-bootstrap-prompt.md +6 -6
  21. package/bundled/dev-pipeline/templates/refactor-bootstrap-prompt.md +5 -5
  22. package/bundled/dev-pipeline/templates/sections/bugfix-phase-commit-report.md +1 -1
  23. package/bundled/dev-pipeline/templates/sections/bugfix-phase-diagnose-plan.md +1 -1
  24. package/bundled/dev-pipeline/templates/sections/bugfix-phase-manual-verification.md +1 -1
  25. package/bundled/dev-pipeline/templates/sections/bugfix-phase-review.md +1 -1
  26. package/bundled/dev-pipeline/templates/sections/bugfix-phase-test.md +13 -10
  27. package/bundled/dev-pipeline/templates/sections/bugfix-reminders.md +1 -1
  28. package/bundled/dev-pipeline/templates/sections/checkpoint-system.md +18 -9
  29. package/bundled/dev-pipeline/templates/sections/headless-commit-authorization.md +1 -1
  30. package/bundled/dev-pipeline/templates/sections/phase-commit-full.md +1 -1
  31. package/bundled/dev-pipeline/templates/sections/phase-commit.md +1 -1
  32. package/bundled/dev-pipeline/templates/sections/phase-context-snapshot-agent-suffix.md +2 -2
  33. package/bundled/dev-pipeline/templates/sections/phase-context-snapshot-base.md +1 -1
  34. package/bundled/dev-pipeline/templates/sections/phase-context-snapshot-lite-suffix.md +1 -1
  35. package/bundled/dev-pipeline/templates/sections/phase-implement-lite.md +1 -1
  36. package/bundled/dev-pipeline/templates/sections/phase-prizmkit-test.md +14 -26
  37. package/bundled/dev-pipeline/templates/sections/phase-review-full.md +2 -2
  38. package/bundled/dev-pipeline/templates/sections/phase-specify-plan-full.md +3 -3
  39. package/bundled/dev-pipeline/templates/sections/refactor-phase-review.md +1 -1
  40. package/bundled/dev-pipeline/templates/sections/refactor-phase-test.md +13 -10
  41. package/bundled/dev-pipeline/templates/sections/refactor-reminders.md +1 -1
  42. package/bundled/dev-pipeline/templates/session-status-schema.json +1 -1
  43. package/bundled/dev-pipeline/tests/test_checkpoint_state.py +494 -413
  44. package/bundled/dev-pipeline/tests/test_generate_bootstrap_prompt.py +144 -684
  45. package/bundled/dev-pipeline/tests/test_generate_bugfix_prompt.py +20 -41
  46. package/bundled/dev-pipeline/tests/test_generate_refactor_prompt.py +24 -24
  47. package/bundled/dev-pipeline/tests/test_python_runner_parity.py +176 -36
  48. package/bundled/skills/_metadata.json +6 -6
  49. package/bundled/skills/prizmkit/SKILL.md +13 -18
  50. package/bundled/skills/prizmkit/references/workflow-state-protocol.md +73 -80
  51. package/bundled/skills/prizmkit-code-review/SKILL.md +7 -7
  52. package/bundled/skills/prizmkit-code-review/references/workflow-state-protocol.md +73 -80
  53. package/bundled/skills/prizmkit-committer/SKILL.md +8 -7
  54. package/bundled/skills/prizmkit-committer/references/workflow-state-protocol.md +73 -80
  55. package/bundled/skills/prizmkit-implement/SKILL.md +3 -2
  56. package/bundled/skills/prizmkit-implement/references/workflow-state-protocol.md +73 -80
  57. package/bundled/skills/prizmkit-plan/SKILL.md +2 -1
  58. package/bundled/skills/prizmkit-plan/references/workflow-state-protocol.md +73 -80
  59. package/bundled/skills/prizmkit-retrospective/SKILL.md +8 -6
  60. package/bundled/skills/prizmkit-retrospective/references/workflow-state-protocol.md +73 -80
  61. package/bundled/skills/prizmkit-test/SKILL.md +217 -189
  62. package/bundled/skills/prizmkit-test/references/external-contract-mock-guidance.md +119 -0
  63. package/bundled/skills/prizmkit-test/references/independent-test-review.md +269 -0
  64. package/bundled/skills/prizmkit-test/references/test-coverage-model.md +193 -0
  65. package/bundled/skills/prizmkit-test/references/test-report-template.md +164 -79
  66. package/bundled/skills/prizmkit-workflow/SKILL.md +28 -36
  67. package/bundled/skills/prizmkit-workflow/references/workflow-state-protocol.md +73 -80
  68. package/package.json +1 -1
  69. package/bundled/skills/prizmkit-test/assets/authoritative-records.schema.json +0 -477
  70. package/bundled/skills/prizmkit-test/assets/behavior-risk-matrix.schema.json +0 -116
  71. package/bundled/skills/prizmkit-test/assets/evidence-manifest.schema.json +0 -111
  72. package/bundled/skills/prizmkit-test/assets/evidence-package-template.json +0 -66
  73. package/bundled/skills/prizmkit-test/references/boundary-coverage-protocol.md +0 -104
  74. package/bundled/skills/prizmkit-test/references/contract-mock-protocol.md +0 -65
  75. package/bundled/skills/prizmkit-test/references/evidence-protocol.md +0 -208
  76. package/bundled/skills/prizmkit-test/references/evidence-request-protocol.md +0 -80
  77. package/bundled/skills/prizmkit-test/references/examples.md +0 -100
  78. package/bundled/skills/prizmkit-test/references/service-boundary-test-catalog.md +0 -228
  79. package/bundled/skills/prizmkit-test/references/test-generation-steps.md +0 -118
  80. package/bundled/skills/prizmkit-test/references/trusted-evidence-execution.md +0 -107
  81. package/bundled/skills/prizmkit-test/references/workflow-state-protocol.md +0 -181
  82. package/bundled/skills/prizmkit-test/scripts/build_test_evidence.py +0 -1930
  83. package/bundled/skills/prizmkit-test/scripts/validate_test_evidence.py +0 -1847
@@ -1,285 +1,313 @@
1
1
  ---
2
2
  name: "prizmkit-test"
3
- description: "Auditable test-evidence protocol for AI-generated behavior changes. Classifies changes, discovers affected modules and regression rings, models observable behavior and risk, builds project-native tests and contract-driven mocks, validates current-workspace test evidence, preserves content-addressed raw evidence, and deterministically returns TEST_PASS, TEST_FAIL, or TEST_BLOCKED. Use after code/config/schema/dependency changes or when users ask to test, verify, add complete module tests, inspect boundaries, or produce auditable evidence. Supports full-project, module, feature, and scope=this-change. (project)"
3
+ description: "AI-led project-native testing for the complete affected business module and its Regression Ring. Adds comprehensive behavior, boundary, error, state, side-effect, permission, concurrency, idempotency, time, dependency, and consumer tests; repairs valid failures in a bounded loop; performs mandatory Main-Agent review plus one optional strict independent Test Reviewer; and returns TEST_PASS, TEST_NEEDS_FIXES, or TEST_BLOCKED with a human report and minimal terminal result. (project)"
4
4
  ---
5
5
 
6
6
  # PrizmKit Test
7
7
 
8
- `/prizmkit-test` converts code changes into auditable testing evidence. Completeness means that the affected module's observable behavior, contracts, boundaries, invariants, errors, state transitions, and coupling risks are represented and proven. It does not mean mechanically reaching a line-coverage threshold.
8
+ `/prizmkit-test` is the independent testing stage for the current workspace. It understands the affected business behavior, fills project-native test gaps, executes the tests, repairs valid failures, reviews the final tests, and reports a truthful testing-domain result.
9
9
 
10
- This skill orchestrates each target project's native runners and conventions. It does not install a cross-language test runtime.
10
+ It does not build an evidence package, prove historical TDD order, validate a previous checkout, or manage pipeline/session state.
11
11
 
12
- ## Model Choice vs Fixed Evidence Mechanics
12
+ ## Atomic Skill Boundary
13
13
 
14
- Choose project-semantic parameters from the target project: framework, commands, cwd, timeout, retry/attempt policy, concurrency, Mock tools, test layers, module boundary, inventory patterns, and environment classification. Do not hard-code ecosystem-specific defaults when the project can decide them.
15
-
16
- Fix only the protocol mechanisms needed for safe, replayable evidence: project/evidence/request locators, schema-shaped requests, path confinement, real process execution, complete raw capture, runner-generated receipts, append-only history, and resume invalidation. Read `${SKILL_DIR}/references/trusted-evidence-execution.md` before inventory, execution, resume, or replay; it defines the trust boundary and builder commands. Read `${SKILL_DIR}/references/evidence-request-protocol.md` when authoring any request or authoritative JSON record; it defines request ownership, structured N/A, and the schema map.
17
-
18
- ## Canonical Evidence Builder Lifecycle
19
-
20
- The Main Agent supplies semantic requests only; it does not author authoritative package records. Initialize and drive one evidence directory through the builder in this order:
14
+ The Main Agent owns this invocation from analysis through final report:
21
15
 
22
16
  ```text
23
- init capture-change inventory → prepare-tests → execute → finalize → validate --attest → resume
17
+ resolve current change and requirement context
18
+ → discover affected module and Regression Ring
19
+ → model observable behavior and applicable risks
20
+ → inspect and add project-native tests
21
+ → execute and repair valid failures: maximum three rounds
22
+ → mandatory Main-Agent test review: maximum ten completed rounds
23
+ → optional independent Test Reviewer: one unit, maximum five responses
24
+ → final native regression on the exact final state
25
+ → write test-report.md and test-result.json
26
+ → return TEST_PASS | TEST_NEEDS_FIXES | TEST_BLOCKED
24
27
  ```
25
28
 
26
- Use `${SKILL_DIR}/scripts/build_test_evidence.py` for every lifecycle command:
27
-
28
- - `init`: creates the package directories and pins the baseline identity.
29
- - `capture-change`: captures the complete canonical change, including untracked, deleted, renamed, and copied files; never substitute plain `git diff`. `inventory` then requires its changed-file set to match the capture exactly and rejects stale captures or unaccounted live paths.
30
- - `inventory`: hashes current source/tests/contracts/lockfiles, rejects category overlap, excludes runtime-only paths, and validates directory roots plus the complete capture-derived changed-file set.
31
- - `prepare-tests`: discovers repository-native test files from the canonical capture and inventory, generates stable builder-owned test IDs, derives existing/added/modified status, and snapshots only added/modified live test sources.
32
- - `execute`: runs native project commands and owns `executions.json`, `receipts/`, and `raw/`. Each request uses exactly one binding mode: `test_ids`, `test_paths`, or `auto_bind=true`. The builder infers recognized package-wide or explicit file/directory/glob command scope, blocks opaque commands or mismatched declarations, and persists verified canonical `test_ids` only.
33
- - `finalize`: derives builder-owned target hashes, runs semantic-record and required-layer checks, rechecks canonical capture freshness, blocks every selected unreliable execution, and owns the payload manifest, verdict, generated-test snapshots, and evidence identity. It leaves the package in a mutable `payload-finalized` state until validator attestation; malformed semantic records fail before package relocation.
34
- - `render-report`: recovery/debugging command that derives `test-report.md`; normal finalization and attestation render it automatically, and it is never authoritative.
35
- - `resume`: preserves prior receipts and refuses to rebind an identity-changing package; create a new package after scope/source identity changes.
29
+ After writing its result, this skill returns control to its caller. It must not:
36
30
 
37
- `test-plan.json` is the only source of builder-generated test IDs. An execution request provides exactly one of native `test_paths`, `auto_bind=true`, or canonical `test_ids`; the actual command must expose the same deterministic scope, including all prepared tests for recognized package-wide commands. The builder blocks opaque or mismatched command scope and rewrites verified requests to canonical `test_ids` only before execution. Assess all five layers explicitly, but set `required=true` only when risk requires it; every omitted layer needs structured N/A evidence. The same binding applies test ↔ behavior ↔ execution; current-workspace receipts are the execution evidence.
31
+ - invoke implementation, Code Review, retrospective, or committer;
32
+ - update a workflow checkpoint or pipeline runtime state;
33
+ - classify an AI CLI session as success, crash, retry, or continuation;
34
+ - decide whether a workflow terminates or resumes;
35
+ - call a testing-domain non-pass result a runtime failure.
38
36
 
39
- The builder/validator, not the Main Agent, produces or updates `manifest.json`, `executions.json`, `receipts/*`, `validation.json`, `verdict.json`, `test-report.md`, and `source-change.patch`. Semantic records are authored through schema-shaped requests and must pass the builder preflight before finalization. `validation.json` is produced only by `validate_test_evidence.py --attest`; a package is not a `TEST_PASS` until the second strict validation succeeds.
37
+ ## Inputs and Outputs
40
38
 
39
+ | Parameter | Required | Description |
40
+ |---|---|---|
41
+ | `scope` | No | `full-project`, `module:<name>`, `feature:<slug>`, or `this-change`. Defaults to `this-change` when current change context exists, otherwise `full-project`. |
42
+ | `artifact_dir` | No | Active requirement directory containing available specification, plan, and prior review context. Reuse the caller-provided path. |
43
+ | `changed_files` | No | Explicit current changed paths. Highest-priority scope locator. |
44
+ | `diff_base` | No | Git comparison base when explicit changed paths are absent. |
45
+ | `test_commands` | No | Project-native commands when repository conventions are ambiguous. |
41
46
 
42
- For a formal requirement, `/prizmkit-test` runs after the Main-Agent code-review stage reaches `PASS` and before retrospective:
47
+ Write exactly two terminal artifacts in `artifact_dir` when it is available; otherwise use a clearly reported caller-visible artifact directory selected from the active request context:
43
48
 
44
49
  ```text
45
- plan → implement → code-review → test → retrospective → committer
50
+ {artifact_dir}/
51
+ ├── test-report.md
52
+ └── test-result.json
46
53
  ```
47
54
 
48
- Read `${SKILL_DIR}/references/workflow-state-protocol.md` before reading or updating lifecycle state. Reuse the caller's `artifact_dir`; do not select another recent plan. When an active `orchestrator` is recorded, return the testing result and next-stage metadata to it instead of invoking another skill independently.
49
-
50
- ## Atomic Stage Boundary
51
-
52
- `prizmkit-test` owns only auditable test evidence generation, validation, and the testing-domain verdict. It writes the authoritative `TEST_*` result and `next_stage`, then returns control. When an external orchestrator is active, it must not invoke retrospective or implementation itself. The active orchestrator owns the next-stage invocation and repair routing.
53
-
55
+ Do not create `.prizmkit/test/evidence/`, manifests, hashes, attestations, package lifecycle records, or test-internal checkpoints.
54
56
 
55
- - behavior-changing source code;
56
- - runtime configuration, schema, migration, generated-runtime, lockfile, or dependency changes;
57
- - public interfaces and data contracts;
58
- - frontend, backend, library, CLI, adapter, and multi-module changes;
59
- - legacy modules whose observable test coverage is incomplete;
60
- - explicit test, boundary, evidence, confidence, or audit requests.
57
+ ## Context Loading and Scope Resolution
61
58
 
62
- Documentation, comment, and formatting-only scopes use deterministic lightweight verification. They do not receive an unjustified full behavior-test claim.
59
+ 1. Reuse current requirement and workspace context already loaded by the Main Agent.
60
+ 2. Load `root.prizm`, the relevant module indexes, and the applicable detail docs before modifying source or tests.
61
+ 3. Inspect current staged, unstaged, untracked, deleted, and renamed paths. Changed lines locate impact but never define completeness.
62
+ 4. Read the available specification, acceptance criteria, plan, and prior review only as needed to establish expected behavior.
63
+ 5. Inspect manifests, runner configuration, CI conventions, existing test assertions, fixtures, fakes, mocks, contracts, and coverage support.
64
+ 6. If necessary scope cannot be determined safely, report the precise missing input and return `TEST_BLOCKED`.
63
65
 
64
- ## Preconditions and Safety
66
+ Do not infer the active requirement solely from the name of an old artifact directory. Do not use a historical worktree, second checkout, mutation proof, baseline-failure proof, or test overlay.
65
67
 
66
- 1. Load `root.prizm`, relevant L1/L2 docs, project manifests, runner configuration, and changed scope once.
67
- 2. Never use production credentials, production databases, production APIs, production queues/storage, or destructive operations against real data.
68
- 3. Preserve complete command/environment/output values as requested. Mark the package `sensitivity=project-controlled`; the project owns access control, retention, and upload policy.
69
- 4. Treat code-level dependencies as mock-first. Real deployed test-environment validation is a separate authorized activity.
70
- 5. Never delete existing tests.
71
- 6. Evidence-protocol recovery is separately bounded: allow at most two package initializations and two semantic preflight failures for one canonical change identity. After the budget is exhausted, return `TEST_BLOCKED`; do not create another versioned package or hand-edit finalized evidence.
72
- 7. For `TEST_FAIL`, classify and persist an evidence-backed `repair_scope` of `test-infrastructure`, `production`, `runtime`, `schema`, `dependency`, `public-interface`, or `unknown`. Test-infrastructure scope routes to implement then test; production-family scope routes to implement, code-review, then test. If evidence cannot support a safe classification, return `TEST_BLOCKED`.
68
+ ## Coverage Model
73
69
 
74
- If test infrastructure is missing, the infrastructure preparation state establishes only the necessary project-native infrastructure autonomously. Do not ask the user to install routine test dependencies in a headless run; verify dependency versions before editing manifests.
70
+ Read `${SKILL_DIR}/references/test-coverage-model.md` before designing or reviewing tests.
75
71
 
76
- ## Input
72
+ ### Affected Business Module
77
73
 
78
- | Parameter | Required | Description |
79
- |-----------|----------|-------------|
80
- | `scope` | No | `full-project`, `module:<name>`, `feature:<slug>`, or `this-change`. Defaults to `this-change` with a valid artifact directory, otherwise `full-project`. |
81
- | `artifact_dir` | No | Change artifact containing `spec.md` + `plan.md`; used to resolve acceptance conditions and changed scope. |
82
- | `changed_files` | No | Explicit changed files; highest-priority scope input. |
83
- | `diff_base` | No | Git baseline used when `changed_files` is absent. |
84
- | `evidence_target` | No | Existing evidence ID/directory to validate and resume. |
85
- | `execution_budget` | No | Bounded time/attempt budget. Incomplete necessary work is `TEST_BLOCKED`, never truncated to pass. |
86
- | `test_commands` | No | Explicit project-native commands when discovery is ambiguous. |
87
-
88
- For `scope=this-change`, determine changed files from `changed_files`, `diff_base`, current diff, artifact plan/context manifest, then spec mapping. Unknown scope is `TEST_BLOCKED`.
74
+ Prefer an explicit project module. If none exists, derive one cohesive boundary from files that jointly implement the same observable responsibility. Cover every discoverable observable behavior of that module, including relevant legacy behavior whose tests are incomplete.
89
75
 
90
- Do not infer a current change artifact from a legacy bug/refactor directory name alone; require explicit `artifact_dir` and changed-file evidence.
76
+ ### Regression Ring
91
77
 
92
- ## Authoritative Resources
78
+ Expand only along concrete coupling evidence:
93
79
 
94
- Load each resource only on its branch:
80
+ - direct callers;
81
+ - consumers of values, errors, events, files, or ordering;
82
+ - shared schemas, types, protocols, generated assets, and adapters;
83
+ - persistence, cache, lock, queue, transaction, and other shared state dependencies.
95
84
 
96
- - Read `${SKILL_DIR}/references/evidence-protocol.md` before a behavior evidence run or validation/resume decision — it defines state contracts, identity, proof, and verdict semantics.
97
- - Load `${SKILL_DIR}/assets/evidence-manifest.schema.json` when creating or validating `manifest.json`.
98
- - Load `${SKILL_DIR}/assets/behavior-risk-matrix.schema.json` during `CONTRACT_MODEL` or matrix validation.
99
- - Load `${SKILL_DIR}/assets/authoritative-records.schema.json` when creating or validating any other authoritative request/record.
100
- - Load `${SKILL_DIR}/assets/evidence-package-template.json` when initializing or resuming an evidence directory.
101
- - Read `${SKILL_DIR}/references/contract-mock-protocol.md` only when dependency, contract, or external-boundary risks require doubles.
102
- - Read `${SKILL_DIR}/references/test-generation-steps.md` during `INFRA_READY`, `TEST_BUILD`, or `EXECUTE_PROVE`.
103
- - Read `${SKILL_DIR}/references/service-boundary-test-catalog.md` only when domain boundary signals need risk discovery.
104
- - Read `${SKILL_DIR}/references/boundary-coverage-protocol.md` when defining module roots, exclusions, Regression Ring, or behavior-risk completeness.
85
+ Do not perform an unconditional repository-wide sweep.
105
86
 
106
- ## Required State Machine
87
+ ### Behavior and Risks
107
88
 
108
- Execute in this exact order: CHANGE_CLASSIFY, SCOPE_DISCOVER, CONTRACT_MODEL, TEST_PLAN, INFRA_READY, TEST_BUILD, EXECUTE_PROVE, EVIDENCE_PACKAGE, EVIDENCE_VALIDATE. Do not claim a later state complete without valid applicable predecessor outputs.
89
+ Model outside-in from business capability or acceptance criterion to public entry point, rules, invariants, state, side effects, low-level logic, dependencies, and consumers. For each behavior, assess applicable:
109
90
 
110
- ### 1. CHANGE_CLASSIFY
111
-
112
- Classify the scope as `behavior` or `lightweight` and write `change-classification.json`.
91
+ ```text
92
+ functional | boundary | error | state | side-effect | permission
93
+ concurrency | idempotency | time | dependency | consumer
94
+ ```
113
95
 
114
- - `behavior`: code, runtime config, schema, migration, generated runtime, dependency, or mixed/uncertain scope; full protocol required.
115
- - `lightweight`: exclusively docs/comments/formatting with deterministic evidence that runtime behavior cannot change.
96
+ A dimension may be omitted only with a concise behavior-specific reason. Coverage percentages are diagnostic signals, never a completion threshold.
116
97
 
117
- Unknown classification follows the full protocol or becomes `TEST_BLOCKED`.
98
+ ### Test Layers
118
99
 
119
- ### 2. SCOPE_DISCOVER
100
+ Use the lowest layer that proves a property without losing relevant composition, then add higher layers for composition risk:
120
101
 
121
- Write `scope.json`, `target-inventory.json`, and `source-change.patch`. Use a model-authored inventory request with the bundled builder so actual discovered files are hashed. Record:
102
+ ```text
103
+ focused/unit
104
+ → module/component
105
+ → contract/integration
106
+ → code-level E2E when applicable
107
+ → complete affected-module regression
108
+ → Regression Ring verification
109
+ ```
122
110
 
123
- - changed files, all declared module roots, and evidence-backed exclusions;
124
- - affected module boundary (`explicit` or `cohesion-derived`);
125
- - Primary Scope containing all observable behavior of that module;
126
- - Regression Ring containing direct callers, consumers, shared contracts, and state dependencies, each mapped to planned tests;
127
- - contract, lockfile, test, and module-boundary discovery evidence;
128
- - Unresolved Edges for dynamic/unprovable coupling.
111
+ Do not mechanically test every private function. Add direct focused tests for critical low-level logic when complexity, boundaries, risk, or failure localization justify them; otherwise prove it through public behavior.
129
112
 
130
- Require every changed file to appear in inventory or a structured exclusion. Inventory and static discovery constrain obvious omissions but cannot deterministically prove semantic completeness; preserve that residual model judgment. Any verdict-capable unresolved edge blocks `TEST_PASS`.
113
+ ## Test Construction
131
114
 
132
- ### 3. CONTRACT_MODEL
115
+ Read `${SKILL_DIR}/references/test-coverage-model.md` and, when external or cross-module contracts apply, `${SKILL_DIR}/references/external-contract-mock-guidance.md`.
133
116
 
134
- Write `behavior-risk-matrix.json`. Resolve test truth in strict precedence:
117
+ Expected behavior follows this precedence:
135
118
 
136
- 1. specifications;
137
- 2. machine-readable contracts;
138
- 3. acceptance conditions;
119
+ 1. confirmed specification;
120
+ 2. machine-readable contract;
121
+ 3. acceptance criteria;
139
122
  4. trusted existing tests;
140
- 5. callers/consumers;
123
+ 5. callers and consumers;
141
124
  6. current implementation.
142
125
 
143
- Unresolved conflicting truth is `TEST_BLOCKED`, not a characterization test.
144
-
145
- For every observable behavior, record contract source, preconditions, input classes/boundaries, outputs, side effects, transitions, errors, and functional/boundary/permission/concurrency/idempotency/time/dependency/consumer risks. Every applicable cell maps to tests and executions or remains explicitly unresolved.
126
+ Conflicting higher-precedence truth that cannot be resolved is `TEST_BLOCKED`; do not preserve a possible implementation defect as a characterization test.
146
127
 
147
- Coverage metrics are diagnostic signals only and never substitute for this matrix.
128
+ Tests must:
148
129
 
149
- ### 4. TEST_PLAN
130
+ - assert observable outputs, errors, state transitions, side effects, ordering, counts, and absence of partial effects;
131
+ - isolate only dependencies irrelevant to the property being proved;
132
+ - follow project-native naming, location, framework, fixture, and helper conventions;
133
+ - control clocks, randomness, concurrency, synchronization, and fixtures deterministically;
134
+ - verify failure cleanup and useful diagnostics;
135
+ - avoid vacuous assertions and assertions that merely duplicate implementation steps;
136
+ - never delete or weaken a valid existing test just to obtain green output.
150
137
 
151
- Write the semantic preparation request under `requests/` and invoke `prepare-tests`; do not hand-author the resulting `test-plan.json`. Provide only repository-native `project_path` values, behavior mappings, selectors/runner hints, and the five layer decisions. The builder derives test IDs, status, snapshots, and hashes from inventory and canonical Git capture. Assess the ordered execution layers:
138
+ Reuse an adequate existing framework. Add only the smallest necessary project-native infrastructure when a required property cannot otherwise be expressed.
152
139
 
153
- 1. focused;
154
- 2. module/component;
155
- 3. contract/integration;
156
- 4. complete affected-module regression;
157
- 5. Regression Ring.
140
+ Minimal behavior-preserving testability seams are allowed, including dependency injection through an existing abstraction, pure-function extraction, an internal adapter, or controllable clock/randomness/state. Classify any production change under the repair boundary below.
158
141
 
159
- Every omitted layer uses a structured N/A decision containing rationale, typed evidence, considered signals, and explicit explanations for detected conflicts. The model owns semantic judgment; deterministic validation checks shape and obvious conflicts.
142
+ ## Native Execution and Failure Repair
160
143
 
161
- ### 5. INFRA_READY
144
+ The target project owns its runner, command, working directory, timeout, concurrency, retry behavior already encoded by the project, coverage tooling, and isolated integration environment. Do not impose an ecosystem-specific command policy.
162
145
 
163
- Discover and reuse native runners and conventions. Install only required dependencies, configure commands, and create fixtures/fakes/contract mock servers/isolated services as needed. Avoid duplicate frameworks and record every infrastructure change and verification in `infrastructure-changes.json`.
146
+ During construction and repair, run the smallest relevant tests first. Before a pass decision, run all required layers, the complete affected-module regression, the Regression Ring, and project-wide regression when concrete coupling or project convention requires it.
164
147
 
165
- External dependencies use `${SKILL_DIR}/references/contract-mock-protocol.md`. Shared machine-readable contracts are preferred; locally derived contract fixtures must record their source. Independently invented incompatible mocks are rejected.
148
+ A nonzero execution is development feedback, not an immediate workflow stop. Classify it:
166
149
 
167
- ### 6. TEST_BUILD
150
+ | Classification | Action |
151
+ |---|---|
152
+ | `test-defect` | Repair the test, fixture, mock, import, syntax, or test infrastructure. |
153
+ | `local-production-defect` | Repair the proven internal implementation defect when public and cross-module contracts remain unchanged. |
154
+ | `high-risk-production-defect` | Repair only when safe, then require delta Code Review and return `TEST_NEEDS_FIXES`. |
155
+ | `environment-unavailable` | Return `TEST_BLOCKED` with the unavailable safe prerequisite. |
156
+ | `truth-unresolved` | Return `TEST_BLOCKED`; never guess the assertion. |
157
+ | `flaky-or-unreliable` | Diagnose within budget; otherwise return `TEST_BLOCKED`. |
168
158
 
169
- Fill gaps across the whole affected module's observable behavior, contracts, boundaries, invariants, failure paths, transitions, and coupling risks. Do not test only the changed happy path or unrelated private implementation lines.
159
+ Use at most three execution-failure repair rounds. One round is reproduce establish truth repair → focused verification → affected-module and Regression Ring verification. Initial test construction is not a repair round. After the third unsuccessful round:
170
160
 
171
- Minimal behavior-preserving production refactoring is allowed only for test seams such as dependency injection, pure-function extraction, adapters, or controllable state. If testing requires changing a public production contract, issue `TEST_BLOCKED`.
161
+ - known remaining correction `TEST_NEEDS_FIXES`;
162
+ - unresolved truth, reliability, safety, or environment → `TEST_BLOCKED`.
172
163
 
173
- If a valid test reveals a business/contract defect, do not fix production behavior in this skill. Preserve the evidence for `TEST_FAIL`.
164
+ Never retry until green or weaken scope/assertions to manufacture a pass.
174
165
 
175
- ### 7. EXECUTE_PROVE
166
+ ## Production Repair Boundary
176
167
 
177
- Run required layers in plan order through `${SKILL_DIR}/scripts/build_test_evidence.py execute`. Use exactly one of `test_paths`, `auto_bind=true`, or canonical `test_ids`; each must exactly match a deterministically inferable native command scope. The builder blocks opaque or mismatched commands and persists verified canonical test IDs before the native command runs. The model selects project-native argv, cwd, timeout, attempts, concurrency, layer, environment classification, and tool probes in schema-shaped requests. The runner performs the actual process execution and appends complete environment/tool versions, raw stdout/stderr hashes, request hash, runner hash, and unique receipt to `executions.json`. Do not accept caller-authored execution JSON and do not retry until green.
168
+ A repair may close inside this skill only when evidence establishes that it is limited to private/internal implementation, a behavior-preserving algorithm correction, test infrastructure, a behavior-preserving testability seam, or local error handling with unchanged public obligations.
178
169
 
179
- Validate the current implementation directly in the project root:
170
+ Set `review_required=true` and `review_scope=delta` when a repair affects or may affect:
180
171
 
181
- - Run every required layer through the project-native runner; do not create historical checkouts or mutate a second copy of the repository.
182
- - Bind each applicable behavior risk to planned tests and selected, reliable execution receipts. A passing current-state test is valid evidence even when the historical implementation would also have passed it.
183
- - Preserve complete runner receipts, raw outputs, request bindings, and replay links. The builder owns execution provenance; the Main Agent does not author receipt records.
184
- - Required test failures produce `TEST_FAIL`; any selected unreliable execution, missing mapping, unavailable execution, unresolved risk, or budget truncation produces `TEST_BLOCKED`. `TEST_PASS` requires every selected execution to be reliable with exit code zero and every planned test to have a selected reliable zero-exit receipt.
172
+ - public API or observable public behavior;
173
+ - schema, migration, serialization, or generated protocol shape;
174
+ - dependency or lockfile;
175
+ - authorization, identity, secrets, permissions, or tenant isolation;
176
+ - persistence, transaction, data integrity, or concurrency semantics;
177
+ - cross-module contracts or consumer obligations;
178
+ - compatibility guarantees;
179
+ - any change whose locality cannot be established confidently.
185
180
 
186
- TDD history, baseline-failure proof, and mutation sensitivity are outside this skill. This skill validates whether the current implementation satisfies the modeled behavior and risk evidence.
181
+ Record completed repairs and test results, return `TEST_NEEDS_FIXES`, and stop. Do not invoke Code Review or prescribe caller routing from this skill.
187
182
 
188
- ### 8. EVIDENCE_PACKAGE
183
+ ## Mandatory Main-Agent Test Review
189
184
 
190
- Create `.prizmkit/test/evidence/<evidence-id>/` from the package template with:
185
+ The Main Agent must personally review the complete current production-and-test state after construction and required execution. This responsibility may not be delegated.
191
186
 
192
- - `manifest.json`
193
- - `change-classification.json`
194
- - `scope.json`
195
- - `behavior-risk-matrix.json`
196
- - `test-plan.json`
197
- - `infrastructure-changes.json`
198
- - `executions.json`
199
- - `verdict.json`
200
- - `source-change.patch`
201
- - complete `raw/` stdout/stderr
202
- - `generated-tests/` snapshots
203
- - `contracts/` snapshots
204
- - derived `test-report.md`
187
+ Use at most ten completed rounds. Track report facts:
205
188
 
206
- Hash baseline, diff, inventoried source/tests/contracts/lockfiles, environment, plan, and every evidence file. Evidence files remain individually content-addressed through `manifest.json`. Set `environment_claim=mocked-code-level-only`.
207
-
208
- ### 9. EVIDENCE_VALIDATE
209
-
210
- The validator runs in two passes:
211
-
212
- **First pass (pre-check, no `--attest`):**
213
-
214
- ```bash
215
- python3 ${SKILL_DIR}/scripts/validate_test_evidence.py \
216
- --evidence-dir .prizmkit/test/evidence/<evidence-id> \
217
- --project-root <target-project-root>
189
+ ```yaml
190
+ main_review_rounds: 0
191
+ accepted_findings: 0
192
+ fixed_findings: 0
193
+ rejected_findings: 0
194
+ unresolved_findings: 0
218
195
  ```
219
196
 
220
- The first pass validates the payload while the package remains `payload-finalized`; `validation.json` is intentionally empty until attestation. Builder-derived target hashes and lifecycle records are generated before this check. If this pass finds semantic errors, repair the mutable staging package and rerun finalize; never repair a sealed package.
197
+ Every complete round checks:
221
198
 
222
- The validator checks every authoritative record against its shipped schema, live target inventories, patch/diff binding, changed-file/module-root/exclusion/Regression Ring cross-links, generated-test snapshots, planned-test execution links, runner-generated receipts/request/raw-output hashes, successful behavior-risk mappings, production/unknown external-target safety, and package integrity.
199
+ - missing behavior, acceptance criteria, Regression Ring edges, and applicable risk dimensions;
200
+ - critical low-level boundary logic;
201
+ - incorrect truth precedence or expected behavior;
202
+ - weak, vacuous, implementation-only, nondeterministic, or flaky assertions;
203
+ - over-mocking, invented contracts, and missing negative side-effect assertions;
204
+ - unjustified layer omissions or tests absent from actual native execution;
205
+ - production repairs outside this skill's safe boundary.
223
206
 
224
- ### Second pass (attestation)
207
+ For each candidate finding record exactly one decision:
225
208
 
226
- ```bash
227
- python3 ${SKILL_DIR}/scripts/validate_test_evidence.py \
228
- --evidence-dir .prizmkit/test/evidence/<evidence-id> \
229
- --project-root <target-project-root> \
230
- --attest
209
+ ```text
210
+ accepted | rejected | unresolved
231
211
  ```
232
212
 
233
- `--attest` writes an integrity/protocol attestation into `validation.json`, marks `EVIDENCE_VALIDATE` complete, re-derives `test-report.md`, re-hashes final records in `manifest.json`, seals the package, and then runs a **second** strict validation against the refreshed package. A package is not a `TEST_PASS` until this second strict validation succeeds. Use builder replay when feasible to rerun recorded requests and create new linked receipts. Store validation output. Tampering, schema/identity/hash mismatch, source/test/lockfile drift, missing matrix mappings, wrong execution order, unjustified structured N/A, unresolved risk, or a false real-environment claim fails validation.
213
+ Round behavior:
234
214
 
235
- ## Resume and Idempotency
215
+ ```text
216
+ accepted = 0 and unresolved = 0
217
+ → Main-Agent review converged
236
218
 
237
- When interrupted or re-invoked:
219
+ accepted > 0 and rounds remain
220
+ → repair directly
221
+ → targeted native verification
222
+ → next complete review round
238
223
 
239
- 1. validate existing manifest, hashes, and stage outputs;
240
- 2. recompute code/test/contract/dependency/environment/plan input hashes;
241
- 3. preserve prior executions and raw output immutably;
242
- 4. invalidate the first changed stage and every downstream result;
243
- 5. resume from the last valid predecessor.
224
+ known correction remains after round ten
225
+ TEST_NEEDS_FIXES
244
226
 
245
- For a large module or bounded budget, persist matrix/checkpoint progress and return `TEST_BLOCKED` until complete evidence exists. Never silently reduce scope.
227
+ truth, safety, input, or environment remains unresolved
228
+ → TEST_BLOCKED
229
+ ```
246
230
 
247
- ## Terminal Verdicts
231
+ All evidence-based rejections with no unresolved item are normal convergence. Any mutation invalidates prior review and execution of that content.
248
232
 
249
- Only these testing-domain outcomes are valid:
233
+ ## Optional Independent Test Reviewer
250
234
 
251
- - `TEST_PASS`: every required module behavior/risk and Regression Ring check is covered by current-workspace tests, required executions pass reliably, no verdict-capable unresolved edge remains, and deterministic validation passes.
252
- - `TEST_FAIL`: a valid reliable test reproduces an implementation or resolved-contract failure. The authoritative verdict record must include `repair_scope` and enough evidence for the next stage to route safely.
253
- - `TEST_BLOCKED`: unknown scope, conflicting truth, unavailable/unreliable/flaky necessary execution, incomplete evidence, unmapped tests, budget truncation, or deterministic validation failure.
235
+ Only after Main-Agent review converges, read `${SKILL_DIR}/references/independent-test-review.md` and follow its complete contract.
254
236
 
255
- No conditional pass exists.
237
+ - Create at most one Reviewer for this invocation.
238
+ - Permit at most five responses.
239
+ - Use native continuation of the exact same Reviewer after an accepted correction.
240
+ - The Reviewer is optional and available only when every structural capability in the reference is proven.
241
+ - If capability is unavailable or unproven, create no weaker replacement; record a visible strict downgrade and preserve the converged mandatory Main-Agent review.
242
+ - The Main Agent retains adjudication, mutation, execution, final result, and reporting authority.
243
+ - If response five still contains an accepted correction, return `TEST_NEEDS_FIXES`.
244
+ - If response five still contains unresolved truth, input, or safety, return `TEST_BLOCKED`.
245
+ - Never claim `TEST_PASS` for a final mutation that was not independently rechecked when independent review had already begun, unless the reference's strict downgrade rule establishes the permitted Main-Agent fallback.
256
246
 
257
- ## Workflow State and Handoff
247
+ ## External Services and Safety
258
248
 
259
- Before writing state, read `${SKILL_DIR}/references/workflow-state-protocol.md`.
249
+ Read `${SKILL_DIR}/references/external-contract-mock-guidance.md` whenever an external service, persistence boundary, queue, filesystem, generated client, or cross-module endpoint affects the tested property.
260
250
 
261
- On `TEST_PASS`, update the active workflow state with `status=TEST_PASS`, `stage_result=TEST_PASS`, and `next_stage=retrospective`. On `TEST_FAIL`, persist the evidence-backed `repair_scope`, set `status=TEST_FAIL` and `stage_result=TEST_FAIL`, and return to implementation; test-infrastructure scope selects `next_stage=implement` with a direct test resume, while production-family scope selects `next_stage=implement` and requires code review before testing again. On `TEST_BLOCKED`, preserve the blocker and any validated finalized evidence paths, do not modify production code speculatively, and return a blocked result. A validator `PASS` means only that the evidence package is structurally valid; it does not mean the testing verdict passed. The authoritative verdict is the validated evidence record; the derived Markdown report is only a compatibility view. If an active orchestrator exists, return the terminal result and `next_stage` to it; do not invoke the next skill independently.
251
+ - Prefer project-owned schemas, generated types, SDK interfaces, fixtures, and documentation.
252
+ - Use approved retrieval of public official documentation only when project artifacts are insufficient.
253
+ - Never expose secrets, private source, proprietary payloads, or repository-private data during retrieval.
254
+ - Never use production credentials, production databases/APIs/queues/storage, real user data, or destructive external operations.
255
+ - If an endpoint cannot be proven non-production, do not connect to it.
256
+ - Use contract-backed local fakes, mocks, fixtures, mock servers, containers, or project-owned isolated services.
257
+ - Record necessary unverified external behavior as a remaining risk or `TEST_BLOCKED` when it prevents the verdict.
262
258
 
263
- ## Derived Report and Handoff
259
+ ## Final Verification
264
260
 
265
- Generate `test-report.md` from structured evidence using `${SKILL_DIR}/references/test-report-template.md`. It is a replaceable view, not the source of truth. Report only:
261
+ Before writing `TEST_PASS`:
266
262
 
267
- - testing-domain verdict;
268
- - evidence directory and ID;
269
- - validator result and key execution pointers;
270
- - sensitivity and mocked-versus-real warning;
271
- - authoritative evidence pointers for external automation and interactive consumers.
263
+ 1. Main-Agent review converged on the exact final state.
264
+ 2. Independent review converged or was strictly and visibly downgraded under its capability contract.
265
+ 3. Every required focused, module, contract/integration, applicable E2E, affected-module, and Regression Ring test passes reliably.
266
+ 4. Any required project-wide regression passes.
267
+ 5. No accepted correction, unresolved finding, verdict-capable scope edge, or required environment check remains.
268
+ 6. No high-risk production repair awaits delta Code Review.
269
+ 7. No mutation occurred after the final applicable review and execution.
272
270
 
273
- Do not embed an independent AI reviewer, decide overall code quality, claim broad Spec compliance, repair a validly detected business defect, authorize commit/release, or perform deployed test/production validation. The derived report is a view; `verdict.json`, validation records, and the evidence manifest remain authoritative.
271
+ ## Terminal Results
274
272
 
275
- ## Compatibility
273
+ Only these testing-domain outcomes are valid:
276
274
 
277
- Consumers must read authoritative `TEST_*` evidence. During migration, a legacy Markdown report may be used only as a compatibility view after the authoritative verdict has passed validation.
275
+ ```text
276
+ TEST_PASS | TEST_NEEDS_FIXES | TEST_BLOCKED
277
+ ```
278
278
 
279
- Existing Pipeline and Skill consumers are not migrated by this protocol. Temporary post-install incompatibility with callers expecting the legacy report is accepted and must be stated, not hidden.
279
+ - `TEST_PASS`: final reviewed state passes every required native test and no high-risk repair awaits review.
280
+ - `TEST_NEEDS_FIXES`: a known correction remains, a review/repair budget did not converge, or a completed high-risk repair requires delta Code Review.
281
+ - `TEST_BLOCKED`: expected truth, safe input, environment, execution reliability, external-target safety, or required review input prevents a safe verdict.
280
282
 
281
- ## Examples
283
+ No conditional pass exists.
284
+
285
+ ## Terminal Report and Result
286
+
287
+ Read `${SKILL_DIR}/references/test-report-template.md` before finalization. Write `test-report.md` and `test-result.json` together for every terminal outcome. The report must use the exact canonical `## Final Result` section and `- Result: <TEST_*>` marker from that template; do not rename it to `Terminal Result`, `Final Decision`, or another heading.
288
+
289
+ `test-result.json` is a terminal projection, not a checkpoint:
290
+
291
+ ```json
292
+ {
293
+ "schema_version": 1,
294
+ "result": "TEST_PASS",
295
+ "report": "test-report.md",
296
+ "main_review_rounds": 3,
297
+ "independent_review": {
298
+ "status": "completed",
299
+ "responses": 2,
300
+ "downgrade_reason": null,
301
+ "final_state_rechecked": true
302
+ },
303
+ "repair_rounds": 1,
304
+ "production_changed": false,
305
+ "review_required": false,
306
+ "review_scope": null,
307
+ "unresolved_items": []
308
+ }
309
+ ```
282
310
 
283
- Read `${SKILL_DIR}/references/examples.md` only when a concrete normal, legacy gap-filling, contract-coupled, failure, blocked, recovery, or tampering example is needed.
311
+ Allowed `independent_review.status` values are `completed`, `downgraded`, and `not_applicable`. `completed` requires one through five responses and `final_state_rechecked=true`; `downgraded` requires zero responses, a non-empty reason, and `final_state_rechecked=false`; `not_applicable` requires zero responses, no reason, and `final_state_rechecked=false`. `review_scope` is `delta` only when `review_required=true`; otherwise it is null. The report and JSON must agree on result, review requirement, and unresolved items. If consistency cannot be established, write a truthful `TEST_BLOCKED` result rather than fabricating pass.
284
312
 
285
- **HANDOFF:** Return the strict testing verdict and structured evidence pointers only.
313
+ **HANDOFF:** Return the testing result plus `test-report.md` and `test-result.json` paths to the caller, then stop.