prizmkit 1.1.119 → 1.1.121

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (91) hide show
  1. package/bin/create-prizmkit.js +1 -5
  2. package/bundled/VERSION.json +3 -3
  3. package/bundled/dev-pipeline/README.md +41 -38
  4. package/bundled/dev-pipeline/assets/skill-subagent-integration.md +76 -0
  5. package/bundled/dev-pipeline/prizmkit_runtime/config.py +2 -8
  6. package/bundled/dev-pipeline/prizmkit_runtime/interoperability.py +1 -1
  7. package/bundled/dev-pipeline/prizmkit_runtime/platform_detection.py +3 -3
  8. package/bundled/dev-pipeline/prizmkit_runtime/runner_classification.py +23 -25
  9. package/bundled/dev-pipeline/prizmkit_runtime/runner_models.py +8 -2
  10. package/bundled/dev-pipeline/prizmkit_runtime/runner_prompts.py +10 -1
  11. package/bundled/dev-pipeline/prizmkit_runtime/runners.py +88 -32
  12. package/bundled/dev-pipeline/prizmkit_runtime/sessions.py +28 -6
  13. package/bundled/dev-pipeline/scripts/continuation.py +51 -6
  14. package/bundled/dev-pipeline/scripts/generate-bootstrap-prompt.py +60 -102
  15. package/bundled/dev-pipeline/scripts/generate-bugfix-prompt.py +1 -19
  16. package/bundled/dev-pipeline/scripts/generate-refactor-prompt.py +1 -19
  17. package/bundled/dev-pipeline/scripts/{init-dev-team.py → init-change-artifact.py} +6 -16
  18. package/bundled/dev-pipeline/scripts/parse-stream-progress.py +364 -76
  19. package/bundled/dev-pipeline/scripts/update-bug-status.py +57 -14
  20. package/bundled/dev-pipeline/scripts/update-feature-status.py +54 -5
  21. package/bundled/dev-pipeline/scripts/update-refactor-status.py +57 -14
  22. package/bundled/dev-pipeline/scripts/utils.py +1 -1
  23. package/bundled/dev-pipeline/templates/bootstrap-prompt.md +2 -2
  24. package/bundled/dev-pipeline/templates/bootstrap-tier3.md +9 -11
  25. package/bundled/dev-pipeline/templates/bugfix-bootstrap-prompt.md +2 -1
  26. package/bundled/dev-pipeline/templates/refactor-bootstrap-prompt.md +2 -2
  27. package/bundled/dev-pipeline/templates/sections/bugfix-phase-review.md +3 -3
  28. package/bundled/dev-pipeline/templates/sections/context-budget-rules.md +1 -1
  29. package/bundled/dev-pipeline/templates/sections/critical-paths-agent.md +0 -1
  30. package/bundled/dev-pipeline/templates/sections/critical-paths-full.md +0 -2
  31. package/bundled/dev-pipeline/templates/sections/phase-commit-full.md +8 -1
  32. package/bundled/dev-pipeline/templates/sections/phase-implement-agent.md +6 -7
  33. package/bundled/dev-pipeline/templates/sections/phase-implement-full.md +6 -7
  34. package/bundled/dev-pipeline/templates/sections/phase-implement-lite.md +4 -2
  35. package/bundled/dev-pipeline/templates/sections/phase-review-agent.md +3 -3
  36. package/bundled/dev-pipeline/templates/sections/phase-review-full.md +5 -3
  37. package/bundled/dev-pipeline/templates/sections/phase-specify-plan-full.md +20 -0
  38. package/bundled/dev-pipeline/templates/sections/refactor-phase-review.md +3 -3
  39. package/bundled/dev-pipeline/tests/test_auto_skip.py +5 -5
  40. package/bundled/dev-pipeline/tests/test_generate_bootstrap_prompt.py +214 -144
  41. package/bundled/dev-pipeline/tests/test_generate_bugfix_prompt.py +25 -12
  42. package/bundled/dev-pipeline/tests/test_generate_refactor_prompt.py +11 -14
  43. package/bundled/dev-pipeline/tests/test_python_runner_parity.py +235 -32
  44. package/bundled/dev-pipeline/tests/test_unified_cli.py +219 -16
  45. package/bundled/skills/_metadata.json +4 -4
  46. package/bundled/skills/app-planner/SKILL.md +2 -2
  47. package/bundled/skills/app-planner/references/architecture-decisions.md +1 -3
  48. package/bundled/skills/prizmkit-code-review/SKILL.md +50 -47
  49. package/bundled/skills/prizmkit-code-review/references/review-report-template.md +13 -8
  50. package/bundled/skills/prizmkit-code-review/references/reviewer-agent-prompt.md +47 -33
  51. package/bundled/skills/prizmkit-code-review/references/reviewer-workspace-protocol.md +90 -0
  52. package/bundled/skills/prizmkit-implement/SKILL.md +21 -1
  53. package/bundled/skills/prizmkit-implement/references/implementation-subagent-procedure.md +67 -0
  54. package/bundled/skills/prizmkit-test/SKILL.md +159 -155
  55. package/bundled/skills/prizmkit-test/assets/authoritative-records.schema.json +420 -0
  56. package/bundled/skills/prizmkit-test/assets/behavior-risk-matrix.schema.json +116 -0
  57. package/bundled/skills/prizmkit-test/assets/evidence-manifest.schema.json +112 -0
  58. package/bundled/skills/prizmkit-test/assets/evidence-package-template.json +97 -0
  59. package/bundled/skills/prizmkit-test/references/boundary-coverage-protocol.md +76 -192
  60. package/bundled/skills/prizmkit-test/references/contract-mock-protocol.md +65 -0
  61. package/bundled/skills/prizmkit-test/references/evidence-protocol.md +194 -0
  62. package/bundled/skills/prizmkit-test/references/evidence-request-protocol.md +78 -0
  63. package/bundled/skills/prizmkit-test/references/examples.md +65 -108
  64. package/bundled/skills/prizmkit-test/references/service-boundary-test-catalog.md +10 -7
  65. package/bundled/skills/prizmkit-test/references/test-generation-steps.md +90 -82
  66. package/bundled/skills/prizmkit-test/references/test-report-template.md +77 -98
  67. package/bundled/skills/prizmkit-test/references/trusted-evidence-execution.md +97 -0
  68. package/bundled/skills/prizmkit-test/scripts/build_test_evidence.py +723 -0
  69. package/bundled/skills/prizmkit-test/scripts/validate_test_evidence.py +1426 -0
  70. package/package.json +1 -1
  71. package/src/clean.js +12 -8
  72. package/src/config.js +24 -42
  73. package/src/index.js +1 -10
  74. package/src/manifest.js +3 -9
  75. package/src/metadata.js +0 -26
  76. package/src/prompts.js +0 -13
  77. package/src/scaffold.js +76 -201
  78. package/src/upgrade.js +16 -33
  79. package/bundled/adapters/claude/agent-adapter.js +0 -96
  80. package/bundled/adapters/claude/team-adapter.js +0 -183
  81. package/bundled/adapters/codebuddy/agent-adapter.js +0 -42
  82. package/bundled/adapters/codebuddy/team-adapter.js +0 -46
  83. package/bundled/adapters/codex/agent-adapter.js +0 -38
  84. package/bundled/adapters/codex/team-adapter.js +0 -37
  85. package/bundled/agents/prizm-dev-team-dev.md +0 -123
  86. package/bundled/agents/prizm-dev-team-reviewer.md +0 -113
  87. package/bundled/dev-pipeline/assets/prizm-dev-team-integration.md +0 -65
  88. package/bundled/dev-pipeline/templates/bootstrap-tier1.md +0 -436
  89. package/bundled/dev-pipeline/templates/bootstrap-tier2.md +0 -510
  90. package/bundled/skills/prizmkit-test/scripts/validate_boundary_report.py +0 -347
  91. package/bundled/team/prizm-dev-team.json +0 -27
@@ -0,0 +1,194 @@
1
+ # Test Evidence Protocol
2
+
3
+ This reference is normative for behavior-changing `/prizmkit-test` runs. Structured JSON is authoritative; `test-report.md` is a derived view.
4
+
5
+ ## Evidence Target and Identity
6
+
7
+ Write packages under `.prizmkit/test/evidence/<evidence-id>/`. Compute `evidence-id` as lowercase SHA-256 of canonical, key-sorted, compact JSON containing exactly:
8
+
9
+ - `baseline_commit`
10
+ - `working_diff_sha256`
11
+ - `scope_sha256`
12
+
13
+ Store those values as `evidence_id_inputs` in `manifest.json`. Record separate aggregate hashes for source, tests, contracts, lockfiles, full environment values, and plan inputs. Hash every evidence file except `manifest.json` in the manifest; do not create a self-referential manifest hash.
14
+
15
+ Hashes and validator receipts provide byte-level binding and drift detection, not cryptographic non-repudiation. A caller with the same OS permissions can replace the runner, requests, project tree, or package before validation. Preserve runner/request hashes and use replay to improve auditability without claiming resistance to a malicious same-privilege producer.
16
+
17
+ `source-change.patch` preserves the complete target diff. Snapshot generated/changed tests under `generated-tests/`, contracts under `contracts/`, and raw stdout/stderr under `raw/`. Never replace old execution/output records.
18
+
19
+ ## Ordered State Machine
20
+
21
+ Run exactly these states in order:
22
+
23
+ 1. `CHANGE_CLASSIFY`
24
+ 2. `SCOPE_DISCOVER`
25
+ 3. `CONTRACT_MODEL`
26
+ 4. `TEST_PLAN`
27
+ 5. `INFRA_READY`
28
+ 6. `TEST_BUILD`
29
+ 7. `EXECUTE_PROVE`
30
+ 8. `EVIDENCE_PACKAGE`
31
+ 9. `EVIDENCE_VALIDATE`
32
+
33
+ Each manifest stage records `name`, `status`, `input_sha256`, and output paths. A stage may be `not_applicable` only with a concrete rationale. A later state cannot complete until every applicable predecessor output exists, validates, and matches its recorded input hash.
34
+
35
+ ### CHANGE_CLASSIFY
36
+
37
+ Classify `behavior` when executable behavior can change through code, runtime configuration, schemas, migrations, generated runtime assets, lockfiles, or dependency changes. Classify `lightweight` only when the scope is exclusively documentation, comments, or formatting with no generated/runtime behavior effect.
38
+
39
+ For lightweight changes, record deterministic checks such as patch classification, parser/format validation, and generated-output consistency. Mark behavior-only stages not applicable with rationale. `TEST_PASS` then means only that the lightweight claim was proven; never describe it as complete runtime behavior evidence.
40
+
41
+ Unknown or mixed classification enters the full protocol; unresolved classification becomes `TEST_BLOCKED`.
42
+
43
+ ### SCOPE_DISCOVER
44
+
45
+ Record:
46
+
47
+ - Affected module: explicit module boundary or cohesion-derived files that jointly express one observable responsibility.
48
+ - Primary Scope: every public behavior of the affected module, not only changed lines.
49
+ - Regression Ring: directly affected callers, consumers, shared contracts, and state dependencies.
50
+ - Unresolved Edges: dynamic loading, reflection, generated clients, runtime registration, remote contracts, or coupling that cannot be proven statically.
51
+
52
+ Record changed files, declared module roots, contract/lockfile/test discovery, evidence-backed exclusions, and Regression Ring mappings in both scope and inventory records. Every changed file must be inventoried or explicitly excluded. The deterministic validator checks these declared relationships and obvious omissions, but inventory cannot prove that the model discovered every semantic behavior or dynamic edge.
53
+
54
+ Every edge records whether it can change the verdict. A verdict-capable unresolved edge prevents `TEST_PASS`.
55
+
56
+ ### CONTRACT_MODEL
57
+
58
+ Resolve expected truth in this order:
59
+
60
+ 1. specifications;
61
+ 2. machine-readable contracts;
62
+ 3. acceptance conditions;
63
+ 4. existing trusted tests;
64
+ 5. callers/consumers;
65
+ 6. current implementation.
66
+
67
+ Record conflicts rather than silently characterizing current behavior. If higher-precedence sources conflict and cannot be resolved, issue `TEST_BLOCKED`.
68
+
69
+ Populate `behavior-risk-matrix.json` using `assets/behavior-risk-matrix.schema.json`. Enumerate observable behavior, preconditions, input classes and boundary values, outputs, side effects, state transitions, errors, and every functional, boundary, permission, concurrency, idempotency, time, dependency, and consumer risk. Every risk is applicable, not applicable with concrete rationale, or unresolved.
70
+
71
+ Coverage percentages are diagnostic omission signals. They can prompt another matrix review but cannot prove or replace behavior-risk completeness.
72
+
73
+ ### TEST_PLAN
74
+
75
+ Assess all test layers:
76
+
77
+ 1. `focused`
78
+ 2. `module-component`
79
+ 3. `contract-integration`
80
+ 4. `affected-module-regression`
81
+ 5. `regression-ring`
82
+
83
+ Choose unit, module/component, contract, integration, and code-level E2E tests from actual risk. Do not require every test type mechanically. Every omitted layer needs a concrete not-applicable rationale. If a necessary layer cannot be executed within the configured budget, persist progress and issue `TEST_BLOCKED` until complete.
84
+
85
+ ### INFRA_READY
86
+
87
+ Discover manifests, existing tests, commands, runner configuration, fixtures, and conventions before changing infrastructure. Reuse an adequate native runner. Install only necessary dependencies after registry/version verification. Create only required configuration, fixtures, fakes, contract-backed mock servers, or isolated services. Never delete existing tests.
88
+
89
+ Record every infrastructure change, command, verification, and cleanup responsibility in `infrastructure-changes.json`. Production credentials, production databases, production APIs, and destructive real-data operations are prohibited.
90
+
91
+ ### TEST_BUILD
92
+
93
+ Fill the whole affected module's observable gaps: public behavior, contracts, boundaries, invariants, failure paths, state transitions, and coupling risks. Do not stop after the changed happy path and do not mechanically exercise unrelated private lines.
94
+
95
+ When code has no controllable seam, the test work may make the minimum behavior-preserving production refactor needed for dependency injection, pure-function extraction, adapters, or controllable state. Record the patch and behavior-preservation check. If a public production contract would need to change, stop with `TEST_BLOCKED`.
96
+
97
+ A valid test that exposes a business/contract defect is evidence for `TEST_FAIL`. Do not repair the business defect inside this skill.
98
+
99
+ ### EXECUTE_PROVE
100
+
101
+ Run progressively:
102
+
103
+ 1. focused tests;
104
+ 2. module/component tests;
105
+ 3. contract/integration tests;
106
+ 4. complete affected-module regression;
107
+ 5. Regression Ring verification.
108
+
109
+ Only omit plan-marked non-required layers. Every N/A is structured with rationale, typed evidence, considered signals, and explanations for detected conflicts. The model retains semantic judgment; validation checks structure and obvious signal conflicts, not hidden intent. Append only runner-generated execution receipts containing:
110
+
111
+ - runner-generated receipt format, runner hash, request path/hash, and receipt-chain predecessor hash;
112
+ - unique selected execution identifier;
113
+ - layer and exact argv;
114
+ - confined working directory;
115
+ - complete effective environment key/value map used by the command;
116
+ - tool/runtime probe results;
117
+ - integer exit code (booleans are invalid);
118
+ - paths and hashes for complete stdout and stderr;
119
+ - reliability/flakiness classification.
120
+
121
+ Do not overwrite failed or superseded attempts. Do not retry until green. Use a bounded documented recovery attempt; required flakiness, unavailable infrastructure, unreliable execution, or cleanup failure is `TEST_BLOCKED`.
122
+
123
+ For each added or changed necessary behavior test, prove the test:
124
+
125
+ 1. Prefer an isolated checkout or equivalent uncontaminated environment where the test fails against the pre-change baseline and passes against current code.
126
+ 2. If baseline execution is inapplicable (for example, the public behavior did not exist), apply a minimal controlled mutation tied to the same matrix risk, observe failure, restore it completely, and observe current success.
127
+ 3. Record `PROVEN`, justified `NOT_APPLICABLE`, or `UNPROVEN`.
128
+
129
+ Record baseline/current/mutation runner receipt IDs, request hash, baseline commit, current and isolation tree hashes, mutation apply/restore hashes, expected failure signal match, and original-tree cleanup result. Necessary `UNPROVEN` behavior prevents `TEST_PASS`.
130
+
131
+ ### EVIDENCE_PACKAGE
132
+
133
+ Create the complete package from `assets/evidence-package-template.json`, then hash its records. Set:
134
+
135
+ - `sensitivity=project-controlled`
136
+ - `environment_claim=mocked-code-level-only`
137
+ - `compatibility=legacy-test-report-interface-not-supported`
138
+
139
+ Preserve complete commands, environment values, stdout, stderr, patches, hashes, and history without automatic redaction. The project owns access control, retention, and upload policy. This does not permit production secrets or services.
140
+
141
+ Generate `test-report.md` solely from structured records. Regeneration must produce the same evidence pointers and verdict. The report is not independently editable authority.
142
+
143
+ Prepare `verdict.json` and the derived report, then let the validator finalize the non-circular attestation. Callers must not predeclare validator success as evidence.
144
+
145
+ ### EVIDENCE_VALIDATE
146
+
147
+ Run:
148
+
149
+ ```bash
150
+ python3 ${SKILL_DIR}/scripts/validate_test_evidence.py \
151
+ --evidence-dir .prizmkit/test/evidence/<evidence-id> \
152
+ --project-root <target-project-root> \
153
+ --attest
154
+ ```
155
+
156
+ `--attest` first validates the immutable payload while excluding only the final attestation/view records, atomically writes `validation.json`, refreshes final file hashes, then immediately performs a complete second validation. Subsequent checks omit `--attest` and verify the stored attestation. This avoids trusting a caller-authored `passed` marker or creating a self-referential manifest. The attestation proves package integrity and protocol consistency only; it is not proof against a same-permission producer.
157
+
158
+ When feasible, replay recorded requests through `build_test_evidence.py execute --replay-receipt ...`. Replay performs a new real execution and emits a new linked receipt; it does not pretend the attestation establishes what historically ran.
159
+
160
+ The validator checks all shipped JSON schemas, evidence-directory identity, live target-project inventories and aggregate hashes, changed-file/module-root/exclusion/Regression Ring cross-links, patch/diff binding, stage output ownership and predecessor hashes, generated-test snapshot existence, planned test/inventory/execution linkage, module/matrix completeness, successful selected runner receipts with request/raw-output binding, proof-linked isolated failures/current successes, controlled-mutation apply/restoration hashes, cleanup, structured N/A and obvious signal conflicts, production/unknown external-target safety, strict verdict semantics, and honest mocked-versus-real claims. `TEST_PASS` requires deterministic validator success.
161
+
162
+ ## Resume and Invalidation
163
+
164
+ On invocation for an existing evidence target:
165
+
166
+ 1. Validate manifest identity and every recorded file hash.
167
+ 2. Recompute source, tests, contracts, lockfiles, environment, and plan target hashes.
168
+ 3. Find the first stage whose input hash or output is invalid.
169
+ 4. Preserve all prior executions and raw outputs immutably.
170
+ 5. Invalidate that stage and every downstream stage; do not reuse their verdict claims.
171
+ 6. Resume from the first invalid stage and write new execution identifiers.
172
+
173
+ Stages are idempotent with respect to the same canonical inputs. Never silently truncate a large module. Persist matrix progress and return `TEST_BLOCKED` until every necessary behavior and Regression Ring cell is complete.
174
+
175
+ ## Terminal Semantics
176
+
177
+ - `TEST_PASS`: all required behavior/risk cells and Regression Ring checks are proven, all required executions are reliable and successful, no verdict-capable unresolved edge remains, cleanup succeeds, code-level requests remain replayable, and deterministic validation passes. This is protocol-validated code-level evidence, not a mathematical proof that no business behavior was omitted.
178
+ - `TEST_FAIL`: valid reliable evidence reproduces an implementation or resolved-contract failure. The package must remain complete enough to prove the failure.
179
+ - `TEST_BLOCKED`: scope/truth uncertainty, unavailable execution, necessary flakiness, incomplete evidence, failed cleanup, budget truncation, unproven behavior, conflicting truth, or deterministic validation failure.
180
+
181
+ No conditional pass exists.
182
+
183
+ ## Responsibility Boundary
184
+
185
+ Return only the testing-domain verdict and evidence pointers. Do not:
186
+
187
+ - embed an independent AI reviewer or adversarial PASS judge;
188
+ - decide overall code quality;
189
+ - claim broad Spec compliance beyond tested contracts;
190
+ - repair a business defect exposed by a valid test;
191
+ - authorize commit, release, or deployment;
192
+ - claim mocked code-level evidence validates a deployed test or production environment.
193
+
194
+ The legacy report/verdict interface is intentionally unsupported. Do not migrate Pipeline or other Skill consumers and do not add a compatibility layer in this change.
@@ -0,0 +1,78 @@
1
+ # Evidence Request Protocol
2
+
3
+ Load this reference when writing an inventory, execution, differential, resume, or replay request. Validate every authoritative record against `${SKILL_DIR}/assets/authoritative-records.schema.json`; the validator loads the named `$defs` for each file.
4
+
5
+ ## Request Ownership
6
+
7
+ The model writes project-semantic requests after inspecting manifests, runner configuration, contracts, test conventions, changed files, and module boundaries. The bundled builder owns process execution, raw output, receipts, isolation, tree hashes, and invalidation calculations.
8
+
9
+ Never hand-author `executions.json` receipt fields. Never copy a command's claimed output into `raw/`. Invoke the builder so provenance is linked to the actual process.
10
+
11
+ ## Inventory Request
12
+
13
+ Place a request under `requests/` with:
14
+
15
+ - `request_version: "1.0"`;
16
+ - `categories`: project-relative glob arrays for source, tests, contracts, and lockfiles;
17
+ - nonempty `changed_files` and declared `module_roots`;
18
+ - structured `exclusions`, each with path, rationale, and discovery evidence;
19
+ - `discovery_evidence` for manifests, runner configuration, filesystem discovery, contracts, lockfiles, and boundary analysis;
20
+ - nonempty `plan_inputs` carrying the semantic planning inputs whose hash controls resume.
21
+
22
+ An empty inventory is not evidence of an empty module. Ensure every changed file exists and is either inventoried or explicitly excluded with evidence. The builder independently enumerates every actual file under each model-declared module root; every enumerated file must appear in one inventory category or a structured exclusion. This does not decide whether the model selected the right semantic module boundary, but it prevents silent omission after that boundary is declared.
23
+
24
+ ## Execution Request
25
+
26
+ Use an argv array so the runner does not interpret a shell string. Required fields are:
27
+
28
+ - `request_version`, `purpose`, `command`, `cwd`;
29
+ - environment additions (the runner starts from a minimal runtime environment and the receipt records every effective value);
30
+ - model-selected `tool_version_commands`;
31
+ - selected `layer` and semantic `test_ids`;
32
+ - structured `external_targets`.
33
+
34
+ Optional `timeout_seconds`, `attempt_policy`, and `concurrency` remain model-chosen. The builder records them through request binding but does not invent values or silently retry.
35
+
36
+ The request path must be beneath the evidence directory. `cwd` resolves beneath the selected execution root. Missing commands produce an actual exit 127 receipt; missing cwd or escaped paths block before execution.
37
+
38
+ ## Differential Request
39
+
40
+ Record:
41
+
42
+ - behavior ID and model-selected `baseline` or `controlled-mutation` method;
43
+ - the nested execution request;
44
+ - actual baseline commit and current project-tree hash;
45
+ - mutation patch path when applicable;
46
+ - concrete expected failure signals.
47
+
48
+ The builder creates isolated copies, executes the failing and current sides, records receipts, binds the differential request hash, and verifies that the original project tree is unchanged. A proof is `PROVEN` only when the intended failure signal appears, the failing side exits nonzero, current exits zero, and cleanup/tree checks pass.
49
+
50
+ ## Structured N/A
51
+
52
+ Use the same structure for omitted layers, not-applicable risk cells, stage N/A, and proof N/A:
53
+
54
+ - `rationale`: behavior-specific reasoning;
55
+ - `evidence`: one or more typed discovery records;
56
+ - `considered_signals`: signals examined before deciding N/A;
57
+ - `conflicts`: every detected contradictory signal plus an explanation.
58
+
59
+ The model retains semantic judgment. The deterministic validator checks shape and obvious signal conflicts only. For example, auth/tenant/role signals conflict with permission N/A; lock/shared-state/worker signals conflict with concurrency N/A; retry/idempotency/dedup signals conflict with idempotency N/A; timeout/TTL/expiry signals conflict with time N/A; clients/contracts/network signals conflict with dependency N/A; callers/consumers/shared-contract mappings conflict with consumer N/A. A conflict can be accepted only when explicitly recorded and explained.
60
+
61
+ ## Record-to-Schema Map
62
+
63
+ | Authoritative record | Schema definition |
64
+ |---|---|
65
+ | `change-classification.json` | `changeClassification` |
66
+ | `scope.json` | `scope` |
67
+ | `target-inventory.json` | `targetInventory` |
68
+ | `test-plan.json` | `testPlan` |
69
+ | `requests/*.execution.json` | `executionRequest` |
70
+ | receipt entries in `executions.json` | `executionReceipt` / `executionLog` |
71
+ | `requests/*differential*.json` | `differentialRequest` |
72
+ | `differential-proof.json` | `differentialProof` |
73
+ | `infrastructure-changes.json` | `infrastructureChanges` |
74
+ | `verdict.json` | `verdict` |
75
+ | `validation.json` | `validation` |
76
+ | `lightweight-verification.json` | `lightweightVerification` |
77
+
78
+ `manifest.json` and `behavior-risk-matrix.json` retain their dedicated schemas. All schemas reject unknown fields where the protocol owns the record.
@@ -1,145 +1,102 @@
1
- # Test Examples
1
+ # Test Evidence Examples
2
2
 
3
- ## Example 1: Full-Project Quality Check with Test Generation
3
+ ## 1. Normally Tested Module Change TEST_PASS
4
4
 
5
- **User**: `/prizmkit-test`
5
+ Input:
6
6
 
7
- **Phase 0 — Architecture Detection**:
8
- ```
9
- Detected: Fullstack (React + Express)
10
- Test framework: Vitest (7 existing test files)
7
+ ```text
8
+ /prizmkit-test scope=this-change artifact_dir=<change-artifact>
11
9
  ```
12
10
 
13
- **Phase 1 Scope Selection** (user selects option 1):
14
- ```
15
- Scope: Full project (all modules, all specs)
16
- ```
11
+ A library parser changes one accepted input form. Scope discovery selects the complete parser module and two direct consumers. Existing module tests already cover most behaviors.
17
12
 
18
- **Phase 2 — Existing Tests**: 7 test files found, 20 tests run, 18 passed, 2 pre-existing failures (recorded, not fixed).
13
+ The run:
19
14
 
20
- **Phase 3 — Gap Analysis** (excerpt):
21
- | Module | Interface | Type | Status |
22
- |--------|-----------|------|--------|
23
- | auth | login() | unit | missing |
24
- | auth | register() | unit | missing |
25
- | payment | processPayment() | unit | missing |
26
- | payment | api/checkout | integration | missing |
27
- | — | Checkout flow | e2e | missing |
15
+ 1. Classifies `behavior`.
16
+ 2. Adds the changed parser behavior plus existing observable parser behavior to the matrix.
17
+ 3. Selects focused, affected-module regression, and Regression Ring layers; module-component and contract-integration are N/A with concrete pure-library rationale.
18
+ 4. Adds missing malformed-input and consumer-shape tests.
19
+ 5. Proves changed tests fail on baseline and pass currently.
20
+ 6. Packages all command/environment/output history.
21
+ 7. Deterministically validates.
28
22
 
29
- **Phase 4 — Boundary Matrix** (excerpt):
30
- | Module | Interface | Service Type | Happy Path | Request Validation | Auth / Permission / Ownership | Domain Invariants | Dependency Failure | Final Status |
31
- |--------|-----------|--------------|------------|--------------------|-------------------------------|-------------------|-------------------|--------------|
32
- | auth | login() | Auth / Session | covered | boundary-missing | boundary-missing | skipped | boundary-missing | boundary-missing |
33
- | payment | processPayment() | Payment / Billing | covered | boundary-missing | boundary-missing | boundary-missing | boundary-missing | boundary-missing |
23
+ Handoff:
34
24
 
35
- **Phase 5**: Generated 5 unit boundary tests (3 passing, 2 assertion failures marked `needs-review`), 1 integration boundary test (passing), 1 E2E test (passing).
36
-
37
- **Phase 6 — Report Summary**:
38
- ```
39
- Generated: 5 unit (2 needs-review), 1 integration, 1 E2E
40
- Boundary gate: failed — 2 interfaces need human review
41
- Report: .prizmkit/test/2026_05_23_14_30_00_testresult/test-report.md
42
- Needs human review: processPayment (returns 400 for negative amounts, expected 422), refund (missing authorization header causes 500 instead of 401)
25
+ ```text
26
+ TEST_PASS
27
+ Evidence: .prizmkit/test/evidence/<evidence-id>/
28
+ Validator: passed
29
+ Mocked code-level evidence does not verify a real deployed environment.
43
30
  ```
44
31
 
45
- ## Example 2: Single Feature/Spec Targeted Test
32
+ ## 2. Incompletely Tested Legacy Module — Whole-Module Gap Filling
46
33
 
47
- This is an interactive selection of one existing spec directory. It is different from headless `scope=this-change`, which is driven by a current change artifact plus changed files.
34
+ A queue worker retry change touches a legacy worker with one happy-path test. Changed lines identify the module, but Primary Scope includes parsing, claim, processing, retry, dead-letter, acknowledgment, idempotency, and dependency-failure behavior.
48
35
 
49
- **User**: `/prizmkit-test`
36
+ `TEST_BUILD` fills observable gaps across the module, not unrelated private utility lines. It adds deterministic fake-clock and contract-backed queue tests for retry just below/at/above maximum, duplicate delivery, malformed payload, lock failure, no partial side effect, ack/nack calls, and dead-letter transition. Regression Ring runs the scheduler and producer contract checks. Coverage percentage is recorded only as a diagnostic signal.
50
37
 
51
- **Phase 0 Architecture Detection**:
52
- ```
53
- Detected: Backend (Express)
54
- Test framework: Vitest
55
- ```
38
+ The run cannot pass until every applicable behavior-risk cell and Regression Ring execution is mapped and proven.
56
39
 
57
- **Phase 1 Scope Selection** (user selects option 3, then picks "042-payment-gateway"):
58
- ```
59
- Scope: Single feature — 042-payment-gateway
60
- Test files in scope: 2
61
- ```
40
+ ## 3. Contract-Coupled Modules Contract-Driven Mocks
62
41
 
63
- **Phase 2 Existing Tests**: 2 test files, 8 tests, all passing.
42
+ An API client and consumer share OpenAPI. The run snapshots the OpenAPI document into `contracts/`, records its source/hash, and generates mock responses for success, schema error, 429, timeout, malformed body, and pagination token boundaries.
64
43
 
65
- **Phase 3 Gap Analysis** (excerpt):
66
- | Module | Interface | Type | Status |
67
- |--------|-----------|------|--------|
68
- | payment | processPayment() | unit | boundary-missing |
69
- | payment | refund() | unit | boundary-missing |
44
+ The mock is rejected if it invents an incompatible field or cannot trace behavior to the contract. Contract and consumer executions pass, and the report states `environment_claim=mocked-code-level-only`; it makes no claim about a deployed SaaS environment.
70
45
 
71
- **Phase 4 — Boundary Matrix**:
72
- | Module | Interface | Service Type | Happy Path | Request Validation | Auth / Permission / Ownership | Domain Invariants | State Transitions | Dependency Failure | Final Status |
73
- |--------|-----------|--------------|------------|--------------------|-------------------------------|-------------------|-------------------|-------------------|--------------|
74
- | payment | processPayment() | Payment / Billing | covered | boundary-missing | boundary-missing | boundary-missing | generated | boundary-missing | boundary-missing |
75
- | payment | refund() | Payment / Refund | covered | boundary-missing | boundary-missing | boundary-missing | boundary-missing | skipped — provider sandbox unavailable | boundary-missing |
46
+ ## 4. Reliable Business Defect TEST_FAIL
76
47
 
77
- **Phase 5**: Generated 8 unit tests covering amount boundaries, currency mismatch, missing entitlement, duplicate idempotency key, partial refund, full refund, over-refund, and repository failure. All passing.
48
+ Resolved specification and OpenAPI require a negative amount to return a validation error without a write. A generated contract test reliably observes success plus a database-adapter call.
78
49
 
79
- **Phase 6 Report Summary**:
80
- ```
81
- Generated: 8 unit tests (all passing)
82
- Boundary gate: passed
83
- Report: .prizmkit/test/2026_05_23_15_00_00_testresult/test-report.md
84
- All tests passing and boundary coverage complete. Ready for commit.
50
+ The test is valid, current and affected-module executions reproduce the failure, and raw output is preserved. `/prizmkit-test` does not change production business behavior. It packages:
51
+
52
+ ```text
53
+ TEST_FAIL
54
+ Reproduced failure: amount-boundary-negative
55
+ Evidence: .prizmkit/test/evidence/<evidence-id>/
85
56
  ```
86
57
 
87
- ## Example 3: Existing Happy Path Is Not Enough
58
+ ## 5. Unknown Coupling or Conflicting Truth TEST_BLOCKED
88
59
 
89
- **User**: `/prizmkit-test user module`
60
+ A dynamically loaded plugin may consume the changed result shape, but its registry is generated remotely and unavailable. The edge can change the verdict, so it remains verdict-capable and unresolved.
90
61
 
91
- **Existing tests found**:
92
- ```
93
- TestHandler_UpdateProfile_Success
94
- TestHandler_GetProfile_Success
95
- ```
62
+ Alternatively, specification and shared schema disagree on nullability with no authoritative resolution. The skill does not freeze current behavior into a characterization test.
96
63
 
97
- **Boundary Matrix result**:
98
- | Module | Interface | Service Type | Happy Path | Request Validation | Auth / Permission / Ownership | Domain Invariants | Dependency Failure | Final Status |
99
- |--------|-----------|--------------|------------|--------------------|-------------------------------|-------------------|-------------------|--------------|
100
- | user | POST /user/update-profile | Validation / CRUD | covered | boundary-missing | boundary-missing | boundary-missing | boundary-missing | boundary-missing |
101
- | user | POST /user/get-profile | CRUD / Auth | covered | N/A — empty request | boundary-missing | N/A — no domain input | boundary-missing | boundary-missing |
64
+ Both produce:
102
65
 
103
- **Correct action**:
104
- Generate tests for:
105
- - missing auth,
106
- - invalid gender,
107
- - invalid activity level,
108
- - implausible height/weight when specified by service contract,
109
- - user not found,
110
- - repository failure,
111
- - response envelope shape.
66
+ ```text
67
+ TEST_BLOCKED
68
+ Blocker: verdict-capable unresolved edge or conflicting truth
69
+ Evidence: .prizmkit/test/evidence/<evidence-id>/
70
+ ```
112
71
 
113
- **Incorrect action**:
114
- Do not report `user` as covered just because both endpoints have success tests.
72
+ ## 6. Baseline Inapplicable — Controlled Mutation
115
73
 
74
+ A newly added CLI command has no pre-change interface, so baseline execution is justified `NOT_APPLICABLE` as a direct proof method. In an uncontaminated checkout, the run applies a minimal mutation that removes the command registration, observes the command test fail for that risk, restores the patch, verifies source hashes and cleanup, then observes current success. The behavior is `PROVEN` by `controlled-mutation`.
116
75
 
117
- ## Example 4: Change Artifact this-change Run
76
+ If restoration cannot be proven, the verdict is `TEST_BLOCKED`.
118
77
 
119
- **Caller**: `/prizmkit-test scope=this-change artifact_dir=.prizmkit/specs/{{FEATURE_SLUG}}/`
78
+ ## 7. Interrupted or Budgeted Run
120
79
 
121
- For non-feature changes, use the same command shape with any explicit change artifact directory that contains `spec.md` + `plan.md` and has a changed-files source. The `{{FEATURE_SLUG}}` path below is a compatibility example for existing feature artifacts; it does not limit `scope=this-change` to feature work.
80
+ On recovery, the skill validates stage outputs and target hashes. A changed lockfile invalidates `INFRA_READY` and every downstream stage but preserves earlier execution history immutably. The run resumes there.
122
81
 
123
- **Expected behavior**:
124
- - Reads the change artifact's `spec.md`, `plan.md`, and optional `context-snapshot.md`.
125
- - Builds in-scope files from changed files and the change artifact.
126
- - Generates or updates only tests for this change's files, public interfaces, and acceptance criteria.
127
- - Records unrelated historical gaps under `Out of Scope Gaps`.
128
- - Does not generate tests for unrelated modules.
129
- - The change artifact may represent a feature, bug fix, refactor, migration, or other scoped change.
82
+ If the budget ends after half a large module's required matrix is tested, progress persists and the result is `TEST_BLOCKED`; no scope is silently truncated.
130
83
 
131
- **Report excerpt**:
132
- ```markdown
133
- ## Scope
134
- - Mode: this-change
135
- - Artifact Dir: .prizmkit/specs/{{CHANGE_SLUG}}/
136
- - Generation Policy: in-scope-only
84
+ ## 8. Tampering
137
85
 
138
- ## Out of Scope Gaps
139
- - payment: missing legacy tests, but this change did not affect payment code.
86
+ After packaging, changing `raw/execution-003.stdout`, a contract snapshot, generated test, source patch, or derived report produces a content-hash mismatch. Changing identity inputs without recomputing the canonical evidence ID also fails.
140
87
 
141
- ## Verdict
142
- PASS
88
+ ```bash
89
+ python3 ${SKILL_DIR}/scripts/validate_test_evidence.py \
90
+ --evidence-dir .prizmkit/test/evidence/<evidence-id> \
91
+ --project-root <target-project-root>
143
92
  ```
144
93
 
145
- **HANDOFF**: Independent skill — no handoff. User may proceed to `/prizmkit-committer` only if tests pass and the boundary completion gate passes, or fix issues manually if tests are marked `needs-review`.
94
+ Result: non-zero with `Test evidence validation: FAIL`. A tampered package cannot retain `TEST_PASS`.
95
+
96
+ ## 9. Lightweight Change
97
+
98
+ A comments-only diff is deterministically classified and parser/format/generated-output checks pass. Behavior-only stages are marked not applicable with reasons. `TEST_PASS` means the lightweight claim is proven; the report explicitly does not claim full runtime test evidence.
99
+
100
+ ## Compatibility Notice
101
+
102
+ The legacy `PASS | NEEDS_FIXES | BLOCKED` Markdown report interface is intentionally not preserved. Existing Pipeline and Skill consumers are not migrated here and may be temporarily incompatible after installation.
@@ -1,14 +1,17 @@
1
- # Service Boundary Test Catalog
1
+ # Service Behavior-Risk Catalog
2
2
 
3
- Use this catalog during `/prizmkit-test` Phase 4a when generating unit tests for service-like functions or class methods. The goal is to turn "write boundary tests" into service-specific scenarios that match the unit's business responsibility.
3
+ Use this catalog during `CONTRACT_MODEL` and `TEST_BUILD` to discover domain-specific observable risks. It supplies questions, not truth: every expected value must come from the recorded truth precedence and every external dependency double must follow `references/contract-mock-protocol.md`.
4
4
 
5
5
  ## How to Use This Catalog
6
6
 
7
- 1. Classify the tested unit by reading its name, module path, parameters, dependencies, state changes, and related Prizm docs/spec acceptance criteria.
8
- 2. Select every service type that applies. A function can be both a payment service and an idempotency service, or both a repository and pagination service.
9
- 3. Map candidate boundaries to the actual contract in code/docs. Assert behavior that is specified or clearly implied; if expected behavior is unclear, mark the case as `needs-review` instead of inventing production behavior.
10
- 4. Mock external dependencies for unit tests: databases, network calls, file storage, queues, payment gateways, email providers, clocks, and random ID generators.
11
- 5. Record non-applicable categories as `N/A` with a short reason in the boundary coverage report so skipped categories are explicit.
7
+ 1. Classify the affected module by its public behavior, dependencies, state changes, and resolved contracts.
8
+ 2. Select every service type that applies; one behavior can combine several types.
9
+ 3. Add applicable cases to `behavior-risk-matrix.json` and map them to test/execution IDs.
10
+ 4. Derive mock requests/responses from shared machine-readable or locally snapshotted contracts. Never independently invent provider/database/queue behavior.
11
+ 5. Record a concrete rationale for every non-applicable risk.
12
+ 6. Treat coverage metrics as diagnostic only; complete observable behavior and risk rather than private lines.
13
+
14
+ Code-level tests are mock-first and must state that they do not verify a real deployed environment. Production credentials, services, and data are prohibited.
12
15
 
13
16
  ## Service Type Detection
14
17