prizmkit 1.1.140 → 1.1.142
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bundled/VERSION.json +3 -3
- package/bundled/dev-pipeline/README.md +1 -1
- package/bundled/dev-pipeline/prizmkit_runtime/checkpoint_state.py +165 -125
- package/bundled/dev-pipeline/prizmkit_runtime/runner_classification.py +13 -1
- package/bundled/dev-pipeline/prizmkit_runtime/runner_models.py +0 -6
- package/bundled/dev-pipeline/prizmkit_runtime/runners.py +5 -3
- package/bundled/dev-pipeline/prizmkit_runtime/test_result.py +160 -0
- package/bundled/dev-pipeline/scripts/check-session-status.py +11 -1
- package/bundled/dev-pipeline/scripts/generate-bootstrap-prompt.py +23 -806
- package/bundled/dev-pipeline/scripts/generate-bugfix-prompt.py +6 -8
- package/bundled/dev-pipeline/scripts/generate-recovery-prompt.py +1 -1
- package/bundled/dev-pipeline/scripts/generate-refactor-prompt.py +6 -8
- package/bundled/dev-pipeline/scripts/prompt_framework.py +68 -56
- package/bundled/dev-pipeline/scripts/update-bug-status.py +21 -2
- package/bundled/dev-pipeline/scripts/update-checkpoint.py +52 -131
- package/bundled/dev-pipeline/scripts/update-feature-status.py +21 -2
- package/bundled/dev-pipeline/scripts/update-refactor-status.py +21 -2
- package/bundled/dev-pipeline/templates/bootstrap-prompt.md +3 -3
- package/bundled/dev-pipeline/templates/bootstrap-tier3.md +8 -19
- package/bundled/dev-pipeline/templates/bugfix-bootstrap-prompt.md +6 -6
- package/bundled/dev-pipeline/templates/refactor-bootstrap-prompt.md +5 -5
- package/bundled/dev-pipeline/templates/sections/bugfix-phase-commit-report.md +1 -1
- package/bundled/dev-pipeline/templates/sections/bugfix-phase-diagnose-plan.md +1 -1
- package/bundled/dev-pipeline/templates/sections/bugfix-phase-manual-verification.md +1 -1
- package/bundled/dev-pipeline/templates/sections/bugfix-phase-review.md +1 -1
- package/bundled/dev-pipeline/templates/sections/bugfix-phase-test.md +13 -10
- package/bundled/dev-pipeline/templates/sections/bugfix-reminders.md +1 -1
- package/bundled/dev-pipeline/templates/sections/checkpoint-system.md +18 -9
- package/bundled/dev-pipeline/templates/sections/headless-commit-authorization.md +1 -1
- package/bundled/dev-pipeline/templates/sections/phase-commit-full.md +1 -1
- package/bundled/dev-pipeline/templates/sections/phase-commit.md +1 -1
- package/bundled/dev-pipeline/templates/sections/phase-context-snapshot-agent-suffix.md +2 -2
- package/bundled/dev-pipeline/templates/sections/phase-context-snapshot-base.md +1 -1
- package/bundled/dev-pipeline/templates/sections/phase-context-snapshot-lite-suffix.md +1 -1
- package/bundled/dev-pipeline/templates/sections/phase-implement-lite.md +1 -1
- package/bundled/dev-pipeline/templates/sections/phase-prizmkit-test.md +14 -26
- package/bundled/dev-pipeline/templates/sections/phase-review-full.md +2 -2
- package/bundled/dev-pipeline/templates/sections/phase-specify-plan-full.md +3 -3
- package/bundled/dev-pipeline/templates/sections/refactor-phase-review.md +1 -1
- package/bundled/dev-pipeline/templates/sections/refactor-phase-test.md +13 -10
- package/bundled/dev-pipeline/templates/sections/refactor-reminders.md +1 -1
- package/bundled/dev-pipeline/templates/session-status-schema.json +1 -1
- package/bundled/dev-pipeline/tests/test_checkpoint_state.py +494 -413
- package/bundled/dev-pipeline/tests/test_generate_bootstrap_prompt.py +144 -684
- package/bundled/dev-pipeline/tests/test_generate_bugfix_prompt.py +20 -41
- package/bundled/dev-pipeline/tests/test_generate_refactor_prompt.py +24 -24
- package/bundled/dev-pipeline/tests/test_python_runner_parity.py +176 -36
- package/bundled/skills/_metadata.json +6 -6
- package/bundled/skills/prizmkit/SKILL.md +13 -18
- package/bundled/skills/prizmkit/references/workflow-state-protocol.md +73 -80
- package/bundled/skills/prizmkit-code-review/SKILL.md +7 -7
- package/bundled/skills/prizmkit-code-review/references/workflow-state-protocol.md +73 -80
- package/bundled/skills/prizmkit-committer/SKILL.md +8 -7
- package/bundled/skills/prizmkit-committer/references/workflow-state-protocol.md +73 -80
- package/bundled/skills/prizmkit-implement/SKILL.md +3 -2
- package/bundled/skills/prizmkit-implement/references/workflow-state-protocol.md +73 -80
- package/bundled/skills/prizmkit-plan/SKILL.md +2 -1
- package/bundled/skills/prizmkit-plan/references/workflow-state-protocol.md +73 -80
- package/bundled/skills/prizmkit-retrospective/SKILL.md +8 -6
- package/bundled/skills/prizmkit-retrospective/references/workflow-state-protocol.md +73 -80
- package/bundled/skills/prizmkit-test/SKILL.md +217 -189
- package/bundled/skills/prizmkit-test/references/external-contract-mock-guidance.md +119 -0
- package/bundled/skills/prizmkit-test/references/independent-test-review.md +269 -0
- package/bundled/skills/prizmkit-test/references/test-coverage-model.md +193 -0
- package/bundled/skills/prizmkit-test/references/test-report-template.md +164 -79
- package/bundled/skills/prizmkit-workflow/SKILL.md +28 -36
- package/bundled/skills/prizmkit-workflow/references/workflow-state-protocol.md +73 -80
- package/package.json +1 -1
- package/bundled/skills/prizmkit-test/assets/authoritative-records.schema.json +0 -477
- package/bundled/skills/prizmkit-test/assets/behavior-risk-matrix.schema.json +0 -116
- package/bundled/skills/prizmkit-test/assets/evidence-manifest.schema.json +0 -111
- package/bundled/skills/prizmkit-test/assets/evidence-package-template.json +0 -66
- package/bundled/skills/prizmkit-test/references/boundary-coverage-protocol.md +0 -104
- package/bundled/skills/prizmkit-test/references/contract-mock-protocol.md +0 -65
- package/bundled/skills/prizmkit-test/references/evidence-protocol.md +0 -208
- package/bundled/skills/prizmkit-test/references/evidence-request-protocol.md +0 -80
- package/bundled/skills/prizmkit-test/references/examples.md +0 -100
- package/bundled/skills/prizmkit-test/references/service-boundary-test-catalog.md +0 -228
- package/bundled/skills/prizmkit-test/references/test-generation-steps.md +0 -118
- package/bundled/skills/prizmkit-test/references/trusted-evidence-execution.md +0 -107
- package/bundled/skills/prizmkit-test/references/workflow-state-protocol.md +0 -181
- package/bundled/skills/prizmkit-test/scripts/build_test_evidence.py +0 -1930
- package/bundled/skills/prizmkit-test/scripts/validate_test_evidence.py +0 -1847
|
@@ -1,100 +0,0 @@
|
|
|
1
|
-
# Test Evidence Examples
|
|
2
|
-
|
|
3
|
-
## 1. Normally Tested Module Change — TEST_PASS
|
|
4
|
-
|
|
5
|
-
Input:
|
|
6
|
-
|
|
7
|
-
```text
|
|
8
|
-
/prizmkit-test scope=this-change artifact_dir=<change-artifact>
|
|
9
|
-
```
|
|
10
|
-
|
|
11
|
-
A library parser changes one accepted input form. Scope discovery selects the complete parser module and two direct consumers. Existing module tests already cover most behaviors.
|
|
12
|
-
|
|
13
|
-
The run:
|
|
14
|
-
|
|
15
|
-
1. Classifies `behavior`.
|
|
16
|
-
2. Adds the changed parser behavior plus existing observable parser behavior to the matrix.
|
|
17
|
-
3. Selects focused, affected-module regression, and Regression Ring layers; module-component and contract-integration are N/A with concrete pure-library rationale.
|
|
18
|
-
4. Adds missing malformed-input and consumer-shape tests.
|
|
19
|
-
5. Proves changed tests fail on baseline and pass currently.
|
|
20
|
-
6. Packages all command/environment/output history.
|
|
21
|
-
7. Deterministically validates.
|
|
22
|
-
|
|
23
|
-
Handoff:
|
|
24
|
-
|
|
25
|
-
```text
|
|
26
|
-
TEST_PASS
|
|
27
|
-
Evidence: .prizmkit/test/evidence/<evidence-id>/
|
|
28
|
-
Validator: passed
|
|
29
|
-
Mocked code-level evidence does not verify a real deployed environment.
|
|
30
|
-
```
|
|
31
|
-
|
|
32
|
-
## 2. Incompletely Tested Legacy Module — Whole-Module Gap Filling
|
|
33
|
-
|
|
34
|
-
A queue worker retry change touches a legacy worker with one happy-path test. Changed lines identify the module, but Primary Scope includes parsing, claim, processing, retry, dead-letter, acknowledgment, idempotency, and dependency-failure behavior.
|
|
35
|
-
|
|
36
|
-
`TEST_BUILD` fills observable gaps across the module, not unrelated private utility lines. It adds deterministic fake-clock and contract-backed queue tests for retry just below/at/above maximum, duplicate delivery, malformed payload, lock failure, no partial side effect, ack/nack calls, and dead-letter transition. Regression Ring runs the scheduler and producer contract checks. Coverage percentage is recorded only as a diagnostic signal.
|
|
37
|
-
|
|
38
|
-
The run cannot pass until every applicable behavior-risk cell and Regression Ring execution is mapped and covered by reliable current-workspace execution.
|
|
39
|
-
|
|
40
|
-
## 3. Contract-Coupled Modules — Contract-Driven Mocks
|
|
41
|
-
|
|
42
|
-
An API client and consumer share OpenAPI. The run snapshots the OpenAPI document into `contracts/`, records its source/hash, and generates mock responses for success, schema error, 429, timeout, malformed body, and pagination token boundaries.
|
|
43
|
-
|
|
44
|
-
The mock is rejected if it invents an incompatible field or cannot trace behavior to the contract. Contract and consumer executions pass, and the report states `environment_claim=mocked-code-level-only`; it makes no claim about a deployed SaaS environment.
|
|
45
|
-
|
|
46
|
-
## 4. Reliable Business Defect — TEST_FAIL
|
|
47
|
-
|
|
48
|
-
Resolved specification and OpenAPI require a negative amount to return a validation error without a write. A generated contract test reliably observes success plus a database-adapter call.
|
|
49
|
-
|
|
50
|
-
The test is valid, current and affected-module executions reproduce the failure, and raw output is preserved. `/prizmkit-test` does not change production business behavior. It packages:
|
|
51
|
-
|
|
52
|
-
```text
|
|
53
|
-
TEST_FAIL
|
|
54
|
-
Reproduced failure: amount-boundary-negative
|
|
55
|
-
Evidence: .prizmkit/test/evidence/<evidence-id>/
|
|
56
|
-
```
|
|
57
|
-
|
|
58
|
-
## 5. Unknown Coupling or Conflicting Truth — TEST_BLOCKED
|
|
59
|
-
|
|
60
|
-
A dynamically loaded plugin may consume the changed result shape, but its registry is generated remotely and unavailable. The edge can change the verdict, so it remains verdict-capable and unresolved.
|
|
61
|
-
|
|
62
|
-
Alternatively, specification and shared schema disagree on nullability with no authoritative resolution. The skill does not freeze current behavior into a characterization test.
|
|
63
|
-
|
|
64
|
-
Both produce:
|
|
65
|
-
|
|
66
|
-
```text
|
|
67
|
-
TEST_BLOCKED
|
|
68
|
-
Blocker: verdict-capable unresolved edge or conflicting truth
|
|
69
|
-
Evidence: .prizmkit/test/evidence/<evidence-id>/
|
|
70
|
-
```
|
|
71
|
-
|
|
72
|
-
## 6. Current-State New Behavior
|
|
73
|
-
|
|
74
|
-
A newly added CLI command has no pre-change interface, so the skill adds a focused native command test and runs it against the current project implementation. The test passes, its behavior-risk and test-plan mappings are complete, and the runner receipt and raw output are preserved. The evidence claims current code-level conformance; it does not claim that the test failed on a historical implementation.
|
|
75
|
-
|
|
76
|
-
## 7. Interrupted or Budgeted Run
|
|
77
|
-
|
|
78
|
-
On recovery, the skill validates stage outputs and target hashes. A changed lockfile invalidates `INFRA_READY` and every downstream stage but preserves earlier execution history immutably. The run resumes there.
|
|
79
|
-
|
|
80
|
-
If the budget ends after half a large module's required matrix is tested, progress persists and the result is `TEST_BLOCKED`; no scope is silently truncated.
|
|
81
|
-
|
|
82
|
-
## 8. Tampering
|
|
83
|
-
|
|
84
|
-
After packaging, changing `raw/execution-003.stdout`, a contract snapshot, generated test, source patch, or derived report produces a content-hash mismatch. Changing identity inputs without recomputing the canonical evidence ID also fails.
|
|
85
|
-
|
|
86
|
-
```bash
|
|
87
|
-
python3 ${SKILL_DIR}/scripts/validate_test_evidence.py \
|
|
88
|
-
--evidence-dir .prizmkit/test/evidence/<evidence-id> \
|
|
89
|
-
--project-root <target-project-root>
|
|
90
|
-
```
|
|
91
|
-
|
|
92
|
-
Result: non-zero with `Test evidence validation: FAIL`. A tampered package cannot retain `TEST_PASS`.
|
|
93
|
-
|
|
94
|
-
## 9. Lightweight Change
|
|
95
|
-
|
|
96
|
-
A comments-only diff is deterministically classified and parser/format/generated-output checks pass. Behavior-only stages are marked not applicable with reasons. `TEST_PASS` means the lightweight claim is proven; the report explicitly does not claim full runtime test evidence.
|
|
97
|
-
|
|
98
|
-
## Compatibility Notice
|
|
99
|
-
|
|
100
|
-
The testing-domain verdict remains `TEST_PASS | TEST_FAIL | TEST_BLOCKED`. Consumers must use validated authoritative evidence and may use `test-report.md` only as a derived compatibility view. The formal lifecycle does not reinterpret a testing result as a review or commit verdict.
|
|
@@ -1,228 +0,0 @@
|
|
|
1
|
-
# Service Behavior-Risk Catalog
|
|
2
|
-
|
|
3
|
-
Use this catalog during `CONTRACT_MODEL` and `TEST_BUILD` to discover domain-specific observable risks. It supplies questions, not truth: every expected value must come from the recorded truth precedence and every external dependency double must follow `references/contract-mock-protocol.md`.
|
|
4
|
-
|
|
5
|
-
## How to Use This Catalog
|
|
6
|
-
|
|
7
|
-
1. Classify the affected module by its public behavior, dependencies, state changes, and resolved contracts.
|
|
8
|
-
2. Select every service type that applies; one behavior can combine several types.
|
|
9
|
-
3. Add applicable cases to `behavior-risk-matrix.json` and map them to test/execution IDs.
|
|
10
|
-
4. Derive mock requests/responses from shared machine-readable or locally snapshotted contracts. Never independently invent provider/database/queue behavior.
|
|
11
|
-
5. Record a concrete rationale for every non-applicable risk.
|
|
12
|
-
6. Treat coverage metrics as diagnostic only; complete observable behavior and risk rather than private lines.
|
|
13
|
-
|
|
14
|
-
Code-level tests are mock-first and must state that they do not verify a real deployed environment. Production credentials, services, and data are prohibited.
|
|
15
|
-
|
|
16
|
-
## Service Type Detection
|
|
17
|
-
|
|
18
|
-
| Signal | Likely Service Type |
|
|
19
|
-
|--------|---------------------|
|
|
20
|
-
| login, logout, signup, password, session, token, jwt, oauth, refresh | Auth / Session / Token |
|
|
21
|
-
| role, permission, policy, access, entitlement, plan, quota, ownership | Authorization / Permission / Entitlement |
|
|
22
|
-
| payment, wallet, billing, subscription, invoice, refund, balance, currency | Payment / Wallet / Billing / Refund |
|
|
23
|
-
| repository, model, dao, create, read, update, delete, insert, save, remove | CRUD / Repository / Database |
|
|
24
|
-
| search, list, page, cursor, filter, sort, query, limit, offset | Search / List / Pagination |
|
|
25
|
-
| validate, parse, normalize, transform, schema, sanitize, format | Validation / Parser / Normalizer |
|
|
26
|
-
| upload, download, storage, file, blob, s3, image, mime, path | File Upload / Storage |
|
|
27
|
-
| notify, email, sms, webhook, template, recipient, provider | Notification / Email / Webhook |
|
|
28
|
-
| job, queue, worker, scheduler, cron, retry, backoff, task | Job / Queue / Scheduler |
|
|
29
|
-
| cache, ttl, rate limit, throttle, idempotency, dedupe, lock | Cache / Rate Limit / Idempotency |
|
|
30
|
-
| client, sdk, request, fetch, provider, api key, endpoint, response | API Client / Third-Party Integration |
|
|
31
|
-
| date, time, expiry, expiration, ttl, schedule, window, timezone | Date / Time / Expiration |
|
|
32
|
-
|
|
33
|
-
When names and behavior disagree, trust behavior and dependencies over names. For example, `createCheckoutSession()` is payment-related even if it lives in a controller module.
|
|
34
|
-
|
|
35
|
-
## Universal Boundary Matrix
|
|
36
|
-
|
|
37
|
-
Apply these categories to every public function/class method before adding service-specific cases:
|
|
38
|
-
|
|
39
|
-
- **Valid input**: typical valid input, minimal valid input, and complex valid input when supported.
|
|
40
|
-
- **Empty input**: `null` / `undefined` / `None`, empty string, empty array/object, whitespace-only string.
|
|
41
|
-
- **Numeric boundaries**: zero, one, minimum allowed value, maximum allowed value, just below/above limits.
|
|
42
|
-
- **Collection boundaries**: zero items, one item, many items, duplicate items, first/last element behavior.
|
|
43
|
-
- **Type and format errors**: wrong type, malformed string, unsupported enum, missing required parameter.
|
|
44
|
-
- **Branch paths**: each `if/else`, `switch/case/default`, ternary side, and loop path with zero/one/many iterations.
|
|
45
|
-
- **State and side effects**: before/after state, writes, emitted events, dependency method calls, call count/order/arguments.
|
|
46
|
-
- **Dependency failures**: mocked database errors, network timeouts, storage failures, provider errors, clock/randomness behavior.
|
|
47
|
-
|
|
48
|
-
## Service-Specific Boundary Cases
|
|
49
|
-
|
|
50
|
-
### Auth / Session / Token Services
|
|
51
|
-
|
|
52
|
-
Generate applicable cases for:
|
|
53
|
-
|
|
54
|
-
- Missing identifier, missing password, empty credentials, malformed email/username.
|
|
55
|
-
- Identifier normalization: uppercase/lowercase email, leading/trailing whitespace, Unicode usernames when supported.
|
|
56
|
-
- User not found, wrong password, unverified account, disabled account, locked account.
|
|
57
|
-
- Failed login count just below/at/above lock threshold.
|
|
58
|
-
- Successful login creates exactly one session/token with expected expiry and persisted state.
|
|
59
|
-
- Token missing, malformed, invalid signature, wrong algorithm, expired, not-yet-valid, revoked.
|
|
60
|
-
- Refresh token rotation: old refresh token reuse, missing refresh token, expired refresh token, revoked refresh token.
|
|
61
|
-
- Logout idempotency: logout with valid token, already logged-out token, missing token.
|
|
62
|
-
- OAuth/callback flows: missing code, missing state, state mismatch, provider error, provider returns incomplete profile.
|
|
63
|
-
- Dependency failures: user store error, password hasher error, token signer/verifier error, clock skew.
|
|
64
|
-
|
|
65
|
-
### Authorization / Permission / Entitlement Services
|
|
66
|
-
|
|
67
|
-
Generate applicable cases for:
|
|
68
|
-
|
|
69
|
-
- Missing principal, anonymous principal, principal without roles/permissions.
|
|
70
|
-
- Default-deny behavior when no policy matches.
|
|
71
|
-
- Role hierarchy: basic role, elevated role, admin override, deny rule precedence.
|
|
72
|
-
- Resource ownership: owner, non-owner, tenant mismatch, organization mismatch.
|
|
73
|
-
- Entitlement states: active, expired, canceled, trial, disabled feature, missing plan.
|
|
74
|
-
- Quota boundaries: usage below limit, exactly at limit, one over limit, reset window boundary.
|
|
75
|
-
- Duplicate or conflicting permissions.
|
|
76
|
-
- Policy conditions with missing attributes, malformed attributes, unknown action/resource.
|
|
77
|
-
- Dependency failures: policy store error, entitlement lookup error, quota service timeout.
|
|
78
|
-
|
|
79
|
-
### Payment / Wallet / Billing / Refund Services
|
|
80
|
-
|
|
81
|
-
Generate applicable cases for:
|
|
82
|
-
|
|
83
|
-
- Amount zero, negative amount, minimum billable unit, maximum allowed amount, just above maximum.
|
|
84
|
-
- Currency precision: too many decimal places, integer minor units, rounding at half-unit boundaries.
|
|
85
|
-
- Balance boundaries: balance exactly equals debit amount, balance one minor unit below amount, zero balance.
|
|
86
|
-
- Currency mismatch between wallet, invoice, and payment request.
|
|
87
|
-
- Idempotency: missing key, duplicate key with same payload, duplicate key with different payload.
|
|
88
|
-
- Transaction states: pending, succeeded, failed, canceled, already paid, already refunded.
|
|
89
|
-
- Refund boundaries: zero refund, partial refund, full refund, cumulative refunds equal original amount, cumulative refunds exceed original amount.
|
|
90
|
-
- Concurrent debit/refund attempts against the same account or transaction.
|
|
91
|
-
- Subscription/billing cycles: trial start/end, expired subscription, canceled subscription, renewal at boundary time.
|
|
92
|
-
- Provider failures: timeout, 4xx/5xx, malformed response, duplicate provider event, out-of-order webhook.
|
|
93
|
-
- Side effects: ledger entry count, balance mutation, webhook/event emission, provider call arguments.
|
|
94
|
-
|
|
95
|
-
### CRUD / Repository / Database Services
|
|
96
|
-
|
|
97
|
-
Generate applicable cases for:
|
|
98
|
-
|
|
99
|
-
- Create with missing required fields, nullable fields set to `null`, optional fields omitted.
|
|
100
|
-
- Duplicate unique key, generated ID collision, invalid foreign key, unknown enum value.
|
|
101
|
-
- Read/update/delete with nonexistent ID, malformed ID, deleted/soft-deleted ID.
|
|
102
|
-
- Partial update semantics: omitted field vs `undefined` vs `null`, empty patch object.
|
|
103
|
-
- Optimistic locking: expected version, stale version, missing version.
|
|
104
|
-
- Soft delete behavior: excluded by default, included only when requested, double delete idempotency.
|
|
105
|
-
- Transaction behavior: rollback when one write fails, no partial side effects after failure.
|
|
106
|
-
- Database dependency errors: connection failure, constraint violation, timeout.
|
|
107
|
-
- Repository side effects: correct query arguments, write count, returned entity shape.
|
|
108
|
-
|
|
109
|
-
### Search / List / Pagination / Sorting Services
|
|
110
|
-
|
|
111
|
-
Generate applicable cases for:
|
|
112
|
-
|
|
113
|
-
- Empty query, whitespace query, special characters, Unicode query, injection-like strings treated as data.
|
|
114
|
-
- No results, one result, many results, duplicate ranking/sort values.
|
|
115
|
-
- Page/offset pagination: page 0, page 1, last page, beyond last page, negative page/offset.
|
|
116
|
-
- Cursor pagination: missing cursor, malformed cursor, expired cursor, cursor at final item.
|
|
117
|
-
- Limit boundaries: omitted limit, limit 0, limit 1, maximum limit, maximum + 1.
|
|
118
|
-
- Sorting: default sort, invalid sort field, invalid direction, stable ordering with ties.
|
|
119
|
-
- Filters: empty filter set, unknown filter, conflicting filters, inclusive/exclusive date ranges.
|
|
120
|
-
- Total count and hasNext/hasPrevious flags at page boundaries.
|
|
121
|
-
- Dependency failures: search index unavailable, database query error, stale index response.
|
|
122
|
-
|
|
123
|
-
### Validation / Parser / Normalizer Services
|
|
124
|
-
|
|
125
|
-
Generate applicable cases for:
|
|
126
|
-
|
|
127
|
-
- Missing input, `null` / `undefined` / `None`, empty string, whitespace-only string.
|
|
128
|
-
- Minimum and maximum length; just below/above configured length boundaries.
|
|
129
|
-
- Special characters, Unicode, emoji, newline/control characters, normalization-sensitive characters.
|
|
130
|
-
- Wrong type, mixed-type arrays, numeric strings vs numbers, booleans represented as strings.
|
|
131
|
-
- Unknown fields in strict mode, extra fields in permissive mode, duplicate keys/items.
|
|
132
|
-
- Invalid formats: email, URL, date, UUID, JSON, CSV, phone number, currency.
|
|
133
|
-
- Normalization idempotency: applying the normalizer twice yields the same result.
|
|
134
|
-
- Error shape: path/field name, code, message, and aggregation of multiple errors.
|
|
135
|
-
- Parser failures: malformed payload, truncated payload, unsupported version/schema.
|
|
136
|
-
|
|
137
|
-
### File Upload / Storage Services
|
|
138
|
-
|
|
139
|
-
Generate applicable cases for:
|
|
140
|
-
|
|
141
|
-
- Zero-byte file, one-byte file, just below max size, exactly max size, just above max size.
|
|
142
|
-
- MIME/type mismatch between content, extension, and declared header.
|
|
143
|
-
- Unsupported file type, missing extension, uppercase extension, compound extension.
|
|
144
|
-
- Filename boundaries: empty name, very long name, Unicode name, duplicate name, path traversal characters.
|
|
145
|
-
- Storage key collisions and deterministic key generation.
|
|
146
|
-
- Checksum/hash mismatch, corrupted stream, interrupted upload, storage write failure.
|
|
147
|
-
- Image/media metadata boundaries: dimensions too small/large, unsupported codec, malformed metadata.
|
|
148
|
-
- Delete/read behavior for missing object, already-deleted object, permission-denied object.
|
|
149
|
-
- Side effects: storage provider call arguments, metadata record creation, cleanup on failure.
|
|
150
|
-
|
|
151
|
-
### Notification / Email / Webhook Services
|
|
152
|
-
|
|
153
|
-
Generate applicable cases for:
|
|
154
|
-
|
|
155
|
-
- Missing recipient, invalid email/phone/URL, empty recipient list, duplicate recipients.
|
|
156
|
-
- Recipient preferences: unsubscribed, disabled channel, quiet hours, blocked user.
|
|
157
|
-
- Template variables: missing required variable, extra variable, `null` variable, special characters/HTML escaping.
|
|
158
|
-
- Provider selection: unsupported channel, fallback provider, disabled provider.
|
|
159
|
-
- Deduplication: duplicate notification key, repeated webhook event, retry with same id.
|
|
160
|
-
- Provider responses: success, rate limit, timeout, 4xx/5xx, malformed response, partial multi-recipient failure.
|
|
161
|
-
- Webhook security: missing signature, invalid signature, timestamp outside tolerance, replayed event.
|
|
162
|
-
- Side effects: event status persisted, retry scheduled or suppressed, provider called with expected payload.
|
|
163
|
-
|
|
164
|
-
### Job / Queue / Scheduler Services
|
|
165
|
-
|
|
166
|
-
Generate applicable cases for:
|
|
167
|
-
|
|
168
|
-
- Missing payload, empty payload, malformed payload, unsupported job type/version.
|
|
169
|
-
- Duplicate job ID, idempotent processing of already-completed job, requeue behavior.
|
|
170
|
-
- Retry boundaries: first failure, retry count just below max, at max, over max, poison/dead-letter path.
|
|
171
|
-
- Backoff calculation: zero delay, minimum delay, maximum delay, jitter bounds when deterministic/mocked.
|
|
172
|
-
- Schedule boundaries: run in past, run now, run in future, invalid cron, daylight-saving transition.
|
|
173
|
-
- Concurrency: two workers claim same job, lock acquisition failure, stale lock recovery.
|
|
174
|
-
- Cancellation: before start, during processing, after completion.
|
|
175
|
-
- Side effects: ack/nack calls, status transitions, retry scheduling, emitted events.
|
|
176
|
-
|
|
177
|
-
### Cache / Rate Limit / Idempotency Services
|
|
178
|
-
|
|
179
|
-
Generate applicable cases for:
|
|
180
|
-
|
|
181
|
-
- Cache miss, hit, stale hit, expired entry, invalidation, key collision.
|
|
182
|
-
- TTL boundaries: zero TTL, negative TTL, one unit TTL, exact expiry instant, just before/after expiry.
|
|
183
|
-
- Serialization/deserialization failure, corrupted cached value, unsupported value type.
|
|
184
|
-
- Cache stampede: concurrent misses for the same key should not duplicate expensive work when guarded.
|
|
185
|
-
- Rate limit boundaries: count below limit, exactly at limit, one above limit, window rollover.
|
|
186
|
-
- Missing actor key: anonymous user, missing IP, shared tenant/global scope.
|
|
187
|
-
- Idempotency key: missing, malformed, reused with same payload, reused with different payload, expired record.
|
|
188
|
-
- Lock behavior: lock unavailable, lock timeout, release on success, release on failure.
|
|
189
|
-
- Backing store failures: Redis/cache down, slow response, partial write.
|
|
190
|
-
|
|
191
|
-
### API Client / Third-Party Integration Services
|
|
192
|
-
|
|
193
|
-
Generate applicable cases for:
|
|
194
|
-
|
|
195
|
-
- Missing base URL, missing API key, malformed configuration, unsupported region/environment.
|
|
196
|
-
- Request building: query encoding, header inclusion, body serialization, path parameter escaping.
|
|
197
|
-
- Response handling: 200 with valid body, 204/no body, malformed JSON, missing required field, unknown extra field.
|
|
198
|
-
- Error responses: 400, 401/403, 404, 409, 429, 500/503; retryable vs non-retryable classification.
|
|
199
|
-
- Timeout, network error, DNS/connect failure, aborted request.
|
|
200
|
-
- Retry boundaries: zero retries, one retry then success, max retries exhausted, backoff capped.
|
|
201
|
-
- Pagination: empty page, last page, next token missing/malformed, duplicate page token.
|
|
202
|
-
- Auth refresh/signing: expired access token, refresh failure, invalid signature, clock skew.
|
|
203
|
-
- Circuit breaker or fallback behavior when present.
|
|
204
|
-
|
|
205
|
-
### Date / Time / Expiration Services
|
|
206
|
-
|
|
207
|
-
Generate applicable cases for:
|
|
208
|
-
|
|
209
|
-
- Exact boundary: `now == startsAt`, `now == expiresAt`, one millisecond/second before and after.
|
|
210
|
-
- Inclusive vs exclusive interval ends according to the contract.
|
|
211
|
-
- Start after end, zero-length interval, negative duration, missing timezone.
|
|
212
|
-
- Time zones: UTC vs local time, DST jump forward/back, leap day, month end, year end.
|
|
213
|
-
- Clock skew tolerance for tokens, webhooks, scheduled jobs, and signed requests.
|
|
214
|
-
- Recurring schedules: first occurrence, last occurrence, skipped occurrence, invalid cron/expression.
|
|
215
|
-
- TTL/window calculations: zero TTL, negative TTL, max TTL, rollover between windows.
|
|
216
|
-
|
|
217
|
-
## When Multiple Service Types Apply
|
|
218
|
-
|
|
219
|
-
Combine the relevant sections and remove duplicate cases. Prioritize tests that exercise business invariants over tests that only repeat generic invalid input handling. For example, a wallet debit function should cover payment amount boundaries, idempotency replay, and date/expiration behavior if it also accepts an expiring authorization.
|
|
220
|
-
|
|
221
|
-
## N/A Rules
|
|
222
|
-
|
|
223
|
-
Mark a category `N/A` when the unit has no corresponding input, state, or dependency. Include a short reason:
|
|
224
|
-
|
|
225
|
-
- `N/A — no external provider dependency`
|
|
226
|
-
- `N/A — function is pure and has no stateful side effects`
|
|
227
|
-
- `N/A — pagination is handled by caller, not this unit`
|
|
228
|
-
- `N/A — concurrency safety belongs to integration test because unit has no shared mutable state`
|
|
@@ -1,118 +0,0 @@
|
|
|
1
|
-
# Test Generation and Infrastructure Steps
|
|
2
|
-
|
|
3
|
-
Use this reference during `INFRA_READY`, `TEST_BUILD`, and `EXECUTE_PROVE`. The authoritative scope and completeness inputs are `scope.json`, `behavior-risk-matrix.json`, and `test-plan.json`.
|
|
4
|
-
|
|
5
|
-
## Existing-Test-First Rule
|
|
6
|
-
|
|
7
|
-
Before creating a test, inspect the repository's native test files, runner configuration, and existing behavior assertions. Execute the smallest relevant existing tests first. Add, modify, or delete a repository-native test only when a concrete behavior gap is identified and recorded in the semantic preparation request. Supply project paths and behavior/layer mappings only; `prepare-tests` derives IDs, status, snapshots, and hashes from the repository and canonical Git capture. An unchanged existing test has no generated snapshot, and `generated-tests/` is never an execution workspace.
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
For `scope=this-change`, changed files locate the affected module; they do not restrict tests to changed lines. Fill observable behavior/risk gaps across the whole affected module and execute its Regression Ring. Unrelated modules remain out of scope unless they are direct callers, consumers, shared contracts, or state dependencies.
|
|
11
|
-
|
|
12
|
-
Do not infer current-change scope from a directory name alone. Unknown scope is `TEST_BLOCKED`.
|
|
13
|
-
|
|
14
|
-
## Native Infrastructure Discovery
|
|
15
|
-
|
|
16
|
-
Before writing tests:
|
|
17
|
-
|
|
18
|
-
1. Inspect manifests, lockfiles, existing test files, runner configuration, scripts, fixtures, and CI commands.
|
|
19
|
-
2. Select the existing adequate runner and naming/import/mock conventions.
|
|
20
|
-
3. Let the target project's semantics determine framework, argv, cwd, timeout, attempts, concurrency, Mock tooling, layers, module roots, inventory patterns; do not import fixed ecosystem defaults from this skill.
|
|
21
|
-
4. If no adequate infrastructure exists, add only the smallest project-native runner/config/dependencies required for the planned layers.
|
|
22
|
-
5. Verify dependency versions against the package registry before writing manifests.
|
|
23
|
-
6. Record every file/dependency/command change and verification in `infrastructure-changes.json`.
|
|
24
|
-
7. Never delete existing tests or add a second framework when the current one is adequate.
|
|
25
|
-
|
|
26
|
-
Create fixtures, fakes, contract-backed mock servers, browser harnesses, or isolated services only when required by `test-plan.json`. Every disposable resource needs a unique ID, bounded lifetime, and verified cleanup.
|
|
27
|
-
|
|
28
|
-
## Matrix-First Test Construction
|
|
29
|
-
|
|
30
|
-
For every behavior row:
|
|
31
|
-
|
|
32
|
-
1. Read the public interface, contract source, preconditions, inputs/boundaries, outputs, side effects, transitions, and errors.
|
|
33
|
-
2. Inspect existing test assertions—not names—to find proven cells.
|
|
34
|
-
3. Assess all eight risks: functional, boundary, permission, concurrency, idempotency, time, dependency, and consumer.
|
|
35
|
-
4. For every applicable risk, map one or more builder-generated test units and expected execution layer; do not invent a separate AI-maintained ID for every test function.
|
|
36
|
-
5. For every non-applicable risk, record a concrete rationale tied to this behavior.
|
|
37
|
-
6. Keep unresolved truth or untestable necessary behavior explicit; it blocks rather than becoming guessed expected behavior.
|
|
38
|
-
|
|
39
|
-
A module is complete when all observable behaviors and applicable risks have mapped, proven tests and the Regression Ring passes. Line/branch/function coverage may reveal an omission and trigger another matrix review, but a percentage cannot complete a cell.
|
|
40
|
-
|
|
41
|
-
## Risk-Adaptive Layers
|
|
42
|
-
|
|
43
|
-
Choose only necessary test types, then assess all ordered layers in `test-plan.json`:
|
|
44
|
-
|
|
45
|
-
- **Unit/focused:** pure behavior, boundaries, errors, deterministic collaborators, and fast defect localization.
|
|
46
|
-
- **Module/component:** public module behavior with internal collaborators composed as production uses them.
|
|
47
|
-
- **Contract:** producer/consumer or adapter compatibility against a shared/snapshotted machine-readable contract.
|
|
48
|
-
- **Integration:** multiple local modules or isolated infrastructure when mocks cannot establish the relevant property.
|
|
49
|
-
- **Code-level E2E:** CLI/UI/API flow through code-level boundaries when user-visible composition is at risk and a native harness is available.
|
|
50
|
-
- **Affected-module regression:** every required test for the complete affected module.
|
|
51
|
-
- **Regression Ring:** direct callers, consumers, shared contracts, and state dependencies.
|
|
52
|
-
|
|
53
|
-
Do not require every layer mechanically. An omitted layer needs a specific reason such as “pure library has no UI or process boundary; module tests prove its public composition.” “Not tested” is not a rationale.
|
|
54
|
-
|
|
55
|
-
## Contract-Driven Doubles
|
|
56
|
-
|
|
57
|
-
Read `${SKILL_DIR}/references/contract-mock-protocol.md` for external databases, queues, filesystems, SaaS, browser backends, and cross-module endpoints.
|
|
58
|
-
|
|
59
|
-
Prefer shared OpenAPI/schema/types/generated models. Otherwise snapshot a locally derived contract and record derivation source. Generate success, boundary, malformed, timeout, and failure variants from that contract. Reject independently invented fields or a mock that cannot be traced to a snapshot.
|
|
60
|
-
|
|
61
|
-
Code-level evidence is mock-first and always reports `environment_claim=mocked-code-level-only`. Do not connect production or a user-linked deployed test database.
|
|
62
|
-
|
|
63
|
-
## Whole-Module Legacy Gap Filling
|
|
64
|
-
|
|
65
|
-
When a changed module has incomplete tests:
|
|
66
|
-
|
|
67
|
-
- enumerate every observable public behavior, not only changed exports;
|
|
68
|
-
- cover valid/minimal/complex inputs and contract-defined boundaries;
|
|
69
|
-
- cover outputs and response shape;
|
|
70
|
-
- assert side effects and absence of partial effects on failure;
|
|
71
|
-
- cover state transitions and idempotency where applicable;
|
|
72
|
-
- cover permissions, concurrency, clocks, external dependency failures, and consumer assumptions where applicable;
|
|
73
|
-
- avoid tests for unrelated private implementation lines.
|
|
74
|
-
|
|
75
|
-
Use `${SKILL_DIR}/references/service-boundary-test-catalog.md` to discover domain-specific risks, then map only contract-supported expectations.
|
|
76
|
-
|
|
77
|
-
## Testability Seams
|
|
78
|
-
|
|
79
|
-
If production code cannot be controlled:
|
|
80
|
-
|
|
81
|
-
1. Prefer injecting an existing dependency abstraction.
|
|
82
|
-
2. Otherwise extract a pure function, add an internal adapter, or make state/clock/randomness controllable.
|
|
83
|
-
3. Keep the public contract and observable behavior unchanged.
|
|
84
|
-
4. Record the production patch and behavior-preservation evidence.
|
|
85
|
-
|
|
86
|
-
If a public production contract must change, do not proceed autonomously; return `TEST_BLOCKED`.
|
|
87
|
-
|
|
88
|
-
## Valid Failures
|
|
89
|
-
|
|
90
|
-
Distinguish:
|
|
91
|
-
|
|
92
|
-
- **Test construction defect:** wrong import, syntax, fixture, mock contract, or runner usage. Fix the test/infrastructure within bounded attempts.
|
|
93
|
-
- **Reliable implementation/contract failure:** valid test against resolved truth repeatedly fails. Preserve the failure and return `TEST_FAIL`; do not repair business behavior here.
|
|
94
|
-
- **Uncertain/flaky/unavailable:** truth unresolved, required test unreliable, infrastructure unavailable, or cleanup fails. Return `TEST_BLOCKED`.
|
|
95
|
-
|
|
96
|
-
Never retry until green or silently weaken an assertion/scope.
|
|
97
|
-
|
|
98
|
-
## Current-State Execution Evidence
|
|
99
|
-
|
|
100
|
-
Run selected tests in order through the bundled evidence builder: focused, module/component, contract/integration, affected-module regression, Regression Ring. The model writes schema-shaped requests; the builder performs the process and appends immutable runner-generated receipts to `executions.json`.
|
|
101
|
-
|
|
102
|
-
For each new/changed necessary behavior test:
|
|
103
|
-
|
|
104
|
-
1. Execute each required layer in order through the builder in the target project root.
|
|
105
|
-
2. Preserve runner-generated receipts and complete stdout/stderr for every selected execution.
|
|
106
|
-
3. Bind each applicable behavior risk to planned tests and selected execution IDs.
|
|
107
|
-
4. Do not create historical checkouts, controlled mutations, or worktrees; current-state conformance is the testing-domain claim.
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
## Completion Check
|
|
111
|
-
|
|
112
|
-
Before packaging:
|
|
113
|
-
|
|
114
|
-
- no applicable matrix cell lacks test/execution mapping;
|
|
115
|
-
- all required planned layers have reliable selected executions in order;
|
|
116
|
-
- affected-module and Regression Ring regressions are complete;
|
|
117
|
-
- no verdict-capable unresolved edge remains for pass;
|
|
118
|
-
- budgeted/staged work is complete, otherwise `TEST_BLOCKED`.
|
|
@@ -1,107 +0,0 @@
|
|
|
1
|
-
# Trusted Evidence Execution
|
|
2
|
-
|
|
3
|
-
Load this reference before `SCOPE_DISCOVER`, `EXECUTE_PROVE`, resume, or replay. It defines which choices remain with the model and which evidence mechanics are fixed.
|
|
4
|
-
|
|
5
|
-
## Two Parameter Classes
|
|
6
|
-
|
|
7
|
-
### Model-Chosen Project Parameters
|
|
8
|
-
|
|
9
|
-
Inspect the target project and choose these semantically:
|
|
10
|
-
|
|
11
|
-
- native test framework, runner, command, flags, and working directory;
|
|
12
|
-
- timeout, attempt/retry policy, concurrency, resource limits, and isolation strategy;
|
|
13
|
-
- unit/module/contract/integration/code-level E2E layers;
|
|
14
|
-
- fixture, fake, Mock, service-double, browser, and database technology;
|
|
15
|
-
- module roots, public behavior boundary, inventory patterns, exclusions, and Regression Ring;
|
|
16
|
-
- environment classification and endpoint allow/deny evidence.
|
|
17
|
-
|
|
18
|
-
The protocol does not prescribe Jest, pytest, a timeout number, a retry count, coverage threshold, runner flag, or mutation package. Record the reason for each choice in request and planning records.
|
|
19
|
-
|
|
20
|
-
### Protocol-Fixed Locators and Mechanisms
|
|
21
|
-
|
|
22
|
-
The protocol fixes only what trustworthy, replayable packaging needs:
|
|
23
|
-
|
|
24
|
-
- locator arguments: project root, evidence directory, request/manifest/inventory/receipt path;
|
|
25
|
-
- schema-shaped request records;
|
|
26
|
-
- project-root and evidence-root path confinement;
|
|
27
|
-
- actual child-process execution without a shell;
|
|
28
|
-
- complete stdout/stderr capture and output hashes;
|
|
29
|
-
- complete effective environment and tool-probe capture;
|
|
30
|
-
- runner-generated UUID receipt, request hash, and runner hash;
|
|
31
|
-
- append-only execution history;
|
|
32
|
-
- direct current-workspace native execution with runner receipts;
|
|
33
|
-
- deterministic resume invalidation calculation;
|
|
34
|
-
- strict schema and cross-record validation.
|
|
35
|
-
|
|
36
|
-
## Trust Boundary
|
|
37
|
-
|
|
38
|
-
A receipt proves that the bundled runner produced a record with the captured inputs and outputs during this invocation. The runner snapshots its own executable bytes inside the evidence package so later validation does not silently depend on whichever PrizmKit version happens to be installed. Hashes bind bytes together and detect later drift. They do not create cryptographic non-repudiation and cannot prove that a same-OS-user, same-permission producer did not replace the runner, requests, repository, or evidence package before validation.
|
|
39
|
-
|
|
40
|
-
Treat evidence as auditable project-controlled provenance, not a defense against a malicious same-privilege actor. The validator checks consistency, obvious conflicts, and replayability; it cannot mathematically prove that the model found every business behavior.
|
|
41
|
-
|
|
42
|
-
## Builder Commands
|
|
43
|
-
|
|
44
|
-
Drive a new package through the canonical lifecycle. Semantic request files remain model-authored; authoritative package records remain tool-owned. The builder applies a bounded default timeout when the request omits one, preserves common native cache variables, and returns a bounded unreliable receipt on timeout. The normal user path should author one semantic request and one execution plan; use individual lifecycle commands only for recovery or protocol debugging.
|
|
45
|
-
|
|
46
|
-
```bash
|
|
47
|
-
python3 ${SKILL_DIR}/scripts/build_test_evidence.py --project-root <root> --evidence-dir <staging> init --baseline <commit>
|
|
48
|
-
python3 ${SKILL_DIR}/scripts/build_test_evidence.py --project-root <root> --evidence-dir <staging> capture-change
|
|
49
|
-
python3 ${SKILL_DIR}/scripts/build_test_evidence.py --project-root <root> --evidence-dir <staging> inventory --request requests/inventory.json
|
|
50
|
-
python3 ${SKILL_DIR}/scripts/build_test_evidence.py --project-root <root> --evidence-dir <staging> execute --request requests/focused.json
|
|
51
|
-
python3 ${SKILL_DIR}/scripts/build_test_evidence.py --project-root <root> --evidence-dir <staging> finalize --request requests/finalize.json
|
|
52
|
-
```
|
|
53
|
-
|
|
54
|
-
`<staging>` must be a child of `.prizmkit/test/evidence/`. After `finalize`, the package is `payload-finalized`; run the validator pre-check and then `validate_test_evidence.py --attest` to seal it. Do not treat the pre-attestation staging package as immutable or invoke `render-report` after sealing.
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
```bash
|
|
58
|
-
python3 ${SKILL_DIR}/scripts/build_test_evidence.py \
|
|
59
|
-
--project-root <target-project-root> \
|
|
60
|
-
--evidence-dir <evidence-dir> \
|
|
61
|
-
inventory --request requests/inventory.json
|
|
62
|
-
|
|
63
|
-
python3 ${SKILL_DIR}/scripts/build_test_evidence.py \
|
|
64
|
-
--project-root <target-project-root> \
|
|
65
|
-
--evidence-dir <evidence-dir> \
|
|
66
|
-
execute --request requests/focused.json
|
|
67
|
-
|
|
68
|
-
python3 ${SKILL_DIR}/scripts/build_test_evidence.py \
|
|
69
|
-
--project-root <target-project-root> \
|
|
70
|
-
--evidence-dir <evidence-dir> \
|
|
71
|
-
execute --request requests/focused.json
|
|
72
|
-
|
|
73
|
-
python3 ${SKILL_DIR}/scripts/build_test_evidence.py \
|
|
74
|
-
--project-root <target-project-root> \
|
|
75
|
-
--evidence-dir <evidence-dir> \
|
|
76
|
-
resume --manifest manifest.json --inventory target-inventory.json
|
|
77
|
-
```
|
|
78
|
-
|
|
79
|
-
The subcommands cover fixed mechanics only:
|
|
80
|
-
|
|
81
|
-
- `inventory`: expand model-selected project-relative patterns, independently enumerate all actual files under each declared module root, reject files that are neither categorized nor structurally excluded, hash actual files, and bind discovery inputs;
|
|
82
|
-
- `execute`: run the declared native command in the current project workspace and append a runner-generated receipt;
|
|
83
|
-
- `resume`: compare hashes and stage outputs to compute an invalidation decision while retaining `semantic_review_required=true`.
|
|
84
|
-
|
|
85
|
-
## Replay
|
|
86
|
-
|
|
87
|
-
Replay a prior receipt through the runner rather than treating attestation as historical proof:
|
|
88
|
-
|
|
89
|
-
```bash
|
|
90
|
-
python3 ${SKILL_DIR}/scripts/build_test_evidence.py \
|
|
91
|
-
--project-root <target-project-root> \
|
|
92
|
-
--evidence-dir <evidence-dir> \
|
|
93
|
-
execute --replay-receipt receipts/<receipt>.json
|
|
94
|
-
```
|
|
95
|
-
|
|
96
|
-
The recorded request hash must still match. Replay emits a new receipt linked with `replay_of`; it never overwrites the historical receipt. A replay can still differ because project state, installed tools, clocks, external test services, or environment values changed. Report those differences instead of claiming historical equivalence.
|
|
97
|
-
|
|
98
|
-
## External Target Safety
|
|
99
|
-
|
|
100
|
-
Do not block merely because a string contains `production`; names can be misleading. Classify every external target using endpoint/configuration evidence:
|
|
101
|
-
|
|
102
|
-
- block `production` and `unknown` external targets;
|
|
103
|
-
- permit only model-classified isolated/test/staging targets with explicit allow evidence and no deny evidence;
|
|
104
|
-
- preserve the complete evidence used for the decision;
|
|
105
|
-
- keep local fakes and in-process contract mocks classified `local` with `external=false`.
|
|
106
|
-
|
|
107
|
-
The model owns the semantic classification; the runner enforces the declared production/unknown safety invariant and rejects contradictory allow/deny records.
|