prizmkit 1.1.136 → 1.1.138
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bundled/VERSION.json +3 -3
- package/bundled/dev-pipeline/prizmkit_runtime/runner_models.py +0 -1
- package/bundled/dev-pipeline/prizmkit_runtime/runner_prompts.py +0 -1
- package/bundled/dev-pipeline/prizmkit_runtime/runners.py +3 -5
- package/bundled/dev-pipeline/scripts/generate-bootstrap-prompt.py +24 -40
- package/bundled/dev-pipeline/scripts/generate-bugfix-prompt.py +3 -5
- package/bundled/dev-pipeline/scripts/generate-recovery-prompt.py +4 -5
- package/bundled/dev-pipeline/scripts/generate-refactor-prompt.py +3 -5
- package/bundled/dev-pipeline/scripts/prompt_framework.py +0 -5
- package/bundled/dev-pipeline/templates/bootstrap-prompt.md +3 -3
- package/bundled/dev-pipeline/templates/bootstrap-tier3.md +24 -26
- package/bundled/dev-pipeline/templates/bugfix-bootstrap-prompt.md +5 -4
- package/bundled/dev-pipeline/templates/feature-list-schema.json +1 -1
- package/bundled/dev-pipeline/templates/refactor-bootstrap-prompt.md +3 -2
- package/bundled/dev-pipeline/templates/sections/bugfix-mission.md +1 -1
- package/bundled/dev-pipeline/templates/sections/bugfix-phase-review.md +2 -10
- package/bundled/dev-pipeline/templates/sections/bugfix-reminders.md +2 -1
- package/bundled/dev-pipeline/templates/sections/context-budget-rules.md +2 -2
- package/bundled/dev-pipeline/templates/sections/phase-implement-agent.md +5 -5
- package/bundled/dev-pipeline/templates/sections/phase-implement-full.md +5 -5
- package/bundled/dev-pipeline/templates/sections/phase-review-agent.md +2 -12
- package/bundled/dev-pipeline/templates/sections/phase-review-full.md +2 -10
- package/bundled/dev-pipeline/templates/sections/phase-specify-plan-full.md +1 -1
- package/bundled/dev-pipeline/templates/sections/refactor-mission.md +1 -1
- package/bundled/dev-pipeline/templates/sections/refactor-phase-implement.md +2 -2
- package/bundled/dev-pipeline/templates/sections/refactor-phase-review.md +2 -10
- package/bundled/dev-pipeline/templates/sections/refactor-reminders.md +2 -1
- package/bundled/dev-pipeline/templates/sections/subagent-timeout-recovery.md +5 -4
- package/bundled/dev-pipeline/tests/test_generate_bootstrap_prompt.py +11 -17
- package/bundled/dev-pipeline/tests/test_generate_bugfix_prompt.py +3 -1
- package/bundled/dev-pipeline/tests/test_python_runner_parity.py +1 -6
- package/bundled/dev-pipeline/tests/test_unified_cli.py +3 -5
- package/bundled/skills/_metadata.json +2 -2
- package/bundled/skills/prizmkit-test/SKILL.md +18 -22
- package/bundled/skills/prizmkit-test/assets/authoritative-records.schema.json +6 -78
- package/bundled/skills/prizmkit-test/assets/evidence-manifest.schema.json +1 -2
- package/bundled/skills/prizmkit-test/assets/evidence-package-template.json +9 -56
- package/bundled/skills/prizmkit-test/references/boundary-coverage-protocol.md +2 -2
- package/bundled/skills/prizmkit-test/references/evidence-protocol.md +13 -13
- package/bundled/skills/prizmkit-test/references/evidence-request-protocol.md +8 -16
- package/bundled/skills/prizmkit-test/references/examples.md +3 -5
- package/bundled/skills/prizmkit-test/references/test-generation-steps.md +6 -8
- package/bundled/skills/prizmkit-test/references/test-report-template.md +4 -7
- package/bundled/skills/prizmkit-test/references/trusted-evidence-execution.md +6 -8
- package/bundled/skills/prizmkit-test/scripts/build_test_evidence.py +70 -247
- package/bundled/skills/prizmkit-test/scripts/validate_test_evidence.py +251 -201
- package/package.json +1 -1
|
@@ -1,23 +1,23 @@
|
|
|
1
|
-
### Implement —
|
|
1
|
+
### Implement — Plan Execution
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Execute the reviewed plan through the active implementation skill. The skill and runtime determine the execution topology; preserve the shared artifact directory, active checkout, and task checkpoints.
|
|
4
4
|
|
|
5
5
|
Before starting, check plan.md Tasks:
|
|
6
6
|
```bash
|
|
7
7
|
{{RUNTIME_HELPER_CMD}} text count .prizmkit/specs/{{FEATURE_SLUG}}/plan.md --pattern "- [ ]"
|
|
8
8
|
```
|
|
9
9
|
- If result is `0` (all tasks already `[x]`) → **SKIP to Review**.
|
|
10
|
-
- If non-zero →
|
|
10
|
+
- If non-zero → execute the remaining tasks below.
|
|
11
11
|
|
|
12
12
|
**Build artifacts rule**: After any build/compile command, ensure output is in `.gitignore`. Never commit binaries/build output.
|
|
13
13
|
|
|
14
14
|
**3a.** Run `/prizmkit-implement` directly:
|
|
15
15
|
- Continue from the `spec.md` / `plan.md` / `context-snapshot.md` content you created in Phase 1-2; do not re-read self-authored artifacts just because implementation starts
|
|
16
16
|
- Re-read `plan.md` or `context-snapshot.md` only when resuming/continuing, when current context was compacted or lost, or when task checkbox state may have changed outside this session
|
|
17
|
-
-
|
|
17
|
+
- Follow the implementation skill's current execution and delegation contract
|
|
18
18
|
- Use context-snapshot.md Section 4 File Manifest for targeted source reads; avoid re-reading source files already summarized there unless they changed after the snapshot
|
|
19
19
|
- Implements task-by-task, marking each `[x]` immediately
|
|
20
|
-
-
|
|
20
|
+
- Remember to update checkpoints after finishing a series of tasks
|
|
21
21
|
|
|
22
22
|
**3b. Focused checks only (no test gate here)**:
|
|
23
23
|
- During implementation, do not run mandatory periodic or full-suite tests.
|
|
@@ -1,23 +1,23 @@
|
|
|
1
|
-
### Implement —
|
|
1
|
+
### Implement — Plan Execution
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Execute the reviewed plan through the active implementation skill. The skill and runtime determine the execution topology; preserve the shared artifact directory, active checkout, and task checkpoints.
|
|
4
4
|
|
|
5
5
|
Before starting, check plan.md Tasks:
|
|
6
6
|
```bash
|
|
7
7
|
{{RUNTIME_HELPER_CMD}} text count .prizmkit/specs/{{FEATURE_SLUG}}/plan.md --pattern "- [ ]"
|
|
8
8
|
```
|
|
9
9
|
- If result is `0` (all tasks already `[x]`) → **SKIP to Review**.
|
|
10
|
-
- If non-zero →
|
|
10
|
+
- If non-zero → execute the remaining tasks below.
|
|
11
11
|
|
|
12
12
|
**Build artifacts rule**: After any build/compile command, ensure output is in `.gitignore`. Never commit binaries/build output.
|
|
13
13
|
|
|
14
14
|
**3a.** Run `/prizmkit-implement` directly:
|
|
15
15
|
- Continue from the `spec.md` / `plan.md` / `context-snapshot.md` content you created in Phase 1-2; do not re-read self-authored artifacts just because implementation starts
|
|
16
16
|
- Re-read `plan.md` or `context-snapshot.md` only when resuming/continuing, when current context was compacted or lost, or when task checkbox state may have changed outside this session
|
|
17
|
-
-
|
|
17
|
+
- Follow the implementation skill's current execution and delegation contract
|
|
18
18
|
- Use context-snapshot.md Section 4 File Manifest for targeted source reads; avoid re-reading source files already summarized there unless they changed after the snapshot
|
|
19
19
|
- Implements task-by-task, marking each `[x]` immediately
|
|
20
|
-
-
|
|
20
|
+
- Remember to update checkpoints after finishing a series of tasks
|
|
21
21
|
|
|
22
22
|
**3b. Focused checks only (no test gate here)**:
|
|
23
23
|
- During implementation, do not run mandatory periodic or full-suite tests.
|
|
@@ -1,18 +1,8 @@
|
|
|
1
1
|
### Review — Code Review
|
|
2
2
|
|
|
3
|
-
Review the completed implementation before the scoped feature test gate runs. Use `artifact_dir=.prizmkit/specs/{{FEATURE_SLUG}}/` so `/prizmkit-code-review` can read the spec, plan, context snapshot, implementation log, and current diff. Do not
|
|
3
|
+
Review the completed implementation before the scoped feature test gate runs. Use `artifact_dir=.prizmkit/specs/{{FEATURE_SLUG}}/` so `/prizmkit-code-review` can read the spec, plan, context snapshot, implementation log, and current diff. Follow the skill's current review contract for review execution, repairs, verification, evidence, and handoff. Do not invoke another review skill or add a second review path outside that contract.
|
|
4
4
|
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
The Code Review execution boundary is closed for this phase:
|
|
8
|
-
|
|
9
|
-
- Do not delegate the mandatory Main-Agent review or invoke another review skill or workflow.
|
|
10
|
-
- Do not add review execution outside the single optional Reviewer owned and gated by `prizmkit-code-review`.
|
|
11
|
-
- The optional Reviewer must be structurally read-only, unable to execute arbitrary commands or downstream execution, and natively resumable as the same unit; otherwise strict downgrade applies.
|
|
12
|
-
- Direct reading, searching, editing, and testing by the current Main Agent remain allowed.
|
|
13
|
-
- `review-report.md` is the only persisted review artifact; do not append separate Review Notes to `context-snapshot.md`.
|
|
14
|
-
|
|
15
|
-
Missing required evidence, an unsafe repair, a failed verification, or failure to converge by round ten produces `NEEDS_FIXES`.
|
|
5
|
+
Missing required evidence, an unsafe repair, a failed verification, or failure to converge by the contract's review limit produces `NEEDS_FIXES`.
|
|
16
6
|
|
|
17
7
|
**Gate Check — Final Review Result**:
|
|
18
8
|
|
|
@@ -2,21 +2,13 @@
|
|
|
2
2
|
|
|
3
3
|
Review the completed implementation before the full Feature test gate runs. A review result is a required predecessor; never accept or route around testing as if a review had passed.
|
|
4
4
|
|
|
5
|
-
Use `artifact_dir=.prizmkit/specs/{{FEATURE_SLUG}}/` so `/prizmkit-code-review` can read the spec, plan, context snapshot, implementation log, and current diff.
|
|
5
|
+
Use `artifact_dir=.prizmkit/specs/{{FEATURE_SLUG}}/` so `/prizmkit-code-review` can read the spec, plan, context snapshot, implementation log, and current diff. Follow the skill's current review contract for review execution, repairs, verification, evidence, and handoff. Do not invoke another review skill or add a second review path outside that contract.
|
|
6
6
|
|
|
7
7
|
- `REVIEW_PASS` → update checkpoint metadata with `stage=code-review`, `status=REVIEW_PASS`, `stage_result=REVIEW_PASS`, `next_stage=prizmkit-test`, and `resume_from=prizmkit-test`; then run the full test gate.
|
|
8
8
|
- `REVIEW_NEEDS_FIXES` → preserve findings, increment the shared outer `repair_round`, route `prizmkit-implement` → `prizmkit-code-review` → `prizmkit-test`, and do not accept any test result from before the repaired review.
|
|
9
9
|
- If the outer repair round is already three, record `WORKFLOW_BLOCKED`; do not start another repair route.
|
|
10
10
|
|
|
11
|
-
Missing required evidence, unsafe repair, failed verification, or failure to converge
|
|
12
|
-
|
|
13
|
-
The Code Review execution boundary is closed:
|
|
14
|
-
|
|
15
|
-
- Do not delegate the mandatory Main-Agent review or invoke another review skill or workflow.
|
|
16
|
-
- Do not add review execution outside the single optional Reviewer owned and gated by `prizmkit-code-review`.
|
|
17
|
-
- The optional Reviewer must be structurally read-only, unable to execute arbitrary commands or downstream execution, and natively resumable as the same unit; otherwise strict downgrade applies.
|
|
18
|
-
- Direct reading, searching, editing, and testing by the current Main Agent remain allowed.
|
|
19
|
-
- `review-report.md` is the only persisted review artifact; do not append separate Review Notes to `context-snapshot.md`.
|
|
11
|
+
Missing required evidence, unsafe repair, failed verification, or failure to converge under the review contract produces `REVIEW_NEEDS_FIXES`.
|
|
20
12
|
|
|
21
13
|
**Gate Check — Final Review Result**:
|
|
22
14
|
|
|
@@ -71,7 +71,7 @@ Before proceeding past CP-1, verify:
|
|
|
71
71
|
|
|
72
72
|
### Plan Self-Check — Orchestrator Direct
|
|
73
73
|
|
|
74
|
-
|
|
74
|
+
The active planning skill and runtime determine whether any additional planning execution is appropriate. Do not treat this bounded self-check as a substitute for the planning skill's own review contract.
|
|
75
75
|
|
|
76
76
|
1. Re-read only `context-snapshot.md`, `spec.md`, and `plan.md`.
|
|
77
77
|
2. Cross-check spec.md and plan.md Tasks against the feature description and Verification Gates in the Task Contract.
|
|
@@ -4,6 +4,6 @@ You are the **refactor session orchestrator**. Execute Refactor {{REFACTOR_ID}}:
|
|
|
4
4
|
|
|
5
5
|
**CRITICAL**: Preserve observable behavior. Do not change product behavior unless the refactor contract explicitly requires it.
|
|
6
6
|
|
|
7
|
-
**Execution model**:
|
|
7
|
+
**Execution model**: Follow the active skills and runtime contracts for planning, implementation, review, recovery, and commit. Those contracts determine the execution topology, permissions, and delegation boundaries.
|
|
8
8
|
|
|
9
9
|
**NON-INTERACTIVE MODE**: There is NO human on the other end. Resolve uncertainty from existing code and conservative project patterns.
|
|
@@ -1,11 +1,11 @@
|
|
|
1
|
-
### Phase 2: Implement —
|
|
1
|
+
### Phase 2: Implement — Plan Execution
|
|
2
2
|
|
|
3
3
|
**Goal**: execute the refactor plan while preserving behavior.
|
|
4
4
|
|
|
5
5
|
Run `/prizmkit-implement` directly with `artifact_dir=.prizmkit/refactor/{{REFACTOR_ID}}/`.
|
|
6
6
|
|
|
7
7
|
Rules:
|
|
8
|
-
-
|
|
8
|
+
- Follow the implementation skill's current execution and delegation contract.
|
|
9
9
|
- Establish a baseline test result before structural changes when feasible.
|
|
10
10
|
- Execute plan tasks in order and mark each `[x]` immediately.
|
|
11
11
|
- After each behavior-sensitive task, run the smallest relevant behavior-preservation check.
|
|
@@ -1,16 +1,8 @@
|
|
|
1
1
|
### Phase 3: Review — Code Review & Behavior Verification
|
|
2
2
|
|
|
3
|
-
Run `/prizmkit-code-review` directly with `artifact_dir=.prizmkit/refactor/{{REFACTOR_ID}}
|
|
3
|
+
Run `/prizmkit-code-review` directly with `artifact_dir=.prizmkit/refactor/{{REFACTOR_ID}}/` and follow the skill's complete review contract for review execution, repairs, verification, evidence, and handoff. Do not invoke another review skill or add a second review path outside that contract.
|
|
4
4
|
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
- Do not delegate the mandatory Main-Agent review or invoke another review skill or workflow.
|
|
8
|
-
- Do not add review execution outside the single optional Reviewer owned and gated by `prizmkit-code-review`.
|
|
9
|
-
- The optional Reviewer must be structurally read-only, unable to execute arbitrary commands or downstream execution, and natively resumable as the same unit; otherwise strict downgrade applies.
|
|
10
|
-
- Direct reading, searching, editing, and testing by the current Main Agent remain allowed.
|
|
11
|
-
- `review-report.md` is the only persisted review artifact; do not append separate Review Notes to another artifact.
|
|
12
|
-
|
|
13
|
-
Missing required evidence, an unsafe repair, a failed verification, or failure to converge by round ten produces `NEEDS_FIXES`.
|
|
5
|
+
Missing required evidence, an unsafe repair, a failed verification, or failure to converge under the review contract produces `NEEDS_FIXES`.
|
|
14
6
|
|
|
15
7
|
**Gate Check — Final Review Result**:
|
|
16
8
|
|
|
@@ -3,7 +3,8 @@
|
|
|
3
3
|
- Use L1 skills: `/prizmkit-plan`, `/prizmkit-implement`, `/prizmkit-code-review`, `/prizmkit-retrospective`, `/prizmkit-committer`.
|
|
4
4
|
- All refactor skills use `artifact_dir=.prizmkit/refactor/{{REFACTOR_ID}}/`.
|
|
5
5
|
- Behavior preservation is the first priority.
|
|
6
|
-
-
|
|
6
|
+
- Follow the active skills and runtime contracts; they determine execution topology, review ownership, and delegation boundaries
|
|
7
|
+
- On execution-unit timeout, preserve the active checkout and artifacts and follow the owning contract's recovery or downgrade path
|
|
7
8
|
- Main orchestrator runs `/prizmkit-code-review` directly.
|
|
8
9
|
- Commit with `refactor(<scope>):`, not `feat:` or `fix:`.
|
|
9
10
|
- Checkpoint state is the recovery truth; skipped steps must be marked `skipped`.
|
|
@@ -1,8 +1,9 @@
|
|
|
1
|
-
##
|
|
1
|
+
## Execution-Unit Timeout Recovery
|
|
2
2
|
|
|
3
|
-
This recovery path applies
|
|
3
|
+
This recovery path applies to any delegated execution unit created by the active skill or runtime. The unit's own skill contract determines whether it may be resumed, replaced, or handled by the Main Agent; do not infer those rules from this prompt.
|
|
4
4
|
|
|
5
|
-
If
|
|
5
|
+
If an execution unit times out:
|
|
6
6
|
1. `{{RUNTIME_HELPER_CMD}} artifact list .prizmkit/specs/{{FEATURE_SLUG}}` — check what exists
|
|
7
7
|
2. If `context-snapshot.md` exists: open a recovery prompt with `"Read .prizmkit/specs/{{FEATURE_SLUG}}/context-snapshot.md for project context and the Implementation Log. Run git diff HEAD to see actual code changes already made. Do NOT re-read individual source files unless the File Manifest directs you to."` + only remaining steps + `model: "lite"`
|
|
8
|
-
3.
|
|
8
|
+
3. Apply the active skill's retry and continuation contract. Do not create a replacement execution unit when that contract requires native continuation or strict downgrade.
|
|
9
|
+
4. After the allowed recovery attempts are exhausted, the Main Agent continues directly when safe or records `failure-log.md` with the timeout and last observable state.
|
|
@@ -51,10 +51,8 @@ FORBIDDEN_THREE_STRIKE = "3-" + "strike"
|
|
|
51
51
|
FORBIDDEN_TWENTY_STEP = "20-" + "step window"
|
|
52
52
|
|
|
53
53
|
REVIEW_PROMPT_REQUIRED = (
|
|
54
|
-
"
|
|
55
|
-
"
|
|
56
|
-
"skill-owned independent Reviewer",
|
|
57
|
-
"`review-report.md` is the only persisted review artifact",
|
|
54
|
+
"review-report.md",
|
|
55
|
+
"Final Result",
|
|
58
56
|
)
|
|
59
57
|
|
|
60
58
|
REVIEW_PROMPT_FORBIDDEN = (
|
|
@@ -246,7 +244,7 @@ def write_fresh_structured_evidence(project_root):
|
|
|
246
244
|
"EVIDENCE_VALIDATE",
|
|
247
245
|
]
|
|
248
246
|
for name in stage_names:
|
|
249
|
-
stages.append({"name": name, "status": "complete", "
|
|
247
|
+
stages.append({"name": name, "status": "complete", "outputs": []})
|
|
250
248
|
manifest = {
|
|
251
249
|
"protocol_version": "1.0",
|
|
252
250
|
"evidence_id": evidence_id,
|
|
@@ -1235,10 +1233,8 @@ class TestDirectOrchestratorImplementationPrompts:
|
|
|
1235
1233
|
assert FORBIDDEN_TEST_CMD_PLACEHOLDER not in rendered
|
|
1236
1234
|
assert FORBIDDEN_THREE_STRIKE not in rendered
|
|
1237
1235
|
assert FORBIDDEN_TWENTY_STEP not in rendered
|
|
1238
|
-
assert "
|
|
1239
|
-
assert "
|
|
1240
|
-
assert "strict structural capability gate" in rendered
|
|
1241
|
-
assert "skill-owned independent Reviewer" in rendered
|
|
1236
|
+
assert "review-report.md" in rendered
|
|
1237
|
+
assert "Final Result" in rendered
|
|
1242
1238
|
assert "low=0" not in rendered
|
|
1243
1239
|
assert "Reviewer 3" not in rendered
|
|
1244
1240
|
|
|
@@ -1273,7 +1269,7 @@ class TestDirectOrchestratorImplementationPrompts:
|
|
|
1273
1269
|
rendered = Path("dev-pipeline/templates/bootstrap-tier3.md").read_text(encoding="utf-8")
|
|
1274
1270
|
|
|
1275
1271
|
assert "/prizmkit-implement" in rendered
|
|
1276
|
-
assert "Implement —
|
|
1272
|
+
assert "Implement — Plan Execution" in rendered
|
|
1277
1273
|
assert "Spawn Dev subagent" not in rendered
|
|
1278
1274
|
assert "Spawn Dev agent" not in rendered
|
|
1279
1275
|
assert "Spawn Reviewer agent" not in rendered
|
|
@@ -1663,7 +1659,7 @@ class TestFeaturePromptUnifiedFullGuidance:
|
|
|
1663
1659
|
prompt = _render_feature_prompt(tmp_path / "fallback-tier3", mode="lite", complexity="low")
|
|
1664
1660
|
|
|
1665
1661
|
assert requested.index("bootstrap-tier3.md") < requested.index("bootstrap-prompt.md") if "bootstrap-prompt.md" in requested else True
|
|
1666
|
-
assert "# Dev-Pipeline Session Bootstrap —
|
|
1662
|
+
assert "# Dev-Pipeline Session Bootstrap — Full Guidance" in prompt
|
|
1667
1663
|
assert "bootstrap-tier1.md" not in requested
|
|
1668
1664
|
assert "bootstrap-tier2.md" not in requested
|
|
1669
1665
|
|
|
@@ -1931,9 +1927,8 @@ class TestHeadlessPromptCleanupF033:
|
|
|
1931
1927
|
refactor_prompt = _render_refactor_prompt(tmp_path / "refactor-mixed", platform="claude")
|
|
1932
1928
|
|
|
1933
1929
|
assert "{{" + "REVIEWER_" + "SUBAGENT_PATH" + "}}" not in replacements
|
|
1934
|
-
assert "
|
|
1935
|
-
assert "
|
|
1936
|
-
assert "skill-owned independent Reviewer" in refactor_prompt
|
|
1930
|
+
assert "review-report.md" in refactor_prompt
|
|
1931
|
+
assert "Final Result" in refactor_prompt
|
|
1937
1932
|
assert "Reviewer 3" not in refactor_prompt
|
|
1938
1933
|
|
|
1939
1934
|
def test_bugfix_refactor_explicit_codex_override(self, tmp_path, monkeypatch):
|
|
@@ -1955,9 +1950,8 @@ class TestHeadlessPromptCleanupF033:
|
|
|
1955
1950
|
refactor_prompt = _render_refactor_prompt(tmp_path / "refactor-codex", platform="claude", explicit_platform="codex")
|
|
1956
1951
|
|
|
1957
1952
|
assert "{{" + "REVIEWER_" + "SUBAGENT_PATH" + "}}" not in replacements
|
|
1958
|
-
assert "
|
|
1959
|
-
assert "
|
|
1960
|
-
assert "skill-owned independent Reviewer" in refactor_prompt
|
|
1953
|
+
assert "review-report.md" in refactor_prompt
|
|
1954
|
+
assert "Final Result" in refactor_prompt
|
|
1961
1955
|
assert "Reviewer 3" not in refactor_prompt
|
|
1962
1956
|
|
|
1963
1957
|
|
|
@@ -251,7 +251,9 @@ class TestHeadlessRecoveryReframing:
|
|
|
251
251
|
def test_bugfix_template_does_not_spawn_monitor_or_require_compact(self):
|
|
252
252
|
content = Path("dev-pipeline/templates/bugfix-bootstrap-prompt.md").read_text(encoding="utf-8")
|
|
253
253
|
|
|
254
|
-
assert "
|
|
254
|
+
assert "On execution-unit timeout, preserve the active checkout and artifacts" in content
|
|
255
|
+
assert "follow the owning contract's recovery or downgrade path" in content
|
|
256
|
+
assert "Do not spawn persistent background subagents for headless recovery" not in content
|
|
255
257
|
assert REMOVED_MAX_LOG_ENV not in content
|
|
256
258
|
assert REMOVED_MAX_LOG_HUMAN not in content
|
|
257
259
|
assert ("209" + "7152") not in content
|
|
@@ -1007,7 +1007,6 @@ def test_prompt_generation_result_preserves_per_item_model(monkeypatch, tmp_path
|
|
|
1007
1007
|
"checkpoint_path": str(paths.project_root / ".prizmkit" / "bugfix" / "B-001" / "workflow-checkpoint.json"),
|
|
1008
1008
|
"model": "per-item-model",
|
|
1009
1009
|
"pipeline_mode": "standard",
|
|
1010
|
-
"agent_count": 3,
|
|
1011
1010
|
}
|
|
1012
1011
|
),
|
|
1013
1012
|
stderr="",
|
|
@@ -1056,7 +1055,6 @@ def test_prompt_generation_passes_explicit_execution_root_to_generator(monkeypat
|
|
|
1056
1055
|
"checkpoint_path": str(execution_root / ".prizmkit" / "specs" / "001-low" / "workflow-checkpoint.json"),
|
|
1057
1056
|
"model": "",
|
|
1058
1057
|
"pipeline_mode": "full",
|
|
1059
|
-
"agent_count": 3,
|
|
1060
1058
|
}
|
|
1061
1059
|
),
|
|
1062
1060
|
stderr="",
|
|
@@ -1121,7 +1119,6 @@ def _install_runner_session_fakes(monkeypatch, runners, *, statuses=("success",)
|
|
|
1121
1119
|
output_path=session_paths.prompt_path,
|
|
1122
1120
|
model="per-item-model",
|
|
1123
1121
|
pipeline_mode="full",
|
|
1124
|
-
agent_count=3,
|
|
1125
1122
|
checkpoint_path=execution_root / ".prizmkit" / "specs" / "001-low" / "workflow-checkpoint.json",
|
|
1126
1123
|
artifact_path=execution_root / ".prizmkit" / "specs" / "001-low",
|
|
1127
1124
|
raw={},
|
|
@@ -1445,7 +1442,6 @@ def test_explicit_worktree_materializes_ignored_support_assets_before_prompt_and
|
|
|
1445
1442
|
output_path=session_paths.prompt_path,
|
|
1446
1443
|
model="",
|
|
1447
1444
|
pipeline_mode="full",
|
|
1448
|
-
agent_count=3,
|
|
1449
1445
|
checkpoint_path=execution_root / ".prizmkit" / "specs" / "001-support" / "workflow-checkpoint.json",
|
|
1450
1446
|
artifact_path=execution_root / ".prizmkit" / "specs" / "001-support",
|
|
1451
1447
|
raw={},
|
|
@@ -1910,8 +1906,7 @@ def test_foreground_runner_emits_structured_feature_progress(monkeypatch, tmp_pa
|
|
|
1910
1906
|
assert "Feature: F-001 — Readable output" in captured.err
|
|
1911
1907
|
assert "Code retry: 1 / 3" in captured.err
|
|
1912
1908
|
assert "Preflight: no user-visible changes to preserve before feature task F-001" in captured.err
|
|
1913
|
-
assert "Pipeline mode: full (
|
|
1914
|
-
assert "Agents: 3" in captured.err
|
|
1909
|
+
assert "Pipeline mode: full (Full Guidance)" in captured.err
|
|
1915
1910
|
assert "Spawning AI CLI session: F-001-session-1" in captured.err
|
|
1916
1911
|
assert "Session result: success" in captured.err
|
|
1917
1912
|
|
|
@@ -812,7 +812,7 @@ def test_session_preamble_writes_redacted_start_command_for_all_launch_profiles(
|
|
|
812
812
|
assert str(prompt) not in "\n".join(lines[:6])
|
|
813
813
|
|
|
814
814
|
|
|
815
|
-
def
|
|
815
|
+
def test_headless_review_guidance_defers_to_skill_review_contract():
|
|
816
816
|
headless_guidance_files = [
|
|
817
817
|
REPO_ROOT / "dev-pipeline" / "templates" / "sections" / "phase-review-full.md",
|
|
818
818
|
REPO_ROOT / "dev-pipeline" / "templates" / "bootstrap-tier3.md",
|
|
@@ -828,11 +828,9 @@ def test_headless_review_guidance_uses_main_agent_review_loop():
|
|
|
828
828
|
|
|
829
829
|
for path in headless_guidance_files:
|
|
830
830
|
text = path.read_text(encoding="utf-8")
|
|
831
|
-
assert "
|
|
832
|
-
assert "
|
|
831
|
+
assert "current review contract" in text, path
|
|
832
|
+
assert "second review path outside that contract" in text, path
|
|
833
833
|
assert "text final-verdict" in text, path
|
|
834
|
-
assert "strict structural capability gate" in text, path
|
|
835
|
-
assert "skill-owned independent Reviewer" in text, path
|
|
836
834
|
assert "low=0" not in text and "Reviewer 3" not in text, path
|
|
837
835
|
assert "forced-worktree changed-content fallback" not in text, path
|
|
838
836
|
assert "isolation:" not in text, path
|
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
{
|
|
2
|
-
"version": "1.1.
|
|
2
|
+
"version": "1.1.138",
|
|
3
3
|
"skills": {
|
|
4
4
|
"prizmkit": {
|
|
5
5
|
"description": "Framework introduction and navigation for the formal single-requirement lifecycle, project initialization, Prizm docs, and independent deployment.",
|
|
@@ -72,7 +72,7 @@
|
|
|
72
72
|
"hasScripts": false
|
|
73
73
|
},
|
|
74
74
|
"prizmkit-test": {
|
|
75
|
-
"description": "Mandatory post-review auditable test evidence with TEST_PASS/TEST_FAIL/TEST_BLOCKED verdicts, repair-scope routing,
|
|
75
|
+
"description": "Mandatory post-review auditable test evidence with TEST_PASS/TEST_FAIL/TEST_BLOCKED verdicts, repair-scope routing, current-workspace execution evidence, replay, and deterministic validation.",
|
|
76
76
|
"tier": "1",
|
|
77
77
|
"category": "prizmkit-skill",
|
|
78
78
|
"hasAssets": true,
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: "prizmkit-test"
|
|
3
|
-
description: "Auditable test-evidence protocol for AI-generated behavior changes. Classifies changes, discovers affected modules and regression rings, models observable behavior and risk, builds project-native tests and contract-driven mocks,
|
|
3
|
+
description: "Auditable test-evidence protocol for AI-generated behavior changes. Classifies changes, discovers affected modules and regression rings, models observable behavior and risk, builds project-native tests and contract-driven mocks, validates current-workspace test evidence, preserves content-addressed raw evidence, and deterministically returns TEST_PASS, TEST_FAIL, or TEST_BLOCKED. Use after code/config/schema/dependency changes or when users ask to test, verify, add complete module tests, inspect boundaries, or produce auditable evidence. Supports full-project, module, feature, and scope=this-change. (project)"
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# PrizmKit Test
|
|
@@ -11,16 +11,16 @@ This skill orchestrates each target project's native runners and conventions. It
|
|
|
11
11
|
|
|
12
12
|
## Model Choice vs Fixed Evidence Mechanics
|
|
13
13
|
|
|
14
|
-
Choose project-semantic parameters from the target project: framework, commands, cwd, timeout, retry/attempt policy, concurrency, Mock tools, test layers, module boundary, inventory patterns,
|
|
14
|
+
Choose project-semantic parameters from the target project: framework, commands, cwd, timeout, retry/attempt policy, concurrency, Mock tools, test layers, module boundary, inventory patterns, and environment classification. Do not hard-code ecosystem-specific defaults when the project can decide them.
|
|
15
15
|
|
|
16
|
-
Fix only the protocol mechanisms needed for safe, replayable evidence: project/evidence/request locators, schema-shaped requests, path confinement, real process execution, complete raw capture, runner-generated
|
|
16
|
+
Fix only the protocol mechanisms needed for safe, replayable evidence: project/evidence/request locators, schema-shaped requests, path confinement, real process execution, complete raw capture, runner-generated receipts, append-only history, and resume invalidation. Read `${SKILL_DIR}/references/trusted-evidence-execution.md` before inventory, execution, resume, or replay; it defines the trust boundary and builder commands. Read `${SKILL_DIR}/references/evidence-request-protocol.md` when authoring any request or authoritative JSON record; it defines request ownership, structured N/A, and the schema map.
|
|
17
17
|
|
|
18
18
|
## Canonical Evidence Builder Lifecycle
|
|
19
19
|
|
|
20
20
|
The Main Agent supplies semantic requests only; it does not author authoritative package records. Initialize and drive one evidence directory through the builder in this order:
|
|
21
21
|
|
|
22
22
|
```text
|
|
23
|
-
init → capture-change → inventory → prepare-tests → execute
|
|
23
|
+
init → capture-change → inventory → prepare-tests → execute → finalize → render-report → validate --attest → resume
|
|
24
24
|
```
|
|
25
25
|
|
|
26
26
|
Use `${SKILL_DIR}/scripts/build_test_evidence.py` for every lifecycle command:
|
|
@@ -29,13 +29,12 @@ Use `${SKILL_DIR}/scripts/build_test_evidence.py` for every lifecycle command:
|
|
|
29
29
|
- `capture-change`: captures the complete canonical change, including untracked, deleted, renamed, and copied files; never substitute plain `git diff`. `inventory` then requires its changed-file set to match the capture exactly and rejects stale captures or unaccounted live paths.
|
|
30
30
|
- `inventory`: hashes current source/tests/contracts/lockfiles, rejects category overlap, excludes runtime-only paths, and validates directory roots plus the complete capture-derived changed-file set.
|
|
31
31
|
- `prepare-tests`: discovers repository-native test files from the canonical capture and inventory, generates stable builder-owned test IDs, derives existing/added/modified status, and snapshots only added/modified live test sources.
|
|
32
|
-
- `execute`: runs native project commands and owns `executions.json`, `receipts/`, and `raw/`.
|
|
33
|
-
- `
|
|
34
|
-
- `finalize`: owns manifest, stage hashes, verdict, generated-test snapshots, and evidence identity.
|
|
32
|
+
- `execute`: runs native project commands and owns `executions.json`, `receipts/`, and `raw/`. Each request uses exactly one binding mode: `test_ids`, `test_paths`, or `auto_bind=true`. The builder infers recognized package-wide or explicit file/directory/glob command scope, blocks opaque commands or mismatched declarations, and persists verified canonical `test_ids` only.
|
|
33
|
+
- `finalize`: rechecks canonical capture freshness, blocks every selected unreliable execution, and owns manifest, stage hashes, verdict, generated-test snapshots, and evidence identity.
|
|
35
34
|
- `render-report`: derives `test-report.md`; it is never authoritative.
|
|
36
35
|
- `resume`: preserves prior receipts and refuses to rebind an identity-changing package; create a new package after scope/source identity changes.
|
|
37
36
|
|
|
38
|
-
`test-plan.json` is the only source of builder-generated test IDs.
|
|
37
|
+
`test-plan.json` is the only source of builder-generated test IDs. An execution request provides exactly one of native `test_paths`, `auto_bind=true`, or canonical `test_ids`; the actual command must expose the same deterministic scope, including all prepared tests for recognized package-wide commands. The builder blocks opaque or mismatched command scope and rewrites verified requests to canonical `test_ids` only before execution. Assess all five layers explicitly, but set `required=true` only when risk requires it; every omitted layer needs structured N/A evidence. The same binding applies test ↔ behavior ↔ execution; current-workspace receipts are the execution evidence.
|
|
39
38
|
|
|
40
39
|
The builder/validator, not the Main Agent, produces or updates `manifest.json`, `executions.json`, `receipts/*`, `validation.json`, `verdict.json`, `test-report.md`, and `source-change.patch`. `validation.json` is produced only by `validate_test_evidence.py --attest`; a package is not a `TEST_PASS` until the second strict validation succeeds.
|
|
41
40
|
|
|
@@ -174,18 +173,16 @@ If a valid test reveals a business/contract defect, do not fix production behavi
|
|
|
174
173
|
|
|
175
174
|
### 7. EXECUTE_PROVE
|
|
176
175
|
|
|
177
|
-
Run required layers in plan order through `${SKILL_DIR}/scripts/build_test_evidence.py execute`. Use `test_paths
|
|
176
|
+
Run required layers in plan order through `${SKILL_DIR}/scripts/build_test_evidence.py execute`. Use exactly one of `test_paths`, `auto_bind=true`, or canonical `test_ids`; each must exactly match a deterministically inferable native command scope. The builder blocks opaque or mismatched commands and persists verified canonical test IDs before the native command runs. The model selects project-native argv, cwd, timeout, attempts, concurrency, layer, environment classification, and tool probes in schema-shaped requests. The runner performs the actual process execution and appends complete environment/tool versions, raw stdout/stderr hashes, request hash, runner hash, and unique receipt to `executions.json`. Do not accept caller-authored execution JSON and do not retry until green.
|
|
178
177
|
|
|
179
|
-
|
|
178
|
+
Validate the current implementation directly in the project root:
|
|
180
179
|
|
|
181
|
-
-
|
|
182
|
-
-
|
|
183
|
-
-
|
|
184
|
-
-
|
|
185
|
-
- Completely remove mutations and verify the isolated current copy is unchanged. Never rebind completed proof to the mutable live project tree.
|
|
186
|
-
- Classify each proof as `PROVEN`, structured `NOT_APPLICABLE`, or `UNPROVEN`.
|
|
180
|
+
- Run every required layer through the project-native runner; do not create historical checkouts or mutate a second copy of the repository.
|
|
181
|
+
- Bind each applicable behavior risk to planned tests and selected, reliable execution receipts. A passing current-state test is valid evidence even when the historical implementation would also have passed it.
|
|
182
|
+
- Preserve complete runner receipts, raw outputs, request bindings, and replay links. The builder owns execution provenance; the Main Agent does not author receipt records.
|
|
183
|
+
- Required test failures produce `TEST_FAIL`; any selected unreliable execution, missing mapping, unavailable execution, unresolved risk, or budget truncation produces `TEST_BLOCKED`. `TEST_PASS` requires every selected execution to be reliable with exit code zero and every planned test to have a selected reliable zero-exit receipt.
|
|
187
184
|
|
|
188
|
-
|
|
185
|
+
TDD history, baseline-failure proof, and mutation sensitivity are outside this skill. This skill validates whether the current implementation satisfies the modeled behavior and risk evidence.
|
|
189
186
|
|
|
190
187
|
### 8. EVIDENCE_PACKAGE
|
|
191
188
|
|
|
@@ -197,7 +194,6 @@ Create `.prizmkit/test/evidence/<evidence-id>/` from the package template with:
|
|
|
197
194
|
- `behavior-risk-matrix.json`
|
|
198
195
|
- `test-plan.json`
|
|
199
196
|
- `infrastructure-changes.json`
|
|
200
|
-
- `differential-proof.json`
|
|
201
197
|
- `executions.json`
|
|
202
198
|
- `verdict.json`
|
|
203
199
|
- `source-change.patch`
|
|
@@ -220,7 +216,7 @@ python3 ${SKILL_DIR}/scripts/validate_test_evidence.py \
|
|
|
220
216
|
--project-root <target-project-root>
|
|
221
217
|
```
|
|
222
218
|
|
|
223
|
-
If this pre-check finds errors, fix them in the package before the second pass. The validator checks every authoritative record against its shipped schema, live target inventories, patch/diff binding,
|
|
219
|
+
If this pre-check finds errors, fix them in the package before the second pass. The validator checks every authoritative record against its shipped schema, live target inventories, patch/diff binding, changed-file/module-root/exclusion/Regression Ring cross-links, generated-test snapshots, planned-test execution links, runner-generated receipts/request/raw-output hashes, successful behavior-risk mappings, production/unknown external-target safety, and package integrity.
|
|
224
220
|
|
|
225
221
|
**Second pass (attestation, with `--attest`):**
|
|
226
222
|
|
|
@@ -231,7 +227,7 @@ python3 ${SKILL_DIR}/scripts/validate_test_evidence.py \
|
|
|
231
227
|
--attest
|
|
232
228
|
```
|
|
233
229
|
|
|
234
|
-
`--attest` writes an integrity/protocol attestation into `validation.json`, re-derives `test-report.md`, re-hashes final records in `manifest.json`, then runs a **second** strict validation against the refreshed package. A package is not a `TEST_PASS` until this second strict validation succeeds. Use builder replay when feasible to rerun recorded requests and create new linked receipts. Store validation output. Tampering, schema/identity/hash mismatch, source/test/lockfile drift, missing matrix mappings, wrong execution order,
|
|
230
|
+
`--attest` writes an integrity/protocol attestation into `validation.json`, re-derives `test-report.md`, re-hashes final records in `manifest.json`, then runs a **second** strict validation against the refreshed package. A package is not a `TEST_PASS` until this second strict validation succeeds. Use builder replay when feasible to rerun recorded requests and create new linked receipts. Store validation output. Tampering, schema/identity/hash mismatch, source/test/lockfile drift, missing matrix mappings, wrong execution order, unjustified structured N/A, unresolved risk, or a false real-environment claim fails validation.
|
|
235
231
|
|
|
236
232
|
## Resume and Idempotency
|
|
237
233
|
|
|
@@ -249,9 +245,9 @@ For a large module or bounded budget, persist matrix/checkpoint progress and ret
|
|
|
249
245
|
|
|
250
246
|
Only these testing-domain outcomes are valid:
|
|
251
247
|
|
|
252
|
-
- `TEST_PASS`: every required module behavior/risk and Regression Ring check is
|
|
248
|
+
- `TEST_PASS`: every required module behavior/risk and Regression Ring check is covered by current-workspace tests, required executions pass reliably, no verdict-capable unresolved edge remains, and deterministic validation passes.
|
|
253
249
|
- `TEST_FAIL`: a valid reliable test reproduces an implementation or resolved-contract failure. The authoritative verdict record must include `repair_scope` and enough evidence for the next stage to route safely.
|
|
254
|
-
- `TEST_BLOCKED`: unknown scope, conflicting truth, unavailable/unreliable/flaky necessary execution, incomplete evidence,
|
|
250
|
+
- `TEST_BLOCKED`: unknown scope, conflicting truth, unavailable/unreliable/flaky necessary execution, incomplete evidence, unmapped tests, budget truncation, or deterministic validation failure.
|
|
255
251
|
|
|
256
252
|
No conditional pass exists.
|
|
257
253
|
|