okstra 0.55.0 → 0.56.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (29) hide show
  1. package/bin/okstra +24 -7
  2. package/docs/project-structure-overview.md +0 -1
  3. package/docs/superpowers/plans/2026-05-25-okstra-project-root-rename.md +0 -1
  4. package/docs/superpowers/plans/2026-06-07-stage-conformance-qa-phase2.md +275 -0
  5. package/docs/superpowers/plans/2026-06-07-stage-conformance-qa-phase3.md +282 -0
  6. package/docs/superpowers/plans/2026-06-07-stage-conformance-qa-phase4a.md +147 -0
  7. package/docs/superpowers/plans/2026-06-07-stage-conformance-qa-phase4b.md +262 -0
  8. package/docs/superpowers/plans/2026-06-07-stage-conformance-qa-phase4c.md +184 -0
  9. package/docs/superpowers/plans/2026-06-07-stage-conformance-qa-phase4d.md +88 -0
  10. package/docs/superpowers/plans/2026-06-07-stage-conformance-qa-phase4e.md +250 -0
  11. package/docs/superpowers/plans/2026-06-07-stage-conformance-qa.md +409 -0
  12. package/docs/superpowers/specs/2026-06-07-stage-conformance-qa-design.md +169 -0
  13. package/package.json +1 -1
  14. package/runtime/BUILD.json +2 -2
  15. package/runtime/bin/lib/okstra/cli.sh +5 -1
  16. package/runtime/bin/lib/okstra/usage.sh +5 -0
  17. package/runtime/bin/okstra.sh +1 -0
  18. package/runtime/prompts/profiles/_implementation-verifier.md +23 -2
  19. package/runtime/prompts/profiles/final-verification.md +1 -0
  20. package/runtime/prompts/profiles/implementation-planning.md +4 -0
  21. package/runtime/python/okstra_ctl/conformance.py +270 -0
  22. package/runtime/python/okstra_ctl/paths.py +2 -0
  23. package/runtime/python/okstra_ctl/run.py +29 -0
  24. package/runtime/skills/okstra-run/SKILL.md +12 -0
  25. package/runtime/skills/okstra-setup/SKILL.md +35 -0
  26. package/runtime/validators/validate-implementation-plan-stages.py +28 -3
  27. package/runtime/validators/validate-run.py +96 -0
  28. package/src/okstra-dirs.mjs +1 -1
  29. package/src/migrate.mjs +0 -146
package/bin/okstra CHANGED
@@ -4,19 +4,36 @@ import { getPackageVersion } from "../src/version.mjs";
4
4
  const COMMANDS = new Map([
5
5
  ["paths", () => import("../src/paths.mjs").then((m) => m.run)],
6
6
  ["install", () => import("../src/install.mjs").then((m) => m.runInstall)],
7
- ["ensure-installed", () => import("../src/install.mjs").then((m) => m.runEnsureInstalled)],
8
- ["uninstall", () => import("../src/uninstall.mjs").then((m) => m.runUninstall)],
7
+ [
8
+ "ensure-installed",
9
+ () => import("../src/install.mjs").then((m) => m.runEnsureInstalled),
10
+ ],
11
+ [
12
+ "uninstall",
13
+ () => import("../src/uninstall.mjs").then((m) => m.runUninstall),
14
+ ],
9
15
  ["doctor", () => import("../src/doctor.mjs").then((m) => m.run)],
10
16
  ["setup", () => import("../src/setup.mjs").then((m) => m.run)],
11
- ["check-project", () => import("../src/check-project.mjs").then((m) => m.run)],
17
+ [
18
+ "check-project",
19
+ () => import("../src/check-project.mjs").then((m) => m.run),
20
+ ],
12
21
  ["config", () => import("../src/config.mjs").then((m) => m.run)],
13
- ["migrate", () => import("../src/migrate.mjs").then((m) => m.run)],
14
22
  ["task-list", () => import("../src/task-list.mjs").then((m) => m.run)],
15
23
  ["task-show", () => import("../src/task-show.mjs").then((m) => m.run)],
16
24
  ["context-cost", () => import("../src/context-cost.mjs").then((m) => m.run)],
17
- ["worktree-lookup", () => import("../src/worktree-lookup.mjs").then((m) => m.run)],
18
- ["plan-validate", () => import("../src/plan-validate.mjs").then((m) => m.run)],
19
- ["render-bundle", () => import("../src/render-bundle.mjs").then((m) => m.run)],
25
+ [
26
+ "worktree-lookup",
27
+ () => import("../src/worktree-lookup.mjs").then((m) => m.run),
28
+ ],
29
+ [
30
+ "plan-validate",
31
+ () => import("../src/plan-validate.mjs").then((m) => m.run),
32
+ ],
33
+ [
34
+ "render-bundle",
35
+ () => import("../src/render-bundle.mjs").then((m) => m.run),
36
+ ],
20
37
  ["render-views", () => import("../src/render-views.mjs").then((m) => m.run)],
21
38
  ["wizard", () => import("../src/wizard.mjs").then((m) => m.run)],
22
39
  ["token-usage", () => import("../src/token-usage.mjs").then((m) => m.run)],
@@ -137,7 +137,6 @@ So `runtime/` is not the only npm-published content. It is the install payload c
137
137
  | `setup` | `src/setup.mjs` | Create/update `<PROJECT_ROOT>/.okstra/project.json` |
138
138
  | `check-project` | `src/check-project.mjs` | Verify project registration |
139
139
  | `config` | `src/config.mjs` | Read/write project/global settings such as PR template path |
140
- | `migrate` | `src/migrate.mjs` | Move legacy `.project-docs/okstra/` state into `.okstra/` |
141
140
  | `task-list`, `task-show` | `src/task-list.mjs`, `src/task-show.mjs` | Task/run introspection for skills |
142
141
  | `context-cost` | `src/context-cost.mjs` | Estimate task bundle file/read context cost |
143
142
  | `worktree-lookup` | `src/worktree-lookup.mjs` | Look up a task-key's registered worktree |
@@ -95,7 +95,6 @@
95
95
  - [ ] `MigrationPlan` 은 dataclass, JSON 직렬화 가능 (dry-run 출력에 그대로 사용).
96
96
 
97
97
  ### Task 3.2: 진입점 3개 (단일 코어 호출 보장)
98
- - [ ] Node CLI: `bin/commands/migrate.mjs` — argparse + Python 모듈 호출.
99
98
  - [ ] Bash 진입점: `scripts/lib/okstra-ctl/cmd-migrate.sh` — thin adapter.
100
99
  - [ ] `bin/okstra` dispatcher 에 `migrate` 서브커맨드 등록.
101
100
  - [ ] `--dry-run` 기본, `--apply` 로 실제 실행, `--quiet` 로 출력 최소화.
@@ -0,0 +1,275 @@
1
+ # Stage Conformance QA — Phase 2 (게이트 판정 코어) Implementation Plan
2
+
3
+ > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
4
+
5
+ **Goal:** stage 별 conformance entry + 실행 결과를 받아 "PASS / WAIVED / EXEMPT / BLOCKING" 을 결정하는 **순수 판정 로직**을 `conformance.py` 에 추가한다(미실행·FAIL·MISSING = BLOCKING, 면제·waiver = 통과). 이게 DEV-9184 의 "silent mock-green" 을 막는 두뇌다.
6
+
7
+ **Architecture:** Phase 1(매니페스트 검증 + `QA-RESULT` 파서)의 사이드카(부수효과 없는) 확장. wiring(verifier 가 결과 기록, validate-run 이 판정 적용, run.py 진입 검증, planning 이 스크립트 생성)은 **Phase 3** 로 분리한다. 본 Phase 는 외부 파일을 읽지 않는 순수 함수만 추가하므로 완전 TDD 가능하다. SSOT: [`docs/superpowers/specs/2026-06-07-stage-conformance-qa-design.md`](../specs/2026-06-07-stage-conformance-qa-design.md) §6(실행·판정), §7(우회).
8
+
9
+ **Tech Stack:** Python 3 (stdlib + pytest).
10
+
11
+ **전제:** Phase 1 완료 — `scripts/okstra_ctl/conformance.py` 에 `validate_conformance_manifest`, `parse_qa_result`, `QaResult`, `CAPABILITY_WHITELIST` 존재.
12
+
13
+ ---
14
+
15
+ ## 판정 규칙 (spec §6, §7)
16
+
17
+ 단일 stage entry + 그 stage 의 실행 결과(`QaResult | None`)로부터:
18
+
19
+ | 조건 | status | ok | conditional |
20
+ |---|---|---|---|
21
+ | `exemption` 있음 | EXEMPT | True | False |
22
+ | `waiver` 있음 | WAIVED | True | **True** (conformance 미검증 — 사용자 확인) |
23
+ | 결과 없음(`None`) | BLOCKING | False | False |
24
+ | `overall == "MISSING"` (마커 없음) | BLOCKING | False | False |
25
+ | `overall == "FAIL"` | BLOCKING | False | False |
26
+ | `overall == "PASS"` | PASS | True | False |
27
+
28
+ 판정 우선순위: exemption → waiver → 결과 평가. (Phase 3 가 "면제 선언했지만 diff 가 실제 surface 를 건드림" 교차검증을 추가한다 — 본 Phase 는 선언을 신뢰한다.)
29
+
30
+ ---
31
+
32
+ ## Task 1: 게이트 판정 (`decide_conformance_gate`)
33
+
34
+ **Files:**
35
+ - Modify: `scripts/okstra_ctl/conformance.py`
36
+ - Test: `tests/test_okstra_ctl_conformance.py`
37
+
38
+ - [ ] **Step 1: 실패 테스트 추가**
39
+
40
+ ```python
41
+ # tests/test_okstra_ctl_conformance.py 의 import 에 추가
42
+ from okstra_ctl.conformance import ( # noqa: E402
43
+ ConformanceVerdict,
44
+ QaResult,
45
+ decide_conformance_gate,
46
+ )
47
+
48
+
49
+ def _gate_entry(**over):
50
+ base = {"stageKey": "t-stage-1", "exemption": None, "waiver": None}
51
+ base.update(over)
52
+ return base
53
+
54
+
55
+ def test_gate_pass_when_result_pass():
56
+ v = decide_conformance_gate(_gate_entry(), QaResult(overall="PASS", requirements={}))
57
+ assert isinstance(v, ConformanceVerdict)
58
+ assert v.status == "PASS" and v.ok is True and v.conditional is False
59
+
60
+
61
+ def test_gate_blocking_when_no_result():
62
+ v = decide_conformance_gate(_gate_entry(), None)
63
+ assert v.status == "BLOCKING" and v.ok is False
64
+ assert "never ran" in v.message
65
+
66
+
67
+ def test_gate_blocking_when_marker_missing():
68
+ v = decide_conformance_gate(_gate_entry(), QaResult(overall="MISSING", requirements={}))
69
+ assert v.status == "BLOCKING" and v.ok is False
70
+ assert "QA-RESULT" in v.message
71
+
72
+
73
+ def test_gate_blocking_when_fail():
74
+ v = decide_conformance_gate(_gate_entry(), QaResult(overall="FAIL", requirements={}))
75
+ assert v.status == "BLOCKING" and v.ok is False
76
+
77
+
78
+ def test_gate_exempt_passes_with_reason():
79
+ entry = _gate_entry(exemption={"reason": "doc-only", "declaredAt": "2026-06-07"})
80
+ v = decide_conformance_gate(entry, None)
81
+ assert v.status == "EXEMPT" and v.ok is True and v.conditional is False
82
+ assert "doc-only" in v.message
83
+
84
+
85
+ def test_gate_waiver_is_conditional():
86
+ entry = _gate_entry(waiver={"acknowledgedBy": "user", "reason": "replica down",
87
+ "at": "2026-06-07", "scope": ["db"]})
88
+ v = decide_conformance_gate(entry, None)
89
+ assert v.status == "WAIVED" and v.ok is True and v.conditional is True
90
+ assert "user" in v.message and "replica down" in v.message
91
+
92
+
93
+ def test_gate_exemption_takes_priority_over_failing_result():
94
+ entry = _gate_entry(exemption={"reason": "n/a", "declaredAt": "2026-06-07"})
95
+ v = decide_conformance_gate(entry, QaResult(overall="FAIL", requirements={}))
96
+ assert v.status == "EXEMPT" and v.ok is True
97
+ ```
98
+
99
+ - [ ] **Step 2: 실패 확인**
100
+
101
+ Run: `python3 -m pytest tests/test_okstra_ctl_conformance.py -q`
102
+ Expected: FAIL — `cannot import name 'ConformanceVerdict'` / `decide_conformance_gate`
103
+
104
+ - [ ] **Step 3: 구현 추가**
105
+
106
+ `conformance.py` 의 `parse_qa_result` 정의 다음에 추가:
107
+
108
+ ```python
109
+ @dataclass
110
+ class ConformanceVerdict:
111
+ stage_key: str
112
+ status: str # "PASS" | "BLOCKING" | "WAIVED" | "EXEMPT"
113
+ ok: bool # 진행 허용 여부 (PASS/WAIVED/EXEMPT 면 True)
114
+ conditional: bool # WAIVED 일 때만 True — conformance 미검증(사용자 확인)
115
+ message: str
116
+
117
+
118
+ def decide_conformance_gate(entry: dict, result: object) -> ConformanceVerdict:
119
+ """단일 stage entry + 실행 결과(`QaResult | None`)로 게이트 판정.
120
+
121
+ 우선순위: exemption → waiver → 결과 평가. 미실행/MISSING/FAIL 은 BLOCKING.
122
+ 면제·waiver 의 형태 검증은 `validate_conformance_manifest` 가 이미 보장한다.
123
+ """
124
+ key = entry.get("stageKey", "<unknown>")
125
+ exemption = entry.get("exemption")
126
+ if exemption:
127
+ return ConformanceVerdict(
128
+ key, "EXEMPT", True, False,
129
+ f"conformance exempted: {exemption.get('reason', '')}",
130
+ )
131
+ waiver = entry.get("waiver")
132
+ if waiver:
133
+ return ConformanceVerdict(
134
+ key, "WAIVED", True, True,
135
+ f"conformance waived by {waiver.get('acknowledgedBy', '?')}: "
136
+ f"{waiver.get('reason', '')}",
137
+ )
138
+ overall = getattr(result, "overall", None) if result is not None else None
139
+ if overall == "PASS":
140
+ return ConformanceVerdict(key, "PASS", True, False, "conformance PASS")
141
+ if overall is None:
142
+ return ConformanceVerdict(
143
+ key, "BLOCKING", False, False,
144
+ "conformance script never ran (no result recorded)",
145
+ )
146
+ if overall == "MISSING":
147
+ return ConformanceVerdict(
148
+ key, "BLOCKING", False, False,
149
+ "conformance script ran but emitted no QA-RESULT marker",
150
+ )
151
+ return ConformanceVerdict(key, "BLOCKING", False, False, f"conformance {overall}")
152
+ ```
153
+
154
+ - [ ] **Step 4: 통과 확인**
155
+
156
+ Run: `python3 -m pytest tests/test_okstra_ctl_conformance.py -q`
157
+ Expected: PASS (24 passed — 기존 17 + 신규 7)
158
+
159
+ - [ ] **Step 5: 커밋**
160
+
161
+ ```bash
162
+ git add scripts/okstra_ctl/conformance.py tests/test_okstra_ctl_conformance.py
163
+ git commit -m "feat(okstra_ctl/conformance): stage 게이트 판정(decide_conformance_gate)"
164
+ ```
165
+
166
+ ## Task 2: 사이드카 변환 + 매니페스트 일괄 평가
167
+
168
+ **Files:**
169
+ - Modify: `scripts/okstra_ctl/conformance.py`
170
+ - Test: `tests/test_okstra_ctl_conformance.py`
171
+
172
+ - [ ] **Step 1: 실패 테스트 추가**
173
+
174
+ ```python
175
+ from okstra_ctl.conformance import ( # noqa: E402 (위 import 에 합쳐도 됨)
176
+ evaluate_conformance,
177
+ qa_result_from_dict,
178
+ )
179
+
180
+
181
+ def test_qa_result_from_dict_roundtrip():
182
+ r = qa_result_from_dict({"overall": "PASS", "requirements": {"R-001": {"status": "PASS"}}})
183
+ assert r.overall == "PASS" and r.requirements["R-001"]["status"] == "PASS"
184
+
185
+
186
+ def test_qa_result_from_dict_defaults_to_missing():
187
+ assert qa_result_from_dict(None).overall == "MISSING"
188
+ assert qa_result_from_dict({"overall": "bogus"}).overall == "MISSING"
189
+ assert qa_result_from_dict({}).requirements == {}
190
+
191
+
192
+ def test_evaluate_conformance_mixes_verdicts():
193
+ manifest = {"entries": [
194
+ {"stageKey": "t-stage-1", "exemption": None, "waiver": None}, # PASS
195
+ {"stageKey": "t-stage-2", "exemption": None, "waiver": None}, # BLOCKING (no result)
196
+ {"stageKey": "t-stage-3",
197
+ "exemption": {"reason": "doc", "declaredAt": "x"}, "waiver": None}, # EXEMPT
198
+ ]}
199
+ results = {"t-stage-1": QaResult(overall="PASS", requirements={})}
200
+ verdicts = evaluate_conformance(manifest, results)
201
+ by_key = {v.stage_key: v for v in verdicts}
202
+ assert by_key["t-stage-1"].status == "PASS"
203
+ assert by_key["t-stage-2"].status == "BLOCKING"
204
+ assert by_key["t-stage-3"].status == "EXEMPT"
205
+
206
+
207
+ def test_evaluate_conformance_empty_manifest():
208
+ assert evaluate_conformance(None, {}) == []
209
+ assert evaluate_conformance({"entries": []}, {}) == []
210
+ ```
211
+
212
+ - [ ] **Step 2: 실패 확인**
213
+
214
+ Run: `python3 -m pytest tests/test_okstra_ctl_conformance.py -q`
215
+ Expected: FAIL — `cannot import name 'evaluate_conformance'` / `qa_result_from_dict`
216
+
217
+ - [ ] **Step 3: 구현 추가**
218
+
219
+ `conformance.py` 의 `decide_conformance_gate` 다음에 추가:
220
+
221
+ ```python
222
+ def qa_result_from_dict(data: object) -> QaResult:
223
+ """결과 사이드카(JSON dict)를 `QaResult` 로 복원. Phase 3 의 verifier 가 쓴
224
+ `result-stage-<N>.json` 을 validate-run 이 로드할 때 쓴다. 형태가 깨졌으면
225
+ overall='MISSING'(=BLOCKING 취급)으로 안전하게 강등한다."""
226
+ if not isinstance(data, dict):
227
+ return QaResult(overall="MISSING", requirements={})
228
+ overall = data.get("overall")
229
+ if overall not in ("PASS", "FAIL", "MISSING"):
230
+ overall = "MISSING"
231
+ reqs = data.get("requirements")
232
+ return QaResult(overall=overall, requirements=reqs if isinstance(reqs, dict) else {})
233
+
234
+
235
+ def evaluate_conformance(manifest: object, results_by_stage: object) -> list[ConformanceVerdict]:
236
+ """매니페스트 전 entry 에 대해 게이트 판정 목록을 반환.
237
+
238
+ `results_by_stage`: stageKey -> `QaResult`. 키가 없으면 미실행(None)으로 본다.
239
+ 매니페스트 구조 검증은 호출 전에 `validate_conformance_manifest` 로 끝낸다는 전제.
240
+ """
241
+ entries = manifest.get("entries") if isinstance(manifest, dict) else None
242
+ if not isinstance(entries, list):
243
+ return []
244
+ results = results_by_stage if isinstance(results_by_stage, dict) else {}
245
+ verdicts: list[ConformanceVerdict] = []
246
+ for entry in entries:
247
+ if not isinstance(entry, dict):
248
+ continue
249
+ result = results.get(entry.get("stageKey"))
250
+ verdicts.append(decide_conformance_gate(entry, result))
251
+ return verdicts
252
+ ```
253
+
254
+ - [ ] **Step 4: 통과 확인 + 전체 회귀**
255
+
256
+ Run: `python3 -m pytest tests/test_okstra_ctl_conformance.py -q` → Expected: PASS (28 passed)
257
+ Run: `python3 -m pytest tests/ -q` → Expected: 전부 통과(회귀 없음)
258
+
259
+ - [ ] **Step 5: 커밋**
260
+
261
+ ```bash
262
+ git add scripts/okstra_ctl/conformance.py tests/test_okstra_ctl_conformance.py
263
+ git commit -m "feat(okstra_ctl/conformance): 사이드카 변환 + evaluate_conformance 일괄 평가"
264
+ ```
265
+
266
+ ---
267
+
268
+ ## Phase 2 Self-Review 체크
269
+
270
+ - [ ] spec §6 의 미실행/MISSING/FAIL = BLOCKING, PASS = 통과가 모두 커버되는가?
271
+ - [ ] spec §7 의 면제(EXEMPT) / waiver(WAIVED, conditional) 가 커버되는가?
272
+ - [ ] Phase 3 가 의존할 공개 API(`decide_conformance_gate`, `evaluate_conformance`, `qa_result_from_dict`, `ConformanceVerdict`)가 노출되는가?
273
+ - [ ] 순수 함수만 추가했는가(파일 I/O·외부 의존 없음)? — wiring 은 Phase 3.
274
+
275
+ Phase 2 완료 후 Phase 3 계획(verifier 결과 기록 + validate-run 게이트 적용 + run.py 진입 검증 + planning 생성)을 작성한다.
@@ -0,0 +1,282 @@
1
+ # Stage Conformance QA — Phase 3 (validate-run 게이트) Implementation Plan
2
+
3
+ > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
4
+
5
+ **Goal:** `validators/validate-run.py` 가 implementation/final-verification 리포트에 대해 run 의 `qa/` 디렉터리(conformance 매니페스트 + 결과 사이드카)를 읽어 Phase 2 의 `evaluate_conformance` 로 게이트를 적용한다. BLOCKING verdict 는 run 검증 실패로 만든다 — DEV-9184 의 "정적 OK, 미검증 → silent pass" 를 실제 FAIL 로 전환하는 강제 지점.
6
+
7
+ **Architecture:** Phase 1(매니페스트 검증) + Phase 2(게이트 판정)의 순수 함수를 validate-run 에 wiring 한다. **매니페스트가 없으면 게이트는 inert**(early return) — 따라서 매니페스트를 생성하는 prompt 측(Phase 4)이 없어도 안전하게 단독 출하된다. SSOT: [`docs/superpowers/specs/2026-06-07-stage-conformance-qa-design.md`](../specs/2026-06-07-stage-conformance-qa-design.md) §6.
8
+
9
+ **Tech Stack:** Python 3 (stdlib + pytest). validate-run.py 는 importlib 로 로드되는 standalone 스크립트.
10
+
11
+ **전제:** Phase 1·2 완료 — `scripts/okstra_ctl/conformance.py` 에 `validate_conformance_manifest`, `evaluate_conformance`, `qa_result_from_dict`, `ConformanceVerdict` 존재.
12
+
13
+ ---
14
+
15
+ ## 결과 사이드카 계약 (이 Phase 에서 확정)
16
+
17
+ - 매니페스트: `<run_dir>/qa/conformance-manifest.json` (run_dir = `report_path.parent.parent`, 기존 `validate_worker_results_audit` 의 `report_path.parent.parent / "worker-results"` 패턴과 동형).
18
+ - 결과 사이드카(Phase 4 의 verifier 가 작성): `<run_dir>/qa/result-<stageKey>.json` = `{"stageKey","overall","ranAt","requirements"}`. 파일 부재 → 미실행(None) → BLOCKING.
19
+
20
+ ---
21
+
22
+ ## Task 1: conformance 게이트 함수 + 결과 로더
23
+
24
+ **Files:**
25
+ - Modify: `validators/validate-run.py`
26
+ - Test: `tests/test_validate_run_conformance.py` (신규)
27
+
28
+ - [ ] **Step 1: 실패 테스트 작성**
29
+
30
+ ```python
31
+ # tests/test_validate_run_conformance.py
32
+ from __future__ import annotations
33
+
34
+ import importlib.util
35
+ import json
36
+ from pathlib import Path
37
+
38
+ REPO_ROOT = Path(__file__).resolve().parent.parent
39
+ VALIDATOR_PATH = REPO_ROOT / "validators" / "validate-run.py"
40
+
41
+
42
+ def _load_validator():
43
+ spec = importlib.util.spec_from_file_location("validate_run", VALIDATOR_PATH)
44
+ module = importlib.util.module_from_spec(spec)
45
+ assert spec.loader is not None
46
+ spec.loader.exec_module(module)
47
+ return module
48
+
49
+
50
+ def _make_run(tmp_path: Path, entries, results):
51
+ run_dir = tmp_path / "runs" / "implementation"
52
+ (run_dir / "reports").mkdir(parents=True)
53
+ qa = run_dir / "qa"
54
+ qa.mkdir()
55
+ (qa / "conformance-manifest.json").write_text(json.dumps({"entries": entries}))
56
+ for key, overall in results.items():
57
+ (qa / f"result-{key}.json").write_text(
58
+ json.dumps({"stageKey": key, "overall": overall, "requirements": {}})
59
+ )
60
+ report = run_dir / "reports" / "final-report-implementation-001.md"
61
+ report.write_text("# fixture\n")
62
+ return report
63
+
64
+
65
+ def _entry(key, **over):
66
+ base = {"stageKey": key, "script": f"qa/{key}.ts", "runCommand": f"run {key}",
67
+ "requirementIds": ["R-001"], "requires": ["db"],
68
+ "passContract": "exit 0 = PASS", "exemption": None, "waiver": None}
69
+ base.update(over)
70
+ return base
71
+
72
+
73
+ def test_no_manifest_is_inert(tmp_path):
74
+ v = _load_validator()
75
+ run_dir = tmp_path / "runs" / "implementation"
76
+ (run_dir / "reports").mkdir(parents=True)
77
+ report = run_dir / "reports" / "final-report-implementation-001.md"
78
+ report.write_text("# fixture\n")
79
+ failures: list[str] = []
80
+ v._validate_conformance(report, failures)
81
+ assert failures == []
82
+
83
+
84
+ def test_passing_result_no_failure(tmp_path):
85
+ v = _load_validator()
86
+ report = _make_run(tmp_path, [_entry("t-stage-1")], {"t-stage-1": "PASS"})
87
+ failures: list[str] = []
88
+ v._validate_conformance(report, failures)
89
+ assert failures == []
90
+
91
+
92
+ def test_missing_result_is_blocking(tmp_path):
93
+ v = _load_validator()
94
+ report = _make_run(tmp_path, [_entry("t-stage-1")], {}) # no result file
95
+ failures: list[str] = []
96
+ v._validate_conformance(report, failures)
97
+ assert any("BLOCKING" in f and "t-stage-1" in f for f in failures)
98
+
99
+
100
+ def test_failing_result_is_blocking(tmp_path):
101
+ v = _load_validator()
102
+ report = _make_run(tmp_path, [_entry("t-stage-1")], {"t-stage-1": "FAIL"})
103
+ failures: list[str] = []
104
+ v._validate_conformance(report, failures)
105
+ assert any("BLOCKING" in f for f in failures)
106
+
107
+
108
+ def test_exemption_passes(tmp_path):
109
+ v = _load_validator()
110
+ entry = _entry("t-stage-1", exemption={"reason": "doc", "declaredAt": "2026-06-07"})
111
+ report = _make_run(tmp_path, [entry], {})
112
+ failures: list[str] = []
113
+ v._validate_conformance(report, failures)
114
+ assert failures == []
115
+
116
+
117
+ def test_waiver_passes_but_records_conditional(tmp_path):
118
+ v = _load_validator()
119
+ entry = _entry("t-stage-1", waiver={"acknowledgedBy": "user", "reason": "replica down",
120
+ "at": "2026-06-07", "scope": ["db"]})
121
+ report = _make_run(tmp_path, [entry], {})
122
+ failures: list[str] = []
123
+ v._validate_conformance(report, failures)
124
+ assert failures == [] # waiver = ok (conditional, not a failure)
125
+
126
+
127
+ def test_malformed_manifest_reported(tmp_path):
128
+ v = _load_validator()
129
+ report = _make_run(tmp_path, [{"stageKey": ""}], {}) # invalid entry
130
+ failures: list[str] = []
131
+ v._validate_conformance(report, failures)
132
+ assert any("conformance manifest" in f for f in failures)
133
+ ```
134
+
135
+ - [ ] **Step 2: 실패 확인**
136
+
137
+ Run: `python3 -m pytest tests/test_validate_run_conformance.py -q`
138
+ Expected: FAIL — `module 'validate_run' has no attribute '_validate_conformance'`
139
+
140
+ - [ ] **Step 3: 구현 추가**
141
+
142
+ `validators/validate-run.py` 상단의 `from okstra_project.dirs import tasks_root as _okstra_tasks_root` (line 33 부근) 다음에 import 추가:
143
+
144
+ ```python
145
+ from okstra_ctl.conformance import ( # noqa: E402
146
+ evaluate_conformance,
147
+ qa_result_from_dict,
148
+ validate_conformance_manifest,
149
+ )
150
+ ```
151
+
152
+ `validate_report` 함수 정의 바로 앞(`def validate_report(` 위)에 두 함수 추가:
153
+
154
+ ```python
155
+ def _load_conformance_results(qa_dir: Path, manifest: dict) -> dict:
156
+ """매니페스트 각 entry 에 대응하는 `result-<stageKey>.json` 을 로드.
157
+ 파일 부재/파손은 키를 비워 둬 미실행(None→BLOCKING)으로 흐르게 한다."""
158
+ results: dict = {}
159
+ for entry in manifest.get("entries", []):
160
+ key = entry.get("stageKey") if isinstance(entry, dict) else None
161
+ if not isinstance(key, str) or not key:
162
+ continue
163
+ sidecar = qa_dir / f"result-{key}.json"
164
+ if not sidecar.is_file():
165
+ continue
166
+ try:
167
+ results[key] = qa_result_from_dict(json.loads(sidecar.read_text()))
168
+ except (OSError, json.JSONDecodeError):
169
+ results[key] = qa_result_from_dict(None) # → MISSING → BLOCKING
170
+ return results
171
+
172
+
173
+ def _validate_conformance(report_path: Path, failures: list[str]) -> None:
174
+ """Tier 3 conformance 게이트(implementation / final-verification).
175
+
176
+ `<run_dir>/qa/conformance-manifest.json` 이 없으면 inert(선언된 conformance
177
+ 가 없다는 뜻 — 선언을 강제하는 것은 planning 계약(Phase 4)의 몫). 매니페스트가
178
+ 있으면 결과 사이드카와 함께 evaluate_conformance 로 판정하고 BLOCKING verdict
179
+ 를 run 검증 실패로 승격한다. WAIVED(conditional)/EXEMPT 는 통과시킨다.
180
+ """
181
+ qa_dir = report_path.parent.parent / "qa"
182
+ manifest_path = qa_dir / "conformance-manifest.json"
183
+ if not manifest_path.is_file():
184
+ return
185
+ try:
186
+ manifest = json.loads(manifest_path.read_text())
187
+ except (OSError, json.JSONDecodeError) as exc:
188
+ failures.append(f"conformance manifest unreadable at {manifest_path}: {exc}")
189
+ return
190
+ schema_errors = validate_conformance_manifest(manifest)
191
+ if schema_errors:
192
+ failures.extend(f"conformance manifest: {e}" for e in schema_errors)
193
+ return
194
+ results = _load_conformance_results(qa_dir, manifest)
195
+ for verdict in evaluate_conformance(manifest, results):
196
+ if not verdict.ok:
197
+ failures.append(
198
+ f"conformance gate BLOCKING for stage {verdict.stage_key}: "
199
+ f"{verdict.message}. Run the stage's conformance script (or declare "
200
+ f"an exemption / user waiver) — see "
201
+ f"docs/superpowers/specs/2026-06-07-stage-conformance-qa-design.md."
202
+ )
203
+ ```
204
+
205
+ - [ ] **Step 4: 통과 확인**
206
+
207
+ Run: `python3 -m pytest tests/test_validate_run_conformance.py -q`
208
+ Expected: PASS (7 passed)
209
+
210
+ - [ ] **Step 5: 커밋**
211
+
212
+ ```bash
213
+ git add validators/validate-run.py tests/test_validate_run_conformance.py
214
+ git commit -m "feat(validators): conformance Tier3 게이트 함수(_validate_conformance)"
215
+ ```
216
+
217
+ ## Task 2: 메인 dispatch 에 게이트 wiring
218
+
219
+ **Files:**
220
+ - Modify: `validators/validate-run.py`
221
+
222
+ - [ ] **Step 1: dispatch 에 호출 추가**
223
+
224
+ `validate-run.py` 의 메인 dispatch 에서 `validate_worker_results_audit` 호출 블록 다음, `if task_type == "improvement-discovery":` 앞에 추가:
225
+
226
+ ```python
227
+ if task_type in ("implementation", "final-verification"):
228
+ _validate_conformance(report_path, failures)
229
+ ```
230
+
231
+ (찾을 앵커 — 현재 코드:)
232
+
233
+ ```python
234
+ if task_type:
235
+ validate_worker_results_audit(report_path, task_type, failures)
236
+ if task_type == "improvement-discovery":
237
+ ```
238
+
239
+ - [ ] **Step 2: 기존 스위트 회귀 + workflow validator 확인**
240
+
241
+ Run: `python3 -m pytest tests/ -q`
242
+ Expected: 전부 통과(신규 7 포함, 회귀 없음). validate-workflow 의 final-verification fixture 는 `qa/` 디렉터리가 없으므로 게이트는 inert → 영향 없음.
243
+
244
+ Run: `bash validators/validate-workflow.sh`
245
+ Expected: 마지막에 PASS (게이트 inert).
246
+
247
+ - [ ] **Step 3: 코드 확인(수동)**
248
+
249
+ `grep -n "_validate_conformance" validators/validate-run.py` → 정의 1 + 호출 1 = 2건.
250
+
251
+ - [ ] **Step 4: 커밋**
252
+
253
+ ```bash
254
+ git add validators/validate-run.py
255
+ git commit -m "feat(validators): implementation/final-verification 에 conformance 게이트 연결"
256
+ ```
257
+
258
+ ---
259
+
260
+ ## Phase 3 Self-Review 체크
261
+
262
+ - [ ] 매니페스트 부재 시 inert(early return) 확인 — 기존 run 들이 깨지지 않는가?
263
+ - [ ] BLOCKING(미실행/MISSING/FAIL) → failures 추가, EXEMPT/WAIVED → 통과가 spec §6/§7 과 일치하는가?
264
+ - [ ] run_dir 해석(`report_path.parent.parent`)이 기존 `validate_worker_results_audit` 와 동형인가?
265
+ - [ ] 전체 pytest + workflow validator 통과(회귀 없음)?
266
+
267
+ ---
268
+
269
+ ## Phase 4 로 넘길 항목 + ⚠️ 미해결 설계 갬
270
+
271
+ 본 게이트는 매니페스트/결과가 존재할 때만 작동한다. 그것을 **생성·실행**하는 다음 작업들은 Phase 4 이며, 일부는 **계획 전 설계 확정이 필요**하다(추측 금지):
272
+
273
+ 1. **verifier 실행(prompt)**: `_implementation-verifier.md` 에 Tier 3 추가 — 각 entry 의 `runCommand` 실행 → `parse_qa_result` → `<run_dir>/qa/result-<stageKey>.json` 작성. (env/secret 접근, replica DB, mutation 가드 포함)
274
+ 2. **planning 생성(prompt)**: `implementation-planning.md` 가 stage 별 스크립트 + `conformance-manifest.json` + `Conformance exemption:` 라인 산출.
275
+ 3. **final-verification(prompt)**: 전 stage 합집합 실행.
276
+ 4. **우회 UX**: `--qa-waiver <stageKey>:"<reason>"` CLI(okstra.sh/cli.sh) + wizard picker → `run.py` 가 매니페스트 entry 의 `waiver` 채움.
277
+ 5. **env**: `project.json.qaEnv`(replicaDbDsn/appBaseUrl/envFile).
278
+
279
+ **⚠️ 설계 확정 필요(Phase 4 계획 전 brainstorm 보강 권장):**
280
+ - **매니페스트의 교차-phase 위치**: planning 이 만든 매니페스트/스크립트가 implementation run, 이어서 final-verification run 으로 어떻게 전달되는가? (task-key worktree vs stage worktree vs instruction-set 복사) — stage worktree isolation 과 정합해야 함.
281
+ - **"diff 가 surface 를 건드렸는데 매니페스트 entry 자체가 없음"** (planning 누락) 강제: 본 게이트는 선언된 entry 만 강제한다. 누락 탐지는 planning 계약 + diff-surface 휴리스틱이 필요(spec §7.1 선언형 면제 surface 교차검증과 함께).
282
+ - **env/secret 운영 모델**: replica DB·실행 env 를 누가 어떻게 주입하는가.