@ai-dev-methodologies/rlp-desk 0.2.4 → 0.3.1

package/README.md

@@ -40,9 +40,11 @@ curl -sSL https://raw.githubusercontent.com/ai-dev-methodologies/rlp-desk/main/i
 
 You'll be asked to confirm each item:
 - **Slug** — project identifier
-- **User Stories** — discrete, testable units of work
+- **User Stories** — discrete, testable units with Given/When/Then acceptance criteria
+- **Task Type & Risk Level** — code/visual/content/integration/infra × LOW/MEDIUM/HIGH/CRITICAL
 - **Iteration Unit** — one story per iteration (incremental) or all at once (fast)
 - **Verification Commands** — how to check the work
+- **Ambiguity Gate** — AC quality scoring (IL-2, 0-12 scale, blocks init if < 6)
 - **Models** — which Claude model for Worker/Verifier
 
 ### 3. Run
@@ -97,12 +99,38 @@ for iteration in 1..max_iter:
   8. Update status, report to user, continue or stop
 ```
 
+### Verification Policy (v0.3.0)
+
+RLP Desk enforces a comprehensive verification policy defined in `governance.md`:
+
+**Iron Laws (§1a)** — 4 absolute rules that cannot be violated:
+- **IL-1**: No completion claims without fresh verification evidence
+- **IL-2**: No init without AC quality score ≥ 6 (Ambiguity Gate)
+- **IL-3**: No pass with TODO in any required verification layer
+- **IL-4**: No pass without test count ≥ AC count × 3
+
+**Evidence Gate (§1b)** — 5-step protocol: IDENTIFY → RUN → READ → VERIFY → ONLY THEN claim
+
+**Risk Classification (§1c)** — Proportional verification layers per risk level:
+
+| Risk | Required Layers |
+|------|----------------|
+| LOW | L1 (Unit) + L3 (E2E) |
+| MEDIUM | L1 + L2 (Integration) + L3 |
+| HIGH | L1 + L2 + L3 + L4 (Deploy) |
+| CRITICAL | L1 + L2 + L3 + L4 + mutation testing |
+
+**Execution Traceability (§1f)** — Always-on, not flag-gated:
+- Worker records `execution_steps` in done-claim.json (what was done, in what order, with evidence)
+- Verifier records `reasoning` in verify-verdict.json (why each judgment was made)
+
 ### Circuit Breakers
 
 | Condition | Action |
 |-----------|--------|
 | Context unchanged for 3 iterations | BLOCKED |
 | Same error repeated twice | Upgrade model, retry once, then BLOCKED |
+| 3 consecutive failures | Architecture Escalation (§7¾) → report to user |
 | Max iterations reached | TIMEOUT |
 
 ### Model Routing
@@ -140,6 +168,8 @@ for iteration in 1..max_iter:
 | `--codex-reasoning low\|medium\|high` | high | Reasoning effort for Codex |
 | `--verify-mode per-us\|batch` | per-us | Verification strategy (see below) |
 | `--verify-consensus` | off | Cross-engine consensus verification (see below) |
+| `--debug` | off | Debug logging to `logs/<slug>/debug.log` |
+| `--with-self-verification` | off | Campaign-level post-loop analysis report |
 
 ## Execution Modes
 
@@ -334,6 +364,7 @@ mkdir my-calc && cd my-calc
 - [Architecture](docs/architecture.md) — Design philosophy, Agent() and tmux execution modes
 - [Getting Started](docs/getting-started.md) — Step-by-step tutorial with the calculator example
 - [Protocol Reference](docs/protocol-reference.md) — Full protocol specification
+- [Future Plans](docs/TODO-verification-next.md) — P3 items and upcoming features
 
 ## Contributing
 

package/docs/TODO-verification-next.md

@@ -0,0 +1,59 @@
+# Verification Policy — Next Iterations
+
+> Items scoped out of the feature/verification-policy branch.
+> P0-P2 (governance + templates) are complete. These items are planned for subsequent iterations.
+
+---
+
+## --with-self-verification Flag (Campaign-Level Analysis)
+
+Post-campaign analysis that reads all iteration artifacts and generates a versioned report.
+
+### Concept
+- Separate from `--debug` (which logs Leader decisions)
+- After COMPLETE/BLOCKED/TIMEOUT, Leader analyzes all done-claims and verdicts
+- Generates `logs/<slug>/self-verification-report-NNN.md` (versioned per run)
+- Cumulative data stored in `logs/<slug>/self-verification-data.json`
+
+### Report Sections (9-section template defined in rlp-desk.md step 9)
+1. Automated Validation Summary
+2. Failure Deep Dive
+3. Worker Process Quality (§1f audit)
+4. Verifier Judgment Quality (§1f audit)
+5. AC Lifecycle
+6. Test-Spec Adherence
+7. Patterns: Strengths & Weaknesses
+8. Recommendations for Next Cycle (Brainstorm / PRD / Test-Spec)
+9. Blind Spots
+
+### Open Design Items
+- [ ] Automated report generation (currently manual Leader analysis)
+- [ ] Cross-campaign trend analysis (compare report-001 vs report-002)
+- [ ] Integration with brainstorm (Leader reads previous report at brainstorm start)
+
+---
+
+## P3: External Tool Integration + Domain Specialization
+
+P0-P2 (governance policies + templates) form the foundation. P3 requires external dependencies and is planned for separate feature branches.
+
+### P3-1: Domain Rule Packs
+- **Purpose**: Domain-specific verification rule sets (finance, healthcare, security)
+- **Why separate**: Different in nature from universal governance. Requires plugin architecture.
+- [ ] Plugin loading mechanism design
+- [ ] Finance domain rule pack (first)
+- [ ] Rule pack authoring guide
+
+### P3-2: Playwright Agents
+- **Purpose**: Automated verification for visual/content task types (screenshot comparison, accessibility checks)
+- **Why separate**: Requires Playwright installation + browser binaries + CI environment setup.
+- [ ] Playwright integration wrapper
+- [ ] Screenshot comparison verification logic
+- [ ] CI environment guide
+
+### P3-3: Mutahunter / Spec Kit
+- **Purpose**: Automated mutation testing execution for CRITICAL risk
+- **Why separate**: Requires language-specific tool wrappers (mutmut, Stryker, go-mutesting). Governance defines the Gate only.
+- [ ] Language-specific mutation tool wrappers
+- [ ] Mutation score collection + verdict integration
+- [ ] Spec Kit: test-spec auto-generation helper

package/docs/architecture.md

@@ -48,6 +48,31 @@ RLP Desk supports two modes for running the Leader loop. Both honor the same gov
 | **Agent() — "Smart mode"** (default) | LLM (current session) | Dynamic — Leader reasons about which model to use each iteration | Active Claude Code session | Interactive development, complex routing decisions |
 | **Tmux — "Lean mode"** | Shell script (`run_ralph_desk.zsh`) | Static — set via `WORKER_MODEL`/`VERIFIER_MODEL` env vars | None (runs detached) | Long campaigns, CI, observability, zero-token orchestration |
 
+### Verification Policy Layer
+
+Both modes enforce the same verification policy (governance §1a-§1f):
+
+```
+                    ┌─────────────────────────────────┐
+                    │     Governance (§1a-§1f)         │
+                    │  Iron Laws · Evidence Gate       │
+                    │  Risk Classification · Layers    │
+                    │  Checkpoints · Traceability      │
+                    └──────────┬──────────────────────┘
+                               │ enforced by
+              ┌────────────────┼────────────────┐
+              ▼                ▼                ▼
+         Worker Template  Verifier Template  Leader Loop
+         (Test-First,     (12-step process,  (Contract review,
+          12 Shortcuts,    5 reasoning       Checkpoints,
+          execution_steps) categories)       Escalation)
+```
+
+Key design decisions:
+- **execution_steps** (Worker) and **reasoning** (Verifier) are always-on (§1f), not gated by flags
+- **`--with-self-verification`** adds post-campaign analysis only — does not change loop behavior
+- **Risk-proportional layers**: LOW gets L1+L3, CRITICAL gets L1+L2+L3+L4+mutation
+
 **Agent() mode** is synchronous and simple: each `Agent()` call blocks until the subprocess finishes, then the Leader reads the filesystem. No polling, no signal files, no tmux.
 
 **Tmux mode** trades dynamic routing for visibility and independence. The shell Leader writes prompts to files, sends short trigger commands via `tmux send-keys`, and polls structured JSON signal files (`iter-signal.json`, `verify-verdict.json`) for control flow. It uses proven tmux patterns — write-then-notify, pane ID stability, copy-mode guards, heartbeat monitoring — for reliable, race-free orchestration.

package/docs/getting-started.md

@@ -45,10 +45,12 @@ The brainstorm phase interactively determines:
 |------|---------|
 | **Slug** | `loop-test` |
 | **Objective** | Implement calc.py + test_calc.py |
-| **User Stories** | US-001: calculator functions, US-002: pytest tests |
+| **User Stories** | US-001: calculator functions (Given/When/Then AC format) |
+| **Task Type & Risk** | code, LOW |
 | **Iteration Unit** | One user story per iteration |
 | **Verification** | `python3 -m pytest test_calc.py -v` |
 | **Models** | Worker: sonnet, Verifier: opus |
+| **Ambiguity Gate (IL-2)** | AC quality score ≥ 6 required to proceed |
 | **Max Iterations** | 10 |
 
 On approval, brainstorm offers to run `init` automatically.
@@ -81,9 +83,11 @@ This creates the scaffold:
 Edit `.claude/ralph-desk/plans/prd-loop-test.md` to define your user stories and acceptance criteria. See [`examples/calculator/`](../examples/calculator/.claude/ralph-desk/plans/prd-loop-test.md) for a complete example.
 
 Key sections:
-- **User Stories** with specific, testable acceptance criteria
+- **User Stories** with Given/When/Then acceptance criteria, Task Type, and Risk Level
+- **Boundary Cases** for each US
+- **Verification Layers** (L1-L4 based on risk level per governance §1c)
 - **Technical Constraints** (e.g., "Python 3 + pytest only")
-- **Done When** conditions
+- **Done When** conditions (must reference Evidence Gate §1b)
 
 ## Step 6: Define the Test Spec
 
@@ -147,7 +151,10 @@ If you want to run the loop again:
 ## Tips
 
 - **Start small**: One or two user stories for your first loop
-- **Be specific in acceptance criteria**: "function returns float" is testable; "function works well" is not
+- **Use Given/When/Then**: "Given 10 and 5, When add, Then 15" — not "function works well"
+- **Set risk levels**: LOW for docs/config, MEDIUM for features, HIGH for deploys, CRITICAL for security/finance
 - **Include verification commands**: The verifier needs concrete commands to run
 - **One story per iteration**: Each worker should do one bounded action
 - **Check logs when stuck**: `logs/<slug>/iter-NNN.worker-prompt.md` shows exactly what the worker received
+- **Review done-claims**: Worker's `execution_steps` show exactly what was done and in what order
+- **Review verdict reasoning**: Verifier's `reasoning` shows why each judgment was made