pi-rnd 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +74 -0
- package/agents/rnd-builder.md +98 -0
- package/agents/rnd-integrator.md +104 -0
- package/agents/rnd-planner.md +208 -0
- package/agents/rnd-verifier.md +164 -0
- package/dist/doctor.js +166 -0
- package/dist/doctor.js.map +1 -0
- package/dist/gates/bash-discipline.js +27 -0
- package/dist/gates/bash-discipline.js.map +1 -0
- package/dist/gates/read-evidence-pack.js +23 -0
- package/dist/gates/read-evidence-pack.js.map +1 -0
- package/dist/gates/registry.js +24 -0
- package/dist/gates/registry.js.map +1 -0
- package/dist/gates/rnd-dir-required.js +31 -0
- package/dist/gates/rnd-dir-required.js.map +1 -0
- package/dist/index.js +20 -0
- package/dist/index.js.map +1 -0
- package/dist/orchestrator/prompts.js +58 -0
- package/dist/orchestrator/prompts.js.map +1 -0
- package/dist/orchestrator/rnd-dir.js +20 -0
- package/dist/orchestrator/rnd-dir.js.map +1 -0
- package/dist/orchestrator/spawn.js +67 -0
- package/dist/orchestrator/spawn.js.map +1 -0
- package/dist/orchestrator/start.js +195 -0
- package/dist/orchestrator/start.js.map +1 -0
- package/dist/orchestrator/state.js +15 -0
- package/dist/orchestrator/state.js.map +1 -0
- package/dist/orchestrator/types.js +2 -0
- package/dist/orchestrator/types.js.map +1 -0
- package/docs/PI-API.md +574 -0
- package/docs/PORTING.md +105 -0
- package/package.json +57 -0
- package/skills/fp-practices/SKILL.md +128 -0
- package/skills/fp-practices/bash.md +114 -0
- package/skills/fp-practices/duckdb.md +116 -0
- package/skills/fp-practices/elixir.md +115 -0
- package/skills/fp-practices/javascript.md +119 -0
- package/skills/fp-practices/koka.md +120 -0
- package/skills/fp-practices/lean.md +120 -0
- package/skills/fp-practices/postgresql.md +120 -0
- package/skills/fp-practices/python.md +120 -0
- package/skills/fp-practices/svelte.md +114 -0
- package/skills/kiss-practices/SKILL.md +41 -0
- package/skills/kiss-practices/bash.md +70 -0
- package/skills/kiss-practices/duckdb.md +30 -0
- package/skills/kiss-practices/elixir.md +38 -0
- package/skills/kiss-practices/javascript.md +43 -0
- package/skills/kiss-practices/koka.md +34 -0
- package/skills/kiss-practices/lean.md +45 -0
- package/skills/kiss-practices/markdown.md +20 -0
- package/skills/kiss-practices/postgresql.md +31 -0
- package/skills/kiss-practices/python.md +64 -0
- package/skills/kiss-practices/svelte.md +59 -0
- package/skills/rnd-building/SKILL.md +256 -0
- package/skills/rnd-decomposition/SKILL.md +188 -0
- package/skills/rnd-experiments/SKILL.md +197 -0
- package/skills/rnd-failure-modes/SKILL.md +222 -0
- package/skills/rnd-iteration/SKILL.md +170 -0
- package/skills/rnd-orchestration/SKILL.md +314 -0
- package/skills/rnd-scaling/SKILL.md +188 -0
- package/skills/rnd-verification/SKILL.md +248 -0
- package/skills/using-rnd-framework/SKILL.md +65 -0
|
@@ -0,0 +1,256 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: rnd-building
|
|
3
|
+
description: "Use when implementing code within the R&D pipeline — TDD discipline, pre-registration compliance, honest self-assessment, and verification artifact production"
|
|
4
|
+
user-invocable: false
|
|
5
|
+
effort: medium
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# R&D Building
|
|
9
|
+
|
|
10
|
+
Implement ONE assigned task against its pre-registered success criteria. Write the test first. Watch it fail. Write minimal code to pass. Produce verification artifacts for the independent Verifier.
|
|
11
|
+
|
|
12
|
+
## The Iron Laws
|
|
13
|
+
|
|
14
|
+
```
|
|
15
|
+
1. NO PRODUCTION CODE WITHOUT A FAILING TEST FIRST
|
|
16
|
+
2. NO SILENT DEVIATIONS FROM THE PRE-REGISTERED APPROACH
|
|
17
|
+
3. DO NOT VERIFY YOUR OWN WORK
|
|
18
|
+
4. USE WRITE/EDIT TOOLS TO CREATE AND MODIFY FILES — NEVER BASH HEREDOCS
|
|
19
|
+
5. IF SLOP-GATE RETURNS WARN OR FAIL WITH ANY SEVERITY 3+ MATCH, RE-EDIT IMMEDIATELY — DO NOT DEFER
|
|
20
|
+
6. WHEN YOU HIT AN ISSUE (ERROR, WARNING, BROKEN TEST, BUG, GAP), FIX IT OR LOG IT TO found-issues.jsonl WITH decision="fixed"|"escalated" — SILENT DISMISSAL IS NOT AN OPTION
|
|
21
|
+
7. EXPLAIN BEFORE YOU WRITE — ONE LOGICAL CHANGE PER WRITE/EDIT, NOT WALLS OF CODE
|
|
22
|
+
8. DO NOT EMBED PIPELINE TASK IDs IN PROJECT CODE — NO TASK IDs IN COMMENTS, TEST NAMES, OR VARIABLE NAMES (RND ARTIFACT FILES IN $RND_DIR ARE EXEMPT)
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
### 0. Resolve RND_DIR
|
|
26
|
+
|
|
27
|
+
`$RND_DIR` is set by the orchestrator in the spawn prompt. If absent, look for it in the environment. Do not attempt to compute it from plugin paths — the orchestrator owns this value.
|
|
28
|
+
|
|
29
|
+
### 1. Read Your Assignment
|
|
30
|
+
|
|
31
|
+
Find your task in `$RND_DIR/plan.md`. Read its pre-registration — especially success criteria, approach, and the `fulfills` field (which links to specific VAL-AREA-NNN assertions in the Validation Contract). Also read:
|
|
32
|
+
- **Environment Setup** — runtime, package manager, dependencies, install commands
|
|
33
|
+
- **Testing Strategy** — test framework, baseline count, exact run commands for unit/integration/live tests
|
|
34
|
+
- **Worker Guidelines** — project boundaries (USE/OFF-LIMITS), coding conventions, architecture notes
|
|
35
|
+
|
|
36
|
+
### 2. Verify Preconditions
|
|
37
|
+
|
|
38
|
+
If the pre-registration has a `Preconditions:` field, check each assertion before writing code:
|
|
39
|
+
- **File existence:** run Glob for the declared path pattern
|
|
40
|
+
- **Content existence:** run Grep for the declared pattern in the declared file
|
|
41
|
+
- **Dependency presence:** run Read on the config file and check for the declared key
|
|
42
|
+
- **On failure:** report status `BLOCKED` with the specific failing assertion. Do not proceed.
|
|
43
|
+
|
|
44
|
+
### 2.5. Read Exploration Cache
|
|
45
|
+
|
|
46
|
+
If `$RND_DIR/exploration/` exists, read it before writing code — file summaries, key patterns, dependencies. The orchestrator may also inject pitfalls from prior builds; review them before writing any code.
|
|
47
|
+
|
|
48
|
+
### 2.75. Verify External Dependencies
|
|
49
|
+
|
|
50
|
+
Before writing any code, verify every external dependency listed in the pre-registration:
|
|
51
|
+
- **Read or query the actual external system** — read DB schema, call API endpoint, inspect file
|
|
52
|
+
- **Record evidence in the build manifest** under `### Evidence Gathered` — cite file path, line range, and what was learned
|
|
53
|
+
- **Flag any contract mismatch as a STOP condition** — same protocol as a plan deviation
|
|
54
|
+
- **If inaccessible**, document as an unverified assumption in your self-assessment
|
|
55
|
+
|
|
56
|
+
### 3. Red-Green-Refactor (per criterion)
|
|
57
|
+
|
|
58
|
+
For EACH success criterion, output in your response (not in thinking) — **SCAN** (mandatory):
|
|
59
|
+
> SCAN: Working on criterion [N]: [criterion text]. Approach: [approach from pre-registration].
|
|
60
|
+
|
|
61
|
+
**RED** — Write one failing test per criterion. Real code, no mocks unless unavoidable. Run it. Watch it fail. Confirm it fails for the right reason.
|
|
62
|
+
**GREEN** — Write minimal code to pass. Run tests after each change.
|
|
63
|
+
**REFACTOR** — After green only: remove duplication, improve names. Keep tests green. Don't add behavior.
|
|
64
|
+
**DEVIATION** — If the pre-registered approach is wrong: **STOP.** Report to the orchestrator. Minor adjustments: document in your self-assessment.
|
|
65
|
+
|
|
66
|
+
### 4. Produce Verification Artifacts
|
|
67
|
+
|
|
68
|
+
Save the build manifest to `$RND_DIR/builds/T<id>-manifest.md`. Write in full narrative prose — describe what was built, what decisions were made, and why. Manifest depth scales with task Criticality.
|
|
69
|
+
|
|
70
|
+
**For `Criticality: LOW` tasks (config changes, doc edits, renaming, log lines, style fixes) — skinny manifest:**
|
|
71
|
+
|
|
72
|
+
```markdown
|
|
73
|
+
# Build Manifest: T<id>
|
|
74
|
+
|
|
75
|
+
## Files Created/Modified
|
|
76
|
+
- [list with paths]
|
|
77
|
+
|
|
78
|
+
## Files written
|
|
79
|
+
path/to/file-a.ext
|
|
80
|
+
path/to/file-b.ext
|
|
81
|
+
|
|
82
|
+
## Tests Written
|
|
83
|
+
- [test name]: Tests [criterion text]
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
Omit Evidence Gathered, Edge Cases Covered, and External References entirely — LOW-criticality tasks don't interact with external systems by definition, and edge-case sections on trivial changes are boilerplate, not signal.
|
|
87
|
+
|
|
88
|
+
**For `Criticality: NORMAL` and `Criticality: HIGH` tasks — full manifest:**
|
|
89
|
+
|
|
90
|
+
```markdown
|
|
91
|
+
# Build Manifest: T<id>
|
|
92
|
+
|
|
93
|
+
## Files Created/Modified
|
|
94
|
+
- [list with paths]
|
|
95
|
+
|
|
96
|
+
## Files written
|
|
97
|
+
path/to/file-a.ext
|
|
98
|
+
path/to/file-b.ext
|
|
99
|
+
|
|
100
|
+
## Evidence Gathered
|
|
101
|
+
- `path/to/file.ext:NN-MM` — [what was learned]
|
|
102
|
+
|
|
103
|
+
## Tests Written
|
|
104
|
+
- [test name]: Tests [criterion text]
|
|
105
|
+
|
|
106
|
+
## Edge Cases Covered
|
|
107
|
+
- [list edge cases and how they're handled]
|
|
108
|
+
|
|
109
|
+
## External References
|
|
110
|
+
- `[reference value]` — type: [URL | email | address | API endpoint | package name | …] — provenance: [verified from user input | from existing codebase file X:line Y | generated from training data]
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
The `## Files written` section is **required** in every manifest. List one file path per line (relative to the repo root), no bullets or backticks. This section is parsed by the surgical-revert helper; omitting or misspelling it makes reverting impossible. Sections with genuinely no content may be written as `(none)` on a single line. Do NOT omit section headers entirely from the full-manifest form — Verifier scans for presence.
|
|
114
|
+
|
|
115
|
+
**Do not game the skinny form.** If a task is tagged LOW but you discover during implementation that it touches an external system (API call, schema assumption, env var contract), upgrade to the full manifest and flag the misclassification to the orchestrator.
|
|
116
|
+
|
|
117
|
+
### 5. Write Honest Self-Assessment
|
|
118
|
+
|
|
119
|
+
Save to `$RND_DIR/builds/T<id>-self-assessment.md` (Verifier will NOT see this — be honest). The format depends on your build status.
|
|
120
|
+
|
|
121
|
+
**For plain `DONE` status (no concerns, all criteria met with HIGH confidence, no deviations, no unverified assumptions):** write a minimal one-line file. No sections, no boilerplate.
|
|
122
|
+
|
|
123
|
+
```markdown
|
|
124
|
+
# Self-Assessment: T<id>
|
|
125
|
+
|
|
126
|
+
All criteria met with HIGH confidence. No deviations. No unverified assumptions.
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
**For `DONE_WITH_CONCERNS`, `NEEDS_CONTEXT`, or `BLOCKED`:** write the full template. Any genuine uncertainty, any confidence below HIGH on any criterion, any unverified external assumption, or any deviation from plan means you are NOT plain `DONE` — use the full template even if it feels borderline.
|
|
130
|
+
|
|
131
|
+
```markdown
|
|
132
|
+
# Self-Assessment: T<id>
|
|
133
|
+
|
|
134
|
+
## Confidence per criterion
|
|
135
|
+
- [criterion 1]: HIGH / MEDIUM / LOW — [brief reason]
|
|
136
|
+
|
|
137
|
+
## Assumptions made
|
|
138
|
+
|
|
139
|
+
### Verified external assumptions
|
|
140
|
+
- [system]: [what was verified] — evidence: [where]
|
|
141
|
+
|
|
142
|
+
### Unverified external assumptions
|
|
143
|
+
- [system]: [what was assumed] — reason unverified: [why]
|
|
144
|
+
|
|
145
|
+
## Uncertainties & risks
|
|
146
|
+
- [what you're not sure about]
|
|
147
|
+
|
|
148
|
+
## Deviations from plan
|
|
149
|
+
- [any changes from pre-registered approach, with reasons]
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
**Do not game the minimal form.** If you are tempted to write the one-liner to avoid effort but you have an unverified external assumption or a MEDIUM-confidence criterion, that is dishonesty — downgrade the status to `DONE_WITH_CONCERNS` and use the full template.
|
|
153
|
+
|
|
154
|
+
## Found Issues Ledger
|
|
155
|
+
|
|
156
|
+
For every issue encountered during a build — error, warning, broken test, bug, or gap — you MUST either fix it or record it. Silently skipping or rationalizing away an issue is not allowed.
|
|
157
|
+
|
|
158
|
+
**Path:** `$RND_DIR/builds/T<id>-found-issues.jsonl`
|
|
159
|
+
|
|
160
|
+
**JSON-line schema:**
|
|
161
|
+
```json
|
|
162
|
+
{"issue": "<description>", "location": "<path:line>", "decision": "fixed"|"escalated", "reason": "<why fixed or why escalated>"}
|
|
163
|
+
```
|
|
164
|
+
|
|
165
|
+
**Rules:**
|
|
166
|
+
- Append one line per issue, even if you fix it immediately.
|
|
167
|
+
- `decision="escalated"` is the only legal path for issues you cannot fix within this task scope.
|
|
168
|
+
- Never omit an issue from the ledger because it seems minor or pre-existing.
|
|
169
|
+
|
|
170
|
+
**Example:**
|
|
171
|
+
```json
|
|
172
|
+
{"issue": "TypeScript strict null check failure on optional field access", "location": "src/parser.ts:47", "decision": "fixed", "reason": "added null guard before property access"}
|
|
173
|
+
```
|
|
174
|
+
|
|
175
|
+
## Good Tests
|
|
176
|
+
|
|
177
|
+
| Quality | Good | Bad |
|
|
178
|
+
|---------|------|-----|
|
|
179
|
+
| **Minimal** | One thing. "and" in name? Split it. | `test('validates and processes')` |
|
|
180
|
+
| **Clear** | Name describes behavior | `test('test1')` |
|
|
181
|
+
| **Real** | Tests actual code | Tests mock behavior |
|
|
182
|
+
| **Intent** | Shows what SHOULD happen | Shows what DOES happen |
|
|
183
|
+
|
|
184
|
+
Prefer property-based tests for invariants, roundtrips, ordering guarantees. Use fast-check, hypothesis, or propcheck if available. For specific-output tests: exact API shapes, known edge cases, error messages, UI rendering. Summarize verbose output: test/build output >50 lines → extract pass/fail counts + error messages. Large files: use offset/limit or Grep.
|
|
185
|
+
|
|
186
|
+
## Common Rationalizations
|
|
187
|
+
|
|
188
|
+
| Excuse | Reality |
|
|
189
|
+
|--------|---------|
|
|
190
|
+
| "Too simple to test" | Simple code breaks. Test takes 30 seconds. |
|
|
191
|
+
| "I'll test after" | Tests passing immediately prove nothing. |
|
|
192
|
+
| "TDD will slow me down" | TDD is faster than debugging. |
|
|
193
|
+
| "The approach is wrong but I'll adapt" | STOP. Report deviation. Don't silently diverge. |
|
|
194
|
+
| "I'll verify it myself" | That's the Verifier's job. Produce artifacts, not verdicts. |
|
|
195
|
+
|
|
196
|
+
## Verification Checklist
|
|
197
|
+
|
|
198
|
+
- [ ] Every success criterion has a corresponding test
|
|
199
|
+
- [ ] Watched each test fail before implementing
|
|
200
|
+
- [ ] All tests pass
|
|
201
|
+
- [ ] Tried to break your own implementation with at least one edge case per criterion
|
|
202
|
+
- [ ] Build manifest lists all files and tests
|
|
203
|
+
- [ ] Self-assessment is honest about uncertainties
|
|
204
|
+
- [ ] No silent deviations from pre-registered approach
|
|
205
|
+
- [ ] Output is clean (no errors, warnings)
|
|
206
|
+
- [ ] Every external dependency verified against the actual system, evidence in manifest
|
|
207
|
+
- [ ] Build manifest `### Evidence Gathered` contains file:line citations for each external contract
|
|
208
|
+
- [ ] All external references (URLs, APIs, packages, addresses) declared in the manifest's `## External References` section with type and provenance
|
|
209
|
+
- [ ] No pipeline-internal context in project code — comments, docstrings, test names, or variable names. Covers task/wave IDs (`T1`, `T01`, `wave-3`), planner phase or disposition labels (`Q4 disposition`, "compatibility audit"), and session artifact paths or meta-references (`research/*.md`, "the R&D session", "the pipeline"). Carve-out applies to RND artifact files only.
|
|
210
|
+
|
|
211
|
+
## Convergent Iteration
|
|
212
|
+
|
|
213
|
+
When receiving NEEDS ITERATION, address **every** failed criterion in a single pass:
|
|
214
|
+
1. **Inventory all failures.** List every criterion marked FAIL or NEEDS ITERATION. Nothing ships until every item is addressed.
|
|
215
|
+
2. **Diagnose root causes.** Multiple failures often share a root cause.
|
|
216
|
+
3. **Check shared code paths.** Re-verify that fixes do not regress passing criteria.
|
|
217
|
+
4. **Re-run ALL tests.** Run the complete test suite — not just tests related to flagged criteria.
|
|
218
|
+
5. **Update the build manifest and self-assessment** to reflect all changes made in this pass.
|
|
219
|
+
|
|
220
|
+
## Status Codes
|
|
221
|
+
|
|
222
|
+
| Code | When to Use |
|
|
223
|
+
|------|-------------|
|
|
224
|
+
| `DONE` | All criteria met, all tests pass, no significant concerns. |
|
|
225
|
+
| `DONE_WITH_CONCERNS` | Criteria met, tests pass, but uncertainty exists. Include `concerns:` line summarizing what Verifier should scrutinize. |
|
|
226
|
+
| `NEEDS_CONTEXT` | Cannot proceed without additional information — ambiguous requirement, missing dependency, conflicting specs. |
|
|
227
|
+
| `BLOCKED` | Hard blocker prevents implementation. Requires orchestrator intervention. |
|
|
228
|
+
|
|
229
|
+
**Completion message format:**
|
|
230
|
+
```
|
|
231
|
+
T<id> build complete — status: DONE — manifest at $RND_DIR/builds/T<id>-manifest.md
|
|
232
|
+
T<id> build complete — status: DONE_WITH_CONCERNS: [brief summary] — manifest at $RND_DIR/builds/T<id>-manifest.md
|
|
233
|
+
```
|
|
234
|
+
|
|
235
|
+
## Evidence Pack (when RND_EVIDENCE_PACK=1)
|
|
236
|
+
|
|
237
|
+
When the orchestrator sets `RND_EVIDENCE_PACK=1`, wrap tool invocations through `run-tool.sh` to capture a reproducible evidence artifact for each test run. This is opt-in — normal builds do not require it.
|
|
238
|
+
|
|
239
|
+
**Invocation syntax:**
|
|
240
|
+
|
|
241
|
+
```bash
|
|
242
|
+
RND_EVIDENCE_PACK=1 RND_DIR="$RND_DIR" RND_TASK_ID="<id>" \
|
|
243
|
+
run-tool.sh [--task-id <id>] -- <command> [args...]
|
|
244
|
+
```
|
|
245
|
+
|
|
246
|
+
**Artifact location:** `$RND_DIR/evidence/T<id>/` — contains `stdout.txt`, `stderr.txt`, and `manifest.json` (tool name, argv, exit code, timestamps, and sha256 hashes of all tracked input files).
|
|
247
|
+
|
|
248
|
+
**Validation:** Before the Verifier reads the pack, `evidence-pack-gate.sh` schema-validates `manifest.json`. Packs that fail validation are rejected; the Verifier receives an error rather than stale or malformed data.
|
|
249
|
+
|
|
250
|
+
**When to use:** Only when instructed by the pre-registration or the orchestrator. Do not wrap every command — only the test runner or tool that produces the primary verification artifact.
|
|
251
|
+
|
|
252
|
+
## Related Skills
|
|
253
|
+
|
|
254
|
+
- `rnd-framework:rnd-debugging` — unexpected test failures
|
|
255
|
+
- `rnd-framework:rnd-iteration` — when Verifier sends back feedback
|
|
256
|
+
- `rnd-framework:rnd-data-science` — numerical analysis, financial calculations, data wiring, chart generation (Julia or DuckDB)
|
|
@@ -0,0 +1,188 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: rnd-decomposition
|
|
3
|
+
description: "Use when breaking a complex task into sub-tasks with pre-registration documents — structured hierarchical decomposition, dependency analysis, and testable success criteria"
|
|
4
|
+
user-invocable: false
|
|
5
|
+
effort: medium
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# R&D Decomposition
|
|
9
|
+
|
|
10
|
+
## Overview
|
|
11
|
+
|
|
12
|
+
Decompose tasks into structured sub-task trees. Every sub-task gets a pre-registration document with testable success criteria BEFORE any code is written.
|
|
13
|
+
|
|
14
|
+
**Core principle:** If you can't write testable success criteria, the task isn't understood well enough to build.
|
|
15
|
+
|
|
16
|
+
## When to Use
|
|
17
|
+
|
|
18
|
+
- Planning phase of `/rnd-framework:rnd-start` or `/rnd-framework:rnd-plan`
|
|
19
|
+
- Any non-trivial feature, refactor, or task with multiple moving parts or unclear success criteria
|
|
20
|
+
|
|
21
|
+
## The Iron Law
|
|
22
|
+
|
|
23
|
+
```
|
|
24
|
+
NO CODING WITHOUT PRE-REGISTRATION
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
## Hierarchical Decomposition
|
|
28
|
+
|
|
29
|
+
```
|
|
30
|
+
System Level: [Feature] <-> [System Validation]
|
|
31
|
+
Module Level: [Components] <-> [Integration Tests]
|
|
32
|
+
Unit Level: [Functions] <-> [Unit Tests]
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
1. Start at System level — what does the feature DO end-to-end?
|
|
36
|
+
2. Identify Modules — what components are needed?
|
|
37
|
+
3. Break into Units — what functions/utilities does each module need?
|
|
38
|
+
4. Pair each level with verification — system validation, integration tests, unit tests
|
|
39
|
+
|
|
40
|
+
A criterion is testable if a skeptical Verifier can evaluate it from evidence alone: **observable outcome** (return value, state change, error thrown), **concrete conditions** (specific inputs, thresholds, error types), **binary result** (met or not met).
|
|
41
|
+
|
|
42
|
+
### Decomposition Heuristics
|
|
43
|
+
|
|
44
|
+
- **Too big:** >5 success criteria → split
|
|
45
|
+
- **Too small:** single function with one criterion → merge up
|
|
46
|
+
- **Too vague:** "works correctly", "handles errors" → rewrite with observable outcomes
|
|
47
|
+
- **Uncertain:** approach unclear → add Phase 0 spike task
|
|
48
|
+
- **Unverified external contract:** DB schema, API shape, env var not confirmed → add Phase 0 spike or dedicated verification step before that task
|
|
49
|
+
- **Local expert available:** set the `Local expert` field in the pre-registration so the Builder knows to invoke it
|
|
50
|
+
|
|
51
|
+
## Decomposition Caps
|
|
52
|
+
|
|
53
|
+
Hard limits that apply after heuristics:
|
|
54
|
+
|
|
55
|
+
- **Maximum 4 tasks per wave** — if a wave would contain more, split into sub-waves or coalesce tasks.
|
|
56
|
+
- **Minimum task scope: 1 hour of work** — tasks smaller than this must coalesce with a sibling. A task that touches one line or one config key is below the minimum scope.
|
|
57
|
+
- **Coalescing rule:** when two tasks share the same file set and could be reviewed in a single pass, merge them unless their success criteria require different verification levels.
|
|
58
|
+
|
|
59
|
+
## Exploration Cache
|
|
60
|
+
|
|
61
|
+
Before decomposition, write structured findings to `$RND_DIR/exploration/` (`mkdir -p`). One kebab-case file per area (e.g., `hooks-architecture.md`). Each file: `## Files Examined`, `## Key Patterns`, `## Relevant Dependencies`, `## Notes for Builders`.
|
|
62
|
+
|
|
63
|
+
## Pre-Registration Document
|
|
64
|
+
|
|
65
|
+
```
|
|
66
|
+
Task ID: T<number>
|
|
67
|
+
Intent: [One sentence — what this accomplishes and why]
|
|
68
|
+
Approach: [Brief planned implementation strategy]
|
|
69
|
+
Expected outputs: [List of files/functions/artifacts to produce]
|
|
70
|
+
Criticality: LOW | NORMAL | HIGH
|
|
71
|
+
Success criteria:
|
|
72
|
+
Correctness:
|
|
73
|
+
- [ ] [Functional requirement, test passing, or contract conformance condition]
|
|
74
|
+
- [ ] [Another must-pass condition]
|
|
75
|
+
Quality:
|
|
76
|
+
- [ ] [Code quality, naming, patterns, or documentation condition]
|
|
77
|
+
Verification level: unit | integration | system
|
|
78
|
+
Dependencies: [Task IDs this depends on]
|
|
79
|
+
Preconditions:
|
|
80
|
+
- [File/content assertion verified before build starts]
|
|
81
|
+
- [Another assertion — if any fails, task is BLOCKED]
|
|
82
|
+
Local expert: [optional — name of project-local agent/skill to invoke, e.g., security-reviewer]
|
|
83
|
+
External Dependencies:
|
|
84
|
+
- system: [DB | API | file | env | service]
|
|
85
|
+
contract: [What is assumed about this system — schema, response shape, format, presence]
|
|
86
|
+
verification: [How this will be confirmed — e.g., Read actual schema, query endpoint, inspect file sample]
|
|
87
|
+
fulfills: [VAL-AREA-NNN, ...]
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
The `fulfills` field creates bidirectional traceability between tasks and Validation Contract assertions.
|
|
91
|
+
|
|
92
|
+
`Preconditions` declares file/content assertions the Builder verifies before writing code — if any fail, task is immediately BLOCKED. Use concrete, tool-checkable assertions (Glob for file existence, Grep for function presence, Read for dependency key). Omit if the task creates new files from scratch.
|
|
93
|
+
|
|
94
|
+
### Tiered Criteria
|
|
95
|
+
|
|
96
|
+
| Correctness (must-pass) | Quality (should-pass) |
|
|
97
|
+
|---|---|
|
|
98
|
+
| "Returns 401 for expired tokens" | "Function names follow project naming convention" |
|
|
99
|
+
| "Throws ValidationError when input is null" | "Inline comments explain the retry logic" |
|
|
100
|
+
| "File exists at the declared output path" | "No magic numbers — constants are named" |
|
|
101
|
+
|
|
102
|
+
**Decision rule:** "Does a user or downstream system observe this outcome?" Yes → Correctness. Maintainability/DX only → Quality. Unmet Correctness → FAIL. Unmet Quality → NEEDS ITERATION.
|
|
103
|
+
|
|
104
|
+
## Environment Discovery
|
|
105
|
+
|
|
106
|
+
Check if `project-facts.md` exists in `$RND_DIR`. If fresh (commit hash matches HEAD), use it directly. Otherwise run the checklist below and confirm findings with the user via `ctx.ui.notify`.
|
|
107
|
+
|
|
108
|
+
| Area | What to scan | How |
|
|
109
|
+
|------|-------------|-----|
|
|
110
|
+
| Package manager | package.json, Cargo.toml, mix.exs, go.mod, pyproject.toml | Glob for config files |
|
|
111
|
+
| Test framework | vitest, jest, pytest, ExUnit, go test configs | Grep for test runner in configs/scripts |
|
|
112
|
+
| CI config | .github/workflows/, .gitlab-ci.yml, Jenkinsfile | Glob for CI files, Read to extract commands |
|
|
113
|
+
| External service URLs | https:// references in source code | Grep for URLs in src/ |
|
|
114
|
+
| Environment variables | .env.example, .env.template, CI secrets config | Read env templates, Grep for process.env/ENV/os.environ |
|
|
115
|
+
| Secrets and off-limits | .gitignore patterns, CI secret names, sensitive file paths | Read .gitignore, infer from CI config |
|
|
116
|
+
|
|
117
|
+
Findings feed into **Environment Setup**, **Infrastructure**, and **Testing Strategy** sections of plan.md.
|
|
118
|
+
|
|
119
|
+
## Dependency Analysis
|
|
120
|
+
|
|
121
|
+
Build a dependency matrix. Assign tasks with zero dependencies to Wave 1, tasks depending only on Wave 1 to Wave 2, and so on. Flag parallel opportunities within each wave.
|
|
122
|
+
|
|
123
|
+
## Output
|
|
124
|
+
|
|
125
|
+
Save to `$RND_DIR/plan.md` with these sections:
|
|
126
|
+
|
|
127
|
+
- **Task Tree** — hierarchical list with task IDs
|
|
128
|
+
- **Environment Setup** — runtime, package manager, dependencies, install commands
|
|
129
|
+
- **Infrastructure** — external services (URL + auth), off-limits items
|
|
130
|
+
- **Testing Strategy** — test framework, baseline count, exact run commands for unit/integration/live tests, user testing instructions
|
|
131
|
+
- **Worker Guidelines** — `USE`/`OFF-LIMITS` boundaries; coding conventions from CLAUDE.md/linters; architectural patterns; design decisions
|
|
132
|
+
- **Validation Contract** — numbered VAL-AREA-NNN assertions (see format below)
|
|
133
|
+
- **Pre-Registration Documents** — one per task
|
|
134
|
+
- **Dependency Matrix** — task dependency table
|
|
135
|
+
- **Execution Schedule** — wave assignments with parallel opportunities
|
|
136
|
+
- **Iteration Budgets** — per-task budgets based on criticality
|
|
137
|
+
|
|
138
|
+
### Validation Contract Format
|
|
139
|
+
|
|
140
|
+
```markdown
|
|
141
|
+
### Area: [Functional Domain]
|
|
142
|
+
|
|
143
|
+
#### VAL-AREA-NNN: [Assertion title]
|
|
144
|
+
[One-sentence description of what must be true]
|
|
145
|
+
Tool: [shell | grep | glob | read | code review]
|
|
146
|
+
Evidence: [Exact command + expected output pattern]
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
ID format: `VAL-` + area abbreviation (2-6 uppercase chars) + `-` + 3-digit number (e.g., `VAL-AUTH-001`). Evidence must be concrete: not "tests pass" but `npx vitest run exits 0, reports >= 50 passed`. Cross-cutting assertions go under `### Area: Cross-Area`. Every assertion must be fulfilled by at least one task; every task should fulfill at least one assertion.
|
|
150
|
+
|
|
151
|
+
## Verification Checklist
|
|
152
|
+
|
|
153
|
+
- [ ] Every task has a complete pre-registration document
|
|
154
|
+
- [ ] Every success criterion is testable and tagged Correctness or Quality
|
|
155
|
+
- [ ] No circular dependencies; waves correctly ordered; parallel opportunities identified
|
|
156
|
+
- [ ] Tasks >5 criteria have been split; uncertain approaches have Phase 0 spikes
|
|
157
|
+
- [ ] Every task touching an external system has an `External Dependencies` field with system type, assumed contract, and verification method
|
|
158
|
+
- [ ] Environment Setup, Infrastructure, and Testing Strategy sections populated
|
|
159
|
+
- [ ] Worker Guidelines contains boundaries, conventions, and architecture notes
|
|
160
|
+
- [ ] Validation Contract has VAL-AREA-NNN assertions with Tool and Evidence for every Correctness criterion
|
|
161
|
+
- [ ] Every task has a `fulfills` field; every VAL assertion is fulfilled by at least one task
|
|
162
|
+
|
|
163
|
+
## Plan Self-Review
|
|
164
|
+
|
|
165
|
+
After writing all sections of plan.md, reread it with fresh eyes. This is a checklist you run yourself before notifying the orchestrator — not a subagent dispatch. The Verifier cannot save you from plan-level mistakes; they cascade through every downstream phase.
|
|
166
|
+
|
|
167
|
+
Run these six checks against the finished plan.md. If any fails, fix inline and re-check.
|
|
168
|
+
|
|
169
|
+
1. **Spec coverage.** For each explicit user requirement or discovery-context constraint, point to the task(s) covering it. Gaps → add a task or explicitly note it as out-of-scope in Worker Guidelines.
|
|
170
|
+
|
|
171
|
+
2. **Placeholder scan.** Grep the plan for `TODO`, `TBD`, `???`, `XXX`, `[...]`, `handle appropriately`, `works correctly`, `as needed`. Any hit → replace with concrete content or remove.
|
|
172
|
+
|
|
173
|
+
3. **VAL traceability.** Every `VAL-AREA-NNN` is named in at least one task's `fulfills` field, and every task has a non-empty `fulfills`. A VAL with no fulfiller, or a task fulfilling nothing, means the Validation Contract and Pre-Registration drifted apart — fix whichever side is wrong.
|
|
174
|
+
|
|
175
|
+
4. **Identifier consistency.** For each function name, file path, type name, or env var that appears in multiple tasks, confirm the spelling matches across mentions. A function named `clearLayers` in T3 and `clearFullLayers` in T7 is the most common cascading plan error.
|
|
176
|
+
|
|
177
|
+
5. **External-dependency completeness.** Any task whose Intent, Approach, or Expected outputs references a DB, API, file, env var, or external service MUST have a populated `External Dependencies` block with `system`, `contract`, and `verification`. Missing block → add it (this is also what gates the Reality Auditor).
|
|
178
|
+
|
|
179
|
+
6. **Verifier test on each Correctness criterion.** Reread each Correctness criterion as if you have no context. If you can't translate it into "run X, expect Y" in under 10 seconds, rewrite it with an observable outcome, concrete condition, and binary result.
|
|
180
|
+
|
|
181
|
+
If the plan has >10 tasks or any HIGH criticality task, consider one additional escalation: spawn a fresh reviewer with the spec and plan.md, asking them to flag only concrete problems from this same checklist. Optional; not required.
|
|
182
|
+
|
|
183
|
+
## Related Skills
|
|
184
|
+
|
|
185
|
+
- `rnd-framework:rnd-scheduling` — For detailed wave scheduling
|
|
186
|
+
- `rnd-framework:rnd-scaling` — For choosing pipeline scale
|
|
187
|
+
- `rnd-framework:rnd-orchestration` — For pipeline overview
|
|
188
|
+
- `rnd-framework:rnd-data-science` — When a task involves analytical or numerical work
|
|
@@ -0,0 +1,197 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: rnd-experiments
|
|
3
|
+
description: "Use when independently verifying built work — defines how verifiers write experiment tests from the spec alone before reading Builder code, preventing false PASSes through mandatory independent validation"
|
|
4
|
+
user-invocable: false
|
|
5
|
+
allowed-tools: [Read, Write, Bash, Grep, Glob]
|
|
6
|
+
effort: medium
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# R&D Experiments
|
|
10
|
+
|
|
11
|
+
## Overview
|
|
12
|
+
|
|
13
|
+
Experiments are independent tests written by the Verifier from the pre-registration spec alone — before reading the Builder's code or tests. They shift verification from "does this look right?" to "does this actually work?" by producing evidence that cannot be anchored to the Builder's implementation choices.
|
|
14
|
+
|
|
15
|
+
**Core principle:** Derive tests from the spec, not from what the Builder built. If your test logic mirrors the Builder's test logic, you haven't independently verified anything — you've just confirmed the Builder was internally consistent.
|
|
16
|
+
|
|
17
|
+
## When to Use
|
|
18
|
+
|
|
19
|
+
- Step 2 of the verification process (before reading Builder code)
|
|
20
|
+
- Any time a success criterion requires observable behavior that can be executed
|
|
21
|
+
- Mandatory for every criterion in the pre-registration — experiments are not on-demand
|
|
22
|
+
|
|
23
|
+
## The Iron Law
|
|
24
|
+
|
|
25
|
+
```
|
|
26
|
+
EXPERIMENTS ARE MANDATORY FOR EVERY CORRECTNESS CRITERION — NOT OPTIONAL, NOT ON-DEMAND
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
If the pre-registration lists N Correctness criteria, the Verifier writes N experiments. Skipping an experiment for a Correctness criterion — because it "looks simple" or "is obviously met" — is a verification failure. The point is to produce independent evidence, not to confirm what already seems true.
|
|
30
|
+
|
|
31
|
+
Quality criteria are optional for experiments — writing experiments for them is encouraged but not required when time is limited. Correctness criteria always get experiments.
|
|
32
|
+
|
|
33
|
+
## What Makes Experiments Different from Builder Tests
|
|
34
|
+
|
|
35
|
+
| Dimension | Builder Tests | Verifier Experiments |
|
|
36
|
+
|-----------|--------------|---------------------|
|
|
37
|
+
| Source of truth | Builder's understanding of the spec | Pre-registration spec text only |
|
|
38
|
+
| When written | Before or during implementation | Before reading Builder's code |
|
|
39
|
+
| What they prove | Implementation is internally consistent | Implementation meets the declared spec |
|
|
40
|
+
| Who writes them | Builder agent | Verifier agent |
|
|
41
|
+
| Allowed to read | Anything | Only the pre-registration and external systems |
|
|
42
|
+
|
|
43
|
+
The distinction matters because a Builder can write tests that pass while implementing the wrong thing. If the Verifier derives tests from the same source as the Builder, the Verifier will reproduce the Builder's blind spots.
|
|
44
|
+
|
|
45
|
+
**The information barrier:** When writing experiments, the Verifier MUST NOT have read the Builder's test files. Reading them first anchors your experiment logic to the Builder's framing. Write experiments first, then run Builder tests in Step 4.
|
|
46
|
+
|
|
47
|
+
## Output Directory
|
|
48
|
+
|
|
49
|
+
Save all experiment files to:
|
|
50
|
+
|
|
51
|
+
```
|
|
52
|
+
$RND_DIR/verifications/T<id>-experiments/
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
Example: `$RND_DIR/verifications/T3-experiments/`
|
|
56
|
+
|
|
57
|
+
**Multi-judge mode:** When operating as one of two parallel judges, the orchestrator passes your judge identity (`judge-a`, `judge-b`, or `tiebreaker`) in the prompt. Use a subdirectory to avoid path collisions:
|
|
58
|
+
|
|
59
|
+
```
|
|
60
|
+
$RND_DIR/verifications/T<id>-experiments/<judge-id>/
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
Example: `$RND_DIR/verifications/T3-experiments/judge-a/`
|
|
64
|
+
|
|
65
|
+
If no judge identity is provided (single-verifier mode), use the flat path.
|
|
66
|
+
|
|
67
|
+
Create this directory before writing any experiment files:
|
|
68
|
+
|
|
69
|
+
```bash
|
|
70
|
+
mkdir -p "$RND_DIR/verifications/T<id>-experiments"
|
|
71
|
+
# or in multi-judge mode:
|
|
72
|
+
mkdir -p "$RND_DIR/verifications/T<id>-experiments/<judge-id>"
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
## Naming Convention
|
|
76
|
+
|
|
77
|
+
Name experiment files after the criterion they test, using kebab-case:
|
|
78
|
+
|
|
79
|
+
```
|
|
80
|
+
exp-<criterion-slug>.test.<ext>
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
Examples:
|
|
84
|
+
- `exp-file-exists-at-declared-path.test.ts`
|
|
85
|
+
- `exp-frontmatter-contains-name-field.test.ts`
|
|
86
|
+
- `exp-output-directory-is-created.test.ts`
|
|
87
|
+
- `exp-returns-401-for-expired-token.test.ts`
|
|
88
|
+
|
|
89
|
+
Use the same file extension and test runner as the project's existing test suite. One experiment file per criterion. Do not bundle multiple criteria into one file — if an experiment fails, you need to know exactly which criterion it covers.
|
|
90
|
+
|
|
91
|
+
## What Makes a Good Experiment
|
|
92
|
+
|
|
93
|
+
A good experiment:
|
|
94
|
+
|
|
95
|
+
1. **Is derived from the spec text alone.** Read the criterion. Write a test that checks the criterion's stated observable outcome. Do not look at the implementation first.
|
|
96
|
+
2. **Has exactly one assertion.** One experiment, one criterion. If you find yourself writing `expect A` and `expect B` in the same test, split it.
|
|
97
|
+
3. **Uses real execution, not inspection.** Run the code. Do not read the source and reason about what it probably does. Observable behavior is evidence; source analysis is interpretation.
|
|
98
|
+
4. **Tests the boundary, not the happy path only.** If the criterion says "returns 401 for expired tokens," also test that valid tokens are not rejected. Boundary cases reveal implementation shortcuts.
|
|
99
|
+
5. **Is independent of Builder test infrastructure.** Do not import helpers, fixtures, or test utilities written by the Builder. Write your own setup — or use the project's production code directly.
|
|
100
|
+
|
|
101
|
+
## Experiment Template
|
|
102
|
+
|
|
103
|
+
```typescript
|
|
104
|
+
// exp-<criterion-slug>.test.ts
|
|
105
|
+
// Criterion: <exact criterion text from pre-registration>
|
|
106
|
+
// Spec source: T<id> pre-registration, <Correctness|Quality> tier
|
|
107
|
+
|
|
108
|
+
import { describe, test, expect } from "bun:test";
|
|
109
|
+
// Import only production code, not Builder test helpers
|
|
110
|
+
|
|
111
|
+
describe("T<id>: <criterion text>", () => {
|
|
112
|
+
test("<observable outcome — what should happen>", async () => {
|
|
113
|
+
// Arrange: set up the minimum conditions from the spec
|
|
114
|
+
// Act: invoke the behavior the criterion describes
|
|
115
|
+
// Assert: check the observable outcome stated in the criterion
|
|
116
|
+
expect(result).toBe(expectedValue);
|
|
117
|
+
});
|
|
118
|
+
|
|
119
|
+
// If the criterion implies a boundary or negative case, add it here
|
|
120
|
+
test("<boundary case — what should NOT happen or edge input>", async () => {
|
|
121
|
+
// ...
|
|
122
|
+
});
|
|
123
|
+
});
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
Adapt the test runner imports to the project's conventions (Bun, Jest, Vitest, etc.).
|
|
127
|
+
|
|
128
|
+
## Process: Writing Experiments
|
|
129
|
+
|
|
130
|
+
For each criterion in the pre-registration:
|
|
131
|
+
|
|
132
|
+
1. **Read only the criterion text.** Do not look at Builder files yet.
|
|
133
|
+
2. **Identify the observable outcome.** What can be measured or asserted from outside the implementation?
|
|
134
|
+
3. **Identify the minimal setup.** What inputs, files, or state does the criterion require to be exercised?
|
|
135
|
+
4. **Write the experiment** following the template above.
|
|
136
|
+
5. **Identify one boundary or negative case** — what input or condition should produce a different outcome?
|
|
137
|
+
6. **Save to `$RND_DIR/verifications/T<id>-experiments/`** with the naming convention above.
|
|
138
|
+
|
|
139
|
+
After writing all experiments (one per criterion), proceed to Step 3 of the verification process: run experiments against the Builder's code.
|
|
140
|
+
|
|
141
|
+
## Recording Experiment Results
|
|
142
|
+
|
|
143
|
+
When running experiments in Step 3, record the output verbatim in the verification report. Do not paraphrase. The raw output is the evidence.
|
|
144
|
+
|
|
145
|
+
If an experiment fails, this is a Correctness-tier finding — the implementation did not satisfy the spec as the experiment interpreted it. If the experiment itself was wrong (e.g., misread the criterion), fix the experiment and note the correction, but do not delete it. The correction is part of the evidence trail.
|
|
146
|
+
|
|
147
|
+
## Reality Experiments
|
|
148
|
+
|
|
149
|
+
When the Builder's code touches external services — SQL schemas, HTTP APIs, MCP tools, environment variables — write at least one reality experiment per external contract in addition to functional criterion experiments.
|
|
150
|
+
|
|
151
|
+
Reality experiments are adversarial: frame them as "if this assumption is wrong, this command will show X." Design experiments to disprove the assumption, not confirm it. If the assumption survives, that is your evidence.
|
|
152
|
+
|
|
153
|
+
### When to write reality experiments
|
|
154
|
+
|
|
155
|
+
- Builder code references a specific table column, API response field, or env var
|
|
156
|
+
- Builder self-assessment lists unverified external assumptions
|
|
157
|
+
- The pre-registration says the Builder verified something against a live system — independently verify it
|
|
158
|
+
|
|
159
|
+
### Example: SQL Schema Verification
|
|
160
|
+
|
|
161
|
+
```
|
|
162
|
+
Builder code assumes: SELECT email, name FROM users
|
|
163
|
+
Adversarial experiment: Run PRAGMA table_info(users) and assert output includes rows for email and name
|
|
164
|
+
If the output does NOT include those columns → the Builder hallucinated the schema
|
|
165
|
+
|
|
166
|
+
Command: sqlite3 ./dev.db "PRAGMA table_info(users);"
|
|
167
|
+
Disproving condition: email or name row is absent, or the table does not exist
|
|
168
|
+
```
|
|
169
|
+
|
|
170
|
+
### Example: HTTP API Shape
|
|
171
|
+
|
|
172
|
+
```
|
|
173
|
+
Builder code assumes: GET /api/v2/users returns { data: User[], total: number }
|
|
174
|
+
Adversarial experiment: curl -s https://api.example.com/api/v2/users | head -c 500
|
|
175
|
+
Disproving condition: Response missing "data" or "total" keys, or is not valid JSON
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
### Naming convention
|
|
179
|
+
|
|
180
|
+
Name reality experiment files with the `exp-reality-` prefix:
|
|
181
|
+
|
|
182
|
+
```
|
|
183
|
+
exp-reality-users-schema.test.ts
|
|
184
|
+
exp-reality-api-users-shape.test.ts
|
|
185
|
+
exp-reality-env-var-database-url.test.ts
|
|
186
|
+
```
|
|
187
|
+
|
|
188
|
+
Save to the same `$RND_DIR/verifications/T<id>-experiments/` directory as functional experiments.
|
|
189
|
+
|
|
190
|
+
The full adversarial methodology — hypothesis structure, verdict rules (VALID/INVALID/UNCHECKED), and reality reports — is defined in `rnd-framework:rnd-reality-auditing`. This section tells you to include reality experiments in your experiment suite; that skill tells you how to structure them.
|
|
191
|
+
|
|
192
|
+
## Related Skills
|
|
193
|
+
|
|
194
|
+
- `rnd-framework:rnd-verification` — Full 6-step verification process; experiments are Step 2 and Step 3
|
|
195
|
+
- `rnd-framework:rnd-failure-modes` — Failure modes that experiments are designed to catch (Premature Satisfaction, Trusting Agent Reports)
|
|
196
|
+
- `rnd-framework:rnd-iteration` — What happens when experiment failures trigger NEEDS ITERATION
|
|
197
|
+
- `rnd-framework:rnd-reality-auditing` — Full adversarial methodology for external service contracts; reality experiments follow its structure
|