@rune-kit/rune 2.2.3 → 2.2.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/compiler/adapters/openclaw.js +63 -0
- package/compiler/emitter.js +10 -0
- package/docs/EXTENSION-TEMPLATE.md +18 -0
- package/docs/SKILL-TEMPLATE.md +31 -0
- package/docs/guides/index.html +84 -21
- package/docs/index.html +227 -30
- package/docs/script.js +206 -15
- package/docs/style.css +355 -59
- package/extensions/saas/PACK.md +13 -8
- package/extensions/saas/skills/billing-integration.md +82 -3
- package/package.json +1 -1
- package/skills/audit/SKILL.md +1 -0
- package/skills/ba/SKILL.md +54 -1
- package/skills/completion-gate/SKILL.md +67 -2
- package/skills/cook/SKILL.md +170 -2
- package/skills/debug/SKILL.md +48 -1
- package/skills/deploy/SKILL.md +46 -1
- package/skills/fix/SKILL.md +28 -1
- package/skills/git/SKILL.md +55 -1
- package/skills/hallucination-guard/SKILL.md +21 -6
- package/skills/journal/SKILL.md +51 -3
- package/skills/plan/SKILL.md +152 -4
- package/skills/preflight/SKILL.md +50 -1
- package/skills/research/SKILL.md +36 -8
- package/skills/retro/SKILL.md +314 -0
- package/skills/review/SKILL.md +1 -0
- package/skills/sentinel/SKILL.md +108 -3
- package/skills/sentinel-env/SKILL.md +31 -1
- package/skills/skill-forge/SKILL.md +85 -3
- package/skills/team/SKILL.md +61 -15
- package/skills/test/SKILL.md +105 -9
- package/skills/verification/SKILL.md +29 -1
|
@@ -3,7 +3,7 @@ name: skill-forge
|
|
|
3
3
|
description: Use when creating new Rune skills, editing existing skills, or verifying skill quality before deployment. Applies TDD discipline to skill authoring — test before write, verify before ship.
|
|
4
4
|
metadata:
|
|
5
5
|
author: runedev
|
|
6
|
-
version: "1.
|
|
6
|
+
version: "1.4.0"
|
|
7
7
|
layer: L2
|
|
8
8
|
model: opus
|
|
9
9
|
group: creation
|
|
@@ -97,11 +97,13 @@ Follow `docs/SKILL-TEMPLATE.md` format. Required sections:
|
|
|
97
97
|
| Frontmatter | YES | Name, description, metadata |
|
|
98
98
|
| Purpose | YES | One paragraph, ecosystem role |
|
|
99
99
|
| Triggers | YES | When to invoke |
|
|
100
|
-
| Calls / Called By | YES | Mesh connections |
|
|
100
|
+
| Calls / Called By | YES | Mesh connections (control flow) |
|
|
101
|
+
| Data Flow | YES | Feeds Into / Fed By / Feedback Loops (data flow) |
|
|
101
102
|
| Workflow | YES | Step-by-step execution |
|
|
102
103
|
| Output Format | YES | Structured, parseable output |
|
|
103
104
|
| Constraints | YES | 3-7 MUST/MUST NOT rules |
|
|
104
105
|
| Sharp Edges | YES | Known failure modes |
|
|
106
|
+
| Self-Validation | YES | Domain-specific QA checklist (per-skill, not centralized) |
|
|
105
107
|
| Done When | YES | Verifiable completion criteria |
|
|
106
108
|
| Cost Profile | YES | Token estimate |
|
|
107
109
|
| Mesh Gates | L1/L2 only | Progression guards |
|
|
@@ -277,7 +279,72 @@ Wire the skill into the mesh:
|
|
|
277
279
|
1. **Update `docs/ARCHITECTURE.md`** — add to correct layer/group table
|
|
278
280
|
2. **Update `CLAUDE.md`** — increment skill count, add to layer list
|
|
279
281
|
3. **Add mesh connections** — update SKILL.md of skills that should call/be called by this one
|
|
280
|
-
4. **
|
|
282
|
+
4. **Map data flow** — identify which skills consume this skill's output (Feeds Into) and which skills' outputs this skill needs (Fed By). Look for feedback loops where two skills refine each other's work
|
|
283
|
+
5. **Write Self-Validation** — 3-5 domain-specific checks unique to this skill's output. Ask: "What quality issues can ONLY this skill catch?"
|
|
284
|
+
6. **Verify no conflicts** — new skill's output format compatible with consumers?
|
|
285
|
+
|
|
286
|
+
### Phase 6.5 — EXTENSION AUTHORING (if building an extension, not a skill)
|
|
287
|
+
|
|
288
|
+
Extensions augment existing skills with optional capabilities. Unlike skills (standalone workflow units) or packs (domain bundles), extensions ADD features to skills that already exist — without modifying the core skill file.
|
|
289
|
+
|
|
290
|
+
#### Extension vs Skill vs Pack
|
|
291
|
+
|
|
292
|
+
| Concept | Purpose | Modifies Core? | Self-contained? |
|
|
293
|
+
|---------|---------|----------------|-----------------|
|
|
294
|
+
| **Skill** | Standalone workflow unit (SKILL.md) | N/A — IS core | Yes |
|
|
295
|
+
| **Pack** | Domain bundle of skills (PACK.md) | No — bundles existing | Yes |
|
|
296
|
+
| **Extension** | Augments existing skill with new capability | No — additive only | Yes — own dir with install/uninstall |
|
|
297
|
+
|
|
298
|
+
#### Extension Directory Structure
|
|
299
|
+
|
|
300
|
+
```
|
|
301
|
+
extensions/<extension-name>/
|
|
302
|
+
├── EXTENSION.md # Manifest: what it extends, how, dependencies
|
|
303
|
+
├── install.sh # Unix installer (non-destructive MCP merge)
|
|
304
|
+
├── install.ps1 # Windows installer
|
|
305
|
+
├── uninstall.sh # Clean removal
|
|
306
|
+
├── uninstall.ps1 # Clean removal (Windows)
|
|
307
|
+
├── skills/
|
|
308
|
+
│ └── <skill-name>/
|
|
309
|
+
│ └── SKILL.md # New skill added by extension
|
|
310
|
+
├── agents/ # Optional subagent definitions
|
|
311
|
+
│ └── <agent-name>.md
|
|
312
|
+
├── references/ # Domain knowledge loaded by extension skills
|
|
313
|
+
│ └── <topic>.md
|
|
314
|
+
├── scripts/ # Executable utilities
|
|
315
|
+
│ └── <script>.py|.sh
|
|
316
|
+
└── docs/
|
|
317
|
+
└── SETUP.md # Extension-specific configuration guide
|
|
318
|
+
```
|
|
319
|
+
|
|
320
|
+
#### EXTENSION.md Manifest
|
|
321
|
+
|
|
322
|
+
```yaml
|
|
323
|
+
---
|
|
324
|
+
name: "<extension-name>"
|
|
325
|
+
extends: "<target-skill-or-pack>"
|
|
326
|
+
description: "What capability this extension adds"
|
|
327
|
+
requires:
|
|
328
|
+
- mcp: "<mcp-server-name>" # Optional: MCP server dependency
|
|
329
|
+
- skill: "<required-skill-name>" # Required core skill
|
|
330
|
+
install_method: "non-destructive" # MUST be non-destructive
|
|
331
|
+
---
|
|
332
|
+
```
|
|
333
|
+
|
|
334
|
+
#### Extension Rules
|
|
335
|
+
|
|
336
|
+
1. **Non-destructive install** — extension MUST NOT modify existing skill files. It adds new files alongside.
|
|
337
|
+
2. **Self-contained** — removing the extension directory restores the system to its pre-install state.
|
|
338
|
+
3. **MCP merge** — if the extension adds MCP tools, install script MUST merge into settings.json without overwriting existing entries.
|
|
339
|
+
4. **Fallback graceful** — if the MCP server or external dependency is unavailable, the extension skill MUST degrade gracefully (report unavailability, don't crash).
|
|
340
|
+
5. **Cost awareness** — if the extension calls paid APIs, the extension skill MUST warn before expensive operations and track usage.
|
|
341
|
+
6. **Pre-flight check** — extension skill Step 1 MUST verify dependencies are available before executing.
|
|
342
|
+
|
|
343
|
+
#### When to Build an Extension (vs a Skill or Pack)
|
|
344
|
+
|
|
345
|
+
- Build an **extension** when: the capability requires an external API/MCP, is optional, and augments an existing skill
|
|
346
|
+
- Build a **skill** when: the capability is self-contained and fits a layer in the mesh
|
|
347
|
+
- Build a **pack** when: you're bundling multiple related skills for a domain
|
|
281
348
|
|
|
282
349
|
### Phase 7 — SHIP
|
|
283
350
|
|
|
@@ -302,6 +369,8 @@ git commit -m "feat: add [skill-name] — [one-line purpose]"
|
|
|
302
369
|
- [ ] At least one observed failure documented
|
|
303
370
|
- [ ] Anti-rationalization table from real test failures
|
|
304
371
|
- [ ] Mesh connections bidirectional (calls AND called-by both updated)
|
|
372
|
+
- [ ] Data flow mapped (Feeds Into / Fed By / Feedback Loops)
|
|
373
|
+
- [ ] Self-Validation has 3-5 domain-specific checks (not generic)
|
|
305
374
|
- [ ] Output format is structured and parseable by other skills
|
|
306
375
|
|
|
307
376
|
**Architecture:**
|
|
@@ -311,6 +380,14 @@ git commit -m "feat: add [skill-name] — [one-line purpose]"
|
|
|
311
380
|
- [ ] ARCHITECTURE.md updated
|
|
312
381
|
- [ ] CLAUDE.md updated
|
|
313
382
|
|
|
383
|
+
**Extension-specific (if building an extension):**
|
|
384
|
+
- [ ] EXTENSION.md manifest present with extends, requires, install_method
|
|
385
|
+
- [ ] install.sh + install.ps1 tested (non-destructive merge)
|
|
386
|
+
- [ ] uninstall.sh + uninstall.ps1 tested (clean removal)
|
|
387
|
+
- [ ] Extension skill has dependency pre-flight check (Step 1)
|
|
388
|
+
- [ ] Fallback behavior documented when external dependency unavailable
|
|
389
|
+
- [ ] Cost warning present if extension calls paid APIs
|
|
390
|
+
|
|
314
391
|
## Adapting Existing Skills
|
|
315
392
|
|
|
316
393
|
When editing, not creating:
|
|
@@ -368,6 +445,8 @@ Techniques:
|
|
|
368
445
|
### Mesh Impact
|
|
369
446
|
- New connections: [count] ([list of skills])
|
|
370
447
|
- Bidirectional check: PASS | FAIL
|
|
448
|
+
- Data flow mapped: [count] feeds-into, [count] fed-by, [count] feedback loops
|
|
449
|
+
- Self-Validation: [count] domain-specific checks written
|
|
371
450
|
```
|
|
372
451
|
|
|
373
452
|
## Constraints
|
|
@@ -393,6 +472,9 @@ Techniques:
|
|
|
393
472
|
| Code blocks in SKILL.md bloat every invocation | HIGH | WHY vs HOW split: SKILL.md ≤10-line code blocks, extract rest to references/ |
|
|
394
473
|
| Writing skill without TDD (no observed failures first) | CRITICAL | Skill TDD: RED (run scenario WITHOUT skill → document failures) → GREEN (write skill targeting failures) → REFACTOR (find bypasses → add blocks) |
|
|
395
474
|
| Description leaks workflow → agent skips full content | HIGH | CSO Discipline: description = triggers only. Test: can you execute from description alone? If yes, it leaks too much |
|
|
475
|
+
| Self-Validation copies completion-gate checks | HIGH | Self-Validation is DOMAIN-specific: "assertions per test", "dependency ordering". NOT generic: "tests pass", "build succeeds" — those belong to completion-gate |
|
|
476
|
+
| Data Flow confused with Calls | MEDIUM | Calls = runtime invocation (skill A calls skill B). Feeds Into = artifact persistence (skill A writes .rune/X.md, skill B reads it later). If it's a direct function call → Calls. If it's via files/context → Data Flow |
|
|
477
|
+
| Feedback Loop missing one direction | MEDIUM | Every Feedback Loop ↻ must document BOTH directions: what A sends to B AND what B sends back to A. One-way = Feeds Into, not a loop |
|
|
396
478
|
|
|
397
479
|
## Done When
|
|
398
480
|
|
package/skills/team/SKILL.md
CHANGED
|
@@ -5,7 +5,7 @@ context: fork
|
|
|
5
5
|
agent: general-purpose
|
|
6
6
|
metadata:
|
|
7
7
|
author: runedev
|
|
8
|
-
version: "0.
|
|
8
|
+
version: "0.5.0"
|
|
9
9
|
layer: L1
|
|
10
10
|
model: opus
|
|
11
11
|
group: orchestrator
|
|
@@ -152,7 +152,11 @@ Mark todo[1] `in_progress`.
|
|
|
152
152
|
|
|
153
153
|
**2a. Launch parallel streams.**
|
|
154
154
|
|
|
155
|
-
Launch independent streams (depends_on: []) in parallel using Task tool with worktree isolation
|
|
155
|
+
Launch independent streams (depends_on: []) in parallel using Task tool with worktree isolation.
|
|
156
|
+
|
|
157
|
+
> From agency-agents (msitarzewski/agency-agents, 50.8k★): "Structured handoff docs prevent the #1 multi-agent failure: context loss between agents."
|
|
158
|
+
|
|
159
|
+
Each stream receives a **NEXUS Handoff Template** — not a bare prompt:
|
|
156
160
|
|
|
157
161
|
```
|
|
158
162
|
For each stream where depends_on == []:
|
|
@@ -160,22 +164,54 @@ For each stream where depends_on == []:
|
|
|
160
164
|
subagent_type: "general-purpose",
|
|
161
165
|
model: "sonnet",
|
|
162
166
|
isolation: "worktree",
|
|
163
|
-
prompt:
|
|
167
|
+
prompt: <NEXUS Handoff below>
|
|
164
168
|
)
|
|
165
169
|
```
|
|
166
170
|
|
|
171
|
+
**NEXUS Handoff Template** (sent to each cook instance):
|
|
172
|
+
|
|
173
|
+
```markdown
|
|
174
|
+
## NEXUS Handoff: Stream [id]
|
|
175
|
+
|
|
176
|
+
### Metadata
|
|
177
|
+
- Stream: [id] of [total]
|
|
178
|
+
- Depends on: [none | stream ids]
|
|
179
|
+
- File ownership: [list — ONLY these files may be modified]
|
|
180
|
+
- Model: sonnet
|
|
181
|
+
|
|
182
|
+
### Context
|
|
183
|
+
- Project: [project name and type]
|
|
184
|
+
- Overall goal: [1-line feature description]
|
|
185
|
+
- This stream's goal: [specific sub-task]
|
|
186
|
+
- Conventions: [key patterns from scout — naming, file structure, test framework]
|
|
187
|
+
|
|
188
|
+
### Deliverable
|
|
189
|
+
- [ ] [specific outcome 1 — e.g., "AuthService with login/register/reset methods"]
|
|
190
|
+
- [ ] [specific outcome 2 — e.g., "Unit tests covering happy path + 3 error cases"]
|
|
191
|
+
- [ ] [specific outcome 3 — e.g., "Types exported for Phase 2 consumers"]
|
|
192
|
+
|
|
193
|
+
### Quality Expectations
|
|
194
|
+
- Tests: must pass with evidence (stdout captured)
|
|
195
|
+
- Types: no `any`, strict mode
|
|
196
|
+
- Security: no hardcoded secrets, parameterized queries
|
|
197
|
+
- Conventions: [project-specific — from scout output]
|
|
198
|
+
|
|
199
|
+
### Evidence Required
|
|
200
|
+
Return a Cook Report with:
|
|
201
|
+
- Exact files modified (git diff --stat)
|
|
202
|
+
- Test output (stdout — not just "tests pass")
|
|
203
|
+
- Any CONCERNS discovered during implementation
|
|
204
|
+
```
|
|
205
|
+
|
|
167
206
|
**2b. Launch dependent streams sequentially.**
|
|
168
207
|
|
|
169
208
|
```
|
|
170
209
|
For each stream where depends_on != []:
|
|
171
210
|
WAIT for all depends_on streams to complete.
|
|
172
|
-
Then launch:
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
isolation: "worktree",
|
|
177
|
-
prompt: "Cook task: [stream.task]. Files in scope: [stream.files]. Patterns from stream [depends_on] are available in worktree. Return cook report."
|
|
178
|
-
)
|
|
211
|
+
Then launch with NEXUS Handoff that includes:
|
|
212
|
+
- Completed stream's deliverables as "Available Context"
|
|
213
|
+
- Exported interfaces/types from prior streams in "Code Contracts" section
|
|
214
|
+
- Any CONCERNS from prior streams in "Known Issues" section
|
|
179
215
|
```
|
|
180
216
|
|
|
181
217
|
**2b.5. Pre-merge scope verification.**
|
|
@@ -367,16 +403,24 @@ Mark todo[4] `completed`.
|
|
|
367
403
|
- **Duration**: [time across streams]
|
|
368
404
|
|
|
369
405
|
### Streams
|
|
370
|
-
| Stream | Task | Status |
|
|
371
|
-
|
|
372
|
-
| A | [task] |
|
|
373
|
-
| B | [task] |
|
|
374
|
-
| C | [task] |
|
|
406
|
+
| Stream | Task | Status | Deliverables | Concerns |
|
|
407
|
+
|--------|------|--------|-------------|----------|
|
|
408
|
+
| A | [task] | DONE | 3/3 delivered | None |
|
|
409
|
+
| B | [task] | DONE_WITH_CONCERNS | 2/2 delivered | Perf regression on large input |
|
|
410
|
+
| C | [task] | DONE | 2/2 delivered | None |
|
|
411
|
+
|
|
412
|
+
### Acceptance Criteria
|
|
413
|
+
| # | Criterion | Stream | Evidence | Verdict |
|
|
414
|
+
|---|-----------|--------|----------|---------|
|
|
415
|
+
| 1 | Auth endpoints return JWT | A | Test stdout: "3 passed" | PASS |
|
|
416
|
+
| 2 | No SQL injection | A | Sentinel: PASS | PASS |
|
|
417
|
+
| 3 | Dashboard loads < 2s | B | No perf test run | UNVERIFIED |
|
|
375
418
|
|
|
376
419
|
### Integration
|
|
377
420
|
- Merge conflicts: [count]
|
|
378
421
|
- Integration tests: [passed]/[total]
|
|
379
422
|
- Coverage: [%]
|
|
423
|
+
- Unresolved concerns: [count — from DONE_WITH_CONCERNS streams]
|
|
380
424
|
```
|
|
381
425
|
|
|
382
426
|
---
|
|
@@ -405,6 +449,8 @@ Known failure modes for this skill. Check these before declaring done.
|
|
|
405
449
|
| Agent modified files outside declared scope | HIGH | Pre-merge scope verification in Phase 2b.5 — flag before merge, not after |
|
|
406
450
|
| Merge failure with no rollback path | HIGH | pre-team-merge tag created before merges — git reset --hard on failure |
|
|
407
451
|
| Poisoned cook report merged blindly | HIGH | Phase 3a.5 integrity-check on all cook reports before merge |
|
|
452
|
+
| Bare prompt to cook instance — no context, conventions, or scope boundary | HIGH | NEXUS Handoff Template: structured handoff with metadata, deliverables, quality expectations, and evidence requirements |
|
|
453
|
+
| Cook returns "done" with no acceptance criteria tracking | MEDIUM | Team Report includes Acceptance Criteria table with per-criterion evidence and PASS/FAIL/UNVERIFIED verdict |
|
|
408
454
|
|
|
409
455
|
## Done When
|
|
410
456
|
|
package/skills/test/SKILL.md
CHANGED
|
@@ -3,7 +3,7 @@ name: test
|
|
|
3
3
|
description: "TDD test writer. Writes failing tests FIRST (red), then verifies they pass after implementation (green). Covers unit, integration, and e2e tests."
|
|
4
4
|
metadata:
|
|
5
5
|
author: runedev
|
|
6
|
-
version: "0.
|
|
6
|
+
version: "0.7.0"
|
|
7
7
|
layer: L2
|
|
8
8
|
model: sonnet
|
|
9
9
|
group: development
|
|
@@ -168,15 +168,79 @@ Input: git diff main --name-only
|
|
|
168
168
|
Output: Prioritized test plan targeting only affected paths
|
|
169
169
|
```
|
|
170
170
|
|
|
171
|
-
## Test Types
|
|
171
|
+
## Test Types — 4-Layer Methodology
|
|
172
172
|
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
|
176
|
-
|
|
177
|
-
|
|
|
178
|
-
|
|
|
179
|
-
|
|
|
173
|
+
Tests are organized in 4 layers. Each layer catches a different failure class. Higher layers are slower but catch integration issues lower layers miss.
|
|
174
|
+
|
|
175
|
+
| Layer | Type | What It Catches | Framework | Speed |
|
|
176
|
+
|-------|------|-----------------|-----------|-------|
|
|
177
|
+
| L1 | **Unit** | Logic bugs, boundary violations, pure function errors | jest/vitest/pytest/cargo test | Fast |
|
|
178
|
+
| L2 | **Integration** | API contract breaks, DB query errors, service interaction failures | supertest/httpx/reqwest | Medium |
|
|
179
|
+
| L3 | **True Backend** | Real tool/service output correctness (not just exit 0) | Same + real software invocation | Medium-Slow |
|
|
180
|
+
| L4 | **E2E / Subprocess** | Full workflow from user/agent perspective, installed app works | Playwright/Cypress/subprocess | Slow |
|
|
181
|
+
|
|
182
|
+
**Layer rules:**
|
|
183
|
+
- **L1 (Unit)**: Synthetic data, no external deps. Every function tested in isolation. Fast, deterministic, CI-friendly
|
|
184
|
+
- **L2 (Integration)**: Tests service boundaries — API endpoints, DB operations, message queues. May need test DB or mock server
|
|
185
|
+
- **L3 (True Backend)**: **Invokes the REAL tool/service** and verifies output programmatically. No graceful degradation — if the dependency isn't installed, tests FAIL (not skip). Verify: magic bytes, file size > 0, content structure. Print artifact paths for manual inspection
|
|
186
|
+
- **L4 (E2E/Subprocess)**: Tests the installed command/app via subprocess or browser automation. Full user workflow: input → process → output → verify
|
|
187
|
+
|
|
188
|
+
**"No graceful degradation" rule** (L3/L4): Hard dependencies MUST be installed. Tests MUST NOT skip or produce fake results when the dependency is missing. A silently skipping test is worse than a loudly failing test.
|
|
189
|
+
|
|
190
|
+
Additional modes:
|
|
191
|
+
|
|
192
|
+
| Type | When | Speed |
|
|
193
|
+
|------|------|-------|
|
|
194
|
+
| Regression | After bug fixes | Fast |
|
|
195
|
+
| Diff-aware | After implementation, large codebases (Phase 6.5) | Fast (targeted) |
|
|
196
|
+
|
|
197
|
+
## TEST.md — Test Plan + Results Document
|
|
198
|
+
|
|
199
|
+
For non-trivial features (3+ test files or 20+ test cases), create a `TEST.md` in the test directory. This is BOTH a planning doc (written BEFORE tests) and results doc (appended AFTER tests pass).
|
|
200
|
+
|
|
201
|
+
### Before writing tests — write the plan:
|
|
202
|
+
```markdown
|
|
203
|
+
# Test Plan: [Feature Name]
|
|
204
|
+
|
|
205
|
+
## Test Inventory
|
|
206
|
+
- `test_core.py`: ~XX unit tests planned (L1)
|
|
207
|
+
- `test_integration.py`: ~XX integration tests planned (L2)
|
|
208
|
+
- `test_e2e.py`: ~XX E2E tests planned (L3/L4)
|
|
209
|
+
|
|
210
|
+
## Unit Test Plan (L1)
|
|
211
|
+
| Module | Functions | Edge Cases | Est. Tests | Req IDs |
|
|
212
|
+
|--------|-----------|------------|------------|---------|
|
|
213
|
+
| `core/auth.py` | login, register, refresh | expired token, invalid creds, rate limit | 12 | REQ-001, REQ-003 |
|
|
214
|
+
|
|
215
|
+
## E2E Scenarios (L3/L4)
|
|
216
|
+
| Workflow | Simulates | Operations | Verified | Req IDs |
|
|
217
|
+
|----------|-----------|------------|----------|---------|
|
|
218
|
+
| User signup | New user onboarding | register → verify → login | Token valid, profile created | REQ-005 |
|
|
219
|
+
|
|
220
|
+
## Realistic Workflow Scenarios
|
|
221
|
+
- **[Name]**: [Step 1] → [Step 2] → verify [output properties]
|
|
222
|
+
```
|
|
223
|
+
|
|
224
|
+
### After tests pass — append results:
|
|
225
|
+
```markdown
|
|
226
|
+
## Test Results
|
|
227
|
+
[Paste full `pytest -v --tb=no` or `npm test` output]
|
|
228
|
+
|
|
229
|
+
## Summary
|
|
230
|
+
- Total: XX | Passed: XX | Failed: 0
|
|
231
|
+
- Execution time: X.Xs | Coverage: XX%
|
|
232
|
+
|
|
233
|
+
## Requirement Coverage
|
|
234
|
+
| Req ID | Test File(s) | Status |
|
|
235
|
+
|--------|-------------|--------|
|
|
236
|
+
| REQ-001 | `test_auth.py::test_login` | ✅ Covered |
|
|
237
|
+
| REQ-002 | — | ❌ Not covered |
|
|
238
|
+
|
|
239
|
+
## Gaps
|
|
240
|
+
- [Areas not covered and why]
|
|
241
|
+
```
|
|
242
|
+
|
|
243
|
+
**Why TEST.md**: Planning tests before code catches missing edge cases early. Appending results creates permanent evidence. One document = complete testing story.
|
|
180
244
|
|
|
181
245
|
## Error Recovery
|
|
182
246
|
|
|
@@ -203,6 +267,25 @@ Output: Prioritized test plan targeting only affected paths
|
|
|
203
267
|
- `browser-pilot` (L3): Phase 4 — e2e and visual testing for UI flows
|
|
204
268
|
- `debug` (L2): Phase 5 — when existing test regresses unexpectedly
|
|
205
269
|
|
|
270
|
+
## Data Flow
|
|
271
|
+
|
|
272
|
+
### Feeds Into →
|
|
273
|
+
|
|
274
|
+
- `cook` (L1): test results (pass/fail/coverage) → cook's Phase 5 quality gate evidence
|
|
275
|
+
- `completion-gate` (L3): test runner stdout → evidence for "tests pass" claims
|
|
276
|
+
- `fix` (L2): failing test output → fix's target (what to make green)
|
|
277
|
+
|
|
278
|
+
### Fed By ←
|
|
279
|
+
|
|
280
|
+
- `plan` (L2): phase file test tasks → test's RED phase targets (what to test)
|
|
281
|
+
- `review` (L2): untested edge cases found during review → new test targets
|
|
282
|
+
- `fix` (L2): implemented code → test's GREEN phase verification target
|
|
283
|
+
|
|
284
|
+
### Feedback Loops ↻
|
|
285
|
+
|
|
286
|
+
- `test` ↔ `fix`: test writes failing tests (RED) → fix implements to pass → test verifies (GREEN) → if new failures emerge, loop continues
|
|
287
|
+
- `test` ↔ `debug`: test discovers regression → debug diagnoses root cause → test writes regression test to prevent recurrence
|
|
288
|
+
|
|
206
289
|
## Anti-Rationalization Table
|
|
207
290
|
|
|
208
291
|
| Excuse | Reality |
|
|
@@ -317,6 +400,18 @@ Known failure modes for this skill. Check these before declaring done.
|
|
|
317
400
|
| Modifying source files to make tests work | HIGH | Role boundary: test writes test files ONLY — source changes go to rune:fix |
|
|
318
401
|
| Test-only methods leaking into production code | MEDIUM | Anti-Pattern 2 gate: if method only called by tests → move to test utilities |
|
|
319
402
|
|
|
403
|
+
## Self-Validation
|
|
404
|
+
|
|
405
|
+
```
|
|
406
|
+
SELF-VALIDATION (run before emitting Test Report):
|
|
407
|
+
- [ ] Every test file has at least one assertion — no empty test bodies
|
|
408
|
+
- [ ] RED phase output shows actual failures (not "0 tests") — tests were real, not stubs
|
|
409
|
+
- [ ] No test modifies source code — test files only, source changes belong to fix
|
|
410
|
+
- [ ] Test names describe behavior, not implementation ("should reject expired token" not "test function X")
|
|
411
|
+
- [ ] No mocks of the thing being tested — only mock external dependencies
|
|
412
|
+
- [ ] If BA requirements exist (REQ-xxx), every requirement has at least one test — check plan's Traceability Matrix
|
|
413
|
+
```
|
|
414
|
+
|
|
320
415
|
## Done When
|
|
321
416
|
|
|
322
417
|
- Test framework detected from project config files
|
|
@@ -325,6 +420,7 @@ Known failure modes for this skill. Check these before declaring done.
|
|
|
325
420
|
- After implementation: all tests PASS (GREEN phase — actual pass output shown)
|
|
326
421
|
- Coverage ≥80% verified via verification
|
|
327
422
|
- Test Report emitted with framework, test count, RED/GREEN status, and coverage
|
|
423
|
+
- Self-Validation: all checks passed
|
|
328
424
|
|
|
329
425
|
## Cost Profile
|
|
330
426
|
|
|
@@ -3,7 +3,7 @@ name: verification
|
|
|
3
3
|
description: "Universal verification runner. Runs lint, type-check, tests, and build. Use after any code change to verify nothing is broken."
|
|
4
4
|
metadata:
|
|
5
5
|
author: runedev
|
|
6
|
-
version: "0.
|
|
6
|
+
version: "0.5.0"
|
|
7
7
|
layer: L3
|
|
8
8
|
model: haiku
|
|
9
9
|
group: validation
|
|
@@ -233,6 +233,33 @@ Overall: [PASS/FAIL]
|
|
|
233
233
|
- Unwired: [files that failed Level 3 with 0 consumers]
|
|
234
234
|
```
|
|
235
235
|
|
|
236
|
+
## Output Completion Enforcement
|
|
237
|
+
|
|
238
|
+
> From taste-skill (Leonxlnx/taste-skill, 3.4k★): Truncated code is worse than no code — it passes reviews but breaks at runtime.
|
|
239
|
+
|
|
240
|
+
When verifying code files (Level 2 SUBSTANTIVE check), also scan for **truncation patterns** — signs that the agent generated partial output and stopped:
|
|
241
|
+
|
|
242
|
+
| Banned Pattern | Language | What It Means |
|
|
243
|
+
|---|---|---|
|
|
244
|
+
| `// ...` or `/* ... */` as a statement | JS/TS | Agent truncated remaining code |
|
|
245
|
+
| `# ...` as a statement (not comment) | Python | Agent truncated |
|
|
246
|
+
| `// rest of code` / `// remaining implementation` | Any | Explicit truncation admission |
|
|
247
|
+
| `// TODO: implement` as sole function body | Any | Placeholder, not implementation |
|
|
248
|
+
| `{ /* same as above */ }` | JS/TS | Copy-paste truncation |
|
|
249
|
+
| `...` (bare ellipsis, not spread operator) | JS/TS/Python | Truncation marker |
|
|
250
|
+
| `[PAUSED]` / `[CONTINUED]` in source | Any | Agent session marker leaked into code |
|
|
251
|
+
|
|
252
|
+
**Action on detection:**
|
|
253
|
+
- Mark file as TRUNCATED (distinct from STUB) in Verification Report
|
|
254
|
+
- TRUNCATED files are Level 2 FAIL — they CANNOT pass verification
|
|
255
|
+
- Report the specific line number and pattern detected
|
|
256
|
+
- If agent claims "done" with truncated files → REJECTED by Evidence-Before-Claims gate
|
|
257
|
+
|
|
258
|
+
**Continuation protocol** — if the agent hit output limits mid-file:
|
|
259
|
+
- Agent MUST log: `[PAUSED — X of Y functions complete]` in its response (NOT in the code file)
|
|
260
|
+
- Agent MUST resume and complete the file in the next turn
|
|
261
|
+
- Verification re-runs after completion to clear the TRUNCATED flag
|
|
262
|
+
|
|
236
263
|
## Evidence-Before-Claims Gate
|
|
237
264
|
|
|
238
265
|
<HARD-GATE>
|
|
@@ -284,6 +311,7 @@ Known failure modes for this skill. Check these before declaring done.
|
|
|
284
311
|
| Trusting exit code 0 without output verification | CRITICAL | Artifact Verification HARD-GATE: always confirm success indicator in stdout (pass count, "0 errors", output file exists) |
|
|
285
312
|
| Existence Theater — file exists but is a stub | HIGH | 3-Level check: Level 2 scans for stub patterns (`<div>Placeholder</div>`, `return null`, `NotImplementedError`) |
|
|
286
313
|
| Dead code — file created but never imported/used | MEDIUM | 3-Level check: Level 3 greps for consumers. 0 importers = UNWIRED |
|
|
314
|
+
| Truncated code — agent hit output limit mid-file | HIGH | Output Completion Enforcement: scan for `// ...`, `// rest of code`, bare ellipsis patterns. TRUNCATED = Level 2 FAIL |
|
|
287
315
|
|
|
288
316
|
## Done When
|
|
289
317
|
|