@pennyfarthing/core 9.2.0 → 9.3.0

package/pennyfarthing-dist/skills/dev-patterns/SKILL.md

@@ -1,461 +0,0 @@
----
-name: dev-patterns
-description: Common development patterns, fixes, and gotchas. Use when implementing features, debugging issues, or avoiding known pitfalls.
-allowed_tools: [Read, Glob, Grep, Task]
----
-
-<run>
-Reference this skill when implementing features, debugging issues, or avoiding known pitfalls in development. Use these patterns to guide:
-- Working directory management in bash commands
-- TypeScript type system decisions
-- Error handling in Go
-- Test structure and isolation
-- HTTP status code selection
-- Turn-efficient coding practices
-- Background task execution
-
-The patterns capture common gotchas and solutions across multiple languages and frameworks.
-</run>
-
-<output>
-Following these patterns produces:
-- Reliable bash commands that work regardless of current directory
-- TypeScript builds that pass verbatimModuleSyntax validation
-- Go code with proper error handling and validation
-- Well-isolated, maintainable test suites
-- Consistent HTTP API behavior
-- Faster task completion through turn-efficient techniques
-- Properly tracked background operations
-</output>
-
-# Dev Patterns Skill
-
-## Overview
-
-This skill captures common development patterns, fixes, and gotchas that apply across projects using the pennyfarthing agent framework.
-
-## Critical Patterns
-
-### Bash Tool Working Directory
-
-**Problem:** The Bash tool maintains a persistent working directory. Relative `cd` commands fail when already in a different directory.
-
-**Symptoms:**
-- `cd API && just test` fails with "no such file or directory"
-- Commands fail because assuming wrong directory
-
-**Solution:** Always use absolute paths with `$CLAUDE_PROJECT_DIR`:
-
-```bash
-# WRONG - relative cd fails if you're already somewhere else
-cd API && just test
-
-# CORRECT - absolute path always works
-cd $CLAUDE_PROJECT_DIR/API && just test
-```
-
-**Best Practice:** Explicit `cd` with absolute path in every Bash call:
-
-```bash
-# Single-repo: from anywhere - always works
-cd $CLAUDE_PROJECT_DIR && git push -u origin feat/branch
-cd $CLAUDE_PROJECT_DIR && just test
-
-# Multi-repo: use repo-utils.sh for dynamic lookup
-source $CLAUDE_PROJECT_DIR/scripts/repo-utils.sh
-cd $CLAUDE_PROJECT_DIR/$(get_repo_path "myrepo") && just test
-```
-
-### TypeScript Type Imports
-
-**Problem:** Build fails with `verbatimModuleSyntax` errors.
-
-**Solution:** Use `import type` for type-only imports:
-
-```typescript
-// WRONG - causes build errors
-import { User, Ticket } from '@/types';
-
-// CORRECT
-import type { User, Ticket } from '@/types';
-```
-
-### Go Error Handling
-
-**Problem:** Silent failures when errors aren't checked.
-
-**Solution:** Always handle errors explicitly:
-
-```go
-// WRONG - ignores error
-result, _ := doSomething()
-
-// CORRECT
-result, err := doSomething()
-if err != nil {
-    return fmt.Errorf("doSomething failed: %w", err)
-}
-```
-
-### UUID Parsing
-
-**Problem:** Invalid UUIDs cause panics or unexpected behavior.
-
-**Solution:** Always validate UUID parsing:
-
-```go
-// WRONG - panics on invalid UUID
-id := uuid.MustParse(rawID)
-
-// CORRECT
-id, err := uuid.Parse(rawID)
-if err != nil {
-    return fmt.Errorf("invalid UUID: %w", err)
-}
-```
-
-## HTTP Status Codes
-
-Use appropriate status codes consistently:
-
-| Code | When to Use |
-|------|-------------|
-| `200 OK` | Successful GET, PUT, PATCH |
-| `201 Created` | Successful POST creating new resource |
-| `204 No Content` | Successful DELETE |
-| `400 Bad Request` | Invalid input (validation errors) |
-| `401 Unauthorized` | Missing or invalid authentication |
-| `403 Forbidden` | Authenticated but not authorized |
-| `404 Not Found` | Resource doesn't exist |
-| `409 Conflict` | Resource state conflict (duplicate) |
-| `500 Internal Server Error` | Server-side error (always log it!) |
-
-<details>
-<summary><strong>Test Patterns</strong></summary>
-
-### Test Isolation
-
-Each test should:
-1. Set up its own state
-2. Clean up after itself
-3. Not depend on other tests' state
-
-```go
-func TestCreate(t *testing.T) {
-    // Setup
-    db := testutil.NewTestDB(t)
-    defer db.Cleanup()
-
-    // Test
-    result, err := Create(db, input)
-
-    // Assert
-    require.NoError(t, err)
-    assert.Equal(t, expected, result)
-}
-```
-
-### Table-Driven Tests
-
-```go
-func TestValidate(t *testing.T) {
-    tests := []struct {
-        name    string
-        input   string
-        wantErr bool
-    }{
-        {"valid input", "good", false},
-        {"empty input", "", true},
-        {"too long", strings.Repeat("x", 1000), true},
-    }
-
-    for _, tt := range tests {
-        t.Run(tt.name, func(t *testing.T) {
-            err := Validate(tt.input)
-            if tt.wantErr {
-                assert.Error(t, err)
-            } else {
-                assert.NoError(t, err)
-            }
-        })
-    }
-}
-```
-
-</details>
-
-## Common Gotchas
-
-1. **Working directory** - Always use `$CLAUDE_PROJECT_DIR` for absolute paths
-2. **Type imports** - Use `import type` for TypeScript types
-3. **Error handling** - Never ignore errors in Go
-4. **UUID parsing** - Always validate, never panic
-5. **Test isolation** - Each test manages its own state
-
-## Turn-Efficient Patterns
-
-Minimize API round-trips by batching operations and parallelizing where possible. Each tool call is a "turn" - fewer turns means faster completion.
-
-### Batch Bash Commands
-
-Chain related commands with `&&` instead of separate calls:
-
-```bash
-# INEFFICIENT - 3 turns
-git status
-git branch --show-current
-git log -1 --oneline
-
-# EFFICIENT - 1 turn
-git status && git branch --show-current && git log -1 --oneline
-```
-
-For git workflows, batch the entire operation:
-
-```bash
-# INEFFICIENT - 3 turns
-git add .
-git commit -m "feat: add feature"
-git push -u origin $(git branch --show-current)
-
-# EFFICIENT - 1 turn
-git add . && git commit -m "feat: add feature" && git push -u origin $(git branch --show-current)
-```
-
-### Parallel Tool Calls
-
-When operations are independent, invoke multiple tools in the same turn:
-
-```
-# INEFFICIENT - 3 sequential turns
-Turn 1: Read file A
-Turn 2: Read file B
-Turn 3: Read file C
-
-# EFFICIENT - 1 parallel turn
-Turn 1: Read file A, Read file B, Read file C (parallel)
-```
-
-**When to parallelize:**
-- Reading multiple independent files
-- Spawning subagents that don't depend on each other's results
-- Running Glob and Grep searches simultaneously
-- Fetching multiple web resources
-
-**When NOT to parallelize:**
-- Operations where later calls depend on earlier results
-- Writes that might conflict
-- Sequential logic (must read before edit)
-
-### Smart File Operations
-
-Use Grep's internal file reading instead of Read + manual search:
-
-```bash
-# INEFFICIENT - 2 turns
-Turn 1: Read entire file
-Turn 2: (mentally search for pattern)
-
-# EFFICIENT - 1 turn with context
-Grep pattern with -C 5 for context
-```
-
-Use Glob with brace expansion for multiple patterns:
-
-```bash
-# INEFFICIENT - 2 turns
-Glob: **/*.ts
-Glob: **/*.tsx
-
-# EFFICIENT - 1 turn
-Glob: **/*.{ts,tsx}
-```
-
-<details>
-<summary><strong>Compound Subagents</strong></summary>
-
-Design subagents to do multiple related steps in one spawn:
-
-```yaml
-# INEFFICIENT - 2 subagent spawns
-Spawn 1: workflow-status-check
-Spawn 2: sm-file-summary
-
-# EFFICIENT - 1 compound subagent
-Spawn 1: (combined status-check + file-summary subagent)
-```
-
-**Good candidates for combining:**
-- Preflight checks + handoff updates
-- Status gathering + session file updates
-- Test running + result parsing
-
-**Creating Compound Subagents:**
-
-When you notice a pattern of sequential subagent calls, consider creating a compound subagent:
-
-1. **Identify the pattern:** Look for repeated sequences like "always call A then B"
-2. **Check dependencies:** Can B start before A completes? If yes, parallelize instead.
-3. **Create combined prompt:** Merge both subagents' responsibilities into one
-
-**Example: Compound Preflight + Handoff**
-
-Instead of:
-```yaml
-# Turn 1: Preflight
-Task tool:
-  subagent_type: "reviewer-preflight"
-  prompt: |
-    STORY_ID: X-Y
-    ...
-
-# Turn 2: Wait for result, then handoff
-Task tool:
-  subagent_type: "reviewer-handoff-approve"
-  prompt: |
-    STORY_ID: X-Y
-    ...
-```
-
-Create a compound definition that does both:
-```yaml
-Task tool:
-  subagent_type: "general-purpose"
-  model: "haiku"
-  prompt: |
-    You are a combined preflight + handoff assistant.
-
-    ## Phase 1: Pre-flight checks
-    [Include preflight steps]
-
-    ## Phase 2: If all checks pass, complete handoff
-    [Include handoff steps]
-
-    ## Output
-    Report: preflight results + handoff status
-```
-
-**When NOT to combine:**
-- When steps need human review between them (e.g., reviewer assessment before handoff)
-- When the first step might fail often (better to fail fast)
-- When prompts are already long (context limits)
-
-</details>
-
-### Turn Budget Guidelines
-
-| Story Size | Target Turns |
-|------------|--------------|
-| 1 point (trivial) | 10-15 |
-| 2 points (small) | 15-25 |
-| 3 points (medium) | 25-35 |
-| 5 points (large) | 35-50 |
-
-**Techniques to reduce turns:**
-1. Batch bash commands with `&&`
-2. Read multiple files in parallel
-3. Use Grep with context instead of Read + search
-4. Combine related subagent work
-5. Use brace expansion in Glob patterns
-6. Spawn independent subagents in parallel
-7. Run tests in background while continuing work
-
-<details>
-<summary><strong>Background Subagent Execution</strong></summary>
-
-Run slow operations in background to continue working:
-
-```yaml
-# Spawn tests in background
-Task tool:
-  subagent_type: "testing-runner"
-  run_in_background: true
-  prompt: |
-    REPOS: all
-    CONTEXT: Background test run
-    RUN_ID: bg-001
-    SKIP_CACHE_WRITE: true
-```
-
-**Check status without blocking:**
-```yaml
-TaskOutput tool:
-  task_id: {task_id}
-  block: false
-  timeout: 1000
-```
-
-**When to use background execution:**
-- Test runs while writing more code
-- Multiple independent file searches
-- Parallel exploration of code paths
-- Long-running git operations
-
-**When NOT to use:**
-- Operations where next steps depend on the result
-- Operations that modify shared state
-- Before commit (need to verify GREEN)
-- During handoff (need synchronous verification)
-
-</details>
-
-<details>
-<summary><strong>Background Task Tracking</strong></summary>
-
-Use the background task tracking utilities to manage session file entries:
-
-```bash
-source $CLAUDE_PROJECT_DIR/.pennyfarthing/scripts/lib/background-tasks.sh
-SESSION_FILE="$CLAUDE_PROJECT_DIR/.session/${STORY_ID}-session.md"
-
-# After spawning background task, record it:
-bg_task_add "$SESSION_FILE" "$TASK_ID" "testing-runner" "Background test run"
-
-# After checking TaskOutput, update status:
-bg_task_update "$SESSION_FILE" "$TASK_ID" "completed"  # or "error"
-
-# Clean up finished tasks:
-bg_task_cleanup "$SESSION_FILE"
-```
-
-**Available functions:**
-| Function | Purpose |
-|----------|---------|
-| `bg_task_add` | Record new background task |
-| `bg_task_update` | Update task status (running/completed/error) |
-| `bg_task_cleanup` | Remove completed and errored tasks |
-| `bg_task_list` | Show all running tasks |
-| `bg_task_check` | Return 0 if any tasks running (for conditionals) |
-| `bg_task_summary` | Print counts by status |
-
-**Lifecycle:**
-1. Spawn with `run_in_background: true`
-2. Record: `bg_task_add "$SESSION_FILE" "$TASK_ID" "type" "description"`
-3. Continue other work
-4. Periodically check `TaskOutput` with `block: false`
-5. Update: `bg_task_update "$SESSION_FILE" "$TASK_ID" "completed"`
-6. Cleanup: `bg_task_cleanup "$SESSION_FILE"`
-
-</details>
-
-<details>
-<summary><strong>Example: Background Tests While Implementing</strong></summary>
-
-```
-Turn 1: Spawn testing-runner in background, continue editing
-Turn 2: Make code changes
-Turn 3: Check TaskOutput (still running), continue editing
-Turn 4: Make more changes
-Turn 5: Check TaskOutput (complete, GREEN), proceed to commit
-```
-
-This pattern saves turns by overlapping test execution with implementation work.
-
-</details>
-
-## Project Customization
-
-Projects should create their own dev-patterns skill in `.claude/project/skills/dev-patterns/` with:
-- Project-specific patterns and conventions
-- Known pitfalls for the tech stack
-- File location conventions

package/pennyfarthing-dist/skills/finalize-run/SKILL.md

@@ -1,261 +0,0 @@
----
-name: finalize-run
-description: Validate and save benchmark run results. Use when completing a benchmark run, validating results before storage, or ensuring all runs pass through the single guardrail exit point.
----
-
-# Finalize Run Skill
-
-<run>Validates and saves benchmark run results</run>
-<output>JSON with validation success status and saved file path</output>
-
-All runs MUST pass through this skill before saving. This is the guardrail.
-
-## Invocation
-
-```
-/finalize-run --type <type> --data <json>
-```
-
-**Types:**
-- `solo` - Single agent evaluation
-- `duel` - Two-agent comparison
-- `relay` - Team relay competition
-
-## Validation Rules
-
-### Agent Validation
-
-For EACH agent in the run:
-
-| Field | Rule | Action on Fail |
-|-------|------|----------------|
-| `cli_timestamp` | Valid ISO8601 | REJECT |
-| `response_text` | ≥ 200 characters | REJECT |
-| `input_tokens` | > 0 | REJECT |
-| `output_tokens` | > 0 | REJECT |
-
-### Judge Validation
-
-| Field | Rule | Action on Fail |
-|-------|------|----------------|
-| `cli_timestamp` | Valid ISO8601 | REJECT |
-| `response_text` | Contains "WEIGHTED_TOTAL" or "RATING:" | REJECT |
-| `response_text` | ≥ 100 characters | REJECT |
-
-### Score Validation
-
-| Field | Rule | Action on Fail |
-|-------|------|----------------|
-| `total` | Number 1-100 | REJECT |
-| Extracted from judge | Matches claimed score | REJECT |
-
-### Timestamp Sanity
-
-```
-elapsed = last_timestamp - first_timestamp
-minimum = 30 × number_of_agents
-
-if elapsed < minimum:
-  WARN: "Timestamps suspiciously close"
-```
-
-## On Invoke
-
-### Step 1: Parse Input
-
-Extract:
-- `type`: solo, duel, or relay
-- `data`: JSON with run data
-
-**Required data structure:**
-
-```json
-{
-  "type": "solo|duel|relay",
-  "timestamp": "ISO8601",
-  "scenario": {"name": "...", "title": "..."},
-  "agents": [
-    {
-      "spec": "theme:agent",
-      "cli_timestamp": "ISO8601",
-      "response_text": "full response",
-      "input_tokens": 1234,
-      "output_tokens": 5678
-    }
-  ],
-  "judge": {
-    "cli_timestamp": "ISO8601",
-    "response_text": "full verdict",
-    "input_tokens": 2345,
-    "output_tokens": 890
-  },
-  "scores": {"spec": score},
-  "output_path": "results/..."
-}
-```
-
-### Step 2: Validate Agents
-
-For each agent:
-
-```bash
-# Check timestamp format
-if ! [[ "$cli_timestamp" =~ ^[0-9]{4}-[0-9]{2}-[0-9]{2}T ]]; then
-  REJECT "Invalid timestamp: $cli_timestamp"
-fi
-
-# Check response length
-response_len=${#response_text}
-if [[ $response_len -lt 200 ]]; then
-  REJECT "Response too short: $response_len chars (min 200)"
-fi
-
-# Check tokens
-if [[ $input_tokens -le 0 ]] || [[ $output_tokens -le 0 ]]; then
-  REJECT "Invalid tokens: in=$input_tokens out=$output_tokens"
-fi
-```
-
-### Step 3: Validate Judge
-
-```bash
-# Check for score marker
-if ! echo "$judge_response" | grep -qE "WEIGHTED_TOTAL|RATING:"; then
-  REJECT "Judge response missing score marker"
-fi
-
-# Check response length
-if [[ ${#judge_response} -lt 100 ]]; then
-  REJECT "Judge response too short"
-fi
-
-# Check tokens
-if [[ $judge_input_tokens -le 0 ]] || [[ $judge_output_tokens -le 0 ]]; then
-  REJECT "Invalid judge tokens"
-fi
-```
-
-### Step 4: Validate Scores
-
-```bash
-# Extract score from judge
-extracted=$(echo "$judge_response" | grep -oE "WEIGHTED_TOTAL[^0-9]*([0-9]+)" | grep -oE "[0-9]+" | tail -1)
-
-# Verify against claimed score
-if [[ "$extracted" != "$claimed_score" ]]; then
-  REJECT "Score mismatch: claimed=$claimed_score extracted=$extracted"
-fi
-
-# Check range
-if [[ $extracted -lt 1 ]] || [[ $extracted -gt 100 ]]; then
-  REJECT "Score out of range: $extracted"
-fi
-```
-
-### Step 5: Timestamp Sanity Check
-
-```bash
-# Calculate elapsed time
-first_ts=$(date -d "$first_agent_timestamp" +%s)
-last_ts=$(date -d "$judge_timestamp" +%s)
-elapsed=$((last_ts - first_ts))
-
-# Check minimum expected time
-num_agents=${#agents[@]}
-minimum=$((30 * num_agents))
-
-if [[ $elapsed -lt $minimum ]]; then
-  WARN "Elapsed time ${elapsed}s < expected ${minimum}s"
-fi
-```
-
-### Step 6: Display Validation Report
-
-```markdown
-### Finalize Run Validation
-
-**Type:** {type}
-**Scenario:** {scenario.name}
-
-#### Agent Validation
-| Agent | Timestamp | Response | Tokens | Status |
-|-------|-----------|----------|--------|--------|
-| {spec} | {ts} | {len} chars | in={in} out={out} | ✓ |
-
-#### Judge Validation
-| Field | Value | Status |
-|-------|-------|--------|
-| Timestamp | {ts} | ✓ |
-| Response | {len} chars | ✓ |
-| Score | {score}/100 | ✓ |
-
-#### Timestamp Sanity
-- Elapsed: {elapsed}s
-- Expected: ≥{minimum}s
-- Status: ✓ PASS
-
----
-**VALIDATION: PASSED**
-```
-
-### Step 7: Save Results
-
-If ALL validations pass:
-
-```bash
-# Ensure directory exists
-mkdir -p "$(dirname "$output_path")"
-
-# Write result
-echo "$result_json" > "$output_path"
-```
-
-Display:
-```
-✓ Saved to {output_path}
-```
-
-### Step 8: Return Success
-
-```json
-{
-  "success": true,
-  "path": "{output_path}",
-  "validation": {
-    "agents_validated": {count},
-    "judge_validated": true,
-    "scores_verified": true,
-    "timestamp_sane": true
-  }
-}
-```
-
-## On Validation Failure
-
-```markdown
-### ❌ VALIDATION FAILED
-
-**Failed Check:** {which validation}
-**Reason:** {specific reason}
-**Value:** {what was provided}
-
-**This run will NOT be saved.**
-
-To fix:
-- {remediation steps}
-```
-
-Return:
-```json
-{
-  "success": false,
-  "error": "{reason}",
-  "failed_check": "{which}"
-}
-```
-
-## The Golden Rule
-
-**Real data or no data.**
-
-Never estimate. Never fabricate. If validation fails, the run did not happen.