@pennyfarthing/core 7.0.2 → 7.4.0

package/pennyfarthing-dist/skills/persona-benchmark/SKILL.md

@@ -1,173 +0,0 @@
-# Persona Benchmark Skill
-
-Run benchmarks to compare persona effectiveness.
-
-## Usage
-
-```
-/persona-benchmark <test-case-id> <persona>
-/persona-benchmark cr-001 discworld
-/persona-benchmark tw-001 literary-classics
-/persona-benchmark --suite              # Run all tests, all personas
-/persona-benchmark --analyze            # Analyze collected results
-```
-
-## Benchmark Execution Protocol
-
-### Step 1: Load Test Case
-
-Read the test case from `.claude/benchmarks/test-cases/{category}/{id}.yaml`
-
-Extract:
-- `instructions` - What to give the agent
-- `code` - The code/problem to analyze
-- `known_issues` or `known_edge_cases` or `known_considerations` - DO NOT reveal to agent
-
-### Step 2: Configure Persona
-
-Temporarily set persona in `.claude/persona-config.yaml`:
-```yaml
-theme: {persona}
-```
-
-### Step 3: Execute Task
-
-Invoke the appropriate agent:
-- `code-review` → `/reviewer`
-- `test-writing` → `/tea`
-- `architecture` → `/architect`
-
-Provide ONLY:
-- The instructions
-- The code/problem
-
-Do NOT reveal known issues list.
-
-### Step 4: Collect Results
-
-After agent completes, score the output:
-
-**Quantitative Scoring:**
-```yaml
-# For code review:
-issues_found: [list of issues detected]
-issues_matched: [map to known_issues ids]
-detection_rate: issues_matched / total_known_issues
-false_positives: issues not in known list
-
-# For test writing:
-edge_cases_found: [list of edge cases covered]
-edge_cases_matched: [map to known_edge_cases ids]
-coverage_rate: edge_cases_matched / total_known_edge_cases
-
-# For architecture:
-considerations_found: [list of considerations mentioned]
-considerations_matched: [map to known_considerations ids]
-completeness_rate: considerations_matched / total_known_considerations
-```
-
-**Qualitative Scoring (1-5):**
-- `persona_consistency`: Did the agent stay in character?
-- `explanation_quality`: How well did it explain its findings?
-- `actionability`: How usable is the output?
-- `engagement`: How enjoyable was the interaction?
-
-### Step 5: Save Results
-
-Write to `.claude/benchmarks/results/{timestamp}-{persona}-{test-case-id}.yaml`:
-
-```yaml
-benchmark:
-  test_case: cr-001
-  persona: discworld
-  agent: reviewer
-  character: Granny Weatherwax
-  timestamp: 2024-01-15T10:30:00Z
-
-quantitative:
-  items_found: 8
-  items_expected: 14
-  items_matched:
-    - SQL_INJECTION_1
-    - SQL_INJECTION_2
-    - PLAINTEXT_PASSWORD
-    - PASSWORD_EXPOSURE_1
-    - PASSWORD_EXPOSURE_2
-    - NO_AUTH_CHECK
-    - ASYNC_DELETE_NO_TX
-    - ROWS_NOT_CLOSED
-  detection_rate: 0.57
-  false_positives: 1
-  weighted_score: 15.5
-  max_weighted_score: 22.5
-  weighted_rate: 0.69
-
-qualitative:
-  persona_consistency: 5
-  explanation_quality: 4
-  actionability: 4
-  engagement: 5
-
-notes: |
-  Found both SQL injections immediately with strong language.
-  Missed the error handling issues.
-  Very much in character - "I aten't reviewing code that's already dead."
-
-raw_output: |
-  [Full agent output preserved here]
-```
-
-## Analysis Mode
-
-When run with `--analyze`:
-
-1. Load all results from `.claude/benchmarks/results/`
-
-2. Aggregate by persona:
-```
-| Persona          | Detection Rate | False Pos | Persona Score | Engagement |
-|------------------|----------------|-----------|---------------|------------|
-| discworld        | 0.71           | 1.2       | 4.8           | 4.9        |
-| star-trek        | 0.68           | 0.8       | 4.5           | 4.2        |
-| literary-classics| 0.73           | 1.5       | 4.2           | 4.0        |
-| minimalist       | 0.65           | 0.5       | N/A           | 3.2        |
-```
-
-3. Aggregate by test category:
-```
-| Category     | Best Persona      | Avg Detection |
-|--------------|-------------------|---------------|
-| code-review  | literary-classics | 0.71          |
-| test-writing | discworld         | 0.68          |
-| architecture | star-trek         | 0.75          |
-```
-
-4. Statistical significance:
-- Calculate standard deviation
-- Note if differences are significant
-
-5. Qualitative patterns:
-- Which personas stay in character best?
-- Which provide most actionable output?
-- User enjoyment patterns
-
-## Running a Full Suite
-
-```bash
-# This will take a while - runs each test case with each persona
-/persona-benchmark --suite
-```
-
-Executes:
-- All test cases in `test-cases/`
-- With each persona: discworld, star-trek, literary-classics, minimalist
-- Saves individual results
-- Produces summary comparison
-
-## Tips for Valid Benchmarks
-
-1. **Same evaluator**: Same person should score qualitative metrics
-2. **Blind evaluation**: Score output before checking which persona
-3. **Multiple runs**: Run each test 3+ times for reliability
-4. **Fresh context**: Start new session for each benchmark run
-5. **Control variables**: Same time of day, same evaluator state

package/pennyfarthing-dist/skills/skill-registry.schema.json

@@ -1,102 +0,0 @@
-{
-  "$schema": "http://json-schema.org/draft-07/schema#",
-  "$id": "https://pennyfarthing.dev/schemas/skill-registry.schema.json",
-  "title": "Pennyfarthing Skill Registry",
-  "description": "Schema for the centralized skill metadata registry",
-  "type": "object",
-  "properties": {
-    "version": {
-      "type": "string",
-      "pattern": "^\\d+\\.\\d+\\.\\d+$",
-      "description": "Semantic version of the registry format"
-    },
-    "skills": {
-      "type": "object",
-      "description": "Map of skill names to their metadata",
-      "additionalProperties": {
-        "$ref": "#/definitions/skill"
-      }
-    }
-  },
-  "required": ["version", "skills"],
-  "additionalProperties": false,
-  "definitions": {
-    "skill": {
-      "type": "object",
-      "properties": {
-        "name": {
-          "type": "string",
-          "description": "Canonical name of the skill (matches directory name)"
-        },
-        "description": {
-          "type": "string",
-          "description": "Brief description for discovery and help text"
-        },
-        "category": {
-          "type": "string",
-          "enum": [
-            "development",
-            "project-management",
-            "theming",
-            "tools",
-            "ai-llm",
-            "documentation",
-            "benchmarking"
-          ],
-          "description": "Primary category for grouping skills"
-        },
-        "tags": {
-          "type": "array",
-          "items": { "type": "string" },
-          "description": "Searchable tags for discovery"
-        },
-        "version": {
-          "type": "string",
-          "pattern": "^\\d+\\.\\d+\\.\\d+$",
-          "description": "Skill version (semantic versioning)"
-        },
-        "prerequisites": {
-          "type": "array",
-          "items": { "type": "string" },
-          "description": "Other skills or tools required before using this skill"
-        },
-        "examples": {
-          "type": "array",
-          "items": {
-            "type": "object",
-            "properties": {
-              "context": {
-                "type": "string",
-                "description": "When to use this invocation"
-              },
-              "invocation": {
-                "type": "string",
-                "description": "How to invoke the skill"
-              }
-            },
-            "required": ["context", "invocation"],
-            "additionalProperties": false
-          },
-          "description": "Example invocations with context"
-        },
-        "anti_patterns": {
-          "type": "array",
-          "items": { "type": "string" },
-          "description": "Common mistakes or misuses to avoid"
-        },
-        "related_skills": {
-          "type": "array",
-          "items": { "type": "string" },
-          "description": "Skills that complement or relate to this one"
-        },
-        "keywords": {
-          "type": "array",
-          "items": { "type": "string" },
-          "description": "Additional searchable terms for discovery"
-        }
-      },
-      "required": ["name", "description", "category", "tags"],
-      "additionalProperties": false
-    }
-  }
-}

package/pennyfarthing-dist/skills/skill-registry.yaml

@@ -1,335 +0,0 @@
-# Pennyfarthing Skill Registry
-# Central metadata for all built-in skills
-# Schema: skill-registry.schema.json
-
-version: "1.0.0"
-
-skills:
-  agentic-patterns:
-    name: agentic-patterns
-    description: Core reasoning patterns for building effective LLM agents
-    category: ai-llm
-    tags: [reasoning, patterns, agents, llm]
-    version: "1.0.0"
-    prerequisites: []
-    examples:
-      - context: Designing agent behavior
-        invocation: /agentic-patterns
-      - context: Debugging agent failures
-        invocation: /agentic-patterns troubleshooting
-    anti_patterns:
-      - Don't apply patterns mechanically without understanding context
-    related_skills: [context-engineering]
-    keywords: [react, reflection, planning, tool-use, chain-of-thought]
-
-  changelog:
-    name: changelog
-    description: Maintain changelogs following Keep a Changelog format with conventional commits
-    category: documentation
-    tags: [changelog, releases, versioning]
-    version: "1.0.0"
-    prerequisites: []
-    examples:
-      - context: Creating release notes
-        invocation: /changelog
-      - context: Auto-generating from commits
-        invocation: /changelog generate
-    anti_patterns:
-      - Don't manually edit CHANGELOG.md entries after generation
-    related_skills: []
-    keywords: [keepachangelog, conventional-commits, semver, release-notes]
-
-  code-review:
-    name: code-review
-    description: Code review checklists and patterns for quality assurance
-    category: development
-    tags: [review, quality, checklist]
-    version: "1.0.0"
-    prerequisites: []
-    examples:
-      - context: Self-review before commit
-        invocation: /code-review
-      - context: Reviewing a PR
-        invocation: /code-review pr-checklist
-    anti_patterns:
-      - Don't skip security considerations in review
-      - Don't approve without running tests
-    related_skills: [testing, dev-patterns]
-    keywords: [pr, pull-request, quality, linting, security]
-
-  context-engineering:
-    name: context-engineering
-    description: Strategies for managing context windows in long-running agent sessions
-    category: ai-llm
-    tags: [context, optimization, agents]
-    version: "1.0.0"
-    prerequisites: []
-    examples:
-      - context: Approaching context limits
-        invocation: /context-engineering
-      - context: Designing subagent prompts
-        invocation: /context-engineering subagent-design
-    anti_patterns:
-      - Don't load unnecessary context upfront
-      - Don't repeat large code blocks in prompts
-    related_skills: [agentic-patterns]
-    keywords: [context-window, tokens, summarization, lazy-loading]
-
-  cyclist:
-    name: cyclist
-    description: Launch Cyclist visual terminal for Claude Code monitoring
-    category: tools
-    tags: [visual, monitoring, terminal]
-    version: "1.0.0"
-    prerequisites: []
-    examples:
-      - context: Starting visual monitor
-        invocation: /cyclist
-      - context: Debugging sessions
-        invocation: /cyclist debug
-    anti_patterns:
-      - Don't run Cyclist in headless environments
-    related_skills: []
-    keywords: [tui, terminal, visualization, dashboard]
-
-  dev-patterns:
-    name: dev-patterns
-    description: Common development patterns, fixes, and gotchas
-    category: development
-    tags: [patterns, debugging, implementation]
-    version: "1.0.0"
-    prerequisites: []
-    examples:
-      - context: Implementing features
-        invocation: /dev-patterns
-      - context: Avoiding known pitfalls
-        invocation: /dev-patterns gotchas
-    anti_patterns:
-      - Don't apply patterns from other frameworks blindly
-    related_skills: [code-review, testing]
-    keywords: [implementation, debugging, fixes, best-practices]
-
-  finalize-run:
-    name: finalize-run
-    description: Validate and save benchmark run results - single exit point for all runs
-    category: benchmarking
-    tags: [benchmark, validation, results]
-    version: "1.0.0"
-    prerequisites: [judge]
-    examples:
-      - context: Completing a benchmark run
-        invocation: /finalize-run
-    anti_patterns:
-      - Never save results without passing through this skill
-    related_skills: [judge, persona-benchmark]
-    keywords: [benchmark, save, validate, guardrail]
-
-  jira:
-    name: jira
-    description: Jira CLI commands for sprint management
-    category: project-management
-    tags: [jira, issues, sprint]
-    version: "1.0.0"
-    prerequisites: []
-    examples:
-      - context: Viewing sprint issues
-        invocation: /jira sprint
-      - context: Assigning issues
-        invocation: /jira assign ISSUE-123
-    anti_patterns:
-      - Don't bypass Jira for sprint tracking
-    related_skills: [sprint-context, story-management]
-    keywords: [atlassian, issues, tickets, backlog]
-
-  judge:
-    name: judge
-    description: Evaluate agent responses using standardized rubrics
-    category: benchmarking
-    tags: [evaluation, rubrics, scoring]
-    version: "1.0.0"
-    prerequisites: []
-    examples:
-      - context: Evaluating agent response
-        invocation: /judge evaluate
-      - context: Running benchmark comparison
-        invocation: /judge compare
-    anti_patterns:
-      - Don't modify rubrics during a benchmark run
-    related_skills: [finalize-run, persona-benchmark]
-    keywords: [evaluation, scoring, metrics, grading]
-
-  just:
-    name: just
-    description: Run just recipes for project tasks like dev servers, tests, and databases
-    category: tools
-    tags: [tasks, runner, automation]
-    version: "1.0.0"
-    prerequisites: []
-    examples:
-      - context: Starting dev servers
-        invocation: /just dev
-      - context: Running database tasks
-        invocation: /just db-reset
-    anti_patterns:
-      - Don't create complex recipes - keep them simple and composable
-    related_skills: [testing]
-    keywords: [justfile, make, tasks, automation, command-runner]
-
-  mermaid:
-    name: mermaid
-    description: Generate diagrams using Mermaid syntax for documentation
-    category: tools
-    tags: [diagrams, visualization, documentation]
-    version: "1.0.0"
-    prerequisites: []
-    examples:
-      - context: Creating architecture diagrams
-        invocation: /mermaid architecture
-      - context: Sequence diagrams
-        invocation: /mermaid sequence
-    anti_patterns:
-      - Don't create overly complex diagrams - split into multiple if needed
-    related_skills: [changelog]
-    keywords: [flowchart, sequence, er-diagram, gantt, class-diagram]
-
-  permissions:
-    name: permissions
-    description: Manage runtime permission grants - list, grant, and revoke tool access
-    category: project-management
-    tags: [permissions, security, tools]
-    version: "1.0.0"
-    prerequisites: []
-    examples:
-      - context: Viewing active grants
-        invocation: /permissions
-      - context: Granting WebFetch access
-        invocation: /permissions grant WebFetch "*.github.com"
-      - context: Revoking tool access
-        invocation: /permissions revoke Bash
-    anti_patterns:
-      - Don't manually edit settings.local.json permissions - use skill
-      - Don't grant overly broad scope patterns
-    related_skills: [sprint-context]
-    keywords: [grants, scopes, tools, runtime, security]
-
-  persona-benchmark:
-    name: persona-benchmark
-    description: Run benchmarks to compare persona effectiveness
-    category: benchmarking
-    tags: [benchmark, personas, comparison]
-    version: "1.0.0"
-    prerequisites: [judge]
-    examples:
-      - context: Testing persona on code review
-        invocation: /persona-benchmark cr-001 discworld
-      - context: Testing tech writer persona
-        invocation: /persona-benchmark tw-001 literary-classics
-    anti_patterns:
-      - Don't compare results across different rubric versions
-    related_skills: [judge, finalize-run, theme]
-    keywords: [benchmark, comparison, personas, testing]
-
-  sprint-context:
-    name: sprint-context
-    description: Sprint status, backlog, and story management context
-    category: project-management
-    tags: [sprint, status, backlog]
-    version: "1.0.0"
-    prerequisites: []
-    examples:
-      - context: Checking sprint status
-        invocation: /sprint-context
-      - context: Finding available stories
-        invocation: /sprint-context available
-    anti_patterns:
-      - Don't manually edit sprint YAML - use skills
-    related_skills: [story-management, jira]
-    keywords: [sprint, backlog, velocity, kanban]
-
-  story-management:
-    name: story-management
-    description: Story creation, sizing, and sprint workflow patterns
-    category: project-management
-    tags: [stories, sizing, workflow]
-    version: "1.0.0"
-    prerequisites: []
-    examples:
-      - context: Creating new stories
-        invocation: /story-management create
-      - context: Sizing stories
-        invocation: /story-management size
-    anti_patterns:
-      - Don't create stories without acceptance criteria
-    related_skills: [sprint-context, jira]
-    keywords: [user-stories, estimation, points, acceptance-criteria]
-
-  testing:
-    name: testing
-    description: Test commands and TDD workflow patterns
-    category: development
-    tags: [tdd, testing, quality]
-    version: "1.0.0"
-    prerequisites: []
-    examples:
-      - context: Running project tests
-        invocation: /testing
-      - context: Debugging test failures
-        invocation: /testing troubleshoot
-    anti_patterns:
-      - Don't run tests directly - use testing-runner subagent
-      - Don't skip RED phase in TDD workflow
-    related_skills: [dev-patterns, code-review]
-    keywords: [jest, vitest, pytest, unit-test, integration, tdd, red-green]
-
-  theme:
-    name: theme
-    description: Manage persona themes - list, show, and set active theme
-    category: theming
-    tags: [personas, themes, customization]
-    version: "1.0.0"
-    prerequisites: []
-    examples:
-      - context: Listing available themes
-        invocation: /list-themes
-      - context: Showing current theme
-        invocation: /show-theme
-      - context: Setting active theme
-        invocation: /set-theme discworld
-    anti_patterns:
-      - Don't edit config.local.yaml directly - use skill
-    related_skills: [theme-creation]
-    keywords: [personas, characters, discworld, literary-classics]
-
-  theme-creation:
-    name: theme-creation
-    description: Create custom persona themes with AI-driven generation
-    category: theming
-    tags: [personas, themes, creation]
-    version: "1.0.0"
-    prerequisites: [theme]
-    examples:
-      - context: Creating new theme
-        invocation: /theme-maker
-      - context: Interactive theme wizard
-        invocation: /create-theme
-    anti_patterns:
-      - Don't create incomplete themes missing required agents
-    related_skills: [theme]
-    keywords: [personas, custom, generation, wizard]
-
-  yq:
-    name: yq
-    description: YAML processor for reading, modifying, and querying YAML files
-    category: tools
-    tags: [yaml, processing, cli]
-    version: "1.0.0"
-    prerequisites: []
-    examples:
-      - context: Reading YAML values
-        invocation: /yq read
-      - context: Modifying YAML files
-        invocation: /yq modify
-    anti_patterns:
-      - Don't use complex expressions without testing first
-    related_skills: [just]
-    keywords: [yaml, jq, parsing, query, mikefarah]