aiag-cli 2.2.2 → 2.3.0

package/templates/skills/prd-taskmaster/.taskmaster/docs/prd.md

@@ -0,0 +1,2571 @@
+# PRD: Agentic Arena-Based Skill Creation & Optimization System
+
+**Author:** anombyte
+**Date:** 2025-01-22
+**Status:** Ready for Implementation
+**Version:** 4.0 (Taskmaster Optimized)
+**Taskmaster Optimized:** Yes
+**Original Version:** v3.0 from skill_creating/planning/PRD.md
+
+---
+
+## Table of Contents
+
+1. [Executive Summary](#executive-summary)
+2. [Problem Statement](#problem-statement)
+3. [Goals & Success Metrics](#goals--success-metrics)
+4. [User Stories](#user-stories)
+5. [Functional Requirements](#functional-requirements)
+6. [Non-Functional Requirements](#non-functional-requirements)
+7. [Technical Considerations](#technical-considerations)
+8. [Implementation Roadmap](#implementation-roadmap)
+9. [Out of Scope](#out-of-scope)
+10. [Open Questions & Risks](#open-questions--risks)
+11. [Validation Checkpoints](#validation-checkpoints)
+12. [Appendix: Task Breakdown Hints](#appendix-task-breakdown-hints)
+
+---
+
+## Executive Summary
+
+Current Claude Code skill creation is "write once" without optimization or validation. Users need an intelligent system that evolves skills through tournament-style arena battles with empirical testing—comparing real outputs, not theoretical code quality. This system provides database-first collective knowledge (show existing skills before creating), delivers quick base skills in 30 seconds for immediate use, then runs background optimization (25-45 min) using agentic orchestration and LLM-as-judge evaluation. Expected impact: average skill scores improve from 78/100 (base) to 93/100 (optimized), with 50% of requests served from collective database within 3 months.
+
+---
+
+## Problem Statement
+
+### Current Situation
+
+Users create Claude Code skills manually by writing SKILL.md files without:
+- **Quality validation**: No way to know if skill will work well before deploying
+- **Optimization**: Skills are written once and never improved
+- **Collective knowledge**: Each user reinvents solutions others have already created
+- **Empirical testing**: Skills are evaluated by reading code, not running them
+- **Evolution mechanism**: No systematic way to iterate and improve skills
+
+**Evidence:**
+- From user requirement: "I can't create PRDs myself. I want the best possible PRD for optimal outcomes."
+- User explicitly stated: "Planning is 95% of the work with vibe coding"
+- Current skill_creating skill guides creation but doesn't optimize or validate
+
+### User Impact
+
+- **Who is affected:** Claude Code users creating custom skills (engineers, technical users)
+- **How they're affected:**
+  - Spend hours writing skills that may not work well
+  - No feedback on skill quality until they use it in production
+  - Reinvent solutions others have already created
+  - No systematic improvement process
+  - Miss opportunities to leverage collective expertise
+- **Severity:** High - Directly impacts development velocity and skill effectiveness
+
+### Business Impact
+
+- **Cost of problem:**
+  - Wasted time creating suboptimal skills (estimated 2-4 hours per skill)
+  - Poor skill quality reduces Claude Code effectiveness
+  - User frustration from trial-and-error skill development
+- **Opportunity cost:**
+  - Missing collective intelligence benefits (GitHub Copilot-like network effects)
+  - Not capitalizing on community improvements
+  - Slower Claude Code adoption due to skill creation friction
+- **Strategic importance:**
+  - Skills are core differentiator for Claude Code vs competitors
+  - Quality skill ecosystem drives user retention and engagement
+  - Collective evolution enables exponential improvement vs linear
+
+### Why Solve This Now?
+
+1. **2025 LLM evaluation best practices available**: Arena-Lite architecture, LLM-as-judge patterns, realistic test generation
+2. **Technical capability ready**: Claude Code Task tool enables background agentic orchestration
+3. **User demand clear**: Explicit request for "best possible" skills with automated optimization
+4. **Competitive timing**: First to market with collective skill evolution for AI coding tools
+5. **Foundation for future**: This enables advanced features (server farms, continuous evolution)
+
+---
+
+## Goals & Success Metrics
+
+### Goal 1: Improve Skill Quality Through Arena Optimization
+
+**Description:** Skills optimized through arena battles score significantly higher than base versions
+
+**Metric:** Average score improvement (final vs base skill)
+
+**Baseline:** 0 points (no optimization exists today)
+
+**Target:** +15 points average (e.g., 78/100 base → 93/100 optimized)
+
+**Timeframe:** Measured per skill, target achieved for 80% of skills within 30 min arena completion
+
+**Measurement Method:** Automated scoring via LLM-as-judge comparing base (v0.1) vs optimized (v1.0) skill outputs
+
+---
+
+### Goal 2: Enable Collective Knowledge Reuse
+
+**Description:** Users find and reuse existing high-quality skills instead of recreating
+
+**Metric:** Reuse rate (% of skill requests served from collective database)
+
+**Baseline:** 0% (no collective database exists)
+
+**Target:** 50% of requests match existing skills within 3 months
+
+**Timeframe:** 3 months post-launch
+
+**Measurement Method:** Track database queries with confidence > 0.8, user selection of existing vs "build custom"
+
+---
+
+### Goal 3: Fast Time-to-First-Value
+
+**Description:** Users get working skill immediately while optimization runs in background
+
+**Metric:** Time to base skill (v0.1) delivery
+
+**Baseline:** N/A (current: manual creation 30-120 min)
+
+**Target:** < 30 seconds for base skill generation
+
+**Timeframe:** Every skill creation
+
+**Measurement Method:** Track timestamp from user request to v0.1 skill deployed and usable
+
+---
+
+### Goal 4: Reliable Arena Completion Time
+
+**Description:** Background optimization completes within predictable time windows
+
+**Metric:** Arena completion time (p95)
+
+**Baseline:** N/A
+
+**Target:** < 30 minutes for moderate-complexity skills (p95)
+
+**Timeframe:** Every arena execution
+
+**Measurement Method:** Track arena start → convergence timestamps, categorize by skill complexity
+
+---
+
+### Goal 5: High User Satisfaction
+
+**Description:** Users rate optimized skills highly and adopt the system
+
+**Metric:** Average user rating of optimized skills
+
+**Baseline:** N/A
+
+**Target:** ≥ 4.5/5 stars average
+
+**Timeframe:** Ongoing (minimum 50 ratings for statistical validity)
+
+**Measurement Method:** Post-execution optional rating prompt (1-5 stars), aggregate in database
+
+---
+
+### Goal 6: Build Thriving Collective Database
+
+**Description:** Grow database of high-quality community-contributed skills
+
+**Metric:** Total unique skills in collective database
+
+**Baseline:** 0 skills
+
+**Target:** 100+ skills across diverse domains in 3 months
+
+**Timeframe:** 3 months post-launch
+
+**Measurement Method:** Count unique skill_id entries in Pinecone database
+
+---
+
+## User Stories
+
+### Story 1: Database-First Skill Discovery
+
+**As a** Claude Code user,
+**I want to** see existing high-quality skills before creating a new one,
+**So that I can** reuse proven solutions instead of reinventing.
+
+**Acceptance Criteria:**
+- [ ] System queries Pinecone database before generating new skill
+- [ ] Shows matching skills with arena scores (e.g., 91.5/100)
+- [ ] Shows user ratings (e.g., ⭐4.7/5 from 342 users)
+- [ ] Shows last updated timestamp and key features
+- [ ] User can select existing skill or choose "Build custom"
+- [ ] Confidence scoring for matches (>0.8 shown, <0.8 clarifies)
+- [ ] Results appear within 2 seconds of query
+
+**Task Breakdown Hint:**
+- Task 1.1: Implement Pinecone vector search integration (6h)
+- Task 1.2: Build requirement fingerprint generation (4h)
+- Task 1.3: Create search results display UI (5h)
+- Task 1.4: Add confidence scoring and ranking logic (3h)
+- Task 1.5: Implement user selection workflow (3h)
+- Task 1.6: Write tests for search accuracy (4h)
+
+**Dependencies:** R11 (Collective API & Database), R4 (Requirement Extraction)
+
+---
+
+### Story 2: Progressive Skill Delivery (Quick Base → Optimized)
+
+**As a** Claude Code user,
+**I want to** get a working skill immediately while optimization runs in background,
+**So that I can** start using it right away without waiting 30 minutes.
+
+**Acceptance Criteria:**
+- [ ] Quick initial research completes in <30 seconds
+- [ ] Base skill (v0.1) generated and deployed immediately
+- [ ] User can use v0.1 skill while arena runs in background
+- [ ] Optional quick scoring of v0.1 (user can decline)
+- [ ] Background arena starts automatically after v0.1 delivery
+- [ ] User receives notification when optimized v1.0 ready
+- [ ] Comparison shows improvement metrics (e.g., +15 points)
+- [ ] User can review outputs and choose to deploy v1.0 or keep v0.1
+
+**Task Breakdown Hint:**
+- Task 2.1: Implement dual-track research system (quick + deep) (8h)
+- Task 2.2: Build base skill generation from quick research (6h)
+- Task 2.3: Create background job orchestration via Task tool (10h)
+- Task 2.4: Implement optional quick scoring workflow (4h)
+- Task 2.5: Build notification system for arena completion (3h)
+- Task 2.6: Create comparison UI for v0.1 vs v1.0 (6h)
+- Task 2.7: Write tests for progressive delivery flow (5h)
+
+**Dependencies:** R2 (Dual-Track Research), R10 (Background Execution), R14 (Adaptive Complexity)
+
+---
+
+### Story 3: Agentic Question Generation
+
+**As a** Claude Code user,
+**I want** domain-specific questions about my requirements,
+**So that** the system can optimize for what matters most to me.
+
+**Acceptance Criteria:**
+- [ ] System analyzes skill domain from user request
+- [ ] Generates 3-7 questions specific to domain (e.g., PRD vs PDF extraction)
+- [ ] Questions map to evaluation weight priorities
+- [ ] Example question: "What's most important: completeness or speed?"
+- [ ] User answers converted to weighted criteria (e.g., Quality: 64%, Speed: 5%)
+- [ ] Default weights provided if user skips questions
+- [ ] Question generation completes within quick research phase (<30s)
+
+**Task Breakdown Hint:**
+- Task 3.1: Build domain analysis agent (6h)
+- Task 3.2: Create question generation templates by domain (8h)
+- Task 3.3: Implement answer-to-weight conversion logic (5h)
+- Task 3.4: Add default weight fallback (2h)
+- Task 3.5: Write tests for question relevance (4h)
+
+**Dependencies:** R3 (Agentic Question Generation), AGENTIC_WEIGHTING_SOLUTIONS.md integration
+
+---
+
+### Story 4: Tournament Arena with Empirical Testing
+
+**As a** system administrator,
+**I want** skills to compete in arena battles using real outputs,
+**So that** winners are selected based on empirical quality, not theoretical code review.
+
+**Acceptance Criteria:**
+- [ ] Arena generates 3 skill variations (A, B, C) in Round 1
+- [ ] All variations execute with identical realistic test input
+- [ ] System captures complete real outputs (PRDs, code, data, etc.)
+- [ ] LLM judge compares outputs directly (not code)
+- [ ] Judge uses weighted criteria from user questions
+- [ ] Pairwise comparisons with position bias mitigation (randomized order)
+- [ ] Bradley-Terry ranking determines winner
+- [ ] Winner advances to next round vs 2 new refined variations
+- [ ] Arena stops when convergence detected (score plateau, time limit, or target achieved)
+- [ ] Maximum 10 rounds or 30 minutes (whichever comes first)
+
+**Task Breakdown Hint:**
+- Task 4.1: Implement skill variation generator agent (10h)
+- Task 4.2: Build skill execution isolation sandbox (8h)
+- Task 4.3: Create output capture system (4h)
+- Task 4.4: Implement LLM-as-judge with pairwise comparison (12h)
+- Task 4.5: Add Bradley-Terry ranking algorithm (6h)
+- Task 4.6: Build convergence detection logic (5h)
+- Task 4.7: Create tournament orchestration loop (8h)
+- Task 4.8: Write comprehensive arena tests (10h)
+
+**Dependencies:** R5 (Realistic Test Data), R6 (Tournament Arena), R7 (Skill Execution), R8 (LLM-as-Judge), R9 (Convergence)
+
+---
+
+### Story 5: Realistic Test Data Generation
+
+**As a** system administrator,
+**I want** realistic test scenarios for skill evaluation,
+**So that** arena battles reflect real-world usage, not toy examples.
+
+**Acceptance Criteria:**
+- [ ] Agent discovers realistic use cases via web search
+- [ ] LLM takes persona appropriate to skill type (e.g., "Product Manager" for PRD skills)
+- [ ] Generates realistic input data (not "Create a PRD" but "Add password reset to fintech SaaS app")
+- [ ] Test scenarios evolve across rounds (simple → complex → edge case)
+- [ ] Validates realism through domain pattern matching
+- [ ] Caches validated scenarios in database for reuse
+- [ ] Each skill tested with minimum 1 realistic scenario per round
+
+**Task Breakdown Hint:**
+- Task 5.1: Build test data generator agent with web search (8h)
+- Task 5.2: Create persona-based scenario generation (6h)
+- Task 5.3: Implement scenario evolution logic (5h)
+- Task 5.4: Add realism validation (4h)
+- Task 5.5: Build scenario caching in Pinecone (4h)
+- Task 5.6: Write tests for scenario quality (4h)
+
+**Dependencies:** R5 (Realistic Test Data Generation), Pinecone database
+
+---
+
+### Story 6: User-in-the-Loop Validation
+
+**As a** Claude Code user,
+**I want to** review arena outputs and score them myself,
+**So that** I can validate automated judgments and provide feedback.
+
+**Acceptance Criteria:**
+- [ ] All arena results stored locally in `.claude/skills/[skill-name]/arena_results/`
+- [ ] Results stored as JSON with inputs, outputs, scores, reasoning
+- [ ] User can browse results after arena completion
+- [ ] UI shows side-by-side comparison of outputs
+- [ ] User can score each output (1-5 stars)
+- [ ] User feedback submitted to database (opt-in)
+- [ ] Feedback improves future arena weights and judgments
+
+**Task Breakdown Hint:**
+- Task 6.1: Create local arena results storage system (4h)
+- Task 6.2: Build results browsing UI (8h)
+- Task 6.3: Implement side-by-side output comparison (6h)
+- Task 6.4: Add user scoring interface (4h)
+- Task 6.5: Build feedback submission to database (3h)
+- Task 6.6: Write tests for validation flow (4h)
+
+**Dependencies:** R12 (User-in-the-Loop Validation), R13 (Feedback Collection)
+
+---
+
+### Story 7: Collective Submission & Leaderboards
+
+**As a** Claude Code user,
+**I want to** submit my optimized skill to the collective database,
+**So that** others can benefit and my skill can evolve further.
+
+**Acceptance Criteria:**
+- [ ] After arena completion, special notification if skill beats database champions
+- [ ] Opt-in prompt to submit to collective
+- [ ] Submission includes: skill content, arena scores (dimensional + overall), weights used, generation/lineage
+- [ ] Privacy-conscious: input hash (not actual input), output samples (500 chars), anonymous user ID
+- [ ] Skills appear in search results for future users
+- [ ] Leaderboard shows top skills by domain
+- [ ] ELO ratings update based on usage and feedback
+
+**Task Breakdown Hint:**
+- Task 7.1: Build champion comparison logic (3h)
+- Task 7.2: Create submission UI with privacy controls (6h)
+- Task 7.3: Implement skill submission API endpoint (8h)
+- Task 7.4: Build leaderboard display (6h)
+- Task 7.5: Add ELO rating calculation (5h)
+- Task 7.6: Write tests for submission flow (5h)
+
+**Dependencies:** R11 (Collective API & Database), R13 (Feedback Collection), skill lineage tracking
+
+---
+
+## Functional Requirements
+
+### Must Have (P0) - Critical for MVP
+
+#### REQ-001: Database-First Query System
+
+**Description:** System MUST query Pinecone collective database before generating new skills, showing existing matches to enable reuse.
+
+**Acceptance Criteria:**
+- [ ] Extract requirements from user request ("Create PRD skill for comprehensive planning")
+- [ ] Generate requirement fingerprint (SHA-256 hash of structured requirements)
+- [ ] Query Pinecone with semantic search (embedding + metadata filtering)
+- [ ] Return matches with confidence scores (0-1 scale)
+- [ ] Show skills with confidence > 0.7
+- [ ] Display: name, domain, arena scores (dimensional + overall), user ratings, last updated
+- [ ] User can select existing skill or choose "Build custom"
+- [ ] Query completes in < 2 seconds
+
+**Technical Specification:**
+```typescript
+interface SkillSearchRequest {
+  userRequest: string;
+  domain?: string;  // Optional: "prd-generation", "pdf-extraction", etc.
+}
+
+interface SkillSearchResult {
+  skillId: string;
+  name: string;
+  domain: string;
+  confidence: number;  // 0-1
+  scores: {
+    completeness: number;
+    clarity: number;
+    quality: number;
+    efficiency: number;
+    overall: number;
+  };
+  feedback: {
+    avgRating: number;
+    totalRatings: number;
+    successRate: number;
+  };
+  lastUpdated: string;  // ISO 8601
+  keyFeatures: string[];
+}
+
+// API Call
+POST /api/collective/search
+{
+  "userRequest": "Create comprehensive PRD skill",
+  "domain": "prd-generation"
+}
+
+// Response
+{
+  "matches": [
+    {
+      "skillId": "uuid-123",
+      "name": "Comprehensive PRD Generator",
+      "confidence": 0.92,
+      "scores": { "overall": 91.5, ... },
+      "feedback": { "avgRating": 4.7, "totalRatings": 342 },
+      ...
+    }
+  ],
+  "queryTime": 1.2
+}
+```
+
+**Task Breakdown:**
+- Implement requirement extraction and fingerprinting: Medium (6h)
+- Build Pinecone semantic search integration: Medium (8h)
+- Add confidence scoring logic: Small (4h)
+- Create search results display: Medium (6h)
+- Write integration tests: Small (4h)
+
+**Dependencies:** Pinecone database setup, embedding model (OpenAI text-embedding-3-small)
+
+---
+
+#### REQ-002: Quick Base Skill Generation (v0.1)
+
+**Description:** Generate working base skill in < 30 seconds using quick initial research, deployable immediately.
+
+**Acceptance Criteria:**
+- [ ] Quick pattern research: Scan awesome-claude-skills examples (< 10s)
+- [ ] Quick domain research: Basic WebSearch for best practices (< 15s)
+- [ ] Generate domain-specific questions (3-7 questions) (< 5s)
+- [ ] User answers questions or accepts defaults
+- [ ] Generate base SKILL.md from quick research + answers (< 5s)
+- [ ] Deploy to `.claude/skills/[skill-name]/SKILL.md`
+- [ ] Skill immediately usable (activates on triggers)
+- [ ] Total time user request → deployed v0.1: < 30 seconds (excluding user answer time)
+
+**Technical Specification:**
+```python
+# Quick Research Flow
+def generate_base_skill(user_request, user_answers):
+    # Phase 1: Quick Research (parallel, 15s total)
+    pattern_research = quick_scan_patterns(user_request)  # 10s
+    domain_research = quick_web_search(user_request)      # 15s (parallel)
+
+    # Phase 2: Question Generation (5s)
+    questions = generate_questions(pattern_research, domain_research)
+
+    # User answers (time not counted)
+    answers = await user.answer(questions) or get_defaults()
+
+    # Phase 3: Skill Generation (5s)
+    skill_content = generate_skill_md(
+        pattern_research,
+        domain_research,
+        answers
+    )
+
+    # Phase 4: Deployment (1s)
+    deploy_skill(skill_content, skill_name)
+
+    return {"version": "0.1", "path": skill_path, "time": elapsed}
+```
+
+**Task Breakdown:**
+- Implement quick pattern scanner: Medium (6h)
+- Build quick domain research: Small (4h)
+- Create question generator agent: Medium (8h)
+- Implement base skill template engine: Medium (6h)
+- Add deployment automation: Small (3h)
+- Write tests for 30s SLA: Small (4h)
+
+**Dependencies:** WebSearch tool, pattern database (awesome-claude-skills)
+
+---
+
+#### REQ-003: Background Arena Orchestration
+
+**Description:** After v0.1 delivery, automatically start background arena optimization using Task tool for non-blocking execution.
+
+**Acceptance Criteria:**
+- [ ] Arena starts automatically after v0.1 deployed
+- [ ] User can continue working (non-blocking)
+- [ ] Orchestrator-worker pattern: central orchestrator + worker agents
+- [ ] Workers run in parallel via Task tool
+- [ ] State persistence (can resume if interrupted)
+- [ ] Progress indicator (optional, non-intrusive)
+- [ ] Graceful handling of interruptions
+- [ ] User can manually stop optimization
+- [ ] Arena completes in < 30 min for moderate skills (p95)
+
+**Technical Specification:**
+```typescript
+interface ArenaOrchestrator {
+  skillName: string;
+  baseSkill: string;  // v0.1 content
+  userWeights: WeightConfig;
+  complexity: 1-10;
+
+  async run(): ArenaResult {
+    // Start background job
+    const jobId = await Task.start({
+      subagent_type: "arena-orchestrator",
+      prompt: `Run arena optimization for ${skillName}...`,
+      async: true
+    });
+
+    // State machine
+    while (!converged) {
+      // Round N
+      variations = await generateVariations(currentBest);
+      testData = await generateRealisticTest(round);
+      outputs = await Promise.all(
+        variations.map(v => executeSkill(v, testData))
+      );
+      scores = await judgeOutputs(outputs, userWeights);
+      currentBest = selectWinner(scores);
+
+      // Check convergence
+      if (shouldStop(scores, elapsed, rounds)) {
+        converged = true;
+      }
+    }
+
+    return { winner: currentBest, rounds, time: elapsed };
+  }
+}
+```
+
+**Task Breakdown:**
+- Implement orchestrator-worker pattern: Large (12h)
+- Build state persistence system: Medium (6h)
+- Add graceful interruption handling: Small (4h)
+- Create progress tracking (optional): Small (3h)
+- Write orchestration tests: Medium (8h)
+
+**Dependencies:** Claude Code Task tool, R6 (Tournament Arena), R9 (Convergence Detection)
+
+---
+
+#### REQ-004: Realistic Test Data Generation
+
+**Description:** Generate realistic test scenarios using web search and persona-based LLM generation, not generic toy examples.
+
+**Acceptance Criteria:**
+- [ ] Web search for domain-specific realistic examples
+- [ ] LLM takes persona appropriate to skill (e.g., "Product Manager" for PRD)
+- [ ] Generates realistic input matching real-world complexity
+- [ ] Example: NOT "Create a PRD" but "Add OAuth 2.0 authentication supporting Google, Microsoft, GitHub to B2B SaaS app with existing auth system, 2FA support, SOC 2 compliance"
+- [ ] Scenarios evolve across rounds: Round 1 (simple), Round 2 (complex), Round 3 (edge case)
+- [ ] Validates realism via domain pattern matching
+- [ ] Caches validated scenarios in Pinecone for reuse
+
+**Technical Specification:**
+```python
+# Realistic Test Data Generation
+class RealisticTestGenerator:
+    def generate(self, skill_domain, round_num):
+        # Step 1: Web search for real examples
+        examples = web_search(f"{skill_domain} use cases 2025")
+
+        # Step 2: Take persona
+        persona = get_persona(skill_domain)
+        # e.g., "Product Manager at B2B SaaS company"
+
+        # Step 3: Generate realistic scenario
+        scenario = llm.generate(
+            prompt=f"""You are a {persona}.
+            Generate a realistic request for {skill_domain}.
+            Based on these real-world examples: {examples}
+
+            Complexity level: {get_complexity(round_num)}
+            - Round 1: Simple, common use case
+            - Round 2: Complex, multi-part scenario
+            - Round 3: Edge case, unusual constraints
+
+            Be specific, include real constraints and context."""
+        )
+
+        # Step 4: Validate realism
+        if validate_realism(scenario, examples):
+            cache_scenario(skill_domain, scenario)
+            return scenario
+        else:
+            return generate(skill_domain, round_num)  # Retry
+```
+
+**Task Breakdown:**
+- Build web search integration for examples: Medium (5h)
+- Create persona mapping by domain: Small (4h)
+- Implement scenario generation with evolution: Medium (8h)
+- Add realism validation: Medium (5h)
+- Build scenario caching in Pinecone: Small (4h)
+- Write tests for scenario quality: Medium (6h)
+
+**Dependencies:** WebSearch tool, Pinecone database, LLM access
+
+---
+
+#### REQ-005: Arena Tournament with Pairwise Comparison
+
+**Description:** Run tournament battles using Arena-Lite architecture with direct pairwise comparison of real skill outputs.
+
+**Acceptance Criteria:**
+- [ ] Round 1: Generate 3 variations (A, B, C)
+- [ ] Execute all variations with identical test input
+- [ ] Capture complete real outputs (not truncated)
+- [ ] Judge compares outputs pairwise (A vs B, B vs C, A vs C)
+- [ ] Judge uses weighted criteria from user questions
+- [ ] Position bias mitigation: randomize output order
+- [ ] Bradley-Terry ranking from pairwise results
+- [ ] Winner (highest rank) advances to next round
+- [ ] Next round: Winner vs 2 new refined variations
+- [ ] Repeat until convergence
+- [ ] Maximum 10 variations tested per round (performance limit)
+
+**Technical Specification:**
+```python
+# Arena Tournament Flow
+class ArenaTournament:
+    def run_round(self, variations, test_input, weights):
+        # 1. Execute all variations
+        outputs = []
+        for var in variations:
+            output = execute_skill(var, test_input)
+            outputs.append({
+                "variation": var.id,
+                "output": output,
+                "exec_time": elapsed,
+                "tokens": token_count
+            })
+
+        # 2. Pairwise comparisons
+        comparisons = []
+        for i, out_a in enumerate(outputs):
+            for j, out_b in enumerate(outputs[i+1:]):
+                # Randomize order to mitigate position bias
+                order = random.choice(['AB', 'BA'])
+
+                result = judge.compare(
+                    output_a=out_a if order=='AB' else out_b,
+                    output_b=out_b if order=='AB' else out_a,
+                    weights=weights,
+                    criteria=["completeness", "clarity", "quality", "efficiency"]
+                )
+
+                comparisons.append({
+                    "pair": (out_a.variation, out_b.variation),
+                    "winner": result.winner,
+                    "reasoning": result.reasoning,
+                    "scores": result.dimensional_scores
+                })
+
+        # 3. Bradley-Terry ranking
+        rankings = bradley_terry_rank(comparisons)
+
+        # 4. Select winner
+        winner = rankings[0]
+
+        return {
+            "winner": winner,
+            "rankings": rankings,
+            "outputs": outputs,
+            "comparisons": comparisons
+        }
+```
+
+**Task Breakdown:**
+- Implement skill variation generator: Large (10h)
+- Build skill execution sandbox: Medium (8h)
+- Create output capture system: Small (4h)
+- Implement pairwise judge with bias mitigation: Large (12h)
+- Add Bradley-Terry ranking: Medium (6h)
+- Build tournament loop: Medium (8h)
+- Write comprehensive tests: Large (10h)
+
+**Dependencies:** R7 (Skill Execution), R8 (LLM-as-Judge), Arena-Lite algorithm
+
+---
+
+#### REQ-006: LLM-as-Judge with Weighted Evaluation
+
+**Description:** Separate judge model evaluates real skill outputs using user-configured weighted criteria with chain-of-thought reasoning.
+
+**Acceptance Criteria:**
+- [ ] Judge model separate from skill execution model (avoid self-evaluation bias)
+- [ ] Judges real outputs, NOT skill code
+- [ ] Weighted dimensions: Completeness, Clarity, Quality, Efficiency (user-configured)
+- [ ] Dimensional scores (0-100) with evidence from outputs
+- [ ] Chain-of-thought reasoning before final score
+- [ ] Anti-verbosity instructions (prefer concise accurate answers)
+- [ ] Position bias mitigation (randomize output order)
+- [ ] Store detailed reasoning for user review
+- [ ] Integration with agentic weighting (failure mode analysis)
+
+**Technical Specification:**
+```typescript
+interface JudgeRequest {
+  outputA: string;
+  outputB: string;
+  weights: {
+    completeness: number;  // 0-100, sums to 100
+    clarity: number;
+    quality: number;
+    efficiency: number;
+  };
+  domain: string;
+}
+
+interface JudgeResponse {
+  winner: "A" | "B" | "TIE";
+  reasoning: string;  // Chain-of-thought
+  dimensionalScores: {
+    A: { completeness: number; clarity: number; quality: number; efficiency: number; };
+    B: { completeness: number; clarity: number; quality: number; efficiency: number; };
+  };
+  overallScores: {
+    A: number;  // Weighted average
+    B: number;
+  };
+  evidence: {
+    dimension: string;
+    winner: "A" | "B";
+    example: string;  // Specific quote from output
+  }[];
+}
+
+// Example Judge Prompt
+const judgePrompt = `
+You are an expert evaluator for ${domain} skills.
+
+Compare these two outputs for the task: "${testInput}"
+
+Output A:
+${outputA}
+
+Output B:
+${outputB}
+
+Evaluation Criteria (weights):
+- Completeness (${weights.completeness}%): Does it address all requirements?
+- Clarity (${weights.clarity}%): Is it clear and understandable?
+- Quality (${weights.quality}%): Is it high quality and detailed?
+- Efficiency (${weights.efficiency}%): Is it concise without unnecessary verbosity?
+
+IMPORTANT: Prefer concise, accurate answers over verbose ones.
+
+Step 1: Analyze each dimension
+[Your chain-of-thought reasoning]
+
+Step 2: Score each dimension (0-100)
+[Dimensional scores with evidence]
+
+Step 3: Calculate weighted overall score
+[Final scores]
+
+Winner: [A/B/TIE]
+`;
+```
+
+**Task Breakdown:**
+- Implement separate judge model call: Small (3h)
+- Build weighted evaluation logic: Medium (6h)
+- Add chain-of-thought prompting: Small (4h)
+- Implement dimensional scoring: Medium (6h)
+- Add evidence extraction: Medium (5h)
+- Build position bias mitigation: Small (3h)
+- Write judge accuracy tests: Medium (8h)
+
+**Dependencies:** LLM API access (Opus for judging), agentic weighting integration
+
+---
+
+#### REQ-007: Convergence Detection (Multi-Criteria)
+
+**Description:** Stop arena when ANY stopping condition met: score plateau, time limit, iteration limit, or target achieved.
+
+**Acceptance Criteria:**
+- [ ] Score plateau: Improvement < 2% for 3 consecutive rounds
+- [ ] Time limit: Elapsed time > MAX_TIME (adaptive: 10min simple, 25min moderate, 45min complex)
+- [ ] Iteration limit: Rounds > MAX_ROUNDS (adaptive: 3-10 based on complexity)
+- [ ] Target achieved: Score >= TARGET_SCORE (e.g., 95/100)
+- [ ] User interruption: Manual stop by user
+- [ ] Log convergence reason for transparency
+- [ ] Early stopping prevents wasted computation
+
+**Technical Specification:**
+```python
+class ConvergenceDetector:
+    def should_stop(self, history, elapsed, complexity):
+        # Adaptive limits based on complexity
+        MAX_TIME = {1-3: 10, 4-7: 25, 8-10: 45}[complexity]  # minutes
+        MAX_ROUNDS = {1-3: 3, 4-7: 5, 8-10: 10}[complexity]
+        TARGET_SCORE = 95
+
+        # Criterion 1: Score plateau
+        if len(history) >= 3:
+            recent = history[-3:]
+            improvements = [
+                recent[i].score - recent[i-1].score
+                for i in range(1, 3)
+            ]
+            if all(imp < 0.02 for imp in improvements):
+                return True, "Score plateau: < 2% improvement for 3 rounds"
+
+        # Criterion 2: Time limit
+        if elapsed > MAX_TIME * 60:
+            return True, f"Time limit reached ({MAX_TIME} min)"
+
+        # Criterion 3: Iteration limit
+        if len(history) >= MAX_ROUNDS:
+            return True, f"Max rounds reached ({MAX_ROUNDS})"
+
+        # Criterion 4: Target achieved
+        if history[-1].score >= TARGET_SCORE:
+            return True, f"Target score achieved ({TARGET_SCORE})"
+
+        # Criterion 5: User interruption (checked elsewhere)
+
+        return False, None
+```
+
+**Task Breakdown:**
+- Implement multi-criteria convergence logic: Medium (6h)
+- Add adaptive limits based on complexity: Small (3h)
+- Build user interruption handling: Small (3h)
+- Add convergence logging: Small (2h)
+- Write convergence tests: Small (4h)
+
+**Dependencies:** R14 (Adaptive Complexity)
+
+---
+
+#### REQ-008: Pinecone Collective Database Integration
+
+**Description:** HTTP API for skill storage, search, and feedback with Pinecone vector database backend (no MCP installation required).
+
+**Acceptance Criteria:**
+- [ ] Direct HTTP API calls via Bash curl (no MCP to install)
+- [ ] POST /api/collective/search - Query for matching skills
+- [ ] POST /api/collective/submit - Submit winning skill
+- [ ] POST /api/collective/feedback - Submit user rating
+- [ ] GET /api/collective/leaderboard - Top skills by domain
+- [ ] Pinecone schema includes: skill_id, embedding (1536-dim), metadata (scores, weights, feedback, lineage, ELO, usage stats)
+- [ ] Authentication for submissions (API key)
+- [ ] Rate limiting (100 requests/min per user)
+- [ ] Response time < 2s for search queries
+
+**Technical Specification:**
+```typescript
+// Pinecone Schema
+interface SkillVector {
+  id: string;  // skill_id
+  values: number[];  // 1536-dim embedding
+  metadata: {
+    // Basic info
+    name: string;
+    domain: string;
+    skill_content: string;  // Full SKILL.md
+
+    // Dimensional scores
+    scores: {
+      completeness: number;
+      clarity: number;
+      quality: number;
+      efficiency: number;
+      overall: number;
+    };
+
+    // Impact-based weights used
+    weights: {
+      completeness: number;
+      clarity: number;
+      quality: number;
+      efficiency: number;
+      reasoning: string;
+    };
+
+    // Real-world user feedback
+    feedback: {
+      avg_rating: number;
+      total_ratings: number;
+      success_rate: number;
+      recent_comments: string[];
+    };
+
+    // Lineage & evolution
+    generation: number;
+    parent_id: string;
+    improvement_pct: number;
+
+    // Rankings
+    elo_rating: number;
+    leaderboard_rank: number;
+
+    // Usage stats
+    usage_count: number;
+    last_used: string;  // ISO 8601
+
+    // Test data
+    test_scenarios: string[];
+    arena_results_url: string;
+  };
+}
+
+// API Endpoints
+POST /api/collective/search
+{
+  "query": "comprehensive PRD skill",
+  "domain": "prd-generation",
+  "top_k": 5
+}
+→ { "matches": [...], "queryTime": 1.2 }
+
+POST /api/collective/submit
+{
+  "skill_content": "...",
+  "scores": {...},
+  "weights": {...},
+  "parent_id": "uuid-123",
+  "test_scenarios": [...]
+}
+→ { "skillId": "uuid-456", "rank": 12 }
+
+POST /api/collective/feedback
+{
+  "skill_id": "uuid-456",
+  "rating": 5,
+  "comment": "Excellent PRD generation",
+  "success": true
+}
+→ { "updated": true }
+
+GET /api/collective/leaderboard?domain=prd-generation&limit=10
+→ { "skills": [...], "lastUpdated": "2025-01-22T10:00:00Z" }
+```
+
+**Task Breakdown:**
+- Set up Pinecone database and schema: Medium (6h)
+- Implement POST /search endpoint: Medium (8h)
+- Implement POST /submit endpoint: Medium (8h)
+- Implement POST /feedback endpoint: Small (4h)
+- Implement GET /leaderboard endpoint: Small (4h)
+- Add authentication and rate limiting: Medium (6h)
+- Write API integration tests: Medium (8h)
+
+**Dependencies:** Pinecone account, OpenAI embeddings API
+
+---
+
+### Should Have (P1) - Important for Full Experience
+
+#### REQ-009: User-in-the-Loop Validation
+
+**Description:** Store arena results locally, allow users to review outputs and score them manually for validation and feedback.
+
+**Acceptance Criteria:**
+- [ ] All arena results stored in `.claude/skills/[skill-name]/arena_results/`
+- [ ] Format: JSON with inputs, outputs, scores, judge reasoning
+- [ ] User can browse results after arena completes
+- [ ] Side-by-side output comparison UI
+- [ ] User can score each output (1-5 stars)
+- [ ] User scores submitted to database (opt-in)
+- [ ] Feedback improves future arena weights
+
+**Task Breakdown:**
+- Create local storage system: Small (4h)
+- Build results browsing UI: Medium (8h)
+- Implement comparison view: Medium (6h)
+- Add user scoring interface: Small (4h)
+- Build feedback submission: Small (3h)
+- Write validation tests: Small (4h)
+
+**Dependencies:** R12 (User-in-the-Loop Validation), Pinecone feedback API
+
+---
+
+#### REQ-010: Adaptive Tournament Sizing
+
+**Description:** Tournament size (variations, rounds) adapts to skill complexity for optimal time/quality trade-off.
+
+**Acceptance Criteria:**
+- [ ] Simple skills (1-3): 3 variations, 3 rounds, ~10 min
+- [ ] Moderate skills (4-7): 5 variations, 5 rounds, ~25 min
+- [ ] Complex skills (8-10): 7 variations, 7 rounds, ~45 min
+- [ ] Complexity auto-detected from user request and domain
+- [ ] User can override default sizing
+- [ ] Adaptive convergence limits (time, iterations)
+
+**Task Breakdown:**
+- Implement complexity detection: Medium (6h)
+- Build adaptive sizing logic: Small (4h)
+- Add user override option: Small (2h)
+- Write adaptive tests: Small (4h)
+
+**Dependencies:** R14 (Adaptive Complexity)
+
+---
+
+#### REQ-011: Skill Lineage Tracking
+
+**Description:** Track skill evolution over time (parent → child relationships, improvement percentages, generation numbers).
+
+**Acceptance Criteria:**
+- [ ] Each skill stores parent_id in metadata
+- [ ] Generation number auto-increments (parent.gen + 1)
+- [ ] Improvement percentage calculated vs parent
+- [ ] Can trace ancestry (skill → parent → grandparent → ...)
+- [ ] Lineage displayed in search results and leaderboard
+
+**Task Breakdown:**
+- Add lineage fields to schema: Small (2h)
+- Implement ancestry tracking: Small (4h)
+- Build lineage display UI: Small (4h)
+- Write lineage tests: Small (3h)
+
+**Dependencies:** Pinecone schema update
+
+---
+
+### Nice to Have (P2) - Future Enhancement
+
+#### REQ-012: Test Scenario Evolution Across Rounds
+
+**Description:** Tests get progressively harder across arena rounds (simple → complex → edge case).
+
+**Acceptance Criteria:**
+- [ ] Round 1: Simple, common use case
+- [ ] Round 2: Complex, multi-part scenario
+- [ ] Round 3: Edge case, unusual constraints
+- [ ] Winner must excel at all difficulty levels
+
+**Task Breakdown:**
+- Implement scenario difficulty progression: Medium (6h)
+- Write evolution tests: Small (4h)
+
+**Dependencies:** R5 (Realistic Test Data)
+
+---
+
+#### REQ-013: Multi-Model Judge Ensemble
+
+**Description:** Use multiple judge models (e.g., Opus + Sonnet) and aggregate results for more reliable judgments.
+
+**Acceptance Criteria:**
+- [ ] Run same comparison with 2+ judge models
+- [ ] Aggregate results (majority vote or average scores)
+- [ ] Higher confidence when judges agree
+- [ ] Flag for human review when judges disagree
+
+**Task Breakdown:**
+- Implement multi-model judging: Medium (8h)
+- Build aggregation logic: Small (4h)
+- Add disagreement detection: Small (3h)
+
+**Dependencies:** Access to multiple LLM models
+
+---
+
+## Non-Functional Requirements
+
+### Performance
+
+**Response Time:**
+- Database search queries: < 2 seconds (p95)
+- Base skill (v0.1) generation: < 30 seconds total
+- Arena completion: < 30 minutes for moderate skills (p95)
+- LLM judge comparison: < 10 seconds per pairwise comparison
+
+**Throughput:**
+- Support 100 concurrent users creating skills
+- Database can handle 1000 queries/hour
+- Arena can run 10 background jobs concurrently
+
+**Resource Usage:**
+- Local storage: < 100MB per skill (arena results)
+- Memory: < 2GB for background arena process
+- Network: Minimize API calls (batch where possible)
+
+---
+
+### Security
+
+**Authentication:**
+- API key required for database submissions
+- User ID anonymized in database (hash)
+- No PII stored in collective database
+
+**Data Protection:**
+- Skill content public (stored in database)
+- User inputs hashed (not stored in plaintext)
+- Output samples truncated (500 chars max)
+- Local arena results private (stored on user machine)
+
+**Privacy:**
+- Opt-in for database submission
+- Opt-in for feedback collection
+- YAML flag: `collect-feedback: true/false`
+- Can disable via setting
+
+---
+
+### Scalability
+
+**User Load:**
+- Support 1000 active users in first 3 months
+- Scale to 10,000 users within 1 year
+- Serverless API (auto-scales)
+
+**Database Volume:**
+- Initial: 100 skills
+- Growth: 100-500 skills/month
+- Storage: 1GB vector database initially
+- Pinecone free tier: 1 index, 100k vectors (sufficient for MVP)
+
+**Arena Jobs:**
+- 10 concurrent background jobs per user machine
+- Each job runs 10-45 min
+- State persistence allows resume
+
+---
+
+### Reliability
+
+**Uptime:**
+- API SLA: 99% monthly uptime
+- Graceful degradation if database unavailable (create skills without search)
+- Resume capability if arena interrupted
+
+**Error Handling:**
+- Retry logic for API failures (3 retries with exponential backoff)
+- Timeout for skill execution (5 min max per skill)
+- Fallback to defaults if web search fails
+
+**Monitoring:**
+- Track arena completion rate
+- Alert on high failure rate (> 10%)
+- Log all convergence reasons
+
+---
+
+### Compatibility
+
+**Claude Code Version:**
+- Requires Claude Code with Task tool support
+- Compatible with latest stable release
+
+**System Requirements:**
+- Linux, macOS, Windows (WSL2)
+- Internet connection for API calls
+- Disk space: 1GB free for arena results
+
+**Dependencies:**
+- Bash (for curl API calls)
+- No MCP installation required
+- No additional software needed
+
+---
+
+## Technical Considerations
+
+### System Architecture
+
+**Current Architecture:**
+Claude Code skill system with:
+- Skills stored in `~/.claude/skills/[skill-name]/SKILL.md`
+- YAML frontmatter for metadata
+- Activation via triggers in user messages
+- No optimization or validation currently
+
+**Proposed Architecture:**
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│                    Claude Code (User Machine)                │
+│                                                               │
+│  ┌──────────────────────────────────────────────────────┐   │
+│  │  skill_creating Skill (Enhanced)                      │   │
+│  │  ┌────────────────────────────────────────────────┐  │   │
+│  │  │  1. Database-First Query                       │  │   │
+│  │  │     └─> Pinecone Search API                    │  │   │
+│  │  └────────────────────────────────────────────────┘  │   │
+│  │  ┌────────────────────────────────────────────────┐  │   │
+│  │  │  2. Quick Base Generation (v0.1)               │  │   │
+│  │  │     ├─> Pattern Research (awesome-claude)      │  │   │
+│  │  │     ├─> Domain Research (WebSearch)            │  │   │
+│  │  │     ├─> Question Generator Agent               │  │   │
+│  │  │     └─> Base Skill Template                    │  │   │
+│  │  └────────────────────────────────────────────────┘  │   │
+│  │  ┌────────────────────────────────────────────────┐  │   │
+│  │  │  3. Background Arena (via Task tool)           │  │   │
+│  │  │                                                 │  │   │
+│  │  │     Orchestrator Agent                         │  │   │
+│  │  │       ├─> Deep Research Agent                  │  │   │
+│  │  │       ├─> Test Data Generator Agent            │  │   │
+│  │  │       │     └─> WebSearch (realistic examples) │  │   │
+│  │  │       ├─> Variation Generator Agent            │  │   │
+│  │  │       ├─> Execution Workers (parallel)         │  │   │
+│  │  │       │     ├─ Worker A: Execute Skill A       │  │   │
+│  │  │       │     ├─ Worker B: Execute Skill B       │  │   │
+│  │  │       │     └─ Worker C: Execute Skill C       │  │   │
+│  │  │       ├─> Judge Agent (LLM-as-judge)           │  │   │
+│  │  │       │     └─> Opus (separate from exec)      │  │   │
+│  │  │       └─> Synthesis Agent (Bradley-Terry)      │  │   │
+│  │  │                                                 │  │   │
+│  │  │     State Persistence: Local JSON              │  │   │
+│  │  │     Arena Results: .claude/skills/.../arena... │  │   │
+│  │  └────────────────────────────────────────────────┘  │   │
+│  │  ┌────────────────────────────────────────────────┐  │   │
+│  │  │  4. User Validation & Submission               │  │   │
+│  │  │     ├─> Review UI (compare outputs)            │  │   │
+│  │  │     ├─> User Scoring (1-5 stars)               │  │   │
+│  │  │     └─> Submit to Collective (opt-in)          │  │   │
+│  │  └────────────────────────────────────────────────┘  │   │
+│  └──────────────────────────────────────────────────────┘   │
+└─────────────────────────────────────────────────────────────┘
+                            │
+                            │ HTTP API (curl)
+                            ▼
+┌─────────────────────────────────────────────────────────────┐
+│                   Collective API (Serverless)                │
+│                                                               │
+│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐      │
+│  │ POST /search │  │ POST /submit │  │ POST /feedback│      │
+│  └──────┬───────┘  └──────┬───────┘  └──────┬────────┘      │
+│         │                  │                  │               │
+│         └──────────────────┼──────────────────┘               │
+│                            ▼                                  │
+│  ┌────────────────────────────────────────────────────┐      │
+│  │         Pinecone Vector Database                   │      │
+│  │  ┌──────────────────────────────────────────────┐ │      │
+│  │  │  Index: claude-skills                        │ │      │
+│  │  │  - Embeddings (1536-dim)                     │ │      │
+│  │  │  - Metadata (scores, weights, feedback, ELO) │ │      │
+│  │  │  - Semantic search                           │ │      │
+│  │  └──────────────────────────────────────────────┘ │      │
+│  └────────────────────────────────────────────────────┘      │
+│                                                               │
+│  ┌──────────────┐  ┌──────────────┐                          │
+│  │ GET /leaderboard│ │ Auth & Rate  │                        │
+│  │              │  │ Limiting     │                          │
+│  └──────────────┘  └──────────────┘                          │
+└─────────────────────────────────────────────────────────────┘
+                            │
+                            │ OpenAI API
+                            ▼
+                  ┌──────────────────┐
+                  │ OpenAI Embeddings│
+                  │ text-embed-3-small│
+                  └──────────────────┘
+```
+
+**Key Components:**
+
+1. **skill_creating Skill (Enhanced):**
+   - Main entry point for user requests
+   - Orchestrates entire workflow
+   - Uses Task tool for background jobs
+
+2. **Agentic Workers:**
+   - **Question Generator:** Domain-specific questions
+   - **Research Agents:** Quick (30s) + Deep (background)
+   - **Test Data Generator:** Realistic scenarios via web search
+   - **Variation Generator:** Creates competing skill variations
+   - **Execution Workers:** Run skills in isolation (parallel)
+   - **Judge Agent:** LLM-as-judge with weighted criteria
+   - **Synthesis Agent:** Bradley-Terry ranking, convergence detection
+
+3. **Collective API:**
+   - Serverless HTTP API (no MCP required)
+   - Pinecone vector database backend
+   - Authentication, rate limiting
+   - Leaderboard and feedback system
+
+4. **Local Storage:**
+   - Arena results: `.claude/skills/[skill-name]/arena_results/`
+   - State persistence for resume capability
+   - User can review and validate
+
+---
+
+### API Specifications
+
+See REQ-008 for detailed API specs.
+
+**Key Endpoints:**
+- `POST /api/collective/search` - Query skills
+- `POST /api/collective/submit` - Submit skill
+- `POST /api/collective/feedback` - Submit rating
+- `GET /api/collective/leaderboard` - Top skills
+
+**Authentication:**
+```bash
+curl -X POST https://api.collective.claude-skills.ai/search \
+  -H "Authorization: Bearer ${API_KEY}" \
+  -H "Content-Type: application/json" \
+  -d '{"query": "PRD skill", "domain": "prd-generation"}'
+```
+
+---
+
+### Technology Stack
+
+**Frontend (User-Facing):**
+- Claude Code CLI interface
+- Text-based UI for questions, results, comparison
+- Markdown formatting for output
+
+**Backend (Skill Logic):**
+- Language: Integrated with Claude Code (no separate backend)
+- Agentic orchestration: Claude Code Task tool
+- State management: Local JSON files
+- LLM calls: Anthropic API (Sonnet for execution, Opus for judging)
+
+**Database:**
+- Vector DB: Pinecone (free tier for MVP)
+- Embeddings: OpenAI text-embedding-3-small (1536 dimensions)
+- Storage format: JSON metadata + vector embeddings
+
+**Infrastructure:**
+- Execution: User's machine (local)
+- API: Serverless (Vercel, AWS Lambda, or Cloudflare Workers)
+- No additional installation required (uses existing Claude Code)
+
+**External Dependencies:**
+- WebSearch tool (Claude Code built-in)
+- WebFetch tool (Claude Code built-in)
+- Anthropic API (Claude Sonnet, Opus)
+- OpenAI Embeddings API
+- Pinecone API
+
+---
+
+### External Dependencies
+
+**Third-Party Services:**
+
+1. **Pinecone Vector Database:**
+   - Purpose: Collective skill storage and semantic search
+   - API: https://docs.pinecone.io/
+   - Rate Limits: 100 requests/second (free tier)
+   - Fallback: If down, create skills without search (graceful degradation)
+   - Cost: Free tier (1 index, 100k vectors)
+
+2. **OpenAI Embeddings API:**
+   - Purpose: Generate 1536-dim embeddings for semantic search
+   - Model: text-embedding-3-small
+   - Rate Limits: 3000 requests/min
+   - Fallback: Cache embeddings, retry on failure
+   - Cost: $0.02 per 1M tokens (~$0.0001 per skill)
+
+3. **Anthropic API:**
+   - Purpose: LLM calls for skill execution and judging
+   - Models: Sonnet (execution), Opus (judging)
+   - Rate Limits: Per user API key
+   - Fallback: Use Sonnet for judging if Opus unavailable
+   - Cost: User pays via their own API key
+
+**Internal Dependencies:**
+- **Claude Code Task tool:** Required for background execution
+- **WebSearch tool:** For realistic test scenario discovery
+- **WebFetch tool:** For pattern research (awesome-claude-skills)
+- **Bash tool:** For curl API calls to collective database
+
+---
+
+### Migration Strategy
+
+**For Existing skill_creating Skill:**
+
+1. **Phase 1: Enhance Existing Skill**
+   - Add database-first query to SKILL.md workflow
+   - Implement quick base generation (v0.1)
+   - No breaking changes to current functionality
+
+2. **Phase 2: Add Background Arena**
+   - Implement arena orchestration via Task tool
+   - Runs after v0.1 delivery (non-blocking)
+   - Opt-in initially (user can skip arena)
+
+3. **Phase 3: Collective Database Integration**
+   - Deploy serverless API
+   - Set up Pinecone database
+   - Enable submissions
+
+4. **Phase 4: Gradual Feature Rollout**
+   - Week 1-2: Database search only
+   - Week 3-4: Base generation + optional arena
+   - Week 5-6: Full arena with convergence
+   - Week 7-8: User validation and feedback
+
+5. **Phase 5: Optimize Based on Usage**
+   - Monitor arena completion rates
+   - Adjust convergence criteria
+   - Improve test data generation
+
+**Rollback Plan:**
+- Disable database queries (fall back to current behavior)
+- Skip arena optimization (deploy v0.1 only)
+- Feature flags control each component independently
+
+---
+
+### Testing Strategy
+
+**Unit Tests:**
+- Test coverage: > 80% for new code
+- Key areas:
+  - Requirement fingerprint generation
+  - Question generation by domain
+  - Skill variation creation
+  - Pairwise comparison logic
+  - Bradley-Terry ranking
+  - Convergence detection
+  - API request/response handling
+
+**Integration Tests:**
+- Full workflow tests:
+  - Database query → results display → user selection
+  - Quick research → base skill → deployment
+  - Arena orchestration → workers → judge → synthesis
+  - Local storage → user review → feedback submission
+
+**E2E Tests:**
+- User journeys:
+  - New user creates PRD skill (finds existing, chooses it)
+  - User builds custom skill (v0.1 → arena → v1.0)
+  - User reviews arena outputs and scores them
+  - User submits winning skill to collective
+  - Another user discovers submitted skill via search
+
+**Performance Tests:**
+- Base skill generation completes < 30s (95th percentile)
+- Database query completes < 2s (95th percentile)
+- Arena completes < 30 min for moderate skills (95th percentile)
+- API endpoints respond < 2s (95th percentile)
+
+**Quality Tests:**
+- Realistic test scenarios validated by domain experts
+- Judge consistency: Same comparison run 3x should agree ≥ 80%
+- Score improvement: v1.0 beats v0.1 in ≥ 80% of cases
+- User satisfaction: ≥ 4.5/5 average rating
+
+**Security Tests:**
+- API authentication required for submissions
+- Rate limiting prevents abuse
+- No PII leaked in database
+- Anonymous user IDs cannot be reversed
+
+---
+
+## Implementation Roadmap
+
+### Phase 1: Foundation (Weeks 1-3)
+
+**Goal:** Database setup, basic API, requirement extraction, quick research
+
+**Tasks:**
+
+- [ ] **Task 1.1:** Set up Pinecone database and schema
+  - Complexity: Medium (6h)
+  - Dependencies: None
+  - Owner: Backend team
+  - Deliverable: Pinecone index created, schema documented
+
+- [ ] **Task 1.2:** Implement OpenAI embeddings generation
+  - Complexity: Small (3h)
+  - Dependencies: Task 1.1
+  - Owner: Backend team
+  - Deliverable: Function to generate 1536-dim vectors
+
+- [ ] **Task 1.3:** Build POST /search API endpoint
+  - Complexity: Medium (8h)
+  - Dependencies: Tasks 1.1, 1.2
+  - Owner: Backend team
+  - Deliverable: Working semantic search API
+
+- [ ] **Task 1.4:** Implement requirement extraction and fingerprinting
+  - Complexity: Medium (6h)
+  - Dependencies: None
+  - Owner: Agent team
+  - Deliverable: Parse user requests into structured requirements
+
+- [ ] **Task 1.5:** Build quick pattern research (awesome-claude-skills scan)
+  - Complexity: Small (4h)
+  - Dependencies: None
+  - Owner: Agent team
+  - Deliverable: Quick scan returns relevant patterns in <10s
+
+- [ ] **Task 1.6:** Build quick domain research (WebSearch integration)
+  - Complexity: Medium (6h)
+  - Dependencies: None
+  - Owner: Agent team
+  - Deliverable: Web search returns best practices in <15s
+
+**Validation Checkpoint:**
+- [ ] Can query database and get relevant results in < 2s
+- [ ] Can extract requirements from user request
+- [ ] Quick research completes in < 20s total
+- [ ] Unit tests passing for all components
+
+**Total Effort:** ~33 hours (~1.5 weeks with 2-person team)
+
+---
+
+### Phase 2: Quick Base Generation (Weeks 4-5)
+
+**Goal:** Question generation, base skill creation, v0.1 deployment
+
+**Tasks:**
+
+- [ ] **Task 2.1:** Build question generator agent
+  - Complexity: Medium (8h)
+  - Dependencies: Phase 1 complete (uses quick research)
+  - Owner: Agent team
+  - Deliverable: Generates 3-7 domain-specific questions
+
+- [ ] **Task 2.2:** Implement answer-to-weight conversion
+  - Complexity: Medium (5h)
+  - Dependencies: Task 2.1
+  - Owner: Agent team
+  - Deliverable: User answers → weighted criteria (e.g., Quality: 64%)
+
+- [ ] **Task 2.3:** Create base skill template engine
+  - Complexity: Medium (6h)
+  - Dependencies: Quick research, questions
+  - Owner: Agent team
+  - Deliverable: Generates SKILL.md from inputs
+
+- [ ] **Task 2.4:** Implement skill deployment automation
+  - Complexity: Small (3h)
+  - Dependencies: Task 2.3
+  - Owner: Agent team
+  - Deliverable: Writes SKILL.md to `.claude/skills/[name]/`
+
+- [ ] **Task 2.5:** Build database-first workflow integration
+  - Complexity: Medium (8h)
+  - Dependencies: Task 1.3, Task 2.4
+  - Owner: Integration team
+  - Deliverable: Full flow: query → show results → user select → deploy
+
+- [ ] **Task 2.6:** Add optional quick scoring of v0.1
+  - Complexity: Small (4h)
+  - Dependencies: Task 2.4
+  - Owner: Agent team
+  - Deliverable: User can score v0.1 immediately (optional)
+
+**Validation Checkpoint:**
+- [ ] Full base generation completes in < 30s (excluding user answer time)
+- [ ] Generated v0.1 skill is valid and activates correctly
+- [ ] Database query → user selection → deployment works end-to-end
+- [ ] Integration tests passing
+
+**Total Effort:** ~34 hours (~1.5 weeks)
+
+---
+
+### Phase 3: Arena Core (Weeks 6-8)
+
+**Goal:** Background orchestration, skill execution, realistic test data, pairwise judge
+
+**Tasks:**
+
+- [ ] **Task 3.1:** Implement orchestrator-worker pattern via Task tool
+  - Complexity: Large (12h)
+  - Dependencies: None (new subsystem)
+  - Owner: Agent team
+  - Deliverable: Orchestrator manages arena rounds in background
+
+- [ ] **Task 3.2:** Build skill variation generator agent
+  - Complexity: Large (10h)
+  - Dependencies: Base skill template
+  - Owner: Agent team
+  - Deliverable: Creates 3 variations (A, B, C) from base
+
+- [ ] **Task 3.3:** Build realistic test data generator agent
+  - Complexity: Medium (8h)
+  - Dependencies: WebSearch tool
+  - Owner: Agent team
+  - Deliverable: Generates realistic scenarios using personas
+
+- [ ] **Task 3.4:** Implement skill execution sandbox
+  - Complexity: Medium (8h)
+  - Dependencies: None
+  - Owner: Execution team
+  - Deliverable: Isolated skill execution with timeout
+
+- [ ] **Task 3.5:** Build output capture system
+  - Complexity: Small (4h)
+  - Dependencies: Task 3.4
+  - Owner: Execution team
+  - Deliverable: Captures full skill outputs (no truncation)
+
+- [ ] **Task 3.6:** Implement LLM-as-judge with pairwise comparison
+  - Complexity: Large (12h)
+  - Dependencies: Weighted criteria from questions
+  - Owner: Judge team
+  - Deliverable: Compares 2 outputs, returns winner with reasoning
+
+- [ ] **Task 3.7:** Add position bias mitigation (randomize order)
+  - Complexity: Small (3h)
+  - Dependencies: Task 3.6
+  - Owner: Judge team
+  - Deliverable: Multiple comparisons with order flipped
+
+- [ ] **Task 3.8:** Implement Bradley-Terry ranking
+  - Complexity: Medium (6h)
+  - Dependencies: Task 3.6
+  - Owner: Judge team
+  - Deliverable: Ranks variations from pairwise results
+
+**Validation Checkpoint:**
+- [ ] Can generate 3 variations from base skill
+- [ ] Can execute all variations with same test input
+- [ ] Judge compares outputs and selects winner
+- [ ] Ranking produces consistent results
+- [ ] Background orchestration runs without blocking user
+
+**Total Effort:** ~63 hours (~3 weeks)
+
+---
+
+### Phase 4: Convergence & Iteration (Weeks 9-10)
+
+**Goal:** Multi-round tournaments, convergence detection, state persistence
+
+**Tasks:**
+
+- [ ] **Task 4.1:** Build tournament iteration loop
+  - Complexity: Medium (8h)
+  - Dependencies: Phase 3 complete
+  - Owner: Orchestrator team
+  - Deliverable: Winner advances to next round vs 2 new variations
+
+- [ ] **Task 4.2:** Implement convergence detection (multi-criteria)
+  - Complexity: Medium (6h)
+  - Dependencies: Tournament loop
+  - Owner: Orchestrator team
+  - Deliverable: Stops when score plateau, time limit, or target met
+
+- [ ] **Task 4.3:** Add adaptive tournament sizing
+  - Complexity: Small (4h)
+  - Dependencies: Complexity detection
+  - Owner: Orchestrator team
+  - Deliverable: Simple (3 rounds), Moderate (5), Complex (7)
+
+- [ ] **Task 4.4:** Build state persistence system
+  - Complexity: Medium (6h)
+  - Dependencies: Tournament loop
+  - Owner: Orchestrator team
+  - Deliverable: Can resume arena if interrupted
+
+- [ ] **Task 4.5:** Implement graceful interruption handling
+  - Complexity: Small (4h)
+  - Dependencies: State persistence
+  - Owner: Orchestrator team
+  - Deliverable: User can stop arena, resume later
+
+- [ ] **Task 4.6:** Build arena completion notification
+  - Complexity: Small (3h)
+  - Dependencies: Convergence detection
+  - Owner: UI team
+  - Deliverable: User notified when v1.0 ready
+
+**Validation Checkpoint:**
+- [ ] Arena runs multiple rounds until convergence
+- [ ] Convergence criteria working correctly (no infinite loops)
+- [ ] Can interrupt and resume arena
+- [ ] Notification appears when complete
+- [ ] Arena completes in < 30 min for moderate skills
+
+**Total Effort:** ~31 hours (~1.5 weeks)
+
+---
+
+### Phase 5: User Validation & Feedback (Weeks 11-12)
+
+**Goal:** Local storage, review UI, user scoring, feedback collection
+
+**Tasks:**
+
+- [ ] **Task 5.1:** Create local arena results storage
+  - Complexity: Small (4h)
+  - Dependencies: Output capture
+  - Owner: Storage team
+  - Deliverable: JSON files in `.claude/skills/.../arena_results/`
+
+- [ ] **Task 5.2:** Build results browsing UI
+  - Complexity: Medium (8h)
+  - Dependencies: Local storage
+  - Owner: UI team
+  - Deliverable: User can view all round results
+
+- [ ] **Task 5.3:** Implement side-by-side output comparison
+  - Complexity: Medium (6h)
+  - Dependencies: Results browsing
+  - Owner: UI team
+  - Deliverable: Compare outputs from different variations
+
+- [ ] **Task 5.4:** Add user scoring interface (1-5 stars)
+  - Complexity: Small (4h)
+  - Dependencies: Comparison UI
+  - Owner: UI team
+  - Deliverable: User can rate each output
+
+- [ ] **Task 5.5:** Build version comparison (v0.1 vs v1.0)
+  - Complexity: Medium (6h)
+  - Dependencies: Arena results
+  - Owner: UI team
+  - Deliverable: Shows improvement metrics (+15 points, etc.)
+
+- [ ] **Task 5.6:** Implement feedback submission to database
+  - Complexity: Small (3h)
+  - Dependencies: User scoring, Pinecone API
+  - Owner: API team
+  - Deliverable: POST /feedback endpoint working
+
+**Validation Checkpoint:**
+- [ ] Arena results stored locally
+- [ ] User can review and compare outputs
+- [ ] User can score outputs and submit feedback
+- [ ] Feedback appears in database
+- [ ] UI is clear and usable
+
+**Total Effort:** ~31 hours (~1.5 weeks)
+
+---
+
+### Phase 6: Collective Database Features (Weeks 13-14)
+
+**Goal:** Skill submission, leaderboards, lineage tracking
+
+**Tasks:**
+
+- [ ] **Task 6.1:** Implement POST /submit API endpoint
+  - Complexity: Medium (8h)
+  - Dependencies: Pinecone database
+  - Owner: API team
+  - Deliverable: Can submit skills to collective
+
+- [ ] **Task 6.2:** Build submission UI with privacy controls
+  - Complexity: Medium (6h)
+  - Dependencies: Arena completion
+  - Owner: UI team
+  - Deliverable: User prompted to submit, opt-in, privacy clear
+
+- [ ] **Task 6.3:** Add champion comparison logic
+  - Complexity: Small (3h)
+  - Dependencies: Database query, arena scores
+  - Owner: API team
+  - Deliverable: Detect when user skill beats database champions
+
+- [ ] **Task 6.4:** Implement GET /leaderboard endpoint
+  - Complexity: Small (4h)
+  - Dependencies: Pinecone database
+  - Owner: API team
+  - Deliverable: Returns top skills by domain
+
+- [ ] **Task 6.5:** Build leaderboard display UI
+  - Complexity: Medium (6h)
+  - Dependencies: Leaderboard API
+  - Owner: UI team
+  - Deliverable: Shows top skills with scores, ratings
+
+- [ ] **Task 6.6:** Add skill lineage tracking
+  - Complexity: Small (4h)
+  - Dependencies: Submission API
+  - Owner: API team
+  - Deliverable: parent_id, generation, improvement_pct stored
+
+- [ ] **Task 6.7:** Implement ELO rating calculation
+  - Complexity: Medium (5h)
+  - Dependencies: Usage data, feedback
+  - Owner: API team
+  - Deliverable: Skills have ELO ratings that update
+
+**Validation Checkpoint:**
+- [ ] Can submit skills to collective
+- [ ] Leaderboard shows top skills accurately
+- [ ] Lineage tracking works (can trace ancestry)
+- [ ] ELO ratings update based on usage
+- [ ] Privacy controls working (anonymized data)
+
+**Total Effort:** ~36 hours (~1.5 weeks)
+
+---
+
+### Phase 7: Testing & Polish (Weeks 15-16)
+
+**Goal:** Comprehensive testing, bug fixes, performance optimization, documentation
+
+**Tasks:**
+
+- [ ] **Task 7.1:** Write comprehensive unit tests
+  - Complexity: Large (16h)
+  - Dependencies: All features implemented
+  - Owner: QA team
+  - Deliverable: > 80% code coverage
+
+- [ ] **Task 7.2:** Write integration tests
+  - Complexity: Large (12h)
+  - Dependencies: All features implemented
+  - Owner: QA team
+  - Deliverable: All workflows tested end-to-end
+
+- [ ] **Task 7.3:** Write E2E tests (user journeys)
+  - Complexity: Medium (10h)
+  - Dependencies: All features implemented
+  - Owner: QA team
+  - Deliverable: Key user journeys automated
+
+- [ ] **Task 7.4:** Performance testing and optimization
+  - Complexity: Medium (8h)
+  - Dependencies: All features implemented
+  - Owner: Performance team
+  - Deliverable: Meet all performance targets (< 30s base, < 30min arena, etc.)
+
+- [ ] **Task 7.5:** Bug fixes from testing
+  - Complexity: Variable (16h estimated)
+  - Dependencies: Tests written
+  - Owner: All teams
+  - Deliverable: All critical bugs fixed, P1 bugs addressed
+
+- [ ] **Task 7.6:** Write user documentation
+  - Complexity: Medium (8h)
+  - Dependencies: All features implemented
+  - Owner: Docs team
+  - Deliverable: README, usage guide, troubleshooting
+
+- [ ] **Task 7.7:** Create example arena results
+  - Complexity: Small (4h)
+  - Dependencies: Arena working
+  - Owner: Docs team
+  - Deliverable: Example outputs for documentation
+
+**Validation Checkpoint:**
+- [ ] All tests passing (unit, integration, E2E)
+- [ ] Performance benchmarks met
+- [ ] Zero critical bugs, minimal P1 bugs
+- [ ] Documentation complete and clear
+- [ ] System ready for production
+
+**Total Effort:** ~74 hours (~3.5 weeks)
+
+---
+
+### Phase 8: Deployment & Rollout (Weeks 17-18)
+
+**Goal:** Deploy to production, gradual rollout, monitoring, iteration
+
+**Tasks:**
+
+- [ ] **Task 8.1:** Deploy serverless API to production
+  - Complexity: Small (4h)
+  - Dependencies: API tested
+  - Owner: DevOps team
+  - Deliverable: API live at production URL
+
+- [ ] **Task 8.2:** Set up Pinecone production database
+  - Complexity: Small (2h)
+  - Dependencies: Schema finalized
+  - Owner: DevOps team
+  - Deliverable: Production Pinecone index ready
+
+- [ ] **Task 8.3:** Deploy enhanced skill_creating skill
+  - Complexity: Small (2h)
+  - Dependencies: All features tested
+  - Owner: DevOps team
+  - Deliverable: Updated SKILL.md deployed
+
+- [ ] **Task 8.4:** Set up monitoring and alerting
+  - Complexity: Small (4h)
+  - Dependencies: API deployed
+  - Owner: DevOps team
+  - Deliverable: Track API uptime, error rates, arena completion rates
+
+- [ ] **Task 8.5:** Gradual rollout (beta users first)
+  - Complexity: Small (2h)
+  - Dependencies: Monitoring set up
+  - Owner: Product team
+  - Deliverable: 10 beta users test system
+
+- [ ] **Task 8.6:** Gather feedback from beta users
+  - Complexity: Medium (8h)
+  - Dependencies: Beta rollout
+  - Owner: Product team
+  - Deliverable: Feedback collected, prioritized
+
+- [ ] **Task 8.7:** Iterate based on feedback
+  - Complexity: Medium (8h)
+  - Dependencies: Feedback gathered
+  - Owner: All teams
+  - Deliverable: Top 3-5 improvements implemented
+
+- [ ] **Task 8.8:** Full public launch
+  - Complexity: Small (2h)
+  - Dependencies: Beta successful
+  - Owner: Product team
+  - Deliverable: Announcement, full availability
+
+**Validation Checkpoint:**
+- [ ] API stable and responding correctly
+- [ ] Beta users successfully creating and optimizing skills
+- [ ] Monitoring shows healthy metrics
+- [ ] Feedback incorporated
+- [ ] Ready for public launch
+
+**Total Effort:** ~32 hours (~1.5 weeks)
+
+---
+
+### Task Dependencies Visualization
+
+```
+Phase 1 (Foundation):
+  1.1 (Pinecone) → 1.2 (Embeddings) → 1.3 (Search API)
+  1.4 (Requirements) ──────────────────┐
+  1.5 (Pattern Research) ──────────────┤
+  1.6 (Domain Research) ───────────────┴─→ Phase 2
+
+Phase 2 (Base Generation):
+  [Phase 1] → 2.1 (Questions) → 2.2 (Weights) → 2.3 (Template) → 2.4 (Deploy)
+  1.3 (Search) ────────────────────────────────────────┐
+  2.4 (Deploy) ────────────────────────────────────────┴─→ 2.5 (Workflow)
+  2.4 (Deploy) → 2.6 (Quick Scoring)
+
+Phase 3 (Arena Core):
+  3.1 (Orchestrator) ──────────────────────────────────┐
+  3.2 (Variation Gen) ─────────────────────────────────┤
+  3.3 (Test Data) ─────────────────────────────────────┤
+  3.4 (Execution) → 3.5 (Output Capture) ──────────────┼─→ 3.8 (Ranking)
+  3.6 (Judge) → 3.7 (Bias Mitigation) ─────────────────┘
+
+Phase 4 (Convergence):
+  [Phase 3] → 4.1 (Tournament Loop) → 4.2 (Convergence) → 4.6 (Notification)
+  4.1 → 4.3 (Adaptive Sizing)
+  4.1 → 4.4 (State Persist) → 4.5 (Interruption)
+
+Phase 5 (User Validation):
+  3.5 (Output Capture) → 5.1 (Local Storage) → 5.2 (Browse UI)
+  5.2 → 5.3 (Comparison) → 5.4 (User Scoring) → 5.6 (Feedback API)
+  4.6 (Notification) → 5.5 (Version Compare)
+
+Phase 6 (Collective):
+  1.1 (Pinecone) ──────────────────────────────────────┐
+  4.2 (Convergence) → 6.1 (Submit API) → 6.2 (Submit UI)│
+  6.1 → 6.3 (Champion Compare) ───────────────────────┤
+  6.1 → 6.4 (Leaderboard API) → 6.5 (Leaderboard UI) ─┤
+  6.1 → 6.6 (Lineage) ─────────────────────────────────┤
+  6.1 → 6.7 (ELO) ─────────────────────────────────────┘
+
+Phase 7 (Testing):
+  [All Phases] → 7.1 (Unit) ─┐
+                 7.2 (Integration) ─┼→ 7.5 (Bug Fixes) → 7.6 (Docs)
+                 7.3 (E2E) ─┘       → 7.4 (Performance) → 7.7 (Examples)
+
+Phase 8 (Deployment):
+  7.5 (Bug Fixes) → 8.1 (API Deploy) ─┐
+  7.5 → 8.2 (Pinecone Prod) ──────────┼→ 8.4 (Monitoring) → 8.5 (Beta)
+  7.5 → 8.3 (Skill Deploy) ───────────┘
+  8.5 → 8.6 (Feedback) → 8.7 (Iterate) → 8.8 (Launch)
+
+Critical Path:
+  1.1 → 1.2 → 1.3 → 2.5 → 3.1 → 3.6 → 4.1 → 4.2 → 5.1 → 6.1 → 7.5 → 8.8
+```
+
+---
+
+### Effort Estimation
+
+**Total Estimated Effort by Phase:**
+- Phase 1 (Foundation): 33 hours
+- Phase 2 (Base Generation): 34 hours
+- Phase 3 (Arena Core): 63 hours
+- Phase 4 (Convergence): 31 hours
+- Phase 5 (User Validation): 31 hours
+- Phase 6 (Collective): 36 hours
+- Phase 7 (Testing): 74 hours
+- Phase 8 (Deployment): 32 hours
+
+**Total: ~334 hours**
+
+**With 2-person team:**
+- ~167 hours per person
+- ~21 weeks at 8 hours/week
+- **~5 months calendar time**
+
+**With 3-person team:**
+- ~111 hours per person
+- ~14 weeks at 8 hours/week
+- **~3.5 months calendar time**
+
+**Risk Buffer:** +25% (84 hours) for unknowns, integration challenges, and iteration
+
+**Final Estimate with Buffer:**
+- 2-person team: **~6-7 months**
+- 3-person team: **~4-5 months**
+
+**MVP (Phases 1-5 only):**
+- Total: 192 hours
+- 2-person team: ~3 months
+- 3-person team: ~2 months
+
+---
+
+## Out of Scope
+
+Explicitly NOT included in this release:
+
+### 1. Server Farm Optimization (Future Phase 2)
+
+**What:** Centralized server farms running continuous global tournaments of all submitted skills to find absolute best across all users.
+
+**Why Out of Scope:**
+- Requires significant infrastructure ($$)
+- MVP focuses on local execution (user pays own tokens)
+- Can be added later without changing architecture
+- Future enhancement: 24/7 deep research and optimization
+
+**Future Consideration:** Document for Phase 2, 6-12 months post-launch
+
+---
+
+### 2. Multi-Language Skill Support
+
+**What:** Skills that work across multiple programming languages or natural languages.
+
+**Why Out of Scope:**
+- MVP focuses on English, code-agnostic skills
+- Adds complexity to question generation and judging
+- Limited user demand in initial research
+
+**Future Consideration:** Add if international users request it
+
+---
+
+### 3. Skill Marketplace / Monetization
+
+**What:** Paid skills, premium collective access, skill authors earning revenue.
+
+**Why Out of Scope:**
+- Collective is free and open initially
+- Monetization complicates launch
+- Focus on quality and adoption first
+
+**Future Consideration:** Evaluate after 100+ skills and 1000+ users
+
+---
+
+### 4. IDE Integration Beyond Claude Code
+
+**What:** VS Code extension, JetBrains plugin, etc.
+
+**Why Out of Scope:**
+- MVP is Claude Code only
+- Different architecture for each IDE
+- Limited resources
+
+**Future Consideration:** Partner integrations post-launch
+
+---
+
+### 5. Advanced Judge Models (GPT-4, Gemini, etc.)
+
+**What:** Support for non-Anthropic judge models.
+
+**Why Out of Scope:**
+- Anthropic models (Sonnet, Opus) sufficient for MVP
+- Cross-platform adds complexity
+- Focus on one provider initially
+
+**Future Consideration:** Add multi-model support if users request it
+
+---
+
+### 6. Real-Time Collaboration on Skills
+
+**What:** Multiple users co-creating skills simultaneously.
+
+**Why Out of Scope:**
+- MVP is single-user workflow
+- Requires collaborative editing infrastructure
+- Not in initial requirements
+
+**Future Consideration:** If teams request it
+
+---
+
+### 7. Automated Skill Maintenance
+
+**What:** System automatically updates skills when dependencies change or new best practices emerge.
+
+**Why Out of Scope:**
+- Requires continuous monitoring
+- Risk of breaking working skills
+- Manual updates sufficient for MVP
+
+**Future Consideration:** Auto-suggest updates, user approves
+
+---
+
+### 8. Skill Analytics Dashboard
+
+**What:** Detailed analytics on skill usage, performance, user demographics.
+
+**Why Out of Scope:**
+- Basic metrics sufficient for MVP (ratings, usage count)
+- Privacy concerns with detailed tracking
+- Focus on core functionality first
+
+**Future Consideration:** Opt-in analytics for skill authors
+
+---
+
+## Open Questions & Risks
+
+### Open Questions
+
+#### Q1: What judge model should we use?
+
+**Current Status:** Considering Opus (highest quality) vs Sonnet (faster, cheaper)
+
+**Options:**
+- **A)** Always use Opus for judging (consistent, highest quality)
+- **B)** Haiku for early rounds, Opus for finals (optimized cost)
+- **C)** User chooses in questions (flexibility)
+
+**Owner:** Technical lead
+
+**Deadline:** End of Phase 2 (before arena implementation)
+
+**Impact:** Medium (affects arena runtime and cost)
+
+**Recommendation:** Option B (progressive) - balances quality and cost. Early rounds don't need Opus-level precision, finals do.
+
+---
+
+#### Q2: How many test scenarios per round?
+
+**Current Status:** Considering 1 vs 3 scenarios per round
+
+**Options:**
+- **A)** 1 scenario per round (faster, less reliable)
+- **B)** 3 scenarios per round (slower, more reliable)
+- **C)** Adaptive based on skill complexity (1 for simple, 3 for complex)
+
+**Owner:** Arena architect
+
+**Deadline:** End of Phase 3 (before convergence testing)
+
+**Impact:** High (affects arena reliability and runtime)
+
+**Recommendation:** Option C (adaptive) - simple skills don't need extensive testing, complex skills do.
+
+---
+
+#### Q3: Should we allow user override of convergence criteria?
+
+**Current Status:** System auto-detects convergence
+
+**Options:**
+- **A)** Fully automatic (no user control)
+- **B)** User can set max time/rounds (simple override)
+- **C)** User can configure all criteria (full control)
+
+**Owner:** Product team
+
+**Deadline:** End of Phase 4 (convergence implementation)
+
+**Impact:** Low (nice-to-have, not critical)
+
+**Recommendation:** Option B (simple override) - most users trust defaults, power users want time control.
+
+---
+
+#### Q4: How should we handle skill activation conflicts?
+
+**Current Status:** Multiple skills might activate on same trigger
+
+**Options:**
+- **A)** First match wins (simple, may not be best)
+- **B)** Highest-scored skill wins (quality-focused)
+- **C)** User prompted to choose (manual, slower)
+
+**Owner:** skill_creating maintainer
+
+**Deadline:** End of Phase 2 (affects base generation)
+
+**Impact:** Medium (affects user experience)
+
+**Recommendation:** Option B (highest-scored) - leverages arena scores for automatic quality selection.
+
+---
+
+#### Q5: Should database submissions be moderated?
+
+**Current Status:** Considering auto-accept vs review queue
+
+**Options:**
+- **A)** Auto-accept all submissions (fast, risk of spam)
+- **B)** Auto-scan for dangerous patterns, flag suspicious (balanced)
+- **C)** Manual review all submissions (slow, thorough)
+
+**Owner:** Security team
+
+**Deadline:** End of Phase 6 (before collective launch)
+
+**Impact:** High (affects collective quality and safety)
+
+**Recommendation:** Option B (auto-scan + flag) - prevents obvious abuse, human review for edge cases.
+
+---
+
+### Risks & Mitigation
+
+| Risk | Likelihood | Impact | Severity | Mitigation | Contingency |
+|------|------------|--------|----------|------------|-------------|
+| Arena takes longer than 30 min target | Medium | Medium | **Medium** | Adaptive tournament sizing (simple skills fewer rounds), early convergence detection, user can interrupt | User can keep using v0.1 if arena takes too long, improve convergence criteria based on data |
+| Test data not realistic enough | Medium | High | **High** | Web search for current examples, persona-based generation, evolution refinement, user validation scoring | Cache proven realistic scenarios, allow users to provide examples, improve generation prompts |
+| Judge produces inconsistent results | Medium | High | **High** | Multiple comparisons with order randomization, chain-of-thought required, user review override, aggregate multiple calls | Flag inconsistent judgments for human review, use ensemble of judges, tune prompts |
+| User machine offline during arena | Low | Medium | **Low** | State persistence (save progress), resume capability, graceful degradation (partial results usable) | Allow arena restart from last checkpoint, warn users before starting |
+| Pinecone API down or rate limited | Low | High | **Medium** | Graceful degradation (create skills without search), retry logic with exponential backoff, cache common queries | Fall back to local-only mode, queue submissions for later |
+| Malicious skill submissions | Low | Critical | **High** | Automated scanning for dangerous patterns (rm -rf, curl to unknown domains), review queue for flagged skills, rate limiting per user, community flagging | Manual review, blocklist, user reputation system |
+| High API costs (OpenAI embeddings) | Medium | Low | **Low** | Cache embeddings aggressively, batch requests, use efficient models (text-embed-3-small), minimal re-embedding | User pays own API costs, optimize caching strategy |
+| Low skill submission rate | Medium | Medium | **Medium** | Encourage submissions (special notification if beats champion), gamification (leaderboards), showcase benefits (others using your skill) | Pre-populate database with high-quality seed skills, improve submission UX |
+| User confusion about v0.1 vs v1.0 | Medium | Low | **Low** | Clear labeling (v0.1 "Quick Base", v1.0 "Arena Optimized"), show improvement metrics (+15 points), user can test both | Improve UI copy, add tooltips, documentation |
+
+---
+
+## Validation Checkpoints
+
+### Checkpoint 1: End of Phase 1 (Foundation)
+
+**Criteria:**
+- [ ] Pinecone database set up and queryable
+- [ ] Can generate embeddings and search semantically
+- [ ] Search returns relevant results in < 2s
+- [ ] Can extract requirements from user requests
+- [ ] Quick research (pattern + domain) completes in < 20s
+- [ ] Unit tests passing for all components
+
+**If Failed:**
+- Debug Pinecone integration
+- Optimize search query performance
+- Improve requirement extraction accuracy
+- Don't proceed to Phase 2 until stable
+
+---
+
+### Checkpoint 2: End of Phase 2 (Base Generation)
+
+**Criteria:**
+- [ ] Question generator produces relevant domain-specific questions
+- [ ] User answers convert to reasonable weights
+- [ ] Base skill (v0.1) generates in < 30s
+- [ ] v0.1 skill is valid SKILL.md and activates correctly
+- [ ] Database-first workflow works end-to-end (query → select → deploy)
+- [ ] Integration tests passing
+
+**If Failed:**
+- Refine question templates by domain
+- Fix skill template generation bugs
+- Improve database query UI
+- Don't proceed to Phase 3 until v0.1 generation reliable
+
+---
+
+### Checkpoint 3: End of Phase 3 (Arena Core)
+
+**Criteria:**
+- [ ] Can generate 3 variations from base skill
+- [ ] All variations execute with same test input
+- [ ] Test data is realistic (validated by spot-checking)
+- [ ] Judge compares outputs and selects winner with reasoning
+- [ ] Bradley-Terry ranking produces consistent results
+- [ ] Background orchestration runs without blocking user
+- [ ] Integration tests for arena passing
+
+**If Failed:**
+- Improve variation generator diversity
+- Enhance test data realism validation
+- Tune judge prompts for consistency
+- Don't proceed to Phase 4 until core arena reliable
+
+---
+
+### Checkpoint 4: End of Phase 4 (Convergence)
+
+**Criteria:**
+- [ ] Arena runs multiple rounds until convergence
+- [ ] Convergence detects score plateau correctly
+- [ ] Time and iteration limits prevent infinite loops
+- [ ] Can interrupt and resume arena from saved state
+- [ ] Arena completes in < 30 min for moderate skills (test with 5+ examples)
+- [ ] User receives notification when arena completes
+
+**If Failed:**
+- Adjust convergence thresholds (may need < 2% → < 3%)
+- Optimize arena performance (reduce round time)
+- Fix state persistence bugs
+- Don't proceed to Phase 5 until convergence robust
+
+---
+
+### Checkpoint 5: End of Phase 5 (User Validation)
+
+**Criteria:**
+- [ ] Arena results stored locally and readable
+- [ ] User can browse all round results
+- [ ] Side-by-side comparison shows outputs clearly
+- [ ] User can score outputs (1-5 stars)
+- [ ] Version comparison (v0.1 vs v1.0) shows improvement metrics
+- [ ] Feedback submission to database works
+
+**If Failed:**
+- Improve UI clarity and usability
+- Fix local storage bugs
+- Ensure feedback API working
+- Don't proceed to Phase 6 until user can validate effectively
+
+---
+
+### Checkpoint 6: End of Phase 6 (Collective)
+
+**Criteria:**
+- [ ] Can submit skills to collective database
+- [ ] Submissions include all required metadata (scores, weights, lineage)
+- [ ] Leaderboard shows top skills accurately
+- [ ] Lineage tracking works (can trace parent → child)
+- [ ] ELO ratings update based on usage
+- [ ] Privacy controls working (anonymized data, no PII)
+- [ ] Champion comparison detects when user skill beats database leaders
+
+**If Failed:**
+- Fix submission API bugs
+- Improve leaderboard ranking algorithm
+- Ensure privacy compliance
+- Don't proceed to Phase 7 until collective reliable
+
+---
+
+### Checkpoint 7: End of Phase 7 (Testing)
+
+**Criteria:**
+- [ ] Unit test coverage > 80%
+- [ ] All integration tests passing
+- [ ] E2E tests cover key user journeys
+- [ ] Performance benchmarks met:
+  - [ ] Base generation < 30s (p95)
+  - [ ] Database query < 2s (p95)
+  - [ ] Arena completion < 30 min moderate skills (p95)
+- [ ] Zero critical bugs (P0)
+- [ ] P1 bugs addressed or accepted as known issues
+- [ ] Documentation complete (README, usage guide, troubleshooting)
+
+**If Failed:**
+- Fix critical bugs before launch
+- Optimize performance to meet benchmarks
+- Complete documentation
+- Don't proceed to Phase 8 until quality gates met
+
+---
+
+### Checkpoint 8: Production Launch
+
+**Criteria (at each rollout stage):**
+
+**Beta (10 users):**
+- [ ] At least 5 skills created successfully
+- [ ] At least 2 arenas completed successfully
+- [ ] Error rate < 5% (acceptable for beta)
+- [ ] User feedback collected (survey or interviews)
+
+**If Beta Failed:**
+- Fix bugs discovered
+- Improve UX based on feedback
+- Iterate before wider rollout
+
+**Public Launch (all users):**
+- [ ] Beta successful with no critical issues
+- [ ] Monitoring shows healthy metrics (API uptime > 99%, error rate < 1%)
+- [ ] Database has at least 10 seed skills (pre-populated)
+- [ ] Documentation published and accessible
+- [ ] Support channel ready (GitHub issues, Discord, etc.)
+
+**If Public Launch Failed:**
+- Rollback to beta (disable for most users)
+- Fix issues
+- Re-launch when stable
+
+---
+
+## Appendix: Task Breakdown Hints
+
+### Suggested Taskmaster Task Structure
+
+**Phase 1: Foundation (10 tasks, ~33 hours)**
+1. Set up Pinecone database and schema (6h) - Dependencies: None
+2. Implement OpenAI embeddings generation (3h) - Dependencies: Task 1
+3. Build POST /search API endpoint (8h) - Dependencies: Tasks 1, 2
+4. Implement requirement extraction (6h) - Dependencies: None
+5. Build quick pattern research (4h) - Dependencies: None
+6. Build quick domain research via WebSearch (6h) - Dependencies: None
+
+**Phase 2: Base Generation (6 tasks, ~34 hours)**
+7. Build question generator agent (8h) - Dependencies: Tasks 5, 6
+8. Implement answer-to-weight conversion (5h) - Dependencies: Task 7
+9. Create base skill template engine (6h) - Dependencies: Tasks 5, 6, 7
+10. Implement skill deployment automation (3h) - Dependencies: Task 9
+11. Build database-first workflow integration (8h) - Dependencies: Tasks 3, 10
+12. Add optional quick scoring of v0.1 (4h) - Dependencies: Task 10
+
+**Phase 3: Arena Core (8 tasks, ~63 hours)**
+13. Implement orchestrator-worker pattern via Task tool (12h) - Dependencies: None
+14. Build skill variation generator agent (10h) - Dependencies: Task 9
+15. Build realistic test data generator with web search (8h) - Dependencies: None
+16. Implement skill execution sandbox (8h) - Dependencies: None
+17. Build output capture system (4h) - Dependencies: Task 16
+18. Implement LLM-as-judge with pairwise comparison (12h) - Dependencies: Task 8
+19. Add position bias mitigation (3h) - Dependencies: Task 18
+20. Implement Bradley-Terry ranking (6h) - Dependencies: Task 18
+
+**Phase 4: Convergence (6 tasks, ~31 hours)**
+21. Build tournament iteration loop (8h) - Dependencies: Phase 3 complete
+22. Implement convergence detection (6h) - Dependencies: Task 21
+23. Add adaptive tournament sizing (4h) - Dependencies: Task 21
+24. Build state persistence system (6h) - Dependencies: Task 21
+25. Implement graceful interruption handling (4h) - Dependencies: Task 24
+26. Build arena completion notification (3h) - Dependencies: Task 22
+
+**Phase 5: User Validation (6 tasks, ~31 hours)**
+27. Create local arena results storage (4h) - Dependencies: Task 17
+28. Build results browsing UI (8h) - Dependencies: Task 27
+29. Implement side-by-side output comparison (6h) - Dependencies: Task 28
+30. Add user scoring interface (4h) - Dependencies: Task 29
+31. Build version comparison (v0.1 vs v1.0) (6h) - Dependencies: Task 27
+32. Implement feedback submission to database (3h) - Dependencies: Task 30
+
+**Phase 6: Collective (7 tasks, ~36 hours)**
+33. Implement POST /submit API endpoint (8h) - Dependencies: Task 1
+34. Build submission UI with privacy controls (6h) - Dependencies: Task 26
+35. Add champion comparison logic (3h) - Dependencies: Tasks 3, 22
+36. Implement GET /leaderboard endpoint (4h) - Dependencies: Task 1
+37. Build leaderboard display UI (6h) - Dependencies: Task 36
+38. Add skill lineage tracking (4h) - Dependencies: Task 33
+39. Implement ELO rating calculation (5h) - Dependencies: Task 33
+
+**Phase 7: Testing (7 tasks, ~74 hours)**
+40. Write comprehensive unit tests (16h) - Dependencies: All features
+41. Write integration tests (12h) - Dependencies: All features
+42. Write E2E tests (10h) - Dependencies: All features
+43. Performance testing and optimization (8h) - Dependencies: All features
+44. Bug fixes from testing (16h) - Dependencies: Tasks 40-43
+45. Write user documentation (8h) - Dependencies: Task 44
+46. Create example arena results (4h) - Dependencies: Task 44
+
+**Phase 8: Deployment (8 tasks, ~32 hours)**
+47. Deploy serverless API to production (4h) - Dependencies: Task 44
+48. Set up Pinecone production database (2h) - Dependencies: Task 44
+49. Deploy enhanced skill_creating skill (2h) - Dependencies: Task 44
+50. Set up monitoring and alerting (4h) - Dependencies: Task 47
+51. Gradual rollout (beta users) (2h) - Dependencies: Task 50
+52. Gather feedback from beta users (8h) - Dependencies: Task 51
+53. Iterate based on feedback (8h) - Dependencies: Task 52
+54. Full public launch (2h) - Dependencies: Task 53
+
+**Total: 54 tasks, ~334 hours**
+
+---
+
+### Parallelizable Tasks
+
+**Can work in parallel:**
+
+**Phase 1:**
+- Tasks 1-2 (sequential), Tasks 4-6 (parallel)
+
+**Phase 2:**
+- Task 7 (after 5-6), Tasks 8-10 (mostly sequential), Task 11 (integrates), Task 12 (parallel with 11)
+
+**Phase 3:**
+- Tasks 13-15 (parallel), Tasks 16-17 (sequential), Tasks 18-20 (sequential)
+- Two sub-teams: Orchestration (13-15) and Execution+Judge (16-20)
+
+**Phase 4:**
+- Tasks 21-26 mostly sequential (each depends on previous)
+
+**Phase 5:**
+- Tasks 27-32 mostly sequential, but Task 31 parallel with 28-30
+
+**Phase 6:**
+- Tasks 33-39 parallel after Task 33 complete
+
+**Phase 7:**
+- Tasks 40-43 parallel, Task 44 after all tests, Tasks 45-46 parallel
+
+**Phase 8:**
+- Tasks 47-49 parallel, Task 50 after 47, Tasks 51-54 sequential
+
+---
+
+### Critical Path Tasks
+
+**Critical path (longest dependency chain):**
+1. Set up Pinecone (Task 1)
+2. Build embeddings (Task 2)
+3. Build search API (Task 3)
+4. Build workflow integration (Task 11)
+5. Implement orchestrator (Task 13)
+6. Implement judge (Task 18)
+7. Build tournament loop (Task 21)
+8. Implement convergence (Task 22)
+9. Create local storage (Task 27)
+10. Implement submit API (Task 33)
+11. Bug fixes (Task 44)
+12. Public launch (Task 54)
+
+**Critical path duration:** ~85 hours (~11 weeks at 8h/week with 1 person on critical path)
+
+**With parallel teams (3 people):**
+- Critical path person: 85 hours
+- Arena team: 63 hours (Phase 3)
+- UI/Validation team: 31 hours (Phase 5)
+- **Total calendar time: ~14 weeks (3.5 months)**
+
+---
+
+**End of PRD**
+
+*This PRD is optimized for taskmaster AI task generation. All requirements include task breakdown hints, complexity estimates, and dependency mapping to enable effective automated task planning.*
+
+**Ready for Implementation:** Yes
+**Next Steps:** Review PRD → Approve → Begin Phase 1 (Foundation) → taskmaster init → taskmaster generate