@tgoodington/intuition 9.2.2 → 9.3.0

package/docs/archive/WALDO_V3_COMPLETE_DOCUMENTATION.md

@@ -0,0 +1,538 @@
+# Waldo v3 - Complete Implementation Documentation
+
+**Version:** 3.0
+**Status:** Fully Implemented & Ready for Testing
+**Date:** February 4, 2025
+
+---
+
+## Executive Summary
+
+Waldo v3 is a complete redesign of the discovery skill from an interrogation model to a **research-informed thinking partnership** model. Users now have two dialogue modes (Guided and Open-Ended), and the system automatically builds a persistent user profile across all projects.
+
+Key improvements:
+- ✅ Research agents launch immediately (best practices, pitfalls, emerging patterns)
+- ✅ Wise confidant role (brings knowledge to conversation, not neutral)
+- ✅ "Yes, and..." collaboration (builds on ideas, doesn't negate)
+- ✅ 1-2 focused questions per turn (never a deluge)
+- ✅ Gentle steering based on research insights
+- ✅ Dual dialogue modes (user choice at start)
+- ✅ Persistent user profile (cross-project, naturally discovered)
+- ✅ All skills integrated with profile system
+
+---
+
+## Part 1: Waldo v3 Core Design
+
+### Philosophy Shift
+
+**From:** Interrogator asking structured questions
+**To:** Wise confidant with domain knowledge collaborating in dialogue
+
+### Core Principles
+
+1. **Research from the start** - Immediate parallel research (best practices, pitfalls, alternatives)
+2. **Wise confidant model** - Brings relevant knowledge; informs every question
+3. **Yes, and... building** - Collaboratively expand thinking, never negate
+4. **One or two questions** - Focused, never rapid-fire
+5. **Gentle steering** - Flag inefficient paths respectfully
+6. **Cross-sector capable** - Research adapts to any domain
+7. **Authentic collaboration** - Think together, not at user
+
+### Key Feature: Immediate Research Launch
+
+When user provides initial context, Waldo immediately delegates:
+- **Research Agent 1:** Best practices and standards in user's domain
+- **Research Agent 2:** Common pitfalls and inefficiencies
+- **Research Agent 3** (optional): Emerging patterns or alternatives
+
+All run in parallel (single message, multiple Task calls). While research completes, Waldo continues dialogue with increasing context.
+
+---
+
+## Part 2: Dual Dialogue Modes
+
+### Mode Selection
+
+**Timing:** First question after greeting
+**Stored In:** `.project-memory-state.json` > `discovery.dialogue_mode`
+
+### Guided Mode
+
+**Best For:** Users who like structure, clarity, options
+
+**How It Works:**
+- Every question uses AskUserQuestion
+- Offers 2-4 focused options per question
+- Always includes "Other" for custom input
+- Structured but fully flexible
+- User drives conversation through selections
+
+**Example:**
+```
+"Given what you're exploring, what matters most to you?"
+
+OPTIONS:
+- Getting to value quickly
+- Building it right
+- Efficiency / constraints
+- Experience
+- Other: [custom answer]
+```
+
+### Open-Ended Mode
+
+**Best For:** Users who prefer natural flow, freedom, spontaneity
+
+**How It Works:**
+- Asks conversational questions (no options)
+- User answers however they want
+- Natural rhythm and flow
+- Just as deep and thorough as Guided
+- Feels like thinking out loud together
+
+**Example:**
+```
+Waldo: "Given what you're exploring, what matters most to you right now?"
+
+User: "We need to move fast but can't break what's working. The board
+is waiting for something in 3 months."
+
+Waldo: "That's a real tension. Tell me more about the board pressure..."
+```
+
+### Both Modes
+
+**Produce identical outcomes:**
+- Same discovery depth and rigor
+- Same GAPP dimension coverage (Problem, Goals, User Context, Personalization)
+- Same research-informed insights
+- Same discovery brief and structured output
+- Difference is pure dialogue style, not substance
+
+**Mode can be changed:**
+- Anytime during session (just ask Waldo)
+- Different mode for different projects
+- Preference stored for next time
+
+---
+
+## Part 3: Persistent User Profile System
+
+### Overview
+
+`.claude/USER_PROFILE.json` is a **global, cross-project** file that captures who the user is professionally, not what they're doing on any specific project.
+
+**Characteristics:**
+- ✅ Lives in `.claude/` (global)
+- ✅ Persists across all projects
+- ✅ Naturally discovered through conversation (not filled out upfront)
+- ✅ Updated by handoff skill after each discovery
+- ✅ Read by Magellan (planning) and Faraday (execution)
+- ✅ Includes confidence scores for learnings
+- ✅ Tracks which projects informed each property
+
+### Profile Structure
+
+```json
+{
+  "user": {
+    "name": "...",
+    "role": "...",
+    "seniority_level": "junior/mid/senior/lead",
+    "years_experience": null,
+    "organization": {
+      "name": "...",
+      "type": "startup/enterprise/non-profit",
+      "industry": "EdTech/FinTech/etc.",
+      "location": "..."
+    },
+    "reports_to": "..."
+  },
+
+  "expertise": {
+    "primary_skills": ["Python", "React", "System Design"],
+    "expertise_areas": ["Backend Architecture", "Team Leadership"],
+    "learning_style": "hands-on/research-first/collaborative",
+    "learning_goals": [...]
+  },
+
+  "communication": {
+    "style": "direct/narrative/bullets",
+    "pace": "async/real-time/batched",
+    "detail_level": "high-level/comprehensive",
+    "decision_making": "data-driven/collaborative"
+  },
+
+  "constraints": {
+    "authority_level": "personal/team/enterprise",
+    "typical_availability": "part-time/full-time",
+    "team_size": "solo/small team/large team",
+    "time_zone": "...",
+    "technical_environment": {
+      "cloud_providers": ["AWS", "GCP"],
+      "preferred_databases": ["PostgreSQL"],
+      "deployment_patterns": ["microservices", "serverless"]
+    }
+  },
+
+  "motivation": {
+    "primary_drives": ["shipping", "learning", "quality"],
+    "cares_about": ["team culture", "technical excellence"],
+    "professional_goals": ["...")
+  },
+
+  "preferences": {
+    "tools_and_frameworks": ["React", "TypeScript", "PostgreSQL"],
+    "methodologies": ["TDD", "Agile"],
+    "collaboration_tools": ["GitHub", "Slack"]
+  },
+
+  "metadata": {
+    "created_at": "2025-02-04T...",
+    "last_updated": "...",
+    "profile_completeness": 0.0-1.0,
+    "source": "Discovered through agent conversations",
+    "projects_contributed_to": ["project1", "project2"],
+    "confidence_scores": {
+      "role": 0.9,
+      "expertise_areas": 0.8,
+      "communication_style": 0.7,
+      "motivation": 0.6
+    }
+  }
+}
+```
+
+### What Gets Stored vs. NOT Stored
+
+**PERSISTENT (Follows You):**
+- Role, seniority, experience
+- Organization context (name, type, industry, location)
+- Expertise and skills
+- Learning style and goals
+- Communication preferences
+- Decision-making approach
+- Motivation and what you care about
+- Authority level and typical constraints
+- Preferred tools and methodologies
+
+**NOT PERSISTENT (Project-Specific):**
+- ❌ Project goals
+- ❌ Project problems
+- ❌ Project constraints
+- ❌ Project decisions
+- ❌ Project scope
+
+---
+
+## Part 4: How Skills Integrate with User Profile
+
+### Initialize Skill
+
+**On Project Setup:**
+- Creates `.claude/USER_PROFILE.json` from template (if doesn't exist)
+- Explains that it's persistent and cross-project
+- Documents that agents will populate it naturally
+
+### Waldo (Discovery)
+
+**During Discovery:**
+- Greets warmly and asks for dialogue mode (Guided or Open-Ended)
+- Launches research agents immediately
+- Naturally learns about user through conversation
+- Documents findings in `discovery_output.json` > `user_profile_learnings`
+- Captures confidence levels for each discovery
+
+**What Waldo Learns:**
+- Role, seniority, years of experience
+- Organization (name, type, industry, location)
+- Expertise areas and primary skills
+- Learning style
+- Communication style, pace, detail preference
+- Decision-making approach
+- What drives/motivates them
+- Typical constraints and authority level
+- Technical environment they work in
+
+### Handoff Orchestrator
+
+**After Discovery → Planning Transition:**
+1. Reads `discovery_output.json` > `user_profile_learnings`
+2. Merges into `.claude/USER_PROFILE.json`:
+   - If field is `null` in profile and has value in learnings → add it
+   - If field is populated, only overwrite if discovery_confidence is "high"
+   - Update `last_updated` timestamp
+   - Track projects that contributed to profile
+   - Update confidence scores
+3. Save updated profile
+
+### Magellan (Planning)
+
+**On Startup:**
+- Reads `.claude/USER_PROFILE.json`
+- Understands user's role, expertise, decision-making style
+- Tailors planning depth and complexity to user's level
+- Adapts communication to user's preferences
+
+### Faraday (Execution)
+
+**On Startup:**
+- Reads `.claude/USER_PROFILE.json`
+- Understands user's authority level, team size, constraints
+- Personalizes execution communication (detailed vs. summary)
+- Makes delegation decisions that fit user's context
+
+### Start Skill
+
+**On Session Start:**
+- Could read `.claude/USER_PROFILE.json` for session context
+- Could mention relevant profile properties when generating briefs
+- (Implementation: optional enhancement)
+
+---
+
+## Part 5: Complete Discovery Workflow
+
+### Step-by-Step
+
+```
+1. User runs /intuition-discovery
+
+2. Waldo greets and asks: "Guided or Open-Ended mode?"
+   → User selects mode
+   → Stored in state.discovery.dialogue_mode
+
+3. Waldo asks for initial context
+   Guided: Offers options for how user wants to frame it
+   Open-Ended: "What's on your mind?"
+
+4. User describes situation
+   "I want to build X"
+   "We're struggling with Y"
+   "I have an idea about Z"
+
+5. Waldo identifies domain/mode and launches research (parallel)
+   - Research Agent 1: Best practices in domain
+   - Research Agent 2: Common pitfalls
+   - Research Agent 3: Emerging patterns
+
+6. While research runs, Waldo asks follow-up questions
+   Guided: Uses AskUserQuestion with options
+   Open-Ended: Conversational questions
+
+7. Research completes, Waldo has full context
+   (Best practices, pitfalls, alternatives, domain knowledge)
+
+8. Conversation continues in chosen mode
+   - 1-2 focused questions per turn
+   - Building on user's answers (yes, and...)
+   - Integrating research insights naturally
+   - Gleaning user profile information
+   - Exploring GAPP dimensions (Problem, Goals, Context, Motivation)
+
+9. Conversation reaches natural depth
+   - All GAPP dimensions explored
+   - Assumptions documented with confidence
+   - Mutual understanding achieved
+
+10. Waldo proposes formalization
+    "Ready to capture what we've learned?"
+    Guided: Uses AskUserQuestion
+    Open-Ended: Natural question
+
+11. User agrees → Waldo creates:
+    - discovery_brief.md (readable narrative)
+    - discovery_output.json (structured data + user_profile_learnings)
+
+12. Waldo directs to handoff
+    "Run /intuition-handoff to transition to planning"
+    (NOT directly to plan)
+
+13. Handoff orchestrator:
+    - Extracts user_profile_learnings
+    - Merges into .claude/USER_PROFILE.json
+    - Updates project memory
+    - Creates planning_brief.md for Magellan
+```
+
+---
+
+## Part 6: State Tracking
+
+### `.project-memory-state.json` Discovery Section
+
+```json
+{
+  "discovery": {
+    "status": "in_progress|complete",
+    "dialogue_mode": "guided|open-ended",
+    "started_at": "2025-02-04T14:30:00Z",
+    "completed_at": null,
+
+    "initial_context": {
+      "user_input": "I want to build...",
+      "timestamp": "2025-02-04T14:30:10Z"
+    },
+
+    "research_performed": [
+      {
+        "task_id": "research-001",
+        "topic": "e-commerce best practices",
+        "launched_at": "2025-02-04T14:30:20Z",
+        "completed_at": "2025-02-04T14:35:40Z",
+        "findings_summary": "Key practices: microservices, event-driven...",
+        "findings_full": "...",
+        "informed_questions": ["How are you thinking about scalability?", "...]
+      }
+    ],
+
+    "conversation_via_mode": {
+      "guided_questions": [
+        {
+          "turn": 1,
+          "question": "What do you want to explore?",
+          "options": ["Build new", "Stuck on problem", "Validate idea", "Other"],
+          "user_selected": "Build new",
+          "timestamp": "2025-02-04T14:30:10Z"
+        }
+      ],
+      "open_ended_questions": [
+        {
+          "turn": 1,
+          "question": "What's on your mind?",
+          "user_response": "I want to build...",
+          "timestamp": "2025-02-04T14:30:10Z"
+        }
+      ]
+    },
+
+    "gapp": {
+      "problem": {
+        "covered": true,
+        "insights": ["Core problem: X", "Root cause: Y"],
+        "confidence": "high"
+      },
+      "goals": { "covered": true, ... },
+      "ux_context": { "covered": false, ... },
+      "personalization": { "covered": false, ... }
+    },
+
+    "quality_score": {
+      "coverage": 0.75,
+      "depth": "medium",
+      "assumptions_documented": true,
+      "ready_for_formalization": false
+    }
+  }
+}
+```
+
+---
+
+## Part 7: Files Modified & Created
+
+### New Files Created
+- ✅ `.claude/USER_PROFILE.json` (persistent user profile)
+- ✅ `skills/intuition-initialize/references/user_profile_template.json` (template)
+
+### Files Updated
+
+**waldo_core.md:**
+- Complete redesign of philosophy and approach
+- Added "Gleaning User Profile Information" section
+- Documented both dialogue modes (Guided and Open-Ended)
+- Updated state tracking to include dialogue_mode
+- Updated complete flow diagram with mode selection
+- Enhanced discovery output schema
+
+**Magellan_core.md (planning):**
+- Added "Reading User Context at Startup" section
+- Documents how to read and use USER_PROFILE.json
+- Shows how to personalize planning based on user profile
+
+**Faraday_core.md (execution):**
+- Added "Reading User Context at Startup" section
+- Documents how to personalize execution based on user profile
+- Shows how to adapt communication and delegation
+
+**handoff_core.md:**
+- Added "User Profile Extraction & Updates" section
+- Documents how to extract user_profile_learnings from discovery
+- Shows merge logic for updating .claude/USER_PROFILE.json
+- Includes confidence scoring and project tracking
+
+**initialize SKILL.md:**
+- Added user profile setup documentation
+- Explains persistent vs. project-specific distinction
+- Documents template usage
+
+**SKILL.md (discovery):**
+- Updated description to emphasize "research-informed thinking partnership"
+- Added mode selection documentation
+- Updated "How to Start" section
+- Updated workflow diagram with mode selection
+- Clarified Guided vs. Open-Ended experience
+
+---
+
+## Part 8: Implementation Checklist
+
+- ✅ Waldo v3 redesign complete (research-informed, wise confidant)
+- ✅ Dual dialogue modes implemented (Guided and Open-Ended)
+- ✅ User profile system created (.claude/USER_PROFILE.json)
+- ✅ Profile template created (for initialize skill)
+- ✅ Waldo updated to glean profile information
+- ✅ Magellan updated to read and use profile
+- ✅ Faraday updated to read and use profile
+- ✅ Handoff updated to extract and merge profile
+- ✅ Initialize updated to create profile
+- ✅ State tracking includes dialogue_mode
+- ✅ Complete flow diagrams documented
+- ✅ Confidence scoring for profile properties
+- ✅ Project tracking for profile properties
+- ✅ All skills reinstalled and current
+
+---
+
+## Part 9: Ready for Testing
+
+The complete system is implemented and ready to test:
+
+1. **Start a discovery session** - Run `/intuition-discovery`
+2. **Choose dialogue mode** - Guided or Open-Ended
+3. **Describe what you're exploring** - Natural description
+4. **Let Waldo guide** - Research agents launch, wise confidant engages
+5. **Watch profile build** - Your context naturally discovered
+6. **Complete discovery** - Move to handoff and planning
+
+---
+
+## Future Enhancements
+
+Possible additions (not yet implemented):
+- Start skill reads and references USER_PROFILE.json
+- Profile completeness dashboard
+- Confidence-based prompts to refine uncertain properties
+- Cross-project pattern analysis (roles across projects)
+- Profile version history
+- Manual profile editing interface
+- Export/import profile for new tools
+
+---
+
+## Key Principles Maintained
+
+✅ **File-Based Architecture** - No APIs, all communication through files
+✅ **Tool-Agnostic** - Works across Claude Code, Cursor, Copilot, etc.
+✅ **Auditable & Transparent** - Everything is readable and tracked
+✅ **Memory Authority** - Clear ownership of which skill updates which files
+✅ **Project Memory Separate** - User profile is global, project memory stays in docs/project_notes/
+✅ **Incremental Discovery** - Profile builds over time, never regresses
+✅ **User Control** - Preference-driven, can switch modes, natural discovery
+
+---
+
+**Status:** Complete and ready for production testing
+**Date:** February 4, 2025
+**Version:** Waldo v3.0 with Dual Modes and Persistent User Profile