aidp 0.10.0 → 0.11.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 2358e7598866519630169447a6ba6fd60704571ca81afc8b6e9ec856916cacc1
-  data.tar.gz: a33aba24daef5e8492d36954b4a36c44fff738dc9b9a54762c81ac28708f2cfa
+  metadata.gz: ffba477d4ca2e297b4fe81d7ecb32ccf37d161df6bcd2dceaa91e94ccbc8ea8f
+  data.tar.gz: 851fdc47c7a12bd8f6f8304badf60ffdb6855787d0982645d80a309d9caee40b
 SHA512:
-  metadata.gz: c37341e9a7a26a5bb23141133f08a67f26f5a88dcf1d5e12b36f638359f6e738cb4ffd86e15b0d8cc1c8282ecea31803faa6b0ff82c55bb1d9b63aaf987cc82b
-  data.tar.gz: 8b78de9305399181898a47542880840487ab667ccb7e151c4ff52d497437a5fd780440844c3b2585d6356c91c25378c7e73b15eebb1fa8f5e321bb46d4a79773
+  metadata.gz: ef68650e5a8c55d8b219a00a42ba7fbff2c87bdd5cacf5f450c8bf406c481d6d1e4c00cd7edabbf41b07928bb28c2b5246c54392e67f52ee8381bac1668d9463
+  data.tar.gz: d2ae0a1bc041b0f8c0cdf1ccb3c328fc7bb1c4ba3cbb0599c983f4937056a8ad1f47e0bcb475a96dd8d4d936cfc6d7452a0333121ed36e29a14708b8e4bd15ec

data/README.md

@@ -1,6 +1,6 @@
 # AI Dev Pipeline (aidp) - Ruby Gem
 
-A portable CLI that automates a complete AI development workflow from idea to implementation using your existing IDE assistants. Now with **Enhanced TUI** - a rich terminal interface that runs complete workflows with intelligent provider management and error recovery.
+A portable CLI that automates AI development workflows from idea to implementation using your existing IDE assistants. Features autonomous work loops, background execution, and comprehensive progress tracking.
 
 ## Quick Start
 
@@ -11,13 +11,16 @@ gem install aidp
 # Navigate to your project
 cd /your/project
 
-# Start the interactive TUI (default)
-aidp
+# Start an interactive workflow
+aidp execute
+
+# Or run in background
+aidp execute --background
 ```
 
 ### First-Time Setup
 
-On the first run in a project without an `aidp.yml`, AIDP now launches a **First-Time Setup Wizard** instead of failing with a configuration error. You'll be prompted to choose one of:
+On the first run in a project without an `aidp.yml`, AIDP launches a **First-Time Setup Wizard**. You'll be prompted to choose one of:
 
 1. Minimal (single provider: cursor)
 2. Development template (multiple providers, safe defaults)
@@ -27,43 +30,156 @@ On the first run in a project without an `aidp.yml`, AIDP now launches a **First
 
 Non-interactive environments (CI, scripts, pipes) automatically receive a minimal `aidp.yml` so workflows can proceed without manual intervention.
 
-You can re-run the wizard manually by removing `aidp.yml` and starting `aidp` again.
+You can re-run the wizard manually:
+
+```bash
+aidp --setup-config
+```
+
+## Core Features
+
+### Work Loops
+
+AIDP implements **work loops** - an iterative execution pattern where AI agents autonomously work on tasks until completion, with automatic testing and linting feedback.
+
+- **Iterative refinement**: Agent works in loops until task is 100% complete
+- **Self-management**: Agent edits PROMPT.md to track its own progress
+- **Automatic validation**: Tests and linters run after each iteration
+- **Self-correction**: Only failures are fed back for the next iteration
+
+See [Work Loops Guide](docs/WORK_LOOPS_GUIDE.md) for details.
+
+### Background Execution
+
+Run workflows in the background while monitoring progress from separate terminals:
+
+```bash
+# Start in background
+aidp execute --background
+✓ Started background job: 20251005_235912_a1b2c3d4
+
+# Monitor progress
+aidp jobs list                        # List all jobs
+aidp jobs status <job_id>             # Show job status
+aidp jobs logs <job_id> --tail        # View recent logs
+aidp checkpoint summary --watch       # Watch metrics in real-time
+
+# Control jobs
+aidp jobs stop <job_id>               # Stop a running job
+```
+
+### Progress Checkpoints
 
-## Enhanced TUI
+Track code quality metrics and task progress throughout execution:
 
-AIDP features a rich terminal interface that transforms it from a step-by-step tool into an intelligent development assistant. The enhanced TUI provides beautiful, interactive terminal components while running complete workflows automatically.
+```bash
+# View current progress
+aidp checkpoint summary
+
+# Watch with auto-refresh
+aidp checkpoint summary --watch
+
+# View historical data
+aidp checkpoint history 20
+```
 
-### Features
+**Tracked Metrics:**
 
-- **🎨 Rich Terminal Interface**: Beautiful CLI UI components with progress bars, spinners, and frames
-- **📋 Interactive Navigation**: Hierarchical menu system with breadcrumb navigation
-- **⌨️ Keyboard Shortcuts**: Full keyboard navigation and control
-- **📊 Real-time Progress**: Live monitoring of progress and system status
-- **🔄 Workflow Control**: Pause, resume, cancel, and stop workflows with visual feedback
-- **💬 Smart Question Collection**: Interactive prompts with validation and error handling
+- Lines of code
+- Test coverage
+- Code quality scores
+- PRD task completion percentage
+- File count and growth trends
 
-### Usage
+## Command Reference
+
+### Execution Modes
 
 ```bash
-# Start the interactive TUI (default)
-aidp
+# Execute mode - Build new features
+aidp execute                    # Interactive workflow selection
+aidp execute --background       # Run in background
+aidp execute --background --follow  # Start and follow logs
+
+# Analyze mode - Analyze codebase
+aidp analyze                    # Interactive analysis
+aidp analyze --background       # Background analysis
+```
 
-# Show version information
-aidp --version
+### Job Management
+
+```bash
+# List all background jobs
+aidp jobs list
 
-# Show help information
-aidp --help
+# Show job status
+aidp jobs status <job_id>
+aidp jobs status <job_id> --follow  # Follow with auto-refresh
+
+# View job logs
+aidp jobs logs <job_id>
+aidp jobs logs <job_id> --tail      # Last 50 lines
+aidp jobs logs <job_id> --follow    # Stream in real-time
+
+# Stop a running job
+aidp jobs stop <job_id>
+```
+
+### Checkpoint Monitoring
+
+```bash
+# View latest checkpoint
+aidp checkpoint show
+
+# Progress summary with trends
+aidp checkpoint summary
+aidp checkpoint summary --watch             # Auto-refresh every 5s
+aidp checkpoint summary --watch --interval 10  # Custom interval
+
+# View checkpoint history
+aidp checkpoint history           # Last 10 checkpoints
+aidp checkpoint history 50        # Last 50 checkpoints
+
+# Detailed metrics
+aidp checkpoint metrics
+
+# Clear checkpoint data
+aidp checkpoint clear
+aidp checkpoint clear --force     # Skip confirmation
+```
+
+### System Commands
+
+```bash
+# Show system status
+aidp status
+
+# Provider health dashboard
+aidp providers
+
+# Harness state management
+aidp harness status
+aidp harness reset
+
+# Configuration
+aidp --setup-config             # Re-run setup wizard
+aidp --help                     # Show all commands
+aidp --version                  # Show version
 ```
 
 ## AI Providers
 
 AIDP intelligently manages multiple providers with automatic switching:
 
-- **Claude API** - Primary provider for complex analysis and code generation
-- **Gemini API** - Cost-effective fallback for general tasks
+- **Anthropic Claude CLI** - Primary provider for complex analysis and code generation
+- **Codex CLI** - OpenAI's Codex command-line interface for code generation
 - **Cursor CLI** - IDE-integrated provider for code-specific tasks
+- **Gemini CLI** - Google's Gemini command-line interface for general tasks
+- **GitHub Copilot CLI** - GitHub's AI pair programmer command-line interface
+- **macOS UI** - macOS-specific UI automation provider
+- **OpenCode** - Alternative open-source code generation provider
 
-The TUI automatically switches providers when:
+The system automatically switches providers when:
 
 - Rate limits are hit
 - Providers fail or timeout
@@ -74,6 +190,15 @@ The TUI automatically switches providers when:
 
 ```yaml
 # aidp.yml
+harness:
+  work_loop:
+    enabled: true
+    max_iterations: 50
+    test_commands:
+      - "bundle exec rspec"
+    lint_commands:
+      - "bundle exec standardrb"
+
 providers:
   claude:
     type: "usage_based"
@@ -190,6 +315,49 @@ When the AI creates a questions file, follow these steps:
 
 The questions file is only created when the AI needs additional information beyond what it can infer from your project structure and existing files. Your answers are preserved for future reference.
 
+## Workflow Examples
+
+### Standard Interactive Workflow
+
+```bash
+# Start execute mode
+aidp execute
+
+# Select workflow type (e.g., "Full PRD to Implementation")
+# Answer any questions interactively
+# Review generated files (PRD, architecture, etc.)
+# Workflow runs automatically with harness managing retries
+```
+
+### Background Workflow with Monitoring
+
+```bash
+# Terminal 1: Start background execution
+aidp execute --background
+✓ Started background job: 20251005_235912_a1b2c3d4
+
+# Terminal 2: Watch progress in real-time
+aidp checkpoint summary --watch
+
+# Terminal 3: Monitor job status
+aidp jobs status 20251005_235912_a1b2c3d4 --follow
+
+# Later: Check final results
+aidp checkpoint summary
+aidp jobs logs 20251005_235912_a1b2c3d4 --tail
+```
+
+### Quick Analysis
+
+```bash
+# Run analysis in background
+aidp analyze --background
+
+# Check progress
+aidp jobs list
+aidp checkpoint summary
+```
+
 ## Debug and Logging
 
 ```bash
@@ -247,7 +415,8 @@ See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup and conventional co
 
 For detailed information:
 
-- **[TUI User Guide](docs/TUI_USER_GUIDE.md)** - Complete guide to using the enhanced TUI
+- **[CLI User Guide](docs/CLI_USER_GUIDE.md)** - Complete guide to using AIDP commands
+- **[Work Loops Guide](docs/WORK_LOOPS_GUIDE.md)** - Iterative workflows with automatic validation
 - **[Configuration Guide](docs/harness-configuration.md)** - Detailed configuration options and examples
 - **[Troubleshooting Guide](docs/harness-troubleshooting.md)** - Common issues and solutions
 

data/lib/aidp/analyze/kb_inspector.rb

@@ -7,27 +7,14 @@ require "tty-prompt"
 module Aidp
   module Analyze
     class KBInspector
+      include Aidp::MessageDisplay
+
       def initialize(kb_dir = ".aidp/kb", prompt: TTY::Prompt.new)
         @kb_dir = File.expand_path(kb_dir)
         @prompt = prompt
         @data = load_kb_data
       end
 
-      # Helper method for consistent message display using TTY::Prompt
-      def display_message(message, type: :info)
-        color = case type
-        when :error then :red
-        when :success then :green
-        when :warning then :yellow
-        when :info then :blue
-        when :highlight then :cyan
-        when :muted then :bright_black
-        else :white
-        end
-
-        @prompt.say(message, color: color)
-      end
-
       def show(type, format: "summary")
         case type
         when "seams"

data/lib/aidp/analyze/progress.rb

@@ -11,7 +11,7 @@ module Aidp
 
       def initialize(project_dir)
         @project_dir = project_dir
-        @progress_file = File.join(project_dir, ".aidp-analyze-progress.yml")
+        @progress_file = File.join(project_dir, ".aidp", "progress", "analyze.yml")
         load_progress
       end
 
@@ -80,6 +80,7 @@ module Aidp
         # In test mode, skip file operations to avoid hanging
         return if ENV["RACK_ENV"] == "test" || defined?(RSpec)
 
+        FileUtils.mkdir_p(File.dirname(@progress_file))
         File.write(@progress_file, @progress.to_yaml)
       end
     end

data/lib/aidp/analyze/ruby_maat_integration.rb

@@ -8,6 +8,8 @@ require "fileutils"
 module Aidp
   module Analyze
     class RubyMaatIntegration
+      include Aidp::MessageDisplay
+
       def initialize(project_dir = Dir.pwd, prompt: TTY::Prompt.new)
         @project_dir = project_dir
         @prompt = prompt
@@ -455,21 +457,6 @@ module Aidp
 
         result.success? && !result.out.strip.empty?
       end
-
-      private
-
-      # Helper method for consistent message display using TTY::Prompt
-      def display_message(message, type: :info)
-        color = case type
-        when :error then :red
-        when :warn then :yellow
-        when :success then :green
-        when :highlight then :cyan
-        else :white
-        end
-
-        @prompt.say(message, color: color)
-      end
     end
   end
 end

data/lib/aidp/analyze/runner.rb

@@ -10,6 +10,7 @@ module Aidp
   module Analyze
     class Runner
       include Aidp::DebugMixin
+      include Aidp::MessageDisplay
 
       def initialize(project_dir, harness_runner = nil, prompt: TTY::Prompt.new)
         @project_dir = project_dir
@@ -25,18 +26,6 @@ module Aidp
 
       private
 
-      def display_message(message, type: :info)
-        color = case type
-        when :error then :red
-        when :success then :green
-        when :warning then :yellow
-        when :info then :blue
-        when :highlight then :cyan
-        else :white
-        end
-        @prompt.say(message, color: color)
-      end
-
       public
 
       def run_step(step_name, options = {})
@@ -60,7 +49,7 @@ module Aidp
       # Harness-aware step execution
       def run_step_with_harness(step_name, options = {})
         # Get current provider from harness
-        current_provider = @harness_runner.current_provider
+        current_provider = get_harness_provider_safely
         provider_type = current_provider || "cursor"
 
         debug_step(step_name, "Harness execution", {
@@ -198,7 +187,10 @@ module Aidp
 
       def template_search_paths
         [
-          File.join(@project_dir, "templates", "ANALYZE"),
+          File.join(@project_dir, "templates"),        # Root templates folder
+          File.join(@project_dir, "templates", "analysis"),
+          File.join(@project_dir, "templates", "planning"),
+          File.join(@project_dir, "templates", "implementation"),
           File.join(@project_dir, "templates", "COMMON")
         ]
       end
@@ -241,11 +233,11 @@ module Aidp
         # Add current execution context
         context_parts << "## Analysis Context"
         context_parts << "Project Directory: #{@project_dir}"
-        context_parts << "Current Step: #{@harness_runner.current_step}"
-        context_parts << "Current Provider: #{@harness_runner.current_provider}"
+        context_parts << "Current Step: #{get_harness_current_step_safely}"
+        context_parts << "Current Provider: #{get_harness_provider_safely}"
 
         # Add user input context
-        user_input = @harness_runner.user_input
+        user_input = get_harness_user_input_safely
         if user_input && !user_input.empty?
           context_parts << "\n## Previous User Input"
           user_input.each do |key, value|
@@ -254,7 +246,7 @@ module Aidp
         end
 
         # Add execution history context
-        execution_log = @harness_runner.execution_log
+        execution_log = get_harness_execution_log_safely
         if execution_log && !execution_log.empty?
           context_parts << "\n## Analysis History"
           recent_logs = execution_log.last(5) # Last 5 entries
@@ -268,8 +260,9 @@ module Aidp
 
       # Execute step with harness provider management
       def execute_with_harness_provider(provider_type, prompt, step_name, _options)
-        # Get provider manager from harness
-        provider_manager = @harness_runner.provider_manager
+        # Get provider manager from harness safely
+        provider_manager = get_harness_provider_manager_safely
+        return {status: "failed", error: "No provider manager available"} unless provider_manager
 
         # Execute with provider
         provider_manager.execute_with_provider(provider_type, prompt, {
@@ -343,6 +336,57 @@ module Aidp
           })
         end
       end
+
+      # Safely get current provider from harness runner
+      def get_harness_provider_safely
+        return "cursor" unless @harness_runner
+        return "cursor" unless @harness_runner.respond_to?(:current_provider)
+
+        @harness_runner.current_provider || "cursor"
+      rescue => e
+        debug_log("⚠️ Failed to get current provider from harness", level: :warn, data: {error: e.message})
+        "cursor"
+      end
+
+      def get_harness_current_step_safely
+        return "unknown" unless @harness_runner
+        return "unknown" unless @harness_runner.respond_to?(:current_step)
+
+        @harness_runner.current_step || "unknown"
+      rescue => e
+        debug_log("⚠️ Failed to get current step from harness", level: :warn, data: {error: e.message})
+        "unknown"
+      end
+
+      def get_harness_user_input_safely
+        return {} unless @harness_runner
+        return {} unless @harness_runner.respond_to?(:user_input)
+
+        @harness_runner.user_input || {}
+      rescue => e
+        debug_log("⚠️ Failed to get user input from harness", level: :warn, data: {error: e.message})
+        {}
+      end
+
+      def get_harness_execution_log_safely
+        return [] unless @harness_runner
+        return [] unless @harness_runner.respond_to?(:execution_log)
+
+        @harness_runner.execution_log || []
+      rescue => e
+        debug_log("⚠️ Failed to get execution log from harness", level: :warn, data: {error: e.message})
+        []
+      end
+
+      def get_harness_provider_manager_safely
+        return nil unless @harness_runner
+        return nil unless @harness_runner.respond_to?(:provider_manager)
+
+        @harness_runner.provider_manager
+      rescue => e
+        debug_log("⚠️ Failed to get provider manager from harness", level: :warn, data: {error: e.message})
+        nil
+      end
     end
   end
 end

data/lib/aidp/analyze/steps.rb

@@ -3,51 +3,53 @@
 module Aidp
   module Analyze
     module Steps
+      # Analysis step specifications
+      # Templates are organized by purpose and named with action verbs
       SPEC = {
         "01_REPOSITORY_ANALYSIS" => {
-          "templates" => ["01_REPOSITORY_ANALYSIS.md"],
+          "templates" => ["analysis/analyze_repository.md"],
           "description" => "Initial code-maat based repository mining",
           "outs" => ["docs/analysis/repository_analysis.md"],
           "gate" => false
         },
         "02_ARCHITECTURE_ANALYSIS" => {
-          "templates" => ["02_ARCHITECTURE_ANALYSIS.md"],
+          "templates" => ["analysis/analyze_architecture.md"],
           "description" => "Identify architectural patterns, dependencies, and violations",
           "outs" => ["docs/analysis/architecture_analysis.md"],
           "gate" => true
         },
         "03_TEST_ANALYSIS" => {
-          "templates" => ["03_TEST_ANALYSIS.md"],
+          "templates" => ["analysis/analyze_tests.md"],
           "description" => "Analyze existing test coverage and identify gaps",
           "outs" => ["docs/analysis/test_analysis.md"],
           "gate" => false
         },
         "04_FUNCTIONALITY_ANALYSIS" => {
-          "templates" => ["04_FUNCTIONALITY_ANALYSIS.md"],
+          "templates" => ["analysis/analyze_functionality.md"],
           "description" => "Map features, identify dead code, analyze complexity",
           "outs" => ["docs/analysis/functionality_analysis.md"],
           "gate" => false
         },
         "05_DOCUMENTATION_ANALYSIS" => {
-          "templates" => ["05_DOCUMENTATION_ANALYSIS.md"],
+          "templates" => ["analysis/analyze_documentation.md"],
           "description" => "Identify missing documentation and generate what's needed",
           "outs" => ["docs/analysis/documentation_analysis.md"],
           "gate" => false
         },
         "06_STATIC_ANALYSIS" => {
-          "templates" => ["06_STATIC_ANALYSIS.md"],
+          "templates" => ["analysis/analyze_static_code.md"],
           "description" => "Check for existing tools and recommend improvements",
           "outs" => ["docs/analysis/static_analysis.md"],
           "gate" => false
         },
         "06A_TREE_SITTER_SCAN" => {
-          "templates" => ["06a_tree_sitter_scan.md"],
+          "templates" => ["analysis/scan_with_tree_sitter.md"],
           "description" => "Tree-sitter powered static analysis to build knowledge base",
           "outs" => [".aidp/kb/symbols.json", ".aidp/kb/seams.json", ".aidp/kb/hotspots.json"],
           "gate" => false
         },
         "07_REFACTORING_RECOMMENDATIONS" => {
-          "templates" => ["07_REFACTORING_RECOMMENDATIONS.md"],
+          "templates" => ["analysis/recommend_refactoring.md"],
           "description" => "Provide actionable refactoring guidance",
           "outs" => ["docs/analysis/refactoring_recommendations.md"],
           "gate" => true

data/lib/aidp/analyze/tree_sitter_grammar_loader.rb

@@ -8,6 +8,8 @@ require "fileutils"
 module Aidp
   module Analyze
     class TreeSitterGrammarLoader
+      include Aidp::MessageDisplay
+
       # Default grammar configurations
       GRAMMAR_CONFIGS = {
         "ruby" => {
@@ -480,19 +482,6 @@ module Aidp
       end
 
       private
-
-      # Helper method for consistent message display using TTY::Prompt
-      def display_message(message, type: :info)
-        color = case type
-        when :error then :red
-        when :warn then :yellow
-        when :success then :green
-        when :highlight then :cyan
-        else :white
-        end
-
-        @prompt.say(message, color: color)
-      end
     end
   end
 end

data/lib/aidp/analyze/tree_sitter_scan.rb

@@ -12,6 +12,8 @@ require_relative "seams"
 module Aidp
   module Analyze
     class TreeSitterScan
+      include Aidp::MessageDisplay
+
       def initialize(root: Dir.pwd, kb_dir: ".aidp/kb", langs: %w[ruby], threads: Etc.nprocessors, prompt: TTY::Prompt.new)
         @root = File.expand_path(root)
         @kb_dir = File.expand_path(kb_dir, @root)
@@ -685,19 +687,6 @@ module Aidp
       end
 
       private
-
-      # Helper method for consistent message display using TTY::Prompt
-      def display_message(message, type: :info)
-        color = case type
-        when :error then :red
-        when :warn then :yellow
-        when :success then :green
-        when :highlight then :cyan
-        else :white
-        end
-
-        @prompt.say(message, color: color)
-      end
     end
   end
 end