desiru 0.1.1 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 6d4dc11edfa1084914d29616c396cb2bb163c4ed74bf2e13a07f23edd4b0fefc
-  data.tar.gz: 86a62bd76bf86d7aedff994dcf93e110a64a1d135cad0249c89658f22e93bc78
+  metadata.gz: dba6d3f283255824630ae2a8fd4119dfcdb7480f1aa8d50e57aa8c8d740128ce
+  data.tar.gz: 26b4a53f8be0de54d4619c7ae7ba85fb868009c3b7407b7f5730b2cdc8b8277d
 SHA512:
-  metadata.gz: bee932d2d4adb82e40e80feee6a07156f1f8c6f753660e11c492b2ff0ccc6fa8711587bdc05c2961b9f80b32b84394c226b8408fb7467d0d5bd5d2063254c6c0
-  data.tar.gz: f59cb824b72814078b1646c997f001b6aae47fa0ffc2ad7c06070b2a2a9cb0e3fe6068574624c9f508ac2f87326c9d76c2040f28ada3fc91b5e938113ac81186
+  metadata.gz: 4be0c144f72095e40bb49060797ebf2e790d053f17bdb773c4e41bf0bd84f9706f4fff91981ea3bba89535afc2b152ca06e79ad11ac5a74d82912055b115d702
+  data.tar.gz: '08db2c001e9dda2c0f1cb934fcd2826f75290324edf7707e898f2ed18a57b675c4d2c50b7592ce3aa0b7d2506cbb8c704b3edb15accc0fbac44ab917abb76180'

data/.claude/settings.local.json

@@ -0,0 +1,11 @@
+{
+  "permissions": {
+    "allow": [
+      "WebFetch(domain:github.com)",
+      "Bash(gh repo view:*)",
+      "Bash(gh issue list:*)",
+      "Bash(claude-swarm:*)"
+    ],
+    "deny": []
+  }
+}

data/CHANGELOG.md

@@ -0,0 +1,73 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [0.2.0] - 2025-06-23
+
+### Added
+- **Core Data Containers**: New `Example` and `Prediction` classes for structured data handling in DSPy programs
+- **Trace Collection System**: Complete tracing infrastructure with `TraceCollector` and `Traceable` for execution monitoring and debugging
+- **Compilation Infrastructure**: New `Compiler` and `CompilerBuilder` classes enabling full DSPy program compilation pipeline
+- **ProgramOfThought Module**: Advanced reasoning module that generates and executes code for complex problem solving
+- **MIPROv2 Optimizer**: State-of-the-art Bayesian optimization algorithm for automatic prompt and few-shot example optimization
+- **BestOfN Module**: Multi-sampling module with configurable selection criteria (confidence, consistency, LLM judge, custom)
+- **Comprehensive Integration Tests**: Full test coverage for all new components ensuring reliability and correctness
+
+### Enhanced
+- **Module Architecture**: Improved base module system to support advanced tracing and compilation features
+- **Optimization Pipeline**: Complete optimization workflow from data collection through model improvement
+- **Error Handling**: Robust error recovery and logging throughout the new components
+
+### Technical Improvements
+- **Type Safety**: Enhanced type checking and validation across all new modules
+- **Performance**: Optimized execution paths for compilation and optimization workflows  
+- **Extensibility**: Modular architecture enabling easy addition of new optimizers and reasoning modules
+
+## [0.1.1] - 2025-06-21
+
+### Added
+- Direct API client implementations for Anthropic, OpenAI, and OpenRouter
+- New modules: Majority, MultiChainComparison, and ProgramOfThought
+- New optimizers: COPRO and KNNFewShot
+- Enhanced error handling with detailed error classes
+- Support for Ruby 3.3.6 (minimum version now 3.3.0)
+- Interactive console with pre-loaded modules (`bin/console`)
+- Examples runner with model selection (`bin/examples`)
+
+### Changed
+- Replaced Raix dependency with direct API integrations for better control
+- Improved console experience with better error messages and debug helpers
+- Updated default max_tokens from 1000 to 4096 to prevent truncated responses
+- Fixed namespace issues in console by including necessary modules
+
+### Fixed
+- Redis mocking in job tests for CI compatibility
+- Rubocop configuration to match required Ruby version
+- Test failures in CI environment
+
+### Removed
+- Raix gem dependency
+- Support for Ruby 3.2.x (minimum version is now 3.3.0)
+
+## [0.1.0] - 2025-06-12
+
+### Added
+- Initial release of Desiru
+- Core DSPy functionality ported to Ruby
+- Basic modules: Predict, ChainOfThought, ReAct
+- Signature system with type validation
+- Model adapters for OpenAI and Anthropic (via Raix)
+- Optimizers: BootstrapFewShot, MIPROv2
+- REST API integration with Grape and Sinatra
+- GraphQL integration with automatic schema generation
+- Background job processing with Sidekiq
+- Database persistence layer with Sequel
+- Comprehensive test suite
+- Documentation and examples
+
+[0.2.0]: https://github.com/obie/desiru/compare/v0.1.1...v0.2.0
+[0.1.1]: https://github.com/obie/desiru/compare/v0.1.0...v0.1.1
+[0.1.0]: https://github.com/obie/desiru/releases/tag/v0.1.0

data/CLAUDE.local.md

@@ -0,0 +1,3 @@
+## Development Workflow
+
+- ALWAYS run rubocop -A before pushing a new PR update. it's super annoying to have the build break because of the linter

data/CLAUDE.md

@@ -23,4 +23,9 @@ This project is in its initial setup phase. When implementing features:
 
 ## Workflow Guidance
 
-- For maximum efficiency, whenever you need to perform multiple independent operations, invoke all relevant tools simultaneously rather than sequentially.
+- For maximum efficiency, whenever you need to perform multiple independent operations, invoke all relevant tools simultaneously rather than sequentially.
+- When you're tempted to respond and return control to me with a message like "The codebase is now in excellent shape with 859 passing tests, 1 failing test, and 5 pending tests. The project is ready for the v0.2.0 release once the team decides how to handle the final test (either fix it or mark it as pending)." then instead, you should invoke the team to go ahead and decide how to handle the final test.
+
+## Release Guidance
+
+- Note that releases are never ready if there are any tests failing in the test suites. Never tell me that a release is ready unless we have a clean build.

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    desiru (0.1.0)
+    desiru (0.2.0)
       forwardable (~> 1.3)
       redis (~> 5.0)
       sidekiq (~> 7.2)

data/README.md

@@ -2,7 +2,7 @@
 
 A Ruby implementation of [DSPy](https://dspy.ai/), the framework for programming—not prompting—language models. Build sophisticated AI systems with modular, composable code instead of brittle prompt strings.
 
-Note: This project is in its earliest stages of development and experimental. Expect many bugs and breaking changes.
+Note: This project is in active development. While core functionality is stable, expect continued rapid evolution and new features.
 
 
 ## Overview
@@ -88,6 +88,12 @@ cot = Desiru::ChainOfThought.new("question -> answer")
 # ReAct pattern for tool use
 react = Desiru::ReAct.new("question -> answer", tools: [calculator, search])
 
+# Program of Thought - generates and executes code
+pot = Desiru::ProgramOfThought.new("problem -> solution: float")
+
+# Best of N - samples multiple outputs and selects the best
+best_of_n = Desiru::BestOfN.new("question -> answer", n_samples: 3, selection_criterion: :consistency)
+
 # Compose modules into programs
 class RAGPipeline < Desiru::Program
   def initialize

data/desiru-development-swarm.yml

@@ -0,0 +1,185 @@
+version: 1
+swarm:
+  name: "Desiru Development Team"
+  main: project_lead
+  before:
+    - "echo '🚀 Starting Desiru development session...'"
+    - "shadowenv exec -- bundle install"
+    - "shadowenv exec -- bundle exec rspec --help > /dev/null || echo 'RSpec ready'"
+  instances:
+    project_lead:
+      description: "Project lead coordinating Desiru development, managing releases, and ensuring code quality"
+      directory: .
+      model: opus
+      connections: [core_architect, feature_implementer, test_specialist, release_manager]
+      prompt: |
+        You are the project lead for Desiru, a Ruby implementation of DSPy. Your responsibilities include:
+        
+        - Coordinating development work across the team
+        - Making architectural decisions and ensuring code quality
+        - Prioritizing GitHub issues based on the roadmap
+        - Reviewing code changes before they're committed
+        - Managing the overall development process
+        
+        Key project context:
+        - This is a Ruby gem implementing DSPy (Declarative Self-Improving) for programming language models
+        - Current version: 0.1.1 (check lib/desiru/version.rb)
+        - Uses RSpec for testing (NEVER use Minitest)
+        - Follows Ruby community conventions
+        - Has a comprehensive roadmap in issue #22
+        
+        Always use 'shadowenv exec --' prefix for Ruby/bundler commands. Use 'be' alias for bundle exec.
+        
+        For maximum efficiency, whenever you need to perform multiple independent operations, invoke all relevant tools simultaneously rather than sequentially.
+      allowed_tools:
+        - Read
+        - Edit
+        - Bash
+        - WebSearch
+        - WebFetch
+
+    core_architect:
+      description: "Senior architect implementing core DSPy infrastructure, modules, and optimizers"
+      directory: ./lib/desiru
+      model: opus
+      connections: [feature_implementer, test_specialist]
+      prompt: |
+        You are the core architect for Desiru's DSPy implementation. Your expertise includes:
+        
+        - Implementing core DSPy modules (ProgramOfThought, MultiChainComparison, BestOfN)
+        - Building optimizers (MIPROv2, COPRO, BootstrapFewShotWithRandomSearch)
+        - Designing the compilation infrastructure and trace collection system
+        - Creating typed predictors and example/prediction classes
+        
+        Focus on high-priority features from the roadmap (issue #22):
+        - Phase 1: Core Functionality (Example/Prediction classes, ProgramOfThought, MIPROv2, Trace collection)
+        - Phase 2: Enhanced Optimization (MultiChainComparison, BestOfN, COPRO)
+        
+        Always follow Ruby conventions and ensure code is clean, well-documented, and tested.
+        Use 'shadowenv exec --' for Ruby commands and 'be' for bundle exec.
+        
+        For maximum efficiency, whenever you need to perform multiple independent operations, invoke all relevant tools simultaneously rather than sequentially.
+      allowed_tools:
+        - Read
+        - Edit
+        - Write
+        - Bash
+
+    feature_implementer:
+      description: "Feature developer implementing specific DSPy components, utilities, and integrations"
+      directory: ./lib/desiru
+      model: opus
+      connections: [test_specialist]
+      prompt: |
+        You are a feature developer specializing in implementing specific DSPy components. Your focus areas:
+        
+        - Implementing modules: Refine, ChainOfThoughtWithHint, streaming support
+        - Building utilities: data loaders, metrics system, serialization
+        - Creating multi-provider LLM abstractions
+        - Adding advanced features like suggestions system
+        
+        Current priority features (from roadmap issue #22):
+        - Medium priority: Refine module, ChainOfThoughtWithHint, advanced metrics
+        - Utilities: Data loaders (CSV, JSON), streaming support, serialization
+        
+        Ensure all implementations follow existing patterns and are thoroughly tested.
+        Use 'shadowenv exec --' for Ruby commands and 'be' for bundle exec.
+        
+        For maximum efficiency, whenever you need to perform multiple independent operations, invoke all relevant tools simultaneously rather than sequentially.
+      allowed_tools:
+        - Read
+        - Edit
+        - Write
+        - Bash
+
+    test_specialist:
+      description: "Testing expert ensuring comprehensive test coverage and quality assurance"
+      directory: ./spec
+      model: sonnet
+      prompt: |
+        You are the testing specialist for Desiru. Your responsibilities include:
+        
+        - Writing comprehensive RSpec tests for all new features
+        - Ensuring test coverage for core functionality
+        - Creating integration tests for DSPy workflows
+        - Maintaining test quality and performance
+        - Running test suites and fixing test failures
+        
+        CRITICAL: This project uses RSpec exclusively. NEVER use Minitest or create test/ directories.
+        All tests must be in spec/ directory using RSpec format.
+        
+        Key testing priorities:
+        - Round-trip serialization tests
+        - Integration tests for modules and optimizers
+        - Performance benchmarks
+        - Cross-version compatibility tests
+        
+        Use 'shadowenv exec --' for Ruby commands and 'be rspec' for running tests.
+        
+        For maximum efficiency, whenever you need to perform multiple independent operations, invoke all relevant tools simultaneously rather than sequentially.
+      allowed_tools:
+        - Read
+        - Edit
+        - Write
+        - Bash
+
+    release_manager:
+      description: "Release engineering specialist handling versioning, changelogs, and gem publishing"
+      directory: .
+      model: sonnet
+      prompt: |
+        You are the release manager for Desiru. Your responsibilities include:
+        
+        - Managing semantic versioning in lib/desiru/version.rb
+        - Updating CHANGELOG.md with new features and fixes
+        - Preparing release documentation
+        - Coordinating gem publishing to RubyGems
+        - Ensuring release readiness (tests pass, docs updated)
+        
+        Current version: 0.1.1 (check lib/desiru/version.rb)
+        
+        Release process:
+        1. Update version number in lib/desiru/version.rb
+        2. Update CHANGELOG.md with version changes
+        3. Commit changes: git commit -am "Bump version to x.y.z"
+        4. Create version tag: git tag -a vx.y.z -m "Release version x.y.z"
+        5. Push changes and tag: git push && git push --tags
+        
+        Use 'shadowenv exec --' for Ruby commands and follow semantic versioning.
+        Coordinate with project_lead before any releases.
+        
+        For maximum efficiency, whenever you need to perform multiple independent operations, invoke all relevant tools simultaneously rather than sequentially.
+      allowed_tools:
+        - Read
+        - Edit
+        - Bash
+        - WebSearch
+
+    documentation_writer:
+      description: "Documentation specialist maintaining comprehensive docs and examples"
+      directory: ./docs
+      model: sonnet
+      connections: [project_lead]
+      prompt: |
+        You are the documentation specialist for Desiru. Your focus areas:
+        
+        - Maintaining comprehensive API documentation
+        - Creating usage examples and tutorials
+        - Updating feature documentation as new capabilities are added
+        - Ensuring documentation accuracy and clarity
+        - Writing integration guides and best practices
+        
+        Key documentation areas:
+        - Feature gap analysis updates
+        - Integration test strategy documentation
+        - API documentation for new modules and optimizers
+        - Usage examples for new features
+        
+        Keep documentation current with development progress and ensure examples work.
+        
+        For maximum efficiency, whenever you need to perform multiple independent operations, invoke all relevant tools simultaneously rather than sequentially.
+      allowed_tools:
+        - Read
+        - Edit
+        - Write
+        - WebSearch

data/lib/desiru/core/compiler.rb

@@ -0,0 +1,231 @@
+# frozen_string_literal: true
+
+module Desiru
+  module Core
+    class CompilationResult
+      attr_reader :program, :metrics, :traces, :metadata
+
+      def initialize(program:, metrics: {}, traces: [], metadata: {})
+        @program = program
+        @metrics = metrics
+        @traces = traces
+        @metadata = metadata
+      end
+
+      def success?
+        @metadata[:success] != false
+      end
+
+      def optimization_score
+        @metrics[:optimization_score] || 0.0
+      end
+
+      def to_h
+        {
+          program: @program.to_h,
+          metrics: @metrics,
+          traces_count: @traces.size,
+          metadata: @metadata
+        }
+      end
+    end
+
+    class Compiler
+      attr_reader :optimizer, :trace_collector, :config
+
+      def initialize(optimizer: nil, trace_collector: nil, config: {})
+        @optimizer = optimizer
+        @trace_collector = trace_collector || Core.trace_collector
+        @config = default_config.merge(config)
+        @compilation_stack = []
+      end
+
+      def compile(program, training_set = [])
+        start_compilation(program)
+        modules_traced = false
+
+        begin
+          # Clear previous traces if configured
+          @trace_collector.clear if @config[:clear_traces]
+
+          # Enable tracing for all modules
+          enable_module_tracing(program)
+          modules_traced = true
+
+          # Run optimizer if provided
+          if @optimizer
+            if @optimizer.respond_to?(:compile)
+              # MIPROv2 style optimizer
+              optimized_program = @optimizer.compile(program, trainset: training_set)
+            elsif @optimizer.respond_to?(:optimize)
+              # Generic optimizer
+              optimized_program = @optimizer.optimize(program, training_set)
+            else
+              raise ArgumentError, "Optimizer must implement either compile or optimize method"
+            end
+          else
+            # Basic compilation without optimization
+            optimized_program = compile_without_optimization(program, training_set)
+          end
+
+          # Collect compilation metrics
+          metrics = collect_metrics(program, optimized_program, training_set)
+
+          # Get relevant traces
+          traces = @trace_collector.traces.dup
+
+          end_compilation(
+            program: optimized_program,
+            metrics: metrics,
+            traces: traces,
+            metadata: { success: true, optimizer: @optimizer&.class&.name }
+          )
+        rescue StandardError => e
+          end_compilation(
+            program: program,
+            metrics: {},
+            traces: @trace_collector.respond_to?(:traces) ? @trace_collector.traces.dup : [],
+            metadata: { success: false, error: e.message, error_class: e.class.name }
+          )
+        ensure
+          disable_module_tracing(program) if @config[:restore_trace_state] && modules_traced
+        end
+      end
+
+      def compile_module(mod, examples = [])
+        # Compile individual module with examples
+        return mod if examples.empty?
+
+        # Extract demonstrations from successful examples
+        demos = examples.select { |ex| ex.is_a?(Example) }.take(@config[:max_demos])
+
+        # Create new module instance with demos
+        mod.with_demos(demos)
+      end
+
+      private
+
+      def default_config
+        {
+          clear_traces: true,
+          restore_trace_state: true,
+          max_demos: 5,
+          evaluate_metrics: true
+        }
+      end
+
+      def start_compilation(program)
+        @compilation_stack.push({
+                                  program: program,
+                                  start_time: Time.now
+                                })
+      end
+
+      def end_compilation(program:, metrics:, traces:, metadata:)
+        compilation_data = @compilation_stack.pop
+        duration = Time.now - compilation_data[:start_time]
+
+        CompilationResult.new(
+          program: program,
+          metrics: metrics.merge(compilation_duration: duration),
+          traces: traces,
+          metadata: metadata
+        )
+      end
+
+      def enable_module_tracing(program)
+        return unless program.respond_to?(:modules)
+
+        program.modules.each do |mod|
+          mod.enable_trace! if mod.respond_to?(:enable_trace!)
+        end
+      end
+
+      def disable_module_tracing(program)
+        return unless program.respond_to?(:modules)
+
+        program.modules.each do |mod|
+          mod.disable_trace! if mod.respond_to?(:disable_trace!)
+        end
+      end
+
+      def compile_without_optimization(program, training_set)
+        # Basic compilation: collect examples as demonstrations
+        return program if training_set.empty? || !program.respond_to?(:modules)
+
+        # Create a copy of the program
+        compiled_program = program.dup
+
+        # Update modules with training examples if program supports it
+        if compiled_program.respond_to?(:modules) && compiled_program.respond_to?(:update_module)
+          compiled_program.modules.each do |mod|
+            next unless mod.respond_to?(:signature) && mod.signature.respond_to?(:input_fields)
+
+            relevant_examples = training_set.select do |ex|
+              ex.respond_to?(:keys) && ex.keys.any? { |k| mod.signature.input_fields.key?(k) }
+            end
+
+            compiled_module = compile_module(mod, relevant_examples)
+            compiled_program.update_module(mod.class, compiled_module) if compiled_module
+          end
+        end
+
+        compiled_program
+      end
+
+      def collect_metrics(original_program, optimized_program, training_set)
+        return { compilation_duration: 0 } unless @config[:evaluate_metrics]
+
+        metrics = {
+          training_set_size: training_set.size,
+          traces_collected: @trace_collector.size
+        }
+
+        # Add module counts if available
+        metrics[:original_modules_count] = original_program.modules.size if original_program.respond_to?(:modules)
+
+        metrics[:optimized_modules_count] = optimized_program.modules.size if optimized_program.respond_to?(:modules)
+
+        # Add success rate if traces available
+        if @trace_collector.size.positive?
+          success_rate = @trace_collector.successful.size.to_f / @trace_collector.size
+          metrics[:success_rate] = success_rate
+          metrics[:optimization_score] = success_rate
+        end
+
+        metrics
+      end
+    end
+
+    class CompilerBuilder
+      def initialize
+        @optimizer = nil
+        @trace_collector = nil
+        @config = {}
+      end
+
+      def with_optimizer(optimizer)
+        @optimizer = optimizer
+        self
+      end
+
+      def with_trace_collector(collector)
+        @trace_collector = collector
+        self
+      end
+
+      def with_config(config)
+        @config.merge!(config)
+        self
+      end
+
+      def build
+        Compiler.new(
+          optimizer: @optimizer,
+          trace_collector: @trace_collector,
+          config: @config
+        )
+      end
+    end
+  end
+end

data/lib/desiru/core/example.rb

@@ -0,0 +1,96 @@
+# frozen_string_literal: true
+
+module Desiru
+  module Core
+    class Example
+      attr_reader :inputs, :labels
+
+      def initialize(**kwargs)
+        @data = kwargs
+        @inputs = {}
+        @labels = {}
+
+        kwargs.each do |key, value|
+          if key.to_s.end_with?('_input')
+            @inputs[key.to_s.sub(/_input$/, '').to_sym] = value
+          elsif key.to_s.end_with?('_output')
+            @labels[key.to_s.sub(/_output$/, '').to_sym] = value
+          else
+            @inputs[key] = value
+          end
+        end
+      end
+
+      def [](key)
+        @data[key]
+      end
+
+      def []=(key, value)
+        @data[key] = value
+        update_inputs_and_labels(key, value)
+      end
+
+      def keys
+        @data.keys
+      end
+
+      def values
+        @data.values
+      end
+
+      def to_h
+        @data.dup
+      end
+
+      def with_inputs(**new_inputs)
+        merged_data = @data.dup
+        new_inputs.each do |key, value|
+          input_key = key.to_s.end_with?('_input') ? key : :"#{key}_input"
+          merged_data[input_key] = value
+        end
+        self.class.new(**merged_data)
+      end
+
+      def method_missing(method_name, *args, &)
+        if @data.key?(method_name)
+          @data[method_name]
+        elsif method_name.to_s.end_with?('=')
+          key = method_name.to_s.chop.to_sym
+          self[key] = args.first
+        else
+          super
+        end
+      end
+
+      def respond_to_missing?(method_name, include_private = false)
+        @data.key?(method_name) || method_name.to_s.end_with?('=') || super
+      end
+
+      def ==(other)
+        return false unless other.is_a?(self.class)
+
+        @data == other.instance_variable_get(:@data)
+      end
+
+      def hash
+        @data.hash
+      end
+
+      def inspect
+        "#<#{self.class.name} inputs=#{@inputs.inspect} labels=#{@labels.inspect}>"
+      end
+
+      private
+
+      def update_inputs_and_labels(key, value)
+        if key.to_s.end_with?('_input')
+          @inputs[key.to_s.sub(/_input$/, '').to_sym] = value
+        elsif key.to_s.end_with?('_output')
+          @labels[key.to_s.sub(/_output$/, '').to_sym] = value
+        else
+          @inputs[key] = value
+        end
+      end
+    end
+  end
+end