dspy 0.9.0 → 0.10.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 564abfdf924b5e3aa3c002a9210a3e1ef73f2d418e80bf58a4038eecee65219a
-  data.tar.gz: c7c8c1b2338d09636a5cca0b9bab7b12affe91d22d5edfb25031b9b4740ba68c
+  metadata.gz: 7efcb1d5900858477fe538790a6d132a8e289260bb326cee5f30316358fa80f0
+  data.tar.gz: c4116d5f2e01e34e23b6bd3baefd5f7480af214284410c77255d4f37fc099743
 SHA512:
-  metadata.gz: d81c9af7ac29706395e20db912af207cfc02f24d5777e247615ad94ebaa07a5f352df7ce87cac0947acb09a9f9d1e1094950c835a47fda5d2441114bdedabbd2
-  data.tar.gz: 9366f67dffe4baa183f40705e8fc5f8f781b5f87e7aca3f0f77f954bb1009edefd773ce68d704a5e3663a0518f940e2ba51cc9b8a6c5c591e4309a7b325d4d51
+  metadata.gz: 487725527cfbe3b91d5694999aecdd6fe916940bb9ed42c2a222346b0f1b09ec7b9cb0bdf0f6509a542d72a07bfd4dab66dc879b6bf76d2dc1ebb7e61064dfa7
+  data.tar.gz: d3674ea0fe89d498d31b5be92e1a24458bfec0c79f55aef061869cf3a7014f2b5ab67ffac24226c6783f3ea0e485e16bdc16a1dafeef906f602241d8fd7077b7

data/README.md

@@ -25,12 +25,13 @@ The result? LLM applications that actually scale and don't break when you sneeze
 - **Basic Optimization** - Simple prompt optimization techniques
 
 **Production Features:**
-- **Reliable JSON Extraction** - Automatic strategy selection for OpenAI structured outputs, Anthropic patterns, and fallback modes
+- **Reliable JSON Extraction** - Native OpenAI structured outputs, Anthropic extraction patterns, and automatic strategy selection with fallback
+- **Type-Safe Configuration** - Strategy enums with automatic provider optimization (Strict/Compatible modes)
 - **Smart Retry Logic** - Progressive fallback with exponential backoff for handling transient failures
 - **Performance Caching** - Schema and capability caching for faster repeated operations
-- **File-based Storage** - Basic optimization result persistence
+- **File-based Storage** - Optimization result persistence with versioning
 - **Multi-Platform Observability** - OpenTelemetry, New Relic, and Langfuse integration
-- **Basic Instrumentation** - Event tracking and logging
+- **Comprehensive Instrumentation** - Event tracking, performance monitoring, and detailed logging
 
 **Developer Experience:**
 - LLM provider support using official Ruby clients:
@@ -40,20 +41,56 @@ The result? LLM applications that actually scale and don't break when you sneeze
 - Type-safe tool definitions for ReAct agents
 - Comprehensive instrumentation and observability
 
-## Fair Warning
+## Development Status
 
-This is fresh off the oven and evolving fast. I'm actively building this as a Ruby port of the [DSPy library](https://dspy.ai/). If you hit bugs or want to contribute, just email me directly!
+DSPy.rb is actively developed and approaching stability at **v0.9.0**. The core framework is production-ready with comprehensive documentation, but I'm battle-testing features through the 0.x series before committing to a stable v1.0 API. 
+
+Real-world usage feedback is invaluable - if you encounter issues or have suggestions, please open a GitHub issue!
 
 ## Quick Start
 
 ### Installation
 
-Skip the gem for now - install straight from this repo while I prep the first release:
+```ruby
+gem 'dspy', '~> 0.9'
+```
+
+Or add to your Gemfile:
 
 ```ruby
-gem 'dspy', github: 'vicentereig/dspy.rb'
+gem 'dspy'
+```
+
+Then run:
+
+```bash
+bundle install
+```
+
+#### System Dependencies for Ubuntu/Pop!_OS
+
+If you need to compile the `polars-df` dependency from source (used for data processing in evaluations), install these system packages:
+
+```bash
+# Update package list
+sudo apt-get update
+
+# Install Ruby development files (if not already installed)
+sudo apt-get install ruby-full ruby-dev
+
+# Install essential build tools
+sudo apt-get install build-essential
+
+# Install Rust and Cargo (required for polars-df compilation)
+curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
+source $HOME/.cargo/env
+
+# Install CMake (often needed for Rust projects)
+sudo apt-get install cmake
 ```
 
+**Note**: The `polars-df` gem compilation can take 15-20 minutes. Pre-built binaries are available for most platforms, so compilation is only needed if a pre-built binary isn't available for your system.
+
 ### Your First DSPy Program
 
 ```ruby
@@ -124,25 +161,30 @@ puts result.confidence   # => 0.85
 - **[RAG Patterns](docs/src/advanced/rag.md)** - Manual RAG implementation with external services
 - **[Custom Metrics](docs/src/advanced/custom-metrics.md)** - Proc-based evaluation logic
 
-## What's Next
-
-These are my goals to release v1.0.
-
-- ✅ Prompt objects foundation - *Done*
-- ✅ Evaluation framework - *Done*  
-- ✅ Teleprompter base classes - *Done*
-- ✅ MIPROv2 optimization algorithm - *Done*
-- ✅ Storage & persistence system - *Done*
-- ✅ Registry & version management - *Done*
-- ✅ OpenTelemetry integration - *Done*
-- ✅ New Relic integration - *Done*
-- ✅ Langfuse integration - *Done*
-- 🚧 Ollama support
-- Context Engineering (see recent research: [How Contexts Fail](https://www.dbreunig.com/2025/06/22/how-contexts-fail-and-how-to-fix-them.html), [How to Fix Your Context](https://www.dbreunig.com/2025/06/26/how-to-fix-your-context.html), [Context Engineering](https://simonwillison.net/2025/Jun/27/context-engineering/))
-- Agentic Memory support
-- MCP Support
-- Documentation website
-- Performance benchmarks
+## Recent Achievements
+
+DSPy.rb has rapidly evolved from experimental to production-ready:
+
+- ✅ **JSON Parsing Reliability** (v0.8.0) - Native OpenAI structured outputs, strategy selection, retry logic
+- ✅ **Type-Safe Strategy Configuration** (v0.9.0) - Provider-optimized automatic strategy selection  
+- ✅ **Documentation Website** (v0.6.4) - Comprehensive docs at [vicentereig.github.io/dspy.rb](https://vicentereig.github.io/dspy.rb)
+- ✅ **Production Observability** - OpenTelemetry, New Relic, and Langfuse integration
+- ✅ **Optimization Framework** - MIPROv2 algorithm with storage & persistence
+- ✅ **Core Module System** - Predict, ChainOfThought, ReAct, CodeAct with type safety
+
+## Roadmap - Battle-Testing Toward v1.0
+
+DSPy.rb is currently at **v0.9.0** and approaching stability. I'm focusing on real-world usage and refinement through the 0.10, 0.11, 0.12+ series before committing to a stable v1.0 API.
+
+**Current Focus Areas:**
+- 🚧 **Ollama Support** - Local model integration
+- 🚧 **Context Engineering** - Advanced prompt optimization techniques
+- 🚧 **MCP Support** - Model Context Protocol integration
+- 🚧 **Agentic Memory** - Persistent agent state management
+- 🚧 **Performance Optimization** - Based on production usage patterns
+
+**v1.0 Philosophy:** 
+v1.0 will be released after extensive production battle-testing, not after checking off features. This ensures a stable, reliable API backed by real-world validation.
 
 ## License
 

data/lib/dspy/example.rb

@@ -192,8 +192,28 @@ module DSPy
     # String representation for debugging
     sig { returns(String) }
     def to_s
-      "DSPy::Example(#{@signature_class.name}) input=#{input_values} expected=#{expected_values}"
+      "DSPy::Example(#{@signature_class.name}) input=#{format_hash(input_values)} expected=#{format_hash(expected_values)}"
     end
+    
+    private
+    
+    # Format hash without escaping Unicode characters
+    sig { params(hash: T::Hash[Symbol, T.untyped]).returns(String) }
+    def format_hash(hash)
+      pairs = hash.map do |k, v|
+        value_str = case v
+                    when String
+                      # Don't escape Unicode characters
+                      "\"#{v}\""
+                    else
+                      v.inspect
+                    end
+        ":#{k} => #{value_str}"
+      end
+      "{#{pairs.join(", ")}}"
+    end
+    
+    public
 
     sig { returns(String) }
     def inspect

data/lib/dspy/prediction.rb

@@ -0,0 +1,464 @@
+# typed: strict
+# frozen_string_literal: true
+
+module DSPy
+  class Prediction
+    extend T::Sig
+    extend T::Generic
+    include T::Props
+    include T::Props::Serializable
+
+    # The underlying struct that holds the actual data
+    sig { returns(T.untyped) }
+    attr_reader :_struct
+
+    # Schema information for type conversion
+    sig { returns(T.nilable(T::Class[T::Struct])) }
+    attr_reader :_schema
+
+    sig do
+      params(
+        schema: T.nilable(T.any(T::Class[T::Struct], T::Types::Base)),
+        attributes: T.untyped
+      ).void
+    end
+    def initialize(schema = nil, **attributes)
+      @_schema = extract_struct_class(schema)
+      
+      # Convert attributes based on schema if provided
+      converted_attributes = if @_schema
+        convert_attributes_with_schema(attributes)
+      else
+        attributes
+      end
+
+      # Create a dynamic struct to hold the data
+      struct_class = create_dynamic_struct(converted_attributes)
+      @_struct = struct_class.new(**converted_attributes)
+    end
+
+    # Delegate all method calls to the underlying struct
+    sig { params(method: Symbol, args: T.untyped, block: T.untyped).returns(T.untyped) }
+    def method_missing(method, *args, &block)
+      if @_struct.respond_to?(method)
+        @_struct.send(method, *args, &block)
+      else
+        super
+      end
+    end
+
+    sig { params(method: Symbol, include_all: T::Boolean).returns(T::Boolean) }
+    def respond_to_missing?(method, include_all = false)
+      @_struct.respond_to?(method, include_all) || super
+    end
+
+    sig { returns(T::Hash[Symbol, T.untyped]) }
+    def to_h
+      @_struct.to_h
+    end
+
+    private
+
+    sig { params(schema: T.untyped).returns(T.nilable(T::Class[T::Struct])) }
+    def extract_struct_class(schema)
+      case schema
+      when Class
+        schema if schema < T::Struct
+      when T::Types::Simple
+        schema.raw_type if schema.raw_type < T::Struct
+      else
+        nil
+      end
+    end
+
+    sig { params(attributes: T::Hash[Symbol, T.untyped]).returns(T::Hash[Symbol, T.untyped]) }
+    def convert_attributes_with_schema(attributes)
+      return attributes unless @_schema
+
+      converted = {}
+      
+      # Get discriminator mappings for T.any() fields
+      discriminator_mappings = detect_discriminator_fields(@_schema)
+
+      # First, add all the fields from the schema with defaults if not provided
+      @_schema.props.each do |field_name, prop_info|
+        # Skip if attribute was provided
+        next if attributes.key?(field_name)
+        
+        # Apply default value if available
+        default_value = prop_info[:default]
+        if !default_value.nil?
+          if default_value.is_a?(Proc)
+            converted[field_name] = default_value.call
+          else
+            converted[field_name] = default_value
+          end
+        elsif prop_info[:fully_optional]
+          # For optional fields without defaults, set to nil
+          converted[field_name] = nil
+        end
+      end
+
+      attributes.each do |key, value|
+        prop_info = @_schema.props[key]
+        
+        if prop_info && discriminator_mappings[key]
+          # This is a T.any() field with a discriminator
+          discriminator_field, type_mapping = discriminator_mappings[key]
+          discriminator_value = attributes[discriminator_field]
+          prop_type = prop_info[:type_object] || prop_info[:type]
+          
+          converted[key] = convert_union_type(value, discriminator_value, type_mapping, prop_type)
+        elsif prop_info
+          prop_type = prop_info[:type_object] || prop_info[:type]
+          
+          # Handle nil values explicitly
+          if value.nil?
+            # Check if there's a default value
+            default_value = prop_info[:default]
+            if !default_value.nil?
+              converted[key] = default_value.is_a?(Proc) ? default_value.call : default_value
+            else
+              converted[key] = nil
+            end
+          elsif is_enum_type?(prop_type) && value.is_a?(String)
+            # Convert string to enum
+            converted[key] = prop_type.raw_type.deserialize(value)
+          elsif value.is_a?(Hash) && needs_struct_conversion?(prop_type)
+            # Regular struct field that needs conversion
+            converted[key] = convert_to_struct(value, prop_type)
+          elsif value.is_a?(Array) && needs_array_conversion?(prop_type)
+            # Array field that might contain structs
+            converted[key] = convert_array_elements(value, prop_type)
+          else
+            converted[key] = value
+          end
+        else
+          converted[key] = value
+        end
+      end
+
+      converted
+    end
+
+    sig { params(schema: T::Class[T::Struct]).returns(T::Hash[Symbol, [Symbol, T::Hash[String, T.untyped]]]) }
+    def detect_discriminator_fields(schema)
+      discriminator_mappings = {}
+      props = schema.props.to_a
+
+      props.each_with_index do |(prop_name, prop_info), index|
+        prop_type = prop_info[:type_object] || prop_info[:type]
+        next unless is_union_type?(prop_type)
+
+        # Look for preceding String or Enum field as potential discriminator
+        if index > 0
+          prev_prop_name, prev_prop_info = props[index - 1]
+          prev_prop_type = prev_prop_info[:type_object] || prev_prop_info[:type]
+          if prev_prop_type && (is_string_type?(prev_prop_type) || is_enum_type?(prev_prop_type))
+            # This String/Enum field might be a discriminator
+            type_mapping = build_type_mapping_from_union(prop_type, prev_prop_type)
+            discriminator_mappings[prop_name] = [prev_prop_name, type_mapping]
+          end
+        end
+      end
+
+      discriminator_mappings
+    end
+
+    sig { params(type: T.untyped).returns(T::Boolean) }
+    def is_union_type?(type)
+      type.is_a?(T::Types::Union) && !is_nilable_type?(type)
+    end
+
+    sig { params(type: T.untyped).returns(T::Boolean) }
+    def is_nilable_type?(type)
+      type.is_a?(T::Types::Union) && type.types.any? { |t| t == T::Utils.coerce(NilClass) }
+    end
+
+    sig { params(type: T.untyped).returns(T::Boolean) }
+    def is_string_type?(type)
+      case type
+      when T::Types::Simple
+        type.raw_type == String
+      else
+        false
+      end
+    end
+
+    sig { params(type: T.untyped).returns(T::Boolean) }
+    def is_enum_type?(type)
+      return false if type.nil?
+      return false unless type.is_a?(T::Types::Simple)
+      
+      begin
+        raw_type = type.raw_type
+        return false unless raw_type.is_a?(Class)
+        result = raw_type < T::Enum
+        return result == true # Force conversion to boolean
+      rescue StandardError
+        return false
+      end
+    end
+
+    sig { params(union_type: T::Types::Union, discriminator_type: T.untyped).returns(T::Hash[String, T.untyped]) }
+    def build_type_mapping_from_union(union_type, discriminator_type)
+      mapping = {}
+      
+      if is_enum_type?(discriminator_type)
+        # For enum discriminators, try to map enum values to struct types
+        enum_class = discriminator_type.raw_type
+        union_type.types.each do |type|
+          next if type == T::Utils.coerce(NilClass)
+          
+          if type.is_a?(T::Types::Simple) && type.raw_type < T::Struct
+            struct_class = type.raw_type
+            struct_name = struct_class.name.split("::").last
+            
+            # Try to find matching enum value by name
+            enum_class.values.each do |enum_value|
+              enum_name = enum_value.instance_variable_get(:@const_name).to_s
+              if enum_name == struct_name
+                # Exact match
+                mapping[enum_value.serialize] = struct_class
+              elsif enum_name.downcase == struct_name.downcase
+                # Case-insensitive match
+                mapping[enum_value.serialize] = struct_class
+              end
+            end
+            
+            # Also add snake_case mapping as fallback
+            discriminator_value = struct_name
+              .gsub(/([A-Z]+)([A-Z][a-z])/, '\1_\2')
+              .gsub(/([a-z\d])([A-Z])/, '\1_\2')
+              .downcase
+            mapping[discriminator_value] = struct_class
+          end
+        end
+      else
+        # String discriminators use snake_case convention
+        union_type.types.each do |type|
+          next if type == T::Utils.coerce(NilClass)
+          
+          if type.is_a?(T::Types::Simple) && type.raw_type < T::Struct
+            struct_class = type.raw_type
+            # Convert class name to snake_case for discriminator value
+            discriminator_value = struct_class.name
+              .split("::").last
+              .gsub(/([A-Z]+)([A-Z][a-z])/, '\1_\2')
+              .gsub(/([a-z\d])([A-Z])/, '\1_\2')
+              .downcase
+            
+            mapping[discriminator_value] = struct_class
+          end
+        end
+      end
+
+      mapping
+    end
+
+    sig do
+      params(
+        value: T.untyped,
+        discriminator_value: T.untyped,
+        type_mapping: T::Hash[String, T.untyped],
+        union_type: T.untyped
+      ).returns(T.untyped)
+    end
+    def convert_union_type(value, discriminator_value, type_mapping, union_type)
+      return value unless value.is_a?(Hash)
+      
+      # Handle enum discriminators
+      discriminator_str = case discriminator_value
+      when T::Enum
+        discriminator_value.serialize
+      when String
+        discriminator_value
+      else
+        return value
+      end
+
+      struct_class = type_mapping[discriminator_str]
+      return value unless struct_class
+
+      # Convert the Hash to the appropriate struct type
+      struct_class.new(**value)
+    rescue TypeError, ArgumentError
+      # If conversion fails, return the original value
+      value
+    end
+
+    sig { params(type: T.untyped).returns(T::Boolean) }
+    def needs_struct_conversion?(type)
+      case type
+      when T::Types::Simple
+        type.raw_type < T::Struct
+      when T::Types::Union
+        # Check if any type in the union is a struct
+        type.types.any? { |t| needs_struct_conversion?(t) }
+      else
+        false
+      end
+    end
+
+    sig { params(value: T::Hash[Symbol, T.untyped], type: T.untyped).returns(T.untyped) }
+    def convert_to_struct(value, type)
+      case type
+      when T::Types::Simple
+        struct_class = type.raw_type
+        # Convert nested hash values to structs if needed
+        converted_hash = {}
+        
+        # First, apply defaults for missing fields
+        struct_class.props.each do |field_name, prop_info|
+          next if value.key?(field_name)
+          
+          default_value = prop_info[:default]
+          if !default_value.nil?
+            converted_hash[field_name] = default_value.is_a?(Proc) ? default_value.call : default_value
+          end
+        end
+        
+        value.each do |k, v|
+          prop_info = struct_class.props[k]
+          if prop_info
+            prop_type = prop_info[:type_object] || prop_info[:type]
+            if v.is_a?(String) && is_enum_type?(prop_type)
+              # Convert string to enum
+              converted_hash[k] = prop_type.raw_type.deserialize(v)
+            elsif v.is_a?(Hash) && needs_struct_conversion?(prop_type)
+              converted_hash[k] = convert_to_struct(v, prop_type)
+            elsif v.is_a?(Array) && needs_array_conversion?(prop_type)
+              converted_hash[k] = convert_array_elements(v, prop_type)
+            else
+              converted_hash[k] = v
+            end
+          else
+            converted_hash[k] = v
+          end
+        end
+        begin
+          struct_class.new(**converted_hash)
+        rescue => e
+          # Return original value if conversion fails
+          value
+        end
+      when T::Types::Union
+        # For unions without discriminator, try each type
+        type.types.each do |t|
+          next if t == T::Utils.coerce(NilClass)
+          
+          begin
+            return convert_to_struct(value, t) if needs_struct_conversion?(t)
+          rescue TypeError, ArgumentError
+            # Try next type
+          end
+        end
+        value
+      else
+        value
+      end
+    rescue TypeError, ArgumentError
+      value
+    end
+
+    sig { params(type: T.untyped).returns(T::Boolean) }
+    def needs_array_conversion?(type)
+      case type
+      when T::Types::TypedArray
+        needs_struct_conversion?(type.type)
+      else
+        false
+      end
+    end
+
+    sig { params(array: T::Array[T.untyped], type: T.untyped).returns(T::Array[T.untyped]) }
+    def convert_array_elements(array, type)
+      return array unless type.is_a?(T::Types::TypedArray)
+
+      element_type = type.type
+      # Check if elements need any conversion (structs or enums)
+      return array unless needs_struct_conversion?(element_type) || is_enum_type?(element_type)
+
+      array.map do |element|
+        if element.is_a?(Hash)
+          # For union types, we need to infer which struct type based on the hash structure
+          if is_union_type?(element_type) && !is_nilable_type?(element_type)
+            convert_hash_to_union_struct(element, element_type)
+          else
+            convert_to_struct(element, element_type)
+          end
+        elsif element.is_a?(String) && is_enum_type?(element_type)
+          # Convert string to enum
+          element_type.raw_type.deserialize(element)
+        else
+          element
+        end
+      end
+    end
+
+    sig { params(hash: T::Hash[Symbol, T.untyped], union_type: T::Types::Union).returns(T.untyped) }
+    def convert_hash_to_union_struct(hash, union_type)
+      # Try to match the hash structure to one of the union types
+      union_type.types.each do |type|
+        next if type == T::Utils.coerce(NilClass)
+        
+        if type.is_a?(T::Types::Simple) && type.raw_type < T::Struct
+          struct_class = type.raw_type
+          
+          # Check if all required fields of this struct are present in the hash
+          required_fields = struct_class.props.reject { |_, info| info[:fully_optional] }.keys
+          if required_fields.all? { |field| hash.key?(field) }
+            begin
+              # Need to convert nested values too
+              converted_hash = {}
+              hash.each do |k, v|
+                prop_info = struct_class.props[k]
+                if prop_info
+                  prop_type = prop_info[:type_object] || prop_info[:type]
+                  if v.is_a?(String) && is_enum_type?(prop_type)
+                    converted_hash[k] = prop_type.raw_type.deserialize(v)
+                  elsif v.is_a?(Hash) && needs_struct_conversion?(prop_type)
+                    converted_hash[k] = convert_to_struct(v, prop_type)
+                  elsif v.is_a?(Array) && needs_array_conversion?(prop_type)
+                    converted_hash[k] = convert_array_elements(v, prop_type)
+                  else
+                    converted_hash[k] = v
+                  end
+                else
+                  converted_hash[k] = v
+                end
+              end
+              return struct_class.new(**converted_hash)
+            rescue TypeError, ArgumentError
+              # This struct didn't match, try the next one
+            end
+          end
+        end
+      end
+      
+      # If no struct matched, return the original hash
+      hash
+    end
+
+    sig { params(attributes: T::Hash[Symbol, T.untyped]).returns(T::Class[T::Struct]) }
+    def create_dynamic_struct(attributes)
+      # If we have a schema, include all fields from it in the dynamic struct
+      all_fields = if @_schema
+        # Merge schema fields with provided attributes
+        schema_fields = @_schema.props.keys.to_h { |k| [k, nil] }
+        schema_fields.merge(attributes)
+      else
+        attributes
+      end
+      
+      Class.new(T::Struct) do
+        const :_prediction_marker, T::Boolean, default: true
+        
+        all_fields.each do |key, value|
+          # Use T.untyped for dynamic properties
+          const key, T.untyped
+        end
+      end
+    end
+  end
+end

data/lib/dspy/re_act.rb

@@ -30,18 +30,9 @@ module DSPy
       }.compact
     end
   end
-  # Defines the signature for ReAct reasoning using Sorbet signatures
-  class Thought < DSPy::Signature
-    description "Generate a thought about what to do next to answer the question."
-
-    input do
-      const :question, String,
-        description: "The question to answer"
-      const :history, T::Array[HistoryEntry],
-        description: "Previous thoughts and actions, including observations from tools. The agent MUST use information from the history to inform its actions and final answer. Each entry is a hash representing a step in the reasoning process."
-      const :available_tools, T::Array[T::Hash[String, T.untyped]],
-        description: "Array of available tools with their JSON schemas. The agent MUST choose an action from the tool names in this list or use \"finish\". For each tool, use the name exactly as specified and provide action_input as a JSON object matching the tool's schema."
-    end
+  # Base class for ReAct thought generation - will be customized per input type
+  class ThoughtBase < DSPy::Signature
+    description "Generate a thought about what to do next to process the given inputs."
 
     output do
       const :thought, String,
@@ -49,7 +40,7 @@ module DSPy
       const :action, String,
         description: "The action to take. MUST be one of the tool names listed in `available_tools` input, or the literal string \"finish\" to provide the final answer."
       const :action_input, T.any(String, T::Hash[T.untyped, T.untyped]),
-        description: "Input for the chosen action. If action is a tool name, this MUST be a JSON object matching the tool's schema. If action is \"finish\", this field MUST contain the final answer to the original question. This answer MUST be directly taken from the relevant Observation in the history if available. For example, if an observation showed \"Observation: 100.0\", and you are finishing, this field MUST be \"100.0\". Do not leave empty if finishing with an observed answer."
+        description: "Input for the chosen action. If action is a tool name, this MUST be a JSON object matching the tool's schema. If action is \"finish\", this field MUST contain the final result based on processing the input data. This result MUST be directly taken from the relevant Observation in the history if available."
     end
   end
 
@@ -60,19 +51,10 @@ module DSPy
     end
   end
 
-  # Defines the signature for processing observations and deciding next steps
-  class ReActObservation < DSPy::Signature
+  # Base class for observation processing - will be customized per input type
+  class ReActObservationBase < DSPy::Signature
     description "Process the observation from a tool and decide what to do next."
 
-    input do
-      const :question, String,
-        description: "The original question"
-      const :history, T::Array[HistoryEntry],
-        description: "Previous thoughts, actions, and observations. Each entry is a hash representing a step in the reasoning process."
-      const :observation, String,
-        description: "The result from the last action"
-    end
-
     output do
       const :interpretation, String,
         description: "Interpretation of the observation"
@@ -108,11 +90,15 @@ module DSPy
       tools.each { |tool| @tools[tool.name.downcase] = tool }
       @max_iterations = max_iterations
 
+      # Create dynamic signature classes that include the original input fields
+      thought_signature = create_thought_signature(signature_class)
+      observation_signature = create_observation_signature(signature_class)
+
       # Create thought generator using Predict to preserve field descriptions
-      @thought_generator = T.let(DSPy::Predict.new(Thought), DSPy::Predict)
+      @thought_generator = T.let(DSPy::Predict.new(thought_signature), DSPy::Predict)
 
       # Create observation processor using Predict to preserve field descriptions
-      @observation_processor = T.let(DSPy::Predict.new(ReActObservation), DSPy::Predict)
+      @observation_processor = T.let(DSPy::Predict.new(observation_signature), DSPy::Predict)
 
       # Create enhanced output struct with ReAct fields
       @enhanced_output_struct = create_enhanced_output_struct(signature_class)
@@ -148,12 +134,11 @@ module DSPy
         max_iterations: @max_iterations,
         available_tools: available_tools
       }) do
-        # Validate input and extract question
+        # Validate input
         input_struct = @original_signature_class.input_struct_class.new(**kwargs)
-        question = T.cast(input_struct.serialize.values.first, String)
 
         # Execute ReAct reasoning loop
-        reasoning_result = execute_react_reasoning_loop(question)
+        reasoning_result = execute_react_reasoning_loop(input_struct)
 
         # Create enhanced output with all ReAct data
         create_enhanced_result(kwargs, reasoning_result)
@@ -164,9 +149,67 @@ module DSPy
 
     private
 
+    # Creates a dynamic Thought signature that includes the original input fields
+    sig { params(signature_class: T.class_of(DSPy::Signature)).returns(T.class_of(DSPy::Signature)) }
+    def create_thought_signature(signature_class)
+      # Create new class that inherits from DSPy::Signature
+      Class.new(DSPy::Signature) do
+        # Set description
+        description "Generate a thought about what to do next to process the given inputs."
+
+        # Define input fields
+        input do
+          const :input_context, String,
+            desc: "Serialized representation of all input fields"
+          const :history, T::Array[HistoryEntry],
+            desc: "Previous thoughts and actions, including observations from tools."
+          const :available_tools, T::Array[T::Hash[String, T.untyped]],
+            desc: "Array of available tools with their JSON schemas."
+        end
+
+        # Define output fields (same as ThoughtBase)
+        output do
+          const :thought, String,
+            desc: "Reasoning about what to do next, considering the history and observations."
+          const :action, String,
+            desc: "The action to take. MUST be one of the tool names listed in `available_tools` input, or the literal string \"finish\" to provide the final answer."
+          const :action_input, T.any(String, T::Hash[T.untyped, T.untyped]),
+            desc: "Input for the chosen action. If action is a tool name, this MUST be a JSON object matching the tool's schema. If action is \"finish\", this field MUST contain the final result based on processing the input data."
+        end
+      end
+    end
+
+    # Creates a dynamic observation signature that includes the original input fields
+    sig { params(signature_class: T.class_of(DSPy::Signature)).returns(T.class_of(DSPy::Signature)) }
+    def create_observation_signature(signature_class)
+      # Create new class that inherits from DSPy::Signature
+      Class.new(DSPy::Signature) do
+        # Set description
+        description "Process the observation from a tool and decide what to do next."
+
+        # Define input fields
+        input do
+          const :input_context, String,
+            desc: "Serialized representation of all input fields"
+          const :history, T::Array[HistoryEntry],
+            desc: "Previous thoughts, actions, and observations."
+          const :observation, String,
+            desc: "The result from the last action"
+        end
+
+        # Define output fields (same as ReActObservationBase)
+        output do
+          const :interpretation, String,
+            desc: "Interpretation of the observation"
+          const :next_step, NextStep,
+            desc: "What to do next: '#{NextStep::Continue}' or '#{NextStep::Finish}'"
+        end
+      end
+    end
+
     # Executes the main ReAct reasoning loop
-    sig { params(question: String).returns(T::Hash[Symbol, T.untyped]) }
-    def execute_react_reasoning_loop(question)
+    sig { params(input_struct: T.untyped).returns(T::Hash[Symbol, T.untyped]) }
+    def execute_react_reasoning_loop(input_struct)
       history = T.let([], T::Array[HistoryEntry])
       available_tools_desc = @tools.map { |name, tool| JSON.parse(tool.schema) }
       final_answer = T.let(nil, T.nilable(String))
@@ -178,7 +221,7 @@ module DSPy
         iterations_count += 1
 
         iteration_result = execute_single_iteration(
-          question, history, available_tools_desc, iterations_count, tools_used, last_observation
+          input_struct, history, available_tools_desc, iterations_count, tools_used, last_observation
         )
 
         if iteration_result[:should_finish]
@@ -202,8 +245,8 @@ module DSPy
     end
 
     # Executes a single iteration of the ReAct loop
-    sig { params(question: String, history: T::Array[HistoryEntry], available_tools_desc: T::Array[T::Hash[String, T.untyped]], iteration: Integer, tools_used: T::Array[String], last_observation: T.nilable(String)).returns(T::Hash[Symbol, T.untyped]) }
-    def execute_single_iteration(question, history, available_tools_desc, iteration, tools_used, last_observation)
+    sig { params(input_struct: T.untyped, history: T::Array[HistoryEntry], available_tools_desc: T::Array[T::Hash[String, T.untyped]], iteration: Integer, tools_used: T::Array[String], last_observation: T.nilable(String)).returns(T::Hash[Symbol, T.untyped]) }
+    def execute_single_iteration(input_struct, history, available_tools_desc, iteration, tools_used, last_observation)
       # Instrument each iteration
       Instrumentation.instrument('dspy.react.iteration', {
         iteration: iteration,
@@ -213,7 +256,7 @@ module DSPy
       }) do
         # Generate thought and action
         thought_obj = @thought_generator.forward(
-          question: question,
+          input_context: input_struct.serialize.to_json,
           history: history,
           available_tools: available_tools_desc
         )
@@ -243,7 +286,7 @@ module DSPy
 
         # Process observation and decide next step
         observation_decision = process_observation_and_decide_next_step(
-          question, history, observation, available_tools_desc, iteration
+          input_struct, history, observation, available_tools_desc, iteration
         )
 
         if observation_decision[:should_finish]
@@ -337,12 +380,12 @@ module DSPy
       )
     end
 
-    sig { params(question: String, history: T::Array[HistoryEntry], observation: String, available_tools_desc: T::Array[T::Hash[String, T.untyped]], iteration: Integer).returns(T::Hash[Symbol, T.untyped]) }
-    def process_observation_and_decide_next_step(question, history, observation, available_tools_desc, iteration)
+    sig { params(input_struct: T.untyped, history: T::Array[HistoryEntry], observation: String, available_tools_desc: T::Array[T::Hash[String, T.untyped]], iteration: Integer).returns(T::Hash[Symbol, T.untyped]) }
+    def process_observation_and_decide_next_step(input_struct, history, observation, available_tools_desc, iteration)
       return { should_finish: false } if observation.include?("Unknown action")
 
       observation_result = @observation_processor.forward(
-        question: question,
+        input_context: input_struct.serialize.to_json,
         history: history,
         observation: observation
       )
@@ -350,16 +393,16 @@ module DSPy
       return { should_finish: false } unless observation_result.next_step == NextStep::Finish
 
       final_answer = generate_forced_final_answer(
-        question, history, available_tools_desc, observation_result, iteration
+        input_struct, history, available_tools_desc, observation_result, iteration
       )
 
       { should_finish: true, final_answer: final_answer }
     end
 
-    sig { params(question: String, history: T::Array[HistoryEntry], available_tools_desc: T::Array[T::Hash[String, T.untyped]], observation_result: T.untyped, iteration: Integer).returns(String) }
-    def generate_forced_final_answer(question, history, available_tools_desc, observation_result, iteration)
+    sig { params(input_struct: T.untyped, history: T::Array[HistoryEntry], available_tools_desc: T::Array[T::Hash[String, T.untyped]], observation_result: T.untyped, iteration: Integer).returns(String) }
+    def generate_forced_final_answer(input_struct, history, available_tools_desc, observation_result, iteration)
       final_thought = @thought_generator.forward(
-        question: question,
+        input_context: input_struct.serialize.to_json,
         history: history,
         available_tools: available_tools_desc
       )

data/lib/dspy/signature.rb

@@ -147,6 +147,11 @@ module DSPy
         }
       end
 
+      sig { returns(T.nilable(T.class_of(T::Struct))) }
+      def input_schema
+        @input_struct_class
+      end
+
       sig { returns(T::Hash[Symbol, T.untyped]) }
       def output_json_schema
         return {} unless @output_struct_class
@@ -169,6 +174,11 @@ module DSPy
         }
       end
 
+      sig { returns(T.nilable(T.class_of(T::Struct))) }
+      def output_schema
+        @output_struct_class
+      end
+
       private
 
       sig { params(type: T.untyped).returns(T::Hash[Symbol, T.untyped]) }

data/lib/dspy/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module DSPy
-  VERSION = "0.9.0"
+  VERSION = "0.10.0"
 end

data/lib/dspy.rb

@@ -123,6 +123,7 @@ require_relative 'dspy/prompt'
 require_relative 'dspy/example'
 require_relative 'dspy/lm'
 require_relative 'dspy/strategy'
+require_relative 'dspy/prediction'
 require_relative 'dspy/predict'
 require_relative 'dspy/chain_of_thought'
 require_relative 'dspy/re_act'

metadata

@@ -1,13 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: dspy
 version: !ruby/object:Gem::Version
-  version: 0.9.0
+  version: 0.10.0
 platform: ruby
 authors:
 - Vicente Reig Rincón de Arellano
+autorequire: 
 bindir: bin
 cert_chain: []
-date: 2025-07-11 00:00:00.000000000 Z
+date: 2025-07-20 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: dry-configurable
@@ -196,6 +197,7 @@ files:
 - lib/dspy/mixins/type_coercion.rb
 - lib/dspy/module.rb
 - lib/dspy/predict.rb
+- lib/dspy/prediction.rb
 - lib/dspy/prompt.rb
 - lib/dspy/propose/grounded_proposer.rb
 - lib/dspy/re_act.rb
@@ -225,6 +227,7 @@ homepage: https://github.com/vicentereig/dspy.rb
 licenses:
 - MIT
 metadata: {}
+post_install_message: 
 rdoc_options: []
 require_paths:
 - lib
@@ -239,7 +242,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.6.5
+rubygems_version: 3.5.22
+signing_key: 
 specification_version: 4
 summary: Ruby port of DSPy 2.6
 test_files: []