dspy 0.9.0 → 0.9.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 564abfdf924b5e3aa3c002a9210a3e1ef73f2d418e80bf58a4038eecee65219a
-  data.tar.gz: c7c8c1b2338d09636a5cca0b9bab7b12affe91d22d5edfb25031b9b4740ba68c
+  metadata.gz: 306fa376ea024edd2fc75d7bc1af6a42e60770e9566b4dd114414a40c685f2a7
+  data.tar.gz: 74f321fdd09a5761d17481af846dcd85dc3a372f573f79659c4fcfc761e92a18
 SHA512:
-  metadata.gz: d81c9af7ac29706395e20db912af207cfc02f24d5777e247615ad94ebaa07a5f352df7ce87cac0947acb09a9f9d1e1094950c835a47fda5d2441114bdedabbd2
-  data.tar.gz: 9366f67dffe4baa183f40705e8fc5f8f781b5f87e7aca3f0f77f954bb1009edefd773ce68d704a5e3663a0518f940e2ba51cc9b8a6c5c591e4309a7b325d4d51
+  metadata.gz: 5efb12df9c365114696857067bb4f869ce2329c0977666331846a5f070aca050312992c176fddb60ab701b5d31b017d23182d7a4e813e80cba152d155fc2091c
+  data.tar.gz: 2fd7f831b6fab1624d2de7702d54d6377a4a96fff4cb0230eb810285828b764033288bc3180e2aa2cf92b1a1026fa6552cdbf851174216a3fc1a651419accc59

data/README.md

@@ -25,12 +25,13 @@ The result? LLM applications that actually scale and don't break when you sneeze
 - **Basic Optimization** - Simple prompt optimization techniques
 
 **Production Features:**
-- **Reliable JSON Extraction** - Automatic strategy selection for OpenAI structured outputs, Anthropic patterns, and fallback modes
+- **Reliable JSON Extraction** - Native OpenAI structured outputs, Anthropic extraction patterns, and automatic strategy selection with fallback
+- **Type-Safe Configuration** - Strategy enums with automatic provider optimization (Strict/Compatible modes)
 - **Smart Retry Logic** - Progressive fallback with exponential backoff for handling transient failures
 - **Performance Caching** - Schema and capability caching for faster repeated operations
-- **File-based Storage** - Basic optimization result persistence
+- **File-based Storage** - Optimization result persistence with versioning
 - **Multi-Platform Observability** - OpenTelemetry, New Relic, and Langfuse integration
-- **Basic Instrumentation** - Event tracking and logging
+- **Comprehensive Instrumentation** - Event tracking, performance monitoring, and detailed logging
 
 **Developer Experience:**
 - LLM provider support using official Ruby clients:
@@ -40,18 +41,30 @@ The result? LLM applications that actually scale and don't break when you sneeze
 - Type-safe tool definitions for ReAct agents
 - Comprehensive instrumentation and observability
 
-## Fair Warning
+## Development Status
 
-This is fresh off the oven and evolving fast. I'm actively building this as a Ruby port of the [DSPy library](https://dspy.ai/). If you hit bugs or want to contribute, just email me directly!
+DSPy.rb is actively developed and approaching stability at **v0.9.0**. The core framework is production-ready with comprehensive documentation, but I'm battle-testing features through the 0.x series before committing to a stable v1.0 API. 
+
+Real-world usage feedback is invaluable - if you encounter issues or have suggestions, please open a GitHub issue!
 
 ## Quick Start
 
 ### Installation
 
-Skip the gem for now - install straight from this repo while I prep the first release:
+```ruby
+gem 'dspy', '~> 0.9'
+```
+
+Or add to your Gemfile:
 
 ```ruby
-gem 'dspy', github: 'vicentereig/dspy.rb'
+gem 'dspy'
+```
+
+Then run:
+
+```bash
+bundle install
 ```
 
 ### Your First DSPy Program
@@ -124,25 +137,30 @@ puts result.confidence   # => 0.85
 - **[RAG Patterns](docs/src/advanced/rag.md)** - Manual RAG implementation with external services
 - **[Custom Metrics](docs/src/advanced/custom-metrics.md)** - Proc-based evaluation logic
 
-## What's Next
-
-These are my goals to release v1.0.
-
-- ✅ Prompt objects foundation - *Done*
-- ✅ Evaluation framework - *Done*  
-- ✅ Teleprompter base classes - *Done*
-- ✅ MIPROv2 optimization algorithm - *Done*
-- ✅ Storage & persistence system - *Done*
-- ✅ Registry & version management - *Done*
-- ✅ OpenTelemetry integration - *Done*
-- ✅ New Relic integration - *Done*
-- ✅ Langfuse integration - *Done*
-- 🚧 Ollama support
-- Context Engineering (see recent research: [How Contexts Fail](https://www.dbreunig.com/2025/06/22/how-contexts-fail-and-how-to-fix-them.html), [How to Fix Your Context](https://www.dbreunig.com/2025/06/26/how-to-fix-your-context.html), [Context Engineering](https://simonwillison.net/2025/Jun/27/context-engineering/))
-- Agentic Memory support
-- MCP Support
-- Documentation website
-- Performance benchmarks
+## Recent Achievements
+
+DSPy.rb has rapidly evolved from experimental to production-ready:
+
+- ✅ **JSON Parsing Reliability** (v0.8.0) - Native OpenAI structured outputs, strategy selection, retry logic
+- ✅ **Type-Safe Strategy Configuration** (v0.9.0) - Provider-optimized automatic strategy selection  
+- ✅ **Documentation Website** (v0.6.4) - Comprehensive docs at [vicentereig.github.io/dspy.rb](https://vicentereig.github.io/dspy.rb)
+- ✅ **Production Observability** - OpenTelemetry, New Relic, and Langfuse integration
+- ✅ **Optimization Framework** - MIPROv2 algorithm with storage & persistence
+- ✅ **Core Module System** - Predict, ChainOfThought, ReAct, CodeAct with type safety
+
+## Roadmap - Battle-Testing Toward v1.0
+
+DSPy.rb is currently at **v0.9.0** and approaching stability. I'm focusing on real-world usage and refinement through the 0.10, 0.11, 0.12+ series before committing to a stable v1.0 API.
+
+**Current Focus Areas:**
+- 🚧 **Ollama Support** - Local model integration
+- 🚧 **Context Engineering** - Advanced prompt optimization techniques
+- 🚧 **MCP Support** - Model Context Protocol integration
+- 🚧 **Agentic Memory** - Persistent agent state management
+- 🚧 **Performance Optimization** - Based on production usage patterns
+
+**v1.0 Philosophy:** 
+v1.0 will be released after extensive production battle-testing, not after checking off features. This ensures a stable, reliable API backed by real-world validation.
 
 ## License
 

data/lib/dspy/re_act.rb

@@ -30,18 +30,9 @@ module DSPy
       }.compact
     end
   end
-  # Defines the signature for ReAct reasoning using Sorbet signatures
-  class Thought < DSPy::Signature
-    description "Generate a thought about what to do next to answer the question."
-
-    input do
-      const :question, String,
-        description: "The question to answer"
-      const :history, T::Array[HistoryEntry],
-        description: "Previous thoughts and actions, including observations from tools. The agent MUST use information from the history to inform its actions and final answer. Each entry is a hash representing a step in the reasoning process."
-      const :available_tools, T::Array[T::Hash[String, T.untyped]],
-        description: "Array of available tools with their JSON schemas. The agent MUST choose an action from the tool names in this list or use \"finish\". For each tool, use the name exactly as specified and provide action_input as a JSON object matching the tool's schema."
-    end
+  # Base class for ReAct thought generation - will be customized per input type
+  class ThoughtBase < DSPy::Signature
+    description "Generate a thought about what to do next to process the given inputs."
 
     output do
       const :thought, String,
@@ -49,7 +40,7 @@ module DSPy
       const :action, String,
         description: "The action to take. MUST be one of the tool names listed in `available_tools` input, or the literal string \"finish\" to provide the final answer."
       const :action_input, T.any(String, T::Hash[T.untyped, T.untyped]),
-        description: "Input for the chosen action. If action is a tool name, this MUST be a JSON object matching the tool's schema. If action is \"finish\", this field MUST contain the final answer to the original question. This answer MUST be directly taken from the relevant Observation in the history if available. For example, if an observation showed \"Observation: 100.0\", and you are finishing, this field MUST be \"100.0\". Do not leave empty if finishing with an observed answer."
+        description: "Input for the chosen action. If action is a tool name, this MUST be a JSON object matching the tool's schema. If action is \"finish\", this field MUST contain the final result based on processing the input data. This result MUST be directly taken from the relevant Observation in the history if available."
     end
   end
 
@@ -60,19 +51,10 @@ module DSPy
     end
   end
 
-  # Defines the signature for processing observations and deciding next steps
-  class ReActObservation < DSPy::Signature
+  # Base class for observation processing - will be customized per input type
+  class ReActObservationBase < DSPy::Signature
     description "Process the observation from a tool and decide what to do next."
 
-    input do
-      const :question, String,
-        description: "The original question"
-      const :history, T::Array[HistoryEntry],
-        description: "Previous thoughts, actions, and observations. Each entry is a hash representing a step in the reasoning process."
-      const :observation, String,
-        description: "The result from the last action"
-    end
-
     output do
       const :interpretation, String,
         description: "Interpretation of the observation"
@@ -108,11 +90,15 @@ module DSPy
       tools.each { |tool| @tools[tool.name.downcase] = tool }
       @max_iterations = max_iterations
 
+      # Create dynamic signature classes that include the original input fields
+      thought_signature = create_thought_signature(signature_class)
+      observation_signature = create_observation_signature(signature_class)
+
       # Create thought generator using Predict to preserve field descriptions
-      @thought_generator = T.let(DSPy::Predict.new(Thought), DSPy::Predict)
+      @thought_generator = T.let(DSPy::Predict.new(thought_signature), DSPy::Predict)
 
       # Create observation processor using Predict to preserve field descriptions
-      @observation_processor = T.let(DSPy::Predict.new(ReActObservation), DSPy::Predict)
+      @observation_processor = T.let(DSPy::Predict.new(observation_signature), DSPy::Predict)
 
       # Create enhanced output struct with ReAct fields
       @enhanced_output_struct = create_enhanced_output_struct(signature_class)
@@ -148,12 +134,11 @@ module DSPy
         max_iterations: @max_iterations,
         available_tools: available_tools
       }) do
-        # Validate input and extract question
+        # Validate input
         input_struct = @original_signature_class.input_struct_class.new(**kwargs)
-        question = T.cast(input_struct.serialize.values.first, String)
 
         # Execute ReAct reasoning loop
-        reasoning_result = execute_react_reasoning_loop(question)
+        reasoning_result = execute_react_reasoning_loop(input_struct)
 
         # Create enhanced output with all ReAct data
         create_enhanced_result(kwargs, reasoning_result)
@@ -164,9 +149,67 @@ module DSPy
 
     private
 
+    # Creates a dynamic Thought signature that includes the original input fields
+    sig { params(signature_class: T.class_of(DSPy::Signature)).returns(T.class_of(DSPy::Signature)) }
+    def create_thought_signature(signature_class)
+      # Create new class that inherits from DSPy::Signature
+      Class.new(DSPy::Signature) do
+        # Set description
+        description "Generate a thought about what to do next to process the given inputs."
+
+        # Define input fields
+        input do
+          const :input_context, String,
+            desc: "Serialized representation of all input fields"
+          const :history, T::Array[HistoryEntry],
+            desc: "Previous thoughts and actions, including observations from tools."
+          const :available_tools, T::Array[T::Hash[String, T.untyped]],
+            desc: "Array of available tools with their JSON schemas."
+        end
+
+        # Define output fields (same as ThoughtBase)
+        output do
+          const :thought, String,
+            desc: "Reasoning about what to do next, considering the history and observations."
+          const :action, String,
+            desc: "The action to take. MUST be one of the tool names listed in `available_tools` input, or the literal string \"finish\" to provide the final answer."
+          const :action_input, T.any(String, T::Hash[T.untyped, T.untyped]),
+            desc: "Input for the chosen action. If action is a tool name, this MUST be a JSON object matching the tool's schema. If action is \"finish\", this field MUST contain the final result based on processing the input data."
+        end
+      end
+    end
+
+    # Creates a dynamic observation signature that includes the original input fields
+    sig { params(signature_class: T.class_of(DSPy::Signature)).returns(T.class_of(DSPy::Signature)) }
+    def create_observation_signature(signature_class)
+      # Create new class that inherits from DSPy::Signature
+      Class.new(DSPy::Signature) do
+        # Set description
+        description "Process the observation from a tool and decide what to do next."
+
+        # Define input fields
+        input do
+          const :input_context, String,
+            desc: "Serialized representation of all input fields"
+          const :history, T::Array[HistoryEntry],
+            desc: "Previous thoughts, actions, and observations."
+          const :observation, String,
+            desc: "The result from the last action"
+        end
+
+        # Define output fields (same as ReActObservationBase)
+        output do
+          const :interpretation, String,
+            desc: "Interpretation of the observation"
+          const :next_step, NextStep,
+            desc: "What to do next: '#{NextStep::Continue}' or '#{NextStep::Finish}'"
+        end
+      end
+    end
+
     # Executes the main ReAct reasoning loop
-    sig { params(question: String).returns(T::Hash[Symbol, T.untyped]) }
-    def execute_react_reasoning_loop(question)
+    sig { params(input_struct: T.untyped).returns(T::Hash[Symbol, T.untyped]) }
+    def execute_react_reasoning_loop(input_struct)
       history = T.let([], T::Array[HistoryEntry])
       available_tools_desc = @tools.map { |name, tool| JSON.parse(tool.schema) }
       final_answer = T.let(nil, T.nilable(String))
@@ -178,7 +221,7 @@ module DSPy
         iterations_count += 1
 
         iteration_result = execute_single_iteration(
-          question, history, available_tools_desc, iterations_count, tools_used, last_observation
+          input_struct, history, available_tools_desc, iterations_count, tools_used, last_observation
         )
 
         if iteration_result[:should_finish]
@@ -202,8 +245,8 @@ module DSPy
     end
 
     # Executes a single iteration of the ReAct loop
-    sig { params(question: String, history: T::Array[HistoryEntry], available_tools_desc: T::Array[T::Hash[String, T.untyped]], iteration: Integer, tools_used: T::Array[String], last_observation: T.nilable(String)).returns(T::Hash[Symbol, T.untyped]) }
-    def execute_single_iteration(question, history, available_tools_desc, iteration, tools_used, last_observation)
+    sig { params(input_struct: T.untyped, history: T::Array[HistoryEntry], available_tools_desc: T::Array[T::Hash[String, T.untyped]], iteration: Integer, tools_used: T::Array[String], last_observation: T.nilable(String)).returns(T::Hash[Symbol, T.untyped]) }
+    def execute_single_iteration(input_struct, history, available_tools_desc, iteration, tools_used, last_observation)
       # Instrument each iteration
       Instrumentation.instrument('dspy.react.iteration', {
         iteration: iteration,
@@ -213,7 +256,7 @@ module DSPy
       }) do
         # Generate thought and action
         thought_obj = @thought_generator.forward(
-          question: question,
+          input_context: input_struct.serialize.to_json,
           history: history,
           available_tools: available_tools_desc
         )
@@ -243,7 +286,7 @@ module DSPy
 
         # Process observation and decide next step
         observation_decision = process_observation_and_decide_next_step(
-          question, history, observation, available_tools_desc, iteration
+          input_struct, history, observation, available_tools_desc, iteration
         )
 
         if observation_decision[:should_finish]
@@ -337,12 +380,12 @@ module DSPy
       )
     end
 
-    sig { params(question: String, history: T::Array[HistoryEntry], observation: String, available_tools_desc: T::Array[T::Hash[String, T.untyped]], iteration: Integer).returns(T::Hash[Symbol, T.untyped]) }
-    def process_observation_and_decide_next_step(question, history, observation, available_tools_desc, iteration)
+    sig { params(input_struct: T.untyped, history: T::Array[HistoryEntry], observation: String, available_tools_desc: T::Array[T::Hash[String, T.untyped]], iteration: Integer).returns(T::Hash[Symbol, T.untyped]) }
+    def process_observation_and_decide_next_step(input_struct, history, observation, available_tools_desc, iteration)
       return { should_finish: false } if observation.include?("Unknown action")
 
       observation_result = @observation_processor.forward(
-        question: question,
+        input_context: input_struct.serialize.to_json,
         history: history,
         observation: observation
       )
@@ -350,16 +393,16 @@ module DSPy
       return { should_finish: false } unless observation_result.next_step == NextStep::Finish
 
       final_answer = generate_forced_final_answer(
-        question, history, available_tools_desc, observation_result, iteration
+        input_struct, history, available_tools_desc, observation_result, iteration
       )
 
       { should_finish: true, final_answer: final_answer }
     end
 
-    sig { params(question: String, history: T::Array[HistoryEntry], available_tools_desc: T::Array[T::Hash[String, T.untyped]], observation_result: T.untyped, iteration: Integer).returns(String) }
-    def generate_forced_final_answer(question, history, available_tools_desc, observation_result, iteration)
+    sig { params(input_struct: T.untyped, history: T::Array[HistoryEntry], available_tools_desc: T::Array[T::Hash[String, T.untyped]], observation_result: T.untyped, iteration: Integer).returns(String) }
+    def generate_forced_final_answer(input_struct, history, available_tools_desc, observation_result, iteration)
       final_thought = @thought_generator.forward(
-        question: question,
+        input_context: input_struct.serialize.to_json,
         history: history,
         available_tools: available_tools_desc
       )

data/lib/dspy/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module DSPy
-  VERSION = "0.9.0"
+  VERSION = "0.9.1"
 end

metadata

@@ -1,13 +1,13 @@
 --- !ruby/object:Gem::Specification
 name: dspy
 version: !ruby/object:Gem::Version
-  version: 0.9.0
+  version: 0.9.1
 platform: ruby
 authors:
 - Vicente Reig Rincón de Arellano
 bindir: bin
 cert_chain: []
-date: 2025-07-11 00:00:00.000000000 Z
+date: 2025-07-16 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: dry-configurable