dspy 0.34.1 → 0.34.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 100e82f4aeff8020a845aa80a63ad86278ead32f34b7846b0624db99dc060325
-  data.tar.gz: 43ed18798e67e829e2decdd8ef0519751d6f4fc2cf52571850f1acfd671f2780
+  metadata.gz: 01f38786c88d525a1031cf41931f578c3d2dcbfa29ee6a8dac1a381cafe47edf
+  data.tar.gz: 6334bfb483b3011fa91e163f688127be763a126ea7cd0edc44f07b0557dc2a30
 SHA512:
-  metadata.gz: 3081d9fada92dcf1f7f5b003212fd6d5b94787b6982c6828bd95704e84f197be3017d08eff81c4c1271e4a1d442e6a235973f4c2ab69c47493e025b2d155b1ca
-  data.tar.gz: 32454139be26918608d4c63f58a706d762caa457ca3df9addb238eb9c9bfd5e97f749054923573838b6e983db129732de290387076423e39293bcf02e2747bc4
+  metadata.gz: 744087dd87e936b247d194539407f2a74b29d5e6a28b4ba872c4aa0ef77103c4a6957c97b6bed3ee7e8ef899824f3e6e0f40c2b429c47312aa10924bb1fbca3c
+  data.tar.gz: 4e343687e84570d199ce9c7695d19d0a0a551cac66693fda131fe03268d3907e2d20f4648530d1e6a5de0a73092b03f3ec7bcec877d9c23662332193aaee0e31

data/README.md

@@ -6,56 +6,94 @@
 [![Documentation](https://img.shields.io/badge/docs-oss.vicente.services%2Fdspy.rb-blue)](https://oss.vicente.services/dspy.rb/)
 [![Discord](https://img.shields.io/discord/1161519468141355160?label=discord&logo=discord&logoColor=white)](https://discord.gg/zWBhrMqn)
 
-> [!NOTE]
-> The core Prompt Engineering Framework is production-ready with
-> comprehensive documentation. I am focusing now on educational content on systematic Prompt Optimization and Context Engineering.
-> Your feedback is invaluable. if you encounter issues, please open an [issue](https://github.com/vicentereig/dspy.rb/issues). If you have suggestions, open a [new thread](https://github.com/vicentereig/dspy.rb/discussions).  
-> 
-> If you want to contribute, feel free to reach out to me to coordinate efforts: hey at vicente.services
->
-
 **Build reliable LLM applications in idiomatic Ruby using composable, type-safe modules.**
 
-DSPy.rb is the Ruby-first surgical port of Stanford's [DSPy paradigm](https://github.com/stanfordnlp/dspy). It delivers structured LLM programming, prompt engineering, and context engineering in the language we love. Instead of wrestling with brittle prompt strings, you define typed signatures in idiomatic Ruby and compose workflows and agents that actually behave.
+DSPy.rb is the Ruby port of Stanford's [DSPy](https://dspy.ai). Instead of wrestling with brittle prompt strings, you define typed signatures and let the framework handle the rest. Prompts become functions. LLM calls become predictable.
 
-**Prompts are just functions.** Traditional prompting is like writing code with string concatenation: it works until it doesn't. DSPy.rb brings you the programming approach pioneered by [dspy.ai](https://dspy.ai/): define modular signatures and let the framework deal with the messy bits.
+```ruby
+require 'dspy'
 
-While we implement the same signatures, predictors, and optimization algorithms as the original library, DSPy.rb leans hard into Ruby conventions with Sorbet-based typing, ReAct loops, and production-ready integrations like non-blocking OpenTelemetry instrumentation.
+DSPy.configure do |c|
+  c.lm = DSPy::LM.new('openai/gpt-4o-mini', api_key: ENV['OPENAI_API_KEY'])
+end
 
-**What you get?** Ruby LLM applications that scale and don't break when you sneeze.
+class Summarize < DSPy::Signature
+  description "Summarize the given text in one sentence."
 
-Check the [examples](examples/) and take them for a spin!
+  input do
+    const :text, String
+  end
 
-## Your First DSPy Program
-### Installation
+  output do
+    const :summary, String
+  end
+end
 
-Add to your Gemfile:
+summarizer = DSPy::Predict.new(Summarize)
+result = summarizer.call(text: "DSPy.rb brings structured LLM programming to Ruby...")
+puts result.summary
+```
+
+That's it. No prompt templates. No JSON parsing. No prayer-based error handling.
+
+## Installation
 
 ```ruby
+# Gemfile
 gem 'dspy'
+gem 'dspy-openai'     # For OpenAI, OpenRouter, or Ollama
+# gem 'dspy-anthropic' # For Claude
+# gem 'dspy-gemini'    # For Gemini
+# gem 'dspy-ruby_llm'  # For 12+ providers via RubyLLM
 ```
 
-and
-
 ```bash
 bundle install
 ```
 
-### Your First Reliable Predictor
+## Quick Start
 
-```ruby
-require 'dspy'
+### Configure Your LLM
 
-# Configure DSPy globally to use your fave LLM (you can override per predictor).
+```ruby
+# OpenAI
 DSPy.configure do |c|
   c.lm = DSPy::LM.new('openai/gpt-4o-mini',
                       api_key: ENV['OPENAI_API_KEY'],
-                      structured_outputs: true)  # Enable OpenAI's native JSON mode
+                      structured_outputs: true)
 end
 
-# Define a signature for sentiment classification - instead of writing a full prompt!
+# Anthropic Claude
+DSPy.configure do |c|
+  c.lm = DSPy::LM.new('anthropic/claude-sonnet-4-20250514',
+                      api_key: ENV['ANTHROPIC_API_KEY'])
+end
+
+# Google Gemini
+DSPy.configure do |c|
+  c.lm = DSPy::LM.new('gemini/gemini-2.5-flash',
+                      api_key: ENV['GEMINI_API_KEY'])
+end
+
+# Ollama (local, free)
+DSPy.configure do |c|
+  c.lm = DSPy::LM.new('ollama/llama3.2')
+end
+
+# OpenRouter (200+ models)
+DSPy.configure do |c|
+  c.lm = DSPy::LM.new('openrouter/deepseek/deepseek-chat-v3.1:free',
+                      api_key: ENV['OPENROUTER_API_KEY'])
+end
+```
+
+### Define a Signature
+
+Signatures are typed contracts for LLM operations. Define inputs, outputs, and let DSPy handle the prompt:
+
+```ruby
 class Classify < DSPy::Signature
-  description "Classify sentiment of a given sentence." # sets the goal of the underlying prompt
+  description "Classify sentiment of a given sentence."
 
   class Sentiment < T::Enum
     enums do
@@ -64,245 +102,130 @@ class Classify < DSPy::Signature
       Neutral = new('neutral')
     end
   end
-  
-  # Structured Inputs: makes sure you are sending only valid prompt inputs to your model
+
   input do
     const :sentence, String, description: 'The sentence to analyze'
   end
 
-  # Structured Outputs: your predictor will validate the output of the model too.
   output do
-    const :sentiment, Sentiment, description: 'The sentiment of the sentence'
-    const :confidence, Float, description: 'A number between 0.0 and 1.0'
+    const :sentiment, Sentiment
+    const :confidence, Float
   end
 end
 
-# Wire it to the simplest prompting technique: a prediction loop.
-classify = DSPy::Predict.new(Classify)
-# it may raise an error if you mess the inputs or your LLM messes the outputs.
-result = classify.call(sentence: "This book was super fun to read!")
+classifier = DSPy::Predict.new(Classify)
+result = classifier.call(sentence: "This book was super fun to read!")
 
-puts result.sentiment    # => #<Sentiment::Positive>  
-puts result.confidence   # => 0.85
+result.sentiment    # => #<Sentiment::Positive>
+result.confidence   # => 0.92
 ```
 
-Save this as `examples/first_predictor.rb` and run it with:
+### Chain of Thought
 
-```bash
-bundle exec ruby examples/first_predictor.rb
-```
-
-### Sibling Gems
-
-DSPy.rb ships multiple gems from this monorepo so you can opt into features with heavier dependency trees (e.g., datasets pull in Polars/Arrow, MIPROv2 requires `numo-*` BLAS bindings) only when you need them. Add these alongside `dspy`:
+For complex reasoning, use `ChainOfThought` to get step-by-step explanations:
 
-| Gem | Description | Status |
-| --- | --- | --- |
-| `dspy-schema` | Exposes `DSPy::TypeSystem::SorbetJsonSchema` for downstream reuse. (Still required by the core `dspy` gem; extraction lets other projects depend on it directly.) | **Stable** (v1.0.0) |
-| `dspy-openai` | Packages the OpenAI/OpenRouter/Ollama adapters plus the official SDK guardrails. Install whenever you call `openai/*`, `openrouter/*`, or `ollama/*`. [Adapter README](https://github.com/vicentereig/dspy.rb/blob/main/lib/dspy/openai/README.md) | **Stable** (v1.0.0) |
-| `dspy-anthropic` | Claude adapters, streaming, and structured-output helpers behind the official `anthropic` SDK. [Adapter README](https://github.com/vicentereig/dspy.rb/blob/main/lib/dspy/anthropic/README.md) | **Stable** (v1.0.0) |
-| `dspy-gemini` | Gemini adapters with multimodal + tool-call support via `gemini-ai`. [Adapter README](https://github.com/vicentereig/dspy.rb/blob/main/lib/dspy/gemini/README.md) | **Stable** (v1.0.0) |
-| `dspy-ruby_llm` | Unified access to 12+ LLM providers (OpenAI, Anthropic, Gemini, Bedrock, Ollama, DeepSeek, etc.) via [RubyLLM](https://rubyllm.com). [Adapter README](https://github.com/vicentereig/dspy.rb/blob/main/lib/dspy/ruby_llm/README.md) | **Stable** (v0.1.0) |
-| `dspy-code_act` | Think-Code-Observe agents that synthesize and execute Ruby safely. (Add the gem or set `DSPY_WITH_CODE_ACT=1` before requiring `dspy/code_act`.) | **Stable** (v1.0.0) |
-| `dspy-datasets` | Dataset helpers plus Parquet/Polars tooling for richer evaluation corpora. (Toggle via `DSPY_WITH_DATASETS`.) | **Stable** (v1.0.0) |
-| `dspy-evals` | High-throughput evaluation harness with metrics, callbacks, and regression fixtures. (Toggle via `DSPY_WITH_EVALS`.) | **Stable** (v1.0.0) |
-| `dspy-miprov2` | Bayesian optimization + Gaussian Process backend for the MIPROv2 teleprompter. (Install or export `DSPY_WITH_MIPROV2=1` before requiring the teleprompter.) | **Stable** (v1.0.0) |
-| `dspy-gepa` | `DSPy::Teleprompt::GEPA`, reflection loops, experiment tracking, telemetry adapters. (Install or set `DSPY_WITH_GEPA=1`.) | **Stable** (v1.0.0) |
-| `gepa` | GEPA optimizer core (Pareto engine, telemetry, reflective proposer). | **Stable** (v1.0.0) |
-| `dspy-o11y` | Core observability APIs: `DSPy::Observability`, async span processor, observation types. (Install or set `DSPY_WITH_O11Y=1`.) | **Stable** (v1.0.0) |
-| `dspy-o11y-langfuse` | Auto-configures DSPy observability to stream spans to Langfuse via OTLP. (Install or set `DSPY_WITH_O11Y_LANGFUSE=1`.) | **Stable** (v1.0.0) |
-| `dspy-deep_search` | Production DeepSearch loop with Exa-backed search/read, token budgeting, and instrumentation (Issue #163). | **Stable** (v1.0.0) |
-| `dspy-deep_research` | Planner/QA orchestration atop DeepSearch plus the memory supervisor used by the CLI example. | **Stable** (v1.0.0) |
-| `sorbet-toon` | Token-Oriented Object Notation (TOON) codec, prompt formatter, and Sorbet mixins for BAML/TOON Enhanced Prompting. [Sorbet::Toon README](https://github.com/vicentereig/dspy.rb/blob/main/lib/sorbet/toon/README.md) | **Alpha** (v0.1.0) |
+```ruby
+solver = DSPy::ChainOfThought.new(MathProblem)
+result = solver.call(problem: "If a train travels 120km in 2 hours, what's its speed?")
 
-**Provider adapters:** Add `dspy-openai`, `dspy-anthropic`, and/or `dspy-gemini` next to `dspy` in your Gemfile depending on which `DSPy::LM` providers you call. Each gem already depends on the official SDK (`openai`, `anthropic`, `gemini-ai`), and DSPy auto-loads the adapters when the gem is present—no extra `require` needed.
+result.reasoning  # => "Speed = Distance / Time = 120km / 2h = 60km/h"
+result.answer     # => "60 km/h"
+```
 
-Set the matching `DSPY_WITH_*` environment variables (see `Gemfile`) to include or exclude each sibling gem when running Bundler locally (for example `DSPY_WITH_GEPA=1` or `DSPY_WITH_O11Y_LANGFUSE=1`). Refer to `adr/013-dependency-tree.md` for the full dependency map and roadmap.
-### Access to 200+ Models Across 5 Providers
+### ReAct Agents
 
-DSPy.rb provides unified access to major LLM providers with provider-specific optimizations:
+Build agents that use tools to accomplish tasks:
 
 ```ruby
-# OpenAI (GPT-4, GPT-4o, GPT-4o-mini, GPT-5, etc.)
-DSPy.configure do |c|
-  c.lm = DSPy::LM.new('openai/gpt-4o-mini',
-                      api_key: ENV['OPENAI_API_KEY'],
-                      structured_outputs: true)  # Native JSON mode
-end
+class SearchTool < DSPy::Tools::Tool
+  tool_name "search"
+  description "Search for information"
 
-# Google Gemini (Gemini 1.5 Pro, Flash, Gemini 2.0, etc.)
-DSPy.configure do |c|
-  c.lm = DSPy::LM.new('gemini/gemini-2.5-flash',
-                      api_key: ENV['GEMINI_API_KEY'],
-                      structured_outputs: true)  # Native structured outputs
-end
+  input do
+    const :query, String
+  end
 
-# Anthropic Claude (Claude 3.5, Claude 4, etc.)
-DSPy.configure do |c|
-  c.lm = DSPy::LM.new('anthropic/claude-sonnet-4-5-20250929',
-                      api_key: ENV['ANTHROPIC_API_KEY'],
-                      structured_outputs: true)  # Tool-based extraction (default)
-end
+  output do
+    const :results, T::Array[String]
+  end
 
-# Ollama - Run any local model (Llama, Mistral, Gemma, etc.)
-DSPy.configure do |c|
-  c.lm = DSPy::LM.new('ollama/llama3.2')  # Free, runs locally, no API key needed
+  def call(query:)
+    # Your search implementation
+    { results: ["Result 1", "Result 2"] }
+  end
 end
 
-# OpenRouter - Access to 200+ models from multiple providers
-DSPy.configure do |c|
-  c.lm = DSPy::LM.new('openrouter/deepseek/deepseek-chat-v3.1:free',
-                      api_key: ENV['OPENROUTER_API_KEY'])
-end
+toolset = DSPy::Tools::Toolset.new(tools: [SearchTool.new])
+agent = DSPy::ReAct.new(signature: ResearchTask, tools: toolset, max_iterations: 5)
+result = agent.call(question: "What's the latest on Ruby 3.4?")
 ```
 
-## What You Get
-
-**Developer Experience:** Official clients, multimodal coverage, and observability baked in.
-<details>
-<summary>Expand for everything included</summary>
-
-- LLM provider support using official Ruby clients:
-  - [OpenAI Ruby](https://github.com/openai/openai-ruby) with vision model support
-  - [Anthropic Ruby SDK](https://github.com/anthropics/anthropic-sdk-ruby) with multimodal capabilities
-  - [Google Gemini API](https://ai.google.dev/) with native structured outputs
-  - [Ollama](https://ollama.com/) via OpenAI compatibility layer for local models
-- **Multimodal Support** - Complete image analysis with DSPy::Image, type-safe bounding boxes, vision-capable models
-- Runtime type checking with [Sorbet](https://sorbet.org/) including T::Enum and union types
-- Type-safe tool definitions for ReAct agents
-- Comprehensive instrumentation and observability
-</details>
-
-**Core Building Blocks:** Predictors, agents, and pipelines wired through type-safe signatures.
-<details>
-<summary>Expand for everything included</summary>
-
-- **Signatures** - Define input/output schemas using Sorbet types with T::Enum and union type support
-- **Predict** - LLM completion with structured data extraction and multimodal support
-- **Chain of Thought** - Step-by-step reasoning for complex problems with automatic prompt optimization
-- **ReAct** - Tool-using agents with type-safe tool definitions and error recovery
-- **Module Composition** - Combine multiple LLM calls into production-ready workflows
-</details>
-
-**Optimization & Evaluation:** Treat prompt optimization like a real ML workflow.
-<details>
-<summary>Expand for everything included</summary>
-
-- **Prompt Objects** - Manipulate prompts as first-class objects instead of strings
-- **Typed Examples** - Type-safe training data with automatic validation
-- **Evaluation Framework** - Advanced metrics beyond simple accuracy with error-resilient pipelines
-- **MIPROv2 Optimization** - Advanced Bayesian optimization with Gaussian Processes, multiple optimization strategies, auto-config presets, and storage persistence
-</details>
-
-**Production Features:** Hardened behaviors for teams shipping actual products.
-<details>
-<summary>Expand for everything included</summary>
-
-- **Reliable JSON Extraction** - Native structured outputs for OpenAI and Gemini, Anthropic tool-based extraction, and automatic strategy selection with fallback
-- **Type-Safe Configuration** - Strategy enums with automatic provider optimization (Strict/Compatible modes)
-- **Smart Retry Logic** - Progressive fallback with exponential backoff for handling transient failures
-- **Zero-Config Langfuse Integration** - Set env vars and get automatic OpenTelemetry traces in Langfuse
-- **Performance Caching** - Schema and capability caching for faster repeated operations
-- **File-based Storage** - Optimization result persistence with versioning
-- **Structured Logging** - JSON and key=value formats with span tracking
-</details>
-
-## Recent Achievements
-
-DSPy.rb has gone from experimental to production-ready in three fast releases.
-<details>
-<summary>Expand for the full changelog highlights</summary>
-
-### Foundation
-- ✅ **JSON Parsing Reliability** - Native OpenAI structured outputs with adaptive retry logic and schema-aware fallbacks
-- ✅ **Type-Safe Strategy Configuration** - Provider-optimized strategy selection and enum-backed optimizer presets
-- ✅ **Core Module System** - Predict, ChainOfThought, ReAct with type safety (add `dspy-code_act` for Think-Code-Observe agents)
-- ✅ **Production Observability** - OpenTelemetry, New Relic, and Langfuse integration
-- ✅ **Advanced Optimization** - MIPROv2 with Bayesian optimization, Gaussian Processes, and multi-mode search
-
-### Recent Advances
-- ✅ **MIPROv2 ADE Integrity (v0.29.1)** - Stratified train/val/test splits, honest precision accounting, and enum-driven `--auto` presets with integration coverage
-- ✅ **Instruction Deduplication (v0.29.1)** - Candidate generation now filters repeated programs so optimization logs highlight unique strategies
-- ✅ **GEPA Teleprompter (v0.29.0)** - Genetic-Pareto reflective prompt evolution with merge proposer scheduling, reflective mutation, and ADE demo parity
-- ✅ **Optimizer Utilities Parity (v0.29.0)** - Bootstrap strategies, dataset summaries, and Layer 3 utilities unlock multi-predictor programs on Ruby
-- ✅ **Observability Hardening (v0.29.0)** - OTLP exporter runs on a single-thread executor preventing frozen SSL contexts without blocking spans
-- ✅ **Documentation Refresh (v0.29.x)** - New GEPA guide plus ADE optimization docs covering presets, stratified splits, and error-handling defaults
-</details>
-
-**Current Focus Areas:** Closing the loop on production patterns and community adoption ahead of v1.0.
-<details>
-<summary>Expand for the roadmap</summary>
-
-### Production Readiness
-- 🚧 **Production Patterns** - Real-world usage validation and performance optimization
-- 🚧 **Ruby Ecosystem Integration** - Rails integration, Sidekiq compatibility, deployment patterns
-
-### Community & Adoption
-- 🚧 **Community Examples** - Real-world applications and case studies
-- 🚧 **Contributor Experience** - Making it easier to contribute and extend
-- 🚧 **Performance Benchmarks** - Comparative analysis vs other frameworks
-</details>
-
-**v1.0 Philosophy:** v1.0 lands after battle-testing, not checkbox bingo. The API is already stable; the milestone marks production confidence.
+## What's Included
 
+**Core Modules**: Predict, ChainOfThought, ReAct agents, and composable pipelines.
 
-## Documentation
+**Type Safety**: Sorbet-based runtime validation. Enums, unions, nested structs—all work.
+
+**Multimodal**: Image analysis with `DSPy::Image` for vision-capable models.
 
-📖 **[Complete Documentation Website](https://oss.vicente.services/dspy.rb/)**
+**Observability**: Zero-config Langfuse integration via OpenTelemetry. Non-blocking, production-ready.
 
-### LLM-Friendly Documentation
+**Optimization**: MIPROv2 (Bayesian optimization) and GEPA (genetic evolution) for prompt tuning.
 
-For LLMs and AI assistants working with DSPy.rb:
-- **[llms.txt](https://oss.vicente.services/dspy.rb/llms.txt)** - Concise reference optimized for LLMs
-- **[llms-full.txt](https://oss.vicente.services/dspy.rb/llms-full.txt)** - Comprehensive API documentation
+**Provider Support**: OpenAI, Anthropic, Gemini, Ollama, and OpenRouter via official SDKs.
+
+## Documentation
+
+**[Full Documentation](https://oss.vicente.services/dspy.rb/)** — Getting started, core concepts, advanced patterns.
+
+**[llms.txt](https://oss.vicente.services/dspy.rb/llms.txt)** — LLM-friendly reference for AI assistants.
 
 ### Claude Skill
 
-A [Claude Skill](https://github.com/vicentereig/dspy-rb-skill) is available to help you build DSPy.rb applications with Claude Code or claude.ai.
+A [Claude Skill](https://github.com/vicentereig/dspy-rb-skill) is available to help you build DSPy.rb applications:
 
-**Claude Code:**
 ```bash
+# Claude Code
 git clone https://github.com/vicentereig/dspy-rb-skill ~/.claude/skills/dspy-rb
 ```
 
-**Claude.ai (Pro/Max):** Download the [skill as a ZIP](https://github.com/vicentereig/dspy-rb-skill/archive/refs/heads/main.zip) and upload via Settings > Skills.
-
-### Getting Started
-- **[Installation & Setup](docs/src/getting-started/installation.md)** - Detailed installation and configuration
-- **[Quick Start Guide](docs/src/getting-started/quick-start.md)** - Your first DSPy programs
-- **[Core Concepts](docs/src/getting-started/core-concepts.md)** - Understanding signatures, predictors, and modules
+For Claude.ai Pro/Max, download the [skill ZIP](https://github.com/vicentereig/dspy-rb-skill/archive/refs/heads/main.zip) and upload via Settings > Skills.
 
-### Prompt Engineering
-- **[Signatures & Types](docs/src/core-concepts/signatures.md)** - Define typed interfaces for LLM operations
-- **[Predictors](docs/src/core-concepts/predictors.md)** - Predict, ChainOfThought, ReAct, and more
-- **[Modules & Pipelines](docs/src/core-concepts/modules.md)** - Compose complex multi-stage workflows
-- **[Multimodal Support](docs/src/core-concepts/multimodal.md)** - Image analysis with vision-capable models
-- **[Examples & Validation](docs/src/core-concepts/examples.md)** - Type-safe training data
-- **[Rich Types](docs/src/advanced/complex-types.md)** - Sorbet type integration with automatic coercion for structs, enums, and arrays
-- **[Composable Pipelines](docs/src/advanced/pipelines.md)** - Manual module composition patterns
+## Examples
 
-### Prompt Optimization
-- **[Evaluation Framework](docs/src/optimization/evaluation.md)** - Advanced metrics beyond simple accuracy
-- **[Prompt Optimization](docs/src/optimization/prompt-optimization.md)** - Manipulate prompts as objects
-- **[MIPROv2 Optimizer](docs/src/optimization/miprov2.md)** - Advanced Bayesian optimization with Gaussian Processes
-- **[GEPA Optimizer](docs/src/optimization/gepa.md)** *(beta)* - Reflective mutation with optional reflection LMs
+The [examples/](examples/) directory has runnable code for common patterns:
 
-### Context Engineering
-- **[Tools](docs/src/core-concepts/toolsets.md)** - Tool wieldint agents.
-- **[Agentic Memory](docs/src/core-concepts/memory.md)** - Memory Tools & Agentic Loops
-- **[RAG Patterns](docs/src/advanced/rag.md)** - Manual RAG implementation with external services
+- Sentiment classification
+- ReAct agents with tools
+- Image analysis
+- Prompt optimization
 
-### Production Features
-- **[Observability](docs/src/production/observability.md)** - Zero-config Langfuse integration with a dedicated export worker that never blocks your LLMs
-- **[Storage System](docs/src/production/storage.md)** - Persistence and optimization result storage
-- **[Custom Metrics](docs/src/advanced/custom-metrics.md)** - Proc-based evaluation logic 
+```bash
+bundle exec ruby examples/first_predictor.rb
+```
 
+## Optional Gems
 
+DSPy.rb ships sibling gems for features with heavier dependencies. Add them as needed:
 
+| Gem | What it does |
+| --- | --- |
+| `dspy-datasets` | Dataset helpers, Parquet/Polars tooling |
+| `dspy-evals` | Evaluation harness with metrics and callbacks |
+| `dspy-miprov2` | Bayesian optimization for prompt tuning |
+| `dspy-gepa` | Genetic-Pareto prompt evolution |
+| `dspy-o11y-langfuse` | Auto-configure Langfuse tracing |
+| `dspy-code_act` | Think-Code-Observe agents |
+| `dspy-deep_search` | Production DeepSearch with Exa |
 
+See [the full list](https://oss.vicente.services/dspy.rb/getting-started/installation/) in the docs.
 
+## Contributing
 
+Feedback is invaluable. If you encounter issues, [open an issue](https://github.com/vicentereig/dspy.rb/issues). For suggestions, [start a discussion](https://github.com/vicentereig/dspy.rb/discussions).
 
+Want to contribute code? Reach out: hey at vicente.services
 
 ## License
-This project is licensed under the MIT License.
+
+MIT License.

data/lib/dspy/chain_of_thought.rb

@@ -47,7 +47,8 @@ module DSPy
         output_schema: @signature_class.output_json_schema,
         few_shot_examples: new_prompt.few_shot_examples,
         signature_class_name: @signature_class.name,
-        schema_format: new_prompt.schema_format
+        schema_format: new_prompt.schema_format,
+        data_format: new_prompt.data_format
       )
       
       instance.instance_variable_set(:@prompt, enhanced_prompt)
@@ -93,7 +94,7 @@ module DSPy
       
       # Create a temporary Predict instance with our enhanced signature to get the prediction
       predict_instance = DSPy::Predict.new(@signature_class)
-      predict_instance.config.lm = self.lm  # Use the same LM configuration
+      predict_instance.configure { |c| c.lm = self.lm }  # Use the same LM configuration
       
       # Call predict's forward method, which will create the Predict span
       prediction_result = predict_instance.forward(**input_values)

data/lib/dspy/context.rb

@@ -7,43 +7,45 @@ module DSPy
   class Context
     class << self
       def current
-        # Use Thread storage as primary source to ensure thread isolation
-        # Fiber storage is used for OpenTelemetry context propagation within the same thread
-        
-        # Create a unique key for this thread to ensure isolation
-        thread_key = :"dspy_context_#{Thread.current.object_id}"
-        
-        # Always check thread-local storage first for proper isolation
-        if Thread.current[thread_key]
-          # Thread has context, ensure fiber inherits it for OpenTelemetry propagation
-          Fiber[:dspy_context] = Thread.current[thread_key]
-          Thread.current[:dspy_context] = Thread.current[thread_key]  # Keep for backward compatibility
-          return Thread.current[thread_key]
+        # Prefer fiber-local context for async safety; fall back to thread root context.
+        fiber_context = Fiber[:dspy_context]
+        if fiber_context && fiber_context[:thread_id] == Thread.current.object_id
+          return fiber_context if fiber_context[:fiber_id] == Fiber.current.object_id
+
+          Fiber[:dspy_context] = fork_context(fiber_context)
+          return Fiber[:dspy_context]
         end
-        
-        # Check if current fiber has context that was set by this same thread
-        # This handles cases where context was set via OpenTelemetry propagation within the thread
-        if Fiber[:dspy_context] && Thread.current[:dspy_context] == Fiber[:dspy_context]
-          # This fiber context was set by this thread, safe to use
-          Thread.current[thread_key] = Fiber[:dspy_context]
+
+        thread_key = :"dspy_context_#{Thread.current.object_id}"
+        thread_context = Thread.current[thread_key]
+
+        if thread_context
+          Fiber[:dspy_context] = fork_context(thread_context)
           return Fiber[:dspy_context]
         end
-        
-        # No existing context or context belongs to different thread - create new one
-        context = {
-          trace_id: SecureRandom.uuid,
-          span_stack: [],
-          otel_span_stack: [],
-          module_stack: []
-        }
-        
-        # Set in both Thread and Fiber storage
+
+        context = build_context
         Thread.current[thread_key] = context
-        Thread.current[:dspy_context] = context  # Keep for backward compatibility
+        Thread.current[:dspy_context] = context  # Backward compatibility (thread root)
         Fiber[:dspy_context] = context
-        
         context
       end
+
+      def with_request(request_id, start_time)
+        previous_request_id = current[:request_id]
+        previous_start_time = current[:request_start_time]
+
+        current[:request_id] = request_id
+        current[:request_start_time] = start_time
+        yield
+      ensure
+        current[:request_id] = previous_request_id
+        current[:request_start_time] = previous_start_time
+      end
+
+      def fork_context(parent_context)
+        clone_context(parent_context)
+      end
       
       def with_span(operation:, **attributes)
         span_id = SecureRandom.uuid
@@ -219,6 +221,31 @@ module DSPy
         false
       end
 
+      def build_context
+        {
+          trace_id: SecureRandom.uuid,
+          thread_id: Thread.current.object_id,
+          fiber_id: Fiber.current.object_id,
+          span_stack: [],
+          otel_span_stack: [],
+          module_stack: [],
+          request_id: nil,
+          request_start_time: nil
+        }
+      end
+
+      def clone_context(context)
+        cloned = context.dup
+        cloned[:span_stack] = Array(context[:span_stack]).dup
+        cloned[:otel_span_stack] = Array(context[:otel_span_stack]).dup
+        cloned[:module_stack] = Array(context[:module_stack]).map { |entry| entry.dup }
+        cloned[:thread_id] = Thread.current.object_id
+        cloned[:fiber_id] = Fiber.current.object_id
+        cloned[:request_id] = context[:request_id]
+        cloned[:request_start_time] = context[:request_start_time]
+        cloned
+      end
+
       def sanitize_span_attributes(attributes)
         attributes.each_with_object({}) do |(key, value), acc|
           sanitized_value = sanitize_attribute_value(value)

data/lib/dspy/evals/version.rb

@@ -2,6 +2,6 @@
 
 module DSPy
   class Evals
-    VERSION = '1.0.1'
+    VERSION = '1.0.2'
   end
 end