open_router_enhanced 1.0.0 → 1.1.0

data/docs/observability.md

@@ -104,12 +104,15 @@ end
 client.on(:on_healing) do |healing_data|
   if healing_data[:healed]
     puts "Successfully healed JSON response"
+    puts "Attempts: #{healing_data[:attempts]}"
   else
     puts "JSON healing failed: #{healing_data[:error]}"
   end
 end
 ```
 
+**Note**: For detailed information about when auto-healing triggers, how it works, and configuration options, see the [Structured Outputs documentation](structured_outputs.md#json-auto-healing).
+
 ### 4. Streaming Observability
 Enhanced streaming support with detailed event callbacks:
 

data/docs/plugins.md

@@ -0,0 +1,183 @@
+# OpenRouter Plugins
+
+OpenRouter provides plugins that extend model capabilities. The gem supports all OpenRouter plugins and automatically enables response healing for structured outputs.
+
+## Available Plugins
+
+| Plugin | ID | Description |
+|--------|-----|-------------|
+| Response Healing | `response-healing` | Fixes malformed JSON responses |
+| Web Search | `web-search` | Augments responses with real-time web search |
+| PDF Inputs | `pdf-inputs` | Parses and extracts content from PDF files |
+
+## Basic Usage
+
+```ruby
+# Specify plugins in your request
+response = client.complete(
+  messages,
+  model: "openai/gpt-4o-mini",
+  plugins: [{ id: "web-search" }]
+)
+
+# Multiple plugins
+response = client.complete(
+  messages,
+  model: "openai/gpt-4o-mini",
+  plugins: [
+    { id: "web-search" },
+    { id: "pdf-inputs" }
+  ]
+)
+```
+
+## Response Healing Plugin
+
+The response-healing plugin fixes common JSON formatting issues server-side:
+
+- Missing brackets, commas, and quotes
+- Trailing commas
+- Markdown-wrapped JSON
+- Text mixed with JSON
+- Unquoted object keys
+
+### Automatic Activation
+
+The gem **automatically adds** the response-healing plugin when:
+1. Using structured outputs (`response_format` is set)
+2. Not streaming
+3. `auto_native_healing` is enabled (default: true)
+
+```ruby
+# Response-healing is automatically added here
+response = client.complete(
+  messages,
+  model: "openai/gpt-4o-mini",
+  response_format: schema
+)
+```
+
+### Disable Automatic Healing
+
+```ruby
+# Via configuration
+OpenRouter.configure do |config|
+  config.auto_native_healing = false
+end
+
+# Via environment variable
+# OPENROUTER_AUTO_NATIVE_HEALING=false
+```
+
+### Manual Control
+
+```ruby
+# Explicitly add response-healing
+response = client.complete(
+  messages,
+  model: "openai/gpt-4o-mini",
+  plugins: [{ id: "response-healing" }],
+  response_format: { type: "json_object" }
+)
+
+# Disable for a specific request (when auto is enabled)
+response = client.complete(
+  messages,
+  model: "openai/gpt-4o-mini",
+  plugins: [{ id: "response-healing", enabled: false }],
+  response_format: schema
+)
+```
+
+### Limitations
+
+- **Non-streaming only**: Does not work with `stream: proc`
+- **Syntax only**: Fixes JSON syntax, not schema conformance
+- **Truncation issues**: May fail if response was cut off by `max_tokens`
+
+For schema validation failures, use the gem's [client-side auto-healing](structured_outputs.md#json-auto-healing-client-side).
+
+## Web Search Plugin
+
+Augments model responses with real-time web search results.
+
+```ruby
+response = client.complete(
+  [{ role: "user", content: "What are the latest AI developments?" }],
+  model: "openai/gpt-4o-mini",
+  plugins: [{ id: "web-search" }]
+)
+```
+
+**Shortcut**: Append `:online` to model ID:
+
+```ruby
+response = client.complete(
+  messages,
+  model: "openai/gpt-4o-mini:online"  # Enables web-search
+)
+```
+
+## PDF Inputs Plugin
+
+Enables models to process PDF file content.
+
+```ruby
+response = client.complete(
+  [{ role: "user", content: "Summarize this PDF: [pdf content]" }],
+  model: "openai/gpt-4o-mini",
+  plugins: [{ id: "pdf-inputs" }]
+)
+```
+
+## Plugin Configuration Options
+
+Plugins can accept additional configuration:
+
+```ruby
+# Enable/disable a plugin explicitly
+plugins: [{ id: "response-healing", enabled: true }]
+
+# Disable a default plugin for one request
+plugins: [{ id: "response-healing", enabled: false }]
+```
+
+## Prediction Parameter (Latency Optimization)
+
+The `prediction` parameter reduces latency by providing the model with an expected output:
+
+```ruby
+response = client.complete(
+  [{ role: "user", content: "What is the capital of France?" }],
+  model: "openai/gpt-4o",
+  prediction: { type: "content", content: "The capital of France is Paris." }
+)
+```
+
+**When to use**:
+- Code completion with predictable boilerplate
+- Template filling where most content is known
+- Minor corrections/refinements to existing text
+
+**How it works**: Instead of generating from scratch, the model confirms/refines your prediction, which is faster when accurate.
+
+## Best Practices
+
+1. **Use native healing for structured outputs**: It's free and adds <1ms latency
+2. **Don't combine response-healing with streaming**: It won't work
+3. **Check model compatibility**: Not all models support all plugins
+4. **Monitor costs**: Web search may add to response latency
+
+## Comparison: Native vs Client-Side Healing
+
+| Aspect | Native (Plugin) | Client-Side (Gem) |
+|--------|-----------------|-------------------|
+| Location | Server-side | Client-side |
+| Cost | Free | API call per attempt |
+| Latency | <1ms | Full LLM call |
+| Fixes syntax | Yes | Yes |
+| Fixes schema | No | Yes |
+| Streaming | No | Yes |
+| Auto-enabled | For structured outputs | When `auto_heal_responses = true` |
+
+**Recommendation**: Use both! Native healing catches 80%+ of issues for free. Client-side healing handles the rest and validates against your schema.

data/docs/streaming.md

@@ -214,7 +214,7 @@ end
 
 ## Structured Outputs with Streaming
 
-Streaming works seamlessly with structured outputs:
+Streaming works seamlessly with structured outputs. The response is streamed in real-time, then validated and parsed after accumulation completes.
 
 ```ruby
 # Define schema
@@ -225,18 +225,33 @@ user_schema = OpenRouter::Schema.define("user") do
 end
 
 # Stream with structured output
+# IMPORTANT: accumulate_response must be true for structured outputs
 response = streaming_client.stream_complete(
   [{ role: "user", content: "Create a user: John Doe, 30, john@example.com" }],
   model: "openai/gpt-4o",
   response_format: user_schema,
-  accumulate_response: true
+  accumulate_response: true  # Required for structured_output access
 )
 
-# Access structured output after streaming
+# Access structured output after streaming completes
 user_data = response.structured_output
 puts "User: #{user_data['name']}, Age: #{user_data['age']}"
 ```
 
+### How Structured Outputs Work with Streaming
+
+1. **During Streaming**: Content chunks are streamed and displayed in real-time
+2. **After Accumulation**: The complete response is validated against your schema
+3. **Auto-Healing**: If enabled and needed, healing occurs after streaming completes
+4. **Validation**: Schema validation happens on the accumulated response
+
+**Important Notes:**
+- You must set `accumulate_response: true` to use `response.structured_output`
+- Auto-healing (if configured) happens after streaming completes, not during streaming
+- The `on_finish` callback receives the final, validated response
+
+For detailed information on auto-healing, native vs forced outputs, and troubleshooting, see the [Structured Outputs documentation](structured_outputs.md).
+
 ## Configuration Options
 
 The streaming client accepts all the same configuration options as the regular client: