open_router_enhanced 1.0.0 → 1.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/README.md CHANGED
@@ -2,26 +2,26 @@
2
2
 
3
3
  The future will bring us hundreds of language models and dozens of providers for each. How will you choose the best?
4
4
 
5
- The [OpenRouter API](https://openrouter.ai/docs) is a single unified interface for all LLMs! And now you can easily use it with Ruby! 🤖🌌
5
+ The [OpenRouter API](https://openrouter.ai/docs) is a single unified interface for all LLMs, accessible through an idiomatic Ruby interface.
6
6
 
7
7
  **OpenRouter Enhanced** is an advanced fork of the [original OpenRouter Ruby gem](https://github.com/OlympiaAI/open_router) by [Obie Fernandez](https://github.com/obie) that adds comprehensive AI application development features including tool calling, structured outputs, intelligent model selection, prompt templates, observability, and automatic response healing—all while maintaining full backward compatibility.
8
8
 
9
- 📖 **[Read the story behind OpenRouter Enhanced](https://lowlevelmagic.io/writings/why-i-built-open-router-enhanced/)** - Learn why this gem was built and the philosophy behind its design.
9
+ **[Read the story behind OpenRouter Enhanced](https://lowlevelmagic.io/writings/why-i-built-open-router-enhanced/)** - Learn why this gem was built and the philosophy behind its design.
10
10
 
11
11
  ## Enhanced Features
12
12
 
13
- This fork extends the original OpenRouter gem with enterprise-grade AI development capabilities:
14
-
15
13
  ### Core AI Features
16
- - **Tool Calling**: Full support for OpenRouter's function calling API with Ruby-idiomatic DSL for tool definitions
17
- - **Structured Outputs**: JSON Schema validation with automatic healing for non-native models and Ruby DSL for schema definitions
18
- - **Smart Model Selection**: Intelligent model selection with fluent DSL for cost optimization, capability requirements, and provider preferences
19
- - **Prompt Templates**: Reusable prompt templates with variable interpolation and few-shot learning support
14
+ - **[Tool Calling](docs/tools.md)**: Full support for OpenRouter's function calling API with Ruby-idiomatic DSL
15
+ - **[Structured Outputs](docs/structured_outputs.md)**: JSON Schema validation with automatic healing for non-native models
16
+ - **[Smart Model Selection](docs/model_selection.md)**: Intelligent model selection with cost optimization and capability matching
17
+ - **[Prompt Templates](docs/prompt_templates.md)**: Reusable prompt templates with variable interpolation and few-shot learning
18
+ - **[Plugins](docs/plugins.md)**: OpenRouter plugins for web search, PDF inputs, and native response healing
20
19
 
21
20
  ### Performance & Reliability
22
21
  - **Model Registry**: Local caching and querying of OpenRouter model data with capability detection
23
22
  - **Enhanced Response Handling**: Rich Response objects with automatic parsing for tool calls and structured outputs
24
- - **Automatic Healing**: Self-healing responses for malformed JSON from models that don't natively support structured outputs
23
+ - **Native Response Healing**: Automatic server-side JSON repair via OpenRouter's response-healing plugin
24
+ - **Client-Side Healing**: Self-healing responses for schema validation failures using LLM-based repair
25
25
  - **Model Fallbacks**: Automatic failover between models with graceful degradation
26
26
  - **Streaming Support**: Enhanced streaming client with callback system and response reconstruction
27
27
 
@@ -31,50 +31,26 @@ This fork extends the original OpenRouter gem with enterprise-grade AI developme
31
31
  - **Callback System**: Extensible event system for monitoring requests, responses, and errors
32
32
  - **Cost Management**: Built-in cost estimation and budget constraints
33
33
 
34
- ### Development & Testing
35
- - **Comprehensive Testing**: VCR-based integration tests with real API interactions
36
- - **Debug Support**: Detailed error reporting and validation feedback
37
- - **Configuration Options**: Extensive configuration for healing, validation, and performance tuning
38
- - **Backward Compatible**: All existing code continues to work unchanged
39
-
40
34
  ### Core OpenRouter Benefits
41
-
42
- - **Prioritize price or performance**: OpenRouter scouts for the lowest prices and best latencies/throughputs across dozens of providers, and lets you choose how to prioritize them.
43
- - **Standardized API**: No need to change your code when switching between models or providers. You can even let users choose and pay for their own.
44
- - **Easy integration**: This Ruby gem provides a simple and intuitive interface to interact with the OpenRouter API, making it effortless to integrate AI capabilities into your Ruby applications.
35
+ - **Prioritize price or performance**: OpenRouter scouts for the lowest prices and best latencies/throughputs across dozens of providers
36
+ - **Standardized API**: No need to change your code when switching between models or providers
37
+ - **Easy integration**: Simple and intuitive Ruby interface for AI capabilities
45
38
 
46
39
  ## Table of Contents
47
40
 
48
41
  - [Installation](#installation)
49
42
  - [Quick Start](#quick-start)
50
43
  - [Configuration](#configuration)
51
- - [Core Features](#core-features)
52
- - [Basic Completions](#basic-completions)
53
- - [Model Selection](#model-selection)
54
- - [Enhanced AI Features](#enhanced-ai-features)
44
+ - [Features](#features)
55
45
  - [Tool Calling](#tool-calling)
56
46
  - [Structured Outputs](#structured-outputs)
57
47
  - [Smart Model Selection](#smart-model-selection)
58
48
  - [Prompt Templates](#prompt-templates)
59
- - [Streaming & Real-time](#streaming--real-time)
60
- - [Streaming Client](#streaming-client)
61
- - [Streaming Callbacks](#streaming-callbacks)
62
- - [Observability & Analytics](#observability--analytics)
49
+ - [Streaming](#streaming)
63
50
  - [Usage Tracking](#usage-tracking)
64
- - [Response Analytics](#response-analytics)
65
- - [Callback System](#callback-system)
66
- - [Cost Management](#cost-management)
67
- - [Advanced Features](#advanced-features)
68
- - [Model Registry](#model-registry)
69
- - [Model Fallbacks](#model-fallbacks)
70
- - [Response Healing](#response-healing)
71
- - [Performance Optimization](#performance-optimization)
72
- - [Testing & Development](#testing--development)
73
- - [Running Tests](#running-tests)
74
- - [VCR Testing](#vcr-testing)
75
- - [Examples](#examples)
51
+ - [Model Exploration](#model-exploration)
52
+ - [Testing](#testing)
76
53
  - [Troubleshooting](#troubleshooting)
77
- - [API Reference](#api-reference)
78
54
  - [Contributing](#contributing)
79
55
  - [License](#license)
80
56
 
@@ -102,19 +78,13 @@ Or install it directly:
102
78
  gem install open_router_enhanced
103
79
  ```
104
80
 
105
- And require it in your code:
106
-
107
- ```ruby
108
- require "open_router"
109
- ```
110
-
111
81
  ## Quick Start
112
82
 
113
83
  ### 1. Get Your API Key
114
84
  - Sign up at [OpenRouter](https://openrouter.ai)
115
85
  - Get your API key from [https://openrouter.ai/keys](https://openrouter.ai/keys)
116
86
 
117
- ### 2. Basic Setup and Usage
87
+ ### 2. Basic Setup
118
88
 
119
89
  ```ruby
120
90
  require "open_router"
@@ -138,16 +108,16 @@ puts response.content
138
108
  # => "The capital of France is Paris."
139
109
  ```
140
110
 
141
- ### 3. Enhanced Features Quick Example
111
+ ### 3. Enhanced Features Example
142
112
 
143
113
  ```ruby
144
- # Smart model selection
114
+ # Smart model selection with capabilities
145
115
  model = OpenRouter::ModelSelector.new
146
- .require(:function_calling)
147
- .optimize_for(:cost)
148
- .choose
116
+ .require(:function_calling)
117
+ .optimize_for(:cost)
118
+ .choose
149
119
 
150
- # Tool calling with structured output
120
+ # Define a tool
151
121
  weather_tool = OpenRouter::Tool.define do
152
122
  name "get_weather"
153
123
  description "Get current weather"
@@ -156,12 +126,14 @@ weather_tool = OpenRouter::Tool.define do
156
126
  end
157
127
  end
158
128
 
129
+ # Define a schema for structured output
159
130
  weather_schema = OpenRouter::Schema.define("weather") do
160
131
  string :location, required: true
161
132
  number :temperature, required: true
162
133
  string :conditions, required: true
163
134
  end
164
135
 
136
+ # Use everything together
165
137
  response = client.complete(
166
138
  [{ role: "user", content: "What's the weather in Tokyo?" }],
167
139
  model: model,
@@ -178,149 +150,66 @@ end
178
150
 
179
151
  ## Configuration
180
152
 
181
- ### Global Configuration
182
-
183
- Configure the gem globally, for example in an `open_router.rb` initializer file. Never hardcode secrets into your codebase - instead use `Rails.application.credentials` or something like [dotenv](https://github.com/motdotla/dotenv) to pass the keys safely into your environments.
153
+ Configure the gem globally (e.g., in an initializer):
184
154
 
185
155
  ```ruby
186
156
  OpenRouter.configure do |config|
157
+ # Required
187
158
  config.access_token = ENV["OPENROUTER_API_KEY"]
188
159
  config.site_name = "Your App Name"
189
160
  config.site_url = "https://yourapp.com"
190
-
191
- # Optional: Configure response healing for non-native structured output models
161
+
162
+ # Optional: Response healing for non-native structured output models
192
163
  config.auto_heal_responses = true
193
164
  config.healer_model = "openai/gpt-4o-mini"
194
165
  config.max_heal_attempts = 2
195
-
196
- # Optional: Configure strict mode for capability validation
166
+
167
+ # Optional: Strict mode for capability validation
197
168
  config.strict_mode = true
198
-
199
- # Optional: Configure automatic forcing for unsupported models
200
- config.auto_force_on_unsupported_models = true
169
+
170
+ # Optional: Request timeout (default: 120 seconds)
171
+ config.request_timeout = 120
201
172
  end
202
173
  ```
203
174
 
204
175
  ### Per-Client Configuration
205
176
 
206
- You can also configure clients individually:
207
-
208
177
  ```ruby
209
178
  client = OpenRouter::Client.new(
210
179
  access_token: ENV["OPENROUTER_API_KEY"],
211
- request_timeout: 120
180
+ request_timeout: 240
212
181
  )
213
182
  ```
214
183
 
215
184
  ### Faraday Configuration
216
185
 
217
- The configuration object exposes a [`faraday`](https://github.com/lostisland/faraday-retry) method that you can pass a block to configure Faraday settings and middleware.
218
-
219
- This example adds `faraday-retry` and a logger that redacts the api key so it doesn't get leaked to logs.
186
+ Configure Faraday middleware for retries, logging, etc:
220
187
 
221
188
  ```ruby
222
189
  require 'faraday/retry'
223
190
 
224
- retry_options = {
225
- max: 2,
226
- interval: 0.05,
227
- interval_randomness: 0.5,
228
- backoff_factor: 2
229
- }
230
-
231
191
  OpenRouter::Client.new(access_token: ENV["ACCESS_TOKEN"]) do |config|
232
192
  config.faraday do |f|
233
- f.request :retry, retry_options
234
- f.response :logger, ::Logger.new($stdout), { headers: true, bodies: true, errors: true } do |logger|
235
- logger.filter(/(Bearer) (\S+)/, '\1[REDACTED]')
236
- end
193
+ f.request :retry, max: 2, interval: 0.05
194
+ f.response :logger, ::Logger.new($stdout), { headers: true, bodies: true }
237
195
  end
238
196
  end
239
197
  ```
240
198
 
241
- #### Change version or timeout
242
-
243
- The default timeout for any request using this library is 120 seconds. You can change that by passing a number of seconds to the `request_timeout` when initializing the client.
244
-
245
- ```ruby
246
- client = OpenRouter::Client.new(
247
- access_token: "access_token_goes_here",
248
- request_timeout: 240 # Optional
249
- )
250
- ```
251
-
252
- ## Core Features
253
-
254
- ### Basic Completions
255
-
256
- Hit the OpenRouter API for a completion:
257
-
258
- ```ruby
259
- messages = [
260
- { role: "system", content: "You are a helpful assistant." },
261
- { role: "user", content: "What is the color of the sky?" }
262
- ]
263
-
264
- response = client.complete(messages)
265
- puts response.content
266
- # => "The sky is typically blue during the day due to a phenomenon called Rayleigh scattering. Sunlight..."
267
- ```
268
-
269
- ### Model Selection
270
-
271
- Pass an array to the `model` parameter to enable [explicit model routing](https://openrouter.ai/docs#model-routing).
272
-
273
- ```ruby
274
- OpenRouter::Client.new.complete(
275
- [
276
- { role: "system", content: SYSTEM_PROMPT },
277
- { role: "user", content: "Provide analysis of the data formatted as JSON:" }
278
- ],
279
- model: [
280
- "mistralai/mixtral-8x7b-instruct:nitro",
281
- "mistralai/mixtral-8x7b-instruct"
282
- ],
283
- extras: {
284
- response_format: {
285
- type: "json_object"
286
- }
287
- }
288
- )
289
- ```
199
+ **[Full configuration documentation](docs/configuration.md)**
290
200
 
291
- [Browse full list of models available](https://openrouter.ai/models) or fetch from the OpenRouter API:
292
-
293
- ```ruby
294
- models = client.models
295
- puts models
296
- # => [{"id"=>"openrouter/auto", "object"=>"model", "created"=>1684195200, "owned_by"=>"openrouter", "permission"=>[], "root"=>"openrouter", "parent"=>nil}, ...]
297
- ```
298
-
299
- ### Generation Stats
300
-
301
- Query the generation stats for a given generation ID:
302
-
303
- ```ruby
304
- generation_id = "generation-abcdefg"
305
- stats = client.query_generation_stats(generation_id)
306
- puts stats
307
- # => {"id"=>"generation-abcdefg", "object"=>"generation", "created"=>1684195200, "model"=>"openrouter/auto", "usage"=>{"prompt_tokens"=>10, "completion_tokens"=>50, "total_tokens"=>60}, "cost"=>0.0006}
308
- ```
309
-
310
- ## Enhanced AI Features
201
+ ## Features
311
202
 
312
203
  ### Tool Calling
313
204
 
314
- Enable AI models to call functions and interact with external APIs using OpenRouter's function calling with an intuitive Ruby DSL.
315
-
316
- #### Quick Example
205
+ Enable AI models to call functions and interact with external APIs.
317
206
 
318
207
  ```ruby
319
- # Define a tool using the DSL
208
+ # Define a tool with the DSL
320
209
  weather_tool = OpenRouter::Tool.define do
321
210
  name "get_weather"
322
211
  description "Get current weather for a location"
323
-
212
+
324
213
  parameters do
325
214
  string :location, required: true, description: "City name"
326
215
  string :units, enum: ["celsius", "fahrenheit"], default: "celsius"
@@ -331,41 +220,30 @@ end
331
220
  response = client.complete(
332
221
  [{ role: "user", content: "What's the weather in London?" }],
333
222
  model: "anthropic/claude-3.5-sonnet",
334
- tools: [weather_tool],
335
- tool_choice: "auto"
223
+ tools: [weather_tool]
336
224
  )
337
225
 
338
226
  # Handle tool calls
339
227
  if response.has_tool_calls?
340
228
  response.tool_calls.each do |tool_call|
341
- result = fetch_weather(tool_call.arguments["location"], tool_call.arguments["units"])
342
- puts "Weather in #{tool_call.arguments['location']}: #{result}"
229
+ result = fetch_weather(tool_call.arguments["location"])
230
+ puts "Weather: #{result}"
343
231
  end
344
232
  end
345
233
  ```
346
234
 
347
- #### Key Features
348
-
349
- - **Ruby DSL**: Define tools with intuitive Ruby syntax
350
- - **Parameter Validation**: Automatic validation against JSON Schema
351
- - **Tool Choice Control**: Auto, required, none, or specific tool selection
352
- - **Conversation Continuation**: Easy message building for multi-turn conversations
353
- - **Error Handling**: Graceful error handling and validation
354
-
355
- 📖 **[Complete Tool Calling Documentation](docs/tools.md)**
235
+ **[Complete Tool Calling Documentation](docs/tools.md)**
356
236
 
357
237
  ### Structured Outputs
358
238
 
359
- Get JSON responses that conform to specific schemas with automatic validation and healing for non-native models.
360
-
361
- #### Quick Example
239
+ Get JSON responses that conform to specific schemas with automatic validation and healing.
362
240
 
363
241
  ```ruby
364
- # Define a schema using the DSL
242
+ # Define a schema
365
243
  user_schema = OpenRouter::Schema.define("user") do
366
244
  string :name, required: true, description: "Full name"
367
245
  integer :age, required: true, minimum: 0, maximum: 150
368
- string :email, required: true, description: "Email address"
246
+ string :email, required: true
369
247
  boolean :premium, description: "Premium account status"
370
248
  end
371
249
 
@@ -376,78 +254,63 @@ response = client.complete(
376
254
  response_format: user_schema
377
255
  )
378
256
 
379
- # Access parsed JSON data
257
+ # Access parsed JSON
380
258
  user = response.structured_output
381
- puts user["name"] # => "John Doe"
382
- puts user["age"] # => 30
383
- puts user["email"] # => "john@example.com"
259
+ puts "#{user['name']} (#{user['age']}) - #{user['email']}"
384
260
  ```
385
261
 
386
- #### Key Features
387
-
388
- - **Ruby DSL**: Define JSON schemas with Ruby syntax
389
- - **Automatic Healing**: Self-healing for models without native structured output support
390
- - **Validation**: Optional validation with detailed error reporting
391
- - **Complex Schemas**: Support for nested objects, arrays, and advanced constraints
392
- - **Fallback Support**: Graceful degradation for unsupported models
262
+ **Key Features:**
263
+ - Ruby DSL for JSON schemas
264
+ - Automatic healing for models without native support
265
+ - Validation with detailed error reporting
266
+ - Support for nested objects and arrays
393
267
 
394
- 📖 **[Complete Structured Outputs Documentation](docs/structured_outputs.md)**
268
+ **[Complete Structured Outputs Documentation](docs/structured_outputs.md)**
395
269
 
396
270
  ### Smart Model Selection
397
271
 
398
- Automatically choose the best AI model based on your specific requirements using a fluent DSL.
399
-
400
- #### Quick Example
272
+ Automatically choose the best model based on requirements.
401
273
 
402
274
  ```ruby
403
- # Find the cheapest model with function calling
275
+ # Find cheapest model with function calling
404
276
  model = OpenRouter::ModelSelector.new
405
- .require(:function_calling)
406
- .optimize_for(:cost)
407
- .choose
277
+ .require(:function_calling)
278
+ .optimize_for(:cost)
279
+ .choose
408
280
 
409
281
  # Advanced selection with multiple criteria
410
282
  model = OpenRouter::ModelSelector.new
411
- .require(:function_calling, :vision)
412
- .within_budget(max_cost: 0.01)
413
- .min_context(50_000)
414
- .prefer_providers("anthropic", "openai")
415
- .optimize_for(:performance)
416
- .choose
417
-
418
- # Get multiple options with fallbacks
283
+ .require(:function_calling, :vision)
284
+ .within_budget(max_cost: 0.01)
285
+ .min_context(50_000)
286
+ .prefer_providers("anthropic", "openai")
287
+ .optimize_for(:performance)
288
+ .choose
289
+
290
+ # Get multiple options for fallbacks
419
291
  models = OpenRouter::ModelSelector.new
420
- .require(:structured_outputs)
421
- .choose_with_fallbacks(limit: 3)
292
+ .require(:structured_outputs)
293
+ .choose_with_fallbacks(limit: 3)
422
294
  # => ["openai/gpt-4o-mini", "anthropic/claude-3-haiku", "google/gemini-flash"]
423
295
  ```
424
296
 
425
- #### Key Features
297
+ **Available capabilities:** `:chat`, `:function_calling`, `:structured_outputs`, `:vision`, `:code_generation`
426
298
 
427
- - **Fluent DSL**: Chain requirements and preferences intuitively
428
- - **Cost Optimization**: Find models within budget constraints
429
- - **Capability Matching**: Require specific features like function calling or vision
430
- - **Provider Preferences**: Prefer or avoid specific providers
431
- - **Graceful Fallbacks**: Automatic fallback with requirement relaxation
432
- - **Performance Tiers**: Choose between cost and performance optimization
299
+ **Optimization strategies:** `:cost`, `:performance`, `:latest`, `:context`
433
300
 
434
- 📖 **[Complete Model Selection Documentation](docs/model_selection.md)**
301
+ **[Complete Model Selection Documentation](docs/model_selection.md)**
435
302
 
436
303
  ### Prompt Templates
437
304
 
438
- Create reusable, parameterized prompts with variable interpolation and few-shot learning support.
439
-
440
- #### Quick Example
305
+ Create reusable, parameterized prompts with variable interpolation.
441
306
 
442
307
  ```ruby
443
- # Basic template with variables
308
+ # Basic template
444
309
  translation_template = OpenRouter::PromptTemplate.new(
445
310
  template: "Translate '{text}' from {source_lang} to {target_lang}",
446
311
  input_variables: [:text, :source_lang, :target_lang]
447
312
  )
448
313
 
449
- # Use with client
450
- client = OpenRouter::Client.new
451
314
  response = client.complete(
452
315
  translation_template.to_messages(
453
316
  text: "Hello world",
@@ -459,505 +322,106 @@ response = client.complete(
459
322
 
460
323
  # Few-shot learning template
461
324
  classification_template = OpenRouter::PromptTemplate.new(
462
- prefix: "Classify the sentiment of the following text. Examples:",
325
+ prefix: "Classify the sentiment:",
463
326
  suffix: "Now classify: {text}",
464
327
  examples: [
465
- { text: "I love this product!", sentiment: "positive" },
466
- { text: "This is terrible.", sentiment: "negative" },
467
- { text: "It's okay, nothing special.", sentiment: "neutral" }
328
+ { text: "I love this!", sentiment: "positive" },
329
+ { text: "This is terrible.", sentiment: "negative" }
468
330
  ],
469
331
  example_template: "Text: {text}\nSentiment: {sentiment}",
470
332
  input_variables: [:text]
471
333
  )
472
-
473
- # Render complete prompt
474
- prompt = classification_template.format(text: "This is amazing!")
475
- puts prompt
476
- # =>
477
- # Classify the sentiment of the following text. Examples:
478
- #
479
- # Text: I love this product!
480
- # Sentiment: positive
481
- #
482
- # Text: This is terrible.
483
- # Sentiment: negative
484
- #
485
- # Text: It's okay, nothing special.
486
- # Sentiment: neutral
487
- #
488
- # Now classify: This is amazing!
489
334
  ```
490
335
 
491
- #### Key Features
336
+ **[Complete Prompt Templates Documentation](docs/prompt_templates.md)**
492
337
 
493
- - **Variable Interpolation**: Use `{variable}` syntax for dynamic content
494
- - **Few-Shot Learning**: Include examples to improve model performance
495
- - **Chat Formatting**: Automatic conversion to OpenRouter message format
496
- - **Partial Variables**: Pre-fill common variables for reuse
497
- - **Template Composition**: Combine templates for complex prompts
498
- - **Validation**: Automatic validation of required input variables
338
+ ### Streaming
499
339
 
500
- 📖 **[Complete Prompt Templates Documentation](docs/prompt_templates.md)**
501
-
502
- ### Model Registry
503
-
504
- Access detailed information about available models and their capabilities.
505
-
506
- #### Quick Example
340
+ Stream responses in real-time with callbacks and automatic response reconstruction.
507
341
 
508
342
  ```ruby
509
- # Get specific model information
510
- model_info = OpenRouter::ModelRegistry.get_model_info("anthropic/claude-3-5-sonnet")
511
- puts model_info[:capabilities] # [:chat, :function_calling, :structured_outputs, :vision]
512
- puts model_info[:cost_per_1k_tokens] # { input: 0.003, output: 0.015 }
513
-
514
- # Find models matching requirements
515
- candidates = OpenRouter::ModelRegistry.models_meeting_requirements(
516
- capabilities: [:function_calling],
517
- max_input_cost: 0.01
518
- )
519
-
520
- # Estimate costs for specific usage
521
- cost = OpenRouter::ModelRegistry.calculate_estimated_cost(
522
- "openai/gpt-4o",
523
- input_tokens: 1000,
524
- output_tokens: 500
525
- )
526
- puts "Estimated cost: $#{cost.round(4)}" # => "Estimated cost: $0.0105"
527
- ```
528
-
529
- #### Key Features
530
-
531
- - **Model Discovery**: Browse all available models and their specifications
532
- - **Capability Detection**: Check which features each model supports
533
- - **Cost Calculation**: Estimate costs for specific token usage
534
- - **Local Caching**: Fast model data access with automatic cache management
535
- - **Real-time Updates**: Refresh model data from OpenRouter API
536
-
537
- ## Streaming & Real-time
538
-
539
- ### Streaming Client
540
-
541
- The enhanced streaming client provides real-time response streaming with callback support and automatic response reconstruction.
542
-
543
- #### Quick Example
544
-
545
- ```ruby
546
- # Create streaming client
547
343
  streaming_client = OpenRouter::StreamingClient.new
548
344
 
549
345
  # Set up callbacks
550
346
  streaming_client
551
- .on_stream(:on_start) { |data| puts "Starting request to #{data[:model]}" }
347
+ .on_stream(:on_start) { |data| puts "Starting..." }
552
348
  .on_stream(:on_chunk) { |chunk| print chunk.content }
553
- .on_stream(:on_tool_call_chunk) { |chunk| puts "Tool call: #{chunk.name}" }
554
- .on_stream(:on_finish) { |response| puts "\nCompleted. Total tokens: #{response.total_tokens}" }
349
+ .on_stream(:on_finish) { |response| puts "\nDone! Tokens: #{response.total_tokens}" }
555
350
  .on_stream(:on_error) { |error| puts "Error: #{error.message}" }
556
351
 
557
352
  # Stream with automatic response accumulation
558
353
  response = streaming_client.stream_complete(
559
- [{ role: "user", content: "Write a short story about a robot" }],
354
+ [{ role: "user", content: "Write a story about a robot" }],
560
355
  model: "openai/gpt-4o-mini",
561
356
  accumulate_response: true
562
357
  )
563
358
 
564
- # Access complete response after streaming
565
359
  puts "Final response: #{response.content}"
566
- puts "Cost: $#{response.cost_estimate}"
567
- ```
568
-
569
- #### Streaming with Tool Calls
570
-
571
- ```ruby
572
- # Define a tool
573
- weather_tool = OpenRouter::Tool.define do
574
- name "get_weather"
575
- description "Get current weather"
576
- parameters { string :location, required: true }
577
- end
578
-
579
- # Stream with tool calling
580
- streaming_client.stream_complete(
581
- [{ role: "user", content: "What's the weather in Tokyo?" }],
582
- model: "anthropic/claude-3-5-sonnet",
583
- tools: [weather_tool]
584
- ) do |chunk|
585
- if chunk.has_tool_calls?
586
- chunk.tool_calls.each do |tool_call|
587
- puts "Calling #{tool_call.name} with #{tool_call.arguments}"
588
- end
589
- else
590
- print chunk.content
591
- end
592
- end
593
360
  ```
594
361
 
595
- ### Streaming Callbacks
596
-
597
- The streaming client supports extensive callback events for monitoring and analytics.
598
-
599
- ```ruby
600
- streaming_client = OpenRouter::StreamingClient.new
601
-
602
- # Monitor token usage in real-time
603
- streaming_client.on_stream(:on_chunk) do |chunk|
604
- if chunk.usage
605
- puts "Tokens so far: #{chunk.usage['total_tokens']}"
606
- end
607
- end
608
-
609
- # Handle errors gracefully
610
- streaming_client.on_stream(:on_error) do |error|
611
- logger.error "Streaming failed: #{error.message}"
612
- # Implement fallback logic
613
- fallback_response = client.complete(messages, model: "openai/gpt-4o-mini")
614
- end
615
-
616
- # Track performance metrics
617
- start_time = nil
618
- streaming_client
619
- .on_stream(:on_start) { |data| start_time = Time.now }
620
- .on_stream(:on_finish) do |response|
621
- duration = Time.now - start_time
622
- puts "Request completed in #{duration.round(2)}s"
623
- puts "Tokens per second: #{response.total_tokens / duration}"
624
- end
625
- ```
626
-
627
- ## Observability & Analytics
362
+ **[Complete Streaming Documentation](docs/streaming.md)**
628
363
 
629
364
  ### Usage Tracking
630
365
 
631
- Track token usage, costs, and performance metrics across all API calls.
632
-
633
- #### Quick Example
366
+ Track token usage, costs, and performance metrics.
634
367
 
635
368
  ```ruby
636
- # Create client with usage tracking enabled
369
+ # Enable tracking
637
370
  client = OpenRouter::Client.new(track_usage: true)
638
371
 
639
- # Make multiple requests
640
- 3.times do |i|
372
+ # Make requests
373
+ 3.times do
641
374
  response = client.complete(
642
- [{ role: "user", content: "Tell me a fact about space #{i + 1}" }],
375
+ [{ role: "user", content: "Tell me a fact" }],
643
376
  model: "openai/gpt-4o-mini"
644
377
  )
645
- puts "Request #{i + 1}: #{response.total_tokens} tokens, $#{response.cost_estimate}"
646
378
  end
647
379
 
648
- # View comprehensive usage statistics
380
+ # View statistics
649
381
  tracker = client.usage_tracker
650
- puts "\n=== Usage Summary ==="
651
382
  puts "Total requests: #{tracker.request_count}"
652
383
  puts "Total tokens: #{tracker.total_tokens}"
653
384
  puts "Total cost: $#{tracker.total_cost.round(4)}"
654
- puts "Average cost per request: $#{(tracker.total_cost / tracker.request_count).round(4)}"
655
385
 
656
- # View per-model breakdown
386
+ # Per-model breakdown
657
387
  tracker.model_usage.each do |model, stats|
658
- puts "\n#{model}:"
659
- puts " Requests: #{stats[:request_count]}"
660
- puts " Tokens: #{stats[:total_tokens]}"
661
- puts " Cost: $#{stats[:cost].round(4)}"
388
+ puts "#{model}: #{stats[:total_tokens]} tokens, $#{stats[:cost].round(4)}"
662
389
  end
663
390
 
664
391
  # Print detailed report
665
392
  tracker.print_summary
666
393
  ```
667
394
 
668
- #### Advanced Usage Tracking
669
-
670
- ```ruby
671
- # Track specific operations
672
- client.usage_tracker.reset! # Start fresh
673
-
674
- # Simulate different workload types
675
- client.complete(messages, model: "openai/gpt-4o") # Expensive, high-quality
676
- client.complete(messages, model: "openai/gpt-4o-mini") # Cheap, fast
677
-
678
- # Get usage metrics
679
- cache_hit_rate = client.usage_tracker.cache_hit_rate
680
- tokens_per_second = client.usage_tracker.tokens_per_second
681
-
682
- puts "Cache hit rate: #{cache_hit_rate}%"
683
- puts "Tokens per second: #{tokens_per_second}"
684
-
685
- # Export usage data as CSV for analysis
686
- csv_data = client.usage_tracker.export_csv
687
- File.write("usage_report.csv", csv_data)
688
- ```
689
-
690
- ### Response Analytics
691
-
692
- Every response includes comprehensive metadata for monitoring and optimization.
693
-
694
- ```ruby
695
- response = client.complete(messages, model: "anthropic/claude-3-5-sonnet")
696
-
697
- # Token metrics
698
- puts "Input tokens: #{response.prompt_tokens}"
699
- puts "Output tokens: #{response.completion_tokens}"
700
- puts "Cached tokens: #{response.cached_tokens}"
701
- puts "Total tokens: #{response.total_tokens}"
702
-
703
- # Cost information (requires generation stats query)
704
- puts "Total cost: $#{response.cost_estimate}"
705
-
706
- # Model information
707
- puts "Provider: #{response.provider}"
708
- puts "Model: #{response.model}"
709
- puts "System fingerprint: #{response.system_fingerprint}"
710
- puts "Finish reason: #{response.finish_reason}"
711
- ```
712
-
713
- ### Callback System
714
-
715
- The client provides an extensible callback system for monitoring requests, responses, and errors.
716
-
717
- #### Basic Callbacks
718
-
719
- ```ruby
720
- client = OpenRouter::Client.new
721
-
722
- # Monitor all requests
723
- client.on(:before_request) do |params|
724
- puts "Making request to #{params[:model]} with #{params[:messages].size} messages"
725
- end
726
-
727
- # Monitor all responses
728
- client.on(:after_response) do |response|
729
- puts "Received response: #{response.total_tokens} tokens, $#{response.cost_estimate}"
730
- end
731
-
732
- # Monitor tool calls
733
- client.on(:on_tool_call) do |tool_calls|
734
- tool_calls.each do |call|
735
- puts "Tool called: #{call.name} with args #{call.arguments}"
736
- end
737
- end
738
-
739
- # Monitor errors
740
- client.on(:on_error) do |error|
741
- logger.error "API error: #{error.message}"
742
- # Send to monitoring service
743
- ErrorReporter.notify(error)
744
- end
745
- ```
746
-
747
- #### Advanced Callback Usage
748
-
749
- ```ruby
750
- # Cost monitoring with alerts
751
- client.on(:after_response) do |response|
752
- if response.cost_estimate > 0.10
753
- AlertService.send_alert(
754
- "High cost request: $#{response.cost_estimate} for #{response.total_tokens} tokens"
755
- )
756
- end
757
- end
758
-
759
- # Performance monitoring
760
- client.on(:before_request) { |params| @start_time = Time.now }
761
- client.on(:after_response) do |response|
762
- duration = Time.now - @start_time
763
- if duration > 10.0
764
- puts "Slow request detected: #{duration.round(2)}s"
765
- end
766
- end
767
-
768
- # Usage analytics
769
- request_count = 0
770
- total_cost = 0.0
771
-
772
- client.on(:after_response) do |response|
773
- request_count += 1
774
- total_cost += response.cost_estimate || 0.0
775
-
776
- if request_count % 100 == 0
777
- puts "100 requests processed. Average cost: $#{(total_cost / request_count).round(4)}"
778
- end
779
- end
780
-
781
- # Chain callbacks for complex workflows
782
- client
783
- .on(:before_request) { |params| log_request(params) }
784
- .on(:after_response) { |response| log_response(response) }
785
- .on(:on_tool_call) { |calls| execute_tools(calls) }
786
- .on(:on_error) { |error| handle_error(error) }
787
- ```
788
-
789
- ### Cost Management
790
-
791
- Built-in cost estimation and usage tracking tools.
792
-
793
- ```ruby
794
- # Pre-flight cost estimation
795
- estimated_cost = OpenRouter::ModelRegistry.calculate_estimated_cost(
796
- "anthropic/claude-3-5-sonnet",
797
- input_tokens: 1500,
798
- output_tokens: 800
799
- )
800
-
801
- puts "Estimated cost: $#{estimated_cost}"
802
-
803
- # Use model selector to stay within budget
804
- if estimated_cost > 0.01
805
- puts "Switching to cheaper model"
806
- model = OpenRouter::ModelSelector.new
807
- .within_budget(max_cost: 0.01)
808
- .require(:chat)
809
- .choose
810
- end
811
-
812
- # Track costs in real-time
813
- client = OpenRouter::Client.new(track_usage: true)
814
-
815
- client.on(:after_response) do |response|
816
- total_spent = client.usage_tracker.total_cost
817
- puts "Total spent this session: $#{total_spent.round(4)}"
818
-
819
- if total_spent > 5.00
820
- puts "⚠️ Session cost exceeds $5.00"
821
- end
822
- end
823
- ```
824
-
825
- ## Advanced Features
826
-
827
- ### Model Fallbacks
828
-
829
- Use multiple models with automatic failover for increased reliability.
830
-
831
- ```ruby
832
- # Define fallback chain
833
- response = client.complete(
834
- messages,
835
- model: ["openai/gpt-4o", "anthropic/claude-3-5-sonnet", "anthropic/claude-3-haiku"],
836
- tools: tools
837
- )
838
-
839
- # Or use ModelSelector for intelligent fallbacks
840
- models = OpenRouter::ModelSelector.new
841
- .require(:function_calling)
842
- .choose_with_fallbacks(limit: 3)
843
-
844
- response = client.complete(messages, model: models, tools: tools)
845
- ```
846
-
847
- ### Response Healing
848
-
849
- Automatically heal malformed responses from models that don't natively support structured outputs.
850
-
851
- ```ruby
852
- # Configure global healing
853
- OpenRouter.configure do |config|
854
- config.auto_heal_responses = true
855
- config.healer_model = "openai/gpt-4o-mini"
856
- config.max_heal_attempts = 2
857
- end
858
-
859
- # The gem automatically heals malformed JSON responses
860
- response = client.complete(
861
- messages,
862
- model: "some/model-without-native-structured-outputs",
863
- response_format: schema # Will be automatically healed if malformed
864
- )
865
- ```
866
-
867
- ### Performance Optimization
395
+ **[Complete Usage Tracking Documentation](docs/usage_tracking.md)**
868
396
 
869
- Optimize performance for high-throughput applications.
397
+ ## Model Exploration
870
398
 
871
- #### Batching and Parallelization
399
+ The gem includes rake tasks for exploring available models:
872
400
 
873
- ```ruby
874
- require 'concurrent-ruby'
875
-
876
- # Process multiple requests in parallel
877
- messages_batch = [
878
- [{ role: "user", content: "Summarize this: #{text1}" }],
879
- [{ role: "user", content: "Summarize this: #{text2}" }],
880
- [{ role: "user", content: "Summarize this: #{text3}" }]
881
- ]
882
-
883
- # Create thread pool
884
- thread_pool = Concurrent::FixedThreadPool.new(5)
885
-
886
- # Process batch with shared model selection
887
- model = OpenRouter::ModelSelector.new
888
- .optimize_for(:performance)
889
- .require(:chat)
890
- .choose
891
-
892
- futures = messages_batch.map do |messages|
893
- Concurrent::Future.execute(executor: thread_pool) do
894
- client.complete(messages, model: model)
895
- end
896
- end
897
-
898
- # Collect results
899
- results = futures.map(&:value)
900
- thread_pool.shutdown
901
- ```
902
-
903
- #### Caching and Optimization
904
-
905
- ```ruby
906
- # Enable aggressive caching
907
- OpenRouter.configure do |config|
908
- config.cache_ttl = 24 * 60 * 60 # 24 hours
909
- config.auto_heal_responses = true
910
- config.strict_mode = false # Better performance
911
- end
912
-
913
- # Use cheaper models for development/testing
914
- if Rails.env.development?
915
- client = OpenRouter::Client.new(
916
- default_model: "openai/gpt-4o-mini", # Cheaper for development
917
- track_usage: true
918
- )
919
- else
920
- client = OpenRouter::Client.new(
921
- default_model: "anthropic/claude-3-5-sonnet" # Production quality
922
- )
923
- end
924
-
925
- # Pre-warm model registry cache
926
- OpenRouter::ModelRegistry.refresh_cache!
927
-
928
- # Optimize for specific workloads
929
- fast_client = OpenRouter::Client.new(
930
- request_timeout: 30, # Shorter timeout
931
- auto_heal_responses: false, # Skip healing for speed
932
- strict_mode: false # Skip capability validation
933
- )
934
- ```
935
-
936
- #### Memory Management
937
-
938
- ```ruby
939
- # Reset usage tracking periodically for long-running apps
940
- client.usage_tracker.reset! if client.usage_tracker.request_count > 1000
401
+ ```bash
402
+ # View model summary with statistics
403
+ bundle exec rake models:summary
941
404
 
942
- # Clear callback chains when not needed
943
- client.clear_callbacks(:after_response) if Rails.env.production?
405
+ # Search by provider
406
+ bundle exec rake models:search provider=anthropic
944
407
 
945
- # Use streaming for large responses to reduce memory usage
946
- streaming_client = OpenRouter::StreamingClient.new
408
+ # Search by capabilities and optimize for cost
409
+ bundle exec rake models:search capability=function_calling optimize=cost limit=10
947
410
 
948
- streaming_client.stream_complete(
949
- [{ role: "user", content: "Write a detailed report on AI trends" }],
950
- model: "anthropic/claude-3-5-sonnet",
951
- accumulate_response: false # Don't store full response
952
- ) do |chunk|
953
- # Process chunk immediately and discard
954
- process_chunk(chunk.content)
955
- end
411
+ # Advanced filtering
412
+ bundle exec rake models:search provider=anthropic capability=function_calling min_context=100000 max_cost=0.01
956
413
  ```
957
414
 
958
- ## Testing & Development
415
+ Available search parameters:
416
+ - `provider=name` - Filter by provider
417
+ - `capability=cap1,cap2` - Required capabilities
418
+ - `optimize=strategy` - Optimization (cost, performance, latest, context)
419
+ - `min_context=tokens` - Minimum context length
420
+ - `max_cost=amount` - Maximum cost per 1k tokens
421
+ - `newer_than=YYYY-MM-DD` - Filter by release date
422
+ - `limit=N` - Maximum results (default: 20)
959
423
 
960
- The gem includes comprehensive test coverage with VCR integration for real API testing.
424
+ ## Testing
961
425
 
962
426
  ### Running Tests
963
427
 
@@ -978,242 +442,78 @@ bundle exec rspec spec/vcr/ # VCR integration tests (requires API key
978
442
  The project includes VCR tests that record real API interactions:
979
443
 
980
444
  ```bash
981
- # Set API key for VCR tests
445
+ # Set API key
982
446
  export OPENROUTER_API_KEY="your_api_key"
983
447
 
984
448
  # Run VCR tests
985
449
  bundle exec rspec spec/vcr/
986
450
 
987
- # Re-record cassettes (deletes old recordings)
451
+ # Re-record cassettes
988
452
  rm -rf spec/fixtures/vcr_cassettes/
989
453
  bundle exec rspec spec/vcr/
990
454
  ```
991
455
 
992
456
  ### Examples
993
457
 
994
- The project includes comprehensive examples for all features:
458
+ Comprehensive examples for all features are available in the `examples/` directory:
995
459
 
996
460
  ```bash
997
461
  # Set your API key
998
462
  export OPENROUTER_API_KEY="your_key_here"
999
463
 
1000
- # Run individual examples
464
+ # Run examples
1001
465
  ruby -I lib examples/basic_completion.rb
1002
466
  ruby -I lib examples/tool_calling_example.rb
1003
467
  ruby -I lib examples/structured_outputs_example.rb
1004
468
  ruby -I lib examples/model_selection_example.rb
1005
- ruby -I lib examples/prompt_template_example.rb
1006
469
  ruby -I lib examples/streaming_example.rb
1007
- ruby -I lib examples/observability_example.rb
1008
- ruby -I lib examples/smart_completion_example.rb
1009
-
1010
- # Run all examples
1011
- find examples -name "*.rb" -exec ruby -I lib {} \;
1012
- ```
1013
-
1014
- ### Model Exploration Rake Tasks
1015
-
1016
- The gem includes convenient rake tasks for exploring and searching available models without writing code:
1017
-
1018
- #### Model Summary
1019
-
1020
- View an overview of all available models, including provider breakdown, capabilities, costs, and context lengths:
1021
-
1022
- ```bash
1023
- bundle exec rake models:summary
1024
- ```
1025
-
1026
- Output includes:
1027
- - Total model count and breakdown by provider
1028
- - Available capabilities across all models
1029
- - Cost analysis (min/max/median for input and output tokens)
1030
- - Context length statistics
1031
- - Performance tier distribution
1032
-
1033
- #### Model Search
1034
-
1035
- Search for models using various filters and optimization strategies:
1036
-
1037
- ```bash
1038
- # Basic search by provider
1039
- bundle exec rake models:search provider=anthropic
1040
-
1041
- # Search by capabilities
1042
- bundle exec rake models:search capability=function_calling,vision
1043
-
1044
- # Optimize for cost with capability requirements
1045
- bundle exec rake models:search capability=function_calling optimize=cost limit=10
1046
-
1047
- # Filter by context length
1048
- bundle exec rake models:search min_context=200000
1049
-
1050
- # Filter by cost
1051
- bundle exec rake models:search max_cost=0.01
1052
-
1053
- # Filter by release date
1054
- bundle exec rake models:search newer_than=2024-01-01
1055
-
1056
- # Combine multiple filters
1057
- bundle exec rake models:search provider=anthropic capability=function_calling min_context=100000 optimize=cost limit=5
1058
- ```
1059
-
1060
- Available search parameters:
1061
- - `provider=name` - Filter by provider (comma-separated for multiple)
1062
- - `capability=cap1,cap2` - Required capabilities (function_calling, vision, structured_outputs, etc.)
1063
- - `optimize=strategy` - Optimization strategy (cost, performance, latest, context)
1064
- - `min_context=tokens` - Minimum context length
1065
- - `max_cost=amount` - Maximum input cost per 1k tokens
1066
- - `max_output_cost=amount` - Maximum output cost per 1k tokens
1067
- - `newer_than=YYYY-MM-DD` - Filter models released after date
1068
- - `limit=N` - Maximum number of results to show (default: 20)
1069
- - `fallbacks=true` - Show models with fallback support
1070
-
1071
- Examples:
1072
-
1073
- ```bash
1074
- # Find cheapest models with vision support
1075
- bundle exec rake models:search capability=vision optimize=cost limit=5
1076
-
1077
- # Find latest Anthropic models with function calling
1078
- bundle exec rake models:search provider=anthropic optimize=latest capability=function_calling
1079
-
1080
- # Find high-context models for long documents
1081
- bundle exec rake models:search min_context=500000 optimize=context
1082
470
  ```
1083
471
 
1084
472
  ## Troubleshooting
1085
473
 
1086
- ### Common Issues and Solutions
1087
-
1088
- #### Authentication Errors
474
+ ### Common Issues
1089
475
 
476
+ **Authentication Errors**
1090
477
  ```ruby
1091
- # Error: "OpenRouter access token missing!"
1092
- # Solution: Set your API key
478
+ # Set your API key
1093
479
  export OPENROUTER_API_KEY="your_key_here"
1094
480
 
1095
- # Or configure in code
481
+ # Or in code
1096
482
  OpenRouter.configure do |config|
1097
483
  config.access_token = ENV["OPENROUTER_API_KEY"]
1098
484
  end
1099
-
1100
- # Error: "Invalid API key"
1101
- # Solution: Verify your key at https://openrouter.ai/keys
1102
485
  ```
1103
486
 
1104
- #### Model Selection Issues
1105
-
487
+ **Model Selection Issues**
1106
488
  ```ruby
1107
- # Error: "Model not found or access denied"
1108
- # Solution: Check model availability and your account limits
1109
- begin
1110
- client.complete(messages, model: "gpt-4")
1111
- rescue OpenRouter::ServerError => e
1112
- if e.message.include?("not found")
1113
- puts "Model not available, falling back to default"
1114
- client.complete(messages, model: "openai/gpt-4o-mini")
1115
- end
1116
- end
1117
-
1118
- # Error: "Model doesn't support feature X"
1119
- # Solution: Use ModelSelector to find compatible models
1120
- model = OpenRouter::ModelSelector.new
1121
- .require(:function_calling)
1122
- .choose
1123
- ```
1124
-
1125
- #### Rate Limiting and Costs
1126
-
1127
- ```ruby
1128
- # Error: "Rate limit exceeded"
1129
- # Solution: Implement exponential backoff
1130
- require 'retries'
1131
-
1132
- with_retries(max_tries: 3, base_sleep_seconds: 1, max_sleep_seconds: 60) do |attempt|
1133
- client.complete(messages, model: model)
1134
- end
1135
-
1136
- # Error: "Request too expensive"
1137
- # Solution: Use cheaper models or budget constraints
1138
- client = OpenRouter::Client.new
489
+ # Use ModelSelector to find compatible models
1139
490
  model = OpenRouter::ModelSelector.new
1140
- .within_budget(max_cost: 0.01)
1141
- .choose
491
+ .require(:function_calling)
492
+ .choose
1142
493
  ```
1143
494
 
1144
- #### Structured Output Issues
1145
-
495
+ **Structured Output Issues**
1146
496
  ```ruby
1147
- # Error: "Invalid JSON response"
1148
- # Solution: Enable response healing
497
+ # Enable automatic healing
1149
498
  OpenRouter.configure do |config|
1150
499
  config.auto_heal_responses = true
1151
500
  config.healer_model = "openai/gpt-4o-mini"
1152
501
  end
1153
-
1154
- # Error: "Schema validation failed"
1155
- # Solution: Check schema definitions and model capability
1156
- schema = OpenRouter::Schema.define("user") do
1157
- string :name, required: true
1158
- integer :age, minimum: 0 # Add constraints
1159
- end
1160
-
1161
- # Use models that support structured outputs natively
1162
- model = OpenRouter::ModelSelector.new
1163
- .require(:structured_outputs)
1164
- .choose
1165
502
  ```
1166
503
 
1167
- #### Performance Issues
1168
-
504
+ **Performance Issues**
1169
505
  ```ruby
1170
- # Issue: Slow responses
1171
- # Solution: Optimize client configuration
506
+ # Optimize configuration
1172
507
  client = OpenRouter::Client.new(
1173
- request_timeout: 30, # Lower timeout
1174
- strict_mode: false, # Skip capability validation
1175
- auto_heal_responses: false # Skip healing for speed
508
+ request_timeout: 30,
509
+ strict_mode: false,
510
+ auto_heal_responses: false
1176
511
  )
1177
-
1178
- # Issue: High memory usage
1179
- # Solution: Use streaming for large responses
1180
- streaming_client = OpenRouter::StreamingClient.new
1181
- streaming_client.stream_complete(messages, accumulate_response: false) do |chunk|
1182
- process_chunk_immediately(chunk)
1183
- end
1184
-
1185
- # Issue: Too many API calls
1186
- # Solution: Implement request batching
1187
- messages_batch = [...] # Multiple message sets
1188
- results = process_batch_concurrently(messages_batch, thread_pool_size: 5)
1189
- ```
1190
-
1191
- #### Tool Calling Issues
1192
-
1193
- ```ruby
1194
- # Error: "Tool not found"
1195
- # Solution: Verify tool definitions match exactly
1196
- tool = OpenRouter::Tool.define do
1197
- name "get_weather" # Must match exactly in model response
1198
- description "Get current weather for a location"
1199
- parameters do
1200
- string :location, required: true
1201
- end
1202
- end
1203
-
1204
- # Error: "Invalid tool parameters"
1205
- # Solution: Add parameter validation
1206
- def handle_weather_tool(tool_call)
1207
- location = tool_call.arguments["location"]
1208
- raise ArgumentError, "Location required" if location.nil? || location.empty?
1209
-
1210
- get_weather_data(location)
1211
- end
1212
512
  ```
1213
513
 
1214
514
  ### Debug Mode
1215
515
 
1216
- Enable detailed logging for troubleshooting:
516
+ Enable detailed logging:
1217
517
 
1218
518
  ```ruby
1219
519
  require 'logger'
@@ -1224,383 +524,29 @@ OpenRouter.configure do |config|
1224
524
  f.response :logger, Logger.new($stdout), { headers: true, bodies: true, errors: true }
1225
525
  end
1226
526
  end
1227
-
1228
- # Enable callback debugging
1229
- client = OpenRouter::Client.new
1230
- client.on(:before_request) { |params| puts "REQUEST: #{params.inspect}" }
1231
- client.on(:after_response) { |response| puts "RESPONSE: #{response.inspect}" }
1232
- client.on(:on_error) { |error| puts "ERROR: #{error.message}" }
1233
- ```
1234
-
1235
- ### Performance Monitoring
1236
-
1237
- ```ruby
1238
- # Monitor request performance
1239
- client.on(:before_request) { @start_time = Time.now }
1240
- client.on(:after_response) do |response|
1241
- duration = Time.now - @start_time
1242
- if duration > 5.0
1243
- puts "SLOW REQUEST: #{duration.round(2)}s for #{response.total_tokens} tokens"
1244
- end
1245
- end
1246
-
1247
- # Monitor costs
1248
- client.on(:after_response) do |response|
1249
- if response.cost_estimate > 0.10
1250
- puts "EXPENSIVE REQUEST: $#{response.cost_estimate}"
1251
- end
1252
- end
1253
-
1254
- # Export usage data as CSV for analysis
1255
- csv_data = client.usage_tracker.export_csv
1256
- File.write("debug_usage.csv", csv_data)
1257
527
  ```
1258
528
 
1259
529
  ### Getting Help
1260
530
 
1261
- 1. **Check the documentation**: Each feature has detailed documentation in the `docs/` directory
1262
- 2. **Review examples**: Look at working examples in the `examples/` directory
1263
- 3. **Enable debug mode**: Turn on logging to see request/response details
1264
- 4. **Check OpenRouter status**: Visit [OpenRouter Status](https://status.openrouter.ai)
1265
- 5. **Open an issue**: Report bugs at [GitHub Issues](https://github.com/estiens/open_router_enhanced/issues)
1266
-
1267
- ## API Reference
1268
-
1269
- ### Client Classes
1270
-
1271
- #### OpenRouter::Client
1272
-
1273
- Main client for OpenRouter API interactions.
1274
-
1275
- ```ruby
1276
- client = OpenRouter::Client.new(
1277
- access_token: "...",
1278
- track_usage: false,
1279
- request_timeout: 120
1280
- )
1281
-
1282
- # Core methods
1283
- client.complete(messages, **options) # Chat completions with full feature support
1284
- client.models # List available models
1285
- client.query_generation_stats(id) # Query generation statistics
1286
-
1287
- # Callback methods
1288
- client.on(event, &block) # Register event callback
1289
- client.clear_callbacks(event) # Clear callbacks for event
1290
- client.trigger_callbacks(event, data) # Manually trigger callbacks
1291
-
1292
- # Usage tracking
1293
- client.usage_tracker # Access usage tracker instance
1294
- ```
1295
-
1296
- #### OpenRouter::StreamingClient
1297
-
1298
- Enhanced streaming client with callback support.
1299
-
1300
- ```ruby
1301
- streaming_client = OpenRouter::StreamingClient.new
531
+ 1. **Check the documentation**: Detailed docs in the `docs/` directory
532
+ 2. **Review examples**: Working examples in `examples/`
533
+ 3. **Enable debug mode**: Turn on logging to see details
534
+ 4. **Check OpenRouter status**: [OpenRouter Status](https://status.openrouter.ai)
535
+ 5. **Open an issue**: [GitHub Issues](https://github.com/estiens/open_router_enhanced/issues)
1302
536
 
1303
- # Streaming methods
1304
- streaming_client.stream_complete(messages, **options) # Stream with callbacks
1305
- streaming_client.on_stream(event, &block) # Register streaming callbacks
1306
-
1307
- # Available streaming events: :on_start, :on_chunk, :on_tool_call_chunk, :on_finish, :on_error
1308
- ```
1309
-
1310
- ### Enhanced Classes
1311
-
1312
- #### OpenRouter::Tool
1313
-
1314
- Define and manage function calling tools.
1315
-
1316
- ```ruby
1317
- # DSL definition
1318
- tool = OpenRouter::Tool.define do
1319
- name "function_name"
1320
- description "Function description"
1321
- parameters do
1322
- string :param1, required: true, description: "Parameter description"
1323
- integer :param2, minimum: 0, maximum: 100
1324
- boolean :param3, default: false
1325
- end
1326
- end
1327
-
1328
- # Hash definition
1329
- tool = OpenRouter::Tool.from_hash({
1330
- name: "function_name",
1331
- description: "Function description",
1332
- parameters: {
1333
- type: "object",
1334
- properties: { ... }
1335
- }
1336
- })
1337
-
1338
- # Methods
1339
- tool.name # Get tool name
1340
- tool.description # Get tool description
1341
- tool.parameters # Get parameters schema
1342
- tool.to_h # Convert to hash format
1343
- tool.validate_arguments(args) # Validate arguments against schema
1344
- ```
1345
-
1346
- #### OpenRouter::Schema
1347
-
1348
- Define JSON schemas for structured outputs.
1349
-
1350
- ```ruby
1351
- # DSL definition
1352
- schema = OpenRouter::Schema.define("schema_name") do
1353
- string :name, required: true, description: "User's name"
1354
- integer :age, minimum: 0, maximum: 150
1355
- boolean :active, default: true
1356
- array :tags, items: { type: "string" }
1357
- object :address do
1358
- string :street, required: true
1359
- string :city, required: true
1360
- string :country, default: "US"
1361
- end
1362
- end
1363
-
1364
- # Hash definition
1365
- schema = OpenRouter::Schema.from_hash("schema_name", {
1366
- type: "object",
1367
- properties: { ... },
1368
- required: [...]
1369
- })
1370
-
1371
- # Methods
1372
- schema.name # Get schema name
1373
- schema.schema # Get JSON schema hash
1374
- schema.validate(data) # Validate data against schema
1375
- schema.to_h # Convert to hash format
1376
- ```
1377
-
1378
- #### OpenRouter::PromptTemplate
1379
-
1380
- Create reusable prompt templates with variable interpolation.
1381
-
1382
- ```ruby
1383
- # Basic template
1384
- template = OpenRouter::PromptTemplate.new(
1385
- template: "Translate '{text}' from {source} to {target}",
1386
- input_variables: [:text, :source, :target]
1387
- )
1388
-
1389
- # Few-shot template
1390
- template = OpenRouter::PromptTemplate.new(
1391
- prefix: "Classification examples:",
1392
- suffix: "Classify: {input}",
1393
- examples: [{ input: "...", output: "..." }],
1394
- example_template: "Input: {input}\nOutput: {output}",
1395
- input_variables: [:input]
1396
- )
1397
-
1398
- # Methods
1399
- template.format(**variables) # Format template with variables
1400
- template.to_messages(**variables) # Convert to OpenRouter message format
1401
- template.input_variables # Get required input variables
1402
- template.partial(**variables) # Create partial template with some variables filled
1403
- ```
1404
-
1405
- #### OpenRouter::ModelSelector
1406
-
1407
- Intelligent model selection with fluent DSL.
1408
-
1409
- ```ruby
1410
- selector = OpenRouter::ModelSelector.new
1411
-
1412
- # Requirement methods
1413
- selector.require(*capabilities) # Require specific capabilities
1414
- selector.within_budget(max_cost: 0.01) # Set maximum cost constraint
1415
- selector.min_context(tokens) # Minimum context length
1416
- selector.prefer_providers(*providers) # Prefer specific providers
1417
- selector.avoid_providers(*providers) # Avoid specific providers
1418
- selector.optimize_for(strategy) # Optimization strategy (:cost, :performance, :balanced)
1419
-
1420
- # Selection methods
1421
- selector.choose # Choose best single model
1422
- selector.choose_with_fallbacks(limit: 3) # Choose multiple models for fallback
1423
- selector.candidates # Get all matching models
1424
- selector.explain_choice # Get explanation of selection
1425
-
1426
- # Available capabilities: :chat, :function_calling, :structured_outputs, :vision, :code_generation
1427
- # Available strategies: :cost, :performance, :balanced
1428
- ```
1429
-
1430
- #### OpenRouter::ModelRegistry
1431
-
1432
- Model information and capability detection.
1433
-
1434
- ```ruby
1435
- # Class methods
1436
- OpenRouter::ModelRegistry.all_models # Get all cached models
1437
- OpenRouter::ModelRegistry.get_model_info(model) # Get specific model info
1438
- OpenRouter::ModelRegistry.models_meeting_requirements(...) # Find models matching criteria
1439
- OpenRouter::ModelRegistry.calculate_estimated_cost(model, tokens) # Estimate cost
1440
- OpenRouter::ModelRegistry.refresh_cache! # Refresh model cache
1441
- OpenRouter::ModelRegistry.cache_status # Get cache status
1442
- ```
1443
-
1444
- #### OpenRouter::UsageTracker
1445
-
1446
- Track token usage, costs, and performance metrics.
1447
-
1448
- ```ruby
1449
- tracker = client.usage_tracker
1450
-
1451
- # Metrics
1452
- tracker.total_tokens # Total tokens used
1453
- tracker.total_cost # Total estimated cost
1454
- tracker.request_count # Number of requests made
1455
- tracker.model_usage # Per-model usage breakdown
1456
- tracker.session_duration # Time since tracking started
1457
-
1458
- # Analysis methods
1459
- tracker.cache_hit_rate # Cache hit rate percentage
1460
- tracker.tokens_per_second # Tokens processed per second
1461
- tracker.print_summary # Print detailed usage report
1462
- tracker.export_csv # Export usage data as CSV
1463
- tracker.summary # Get usage summary hash
1464
- tracker.reset! # Reset all counters
1465
- ```
1466
-
1467
- ### Response Objects
1468
-
1469
- #### OpenRouter::Response
1470
-
1471
- Enhanced response wrapper with metadata and feature support.
1472
-
1473
- ```ruby
1474
- response = client.complete(messages)
1475
-
1476
- # Content access
1477
- response.content # Response content
1478
- response.structured_output # Parsed JSON for structured outputs
1479
-
1480
- # Tool calling
1481
- response.has_tool_calls? # Check if response has tool calls
1482
- response.tool_calls # Array of ToolCall objects
1483
-
1484
- # Token metrics
1485
- response.prompt_tokens # Input tokens
1486
- response.completion_tokens # Output tokens
1487
- response.cached_tokens # Cached tokens
1488
- response.total_tokens # Total tokens
1489
-
1490
- # Cost information
1491
- response.input_cost # Input cost
1492
- response.output_cost # Output cost
1493
- response.cost_estimate # Total estimated cost
1494
-
1495
- # Performance metrics
1496
- response.response_time # Response time in milliseconds
1497
- response.tokens_per_second # Processing speed
1498
-
1499
- # Model information
1500
- response.model # Model used
1501
- response.provider # Provider name
1502
- response.system_fingerprint # System fingerprint
1503
- response.finish_reason # Why generation stopped
1504
-
1505
- # Cache information
1506
- response.cache_hit? # Whether response used cache
1507
- response.cache_efficiency # Cache efficiency percentage
1508
-
1509
- # Backward compatibility - delegates hash methods to raw response
1510
- response["key"] # Hash-style access
1511
- response.dig("path", "to", "value") # Deep hash access
1512
- ```
1513
-
1514
- #### OpenRouter::ToolCall
1515
-
1516
- Individual tool call handling and execution.
1517
-
1518
- ```ruby
1519
- tool_call = response.tool_calls.first
1520
-
1521
- # Properties
1522
- tool_call.id # Tool call ID
1523
- tool_call.name # Tool name
1524
- tool_call.arguments # Tool arguments (Hash)
1525
-
1526
- # Methods
1527
- tool_call.validate_arguments! # Validate arguments against tool schema
1528
- tool_call.to_message # Convert to continuation message format
1529
- tool_call.execute(&block) # Execute tool with block
1530
- ```
1531
-
1532
- ### Error Classes
1533
-
1534
- ```ruby
1535
- OpenRouter::Error # Base error class
1536
- OpenRouter::ConfigurationError # Configuration issues
1537
- OpenRouter::CapabilityError # Capability validation errors
1538
- OpenRouter::ServerError # API server errors
1539
- OpenRouter::ToolCallError # Tool execution errors
1540
- OpenRouter::SchemaValidationError # Schema validation errors
1541
- OpenRouter::StructuredOutputError # JSON parsing/healing errors
1542
- OpenRouter::ModelRegistryError # Model registry errors
1543
- OpenRouter::ModelSelectionError # Model selection errors
1544
- ```
1545
-
1546
- ### Configuration Options
1547
-
1548
- ```ruby
1549
- OpenRouter.configure do |config|
1550
- # Authentication
1551
- config.access_token = "sk-..."
1552
- config.site_name = "Your App Name"
1553
- config.site_url = "https://yourapp.com"
1554
-
1555
- # Request settings
1556
- config.request_timeout = 120
1557
- config.api_version = "v1"
1558
- config.uri_base = "https://openrouter.ai/api"
1559
- config.extra_headers = {}
1560
-
1561
- # Response healing
1562
- config.auto_heal_responses = true
1563
- config.healer_model = "openai/gpt-4o-mini"
1564
- config.max_heal_attempts = 2
1565
-
1566
- # Capability validation
1567
- config.strict_mode = true
1568
- config.auto_force_on_unsupported_models = true
1569
-
1570
- # Structured outputs
1571
- config.default_structured_output_mode = :strict
1572
-
1573
- # Caching
1574
- config.cache_ttl = 7 * 24 * 60 * 60 # 7 days
1575
-
1576
- # Model registry
1577
- config.model_registry_timeout = 30
1578
- config.model_registry_retries = 3
1579
-
1580
- # Logging
1581
- config.log_errors = false
1582
- config.faraday do |f|
1583
- f.response :logger
1584
- end
1585
- end
1586
- ```
537
+ **[Complete Troubleshooting Guide](docs/troubleshooting.md)**
1587
538
 
1588
539
  ## Contributing
1589
540
 
1590
541
  Bug reports and pull requests are welcome on GitHub at <https://github.com/estiens/open_router_enhanced>.
1591
542
 
1592
- This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](https://www.contributor-covenant.org/) code of conduct.
1593
-
1594
- For detailed contribution guidelines, see [CONTRIBUTING.md](.github/CONTRIBUTING.md).
1595
-
1596
543
  ### Branch Strategy
1597
544
 
1598
545
  We use a two-branch workflow:
1599
-
1600
546
  - **`main`** - Stable releases only. Protected branch.
1601
- - **`dev`** - Active development. All PRs should target this branch.
547
+ - **`dev`** - Active development. **All PRs should target this branch.**
1602
548
 
1603
- **⚠️ Important:** Always target your PRs to the `dev` branch, not `main`. The `main` branch is reserved for stable releases.
549
+ **Important:** Always target your PRs to the `dev` branch, not `main`.
1604
550
 
1605
551
  ### Development Setup
1606
552
 
@@ -1611,42 +557,21 @@ bundle install
1611
557
  bundle exec rspec
1612
558
  ```
1613
559
 
1614
- ### Running Examples
1615
-
1616
- ```bash
1617
- # Set your API key
1618
- export OPENROUTER_API_KEY="your_key_here"
1619
-
1620
- # Run examples
1621
- ruby -I lib examples/tool_calling_example.rb
1622
- ruby -I lib examples/structured_outputs_example.rb
1623
- ruby -I lib examples/model_selection_example.rb
1624
- ```
560
+ For detailed contribution guidelines, see [CONTRIBUTING.md](.github/CONTRIBUTING.md).
1625
561
 
1626
562
  ## Acknowledgments
1627
563
 
1628
- This enhanced fork builds upon the excellent foundation laid by [Obie Fernandez](https://github.com/obie) and the original OpenRouter Ruby gem. The original library was bootstrapped from the [Anthropic gem](https://github.com/alexrudall/anthropic) by [Alex Rudall](https://github.com/alexrudall) and extracted from the codebase of [Olympia](https://olympia.chat), Obie's AI startup.
564
+ This enhanced fork builds upon the excellent foundation laid by [Obie Fernandez](https://github.com/obie) and the original OpenRouter Ruby gem. The original library was bootstrapped from the [Anthropic gem](https://github.com/alexrudall/anthropic) by [Alex Rudall](https://github.com/alexrudall).
1629
565
 
1630
566
  We extend our heartfelt gratitude to:
1631
-
1632
- - **Obie Fernandez** - Original OpenRouter gem author and visionary
1633
- - **Alex Rudall** - Creator of the Anthropic gem that served as the foundation
567
+ - **Obie Fernandez** - Original OpenRouter gem author
568
+ - **Alex Rudall** - Creator of the Anthropic gem foundation
1634
569
  - **The OpenRouter Team** - For creating an amazing unified AI API
1635
570
  - **The Ruby Community** - For continuous support and contributions
1636
571
 
1637
- ## Maintainer & Consulting
1638
-
1639
- This enhanced fork is maintained by:
572
+ ## Consulting
1640
573
 
1641
- **Eric Stiens**
1642
- - Email: hello@ericstiens.dev
1643
- - Website: [ericstiens.dev](http://ericstiens.dev)
1644
- - GitHub: [@estiens](https://github.com/estiens)
1645
- - Blog: [Low Level Magic](https://lowlevelmagic.io)
1646
-
1647
- ### Need Help with AI Integration?
1648
-
1649
- I'm available for consulting on Ruby AI applications, LLM integration, and building production-ready AI systems. My work extends beyond Ruby to include real-time AI orchestration, character-based AI systems, multi-agent architectures, and low-latency voice/streaming applications. Whether you need help with tool calling workflows, cost optimization, building AI characters with persistent memory, or orchestrating complex multi-model systems, I'd be happy to help.
574
+ I'm available for consulting on Ruby AI applications, LLM integration, and building production-ready AI systems. My work extends beyond Ruby to include real-time AI orchestration, character-based AI systems, multi-agent architectures, and low-latency voice/streaming applications.
1650
575
 
1651
576
  **Get in touch:**
1652
577
  - Email: hello@lowlevelmagic.io
@@ -1656,5 +581,3 @@ I'm available for consulting on Ruby AI applications, LLM integration, and build
1656
581
  ## License
1657
582
 
1658
583
  The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
1659
-
1660
- MIT License is chosen for maximum permissiveness and compatibility, allowing unrestricted use, modification, and distribution while maintaining attribution requirements.