open_router_enhanced 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/.env.example +1 -0
- data/.rspec +3 -0
- data/.rubocop.yml +13 -0
- data/.rubocop_todo.yml +130 -0
- data/.ruby-version +1 -0
- data/CHANGELOG.md +41 -0
- data/CODE_OF_CONDUCT.md +84 -0
- data/CONTRIBUTING.md +384 -0
- data/Gemfile +22 -0
- data/Gemfile.lock +138 -0
- data/LICENSE.txt +21 -0
- data/MIGRATION.md +556 -0
- data/README.md +1660 -0
- data/Rakefile +334 -0
- data/SECURITY.md +150 -0
- data/VCR_CONFIGURATION.md +80 -0
- data/docs/model_selection.md +637 -0
- data/docs/observability.md +430 -0
- data/docs/prompt_templates.md +422 -0
- data/docs/streaming.md +467 -0
- data/docs/structured_outputs.md +466 -0
- data/docs/tools.md +1016 -0
- data/examples/basic_completion.rb +122 -0
- data/examples/model_selection_example.rb +141 -0
- data/examples/observability_example.rb +199 -0
- data/examples/prompt_template_example.rb +184 -0
- data/examples/smart_completion_example.rb +89 -0
- data/examples/streaming_example.rb +176 -0
- data/examples/structured_outputs_example.rb +191 -0
- data/examples/tool_calling_example.rb +149 -0
- data/lib/open_router/client.rb +552 -0
- data/lib/open_router/http.rb +118 -0
- data/lib/open_router/json_healer.rb +263 -0
- data/lib/open_router/model_registry.rb +378 -0
- data/lib/open_router/model_selector.rb +462 -0
- data/lib/open_router/prompt_template.rb +290 -0
- data/lib/open_router/response.rb +371 -0
- data/lib/open_router/schema.rb +288 -0
- data/lib/open_router/streaming_client.rb +210 -0
- data/lib/open_router/tool.rb +221 -0
- data/lib/open_router/tool_call.rb +180 -0
- data/lib/open_router/usage_tracker.rb +277 -0
- data/lib/open_router/version.rb +5 -0
- data/lib/open_router.rb +123 -0
- data/sig/open_router.rbs +20 -0
- metadata +186 -0
data/README.md
ADDED
|
@@ -0,0 +1,1660 @@
|
|
|
1
|
+
# OpenRouter Enhanced - Ruby Gem
|
|
2
|
+
|
|
3
|
+
The future will bring us hundreds of language models and dozens of providers for each. How will you choose the best?
|
|
4
|
+
|
|
5
|
+
The [OpenRouter API](https://openrouter.ai/docs) is a single unified interface for all LLMs! And now you can easily use it with Ruby! 🤖🌌
|
|
6
|
+
|
|
7
|
+
**OpenRouter Enhanced** is an advanced fork of the [original OpenRouter Ruby gem](https://github.com/OlympiaAI/open_router) by [Obie Fernandez](https://github.com/obie) that adds comprehensive AI application development features including tool calling, structured outputs, intelligent model selection, prompt templates, observability, and automatic response healing—all while maintaining full backward compatibility.
|
|
8
|
+
|
|
9
|
+
đź“– **[Read the story behind OpenRouter Enhanced](https://lowlevelmagic.io/writings/why-i-built-open-router-enhanced/)** - Learn why this gem was built and the philosophy behind its design.
|
|
10
|
+
|
|
11
|
+
## Enhanced Features
|
|
12
|
+
|
|
13
|
+
This fork extends the original OpenRouter gem with enterprise-grade AI development capabilities:
|
|
14
|
+
|
|
15
|
+
### Core AI Features
|
|
16
|
+
- **Tool Calling**: Full support for OpenRouter's function calling API with Ruby-idiomatic DSL for tool definitions
|
|
17
|
+
- **Structured Outputs**: JSON Schema validation with automatic healing for non-native models and Ruby DSL for schema definitions
|
|
18
|
+
- **Smart Model Selection**: Intelligent model selection with fluent DSL for cost optimization, capability requirements, and provider preferences
|
|
19
|
+
- **Prompt Templates**: Reusable prompt templates with variable interpolation and few-shot learning support
|
|
20
|
+
|
|
21
|
+
### Performance & Reliability
|
|
22
|
+
- **Model Registry**: Local caching and querying of OpenRouter model data with capability detection
|
|
23
|
+
- **Enhanced Response Handling**: Rich Response objects with automatic parsing for tool calls and structured outputs
|
|
24
|
+
- **Automatic Healing**: Self-healing responses for malformed JSON from models that don't natively support structured outputs
|
|
25
|
+
- **Model Fallbacks**: Automatic failover between models with graceful degradation
|
|
26
|
+
- **Streaming Support**: Enhanced streaming client with callback system and response reconstruction
|
|
27
|
+
|
|
28
|
+
### Observability & Analytics
|
|
29
|
+
- **Usage Tracking**: Comprehensive token usage and cost tracking across all API calls
|
|
30
|
+
- **Response Analytics**: Detailed metadata including tokens, costs, cache hits, and performance metrics
|
|
31
|
+
- **Callback System**: Extensible event system for monitoring requests, responses, and errors
|
|
32
|
+
- **Cost Management**: Built-in cost estimation and budget constraints
|
|
33
|
+
|
|
34
|
+
### Development & Testing
|
|
35
|
+
- **Comprehensive Testing**: VCR-based integration tests with real API interactions
|
|
36
|
+
- **Debug Support**: Detailed error reporting and validation feedback
|
|
37
|
+
- **Configuration Options**: Extensive configuration for healing, validation, and performance tuning
|
|
38
|
+
- **Backward Compatible**: All existing code continues to work unchanged
|
|
39
|
+
|
|
40
|
+
### Core OpenRouter Benefits
|
|
41
|
+
|
|
42
|
+
- **Prioritize price or performance**: OpenRouter scouts for the lowest prices and best latencies/throughputs across dozens of providers, and lets you choose how to prioritize them.
|
|
43
|
+
- **Standardized API**: No need to change your code when switching between models or providers. You can even let users choose and pay for their own.
|
|
44
|
+
- **Easy integration**: This Ruby gem provides a simple and intuitive interface to interact with the OpenRouter API, making it effortless to integrate AI capabilities into your Ruby applications.
|
|
45
|
+
|
|
46
|
+
## Table of Contents
|
|
47
|
+
|
|
48
|
+
- [Installation](#installation)
|
|
49
|
+
- [Quick Start](#quick-start)
|
|
50
|
+
- [Configuration](#configuration)
|
|
51
|
+
- [Core Features](#core-features)
|
|
52
|
+
- [Basic Completions](#basic-completions)
|
|
53
|
+
- [Model Selection](#model-selection)
|
|
54
|
+
- [Enhanced AI Features](#enhanced-ai-features)
|
|
55
|
+
- [Tool Calling](#tool-calling)
|
|
56
|
+
- [Structured Outputs](#structured-outputs)
|
|
57
|
+
- [Smart Model Selection](#smart-model-selection)
|
|
58
|
+
- [Prompt Templates](#prompt-templates)
|
|
59
|
+
- [Streaming & Real-time](#streaming--real-time)
|
|
60
|
+
- [Streaming Client](#streaming-client)
|
|
61
|
+
- [Streaming Callbacks](#streaming-callbacks)
|
|
62
|
+
- [Observability & Analytics](#observability--analytics)
|
|
63
|
+
- [Usage Tracking](#usage-tracking)
|
|
64
|
+
- [Response Analytics](#response-analytics)
|
|
65
|
+
- [Callback System](#callback-system)
|
|
66
|
+
- [Cost Management](#cost-management)
|
|
67
|
+
- [Advanced Features](#advanced-features)
|
|
68
|
+
- [Model Registry](#model-registry)
|
|
69
|
+
- [Model Fallbacks](#model-fallbacks)
|
|
70
|
+
- [Response Healing](#response-healing)
|
|
71
|
+
- [Performance Optimization](#performance-optimization)
|
|
72
|
+
- [Testing & Development](#testing--development)
|
|
73
|
+
- [Running Tests](#running-tests)
|
|
74
|
+
- [VCR Testing](#vcr-testing)
|
|
75
|
+
- [Examples](#examples)
|
|
76
|
+
- [Troubleshooting](#troubleshooting)
|
|
77
|
+
- [API Reference](#api-reference)
|
|
78
|
+
- [Contributing](#contributing)
|
|
79
|
+
- [License](#license)
|
|
80
|
+
|
|
81
|
+
## Installation
|
|
82
|
+
|
|
83
|
+
### Bundler
|
|
84
|
+
|
|
85
|
+
Add this line to your application's Gemfile:
|
|
86
|
+
|
|
87
|
+
```ruby
|
|
88
|
+
gem "open_router_enhanced"
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
And then execute:
|
|
92
|
+
|
|
93
|
+
```bash
|
|
94
|
+
bundle install
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
### Gem install
|
|
98
|
+
|
|
99
|
+
Or install it directly:
|
|
100
|
+
|
|
101
|
+
```bash
|
|
102
|
+
gem install open_router_enhanced
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
And require it in your code:
|
|
106
|
+
|
|
107
|
+
```ruby
|
|
108
|
+
require "open_router"
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
## Quick Start
|
|
112
|
+
|
|
113
|
+
### 1. Get Your API Key
|
|
114
|
+
- Sign up at [OpenRouter](https://openrouter.ai)
|
|
115
|
+
- Get your API key from [https://openrouter.ai/keys](https://openrouter.ai/keys)
|
|
116
|
+
|
|
117
|
+
### 2. Basic Setup and Usage
|
|
118
|
+
|
|
119
|
+
```ruby
|
|
120
|
+
require "open_router"
|
|
121
|
+
|
|
122
|
+
# Configure the gem
|
|
123
|
+
OpenRouter.configure do |config|
|
|
124
|
+
config.access_token = ENV["OPENROUTER_API_KEY"]
|
|
125
|
+
config.site_name = "Your App Name"
|
|
126
|
+
config.site_url = "https://yourapp.com"
|
|
127
|
+
end
|
|
128
|
+
|
|
129
|
+
# Create a client
|
|
130
|
+
client = OpenRouter::Client.new
|
|
131
|
+
|
|
132
|
+
# Basic completion
|
|
133
|
+
response = client.complete([
|
|
134
|
+
{ role: "user", content: "What is the capital of France?" }
|
|
135
|
+
])
|
|
136
|
+
|
|
137
|
+
puts response.content
|
|
138
|
+
# => "The capital of France is Paris."
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
### 3. Enhanced Features Quick Example
|
|
142
|
+
|
|
143
|
+
```ruby
|
|
144
|
+
# Smart model selection
|
|
145
|
+
model = OpenRouter::ModelSelector.new
|
|
146
|
+
.require(:function_calling)
|
|
147
|
+
.optimize_for(:cost)
|
|
148
|
+
.choose
|
|
149
|
+
|
|
150
|
+
# Tool calling with structured output
|
|
151
|
+
weather_tool = OpenRouter::Tool.define do
|
|
152
|
+
name "get_weather"
|
|
153
|
+
description "Get current weather"
|
|
154
|
+
parameters do
|
|
155
|
+
string :location, required: true
|
|
156
|
+
end
|
|
157
|
+
end
|
|
158
|
+
|
|
159
|
+
weather_schema = OpenRouter::Schema.define("weather") do
|
|
160
|
+
string :location, required: true
|
|
161
|
+
number :temperature, required: true
|
|
162
|
+
string :conditions, required: true
|
|
163
|
+
end
|
|
164
|
+
|
|
165
|
+
response = client.complete(
|
|
166
|
+
[{ role: "user", content: "What's the weather in Tokyo?" }],
|
|
167
|
+
model: model,
|
|
168
|
+
tools: [weather_tool],
|
|
169
|
+
response_format: weather_schema
|
|
170
|
+
)
|
|
171
|
+
|
|
172
|
+
# Process results
|
|
173
|
+
if response.has_tool_calls?
|
|
174
|
+
weather_data = response.structured_output
|
|
175
|
+
puts "Temperature in #{weather_data['location']}: #{weather_data['temperature']}°"
|
|
176
|
+
end
|
|
177
|
+
```
|
|
178
|
+
|
|
179
|
+
## Configuration
|
|
180
|
+
|
|
181
|
+
### Global Configuration
|
|
182
|
+
|
|
183
|
+
Configure the gem globally, for example in an `open_router.rb` initializer file. Never hardcode secrets into your codebase - instead use `Rails.application.credentials` or something like [dotenv](https://github.com/motdotla/dotenv) to pass the keys safely into your environments.
|
|
184
|
+
|
|
185
|
+
```ruby
|
|
186
|
+
OpenRouter.configure do |config|
|
|
187
|
+
config.access_token = ENV["OPENROUTER_API_KEY"]
|
|
188
|
+
config.site_name = "Your App Name"
|
|
189
|
+
config.site_url = "https://yourapp.com"
|
|
190
|
+
|
|
191
|
+
# Optional: Configure response healing for non-native structured output models
|
|
192
|
+
config.auto_heal_responses = true
|
|
193
|
+
config.healer_model = "openai/gpt-4o-mini"
|
|
194
|
+
config.max_heal_attempts = 2
|
|
195
|
+
|
|
196
|
+
# Optional: Configure strict mode for capability validation
|
|
197
|
+
config.strict_mode = true
|
|
198
|
+
|
|
199
|
+
# Optional: Configure automatic forcing for unsupported models
|
|
200
|
+
config.auto_force_on_unsupported_models = true
|
|
201
|
+
end
|
|
202
|
+
```
|
|
203
|
+
|
|
204
|
+
### Per-Client Configuration
|
|
205
|
+
|
|
206
|
+
You can also configure clients individually:
|
|
207
|
+
|
|
208
|
+
```ruby
|
|
209
|
+
client = OpenRouter::Client.new(
|
|
210
|
+
access_token: ENV["OPENROUTER_API_KEY"],
|
|
211
|
+
request_timeout: 120
|
|
212
|
+
)
|
|
213
|
+
```
|
|
214
|
+
|
|
215
|
+
### Faraday Configuration
|
|
216
|
+
|
|
217
|
+
The configuration object exposes a [`faraday`](https://github.com/lostisland/faraday-retry) method that you can pass a block to configure Faraday settings and middleware.
|
|
218
|
+
|
|
219
|
+
This example adds `faraday-retry` and a logger that redacts the api key so it doesn't get leaked to logs.
|
|
220
|
+
|
|
221
|
+
```ruby
|
|
222
|
+
require 'faraday/retry'
|
|
223
|
+
|
|
224
|
+
retry_options = {
|
|
225
|
+
max: 2,
|
|
226
|
+
interval: 0.05,
|
|
227
|
+
interval_randomness: 0.5,
|
|
228
|
+
backoff_factor: 2
|
|
229
|
+
}
|
|
230
|
+
|
|
231
|
+
OpenRouter::Client.new(access_token: ENV["ACCESS_TOKEN"]) do |config|
|
|
232
|
+
config.faraday do |f|
|
|
233
|
+
f.request :retry, retry_options
|
|
234
|
+
f.response :logger, ::Logger.new($stdout), { headers: true, bodies: true, errors: true } do |logger|
|
|
235
|
+
logger.filter(/(Bearer) (\S+)/, '\1[REDACTED]')
|
|
236
|
+
end
|
|
237
|
+
end
|
|
238
|
+
end
|
|
239
|
+
```
|
|
240
|
+
|
|
241
|
+
#### Change version or timeout
|
|
242
|
+
|
|
243
|
+
The default timeout for any request using this library is 120 seconds. You can change that by passing a number of seconds to the `request_timeout` when initializing the client.
|
|
244
|
+
|
|
245
|
+
```ruby
|
|
246
|
+
client = OpenRouter::Client.new(
|
|
247
|
+
access_token: "access_token_goes_here",
|
|
248
|
+
request_timeout: 240 # Optional
|
|
249
|
+
)
|
|
250
|
+
```
|
|
251
|
+
|
|
252
|
+
## Core Features
|
|
253
|
+
|
|
254
|
+
### Basic Completions
|
|
255
|
+
|
|
256
|
+
Hit the OpenRouter API for a completion:
|
|
257
|
+
|
|
258
|
+
```ruby
|
|
259
|
+
messages = [
|
|
260
|
+
{ role: "system", content: "You are a helpful assistant." },
|
|
261
|
+
{ role: "user", content: "What is the color of the sky?" }
|
|
262
|
+
]
|
|
263
|
+
|
|
264
|
+
response = client.complete(messages)
|
|
265
|
+
puts response.content
|
|
266
|
+
# => "The sky is typically blue during the day due to a phenomenon called Rayleigh scattering. Sunlight..."
|
|
267
|
+
```
|
|
268
|
+
|
|
269
|
+
### Model Selection
|
|
270
|
+
|
|
271
|
+
Pass an array to the `model` parameter to enable [explicit model routing](https://openrouter.ai/docs#model-routing).
|
|
272
|
+
|
|
273
|
+
```ruby
|
|
274
|
+
OpenRouter::Client.new.complete(
|
|
275
|
+
[
|
|
276
|
+
{ role: "system", content: SYSTEM_PROMPT },
|
|
277
|
+
{ role: "user", content: "Provide analysis of the data formatted as JSON:" }
|
|
278
|
+
],
|
|
279
|
+
model: [
|
|
280
|
+
"mistralai/mixtral-8x7b-instruct:nitro",
|
|
281
|
+
"mistralai/mixtral-8x7b-instruct"
|
|
282
|
+
],
|
|
283
|
+
extras: {
|
|
284
|
+
response_format: {
|
|
285
|
+
type: "json_object"
|
|
286
|
+
}
|
|
287
|
+
}
|
|
288
|
+
)
|
|
289
|
+
```
|
|
290
|
+
|
|
291
|
+
[Browse full list of models available](https://openrouter.ai/models) or fetch from the OpenRouter API:
|
|
292
|
+
|
|
293
|
+
```ruby
|
|
294
|
+
models = client.models
|
|
295
|
+
puts models
|
|
296
|
+
# => [{"id"=>"openrouter/auto", "object"=>"model", "created"=>1684195200, "owned_by"=>"openrouter", "permission"=>[], "root"=>"openrouter", "parent"=>nil}, ...]
|
|
297
|
+
```
|
|
298
|
+
|
|
299
|
+
### Generation Stats
|
|
300
|
+
|
|
301
|
+
Query the generation stats for a given generation ID:
|
|
302
|
+
|
|
303
|
+
```ruby
|
|
304
|
+
generation_id = "generation-abcdefg"
|
|
305
|
+
stats = client.query_generation_stats(generation_id)
|
|
306
|
+
puts stats
|
|
307
|
+
# => {"id"=>"generation-abcdefg", "object"=>"generation", "created"=>1684195200, "model"=>"openrouter/auto", "usage"=>{"prompt_tokens"=>10, "completion_tokens"=>50, "total_tokens"=>60}, "cost"=>0.0006}
|
|
308
|
+
```
|
|
309
|
+
|
|
310
|
+
## Enhanced AI Features
|
|
311
|
+
|
|
312
|
+
### Tool Calling
|
|
313
|
+
|
|
314
|
+
Enable AI models to call functions and interact with external APIs using OpenRouter's function calling with an intuitive Ruby DSL.
|
|
315
|
+
|
|
316
|
+
#### Quick Example
|
|
317
|
+
|
|
318
|
+
```ruby
|
|
319
|
+
# Define a tool using the DSL
|
|
320
|
+
weather_tool = OpenRouter::Tool.define do
|
|
321
|
+
name "get_weather"
|
|
322
|
+
description "Get current weather for a location"
|
|
323
|
+
|
|
324
|
+
parameters do
|
|
325
|
+
string :location, required: true, description: "City name"
|
|
326
|
+
string :units, enum: ["celsius", "fahrenheit"], default: "celsius"
|
|
327
|
+
end
|
|
328
|
+
end
|
|
329
|
+
|
|
330
|
+
# Use in completion
|
|
331
|
+
response = client.complete(
|
|
332
|
+
[{ role: "user", content: "What's the weather in London?" }],
|
|
333
|
+
model: "anthropic/claude-3.5-sonnet",
|
|
334
|
+
tools: [weather_tool],
|
|
335
|
+
tool_choice: "auto"
|
|
336
|
+
)
|
|
337
|
+
|
|
338
|
+
# Handle tool calls
|
|
339
|
+
if response.has_tool_calls?
|
|
340
|
+
response.tool_calls.each do |tool_call|
|
|
341
|
+
result = fetch_weather(tool_call.arguments["location"], tool_call.arguments["units"])
|
|
342
|
+
puts "Weather in #{tool_call.arguments['location']}: #{result}"
|
|
343
|
+
end
|
|
344
|
+
end
|
|
345
|
+
```
|
|
346
|
+
|
|
347
|
+
#### Key Features
|
|
348
|
+
|
|
349
|
+
- **Ruby DSL**: Define tools with intuitive Ruby syntax
|
|
350
|
+
- **Parameter Validation**: Automatic validation against JSON Schema
|
|
351
|
+
- **Tool Choice Control**: Auto, required, none, or specific tool selection
|
|
352
|
+
- **Conversation Continuation**: Easy message building for multi-turn conversations
|
|
353
|
+
- **Error Handling**: Graceful error handling and validation
|
|
354
|
+
|
|
355
|
+
đź“– **[Complete Tool Calling Documentation](docs/tools.md)**
|
|
356
|
+
|
|
357
|
+
### Structured Outputs
|
|
358
|
+
|
|
359
|
+
Get JSON responses that conform to specific schemas with automatic validation and healing for non-native models.
|
|
360
|
+
|
|
361
|
+
#### Quick Example
|
|
362
|
+
|
|
363
|
+
```ruby
|
|
364
|
+
# Define a schema using the DSL
|
|
365
|
+
user_schema = OpenRouter::Schema.define("user") do
|
|
366
|
+
string :name, required: true, description: "Full name"
|
|
367
|
+
integer :age, required: true, minimum: 0, maximum: 150
|
|
368
|
+
string :email, required: true, description: "Email address"
|
|
369
|
+
boolean :premium, description: "Premium account status"
|
|
370
|
+
end
|
|
371
|
+
|
|
372
|
+
# Get structured response
|
|
373
|
+
response = client.complete(
|
|
374
|
+
[{ role: "user", content: "Create a user: John Doe, 30, john@example.com" }],
|
|
375
|
+
model: "openai/gpt-4o",
|
|
376
|
+
response_format: user_schema
|
|
377
|
+
)
|
|
378
|
+
|
|
379
|
+
# Access parsed JSON data
|
|
380
|
+
user = response.structured_output
|
|
381
|
+
puts user["name"] # => "John Doe"
|
|
382
|
+
puts user["age"] # => 30
|
|
383
|
+
puts user["email"] # => "john@example.com"
|
|
384
|
+
```
|
|
385
|
+
|
|
386
|
+
#### Key Features
|
|
387
|
+
|
|
388
|
+
- **Ruby DSL**: Define JSON schemas with Ruby syntax
|
|
389
|
+
- **Automatic Healing**: Self-healing for models without native structured output support
|
|
390
|
+
- **Validation**: Optional validation with detailed error reporting
|
|
391
|
+
- **Complex Schemas**: Support for nested objects, arrays, and advanced constraints
|
|
392
|
+
- **Fallback Support**: Graceful degradation for unsupported models
|
|
393
|
+
|
|
394
|
+
đź“– **[Complete Structured Outputs Documentation](docs/structured_outputs.md)**
|
|
395
|
+
|
|
396
|
+
### Smart Model Selection
|
|
397
|
+
|
|
398
|
+
Automatically choose the best AI model based on your specific requirements using a fluent DSL.
|
|
399
|
+
|
|
400
|
+
#### Quick Example
|
|
401
|
+
|
|
402
|
+
```ruby
|
|
403
|
+
# Find the cheapest model with function calling
|
|
404
|
+
model = OpenRouter::ModelSelector.new
|
|
405
|
+
.require(:function_calling)
|
|
406
|
+
.optimize_for(:cost)
|
|
407
|
+
.choose
|
|
408
|
+
|
|
409
|
+
# Advanced selection with multiple criteria
|
|
410
|
+
model = OpenRouter::ModelSelector.new
|
|
411
|
+
.require(:function_calling, :vision)
|
|
412
|
+
.within_budget(max_cost: 0.01)
|
|
413
|
+
.min_context(50_000)
|
|
414
|
+
.prefer_providers("anthropic", "openai")
|
|
415
|
+
.optimize_for(:performance)
|
|
416
|
+
.choose
|
|
417
|
+
|
|
418
|
+
# Get multiple options with fallbacks
|
|
419
|
+
models = OpenRouter::ModelSelector.new
|
|
420
|
+
.require(:structured_outputs)
|
|
421
|
+
.choose_with_fallbacks(limit: 3)
|
|
422
|
+
# => ["openai/gpt-4o-mini", "anthropic/claude-3-haiku", "google/gemini-flash"]
|
|
423
|
+
```
|
|
424
|
+
|
|
425
|
+
#### Key Features
|
|
426
|
+
|
|
427
|
+
- **Fluent DSL**: Chain requirements and preferences intuitively
|
|
428
|
+
- **Cost Optimization**: Find models within budget constraints
|
|
429
|
+
- **Capability Matching**: Require specific features like function calling or vision
|
|
430
|
+
- **Provider Preferences**: Prefer or avoid specific providers
|
|
431
|
+
- **Graceful Fallbacks**: Automatic fallback with requirement relaxation
|
|
432
|
+
- **Performance Tiers**: Choose between cost and performance optimization
|
|
433
|
+
|
|
434
|
+
đź“– **[Complete Model Selection Documentation](docs/model_selection.md)**
|
|
435
|
+
|
|
436
|
+
### Prompt Templates
|
|
437
|
+
|
|
438
|
+
Create reusable, parameterized prompts with variable interpolation and few-shot learning support.
|
|
439
|
+
|
|
440
|
+
#### Quick Example
|
|
441
|
+
|
|
442
|
+
```ruby
|
|
443
|
+
# Basic template with variables
|
|
444
|
+
translation_template = OpenRouter::PromptTemplate.new(
|
|
445
|
+
template: "Translate '{text}' from {source_lang} to {target_lang}",
|
|
446
|
+
input_variables: [:text, :source_lang, :target_lang]
|
|
447
|
+
)
|
|
448
|
+
|
|
449
|
+
# Use with client
|
|
450
|
+
client = OpenRouter::Client.new
|
|
451
|
+
response = client.complete(
|
|
452
|
+
translation_template.to_messages(
|
|
453
|
+
text: "Hello world",
|
|
454
|
+
source_lang: "English",
|
|
455
|
+
target_lang: "French"
|
|
456
|
+
),
|
|
457
|
+
model: "openai/gpt-4o-mini"
|
|
458
|
+
)
|
|
459
|
+
|
|
460
|
+
# Few-shot learning template
|
|
461
|
+
classification_template = OpenRouter::PromptTemplate.new(
|
|
462
|
+
prefix: "Classify the sentiment of the following text. Examples:",
|
|
463
|
+
suffix: "Now classify: {text}",
|
|
464
|
+
examples: [
|
|
465
|
+
{ text: "I love this product!", sentiment: "positive" },
|
|
466
|
+
{ text: "This is terrible.", sentiment: "negative" },
|
|
467
|
+
{ text: "It's okay, nothing special.", sentiment: "neutral" }
|
|
468
|
+
],
|
|
469
|
+
example_template: "Text: {text}\nSentiment: {sentiment}",
|
|
470
|
+
input_variables: [:text]
|
|
471
|
+
)
|
|
472
|
+
|
|
473
|
+
# Render complete prompt
|
|
474
|
+
prompt = classification_template.format(text: "This is amazing!")
|
|
475
|
+
puts prompt
|
|
476
|
+
# =>
|
|
477
|
+
# Classify the sentiment of the following text. Examples:
|
|
478
|
+
#
|
|
479
|
+
# Text: I love this product!
|
|
480
|
+
# Sentiment: positive
|
|
481
|
+
#
|
|
482
|
+
# Text: This is terrible.
|
|
483
|
+
# Sentiment: negative
|
|
484
|
+
#
|
|
485
|
+
# Text: It's okay, nothing special.
|
|
486
|
+
# Sentiment: neutral
|
|
487
|
+
#
|
|
488
|
+
# Now classify: This is amazing!
|
|
489
|
+
```
|
|
490
|
+
|
|
491
|
+
#### Key Features
|
|
492
|
+
|
|
493
|
+
- **Variable Interpolation**: Use `{variable}` syntax for dynamic content
|
|
494
|
+
- **Few-Shot Learning**: Include examples to improve model performance
|
|
495
|
+
- **Chat Formatting**: Automatic conversion to OpenRouter message format
|
|
496
|
+
- **Partial Variables**: Pre-fill common variables for reuse
|
|
497
|
+
- **Template Composition**: Combine templates for complex prompts
|
|
498
|
+
- **Validation**: Automatic validation of required input variables
|
|
499
|
+
|
|
500
|
+
đź“– **[Complete Prompt Templates Documentation](docs/prompt_templates.md)**
|
|
501
|
+
|
|
502
|
+
### Model Registry
|
|
503
|
+
|
|
504
|
+
Access detailed information about available models and their capabilities.
|
|
505
|
+
|
|
506
|
+
#### Quick Example
|
|
507
|
+
|
|
508
|
+
```ruby
|
|
509
|
+
# Get specific model information
|
|
510
|
+
model_info = OpenRouter::ModelRegistry.get_model_info("anthropic/claude-3-5-sonnet")
|
|
511
|
+
puts model_info[:capabilities] # [:chat, :function_calling, :structured_outputs, :vision]
|
|
512
|
+
puts model_info[:cost_per_1k_tokens] # { input: 0.003, output: 0.015 }
|
|
513
|
+
|
|
514
|
+
# Find models matching requirements
|
|
515
|
+
candidates = OpenRouter::ModelRegistry.models_meeting_requirements(
|
|
516
|
+
capabilities: [:function_calling],
|
|
517
|
+
max_input_cost: 0.01
|
|
518
|
+
)
|
|
519
|
+
|
|
520
|
+
# Estimate costs for specific usage
|
|
521
|
+
cost = OpenRouter::ModelRegistry.calculate_estimated_cost(
|
|
522
|
+
"openai/gpt-4o",
|
|
523
|
+
input_tokens: 1000,
|
|
524
|
+
output_tokens: 500
|
|
525
|
+
)
|
|
526
|
+
puts "Estimated cost: $#{cost.round(4)}" # => "Estimated cost: $0.0105"
|
|
527
|
+
```
|
|
528
|
+
|
|
529
|
+
#### Key Features
|
|
530
|
+
|
|
531
|
+
- **Model Discovery**: Browse all available models and their specifications
|
|
532
|
+
- **Capability Detection**: Check which features each model supports
|
|
533
|
+
- **Cost Calculation**: Estimate costs for specific token usage
|
|
534
|
+
- **Local Caching**: Fast model data access with automatic cache management
|
|
535
|
+
- **Real-time Updates**: Refresh model data from OpenRouter API
|
|
536
|
+
|
|
537
|
+
## Streaming & Real-time
|
|
538
|
+
|
|
539
|
+
### Streaming Client
|
|
540
|
+
|
|
541
|
+
The enhanced streaming client provides real-time response streaming with callback support and automatic response reconstruction.
|
|
542
|
+
|
|
543
|
+
#### Quick Example
|
|
544
|
+
|
|
545
|
+
```ruby
|
|
546
|
+
# Create streaming client
|
|
547
|
+
streaming_client = OpenRouter::StreamingClient.new
|
|
548
|
+
|
|
549
|
+
# Set up callbacks
|
|
550
|
+
streaming_client
|
|
551
|
+
.on_stream(:on_start) { |data| puts "Starting request to #{data[:model]}" }
|
|
552
|
+
.on_stream(:on_chunk) { |chunk| print chunk.content }
|
|
553
|
+
.on_stream(:on_tool_call_chunk) { |chunk| puts "Tool call: #{chunk.name}" }
|
|
554
|
+
.on_stream(:on_finish) { |response| puts "\nCompleted. Total tokens: #{response.total_tokens}" }
|
|
555
|
+
.on_stream(:on_error) { |error| puts "Error: #{error.message}" }
|
|
556
|
+
|
|
557
|
+
# Stream with automatic response accumulation
|
|
558
|
+
response = streaming_client.stream_complete(
|
|
559
|
+
[{ role: "user", content: "Write a short story about a robot" }],
|
|
560
|
+
model: "openai/gpt-4o-mini",
|
|
561
|
+
accumulate_response: true
|
|
562
|
+
)
|
|
563
|
+
|
|
564
|
+
# Access complete response after streaming
|
|
565
|
+
puts "Final response: #{response.content}"
|
|
566
|
+
puts "Cost: $#{response.cost_estimate}"
|
|
567
|
+
```
|
|
568
|
+
|
|
569
|
+
#### Streaming with Tool Calls
|
|
570
|
+
|
|
571
|
+
```ruby
|
|
572
|
+
# Define a tool
|
|
573
|
+
weather_tool = OpenRouter::Tool.define do
|
|
574
|
+
name "get_weather"
|
|
575
|
+
description "Get current weather"
|
|
576
|
+
parameters { string :location, required: true }
|
|
577
|
+
end
|
|
578
|
+
|
|
579
|
+
# Stream with tool calling
|
|
580
|
+
streaming_client.stream_complete(
|
|
581
|
+
[{ role: "user", content: "What's the weather in Tokyo?" }],
|
|
582
|
+
model: "anthropic/claude-3-5-sonnet",
|
|
583
|
+
tools: [weather_tool]
|
|
584
|
+
) do |chunk|
|
|
585
|
+
if chunk.has_tool_calls?
|
|
586
|
+
chunk.tool_calls.each do |tool_call|
|
|
587
|
+
puts "Calling #{tool_call.name} with #{tool_call.arguments}"
|
|
588
|
+
end
|
|
589
|
+
else
|
|
590
|
+
print chunk.content
|
|
591
|
+
end
|
|
592
|
+
end
|
|
593
|
+
```
|
|
594
|
+
|
|
595
|
+
### Streaming Callbacks
|
|
596
|
+
|
|
597
|
+
The streaming client supports extensive callback events for monitoring and analytics.
|
|
598
|
+
|
|
599
|
+
```ruby
|
|
600
|
+
streaming_client = OpenRouter::StreamingClient.new
|
|
601
|
+
|
|
602
|
+
# Monitor token usage in real-time
|
|
603
|
+
streaming_client.on_stream(:on_chunk) do |chunk|
|
|
604
|
+
if chunk.usage
|
|
605
|
+
puts "Tokens so far: #{chunk.usage['total_tokens']}"
|
|
606
|
+
end
|
|
607
|
+
end
|
|
608
|
+
|
|
609
|
+
# Handle errors gracefully
|
|
610
|
+
streaming_client.on_stream(:on_error) do |error|
|
|
611
|
+
logger.error "Streaming failed: #{error.message}"
|
|
612
|
+
# Implement fallback logic
|
|
613
|
+
fallback_response = client.complete(messages, model: "openai/gpt-4o-mini")
|
|
614
|
+
end
|
|
615
|
+
|
|
616
|
+
# Track performance metrics
|
|
617
|
+
start_time = nil
|
|
618
|
+
streaming_client
|
|
619
|
+
.on_stream(:on_start) { |data| start_time = Time.now }
|
|
620
|
+
.on_stream(:on_finish) do |response|
|
|
621
|
+
duration = Time.now - start_time
|
|
622
|
+
puts "Request completed in #{duration.round(2)}s"
|
|
623
|
+
puts "Tokens per second: #{response.total_tokens / duration}"
|
|
624
|
+
end
|
|
625
|
+
```
|
|
626
|
+
|
|
627
|
+
## Observability & Analytics
|
|
628
|
+
|
|
629
|
+
### Usage Tracking
|
|
630
|
+
|
|
631
|
+
Track token usage, costs, and performance metrics across all API calls.
|
|
632
|
+
|
|
633
|
+
#### Quick Example
|
|
634
|
+
|
|
635
|
+
```ruby
|
|
636
|
+
# Create client with usage tracking enabled
|
|
637
|
+
client = OpenRouter::Client.new(track_usage: true)
|
|
638
|
+
|
|
639
|
+
# Make multiple requests
|
|
640
|
+
3.times do |i|
|
|
641
|
+
response = client.complete(
|
|
642
|
+
[{ role: "user", content: "Tell me a fact about space #{i + 1}" }],
|
|
643
|
+
model: "openai/gpt-4o-mini"
|
|
644
|
+
)
|
|
645
|
+
puts "Request #{i + 1}: #{response.total_tokens} tokens, $#{response.cost_estimate}"
|
|
646
|
+
end
|
|
647
|
+
|
|
648
|
+
# View comprehensive usage statistics
|
|
649
|
+
tracker = client.usage_tracker
|
|
650
|
+
puts "\n=== Usage Summary ==="
|
|
651
|
+
puts "Total requests: #{tracker.request_count}"
|
|
652
|
+
puts "Total tokens: #{tracker.total_tokens}"
|
|
653
|
+
puts "Total cost: $#{tracker.total_cost.round(4)}"
|
|
654
|
+
puts "Average cost per request: $#{(tracker.total_cost / tracker.request_count).round(4)}"
|
|
655
|
+
|
|
656
|
+
# View per-model breakdown
|
|
657
|
+
tracker.model_usage.each do |model, stats|
|
|
658
|
+
puts "\n#{model}:"
|
|
659
|
+
puts " Requests: #{stats[:request_count]}"
|
|
660
|
+
puts " Tokens: #{stats[:total_tokens]}"
|
|
661
|
+
puts " Cost: $#{stats[:cost].round(4)}"
|
|
662
|
+
end
|
|
663
|
+
|
|
664
|
+
# Print detailed report
|
|
665
|
+
tracker.print_summary
|
|
666
|
+
```
|
|
667
|
+
|
|
668
|
+
#### Advanced Usage Tracking
|
|
669
|
+
|
|
670
|
+
```ruby
|
|
671
|
+
# Track specific operations
|
|
672
|
+
client.usage_tracker.reset! # Start fresh
|
|
673
|
+
|
|
674
|
+
# Simulate different workload types
|
|
675
|
+
client.complete(messages, model: "openai/gpt-4o") # Expensive, high-quality
|
|
676
|
+
client.complete(messages, model: "openai/gpt-4o-mini") # Cheap, fast
|
|
677
|
+
|
|
678
|
+
# Get usage metrics
|
|
679
|
+
cache_hit_rate = client.usage_tracker.cache_hit_rate
|
|
680
|
+
tokens_per_second = client.usage_tracker.tokens_per_second
|
|
681
|
+
|
|
682
|
+
puts "Cache hit rate: #{cache_hit_rate}%"
|
|
683
|
+
puts "Tokens per second: #{tokens_per_second}"
|
|
684
|
+
|
|
685
|
+
# Export usage data as CSV for analysis
|
|
686
|
+
csv_data = client.usage_tracker.export_csv
|
|
687
|
+
File.write("usage_report.csv", csv_data)
|
|
688
|
+
```
|
|
689
|
+
|
|
690
|
+
### Response Analytics
|
|
691
|
+
|
|
692
|
+
Every response includes comprehensive metadata for monitoring and optimization.
|
|
693
|
+
|
|
694
|
+
```ruby
|
|
695
|
+
response = client.complete(messages, model: "anthropic/claude-3-5-sonnet")
|
|
696
|
+
|
|
697
|
+
# Token metrics
|
|
698
|
+
puts "Input tokens: #{response.prompt_tokens}"
|
|
699
|
+
puts "Output tokens: #{response.completion_tokens}"
|
|
700
|
+
puts "Cached tokens: #{response.cached_tokens}"
|
|
701
|
+
puts "Total tokens: #{response.total_tokens}"
|
|
702
|
+
|
|
703
|
+
# Cost information (requires generation stats query)
|
|
704
|
+
puts "Total cost: $#{response.cost_estimate}"
|
|
705
|
+
|
|
706
|
+
# Model information
|
|
707
|
+
puts "Provider: #{response.provider}"
|
|
708
|
+
puts "Model: #{response.model}"
|
|
709
|
+
puts "System fingerprint: #{response.system_fingerprint}"
|
|
710
|
+
puts "Finish reason: #{response.finish_reason}"
|
|
711
|
+
```
|
|
712
|
+
|
|
713
|
+
### Callback System
|
|
714
|
+
|
|
715
|
+
The client provides an extensible callback system for monitoring requests, responses, and errors.
|
|
716
|
+
|
|
717
|
+
#### Basic Callbacks
|
|
718
|
+
|
|
719
|
+
```ruby
|
|
720
|
+
client = OpenRouter::Client.new
|
|
721
|
+
|
|
722
|
+
# Monitor all requests
|
|
723
|
+
client.on(:before_request) do |params|
|
|
724
|
+
puts "Making request to #{params[:model]} with #{params[:messages].size} messages"
|
|
725
|
+
end
|
|
726
|
+
|
|
727
|
+
# Monitor all responses
|
|
728
|
+
client.on(:after_response) do |response|
|
|
729
|
+
puts "Received response: #{response.total_tokens} tokens, $#{response.cost_estimate}"
|
|
730
|
+
end
|
|
731
|
+
|
|
732
|
+
# Monitor tool calls
|
|
733
|
+
client.on(:on_tool_call) do |tool_calls|
|
|
734
|
+
tool_calls.each do |call|
|
|
735
|
+
puts "Tool called: #{call.name} with args #{call.arguments}"
|
|
736
|
+
end
|
|
737
|
+
end
|
|
738
|
+
|
|
739
|
+
# Monitor errors
|
|
740
|
+
client.on(:on_error) do |error|
|
|
741
|
+
logger.error "API error: #{error.message}"
|
|
742
|
+
# Send to monitoring service
|
|
743
|
+
ErrorReporter.notify(error)
|
|
744
|
+
end
|
|
745
|
+
```
|
|
746
|
+
|
|
747
|
+
#### Advanced Callback Usage
|
|
748
|
+
|
|
749
|
+
```ruby
|
|
750
|
+
# Cost monitoring with alerts
|
|
751
|
+
client.on(:after_response) do |response|
|
|
752
|
+
if response.cost_estimate > 0.10
|
|
753
|
+
AlertService.send_alert(
|
|
754
|
+
"High cost request: $#{response.cost_estimate} for #{response.total_tokens} tokens"
|
|
755
|
+
)
|
|
756
|
+
end
|
|
757
|
+
end
|
|
758
|
+
|
|
759
|
+
# Performance monitoring
|
|
760
|
+
client.on(:before_request) { |params| @start_time = Time.now }
|
|
761
|
+
client.on(:after_response) do |response|
|
|
762
|
+
duration = Time.now - @start_time
|
|
763
|
+
if duration > 10.0
|
|
764
|
+
puts "Slow request detected: #{duration.round(2)}s"
|
|
765
|
+
end
|
|
766
|
+
end
|
|
767
|
+
|
|
768
|
+
# Usage analytics
|
|
769
|
+
request_count = 0
|
|
770
|
+
total_cost = 0.0
|
|
771
|
+
|
|
772
|
+
client.on(:after_response) do |response|
|
|
773
|
+
request_count += 1
|
|
774
|
+
total_cost += response.cost_estimate || 0.0
|
|
775
|
+
|
|
776
|
+
if request_count % 100 == 0
|
|
777
|
+
puts "100 requests processed. Average cost: $#{(total_cost / request_count).round(4)}"
|
|
778
|
+
end
|
|
779
|
+
end
|
|
780
|
+
|
|
781
|
+
# Chain callbacks for complex workflows
|
|
782
|
+
client
|
|
783
|
+
.on(:before_request) { |params| log_request(params) }
|
|
784
|
+
.on(:after_response) { |response| log_response(response) }
|
|
785
|
+
.on(:on_tool_call) { |calls| execute_tools(calls) }
|
|
786
|
+
.on(:on_error) { |error| handle_error(error) }
|
|
787
|
+
```
|
|
788
|
+
|
|
789
|
+
### Cost Management
|
|
790
|
+
|
|
791
|
+
Built-in cost estimation and usage tracking tools.
|
|
792
|
+
|
|
793
|
+
```ruby
|
|
794
|
+
# Pre-flight cost estimation
|
|
795
|
+
estimated_cost = OpenRouter::ModelRegistry.calculate_estimated_cost(
|
|
796
|
+
"anthropic/claude-3-5-sonnet",
|
|
797
|
+
input_tokens: 1500,
|
|
798
|
+
output_tokens: 800
|
|
799
|
+
)
|
|
800
|
+
|
|
801
|
+
puts "Estimated cost: $#{estimated_cost}"
|
|
802
|
+
|
|
803
|
+
# Use model selector to stay within budget
|
|
804
|
+
if estimated_cost > 0.01
|
|
805
|
+
puts "Switching to cheaper model"
|
|
806
|
+
model = OpenRouter::ModelSelector.new
|
|
807
|
+
.within_budget(max_cost: 0.01)
|
|
808
|
+
.require(:chat)
|
|
809
|
+
.choose
|
|
810
|
+
end
|
|
811
|
+
|
|
812
|
+
# Track costs in real-time
|
|
813
|
+
client = OpenRouter::Client.new(track_usage: true)
|
|
814
|
+
|
|
815
|
+
client.on(:after_response) do |response|
|
|
816
|
+
total_spent = client.usage_tracker.total_cost
|
|
817
|
+
puts "Total spent this session: $#{total_spent.round(4)}"
|
|
818
|
+
|
|
819
|
+
if total_spent > 5.00
|
|
820
|
+
puts "⚠️ Session cost exceeds $5.00"
|
|
821
|
+
end
|
|
822
|
+
end
|
|
823
|
+
```
|
|
824
|
+
|
|
825
|
+
## Advanced Features
|
|
826
|
+
|
|
827
|
+
### Model Fallbacks
|
|
828
|
+
|
|
829
|
+
Use multiple models with automatic failover for increased reliability.
|
|
830
|
+
|
|
831
|
+
```ruby
|
|
832
|
+
# Define fallback chain
|
|
833
|
+
response = client.complete(
|
|
834
|
+
messages,
|
|
835
|
+
model: ["openai/gpt-4o", "anthropic/claude-3-5-sonnet", "anthropic/claude-3-haiku"],
|
|
836
|
+
tools: tools
|
|
837
|
+
)
|
|
838
|
+
|
|
839
|
+
# Or use ModelSelector for intelligent fallbacks
|
|
840
|
+
models = OpenRouter::ModelSelector.new
|
|
841
|
+
.require(:function_calling)
|
|
842
|
+
.choose_with_fallbacks(limit: 3)
|
|
843
|
+
|
|
844
|
+
response = client.complete(messages, model: models, tools: tools)
|
|
845
|
+
```
|
|
846
|
+
|
|
847
|
+
### Response Healing
|
|
848
|
+
|
|
849
|
+
Automatically heal malformed responses from models that don't natively support structured outputs.
|
|
850
|
+
|
|
851
|
+
```ruby
|
|
852
|
+
# Configure global healing
|
|
853
|
+
OpenRouter.configure do |config|
|
|
854
|
+
config.auto_heal_responses = true
|
|
855
|
+
config.healer_model = "openai/gpt-4o-mini"
|
|
856
|
+
config.max_heal_attempts = 2
|
|
857
|
+
end
|
|
858
|
+
|
|
859
|
+
# The gem automatically heals malformed JSON responses
|
|
860
|
+
response = client.complete(
|
|
861
|
+
messages,
|
|
862
|
+
model: "some/model-without-native-structured-outputs",
|
|
863
|
+
response_format: schema # Will be automatically healed if malformed
|
|
864
|
+
)
|
|
865
|
+
```
|
|
866
|
+
|
|
867
|
+
### Performance Optimization
|
|
868
|
+
|
|
869
|
+
Optimize performance for high-throughput applications.
|
|
870
|
+
|
|
871
|
+
#### Batching and Parallelization
|
|
872
|
+
|
|
873
|
+
```ruby
|
|
874
|
+
require 'concurrent-ruby'
|
|
875
|
+
|
|
876
|
+
# Process multiple requests in parallel
|
|
877
|
+
messages_batch = [
|
|
878
|
+
[{ role: "user", content: "Summarize this: #{text1}" }],
|
|
879
|
+
[{ role: "user", content: "Summarize this: #{text2}" }],
|
|
880
|
+
[{ role: "user", content: "Summarize this: #{text3}" }]
|
|
881
|
+
]
|
|
882
|
+
|
|
883
|
+
# Create thread pool
|
|
884
|
+
thread_pool = Concurrent::FixedThreadPool.new(5)
|
|
885
|
+
|
|
886
|
+
# Process batch with shared model selection
|
|
887
|
+
model = OpenRouter::ModelSelector.new
|
|
888
|
+
.optimize_for(:performance)
|
|
889
|
+
.require(:chat)
|
|
890
|
+
.choose
|
|
891
|
+
|
|
892
|
+
futures = messages_batch.map do |messages|
|
|
893
|
+
Concurrent::Future.execute(executor: thread_pool) do
|
|
894
|
+
client.complete(messages, model: model)
|
|
895
|
+
end
|
|
896
|
+
end
|
|
897
|
+
|
|
898
|
+
# Collect results
|
|
899
|
+
results = futures.map(&:value)
|
|
900
|
+
thread_pool.shutdown
|
|
901
|
+
```
|
|
902
|
+
|
|
903
|
+
#### Caching and Optimization
|
|
904
|
+
|
|
905
|
+
```ruby
|
|
906
|
+
# Enable aggressive caching
|
|
907
|
+
OpenRouter.configure do |config|
|
|
908
|
+
config.cache_ttl = 24 * 60 * 60 # 24 hours
|
|
909
|
+
config.auto_heal_responses = true
|
|
910
|
+
config.strict_mode = false # Better performance
|
|
911
|
+
end
|
|
912
|
+
|
|
913
|
+
# Use cheaper models for development/testing
|
|
914
|
+
if Rails.env.development?
|
|
915
|
+
client = OpenRouter::Client.new(
|
|
916
|
+
default_model: "openai/gpt-4o-mini", # Cheaper for development
|
|
917
|
+
track_usage: true
|
|
918
|
+
)
|
|
919
|
+
else
|
|
920
|
+
client = OpenRouter::Client.new(
|
|
921
|
+
default_model: "anthropic/claude-3-5-sonnet" # Production quality
|
|
922
|
+
)
|
|
923
|
+
end
|
|
924
|
+
|
|
925
|
+
# Pre-warm model registry cache
|
|
926
|
+
OpenRouter::ModelRegistry.refresh_cache!
|
|
927
|
+
|
|
928
|
+
# Optimize for specific workloads
|
|
929
|
+
fast_client = OpenRouter::Client.new(
|
|
930
|
+
request_timeout: 30, # Shorter timeout
|
|
931
|
+
auto_heal_responses: false, # Skip healing for speed
|
|
932
|
+
strict_mode: false # Skip capability validation
|
|
933
|
+
)
|
|
934
|
+
```
|
|
935
|
+
|
|
936
|
+
#### Memory Management
|
|
937
|
+
|
|
938
|
+
```ruby
|
|
939
|
+
# Reset usage tracking periodically for long-running apps
|
|
940
|
+
client.usage_tracker.reset! if client.usage_tracker.request_count > 1000
|
|
941
|
+
|
|
942
|
+
# Clear callback chains when not needed
|
|
943
|
+
client.clear_callbacks(:after_response) if Rails.env.production?
|
|
944
|
+
|
|
945
|
+
# Use streaming for large responses to reduce memory usage
|
|
946
|
+
streaming_client = OpenRouter::StreamingClient.new
|
|
947
|
+
|
|
948
|
+
streaming_client.stream_complete(
|
|
949
|
+
[{ role: "user", content: "Write a detailed report on AI trends" }],
|
|
950
|
+
model: "anthropic/claude-3-5-sonnet",
|
|
951
|
+
accumulate_response: false # Don't store full response
|
|
952
|
+
) do |chunk|
|
|
953
|
+
# Process chunk immediately and discard
|
|
954
|
+
process_chunk(chunk.content)
|
|
955
|
+
end
|
|
956
|
+
```
|
|
957
|
+
|
|
958
|
+
## Testing & Development
|
|
959
|
+
|
|
960
|
+
The gem includes comprehensive test coverage with VCR integration for real API testing.
|
|
961
|
+
|
|
962
|
+
### Running Tests
|
|
963
|
+
|
|
964
|
+
```bash
|
|
965
|
+
# Run all tests
|
|
966
|
+
bundle exec rspec
|
|
967
|
+
|
|
968
|
+
# Run with documentation format
|
|
969
|
+
bundle exec rspec --format documentation
|
|
970
|
+
|
|
971
|
+
# Run specific test types
|
|
972
|
+
bundle exec rspec spec/unit/ # Unit tests only
|
|
973
|
+
bundle exec rspec spec/vcr/ # VCR integration tests (requires API key)
|
|
974
|
+
```
|
|
975
|
+
|
|
976
|
+
### VCR Testing
|
|
977
|
+
|
|
978
|
+
The project includes VCR tests that record real API interactions:
|
|
979
|
+
|
|
980
|
+
```bash
|
|
981
|
+
# Set API key for VCR tests
|
|
982
|
+
export OPENROUTER_API_KEY="your_api_key"
|
|
983
|
+
|
|
984
|
+
# Run VCR tests
|
|
985
|
+
bundle exec rspec spec/vcr/
|
|
986
|
+
|
|
987
|
+
# Re-record cassettes (deletes old recordings)
|
|
988
|
+
rm -rf spec/fixtures/vcr_cassettes/
|
|
989
|
+
bundle exec rspec spec/vcr/
|
|
990
|
+
```
|
|
991
|
+
|
|
992
|
+
### Examples
|
|
993
|
+
|
|
994
|
+
The project includes comprehensive examples for all features:
|
|
995
|
+
|
|
996
|
+
```bash
|
|
997
|
+
# Set your API key
|
|
998
|
+
export OPENROUTER_API_KEY="your_key_here"
|
|
999
|
+
|
|
1000
|
+
# Run individual examples
|
|
1001
|
+
ruby -I lib examples/basic_completion.rb
|
|
1002
|
+
ruby -I lib examples/tool_calling_example.rb
|
|
1003
|
+
ruby -I lib examples/structured_outputs_example.rb
|
|
1004
|
+
ruby -I lib examples/model_selection_example.rb
|
|
1005
|
+
ruby -I lib examples/prompt_template_example.rb
|
|
1006
|
+
ruby -I lib examples/streaming_example.rb
|
|
1007
|
+
ruby -I lib examples/observability_example.rb
|
|
1008
|
+
ruby -I lib examples/smart_completion_example.rb
|
|
1009
|
+
|
|
1010
|
+
# Run all examples
|
|
1011
|
+
find examples -name "*.rb" -exec ruby -I lib {} \;
|
|
1012
|
+
```
|
|
1013
|
+
|
|
1014
|
+
### Model Exploration Rake Tasks
|
|
1015
|
+
|
|
1016
|
+
The gem includes convenient rake tasks for exploring and searching available models without writing code:
|
|
1017
|
+
|
|
1018
|
+
#### Model Summary
|
|
1019
|
+
|
|
1020
|
+
View an overview of all available models, including provider breakdown, capabilities, costs, and context lengths:
|
|
1021
|
+
|
|
1022
|
+
```bash
|
|
1023
|
+
bundle exec rake models:summary
|
|
1024
|
+
```
|
|
1025
|
+
|
|
1026
|
+
Output includes:
|
|
1027
|
+
- Total model count and breakdown by provider
|
|
1028
|
+
- Available capabilities across all models
|
|
1029
|
+
- Cost analysis (min/max/median for input and output tokens)
|
|
1030
|
+
- Context length statistics
|
|
1031
|
+
- Performance tier distribution
|
|
1032
|
+
|
|
1033
|
+
#### Model Search
|
|
1034
|
+
|
|
1035
|
+
Search for models using various filters and optimization strategies:
|
|
1036
|
+
|
|
1037
|
+
```bash
|
|
1038
|
+
# Basic search by provider
|
|
1039
|
+
bundle exec rake models:search provider=anthropic
|
|
1040
|
+
|
|
1041
|
+
# Search by capabilities
|
|
1042
|
+
bundle exec rake models:search capability=function_calling,vision
|
|
1043
|
+
|
|
1044
|
+
# Optimize for cost with capability requirements
|
|
1045
|
+
bundle exec rake models:search capability=function_calling optimize=cost limit=10
|
|
1046
|
+
|
|
1047
|
+
# Filter by context length
|
|
1048
|
+
bundle exec rake models:search min_context=200000
|
|
1049
|
+
|
|
1050
|
+
# Filter by cost
|
|
1051
|
+
bundle exec rake models:search max_cost=0.01
|
|
1052
|
+
|
|
1053
|
+
# Filter by release date
|
|
1054
|
+
bundle exec rake models:search newer_than=2024-01-01
|
|
1055
|
+
|
|
1056
|
+
# Combine multiple filters
|
|
1057
|
+
bundle exec rake models:search provider=anthropic capability=function_calling min_context=100000 optimize=cost limit=5
|
|
1058
|
+
```
|
|
1059
|
+
|
|
1060
|
+
Available search parameters:
|
|
1061
|
+
- `provider=name` - Filter by provider (comma-separated for multiple)
|
|
1062
|
+
- `capability=cap1,cap2` - Required capabilities (function_calling, vision, structured_outputs, etc.)
|
|
1063
|
+
- `optimize=strategy` - Optimization strategy (cost, performance, latest, context)
|
|
1064
|
+
- `min_context=tokens` - Minimum context length
|
|
1065
|
+
- `max_cost=amount` - Maximum input cost per 1k tokens
|
|
1066
|
+
- `max_output_cost=amount` - Maximum output cost per 1k tokens
|
|
1067
|
+
- `newer_than=YYYY-MM-DD` - Filter models released after date
|
|
1068
|
+
- `limit=N` - Maximum number of results to show (default: 20)
|
|
1069
|
+
- `fallbacks=true` - Show models with fallback support
|
|
1070
|
+
|
|
1071
|
+
Examples:
|
|
1072
|
+
|
|
1073
|
+
```bash
|
|
1074
|
+
# Find cheapest models with vision support
|
|
1075
|
+
bundle exec rake models:search capability=vision optimize=cost limit=5
|
|
1076
|
+
|
|
1077
|
+
# Find latest Anthropic models with function calling
|
|
1078
|
+
bundle exec rake models:search provider=anthropic optimize=latest capability=function_calling
|
|
1079
|
+
|
|
1080
|
+
# Find high-context models for long documents
|
|
1081
|
+
bundle exec rake models:search min_context=500000 optimize=context
|
|
1082
|
+
```
|
|
1083
|
+
|
|
1084
|
+
## Troubleshooting
|
|
1085
|
+
|
|
1086
|
+
### Common Issues and Solutions
|
|
1087
|
+
|
|
1088
|
+
#### Authentication Errors
|
|
1089
|
+
|
|
1090
|
+
```ruby
|
|
1091
|
+
# Error: "OpenRouter access token missing!"
|
|
1092
|
+
# Solution: Set your API key
|
|
1093
|
+
export OPENROUTER_API_KEY="your_key_here"
|
|
1094
|
+
|
|
1095
|
+
# Or configure in code
|
|
1096
|
+
OpenRouter.configure do |config|
|
|
1097
|
+
config.access_token = ENV["OPENROUTER_API_KEY"]
|
|
1098
|
+
end
|
|
1099
|
+
|
|
1100
|
+
# Error: "Invalid API key"
|
|
1101
|
+
# Solution: Verify your key at https://openrouter.ai/keys
|
|
1102
|
+
```
|
|
1103
|
+
|
|
1104
|
+
#### Model Selection Issues
|
|
1105
|
+
|
|
1106
|
+
```ruby
|
|
1107
|
+
# Error: "Model not found or access denied"
|
|
1108
|
+
# Solution: Check model availability and your account limits
|
|
1109
|
+
begin
|
|
1110
|
+
client.complete(messages, model: "gpt-4")
|
|
1111
|
+
rescue OpenRouter::ServerError => e
|
|
1112
|
+
if e.message.include?("not found")
|
|
1113
|
+
puts "Model not available, falling back to default"
|
|
1114
|
+
client.complete(messages, model: "openai/gpt-4o-mini")
|
|
1115
|
+
end
|
|
1116
|
+
end
|
|
1117
|
+
|
|
1118
|
+
# Error: "Model doesn't support feature X"
|
|
1119
|
+
# Solution: Use ModelSelector to find compatible models
|
|
1120
|
+
model = OpenRouter::ModelSelector.new
|
|
1121
|
+
.require(:function_calling)
|
|
1122
|
+
.choose
|
|
1123
|
+
```
|
|
1124
|
+
|
|
1125
|
+
#### Rate Limiting and Costs
|
|
1126
|
+
|
|
1127
|
+
```ruby
|
|
1128
|
+
# Error: "Rate limit exceeded"
|
|
1129
|
+
# Solution: Implement exponential backoff
|
|
1130
|
+
require 'retries'
|
|
1131
|
+
|
|
1132
|
+
with_retries(max_tries: 3, base_sleep_seconds: 1, max_sleep_seconds: 60) do |attempt|
|
|
1133
|
+
client.complete(messages, model: model)
|
|
1134
|
+
end
|
|
1135
|
+
|
|
1136
|
+
# Error: "Request too expensive"
|
|
1137
|
+
# Solution: Use cheaper models or budget constraints
|
|
1138
|
+
client = OpenRouter::Client.new
|
|
1139
|
+
model = OpenRouter::ModelSelector.new
|
|
1140
|
+
.within_budget(max_cost: 0.01)
|
|
1141
|
+
.choose
|
|
1142
|
+
```
|
|
1143
|
+
|
|
1144
|
+
#### Structured Output Issues
|
|
1145
|
+
|
|
1146
|
+
```ruby
|
|
1147
|
+
# Error: "Invalid JSON response"
|
|
1148
|
+
# Solution: Enable response healing
|
|
1149
|
+
OpenRouter.configure do |config|
|
|
1150
|
+
config.auto_heal_responses = true
|
|
1151
|
+
config.healer_model = "openai/gpt-4o-mini"
|
|
1152
|
+
end
|
|
1153
|
+
|
|
1154
|
+
# Error: "Schema validation failed"
|
|
1155
|
+
# Solution: Check schema definitions and model capability
|
|
1156
|
+
schema = OpenRouter::Schema.define("user") do
|
|
1157
|
+
string :name, required: true
|
|
1158
|
+
integer :age, minimum: 0 # Add constraints
|
|
1159
|
+
end
|
|
1160
|
+
|
|
1161
|
+
# Use models that support structured outputs natively
|
|
1162
|
+
model = OpenRouter::ModelSelector.new
|
|
1163
|
+
.require(:structured_outputs)
|
|
1164
|
+
.choose
|
|
1165
|
+
```
|
|
1166
|
+
|
|
1167
|
+
#### Performance Issues
|
|
1168
|
+
|
|
1169
|
+
```ruby
|
|
1170
|
+
# Issue: Slow responses
|
|
1171
|
+
# Solution: Optimize client configuration
|
|
1172
|
+
client = OpenRouter::Client.new(
|
|
1173
|
+
request_timeout: 30, # Lower timeout
|
|
1174
|
+
strict_mode: false, # Skip capability validation
|
|
1175
|
+
auto_heal_responses: false # Skip healing for speed
|
|
1176
|
+
)
|
|
1177
|
+
|
|
1178
|
+
# Issue: High memory usage
|
|
1179
|
+
# Solution: Use streaming for large responses
|
|
1180
|
+
streaming_client = OpenRouter::StreamingClient.new
|
|
1181
|
+
streaming_client.stream_complete(messages, accumulate_response: false) do |chunk|
|
|
1182
|
+
process_chunk_immediately(chunk)
|
|
1183
|
+
end
|
|
1184
|
+
|
|
1185
|
+
# Issue: Too many API calls
|
|
1186
|
+
# Solution: Implement request batching
|
|
1187
|
+
messages_batch = [...] # Multiple message sets
|
|
1188
|
+
results = process_batch_concurrently(messages_batch, thread_pool_size: 5)
|
|
1189
|
+
```
|
|
1190
|
+
|
|
1191
|
+
#### Tool Calling Issues
|
|
1192
|
+
|
|
1193
|
+
```ruby
|
|
1194
|
+
# Error: "Tool not found"
|
|
1195
|
+
# Solution: Verify tool definitions match exactly
|
|
1196
|
+
tool = OpenRouter::Tool.define do
|
|
1197
|
+
name "get_weather" # Must match exactly in model response
|
|
1198
|
+
description "Get current weather for a location"
|
|
1199
|
+
parameters do
|
|
1200
|
+
string :location, required: true
|
|
1201
|
+
end
|
|
1202
|
+
end
|
|
1203
|
+
|
|
1204
|
+
# Error: "Invalid tool parameters"
|
|
1205
|
+
# Solution: Add parameter validation
|
|
1206
|
+
def handle_weather_tool(tool_call)
|
|
1207
|
+
location = tool_call.arguments["location"]
|
|
1208
|
+
raise ArgumentError, "Location required" if location.nil? || location.empty?
|
|
1209
|
+
|
|
1210
|
+
get_weather_data(location)
|
|
1211
|
+
end
|
|
1212
|
+
```
|
|
1213
|
+
|
|
1214
|
+
### Debug Mode
|
|
1215
|
+
|
|
1216
|
+
Enable detailed logging for troubleshooting:
|
|
1217
|
+
|
|
1218
|
+
```ruby
|
|
1219
|
+
require 'logger'
|
|
1220
|
+
|
|
1221
|
+
OpenRouter.configure do |config|
|
|
1222
|
+
config.log_errors = true
|
|
1223
|
+
config.faraday do |f|
|
|
1224
|
+
f.response :logger, Logger.new($stdout), { headers: true, bodies: true, errors: true }
|
|
1225
|
+
end
|
|
1226
|
+
end
|
|
1227
|
+
|
|
1228
|
+
# Enable callback debugging
|
|
1229
|
+
client = OpenRouter::Client.new
|
|
1230
|
+
client.on(:before_request) { |params| puts "REQUEST: #{params.inspect}" }
|
|
1231
|
+
client.on(:after_response) { |response| puts "RESPONSE: #{response.inspect}" }
|
|
1232
|
+
client.on(:on_error) { |error| puts "ERROR: #{error.message}" }
|
|
1233
|
+
```
|
|
1234
|
+
|
|
1235
|
+
### Performance Monitoring
|
|
1236
|
+
|
|
1237
|
+
```ruby
|
|
1238
|
+
# Monitor request performance
|
|
1239
|
+
client.on(:before_request) { @start_time = Time.now }
|
|
1240
|
+
client.on(:after_response) do |response|
|
|
1241
|
+
duration = Time.now - @start_time
|
|
1242
|
+
if duration > 5.0
|
|
1243
|
+
puts "SLOW REQUEST: #{duration.round(2)}s for #{response.total_tokens} tokens"
|
|
1244
|
+
end
|
|
1245
|
+
end
|
|
1246
|
+
|
|
1247
|
+
# Monitor costs
|
|
1248
|
+
client.on(:after_response) do |response|
|
|
1249
|
+
if response.cost_estimate > 0.10
|
|
1250
|
+
puts "EXPENSIVE REQUEST: $#{response.cost_estimate}"
|
|
1251
|
+
end
|
|
1252
|
+
end
|
|
1253
|
+
|
|
1254
|
+
# Export usage data as CSV for analysis
|
|
1255
|
+
csv_data = client.usage_tracker.export_csv
|
|
1256
|
+
File.write("debug_usage.csv", csv_data)
|
|
1257
|
+
```
|
|
1258
|
+
|
|
1259
|
+
### Getting Help
|
|
1260
|
+
|
|
1261
|
+
1. **Check the documentation**: Each feature has detailed documentation in the `docs/` directory
|
|
1262
|
+
2. **Review examples**: Look at working examples in the `examples/` directory
|
|
1263
|
+
3. **Enable debug mode**: Turn on logging to see request/response details
|
|
1264
|
+
4. **Check OpenRouter status**: Visit [OpenRouter Status](https://status.openrouter.ai)
|
|
1265
|
+
5. **Open an issue**: Report bugs at [GitHub Issues](https://github.com/estiens/open_router_enhanced/issues)
|
|
1266
|
+
|
|
1267
|
+
## API Reference
|
|
1268
|
+
|
|
1269
|
+
### Client Classes
|
|
1270
|
+
|
|
1271
|
+
#### OpenRouter::Client
|
|
1272
|
+
|
|
1273
|
+
Main client for OpenRouter API interactions.
|
|
1274
|
+
|
|
1275
|
+
```ruby
|
|
1276
|
+
client = OpenRouter::Client.new(
|
|
1277
|
+
access_token: "...",
|
|
1278
|
+
track_usage: false,
|
|
1279
|
+
request_timeout: 120
|
|
1280
|
+
)
|
|
1281
|
+
|
|
1282
|
+
# Core methods
|
|
1283
|
+
client.complete(messages, **options) # Chat completions with full feature support
|
|
1284
|
+
client.models # List available models
|
|
1285
|
+
client.query_generation_stats(id) # Query generation statistics
|
|
1286
|
+
|
|
1287
|
+
# Callback methods
|
|
1288
|
+
client.on(event, &block) # Register event callback
|
|
1289
|
+
client.clear_callbacks(event) # Clear callbacks for event
|
|
1290
|
+
client.trigger_callbacks(event, data) # Manually trigger callbacks
|
|
1291
|
+
|
|
1292
|
+
# Usage tracking
|
|
1293
|
+
client.usage_tracker # Access usage tracker instance
|
|
1294
|
+
```
|
|
1295
|
+
|
|
1296
|
+
#### OpenRouter::StreamingClient
|
|
1297
|
+
|
|
1298
|
+
Enhanced streaming client with callback support.
|
|
1299
|
+
|
|
1300
|
+
```ruby
|
|
1301
|
+
streaming_client = OpenRouter::StreamingClient.new
|
|
1302
|
+
|
|
1303
|
+
# Streaming methods
|
|
1304
|
+
streaming_client.stream_complete(messages, **options) # Stream with callbacks
|
|
1305
|
+
streaming_client.on_stream(event, &block) # Register streaming callbacks
|
|
1306
|
+
|
|
1307
|
+
# Available streaming events: :on_start, :on_chunk, :on_tool_call_chunk, :on_finish, :on_error
|
|
1308
|
+
```
|
|
1309
|
+
|
|
1310
|
+
### Enhanced Classes
|
|
1311
|
+
|
|
1312
|
+
#### OpenRouter::Tool
|
|
1313
|
+
|
|
1314
|
+
Define and manage function calling tools.
|
|
1315
|
+
|
|
1316
|
+
```ruby
|
|
1317
|
+
# DSL definition
|
|
1318
|
+
tool = OpenRouter::Tool.define do
|
|
1319
|
+
name "function_name"
|
|
1320
|
+
description "Function description"
|
|
1321
|
+
parameters do
|
|
1322
|
+
string :param1, required: true, description: "Parameter description"
|
|
1323
|
+
integer :param2, minimum: 0, maximum: 100
|
|
1324
|
+
boolean :param3, default: false
|
|
1325
|
+
end
|
|
1326
|
+
end
|
|
1327
|
+
|
|
1328
|
+
# Hash definition
|
|
1329
|
+
tool = OpenRouter::Tool.from_hash({
|
|
1330
|
+
name: "function_name",
|
|
1331
|
+
description: "Function description",
|
|
1332
|
+
parameters: {
|
|
1333
|
+
type: "object",
|
|
1334
|
+
properties: { ... }
|
|
1335
|
+
}
|
|
1336
|
+
})
|
|
1337
|
+
|
|
1338
|
+
# Methods
|
|
1339
|
+
tool.name # Get tool name
|
|
1340
|
+
tool.description # Get tool description
|
|
1341
|
+
tool.parameters # Get parameters schema
|
|
1342
|
+
tool.to_h # Convert to hash format
|
|
1343
|
+
tool.validate_arguments(args) # Validate arguments against schema
|
|
1344
|
+
```
|
|
1345
|
+
|
|
1346
|
+
#### OpenRouter::Schema
|
|
1347
|
+
|
|
1348
|
+
Define JSON schemas for structured outputs.
|
|
1349
|
+
|
|
1350
|
+
```ruby
|
|
1351
|
+
# DSL definition
|
|
1352
|
+
schema = OpenRouter::Schema.define("schema_name") do
|
|
1353
|
+
string :name, required: true, description: "User's name"
|
|
1354
|
+
integer :age, minimum: 0, maximum: 150
|
|
1355
|
+
boolean :active, default: true
|
|
1356
|
+
array :tags, items: { type: "string" }
|
|
1357
|
+
object :address do
|
|
1358
|
+
string :street, required: true
|
|
1359
|
+
string :city, required: true
|
|
1360
|
+
string :country, default: "US"
|
|
1361
|
+
end
|
|
1362
|
+
end
|
|
1363
|
+
|
|
1364
|
+
# Hash definition
|
|
1365
|
+
schema = OpenRouter::Schema.from_hash("schema_name", {
|
|
1366
|
+
type: "object",
|
|
1367
|
+
properties: { ... },
|
|
1368
|
+
required: [...]
|
|
1369
|
+
})
|
|
1370
|
+
|
|
1371
|
+
# Methods
|
|
1372
|
+
schema.name # Get schema name
|
|
1373
|
+
schema.schema # Get JSON schema hash
|
|
1374
|
+
schema.validate(data) # Validate data against schema
|
|
1375
|
+
schema.to_h # Convert to hash format
|
|
1376
|
+
```
|
|
1377
|
+
|
|
1378
|
+
#### OpenRouter::PromptTemplate
|
|
1379
|
+
|
|
1380
|
+
Create reusable prompt templates with variable interpolation.
|
|
1381
|
+
|
|
1382
|
+
```ruby
|
|
1383
|
+
# Basic template
|
|
1384
|
+
template = OpenRouter::PromptTemplate.new(
|
|
1385
|
+
template: "Translate '{text}' from {source} to {target}",
|
|
1386
|
+
input_variables: [:text, :source, :target]
|
|
1387
|
+
)
|
|
1388
|
+
|
|
1389
|
+
# Few-shot template
|
|
1390
|
+
template = OpenRouter::PromptTemplate.new(
|
|
1391
|
+
prefix: "Classification examples:",
|
|
1392
|
+
suffix: "Classify: {input}",
|
|
1393
|
+
examples: [{ input: "...", output: "..." }],
|
|
1394
|
+
example_template: "Input: {input}\nOutput: {output}",
|
|
1395
|
+
input_variables: [:input]
|
|
1396
|
+
)
|
|
1397
|
+
|
|
1398
|
+
# Methods
|
|
1399
|
+
template.format(**variables) # Format template with variables
|
|
1400
|
+
template.to_messages(**variables) # Convert to OpenRouter message format
|
|
1401
|
+
template.input_variables # Get required input variables
|
|
1402
|
+
template.partial(**variables) # Create partial template with some variables filled
|
|
1403
|
+
```
|
|
1404
|
+
|
|
1405
|
+
#### OpenRouter::ModelSelector
|
|
1406
|
+
|
|
1407
|
+
Intelligent model selection with fluent DSL.
|
|
1408
|
+
|
|
1409
|
+
```ruby
|
|
1410
|
+
selector = OpenRouter::ModelSelector.new
|
|
1411
|
+
|
|
1412
|
+
# Requirement methods
|
|
1413
|
+
selector.require(*capabilities) # Require specific capabilities
|
|
1414
|
+
selector.within_budget(max_cost: 0.01) # Set maximum cost constraint
|
|
1415
|
+
selector.min_context(tokens) # Minimum context length
|
|
1416
|
+
selector.prefer_providers(*providers) # Prefer specific providers
|
|
1417
|
+
selector.avoid_providers(*providers) # Avoid specific providers
|
|
1418
|
+
selector.optimize_for(strategy) # Optimization strategy (:cost, :performance, :balanced)
|
|
1419
|
+
|
|
1420
|
+
# Selection methods
|
|
1421
|
+
selector.choose # Choose best single model
|
|
1422
|
+
selector.choose_with_fallbacks(limit: 3) # Choose multiple models for fallback
|
|
1423
|
+
selector.candidates # Get all matching models
|
|
1424
|
+
selector.explain_choice # Get explanation of selection
|
|
1425
|
+
|
|
1426
|
+
# Available capabilities: :chat, :function_calling, :structured_outputs, :vision, :code_generation
|
|
1427
|
+
# Available strategies: :cost, :performance, :balanced
|
|
1428
|
+
```
|
|
1429
|
+
|
|
1430
|
+
#### OpenRouter::ModelRegistry
|
|
1431
|
+
|
|
1432
|
+
Model information and capability detection.
|
|
1433
|
+
|
|
1434
|
+
```ruby
|
|
1435
|
+
# Class methods
|
|
1436
|
+
OpenRouter::ModelRegistry.all_models # Get all cached models
|
|
1437
|
+
OpenRouter::ModelRegistry.get_model_info(model) # Get specific model info
|
|
1438
|
+
OpenRouter::ModelRegistry.models_meeting_requirements(...) # Find models matching criteria
|
|
1439
|
+
OpenRouter::ModelRegistry.calculate_estimated_cost(model, tokens) # Estimate cost
|
|
1440
|
+
OpenRouter::ModelRegistry.refresh_cache! # Refresh model cache
|
|
1441
|
+
OpenRouter::ModelRegistry.cache_status # Get cache status
|
|
1442
|
+
```
|
|
1443
|
+
|
|
1444
|
+
#### OpenRouter::UsageTracker
|
|
1445
|
+
|
|
1446
|
+
Track token usage, costs, and performance metrics.
|
|
1447
|
+
|
|
1448
|
+
```ruby
|
|
1449
|
+
tracker = client.usage_tracker
|
|
1450
|
+
|
|
1451
|
+
# Metrics
|
|
1452
|
+
tracker.total_tokens # Total tokens used
|
|
1453
|
+
tracker.total_cost # Total estimated cost
|
|
1454
|
+
tracker.request_count # Number of requests made
|
|
1455
|
+
tracker.model_usage # Per-model usage breakdown
|
|
1456
|
+
tracker.session_duration # Time since tracking started
|
|
1457
|
+
|
|
1458
|
+
# Analysis methods
|
|
1459
|
+
tracker.cache_hit_rate # Cache hit rate percentage
|
|
1460
|
+
tracker.tokens_per_second # Tokens processed per second
|
|
1461
|
+
tracker.print_summary # Print detailed usage report
|
|
1462
|
+
tracker.export_csv # Export usage data as CSV
|
|
1463
|
+
tracker.summary # Get usage summary hash
|
|
1464
|
+
tracker.reset! # Reset all counters
|
|
1465
|
+
```
|
|
1466
|
+
|
|
1467
|
+
### Response Objects
|
|
1468
|
+
|
|
1469
|
+
#### OpenRouter::Response
|
|
1470
|
+
|
|
1471
|
+
Enhanced response wrapper with metadata and feature support.
|
|
1472
|
+
|
|
1473
|
+
```ruby
|
|
1474
|
+
response = client.complete(messages)
|
|
1475
|
+
|
|
1476
|
+
# Content access
|
|
1477
|
+
response.content # Response content
|
|
1478
|
+
response.structured_output # Parsed JSON for structured outputs
|
|
1479
|
+
|
|
1480
|
+
# Tool calling
|
|
1481
|
+
response.has_tool_calls? # Check if response has tool calls
|
|
1482
|
+
response.tool_calls # Array of ToolCall objects
|
|
1483
|
+
|
|
1484
|
+
# Token metrics
|
|
1485
|
+
response.prompt_tokens # Input tokens
|
|
1486
|
+
response.completion_tokens # Output tokens
|
|
1487
|
+
response.cached_tokens # Cached tokens
|
|
1488
|
+
response.total_tokens # Total tokens
|
|
1489
|
+
|
|
1490
|
+
# Cost information
|
|
1491
|
+
response.input_cost # Input cost
|
|
1492
|
+
response.output_cost # Output cost
|
|
1493
|
+
response.cost_estimate # Total estimated cost
|
|
1494
|
+
|
|
1495
|
+
# Performance metrics
|
|
1496
|
+
response.response_time # Response time in milliseconds
|
|
1497
|
+
response.tokens_per_second # Processing speed
|
|
1498
|
+
|
|
1499
|
+
# Model information
|
|
1500
|
+
response.model # Model used
|
|
1501
|
+
response.provider # Provider name
|
|
1502
|
+
response.system_fingerprint # System fingerprint
|
|
1503
|
+
response.finish_reason # Why generation stopped
|
|
1504
|
+
|
|
1505
|
+
# Cache information
|
|
1506
|
+
response.cache_hit? # Whether response used cache
|
|
1507
|
+
response.cache_efficiency # Cache efficiency percentage
|
|
1508
|
+
|
|
1509
|
+
# Backward compatibility - delegates hash methods to raw response
|
|
1510
|
+
response["key"] # Hash-style access
|
|
1511
|
+
response.dig("path", "to", "value") # Deep hash access
|
|
1512
|
+
```
|
|
1513
|
+
|
|
1514
|
+
#### OpenRouter::ToolCall
|
|
1515
|
+
|
|
1516
|
+
Individual tool call handling and execution.
|
|
1517
|
+
|
|
1518
|
+
```ruby
|
|
1519
|
+
tool_call = response.tool_calls.first
|
|
1520
|
+
|
|
1521
|
+
# Properties
|
|
1522
|
+
tool_call.id # Tool call ID
|
|
1523
|
+
tool_call.name # Tool name
|
|
1524
|
+
tool_call.arguments # Tool arguments (Hash)
|
|
1525
|
+
|
|
1526
|
+
# Methods
|
|
1527
|
+
tool_call.validate_arguments! # Validate arguments against tool schema
|
|
1528
|
+
tool_call.to_message # Convert to continuation message format
|
|
1529
|
+
tool_call.execute(&block) # Execute tool with block
|
|
1530
|
+
```
|
|
1531
|
+
|
|
1532
|
+
### Error Classes
|
|
1533
|
+
|
|
1534
|
+
```ruby
|
|
1535
|
+
OpenRouter::Error # Base error class
|
|
1536
|
+
OpenRouter::ConfigurationError # Configuration issues
|
|
1537
|
+
OpenRouter::CapabilityError # Capability validation errors
|
|
1538
|
+
OpenRouter::ServerError # API server errors
|
|
1539
|
+
OpenRouter::ToolCallError # Tool execution errors
|
|
1540
|
+
OpenRouter::SchemaValidationError # Schema validation errors
|
|
1541
|
+
OpenRouter::StructuredOutputError # JSON parsing/healing errors
|
|
1542
|
+
OpenRouter::ModelRegistryError # Model registry errors
|
|
1543
|
+
OpenRouter::ModelSelectionError # Model selection errors
|
|
1544
|
+
```
|
|
1545
|
+
|
|
1546
|
+
### Configuration Options
|
|
1547
|
+
|
|
1548
|
+
```ruby
|
|
1549
|
+
OpenRouter.configure do |config|
|
|
1550
|
+
# Authentication
|
|
1551
|
+
config.access_token = "sk-..."
|
|
1552
|
+
config.site_name = "Your App Name"
|
|
1553
|
+
config.site_url = "https://yourapp.com"
|
|
1554
|
+
|
|
1555
|
+
# Request settings
|
|
1556
|
+
config.request_timeout = 120
|
|
1557
|
+
config.api_version = "v1"
|
|
1558
|
+
config.uri_base = "https://openrouter.ai/api"
|
|
1559
|
+
config.extra_headers = {}
|
|
1560
|
+
|
|
1561
|
+
# Response healing
|
|
1562
|
+
config.auto_heal_responses = true
|
|
1563
|
+
config.healer_model = "openai/gpt-4o-mini"
|
|
1564
|
+
config.max_heal_attempts = 2
|
|
1565
|
+
|
|
1566
|
+
# Capability validation
|
|
1567
|
+
config.strict_mode = true
|
|
1568
|
+
config.auto_force_on_unsupported_models = true
|
|
1569
|
+
|
|
1570
|
+
# Structured outputs
|
|
1571
|
+
config.default_structured_output_mode = :strict
|
|
1572
|
+
|
|
1573
|
+
# Caching
|
|
1574
|
+
config.cache_ttl = 7 * 24 * 60 * 60 # 7 days
|
|
1575
|
+
|
|
1576
|
+
# Model registry
|
|
1577
|
+
config.model_registry_timeout = 30
|
|
1578
|
+
config.model_registry_retries = 3
|
|
1579
|
+
|
|
1580
|
+
# Logging
|
|
1581
|
+
config.log_errors = false
|
|
1582
|
+
config.faraday do |f|
|
|
1583
|
+
f.response :logger
|
|
1584
|
+
end
|
|
1585
|
+
end
|
|
1586
|
+
```
|
|
1587
|
+
|
|
1588
|
+
## Contributing
|
|
1589
|
+
|
|
1590
|
+
Bug reports and pull requests are welcome on GitHub at <https://github.com/estiens/open_router_enhanced>.
|
|
1591
|
+
|
|
1592
|
+
This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](https://www.contributor-covenant.org/) code of conduct.
|
|
1593
|
+
|
|
1594
|
+
For detailed contribution guidelines, see [CONTRIBUTING.md](.github/CONTRIBUTING.md).
|
|
1595
|
+
|
|
1596
|
+
### Branch Strategy
|
|
1597
|
+
|
|
1598
|
+
We use a two-branch workflow:
|
|
1599
|
+
|
|
1600
|
+
- **`main`** - Stable releases only. Protected branch.
|
|
1601
|
+
- **`dev`** - Active development. All PRs should target this branch.
|
|
1602
|
+
|
|
1603
|
+
**⚠️ Important:** Always target your PRs to the `dev` branch, not `main`. The `main` branch is reserved for stable releases.
|
|
1604
|
+
|
|
1605
|
+
### Development Setup
|
|
1606
|
+
|
|
1607
|
+
```bash
|
|
1608
|
+
git clone https://github.com/estiens/open_router_enhanced.git
|
|
1609
|
+
cd open_router_enhanced
|
|
1610
|
+
bundle install
|
|
1611
|
+
bundle exec rspec
|
|
1612
|
+
```
|
|
1613
|
+
|
|
1614
|
+
### Running Examples
|
|
1615
|
+
|
|
1616
|
+
```bash
|
|
1617
|
+
# Set your API key
|
|
1618
|
+
export OPENROUTER_API_KEY="your_key_here"
|
|
1619
|
+
|
|
1620
|
+
# Run examples
|
|
1621
|
+
ruby -I lib examples/tool_calling_example.rb
|
|
1622
|
+
ruby -I lib examples/structured_outputs_example.rb
|
|
1623
|
+
ruby -I lib examples/model_selection_example.rb
|
|
1624
|
+
```
|
|
1625
|
+
|
|
1626
|
+
## Acknowledgments
|
|
1627
|
+
|
|
1628
|
+
This enhanced fork builds upon the excellent foundation laid by [Obie Fernandez](https://github.com/obie) and the original OpenRouter Ruby gem. The original library was bootstrapped from the [Anthropic gem](https://github.com/alexrudall/anthropic) by [Alex Rudall](https://github.com/alexrudall) and extracted from the codebase of [Olympia](https://olympia.chat), Obie's AI startup.
|
|
1629
|
+
|
|
1630
|
+
We extend our heartfelt gratitude to:
|
|
1631
|
+
|
|
1632
|
+
- **Obie Fernandez** - Original OpenRouter gem author and visionary
|
|
1633
|
+
- **Alex Rudall** - Creator of the Anthropic gem that served as the foundation
|
|
1634
|
+
- **The OpenRouter Team** - For creating an amazing unified AI API
|
|
1635
|
+
- **The Ruby Community** - For continuous support and contributions
|
|
1636
|
+
|
|
1637
|
+
## Maintainer & Consulting
|
|
1638
|
+
|
|
1639
|
+
This enhanced fork is maintained by:
|
|
1640
|
+
|
|
1641
|
+
**Eric Stiens**
|
|
1642
|
+
- Email: hello@ericstiens.dev
|
|
1643
|
+
- Website: [ericstiens.dev](http://ericstiens.dev)
|
|
1644
|
+
- GitHub: [@estiens](https://github.com/estiens)
|
|
1645
|
+
- Blog: [Low Level Magic](https://lowlevelmagic.io)
|
|
1646
|
+
|
|
1647
|
+
### Need Help with AI Integration?
|
|
1648
|
+
|
|
1649
|
+
I'm available for consulting on Ruby AI applications, LLM integration, and building production-ready AI systems. My work extends beyond Ruby to include real-time AI orchestration, character-based AI systems, multi-agent architectures, and low-latency voice/streaming applications. Whether you need help with tool calling workflows, cost optimization, building AI characters with persistent memory, or orchestrating complex multi-model systems, I'd be happy to help.
|
|
1650
|
+
|
|
1651
|
+
**Get in touch:**
|
|
1652
|
+
- Email: hello@lowlevelmagic.io
|
|
1653
|
+
- Visit: [lowlevelmagic.io](https://lowlevelmagic.io)
|
|
1654
|
+
- Read more: [Why I Built OpenRouter Enhanced](https://lowlevelmagic.io/writings/why-i-built-open-router-enhanced/)
|
|
1655
|
+
|
|
1656
|
+
## License
|
|
1657
|
+
|
|
1658
|
+
The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
|
|
1659
|
+
|
|
1660
|
+
MIT License is chosen for maximum permissiveness and compatibility, allowing unrestricted use, modification, and distribution while maintaining attribution requirements.
|