open_router_enhanced 1.0.0 → 1.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +36 -0
- data/Gemfile.lock +45 -47
- data/README.md +151 -1228
- data/docs/observability.md +3 -0
- data/docs/plugins.md +183 -0
- data/docs/responses_api.md +298 -0
- data/docs/streaming.md +18 -3
- data/docs/structured_outputs.md +466 -146
- data/lib/open_router/client.rb +122 -5
- data/lib/open_router/responses_response.rb +192 -0
- data/lib/open_router/responses_tool_call.rb +95 -0
- data/lib/open_router/tool_call.rb +13 -59
- data/lib/open_router/tool_call_base.rb +69 -0
- data/lib/open_router/version.rb +1 -1
- data/lib/open_router.rb +9 -0
- metadata +7 -2
data/README.md
CHANGED
|
@@ -2,26 +2,26 @@
|
|
|
2
2
|
|
|
3
3
|
The future will bring us hundreds of language models and dozens of providers for each. How will you choose the best?
|
|
4
4
|
|
|
5
|
-
The [OpenRouter API](https://openrouter.ai/docs) is a single unified interface for all LLMs
|
|
5
|
+
The [OpenRouter API](https://openrouter.ai/docs) is a single unified interface for all LLMs, accessible through an idiomatic Ruby interface.
|
|
6
6
|
|
|
7
7
|
**OpenRouter Enhanced** is an advanced fork of the [original OpenRouter Ruby gem](https://github.com/OlympiaAI/open_router) by [Obie Fernandez](https://github.com/obie) that adds comprehensive AI application development features including tool calling, structured outputs, intelligent model selection, prompt templates, observability, and automatic response healing—all while maintaining full backward compatibility.
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
**[Read the story behind OpenRouter Enhanced](https://lowlevelmagic.io/writings/why-i-built-open-router-enhanced/)** - Learn why this gem was built and the philosophy behind its design.
|
|
10
10
|
|
|
11
11
|
## Enhanced Features
|
|
12
12
|
|
|
13
|
-
This fork extends the original OpenRouter gem with enterprise-grade AI development capabilities:
|
|
14
|
-
|
|
15
13
|
### Core AI Features
|
|
16
|
-
- **Tool Calling**: Full support for OpenRouter's function calling API with Ruby-idiomatic DSL
|
|
17
|
-
- **Structured Outputs**: JSON Schema validation with automatic healing for non-native models
|
|
18
|
-
- **Smart Model Selection**: Intelligent model selection with
|
|
19
|
-
- **Prompt Templates**: Reusable prompt templates with variable interpolation and few-shot learning
|
|
14
|
+
- **[Tool Calling](docs/tools.md)**: Full support for OpenRouter's function calling API with Ruby-idiomatic DSL
|
|
15
|
+
- **[Structured Outputs](docs/structured_outputs.md)**: JSON Schema validation with automatic healing for non-native models
|
|
16
|
+
- **[Smart Model Selection](docs/model_selection.md)**: Intelligent model selection with cost optimization and capability matching
|
|
17
|
+
- **[Prompt Templates](docs/prompt_templates.md)**: Reusable prompt templates with variable interpolation and few-shot learning
|
|
18
|
+
- **[Plugins](docs/plugins.md)**: OpenRouter plugins for web search, PDF inputs, and native response healing
|
|
20
19
|
|
|
21
20
|
### Performance & Reliability
|
|
22
21
|
- **Model Registry**: Local caching and querying of OpenRouter model data with capability detection
|
|
23
22
|
- **Enhanced Response Handling**: Rich Response objects with automatic parsing for tool calls and structured outputs
|
|
24
|
-
- **
|
|
23
|
+
- **Native Response Healing**: Automatic server-side JSON repair via OpenRouter's response-healing plugin
|
|
24
|
+
- **Client-Side Healing**: Self-healing responses for schema validation failures using LLM-based repair
|
|
25
25
|
- **Model Fallbacks**: Automatic failover between models with graceful degradation
|
|
26
26
|
- **Streaming Support**: Enhanced streaming client with callback system and response reconstruction
|
|
27
27
|
|
|
@@ -31,50 +31,26 @@ This fork extends the original OpenRouter gem with enterprise-grade AI developme
|
|
|
31
31
|
- **Callback System**: Extensible event system for monitoring requests, responses, and errors
|
|
32
32
|
- **Cost Management**: Built-in cost estimation and budget constraints
|
|
33
33
|
|
|
34
|
-
### Development & Testing
|
|
35
|
-
- **Comprehensive Testing**: VCR-based integration tests with real API interactions
|
|
36
|
-
- **Debug Support**: Detailed error reporting and validation feedback
|
|
37
|
-
- **Configuration Options**: Extensive configuration for healing, validation, and performance tuning
|
|
38
|
-
- **Backward Compatible**: All existing code continues to work unchanged
|
|
39
|
-
|
|
40
34
|
### Core OpenRouter Benefits
|
|
41
|
-
|
|
42
|
-
- **
|
|
43
|
-
- **
|
|
44
|
-
- **Easy integration**: This Ruby gem provides a simple and intuitive interface to interact with the OpenRouter API, making it effortless to integrate AI capabilities into your Ruby applications.
|
|
35
|
+
- **Prioritize price or performance**: OpenRouter scouts for the lowest prices and best latencies/throughputs across dozens of providers
|
|
36
|
+
- **Standardized API**: No need to change your code when switching between models or providers
|
|
37
|
+
- **Easy integration**: Simple and intuitive Ruby interface for AI capabilities
|
|
45
38
|
|
|
46
39
|
## Table of Contents
|
|
47
40
|
|
|
48
41
|
- [Installation](#installation)
|
|
49
42
|
- [Quick Start](#quick-start)
|
|
50
43
|
- [Configuration](#configuration)
|
|
51
|
-
- [
|
|
52
|
-
- [Basic Completions](#basic-completions)
|
|
53
|
-
- [Model Selection](#model-selection)
|
|
54
|
-
- [Enhanced AI Features](#enhanced-ai-features)
|
|
44
|
+
- [Features](#features)
|
|
55
45
|
- [Tool Calling](#tool-calling)
|
|
56
46
|
- [Structured Outputs](#structured-outputs)
|
|
57
47
|
- [Smart Model Selection](#smart-model-selection)
|
|
58
48
|
- [Prompt Templates](#prompt-templates)
|
|
59
|
-
- [Streaming
|
|
60
|
-
- [Streaming Client](#streaming-client)
|
|
61
|
-
- [Streaming Callbacks](#streaming-callbacks)
|
|
62
|
-
- [Observability & Analytics](#observability--analytics)
|
|
49
|
+
- [Streaming](#streaming)
|
|
63
50
|
- [Usage Tracking](#usage-tracking)
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
- [Cost Management](#cost-management)
|
|
67
|
-
- [Advanced Features](#advanced-features)
|
|
68
|
-
- [Model Registry](#model-registry)
|
|
69
|
-
- [Model Fallbacks](#model-fallbacks)
|
|
70
|
-
- [Response Healing](#response-healing)
|
|
71
|
-
- [Performance Optimization](#performance-optimization)
|
|
72
|
-
- [Testing & Development](#testing--development)
|
|
73
|
-
- [Running Tests](#running-tests)
|
|
74
|
-
- [VCR Testing](#vcr-testing)
|
|
75
|
-
- [Examples](#examples)
|
|
51
|
+
- [Model Exploration](#model-exploration)
|
|
52
|
+
- [Testing](#testing)
|
|
76
53
|
- [Troubleshooting](#troubleshooting)
|
|
77
|
-
- [API Reference](#api-reference)
|
|
78
54
|
- [Contributing](#contributing)
|
|
79
55
|
- [License](#license)
|
|
80
56
|
|
|
@@ -102,19 +78,13 @@ Or install it directly:
|
|
|
102
78
|
gem install open_router_enhanced
|
|
103
79
|
```
|
|
104
80
|
|
|
105
|
-
And require it in your code:
|
|
106
|
-
|
|
107
|
-
```ruby
|
|
108
|
-
require "open_router"
|
|
109
|
-
```
|
|
110
|
-
|
|
111
81
|
## Quick Start
|
|
112
82
|
|
|
113
83
|
### 1. Get Your API Key
|
|
114
84
|
- Sign up at [OpenRouter](https://openrouter.ai)
|
|
115
85
|
- Get your API key from [https://openrouter.ai/keys](https://openrouter.ai/keys)
|
|
116
86
|
|
|
117
|
-
### 2. Basic Setup
|
|
87
|
+
### 2. Basic Setup
|
|
118
88
|
|
|
119
89
|
```ruby
|
|
120
90
|
require "open_router"
|
|
@@ -138,16 +108,16 @@ puts response.content
|
|
|
138
108
|
# => "The capital of France is Paris."
|
|
139
109
|
```
|
|
140
110
|
|
|
141
|
-
### 3. Enhanced Features
|
|
111
|
+
### 3. Enhanced Features Example
|
|
142
112
|
|
|
143
113
|
```ruby
|
|
144
|
-
# Smart model selection
|
|
114
|
+
# Smart model selection with capabilities
|
|
145
115
|
model = OpenRouter::ModelSelector.new
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
116
|
+
.require(:function_calling)
|
|
117
|
+
.optimize_for(:cost)
|
|
118
|
+
.choose
|
|
149
119
|
|
|
150
|
-
#
|
|
120
|
+
# Define a tool
|
|
151
121
|
weather_tool = OpenRouter::Tool.define do
|
|
152
122
|
name "get_weather"
|
|
153
123
|
description "Get current weather"
|
|
@@ -156,12 +126,14 @@ weather_tool = OpenRouter::Tool.define do
|
|
|
156
126
|
end
|
|
157
127
|
end
|
|
158
128
|
|
|
129
|
+
# Define a schema for structured output
|
|
159
130
|
weather_schema = OpenRouter::Schema.define("weather") do
|
|
160
131
|
string :location, required: true
|
|
161
132
|
number :temperature, required: true
|
|
162
133
|
string :conditions, required: true
|
|
163
134
|
end
|
|
164
135
|
|
|
136
|
+
# Use everything together
|
|
165
137
|
response = client.complete(
|
|
166
138
|
[{ role: "user", content: "What's the weather in Tokyo?" }],
|
|
167
139
|
model: model,
|
|
@@ -178,149 +150,66 @@ end
|
|
|
178
150
|
|
|
179
151
|
## Configuration
|
|
180
152
|
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
Configure the gem globally, for example in an `open_router.rb` initializer file. Never hardcode secrets into your codebase - instead use `Rails.application.credentials` or something like [dotenv](https://github.com/motdotla/dotenv) to pass the keys safely into your environments.
|
|
153
|
+
Configure the gem globally (e.g., in an initializer):
|
|
184
154
|
|
|
185
155
|
```ruby
|
|
186
156
|
OpenRouter.configure do |config|
|
|
157
|
+
# Required
|
|
187
158
|
config.access_token = ENV["OPENROUTER_API_KEY"]
|
|
188
159
|
config.site_name = "Your App Name"
|
|
189
160
|
config.site_url = "https://yourapp.com"
|
|
190
|
-
|
|
191
|
-
# Optional:
|
|
161
|
+
|
|
162
|
+
# Optional: Response healing for non-native structured output models
|
|
192
163
|
config.auto_heal_responses = true
|
|
193
164
|
config.healer_model = "openai/gpt-4o-mini"
|
|
194
165
|
config.max_heal_attempts = 2
|
|
195
|
-
|
|
196
|
-
# Optional:
|
|
166
|
+
|
|
167
|
+
# Optional: Strict mode for capability validation
|
|
197
168
|
config.strict_mode = true
|
|
198
|
-
|
|
199
|
-
# Optional:
|
|
200
|
-
config.
|
|
169
|
+
|
|
170
|
+
# Optional: Request timeout (default: 120 seconds)
|
|
171
|
+
config.request_timeout = 120
|
|
201
172
|
end
|
|
202
173
|
```
|
|
203
174
|
|
|
204
175
|
### Per-Client Configuration
|
|
205
176
|
|
|
206
|
-
You can also configure clients individually:
|
|
207
|
-
|
|
208
177
|
```ruby
|
|
209
178
|
client = OpenRouter::Client.new(
|
|
210
179
|
access_token: ENV["OPENROUTER_API_KEY"],
|
|
211
|
-
request_timeout:
|
|
180
|
+
request_timeout: 240
|
|
212
181
|
)
|
|
213
182
|
```
|
|
214
183
|
|
|
215
184
|
### Faraday Configuration
|
|
216
185
|
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
This example adds `faraday-retry` and a logger that redacts the api key so it doesn't get leaked to logs.
|
|
186
|
+
Configure Faraday middleware for retries, logging, etc:
|
|
220
187
|
|
|
221
188
|
```ruby
|
|
222
189
|
require 'faraday/retry'
|
|
223
190
|
|
|
224
|
-
retry_options = {
|
|
225
|
-
max: 2,
|
|
226
|
-
interval: 0.05,
|
|
227
|
-
interval_randomness: 0.5,
|
|
228
|
-
backoff_factor: 2
|
|
229
|
-
}
|
|
230
|
-
|
|
231
191
|
OpenRouter::Client.new(access_token: ENV["ACCESS_TOKEN"]) do |config|
|
|
232
192
|
config.faraday do |f|
|
|
233
|
-
f.request :retry,
|
|
234
|
-
f.response :logger, ::Logger.new($stdout), { headers: true, bodies: true
|
|
235
|
-
logger.filter(/(Bearer) (\S+)/, '\1[REDACTED]')
|
|
236
|
-
end
|
|
193
|
+
f.request :retry, max: 2, interval: 0.05
|
|
194
|
+
f.response :logger, ::Logger.new($stdout), { headers: true, bodies: true }
|
|
237
195
|
end
|
|
238
196
|
end
|
|
239
197
|
```
|
|
240
198
|
|
|
241
|
-
|
|
242
|
-
|
|
243
|
-
The default timeout for any request using this library is 120 seconds. You can change that by passing a number of seconds to the `request_timeout` when initializing the client.
|
|
244
|
-
|
|
245
|
-
```ruby
|
|
246
|
-
client = OpenRouter::Client.new(
|
|
247
|
-
access_token: "access_token_goes_here",
|
|
248
|
-
request_timeout: 240 # Optional
|
|
249
|
-
)
|
|
250
|
-
```
|
|
251
|
-
|
|
252
|
-
## Core Features
|
|
253
|
-
|
|
254
|
-
### Basic Completions
|
|
255
|
-
|
|
256
|
-
Hit the OpenRouter API for a completion:
|
|
257
|
-
|
|
258
|
-
```ruby
|
|
259
|
-
messages = [
|
|
260
|
-
{ role: "system", content: "You are a helpful assistant." },
|
|
261
|
-
{ role: "user", content: "What is the color of the sky?" }
|
|
262
|
-
]
|
|
263
|
-
|
|
264
|
-
response = client.complete(messages)
|
|
265
|
-
puts response.content
|
|
266
|
-
# => "The sky is typically blue during the day due to a phenomenon called Rayleigh scattering. Sunlight..."
|
|
267
|
-
```
|
|
268
|
-
|
|
269
|
-
### Model Selection
|
|
270
|
-
|
|
271
|
-
Pass an array to the `model` parameter to enable [explicit model routing](https://openrouter.ai/docs#model-routing).
|
|
272
|
-
|
|
273
|
-
```ruby
|
|
274
|
-
OpenRouter::Client.new.complete(
|
|
275
|
-
[
|
|
276
|
-
{ role: "system", content: SYSTEM_PROMPT },
|
|
277
|
-
{ role: "user", content: "Provide analysis of the data formatted as JSON:" }
|
|
278
|
-
],
|
|
279
|
-
model: [
|
|
280
|
-
"mistralai/mixtral-8x7b-instruct:nitro",
|
|
281
|
-
"mistralai/mixtral-8x7b-instruct"
|
|
282
|
-
],
|
|
283
|
-
extras: {
|
|
284
|
-
response_format: {
|
|
285
|
-
type: "json_object"
|
|
286
|
-
}
|
|
287
|
-
}
|
|
288
|
-
)
|
|
289
|
-
```
|
|
199
|
+
**[Full configuration documentation](docs/configuration.md)**
|
|
290
200
|
|
|
291
|
-
|
|
292
|
-
|
|
293
|
-
```ruby
|
|
294
|
-
models = client.models
|
|
295
|
-
puts models
|
|
296
|
-
# => [{"id"=>"openrouter/auto", "object"=>"model", "created"=>1684195200, "owned_by"=>"openrouter", "permission"=>[], "root"=>"openrouter", "parent"=>nil}, ...]
|
|
297
|
-
```
|
|
298
|
-
|
|
299
|
-
### Generation Stats
|
|
300
|
-
|
|
301
|
-
Query the generation stats for a given generation ID:
|
|
302
|
-
|
|
303
|
-
```ruby
|
|
304
|
-
generation_id = "generation-abcdefg"
|
|
305
|
-
stats = client.query_generation_stats(generation_id)
|
|
306
|
-
puts stats
|
|
307
|
-
# => {"id"=>"generation-abcdefg", "object"=>"generation", "created"=>1684195200, "model"=>"openrouter/auto", "usage"=>{"prompt_tokens"=>10, "completion_tokens"=>50, "total_tokens"=>60}, "cost"=>0.0006}
|
|
308
|
-
```
|
|
309
|
-
|
|
310
|
-
## Enhanced AI Features
|
|
201
|
+
## Features
|
|
311
202
|
|
|
312
203
|
### Tool Calling
|
|
313
204
|
|
|
314
|
-
Enable AI models to call functions and interact with external APIs
|
|
315
|
-
|
|
316
|
-
#### Quick Example
|
|
205
|
+
Enable AI models to call functions and interact with external APIs.
|
|
317
206
|
|
|
318
207
|
```ruby
|
|
319
|
-
# Define a tool
|
|
208
|
+
# Define a tool with the DSL
|
|
320
209
|
weather_tool = OpenRouter::Tool.define do
|
|
321
210
|
name "get_weather"
|
|
322
211
|
description "Get current weather for a location"
|
|
323
|
-
|
|
212
|
+
|
|
324
213
|
parameters do
|
|
325
214
|
string :location, required: true, description: "City name"
|
|
326
215
|
string :units, enum: ["celsius", "fahrenheit"], default: "celsius"
|
|
@@ -331,41 +220,30 @@ end
|
|
|
331
220
|
response = client.complete(
|
|
332
221
|
[{ role: "user", content: "What's the weather in London?" }],
|
|
333
222
|
model: "anthropic/claude-3.5-sonnet",
|
|
334
|
-
tools: [weather_tool]
|
|
335
|
-
tool_choice: "auto"
|
|
223
|
+
tools: [weather_tool]
|
|
336
224
|
)
|
|
337
225
|
|
|
338
226
|
# Handle tool calls
|
|
339
227
|
if response.has_tool_calls?
|
|
340
228
|
response.tool_calls.each do |tool_call|
|
|
341
|
-
result = fetch_weather(tool_call.arguments["location"]
|
|
342
|
-
puts "Weather
|
|
229
|
+
result = fetch_weather(tool_call.arguments["location"])
|
|
230
|
+
puts "Weather: #{result}"
|
|
343
231
|
end
|
|
344
232
|
end
|
|
345
233
|
```
|
|
346
234
|
|
|
347
|
-
|
|
348
|
-
|
|
349
|
-
- **Ruby DSL**: Define tools with intuitive Ruby syntax
|
|
350
|
-
- **Parameter Validation**: Automatic validation against JSON Schema
|
|
351
|
-
- **Tool Choice Control**: Auto, required, none, or specific tool selection
|
|
352
|
-
- **Conversation Continuation**: Easy message building for multi-turn conversations
|
|
353
|
-
- **Error Handling**: Graceful error handling and validation
|
|
354
|
-
|
|
355
|
-
📖 **[Complete Tool Calling Documentation](docs/tools.md)**
|
|
235
|
+
**[Complete Tool Calling Documentation](docs/tools.md)**
|
|
356
236
|
|
|
357
237
|
### Structured Outputs
|
|
358
238
|
|
|
359
|
-
Get JSON responses that conform to specific schemas with automatic validation and healing
|
|
360
|
-
|
|
361
|
-
#### Quick Example
|
|
239
|
+
Get JSON responses that conform to specific schemas with automatic validation and healing.
|
|
362
240
|
|
|
363
241
|
```ruby
|
|
364
|
-
# Define a schema
|
|
242
|
+
# Define a schema
|
|
365
243
|
user_schema = OpenRouter::Schema.define("user") do
|
|
366
244
|
string :name, required: true, description: "Full name"
|
|
367
245
|
integer :age, required: true, minimum: 0, maximum: 150
|
|
368
|
-
string :email, required: true
|
|
246
|
+
string :email, required: true
|
|
369
247
|
boolean :premium, description: "Premium account status"
|
|
370
248
|
end
|
|
371
249
|
|
|
@@ -376,78 +254,63 @@ response = client.complete(
|
|
|
376
254
|
response_format: user_schema
|
|
377
255
|
)
|
|
378
256
|
|
|
379
|
-
# Access parsed JSON
|
|
257
|
+
# Access parsed JSON
|
|
380
258
|
user = response.structured_output
|
|
381
|
-
puts user[
|
|
382
|
-
puts user["age"] # => 30
|
|
383
|
-
puts user["email"] # => "john@example.com"
|
|
259
|
+
puts "#{user['name']} (#{user['age']}) - #{user['email']}"
|
|
384
260
|
```
|
|
385
261
|
|
|
386
|
-
|
|
387
|
-
|
|
388
|
-
-
|
|
389
|
-
-
|
|
390
|
-
-
|
|
391
|
-
- **Complex Schemas**: Support for nested objects, arrays, and advanced constraints
|
|
392
|
-
- **Fallback Support**: Graceful degradation for unsupported models
|
|
262
|
+
**Key Features:**
|
|
263
|
+
- Ruby DSL for JSON schemas
|
|
264
|
+
- Automatic healing for models without native support
|
|
265
|
+
- Validation with detailed error reporting
|
|
266
|
+
- Support for nested objects and arrays
|
|
393
267
|
|
|
394
|
-
|
|
268
|
+
**[Complete Structured Outputs Documentation](docs/structured_outputs.md)**
|
|
395
269
|
|
|
396
270
|
### Smart Model Selection
|
|
397
271
|
|
|
398
|
-
Automatically choose the best
|
|
399
|
-
|
|
400
|
-
#### Quick Example
|
|
272
|
+
Automatically choose the best model based on requirements.
|
|
401
273
|
|
|
402
274
|
```ruby
|
|
403
|
-
# Find
|
|
275
|
+
# Find cheapest model with function calling
|
|
404
276
|
model = OpenRouter::ModelSelector.new
|
|
405
|
-
|
|
406
|
-
|
|
407
|
-
|
|
277
|
+
.require(:function_calling)
|
|
278
|
+
.optimize_for(:cost)
|
|
279
|
+
.choose
|
|
408
280
|
|
|
409
281
|
# Advanced selection with multiple criteria
|
|
410
282
|
model = OpenRouter::ModelSelector.new
|
|
411
|
-
|
|
412
|
-
|
|
413
|
-
|
|
414
|
-
|
|
415
|
-
|
|
416
|
-
|
|
417
|
-
|
|
418
|
-
# Get multiple options
|
|
283
|
+
.require(:function_calling, :vision)
|
|
284
|
+
.within_budget(max_cost: 0.01)
|
|
285
|
+
.min_context(50_000)
|
|
286
|
+
.prefer_providers("anthropic", "openai")
|
|
287
|
+
.optimize_for(:performance)
|
|
288
|
+
.choose
|
|
289
|
+
|
|
290
|
+
# Get multiple options for fallbacks
|
|
419
291
|
models = OpenRouter::ModelSelector.new
|
|
420
|
-
|
|
421
|
-
|
|
292
|
+
.require(:structured_outputs)
|
|
293
|
+
.choose_with_fallbacks(limit: 3)
|
|
422
294
|
# => ["openai/gpt-4o-mini", "anthropic/claude-3-haiku", "google/gemini-flash"]
|
|
423
295
|
```
|
|
424
296
|
|
|
425
|
-
|
|
297
|
+
**Available capabilities:** `:chat`, `:function_calling`, `:structured_outputs`, `:vision`, `:code_generation`
|
|
426
298
|
|
|
427
|
-
|
|
428
|
-
- **Cost Optimization**: Find models within budget constraints
|
|
429
|
-
- **Capability Matching**: Require specific features like function calling or vision
|
|
430
|
-
- **Provider Preferences**: Prefer or avoid specific providers
|
|
431
|
-
- **Graceful Fallbacks**: Automatic fallback with requirement relaxation
|
|
432
|
-
- **Performance Tiers**: Choose between cost and performance optimization
|
|
299
|
+
**Optimization strategies:** `:cost`, `:performance`, `:latest`, `:context`
|
|
433
300
|
|
|
434
|
-
|
|
301
|
+
**[Complete Model Selection Documentation](docs/model_selection.md)**
|
|
435
302
|
|
|
436
303
|
### Prompt Templates
|
|
437
304
|
|
|
438
|
-
Create reusable, parameterized prompts with variable interpolation
|
|
439
|
-
|
|
440
|
-
#### Quick Example
|
|
305
|
+
Create reusable, parameterized prompts with variable interpolation.
|
|
441
306
|
|
|
442
307
|
```ruby
|
|
443
|
-
# Basic template
|
|
308
|
+
# Basic template
|
|
444
309
|
translation_template = OpenRouter::PromptTemplate.new(
|
|
445
310
|
template: "Translate '{text}' from {source_lang} to {target_lang}",
|
|
446
311
|
input_variables: [:text, :source_lang, :target_lang]
|
|
447
312
|
)
|
|
448
313
|
|
|
449
|
-
# Use with client
|
|
450
|
-
client = OpenRouter::Client.new
|
|
451
314
|
response = client.complete(
|
|
452
315
|
translation_template.to_messages(
|
|
453
316
|
text: "Hello world",
|
|
@@ -459,505 +322,106 @@ response = client.complete(
|
|
|
459
322
|
|
|
460
323
|
# Few-shot learning template
|
|
461
324
|
classification_template = OpenRouter::PromptTemplate.new(
|
|
462
|
-
prefix: "Classify the sentiment
|
|
325
|
+
prefix: "Classify the sentiment:",
|
|
463
326
|
suffix: "Now classify: {text}",
|
|
464
327
|
examples: [
|
|
465
|
-
{ text: "I love this
|
|
466
|
-
{ text: "This is terrible.", sentiment: "negative" }
|
|
467
|
-
{ text: "It's okay, nothing special.", sentiment: "neutral" }
|
|
328
|
+
{ text: "I love this!", sentiment: "positive" },
|
|
329
|
+
{ text: "This is terrible.", sentiment: "negative" }
|
|
468
330
|
],
|
|
469
331
|
example_template: "Text: {text}\nSentiment: {sentiment}",
|
|
470
332
|
input_variables: [:text]
|
|
471
333
|
)
|
|
472
|
-
|
|
473
|
-
# Render complete prompt
|
|
474
|
-
prompt = classification_template.format(text: "This is amazing!")
|
|
475
|
-
puts prompt
|
|
476
|
-
# =>
|
|
477
|
-
# Classify the sentiment of the following text. Examples:
|
|
478
|
-
#
|
|
479
|
-
# Text: I love this product!
|
|
480
|
-
# Sentiment: positive
|
|
481
|
-
#
|
|
482
|
-
# Text: This is terrible.
|
|
483
|
-
# Sentiment: negative
|
|
484
|
-
#
|
|
485
|
-
# Text: It's okay, nothing special.
|
|
486
|
-
# Sentiment: neutral
|
|
487
|
-
#
|
|
488
|
-
# Now classify: This is amazing!
|
|
489
334
|
```
|
|
490
335
|
|
|
491
|
-
|
|
336
|
+
**[Complete Prompt Templates Documentation](docs/prompt_templates.md)**
|
|
492
337
|
|
|
493
|
-
|
|
494
|
-
- **Few-Shot Learning**: Include examples to improve model performance
|
|
495
|
-
- **Chat Formatting**: Automatic conversion to OpenRouter message format
|
|
496
|
-
- **Partial Variables**: Pre-fill common variables for reuse
|
|
497
|
-
- **Template Composition**: Combine templates for complex prompts
|
|
498
|
-
- **Validation**: Automatic validation of required input variables
|
|
338
|
+
### Streaming
|
|
499
339
|
|
|
500
|
-
|
|
501
|
-
|
|
502
|
-
### Model Registry
|
|
503
|
-
|
|
504
|
-
Access detailed information about available models and their capabilities.
|
|
505
|
-
|
|
506
|
-
#### Quick Example
|
|
340
|
+
Stream responses in real-time with callbacks and automatic response reconstruction.
|
|
507
341
|
|
|
508
342
|
```ruby
|
|
509
|
-
# Get specific model information
|
|
510
|
-
model_info = OpenRouter::ModelRegistry.get_model_info("anthropic/claude-3-5-sonnet")
|
|
511
|
-
puts model_info[:capabilities] # [:chat, :function_calling, :structured_outputs, :vision]
|
|
512
|
-
puts model_info[:cost_per_1k_tokens] # { input: 0.003, output: 0.015 }
|
|
513
|
-
|
|
514
|
-
# Find models matching requirements
|
|
515
|
-
candidates = OpenRouter::ModelRegistry.models_meeting_requirements(
|
|
516
|
-
capabilities: [:function_calling],
|
|
517
|
-
max_input_cost: 0.01
|
|
518
|
-
)
|
|
519
|
-
|
|
520
|
-
# Estimate costs for specific usage
|
|
521
|
-
cost = OpenRouter::ModelRegistry.calculate_estimated_cost(
|
|
522
|
-
"openai/gpt-4o",
|
|
523
|
-
input_tokens: 1000,
|
|
524
|
-
output_tokens: 500
|
|
525
|
-
)
|
|
526
|
-
puts "Estimated cost: $#{cost.round(4)}" # => "Estimated cost: $0.0105"
|
|
527
|
-
```
|
|
528
|
-
|
|
529
|
-
#### Key Features
|
|
530
|
-
|
|
531
|
-
- **Model Discovery**: Browse all available models and their specifications
|
|
532
|
-
- **Capability Detection**: Check which features each model supports
|
|
533
|
-
- **Cost Calculation**: Estimate costs for specific token usage
|
|
534
|
-
- **Local Caching**: Fast model data access with automatic cache management
|
|
535
|
-
- **Real-time Updates**: Refresh model data from OpenRouter API
|
|
536
|
-
|
|
537
|
-
## Streaming & Real-time
|
|
538
|
-
|
|
539
|
-
### Streaming Client
|
|
540
|
-
|
|
541
|
-
The enhanced streaming client provides real-time response streaming with callback support and automatic response reconstruction.
|
|
542
|
-
|
|
543
|
-
#### Quick Example
|
|
544
|
-
|
|
545
|
-
```ruby
|
|
546
|
-
# Create streaming client
|
|
547
343
|
streaming_client = OpenRouter::StreamingClient.new
|
|
548
344
|
|
|
549
345
|
# Set up callbacks
|
|
550
346
|
streaming_client
|
|
551
|
-
.on_stream(:on_start) { |data| puts "Starting
|
|
347
|
+
.on_stream(:on_start) { |data| puts "Starting..." }
|
|
552
348
|
.on_stream(:on_chunk) { |chunk| print chunk.content }
|
|
553
|
-
.on_stream(:
|
|
554
|
-
.on_stream(:on_finish) { |response| puts "\nCompleted. Total tokens: #{response.total_tokens}" }
|
|
349
|
+
.on_stream(:on_finish) { |response| puts "\nDone! Tokens: #{response.total_tokens}" }
|
|
555
350
|
.on_stream(:on_error) { |error| puts "Error: #{error.message}" }
|
|
556
351
|
|
|
557
352
|
# Stream with automatic response accumulation
|
|
558
353
|
response = streaming_client.stream_complete(
|
|
559
|
-
[{ role: "user", content: "Write a
|
|
354
|
+
[{ role: "user", content: "Write a story about a robot" }],
|
|
560
355
|
model: "openai/gpt-4o-mini",
|
|
561
356
|
accumulate_response: true
|
|
562
357
|
)
|
|
563
358
|
|
|
564
|
-
# Access complete response after streaming
|
|
565
359
|
puts "Final response: #{response.content}"
|
|
566
|
-
puts "Cost: $#{response.cost_estimate}"
|
|
567
|
-
```
|
|
568
|
-
|
|
569
|
-
#### Streaming with Tool Calls
|
|
570
|
-
|
|
571
|
-
```ruby
|
|
572
|
-
# Define a tool
|
|
573
|
-
weather_tool = OpenRouter::Tool.define do
|
|
574
|
-
name "get_weather"
|
|
575
|
-
description "Get current weather"
|
|
576
|
-
parameters { string :location, required: true }
|
|
577
|
-
end
|
|
578
|
-
|
|
579
|
-
# Stream with tool calling
|
|
580
|
-
streaming_client.stream_complete(
|
|
581
|
-
[{ role: "user", content: "What's the weather in Tokyo?" }],
|
|
582
|
-
model: "anthropic/claude-3-5-sonnet",
|
|
583
|
-
tools: [weather_tool]
|
|
584
|
-
) do |chunk|
|
|
585
|
-
if chunk.has_tool_calls?
|
|
586
|
-
chunk.tool_calls.each do |tool_call|
|
|
587
|
-
puts "Calling #{tool_call.name} with #{tool_call.arguments}"
|
|
588
|
-
end
|
|
589
|
-
else
|
|
590
|
-
print chunk.content
|
|
591
|
-
end
|
|
592
|
-
end
|
|
593
360
|
```
|
|
594
361
|
|
|
595
|
-
|
|
596
|
-
|
|
597
|
-
The streaming client supports extensive callback events for monitoring and analytics.
|
|
598
|
-
|
|
599
|
-
```ruby
|
|
600
|
-
streaming_client = OpenRouter::StreamingClient.new
|
|
601
|
-
|
|
602
|
-
# Monitor token usage in real-time
|
|
603
|
-
streaming_client.on_stream(:on_chunk) do |chunk|
|
|
604
|
-
if chunk.usage
|
|
605
|
-
puts "Tokens so far: #{chunk.usage['total_tokens']}"
|
|
606
|
-
end
|
|
607
|
-
end
|
|
608
|
-
|
|
609
|
-
# Handle errors gracefully
|
|
610
|
-
streaming_client.on_stream(:on_error) do |error|
|
|
611
|
-
logger.error "Streaming failed: #{error.message}"
|
|
612
|
-
# Implement fallback logic
|
|
613
|
-
fallback_response = client.complete(messages, model: "openai/gpt-4o-mini")
|
|
614
|
-
end
|
|
615
|
-
|
|
616
|
-
# Track performance metrics
|
|
617
|
-
start_time = nil
|
|
618
|
-
streaming_client
|
|
619
|
-
.on_stream(:on_start) { |data| start_time = Time.now }
|
|
620
|
-
.on_stream(:on_finish) do |response|
|
|
621
|
-
duration = Time.now - start_time
|
|
622
|
-
puts "Request completed in #{duration.round(2)}s"
|
|
623
|
-
puts "Tokens per second: #{response.total_tokens / duration}"
|
|
624
|
-
end
|
|
625
|
-
```
|
|
626
|
-
|
|
627
|
-
## Observability & Analytics
|
|
362
|
+
**[Complete Streaming Documentation](docs/streaming.md)**
|
|
628
363
|
|
|
629
364
|
### Usage Tracking
|
|
630
365
|
|
|
631
|
-
Track token usage, costs, and performance metrics
|
|
632
|
-
|
|
633
|
-
#### Quick Example
|
|
366
|
+
Track token usage, costs, and performance metrics.
|
|
634
367
|
|
|
635
368
|
```ruby
|
|
636
|
-
#
|
|
369
|
+
# Enable tracking
|
|
637
370
|
client = OpenRouter::Client.new(track_usage: true)
|
|
638
371
|
|
|
639
|
-
# Make
|
|
640
|
-
3.times do
|
|
372
|
+
# Make requests
|
|
373
|
+
3.times do
|
|
641
374
|
response = client.complete(
|
|
642
|
-
[{ role: "user", content: "Tell me a fact
|
|
375
|
+
[{ role: "user", content: "Tell me a fact" }],
|
|
643
376
|
model: "openai/gpt-4o-mini"
|
|
644
377
|
)
|
|
645
|
-
puts "Request #{i + 1}: #{response.total_tokens} tokens, $#{response.cost_estimate}"
|
|
646
378
|
end
|
|
647
379
|
|
|
648
|
-
# View
|
|
380
|
+
# View statistics
|
|
649
381
|
tracker = client.usage_tracker
|
|
650
|
-
puts "\n=== Usage Summary ==="
|
|
651
382
|
puts "Total requests: #{tracker.request_count}"
|
|
652
383
|
puts "Total tokens: #{tracker.total_tokens}"
|
|
653
384
|
puts "Total cost: $#{tracker.total_cost.round(4)}"
|
|
654
|
-
puts "Average cost per request: $#{(tracker.total_cost / tracker.request_count).round(4)}"
|
|
655
385
|
|
|
656
|
-
#
|
|
386
|
+
# Per-model breakdown
|
|
657
387
|
tracker.model_usage.each do |model, stats|
|
|
658
|
-
puts "
|
|
659
|
-
puts " Requests: #{stats[:request_count]}"
|
|
660
|
-
puts " Tokens: #{stats[:total_tokens]}"
|
|
661
|
-
puts " Cost: $#{stats[:cost].round(4)}"
|
|
388
|
+
puts "#{model}: #{stats[:total_tokens]} tokens, $#{stats[:cost].round(4)}"
|
|
662
389
|
end
|
|
663
390
|
|
|
664
391
|
# Print detailed report
|
|
665
392
|
tracker.print_summary
|
|
666
393
|
```
|
|
667
394
|
|
|
668
|
-
|
|
669
|
-
|
|
670
|
-
```ruby
|
|
671
|
-
# Track specific operations
|
|
672
|
-
client.usage_tracker.reset! # Start fresh
|
|
673
|
-
|
|
674
|
-
# Simulate different workload types
|
|
675
|
-
client.complete(messages, model: "openai/gpt-4o") # Expensive, high-quality
|
|
676
|
-
client.complete(messages, model: "openai/gpt-4o-mini") # Cheap, fast
|
|
677
|
-
|
|
678
|
-
# Get usage metrics
|
|
679
|
-
cache_hit_rate = client.usage_tracker.cache_hit_rate
|
|
680
|
-
tokens_per_second = client.usage_tracker.tokens_per_second
|
|
681
|
-
|
|
682
|
-
puts "Cache hit rate: #{cache_hit_rate}%"
|
|
683
|
-
puts "Tokens per second: #{tokens_per_second}"
|
|
684
|
-
|
|
685
|
-
# Export usage data as CSV for analysis
|
|
686
|
-
csv_data = client.usage_tracker.export_csv
|
|
687
|
-
File.write("usage_report.csv", csv_data)
|
|
688
|
-
```
|
|
689
|
-
|
|
690
|
-
### Response Analytics
|
|
691
|
-
|
|
692
|
-
Every response includes comprehensive metadata for monitoring and optimization.
|
|
693
|
-
|
|
694
|
-
```ruby
|
|
695
|
-
response = client.complete(messages, model: "anthropic/claude-3-5-sonnet")
|
|
696
|
-
|
|
697
|
-
# Token metrics
|
|
698
|
-
puts "Input tokens: #{response.prompt_tokens}"
|
|
699
|
-
puts "Output tokens: #{response.completion_tokens}"
|
|
700
|
-
puts "Cached tokens: #{response.cached_tokens}"
|
|
701
|
-
puts "Total tokens: #{response.total_tokens}"
|
|
702
|
-
|
|
703
|
-
# Cost information (requires generation stats query)
|
|
704
|
-
puts "Total cost: $#{response.cost_estimate}"
|
|
705
|
-
|
|
706
|
-
# Model information
|
|
707
|
-
puts "Provider: #{response.provider}"
|
|
708
|
-
puts "Model: #{response.model}"
|
|
709
|
-
puts "System fingerprint: #{response.system_fingerprint}"
|
|
710
|
-
puts "Finish reason: #{response.finish_reason}"
|
|
711
|
-
```
|
|
712
|
-
|
|
713
|
-
### Callback System
|
|
714
|
-
|
|
715
|
-
The client provides an extensible callback system for monitoring requests, responses, and errors.
|
|
716
|
-
|
|
717
|
-
#### Basic Callbacks
|
|
718
|
-
|
|
719
|
-
```ruby
|
|
720
|
-
client = OpenRouter::Client.new
|
|
721
|
-
|
|
722
|
-
# Monitor all requests
|
|
723
|
-
client.on(:before_request) do |params|
|
|
724
|
-
puts "Making request to #{params[:model]} with #{params[:messages].size} messages"
|
|
725
|
-
end
|
|
726
|
-
|
|
727
|
-
# Monitor all responses
|
|
728
|
-
client.on(:after_response) do |response|
|
|
729
|
-
puts "Received response: #{response.total_tokens} tokens, $#{response.cost_estimate}"
|
|
730
|
-
end
|
|
731
|
-
|
|
732
|
-
# Monitor tool calls
|
|
733
|
-
client.on(:on_tool_call) do |tool_calls|
|
|
734
|
-
tool_calls.each do |call|
|
|
735
|
-
puts "Tool called: #{call.name} with args #{call.arguments}"
|
|
736
|
-
end
|
|
737
|
-
end
|
|
738
|
-
|
|
739
|
-
# Monitor errors
|
|
740
|
-
client.on(:on_error) do |error|
|
|
741
|
-
logger.error "API error: #{error.message}"
|
|
742
|
-
# Send to monitoring service
|
|
743
|
-
ErrorReporter.notify(error)
|
|
744
|
-
end
|
|
745
|
-
```
|
|
746
|
-
|
|
747
|
-
#### Advanced Callback Usage
|
|
748
|
-
|
|
749
|
-
```ruby
|
|
750
|
-
# Cost monitoring with alerts
|
|
751
|
-
client.on(:after_response) do |response|
|
|
752
|
-
if response.cost_estimate > 0.10
|
|
753
|
-
AlertService.send_alert(
|
|
754
|
-
"High cost request: $#{response.cost_estimate} for #{response.total_tokens} tokens"
|
|
755
|
-
)
|
|
756
|
-
end
|
|
757
|
-
end
|
|
758
|
-
|
|
759
|
-
# Performance monitoring
|
|
760
|
-
client.on(:before_request) { |params| @start_time = Time.now }
|
|
761
|
-
client.on(:after_response) do |response|
|
|
762
|
-
duration = Time.now - @start_time
|
|
763
|
-
if duration > 10.0
|
|
764
|
-
puts "Slow request detected: #{duration.round(2)}s"
|
|
765
|
-
end
|
|
766
|
-
end
|
|
767
|
-
|
|
768
|
-
# Usage analytics
|
|
769
|
-
request_count = 0
|
|
770
|
-
total_cost = 0.0
|
|
771
|
-
|
|
772
|
-
client.on(:after_response) do |response|
|
|
773
|
-
request_count += 1
|
|
774
|
-
total_cost += response.cost_estimate || 0.0
|
|
775
|
-
|
|
776
|
-
if request_count % 100 == 0
|
|
777
|
-
puts "100 requests processed. Average cost: $#{(total_cost / request_count).round(4)}"
|
|
778
|
-
end
|
|
779
|
-
end
|
|
780
|
-
|
|
781
|
-
# Chain callbacks for complex workflows
|
|
782
|
-
client
|
|
783
|
-
.on(:before_request) { |params| log_request(params) }
|
|
784
|
-
.on(:after_response) { |response| log_response(response) }
|
|
785
|
-
.on(:on_tool_call) { |calls| execute_tools(calls) }
|
|
786
|
-
.on(:on_error) { |error| handle_error(error) }
|
|
787
|
-
```
|
|
788
|
-
|
|
789
|
-
### Cost Management
|
|
790
|
-
|
|
791
|
-
Built-in cost estimation and usage tracking tools.
|
|
792
|
-
|
|
793
|
-
```ruby
|
|
794
|
-
# Pre-flight cost estimation
|
|
795
|
-
estimated_cost = OpenRouter::ModelRegistry.calculate_estimated_cost(
|
|
796
|
-
"anthropic/claude-3-5-sonnet",
|
|
797
|
-
input_tokens: 1500,
|
|
798
|
-
output_tokens: 800
|
|
799
|
-
)
|
|
800
|
-
|
|
801
|
-
puts "Estimated cost: $#{estimated_cost}"
|
|
802
|
-
|
|
803
|
-
# Use model selector to stay within budget
|
|
804
|
-
if estimated_cost > 0.01
|
|
805
|
-
puts "Switching to cheaper model"
|
|
806
|
-
model = OpenRouter::ModelSelector.new
|
|
807
|
-
.within_budget(max_cost: 0.01)
|
|
808
|
-
.require(:chat)
|
|
809
|
-
.choose
|
|
810
|
-
end
|
|
811
|
-
|
|
812
|
-
# Track costs in real-time
|
|
813
|
-
client = OpenRouter::Client.new(track_usage: true)
|
|
814
|
-
|
|
815
|
-
client.on(:after_response) do |response|
|
|
816
|
-
total_spent = client.usage_tracker.total_cost
|
|
817
|
-
puts "Total spent this session: $#{total_spent.round(4)}"
|
|
818
|
-
|
|
819
|
-
if total_spent > 5.00
|
|
820
|
-
puts "⚠️ Session cost exceeds $5.00"
|
|
821
|
-
end
|
|
822
|
-
end
|
|
823
|
-
```
|
|
824
|
-
|
|
825
|
-
## Advanced Features
|
|
826
|
-
|
|
827
|
-
### Model Fallbacks
|
|
828
|
-
|
|
829
|
-
Use multiple models with automatic failover for increased reliability.
|
|
830
|
-
|
|
831
|
-
```ruby
|
|
832
|
-
# Define fallback chain
|
|
833
|
-
response = client.complete(
|
|
834
|
-
messages,
|
|
835
|
-
model: ["openai/gpt-4o", "anthropic/claude-3-5-sonnet", "anthropic/claude-3-haiku"],
|
|
836
|
-
tools: tools
|
|
837
|
-
)
|
|
838
|
-
|
|
839
|
-
# Or use ModelSelector for intelligent fallbacks
|
|
840
|
-
models = OpenRouter::ModelSelector.new
|
|
841
|
-
.require(:function_calling)
|
|
842
|
-
.choose_with_fallbacks(limit: 3)
|
|
843
|
-
|
|
844
|
-
response = client.complete(messages, model: models, tools: tools)
|
|
845
|
-
```
|
|
846
|
-
|
|
847
|
-
### Response Healing
|
|
848
|
-
|
|
849
|
-
Automatically heal malformed responses from models that don't natively support structured outputs.
|
|
850
|
-
|
|
851
|
-
```ruby
|
|
852
|
-
# Configure global healing
|
|
853
|
-
OpenRouter.configure do |config|
|
|
854
|
-
config.auto_heal_responses = true
|
|
855
|
-
config.healer_model = "openai/gpt-4o-mini"
|
|
856
|
-
config.max_heal_attempts = 2
|
|
857
|
-
end
|
|
858
|
-
|
|
859
|
-
# The gem automatically heals malformed JSON responses
|
|
860
|
-
response = client.complete(
|
|
861
|
-
messages,
|
|
862
|
-
model: "some/model-without-native-structured-outputs",
|
|
863
|
-
response_format: schema # Will be automatically healed if malformed
|
|
864
|
-
)
|
|
865
|
-
```
|
|
866
|
-
|
|
867
|
-
### Performance Optimization
|
|
395
|
+
**[Complete Usage Tracking Documentation](docs/usage_tracking.md)**
|
|
868
396
|
|
|
869
|
-
|
|
397
|
+
## Model Exploration
|
|
870
398
|
|
|
871
|
-
|
|
399
|
+
The gem includes rake tasks for exploring available models:
|
|
872
400
|
|
|
873
|
-
```
|
|
874
|
-
|
|
875
|
-
|
|
876
|
-
# Process multiple requests in parallel
|
|
877
|
-
messages_batch = [
|
|
878
|
-
[{ role: "user", content: "Summarize this: #{text1}" }],
|
|
879
|
-
[{ role: "user", content: "Summarize this: #{text2}" }],
|
|
880
|
-
[{ role: "user", content: "Summarize this: #{text3}" }]
|
|
881
|
-
]
|
|
882
|
-
|
|
883
|
-
# Create thread pool
|
|
884
|
-
thread_pool = Concurrent::FixedThreadPool.new(5)
|
|
885
|
-
|
|
886
|
-
# Process batch with shared model selection
|
|
887
|
-
model = OpenRouter::ModelSelector.new
|
|
888
|
-
.optimize_for(:performance)
|
|
889
|
-
.require(:chat)
|
|
890
|
-
.choose
|
|
891
|
-
|
|
892
|
-
futures = messages_batch.map do |messages|
|
|
893
|
-
Concurrent::Future.execute(executor: thread_pool) do
|
|
894
|
-
client.complete(messages, model: model)
|
|
895
|
-
end
|
|
896
|
-
end
|
|
897
|
-
|
|
898
|
-
# Collect results
|
|
899
|
-
results = futures.map(&:value)
|
|
900
|
-
thread_pool.shutdown
|
|
901
|
-
```
|
|
902
|
-
|
|
903
|
-
#### Caching and Optimization
|
|
904
|
-
|
|
905
|
-
```ruby
|
|
906
|
-
# Enable aggressive caching
|
|
907
|
-
OpenRouter.configure do |config|
|
|
908
|
-
config.cache_ttl = 24 * 60 * 60 # 24 hours
|
|
909
|
-
config.auto_heal_responses = true
|
|
910
|
-
config.strict_mode = false # Better performance
|
|
911
|
-
end
|
|
912
|
-
|
|
913
|
-
# Use cheaper models for development/testing
|
|
914
|
-
if Rails.env.development?
|
|
915
|
-
client = OpenRouter::Client.new(
|
|
916
|
-
default_model: "openai/gpt-4o-mini", # Cheaper for development
|
|
917
|
-
track_usage: true
|
|
918
|
-
)
|
|
919
|
-
else
|
|
920
|
-
client = OpenRouter::Client.new(
|
|
921
|
-
default_model: "anthropic/claude-3-5-sonnet" # Production quality
|
|
922
|
-
)
|
|
923
|
-
end
|
|
924
|
-
|
|
925
|
-
# Pre-warm model registry cache
|
|
926
|
-
OpenRouter::ModelRegistry.refresh_cache!
|
|
927
|
-
|
|
928
|
-
# Optimize for specific workloads
|
|
929
|
-
fast_client = OpenRouter::Client.new(
|
|
930
|
-
request_timeout: 30, # Shorter timeout
|
|
931
|
-
auto_heal_responses: false, # Skip healing for speed
|
|
932
|
-
strict_mode: false # Skip capability validation
|
|
933
|
-
)
|
|
934
|
-
```
|
|
935
|
-
|
|
936
|
-
#### Memory Management
|
|
937
|
-
|
|
938
|
-
```ruby
|
|
939
|
-
# Reset usage tracking periodically for long-running apps
|
|
940
|
-
client.usage_tracker.reset! if client.usage_tracker.request_count > 1000
|
|
401
|
+
```bash
|
|
402
|
+
# View model summary with statistics
|
|
403
|
+
bundle exec rake models:summary
|
|
941
404
|
|
|
942
|
-
#
|
|
943
|
-
|
|
405
|
+
# Search by provider
|
|
406
|
+
bundle exec rake models:search provider=anthropic
|
|
944
407
|
|
|
945
|
-
#
|
|
946
|
-
|
|
408
|
+
# Search by capabilities and optimize for cost
|
|
409
|
+
bundle exec rake models:search capability=function_calling optimize=cost limit=10
|
|
947
410
|
|
|
948
|
-
|
|
949
|
-
|
|
950
|
-
model: "anthropic/claude-3-5-sonnet",
|
|
951
|
-
accumulate_response: false # Don't store full response
|
|
952
|
-
) do |chunk|
|
|
953
|
-
# Process chunk immediately and discard
|
|
954
|
-
process_chunk(chunk.content)
|
|
955
|
-
end
|
|
411
|
+
# Advanced filtering
|
|
412
|
+
bundle exec rake models:search provider=anthropic capability=function_calling min_context=100000 max_cost=0.01
|
|
956
413
|
```
|
|
957
414
|
|
|
958
|
-
|
|
415
|
+
Available search parameters:
|
|
416
|
+
- `provider=name` - Filter by provider
|
|
417
|
+
- `capability=cap1,cap2` - Required capabilities
|
|
418
|
+
- `optimize=strategy` - Optimization (cost, performance, latest, context)
|
|
419
|
+
- `min_context=tokens` - Minimum context length
|
|
420
|
+
- `max_cost=amount` - Maximum cost per 1k tokens
|
|
421
|
+
- `newer_than=YYYY-MM-DD` - Filter by release date
|
|
422
|
+
- `limit=N` - Maximum results (default: 20)
|
|
959
423
|
|
|
960
|
-
|
|
424
|
+
## Testing
|
|
961
425
|
|
|
962
426
|
### Running Tests
|
|
963
427
|
|
|
@@ -978,242 +442,78 @@ bundle exec rspec spec/vcr/ # VCR integration tests (requires API key
|
|
|
978
442
|
The project includes VCR tests that record real API interactions:
|
|
979
443
|
|
|
980
444
|
```bash
|
|
981
|
-
# Set API key
|
|
445
|
+
# Set API key
|
|
982
446
|
export OPENROUTER_API_KEY="your_api_key"
|
|
983
447
|
|
|
984
448
|
# Run VCR tests
|
|
985
449
|
bundle exec rspec spec/vcr/
|
|
986
450
|
|
|
987
|
-
# Re-record cassettes
|
|
451
|
+
# Re-record cassettes
|
|
988
452
|
rm -rf spec/fixtures/vcr_cassettes/
|
|
989
453
|
bundle exec rspec spec/vcr/
|
|
990
454
|
```
|
|
991
455
|
|
|
992
456
|
### Examples
|
|
993
457
|
|
|
994
|
-
|
|
458
|
+
Comprehensive examples for all features are available in the `examples/` directory:
|
|
995
459
|
|
|
996
460
|
```bash
|
|
997
461
|
# Set your API key
|
|
998
462
|
export OPENROUTER_API_KEY="your_key_here"
|
|
999
463
|
|
|
1000
|
-
# Run
|
|
464
|
+
# Run examples
|
|
1001
465
|
ruby -I lib examples/basic_completion.rb
|
|
1002
466
|
ruby -I lib examples/tool_calling_example.rb
|
|
1003
467
|
ruby -I lib examples/structured_outputs_example.rb
|
|
1004
468
|
ruby -I lib examples/model_selection_example.rb
|
|
1005
|
-
ruby -I lib examples/prompt_template_example.rb
|
|
1006
469
|
ruby -I lib examples/streaming_example.rb
|
|
1007
|
-
ruby -I lib examples/observability_example.rb
|
|
1008
|
-
ruby -I lib examples/smart_completion_example.rb
|
|
1009
|
-
|
|
1010
|
-
# Run all examples
|
|
1011
|
-
find examples -name "*.rb" -exec ruby -I lib {} \;
|
|
1012
|
-
```
|
|
1013
|
-
|
|
1014
|
-
### Model Exploration Rake Tasks
|
|
1015
|
-
|
|
1016
|
-
The gem includes convenient rake tasks for exploring and searching available models without writing code:
|
|
1017
|
-
|
|
1018
|
-
#### Model Summary
|
|
1019
|
-
|
|
1020
|
-
View an overview of all available models, including provider breakdown, capabilities, costs, and context lengths:
|
|
1021
|
-
|
|
1022
|
-
```bash
|
|
1023
|
-
bundle exec rake models:summary
|
|
1024
|
-
```
|
|
1025
|
-
|
|
1026
|
-
Output includes:
|
|
1027
|
-
- Total model count and breakdown by provider
|
|
1028
|
-
- Available capabilities across all models
|
|
1029
|
-
- Cost analysis (min/max/median for input and output tokens)
|
|
1030
|
-
- Context length statistics
|
|
1031
|
-
- Performance tier distribution
|
|
1032
|
-
|
|
1033
|
-
#### Model Search
|
|
1034
|
-
|
|
1035
|
-
Search for models using various filters and optimization strategies:
|
|
1036
|
-
|
|
1037
|
-
```bash
|
|
1038
|
-
# Basic search by provider
|
|
1039
|
-
bundle exec rake models:search provider=anthropic
|
|
1040
|
-
|
|
1041
|
-
# Search by capabilities
|
|
1042
|
-
bundle exec rake models:search capability=function_calling,vision
|
|
1043
|
-
|
|
1044
|
-
# Optimize for cost with capability requirements
|
|
1045
|
-
bundle exec rake models:search capability=function_calling optimize=cost limit=10
|
|
1046
|
-
|
|
1047
|
-
# Filter by context length
|
|
1048
|
-
bundle exec rake models:search min_context=200000
|
|
1049
|
-
|
|
1050
|
-
# Filter by cost
|
|
1051
|
-
bundle exec rake models:search max_cost=0.01
|
|
1052
|
-
|
|
1053
|
-
# Filter by release date
|
|
1054
|
-
bundle exec rake models:search newer_than=2024-01-01
|
|
1055
|
-
|
|
1056
|
-
# Combine multiple filters
|
|
1057
|
-
bundle exec rake models:search provider=anthropic capability=function_calling min_context=100000 optimize=cost limit=5
|
|
1058
|
-
```
|
|
1059
|
-
|
|
1060
|
-
Available search parameters:
|
|
1061
|
-
- `provider=name` - Filter by provider (comma-separated for multiple)
|
|
1062
|
-
- `capability=cap1,cap2` - Required capabilities (function_calling, vision, structured_outputs, etc.)
|
|
1063
|
-
- `optimize=strategy` - Optimization strategy (cost, performance, latest, context)
|
|
1064
|
-
- `min_context=tokens` - Minimum context length
|
|
1065
|
-
- `max_cost=amount` - Maximum input cost per 1k tokens
|
|
1066
|
-
- `max_output_cost=amount` - Maximum output cost per 1k tokens
|
|
1067
|
-
- `newer_than=YYYY-MM-DD` - Filter models released after date
|
|
1068
|
-
- `limit=N` - Maximum number of results to show (default: 20)
|
|
1069
|
-
- `fallbacks=true` - Show models with fallback support
|
|
1070
|
-
|
|
1071
|
-
Examples:
|
|
1072
|
-
|
|
1073
|
-
```bash
|
|
1074
|
-
# Find cheapest models with vision support
|
|
1075
|
-
bundle exec rake models:search capability=vision optimize=cost limit=5
|
|
1076
|
-
|
|
1077
|
-
# Find latest Anthropic models with function calling
|
|
1078
|
-
bundle exec rake models:search provider=anthropic optimize=latest capability=function_calling
|
|
1079
|
-
|
|
1080
|
-
# Find high-context models for long documents
|
|
1081
|
-
bundle exec rake models:search min_context=500000 optimize=context
|
|
1082
470
|
```
|
|
1083
471
|
|
|
1084
472
|
## Troubleshooting
|
|
1085
473
|
|
|
1086
|
-
### Common Issues
|
|
1087
|
-
|
|
1088
|
-
#### Authentication Errors
|
|
474
|
+
### Common Issues
|
|
1089
475
|
|
|
476
|
+
**Authentication Errors**
|
|
1090
477
|
```ruby
|
|
1091
|
-
#
|
|
1092
|
-
# Solution: Set your API key
|
|
478
|
+
# Set your API key
|
|
1093
479
|
export OPENROUTER_API_KEY="your_key_here"
|
|
1094
480
|
|
|
1095
|
-
# Or
|
|
481
|
+
# Or in code
|
|
1096
482
|
OpenRouter.configure do |config|
|
|
1097
483
|
config.access_token = ENV["OPENROUTER_API_KEY"]
|
|
1098
484
|
end
|
|
1099
|
-
|
|
1100
|
-
# Error: "Invalid API key"
|
|
1101
|
-
# Solution: Verify your key at https://openrouter.ai/keys
|
|
1102
485
|
```
|
|
1103
486
|
|
|
1104
|
-
|
|
1105
|
-
|
|
487
|
+
**Model Selection Issues**
|
|
1106
488
|
```ruby
|
|
1107
|
-
#
|
|
1108
|
-
# Solution: Check model availability and your account limits
|
|
1109
|
-
begin
|
|
1110
|
-
client.complete(messages, model: "gpt-4")
|
|
1111
|
-
rescue OpenRouter::ServerError => e
|
|
1112
|
-
if e.message.include?("not found")
|
|
1113
|
-
puts "Model not available, falling back to default"
|
|
1114
|
-
client.complete(messages, model: "openai/gpt-4o-mini")
|
|
1115
|
-
end
|
|
1116
|
-
end
|
|
1117
|
-
|
|
1118
|
-
# Error: "Model doesn't support feature X"
|
|
1119
|
-
# Solution: Use ModelSelector to find compatible models
|
|
1120
|
-
model = OpenRouter::ModelSelector.new
|
|
1121
|
-
.require(:function_calling)
|
|
1122
|
-
.choose
|
|
1123
|
-
```
|
|
1124
|
-
|
|
1125
|
-
#### Rate Limiting and Costs
|
|
1126
|
-
|
|
1127
|
-
```ruby
|
|
1128
|
-
# Error: "Rate limit exceeded"
|
|
1129
|
-
# Solution: Implement exponential backoff
|
|
1130
|
-
require 'retries'
|
|
1131
|
-
|
|
1132
|
-
with_retries(max_tries: 3, base_sleep_seconds: 1, max_sleep_seconds: 60) do |attempt|
|
|
1133
|
-
client.complete(messages, model: model)
|
|
1134
|
-
end
|
|
1135
|
-
|
|
1136
|
-
# Error: "Request too expensive"
|
|
1137
|
-
# Solution: Use cheaper models or budget constraints
|
|
1138
|
-
client = OpenRouter::Client.new
|
|
489
|
+
# Use ModelSelector to find compatible models
|
|
1139
490
|
model = OpenRouter::ModelSelector.new
|
|
1140
|
-
|
|
1141
|
-
|
|
491
|
+
.require(:function_calling)
|
|
492
|
+
.choose
|
|
1142
493
|
```
|
|
1143
494
|
|
|
1144
|
-
|
|
1145
|
-
|
|
495
|
+
**Structured Output Issues**
|
|
1146
496
|
```ruby
|
|
1147
|
-
#
|
|
1148
|
-
# Solution: Enable response healing
|
|
497
|
+
# Enable automatic healing
|
|
1149
498
|
OpenRouter.configure do |config|
|
|
1150
499
|
config.auto_heal_responses = true
|
|
1151
500
|
config.healer_model = "openai/gpt-4o-mini"
|
|
1152
501
|
end
|
|
1153
|
-
|
|
1154
|
-
# Error: "Schema validation failed"
|
|
1155
|
-
# Solution: Check schema definitions and model capability
|
|
1156
|
-
schema = OpenRouter::Schema.define("user") do
|
|
1157
|
-
string :name, required: true
|
|
1158
|
-
integer :age, minimum: 0 # Add constraints
|
|
1159
|
-
end
|
|
1160
|
-
|
|
1161
|
-
# Use models that support structured outputs natively
|
|
1162
|
-
model = OpenRouter::ModelSelector.new
|
|
1163
|
-
.require(:structured_outputs)
|
|
1164
|
-
.choose
|
|
1165
502
|
```
|
|
1166
503
|
|
|
1167
|
-
|
|
1168
|
-
|
|
504
|
+
**Performance Issues**
|
|
1169
505
|
```ruby
|
|
1170
|
-
#
|
|
1171
|
-
# Solution: Optimize client configuration
|
|
506
|
+
# Optimize configuration
|
|
1172
507
|
client = OpenRouter::Client.new(
|
|
1173
|
-
request_timeout: 30,
|
|
1174
|
-
strict_mode: false,
|
|
1175
|
-
auto_heal_responses: false
|
|
508
|
+
request_timeout: 30,
|
|
509
|
+
strict_mode: false,
|
|
510
|
+
auto_heal_responses: false
|
|
1176
511
|
)
|
|
1177
|
-
|
|
1178
|
-
# Issue: High memory usage
|
|
1179
|
-
# Solution: Use streaming for large responses
|
|
1180
|
-
streaming_client = OpenRouter::StreamingClient.new
|
|
1181
|
-
streaming_client.stream_complete(messages, accumulate_response: false) do |chunk|
|
|
1182
|
-
process_chunk_immediately(chunk)
|
|
1183
|
-
end
|
|
1184
|
-
|
|
1185
|
-
# Issue: Too many API calls
|
|
1186
|
-
# Solution: Implement request batching
|
|
1187
|
-
messages_batch = [...] # Multiple message sets
|
|
1188
|
-
results = process_batch_concurrently(messages_batch, thread_pool_size: 5)
|
|
1189
|
-
```
|
|
1190
|
-
|
|
1191
|
-
#### Tool Calling Issues
|
|
1192
|
-
|
|
1193
|
-
```ruby
|
|
1194
|
-
# Error: "Tool not found"
|
|
1195
|
-
# Solution: Verify tool definitions match exactly
|
|
1196
|
-
tool = OpenRouter::Tool.define do
|
|
1197
|
-
name "get_weather" # Must match exactly in model response
|
|
1198
|
-
description "Get current weather for a location"
|
|
1199
|
-
parameters do
|
|
1200
|
-
string :location, required: true
|
|
1201
|
-
end
|
|
1202
|
-
end
|
|
1203
|
-
|
|
1204
|
-
# Error: "Invalid tool parameters"
|
|
1205
|
-
# Solution: Add parameter validation
|
|
1206
|
-
def handle_weather_tool(tool_call)
|
|
1207
|
-
location = tool_call.arguments["location"]
|
|
1208
|
-
raise ArgumentError, "Location required" if location.nil? || location.empty?
|
|
1209
|
-
|
|
1210
|
-
get_weather_data(location)
|
|
1211
|
-
end
|
|
1212
512
|
```
|
|
1213
513
|
|
|
1214
514
|
### Debug Mode
|
|
1215
515
|
|
|
1216
|
-
Enable detailed logging
|
|
516
|
+
Enable detailed logging:
|
|
1217
517
|
|
|
1218
518
|
```ruby
|
|
1219
519
|
require 'logger'
|
|
@@ -1224,383 +524,29 @@ OpenRouter.configure do |config|
|
|
|
1224
524
|
f.response :logger, Logger.new($stdout), { headers: true, bodies: true, errors: true }
|
|
1225
525
|
end
|
|
1226
526
|
end
|
|
1227
|
-
|
|
1228
|
-
# Enable callback debugging
|
|
1229
|
-
client = OpenRouter::Client.new
|
|
1230
|
-
client.on(:before_request) { |params| puts "REQUEST: #{params.inspect}" }
|
|
1231
|
-
client.on(:after_response) { |response| puts "RESPONSE: #{response.inspect}" }
|
|
1232
|
-
client.on(:on_error) { |error| puts "ERROR: #{error.message}" }
|
|
1233
|
-
```
|
|
1234
|
-
|
|
1235
|
-
### Performance Monitoring
|
|
1236
|
-
|
|
1237
|
-
```ruby
|
|
1238
|
-
# Monitor request performance
|
|
1239
|
-
client.on(:before_request) { @start_time = Time.now }
|
|
1240
|
-
client.on(:after_response) do |response|
|
|
1241
|
-
duration = Time.now - @start_time
|
|
1242
|
-
if duration > 5.0
|
|
1243
|
-
puts "SLOW REQUEST: #{duration.round(2)}s for #{response.total_tokens} tokens"
|
|
1244
|
-
end
|
|
1245
|
-
end
|
|
1246
|
-
|
|
1247
|
-
# Monitor costs
|
|
1248
|
-
client.on(:after_response) do |response|
|
|
1249
|
-
if response.cost_estimate > 0.10
|
|
1250
|
-
puts "EXPENSIVE REQUEST: $#{response.cost_estimate}"
|
|
1251
|
-
end
|
|
1252
|
-
end
|
|
1253
|
-
|
|
1254
|
-
# Export usage data as CSV for analysis
|
|
1255
|
-
csv_data = client.usage_tracker.export_csv
|
|
1256
|
-
File.write("debug_usage.csv", csv_data)
|
|
1257
527
|
```
|
|
1258
528
|
|
|
1259
529
|
### Getting Help
|
|
1260
530
|
|
|
1261
|
-
1. **Check the documentation**:
|
|
1262
|
-
2. **Review examples**:
|
|
1263
|
-
3. **Enable debug mode**: Turn on logging to see
|
|
1264
|
-
4. **Check OpenRouter status**:
|
|
1265
|
-
5. **Open an issue**:
|
|
1266
|
-
|
|
1267
|
-
## API Reference
|
|
1268
|
-
|
|
1269
|
-
### Client Classes
|
|
1270
|
-
|
|
1271
|
-
#### OpenRouter::Client
|
|
1272
|
-
|
|
1273
|
-
Main client for OpenRouter API interactions.
|
|
1274
|
-
|
|
1275
|
-
```ruby
|
|
1276
|
-
client = OpenRouter::Client.new(
|
|
1277
|
-
access_token: "...",
|
|
1278
|
-
track_usage: false,
|
|
1279
|
-
request_timeout: 120
|
|
1280
|
-
)
|
|
1281
|
-
|
|
1282
|
-
# Core methods
|
|
1283
|
-
client.complete(messages, **options) # Chat completions with full feature support
|
|
1284
|
-
client.models # List available models
|
|
1285
|
-
client.query_generation_stats(id) # Query generation statistics
|
|
1286
|
-
|
|
1287
|
-
# Callback methods
|
|
1288
|
-
client.on(event, &block) # Register event callback
|
|
1289
|
-
client.clear_callbacks(event) # Clear callbacks for event
|
|
1290
|
-
client.trigger_callbacks(event, data) # Manually trigger callbacks
|
|
1291
|
-
|
|
1292
|
-
# Usage tracking
|
|
1293
|
-
client.usage_tracker # Access usage tracker instance
|
|
1294
|
-
```
|
|
1295
|
-
|
|
1296
|
-
#### OpenRouter::StreamingClient
|
|
1297
|
-
|
|
1298
|
-
Enhanced streaming client with callback support.
|
|
1299
|
-
|
|
1300
|
-
```ruby
|
|
1301
|
-
streaming_client = OpenRouter::StreamingClient.new
|
|
531
|
+
1. **Check the documentation**: Detailed docs in the `docs/` directory
|
|
532
|
+
2. **Review examples**: Working examples in `examples/`
|
|
533
|
+
3. **Enable debug mode**: Turn on logging to see details
|
|
534
|
+
4. **Check OpenRouter status**: [OpenRouter Status](https://status.openrouter.ai)
|
|
535
|
+
5. **Open an issue**: [GitHub Issues](https://github.com/estiens/open_router_enhanced/issues)
|
|
1302
536
|
|
|
1303
|
-
|
|
1304
|
-
streaming_client.stream_complete(messages, **options) # Stream with callbacks
|
|
1305
|
-
streaming_client.on_stream(event, &block) # Register streaming callbacks
|
|
1306
|
-
|
|
1307
|
-
# Available streaming events: :on_start, :on_chunk, :on_tool_call_chunk, :on_finish, :on_error
|
|
1308
|
-
```
|
|
1309
|
-
|
|
1310
|
-
### Enhanced Classes
|
|
1311
|
-
|
|
1312
|
-
#### OpenRouter::Tool
|
|
1313
|
-
|
|
1314
|
-
Define and manage function calling tools.
|
|
1315
|
-
|
|
1316
|
-
```ruby
|
|
1317
|
-
# DSL definition
|
|
1318
|
-
tool = OpenRouter::Tool.define do
|
|
1319
|
-
name "function_name"
|
|
1320
|
-
description "Function description"
|
|
1321
|
-
parameters do
|
|
1322
|
-
string :param1, required: true, description: "Parameter description"
|
|
1323
|
-
integer :param2, minimum: 0, maximum: 100
|
|
1324
|
-
boolean :param3, default: false
|
|
1325
|
-
end
|
|
1326
|
-
end
|
|
1327
|
-
|
|
1328
|
-
# Hash definition
|
|
1329
|
-
tool = OpenRouter::Tool.from_hash({
|
|
1330
|
-
name: "function_name",
|
|
1331
|
-
description: "Function description",
|
|
1332
|
-
parameters: {
|
|
1333
|
-
type: "object",
|
|
1334
|
-
properties: { ... }
|
|
1335
|
-
}
|
|
1336
|
-
})
|
|
1337
|
-
|
|
1338
|
-
# Methods
|
|
1339
|
-
tool.name # Get tool name
|
|
1340
|
-
tool.description # Get tool description
|
|
1341
|
-
tool.parameters # Get parameters schema
|
|
1342
|
-
tool.to_h # Convert to hash format
|
|
1343
|
-
tool.validate_arguments(args) # Validate arguments against schema
|
|
1344
|
-
```
|
|
1345
|
-
|
|
1346
|
-
#### OpenRouter::Schema
|
|
1347
|
-
|
|
1348
|
-
Define JSON schemas for structured outputs.
|
|
1349
|
-
|
|
1350
|
-
```ruby
|
|
1351
|
-
# DSL definition
|
|
1352
|
-
schema = OpenRouter::Schema.define("schema_name") do
|
|
1353
|
-
string :name, required: true, description: "User's name"
|
|
1354
|
-
integer :age, minimum: 0, maximum: 150
|
|
1355
|
-
boolean :active, default: true
|
|
1356
|
-
array :tags, items: { type: "string" }
|
|
1357
|
-
object :address do
|
|
1358
|
-
string :street, required: true
|
|
1359
|
-
string :city, required: true
|
|
1360
|
-
string :country, default: "US"
|
|
1361
|
-
end
|
|
1362
|
-
end
|
|
1363
|
-
|
|
1364
|
-
# Hash definition
|
|
1365
|
-
schema = OpenRouter::Schema.from_hash("schema_name", {
|
|
1366
|
-
type: "object",
|
|
1367
|
-
properties: { ... },
|
|
1368
|
-
required: [...]
|
|
1369
|
-
})
|
|
1370
|
-
|
|
1371
|
-
# Methods
|
|
1372
|
-
schema.name # Get schema name
|
|
1373
|
-
schema.schema # Get JSON schema hash
|
|
1374
|
-
schema.validate(data) # Validate data against schema
|
|
1375
|
-
schema.to_h # Convert to hash format
|
|
1376
|
-
```
|
|
1377
|
-
|
|
1378
|
-
#### OpenRouter::PromptTemplate
|
|
1379
|
-
|
|
1380
|
-
Create reusable prompt templates with variable interpolation.
|
|
1381
|
-
|
|
1382
|
-
```ruby
|
|
1383
|
-
# Basic template
|
|
1384
|
-
template = OpenRouter::PromptTemplate.new(
|
|
1385
|
-
template: "Translate '{text}' from {source} to {target}",
|
|
1386
|
-
input_variables: [:text, :source, :target]
|
|
1387
|
-
)
|
|
1388
|
-
|
|
1389
|
-
# Few-shot template
|
|
1390
|
-
template = OpenRouter::PromptTemplate.new(
|
|
1391
|
-
prefix: "Classification examples:",
|
|
1392
|
-
suffix: "Classify: {input}",
|
|
1393
|
-
examples: [{ input: "...", output: "..." }],
|
|
1394
|
-
example_template: "Input: {input}\nOutput: {output}",
|
|
1395
|
-
input_variables: [:input]
|
|
1396
|
-
)
|
|
1397
|
-
|
|
1398
|
-
# Methods
|
|
1399
|
-
template.format(**variables) # Format template with variables
|
|
1400
|
-
template.to_messages(**variables) # Convert to OpenRouter message format
|
|
1401
|
-
template.input_variables # Get required input variables
|
|
1402
|
-
template.partial(**variables) # Create partial template with some variables filled
|
|
1403
|
-
```
|
|
1404
|
-
|
|
1405
|
-
#### OpenRouter::ModelSelector
|
|
1406
|
-
|
|
1407
|
-
Intelligent model selection with fluent DSL.
|
|
1408
|
-
|
|
1409
|
-
```ruby
|
|
1410
|
-
selector = OpenRouter::ModelSelector.new
|
|
1411
|
-
|
|
1412
|
-
# Requirement methods
|
|
1413
|
-
selector.require(*capabilities) # Require specific capabilities
|
|
1414
|
-
selector.within_budget(max_cost: 0.01) # Set maximum cost constraint
|
|
1415
|
-
selector.min_context(tokens) # Minimum context length
|
|
1416
|
-
selector.prefer_providers(*providers) # Prefer specific providers
|
|
1417
|
-
selector.avoid_providers(*providers) # Avoid specific providers
|
|
1418
|
-
selector.optimize_for(strategy) # Optimization strategy (:cost, :performance, :balanced)
|
|
1419
|
-
|
|
1420
|
-
# Selection methods
|
|
1421
|
-
selector.choose # Choose best single model
|
|
1422
|
-
selector.choose_with_fallbacks(limit: 3) # Choose multiple models for fallback
|
|
1423
|
-
selector.candidates # Get all matching models
|
|
1424
|
-
selector.explain_choice # Get explanation of selection
|
|
1425
|
-
|
|
1426
|
-
# Available capabilities: :chat, :function_calling, :structured_outputs, :vision, :code_generation
|
|
1427
|
-
# Available strategies: :cost, :performance, :balanced
|
|
1428
|
-
```
|
|
1429
|
-
|
|
1430
|
-
#### OpenRouter::ModelRegistry
|
|
1431
|
-
|
|
1432
|
-
Model information and capability detection.
|
|
1433
|
-
|
|
1434
|
-
```ruby
|
|
1435
|
-
# Class methods
|
|
1436
|
-
OpenRouter::ModelRegistry.all_models # Get all cached models
|
|
1437
|
-
OpenRouter::ModelRegistry.get_model_info(model) # Get specific model info
|
|
1438
|
-
OpenRouter::ModelRegistry.models_meeting_requirements(...) # Find models matching criteria
|
|
1439
|
-
OpenRouter::ModelRegistry.calculate_estimated_cost(model, tokens) # Estimate cost
|
|
1440
|
-
OpenRouter::ModelRegistry.refresh_cache! # Refresh model cache
|
|
1441
|
-
OpenRouter::ModelRegistry.cache_status # Get cache status
|
|
1442
|
-
```
|
|
1443
|
-
|
|
1444
|
-
#### OpenRouter::UsageTracker
|
|
1445
|
-
|
|
1446
|
-
Track token usage, costs, and performance metrics.
|
|
1447
|
-
|
|
1448
|
-
```ruby
|
|
1449
|
-
tracker = client.usage_tracker
|
|
1450
|
-
|
|
1451
|
-
# Metrics
|
|
1452
|
-
tracker.total_tokens # Total tokens used
|
|
1453
|
-
tracker.total_cost # Total estimated cost
|
|
1454
|
-
tracker.request_count # Number of requests made
|
|
1455
|
-
tracker.model_usage # Per-model usage breakdown
|
|
1456
|
-
tracker.session_duration # Time since tracking started
|
|
1457
|
-
|
|
1458
|
-
# Analysis methods
|
|
1459
|
-
tracker.cache_hit_rate # Cache hit rate percentage
|
|
1460
|
-
tracker.tokens_per_second # Tokens processed per second
|
|
1461
|
-
tracker.print_summary # Print detailed usage report
|
|
1462
|
-
tracker.export_csv # Export usage data as CSV
|
|
1463
|
-
tracker.summary # Get usage summary hash
|
|
1464
|
-
tracker.reset! # Reset all counters
|
|
1465
|
-
```
|
|
1466
|
-
|
|
1467
|
-
### Response Objects
|
|
1468
|
-
|
|
1469
|
-
#### OpenRouter::Response
|
|
1470
|
-
|
|
1471
|
-
Enhanced response wrapper with metadata and feature support.
|
|
1472
|
-
|
|
1473
|
-
```ruby
|
|
1474
|
-
response = client.complete(messages)
|
|
1475
|
-
|
|
1476
|
-
# Content access
|
|
1477
|
-
response.content # Response content
|
|
1478
|
-
response.structured_output # Parsed JSON for structured outputs
|
|
1479
|
-
|
|
1480
|
-
# Tool calling
|
|
1481
|
-
response.has_tool_calls? # Check if response has tool calls
|
|
1482
|
-
response.tool_calls # Array of ToolCall objects
|
|
1483
|
-
|
|
1484
|
-
# Token metrics
|
|
1485
|
-
response.prompt_tokens # Input tokens
|
|
1486
|
-
response.completion_tokens # Output tokens
|
|
1487
|
-
response.cached_tokens # Cached tokens
|
|
1488
|
-
response.total_tokens # Total tokens
|
|
1489
|
-
|
|
1490
|
-
# Cost information
|
|
1491
|
-
response.input_cost # Input cost
|
|
1492
|
-
response.output_cost # Output cost
|
|
1493
|
-
response.cost_estimate # Total estimated cost
|
|
1494
|
-
|
|
1495
|
-
# Performance metrics
|
|
1496
|
-
response.response_time # Response time in milliseconds
|
|
1497
|
-
response.tokens_per_second # Processing speed
|
|
1498
|
-
|
|
1499
|
-
# Model information
|
|
1500
|
-
response.model # Model used
|
|
1501
|
-
response.provider # Provider name
|
|
1502
|
-
response.system_fingerprint # System fingerprint
|
|
1503
|
-
response.finish_reason # Why generation stopped
|
|
1504
|
-
|
|
1505
|
-
# Cache information
|
|
1506
|
-
response.cache_hit? # Whether response used cache
|
|
1507
|
-
response.cache_efficiency # Cache efficiency percentage
|
|
1508
|
-
|
|
1509
|
-
# Backward compatibility - delegates hash methods to raw response
|
|
1510
|
-
response["key"] # Hash-style access
|
|
1511
|
-
response.dig("path", "to", "value") # Deep hash access
|
|
1512
|
-
```
|
|
1513
|
-
|
|
1514
|
-
#### OpenRouter::ToolCall
|
|
1515
|
-
|
|
1516
|
-
Individual tool call handling and execution.
|
|
1517
|
-
|
|
1518
|
-
```ruby
|
|
1519
|
-
tool_call = response.tool_calls.first
|
|
1520
|
-
|
|
1521
|
-
# Properties
|
|
1522
|
-
tool_call.id # Tool call ID
|
|
1523
|
-
tool_call.name # Tool name
|
|
1524
|
-
tool_call.arguments # Tool arguments (Hash)
|
|
1525
|
-
|
|
1526
|
-
# Methods
|
|
1527
|
-
tool_call.validate_arguments! # Validate arguments against tool schema
|
|
1528
|
-
tool_call.to_message # Convert to continuation message format
|
|
1529
|
-
tool_call.execute(&block) # Execute tool with block
|
|
1530
|
-
```
|
|
1531
|
-
|
|
1532
|
-
### Error Classes
|
|
1533
|
-
|
|
1534
|
-
```ruby
|
|
1535
|
-
OpenRouter::Error # Base error class
|
|
1536
|
-
OpenRouter::ConfigurationError # Configuration issues
|
|
1537
|
-
OpenRouter::CapabilityError # Capability validation errors
|
|
1538
|
-
OpenRouter::ServerError # API server errors
|
|
1539
|
-
OpenRouter::ToolCallError # Tool execution errors
|
|
1540
|
-
OpenRouter::SchemaValidationError # Schema validation errors
|
|
1541
|
-
OpenRouter::StructuredOutputError # JSON parsing/healing errors
|
|
1542
|
-
OpenRouter::ModelRegistryError # Model registry errors
|
|
1543
|
-
OpenRouter::ModelSelectionError # Model selection errors
|
|
1544
|
-
```
|
|
1545
|
-
|
|
1546
|
-
### Configuration Options
|
|
1547
|
-
|
|
1548
|
-
```ruby
|
|
1549
|
-
OpenRouter.configure do |config|
|
|
1550
|
-
# Authentication
|
|
1551
|
-
config.access_token = "sk-..."
|
|
1552
|
-
config.site_name = "Your App Name"
|
|
1553
|
-
config.site_url = "https://yourapp.com"
|
|
1554
|
-
|
|
1555
|
-
# Request settings
|
|
1556
|
-
config.request_timeout = 120
|
|
1557
|
-
config.api_version = "v1"
|
|
1558
|
-
config.uri_base = "https://openrouter.ai/api"
|
|
1559
|
-
config.extra_headers = {}
|
|
1560
|
-
|
|
1561
|
-
# Response healing
|
|
1562
|
-
config.auto_heal_responses = true
|
|
1563
|
-
config.healer_model = "openai/gpt-4o-mini"
|
|
1564
|
-
config.max_heal_attempts = 2
|
|
1565
|
-
|
|
1566
|
-
# Capability validation
|
|
1567
|
-
config.strict_mode = true
|
|
1568
|
-
config.auto_force_on_unsupported_models = true
|
|
1569
|
-
|
|
1570
|
-
# Structured outputs
|
|
1571
|
-
config.default_structured_output_mode = :strict
|
|
1572
|
-
|
|
1573
|
-
# Caching
|
|
1574
|
-
config.cache_ttl = 7 * 24 * 60 * 60 # 7 days
|
|
1575
|
-
|
|
1576
|
-
# Model registry
|
|
1577
|
-
config.model_registry_timeout = 30
|
|
1578
|
-
config.model_registry_retries = 3
|
|
1579
|
-
|
|
1580
|
-
# Logging
|
|
1581
|
-
config.log_errors = false
|
|
1582
|
-
config.faraday do |f|
|
|
1583
|
-
f.response :logger
|
|
1584
|
-
end
|
|
1585
|
-
end
|
|
1586
|
-
```
|
|
537
|
+
**[Complete Troubleshooting Guide](docs/troubleshooting.md)**
|
|
1587
538
|
|
|
1588
539
|
## Contributing
|
|
1589
540
|
|
|
1590
541
|
Bug reports and pull requests are welcome on GitHub at <https://github.com/estiens/open_router_enhanced>.
|
|
1591
542
|
|
|
1592
|
-
This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](https://www.contributor-covenant.org/) code of conduct.
|
|
1593
|
-
|
|
1594
|
-
For detailed contribution guidelines, see [CONTRIBUTING.md](.github/CONTRIBUTING.md).
|
|
1595
|
-
|
|
1596
543
|
### Branch Strategy
|
|
1597
544
|
|
|
1598
545
|
We use a two-branch workflow:
|
|
1599
|
-
|
|
1600
546
|
- **`main`** - Stable releases only. Protected branch.
|
|
1601
|
-
- **`dev`** - Active development. All PRs should target this branch
|
|
547
|
+
- **`dev`** - Active development. **All PRs should target this branch.**
|
|
1602
548
|
|
|
1603
|
-
|
|
549
|
+
**Important:** Always target your PRs to the `dev` branch, not `main`.
|
|
1604
550
|
|
|
1605
551
|
### Development Setup
|
|
1606
552
|
|
|
@@ -1611,42 +557,21 @@ bundle install
|
|
|
1611
557
|
bundle exec rspec
|
|
1612
558
|
```
|
|
1613
559
|
|
|
1614
|
-
|
|
1615
|
-
|
|
1616
|
-
```bash
|
|
1617
|
-
# Set your API key
|
|
1618
|
-
export OPENROUTER_API_KEY="your_key_here"
|
|
1619
|
-
|
|
1620
|
-
# Run examples
|
|
1621
|
-
ruby -I lib examples/tool_calling_example.rb
|
|
1622
|
-
ruby -I lib examples/structured_outputs_example.rb
|
|
1623
|
-
ruby -I lib examples/model_selection_example.rb
|
|
1624
|
-
```
|
|
560
|
+
For detailed contribution guidelines, see [CONTRIBUTING.md](.github/CONTRIBUTING.md).
|
|
1625
561
|
|
|
1626
562
|
## Acknowledgments
|
|
1627
563
|
|
|
1628
|
-
This enhanced fork builds upon the excellent foundation laid by [Obie Fernandez](https://github.com/obie) and the original OpenRouter Ruby gem. The original library was bootstrapped from the [Anthropic gem](https://github.com/alexrudall/anthropic) by [Alex Rudall](https://github.com/alexrudall)
|
|
564
|
+
This enhanced fork builds upon the excellent foundation laid by [Obie Fernandez](https://github.com/obie) and the original OpenRouter Ruby gem. The original library was bootstrapped from the [Anthropic gem](https://github.com/alexrudall/anthropic) by [Alex Rudall](https://github.com/alexrudall).
|
|
1629
565
|
|
|
1630
566
|
We extend our heartfelt gratitude to:
|
|
1631
|
-
|
|
1632
|
-
- **
|
|
1633
|
-
- **Alex Rudall** - Creator of the Anthropic gem that served as the foundation
|
|
567
|
+
- **Obie Fernandez** - Original OpenRouter gem author
|
|
568
|
+
- **Alex Rudall** - Creator of the Anthropic gem foundation
|
|
1634
569
|
- **The OpenRouter Team** - For creating an amazing unified AI API
|
|
1635
570
|
- **The Ruby Community** - For continuous support and contributions
|
|
1636
571
|
|
|
1637
|
-
##
|
|
1638
|
-
|
|
1639
|
-
This enhanced fork is maintained by:
|
|
572
|
+
## Consulting
|
|
1640
573
|
|
|
1641
|
-
|
|
1642
|
-
- Email: hello@ericstiens.dev
|
|
1643
|
-
- Website: [ericstiens.dev](http://ericstiens.dev)
|
|
1644
|
-
- GitHub: [@estiens](https://github.com/estiens)
|
|
1645
|
-
- Blog: [Low Level Magic](https://lowlevelmagic.io)
|
|
1646
|
-
|
|
1647
|
-
### Need Help with AI Integration?
|
|
1648
|
-
|
|
1649
|
-
I'm available for consulting on Ruby AI applications, LLM integration, and building production-ready AI systems. My work extends beyond Ruby to include real-time AI orchestration, character-based AI systems, multi-agent architectures, and low-latency voice/streaming applications. Whether you need help with tool calling workflows, cost optimization, building AI characters with persistent memory, or orchestrating complex multi-model systems, I'd be happy to help.
|
|
574
|
+
I'm available for consulting on Ruby AI applications, LLM integration, and building production-ready AI systems. My work extends beyond Ruby to include real-time AI orchestration, character-based AI systems, multi-agent architectures, and low-latency voice/streaming applications.
|
|
1650
575
|
|
|
1651
576
|
**Get in touch:**
|
|
1652
577
|
- Email: hello@lowlevelmagic.io
|
|
@@ -1656,5 +581,3 @@ I'm available for consulting on Ruby AI applications, LLM integration, and build
|
|
|
1656
581
|
## License
|
|
1657
582
|
|
|
1658
583
|
The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
|
|
1659
|
-
|
|
1660
|
-
MIT License is chosen for maximum permissiveness and compatibility, allowing unrestricted use, modification, and distribution while maintaining attribution requirements.
|