llm.rb 15.0.2 → 15.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +161 -1
- data/README.md +134 -85
- data/bin/llm.rb +38 -3
- data/data/bedrock.json +250 -0
- data/data/deepinfra.json +162 -0
- data/data/deepseek.json +50 -0
- data/data/google.json +19 -21
- data/data/openrouter.json +14280 -0
- data/data/xai.json +28 -16
- data/docs/deepdive/advanced/context.md +8 -6
- data/docs/deepdive/fundamentals/agents.md +11 -10
- data/docs/deepdive/fundamentals/stream.md +4 -4
- data/lib/llm/active_record.rb +1 -1
- data/lib/llm/agent.rb +6 -0
- data/lib/llm/context.rb +28 -13
- data/lib/llm/cost.rb +13 -0
- data/lib/llm/function/fork/task.rb +14 -10
- data/lib/llm/provider.rb +29 -8
- data/lib/llm/providers/anthropic.rb +1 -1
- data/lib/llm/providers/bedrock/models.rb +2 -2
- data/lib/llm/providers/bedrock.rb +1 -1
- data/lib/llm/providers/google.rb +1 -1
- data/lib/llm/providers/ollama.rb +1 -1
- data/lib/llm/providers/openai/responses.rb +2 -1
- data/lib/llm/providers/openai.rb +1 -1
- data/lib/llm/providers/openrouter.rb +87 -0
- data/lib/llm/repl/buffer.rb +20 -5
- data/lib/llm/repl/input.rb +5 -4
- data/lib/llm/repl/markdown/table.rb +5 -2
- data/lib/llm/repl/node.rb +26 -1
- data/lib/llm/repl/stream.rb +30 -3
- data/lib/llm/repl/window.rb +7 -7
- data/lib/llm/repl.rb +1 -0
- data/lib/llm/skill.rb +7 -1
- data/lib/llm/stream.rb +8 -3
- data/lib/llm/tools/git.rb +2 -2
- data/lib/llm/tools/mkdir.rb +2 -2
- data/lib/llm/tools/rg.rb +2 -2
- data/lib/llm/tools/ruby.rb +2 -2
- data/lib/llm/tools/shell.rb +2 -2
- data/lib/llm/tools/utils.rb +1 -1
- data/lib/llm/transport/curb.rb +5 -3
- data/lib/llm/transport/http.rb +5 -2
- data/lib/llm/transport/persistent_http.rb +6 -4
- data/lib/llm/transport/utils.rb +8 -6
- data/lib/llm/version.rb +1 -1
- data/lib/llm.rb +16 -1
- data/llm.gemspec +2 -1
- metadata +19 -3
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 2b496d3aec5a309af0ff9eaac32cabb6c7191bf2bac1f3b664c02a56c593402e
|
|
4
|
+
data.tar.gz: 0ae83d7794b4dad881de7b9866b3894746efa1853e349fc5707416e367802db5
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 9f890dd0720d0acbc79f9df2196f3595a68cd618fd0d09dc02fcab9a6fe27758ae72efd4b764614d5511bdb7914f586fba7194b3d1a5bec186562b9196753783
|
|
7
|
+
data.tar.gz: f25bc47b9bd5dcb0afea4f2ddbcb1ad37f74863cff962de6ceefa6157af073997589fd08fad0b2df11288aa24db41685127542b4d1748a71c4fef607d6b76ba2
|
data/CHANGELOG.md
CHANGED
|
@@ -15,6 +15,166 @@
|
|
|
15
15
|
|
|
16
16
|
## What's next
|
|
17
17
|
|
|
18
|
+
## v15.1.0
|
|
19
|
+
|
|
20
|
+
Changes since `v15.0.3`.
|
|
21
|
+
|
|
22
|
+
This release adds the OpenRouter provider, splits provider timeouts
|
|
23
|
+
into `connect_timeout` and `read_timeout`, retries timed-out requests,
|
|
24
|
+
and renames `on_rate_limit` to `on_retry`. Skills now gain a
|
|
25
|
+
frontmatter `model:` parameter and inherit the parent agent's model,
|
|
26
|
+
while the CLI gains `-m` and `-x` switches, and the REPL shows retry
|
|
27
|
+
progress and measures text by display width.
|
|
28
|
+
|
|
29
|
+
### Provider
|
|
30
|
+
|
|
31
|
+
* **add `LLM::OpenRouter` for the OpenRouter provider** <br>
|
|
32
|
+
[`LLM::OpenRouter`](https://r.uby.dev/api-docs/llm.rb/LLM/OpenRouter.html)
|
|
33
|
+
is a new provider that talks to [OpenRouter](https://openrouter.ai)
|
|
34
|
+
through its OpenAI-compatible API, contributed via
|
|
35
|
+
[PR #165](https://github.com/r-uby-dev/llm.rb/pull/165). Create an
|
|
36
|
+
instance with
|
|
37
|
+
[`LLM.openrouter`](https://r.uby.dev/api-docs/llm.rb/LLM.html#openrouter-class_method),
|
|
38
|
+
which accepts the same `key:`, `host:`, and `base_path:` options as the
|
|
39
|
+
OpenAI provider. It defaults to the `openrouter/auto` router model and
|
|
40
|
+
supports chat completions, streaming, tool calls, structured output,
|
|
41
|
+
and embeddings; image, audio, moderation, files, and vector store
|
|
42
|
+
endpoints raise `NotImplementedError`. Model metadata ships in
|
|
43
|
+
`data/openrouter.json` for the registry.
|
|
44
|
+
|
|
45
|
+
* **provider: `LLM::Provider#with` accepts headers without the `headers:` keyword** <br>
|
|
46
|
+
[`LLM::Provider#with`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#with-instance_method)
|
|
47
|
+
now accepts headers directly as a Hash (`llm.with("User-Agent" => "llmrb/1.0")`)
|
|
48
|
+
in addition to the legacy `headers:` keyword form. Both are merged, and
|
|
49
|
+
the keyword form remains for backwards compatibility.
|
|
50
|
+
|
|
51
|
+
* **providers: `model: nil` falls back to `default_model`** <br>
|
|
52
|
+
[`LLM::Provider`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html)
|
|
53
|
+
now treats a `model: nil` payload as "use the default model" instead of
|
|
54
|
+
sending a model value that providers reject. Previously a `{model: nil}`
|
|
55
|
+
param overwrote the default and then was removed by `.compact`, leaving
|
|
56
|
+
undefined behavior where providers could reject the request.
|
|
57
|
+
|
|
58
|
+
* **openai: handle `model: nil` in the Responses API** <br>
|
|
59
|
+
The OpenAI Responses API path now also falls back to the default model
|
|
60
|
+
when `model: nil`, matching the completions path.
|
|
61
|
+
|
|
62
|
+
* **provider: separate `connect_timeout` and `read_timeout`** <br>
|
|
63
|
+
[`LLM::Provider`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html)
|
|
64
|
+
now accepts a `connect_timeout:` for opening the connection (default 5
|
|
65
|
+
seconds) and a `read_timeout:` for waiting on an idle connection (default
|
|
66
|
+
600 seconds), instead of a single `timeout`. The legacy `timeout:` option
|
|
67
|
+
remains as a shorthand for `read_timeout`. The provider exposes the new
|
|
68
|
+
`#read_timeout` and `#connect_timeout` accessors.
|
|
69
|
+
|
|
70
|
+
* **provider: retry timed-out requests** <br>
|
|
71
|
+
Requests that time out (`Net::OpenTimeout` or `Net::ReadTimeout`) are now
|
|
72
|
+
retried along with rate-limited requests, up to the `retry_budget`. When
|
|
73
|
+
a heavily loaded provider (for example DeepSeek) drops the connection
|
|
74
|
+
during the connect or read phase, the request is retried instead of
|
|
75
|
+
failing, and the stream is notified through `on_retry`.
|
|
76
|
+
|
|
77
|
+
* **context: retry on `LLM::InsufficientQuotaError`, too** <br>
|
|
78
|
+
A request that raises
|
|
79
|
+
[`LLM::InsufficientQuotaError`](https://r.uby.dev/api-docs/llm.rb/LLM/InsufficientQuotaError.html)
|
|
80
|
+
is now retried like other rate-limited requests. The error is a subclass
|
|
81
|
+
of `LLM::RateLimitError` and can be raised at regular intervals, in
|
|
82
|
+
particular by the Alibaba provider.
|
|
83
|
+
|
|
84
|
+
* **cost: return `LLM::Cost.zero` when the registry has no pricing** <br>
|
|
85
|
+
Add
|
|
86
|
+
[`LLM::Cost.zero`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html#zero-class_method),
|
|
87
|
+
a factory for a zero-valued cost breakdown.
|
|
88
|
+
[`LLM::Context#cost`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#cost-instance_method)
|
|
89
|
+
now returns it when the active model has no pricing in the registry (for
|
|
90
|
+
example OpenRouter's `openrouter/auto` router model), instead of crashing
|
|
91
|
+
on a nil pricing entry.
|
|
92
|
+
|
|
93
|
+
### Skills
|
|
94
|
+
|
|
95
|
+
* **skills: add a `model` frontmatter parameter** <br>
|
|
96
|
+
A skill's `SKILL.md` frontmatter can now declare a `model:` value, so a
|
|
97
|
+
skill's sub-agent runs on a specific model instead of the default. This
|
|
98
|
+
lets a parent agent run on one model (for example `deepseek-v4-flash`)
|
|
99
|
+
while the skill runs on another (`deepseek-v4-pro`). The `model` value is
|
|
100
|
+
not strictly portable between providers.
|
|
101
|
+
|
|
102
|
+
* **skills: inherit the model of the parent agent** <br>
|
|
103
|
+
A skill's sub-agent now inherits the active model of the agent that
|
|
104
|
+
spawns it instead of falling back to the provider default. The frontmatter
|
|
105
|
+
`model:` parameter overrides it explicitly when set.
|
|
106
|
+
[`LLM::Context#model`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#model-instance_method)
|
|
107
|
+
now falls back to the provider's default model when no model is set.
|
|
108
|
+
|
|
109
|
+
### Stream
|
|
110
|
+
|
|
111
|
+
* **stream: rename `on_rate_limit` to `on_retry`** <br>
|
|
112
|
+
[`LLM::Stream#on_rate_limit`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_rate_limit-instance_method)
|
|
113
|
+
is renamed to
|
|
114
|
+
[`LLM::Stream#on_retry`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_retry-instance_method),
|
|
115
|
+
which fires when a request is retried after a rate limit or a timeout,
|
|
116
|
+
and now receives the one-based retry attempt number as a second argument.
|
|
117
|
+
The old `on_rate_limit` name remains as an alias.
|
|
118
|
+
|
|
119
|
+
### CLI
|
|
120
|
+
|
|
121
|
+
* **cli: add a `-m` switch for choosing the model** <br>
|
|
122
|
+
`bin/llm.rb` now accepts `-m MODEL` to run the session on a model other
|
|
123
|
+
than the provider default, for example
|
|
124
|
+
`llm.rb -p deepseek -m deepseek-v4-pro`. A model unknown to the provider's
|
|
125
|
+
registry prints an error and exits.
|
|
126
|
+
|
|
127
|
+
* **cli: add a `-x` switch for the read timeout** <br>
|
|
128
|
+
`bin/llm.rb` now accepts `-x SECONDS` to set the provider read timeout.
|
|
129
|
+
|
|
130
|
+
* **cli: print a backtrace on fatal crashes** <br>
|
|
131
|
+
When `bin/llm.rb` hits an unexpected error, the crash message now includes
|
|
132
|
+
up to three stack lines from the backtrace, so the failure is easier to
|
|
133
|
+
locate and report than a bare diagnostic.
|
|
134
|
+
|
|
135
|
+
### Repl
|
|
136
|
+
|
|
137
|
+
* **repl: show retry progress in the status bar** <br>
|
|
138
|
+
When a request is rate limited or times out, the curses-based REPL status
|
|
139
|
+
bar shows a retry indicator with the error and the remaining attempts, for
|
|
140
|
+
example `🔁 Rate limited • attempt 2 of 5`.
|
|
141
|
+
|
|
142
|
+
* **repl: measure text width with `unicode-display_width`** <br>
|
|
143
|
+
The curses-based REPL now counts and slices text by display column width
|
|
144
|
+
instead of character count, so wrapping, table columns, and clipping stay
|
|
145
|
+
aligned for wide characters such as emoji. It requires the optional
|
|
146
|
+
`unicode-display_width` gem.
|
|
147
|
+
|
|
148
|
+
* **repl: treat `LLM::InsufficientQuotaError` as a rate limit in the status bar** <br>
|
|
149
|
+
The curses-based REPL status bar now shows `Rate limited` when a request
|
|
150
|
+
raises
|
|
151
|
+
[`LLM::InsufficientQuotaError`](https://r.uby.dev/api-docs/llm.rb/LLM/InsufficientQuotaError.html),
|
|
152
|
+
matching how ordinary `LLM::RateLimitError`s are shown, instead of falling
|
|
153
|
+
through to the raw class name.
|
|
154
|
+
|
|
155
|
+
### Registry
|
|
156
|
+
|
|
157
|
+
* **refresh model metadata** <br>
|
|
158
|
+
Update `data/*.json` with current model listings and pricing, adding
|
|
159
|
+
GPT-5.6 Sol, Terra, and Luna models to Bedrock, Grok 4.6 and Grok Imagine
|
|
160
|
+
Image 2.0 to xAI, DeepSeek V4 Flash Vision to DeepSeek, and DeepSeek V4
|
|
161
|
+
Pro 0813, Qwen3 VL, and Qwen3.8 models to DeepInfra.
|
|
162
|
+
|
|
163
|
+
## v15.0.3
|
|
164
|
+
|
|
165
|
+
Changes since `v15.0.2`.
|
|
166
|
+
|
|
167
|
+
This release fixes ActiveRecord `:json`/`:jsonb` serialization so tool
|
|
168
|
+
call arguments round-trip as JSON objects instead of arrays of pairs.
|
|
169
|
+
|
|
170
|
+
### Fix
|
|
171
|
+
|
|
172
|
+
* **activerecord: serialize tool call arguments properly** <br>
|
|
173
|
+
Fix a bug where the ActiveRecord `:json`/`:jsonb` layer serialized tool
|
|
174
|
+
call arguments as an array of pairs instead of a Hash. The context is
|
|
175
|
+
now serialized through its JSON form, so tool call arguments round-trip
|
|
176
|
+
as JSON objects that providers accept.
|
|
177
|
+
|
|
18
178
|
## v15.0.2
|
|
19
179
|
|
|
20
180
|
Changes since `v15.0.1`.
|
|
@@ -1062,7 +1222,7 @@ reliable across all six concurrency backends. The `functions` and
|
|
|
1062
1222
|
`LLM.require` now accepts a second `version` parameter that is passed
|
|
1063
1223
|
to `Kernel#gem` before loading, enabling version constraints for
|
|
1064
1224
|
optional runtime dependencies. For example,
|
|
1065
|
-
`LLM.require "test-cmd.rb", "~>
|
|
1225
|
+
`LLM.require "test-cmd.rb", "~> 2.1"` ensures a minimum gem version
|
|
1066
1226
|
is available. This is used internally by the `Git`, `Rg`, `Mkdir`,
|
|
1067
1227
|
and `Shell` tools to enforce compatibility with the `test-cmd.rb` gem.
|
|
1068
1228
|
|
data/README.md
CHANGED
|
@@ -15,22 +15,16 @@
|
|
|
15
15
|
Welcome to the canonical llm.rb repository.
|
|
16
16
|
|
|
17
17
|
llm.rb is an advanced runtime for building agentic AI applications
|
|
18
|
-
on CRuby. It has zero runtime dependencies by default,
|
|
18
|
+
on CRuby. It has zero runtime dependencies by default, supports
|
|
19
19
|
concurrent and parallel tool execution and has a single coherent API
|
|
20
|
-
that spans
|
|
20
|
+
that spans 14+ providers. Streaming, tools, guards, compaction, the
|
|
21
21
|
REPL, builtin MCP/A2A support and the database integrations all build
|
|
22
22
|
on the same three concepts: providers, contexts, and agents.
|
|
23
23
|
|
|
24
|
-
|
|
25
|
-
naturally. Some features, such as ActiveRecord support, require
|
|
26
|
-
optional dependencies that are opt-in.
|
|
27
|
-
|
|
28
|
-
The best way to learn about llm.rb is to ask [the official chatbot](https://r.uby.dev)
|
|
24
|
+
The easiest way to learn about llm.rb is to ask [the r.uby.dev chatbot](https://r.uby.dev)
|
|
29
25
|
a question. It is connected to the llm.rb GitHub repository, backed by
|
|
30
|
-
ActiveRecord and uses the builtin MCP feature to connect to GitHub.
|
|
31
|
-
|
|
32
|
-
you should get an answer that is grounded in the source code. It is 100%
|
|
33
|
-
free to use and powered by DeepSeek under the hood.
|
|
26
|
+
ActiveRecord and uses the builtin MCP feature to connect to GitHub. All
|
|
27
|
+
answers are grounded in the llm.rb source code.
|
|
34
28
|
|
|
35
29
|
## Install
|
|
36
30
|
|
|
@@ -46,8 +40,7 @@ The
|
|
|
46
40
|
[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
|
|
47
41
|
class is the default high-level interface,
|
|
48
42
|
and it is recommended for most use-cases. It manages tool execution
|
|
49
|
-
automatically and
|
|
50
|
-
[guards against infinite loops](https://r.uby.dev/llm/deepdive/advanced/guard),
|
|
43
|
+
automatically and guards against infinite loops,
|
|
51
44
|
manages conversation state, and much more.
|
|
52
45
|
|
|
53
46
|
```ruby
|
|
@@ -68,9 +61,6 @@ reasoning, tool calls, tool returns, and compaction.
|
|
|
68
61
|
Streams can also observe message transformers, which rewrite
|
|
69
62
|
outgoing messages before they reach the provider.
|
|
70
63
|
|
|
71
|
-
See the [deepdive.md](https://r.uby.dev/llm/deepdive/fundamentals/stream/)
|
|
72
|
-
to learn more.
|
|
73
|
-
|
|
74
64
|
```ruby
|
|
75
65
|
class MyStream < LLM::Stream
|
|
76
66
|
# Visible assistant output.
|
|
@@ -118,8 +108,8 @@ class MyStream < LLM::Stream
|
|
|
118
108
|
def on_skill_return(agent, skill, result)
|
|
119
109
|
end
|
|
120
110
|
|
|
121
|
-
# A request was rate limited and will be retried.
|
|
122
|
-
def
|
|
111
|
+
# A request was rate limited or timed out and will be retried.
|
|
112
|
+
def on_retry(error, attempt)
|
|
123
113
|
end
|
|
124
114
|
end
|
|
125
115
|
|
|
@@ -140,7 +130,7 @@ call them on your behalf, and they're one of the most powerful features
|
|
|
140
130
|
for extending the feature set or abilities of a model.
|
|
141
131
|
|
|
142
132
|
The runtime also ships with a catalog of built-in tools for
|
|
143
|
-
filesystem, search, and shell operations.
|
|
133
|
+
filesystem, search, and shell operations.
|
|
144
134
|
|
|
145
135
|
```ruby
|
|
146
136
|
class ReadFile < LLM::Tool
|
|
@@ -175,8 +165,6 @@ callback hands back the subagent that ran the skill, so you can inspect
|
|
|
175
165
|
its conversation, measure its usage, track costs or add a verification
|
|
176
166
|
step (eg `subagent.talk("verify your work")`).
|
|
177
167
|
|
|
178
|
-
See the [deepdive.md](https://r.uby.dev/llm/deepdive/fundamentals/skills) to learn more.
|
|
179
|
-
|
|
180
168
|
##### summary.md
|
|
181
169
|
|
|
182
170
|
```markdown
|
|
@@ -216,7 +204,11 @@ with the `:fork` and `:ractor` strategies. The
|
|
|
216
204
|
The `:fork` strategy also provides a separate process that offers
|
|
217
205
|
isolation from its parent.
|
|
218
206
|
|
|
219
|
-
|
|
207
|
+
A couple of concurrency strategies require optional, opt-in dependencies.
|
|
208
|
+
The `async` strategy requires the [async](https://github.com/socketry/async)
|
|
209
|
+
gem and the `fork` strategy requires the [xchan.rb](https://github.com/0x1eef/xchan.rb)
|
|
210
|
+
gem. The `fiber` strategy requires a scheduler (`Fiber.scheduler`) but by
|
|
211
|
+
default Ruby does not provide one.
|
|
220
212
|
|
|
221
213
|
```ruby
|
|
222
214
|
require "llm"
|
|
@@ -242,8 +234,6 @@ the control channel, a ractor via message passing, and pending tools
|
|
|
242
234
|
are stopped before they run. The in-flight HTTP request is closed
|
|
243
235
|
too, so a turn you no longer want stops without burning tokens.
|
|
244
236
|
|
|
245
|
-
See the [deepdive.md](https://r.uby.dev/llm/deepdive/advanced/cancellation) to learn more.
|
|
246
|
-
|
|
247
237
|
```ruby
|
|
248
238
|
llm = LLM.deepseek(key: ENV["KEY"])
|
|
249
239
|
agent = LLM::Agent.new(llm)
|
|
@@ -268,8 +258,6 @@ The REPL stands out because it connects to the surrounding
|
|
|
268
258
|
runtime and it can be extended by your code. Think of it as
|
|
269
259
|
`binding.pry` but for agents.
|
|
270
260
|
|
|
271
|
-
See the [deepdive.md](https://r.uby.dev/llm/deepdive/features/repl) to learn more.
|
|
272
|
-
|
|
273
261
|
##### Demo
|
|
274
262
|
|
|
275
263
|
[Watch in high quality on asciinema](https://asciinema.org/a/OsS8wwaasKasoDDz)
|
|
@@ -284,7 +272,7 @@ a separate gem but it requires a number of optional dependencies
|
|
|
284
272
|
to be installed separately. The following gems provide the full
|
|
285
273
|
experience:
|
|
286
274
|
|
|
287
|
-
gem install curses kramdown xchan.rb test-cmd.rb
|
|
275
|
+
gem install unicode-display_width curses kramdown xchan.rb test-cmd.rb
|
|
288
276
|
|
|
289
277
|
##### Persistence
|
|
290
278
|
|
|
@@ -327,8 +315,6 @@ code. For database-backed persistence, ActiveRecord and Sequel
|
|
|
327
315
|
integrations are also available. All persistence options use the same
|
|
328
316
|
underlying serialization.
|
|
329
317
|
|
|
330
|
-
See the [deepdive.md](https://r.uby.dev/llm/deepdive/features/database) to learn more.
|
|
331
|
-
|
|
332
318
|
```ruby
|
|
333
319
|
require "llm"
|
|
334
320
|
|
|
@@ -355,55 +341,90 @@ for both Rack-based / Rails-based applications. On databases
|
|
|
355
341
|
where it is supported, such as PostgreSQL, the column can be optimized by using
|
|
356
342
|
the `jsonb` type.
|
|
357
343
|
|
|
358
|
-
|
|
344
|
+
The following example is based on the agent used to power the
|
|
345
|
+
[r.uby.dev chatbot](https://r.uby.dev).
|
|
359
346
|
|
|
360
347
|
```ruby
|
|
361
348
|
require "active_record"
|
|
362
349
|
require "llm"
|
|
363
350
|
require "llm/active_record"
|
|
364
351
|
|
|
365
|
-
class
|
|
366
|
-
acts_as_agent do |agent|
|
|
367
|
-
agent.set name: "
|
|
368
|
-
|
|
369
|
-
|
|
352
|
+
class Raven < ActiveRecord::Base
|
|
353
|
+
acts_as_agent(format: :jsonb) do |agent|
|
|
354
|
+
agent.set name: "raven",
|
|
355
|
+
description: "a chatbot for the r.uby.dev website",
|
|
356
|
+
instructions: proc { File.read(File.join(__dir__, "raven", "prompt.md")) },
|
|
357
|
+
tools: :tools,
|
|
358
|
+
concurrency: :async
|
|
370
359
|
end
|
|
371
360
|
|
|
372
|
-
def
|
|
373
|
-
talk("
|
|
361
|
+
def research_issues
|
|
362
|
+
talk("research open pull requests on r-uby-dev/llm")
|
|
374
363
|
end
|
|
375
364
|
|
|
376
|
-
def
|
|
377
|
-
talk("
|
|
365
|
+
def research_codebase
|
|
366
|
+
talk("research the codebase on r-uby-dev/llm")
|
|
378
367
|
end
|
|
379
368
|
|
|
380
|
-
|
|
369
|
+
##
|
|
370
|
+
# @return [LLM::MCP]
|
|
371
|
+
def github
|
|
372
|
+
@github ||= LLM::MCP.http(
|
|
373
|
+
url: "https://api.githubcopilot.com/mcp/",
|
|
374
|
+
headers: {"Authorization" => "Bearer #{ENV['GITHUB_RUBYDEV_PAT']}"},
|
|
375
|
+
transport: :net_http_persistent
|
|
376
|
+
)
|
|
377
|
+
end
|
|
381
378
|
|
|
382
379
|
##
|
|
383
|
-
#
|
|
384
|
-
|
|
380
|
+
# @return [Array<LLM::Tool>]
|
|
381
|
+
def tools
|
|
382
|
+
github.tools.select { allowlist.include?(_1.name.to_s) }
|
|
383
|
+
end
|
|
384
|
+
|
|
385
|
+
private
|
|
386
|
+
|
|
385
387
|
def set_provider
|
|
386
|
-
LLM.deepseek
|
|
388
|
+
LLM.deepseek
|
|
387
389
|
end
|
|
388
390
|
|
|
389
|
-
|
|
390
|
-
|
|
391
|
-
|
|
392
|
-
|
|
393
|
-
|
|
391
|
+
def allowlist
|
|
392
|
+
%w[
|
|
393
|
+
get_commit
|
|
394
|
+
get_file_contents
|
|
395
|
+
list_branches
|
|
396
|
+
list_commits
|
|
397
|
+
search_code
|
|
398
|
+
search_commits
|
|
399
|
+
search_repositories
|
|
400
|
+
search_issues
|
|
401
|
+
pull_request_read
|
|
402
|
+
list_pull_requests
|
|
403
|
+
list_issues
|
|
404
|
+
issue_read
|
|
405
|
+
].freeze
|
|
394
406
|
end
|
|
395
407
|
end
|
|
396
408
|
|
|
397
|
-
|
|
398
|
-
|
|
409
|
+
agent = Raven.create!
|
|
410
|
+
|
|
411
|
+
##
|
|
412
|
+
# Every call to `talk` automatically persists
|
|
413
|
+
# to the database (under the hood research_issues
|
|
414
|
+
# calls the talk method)
|
|
415
|
+
agent.research_issues
|
|
416
|
+
|
|
417
|
+
##
|
|
418
|
+
# The conversation was persisted to database. A
|
|
419
|
+
# fresh instance restores it and continues where
|
|
420
|
+
# we left off
|
|
421
|
+
agent = Raven.find(agent.id).tap(&:research_codebase)
|
|
399
422
|
|
|
400
423
|
##
|
|
401
|
-
#
|
|
402
|
-
#
|
|
403
|
-
#
|
|
404
|
-
|
|
405
|
-
# already drafted:
|
|
406
|
-
Email.find(email.id).summarize
|
|
424
|
+
# Start an agent console.
|
|
425
|
+
# Query agent's state, debug, etc.
|
|
426
|
+
# The REPL does not persist back to the database.
|
|
427
|
+
agent.repl
|
|
407
428
|
```
|
|
408
429
|
</details>
|
|
409
430
|
|
|
@@ -418,9 +439,6 @@ used with
|
|
|
418
439
|
[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html) or
|
|
419
440
|
[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html).
|
|
420
441
|
|
|
421
|
-
See the [deepdive.md](https://r.uby.dev/llm/deepdive/protocols/mcp/), and the
|
|
422
|
-
[deepdive.md on persistent connections](https://r.uby.dev/llm/deepdive/features/transports) to learn more.
|
|
423
|
-
|
|
424
442
|
```ruby
|
|
425
443
|
require "llm"
|
|
426
444
|
|
|
@@ -441,9 +459,6 @@ used with
|
|
|
441
459
|
[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html) or
|
|
442
460
|
[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html).
|
|
443
461
|
|
|
444
|
-
See the [deepdive.md](https://r.uby.dev/llm/deepdive/protocols/a2a/), and the
|
|
445
|
-
[deepdive.md on persistent connections](https://r.uby.dev/llm/deepdive/features/transports) to learn more.
|
|
446
|
-
|
|
447
462
|
```ruby
|
|
448
463
|
require "llm"
|
|
449
464
|
|
|
@@ -516,8 +531,6 @@ and implement
|
|
|
516
531
|
The pending call arrives as `function:`. Return a value to close
|
|
517
532
|
the call, or `nil` to let it run:
|
|
518
533
|
|
|
519
|
-
See the [deepdive.md](https://r.uby.dev/llm/deepdive/advanced/guard) to learn more.
|
|
520
|
-
|
|
521
534
|
```ruby
|
|
522
535
|
class PolicyGuard < LLM::Guard
|
|
523
536
|
def call(function:)
|
|
@@ -543,8 +556,6 @@ Create a subclass and implement `call(message:)` to scrub sensitive data,
|
|
|
543
556
|
inject context, or normalize content. The transform runs automatically
|
|
544
557
|
on every turn, so you never have to change your prompt code.
|
|
545
558
|
|
|
546
|
-
See the [deepdive.md](https://r.uby.dev/llm/deepdive/advanced/transformer) to learn more.
|
|
547
|
-
|
|
548
559
|
```ruby
|
|
549
560
|
class RedactEmails < LLM::Transformer
|
|
550
561
|
def call(message:)
|
|
@@ -581,8 +592,6 @@ and
|
|
|
581
592
|
[`LLM::Stream#on_compaction_finish`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_compaction_finish)
|
|
582
593
|
callbacks.
|
|
583
594
|
|
|
584
|
-
See the [deepdive.md](https://r.uby.dev/llm/deepdive/advanced/compaction) to learn more.
|
|
585
|
-
|
|
586
595
|
```ruby
|
|
587
596
|
llm = LLM.deepseek(key: ENV["KEY"])
|
|
588
597
|
agent = LLM::Agent.new(
|
|
@@ -600,7 +609,8 @@ agent.talk "Hello"
|
|
|
600
609
|
|
|
601
610
|
Rate-limited requests are retried automatically by default. Agents
|
|
602
611
|
retry a 429 up to five times with a growing backoff before giving
|
|
603
|
-
up, so most request failures resolve on their own.
|
|
612
|
+
up, so most request failures resolve on their own. Connection and
|
|
613
|
+
read timeouts are retried the same way. Set `retry_budget`
|
|
604
614
|
to change the number of retries, or `retry_budget: 0` to disable
|
|
605
615
|
them.
|
|
606
616
|
|
|
@@ -631,8 +641,6 @@ exports spans via OTLP for OpenTelemetry in production.
|
|
|
631
641
|
* [`LLM::Tracer::Logger`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/Logger.html):
|
|
632
642
|
structured JSON to stdout or a file.
|
|
633
643
|
|
|
634
|
-
See the [deepdive.md](https://r.uby.dev/llm/deepdive/reference/tracer) to learn more.
|
|
635
|
-
|
|
636
644
|
```ruby
|
|
637
645
|
llm = LLM.deepseek(key: ENV["KEY"])
|
|
638
646
|
agent = LLM::Agent.new(llm, tracer: LLM::Tracer::PrettyLogger.new(llm))
|
|
@@ -676,9 +684,8 @@ Each provider is constructed with a class-level factory method on
|
|
|
676
684
|
[`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
|
|
677
685
|
or
|
|
678
686
|
[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html). The
|
|
679
|
-
same API drives every one of them, so switching
|
|
680
|
-
change.
|
|
681
|
-
for a full provider reference.
|
|
687
|
+
same API drives every one of them, so switching providers is a one-line
|
|
688
|
+
change.
|
|
682
689
|
|
|
683
690
|
#### What providers does llm.rb support?
|
|
684
691
|
|
|
@@ -690,6 +697,7 @@ for a full provider reference.
|
|
|
690
697
|
* **xAI** (`LLM.xai`)
|
|
691
698
|
* **Z.ai** (`LLM.zai`)
|
|
692
699
|
* **Moonshot (Kimi)** (`LLM.moonshot`)
|
|
700
|
+
* **OpenRouter** (`LLM.openrouter`)
|
|
693
701
|
* **Alibaba (Qwen3)** (`LLM.alibaba`, also `LLM.aliyun`)
|
|
694
702
|
* **Mistral** (`LLM.mistral`)
|
|
695
703
|
* **AWS Bedrock** (`LLM.bedrock`)
|
|
@@ -711,6 +719,7 @@ llm = LLM.anthropic
|
|
|
711
719
|
llm = LLM.deepseek
|
|
712
720
|
llm = LLM.alibaba # also: LLM.aliyun
|
|
713
721
|
llm = LLM.moonshot
|
|
722
|
+
llm = LLM.openrouter
|
|
714
723
|
llm = LLM.mistral
|
|
715
724
|
```
|
|
716
725
|
</details>
|
|
@@ -728,6 +737,7 @@ llm = LLM.anthropic(key: ENV["ANTHROPIC_API_KEY"])
|
|
|
728
737
|
llm = LLM.deepseek(key: ENV["DEEPSEEK_API_KEY"])
|
|
729
738
|
llm = LLM.alibaba(key: ENV["DASHSCOPE_API_KEY"]) # also: LLM.aliyun
|
|
730
739
|
llm = LLM.moonshot(key: ENV["MOONSHOT_API_KEY"])
|
|
740
|
+
llm = LLM.openrouter(key: ENV["OPENROUTER_API_KEY"])
|
|
731
741
|
llm = LLM.mistral(key: ENV["MISTRAL_API_KEY"])
|
|
732
742
|
```
|
|
733
743
|
</details>
|
|
@@ -741,8 +751,6 @@ modalities with the gem, sourced from [models.dev](https://models.dev).
|
|
|
741
751
|
Reach it from any provider, context, or agent, enumerate models, or
|
|
742
752
|
sort them by price.
|
|
743
753
|
|
|
744
|
-
See the [deepdive.md](https://r.uby.dev/llm/deepdive/reference/model_registry) to learn more.
|
|
745
|
-
|
|
746
754
|
```ruby
|
|
747
755
|
require "llm"
|
|
748
756
|
|
|
@@ -765,8 +773,6 @@ is always available and the default, `net/http/persistent` pools
|
|
|
765
773
|
connections for many requests to the same host, and `curb` wraps
|
|
766
774
|
libcurl. They share one interface, so switching is a one-word change.
|
|
767
775
|
|
|
768
|
-
See the [deepdive.md](https://r.uby.dev/llm/deepdive/advanced/transports) to learn more.
|
|
769
|
-
|
|
770
776
|
```ruby
|
|
771
777
|
llm = LLM.deepseek(
|
|
772
778
|
key: ENV["KEY"],
|
|
@@ -775,6 +781,51 @@ llm = LLM.deepseek(
|
|
|
775
781
|
```
|
|
776
782
|
</details>
|
|
777
783
|
|
|
784
|
+
<details>
|
|
785
|
+
<summary>Timeouts</summary>
|
|
786
|
+
<br>
|
|
787
|
+
|
|
788
|
+
Providers accept two timeouts:
|
|
789
|
+
|
|
790
|
+
* `connect_timeout` - opening the connection. Defaults to 5 seconds.
|
|
791
|
+
* `read_timeout` - waiting for a response on an idle connection.
|
|
792
|
+
Defaults to 600 seconds (10 minutes).
|
|
793
|
+
|
|
794
|
+
The longer read timeout leaves room for slow reasoning models and
|
|
795
|
+
local models. The legacy `timeout:` option remains as a shorthand for
|
|
796
|
+
`read_timeout`. Timeouts are retriable:
|
|
797
|
+
[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
|
|
798
|
+
retries a timed out request up to its
|
|
799
|
+
[`retry_budget`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#retry_budget-class_method)
|
|
800
|
+
(five by default), so a dropped connection or a slow first token
|
|
801
|
+
is often something we can recover from.
|
|
802
|
+
|
|
803
|
+
```ruby
|
|
804
|
+
llm = LLM.deepseek(
|
|
805
|
+
connect_timeout: 5, # opening the connection
|
|
806
|
+
read_timeout: 600 # waiting for the next bytes
|
|
807
|
+
)
|
|
808
|
+
```
|
|
809
|
+
</details>
|
|
810
|
+
|
|
811
|
+
<details>
|
|
812
|
+
<summary>Headers</summary>
|
|
813
|
+
<br>
|
|
814
|
+
|
|
815
|
+
Providers can accept a custom set of headers with
|
|
816
|
+
the [`LLM::Provider#with`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#with-instance_method) method.
|
|
817
|
+
For example, you could set a custom User-Agent header,
|
|
818
|
+
or provide headers that carry special meaning to
|
|
819
|
+
certain providers (eg OpenAI, OpenRouter).
|
|
820
|
+
|
|
821
|
+
```ruby
|
|
822
|
+
llm = LLM.openrouter
|
|
823
|
+
llm = llm.with("HTTP-Referer" => "https://example.com")
|
|
824
|
+
llm = llm.with("X-OpenRouter-Title" => "Example App")
|
|
825
|
+
```
|
|
826
|
+
|
|
827
|
+
</details>
|
|
828
|
+
|
|
778
829
|
### RAG
|
|
779
830
|
|
|
780
831
|
Most providers offer an embedding model that can be
|
|
@@ -788,8 +839,6 @@ llm.rb also includes support for OpenAI's vector store API. It
|
|
|
788
839
|
provides a vector database as a HTTP service but we won't cover
|
|
789
840
|
that here.
|
|
790
841
|
|
|
791
|
-
See the [deepdive.md](https://r.uby.dev/llm/deepdive/features/embeddings) to learn more.
|
|
792
|
-
|
|
793
842
|
```ruby
|
|
794
843
|
require "llm"
|
|
795
844
|
|
|
@@ -928,11 +977,11 @@ it as my primary driver for development.
|
|
|
928
977
|
|
|
929
978
|
## Resources
|
|
930
979
|
|
|
931
|
-
|
|
932
|
-
to
|
|
933
|
-
|
|
934
|
-
|
|
935
|
-
|
|
980
|
+
The [r.uby.dev chatbot](https://r.uby.dev) is connected
|
|
981
|
+
to this very GitHub repository. It can read documentation,
|
|
982
|
+
source code, issues, and pull requests. The [docs/](docs/)
|
|
983
|
+
directory contains the full documentation and the chatbot
|
|
984
|
+
can find the answers to your questions there.
|
|
936
985
|
|
|
937
986
|
## License
|
|
938
987
|
|