llm.rb 15.0.2 → 15.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (50) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +161 -1
  3. data/README.md +134 -85
  4. data/bin/llm.rb +38 -3
  5. data/data/bedrock.json +250 -0
  6. data/data/deepinfra.json +162 -0
  7. data/data/deepseek.json +50 -0
  8. data/data/google.json +19 -21
  9. data/data/openrouter.json +14280 -0
  10. data/data/xai.json +28 -16
  11. data/docs/deepdive/advanced/context.md +8 -6
  12. data/docs/deepdive/fundamentals/agents.md +11 -10
  13. data/docs/deepdive/fundamentals/stream.md +4 -4
  14. data/lib/llm/active_record.rb +1 -1
  15. data/lib/llm/agent.rb +6 -0
  16. data/lib/llm/context.rb +28 -13
  17. data/lib/llm/cost.rb +13 -0
  18. data/lib/llm/function/fork/task.rb +14 -10
  19. data/lib/llm/provider.rb +29 -8
  20. data/lib/llm/providers/anthropic.rb +1 -1
  21. data/lib/llm/providers/bedrock/models.rb +2 -2
  22. data/lib/llm/providers/bedrock.rb +1 -1
  23. data/lib/llm/providers/google.rb +1 -1
  24. data/lib/llm/providers/ollama.rb +1 -1
  25. data/lib/llm/providers/openai/responses.rb +2 -1
  26. data/lib/llm/providers/openai.rb +1 -1
  27. data/lib/llm/providers/openrouter.rb +87 -0
  28. data/lib/llm/repl/buffer.rb +20 -5
  29. data/lib/llm/repl/input.rb +5 -4
  30. data/lib/llm/repl/markdown/table.rb +5 -2
  31. data/lib/llm/repl/node.rb +26 -1
  32. data/lib/llm/repl/stream.rb +30 -3
  33. data/lib/llm/repl/window.rb +7 -7
  34. data/lib/llm/repl.rb +1 -0
  35. data/lib/llm/skill.rb +7 -1
  36. data/lib/llm/stream.rb +8 -3
  37. data/lib/llm/tools/git.rb +2 -2
  38. data/lib/llm/tools/mkdir.rb +2 -2
  39. data/lib/llm/tools/rg.rb +2 -2
  40. data/lib/llm/tools/ruby.rb +2 -2
  41. data/lib/llm/tools/shell.rb +2 -2
  42. data/lib/llm/tools/utils.rb +1 -1
  43. data/lib/llm/transport/curb.rb +5 -3
  44. data/lib/llm/transport/http.rb +5 -2
  45. data/lib/llm/transport/persistent_http.rb +6 -4
  46. data/lib/llm/transport/utils.rb +8 -6
  47. data/lib/llm/version.rb +1 -1
  48. data/lib/llm.rb +16 -1
  49. data/llm.gemspec +2 -1
  50. metadata +19 -3
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: c86e9ccde2cc0844b5172935bc222c1adf3cda8b0f49e84730179bcc635b2d2b
4
- data.tar.gz: 60c740c3dc2c20d8b1d86de55ae5ef25eb7bbe5b5b567f3766c059b37520efcc
3
+ metadata.gz: 2b496d3aec5a309af0ff9eaac32cabb6c7191bf2bac1f3b664c02a56c593402e
4
+ data.tar.gz: 0ae83d7794b4dad881de7b9866b3894746efa1853e349fc5707416e367802db5
5
5
  SHA512:
6
- metadata.gz: fd1901c5bebd659b242db55ed91ea4f538832c24b4ab119b80f46ae987fc5166823fa70174d9f545c079addd90487eb379b1dfd6eeea2365d045a831b861134e
7
- data.tar.gz: 5949efbaa7381fb0cab7328e64c08af4022a8b976a24b8b24901c167c67cd060f912a7054956a7942cd7cf67b3f0aa226ad5aec608bcd273bfa1aa25d3ffb5d0
6
+ metadata.gz: 9f890dd0720d0acbc79f9df2196f3595a68cd618fd0d09dc02fcab9a6fe27758ae72efd4b764614d5511bdb7914f586fba7194b3d1a5bec186562b9196753783
7
+ data.tar.gz: f25bc47b9bd5dcb0afea4f2ddbcb1ad37f74863cff962de6ceefa6157af073997589fd08fad0b2df11288aa24db41685127542b4d1748a71c4fef607d6b76ba2
data/CHANGELOG.md CHANGED
@@ -15,6 +15,166 @@
15
15
 
16
16
  ## What's next
17
17
 
18
+ ## v15.1.0
19
+
20
+ Changes since `v15.0.3`.
21
+
22
+ This release adds the OpenRouter provider, splits provider timeouts
23
+ into `connect_timeout` and `read_timeout`, retries timed-out requests,
24
+ and renames `on_rate_limit` to `on_retry`. Skills now gain a
25
+ frontmatter `model:` parameter and inherit the parent agent's model,
26
+ while the CLI gains `-m` and `-x` switches, and the REPL shows retry
27
+ progress and measures text by display width.
28
+
29
+ ### Provider
30
+
31
+ * **add `LLM::OpenRouter` for the OpenRouter provider** <br>
32
+ [`LLM::OpenRouter`](https://r.uby.dev/api-docs/llm.rb/LLM/OpenRouter.html)
33
+ is a new provider that talks to [OpenRouter](https://openrouter.ai)
34
+ through its OpenAI-compatible API, contributed via
35
+ [PR #165](https://github.com/r-uby-dev/llm.rb/pull/165). Create an
36
+ instance with
37
+ [`LLM.openrouter`](https://r.uby.dev/api-docs/llm.rb/LLM.html#openrouter-class_method),
38
+ which accepts the same `key:`, `host:`, and `base_path:` options as the
39
+ OpenAI provider. It defaults to the `openrouter/auto` router model and
40
+ supports chat completions, streaming, tool calls, structured output,
41
+ and embeddings; image, audio, moderation, files, and vector store
42
+ endpoints raise `NotImplementedError`. Model metadata ships in
43
+ `data/openrouter.json` for the registry.
44
+
45
+ * **provider: `LLM::Provider#with` accepts headers without the `headers:` keyword** <br>
46
+ [`LLM::Provider#with`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#with-instance_method)
47
+ now accepts headers directly as a Hash (`llm.with("User-Agent" => "llmrb/1.0")`)
48
+ in addition to the legacy `headers:` keyword form. Both are merged, and
49
+ the keyword form remains for backwards compatibility.
50
+
51
+ * **providers: `model: nil` falls back to `default_model`** <br>
52
+ [`LLM::Provider`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html)
53
+ now treats a `model: nil` payload as "use the default model" instead of
54
+ sending a model value that providers reject. Previously a `{model: nil}`
55
+ param overwrote the default and then was removed by `.compact`, leaving
56
+ undefined behavior where providers could reject the request.
57
+
58
+ * **openai: handle `model: nil` in the Responses API** <br>
59
+ The OpenAI Responses API path now also falls back to the default model
60
+ when `model: nil`, matching the completions path.
61
+
62
+ * **provider: separate `connect_timeout` and `read_timeout`** <br>
63
+ [`LLM::Provider`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html)
64
+ now accepts a `connect_timeout:` for opening the connection (default 5
65
+ seconds) and a `read_timeout:` for waiting on an idle connection (default
66
+ 600 seconds), instead of a single `timeout`. The legacy `timeout:` option
67
+ remains as a shorthand for `read_timeout`. The provider exposes the new
68
+ `#read_timeout` and `#connect_timeout` accessors.
69
+
70
+ * **provider: retry timed-out requests** <br>
71
+ Requests that time out (`Net::OpenTimeout` or `Net::ReadTimeout`) are now
72
+ retried along with rate-limited requests, up to the `retry_budget`. When
73
+ a heavily loaded provider (for example DeepSeek) drops the connection
74
+ during the connect or read phase, the request is retried instead of
75
+ failing, and the stream is notified through `on_retry`.
76
+
77
+ * **context: retry on `LLM::InsufficientQuotaError`, too** <br>
78
+ A request that raises
79
+ [`LLM::InsufficientQuotaError`](https://r.uby.dev/api-docs/llm.rb/LLM/InsufficientQuotaError.html)
80
+ is now retried like other rate-limited requests. The error is a subclass
81
+ of `LLM::RateLimitError` and can be raised at regular intervals, in
82
+ particular by the Alibaba provider.
83
+
84
+ * **cost: return `LLM::Cost.zero` when the registry has no pricing** <br>
85
+ Add
86
+ [`LLM::Cost.zero`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html#zero-class_method),
87
+ a factory for a zero-valued cost breakdown.
88
+ [`LLM::Context#cost`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#cost-instance_method)
89
+ now returns it when the active model has no pricing in the registry (for
90
+ example OpenRouter's `openrouter/auto` router model), instead of crashing
91
+ on a nil pricing entry.
92
+
93
+ ### Skills
94
+
95
+ * **skills: add a `model` frontmatter parameter** <br>
96
+ A skill's `SKILL.md` frontmatter can now declare a `model:` value, so a
97
+ skill's sub-agent runs on a specific model instead of the default. This
98
+ lets a parent agent run on one model (for example `deepseek-v4-flash`)
99
+ while the skill runs on another (`deepseek-v4-pro`). The `model` value is
100
+ not strictly portable between providers.
101
+
102
+ * **skills: inherit the model of the parent agent** <br>
103
+ A skill's sub-agent now inherits the active model of the agent that
104
+ spawns it instead of falling back to the provider default. The frontmatter
105
+ `model:` parameter overrides it explicitly when set.
106
+ [`LLM::Context#model`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#model-instance_method)
107
+ now falls back to the provider's default model when no model is set.
108
+
109
+ ### Stream
110
+
111
+ * **stream: rename `on_rate_limit` to `on_retry`** <br>
112
+ [`LLM::Stream#on_rate_limit`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_rate_limit-instance_method)
113
+ is renamed to
114
+ [`LLM::Stream#on_retry`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_retry-instance_method),
115
+ which fires when a request is retried after a rate limit or a timeout,
116
+ and now receives the one-based retry attempt number as a second argument.
117
+ The old `on_rate_limit` name remains as an alias.
118
+
119
+ ### CLI
120
+
121
+ * **cli: add a `-m` switch for choosing the model** <br>
122
+ `bin/llm.rb` now accepts `-m MODEL` to run the session on a model other
123
+ than the provider default, for example
124
+ `llm.rb -p deepseek -m deepseek-v4-pro`. A model unknown to the provider's
125
+ registry prints an error and exits.
126
+
127
+ * **cli: add a `-x` switch for the read timeout** <br>
128
+ `bin/llm.rb` now accepts `-x SECONDS` to set the provider read timeout.
129
+
130
+ * **cli: print a backtrace on fatal crashes** <br>
131
+ When `bin/llm.rb` hits an unexpected error, the crash message now includes
132
+ up to three stack lines from the backtrace, so the failure is easier to
133
+ locate and report than a bare diagnostic.
134
+
135
+ ### Repl
136
+
137
+ * **repl: show retry progress in the status bar** <br>
138
+ When a request is rate limited or times out, the curses-based REPL status
139
+ bar shows a retry indicator with the error and the remaining attempts, for
140
+ example `🔁 Rate limited • attempt 2 of 5`.
141
+
142
+ * **repl: measure text width with `unicode-display_width`** <br>
143
+ The curses-based REPL now counts and slices text by display column width
144
+ instead of character count, so wrapping, table columns, and clipping stay
145
+ aligned for wide characters such as emoji. It requires the optional
146
+ `unicode-display_width` gem.
147
+
148
+ * **repl: treat `LLM::InsufficientQuotaError` as a rate limit in the status bar** <br>
149
+ The curses-based REPL status bar now shows `Rate limited` when a request
150
+ raises
151
+ [`LLM::InsufficientQuotaError`](https://r.uby.dev/api-docs/llm.rb/LLM/InsufficientQuotaError.html),
152
+ matching how ordinary `LLM::RateLimitError`s are shown, instead of falling
153
+ through to the raw class name.
154
+
155
+ ### Registry
156
+
157
+ * **refresh model metadata** <br>
158
+ Update `data/*.json` with current model listings and pricing, adding
159
+ GPT-5.6 Sol, Terra, and Luna models to Bedrock, Grok 4.6 and Grok Imagine
160
+ Image 2.0 to xAI, DeepSeek V4 Flash Vision to DeepSeek, and DeepSeek V4
161
+ Pro 0813, Qwen3 VL, and Qwen3.8 models to DeepInfra.
162
+
163
+ ## v15.0.3
164
+
165
+ Changes since `v15.0.2`.
166
+
167
+ This release fixes ActiveRecord `:json`/`:jsonb` serialization so tool
168
+ call arguments round-trip as JSON objects instead of arrays of pairs.
169
+
170
+ ### Fix
171
+
172
+ * **activerecord: serialize tool call arguments properly** <br>
173
+ Fix a bug where the ActiveRecord `:json`/`:jsonb` layer serialized tool
174
+ call arguments as an array of pairs instead of a Hash. The context is
175
+ now serialized through its JSON form, so tool call arguments round-trip
176
+ as JSON objects that providers accept.
177
+
18
178
  ## v15.0.2
19
179
 
20
180
  Changes since `v15.0.1`.
@@ -1062,7 +1222,7 @@ reliable across all six concurrency backends. The `functions` and
1062
1222
  `LLM.require` now accepts a second `version` parameter that is passed
1063
1223
  to `Kernel#gem` before loading, enabling version constraints for
1064
1224
  optional runtime dependencies. For example,
1065
- `LLM.require "test-cmd.rb", "~> 1.1"` ensures a minimum gem version
1225
+ `LLM.require "test-cmd.rb", "~> 2.1"` ensures a minimum gem version
1066
1226
  is available. This is used internally by the `Git`, `Rg`, `Mkdir`,
1067
1227
  and `Shell` tools to enforce compatibility with the `test-cmd.rb` gem.
1068
1228
 
data/README.md CHANGED
@@ -15,22 +15,16 @@
15
15
  Welcome to the canonical llm.rb repository.
16
16
 
17
17
  llm.rb is an advanced runtime for building agentic AI applications
18
- on CRuby. It has zero runtime dependencies by default, it supports
18
+ on CRuby. It has zero runtime dependencies by default, supports
19
19
  concurrent and parallel tool execution and has a single coherent API
20
- that spans 13+ providers. Streaming, tools, guards, compaction, the
20
+ that spans 14+ providers. Streaming, tools, guards, compaction, the
21
21
  REPL, builtin MCP/A2A support and the database integrations all build
22
22
  on the same three concepts: providers, contexts, and agents.
23
23
 
24
- Once you learn the fundamentals, everything else falls into place
25
- naturally. Some features, such as ActiveRecord support, require
26
- optional dependencies that are opt-in.
27
-
28
- The best way to learn about llm.rb is to ask [the official chatbot](https://r.uby.dev)
24
+ The easiest way to learn about llm.rb is to ask [the r.uby.dev chatbot](https://r.uby.dev)
29
25
  a question. It is connected to the llm.rb GitHub repository, backed by
30
- ActiveRecord and uses the builtin MCP feature to connect to GitHub. If
31
- you want to know more then simply [ask the chatbot](https://r.uby.dev) and
32
- you should get an answer that is grounded in the source code. It is 100%
33
- free to use and powered by DeepSeek under the hood.
26
+ ActiveRecord and uses the builtin MCP feature to connect to GitHub. All
27
+ answers are grounded in the llm.rb source code.
34
28
 
35
29
  ## Install
36
30
 
@@ -46,8 +40,7 @@ The
46
40
  [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
47
41
  class is the default high-level interface,
48
42
  and it is recommended for most use-cases. It manages tool execution
49
- automatically and
50
- [guards against infinite loops](https://r.uby.dev/llm/deepdive/advanced/guard),
43
+ automatically and guards against infinite loops,
51
44
  manages conversation state, and much more.
52
45
 
53
46
  ```ruby
@@ -68,9 +61,6 @@ reasoning, tool calls, tool returns, and compaction.
68
61
  Streams can also observe message transformers, which rewrite
69
62
  outgoing messages before they reach the provider.
70
63
 
71
- See the [deepdive.md](https://r.uby.dev/llm/deepdive/fundamentals/stream/)
72
- to learn more.
73
-
74
64
  ```ruby
75
65
  class MyStream < LLM::Stream
76
66
  # Visible assistant output.
@@ -118,8 +108,8 @@ class MyStream < LLM::Stream
118
108
  def on_skill_return(agent, skill, result)
119
109
  end
120
110
 
121
- # A request was rate limited and will be retried.
122
- def on_rate_limit(error)
111
+ # A request was rate limited or timed out and will be retried.
112
+ def on_retry(error, attempt)
123
113
  end
124
114
  end
125
115
 
@@ -140,7 +130,7 @@ call them on your behalf, and they're one of the most powerful features
140
130
  for extending the feature set or abilities of a model.
141
131
 
142
132
  The runtime also ships with a catalog of built-in tools for
143
- filesystem, search, and shell operations. <br> See the [deepdive.md](https://r.uby.dev/llm/deepdive/fundamentals/tools) to learn more.
133
+ filesystem, search, and shell operations.
144
134
 
145
135
  ```ruby
146
136
  class ReadFile < LLM::Tool
@@ -175,8 +165,6 @@ callback hands back the subagent that ran the skill, so you can inspect
175
165
  its conversation, measure its usage, track costs or add a verification
176
166
  step (eg `subagent.talk("verify your work")`).
177
167
 
178
- See the [deepdive.md](https://r.uby.dev/llm/deepdive/fundamentals/skills) to learn more.
179
-
180
168
  ##### summary.md
181
169
 
182
170
  ```markdown
@@ -216,7 +204,11 @@ with the `:fork` and `:ractor` strategies. The
216
204
  The `:fork` strategy also provides a separate process that offers
217
205
  isolation from its parent.
218
206
 
219
- See the [deepdive.md](https://r.uby.dev/llm/deepdive/features/concurrency) to learn more.
207
+ A couple of concurrency strategies require optional, opt-in dependencies.
208
+ The `async` strategy requires the [async](https://github.com/socketry/async)
209
+ gem and the `fork` strategy requires the [xchan.rb](https://github.com/0x1eef/xchan.rb)
210
+ gem. The `fiber` strategy requires a scheduler (`Fiber.scheduler`) but by
211
+ default Ruby does not provide one.
220
212
 
221
213
  ```ruby
222
214
  require "llm"
@@ -242,8 +234,6 @@ the control channel, a ractor via message passing, and pending tools
242
234
  are stopped before they run. The in-flight HTTP request is closed
243
235
  too, so a turn you no longer want stops without burning tokens.
244
236
 
245
- See the [deepdive.md](https://r.uby.dev/llm/deepdive/advanced/cancellation) to learn more.
246
-
247
237
  ```ruby
248
238
  llm = LLM.deepseek(key: ENV["KEY"])
249
239
  agent = LLM::Agent.new(llm)
@@ -268,8 +258,6 @@ The REPL stands out because it connects to the surrounding
268
258
  runtime and it can be extended by your code. Think of it as
269
259
  `binding.pry` but for agents.
270
260
 
271
- See the [deepdive.md](https://r.uby.dev/llm/deepdive/features/repl) to learn more.
272
-
273
261
  ##### Demo
274
262
 
275
263
  [Watch in high quality on asciinema](https://asciinema.org/a/OsS8wwaasKasoDDz)
@@ -284,7 +272,7 @@ a separate gem but it requires a number of optional dependencies
284
272
  to be installed separately. The following gems provide the full
285
273
  experience:
286
274
 
287
- gem install curses kramdown xchan.rb test-cmd.rb
275
+ gem install unicode-display_width curses kramdown xchan.rb test-cmd.rb
288
276
 
289
277
  ##### Persistence
290
278
 
@@ -327,8 +315,6 @@ code. For database-backed persistence, ActiveRecord and Sequel
327
315
  integrations are also available. All persistence options use the same
328
316
  underlying serialization.
329
317
 
330
- See the [deepdive.md](https://r.uby.dev/llm/deepdive/features/database) to learn more.
331
-
332
318
  ```ruby
333
319
  require "llm"
334
320
 
@@ -355,55 +341,90 @@ for both Rack-based / Rails-based applications. On databases
355
341
  where it is supported, such as PostgreSQL, the column can be optimized by using
356
342
  the `jsonb` type.
357
343
 
358
- See the [deepdive.md](https://r.uby.dev/llm/deepdive/features/database) to learn more.
344
+ The following example is based on the agent used to power the
345
+ [r.uby.dev chatbot](https://r.uby.dev).
359
346
 
360
347
  ```ruby
361
348
  require "active_record"
362
349
  require "llm"
363
350
  require "llm/active_record"
364
351
 
365
- class Email < ApplicationRecord
366
- acts_as_agent do |agent|
367
- agent.set name: "mail",
368
- instructions: "Write concise, friendly replies to emails",
369
- model: "deepseek-v4-pro"
352
+ class Raven < ActiveRecord::Base
353
+ acts_as_agent(format: :jsonb) do |agent|
354
+ agent.set name: "raven",
355
+ description: "a chatbot for the r.uby.dev website",
356
+ instructions: proc { File.read(File.join(__dir__, "raven", "prompt.md")) },
357
+ tools: :tools,
358
+ concurrency: :async
370
359
  end
371
360
 
372
- def draft_reply!
373
- talk("Draft a reply to:\n\n#{body}")
361
+ def research_issues
362
+ talk("research open pull requests on r-uby-dev/llm")
374
363
  end
375
364
 
376
- def summarize
377
- talk("Summarize this email thread in a few sentences")
365
+ def research_codebase
366
+ talk("research the codebase on r-uby-dev/llm")
378
367
  end
379
368
 
380
- private
369
+ ##
370
+ # @return [LLM::MCP]
371
+ def github
372
+ @github ||= LLM::MCP.http(
373
+ url: "https://api.githubcopilot.com/mcp/",
374
+ headers: {"Authorization" => "Bearer #{ENV['GITHUB_RUBYDEV_PAT']}"},
375
+ transport: :net_http_persistent
376
+ )
377
+ end
381
378
 
382
379
  ##
383
- # By convention, this method defines the provider for a model.
384
- # If necessary, it can be renamed with: provider: :your_method.
380
+ # @return [Array<LLM::Tool>]
381
+ def tools
382
+ github.tools.select { allowlist.include?(_1.name.to_s) }
383
+ end
384
+
385
+ private
386
+
385
387
  def set_provider
386
- LLM.deepseek(key: ENV["KEY"])
388
+ LLM.deepseek
387
389
  end
388
390
 
389
- ##
390
- # By convention, this method returns the context options given
391
- # to LLM::Context or LLM::Agent. This method can be left undefined.
392
- def set_context
393
- {}
391
+ def allowlist
392
+ %w[
393
+ get_commit
394
+ get_file_contents
395
+ list_branches
396
+ list_commits
397
+ search_code
398
+ search_commits
399
+ search_repositories
400
+ search_issues
401
+ pull_request_read
402
+ list_pull_requests
403
+ list_issues
404
+ issue_read
405
+ ].freeze
394
406
  end
395
407
  end
396
408
 
397
- email = Email.create!(subject: "Streaming support", body: "How do I stream responses?")
398
- email.draft_reply!
409
+ agent = Raven.create!
410
+
411
+ ##
412
+ # Every call to `talk` automatically persists
413
+ # to the database (under the hood research_issues
414
+ # calls the talk method)
415
+ agent.research_issues
416
+
417
+ ##
418
+ # The conversation was persisted to database. A
419
+ # fresh instance restores it and continues where
420
+ # we left off
421
+ agent = Raven.find(agent.id).tap(&:research_codebase)
399
422
 
400
423
  ##
401
- # The conversation (the email and the draft
402
- # reply) is persisted to the email's column. A
403
- # fresh instance restores it and continues the
404
- # thread, so the summary below knows what was
405
- # already drafted:
406
- Email.find(email.id).summarize
424
+ # Start an agent console.
425
+ # Query agent's state, debug, etc.
426
+ # The REPL does not persist back to the database.
427
+ agent.repl
407
428
  ```
408
429
  </details>
409
430
 
@@ -418,9 +439,6 @@ used with
418
439
  [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html) or
419
440
  [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html).
420
441
 
421
- See the [deepdive.md](https://r.uby.dev/llm/deepdive/protocols/mcp/), and the
422
- [deepdive.md on persistent connections](https://r.uby.dev/llm/deepdive/features/transports) to learn more.
423
-
424
442
  ```ruby
425
443
  require "llm"
426
444
 
@@ -441,9 +459,6 @@ used with
441
459
  [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html) or
442
460
  [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html).
443
461
 
444
- See the [deepdive.md](https://r.uby.dev/llm/deepdive/protocols/a2a/), and the
445
- [deepdive.md on persistent connections](https://r.uby.dev/llm/deepdive/features/transports) to learn more.
446
-
447
462
  ```ruby
448
463
  require "llm"
449
464
 
@@ -516,8 +531,6 @@ and implement
516
531
  The pending call arrives as `function:`. Return a value to close
517
532
  the call, or `nil` to let it run:
518
533
 
519
- See the [deepdive.md](https://r.uby.dev/llm/deepdive/advanced/guard) to learn more.
520
-
521
534
  ```ruby
522
535
  class PolicyGuard < LLM::Guard
523
536
  def call(function:)
@@ -543,8 +556,6 @@ Create a subclass and implement `call(message:)` to scrub sensitive data,
543
556
  inject context, or normalize content. The transform runs automatically
544
557
  on every turn, so you never have to change your prompt code.
545
558
 
546
- See the [deepdive.md](https://r.uby.dev/llm/deepdive/advanced/transformer) to learn more.
547
-
548
559
  ```ruby
549
560
  class RedactEmails < LLM::Transformer
550
561
  def call(message:)
@@ -581,8 +592,6 @@ and
581
592
  [`LLM::Stream#on_compaction_finish`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_compaction_finish)
582
593
  callbacks.
583
594
 
584
- See the [deepdive.md](https://r.uby.dev/llm/deepdive/advanced/compaction) to learn more.
585
-
586
595
  ```ruby
587
596
  llm = LLM.deepseek(key: ENV["KEY"])
588
597
  agent = LLM::Agent.new(
@@ -600,7 +609,8 @@ agent.talk "Hello"
600
609
 
601
610
  Rate-limited requests are retried automatically by default. Agents
602
611
  retry a 429 up to five times with a growing backoff before giving
603
- up, so most request failures resolve on their own. Set `retry_budget`
612
+ up, so most request failures resolve on their own. Connection and
613
+ read timeouts are retried the same way. Set `retry_budget`
604
614
  to change the number of retries, or `retry_budget: 0` to disable
605
615
  them.
606
616
 
@@ -631,8 +641,6 @@ exports spans via OTLP for OpenTelemetry in production.
631
641
  * [`LLM::Tracer::Logger`](https://r.uby.dev/api-docs/llm.rb/LLM/Tracer/Logger.html):
632
642
  structured JSON to stdout or a file.
633
643
 
634
- See the [deepdive.md](https://r.uby.dev/llm/deepdive/reference/tracer) to learn more.
635
-
636
644
  ```ruby
637
645
  llm = LLM.deepseek(key: ENV["KEY"])
638
646
  agent = LLM::Agent.new(llm, tracer: LLM::Tracer::PrettyLogger.new(llm))
@@ -676,9 +684,8 @@ Each provider is constructed with a class-level factory method on
676
684
  [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
677
685
  or
678
686
  [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html). The
679
- same API drives every one of them, so switching models is a one-line
680
- change. See the [deepdive](https://r.uby.dev/llm/deepdive/fundamentals/providers)
681
- for a full provider reference.
687
+ same API drives every one of them, so switching providers is a one-line
688
+ change.
682
689
 
683
690
  #### What providers does llm.rb support?
684
691
 
@@ -690,6 +697,7 @@ for a full provider reference.
690
697
  * **xAI** (`LLM.xai`)
691
698
  * **Z.ai** (`LLM.zai`)
692
699
  * **Moonshot (Kimi)** (`LLM.moonshot`)
700
+ * **OpenRouter** (`LLM.openrouter`)
693
701
  * **Alibaba (Qwen3)** (`LLM.alibaba`, also `LLM.aliyun`)
694
702
  * **Mistral** (`LLM.mistral`)
695
703
  * **AWS Bedrock** (`LLM.bedrock`)
@@ -711,6 +719,7 @@ llm = LLM.anthropic
711
719
  llm = LLM.deepseek
712
720
  llm = LLM.alibaba # also: LLM.aliyun
713
721
  llm = LLM.moonshot
722
+ llm = LLM.openrouter
714
723
  llm = LLM.mistral
715
724
  ```
716
725
  </details>
@@ -728,6 +737,7 @@ llm = LLM.anthropic(key: ENV["ANTHROPIC_API_KEY"])
728
737
  llm = LLM.deepseek(key: ENV["DEEPSEEK_API_KEY"])
729
738
  llm = LLM.alibaba(key: ENV["DASHSCOPE_API_KEY"]) # also: LLM.aliyun
730
739
  llm = LLM.moonshot(key: ENV["MOONSHOT_API_KEY"])
740
+ llm = LLM.openrouter(key: ENV["OPENROUTER_API_KEY"])
731
741
  llm = LLM.mistral(key: ENV["MISTRAL_API_KEY"])
732
742
  ```
733
743
  </details>
@@ -741,8 +751,6 @@ modalities with the gem, sourced from [models.dev](https://models.dev).
741
751
  Reach it from any provider, context, or agent, enumerate models, or
742
752
  sort them by price.
743
753
 
744
- See the [deepdive.md](https://r.uby.dev/llm/deepdive/reference/model_registry) to learn more.
745
-
746
754
  ```ruby
747
755
  require "llm"
748
756
 
@@ -765,8 +773,6 @@ is always available and the default, `net/http/persistent` pools
765
773
  connections for many requests to the same host, and `curb` wraps
766
774
  libcurl. They share one interface, so switching is a one-word change.
767
775
 
768
- See the [deepdive.md](https://r.uby.dev/llm/deepdive/advanced/transports) to learn more.
769
-
770
776
  ```ruby
771
777
  llm = LLM.deepseek(
772
778
  key: ENV["KEY"],
@@ -775,6 +781,51 @@ llm = LLM.deepseek(
775
781
  ```
776
782
  </details>
777
783
 
784
+ <details>
785
+ <summary>Timeouts</summary>
786
+ <br>
787
+
788
+ Providers accept two timeouts:
789
+
790
+ * `connect_timeout` - opening the connection. Defaults to 5 seconds.
791
+ * `read_timeout` - waiting for a response on an idle connection.
792
+ Defaults to 600 seconds (10 minutes).
793
+
794
+ The longer read timeout leaves room for slow reasoning models and
795
+ local models. The legacy `timeout:` option remains as a shorthand for
796
+ `read_timeout`. Timeouts are retriable:
797
+ [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
798
+ retries a timed out request up to its
799
+ [`retry_budget`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#retry_budget-class_method)
800
+ (five by default), so a dropped connection or a slow first token
801
+ is often something we can recover from.
802
+
803
+ ```ruby
804
+ llm = LLM.deepseek(
805
+ connect_timeout: 5, # opening the connection
806
+ read_timeout: 600 # waiting for the next bytes
807
+ )
808
+ ```
809
+ </details>
810
+
811
+ <details>
812
+ <summary>Headers</summary>
813
+ <br>
814
+
815
+ Providers can accept a custom set of headers with
816
+ the [`LLM::Provider#with`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#with-instance_method) method.
817
+ For example, you could set a custom User-Agent header,
818
+ or provide headers that carry special meaning to
819
+ certain providers (eg OpenAI, OpenRouter).
820
+
821
+ ```ruby
822
+ llm = LLM.openrouter
823
+ llm = llm.with("HTTP-Referer" => "https://example.com")
824
+ llm = llm.with("X-OpenRouter-Title" => "Example App")
825
+ ```
826
+
827
+ </details>
828
+
778
829
  ### RAG
779
830
 
780
831
  Most providers offer an embedding model that can be
@@ -788,8 +839,6 @@ llm.rb also includes support for OpenAI's vector store API. It
788
839
  provides a vector database as a HTTP service but we won't cover
789
840
  that here.
790
841
 
791
- See the [deepdive.md](https://r.uby.dev/llm/deepdive/features/embeddings) to learn more.
792
-
793
842
  ```ruby
794
843
  require "llm"
795
844
 
@@ -928,11 +977,11 @@ it as my primary driver for development.
928
977
 
929
978
  ## Resources
930
979
 
931
- If you like what you read so far, check out the [deepdive.md](https://r.uby.dev/llm/deepdive/)
932
- to learn more. Unfortunately it
933
- wasn't possible to cover every feature without the README becoming a small book.
934
- The [r.uby.dev](https://r.uby.dev) homepage also includes more learning material
935
- and resources.
980
+ The [r.uby.dev chatbot](https://r.uby.dev) is connected
981
+ to this very GitHub repository. It can read documentation,
982
+ source code, issues, and pull requests. The [docs/](docs/)
983
+ directory contains the full documentation and the chatbot
984
+ can find the answers to your questions there.
936
985
 
937
986
  ## License
938
987