llm.rb 15.0.3 → 15.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b2700592b8b9fbb85694c24a0a7379a84494b0e4f2157d6302fad1552df9d684
-  data.tar.gz: 43864e55f008478a40860511f9ca138f86010695225f91f0b2a07fbcff1ed448
+  metadata.gz: 2b496d3aec5a309af0ff9eaac32cabb6c7191bf2bac1f3b664c02a56c593402e
+  data.tar.gz: 0ae83d7794b4dad881de7b9866b3894746efa1853e349fc5707416e367802db5
 SHA512:
-  metadata.gz: 464e203f6e4dc1fb2ea2aaa5bd4a2f95753d569190e63e4cd590b014aed4e369fa2f2835a4d6d95394b6988210c880e976abd5d8e8c69f015b057bb1f193609d
-  data.tar.gz: 9303d2475e2bd32dc53b65b7b9d54fa28611616a0714015bd1c0d74e8ca7cd66e6d88889b73231ed19dda38b2f2888ded08569694f23bd834a8e7c1f3fd6db35
+  metadata.gz: 9f890dd0720d0acbc79f9df2196f3595a68cd618fd0d09dc02fcab9a6fe27758ae72efd4b764614d5511bdb7914f586fba7194b3d1a5bec186562b9196753783
+  data.tar.gz: f25bc47b9bd5dcb0afea4f2ddbcb1ad37f74863cff962de6ceefa6157af073997589fd08fad0b2df11288aa24db41685127542b4d1748a71c4fef607d6b76ba2

data/CHANGELOG.md

@@ -15,6 +15,151 @@
 
 ## What's next
 
+## v15.1.0
+
+Changes since `v15.0.3`.
+
+This release adds the OpenRouter provider, splits provider timeouts
+into `connect_timeout` and `read_timeout`, retries timed-out requests,
+and renames `on_rate_limit` to `on_retry`. Skills now gain a
+frontmatter `model:` parameter and inherit the parent agent's model,
+while the CLI gains `-m` and `-x` switches, and the REPL shows retry
+progress and measures text by display width.
+
+### Provider
+
+* **add `LLM::OpenRouter` for the OpenRouter provider** <br>
+  [`LLM::OpenRouter`](https://r.uby.dev/api-docs/llm.rb/LLM/OpenRouter.html)
+  is a new provider that talks to [OpenRouter](https://openrouter.ai)
+  through its OpenAI-compatible API, contributed via
+  [PR #165](https://github.com/r-uby-dev/llm.rb/pull/165). Create an
+  instance with
+  [`LLM.openrouter`](https://r.uby.dev/api-docs/llm.rb/LLM.html#openrouter-class_method),
+  which accepts the same `key:`, `host:`, and `base_path:` options as the
+  OpenAI provider. It defaults to the `openrouter/auto` router model and
+  supports chat completions, streaming, tool calls, structured output,
+  and embeddings; image, audio, moderation, files, and vector store
+  endpoints raise `NotImplementedError`. Model metadata ships in
+  `data/openrouter.json` for the registry.
+
+* **provider: `LLM::Provider#with` accepts headers without the `headers:` keyword** <br>
+  [`LLM::Provider#with`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#with-instance_method)
+  now accepts headers directly as a Hash (`llm.with("User-Agent" => "llmrb/1.0")`)
+  in addition to the legacy `headers:` keyword form. Both are merged, and
+  the keyword form remains for backwards compatibility.
+
+* **providers: `model: nil` falls back to `default_model`** <br>
+  [`LLM::Provider`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html)
+  now treats a `model: nil` payload as "use the default model" instead of
+  sending a model value that providers reject. Previously a `{model: nil}`
+  param overwrote the default and then was removed by `.compact`, leaving
+  undefined behavior where providers could reject the request.
+
+* **openai: handle `model: nil` in the Responses API** <br>
+  The OpenAI Responses API path now also falls back to the default model
+  when `model: nil`, matching the completions path.
+
+* **provider: separate `connect_timeout` and `read_timeout`** <br>
+  [`LLM::Provider`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html)
+  now accepts a `connect_timeout:` for opening the connection (default 5
+  seconds) and a `read_timeout:` for waiting on an idle connection (default
+  600 seconds), instead of a single `timeout`. The legacy `timeout:` option
+  remains as a shorthand for `read_timeout`. The provider exposes the new
+  `#read_timeout` and `#connect_timeout` accessors.
+
+* **provider: retry timed-out requests** <br>
+  Requests that time out (`Net::OpenTimeout` or `Net::ReadTimeout`) are now
+  retried along with rate-limited requests, up to the `retry_budget`. When
+  a heavily loaded provider (for example DeepSeek) drops the connection
+  during the connect or read phase, the request is retried instead of
+  failing, and the stream is notified through `on_retry`.
+
+* **context: retry on `LLM::InsufficientQuotaError`, too** <br>
+  A request that raises
+  [`LLM::InsufficientQuotaError`](https://r.uby.dev/api-docs/llm.rb/LLM/InsufficientQuotaError.html)
+  is now retried like other rate-limited requests. The error is a subclass
+  of `LLM::RateLimitError` and can be raised at regular intervals, in
+  particular by the Alibaba provider.
+
+* **cost: return `LLM::Cost.zero` when the registry has no pricing** <br>
+  Add
+  [`LLM::Cost.zero`](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html#zero-class_method),
+  a factory for a zero-valued cost breakdown.
+  [`LLM::Context#cost`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#cost-instance_method)
+  now returns it when the active model has no pricing in the registry (for
+  example OpenRouter's `openrouter/auto` router model), instead of crashing
+  on a nil pricing entry.
+
+### Skills
+
+* **skills: add a `model` frontmatter parameter** <br>
+  A skill's `SKILL.md` frontmatter can now declare a `model:` value, so a
+  skill's sub-agent runs on a specific model instead of the default. This
+  lets a parent agent run on one model (for example `deepseek-v4-flash`)
+  while the skill runs on another (`deepseek-v4-pro`). The `model` value is
+  not strictly portable between providers.
+
+* **skills: inherit the model of the parent agent** <br>
+  A skill's sub-agent now inherits the active model of the agent that
+  spawns it instead of falling back to the provider default. The frontmatter
+  `model:` parameter overrides it explicitly when set.
+  [`LLM::Context#model`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#model-instance_method)
+  now falls back to the provider's default model when no model is set.
+
+### Stream
+
+* **stream: rename `on_rate_limit` to `on_retry`** <br>
+  [`LLM::Stream#on_rate_limit`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_rate_limit-instance_method)
+  is renamed to
+  [`LLM::Stream#on_retry`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_retry-instance_method),
+  which fires when a request is retried after a rate limit or a timeout,
+  and now receives the one-based retry attempt number as a second argument.
+  The old `on_rate_limit` name remains as an alias.
+
+### CLI
+
+* **cli: add a `-m` switch for choosing the model** <br>
+  `bin/llm.rb` now accepts `-m MODEL` to run the session on a model other
+  than the provider default, for example
+  `llm.rb -p deepseek -m deepseek-v4-pro`. A model unknown to the provider's
+  registry prints an error and exits.
+
+* **cli: add a `-x` switch for the read timeout** <br>
+  `bin/llm.rb` now accepts `-x SECONDS` to set the provider read timeout.
+
+* **cli: print a backtrace on fatal crashes** <br>
+  When `bin/llm.rb` hits an unexpected error, the crash message now includes
+  up to three stack lines from the backtrace, so the failure is easier to
+  locate and report than a bare diagnostic.
+
+### Repl
+
+* **repl: show retry progress in the status bar** <br>
+  When a request is rate limited or times out, the curses-based REPL status
+  bar shows a retry indicator with the error and the remaining attempts, for
+  example `🔁 Rate limited • attempt 2 of 5`.
+
+* **repl: measure text width with `unicode-display_width`** <br>
+  The curses-based REPL now counts and slices text by display column width
+  instead of character count, so wrapping, table columns, and clipping stay
+  aligned for wide characters such as emoji. It requires the optional
+  `unicode-display_width` gem.
+
+* **repl: treat `LLM::InsufficientQuotaError` as a rate limit in the status bar** <br>
+  The curses-based REPL status bar now shows `Rate limited` when a request
+  raises
+  [`LLM::InsufficientQuotaError`](https://r.uby.dev/api-docs/llm.rb/LLM/InsufficientQuotaError.html),
+  matching how ordinary `LLM::RateLimitError`s are shown, instead of falling
+  through to the raw class name.
+
+### Registry
+
+* **refresh model metadata** <br>
+  Update `data/*.json` with current model listings and pricing, adding
+  GPT-5.6 Sol, Terra, and Luna models to Bedrock, Grok 4.6 and Grok Imagine
+  Image 2.0 to xAI, DeepSeek V4 Flash Vision to DeepSeek, and DeepSeek V4
+  Pro 0813, Qwen3 VL, and Qwen3.8 models to DeepInfra.
+
 ## v15.0.3
 
 Changes since `v15.0.2`.
@@ -1077,7 +1222,7 @@ reliable across all six concurrency backends. The `functions` and
   `LLM.require` now accepts a second `version` parameter that is passed
   to `Kernel#gem` before loading, enabling version constraints for
   optional runtime dependencies. For example,
-  `LLM.require "test-cmd.rb", "~> 1.1"` ensures a minimum gem version
+  `LLM.require "test-cmd.rb", "~> 2.1"` ensures a minimum gem version
   is available. This is used internally by the `Git`, `Rg`, `Mkdir`,
   and `Shell` tools to enforce compatibility with the `test-cmd.rb` gem.
 

data/README.md

@@ -17,11 +17,11 @@ Welcome to the canonical llm.rb repository.
 llm.rb is an advanced runtime for building agentic AI applications
 on CRuby. It has zero runtime dependencies by default, supports
 concurrent and parallel tool execution and has a single coherent API
-that spans 13+ providers. Streaming, tools, guards, compaction, the
+that spans 14+ providers. Streaming, tools, guards, compaction, the
 REPL, builtin MCP/A2A support and the database integrations all build
 on the same three concepts: providers, contexts, and agents.
 
-The most effective way to learn about llm.rb is to ask [the r.uby.dev chatbot](https://r.uby.dev)
+The easiest way to learn about llm.rb is to ask [the r.uby.dev chatbot](https://r.uby.dev)
 a question. It is connected to the llm.rb GitHub repository, backed by
 ActiveRecord and uses the builtin MCP feature to connect to GitHub. All
 answers are grounded in the llm.rb source code.
@@ -108,8 +108,8 @@ class MyStream < LLM::Stream
   def on_skill_return(agent, skill, result)
   end
 
-  # A request was rate limited and will be retried.
-  def on_rate_limit(error)
+  # A request was rate limited or timed out and will be retried.
+  def on_retry(error, attempt)
   end
 end
 
@@ -204,6 +204,12 @@ with the `:fork` and `:ractor` strategies. The
 The `:fork` strategy also provides a separate process that offers
 isolation from its parent.
 
+A couple of concurrency strategies require optional, opt-in dependencies.
+The `async` strategy requires the [async](https://github.com/socketry/async)
+gem and the `fork` strategy requires the [xchan.rb](https://github.com/0x1eef/xchan.rb)
+gem. The `fiber` strategy requires a scheduler (`Fiber.scheduler`) but by
+default Ruby does not provide one.
+
 ```ruby
 require "llm"
 require "llm/tools"
@@ -266,7 +272,7 @@ a separate gem but it requires a number of optional dependencies
 to be installed separately. The following gems provide the full
 experience:
 
-    gem install curses kramdown xchan.rb test-cmd.rb
+    gem install unicode-display_width curses kramdown xchan.rb test-cmd.rb
 
 ##### Persistence
 
@@ -335,53 +341,90 @@ for both Rack-based / Rails-based applications. On databases
 where it is supported, such as PostgreSQL, the column can be optimized by using
 the `jsonb` type.
 
+The following example is based on the agent used to power the
+[r.uby.dev chatbot](https://r.uby.dev).
+
 ```ruby
 require "active_record"
 require "llm"
 require "llm/active_record"
 
-class Email < ApplicationRecord
-  acts_as_agent do |agent|
-    agent.set name: "mail",
-              instructions: "Write concise, friendly replies to emails",
-              model: "deepseek-v4-pro"
+class Raven < ActiveRecord::Base
+  acts_as_agent(format: :jsonb) do |agent|
+    agent.set name: "raven",
+              description: "a chatbot for the r.uby.dev website",
+              instructions: proc { File.read(File.join(__dir__, "raven", "prompt.md")) },
+              tools: :tools,
+              concurrency: :async
   end
 
-  def draft_reply!
-    talk("Draft a reply to:\n\n#{body}")
+  def research_issues
+    talk("research open pull requests on r-uby-dev/llm")
   end
 
-  def summarize
-    talk("Summarize this email thread in a few sentences")
+  def research_codebase
+    talk("research the codebase on r-uby-dev/llm")
   end
 
-  private
+  ##
+  # @return [LLM::MCP]
+  def github
+    @github ||= LLM::MCP.http(
+      url: "https://api.githubcopilot.com/mcp/",
+      headers: {"Authorization" => "Bearer #{ENV['GITHUB_RUBYDEV_PAT']}"},
+      transport: :net_http_persistent
+    )
+  end
 
   ##
-  # By convention, this method defines the provider for a model.
-  # If necessary, it can be renamed with: provider: :your_method.
+  # @return [Array<LLM::Tool>]
+  def tools
+    github.tools.select { allowlist.include?(_1.name.to_s) }
+  end
+
+  private
+
   def set_provider
-    LLM.deepseek(key: ENV["KEY"])
+    LLM.deepseek
   end
 
-  ##
-  # By convention, this method returns the context options given
-  # to LLM::Context or LLM::Agent. This method can be left undefined.
-  def set_context
-    {}
+  def allowlist
+    %w[
+        get_commit
+        get_file_contents
+        list_branches
+        list_commits
+        search_code
+        search_commits
+        search_repositories
+        search_issues
+        pull_request_read
+        list_pull_requests
+        list_issues
+        issue_read
+    ].freeze
   end
 end
 
-email = Email.create!(subject: "Streaming support", body: "How do I stream responses?")
-email.draft_reply!
+agent = Raven.create!
+
+##
+# Every call to `talk` automatically persists
+# to the database (under the hood research_issues
+# calls the talk method)
+agent.research_issues
+
+##
+# The conversation was persisted to database. A
+# fresh instance restores it and continues where
+# we left off
+agent = Raven.find(agent.id).tap(&:research_codebase)
 
 ##
-# The conversation (the email and the draft
-# reply) is persisted to the email's column. A
-# fresh instance restores it and continues the
-# thread, so the summary below knows what was
-# already drafted:
-Email.find(email.id).summarize
+# Start an agent console.
+# Query agent's state, debug, etc.
+# The REPL does not persist back to the database.
+agent.repl
 ```
 </details>
 
@@ -566,7 +609,8 @@ agent.talk "Hello"
 
 Rate-limited requests are retried automatically by default. Agents
 retry a 429 up to five times with a growing backoff before giving
-up, so most request failures resolve on their own. Set `retry_budget`
+up, so most request failures resolve on their own. Connection and
+read timeouts are retried the same way. Set `retry_budget`
 to change the number of retries, or `retry_budget: 0` to disable
 them.
 
@@ -653,6 +697,7 @@ change.
 * **xAI** (`LLM.xai`)
 * **Z.ai** (`LLM.zai`)
 * **Moonshot (Kimi)** (`LLM.moonshot`)
+* **OpenRouter** (`LLM.openrouter`)
 * **Alibaba (Qwen3)** (`LLM.alibaba`, also `LLM.aliyun`)
 * **Mistral** (`LLM.mistral`)
 * **AWS Bedrock** (`LLM.bedrock`)
@@ -674,6 +719,7 @@ llm = LLM.anthropic
 llm = LLM.deepseek
 llm = LLM.alibaba  # also: LLM.aliyun
 llm = LLM.moonshot
+llm = LLM.openrouter
 llm = LLM.mistral
 ```
 </details>
@@ -691,6 +737,7 @@ llm = LLM.anthropic(key: ENV["ANTHROPIC_API_KEY"])
 llm = LLM.deepseek(key: ENV["DEEPSEEK_API_KEY"])
 llm = LLM.alibaba(key: ENV["DASHSCOPE_API_KEY"]) # also: LLM.aliyun
 llm = LLM.moonshot(key: ENV["MOONSHOT_API_KEY"])
+llm = LLM.openrouter(key: ENV["OPENROUTER_API_KEY"])
 llm = LLM.mistral(key: ENV["MISTRAL_API_KEY"])
 ```
 </details>
@@ -734,6 +781,51 @@ llm = LLM.deepseek(
 ```
 </details>
 
+<details>
+<summary>Timeouts</summary>
+<br>
+
+Providers accept two timeouts:
+
+* `connect_timeout` - opening the connection. Defaults to 5 seconds.
+* `read_timeout` - waiting for a response on an idle connection.
+  Defaults to 600 seconds (10 minutes).
+
+The longer read timeout leaves room for slow reasoning models and
+local models. The legacy `timeout:` option remains as a shorthand for
+`read_timeout`. Timeouts are retriable:
+[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)
+retries a timed out request up to its
+[`retry_budget`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#retry_budget-class_method)
+(five by default), so a dropped connection or a slow first token
+is often something we can recover from.
+
+```ruby
+llm = LLM.deepseek(
+  connect_timeout: 5,   # opening the connection
+  read_timeout: 600     # waiting for the next bytes
+)
+```
+</details>
+
+<details>
+<summary>Headers</summary>
+<br>
+
+Providers can accept a custom set of headers with
+the [`LLM::Provider#with`](https://r.uby.dev/api-docs/llm.rb/LLM/Provider.html#with-instance_method) method.
+For example, you could set a custom User-Agent header,
+or provide headers that carry special meaning to
+certain providers (eg OpenAI, OpenRouter).
+
+```ruby
+llm = LLM.openrouter
+llm = llm.with("HTTP-Referer" => "https://example.com")
+llm = llm.with("X-OpenRouter-Title" => "Example App")
+```
+
+</details>
+
 ### RAG
 
 Most providers offer an embedding model that can be

data/bin/llm.rb

@@ -49,16 +49,20 @@ def help
   warn ""
   warn "Options:"
   warn "  -p PROVIDER    Choose a provider"
+  warn "  -m MODEL       Choose a model"
   warn "  -c STRATEGY    Concurrency strategy for tool calls (eg thread, async, fork)"
   warn "  -n TRANSPORT   HTTP transports - net-http (default), net-http-persistent, and curb"
+  warn "  -x TIMEOUT     The default read timeout (in seconds)"
   warn "  -t             Temporary session that doesn't persist to disk"
   warn "  -h             Show this help"
   warn ""
   warn "Examples:"
   warn "  #{prog}                     # auto-detect provider from $PROVIDER_API_KEY"
   warn "  #{prog} -p openai           # use OpenAI"
+  warn "  #{prog} -m gpt-5.6          # use a model other than the provider default"
   warn "  #{prog} -n curb             # use libcurl"
   warn "  #{prog} -c thread           # run tool calls on a separate thread"
+  warn "  #{prog} -x 900              # read timeout of 15mins"
   warn "  #{prog} -h                  # this help"
   warn ""
 end
@@ -94,6 +98,11 @@ def fatal(ex)
     wrapped detail, "     "
   end
   warn ""
+  warn "     Backtrace:"
+  ex.backtrace.drop(1).first(3).each do |line|
+    wrapped line, "       "
+  end
+  warn ""
   warn "     This is an unexpected error. If it keeps happening,"
   warn "     consider opening an issue at"
   warn "     https://github.com/r-uby-dev/llm/issues"
@@ -153,6 +162,22 @@ def main(argv)
           help
           exit 1
         end
+      when '-m'
+        model = argv.shift
+        if model.nil?
+          warn "llm.rb: -m switch requires an argument"
+          help
+          exit 1
+        end
+      when '-x'
+        timeout = argv.shift
+        if timeout.nil?
+          warn "llm.rb: -x switch requires an argument"
+          help
+          exit 1
+        else
+          timeout = Integer(timeout)
+        end
       else
         warn "llm.rb: unknown option #{option}"
         help
@@ -164,9 +189,10 @@ def main(argv)
     # No provider has been given.
     # Try to infer one.
     transport ||= :net_http
+    options = timeout ? {timeout:, transport:} : {transport:}
     if provider.nil?
       llm = providers.filter_map do
-        LLM.method(_1).call(transport:)
+        LLM.method(_1).call(**options)
       rescue ArgumentError
       end.first
       if llm.nil?
@@ -175,7 +201,7 @@ def main(argv)
       end
     else
       begin
-        llm = LLM.method(provider).call(transport:)
+        llm = LLM.method(provider).call(**options)
       rescue ArgumentError
         warn "llm.rb: set credentials for #{provider}"
         exit 1
@@ -209,11 +235,20 @@ def main(argv)
       File.binwrite file, JSON.pretty_generate(data)
     end
 
+    if model
+      if not llm.registry.keys.include?(model)
+        warn "llm.rb: #{model} is not a valid #{llm.name} model"
+        exit 1
+      end
+    else
+      model = llm.default_model
+    end
+
     ##
     # Let's go!
     concurrency ||= :sequential
     path  = temp ? nil : data[Dir.getwd]
-    agent = LLM::Agent.new(llm, path:, concurrency:, tools: LLM::Tool.subclasses)
+    agent = LLM::Agent.new(llm, model:, path:, concurrency:, tools: LLM::Tool.subclasses)
     agent.repl
   rescue Interrupt
     warn "llm.rb: Bye!"