llm.rb 16.0.0.pre.9 → 16.0.0

data/README.md

@@ -235,9 +235,13 @@ isolation from its parent.
 A couple of concurrency strategies require optional, opt-in dependencies.
 The `async` strategy requires the [async](https://github.com/socketry/async)
 gem and the `fork` strategy requires the [xchan.rb](https://github.com/r-uby-dev/xchan.rb)
-gem. The `fiber` strategy requires a scheduler (`Fiber.scheduler`) but by
+gem (`~> 0.24`). The `fiber` strategy requires a scheduler (`Fiber.scheduler`) but by
 default Ruby does not provide one.
 
+The `:ractor` strategy is the least interchangeable of the six. It runs
+class-based tools only, and a tool's arguments have to be
+ractor-shareable.
+
 ```ruby
 require "llm"
 require "llm/tools"
@@ -349,6 +353,7 @@ llm.rb -n curb             # use libcurl as the HTTP transport
 llm.rb -x 900              # read timeout of 15 minutes
 llm.rb -t                  # temporary session, no persistence
 llm.rb -v                  # print the version
+llm.rb -h                  # print usage
 ```
 </details>
 <details>
@@ -762,6 +767,41 @@ agent.talk "Hello"
 </details>
 
 
+<details>
+<summary>Usage and cost</summary>
+<br>
+
+Every context and agent reports what a conversation has spent and how
+much room is left, and the numbers answer different questions. A
+`token_usage` is the whole conversation, summed as an
+[LLM::Usage](https://r.uby.dev/api-docs/llm.rb/LLM/Usage.html), and it
+is what [LLM::Cost](https://r.uby.dev/api-docs/llm.rb/LLM/Cost.html)
+prices against the model registry:
+
+```ruby
+require "llm"
+
+llm = LLM.deepseek(key: ENV["KEY"])
+agent = LLM::Agent.new(llm)
+agent.talk "Hello"
+
+agent.token_usage  # => LLM::Usage for the whole conversation
+agent.cost         # => LLM::Cost, priced from the registry
+```
+
+A `context_used` is one turn's worth - the live size of the most recent
+assistant message - so it is what a context window is really being
+spent on, and `context_usage` is that as a fraction of the window:
+
+```ruby
+agent.context_used    # => tokens in the latest turn
+agent.context_window  # => the model's limit, or nil when unknown
+agent.context_usage   # => Rational, eg Rational(100, 10_000)
+```
+
+</details>
+
+
 <details>
 <summary>Observability</summary>
 <br>
@@ -892,6 +932,11 @@ llm = LLM.moonshot(key: ENV["MOONSHOT_API_KEY"])
 llm = LLM.openrouter(key: ENV["OPENROUTER_API_KEY"])
 llm = LLM.alibaba(key: ENV["DASHSCOPE_API_KEY"]) # also: LLM.aliyun
 llm = LLM.mistral(key: ENV["MISTRAL_API_KEY"])
+llm = LLM.bedrock(
+  access_key_id: ENV["AWS_ACCESS_KEY_ID"],
+  secret_access_key: ENV["AWS_SECRET_ACCESS_KEY"],
+  region: ENV["AWS_REGION"]
+)
 ```
 </details>
 

data/docs/deepdive/advanced/cancellation.md

@@ -85,7 +85,8 @@ make is one nobody asked for any more.
 
 How an interrupt reaches a tool depends on the strategy. The
 `:ractor` strategy delivers it through ractor message passing, and
-`:fork` via a message over the xchan control channel. The `:thread`
+`:fork` via a message over the control channel the `xchan.rb` gem
+provides. The `:thread`
 strategy raises it on the thread that runs the tool. The `:fiber`
 and `:async` strategies ask the fiber scheduler for the raise,
 because a scheduled fiber cannot be entered from another thread -
@@ -109,3 +110,19 @@ The runtime's own path out of a turn does not swallow an interrupt:
 [`LLM::Context#try`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#try)
 re-raises anything that is not a retryable failure, and the tracer
 interface has no broad rescues of its own.
+
+The names are worth separating.
+[`LLM::Context#interrupt!`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#interrupt!-instance_method)
+and
+[`LLM::Function#interrupt!`](https://r.uby.dev/api-docs/llm.rb/LLM/Function.html#interrupt!-instance_method)
+are each aliased to `cancel!`, so `cancel!` ends what is in flight and
+tells a tool, and the two spellings are the same call.
+[`LLM::Function#cancel`](https://r.uby.dev/api-docs/llm.rb/LLM/Function.html#cancel-instance_method)
+is something else: it answers a call that has not run with a cancelled
+return and marks it cancelled, which is how a caller declines a pending
+tool rather than interrupting one. A tool hears about an interrupt
+through
+[`LLM::Tool#on_interrupt`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#on_interrupt-instance_method),
+or
+[`LLM::Tool#on_cancel`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#on_cancel-instance_method),
+which takes precedence when both are defined.

data/docs/deepdive/advanced/compaction.md

@@ -24,7 +24,10 @@ It preserves tool call/return pairs so the conversation never
 contains an orphaned result.
 
 The `keep:` parameter accepts an integer count or a percentage
-string like `"80%"`. The default compactor is
+string like `"80%"`, and defaults to `64`. When `Truncate` is used
+through a context or an agent, `keep:` is given as
+`compactor_options: {keep: N}` rather than to the compactor itself.
+The default compactor is
 [`LLM::Compactor::Null`](https://r.uby.dev/api-docs/llm.rb/LLM/Compactor/Null.html),
 which does nothing. A compactor can also be used standalone:
 

data/docs/deepdive/advanced/context.md

@@ -332,7 +332,8 @@ which can be helpful when the interrupt is temporary.
 
 The mechanism is the same across all six concurrency strategies.
 The `:ractor` strategy delivers the interrupt through ractor
-message passing. The `:fork` strategy delivers it via xchan.
+message passing. The `:fork` strategy delivers it over the control
+channel the `xchan.rb` gem provides.
 
 ### Messages
 

data/docs/deepdive/features/concurrency.md

@@ -94,7 +94,7 @@ inside a reactor. Requires the `async` gem.
 **fork**: Each tool runs in a forked child process. True parallelism
 and process isolation. Best for shell commands, native extensions,
 or anything you do not want touching the parent's memory. Requires
-the `xchan` gem.
+the `xchan.rb` gem (`~> 0.24`).
 
 **ractor**: Each class-based tool runs in a Ruby Ractor. True
 parallelism without the overhead of forking full processes. Only
@@ -106,5 +106,5 @@ class-based tools are supported. Arguments must be ractor-shareable.
 | `:thread` | `Thread` | IO only (GVL) | No | None |
 | `:fiber` | `Fiber.schedule` | Cooperative | No | `Fiber.scheduler` |
 | `:async` | `Async::Reactor` | Cooperative | No | `async` gem |
-| `:fork` | `Kernel.fork` | Yes (process) | Yes (memory) | `xchan` gem |
+| `:fork` | `Kernel.fork` | Yes (process) | Yes (memory) | `xchan.rb` gem |
 | `:ractor` | `Ractor` | Yes (CPU) | Limited | None |

data/docs/deepdive/features/console.md

@@ -79,6 +79,46 @@ and
 helpers write a formatted `user:` message with a trailing newline,
 and `LLM::Command#write_message` matches the same interface.
 
+### The `llm.rb` executable
+
+#### Overview
+
+The console also starts from a shell. After the gem is installed,
+`llm.rb` puts a console on your PATH and opens a session in the current
+directory, so there is no Ruby file to write.
+
+#### How it works
+
+Run `llm.rb` with no arguments to auto-detect a provider from the usual
+environment variables, or name one with `-p`. The session for the
+current directory is kept under `~/.llm.rb/` and restored on the next
+visit, unless `-t` asks for a temporary one:
+
+```bash
+llm.rb                     # auto-detect from $PROVIDER_API_KEY
+llm.rb -p openai           # use OpenAI explicitly
+llm.rb -m gpt-5.6          # use a model other than the provider default
+llm.rb -c thread           # run tool calls on a separate thread
+llm.rb -n curb             # use libcurl as the HTTP transport
+llm.rb -x 900              # read timeout of 15 minutes
+llm.rb -t                  # temporary session, no persistence
+llm.rb -v                  # print the version
+llm.rb -h                  # print usage
+```
+
+#### Why would I use it?
+
+It is the shortest path to a working session: no script, no `require`,
+and a conversation that is still there tomorrow. Reach for
+`agent.console` instead when the agent needs a provider, tools, or a
+tracer that only your application can supply.
+
+#### Notes
+
+The switches are the console's own, so `-c`, `-m`, and `-p` set the
+same things `agent.console` takes from its agent. `-v` and `-h` print
+and exit.
+
 ### Switch the model
 
 The active model is exposed through

data/docs/deepdive/features/embeddings.md

@@ -42,8 +42,8 @@ pattern is one of the most common AI workflows.
 
 #### Notes
 
-Not all providers support embeddings. The following providers have
-an `embed` method:
+Not all providers support embeddings. The following providers have an
+`embed` method of their own:
 
 - OpenAI (`text-embedding-3-small`)
 - DeepInfra (`BAAI/bge-m3`)
@@ -53,6 +53,11 @@ an `embed` method:
 - OpenRouter (`openai/text-embedding-3-small`)
 - Bedrock (the model is given per call)
 
+The providers built on OpenAI inherit its `embed`, so DeepSeek, xAI,
+Z.ai, Moonshot, Alibaba, and llama.cpp answer it too, against their own
+OpenAI-compatible endpoint. Anthropic is the exception: it defines no
+`embed`, so the call raises `NotImplementedError`.
+
 ### Vector stores (OpenAI)
 
 #### Overview

data/docs/deepdive/features/files.md

@@ -45,8 +45,10 @@ assistants that read them, get their content.
 
 #### Notes
 
-OpenAI, Anthropic, and Google implement the Files API; a provider that
-does not raises `NotImplementedError`. `download` is OpenAI-only. A
-file id belongs to the provider that minted it: DeepSeek, for example,
-has no Files API and raises `LLM::PromptError` when it is handed a file
-id from another provider.
+Anthropic, Google, and OpenAI implement the Files API, and so do the
+providers built on OpenAI's - Moonshot and Alibaba among them, which
+inherit it. A provider that has no Files API of its own, such as
+DeepSeek, Mistral, or xAI, raises `NotImplementedError`. `download` is
+OpenAI-only, so the OpenAI-compatible providers inherit that too. A file
+id belongs to the provider that minted it: DeepSeek, for example, raises
+`LLM::PromptError` when it is handed a file id from another provider.

data/docs/deepdive/fundamentals/providers.md

@@ -318,4 +318,55 @@ endpoint. Token Plan users should point the provider at their own
 Token Plan URL via `DASHSCOPE_API_HOST` or `host:`. Check your
 Model Studio dashboard for the correct endpoint.
 
-Model metadata ships in `data/alibaba.json` for the registry.
+Model metadata ships in `data/alibaba.json` for the registry.
+
+### Bedrock
+
+#### Overview
+
+[`LLM::Bedrock`](https://r.uby.dev/api-docs/llm.rb/LLM/Bedrock.html)
+talks to [Amazon Bedrock](https://aws.amazon.com/bedrock/) through the
+[Converse API](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html),
+which reaches models from Anthropic, Meta, Mistral, Cohere, and others
+behind one endpoint. It is created with
+[`LLM.bedrock`](https://r.uby.dev/api-docs/llm.rb/LLM.html#bedrock-class_method).
+
+#### How it works
+
+Bedrock is the one provider that does not take a `key:`. It signs each
+request with AWS Signature V4, so it wants an access key, a secret key,
+and a region:
+
+```ruby
+require "llm"
+
+llm = LLM.bedrock(
+  access_key_id: ENV["AWS_ACCESS_KEY_ID"],
+  secret_access_key: ENV["AWS_SECRET_ACCESS_KEY"],
+  region: "us-east-1"
+)
+ctx = LLM::Context.new(llm)
+ctx.talk "Hello"
+```
+
+Each of the three falls back to its own environment variable
+(`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_REGION`), so
+`LLM.bedrock` alone works in an environment that already has
+credentials. `region:` defaults to `us-east-1`, and the host is
+`bedrock-runtime.<region>.amazonaws.com`.
+
+#### Why would I use it?
+
+One credential reaches every model Bedrock serves, so a model can be
+swapped with `model:` alone rather than by moving to another vendor's
+API. For an application that already runs on AWS, it also keeps the
+traffic inside the account's own region.
+
+#### Notes
+
+A model is named by its regional id rather than a short name - the
+entries in `data/bedrock.json` look like
+`anthropic.claude-sonnet-4-20250514-v1:0` or `google.gemma-4-31b`, and
+some are prefixed by the region that serves them (`us-gov.`, `eu.`,
+`global.`). Passing `key:` raises `ArgumentError`, because there is no
+API key to pass.

data/docs/deepdive/fundamentals/schema.md

@@ -133,6 +133,23 @@ renders the schema as a prompt-friendly string. `required` and
 `defaults` refer to properties that already exist, so declare the
 property first and mark it afterwards.
 
+A property also takes the constraints its type supports, as options.
+`min:` and `max:` bound a number's value or a string's length,
+`multiple_of:` constrains a number, `const:` fixes one value, `enum:`
+lists the allowed values, and `default:` gives one. Pass
+`required: true`, or `required: false`, to mark a single property
+without the class-level call:
+
+```ruby
+class Form < LLM::Schema
+  property :name, String, "The name", min: 2, max: 8
+  property :age, Integer, "The age", min: 0, max: 120
+  property :currency, String, "The currency", enum: %w[usd eur]
+  property :note, String, "An optional note", required: false
+  required %i[name age currency]
+end
+```
+
 ### Parsing
 
 #### Overview

data/docs/deepdive/fundamentals/stream.md

@@ -151,3 +151,60 @@ agent.talk "Explain Ruby fibers."
 A stream subclass gives you visibility into more than just content chunks. React to tool
 calls as they happen, show compaction progress, or integrate with
 an existing observability stack.
+
+### Streamed tool execution
+
+#### Overview
+
+A tool call is a round trip. By default the runtime makes it after the
+response has finished streaming, so the model sits idle for the length
+of the tool. A stream can start the tool the moment its call is parsed
+instead, and hand the result to the turn when it is asked for.
+
+#### How it works
+
+Start the tool from
+[`LLM::Stream#on_tool_call`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_tool_call)
+and enqueue its task on
+[`LLM::Stream#queue`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#queue-instance_method).
+[`LLM::Context#wait`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#wait-instance_method)
+drains that queue in preference to the pending functions, so a tool that
+was started early is waited for, not run twice:
+
+```ruby
+class FastStream < LLM::Stream
+  def on_tool_call(tool)
+    queue << tool.task(:thread).tap(&:spawn)
+  end
+end
+
+llm = LLM.deepseek(key: ENV["KEY"])
+agent = LLM::Agent.new(llm, stream: FastStream.new)
+agent.talk "Read every file in lib/ and summarise them."
+```
+
+[`LLM::Stream::Queue`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream/Queue.html)
+takes either a running task or an immediate
+[`LLM::Function::Return`](https://r.uby.dev/api-docs/llm.rb/LLM/Function/Return.html),
+and
+[`LLM::Stream#wait`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#wait-instance_method)
+resolves whatever it holds and returns the returns. The task type is
+remembered, so no strategy is passed at wait time.
+
+#### Why would I use it?
+
+A turn that calls several tools spends most of its wall-clock time
+waiting. Starting each tool as its call arrives overlaps that wait with
+the rest of the stream, so a turn that reads ten files costs roughly one
+file's latency rather than ten.
+
+#### Notes
+
+The queue is only consulted when it is non-empty. A turn that enqueues
+nothing behaves exactly as before, and a turn that enqueues from
+`on_tool_call` still runs any *other* pending function through the
+strategy given to
+[`LLM::Context#wait`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#wait-instance_method).
+A cancel reaches the queue
+([`LLM::Stream::Queue#interrupt!`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream/Queue.html#interrupt!-instance_method)),
+so tools started this way are interrupted with the rest of the turn.

data/docs/deepdive/fundamentals/tools.md

@@ -108,6 +108,14 @@ in the How it works section) or with
 [`LLM::Tool.set`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#set-class_method)
 (see the Set subsection). Both approaches work the same way.
 
+A parameter takes the same constraints a schema property does, as
+options: `min:` and `max:` bound a number or a string's length,
+`multiple_of:` constrains a number, `const:` fixes one value, and
+`enum:` lists the allowed values. So
+`parameter :age, Integer, "Age", min: 0, max: 120` sends the range to
+the model rather than a bare integer. The Schema topic covers the
+keywords in full.
+
 ### Confirmation
 
 #### Overview

data/lib/llm/tools/utils.rb

@@ -48,12 +48,22 @@ class LLM::Tool
     # Wait for a command to finish, or abort
     # with an error when it exceeds the
     # specified timeout.
+    #
+    # **A command is waited on until its status is decided, not only until it
+    # has stopped running.** `running?` answers whether the process is there,
+    # and `success?` answers `nil` until the command has been reaped - so a
+    # loop that trusts `running?` alone can leave before the status exists, and
+    # the caller reads `ok: nil` beside an output the read had waited for.
+    #
+    # **`running?` is what decides it.** The command reaps the process as it
+    # answers, so every turn of this loop is what makes a status exist - which
+    # is why the condition is asked again on each turn rather than read once.
     # @param [Test::Command] command
     # @param [Integer] timeout
     # @return [void]
     def wait(command:, timeout:)
       start = now
-      while command.running?
+      while command.running? || status_unavailable?(command)
         if now - start > timeout
           command.kill!
           raise "command timed out after #{timeout}s"
@@ -96,6 +106,20 @@ class LLM::Tool
       Process.clock_gettime(Process::CLOCK_MONOTONIC)
     end
 
+    private
+
+    ##
+    # Whether a command has not been reaped yet.
+    #
+    # `success?` answers `nil` until the status exists, and a command that was
+    # never found is the one case where it never will: there was nothing to
+    # reap, and `not_found?` is the answer the caller wants.
+    # @param [Test::Command] command
+    # @return [Boolean]
+    def status_unavailable?(command)
+      command.success?.nil? and !command.not_found?
+    end
+
     ##
     # requires test-cmd.rb
     LLM.require "test-cmd.rb", "~> 2.7.1"

data/lib/llm/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module LLM
-  VERSION = "16.0.0.pre.9"
+  VERSION = "16.0.0"
 end

data/llm.gemspec

@@ -10,9 +10,11 @@ Gem::Specification.new do |spec|
 
   spec.summary = "An agentic runtime for Ruby"
   spec.description = <<~DESCRIPTION
-llm.rb is a runtime for building agentic AI applications on CRuby.
-It has zero runtime dependencies by default, supports concurrent and parallel
-tool execution and has a single coherent API that spans 14+ providers.
+llm.rb is an agentic runtime for CRuby.
+It has zero runtime dependencies by default,
+supports concurrent and parallel tool execution
+and has a single coherent API that spans
+14+ providers.
 DESCRIPTION
 
   spec.license = "MIT"

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: llm.rb
 version: !ruby/object:Gem::Version
-  version: 16.0.0.pre.9
+  version: 16.0.0
 platform: ruby
 authors:
 - Robert Gleeson
@@ -362,9 +362,11 @@ dependencies:
       - !ruby/object:Gem::Version
         version: '2.45'
 description: |
-  llm.rb is a runtime for building agentic AI applications on CRuby.
-  It has zero runtime dependencies by default, supports concurrent and parallel
-  tool execution and has a single coherent API that spans 14+ providers.
+  llm.rb is an agentic runtime for CRuby.
+  It has zero runtime dependencies by default,
+  supports concurrent and parallel tool execution
+  and has a single coherent API that spans
+  14+ providers.
 email:
 - robert@r.uby.dev
 executables: