llm.rb 16.0.0.pre.3 → 16.0.0.pre.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a8bd3b4f303036ba390981de3ec63ea0bb523cc653c4b77b221e4591e9b9a446
-  data.tar.gz: 3036ce3c4248e3ddad524657f31ea2c300f98282ad0aa68791848f56cd1b91f5
+  metadata.gz: a83847d3c33a48980bd9740d7aff397db757ec91224440a97c10240c0ae3a290
+  data.tar.gz: 402d6ea4a68fc987e3597d1b6dd84089f2804c7774ef35e38fb8fc424b0e62b0
 SHA512:
-  metadata.gz: 604bafd36e8a55405e3fc487957ae97b9e4ffd67777711c071f7bfb9cfe005e0400294c089e72fbec3019538f4706c92cfe5bb15f3d0c2f994a324b9ba1a3322
-  data.tar.gz: d3bf4d7e1f9337cd7bdedb40ac4cfb4f95b6195e6df3360d428bfa6c220994edfaf8c5f053256e968c1feb9ae025b29854586dd9834dc84948729644f7a7b8ff
+  metadata.gz: f75dda6e725c6aa48748a407e744c94c006d3db4973cf45064f7f0252fc2bc818ee7a99b564e66c4bad6ad4066bf5214b912a637b3b25b8b9f335d45d226c7e2
+  data.tar.gz: ea3e7fe4a32687c7a34f4b85a0bd4aeee5b7b68387452dd1ac98a3572f1b1f74b4f19ac3ee32b10cf6f53563b9eceb5a8e9292e6001b90728e4997c7b9a301a2

data/CHANGELOG.md

@@ -46,6 +46,16 @@
   [`LLM::Message`](https://r.uby.dev/api-docs/llm.rb/LLM/Message.html) objects.
   Every other format, and Sequel, still loads the runtime.
 
+### ActiveRecord
+
+* **activerecord: add `messages!` for the runtime's own messages** <br>
+  [`LLM::ActiveRecord#messages!`](https://r.uby.dev/api-docs/llm.rb/LLM/ActiveRecord.html#messages!-instance_method)
+  returns the messages the runtime holds whatever the storage format is, so a
+  jsonb record can still reach them now that its `#messages` answers with the
+  view over the column. It reads what the context holds, including state that
+  has not been saved, where `#messages` reads the column. The Sequel plugin
+  answers to `#messages!` too, where it is the same call as `#messages`.
+
 ### Console
 
 * **console: erase with the backspace key on OpenBSD** <br>
@@ -115,6 +125,23 @@
   `respond_to?` with false for an instance method. A hook that raises becomes
   what the caller sees in place of the call's result.
 
+### ORM
+
+* **orm: save a conversation after each request, not once per turn** <br>
+  [`LLM::Step`](https://r.uby.dev/api-docs/llm.rb/LLM/Step.html) is prepended
+  onto every stream by
+  [`LLM::Stream.try`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#try-class_method),
+  so a conversation bound to a record is written down as it goes and a turn
+  that is interrupted can be continued rather than started over. It saves
+  through
+  [`LLM::Context#record`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#record-instance_method),
+  an ActiveRecord model through `LLM::ActiveRecord::Utils.save!` and a Sequel
+  model through `LLM::Sequel::Plugin::Utils.save!`, and the write is best
+  effort, so a failure is swallowed rather than raised inside the request that
+  has just completed. Before this, `talk` and `ask` on both wrappers saved
+  once, when the whole turn had finished, so an interrupted turn persisted
+  nothing.
+
 ### Provider
 
 * **deepseek: support image attachments in chat completions** <br>
@@ -155,6 +182,19 @@
   discarded a false default, and `required: false` marks the parameter optional
   rather than being ignored.
 
+### Stream
+
+* **stream: add `on_step`, called when a request completes** <br>
+  [`LLM::Stream#on_step`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_step-instance_method)
+  is a new callback, emitted from
+  [`LLM::Context#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#talk-instance_method)
+  once the prompt and the response are in the conversation. It marks the
+  boundary between one request and the next, which is the point a stream can
+  checkpoint the conversation at, because a provider will accept it again
+  there. It fires once per successful request, so a request retried after a
+  rate limit or a timeout calls it when it lands and not once per attempt, and
+  it fires even when the stream is disabled.
+
 ## v15.5.0
 
 Changes since `v15.4.1`.

data/docs/deepdive/advanced/context.md

@@ -81,9 +81,10 @@ Pass `retry_budget:` to retry a request that was rate limited
 Each retry sleeps a growing interval (2s, 4s, 6s, ...) and notifies
 the stream through
 [`LLM::Stream#on_retry`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_retry-instance_method)
-before trying again. An `LLM::Agent` enables a budget of 5 by
-default (or 8 on the Alibaba provider, which rate limits more
-often), so most users never touch this directly.
+before trying again. The budget lives on the provider, where it
+defaults to 5 (8 on Alibaba, which rate limits more often), and an
+agent that sets none of its own takes the provider's, so most users
+never touch this directly.
 
 ### Identity
 

data/docs/deepdive/features/database.md

@@ -419,11 +419,12 @@ records rather than
 [`LLM::Message`](https://r.uby.dev/api-docs/llm.rb/LLM/Message.html)
 objects, until
 [`#unwrap!`](https://r.uby.dev/api-docs/llm.rb/LLM/ActiveRecord/Message.html#unwrap!-instance_method)
-turns them back.
+turns them back. `#messages!` is the runtime's own list, whatever the
+format is, so a jsonb record reaches it in one call.
 
-Every other format loads the runtime and returns the messages it
-holds, as it always did, and that is the default. Sequel has no
-equivalent: its plugin persists the same state, and there is no view to
+Every other format answers `#messages` with the runtime's messages, as
+it always did, and that is the default. Sequel has no equivalent to any
+of this: its plugin persists the same state, and there is no view to
 read it back in the database.
 
 ### Sequel

data/docs/deepdive/features/embeddings.md

@@ -50,6 +50,8 @@ an `embed` method:
 - Ollama (`qwen3:latest`)
 - Google (`gemini-embedding-2`)
 - Mistral (`mistral-embed`)
+- OpenRouter (`openai/text-embedding-3-small`)
+- Bedrock (the model is given per call)
 
 ### Vector stores (OpenAI)
 

data/docs/deepdive/features/files.md

@@ -0,0 +1,52 @@
+
+## Files
+
+### Introduction
+
+#### Overview
+
+Some providers keep files you upload, and let a later request refer to
+one by id instead of resending its bytes. The Files API is how a file
+is uploaded, listed, fetched, downloaded, and deleted, and a file id is
+then passed to an endpoint that reads it.
+
+#### How it works
+
+Call the file methods on a provider that implements them. `create`
+uploads a file and returns its record, `get` reads a record, `download`
+returns the bytes, `all` lists the files, and `delete` removes one:
+
+```ruby
+require "llm"
+
+llm  = LLM.openai(key: ENV["KEY"])
+ctx  = LLM::Context.new(llm)
+
+file = llm.files.create(file: "/books/goodread.pdf")
+ctx.talk ["Tell me about this PDF", file]
+
+llm.files.all                         # every uploaded file
+llm.files.get(file: file.id)          # one file's record
+llm.files.download(file: file.id)     # its bytes, under #file
+llm.files.delete(file: file.id)
+```
+
+`create` takes a path, an `LLM::File`, or a `File` object, and a
+`purpose:` that tells the provider what the file is for. An uploaded
+file carries its id, so it can be passed to a turn directly, as above,
+and a response from `download` exposes the bytes through `#file`.
+
+#### Why would I use it?
+
+A file that a provider stores can be reused across requests without
+being uploaded each time. That keeps a large document or an image out
+of the request body, and it is how OpenAI's vector stores, and the
+assistants that read them, get their content.
+
+#### Notes
+
+OpenAI, Anthropic, and Google implement the Files API; a provider that
+does not raises `NotImplementedError`. `download` is OpenAI-only. A
+file id belongs to the provider that minted it: DeepSeek, for example,
+has no Files API and raises `LLM::PromptError` when it is handed a file
+id from another provider.

data/docs/deepdive/features/moderations.md

@@ -0,0 +1,48 @@
+
+## Moderations
+
+### Introduction
+
+#### Overview
+
+A moderation classifies text or an image against a set of harm
+categories, so an application can decide what to do with it before
+showing it, storing it, or sending it to a model. OpenAI hosts the
+moderation endpoint.
+
+#### How it works
+
+Call `llm.moderations` with the content to check. The response lists a
+moderation per input, and each one reports whether it was flagged, the
+categories it matched, and their scores:
+
+```ruby
+require "llm"
+
+llm = LLM.openai(key: ENV["KEY"])
+
+res = llm.moderations.create(input: "I hate you")
+mod = res.moderations[0]
+
+mod.flagged?     # => true
+mod.categories   # the categories that matched
+mod.scores       # a score per category
+```
+
+`input` accepts a string or a URL, so an image URL is classified the
+same way text is.
+
+#### Why would I use it?
+
+A moderation is a cheap check that runs before an expensive or public
+step. Screen user-generated content before publishing it, gate a
+request before it reaches a model, or keep a record of what was
+filtered and why.
+
+#### Notes
+
+Only OpenAI implements moderations; another provider raises
+`NotImplementedError`. The default model is `omni-moderation-latest`,
+and it classifies both text and images. The scores are model
+estimates, so a threshold is a product decision rather than a fixed
+value.

data/docs/deepdive/features/responses.md

@@ -0,0 +1,54 @@
+
+## Responses
+
+### Introduction
+
+#### Overview
+
+OpenAI's Responses API gives every reply an id, and a later call can
+continue from one by passing `previous_response_id:`. When the provider
+is asked to store the reply, it keeps the history, so a client can hand
+the thread back instead of resending it.
+
+#### How it works
+
+Create a response with `llm.responses.create`, then continue it by
+passing `previous_response_id:` the id of the last one. A response can
+be read back with `get` and removed with `delete`:
+
+```ruby
+require "llm"
+
+llm = LLM.openai(key: ENV["KEY"])
+
+first  = llm.responses.create "Your task is to answer the user's questions",
+                              role: :developer
+second = llm.responses.create "5 + 5 = X ?",
+                              role: :user,
+                              previous_response_id: first.id
+
+puts second.content
+
+llm.responses.get(second)     # read it back
+llm.responses.delete(second)  # remove it
+```
+
+`create` accepts the same prompt shapes as a completion (a string or a
+list), plus `role:`, `model:`, `tools:`, and a `stream:` target. It
+returns an `LLM::Response` like any other call.
+
+#### Why would I use it?
+
+Server-side state means the conversation lives with the provider
+instead of in the request. Passing one id continues a thread without
+rebuilding and resending the messages, which keeps requests small and
+lets a client that lost its local history pick the thread back up.
+
+#### Notes
+
+Only OpenAI implements the Responses API. A context in `mode:
+:responses` uses this endpoint and defaults to `store: false`, so the
+conversation is not kept on the provider and the context rebuilds the
+input itself; set `store: true` to keep it there and have the next turn
+continue with `previous_response_id`. A plain `llm.responses.create`
+call continues only when `previous_response_id:` is given.

data/docs/deepdive/fundamentals/agents.md

@@ -245,9 +245,12 @@ of hanging.
 
 The retry budget applies to rate-limited requests
 (`LLM::RateLimitError`) and timeouts (`Timeout::Error`, covering
-`Net::OpenTimeout` and `Net::ReadTimeout`), other errors are never
-retried. The budget defaults to five for agents, while a raw
+`Net::OpenTimeout` and `Net::ReadTimeout`); other errors are never
+retried. The budget lives on the provider, where it defaults to five
+(eight on Alibaba, which rate limits more often), and an agent that
+sets none of its own takes the provider's. A raw
 [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)
-defaults to zero (`retry_budget: 0`). A 429 is refused before any
-content streams, so retrying the same request loses nothing. Pass
-`retry_budget: 0` to disable retries on an agent.
+defaults to zero (`retry_budget: 0`), so retries are opt-in there. A
+429 is refused before any content streams, so retrying the same
+request loses nothing. Pass `retry_budget: 0` to disable retries on
+an agent.

data/docs/deepdive/fundamentals/providers.md

@@ -92,6 +92,43 @@ missing model or registry raises `LLM::NoSuchModelError` or
 `LLM::NoSuchRegistryError`, which the runtime rescues to default
 gracefully (for example, an unknown context window reads as `nil`).
 
+### Model listings
+
+#### Overview
+
+The registry is the catalog llm.rb ships. A provider can also list the
+models it serves right now, which shows a model added since your copy
+of `data/` was written, or one available to your account alone.
+
+#### How it works
+
+A provider with a models endpoint exposes it through `llm.models`, and
+`all` returns the list:
+
+```ruby
+require "llm"
+
+llm = LLM.openai(key: ENV["KEY"])
+llm.models.all.each { puts _1.id }
+```
+
+OpenAI, Anthropic, Google, Bedrock, and Ollama implement it. Bedrock
+lists the foundation models in the configured region and accepts
+`byProvider:` to narrow the list; Ollama lists the models pulled into
+the local daemon.
+
+#### Why would I use it?
+
+A shipped catalog goes stale between releases. Asking the provider
+keeps a model picker current and reveals models the catalog does not
+know about.
+
+#### Notes
+
+Not every provider implements a models endpoint; calling `models` on
+one that does not raises `NotImplementedError`. Unlike the registry,
+the call needs credentials.
+
 ### Request headers
 
 #### Overview

data/docs/deepdive/fundamentals/schema.md

@@ -178,3 +178,53 @@ DSL builds, so the result is used the same way.
 An unsupported `type`, an unresolvable `$ref`, or a `$ref` that does
 not start with `#/` raises `TypeError`. When a schema omits `type`,
 the parser infers it from `const`, `enum`, or `default`.
+
+### Deferred types
+
+#### Overview
+
+A property or parameter type can be given as a block, in which case the
+type is worked out when the schema is serialized rather than when the
+class that declares it is defined. That is how a type that only exists
+at runtime, such as the values a store holds right now, is described.
+
+#### How it works
+
+Pass a block in place of the type. It is called each time the schema is
+serialized, and whatever it returns is resolved in turn, so it can
+return a leaf, a class, or an array of either:
+
+```ruby
+class Article < LLM::Schema
+  property :kind, proc { LLM::Schema::Enum[*Article.kinds] }, "The article kind"
+  required %i[kind]
+end
+```
+
+The same works for a tool parameter:
+
+```ruby
+class Publish < LLM::Tool
+  name "publish"
+  parameter :kind, proc { LLM::Schema::Enum[*Article.kinds] }, "The article kind"
+  required %i[kind]
+
+  def call(kind:)
+    Article.publish(kind:)
+  end
+end
+```
+
+#### Why would I use it?
+
+A schema is a class-level definition, but the choices it offers are
+sometimes only known when a request is made. A deferred type lets the
+schema be declared once and still send the values that are current at
+the time of the request.
+
+#### Notes
+
+The block is called on every serialization, so it runs once per
+request and sees the latest values. Everything a leaf accepts, such as
+`required`, `default`, and `enum`, is applied to the resolved leaf
+after the block returns.

data/docs/deepdive/fundamentals/tools.md

@@ -435,6 +435,12 @@ needs most. See the
 The tools that spawn subprocesses use the optional `test-cmd.rb`
 gem for process management and interrupt handling.
 
+`LLM::Tool.subclasses` lists the direct subclasses of
+[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html), so a
+tool defined under an intermediate base class is not in the list.
+[`LLM::Tool.registry`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html#registry-class_method)
+lists every tool that has a name, wherever it sits in the hierarchy.
+
 ### Interrupts
 
 #### Overview

data/docs/deepdive/reference/object.md

@@ -61,6 +61,12 @@ is defined explicitly and serializes through the configured JSON
 adapter via
 [`LLM.json.dump`](https://r.uby.dev/api-docs/llm.rb/LLM.html#json-class_method).
 
+The adapter is `LLM::JSONAdapter::JSON`, the standard library, by
+default. Set `LLM.json = :oj` or `LLM.json = :yajl` to use a faster
+backend when the matching gem is present; every adapter normalizes
+strings to valid UTF-8 before it serializes, so the choice affects
+speed rather than behavior.
+
 ### Reading and writing
 
 #### Overview

data/docs/deepdive/reference/response.md

@@ -0,0 +1,55 @@
+
+## LLM::Response
+
+### Introduction
+
+#### Overview
+
+Every call returns an
+[`LLM::Response`](https://r.uby.dev/api-docs/llm.rb/LLM/Response.html).
+It is one normalized shape that a provider, an endpoint, or a context
+extends with the accessors that response needs, so chat completions,
+embeddings, images, and OCR all share a common surface without losing
+their own.
+
+#### How it works
+
+`LLM::Response` carries the transport response and a little common
+state: `#res` (the underlying HTTP response), `#body` (the parsed body,
+or a raw string), `#ok?`, `#id`, and `#file?`. A name the base class
+does not define falls through to the body, so a provider field such as
+`content` is reachable directly.
+
+A completion response also includes
+[`LLM::Contract::Completion`](https://r.uby.dev/api-docs/llm.rb/LLM/Contract/Completion.html),
+which adds the pieces a chat reply needs:
+
+```ruby
+res = agent.talk "Weather in Paris?", schema: Weather
+
+res.content            # the raw text, or the structured reply
+res.content!           # the reply parsed as an LLM::Object
+res.messages           # the messages this turn produced
+res.usage              # an LLM::Usage (input, output, reasoning, cache, ...)
+res.reasoning_content  # the model's reasoning, when the provider exposes it
+res.model              # the model that answered
+```
+
+#### Why would I use it?
+
+One shape means one set of methods to learn: the same `content!`,
+`usage`, and `messages` work whatever the provider. When an endpoint
+returns something shaped differently, it adds an accessor rather than
+changing the contract.
+
+#### Notes
+
+`#usage` returns an
+[`LLM::Usage`](https://r.uby.dev/api-docs/llm.rb/LLM/Usage.html), so
+token counts are read the same way everywhere; the individual token
+readers (`input_tokens`, `output_tokens`, `cache_read_tokens`, and so
+on) live on the completion contract. Endpoint-specific accessors such
+as `images`, `pages`, `audio`, or `embeddings` come from the adapter
+that extends the response, so they are present only on responses that
+carry that data. `#content!` parses the content as JSON, which is the
+read to use with a schema.

data/docs/deepdive.md

@@ -68,6 +68,9 @@ is the best place to start if you are new to llm.rb.
 - [Built-in tools](deepdive/features/builtin_tools.md)
 - [Concurrency](deepdive/features/concurrency.md)
 - [Embeddings](deepdive/features/embeddings.md)
+- [Files](deepdive/features/files.md)
+- [Moderations](deepdive/features/moderations.md)
+- [Responses](deepdive/features/responses.md)
 - [Database](deepdive/features/database.md)
 - [Console](deepdive/features/console.md)
 
@@ -88,6 +91,7 @@ is the best place to start if you are new to llm.rb.
 
 ## Reference
 
+- [LLM::Response](deepdive/reference/response.md)
 - [LLM::Object](deepdive/reference/object.md)
 - [LLM::Cost](deepdive/reference/cost.md)
 - [Tracer](deepdive/reference/tracer.md)

data/lib/llm/active_record/acts_as_llm.rb

@@ -56,21 +56,23 @@ module LLM::ActiveRecord
 
     module InstanceMethods
       ##
-      # Continues the stored context with new input and flushes it.
+      # Continues the stored context with new input.
+      #
+      # The conversation is saved by {LLM::Step LLM::Step} at each request.
       # @see LLM::Context#talk
       # @return [LLM::Response]
       def talk(...)
-        options = self.class.llm_plugin_options
-        ctx.talk(...).tap { Utils.save!(self, ctx, options) }
+        ctx.talk(...)
       end
 
       ##
-      # Continues the stored context with new input and flushes it.
+      # Continues the stored context with new input.
+      #
+      # The conversation is saved by {LLM::Step LLM::Step} at each request.
       # @see LLM::Context#ask
       # @return [LLM::Response]
       def ask(...)
-        options = self.class.llm_plugin_options
-        ctx.ask(...).tap { Utils.save!(self, ctx, options) }
+        ctx.ask(...)
       end
 
       ##
@@ -101,7 +103,8 @@ module LLM::ActiveRecord
       # unordered - conversation order is `position`, not `id` - and its rows
       # are {LLM::ActiveRecord::Message} records until
       # {LLM::ActiveRecord::Message#unwrap!} turns them back into
-      # {LLM::Message} objects.
+      # {LLM::Message} objects. {#messages!} is the runtime's own collection,
+      # for a caller who wants that instead.
       # @see LLM::Context#messages
       # @return [ActiveRecord::Relation, LLM::Buffer]
       def messages
@@ -113,6 +116,20 @@ module LLM::ActiveRecord
         end
       end
 
+      ##
+      # Returns the messages the runtime holds, whatever the storage format is.
+      #
+      # This is what {#messages} answers with for every format but jsonb, and
+      # it is the way back to it for a jsonb record. The difference between
+      # the two is what each reads: this reads what the context holds,
+      # including state that has not been saved, where {#messages} reads the
+      # column.
+      # @see LLM::Context#messages
+      # @return [LLM::Buffer]
+      def messages!
+        ctx.messages
+      end
+
       ##
       # @note The bang keeps the ActiveRecord and Sequel wrappers aligned.
       # @see LLM::Context#model

data/lib/llm/agent.rb

@@ -441,7 +441,10 @@ module LLM
       # knows its own API, and how often it recovers from a
       # rate limit rather than failing outright.
       params[:retry_budget] = llm.retry_budget if params[:retry_budget].equal?(UNDEFINED)
-      @ctx = LLM::Context.new(llm, params)
+      ##
+      # The context is bound to the same record the agent holds, so code
+      # written against a context alone can find it through `ctx.record`.
+      @ctx = LLM::Context.new(llm, params.merge(record: @record).compact)
       @path and File.readable?(@path) ? @ctx.restore(path:) : nil
     end
 

data/lib/llm/context.rb

@@ -222,6 +222,7 @@ module LLM
         @messages.concat(prompt)
       end
       @messages.concat([res.choices[-1]].compact)
+      stream.on_step(self, res)
       res
     ensure
       @owner = nil

data/lib/llm/sequel/plugin.rb

@@ -155,21 +155,23 @@ module LLM::Sequel
     Utils = Plugin::Utils
 
     ##
-    # Continues the stored context with new input and flushes it.
+    # Continues the stored context with new input.
+    #
+    # The conversation is saved by {LLM::Step LLM::Step} at each request.
     # @see LLM::Context#talk
     # @return [LLM::Response]
     def talk(...)
-      options = self.class.llm_plugin_options
-      ctx.talk(...).tap { Utils.save!(self, ctx, options) }
+      ctx.talk(...)
     end
 
     ##
-    # Continues the stored context with new input and flushes it.
+    # Continues the stored context with new input.
+    #
+    # The conversation is saved by {LLM::Step LLM::Step} at each request.
     # @see LLM::Context#ask
     # @return [LLM::Response]
     def ask(...)
-      options = self.class.llm_plugin_options
-      ctx.ask(...).tap { Utils.save!(self, ctx, options) }
+      ctx.ask(...)
     end
 
     ##
@@ -199,6 +201,17 @@ module LLM::Sequel
       ctx.messages
     end
 
+    ##
+    # @note The same call {#messages} makes here, and here for the sake of the
+    #   name: on {LLM::ActiveRecord} the two differ for a jsonb record, and a
+    #   wrapper that answered to a name the other one does not is the next
+    #   reader's surprise.
+    # @see LLM::Context#messages
+    # @return [LLM::Buffer]
+    def messages!
+      ctx.messages
+    end
+
     ##
     # @note The bang is used because Sequel reserves `model` for the
     #   underlying model class on instances.

data/lib/llm/step.rb

@@ -0,0 +1,59 @@
+# frozen_string_literal: true
+
+module LLM
+  ##
+  # Writes the conversation down when a request completes.
+  #
+  # {LLM::Step LLM::Step} is prepended onto a stream's singleton class by
+  # {LLM::Stream.try}, so a conversation bound to a record is saved as it goes:
+  # a turn that is interrupted - by a crash, by a deploy, or by a provider that
+  # refuses - can be continued rather than started over.
+  #
+  # It saves through the record the context is bound to ({LLM::Context#record}),
+  # and it knows both the ORMs llm.rb speaks: an ActiveRecord model is saved
+  # with {LLM::ActiveRecord::Utils.save!} and a Sequel model with
+  # {LLM::Sequel::Plugin::Utils.save!}. A context with no record - which is
+  # every context that is not backed by a model - has nothing to save, and
+  # neither has a record that carries neither plugin, so the step is passed
+  # along untouched.
+  #
+  # All of this is one method on purpose. The module is put onto an object the
+  # caller owns, and a helper would be one more name on that object for as long
+  # as it lives.
+  #
+  # The write is best effort: a failure is swallowed rather than raised, because
+  # this runs inside the request that has just completed, and a stale checkpoint
+  # is a smaller failure than a turn that ends because the database was busy.
+  # Only the write is covered, so a stream's own `on_step` is reached whether or
+  # not the write lands - a step that a tracer hears about does not depend on a
+  # database having been free.
+  #
+  # @see LLM::Stream#on_step
+  # @see LLM::ActiveRecord::Utils.save!
+  module Step
+    ##
+    # Saves the conversation the context holds, then passes the step along.
+    #
+    # @param [LLM::Context] ctx
+    #  The context the request belongs to
+    # @param [LLM::Response] res
+    #  The response for the request that completed
+    # @return [nil]
+    def on_step(ctx, res)
+      record = ctx.record
+      if record and record.class.respond_to?(:llm_plugin_options)
+        options = record.class.llm_plugin_options
+        begin
+          if defined?(::ActiveRecord::Base) and ::ActiveRecord::Base === record
+            LLM::ActiveRecord::Utils.save!(record, ctx, options)
+          elsif defined?(::Sequel::Model) and ::Sequel::Model === record
+            LLM::Sequel::Plugin::Utils.save!(record, ctx, options)
+          end
+        rescue
+          nil
+        end
+      end
+      super
+    end
+  end
+end

data/lib/llm/stream.rb

@@ -37,10 +37,12 @@ module LLM
   #
   # @see LLM::Agent Where streams are typically attached
   # @see LLM::Context Where streams are bound per-turn
+  # @see LLM::Step An extension that saves the conversation at each step
   class Stream
     require_relative "stream/queue"
     require_relative "stream/io"
     require_relative "stream/disabled"
+    require_relative "step"
 
     ##
     # This method will try to convert its argument into
@@ -51,18 +53,22 @@ module LLM
     # objects, IO objects who implement `#<<`, true, false,
     # and nil. Anything else raises a TypeError.
     #
+    # The stream it answers with is ready to be told about a step:
+    # {LLM::Step LLM::Step} is prepended onto it here, once, rather than by
+    # every caller that means to save a conversation.
+    #
     # @raise [TypeError]
     # @param [LLM::Stream, #<<, Boolean, NilClass] obj
     # @return [LLM::Stream]
     def self.try(obj, extra: {})
       if LLM::Stream === obj
-        obj.tap { _1.extra.merge!(extra) }
+        obj.tap { _1.extra.merge!(extra) }.tap { _1.singleton_class.prepend(LLM::Step) }
       elsif obj.respond_to?(:<<)
-        LLM::Stream::IO.new(obj).tap { _1.extra.merge!(extra) }
+        LLM::Stream::IO.new(obj).tap { _1.extra.merge!(extra) }.tap { _1.singleton_class.prepend(LLM::Step) }
       elsif obj == true
-        LLM::Stream.new.tap { _1.extra.merge!(extra) }
+        LLM::Stream.new.tap { _1.extra.merge!(extra) }.tap { _1.singleton_class.prepend(LLM::Step) }
       elsif obj.nil? || obj == false
-        LLM::Stream::Disabled.new.tap { _1.extra.merge!(extra) }
+        LLM::Stream::Disabled.new.tap { _1.extra.merge!(extra) }.tap { _1.singleton_class.prepend(LLM::Step) }
       else
         raise TypeError, "invalid stream object"
       end
@@ -152,6 +158,32 @@ module LLM
       nil
     end
 
+    ##
+    # Called when a request in a turn has completed.
+    #
+    # This is the boundary between one request and the next: the response
+    # is in the context's message history, and a turn that asked for tools
+    # runs them after this while a turn that is finished ends here.
+    #
+    # It is emitted once per successful request, so a request that is
+    # retried after a rate limit or a timeout calls this when it lands and
+    # not once per attempt. It is also emitted when the stream is disabled:
+    # a disabled stream has no output to report, but it is still told that
+    # a step ended.
+    #
+    # The context is passed with the response so that a stream does not have
+    # to reach for it: {LLM::Step LLM::Step}, which saves the conversation the
+    # context holds, is written against this callback alone.
+    #
+    # @param [LLM::Context] ctx
+    #  The context the request belongs to
+    # @param [LLM::Response] res
+    #  The response for the request that completed
+    # @return [nil]
+    def on_step(ctx, res)
+      nil
+    end
+
     ##
     # Called before a context transformer rewrites a prompt.
     # @param [LLM::Transformer] transformer

data/lib/llm/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module LLM
-  VERSION = "16.0.0.pre.3"
+  VERSION = "16.0.0.pre.4"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: llm.rb
 version: !ruby/object:Gem::Version
-  version: 16.0.0.pre.3
+  version: 16.0.0.pre.4
 platform: ruby
 authors:
 - Robert Gleeson
@@ -400,6 +400,9 @@ files:
 - docs/deepdive/features/console.md
 - docs/deepdive/features/database.md
 - docs/deepdive/features/embeddings.md
+- docs/deepdive/features/files.md
+- docs/deepdive/features/moderations.md
+- docs/deepdive/features/responses.md
 - docs/deepdive/fundamentals/agents.md
 - docs/deepdive/fundamentals/providers.md
 - docs/deepdive/fundamentals/schema.md
@@ -414,6 +417,7 @@ files:
 - docs/deepdive/reference/cost.md
 - docs/deepdive/reference/model_registry.md
 - docs/deepdive/reference/object.md
+- docs/deepdive/reference/response.md
 - docs/deepdive/reference/tracer.md
 - lib/llm.rb
 - lib/llm/a2a.rb
@@ -642,6 +646,7 @@ files:
 - lib/llm/server_tool.rb
 - lib/llm/session.rb
 - lib/llm/skill.rb
+- lib/llm/step.rb
 - lib/llm/stream.rb
 - lib/llm/stream/disabled.rb
 - lib/llm/stream/io.rb