llm.rb 16.0.0.pre.4 → 16.0.0.pre.5

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a83847d3c33a48980bd9740d7aff397db757ec91224440a97c10240c0ae3a290
-  data.tar.gz: 402d6ea4a68fc987e3597d1b6dd84089f2804c7774ef35e38fb8fc424b0e62b0
+  metadata.gz: f5558a0514d6bd6b9648f743a308ce8e6dbb3802b8709edc6caf45d7c1ab3de8
+  data.tar.gz: a5d3178f0c79e152b8032cce974be69edd1e52cc629658034a750183ccb41ea8
 SHA512:
-  metadata.gz: f75dda6e725c6aa48748a407e744c94c006d3db4973cf45064f7f0252fc2bc818ee7a99b564e66c4bad6ad4066bf5214b912a637b3b25b8b9f335d45d226c7e2
-  data.tar.gz: ea3e7fe4a32687c7a34f4b85a0bd4aeee5b7b68387452dd1ac98a3572f1b1f74b4f19ac3ee32b10cf6f53563b9eceb5a8e9292e6001b90728e4997c7b9a301a2
+  metadata.gz: 3a0288112d140d3e0a3e34dc35e570ceb25ad62fd7f3c33a2942cb111432992b7194e18fb406bf55a25a945aff0e89ce055c292a1de50c00b66c801027b95c52
+  data.tar.gz: 454a1c74a59401689c88947714f741ada1d39f6ce95013bf8bdd6fab76690adafd3a4be8af02e9eace2a71ae26c3f4daf3a1751fdf326549f722da888cbcb309

data/CHANGELOG.md

@@ -56,6 +56,29 @@
   has not been saved, where `#messages` reads the column. The Sequel plugin
   answers to `#messages!` too, where it is the same call as `#messages`.
 
+### Agent
+
+* **agent: keep the instructions it injected in step** <br>
+  [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html) now brings
+  the instructions it injected up to date before a request goes out, so a
+  conversation restored from saved state runs on the agent's current
+  instructions rather than on the ones it was saved with. The message is
+  found by a mark the agent puts on it, so a message the caller composed is
+  never replaced, and the mark is written by
+  [`LLM::Message#to_h`](https://r.uby.dev/api-docs/llm.rb/LLM/Message.html#to_h-instance_method)
+  and read back on a restore, which is the conversation the refresh exists
+  for. Before this, the instructions were injected once and the message
+  stayed as it was for the life of the conversation.
+
+* **agent: identify its own instructions by the mark, not by the role** <br>
+  Fix a bug where the agent decided whether it had already injected its
+  instructions by looking for a message with the `system` role. Google gives
+  instructions the `user` role, so the agent did not recognise the message it
+  had already injected and added another copy of the instructions on every
+  turn. It now asks the provider which role instructions are given, and
+  identifies the message it wrote by its mark, because on Google the message
+  a caller seeds a conversation with is a user message too.
+
 ### Console
 
 * **console: erase with the backspace key on OpenBSD** <br>
@@ -142,6 +165,18 @@
   once, when the whole turn had finished, so an interrupted turn persisted
   nothing.
 
+### Prompt
+
+* **prompt: let a message carry fields a caller attaches to it** <br>
+  [`LLM::Prompt#system`](https://r.uby.dev/api-docs/llm.rb/LLM/Prompt.html#system-instance_method),
+  [`#user`](https://r.uby.dev/api-docs/llm.rb/LLM/Prompt.html#user-instance_method),
+  [`#developer`](https://r.uby.dev/api-docs/llm.rb/LLM/Prompt.html#developer-instance_method),
+  and [`#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Prompt.html#talk-instance_method),
+  along with its `#chat` alias, now take an `extra:` keyword, so a message a
+  caller composes can carry fields of its own. The runtime reads those fields
+  back rather than inferring provenance from a role, which is how an agent
+  tells the instructions it injected from a message the caller wrote.
+
 ### Provider
 
 * **deepseek: support image attachments in chat completions** <br>

data/README.md

@@ -78,7 +78,8 @@ agent.talk "hello world"
 Streams can be simple IO objects or subclasses of
 [`LLM::Stream`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html)
 with structured callbacks for content,
-reasoning, tool calls, tool returns, and compaction.
+reasoning, tool calls, tool returns, the boundary a completed
+request marks, and compaction.
 Streams can also observe message transformers, which rewrite
 outgoing messages before they reach the provider.
 
@@ -102,6 +103,11 @@ class MyStream < LLM::Stream
   def on_tool_return(tool, result)
   end
 
+  # A request has completed: the response is in the conversation, and
+  # any tools it asked for run after this.
+  def on_step(ctx, res)
+  end
+
   # Before a transformer rewrites an outgoing message.
   def on_transform(transformer)
   end

data/docs/deepdive/advanced/context.md

@@ -424,7 +424,9 @@ The block receives the prompt object when it takes an argument, and
 otherwise runs in the prompt's context:
 [`LLM::Prompt#to_a`](https://r.uby.dev/api-docs/llm.rb/LLM/Prompt.html#to_a)
 returns the messages in order, and two prompts are equal when their
-messages match.
+messages match. Each of `system`, `user`, `developer`, and `talk`
+accepts an `extra:` hash, which is attached to the message it builds,
+so a caller can carry a field of its own alongside the content.
 
 #### Why would I use it?
 

data/docs/deepdive/features/console.md

@@ -66,9 +66,9 @@ should use the `console` names.
 When `bin/llm.rb` starts, it looks for an `AGENTS.md` file in the
 current working directory. If one exists, its contents become the
 agent's instructions for the session, so a project can describe how it
-wants the agent to work without passing flags. The instructions are
-injected once, so a resumed session that already has a system message
-keeps the one it has.
+wants the agent to work without passing flags. The agent brings its
+instructions up to date before each request, so a resumed session runs
+on the current file rather than the one it was saved with.
 
 The user-message label is exposed through
 [`LLM::Console#sender`](https://r.uby.dev/api-docs/llm.rb/LLM/Console.html#sender-instance_method),

data/docs/deepdive/features/database.md

@@ -34,13 +34,23 @@ and
 for serialization and
 [`LLM::Context#restore`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html#restore)
 for deserialization. Save writes the current state, restore loads
-it back and picks up where the conversation left off. The ORM
-wrappers automate this; each
-[`LLM::Agent#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#talk)
-or
-[`LLM::Agent#ask`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#ask)
-call persists the
-updated state back to the column automatically.
+it back and picks up where the conversation left off.
+
+The ORM wrappers write on a different boundary than the file does, and
+the two are worth telling apart.
+[`LLM::Step`](https://r.uby.dev/api-docs/llm.rb/LLM/Step.html) is
+prepended onto the stream, and it saves the conversation through the
+record the context is bound to each time
+[`LLM::Stream#on_step`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_step-instance_method)
+fires - after a response is in the conversation and before the tools it
+asked for run. A turn that runs tools before it answers is therefore
+saved more than once, and an application that writes it down can
+continue an interrupted turn instead of redoing it.
+
+A `path:` has no record behind it to write to, so that one saves once
+per turn. The difference is what each is for: the column is a
+checkpoint after every completed request, and the file is a session at
+the end of one.
 
 #### Why would I use it?
 
@@ -242,12 +252,10 @@ first call, a fresh agent is created and the conversation starts
 from scratch.
 
 On subsequent calls, the stored state is restored and the
-conversation continues. Every
-[`LLM::Agent#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#talk)
-or
-[`LLM::Agent#ask`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#ask)
-call persists the
-updated state back to the column automatically.
+conversation continues. The column is written after each completed
+request rather than at the end of a turn, so a turn that runs tools
+before it answers is saved more than once - and a turn that is
+interrupted can be resumed from its last completed request.
 
 The `data_column:` option lets you use a different column name.
 The `format:` option controls the storage type. Use `:string` for
@@ -327,7 +335,7 @@ agent.talk "perform research"
 #### Why would I use it?
 
 The ActiveRecord wrapper integrates with your existing models.
-Agent state is automatically persisted after each conversation turn
+Agent state is automatically persisted after each completed request,
 using the same `create!`, `save!`, and query methods you already
 use.
 
@@ -447,11 +455,8 @@ the plugin in the model class. The `data` column stores
 the full agent state as JSON, same structure as ActiveRecord.
 On first call, a fresh agent is created. On subsequent calls,
 the stored state is restored and the conversation continues.
-Every
-[`LLM::Agent#talk`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#talk)
-or
-[`LLM::Agent#ask`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#ask)
-call persists the updated state.
+The state is written after each completed request, as it is under
+ActiveRecord.
 
 The `data_column:` and `format:` options work identically to
 ActiveRecord. For `:jsonb`, Sequel loads the `pg_json` extension

data/docs/deepdive/fundamentals/agents.md

@@ -21,8 +21,9 @@ the model responds, and if it requests tools the agent
 executes them automatically and feeds the results back. The tool
 loop can be bounded with
 [`LLM::Agent.tool_budget`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#tool_budget-class_method)
-(see the Tool budget section). Instructions are injected once
-unless a system message is already present.
+(see the Tool budget section). Instructions are injected once and
+brought up to date from the agent before each request, unless the
+prompt already carries a system message of the caller's own.
 
 #### Why would I use it?
 

data/docs/deepdive/fundamentals/stream.md

@@ -67,6 +67,17 @@ receives tokens as they arrive.
 fires when the model requests a tool.
 [`LLM::Stream#on_tool_return`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_tool_return)
 fires when the tool completes.
+[`LLM::Stream#on_step`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_step-instance_method)
+fires when a request completes, which is the boundary between one
+request and the next: the response is in the conversation, and a
+turn that asked for tools runs them after this while a turn that is
+finished ends here. That boundary is where a conversation is whole,
+and the runtime's
+[`LLM::Step`](https://r.uby.dev/api-docs/llm.rb/LLM/Step.html) is
+written against it - it is prepended onto a stream, and saves the
+conversation through the record the context is bound to, so a turn
+that is interrupted can be continued from its last completed request
+rather than started over.
 [`LLM::Stream#on_retry`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_retry)
 fires each time a failed request is retried. Compaction hooks
 let you show progress or log what was trimmed. Skill hooks bracket a
@@ -96,6 +107,11 @@ class MyStream < LLM::Stream
   def on_tool_return(tool, result)
   end
 
+  # A request has completed: the response is in the conversation, and
+  # any tools it asked for run after this.
+  def on_step(ctx, res)
+  end
+
   # Before a transformer rewrites an outgoing message.
   def on_transform(transformer)
   end
@@ -135,9 +151,3 @@ agent.talk "Explain Ruby fibers."
 A stream subclass gives you visibility into more than just content chunks. React to tool
 calls as they happen, show compaction progress, or integrate with
 an existing observability stack.
-
-#### Notes
-
-The IO-like form is equivalent to
-[`LLM::Stream#on_content`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html#on_content)
-and does not include the other hooks.

data/lib/llm/agent.rb

@@ -15,7 +15,10 @@ module LLM
   # instead of leaving tool loops to the caller.
   #
   # **Notes:**
-  # * Instructions are injected once unless a system message is already present.
+  # * Instructions are injected once, and kept in step after that. A
+  #   restored conversation carries the instructions that were current
+  #   when it started, and the message is brought up to date before the
+  #   next request goes out.
   # * An agent automatically executes tool loops (unlike {LLM::Context LLM::Context}).
   # * The tool loop can be bounded with `tool_budget`. Once the budget is
   #   spent, no further tool calls are run for that turn: the agent sends an
@@ -53,6 +56,14 @@ module LLM
     UNDEFINED = Object.new
     private_constant :UNDEFINED
 
+    ##
+    # Marks the message an agent injects as its own, so that a later
+    # turn can tell it from a message the caller composed. The role
+    # cannot do that: it is true of whatever the caller put there.
+    # @api private
+    INSTRUCTIONS = {instructions: true}.freeze
+    private_constant :INSTRUCTIONS
+
     ##
     # Ugh :)
     # @api private
@@ -802,27 +813,92 @@ module LLM
     # @return [LLM::Prompt]
     def apply_instructions(new_prompt)
       return new_prompt unless @instructions
+      ##
+      # A prompt that carries a system message of its own is the
+      # caller's, and the instructions already in the conversation are
+      # left as they are. The context is deliberately not consulted: a
+      # restored context always holds the message this exists to bring
+      # up to date, which is what `inject_instructions?` tests for.
+      refresh_instructions! unless carries_system_message?(new_prompt)
       if LLM::Prompt === new_prompt
-        new_prompt.system(@instructions) if inject_instructions?(new_prompt)
+        new_prompt.system(@instructions, extra: INSTRUCTIONS) if inject_instructions?(new_prompt)
         new_prompt
       else
         prompt do
-          _1.system(@instructions) if inject_instructions?
+          _1.system(@instructions, extra: INSTRUCTIONS) if inject_instructions?
           _1.user(new_prompt)
         end
       end
     end
 
+    ##
+    # Brings the instructions stored in the conversation up to date.
+    #
+    # The message is identified by the mark the agent put on it when it
+    # injected it, and not by its role or its position. A role is true
+    # of whatever the caller put there - on a provider whose system role
+    # is `:user`, the first message of a conversation the caller seeded
+    # is a user message too - so a rule written on either of those
+    # replaces a message that was never the agent's to replace.
+    #
+    # A message with no mark is left alone, which is the safe direction:
+    # a conversation written before the mark existed keeps its stale
+    # instructions rather than losing a message. Stale is recoverable.
+    #
+    # An agent with no instructions returns first. The caller reaches
+    # this through `apply_instructions`, which already returns early for
+    # the same reason, and the guard is repeated here so that the method
+    # cannot replace a message with nil however it is reached.
+    #
+    # Comparing content rather than the message is deliberate.
+    # {LLM::Message#==} compares everything a message carries apart from
+    # its id, which drags fields that have nothing to do with
+    # instructions into the comparison. The mark is carried onto the
+    # replacement so that the turn after this one can still find it, and
+    # so is the role the message already had: it is the provider's, and
+    # rebuilding it here would undo the reason the replacement is safe.
+    # @api private
+    # @return [void]
+    def refresh_instructions!
+      return unless @instructions
+      message = @ctx.messages.first
+      return unless message&.extra&.key?(:instructions)
+      return if message.content == @instructions
+      @ctx.messages.replace(
+        [LLM::Message.new(message.role, @instructions, message.extra), *@ctx.messages.drop(1)]
+      )
+    end
+
+    ##
+    # Returns true when a prompt carries a system message of its own.
+    #
+    # This is the half of `inject_instructions?` that asks about the
+    # prompt rather than about the context, and it is the one a refresh
+    # needs. The context half says "do not inject again", which is not
+    # the same question as "whose instructions are these".
+    #
+    # The role asked about is the provider's: a prompt's `system` is
+    # built with the role the provider gives instructions, which is not
+    # the string "system" everywhere.
+    # @param [LLM::Prompt, Object] prompt
+    # @return [Boolean]
+    def carries_system_message?(prompt)
+      role = @llm.system_role.to_s
+      LLM::Prompt === prompt && prompt.to_a.any? { _1.role.to_s == role }
+    end
+
     ##
     # Returns true when agent instructions should be injected for the turn.
-    # Instructions are injected once unless a system message is already
-    # present in the existing context or the prompt being sent.
+    # Instructions are injected once unless a message with the provider's
+    # system role is already present in the existing context or the prompt
+    # being sent.
     # @param [LLM::Prompt, nil] prompt
     # @return [Boolean]
     def inject_instructions?(prompt = nil)
-      return false if @ctx.messages.any?(&:system?)
+      role = @llm.system_role.to_s
+      return false if @ctx.messages.any? { _1.role.to_s == role }
       return true if prompt.nil?
-      !prompt.to_a.any?(&:system?)
+      !prompt.to_a.any? { _1.role.to_s == role }
     end
 
     ##

data/lib/llm/context/deserializer.rb

@@ -40,6 +40,19 @@ class LLM::Context
 
     private
 
+    ##
+    # Rebuilds one message from a payload.
+    #
+    # The fields that describe a message are named here rather than
+    # taken from everything the payload holds, so a field that
+    # {LLM::Message#to_h} writes has to be read here as well - and one
+    # that is not named is dropped on the way in, whatever the payload
+    # carries. `instructions` marks the message an agent injected as its
+    # own, and the agent looks for that mark rather than for a role or a
+    # position when it refreshes the instructions in a restored
+    # conversation.
+    # @param [Hash] payload
+    # @return [LLM::Message]
     def deserialize_message(payload)
       tool_calls = deserialize_tool_calls(payload["tools"])
       returns = deserialize_returns(payload["content"]) if returns.nil?
@@ -47,8 +60,18 @@ class LLM::Context
       usage = payload["usage"]
       reasoning_content = payload["reasoning_content"]
       compaction = payload["compaction"]
+      instructions = payload["instructions"]
       id = payload["id"]
-      extra = {tool_calls:, original_tool_calls:, tools: @params[:tools], usage:, reasoning_content:, compaction:, id:}.compact
+      extra = {
+        tool_calls:,
+        original_tool_calls:,
+        tools: @params[:tools],
+        usage:,
+        reasoning_content:,
+        compaction:,
+        instructions:,
+        id:
+      }.compact
       content = returns.nil? ? deserialize_content(payload["content"]) : returns
       LLM::Message.new(payload["role"], content, extra)
     end

data/lib/llm/message.rb

@@ -46,6 +46,12 @@ module LLM
 
     ##
     # Returns a Hash representation of the message.
+    #
+    # This is the whole of a message as far as storage is concerned:
+    # {LLM::Context#to_h} maps its messages through here, and restore
+    # reads them back from the same list. A field that is not named
+    # here does not survive a save, so a field the runtime relies on
+    # after a restore has to be named.
     # @return [Hash]
     def to_h
       {
@@ -54,6 +60,7 @@ module LLM
         content:,
         reasoning_content:,
         compaction: extra.compaction,
+        instructions: extra.instructions,
         tools: extra.tool_calls&.map { LLM::Object === _1 ? _1.to_h : _1 },
         usage:,
         original_tool_calls: extra.original_tool_calls

data/lib/llm/prompt.rb

@@ -40,40 +40,51 @@ class LLM::Prompt
   #  The message
   # @param [Symbol] role
   #  The role (eg user, system)
+  # @param [Hash, nil] extra
+  #  Fields carried alongside the message. A caller that composes a
+  #  message of its own can mark it here, and the runtime reads the
+  #  mark back rather than inferring it from the role.
   # @return [void]
-  def talk(content, role: @provider.user_role)
+  def talk(content, role: @provider.user_role, extra: nil)
     role = case role.to_sym
     when :system then @provider.system_role
     when :user then @provider.user_role
     when :developer then @provider.developer_role
     else role
     end
-    @buffer << LLM::Message.new(role, content)
+    message = extra ? LLM::Message.new(role, content, extra) : LLM::Message.new(role, content)
+    @buffer << message
   end
   alias_method :chat, :talk
 
   ##
   # @param [String] content
   #  The message content
+  # @param [Hash, nil] extra
+  #  Fields carried alongside the message
   # @return [void]
-  def user(content)
-    talk(content, role: @provider.user_role)
+  def user(content, extra: nil)
+    talk(content, role: @provider.user_role, extra:)
   end
 
   ##
   # @param [String] content
   #  The message content
+  # @param [Hash, nil] extra
+  #  Fields carried alongside the message
   # @return [void]
-  def system(content)
-    talk(content, role: @provider.system_role)
+  def system(content, extra: nil)
+    talk(content, role: @provider.system_role, extra:)
   end
 
   ##
   # @param [String] content
   #  The message content
+  # @param [Hash, nil] extra
+  #  Fields carried alongside the message
   # @return [void]
-  def developer(content)
-    talk(content, role: @provider.developer_role)
+  def developer(content, extra: nil)
+    talk(content, role: @provider.developer_role, extra:)
   end
 
   ##

data/lib/llm/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module LLM
-  VERSION = "16.0.0.pre.4"
+  VERSION = "16.0.0.pre.5"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: llm.rb
 version: !ruby/object:Gem::Version
-  version: 16.0.0.pre.4
+  version: 16.0.0.pre.5
 platform: ruby
 authors:
 - Robert Gleeson