layered-assistant-rails 0.7.0 → 0.8.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: '0849058555cc1dd6e7a2dffb73b3919086413374156a1c3300999eaa543c8a4b'
-  data.tar.gz: fa46c8ac4e95587216f467be30eb25d8c01c427d642b589b63a0ab849984e4fb
+  metadata.gz: 3c4266f3d5e33f449619fe77a6e9b0819bcc22c038ffe3498c96ea56549faa80
+  data.tar.gz: 6b4e8a671a99e0e0cfd987b631dc316616743f5b3dbe3b2d3370d8c0a7f1a614
 SHA512:
-  metadata.gz: 6b4ae1f8d54d5a753b9560fd532cb04e80cbe1959a728deaa9dfaaa82f83d5a7a65edef6a341a61879f73d1c22d76785692ef430242c7e8f995abc19754bac83
-  data.tar.gz: d43a15e7ed0b30c8a85dc578a4887b46b537d37e2d881f83fc2135539e60a8b4751dc3bc06fba37ed75e3ee0281b2fa52e089594abf91d31017a94156b065c0a
+  metadata.gz: '0681356f7a030b95dcc62e0b589647b4e4c650fa1683dbf2b98ddceefd89cb74ea1b73578053de064c32420faefd21c2bd28cd7ef5ffc1660d39cd991d954713'
+  data.tar.gz: df9c6fde84b9897e4833fd7beedcb0557c319f6603b663af10f9fa1b4ce1c69ddf4041dd9aa7aea335c09b2b26764aecf53fdbb3ac21d3871796f11fdcc0bc28

data/README.md

@@ -251,13 +251,15 @@ assistant.update!(tool_names: [ "weather", "order-lookup" ])
 | `argument` | An argument the model may supply: `argument :name, :type, required:, description:, enum:, items:` |
 | `tool_name` | The name the model calls the tool by. Defaults to the class name without its `Tool` suffix, namespaces hyphenated: `Weather::ForecastTool` becomes `weather-forecast` |
 | `self.public =` | Whether the tool may be offered to a public assistant. `false` by default - see below |
+| `permit` | A block deciding which conversations may call the tool - see [Who may call a tool](#who-may-call-a-tool) |
+| `consent` | `:always` to put each call to the person talking before it runs. `:never` by default - see [Asking before a tool runs](#asking-before-a-tool-runs) |
 
 Argument types are `:string`, `:integer`, `:number`, `:boolean`, `:array` and
 `:object`. An `:array` takes `items:` to name its element type.
 
 Subclass a tool to share logic and the declarations come with it: the child
-inherits its parent's `description`, `public` flag and arguments, adds any
-arguments of its own, and may redeclare one by name to narrow it. The name is
+inherits its parent's `description`, `public` flag, `permit` block, `consent`
+setting and arguments, adds any arguments of its own, and may redeclare one by name to narrow it. The name is
 the exception - the child derives its own from its class name, since two tools
 answering to one name would collide in the registry.
 
@@ -285,6 +287,79 @@ def call(reference:)
 end
 ```
 
+### Who may call a tool
+
+Which assistants have a tool is a configuration decision; who may call it is a
+runtime one. A `permit` block narrows a tool to the conversations that satisfy
+it:
+
+```ruby
+class RefundTool < Layered::Assistant::Tool
+  description "Refund an order."
+
+  permit { |conversation| conversation.user&.admin? }
+end
+```
+
+An unpermitted tool is left out of the definitions sent to the provider, so
+the model is never told it exists, and is refused if it asks for it anyway
+from an earlier turn's history. The block is inherited like the other
+declarations, so a tool subclassed to share logic keeps its parent's policy
+unless it declares its own.
+
+To apply one rule across every tool, configure an `authorize_tool` block in
+your initialiser. It is given the tool class and the conversation, and
+returning falsey withholds the tool:
+
+```ruby
+Layered::Assistant.authorize_tool do |tool, conversation|
+  conversation.user&.permitted_tools&.include?(tool.tool_name)
+end
+```
+
+Unlike `authorize`, which guards the engine's routes, this is left open when
+unconfigured: tool calls already sit behind that block and behind the set of
+tools each assistant has been given. A block that raises denies the tool and
+logs, rather than handing it over or failing the response.
+
+### Asking before a tool runs
+
+Reads are usually fine unattended. Anything that writes, spends or sends is
+worth putting to the person talking first:
+
+```ruby
+class RefundTool < Layered::Assistant::Tool
+  description "Refund an order."
+  consent :always
+
+  argument :reference, :string, required: true
+
+  def call(reference:)
+    owner.orders.find_by!(reference: reference).refund!
+  end
+end
+```
+
+The call is recorded in the conversation with its arguments shown and nothing
+in it, and the response stops there - the composer stays disabled, and the
+call waits as long as it needs to, surviving a reload. Approving runs the tool
+and the response picks up where it left off. Declining reports the refusal to
+the model as the tool's result, so it can say something useful about being
+turned down rather than the conversation dead-ending. Stopping the response
+declines whatever is outstanding, including a call that has been approved but
+has yet to run - so a tool that writes, spends or sends does not slip through
+after the Stop. A call whose tool is already running is left to finish, its
+result recorded, and the response is still not picked back up.
+
+Calls in the same batch that need no consent still run while one waits: each
+call stands on its own.
+
+A tool that asks for consent needs both a user and an owner, so it is never
+offered to a public assistant - even one an anonymous visitor happens to be
+signed in for. There is nobody to ask an anonymous visitor, and a decision is
+recorded through the owner-scoped route a conversation with no owner cannot
+reach. The tool is withheld rather than offered and then stuck waiting.
+
 ### Tools and public assistants
 
 A conversation with a public assistant has no owner - it belongs to an
@@ -319,7 +394,8 @@ composer stays disabled until a response completes without asking for
 anything, and `max_tool_cycles` (default 10) bounds the loop.
 
 Results are shown in the conversation as a collapsible panel naming the tool,
-with its input and output.
+with its input and output. A call waiting to be approved holds the loop where
+it is until it has been answered.
 
 ## Configuration
 

data/app/controllers/concerns/layered/assistant/message_creation.rb

@@ -1,9 +1,24 @@
 module Layered
   module Assistant
     module MessageCreation
+      WAITING_ON_TOOL_CALL = "A tool call is waiting on you. Answer it before sending another message.".freeze
+
       private
 
       def create_messages_for(conversation:, content:, model_id:)
+        # A tool call that has yet to reach an answer holds the conversation.
+        # The composer is disabled while it waits, so anything arriving here is
+        # a stale tab - and answering it would send the model a call with no
+        # result yet, which the provider rejects. Said rather than dropped, or
+        # the message would vanish with nothing to explain it.
+        if conversation.unresolved_tool_call?
+          return {
+            message: conversation.messages.new(role: :user, content: content),
+            error: WAITING_ON_TOOL_CALL,
+            responding: true
+          }
+        end
+
         message = conversation.messages.create(
           role: :user,
           content: content,
@@ -37,7 +52,8 @@ module Layered
         {
           message: message,
           assistant_message: assistant_message,
-          error: error
+          error: error,
+          responding: error.nil?
         }
       end
     end

data/app/controllers/layered/assistant/messages_controller.rb

@@ -20,15 +20,16 @@ module Layered
           model_id: model_id
         )
         @message = result[:message]
+        @error = result[:error]
+        @responding = result[:responding]
 
-        unless @message.persisted?
+        unless @message.persisted? || @error
           return head :unprocessable_entity
         end
 
         @assistant_message = result[:assistant_message]
         @models = scoped_models
         @selected_model_id = model_id
-        @error = result[:error]
 
         respond_to do |format|
           format.turbo_stream

data/app/controllers/layered/assistant/panel/messages_controller.rb

@@ -16,15 +16,16 @@ module Layered
             model_id: model_id
           )
           @message = result[:message]
+          @error = result[:error]
+          @responding = result[:responding]
 
-          unless @message.persisted?
+          unless @message.persisted? || @error
             return head :unprocessable_entity
           end
 
           @assistant_message = result[:assistant_message]
           @models = scoped_models
           @selected_model_id = model_id
-          @error = result[:error]
 
           respond_to do |format|
             format.turbo_stream

data/app/controllers/layered/assistant/public/messages_controller.rb

@@ -13,13 +13,13 @@ module Layered
             model_id: @conversation.assistant.default_model_id
           )
           @message = result[:message]
+          @error = result[:error]
+          @responding = result[:responding]
 
-          unless @message.persisted?
+          unless @message.persisted? || @error
             return head :unprocessable_entity
           end
 
-          @error = result[:error]
-
           respond_to do |format|
             format.turbo_stream
           end

data/app/controllers/layered/assistant/public/panel/messages_controller.rb

@@ -15,13 +15,13 @@ module Layered
               model_id: @conversation.assistant.default_model_id
             )
             @message = result[:message]
+            @error = result[:error]
+            @responding = result[:responding]
 
-            unless @message.persisted?
+            unless @message.persisted? || @error
               return head :unprocessable_entity
             end
 
-            @error = result[:error]
-
             respond_to do |format|
               format.turbo_stream
             end

data/app/controllers/layered/assistant/tool_calls_controller.rb

@@ -0,0 +1,34 @@
+module Layered
+  module Assistant
+    # Approving or declining a tool call that asked for consent before it ran.
+    class ToolCallsController < ApplicationController
+      DECISIONS = { "approve" => "approved", "decline" => "declined" }.freeze
+
+      before_action :set_conversation
+
+      def update
+        status = DECISIONS[params[:decision]]
+        return head :unprocessable_entity unless status
+
+        message = @conversation.messages.find(params[:id])
+
+        # Conditional so that two decisions arriving at once cannot both win,
+        # and so a second click or a stale tab changes nothing.
+        decided = Message.where(id: message.id, tool_status: "pending")
+          .update_all(tool_status: status, updated_at: Time.current)
+        return head :no_content if decided.zero?
+
+        message.reload.broadcast_updated
+        Messages::ToolCallJob.perform_later(message.id)
+
+        head :no_content
+      end
+
+      private
+
+      def set_conversation
+        @conversation = scoped(Conversation).find_by!(uid: params[:conversation_id])
+      end
+    end
+  end
+end

data/app/javascript/layered_assistant/composer_controller.js

@@ -7,11 +7,15 @@ export default class extends Controller {
   static targets = ["form", "input", "sendButton", "stopButton"]
   static values = {
     responding: { type: Boolean, default: false },
+    waiting: { type: Boolean, default: false },
     stopUrl: { type: String, default: "" }
   }
 
   connect() {
-    this._onChunkReceived = () => this._resetRespondingTimeout()
+    this._onChunkReceived = () => {
+      this.waitingValue = false
+      this._resetRespondingTimeout()
+    }
     document.addEventListener("assistant:chunk-received", this._onChunkReceived)
     this._applyRespondingState()
     this.updateButtonDisabled()
@@ -26,6 +30,16 @@ export default class extends Controller {
     this._applyRespondingState()
   }
 
+  // A tool call waiting to be approved sends nothing until it is answered,
+  // so the safety timeout is stood down rather than giving up on it.
+  waitingValueChanged() {
+    if (this.waitingValue) {
+      clearTimeout(this._respondingTimeout)
+    } else {
+      this._resetRespondingTimeout()
+    }
+  }
+
   // Enter submits, Shift+Enter is a no-op (default newline),
   // Alt+Enter inserts a newline without submitting.
   submitOnEnter(event) {
@@ -81,9 +95,10 @@ export default class extends Controller {
   }
 
   // Toggle visibility of the Send and Stop buttons. While responding a
-  // 60-second safety timeout resets the composer in case the server
+  // 30-second safety timeout resets the composer in case the server
   // never signals completion. The timeout is reset each time a chunk
-  // is received so long-running responses are not interrupted.
+  // is received so long-running responses are not interrupted, and is
+  // not armed at all while a tool call waits to be approved.
   _applyRespondingState() {
     clearTimeout(this._respondingTimeout)
 
@@ -104,7 +119,7 @@ export default class extends Controller {
   }
 
   _resetRespondingTimeout() {
-    if (!this.respondingValue) return
+    if (!this.respondingValue || this.waitingValue) return
     clearTimeout(this._respondingTimeout)
     this._respondingTimeout = setTimeout(() => {
       this.respondingValue = false

data/app/javascript/layered_assistant/message_streaming.js

@@ -18,10 +18,19 @@ const pendingRender = new WeakMap()
 
 Turbo.StreamActions.enable_composer = function () {
   this.targetElements.forEach((form) => {
+    form.setAttribute("data-composer-waiting-value", "false")
     form.setAttribute("data-composer-responding-value", "false")
   })
 }
 
+// A tool call is waiting to be approved: the composer stays disabled, and
+// holds there for as long as it takes rather than timing out.
+Turbo.StreamActions.wait_composer = function () {
+  this.targetElements.forEach((form) => {
+    form.setAttribute("data-composer-waiting-value", "true")
+  })
+}
+
 function syncStreamingContent(target, markdown) {
   const tmp = document.createElement("div")
   tmp.innerHTML = renderMarkdown(markdown)

data/app/jobs/layered/assistant/messages/tool_call_job.rb

@@ -0,0 +1,22 @@
+module Layered
+  module Assistant
+    module Messages
+      # Carries out a tool call once it has been approved, or writes down the
+      # refusal, then picks the response back up. Runs out of band because a
+      # tool can be slow and the answer streams back like any other.
+      class ToolCallJob < ApplicationJob
+        queue_as :default
+
+        def perform(message_id)
+          message = Message.find(message_id)
+          # Nothing to do for a call still waiting on a decision. A call that
+          # is already answered is not run again, but is still passed on: a
+          # retry may be here because resuming the response is what failed.
+          return if message.consent_pending?
+
+          ToolRunnerService.new.resolve(message: message)
+        end
+      end
+    end
+  end
+end

data/app/models/layered/assistant/conversation.rb

@@ -42,12 +42,50 @@ module Layered
         "New conversation"
       end
 
+      # The composer stays disabled while either is true: the assistant is
+      # still writing, or a tool call has yet to reach an answer.
       def responding?
+        generating? || unresolved_tool_call?
+      end
+
+      def generating?
         messages.where(role: :assistant, stopped: false, output_tokens: nil).exists?
       end
 
+      # Whether the response was stopped where it stands. The latest assistant
+      # message carries the mark, so a fresh turn clears it.
+      def stopped?
+        last_assistant_message&.stopped? || false
+      end
+
+      def awaiting_consent?
+        pending_tool_calls.exists?
+      end
+
+      def pending_tool_calls
+        messages.where(role: :tool, tool_status: :pending)
+      end
+
+      # A call that was put to the person talking and has yet to reach an
+      # answer. The decision is recorded before the result is written, so a
+      # call that has been answered is still unresolved until the job says
+      # what came of it - a refusal included. Nothing may be sent to the model
+      # until every one of them holds a result.
+      def unresolved_tool_calls
+        messages.where(role: :tool, content: nil).where.not(tool_status: nil)
+      end
+
+      def unresolved_tool_call?
+        unresolved_tool_calls.exists?
+      end
+
       def stop_response!
         with_lock do
+          # Stopping while a tool call is unanswered is an answer: the calls
+          # are abandoned and the response is not picked back up, which would
+          # only ask the model to try again.
+          return abandon_unresolved_tool_calls! if unresolved_tool_call?
+
           message = messages.where(role: :assistant, stopped: false).order(created_at: :desc).first
           return false unless message
 
@@ -83,6 +121,31 @@ module Layered
 
       private
 
+      def abandon_unresolved_tool_calls!
+        last = nil
+
+        unresolved_tool_calls.each do |message|
+          next unless ToolRunnerService.abandon(message)
+
+          message.broadcast_updated
+          last = message
+        end
+
+        # The message that asked for the tools is already complete, with its
+        # real token counts, so it is marked stopped without being estimated
+        # over. That mark is what keeps an approved call still running, or a
+        # job retried later, from picking the response back up.
+        last_assistant_message&.update!(stopped: true)
+
+        update_token_totals!
+        last&.broadcast_response_complete
+        true
+      end
+
+      def last_assistant_message
+        messages.where(role: :assistant).order(created_at: :desc, id: :desc).first
+      end
+
       def create_system_message
         prompt = SystemPromptService.new.call(assistant: assistant)
         return if prompt.blank?

data/app/models/layered/assistant/message.rb

@@ -15,8 +15,21 @@ module Layered
         tool: "tool"
       }
 
+      # Where a tool call that asked for consent has got to. Null for a call
+      # that ran unattended, which is most of them.
+      enum :tool_status, {
+        pending: "pending",
+        approved: "approved",
+        running: "running",
+        declined: "declined"
+      }, prefix: :consent
+
       # Validations
-      validates :content, presence: true, unless: :assistant?
+      # A tool call under consent is written in two steps - the decision is
+      # recorded, then the call runs and its answer is written - so it holds
+      # no content in between. Every other tool message has its answer from
+      # the moment it exists.
+      validates :content, presence: true, unless: -> { assistant? || under_consent? }
 
       # Associations
       belongs_to :conversation, counter_cache: true
@@ -31,6 +44,51 @@ module Layered
       # Scopes
       scope :by_created_at, -> { order(created_at: :asc, id: :asc) }
 
+      # Whether this message is a tool call that was put to the person
+      # talking, whatever they said to it.
+      def under_consent?
+        tool_status.present?
+      end
+
+      # Writes the outcome of a call that was waiting to be approved. Until
+      # this runs the message is the question; afterwards it is the answer,
+      # and reads like any other tool message.
+      #
+      # Conditional on the call still being unanswered, because stopping the
+      # response answers a waiting call on its behalf: whichever gets there
+      # first wins, and the loser is told so rather than overwriting it.
+      # `from` narrows that to particular states, so a caller can decline to
+      # overtake a call whose tool is already running.
+      def resolve_tool_call!(status:, content:, from: nil)
+        scope = self.class.where(id: id, content: nil)
+        scope = scope.where(tool_status: from) if from
+
+        written = scope.update_all(
+          tool_status: status,
+          content: content,
+          input_tokens: TokenEstimator.estimate(content),
+          tokens_estimated: true,
+          updated_at: Time.current
+        )
+        return false if written.zero?
+
+        reload
+        true
+      end
+
+      # Claims an approved call, so that the tool runs once and only once. The
+      # claim is the same row stopping the response competes for: a tool under
+      # consent writes, spends or sends, so it must not run after a Stop, and
+      # must not run twice because a job was delivered twice.
+      def claim_tool_call!
+        claimed = self.class.where(id: id, tool_status: "approved", content: nil)
+          .update_all(tool_status: "running", updated_at: Time.current)
+        return false if claimed.zero?
+
+        reload
+        true
+      end
+
       # Broadcasting
       def broadcast_created
         broadcast_append_to conversation,
@@ -46,6 +104,15 @@ module Layered
           locals: { message: self }
       end
 
+      # Tells the composer the response is not lost, only waiting: it holds
+      # its ground rather than giving up on a response that is doing exactly
+      # what it should - nothing, until the call is answered.
+      def broadcast_response_waiting
+        broadcast_action_to conversation,
+          action: :wait_composer,
+          targets: ".#{dom_id(conversation)}_composer"
+      end
+
       def broadcast_response_complete
         broadcast_action_to conversation,
           action: :enable_composer,

data/app/services/layered/assistant/tool_runner_service.rb

@@ -4,7 +4,29 @@ module Layered
     # result, then queues a fresh assistant message so the model can answer
     # with what the tools returned. That message may ask for tools again,
     # which is the loop max_tool_cycles bounds.
+    #
+    # A tool that declares `consent :always` is not run here. Its message is
+    # recorded with nothing in it, the response stops where it is, and the
+    # person talking approves or declines it - at which point #resolve
+    # finishes the call and picks the response back up.
     class ToolRunnerService
+      DECLINED = "The person you are talking to declined this tool call.".freeze
+      STOPPED = "The response was stopped before this tool ran.".freeze
+
+      # Results are reported to the model as JSON, a refusal included, so
+      # that being turned down reads like any other unhappy answer.
+      def self.error(reason)
+        { error: reason }.to_json
+      end
+
+      # Writes down a call that was never answered because the response was
+      # stopped. Nothing is resumed: stopping means stopping. Returns false if
+      # the call is beyond stopping - already answered, or claimed by the job
+      # that is running its tool, whose own result is the true one.
+      def self.abandon(message)
+        message.resolve_tool_call!(status: :declined, content: error(STOPPED), from: %w[pending approved])
+      end
+
       def call(message:)
         return if message.tool_calls.blank?
 
@@ -16,11 +38,43 @@ module Layered
         message.tool_calls.each { |tool_call| record_result(message, tool_call, available) }
         conversation.update_token_totals!
 
-        if cycles_since_last_prompt(conversation) >= Layered::Assistant.max_tool_cycles
-          halt(message)
-        else
-          continue(message)
+        message.broadcast_response_waiting if conversation.awaiting_consent?
+
+        resume(message)
+      end
+
+      # Carries out a call once it has been approved, or writes down the
+      # refusal, and resumes the response. The refusal is reported to the
+      # model as the tool's result rather than ending the conversation: it
+      # can say something useful about being turned down.
+      def resolve(message:)
+        # An answered call is one this job has already run, or one the response
+        # was stopped over. The tool does not run twice - but resuming may be
+        # what failed last time, so that is still attempted below.
+        if message.content.blank?
+          # Held from before the claim, because claiming moves the call to
+          # running. Running is the tool being carried out, not an outcome, so
+          # the answer is written back under the decision that allowed it.
+          decided = message.tool_status
+
+          content = if message.consent_declined?
+            error(DECLINED)
+          elsif message.claim_tool_call!
+            execute(message, message.tool_name, message.tool_arguments, ToolRegistry.for(message.conversation))
+          else
+            # The claim went elsewhere: the response was stopped before the
+            # tool ran, or this job was delivered twice. Either way the tool
+            # does not run here, and whoever holds the claim finishes the job.
+            return
+          end
+
+          return unless message.resolve_tool_call!(status: decided, content: content)
+
+          message.broadcast_updated
+          message.conversation.update_token_totals!
         end
+
+        resume(message)
       end
 
       private
@@ -28,7 +82,7 @@ module Layered
       def record_result(message, tool_call, available)
         name = tool_call.dig("function", "name")
         arguments = tool_call.dig("function", "arguments")
-        content = execute(message, name, arguments, available)
+        tool = available.find { |candidate| candidate.tool_name == name }
 
         # Both protocols name the call they are asking for. Without an id the
         # result cannot be paired back to it, and the provider rejects the next
@@ -37,6 +91,14 @@ module Layered
           Rails.logger.error("Tool call for '#{name}' arrived with no id on message #{message.id}")
         end
 
+        if tool&.consent_required?
+          record(message, tool_call, name, arguments, content: nil, status: :pending)
+        else
+          record(message, tool_call, name, arguments, content: execute(message, name, arguments, available))
+        end
+      end
+
+      def record(message, tool_call, name, arguments, content:, status: nil)
         result = message.conversation.messages.create!(
           role: :tool,
           content: content,
@@ -44,6 +106,7 @@ module Layered
           tool_call_id: tool_call["id"],
           tool_name: name,
           tool_arguments: arguments,
+          tool_status: status,
           input_tokens: TokenEstimator.estimate(content),
           tokens_estimated: true
         )
@@ -92,7 +155,7 @@ module Layered
       end
 
       def error(reason)
-        { error: reason }.to_json
+        self.class.error(reason)
       end
 
       # One cycle is an assistant message that asked for tools. Counted from
@@ -105,6 +168,72 @@ module Layered
         scope.where.not(tool_calls: nil).count
       end
 
+      # Picks the response back up once every call in the batch has an answer.
+      # Taken under a lock, and refusing to queue a second follow-up, because
+      # two calls approved at once would otherwise both find themselves last.
+      def resume(message)
+        conversation = message.conversation
+
+        conversation.with_lock do
+          if conversation.stopped?
+            # A tool claimed before the Stop still finishes, and the response
+            # is not picked back up. But a tab that loaded while it was
+            # running is waiting on it, so say the response is over once
+            # nothing is left running - otherwise its composer never comes
+            # back.
+            message.broadcast_response_complete unless conversation.unresolved_tool_call?
+            return
+          end
+
+          # Every call in the batch has to hold a result before the model is
+          # shown any of them: approving two at once means the first to finish
+          # would otherwise send a turn with the second still empty.
+          return if conversation.unresolved_tool_call?
+          return if awaiting_results?(conversation)
+          return if followed_up?(conversation, message)
+
+          if cycles_since_last_prompt(conversation) >= Layered::Assistant.max_tool_cycles
+            halt(message)
+          else
+            continue(message)
+          end
+        end
+      end
+
+      # Results are written down one call at a time, so a batch is briefly
+      # part-recorded. A call put to the person talking is answerable the
+      # moment its own row exists, and approving it while a slower call in the
+      # same batch is still running would otherwise find nothing unresolved -
+      # there being no row yet to be unresolved - and send a turn missing that
+      # result. So the batch is measured against what the model asked for
+      # rather than against what has been written down so far.
+      def awaiting_results?(conversation)
+        asked = conversation.messages
+          .where(role: :assistant).where.not(tool_calls: nil)
+          .order(created_at: :desc, id: :desc).first
+        return false unless asked
+
+        ids = asked.tool_calls.filter_map { |tool_call| tool_call["id"].presence }
+        return false if ids.empty?
+
+        conversation.messages.where(role: :tool, tool_call_id: ids).count < ids.size
+      end
+
+      # Whether the conversation has already moved past this batch, which is
+      # what makes resuming safe to attempt twice. Any assistant message from
+      # this point on is that move, finished or not: a follow-up that has since
+      # completed still means the batch was resumed, and a job delivered late
+      # must not queue a second response for it. The message passed in is
+      # excluded because on the unattended path it is itself an assistant
+      # message - the one that asked for the tools.
+      def followed_up?(conversation, message)
+        conversation.messages
+          .where(role: :assistant)
+          .where(created_at: message.created_at..)
+          .where.not(id: message.id)
+          .exists?
+      end
+
       def continue(message)
         follow_up = message.conversation.messages.create!(
           role: :assistant,

data/app/tools/layered/assistant/tool.rb

@@ -58,6 +58,50 @@ module Layered
           from_superclass(:public?) || false
         end
 
+        # Who may call the tool, as a block returning truthy to allow it:
+        #
+        #   permit { |conversation| conversation.user&.admin? }
+        #
+        # This narrows what an assistant has been given rather than replacing
+        # it: a tool still has to be selected on the assistant before anyone
+        # can call it. An unpermitted tool is left out of the definitions sent
+        # to the provider, and refused if the model asks for it anyway from an
+        # earlier turn's history.
+        #
+        # Inherited like the other declarations, so a tool subclassed to share
+        # logic keeps its parent's policy unless it declares its own.
+        def permit(&block)
+          @permit = block if block
+          @permit || from_superclass(:permit)
+        end
+
+        # Whether a call has to be approved by the person talking before it
+        # runs. Reads are usually fine unattended; anything that writes, spends
+        # or sends is worth asking about first.
+        #
+        #   consent :always
+        #
+        # A tool that asks for consent is withheld from a conversation with no
+        # user or no owner: an anonymous visitor gives nobody to ask, and a
+        # public assistant's conversation has no owner to answer as, so a
+        # waiting call there could never be approved. Inherited like the other
+        # declarations.
+        CONSENT = %i[never always].freeze
+
+        def consent(value = nil)
+          if value
+            raise ::ArgumentError, "Unsupported consent: #{value}" unless CONSENT.include?(value.to_sym)
+
+            @consent = value.to_sym
+          end
+
+          @consent || from_superclass(:consent) || :never
+        end
+
+        def consent_required?
+          consent == :always
+        end
+
         def argument(name, type = :string, required: false, description: nil, enum: nil, items: nil)
           type = type.to_s
           raise ::ArgumentError, "Unsupported argument type: #{type}" unless TYPES.include?(type)
@@ -97,8 +141,17 @@ module Layered
           }
         end
 
+        # Whether a conversation may be offered the tool at all. Cheapest
+        # gate first: a private tool needs an owner to scope its reads to, a
+        # tool that asks for consent needs somebody to ask and an owner to
+        # answer as, the host's authorize_tool block may narrow every tool at
+        # once, and the tool's own permit block has the last word.
         def available_for?(conversation)
-          public? || conversation&.owner.present?
+          return false unless public? || conversation&.owner.present?
+          return false if consent_required? && !consentable?(conversation)
+          return false unless host_permits?(conversation)
+
+          permits?(conversation)
         end
 
         # Checks what the model supplied against the schema and returns it as
@@ -136,6 +189,40 @@ module Layered
         def default_tool_name
           name.underscore.sub(/_tool\z/, "").tr("/", "-")
         end
+
+        # A waiting call is approved through the owner-scoped route, so a
+        # conversation with no owner - a public assistant's - has no way to
+        # answer one, whoever is signed in. Both are required: somebody to
+        # ask, and an owner to reach the decision with.
+        def consentable?(conversation)
+          conversation&.user.present? && conversation&.owner.present?
+        end
+
+        def permits?(conversation)
+          block = permit
+          return true unless block
+
+          allowed?("The permit block for '#{tool_name}'") do
+            block.arity.zero? ? block.call : block.call(conversation)
+          end
+        end
+
+        def host_permits?(conversation)
+          block = Layered::Assistant.authorize_tool_block
+          return true unless block
+
+          allowed?("The authorize_tool block") { block.call(self, conversation) }
+        end
+
+        # A policy that raises denies the tool rather than failing the whole
+        # response: a broken block should not hand the tool over, and should
+        # not take the conversation down with it either.
+        def allowed?(subject)
+          !!yield
+        rescue => e
+          Rails.logger.error("#{subject} raised #{e.class}: #{e.message} - denying the tool")
+          false
+        end
       end
 
       attr_reader :message

data/app/views/layered/assistant/messages/_composer.html.erb

@@ -1,3 +1,3 @@
-<%= form_with url: layered_assistant.conversation_messages_path(conversation), id: "composer-form", class: "#{dom_id(conversation)}_composer", data: { controller: "composer", composer_target: "form", action: "submit->composer#submit", composer_responding_value: local_assigns.fetch(:responding, false), composer_stop_url_value: layered_assistant.stop_conversation_path(conversation) } do |f| %>
+<%= form_with url: layered_assistant.conversation_messages_path(conversation), id: "composer-form", class: "#{dom_id(conversation)}_composer", data: { controller: "composer", composer_target: "form", action: "submit->composer#submit", composer_responding_value: local_assigns.fetch(:responding, false), composer_waiting_value: conversation.unresolved_tool_call?, composer_stop_url_value: layered_assistant.stop_conversation_path(conversation) } do |f| %>
   <%= render "layered/assistant/messages/composer_fields", f: f, models: models, selected_model_id: selected_model_id %>
 <% end %>

data/app/views/layered/assistant/messages/_tool_message.html.erb

@@ -1,6 +1,12 @@
+<% pending = message.consent_pending? %>
+<%# An approved call holds nothing until the tool has run, which for a slow %>
+<%# one is a while. Saying so beats an empty Output block. %>
+<% running = message.under_consent? && message.content.blank? %>
 <%= tag.div id: dom_id(message), data: { created_at: (message.created_at.to_f * 1000).to_i }, class: "#{dom_id(message)} l-ui-message" do %>
   <div class="l-ui-message__bubble">
-    <details class="l-ui-surface l-ui-surface--collapsible-highlighted l-ui-surface--sm">
+    <%# A call waiting to be approved is opened: nobody should be asked to %>
+    <%# approve arguments they have to go looking for. %>
+    <%= tag.details class: "l-ui-surface l-ui-surface--collapsible-highlighted l-ui-surface--sm", open: pending do %>
       <summary class="l-ui-surface__summary">
         <span><strong>Tool:</strong> <%= message.tool_name %></span>
         <%= image_tag "layered_ui/icon_chevron_right.svg", alt: "", class: "l-ui-icon l-ui-icon--sm l-ui-surface__chevron", aria: { hidden: true } %>
@@ -8,10 +14,39 @@
       <div class="l-ui-surface__content l-ui-markdown">
         <p>Input:</p>
         <pre><code><%= tool_payload(message.tool_arguments) %></code></pre>
-        <p>Output:</p>
-        <pre><code><%= tool_payload(message.content) %></code></pre>
+        <% if message.content.present? %>
+          <p>Output:</p>
+          <pre><code><%= tool_payload(message.content) %></code></pre>
+        <% end %>
       </div>
-    </details>
+    <% end %>
+    <% if pending %>
+      <%# The notice and the decision are one thing, so they sit in one child %>
+      <%# of the bubble: its gap would otherwise stack with the notice's own %>
+      <%# bottom margin. A plain row, not l-ui-form__actions - that is a page %>
+      <%# form's footer, right-aligned and set well clear of the fields above. %>
+      <div>
+        <div class="l-ui-notice l-ui-notice--warning" role="status">This tool needs your approval before it runs.</div>
+        <div class="l-ui-tag-row">
+          <%= button_to "Approve",
+                layered_assistant.conversation_tool_call_path(message.conversation, message),
+                method: :patch,
+                params: { decision: "approve" },
+                class: "l-ui-button l-ui-button--primary l-ui-button--small",
+                aria: { label: "Approve the call to #{message.tool_name}" } %>
+          <%= button_to "Decline",
+                layered_assistant.conversation_tool_call_path(message.conversation, message),
+                method: :patch,
+                params: { decision: "decline" },
+                class: "l-ui-button l-ui-button--outline-danger l-ui-button--small",
+                aria: { label: "Decline the call to #{message.tool_name}" } %>
+        </div>
+      </div>
+    <% elsif running %>
+      <div class="l-ui-notice" role="status">This tool call is running.</div>
+    <% elsif message.consent_declined? %>
+      <div class="l-ui-notice l-ui-notice--warning" role="status">This tool call was declined.</div>
+    <% end %>
     <div class="l-ui-message__footer">
       <span class="l-ui-message__timestamp"><%= message.created_at.to_fs(:short) %></span>
     </div>

data/app/views/layered/assistant/messages/create.turbo_stream.erb

@@ -5,5 +5,5 @@
 <% end %>
 
 <%= turbo_stream.replace "composer-form" do %>
-  <%= render "layered/assistant/messages/composer", conversation: @conversation, models: @models, selected_model_id: @selected_model_id, responding: @error.nil? %>
+  <%= render "layered/assistant/messages/composer", conversation: @conversation, models: @models, selected_model_id: @selected_model_id, responding: @responding %>
 <% end %>

data/app/views/layered/assistant/panel/messages/_composer.html.erb

@@ -1,3 +1,3 @@
-<%= form_with url: layered_assistant.panel_conversation_messages_path(conversation), id: "panel-composer-form", class: "#{dom_id(conversation)}_composer", data: { controller: "composer", composer_target: "form", turbo_frame: "_top", action: "submit->composer#submit", composer_responding_value: local_assigns.fetch(:responding, false), composer_stop_url_value: layered_assistant.stop_panel_conversation_path(conversation) } do |f| %>
+<%= form_with url: layered_assistant.panel_conversation_messages_path(conversation), id: "panel-composer-form", class: "#{dom_id(conversation)}_composer", data: { controller: "composer", composer_target: "form", turbo_frame: "_top", action: "submit->composer#submit", composer_responding_value: local_assigns.fetch(:responding, false), composer_waiting_value: conversation.unresolved_tool_call?, composer_stop_url_value: layered_assistant.stop_panel_conversation_path(conversation) } do |f| %>
   <%= render "layered/assistant/messages/composer_fields", f: f, models: models, selected_model_id: selected_model_id %>
 <% end %>

data/app/views/layered/assistant/panel/messages/create.turbo_stream.erb

@@ -5,5 +5,5 @@
 <% end %>
 
 <%= turbo_stream.replace "panel-composer-form" do %>
-  <%= render "layered/assistant/panel/messages/composer", conversation: @conversation, models: @models, selected_model_id: @selected_model_id, responding: @error.nil? %>
+  <%= render "layered/assistant/panel/messages/composer", conversation: @conversation, models: @models, selected_model_id: @selected_model_id, responding: @responding %>
 <% end %>

data/app/views/layered/assistant/public/messages/_composer.html.erb

@@ -1,3 +1,3 @@
-<%= form_with url: layered_assistant.public_conversation_messages_path(conversation), id: "public-composer-form", class: "#{dom_id(conversation)}_composer", data: { controller: "composer", composer_target: "form", turbo_frame: "_top", action: "submit->composer#submit", composer_responding_value: local_assigns.fetch(:responding, false), composer_stop_url_value: layered_assistant.stop_public_conversation_path(conversation) } do |f| %>
+<%= form_with url: layered_assistant.public_conversation_messages_path(conversation), id: "public-composer-form", class: "#{dom_id(conversation)}_composer", data: { controller: "composer", composer_target: "form", turbo_frame: "_top", action: "submit->composer#submit", composer_responding_value: local_assigns.fetch(:responding, false), composer_waiting_value: conversation.unresolved_tool_call?, composer_stop_url_value: layered_assistant.stop_public_conversation_path(conversation) } do |f| %>
   <%= render "layered/assistant/messages/composer_fields", f: f %>
 <% end %>

data/app/views/layered/assistant/public/messages/create.turbo_stream.erb

@@ -5,5 +5,5 @@
 <% end %>
 
 <%= turbo_stream.replace "public-composer-form" do %>
-  <%= render "layered/assistant/public/messages/composer", conversation: @conversation, responding: @error.nil? %>
+  <%= render "layered/assistant/public/messages/composer", conversation: @conversation, responding: @responding %>
 <% end %>

data/app/views/layered/assistant/public/panel/messages/_composer.html.erb

@@ -1,3 +1,3 @@
-<%= form_with url: layered_assistant.public_panel_conversation_messages_path(conversation), id: "public-panel-composer-form", class: "#{dom_id(conversation)}_composer", data: { controller: "composer", composer_target: "form", turbo_frame: "_top", action: "submit->composer#submit", composer_responding_value: local_assigns.fetch(:responding, false), composer_stop_url_value: layered_assistant.stop_public_panel_conversation_path(conversation) } do |f| %>
+<%= form_with url: layered_assistant.public_panel_conversation_messages_path(conversation), id: "public-panel-composer-form", class: "#{dom_id(conversation)}_composer", data: { controller: "composer", composer_target: "form", turbo_frame: "_top", action: "submit->composer#submit", composer_responding_value: local_assigns.fetch(:responding, false), composer_waiting_value: conversation.unresolved_tool_call?, composer_stop_url_value: layered_assistant.stop_public_panel_conversation_path(conversation) } do |f| %>
   <%= render "layered/assistant/messages/composer_fields", f: f %>
 <% end %>

data/app/views/layered/assistant/public/panel/messages/create.turbo_stream.erb

@@ -5,5 +5,5 @@
 <% end %>
 
 <%= turbo_stream.replace "public-panel-composer-form" do %>
-  <%= render "layered/assistant/public/panel/messages/composer", conversation: @conversation, responding: @error.nil? %>
+  <%= render "layered/assistant/public/panel/messages/composer", conversation: @conversation, responding: @responding %>
 <% end %>

data/config/routes.rb

@@ -13,6 +13,12 @@ Layered::Assistant::Engine.routes.draw do
   resources :conversations, only: [ :index, :show, :new, :create, :edit, :update, :destroy ] do
     patch :stop, on: :member
     resources :messages, only: [ :index, :create, :destroy ]
+    # Approving a tool call is the same act wherever it is rendered, and the
+    # partial carrying the buttons is broadcast rather than requested, so it
+    # cannot know which namespace it landed in. One route serves them all.
+    # Public conversations are not among them: a tool that asks for consent
+    # is withheld from a conversation with nobody to ask.
+    resources :tool_calls, only: [ :update ]
   end
 
   namespace :panel do

data/db/migrate/20260919000000_add_tool_status_to_layered_assistant_messages.rb

@@ -0,0 +1,8 @@
+class AddToolStatusToLayeredAssistantMessages < ActiveRecord::Migration[8.0]
+  def change
+    # Set on a tool message whose tool asked to be approved before it ran.
+    # Null for a call that needed no consent, which is every call made before
+    # this column existed.
+    add_column :layered_assistant_messages, :tool_status, :string
+  end
+end

data/lib/generators/layered/assistant/templates/initializer.rb

@@ -88,6 +88,34 @@
 # Each assistant is given its own set on its edit screen, and an assistant
 # with none calls nothing - so adding a tool to the application does not hand
 # it to every assistant at once.
+#
+# Who may call a tool is separate from which assistants have it. A `permit`
+# block narrows a tool to the conversations that satisfy it:
+#
+# class RefundTool < Layered::Assistant::Tool
+#   permit { |conversation| conversation.user&.admin? }
+# end
+#
+# An unpermitted tool is left out of the definitions sent to the provider, so
+# the model is never told about it, and is refused if it asks from an earlier
+# turn's history anyway. To apply one rule across every tool, configure an
+# authorize_tool block - it is given the tool class and the conversation, and
+# returning falsey withholds the tool:
+#
+# Layered::Assistant.authorize_tool do |tool, conversation|
+#   conversation.user&.permitted_tools&.include?(tool.tool_name)
+# end
+#
+# Separately from who may call a tool, a tool that writes, spends or sends
+# can put each call to the person talking before it runs:
+#
+# class RefundTool < Layered::Assistant::Tool
+#   consent :always
+# end
+#
+# The call waits in the conversation with its arguments shown, the composer
+# stays disabled, and the response picks up once it has been approved or
+# declined.
 
 # Optional settings (uncomment to enable):
 # Layered::Assistant.log_errors = true              # log API errors to stdout

data/lib/layered/assistant/version.rb

@@ -1,5 +1,5 @@
 module Layered
   module Assistant
-    VERSION = "0.7.0"
+    VERSION = "0.8.0"
   end
 end

data/lib/layered/assistant.rb

@@ -15,6 +15,7 @@ module Layered
     mattr_reader :authorize_block
     mattr_reader :owner_block
     mattr_reader :tools_block
+    mattr_reader :authorize_tool_block
     mattr_accessor :log_errors, default: false
     mattr_accessor :api_request_timeout, default: 210
     mattr_accessor :skip_db_encryption, default: false
@@ -33,5 +34,16 @@ module Layered
     def self.tools(&block)
       @@tools_block = block
     end
+
+    # A veto over every tool call, given the tool class and the conversation
+    # it was asked for in. Returning falsey withholds the tool, as a tool's
+    # own `permit` block does for itself.
+    #
+    # Unlike `authorize`, which guards the engine's routes, this is left open
+    # when unconfigured: tool calls are already behind that block and behind
+    # the set of tools each assistant has been given.
+    def self.authorize_tool(&block)
+      @@authorize_tool_block = block
+    end
   end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: layered-assistant-rails
 version: !ruby/object:Gem::Version
-  version: 0.7.0
+  version: 0.8.0
 platform: ruby
 authors:
 - layered.ai
@@ -327,6 +327,7 @@ files:
 - app/controllers/layered/assistant/public/panel/messages_controller.rb
 - app/controllers/layered/assistant/resources_controller.rb
 - app/controllers/layered/assistant/setup_controller.rb
+- app/controllers/layered/assistant/tool_calls_controller.rb
 - app/helpers/layered/assistant/access_helper.rb
 - app/helpers/layered/assistant/messages_helper.rb
 - app/helpers/layered/assistant/panel_helper.rb
@@ -346,6 +347,7 @@ files:
 - app/javascript/layered_assistant/vendor/marked.js
 - app/jobs/layered/assistant/application_job.rb
 - app/jobs/layered/assistant/messages/response_job.rb
+- app/jobs/layered/assistant/messages/tool_call_job.rb
 - app/layered_resources/layered/assistant/assistant_resource.rb
 - app/layered_resources/layered/assistant/model_resource.rb
 - app/layered_resources/layered/assistant/persona_resource.rb
@@ -427,6 +429,7 @@ files:
 - db/migrate/20260831000000_add_tool_calling_to_layered_assistant_messages.rb
 - db/migrate/20260901000000_create_layered_assistant_assistant_tools.rb
 - db/migrate/20260901000001_add_user_to_layered_assistant_conversations.rb
+- db/migrate/20260919000000_add_tool_status_to_layered_assistant_messages.rb
 - lib/generators/layered/assistant/install_agent_skill_generator.rb
 - lib/generators/layered/assistant/install_generator.rb
 - lib/generators/layered/assistant/migrations_generator.rb