layered-assistant-rails 0.7.0 → 0.9.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: '0849058555cc1dd6e7a2dffb73b3919086413374156a1c3300999eaa543c8a4b'
-  data.tar.gz: fa46c8ac4e95587216f467be30eb25d8c01c427d642b589b63a0ab849984e4fb
+  metadata.gz: 68ea0fb66720732e6e05a139e873ec6eb90912967e1a6e9c956461e76d7e347c
+  data.tar.gz: f4e561987280115a3e160a42a747ee036aab7373c8f65fcc33d36eac01dafec7
 SHA512:
-  metadata.gz: 6b4ae1f8d54d5a753b9560fd532cb04e80cbe1959a728deaa9dfaaa82f83d5a7a65edef6a341a61879f73d1c22d76785692ef430242c7e8f995abc19754bac83
-  data.tar.gz: d43a15e7ed0b30c8a85dc578a4887b46b537d37e2d881f83fc2135539e60a8b4751dc3bc06fba37ed75e3ee0281b2fa52e089594abf91d31017a94156b065c0a
+  metadata.gz: 56758769136c2e033813dcf466444c778643483739bcafa5048962bca0786e167a04a6a9de9085eed038a8d492cd08c2328363cc1a98f2ea58024bd6096efd99
+  data.tar.gz: 054c1b233c696bbec3d6c7e4e643c28fc69ffc378ce5ffa11b3110de59ed9cf8bbb87c9ed582b442f437a724a7d502609171c3022b67eb43f8b2a759a8190059

data/README.md

@@ -251,13 +251,15 @@ assistant.update!(tool_names: [ "weather", "order-lookup" ])
 | `argument` | An argument the model may supply: `argument :name, :type, required:, description:, enum:, items:` |
 | `tool_name` | The name the model calls the tool by. Defaults to the class name without its `Tool` suffix, namespaces hyphenated: `Weather::ForecastTool` becomes `weather-forecast` |
 | `self.public =` | Whether the tool may be offered to a public assistant. `false` by default - see below |
+| `permit` | A block deciding which conversations may call the tool - see [Who may call a tool](#who-may-call-a-tool) |
+| `consent` | `:always` to put each call to the person talking before it runs. `:never` by default - see [Asking before a tool runs](#asking-before-a-tool-runs) |
 
 Argument types are `:string`, `:integer`, `:number`, `:boolean`, `:array` and
 `:object`. An `:array` takes `items:` to name its element type.
 
 Subclass a tool to share logic and the declarations come with it: the child
-inherits its parent's `description`, `public` flag and arguments, adds any
-arguments of its own, and may redeclare one by name to narrow it. The name is
+inherits its parent's `description`, `public` flag, `permit` block, `consent`
+setting and arguments, adds any arguments of its own, and may redeclare one by name to narrow it. The name is
 the exception - the child derives its own from its class name, since two tools
 answering to one name would collide in the registry.
 
@@ -285,6 +287,79 @@ def call(reference:)
 end
 ```
 
+### Who may call a tool
+
+Which assistants have a tool is a configuration decision; who may call it is a
+runtime one. A `permit` block narrows a tool to the conversations that satisfy
+it:
+
+```ruby
+class RefundTool < Layered::Assistant::Tool
+  description "Refund an order."
+
+  permit { |conversation| conversation.user&.admin? }
+end
+```
+
+An unpermitted tool is left out of the definitions sent to the provider, so
+the model is never told it exists, and is refused if it asks for it anyway
+from an earlier turn's history. The block is inherited like the other
+declarations, so a tool subclassed to share logic keeps its parent's policy
+unless it declares its own.
+
+To apply one rule across every tool, configure an `authorize_tool` block in
+your initialiser. It is given the tool class and the conversation, and
+returning falsey withholds the tool:
+
+```ruby
+Layered::Assistant.authorize_tool do |tool, conversation|
+  conversation.user&.permitted_tools&.include?(tool.tool_name)
+end
+```
+
+Unlike `authorize`, which guards the engine's routes, this is left open when
+unconfigured: tool calls already sit behind that block and behind the set of
+tools each assistant has been given. A block that raises denies the tool and
+logs, rather than handing it over or failing the response.
+
+### Asking before a tool runs
+
+Reads are usually fine unattended. Anything that writes, spends or sends is
+worth putting to the person talking first:
+
+```ruby
+class RefundTool < Layered::Assistant::Tool
+  description "Refund an order."
+  consent :always
+
+  argument :reference, :string, required: true
+
+  def call(reference:)
+    owner.orders.find_by!(reference: reference).refund!
+  end
+end
+```
+
+The call is recorded in the conversation with its arguments shown and nothing
+in it, and the response stops there - the composer stays disabled, and the
+call waits as long as it needs to, surviving a reload. Approving runs the tool
+and the response picks up where it left off. Declining reports the refusal to
+the model as the tool's result, so it can say something useful about being
+turned down rather than the conversation dead-ending. Stopping the response
+declines whatever is outstanding, including a call that has been approved but
+has yet to run - so a tool that writes, spends or sends does not slip through
+after the Stop. A call whose tool is already running is left to finish, its
+result recorded, and the response is still not picked back up.
+
+Calls in the same batch that need no consent still run while one waits: each
+call stands on its own.
+
+A tool that asks for consent needs both a user and an owner, so it is never
+offered to a public assistant - even one an anonymous visitor happens to be
+signed in for. There is nobody to ask an anonymous visitor, and a decision is
+recorded through the owner-scoped route a conversation with no owner cannot
+reach. The tool is withheld rather than offered and then stuck waiting.
+
 ### Tools and public assistants
 
 A conversation with a public assistant has no owner - it belongs to an
@@ -319,7 +394,8 @@ composer stays disabled until a response completes without asking for
 anything, and `max_tool_cycles` (default 10) bounds the loop.
 
 Results are shown in the conversation as a collapsible panel naming the tool,
-with its input and output.
+with its input and output. A call waiting to be approved holds the loop where
+it is until it has been answered.
 
 ## Configuration
 
@@ -358,16 +434,17 @@ Then visit `http://localhost:3000`.
 
 ### Deploying the dummy app
 
-The dummy app can be deployed with [Kamal](https://kamal-deploy.org). Set the required environment variables and deploy from `test/dummy`:
+The dummy app can be deployed with [Kamal](https://kamal-deploy.org). Copy `.env.deploy.example` to `.env.deploy` and fill in the target, then deploy with `bin/deploy`:
 
 ```bash
-cd test/dummy
-export KAMAL_DEPLOY_IP=<server-ip>
-export KAMAL_DEPLOY_DOMAIN=<domain>
-export KAMAL_SSH_KEY=<path-to-ssh-key>
-kamal deploy
+cp .env.deploy.example .env.deploy
+bin/deploy           # kamal deploy
+bin/deploy setup     # first run on a new server
+bin/deploy logs      # any other kamal subcommand passes straight through
 ```
 
+`.env.deploy` is gitignored, so the server address stays out of the repo. The script reads it, checks the three `KAMAL_*` variables are set, and runs Kamal from `test/dummy`, where `config/deploy.yml` lives.
+
 ## Testing
 
 Run the gem tests from the root directory:

data/app/controllers/concerns/layered/assistant/message_creation.rb

@@ -1,9 +1,24 @@
 module Layered
   module Assistant
     module MessageCreation
+      WAITING_ON_TOOL_CALL = "A tool call is waiting on you. Answer it before sending another message.".freeze
+
       private
 
       def create_messages_for(conversation:, content:, model_id:)
+        # A tool call that has yet to reach an answer holds the conversation.
+        # The composer is disabled while it waits, so anything arriving here is
+        # a stale tab - and answering it would send the model a call with no
+        # result yet, which the provider rejects. Said rather than dropped, or
+        # the message would vanish with nothing to explain it.
+        if conversation.unresolved_tool_call?
+          return {
+            message: conversation.messages.new(role: :user, content: content),
+            error: WAITING_ON_TOOL_CALL,
+            responding: true
+          }
+        end
+
         message = conversation.messages.create(
           role: :user,
           content: content,
@@ -37,7 +52,8 @@ module Layered
         {
           message: message,
           assistant_message: assistant_message,
-          error: error
+          error: error,
+          responding: error.nil?
         }
       end
     end

data/app/controllers/layered/assistant/conversations_controller.rb

@@ -1,80 +1,60 @@
 module Layered
   module Assistant
-    class ConversationsController < ApplicationController
+    # Conversations are a layered resource for listing, starting, renaming
+    # and deleting them; only `show` - the conversation itself - and `stop`
+    # are this engine's own.
+    class ConversationsController < ResourcesController
       include StoppableResponse
 
-      before_action :set_conversation, only: [ :show, :edit, :update, :destroy, :stop ]
-      before_action :set_assistants, only: [ :new, :create ]
-
-      def index
-        if params[:assistant_id]
-          @assistant = scoped(Assistant).find(params[:assistant_id])
-          @page_title = "Conversations - #{@assistant.name}"
-          @pagy, @conversations = pagy(@assistant.conversations.merge(scoped(Conversation)).includes(:owner, :user).by_created_at)
-        else
-          @page_title = "Conversations"
-          @pagy, @conversations = pagy(scoped(Conversation).includes(:assistant, :owner, :user).by_created_at)
-        end
-      end
+      before_action :set_conversation, only: [ :stop ]
+      before_action :drop_assistant_column, only: [ :index ], if: -> { params[:assistant_id] }
+      before_action :scope_choice_fields, only: [ :new, :create, :edit, :update ]
 
       def show
-        @page_title = @conversation.name
+        super
+        @conversation = @record
         @messages = @conversation.messages.includes(:model).by_created_at
         @models = scoped_models
         @selected_model_id = @messages.last&.model_id || @conversation.assistant.default_model_id || @models.first&.id
       end
 
-      def new
-        @page_title = "New conversation"
-        @conversation = Conversation.new(params.permit(conversation: [ :assistant_id ])[:conversation])
-      end
+      private
 
-      def create
-        @conversation = Conversation.new(conversation_params)
-        @conversation.owner = current_owner!
-        @conversation.user = current_conversation_user
-        @conversation.assistant = scoped(Assistant).find(conversation_params[:assistant_id]) if conversation_params[:assistant_id].present?
-        @conversation.name = Conversation.default_name if @conversation.name.blank?
-        if @conversation.save
-          redirect_to layered_assistant.conversation_path(@conversation)
-        else
-          render :new, status: :unprocessable_entity
-        end
+      # `stop` is a custom member action, so the gem has loaded @record.
+      def set_conversation
+        @conversation = @record
       end
 
-      def edit
-        @page_title = "Edit conversation"
+      # Listed under an assistant, every row would name the same one.
+      def drop_assistant_column
+        @columns = @columns.reject { |column| column[:attribute] == :assistant_id }
       end
 
-      def update
-        if @conversation.update(conversation_params)
-          redirect_to layered_assistant.conversations_path, notice: "Conversation was successfully updated."
+      # The assistant is chosen once, when the conversation is started, and
+      # only from the owner's own.
+      def scope_choice_fields
+        if action_name.in?(%w[new create])
+          assistants = scoped(Assistant).by_name.map { |assistant| [ assistant.name, assistant.id ] }
+          @fields = @fields.map { |field| field[:attribute] == :assistant_id ? field.merge(collection: assistants) : field }
         else
-          render :edit, status: :unprocessable_entity
+          @fields = @fields.reject { |field| field[:attribute] == :assistant_id }
         end
       end
 
-      def destroy
-        @conversation.destroy
-        redirect_to layered_assistant.conversations_path, notice: "Conversation was successfully deleted."
-      end
-
-      private
-
-      def set_conversation
-        @conversation = scoped(Conversation).find_by!(uid: params[:id])
-      end
-
-      def set_assistants
-        @assistants = scoped(Assistant).by_name
-      end
+      # An out-of-scope assistant 404s rather than being silently dropped,
+      # matching how a record itself is looked up. A blank one leaves any
+      # assistant the route already chose in place.
+      def layered_resource_params
+        attributes = super
+        return attributes.except(:assistant_id) unless action_name == "create"
 
-      def conversation_params
-        if action_name == "create"
-          params.require(:conversation).permit(:name, :assistant_id)
+        if attributes[:assistant_id].present?
+          attributes[:assistant_id] = scoped(Assistant).find(attributes[:assistant_id]).id
         else
-          params.require(:conversation).permit(:name)
+          attributes.delete(:assistant_id)
         end
+        attributes[:name] = Conversation.default_name if attributes[:name].blank?
+        attributes
       end
     end
   end

data/app/controllers/layered/assistant/messages_controller.rb

@@ -20,15 +20,16 @@ module Layered
           model_id: model_id
         )
         @message = result[:message]
+        @error = result[:error]
+        @responding = result[:responding]
 
-        unless @message.persisted?
+        unless @message.persisted? || @error
           return head :unprocessable_entity
         end
 
         @assistant_message = result[:assistant_message]
         @models = scoped_models
         @selected_model_id = model_id
-        @error = result[:error]
 
         respond_to do |format|
           format.turbo_stream

data/app/controllers/layered/assistant/panel/messages_controller.rb

@@ -16,15 +16,16 @@ module Layered
             model_id: model_id
           )
           @message = result[:message]
+          @error = result[:error]
+          @responding = result[:responding]
 
-          unless @message.persisted?
+          unless @message.persisted? || @error
             return head :unprocessable_entity
           end
 
           @assistant_message = result[:assistant_message]
           @models = scoped_models
           @selected_model_id = model_id
-          @error = result[:error]
 
           respond_to do |format|
             format.turbo_stream

data/app/controllers/layered/assistant/public/messages_controller.rb

@@ -13,13 +13,13 @@ module Layered
             model_id: @conversation.assistant.default_model_id
           )
           @message = result[:message]
+          @error = result[:error]
+          @responding = result[:responding]
 
-          unless @message.persisted?
+          unless @message.persisted? || @error
             return head :unprocessable_entity
           end
 
-          @error = result[:error]
-
           respond_to do |format|
             format.turbo_stream
           end

data/app/controllers/layered/assistant/public/panel/messages_controller.rb

@@ -15,13 +15,13 @@ module Layered
               model_id: @conversation.assistant.default_model_id
             )
             @message = result[:message]
+            @error = result[:error]
+            @responding = result[:responding]
 
-            unless @message.persisted?
+            unless @message.persisted? || @error
               return head :unprocessable_entity
             end
 
-            @error = result[:error]
-
             respond_to do |format|
               format.turbo_stream
             end

data/app/controllers/layered/assistant/tool_calls_controller.rb

@@ -0,0 +1,34 @@
+module Layered
+  module Assistant
+    # Approving or declining a tool call that asked for consent before it ran.
+    class ToolCallsController < ApplicationController
+      DECISIONS = { "approve" => "approved", "decline" => "declined" }.freeze
+
+      before_action :set_conversation
+
+      def update
+        status = DECISIONS[params[:decision]]
+        return head :unprocessable_entity unless status
+
+        message = @conversation.messages.find(params[:id])
+
+        # Conditional so that two decisions arriving at once cannot both win,
+        # and so a second click or a stale tab changes nothing.
+        decided = Message.where(id: message.id, tool_status: "pending")
+          .update_all(tool_status: status, updated_at: Time.current)
+        return head :no_content if decided.zero?
+
+        message.reload.broadcast_updated
+        Messages::ToolCallJob.perform_later(message.id)
+
+        head :no_content
+      end
+
+      private
+
+      def set_conversation
+        @conversation = scoped(Conversation).find_by!(uid: params[:conversation_id])
+      end
+    end
+  end
+end

data/app/javascript/layered_assistant/composer_controller.js

@@ -7,11 +7,15 @@ export default class extends Controller {
   static targets = ["form", "input", "sendButton", "stopButton"]
   static values = {
     responding: { type: Boolean, default: false },
+    waiting: { type: Boolean, default: false },
     stopUrl: { type: String, default: "" }
   }
 
   connect() {
-    this._onChunkReceived = () => this._resetRespondingTimeout()
+    this._onChunkReceived = () => {
+      this.waitingValue = false
+      this._resetRespondingTimeout()
+    }
     document.addEventListener("assistant:chunk-received", this._onChunkReceived)
     this._applyRespondingState()
     this.updateButtonDisabled()
@@ -26,6 +30,16 @@ export default class extends Controller {
     this._applyRespondingState()
   }
 
+  // A tool call waiting to be approved sends nothing until it is answered,
+  // so the safety timeout is stood down rather than giving up on it.
+  waitingValueChanged() {
+    if (this.waitingValue) {
+      clearTimeout(this._respondingTimeout)
+    } else {
+      this._resetRespondingTimeout()
+    }
+  }
+
   // Enter submits, Shift+Enter is a no-op (default newline),
   // Alt+Enter inserts a newline without submitting.
   submitOnEnter(event) {
@@ -81,9 +95,10 @@ export default class extends Controller {
   }
 
   // Toggle visibility of the Send and Stop buttons. While responding a
-  // 60-second safety timeout resets the composer in case the server
+  // 30-second safety timeout resets the composer in case the server
   // never signals completion. The timeout is reset each time a chunk
-  // is received so long-running responses are not interrupted.
+  // is received so long-running responses are not interrupted, and is
+  // not armed at all while a tool call waits to be approved.
   _applyRespondingState() {
     clearTimeout(this._respondingTimeout)
 
@@ -104,7 +119,7 @@ export default class extends Controller {
   }
 
   _resetRespondingTimeout() {
-    if (!this.respondingValue) return
+    if (!this.respondingValue || this.waitingValue) return
     clearTimeout(this._respondingTimeout)
     this._respondingTimeout = setTimeout(() => {
       this.respondingValue = false

data/app/javascript/layered_assistant/message_streaming.js

@@ -18,10 +18,19 @@ const pendingRender = new WeakMap()
 
 Turbo.StreamActions.enable_composer = function () {
   this.targetElements.forEach((form) => {
+    form.setAttribute("data-composer-waiting-value", "false")
     form.setAttribute("data-composer-responding-value", "false")
   })
 }
 
+// A tool call is waiting to be approved: the composer stays disabled, and
+// holds there for as long as it takes rather than timing out.
+Turbo.StreamActions.wait_composer = function () {
+  this.targetElements.forEach((form) => {
+    form.setAttribute("data-composer-waiting-value", "true")
+  })
+}
+
 function syncStreamingContent(target, markdown) {
   const tmp = document.createElement("div")
   tmp.innerHTML = renderMarkdown(markdown)

data/app/jobs/layered/assistant/messages/tool_call_job.rb

@@ -0,0 +1,22 @@
+module Layered
+  module Assistant
+    module Messages
+      # Carries out a tool call once it has been approved, or writes down the
+      # refusal, then picks the response back up. Runs out of band because a
+      # tool can be slow and the answer streams back like any other.
+      class ToolCallJob < ApplicationJob
+        queue_as :default
+
+        def perform(message_id)
+          message = Message.find(message_id)
+          # Nothing to do for a call still waiting on a decision. A call that
+          # is already answered is not run again, but is still passed on: a
+          # retry may be here because resuming the response is what failed.
+          return if message.consent_pending?
+
+          ToolRunnerService.new.resolve(message: message)
+        end
+      end
+    end
+  end
+end

data/app/layered_resources/layered/assistant/conversation_resource.rb

@@ -0,0 +1,87 @@
+module Layered
+  module Assistant
+    class ConversationResource < Layered::Resource::Base
+      model Conversation
+
+      # Conversations are addressed by uid, not id.
+      lookup_attribute :uid
+
+      columns [
+        # The name links to the conversation itself, not its edit form.
+        { attribute: :name, primary: true, link: false,
+          render: ->(record, view) {
+            view.link_to(record.name, view.layered_assistant.conversation_path(record),
+                         data: { turbo_frame: "_top" })
+          } },
+        { attribute: :assistant_id, label: "Assistant", sortable: false,
+          render: ->(record, view) {
+            view.link_to(record.assistant.name,
+                         view.layered_assistant.assistant_conversations_path(record.assistant),
+                         data: { turbo_frame: "_top" })
+          } },
+        { attribute: :messages_count, label: "Messages",
+          render: ->(record, view) {
+            view.link_to(record.messages_count.to_i,
+                         view.layered_assistant.conversation_messages_path(record),
+                         data: { turbo_frame: "_top" })
+          } },
+        { attribute: :token_estimate, label: "Tokens",
+          render: ->(record, view) { view.number_with_delimiter(record.token_estimate.to_i) } },
+        # The person who did the talking, which is the owner itself until an
+        # owner block scopes ownership elsewhere. Neither is set for an
+        # anonymous visitor on a public assistant.
+        { attribute: :user_id, label: "User", sortable: false,
+          render: ->(record) { record.user.try(:name) || record.owner.try(:name) || "Guest" } },
+        { attribute: :created_at, label: "Created",
+          render: ->(record) { record.created_at.to_fs(:short) } }
+      ]
+
+      search_fields [ :name ]
+
+      default_sort attribute: :created_at, direction: :desc
+
+      # `assistant_id` carries no collection here: the options have to be
+      # scoped to the owner, which only the controller can resolve.
+      # ConversationsController fills them in per request, and drops the
+      # field once the conversation exists - its assistant is fixed.
+      fields [
+        { attribute: :assistant_id, label: "Assistant", as: :select, include_blank: "Select an assistant:", required: true },
+        # Left blank, the name is set from the first message, so the form
+        # must not insist on one even though the model does.
+        { attribute: :name, required: false, hint: "Auto-set from first message if left blank" }
+      ]
+
+      # Nested under an assistant, only that assistant's conversations are
+      # listed, and an out-of-scope assistant 404s before any are reached.
+      def self.scope(controller)
+        scope = controller.send(:scoped, Conversation)
+        if (assistant_id = controller.params[:assistant_id])
+          scope = scope.where(assistant: controller.send(:scoped, Assistant).find(assistant_id))
+        end
+        scope.includes(:assistant, :owner, :user)
+      end
+
+      # Started from an assistant's conversations, the assistant is chosen
+      # already - resolved through `scoped` so an out-of-scope one 404s.
+      def self.build_record(controller)
+        assistant_id = controller.params[:assistant_id]
+
+        Conversation.new(
+          owner: controller.send(:current_owner!),
+          user: controller.send(:current_conversation_user),
+          assistant: (controller.send(:scoped, Assistant).find(assistant_id) if assistant_id)
+        )
+      end
+
+      # A new conversation opens straight into the chat; edits and deletes
+      # return to the list they were made from.
+      def self.after_save_path(controller, record)
+        if controller.action_name == "create"
+          controller.layered_assistant.conversation_path(record)
+        else
+          controller.layered_assistant.conversations_path
+        end
+      end
+    end
+  end
+end