samagotchi 0.36.0 → 0.38.0

data/docs/plugins.md

@@ -162,8 +162,8 @@ in the chat path's `tools:`, after chi's own tools.
   it raises, the default is shown. Both are saved with the call's result, so
   a web page reloaded later shows the same row: the web server doesn't run
   plugins.
-- `targets`: `->(args) { { paths: [...], command: "…", cwd: "…" } }`, each
-  key optional, says what a call acts on, for [guardrails](#guardrails).
+- `targets`: `->(args) { { paths: [...], command: "…", cwd: "…", acts_as: "…", args: {…}, label: "…" } }`,
+  each key optional, says what a call acts on, for [guardrails](#guardrails).
 
 The block returns the result text. If it starts with `Error:`, it counts as
 a failure. If it raises, the model gets `Error: <message>`. To return images
@@ -234,6 +234,27 @@ is what `targets:` says:
   paths (chi's config, …) match them.
 - `command:`: a shell command the call runs; `command:` rules match it.
 - `cwd:`: where it runs, for the repo root and relative paths.
+- `acts_as:`: the tool a call stands for, when the tool runs other tools
+  (the mcp bundle's `mcp_call`). Rules keyed by that tool's name match the
+  call too, as well as rules on the tool's own name. It can't be one of
+  chi's own tools or another bundle's tool: that is dropped (and logged).
+- `args:`: the arguments it acts with (a Hash), when they aren't the call's
+  own (a dispatcher's inner arguments). A call with no `command:` or
+  `paths:` is asked about with them, and "allow this call" is keyed by them.
+- `label:`: what the approval question names the call by (`github: x`), in
+  place of the tool's `label` (or, with `acts_as:`, that tool's). Display
+  only: rules and approvals don't read it. One line of at most 80
+  characters, else it is ignored (and logged).
+
+```ruby
+targets: ->(args) { { acts_as: "mcp_#{args["server"]}_#{args["tool"]}", args: args["args"],
+                      label: "#{args["server"]}: #{args["tool"]}" } }
+```
+
+An approval of an `acts_as:` call keys on the tool's own name as well
+(`mcp_call>mcp_github_x:…`), so it never stands in for an approval of the
+tool it acts as, nor that one for it. Hooks see the call's own name, and so
+does loop-guard's `ignore_tools`.
 
 A tool without `targets:` is matched by its name only. A `targets:` that
 raises counts as nothing (it is logged). See [guardrails.md](guardrails.md).
@@ -431,8 +452,10 @@ end
   start`; the card shows the bundle beside it), and its body the message;
   without `failed:` the title is `setup failed` and the body
   `<label>: <message>`.
-- **`provides_tools: true`**: the task brings tools (with
-  `chi.replace_tools`). A turn sent while it runs starts at once (the user's
+- **`provides_tools: true`**: the task provides what the plugin's tools
+  need: tools of its own (with `chi.replace_tools`), or what fixed tools
+  read (the mcp bundle's index of a first-run server's tools, which
+  `find_mcp_tools` searches). A turn sent while it runs starts at once (the user's
   message shows), then waits for it **before its first model request**, so
   the model sees the tools; the UIs keep showing the task meanwhile. The
   wait lasts at most `timeout` seconds from the task's start (default 60).
@@ -737,10 +760,12 @@ only against this API (`lib/samagotchi/bundles/btw/plugin.rb`).
 ## The mcp bundle
 
 `chi bundle install mcp` (or the `dev` profile) installs the bundle shipped with chi. It is written
-only against this API (`lib/samagotchi/bundles/mcp/plugin.rb`), and adds
-tools from [MCP](https://modelcontextprotocol.io) servers. It has no memory
-file, so it costs the prompt nothing but its tools. Stdio servers only, for
-now.
+only against this API (`lib/samagotchi/bundles/mcp/plugin.rb`), and gives
+the model the tools of [MCP](https://modelcontextprotocol.io) servers. It
+has no memory file, and the servers' tools aren't declared to the model one
+by one: it has two tools, `find_mcp_tools` and `mcp_call` (~250 tokens in
+every request plus a line per server, whatever the servers have; `/mcp`
+says how many). Stdio servers only, for now.
 
 ```yaml
 # config.yml
@@ -756,6 +781,7 @@ bundles:
         env: {NODE_OPTIONS: "--no-warnings"}   # added to chi's environment
         cwd: ~/scratch                         # default: where chi runs
         tools: [read_*, list_directory]        # optional: only these (globs)
+        description: files in ~/scratch        # optional: its line in find_mcp_tools
         timeout: 120                           # optional: this server's per-call timeout
       chrome:
         command: [npx, -y, "chrome-devtools-mcp@latest", --slim, --headless]
@@ -767,29 +793,33 @@ bundles:
   saved in the bundle's data dir (`$XDG_STATE_HOME/samagotchi/plugins/mcp/
   tools-<server>.json`), keyed by a digest of its `command`, `env` (names
   and values: only the digest is stored) and `cwd`. A session with a saved
-  list registers the tools at once and **doesn't start the server**: the
+  list can search its tools at once and **doesn't start the server**: the
   first call of one of its tools does (the call's row shows the wait). So a
   session that never uses MCP spawns nothing, and a new chat opens without
   waiting for `npx`. If the live list differs from the saved one, the saved
-  one is replaced, and so are the tools, from the next turn on.
+  one is replaced, and so is what a search finds. The model's tools never
+  change, so the prompt cache keeps. The saved file also keeps the first
+  sentence of the server's `instructions` (from `initialize`), for its line
+  in `find_mcp_tools`.
 - **The first run** (no saved list, or the config changed) starts the
   server in an [init task](#chiinitlabel-provides_tools-false-quiet-false-timeout-nil--ctx--):
   every UI shows `Starting MCP server x (first run, saving its tools)`, and
   a turn sent meanwhile waits for its tools. A server that doesn't start,
   answer or list its tools within `startup_timeout` (each step) is a warn
-  card, `…: failed`, and its tools are left out. The rest of chi works as
-  usual.
+  card, `…: failed`, and its calls fail until chi restarts. The rest of chi
+  works as usual.
 - **Freshness.** A saved list older than a day is still used, and a quiet
   background task lists the tools again with a server of its own (then
-  stops it), saves them, and replaces the tools if they changed. One worker
-  does it at a time.
+  stops it), saves them, and replaces the searched ones if they changed.
+  One worker does it at a time.
 - **A server that says its tools changed** (`notifications/tools/list_changed`)
-  is asked for its `tools/list` again; the saved list and the tools are
-  replaced from the next turn on.
+  is asked for its `tools/list` again; the saved list and the searched tools
+  are replaced.
 - **A cached server that doesn't start** (the command is gone, it crashes)
   fails that call with `Error: MCP server x didn't start: …` and one notice;
-  later calls answer the same at once, and its tools are left out from the
-  next turn. The saved list stays: the next session tries again.
+  later calls answer the same at once until chi restarts, and a search shows
+  its tools as `(failed: …)`. The saved list stays: the next session tries
+  again.
 - **`start: eager`** on a server starts it with every session (in an init
   task, after the Bridge is up), for a server whose start does something
   you want at once.
@@ -803,17 +833,45 @@ bundles:
     command: [chrome-devtools-mcp, --slim, --headless]   # after npm i -g chrome-devtools-mcp
   ```
 
-- **Tools.** Each tool is the model's as `mcp_<server>_<tool>`, lower case,
-  with anything but a-z, 0-9 and `_` made `_`, cut at 48 characters. A name
-  that clashes is left out, with a notice. The tool's `inputSchema` is its
-  schema (flattened on the native paths, see
-  [Schemas on the native paths](#schemas-on-the-native-paths)); its label is
-  `<server>: <tool>` and its preview the arguments, short.
+- **Tools: search, then call.** The servers' tools are kept in an index;
+  the model has two fixed tools:
+  - `find_mcp_tools(query, server?)` searches it by keywords over the
+    server's name, the tool's name and its description, and answers up to 5
+    tools, each with its name `<server>/<tool>`, its description and its
+    whole `inputSchema`. A word counts more the fewer tools have it (a word
+    most of a browser server's tools share, "page", counts little), and
+    three times as much in a name as in a description. An empty query (and
+    a search with no match) lists every server's tool names, without
+    schemas. A tool whose `inputSchema` (or a property's schema) isn't an
+    object shows
+    `(bad schema)` (and is said once, as a notice), a failed server's tools
+    `(failed: …)`, a first-run server still starting `(starting, try
+    again)`. Its description names each server on one line, never its
+    tools: the server's `description:`, else the first sentence of its
+    `instructions`, else its first 6 tool names, and how many tools it has
+    (a first run's server is just its name until the next session).
+  - `mcp_call(tool, args)` calls one: `tool` is `<server>/<tool>`, and
+    `mcp_<server>_<tool>` works too (looked up, not parsed). Two tools whose
+    `mcp_<server>_<tool>` comes out the same (servers `git-hub` and
+    `git_hub`, names alike up to the 48-character cut) are both refused,
+    since guardrail rules and approvals couldn't tell them apart: a search
+    and `/mcp` mark them `(name clash with …)`, and there is one notice.
+    `args` are typed by the tool's
+    `inputSchema` (numbers and booleans given as text, JSON text for an
+    object). A name it doesn't know answers the closest ones (`Error: no
+    MCP tool chrome/open_url. Closest: chrome/new_page, …`). Its row shows
+    `mcp chrome/take_screenshot fullPage=true`.
+  - A tool turn takes a few more steps than when every tool was declared
+    (the P1 spike: about 1.5 more on a 30-tool server), and each request is
+    smaller (about half there).
 - **Calls.** A call is `tools/call`. The text blocks of the answer are joined;
   audio or a resource without text is a short placeholder
-  (`[audio: audio/wav]`). `isError` makes it `Error: …`. A call that takes
-  longer than the timeout is an `Error:`, and a cancelled turn stops the
-  wait; both send `notifications/cancelled` to the server.
+  (`[audio: audio/wav]`). `isError` makes it `Error: …`; it, a JSON-RPC
+  error and `args` that aren't an object end with the tool's schema
+  (`chrome/click's inputSchema: {…}`), so a call made without a search can
+  be fixed on the next step. A call that takes longer than the timeout is
+  an `Error:`, and a cancelled turn stops the wait; both send
+  `notifications/cancelled` to the server.
 - **Images.** An `image` block goes to the model as a picture
   ([Returning images](#returning-images): at most 4 per call, a line
   instead when the model can't see images). Its place in the text is a line,
@@ -835,17 +893,20 @@ bundles:
 - **Stop.** The servers stop with chi ([Shutdown](#shutdown)): stdin is
   closed, then TERM and KILL go to the server's process group.
 - **`/mcp`** (anytime) shows a card with the servers, their state (cached
-  (not started), running with its pid, failed, stopped) and their tools.
+  (not started), running with its pid, failed with its count, stopped) and
+  their tools as `<server>/<tool>`, the name `mcp_call` takes.
   A cached or running server's line says roughly how many tokens its tool
-  definitions take (`~4,232 tokens`), and the last line totals the servers
-  whose tools the model has: what every request carries. It is an estimate:
+  definitions take (`~4,232 tokens`), which only a search's answer carries
+  now; the last line says what every request carries (`find_mcp_tools` and
+  `mcp_call`) and totals the servers'. It is an estimate:
   the definitions' JSON as the chat API gets it (`api: openai`), divided by
   4. The native prompts (Gemma, Qwen) render a flatter schema, so there they
   take less. The log records each server's estimate when it changes
   (`mcp_tools_estimated`).
 - The server's stderr goes to the debug log (`plugins` records, bundle=mcp).
 - **Guardrails.** A rule's `tool:` can be a glob, so one rule covers every
-  MCP tool:
+  MCP tool (a search is `find_mcp_tools`, outside `mcp_*`, so it isn't
+  asked about):
 
   ```yaml
   guardrails:
@@ -856,10 +917,22 @@ bundles:
         reason: an MCP server's tool
   ```
 
-  An MCP tool has no `targets:`, so the question shows its arguments,
-  under the tool's label as its row shows it (`everything: get_sum: a=20
-  b=22`; any plugin tool with a label is asked about by it), and "Allow this call for the
+  `mcp_call`'s `targets:` act as the tool it calls, by its
+  `mcp_<server>_<tool>` name ([acts_as:](#guardrails)), so `tool:
+  "mcp_github_*"` or `tool: mcp_github_merge_pull_request` match an
+  `mcp_call` to it, as they matched the tool when it was declared by that
+  name. A call made before the server's tools are known (a first run
+  still starting) is matched by the name the model gave, so the rule fires
+  before the call waits; if the tool it then finds isn't that one, the
+  call is refused. The question shows the inner arguments under `<server>: <tool>`
+  (`everything: get_sum: a=20 b=22`), and "Allow this call for the
   session" (or in this repo) allows that tool with those arguments only.
+  (Approvals stored before 0.6.0, under the old name, don't carry over.)
+  Two things see `mcp_call`, not the tool: hooks (a `before_tool_call`
+  event's `tool:` is `"mcp_call"`, the tool in its args), and loop-guard's
+  `ignore_tools` (`mcp_call` ignores every MCP call; calls to different
+  tools, or with different args, stay distinct, since the tool is in the
+  args).
 
 ## The loop-guard bundle
 

data/lib/samagotchi/broadcast/scope_card.rb

@@ -74,16 +74,29 @@ module Samagotchi
       end
       private_class_method :user_texts
 
+      # Both paths resolved first: a linked worktree's project root comes
+      # back resolved (/private/tmp/… on macOS) while the session's folder
+      # may go through a symlink (/tmp/…).
       def folder(cwd, project)
         return ContextNote.home_relative(cwd) if project.nil?
-        return nil if cwd.to_s == project.to_s
 
-        Pathname.new(cwd.to_s).relative_path_from(Pathname.new(project.to_s)).to_s
+        cwd = real(cwd)
+        project = real(project)
+        return nil if cwd == project
+
+        Pathname.new(cwd).relative_path_from(Pathname.new(project)).to_s
       rescue ArgumentError
         cwd
       end
       private_class_method :folder
 
+      def real(path)
+        File.realpath(path.to_s)
+      rescue SystemCallError
+        File.expand_path(path.to_s)
+      end
+      private_class_method :real
+
       def cut(text, limit)
         line = text.to_s.gsub(/\s+/, " ").strip
         line.length > limit ? "#{line[0, limit - 1]}…" : line

data/lib/samagotchi/broadcast/triage.rb

@@ -0,0 +1,284 @@
+# frozen_string_literal: true
+
+require_relative "../idle_client"
+require_relative "../idle_target"
+require_relative "../cancellation_controller"
+require_relative "../log"
+require_relative "tags"
+
+module Samagotchi
+  module Broadcast
+    # Whether a broadcast concerns a recipient its tags didn't match: a
+    # small model reads the note and the session's ScopeCard and answers
+    # yes or no (Triage::LLM). Many recipients are judged in parallel under
+    # one deadline (Triage.judge_all); whatever isn't judged by then, or
+    # fails, is delivered unchecked (fail open: a missed recipient is worse
+    # than an extra note).
+    module Triage
+      # One recipient's verdict.
+      # @!attribute relevant [Boolean] it gets the note
+      # @!attribute p [Float, nil] the model's P(yes) from its logprobs,
+      #   1.0 or 0.0 from a plain answer, nil when no model judged it
+      # @!attribute reason [String] why, for the output and the log
+      # @!attribute by [String] who decided: "tags" (a shared tag),
+      #   "model", "fallback" (delivered unchecked: deadline, error, an
+      #   answer that is no yes or no), "scope" (the note's first line
+      #   names another project), "all" (--all) or "repl" (a chi REPL owns
+      #   it: no notes)
+      # @!attribute match [Match, nil] the shared tag (by "tags")
+      Verdict = Data.define(:relevant, :p, :reason, :by, :match) do
+        def initialize(match: nil, **) = super
+
+        def unchecked? = by == "fallback"
+      end
+
+      # The spike's prompt (plain yes/no beat JSON with a p and a reason
+      # for every model tried: writing a reason pushes towards "relevant").
+      SYSTEM = <<~TEXT
+        You route short notes that a developer broadcasts to their running coding-agent sessions.
+        You get one note and one session's card (project, branch, title, what the session was started for, its latest recap, the last request).
+        Decide whether the note may affect THIS session's current work: would the agent working on it want to know?
+        A note can start with an informal scope line (a project, a component, a team); use it as a hint.
+        Notes about things unrelated to the session's work (social, other projects, other components) are not relevant.
+        Answer with exactly one word: yes or no.
+      TEXT
+
+      DEFAULT_PARALLEL = 4
+      DEFAULT_DEADLINE = 20.0
+      DEFAULT_THRESHOLD = 0.5
+      # How long the threads still asking get to stop once the deadline
+      # passed, all of them together (then they are killed).
+      JOIN_GRACE = 1.0
+
+      module_function
+
+      # @return [Verdict] delivered unchecked, +why+ in the reason
+      def unchecked(why) = Verdict.new(relevant: true, p: nil, reason: "unchecked: #{why}", by: "fallback")
+
+      # The messages a model judges +note+ for +card+ by.
+      def messages(note, card)
+        [{ role: "system", content: SYSTEM }, { role: "user", content: "NOTE:\n#{note}\n\nSESSION CARD:\n#{card}" }]
+      end
+
+      # The one project the note's first line names, as a whole word (any
+      # case), among +projects+ (the recipients' project names); nil when it
+      # names none or several. A scope line ("shopfront/checkout", "agentd
+      # web") then keeps the note to that project's sessions; anything
+      # fuzzier is the model's job.
+      # @param projects [Array<String>]
+      # @return [String, nil]
+      def scope_project(note, projects)
+        line = note.to_s.lines.first.to_s.downcase
+        named = projects.compact.uniq.select do |project|
+          line.match?(/(?<![[:alnum:]_-])#{Regexp.escape(project.downcase)}(?![[:alnum:]_-])/)
+        end
+        named.one? ? named.first : nil
+      end
+
+      # Every card's verdict: a shared tag delivers (Tags.match); a note
+      # whose first line names one of the projects (scope_project) skips
+      # the other projects' sessions; the model judges the rest
+      # (judge_all, +judge+ its keywords).
+      # @param cards [Array<ScopeCard>]
+      # @param note_tags [Array<Tag>]
+      # @return [Hash{String => Verdict}] by card id, in +cards+' order
+      def verdicts(note, cards, note_tags:, new_backend:, **judge)
+        scope = scope_project(note, cards.map(&:project))
+        decided = cards.to_h do |card|
+          match = Tags.match(note_tags, card.tags)
+          verdict = if match
+                      Verdict.new(relevant: true, p: nil, reason: match.reason, by: "tags", match: match)
+                    elsif scope && card.project != scope
+                      Verdict.new(relevant: false, p: nil, reason: "scope line names #{scope}", by: "scope")
+                    end
+          [card.id, verdict]
+        end
+        judged = judge_all(note, cards.reject { |card| decided[card.id] }, new_backend: new_backend, **judge)
+        cards.to_h { |card| [card.id, decided[card.id] || judged.fetch(card.id)] }
+      end
+
+      # Judges +cards+ with at most +parallel+ backends at a time, one per
+      # thread (a backend keeps per-host state), all under +deadline+
+      # seconds. A card not judged by then is delivered unchecked; the
+      # requests still running are cancelled.
+      # @param new_backend [#call] (CancellationController) → a backend:
+      #   #judge(note, card) → Verdict
+      # @param cards [Array<ScopeCard>]
+      # @return [Hash{String => Verdict}] by card id
+      def judge_all(note, cards, new_backend:, parallel: DEFAULT_PARALLEL, deadline: DEFAULT_DEADLINE,
+                    clock: -> { Process.clock_gettime(Process::CLOCK_MONOTONIC) })
+        return {} if cards.empty?
+
+        queue = Queue.new
+        cards.each { |card| queue << card }
+        queue.close
+        results = {}
+        lock = Mutex.new
+        changed = ConditionVariable.new
+        cancel = CancellationController.new
+        ends = clock.call + deadline.to_f
+        finished = 0
+        threads = Array.new(parallel.to_i.clamp(1, cards.size)) do
+          Thread.new do
+            card = nil
+            backend = new_backend.call(cancel)
+            while (card = queue.pop)
+              verdict = judge_one(backend, note, card)
+              lock.synchronize do
+                results[card.id] = verdict unless cancel.cancelled?
+                changed.broadcast
+              end
+              card = nil
+            end
+          rescue StandardError => e
+            # No backend (or a bug around it): this thread's card and the
+            # ones it takes from here on are delivered unchecked at once,
+            # so nobody waits for the deadline.
+            Log.exception(:broadcast, "triage_thread_failed", e)
+            failed = unchecked("triage failed (#{e.class})")
+            lock.synchronize { results[card.id] ||= failed if card }
+            while (card = queue.pop)
+              lock.synchronize { results[card.id] ||= failed }
+            end
+          ensure
+            lock.synchronize do
+              finished += 1
+              changed.broadcast
+            end
+          end
+        end
+        lock.synchronize do
+          loop do
+            break if results.size == cards.size || finished == threads.size
+
+            left = ends - clock.call
+            break if left <= 0
+
+            changed.wait(lock, left)
+          end
+          cancel.cancel!(:deadline)
+          queue.clear
+        end
+        grace_ends = clock.call + JOIN_GRACE
+        threads.each { |thread| thread.join([grace_ends - clock.call, 0].max) || thread.kill }
+        cards.to_h { |card| [card.id, results[card.id] || unchecked("triage deadline")] }
+      end
+
+      def judge_one(backend, note, card)
+        backend.judge(note, card)
+      rescue Samagotchi::LLM::RequestCancelled
+        unchecked("triage deadline")
+      rescue StandardError => e
+        Log.exception(:broadcast, "triage_failed", e)
+        unchecked("triage failed (#{e.class})")
+      end
+      private_class_method :judge_one
+
+      # A triage model on an OpenAI-compatible host (an IdleTarget), asked
+      # through its own IdleClient: thinking off, a few tokens, the first
+      # token's logprobs when the host gives them. P(yes) / (P(yes) +
+      # P(no)) from those is graded against +threshold+; a plain yes or no
+      # is 1.0 or 0.0.
+      class LLM
+        MAX_TOKENS = 16
+        TOP_LOGPROBS = 5
+        LOGPROB_OPTIONS = { logprobs: true, top_logprobs: TOP_LOGPROBS }.freeze
+        # Hosts (base urls) that refused logprobs with a 400: asked without
+        # them from then on, in this process. Shared by every thread's LLM:
+        # a parallel batch's first requests may each meet the 400 once, the
+        # requests after them don't.
+        @no_logprobs = Set.new
+        @no_logprobs_lock = Mutex.new
+
+        class << self
+          def logprobs?(base_url) = @no_logprobs_lock.synchronize { !@no_logprobs.include?(base_url) }
+
+          def no_logprobs!(base_url) = @no_logprobs_lock.synchronize { @no_logprobs << base_url }
+
+          # Specs: forget the hosts that refused.
+          def reset_logprobs! = @no_logprobs_lock.synchronize { @no_logprobs.clear }
+        end
+
+        # @param target [IdleTarget]
+        # @param timeout [Numeric] each request's
+        # @param client [IdleClient, nil] specs inject one
+        def initialize(target:, timeout:, threshold: DEFAULT_THRESHOLD, cancel_controller: nil, client: nil)
+          @target = target
+          @threshold = threshold.to_f
+          @cancel = cancel_controller
+          @client = client || IdleClient.for(target, timeout: timeout, purpose: "broadcast")
+        end
+
+        # @return [Verdict]
+        # @raise [Samagotchi::LLM::RequestCancelled] the deadline passed
+        def judge(note, card)
+          answer = ask(Triage.messages(note, card))
+          word = self.class.word(answer.text)
+          return Triage.unchecked("the model answered #{answer.text.to_s.strip[0, 40].inspect}") unless word
+
+          p = self.class.logprob_p(answer.top_logprobs, word: word)
+          relevant = p ? p >= @threshold : word == "yes"
+          reason = "model: #{relevant ? "yes" : "no"}#{format(" (p %.2f)", p) if p}"
+          Verdict.new(relevant: relevant, p: p || (word == "yes" ? 1.0 : 0.0), reason: reason, by: "model")
+        rescue IdleClient::SummarizeError => e
+          Triage.unchecked("the model failed (#{e.message[0, 120]})")
+        end
+
+        # "yes" or "no": the answer's first such word (thinking left out),
+        # nil when it has neither.
+        def self.word(text)
+          text.to_s.gsub(%r{<think>.*?</think>}m, "").strip.downcase[/\A\W*(yes|no)\b/, 1]
+        end
+
+        # P(yes) / (P(yes) + P(no)) over the first token's alternatives
+        # ("Yes", " yes" count as yes), when that token is the answer +word+
+        # itself: its likeliest alternative is yes or no and agrees with
+        # +word+. nil otherwise (an answer starting "**", "<think>" or a
+        # quote: those alternatives say nothing about yes or no).
+        # @param top [Array<Samagotchi::LLM::TokenLogprob>]
+        # @return [Float, nil]
+        def self.logprob_p(top, word:)
+          first = top.max_by(&:logprob)
+          return nil unless first && first.token.strip.downcase == word
+
+          sum = ->(want) { top.select { |t| t.token.strip.downcase == want }.sum { |t| Math.exp(t.logprob) } }
+          yes = sum.call("yes")
+          no = sum.call("no")
+          (yes / (yes + no)).round(4)
+        end
+
+        private
+
+        def ask(messages)
+          base_url = @target.base_url
+          return request(messages, {}) unless self.class.logprobs?(base_url)
+
+          begin
+            request(messages, LOGPROB_OPTIONS)
+          rescue IdleClient::SummarizeError => e
+            raise unless e.cause.is_a?(Samagotchi::LLM::BadRequest)
+
+            # The host may not give logprobs (a server that refuses "n and
+            # logprobs"): ask again without. Leave them out from now on when
+            # the 400 said so or the retry answered; a 400 the retry gets
+            # too was about something else.
+            refused = e.cause.message.to_s.match?(/logprob/i)
+            no_logprobs!(e.cause) if refused
+            answer = request(messages, {})
+            no_logprobs!(e.cause) unless refused
+            answer
+          end
+        end
+
+        def no_logprobs!(error)
+          self.class.no_logprobs!(@target.base_url)
+          Log.info(:broadcast, "logprobs_refused", model: @target.label, detail: error.message.to_s[0, 200])
+        end
+
+        def request(messages, options)
+          @client.ask(messages, max_tokens: MAX_TOKENS, cancel_controller: @cancel, kind: "broadcast", options: options)
+        end
+      end
+    end
+  end
+end