ask-decisions 0.1.1 → 0.2.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 0eecbf122542bb526917a22123bd02d15ed9aaee01db6c3283024858cbb5a867
-  data.tar.gz: 6192d3b7bab381fe65d3f43119ee836598aa68b2639eae0fb87379b432bd838a
+  metadata.gz: 3dd4560892d89a85e39ed1e73474268724f2f55fca39f1471e9e490e2044168c
+  data.tar.gz: d960989693373e6ce7afe0ddcec17bc88e59addafb89d5269fa67ef52bb9b299
 SHA512:
-  metadata.gz: f90ced1aac865272ac6bdbc55f734ba55be112d0178b3fdc07214b3758b12399606bc6ccb3df48f2ca20f2968288759fa02bf975e20add4e1faeb8f8b52fc7a3
-  data.tar.gz: efad93c3e452c09bc1b0e188c6445066dd6787a13ed9d6d9698e2198f7a90a17faff13b7aa5c26f92bd87c7c0fcefd1edda23dd84f489ae15ef7ae1497216ef4
+  metadata.gz: 3e2616a2e442dd2c764e5dd3d020a7ba7086f40ba24acca442e272db358817727e4bbaa6c57e84b87fea7c3e64103a6c6a741d7cac24f29f72504708a086b6ea
+  data.tar.gz: 99ea57a135e7f4265245f246f908af5b821b4780ffbf9cafe8f68c94c9d63124e4298666b106c556ccbb9c9aa1be75bb45e690747f9c063fc1616ee053553248

data/CHANGELOG.md

@@ -4,7 +4,56 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
-## [0.1.0] - 2026-09-18
+## [0.2.1] - 2026-09-17
+
+### Added
+
+- **The host owns the judgement: `Gate.new(provider, questions:, thresholds:)`.**
+  The questions a gate asks are a property of the host's tools, not of the gem.
+  A booking tool and a shell tool are not dangerous for the same reason, and
+  pi-jev's questions — "is this action destructive?", "does this send local data
+  off-machine?" — say nothing useful about booking an appointment. A host now
+  asks its own: "does this commit the customer to a booking?", "does this spend
+  the owner's money?". Every question must be armed with a threshold, and a
+  question without one is refused at construction rather than armed with a bar
+  that never fires. A threshold given for one of the default questions still
+  keeps the rest of the defaults, so raising one bar stays one line.
+
+- **`OutputJudge.new(provider, questions:, advice:)`.** Same reasoning: what an
+  output *is* — a leak, a failure class, the advice to give — belongs to the
+  host. The two ids the result reads (`:leaks_secret`, `:failure_class`) stay
+  the gem's contract, and the outcome question must offer `no_failure` among its
+  criteria, because the judge runs after every judged call and not only after a
+  suspicious one; both are refused at construction when they are missing, since
+  a judge that reads nothing judges nothing, silently.
+
+- **`AgentAdapter` passes the judgement through** — `gate_questions`,
+  `output_questions`, `output_advice` — so a host configures the guards for its
+  own tools in one place.
+
+### Notes
+
+- 0.2.0 is what it was released as: the router removal below, and the
+  host-owned judgement arrived after it.
+
+## [0.2.0] - 2026-09-17
+
+### Removed
+
+- **`Ask::Decisions::ToolRouter` — a router that picks one of N tools from the
+  message alone.** It is gone because the measurement below is the argument
+  against it: routing to a tool was right 10/16 where routing to a lane was
+  right 19/20, on the same roster with the same model. Overlapping tools
+  cannot be separated by a message — which of seven knowledge tools holds the
+  answer is discovered by calling them — so a router answering that question
+  is guessing with confidence. The lane withholds the tools a turn cannot
+  need and the model chooses within the narrow roster, where it can see the
+  candidates' results.
+
+  Nothing in the ecosystem called it: not ask-agent, not ask-anychat, not any
+  app. Take this as the cheap moment — the class has no users yet.
+
+## [0.1.1] - 2026-09-18
 
 The first release, so everything here is new. It is a decision layer for the
 ask-rb ecosystem: ask Jev (or any System One model) typed questions and get
@@ -15,7 +64,7 @@ decide.
 
 - `Ask::Decisions::Reader` — asks a described set of options as one Choice,
   with anything else the caller needs riding along in the same request.
-  `ToolRouter` and `Triage` are façades over it.
+  `Triage` is a façade over it.
 - `Ask::Decisions::Triage` — reads a message into a caller-defined lane and
   asks the mood and whether the person wants a human, all in one request.
   Measured against a 19-tool roster: lane-level routing was right 19/20 where

data/README.md

@@ -177,16 +177,28 @@ not carry. A lane is the part that *is* decidable from the message alone.
 A reading should narrow, never grant: let the lane take tools away from a
 turn, and let the agent's own definition stay the ceiling.
 
-## ToolRouter
+### Why there is no tool router
 
-Routes user turns to the right tool:
+There was one — `ToolRouter`, a Choice over a tool roster. It is gone, because
+the measurement above is the argument against it: asked to pick one of
+nineteen tools the answer was right 10 times in 16, and asked to pick a lane
+19 times in 20. Same model, same messages.
 
-```ruby
-router = Ask::Decisions::ToolRouter.new(provider, tools: tool_roster)
-result = router.route(user_turn: "run the tests")
-result.tool         # => "bash"
-result.confidence   # => 0.92
-```
+The reason is structural, not a tuning problem. Overlapping tools cannot be
+separated by a message: which of seven knowledge tools holds the answer is
+discovered by *calling* them. Routing to a tool asks a question the message
+does not carry, so a router that answers it is guessing with confidence.
+
+What replaces it is the lane plus code:
+
+- the lane withholds the tools the turn cannot need,
+- the lane's pre-read fetches what the turn will obviously ask for,
+- and the model chooses within the narrow roster, where choosing is a
+  decision it can actually make — because it can see the candidates' results.
+
+If you do need a decider to pick a tool, the roster it picks from has to be
+small and disjoint — a handful of tools a message can actually distinguish.
+If it is not, the fix is a narrower lane or a pre-read, not a better prompt.
 
 ## ConfidencePolicy
 

data/lib/ask/decisions/agent_adapter.rb

@@ -106,12 +106,25 @@ module Ask
         Ask::Decisions.resolve_provider(name)
       end
 
+      # What the host judges, and how high the bar is. The questions are the
+      # host's because risk is: a booking tool and a shell tool are not
+      # dangerous for the same reason, and a gate written for one says nothing
+      # useful about the other.
       def gate_config
-        { tools: @config[:gate_tools], thresholds: @config[:gate_thresholds] || {} }.compact
+        {
+          questions: @config[:gate_questions],
+          thresholds: @config[:gate_thresholds],
+          tools: @config[:gate_tools]
+        }.compact
       end
 
       def output_judge_config
-        { tools: @config[:output_tools], output_limit: @config[:output_limit] }.compact
+        {
+          questions: @config[:output_questions],
+          advice: @config[:output_advice],
+          tools: @config[:output_tools],
+          output_limit: @config[:output_limit]
+        }.compact
       end
 
       def failure_classifier_config

data/lib/ask/decisions/gate.rb

@@ -49,11 +49,20 @@ module Ask
       }.freeze
 
       # @param provider [Ask::DecisionProvider] the decision provider to use
-      # @param thresholds [Hash] override specific thresholds
+      # @param questions [Hash{Symbol => Decision::Noul,Decision::Score}] what
+      #   to ask about a call. The defaults above are pi-jev's, written for a
+      #   coding agent's tools; a host with other tools should say what risk
+      #   means for them ("does this commit the customer to a booking?", "does
+      #   this spend the owner's money?") rather than inherit a vocabulary
+      #   about shell commands.
+      # @param thresholds [Hash{Symbol => Numeric}] the bar for each question.
+      #   Every question needs one: a question with no threshold can never
+      #   flag, and a gate that looks armed and never fires is worse than none.
       # @param tools [Array<String>, nil] tools to gate (nil = all)
-      def initialize(provider, thresholds: {}, tools: nil)
+      def initialize(provider, questions: QUESTIONS, thresholds: DEFAULT_THRESHOLDS, tools: nil)
         @provider = provider
-        @thresholds = DEFAULT_THRESHOLDS.merge(thresholds)
+        @questions = questions
+        @thresholds = arming_thresholds(questions, thresholds)
         @tools = tools
       end
 
@@ -71,7 +80,7 @@ module Ask
 
         result = @provider.evaluate(
           state: state,
-          decisions: QUESTIONS
+          decisions: @questions
         )
 
         Verdict.new(result, @thresholds)
@@ -79,6 +88,27 @@ module Ask
 
       private
 
+      # Every question must be armed. A host that supplies its own questions
+      # and forgets a threshold would otherwise get a gate that silently
+      # ignores one of its own risk questions, which is the failure mode a
+      # gate exists to prevent.
+      #
+      # A threshold given for a question the defaults also ask keeps the rest
+      # of the defaults, so raising one bar is one line rather than a copy of
+      # the table.
+      def arming_thresholds(questions, thresholds)
+        keys = questions.keys.map(&:to_sym)
+        armed = DEFAULT_THRESHOLDS.slice(*keys).merge(thresholds.to_h.transform_keys(&:to_sym))
+        unarmed = keys - armed.keys
+
+        unless unarmed.empty?
+          raise ArgumentError,
+            "no threshold for #{unarmed.inspect}: a question that cannot fire is not a gate"
+        end
+
+        armed
+      end
+
       def build_state(tool:, args:, working_dir: nil, user_message: nil)
         {
           tool: tool,

data/lib/ask/decisions/output_judge.rb

@@ -46,12 +46,35 @@ module Ask
       }.freeze
 
       # @param provider [Ask::DecisionProvider]
-      # @param tools [Array<String>, nil] tools to judge (nil = ["bash"])
+      # The class that means nothing went wrong. A host's outcome question has
+      # to offer it, because the judge runs after every judged call and not
+      # only after a suspicious one: without it, every successful call would
+      # read as a failure.
+      SUCCESS_CLASS = "no_failure"
+
+      # A host judges two things about an output: whether it leaked something
+      # (`:leaks_secret`), and what happened (`:failure_class`). Those two ids
+      # are the gem's contract and stay fixed; the questions' words, the
+      # classes they can answer with, and the advice per class are the host's —
+      # a coding agent's failures are code bugs and broken environments, a
+      # business's are "we don't offer that" and "the system is down".
+      #
+      # @param questions [Hash{Symbol => Decision}] what to ask about a result.
+      #   Must answer under :leaks_secret and :failure_class, and the outcome
+      #   question must offer the class above among its criteria.
+      # @param advice [Hash{String => String,nil}] one line per class the
+      #   outcome question can answer with. A class with no line is advice the
+      #   model does not get.
+      # @param tools [Array<String>, nil] tools to judge (nil = the gem's own
+      #   default, the coding agent's shell tool — pass the host's own)
       # @param leak_threshold [Float] noul threshold for leak detection
       # @param failure_threshold [Float] confidence threshold for failure classification
       # @param output_limit [Integer] max characters of output to send
-      def initialize(provider, tools: nil, leak_threshold: 0.90, failure_threshold: 0.60, output_limit: 2000)
+      def initialize(provider, questions: QUESTIONS, advice: ADVICE, tools: nil,
+        leak_threshold: 0.90, failure_threshold: 0.60, output_limit: 2000)
         @provider = provider
+        @questions = judgeable(questions)
+        @advice = advice
         @tools = tools || ["bash"]
         @leak_threshold = leak_threshold
         @failure_threshold = failure_threshold
@@ -70,12 +93,32 @@ module Ask
         truncated = truncate(output, @output_limit)
         state = { output: truncated, tool_arguments: truncate_values(args, 400) }
 
-        result = @provider.evaluate(state: state, decisions: QUESTIONS)
-        OutputResult.new(result, @leak_threshold, @failure_threshold)
+        result = @provider.evaluate(state: state, decisions: @questions)
+        OutputResult.new(result, @leak_threshold, @failure_threshold, advice: @advice)
       end
 
       private
 
+      # A host's questions have to answer the two things the result reads, and
+      # the outcome question has to be able to say that nothing went wrong.
+      # Both are silent failures otherwise — a judge that reads nothing judges
+      # nothing — so they are refused at construction instead.
+      def judgeable(questions)
+        missing = %i[leaks_secret failure_class] - questions.keys.map(&:to_sym)
+        unless missing.empty?
+          raise ArgumentError, "the output judge needs questions for #{missing.inspect}"
+        end
+
+        criteria = Array(questions[:failure_class].criteria&.keys)
+        unless criteria.include?(SUCCESS_CLASS)
+          raise ArgumentError,
+            "the failure_class question must offer #{SUCCESS_CLASS.inspect} among its criteria, " \
+            "or every successful call reads as a failure"
+        end
+
+        questions
+      end
+
       def truncate(str, limit)
         return "" if str.nil?
         str.length > limit ? "#{str[0, limit]}…[#{str.length - limit} chars elided]" : str
@@ -91,17 +134,17 @@ module Ask
       class OutputResult
         attr_reader :leak_noul, :failure_class, :failure_confidence, :advice
 
-        def initialize(batch, leak_threshold, failure_threshold)
+        def initialize(batch, leak_threshold, failure_threshold, advice: ADVICE)
           leak_answer = batch["leaks_secret"]
           failure_answer = batch["failure_class"]
 
           @leak_noul = leak_answer&.noul || 0.0
           @leak_threshold = leak_threshold
 
-          @failure_class = failure_answer&.choice || "no_failure"
+          @failure_class = failure_answer&.choice || SUCCESS_CLASS
           @failure_confidence = failure_answer&.confidence || 0.0
           @failure_threshold = failure_threshold
-          @advice = ADVICE[@failure_class]
+          @advice = advice[@failure_class]
         end
 
         def leak?
@@ -109,7 +152,7 @@ module Ask
         end
 
         def failure?
-          @failure_class != "no_failure" && @failure_confidence >= @failure_threshold
+          @failure_class != SUCCESS_CLASS && @failure_confidence >= @failure_threshold
         end
 
         def to_s

data/lib/ask/decisions/reader.rb

@@ -12,8 +12,7 @@ module Ask
     # along in the same request.
     #
     # The caller owns *what* to ask — which options, in what words. This owns
-    # asking it once and handing back the answers. `ToolRouter` and `Triage`
-    # are both façades over it.
+    # asking it once and handing back the answers.
     #
     #   reader = Ask::Decisions::Reader.new(
     #     provider,

data/lib/ask/decisions/version.rb

@@ -2,6 +2,6 @@
 
 module Ask
   module Decisions
-    VERSION = "0.1.1"
+    VERSION = "0.2.1"
   end
 end

data/lib/ask-decisions.rb

@@ -15,7 +15,6 @@ require_relative "ask/decisions/gate"
 require_relative "ask/decisions/output_judge"
 require_relative "ask/decisions/reader"
 require_relative "ask/decisions/triage"
-require_relative "ask/decisions/tool_router"
 require_relative "ask/decisions/argument_resolver"
 require_relative "ask/decisions/decision_state"
 require_relative "ask/decisions/failure_classifier"

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: ask-decisions
 version: !ruby/object:Gem::Version
-  version: 0.1.1
+  version: 0.2.1
 platform: ruby
 authors:
 - Kaka Ruto
@@ -169,7 +169,6 @@ files:
 - lib/ask/decisions/static.rb
 - lib/ask/decisions/structured_state_loop.rb
 - lib/ask/decisions/tool_repairer.rb
-- lib/ask/decisions/tool_router.rb
 - lib/ask/decisions/triage.rb
 - lib/ask/decisions/typesafe.rb
 - lib/ask/decisions/version.rb

data/lib/ask/decisions/tool_router.rb

@@ -1,137 +0,0 @@
-# frozen_string_literal: true
-
-module Ask
-  module Decisions
-    # Routes a user turn to the right tool by asking Jev to pick from the
-    # tool roster. Emits the same shape as a tool call so the existing
-    # ToolExecutor can run it unchanged.
-    #
-    #   router = Ask::Decisions::ToolRouter.new(provider, tools: tool_roster)
-    #   result = router.route(
-    #     user_turn: "check the weather in seattle tomorrow",
-    #     recent_turns: [...]
-    #   )
-    #   result.tool      # => "web_search"
-    #   result.confidence  # => 0.92
-    #   result.answer_directly?  # => false
-    #
-    class ToolRouter
-      # Non-tool outcomes that the Choice question includes.
-      NON_TOOL_OUTCOMES = {
-        "answer_directly" => "Answer the user directly without calling any tool",
-        "ask_clarifying_question" => "Ask the user a clarifying question before acting",
-        "none" => "No action needed; the turn is a follow-up or acknowledgment"
-      }.freeze
-
-      # The question the roster answers. The ids in +criteria+ are the tool
-      # names, so the answer comes back as a tool the caller can run.
-      INSTRUCTIONS = "Which tool should the assistant use to handle the user's latest request? " \
-                     "If no tool is needed, pick answer_directly, ask_clarifying_question, or none."
-
-      # @param provider [Ask::DecisionProvider]
-      # @param tools [Array<Hash>] tool roster, each with "name" and "description"
-      # @param none_threshold [Float] below this confidence, fall back to LLM
-      # @param criteria [Hash, nil] routing-grade descriptions, tool name =>
-      #   when to choose it. Worth supplying whenever the roster holds tools
-      #   that overlap: a tool's own description is written for the model that
-      #   already holds it, and two accurate descriptions can still fail to
-      #   separate their tools from the outside. Omitted, each tool's own
-      #   description is used.
-      # @param limit [Integer] characters kept per description
-      def initialize(provider, tools:, none_threshold: 0.5, criteria: nil, limit: 160)
-        @provider = provider
-        @tools = tools
-        @none_threshold = none_threshold
-        @criteria = criteria
-        @limit = limit
-      end
-
-      # Route a user turn to a tool or non-tool outcome.
-      #
-      # @param user_turn [String] the latest user message
-      # @param recent_turns [String, nil] recent conversation context (truncated)
-      # @param model [String, nil] model override
-      # @return [RouteResult]
-      def route(user_turn:, recent_turns: nil, model: nil)
-        state = build_state(user_turn: user_turn, recent_turns: recent_turns)
-        RouteResult.new(reader.choice(reader.read(state: state, model: model)))
-      end
-
-      private
-
-      # The roster, as the options of one Choice question.
-      def reader
-        @reader ||= Ask::Decisions::Reader.new(
-          @provider,
-          id: "tool.route",
-          instructions: INSTRUCTIONS,
-          options: described_roster,
-          limit: @limit
-        )
-      end
-
-      def described_roster
-        described = @tools.each_with_object({}) do |tool, options|
-          name = tool[:name] || tool["name"]
-          options[name] = @criteria&.dig(name) || @criteria&.dig(name.to_s) ||
-                          tool[:description] || tool["description"] || ""
-        end
-        described.merge(NON_TOOL_OUTCOMES)
-      end
-
-      def build_state(user_turn:, recent_turns: nil)
-        state = {user_turn: user_turn}
-        state[:recent_turns] = truncate(recent_turns, 2000) if recent_turns
-        state
-      end
-
-      def truncate(str, limit)
-        return "" if str.nil?
-        str.length > limit ? "#{str[0, limit]}…" : str
-      end
-
-
-      # Result of routing.
-      class RouteResult
-        attr_reader :choice_answer
-
-        def initialize(choice_answer)
-          @choice_answer = choice_answer
-        end
-
-        # The selected tool name or non-tool outcome.
-        def tool = choice_answer&.choice
-
-        def confidence = choice_answer&.confidence
-
-        def probabilities = choice_answer&.probabilities
-
-        # Should we call a tool, or handle this differently?
-        def answer_directly?  = tool == "answer_directly"
-        def ask_clarifying?   = tool == "ask_clarifying_question"
-        def no_action?        = tool == "none"
-        def call_tool?        = !answer_directly? && !ask_clarifying? && !no_action?
-
-        # Is the confidence above the threshold for autonomous action?
-        def confident?(threshold = nil)
-          threshold ||= 0.7
-          return false if confidence.nil?
-          confidence >= threshold
-        end
-
-        # Should we fall back to the LLM loop?
-        def fallback?(none_threshold = 0.5)
-          confidence.nil? || confidence < none_threshold
-        end
-
-        def to_s
-          if call_tool?
-            "tool: #{tool} (#{('%.2f' % (confidence || 0))})"
-          else
-            "#{tool} (#{('%.2f' % (confidence || 0))})"
-          end
-        end
-      end
-    end
-  end
-end