ask-decisions 0.1.1 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +19 -2
- data/README.md +20 -8
- data/lib/ask/decisions/reader.rb +1 -2
- data/lib/ask/decisions/version.rb +1 -1
- data/lib/ask-decisions.rb +0 -1
- metadata +1 -2
- data/lib/ask/decisions/tool_router.rb +0 -137
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: '09c0d2c768b78986cc452b899fca0579c7030fe8d4e9b067dcc6d38b8aebe544'
|
|
4
|
+
data.tar.gz: 6af44122d2741d29f5897c655cba0f38e99596dff88fec8152a95237957d514f
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: fd023d73dd39cffa709fbf27f651dfddfe39590bc77ca6874eac5d2c85fe10e8bcc3b1df51757eb6d612df6fde8f1691ad1cb704c74b26ace67349c845b6e8c3
|
|
7
|
+
data.tar.gz: ef8451ad571c3d52d20b17641ddee5a01e24ae39ea1e7a1e976ab0743dd375d2775f27d569b4cc451111df9543359a2bd00ce0f4ef5cc40ee8ce2cb3cdefeb14
|
data/CHANGELOG.md
CHANGED
|
@@ -4,7 +4,24 @@ All notable changes to this project will be documented in this file.
|
|
|
4
4
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
|
6
6
|
|
|
7
|
-
## [0.
|
|
7
|
+
## [0.2.0] - 2026-09-18
|
|
8
|
+
|
|
9
|
+
### Removed
|
|
10
|
+
|
|
11
|
+
- **`Ask::Decisions::ToolRouter` — a router that picks one of N tools from the
|
|
12
|
+
message alone.** It is gone because the measurement below is the argument
|
|
13
|
+
against it: routing to a tool was right 10/16 where routing to a lane was
|
|
14
|
+
right 19/20, on the same roster with the same model. Overlapping tools
|
|
15
|
+
cannot be separated by a message — which of seven knowledge tools holds the
|
|
16
|
+
answer is discovered by calling them — so a router answering that question
|
|
17
|
+
is guessing with confidence. The lane withholds the tools a turn cannot
|
|
18
|
+
need and the model chooses within the narrow roster, where it can see the
|
|
19
|
+
candidates' results.
|
|
20
|
+
|
|
21
|
+
Nothing in the ecosystem called it: not ask-agent, not ask-anychat, not any
|
|
22
|
+
app. Take this as the cheap moment — the class has no users yet.
|
|
23
|
+
|
|
24
|
+
## [0.1.1] - 2026-09-18
|
|
8
25
|
|
|
9
26
|
The first release, so everything here is new. It is a decision layer for the
|
|
10
27
|
ask-rb ecosystem: ask Jev (or any System One model) typed questions and get
|
|
@@ -15,7 +32,7 @@ decide.
|
|
|
15
32
|
|
|
16
33
|
- `Ask::Decisions::Reader` — asks a described set of options as one Choice,
|
|
17
34
|
with anything else the caller needs riding along in the same request.
|
|
18
|
-
`
|
|
35
|
+
`Triage` is a façade over it.
|
|
19
36
|
- `Ask::Decisions::Triage` — reads a message into a caller-defined lane and
|
|
20
37
|
asks the mood and whether the person wants a human, all in one request.
|
|
21
38
|
Measured against a 19-tool roster: lane-level routing was right 19/20 where
|
data/README.md
CHANGED
|
@@ -177,16 +177,28 @@ not carry. A lane is the part that *is* decidable from the message alone.
|
|
|
177
177
|
A reading should narrow, never grant: let the lane take tools away from a
|
|
178
178
|
turn, and let the agent's own definition stay the ceiling.
|
|
179
179
|
|
|
180
|
-
|
|
180
|
+
### Why there is no tool router
|
|
181
181
|
|
|
182
|
-
|
|
182
|
+
There was one — `ToolRouter`, a Choice over a tool roster. It is gone, because
|
|
183
|
+
the measurement above is the argument against it: asked to pick one of
|
|
184
|
+
nineteen tools the answer was right 10 times in 16, and asked to pick a lane
|
|
185
|
+
19 times in 20. Same model, same messages.
|
|
183
186
|
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
187
|
+
The reason is structural, not a tuning problem. Overlapping tools cannot be
|
|
188
|
+
separated by a message: which of seven knowledge tools holds the answer is
|
|
189
|
+
discovered by *calling* them. Routing to a tool asks a question the message
|
|
190
|
+
does not carry, so a router that answers it is guessing with confidence.
|
|
191
|
+
|
|
192
|
+
What replaces it is the lane plus code:
|
|
193
|
+
|
|
194
|
+
- the lane withholds the tools the turn cannot need,
|
|
195
|
+
- the lane's pre-read fetches what the turn will obviously ask for,
|
|
196
|
+
- and the model chooses within the narrow roster, where choosing is a
|
|
197
|
+
decision it can actually make — because it can see the candidates' results.
|
|
198
|
+
|
|
199
|
+
If you do need a decider to pick a tool, the roster it picks from has to be
|
|
200
|
+
small and disjoint — a handful of tools a message can actually distinguish.
|
|
201
|
+
If it is not, the fix is a narrower lane or a pre-read, not a better prompt.
|
|
190
202
|
|
|
191
203
|
## ConfidencePolicy
|
|
192
204
|
|
data/lib/ask/decisions/reader.rb
CHANGED
|
@@ -12,8 +12,7 @@ module Ask
|
|
|
12
12
|
# along in the same request.
|
|
13
13
|
#
|
|
14
14
|
# The caller owns *what* to ask — which options, in what words. This owns
|
|
15
|
-
# asking it once and handing back the answers.
|
|
16
|
-
# are both façades over it.
|
|
15
|
+
# asking it once and handing back the answers.
|
|
17
16
|
#
|
|
18
17
|
# reader = Ask::Decisions::Reader.new(
|
|
19
18
|
# provider,
|
data/lib/ask-decisions.rb
CHANGED
|
@@ -15,7 +15,6 @@ require_relative "ask/decisions/gate"
|
|
|
15
15
|
require_relative "ask/decisions/output_judge"
|
|
16
16
|
require_relative "ask/decisions/reader"
|
|
17
17
|
require_relative "ask/decisions/triage"
|
|
18
|
-
require_relative "ask/decisions/tool_router"
|
|
19
18
|
require_relative "ask/decisions/argument_resolver"
|
|
20
19
|
require_relative "ask/decisions/decision_state"
|
|
21
20
|
require_relative "ask/decisions/failure_classifier"
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: ask-decisions
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.2.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Kaka Ruto
|
|
@@ -169,7 +169,6 @@ files:
|
|
|
169
169
|
- lib/ask/decisions/static.rb
|
|
170
170
|
- lib/ask/decisions/structured_state_loop.rb
|
|
171
171
|
- lib/ask/decisions/tool_repairer.rb
|
|
172
|
-
- lib/ask/decisions/tool_router.rb
|
|
173
172
|
- lib/ask/decisions/triage.rb
|
|
174
173
|
- lib/ask/decisions/typesafe.rb
|
|
175
174
|
- lib/ask/decisions/version.rb
|
|
@@ -1,137 +0,0 @@
|
|
|
1
|
-
# frozen_string_literal: true
|
|
2
|
-
|
|
3
|
-
module Ask
|
|
4
|
-
module Decisions
|
|
5
|
-
# Routes a user turn to the right tool by asking Jev to pick from the
|
|
6
|
-
# tool roster. Emits the same shape as a tool call so the existing
|
|
7
|
-
# ToolExecutor can run it unchanged.
|
|
8
|
-
#
|
|
9
|
-
# router = Ask::Decisions::ToolRouter.new(provider, tools: tool_roster)
|
|
10
|
-
# result = router.route(
|
|
11
|
-
# user_turn: "check the weather in seattle tomorrow",
|
|
12
|
-
# recent_turns: [...]
|
|
13
|
-
# )
|
|
14
|
-
# result.tool # => "web_search"
|
|
15
|
-
# result.confidence # => 0.92
|
|
16
|
-
# result.answer_directly? # => false
|
|
17
|
-
#
|
|
18
|
-
class ToolRouter
|
|
19
|
-
# Non-tool outcomes that the Choice question includes.
|
|
20
|
-
NON_TOOL_OUTCOMES = {
|
|
21
|
-
"answer_directly" => "Answer the user directly without calling any tool",
|
|
22
|
-
"ask_clarifying_question" => "Ask the user a clarifying question before acting",
|
|
23
|
-
"none" => "No action needed; the turn is a follow-up or acknowledgment"
|
|
24
|
-
}.freeze
|
|
25
|
-
|
|
26
|
-
# The question the roster answers. The ids in +criteria+ are the tool
|
|
27
|
-
# names, so the answer comes back as a tool the caller can run.
|
|
28
|
-
INSTRUCTIONS = "Which tool should the assistant use to handle the user's latest request? " \
|
|
29
|
-
"If no tool is needed, pick answer_directly, ask_clarifying_question, or none."
|
|
30
|
-
|
|
31
|
-
# @param provider [Ask::DecisionProvider]
|
|
32
|
-
# @param tools [Array<Hash>] tool roster, each with "name" and "description"
|
|
33
|
-
# @param none_threshold [Float] below this confidence, fall back to LLM
|
|
34
|
-
# @param criteria [Hash, nil] routing-grade descriptions, tool name =>
|
|
35
|
-
# when to choose it. Worth supplying whenever the roster holds tools
|
|
36
|
-
# that overlap: a tool's own description is written for the model that
|
|
37
|
-
# already holds it, and two accurate descriptions can still fail to
|
|
38
|
-
# separate their tools from the outside. Omitted, each tool's own
|
|
39
|
-
# description is used.
|
|
40
|
-
# @param limit [Integer] characters kept per description
|
|
41
|
-
def initialize(provider, tools:, none_threshold: 0.5, criteria: nil, limit: 160)
|
|
42
|
-
@provider = provider
|
|
43
|
-
@tools = tools
|
|
44
|
-
@none_threshold = none_threshold
|
|
45
|
-
@criteria = criteria
|
|
46
|
-
@limit = limit
|
|
47
|
-
end
|
|
48
|
-
|
|
49
|
-
# Route a user turn to a tool or non-tool outcome.
|
|
50
|
-
#
|
|
51
|
-
# @param user_turn [String] the latest user message
|
|
52
|
-
# @param recent_turns [String, nil] recent conversation context (truncated)
|
|
53
|
-
# @param model [String, nil] model override
|
|
54
|
-
# @return [RouteResult]
|
|
55
|
-
def route(user_turn:, recent_turns: nil, model: nil)
|
|
56
|
-
state = build_state(user_turn: user_turn, recent_turns: recent_turns)
|
|
57
|
-
RouteResult.new(reader.choice(reader.read(state: state, model: model)))
|
|
58
|
-
end
|
|
59
|
-
|
|
60
|
-
private
|
|
61
|
-
|
|
62
|
-
# The roster, as the options of one Choice question.
|
|
63
|
-
def reader
|
|
64
|
-
@reader ||= Ask::Decisions::Reader.new(
|
|
65
|
-
@provider,
|
|
66
|
-
id: "tool.route",
|
|
67
|
-
instructions: INSTRUCTIONS,
|
|
68
|
-
options: described_roster,
|
|
69
|
-
limit: @limit
|
|
70
|
-
)
|
|
71
|
-
end
|
|
72
|
-
|
|
73
|
-
def described_roster
|
|
74
|
-
described = @tools.each_with_object({}) do |tool, options|
|
|
75
|
-
name = tool[:name] || tool["name"]
|
|
76
|
-
options[name] = @criteria&.dig(name) || @criteria&.dig(name.to_s) ||
|
|
77
|
-
tool[:description] || tool["description"] || ""
|
|
78
|
-
end
|
|
79
|
-
described.merge(NON_TOOL_OUTCOMES)
|
|
80
|
-
end
|
|
81
|
-
|
|
82
|
-
def build_state(user_turn:, recent_turns: nil)
|
|
83
|
-
state = {user_turn: user_turn}
|
|
84
|
-
state[:recent_turns] = truncate(recent_turns, 2000) if recent_turns
|
|
85
|
-
state
|
|
86
|
-
end
|
|
87
|
-
|
|
88
|
-
def truncate(str, limit)
|
|
89
|
-
return "" if str.nil?
|
|
90
|
-
str.length > limit ? "#{str[0, limit]}…" : str
|
|
91
|
-
end
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
# Result of routing.
|
|
95
|
-
class RouteResult
|
|
96
|
-
attr_reader :choice_answer
|
|
97
|
-
|
|
98
|
-
def initialize(choice_answer)
|
|
99
|
-
@choice_answer = choice_answer
|
|
100
|
-
end
|
|
101
|
-
|
|
102
|
-
# The selected tool name or non-tool outcome.
|
|
103
|
-
def tool = choice_answer&.choice
|
|
104
|
-
|
|
105
|
-
def confidence = choice_answer&.confidence
|
|
106
|
-
|
|
107
|
-
def probabilities = choice_answer&.probabilities
|
|
108
|
-
|
|
109
|
-
# Should we call a tool, or handle this differently?
|
|
110
|
-
def answer_directly? = tool == "answer_directly"
|
|
111
|
-
def ask_clarifying? = tool == "ask_clarifying_question"
|
|
112
|
-
def no_action? = tool == "none"
|
|
113
|
-
def call_tool? = !answer_directly? && !ask_clarifying? && !no_action?
|
|
114
|
-
|
|
115
|
-
# Is the confidence above the threshold for autonomous action?
|
|
116
|
-
def confident?(threshold = nil)
|
|
117
|
-
threshold ||= 0.7
|
|
118
|
-
return false if confidence.nil?
|
|
119
|
-
confidence >= threshold
|
|
120
|
-
end
|
|
121
|
-
|
|
122
|
-
# Should we fall back to the LLM loop?
|
|
123
|
-
def fallback?(none_threshold = 0.5)
|
|
124
|
-
confidence.nil? || confidence < none_threshold
|
|
125
|
-
end
|
|
126
|
-
|
|
127
|
-
def to_s
|
|
128
|
-
if call_tool?
|
|
129
|
-
"tool: #{tool} (#{('%.2f' % (confidence || 0))})"
|
|
130
|
-
else
|
|
131
|
-
"#{tool} (#{('%.2f' % (confidence || 0))})"
|
|
132
|
-
end
|
|
133
|
-
end
|
|
134
|
-
end
|
|
135
|
-
end
|
|
136
|
-
end
|
|
137
|
-
end
|