smith-agents 0.12.7 → 0.13.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +78 -0
- data/README.md +6 -2
- data/docs/AGENTS.md +9 -5
- data/docs/ERRORS_AND_RETRIES.md +20 -4
- data/docs/PATTERNS.md +5 -3
- data/docs/PERSISTENCE.md +4 -1
- data/docs/TOOLS_AND_GUARDRAILS.md +86 -6
- data/lib/smith/agent/attempt_captures.rb +62 -0
- data/lib/smith/agent/lifecycle.rb +7 -3
- data/lib/smith/agent/output_integrity.rb +7 -6
- data/lib/smith/agent/output_recovery.rb +6 -4
- data/lib/smith/agent/provider_attempt.rb +3 -0
- data/lib/smith/agent/provider_completion.rb +18 -16
- data/lib/smith/agent/provider_failure_handling.rb +5 -3
- data/lib/smith/agent/provider_streaming.rb +3 -1
- data/lib/smith/errors.rb +6 -3
- data/lib/smith/response_memo.rb +2 -1
- data/lib/smith/tool/budget_enforcement.rb +1 -1
- data/lib/smith/tool/capability_builder.rb +14 -1
- data/lib/smith/tool/capability_declaration.rb +45 -0
- data/lib/smith/tool/capture_configuration.rb +5 -2
- data/lib/smith/tool/declarations.rb +73 -0
- data/lib/smith/tool/execution_tracker.rb +13 -2
- data/lib/smith/tool.rb +8 -38
- data/lib/smith/version.rb +1 -1
- data/lib/smith/workflow/budget_pause.rb +8 -1
- data/lib/smith/workflow/guarded_step_execution.rb +8 -8
- data/lib/smith/workflow/nested_execution.rb +3 -3
- data/lib/smith/workflow/remembered_calls.rb +14 -2
- data/lib/smith/workflow/retry_execution.rb +24 -3
- data/lib/smith/workflow/retry_wait.rb +2 -0
- data/lib/smith/workflow.rb +11 -0
- metadata +4 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 22c04af0f6062eb36758bb0fedc62663d68a27dce67f016942038ffc1ee3cc07
|
|
4
|
+
data.tar.gz: 0243fe30beb96d8bccc37869cb6ece9a3525ecd83004de98e18d8f97f4de8b8c
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 41b4452c51e7aa3e6946c192e0f6a9eb9bd89163c90886f013dfffd891da55a53f40ce30b00eb90ecebdc5adab3f7373687c7e0ae64be3e32085d6bfc05af47d
|
|
7
|
+
data.tar.gz: 4007dfee19e37843c993cf9abf5ea57a6e7450141594090fb0579ae013437de6e3053de16c382a88180bb5536b84fdbe951270e0e6707e7a0c73daec9036a569
|
data/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,84 @@ All notable changes to Smith are documented in this file.
|
|
|
4
4
|
|
|
5
5
|
Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). Versioning follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html). Smith is pre-1.0 and under active development; expect occasional contract tightening between minor versions until 1.0.
|
|
6
6
|
|
|
7
|
+
## [0.13.0] - 2026-11-01
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
|
|
11
|
+
- A tool can declare `capabilities { replay :safe }`: running it again under
|
|
12
|
+
the same keys does what running it once did (a lookup, a search, a read,
|
|
13
|
+
one that calls a paid API included). Once a tool had run in a step, Smith
|
|
14
|
+
made no output repair, refusal fallback or truncation retry, took no
|
|
15
|
+
fallback model after a provider failure, and turned the step's failure
|
|
16
|
+
into `Smith::ToolOutcomeUncertain`, which no retry takes. Now only a tool
|
|
17
|
+
that is not replay-safe does that (any tool without the declaration, a
|
|
18
|
+
plain RubyLLM tool included). When every tool that ran in a step is
|
|
19
|
+
replay-safe, the step keeps its follow-ups, its fallback models and its
|
|
20
|
+
`retry_on`, durable retries included; each runs the tools again, under the
|
|
21
|
+
same idempotency keys and within the same `tool_calls` allowances. A
|
|
22
|
+
provider attempt set aside for a follow-up or a fallback model takes its
|
|
23
|
+
tools' captures out of the step's `tool_results` once the call has its
|
|
24
|
+
output (by identity, so a concurrent fan-out branch's stay; a call that
|
|
25
|
+
fails after all keeps every capture), and so does a retried step attempt
|
|
26
|
+
once its retry is certain, unless the retry continues a tool loop the
|
|
27
|
+
step paused for budget. Tool evidence reaches a parent step from a nested
|
|
28
|
+
workflow as it was, and a call the response memo replays counts its kept
|
|
29
|
+
tool results by the tools its chat declares. `replay :unsafe` states the
|
|
30
|
+
default; any other value raises `ArgumentError`; `ToolClass.replay_safe?`
|
|
31
|
+
reads it. Each class declares it for itself: a subclass of a replay-safe
|
|
32
|
+
tool is unsafe until it says `replay :safe` too.
|
|
33
|
+
|
|
34
|
+
### Changed
|
|
35
|
+
|
|
36
|
+
- A `Smith::Tool` subclass inherits its parent's declarations, each read
|
|
37
|
+
from the nearest class that made it, so a subclass overrides one by
|
|
38
|
+
declaring it again. Before, a subclass inherited none of them: it ran
|
|
39
|
+
without its parent's `authorize` (failing open), `privilege`, trace
|
|
40
|
+
`sensitivity` and `capture_result`, and reached the model without its
|
|
41
|
+
parent's description and declared parameters (a host subclass of the
|
|
42
|
+
shipped `Tools::WebSearch` or `Tools::UrlFetcher` stub sent no
|
|
43
|
+
description and parameters guessed from its `perform`, all strings).
|
|
44
|
+
`description`, `with_params`, `category`, `authorize`, `before_execute`,
|
|
45
|
+
`compatible_with` and `capture_result` (its block and strictness
|
|
46
|
+
together) are the nearest declared; `param`s add up by name, up to a
|
|
47
|
+
class that gave a `params` schema; a `params` schema is the nearest
|
|
48
|
+
unless the subclass declares `param`s of its own; `capabilities` merge
|
|
49
|
+
key by key, except `replay`, which is never inherited: a subclass of a
|
|
50
|
+
replay-safe tool is replay-safe only when it declares `replay :safe`
|
|
51
|
+
itself, as safety to run again depends on its own `perform`. A tool none
|
|
52
|
+
of whose classes declares `authorize` runs as an undeclared tool always
|
|
53
|
+
has, and a declaration made on a parent after its subclass was created
|
|
54
|
+
reaches the subclass (`compatible_with` was copied when the subclass was
|
|
55
|
+
made). Capture was documented as opt-in per concrete class; it is now
|
|
56
|
+
inherited too.
|
|
57
|
+
|
|
58
|
+
### Fixed
|
|
59
|
+
|
|
60
|
+
- A budget refusal that carries a paused tool loop
|
|
61
|
+
(`Smith::BudgetUnavailable#continuation`) spares a step from
|
|
62
|
+
`Smith::ToolOutcomeUncertain`, and is continued by its retry, only in the
|
|
63
|
+
run whose step paused (`paused_by`). One raised from a nested workflow, or
|
|
64
|
+
made by host code, reached the parent step as if the parent could
|
|
65
|
+
continue it, so the parent's retry started the child over and ran its
|
|
66
|
+
tools again, and could take the child's paused messages as its own.
|
|
67
|
+
|
|
68
|
+
### Upgrading
|
|
69
|
+
|
|
70
|
+
- A tool subclass that declared less than its parent now runs with its
|
|
71
|
+
parent's declarations: its parent's `authorize`, `privilege` and
|
|
72
|
+
`before_execute` apply (a call they deny raises `Smith::ToolPolicyDenied`),
|
|
73
|
+
its strict capture fails closed without a collector, its traces follow
|
|
74
|
+
its parent's `sensitivity`, and its requests carry its parent's
|
|
75
|
+
description and parameters. A subclass meant to differ declares its own
|
|
76
|
+
(`authorize { ... }`, `capabilities { privilege :none }`,
|
|
77
|
+
`capture_result(strict: false) { ... }`). As its
|
|
78
|
+
requests change, the response memo misses once for a call with such a
|
|
79
|
+
tool (its digest covers tools' descriptions and parameters), so a step
|
|
80
|
+
cut by the upgrade's deploy pays for that call again and runs its tools
|
|
81
|
+
again.
|
|
82
|
+
- `Smith::EXECUTION_SEMANTICS_VERSION` stays `"7"`: no persisted state,
|
|
83
|
+
wait, pending retry or durable plan changes shape.
|
|
84
|
+
|
|
7
85
|
## [0.12.7] - 2026-10-13
|
|
8
86
|
|
|
9
87
|
### Added
|
data/README.md
CHANGED
|
@@ -30,7 +30,7 @@ What you get:
|
|
|
30
30
|
|
|
31
31
|
```ruby
|
|
32
32
|
# Gemfile
|
|
33
|
-
gem "smith-agents", "0.
|
|
33
|
+
gem "smith-agents", "0.13.0", require: "smith"
|
|
34
34
|
```
|
|
35
35
|
|
|
36
36
|
In a Rails app:
|
|
@@ -240,7 +240,11 @@ everything else. Messages can carry documents and images.
|
|
|
240
240
|
## Tools and guardrails
|
|
241
241
|
|
|
242
242
|
Agents can use `Smith::Tool` subclasses, with a budget on how many calls a
|
|
243
|
-
step may make.
|
|
243
|
+
step may make. Once a tool has run, a step is not retried and takes no
|
|
244
|
+
fallback, as its tools would run again; a tool that is safe to run again (a
|
|
245
|
+
lookup) declares `capabilities { replay :safe }`, and a step whose tools were
|
|
246
|
+
all safe keeps its retries, fallbacks and output repair. Guardrails check
|
|
247
|
+
inputs, outputs and tool calls. Smith ships
|
|
244
248
|
`Tools::Think`; `Tools::WebSearch` and `Tools::UrlFetcher` are stubs that
|
|
245
249
|
declare the interface, which you subclass with your own search or fetch.
|
|
246
250
|
[`docs/TOOLS_AND_GUARDRAILS.md`](docs/TOOLS_AND_GUARDRAILS.md)
|
data/docs/AGENTS.md
CHANGED
|
@@ -50,11 +50,15 @@ end
|
|
|
50
50
|
problems as "path: problem" lines. It then raises `Smith::OutputInvalid`,
|
|
51
51
|
whose `problems` lists at most 20 of them. A blank output still raises
|
|
52
52
|
`Smith::BlankAgentOutputError`.
|
|
53
|
-
- No follow-up call is made once a tool
|
|
54
|
-
run again; the step fails with
|
|
55
|
-
error) or once the call has
|
|
56
|
-
|
|
57
|
-
|
|
53
|
+
- No follow-up call is made once a tool that is not replay-safe has run in
|
|
54
|
+
the step (the tools would run again; the step fails with
|
|
55
|
+
`Smith::ToolOutcomeUncertain` around the output error) or once the call has
|
|
56
|
+
spent what its step reserved from a budget. After replay-safe tools only
|
|
57
|
+
(`capabilities { replay :safe }`, docs/TOOLS_AND_GUARDRAILS.md), the
|
|
58
|
+
follow-up is made and its model may call them again; once there is an
|
|
59
|
+
output, the set-aside reply's tool captures leave `tool_results`. A
|
|
60
|
+
follow-up that fails raises the output error it was making up for, with
|
|
61
|
+
the provider's error as its cause.
|
|
58
62
|
- Anthropic's structured outputs reject numeric, string-length and most
|
|
59
63
|
array-size constraints: Smith sends Anthropic the schema without them, with
|
|
60
64
|
`additionalProperties: false` on every object, and checks them on the parsed
|
data/docs/ERRORS_AND_RETRIES.md
CHANGED
|
@@ -15,9 +15,24 @@ Smith::Errors.retry_forbidden?(error)
|
|
|
15
15
|
```
|
|
16
16
|
|
|
17
17
|
A nested workflow's step fails with the child's own error when the child
|
|
18
|
-
raises it, so these answers carry through nesting. When a tool
|
|
19
|
-
child, the parent step's failure becomes
|
|
20
|
-
retried.
|
|
18
|
+
raises it, so these answers carry through nesting. When a tool that is not
|
|
19
|
+
replay-safe ran inside the child, the parent step's failure becomes
|
|
20
|
+
`ToolOutcomeUncertain`, which is never retried.
|
|
21
|
+
|
|
22
|
+
## After tools ran
|
|
23
|
+
|
|
24
|
+
A step's failure after one of its tools ran is `ToolOutcomeUncertain`, which
|
|
25
|
+
no retry takes, and the step takes no fallback model and makes no follow-up
|
|
26
|
+
call for its output: running the tools again could repeat what they did. A
|
|
27
|
+
tool that declares `capabilities { replay :safe }` (a lookup, a search, a
|
|
28
|
+
read) says running it again does no harm. When every tool that ran in the
|
|
29
|
+
step is replay-safe, the step keeps its output repair, its refusal fallback
|
|
30
|
+
and truncation retry, its fallback models after a provider failure, and its
|
|
31
|
+
`retry_on`, durable retries included; each runs the tools again, under the
|
|
32
|
+
same idempotency keys and within the same `tool_calls` allowances, and the
|
|
33
|
+
captures of an attempt it sets aside leave `tool_results` once the step has
|
|
34
|
+
its output or its retry is certain. See
|
|
35
|
+
[TOOLS_AND_GUARDRAILS.md](TOOLS_AND_GUARDRAILS.md), Replay-Safe Tools.
|
|
21
36
|
|
|
22
37
|
Workflow transitions can also declare a bounded local retry policy:
|
|
23
38
|
|
|
@@ -185,7 +200,8 @@ had just come, once, and its `:provider_call` trace says `memo: true`; a reply
|
|
|
185
200
|
that was cut off or refused fails the step as it did. One that had not ended
|
|
186
201
|
goes on from where it was kept. Kept after its tools' results, it continues
|
|
187
202
|
from them: kept tool results count as tools the step ran, so no retry or
|
|
188
|
-
fallback runs them again
|
|
203
|
+
fallback runs them again (unless each is the result of a tool the chat
|
|
204
|
+
declares replay-safe), and each is charged to the agent's `tool_calls`
|
|
189
205
|
allowance and the workflow's `tool_calls` budget as it was. Kept while its
|
|
190
206
|
tools ran (the step was cut inside a tool), the reply that called them is not
|
|
191
207
|
asked for again: its tools run again, under the same idempotency keys, and the
|
data/docs/PATTERNS.md
CHANGED
|
@@ -332,9 +332,11 @@ What you get:
|
|
|
332
332
|
- a child that routes its failure through its own `on_failure` has handled it:
|
|
333
333
|
the parent step fails with a non-retryable `Smith::WorkflowError`
|
|
334
334
|
(`nested workflow failed: <child message>`, bounded to 1 KiB)
|
|
335
|
-
- tool evidence travels up: when a tool ran in any of
|
|
336
|
-
failure of the parent step becomes
|
|
337
|
-
retried, exactly as when the tool
|
|
335
|
+
- tool evidence travels up: when a tool that is not replay-safe ran in any of
|
|
336
|
+
the child's steps, a failure of the parent step becomes
|
|
337
|
+
`Smith::ToolOutcomeUncertain` and is never retried, exactly as when the tool
|
|
338
|
+
runs in the parent step itself (a child whose tools were all
|
|
339
|
+
`replay :safe` leaves the parent's retry)
|
|
338
340
|
- parent step count stays parent-scoped
|
|
339
341
|
- parent and child share the outer budget ledger
|
|
340
342
|
- nested best-known token/cost totals roll up into the parent result
|
data/docs/PERSISTENCE.md
CHANGED
|
@@ -935,7 +935,10 @@ end
|
|
|
935
935
|
runs under the step-in-progress mark, so the restore of a run caught in it
|
|
936
936
|
raises `Smith::StepInProgressOnRestore`.
|
|
937
937
|
- No retry wait is entered once the run's wall-clock deadline has passed, after
|
|
938
|
-
a tool ran (`Smith::ToolOutcomeUncertain` is never
|
|
938
|
+
a tool that is not replay-safe ran (`Smith::ToolOutcomeUncertain` is never
|
|
939
|
+
retried; after replay-safe tools only, the wait is entered, the run stored
|
|
940
|
+
without the failed attempt's tool captures, and the next attempt runs them
|
|
941
|
+
again), or for a backoff
|
|
939
942
|
that does not end within the store's persistence TTL (the step fails with a
|
|
940
943
|
`Smith::WorkflowError` whose cause is the attempt's error); runtime readiness
|
|
941
944
|
reports a policy whose longest backoff, jitter included, reaches the TTL
|
|
@@ -41,6 +41,85 @@ configuration, programming, authorization-integrity, or durability failures.
|
|
|
41
41
|
Smith applies policy and budgeting around that host boundary; it does not
|
|
42
42
|
interpret a host's input schema or failure envelope.
|
|
43
43
|
|
|
44
|
+
### Subclasses
|
|
45
|
+
|
|
46
|
+
A subclass inherits every declaration of its parent's but `replay`, each
|
|
47
|
+
read from the nearest class that made it, so a subclass overrides one by
|
|
48
|
+
declaring it again:
|
|
49
|
+
|
|
50
|
+
- `description`, `with_params`, `category`, `authorize`, `before_execute`,
|
|
51
|
+
`compatible_with` and `capture_result` (its block and strictness
|
|
52
|
+
together): the nearest declared wins
|
|
53
|
+
- `param`: parameters add up by name, so a subclass adds parameters or
|
|
54
|
+
replaces one of its parent's; a schema given with `params` is the
|
|
55
|
+
nearest one, unless the subclass declares `param`s of its own
|
|
56
|
+
- `capabilities`: merged key by key, so a subclass's block sets or replaces
|
|
57
|
+
only the keys it names and keeps its parent's `privilege` and
|
|
58
|
+
`sensitivity`; `replay` is never inherited (Replay-Safe Tools)
|
|
59
|
+
|
|
60
|
+
An inherited `authorize` guards the subclass as it guards its parent; a
|
|
61
|
+
subclass's own replaces it (call `superclass.authorize` inside it to keep
|
|
62
|
+
both checks), and a tool none of whose classes declares one runs as an
|
|
63
|
+
undeclared tool does. A subclass made before its parent declares something
|
|
64
|
+
still reads it.
|
|
65
|
+
|
|
66
|
+
### Replay-Safe Tools
|
|
67
|
+
|
|
68
|
+
A tool that does the same when it runs again under the same keys (a lookup
|
|
69
|
+
over the host's own data, a search, a read, even one that calls a paid API)
|
|
70
|
+
can say so:
|
|
71
|
+
|
|
72
|
+
```ruby
|
|
73
|
+
class FindPlaces < Smith::Tool
|
|
74
|
+
description "Find places that match a description."
|
|
75
|
+
param :query, type: :string, required: true
|
|
76
|
+
|
|
77
|
+
capabilities do
|
|
78
|
+
replay :safe
|
|
79
|
+
end
|
|
80
|
+
|
|
81
|
+
def perform(query:) = PlaceIndex.search(query, limit: 5)
|
|
82
|
+
end
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
Smith follows the tools each step attempt runs. Once a tool that is not
|
|
86
|
+
replay-safe has run (any tool without the declaration, a plain RubyLLM tool
|
|
87
|
+
included), the step makes no follow-up call for its output (the step fails
|
|
88
|
+
with `Smith::ToolOutcomeUncertain` around the output error), takes no
|
|
89
|
+
fallback model after a provider failure (`ToolOutcomeUncertain`), and a
|
|
90
|
+
failure of the step becomes `ToolOutcomeUncertain`, which no retry takes:
|
|
91
|
+
running the step's tools again could repeat what they did. When every tool
|
|
92
|
+
that ran is replay-safe, the step keeps all three:
|
|
93
|
+
|
|
94
|
+
- an output repair, a refusal fallback and a truncation retry are each a new
|
|
95
|
+
call from the step's messages (a repair adds the rejected reply and its
|
|
96
|
+
problems), so the model may call the tools again
|
|
97
|
+
- a provider failure takes the agent's fallback models, each with a tool
|
|
98
|
+
loop of its own
|
|
99
|
+
- a failure of the step is its own error, which `retry_on` retries (with
|
|
100
|
+
`durable: true`, a wait of the run), and the next attempt runs its tools
|
|
101
|
+
again
|
|
102
|
+
|
|
103
|
+
An attempt set aside for any of these takes its tools' captures
|
|
104
|
+
(`capture_result`, strict or not) out of the step's `tool_results` once the
|
|
105
|
+
call has its output, by identity, so a concurrent branch's are left; a call
|
|
106
|
+
that fails after all keeps every capture. A retried attempt's go too, once
|
|
107
|
+
the retry is certain (a durable retry's wait entered), unless the retry
|
|
108
|
+
continues a tool loop the step paused for budget, which keeps its tools'
|
|
109
|
+
results (one a nested workflow paused counts as any failure of its tools).
|
|
110
|
+
Tools that run again send the same idempotency keys, and count
|
|
111
|
+
against the agent's `tool_calls` allowance and the workflow's `tool_calls`
|
|
112
|
+
budget as any call does, so a step's tool calls stay bounded. Tool evidence
|
|
113
|
+
from a nested workflow reaches its parent step as it was: a child whose tools
|
|
114
|
+
were all replay-safe leaves the parent's retry. A call the response memo
|
|
115
|
+
replays counts its kept tool results the same way, by the tools its chat
|
|
116
|
+
declares. The declaration is never inherited, as whether a tool is safe to
|
|
117
|
+
run again depends on its own `perform`: a subclass of a replay-safe tool is
|
|
118
|
+
replay-safe only when it declares `replay :safe` itself (its other
|
|
119
|
+
capabilities, description, parameters and capture it inherits). `replay
|
|
120
|
+
:unsafe` states the default, any other value raises `ArgumentError`, and
|
|
121
|
+
`ToolClass.replay_safe?` reads the class's own declaration.
|
|
122
|
+
|
|
44
123
|
### Bounded Completion
|
|
45
124
|
|
|
46
125
|
The default exhaustion policy remains fail-fast:
|
|
@@ -186,9 +265,9 @@ entry by raising an exception. A collector return value of `nil` or `false` is
|
|
|
186
265
|
accepted; delivery is rejected only when the collector raises. Best-effort
|
|
187
266
|
capture logs and continues as before.
|
|
188
267
|
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
|
|
268
|
+
A subclass inherits its parent's capture, strictness included, and replaces it
|
|
269
|
+
with a `capture_result` of its own (Subclasses), so a subclass of a strictly
|
|
270
|
+
capturing tool fails closed as its parent does.
|
|
192
271
|
|
|
193
272
|
Strict capture validates that a collector exists before `perform`, but the
|
|
194
273
|
capture block and collector run after `perform` returns. It is therefore
|
|
@@ -459,7 +538,8 @@ Smith ships `Tools::WebSearch`, `Tools::UrlFetcher`, and `Tools::Think`. Tools d
|
|
|
459
538
|
`Tools::WebSearch` and `Tools::UrlFetcher` are stubs: they declare each
|
|
460
539
|
tool's parameters and description, and their `perform` raises
|
|
461
540
|
`NotImplementedError`. A host subclasses them with its own search or fetch
|
|
462
|
-
behind `perform
|
|
541
|
+
behind `perform`, and its subclass keeps their parameters, description and
|
|
542
|
+
compatibility (Subclasses). `Tools::Think` works as shipped.
|
|
463
543
|
|
|
464
544
|
```ruby
|
|
465
545
|
class WebSearch < Smith::Tools::WebSearch
|
|
@@ -499,8 +579,8 @@ Graceful exhaustion is an in-process protocol guarantee, not durable recovery
|
|
|
499
579
|
for the model/tool loop. If the process exits after a tool begins and before the
|
|
500
580
|
agent completes, the host must treat the outcome as uncertain unless it owns a
|
|
501
581
|
durable per-invocation receipt and reconciliation contract. Smith also refuses
|
|
502
|
-
to start a fresh fallback-model chat after tool
|
|
503
|
-
|
|
582
|
+
to start a fresh fallback-model chat after a tool that is not replay-safe has
|
|
583
|
+
begun, because the new chat would run its tools again (Replay-Safe Tools).
|
|
504
584
|
|
|
505
585
|
Within a host invocation scope, `Smith::Tool.current_invocation` exposes the
|
|
506
586
|
normalized tool-call ID and deterministic execution/batch ordinals. The ID is
|
|
@@ -0,0 +1,62 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Smith
|
|
4
|
+
class Agent
|
|
5
|
+
# The tool results one provider attempt captures, kept apart so an
|
|
6
|
+
# attempt the call sets aside (for a fallback model or a follow-up,
|
|
7
|
+
# possible only when its tools were replay-safe) takes them with it once
|
|
8
|
+
# the call has its output: the attempt that replaced it ran its tools
|
|
9
|
+
# again and captured anew. Each entry still reaches the step's collector
|
|
10
|
+
# as it comes.
|
|
11
|
+
module AttemptCaptures
|
|
12
|
+
# A collector that passes each entry on and remembers it.
|
|
13
|
+
class Scope
|
|
14
|
+
def initialize(collector)
|
|
15
|
+
@collector = collector
|
|
16
|
+
@entries = []
|
|
17
|
+
@lock = Mutex.new
|
|
18
|
+
end
|
|
19
|
+
|
|
20
|
+
def call(entry)
|
|
21
|
+
accepted = @collector.call(entry)
|
|
22
|
+
@lock.synchronize { @entries << entry }
|
|
23
|
+
accepted
|
|
24
|
+
end
|
|
25
|
+
|
|
26
|
+
def entries = @lock.synchronize { @entries.dup.freeze }
|
|
27
|
+
end
|
|
28
|
+
private_constant :Scope
|
|
29
|
+
|
|
30
|
+
private
|
|
31
|
+
|
|
32
|
+
# The chat's completion (ProviderStreaming#complete_chat), with what
|
|
33
|
+
# its tools capture recorded as `facts[:captures]`, the attempt's,
|
|
34
|
+
# however it ends. A step without a callable collector is left as it
|
|
35
|
+
# is, so strict capture still fails closed there.
|
|
36
|
+
def captured_completion(chat, model_reference, collected, facts)
|
|
37
|
+
collector = Tool.current_tool_result_collector
|
|
38
|
+
return complete_chat(chat, model_reference, collected, facts) unless collector.respond_to?(:call)
|
|
39
|
+
|
|
40
|
+
scope = Scope.new(collector)
|
|
41
|
+
Thread.handle_interrupt(Object => :never) do
|
|
42
|
+
Tool.current_tool_result_collector = scope
|
|
43
|
+
begin
|
|
44
|
+
Thread.handle_interrupt(Object => :immediate) { complete_chat(chat, model_reference, collected, facts) }
|
|
45
|
+
ensure
|
|
46
|
+
Tool.current_tool_result_collector = collector
|
|
47
|
+
facts[:captures] = scope.entries
|
|
48
|
+
end
|
|
49
|
+
end
|
|
50
|
+
end
|
|
51
|
+
|
|
52
|
+
# The accepted output; only now do the attempts set aside on the way
|
|
53
|
+
# take what their tools captured with them, so an invocation that
|
|
54
|
+
# fails after all keeps every capture, as before.
|
|
55
|
+
def accepted_output(agent_class, result, set_aside)
|
|
56
|
+
output = finalize_output(agent_class, result)
|
|
57
|
+
drop_tool_results(set_aside)
|
|
58
|
+
output
|
|
59
|
+
end
|
|
60
|
+
end
|
|
61
|
+
end
|
|
62
|
+
end
|
|
@@ -21,18 +21,22 @@ module Smith
|
|
|
21
21
|
end
|
|
22
22
|
|
|
23
23
|
# One invocation: the provider call, then whatever OutputIntegrity
|
|
24
|
-
# asks for until the response can be the output.
|
|
24
|
+
# asks for until the response can be the output. The attempts set aside
|
|
25
|
+
# on the way (`set_aside`) take what their tools captured with them
|
|
26
|
+
# once there is an output (AttemptCaptures).
|
|
25
27
|
def invoke_agent(agent_class, prepared_input, output_schema: effective_output_schema(agent_class))
|
|
26
28
|
check_deadline!
|
|
27
29
|
plan = output_plan(agent_class, prepared_input, output_schema)
|
|
28
|
-
|
|
30
|
+
set_aside = []
|
|
31
|
+
attempt = complete_with_provider(agent_class, prepared_input, shape: plan.shape, set_aside:)
|
|
29
32
|
loop do
|
|
30
33
|
result = account_attempt(agent_class, attempt.completion, attempt.model_reference,
|
|
31
34
|
attempt_id: attempt.attempt_id)
|
|
32
35
|
follow_up = output_follow_up(plan, attempt, result)
|
|
33
|
-
return
|
|
36
|
+
return accepted_output(agent_class, result, set_aside) unless follow_up
|
|
34
37
|
|
|
35
38
|
settle_rejected(result)
|
|
39
|
+
set_aside.concat(attempt.captures)
|
|
36
40
|
attempt = follow_up.call
|
|
37
41
|
end
|
|
38
42
|
end
|
|
@@ -16,12 +16,13 @@ module Smith
|
|
|
16
16
|
# output contract is sent back for repair up to `output_repairs`, then
|
|
17
17
|
# raises OutputInvalid.
|
|
18
18
|
#
|
|
19
|
-
# No follow-up call is made once a tool
|
|
20
|
-
# would run again), once the call has spent what
|
|
21
|
-
# past MAX_CALLS; a follow-up that fails raises the
|
|
22
|
-
# was making up for, with the provider's error as its
|
|
23
|
-
# response is billed; one that is set aside settles with
|
|
24
|
-
# invocation's failed attempts,
|
|
19
|
+
# No follow-up call is made once a tool that is not replay-safe has run
|
|
20
|
+
# in the step (the tools would run again), once the call has spent what
|
|
21
|
+
# its step reserved, or past MAX_CALLS; a follow-up that fails raises the
|
|
22
|
+
# output failure it was making up for, with the provider's error as its
|
|
23
|
+
# cause. Every response is billed; one that is set aside settles with
|
|
24
|
+
# the invocation's failed attempts, takes what its tools captured with
|
|
25
|
+
# it, and an :output_rejected trace names it.
|
|
25
26
|
module OutputIntegrity
|
|
26
27
|
include OutputRecovery
|
|
27
28
|
|
|
@@ -6,8 +6,10 @@ module Smith
|
|
|
6
6
|
class Agent
|
|
7
7
|
# The follow-up call an output failure may get (OutputIntegrity): a
|
|
8
8
|
# refusal fallback, a truncation retry, or a repair, each within the
|
|
9
|
-
# agent's settings and never once a tool
|
|
10
|
-
# call has spent what its step reserved, or past
|
|
9
|
+
# agent's settings and never once a tool that is not replay-safe has run
|
|
10
|
+
# in the step, once the call has spent what its step reserved, or past
|
|
11
|
+
# MAX_CALLS calls. A follow-up is a new chat from the step's messages,
|
|
12
|
+
# so the model may call replay-safe tools again.
|
|
11
13
|
module OutputRecovery
|
|
12
14
|
MAX_CALLS = 3 + OutputPolicy::MAX_REPAIRS
|
|
13
15
|
REPAIR_NOTE = "Your previous reply could not be used as the required output:\n%s\n" \
|
|
@@ -20,7 +22,7 @@ module Smith
|
|
|
20
22
|
private
|
|
21
23
|
|
|
22
24
|
def recovery(plan, attempt, result, failure)
|
|
23
|
-
return if
|
|
25
|
+
return if tools_uncertain? || plan.calls >= MAX_CALLS || budget_spent?(result)
|
|
24
26
|
|
|
25
27
|
case failure
|
|
26
28
|
when OutputRefused then refusal_recovery(plan, attempt, failure)
|
|
@@ -29,7 +31,7 @@ module Smith
|
|
|
29
31
|
end
|
|
30
32
|
end
|
|
31
33
|
|
|
32
|
-
def
|
|
34
|
+
def tools_uncertain? = Tool.current_tool_execution_tracker&.uncertain? || false
|
|
33
35
|
|
|
34
36
|
def refusal_recovery(plan, attempt, failure)
|
|
35
37
|
fallback = plan.agent_class.on_refusal[:model]
|
|
@@ -30,6 +30,9 @@ module Smith
|
|
|
30
30
|
# True for a response a host's response memo kept (ResponseMemo): the
|
|
31
31
|
# call sent nothing.
|
|
32
32
|
attribute :memo, Types::Bool.optional.default(nil)
|
|
33
|
+
# The tool results the attempt's tools captured (AttemptCaptures),
|
|
34
|
+
# which leave the step's with the attempt if it is set aside.
|
|
35
|
+
attribute :captures, Types::Array.default([].freeze)
|
|
33
36
|
|
|
34
37
|
def self.success(completion:, model_reference:, **facts)
|
|
35
38
|
new(completion:, model_reference:, error: nil, **facts)
|
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
require_relative "provider_failure_handling"
|
|
4
|
+
require_relative "attempt_captures"
|
|
4
5
|
require_relative "content_bytes"
|
|
5
6
|
require_relative "content_parts"
|
|
6
7
|
require_relative "invocation_preparation"
|
|
@@ -15,6 +16,7 @@ module Smith
|
|
|
15
16
|
class Agent
|
|
16
17
|
module ProviderCompletion
|
|
17
18
|
include ProviderFailureHandling
|
|
19
|
+
include AttemptCaptures
|
|
18
20
|
include InvocationPreparation
|
|
19
21
|
include ContentParts
|
|
20
22
|
include ProviderCallTiming
|
|
@@ -26,13 +28,14 @@ module Smith
|
|
|
26
28
|
|
|
27
29
|
# The input's documents and images are read once (ContentBytes), and
|
|
28
30
|
# its cached parts checked against every Anthropic model in the chain,
|
|
29
|
-
# before the first attempt.
|
|
30
|
-
|
|
31
|
+
# before the first attempt. A candidate that fails over to the next
|
|
32
|
+
# joins `set_aside` (AttemptCaptures).
|
|
33
|
+
def complete_with_provider(agent_class, prepared_input, shape:, set_aside: [])
|
|
31
34
|
chain = build_model_chain(agent_class).tap { check_chain_cache_breakpoints!(_1, prepared_input) }
|
|
32
|
-
ContentBytes.with(prepared_input) { attempt_candidates(agent_class, prepared_input, chain, shape) }
|
|
35
|
+
ContentBytes.with(prepared_input) { attempt_candidates(agent_class, prepared_input, chain, shape, set_aside) }
|
|
33
36
|
end
|
|
34
37
|
|
|
35
|
-
def attempt_candidates(agent_class, prepared_input, chain, shape)
|
|
38
|
+
def attempt_candidates(agent_class, prepared_input, chain, shape, set_aside)
|
|
36
39
|
candidates = ProviderCandidateSequence.new(chain)
|
|
37
40
|
interrupted = nil
|
|
38
41
|
candidates.each do |model_reference, index|
|
|
@@ -43,7 +46,7 @@ module Smith
|
|
|
43
46
|
|
|
44
47
|
interrupted ||= attempt if Interruptions.find(attempt.error)
|
|
45
48
|
candidates.suppress(account_failed_provider(attempt, model_reference))
|
|
46
|
-
fail_over!(attempt, agent_class, candidates, interrupted)
|
|
49
|
+
fail_over!(attempt, agent_class, candidates, interrupted, set_aside)
|
|
47
50
|
end
|
|
48
51
|
|
|
49
52
|
# Fail closed: an empty candidate chain (agent without a model)
|
|
@@ -53,16 +56,15 @@ module Smith
|
|
|
53
56
|
"no executable model candidate for #{agent_class}; declare a model or fallback_models"
|
|
54
57
|
end
|
|
55
58
|
|
|
56
|
-
# Returns when another candidate is left to try
|
|
57
|
-
#
|
|
58
|
-
#
|
|
59
|
-
#
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
)
|
|
59
|
+
# Returns when another candidate is left to try, the failed attempt's
|
|
60
|
+
# captures set aside. A candidate that met an interrupting error (an
|
|
61
|
+
# account out of credit) outlasts how a later one fails: the step was
|
|
62
|
+
# cut short, not failed, and is taken again from its first candidate
|
|
63
|
+
# once the host can do its part.
|
|
64
|
+
def fail_over!(attempt, agent_class, candidates, interrupted, set_aside)
|
|
65
|
+
handle_provider_failure!(attempt.error, attempt.model_reference, agent_class,
|
|
66
|
+
fallback_available: candidates.fallback_available?, attempt_id: attempt.attempt_id)
|
|
67
|
+
set_aside.concat(attempt.captures)
|
|
66
68
|
rescue StandardError => e
|
|
67
69
|
raise if interrupted.nil? || Interruptions.find(e)
|
|
68
70
|
|
|
@@ -126,7 +128,7 @@ module Smith
|
|
|
126
128
|
message_count = chat_message_count(chat)
|
|
127
129
|
observed_reference = observed_model_reference(chat, fallback: model_reference)
|
|
128
130
|
timer = ProviderCallTiming::Timer.start
|
|
129
|
-
response, stream, mode, call_facts =
|
|
131
|
+
response, stream, mode, call_facts = captured_completion(chat, observed_reference, collected = [], facts)
|
|
130
132
|
timer.stop
|
|
131
133
|
completion = Completion.from_messages(response:, messages: new_chat_messages(chat, message_count), stream:,
|
|
132
134
|
mode:)
|
|
@@ -23,7 +23,7 @@ module Smith
|
|
|
23
23
|
|
|
24
24
|
def handle_provider_failure!(error, model_reference, agent_class, fallback_available:, attempt_id: nil)
|
|
25
25
|
account_failed_attempt(error, model_reference, agent_class, attempt_id:)
|
|
26
|
-
if
|
|
26
|
+
if uncertain_tool_calls?
|
|
27
27
|
raise Smith::ToolOutcomeUncertain.new(
|
|
28
28
|
"provider failed after tool execution began; retry or fallback could replay an uncertain outcome"
|
|
29
29
|
), cause: error
|
|
@@ -44,8 +44,10 @@ module Smith
|
|
|
44
44
|
PROVIDER_ACCOUNT_ERRORS.any? { |error_class| error.is_a?(error_class) }
|
|
45
45
|
end
|
|
46
46
|
|
|
47
|
-
|
|
48
|
-
|
|
47
|
+
# A tool that is not replay-safe has run in the step: a fallback would
|
|
48
|
+
# run it again.
|
|
49
|
+
def uncertain_tool_calls?
|
|
50
|
+
Tool.current_tool_execution_tracker&.uncertain?
|
|
49
51
|
end
|
|
50
52
|
|
|
51
53
|
def fallback_eligible?(error)
|
|
@@ -72,7 +72,9 @@ module Smith
|
|
|
72
72
|
|
|
73
73
|
def paused_loop(error, chat, start)
|
|
74
74
|
continuation = Budget::Continuation.dump(@budget_continuation, chat.messages.drop(start || 0))
|
|
75
|
-
|
|
75
|
+
return unless continuation && storable_pause?(continuation)
|
|
76
|
+
|
|
77
|
+
BudgetUnavailable.new(error.message, continuation:, paused_by: self)
|
|
76
78
|
end
|
|
77
79
|
|
|
78
80
|
# Whether the run's state, with the paused loop kept, stays within
|
data/lib/smith/errors.rb
CHANGED
|
@@ -74,12 +74,15 @@ module Smith
|
|
|
74
74
|
# hold; nothing was sent. A BudgetExceeded, so it is not retried unless a
|
|
75
75
|
# step's `retry_on` names it (a durable retry waits for budget). One that
|
|
76
76
|
# paused a tool loop carries its `continuation` (Budget::Continuation),
|
|
77
|
-
# which the step's retry continues from
|
|
77
|
+
# which the step's retry continues from, and the run whose step paused
|
|
78
|
+
# (`paused_by`): only that run's retry continues it, never a parent's,
|
|
79
|
+
# whose retry would start a nested workflow over.
|
|
78
80
|
class BudgetUnavailable < BudgetExceeded
|
|
79
|
-
attr_reader :continuation
|
|
81
|
+
attr_reader :continuation, :paused_by
|
|
80
82
|
|
|
81
|
-
def initialize(message = "the budget ledger refused the request", continuation: nil)
|
|
83
|
+
def initialize(message = "the budget ledger refused the request", continuation: nil, paused_by: nil)
|
|
82
84
|
@continuation = continuation
|
|
85
|
+
@paused_by = paused_by
|
|
83
86
|
super(message)
|
|
84
87
|
end
|
|
85
88
|
end
|
data/lib/smith/response_memo.rb
CHANGED
|
@@ -31,7 +31,8 @@ module Smith
|
|
|
31
31
|
# whose tools had not run answers the call's first request without
|
|
32
32
|
# sending it, and its tools run again, under the same keys; kept tool
|
|
33
33
|
# results count as tools the step ran (charged to its tool-call
|
|
34
|
-
# allowances, and never run again by a retry or fallback
|
|
34
|
+
# allowances, and never run again by a retry or fallback unless each is a
|
|
35
|
+
# replay-safe tool's). A call whose
|
|
35
36
|
# messages cannot be kept (an attachment, or too many bytes) is not
|
|
36
37
|
# stored, and is logged. A record Smith does not keep (its shape, or tool
|
|
37
38
|
# results that do not answer its replies' calls) is ignored, and an error
|
|
@@ -31,7 +31,7 @@ module Smith
|
|
|
31
31
|
end
|
|
32
32
|
|
|
33
33
|
def mark_tool_execution_started!
|
|
34
|
-
self.class.current_tool_execution_tracker&.mark_started!
|
|
34
|
+
self.class.current_tool_execution_tracker&.mark_started!(replay_safe: self.class.replay_safe?)
|
|
35
35
|
self.class.__send__(:current_tool_dispatch_start_handler)&.call(self)
|
|
36
36
|
end
|
|
37
37
|
end
|
|
@@ -3,6 +3,9 @@
|
|
|
3
3
|
module Smith
|
|
4
4
|
class Tool < RubyLLM::Tool
|
|
5
5
|
class CapabilityBuilder
|
|
6
|
+
REPLAY = %i[safe unsafe].freeze
|
|
7
|
+
private_constant :REPLAY
|
|
8
|
+
|
|
6
9
|
def initialize
|
|
7
10
|
@capabilities = {}
|
|
8
11
|
end
|
|
@@ -12,7 +15,17 @@ module Smith
|
|
|
12
15
|
def network(value) = @capabilities[:network] = value
|
|
13
16
|
def approval(value) = @capabilities[:approval] = value
|
|
14
17
|
def data_volume(value) = @capabilities[:data_volume] = value
|
|
15
|
-
|
|
18
|
+
|
|
19
|
+
# :safe when running the tool again under the same keys does what
|
|
20
|
+
# running it once did (a lookup), so a step whose tools are all safe
|
|
21
|
+
# keeps its retries and fallbacks; :unsafe, the default, otherwise.
|
|
22
|
+
def replay(value)
|
|
23
|
+
raise ArgumentError, "replay is :safe or :unsafe" unless REPLAY.include?(value)
|
|
24
|
+
|
|
25
|
+
@capabilities[:replay] = value
|
|
26
|
+
end
|
|
27
|
+
|
|
28
|
+
def to_h = @capabilities
|
|
16
29
|
end
|
|
17
30
|
end
|
|
18
31
|
end
|
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Smith
|
|
4
|
+
class Tool < RubyLLM::Tool
|
|
5
|
+
# What a tool class declares about itself (`capabilities do ... end`,
|
|
6
|
+
# CapabilityBuilder), read for its privilege check, its traces'
|
|
7
|
+
# sensitivity and whether a step may run it again. A subclass inherits
|
|
8
|
+
# its parent's, key by key: its own block sets or replaces only the keys
|
|
9
|
+
# it names, so a subclass is never less guarded than its parent unless
|
|
10
|
+
# it says so. `replay` alone is not inherited: whether a tool is safe to
|
|
11
|
+
# run again is a fact about its own `perform`, so a subclass is
|
|
12
|
+
# replay-safe only when it declares `replay :safe` itself.
|
|
13
|
+
module CapabilityDeclaration
|
|
14
|
+
NOT_INHERITED = %i[replay].freeze
|
|
15
|
+
private_constant :NOT_INHERITED
|
|
16
|
+
|
|
17
|
+
def capabilities(&)
|
|
18
|
+
return inherited_capabilities unless block_given?
|
|
19
|
+
|
|
20
|
+
builder = CapabilityBuilder.new
|
|
21
|
+
builder.instance_eval(&)
|
|
22
|
+
@capabilities = builder.to_h.freeze
|
|
23
|
+
end
|
|
24
|
+
|
|
25
|
+
# Whether this class itself declares `capabilities { replay :safe }`.
|
|
26
|
+
def replay_safe? = capabilities&.dig(:replay) == :safe
|
|
27
|
+
|
|
28
|
+
private
|
|
29
|
+
|
|
30
|
+
def inherited_capabilities
|
|
31
|
+
inherited = parent_capabilities
|
|
32
|
+
own = @capabilities if instance_variable_defined?(:@capabilities)
|
|
33
|
+
return own unless inherited
|
|
34
|
+
|
|
35
|
+
own ? inherited.merge(own) : inherited
|
|
36
|
+
end
|
|
37
|
+
|
|
38
|
+
# The parent's capabilities this class takes (all but `replay`), or nil.
|
|
39
|
+
def parent_capabilities
|
|
40
|
+
inherited = parent_tool&.capabilities&.except(*NOT_INHERITED)
|
|
41
|
+
inherited if inherited&.any?
|
|
42
|
+
end
|
|
43
|
+
end
|
|
44
|
+
end
|
|
45
|
+
end
|
|
@@ -2,9 +2,12 @@
|
|
|
2
2
|
|
|
3
3
|
module Smith
|
|
4
4
|
class Tool < RubyLLM::Tool
|
|
5
|
+
# A tool class's capture, inherited (Declarations) with its strictness: a
|
|
6
|
+
# subclass's own `capture_result` replaces its parent's, block and
|
|
7
|
+
# strictness together.
|
|
5
8
|
module CaptureConfiguration
|
|
6
9
|
def capture_result(strict: false, &block)
|
|
7
|
-
return
|
|
10
|
+
return declared(:@capture_result) unless block
|
|
8
11
|
|
|
9
12
|
raise ArgumentError, "capture_result strict must be true or false" unless [true, false].include?(strict)
|
|
10
13
|
|
|
@@ -13,7 +16,7 @@ module Smith
|
|
|
13
16
|
end
|
|
14
17
|
|
|
15
18
|
def capture_result_strict?
|
|
16
|
-
|
|
19
|
+
declaring_tool(:@capture_result)&.instance_variable_get(:@capture_result_strict) == true
|
|
17
20
|
end
|
|
18
21
|
end
|
|
19
22
|
end
|
|
@@ -0,0 +1,73 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Smith
|
|
4
|
+
class Tool < RubyLLM::Tool
|
|
5
|
+
# A tool class's declarations, which its subclasses inherit: each reads
|
|
6
|
+
# the nearest class up the chain that declared it, so a subclass
|
|
7
|
+
# overrides one by declaring it again. Parameters add up by name (a
|
|
8
|
+
# subclass's `param` adds one or replaces its parent's of that name), up
|
|
9
|
+
# to a class that gave a schema with `params`, which replaced the ones
|
|
10
|
+
# above it; that schema gives way to a subclass's own `param`s.
|
|
11
|
+
# Capabilities (CapabilityDeclaration) merge key by key, and a capture
|
|
12
|
+
# (CaptureConfiguration) comes with its strictness. A tool none of whose
|
|
13
|
+
# classes declares `authorize` or `before_execute` runs without one, as
|
|
14
|
+
# before. `O(D)` for a chain `D` classes deep.
|
|
15
|
+
module Declarations
|
|
16
|
+
def description(text = nil)
|
|
17
|
+
return declared(:@description) unless text
|
|
18
|
+
|
|
19
|
+
@description = text
|
|
20
|
+
end
|
|
21
|
+
alias desc description
|
|
22
|
+
|
|
23
|
+
def param(name, **) = own_parameters[name] = RubyLLM::Parameter.new(name, **)
|
|
24
|
+
|
|
25
|
+
def parameters
|
|
26
|
+
parent = parent_tool
|
|
27
|
+
inherited = parent.parameters if parent && !parent.instance_variable_defined?(:@params_schema_definition)
|
|
28
|
+
inherited.nil? || inherited.empty? ? own_parameters : inherited.merge(own_parameters)
|
|
29
|
+
end
|
|
30
|
+
|
|
31
|
+
def params_schema_definition
|
|
32
|
+
return @params_schema_definition if instance_variable_defined?(:@params_schema_definition)
|
|
33
|
+
|
|
34
|
+
parent_tool&.params_schema_definition if own_parameters.empty?
|
|
35
|
+
end
|
|
36
|
+
|
|
37
|
+
def provider_params = declared(:@provider_params) || {}
|
|
38
|
+
|
|
39
|
+
def category(value = nil)
|
|
40
|
+
return declared(:@category) if value.nil?
|
|
41
|
+
|
|
42
|
+
@category = value
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
def authorize(&block)
|
|
46
|
+
return declared(:@authorize) unless block
|
|
47
|
+
|
|
48
|
+
@authorize = block
|
|
49
|
+
end
|
|
50
|
+
|
|
51
|
+
def before_execute(&block)
|
|
52
|
+
return declared(:@before_execute) unless block
|
|
53
|
+
|
|
54
|
+
@before_execute = block
|
|
55
|
+
end
|
|
56
|
+
|
|
57
|
+
private
|
|
58
|
+
|
|
59
|
+
def own_parameters = (@own_parameters ||= {})
|
|
60
|
+
|
|
61
|
+
def parent_tool = (superclass if superclass <= Tool)
|
|
62
|
+
|
|
63
|
+
# The nearest class from this one up that set `ivar`, or nil.
|
|
64
|
+
def declaring_tool(ivar)
|
|
65
|
+
tool = self
|
|
66
|
+
tool = tool.__send__(:parent_tool) while tool && !tool.instance_variable_defined?(ivar)
|
|
67
|
+
tool
|
|
68
|
+
end
|
|
69
|
+
|
|
70
|
+
def declared(ivar) = declaring_tool(ivar)&.instance_variable_get(ivar)
|
|
71
|
+
end
|
|
72
|
+
end
|
|
73
|
+
end
|
|
@@ -2,19 +2,30 @@
|
|
|
2
2
|
|
|
3
3
|
module Smith
|
|
4
4
|
class Tool < RubyLLM::Tool
|
|
5
|
+
# What the tools of one step attempt did: whether any began, and whether
|
|
6
|
+
# one that is not replay-safe did, which leaves the step's outcome
|
|
7
|
+
# uncertain (nothing may run its tools again).
|
|
5
8
|
class ExecutionTracker
|
|
6
9
|
def initialize
|
|
7
10
|
@started = false
|
|
11
|
+
@uncertain = false
|
|
8
12
|
@mutex = Mutex.new
|
|
9
13
|
end
|
|
10
14
|
|
|
11
|
-
def mark_started!
|
|
12
|
-
@mutex.synchronize
|
|
15
|
+
def mark_started!(replay_safe: false)
|
|
16
|
+
@mutex.synchronize do
|
|
17
|
+
@started = true
|
|
18
|
+
@uncertain = true unless replay_safe
|
|
19
|
+
end
|
|
13
20
|
end
|
|
14
21
|
|
|
15
22
|
def started?
|
|
16
23
|
@mutex.synchronize { @started }
|
|
17
24
|
end
|
|
25
|
+
|
|
26
|
+
def uncertain?
|
|
27
|
+
@mutex.synchronize { @uncertain }
|
|
28
|
+
end
|
|
18
29
|
end
|
|
19
30
|
end
|
|
20
31
|
end
|
data/lib/smith/tool.rb
CHANGED
|
@@ -2,7 +2,9 @@
|
|
|
2
2
|
|
|
3
3
|
require "ruby_llm"
|
|
4
4
|
|
|
5
|
+
require_relative "tool/declarations"
|
|
5
6
|
require_relative "tool/capability_builder"
|
|
7
|
+
require_relative "tool/capability_declaration"
|
|
6
8
|
require_relative "tool/policy"
|
|
7
9
|
require_relative "tool/call_budget"
|
|
8
10
|
require_relative "tool/call_batch"
|
|
@@ -61,22 +63,14 @@ module Smith
|
|
|
61
63
|
include ExecutionLifecycle
|
|
62
64
|
include BudgetEnforcement
|
|
63
65
|
include Capture
|
|
66
|
+
extend Declarations
|
|
67
|
+
extend CapabilityDeclaration
|
|
64
68
|
extend CaptureConfiguration
|
|
65
69
|
extend ScopedContext
|
|
66
70
|
|
|
67
71
|
private_constant :ExecutionAuthority
|
|
68
72
|
|
|
69
73
|
class << self
|
|
70
|
-
# Tool subclasses inherit the parent's compatible_with spec by
|
|
71
|
-
# reference (the spec is a frozen Hash; immutability makes shared
|
|
72
|
-
# references safe). Subclasses can override by calling
|
|
73
|
-
# `compatible_with` again — assigns a NEW frozen Hash to its own
|
|
74
|
-
# @compatible_with_spec, leaving the parent untouched.
|
|
75
|
-
def inherited(subclass)
|
|
76
|
-
super
|
|
77
|
-
subclass.instance_variable_set(:@compatible_with_spec, @compatible_with_spec)
|
|
78
|
-
end
|
|
79
|
-
|
|
80
74
|
# Declarative compatibility DSL. Examples:
|
|
81
75
|
# compatible_with :anthropic, :gemini
|
|
82
76
|
# compatible_with :anthropic, :gemini, openai: :responses
|
|
@@ -85,38 +79,14 @@ module Smith
|
|
|
85
79
|
# Tools that NEVER declare compatible_with are universally compatible.
|
|
86
80
|
# Consumed by Smith::Models::Normalizer.drop_incompatible_tools when
|
|
87
81
|
# the resolved model rejects the (tools + thinking) combo and no
|
|
88
|
-
# routing fallback (e.g., openai_api_mode :auto) is available.
|
|
82
|
+
# routing fallback (e.g., openai_api_mode :auto) is available. The
|
|
83
|
+
# spec is a frozen Hash, read like the other declarations: a subclass
|
|
84
|
+
# takes the nearest one up its chain (Declarations).
|
|
89
85
|
def compatible_with(*providers, except: nil, **provider_endpoints)
|
|
90
86
|
@compatible_with_spec = Compatibility.parse(providers, except: except, **provider_endpoints)
|
|
91
87
|
end
|
|
92
88
|
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
def category(value = nil)
|
|
96
|
-
return @category if value.nil?
|
|
97
|
-
|
|
98
|
-
@category = value
|
|
99
|
-
end
|
|
100
|
-
|
|
101
|
-
def capabilities(&)
|
|
102
|
-
return @capabilities unless block_given?
|
|
103
|
-
|
|
104
|
-
builder = CapabilityBuilder.new
|
|
105
|
-
builder.instance_eval(&)
|
|
106
|
-
@capabilities = builder.to_h
|
|
107
|
-
end
|
|
108
|
-
|
|
109
|
-
def authorize(&block)
|
|
110
|
-
return @authorize unless block_given?
|
|
111
|
-
|
|
112
|
-
@authorize = block
|
|
113
|
-
end
|
|
114
|
-
|
|
115
|
-
def before_execute(&block)
|
|
116
|
-
return @before_execute unless block_given?
|
|
117
|
-
|
|
118
|
-
@before_execute = block
|
|
119
|
-
end
|
|
89
|
+
def compatible_with_spec = declared(:@compatible_with_spec)
|
|
120
90
|
end
|
|
121
91
|
|
|
122
92
|
def execute(**kwargs)
|
data/lib/smith/version.rb
CHANGED
|
@@ -37,7 +37,14 @@ module Smith
|
|
|
37
37
|
def budget_pausable? = !@budget_pause_attempts.nil? && @step_attempt.to_i < @budget_pause_attempts
|
|
38
38
|
|
|
39
39
|
def keep_budget_continuation(error)
|
|
40
|
-
@budget_continuation = error.continuation if
|
|
40
|
+
@budget_continuation = error.continuation if own_pause?(error)
|
|
41
|
+
end
|
|
42
|
+
|
|
43
|
+
# A tool loop this run's own step paused, which its retry continues
|
|
44
|
+
# from its tools' results. One a nested workflow paused is not: the
|
|
45
|
+
# retry starts the child over, and its tools run again.
|
|
46
|
+
def own_pause?(error)
|
|
47
|
+
error.is_a?(BudgetUnavailable) && !error.continuation.nil? && error.paused_by.equal?(self)
|
|
41
48
|
end
|
|
42
49
|
|
|
43
50
|
# The paused loop's messages, after the attempt's prepared input.
|
|
@@ -22,20 +22,20 @@ module Smith
|
|
|
22
22
|
ensure
|
|
23
23
|
# Tool evidence travels up: a step running inside another step
|
|
24
24
|
# (a nested workflow's step) marks the enclosing step too, so the
|
|
25
|
-
# enclosing step is never retried over tools that already ran
|
|
26
|
-
|
|
25
|
+
# enclosing step is never retried over tools that already ran,
|
|
26
|
+
# unless they were all replay-safe.
|
|
27
|
+
previous_tracker&.mark_started!(replay_safe: !tracker.uncertain?) if tracker.started?
|
|
27
28
|
Tool.current_tool_execution_tracker = previous_tracker
|
|
28
29
|
end
|
|
29
30
|
end
|
|
30
31
|
end
|
|
31
32
|
|
|
32
|
-
# A failure after a tool ran leaves its outcome
|
|
33
|
-
# retry could not run the tool again: a terminal
|
|
34
|
-
# loop paused for budget, whose
|
|
35
|
-
# results.
|
|
33
|
+
# A failure after a tool that is not replay-safe ran leaves its outcome
|
|
34
|
+
# uncertain, unless its retry could not run the tool again: a terminal
|
|
35
|
+
# failure, or a tool loop this run's step paused for budget, whose
|
|
36
|
+
# retry continues from its tools' results (BudgetPause#own_pause?).
|
|
36
37
|
def uncertain_after_tools?(tracker, error)
|
|
37
|
-
tracker.
|
|
38
|
-
!(error.is_a?(BudgetUnavailable) && error.continuation)
|
|
38
|
+
tracker.uncertain? && !terminal_retry_error?(error) && !own_pause?(error)
|
|
39
39
|
end
|
|
40
40
|
|
|
41
41
|
def run_tracked_guarded_step(transition)
|
|
@@ -16,9 +16,9 @@ module Smith
|
|
|
16
16
|
end
|
|
17
17
|
|
|
18
18
|
# A child that raises passes its error to the parent unchanged, after
|
|
19
|
-
# its billable usage is rolled up. If a tool
|
|
20
|
-
# parent's step already carries that evidence
|
|
21
|
-
# into ToolOutcomeUncertain (GuardedStepExecution).
|
|
19
|
+
# its billable usage is rolled up. If a tool that is not replay-safe
|
|
20
|
+
# ran in the child, the parent's step already carries that evidence
|
|
21
|
+
# and turns the failure into ToolOutcomeUncertain (GuardedStepExecution).
|
|
22
22
|
def run_child_workflow(child)
|
|
23
23
|
child.run!
|
|
24
24
|
rescue StandardError
|
|
@@ -53,12 +53,24 @@ module Smith
|
|
|
53
53
|
messages = kept.fetch(:messages)
|
|
54
54
|
messages.each { chat.add_message(_1) }
|
|
55
55
|
ran = messages.filter_map { _1.tool_call_id if _1.role == :tool }
|
|
56
|
-
|
|
57
|
-
|
|
56
|
+
calls = ran_calls(messages, ran)
|
|
57
|
+
mark_kept_tools(chat, ran, calls) if ran.any?
|
|
58
|
+
charge_kept_calls(calls)
|
|
58
59
|
end
|
|
59
60
|
|
|
60
61
|
def ran_calls(messages, ran) = messages.flat_map { _1.tool_calls&.values.to_a }.select { ran.include?(_1.id) }
|
|
61
62
|
|
|
63
|
+
# As the tools themselves would mark the step: replay-safe only when
|
|
64
|
+
# each kept result answers a call to a tool of the chat's that
|
|
65
|
+
# declares it.
|
|
66
|
+
def mark_kept_tools(chat, ran, calls)
|
|
67
|
+
tools = calls.to_h { [_1.id, chat.tools[_1.name.to_sym]] }
|
|
68
|
+
safe = ran.all? { replay_safe_tool?(tools[_1]) }
|
|
69
|
+
Tool.current_tool_execution_tracker&.mark_started!(replay_safe: safe)
|
|
70
|
+
end
|
|
71
|
+
|
|
72
|
+
def replay_safe_tool?(tool) = tool.is_a?(Tool) && tool.class.replay_safe?
|
|
73
|
+
|
|
62
74
|
# Each kept call against the agent's tool-call allowance and the
|
|
63
75
|
# workflow's `tool_calls` budget, as a call that runs is, so a replay
|
|
64
76
|
# runs no more tools than the step may. A call past them (a budget
|
|
@@ -34,9 +34,7 @@ module Smith
|
|
|
34
34
|
schedule = retry_schedule(config)
|
|
35
35
|
begin
|
|
36
36
|
attempt += 1
|
|
37
|
-
|
|
38
|
-
@step_key_generation = key_generation
|
|
39
|
-
attempt_scope = attempt_idempotency_scope(key_generation)
|
|
37
|
+
attempt_scope = enter_attempt(attempt, key_generation)
|
|
40
38
|
run_attempt_under_keys(transition, attempt_scope)
|
|
41
39
|
rescue StandardError => e
|
|
42
40
|
Interruptions.pass!(e)
|
|
@@ -48,10 +46,33 @@ module Smith
|
|
|
48
46
|
return wait_to_retry(transition, delay, attempt + 1, key_generation, e) if config[:durable]
|
|
49
47
|
|
|
50
48
|
sleep(delay) if delay.positive?
|
|
49
|
+
drop_attempt_tool_results(e)
|
|
51
50
|
retry
|
|
52
51
|
end
|
|
53
52
|
end
|
|
54
53
|
|
|
54
|
+
# The attempt under way, under `key_generation`; returns its
|
|
55
|
+
# idempotency scope. `@attempt_tool_results_from` (transient, like
|
|
56
|
+
# `@step_attempt`) is where its tool results start.
|
|
57
|
+
def enter_attempt(attempt, key_generation)
|
|
58
|
+
@step_attempt = attempt
|
|
59
|
+
@step_key_generation = key_generation
|
|
60
|
+
@attempt_tool_results_from = @tool_results_mutex.synchronize { @tool_results.length }
|
|
61
|
+
attempt_idempotency_scope(key_generation)
|
|
62
|
+
end
|
|
63
|
+
|
|
64
|
+
# Once its retry is certain (in process, or waiting), a failed attempt's
|
|
65
|
+
# tool results go: its tools were replay-safe, or it would not be
|
|
66
|
+
# retried, and the next attempt runs them again. A tool loop this
|
|
67
|
+
# run's step paused for budget keeps them, as its retry goes on from
|
|
68
|
+
# them.
|
|
69
|
+
def drop_attempt_tool_results(error)
|
|
70
|
+
from = @attempt_tool_results_from
|
|
71
|
+
return if from.nil? || own_pause?(error)
|
|
72
|
+
|
|
73
|
+
@tool_results_mutex.synchronize { @tool_results.slice!(from..) }
|
|
74
|
+
end
|
|
75
|
+
|
|
55
76
|
def attempt_idempotency_scope(key_generation)
|
|
56
77
|
step_scope = IdempotencyKey.__send__(:current_scope)
|
|
57
78
|
return step_scope if step_scope.nil? || key_generation.zero?
|
|
@@ -24,6 +24,7 @@ module Smith
|
|
|
24
24
|
# it waits to make. Like any wait it is not a step. The pending retry
|
|
25
25
|
# is recorded before the wait, so an interrupt between the two leaves
|
|
26
26
|
# a step that ended without a wait, whose pending retry is cleared.
|
|
27
|
+
# The run waits without the failed attempt's tool captures.
|
|
27
28
|
def wait_to_retry(transition, seconds, attempt, key_generation, error)
|
|
28
29
|
if @split_step_phase == :executing
|
|
29
30
|
raise WorkflowError, "a durable retry cannot wait inside split-step execution"
|
|
@@ -38,6 +39,7 @@ module Smith
|
|
|
38
39
|
@awaiting = WaitRecord.build_retry(transition, wait_token_for(transition, ["retry", attempt]), seconds,
|
|
39
40
|
Time.now.utc)
|
|
40
41
|
record_retry_trace(transition, :retrying, attempt:, error: error_name, deadline_at: @awaiting["deadline_at"])
|
|
42
|
+
drop_attempt_tool_results(error)
|
|
41
43
|
WAIT_PENDING
|
|
42
44
|
end
|
|
43
45
|
|
data/lib/smith/workflow.rb
CHANGED
|
@@ -465,6 +465,17 @@ module Smith
|
|
|
465
465
|
->(entry) { @tool_results_mutex.synchronize { @tool_results << entry } }
|
|
466
466
|
end
|
|
467
467
|
|
|
468
|
+
# Takes out what an attempt the step set aside had captured
|
|
469
|
+
# (Agent::AttemptCaptures), by identity, so an equal entry of another
|
|
470
|
+
# attempt or branch stays. `O(N + K)`.
|
|
471
|
+
def drop_tool_results(entries)
|
|
472
|
+
return if entries.empty?
|
|
473
|
+
|
|
474
|
+
dropped = {}.compare_by_identity
|
|
475
|
+
entries.each { dropped[_1] = true }
|
|
476
|
+
@tool_results_mutex.synchronize { @tool_results.reject! { dropped.key?(_1) } }
|
|
477
|
+
end
|
|
478
|
+
|
|
468
479
|
def initialize_tool_result_state
|
|
469
480
|
@tool_results = []
|
|
470
481
|
@tool_results_mutex = Mutex.new
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: smith-agents
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.13.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Samuel Ralak
|
|
@@ -161,6 +161,7 @@ files:
|
|
|
161
161
|
- lib/generators/smith/install/templates/workflow_state.rb.tt
|
|
162
162
|
- lib/smith.rb
|
|
163
163
|
- lib/smith/agent.rb
|
|
164
|
+
- lib/smith/agent/attempt_captures.rb
|
|
164
165
|
- lib/smith/agent/attempt_chat.rb
|
|
165
166
|
- lib/smith/agent/chat_construction.rb
|
|
166
167
|
- lib/smith/agent/completion.rb
|
|
@@ -348,11 +349,13 @@ files:
|
|
|
348
349
|
- lib/smith/tool/call_budget.rb
|
|
349
350
|
- lib/smith/tool/call_reservation.rb
|
|
350
351
|
- lib/smith/tool/capability_builder.rb
|
|
352
|
+
- lib/smith/tool/capability_declaration.rb
|
|
351
353
|
- lib/smith/tool/capture.rb
|
|
352
354
|
- lib/smith/tool/capture_configuration.rb
|
|
353
355
|
- lib/smith/tool/chat_execution_callbacks.rb
|
|
354
356
|
- lib/smith/tool/chat_execution_context.rb
|
|
355
357
|
- lib/smith/tool/compatibility.rb
|
|
358
|
+
- lib/smith/tool/declarations.rb
|
|
356
359
|
- lib/smith/tool/execution_authority.rb
|
|
357
360
|
- lib/smith/tool/execution_authorization.rb
|
|
358
361
|
- lib/smith/tool/execution_batch.rb
|