smith-agents 0.12.7 → 0.13.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 8158004e28c8bec8f58a83727cbfe9d91a7fff89dd24cc3435aefd06f87d826f
-  data.tar.gz: 6b7088fd850554492cbe98edb93c76b2d1958024206eec9a497fd423f7d2b794
+  metadata.gz: 7c3bc332fe08b2ff077b3039603ddc8356ac65d5034f47abcf4253c3e167f8c6
+  data.tar.gz: 9eaf0e8548b473da25838206b63e2a5639d7ee4391f7cb6e30980a5b5aa46e67
 SHA512:
-  metadata.gz: 442ebb9375a3e912f6d98c08da04e67d3aeec1afa8aa1b0f1a45f936e4e33ef9b9dd477d847b113510545a34327451566bd264b3ca2970a26762d61291c5d5f4
-  data.tar.gz: dc3fb7433a59f12a2e947e8eeec20670f4430e5c86b654ad5e1795c4f6e75da753450905de141b409e19162c2f034f070685bf40ef83aa3a93b370abd21fcab6
+  metadata.gz: 1ff7f020acb2efd11f7143a074d3c65093137be5bc2646e10a668e1e5242b37255db15909f0c99aa3b6073ad73e31a2557f61fa3d2a35d2de716ba122d7deb4e
+  data.tar.gz: fab4fd19cd33c21b6800a8d75740bc62533aa4100a2d9ae5e26bad31ad5a1fa57b4869cce9efc9a351b41366ee5ee5a8cb24c0d21dc8369effdbe5b8fd8e703c

data/CHANGELOG.md

@@ -4,6 +4,132 @@ All notable changes to Smith are documented in this file.
 
 Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). Versioning follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html). Smith is pre-1.0 and under active development; expect occasional contract tightening between minor versions until 1.0.
 
+## [0.13.1] - 2026-11-08
+
+### Changed
+
+- Split-step persistence refuses a step declared `from: nil` when it is
+  prepared: `prepare_persisted_step!` raises `Smith::WorkflowError` naming
+  the step and saying to run it with `advance_persisted!`, before anything
+  is written, as a prepared step is checked against the state its step is
+  declared from. It is not supported yet. Under 0.13.0 such a step was
+  prepared, which stores its run as a step in progress, and then failed:
+  its work ran first, a parallel or fan-out step's branches included, and
+  then "prepared execution did not return the claimed transition", so a
+  retry ran the work again. Only a parallel or fan-out step whose branches
+  the host ran through `prepare_composite_step!` failed in that call,
+  before any branch ran, its run left in progress all the same. A run
+  stored that way still raises `Smith::StepInProgressOnRestore` in strict
+  mode.
+
+### Fixed
+
+- A failure in a step declared `from: nil` (one that runs from whichever
+  state routes to it) restores. Smith records the failure with the step's
+  declared origin, `from: nil`, as its step result, its `StepFailed` event
+  and its trace have it, but restore refused a record without a `from`
+  state (`Smith::PersistedFailureInvalid`), so a stored run could not go on
+  to the step's `on_failure`, and a run that ended failed there could not
+  be read back. Restore now takes a nil `from`, in lax and strict modes and
+  through every store adapter, and a run stored by 0.13.0 restores as it
+  is. Any other `from`, and its `transition` and `to`, must still be text
+  of 1 to 256 bytes, and a record without the field is still refused.
+- A run whose last failed step was renamed (`renamed_transition`, or
+  `renamed_state` for the state it failed from or was headed to) restores.
+  The names a rename leads to are US-ASCII text, as a Symbol's name is,
+  and restore refused any failure record name not marked UTF-8
+  (`Smith::PersistedFailureInvalid`, "transition is invalid"), so such a
+  run could not go on to the step's `on_failure`, or be read back once it
+  ended failed. A failure record restored from `to_state` itself, without a
+  JSON round trip, was refused the same way. US-ASCII text is UTF-8 as it
+  stands and is now taken as such; text in any other encoding, or with
+  bytes outside its encoding, is still refused. Runs stored by 0.12.7 and
+  later restore as they are.
+
+### Upgrading
+
+- `Smith::EXECUTION_SEMANTICS_VERSION` stays `"7"`: no persisted state,
+  wait, pending retry or durable plan changes shape, and a run stored by
+  0.13.0 restores as it is.
+
+## [0.13.0] - 2026-11-01
+
+### Added
+
+- A tool can declare `capabilities { replay :safe }`: running it again under
+  the same keys does what running it once did (a lookup, a search, a read,
+  one that calls a paid API included). Once a tool had run in a step, Smith
+  made no output repair, refusal fallback or truncation retry, took no
+  fallback model after a provider failure, and turned the step's failure
+  into `Smith::ToolOutcomeUncertain`, which no retry takes. Now only a tool
+  that is not replay-safe does that (any tool without the declaration, a
+  plain RubyLLM tool included). When every tool that ran in a step is
+  replay-safe, the step keeps its follow-ups, its fallback models and its
+  `retry_on`, durable retries included; each runs the tools again, under the
+  same idempotency keys and within the same `tool_calls` allowances. A
+  provider attempt set aside for a follow-up or a fallback model takes its
+  tools' captures out of the step's `tool_results` once the call has its
+  output (by identity, so a concurrent fan-out branch's stay; a call that
+  fails after all keeps every capture), and so does a retried step attempt
+  once its retry is certain, unless the retry continues a tool loop the
+  step paused for budget. Tool evidence reaches a parent step from a nested
+  workflow as it was, and a call the response memo replays counts its kept
+  tool results by the tools its chat declares. `replay :unsafe` states the
+  default; any other value raises `ArgumentError`; `ToolClass.replay_safe?`
+  reads it. Each class declares it for itself: a subclass of a replay-safe
+  tool is unsafe until it says `replay :safe` too.
+
+### Changed
+
+- A `Smith::Tool` subclass inherits its parent's declarations, each read
+  from the nearest class that made it, so a subclass overrides one by
+  declaring it again. Before, a subclass inherited none of them: it ran
+  without its parent's `authorize` (failing open), `privilege`, trace
+  `sensitivity` and `capture_result`, and reached the model without its
+  parent's description and declared parameters (a host subclass of the
+  shipped `Tools::WebSearch` or `Tools::UrlFetcher` stub sent no
+  description and parameters guessed from its `perform`, all strings).
+  `description`, `with_params`, `category`, `authorize`, `before_execute`,
+  `compatible_with` and `capture_result` (its block and strictness
+  together) are the nearest declared; `param`s add up by name, up to a
+  class that gave a `params` schema; a `params` schema is the nearest
+  unless the subclass declares `param`s of its own; `capabilities` merge
+  key by key, except `replay`, which is never inherited: a subclass of a
+  replay-safe tool is replay-safe only when it declares `replay :safe`
+  itself, as safety to run again depends on its own `perform`. A tool none
+  of whose classes declares `authorize` runs as an undeclared tool always
+  has, and a declaration made on a parent after its subclass was created
+  reaches the subclass (`compatible_with` was copied when the subclass was
+  made). Capture was documented as opt-in per concrete class; it is now
+  inherited too.
+
+### Fixed
+
+- A budget refusal that carries a paused tool loop
+  (`Smith::BudgetUnavailable#continuation`) spares a step from
+  `Smith::ToolOutcomeUncertain`, and is continued by its retry, only in the
+  run whose step paused (`paused_by`). One raised from a nested workflow, or
+  made by host code, reached the parent step as if the parent could
+  continue it, so the parent's retry started the child over and ran its
+  tools again, and could take the child's paused messages as its own.
+
+### Upgrading
+
+- A tool subclass that declared less than its parent now runs with its
+  parent's declarations: its parent's `authorize`, `privilege` and
+  `before_execute` apply (a call they deny raises `Smith::ToolPolicyDenied`),
+  its strict capture fails closed without a collector, its traces follow
+  its parent's `sensitivity`, and its requests carry its parent's
+  description and parameters. A subclass meant to differ declares its own
+  (`authorize { ... }`, `capabilities { privilege :none }`,
+  `capture_result(strict: false) { ... }`). As its
+  requests change, the response memo misses once for a call with such a
+  tool (its digest covers tools' descriptions and parameters), so a step
+  cut by the upgrade's deploy pays for that call again and runs its tools
+  again.
+- `Smith::EXECUTION_SEMANTICS_VERSION` stays `"7"`: no persisted state,
+  wait, pending retry or durable plan changes shape.
+
 ## [0.12.7] - 2026-10-13
 
 ### Added

data/README.md

@@ -30,7 +30,7 @@ What you get:
 
 ```ruby
 # Gemfile
-gem "smith-agents", "0.12.7", require: "smith"
+gem "smith-agents", "0.13.1", require: "smith"
 ```
 
 In a Rails app:
@@ -240,7 +240,11 @@ everything else. Messages can carry documents and images.
 ## Tools and guardrails
 
 Agents can use `Smith::Tool` subclasses, with a budget on how many calls a
-step may make. Guardrails check inputs, outputs and tool calls. Smith ships
+step may make. Once a tool has run, a step is not retried and takes no
+fallback, as its tools would run again; a tool that is safe to run again (a
+lookup) declares `capabilities { replay :safe }`, and a step whose tools were
+all safe keeps its retries, fallbacks and output repair. Guardrails check
+inputs, outputs and tool calls. Smith ships
 `Tools::Think`; `Tools::WebSearch` and `Tools::UrlFetcher` are stubs that
 declare the interface, which you subclass with your own search or fetch.
 [`docs/TOOLS_AND_GUARDRAILS.md`](docs/TOOLS_AND_GUARDRAILS.md)

data/docs/AGENTS.md

@@ -50,11 +50,15 @@ end
   problems as "path: problem" lines. It then raises `Smith::OutputInvalid`,
   whose `problems` lists at most 20 of them. A blank output still raises
   `Smith::BlankAgentOutputError`.
-- No follow-up call is made once a tool has run in the step (the tools would
-  run again; the step fails with `Smith::ToolOutcomeUncertain` around the output
-  error) or once the call has spent what its step reserved from a budget. A
-  follow-up that fails raises the output error it was making up for, with the
-  provider's error as its cause.
+- No follow-up call is made once a tool that is not replay-safe has run in
+  the step (the tools would run again; the step fails with
+  `Smith::ToolOutcomeUncertain` around the output error) or once the call has
+  spent what its step reserved from a budget. After replay-safe tools only
+  (`capabilities { replay :safe }`, docs/TOOLS_AND_GUARDRAILS.md), the
+  follow-up is made and its model may call them again; once there is an
+  output, the set-aside reply's tool captures leave `tool_results`. A
+  follow-up that fails raises the output error it was making up for, with
+  the provider's error as its cause.
 - Anthropic's structured outputs reject numeric, string-length and most
   array-size constraints: Smith sends Anthropic the schema without them, with
   `additionalProperties: false` on every object, and checks them on the parsed

data/docs/ERRORS_AND_RETRIES.md

@@ -15,9 +15,24 @@ Smith::Errors.retry_forbidden?(error)
 ```
 
 A nested workflow's step fails with the child's own error when the child
-raises it, so these answers carry through nesting. When a tool ran inside the
-child, the parent step's failure becomes `ToolOutcomeUncertain`, which is never
-retried.
+raises it, so these answers carry through nesting. When a tool that is not
+replay-safe ran inside the child, the parent step's failure becomes
+`ToolOutcomeUncertain`, which is never retried.
+
+## After tools ran
+
+A step's failure after one of its tools ran is `ToolOutcomeUncertain`, which
+no retry takes, and the step takes no fallback model and makes no follow-up
+call for its output: running the tools again could repeat what they did. A
+tool that declares `capabilities { replay :safe }` (a lookup, a search, a
+read) says running it again does no harm. When every tool that ran in the
+step is replay-safe, the step keeps its output repair, its refusal fallback
+and truncation retry, its fallback models after a provider failure, and its
+`retry_on`, durable retries included; each runs the tools again, under the
+same idempotency keys and within the same `tool_calls` allowances, and the
+captures of an attempt it sets aside leave `tool_results` once the step has
+its output or its retry is certain. See
+[TOOLS_AND_GUARDRAILS.md](TOOLS_AND_GUARDRAILS.md), Replay-Safe Tools.
 
 Workflow transitions can also declare a bounded local retry policy:
 
@@ -185,7 +200,8 @@ had just come, once, and its `:provider_call` trace says `memo: true`; a reply
 that was cut off or refused fails the step as it did. One that had not ended
 goes on from where it was kept. Kept after its tools' results, it continues
 from them: kept tool results count as tools the step ran, so no retry or
-fallback runs them again, and each is charged to the agent's `tool_calls`
+fallback runs them again (unless each is the result of a tool the chat
+declares replay-safe), and each is charged to the agent's `tool_calls`
 allowance and the workflow's `tool_calls` budget as it was. Kept while its
 tools ran (the step was cut inside a tool), the reply that called them is not
 asked for again: its tools run again, under the same idempotency keys, and the

data/docs/PATTERNS.md

@@ -332,9 +332,11 @@ What you get:
 - a child that routes its failure through its own `on_failure` has handled it:
   the parent step fails with a non-retryable `Smith::WorkflowError`
   (`nested workflow failed: <child message>`, bounded to 1 KiB)
-- tool evidence travels up: when a tool ran in any of the child's steps, a
-  failure of the parent step becomes `Smith::ToolOutcomeUncertain` and is never
-  retried, exactly as when the tool runs in the parent step itself
+- tool evidence travels up: when a tool that is not replay-safe ran in any of
+  the child's steps, a failure of the parent step becomes
+  `Smith::ToolOutcomeUncertain` and is never retried, exactly as when the tool
+  runs in the parent step itself (a child whose tools were all
+  `replay :safe` leaves the parent's retry)
 - parent step count stays parent-scoped
 - parent and child share the outer budget ledger
 - nested best-known token/cost totals roll up into the parent result

data/docs/PERSISTENCE.md

@@ -60,6 +60,11 @@ What Smith does for you:
   not the caller's Hash
 - persists accepted session history
 - preserves chosen next transitions across persistence
+- keeps a step's failure across persistence: a restored run takes the step's
+  `on_failure`, and `RunResult#last_error` and `#failure_detail` read as they
+  did in the run that failed. A step declared `from: nil`, which runs from
+  whichever state routes to it, keeps `from: nil` there, as its step result
+  and its `StepFailed` event do
 - supports JSON host round-trips through `to_state` and `.from_state`
 - refuses to restore a payload whose state, or whose pending transition, the
   workflow class does not know (a deploy renamed or removed it, or the payload
@@ -307,6 +312,11 @@ a versioned persistence adapter whose `store_versioned` method accepts `ttl:`,
 and non-expiring workflow persistence. Smith explicitly pins `ttl: nil` for
 every boundary write, so a later global configuration change cannot turn an
 accepted checkpoint into expiring state.
+It does not run a step declared `from: nil` yet: `prepare_persisted_step!`
+refuses one with a `Smith::WorkflowError` naming the step, before anything is
+written, since a prepared step is checked against the state its step is
+declared from. The run itself is unchanged: run that step with
+`advance_persisted!`.
 `prepare_persisted_step!` writes the normal `step_in_progress` marker and returns
 the pending transition name without consuming it. Duplicate preparation on the
 same workflow object is rejected. The boundary is pinned to the exact
@@ -935,7 +945,10 @@ end
   runs under the step-in-progress mark, so the restore of a run caught in it
   raises `Smith::StepInProgressOnRestore`.
 - No retry wait is entered once the run's wall-clock deadline has passed, after
-  a tool ran (`Smith::ToolOutcomeUncertain` is never retried), or for a backoff
+  a tool that is not replay-safe ran (`Smith::ToolOutcomeUncertain` is never
+  retried; after replay-safe tools only, the wait is entered, the run stored
+  without the failed attempt's tool captures, and the next attempt runs them
+  again), or for a backoff
   that does not end within the store's persistence TTL (the step fails with a
   `Smith::WorkflowError` whose cause is the attempt's error); runtime readiness
   reports a policy whose longest backoff, jitter included, reaches the TTL

data/docs/TOOLS_AND_GUARDRAILS.md

@@ -41,6 +41,85 @@ configuration, programming, authorization-integrity, or durability failures.
 Smith applies policy and budgeting around that host boundary; it does not
 interpret a host's input schema or failure envelope.
 
+### Subclasses
+
+A subclass inherits every declaration of its parent's but `replay`, each
+read from the nearest class that made it, so a subclass overrides one by
+declaring it again:
+
+- `description`, `with_params`, `category`, `authorize`, `before_execute`,
+  `compatible_with` and `capture_result` (its block and strictness
+  together): the nearest declared wins
+- `param`: parameters add up by name, so a subclass adds parameters or
+  replaces one of its parent's; a schema given with `params` is the
+  nearest one, unless the subclass declares `param`s of its own
+- `capabilities`: merged key by key, so a subclass's block sets or replaces
+  only the keys it names and keeps its parent's `privilege` and
+  `sensitivity`; `replay` is never inherited (Replay-Safe Tools)
+
+An inherited `authorize` guards the subclass as it guards its parent; a
+subclass's own replaces it (call `superclass.authorize` inside it to keep
+both checks), and a tool none of whose classes declares one runs as an
+undeclared tool does. A subclass made before its parent declares something
+still reads it.
+
+### Replay-Safe Tools
+
+A tool that does the same when it runs again under the same keys (a lookup
+over the host's own data, a search, a read, even one that calls a paid API)
+can say so:
+
+```ruby
+class FindPlaces < Smith::Tool
+  description "Find places that match a description."
+  param :query, type: :string, required: true
+
+  capabilities do
+    replay :safe
+  end
+
+  def perform(query:) = PlaceIndex.search(query, limit: 5)
+end
+```
+
+Smith follows the tools each step attempt runs. Once a tool that is not
+replay-safe has run (any tool without the declaration, a plain RubyLLM tool
+included), the step makes no follow-up call for its output (the step fails
+with `Smith::ToolOutcomeUncertain` around the output error), takes no
+fallback model after a provider failure (`ToolOutcomeUncertain`), and a
+failure of the step becomes `ToolOutcomeUncertain`, which no retry takes:
+running the step's tools again could repeat what they did. When every tool
+that ran is replay-safe, the step keeps all three:
+
+- an output repair, a refusal fallback and a truncation retry are each a new
+  call from the step's messages (a repair adds the rejected reply and its
+  problems), so the model may call the tools again
+- a provider failure takes the agent's fallback models, each with a tool
+  loop of its own
+- a failure of the step is its own error, which `retry_on` retries (with
+  `durable: true`, a wait of the run), and the next attempt runs its tools
+  again
+
+An attempt set aside for any of these takes its tools' captures
+(`capture_result`, strict or not) out of the step's `tool_results` once the
+call has its output, by identity, so a concurrent branch's are left; a call
+that fails after all keeps every capture. A retried attempt's go too, once
+the retry is certain (a durable retry's wait entered), unless the retry
+continues a tool loop the step paused for budget, which keeps its tools'
+results (one a nested workflow paused counts as any failure of its tools).
+Tools that run again send the same idempotency keys, and count
+against the agent's `tool_calls` allowance and the workflow's `tool_calls`
+budget as any call does, so a step's tool calls stay bounded. Tool evidence
+from a nested workflow reaches its parent step as it was: a child whose tools
+were all replay-safe leaves the parent's retry. A call the response memo
+replays counts its kept tool results the same way, by the tools its chat
+declares. The declaration is never inherited, as whether a tool is safe to
+run again depends on its own `perform`: a subclass of a replay-safe tool is
+replay-safe only when it declares `replay :safe` itself (its other
+capabilities, description, parameters and capture it inherits). `replay
+:unsafe` states the default, any other value raises `ArgumentError`, and
+`ToolClass.replay_safe?` reads the class's own declaration.
+
 ### Bounded Completion
 
 The default exhaustion policy remains fail-fast:
@@ -186,9 +265,9 @@ entry by raising an exception. A collector return value of `nil` or `false` is
 accepted; delivery is rejected only when the collector raises. Best-effort
 capture logs and continues as before.
 
-Capture policy is declared per concrete tool class and is not inherited. This
-preserves the historical opt-in contract and prevents a subclass from beginning
-to capture or fail strictly without its own declaration.
+A subclass inherits its parent's capture, strictness included, and replaces it
+with a `capture_result` of its own (Subclasses), so a subclass of a strictly
+capturing tool fails closed as its parent does.
 
 Strict capture validates that a collector exists before `perform`, but the
 capture block and collector run after `perform` returns. It is therefore
@@ -459,7 +538,8 @@ Smith ships `Tools::WebSearch`, `Tools::UrlFetcher`, and `Tools::Think`. Tools d
 `Tools::WebSearch` and `Tools::UrlFetcher` are stubs: they declare each
 tool's parameters and description, and their `perform` raises
 `NotImplementedError`. A host subclasses them with its own search or fetch
-behind `perform`. `Tools::Think` works as shipped.
+behind `perform`, and its subclass keeps their parameters, description and
+compatibility (Subclasses). `Tools::Think` works as shipped.
 
 ```ruby
 class WebSearch < Smith::Tools::WebSearch
@@ -499,8 +579,8 @@ Graceful exhaustion is an in-process protocol guarantee, not durable recovery
 for the model/tool loop. If the process exits after a tool begins and before the
 agent completes, the host must treat the outcome as uncertain unless it owns a
 durable per-invocation receipt and reconciliation contract. Smith also refuses
-to start a fresh fallback-model chat after tool execution has begun because that
-would discard the captured evidence and conversation protocol.
+to start a fresh fallback-model chat after a tool that is not replay-safe has
+begun, because the new chat would run its tools again (Replay-Safe Tools).
 
 Within a host invocation scope, `Smith::Tool.current_invocation` exposes the
 normalized tool-call ID and deterministic execution/batch ordinals. The ID is

data/lib/smith/agent/attempt_captures.rb

@@ -0,0 +1,62 @@
+# frozen_string_literal: true
+
+module Smith
+  class Agent
+    # The tool results one provider attempt captures, kept apart so an
+    # attempt the call sets aside (for a fallback model or a follow-up,
+    # possible only when its tools were replay-safe) takes them with it once
+    # the call has its output: the attempt that replaced it ran its tools
+    # again and captured anew. Each entry still reaches the step's collector
+    # as it comes.
+    module AttemptCaptures
+      # A collector that passes each entry on and remembers it.
+      class Scope
+        def initialize(collector)
+          @collector = collector
+          @entries = []
+          @lock = Mutex.new
+        end
+
+        def call(entry)
+          accepted = @collector.call(entry)
+          @lock.synchronize { @entries << entry }
+          accepted
+        end
+
+        def entries = @lock.synchronize { @entries.dup.freeze }
+      end
+      private_constant :Scope
+
+      private
+
+      # The chat's completion (ProviderStreaming#complete_chat), with what
+      # its tools capture recorded as `facts[:captures]`, the attempt's,
+      # however it ends. A step without a callable collector is left as it
+      # is, so strict capture still fails closed there.
+      def captured_completion(chat, model_reference, collected, facts)
+        collector = Tool.current_tool_result_collector
+        return complete_chat(chat, model_reference, collected, facts) unless collector.respond_to?(:call)
+
+        scope = Scope.new(collector)
+        Thread.handle_interrupt(Object => :never) do
+          Tool.current_tool_result_collector = scope
+          begin
+            Thread.handle_interrupt(Object => :immediate) { complete_chat(chat, model_reference, collected, facts) }
+          ensure
+            Tool.current_tool_result_collector = collector
+            facts[:captures] = scope.entries
+          end
+        end
+      end
+
+      # The accepted output; only now do the attempts set aside on the way
+      # take what their tools captured with them, so an invocation that
+      # fails after all keeps every capture, as before.
+      def accepted_output(agent_class, result, set_aside)
+        output = finalize_output(agent_class, result)
+        drop_tool_results(set_aside)
+        output
+      end
+    end
+  end
+end

data/lib/smith/agent/lifecycle.rb

@@ -21,18 +21,22 @@ module Smith
       end
 
       # One invocation: the provider call, then whatever OutputIntegrity
-      # asks for until the response can be the output.
+      # asks for until the response can be the output. The attempts set aside
+      # on the way (`set_aside`) take what their tools captured with them
+      # once there is an output (AttemptCaptures).
       def invoke_agent(agent_class, prepared_input, output_schema: effective_output_schema(agent_class))
         check_deadline!
         plan = output_plan(agent_class, prepared_input, output_schema)
-        attempt = complete_with_provider(agent_class, prepared_input, shape: plan.shape)
+        set_aside = []
+        attempt = complete_with_provider(agent_class, prepared_input, shape: plan.shape, set_aside:)
         loop do
           result = account_attempt(agent_class, attempt.completion, attempt.model_reference,
                                    attempt_id: attempt.attempt_id)
           follow_up = output_follow_up(plan, attempt, result)
-          return finalize_output(agent_class, result) unless follow_up
+          return accepted_output(agent_class, result, set_aside) unless follow_up
 
           settle_rejected(result)
+          set_aside.concat(attempt.captures)
           attempt = follow_up.call
         end
       end

data/lib/smith/agent/output_integrity.rb

@@ -16,12 +16,13 @@ module Smith
     # output contract is sent back for repair up to `output_repairs`, then
     # raises OutputInvalid.
     #
-    # No follow-up call is made once a tool has run in the step (the tools
-    # would run again), once the call has spent what its step reserved, or
-    # past MAX_CALLS; a follow-up that fails raises the output failure it
-    # was making up for, with the provider's error as its cause. Every
-    # response is billed; one that is set aside settles with the
-    # invocation's failed attempts, and an :output_rejected trace names it.
+    # No follow-up call is made once a tool that is not replay-safe has run
+    # in the step (the tools would run again), once the call has spent what
+    # its step reserved, or past MAX_CALLS; a follow-up that fails raises the
+    # output failure it was making up for, with the provider's error as its
+    # cause. Every response is billed; one that is set aside settles with
+    # the invocation's failed attempts, takes what its tools captured with
+    # it, and an :output_rejected trace names it.
     module OutputIntegrity
       include OutputRecovery
 

data/lib/smith/agent/output_recovery.rb

@@ -6,8 +6,10 @@ module Smith
   class Agent
     # The follow-up call an output failure may get (OutputIntegrity): a
     # refusal fallback, a truncation retry, or a repair, each within the
-    # agent's settings and never once a tool has run in the step, once the
-    # call has spent what its step reserved, or past MAX_CALLS calls.
+    # agent's settings and never once a tool that is not replay-safe has run
+    # in the step, once the call has spent what its step reserved, or past
+    # MAX_CALLS calls. A follow-up is a new chat from the step's messages,
+    # so the model may call replay-safe tools again.
     module OutputRecovery
       MAX_CALLS = 3 + OutputPolicy::MAX_REPAIRS
       REPAIR_NOTE = "Your previous reply could not be used as the required output:\n%s\n" \
@@ -20,7 +22,7 @@ module Smith
       private
 
       def recovery(plan, attempt, result, failure)
-        return if tools_ran? || plan.calls >= MAX_CALLS || budget_spent?(result)
+        return if tools_uncertain? || plan.calls >= MAX_CALLS || budget_spent?(result)
 
         case failure
         when OutputRefused then refusal_recovery(plan, attempt, failure)
@@ -29,7 +31,7 @@ module Smith
         end
       end
 
-      def tools_ran? = Tool.current_tool_execution_tracker&.started? || false
+      def tools_uncertain? = Tool.current_tool_execution_tracker&.uncertain? || false
 
       def refusal_recovery(plan, attempt, failure)
         fallback = plan.agent_class.on_refusal[:model]

data/lib/smith/agent/provider_attempt.rb

@@ -30,6 +30,9 @@ module Smith
       # True for a response a host's response memo kept (ResponseMemo): the
       # call sent nothing.
       attribute :memo, Types::Bool.optional.default(nil)
+      # The tool results the attempt's tools captured (AttemptCaptures),
+      # which leave the step's with the attempt if it is set aside.
+      attribute :captures, Types::Array.default([].freeze)
 
       def self.success(completion:, model_reference:, **facts)
         new(completion:, model_reference:, error: nil, **facts)

data/lib/smith/agent/provider_completion.rb

@@ -1,6 +1,7 @@
 # frozen_string_literal: true
 
 require_relative "provider_failure_handling"
+require_relative "attempt_captures"
 require_relative "content_bytes"
 require_relative "content_parts"
 require_relative "invocation_preparation"
@@ -15,6 +16,7 @@ module Smith
   class Agent
     module ProviderCompletion
       include ProviderFailureHandling
+      include AttemptCaptures
       include InvocationPreparation
       include ContentParts
       include ProviderCallTiming
@@ -26,13 +28,14 @@ module Smith
 
       # The input's documents and images are read once (ContentBytes), and
       # its cached parts checked against every Anthropic model in the chain,
-      # before the first attempt.
-      def complete_with_provider(agent_class, prepared_input, shape:)
+      # before the first attempt. A candidate that fails over to the next
+      # joins `set_aside` (AttemptCaptures).
+      def complete_with_provider(agent_class, prepared_input, shape:, set_aside: [])
         chain = build_model_chain(agent_class).tap { check_chain_cache_breakpoints!(_1, prepared_input) }
-        ContentBytes.with(prepared_input) { attempt_candidates(agent_class, prepared_input, chain, shape) }
+        ContentBytes.with(prepared_input) { attempt_candidates(agent_class, prepared_input, chain, shape, set_aside) }
       end
 
-      def attempt_candidates(agent_class, prepared_input, chain, shape)
+      def attempt_candidates(agent_class, prepared_input, chain, shape, set_aside)
         candidates = ProviderCandidateSequence.new(chain)
         interrupted = nil
         candidates.each do |model_reference, index|
@@ -43,7 +46,7 @@ module Smith
 
           interrupted ||= attempt if Interruptions.find(attempt.error)
           candidates.suppress(account_failed_provider(attempt, model_reference))
-          fail_over!(attempt, agent_class, candidates, interrupted)
+          fail_over!(attempt, agent_class, candidates, interrupted, set_aside)
         end
 
         # Fail closed: an empty candidate chain (agent without a model)
@@ -53,16 +56,15 @@ module Smith
               "no executable model candidate for #{agent_class}; declare a model or fallback_models"
       end
 
-      # Returns when another candidate is left to try. A candidate that met
-      # an interrupting error (an account out of credit) outlasts how a later
-      # one fails: the step was cut short, not failed, and is taken again
-      # from its first candidate once the host can do its part.
-      def fail_over!(attempt, agent_class, candidates, interrupted)
-        handle_provider_failure!(
-          attempt.error, attempt.model_reference, agent_class,
-          fallback_available: candidates.fallback_available?,
-          attempt_id: attempt.attempt_id
-        )
+      # Returns when another candidate is left to try, the failed attempt's
+      # captures set aside. A candidate that met an interrupting error (an
+      # account out of credit) outlasts how a later one fails: the step was
+      # cut short, not failed, and is taken again from its first candidate
+      # once the host can do its part.
+      def fail_over!(attempt, agent_class, candidates, interrupted, set_aside)
+        handle_provider_failure!(attempt.error, attempt.model_reference, agent_class,
+                                 fallback_available: candidates.fallback_available?, attempt_id: attempt.attempt_id)
+        set_aside.concat(attempt.captures)
       rescue StandardError => e
         raise if interrupted.nil? || Interruptions.find(e)
 
@@ -126,7 +128,7 @@ module Smith
         message_count = chat_message_count(chat)
         observed_reference = observed_model_reference(chat, fallback: model_reference)
         timer = ProviderCallTiming::Timer.start
-        response, stream, mode, call_facts = complete_chat(chat, observed_reference, collected = [], facts)
+        response, stream, mode, call_facts = captured_completion(chat, observed_reference, collected = [], facts)
         timer.stop
         completion = Completion.from_messages(response:, messages: new_chat_messages(chat, message_count), stream:,
                                               mode:)