smith-agents 0.12.6 → 0.13.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 97813bcba79ce652b05eb860da067e3c566b8ba3450344d66940c9f65d23a1dc
-  data.tar.gz: ea20e0a30b7517aab44a43b2668cb5e59fde6b98a829056e8e73ba585338f216
+  metadata.gz: 22c04af0f6062eb36758bb0fedc62663d68a27dce67f016942038ffc1ee3cc07
+  data.tar.gz: 0243fe30beb96d8bccc37869cb6ece9a3525ecd83004de98e18d8f97f4de8b8c
 SHA512:
-  metadata.gz: 37d47a1f6268dc96cd05ce69f4138998d1c5a9649bbb41921882b6ae93a8f5d030586c273f37231501c121dd43492e1a37034cefe88981a5f89138cbf1ccd128
-  data.tar.gz: 40eb2c1a4ac979411860e4971b4814e0c2cdaebc39f1a827e956f7fcb2a5ca33c3f34d18aad339edfa1460a8fce07188f505aa174a2e59233d735c50bab258d3
+  metadata.gz: 41b4452c51e7aa3e6946c192e0f6a9eb9bd89163c90886f013dfffd891da55a53f40ce30b00eb90ecebdc5adab3f7373687c7e0ae64be3e32085d6bfc05af47d
+  data.tar.gz: 4007dfee19e37843c993cf9abf5ea57a6e7450141594090fb0579ae013437de6e3053de16c382a88180bb5536b84fdbe951270e0e6707e7a0c73daec9036a569

data/CHANGELOG.md

@@ -4,6 +4,111 @@ All notable changes to Smith are documented in this file.
 
 Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). Versioning follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html). Smith is pre-1.0 and under active development; expect occasional contract tightening between minor versions until 1.0.
 
+## [0.13.0] - 2026-11-01
+
+### Added
+
+- A tool can declare `capabilities { replay :safe }`: running it again under
+  the same keys does what running it once did (a lookup, a search, a read,
+  one that calls a paid API included). Once a tool had run in a step, Smith
+  made no output repair, refusal fallback or truncation retry, took no
+  fallback model after a provider failure, and turned the step's failure
+  into `Smith::ToolOutcomeUncertain`, which no retry takes. Now only a tool
+  that is not replay-safe does that (any tool without the declaration, a
+  plain RubyLLM tool included). When every tool that ran in a step is
+  replay-safe, the step keeps its follow-ups, its fallback models and its
+  `retry_on`, durable retries included; each runs the tools again, under the
+  same idempotency keys and within the same `tool_calls` allowances. A
+  provider attempt set aside for a follow-up or a fallback model takes its
+  tools' captures out of the step's `tool_results` once the call has its
+  output (by identity, so a concurrent fan-out branch's stay; a call that
+  fails after all keeps every capture), and so does a retried step attempt
+  once its retry is certain, unless the retry continues a tool loop the
+  step paused for budget. Tool evidence reaches a parent step from a nested
+  workflow as it was, and a call the response memo replays counts its kept
+  tool results by the tools its chat declares. `replay :unsafe` states the
+  default; any other value raises `ArgumentError`; `ToolClass.replay_safe?`
+  reads it. Each class declares it for itself: a subclass of a replay-safe
+  tool is unsafe until it says `replay :safe` too.
+
+### Changed
+
+- A `Smith::Tool` subclass inherits its parent's declarations, each read
+  from the nearest class that made it, so a subclass overrides one by
+  declaring it again. Before, a subclass inherited none of them: it ran
+  without its parent's `authorize` (failing open), `privilege`, trace
+  `sensitivity` and `capture_result`, and reached the model without its
+  parent's description and declared parameters (a host subclass of the
+  shipped `Tools::WebSearch` or `Tools::UrlFetcher` stub sent no
+  description and parameters guessed from its `perform`, all strings).
+  `description`, `with_params`, `category`, `authorize`, `before_execute`,
+  `compatible_with` and `capture_result` (its block and strictness
+  together) are the nearest declared; `param`s add up by name, up to a
+  class that gave a `params` schema; a `params` schema is the nearest
+  unless the subclass declares `param`s of its own; `capabilities` merge
+  key by key, except `replay`, which is never inherited: a subclass of a
+  replay-safe tool is replay-safe only when it declares `replay :safe`
+  itself, as safety to run again depends on its own `perform`. A tool none
+  of whose classes declares `authorize` runs as an undeclared tool always
+  has, and a declaration made on a parent after its subclass was created
+  reaches the subclass (`compatible_with` was copied when the subclass was
+  made). Capture was documented as opt-in per concrete class; it is now
+  inherited too.
+
+### Fixed
+
+- A budget refusal that carries a paused tool loop
+  (`Smith::BudgetUnavailable#continuation`) spares a step from
+  `Smith::ToolOutcomeUncertain`, and is continued by its retry, only in the
+  run whose step paused (`paused_by`). One raised from a nested workflow, or
+  made by host code, reached the parent step as if the parent could
+  continue it, so the parent's retry started the child over and ran its
+  tools again, and could take the child's paused messages as its own.
+
+### Upgrading
+
+- A tool subclass that declared less than its parent now runs with its
+  parent's declarations: its parent's `authorize`, `privilege` and
+  `before_execute` apply (a call they deny raises `Smith::ToolPolicyDenied`),
+  its strict capture fails closed without a collector, its traces follow
+  its parent's `sensitivity`, and its requests carry its parent's
+  description and parameters. A subclass meant to differ declares its own
+  (`authorize { ... }`, `capabilities { privilege :none }`,
+  `capture_result(strict: false) { ... }`). As its
+  requests change, the response memo misses once for a call with such a
+  tool (its digest covers tools' descriptions and parameters), so a step
+  cut by the upgrade's deploy pays for that call again and runs its tools
+  again.
+- `Smith::EXECUTION_SEMANTICS_VERSION` stays `"7"`: no persisted state,
+  wait, pending retry or durable plan changes shape.
+
+## [0.12.7] - 2026-10-13
+
+### Added
+
+- A workflow can rename a state or a transition and keep its persisted runs:
+  `renamed_state :old, to: :new` and `renamed_transition :old, to: :new`.
+  On restore, a run persisted under an old name is mapped to the new one
+  before anything checks it: its state, the transition it takes next, the
+  transition its wait, pending retry or pending batch is in, and its last
+  failed step's names, each kept the type it was persisted as. A run waiting
+  in a renamed step resumes there under its wait's own token; one in a
+  renamed state carries on from the new one. A renamed transition keeps its
+  keys: its steps' idempotency keys and wait tokens are made under the first
+  name in its chain (`Workflow.transition_key_name`), so a step cut short, a
+  retry or an early delivery meets the keys its earlier attempt used. Renames
+  chain (a to b, then b to c); a name renamed twice or back to a name it had,
+  and two names renamed to one transition, are refused where declared. A name
+  the class still declares is never taken for an old one, a rename to a name
+  the class does not know is still refused on restore
+  (`Smith::UnknownStateOnRestore`, `UnknownTransitionOnRestore`), and
+  `validate_graph` reports both (`:rename_shadowed`,
+  `:renamed_to_unknown_state`, `:renamed_to_unknown_transition`). Subclasses
+  inherit their parent's renames, and `stuck_for?` follows them. Renames apply
+  after `migrate_from`; no `persistence_schema_version` is needed for one.
+  `Workflow.current_state_name` and `current_transition_name` give what a
+  persisted name stands for now, renames followed.
+
 ## [0.12.6] - 2026-10-13
 
 ### Fixed

data/README.md

@@ -30,7 +30,7 @@ What you get:
 
 ```ruby
 # Gemfile
-gem "smith-agents", "0.12.6", require: "smith"
+gem "smith-agents", "0.13.0", require: "smith"
 ```
 
 In a Rails app:
@@ -137,8 +137,11 @@ Your app delivers events (`resume!`), schedules the timeout
 (`expire_wait!` once `deadline_at` passes), and runs the next step from a
 job. Smith stores runs through an adapter (`:active_record`, `:redis`,
 `:rails_cache`, `:memory` or your own) and refuses a state past
-`persistence_max_bytes`. Waits, child runs, durable retries, replay safety
-and storage: [`docs/PERSISTENCE.md`](docs/PERSISTENCE.md).
+`persistence_max_bytes`. A later version can rename a state or a step
+(`renamed_state :drafted, to: :written`, `renamed_transition :approve, to:
+:review`), and runs persisted under the old names carry on under the new ones,
+waits included, under the keys they had. Waits, child runs, durable retries, replay safety, renames and
+storage: [`docs/PERSISTENCE.md`](docs/PERSISTENCE.md).
 
 ## Agent outputs
 
@@ -237,7 +240,11 @@ everything else. Messages can carry documents and images.
 ## Tools and guardrails
 
 Agents can use `Smith::Tool` subclasses, with a budget on how many calls a
-step may make. Guardrails check inputs, outputs and tool calls. Smith ships
+step may make. Once a tool has run, a step is not retried and takes no
+fallback, as its tools would run again; a tool that is safe to run again (a
+lookup) declares `capabilities { replay :safe }`, and a step whose tools were
+all safe keeps its retries, fallbacks and output repair. Guardrails check
+inputs, outputs and tool calls. Smith ships
 `Tools::Think`; `Tools::WebSearch` and `Tools::UrlFetcher` are stubs that
 declare the interface, which you subclass with your own search or fetch.
 [`docs/TOOLS_AND_GUARDRAILS.md`](docs/TOOLS_AND_GUARDRAILS.md)

data/docs/AGENTS.md

@@ -50,11 +50,15 @@ end
   problems as "path: problem" lines. It then raises `Smith::OutputInvalid`,
   whose `problems` lists at most 20 of them. A blank output still raises
   `Smith::BlankAgentOutputError`.
-- No follow-up call is made once a tool has run in the step (the tools would
-  run again; the step fails with `Smith::ToolOutcomeUncertain` around the output
-  error) or once the call has spent what its step reserved from a budget. A
-  follow-up that fails raises the output error it was making up for, with the
-  provider's error as its cause.
+- No follow-up call is made once a tool that is not replay-safe has run in
+  the step (the tools would run again; the step fails with
+  `Smith::ToolOutcomeUncertain` around the output error) or once the call has
+  spent what its step reserved from a budget. After replay-safe tools only
+  (`capabilities { replay :safe }`, docs/TOOLS_AND_GUARDRAILS.md), the
+  follow-up is made and its model may call them again; once there is an
+  output, the set-aside reply's tool captures leave `tool_results`. A
+  follow-up that fails raises the output error it was making up for, with
+  the provider's error as its cause.
 - Anthropic's structured outputs reject numeric, string-length and most
   array-size constraints: Smith sends Anthropic the schema without them, with
   `additionalProperties: false` on every object, and checks them on the parsed

data/docs/ERRORS_AND_RETRIES.md

@@ -15,9 +15,24 @@ Smith::Errors.retry_forbidden?(error)
 ```
 
 A nested workflow's step fails with the child's own error when the child
-raises it, so these answers carry through nesting. When a tool ran inside the
-child, the parent step's failure becomes `ToolOutcomeUncertain`, which is never
-retried.
+raises it, so these answers carry through nesting. When a tool that is not
+replay-safe ran inside the child, the parent step's failure becomes
+`ToolOutcomeUncertain`, which is never retried.
+
+## After tools ran
+
+A step's failure after one of its tools ran is `ToolOutcomeUncertain`, which
+no retry takes, and the step takes no fallback model and makes no follow-up
+call for its output: running the tools again could repeat what they did. A
+tool that declares `capabilities { replay :safe }` (a lookup, a search, a
+read) says running it again does no harm. When every tool that ran in the
+step is replay-safe, the step keeps its output repair, its refusal fallback
+and truncation retry, its fallback models after a provider failure, and its
+`retry_on`, durable retries included; each runs the tools again, under the
+same idempotency keys and within the same `tool_calls` allowances, and the
+captures of an attempt it sets aside leave `tool_results` once the step has
+its output or its retry is certain. See
+[TOOLS_AND_GUARDRAILS.md](TOOLS_AND_GUARDRAILS.md), Replay-Safe Tools.
 
 Workflow transitions can also declare a bounded local retry policy:
 
@@ -185,7 +200,8 @@ had just come, once, and its `:provider_call` trace says `memo: true`; a reply
 that was cut off or refused fails the step as it did. One that had not ended
 goes on from where it was kept. Kept after its tools' results, it continues
 from them: kept tool results count as tools the step ran, so no retry or
-fallback runs them again, and each is charged to the agent's `tool_calls`
+fallback runs them again (unless each is the result of a tool the chat
+declares replay-safe), and each is charged to the agent's `tool_calls`
 allowance and the workflow's `tool_calls` budget as it was. Kept while its
 tools ran (the step was cut inside a tool), the reply that called them is not
 asked for again: its tools run again, under the same idempotency keys, and the

data/docs/PATTERNS.md

@@ -332,9 +332,11 @@ What you get:
 - a child that routes its failure through its own `on_failure` has handled it:
   the parent step fails with a non-retryable `Smith::WorkflowError`
   (`nested workflow failed: <child message>`, bounded to 1 KiB)
-- tool evidence travels up: when a tool ran in any of the child's steps, a
-  failure of the parent step becomes `Smith::ToolOutcomeUncertain` and is never
-  retried, exactly as when the tool runs in the parent step itself
+- tool evidence travels up: when a tool that is not replay-safe ran in any of
+  the child's steps, a failure of the parent step becomes
+  `Smith::ToolOutcomeUncertain` and is never retried, exactly as when the tool
+  runs in the parent step itself (a child whose tools were all
+  `replay :safe` leaves the parent's retry)
 - parent step count stays parent-scoped
 - parent and child share the outer budget ledger
 - nested best-known token/cost totals roll up into the parent result

data/docs/PERSISTENCE.md

@@ -72,12 +72,40 @@ What Smith does for you:
   `migrate_from` and after the `definition_digest` check, so a digest
   mismatch is reported as one and a migration can map an old name: bump
   `persistence_schema_version` and declare `migrate_from(<previous version>)`,
-  since migrations run only when the stored version is behind. Finished runs
-  are refused too, by `restore`, `restore_or_initialize`,
-  `restorable_billing_state?` and the class-level `run_persisted!`: to read the
-  outcome of a finished run whose terminal state was renamed, migrate it or
-  read the stored JSON directly. `stuck_for?` reports a stored run in an
-  unknown state as stuck, since it can neither advance nor restore
+  since migrations run only when the stored version is behind. A rename needs
+  neither: `renamed_state :old, to: :new` and `renamed_transition :old, to:
+  :new` map a run persisted under the old name to the new one after
+  `migrate_from` (a migration sees the names as they were persisted) and
+  before anything checks it (its state, the transition it takes next, the
+  transition its wait, pending retry or pending batch is in, and its last
+  failed step's names, each kept the type it was persisted as), so a run
+  waiting in a renamed step resumes there under its wait's own token. A
+  renamed transition keeps its keys: its steps' idempotency keys and wait
+  tokens are made under the first name in its chain
+  (`Workflow.transition_key_name`), so a step a deploy cut short runs again, a
+  retry makes its next attempt, and a delivery that came before its wait was
+  stored meets the block's token, all under the keys they had before the
+  rename. Renames chain (a to b, then b to c); a name is renamed at most once,
+  never back to a name it had, and at most one name is renamed to a given
+  transition, each refused with `ArgumentError` where it is declared. A name
+  the class still declares is never taken for an old one, and a rename to a
+  name the class does not know is refused as above when a run restores
+  through it; `validate_graph` reports both before then (`:rename_shadowed`,
+  a warning; `:renamed_to_unknown_state` and `:renamed_to_unknown_transition`,
+  errors). A subclass inherits its parent's renames. A state's rename stays
+  declared while runs persisted under its old name may remain; a
+  transition's, for good, since dropping it moves the transition's keys to
+  its new name.
+  `Workflow.current_state_name` and `current_transition_name` give what a
+  persisted name stands for now, renames followed. In strict mode a payload
+  caught mid-step is refused before renames apply, so
+  `Smith::StepInProgressOnRestore` names its transition as persisted. Finished
+  runs are refused too, by `restore`, `restore_or_initialize`,
+  `restorable_billing_state?` and the class-level `run_persisted!`: a finished
+  run whose terminal state was renamed restores while the rename is declared;
+  to read one whose terminal state was removed, migrate it or read the stored
+  JSON directly. `stuck_for?` reports a stored run in an unknown state
+  (renames followed) as stuck, since it can neither advance nor restore
 
 `preserve_seed: true` is intended for workflows whose seed is durable external
 conversation and whose later accepted outputs are internal stage handoffs. Smith
@@ -860,8 +888,9 @@ end
   transition raises `Smith::SerializationError`; one whose transition the
   class no longer declares exactly (kind, event, `into:`) or that does not
   start from the run's state raises `Smith::UnknownTransitionOnRestore`. A
-  `migrate_from` that renames such things must rewrite `awaiting` too, and
-  `pending_retry` for a retry wait.
+  rename declared with `renamed_transition` (or `renamed_state`) maps the
+  wait's transition first; a `migrate_from` that renames such things itself
+  must rewrite `awaiting` too, and `pending_retry` for a retry wait.
 - A wait that does not end within the store's persistence TTL is refused when
   it is entered, before the block runs. `stuck_for?` reports a waiting run as stuck
   only once its deadline has passed by the threshold.
@@ -906,7 +935,10 @@ end
   runs under the step-in-progress mark, so the restore of a run caught in it
   raises `Smith::StepInProgressOnRestore`.
 - No retry wait is entered once the run's wall-clock deadline has passed, after
-  a tool ran (`Smith::ToolOutcomeUncertain` is never retried), or for a backoff
+  a tool that is not replay-safe ran (`Smith::ToolOutcomeUncertain` is never
+  retried; after replay-safe tools only, the wait is entered, the run stored
+  without the failed attempt's tool captures, and the next attempt runs them
+  again), or for a backoff
   that does not end within the store's persistence TTL (the step fails with a
   `Smith::WorkflowError` whose cause is the attempt's error); runtime readiness
   reports a policy whose longest backoff, jitter included, reaches the TTL

data/docs/TOOLS_AND_GUARDRAILS.md

@@ -41,6 +41,85 @@ configuration, programming, authorization-integrity, or durability failures.
 Smith applies policy and budgeting around that host boundary; it does not
 interpret a host's input schema or failure envelope.
 
+### Subclasses
+
+A subclass inherits every declaration of its parent's but `replay`, each
+read from the nearest class that made it, so a subclass overrides one by
+declaring it again:
+
+- `description`, `with_params`, `category`, `authorize`, `before_execute`,
+  `compatible_with` and `capture_result` (its block and strictness
+  together): the nearest declared wins
+- `param`: parameters add up by name, so a subclass adds parameters or
+  replaces one of its parent's; a schema given with `params` is the
+  nearest one, unless the subclass declares `param`s of its own
+- `capabilities`: merged key by key, so a subclass's block sets or replaces
+  only the keys it names and keeps its parent's `privilege` and
+  `sensitivity`; `replay` is never inherited (Replay-Safe Tools)
+
+An inherited `authorize` guards the subclass as it guards its parent; a
+subclass's own replaces it (call `superclass.authorize` inside it to keep
+both checks), and a tool none of whose classes declares one runs as an
+undeclared tool does. A subclass made before its parent declares something
+still reads it.
+
+### Replay-Safe Tools
+
+A tool that does the same when it runs again under the same keys (a lookup
+over the host's own data, a search, a read, even one that calls a paid API)
+can say so:
+
+```ruby
+class FindPlaces < Smith::Tool
+  description "Find places that match a description."
+  param :query, type: :string, required: true
+
+  capabilities do
+    replay :safe
+  end
+
+  def perform(query:) = PlaceIndex.search(query, limit: 5)
+end
+```
+
+Smith follows the tools each step attempt runs. Once a tool that is not
+replay-safe has run (any tool without the declaration, a plain RubyLLM tool
+included), the step makes no follow-up call for its output (the step fails
+with `Smith::ToolOutcomeUncertain` around the output error), takes no
+fallback model after a provider failure (`ToolOutcomeUncertain`), and a
+failure of the step becomes `ToolOutcomeUncertain`, which no retry takes:
+running the step's tools again could repeat what they did. When every tool
+that ran is replay-safe, the step keeps all three:
+
+- an output repair, a refusal fallback and a truncation retry are each a new
+  call from the step's messages (a repair adds the rejected reply and its
+  problems), so the model may call the tools again
+- a provider failure takes the agent's fallback models, each with a tool
+  loop of its own
+- a failure of the step is its own error, which `retry_on` retries (with
+  `durable: true`, a wait of the run), and the next attempt runs its tools
+  again
+
+An attempt set aside for any of these takes its tools' captures
+(`capture_result`, strict or not) out of the step's `tool_results` once the
+call has its output, by identity, so a concurrent branch's are left; a call
+that fails after all keeps every capture. A retried attempt's go too, once
+the retry is certain (a durable retry's wait entered), unless the retry
+continues a tool loop the step paused for budget, which keeps its tools'
+results (one a nested workflow paused counts as any failure of its tools).
+Tools that run again send the same idempotency keys, and count
+against the agent's `tool_calls` allowance and the workflow's `tool_calls`
+budget as any call does, so a step's tool calls stay bounded. Tool evidence
+from a nested workflow reaches its parent step as it was: a child whose tools
+were all replay-safe leaves the parent's retry. A call the response memo
+replays counts its kept tool results the same way, by the tools its chat
+declares. The declaration is never inherited, as whether a tool is safe to
+run again depends on its own `perform`: a subclass of a replay-safe tool is
+replay-safe only when it declares `replay :safe` itself (its other
+capabilities, description, parameters and capture it inherits). `replay
+:unsafe` states the default, any other value raises `ArgumentError`, and
+`ToolClass.replay_safe?` reads the class's own declaration.
+
 ### Bounded Completion
 
 The default exhaustion policy remains fail-fast:
@@ -186,9 +265,9 @@ entry by raising an exception. A collector return value of `nil` or `false` is
 accepted; delivery is rejected only when the collector raises. Best-effort
 capture logs and continues as before.
 
-Capture policy is declared per concrete tool class and is not inherited. This
-preserves the historical opt-in contract and prevents a subclass from beginning
-to capture or fail strictly without its own declaration.
+A subclass inherits its parent's capture, strictness included, and replaces it
+with a `capture_result` of its own (Subclasses), so a subclass of a strictly
+capturing tool fails closed as its parent does.
 
 Strict capture validates that a collector exists before `perform`, but the
 capture block and collector run after `perform` returns. It is therefore
@@ -459,7 +538,8 @@ Smith ships `Tools::WebSearch`, `Tools::UrlFetcher`, and `Tools::Think`. Tools d
 `Tools::WebSearch` and `Tools::UrlFetcher` are stubs: they declare each
 tool's parameters and description, and their `perform` raises
 `NotImplementedError`. A host subclasses them with its own search or fetch
-behind `perform`. `Tools::Think` works as shipped.
+behind `perform`, and its subclass keeps their parameters, description and
+compatibility (Subclasses). `Tools::Think` works as shipped.
 
 ```ruby
 class WebSearch < Smith::Tools::WebSearch
@@ -499,8 +579,8 @@ Graceful exhaustion is an in-process protocol guarantee, not durable recovery
 for the model/tool loop. If the process exits after a tool begins and before the
 agent completes, the host must treat the outcome as uncertain unless it owns a
 durable per-invocation receipt and reconciliation contract. Smith also refuses
-to start a fresh fallback-model chat after tool execution has begun because that
-would discard the captured evidence and conversation protocol.
+to start a fresh fallback-model chat after a tool that is not replay-safe has
+begun, because the new chat would run its tools again (Replay-Safe Tools).
 
 Within a host invocation scope, `Smith::Tool.current_invocation` exposes the
 normalized tool-call ID and deterministic execution/batch ordinals. The ID is

data/lib/smith/agent/attempt_captures.rb

@@ -0,0 +1,62 @@
+# frozen_string_literal: true
+
+module Smith
+  class Agent
+    # The tool results one provider attempt captures, kept apart so an
+    # attempt the call sets aside (for a fallback model or a follow-up,
+    # possible only when its tools were replay-safe) takes them with it once
+    # the call has its output: the attempt that replaced it ran its tools
+    # again and captured anew. Each entry still reaches the step's collector
+    # as it comes.
+    module AttemptCaptures
+      # A collector that passes each entry on and remembers it.
+      class Scope
+        def initialize(collector)
+          @collector = collector
+          @entries = []
+          @lock = Mutex.new
+        end
+
+        def call(entry)
+          accepted = @collector.call(entry)
+          @lock.synchronize { @entries << entry }
+          accepted
+        end
+
+        def entries = @lock.synchronize { @entries.dup.freeze }
+      end
+      private_constant :Scope
+
+      private
+
+      # The chat's completion (ProviderStreaming#complete_chat), with what
+      # its tools capture recorded as `facts[:captures]`, the attempt's,
+      # however it ends. A step without a callable collector is left as it
+      # is, so strict capture still fails closed there.
+      def captured_completion(chat, model_reference, collected, facts)
+        collector = Tool.current_tool_result_collector
+        return complete_chat(chat, model_reference, collected, facts) unless collector.respond_to?(:call)
+
+        scope = Scope.new(collector)
+        Thread.handle_interrupt(Object => :never) do
+          Tool.current_tool_result_collector = scope
+          begin
+            Thread.handle_interrupt(Object => :immediate) { complete_chat(chat, model_reference, collected, facts) }
+          ensure
+            Tool.current_tool_result_collector = collector
+            facts[:captures] = scope.entries
+          end
+        end
+      end
+
+      # The accepted output; only now do the attempts set aside on the way
+      # take what their tools captured with them, so an invocation that
+      # fails after all keeps every capture, as before.
+      def accepted_output(agent_class, result, set_aside)
+        output = finalize_output(agent_class, result)
+        drop_tool_results(set_aside)
+        output
+      end
+    end
+  end
+end

data/lib/smith/agent/lifecycle.rb

@@ -21,18 +21,22 @@ module Smith
       end
 
       # One invocation: the provider call, then whatever OutputIntegrity
-      # asks for until the response can be the output.
+      # asks for until the response can be the output. The attempts set aside
+      # on the way (`set_aside`) take what their tools captured with them
+      # once there is an output (AttemptCaptures).
       def invoke_agent(agent_class, prepared_input, output_schema: effective_output_schema(agent_class))
         check_deadline!
         plan = output_plan(agent_class, prepared_input, output_schema)
-        attempt = complete_with_provider(agent_class, prepared_input, shape: plan.shape)
+        set_aside = []
+        attempt = complete_with_provider(agent_class, prepared_input, shape: plan.shape, set_aside:)
         loop do
           result = account_attempt(agent_class, attempt.completion, attempt.model_reference,
                                    attempt_id: attempt.attempt_id)
           follow_up = output_follow_up(plan, attempt, result)
-          return finalize_output(agent_class, result) unless follow_up
+          return accepted_output(agent_class, result, set_aside) unless follow_up
 
           settle_rejected(result)
+          set_aside.concat(attempt.captures)
           attempt = follow_up.call
         end
       end

data/lib/smith/agent/output_integrity.rb

@@ -16,12 +16,13 @@ module Smith
     # output contract is sent back for repair up to `output_repairs`, then
     # raises OutputInvalid.
     #
-    # No follow-up call is made once a tool has run in the step (the tools
-    # would run again), once the call has spent what its step reserved, or
-    # past MAX_CALLS; a follow-up that fails raises the output failure it
-    # was making up for, with the provider's error as its cause. Every
-    # response is billed; one that is set aside settles with the
-    # invocation's failed attempts, and an :output_rejected trace names it.
+    # No follow-up call is made once a tool that is not replay-safe has run
+    # in the step (the tools would run again), once the call has spent what
+    # its step reserved, or past MAX_CALLS; a follow-up that fails raises the
+    # output failure it was making up for, with the provider's error as its
+    # cause. Every response is billed; one that is set aside settles with
+    # the invocation's failed attempts, takes what its tools captured with
+    # it, and an :output_rejected trace names it.
     module OutputIntegrity
       include OutputRecovery
 

data/lib/smith/agent/output_recovery.rb

@@ -6,8 +6,10 @@ module Smith
   class Agent
     # The follow-up call an output failure may get (OutputIntegrity): a
     # refusal fallback, a truncation retry, or a repair, each within the
-    # agent's settings and never once a tool has run in the step, once the
-    # call has spent what its step reserved, or past MAX_CALLS calls.
+    # agent's settings and never once a tool that is not replay-safe has run
+    # in the step, once the call has spent what its step reserved, or past
+    # MAX_CALLS calls. A follow-up is a new chat from the step's messages,
+    # so the model may call replay-safe tools again.
     module OutputRecovery
       MAX_CALLS = 3 + OutputPolicy::MAX_REPAIRS
       REPAIR_NOTE = "Your previous reply could not be used as the required output:\n%s\n" \
@@ -20,7 +22,7 @@ module Smith
       private
 
       def recovery(plan, attempt, result, failure)
-        return if tools_ran? || plan.calls >= MAX_CALLS || budget_spent?(result)
+        return if tools_uncertain? || plan.calls >= MAX_CALLS || budget_spent?(result)
 
         case failure
         when OutputRefused then refusal_recovery(plan, attempt, failure)
@@ -29,7 +31,7 @@ module Smith
         end
       end
 
-      def tools_ran? = Tool.current_tool_execution_tracker&.started? || false
+      def tools_uncertain? = Tool.current_tool_execution_tracker&.uncertain? || false
 
       def refusal_recovery(plan, attempt, failure)
         fallback = plan.agent_class.on_refusal[:model]

data/lib/smith/agent/provider_attempt.rb

@@ -30,6 +30,9 @@ module Smith
       # True for a response a host's response memo kept (ResponseMemo): the
       # call sent nothing.
       attribute :memo, Types::Bool.optional.default(nil)
+      # The tool results the attempt's tools captured (AttemptCaptures),
+      # which leave the step's with the attempt if it is set aside.
+      attribute :captures, Types::Array.default([].freeze)
 
       def self.success(completion:, model_reference:, **facts)
         new(completion:, model_reference:, error: nil, **facts)