smith-agents 0.10.0 → 0.11.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 06665b52b660baf1f846ebd104caec677044595bd4aaf71ecf6ac3120e18d478
-  data.tar.gz: '09dd67e047e0f2a3dc6afd0f6b320ab7d5b7f2f23460ddae9576860e7b2e1329'
+  metadata.gz: b3692cc78203afacc91c1ede53a150d8bc864d8be5b9f549739570b5fe823dde
+  data.tar.gz: 6f9981471f8f8ddf2c6fb4141bfe2b6d8631bd21aaf78215c4c87d311aa6d3f2
 SHA512:
-  metadata.gz: 76771bcb4d58d45f110b16a590579a34f698f1e9e2b75cca7806a5c5864f50c8209afc4a258a587069c6dd97f6ae87c1d0a62d434afd4b415f04e199561e79a6
-  data.tar.gz: 5160511ae311634881c774fa8040d3b69f93171e20e76aca595821c9eb7a767c76d0165ffe562190b16f624a4ba2e3e272822b3a1cf6b852ae2d2af7d7521994
+  metadata.gz: 870199be5d2163608606ab5a3eacca29c967e49e3f1f3fd193d88f4bdd57e455e973f03346b2b9769387b363c77147aaeb78a614837e4a665fca1fd4cfc66dc8
+  data.tar.gz: d555df7da59578922841547c53607a81ab7262a427d80a2140e3b9072158764cc33acca058a1fb380b20242315aa5161ca8659c7a26107e35527baa9efea9391

data/CHANGELOG.md

@@ -4,6 +4,209 @@ All notable changes to Smith are documented in this file.
 
 Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). Versioning follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html). Smith is pre-1.0 and under active development; expect occasional contract tightening between minor versions until 1.0.
 
+## [0.11.0] - 2026-10-07
+
+### Added
+
+- `fallback_models` accepts a context block, matching the dynamic
+  `model { |context| ... }`. `fallback_models { |context| [...] }` is evaluated
+  once per agent invocation with the workflow context, and its entries pass
+  exactly the static form's validation: every entry needs an explicit
+  provider. An unqualified or invalid entry fails the step with
+  `Smith::WorkflowError` before any provider attempt, and that error is not
+  retried: `Smith::Errors.retryable?` is false for it, so a `retry_on` without
+  error classes, `retry_on Smith::AgentError`, and host retries built on
+  `Smith::Errors.retryable_classes` all leave it failed, unlike an invalid
+  `model` block result, which raises the retryable `Smith::AgentError`. The
+  chain keeps its de-duplication. Declaring entries and a block together raises
+  `ArgumentError`; a later static declaration clears the block and a later
+  block clears the static list; subclasses inherit the block. Graph
+  inspection, runtime readiness, and the doctor model check never call the
+  block: `fallback_models` returns `nil` for the block form and
+  `fallback_models_block` exposes the block. `resolve_fallback_models(context)`
+  evaluates the block, or returns the static qualified list (empty when none
+  is declared) when no block is configured. Resolution is `O(F)` time and
+  space for `F` returned entries. The static form is unchanged.
+- Every `:provider_call` trace carries `agent_name`, the registered name of
+  the agent that made the attempt (the value its usage entries carry), so a
+  generator and an evaluator sharing one `optimize` round, and so one
+  transition and round, stay apart; it is omitted for an unregistered agent.
+  When the provider reported usage, the trace also carries `input_tokens` and
+  `output_tokens`: the totals of the attempt's usage entries, covering a
+  successful completion, a failed attempt's completed tool-loop prefix, and
+  the usage a failed attempt's error reported. Both are omitted when the
+  provider reported none. A trace adapter can therefore record a complete
+  attempt when it ends instead of waiting for a step checkpoint. Token counts
+  are metadata, never content: `trace_content false` does not hide them.
+  Hosts with a `trace_fields` allowlist for `:provider_call` must add these
+  keys to receive them. The fields are documented in `docs/CONFIGURATION.md`.
+- `optimize`'s `before_eval` may reject a candidate. A returned Hash whose
+  `accept` (Symbol or String key) is `false` is that round's evaluation: it is
+  normalized and validated exactly as evaluator output (`feedback` required,
+  a numeric `score` when `improvement_threshold` is set), the evaluator is not
+  called for the round, and the loop continues as a rejection, so the
+  feedback reaches the generator's refinement turn and `on_exhaustion` applies
+  when rounds run out. Any other return value, including `nil` and a Hash
+  with `accept: true`, is ignored exactly as before and the evaluator judges.
+  A rejected round records only the generator's usage and `:provider_call`.
+  Documented in the Evaluator-Optimizer section of `docs/PATTERNS.md`.
+- `optimize`'s verdicts are an output. Each valid round's evaluation is
+  recorded as `{ round:, source:, verdict: }`: the optimizer round (the one
+  usage entries and traces carry), `:evaluator` or `:before_eval` for the one
+  that gave it, and the evaluation as normalized, every field the evaluator's
+  schema declares included, as a frozen copy. Once its loop has run, the
+  step's record carries its verdicts under `:evaluations` whether the step
+  completed or failed (a completion that fails after the loop, such as a split
+  step's snapshot, included), the verdict that ends the loop included, each
+  also naming its `attempt`, the retry policy's attempt that ran the loop: a
+  step `retry_on` runs again starts its loop afresh with rounds from 0, and the
+  earlier attempt's verdicts stay before the new ones. `RunResult#evaluations`
+  lists every optimize step's verdicts in step, attempt and round order as
+  frozen records with the step's `transition`, and
+  `OptimizationState#evaluations` gives a callable a frozen copy of its own
+  loop's verdicts so far. A step that failed before its loop, and every other
+  step, gains no key; a nested workflow's verdicts stay on its own run.
+  Verdicts are content, so no trace carries them, and like `steps` they
+  belong to the run that executed the step: nothing is persisted and a
+  terminal restore has none. Before this, a rejection's feedback lived only in
+  the loop and an accepted verdict was discarded, so a host could not see why
+  a candidate was sent back. Documented in the Evaluator-Optimizer section of
+  `docs/PATTERNS.md`.
+- `Smith::StepInProgressOnRestore` gains `state`, the persisted state the
+  interrupted step started from (a Symbol), and `transition`, the next
+  transition the payload records (such as a routed one), else `nil`, so a
+  host can tell which step died without reading Smith's payload. Both are set
+  wherever restore raises it and default to `nil` for any other construction;
+  the constructor stays backward compatible and the message gains only
+  ` state=...` when the state is known.
+
+### Changed
+
+- Contract tightening: an `optimize` round's verdict, now recorded as an
+  output, must hold JSON values only (Hashes with String or Symbol keys,
+  Arrays, Strings, Symbols, Integers, finite Floats, true, false, nil), and a
+  step's verdicts together must fit the limits a split step's record is held
+  to (`ExecutionResultSnapshot`'s depth, node and byte caps). A round whose
+  verdict breaks either, most likely a rejection `before_eval` returned
+  holding an object or a very large feedback, now fails the step with
+  `Smith::WorkflowError` at that round, in every run mode, keeping the
+  verdicts recorded before it. Before, such a verdict was never recorded and
+  the loop went on.
+- **Breaking:** a deterministic step's `write_outcome` is deep-symbolized
+  when written, matching restore. A live `RunResult#outcome_payload` (and
+  `RunResult#outcome`) now has Symbol keys at every depth, so a host reading
+  String keys from a live payload must switch to Symbol keys. Restored
+  payloads were already symbolized, so an outcome now reads identically live
+  and after a JSON restore. Persisted JSON is unchanged.
+- Contract tightening: `ActiveRecordStore` refuses to write with a non-nil
+  TTL instead of silently ignoring it, since it has no expiry column.
+  `store`, `store_versioned`, and `replace_exact` raise `ArgumentError`
+  before touching the database. Reads and deletes take no TTL, and resolving
+  the adapter never refuses, so `restore`, `persisted_state_exists?`,
+  `clear_persisted!`, `stuck_for?`, and `heartbeat_age` keep working while a
+  TTL is configured. A global or per-workflow `persistence_ttl` with this
+  adapter is refused by the initial checkpoint that `run_persisted!` and
+  `advance_persisted!` write before their first step, so no step runs before
+  the refusal. A nil TTL (the default) is unaffected. `smith doctor
+  --durability` fails a new `durability.ttl` check when the configured
+  adapter is an `ActiveRecordStore` and `Smith.config.persistence_ttl` is
+  set. The README, `docs/CONFIGURATION.md`, and the `persistence_ttl` DSL
+  comment no longer claim this adapter honours TTL.
+- Contract tightening: workflow and agent `budget` declarations raise
+  `ArgumentError`, naming the accepted keys, when given a key Smith does not
+  read; previously such a key was accepted and ignored. Workflow budgets
+  accept `total_tokens`, `token_limit`, `total_cost`, `tool_calls`, and
+  `wall_clock`; agent budgets accept `token_limit`, `cost`, `wall_clock`,
+  `tool_calls`, `total_tokens`, and `total_cost`. `wall_clock` is in seconds:
+  the README budget example used the ignored `wall_clock_ms: 30_000` and now
+  uses `wall_clock: 30`. The check is `O(K)` time and space for `K` declared
+  keys. State written by 0.10.0 keeps restoring after the host removes such a
+  key: 0.10.0 reserved every declared key at each agent call, so it could
+  persist a consumed entry like `"wall_clock_ms" => 0`, and restore now drops
+  consumed entries (String or Symbol keys) for dimensions the workflow no
+  longer declares. An undeclared dimension has no limit, so dropping it is
+  safe. The filter is `O(D + C)` for `D` declared and `C` consumed keys.
+
+### Fixed
+
+- `DeterministicStep#last_output` returned `nil` after a JSON restore,
+  because a restored session message carries a Symbol key with a String role
+  value. `last_output` now accepts a Symbol or String role under either key
+  form. Restored messages keep the shape 0.10.0 gives them (Symbol keys, JSON
+  values such as `role: "assistant"`).
+- Restored failures keep their type. `ProviderPermanentFailure`,
+  `BudgetExceeded`, and `GuardrailFailed` get their own failure families
+  (`provider_permanent_failure`, `budget_exceeded`, `guardrail_failed`) where
+  they were `other`, and restore as their own classes where they restored as
+  `RuntimeError`. `BlankAgentOutputError` keeps the `agent_error` family and
+  restores as itself where it restored as a plain `AgentError`.
+  `ProviderPermanentFailure` and `BlankAgentOutputError` gain `details` (each
+  value bounded to 512 bytes) and `from_details`, so `provider`, `model_id`,
+  `source_error_class`, `agent_name`, and `model_used` survive a restore.
+  Details stay JSON-normalized, and `FailureRecordValidator` rejects malformed
+  details, or one of these classes claiming another family, with
+  `Smith::PersistedFailureInvalid`. Records written by earlier versions (these
+  classes under `other`, or `BlankAgentOutputError` without details) keep
+  restoring exactly as before. The new families also appear as `error_family`
+  on failed `:transition` traces and `StepFailed` events. Composite branch
+  failure evidence (`Workflow::Composite::ErrorEvidence`) still classifies
+  these three errors as `other`. Rollback: 0.10.0 checks `error_family` and
+  `error_cause_family` against its own family list, so any persisted payload
+  whose `last_failed_step`, or its cause, carries one of the new families
+  fails to restore under 0.10.0, whether the run is terminal or not; clear or
+  settle such runs before rolling back.
+- A failed or aborted attempt's `:provider_call` trace now carries
+  `error_class`: the exception's class name, bounded to 512 bytes by the
+  diagnostic text helper (`anonymous_error` for an anonymous class). It also
+  carries `error_cause_class`, the class name of the error's direct cause
+  (`Exception#cause`, else the error's `wrapped_exception`), captured the same
+  way and omitted when there is no cause: a `Faraday::ConnectionFailed`
+  wraps both never-connected causes (`Errno::ECONNREFUSED`, `SocketError`,
+  `Net::OpenTimeout`) and lost-after-sending ones (`EOFError`,
+  `Errno::ECONNRESET`), and a host deciding whether an attempt may have been
+  billed needs to tell them apart. Messages never ride the trace. Hosts with a
+  `trace_fields` allowlist for `:provider_call` must add `:error_class` and
+  `:error_cause_class` to receive them.
+- `inject_state` injects only non-blank text. A formatter returning `nil` or
+  whitespace-only text no longer adds a bare `[smith:injected-state]` system
+  message, and when a formatter that returned text now returns blank, the
+  stale injected-state message is removed from session history. Injection
+  remains one linear scan over the session.
+- Restore and timestamp asymmetries. The `provider` in
+  `last_agent_execution`, read by `DeterministicStep#last_agent_provider`,
+  restores as the Symbol a live run holds where it restored as a String.
+  `created_at` is now written with microsecond precision (`iso8601(6)`), so a
+  `wall_clock` deadline no longer fires up to a second early; the early firing
+  affected live runs too, since the deadline is always computed from the
+  stored string. `updated_at` is written with the same precision at every
+  step, so it never reads earlier than `created_at`. Payloads with
+  whole-second timestamps still restore.
+- `optimize` sends a structured candidate (a Hash or Array from an
+  `output_schema` generator) to the evaluator, and replays it as the
+  refinement round's assistant turn, as JSON, using the serialization Smith
+  applies at the provider boundary, where it sent Ruby `inspect` notation.
+  String candidates are unchanged.
+- A budget reservation of the reported remaining amount no longer raises
+  `BudgetExceeded` far below the limit. `Budget::Ledger#remaining` returned
+  the Float nearest the exact remaining amount, and reserving it re-reads its
+  shortest decimal form, which after Float-priced costs sits above the exact
+  amount about half the time: a serial step on a $0.50 `total_cost` budget
+  failed its third call at $0.0707. `remaining` now returns a value that
+  never reads back above the exact amount, so it may read one ulp lower than
+  before. `Budget::Ledger#remaining_share(key, parts)` is a new public method
+  that returns one of `parts` equal shares that all fit, which fan-out branch
+  estimates now use (a `total_cost: 0.2` budget split across three branches
+  failed with nothing spent). Integers keep floor division; a Float steps
+  down in `O(1)` ulp steps. Amounts stay Integer or Float in and JSON-safe
+  numerics out.
+- `smith doctor`'s `persistence.capabilities` check reports an adapter that
+  fails to resolve as a failed "Persistence adapter configuration is invalid"
+  check carrying the resolution error, where it reported "No persistence
+  adapter configured".
+- A swallowed `on_step:` callback error is logged with its class after the
+  message (`Smith::Workflow on_step callback error: <message> (<class>)`),
+  where only the message was logged. It is still swallowed.
+
 ## [0.10.0] - 2026-08-24
 
 ### Added

data/README.md

@@ -217,10 +217,12 @@ message batch, appends it under Smith's workflow lifecycle lock, and returns an
 immutable digest witness. The host still owns session identity, transaction
 coordination, persistence, idempotency, and resume policy.
 
-Built-in adapters (all support TTL where the backend allows; `Redis`,
-`ActiveRecord`, and `Memory` support optimistic locking via `store_versioned`
-and exact dispatch claims via `replace_exact`; `ActiveRecord` participates in
-the caller's transaction for host-record coordination;
+Built-in adapters (all except `ActiveRecord` support TTL; `ActiveRecord` has no
+expiry column, so its writes refuse a non-nil `persistence_ttl`, while reads and
+deletes keep working and `smith doctor --durability` flags the combination;
+`Redis`, `ActiveRecord`, and `Memory` support optimistic locking via
+`store_versioned` and exact dispatch claims via `replace_exact`; `ActiveRecord`
+participates in the caller's transaction for host-record coordination;
 `Redis` and `Memory` also support heartbeat probes via `record_heartbeat` /
 `last_heartbeat`):
 
@@ -407,10 +409,16 @@ Guardrails run as input/output gates around agent calls. See [`docs/TOOLS_AND_GU
 
 ```ruby
 class BudgetedWorkflow < Smith::Workflow
-  budget total_tokens: 10_000, total_cost: 0.50, wall_clock_ms: 30_000
+  budget total_tokens: 10_000, total_cost: 0.50, wall_clock: 30
 end
 ```
 
+`wall_clock` is in seconds. A workflow budget accepts `total_tokens`,
+`token_limit`, `total_cost`, `tool_calls`, and `wall_clock`; an agent budget
+accepts `token_limit`, `cost`, `wall_clock`, `tool_calls`, `total_tokens`, and
+`total_cost`. Any other key raises `ArgumentError` naming the accepted keys. A
+restored run drops consumed amounts for keys the workflow no longer declares.
+
 Budgets reserve serially at each step and reconcile after the agent call. Parallel branches reserve scoped envelopes that release back to the parent ledger. The `Workflow::RunResult` carries `total_tokens`, `total_cost`, and per-call `usage_entries`.
 
 Direct ledger integrations settle the exact receipt returned by reservation:
@@ -422,7 +430,9 @@ ledger.reconcile_many!(reservation, actual: { total_tokens: 640, total_cost: 0.0
 
 `release!` / `release_many!` accept the same receipt when work never starts.
 Receipts are ledger-owned and one-shot: another ledger, a replay, or an
-amount-only settlement is rejected.
+amount-only settlement is rejected. `ledger.remaining(key)` never reads back
+above the exact remaining amount, and `ledger.remaining_share(key, parts)`
+returns one of `parts` equal shares of it such that reserving all `parts` fits.
 
 Budget amounts accept finite, non-negative `Integer` and `Float` values. Smith
 uses an exact `BigDecimal` representation derived from each value's canonical

data/docs/CONFIGURATION.md

@@ -124,7 +124,7 @@ observes a provider but finds only a model-only rate, it raises
 | `logger` | Smith's runtime logger | Usually the first setting to add |
 | `persistence_adapter` | Adapter for durable workflow state | `:redis`, `:rails_cache`, `:active_record`, `:memory`, or a custom object |
 | `persistence_options` | Per-adapter options (client, namespace, model, columns) | See "Built-In Persistence Adapters" |
-| `persistence_ttl` | Global TTL for persisted state (Integer/Float seconds; nil = no expiry) | Set when long-tail abandoned workflows accumulate in storage |
+| `persistence_ttl` | Global TTL for persisted state (Integer/Float seconds; nil = no expiry). `:active_record` cannot expire state, so its writes refuse a non-nil TTL (reads and deletes still work, and `smith doctor --durability` flags the combination) | Set when long-tail abandoned workflows accumulate in storage |
 | `persistence_retry_policy` | Exponential-backoff policy for transient adapter I/O failures | Defaults to `{ attempts: 3, base_delay: 0.1, max_delay: 1.0 }` |
 | `retry_attempt_limit` | Maximum attempts accepted by workflow and persistence retry policies | Defaults to `100` |
 | `parallel_branch_limit` | Maximum branches accepted by one homogeneous or heterogeneous fan-out | Defaults to `1_000` |
@@ -135,6 +135,26 @@ observes a provider but finds only a model-only rate, it raises
 | `trace_normalizer` | Emit `:normalizer_decision` trace events from `Smith::Models::Normalizer` | Useful when debugging cross-provider request shaping |
 | `ruby_llm_model_registry` | `:database` to require an AR-backed RubyLLM model registry; `:bundled` for the JSON fallback | Leave at default unless you've migrated to DB-backed |
 
+### The `:provider_call` Trace
+
+With `trace_provider_calls` on, every provider attempt emits one
+`:provider_call` trace when it ends: `model`, `provider`, `attempt_id` (shared
+with the attempt's usage entries), `attempt_index` (its position in the
+fallback chain), `outcome` (`:success`, `:failure`, or `:aborted`), and
+`duration_ms` once the timed call started. `agent_name` is the registered name
+of the agent that made the attempt, so a generator and an evaluator sharing an
+optimize round stay apart. `input_tokens` and `output_tokens` total the
+provider-reported usage that the attempt's usage entries record (a completed
+tool-loop prefix, and a failed attempt's own reported usage); both are omitted
+when the provider reported none. A failed or aborted attempt adds `error_class`,
+and `error_cause_class` for the error's direct cause (`Exception#cause`, else a
+wrapping error's `wrapped_exception`, such as the `EOFError` inside a
+`Faraday::ConnectionFailed`), omitted when there is none. Class names and token
+counts are metadata, never content: `trace_content false` keeps them, messages
+never appear, and a `trace_fields` allowlist for `:provider_call` must name
+each field it wants. Ambient attribution (`transition`, `round`, and so on)
+merges in as for every trace.
+
 ### Recommended First Additions
 
 Add settings in this order:

data/docs/PATTERNS.md

@@ -377,6 +377,65 @@ class TranslationWorkflow < Smith::Workflow
 end
 ```
 
+`before_eval: ->(state, context) { ... }` runs after each candidate is
+generated and before the evaluator is called, and may write to the workflow
+context. When it returns a Hash whose `accept` (Symbol or String key) is
+`false`, that Hash is the round's evaluation: Smith validates it exactly as
+evaluator output (`feedback` is required, and a numeric `score` when
+`improvement_threshold` is set), skips the evaluator for that round, and
+continues as a rejection, so the feedback reaches the generator's refinement
+turn and `on_exhaustion` applies when rounds run out. Convergence and the
+improvement threshold read it as they read evaluator output. Any other return
+value, including a Hash with `accept: true`, is ignored and the evaluator
+judges as before. A deterministic check that makes a candidate unacceptable
+can reject it this way without spending an evaluator call.
+
+Every round's verdict is an output. Once its loop has run, the step's record
+carries its verdicts under `:evaluations`, in attempt and round order, whether
+the step completed or failed (a step that failed before its loop, or any other
+step, has no such key), and `RunResult#evaluations` lists every optimize step's
+verdicts with the step's transition:
+
+```ruby
+result = TranslationWorkflow.new.run!
+result.evaluations
+# => [{ transition: :translate, attempt: 1, round: 0, source: :evaluator,
+#       verdict: { accept: false, feedback: "Keep the formal register.", score: 0.6 } },
+#     { transition: :translate, attempt: 1, round: 1, source: :evaluator,
+#       verdict: { accept: true, feedback: "ok", score: 0.93 } }]
+```
+
+`attempt` is the retry policy's attempt that ran the loop, from 1: when
+`retry_on` runs the step again, the loop starts afresh and its rounds count
+from 0 again, so an earlier attempt's verdicts, given and paid for all the
+same, stay before it, and an attempt that failed before its loop leaves a gap.
+`round` is the optimizer round, the one usage entries and traces carry.
+`source` is `:evaluator`, or `:before_eval` for a rejection `before_eval`
+returned. `verdict` is the evaluation as normalized (Symbol keys at every
+depth), with every field the evaluator's schema declares, not only the
+contract's. A verdict holds JSON values only (Hashes with String or Symbol
+keys, Arrays, Strings, Symbols, Integers, finite Floats, true, false, nil), as
+a provider's structured output does; anything else, most likely an object a
+`before_eval` returned, fails the step with `Smith::WorkflowError`, the same
+way in every run mode. So does a round whose verdict would take the step's
+verdicts past the limits a split step's record is held to (depth, nodes and
+bytes, `ExecutionResultSnapshot`'s), keeping the verdicts before it; a step's
+output counts toward those limits only in a split step. The records on the
+step and on `RunResult#evaluations` are frozen copies; a split step's
+execution snapshot hands the host its own mutable copy of the whole step,
+verdicts included. Only a valid verdict is recorded, and the one that ends the
+loop is kept, whether it accepts, converges, or trips the improvement
+threshold. A step whose completion fails after its loop (a split step's
+snapshot over its limits) keeps its verdicts on the failure record. A nested
+workflow's verdicts stay on its own run: the parent's step carries the child's
+usage, not its verdicts. A callable
+(`before_eval` or an exit mode) reads its own loop's verdicts so far from
+`state.evaluations`, a frozen copy without `attempt`, so it cannot rewrite the
+step's. Verdicts are content, so they are an output and never a
+trace: nothing about them reaches a trace adapter. Like `steps`, they belong to
+the run that executed the step, so a terminal restore has none; a host that
+needs them later stores them when the run returns.
+
 Why this matters:
 
 - the loop is explicit, bounded, and observable

data/docs/PERSISTENCE.md

@@ -534,7 +534,11 @@ before writing when an open transaction has no exact identity capability.
 If execution raises or the process dies after external work, the durable marker
 remains set. A strict restore therefore fails closed with
 `StepInProgressOnRestore`; the host must reconcile operation results or classify
-the run as uncertain rather than blindly replaying the transition. The same
+the run as uncertain rather than blindly replaying the transition. The error's
+`state` is the persisted state the interrupted step started from (a Symbol) and
+its `transition` is the next transition the payload records, such as a routed
+one, or nil when it records none, so the host can tell which step died without
+reading Smith's payload itself. The same
 in-memory workflow object cannot retry an attempted transition. If a host
 transaction rolls back after a successful `persist!`,
 `complete_persisted_step!` rejects the rolled-back checkpoint. When the adapter

data/lib/smith/agent/fallback_configuration.rb

@@ -3,15 +3,38 @@
 module Smith
   class Agent
     module FallbackConfiguration
-      def fallback_models(*models)
+      attr_reader :fallback_models_block
+
+      def fallback_models(*models, &block)
+        return configure_dynamic_fallback_models(models, block) if block
         return @fallback_models_list if models.empty?
 
-        entries = models.flatten.compact.map { qualified_fallback_reference(_1) }
-        @fallback_models_list = entries.uniq(&:key).freeze
+        @fallback_models_block = nil
+        @fallback_models_list = qualified_fallback_references(models)
+      end
+
+      # Evaluated per invocation with the workflow context. The entries pass
+      # the static form's validation and de-duplication: O(F) time and space
+      # for F returned entries.
+      def resolve_fallback_models(context)
+        return @fallback_models_list || [].freeze unless fallback_models_block
+
+        qualified_fallback_references([fallback_models_block.call(context)])
       end
 
       private
 
+      def configure_dynamic_fallback_models(models, block)
+        raise ArgumentError, "fallback_models can take model entries OR a block, not both" if models.any?
+
+        @fallback_models_list = nil
+        @fallback_models_block = block
+      end
+
+      def qualified_fallback_references(models)
+        models.flatten.compact.map { qualified_fallback_reference(_1) }.uniq(&:key).freeze
+      end
+
       def qualified_fallback_reference(model)
         reference = ModelReference.coerce(model)
         return reference if reference.provider

data/lib/smith/agent/provider_attempt.rb

@@ -4,6 +4,7 @@ require "dry-struct"
 
 require_relative "../types"
 require_relative "model_reference"
+require_relative "provider_usage"
 
 module Smith
   class Agent
@@ -17,13 +18,19 @@ module Smith
       # completion call started.
       attribute :attempt_id, Types::String.optional.default(nil)
       attribute :duration_ms, Types::Integer.optional.default(nil)
+      # The candidate's position in the model chain, the registered name of
+      # the agent that made the attempt, and the provider-reported usage its
+      # usage entries record (nil when the provider reported none).
+      attribute :attempt_index, Types::Integer.optional.default(nil)
+      attribute :agent_name, (Types::Symbol | Types::String).optional.default(nil)
+      attribute :usage, Types.Instance(ProviderUsage).optional.default(nil)
 
-      def self.success(completion:, model_reference:, attempt_id: nil, duration_ms: nil)
-        new(completion:, model_reference:, error: nil, attempt_id:, duration_ms:)
+      def self.success(completion:, model_reference:, **facts)
+        new(completion:, model_reference:, error: nil, **facts)
       end
 
-      def self.failure(error:, model_reference:, attempt_id: nil, duration_ms: nil)
-        new(completion: nil, model_reference:, error:, attempt_id:, duration_ms:)
+      def self.failure(error:, model_reference:, **facts)
+        new(completion: nil, model_reference:, error:, **facts)
       end
 
       def success?

data/lib/smith/agent/provider_call_timing.rb

@@ -1,5 +1,7 @@
 # frozen_string_literal: true
 
+require_relative "../diagnostic_text"
+
 module Smith
   class Agent
     # Monotonic timing and trace emission for one provider attempt: the
@@ -7,6 +9,9 @@ module Smith
     # (including any provider-side tool loop). Per-network-round timing
     # belongs to the host's RubyLLM notification subscriptions, not Smith.
     module ProviderCallTiming
+      EXCEPTION_CAUSE = Exception.instance_method(:cause)
+      private_constant :EXCEPTION_CAUSE
+
       class Timer
         def self.start
           new
@@ -38,7 +43,9 @@ module Smith
       # durations across entries. duration_ms is present only when the timed
       # provider call actually started: a failure before dispatch (model
       # resolution, chat construction) emits its attempt without a duration.
-      def record_provider_call_trace(attempt, attempt_index, aborted: false)
+      # input_tokens and output_tokens total the attempt's usage entries, so
+      # a host can record the whole attempt when it ends.
+      def record_provider_call_trace(attempt, aborted: false)
         Smith::Trace.record(
           type: :provider_call,
           data: {
@@ -46,12 +53,23 @@ module Smith
             provider: attempt.model_reference.provider,
             duration_ms: attempt.duration_ms,
             attempt_id: attempt.attempt_id,
-            attempt_index: attempt_index,
-            outcome: provider_call_outcome(attempt, aborted)
+            attempt_index: attempt.attempt_index,
+            outcome: provider_call_outcome(attempt, aborted),
+            **provider_call_error_classes(attempt.error),
+            agent_name: attempt.agent_name,
+            **provider_call_usage(attempt.usage)
           }.compact
         )
       end
 
+      # Token counts are metadata, never content, so the content policy
+      # leaves them in place.
+      def provider_call_usage(usage)
+        return {} unless usage
+
+        { input_tokens: usage.input_tokens, output_tokens: usage.output_tokens }
+      end
+
       def provider_call_outcome(attempt, aborted)
         return :aborted if aborted
         return :success if attempt.success?
@@ -59,14 +77,51 @@ module Smith
         :failure
       end
 
+      # Class names only, never a message: the same bounded identifier the
+      # failed :transition trace carries, for the error and its direct cause.
+      def provider_call_error_classes(error)
+        return {} unless error
+
+        cause = provider_call_error_cause(error)
+        {
+          error_class: DiagnosticText.error_class_name(error),
+          error_cause_class: cause && DiagnosticText.error_class_name(cause)
+        }
+      end
+
+      # A transport error wraps what actually happened (never connected, or
+      # lost the connection after sending); Faraday keeps it as
+      # wrapped_exception when the error was not raised inside a rescue.
+      def provider_call_error_cause(error)
+        cause = EXCEPTION_CAUSE.bind_call(error)
+        cause ||= error.wrapped_exception if error.respond_to?(:wrapped_exception)
+        cause if cause.is_a?(Exception)
+      rescue StandardError
+        nil
+      end
+
+      def completed_provider_attempt(completion, observed_reference, timer, facts)
+        attempt = ProviderAttempt.success(
+          completion:, model_reference: observed_reference, duration_ms: timer.elapsed_ms,
+          usage: ProviderUsage.sum(completion.provider_usages), **facts
+        )
+        record_provider_call_trace(attempt)
+        attempt
+      end
+
       # Aborted (non-provider) attempts emit a :provider_call too, so the
       # prefix-accounted usage entries stamped with this attempt_id always
-      # have their join target; the error then propagates unchanged.
-      def failed_provider_attempt(error, observed_reference, attempt_id, timer, attempt_index)
+      # have their join target; the error then propagates unchanged. A
+      # provider failure's own reported usage is accounted beside the
+      # completed prefix, so its trace totals both.
+      def failed_provider_attempt(error, observed_reference, timer, facts, prefix_usages)
+        aborted = !provider_failure?(error)
+        usages = aborted ? prefix_usages : [*prefix_usages, ProviderUsage.from_message(error)].compact
         attempt = ProviderAttempt.failure(
-          error: error, model_reference: observed_reference, attempt_id:, duration_ms: timer&.elapsed_ms
+          error:, model_reference: observed_reference, duration_ms: timer&.elapsed_ms,
+          usage: ProviderUsage.sum(usages), **facts
         )
-        record_provider_call_trace(attempt, attempt_index, aborted: !provider_failure?(error))
+        record_provider_call_trace(attempt, aborted:)
         attempt
       end
     end

data/lib/smith/agent/provider_completion.rb

@@ -68,6 +68,8 @@ module Smith
       end
 
       def fallback_model_references(agent_class)
+        return agent_class.resolve_fallback_models(@context || {}) if agent_class.fallback_models_block
+
         Array(agent_class.fallback_models).map { |model| ModelReference.coerce(model) }
       end
 
@@ -87,7 +89,7 @@ module Smith
       # completion, prefix accounting on failure); splitting it would scatter
       # the rescue-path accounting away from what it accounts for.
       def attempt_model(agent_class, prepared_input, model_reference, output_schema:, attempt_index:)
-        attempt_id = SecureRandom.uuid
+        facts = { attempt_id: SecureRandom.uuid, attempt_index:, agent_name: agent_class.register_as }
         chat = prepared_attempt_chat(agent_class, prepared_input, model_reference, output_schema:)
         message_count = chat_message_count(chat)
         observed_reference = observed_model_reference(chat, fallback: model_reference)
@@ -96,16 +98,14 @@ module Smith
         timer.stop
         completion = Completion.from_messages(response: response, messages: new_chat_messages(chat, message_count))
 
-        attempt = ProviderAttempt.success(
-          completion:, model_reference: observed_reference, attempt_id:, duration_ms: timer.elapsed_ms
-        )
-        record_provider_call_trace(attempt, attempt_index)
-        attempt
+        completed_provider_attempt(completion, observed_reference, timer, facts)
       rescue StandardError => e
         timer&.stop
         observed_reference ||= observed_model_reference(chat, fallback: model_reference)
-        account_completed_prefix(agent_class, observed_reference, new_chat_messages(chat, message_count), attempt_id:)
-        attempt = failed_provider_attempt(e, observed_reference, attempt_id, timer, attempt_index)
+        prefix_usages = account_completed_prefix(
+          agent_class, observed_reference, new_chat_messages(chat, message_count), attempt_id: facts[:attempt_id]
+        )
+        attempt = failed_provider_attempt(e, observed_reference, timer, facts, prefix_usages)
         raise unless provider_failure?(e)
 
         attempt

data/lib/smith/agent/provider_usage.rb

@@ -20,6 +20,12 @@ module Smith
         new(input_tokens:, output_tokens:)
       end
 
+      def self.sum(usages)
+        return if usages.empty?
+
+        new(input_tokens: usages.sum(&:input_tokens), output_tokens: usages.sum(&:output_tokens))
+      end
+
       def total_tokens
         input_tokens + output_tokens
       end

data/lib/smith/agent/usage_tracking.rb

@@ -23,9 +23,12 @@ module Smith
         record_failed_usage(agent_class, model_reference, input, output, attempt_id:)
       end
 
+      # Returns the provider usages it recorded, so the attempt's trace can
+      # carry the same totals its usage entries hold.
       def account_completed_prefix(agent_class, model_reference, messages, attempt_id: nil)
         completion = Completion.from_messages(response: nil, messages: messages)
         record_completion_usage(agent_class, completion, :partial_attempt, model_reference, attempt_id:)
+        completion.provider_usages
       end
 
       def record_failed_usage(agent_class, model_reference, input_tokens, output_tokens, attempt_id: nil)