smith-agents 0.12.2 → 0.12.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 746bb384c941e4a7276937eec58e74e1b1f11230e7310190a64401e348a92f61
-  data.tar.gz: 26b5e327966a11488d166214a0363bc2f554a79230b54392775609bcc2d25b6f
+  metadata.gz: e2c791c5165889ff1098cc5c92976b1affe2c3f82108815fda1c98fae56c0430
+  data.tar.gz: 5b7d1d9f41e2949c335d6dc8b26a7f69aff73c2edf4ee39e100760ca3c4f4145
 SHA512:
-  metadata.gz: 652ec2c82ed914ce2aa798ccd87ee6a70afbc9149b731bd16e2ff2a14eecef13c23161b2039d46a5943c2a0eb4bcf9135b48c1ba8308e8585fba827aded3cdc7
-  data.tar.gz: 2f86f38c713ce0e62cd01ebfbc7b62270d30c30a34f8db2954526c73a9996be0792fd3ef874dbc500e2489883a178e267215b17b53494aaab648d3f9bd612f70
+  metadata.gz: e76354065df47cf04703bc41cda18256df9f2118a9d714f046f3fe04c567256d969be828235b0a6da2ff0ee83fe8953ba8c381ed4209c4f49776cde1658ab5d0
+  data.tar.gz: 3e73cddd45cd37191712cc2ed4d5cdc1fe62dbe07a2de73b86bd626f43eda0ba74efcbf8b40fabbbc15db21e6517495567ac346c4339f3c9864e316894170d3f

data/CHANGELOG.md

@@ -4,6 +4,56 @@ All notable changes to Smith are documented in this file.
 
 Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). Versioning follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html). Smith is pre-1.0 and under active development; expect occasional contract tightening between minor versions until 1.0.
 
+## [0.12.4] - 2026-10-13
+
+### Changed
+
+- A provider's refusal for want of credit or past a spending limit is
+  `RubyLLM::PaymentRequiredError` whatever status it came with
+  (`Smith::BillingRefusals`), with the error RubyLLM raised as its cause. A
+  refusal is told by what the provider says it is: Anthropic's `billing_error`,
+  its tier's monthly spend cap (a 429 whose `error_code` is
+  `enforced_spend_limit_reached`), its 400s for spent credit ("Your credit
+  balance is too low") and for a spend limit the account set ("You have
+  reached your specified API usage limits"), and OpenAI's
+  `insufficient_quota`. A streamed request's refusal, which RubyLLM raises as
+  a `RubyLLM::ServerError`, is one too, and so is one met by
+  `Smith::Providers::Anthropic::Batches#submit`. It is the provider account's
+  failure, so another provider in the fallback chain is tried, and a host can
+  name `RubyLLM::PaymentRequiredError` in `interrupting_errors` to have a step
+  wait for credit instead of failing. Any other error is unchanged: a rate
+  limit that mentions a quota (Gemini's) stays a rate limit.
+- A model candidate that met an interrupting error outlasts how a later
+  fallback fails: the step is cut short, as it would have been with no
+  fallback, instead of failing on the fallback's error.
+
+## [0.12.3] - 2026-10-12
+
+### Changed
+
+- What identifies a request leaves prompt-cache markers out
+  (`Smith::CacheMarkers`): the response memo's key, a batch request's digest
+  and a seed's drift digest (`seed_validation`). A marker says how a request
+  is billed, never what it asks, so a host may decide call by call whether to
+  cache a part (an agent's instructions, a content part, the whole request)
+  and a replay still finds what the first call got, as long as the part keeps
+  its form (the same text block, with or without its marker). Only the places
+  Anthropic puts `cache_control` are looked at (the request, its system and
+  message blocks, a tool result's blocks, its tools) and Smith's own
+  `"cache" => true` on a seed's content parts; a tool's input or schema, or
+  anything else a request carries as data, is never touched. A batch result
+  for a request that differs from the one sent only in its markers is no
+  longer taken as a changed request.
+
+### Upgrading
+
+- Response memo records kept by 0.12.2 for calls that carried a cache marker
+  are not found after the upgrade (their key took the marker in), so such a
+  call cut by the upgrade's deploy is paid for once more. A batch request
+  captured by 0.12.2, and a run's seed digest taken by 0.12.2, still match as
+  they were taken, cache markers included: one whose markers change after the
+  upgrade counts as changed, as before.
+
 ## [0.12.2] - 2026-10-12
 
 ### Added

data/README.md

@@ -30,7 +30,7 @@ What you get:
 
 ```ruby
 # Gemfile
-gem "smith-agents", "0.12.2", require: "smith"
+gem "smith-agents", "0.12.4", require: "smith"
 ```
 
 In a Rails app:

data/docs/AGENTS.md

@@ -216,7 +216,10 @@ without sleeping: `Smith::Providers::Anthropic::Batches` gives the host
 `submit`, `status`, `results` and `cancel`. `resume!` with
 `{custom_id => result}` runs the step again, each call taking its result as
 the reply it would have got. A call that renders another request by then (a
-date in its prompt, data read afresh, a prompt changed in a deploy) does what
+date in its prompt, data read afresh, a prompt changed in a deploy; a prompt
+cache marker added or dropped is no change, as it never changes what the
+request asks, though a request captured by Smith 0.12.2 is matched as it was
+sent, markers included) does what
 its step's `on_request_change:` says: `:keep` (the default) takes the result,
 which answers the request as sent, and its `:provider_call` trace says
 `request_changed: true`; `:resend` sends the request as rendered now over the

data/docs/ERRORS_AND_RETRIES.md

@@ -99,6 +99,23 @@ next advanced; a strict run is found under way on restore
 (`Smith::StepInProgressOnRestore`), as after a crash. The instance that
 raised is not advanced again: restore the run.
 
+A provider's refusal for want of credit or past a spending limit is
+`RubyLLM::PaymentRequiredError` whatever status it came with, its cause the
+error RubyLLM raised. Smith tells one by what the provider says it is:
+Anthropic's `billing_error` (402), its tier's monthly spend cap (a 429 whose
+`error.details.error_code` is `enforced_spend_limit_reached`), its 400s for
+spent credit ("Your credit balance is too low") and for a spend limit the
+account set ("You have reached your specified API usage limits"), and
+OpenAI's `insufficient_quota` (429). RubyLLM raises a streamed request's
+refusal as a `RubyLLM::ServerError`; it is one all the same. It is the
+provider account's, so another provider in the fallback chain is tried;
+naming `RubyLLM::PaymentRequiredError` makes a step that meets it wait for
+credit instead of failing, even when a fallback then fails some other way.
+`Smith::Providers::Anthropic::Batches#submit` raises it too. Any other error
+stays as it is: a rate limit whose message mentions a quota (Gemini's) is a
+rate limit. RubyLLM's own HTTP retries (`max_retries`) may repeat a 429 or a
+streamed refusal before Smith sees it; a refused request is not charged.
+
 Name only errors that mean your own systems failed. A provider's own failure
 (`RubyLLM::Error`, or its HTTP client's `Faraday::Error`) is the provider's
 whatever it was raised from, so naming a low-level class such as
@@ -138,10 +155,12 @@ attempts it runs under (its step's, and each enclosing run's for a nested
 workflow, so a retry never takes the attempt before's answer), its attempt
 index (its model in the fallback chain, or a follow-up's place), its place
 among the attempt's calls, and a digest of its request (model, messages,
-tools with their descriptions and parameters, schema, params). A replay
-reaches the same key; a call whose request changed since (a deployed prompt or
-tool, another model, a date in the prompt) misses and is sent again, as a miss
-always may be.
+tools with their descriptions and parameters, schema, params), Anthropic's
+prompt-cache markers aside: whether a part is cached never changes what the
+call asks, so a host may decide it call by call, keeping the part's form (the
+same text block, with or without its marker). A replay reaches the same key; a
+call whose request changed since (a deployed prompt or tool, another model, a
+date in the prompt) misses and is sent again, as a miss always may be.
 
 Its record (`{"version" => 1 | 2, "messages" => [...], "complete" => true |
 false}`, JSON) holds the messages the call added: the model's replies

data/lib/smith/agent/provider_call_timing.rb

@@ -123,7 +123,9 @@ module Smith
       # have their join target; the error then propagates unchanged. A
       # provider failure's own reported usage is accounted beside the
       # completed prefix, so its trace totals both.
+      # A billing refusal is recorded as the provider account's (BillingRefusals).
       def failed_provider_attempt(error, observed_reference, timer, facts, prefix_usages)
+        error = BillingRefusals.classified(error)
         aborted = !provider_failure?(error)
         usages = aborted ? prefix_usages : [*prefix_usages, ProviderUsage.from_message(error)].compact
         attempt = ProviderAttempt.failure(

data/lib/smith/agent/provider_completion.rb

@@ -34,18 +34,16 @@ module Smith
 
       def attempt_candidates(agent_class, prepared_input, chain, shape)
         candidates = ProviderCandidateSequence.new(chain)
+        interrupted = nil
         candidates.each do |model_reference, index|
           check_deadline! if index.positive?
           withdraw_streamed(index.positive? ? :fallback : :retry)
           attempt = attempt_model(agent_class, prepared_input, model_reference, shape:, attempt_index: index)
           return attempt if attempt.success?
 
+          interrupted ||= attempt if Interruptions.find(attempt.error)
           candidates.suppress(account_failed_provider(attempt, model_reference))
-          handle_provider_failure!(
-            attempt.error, attempt.model_reference, agent_class,
-            fallback_available: candidates.fallback_available?,
-            attempt_id: attempt.attempt_id
-          )
+          fail_over!(attempt, agent_class, candidates, interrupted)
         end
 
         # Fail closed: an empty candidate chain (agent without a model)
@@ -55,6 +53,22 @@ module Smith
               "no executable model candidate for #{agent_class}; declare a model or fallback_models"
       end
 
+      # Returns when another candidate is left to try. A candidate that met
+      # an interrupting error (an account out of credit) outlasts how a later
+      # one fails: the step was cut short, not failed, and is taken again
+      # from its first candidate once the host can do its part.
+      def fail_over!(attempt, agent_class, candidates, interrupted)
+        handle_provider_failure!(
+          attempt.error, attempt.model_reference, agent_class,
+          fallback_available: candidates.fallback_available?,
+          attempt_id: attempt.attempt_id
+        )
+      rescue StandardError => e
+        raise if interrupted.nil? || Interruptions.find(e)
+
+        raise terminal_provider_error(interrupted.error, interrupted.model_reference), cause: interrupted.error
+      end
+
       # Suppression keys on the provider that actually rejected the
       # account; when the chat is unobservable the attempted reference's
       # declared provider is the best available attribution, so a dead

data/lib/smith/batch.rb

@@ -45,16 +45,18 @@ module Smith
       deep_freeze(JSON.parse(encoded))
     end
 
-    # What identifies a request's content: a SHA-256 of its payload and
-    # headers as JSON gives them, so the request a delivery answers can be
-    # told from one the step would render now.
-    def digest(request) = digest_of(request.fetch("params"), request.fetch("headers"))
-
-    # Whether a call renders a request other than the one sent (`digest`).
-    # Nothing is sent, so nothing in it is refused; one JSON cannot give
-    # counts as changed.
+    # What identifies a request's content: a SHA-256 of its payload, prompt
+    # cache markers aside (CacheMarkers), and headers as JSON gives them, so
+    # the request a delivery answers can be told from one the step would
+    # render now.
+    def digest(request) = digest_of(CacheMarkers.strip_request(request.fetch("params")), request.fetch("headers"))
+
+    # Whether a call renders a request other than the one sent (`digest`),
+    # cache markers aside; a digest Smith 0.12.2 took with its markers still
+    # matches the request as it was. Nothing is sent, so nothing in it is
+    # refused; one JSON cannot give counts as changed.
     def changed?(digest, params, headers)
-      digest_of(params, headers.to_h) != digest
+      [CacheMarkers.strip_request(params), params].none? { digest_of(_1, headers.to_h) == digest }
     rescue JSON::GeneratorError, JSON::NestingError, Encoding::UndefinedConversionError
       true
     end

data/lib/smith/billing_refusals.rb

@@ -0,0 +1,68 @@
+# frozen_string_literal: true
+
+require "json"
+require "ruby_llm"
+
+module Smith
+  # A provider's refusal for want of credit or past a spending limit, whatever
+  # status it came with. Smith takes each as RubyLLM::PaymentRequiredError,
+  # the error RubyLLM raises for a 402 (its cause the error RubyLLM raised),
+  # so the refusal is the provider account's (another provider in the
+  # fallback chain is tried) and a host can name that one class in
+  # interrupting_errors to have a step wait for credit instead of failing.
+  #
+  # A refusal is told by what the provider says it is, never by its status
+  # alone: Anthropic's billing_error (402), its tier's monthly spend cap (429,
+  # error_code enforced_spend_limit_reached), its spent credit and a spend
+  # limit the account set (400s known only by how their message begins), and
+  # OpenAI's insufficient_quota (429). A streamed request's refusal arrives
+  # as a RubyLLM::ServerError, since RubyLLM takes every streamed error as a
+  # 500. Any other error stays as it is: a rate limit that mentions a quota
+  # (Gemini's) is a rate limit.
+  module BillingRefusals
+    CLASSES = [RubyLLM::BadRequestError, RubyLLM::RateLimitError, RubyLLM::ServerError].freeze
+    TYPES = %w[billing_error insufficient_quota].freeze
+    CODES = %w[enforced_spend_limit_reached].freeze
+    MESSAGES = [
+      /\AYour credit balance is too low/i,
+      /\AYou have reached your specified (?:workspace )?API usage limits/i
+    ].freeze
+    private_constant :CLASSES, :TYPES, :CODES, :MESSAGES
+
+    module_function
+
+    # The error as Smith takes it: a billing refusal as a
+    # RubyLLM::PaymentRequiredError, anything else unchanged.
+    def classified(error)
+      return error unless refusal?(error)
+
+      begin
+        raise RubyLLM::PaymentRequiredError.new(error.response, error.message), cause: error
+      rescue RubyLLM::PaymentRequiredError => e
+        e
+      end
+    end
+
+    def refusal?(error)
+      return false unless CLASSES.any? { error.is_a?(_1) }
+
+      said = provider_error(error)
+      details = said["details"].is_a?(Hash) ? said["details"] : {}
+
+      TYPES.include?(said["type"]) ||
+        CODES.include?(details["error_code"]) ||
+        MESSAGES.any? { error.message.to_s.match?(_1) }
+    end
+
+    # The provider's own error object in the response body, or {} when the
+    # body carries none.
+    def provider_error(error)
+      body = error.response.respond_to?(:body) ? error.response.body : nil
+      body = JSON.parse(body) if body.is_a?(String)
+      said = body["error"] if body.is_a?(Hash)
+      said.is_a?(Hash) ? said : {}
+    rescue JSON::ParserError
+      {}
+    end
+  end
+end

data/lib/smith/cache_markers.rb

@@ -0,0 +1,86 @@
+# frozen_string_literal: true
+
+module Smith
+  # Anthropic's prompt-cache markers (`cache_control: { type: "ephemeral" }`
+  # on a request, a system block, a content block, a tool result's block or
+  # a tool) and Smith's own `"cache" => true` on a content part say how a
+  # request is billed, never what it asks: the reply is the same with or
+  # without them. What identifies a request (the response memo's key, a batch
+  # request's digest, a seed's drift digest) leaves them out, so a host that
+  # caches a prompt on one call and not on the next still finds what the
+  # first call got. Only the places markers can sit are looked at: a tool's
+  # input or schema, or anything else a request carries as data, is never
+  # touched, and nothing is walked deeper than a tool result's blocks.
+  module CacheMarkers
+    MARKER_KEYS = [:cache_control, "cache_control"].freeze
+    private_constant :MARKER_KEYS
+
+    module_function
+
+    # A request payload (a batch request's params, a chat's params): its own
+    # marker, and those of its system blocks, its messages' blocks and its
+    # tools.
+    def strip_request(request)
+      return request unless request.is_a?(Hash)
+
+      request = unmarked(request)
+      request = replaced(request, :system) { strip_blocks(_1) }
+      request = replaced(request, :messages) do |messages|
+        messages.is_a?(Array) ? messages.map { |message| replaced(message, :content) { strip_blocks(_1) } } : messages
+      end
+      replaced(request, :tools) { |tools| tools.is_a?(Array) ? tools.map { unmarked(_1) } : tools }
+    end
+
+    # A message's content: a block or blocks, a tool result's own blocks
+    # included.
+    def strip_blocks(content)
+      case content
+      when Array then content.map { strip_block(_1) }
+      when Hash then strip_block(content)
+      else content
+      end
+    end
+
+    # Seed messages as Smith keeps them: each content part without its
+    # `"cache" => true`.
+    def strip_parts(messages)
+      return messages unless messages.is_a?(Array)
+
+      messages.map do |message|
+        replaced(message, :content) { |content| content.is_a?(Array) ? content.map { unflagged(_1) } : content }
+      end
+    end
+
+    def unflagged(part) = part.is_a?(Hash) ? part.except("cache", :cache) : part
+
+    def strip_block(block)
+      return block unless block.is_a?(Hash)
+
+      block = unmarked(block)
+      return block unless (block[:type] || block["type"]).to_s == "tool_result"
+
+      replaced(block, :content) { |content| content.is_a?(Array) ? content.map { unmarked(_1) } : unmarked(content) }
+    end
+
+    def unmarked(value)
+      return value unless value.is_a?(Hash)
+
+      value.reject { |key, item| MARKER_KEYS.include?(key) && marker?(item) }
+    end
+
+    def marker?(value) = value.is_a?(Hash) && (value[:type] || value["type"]).to_s == "ephemeral"
+
+    # The Hash with its value under `key` (a Symbol or a String key) passed
+    # through the block.
+    def replaced(hash, key)
+      return hash unless hash.is_a?(Hash)
+
+      [key, key.to_s].each do |name|
+        hash = hash.merge(name => yield(hash[name])) if hash.key?(name)
+      end
+      hash
+    end
+
+    private_class_method :strip_block, :unflagged, :unmarked, :marker?, :replaced
+  end
+end

data/lib/smith/providers/anthropic/batches.rb

@@ -34,13 +34,13 @@ module Smith
           @connection = RubyLLM::Providers::Anthropic.new(config).connection
         end
 
-        # Submits the requests; returns the batch id.
+        # Submits the requests; returns the batch id. A refusal for want of
+        # credit raises RubyLLM::PaymentRequiredError (BillingRefusals).
         def submit(requests)
           entries = Array(requests).map { |entry| entry.transform_keys(&:to_sym) }
           body = { requests: entries.map { |entry| { custom_id: entry.fetch(:custom_id), params: params(entry) } } }
           check_batch!(entries, body)
-          headers = shared_headers(entries)
-          @connection.post(PATH, body) { |request| add_headers(request.headers, headers) }.body.fetch("id")
+          create(body, shared_headers(entries))
         end
 
         # The batch's processing status ("in_progress", "canceling" or
@@ -67,6 +67,12 @@ module Smith
 
         private
 
+        def create(body, headers)
+          @connection.post(PATH, body) { |request| add_headers(request.headers, headers) }.body.fetch("id")
+        rescue RubyLLM::Error => e
+          raise BillingRefusals.classified(e)
+        end
+
         def params(entry)
           request = entry.fetch(:request).transform_keys(&:to_s)
           raise ArgumentError, "a request for #{request["provider"]} cannot go in an Anthropic batch" unless

data/lib/smith/response_memo.rb

@@ -149,7 +149,7 @@ module Smith
 
     def request_digest(chat)
       shape = [chat.model.id, chat.model.provider.to_s, chat.messages.map { message_shape(_1) },
-               tools_shape(chat.tools), chat.schema, chat.params]
+               tools_shape(chat.tools), chat.schema, CacheMarkers.strip_request(chat.params)]
       Digest::SHA256.hexdigest(JSON.generate(shape))
     rescue StandardError
       nil
@@ -158,7 +158,7 @@ module Smith
     def message_shape(message)
       content = message.content
       body = case content
-             when RubyLLM::Content::Raw then content.value
+             when RubyLLM::Content::Raw then CacheMarkers.strip_blocks(content.value)
              when RubyLLM::Content
                [content.text, content.attachments.map { [_1.filename, _1.mime_type, _1.content.to_s.bytesize] }]
              else content

data/lib/smith/version.rb

@@ -1,6 +1,6 @@
 # frozen_string_literal: true
 
 module Smith
-  VERSION = "0.12.2"
+  VERSION = "0.12.4"
   EXECUTION_SEMANTICS_VERSION = "7"
 end

data/lib/smith/workflow/persistence.rb

@@ -228,13 +228,16 @@ module Smith
         validate_step_in_progress!(fields.to_h { |field| [field, state[field] || state[field.to_s]] })
       end
 
+      # A digest Smith before 0.12.3 took, its parts' cache flags included.
+      def flagged_seed_digest?(messages, stored) = compute_seed_digest(messages, cache_flags: true) == stored
+
       def validate_seed_digest!(normalized)
         stored_digest = normalized[:seed_digest]
         return if stored_digest.nil?
 
         current_messages = compute_seed_messages
         current_digest = compute_seed_digest(current_messages)
-        return if current_digest == stored_digest
+        return if current_digest == stored_digest || flagged_seed_digest?(current_messages, stored_digest)
 
         case self.class.seed_validation
         when :strict

data/lib/smith/workflow.rb

@@ -412,10 +412,12 @@ module Smith
       normalize_seed_messages(seeded)
     end
 
-    def compute_seed_digest(messages)
+    # A seed's drift digest, its parts' cache flags aside (CacheMarkers);
+    # with them, as Smith before 0.12.3 took it.
+    def compute_seed_digest(messages, cache_flags: false)
       return nil if messages.nil? || messages.empty?
 
-      Digest::SHA256.hexdigest(JSON.generate(messages))
+      Digest::SHA256.hexdigest(JSON.generate(cache_flags ? messages : CacheMarkers.strip_parts(messages)))
     rescue JSON::GeneratorError, Encoding::UndefinedConversionError, Encoding::InvalidByteSequenceError => e
       raise WorkflowError,
             "seed_messages content must be valid UTF-8 (Smith hashes via JSON.generate " \

data/lib/smith.rb

@@ -212,6 +212,7 @@ require_relative "smith/tool_failure_notification_failed"
 require_relative "smith/persistence_payload_conflict"
 require_relative "smith/exponential_backoff"
 require_relative "smith/interruptions"
+require_relative "smith/billing_refusals"
 
 # Event system (depends on Types)
 require_relative "smith/event"
@@ -227,6 +228,7 @@ require_relative "smith/budget/ledger"
 require_relative "smith/budget/host_ledger"
 require_relative "smith/budget/request_hold"
 require_relative "smith/budget/continuation"
+require_relative "smith/cache_markers"
 require_relative "smith/response_memo"
 
 # Pricing (depends on Smith config)

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: smith-agents
 version: !ruby/object:Gem::Version
-  version: 0.12.2
+  version: 0.12.4
 platform: ruby
 authors:
 - Samuel Ralak
@@ -203,6 +203,7 @@ files:
 - lib/smith/attribution.rb
 - lib/smith/batch.rb
 - lib/smith/batch/message_shape.rb
+- lib/smith/billing_refusals.rb
 - lib/smith/budget.rb
 - lib/smith/budget/amount.rb
 - lib/smith/budget/amount_contract.rb
@@ -219,6 +220,7 @@ files:
 - lib/smith/budget/reservation.rb
 - lib/smith/budget/reservation_contract.rb
 - lib/smith/budget/spent_reservation.rb
+- lib/smith/cache_markers.rb
 - lib/smith/child_run.rb
 - lib/smith/child_runs_failed.rb
 - lib/smith/cli.rb