ruby_llm-contract 1.1.2 → 1.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 7ae19c7459f4999409748c604e8b80aa756e5b2333863c8cc16b4070bf57dff9
-  data.tar.gz: 3be220b3be5c0c313f4b3f5160da5416dc9be8c35629c0012d756e0d027aa646
+  metadata.gz: fead48c5516f8e0aa738d3ab1633ca2d07f7173aba6466a6d29e37634263b857
+  data.tar.gz: 4ae4c314cd6365c5f5de455f9dc5ddc0741abd26fa751804b22c5ffded0e6cb1
 SHA512:
-  metadata.gz: 16bc0227706b8e1cc16f5d93c7771f82e7835c40c20888cec9163a9537d30f80a86f761c47fd10bd778df8f7023ce041bd0d574248950e11bdad5dd3ac4860da
-  data.tar.gz: 4d4374cc2d65aa555f30656c2475bae261b0af1be0656b7449c22c38467b7f1e32f7ba159d215adc17254d0c624f7144f91087644f578b5cb2727fc8f9087c91
+  metadata.gz: 9b66df1e490e238fdacc84a728b874a008b80c8b44cdd882b444a84389bb77996ee6132efb6adaad79c91a04bee30eb7351241a4989ff1a4ed5df8d7a6cff619
+  data.tar.gz: a055fd3d06ee4fa9591fa2a6cfed6872142febd950fe3a455daa6e588f04e157a585701456f4815483ecc3accb803cf3bbcffa9308c6dfa5011c00f6a7143274

data/CHANGELOG.md

@@ -1,5 +1,52 @@
 # Changelog
 
+## 1.2.0 (2026-10-10)
+
+Everything new is opt-in, except the temperature fix below.
+
+### Added
+
+- **`token_count :exact`** on a step: `max_input` and `max_cost` measure the input with
+  the provider's count (RubyLLM's `chat.count_tokens`: OpenAI Responses, Anthropic,
+  Gemini, Vertex AI, Bedrock Converse) instead of the chars/4 heuristic, attachments
+  included, so `attachment_token_estimate` is not needed. One extra request per attempt.
+  A provider without the endpoint, a failed count or an adapter without `count_tokens`
+  refuses the call (`:limit_exceeded`) rather than falling back to the heuristic; the
+  `Test` adapter counts with the heuristic. What a provider counts beyond prompt and
+  attachments is its own (Gemini's count leaves the output schema out).
+  `Adapters::RubyLLM#count_tokens`.
+- **`on_incomplete_output :refuse`** on a step: a response the provider stopped at a token
+  limit fails as `:output_truncated`, one stopped by a content filter as
+  `:content_filtered`, before validation, so it is never `:ok`. Both statuses are outside
+  the default `retry_on`. The default, `:accept`, keeps 1.1.2's behaviour.
+- **`retry_on` takes exception classes**: `retry_on :validation_failed,
+  RubyLLM::RateLimitError` retries an adapter error only of that class or a subclass;
+  `:adapter_error` still retries any. An adapter error's trace (and attempt) has
+  `error_class`.
+- **`provider_options`** on a step (inherited, `:default` resets) and per call
+  (`context: { provider_options: }`, merged over the step's), passed to RubyLLM's
+  `with_provider_options`. Keys the step controls raise `ArgumentError`.
+- **`workflow_instrumentation`** in `RubyLLM::Contract.configure`: a pipeline runs as one
+  `RubyLLM.workflow` with a `workflow.step` per alias, a step on its own as a workflow
+  named after its class, so RubyLLM's events and OpenTelemetry spans carry the step.
+- **`register_model` takes cache prices**: `cache_read_per_1m:` and `cache_write_per_1m:`,
+  both optional.
+- Guides: retry by error class, cut-off responses, exact token counts, local and custom
+  models (Ollama), provider options, workflow instrumentation, and `RubyLLM::Judge` (2.1)
+  in the LLM-judge guide. The README lists the new options and answers how to run local
+  models.
+
+### Changed
+
+- **A temperature is no longer sent to a model RubyLLM's registry marks as taking none**
+  (OpenAI's o-series and gpt-5 family). RubyLLM 1.x quietly sent 1.0 to those models;
+  2.x sends the value as given, so a step with `temperature` that escalated to one failed
+  with an adapter error. The adapter now leaves it out and warns once per model; a model
+  the registry does not know still gets it.
+- **`retry_on` raises on a condition it cannot match.** A status given as a String (or
+  any value that is neither a Symbol nor an exception class) never matched a result, so
+  it silently turned retries off; it now raises `ArgumentError` when the step is defined.
+
 ## 1.1.2 (2026-10-10)
 
 ### Changed - costs and gates may move

data/README.md

@@ -131,6 +131,10 @@ Everything below is optional — the example above is a complete step. Reach for
 - **[A/B test prompts](docs/guide/eval_first.md)** — measure whether a new prompt is safe to ship before merging.
 - **[Budget caps](docs/guide/getting_started.md)** — refuse the request pre-flight when an estimate exceeds the limit.
 - **[Cost tracking](docs/guide/getting_started.md#what-a-traces-usage-and-cost-count)** - per-call cost with prompt-cache and per-provider prices; eval cost gates fail closed when a cost is unknown.
+- **[Exact token counts](docs/guide/getting_started.md#counting-input-tokens-exactly)** - `token_count :exact` checks `max_input` / `max_cost` against the provider's own count instead of a heuristic.
+- **[Cut-off responses](docs/guide/getting_started.md#responses-cut-off-by-a-token-limit)** - `on_incomplete_output :refuse` fails an answer the provider stopped at a token limit, even one that would validate.
+- **[Retry by error class](docs/guide/getting_started.md#retrying-provider-errors-by-class)** - `retry_on RubyLLM::RateLimitError` retries a rate limit without retrying an exhausted balance.
+- **[Provider options](docs/guide/getting_started.md#provider-options)** - send `service_tier`, `seed` and other provider settings from the step or per call.
 - **[Reasoning effort / thinking config](docs/guide/optimizing_retry_policy.md)** — Anthropic / OpenAI thinking configuration on the Step class.
 
 Also supports [multi-step pipelines](docs/guide/pipeline.md) with fail-fast and per-step models.
@@ -194,6 +198,8 @@ what 1.0.0 itself was.
 
 **Costs went up, or an eval cost gate turned red, after upgrading to 1.1.2?** Earlier versions left prompt-cache tokens out of `input_tokens` and the cost, priced every model at its default provider, counted a missing price as $0, and treated a call without token counts as free. The [CHANGELOG](CHANGELOG.md) lists each change; `on_unknown_pricing: :warn` turns the unknown-cost refusal into a warning.
 
+**Local models (Ollama)?** Point RubyLLM at the server (`c.ollama_api_base = "http://localhost:11434/v1"`) and run steps with `context: { provider: :ollama, model: "...", assume_model_exists: true }`. Local models have no price in RubyLLM's registry, so register one (`CostCalculator.register_model("gemma-fast:latest", input_per_1m: 0, output_per_1m: 0)`) or the eval cost gates treat their cost as unknown. Ollama has no token-counting endpoint, so leave `token_count` at `:estimate`. See [local and custom models](docs/guide/getting_started.md#local-and-custom-models).
+
 **Upgraded from pre-0.10.0 and getting `:limit_exceeded` with attachments?** Multimodal contracts with `max_cost`/`max_input` need `attachment_token_estimate`. See [multimodal input guide](docs/guide/multimodal_input.md#cost-attachment_token_estimate-is-required) for setup, fail-closed behaviour, and `on_unknown_attachment_size :warn` opt-out.
 
 ## License

data/docs/guide/getting_started.md

@@ -65,6 +65,32 @@ result.trace[:attempts]
 
 If the whole chain exhausts, `result.status` is the status of the last attempt (`:validation_failed` or `:parse_error`) and `result.parsed_output` is the last attempt's output. The caller decides what to do — ship it anyway, fall back to a template, or raise.
 
+### Retrying provider errors by class
+
+A provider error that RubyLLM's own retries could not get past ends as `:adapter_error`, with the exception class in `trace[:error_class]` (e.g. `"RubyLLM::RateLimitError"`). Adapter errors are not retried by default. List exception classes in `retry_on` to retry only those, typically together with `escalate` so the next attempt goes to another model:
+
+```ruby
+retry_policy do
+  escalate "gpt-4.1-mini", "claude-haiku-4-5"
+  retry_on :validation_failed, :parse_error, RubyLLM::RateLimitError, RubyLLM::OverloadedError
+end
+```
+
+A class matches its subclasses; `:adapter_error` in the list retries every adapter error. As with statuses, the list replaces the defaults. Switching to a fallback model on transport errors inside one call is RubyLLM's `chat.with_fallbacks`; the contract's adapter does not call it, so fallbacks across models belong in `escalate`.
+
+### Responses cut off by a token limit
+
+When a provider stops at a token limit or a content filter, `trace[:finish_reason]` says so (`:max_tokens`, `:content_filter`), and a failed result gets a last validation error naming it. A cut-off answer that still parses and validates stays `:ok` unless the step refuses it:
+
+```ruby
+class SummarizeArticle < RubyLLM::Contract::Step::Base
+  max_output 400
+  on_incomplete_output :refuse   # default :accept
+end
+```
+
+With `:refuse`, such a response fails before validation as `:output_truncated` (token limit) or `:content_filtered`, so `validate` blocks and observers never see it. Raw output, usage and cost stay on the result. Neither status is retried by default; `retry_on :output_truncated` opts in (useful when a later attempt has a larger `max_output`). Anthropic reports an exhausted context window as a token-limit stop too, so `:output_truncated` means "stopped on tokens", not necessarily "hit max_output".
+
 ### Per-attempt reasoning effort
 
 `models:` accepts config hashes as well as model-name strings, so a fallback can "try harder" (more reasoning) on retry, not just switch model:
@@ -191,6 +217,31 @@ When a cost does not cover the whole call, `result.trace.cost_unknown?` is true
 
 A custom adapter returning `Response.new(content:, usage:)` is priced from `usage` as before. To report its own cost, pass `cost:` - `nil` means unknown - with `cost_complete:` and `usage_complete:`.
 
+### Counting input tokens exactly
+
+`max_input` and `max_cost` measure the input with a chars/4 heuristic (±30%). With `token_count :exact` they ask the provider instead:
+
+```ruby
+class SummarizeArticle < RubyLLM::Contract::Step::Base
+  token_count :exact   # default :estimate
+  max_input 20_000
+end
+```
+
+The count comes from RubyLLM's `chat.count_tokens` (OpenAI Responses, Anthropic, Gemini, Vertex AI, Bedrock Converse) and covers the prompt and attachments, so `attachment_token_estimate` is not needed. What else a provider counts is its own: OpenAI's and Anthropic's counts include the output schema, Gemini's leaves it out. It costs one extra request per attempt, and RubyLLM leaves `provider_options` out of it. Where no count can be had - a provider without the endpoint (Ollama, for one), a failed request, or an adapter without `count_tokens` - the call is refused as `:limit_exceeded` with the reason, never measured with the heuristic under the name "exact". The `Test` adapter counts with the heuristic, since it has no provider to ask. The output side of `max_cost` is still an estimate (`max_output`, or the input size without one).
+
+### Local and custom models
+
+A local model (Ollama) or a fine-tuned one has no price in RubyLLM's registry, so its cost is unknown and the eval cost gates refuse. Register a price - zero for a model that costs nothing per token:
+
+```ruby
+RubyLLM::Contract::CostCalculator.register_model("gemma-fast:latest", input_per_1m: 0, output_per_1m: 0)
+RubyLLM::Contract::CostCalculator.register_model("ft:gpt-4o-custom",
+  input_per_1m: 3.0, output_per_1m: 6.0, cache_read_per_1m: 1.5, cache_write_per_1m: 3.75)
+```
+
+`register_model` sets a price for a model id, not a model RubyLLM can call: the step still needs `provider:` (and `assume_model_exists: true` for an id RubyLLM does not list). Cache prices are optional; without them cache reads are charged at the input price and a call that wrote to the cache has no price. A price does not make up for missing counts: a provider that sends no token counts still leaves the cost unknown.
+
 ### Preflight cost estimates
 
 Check what a call is likely to cost before invoking it:
@@ -211,6 +262,24 @@ SummarizeArticle.estimate_eval_cost("regression",
 
 `estimate_cost` returns `nil` when pricing isn't registered. `estimate_eval_cost` silently treats unknown-pricing cases as `$0.00` and sums the rest — it does **not** fail closed the way `max_cost` does. Treat its output as a floor, not a guarantee; register pricing via `CostCalculator.register_model` before relying on it for budget decisions.
 
+## Provider options
+
+Settings RubyLLM has no method for go to the provider as-is through `provider_options`, on the step or per call:
+
+```ruby
+class SummarizeArticle < RubyLLM::Contract::Step::Base
+  provider_options service_tier: "flex"
+end
+
+SummarizeArticle.run(text, context: { provider_options: { service_tier: "priority", seed: 7 } })
+```
+
+The call's keys win over the step's; a subclass replaces its parent's hash, and `provider_options :default` stops inheriting it. Keys the step itself sets (`model`, `temperature`, the max-token keys, the messages, and on a step with `output_schema` the structured-output keys `response_format` and `text`) raise `ArgumentError`, so a call cannot differ from what its limits and schema were checked against. Only top-level keys are checked. `compare_models` and `optimize_retry_policy` candidates do not carry provider options.
+
+## Temperature on reasoning models
+
+RubyLLM 1.x sent temperature 1.0 to OpenAI reasoning models (o-series, gpt-5), which reject other values; RubyLLM 2.x sends what it is given. A step with `temperature` therefore leaves it out for a model RubyLLM's registry marks as taking none, and warns once per model - so `temperature 0` with `escalate "gpt-4.1-mini", "gpt-5-mini"` still works. A model the registry does not know gets the temperature as set.
+
 ## `output_schema` vs `with_schema`
 
 `with_schema` in `ruby_llm` tells the provider to force a specific JSON structure. `output_schema` in this gem does the same thing (calls `with_schema` under the hood) **plus** validates the response client-side. Cheaper models sometimes ignore schema constraints — `with_schema` is a request; `output_schema` is a request plus verification.

data/docs/guide/llm_judge.md

@@ -195,6 +195,20 @@ What to do operationally:
 
 The drop is your signal to refine the judge's prompt, not to lower the gate.
 
+## `RubyLLM::Judge` (RubyLLM 2.1)
+
+RubyLLM 2.1 adds `RubyLLM::Judge` for typed questions - `probability`, `choice` and `score` - answered by **decision models**: OpenAI's judgment models, TypeSafe, or local decision models through Ollama. It does not run on ordinary chat models; those raise that they do not support judgments. Where you have such a model, a judge class can stand in for the judge Step in step 1 above:
+
+```ruby
+class SummaryFaithfulness < RubyLLM::Judge
+  probability :faithful, "Is every claim in the summary supported by the article?"
+end
+
+SummaryFaithfulness.judge(article_and_summary).faithful.probability   # => 0.93
+```
+
+Calibrate it against human labels exactly as in step 2; the threshold is still yours to set. On an ordinary chat model, keep the judge as a Contract Step - the pattern this guide builds.
+
 ## When to reach for Tribunal instead
 
 [`ruby_llm-tribunal`](https://github.com/Alqemist-labs/ruby_llm-tribunal) ships an off-the-shelf catalog of common LLM-as-judge assertions (`assert_faithful`, `assert_hallucination`, `assert_refusal`, `assert_no_pii`, etc.) — a shortcut when your check matches one of those domain-general categories. The methodology in this guide still applies: calibrate the judge against your human-labeled production data **before** trusting Tribunal's `default_threshold = 0.8`, refine the prompt when it over-flags, watch for the anti-patterns above. See [Relation to Tribunal](relation_to_tribunal.md) for the full positioning — what each gem documents (and doesn't), a concrete decision tree on catalog-vs-custom-judge, and three working integration patterns.

data/docs/guide/rails_integration.md

@@ -125,6 +125,14 @@ end
 
 Trace inspection in an admin UI: `result.trace[:attempts]` gives you per-attempt model, status, cost, latency — render it in a partial to debug production failures without re-running.
 
+RubyLLM 2.1 emits its own events and OpenTelemetry spans for every call. To group them by step, turn on workflow instrumentation in the initializer:
+
+```ruby
+RubyLLM::Contract.configure { |c| c.workflow_instrumentation = true }
+```
+
+A pipeline then runs as one `RubyLLM.workflow` named after its class, with each step as `workflow.step(alias)`; a step run on its own is a workflow named after its class. RubyLLM's events carry `workflow_name` and `workflow_step_name`, and a workflow your code opened around the call becomes the parent. Off by default. In a Rails app RubyLLM also records each call in its usage ledger when configured; that figure and `trace[:cost]` are the same RubyLLM price, unless `CostCalculator.register_model` overrides it for the model.
+
 ## 5. Testing — RSpec and Minitest
 
 Add to `spec/spec_helper.rb` (or `test_helper.rb`):

data/lib/ruby_llm/contract/adapters/ruby_llm.rb

@@ -6,29 +6,59 @@ module RubyLLM
   module Contract
     module Adapters
       class RubyLLM < Base
+        # `with: nil` adds no attachment. The text is never nil (`fetch(:content, "")`),
+        # and RubyLLM raises only when both text and attachments are nil.
         def call(messages:, **options)
-          system_contents, conversation = partition_messages(messages)
-          conversation = fallback_conversation(system_contents, conversation)
-
-          chat = build_chat(options, system_contents)
-          add_history(chat, conversation[0..-2])
-
-          # `with: nil` adds no attachment. The text is never nil (`fetch(:content, "")`),
-          # and RubyLLM raises only when both text and attachments are nil.
-          response = chat.ask(
-            conversation.last&.fetch(:content, ""),
-            with: options[:attachment]
-          )
+          chat, text = prepared_chat(messages, options)
+          response = chat.ask(text, with: options[:attachment])
           build_response(response, options[:model])
         end
 
+        # Input tokens of the request `call` would send, counted by the
+        # provider (`chat.count_tokens`), attachments included. `ask` is
+        # `ask_later` plus `complete` in RubyLLM, so the staged message is the
+        # one `call` sends. RubyLLM leaves provider_options out of the count.
+        # Raises RubyLLM::Error where the provider has no counting endpoint.
+        def count_tokens(messages:, **options)
+          chat, text = prepared_chat(messages, options)
+          chat.ask_later(text, with: options[:attachment]).count_tokens
+        rescue KeyError => e
+          # RubyLLM reads OpenAI's count with `fetch`; a reply without the
+          # field is a failed count, not a bug in the caller.
+          raise ::RubyLLM::Error, "count response without input_tokens (#{e.message})"
+        end
+
         CHAT_OPTION_METHODS = {
-          temperature: :with_temperature,
           schema: :with_schema
         }.freeze
 
+        @dropped_temperature_models = Set.new
+        @dropped_temperature_lock = Mutex.new
+
+        # Warns once per provider and model, across threads and subclasses
+        # (called on this class, whose state subclasses do not inherit).
+        def self.warn_dropped_temperature(model)
+          key = "#{model.provider}:#{model.id}"
+          first = @dropped_temperature_lock.synchronize { @dropped_temperature_models.add?(key) }
+          return unless first
+
+          warn "[ruby_llm-contract] #{model.id} (#{model.provider}) does not accept temperature " \
+               "according to RubyLLM's model registry; the step's temperature is not sent"
+        end
+
         private
 
+        # The configured chat with the history added, and the text of the last
+        # message, which sending and counting both stage themselves.
+        def prepared_chat(messages, options)
+          system_contents, conversation = partition_messages(messages)
+          conversation = fallback_conversation(system_contents, conversation)
+
+          chat = build_chat(options, system_contents)
+          add_history(chat, conversation[0..-2])
+          [chat, conversation.last&.fetch(:content, "")]
+        end
+
         # When prompt has only system/section/rule nodes and no user message,
         # pop the last system message and use it as the user ask.
         def fallback_conversation(system_contents, conversation)
@@ -56,6 +86,7 @@ module RubyLLM
           CHAT_OPTION_METHODS.each do |key, method_name|
             chat.public_send(method_name, options[key]) if options[key]
           end
+          apply_temperature(chat, options[:temperature]) unless options[:temperature].nil?
 
           # Resolve thinking config from BOTH sources, with `:reasoning_effort`
           # taking precedence over `:thinking[:effort]`. This is the per-attempt
@@ -69,6 +100,29 @@ module RubyLLM
           # replacement for this one passthrough. `reasoning_effort` is not
           # forwarded here — it goes through `with_thinking` above.
           chat.with_max_output_tokens(options[:max_tokens]) if options[:max_tokens]
+          apply_provider_options(chat, options)
+        end
+
+        def apply_provider_options(chat, options)
+          provider_options = options[:provider_options]
+          return if provider_options.nil? || provider_options.empty?
+
+          ProviderOptions.validate!(provider_options, schema: !options[:schema].nil?)
+          chat.with_provider_options(provider_options)
+        end
+
+        # RubyLLM 1.x set temperature to 1.0 for OpenAI reasoning models, which
+        # reject any other value; 2.x sends it as given. A step escalating from
+        # a sampling model to one of those would fail, so a temperature is left
+        # out where the registry says the model takes none. A model RubyLLM does
+        # not know (assume_model_exists) gets it as before.
+        def apply_temperature(chat, temperature)
+          model = chat.respond_to?(:model) ? chat.model : nil
+          if model.respond_to?(:metadata) && model.metadata.is_a?(Hash) && model.metadata[:temperature] == false
+            Adapters::RubyLLM.warn_dropped_temperature(model)
+          else
+            chat.with_temperature(temperature)
+          end
         end
 
         # Returns merged `{ effort:, budget: }` or nil. `options[:reasoning_effort]`

data/lib/ruby_llm/contract/adapters/test.rb

@@ -43,6 +43,12 @@ module RubyLLM
           end
         end
 
+        # There is no provider to ask, so `token_count :exact` steps measure
+        # their input with the same heuristic as `:estimate` under this adapter.
+        def count_tokens(messages:, **_options)
+          TokenEstimator.estimate(messages)
+        end
+
         def call(messages:, **_options) # rubocop:disable Lint/UnusedMethodArgument
           content = if @responses
                       c = @responses[@index] || @responses.last

data/lib/ruby_llm/contract/configuration.rb

@@ -9,13 +9,17 @@ module RubyLLM
     #
     # Then configure contract-specific options:
     #   RubyLLM::Contract.configure { |c| c.default_model = "gpt-4.1-mini" }
+    #
+    # `workflow_instrumentation = true` wraps steps and pipelines in
+    # `RubyLLM.workflow` (see WorkflowScope).
     class Configuration
-      attr_accessor :default_adapter, :default_model, :logger
+      attr_accessor :default_adapter, :default_model, :logger, :workflow_instrumentation
 
       def initialize
         @default_adapter = nil
         @default_model = nil
         @logger = nil
+        @workflow_instrumentation = false
       end
     end
   end

data/lib/ruby_llm/contract/cost_calculator.rb

@@ -29,7 +29,9 @@ module RubyLLM
     # which calls `calculate` per attempt and sums; not in this module).
     module CostCalculator
       # Simple struct for custom-registered model pricing
-      RegisteredModel = Struct.new(:input_price_per_million, :output_price_per_million, keyword_init: true)
+      RegisteredModel = Struct.new(:input_price_per_million, :output_price_per_million,
+                                   :cache_read_price_per_million, :cache_write_price_per_million,
+                                   keyword_init: true)
 
       @custom_models = {}
 
@@ -40,13 +42,22 @@ module RubyLLM
       #   CostCalculator.register_model("ft:gpt-4o-custom",
       #     input_per_1m: 3.0, output_per_1m: 6.0)
       #
-      def self.register_model(model_name, input_per_1m:, output_per_1m:)
+      # Prompt-cache prices are optional: without `cache_read_per_1m` cache
+      # reads are charged at the input price; without `cache_write_per_1m` a
+      # call that wrote to the cache has no price (writes can cost more than
+      # input).
+      def self.register_model(model_name, input_per_1m:, output_per_1m:, cache_read_per_1m: nil,
+                              cache_write_per_1m: nil)
         validate_price!(:input_per_1m, input_per_1m)
         validate_price!(:output_per_1m, output_per_1m)
+        validate_price!(:cache_read_per_1m, cache_read_per_1m) unless cache_read_per_1m.nil?
+        validate_price!(:cache_write_per_1m, cache_write_per_1m) unless cache_write_per_1m.nil?
 
         @custom_models[model_name] = RegisteredModel.new(
           input_price_per_million: input_per_1m,
-          output_price_per_million: output_per_1m
+          output_price_per_million: output_per_1m,
+          cache_read_price_per_million: cache_read_per_1m,
+          cache_write_price_per_million: cache_write_per_1m
         )
       end
 
@@ -113,18 +124,32 @@ module RubyLLM
         end
       end
 
-      # Our registry knows input and output prices only. Cache reads are
+      # input_tokens includes the cache reads and writes, so each part is
+      # charged once, at its own price. Without a cache-read price, reads are
       # charged at the input price, which assumes they cost no more than
       # uncached input (true of every provider in RubyLLM's registry, not
       # guaranteed for a custom one). Cache writes can cost more than input
-      # (Anthropic charges 1.25x-2x), so a call that wrote to the cache has
-      # no price here rather than an undercount.
+      # (Anthropic charges 1.25x-2x), so without a cache-write price a call
+      # that wrote to the cache has no price rather than an undercount.
       def self.registered_cost(model_info, usage)
-        return nil if (usage[:cache_write_tokens] || 0).positive?
+        cache_read = usage[:cache_read_tokens] || 0
+        cache_write = usage[:cache_write_tokens] || 0
+        read_price, write_price = registered_cache_prices(model_info)
+        return nil if cache_write.positive? && write_price.nil?
+
+        uncached = [(usage[:input_tokens] || 0) - cache_read - cache_write, 0].max
+        (token_cost(uncached, model_info.input_price_per_million) +
+          token_cost(cache_read, read_price) +
+          token_cost(cache_write, write_price.to_f) +
+          token_cost(usage[:output_tokens], model_info.output_price_per_million)).round(6)
+      end
 
-        input_cost = token_cost(usage[:input_tokens], model_info.input_price_per_million)
-        output_cost = token_cost(usage[:output_tokens], model_info.output_price_per_million)
-        (input_cost + output_cost).round(6)
+      # [read price, write price]; a model without a read price reads at its
+      # input price. Duck-typed registry entries may lack both readers.
+      def self.registered_cache_prices(model_info)
+        read = model_info.cache_read_price_per_million if model_info.respond_to?(:cache_read_price_per_million)
+        write = model_info.cache_write_price_per_million if model_info.respond_to?(:cache_write_price_per_million)
+        [read || model_info.input_price_per_million, write]
       end
 
       # RubyLLM counts uncached input apart from cache reads and writes; our
@@ -174,7 +199,8 @@ module RubyLLM
       # to inspect model pricing before invoking `calculate` (e.g., to short-
       # circuit estimate when the model is unknown). Exposing it removes
       # `CostCalculator.send(:find_model)` workarounds at call sites.
-      private_class_method :compute_cost, :registered_cost, :tokens_for, :token_cost, :validate_price!
+      private_class_method :compute_cost, :registered_cost, :registered_cache_prices, :tokens_for, :token_cost,
+                           :validate_price!
     end
   end
 end

data/lib/ruby_llm/contract/pipeline/base.rb

@@ -48,7 +48,9 @@ module RubyLLM
           end
 
           def run(input, context: {}, timeout_ms: nil)
-            Runner.new(steps: steps, context: context, timeout_ms: timeout_ms, token_budget: token_budget).call(input)
+            WorkflowScope.workflow(name || "anonymous pipeline") do
+              Runner.new(steps: steps, context: context, timeout_ms: timeout_ms, token_budget: token_budget).call(input)
+            end
           end
 
           def test(input, responses: {}, timeout_ms: nil)

data/lib/ruby_llm/contract/pipeline/runner.rb

@@ -37,7 +37,9 @@ module RubyLLM
 
         def execute_step(step_def, execution)
           step_context = build_step_context(step_def)
-          result = step_def[:step_class].run(execution.current_input, context: step_context)
+          result = WorkflowScope.step(step_def[:alias]) do
+            step_def[:step_class].run(execution.current_input, context: step_context)
+          end
 
           execution.record_step(step_def[:alias], result)
         end

data/lib/ruby_llm/contract/provider_options.rb

@@ -0,0 +1,45 @@
+# frozen_string_literal: true
+
+module RubyLLM
+  module Contract
+    # Options in the provider's own request vocabulary, passed to
+    # `chat.with_provider_options` and merged into the request as-is. Keys the
+    # contract itself sets are refused: an override would make the call differ
+    # from what max_cost, max_output and the schema were checked against.
+    # Only top-level keys are checked; a provider that nests these settings
+    # (Gemini's generationConfig, for one) is not.
+    module ProviderOptions
+      RESERVED_KEYS = %w[model messages input instructions system temperature stream
+                         max_tokens max_output_tokens max_completion_tokens].freeze
+      # Where OpenAI's protocols put the structured-output schema.
+      SCHEMA_KEYS = %w[response_format text].freeze
+
+      def self.validate!(options, schema: false)
+        raise ArgumentError, "provider_options must be a Hash, got #{options.class}" unless options.is_a?(Hash)
+
+        reserved = schema ? RESERVED_KEYS + SCHEMA_KEYS : RESERVED_KEYS
+        clashes = options.keys.map(&:to_s) & reserved
+        return options if clashes.empty?
+
+        raise ArgumentError, "provider_options cannot set #{clashes.join(", ")}: the step sets " \
+                             "#{clashes.one? ? "it" : "them"} (use the step DSL or context instead)"
+      end
+
+      # A frozen deep copy, so a hash the caller keeps mutating cannot change
+      # a step definition shared across threads.
+      def self.frozen_copy(object)
+        case object
+        when Hash then object.to_h { |key, value| [key, frozen_copy(value)] }.freeze
+        when Array then object.map { |value| frozen_copy(value) }.freeze
+        else object.frozen? ? object : object.dup.freeze
+        end
+      end
+
+      # Context keys win over the class-level ones; nil when nothing is set.
+      def self.merge(class_level, context_level)
+        merged = (class_level || {}).merge(context_level || {})
+        merged.empty? ? nil : merged
+      end
+    end
+  end
+end

data/lib/ruby_llm/contract/step/adapter_caller.rb

@@ -30,7 +30,8 @@ module RubyLLM
             status: :adapter_error,
             raw_output: nil,
             parsed_output: nil,
-            validation_errors: [e.message]
+            validation_errors: [e.message],
+            trace: Trace.new(error_class: e.class.name, error_type: e.class)
           )
           [result, 0]
         end

data/lib/ruby_llm/contract/step/base.rb

@@ -93,7 +93,8 @@ module RubyLLM
 
           # Forwarded to the adapter as-is. A key known but not forwarded would be
           # silently dropped, so the known list is built from this one.
-          ADAPTER_CONTEXT_KEYS = %i[provider assume_model_exists max_tokens reasoning_effort attachment].freeze
+          ADAPTER_CONTEXT_KEYS = %i[provider assume_model_exists max_tokens reasoning_effort attachment
+                                    provider_options].freeze
           KNOWN_CONTEXT_KEYS = (%i[adapter model temperature retry_policy_override] + ADAPTER_CONTEXT_KEYS).freeze
 
           include Concerns::ContextHelpers
@@ -102,7 +103,7 @@ module RubyLLM
             context = safe_context(context)
             warn_unknown_context_keys(context)
 
-            result = dispatch_run(input, context)
+            result = WorkflowScope.workflow(name || "anonymous step") { dispatch_run(input, context) }
             log_result(result)
             invoke_around_call(input, result)
           end
@@ -196,7 +197,7 @@ module RubyLLM
 
           def runtime_settings(context)
             policy = context.key?(:retry_policy_override) ? context[:retry_policy_override] : retry_policy
-            extra = context.slice(*ADAPTER_CONTEXT_KEYS)
+            extra = context.slice(*ADAPTER_CONTEXT_KEYS).merge(merged_provider_options(context))
 
             # Always pass the class-level `thinking` config to the adapter when
             # set, so fields like `budget` survive a per-call `reasoning_effort`
@@ -221,6 +222,13 @@ module RubyLLM
             }
           end
 
+          # Class-level provider_options with the call's merged over them; a
+          # one-key hash so `merge` drops it entirely when neither is set.
+          def merged_provider_options(context)
+            merged = ProviderOptions.merge(provider_options, context[:provider_options])
+            merged ? { provider_options: merged } : {}
+          end
+
           def current_model_config
             policy = retry_policy
             if policy && policy.config_list.any?
@@ -275,7 +283,9 @@ module RubyLLM
               on_unknown_attachment_size: on_unknown_attachment_size,
               temperature: context_temperature || temperature,
               extra_options: extra_options,
-              observers: class_observers
+              observers: class_observers,
+              on_incomplete_output: on_incomplete_output,
+              token_count: token_count
             )
           end
 

data/lib/ruby_llm/contract/step/dsl.rb

@@ -185,6 +185,46 @@ module RubyLLM
           inherited_value(:on_unknown_attachment_size) || UnknownPolicy::DEFAULT
         end
 
+        TOKEN_COUNT_MODES = %i[estimate exact].freeze
+        TOKEN_COUNT_DEFAULT = :estimate
+
+        # How max_input and max_cost measure the input before a call.
+        # `:estimate` (default) is the local chars/4 heuristic. `:exact` asks
+        # the provider (`chat.count_tokens`): one extra request per attempt,
+        # attachments included, no attachment_token_estimate needed. A provider
+        # or adapter that cannot count refuses the call (:limit_exceeded).
+        def token_count(mode = nil)
+          if mode
+            unless TOKEN_COUNT_MODES.include?(mode)
+              raise ArgumentError, "token_count must be one of #{TOKEN_COUNT_MODES.inspect}, got #{mode.inspect}"
+            end
+
+            return @token_count = mode
+          end
+
+          inherited_value(:token_count) || TOKEN_COUNT_DEFAULT
+        end
+
+        INCOMPLETE_OUTPUT_MODES = %i[accept refuse].freeze
+        INCOMPLETE_OUTPUT_DEFAULT = :accept
+
+        # `:refuse` fails a response the provider cut off at a token limit
+        # (:output_truncated) or stopped with a content filter
+        # (:content_filtered), before validation, so it is never :ok.
+        # `:accept` (default) validates it like any other response.
+        def on_incomplete_output(mode = nil)
+          if mode
+            unless INCOMPLETE_OUTPUT_MODES.include?(mode)
+              raise ArgumentError, "on_incomplete_output must be one of #{INCOMPLETE_OUTPUT_MODES.inspect}, " \
+                                   "got #{mode.inspect}"
+            end
+
+            return @on_incomplete_output = mode
+          end
+
+          inherited_value(:on_incomplete_output) || INCOMPLETE_OUTPUT_DEFAULT
+        end
+
         def model(name = nil)
           if name == :default
             @model = UNSET
@@ -215,6 +255,20 @@ module RubyLLM
           inherited_value_with_reset(:temperature)
         end
 
+        # Request options in the provider's vocabulary, e.g.
+        # `provider_options service_tier: "flex"`. Replaces an inherited hash;
+        # `provider_options :default` stops inheriting. `context: { provider_options: }`
+        # is merged over it per call.
+        def provider_options(options = nil)
+          if options == :default
+            @provider_options = UNSET
+            return nil
+          end
+          return @provider_options = ProviderOptions.frozen_copy(ProviderOptions.validate!(options)) if options
+
+          inherited_value_with_reset(:provider_options)
+        end
+
         def thinking(effort: nil, budget: nil)
           if effort == :default
             @thinking = UNSET

data/lib/ruby_llm/contract/step/limit_checker.rb

@@ -8,6 +8,7 @@ module RubyLLM
 
         def check_limits(messages)
           return nil unless max_input || max_cost
+          return check_counted_limits(messages) if exact_token_count?
 
           text_tokens = TokenEstimator.estimate(messages)
           attachment_tokens, attachment_error = resolve_attachment_tokens
@@ -23,6 +24,31 @@ module RubyLLM
           build_limit_result(messages, estimated, errors)
         end
 
+        # `token_count :exact`: the provider counts the input, attachments
+        # included. A count that cannot be had refuses the call rather than
+        # falling back to the heuristic under the name "exact".
+        def check_counted_limits(messages)
+          counted, count_error = count_input_tokens(messages)
+          return build_limit_result(messages, nil, [count_error], method: :exact) if count_error
+
+          errors = collect_limit_errors(counted, method: :exact)
+          errors.empty? ? nil : build_limit_result(messages, counted, errors, method: :exact)
+        end
+
+        def count_input_tokens(messages)
+          unless count_adapter.respond_to?(:count_tokens)
+            return [nil, "token_count :exact needs an adapter that counts tokens; " \
+                         "#{count_adapter.class} does not"]
+          end
+
+          counted = count_adapter.count_tokens(messages: messages, **count_options)
+          return [counted, nil] if counted.is_a?(Integer) && !counted.negative?
+
+          [nil, "token_count :exact: no usable token count (got #{counted.inspect})"]
+        rescue ::RubyLLM::Error, ::Faraday::Error => e
+          [nil, "token_count :exact: the provider could not count the input tokens (#{e.message})"]
+        end
+
         # Fail-closed: when an attachment is passed via context but no
         # attachment_token_estimate is declared, the gem cannot bound vision/
         # PDF cost. Refuses with a clear error unless on_unknown_attachment_size
@@ -50,12 +76,17 @@ module RubyLLM
           [estimate, nil]
         end
 
-        def collect_limit_errors(estimated)
+        def collect_limit_errors(estimated, method: :heuristic)
           errors = []
           if max_input && estimated > max_input
-            errors << "Input token limit exceeded: estimated #{estimated} tokens (heuristic ±30%), max #{max_input}"
+            measured = if method == :exact
+                         "#{estimated} tokens (counted by the provider)"
+                       else
+                         "estimated #{estimated} tokens (heuristic ±30%)"
+                       end
+            errors << "Input token limit exceeded: #{measured}, max #{max_input}"
           end
-          append_cost_error(estimated, errors) if max_cost
+          append_cost_error(estimated, errors, method) if max_cost
           errors
         end
 
@@ -66,7 +97,7 @@ module RubyLLM
         # for models expensive on completion side.
         DEFAULT_OUTPUT_RATIO = 1
 
-        def append_cost_error(estimated, errors)
+        def append_cost_error(estimated, errors, method = :heuristic)
           estimated_output = effective_max_output || (estimated * DEFAULT_OUTPUT_RATIO)
           estimated_cost = CostCalculator.calculate(
             model_name: model_name,
@@ -77,8 +108,12 @@ module RubyLLM
           if estimated_cost.nil?
             handle_unknown_pricing(errors)
           elsif estimated_cost > max_cost
-            errors << "Cost limit exceeded: estimated $#{format("%.6f", estimated_cost)} " \
-                      "(#{estimated} input + #{estimated_output} output tokens, heuristic ±30%), " \
+            tokens = if method == :exact
+                       "#{estimated} input tokens counted by the provider + #{estimated_output} output"
+                     else
+                       "#{estimated} input + #{estimated_output} output tokens, heuristic ±30%"
+                     end
+            errors << "Cost limit exceeded: estimated $#{format("%.6f", estimated_cost)} (#{tokens}), " \
                       "max $#{format("%.6f", max_cost)}"
           end
         end
@@ -94,17 +129,16 @@ module RubyLLM
           end
         end
 
-        def build_limit_result(messages, estimated, errors)
+        # No request was sent, so usage is zero; the measured input goes
+        # alongside, and is left out when it could not be measured.
+        def build_limit_result(messages, estimated, errors, method: :heuristic)
+          usage = { input_tokens: 0, output_tokens: 0, estimated_input_tokens: estimated, estimate_method: method }
           Result.new(
             status: :limit_exceeded,
             raw_output: nil,
             parsed_output: nil,
             validation_errors: errors,
-            trace: Trace.new(
-              messages: messages, model: model_name,
-              usage: { input_tokens: 0, output_tokens: 0, estimated_input_tokens: estimated,
-                       estimate_method: :heuristic }
-            )
+            trace: Trace.new(messages: messages, model: model_name, usage: usage.compact)
           )
         end
       end

data/lib/ruby_llm/contract/step/result_builder.rb

@@ -12,13 +12,18 @@ module RubyLLM
           content_filter: "provider reported content filtering or refusal"
         }.freeze
 
-        def initialize(contract_definition:, output_type:, output_schema:, model:, observers:, max_output: nil)
+        # Statuses for `on_incomplete_output :refuse`.
+        INCOMPLETE_STATUSES = { max_tokens: :output_truncated, content_filter: :content_filtered }.freeze
+
+        def initialize(contract_definition:, output_type:, output_schema:, model:, observers:, max_output: nil,
+                       on_incomplete_output: Dsl::INCOMPLETE_OUTPUT_DEFAULT)
           @contract_definition = contract_definition
           @output_type = output_type
           @output_schema = output_schema
           @model = model
           @observers = observers
           @max_output = max_output
+          @on_incomplete_output = on_incomplete_output
         end
 
         def error_result(error_result:, messages:)
@@ -27,15 +32,18 @@ module RubyLLM
             raw_output: error_result.raw_output,
             parsed_output: error_result.parsed_output,
             validation_errors: error_result.validation_errors,
-            trace: Trace.new(messages: messages, model: @model)
+            trace: Trace.new(messages: messages, model: @model, **error_fields(error_result.trace))
           )
         end
 
         def success_result(response:, messages:, latency_ms:, input:)
           raw_output = response.content
-          validation_result = validate_output(raw_output, input)
           trace = Trace.new(messages: messages, model: @model, latency_ms: latency_ms, usage: response.usage,
                             **response_accounting(response))
+          incomplete = incomplete_result(response, trace)
+          return incomplete if incomplete
+
+          validation_result = validate_output(raw_output, input)
 
           Result.new(
             status: validation_result[:status],
@@ -49,18 +57,43 @@ module RubyLLM
 
         private
 
+        def error_fields(trace)
+          return {} unless trace.respond_to?(:error_class) && trace.error_class
+
+          { error_class: trace.error_class, error_type: trace.error_type }
+        end
+
+        # Decided before validation, so neither validate blocks nor observers
+        # see a response the step refuses.
+        def incomplete_result(response, trace)
+          return unless @on_incomplete_output == :refuse
+
+          status = INCOMPLETE_STATUSES[finish_reason(response)]
+          return unless status
+
+          Result.new(status: status, raw_output: response.content, parsed_output: nil,
+                     validation_errors: [finish_reason_error(finish_reason(response))], trace: trace)
+        end
+
+        def finish_reason(response)
+          response.respond_to?(:finish_reason) ? response.finish_reason : nil
+        end
+
+        def finish_reason_error(reason)
+          message = FINISH_REASON_ERRORS[reason]
+          return message unless message && reason == :max_tokens && @max_output
+
+          "#{message}; configured max_output: #{@max_output}"
+        end
+
         # Only a failed result says why the provider stopped; a result that
         # passed validation keeps finish_reason in its trace alone.
         def validation_errors(validation_result, response)
           errors = validation_result[:errors]
           return errors if validation_result[:status] == :ok
 
-          reason = response.respond_to?(:finish_reason) ? response.finish_reason : nil
-          message = FINISH_REASON_ERRORS[reason]
-          return errors unless message
-
-          message += "; configured max_output: #{@max_output}" if reason == :max_tokens && @max_output
-          [*errors, message]
+          message = finish_reason_error(finish_reason(response))
+          message ? [*errors, message] : errors
         end
 
         # Custom adapters may return any object with `content` and `usage`.

data/lib/ruby_llm/contract/step/retry_executor.rb

@@ -7,7 +7,7 @@ module RubyLLM
         include Concerns::UsageAggregator
 
         # Copied from each attempt's trace into its attempts entry when present.
-        ATTEMPT_TRACE_FIELDS = %i[usage latency_ms cost finish_reason].freeze
+        ATTEMPT_TRACE_FIELDS = %i[usage latency_ms cost finish_reason error_class].freeze
 
         private
 

data/lib/ruby_llm/contract/step/retry_policy.rb

@@ -4,7 +4,7 @@ module RubyLLM
   module Contract
     module Step
       class RetryPolicy
-        attr_reader :max_attempts, :retryable_statuses
+        attr_reader :max_attempts, :retryable_statuses, :retryable_errors
 
         # Breaking (0.7.0): :adapter_error removed from defaults. ruby_llm's Faraday
         # middleware already retries transport errors (RateLimitError, ServerError,
@@ -18,6 +18,7 @@ module RubyLLM
         def initialize(models: nil, attempts: nil, retry_on: nil, &block)
           @configs = []
           @retryable_statuses = DEFAULT_RETRY_ON.dup
+          @retryable_errors = []
 
           if block
             @max_attempts = 1
@@ -49,12 +50,20 @@ module RubyLLM
           @configs
         end
 
-        def retry_on(*statuses)
-          @retryable_statuses = statuses.flatten
+        # Statuses, and exception classes for adapter errors:
+        # `retry_on :validation_failed, RubyLLM::RateLimitError` retries an
+        # adapter error only when it was a RateLimitError (or a subclass), while
+        # `:adapter_error` retries any. Replaces the defaults, as before.
+        def retry_on(*conditions)
+          @retryable_statuses, @retryable_errors = split_conditions(conditions.flatten)
         end
 
         def retryable?(result)
-          retryable_statuses.include?(result.status)
+          return true if retryable_statuses.include?(result.status)
+          return false unless result.status == :adapter_error && retryable_errors.any?
+
+          error_type = result.trace.respond_to?(:error_type) ? result.trace.error_type : nil
+          !error_type.nil? && retryable_errors.any? { |klass| error_type <= klass }
         end
 
         def config_for_attempt(attempt, default_config)
@@ -78,7 +87,24 @@ module RubyLLM
           else
             @max_attempts = attempts || 1
           end
-          @retryable_statuses = Array(retry_on).dup if retry_on
+          @retryable_statuses, @retryable_errors = split_conditions(Array(retry_on)) if retry_on
+        end
+
+        # Symbols are statuses; exception classes match adapter errors. Anything
+        # else (a String status among them) would never match, so it raises
+        # instead of quietly turning retries off.
+        def split_conditions(conditions)
+          invalid = conditions.reject { |condition| condition.is_a?(Symbol) || exception_class?(condition) }
+          unless invalid.empty?
+            raise ArgumentError,
+                  "retry_on takes status symbols and exception classes, got #{invalid.map(&:inspect).join(", ")}"
+          end
+
+          conditions.partition { |condition| condition.is_a?(Symbol) }
+        end
+
+        def exception_class?(condition)
+          condition.is_a?(Class) && condition <= Exception
         end
 
         def normalize_config(entry)

data/lib/ruby_llm/contract/step/runner.rb

@@ -60,7 +60,8 @@ module RubyLLM
             output_schema: @config.output_schema,
             model: @config.model,
             observers: @config.observers,
-            max_output: @config.effective_max_output
+            max_output: @config.effective_max_output,
+            on_incomplete_output: @config.on_incomplete_output
           )
         end
 
@@ -84,6 +85,18 @@ module RubyLLM
           @config.attachment_token_estimate
         end
 
+        def exact_token_count?
+          @config.token_count == :exact
+        end
+
+        def count_adapter
+          @config.adapter
+        end
+
+        def count_options
+          @config.adapter_options
+        end
+
         def on_unknown_attachment_size
           @config.on_unknown_attachment_size
         end

data/lib/ruby_llm/contract/step/runner_config.rb

@@ -19,7 +19,9 @@ module RubyLLM
         :on_unknown_attachment_size,
         :temperature,
         :extra_options,
-        :observers
+        :observers,
+        :on_incomplete_output,
+        :token_count
       ) do
         # Factory with sensible defaults for optional fields. Lets callers
         # (Step::Base#run_once and tests) construct a RunnerConfig without
@@ -30,7 +32,8 @@ module RubyLLM
                        output_schema: nil, max_output: nil,
                        max_input: nil, max_cost: nil, on_unknown_pricing: UnknownPolicy::DEFAULT,
                        attachment_token_estimate: nil, on_unknown_attachment_size: UnknownPolicy::DEFAULT,
-                       temperature: nil, extra_options: {}, observers: [])
+                       temperature: nil, extra_options: {}, observers: [],
+                       on_incomplete_output: Dsl::INCOMPLETE_OUTPUT_DEFAULT, token_count: Dsl::TOKEN_COUNT_DEFAULT)
           new(
             input_type: input_type, output_type: output_type,
             prompt_block: prompt_block, contract_definition: contract_definition,
@@ -41,7 +44,7 @@ module RubyLLM
             attachment_token_estimate: attachment_token_estimate,
             on_unknown_attachment_size: on_unknown_attachment_size,
             temperature: temperature, extra_options: extra_options,
-            observers: observers
+            observers: observers, on_incomplete_output: on_incomplete_output, token_count: token_count
           )
         end
 

data/lib/ruby_llm/contract/step/trace.rb

@@ -8,7 +8,7 @@ module RubyLLM
         include Concerns::DeepFreeze
 
         attr_reader :messages, :model, :latency_ms, :usage, :attempts, :cost,
-                    :usage_complete, :cost_complete, :finish_reason
+                    :usage_complete, :cost_complete, :finish_reason, :error_class, :error_type
 
         # Marks `cost:` as not given, as distinct from an explicit nil. Only an
         # omitted cost is priced from the registry; nil from an adapter that
@@ -18,8 +18,12 @@ module RubyLLM
         # usage_complete / cost_complete are nil for traces that carry no
         # accounting state (Test adapter, older adapters, hand-built traces);
         # those keep the token-count rule in `unpriced?`.
+        # error_class names the exception behind an :adapter_error and is part
+        # of to_h; error_type is the class itself, kept in memory only so
+        # `retry_on SomeError` can match subclasses. A trace rebuilt from a
+        # hash has the name but not the class.
         def initialize(messages: nil, model: nil, latency_ms: nil, usage: nil, attempts: nil, cost: COST_UNSET,
-                       usage_complete: nil, cost_complete: nil, finish_reason: nil)
+                       usage_complete: nil, cost_complete: nil, finish_reason: nil, error_class: nil, error_type: nil)
           @messages = deep_dup_freeze(messages)
           @model = model.frozen? ? model : model&.dup&.freeze
           @latency_ms = latency_ms
@@ -28,13 +32,15 @@ module RubyLLM
           @usage_complete = usage_complete
           @cost_complete = cost_complete
           @finish_reason = finish_reason
+          @error_class = error_class
+          @error_type = error_type
           @cost = resolve_cost(cost)
           @keep_nil_cost = keep_nil_cost?(cost)
           freeze
         end
 
         KNOWN_KEYS = %i[messages model latency_ms usage attempts cost
-                        usage_complete cost_complete finish_reason].freeze
+                        usage_complete cost_complete finish_reason error_class].freeze
 
         def [](key)
           return nil unless KNOWN_KEYS.include?(key.to_sym)
@@ -57,7 +63,8 @@ module RubyLLM
         # Never re-prices: a retry merges a subtotal and an aggregate usage that
         # no single model's pricing describes.
         def merge(**overrides)
-          self.class.new(**KNOWN_KEYS.to_h { |key| [key, overrides.fetch(key) { public_send(key) }] })
+          self.class.new(**KNOWN_KEYS.to_h { |key| [key, overrides.fetch(key) { public_send(key) }] },
+                         error_type: overrides.fetch(:error_type, @error_type))
         end
 
         # True when a total built from this trace undercounts: the adapter said
@@ -87,7 +94,7 @@ module RubyLLM
         def to_h
           hash = { messages: @messages, model: @model, latency_ms: @latency_ms,
                    usage: @usage, attempts: @attempts, cost: @cost, usage_complete: @usage_complete,
-                   cost_complete: @cost_complete, finish_reason: @finish_reason }.compact
+                   cost_complete: @cost_complete, finish_reason: @finish_reason, error_class: @error_class }.compact
           hash[:cost] = nil if @keep_nil_cost
           hash
         end

data/lib/ruby_llm/contract/version.rb

@@ -2,6 +2,6 @@
 
 module RubyLLM
   module Contract
-    VERSION = "1.1.2"
+    VERSION = "1.2.0"
   end
 end

data/lib/ruby_llm/contract/workflow_scope.rb

@@ -0,0 +1,48 @@
+# frozen_string_literal: true
+
+module RubyLLM
+  module Contract
+    # Groups a step's or pipeline's RubyLLM calls under `RubyLLM.workflow`, so
+    # OpenTelemetry spans and instrumentation events carry the step's name.
+    # Off unless `RubyLLM::Contract.configure { |c| c.workflow_instrumentation = true }`.
+    #
+    # A pipeline opens one workflow and runs each step as `workflow.step(alias)`;
+    # a step run inside it does not open another. A step run on its own opens a
+    # workflow named after its class. A workflow the application opened itself
+    # becomes the parent, as RubyLLM links nested workflows.
+    module WorkflowScope
+      CURRENT_KEY = :ruby_llm_contract_workflow
+
+      def self.workflow(name, &block)
+        return yield unless enabled? && current.nil?
+
+        ::RubyLLM.workflow(name.to_s) { |workflow| within(workflow, &block) }
+      end
+
+      def self.step(name, &)
+        workflow = current
+        return yield unless enabled? && workflow
+
+        workflow.step(name.to_s, &)
+      end
+
+      def self.enabled?
+        Contract.configuration.workflow_instrumentation
+      end
+
+      # Fiber-local, so concurrent eval threads each get their own.
+      def self.current
+        Thread.current[CURRENT_KEY]
+      end
+
+      def self.within(workflow)
+        previous = Thread.current[CURRENT_KEY]
+        Thread.current[CURRENT_KEY] = workflow
+        yield
+      ensure
+        Thread.current[CURRENT_KEY] = previous
+      end
+      private_class_method :within
+    end
+  end
+end

data/lib/ruby_llm/contract.rb

@@ -3,6 +3,8 @@
 require_relative "contract/version"
 require_relative "contract/errors"
 require_relative "contract/unknown_policy"
+require_relative "contract/provider_options"
+require_relative "contract/workflow_scope"
 require_relative "contract/types"
 
 module RubyLLM

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: ruby_llm-contract
 version: !ruby/object:Gem::Version
-  version: 1.1.2
+  version: 1.2.0
 platform: ruby
 authors:
 - Justyna
@@ -170,6 +170,7 @@ files:
 - lib/ruby_llm/contract/prompt/nodes/system_node.rb
 - lib/ruby_llm/contract/prompt/nodes/user_node.rb
 - lib/ruby_llm/contract/prompt/renderer.rb
+- lib/ruby_llm/contract/provider_options.rb
 - lib/ruby_llm/contract/railtie.rb
 - lib/ruby_llm/contract/rake_task.rb
 - lib/ruby_llm/contract/rake_task/suite_gate.rb
@@ -195,6 +196,7 @@ files:
 - lib/ruby_llm/contract/types.rb
 - lib/ruby_llm/contract/unknown_policy.rb
 - lib/ruby_llm/contract/version.rb
+- lib/ruby_llm/contract/workflow_scope.rb
 - ruby_llm-contract.gemspec
 homepage: https://github.com/justi/ruby_llm-contract
 licenses: