activeagent 1.9.7 → 1.9.8

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a2e216123a2bdec63446cf6f6af685c0c61b285a318f5a98845e09b188838427
-  data.tar.gz: 434fbcc22e9a7ffa10434c93ba4b3d63306e6f0ae5502742eb2c83a31abb9338
+  metadata.gz: 560139e6515a8b3568aabc383ec0ae4219764a2d8ca91b14b4defc4fdec3439d
+  data.tar.gz: b1d8651cb0a058b8922e579374720efd42c3babcadfddc3c123c5b7c6c2afd8b
 SHA512:
-  metadata.gz: 442971fa937ce0d4f44c6add3d3e8841f48789da87de99363e8fcc0a891bcf4bc49461b914c5816ee2754a8c8b22ed92d42b0a26cff3423f3c5de8fbdd3e653c
-  data.tar.gz: 971714f2043a08e213f3a646706da229d77f289f2a98e7674169406ee1b5767235b98988a71113868dcd93d680da72dd9721859d49c30577f812ac902e84704f
+  metadata.gz: 722dcbfe0e8bc2254871264f42ddf5394eb4f26acfc47f53ce17f2c9b382db627248d90ded1c9164a30bd176f908ee4510db11332e26bd6e25c91d485b251fde
+  data.tar.gz: e97465f43ad9dfee0eecfa0e3b3207b7c3a67dc620a1fa656c451800b9e49ca89259a75dd7ce7ad7066f05bfeece5d719fdf869e30a3cf89ae11f6e6f9de42dd

data/CHANGELOG.md

@@ -5,7 +5,183 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## [Unreleased]
+## [1.9.8] - 2026-11-12
+
+The dashboard holds a run to its catalog's pass bars: a product's and a
+set's `pass` blocks and a product's named criteria lists survive import, a
+run records the bars it started under and shows each group's gate beside
+its pass rate, and `evals run` exits 4 when a gated group misses, so CI
+fails on the group that slipped rather than on the whole-run average. A
+catalog that declares a `threshold` now scores at it instead of the fixed
+0.7. Around that: the empty agent list shows the ways an agent arrives,
+telemetry first; the RubyLLM provider works with ruby_llm 2.2 and the
+Anthropic provider with the `anthropic` gem 2.x; Runcrate joins the
+OpenAI-compatible providers; and smaller fixes to Anthropic usage counts,
+Requesty embeddings, `top_k` for thinking-first models, late cancels,
+`tool_observer` under Capture and Responses streaming errors.
+
+Upgrading: two dashboard migrations, 034 `add_catalog_product_criteria`
+(the `criteria` column on scenario products) and 035
+`add_catalog_pass_bars` (the `pass` column on scenario products and sets):
+run `bin/rails generate action_agent:install --skip` and `bin/rails
+db:migrate`. Until they run, catalog imports keep working but still drop
+named criteria lists and pass bars, so dashboard runs stay ungated; a
+catalog imported before the upgrade needs importing again to gain them.
+033 is left free for a later release. A finished run that carries gates
+now fails `evals run --wait` with exit 4 when any gated group missed its
+bar; a run without gates is reported as before. ruby_llm 2.2.0 (2026-11-10)
+breaks the RubyLLM provider in 1.9.7 and earlier (every tool call reads as
+`nil`); upgrade, or hold `ruby_llm < 2.2` until you do.
+
+### Added
+
+- **The dashboard shows a run's per-group pass gates** (`actionagent`). A
+  published report's `pass_thresholds` are kept with the run, so the
+  rebuilt report, the export and the run page gate the stored results as
+  the application's report did, and a run started from the dashboard on a
+  catalog set is held to the set's and product's `pass` bars, recorded as
+  the run starts so a later catalog change does not regrade it. The run
+  page adds a **Pass rate by group** table, and the run's badge reads
+  `gate missed` when a gate was not met even if the whole-run pass rate
+  looks fine. The run's JSON adds `gates` and `gates_met`; a run without
+  bars reads as before (#678).
+- **An empty agent list shows how agents arrive** (`actionagent`). With no
+  agents, the Agents page offered only Browse Templates and Create From
+  Scratch. It now shows three ways in, telemetry first:
+  - **Connect your app** asks what runs the agents. For ActiveAgent it shows
+    the `config/active_agent.yml` telemetry block. For RubyLLM it shows the
+    `activeagents-telemetry-ruby_llm` Gemfile line and the initializer that
+    subscribes it. Both point at this dashboard's endpoint and read the key
+    from `ACTIVEAGENTS_API_KEY`. The first trace registers the agent, and
+    **Check for agents** reloads the list.
+  - **Link a GitHub repository** opens New project.
+  - **Build one here** keeps the template library and the builder.
+- **Runcrate as an OpenAI-compatible provider** (`activeagent`).
+  `service: "Runcrate"` generates and embeds through
+  `https://api.runcrate.ai/v1` (open-weight models such as
+  `deepseek-ai/DeepSeek-V4.1-Flash` and `Qwen/Qwen3-Embedding-8B`), with the
+  key from `RUNCRATE_API_KEY`; a thin subclass of the OpenAI provider, like
+  Cheaper Inference.
+
+### Fixed
+
+- **The RubyLLM provider works with ruby_llm 2.2** (`activeagent`). RubyLLM
+  2.2.0 (2026-11-10) changed three things the provider leaned on. A
+  message's `tool_calls` became a `ToolCall::Collection` that yields the
+  calls rather than ID-and-call pairs, so every tool call read as `nil`; a
+  streamed delta now carries the provider's call ID in `provider_call_id`
+  and leaves `id` empty, so every fragment joined the latest call and
+  arguments doubled up; and a chat asks each tool for its `namespace`,
+  `qualified_name` and `deferred?`, which ActiveAgent's tool proxy did not
+  answer. The provider now iterates either shape, keys streamed fragments
+  by the provider's ID, and the proxy answers the three. ruby_llm 1.16
+  through 2.1 behave as before. The evals conversation check also learned
+  2.2's `acts_as_chat` modules, columns and reworded unavailable-tool error.
+- **`evals run` fails CI on a missed group gate** (`activeagent`). A run
+  whose critical group slipped to 50% passed `evals run --min-pass-rate`
+  whenever the other groups lifted the whole-run pass rate over the bar.
+  A finished run that the dashboard holds to per-group pass gates now fails
+  (exit 4) when any gated group missed its bar, with `--wait` alone or with
+  `--min-pass-rate`, which stays the whole-run bar; each missed gate prints
+  as one line, and `--json` adds `gates`. A run without gates is gated as
+  before (#677).
+- **Responses streaming failures retain their error details** (`activeagent`).
+  `error` and `response.failed` events raise a provider error with the API
+  code and message instead of an unexpected chunk type error.
+- **The dashboard keeps a catalog's pass bars** (`actionagent`). A
+  product's and a set's `pass` blocks (`min_pass_rate`, `threshold`,
+  `gated`) parsed in `ActiveAgent::Evals::Catalog`, but neither
+  `ScenarioProduct` nor `ScenarioSet` kept them: an import dropped the
+  bars, the export left them out, the stored catalog's `pass_thresholds`
+  fell back to the defaults (every group gated at 100%), and the import
+  preview reported the product and its sets as `update` on every re-import
+  of an unchanged file. The bars are now stored and exported as imported,
+  and both import modes take the document's. Existing installs add the
+  columns with `bin/rails generate action_agent:install --skip` and
+  `bin/rails db:migrate` (`add_catalog_pass_bars`); imports keep working,
+  without the bars, until it runs. Bars dropped by an earlier version come
+  back when the original document is imported again.
+- **The dashboard keeps a catalog product's named criteria lists**
+  (`actionagent`). A product's `criteria: { shared: [...], strict: [...] }`
+  parsed and round-tripped in `ActiveAgent::Evals::Catalog`, but
+  `ScenarioProduct` had nowhere to keep it: an import dropped the lists, the
+  export left them out, the stored digest never matched the records'
+  export, and the import preview reported the product as `update` on every
+  re-import of an unchanged file. The lists are now stored on the product
+  and exported as imported, so the documents and digests match and an
+  unchanged file previews as unchanged. Both import modes take the
+  document's lists, as they take a set's `criteria`. A set still runs on
+  its own `criteria`; which product lists a run uses is left for later.
+  Existing installs add the column with
+  `bin/rails generate action_agent:install --skip` and `bin/rails db:migrate`
+  (`add_catalog_product_criteria`); imports keep working, without the lists,
+  until it runs. Lists dropped by an earlier version come back when the
+  original document is imported again (#687).
+- **A self-hosted dashboard shows its own trace endpoint** (`actionagent`).
+  The Organization page's Telemetry card showed the platform's `/v1/traces`.
+  A self-hosted engine takes traces at `<mount>/api/traces`, and the card
+  now shows that. Multi-tenant installs are unchanged.
+- **The agent list's filter arrows sit inside the select** (`actionagent`).
+  The browser's own arrow ignores padding and sat against the right border.
+  The three filters now draw an inset chevron (`aa-select`).
+- **The Anthropic provider loads the `anthropic` gem 2.x** (`activeagent`).
+  The provider pinned its client gem to `~> 1.12`, so a bundle holding
+  anthropic 2.0 or later raised `LoadError` the moment the provider loaded,
+  although the provider builds its requests as hashes and the 2.0 SDK
+  (role-union `MessageParam`, one-value enums as constants, Text Completions
+  removed) breaks none of them. The provider and the gemspec's development
+  dependency now accept anthropic 1.12 through 2.x, and CI runs the provider
+  tests against 2.3 beside the 1.12, 1.14 and 1.55 legs. Two 2.0 changes do
+  reach the code: Text Completions went away with
+  `Anthropic::Resources::Completions`, which the Bedrock bearer client now
+  sets up only when it exists, and the client sends its key as
+  `Authorization: Bearer` instead of `x-api-key`. The 2.0 SDK also dropped
+  the `?beta=true` query string from its beta Messages requests, so the test
+  suite matches recorded cassettes with or without it.
+- **A streamed Anthropic response reports its token usage**
+  (`activeagent`). With `stream: true`, `response.usage` was nil, so a
+  streamed generation counted 0 tokens and cost nothing. The provider now
+  records the usage `message_start` reports and the cumulative counts
+  `message_delta` carries, as the OpenAI providers do from their chunks.
+- **Anthropic's `total_tokens` counts cached tokens** (`activeagent`).
+  Anthropic's `input_tokens` leaves out cache reads and writes, which it
+  reports separately, so `total_tokens` undercounted any prompt that used
+  the cache. It is now input + output + cache read + cache creation. The
+  other counts are unchanged, and so is cost: pricing still reads
+  `input_tokens` and `output_tokens` only.
+- **Added usage keeps its duration** (`activeagent`). `Usage#+` dropped
+  `duration_ms`, so the usage of a tool-calling loop (Ollama reports a
+  duration) came back without one. It is now summed like the other
+  optional counts.
+- **Requesty embeddings work** (`activeagent`). The Requesty provider
+  declared an embedding request but never sent one, so `embed` raised
+  `NotImplementedError`. It now calls Requesty's OpenAI-compatible
+  `/v1/embeddings`.
+- **`top_k` is stripped for thinking-first models** (`activeagent`).
+  `ModelCapabilities` removed `temperature` and `top_p` for the models that
+  reject sampling parameters, but not `top_k`, which they reject too, so an
+  agent configured with it still got a 400.
+- **A late cancel leaves a finished run as it finished** (`actionagent`).
+  `AgentRun#cancel!` checked the status of its own copy of the run and
+  then wrote `cancelled` unconditionally. A cancel request that loaded the
+  run while it was running, and wrote after the worker had recorded its
+  result, turned a complete run into a cancelled one that still carried
+  its output, and replaced a failed run's error with "Cancelled by user".
+  The cancel now moves the run to `cancelled` in one conditional update,
+  as the execution job claims a pending run, and only from pending,
+  running or awaiting input. A run that finished first keeps its status,
+  output, error and completion time; `cancel!` returns false and reloads
+  the copy, so the API answers with the status the run ended in. A cancel
+  is broadcast once, where the callback and the method each sent it
+  before. Cancelling still does not stop a provider call already in
+  flight; it keeps that call's result from being recorded.
+- **Capture keeps an agent's own `tool_observer`** (`activeagent`). Since
+  1.9.7, `Evals::Capture` records client-side MCP calls through the
+  agent's `tool_observer`, and it replaced an observer the agent defined
+  itself, so that observer stopped running whenever the agent was
+  captured. Capture now runs the agent's observer around the call it
+  records: the server is still called once, both see the same result or
+  exception, and a call the observer withholds is not recorded as made.
 
 ## [1.9.7] - 2026-11-04
 

data/README.md

@@ -66,6 +66,9 @@ bundle add openai
 # Cheaper Inference (uses OpenAI-compatible API)
 bundle add openai
 
+# Runcrate (uses OpenAI-compatible API)
+bundle add openai
+
 # DeepSeek (uses OpenAI-compatible API)
 bundle add openai
 

data/lib/active_agent/cli/eval_commands.rb

@@ -86,8 +86,8 @@ module ActiveAgent
           p.on("--keys LIST", "Replay only these scenario keys, comma-separated") { |v| arguments[:keys] = list_option(v) }
           p.on("--group GROUP", "Replay only this scenario group") { |v| arguments[:group] = v }
           p.on("--sandbox ID", "Replay against this ready checkout sandbox") { |v| arguments[:sandbox_id] = v }
-          p.on("--wait", "Wait for the run to finish") { wait = true }
-          p.on("--min-pass-rate RATE", "Fail (exit #{EXIT_GATE}) below this pass rate: 0.8 or 80%; implies --wait") do |v|
+          p.on("--wait", "Wait for the run to finish; fail (exit #{EXIT_GATE}) when it failed or missed a pass gate") { wait = true }
+          p.on("--min-pass-rate RATE", "Also fail below this whole-run pass rate: 0.8 or 80%; implies --wait") do |v|
             min_pass_rate = pass_rate_option(v)
             wait = true
           end
@@ -199,18 +199,28 @@ module ActiveAgent
         end
       end
 
+      # A finished run fails the gate when it failed, when its whole pass
+      # rate is under --min-pass-rate, or when it was held to per-group pass
+      # gates (the run's "gates", from the suite's or catalog set's `pass`
+      # bars) and the dashboard says it did not meet every one. A run
+      # without gates is gated on its status and rate alone, as it was.
       def gate(payload, min_pass_rate)
         run = payload.fetch("run", {})
         rate = pass_rate(run)
         failed = run["status"] == "failed"
         below = min_pass_rate && (rate.nil? || rate < min_pass_rate)
-        verdict = failed || below ? "fail" : "pass"
+        gates = run_gates(run)
+        missed = !gates.nil? && run["gates_met"] != true
+        verdict = failed || below || missed ? "fail" : "pass"
         failures = payload.fetch("results", [])
+        summary = { "pass_rate" => rate, "min_pass_rate" => min_pass_rate }
+        summary["gates"] = gates if gates
 
-        emit(payload.merge("pass_rate" => rate, "min_pass_rate" => min_pass_rate, "verdict" => verdict)) do
+        emit(payload.merge(summary).merge("verdict" => verdict)) do
           @out.puts "Run #{run["id"]} #{run["status"]}: #{passed(run) || "no results"}" \
                     "#{rate ? format(" (%.1f%%)", rate * 100) : ""}#{min_pass_rate ? format(", threshold %.1f%%", min_pass_rate * 100) : ""}."
           @out.puts "Error: #{run["error_message"]}" if run["error_message"]
+          print_gates(gates, missed) if gates
           unless failures.empty?
             @out.puts "Failed results:"
             table(failures, {
@@ -227,6 +237,39 @@ module ActiveAgent
         verdict == "pass" ? EXIT_SUCCESS : EXIT_GATE
       end
 
+      # The run's per-group pass gates as the dashboard worked them out
+      # (EvaluationRun#gates), or nil for a run held to none.
+      def run_gates(run)
+        rows = run["gates"]
+        rows.select { |row| row.is_a?(Hash) } if rows.is_a?(Array)
+      end
+
+      def print_gates(gates, missed)
+        unmet = gates.reject { |gate| gate["met"] == true }
+        unless missed
+          @out.puts "Gates: every gate met (#{gates.size})."
+          return
+        end
+
+        @out.puts(unmet.empty? ? "Gates: no gate was reached." : "Gates not met:")
+        unmet.each { |gate| @out.puts "  #{gate_sentence(gate)}" }
+      end
+
+      # A gate the run missed, as ActiveAgent::Evals::Report::Gate#to_s
+      # reads it: "Orders · gpt-5-mini: 1/2 (50%) passed, below 100%".
+      def gate_sentence(gate)
+        passed = gate["passed"].to_i
+        total = gate["total"].to_i
+        passes = total.positive? ? "#{passed}/#{total} (#{whole_percent(passed.to_f / total)})" : "no results"
+        label = cell(gate["label"]) unless gate["label"].to_s.empty?
+        "#{cell(gate["name"] || gate["group"] || "All")}#{label ? " · #{label}" : ""}: #{passes} passed, below #{whole_percent(gate["min_pass_rate"])}"
+      end
+
+      def whole_percent(fraction)
+        value = Float(fraction, exception: false)
+        value ? "#{(value * 100).round}%" : "?"
+      end
+
       def pass_rate(run)
         evaluated = run["samples_evaluated"].to_i
         evaluated.positive? ? run["samples_passed"].to_f / evaluated : nil

data/lib/active_agent/concerns/tooling.rb

@@ -30,8 +30,10 @@ module ActiveAgent
     # A callback the provider runs the calls it serves from an MCP server
     # around, which never reach {#tools_function}. It receives the tool name,
     # its arguments and a block that makes the call, and returns the block's
-    # result. Nil unless a host that records a generation's tool calls (see
-    # Evals::Capture) sets it on the instance.
+    # result. Nil unless the agent defines one, to audit its MCP calls say,
+    # or a host that records a generation's tool calls sets it on the
+    # instance. Evals::Capture keeps an agent's own observer and records the
+    # call that observer makes.
     #
     # @return [Proc, nil]
     def tool_observer = nil

data/lib/active_agent/evals/capture.rb

@@ -19,8 +19,10 @@ module ActiveAgent
     # (`mcp_strategy: :client`, or a provider with no MCP of its own) is
     # dispatched before that proc, so the provider reports it through the
     # agent's `tool_observer` instead, with a failure the server reported
-    # marked as one. A tool a provider runs on its own side (Anthropic's or
-    # OpenAI's hosted MCP) never reaches this process and is not recorded.
+    # marked as one. An observer the agent defines itself still runs, around
+    # the call that is recorded. A tool a provider runs on its own side
+    # (Anthropic's or OpenAI's hosted MCP) never reaches this process and is
+    # not recorded.
     #
     # `model:` and `provider:` replay the same action under another model,
     # the way a comparison run does. A model is an option on the agent
@@ -190,9 +192,15 @@ module ActiveAgent
       # the provider runs that call inside the agent's `tool_observer`, which
       # records it the same way. The two paths are disjoint, so a call is
       # recorded once, and the observer sees the call without making it again.
+      #
+      # An observer the agent already has is kept, as tools_function's `base`
+      # is: it runs around the call this records, so it sees the same name,
+      # arguments, result and exception, and a call it does not make is not
+      # recorded as made.
       def record_tool_calls(agent)
         calls = @tool_calls
         base = agent.tools_function
+        observer = agent.tool_observer
         capture = self.class
 
         agent.define_singleton_method(:tools_function) do
@@ -203,7 +211,10 @@ module ActiveAgent
         end
 
         agent.define_singleton_method(:tool_observer) do
-          proc { |name, arguments, &call| capture.record_call(calls, name, arguments.presence, &call) }
+          record = proc { |name, arguments, &call| capture.record_call(calls, name, arguments.presence, &call) }
+          next record unless observer
+
+          proc { |name, arguments, &call| observer.call(name, arguments) { record.call(name, arguments, &call) } }
         end
       end
 

data/lib/active_agent/model_capabilities.rb

@@ -17,7 +17,7 @@ module ActiveAgent
   # @example Disable sanitization entirely
   #   ActiveAgent::ModelCapabilities.enabled = false
   module ModelCapabilities
-    SAMPLING_PARAMS = [ :temperature, :top_p ].freeze
+    SAMPLING_PARAMS = [ :temperature, :top_p, :top_k ].freeze
 
     # Model families that reject sampling parameters with a 400:
     # - Anthropic thinking-first models (Opus 4.7+, Opus 5, Sonnet 5,

data/lib/active_agent/providers/_base_provider.rb

@@ -10,7 +10,7 @@ require_relative "concerns/tool_choice_clearing"
 
 # @private
 GEM_LOADERS = {
-  anthropic: [ "anthropic", "~> 1.12", "anthropic" ],
+  anthropic: [ "anthropic", [ ">= 1.12", "< 3" ], "anthropic" ],
   openai:    [ "openai",    "~> 0.34", "openai" ],
   # Keep a tested floor: 1.2 lacks the provider APIs this adapter uses.
   ruby_llm:  [ "ruby_llm",  [ ">= 1.16", "< 3" ], "ruby_llm" ]

data/lib/active_agent/providers/anthropic_provider.rb

@@ -197,6 +197,7 @@ module ActiveAgent
         when :message_start
           api_message = Anthropic::Transforms.gem_to_hash(api_response_chunk.message)
           self.message_stack.push(api_message)
+          record_stream_usage(api_message[:usage])
           broadcast_stream_update(message_stack.last)
 
         # -> Content Block Create
@@ -239,6 +240,15 @@ module ActiveAgent
           self.message_stack.last.merge!(delta)
           self.stream_finish_reason = delta[:stop_reason] if delta[:stop_reason]
 
+          # message_delta carries the cumulative output tokens (and, on newer
+          # API versions, refreshed input counts); layer them over the usage
+          # message_start reported so the turn keeps a single running total.
+          delta_usage = Anthropic::Transforms.gem_to_hash(api_response_chunk.usage)&.compact
+          if delta_usage.present?
+            message_stack.last[:usage] = (message_stack.last[:usage] || {}).merge(delta_usage)
+            record_stream_usage(message_stack.last[:usage])
+          end
+
         # Message Completed [Full Message]
         when :message_stop
           api_message = Anthropic::Transforms.gem_to_hash(api_response_chunk.message)

data/lib/active_agent/providers/bedrock/bearer_client.rb

@@ -51,7 +51,8 @@ module ActiveAgent
           )
 
           @messages = ::Anthropic::Resources::Messages.new(client: self)
-          @completions = ::Anthropic::Resources::Completions.new(client: self)
+          # The anthropic gem 2.0 removed Text Completions along with this resource.
+          @completions = ::Anthropic::Resources::Completions.new(client: self) if defined?(::Anthropic::Resources::Completions)
           @beta = ::Anthropic::Resources::Beta.new(client: self)
         end
 

data/lib/active_agent/providers/common/usage.rb

@@ -148,7 +148,8 @@ module ActiveAgent
             cached_tokens:         sum_optional(self.cached_tokens,         other.cached_tokens),
             cache_creation_tokens: sum_optional(self.cache_creation_tokens, other.cache_creation_tokens),
             reasoning_tokens:      sum_optional(self.reasoning_tokens,      other.reasoning_tokens),
-            audio_tokens:          sum_optional(self.audio_tokens,          other.audio_tokens)
+            audio_tokens:          sum_optional(self.audio_tokens,          other.audio_tokens),
+            duration_ms:           sum_optional(self.duration_ms,           other.duration_ms)
           )
         end
 
@@ -263,6 +264,10 @@ module ActiveAgent
             **usage.slice(:input_tokens, :output_tokens, :service_tier),
             input_tokens: usage[:input_tokens] || 0,
             output_tokens: usage[:output_tokens] || 0,
+            # Anthropic's input_tokens excludes cache reads and writes, which are
+            # reported separately, so they are added to get every token processed.
+            total_tokens: (usage[:input_tokens] || 0) + (usage[:output_tokens] || 0) +
+              (usage[:cache_read_input_tokens] || 0) + (usage[:cache_creation_input_tokens] || 0),
             cached_tokens: usage[:cache_read_input_tokens],
             cache_creation_tokens: usage[:cache_creation_input_tokens],
             provider_details: usage.slice(:cache_creation, :server_tool_use).compact

data/lib/active_agent/providers/open_ai/responses_provider.rb

@@ -110,6 +110,7 @@ module ActiveAgent
         #   reasoning progress, kept from the reasoning item once it is done
         # - `:"response.content_part.done"` - content part completed
         # - `:"response.output_item.done"` - message, reasoning or function call completed
+        # - `:error`, `:"response.failed"` - provider failure
         # - `:"response.completed"` - response finished
         # - `:"response.incomplete"` - response cut short (e.g. max_output_tokens)
         #
@@ -174,6 +175,12 @@ module ActiveAgent
           when :"response.output_item.done"
             process_stream_output_item_done(api_response_event)
 
+          # Errors arrive as events even though the HTTP request succeeded.
+          when :error, :"response.failed"
+            error = api_response_event.type == :error ? api_response_event : api_response_event.response.error
+            message = [ error&.code, error&.message ].compact.join(": ")
+            raise Errors::ProviderError.new(message.presence || "OpenAI response failed", provider_tag: tag_name)
+
           # Response Completed, or cut short (e.g. by max_output_tokens) with
           # the output it has so far
           when :"response.completed", :"response.incomplete"

data/lib/active_agent/providers/requesty_provider.rb

@@ -50,6 +50,14 @@ module ActiveAgent
 
       protected
 
+      # Executes an embedding request via Requesty's OpenAI-compatible API.
+      #
+      # @param parameters [Hash]
+      # @return [Hash] symbolized API response
+      def api_embed_execute(parameters)
+        client.embeddings.create(**parameters).as_json.deep_symbolize_keys
+      end
+
       # @see BaseProvider#api_response_normalize
       # @param api_response [OpenAI::Models::ChatCompletion]
       # @return [Hash] normalized response hash

data/lib/active_agent/providers/ruby_llm/tool_proxy.rb

@@ -32,6 +32,21 @@ module ActiveAgent
         alias_method :declared_parameters, :parameters
         alias_method :provider_options, :provider_params
 
+        # RubyLLM 2.2 registers a tool under its qualified name (namespace
+        # plus name) and asks each tool whether it defers. ActiveAgent's
+        # tools have no namespace and run when called.
+        def namespace
+          nil
+        end
+
+        def qualified_name
+          name
+        end
+
+        def deferred?
+          false
+        end
+
         private
 
         def deep_stringify(obj)

data/lib/active_agent/providers/ruby_llm_provider.rb

@@ -152,15 +152,16 @@ module ActiveAgent
         # Handle tool calls in chunk
         if chunk.tool_calls&.any?
           message[:tool_calls] ||= []
-          chunk.tool_calls.each do |key, tool_call|
+          each_tool_call(chunk.tool_calls) do |key, tool_call|
             # Only the first delta of a call has its ID and name; the rest
             # are fragments of its arguments. A fragment belongs to the call
             # that shares its stream key (the index RubyLLM 2 numbers calls
             # by), or, without one as RubyLLM 1.16 leaves it for OpenAI, to
             # the latest call. RubyLLM 1.16 cannot preserve interleaved
             # OpenAI calls: its parser drops their indices and fragments.
-            existing = if tool_call.id
-              message[:tool_calls].find { |tc| tc[:id] == tool_call.id }
+            call_id = streamed_tool_call_id(tool_call)
+            existing = if call_id
+              message[:tool_calls].find { |tc| tc[:id] == call_id }
             else
               @stream_tool_calls[key] || message[:tool_calls].last
             end
@@ -169,7 +170,7 @@ module ActiveAgent
               existing[:function][:arguments] += fragment
             else
               existing = {
-                id: tool_call.id,
+                id: call_id,
                 type: "function",
                 function: {
                   name: tool_call.name,
@@ -492,7 +493,7 @@ module ActiveAgent
 
         # Handle tool calls
         if response.tool_calls&.any?
-          hash[:tool_calls] = response.tool_calls.map do |id, tc|
+          hash[:tool_calls] = each_tool_call(response.tool_calls).map do |id, tc|
             {
               id: id,
               type: "function",
@@ -523,6 +524,41 @@ module ActiveAgent
         hash
       end
 
+      # Yields each of a message's tool calls with its key. RubyLLM 1.x and
+      # 2.0 to 2.1 keep them in a Hash keyed by call ID (or, on a streamed
+      # chunk, by stream index), so iterating yields pairs. RubyLLM 2.2's
+      # `ToolCall::Collection` yields the calls themselves and keeps the
+      # keys behind `each_key`, in the same order.
+      #
+      # @param calls [Hash, #each_key, nil]
+      # @return [Enumerator, Array] key and call pairs
+      def each_tool_call(calls, &block)
+        pairs = if calls.nil?
+          []
+        elsif calls.is_a?(Hash)
+          calls.to_a
+        elsif calls.respond_to?(:each_key)
+          calls.each_key.to_a.zip(calls.each.to_a)
+        else
+          calls.each.to_a.map { |call| [ call.id, call ] }
+        end
+        block ? pairs.each(&block) : pairs
+      end
+
+      # The provider's ID for a streamed tool call, present on its first
+      # delta and nil on the fragments. RubyLLM 2.2 keeps it in
+      # `provider_call_id` and leaves `id` (its own identifier, assigned when
+      # the message completes) nil on a chunk; earlier releases put the
+      # provider's ID in `id`.
+      #
+      # @param tool_call [RubyLLM::ToolCall]
+      # @return [String, nil]
+      def streamed_tool_call_id(tool_call)
+        provider_id = tool_call.provider_call_id if tool_call.respond_to?(:provider_call_id)
+        provider_id = nil if provider_id.to_s.empty?
+        provider_id || tool_call.id
+      end
+
       # A streamed tool call's arguments as the text they add to the JSON
       # string the call builds up. Most APIs stream that string; Anthropic
       # opens a call with an empty Hash for its input, and Gemini sends the

data/lib/active_agent/providers/runcrate/_types.rb

@@ -0,0 +1,17 @@
+# frozen_string_literal: true
+
+require_relative "../open_ai/chat/_types"
+require_relative "options"
+
+module ActiveAgent
+  module Providers
+    module Runcrate
+      # ActiveModel type for casting and serializing Runcrate requests.
+      #
+      # Runcrate is OpenAI-compatible, so requests use the same shape as the
+      # OpenAI Chat API. This delegates entirely to
+      # OpenAI::Chat::RequestType.
+      RequestType = ActiveAgent::Providers::OpenAI::Chat::RequestType
+    end
+  end
+end

data/lib/active_agent/providers/runcrate/options.rb

@@ -0,0 +1,39 @@
+# frozen_string_literal: true
+
+require_relative "../open_ai/options"
+
+module ActiveAgent
+  module Providers
+    module Runcrate
+      # Configuration options for the Runcrate provider.
+      #
+      # Extends OpenAI::Options, overriding the base URL to point at
+      # Runcrate's OpenAI-compatible inference API and resolving the API key
+      # from RUNCRATE_API_KEY. Runcrate does not use organization or project
+      # identifiers.
+      #
+      # @example Basic configuration
+      #   options = Options.new(api_key: ENV["RUNCRATE_API_KEY"])
+      #
+      # @see https://runcrate.ai/docs/models/overview
+      # @see https://runcrate.ai/dashboard/api-keys Runcrate API Keys
+      class Options < ActiveAgent::Providers::OpenAI::Options
+        # @!attribute base_url
+        #   @return [String] API endpoint (default: "https://api.runcrate.ai/v1")
+        attribute :base_url, :string, as: "https://api.runcrate.ai/v1"
+
+        private
+
+        def resolve_api_key(kwargs)
+          kwargs[:api_key] ||
+            kwargs[:access_token] ||
+            ENV["RUNCRATE_API_KEY"]
+        end
+
+        # Not used as part of Runcrate
+        def resolve_organization_id(kwargs) = nil
+        def resolve_project_id(kwargs)      = nil
+      end
+    end
+  end
+end

data/lib/active_agent/providers/runcrate_provider.rb

@@ -0,0 +1,72 @@
+require_relative "_base_provider"
+
+require_gem!(:openai, __FILE__)
+
+require_relative "open_ai_provider"
+require_relative "runcrate/_types"
+
+module ActiveAgent
+  module Providers
+    # Provides access to Runcrate's OpenAI-compatible inference API for
+    # open-weight models.
+    #
+    # Extends the OpenAI provider to work with Runcrate's OpenAI-compatible
+    # API. Model ids are the vendor-prefixed names Runcrate lists at
+    # +/v1/models+ (e.g. +deepseek-ai/DeepSeek-V4.1-Flash+).
+    #
+    # Runcrate is a plain OpenAI-compatible API: requests, responses and
+    # transforms are identical to the OpenAI Chat API, so this provider
+    # reuses OpenAI::Chat::RequestType and OpenAI::Chat::Transforms directly.
+    # The only Runcrate-specific configuration is the base URL and API key,
+    # which live in Runcrate::Options.
+    #
+    # @example Configuration in active_agent.yml
+    #   runcrate:
+    #     service: "Runcrate"
+    #     api_key: <%= ENV["RUNCRATE_API_KEY"] %>
+    #     model: "deepseek-ai/DeepSeek-V4.1-Flash"
+    #
+    # @see OpenAI::ChatProvider
+    # @see https://runcrate.ai/docs/models/overview
+    class RuncrateProvider < OpenAI::ChatProvider
+      # @return [String]
+      def self.service_name
+        "Runcrate"
+      end
+
+      # @return [Class]
+      def self.options_klass
+        Runcrate::Options
+      end
+
+      # @return [ActiveModel::Type::Value]
+      def self.prompt_request_type
+        OpenAI::Chat::RequestType.new
+      end
+
+      # @return [ActiveModel::Type::Value]
+      def self.embed_request_type
+        OpenAI::Embedding::RequestType.new
+      end
+
+      protected
+
+      # Executes an embedding request via Runcrate's OpenAI-compatible API.
+      #
+      # @param parameters [Hash]
+      # @return [Hash] symbolized API response
+      def api_embed_execute(parameters)
+        client.embeddings.create(**parameters).as_json.deep_symbolize_keys
+      end
+
+      # @see BaseProvider#api_response_normalize
+      # @param api_response [OpenAI::Models::ChatCompletion]
+      # @return [Hash] normalized response hash
+      def api_response_normalize(api_response)
+        return api_response unless api_response
+
+        OpenAI::Chat::Transforms.gem_to_hash(api_response)
+      end
+    end
+  end
+end

data/lib/active_agent/version.rb

@@ -1,3 +1,3 @@
 module ActiveAgent
-  VERSION = "1.9.7"
+  VERSION = "1.9.8"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: activeagent
 version: !ruby/object:Gem::Version
-  version: 1.9.7
+  version: 1.9.8
 platform: ruby
 authors:
 - Justin Bowen
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2026-11-05 00:00:00.000000000 Z
+date: 2026-11-12 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: actionpack
@@ -162,16 +162,22 @@ dependencies:
   name: anthropic
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
         version: '1.12'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '3'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
         version: '1.12'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '3'
 - !ruby/object:Gem::Dependency
   name: aws-sdk-bedrockruntime
   requirement: !ruby/object:Gem::Requirement
@@ -630,6 +636,9 @@ files:
 - lib/active_agent/providers/ruby_llm/tool_proxy.rb
 - lib/active_agent/providers/ruby_llm_provider.rb
 - lib/active_agent/providers/rubyllm_provider.rb
+- lib/active_agent/providers/runcrate/_types.rb
+- lib/active_agent/providers/runcrate/options.rb
+- lib/active_agent/providers/runcrate_provider.rb
 - lib/active_agent/railtie.rb
 - lib/active_agent/railtie/schema_generator_extension.rb
 - lib/active_agent/schema_generator.rb