activeagent 1.9.1 → 1.9.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: c47ac88022fb9c9fe8919efaaf8cfb6824dfa7ba102cfc31df2bf42d8f4b19ef
-  data.tar.gz: 9670044e2fd2fceae17068327a7f0b5e9695ae049f81cd2b00e7f56387c20286
+  metadata.gz: a7ecc64cad33188dd3cdfc563b553e49d0a05d42192698365a467061e6bb6d59
+  data.tar.gz: c3923cee8e401e17e3170c28f2fc83d8c290d70c5a4bc4d301951031d7d9cf43
 SHA512:
-  metadata.gz: a2ee2907d4449caef1fd7f22d0b5418dfb19aa0140aaa02b47d7a394671dc16653d315ee80bcd6699a9baa0085c6202726ccbbefe9514689d1b0c53b708482e5
-  data.tar.gz: f98937727b17855137cbf76434ebd5657dbd8d4c37a94119edafadd9da412e46ec44a068b4cbafbc19d04e2726dfa75e62f4eaad503e9db40709d9ca21aacc88
+  metadata.gz: 66e3244ce04d56625f880e26c18cbe3f353f35eca2f4dd5922a1adafb1d542ee5216065401e7ddbbe88cccc1c40336076ab6d4a30d54a5360846cad595ae9717
+  data.tar.gz: b023e5fe43f6674b7040086cc5a47c19fffc4bf3e919d0bb0ab0138d5c4839d3463f09e5a66d7047e4e28262628465a066b343ee76a503732b803ddc5d270bff

data/CHANGELOG.md

@@ -5,7 +5,413 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## [Unreleased]
+## [1.9.3] - 2026-10-21
+
+Releases `activeagent` and `actionagent` 1.9.3, with the matching browser
+sidecar: browser upload and code conversations (the sandbox browser's live
+view takes files from the person driving it, and a Claude Code or Codex
+session is a conversation with the sandbox's browser beside the chat and a
+hand-off either way, Codex on this machine's own login), evaluation run
+reports exported as JSON, HTML and Markdown with Copy as Markdown, an agent
+builder that offers only tools that work, a model leaderboard on the
+Evaluations page, complete trace window statistics, a judge pin that runs
+on its own provider, Opper as a provider and Parallel Search in the MCP
+catalog, and streaming fixes (unindexed tool call fragments, the finish
+reason of streamed and Responses generations, Anthropic cache-token usage).
+The framework, the dashboard engine and `@activeagents/browser-sidecar`
+move to 1.9.3 together.
+
+Upgrading: migration 028 adds the conversation columns to `code_sessions`.
+A host runs `bin/rails generate action_agent:install --skip` then
+`bin/rails db:migrate`, after 1.9.1's 027. Upgrade the sidecar with the
+engine: the file chooser relay and the per-turn browser tokens are in the
+1.9.3 sidecar. On a single-tenant install the new report export
+(`GET <mount>/api/evaluation_reports/:run_id`) answers 401 until
+`ActionAgent.ingest_api_key` is set. A `judge_model` pin written
+`anthropic/…` or `openai/…` now runs on that native provider; an install
+that reached such a model through OpenRouter pins `openrouter/anthropic/…`.
+
+### Added
+
+- **Export evaluation run reports as JSON, HTML and Markdown** (`activeagent`,
+  `actionagent`). `GET <mount>/api/evaluation_reports/:run_id`, with the
+  ingest key and scoped to the tenant like the POST (and refused on a
+  single-tenant install that set no key), returns a complete run
+  as the version-1 envelope the collector takes, so a run moves to another
+  install with one POST and posting it back is an identical retry (200);
+  `?format=html` returns the run's report page and `?format=md` its
+  Markdown, and `?download=1` an attachment of any of them.
+  An imported run exports under its published `run_id`, a dashboard run
+  under `dashboard-run-<id>-<created_at>`. `ActiveAgent::Evals::Publisher.envelope`
+  builds the envelope for both the publisher and the export. The dashboard's
+  run report adds `report.json` and `report.md` (one piece with
+  `?part=verdict|fix_items|scenario`), and `?download=1` saves any format
+  as `<evaluation>-run-<n>.<ext>`; the run report view and the suite's run
+  header get *Download HTML / JSON / Markdown*. New rake tasks:
+  `action_agent:evaluations:export RUN=… [EVALUATION=…] FORMAT=json|html|md [FILE=…]`
+  and `action_agent:evaluations:import FILE=… [ACCOUNT_ID=…]`.
+
+- **Copy as Markdown** (`activeagent`, `actionagent`). `Report#to_markdown`
+  now has the HTML page's content in GitHub-flavored Markdown: the verdict's
+  rationale, What to fix from `Report#fix_items` (replacing the old
+  Recommendations list), and every scenario's answers in a `<details>` block
+  with a per-model table; table cells escape `|` and line breaks. Its pieces
+  are public — `verdict_markdown`, `fix_items_markdown(model:)`,
+  `fix_item_markdown(item)`, `scenario_markdown(key)` — and the dashboard
+  and the standalone HTML page copy each of them: the whole report, the
+  verdict with the comparison, What to fix for the filtered model, a fix
+  card, and a scenario. The HTML page now carries one inline script for
+  those buttons, with a clipboard fallback.
+
+- **Upload files to a page through the sandbox browser's live view**
+  (`actionagent`, `@activeagents/browser-sidecar`). A file chooser the page
+  opens while someone drives is answered from the driver's own machine
+  (picker or drop), over the live-view WebSocket.
+
+- **Claude Code and Codex sessions as Interactions, with chat and browser
+  handoff** (`actionagent`, `@activeagents/browser-sidecar`). A coding
+  session is now a conversation: a later message resumes the CLI session
+  (`--resume`, or `codex exec resume` for Codex), a message sent while a
+  turn runs waits for it, and
+  `POST /api/sandboxes/:sandbox_id/code_sessions/:id/messages` starts or
+  queues a turn under the new `:chat_code_session` permission.
+  `GET /api/interactions` lists agent, Claude Code and Codex interactions
+  together, and the dashboard's `/interactions/code-<id>` shows the turns
+  with a composer and the sandbox's browser beside the chat, with a hand
+  back to the agent. The sidecar mints a per-turn token for the browser
+  MCP, tells the agent once when a person took or returned control, lets
+  the agent ask for a person (`browser_request_person`), offers a person
+  the file chooser the agent left open, and names the waiting agent in the
+  live view. Every Claude Code session runs with `--strict-mcp-config`, and
+  every session gets a system note (on stdin, for Codex) that repository
+  content, tool output and pages are data.
+
+- **Codex turns reach the sandbox browser, and Codex can run on this
+  machine's own login** (`actionagent`). A Codex turn now gets the same
+  per-turn browser token as a Claude Code turn: the sidecar's MCP server
+  goes in as `--config 'mcp_servers.browser.url="…"'`, with the token in an
+  environment variable Codex reads (`bearer_token_env_var`) rather than on
+  the command line, the server's tools pre-approved
+  (`default_tools_approval_mode="approve"`, since Codex would otherwise
+  refuse every tool not marked read-only under `approval_policy="never"`),
+  and is revoked when the turn ends; hand-offs and
+  `browser_request_person` work the same. `config.codex_auth = :local_login`
+  (with the `:local` backend) runs Codex on whatever `codex login` set up
+  for the dashboard's OS user, a ChatGPT plan included, with
+  `--ignore-user-config` so none of the user's own `config.toml` reaches a
+  session; the dashboard never reads the credential and only asks `codex
+  login status` whether there is one. `GET /api/sandboxes` adds `codex_auth`
+  and, in that mode, `codex_login`, the assistant's configuration a
+  `connections.codex`, and the Codex card in Settings shows the machine's
+  login instead of a key form. The default stays `:api_key`.
+
+- **The agent builder offers only tools that work, and every built-in
+  capability has functions** (`actionagent`). The Configure step reads the
+  agent's tool roster (`GET /api/agents/tool_roster` for an agent not
+  created yet) instead of fixed chips, offers the MCP servers a dashboard
+  run can call, and names each one it cannot with the reason
+  (`MCPToolDispatcher.unreachable_reason`: a stdio server, no URL, missing
+  credentials). The six capabilities that gave the model nothing now have
+  functions: `terminal` (`run_command`), `filesystem` (`list_files`,
+  `read_file`, `write_file`), `edit` (`edit_file`) and `database`
+  (`query_database`, read-only) act in the run's checkout sandbox through
+  new `SandboxOrchestrator` verbs (on the `:local` backend; others return
+  `UnsupportedBackendError` to the model); `translate` (`translate_text`)
+  generates on the run's own provider; `slack` lists, reads and posts with
+  a bot token connected in Settings -> Integrations -> Slack (a `slack`
+  connection `ProviderKey`). Instruction sets are composed into the
+  instructions a run executes under. `run_command`, `write_file`,
+  `edit_file` and `slack_post_message` wait for approval by default: an
+  agent that gains `terminal`, `filesystem`, `edit` or `slack` — from the
+  builder, a template, the Tools tab or the agents API — has them on its
+  `approval_required_tools` unless the save names its own list, and the
+  Tools tab's Approval switch takes them off. An agent that enabled one of
+  those capabilities before this release keeps its list as it is. A host's
+  `ActionAgent.mcp_catalog` headers now reach the MCP client (they were
+  dropped).
+
+- **Start an interaction with your own agent from Interactions and Run
+  Agents** (`actionagent`). Interactions has a "New interaction" picker and
+  Run Agents a "Your agents" list; both open the agent's runner on a fresh
+  conversation (`/agents/:id/run?new=1`). The runner lists the agent's tools
+  and MCP servers, and can run against a checkout sandbox (`sandbox_id`),
+  which the checkout tools need.
+
+- **A model leaderboard on the Evaluations page** (`actionagent`). The Pass
+  rate tile listed each model's passes on a line of its own, a dozen lines
+  for seven models; it now shows the overall rate, the leading model and a
+  "compare N models" toggle, which opens a leaderboard below the tiles: one
+  row per model with its pooled passes and rate, and one column per counted
+  evaluation (`—` where the model did not run). Models rank by the lower
+  bound of the Wilson score interval (95%), so a lucky 7/8 does not outrank
+  75/173.
+
+- **Opper as an OpenAI-compatible provider** (`activeagent`). `service:
+  "Opper"` generates and embeds through Opper's compatibility endpoint
+  (`https://api.opper.ai/v3/compat`), with the key from `OPPER_API_KEY`
+  when none is configured, as the Requesty provider does (#583).
+
+- **Parallel Search in the MCP catalog** (`actionagent`). An opt-in,
+  keyless `parallel` server (`web_search`, `web_fetch`) over HTTP; queries
+  and URLs go to parallel.ai. Every MCP HTTP request now sends
+  `User-Agent: ActionAgent/<version>` unless a header names another (#575).
+
+### Fixed
+
+- **Trace totals and timeline buckets cover the whole filtered window**
+  (`actionagent`). `GET /api/traces` adds `total_count`, `returned_count`,
+  `truncated`, a `summary` (requests, errors, average latency, throughput
+  per minute) and zero-filled `buckets` computed over every trace the
+  filters match, independently of the capped detail list, and the Traces
+  view reports truncation and averages recorded request durations instead
+  of time-bucket means (part of #384).
+
+- **A pinned evaluation judge runs on its own provider** (`actionagent`).
+  A `judge_model` pin selects the model's native provider, or the one its
+  qualifier names (`openrouter/anthropic/…`), instead of the first provider
+  with credentials; the qualifier is stripped before the request, the pin
+  wins over the credential resolver's default model, and a pinned route
+  with no credentials fails the run clearly when another provider is
+  configured. Native-to-gateway alias conversion stays out of scope (#431).
+  Upgrading: a pin written `anthropic/…` or `openai/…` now selects that
+  native provider, as candidate model names already did; an install that
+  reached such a model through OpenRouter pins `openrouter/anthropic/…`.
+
+- **Streamed tool calls without an `index` are no longer split into
+  fragments** (`activeagent`). OpenAI-compatible servers (Gemini's
+  compatibility endpoint, some Ollama and OpenRouter backends) stream tool
+  calls without the `index` that ties argument fragments to their call, and
+  the Chat provider turned every fragment into its own tool call with
+  partial arguments. A fragment now continues the call with the same `id`,
+  or the last call when it has no `id` and no function name.
+
+- **A streamed generation reports why it stopped** (`activeagent`).
+  `response.finish_reason` (and `stop_reason`), and the `finish_reason` of
+  the `prompt.active_agent` event, were `nil` for every streamed Chat and
+  Anthropic generation, since a stream has no response body to read it
+  from. The Chat provider now records the chunk's `finish_reason` and the
+  Anthropic provider the `message_delta`'s `stop_reason`, and the response
+  falls back to it.
+
+- **OpenAI Responses generations report why they stopped** (`activeagent`).
+  `response.finish_reason` (and `stop_reason`) was `nil` for every
+  Responses generation, streamed or not. It is now
+  `incomplete_details.reason` when the response was cut short
+  (`max_output_tokens`, `content_filter`), else the response's `status`
+  (`completed`).
+
+- **Anthropic usage with cache token fields no longer raises** (`activeagent`).
+  `Usage.from_provider_usage` recognised Anthropic's usage only by
+  `cache_creation` or `service_tier`; a payload carrying
+  `cache_read_input_tokens` or `cache_creation_input_tokens` without them
+  fell through to `Usage.new` and raised `UnknownAttributeError`. The cache
+  token fields now mark the payload as Anthropic's (#611).
+
+## [1.9.2] - 2026-10-20
+
+Releases `activeagent` and `actionagent` 1.9.2, with the matching browser
+sidecar: evaluations at scale (the scenario suite as a build page, model
+comparison with intervals and P95 latency, opt-in parallel scenario runs,
+the judge named after its model), provider streaming fixes (Anthropic
+citations, OpenAI Chat refusals and logprobs, Responses built-in tools),
+nested `has_many` schemas, and delegation contracts from `SchemaGenerator`
+classes.
+
+Upgrading: structured-output property names are now sent to the provider,
+and answered, as declared. A `response_format` schema with multiword
+property names (`first_name`) used to reach the provider camelized
+(`firstName`) while `required` stayed snake_case, which strict mode
+rejected; both are now as declared. `parsed_json` is unaffected; code that
+reads `message.content` itself, or matches on request payloads, sees
+`first_name` where it saw `firstName`. In a Rails app, `has_many` is
+available on every `ActiveModel::Model` class, since the railtie includes
+`SchemaGenerator` there, not only on classes that include it themselves.
+Scenario fan-out is off by default: `config.scenario_evaluation_fan_out =
+true` turns it on, and its jobs run on the `:agents` queue. No new
+migration; a host still on 1.9.0 needs 1.9.1's migration 027.
+
+### Added
+
+- **P50 and P95 latency per model in evaluation reports** (`activeagent`).
+  `Report#summary_by_model` adds `p50_duration_ms` and `p95_duration_ms`
+  beside `avg_duration_ms`, by nearest rank as the dashboard's Metrics view
+  computes them, and the Markdown and HTML reports show them next to the
+  mean, which hides the tail (#428).
+
+- **Cost per 1,000 scenario runs in the Markdown report** (`activeagent`).
+  The summary table gains `Cost / 1,000`, each model's mean cost per priced
+  replay on a common denominator, marked `~` when it is estimated or
+  covers only some replays (#428).
+
+- **`ActiveAgent::Evals::Comparison`: is the gap between models real?**
+  (`activeagent`). A 95% interval for each model's pass rate and for the
+  difference between every pair of models, from 5,000 seeded resamples of
+  whole scenarios (a paired bootstrap for the pairs, over the scenarios both
+  ran). A pair is decisive when its interval excludes zero, and
+  `#conclusion` says in one sentence which pairs are, or that the ranking
+  is not decisive evidence of superiority (#428).
+
+- **The model comparison in evaluation reports** (`activeagent`).
+  `Report#comparison` and `to_h["comparison"]` carry it for a report that
+  compares models, the Markdown report gains a Comparison section, and
+  `Report.new(comparison:)` renders one already recorded for the results
+  instead of resampling, as `verdict:` does (#428).
+
+- **A ranked interval plot in the HTML evaluation report** (`activeagent`).
+  Under the comparison table, each model's pass rate is a dot inside its
+  95% interval on a fixed 0 to 100% axis, best first, with the figures
+  beside it, the conclusion, and how the intervals were computed. Plain
+  CSS on the design tokens: no script, light and dark (#428).
+
+- **Each comparison run keeps its model comparison** (`actionagent`). A
+  scenario run that compares models records `Report#comparison` under
+  `scores["_comparison"]` when it completes; a published report's is
+  worked out from the stored results rather than taken from the payload;
+  and the run's report page renders the recorded intervals instead of
+  resampling on every view. A run recorded before gets one from its
+  results (#428).
+
+- **P50 and P95 latency in the dashboard's model comparison**
+  (`actionagent`). The comparison table gains a P50 / P95 column and each
+  model's scorecard a p50 / p95 figure, from the percentiles the run
+  recorded; a run recorded before shows neither (#428).
+
+- **The model comparison plotted in the dashboard** (`actionagent`). A
+  run that compared models shows, under its comparison table, each model's
+  pass rate inside its 95% interval, best first, with the run's
+  conclusion and how the intervals were computed, from the comparison the
+  run recorded (#428).
+
+- Parallel scenario evaluations in `actionagent`: with
+  `config.scenario_evaluation_fan_out = true`, a run enqueues one
+  `EvaluationReplayJob` per scenario × model, each stored replay is scored by
+  its own `EvaluationJudgeJob`, and the last result to be scored completes the
+  run with the scores a sequential run writes. Runs as wide as the host's
+  `:agents` queue. Off by default; browser and host-adapter runs stay
+  sequential (#422).
+- `ActiveAgent::Evals::Runner.new(refine_gate:)`, a callable asked before each
+  recommendation call so per-scenario runners share one run-wide judge limit.
+
+- **Nested arrays of objects in ActiveModel schemas** (`activeagent`).
+  `has_many :addresses, class_name: "Address"` on an ActiveModel class that
+  includes `ActiveAgent::SchemaGenerator` makes `to_json_schema` emit the
+  attribute as an array of `$ref`s to the item class's schema, which goes
+  under the root `$defs`. `strict:` and `additional_properties:` reach the
+  nested schema, a nested class's own definitions move up to the root, and
+  a class that nests itself raises `ArgumentError`. A length validation on
+  the array sets its item count (`min_items` / `max_items`) rather than a
+  string's length. `has_many` also declares the attribute with an
+  empty-array default, and is added only to a class without one, so
+  ActiveRecord's is untouched (#260).
+
+- **Evaluation runs record their judge** (`actionagent`), in
+  `scores["_judge"]`: the `EvaluationJudgeAgent` it runs as, its provider,
+  the model it asked for and where that setting came from (the evaluation,
+  the provider key, the host's config, or the provider's default), and the
+  models that answered, with call counts. The suite's Models panel shows
+  "judged by <model>" with an `auto` badge when no model is pinned, and
+  opens the judge's configuration, with a field that pins `judge_model` on
+  the evaluation through `PATCH /api/evaluations/:id` (#605).
+
+
+### Changed
+
+- **A scenario suite's page reads like a build page** (`actionagent`). The
+  page opened on a summary, every run, the costs, the models and every
+  scenario stacked down the page, so on a suite of 100 scenarios and 12 runs
+  the selected run's results started three screens down. It now opens on the
+  selected run: a run header with the run's passed count, models, agent and
+  judge spend, a to-fix count and a progress bar; beside it a rail of runs,
+  each with a pass bar per model, and a box of the run's scenarios that
+  scrolls on its own, one line per step (`[+]` passed, `[!]` failed, `[~]`
+  running, `[ ]` queued, `[?]` errored, `[-]` skipped) under a sticky header
+  and foldable group rows, with a step's detail opening in place. A toolbar
+  filters the steps to failed only, puts failed first, shows the tool cells
+  and follows the running step until the person scrolls. Models, what to fix
+  and the footer follow below; a failed run shows its error in the header
+  instead of models. Selecting a run keeps the page where it is, polling
+  backs off and stops in a hidden tab, and below 940px the rail becomes a
+  strip of the latest runs. The same suite is now 2,786px tall where it was
+  11,354px (#602).
+
+### Fixed
+
+- **`max_retries` sets the Anthropic `json_object` retries** (`activeagent`).
+  `AnthropicProvider` read the count from its constructor's arguments after
+  `BaseProvider` had already moved `max_retries` into the provider's options,
+  so an unparseable `json_object` answer was always retried twice, whatever
+  `max_retries` said, on Anthropic and on Bedrock, which inherits it. The
+  count now comes from the options: `max_retries: 0` sends the request once,
+  `max_retries: 5` up to six times, and the default is still two retries
+  (#531).
+- **A streamed Anthropic answer with citations no longer raises.** Claude
+  streams the citations of a text block as `citations_delta` deltas when it
+  cites a document or a web search result, and the Anthropic provider raised
+  `Unexpected delta type: citations_delta` on the first one. The delta is now
+  skipped like `input_json_delta`: the full block, citations included, arrives
+  at `content_block_stop`.
+- **A streamed OpenAI Chat refusal, or a stream with logprobs, no longer
+  raises.** The openai gem's chat stream emits `refusal.delta` and
+  `refusal.done` when the model refuses, and `logprobs.*` events when the
+  request sets `logprobs`, and the Chat provider failed on them with
+  `Unexpected Response Event Type`. They are now handled, and `refusal.done`
+  ends the generation, since a refused message never gets a `content.done`.
+- **A streamed Responses answer that used a built-in tool no longer raises.**
+  With `tools: [{ type: "web_search_preview" }]` (or file search, image
+  generation, code interpreter, remote MCP), the stream carries the tool's
+  call as an output item (`web_search_call`, ...), its progress events
+  (`response.web_search_call.searching`, ...), and
+  `response.output_text.annotation.added` for the citations, and the
+  Responses provider raised `Unexpected Item Type: web_search_call` on the
+  first of them. The tool calls are now kept like in a non-streamed response,
+  and their events are skipped.
+- **A streamed Responses answer cut short no longer raises.** When a response
+  hits `max_output_tokens`, the stream ends with `response.incomplete` instead
+  of `response.completed`, and the Responses provider raised `Unexpected
+  Response Chunk Type: response.incomplete`. It now ends the generation with
+  the text streamed so far, like `response.completed`.
+- **A streamed Responses refusal no longer raises.** The provider raised
+  `Unexpected Response Chunk Type: response.refusal.delta` when the model
+  refused; the refusal events are now skipped and the response completes.
+- **A streamed Responses generation reports its token usage.** The usage of a
+  streamed response is only on its final event, and the Responses provider
+  never read it, so `response.usage` was `nil`. It is now recorded from
+  `response.completed` or `response.incomplete`, as the Chat provider does
+  from its usage chunk.
+- **Delegation contracts built from a `SchemaGenerator` class**
+  (`activeagent`). `delegation ..., schema: TicketForm` and
+  `returns TicketForm` produced a contract with no properties, because
+  `to_json_schema` wraps the schema in `{ name:, schema:, strict: }` and
+  the contract read only a top-level `properties`. The wrapper is now
+  unwrapped, and `$defs` are kept so the `$ref`s that nested schemas use
+  still resolve. A contract built from a class is sent with the strict
+  setting the class declared: `Model.to_json_schema` is not strict unless
+  asked, and its `required` then names only the presence-validated
+  attributes, which OpenAI's strict mode refuses, so `returns Model` goes
+  out with `strict: false` and `returns Model.to_json_schema(strict: true)`
+  strict with every attribute required.
+
+- **Structured-output schema normalization preserves field names**
+  (`activeagent`). `prompt` converts JSON Schema keywords to camel case
+  without renaming properties or definitions, keeping multiword attributes
+  and `has_many` associations consistent with `required` and `$ref` in
+  strict schemas. Literal data in `default`, `const`, `enum` and `examples`
+  is preserved too (#260).
+
+  Property names now reach the provider as declared, so a `first_name`
+  attribute is sent as `first_name` rather than `firstName`, and the model
+  answers with that key. `parsed_json` underscores keys either way, so code
+  reading it is unaffected; code reading the raw `message.content`, or
+  matching on request payloads, sees snake_case where it saw camelCase.
+
+  A delegation's `returns` schema follows suit: its `required` entries are
+  no longer camelized, so a multiword required key such as `max_points`
+  matches its property in strict mode instead of being sent as
+  `maxPoints`.
+- **A judge with no `judge_model` is labelled with the model that
+  answered** (`actionagent`), not its provider ("judged by openrouter").
+  OpenRouter's default, `openrouter/auto`, routes each call, so a run judged
+  by more than one model is labelled "<provider> default" (#605).
 
 ## [1.9.1] - 2026-10-16
 

data/README.md

@@ -57,6 +57,9 @@ bundle add openai
 # Requesty (uses OpenAI-compatible API)
 bundle add openai
 
+# Opper (uses OpenAI-compatible API)
+bundle add openai
+
 # DeepSeek (uses OpenAI-compatible API)
 bundle add openai
 

data/lib/active_agent/base.rb

@@ -26,7 +26,7 @@ module ActiveAgent
   #
   # @example Basic agent
   #   class MyAgent < ActiveAgent::Base
-  #     generate_with :openai, model: "gpt-4"
+  #     generate_with :openai, model: "gpt-5"
   #
   #     def greet
   #       prompt instructions: "Greet the user warmly"
@@ -112,7 +112,7 @@ module ActiveAgent
     # @return [void]
     #
     # @example
-    #   generate_with :openai, model: "gpt-4", temperature: 0.7
+    #   generate_with :openai, model: "gpt-5"
     def self.generate_with(provider_reference, **agent_options)
       self.prompt_provider = provider_reference
 
@@ -397,13 +397,38 @@ module ActiveAgent
 
       if response_format[:type].to_sym == :json_schema
         response_format = nest_flat_json_schema(response_format)
-        response_format[:json_schema] = prompt_view_schema(response_format[:json_schema])
-        response_format[:json_schema].deep_transform_keys! { _1.to_s.camelize(:lower) }
+        response_format[:json_schema] = normalize_json_schema(prompt_view_schema(response_format[:json_schema]))
       end
 
       response_format
     end
 
+    # Convert Ruby-style schema keywords without changing property names,
+    # definition names, references or literal data (default, const, enum,
+    # examples). Only schema-valued keywords are traversed. Named maps keep
+    # their keys, so required entries and $ref paths still address them.
+    #
+    # @api private
+    def normalize_json_schema(schema)
+      return schema.map { |item| normalize_json_schema(item) } if schema.is_a?(Array)
+      return schema unless schema.is_a?(Hash)
+
+      schema.to_h do |key, value|
+        keyword = key.to_s.camelize(:lower)
+        normalized = case keyword
+        when "properties", "patternProperties", "$defs", "definitions", "dependentSchemas", "dependencies"
+          value.is_a?(Hash) ? value.transform_values { |definition| normalize_json_schema(definition) } : value
+        when "schema", "items", "prefixItems", "additionalItems", "contains", "additionalProperties",
+             "unevaluatedItems", "unevaluatedProperties", "propertyNames", "contentSchema",
+             "allOf", "anyOf", "oneOf", "not", "if", "then", "else"
+          normalize_json_schema(value)
+        else
+          value
+        end
+        [ keyword, normalized ]
+      end
+    end
+
     # Moves a json_schema given flat, with its name, schema and strict flag
     # beside +type+, under +json_schema+, so it is not mistaken for a
     # json_schema that names a view. Without a top-level schema, or with a

data/lib/active_agent/delegation/schema.rb

@@ -56,15 +56,25 @@ module ActiveAgent
           schema
         end
 
-        # @param hash [Hash] JSON Schema object
+        # @param hash [Hash] JSON Schema object, or the +{ name:, schema:, strict: }+
+        #   wrapper {ActiveAgent::SchemaGenerator} returns around one
         # @return [Schema]
         def from_hash(hash)
           hash = hash.deep_symbolize_keys
+          strict = nil
+          if !hash.key?(:properties) && hash[:schema].is_a?(Hash)
+            # SchemaGenerator's wrapper carries +strict: true+ only when the
+            # schema was built for strict mode; otherwise it is not strict.
+            strict = hash[:strict] == true
+            hash = hash[:schema]
+          end
 
           new.tap do |schema|
             (hash[:properties] || {}).each { |name, definition| schema.property(name, definition) }
             schema.required(*Array(hash[:required]))
             schema.additional_properties(hash.fetch(:additionalProperties, hash.fetch(:additional_properties, false)))
+            schema.definitions(hash[:$defs]) if hash[:$defs]
+            schema.strict(strict) unless strict.nil?
           end
         end
       end
@@ -73,6 +83,8 @@ module ActiveAgent
         @properties           = {}
         @required             = []
         @additional_properties = false
+        @definitions           = {}
+        @strict                = nil
       end
 
       # @return [Boolean] true when nothing has been declared
@@ -158,6 +170,29 @@ module ActiveAgent
         @additional_properties = value
       end
 
+      # Declares whether the schema was built for a provider's strict mode,
+      # as the +strict:+ of the wrapper {ActiveAgent::SchemaGenerator} returns
+      # says. Strict mode wants every property in +required+; a
+      # SchemaGenerator schema names only the attributes a presence
+      # validation covers unless it was built with +strict: true+, so a
+      # class is sent as it declared itself (see #to_response_format).
+      #
+      # @param value [Boolean]
+      # @return [void]
+      def strict(value = true)
+        @strict = value
+      end
+
+      # Declares the subschemas that properties reference with
+      # +"$ref": "#/$defs/name"+, such as the nested schemas
+      # {ActiveAgent::SchemaGenerator} emits for associations.
+      #
+      # @param definitions [Hash{Symbol => Hash}]
+      # @return [void]
+      def definitions(definitions)
+        @definitions = @definitions.merge(definitions.deep_symbolize_keys)
+      end
+
       # @return [Array<Symbol>] declared property names
       def keys
         @properties.keys
@@ -177,27 +212,30 @@ module ActiveAgent
           type: "object",
           properties: @properties.deep_dup,
           required: @required.map(&:to_s),
-          additionalProperties: @additional_properties
-        }
+          additionalProperties: @additional_properties,
+          "$defs": (@definitions.deep_dup if @definitions.any?)
+        }.compact
       end
       alias_method :to_h, :to_json_schema
 
       # JSON Schema for a +response_format+ payload.
       #
-      # ActiveAgent camelizes response-format schema *keys* on the way to the
-      # provider, but the +required+ array holds string *values* — so they are
-      # camelized here to keep the emitted schema internally consistent.
-      # {Assistant#parsed_json} underscores the keys again on the way back, so
-      # agent code only ever sees snake_case.
+      # ActiveAgent sends property names to the provider as declared, so the
+      # +required+ array names them as declared too.
+      # {Assistant#parsed_json} underscores the keys of the answer, so agent
+      # code only ever sees snake_case.
       #
       # @param name [String] schema name reported to the provider
-      # @param strict [Boolean]
+      # @param strict [Boolean, nil] nil follows the source: what a
+      #   SchemaGenerator class declared (not strict unless built with
+      #   +strict: true+, since its +required+ otherwise lists only the
+      #   presence-validated attributes, which strict mode refuses), else true
       # @return [Hash]
-      def to_response_format(name:, strict: true)
-        schema = to_json_schema
-        schema[:required] = schema[:required].map { |key| key.camelize(:lower) }
+      def to_response_format(name:, strict: nil)
+        strict = @strict if strict.nil?
+        strict = true if strict.nil?
 
-        { name: name, schema: schema, strict: strict }
+        { name: name, schema: to_json_schema, strict: strict }
       end
 
       # Validates a parsed payload against the declared required properties.