activeagent 1.5.0 → 1.6.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 7401cdff6a7895238a082383f2ef32683b743946445e11dacbf377470bf31071
-  data.tar.gz: b0f847d04eedf620c7f5320950d65e5ffc712296ef7db2b7ea6be25f3bc433b7
+  metadata.gz: 1838cd05b12c4bc56330e8c4b4ec98295e885990bc67b600fefaa401f2543593
+  data.tar.gz: dcd3590de72aef1a8059a6fb03d9303672e73260a59c33e9244ccf326d48b7b6
 SHA512:
-  metadata.gz: 0a29cb99c3139f9c264eb6e86677d47ed592fb165c39489be2efc8ea2bb2dd1614bc8802643b812b04319b7356bc37177943bc9c2e0174de276555ba811ba426
-  data.tar.gz: 6fb392e7cb2f4381d79ae16567edca55d62c585b06f04a6a3da6b31cd0bb7e6149b2e67ceb7e5e143b04bc41123dd1d5bba14f38b33be6e21813e9923308d524
+  metadata.gz: 1534a469b0eeec15765b71e7b43e8362660d9fc7610531c757acb882e130a95cc938a2033b2d0982233558123a0b162d6efb9c51da3e7038d0f0f1d6fcfe8d9e
+  data.tar.gz: eb849f183afd357f381836947891c8029d5ca4772f2e4bf1fda3fedb30ebb7f71db1021f9f94a8ba62337a99493b54b0124f9f0e28d8342fde4182d8ebf02411

data/CHANGELOG.md

@@ -7,6 +7,221 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [1.6.0] - 2026-09-14
+
+Releases `activeagent` and `actionagent` 1.6.0 from one tag.
+
+A minor, not a patch. The cycle that began after 1.5.2 gives an agent a
+caller — `current_user`, carried from whatever authenticated the call into
+every `before_action`, every tool, every delegated sub-agent, every run over
+MCP and every evaluation replay — so an authorization gem has something to
+decide against. Around it: schema tools defined at runtime rather than only
+in a file, a generator that writes the first one, those tools served
+directly over MCP, and an evaluation that calls a fabricated answer a fault
+instead of grading it as an honest gap. That is new public surface in both
+gems, which is a minor under semver even though 1.5.2 shipped a feature as a
+patch.
+
+Two notes for upgrades. `tools_succeeded` is now awarded only for a tool the
+scenario expected, so a suite that was quietly scoring wrong-tool runs as
+partial successes will report lower — read the first run as a corrected
+baseline. And `actor:` is now stripped from tool arguments and from
+`params[params][actor]`: the caller is a property of the run, set once by
+whatever authenticated it, and can no longer be named by the model or by a
+client.
+
+The engine's floor on the framework (`activeagent >= 1.4`) is unchanged and
+still correct: 1.6.0 satisfies it.
+
+### Added
+
+- **An evaluation replay runs as the evaluation's owner.** The scenario runner
+  handed `Agent#test_execute` no caller, so every tool a replay called ran
+  unattributed and a host scope answered empty — the suite graded an agent
+  that never saw a row. When agents are owned per user, a replay now runs as
+  the user who owns the evaluation, as a run over MCP runs as the key's
+  owner. A multi-tenant install still replays unattributed (an account is who
+  is billed, not who is allowed) unless a host adapter runs the suite itself.
+- **The MCP facade serves the host's schema tools directly.** `tools/list`
+  at `POST <mount>/mcp` now offers every tool the dashboard's discovered
+  `ActiveAgent::SchemaTools` classes generate — `find_<records>`,
+  `count_<records>`, `get_<record>` — beside the `run_<slug>` agents, each
+  with its own parameter schema, and `tools/call` runs one as the key's
+  caller through the host's own scope, exactly as it would inside an agent
+  run. A client that only needs the rows no longer has to ask an agent for
+  them. A boundary violation is a tool result with `isError`, a refusal from
+  the host's scope is a JSON-RPC `-32003`, and neither the execution switch
+  nor the execution quota applies, because nothing generates. Set
+  `ActionAgent.mcp_schema_tools = false` to keep the tools reachable only
+  through agents. Closes #439.
+- **A delegated run inherits its parent's caller.** `delegate_to` hands the
+  sub-agent the parent's `current_user` before its action runs, so its own
+  `before_action` callbacks and any scope its tools read through decide
+  against the same person. A parent authorized as one user no longer hands
+  its specialists an unattributed run — which a correctly written host scope
+  reads as "no access", a wrong answer wearing a right one's clothes. A
+  parent with no caller still delegates an unattributed run, never someone
+  else's.
+- **`rails generate active_agent:schema_tools Reservation`** writes a starter
+  `ActiveAgent::SchemaTools` class under `app/agent_tools`. It exposes nothing
+  beyond `id` until a column is moved into `filterable` or `returns`; every
+  column the model has is listed, commented out, with its type, so the
+  allowlist is a review step rather than a blank page, and columns that look
+  like secrets are left off the list. Reads are scoped through
+  `<Model>Policy::Scope` when it exists (`--policy` / `--no-policy` decide
+  explicitly). This is #440's second option: the roster is still declared,
+  once, but the declaration is no longer written from scratch. (#440)
+- **An agent knows who it is running for, so an authorization gem has
+  something to decide against.** `ActiveAgent::Base#current_user` carries the
+  caller, assigned by whatever authenticated the call
+  (`MyAgent.as(current_user).ask(...)`) and readable from every
+  `before_action`, so Pundit, CanCanCan or Action Policy authorize an agent
+  the way they authorize a controller. A refusal inside a tool call is
+  returned to the model as `{ error: ... }` so it can say it is not allowed;
+  a refusal anywhere else is raised to the caller. `denies_with` names a
+  gem's own error as a refusal.
+- **The dashboard fills that seam.** A run records the caller that started it
+  (a Global ID, so a worker on another machine authorizes as the same
+  person), passes it to every tool as `actor:`, and runs the agent through
+  `as(actor)` — which is what a `SchemaTools` `scope` block has been waiting
+  for since 1.5.0. `ActionAgent.agent_actor_resolver` overrides who that is.
+- **Agents reached over MCP run as the key's caller** rather than
+  unattributed, and an agent that refuses answers as a JSON-RPC error
+  (`-32003`) instead of as an empty result.
+- **Schema tools can be defined at runtime.** `ActiveAgent::SchemaTools.define(Reservation,
+  filterable:, returns:, scope: | policy:)` builds the same bounded roster a
+  file under `app/agent_tools` would — same allowlists, same `call`, named
+  `ReservationTools` for logs — from a declaration held anywhere: a table, a
+  dashboard form, a test. The class is registered under its model and a
+  redefinition replaces the previous one, so a registry rebuilt on every
+  change holds one class per model; `undefine` drops it. The dashboard
+  discovers registry entries beside the files and lets a runtime definition
+  supersede a file for the same model. What is persisted, and where, stays the
+  host's decision; this is the seam a persisted declaration builds on. (#441)
+- **A fabricated answer is now a fault.** `Diagnosis` raises `ungrounded_answer`
+  when an agent that had tools called none, did not say it could not answer,
+  and still stated specifics — a count, a record id, a date — that no tool
+  supplied. Where the scenario names an expected tool, `expected_tool_not_called`
+  says the same thing in its summary and carries `ungrounded: true`, so an
+  invented answer no longer reads like an honest gap. Both reach the judge,
+  which is what turns them into a suggested tool. An agent with no tools at all
+  is not flagged: it answers from its instructions by design, and whether that
+  is acceptable is the judge's grade, not a mechanical one. (#433)
+
+### Changed
+
+- **A wrong tool no longer outscores no tool.** `tools_succeeded` is awarded
+  only for a tool the scenario expected (or any tool when it expects none):
+  a tool that ran without erroring was evidence of the task only by accident,
+  and a scenario that called the wrong tool scored higher than one that called
+  nothing. (#433)
+- **The judge reads more of a scenario's notes** — 1,500 characters rather
+  than 300 — because a suite's notes are often its rubric and the "must not"
+  clause tends to come last. (#433)
+
+### Fixed
+
+- **The caller can no longer be named by the model, or by the client.**
+  `actor:` reached `AgentToolbox.call` in the same keyword namespace as the
+  arguments a provider parsed out of a model's tool call, and
+  `params[params][actor]` in an execute request would have won over the
+  controller's own. Both are now stripped: the caller is a property of the
+  run, set once by whatever authenticated it. Scoped `SchemaTools` reads are
+  also excluded from the tool-result cache, so one caller's rows are never
+  replayed for the next.
+- **A superseded runtime tool class is no longer offered twice.** Discovery
+  read every `SchemaTools` subclass out of `descendants`, where a class built
+  at runtime stays until it is collected, so rebuilding a model's tools
+  accumulated stale duplicates. Runtime-built classes are now read from the
+  registry only. (#441)
+- **A persisted model selection re-runs under the provider it ran under.**
+  `Evals::ModelSpec.parse_all` re-parsed a round-tripped spec from its label,
+  so `anthropic/claude-sonnet-4.5` run through OpenRouter came back as
+  Anthropic's own `claude-sonnet-4.5` as soon as that provider's gem was
+  installed — and the re-run failed for want of an Anthropic credential. A
+  hash naming both `provider` and `model` is now rebuilt as it was; a bare
+  label is still parsed. The dashboard's "re-run" of a saved selection is
+  the path this fixes.
+
+## [1.5.2] - 2026-09-11
+
+Releases `activeagent` and `actionagent` 1.5.2 from one tag.
+
+**1.5.1 was never tagged.** Its version bump reached `main`, but three PRs
+that change `actionagent` merged alongside it, and that release deliberately
+held `actionagent` at 1.5.0 — publishing it would have shipped the fix below
+while leaving every dashboard change of this cycle unpublished, because
+`release.yml` skips a version already on RubyGems. 1.5.2 supersedes it and
+carries both gems. The 1.5.1 notes are kept below as the record of what that
+bump contained.
+
+### Added
+
+- **Schema-derived agent tools.** `ActiveAgent::SchemaTools` turns an
+  ActiveRecord model plus a declared boundary into a bounded, enumerable tool
+  roster — `find_*`, `count_*`, `get_*` — with `filterable` and `returns`
+  allowlists. An undeclared column is rejected rather than silently dropped:
+  ignoring an unknown filter answers a broader question than was asked while
+  still looking like success. Results are capped (25 default, 100 max) with a
+  truncation marker the model can see. (#435)
+
+- **Host tools reach the dashboard.** `ActionAgent.schema_tools` offers each
+  generated tool beside `AgentToolbox`'s built-ins: individually selectable in
+  the agent editor, dispatched by name at execution, and nameable in an
+  evaluation's `tools:` expectation. Previously a declared schema tool was
+  invisible — `definitions_for` returned nothing, the model received no
+  schemas and invented tool names in prose while the run scored 0.0 for what
+  looked like a model failure. (#435, closes #438)
+
+- **Tools are discovered, not declared twice.** Leave `schema_tools` unset and
+  every subclass under `schema_tools_path` (`app/agent_tools`) is offered.
+  Adding a tool is adding a file. Anonymous classes are excluded from
+  discovery: a runtime-built class cannot supersede itself, so it would
+  accumulate one per reload. (#435, refs #440)
+
+- **`scope_by_policy`** resolves a model's policy by name —
+  `Reservation` → `ReservationPolicy::Scope` — instead of hand-writing the
+  block. Opt-in, because silently scoping a class that declared none would
+  change what an existing tool returns; a missing policy raises at declaration
+  rather than quietly reading the whole table. (#435)
+
+- **A model's agent starts with that model's tools.** `ReservationAgent` is
+  seeded from `ReservationTools` on create. A default, never a restriction:
+  any agent may enable any tool, and an explicit selection — including a
+  deliberate empty one — is never overwritten. (#435)
+
+### Fixed
+
+- **MCP tool-discovery failures are reported instead of running tool-less and
+  silent.** `MCPToolDispatcher#tool_definitions` rescued a failed `tools/list`
+  to `[]`, so a server that 401s and one that legitimately serves no tools
+  were indistinguishable: the agent ran without tools, the model fabricated,
+  and the report offered prompt advice for what was a transport failure.
+  `discovery_errors` now names the server, its URL and the underlying error,
+  and `all_servers_failed?` lets a caller fail loudly rather than grade an
+  invented answer. (#434, closes #425)
+
+- **Nil VCR filters no longer flake replays**, and the MCP plural is spelled
+  correctly. (#436)
+
+## [1.5.1] - 2026-09-11 [UNRELEASED — superseded by 1.5.2]
+
+Bumped `activeagent` to 1.5.1 and held `actionagent` at 1.5.0. Never tagged;
+its contents ship in 1.5.2.
+
+### Fixed
+
+- **A spec hash names its model rather than reaching the provider as an
+  inspected Hash.** `Evals::ModelSpec.parse_all` called `to_s` on each value,
+  so a Hash travelled as the model ID and the provider answered
+  `{"label" => "openrouter/openai/gpt-4o-mini", ...} is not a valid model ID`
+  — every scenario of the run failing before it reached the model. A caller
+  passing a plain string was unaffected, which is why a single run worked
+  while a whole suite failed. The path is reachable by design rather than by
+  misuse: a run persists its models as `specs.map(&:to_h)`, so re-running that
+  selection hands the hashes back. `parse_all` now reads a hash's `label`,
+  then its `model`, and leaves strings alone.
+
 ## [1.5.0] - 2026-09-10
 
 Releases `activeagent` 1.5.0 and `actionagent` 1.5.0 from one tag.

data/lib/active_agent/base.rb

@@ -4,6 +4,7 @@ require "active_support/core_ext/hash/except"
 require "active_support/core_ext/module/anonymous"
 require "active_support/core_ext/string/inflections"
 
+require "active_agent/concerns/authorization"
 require "active_agent/concerns/callbacks"
 require "active_agent/concerns/delegation"
 require "active_agent/concerns/observers"
@@ -43,6 +44,7 @@ module ActiveAgent
     include AbstractController::Caching
 
     include Callbacks
+    # After Rescue: the refusal handler is registered with rescue_from.
     include Delegation
     include Parameterized
     include Provider
@@ -52,6 +54,11 @@ module ActiveAgent
     include Tooling
     include View
 
+    # Last of the behaviour concerns: its rescue_from must sit on top of the
+    # handlers an agent registers for its own authorization gem, so a host
+    # mapping Pundit::NotAuthorizedError keeps deciding what a refusal means.
+    include Authorization
+
     include Observers
     include Previews
 

data/lib/active_agent/concerns/authorization.rb

@@ -0,0 +1,172 @@
+# frozen_string_literal: true
+
+module ActiveAgent
+  # Raised when an agent refuses a call on the current caller's behalf.
+  #
+  # Hosts usually raise their authorization gem's own error instead
+  # (+Pundit::NotAuthorizedError+, +CanCan::AccessDenied+) and name it with
+  # {Authorization::ClassMethods#denies_with}; this exists so an agent that
+  # has no gem still has something to raise, and so the framework has one
+  # class to describe a refusal with.
+  class NotAuthorized < StandardError
+    # @return [Symbol, String, nil] the action or tool that was refused
+    attr_reader :action
+    # @return [Object, nil] the caller the refusal was decided against
+    attr_reader :actor
+
+    def initialize(message = nil, action: nil, actor: nil)
+      @action = action
+      @actor = actor
+      super(message || default_message)
+    end
+
+    private
+
+    def default_message
+      who = actor.nil? ? "an unauthenticated caller" : "the current caller"
+      what = action ? "`#{action}`" : "this agent"
+      "#{who} is not allowed to call #{what}"
+    end
+  end
+
+  # Carries the caller an agent runs on behalf of, so an agent's callbacks can
+  # authorize with whatever the host app already uses.
+  #
+  # An agent reached over MCP, from a dashboard run, or from a controller is
+  # acting *for someone*. Without that someone, an authorization gem has
+  # nothing to decide against: a Pundit scope handed +nil+ correctly resolves
+  # to the empty set, so a perfectly wired agent answers "there are no
+  # tickets" instead of refusing — a wrong answer that reads like a true one.
+  #
+  # == The seam
+  #
+  # {#current_user} is assigned by whatever authenticated the call and is
+  # readable from every callback and action:
+  #
+  #   class TicketAgent < ApplicationAgent
+  #     before_action :authorize_tickets!
+  #
+  #     def find_tickets(**filters)
+  #       TicketTools.call("find_tickets", actor: current_user, **filters)
+  #     end
+  #
+  #     private
+  #
+  #     def authorize_tickets!
+  #       raise ActiveAgent::NotAuthorized.new(action: action_name, actor: current_user) unless
+  #         TicketPolicy.new(current_user, Ticket).index?
+  #     end
+  #   end
+  #
+  #   TicketAgent.as(current_user).find_tickets.generate_now
+  #
+  # Any gem works, because the framework never interprets the actor — Pundit's
+  # +authorize+/+policy_scope+, CanCanCan's +can?+, Action Policy's
+  # +authorize!+, or a plain predicate. +before_action+ is the same
+  # +AbstractController+ chain a controller uses, including +only:+/+except:+,
+  # so a roster can be authorized tool by tool.
+  #
+  # == It is never model input
+  #
+  # The actor is an attribute of the run, set out of band by the caller. It is
+  # deliberately not part of +params+ and not a tool argument: everything a
+  # model emits is attacker-reachable through the documents it reads, and an
+  # actor a model can name is not an authorization boundary. Assigning it is
+  # the caller's job, once, before the generation starts.
+  #
+  # == What a refusal does
+  #
+  # A refusal inside a *tool call* is returned to the model as an error result
+  # ({ error: ... }), so it can tell the user it is not allowed to look rather
+  # than dying mid-run or, worse, reporting an empty result set as fact. A
+  # refusal anywhere else — the action the caller asked for — is raised, so
+  # the MCP client or controller that asked gets an error instead of an
+  # answer that silently covers less ground than it appears to.
+  #
+  # {ClassMethods#denies_with} is how a gem's own error joins that rule:
+  #
+  #   class ApplicationAgent < ActiveAgent::Base
+  #     denies_with Pundit::NotAuthorizedError
+  #   end
+  module Authorization
+    extend ActiveSupport::Concern
+
+    included do
+      # The caller this generation runs on behalf of, or nil when it runs
+      # unattributed. Whatever the host uses as an actor: a User, an API
+      # key's owner, a service account.
+      attr_accessor :current_user
+
+      # Exception classes that mean "the caller may not do this". Declared
+      # rather than guessed: only the host knows which of its errors are a
+      # refusal and which are a bug.
+      class_attribute :authorization_errors, instance_writer: false,
+        default: [ ActiveAgent::NotAuthorized ].freeze
+    end
+
+    class_methods do
+      # Treats +classes+ as refusals, so raising one inside a tool call
+      # reports to the model instead of ending the run.
+      #
+      # @param classes [Array<Class>] exception classes from an authorization gem
+      # @return [Array<Class>] every class now treated as a refusal
+      #
+      # @example
+      #   denies_with Pundit::NotAuthorizedError, CanCan::AccessDenied
+      def denies_with(*classes)
+        self.authorization_errors = (authorization_errors | classes.flatten).freeze
+      end
+
+      # Runs the agent on behalf of +actor+.
+      #
+      # @param actor [Object, nil] the caller, or nil to run unattributed
+      # @return [ActiveAgent::Parameterized::Agent] a proxy carrying the actor
+      #
+      # @example
+      #   SupportAgent.as(current_user).answer(question).generate_now
+      #
+      # @example With parameters
+      #   SupportAgent.as(current_user).with(locale: :en).answer(question)
+      def as(actor)
+        ActiveAgent::Parameterized::Agent.new(self, {}, actor: actor)
+      end
+    end
+
+    # Whether a refusal right now would be reported to the model rather than
+    # raised to the caller.
+    # @return [Boolean]
+    def tool_call?
+      @_active_agent_tool_call ||= false
+    end
+
+    # Marks the block as a tool call, so a refusal inside it becomes a result
+    # the model can read. Nested calls keep the outer marking.
+    # @api private
+    def with_tool_call
+      previous = @_active_agent_tool_call
+      @_active_agent_tool_call = true
+
+      result = yield
+      # A host's own rescue_from handler runs inside `process`, and
+      # ActiveSupport::Rescuable answers with the exception itself — so a
+      # refusal arrives either raised or returned, and both mean the same
+      # thing here.
+      refusal?(result) ? refused(result) : result
+    rescue *authorization_errors => exception
+      refused(exception)
+    ensure
+      @_active_agent_tool_call = previous
+    end
+
+    private
+
+    def refusal?(value)
+      value.is_a?(Exception) && authorization_errors.any? { |klass| value.is_a?(klass) }
+    end
+
+    def refused(exception)
+      logger&.info("[#{self.class.name}] refused #{action_name}: #{exception.message}")
+      { error: exception.message }
+    end
+  end
+end

data/lib/active_agent/concerns/parameterized.rb

@@ -110,9 +110,28 @@ module ActiveAgent
     class Agent
       # @param agent [Class] the agent class to proxy
       # @param params [Hash] the parameters to pass to agent instances
-      def initialize(agent, params)
+      # @param actor [Object, nil] the caller the generation runs on behalf
+      #   of (see ActiveAgent::Authorization). Kept beside params rather than
+      #   in them: params reach templates and are part of what a generation
+      #   is *about*, while the actor is who it is *for*, and only the caller
+      #   may say.
+      def initialize(agent, params, actor: nil)
         @agent = agent
         @params = params
+        @actor = actor
+      end
+
+      # Adds parameters, keeping the actor. Lets +as+ and +with+ chain in
+      # either order.
+      # @return [ActiveAgent::Parameterized::Agent]
+      def with(params = {})
+        self.class.new(@agent, @params.merge(params), actor: @actor)
+      end
+
+      # Runs on behalf of +actor+, keeping any parameters.
+      # @return [ActiveAgent::Parameterized::Agent]
+      def as(actor)
+        self.class.new(@agent, @params, actor: actor)
       end
 
       # Intercepts calls to agent action methods and creates parameterized generations.
@@ -124,7 +143,9 @@ module ActiveAgent
       # @raise [NoMethodError] if the method doesn't exist on the agent class
       def method_missing(method_name, ...)
         if @agent.public_instance_methods.include?(method_name)
-          ActiveAgent::Parameterized::Generation.new(@agent, method_name, @params, ...)
+          ActiveAgent::Parameterized::Generation.new(@agent, method_name, @params, ...).tap do |generation|
+            generation.actor = @actor
+          end
         else
           super
         end
@@ -146,6 +167,17 @@ module ActiveAgent
     #
     # @api private
     class Generation < ActiveAgent::Generation
+      # @param agent_class [Class] the agent class
+      # @param action [Symbol, String] the action method name
+      # @param params [Hash] the parameters to set on the agent instance
+      # @param args [Array] additional arguments for the action method
+      # The caller the generation runs on behalf of. Assigned after
+      # construction rather than taken as a keyword, because the action's own
+      # arguments are forwarded here and an actor keyword would collide with
+      # one of the same name.
+      # @return [Object, nil]
+      attr_accessor :actor
+
       # @param agent_class [Class] the agent class
       # @param action [Symbol, String] the action method name
       # @param params [Hash] the parameters to set on the agent instance
@@ -163,6 +195,7 @@ module ActiveAgent
       def agent
         @agent ||= agent_class.new.tap do |agent|
           agent.params = @params
+          agent.current_user = @actor
           agent.process(action_name, *args, **kwargs)
         end
       end
@@ -180,8 +213,12 @@ module ActiveAgent
         if processed?
           super
         else
+          # The actor rides as an ordinary job argument, which ActiveJob
+          # serializes through GlobalID like any other record, so a worker on
+          # another machine authorizes as the same caller.
           agent_class.generation_job.set(job_options).perform_later(
-            agent_class.name, action_name.to_s, generation_method.to_s, params: @params, args: args, kwargs: kwargs
+            agent_class.name, action_name.to_s, generation_method.to_s,
+            params: @params, args: args, kwargs: kwargs, actor: @actor
           )
         end
       end

data/lib/active_agent/concerns/tooling.rb

@@ -16,7 +16,9 @@ module ActiveAgent
     # @return [Proc] callback proc that accepts (action_name, *args, **kwargs)
     def tools_function
       proc do |action_name, *args, **kwargs|
-        process(action_name, *args, **kwargs)
+        # Marked as a tool call so a refusal is reported to the model as a
+        # result rather than raised through the run (see Authorization).
+        with_tool_call { process(action_name, *args, **kwargs) }
       end
     end
   end

data/lib/active_agent/delegation/runner.rb

@@ -117,6 +117,7 @@ module ActiveAgent
       def generate(arguments)
         agent = definition.resolved_agent_class.new
         agent.params = resolved_params(arguments)
+        inherit_actor(agent)
         agent.process(definition.action, **arguments)
 
         definition.backend.apply(agent)
@@ -126,6 +127,22 @@ module ActiveAgent
         agent.process_prompt
       end
 
+      # A delegated generation runs on behalf of whoever the parent runs for.
+      # The sub-agent gets the parent's caller before its action runs, so its
+      # own before_action callbacks and any scope its tools read through decide
+      # against the same person — a parent authorized as one user must not
+      # hand its specialists an unattributed run, which a correctly written
+      # host scope reads as "no access". Hosts on an older framework, where an
+      # agent has no caller to carry, are left as they were.
+      #
+      # @param agent [ActiveAgent::Base]
+      # @return [void]
+      def inherit_actor(agent)
+        return unless owner.respond_to?(:current_user) && agent.respond_to?(:current_user=)
+
+        agent.current_user = owner.current_user
+      end
+
       # A delegated generation is part of its parent's work, so it carries the
       # parent's trace id — otherwise the sub-agent's tokens and latency land
       # in a separate trace and the budget you set has nothing to show for it.

data/lib/active_agent/evals/diagnosis.rb

@@ -12,6 +12,7 @@ module ActiveAgent
     #   tool_error               — a tool the agent called returned an error
     #   missing_capability       — the agent said no tool covers the task
     #   expected_tool_not_called — the scenario expects a tool the agent did not call
+    #   ungrounded_answer        — the answer states specifics no tool call supplied
     #   forbidden_content        — the answer contains a pattern the scenario forbids
     #   missing_content          — the answer lacks a pattern the scenario expects
     #   low_quality              — the answer scored below the threshold
@@ -21,7 +22,7 @@ module ActiveAgent
     # Returns nil for a passing result.
     class Diagnosis
       FAULTS = %w[
-        run_error tool_error missing_capability expected_tool_not_called
+        run_error tool_error missing_capability expected_tool_not_called ungrounded_answer
         forbidden_content missing_content low_quality judge_unavailable
       ].freeze
 
@@ -38,6 +39,19 @@ module ActiveAgent
         /\bcan(?:'|no)t (?:be )?(?:done|determined|answered) with (?:the|my) (?:current|available) tools\b/i
       ].freeze
 
+      # Phrasings that state a specific fact — a record id, a date, a count of
+      # things — which an agent that called no tool can only have invented.
+      # Deliberately narrow: a number inside prose ("here are three options",
+      # "within 30 days") is not a claim about data, and a false positive here
+      # fails a scenario that may have passed on its merits.
+      SPECIFIC_CLAIMS = [
+        /#\d+\b/,
+        /\b\d{4}-\d{2}-\d{2}\b/,
+        /\b(?:you have|there are|there is|we have|I found|found|showing|a total of)\s+(?:\*\*)?\d+\b/i,
+        /\b\d+\s+(?:\*\*)?(?:open|overdue|pending|active|closed|resolved|completed|unpaid|outstanding|new|matching|
+          records?|results?|rows?|entries|items?|tickets?|orders?|tasks?|issues?|invoices?|customers?|users?|milestones?)\b/ix
+      ].freeze
+
       Result = Struct.new(:fault, :summary, :recommendation, :evidence, keyword_init: true) do
         def to_h
           {
@@ -76,7 +90,7 @@ module ActiveAgent
       end
 
       def call
-        run_error || tool_error || missing_capability || expected_tool_not_called ||
+        run_error || tool_error || missing_capability || expected_tool_not_called || ungrounded_answer ||
           forbidden_content || missing_content || low_quality
       end
 
@@ -178,16 +192,58 @@ module ActiveAgent
             "#{agent} answered with #{called_tools.uniq.join(', ')} instead of #{expected.join(', ')}. Sharpen " \
             "both tools' descriptions so the model can tell them apart, or say in the instructions which tool " \
             "answers this kind of task."
+          elsif asserts_specifics?
+            "#{expected.join(', ')} is available but #{agent.downcase} answered without calling any tool and " \
+            "stated specifics it could not have looked up (\"#{claim_excerpt}\"). Treat the answer as invented: " \
+            "instruct it to answer this kind of task only from a tool result, and to say so when it has none."
           else
             "#{expected.join(', ')} is available but #{agent.downcase} answered without calling any tool. Tell " \
             "it in the instructions to prefer tool-backed answers for this kind of task, and check the tool's " \
             "description says what it returns."
           end
 
-        result("expected_tool_not_called",
-               "Expected #{expected.join(' or ')} to be called; #{agent.downcase} called " \
-               "#{called_tools.uniq.presence&.join(', ') || 'nothing'}.",
-               recommendation, "expected" => expected, "called" => called_tools, "unavailable" => unavailable)
+        summary = "Expected #{expected.join(' or ')} to be called; #{agent.downcase} called " \
+                  "#{called_tools.uniq.presence&.join(', ') || 'nothing'}"
+        summary += " and answered with specifics no tool supplied" if called_tools.empty? && asserts_specifics?
+
+        result("expected_tool_not_called", "#{summary}.", recommendation,
+               "expected" => expected, "called" => called_tools, "unavailable" => unavailable,
+               "ungrounded" => (called_tools.empty? && asserts_specifics?) || nil, "claim" => (claim_excerpt if called_tools.empty?))
+      end
+
+      # The answer states specifics — a count, an id, a date — that no tool
+      # call could have supplied. Reached only when the scenario names no
+      # expected tool (expected_tool_not_called reports the same fabrication
+      # otherwise) and only for an agent that had tools to call: one with
+      # none answers from its instructions by design, and whether that is
+      # acceptable is the judge's call, not a mechanical one.
+      def ungrounded_answer
+        return nil if @available_tools.empty? || called_tools.any?
+        return nil unless asserts_specifics?
+
+        result("ungrounded_answer",
+               "#{agent} stated specifics (\"#{claim_excerpt}\") without calling any tool that could have supplied them.",
+               "Nothing in the answer came from a tool, so the figures in it are invented. Tell #{agent.downcase} in its " \
+               "instructions to answer this kind of task only from a tool result and to say when it has none; if none of " \
+               "#{@available_tools.join(', ')} returns this data, add a tool that does.",
+               "claim" => claim_excerpt, "tools_available" => @available_tools)
+      end
+
+      def asserts_specifics?
+        specific_claim.present?
+      end
+
+      def specific_claim
+        return @specific_claim if defined?(@specific_claim)
+
+        @specific_claim = SPECIFIC_CLAIMS.lazy.filter_map { |pattern| answer.match(pattern) }.first
+      end
+
+      def claim_excerpt
+        match = specific_claim
+        return nil unless match
+
+        answer[[ match.begin(0) - 40, 0 ].max, 120].to_s.strip
       end
 
       def forbidden_content