phronomy 0.32.0 → 0.33.0

data/docs/shared-state-sharing.md

@@ -0,0 +1,223 @@
+# Custom sharing in SharedState
+
+SharedState can coordinate member Agents while an application owns the shared
+data and the Tools that access it. The application provides one invocation-scoped
+binding; SharedState owns member order, cycles, stopping decisions and final
+aggregation. This Experimental extension is available since v0.33.0.
+
+For example, an issue-review team can share records with `category` and `status`
+fields. Its search Tool accepts those fields as separate arguments instead of
+forcing the application to encode every record and condition as finding text.
+
+## Application provider and binding
+
+A provider scopes shared resources around one synchronous team invocation. Its
+binding contains an opaque store and a Tool-definition builder. SharedState
+passes the store to application callbacks and does not require store methods or
+a particular record schema.
+
+For example, an application provider can open an issue board, yield its binding,
+and close the board in `ensure`:
+
+```ruby
+class IssueSharing
+  def open(input:, config:)
+    board = IssueBoard.new
+    yield Phronomy::MultiAgent::SharedState::Binding.new(
+      store: board,
+      tools: ->(member:, cycle:) { build_issue_tools(board, member, cycle) }
+    )
+  ensure
+    board&.close
+  end
+end
+```
+
+This excerpt shows the provider boundary. The
+[complete executable application example](examples/shared_state_custom_sharing.rb)
+defines its store, Tool builder, member Agent, team and offline check.
+
+The provider must respond to `open(input:, config:)` and yield exactly one
+`Phronomy::MultiAgent::SharedState::Binding` synchronously during that call.
+Zero yields, multiple yields or an invalid binding violate the protocol and
+raise `ArgumentError`. Do not retain the block for deferred execution. The
+provider owns cleanup, including failures raised by member execution or
+aggregation; SharedState does not call `close` on the opaque store.
+
+`Binding.new(store:, tools:)` accepts a callable Tool builder. The public
+`binding.store` returns the opaque store, and
+`binding.tool_definitions(member:, cycle:)` calls the builder. `member` is the
+registered Agent class; `cycle` is the current positive cycle number. The builder
+must return a Hash whose keys are Tool classes and whose values are aliases or
+`nil`, using the same Tool-definition shape as the existing Agent Tool API.
+Arrays are not accepted. Invalid configuration or protocol values raise
+`ArgumentError`.
+
+## Team composition and definition identity
+
+A team selects its provider and supplies the application operations that consume
+the shared store. The provider's stable ID and version identify the meaning of
+its injected Tools independently from an individual member's definition.
+
+For example, the complete issue-review example declares:
+
+```ruby
+class ReviewTeam < Phronomy::MultiAgent::SharedState
+  sharing IssueSharing.new, id: "shared-issues", version: 1
+  member Researcher
+  max_cycles 2
+  coordination "Use search_issues to inspect relevant issues. " \
+               "Use record_issue to record each new issue with its category and status."
+  aggregate { |board| board.snapshot }
+end
+```
+
+Custom sharing requires an explicit `aggregate` block because SharedState does
+not know how to read the application store. An optional `terminate_when` block
+also receives that same store. Existing cycle order, stopping rules and result
+shape remain in place. `config:` is provided to the sharing provider; this does
+not introduce forwarding of `config:` to member invocations.
+
+In custom mode, SharedState adds only the Tools returned by the application's
+builder. It does not add `read_store` or `write_finding`, and it does not append
+stored records to member prompts. The prompt consists of coordination text, the
+task input and any member instruction. The member's existing Tools and aliases
+remain available. Conflicting effective Tool names are rejected rather than
+silently replacing an existing Tool.
+
+Use a stable sharing ID and increment its version whenever the injected Tool
+schemas or behavior change. Custom generated Agent identity incorporates that
+ID/version alongside the member definition. A new store instance per invocation
+does not by itself require a new sharing ID. The application must not substitute
+different Tool semantics under an unchanged sharing revision.
+The ID must be a nonempty String and the version must be a positive Integer.
+
+The default mode remains available when `.sharing` is not configured. It retains
+the existing invocation-local KnowledgeStore, built-in Tools, full-store prompt
+behavior and instrumentation revision 1. This extension does not add a `query`
+argument to the default `read_store` Tool or a search method to that store.
+
+## Deriving an Agent with a complete Tool set
+
+The public `Agent::Base.with_tools(definitions, id:, version:)` factory creates
+an Agent subclass with a replacement Tool map and an explicit definition
+identity. Agent owns preservation of its authoring settings; SharedState uses
+this public operation after composing the member and application Tools. It does
+not need to read Agent private settings.
+
+For example, inside the complete example's module, a provider can supply the
+entire Tool map for a derived researcher class:
+
+```ruby
+IssueSharing.new.open(input: "Review authentication", config: {}) do |binding|
+  tool_map = binding.tool_definitions(member: Researcher, cycle: 1)
+  derived = Researcher.with_tools(
+    tool_map, id: "issue-researcher-with-board", version: 1
+  )
+  # `derived` is a class; no Agent instance or Runtime resource was created.
+end
+```
+
+`definitions` is the complete Hash of Tool classes to aliases or `nil`; this
+factory does not merge Tools or detect naming collisions. SharedState performs
+those operations before calling it. A missing parent Tool is removed, and a
+`nil` alias uses the Tool's own name even when the parent assigned an alias.
+The supplied ID and version identify the derived definition directly.
+
+The factory preserves Phronomy-authored model, provider, instructions,
+ContextPolicy, generation settings, LLM-input hook and filters. Filter lists
+are independent while configured filter and callback instances remain shared.
+The original class is unchanged. This explicit factory does not change general
+Agent subclass inheritance or define copying rules for application-specific
+class state.
+
+## Application search and LLM input
+
+Application Tools own record fields and search conditions. ContextPolicy and the
+existing input-validation path continue to own LLM-input selection and budget
+checks. A custom provider does not become another Context engine.
+
+For example, `search_issues(category: "security", status: "open")` in the complete
+example applies both field conditions and returns matching records as JSON. A
+search can leave three records in the store while returning only one match. Its
+Tool parameter types and enumerations use the existing Tool schema API.
+
+The example has no result-size cap. A broad query may still produce a large Tool
+result, and this extension does not make that result optional, rewrite recorded
+Tool output or bypass the final input-budget check. Applications that need
+paging, compaction or another acquisition policy must define that behavior
+explicitly through the existing appropriate contracts.
+
+SharedState continues to use the member's configured ContextPolicy. The binding
+does not automatically populate `before_llm_input` candidates, inspect Context
+internals or manage a provider message buffer. Example 22's adaptive reader is a
+separate application implementation; this extension alone does not migrate it
+or remove its existing private-API dependency.
+
+## Lifetime and verification
+
+The provider owns resources for one synchronous invocation. SharedState does not
+define a database schema, durable binding identity or process-transfer protocol
+for the application's store. A live binding does not become a persisted
+continuation. Custom sharing requires each member invocation to reach a terminal
+state before coordination proceeds.
+
+For example, the application file can check its store and Tools without calling
+an LLM:
+
+```sh
+bundle exec ruby docs/examples/shared_state_custom_sharing.rb --offline
+```
+
+The check opens the provider, records three issues through `Tool#call`, searches
+with both category and status, verifies one match and three retained records,
+and verifies provider cleanup. It uses the real Tool argument-validation path.
+It does not exercise the SharedState Agent loop or establish live-model,
+database or multi-process acceptance.
+
+In custom mode, SharedState releases terminal member instances through the
+Agent's public `unload!` operation. A normal Agent approval wait keeps the public
+`invoke` call pending. SharedState consequently stays inside the provider's
+`open` block and retains its resources; it does not return a suspended result or
+close the provider merely because an internal execution reaches an approval
+wait. The application remains responsible for answering or cancelling approvals.
+
+For example, when a member waits for approval of `record_issue`, the board stays
+open while that invocation waits. An approval allows the Tool and member to
+complete before aggregation and provider cleanup. A cancellation that ends the
+member invocation also allows cleanup to run. The in-memory example's `ensure`
+is scoped to the end of the invocation, not the start of an approval wait. This
+is a live wait in the current process, not a way to release a Worker and restore
+the board on another one.
+
+The [SharedState integration suite](../spec/integration/shared_state_spec.rb)
+separately exercises this approval lifecycle using a real Agent and fixed LLM
+responses. It checks that the provider stays open and aggregation has not run
+during the wait; after a public approval response, the Tool writes its record,
+aggregation runs, the provider closes and the member is released. Fixed
+responses verify the execution path, not a live model's Tool-selection behavior.
+
+The lifecycle guard also detects a custom `invoke` override that returns a Hash
+with a suspended status before reaching completion. That exceptional early
+return raises `SharedState::MemberLifecycleError` with `reason: :suspended` and
+stops coordination before later members or aggregation. It is not the behavior
+of normal Agent HITL. SharedState does not approve, deny, cancel or unload the
+pending execution on the application's behalf. Providers handling this case
+must decide whether to retain their resources; merely retaining a binding adds
+no SharedState resume operation.
+
+`MemberLifecycleError#reason` also reports `:release_failed`. The error exposes
+`agent`, the available `result`, `primary_error` and `release_error`. A failed
+release preserves the Agent reference and both failure details when invocation
+and release fail together, so the application can inspect the original failure
+and decide whether to retry release. The provider's `open` scope can rescue this
+error before its cleanup policy runs. Rescuing a coordination failure inside
+`open` does not turn the invocation into success: SharedState still reports the
+failure if the provider returns normally.
+
+To run the team from an application, configure Phronomy's LLM backend normally,
+set `PHRONOMY_EXAMPLE_MODEL` to that backend's model ID before requiring the file,
+then invoke `SharedStateCustomSharingExample::ReviewTeam.new.invoke(task)`.
+The example's fallback model name is a placeholder, not a remotely available
+model. A DB-backed provider may replace the example's in-memory board, but its
+transactions, isolation and recovery behavior remain application responsibilities.

data/lib/phronomy/agent/base.rb

@@ -77,6 +77,8 @@ module Phronomy
 
         def tool_aliases
           own = @tool_aliases || {}
+          return own if @tool_aliases_replace_inherited
+
           if superclass.respond_to?(:tool_aliases)
             superclass.tool_aliases.merge(own)
           else
@@ -84,6 +86,29 @@ module Phronomy
           end
         end
 
+        # Derive an Agent definition with a complete replacement Tool map.
+        # Phronomy authoring settings and filters are preserved; mutable filter
+        # lists are independent, while configured callable instances are shared.
+        # No Agent instance or Runtime resource is created by this factory.
+        # @param definitions [Hash<Class, String, nil>] Tool classes and aliases
+        # @param id [String] identity for the derived Agent definition
+        # @param version [Integer] revision for the derived Agent definition
+        # @return [Class<Phronomy::Agent::Base>]
+        # @api public
+        def with_tools(definitions, id:, version:)
+          unless definitions.is_a?(Hash)
+            raise ArgumentError, "with_tools expects a Hash of ToolClass => alias_or_nil"
+          end
+
+          derived = Class.new(self)
+          derived.agent_definition(id: id, version: version)
+          copy_authoring_settings_to(derived)
+          copy_authoring_filters_to(derived)
+          derived.tools(definitions)
+          derived.instance_variable_set(:@tool_aliases_replace_inherited, true)
+          derived
+        end
+
         def provider(name = nil)
           if name
             @provider = name
@@ -263,6 +288,34 @@ module Phronomy
 
         private
 
+        def copy_authoring_settings_to(derived)
+          settings = {
+            model: model,
+            provider: provider,
+            instructions: instructions,
+            context_policy: context_policy,
+            temperature: temperature,
+            max_iterations: max_iterations,
+            cache_instructions: cache_instructions,
+            max_output_tokens: max_output_tokens,
+            before_llm_input: before_llm_input
+          }
+          settings.each do |name, value|
+            derived.public_send(name, value) unless value.nil?
+          end
+          # These getters inherit by ivar presence. An explicit nil preserves
+          # absence even if the source class is configured again later.
+          %i[provider instructions].each do |name|
+            derived.instance_variable_set(:"@#{name}", nil) if settings[name].nil?
+          end
+        end
+
+        def copy_authoring_filters_to(derived)
+          _class_input_filters.each { |filter| derived.input_filter(filter) }
+          _class_output_filters.each { |filter| derived.output_filter(filter) }
+          _class_tool_result_filters.each { |filter| derived.tool_result_filter(filter) }
+        end
+
         def __construct_owned_agent(owner_runtime, reserved_agent_id, *args, **kwargs, &block)
           instance = allocate
           instance.send(:__prepare_runtime_owner!, owner_runtime, reserved_agent_id)

data/lib/phronomy/agent/execution/execution_coordinator.rb

@@ -105,6 +105,7 @@ module Phronomy
         :coordinator, :operation, :delivery, :outcome, :error
       )
       DetachHostedWait = Data.define(:coordinator, :wait, :task, :parent, :completion)
+      private_constant :DetachHostedWait
       SuspensionCancelledCommand = Data.define(:coordinator, :execution_id)
 
       DeferredTerminalCommand = Data.define(
@@ -179,6 +180,16 @@ module Phronomy
         result_task
       end
 
+      # Submit the committed wait for final ownership checks on the EventLoop.
+      # @api private
+      def request_hosted_detach(wait, task:, parent:)
+        completion = Phronomy::TaskResult.deferred(name: "detach-hosted-approval")
+        command = DetachHostedWait.new(coordinator: self, wait: wait, task: task, parent: parent, completion: completion)
+        accepted = @agent.__execution_environment.registry.post(command, completion: completion)
+        raise Phronomy::RuntimeShutdownError, "EventLoop rejected hosted detach" unless accepted
+        completion
+      end
+
       # Called only from an Agent FSMSession external-operation dispatch state on
       # EventLoop. Each method captures one operation-specific immutable snapshot
       # and sends only that snapshot to OffloadPool.

data/lib/phronomy/agent/lifecycle/hosted_invocation.rb

@@ -37,15 +37,12 @@ module Phronomy
         unless wait && wait.approval_request_id == approval_request_id
           raise Phronomy::AgentBusyError, "No matching hosted approval wait"
         end
-        completion = Phronomy::TaskResult.deferred(name: "detach-hosted-approval")
         owner = @environment.registry.agent_execution_owner(execution_id)
         unless owner && (child ? owner.agent.persistence.equal?(@agent.persistence) : owner.agent.equal?(@agent))
           raise Phronomy::AgentBusyError, "Hosted execution has no matching local owner"
         end
-        command = ExecutionCoordinator::DetachHostedWait.new(coordinator: owner.coordinator, wait: wait,
-          task: (child && !@resumed_from) ? nil : @task, parent: child ? @agent : nil, completion: completion)
-        accepted = @environment.registry.post(command, completion: completion)
-        raise Phronomy::RuntimeShutdownError, "EventLoop rejected hosted detach" unless accepted
+        completion = owner.coordinator.request_hosted_detach(wait,
+          task: (child && !@resumed_from) ? nil : @task, parent: child ? @agent : nil)
         completion.wait_result
       end
 

data/lib/phronomy/agent/tool_execution/tool_approval_request.rb

@@ -101,12 +101,13 @@ module Phronomy
       end
 
       def self.build_item(invocation)
+        display = invocation.approval_display
         Item.new(
           tool_invocation_id: invocation.id,
           tool_call_id: invocation.tool_call_id,
           tool_name: invocation.tool_name,
-          arguments: invocation.display_arguments,
-          facts: invocation.display_facts,
+          arguments: display.fetch(:arguments),
+          facts: display.fetch(:facts),
           reason: invocation.authorization_reason,
           origin: invocation.origin,
           metadata: invocation.metadata

data/lib/phronomy/agent/tool_execution/tool_invocation.rb

@@ -333,14 +333,21 @@ module Phronomy
         @tool&.respond_to?(:parameters_schema) ? @tool.parameters_schema : {}
       end
 
-      def display_arguments
-        redact_for_display(@arguments || @raw_arguments)
-      end
+      # The approval Item snapshots these fields at its own construction boundary.
+      # @api private
+      def approval_display
+        return {arguments: {}.freeze, facts: {}.freeze}.freeze unless @tool&.respond_to?(:approval_display)
 
-      def display_facts
-        redact_value(@facts, sensitive_argument_values)
+        display = @tool.approval_display(arguments: @arguments || @raw_arguments, facts: @facts)
+        unless Hash === display && Hash === display.fetch(:arguments, nil) && Hash === display.fetch(:facts, nil)
+          raise Phronomy::ConfigurationError, "Tool approval_display must return :arguments and :facts Hashes"
+        end
+        display
       end
 
+      def display_arguments = immutable_copy(approval_display.fetch(:arguments))
+      def display_facts = immutable_copy(approval_display.fetch(:facts))
+
       private
 
       def restore_saved_state!(status, result)
@@ -544,37 +551,6 @@ module Phronomy
       def immutable_copy(value)
         Phronomy::Values::Immutable.copy(value)
       end
-
-      def redact_for_display(value)
-        if @tool&.respond_to?(:redacted_args, true)
-          immutable_copy(@tool.send(:redacted_args, value || {}))
-        else
-          immutable_copy(value || {})
-        end
-      end
-
-      def sensitive_argument_values
-        return [] unless @tool&.class&.respond_to?(:redact_params)
-        normalized = (@arguments || @raw_arguments || {}).transform_keys(&:to_sym)
-        @tool.class.redact_params.filter_map { |name| normalized[name] }
-      end
-
-      def redact_value(value, sensitive_values)
-        return "[REDACTED]" if sensitive_values.any? { |sensitive| sensitive == value }
-
-        case value
-        when Hash
-          value.each_with_object({}) do |(key, item), result|
-            result[key] = redact_value(item, sensitive_values)
-          end.freeze
-        when Array
-          value.map { |item| redact_value(item, sensitive_values) }.freeze
-        when String
-          "[REDACTED]"
-        else
-          value
-        end
-      end
     end
   end
 end

data/lib/phronomy/context/context_policy.rb

@@ -11,6 +11,16 @@ module Phronomy
 
       protected
 
+      # Uses the same canonical representation and estimator as generated Policy
+      # items. Estimates are planning values, not a model tokenizer guarantee.
+      # @param content [Object] text or JSON-serializable content
+      # @return [Integer] estimated token count
+      # @api public
+      def estimate_tokens(content)
+        bytes = content.is_a?(String) ? content : Phronomy::CanonicalJSON.dump(content)
+        Phronomy::LlmContextWindow::TokenEstimator.estimate(bytes)
+      end
+
       def instruction_item(
         content:,
         role: :system,
@@ -113,8 +123,7 @@ module Phronomy
       end
 
       def estimate_content(content)
-        bytes = content.is_a?(String) ? content : Phronomy::CanonicalJSON.dump(content)
-        Phronomy::LlmContextWindow::TokenEstimator.estimate(bytes)
+        estimate_tokens(content)
       end
     end
   end

data/lib/phronomy/context/context_policy_input.rb

@@ -56,6 +56,32 @@ module Phronomy
         freeze
       end
 
+      # Reports the input limit supplied to this Policy without exposing the
+      # framework's budget representation. Unknown capability remains nil.
+      # @return [Integer, nil]
+      # @api public
+      def input_token_limit
+        token_budget&.effective_input_limit
+      end
+
+      # Returns a selection input with an application limit. A known limit may
+      # only be retained or reduced. An unknown limit accepts an explicit cap
+      # without asserting that the model can accept that many tokens.
+      # @param limit [Integer] positive selection limit
+      # @return [ContextPolicyInput]
+      # @raise [ArgumentError] for an invalid limit or a known-limit increase
+      # @api public
+      def with_input_token_limit(limit)
+        unless limit.is_a?(Integer) && limit.positive?
+          raise ArgumentError, "input token limit must be a positive Integer"
+        end
+        if input_token_limit && limit > input_token_limit
+          raise ArgumentError, "input token limit must not exceed the current limit"
+        end
+
+        with(token_budget: Phronomy::LlmContextWindow::TokenBudget.new(max_input_tokens: limit))
+      end
+
       private
 
       def validate_items!(items, expected_class, label)

data/lib/phronomy/multi_agent/shared_state/binding.rb

@@ -0,0 +1,44 @@
+# frozen_string_literal: true
+
+module Phronomy
+  module MultiAgent
+    class SharedState
+      # Connects one application's shared data to the Tools offered to members.
+      # The application owns the store's shape, operations and resource lifetime.
+      # @api public
+      class Binding
+        # @param store [Object] opaque application data passed to coordination callbacks
+        # @param tools [#call] builder accepting member: (Agent class) and cycle:
+        # @api public
+        def initialize(store:, tools:)
+          raise ArgumentError, "sharing tools must respond to call" unless tools.respond_to?(:call)
+
+          @store = store
+          @tools = tools
+          freeze
+        end
+
+        # @api public
+        attr_reader :store
+
+        # Build invocation-bound Tools using the existing Agent.tools declaration shape.
+        # The builder receives the registered Agent class, not an Agent instance.
+        # @return [Hash<Class, String, nil>] Tool classes mapped to aliases or nil
+        # @api public
+        def tool_definitions(member:, cycle:)
+          definitions = @tools.call(member: member, cycle: cycle)
+          unless definitions.is_a?(Hash)
+            raise ArgumentError, "sharing tools must return a Hash of ToolClass => alias_or_nil"
+          end
+
+          definitions.to_h do |tool_class, alias_name|
+            unless tool_class.is_a?(Class) && tool_class <= Phronomy::Tool::Base
+              raise ArgumentError, "sharing tools must contain Phronomy::Tool::Base classes"
+            end
+            [tool_class, alias_name&.to_s&.dup&.freeze]
+          end.freeze
+        end
+      end
+    end
+  end
+end

data/lib/phronomy/multi_agent/shared_state/member_lifecycle_error.rb

@@ -0,0 +1,32 @@
+# frozen_string_literal: true
+
+module Phronomy
+  module MultiAgent
+    class SharedState
+      # Keeps the member handle when invocation-scoped sharing cannot finish safely.
+      # Callers choose approval, cancellation or release through existing Agent APIs.
+      # @api public
+      class MemberLifecycleError < Phronomy::Error
+        # @api public
+        attr_reader :agent, :result, :reason, :primary_error, :release_error
+
+        # The live Agent and application resources are process-local, not a
+        # durable recipe for rebuilding the sharing provider or its Tools.
+        # @api public
+        def initialize(agent:, result:, reason:, primary_error: nil, release_error: nil)
+          @agent = agent
+          @result = result
+          @reason = reason
+          @primary_error = primary_error
+          @release_error = release_error
+          message = if reason == :suspended
+            "SharedState sharing cannot complete while a member is suspended"
+          else
+            "SharedState sharing could not release a member: #{release_error&.message}"
+          end
+          super(message)
+        end
+      end
+    end
+  end
+end