phronomy 0.23.0 → 0.24.0

data/docs/migrations/durable-semantic-coordination-v2.md

@@ -0,0 +1,65 @@
+# Durable semantic coordination V2 migration
+
+This Beta clean break applies to baseline `5472116cd99a63ec27875024c955ea82be612d6b`.
+
+| Previous surface | Current surface |
+|---|---|
+| `MultiAgent::Handoff`, `HandoffPolicy`, `Runner` | `Agent::Handoff`, `HandoffPolicy`, `HandoffRunner` (no aliases) |
+| Runtime-local Handoff routing | HandoffState at original main Agent ID; same Persistence domain required |
+| Anonymous Team/new Runtime queue | Explicit `team_definition id:, version:` and stable `team_id`; TeamRoot/TeamExecution |
+| Arbitrary aggregate object/error exception result | Pure replay-safe aggregate, canonical JSON result/error data |
+| Five backend repositories | Eight mandatory repositories; extend root and transaction views |
+| Direct standalone fan-out | Remains Runtime-only; no synthetic parent AgentExecution |
+
+```ruby
+class WorkTeam < Phronomy::MultiAgent::TeamCoordinator
+  team_definition id: "work-team", version: 1
+  coordinator_model "gpt-4o-mini"
+  coordinator_provider :openai
+  pool size: 2, agent: WorkerAgent
+  aggregate { |assignments| assignments.map { |a| a[:result] }.compact.join("\n") }
+end
+
+team = WorkTeam.create(team_id: "team-42", persistence: store)
+value = team.invoke("Prepare the report")
+runs = store.list_team_executions("team-42")
+run_id = runs.first.team_execution_id
+retained = store.team_execution_result(run_id)
+# After a restart with the same declared Team and worker definitions:
+team = WorkTeam.load("team-42", persistence: store, on_event: recovery_listener)
+value = team.resume(run_id)
+```
+
+`resume` consumes a stored terminal outcome without running the aggregator again.
+For unfinished runs, stored input/context/assignments and child reservations win.
+Compatible current Agent/Team classes, static registrations and Handoff graph are
+execution wiring, not serialized objects. Version matching checks declarations;
+Application must change versions for incompatible behavior and retain compatible
+wiring for old runs. It is not a source-code hash or automatic migration system.
+
+Use `Persistence#execution_result(id)` and owner-scoped `list_executions` for Agent
+results without hydration/callbacks. `Persistence#handoff_result(source_id)`
+follows the exact transfer chain without constructing a graph or Agent owners. `Orchestrator#resume(id)` continues existing
+static subagent coordination. There is no global generic class registry.
+
+Observer Task wait timeouts and shutdown do not request cancellation. Team's
+`cancel(run_id)` persists a run-scoped request before forwarding its live token;
+resume settles exact children or returns the existing rehydration error. Handoff's
+`cancel(execution_id)` follows only that turn. A cancellation token passed to
+invoke remains the existing Agent execution cancellation mechanism; it does not
+promise rollback of external effects. Persisted pending child IDs remain
+available for recovery when external factual resolution is required.
+
+Callbacks and streaming progress may be lost after Runtime/process loss.
+Application notifications, outboxes, retries and external-effect deduplication
+belong to Application. No callback delivery ACK or restart replay is added.
+InMemory is useful for tests but a durable backend is required for actual F4
+storage retention. See ADR-018 and the V2 RC contract for guarantee limits.
+
+The existing `Agent#purge!` removes its own Handoff anchor atomically with its
+Agent records, after the anchor has no pending/active turn. Referenced outcomes
+deleted through retention/purge are unavailable, not evidence of a new reservation.
+Do not purge participating child/Target owners while their coordination is active.
+Team `stream` observes committed assignment progress; the construction `on_event`
+listener is passed to hidden Agents for their current Runtime events and Recovery.
+There is no additional Team callback delivery subsystem.

data/docs/persistence-backends.md

@@ -44,7 +44,7 @@ contract.
 
 ## Required root surface
 
-A Persistence backend exposes five durable repositories:
+A Persistence backend exposes eight durable repositories:
 
 ```text
 contents
@@ -52,6 +52,9 @@ agents
 journals
 executions
 workflow_states
+handoff_states
+teams
+team_executions
 ```
 
 and two root operations:
@@ -66,7 +69,7 @@ persistence.assert_agent_watermark!(
 ```
 
 The object yielded by `transaction` is a transaction-scoped Persistence view. It
-must respond to all five repository accessors and
+must respond to all eight repository accessors and
 `assert_agent_watermark!`. It may be the Persistence instance itself, but SQL
 backends may instead yield an object bound to a checked-out connection or
 transaction session.
@@ -90,7 +93,7 @@ this requirement.
 
 All durable repositories must be able to participate in one atomic transaction
 domain. A transaction may change `contents`, `agents`, `journals`, `executions`,
-and `workflow_states` and then either commit all changes or roll them all back.
+`workflow_states`, `handoff_states`, `teams`, and `team_executions`, and then either commit all changes or roll them all back.
 
 This requirement deliberately does not claim exactly-once semantics after an
 indeterminate database/network failure. If the underlying database cannot tell
@@ -269,6 +272,7 @@ def create_active(execution)
 def load(execution_id)
 def save(execution_id, expected_revision:, execution:)
 def list_active(agent_id)
+def list(agent_id, after: nil, limit: 100)
 def delete(execution_id)
 def delete_for_agent(agent_id)
 def assert_idle!(agent_id)
@@ -521,6 +525,9 @@ RSpec.describe MyPersistenceBackend do
   it_behaves_like "a Journal repository"
   it_behaves_like "an Execution repository"
   it_behaves_like "a workflow state repository"
+  it_behaves_like "a Handoff state repository"
+  it_behaves_like "a Team repository"
+  it_behaves_like "a Team execution repository"
   it_behaves_like "a Persistence backend"
 end
 ```
@@ -552,3 +559,35 @@ may use combinations of:
 
 Backend-specific database exceptions should be translated to the Phronomy error
 contract where their meaning is known.
+
+## V2 coordination Backend SPI (clean break)
+
+Both `Persistence.new` and `build_transaction_view` require all eight raw
+repositories. Five-repository fallback is removed. Existing record formats remain
+`0.1`; three additional record types are `phronomy.handoff_state`,
+`phronomy.team_root`, and `phronomy.team_execution`. Backend index metadata is
+explicit and MUST NOT be reconstructed by parsing DurableRecord payloads.
+
+| Raw repository | Required operations and explicit metadata |
+|---|---|
+| `handoff_states` | `load(main_agent_id)`; `save(main_agent_id, expected_revision:, next_revision:, active_agent_id:, record:)`; `delete(main_agent_id, expected_revision:)` |
+| `teams` | `create(team_id:, team_revision:, record:)`; `load(team_id)`; `save(team_id, expected_revision:, next_revision:, record:)`; `delete(team_id)` |
+| `team_executions` | `create_active(team_execution_id:, team_id:, execution_revision:, record:)`; `load(id)`; `save(id, expected_revision:, next_revision:, team_id:, active:, record:)`; `list_active(team_id)`; `list(team_id, after: nil, limit: 100)`; `delete(id)`; `delete_for_team(team_id)`; `assert_idle!(team_id)` |
+| `executions` extension | `list(agent_id, after: nil, limit: 100)` for retained active and terminal records |
+
+Handoff starts at revision 1 with expected revision nil; later saves advance
+exactly one revision. Team roots and executions begin at revision 0. Team
+admission has the existing `AgentBusyError` contract for an already active owner;
+stale CAS/duplicate identity uses `ConflictError`. Missing Team/execution loads
+raise `NotFoundError`; absent Handoff state returns nil. Unavailable reads and
+codec errors must propagate separately.
+
+Lists sort IDs lexically, use an exclusive ID cursor, return at most the positive
+integer limit, include only the requested owner, and return immutable copies.
+Backends define retention; enumeration does not implement input correlation or
+request deduplication. Terminal-to-active rewrites are invalid.
+
+For F1, atomic commit does not imply a known response. Phronomy reads back the
+same intended ID/fact before advancing. A backend must provide authoritative
+reads/CAS; failure of readback is returned without new semantic work. No callback
+ACK/index is part of the SPI. Existing cross-process exclusion limitations remain.

data/lib/phronomy/agent/agent_execution.rb

@@ -28,10 +28,10 @@ module Phronomy
       ].freeze
       attr_reader(*ATTRIBUTES)
 
-      def self.start(agent_root:, input_record:, metadata: {})
+      def self.start(agent_root:, input_record:, metadata: {}, execution_id: SecureRandom.uuid)
         now = Time.now.utc.iso8601(6)
         new(
-          execution_id: SecureRandom.uuid,
+          execution_id: execution_id.to_s,
           agent_id: agent_root.agent_id,
           execution_revision: 0,
           status: :preparing,

data/lib/phronomy/agent/agent_invocation.rb

@@ -458,7 +458,7 @@ module Phronomy
           key = :"include_#{category}"
           result[category] = args[key] if args.key?(key)
         end
-        Phronomy::MultiAgent::HandoffRequest.new(
+        Phronomy::Agent::HandoffRequest.new(
           handoff: handoff,
           responsibility: args.fetch(:responsibility),
           selection_intent: selection,

data/lib/phronomy/agent/async_event_api.rb

@@ -168,6 +168,7 @@ module Phronomy
       end
 
       def _start_agent_operation(input, config:, mode:, listener:)
+        config = _snapshot_durable_context(config)
         approval = _approval_configuration_snapshot(nil)
         execution_coordinator_for(config).start(
           input,
@@ -179,6 +180,22 @@ module Phronomy
         )
       end
 
+      def _snapshot_durable_context(config)
+        return config unless config.key?(:durable_context)
+
+        value = config[:durable_context]
+        unless value.is_a?(Hash)
+          raise ArgumentError,
+            "config[:durable_context] must be a canonical JSON-compatible Hash"
+        end
+
+        bytes = Phronomy::CanonicalJSON.dump(value)
+        snapshot = Phronomy::Agent::Immutable.copy(
+          Phronomy::CanonicalJSON.load(bytes)
+        )
+        config.merge(durable_context: snapshot)
+      end
+
       def _required_stream_listener!(method_name)
         listener = _phronomy_event_listener
         return listener if listener
@@ -209,7 +226,7 @@ module Phronomy
         return execution_coordinator unless multi_agent
 
         @multi_agent_execution_coordinator ||=
-          Phronomy::MultiAgent::ExecutionCoordinator.new(self)
+          Phronomy::Agent::HandoffExecutionCoordinator.new(self)
       end
     end
   end

data/lib/phronomy/agent/base.rb

@@ -267,6 +267,28 @@ module Phronomy
         end
       end
 
+      # @api private
+      def __coordination_config
+        @_phronomy_coordination_config || {}.freeze
+      end
+
+      # Framework-owned idempotent Tool operations opt into exact replay.
+      # @api private
+      def __framework_tool_replayable?(_name) = false
+
+      # @api private
+      def __framework_call?(name)
+        __framework_tool_replayable?(name) || Array(__coordination_config[:phronomy_handoff_bindings]).any? { |binding| binding.tool_name == name }
+      end
+
+      # Captured Tool wiring is supplied explicitly; this hook only writes semantic values.
+      # @api private
+      def __prepare_coordination_record(execution, tx:) = execution
+
+      # Runtime configuration only; no value returned here is serialized.
+      # @api private
+      def __invocation_config(config) = config
+
       attr_reader :agent_id, :persistence
 
       def initialize(
@@ -438,6 +460,12 @@ module Phronomy
         begin
           persistence.transaction do |tx|
             tx.executions.assert_idle!(agent_id)
+            if (routing = tx.handoff_states.load(agent_id))
+              unless routing.phase == "stable" && tx.executions.list_active(routing.active_agent_id).empty?
+                raise Phronomy::AgentBusyError, "Handoff anchor #{agent_id} owns an unfinished turn"
+              end
+              tx.handoff_states.delete(agent_id, expected_revision: routing.handoff_revision)
+            end
             tx.journals.delete(agent_id)
             tx.executions.delete_for_agent(agent_id)
             tx.agents.delete(agent_id)

data/lib/phronomy/agent/context_assembler.rb

@@ -552,7 +552,7 @@ module Phronomy
       def merge_handoff_candidates(candidates, handoff_context, execution:)
         return Array(candidates).freeze unless handoff_context
 
-        unless handoff_context.is_a?(Phronomy::MultiAgent::HandoffContext)
+        unless handoff_context.is_a?(Phronomy::Agent::HandoffContext)
           raise ArgumentError, "phronomy_handoff_context must be a HandoffContext"
         end
 

data/lib/phronomy/agent/exact_execution.rb

@@ -0,0 +1,153 @@
+# frozen_string_literal: true
+
+module Phronomy
+  module Agent
+    # Observes a reserved execution using ordinary Agent admission and Recovery.
+    # Durable reads/materialization run off EventLoop. Workers never wait for a
+    # child lifecycle: the existing EventLoop completion waiter settles Task.
+    # @api private
+    class ExactExecution
+      Wait = Data.define(:coordinator, :agent, :execution_id)
+      private_constant :Wait
+
+      def self.start(agent:, execution_id:, input:, config: {})
+        new(agent, execution_id, input, config).start
+      end
+
+      def initialize(agent, execution_id, input, config)
+        @agent, @id, @input, @config = agent, execution_id.to_s.freeze, input, config.freeze
+        @runtime = Phronomy::Runtime.instance
+        @completion = Phronomy::Task.deferred(name: "exact-execution:#{@id}")
+      end
+
+      def start
+        preparation = @runtime.offload.submit(on_full: :raise) do
+          execution = read_execution
+          if execution&.terminal?
+            [:terminal, materialize(execution)]
+          elsif execution
+            unless @runtime.__agent_execution_owner(@id)
+              @agent.instance_variable_set(:@_phronomy_coordination_config, @config)
+              begin
+                RecoveryCoordinator.new(@agent).recover_on_load!
+              rescue Phronomy::ConfigurationError => error
+                raise Phronomy::ExecutionRehydrationRequiredError,
+                  "Execution #{@id} needs current recovery wiring: #{error.message}"
+              end
+            end
+            [:active, nil]
+          else
+            @config[:cancellation_token]&.raise_if_cancelled!
+            [:absent, nil]
+          end
+        end
+        preparation.on_complete do |prepared, error|
+          if error
+            @completion.fail(error)
+          else
+            disposition, result = prepared
+            case disposition
+            when :terminal then @completion.complete(result)
+            when :absent
+              source = @agent.send(:_start_agent_operation, @input,
+                config: @config.merge(phronomy_reserved_execution_id: @id, phronomy_exact_observers: [@completion].freeze),
+                mode: :invoke, listener: @agent.send(:_phronomy_event_listener))
+              source.on_complete { |_value, failure| reconcile(failure) }
+            when :active
+              command = Wait.new(coordinator: self, agent: @agent, execution_id: @id)
+              posted = @runtime.event_loop.post(Phronomy::Event.new(type: :agent_terminal_ready,
+                target_id: Phronomy::EventLoop::SYSTEM_CHANNEL_ID, payload: {command: command}))
+              @completion.fail(Phronomy::RuntimeShutdownError.new("Exact execution observer rejected")) unless posted
+            end
+          end
+        rescue => failure
+          @completion.fail(failure)
+        end
+        @completion
+      rescue => failure
+        @completion.fail(failure)
+        @completion
+      end
+
+      def deliver_on_event_loop(command)
+        state = @runtime.event_loop.agent_execution_state(command.execution_id)
+        unless state
+          # Ownership can be released between the durable read and this command.
+          # Reinstall once with the caller's current wiring/cancellation request.
+          if !@reinstall_attempted
+            @reinstall_attempted = true
+            return start
+          end
+          return reconcile(nil)
+        end
+        unless state.agent.equal?(command.agent)
+          raise Phronomy::Persistence::ConflictError, "Exact execution #{@id} owner mismatch"
+        end
+        if @config[:cancellation_token]&.cancelled?
+          state.invocation&.config&.fetch(:cancellation_token, nil)&.cancel!
+        end
+        if state.execution.status == :suspended || state.fsm_session_id.nil?
+          raise Phronomy::ExecutionRehydrationRequiredError,
+            "Execution #{@id} requires approval, factual resolution or current recovery wiring"
+        end
+        observers = Array(state.invocation.config[:phronomy_exact_observers])
+        state.invocation.merge_config!(phronomy_exact_observers: (observers + [@completion]).uniq.freeze)
+        waiter = Phronomy::Task.deferred(name: "exact-wait:#{@id}")
+        waiter.on_complete { |_result, failure| reconcile(failure) }
+        @runtime.event_loop.register_agent_completion_waiter(@id, waiter)
+      rescue => failure
+        @completion.fail(failure)
+      end
+
+      private
+
+      def read_execution
+        begin
+          execution = @agent.persistence.executions.load(@id)
+        rescue Phronomy::Persistence::NotFoundError
+          return nil
+        end
+        unless execution.agent_id == @agent.agent_id
+          raise Phronomy::Persistence::ConflictError, "Reserved execution #{@id} belongs to another Agent"
+        end
+        expected = @config[:phronomy_coordination]
+        stored = execution.metadata["coordination"]
+        if expected && stored != expected
+          actual_identity = stored&.except("handoff_revision")
+          expected_identity = expected.except("handoff_revision")
+          unless actual_identity == expected_identity
+            raise Phronomy::Persistence::ConflictError, "Reserved execution #{@id} coordination mismatch"
+          end
+        end
+        ref = execution.metadata["current_input_ref"]
+        if ref && @agent.persistence.contents.fetch_text(ref) != @input
+          raise Phronomy::Persistence::ConflictError, "Reserved execution #{@id} input mismatch"
+        end
+        execution
+      end
+
+      def reconcile(failure)
+        task = @runtime.offload.submit(on_full: :raise) do
+          execution = read_execution
+          if execution&.terminal?
+            materialize(execution)
+          else
+            raise(failure || Phronomy::ExecutionRehydrationRequiredError.new(
+              "Exact execution #{@id} is unfinished; resume with current wiring"
+            ))
+          end
+        end
+        task.on_complete do |result, error|
+          error ? @completion.fail(error) : @completion.complete(result)
+        end
+      rescue => error
+        @completion.fail(error)
+      end
+
+      def materialize(execution)
+        result = @agent.persistence.execution_result(execution.execution_id)
+        result.merge(output: result[:result]).freeze
+      end
+    end
+  end
+end

data/lib/phronomy/agent/execution_cancellation.rb

@@ -0,0 +1,25 @@
+# frozen_string_literal: true
+
+module Phronomy
+  module Agent
+    # Wakes the existing execution cancellation token. Durable owners record the
+    # request first; this notification is replaceable after Runtime loss.
+    # @api private
+    class ExecutionCancellation
+      Command = Data.define(:coordinator, :execution_id, :agent_id)
+      private_constant :Command
+
+      def self.signal(execution_id, agent_id)
+        command = Command.new(coordinator: new, execution_id: execution_id, agent_id: agent_id)
+        Phronomy::Runtime.instance.event_loop.post(Phronomy::Event.new(type: :agent_terminal_ready,
+          target_id: Phronomy::EventLoop::SYSTEM_CHANNEL_ID, payload: {command: command}))
+      end
+
+      def deliver_on_event_loop(command)
+        state = Phronomy::Runtime.instance.event_loop.agent_execution_state(command.execution_id)
+        return unless state && state.agent.agent_id == command.agent_id
+        state.invocation&.config&.fetch(:cancellation_token, nil)&.cancel!
+      end
+    end
+  end
+end