phronomy 0.25.0 → 0.27.0

data/docs/runtime-and-concurrency.md

@@ -2,7 +2,7 @@
 
 Phronomy uses an **EventLoop / FSMSession first** architecture for framework
 lifecycle coordination. `FSMSession` represents explicit lifecycle state and
-events. `Phronomy::Task` is the common caller-facing completion handle, not an
+events. `Phronomy::TaskResult` is the common caller-facing completion handle, not an
 execution backend. Synchronous work that must stay off EventLoop is isolated in
 the bounded `OffloadPool`.
 
@@ -23,37 +23,38 @@ are defined by
 
 ## Runtime model
 
-```text
-Runtime
-├─ Agent ownership registry
-│  └─ agent_id -> one mutable live Agent instance
-├─ EventLoop (one control-plane operating-system Thread)
-│  ├─ FSMSession
-│  │  ├─ Agent
-│  │  ├─ Workflow
-│  │  ├─ ToolInvocation
-│  │  └─ MultiAgent fan-out
-│  ├─ Agent top-level admission
-│  │  └─ agent_id -> one nonterminal logical Execution admission
-│  └─ Agent execution directory
-│     └─ execution_id -> immutable live-state record
-├─ OffloadPool (bounded operating-system Threads)
-│  ├─ private Operation records
-│  ├─ blocking input/output (I/O)
-│  ├─ central-processing-unit (CPU)-bound synchronous work
-│  └─ operation-specific durable Agent/Workflow work
-├─ named OffloadPools
-└─ EventLoop-driven timers
-
-EventLoop / FSMSession ─┐
-                       ├─> Task = completion handle
-OffloadPool ────────────┘
-```
+| Runtime-retained component | Implementation owner and responsibility |
+|---|---|
+| Shutdown participants | Feature-owned registries: Agent identity, Team identity, and MultiAgent synchronous-call admission |
+| EventLoop | Engine dispatcher thread, FSMSessions and registered feature receivers |
+| Execution receivers | Agent/Workflow own execution admission, state and idle policies; mutations run on EventLoop |
+| OffloadPool and named pools | Engine bounded workers and private physical-operation records |
+| Timers | Engine EventLoop-driven scheduling |
+
+Runtime retains participants through a generic lifecycle protocol; it does not
+construct Agent or Team identity registries. Each feature registers its own
+instance. See [ADR-041](decisions/041-feature-owned-identity-registries.md).
 
 The framework does not allocate one operating-system Thread per logical
 Agent/Workflow/Tool lifecycle. Logical waits remain explicit states plus later
 EventLoop events.
 
+## Feature execution receivers
+
+[ADR-042](decisions/042-feature-owned-execution-state.md) separates implementation
+ownership from the single-writer execution context. Agent and Workflow create
+`Agent::ExecutionRegistry` and `WorkflowExecutionRegistry` respectively; Engine
+retains registered receivers without selecting concrete feature types.
+
+Receiver registration and new request admission close before idle checks.
+Already queued requests, pending durable operations and supervised physical
+work are counted until their feature continuation settles. Worker callbacks
+only enqueue results. Workflow instance routing and generic FSM enqueue share
+one lock. After a clean join, receiver cleanup invalidates retained state;
+a failing receiver makes Runtime cleanup incomplete. Dispatcher failure instead
+notifies receivers on the failing loop thread with the original error.
+
+
 ## Live state and durable state
 
 A live Agent or Workflow owns its current logical state. `Persistence` is the
@@ -61,7 +62,7 @@ last committed durable representation and recovery source; it is not reloaded at
 every semantic boundary.
 
 For active Agents, **EventLoop is the single writer of Phronomy-managed live
-execution state**. EventLoop owns a process-local execution directory keyed by
+execution state**. `Agent::ExecutionRegistry` owns the process-local directory keyed by
 canonical `execution_id`. Each directory value is immutable and is replaced on
 EventLoop when the current AgentExecution, RuntimeProjection, AgentInvocation, or
 owning FSMSession changes. The former mutex-protected
@@ -75,7 +76,7 @@ handling; workers do not receive AgentInvocation as a mutable state authority.
 Mutable Agent/Execution/Journal state is not automatically reloaded before every
 LLM or Tool step. Durable writes use optimistic revision/position guardrails; an
 external writer that advances the durable base causes
-`Persistence::ConflictError` rather than automatic reload or merge.
+`Storage::ConflictError` rather than automatic reload or merge.
 
 For Workflows, the current `WorkflowContext` and FSMSession own the active
 logical state. A durable Workflow hydrates once at invocation/resume and saves at
@@ -88,8 +89,9 @@ content reference is value materialization rather than mutable state refresh.
 
 `agent_id` identifies one logical Agent, not a reusable lookup key for independent
 mutable objects. One Runtime therefore publishes at most one mutable live Agent
-instance for a given `agent_id`. The Runtime-owned registry is an authority, not a
-cache, and reserves the identity before create/load materialization.
+instance for a given `agent_id`. The Agent-owned registry, strongly retained by
+Runtime, is an authority, not a cache, and reserves the identity before
+create/load materialization.
 
 The application-facing identity operations are distinct:
 
@@ -100,7 +102,7 @@ new / create
 load(agent_id, persistence:)
   live -> exact same Ruby object, with no Persistence reload
   durable-only -> hydrate and publish once
-  missing -> Persistence::NotFoundError
+  missing -> Storage::NotFoundError
 
 get(agent_id)
   live Runtime lookup only; missing -> nil
@@ -114,7 +116,8 @@ invalidates the old object, deletes durable state, releases the process-local
 identity, and allows a later new Agent to reuse the textual ID.
 
 Live Agent ownership and top-level Execution admission are separate lifetimes.
-For one live Agent, EventLoop admits at most one nonterminal top-level Execution.
+For one live Agent, `Agent::ExecutionRegistry` admits at most one nonterminal
+top-level Execution on EventLoop.
 Admission is acquired **before** the initial Offload/Persistence operation:
 
 ```text
@@ -195,6 +198,15 @@ Phronomy-managed live domain objects are rejected from that value data. A comple
 value-type/serialization contract for arbitrary Application-owned opaque objects is
 deferred; such objects remain Application-owned and must be worker-safe.
 
+The existing restricted types declare the internal methodless
+`Concurrency::WorkerInputRestricted` marker at their own definitions.
+ToolInvocation checks that execution-boundary contract instead of enumerating
+Agent/Workflow/Runtime classes. Frozen marked values and subclasses remain
+restricted; Hash keys and values, nested Arrays and behavior handles are all
+checked. The marker does not inspect opaque application fields or closure
+captures and is not a general restriction on every OffloadPool command.
+See [ADR-045](decisions/045-worker-input-restriction-ownership.md).
+
 Worker authorization/execution outcomes return as values carrying
 `tool_invocation_id`; the Tool FSMSession consumes a mismatched semantic result
 without advancing its current state.
@@ -254,7 +266,7 @@ lifecycle. Logical halt/completion first enters a private
 `persisting_terminal` lifecycle condition; the FSMSession remains nonterminal
 while WorkflowRunner saves the terminal snapshot through OffloadPool. Only a
 known-successful save result returned to that same FSMSession permits
-`HALTED`/`COMPLETED`, admission release, and caller Task settlement.
+`HALTED`/`COMPLETED`, admission release, and caller TaskResult settlement.
 
 ```text
 RUNNING
@@ -279,13 +291,39 @@ remain durable conflict defense rather than distributed ownership.
 
 ## Tool execution modes
 
+The default dispatch helper is the private
+`Phronomy::Agent::Context::Capability::ToolExecutor`, colocated with Capability
+Base. Agent ToolInvocation supplies Runtime and admission policy for the
+standard path, and owns authorization and logical result handling. Custom
+`call_async` implementations keep the public Tool protocol. This ownership is
+defined by [ADR-035](decisions/035-tool-executor-capability-ownership.md); the
+public `Phronomy::Tool::Base` facade and its Class identity remain unchanged.
+
+Agent uses ordinary `RubyLLM::Chat` for both complete and streaming Provider
+calls. The `before_tool_call` callback takes the complete assistant Tool-call
+batch before RubyLLM executes the first Tool body. `AgentInvocation` then owns
+authorization, dispatch, approval suspension and result collection. Offloaded
+Tools can overlap within Runtime capacity; short cooperative Tools run according
+to their existing execution contract.
+
+On successful batch completion, Agent records one Tool result per call ID in
+request order and includes the whole exchange in the next Provider request.
+Worker completion order does not split the conversation or trigger a partial
+Provider continuation. Failure, cancellation and approval keep their existing
+Agent lifecycle semantics.
+
+There is no separate Chat execution path or `parallel_tool_execution` switch.
+The removed internal `MultiAgent::ParallelToolChat` class is not replaced by an
+alias. See the [Chat migration guide](migrations/parallel-tool-chat-removal.md)
+for application configuration and existing stored model-config records.
+
 Phronomy exposes two execution modes for capabilities:
 
 - `:cooperative` — short EventLoop-safe work, or specialized asynchronous work
-  that starts another Phronomy lifecycle and returns a Task immediately;
+  that starts another Phronomy lifecycle and returns a TaskResult immediately;
 - `:offloaded` — synchronous work that must not run to completion on EventLoop.
 
-Both paths return `Phronomy::Task`. The execution mechanism differs; the
+Both paths return `Phronomy::TaskResult`. The execution mechanism differs; the
 completion abstraction does not.
 
 Phronomy does not classify application work into framework-level I/O/CPU/process
@@ -310,7 +348,7 @@ Correct shape:
 parent FSMSession
   → start child lifecycle
   → return immediately
-  → child Task settles
+  → child TaskResult settles
   → post parent EventLoop event
 ```
 
@@ -326,7 +364,7 @@ Agent preparation/commit and Workflow hydrate/save operations are submitted to
 
 A durable barrier may pause one logical lifecycle without blocking EventLoop.
 Persistence does not implement async repository variants and must not depend on
-EventLoop, FSMSession, Task settlement internals, or private OffloadPool operation
+EventLoop, FSMSession, TaskResult settlement internals, or private OffloadPool operation
 records.
 
 ## Sync versus async application APIs
@@ -336,7 +374,7 @@ records.
 | Top-level application code | `agent.invoke(...)` when blocking the caller is acceptable |
 | Top-level explicit async | `task = agent.invoke_async(...)`; optionally `task.wait_result` outside EventLoop |
 | Workflow entry/transition action | Start async work and continue through `Workflow#signal` |
-| EventLoop callback | Never block waiting for a Task that requires EventLoop progress |
+| EventLoop callback | Never block waiting for a TaskResult that requires EventLoop progress |
 | Top-level streaming | `agent.stream(...)` |
 | Non-blocking streaming | `agent.stream_async(...)` |
 | Approval from EventLoop callback | Resolve with `live_for_execution`, call `agent.approve_async(...)`, and return immediately |
@@ -345,37 +383,88 @@ Blocking synchronous APIs reject EventLoop re-entry with
 `Phronomy::EventLoopReentrancyError` when waiting would stall the same EventLoop
 needed for progress.
 
-## Task
+## TaskResult
 
-`Phronomy::Task` is thread-free. It represents one terminal result:
+`Phronomy::TaskResult` is thread-free. It represents one terminal result:
 
 - completed value;
 - failure;
 - cancellation.
 
-Task is the common completion abstraction for logical EventLoop/FSMSession
+TaskResult is the common completion abstraction for logical EventLoop/FSMSession
 lifecycles and OffloadPool-backed synchronous work.
 
-`Task#wait_result(timeout:)` is a bridge for external synchronous callers. Its
-timeout is waiter-local: it does not settle/cancel the Task or alter OffloadPool
+`TaskResult#wait_result(timeout:)` is a bridge for external synchronous callers. Its
+timeout is waiter-local: it does not settle/cancel the TaskResult or alter OffloadPool
 abandonment state.
 
-`Task#on_complete` registers an independent notification callback. Callback
+`TaskResult#on_complete` registers an independent notification callback. Callback
 execution thread is not guaranteed. A callback may be delivered by an OffloadPool
 worker, a timer/cancellation caller, an EventLoop-related control path, or the
 thread that registers after settlement. Callbacks must therefore be thread-safe
 and should complete quickly. Framework lifecycle code normally converts worker
 completion into an explicit EventLoop event before applying live state.
 
-`Task#map` is application-level composition. A transformation exception settles
-the mapped Task as failed.
+`TaskResult#map` is application-level composition. A transformation exception settles
+the mapped TaskResult as failed.
 
-Framework components own Task settlement. Application code should not use
-`Task#complete`, `Task#fail`, or `Task#cancel!` as operation-control APIs. Request
+`flat_map` connects a returned TaskResult's completion, while `all_settled`
+observes an input-order list of results. `Execution.run_async` starts and joins
+JOBs under whole-execution controls. See [Result composition and Execution](async-composition.md)
+for the complete contracts, ownership boundaries and development-release migration.
+
+Framework components own TaskResult settlement. Application code should not use
+`TaskResult#complete`, `TaskResult#fail`, or `TaskResult#cancel!` as operation-control APIs. Request
 operation-wide cancellation through the `CancellationToken` accepted by the API
-that created the Task. Task settlement never propagates backwards to cancel a
+that created the TaskResult. TaskResult settlement never propagates backwards to cancel a
 shared CancellationToken.
 
+### Settled TaskResult factories
+
+`TaskResult.completed(value)` and `TaskResult.failed(error)` create results that are already
+settled. They are useful when application code has a result or an error in hand
+and needs to present it through the same `TaskResult` contract as a lifecycle-backed
+completion:
+
+```ruby
+# Already-computed value — no execution started.
+task = Phronomy::TaskResult.completed("cached result")
+
+# Known failure — no execution started.
+task = Phronomy::TaskResult.failed(StandardError.new("precondition not met"))
+```
+
+Settled results satisfy `TaskResult#wait_result`, `TaskResult#on_complete`, and `TaskResult#map` with
+the same contract as a normally-completed TaskResult.
+
+## Blocking
+
+`Phronomy::Blocking.call_async(&block)` submits a synchronous block to the
+existing default Runtime OffloadPool and returns a `TaskResult`. It is the public
+application entry point for one-off synchronous work that should not block the
+caller:
+
+```ruby
+task = Phronomy::Blocking.call_async { expensive_io_call }
+result = task.wait_result
+```
+
+The optional `invocation_context:` connects this operation to an Execution
+scope or an existing invocation context. Context controls and the optional
+individual `cancellation_token:` both apply, without reverse cancellation of
+shared controls. The application chooses when synchronous work needs OffloadPool.
+
+The method uses the existing bounded OffloadPool. It does not create a new thread
+or scheduler. Capacity and backpressure semantics are inherited from the pool
+configuration (`offload_pool_size` / `offload_queue_size`).
+
+`Blocking.call_async` always uses non-waiting admission (`on_full: :raise`) and can
+be called from within the EventLoop thread. Admission `StandardError` failures (for
+example `BackpressureError` when the queue is full) are caught and returned as a
+failed `TaskResult`. Accepted work retains the pool's original `TaskResult` unchanged. Do not
+call `TaskResult#wait_result` on the returned `TaskResult` from within the EventLoop — waiting
+for a `TaskResult` that requires EventLoop progress will deadlock.
+
 ## OffloadPool
 
 `OffloadPool` is a bounded worker pool for synchronous work that must not execute
@@ -391,7 +480,7 @@ Its guarantees include:
 - runtime metrics;
 - shutdown/drain behavior.
 
-`OffloadPool#submit` returns a `Phronomy::Task`. OffloadPool does not expose its
+`OffloadPool#submit` returns a `Phronomy::TaskResult`. OffloadPool does not expose its
 execution record as a caller-facing future/promise. Its private `Operation` owns:
 
 - the submitted block;
@@ -403,27 +492,36 @@ execution record as a caller-facing future/promise. Its private `Operation` owns
 - metrics state needed by the pool.
 
 This separation keeps execution details private while allowing every asynchronous
-Phronomy API to expose the same Task completion contract.
+Phronomy API to expose the same TaskResult completion contract.
 
 ### EventLoop queue admission
 
 Framework-owned EventLoop-origin submissions must not wait for a free worker
 queue slot. They use non-blocking admission (`on_full: :raise`) and route
-`BackpressureError` through the ordinary FSM/Task completion path.
+`BackpressureError` through the ordinary FSM/TaskResult completion path.
+
+Any internal `OffloadPool#submit` call that requests waiting admission
+(`on_full: :wait` or `:timeout`) is **rejected before the Operation is created**
+when the caller is on the EventLoop thread. The rejection raises
+`Phronomy::EventLoopReentrancyError` directly; it does not create an Operation,
+timer, cancellation subscription, or TaskResult. The public `Phronomy::Blocking.call_async`
+always uses `on_full: :raise` and therefore does not trigger this guard; it converts
+admission `StandardError` (such as `BackpressureError`) into a failed `TaskResult` at the
+public-API layer.
 
 External management threads may choose a blocking admission policy when blocking
 the caller is acceptable.
 
 ## Submit timeout and cancellation
 
-Submit-time timeout and submit cancellation settle the caller-facing Task. They
+Submit-time timeout and submit cancellation settle the caller-facing TaskResult. They
 do **not** asynchronously interrupt an already-running synchronous worker.
 
 ### Before worker start
 
 If timeout/cancellation wins before execution starts:
 
-- the Task settles (`TimeoutError` failure or cancellation);
+- the TaskResult settles (`TimeoutError` failure or cancellation);
 - the submitted block does not run;
 - the private Operation is not counted as abandoned.
 
@@ -431,7 +529,7 @@ If timeout/cancellation wins before execution starts:
 
 If timeout/cancellation wins after execution starts:
 
-- the Task settles immediately;
+- the TaskResult settles immediately;
 - the private Operation is marked abandoned;
 - the worker continues until its synchronous call returns;
 - the eventual worker result is discarded.
@@ -447,7 +545,7 @@ requiring callback delivery promote the deadline to explicit `cancel!` through
 the Runtime timer queue. OffloadPool does this for its submit cancellation token.
 
 A CancellationToken may be shared by multiple operations. For that reason,
-settling or cancelling one Task does not cancel the token in the reverse
+settling or cancelling one TaskResult does not cancel the token in the reverse
 direction.
 
 ## Native async boundary
@@ -455,7 +553,7 @@ direction.
 A genuine native-async driver that does not create a Phronomy-owned OS Thread and
 does not block EventLoop need not consume an OffloadPool worker. If Phronomy
 formally exposes such an extension point, it must adapt completion to
-`Phronomy::Task` rather than exposing a provider-specific future or a private
+`Phronomy::TaskResult` rather than exposing a provider-specific future or a private
 Runtime type.
 
 The current Persistence, VectorStore, Embeddings, and LLM call-extension
@@ -473,8 +571,8 @@ Two metrics answer different operational questions:
 - `offload_pool_abandoned_active` — current number of abandoned operations whose
   synchronous workers still occupy pool capacity.
 
-The abandonment state belongs to the private OffloadPool Operation, not to Task.
-Task reports only caller-facing settlement.
+The abandonment state belongs to the private OffloadPool Operation, not to TaskResult.
+TaskResult reports only caller-facing settlement.
 
 ## EventLoop metrics
 
@@ -487,6 +585,53 @@ Use these to distinguish worker saturation from EventLoop backlog/latency.
 Runtime-owned EventLoop, then closes pools and timers according to the Runtime
 shutdown contract.
 
+Internal subsystems can register one shutdown participant per Runtime-local key
+through `Runtime#__register_shutdown_participant`. This is an internal lifecycle
+contract, not a public extension API or an execution registry. Runtime does not
+construct participants or interpret their admission rules.
+
+Each participant implements two required operations:
+
+- `begin_draining` closes admission under the participant's admission lock. It
+  must be idempotent, short and nonblocking, and must not call back into Runtime.
+  Runtime invokes it under its lifecycle lock when shutdown starts or EventLoop
+  fails, so no participant can be registered after closure begins.
+- `wait_until_idle(deadline)` returns whether admitted calls have finished before
+  the shared absolute monotonic deadline. Runtime closes every participant before
+  waiting, and waits without holding its lifecycle lock. A timeout or failed hook
+  makes cleanup incomplete; other participants, pools and timers still receive
+  their shutdown calls. Incomplete cleanup prevents default Runtime replacement.
+
+Participants may also implement `after_runtime_shutdown`. Runtime calls it
+outside its lifecycle lock after every wait succeeds, EventLoop has stopped,
+and pools/timers have shut down successfully. The hook must be idempotent,
+short, and perform no I/O; its return value is ignored. A hook exception marks
+cleanup incomplete, retains the first failure, and does not skip later hooks.
+Completed releases are not rolled back; repeated shutdown returns its cached
+result without retrying hooks. A prior closure/wait failure, timeout, or worker
+shutdown failure skips finalization entirely.
+
+`Runtime#__shutdown_participant(key:)` only retrieves an existing participant,
+including during/after shutdown. Registration still requires a running Runtime,
+even for an existing key. Agent/Team `get` uses lookup without registration;
+feature gates reject new ownership changes through retained registries after
+closure. Already admitted transitions may finish. Identity registries wait only
+for active construction/purge, then detach Agent references or clear Team
+owners at finalization; live idle objects do not themselves prolong shutdown.
+
+`MultiAgent::AdmissionRegistry.for(runtime)` constructs and registers the shared
+coordination admission registry. MultiAgent::HandoffRunner and TeamCoordinator retain that
+registry, admit each synchronous call directly and release it in `ensure` after
+successful admission. The registry closes admission atomically with respect to
+`admit!`; a call admitted before closure is included in the wait, and a call after
+closure raises `RuntimeShutdownError`, including after EventLoop failure. The
+existing `HandoffError` for duplicate admission remains unchanged.
+
+This tracks the current synchronous Handoff/Team call, not the lifetime or
+completion of a durable AgentExecution or TeamExecution. Multi-Agent policy stays
+in `multi_agent`; Runtime depends only on the shutdown operations. Application
+authors do not register these participants themselves.
+
 Workflow durable admission participates in EventLoop idleness: a Workflow whose
 FSMSession has ended but whose durable save is still in flight remains owned until
 that save completes and owner-aware admission is released.

data/lib/phronomy/agent/api/agent.rb

@@ -0,0 +1,17 @@
+# frozen_string_literal: true
+
+module Phronomy
+  module Agent
+    StreamEvent = Data.define(:type, :payload)
+  end
+end
+
+require_relative "../async_event_api"
+
+unless Phronomy::Agent::Base < Phronomy::Agent::AsyncEventApi
+  Phronomy::Agent::Base.prepend(Phronomy::Agent::AsyncEventApi)
+end
+
+# Durable Agent Recovery and Agent-incarnation event binding.
+require_relative "../recovery/recovery_support"
+require_relative "../recovery/recovery_coordinator"

data/lib/phronomy/agent/async_event_api.rb

@@ -96,7 +96,7 @@ module Phronomy
 
       def approve_async(execution_id, approval_request_id:, approved: true, config: {})
         _reject_removed_generic_identity_keys!(config)
-        owner = Phronomy::Runtime.instance.__agent_execution_owner(execution_id)
+        owner = Phronomy::Agent::ExecutionRegistry.existing_for(Phronomy::Runtime.instance)&.agent_execution_owner(execution_id)
         coordinator = if owner&.agent&.equal?(self)
           owner.coordinator
         else
@@ -190,7 +190,7 @@ module Phronomy
         end
 
         bytes = Phronomy::CanonicalJSON.dump(value)
-        snapshot = Phronomy::Agent::Immutable.copy(
+        snapshot = Phronomy::Values::Immutable.copy(
           Phronomy::CanonicalJSON.load(bytes)
         )
         config.merge(durable_context: snapshot)