phronomy 0.22.0 → 0.23.0

data/docs/runtime-and-concurrency.md

@@ -8,24 +8,40 @@ the bounded `OffloadPool`.
 
 For the design rationale, see [ADR-010](decisions/010-cooperative-first-concurrency.md).
 Durable-state ownership is defined by
-[ADR-014](decisions/014-unified-persistence-durable-state.md).
+[ADR-014](decisions/014-unified-persistence-durable-state.md), with live Agent
+Runtime execution-state ownership refined by
+[ADR-024](decisions/024-event-loop-single-writer-agent-runtime.md) and process-local
+Agent identity/admission ownership defined by
+[ADR-025](decisions/025-process-local-agent-ownership-and-runtime-admission.md).
+Canonical Workflow instance identity is defined by
+[ADR-020](decisions/020-canonical-workflow-instance-identity.md).
+Concrete FSMSession incarnation identity and session-local Runtime routing are
+defined by [ADR-023](decisions/023-fsm-session-incarnation-identity-and-routing.md).
+Same-process Workflow admission ownership and durable terminal-barrier ordering
+are defined by
+[ADR-026](decisions/026-workflow-runtime-admission-and-durable-terminal-barrier.md).
 
 ## Runtime model
 
 ```text
 Runtime
+├─ Agent ownership registry
+│  └─ agent_id -> one mutable live Agent instance
 ├─ EventLoop (one control-plane operating-system Thread)
-│  └─ FSMSession
-│     ├─ Agent
-│     ├─ Workflow
-│     ├─ ToolInvocation
-│     └─ MultiAgent fan-out
-├─ process-local Agent ActivationRegistry
+│  ├─ FSMSession
+│  │  ├─ Agent
+│  │  ├─ Workflow
+│  │  ├─ ToolInvocation
+│  │  └─ MultiAgent fan-out
+│  ├─ Agent top-level admission
+│  │  └─ agent_id -> one nonterminal logical Execution admission
+│  └─ Agent execution directory
+│     └─ execution_id -> immutable live-state record
 ├─ OffloadPool (bounded operating-system Threads)
 │  ├─ private Operation records
 │  ├─ blocking input/output (I/O)
 │  ├─ central-processing-unit (CPU)-bound synchronous work
-│  └─ other long synchronous work
+│  └─ operation-specific durable Agent/Workflow work
 ├─ named OffloadPools
 └─ EventLoop-driven timers
 
@@ -44,12 +60,22 @@ A live Agent or Workflow owns its current logical state. `Persistence` is the
 last committed durable representation and recovery source; it is not reloaded at
 every semantic boundary.
 
-For Agents, the live owner consists of the Agent instance plus its current
-`AgentRoot`, hydrated Journal view, and `AgentExecutionActivation`. Mutable
-Agent/Execution/Journal state is not automatically reloaded before every LLM or
-Tool step. Durable writes use optimistic revision/position guardrails; an
-external writer that advances the durable base causes `Persistence::ConflictError`
-rather than automatic reload or merge.
+For active Agents, **EventLoop is the single writer of Phronomy-managed live
+execution state**. EventLoop owns a process-local execution directory keyed by
+canonical `execution_id`. Each directory value is immutable and is replaced on
+EventLoop when the current AgentExecution, RuntimeProjection, AgentInvocation, or
+owning FSMSession changes. The former mutex-protected
+`AgentExecutionActivation` / `ActivationRegistry` model is removed.
+
+`AgentInvocation` is the FSM-local mutable context and holds uncommitted Provider
+outcomes, Tool/runtime events, active Provider-call provenance, and callback
+failure state. These fields are advanced only by EventLoop-driven FSMSession
+handling; workers do not receive AgentInvocation as a mutable state authority.
+
+Mutable Agent/Execution/Journal state is not automatically reloaded before every
+LLM or Tool step. Durable writes use optimistic revision/position guardrails; an
+external writer that advances the durable base causes
+`Persistence::ConflictError` rather than automatic reload or merge.
 
 For Workflows, the current `WorkflowContext` and FSMSession own the active
 logical state. A durable Workflow hydrates once at invocation/resume and saves at
@@ -58,34 +84,198 @@ the halted/terminal boundary.
 Content-addressed `Persistence#contents` values are immutable. Fetching a known
 content reference is value materialization rather than mutable state refresh.
 
-## Workflow identities and durable admission
+## Process-local Agent identity ownership and admission
+
+`agent_id` identifies one logical Agent, not a reusable lookup key for independent
+mutable objects. One Runtime therefore publishes at most one mutable live Agent
+instance for a given `agent_id`. The Runtime-owned registry is an authority, not a
+cache, and reserves the identity before create/load materialization.
+
+The application-facing identity operations are distinct:
+
+```text
+new / create
+  create a new Agent; existing live or durable identity is an error
+
+load(agent_id, persistence:)
+  live -> exact same Ruby object, with no Persistence reload
+  durable-only -> hydrate and publish once
+  missing -> Persistence::NotFoundError
+
+get(agent_id)
+  live Runtime lookup only; missing -> nil
+```
+
+A live Agent is strongly owned for the Runtime lifetime even while idle and
+across sequential Executions. Execution completion does not evict it. Clean
+Runtime shutdown detaches old Agent objects so they cannot remain mutable beside
+a later Runtime owner. `purge!` is the explicit earlier destruction boundary: it
+invalidates the old object, deletes durable state, releases the process-local
+identity, and allows a later new Agent to reuse the textual ID.
+
+Live Agent ownership and top-level Execution admission are separate lifetimes.
+For one live Agent, EventLoop admits at most one nonterminal top-level Execution.
+Admission is acquired **before** the initial Offload/Persistence operation:
 
-Workflow execution keeps three identities separate:
+```text
+invoke
+  -> EventLoop Agent admission
+  -> Offload/Persistence executions.create_active
+  -> EventLoop live execution state
+```
+
+`preparing`, `active`, and `suspended` all retain the slot. A competing request is
+rejected with `AgentBusyError`; core does not promise automatic queueing. A
+known-successful durable terminal transition releases the slot. A known
+pre-durable failure may release it; an uncertain durable outcome remains
+fail-closed/recovery-required.
+
+`Persistence#executions.create_active`, optimistic revision, Journal position, and
+watermark checks remain required durable defenses. They do not become the
+primary same-process live ownership/admission mechanism and do not provide
+cross-process exclusion.
+
+## EventLoop single-writer and Offload result application
+
+Persistence repositories are synchronous, so durable work must remain off the
+EventLoop thread. The ownership rule is therefore not "run everything on
+EventLoop". It is:
+
+```text
+EventLoop
+  capture operation-specific immutable state
+      ↓
+OffloadPool
+  blocking I/O / CPU / operation-local calculation
+  durable commit
+      ↓ operation-specific result
+EventLoop
+  validate current authority
+  apply committed result to live state
+```
+
+Agent initial preparation, follow-up Manifest preparation, approval resume, and
+terminal commit use distinct command/result values. An Offload worker may commit
+Persistence but does not update the live Agent root, Journal view, current
+AgentExecution, RuntimeProjection, AgentInvocation runtime queues, or EventLoop
+execution directory.
+
+Completion callbacks are lightweight bridges that enqueue the result back to the
+EventLoop. If EventLoop no longer accepts the result, the callback does not fall
+back to direct live mutation.
+
+## Provider Call result authority
+
+Provider Call identity is purpose-specific semantic provenance. EventLoop
+allocates `llm_call_id` before transport begins and binds it to the Manifest used
+for that call.
+
+Provider completion and streaming chunks return through the owning FSMSession's
+EventSink and carry the `llm_call_id`. A result is applicable only when:
+
+- it still targets the current FSMSession incarnation;
+- the FSM is in the state that accepts that result; and
+- the AgentInvocation still owns the same active `llm_call_id`.
+
+A callback to an old FSMSession incarnation is dropped by session-local routing.
+A result with a stale `llm_call_id` is consumed without advancing the current
+FSM. Phronomy does not add a generic generation/correlation token as another
+result authority.
+
+Tool operations follow the same ownership direction. `tool_invocation_id` is the
+semantic Tool-operation identity, while FSMSession ID is Runtime routing identity.
+Tool authorization captures Agent identity and Tool description data as values on
+EventLoop before offload. The authorization worker receives no live Agent, Tool, or
+ToolInvocation reference. Application-owned approval/facts/requirement callables
+are explicitly classified behavior handles and receive a value-only
+`ApprovalEvaluationRequest`.
+
+Hash, Array, and String authorization command data is recursively copied/frozen.
+Phronomy-managed live domain objects are rejected from that value data. A complete
+value-type/serialization contract for arbitrary Application-owned opaque objects is
+deferred; such objects remain Application-owned and must be worker-safe.
+
+Worker authorization/execution outcomes return as values carrying
+`tool_invocation_id`; the Tool FSMSession consumes a mismatched semantic result
+without advancing its current state.
+
+## Approval suspension and live owner lookup
+
+Approval suspension retains the same process-local Agent and AgentInvocation but
+has no active owning FSMSession until resume. EventLoop retains the suspended
+execution entry.
+
+`Agent::Base.live_for_execution(execution_id)` resolves a read-only Runtime owner
+view and returns the existing Agent instance. `agent.approve_async(...)` routes to
+the same live coordinator. Neither operation reloads a replacement Agent or
+Execution from Persistence.
+
+A resume performs its durable approval transition through OffloadPool, applies
+the result on EventLoop, and then builds a **fresh** FSMSession incarnation.
+If the process-local owner no longer exists, durable continuation reconstruction
+is not implied; `ExecutionRehydrationRequiredError` is raised.
+
+## Workflow identities, admission, and durable terminal barrier
+
+Workflow runtime keeps identity and coordination responsibilities separate:
 
 ```text
 session_id
-  application session/correlation identity
+  application session/correlation metadata
 
-thread_id
-  durable Workflow identity and Persistence#workflow_states key
+workflow_instance_id
+  logical/durable Workflow identity and Persistence#workflow_states key
+
+admission owner token
+  opaque process-local Runtime coordination capability
 
 fsm_session_id
-  one Runtime FSMSession execution identity; generated again for each invoke/resume
+  one concrete Runtime FSMSession routing identity
+```
+
+EventLoop acquires the `workflow_instance_id` admission with a fresh opaque owner
+token **before** `workflow_states.load(workflow_instance_id)` is submitted. The
+token is not a domain identity and is never an Event target. After durable
+hydration, EventLoop constructs the concrete FSMSession and binds its fresh
+`fsm_session_id` to the existing admission for `Workflow#signal` routing.
+
+```text
+admit workflow_instance_id with owner token
+    ↓
+Offload workflow_states.load
+    ↓
+EventLoop hydrate / create FSMSession
+    ↓
+bind fsm_session_id for routing
 ```
 
-The application `session_id` is tracing/caller metadata and is not used for
-durable Workflow ownership. EventLoop registers active FSMs by `fsm_session_id`;
-durable Workflow admission is a separate owner map:
+A durable Workflow also keeps terminal persistence inside the FSMSession
+lifecycle. Logical halt/completion first enters a private
+`persisting_terminal` lifecycle condition; the FSMSession remains nonterminal
+while WorkflowRunner saves the terminal snapshot through OffloadPool. Only a
+known-successful save result returned to that same FSMSession permits
+`HALTED`/`COMPLETED`, admission release, and caller Task settlement.
 
 ```text
-thread_id -> owner_fsm_session_id
+RUNNING
+    ↓ logical halt/completion
+PERSISTING_TERMINAL
+    ├─ known success  -> HALTED / COMPLETED -> release
+    ├─ known failure  -> ERROR -> release
+    └─ outcome unknown -> RECOVERY_REQUIRED (fail closed)
 ```
 
-The owner is acquired before `workflow_states.load(thread_id)` and remains held
-until the halted/terminal `workflow_states.save(...)` completes. The admission map
-is process-local. Cross-process duplicate execution requires application-level
-distributed coordination; optimistic revisions detect stale terminal commits but
-do not prevent duplicate side effects before that conflict is detected.
+The FSMSession does not know whether Persistence is local, remote, SQL, HTTP, or
+networked. The Workflow persistence operation normalizes the result into
+`success`, `known_failure`, or `outcome_unknown`. Only known success crosses the
+durable barrier. If the backend/storage error does not establish non-commit,
+Phronomy treats the terminal outcome as uncertain rather than guessing.
+
+`recovery_required` prevents a fresh same-Workflow execution segment from being
+admitted, but ACS-13 does not claim restart-safe reconciliation; that remains
+ACS-15 work. The admission map itself is process-local. Cross-process duplicate
+execution requires the later coordination/fencing work; optimistic revisions
+remain durable conflict defense rather than distributed ownership.
 
 ## Tool execution modes
 
@@ -132,8 +322,7 @@ waiting at the same time.
 `Persistence` repositories expose synchronous operations. Framework lifecycle
 code must not perform potentially blocking durable reads/writes on EventLoop.
 Agent preparation/commit and Workflow hydrate/save operations are submitted to
-`OffloadPool`; completion continues through Task callbacks or explicit EventLoop
-events.
+`OffloadPool`; completion continues through explicit EventLoop events.
 
 A durable barrier may pause one logical lifecycle without blocking EventLoop.
 Persistence does not implement async repository variants and must not depend on
@@ -174,8 +363,9 @@ abandonment state.
 `Task#on_complete` registers an independent notification callback. Callback
 execution thread is not guaranteed. A callback may be delivered by an OffloadPool
 worker, a timer/cancellation caller, an EventLoop-related control path, or the
-thread that registers after settlement. Callbacks must be thread-safe and should
-complete quickly.
+thread that registers after settlement. Callbacks must therefore be thread-safe
+and should complete quickly. Framework lifecycle code normally converts worker
+completion into an explicit EventLoop event before applying live state.
 
 `Task#map` is application-level composition. A transformation exception settles
 the mapped Task as failed.
@@ -301,6 +491,10 @@ Workflow durable admission participates in EventLoop idleness: a Workflow whose
 FSMSession has ended but whose durable save is still in flight remains owned until
 that save completes and owner-aware admission is released.
 
+Suspended Agent execution owner entries are process-local continuation state and
+do not by themselves keep Runtime shutdown waiting. Runtime/process loss does not
+imply durable Agent continuation reconstruction.
+
 `Phronomy.reset_runtime!` exists primarily for test isolation and performs a real
 Runtime shutdown before resetting configuration.
 

data/examples/README.md

@@ -0,0 +1,13 @@
+# Phronomy examples
+
+Runnable application examples are maintained in the dedicated
+[`Raizo-TCS/phronomy-examples`](https://github.com/Raizo-TCS/phronomy-examples)
+repository.
+
+This repository intentionally does not keep a second runnable example set under
+`examples/`. Keeping one canonical sample repository avoids API drift between
+Phronomy's implementation/documentation and independently maintained copies of
+sample code.
+
+For framework-level executable behavior and regression coverage, use the specs
+under `spec/`. For application-oriented examples, use `phronomy-examples`.

data/lib/phronomy/agent/agent_execution.rb

@@ -7,12 +7,13 @@ module Phronomy
   module Agent
     class AgentExecution
       ACTIVE_STATUSES = %i[preparing active suspended].freeze
-      TERMINAL_STATUSES = %i[completed failed cancelled rejected blocked].freeze
+      TERMINAL_STATUSES = %i[completed handed_off failed cancelled rejected blocked].freeze
       TRANSITIONS = {
         preparing: %i[preparing active failed cancelled blocked],
-        active: %i[active suspended completed failed cancelled rejected blocked],
+        active: %i[active suspended completed handed_off failed cancelled rejected blocked],
         suspended: %i[suspended active failed cancelled],
         completed: %i[completed],
+        handed_off: %i[handed_off],
         failed: %i[failed],
         cancelled: %i[cancelled],
         rejected: %i[rejected],
@@ -84,10 +85,8 @@ module Phronomy
         self.class.new(**values)
       end
 
-      # Returns the canonical durable representation of this execution.
-      # Nested JournalRecord and LLMCallRecord values are recursively encoded.
-      #
-      # @return [Hash{String => Object}]
+      # Current semantic payload representation. Persistence format identity and
+      # compatibility validation are owned by Persistence::DurableCodec.
       # @api public
       def to_h
         ATTRIBUTES.to_h do |name|
@@ -98,18 +97,23 @@ module Phronomy
         end
       end
 
-      # Restores an execution from its canonical durable representation.
-      # String and Symbol top-level keys are accepted. Nested working Journal
-      # records and LLM Call records are restored through their public codecs so
-      # storage backends do not need to know their constructor details.
-      #
-      # @param hash [Hash]
-      # @return [AgentExecution]
+      # Restores only the current semantic payload shape. Historical durable
+      # representations must go through explicit Persistence migration first.
       # @api public
       def self.from_h(hash)
+        source = hash.to_h { |key, value| [key.to_s, value] }
+        expected = ATTRIBUTES.map(&:to_s).sort
+        actual = source.keys.sort
+        unless actual == expected
+          missing = expected - actual
+          unknown = actual - expected
+          raise ArgumentError,
+            "AgentExecution payload schema mismatch: " \
+            "missing=#{missing.inspect}, unknown=#{unknown.inspect}"
+        end
+
         attributes = ATTRIBUTES.to_h do |name|
-          key = hash.key?(name.to_s) ? name.to_s : name
-          [name, hash.fetch(key)]
+          [name, source.fetch(name.to_s)]
         end
 
         attributes[:working_records] = attributes.fetch(:working_records).map do |record|