phronomy 0.22.0 → 0.24.0

data/docs/decisions/025-process-local-agent-ownership-and-runtime-admission.md

@@ -0,0 +1,249 @@
+# ADR-025: Process-Local Agent Ownership and Runtime Admission
+
+**Status**: Accepted
+**Date**: 2026-08-24
+**Partially supersedes**: [ADR-014](014-unified-persistence-durable-state.md) for same-process Agent live-instance ownership and top-level execution admission
+**Complements**: [ADR-018](018-durability-guarantees-and-failure-model.md), [ADR-022](022-agent-execution-parent-identity-and-runtime-routing-boundary.md), [ADR-023](023-fsm-session-incarnation-identity-and-routing.md), [ADR-024](024-event-loop-single-writer-agent-runtime.md)
+
+---
+
+## Context
+
+Phronomy's durable Agent identity is `agent_id`. The architecture baseline treats
+that value as the identity of one logical Agent, not as a database lookup key
+that may be reused by several independent mutable Ruby objects.
+
+Before this decision, `Agent::Base.load(agent_id, persistence:)` could hydrate a
+new mutable Agent every time it was called. The same process could therefore
+hold two independent live objects that both represented the same logical Agent.
+Separately, top-level `invoke` requests reached EventLoop, but the first concrete
+same-Agent exclusion was still `Persistence#executions.create_active` inside the
+initial Offload/Persistence operation.
+
+That placement confused three different authorities:
+
+```text
+Runtime
+  process-local live Agent ownership
+
+EventLoop
+  process-local top-level execution admission and live execution progression
+
+Persistence
+  last confirmed durable representation, atomic durable transition,
+  optimistic conflict detection and recovery source
+```
+
+ADR-024 made EventLoop the single writer of Phronomy-managed live Agent
+execution state. ACS-12 adds the process-local logical-Agent ownership and
+admission layer on top of that foundation without turning Persistence into a
+live ownership service.
+
+## Decision
+
+### One `agent_id` has one mutable live Agent owner per Runtime
+
+A Runtime owns a purpose-specific Agent ownership registry keyed only by
+`agent_id`:
+
+```text
+agent_id
+  -> one mutable live Agent instance
+```
+
+The registry is an ownership authority, not a cache. Immutable snapshots,
+durable records, read-only projections and handles may still have multiple
+representations of the same Agent identity.
+
+Agent ownership is reserved before durable create/load and before the mutable
+Agent is published. Concurrent materialization of the same identity therefore
+cannot create two independent live objects.
+
+The registry is separate from EventLoop's `execution_id -> AgentExecutionState`
+directory. Agent lifetime and Execution lifetime are different semantic
+lifetimes and must not be represented by one registry.
+
+### Public Agent construction/resolution semantics
+
+The public operations have distinct meanings:
+
+```text
+new / create
+  create a new logical Agent
+  existing live or durable identity -> AgentAlreadyExistsError
+
+load(agent_id, persistence:)
+  resolve an existing logical Agent
+  live owner -> return the exact same Ruby object without Persistence reload
+  durable-only -> hydrate once and publish as the live owner
+  missing durable Agent -> Persistence::NotFoundError
+
+get(agent_id)
+  process-local live-owner lookup only
+  live -> exact same Ruby object
+  not live -> nil
+  never loads Persistence
+```
+
+A live identity resolved through an incompatible Agent class/definition is an
+explicit configuration error; it is not treated as a cache miss. A `load` call
+that supplies a different Persistence instance from the already-live Agent is
+also rejected rather than silently ignoring the caller's backend argument.
+
+Ruby `.new` remains supported because it is an established application-facing
+construction path. Its meaning is creation, not lookup: `.new(agent_id: "A")`
+does not return an already-existing A.
+
+### Ownership normally lasts for the Runtime lifetime
+
+Once a mutable Agent is live, the Runtime keeps a strong ownership reference.
+Execution completion, idleness and Ruby GC do not release that identity.
+
+A clean Runtime shutdown detaches all live Agent objects from that Runtime. An
+old Ruby reference is no longer a usable mutable Agent after shutdown; attempts
+to operate on it fail with `RuntimeShutdownError`. This prevents a stale object
+from remaining mutable while a new Runtime hydrates the same `agent_id`.
+
+Live Agent entries do not themselves keep Runtime shutdown from completing.
+Only an in-progress ownership transition such as construction or purge must
+settle before clean ownership detachment.
+
+### `purge!` is explicit logical-Agent destruction
+
+`purge!` is the explicit exception to Runtime-lifetime ownership. It first moves
+the exact current live owner into a process-local purging state, preventing new
+materialization/admission, then deletes the durable Agent state.
+
+On known successful purge:
+
+```text
+old Agent object -> permanently purged / unusable
+Runtime registry  -> identity released
+Persistence       -> Agent/Journal/Execution records deleted
+```
+
+The same textual `agent_id` may then be used to create a new logical Agent. A
+stale Ruby reference to the old purged object remains invalid and cannot purge
+or mutate the replacement. Repeating `purge!` on that already-purged stale
+object is an idempotent no-op.
+
+If the purge is known not to have committed, the process-local purging state is
+rolled back to live. If durable outcome is uncertain, the transition becomes a
+stable `RECOVERY_REQUIRED` ownership state. The identity stays fail-closed, but
+load/shutdown waiters are not left blocked on a transition that can no longer
+settle by itself. ACS-15 recovery/reconciliation work is responsible for
+resolving that uncertainty.
+
+### EventLoop owns same-process top-level execution admission
+
+A live Agent may be idle while still owned. Top-level Execution admission is a
+separate EventLoop-owned map keyed by `agent_id`.
+
+For one logical Agent, at most one nonterminal top-level Execution is admitted:
+
+```text
+IDLE + invoke(E1)
+  -> ADMITTING(E1)
+
+ADMITTING / EXECUTING / SUSPENDED / RECOVERY_REQUIRED + invoke(E2)
+  -> AgentBusyError
+```
+
+The admission is acquired on EventLoop before the initial Offload/Persistence
+operation. `Persistence#executions.create_active` remains in that operation as a
+durable second line of defense.
+
+A short-lived opaque owner token protects the pre-durable `ADMITTING` entry so
+only the request that acquired it may release/bind it. The token is a
+process-local coordination capability only. It is not a domain identity,
+semantic operation ID, generic generation counter or asynchronous result
+authority. After durable establishment, the admission is bound to the canonical
+`execution_id`.
+
+### Admission follows the logical Execution lifetime
+
+`preparing`, `active` and `suspended` are all nonterminal. Suspension keeps the
+same admission; approval resume continues the same `execution_id` and does not
+create a new top-level Execution.
+
+The slot is released only after a known-successful durable terminal transition.
+Caller-facing Task settlement and terminal callbacks are notification boundaries
+and do not extend the logical Execution lifetime.
+
+A known pre-durable failure releases the process-local admission. An uncertain
+durable establishment/terminal outcome does not. It is marked
+`RECOVERY_REQUIRED`/fail-closed so another top-level Execution cannot be admitted
+from an unproven state lineage.
+
+### Persistence admission remains a durable integrity capability
+
+`atomic_admission` and `executions.create_active` remain required Persistence
+capabilities. They continue to guarantee durable execution-ID uniqueness and
+that durable nonterminal executions for one `agent_id` do not overlap.
+
+They are no longer described as the primary same-process live/execution
+ownership mechanism. Their role is durable integrity and defensive conflict
+detection, including protection against stale code paths and unsupported
+multi-process races.
+
+Optimistic revision, Journal position and watermark guards remain unchanged.
+Phronomy does not reload/merge durable state after a conflict to continue the
+same logical execution.
+
+## Required invariants
+
+1. One Runtime never publishes two independent mutable Agent objects for the same `agent_id`.
+2. Repeated `load` of a live Agent returns the same Ruby object and does not reload durable state.
+3. `get` is process-local and never performs Persistence I/O.
+4. Agent ownership and EventLoop execution-state directories remain separate responsibilities.
+5. EventLoop admission occurs before initial Persistence execution establishment.
+6. Competing same-Agent top-level requests are rejected with `AgentBusyError`; core does not promise automatic queueing.
+7. Suspension retains the same logical Execution admission.
+8. Known durable terminal success releases admission before caller notification is required to settle.
+9. Unknown durable outcome never causes a blind admission release.
+10. Persistence `create_active`/CAS/revision/watermark protection remains enabled as durable defense.
+11. Runtime shutdown makes old live Agent objects unusable before a later Runtime can authoritatively hydrate the same identity.
+12. Successful `purge!` invalidates the old object before the identity may represent a replacement live Agent.
+
+## Explicitly deferred work
+
+This decision does **not** implement:
+
+- cross-process Agent ownership, lease/fencing or stable routing (ACS-14);
+- durable outcome reconciliation/Agent execution rehydration (ACS-15);
+- cancellation-wide physical-work quiescence supervision (ACS-16);
+- semantic external-operation retry/causal barriers (ACS-17);
+- a general Agent duplication contract.
+
+The future Agent duplication operation is named **`copy`**, not `fork`. It must
+create a new `agent_id`. Which Context, Knowledge, Journal history, metadata or
+provenance is copied, and how nonterminal Execution state is handled, remain a
+separate API/semantic decision. ACS-12 does not add `copy` to the runtime API.
+
+## Rejected alternatives
+
+### Use Persistence `create_active` as the only same-process exclusion
+
+Rejected. It preserves durable exclusion behavior but leaves Runtime authority
+ambiguous and allows competing requests to reach durable I/O before the
+process-local owner has decided which continuation is authoritative.
+
+### Put Agent live ownership into EventLoop's execution directory
+
+Rejected. A live Agent exists while idle and across many sequential Executions.
+`agent_id` ownership and `execution_id` Runtime state have different lifetimes.
+
+### Let repeated `load` create a new object and rely on CAS later
+
+Rejected. CAS detects stale durable writes after two mutable owners already
+exist; it does not satisfy the logical Agent identity invariant.
+
+### Make `.new` return an existing instance
+
+Rejected. Ruby `.new` is creation semantics. Returning an existing object would
+make identity lookup implicit and surprising; `load`/`get` provide resolution.
+
+### Release ownership when an Execution completes or when GC collects the Agent
+
+Rejected. Execution lifetime is shorter than logical Agent live lifetime, and GC
+timing is not an architecture ownership protocol.

data/docs/decisions/026-workflow-runtime-admission-and-durable-terminal-barrier.md

@@ -0,0 +1,257 @@
+# ADR-026: Workflow Runtime Admission and Durable Terminal Barrier
+
+**Status**: Accepted
+**Date**: 2026-08-24
+**Partially supersedes**: [ADR-014](014-unified-persistence-durable-state.md) for same-process Workflow admission ownership and terminal durable-barrier ordering
+**Refines**: [ADR-023](023-fsm-session-incarnation-identity-and-routing.md) by removing the transitional Workflow FSMSession-identity reservation bridge
+**Complements**: [ADR-018](018-durability-guarantees-and-failure-model.md), [ADR-020](020-canonical-workflow-instance-identity.md), [ADR-024](024-event-loop-single-writer-agent-runtime.md), [ADR-025](025-process-local-agent-ownership-and-runtime-admission.md)
+
+---
+
+## Context
+
+`workflow_instance_id` is the canonical logical/durable Workflow identity.
+`fsm_session_id` identifies one concrete Runtime FSMSession incarnation. Those
+identities have different responsibilities and lifetimes.
+
+Before this decision, WorkflowRunner reserved the future FSMSession identity
+before durable hydration and reused that value as the process-local Workflow
+admission owner. This preserved admission-before-load ordering, but it conflated
+Runtime coordination ownership with concrete event-routing identity.
+
+Durable Workflow terminalization also had the wrong lifecycle order. A
+FSMSession first became `halted`/`finished`, EventLoop removed the session and
+settled its source completion, and only then WorkflowRunner saved the final
+Workflow snapshot. A save failure could therefore occur after the runtime
+lifecycle had already declared the execution segment terminal.
+
+The Architecture Baseline requires both inconsistencies to be removed:
+
+```text
+acquire Workflow admission
+    ↓
+load / hydrate durable Workflow state
+    ↓
+create and run one concrete FSMSession
+    ↓
+logical halt / completion reached
+    ↓
+persist terminal snapshot
+    ↓
+known-successful persistence result returns to that FSMSession
+    ↓
+HALTED / COMPLETED
+    ↓
+release Workflow admission
+    ↓
+settle caller-facing Task
+```
+
+## Decision
+
+### Workflow admission uses an opaque Runtime owner token
+
+EventLoop owns a process-local Workflow admission entry keyed by
+`workflow_instance_id`. Admission ownership uses a fresh opaque owner token that
+is independent from the later concrete `fsm_session_id`:
+
+```text
+workflow_instance_id = W
+        ↓
+admission owner token = T1
+        ↓
+concrete FSMSession id = S1
+```
+
+The token is a Runtime coordination capability only. It is not a domain
+identity, durable identifier, application correlation value, generic generation
+counter, or asynchronous semantic-result authority.
+
+Only the exact owner token that acquired the admission may bind or release it.
+A competing/stale token cannot release the current Workflow execution segment.
+
+### Admission precedes mutable durable hydration
+
+Start/resume requests are first posted to EventLoop. EventLoop acquires the
+`workflow_instance_id` admission before any `workflow_states.load` is submitted
+to OffloadPool.
+
+A durable load returns only operation-specific value/snapshot data. Hydration,
+WorkflowContext construction, concrete FSMSession construction, admission-to-
+FSMSession binding, and registration are applied on EventLoop.
+
+This preserves the ordering:
+
+```text
+admit W with T1
+    ↓
+load W
+    ↓
+hydrate live Workflow state
+    ↓
+create S1
+    ↓
+bind T1 -> S1 for routing
+```
+
+`fsm_session_id` remains the direct EventLoop routing identity. Application
+`Workflow#signal(workflow_instance_id: ...)` resolves the current admission to
+its bound `fsm_session_id`; the owner token is never used as an event target.
+
+### Durable Workflow terminalization is part of the FSMSession lifecycle
+
+A durable Workflow that reaches logical halt/completion does not immediately
+become runtime-terminal. The logical Workflow phase and the terminal-persistence
+lifecycle are separate.
+
+Conceptually:
+
+```text
+RUNNING
+   ↓ logical halt/completion result
+PERSISTING_TERMINAL
+   ↓ terminal persistence result
+   ├─ known success  -> HALTED / COMPLETED
+   ├─ known failure  -> ERROR
+   └─ outcome unknown -> RECOVERY_REQUIRED
+```
+
+`PERSISTING_TERMINAL` is not injected into the application-defined Workflow
+state graph. It is private FSMSession/runtime lifecycle state.
+
+When a durable terminal boundary is reached, FSMSession keeps its concrete
+session alive and asks WorkflowRunner to persist the terminal snapshot.
+WorkflowRunner submits a Workflow-specific immutable/value persistence command
+to OffloadPool. The persistence result returns through the same FSMSession's
+session-local event sink. Only that FSMSession may accept the result and advance
+its terminal lifecycle.
+
+Ephemeral Workflow executions that do not require durable terminal persistence
+retain their direct terminal behavior.
+
+### The FSM consumes semantic persistence outcomes, not backend details
+
+FSMSession does not know whether the backend is local, remote, SQL, HTTP-based,
+or otherwise networked. It does not classify database-driver or transport
+exceptions.
+
+The Workflow persistence operation normalizes the save result into three
+semantic outcomes:
+
+```text
+success
+  Phronomy has a known-successful durable result.
+
+known_failure
+  the Phronomy Persistence contract establishes that the intended terminal
+  save did not become the successful durable transition.
+
+outcome_unknown
+  Phronomy cannot establish whether the durable transition committed.
+```
+
+For the durable-barrier question, only `success` is permission to proceed.
+Both other outcomes keep the success barrier closed.
+
+Portable Persistence semantic errors whose contract establishes ordinary
+failure, such as optimistic conflict or serialization rejection, may be treated
+as `known_failure`. An arbitrary backend/storage/transport error is not assumed
+to mean "not committed"; when non-commit is not established by contract, the
+result is conservatively `outcome_unknown`.
+
+This classification depends on semantic certainty, not on whether the physical
+backend is on the same machine or reached over a network.
+
+### Known failure follows the Workflow error path
+
+A `known_failure` terminal save result is delivered back to the owning
+FSMSession. The FSMSession does not enter `HALTED`/`COMPLETED`; it terminalizes
+through its error path instead. EventLoop then releases the admission using the
+opaque owner token and the caller-facing Task settles as failed.
+
+Phronomy does not automatically retry the entire Workflow segment merely because
+the terminal durable save failed. External-effect retry/duplicate semantics are
+separate architecture concerns.
+
+### Outcome uncertainty fails closed
+
+An `outcome_unknown` result is not converted to ordinary failure and is not
+assumed to be success. The known-success durable barrier remains closed.
+
+The concrete FSMSession loses normal continuation authority and is retired. The
+process-local Workflow admission becomes `recovery_required` and remains owned,
+so the Runtime cannot admit a fresh top-level segment from an unproven lineage.
+The caller-facing Workflow Task is not falsely settled as success or failure.
+
+Actual persistence-outcome reconciliation and restart-safe continuation are
+ACS-15 responsibilities. ACS-13 establishes the fail-closed boundary but does
+not claim that recovery is already implemented.
+
+### Successful terminalization releases admission after FSM acceptance
+
+On a known-successful terminal save, the result first returns to the same
+FSMSession. The FSMSession accepts it and only then emits its normal
+`halted`/`finished` terminal event.
+
+EventLoop removes the concrete session, and WorkflowRunner releases the
+`workflow_instance_id` admission using its opaque owner token before settling
+the caller-facing Task.
+
+The save completing on a worker thread is therefore not itself Workflow
+completion. Authoritative logical completion occurs only after the EventLoop-
+owned lifecycle accepts that save result.
+
+## Required invariants
+
+1. One Runtime admits at most one live Workflow execution segment for a `workflow_instance_id`.
+2. Workflow admission is acquired before mutable durable Workflow load/hydration.
+3. Workflow admission owner token and `fsm_session_id` are different Runtime concepts.
+4. Only the exact admission owner token may bind/release its Workflow admission.
+5. `fsm_session_id` remains the concrete EventLoop event-routing identity.
+6. Durable halt/completion does not become runtime-terminal before a known-successful terminal snapshot save is accepted by the owning FSMSession.
+7. Terminal Persistence I/O never blocks EventLoop and never mutates the live FSMSession from a worker.
+8. FSMSession receives backend-independent semantic save outcomes, not backend/transport classifications.
+9. Known terminal save failure does not become successful `HALTED`/`COMPLETED`.
+10. Unknown terminal save outcome does not release admission or settle the caller Workflow Task as a terminal success/failure.
+11. Ephemeral Workflows do not acquire a fake durable barrier merely to match the durable path.
+12. Persistence optimistic revision remains a durable integrity/conflict defense and is not the same-process ownership mechanism.
+
+## Explicitly deferred work
+
+This decision does **not** implement:
+
+- cross-process Workflow ownership, routing, lease/fencing (ACS-14);
+- Persistence F1 outcome reconciliation or restart-safe Workflow rehydration (ACS-15);
+- cancellation/semantic-deadline quiescence and terminalization integration (ACS-16);
+- general external semantic-operation retry/idempotency protocol (ACS-17);
+- a new public Workflow lifecycle-state API;
+- Persistence-backend-specific network/driver exception taxonomies in FSM code.
+
+## Rejected alternatives
+
+### Continue using the future `fsm_session_id` as admission owner
+
+Rejected. It preserves pre-load exclusion but conflates coordination ownership
+with concrete routing identity and makes the admission authority depend on a
+session that does not yet exist.
+
+### Save only after FSMSession has emitted `halted`/`finished`
+
+Rejected. Runtime terminalization would precede the durable outcome that is
+required to justify it, violating the Workflow durable barrier.
+
+### Block EventLoop until Persistence returns
+
+Rejected. A durable barrier is logical execution ordering, not EventLoop-wide
+blocking. Persistence remains OffloadPool work.
+
+### Treat every save exception as known failure
+
+Rejected. A backend may have committed even though its success response was not
+observed. Blind release/retry could branch Workflow lineage or duplicate later
+semantic work.
+
+### Teach FSMSession about SQL/network/backend exception classes
+
+Rejected. Physical backend topology and driver error taxonomies belong below the
+Workflow FSM boundary. FSMSession consumes semantic persistence outcomes only.

data/docs/decisions/027-llm-adapter-provider-boundary.md

@@ -0,0 +1,93 @@
+# ADR-027: Phronomy-Owned LLM Adapter and RubyLLM Default Provider Boundary
+
+## Status
+
+Accepted
+
+## Context
+
+ADR-001 selected RubyLLM as Phronomy's LLM provider layer. Its original
+Decision text also encoded an implementation boundary in which Agent code and
+the legacy Chain API called `RubyLLM.chat` directly.
+
+The current architecture has evolved:
+
+```text
+Phronomy Agent / Context / Manifest
+        |
+        v
+RubyLLMMaterializer
+        |
+        v
+Phronomy::LLMAdapter::Base
+        |
+        v
+configured adapter
+(default: Phronomy::LLMAdapter::RubyLLM)
+```
+
+Phronomy now owns Agent lifecycle, Runtime coordination, canonical Journal /
+Context Policy / Manifest semantics, durable execution state, and the
+framework-owned asynchronous/offload bridge around Provider calls.
+
+At the same time, the current materialization path is still RubyLLM-specific:
+the LLMAdapter SPI receives the configured/materialized chat runtime object.
+Making the Provider-call boundary replaceable does not by itself make the
+complete LLM-input materialization pipeline provider-neutral.
+
+The public LLMAdapter SPI is currently classified Beta. Architecture
+reconciliation must not silently promote its API stability.
+
+## Decision
+
+1. Phronomy owns Agent lifecycle, Context/Manifest authority, Runtime
+   coordination, and durable execution semantics. These responsibilities are
+   not delegated to `RubyLLM::Agent`.
+2. `Phronomy::LLMAdapter::Base#complete` and `#stream` are the public
+   Provider-call extension boundary. Phronomy owns the framework-side
+   asynchronous/offload wrappers around that synchronous adapter contract.
+3. `Phronomy::LLMAdapter::RubyLLM` remains the default configured LLM adapter
+   and RubyLLM remains Phronomy's default Provider integration.
+4. A custom LLMAdapter may replace Provider-call behavior, but the LLMAdapter
+   SPI does not imply that the complete input-materialization pipeline is
+   Provider-neutral. The current Agent pipeline still materializes canonical
+   LLM input through RubyLLM-specific runtime objects.
+5. Provider transport timeout, retry, backoff, jitter, and rate-limit handling
+   remain adapter/provider-client responsibilities, consistent with
+   [`011-delegate-transport-policy-to-adapters`](011-delegate-transport-policy-to-adapters.md).
+6. This decision does not expand Phronomy scope merely because RubyLLM or
+   another Provider exposes additional features.
+7. This decision does not change the existing API stability classification of
+   the LLMAdapter SPI.
+
+## Consequences
+
+### Positive
+
+- The normative architecture matches the current Phronomy-owned Agent /
+  Context / Manifest pipeline.
+- RubyLLM remains the default integration without making direct RubyLLM calls
+  the Phronomy extension contract.
+- Applications can supply a custom call adapter through one explicit boundary.
+- Runtime/offload semantics stay Phronomy-owned while transport policy stays
+  adapter-owned.
+- Provider-call replaceability is not confused with full materialization
+  neutrality.
+
+### Tradeoffs
+
+- Custom adapters currently receive Phronomy's materialized chat runtime object
+  and therefore may still depend on the RubyLLM-shaped materialization boundary.
+- Replacing the current RubyLLM-specific materializer would require a separate
+  architecture/API decision if Phronomy later wants end-to-end Provider-neutral
+  materialization.
+- The Beta LLMAdapter SPI may still evolve according to the repository's
+  compatibility policy.
+
+## Supersession
+
+This decision supersedes
+[`001-rubyllm-as-provider-layer`](001-rubyllm-as-provider-layer.md).
+
+ADR-001 remains preserved as historical rationale for adopting RubyLLM, but its
+direct-call boundary is no longer normative.