phronomy 0.22.0 → 0.23.0

data/docs/migrations/0.22.md

@@ -0,0 +1,390 @@
+# Migration to 0.22
+
+Phronomy 0.22 includes clean-break Agent definition identity normalization and
+semantic Multi-Agent Handoff changes.
+
+## Agent definition identity and revision
+
+Agent instance identity and Agent definition identity are separate:
+
+```text
+Agent instance                  agent_id
+Agent definition lineage       agent_definition_id
+Agent definition revision      agent_definition_version
+```
+
+A named concrete Agent may now use its fully-qualified Ruby class name as the
+default definition lineage:
+
+```ruby
+module Billing
+  class Agent < Phronomy::Agent::Base
+    agent_definition version: 2
+  end
+end
+```
+
+The definition ID above is `Billing::Agent`. An explicit stable lineage remains
+supported:
+
+```ruby
+agent_definition id: "billing-agent", version: 2
+```
+
+Concrete subclasses do not implicitly inherit a parent's definition revision.
+Anonymous Agent subclasses cannot derive a fully-qualified class-name lineage
+and therefore require explicit `id:` when declaring their definition identity.
+
+The semantic revision field is a clean rename:
+
+```text
+AgentRoot#definition_version
+  -> AgentRoot#agent_definition_version
+
+AgentRoot durable key "definition_version"
+  -> "agent_definition_version"
+
+LLMInputBuildContext#definition_version
+  -> #agent_definition_version
+```
+
+No deprecated field alias or legacy AgentRoot backward decoder is provided.
+Persisted AgentRoot hashes that contain only `"definition_version"` must be
+migrated by the application before they are loaded by 0.22. Phronomy does not
+rewrite existing AgentRoot data automatically in this change.
+
+Existing `agent_id` values keep their identity semantics. Agent loading still
+uses the current exact-match policy for `(agent_definition_id,
+agent_definition_version)`. This semantic definition revision is not a
+Persistence codec/schema/format version.
+
+## Multi-Agent Handoff
+
+Multi-Agent Handoff changes from Tool-result/sentinel routing to an explicit
+framework control-transfer model. This is also a clean break: compatibility
+aliases for the removed Runner and Handoff routing API are intentionally not
+provided.
+
+## Runner API
+
+Before:
+
+```ruby
+runner = Phronomy::Agent::Runner.new(
+  agents: [triage, billing],
+  routes: {triage => [billing]}
+)
+```
+
+After:
+
+```ruby
+handoff = Phronomy::MultiAgent::Handoff.new(
+  source_agent: triage,
+  target_agent: billing
+)
+
+runner = Phronomy::MultiAgent::Runner.new(
+  main_agent: triage,
+  handoffs: [handoff]
+)
+```
+
+Removed public surface:
+
+- `Phronomy::Agent::Runner`
+- Runner constructor keywords `agents:` and `routes:`
+- Runner `#agents`
+- sentinel-based routing internals as a public compatibility boundary
+
+## Handoff model
+
+`Phronomy::MultiAgent::Handoff` now represents one semantic Source-to-Target
+edge between concrete live Agent instances:
+
+```ruby
+handoff = Phronomy::MultiAgent::Handoff.new(
+  source_agent: triage,
+  target_agent: billing,
+  description: "Transfer billing responsibility"
+)
+```
+
+Generated Tool names are transport details only. Applications must not persist,
+inspect, or route by those generated names. `Handoff#tool_name`,
+`Handoff#sentinel`, and `Handoff#to_tool_class` are not part of the new API.
+
+A Handoff request contains a required `responsibility` string. It is intercepted
+before ordinary Tool execution, so no synthetic Tool result is inserted merely
+to signal routing.
+
+One Provider outcome may contain either ordinary Tool Calls or one Handoff
+request. Mixing Handoff with ordinary Tool Calls, or returning multiple Handoff
+requests in one Provider outcome, raises `Phronomy::HandoffError`.
+
+## Handoff Policy and Context transfer
+
+`Phronomy::MultiAgent::HandoffPolicy` controls what may cross the Agent ownership
+boundary. The initial categories are:
+
+- `current_request`
+- `history`
+- `knowledge`
+- `tool_exchanges`
+
+Each category is `required`, `forbidden`, or `selectable`. Selectable categories
+have an Application-defined default of include or exclude. Required and forbidden
+categories are not exposed to the Source Agent as switches.
+
+The default policy requires the current request, includes history and Tool
+exchanges by default, and keeps persistent Knowledge opt-in.
+
+Transferred material is immutable request-scoped Handoff Context. It is not
+adopted automatically into the Target Journal or Target Knowledge. Provenance is
+preserved across multi-hop transfers.
+
+Handoff Policy and Context Policy have different authority:
+
+```text
+Handoff Policy
+  -> decides what may cross Source -> Target
+
+Target Context Policy
+  -> decides what is used for one Target LLM call
+```
+
+Transferred items therefore enter Target context assembly as ordinary selectable
+Context candidates before Target Context Policy runs.
+
+## Active Agent continuity
+
+The `main_agent` instance anchors one Runtime-local coordination lifetime. After
+a successful Handoff, the Target remains active for later user turns while the
+same main Agent instance and Runtime remain alive. Recreating only the
+`MultiAgent::Runner` facade does not reset that state.
+
+This active-Agent ownership is deliberately not durable. A process or Runtime
+reset starts a new coordination lifetime at `main_agent`; Phronomy does not infer
+active-Agent continuation from historical Handoff audit records.
+
+## Source execution outcome
+
+A Source Agent execution that transfers responsibility terminates explicitly as
+`handed_off`. The complete Provider outcome is retained for audit, but the
+Handoff control Tool Call is not made a future conversational Context candidate
+and no ordinary Handoff Tool result is generated.
+
+## Context Policy clean break (ACS-04)
+
+ACS-04 replaces the intermediate Context selection composition API with the
+final four-category Context Policy SPI. Application-defined policies receive an
+immutable `Phronomy::Agent::ContextPolicyInput` and return an immutable
+`Phronomy::Agent::ContextPlan`, each organized as `instruction`, `knowledge`,
+`tools`, and grouped `conversation`.
+
+Agent classes bind one Application-constructed Policy instance:
+
+```ruby
+class ResearchAgent < Phronomy::Agent::Base
+  context_policy RESEARCH_CONTEXT_POLICY
+end
+```
+
+The following intermediate APIs/internals are removed without compatibility
+aliases:
+
+- `ContextRequest`
+- `ContextPolicyDescriptor` / `ContextPolicyRegistry`
+- `DerivedContentSpec` and `ContextPlan#derived_contents`
+- `ContextPlan#selected_unit_ids`, `#ordering_hints`, `#policy_descriptor`
+- `Selection::Unit` / `Selection::Validator`
+- `DependencyAwareUnitBuilder`
+- `RequiredContextResolver`
+- `RecentFirstSelector`
+- `TokenBudgetPacker`
+- public Policy `parts` composition
+
+`Selection::Candidate` and `Selection::Constraint` remain internal normalization
+mechanisms while `ContextPolicyInput` is built; they are not the Application
+Policy SPI.
+
+ContextPolicy is Application code/runtime wiring rather than durable Agent
+state. The Policy object/descriptor is not persisted or reconstructed. A
+finalized `LLMInputManifest` remains the Recovery authority for an already-fixed
+Provider input.
+
+No Phronomy Persistence transaction spans `ContextPolicy#call`. Context is
+prepared from an immutable snapshot outside the transaction, then the durable
+base/revision is revalidated before the short Manifest/execution commit.
+
+The Context assembly policy version is now `8` because the semantic input/Plan
+contract, Tool subset handling, Policy-generated current-call material, and
+transaction boundary changed. Persisted Manifest readers that validate this
+version should accept version 8.
+
+## Workflow instance identity
+
+Workflow durable identity is a clean rename:
+
+```text
+thread_id
+  -> workflow_instance_id
+```
+
+The new name identifies one logical Workflow instance and is also the logical
+key supplied to `Persistence#workflow_states`.
+
+Application code must change Workflow invocation configuration and live signals:
+
+```ruby
+workflow.invoke(
+  input,
+  config: {workflow_instance_id: "order-123"}
+)
+
+workflow.signal(
+  workflow_instance_id: "order-123",
+  event: :external_result,
+  payload: result
+)
+```
+
+`WorkflowContext#thread_id` is removed and replaced by
+`WorkflowContext#workflow_instance_id`. No deprecated alias is provided.
+Supplying the old Workflow config key `:thread_id` raises `ArgumentError`
+instead of silently allocating a new Workflow identity.
+
+`workflow_instance_id` is framework-owned WorkflowContext metadata and is now a
+reserved application field name. If an existing context declares
+`field :workflow_instance_id`, rename that application field before upgrading;
+0.22 rejects the declaration with `ArgumentError`.
+
+The identifier value itself does not change. Existing durable key value
+`"order-123"` remains `"order-123"` and is interpreted as the same
+`workflow_instance_id`. Custom Persistence backends must rename the logical
+parameter in their Workflow repository SPI, but a database adapter does not need
+to rename a physical column solely for this migration.
+
+`InvocationContext#thread_id` is not used as an implicit Workflow identity after
+this change. CG-01 established the Workflow identity boundary; the generic Agent
+invocation identity cleanup described below completes the separate Agent-side
+removal.
+
+## Generic Agent invocation identity
+
+Agent-side generic invocation correlation is removed as a clean break.
+
+Removed public surface:
+
+```text
+InvocationContext#thread_id
+InvocationContext#session_id
+Agent#invoke(thread_id:)
+Agent#invoke_async(thread_id:)
+Agent#stream(thread_id:)
+Agent#stream_async(thread_id:)
+MultiAgent::Orchestrator#fan_out(thread_id:)
+MultiAgent::Orchestrator#fan_out_async(thread_id:)
+MultiAgent::Orchestrator#subagent(thread_id:)
+```
+
+No `correlation_id`, `conversation_id`, or `application_session_id` replacement
+is introduced. Applications that need correlation should keep it in their own
+tracing/observability metadata or use the purpose-specific domain identifier
+that owns the relevant lifecycle.
+
+`InvocationContext#task_id` and `#parent_task_id` remain tracing/observability
+identifiers. They are not Agent, Workflow, Tool, or Runtime routing identity.
+
+Agent invocation config also rejects the removed `thread_id` and `session_id`
+keys so the old generic identity cannot survive through an untyped config
+backdoor.
+
+The later CG-03b / ACS-10 Runtime foundation removes the shared generic
+FSMSession graph-identity metadata bridge. Agent, Tool, and Multi-Agent routing
+now uses concrete session-local sinks; Workflow metadata is passed explicitly as
+`workflow_instance_id` while its admission-owner redesign remains ACS-13 work.
+
+`JournalRecord#correlation_id` is also removed from the current canonical
+Journal model and `JournalRecord#to_h` representation. New approval and
+execution Journal records do not populate a generic correlation field.
+
+Existing durable Journal Hashes are not rewritten merely to remove the legacy
+key. `JournalRecord.from_h` accepts a legacy `correlation_id` key and ignores it
+while restoring the current canonical fields. The same rule applies to legacy
+Journal records nested in `AgentExecution#working_records`, because
+`AgentExecution.from_h` delegates their reconstruction to `JournalRecord.from_h`.
+
+Custom Persistence backends should stop producing `correlation_id` in new
+canonical Journal representations. A backend does not need to rewrite every
+existing stored row or drop a physical legacy column solely for this migration;
+old values may remain backend-internal while legacy records are decoded into the
+current correlation-free domain model.
+
+## Agent execution parent identity
+
+Tool and approval work now identifies its owning Agent execution with
+`execution_id`. The old `agent_invocation_id` terminology is not retained as a
+deprecated alias.
+
+Application-facing changes:
+
+```text
+ToolApprovalRequest#agent_invocation_id
+  -> ToolApprovalRequest#execution_id
+
+ToolApprovalRequest#to_h[:agent_invocation_id]
+  -> ToolApprovalRequest#to_h[:execution_id]
+
+ApprovalEvaluationRequest#agent_invocation_id
+  -> ApprovalEvaluationRequest#execution_id
+```
+
+ACS-11 also makes Tool approval-policy input a value-only worker-boundary
+contract. The live-object accessors are removed as a clean break:
+
+```text
+ApprovalEvaluationRequest#agent
+  -> #agent_id / #agent_definition_id / #agent_definition_version
+
+ApprovalEvaluationRequest#tool
+  -> #tool_name / #tool_schema / #origin / #metadata
+```
+
+The authorization worker captures Tool approval behavior before offload and
+does not receive a live Agent, Tool, or ToolInvocation object. Application
+approval/facts/requirement callables remain supported as Application-owned
+behavior handles.
+
+`tool_invocation_id`, provider `tool_call_id`, and the approval request's own
+`id` remain separate purpose-specific identifiers.
+
+Agent invocation config also rejects `agent_invocation_id` so application code
+cannot continue controlling the temporary Runtime routing identifier through an
+untyped config key.
+
+Suspended `AgentExecution` records created by an older version may contain
+`approval_request["agent_invocation_id"]`. `AgentExecution.from_h` accepts that
+legacy embedded shape, discards the old Runtime-oriented value, and derives the
+current logical parent from the enclosing execution's own `execution_id`. It
+does not reinterpret the historical `agent_invocation_id` value as an execution
+ID.
+
+Existing content-addressed approval audit bodies are not rewritten and may
+retain their historical key. New approval request writes use `execution_id`.
+This targeted compatibility handling does not define the general Persistence
+schema/versioning policy.
+
+CG-03b / ACS-10 now advances the Runtime routing half. `AgentInvocation` has no
+independent routing ID or duplicate `session_id`; `ToolInvocation#id` remains
+only the Tool lifecycle identity; and the private `parent_agent_invocation_id`
+routing carrier is removed. Agent, Tool, and Multi-Agent concrete sessions use
+fresh FSMSession IDs and session-local event sinks. Provider completion is routed
+to the owning FSMSession and carries `llm_call_id` before any live result apply.
+
+ACS-11 removes `AgentExecutionActivation` / `ActivationRegistry`: EventLoop is the
+single writer of Phronomy-managed live Agent execution state, and operation-
+specific Offload results are validated against current FSMSession/FSM state and
+semantic IDs before apply. Together with ADR-023, this closes the Agent/Tool
+CG-03 Runtime-foundation gap. Workflow opaque admission ownership remains
+separate ACS-13 work.

data/docs/persistence-backends.md

@@ -23,10 +23,13 @@ Phronomy::Persistence synchronous Backend SPI
 Database / durable storage
 ```
 
-Persistence does not own live execution state. In particular, a backend must not
-persist or reconstruct the following as part of this SPI:
+Persistence does not own live Agent identity, top-level Runtime admission, or
+live execution state. In particular, a backend must not persist or reconstruct
+the following as part of this SPI:
 
-- `AgentExecutionActivation`;
+- Runtime Agent ownership-registry entries;
+- EventLoop Agent top-level admission entries;
+- EventLoop Agent execution-directory entries;
 - `AgentInvocation`;
 - `FSMSession`;
 - `Task` or callbacks;
@@ -96,8 +99,9 @@ failure rather than pretending the outcome is known.
 
 ### `atomic_admission`
 
-This capability refers to **Agent execution admission**, not Workflow distributed
-locking.
+This capability is a **durable Agent execution integrity defense**. It is not the
+primary same-process Agent ownership/admission mechanism and it is not Workflow
+distributed locking.
 
 For one Agent, `executions.create_active` must atomically guarantee both:
 
@@ -110,9 +114,15 @@ no active/suspended execution already exists for agent_id
 A conflict with an existing active/suspended execution raises
 `Phronomy::AgentBusyError`.
 
-Workflow admission remains Runtime/process-local. Cross-process Workflow
-lease/fencing is an application/distributed-coordination concern and is not part
-of this Backend SPI.
+Within one process, Runtime/EventLoop admission is acquired before the initial
+Persistence operation and is the primary competing-execution exclusion
+mechanism. `atomic_admission` remains required as the durable second line of
+defense against stale paths, durable conflicts, and unsupported cross-process
+races. It must not be removed merely because Runtime admission exists.
+
+Workflow admission remains Runtime/process-local. Cross-process Agent or
+Workflow ownership/lease/fencing is a separate distributed-coordination concern
+and is not part of this Backend SPI.
 
 ### `optimistic_revision`
 
@@ -137,8 +147,10 @@ identity, duplicate-ID, or compare-and-swap conflicts.
 
 ### `Phronomy::AgentBusyError`
 
-An Agent already has an active or suspended execution and another execution
-cannot be admitted.
+A durable nonterminal Agent execution already exists and another durable
+execution record cannot be established. Phronomy also uses the same public error
+for a competing process-local top-level request rejected by Runtime/EventLoop
+before the backend is called.
 
 ### `Phronomy::Persistence::SerializationError`
 
@@ -202,6 +214,11 @@ def delete(agent_id)
 - returns `Phronomy::Agent::AgentRoot`, not a raw database Hash;
 - raises `NotFoundError` when missing.
 
+These repository operations are durable-storage primitives. The higher-level
+`Agent::Base.load` API first consults Runtime's process-local live ownership
+registry and does not call the repository when the requested Agent is already
+live.
+
 `save` atomically checks:
 
 ```text
@@ -257,8 +274,11 @@ def delete_for_agent(agent_id)
 def assert_idle!(agent_id)
 ```
 
-`create_active` performs atomic Agent admission. A duplicate `execution_id`
-raises `ConflictError`; an already busy Agent raises `AgentBusyError`.
+`create_active` performs atomic **durable** Agent execution admission. A duplicate
+`execution_id` raises `ConflictError`; an already busy Agent raises
+`AgentBusyError`. Runtime/EventLoop has already acquired the process-local
+logical execution slot on the normal Phronomy path before this repository method
+runs.
 
 `load` returns `Phronomy::Agent::AgentExecution`, not a raw database Hash, and
 raises `NotFoundError` when missing.
@@ -278,17 +298,19 @@ A failed precondition raises `ConflictError`.
 `assert_idle!` is used inside transactions before Agent context/Knowledge changes
 and destructive operations. It must raise `AgentBusyError` if an active/suspended
 execution exists. A SQL implementation must make this check part of a consistency
-boundary that cannot race with Agent execution admission; a best-effort SELECT
-outside the transaction is not sufficient.
+boundary that cannot race with durable Agent execution admission; a best-effort
+SELECT outside the transaction is not sufficient. Process-local Runtime
+admission is an additional upstream coordination layer, not a replacement for
+this durable check.
 
 ## Workflow states repository
 
 Required surface:
 
 ```ruby
-def load(thread_id)
-def save(thread_id, expected_revision:, snapshot:)
-def delete(thread_id, expected_revision:)
+def load(workflow_instance_id)
+def save(workflow_instance_id, expected_revision:, snapshot:)
+def delete(workflow_instance_id, expected_revision:)
 ```
 
 `load` returns `nil` when no row exists. Otherwise it returns a Hash containing a
@@ -315,6 +337,37 @@ snapshot and revision. String or Symbol Hash keys are accepted by Phronomy:
 
 Caller mutation of a loaded snapshot must not mutate durable storage.
 
+### Workflow Runtime admission and terminal-save outcome
+
+Same-process Workflow admission is owned by Runtime/EventLoop, not by this
+repository. EventLoop acquires an opaque owner token for `workflow_instance_id`
+before mutable durable load/hydration, then binds a separately generated
+`fsm_session_id` only after the concrete FSMSession is constructed.
+
+A durable Workflow terminal/halt snapshot is saved through OffloadPool while the
+owning FSMSession remains nonterminal. The backend still implements only the
+synchronous `save` contract above; it does not post Runtime events or decide FSM
+state.
+
+Phronomy interprets terminal-save results by semantic certainty:
+
+```text
+known successful save
+  -> durable barrier may be crossed
+
+portable known failure / known not committed
+  -> barrier remains closed; Workflow error path
+
+arbitrary storage/transport failure whose commit outcome is not established
+  -> outcome unknown; barrier remains closed and Runtime fails closed
+```
+
+This distinction is independent of physical topology. A local backend can have
+an uncertain outcome, and a remote backend can return a definite optimistic
+conflict. Backends must therefore preserve meaningful portable errors when the
+contract establishes them, and must surface other storage/transport failures
+honestly rather than converting them into `ConflictError`.
+
 ### Workflow value serialization
 
 `WorkflowContext#to_h` may contain ordinary Ruby application values. The
@@ -414,6 +467,17 @@ Phronomy::Agent::AgentExecution.from_h(hash)
 String and Symbol top-level keys are accepted by these new execution/call codecs,
 which permits adapters to use parsed JSON without reimplementing constructors.
 
+The current canonical `JournalRecord` Hash does not contain `correlation_id`.
+Legacy durable Journal Hashes that still contain that key may be passed to
+`JournalRecord.from_h`; the legacy key is accepted and ignored. Backends must not
+synthesize or populate `correlation_id` for new canonical Journal records, and
+they are not required to eagerly rewrite existing durable rows solely to remove
+the old physical value.
+
+This is a targeted migration rule for the removed generic identity field. It does
+not establish a general unknown-field or long-term codec/schema-versioning
+policy.
+
 The canonical Hash representation is the Phronomy/domain boundary. A backend is
 free to map that representation to normalized SQL columns, JSON, or another
 storage format internally.
@@ -467,10 +531,12 @@ require wrappers only; the authoritative shared-example implementations live
 under `lib/phronomy/testing/persistence_contract/` so the core suite and external
 backends cannot drift through copied definitions.
 
-The generic suite verifies repository behavior, CAS semantics, admission,
-mutation isolation, and whole-backend transaction behavior. Database-specific
-concurrency/locking mechanisms remain backend integration-test concerns; the SPI
-specifies outcomes rather than a particular SQL locking strategy.
+The generic suite verifies repository behavior, CAS semantics, durable execution
+admission, mutation isolation, and whole-backend transaction behavior. Runtime
+same-process ownership/admission is tested separately because it is not a
+Persistence Backend SPI responsibility. Database-specific concurrency/locking
+mechanisms remain backend integration-test concerns; the SPI specifies outcomes
+rather than a particular SQL locking strategy.
 
 ## SQL implementation guidance
 
@@ -481,24 +547,8 @@ may use combinations of:
 - conditional `UPDATE ... WHERE revision = ?`;
 - row locks;
 - serializable/repeatable-read isolation where appropriate;
-- partial unique indexes for active Agent execution admission;
+- partial unique indexes for durable active Agent execution admission;
 - transaction-scoped checks for Agent revision + Journal head.
 
 Backend-specific database exceptions should be translated to the Phronomy error
 contract where their meaning is known.
-
-## Explicit non-goals
-
-This Backend SPI does not provide:
-
-- durable reconstruction of a lost Agent Activation;
-- serialization of Runtime objects;
-- cross-process Workflow execution exclusion;
-- exactly-once external Tool side effects;
-- automatic conflict reload/merge;
-- a generic serializer registry for arbitrary Workflow field classes;
-- an async Persistence API.
-
-For the architectural reasons behind these boundaries, see
-[ADR-014: Unified Persistence for Durable State](decisions/014-unified-persistence-durable-state.md)
-and [Runtime and concurrency](runtime-and-concurrency.md).