phronomy 0.22.0 → 0.24.0

data/docs/decisions/028-preparing-recovery-replay-contract.md

@@ -0,0 +1,106 @@
+# ADR-028: Replay-safe recovery for durably admitted `:preparing` Agent executions
+
+Status: Accepted
+
+## Context
+
+ADR-018 defines Agent execution resumption as continuation of the same logical
+`execution_id` from durable recovery state when there is no unresolved semantic
+operation that prevents safe continuation. The implementation already persists a
+new `AgentExecution` in `status: :preparing, phase: :preparing` before input
+filters, `before_llm_input`, Context Policy, and Manifest finalization run.
+
+Before this decision, process loss in that window left a durable active execution
+with no automatic continuation path.
+
+## Decision
+
+A `:preparing` execution may be resumed automatically only when the framework
+durably recorded `preparation_replayable == true`. Missing, false, or unsupported
+values fail closed. The recovered continuation keeps the same `execution_id`.
+
+The preparation region is **replay-safe, not deterministic**. Input filtering,
+`before_llm_input`, Context Policy, retrieval, and other preparation work may be
+executed again after process loss. Applications must ensure those callbacks are
+safe under at-least-once execution. The framework does not require repeated
+preparation to produce byte-identical results.
+
+Before Manifest finalization, preparation results may be recomputed. After the
+Manifest is durably committed, the Manifest is the authority and existing
+post-Manifest recovery rules apply.
+
+### Durable application context
+
+`config` remains a runtime/application Hash and is not generally durable. An
+Application value that affects preparation semantics and must survive restart is
+placed under one reserved key:
+
+```ruby
+config: {
+  durable_context: {
+    "tenant" => "A",
+    "search_profile" => "legal"
+  }
+}
+```
+
+`config[:durable_context]`, when present, must be a Hash accepted by
+`Phronomy::CanonicalJSON`. The framework performs a Canonical JSON round trip
+before execution admission and uses the detached immutable snapshot for both the
+initial run and any recovery replay. `durable_context: nil` and non-Hash values
+are rejected before an `AgentExecution` is created. Missing and explicit `{}` are
+distinct.
+
+The snapshot is stored in the Content Store and the execution retains only
+`durable_context_ref`.
+
+### Conservative replay eligibility
+
+This change intentionally does not add a new arbitrary raw-input serialization
+format. Current initial admission stores `extract_message(input)` as text, while
+filters and instruction construction may inspect the original Ruby input.
+Therefore automatic `:preparing` replay is enabled only for String invocation
+inputs. Non-String inputs fail closed after process loss at this phase.
+
+Automatic preparation replay is also disabled when the framework can see a
+Runtime-only semantic dependency that it cannot reconstruct, including:
+
+- Multi-Agent handoff/routing wiring;
+- a custom Agent invocation approval policy;
+- invocation-context approval/redaction/token-budget policy values.
+
+These conditions are represented by the single durable boolean
+`preparation_replayable`; the framework does not persist those Runtime objects.
+A true value records that no framework-known blocker was present; it does not
+replace the Application replay-safety contract for callbacks or custom config.
+
+Applications that use other ordinary `config` entries to influence preparation,
+Provider-adapter, Tool, or other continuation semantics are responsible for moving
+restart-required values into `config[:durable_context]` and reading them from that
+sub-Hash after recovery.
+
+### Agent definition compatibility
+
+Existing `agent_definition_id` / `agent_definition_version` load validation
+continues to guard runtime definition compatibility. Applications must increment
+the Agent definition version when a change to filters, hooks, Context Policy, or
+other preparation behavior is not recovery-compatible.
+
+## Non-goals
+
+This decision does not add:
+
+- recovery of caller `Task` objects;
+- durable Multi-Agent active routing or Handoff Context;
+- a durable execution-query API;
+- a new Recovery event;
+- Content Store garbage collection;
+- Workflow checkpoint changes;
+- deterministic replay of Application callbacks.
+
+## Consequences
+
+Direct, replay-safe String invocations can continue the same durable logical
+execution after process loss in the initial preparation window. Unsupported or
+ambiguous cases remain fail-closed rather than being guessed or silently
+abandoned.

data/docs/decisions/029-semantic-completion-and-application-effect-boundary.md

@@ -0,0 +1,220 @@
+# ADR-029: Semantic Completion and Application Effect Boundary
+
+## Status
+
+Accepted. V2 revision 2, 2026-09-06.
+
+User approval covers the V2 boundary and the five recovery-contract clarifications.
+Acceptance is design authority; it is not a claim of repository integration or test success.
+
+## Date
+
+2026-09-06
+
+## Context
+
+Phronomy durably records Agent semantic execution progress and terminal outcomes.
+After a terminal durable commit, Runtime may notify Application code through
+`on_event` and settle process-local caller Tasks.
+
+A previous proposal attempted to make terminal callback delivery itself durable
+across process loss by storing a pending-delivery descriptor, rediscovering
+pending callbacks on `Agent.load`, and durably acknowledging callback attempts.
+
+That approach crossed an architectural boundary.
+
+An arbitrary Application callback may perform an external effect outside
+Phronomy's Persistence transaction domain. Phronomy cannot make that effect
+exactly once. A process can die after the callback has produced an effect but
+before Phronomy records an acknowledgement, so retrying the callback still
+requires Application-level idempotency.
+
+Maintaining a framework outbox/ACK protocol therefore does not remove the
+Application responsibility that matters most, while it adds persistence indexes,
+recovery ordering, acknowledgement reconciliation and callback-specific state to
+the Agent execution engine.
+
+## Decision
+
+### 1. Semantic terminal state remains the durable boundary
+
+The existing AgentExecution terminal statuses remain semantic terminal states:
+
+```text
+completed
+handed_off
+failed
+cancelled
+rejected
+blocked
+```
+
+Phronomy does not add a callback-only `:completing` status.
+
+A known-successful terminal durable transition ends the logical AgentExecution and
+releases normal Agent admission according to ADR-025.
+
+### 2. Terminal semantic result/error evidence is durable
+
+The terminal transaction continues to persist the canonical semantic evidence
+already required by Agent durability, including as applicable:
+
+```text
+terminal AgentExecution
+AgentRoot revision/lifecycle state
+Journal terminal facts
+result_ref / error_ref
+Provider / Tool durable evidence already owned by Agent execution
+```
+
+Process loss after this commit must not cause the semantic execution to be rerun
+merely because the Application did not observe its completion callback.
+
+### 3. `on_event` is a process-local observation contract
+
+Application `on_event` callbacks are Runtime observations.
+
+Phronomy invokes them in the current process according to the existing callback
+error policy, but does not create a restart-spanning delivery obligation.
+
+Phronomy does not persist:
+
+```text
+terminal_delivery
+delivery_pending
+callback attempt_count
+callback acknowledgement state
+callback/Proc/Task references
+```
+
+`Agent.load` does not scan for or redeliver missed terminal callbacks.
+
+### 4. Callback loss after process loss is allowed
+
+The following failure is explicitly permitted:
+
+```text
+terminal semantic commit succeeds
+  -> process dies before Application callback
+  -> callback is not reconstructed/redelivered
+```
+
+The semantic outcome remains authoritative and must not be replayed.
+
+An Application that requires restart-spanning notification must implement that
+requirement in an Application-owned durable mechanism such as:
+
+```text
+outbox
+job queue
+database transaction/status row
+idempotency key
+Application Workflow
+```
+
+### 5. Callback effects are outside Phronomy's exactly-once guarantee
+
+Phronomy does not claim exactly-once semantics for:
+
+```text
+email
+webhook
+external database mutation
+message-broker publication
+arbitrary Application callback side effect
+```
+
+Applications own idempotency/deduplication appropriate to those systems.
+
+### 6. Caller Tasks remain Runtime-only
+
+A caller-facing Task can observe same-process success/failure and callback policy.
+It is not rehydrated after process loss.
+
+No durable state exists solely to recreate or settle a lost caller Task.
+
+### 7. Handoff routing is independent from callback delivery
+
+A Source Agent may terminalize as `:handed_off`.
+
+Durable Handoff responsibility transfer is governed by ADR-030 and does not depend
+on whether a local `:handoff` Application event was observed.
+
+Losing the local callback must not lose the Target routing state.
+
+### 8. Read-only outcome access and execution discovery
+
+Applications must be able to query an execution's owner, status and durable
+result/error by exact semantic execution ID without invoking/recovering it or
+redelivering callbacks. If admission committed before the caller received its ID,
+a public discovery path from the known Agent/Team identity must cover retained
+terminal as well as active executions. Candidate discovery does not guarantee
+request deduplication or unambiguous correlation. Existing retention applies.
+
+Reuse existing APIs where they meet this contract; map missing capabilities only
+after baseline inspection. Do not invent a framework outbox or request registry.
+See [RC-01](RECOVERY_CONTRACT_CLARIFICATIONS.md#1-rc-01--確定結果の参照と実行の発見).
+
+### 9. Unknown commit outcome and cancellation
+
+A failed read is not proof of absence. A lost commit acknowledgement requires
+readback with the same operation/reserved identities before new semantic work.
+Unresolved storage uncertainty follows existing Persistence error/retry rules,
+not Application factual invention (RC-02).
+
+Stopping observation, losing a caller, or shutting down Runtime does not by
+itself request semantic cancellation. Explicit semantic cancellation uses the
+existing Agent contract and preserves exact child identities and confirmed
+outcomes through its existing terminal/settlement boundaries (RC-04).
+
+The guarantee is reuse of confirmed durable outcomes and recovery of unfinished
+executions under the same semantic identity. Unknown external Provider/Tool
+effects follow existing Agent Recovery; external effects are not exactly once
+(RC-05).
+
+## Persistence / Runtime boundary
+
+The ordering remains:
+
+```text
+EventLoop-owned live authority
+  -> OffloadPool durable semantic transaction
+  -> Persistence commit
+  -> EventLoop apply / release admission
+  -> optional current-process Application callback
+  -> current-process Task settlement
+```
+
+No callback acknowledgement transaction follows.
+
+## Relationship to existing ADRs
+
+- Clarifies ADR-018 X0 boundaries: Application effects remain external.
+- Preserves ADR-023: Runtime routing identities remain non-durable.
+- Preserves ADR-024: EventLoop remains live-state writer.
+- Preserves ADR-025: AgentExecution lifetime ends at semantic terminal commit.
+- Does not alter ADR-028 preparation replay rules.
+- ADR-030/031 may persist additional **semantic coordination facts**, but not
+  restart-spanning Application callback delivery.
+
+## Required invariants
+
+1. Known terminal semantic work is never replayed merely because a callback may
+   have been lost.
+2. No terminal callback-delivery metadata/index/ACK protocol is required.
+3. Callbacks, Tasks and external side effects remain Runtime/Application concerns.
+4. Agent terminal statuses remain semantic states, not notification states.
+5. Phronomy does not claim exactly-once external effects.
+6. Read-only result access and retained execution discovery do not trigger work.
+7. Read/commit uncertainty and observation loss do not authorize replacement work.
+
+## Non-goals
+
+This ADR does not:
+
+- provide restart-spanning callback delivery;
+- provide arbitrary callback-side-effect deduplication;
+- recover caller Tasks;
+- introduce an Application outbox inside Persistence;
+- change Handoff routing semantics;
+- change cross-process ownership guarantees.

data/docs/decisions/030-agent-handoff-domain-and-durable-responsibility.md

@@ -0,0 +1,235 @@
+# ADR-030: Agent Handoff Domain and Durable Responsibility
+
+## Status
+
+Accepted. V2 revision 2, 2026-09-06.
+
+User approval covers the V2 boundary and the five recovery-contract clarifications.
+Acceptance is design authority; it is not a claim of repository integration or test success.
+
+## Date
+
+2026-09-06
+
+## Partially supersedes
+
+`016-semantic-multi-agent-handoff` for:
+
+- namespace/domain placement of Handoff;
+- Runtime-local-only active responsibility;
+- allowance for independent Source/Target Persistence domains in the durable
+  Handoff path; and
+- the public Handoff Runner namespace.
+
+ADR-016 remains the historical rationale and remains authoritative for the
+Source-to-Target semantic transfer model, HandoffPolicy category semantics,
+Context dependency grouping, immutable transferred Context, Target ContextPolicy
+ownership, and provenance rules except where this ADR explicitly changes them.
+
+## Context
+
+ADR-016 intentionally made active Handoff responsibility Runtime-local. Process
+reset therefore restarted responsibility at the main Agent.
+
+That behavior is insufficient for a framework-owned Handoff abstraction once the
+Source execution has durably committed `:handed_off`: process loss must not force
+Source semantic work to run again merely to rediscover the Target.
+
+This is a framework semantic-routing concern, not an Application callback concern.
+ADR-029 therefore does not make Handoff durability depend on restart-spanning
+notification delivery.
+
+## Decision
+
+### 1. Handoff moves to the Agent domain
+
+Public types move as a clean break:
+
+```text
+Phronomy::MultiAgent::Handoff
+  -> Phronomy::Agent::Handoff
+
+Phronomy::MultiAgent::HandoffPolicy
+  -> Phronomy::Agent::HandoffPolicy
+
+Phronomy::MultiAgent::Runner
+  -> Phronomy::Agent::HandoffRunner
+```
+
+Handoff-specific private types move under the Agent Handoff implementation
+boundary.
+
+No compatibility alias is required by this ADR.
+
+### 2. `main_agent.agent_id` is the durable routing anchor
+
+No generic coordination/thread/session identity is introduced.
+
+```text
+main_agent.agent_id
+  = durable Handoff routing anchor
+```
+
+One durable HandoffState is keyed by that identity.
+
+### 3. Persistence adds `handoff_states`
+
+The durable state contains semantic routing facts only:
+
+```text
+main_agent_id
+handoff_revision
+active_agent_id
+active_handoff_context_ref
+phase
+pending_source_execution_id
+pending_target_execution_id
+created_at
+updated_at
+metadata
+```
+
+It never stores Agent instances, Agent classes, Procs, HandoffPolicy objects,
+Tasks, FSMSessions or EventLoop routing state.
+
+### 4. One durable Handoff graph uses one Persistence domain
+
+The main/source/target Agents and HandoffState must use the same
+`Phronomy::Persistence` transaction domain.
+
+A graph requiring a distributed transaction across independent Persistence
+domains is rejected before semantic work.
+
+### 5. Source terminalization and responsibility transfer are one semantic transaction
+
+When a Source chooses a valid Handoff, Phronomy:
+
+1. resolves the current finalized Source Manifest;
+2. applies HandoffPolicy projection;
+3. materializes immutable HandoffContext;
+4. reserves the exact Target `execution_id`;
+5. commits Source `:handed_off` and HandoffState transfer atomically.
+
+The transaction records at least:
+
+```text
+Source AgentExecution -> :handed_off
+Source AgentRoot terminal/idle revision
+Source Journal audit facts
+HandoffContext content reference
+HandoffState.active_agent_id -> Target
+HandoffState.active_handoff_context_ref -> transferred Context
+HandoffState.phase -> target_pending
+HandoffState.pending_source_execution_id -> Source execution_id
+HandoffState.pending_target_execution_id -> reserved Target execution_id
+```
+
+There is no terminal callback-delivery descriptor in this transaction.
+
+### 6. Target execution identity is reserved before Target semantic work
+
+Recovery uses the exact reserved Target `execution_id`:
+
+```text
+authoritatively absent after a successful read
+  -> establish that exact reserved execution only after admission is confirmed
+
+nonterminal
+  -> recover that exact execution
+
+terminal
+  -> consume its durable outcome; never create a replacement execution
+```
+
+The execution ID is semantic Agent identity, not Runtime FSMSession identity.
+
+### 7. Active responsibility survives later turns and process loss
+
+When a Handoff turn ends normally at Target B, `active_agent_id` remains B.
+
+The next HandoffRunner turn starts at B.
+
+After process reset, compatible HandoffRunner wiring loads the same HandoffState
+and again starts/resumes from B rather than reverting to the original main Agent.
+
+### 8. Multi-hop updates the same HandoffState
+
+A -> B -> C updates the original main-Agent-anchored HandoffState.
+
+No nested generic coordination IDs are created.
+
+### 9. Runtime graph/Policy wiring is Application code
+
+HandoffRunner requires the current Application-supplied Handoff graph and Policies
+to reconstruct Runtime behavior.
+
+Those Ruby objects are never persisted.
+
+If required wiring is absent or incompatible, recovery fails closed instead of:
+
+- reverting to main Agent;
+- inventing a graph;
+- blindly replaying Source work.
+
+### 10. HandoffContext is durably materializable but not adopted automatically
+
+The canonical immutable HandoffContext value is stored in ContentStore and
+referenced by HandoffState/execution metadata.
+
+Transferred material remains request-scoped Target Context unless Target
+execution creates its own canonical Journal/Knowledge facts.
+
+### 11. Local Application events are Runtime-only
+
+A Source may emit a same-process `:handoff` event.
+
+That event is not durable routing authority and is not redelivered after restart.
+
+Handoff coordination correctness depends only on durable semantic routing facts.
+
+### 12. Recovery evidence, compatibility and cancellation
+
+Apply [RC-01 through RC-05](../design/durable-semantic-coordination/RECOVERY_CONTRACT_CLARIFICATIONS.md).
+Read failures/unknown commit outcomes must not be treated as Target absence.
+Readback reconciles the same reserved execution and transfer facts; admission
+races use existing atomic admission/CAS, never a replacement Target ID.
+
+Before continuation, check main/active/Target identities, current required graph
+connections, declared definition compatibility and the same Persistence instance.
+Absent an existing explicit migration/compatibility contract, definition id/version
+must match. Current wiring never reprojects committed HandoffContext. Proc/code
+hashing and automatic semantic code-compatibility detection are not introduced.
+
+Result reads follow the specified run's recorded Target, not an unrelated later
+turn's latest active result. Continuation wiring is not required merely to read
+stored status/canonical results.
+
+After transfer commit the Source remains handed_off. Observation loss or cancel
+must not roll active responsibility back to main. An explicit cancellation of the
+current turn is routed to that turn's exact reserved Target under existing Agent
+cancellation rules; it does not cancel unrelated/later executions. Preserve the
+facts needed to reconcile cancellation/admission races and process loss.
+
+## Required invariants
+
+1. Handoff is an Agent-domain capability.
+2. `main_agent.agent_id` is the durable routing anchor.
+3. Active responsibility survives process loss.
+4. Source `:handed_off` and durable responsibility transfer cannot diverge.
+5. Target semantic work never starts without a recoverable reserved execution ID.
+6. A committed Source Handoff is never blindly replayed.
+7. Recovery reconstructs fresh Runtime objects.
+8. Graph/Policy Ruby objects are supplied by Application code, not persisted.
+9. All durable graph participants share one Persistence domain.
+10. Application callback delivery is not part of Handoff durability.
+
+## Non-goals
+
+This ADR does not:
+
+- make arbitrary external effects exactly once;
+- provide restart-spanning local Handoff callback delivery;
+- add distributed transactions across Persistence domains;
+- add generic coordination/session/thread identity;
+- persist HandoffPolicy/Application code;
+- merge Source and Target Agent state.