phronomy 0.22.0 → 0.24.0

data/docs/decisions/031-durable-multi-agent-coordination.md

@@ -0,0 +1,301 @@
+# ADR-031: Durable Multi-Agent Semantic Coordination
+
+## Status
+
+Accepted. V2 revision 2, 2026-09-06.
+
+User approval covers the V2 boundary and the five recovery-contract clarifications.
+Acceptance is design authority; it is not a claim of repository integration or test success.
+
+## Date
+
+2026-09-06
+
+## Context
+
+Phronomy exposes:
+
+- `MultiAgent::Orchestrator < Agent::Base`, with child Agents/Tools and fan-out
+  helpers; and
+- `MultiAgent::TeamCoordinator`, with coordinator-generated tasks and a worker pool.
+
+Individual AgentExecutions are durable, but process-local coordination state is
+not. Losing child identities, task assignments or confirmed worker outcomes can
+cause Phronomy to repeat semantic work it owns.
+
+The previous proposal also attempted to make standalone fan-out convenience APIs,
+arbitrary Team aggregation callbacks and terminal Application callbacks durable.
+Responsibility review found those parts too broad.
+
+This ADR therefore defines durability at the **framework-owned semantic
+coordination boundary**, not at every callback/convenience API boundary.
+
+## Decision
+
+### 1. MultiAgent durability means semantic progress recovery
+
+For a durable MultiAgent operation:
+
+```text
+process/runtime loss
+  -> reconstruct from durable coordination facts
+  -> reuse confirmed child/worker outcomes
+  -> recover exact unfinished executions
+  -> admit absent reserved executions only after authoritative read/admission
+  -> resolve unknown external effects through existing Agent Recovery
+```
+
+FSMSession, Task, callback and Runtime queues remain disposable.
+
+### 2. Orchestrator durability is rooted in an existing Orchestrator AgentExecution
+
+`MultiAgent::Orchestrator < Agent::Base` already has canonical Agent identity.
+
+When child coordination occurs **inside an existing Orchestrator AgentExecution**,
+that AgentExecution references an immutable coordination snapshot containing, as
+needed:
+
+```text
+coordination kind / phase
+child slot
+child Agent definition identity/version
+reserved child agent_id
+reserved child execution_id
+restart-required input/config refs
+child semantic state
+result_ref / error_ref
+on_error / max_concurrency semantic options
+```
+
+No separate Orchestrator repository or generic multi-agent execution ID is added.
+
+### 3. Existing fan-out Runtime FSM remains reconstructable machinery
+
+`FanOutInvocation` / `FSMSession` remain Runtime projections.
+
+Recovery builds a fresh runtime invocation/session from the durable coordination
+snapshot and fresh Runtime identities.
+
+### 4. Framework-owned child identities are reserved before semantic work
+
+Before a durable Orchestrator child begins:
+
+```text
+reserve child agent_id
+reserve child execution_id
+persist parent child slot + restart-required refs
+then create/load child and start exact reserved execution
+```
+
+Recovery distinguishes absent/nonterminal/terminal exact child state and never
+creates a replacement simply because Runtime callbacks were lost.
+
+### 5. Durable child definitions must be reconstructable from stable wiring
+
+For the durable path, a child Agent class must be resolvable from stable current
+Orchestrator/Application wiring, initially the concrete Orchestrator class's
+registered subagents.
+
+Invocation-only anonymous/arbitrary class handles are not persisted.
+
+This restriction applies only to APIs/paths that claim restart durability.
+
+### 6. Standalone `dispatch_parallel*` / `fan_out*` remain Runtime-only convenience APIs
+
+A direct Application call to:
+
+```ruby
+orchestrator.dispatch_parallel(...)
+orchestrator.fan_out(...)
+```
+
+outside a live parent Orchestrator AgentExecution does **not** create a synthetic
+AgentExecution solely for durability.
+
+It keeps current-process semantics and may continue accepting Runtime-only Agent
+class wiring.
+
+If durable fan-out is required, the Application must place the work under:
+
+- an Orchestrator AgentExecution with stable child wiring; or
+- an Application Workflow when the operation is application-process
+  orchestration.
+
+This avoids inventing durable parent identity for a convenience call.
+
+### 7. Durable framework-owned Orchestrator children share the parent Persistence domain
+
+A durable parent/child coordination path uses the same Persistence domain.
+
+External remote effects reached through Tools/Application integration remain X0
+external effects.
+
+### 8. TeamCoordinator is a purpose-specific durable semantic entity
+
+TeamCoordinator remains under `MultiAgent`, not `Agent::Base`.
+
+It gains:
+
+```text
+team_id
+team_execution_id
+team_definition id/version
+TeamRoot
+TeamExecution
+```
+
+This is retained because Phronomy itself owns the Team queue/assignment/worker
+abstraction.
+
+### 9. Team task queue and worker assignment are durable facts
+
+TeamExecution durably records:
+
+```text
+canonical tasks
+task-generation finalized state
+worker slots / stable worker agent_id
+task -> worker assignment
+reserved worker execution_id
+worker terminal result/error
+final Team result/error
+```
+
+Before worker semantic execution starts, its assignment and reserved exact
+execution identity are committed.
+
+### 10. Worker identity is stable within one TeamExecution
+
+A worker slot reuses its logical Agent identity for that TeamExecution so worker
+context/transcript semantics remain coherent across assigned tasks.
+
+This ADR does not require worker identity/history to survive into another
+TeamExecution.
+
+### 11. Team scheduler is a replay-safe decision function
+
+Application `schedule` may run again while no assignment has been durably
+committed.
+
+Contract:
+
+- it must not perform externally observable one-shot effects;
+- it selects from the supplied available worker projection;
+- repeated execution before assignment commit is allowed;
+- after assignment commit, recovery uses the durable assignment and does not
+  rerun scheduling for that task.
+
+No scheduler result needs a separate unknown-outcome protocol.
+
+### 12. Team aggregation is pure/replay-safe result computation
+
+Application `aggregate` receives canonical durable assignment results.
+
+Contract:
+
+- no externally observable one-shot side effects;
+- repeated invocation with the same canonical assignments must be semantically
+  equivalent;
+- if process loss occurs before aggregate result/error is durably committed,
+  Phronomy may invoke `aggregate` again;
+- on normal return/raise, Phronomy durably records the final result/error before
+  Team terminalization.
+
+There is no `aggregation=started -> outcome unknown -> manual resolve` protocol.
+
+Applications perform external post-Team effects after obtaining/reconciling the
+Team result.
+
+### 13. Team progress and terminal callbacks are Runtime-only
+
+Streaming task-completion callbacks and final Application notifications are
+current-process observations.
+
+Canonical task/worker/Team semantic outcomes are durable; callback delivery is
+not.
+
+No Team delivery-pending index or restart redelivery obligation is introduced.
+
+### 14. Workflow remains the Application process-orchestration domain
+
+Public domain ownership remains:
+
+```text
+Agent
+  Agent semantic execution + Agent Handoff
+
+MultiAgent
+  durable framework-owned Orchestrator child coordination
+  durable Team queue/assignment/worker coordination
+  Runtime-only convenience fan-out outside a durable parent
+
+Workflow
+  explicit Application-defined durable process/state-machine orchestration
+  durable ordering of Application-owned steps/effects when modeled by the app
+```
+
+### 15. Recovery contract clarifications
+
+The normative [RC-01 through RC-05 contracts](../design/durable-semantic-coordination/RECOVERY_CONTRACT_CLARIFICATIONS.md)
+apply to existing Orchestrator executions and Team executions:
+
+- expose read-only status/result access and retained execution discovery;
+- distinguish authoritative absence from read failure/unknown commit outcome;
+- reconcile writes with the same reserved identities using existing Persistence
+  protocols, including Team-owned operation facts;
+- verify declared definition id/version, registered slot wiring and reserved
+  owner/Agent/execution identities before continuation;
+- preserve committed inputs/config, assignments and outcomes instead of
+  recalculating them with changed current code;
+- distinguish observation loss/shutdown from semantic cancellation;
+- stop new dispatch on accepted parent-run cancellation, reconcile already
+  admitted children using existing Agent cancellation and terminal barriers,
+  retain terminal outcomes and exact identities needed after restart;
+- restrict cancellation to the current run's owned children, never unrelated
+  executions or later runs.
+
+Stable version declarations do not prove Ruby code equivalence. Application code
+must maintain declared compatibility; no Proc hash or generic registry is added.
+Uncommitted replay-safe schedule/aggregate computation remains replayable.
+No dedicated cancellation execution engine or callback recovery service is added.
+
+## Persistence changes
+
+The durable root surface expands to include:
+
+```text
+handoff_states
+teams
+team_executions
+```
+
+Orchestrator continues to use Agent repositories.
+
+No execution/team terminal-delivery pending index is added.
+
+## Required invariants
+
+1. Durable coordination is claimed only where a durable semantic root exists.
+2. Confirmed child/worker terminal work is never blindly rerun.
+3. Child/worker semantic work begins only after exact recoverable identity is
+   durably known.
+4. Orchestrator durable child coordination uses its existing AgentExecution.
+5. Standalone convenience fan-out is explicitly Runtime-only.
+6. Team has stable identity/execution and durable tasks/assignments/worker results.
+7. Scheduler and aggregator are replay-safe Application functions, not arbitrary
+   side-effect transaction boundaries.
+8. Callback/Task/FSMSession/Proc objects are never persisted.
+9. Workflow remains the appropriate domain for Application-defined durable process
+   orchestration.
+
+## Non-goals
+
+This ADR does not:
+
+- make standalone convenience fan-out restart-durable;
+- make external Provider/Tool/Application effects exactly once;
+- persist scheduler/aggregator Procs or callbacks;
+- provide restart-spanning Team callback delivery;
+- require worker history across TeamExecutions;
+- add a generic global Agent class registry;
+- turn MultiAgent into a public Workflow alias.

data/docs/decisions/README.md

@@ -0,0 +1,176 @@
+# Architecture Decision Records
+
+This directory contains Phronomy Architecture Decision Records (ADRs).
+
+The repository-wide architecture authority model is defined by
+[`017-design-authority-and-adr-governance`](017-design-authority-and-adr-governance.md).
+This index is the canonical navigation surface for ADR identity, status, and
+supersession relationships.
+
+## Canonical ADR identity
+
+The **canonical decision key** is the ADR filename basename, for example:
+
+```text
+012-canonical-execution-log-and-context-policy
+```
+
+The three-digit numeric prefix is an ordering/display number. It is not, by
+itself, a globally unique historical identifier because this repository
+already contains two legacy ADRs numbered `011`.
+
+Therefore:
+
+- new or modified normative material SHOULD link to the ADR file or use its
+  full canonical decision key;
+- the bare label `ADR-011` is ambiguous and MUST NOT be used to identify one
+  of the two legacy `011` decisions;
+- existing historical text is not rewritten merely to normalize old
+  references.
+
+## Status and authority
+
+ADR status has architecture meaning:
+
+- **Accepted** — normative architecture intent unless explicitly superseded.
+- **Amended** — normative architecture intent including its recorded
+  amendments.
+- **Superseded** — retained as decision history; non-normative for the
+  superseded scope.
+- **Proposed** — not normative until accepted.
+
+An Accepted ADR can temporarily disagree with implementation reality. That is
+an **architecture inconsistency**, not permission to silently treat whichever
+artifact is newer as authoritative. The inconsistency must be tracked and
+resolved explicitly.
+
+## ADR index
+
+| Canonical decision key | Status | Normative now? | Supersession / note |
+|---|---|---:|---|
+| [`001-rubyllm-as-provider-layer`](001-rubyllm-as-provider-layer.md) | Superseded | No | Superseded by [`027-llm-adapter-provider-boundary`](027-llm-adapter-provider-boundary.md); historical RubyLLM adoption rationale retained. |
+| [`002-workflow-context-immutability`](002-workflow-context-immutability.md) | Amended | Yes | `WorkflowContext#merge` remains new-instance semantics; direct generated field writers are EventLoop-owned guarded mutation APIs. |
+| [`003-event-loop-singleton`](003-event-loop-singleton.md) | Accepted | Yes | Current until explicitly superseded/refined. |
+| [`004-invoke-timeout-is-not-cancellation`](004-invoke-timeout-is-not-cancellation.md) | Superseded | No | Superseded by [`011-delegate-transport-policy-to-adapters`](011-delegate-transport-policy-to-adapters.md). |
+| [`005-static-knowledge-class-level-cache`](005-static-knowledge-class-level-cache.md) | Superseded | No | Superseded by [`013-journal-backed-knowledge-as-context-candidates`](013-journal-backed-knowledge-as-context-candidates.md). |
+| [`006-no-built-in-guardrails`](006-no-built-in-guardrails.md) | Superseded | No | Superseded by [`019-filter-contract-and-security-boundaries`](019-filter-contract-and-security-boundaries.md); historical minimal-built-in/Guardrail rationale retained. |
+| [`007-mcp-is-beta-stability`](007-mcp-is-beta-stability.md) | Accepted | Yes | Current. |
+| [`008-orchestrator-uses-os-threads`](008-orchestrator-uses-os-threads.md) | Superseded | No | Superseded by [`010-cooperative-first-concurrency`](010-cooperative-first-concurrency.md). |
+| [`009-state-store-abstraction`](009-state-store-abstraction.md) | Superseded | No | Superseded by [`014-unified-persistence-durable-state`](014-unified-persistence-durable-state.md). |
+| [`010-cooperative-first-concurrency`](010-cooperative-first-concurrency.md) | Accepted | Yes | Current until explicitly refined/superseded. |
+| [`011-build-context-as-single-llm-input-authority`](011-build-context-as-single-llm-input-authority.md) | Superseded | No | Superseded by [`012-canonical-execution-log-and-context-policy`](012-canonical-execution-log-and-context-policy.md). |
+| [`011-delegate-transport-policy-to-adapters`](011-delegate-transport-policy-to-adapters.md) | Accepted | Yes | Legacy duplicate numeric prefix; use the full canonical key. |
+| [`012-canonical-execution-log-and-context-policy`](012-canonical-execution-log-and-context-policy.md) | Accepted | Yes | Current Journal / Manifest / Context authority. |
+| [`013-journal-backed-knowledge-as-context-candidates`](013-journal-backed-knowledge-as-context-candidates.md) | Accepted | Yes | Current persistent Knowledge authority. |
+| [`014-unified-persistence-durable-state`](014-unified-persistence-durable-state.md) | Accepted | Yes | Durable-backend and live-owner/no-reload intent remains current; live Agent execution mutation is refined by ADR-024, same-process Agent identity/admission ownership by ADR-025, and same-process Workflow admission/terminal-barrier ordering by ADR-026. Workflow identity terminology is superseded by ADR-020, generic `InvocationContext` / Agent correlation semantics by ADR-021, and concrete FSMSession/Agent-Tool routing identity by ADR-023. |
+| [`015-tool-public-facade-and-rbs-boundary`](015-tool-public-facade-and-rbs-boundary.md) | Accepted | Yes | Current Tool façade / extension-SPI / RBS boundary. |
+| [`016-semantic-multi-agent-handoff`](016-semantic-multi-agent-handoff.md) | Superseded | No | Superseded by [ADR-030](030-agent-handoff-domain-and-durable-responsibility.md), including Agent-domain ownership and exact Target recovery. |
+| [`017-design-authority-and-adr-governance`](017-design-authority-and-adr-governance.md) | Accepted | Yes | Repository-wide architecture authority and ADR governance. |
+| [`018-durability-guarantees-and-failure-model`](018-durability-guarantees-and-failure-model.md) | Accepted | Yes | Repository-wide durability/concurrency/external-effect guarantee vocabulary and F0-F4/X0 failure model. |
+| [`019-filter-contract-and-security-boundaries`](019-filter-contract-and-security-boundaries.md) | Accepted | Yes | Current Filter transform/block and bounded PromptInjectionFilter/isolation boundaries; the follow-up review adds no fourth Context Filter call site and places semantic Context trust in Application ContextPolicy. |
+| [`020-canonical-workflow-instance-identity`](020-canonical-workflow-instance-identity.md) | Accepted | Yes | Canonical logical/durable Workflow identity and CG-01 clean-break migration. |
+| [`021-generic-agent-invocation-identity-removal`](021-generic-agent-invocation-identity-removal.md) | Accepted | Yes | Removes generic Agent/InvocationContext identity and canonical Journal `correlation_id`; CG-02 is closed, with targeted legacy durable-key read compatibility and no eager rewrite. |
+| [`022-agent-execution-parent-identity-and-runtime-routing-boundary`](022-agent-execution-parent-identity-and-runtime-routing-boundary.md) | Accepted | Yes | Canonicalizes Agent-owned Tool/approval logical parent as `execution_id`; CG-03a is reconciled, ADR-023 supplies incarnation routing, and ADR-024 supplies EventLoop result/live-state authority. |
+| [`023-fsm-session-incarnation-identity-and-routing`](023-fsm-session-incarnation-identity-and-routing.md) | Accepted | Yes | FSMSession-owned incarnation identity, session-local Agent/Tool/Multi-Agent routing, and stale-target drop remain current. Its transitional Workflow identity-reservation/admission bridge is superseded by ADR-026; Agent result authority is completed by ADR-024. |
+| [`024-event-loop-single-writer-agent-runtime`](024-event-loop-single-writer-agent-runtime.md) | Accepted | Yes | EventLoop is the single writer of Phronomy-managed live Agent execution state; removes Activation/ActivationRegistry and defines operation-specific Offload result application with current FSM + semantic-ID authority. |
+| [`025-process-local-agent-ownership-and-runtime-admission`](025-process-local-agent-ownership-and-runtime-admission.md) | Accepted | Yes | One mutable live Agent owner per `agent_id` per Runtime; EventLoop is the primary same-process top-level admission authority while Persistence admission remains durable defense. |
+| [`026-workflow-runtime-admission-and-durable-terminal-barrier`](026-workflow-runtime-admission-and-durable-terminal-barrier.md) | Accepted | Yes | EventLoop-owned opaque Workflow admission owner, admission-before-hydration ordering, and FSMSession-integrated durable terminal save barrier with fail-closed uncertain outcomes. |
+| [`027-llm-adapter-provider-boundary`](027-llm-adapter-provider-boundary.md) | Accepted | Yes | Phronomy-owned Provider-call extension boundary; RubyLLM is the default adapter/integration while current input materialization remains RubyLLM-specific. |
+| [`028-preparing-recovery-replay-contract`](028-preparing-recovery-replay-contract.md) | Accepted | Yes | Adds replay-safe same-`execution_id` recovery for durably admitted Agent `:preparing` executions when replayability is durably established; unsupported Runtime-only dependencies fail closed. |
+| [`029-semantic-completion-and-application-effect-boundary`](029-semantic-completion-and-application-effect-boundary.md) | Accepted | Yes | V2 revision 2; framework semantic coordination and Application effect boundary. |
+| [`030-agent-handoff-domain-and-durable-responsibility`](030-agent-handoff-domain-and-durable-responsibility.md) | Accepted | Yes | V2 revision 2; framework semantic coordination and Application effect boundary. |
+| [`031-durable-multi-agent-coordination`](031-durable-multi-agent-coordination.md) | Accepted | Yes | V2 revision 2; framework semantic coordination and Application effect boundary. |
+
+## Legacy duplicate `011`
+
+The repository intentionally preserves both legacy files:
+
+```text
+011-build-context-as-single-llm-input-authority
+011-delegate-transport-policy-to-adapters
+```
+
+They are different decisions. The first is superseded by ADR-012; the second
+remains Accepted.
+
+They MUST NOT be silently renumbered. Their filenames are stable historical
+decision keys. New decisions may not introduce another duplicate numeric
+prefix.
+
+## Allocating a new ADR identifier
+
+For a new ADR:
+
+1. inspect all `NNN-*.md` files in this directory;
+2. allocate `max(existing numeric prefix) + 1`;
+3. format the number with three decimal digits;
+4. never fill an old gap and never reuse an existing prefix;
+5. if two unmerged changes select the same number, the change merged later
+   rebases and renumbers **only its new, not-yet-historical ADR**;
+6. add the ADR to this index in the same change.
+
+Existing ADRs are not renumbered merely because a numbering defect is later
+discovered. `016-semantic-multi-agent-handoff` is therefore preserved as-is.
+Under this rule the Design Authority decision is ADR-017.
+
+## Superseding or amending a decision
+
+Do not rewrite historical rationale into a fictional current history.
+
+When superseding an ADR:
+
+- add a new decision describing the new architecture;
+- update the old ADR's status/supersession note only as needed;
+- preserve the old rationale;
+- update this index in the same change;
+- use canonical decision keys/links where a numeric label would be ambiguous.
+
+Amendments are appropriate only when the original decision remains the same
+decision and the amendment can be understood without erasing the historical
+rationale. Materially different architecture should normally be a new ADR.
+
+## Architecture inconsistency process
+
+Architecture intent, public contract, and implementation reality are separate
+authority domains. If they disagree, do not resolve the conflict by recency.
+
+An unresolved architecture inconsistency must be recorded in the active
+repository work item (normally a GitHub issue or PR) with at least:
+
+```text
+Concern
+Normative authority
+Implementation / contract reality
+Conflict
+Resolution dependency
+Status: OPEN | DEFERRED | RESOLVING | RESOLVED
+Final resolution
+```
+
+Existing Workstream carry-forward (`CF-*`) and architecture-inconsistency
+(`AI-*`) registers are discovery/planning evidence, not normative architecture.
+During the current reconciliation program their unresolved contents are
+consolidated into named Architecture Change Sets (ACS) and Compatibility Gates
+(CG). A repository change that resolves such an item should identify the
+corresponding ACS/CG in its PR/commit rationale.
+
+A legacy carry-forward proposition is closed only by one of:
+
+- adoption into current source/public contract/current documentation;
+- an Accepted ADR;
+- explicit rejection/non-carry disposition; or
+- an open repository work item when the resolution is intentionally deferred.
+
+Working registers must not be copied wholesale into current architecture
+documentation as if they were normative decisions.
+
+## Relationship to explanatory architecture documentation
+
+ADRs record normative decisions. Explanatory architecture documents describe
+the current reconciled system but do not supersede ADRs merely by being newer.
+
+Current explanatory architecture starts at
+[`docs/architecture.md`](../architecture.md). Non-current design snapshots are
+segregated under `docs/archive/design/` and are non-normative. Explanatory
+architecture documents describe the reconciled current system but do not
+supersede ADRs merely by being newer.

data/docs/design/durable-semantic-coordination/CHANGELOG_V2_REVISION_2.md

@@ -0,0 +1,33 @@
+# V2 revision 2 — 改訂内容
+
+更新日: 2026-09-06 / 状態: Accepted
+
+## 合意反映
+
+| 項目 | 今回の修正 |
+|---|---|
+| RC-01 結果参照 | exact実行IDで状態・結果を読み取る契約と、ID受領前の停止に備えたownerからの候補発見を追加 |
+| RC-02 保存成否不明 | 正常な不存在、read障害、CAS競合、commit成否不明を区別。同じ予約済みidentityで照合 |
+| RC-03 wiring互換性 | 定義id/version・slot・graph・Persistence境界の最低照合と、確定済みfactの優先を明記 |
+| RC-04 cancellation | 観測終了/shutdownとsemantic cancelを区別。子の状態別処理、競合、restart後の再発見を明記 |
+| RC-05 保証表現 | 確定outcome再利用・同一identity復旧・外部成否不明は既存Recovery、に統一 |
+
+ADR-029/030/031、実装設計、責務境界レビュー、READMEへ対応する規範的記述を反映した。
+詳細契約と検証条件はRECOVERY_CONTRACT_CLARIFICATIONS.mdにまとめ、資料間で参照する。
+承認前のProposed表記をAcceptedへ更新した。新機能の再承認を要求しない。
+
+## 実装状態の訂正
+
+引き継ぎ文書の旧実装件数・静的検証成功は、前セッションの報告として残す。
+この分岐先で実装を再検証した事実にはしない。現在取得できたのは設計資料と引き継ぎ文書であり、
+旧overlay/apply.py/APPLY.sh/VERIFY.shは未取得。再開手順も実ファイルの回収から始める形に更新した。
+
+## 今回実施した確認
+
+- 対象資料の承認状態、baseline、相互参照の整合確認。
+- 旧い「未承認のため実装停止」表現と、外部効果のexactly-onceに読める保証表現の修正。
+- RC-01〜05のADR・実装設計・検証条件への対応確認。
+- ZIP構成、全文書のSHA256SUMS、ZIP/引き継ぎ文書の外部SHA256の検証。
+
+対象repositoryのコード変更・RSpec実行は行っていない。
+API名や例外名は未確認のものを作らず、baselineへの対応付けを実装前の必須作業とした。