phronomy 0.22.0 → 0.23.0

data/docs/decisions/012-canonical-execution-log-and-context-policy.md

@@ -2,68 +2,150 @@
 
 ## Status
 
-Accepted for the stateful Agent refactor. This decision supersedes the parts of ADR-011 that made the legacy `build_context`/Assembler path the long-term LLM-input authority.
+Accepted and implemented by the stateful Agent / ACS-04 Context Policy refactor.
+This decision supersedes the legacy `build_context` / Assembler authority model and
+the intermediate descriptor/registry-based Context Policy design.
 
 ## Decision
 
-Phronomy records the logical execution facts it observes as an append-only Canonical Complete Execution Log. Context selection, pruning and compaction do not rewrite or delete those raw facts. They decide only which representation is materialized into one LLM Call Manifest.
+Phronomy records the logical execution facts it observes as an append-only Canonical
+Complete Execution Log. Context selection, pruning, ordering, Tool subset selection,
+and compaction do not rewrite or delete those raw facts. They decide only what is
+materialized into one LLM Call Manifest.
 
 Journal and Manifest are separate authorities:
 
-- **Journal** is the authoritative record of logical execution facts that Phronomy observed.
-- **Manifest** is the authoritative record of the logical input fixed for one particular LLM Call.
-- Runtime Projection is derived from the Manifest and must not add semantic content that the Manifest did not select.
+- **Journal** is the authoritative record of logical execution facts observed by Phronomy.
+- **Manifest** is the authoritative record of the logical input fixed for one LLM Call.
+- Runtime Projection is derived from the Manifest and must not add semantic content that
+  the Manifest did not select.
 
-The persistent identity axes have narrow responsibilities:
+`agent_id`, `execution_id`, `llm_call_id`, `tool_call_id`, and Journal `sequence` retain
+their narrow identity/provenance meanings. They are not generic Context-selection
+boundaries. Tool protocol dependencies are represented as indivisible conversation
+groups rather than by grouping all records from one execution.
 
-- `agent_id` identifies the owning Agent.
-- `execution_id` identifies one AgentExecution. It is provenance, not a Context-selection atom.
-- `llm_call_id` identifies one runtime Provider LLM Call. It is allocated before transport starts and correlates that call's outcome. It is provenance, not a semantic-compaction boundary.
-- `tool_call_id` links an assistant message's Tool Call with the corresponding Tool execution/message.
-- Journal `sequence` is canonical chronology.
+## Context Policy semantic boundary
 
-No `message_group_id`, import-only source provenance ID, or synthetic imported `execution_id` / `llm_call_id` is introduced.
+One LLM Call is prepared through:
 
-### Message preservation
+```text
+Context sources
+  -> ContextPolicyInput
+  -> ContextPolicy
+  -> ContextPlan
+  -> ContextAssembler validation/canonicalization
+  -> LLMInputManifest
+```
+
+`ContextPolicyInput` exposes four top-level semantic categories:
 
-A logical message that Phronomy receives is not flattened merely to make later Context assembly convenient.
+```text
+instruction
+knowledge
+tools
+conversation
+```
 
-- A Provider assistant response is captured as one complete assistant message containing its observable `content` and all Tool Calls.
-- An imported assistant message is journaled as one assistant message with the structure supplied by the Import contract.
-- A Tool value returned by Phronomy Tool execution is an execution fact (`tool_result`).
-- The Tool-role message actually appended to the LLM conversation is a separate logical fact (`tool_message`).
-- Imported Tool-role messages are journaled directly as `tool_message` records; Phronomy does not invent a separate raw Tool execution result for an execution it did not observe.
+The values are immutable Phronomy-defined typed items. Conversation is exposed as an
+ordered array of indivisible groups. An ordinary message is a singleton group; an
+assistant Tool Call and its corresponding Tool-role message(s) form one atomic group.
 
-The Journal therefore does not need to infer or reconstruct a source message boundary that Phronomy already observed. Context Policy can inspect Tool Call IDs contained in an assistant message and form protocol-safe selection units with the corresponding Tool messages.
+`ContextPlan` uses the same four categories. Items omitted from the Plan are omitted from
+that LLM Call. Plan ordering expresses Policy ordering within the Framework-owned
+structural layout. ContextAssembler validates required material, group integrity, Tool
+configuration, and the final token budget before it stores the Manifest.
 
-### Import boundary
+A custom ContextPolicy is ordinary Ruby strategy code. It may select, omit, order,
+compact, retrieve, or otherwise compute its Plan. Phronomy does not expose a public
+Pipeline/Selector/UnitBuilder composition DSL as the Context Policy SPI.
 
-The application supplying imported history is responsible for satisfying Phronomy's Import contract. Phronomy interprets valid input according to that contract and rejects only data that is invalid under the contract, such as unsupported roles, missing Tool Call IDs, orphan/duplicate Tool results, unresolved Tool calls, or malformed message structure.
+## Agent binding and lifetime
 
-External resource acquisition is not a `ContextImporter` responsibility. Files, URLs and similar resources are obtained and interpreted by the Application or Tool that owns that capability; Phronomy journals the logical content/results it actually receives. `ContextImporter` therefore does not introduce RubyLLM-specific attachment handling or an attachment-specific reject path.
+A ContextPolicy is Application code/runtime wiring, not durable Agent state.
 
-Phronomy must not reject an otherwise valid input merely because an internal flattened representation would lose information. In particular, two separately supplied assistant messages remain two separate Journal messages.
+An Application binds a **ContextPolicy instance** on the Agent class:
 
-### Manifest boundary
+```ruby
+SEARCH_POLICY = SearchContextPolicy.new(vector_store: VECTOR_STORE)
 
-The Manifest fixes what one LLM Call will actually receive after Context Policy selection. A historical raw Tool return value and the Tool message produced from it are not interchangeable: the raw result belongs to the execution log, while the message selected for an LLM Call belongs to the Manifest input path.
+class ResearchAgent < Phronomy::Agent::Base
+  context_policy SEARCH_POLICY
+end
+```
 
-One Provider response is captured as a Phronomy-owned `ProviderCallOutcome` before Agent-owned Tool execution starts. The canonical assistant-message record is produced from that outcome, not from Application callback delivery.
+If no Policy is bound, the built-in Default instance is used. There is no Policy override
+on Agent instance creation/loading or on `invoke` / `stream` calls. A Policy instance may
+be shared by multiple Agent classes; concurrency safety of a shared Policy and its runtime
+dependencies is the Application's responsibility.
 
-Context selection is expressed separately through `ContextCandidate`, dependency-aware `ContextSelectionUnit`, `ContextRequest`, `ContextPolicy`, validated `ContextPlan`, and final token-budget validation. An assistant message containing Tool Calls and the corresponding Tool messages form an atomic protocol unit. Ordinary messages in the same `execution_id` remain independently selectable.
+Phronomy does not persist or reconstruct a Policy instance and defines no
+`ContextPolicyDescriptor`, Policy registry, serialized Policy config, or Policy version
+contract. Recovery hydrates finalized `LLMInputManifest` values directly. Future Context
+preparation after Recovery uses the ContextPolicy supplied by the currently loaded
+Application code.
 
-## RubyLLM boundary
+## Policy-generated material
 
-Agent-owned Tool execution requires RubyLLM's additive callback contract introduced in RubyLLM 1.15. Phronomy therefore requires `ruby_llm >= 1.15, < 2`.
+A Policy may create new instruction, knowledge, or conversation items through the small
+protected helper/factory API on `ContextPolicy`. Such material is an ordinary current-call
+Plan item; there is no separate `DerivedContentSpec` collection.
 
-RubyLLM 1.15 adds the complete assistant message to `Chat#messages` before `before_tool_call` callbacks run. Phronomy captures the immutable Provider outcome at that boundary and raises `ToolCallIntercepted` only as an internal control transfer so approval, suspension, parallel dispatch and durable state remain Phronomy-owned.
+Phronomy assigns current-call identity, estimates tokens, freezes/canonicalizes the value,
+and stores content when needed by the finalized Manifest. The Application owns the
+semantic transformation and any internal source mapping/provenance it requires. Merely
+using a generated item in a Manifest does not promote it to a Journal fact or reusable
+future Context candidate.
+
+The ACS-04 Tool category is selection-only: a Policy may choose a subset of the effective
+Agent Tool definitions but may not invent a runtime Tool implementation from schema-only
+data.
+
+## Default Context Policy
+
+The built-in Default is deterministic and model-free:
+
+- retain effective instructions in stable order;
+- retain the effective Agent Tool configuration;
+- retain required/current conversation and choose a contiguous recent optional history;
+- choose Knowledge in stable order, skipping an oversized item and continuing with later
+  items that fit;
+- allocate variable remainder approximately 60% to conversation and 40% to Knowledge,
+  allowing unused share to be reused by the other category;
+- perform no automatic compaction, embedding search, reranking, or additional LLM Call.
+
+If required/fixed Context cannot fit, preparation fails rather than silently deleting it.
+
+## Execution / Persistence boundary
+
+ContextPolicy executes synchronously on an OffloadPool worker, never on the Runtime
+EventLoop. No Phronomy Persistence transaction spans `ContextPolicy#call`.
+
+The required preparation shape is:
+
+```text
+capture authoritative local/durable snapshot
+  -> build immutable ContextPolicyInput
+  -> ContextPolicy#call outside Persistence transaction
+  -> revalidate durable base/revision/lineage
+  -> short commit transaction
+       validate Plan
+       canonicalize selected/generated content
+       final budget validation
+       store LLMInputManifest
+       save execution state
+```
+
+A stale Policy result is rejected by the final revision/watermark precondition. Phronomy
+does not automatically retry or fall back to another Policy after Policy failure.
 
 ## Consequences
 
-- Canonical execution history is independent of the current Context budget or policy.
-- Import and runtime histories converge on the same canonical assistant/tool-message model without synthetic grouping identity.
-- Old optional working history may be excluded from a follow-up Manifest without being deleted.
-- Raw Tool results remain available as execution facts even when the corresponding Tool message is omitted from a later Manifest.
-- Tool protocol dependencies are validated independently from semantic selection policy.
-- Context Policy can later introduce deterministic derived/compacted records without replacing their raw sources.
-- Public custom Context Policy APIs, transaction-boundary restructuring, deterministic compaction, Manifest v2/tool subsets, and legacy Assembler removal remain later phases.
+- Agent/Workflow durability is independent of ContextPolicy durability.
+- The Manifest, not the historical Policy implementation, is the Recovery authority for a
+  finalized Provider input.
+- Application Context strategies remain ordinary reusable Ruby objects with ordinary DI.
+- Framework-internal protocol grouping/validation may use private helpers, but those
+  helpers are not the Application Policy API.
+- Replaced descriptor/registry/request/derived-content and selector-pipeline abstractions
+  are removed rather than retained as compatibility aliases.

data/docs/decisions/014-unified-persistence-durable-state.md

@@ -1,8 +1,15 @@
 # ADR 014: Unified Persistence for Durable State
 
-**Status**: Accepted  
-**Date**: 2026-08-14  
+**Status**: Accepted
+**Date**: 2026-08-14
 **Supersedes**: ADR-009
+**Partially superseded by**:
+- [ADR-020](020-canonical-workflow-instance-identity.md) for Workflow identity terminology only
+- [ADR-021](021-generic-agent-invocation-identity-removal.md) for `InvocationContext` generic session/correlation semantics and Agent-side generic invocation identity
+- [ADR-023](023-fsm-session-incarnation-identity-and-routing.md) for concrete FSMSession incarnation identity and Agent/Tool Runtime routing
+- [ADR-024](024-event-loop-single-writer-agent-runtime.md) for live Agent Runtime ownership, EventLoop single-writer apply, and `AgentExecutionActivation` removal
+- [ADR-025](025-process-local-agent-ownership-and-runtime-admission.md) for one-live-Agent-per-`agent_id` process ownership and same-process top-level execution admission
+- [ADR-026](026-workflow-runtime-admission-and-durable-terminal-barrier.md) for same-process Workflow admission ownership and durable terminal-barrier ordering
 
 ---
 

data/docs/decisions/016-semantic-multi-agent-handoff.md

@@ -0,0 +1,112 @@
+# ADR-016: Semantic Multi-Agent Handoff and Runtime-local Ownership
+
+## Status
+
+Accepted.
+
+## Context
+
+The previous Multi-Agent Handoff implementation encoded routing through a
+special Tool name and sentinel Tool result. That mechanism could identify a
+Target Agent, but it conflated control-plane routing with ordinary Tool execution
+and did not define sufficient Context-transfer, provenance, ownership, or
+next-turn semantics.
+
+Handoff is also semantically different from Agent-as-Tool delegation. Delegation
+returns control to the parent coordinator. Handoff transfers active
+responsibility to another Agent.
+
+ADR-012 remains authoritative for the canonical execution log and per-call
+Context Policy/Manifest pipeline. ADR-013 remains authoritative for persistent
+Journal-backed Knowledge. This ADR defines the Agent-boundary transfer semantics
+without replacing either decision.
+
+## Decision
+
+### 1. Handoff is an explicit Source-to-Target semantic edge
+
+`Phronomy::MultiAgent::Handoff` binds concrete live `source_agent` and
+`target_agent` instances plus a `HandoffPolicy` and description.
+
+Generated Tool names are private transport encoding. They are not Handoff
+identity, durable routing identity, or public semantic contract.
+
+### 2. Handoff is control-plane transfer, not an ordinary Tool effect
+
+The LLM-facing capability is intercepted before `ToolInvocation` creation. A
+valid Handoff produces a typed private `HandoffRequest` and a Source execution
+terminal outcome of `:handed_off`.
+
+No sentinel Tool result is generated. One Provider outcome may not mix Handoff
+with ordinary Tool Calls or contain multiple Handoff requests.
+
+### 3. Handoff Policy owns the Agent-boundary transfer decision
+
+The initial transfer categories are `current_request`, `history`, `knowledge`,
+and `tool_exchanges`. Application policy classifies each as `required`,
+`forbidden`, or `selectable`, with an include/exclude default for selectable
+categories.
+
+Required and forbidden decisions cannot be overridden by the Source Agent.
+Source selection is bounded to the effective Source Context represented by the
+current Manifest.
+
+### 4. Context dependency grouping is preserved across the Handoff boundary
+
+Context Policy uses typed `ContextPolicyInput` conversation groups rather than
+the removed `Selection::Unit` SPI. Assistant Tool Calls and their corresponding
+Tool-role messages are one indivisible conversation group. When the selected
+Plan is realized, the Manifest records the Framework conversation-group identity
+and classifies canonical Tool exchanges as `tool_exchanges` for Handoff.
+
+`HandoffProjection` groups current-format Manifest segments by that conversation
+group identity. It may still read legacy `selection_unit_id` metadata from a
+finalized pre-ACS-04 Manifest, but new Context assembly does not recreate
+`Selection::Unit`. `Selection::Candidate` and `Selection::Constraint` remain
+internal Context-input normalization details only.
+
+Handoff Policy answers what may cross the Agent boundary. Target Context Policy
+remains the final authority for what enters each Target LLM call. Transferred
+Handoff Context is therefore converted to selectable Target Context candidates
+before Target Context Policy runs.
+
+### 5. Handoff Context is immutable, reference-only transferred material
+
+Selected Source content is materialized into an immutable private
+`HandoffContext`; Target execution does not depend on later dereferencing mutable
+Source state. Source and Target may use different Persistence adapters.
+
+Transferred material is not automatically adopted into Target Journal or
+persistent Knowledge. Provenance records the original Agent/record/execution/LLM
+call/Tool call where available, plus a multi-hop transfer path.
+
+### 6. Active responsibility is Runtime-local
+
+One `main_agent` instance anchors one Multi-Agent coordination lifetime. The
+Runtime/EventLoop is the sole mutation authority for `active_agent` and active
+Handoff Context. The current Target remains active across user turns and across
+Runner-facade recreation while the same main Agent instance and Runtime live.
+
+Concurrent turns for the same coordination lifetime are rejected by Runtime
+admission rather than racing active-Agent transitions.
+
+This coordination state is not durably rehydrated. Runtime/process reset starts
+again at `main_agent`; historical `execution_handed_off` audit facts do not imply
+restored active-Agent ownership.
+
+### 7. Public Runner cutover is a clean break
+
+The public coordinator is `Phronomy::MultiAgent::Runner.new(main_agent:,
+handoffs:)`. `Phronomy::Agent::Runner`, `agents:`, `routes:`, and the old Agent-owned
+Handoff Tool registry are removed without compatibility aliases.
+
+## Consequences
+
+Applications gain explicit Handoff semantics, bounded multi-hop continuation,
+Context transfer with provenance, and predictable next-turn ownership.
+
+The framework has a stronger separation between control-plane operations and
+ordinary Tools, but applications migrating from the old Runner must update their
+configuration code. Active-Agent continuation remains intentionally process-local
+until a future decision defines a durable coordination identity and rehydration
+contract.

data/docs/decisions/017-design-authority-and-adr-governance.md

@@ -0,0 +1,200 @@
+# ADR-017: Design Authority and ADR Governance
+
+## Status
+
+Accepted.
+
+## Date
+
+2026-08-23
+
+## Context
+
+Phronomy contains multiple artifact classes that answer different questions:
+
+- ADRs record architecture intent;
+- source/runtime behavior records implementation reality;
+- public API documentation, `@api` classification, compatibility/contract
+  tests and runtime behavior collectively define public/extension contracts;
+- RBS represents an already-established typed contract;
+- ordinary tests provide regression evidence;
+- historical and archived design documents preserve context but may describe
+  architecture that is no longer current.
+
+Treating those artifacts as one universal precedence list creates two failure
+modes: stale design can be reimplemented because it looks authoritative, or an
+accidental implementation change can silently redefine architecture because
+it is newer.
+
+The repository also contains two historical ADR files with numeric prefix
+`011`. Numeric labels alone therefore cannot provide an unambiguous identity
+for every existing decision.
+
+## Decision
+
+### 1. Authority is question-specific
+
+Phronomy does **not** define one universal precedence order for every artifact.
+
+- Accepted and non-superseded ADRs are the normative authority for
+  architecture decisions.
+- Source code and runtime behavior are the authority for current
+  implementation reality. They do not silently amend architecture intent.
+- Public APIs and extension SPIs are composite contracts established by
+  runtime behavior, `@api` classification, formal API documentation and
+  explicit compatibility/contract tests.
+- RBS represents a contract that already exists. RBS does not create new
+  public semantics or make an internal API public.
+- Ordinary implementation tests are regression evidence. A test is an
+  architecture/contract authority only when the repository explicitly treats
+  it as an architecture guard or compatibility/contract test.
+- Historical, Archived and Superseded artifacts are non-normative for current
+  architecture.
+
+### 2. Inconsistency is explicit
+
+When normative architecture, public contract and implementation reality
+disagree, no artifact wins automatically because it is newer.
+
+The discrepancy is an architecture inconsistency. It must be recorded,
+reviewed and resolved explicitly by changing the appropriate architecture
+decision, contract, implementation, or combination of them.
+
+The required tracking fields and lifecycle are defined in
+[`docs/decisions/README.md`](README.md).
+
+### 3. ADR canonical identity is the filename basename
+
+The canonical key for an ADR is its filename basename:
+
+```text
+NNN-kebab-case-slug
+```
+
+The numeric prefix is an ordering/display field. Because the repository
+contains two legacy `011` decisions, a bare numeric label is not a globally
+unique historical key.
+
+The two legacy `011` ADRs remain in place and are distinguished by:
+
+```text
+011-build-context-as-single-llm-input-authority
+011-delegate-transport-policy-to-adapters
+```
+
+New or modified normative references to either decision must use the full
+canonical key or an explicit file link.
+
+### 4. Existing decision history is not renumbered
+
+Existing ADR files are not silently renumbered to repair historical numbering
+defects. Renumbering changes references and obscures the actual decision
+history.
+
+`016-semantic-multi-agent-handoff` is preserved under its existing key even
+though it was created before this governance rule was canonicalized.
+
+### 5. New ADR numbers are monotonic and unique
+
+A new ADR receives:
+
+```text
+max(existing numeric prefix) + 1
+```
+
+at the point the new decision is prepared for merge.
+
+New ADRs:
+
+- use a three-digit numeric prefix;
+- do not fill gaps;
+- do not reuse any existing prefix;
+- update the decision index in the same change.
+
+Concurrent unmerged ADRs that select the same number are resolved by rebasing:
+only the not-yet-historical ADR that merges later is renumbered.
+
+This decision follows the rule it establishes: `016` already existed and `017`
+was unused, so this Design Authority decision is ADR-017.
+
+### 6. Supersession preserves history
+
+A superseding decision is normally a new ADR.
+
+The superseded ADR remains in the repository with its historical rationale.
+Its status/supersession relation and the decision index are updated so the
+current authority is unambiguous.
+
+A materially different architecture is not retroactively edited into an old
+Accepted ADR merely to make repository text appear internally consistent.
+
+### 7. Working review registers are planning evidence
+
+Carry-forward (`CF-*`), architecture-inconsistency (`AI-*`), Architecture
+Change Set (ACS), and Compatibility Gate (CG) artifacts used during the
+architecture reconciliation program are not themselves normative ADRs.
+
+They identify work and evidence. Their output becomes repository authority
+only through the appropriate Accepted ADR, public contract, source/runtime
+implementation, architecture guard, or current explanatory documentation.
+
+Unresolved items that survive a working review must be represented by an
+active repository work item or an explicit deferred dependency; they must not
+disappear merely because a working document is retired.
+
+### 8. Explanatory architecture documentation is separate
+
+Current explanatory architecture documentation will be organized under a
+dedicated canonical entry as part of the documentation-lifecycle migration.
+
+Those documents explain the reconciled architecture. They do not replace the
+ADR system as the authority for architecture decisions.
+
+## Repository contract
+
+Contributors making architecture-sensitive changes must:
+
+1. consult [`docs/decisions/README.md`](README.md);
+2. identify relevant Accepted/non-superseded ADRs by canonical key;
+3. distinguish architecture intent from implementation reality and public
+   contract evidence;
+4. record unresolved inconsistencies rather than choosing a winner by recency;
+5. update ADR status/index relationships when accepting, amending or
+   superseding a decision;
+6. preserve the public-API composite contract and RBS non-authority rule.
+
+`CONTRIBUTING.md` carries the developer-facing form of this contract.
+
+## Consequences
+
+### Positive
+
+- Current architecture intent has an explicit repository authority model.
+- Duplicate historical ADR number `011` no longer makes decision references
+  ambiguous.
+- New ADR identifiers cannot silently collide.
+- Source drift does not silently become architecture.
+- Stale ADR text does not silently override implementation reality; conflicts
+  are visible and reviewable.
+- RBS and ordinary tests cannot accidentally become architecture-definition
+  mechanisms.
+- Historical decision rationale is preserved.
+
+### Trade-offs
+
+- Architecture-sensitive changes must update an index and sometimes track an
+  explicit inconsistency.
+- Existing bare references to the legacy numeric label `ADR-011` can remain
+  ambiguous in historical material until that material is otherwise touched.
+- Accepted ADRs with known inconsistencies remain visible as normative intent
+  until an explicit successor is accepted; the inconsistency process is
+  therefore required rather than optional.
+
+## Non-goals
+
+This ADR does not:
+
+- reorganize `spec/design/` into the final current/archive documentation tree;
+- resolve domain-specific architecture inconsistencies;
+- supersede ADR-010, ADR-012, ADR-013, ADR-014, ADR-015 or ADR-016;
+- change Runtime behavior, public APIs, persistence formats, or durable data.