phronomy 0.26.0 → 0.28.0

data/docs/architecture/agent-chat-and-state-ownership.md

@@ -0,0 +1,153 @@
+# Agent Chat and explicit state ownership (R09, second slice)
+
+> Current boundary: r8 unit 6 supersedes the historical Chat/SDK ownership and
+> adapter extension SPI below. Agent uses LLMAdapter values; the backend owns SDK
+> construction/interception; Tool owns canonical schema validation. See
+> `docs/architecture/r8-unit6.md` and `docs/migrations/r8-unit6.md`.
+
+
+## Purpose and clients
+
+Agent::Base is the declaration and lifecycle facade used by application Agent
+subclasses. It also supplies internal hooks to normal execution, follow-up LLM
+calls and recovery. Those callers need a Chat built from the selected or saved
+model settings and populated with an already materialized projection. They do
+not ask the Chat constructor to make Context Policy decisions again.
+
+Applications create Agents and explicitly add knowledge, clear context/history,
+reset context or close an idle Agent through Base. These operations append
+Journal records and advance an AgentRoot together. The existing live root is
+authoritative; an operation must not silently reload a newer root and retry.
+
+Refactor 42 extracted ToolBinding and documented existing declaration rules.
+Refactor 43 separates the remaining Chat construction and explicit state writes
+while keeping the Base facade and live-state publication boundary.
+
+## Ownership and placement
+
+| Owner | Responsibility |
+|---|---|
+| Base | Public declarations and lifecycle entry points, live-owner checks, selected class settings, operation-specific root changes, projection installation and publication of successful state writes. |
+| RuntimeChatBuilder | RubyLLM Chat options and setters; provider-specific instruction cache representation. |
+| StateWriter | Initial root/context/knowledge persistence and explicit idle-Agent Journal/root changes in one transaction. |
+| ContextImporter / JournalRecord / AgentRoot | Existing history validation, record meanings and immutable root values. |
+| ToolBinding / Orchestrator | Existing Tool decoration and invocation-specific subagent context. |
+| ExecutionCoordinator and its workers | Existing execution-time writes and EventLoop-side publication, unchanged here. |
+
+RuntimeChatBuilder and StateWriter live in agent/context_assembly, whose existing
+collaborators already turn context inputs into runtime or durable records.
+StateWriter is orchestration over ContextImporter, JournalRecord and AgentRoot;
+it is not a low-level lifecycle primitive or a raw storage adapter. Placing it
+in lifecycle would introduce lifecycle -> context_assembly and enlarge the
+existing directory cycle. That placement was rejected during design validation.
+No new directory, loader rule, mixin or namespace alias is needed.
+
+There are two new internal collaborator classes, plus StateWriter's private
+Update data value containing root and appended records. Neither collaborator
+holds an Agent reference or invokes private Agent methods through send. The
+mutation block passed to StateWriter is the existing operation-specific root
+proposal, called at the same point inside the transaction.
+
+## Chat construction and installation
+
+Base#build_chat selects its default model/provider/temperature/token settings
+only when model_config is falsey. An explicit empty Hash remains an explicit
+config. RuntimeChatBuilder then builds options, creates RubyLLM.chat, applies
+temperature, optionally applies max_output_tokens and returns the original
+Chat, irrespective of setter return values. Zero remains a valid setting.
+
+Base#apply_instructions keeps its signature and delegates the provider detail.
+Only a truthy cache flag with provider.to_s == "anthropic" wraps instructions
+in Anthropic::Content(cache: true). All other values pass through unchanged;
+the with_instructions return value is retained.
+
+Base#_apply_runtime_projection_to_chat remains a short orchestration hook. It
+installs system instructions, prepares and installs each Tool in order, then
+appends the original message objects to the existing Chat messages. Keeping the
+hook retains application/test overrides of apply_instructions and Orchestrator's
+prepare_tool_class(invocation:) cooperation. Moving it into a builder receiving
+the entire Agent would merely introduce callbacks back into Base.
+
+InvocationActions and EngineSessionBuilder and InvocationRestorer keep using the same hooks.
+Saved config is authoritative during recovery; current class declarations do
+not overwrite it. A failure stops installation at that operation, propagates
+the same error and does not install later Tools/messages. No new Chat cache,
+provider-independent SPI, LLM call or Tool execution path is introduced.
+
+## Initial persistence
+
+Base resolves the Agent definition before calling StateWriter#create_root.
+StateWriter constructs the initial root, then performs the existing sequence:
+
+1. Create the root in the transaction.
+2. Import/encode context records, followed by knowledge records, in order.
+3. If records exist, append them at position zero, advance both revisions to
+   one and save the root with expected_revision zero.
+4. Return the resulting root after the transaction call returns successfully.
+
+Empty initial input does not append or advance revisions. String coercion of
+knowledge, metadata normalization, imported text/JSON handling and invalid-format
+errors remain unchanged. Base's initialize_owned_state still assigns the root,
+reads the committed Journal and freezes its local record array. Hydration and
+definition compatibility checks remain unchanged in Base.
+
+## Explicit mutation and publication
+
+For add_knowledge and context/lifecycle changes, Base first checks live ownership
+and captures agent_root. StateWriter takes that root, persistence and agent ID
+explicitly. Each operation retains this transaction sequence:
+
+1. Assert durable execution idleness.
+2. Construct the content/state record and append with the captured Journal head.
+3. Build the next root; a context mutation's proposal block runs after append.
+4. Save with the captured expected Agent revision.
+5. After the transaction call returns, return Update(root, appended records).
+
+Base applies the returned Journal records before replacing @root. StateWriter
+never publishes live state and never reloads the mutable root or Journal.
+add_knowledge still returns self; clear/reset/close still return the new root.
+close! increments Agent revision while preserving context revision. Context
+changes preserve the old generation in the event and the existing revision
+fallback rule in the resulting root.
+
+Block, content, append, CAS and commit failures escape without publication.
+Failures inside a supported transaction roll back its durable writes. An error
+is not wrapped, retried or turned into a false successful return. The transaction
+method's own return value is ignored as before.
+
+This is not a stronger transaction or concurrency contract. If a backend commits
+then reports an unknown outcome, or local Journal publication fails after a
+successful commit, durable and live state can differ just as before. There is
+no new F1 reconciliation for these manual operations. If an application encloses
+an Agent mutation in an outer Persistence transaction, returning from the inner
+scope is not proof of the outer commit; this refactor adds no deferred-publication
+or outer-rollback coordination. Application-owned outer transactions need a
+separate design decision, not an implicit guarantee from this extraction.
+
+The public lifecycle checks, purge ownership protocol, admission, hydration,
+execution commits, SQL and stored formats are unchanged. Relocated private
+record-building helpers are not retained as an extension API. Retained Chat,
+Tool and root-proposal hooks keep their names and signatures.
+
+## Validation and completion rule
+
+Thirty-seven additional behavioral examples run through Base against both the
+unchanged Refactor 42 and this candidate. They cover saved/default/empty config,
+provider options, setter and hook ordering, original objects/errors, import
+formats, atomic rollback, idle checks, event/revision rules and publication.
+The existing architecture guard now checks StateWriter as well as Base for
+forbidden mutable-root/Journal reload and live-state publication paths.
+
+Full core/integration/examples, real SQLite, public API/SPI, RBS, style, isolated
+gem loading and independent package application are recorded in the release
+evidence. New private helpers are distinguished from the unchanged public
+contract and retained hook signatures. Static dependency analysis rejects new
+cycle membership rather than merely checking unchanged edge counts.
+
+Refactor 43 is applied and verified at core 5c4c0392; R09 is closed. Its Tool
+binding and declaration rules were already verified in Refactor 42. Base remains
+the public facade; keeping short orchestration methods there is intentional.
+The DSL behavior itself remains unchanged. R10 is implemented in Refactor 44,
+pending application verification. The published SVG is applied43-01. See the
+[naming boundary](entry-action-and-team-wording.md) and the separate
+[open Tool schema finding](tool-schema-recording-gap.md).

data/docs/architecture/agent-configuration-and-tool-binding.md

@@ -0,0 +1,139 @@
+# Agent configuration and Tool binding (R09, first slice)
+
+## Role and boundary
+
+Agent::Base is the public declaration and lifecycle facade. Before tools are
+advertised to an LLM or installed in a runtime chat, its prepare_tool_class hook
+selects the Agent's alias and result filters for the original Tool class.
+ToolDefinitionSet and runtime projection installation use that same hook.
+MultiAgent::Orchestrator overrides it, calls super, then adds subagent context.
+
+Previously the hook also generated both decorator subclasses and bridged custom
+asynchronous results, including cancellation and physical completion. Refactor 42
+extracts that mechanism into Agent::ToolBinding in agent/tool_execution. Base
+retains the selection hook and its invocation keyword for Orchestrator cooperation.
+ToolBinding takes an explicit Tool class, alias and filter list; it holds no Agent
+reference and does not call private Agent methods back through send.
+
+| Owner | Responsibility |
+|---|---|
+| Base / Filterable | Select the original class's alias and ordered class, instance and scoped filters. |
+| ToolBinding | Construct alias/result-filter decorators, preserve effective Tool names, and bridge custom async results. |
+| Capability::Base / ToolExecutor | Default Tool call protocol and physical execution. |
+| ToolInvocation | Validation, authorization, execution supervision and state restoration. |
+| Orchestrator | Capture subagent parent/config/knowledge context after Base preparation. |
+
+There is one new internal class and one production file. ToolBinding is marked
+@api private, like other Agent implementation collaborators; it is not a new
+supported application API. No loader change, mixin, alias or public signature
+change is introduced. Definitions, approval, durable schemas and stored Tool
+state are unchanged.
+
+## Preparation and reading order
+
+Base first returns non-Class values unchanged. For classes it constructs the
+alias binding, then resolves filters, then prepares the filtered class. This
+order preserves subclass creation before filter selection. With neither alias
+nor filters the exact original class is returned. With an alias and no filters,
+the alias subclass is returned without instantiating a Tool.
+
+ToolBinding#prepare reads the effective name and the call_async method owner,
+then defines the synchronous decorator and, only when needed, the custom async
+decorator. filter_async_result registers physical and logical completion in that
+order. propagate_failure and complete_filtered_result separate terminal outcomes.
+The methods express one level below their operation names rather than leaving
+the whole callback protocol inside Base.
+
+The original Tool class remains the scoped-filter key even when an alias creates
+a subclass. Each preparation captures a newly collected filter list; it does not
+cache generated classes or deep-copy filter objects. Later registrations affect
+later preparations. The synchronous call passes the original args/keyword values
+to super and applies class, instance and scoped filters in that order, with the
+effective name and original args. Filter exceptions propagate without ToolError
+wrapping by this decorator.
+
+## Asynchronous contract retained
+
+- If call_async is owned by Capability::Base, leave it inherited: its ordinary
+  execution path reaches the decorated call. A second async filter wrapper would
+  apply the same filters twice.
+- Any other method owner, including an inherited custom implementation, receives
+  the custom async decorator. It forwards args/kwargs to super and wraps the
+  returned operation in PhysicalCompletionTask. It does not wait for completion.
+- Preserve logical completed/failed/cancelled status, values and error identity.
+  Failure skips filters. A filter StandardError fails the derived result with
+  that same exception. A synchronous startup exception still escapes call_async.
+- With on_physical_complete support, forward that signal independently of the
+  logical result. Logical cancellation/failure must not fabricate physical
+  completion while the source is still working. Physical completion can precede
+  the logical callback or already be true when the listener is registered.
+- Without that signal, mark physical completion before settling the derived
+  result, including filter failure. Listener registration remains physical first,
+  logical second, so already-settled operations are handled by existing callbacks.
+- Cancelling the derived result does not cancel the source. A later successful
+  source completion can still execute result filters, while its derived status
+  remains cancelled. This is existing behavior, not a new cancellation guarantee.
+
+No retry, additional cancellation propagation, thread/pool policy or physical
+completion guarantee is added. The source signal still describes source work,
+not a new synchronization boundary around the filter callback itself.
+
+## Agent declaration inheritance as currently implemented
+
+Do not infer one uniform inheritance rule from the word "configuration". The
+following describes current compatibility behavior; Refactor 42 does not change
+the DSL methods. Parent reads are live lookups, not snapshots taken at subclass
+creation.
+
+| Declaration / getter | Child without an own value | Own value / special behavior |
+|---|---|---|
+| model | Global configuration.default_model; does not read parent's model | Truthy setter, nil/false are reads. Global changes remain visible until an own value is set. |
+| instructions | Parent instructions, including the same Proc or template object | Text or block replaces the own value. nil without a block is a read, not a clear. |
+| provider | Parent provider | Truthy setter; nil/false read. |
+| tools | Parent tools array by reference | A Hash replaces the own class list. An empty Hash is an explicit empty list. nil reads. |
+| tool_aliases | Parent aliases merged with own aliases | Alias values are stringified; nil entries are omitted. A child nil alias or empty tools list does not erase inherited aliases. |
+| context_policy | Parent policy object, ultimately ContextPolicies::Default.instance | Exactly one ContextPolicy object replaces it. nil is invalid, not reset. |
+| temperature | nil; does not inherit | Truthy setter; zero is accepted; nil/false read. |
+| max_iterations | 10; does not inherit | Truthy setter; zero is accepted. |
+| cache_instructions | nil; does not inherit | nil reads, false is an explicit value. |
+| max_output_tokens | nil; does not inherit | nil reads; positive Integer conversion. context_window moved to the RubyLLM registry in Refactor 45. |
+| agent_definition | Raises until that class declares identity/revision | Each subclass requires its own definition; this is not inherited model configuration. |
+| input/output/tool_result_filter registries | Empty class registry; do not inherit | Registrations append on that class. Instance filters are separate. |
+| before_llm_input / _before_llm_input | nil class callback; does not inherit | Instance and global hook behavior remains separate. |
+
+The generic Agent rule does not override specialised subclass DSL such as
+Orchestrator's subagent registry. Tool/Capability declaration inheritance is also
+a different contract, already covered by its configuration-inheritance tests.
+
+The existing test description that said "normal class configuration still
+inherits" referred only to instructions and was too broad. It now names the
+actual checked rule. No new policy is inferred from that earlier wording.
+
+For compatibility, retain these rules in this source refactor. Uniform parent
+fallback, alias removal and copying inherited mutable values would be separate
+behavior changes requiring an explicit migration decision. They can change
+model/provider combinations, budgets, Tool names and application callbacks.
+
+## Verification and remaining R09 work
+
+Thirty-five new contract examples run against both the unchanged Refactor 41 and
+the candidate. They exercise selection/alias order, names, filter ordering,
+forwarded object identity, default/custom async behavior, logical/physical
+completion, cancellation and DSL inheritance. Existing Tool definition, approval,
+Orchestrator and full integration suites cover the cooperation paths.
+The release, API/SPI, RBS, style, isolated gem and examples gates are recorded in
+the distribution. Candidate remote CI, live PostgreSQL, live LLM and performance
+are not measured by this change.
+
+Refactor 42 is applied and independently verified at core 4a57a3c2. This
+completes the Tool binding and declaration-contract slice. Refactor 43 addresses
+the original Chat construction and explicit state-mutation scope; see the
+[Chat/state design](agent-chat-and-state-ownership.md). It preserves the Base
+hooks and operation/publication order rather than changing DSL semantics.
+
+Refactor 43 application was verified at core 5c4c0392; R09 is closed.
+R10 is implemented in Refactor 44, pending application verification.
+The published SVG is applied43-01. See the [naming boundary](entry-action-and-team-wording.md)
+and the separate [open Tool schema finding](tool-schema-recording-gap.md).
+
+Refactor 45 changes the token declarations as described in [the migration](rubyllm-2-token-ownership.md). Other Agent inheritance rules above remain unchanged. Tool schema/provider configuration follows the RubyLLM 2 inherited configuration contract.

data/docs/architecture/agent-context.md

@@ -84,8 +84,10 @@ See [Context Management](context-management.md) and
 ## 3. Live mutable ownership
 
 Within one Runtime/process, one mutable live Agent instance owns one `agent_id`.
-Loading or creating a second mutable live instance for the same `agent_id` is
-rejected.
+`Agent::OwnershipRegistry` reserves that identity before materialization;
+Runtime strongly retains the registry through the generic shutdown contract.
+Repeated load returns the same live instance, while duplicate create is rejected.
+See [ADR-041](../decisions/041-feature-owned-identity-registries.md).
 
 During active execution, EventLoop is the single writer of Phronomy-managed live
 Agent execution state. Blocking Persistence work and other synchronous work that

data/docs/architecture/agent-transition-ownership.md

@@ -0,0 +1,13 @@
+# Agent transition ownership (r8 unit 7)
+
+InvocationTransitions owns both automatic and external transitions, including
+ordered guard evaluation. InvocationActions owns Agent entry operations.
+EngineSessionBuilder configures FSMSession using those definitions;
+PhaseMachineBuilder adapts them to state_machines without defining progress rules.
+ExecutionCoordinator and RecoveryCoordinator retain durable decisions and use
+the Agent-owned ExecutionEnvironment port for execution connection.
+
+See [r8 unit 7](r8-unit7.md) and [ADR-067](../decisions/067-agent-progress-and-engine-connection.md).
+Existing event source validation, callback failure priority, short-circuiting,
+Tool outcome priority, suspended/wait/terminal boundaries and session incarnation
+rules are preserved. Agent's Tool child FSM is still a separate remaining task.

data/docs/architecture/backend-groups.md

@@ -0,0 +1,152 @@
+# Backend responsibilities and asynchronous execution clients
+
+The diagram distinguishes responsibility **groups** from its display layers.
+The earlier B1-B6 horizontal bands and topical columns are restored for
+readability. `G14`, `G46`, `G47` and `G48` remain stable responsibility identifiers;
+boundary permissions follow those responsibilities, not B/G numbers or positions.
+Each source directory keeps its own module box and stable module ID. Restoring
+the display does not undo the separation of backend contracts and execution.
+
+The restored display puts separated Backend Contracts in B5, alongside but
+distinct from Engine Internals and Execution Contracts. B4 holds Async Clients and separately grouped Implementations. Execution
+operations and values now share G58; the former G59 service split is removed. Backend directories awaiting separation remain visibly mixed
+in B4; their contracts move to B5 in their implemented phase. These are display
+positions, not a shared architectural level or namespace.
+
+| Group | Responsibility | Dependency rule |
+|---|---|---|
+| G14 Engine Internals | Runtime, workers, FSM and EventLoop | No feature-specific backend or client dependencies |
+| G58 Execution Framework | Submission protocol, TaskResult, Execution, Blocking, controls and common rules | Only Execution and Common, including neutral settings; Engine implements the mechanism |
+| G46 Async Clients | Execute synchronous backend operations through Execution | Use Backend and Execution contracts; no direct Engine or concrete backend selection |
+| G47 Backend Contracts | Synchronous SPI, values, errors and shared rules | No Engine, Execution Services, Async Client or concrete implementation dependencies |
+| G48 Backend Implementations | Implement the synchronous backend SPI | Depend on the corresponding Contracts; no Engine, Execution Services or client dependencies |
+| G57 Content Service | Content IDs, canonical text/JSON and digest integrity over Storage | Uses neutral Storage and Common; no physical backend, Engine or domain orchestration dependencies |
+
+Composition chooses concrete backends and supplies them to consumers. These
+are source-ownership groups, not shared Ruby namespaces. Synchronous APIs can
+be called directly. Diagram links represent source references; calls through
+injected objects require separate runtime review.
+
+## Implemented phase: P1/P2/P3/P4 (`storage`)
+
+P3 adds public VectorStore and Embeddings AsyncClients. Their synchronous
+contracts and implementations have no Engine dependencies. See the
+[public receiver migration](../migrations/vector-async-clients.md).
+
+### P2: LLM
+
+- Install the AST analyzer, group annotations, SVG formatter and boundary gate
+  in `tools/architecture/`. The Architecture workflow checks the committed
+  phase and generates evidence for its actual checkout SHA.
+- `LLMAdapter::Base` now defines synchronous generation plus chat construction,
+  configuration, message/tool-call values, identity and optional input budget.
+  RubyLLM-specific construction and error translation belong to its backend.
+  Adapter extensions must implement the new operations; no legacy wrapper remains.
+- `LLMAdapter::AsyncClient` owns the internal `complete_async` and `stream_async`
+  operations in `llm_adapter/async/`. It is not a public extension API.
+- `LLMAdapter::RubyLLM` lives in `llm_adapter/backends/` and still inherits Base.
+  Its public name, synchronous signatures and default configuration are unchanged.
+- Zeitwerk collapses these two new directories. Neither `Phronomy::Async` nor
+  `LLMAdapter::Async` / `LLMAdapter::Backends` is introduced.
+- Agent constructs the framework client around the configured synchronous
+  adapter. Custom adapters implement no async methods and need no pool logic.
+
+The client preserves `on_full: :raise`, the original pool TaskResult, token
+forwarding and per-chunk cancellation. Transport timeout/retry stays with the
+adapter/provider. Construction starts no Runtime, and the default pool is
+resolved again on each operation; an injected pool stays caller-owned. Only an
+internal lightweight sink runs on the worker; application event callbacks
+continue to run on EventLoop.
+
+Use `require "phronomy"` and the public constants. The old implementation path
+`phronomy/llm_adapter/ruby_llm` moves to
+`phronomy/llm_adapter/backends/ruby_llm`; arbitrary partial-load paths are not the
+public loading contract. No shim in the contract directory reintroduces an
+implementation dependency. No public RBS or Stable/Beta API snapshot changes
+are needed in this phase; both are verified unchanged.
+
+### P3: VectorStore and Embeddings
+
+- `VectorStore::AsyncClient` owns public add/search/remove/clear async operations;
+  `VectorStore::Embeddings::AsyncClient` owns public `embed_async`.
+- Backends implement only synchronous methods. The old `AsyncBackend` mixin
+  and inherited async methods are removed; application callers change receiver.
+- InMemory, Pgvector, RedisSearch and RubyLLMEmbeddings move into `backends/`
+  directories. Their public constants, synchronous signatures and bodies stay
+  unchanged. Zeitwerk explicitly collapses the new `async/` and `backends/` paths.
+- Synchronous RBS uses structural `_CancellationSignal`; async signatures keep
+  TaskResult and the execution token. The public API snapshot records the new
+  receivers and the intentional removals from Base/InMemory.
+- All three original backend-to-Engine edges are removed. In the measured view,
+  M28/M29 join G47 Contracts in B5. M67/M68 are clients in B4 and M71/M72 are
+  separately grouped implementations. No pending Storage clients are drawn.
+
+### P4: Storage execution
+
+- Public `Storage::AsyncClient#transaction_async` opens the raw backend transaction
+  on a worker and yields its scoped Storage::View. Backends stay synchronous.
+- Private `Storage::AsyncClient.submit(pool:)` executes existing domain units
+  without adding a transaction, timeout or token. All 16 durable submission sites
+  are recorded in the [execution inventory](storage-execution-boundaries.md).
+- M69 is an Async Client and M35 a concrete Storage backend in B4.
+  Existing `Storage::Backends` is not collapsed.
+- See the [public API and transaction rules](../migrations/storage-async-client.md).
+
+### ContentStore service ownership
+
+ContentStore is a content management service using Storage, not a peer physical
+backend. Its API, canonicalization, StoredContents implementation and owned
+StorageSchema live in `content_store/`. Base remains shared by StoredContents
+and the Persistence exception facade. The unnecessary `backends/` subdivision
+and Zeitwerk collapse are removed without a forwarding file or alias.
+
+M10 belongs to G57 Content Service in the persistence column of display band B3.
+M73 is retired, not reused. G47/G48 contain no ContentStore modules. The real
+M10-to-Storage dependency remains in both the drawing and the full graph.
+The boundary gate rejects reverse Storage-to-ContentStore dependencies and any
+service path to a physical backend, Engine, client or domain orchestration.
+Public constants, signatures, content IDs, exceptions and transaction behavior
+remain unchanged. See [ADR-062](../decisions/062-content-store-service.md).
+
+## r8 unit 1 framework boundaries
+
+G58/M76/M77 now owns the common Execution framework under `execution/`.
+Engine owns FSM-specific values and the pool/timer/runtime binding, while
+composition selects that binding. Tracing owns explicit observation. Context
+and Tool now own their shared operations, not merely value/error declarations.
+The former G59/M78 split is retired. See [r8 unit 1](r8-unit1.md) for the exact
+implemented scope, API removals and incomplete Persistence/domain work.
+
+The full architecture still contains existing cycles outside the checked scope.
+The recorded graph is not an ideal final r8 diagram.
+
+## Pending phase
+
+P5 completes the final cross-repository/API/docs review after application.
+All retained P2-P4 responsibilities exist in source; the unnecessary M73 split
+is superseded by G57 service ownership. The measured SVG still comes from
+actual Ruby analysis; the target concept is a separate explanatory artifact.
+
+## Regenerate and review
+
+See [tool instructions](../../tools/architecture/README.md). The normal command
+reads `tools/architecture/phase.json`:
+
+```sh
+python tools/architecture/refresh_diagram.py . tmp/architecture
+```
+
+The output includes the matrix, source hashes, boundary report, module/file
+cycles and the delta from baseline 84668606. Source/module/group changes require
+matching annotation updates. No up/down rule or coordinate controls the gate.
+Ruby loader, RBS, cancellation, admission, streaming and configuration tests
+remain necessary: static analysis does not replace them.
+
+An unpublished local candidate must use `--candidate`. Its SVG states that it
+is unapplied and disables GitHub source links. Applied commits and CI output
+use actual source links. CI artifacts are generated after the source commit;
+there is no self-referential SVG commit hash or automatic commit/push.
+
+Text panels are transparent and module outlines/group colors remain intact.
+Arrows into M33/M41/M44 are hidden by presentation settings; their AST edges,
+source evidence and matrix cells are preserved. Boundary rules inspect all edges.

data/docs/architecture/before-llm-input.md

@@ -28,13 +28,19 @@ candidates are accumulated in hook order.
 
 ## 3. Input and result
 
-Hooks receive immutable `Phronomy::Agent::LLMInputBuildContext` metadata. They do
+Hooks receive immutable `Phronomy::Context::LLMInputBuildContext` metadata. They do
 not receive a mutable Provider chat/message array.
 
-A hook returns `Phronomy::Agent::LLMInputPatch` or `nil`.
+A hook returns `Phronomy::Context::LLMInputPatch` or `nil`.
+
+These two types live in the shared `agent/context_contract/` implementation
+directory together with the Policy input and result contracts. Zeitwerk
+collapses the directory, preserving their existing public `Phronomy::Agent`
+names. Hook registration, invocation, and result handling are unchanged; see
+[ADR-036](../decisions/036-context-contract-ownership.md).
 
 ```ruby
-Phronomy::Agent::LLMInputPatch.new(
+Phronomy::Context::LLMInputPatch.new(
   model_config_patch: {temperature: 0.2},
   segment_candidates: [
     {

data/docs/architecture/context-management.md

@@ -50,12 +50,42 @@ RubyLLMMaterializer
 The normative Journal/Manifest split is
 [ADR-012](../decisions/012-canonical-execution-log-and-context-policy.md).
 
+### Contract ownership and physical layout
+
+The contracts shared by Agent execution, Context policies, and input hooks live
+in `lib/phronomy/agent/context_contract/`:
+
+| Files | Responsibility |
+|---|---|
+| `context_policy.rb`, `context_policy_input.rb` | Policy protocol, helper methods, and typed input material |
+| `context_plan.rb`, `context_plan_validator.rb` | Policy result and its validity rules |
+| `llm_input_manifest.rb` | Final logical input representation and its format validation |
+| `llm_input_build_context.rb`, `llm_input_patch.rb` | Input metadata and returned customization for hooks |
+
+Zeitwerk collapses this directory. The Ruby constants keep their existing
+`Phronomy::Agent::*` names; there is no `ContextContract` namespace or new
+registration mechanism. Applications continue to load `phronomy` and use the
+documented constants. The previous individual implementation file paths are
+not retained as forwarding files.
+
+`ContextPolicies::Default` remains a concrete strategy, while
+`ContextAssembler`, input builders, candidate resolution, Runtime connections,
+and persistence transactions remain on the execution side. The Manifest
+validates and converts record values; it does not write to a storage backend.
+Its format version, keys, and validation rules are unchanged.
+
+This ownership decision is defined by
+[ADR-036](../decisions/036-context-contract-ownership.md). Physical containment
+under `agent/` does not assign the contracts to the execution layer. The
+concrete policies and the shared contracts can occupy distinct groups at the
+same abstraction level.
+
 ## 2. Public Context Policy SPI
 
 Application code supplies an ordinary reusable Ruby strategy object:
 
 ```ruby
-class SearchContextPolicy < Phronomy::Agent::ContextPolicy
+class SearchContextPolicy < Phronomy::Context::ContextPolicy
   def initialize(search:)
     @search = search
   end

data/docs/architecture/context-preparation-steps.md

@@ -0,0 +1,90 @@
+# Context preparation steps
+
+## Role and scope
+
+ContextAssembler prepares the immutable semantic input supplied to Application
+ContextPolicy and later validates and stores its decision as an LLMInputManifest.
+InitialPreparation and DispatchPreparation run preparation outside their commit
+transactions, recheck durable state, then call finalize in the commit transaction.
+The public boundary and those callers already have separate responsibilities.
+
+R07 addresses a narrower readability problem inside prepare_initial and
+prepare_followup: orchestration was mixed with individual item IDs, provenance
+and metadata, and record filtering plus Hook/Handoff merging were duplicated.
+The original proposal calls for private extraction before adding collaborators.
+Refactor 40 follows that limit: one production file changes; no class is added.
+
+## Reading order and responsibilities
+
+| Private operation | Responsibility |
+|---|---|
+| initial_instruction_items | Choose the initial Agent instruction and optional Handoff responsibility, in order. |
+| agent_instruction_item | Build the required system instruction with its stable ID, estimate and Agent provenance. |
+| handoff_instruction_item | Build the required user responsibility with its ID, estimate and Handoff origin. |
+| retained_instruction_items | Read only retained base instructions from the previous Manifest, using the existing compatibility predicate and converter. |
+| resolve_record_candidates | Filter working records by eligibility/current generation and resolve them with Journal candidates and caller-provided exclusions. |
+| merge_context_candidates | Add Hook candidates, then Handoff candidates, using the existing creation, sorting and storage operations. |
+| current_input_item | Build the initial required ask argument after all candidates, including ID, sequence, provenance and metadata. |
+
+Public preparation keeps model/Tool settings, source loading and the sequence of
+these operations visible. Record collection and candidate augmentation stay as
+two steps: follow-up computes its next call sequence after record resolution,
+exactly as before. No new options object, pipeline DSL, mutable preparation state
+or cross-feature factory is introduced. Helpers remain private to the assembler.
+
+## Preserved contracts
+
+- Initial instructions come from the Agent configuration plus optional Handoff
+  responsibility. Follow-up instructions come from the retained base Manifest.
+  Current Hook instructions are rebuilt; previous Hook/Policy-generated items
+  do not become permanent base instructions. Legacy origin handling is unchanged.
+- Working records must remain context candidates in the active transcript
+  generation. Initial preparation excludes the current input record; follow-up
+  includes eligible working records without that exclusion.
+- Record candidates are resolved before Hook and then Handoff augmentation. The
+  current-input sequence follows all candidates, including non-conversation ones.
+  IDs, source fields, trust metadata, estimates, required flags, ordering, frozen
+  values and delivery modes retain their existing values.
+- Initial input content is loaded before instruction-item construction and
+  candidate resolution. Initial instruction callbacks and patch normalization
+  retain their order. Hook content writes still precede Handoff type validation;
+  moving that validation earlier would be a behavior change and is not included.
+- Preparation is not side-effect-free: Hook/Handoff content may be stored before
+  Policy succeeds. This extraction neither adds rollback nor suppresses existing
+  exceptions. Missing-input, reserved-metadata, resolver and Policy failures keep
+  their ordering; collaborator exception objects propagate unchanged.
+- Application Policy executes once during prepare; finalize does not invoke it.
+  Finalization, its validation/encoding/store helpers, assembly policy version 8,
+  schemas, public signatures and transaction-owning callers are unchanged.
+
+The new helpers use the same constructor field evaluation and call ordering.
+They do not cache or deduplicate reads, normalize values earlier, or move work
+across the prepare/finalize boundary merely to shorten the source.
+
+## Verification and trade-off
+
+Ten behavior examples cover initial item provenance/immutability, generation and
+exclusion rules, content-write order, absent instructions/Handoff, follow-up
+retention and current hooks, the prepare/finalize boundary and failure precedence.
+The same examples pass on Refactor 39. Four independent baseline/candidate
+scenario pairs compare complete Policy input, Prepared values, Manifest bytes and
+content references, plus preparation-time content read/write order. The pairs
+cover initial and follow-up calls with/without base instructions and Handoff.
+
+Full core/integration suites, common examples and real SQLite, stable/beta API
+and Storage SPI snapshots, RBS, style, annotations and isolated gem checks are
+also run. Live PostgreSQL, live LLM, remote CI and performance are not exercised
+for this candidate; prior CI is not evidence for the candidate's core.
+
+prepare_initial is 103 -> 55 lines and prepare_followup 57 -> 47. The whole file
+is 625 -> 672 lines because seven named private methods and explicit arguments
+add structure. This is a readability and duplicated-procedure improvement, not a
+net line-count reduction or a directory-cycle change. The methods' names give
+readers a place to choose whether they need the detailed representation.
+
+R07 was independently verified after application at core b3dfbc5a, tree
+dcd9d1bf8efb1b488f9c2b3b1c4e99bdeda6096c. All six files and the full tree matched;
+core/integration/examples/SQLite/API/type/gem and four paired scenarios passed.
+R07 is complete. The published diagram is applied40-01. R08/D08 now has a Refactor
+41 candidate, with R09 and R10 remaining afterward. Their behavior and naming
+decisions are not part of Context preparation extraction.