phronomy 0.26.0 → 0.28.0

data/docs/migrations/r8-unit8.md

@@ -0,0 +1,33 @@
+# Migrating to r8 unit 8
+
+Apply after core a601a0654ace42640f0251b129359ca1f326e3ad (unit 7).
+This is a core-only change. The unit 6 examples at
+ed455c52018338f7c5a4c2948ae0d7db131aa090 require no source changes.
+
+Public Agent invocation/stream/approval/cancellation/recovery signatures and
+Tool#call_async(args, cancellation_token:, config:) are unchanged. Tool::Operation
+adds an explicit C06 operation entry and result transformation API; application
+Tools do not need to adopt it. The supplied submitter applies to the default
+offloaded operation only. Custom async implementations retain their existing
+execution mechanism and configuration.
+
+| Internal use | Replacement |
+| --- | --- |
+| Agent::ToolInvocationSessionBuilder.build / build_for_resume | The owning ExecutionEnvironment.build_tool_session, with resume_event/resume_phase when needed |
+| ToolInvocationSessionBuilder transition constants | Agent::ToolInvocationTransitions |
+| ToolInvocationSessionBuilder entry operations | Agent::ToolInvocationActions |
+| ToolInvocation.start_authorization/runtime: and start_execution/runtime: | Pass environment: explicitly; never reconstruct it from a new global Runtime |
+| Tool::ToolExecutor.call_async(runtime:) | Tool::Operation.call_async(submitter:) for contract consumers; default executor takes submitter: internally |
+| Agent ToolBinding's custom-async detection and mapping mechanism | Tool::Operation.with_result_transform; Agent retains filter configuration |
+
+Removed internal names have no aliases. Deploy all internal call sites together.
+Do not persist execution environments, submitters, FSM sessions or parent sinks.
+Durable IDs, record schemas, revision checks and saved approval evidence are
+unchanged. The change does not certify new in-flight cross-version compatibility.
+Use the existing stop/drain procedure before replacing running processes.
+
+The application ZIP contains strict-base --check/--apply/--verify commands,
+complete replacement files and deletions, expected-tree verification, a review
+patch, validation evidence and rollback instructions. Run --verify before and
+after commit. Verify examples with PHRONOMY_PATH pointing at the candidate and
+unset a stale BUNDLE_GEMFILE before entering each Bundler context.

data/docs/migrations/r8-unit9.md

@@ -0,0 +1,28 @@
+# R8 unit 9 migration
+
+Apply to core unit 8, commit e9f6cc16da19be82a0b7777486926ba46d070def.
+Examples source remains at unit 6, ed455c52018338f7c5a4c2948ae0d7db131aa090.
+
+No public Tool or Agent signature, stored schema, migration CLI or example source
+change is required. Existing configured filter order and repeated registrations
+are preserved. The correction removes only the extra transformation caused by
+Base's standard async-to-sync delegation under a custom async result decorator.
+
+A custom call_async using super now passes its raw delegated result through its
+own processing before the async decorator applies the configured transform.
+Code accidentally relying on the duplicate/intermediate transformation should
+express the required stages explicitly in application filter configuration.
+
+A custom call_async explicitly calling public call retains that call's filters;
+the enclosing async result decorator also retains its stage. Do not substitute
+call for super when the intent is to delegate to Base's normal async execution.
+The framework does not suppress application-authored calls or infer their intent.
+
+Private changes: Base uses Tool::Operation.synchronous_delegate; ToolExecutor
+accepts a bound synchronous_call keyword. Applications should not use these
+private bridge helpers. Custom async implementations retain their existing
+execution mechanism and keyword protocol.
+
+Use the normal process stop/drain and restart procedure. This unit does not add
+cross-version in-flight compatibility or rollback of external Tool side effects.
+See docs/architecture/r8-unit9.md for operation-boundary examples.

data/docs/migrations/shared-state-multi-agent.md

@@ -0,0 +1,44 @@
+# SharedState moves to MultiAgent
+
+This change applies to the architecture refactoring branch after core commit
+`aa9c3d53090bda4deb01b8cab16a43539b6b1deb`. Use the matching core and examples
+checkouts; the unchanged gem version number alone does not identify this API.
+
+| Previous Experimental API | API after this change |
+|---|---|
+| `Phronomy::Agent::SharedState` | `Phronomy::MultiAgent::SharedState` |
+| `Phronomy::Agent::SharedState::KnowledgeStore` | `Phronomy::MultiAgent::SharedState::KnowledgeStore` |
+
+The previous name is removed without an alias. Update superclass declarations
+and explicit constant references. For example:
+
+```ruby
+require "phronomy"
+
+class ResearchTeam < Phronomy::MultiAgent::SharedState
+  member Researcher
+  max_cycles 3
+end
+```
+
+Use the public application entry point `require "phronomy"`. The old internal
+file `phronomy/agent/shared_state.rb` is removed; arbitrary implementation-file
+requires are not a supported partial-loading API.
+
+The DSL, `invoke(input, config: {})` signature, sequential order, shared findings,
+termination rules and `{output:, cycles:, terminated_by:}` result remain the same.
+`config:` remains accepted but is not forwarded to member invocations. Timeout
+is checked between complete cycles and does not cancel an active Agent.
+
+Set `PHRONOMY_PATH` to the matching local core checkout before running
+`bundle install` or the examples verification. Example `22_shared_state` now
+uses the new superclass, and the examples API preflight rejects mismatched core.
+
+No saved-record or SQL schema migration is required. Do not rewrite the existing
+`Phronomy::Agent::SharedState::Instrumented/` generated definition ID: it is
+deliberately preserved together with instrumentation version 1. It is not an
+alias for the removed Ruby constant. The in-memory KnowledgeStore still lasts
+for one invocation only.
+
+See [ADR-053](../decisions/053-shared-state-coordination-ownership.md) for the
+ownership and compatibility decision.

data/docs/migrations/storage-async-client.md

@@ -0,0 +1,103 @@
+# Storage asynchronous execution and ContentStore ownership
+
+P4 adds the public Beta `Phronomy::Storage::AsyncClient`. Synchronous storage
+backends and `Persistence#transaction` keep their existing interfaces. Backend
+authors implement no scheduler or async methods.
+
+## Public transaction API
+
+Construct `Storage::AsyncClient.new(backend: raw_backend, pool: nil)` around a
+Storage SPI 2 backend. For an existing Persistence facade, use
+`persistence.backend`. The transaction block receives a **raw Storage::View**,
+not the domain repositories yielded by `Persistence#transaction`.
+
+`transaction_async(cancellation_token: nil, timeout: nil) { |view| ... }`
+returns the pool's original TaskResult. The complete transaction runs on one
+worker, using the backend's normal connection and scope. Do not split a
+transaction across workers or wait for another TaskResult inside its block.
+Explicit synchronous `backend.transaction` calls within the block retain nested
+savepoint behavior.
+
+### Complete offline example
+
+This requires a checkout containing P4; the previously released 0.27.0 gem does
+not contain the new client. It uses only the public raw storage contract.
+
+```ruby runnable
+require "phronomy"
+require "json"
+
+resource = Phronomy::Storage::Resource.new(id: "example.notes", kind: :records)
+backend = Phronomy::Storage::Backends::InMemory.new(resources: [resource])
+client = Phronomy::Storage::AsyncClient.new(backend: backend)
+
+begin
+  record = Phronomy::Storage::DurableRecord.new(
+    record_type: "example.note", format_version: "0.1", payload: {"text" => "saved"}
+  )
+  task = client.transaction_async do |view|
+    rows = view.records(resource)
+    rows.insert(key: "note-1", revision: 1, attributes: {}, record: record)
+    rows.fetch("note-1").record.payload
+  end
+  puts JSON.generate(task.wait_result)
+ensure
+  Phronomy.reset_runtime!
+end
+```
+
+The output is `{"text":"saved"}`. Return a value from the transaction. A bound
+view or resource handle becomes invalid after the block finishes, even if it
+was returned through TaskResult. Scope also rejects access from another thread.
+Exceptions and invalid non-local block exits roll back through the existing
+backend rules; no new transaction implementation is introduced.
+
+## Admission, cancellation and ownership
+
+Construction starts no Runtime. Each call resolves the current default pool,
+unless an explicit caller-owned `pool:` was supplied. Runtime reset does not
+transfer ownership of an injected pool.
+
+Admission uses `on_full: :raise`: a full or stopped pool raises immediately.
+Queue wait counts toward `timeout`. Cancellation/timeout before worker pickup
+skips the transaction. After work starts, TaskResult may settle while the
+physical transaction continues and commits. A timeout or cancellation is **not
+a rollback guarantee**. The client does not kill a thread, add retries, or claim
+that an uncertain commit succeeded or failed. Applications needing a durable
+outcome must reconcile through their domain's identities and stored state.
+
+## Existing Agent, Workflow and coordination operations
+
+Internal `Storage::AsyncClient.submit(pool:)` accepts an existing synchronous
+storage unit. This is private framework API, not an application extension API.
+It submits exactly once and adds no transaction, token, or timeout. This lets
+domain operations preserve their own transaction and reconciliation boundaries.
+See the [operation inventory](../architecture/storage-execution-boundaries.md).
+
+In particular, a Workflow runner may have only an injected repository rather
+than access to its raw backend. Internal submit therefore accepts a callable
+and the owner's pool; it does not require a dummy backend or a Persistence
+facade. Public transaction_async requires a real SPI 2 backend.
+
+Durable commit/readback policy, admission release, callback delivery and the
+application of results to live state remain with Agent/Workflow/MultiAgent.
+No cancellation condition is added to existing terminal saves. Tool execution,
+authorization, tracing and application approval listeners retain their own
+execution boundaries.
+
+## ContentStore path and diagram
+
+P4 originally moved `ContentStore::StoredContents` into
+`content_store/backends/stored_contents.rb`. [ADR-062](../decisions/062-content-store-service.md)
+supersedes that split: the implementation now lives at
+`content_store/stored_contents.rb`, beside its shared API and owned schema.
+The obsolete backend directory and collapse entry are deleted without a shim.
+Its constant, synchronous operations, digest rules and exceptions are unchanged.
+Use normal `require "phronomy"`; internal file paths are not a public load API.
+Zeitwerk still collapses `storage/async/`. `Storage::Backends` remains a namespace.
+ContentStore receives no artificial async API.
+
+M10 is G57 Content Service in B3, outside Backend Contracts/Implementations;
+M73 is retired. M69 Storage AsyncClient stays in B4. Text panels are transparent.
+Arrows targeting M33/M41/M44 are hidden by display settings; all measured
+dependencies, source evidence and matrix cells remain available.

data/docs/migrations/storage-backend-composition.md

@@ -0,0 +1,117 @@
+# Storage Backend composition migration
+
+Historical migration step. Its old raw SPI surface is superseded by the
+[neutral Storage SPI 2 migration](neutral-storage-spi.md); use that guide for the
+current backend contract.
+
+
+This change targets the `refactor/architecture` branch after `1b481f6`, with
+shared value operations already owned by `Values::Immutable`.
+It replaces the Beta Backend SPI; the gem version and persisted record formats
+are not changed by this branch-only refactoring package.
+
+## Application construction
+
+An isolated in-memory domain is now constructed with:
+
+```ruby
+require "phronomy"
+
+persistence = Phronomy::Persistence.in_memory
+Phronomy.configure { |config| config.persistence = persistence }
+```
+
+An explicitly selected raw backend is composed once:
+
+```ruby
+require "phronomy"
+
+backend = Phronomy::Storage::Backends::InMemory.new
+persistence = Phronomy::Persistence.new(backend: backend)
+Phronomy.configure { |config| config.persistence = persistence }
+```
+
+Use the same `persistence` instance wherever one storage/ownership domain is
+required. Constructing several facades over one physical backend does not create
+independent data, and their object identities are distinct for callers that use
+Persistence identity to determine a domain. `persistence.backend` exposes the
+selected raw backend for backend-specific administration such as accessing its
+connection pool. Ordinary domain reads and writes still use `persistence`.
+
+## Replaced API
+
+| Previous API | Replacement |
+|---|---|
+| `Persistence::InMemory.new` | `Persistence.in_memory` |
+| Backend subclass of `Persistence` | Backend subclass of `Storage::Backend`, or the same raw protocol |
+| `Persistence.new` with eight raw repository keyword arguments | `Persistence.new(backend: raw_backend)` |
+| Backend `super` with eight repositories | The same arguments passed to `Storage::Backend#initialize` |
+| `Persistence#build_transaction_view` | A raw `Storage::Repositories` view yielded by the backend |
+| `Persistence::DurableRecord` | `Storage::DurableRecord` |
+| `Persistence::ConflictError` / `NotFoundError` / `SerializationError` / `UnsupportedBackendError` | The same names under `Storage` |
+| `Persistence::REQUIRED_CAPABILITIES` | `Storage::Backend::REQUIRED_CAPABILITIES` |
+
+The previous constants/helper/constructor are removed. Rescue the new storage
+error constants. The public `Phronomy::AgentBusyError` admission exception remains.
+This is a source/API migration, not a persisted payload migration.
+
+## SQL implementation changes
+
+A raw SQL backend implements `capabilities`, `transaction`, and
+`assert_agent_watermark!`, and supplies all eight raw repositories to the common
+Backend constructor. It exchanges `Storage::DurableRecord` and explicit ID,
+revision, position, and admission metadata. It does not decode domain types.
+
+In its transaction implementation:
+
+1. Check out a connection and open the database transaction as before.
+2. Bind contents, agents, journals, executions, workflow_states, handoff_states,
+   teams, team_executions, and the watermark object to that same connection.
+3. Construct `Storage::Repositories` with those eight keyword arguments and
+   `watermark:`; yield this raw view inside the database transaction block.
+4. Return the block result; propagate failures through the existing transaction
+   rollback path. Keep the existing handling of indeterminate commit outcomes.
+
+`Persistence#transaction` constructs the corresponding domain facades inside
+that block. SQL code no longer calls `build_transaction_view` or private
+`PersistenceComposition::Repositories` or domain repository wrappers. Application/bootstrap code creates the raw SQL backend,
+then passes it to `Persistence.new(backend:)`.
+
+The SQLite and PostgreSQL implementations in `phronomy-examples` at `231d253`
+use the previous SPI and require this migration. Do not use their unchanged
+sources with the new core. Keep this branch's API change separate from a release
+until the applications/backends to be shipped with it have also migrated.
+
+## Validation
+
+Use the existing explicitly loaded shared contract suite with a composed
+`persistence` value. All eight repositories must still commit or roll back
+together. Validate root and transaction repository binding, CAS/admission and
+watermark checks, record round trips, and codec failure rollback. Perform actual
+SQL tests with the backend's supported database and driver versions; an
+in-memory success does not prove SQL transaction/concurrency behavior.
+
+## Execution constraint errors (ADR-043)
+
+Raw `Storage::Backend` execution repositories must report
+`Storage::ActiveExecutionConflictError`, a `Storage::ConflictError` subtype, when
+an existing nonterminal execution prevents another admission or an idle-only
+operation for that owner. Update the same constraint branches in `save` if the
+backend already checks them. Do not convert unrelated identity/revision failures.
+
+The Agent and Team repository facades translate this subtype into the existing
+`Phronomy::AgentBusyError` while preserving its message and cause. A transaction
+still sees the exception before commit. Public Persistence/Agent/Team callers
+keep their previous busy exception; raw backend callers must update their rescue.
+
+Apply the new core before the migrated SQLite/PostgreSQL sources. Both examples
+require the new storage constant; for coordinated source development set
+`PHRONOMY_PATH` to the updated core checkout before resolving/running their
+bundles. An unchanged legacy backend remains usable through the domain facade
+because its old `AgentBusyError` is passed through, but it does not satisfy the
+new raw conformance checks. No fallback constant aliases are supplied.
+
+Run `a Persistence backend` from `phronomy/testing/persistence_contract`; it now
+includes `storage execution constraint notifications` for both raw execution
+repositories. Keep running the domain repository suites and backend concurrency
+and rollback tests. No schema or serialized data migration is required.

data/docs/migrations/storage-transaction-boundaries.md

@@ -0,0 +1,74 @@
+# Storage transaction boundaries (Refactor 34)
+
+Historical migration step. Its old raw SPI surface is superseded by the
+[neutral Storage SPI 2 migration](neutral-storage-spi.md); use that guide for the
+current backend contract.
+
+
+Update core and the SQLite/PostgreSQL reference adapters together. No data
+migration is required. The public method signatures, eight repository accessors,
+required capability keys and stored record formats are unchanged.
+
+## Explicit nested calls
+
+Nested `Backend#transaction` and `Persistence#transaction` calls on the same
+backend and synchronous execution context now have savepoint semantics.
+
+```ruby
+persistence.transaction do |outer|
+  outer.contents.put_text("keep if outer succeeds")
+  begin
+    persistence.transaction do |inner|
+      inner.contents.put_text("discard on inner failure")
+      raise "inner failure"
+    end
+  rescue RuntimeError
+    # Inner changes have already been rolled back. Re-read conditions needed
+    # for subsequent work through the valid outer scope.
+    outer.contents.put_text("outer may continue")
+  end
+end
+```
+
+Earlier SQL reference versions joined the outer scope. Catching the inner error
+could preserve inner writes. Such writes are now rolled back. A successful inner
+call does not commit independently; failure of the outer call rolls back both.
+The same exception object is propagated after rollback, including
+`ActiveRecord::Rollback`, which ActiveRecord itself normally consumes. Catch
+that exception outside the Storage/Persistence block if continued execution is
+intended. No separate database connection is introduced.
+
+## Invalid Journal batches and database failures
+
+The SQL references now validate and serialize every record and normalize the
+expected position before the first write. An invalid second record cannot leave
+the first inserted with an unchanged Journal head. Record IDs from a rejected
+input batch can be retried with valid data.
+
+This is an input-validation guarantee. A database failure after writing must
+escape the current transaction scope. To continue outer work, establish an
+explicit inner scope before the operation, and catch outside that inner scope.
+Do not catch and ignore database errors inside the failed scope. Unknown commit
+outcomes still require existing reconciliation rules, not blind retries.
+
+## Active Agent execution updates
+
+InMemory now enforces the same active-owner exclusion during update as SQL.
+Trying to reactivate an Agent execution while a different execution for that
+Agent is active raises `Storage::ActiveExecutionConflictError` at the raw boundary
+and does not advance the revision or replace the record. The domain-facing
+repository retains its existing error translation. Team terminal execution
+reactivation remains forbidden.
+
+## Backend authors and remaining migration
+
+Run `phronomy/testing/persistence_contract`, including `a Persistence backend`,
+against each real backend. Explicit nested calls must preserve block results,
+exception identity, inner rollback and outer atomicity across repositories.
+SQL integrations additionally exercise physical constraints, concurrent writers,
+connection failure and durable reload; mocked connection responses do not cover
+those guarantees.
+
+Records/Streams/Blobs, failed-view enforcement and explicit non-local block-exit
+handling are later S2b work. Do not use `return` / `break` / `throw` as portable
+commit controls in the current SPI. See [ADR-057](../decisions/057-storage-transaction-boundaries.md).

data/docs/migrations/vector-async-clients.md

@@ -0,0 +1,110 @@
+# VectorStore and Embeddings: asynchronous client migration
+
+This unreleased P3 change moves the Beta asynchronous API to explicit clients.
+It changes the receiver used by callers. Backend authors continue to implement
+only synchronous methods; method arguments and results remain unchanged.
+
+| Previous receiver | New receiver |
+|---|---|
+| `store.add_async(...)` | `async_store.add_async(...)` |
+| `store.search_async(...)` | `async_store.search_async(...)` |
+| `store.remove_async(...)` | `async_store.remove_async(...)` |
+| `store.clear_async(...)` | `async_store.clear_async(...)` |
+| `embeddings.embed_async(text, token, timeout: seconds)` | `async_embeddings.embed_async(text, token, timeout: seconds)` |
+
+Construct the clients with `VectorStore::AsyncClient.new(backend: store)` and
+`VectorStore::Embeddings::AsyncClient.new(adapter: embeddings)`. The embedding
+cancellation token stays positional. Synchronous `add`, `search`, `remove`,
+`clear`, `size` and `embed` stay on the existing backend objects.
+
+`VectorStore::AsyncBackend` and inherited async methods are removed. There is
+no include/prepend, backend `.async` factory, or automatic compatibility shim.
+Update the receiver wherever an application previously called an async method.
+The clients are public Beta API; their `async/` directories do not introduce
+`Phronomy::Async` or feature-specific `Async` namespaces.
+
+## Complete offline example
+
+This example uses an intentionally simple local embedding adapter to demonstrate
+the API. It needs no provider key, network request, or database server. The
+implementation supplies only `embed`; Phronomy supplies asynchronous execution.
+Run it against a checkout containing P3, not the previously released 0.27.0 gem.
+
+```ruby runnable
+require "phronomy"
+require "json"
+
+class KeywordEmbeddings < Phronomy::Embeddings::Base
+  protected
+
+  def perform_embed(text, cancellation_token = nil)
+    cancellation_token&.raise_if_cancelled!
+    words = text.downcase
+    [words.include?("ruby") ? 1.0 : 0.0,
+      words.include?("python") ? 1.0 : 0.0]
+  end
+end
+
+begin
+  store = Phronomy::VectorStore::InMemory.new(dimension: 2)
+  embeddings = KeywordEmbeddings.new
+  async_store = Phronomy::VectorStore::AsyncClient.new(backend: store)
+  async_embeddings = Phronomy::Embeddings::AsyncClient.new(adapter: embeddings)
+
+  {"ruby" => "Ruby guide", "python" => "Python guide"}.each do |id, title|
+    async_embeddings.embed_async(title).flat_map do |vector|
+      async_store.add_async(id: id, embedding: vector, metadata: {title: title})
+    end.wait_result
+  end
+
+  matches = async_embeddings.embed_async("Ruby").flat_map do |vector|
+    async_store.search_async(query_embedding: vector, k: 1)
+  end.wait_result
+  puts JSON.generate(matches)
+ensure
+  Phronomy.reset_runtime!
+end
+```
+
+The result contains the `ruby` document. The application waits only outside
+worker callbacks; `flat_map` returns the next TaskResult without blocking a
+worker on another operation. `Tools::VectorSearch` continues to call synchronous
+`embed` and `search` inside its existing execution boundary, so this migration
+does not add nested offload work to that Tool.
+
+## Execution and cancellation
+
+Both clients accept optional `pool:` injection. Usually omit it. Construction
+starts no Runtime, and each operation resolves the current default pool. An
+explicit pool remains caller-owned across Runtime resets; it must provide the
+existing OffloadPool submit contract, including its timer service for timeouts.
+
+Queue admission uses `on_full: :raise`. A full or stopped pool raises immediately;
+an accepted operation returns the original TaskResult. Timeout includes queue
+wait. Cancellation or timeout before worker pickup skips the operation; after
+execution starts it settles the result without forcibly stopping I/O or undoing
+backend effects. These are the existing OffloadPool semantics.
+
+Synchronous RBS signatures accept the structural `_CancellationSignal`
+interface, which requires only `raise_if_cancelled!`. Existing
+`Concurrency::CancellationToken` objects still satisfy it. Async RBS signatures
+live in `sig/phronomy/vector_store/async/async_client.rbs` and
+`sig/phronomy/embeddings/async/async_client.rbs` and retain the full CancellationToken
+and TaskResult contract needed by the execution pool. Do not substitute a
+minimal synchronous signal for an async operation's full token.
+
+## Source ownership
+
+The public class names below are unchanged. Their implementation files move:
+
+| Public class | New path under `lib/phronomy/` |
+|---|---|
+| `VectorStore::InMemory` | `vector_store/backends/in_memory.rb` |
+| `VectorStore::Pgvector` | `vector_store/backends/pgvector.rb` |
+| `VectorStore::RedisSearch` | `vector_store/backends/redis_search.rb` |
+| `VectorStore::Embeddings::RubyLLMEmbeddings` | `embeddings/backends/ruby_llm_embeddings.rb` |
+
+Use `require "phronomy"` and the public constants. Direct partial-load paths are
+not the public loading contract; no require shim is kept in the contract
+directories. Backend Contracts have no execution or implementation dependency.
+P4 Storage/ContentStore work remains pending.