phronomy 0.27.0 → 0.28.0

data/docs/migrations/vector-async-clients.md

@@ -0,0 +1,110 @@
+# VectorStore and Embeddings: asynchronous client migration
+
+This unreleased P3 change moves the Beta asynchronous API to explicit clients.
+It changes the receiver used by callers. Backend authors continue to implement
+only synchronous methods; method arguments and results remain unchanged.
+
+| Previous receiver | New receiver |
+|---|---|
+| `store.add_async(...)` | `async_store.add_async(...)` |
+| `store.search_async(...)` | `async_store.search_async(...)` |
+| `store.remove_async(...)` | `async_store.remove_async(...)` |
+| `store.clear_async(...)` | `async_store.clear_async(...)` |
+| `embeddings.embed_async(text, token, timeout: seconds)` | `async_embeddings.embed_async(text, token, timeout: seconds)` |
+
+Construct the clients with `VectorStore::AsyncClient.new(backend: store)` and
+`VectorStore::Embeddings::AsyncClient.new(adapter: embeddings)`. The embedding
+cancellation token stays positional. Synchronous `add`, `search`, `remove`,
+`clear`, `size` and `embed` stay on the existing backend objects.
+
+`VectorStore::AsyncBackend` and inherited async methods are removed. There is
+no include/prepend, backend `.async` factory, or automatic compatibility shim.
+Update the receiver wherever an application previously called an async method.
+The clients are public Beta API; their `async/` directories do not introduce
+`Phronomy::Async` or feature-specific `Async` namespaces.
+
+## Complete offline example
+
+This example uses an intentionally simple local embedding adapter to demonstrate
+the API. It needs no provider key, network request, or database server. The
+implementation supplies only `embed`; Phronomy supplies asynchronous execution.
+Run it against a checkout containing P3, not the previously released 0.27.0 gem.
+
+```ruby runnable
+require "phronomy"
+require "json"
+
+class KeywordEmbeddings < Phronomy::Embeddings::Base
+  protected
+
+  def perform_embed(text, cancellation_token = nil)
+    cancellation_token&.raise_if_cancelled!
+    words = text.downcase
+    [words.include?("ruby") ? 1.0 : 0.0,
+      words.include?("python") ? 1.0 : 0.0]
+  end
+end
+
+begin
+  store = Phronomy::VectorStore::InMemory.new(dimension: 2)
+  embeddings = KeywordEmbeddings.new
+  async_store = Phronomy::VectorStore::AsyncClient.new(backend: store)
+  async_embeddings = Phronomy::Embeddings::AsyncClient.new(adapter: embeddings)
+
+  {"ruby" => "Ruby guide", "python" => "Python guide"}.each do |id, title|
+    async_embeddings.embed_async(title).flat_map do |vector|
+      async_store.add_async(id: id, embedding: vector, metadata: {title: title})
+    end.wait_result
+  end
+
+  matches = async_embeddings.embed_async("Ruby").flat_map do |vector|
+    async_store.search_async(query_embedding: vector, k: 1)
+  end.wait_result
+  puts JSON.generate(matches)
+ensure
+  Phronomy.reset_runtime!
+end
+```
+
+The result contains the `ruby` document. The application waits only outside
+worker callbacks; `flat_map` returns the next TaskResult without blocking a
+worker on another operation. `Tools::VectorSearch` continues to call synchronous
+`embed` and `search` inside its existing execution boundary, so this migration
+does not add nested offload work to that Tool.
+
+## Execution and cancellation
+
+Both clients accept optional `pool:` injection. Usually omit it. Construction
+starts no Runtime, and each operation resolves the current default pool. An
+explicit pool remains caller-owned across Runtime resets; it must provide the
+existing OffloadPool submit contract, including its timer service for timeouts.
+
+Queue admission uses `on_full: :raise`. A full or stopped pool raises immediately;
+an accepted operation returns the original TaskResult. Timeout includes queue
+wait. Cancellation or timeout before worker pickup skips the operation; after
+execution starts it settles the result without forcibly stopping I/O or undoing
+backend effects. These are the existing OffloadPool semantics.
+
+Synchronous RBS signatures accept the structural `_CancellationSignal`
+interface, which requires only `raise_if_cancelled!`. Existing
+`Concurrency::CancellationToken` objects still satisfy it. Async RBS signatures
+live in `sig/phronomy/vector_store/async/async_client.rbs` and
+`sig/phronomy/embeddings/async/async_client.rbs` and retain the full CancellationToken
+and TaskResult contract needed by the execution pool. Do not substitute a
+minimal synchronous signal for an async operation's full token.
+
+## Source ownership
+
+The public class names below are unchanged. Their implementation files move:
+
+| Public class | New path under `lib/phronomy/` |
+|---|---|
+| `VectorStore::InMemory` | `vector_store/backends/in_memory.rb` |
+| `VectorStore::Pgvector` | `vector_store/backends/pgvector.rb` |
+| `VectorStore::RedisSearch` | `vector_store/backends/redis_search.rb` |
+| `VectorStore::Embeddings::RubyLLMEmbeddings` | `embeddings/backends/ruby_llm_embeddings.rb` |
+
+Use `require "phronomy"` and the public constants. Direct partial-load paths are
+not the public loading contract; no require shim is kept in the contract
+directories. Backend Contracts have no execution or implementation dependency.
+P4 Storage/ContentStore work remains pending.

data/docs/persistence-backends.md

@@ -2,8 +2,11 @@
 
 [ADR-058](decisions/058-neutral-storage-primitives.md) defines the current
 extension contract. This is an intentional breaking replacement of the old
-fixed-repository Backend SPI. Application code keeps `Persistence.new(backend:)`,
-`Persistence.in_memory`, its eight domain repositories and result queries.
+fixed-repository Backend SPI. Application construction now uses
+`PersistenceComposition.build(backend:)` or `.in_memory`; pass its `agent`, `team`,
+or `workflow` component to that domain. `Persistence.new(backend:)` provides only
+the neutral coordinator. The old eight-repository facade and its queries were
+removed; see [r8 unit 3](architecture/r8-unit3.md).
 
 ## Ownership and composition
 
@@ -13,9 +16,10 @@ record types, active-execution policy, content digest algorithm or watermark API
 
 Agent, Team, Workflow and ContentStore declare their own resource schemas.
 `PersistenceComposition::StorageSchema` gathers those declarations and
-`PersistenceComposition::Repositories` assembles domain wrappers over a View.
-The Team and Workflow schema files live beside their features in nested loader
-roots; loading metadata does not load their runtime implementations.
+`PersistenceComposition` binds domain record adapters to one coordinator.
+Team and Workflow schemas live in their existing `persistence/` implementation
+directories. An isolated Agent uses `PersistenceComposition.agent` and does not
+construct Team or Workflow stores.
 `ContentStore::StoredContents` owns SHA-256 identity and digest verification.
 The Agent-owned `Watermark` composes guarded revision and stream-head conditions.
 
@@ -41,11 +45,10 @@ view.check!(guards: guards, conditions: conditions)
 
 The backend declares `spi_version: 2` and true values for `atomic_resources`,
 `record_cas`, `stream_cas`, `conditional_unique`, `guarded_checks` and
-`nested_savepoints`. Persistence validates these capabilities and the required
-resource declarations before exposing repositories. Old duck-typed backends are
-rejected with `UnsupportedBackendError`; there is no eight-slot compatibility
-adapter. Public Persistence capabilities retain `atomic_all`, `atomic_admission`
-and `optimistic_revision`, derived by composition from these primitives.
+`nested_savepoints`. Persistence validates backend capabilities; composition
+validates the required domain resource declarations before binding adapters.
+Unsupported backends raise `Persistence::UnsupportedBackendError`. There is no
+eight-slot compatibility adapter or duplicated facade capability API.
 
 A root handle routes to the current transaction on the same backend/thread.
 A bound view and all its handles expire on commit or rollback and reject use from
@@ -139,12 +142,24 @@ stale revisions and ordinary constraint conflicts remain `ConflictError`.
 meanings; `TransactionError` identifies invalid scope use. The removed
 `ActiveExecutionConflictError` and `Storage::Repositories` have no aliases.
 
-The following sections describe the retained **domain Persistence** surface.
+The following sections describe the **domain-owned record protocols**, used by
+domain implementations and backend conformance tests. They are not a shared
+application facade. Unqualified
+ConflictError, NotFoundError, SerializationError and UnsupportedBackendError in
+these domain sections refer to `Phronomy::Persistence` errors. Repository exits
+translate the four raw Storage categories, preserving `cause`, message and trace.
+Backend implementation code continues to raise `Phronomy::Storage` errors as
+specified above. Unknown/transport errors and Storage::TransactionError are not
+translated. See [ADR-061](decisions/061-persistence-failure-contracts.md) and the
+[application migration](migrations/persistence-failure-contracts.md).
 
 ## Contents repository
 
-`ContentStore::StoredContents < ContentStore::Base` supplies this domain surface
-over neutral Blobs, with text/JSON helpers and canonical content-ID calculation.
+`ContentStore::StoredContents < ContentStore::Base` supplies text/JSON helpers
+and canonical content-ID calculation over neutral Blobs. The private
+Persistence::ContentRepository adapts its primitive methods to the upper failure
+contract when accessed through `persistence.contents`. Direct StoredContents
+callers retain the raw Storage error surface.
 
 Required primitive surface:
 
@@ -159,7 +174,7 @@ Required semantics:
 - content is immutable and content-addressed;
 - writing identical bytes is idempotent and returns the same content ID;
 - `fetch` returns a binary `String` isolated from caller mutation;
-- a missing content ID raises `Storage::NotFoundError`;
+- a missing content ID through `persistence.contents` raises `Persistence::NotFoundError`;
 - one content ID must never resolve to different bytes; a digest-integrity
   violation raises `ContentStore::IntegrityError`.
 
@@ -361,8 +376,9 @@ Array of supported values
 Hash with String/Symbol keys and supported values
 ```
 
-If a value cannot be represented, raise `Storage::SerializationError` rather
-than silently converting it into a lossy form. JSON backends may return String
+If a value cannot be represented, the backend/codec raises
+`Storage::SerializationError`; the domain repository exposes
+`Persistence::SerializationError`. Do not silently convert it into a lossy form. JSON backends may return String
 keys after decoding; `WorkflowRunner` deliberately accepts String and Symbol keys
 and normalizes them when comparing durable snapshots.
 
@@ -391,16 +407,17 @@ differs, raise `ConflictError`. On success return `true`.
 The operation must not return a replacement AgentRoot or Journal. A mismatch is a
 conflict, not a request to reload/merge mutable state.
 
-When used inside `Persistence#transaction`, a SQL backend should perform the
+When used inside `Agent::Store#transaction`, a SQL backend should perform the
 watermark check in the same database transaction as the subsequent durable
 write.
 
 ## Transaction contract
 
-A transaction block may combine operations across all repositories:
+A domain transaction combines its own record operations. Multiple domains join
+one coordinator scope explicitly; each receives only its own record protocol:
 
 ```ruby
-persistence.transaction do |tx|
+stores.agent.transaction do |tx|
   tx.assert_agent_watermark!(...)
   content_ref = tx.contents.put_text("...")
   tx.journals.append(...)
@@ -493,7 +510,7 @@ require "phronomy"
 require "phronomy/testing/persistence_contract"
 
 RSpec.describe MyPersistenceBackend do
-  let(:persistence) { Phronomy::Persistence.new(backend: described_class.new(...)) }
+  let(:stores) { Phronomy::PersistenceComposition.build(backend: described_class.new(...)) }
 
   it_behaves_like "a persistence content store"
   it_behaves_like "an Agent repository"
@@ -507,7 +524,7 @@ RSpec.describe MyPersistenceBackend do
 end
 ```
 
-`Persistence.in_memory` is run through the same shipped contract source in
+`PersistenceComposition.in_memory` is run through the same shipped contract source in
 Phronomy CI. The files under `spec/support/shared_examples/` are compatibility
 require wrappers only; the authoritative shared-example implementations live
 under `lib/phronomy/testing/persistence_contract/` so the core suite and external

data/docs/runtime-and-concurrency.md

@@ -292,7 +292,7 @@ remain durable conflict defense rather than distributed ownership.
 ## Tool execution modes
 
 The default dispatch helper is the private
-`Phronomy::Agent::Context::Capability::ToolExecutor`, colocated with Capability
+`Phronomy::Tool::ToolExecutor`, colocated with Capability
 Base. Agent ToolInvocation supplies Runtime and admission policy for the
 standard path, and owns authorization and logical result handling. Custom
 `call_async` implementations keep the public Tool protocol. This ownership is
@@ -577,10 +577,35 @@ TaskResult reports only caller-facing settlement.
 ## EventLoop metrics
 
 `Phronomy::Metrics.snapshot` also reports EventLoop queue depth and lag values.
+
+Metrics and Diagnostics only observe resources already owned by the default
+Runtime. Calling `Metrics.snapshot`, `Diagnostics.snapshot` or `Diagnostics.dump`
+does not initialize a Runtime, pool, EventLoop or timer. Missing resources report
+zero values, including `offload_pool_size: 0` when no default pool exists. Once a
+pool exists, this field remains its configured capacity, not a live-worker count.
+Named pools are not aggregated into the default-pool metrics.
+
+Retained resources remain observable during and after shutdown, including
+incomplete cleanup. Observation does not reopen admission or change a shutdown
+result. After the default Runtime is reset, missing resources again report zero;
+the next snapshot uses any newly initialized default Runtime. Counters are sampled
+independently and do not constitute an atomic cross-resource snapshot.
 Use these to distinguish worker saturation from EventLoop backlog/latency.
 
 ## Shutdown
 
+Pool creation and registration close atomically before the shutdown resource set
+is captured. Accepted continuations can still acquire resources during Runtime's
+draining phase. Once resource shutdown begins, all pools close admission before
+any pool waits for its workers.
+
+The stop-phase monotonic deadline is shared by EventLoop and every pool join.
+A join timeout does not stop a synchronous worker. Runtime reports incomplete
+cleanup while any worker remains, even if that worker's TaskResult has already
+been cancelled or timed out. It does not forcefully interrupt worker threads.
+See [R8 unit19](architecture/r8-unit19.md) and its
+[migration notes](migrations/r8-unit19.md) for the shutdown budget and result rules.
+
 `Runtime#shutdown` is terminal for that Runtime. It drains/terminates the
 Runtime-owned EventLoop, then closes pools and timers according to the Runtime
 shutdown contract.

data/lib/phronomy/agent/admission.rb

@@ -0,0 +1,78 @@
+# frozen_string_literal: true
+
+require "securerandom"
+
+module Phronomy
+  module Agent
+    # The short durable phase of an Agent's already acquired runtime admission.
+    # A coordinator may join it to its reservation check; it cannot start an FSM,
+    # publish a provisional result or make the Agent interpret a parent's schema.
+    # @api public
+    class Admission
+      attr_reader :agent_id, :execution_id, :correlation
+
+      # Constructed by Agent execution, never restored from a durable record.
+      # @api private
+      def initialize(persistence:, root:, input:, config:, preparation_metadata:)
+        @persistence, @root, @input, @config = persistence, root, input, config
+        @preparation_metadata = preparation_metadata
+        @agent_id = root.agent_id
+        @execution_id = (config[:phronomy_reserved_execution_id] || SecureRandom.uuid).to_s.freeze
+        @correlation = Phronomy::Values::Immutable.copy(config[:phronomy_reservation])
+        @thread = Thread.current
+      end
+
+      # Own a standalone admission transaction.
+      # @api public
+      def accept
+        @persistence.coordinator.atomic { |scope| accept_in(scope) }
+        nil
+      end
+
+      # Participate without committing or publishing live state.
+      # @api public
+      def accept_in(scope)
+        check_thread!
+        raise Phronomy::ConfigurationError, "Agent admission is single-use" if @scope
+        @scope = scope
+        @persistence.participate(scope) do |records|
+          raise Phronomy::Error, "agent is closed: #{agent_id}" if @root.lifecycle_status == :closed
+          input_ref = records.contents.put_text(@input)
+          context_ref = records.contents.put_json(@config.fetch(:durable_context)) if @config.key?(:durable_context)
+          input_record = JournalRecord.new(agent_id: agent_id, kind: :input_received,
+            channel: :external, role: :user, content_ref: input_ref,
+            context_generation: @root.transcript_generation, context_candidate: false)
+          execution = AgentExecution.start(agent_root: @root, input_record: input_record,
+            execution_id: execution_id, metadata: {
+              "reservation" => correlation, "current_input_ref" => input_ref,
+              "durable_context_ref" => context_ref,
+              "execution_extension" => @config[:phronomy_execution_participant]&.binding&.to_h
+            }.merge(@preparation_metadata).compact)
+          input_record = JournalRecord.from_h(input_record.to_h.merge("execution_id" => execution_id))
+          execution = execution.with(execution_revision: 0, working_records: [input_record])
+          records.executions.create_active(execution)
+          next_root = @root.with(agent_revision: @root.agent_revision + 1, lifecycle_status: :active)
+          records.agents.save(agent_id, expected_revision: @root.agent_revision, root: next_root)
+          @result = [execution, next_root].freeze
+        end
+        nil
+      end
+
+      # Available to Agent execution only after the outermost successful response.
+      # @api private
+      def result
+        check_thread!
+        unless @result && @scope&.committed?
+          raise Phronomy::Persistence::TransactionError, "Agent admission has not committed"
+        end
+        @result
+      end
+
+      private
+
+      def check_thread!
+        raise Phronomy::Persistence::TransactionError, "Agent admission belongs to another execution context" unless @thread.equal?(Thread.current)
+      end
+    end
+  end
+end

data/lib/phronomy/agent/{lifecycle_contract/agent_already_exists_error.rb → agent_already_exists_error.rb}

@@ -1,6 +1,6 @@
 # frozen_string_literal: true
 
-require_relative "../../common/error"
+require_relative "../common/error"
 
 module Phronomy
   class AgentAlreadyExistsError < Phronomy::Error; end

data/lib/phronomy/agent/{lifecycle_contract/agent_busy_error.rb → agent_busy_error.rb}

@@ -1,6 +1,6 @@
 # frozen_string_literal: true
 
-require_relative "../../common/error"
+require_relative "../common/error"
 
 module Phronomy
   class AgentBusyError < Phronomy::Error; end

data/lib/phronomy/agent/{lifecycle_contract/agent_purged_error.rb → agent_purged_error.rb}

@@ -1,6 +1,6 @@
 # frozen_string_literal: true
 
-require_relative "../../common/error"
+require_relative "../common/error"
 
 module Phronomy
   class AgentPurgedError < Phronomy::Error; end

data/lib/phronomy/{recovery → agent/api}/execution_rehydration_required_error.rb

@@ -1,6 +1,6 @@
 # frozen_string_literal: true
 
-require_relative "../common/error"
+require_relative "../../common/error"
 
 module Phronomy
   class ExecutionRehydrationRequiredError < Phronomy::Error; end

data/lib/phronomy/agent/async_event_api.rb

@@ -12,6 +12,35 @@ module Phronomy
     # - approval notifications share the Agent listener as :approval_required;
     # - Recovery resolution uses resolve/resolve_async.
     module AsyncEventApi
+      def start_reserved_async(reservation:, input:, config: {})
+        unless reservation.is_a?(ReservedExecution) && reservation.agent_id == agent_id
+          raise ArgumentError, "reservation must belong to this Agent"
+        end
+        config = __invocation_config(_prepare_invocation_config(config, nil))
+        ExactExecution.start(agent: self, execution_id: reservation.execution_id, input: input,
+          config: config.merge(phronomy_reservation: reservation.correlation))
+      end
+
+      def resume_async(execution_id, config: {})
+        config = __invocation_config(_prepare_invocation_config(config, nil))
+        ExactExecution.start(agent: self, execution_id: execution_id, input: nil, config: config, resume_only: true)
+      end
+
+      def resume(execution_id, config: {})
+        _check_event_loop_reentrancy(:resume, :resume_async)
+        resume_async(execution_id, config: config).wait_result
+      end
+
+      def cancel_async(execution_id)
+        __assert_live_agent!
+        environment = __execution_environment
+        environment.submit(on_full: :raise) do
+          requested = persistence.request_cancellation(agent_id: agent_id, execution_id: execution_id)
+          ExecutionCancellation.signal(execution_id, agent_id, environment: environment) if requested
+          persistence.observe_execution(agent_id: agent_id, execution_id: execution_id)
+        end
+      end
+
       def invoke(
         input,
         config: {},
@@ -96,7 +125,7 @@ module Phronomy
 
       def approve_async(execution_id, approval_request_id:, approved: true, config: {})
         _reject_removed_generic_identity_keys!(config)
-        owner = Phronomy::Agent::ExecutionRegistry.existing_for(Phronomy::Runtime.instance)&.agent_execution_owner(execution_id)
+        owner = Phronomy::Agent::ExecutionRegistry.existing_current&.agent_execution_owner(execution_id)
         coordinator = if owner&.agent&.equal?(self)
           owner.coordinator
         else
@@ -115,8 +144,8 @@ module Phronomy
         expected_execution_revision:,
         subject:,
         outcome:,
-        result: Phronomy::Recovery::MISSING,
-        error: Phronomy::Recovery::MISSING
+        result: Phronomy::Agent::RecoveryRules::MISSING,
+        error: Phronomy::Agent::RecoveryRules::MISSING
       )
         _check_event_loop_reentrancy(:resolve, :resolve_async)
         resolve_async(
@@ -134,8 +163,8 @@ module Phronomy
         expected_execution_revision:,
         subject:,
         outcome:,
-        result: Phronomy::Recovery::MISSING,
-        error: Phronomy::Recovery::MISSING
+        result: Phronomy::Agent::RecoveryRules::MISSING,
+        error: Phronomy::Agent::RecoveryRules::MISSING
       )
         Phronomy::Agent::RecoveryCoordinator.new(self).resolve(
           execution_id,
@@ -149,26 +178,9 @@ module Phronomy
 
       private
 
-      # Framework-private execution-local event routing. This is intentionally
-      # not a public compatibility path for invoke_async(..., on_event:).
-      def __invoke_async_with_event_sink(
-        input,
-        on_event:, config: {},
-        invocation_context: nil
-      )
-        raise ArgumentError, "on_event is required" unless on_event
-
-        config = _prepare_invocation_config(config, invocation_context)
-        _start_agent_operation(
-          input,
-          config: config,
-          mode: :invoke,
-          listener: on_event
-        )
-      end
-
       def _start_agent_operation(input, config:, mode:, listener:)
-        config = _snapshot_durable_context(config)
+        config = __invocation_config(_snapshot_durable_context(config))
+        @_phronomy_execution_wiring = config.dup.freeze
         approval = _approval_configuration_snapshot(nil)
         execution_coordinator_for(config).start(
           input,
@@ -220,14 +232,7 @@ module Phronomy
         @execution_coordinator ||= Agent::ExecutionCoordinator.new(self)
       end
 
-      def execution_coordinator_for(config)
-        multi_agent = config.key?(:phronomy_handoff_bindings) ||
-          config.key?(:phronomy_handoff_context)
-        return execution_coordinator unless multi_agent
-
-        @multi_agent_execution_coordinator ||=
-          Phronomy::Agent::HandoffExecutionCoordinator.new(self)
-      end
+      def execution_coordinator_for(_config) = execution_coordinator
     end
   end
 end