phronomy 0.18.0 → 0.19.0

data/docs/runtime-and-concurrency.md

@@ -8,6 +8,8 @@ handle, not an execution backend. Synchronous work that must stay off EventLoop
 
 For the design rationale, see Architecture Decision Record (ADR)
 [ADR-010: EventLoop / FSMSession First Concurrency](decisions/010-cooperative-first-concurrency.md).
+Durable-state ownership is defined by
+[ADR-014: Unified Persistence and Durable-State Ownership](decisions/014-unified-persistence-durable-state.md).
 
 ## Runtime model
 
@@ -19,6 +21,7 @@ Runtime
 │     ├─ Workflow
 │     ├─ ToolInvocation
 │     └─ MultiAgent fan-out
+├─ process-local Agent ActivationRegistry
 ├─ OffloadPool (bounded operating-system Threads)
 │  ├─ blocking input/output (I/O)
 │  ├─ central-processing-unit (CPU)-bound synchronous work
@@ -32,6 +35,67 @@ Task = completion handle
 The framework does not allocate one operating-system Thread per logical Agent/Workflow/Tool
 lifecycle. Logical waits remain explicit states plus later EventLoop events.
 
+## Live state and durable state
+
+A live Agent or Workflow owns its current logical state. `Persistence` is the
+last committed durable representation and recovery source; it is not reloaded at
+every semantic boundary.
+
+For Agents, the live owner consists of the Agent instance plus its current
+`AgentRoot`, hydrated Journal view, and `AgentExecutionActivation`. Mutable
+Agent/Execution/Journal state is not automatically reloaded before every LLM or
+Tool step. Durable writes use optimistic revision/position guardrails; an
+external writer that advances the durable base causes `Persistence::ConflictError`
+rather than automatic reload or merge.
+
+For Workflows, the current `WorkflowContext` and FSMSession own the active
+logical state. A durable Workflow hydrates once at invocation/resume and saves
+at the halted/terminal boundary.
+
+Content-addressed `Persistence#contents` values are immutable. Fetching a known
+content reference is value materialization rather than mutable state refresh.
+
+## Workflow identities and durable admission
+
+Workflow execution keeps three identities separate:
+
+```text
+session_id
+  application session/correlation identity, for example a Rails session
+
+thread_id
+  durable Workflow identity and Persistence#workflow_states key
+
+fsm_session_id
+  one Runtime FSMSession execution identity; generated again for each
+  invoke/resume operation
+```
+
+The existing application `session_id` is tracing/caller metadata and is not used
+for durable Workflow ownership. EventLoop registers active FSMs by
+`fsm_session_id`; durable Workflow admission is a separate owner map:
+
+```text
+thread_id -> owner_fsm_session_id
+```
+
+The owner is acquired before `workflow_states.load(thread_id)` and remains held
+until the halted/terminal `workflow_states.save(...)` completes. Only the current
+owner may release the admission. `fsm_session_id` is Runtime-only metadata and is
+not stored in Workflow fields or durable snapshots.
+
+The admission map belongs to one Runtime and is process-local. It prevents two
+executions with the same durable `thread_id` from being admitted concurrently
+inside that Runtime, but it is not shared across Ruby processes, containers, or
+service replicas. Separate processes may therefore execute the same `thread_id`
+concurrently unless the application adds distributed coordination.
+
+`workflow_states` optimistic revisions detect stale terminal commits across those
+processes. They do not prevent duplicate execution from starting and cannot undo
+external side effects that both executions already performed before one save
+loses the revision race. CAS is stale/double-commit detection, not a distributed
+execution lock or duplicate-side-effect prevention mechanism.
+
 ## Tool execution modes
 
 Phronomy exposes two execution modes for capabilities:
@@ -72,6 +136,28 @@ parent FSMSession
 This distinction prevents worker-slot starvation when many logical lifecycles are
 waiting at the same time.
 
+## Persistence I/O boundary
+
+`Persistence` repositories expose synchronous operations. Framework lifecycle
+code must not perform potentially blocking durable reads/writes on EventLoop.
+Agent preparation/commit and Workflow hydrate/save operations are submitted to
+`OffloadPool`; completion continues through completion callbacks or explicit
+EventLoop events.
+
+A durable barrier may pause one logical lifecycle without blocking EventLoop.
+The next Agent provider call does not start until the corresponding Manifest and
+logical execution snapshot commit succeeds. A persistence failure or optimistic
+conflict fails that step rather than continuing with stale state.
+
+Approval wait is not a hydration boundary. The same live Agent instance,
+Activation, and AgentInvocation remain the owner and are resumed after approval.
+Approval itself remains an Agent-instance operation. An application that only has
+an `execution_id` first resolves the current process's owner with
+`Phronomy::Agent::Base.live_for_execution(execution_id)` or the expected concrete
+Agent class, then calls `agent.approve(...)` or `agent.approve_async(...)`.
+`live_for_execution` consults the Runtime-local ActivationRegistry and does not
+load a replacement Agent from Persistence.
+
 ## Sync versus async application APIs
 
 | Calling context | Recommended approach |
@@ -82,11 +168,12 @@ waiting at the same time.
 | EventLoop callback | Never block waiting for a Task that requires EventLoop progress |
 | Top-level streaming | `agent.stream(...)` |
 | Non-blocking streaming | `agent.stream_async(...)` |
-| Approval from EventLoop callback | `approve_async` |
+| Approval from EventLoop callback | Resolve with `live_for_execution`, call `agent.approve_async(...)`, and return immediately |
 
 Blocking synchronous APIs reject EventLoop re-entry with
 `Phronomy::EventLoopReentrancyError` when waiting would stall the same EventLoop
-needed for progress.
+needed for progress. `live_for_execution` itself only performs a Runtime-local
+registry lookup and does not wait for Task progress.
 
 ## Task
 
@@ -247,6 +334,10 @@ Use these to distinguish worker saturation from EventLoop backlog/latency.
 Runtime-owned EventLoop, then closes pools and timers according to the Runtime
 shutdown contract.
 
+Workflow durable admission participates in EventLoop idleness: a Workflow whose
+FSMSession has ended but whose durable save is still in flight remains owned until
+that save completes and owner-aware admission is released.
+
 `Phronomy.reset_runtime!` exists primarily for test isolation and performs a real
 Runtime shutdown before resetting configuration.
 

data/lib/phronomy/agent/base.rb

@@ -252,24 +252,22 @@ module Phronomy
           new(agent_id: agent_id, persistence: persistence, load_existing: true)
         end
 
-        def approve(execution_id, approval_request_id:, persistence:, approved: true, config: {})
-          approve_async(
-            execution_id,
-            approval_request_id: approval_request_id,
-            approved: approved,
-            config: config,
-            persistence: persistence
-          ).wait_result
-        end
-
-        def approve_async(execution_id, approval_request_id:, persistence:, approved: true, config: {})
-          execution = persistence.executions.load(execution_id)
-          load(execution.agent_id, persistence: persistence).approve_async(
-            execution_id,
-            approval_request_id: approval_request_id,
-            approved: approved,
-            config: config
-          )
+        # Resolves the live Agent instance that currently owns execution_id in
+        # this process. This is a Runtime-local lookup, not durable rehydration.
+        def live_for_execution(execution_id)
+          activation = Phronomy::Runtime.instance.__agent_activations.fetch(execution_id)
+          unless activation
+            raise Phronomy::ExecutionRehydrationRequiredError,
+              "no live activation for #{execution_id}; durable rehydration is required"
+          end
+
+          agent = activation.agent
+          unless agent.is_a?(self)
+            raise ArgumentError,
+              "live activation #{execution_id} belongs to #{agent.class}, not #{self}"
+          end
+
+          agent
         end
       end
 
@@ -283,21 +281,33 @@ module Phronomy
         metadata: {},
         load_existing: false
       )
-        @persistence = persistence || Phronomy::Persistence::InMemory.new
+        @persistence = persistence ||
+          Phronomy.configuration.persistence ||
+          Phronomy::Persistence::InMemory.new
         @agent_id = agent_id.to_s.freeze
-        @root = if load_existing
-          loaded = @persistence.agents.load(@agent_id)
-          definition = self.class.agent_definition
-          unless loaded.agent_definition_id == definition.fetch(:id) &&
-              loaded.definition_version == definition.fetch(:version)
-            raise Phronomy::ConfigurationError,
-              "Agent definition mismatch for #{@agent_id}: stored " \
-              "#{loaded.agent_definition_id}@#{loaded.definition_version}, runtime " \
-              "#{definition.fetch(:id)}@#{definition.fetch(:version)}"
+
+        if load_existing
+          root = records = nil
+          @persistence.transaction do |tx|
+            root = tx.agents.load(@agent_id)
+            records = tx.journals.read(
+              @agent_id,
+              limit: root.journal_position
+            )
           end
-          loaded
+          validate_loaded_definition!(root)
+          @root = root
+          @_phronomy_journal_records = Array(records).dup.freeze
         else
-          create_agent_root!(context: context, knowledge: knowledge, metadata: metadata)
+          @root = create_agent_root!(
+            context: context,
+            knowledge: knowledge,
+            metadata: metadata
+          )
+          @_phronomy_journal_records = @persistence.journals.read(
+            @agent_id,
+            limit: @root.journal_position
+          ).dup.freeze
         end
       end
 
@@ -306,7 +316,10 @@ module Phronomy
       end
 
       def journal_projection
-        Agent::JournalProjection.new(persistence: persistence, agent_root: @root)
+        Agent::JournalProjection.new(
+          agent_root: @root,
+          records: _journal_records_snapshot
+        )
       end
 
       def transcript
@@ -334,13 +347,14 @@ module Phronomy
         end
       end
 
-      # Appends persistent Knowledge to the Agent Journal.
-      # Knowledge is an optional Context candidate; it is not part of #transcript.
+      # Appends persistent Knowledge to the Agent Journal. The live Agent owns the
+      # current logical root/Journal view; Persistence is advanced optimistically.
       def add_knowledge(content, metadata: {})
+        current = agent_root
         next_root = nil
+        appended = nil
         persistence.transaction do |tx|
           tx.executions.assert_idle!(agent_id)
-          current = tx.agents.load(agent_id)
           record = build_knowledge_record(
             tx: tx,
             root: current,
@@ -363,6 +377,7 @@ module Phronomy
             root: next_root
           )
         end
+        _append_journal_records(appended)
         @root = next_root
         self
       end
@@ -394,6 +409,7 @@ module Phronomy
           tx.agents.delete(agent_id)
         end
         @root = nil
+        @_phronomy_journal_records = [].freeze
         true
       end
 
@@ -404,6 +420,17 @@ module Phronomy
 
       private
 
+      def validate_loaded_definition!(loaded)
+        definition = self.class.agent_definition
+        return if loaded.agent_definition_id == definition.fetch(:id) &&
+          loaded.definition_version == definition.fetch(:version)
+
+        raise Phronomy::ConfigurationError,
+          "Agent definition mismatch for #{@agent_id}: stored " \
+          "#{loaded.agent_definition_id}@#{loaded.definition_version}, runtime " \
+          "#{definition.fetch(:id)}@#{definition.fetch(:version)}"
+      end
+
       def create_agent_root!(context:, knowledge:, metadata:)
         definition = self.class.agent_definition
         root = Agent::AgentRoot.create(
@@ -480,10 +507,11 @@ module Phronomy
       end
 
       def mutate_context!(kind, context_affecting: true)
+        current = agent_root
         next_root = nil
+        appended = nil
         persistence.transaction do |tx|
           tx.executions.assert_idle!(agent_id)
-          current = tx.agents.load(agent_id)
           record = Agent::JournalRecord.new(
             agent_id: agent_id,
             kind: kind,
@@ -502,8 +530,13 @@ module Phronomy
             context_revision: context_affecting ?
               yield_context_revision(current, proposed) : current.context_revision
           )
-          tx.agents.save(agent_id, expected_revision: current.agent_revision, root: next_root)
+          tx.agents.save(
+            agent_id,
+            expected_revision: current.agent_revision,
+            root: next_root
+          )
         end
+        _append_journal_records(appended)
         @root = next_root
       end
 
@@ -511,6 +544,18 @@ module Phronomy
         (proposed.context_revision == current.context_revision) ? current.context_revision + 1 : proposed.context_revision
       end
 
+      def _journal_records_snapshot
+        @_phronomy_journal_records || [].freeze
+      end
+
+      def _append_journal_records(records)
+        incoming = Array(records)
+        return _journal_records_snapshot if incoming.empty?
+
+        @_phronomy_journal_records =
+          (_journal_records_snapshot + incoming).freeze
+      end
+
       public
 
       def _add_handoff_tool(tool_class)

data/lib/phronomy/agent/context_assembler.rb

@@ -14,18 +14,20 @@ module Phronomy
         agent:,
         persistence:,
         policy: ContextPolicies::Default.new,
-        candidate_resolver: nil
+        candidate_resolver: nil,
+        journal_records: nil
       )
         @agent = agent
         @persistence = persistence
         @policy = policy
+        @journal_records = journal_records
         @candidate_resolver = candidate_resolver || ContextCandidateResolver.new(
           content_loader: method(:fetch_content)
         )
       end
 
       def build_initial(input:, agent_root:, execution:, config: {}, patch: LLMInputPatch.empty)
-        projection = JournalProjection.new(persistence: @persistence, agent_root: agent_root)
+        projection = journal_projection(agent_root)
         model_cfg = effective_model_config(config, patch)
         tool_set = ToolDefinitionSet.build(@agent)
         system_text = build_system_text(input)
@@ -65,7 +67,7 @@ module Phronomy
         config: {},
         patch: LLMInputPatch.empty
       )
-        projection = JournalProjection.new(persistence: @persistence, agent_root: agent_root)
+        projection = journal_projection(agent_root)
         model_cfg = effective_model_config(config, patch)
         hook_candidates = normalize_candidates(patch.segment_candidates)
         system_segments = base_manifest.segments.select do |segment|
@@ -96,6 +98,14 @@ module Phronomy
 
       private
 
+      def journal_projection(agent_root)
+        if @journal_records
+          JournalProjection.new(agent_root: agent_root, records: @journal_records)
+        else
+          JournalProjection.new(persistence: @persistence, agent_root: agent_root)
+        end
+      end
+
       def assemble(
         agent_root:,
         execution:,