phronomy 0.16.0 → 0.18.0

data/docs/runtime-and-concurrency.md

@@ -0,0 +1,258 @@
+# Runtime and concurrency
+
+Phronomy uses an **EventLoop / FSMSession first** architecture for framework
+lifecycle coordination. `FSMSession` is the framework finite-state-machine session
+used to represent explicit lifecycle state and events. A `Task` is a completion
+handle, not an execution backend. Synchronous work that must stay off EventLoop is isolated in the bounded
+`OffloadPool`.
+
+For the design rationale, see Architecture Decision Record (ADR)
+[ADR-010: EventLoop / FSMSession First Concurrency](decisions/010-cooperative-first-concurrency.md).
+
+## Runtime model
+
+```text
+Runtime
+├─ EventLoop (one control-plane operating-system Thread)
+│  └─ FSMSession
+│     ├─ Agent
+│     ├─ Workflow
+│     ├─ ToolInvocation
+│     └─ MultiAgent fan-out
+├─ OffloadPool (bounded operating-system Threads)
+│  ├─ blocking input/output (I/O)
+│  ├─ central-processing-unit (CPU)-bound synchronous work
+│  └─ other long synchronous work
+├─ named OffloadPools
+└─ EventLoop-driven timers
+
+Task = completion handle
+```
+
+The framework does not allocate one operating-system Thread per logical Agent/Workflow/Tool
+lifecycle. Logical waits remain explicit states plus later EventLoop events.
+
+## Tool execution modes
+
+Phronomy exposes two execution modes for capabilities:
+
+- `:cooperative` — short EventLoop-safe work, or a specialized asynchronous Tool
+  that starts another Phronomy lifecycle and returns immediately.
+- `:offloaded` — synchronous work that must not run to completion on EventLoop.
+
+Phronomy does not classify application work into framework-level I/O/CPU/process
+execution modes. That workload classification and capacity planning belong to the
+application.
+
+A CPU-heavy operation may therefore use `:offloaded`, but thread offload does not
+remove CRuby Global VM Lock contention or physical CPU contention.
+
+## Logical waiting versus offload
+
+Do not offload a logical wait merely to make it asynchronous.
+
+Prohibited shape:
+
+```text
+OffloadPool worker
+  → child_agent.invoke_async
+  → wait_result
+```
+
+Correct shape:
+
+```text
+parent FSMSession
+  → start child lifecycle
+  → return immediately
+  → child settles
+  → post parent EventLoop event
+```
+
+This distinction prevents worker-slot starvation when many logical lifecycles are
+waiting at the same time.
+
+## Sync versus async application APIs
+
+| Calling context | Recommended approach |
+|---|---|
+| Top-level application code | `agent.invoke(...)` when blocking the caller is acceptable |
+| Top-level explicit async | `task = agent.invoke_async(...)`; optionally `task.wait_result` outside EventLoop |
+| Workflow entry/transition action | Start async work and continue through `Workflow#signal` |
+| EventLoop callback | Never block waiting for a Task that requires EventLoop progress |
+| Top-level streaming | `agent.stream(...)` |
+| Non-blocking streaming | `agent.stream_async(...)` |
+| Approval from EventLoop callback | `approve_async` |
+
+Blocking synchronous APIs reject EventLoop re-entry with
+`Phronomy::EventLoopReentrancyError` when waiting would stall the same EventLoop
+needed for progress.
+
+## Task
+
+`Phronomy::Task` is thread-free. It represents one terminal result:
+
+- completed value,
+- failure,
+- cancellation.
+
+`Task#wait_result(timeout:)` is a bridge for external synchronous callers. It is
+not the framework continuation mechanism.
+
+`Task#map` is application-level composition. A transformation exception settles
+the mapped Task as failed. This is different from independent notification
+callbacks, described below.
+
+## OffloadPool
+
+`OffloadPool` is a bounded worker pool for synchronous work that must not execute
+on EventLoop.
+
+Its guarantees include:
+
+- bounded worker count,
+- bounded queue depth,
+- queue backpressure,
+- operation-wide submit timeout/cancellation settlement,
+- abandoned-worker accounting,
+- runtime metrics,
+- shutdown/drain behavior.
+
+It does not guarantee CPU/I/O fairness or CPU isolation. Applications that need
+resource isolation can create named Runtime pools.
+
+### EventLoop queue admission
+
+Framework-owned EventLoop-origin submissions must not wait for a free worker
+queue slot. They use non-blocking admission (`on_full: :raise`) and route
+`BackpressureError` through the ordinary FSM/completion path.
+
+External management threads may choose a blocking admission policy when blocking
+the caller is acceptable.
+
+## PendingOperation and blocking_wait
+
+`OffloadPool#submit` returns a private `PendingOperation` immediately after queue
+admission.
+
+`PendingOperation#blocking_wait(timeout:)` is intentionally a **low-level
+synchronous bridge** for non-EventLoop callers such as tests and diagnostics.
+The timeout belongs only to that waiter:
+
+- it raises `TimeoutError` to that calling thread,
+- it does not settle the PendingOperation,
+- it does not cancel the submitted operation,
+- it does not mark the operation abandoned.
+
+There is no waiter-local `cancellation_token:` argument. Operation-wide
+cancellation belongs exclusively to `OffloadPool#submit(cancellation_token:)`.
+
+## Submit timeout and cancellation
+
+Submit-time timeout and submit cancellation settle the caller-facing operation.
+They do **not** asynchronously interrupt an already-running synchronous worker.
+
+### Before worker start
+
+If timeout/cancellation wins before execution starts:
+
+- the PendingOperation settles,
+- the submitted block does not run,
+- the operation is not counted as abandoned.
+
+### After worker start
+
+If timeout/cancellation wins after execution starts:
+
+- the PendingOperation settles immediately,
+- the operation is marked abandoned,
+- the worker is allowed to continue until its synchronous call returns,
+- the eventual worker result is discarded.
+
+Phronomy does not use `Thread#raise` to inject an exception into the worker.
+Application/library code that needs hard or transport-level deadlines should use
+its native timeout or, in the future, an appropriate process-isolation mechanism.
+
+### CancellationToken deadlines
+
+`CancellationToken.timeout_after(seconds)` uses a monotonic deadline.
+`cancelled?` becomes true after that deadline, but the token itself does not own a
+Thread.
+
+Components requiring callback delivery for a monotonic deadline must promote the
+deadline to explicit `cancel!` through the Runtime timer queue. OffloadPool does
+this for its submit cancellation token.
+
+`CancellationScope#deadline_in` is appropriate when the application needs a
+Runtime-timer-backed cancellation scope whose `on_cancel` subscribers are fired
+on expiry.
+
+## Independent notification callbacks
+
+Independent notification fan-out is fault-isolated.
+
+The rule applies to:
+
+- `CancellationToken#on_cancel`,
+- `Task#on_complete`,
+- `PendingOperation#on_complete`,
+- EventLoop timer callbacks.
+
+A `StandardError` from one independent subscriber is logged and does not suppress
+later subscribers.
+
+This is deliberately different from a continuation/transform such as
+`Task#map`: a transform exception is the outcome of the derived operation and is
+therefore propagated into that derived Task.
+
+Callback execution thread is not guaranteed for low-level completion handles.
+Callbacks must therefore be thread-safe and should complete quickly. Framework
+lifecycle code normally turns completion into an explicit EventLoop event rather
+than mutating unrelated logical state from a worker thread.
+
+## Abandoned-worker metrics
+
+Two metrics answer different operational questions:
+
+- `offload_pool_abandoned_total` — cumulative count of operations that became
+  abandoned after worker execution had started.
+- `offload_pool_abandoned_active` — current number of abandoned operations whose
+  synchronous workers still occupy pool capacity.
+
+Example:
+
+```text
+offload_pool_size             = 10
+offload_pool_active           = 10
+offload_pool_abandoned_active = 8
+offload_pool_abandoned_total  = 523
+offload_pool_queue_length     = 40
+```
+
+This means 10 workers are currently executing, 8 of them are doing work whose
+caller-facing result has already been abandoned, 523 abandonment events have
+occurred since process start, and 40 operations are queued.
+
+`Phronomy::Diagnostics.dump` exposes the same distinction for point-in-time
+troubleshooting.
+
+## EventLoop metrics
+
+`Phronomy::Metrics.snapshot` also reports EventLoop queue depth and lag values.
+Use these to distinguish worker saturation from EventLoop backlog/latency.
+
+## Shutdown
+
+`Runtime#shutdown` is terminal for that Runtime. It drains/terminates the
+Runtime-owned EventLoop, then closes pools and timers according to the Runtime
+shutdown contract.
+
+`Phronomy.reset_runtime!` exists primarily for test isolation and performs a real
+Runtime shutdown before resetting configuration.
+
+## Further design records
+
+The `docs/decisions/` directory contains the historical and current Architecture
+Decision Records (ADRs). When an older ADR is superseded, use the superseding
+section/current ADR as the active design contract and keep the earlier document
+as historical rationale.

data/examples/workflows/generic_task_event_mapping.rb

@@ -19,18 +19,25 @@ class ImportContext
   end
 end
 
+# Example application service for synchronous work that must stay off EventLoop.
+# The worker Thread belongs to Phronomy's bounded OffloadPool; the Workflow
+# itself never blocks EventLoop waiting for the operation.
+def start_import_async
+  Phronomy::Runtime.instance.offload.submit(on_full: :raise) do
+    # Replace with blocking file/DB/network work or another long synchronous call.
+    100
+  end
+end
+
 workflow = nil
 
 workflow = Phronomy::Workflow.define(ImportContext) do
   initial :importing
 
   state :importing, action: ->(context) {
-    task = Phronomy::Runtime.instance.spawn do
-      # Replace with application-owned asynchronous work.
-      100
-    end
+    operation = start_import_async
 
-    task.on_complete do |record_count, error|
+    operation.on_complete do |record_count, error|
       workflow.signal(
         thread_id: context.thread_id,
         event: error ? :import_failed : :import_completed,
@@ -41,7 +48,8 @@ workflow = Phronomy::Workflow.define(ImportContext) do
       )
     end
 
-    # Do not return task. The state is active after this synchronous entry ends.
+    # Do not return the completion handle. The state remains active after this
+    # synchronous action returns, and later completion arrives as an FSM event.
     context
   }
 

data/lib/phronomy/agent/agent_invocation.rb

@@ -29,7 +29,6 @@ module Phronomy
       CALLBACK_FAILED_EVENTS = %i[application_callback_failed].freeze
 
       attr_accessor :input,
-        :messages,
         :chat,
         :output,
         :usage,
@@ -59,18 +58,15 @@ module Phronomy
       def initialize(
         agent:,
         input:,
-        messages:,
         config:,
         approval_policy: nil,
         approval_listener: nil,
         event_listener: nil,
-        stream_listener: nil,
         mode: nil,
         id: nil
       )
         @agent = agent
         @input = input
-        @messages = Array(messages)
         @config = config
         @thread_id = config[:thread_id]
         @id = (id || config[:agent_invocation_id] || SecureRandom.uuid).to_s
@@ -80,8 +76,8 @@ module Phronomy
         end
         @approval_policy = invocation_policy || approval_policy
         @approval_listener = approval_listener
-        @event_listener = event_listener || stream_listener
-        @mode = (mode || (stream_listener ? :stream : :invoke)).to_sym
+        @event_listener = event_listener
+        @mode = (mode || :invoke).to_sym
 
         @chat = nil
         @output = nil
@@ -104,14 +100,6 @@ module Phronomy
         @tool_batch_llm_call_id = nil
       end
 
-      def stream_listener
-        @event_listener
-      end
-
-      def stream_listener=(listener)
-        @event_listener = listener
-      end
-
       def streaming?
         @mode == :stream
       end
@@ -180,29 +168,9 @@ module Phronomy
         true
       end
 
-      def apply_fsm_action_result(result)
-        event_type =
-          if result.respond_to?(:error) &&
-              result.error &&
-              !result.error.is_a?(ToolCallIntercepted)
-            :llm_failed
-          else
-            :llm_completed
-          end
-        handle_fsm_event(
-          Phronomy::Event.new(
-            type: event_type,
-            target_id: @id,
-            payload: result
-          )
-        )
-        self
-      end
-
       def accept_tool_calls!(tool_calls, llm_call_id: nil)
         @user_message_sent = true
         @pending_tool_calls = Array(tool_calls)
-        @messages = @chat.messages
         @tool_batch_llm_call_id = (llm_call_id || @current_llm_call_id)&.to_s
         @current_llm_call_id = nil
         @pending_tool_calls.each do |tool_call|
@@ -227,7 +195,6 @@ module Phronomy
         @user_message_sent = true
         @output = response.content
         @usage = Phronomy::TokenUsage.from_tokens(response.tokens)
-        @messages = @chat.messages
         @pending_tool_calls = []
         @current_llm_call_id = nil
         self
@@ -298,7 +265,6 @@ module Phronomy
             )
           )
         end
-        @messages = @chat.messages
         clear_tool_batch!
         self
       end