phronomy 0.17.0 → 0.19.0

data/docs/decisions/008-orchestrator-uses-os-threads.md

@@ -2,51 +2,49 @@
 
 ## Status
 
-Accepted
-
-## Context
-
-`Agent::Orchestrator#dispatch_parallel` runs multiple sub-agent invocations
-concurrently. The Ruby concurrency primitives available are:
-
-1. **OS threads** (`Thread`): true OS-level threads, subject to Ruby's GVL for
-   CPU-bound work, but I/O-bound work (LLM API calls, tool HTTP requests)
-   releases the GVL and runs in parallel.
-2. **Ractors**: actor-model isolation, no shared mutable state between Ractors.
-   True parallel for CPU-bound work but requires strict object isolation.
-3. **Fibers / async**: cooperative concurrency via Fiber scheduler (e.g.,
-   `async` gem). Non-blocking I/O without multiple threads.
-4. **`concurrent-ruby` thread pool**: managed pool of OS threads.
-
-LLM calls and tool invocations are overwhelmingly I/O-bound (HTTP requests).
-Under the GVL, OS threads are sufficient to achieve meaningful parallelism for
-these workloads. Ractors require that all objects passed between them are
-shareable, which is incompatible with RubyLLM's mutable chat objects and
-`WorkflowContext` instances without significant refactoring.
-
-Fibers require an async-compatible HTTP library stack throughout (RubyLLM,
-Faraday, etc.), which is not guaranteed today.
-
-## Decision
-
-`dispatch_parallel` spawns one OS thread per task using Ruby's `Thread.new`.
-A `max_concurrency:` cap (default: unlimited) uses a `Mutex`-guarded counter to
-limit the number of simultaneously active threads when specified.
-
-## Consequences
-
-**Positive:**
-- Transparent parallelism for I/O-bound LLM/tool calls with no dependency
-  changes.
-- Compatible with all Ruby versions in the support matrix (3.2, 3.3, 3.4, head).
-- Simple to reason about: each task is an independent thread; results are
-  collected in input order.
-
-**Negative / Tradeoffs:**
-- CPU-bound work inside agents does not benefit from true parallelism due to
-  the GVL. (In practice, agents are almost always I/O-bound.)
-- Spawning many threads simultaneously (no `max_concurrency:`) can exhaust
-  system thread limits under high load. Users should set `max_concurrency:` for
-  large fan-outs.
-- Ractor-based isolation (if ever needed for security sandboxing) would require
-  significant API changes to `WorkflowContext` and RubyLLM integration.
+**Superseded by ADR-010 on 2026-08-13.**
+
+This file is retained as decision history. Its original implementation — one OS
+Thread per `dispatch_parallel` child — is no longer the active architecture.
+
+## Historical context
+
+The original Orchestrator implementation needed concurrent subagent execution
+before Phronomy had a common EventLoop/FSMSession control model. It therefore
+used `Thread.new` per child Agent, optionally bounded by `max_concurrency`.
+
+That choice provided straightforward parallelism for I/O-heavy Agent calls but
+also tied logical child concurrency directly to OS-thread count.
+
+## Superseding decision
+
+Phronomy now models fan-out as a parent `FanOutInvocation` / FSMSession:
+
+```text
+FanOut FSMSession
+    |
+    +-- start child Agent A asynchronously
+    +-- start child Agent B asynchronously
+    +-- start child Agent C asynchronously
+    |
+    +-- child completion events
+    |
+    +-- aggregate / fail / timeout / cancel
+```
+
+`max_concurrency` limits how many child Agent invocations are active. It does
+not create a corresponding set of Threads.
+
+Each child Agent runs through the common Agent FSMSession/EventLoop lifecycle.
+When a child performs synchronous work that must not run on EventLoop, only that
+synchronous operation is executed on the bounded `OffloadPool`.
+
+Therefore:
+
+- `dispatch_parallel` does not create raw Threads;
+- no `Runtime#spawn` or Task execution backend is involved;
+- child completion is delivered by EventLoop events;
+- result ordering, fail/skip policy, timeout, and cancellation belong to the
+  FanOut state machine.
+
+See ADR-010 for the current concurrency model and Thread boundary.

data/docs/decisions/009-state-store-abstraction.md

@@ -1,6 +1,6 @@
 # ADR 009: StateStore Abstraction for Workflow Persistence
 
-**Status**: Accepted  
+**Status**: Superseded by ADR-014  
 **Date**: 2025-01  
 **Issue**: [#250](https://github.com/Raizo-TCS/phronomy/issues/250)
 

data/docs/decisions/010-cooperative-first-concurrency.md

@@ -1,255 +1,175 @@
-# ADR-010: Cooperative-First Concurrency — BlockingAdapterPool for Uncontrollable I/O
+# ADR-010: EventLoop / FSMSession First Concurrency
 
 ## Status
 
-Accepted — updated 2026-05-25 to document current scheduler landscape and
-production-cooperative roadmap (Issues #331, #332, #334).
-
-> **Historical implementation note (2026-08-09):** The implementation-detail
-> inventory below is intentionally preserved as decision history. Compatibility
-> APIs subsequently removed from the active contract — including the
-> `runtime_backend :cooperative` alias and direct Runtime singleton replacement —
-> must not be read as current API guidance. The cooperative-first layering
-> principle remains the decision; use README/current source for current symbols.
+Accepted — revised for the OffloadPool execution boundary.
 
 ## Context
 
-Phronomy provides its own concurrency primitives:
-
-- **`Phronomy::Runtime`** — task scheduler; backend is configurable.
-  The current production default is `:thread` (`ThreadScheduler` / `ThreadBackend`).
-  See the backend landscape table below for all supported values.
-  *(Historical note: early versions used `:cooperative` as the default; it is now
-  a deprecated alias for `:immediate`.)*
-- **`Phronomy::EventLoop`** — singleton event dispatcher; drives the cooperative
-  task cycle.
-
-Early implementations occasionally forced `ThreadBackend` inside framework
-components (e.g., `Agent::FSM#spawn_agent_task` in commit `0cb8510`) to work
-around a perceived limitation: if an agent makes a blocking I/O call with the
-cooperative backend active, it blocks the calling thread.
-
-This approach was identified as wrong for two reasons:
-
-1. **Misplaced responsibility** — threading the *application's* I/O at the
-   *framework* layer prevents the `runtime_backend` configuration from being
-   honored and removes the app developer's control over the concurrency model.
-2. **Violation of layering** — the framework should not assume that its callers
-   are cooperative-scheduler-aware; it should provide primitives and let the
-   caller decide the backend.
+Phronomy must support many concurrently waiting Agent, Workflow, ToolInvocation,
+and MultiAgent lifecycles without allocating one OS Thread per logical task.
+At the same time, application and third-party code may contain synchronous work
+that must not execute on the single Runtime EventLoop thread.
 
-There is a separate, legitimate category of blocking I/O: third-party gems
-(RubyLLM, ActiveRecord, Redis, Faraday, etc.) that perform blocking system
-calls internally and cannot be made non-blocking from the Phronomy side.
-These must be handled differently from application-controlled I/O.
+Classifying arbitrary application work as I/O-bound, CPU-bound, or external
+process work is not a responsibility the framework can reliably infer. The
+architecturally relevant distinction for Phronomy is whether a unit of work can
+safely run to completion on EventLoop or must be moved off the control thread.
 
 ## Decision
 
-### Runtime backend landscape
-
-| Backend | Scheduler class | Role | Production use? |
-|---------|-----------------|------|----------------|
-| `:thread` | `ThreadScheduler` / `ThreadBackend` | **Default.** One OS thread per task. Provides true parallelism for blocking I/O workflows. | Yes |
-| `:immediate` | `FakeScheduler` / `ImmediateBackend` | **Unit test double.** Tasks run synchronously on the caller's thread; no extra threads. | Tests only |
-| `:fiber` | `DeterministicScheduler` / `FiberBackend` | **Experimental validation backend.** Runs tasks as Ruby Fibers to verify that framework components are truly non-blocking. Use in CI to catch inadvertent blocking; never use in production. Not a planned production replacement for `:thread`; preemptive scheduling will not be added. | No |
-
-Note: `:cooperative` is a deprecated alias for `:immediate` and must not be used in new code.
-
-### Rule 1 — Cooperative-first for core control
-
-The core control flow of every Phronomy component — **Agent, Workflow, Tool
-orchestration, Orchestrator, RAG pipeline, Streaming** — MUST be implemented
-using cooperative task / event / scheduler primitives:
-
+Framework lifecycle coordination uses one Runtime-owned EventLoop and explicit
+FSMSession state/events. **Task is a completion handle**, not an execution
+backend.
+
+Phronomy defines two Tool execution modes:
+
+- `:cooperative` — short, EventLoop-safe work, or a specialized asynchronous
+  implementation that starts another Phronomy lifecycle and immediately returns
+  a completion handle.
+- `:offloaded` — synchronous work that must not run on EventLoop. It executes in
+  the bounded `OffloadPool`.
+
+`OffloadPool` is the thread execution boundary for synchronous work that must be
+kept off EventLoop. It may contain blocking I/O, CPU-bound Ruby work, or other
+application-defined long-running synchronous calls.
+
+Workload classification such as I/O-bound versus CPU-bound is application-owned.
+Phronomy does not provide separate `:blocking_io`, `:cpu_bound`, or
+`:external_process` Tool execution modes.
+
+Logical waits are never offloaded merely to obtain concurrency. Waiting for an
+Agent, Workflow, ToolInvocation, approval, timer, or another Task is represented
+as FSMSession state plus a later EventLoop event.
+
+## Runtime model
+
+```text
+Runtime
+├─ EventLoop (one control-plane OS Thread)
+│  └─ FSMSession
+│     ├─ Agent
+│     ├─ Workflow
+│     ├─ ToolInvocation
+│     └─ MultiAgent fan-out
+├─ OffloadPool (bounded OS Threads)
+│  ├─ blocking I/O
+│  ├─ CPU-bound synchronous work
+│  └─ other long synchronous work
+├─ named OffloadPools
+└─ EventLoop-driven timers
+
+Task = completion handle
 ```
-Runtime.instance.spawn(name: "...") { ... }   # respects configured backend
-EventLoop.instance.post { ... }               # event dispatch
-```
-
-Do NOT force `ThreadBackend` or `ThreadScheduler` inside framework components
-unless Rule 2 explicitly applies.
-
-### Rule 2 — ThreadScheduler only for framework-owned infinite loops
-
-A framework component MAY use a dedicated `ThreadScheduler` (or `ThreadBackend`)
-if and only if **not** threading would unconditionally block the framework's own
-infinite loop.
-
-The only current example satisfying this criterion:
 
-```ruby
-# EventLoop#start — the run_loop is the framework's own infinite dispatch loop.
-# It MUST run in its own thread; the cooperative backend cannot yield itself.
-thread_runtime = Phronomy::Runtime.new(
-  scheduler: Phronomy::Runtime::ThreadScheduler.new
-)
-@task = thread_runtime.spawn(name: "event-loop") { run_loop }
+## Capacity and starvation
+
+The default OffloadPool is a shared bounded resource. CPU-heavy work can occupy
+slots that would otherwise be available to I/O, and slow I/O can do the same in
+reverse. Phronomy guarantees bounded worker count, bounded queue depth,
+backpressure, timeout/cancellation settlement, abandoned-worker accounting,
+metrics, and lifecycle shutdown. It does **not** guarantee work-class fairness,
+CPU isolation, core reservation, or CPU-bound speedup.
+
+Applications own capacity planning through `offload_pool_size` and
+`offload_queue_size`. Where isolation is required, applications may use
+`Runtime#pool(name, size:, queue_size:)` to create independent named OffloadPool
+resource domains.
+
+## EventLoop admission rule
+
+An EventLoop action must not block while waiting for a free OffloadPool queue
+slot. Framework-owned EventLoop-origin submissions therefore use non-blocking
+admission (`on_full: :raise`) and propagate `BackpressureError` through the
+normal FSM/completion path.
+
+External management threads may deliberately choose other admission policies
+when blocking the caller is acceptable.
+
+## Timeout and cancellation
+
+An OffloadPool submit-time timeout settles the caller-facing PendingOperation.
+It does not asynchronously interrupt a running worker Thread. If execution has
+already started, the operation becomes abandoned, the worker may continue until
+the submitted synchronous call returns, and that eventual worker result is
+discarded.
+
+The cancellation token passed to `OffloadPool#submit` follows the same
+caller-facing settlement model:
+
+- cancellation before worker execution prevents the submitted block from
+  starting;
+- cancellation after worker execution starts settles the caller-facing
+  PendingOperation immediately, marks the operation abandoned, and allows the
+  worker to continue until the synchronous call returns;
+- cancellation does not use `Thread#raise`;
+- application code may observe the same CancellationToken and terminate its own
+  synchronous operation cooperatively.
+
+A submit token with a monotonic deadline is connected to the Runtime timer queue,
+so deadline expiry becomes explicit cancellation without a polling Thread.
+
+`PendingOperation#blocking_wait(timeout:)` is a low-level synchronous bridge for
+non-EventLoop callers such as tests and diagnostics. Its timeout is waiter-local:
+it raises `TimeoutError` only to that caller and does not settle the
+PendingOperation, cancel the submitted operation, or mark it abandoned.
+PendingOperation does not define a waiter-local cancellation token;
+operation-wide cancellation is represented only by the token passed to
+`OffloadPool#submit`.
+
+Independent notification callbacks are fault-isolated. A `StandardError` from one
+`CancellationToken#on_cancel`, `Task#on_complete`, or
+`PendingOperation#on_complete` subscriber is logged and does not suppress later
+subscribers. This rule applies to notification fan-out; continuation or
+transformation callbacks still report their own failures through the operation
+they construct.
+
+`abandoned_count` and the exported `offload_pool_abandoned_total` metric are
+cumulative: they count operations whose caller-facing submit timeout or submit
+cancellation settled after worker execution had already started.
+`abandoned_active_count` and `offload_pool_abandoned_active` are current-state
+values: they count only abandoned operations whose synchronous worker is still
+occupying OffloadPool capacity.
+
+## CPU-bound work
+
+CPU-bound work is allowed through `:offloaded`. Thread offload protects the
+EventLoop from direct long synchronous execution but does not remove CRuby GVL
+contention or physical CPU contention. Those are explicitly outside the core
+OffloadPool guarantee.
+
+A future subprocess capability may provide process isolation, hard process
+termination, stdout/stderr capture, and CPU-worker separation. That future
+implementation belongs to the offload subsystem and does not reintroduce a
+Tool-level `:external_process` execution class.
+
+## Prohibited pattern
+
+```text
+OffloadPool worker
+  → child_agent.invoke_async
+  → wait_result
 ```
 
-**Legitimate ThreadScheduler exceptions (exhaustive list):**
-
-| Component | Reason |
-|---|---|
-| `EventLoop#start` | `run_loop` is the framework's own `while @running` infinite dispatch loop; running it on the shared scheduler would consume the scheduler, preventing all other tasks from running |
-
-**Handler constraints for EventLoop:**
-
-- Handler code runs **on the EventLoop thread**.  Do not perform blocking
-  operations (database, LLM, HTTP) directly inside a handler — this stalls all
-  session processing.
-- Do **not** call `Workflow#invoke` from within a handler.  That call blocks
-  until the EventLoop processes events, causing a deadlock.  Use the async
-  pattern: schedule work via `Runtime.instance.spawn` or `BlockingAdapterPool`,
-  then post results back with `EventLoop#post`.
-
-All other framework components — including FSM, orchestration, RAG, streaming —
-do NOT own an infinite loop and therefore MUST use `Runtime.instance.spawn`.
-
-### Rule 3 — BlockingAdapterPool for uncontrollable blocking I/O
+and equivalently for Workflow/ToolInvocation/Task lifecycles.
 
-Third-party gems whose internal I/O Phronomy cannot control (RubyLLM, ActiveRecord,
-Redis client, Faraday, etc.) MUST be isolated behind a bounded
-`BlockingAdapterPool`.
+That pattern converts a logical wait into worker-slot occupancy and can create
+pool starvation. The correct model is:
 
+```text
+parent FSMSession
+  → start child lifecycle
+  → return completion handle immediately
+  → child settles
+  → post parent EventLoop event
 ```
-           ┌─────────────────────────────────────┐
-           │  Cooperative EventLoop / Runtime     │
-           │  (FakeScheduler, ImmediateBackend)   │
-           │                                     │
-           │  Agent ──► BlockingAdapterPool ──► RubyLLM (HTTP)
-           │  RAG   ──► BlockingAdapterPool ──► ActiveRecord / Redis
-           └─────────────────────────────────────┘
-                              │
-                    ┌─────────┴─────────┐
-                    │  Thread pool      │  ← bounded (max_threads:)
-                    │  (OS threads)     │
-                    └───────────────────┘
-```
-
-Properties of `BlockingAdapterPool`:
-- Uses OS threads internally (they are the correct tool for I/O that releases
-  the GVL).
-- **Always bounded** — configurable `max_threads:` (no unbounded `Thread.new`).
-- Returns a `Phronomy::Task`-compatible future so the cooperative layer can
-  await results without blocking the EventLoop.
-- Is the **only** place in the framework that creates raw threads for I/O.
-
-> **Note:** `dispatch_parallel` (ADR-008) currently creates one thread per
-> sub-agent via `Thread.new`. This predates the `BlockingAdapterPool` concept
-> and is a candidate for future migration. Until that migration, it remains an
-> accepted exception per ADR-008.
-
-## Scheduler Landscape (as of 2026-05-25)
-
-Three scheduler backends currently exist:
-
-| Symbol / Class | Status | Purpose |
-|---|---|---|
-| `:thread` — `ThreadScheduler` / `ThreadBackend` | **Production default** | One OS thread per task; GVL-releasing I/O works transparently |
-| `:immediate` — `FakeScheduler` / `ImmediateBackend` | Test / CI | Synchronous; block runs to completion before `spawn` returns; no threads |
-| `:fiber` — `DeterministicScheduler` / `FiberBackend` | **Experimental validation backend** | Tick-based Fiber scheduler with virtual clock; enables deterministic concurrency tests without wall-clock timers. Named backend added in Issue #334. |
-
-### `:cooperative` deprecation
-
-`:cooperative` was a silent alias for `:immediate` (mapped to `FakeScheduler`).
-As of Issue #332, it now emits a `WARN`-level deprecation message and must not
-be used in new code. Issue #334 introduced `:fiber` as the named experimental
-validation backend; `:cooperative` is retained only for backwards compatibility.
-
-### DeterministicScheduler as a stepping stone
-
-`DeterministicScheduler` is the foundation for the long-term production
-cooperative runtime goal. It provides:
-
-- Ready-queue dispatch, virtual timer heap, scheduler signal API
-- `FiberBackend` wrapping tasks as Fibers
-- `tick` / `run_until_idle` / `advance(seconds)` for deterministic test control
-
-**Remaining gaps versus a full production cooperative runtime:**
-
-- No real-wall-clock timer integration (`TimerQueue` runs as a background thread — Issue #331)
-- No `BlockingAdapterPool` integration for LLM/network calls (Issue #280)
-- Scheduler must be driven manually (`tick`) rather than event-loop-driven
-
-The minimum delta to promote `DeterministicScheduler` to production use:
-
-1. Integrate `TimerQueue` into the scheduler `tick` cycle (eliminates timer thread — Issue #331)
-2. Implement `BlockingAdapterPool` and wire LLM/tool calls through it (Issue #280)
-3. ~~Expose as a named `runtime_backend` (`:fiber`)~~ — **done** (Issue #334)
-4. Add real-wall-clock integration (replace virtual time with `Process.clock_gettime`)
-
-Progress is tracked in Issues #331 and #280.
-
-### TimerQueue background thread
-
-`Runtime::TimerQueue` currently runs as a dedicated background OS thread
-(`Thread.new { run_loop }`). This is an accepted interim implementation — it is
-robust for production but violates the cooperative-first goal. The long-term
-target is to integrate timer firing into the scheduler's own `tick` cycle so
-that no separate timer thread is needed. Progress tracked in Issue #331.
 
 ## Consequences
 
-### Positive
-  truly means cooperative for all framework-controlled paths.
-- Thread creation is isolated and bounded; no accidental unbounded thread
-  proliferation from deep inside the framework.
-- The concurrency model is layered and explicit:
-  cooperative layer → adapter boundary → bounded thread pool → blocking gem.
-- Tests can reliably use `FakeScheduler` by default and opt in to `:thread`
-  only where true concurrency is required.
-- `DeterministicScheduler` + `FiberBackend` provide a deterministic test
-  foundation for cooperative concurrency without wall-clock timers.
-
-### Negative / Tradeoffs
-- `BlockingAdapterPool` is not yet implemented (as of ADR-010 acceptance).
-  Until it is, callers that need real blocking I/O must opt in to `:thread`
-  backend explicitly (e.g., in tests via `around` blocks).
-- `dispatch_parallel` remains on raw threads (ADR-008) until migrated.
-- `TimerQueue` runs as a background OS thread (interim; see Issue #331).
-- `DeterministicScheduler` backs the named `:fiber` backend (Issue #334
-  resolved). It remains experimental and not for production use; the remaining
-  steps toward a full production cooperative runtime are `TimerQueue`
-  integration (Issue #331) and `BlockingAdapterPool` completion (Issue #280).
-
-## Derived Checklist
-
-When writing or reviewing any Phronomy component, apply this checklist:
-
-| Question | Correct action |
-|---|---|
-| Does this component own a `while true` / `loop do` that blocks forever? | May use dedicated `ThreadScheduler` (Rule 2) |
-| Does this component call into app/agent code (LLM, tools)? | `Runtime.instance.spawn` — no ThreadBackend (Rule 1) |
-| Does this component call a blocking gem (RubyLLM, AR, Redis)? | Route through `BlockingAdapterPool` (Rule 3) |
-| Is this a test that needs real concurrency (sleep + cancel)? | `around` block with `c.runtime_backend = :thread` opt-in |
-| Anything else | Default: `Runtime.instance.spawn` |
-
-## Related ADRs and Issues
-
-- ADR-008: Orchestrator Uses OS Threads for Parallel Dispatch (predates this
-  ADR; `dispatch_parallel` is a future migration candidate)
-- Issue #331: TimerQueue scheduler integration (long-term — eliminate timer thread)
-- Issue #334: Promote DeterministicScheduler to production cooperative runtime
-- Issue #332: `:cooperative` alias deprecation (resolved 2026-05-25)
-- Issue #280: MCP transports behind BlockingAdapterPool (pending)
-
-## Non-goals
-
-The following capabilities are intentionally out of scope for this framework's
-concurrency layer:
-
-- **CPU-bound process pool** — A `ProcessPoolExecutor` equivalent is not part
-  of core framework by default. CPU-intensive tool work belongs at the
-  application layer (fork, Sidekiq, etc.). A separate ADR would be required
-  to introduce one.
-- **External process manager** — Spawning, monitoring, or restarting external
-  subprocesses is not currently a framework responsibility. A separate ADR
-  would be required.
-- **Preemptive scheduling** — The cooperative-first model is non-preemptive by
-  design. Introducing a preemptive scheduler or promoting `:fiber` to
-  production default is not currently planned; a separate ADR would be required.
-- **Additional ToolExecutor core execution routes** — Only `:cooperative` and
-  `:blocking_io` are core dispatch routes. `:cpu_bound` and `:external_process`
-  are compatibility aliases that fall back to `:blocking_io` with a warning.
-  Any genuinely new core execution route requires a new ADR.
-
+- There is one explicit framework continuation model: FSMSession + EventLoop.
+- `Task` stays thread-free and represents settlement only.
+- Tool execution classification becomes `:cooperative` / `:offloaded`.
+- CPU/I/O classification and resource sizing are application responsibilities.
+- Named pools remain available for application-managed resource isolation.
+- Production Fiber execution is not part of the architecture.
+- Raw production Threads remain confined to EventLoop and OffloadPool.