phronomy 0.7.0 → 0.8.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: fbca82a7a23706719deda2e827af5a9b342c9b388d700929ca9eca19a531a2c9
-  data.tar.gz: b9727e2010acefc14738dbd5b71b5ea06b10f5ebd994a858dfbf1133e15ed003
+  metadata.gz: d4410424efcdcdf0ab529106ba2c872bddae9decc322995f37065f426255a05b
+  data.tar.gz: c9ae0dff7f184244b92fc91c585536f767aded5ff6ea8ecaafe86221863738e8
 SHA512:
-  metadata.gz: c33ee2c26a4b6e3d0470f4d95e30b04e9bc228cc87bdd807db1569c707888817f2d904f085822f91e94441e5c21e95ab348f0d7bc56b1ef87c72770ed4976e1d
-  data.tar.gz: 706148e7047ab570ca7d69f735f5767c2a983cf35312a70732723595ea80ed3b5f3efaaddd59d393f5680ec91c36c6ca3e01a70dcb671faa940ae9211df59b5a
+  metadata.gz: e7afa1749dc1431e27e225dfe7a8eafebb2e781c0e6a6ca6e0bdda9712c22b4b5b68d3a9897bc92b466026c698070045774d79a0197ad0463da3f81ff103b36c
+  data.tar.gz: be93a29c2b98b2069ef912815e847f0e0115de07bb435e36fbcb834433757fc72442d7c5db129c9f97dd891113ecf75c46606a6295b2e9f3357831c24246d974

data/.mutant.yml

@@ -12,10 +12,11 @@ includes:
 requires:
   - phronomy
 
-subjects:
-  - Phronomy::WorkflowContext
-  - Phronomy::WorkflowRunner
-  - Phronomy::Tool::Base
-  - Phronomy::Context::TokenBudget
-  - Phronomy::Context::TokenEstimator
-  - Phronomy::VectorStore::InMemory
+matcher:
+  subjects:
+    - Phronomy::WorkflowContext
+    - Phronomy::WorkflowRunner
+    - Phronomy::Tool::Base
+    - Phronomy::Context::TokenBudget
+    - Phronomy::Context::TokenEstimator
+    - Phronomy::VectorStore::InMemory

data/CHANGELOG.md

@@ -11,6 +11,104 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Added
 
+- **`Phronomy::Diagnostics` and `SchedulerReentrancyError`** (#278, #279):
+  `Phronomy::Diagnostics` exposes a snapshot of current scheduler state
+  (`pending_count`, `active_tasks`, `pool_utilization`, etc.) for debugging and
+  monitoring. `SchedulerReentrancyError` is raised when a scheduler operation is
+  attempted from within a scheduler callback, preventing deadlocks.
+  `Phronomy.configure { |c| c.scheduler_debug = true }` enables verbose scheduler
+  logging.
+
+- **`task_id` / `parent_task_id` on `InvocationContext`** (#277):
+  Every task spawned via `Task.spawn` now carries a `task_id` (a random UUID) and
+  an optional `parent_task_id`. These fields enable hierarchical task-tree tracing
+  and are forwarded automatically by `TaskGroup`.
+
+- **`Phronomy::Metrics` — task-centric observability snapshot** (#276):
+  `Phronomy::Metrics.snapshot` returns a hash with scheduler statistics:
+  `tasks_started`, `tasks_completed`, `tasks_failed`, `pool_queue_depth`, and
+  `pool_active_threads`. Intended for metrics export and health-check endpoints.
+
+- **`Phronomy::Testing::FakeClock` and `FakeScheduler`** (#273):
+  Two test helpers for deterministic concurrency testing.
+  `FakeClock` exposes `advance(seconds)` to control the passage of time without
+  sleeping. `FakeScheduler` replaces the real scheduler in specs, providing
+  synchronous execution and `flush` / `drain` helpers to drive task completion.
+
+- **`ScopePolicy` and approval gate integration** (#270):
+  `Phronomy::Tool::ScopePolicy` is a callable that maps `(tool_class, scope, agent)`
+  to `:allow`, `:approve`, or `:reject`. The default policy (`ScopePolicy::DEFAULT`)
+  automatically routes tools declaring high-risk scopes (`:write`, `:admin`,
+  `:external_network`, `:filesystem`, `:process`, `:external_process`) through the
+  existing approval gate; tools with `scope :read_only` or no scope are allowed
+  unconditionally. Per-agent policy overrides are available via
+  `agent.scope_policy = my_policy`.
+  **Behaviour change**: tools with the above scopes that previously executed without
+  an approval handler will now be **rejected** unless an approval handler is
+  registered or the agent uses a custom permissive policy.
+
+- **`PromptInjectionGuardrail`, `Tool::Base#redact_params`, and `#max_result_size`** (#271):
+  `Phronomy::Guardrail::PromptInjectionGuardrail` is a built-in `InputGuardrail`
+  subclass that detects prompt-injection patterns in user input.
+  `Tool::Base.redact_params(*names)` marks parameter names as sensitive; their
+  values are replaced with `"[REDACTED]"` in log and trace output.
+  `Tool::Base.max_result_size(n)` sets a per-tool character limit; results
+  exceeding the limit are truncated and a warning is logged. The global fallback is
+  `Phronomy.configure { |c| c.tool_result_max_size = n }` (default: no limit).
+
+- **`execution_mode` DSL on `Tool::Base`** (#263):
+  `Tool::Base.execution_mode` accepts `:cooperative`, `:blocking_io` (default),
+  `:cpu_bound`, or `:external_process`. Tools marked `:blocking_io` (the default)
+  are dispatched through `BlockingAdapterPool` when a `Runtime` is available,
+  keeping the scheduler thread unblocked. Tools marked `:cooperative` are called
+  directly on the scheduler thread (suitable for pure in-memory operations).
+
+- **`invoke_async` and `call_async` — async entry points** (#262):
+  `Agent::Base#invoke_async(input, **opts)` returns a `Phronomy::Task` wrapping
+  `#invoke`. `Workflow#invoke_async(input, config:)` does the same for workflows.
+  `Tool::Base#call_async(args, cancellation_token:)` returns a `Task` wrapping
+  `#call`. All three are backward-compatible with existing synchronous callers.
+
+- **`LLMAdapter` abstraction** (#266):
+  `Phronomy::LLMAdapter::Base` decouples the agent pipeline from RubyLLM.
+  `Phronomy::LLMAdapter::RubyLLM` (registered by default) wraps the existing
+  integration. Custom adapters can be registered via
+  `Phronomy.configure { |c| c.llm_adapter = MyAdapter }` for testing or
+  alternative LLM backends.
+
+- **`BlockingAdapterPool` backpressure limits** (#268):
+  `BlockingAdapterPool` now enforces configurable `pool_size` (default: 10) and
+  `queue_size` (default: 100) limits. Tasks submitted when the queue is full raise
+  `Phronomy::BackpressureError` immediately instead of growing the queue without
+  bound.
+
+- **Cooperative scheduler fairness** (#269):
+  The scheduler measures per-task lag and emits starvation and dispatch warnings
+  via `Phronomy.configuration.logger` when tasks wait longer than configured
+  thresholds. Configurable via `scheduler_starvation_warn_ms` and
+  `scheduler_dispatch_warn_ms`.
+
+- **Workflow entry actions awaitable with Task** (#264):
+  Entry action lambdas may now return a `Phronomy::Task`. The FSMSession awaits
+  the task on a background thread and posts `:action_completed` (with the resulting
+  `WorkflowContext`) or `:state_completed` back to the EventLoop without blocking
+  it. Backward-compatible: lambdas that return a `WorkflowContext` or `nil`
+  continue to work as before.
+
+- **`Task`, `TaskGroup`, `AsyncQueue`, `Deadline`, `InvocationContext`, `Runtime` concurrency abstractions** (#255):
+  Six new concurrency primitives form the foundation of the async execution layer.
+  `Task` wraps a callable with cancellation, timeout (`Deadline`), and context
+  propagation (`InvocationContext`). `TaskGroup` runs tasks concurrently and waits
+  for all to finish (or the first failure). `AsyncQueue` is a bounded, cancellable
+  queue. `Runtime` is the top-level façade that resolves a `BlockingAdapterPool`
+  and provides `blocking_io { }` and `cpu_bound { }` dispatch helpers.
+
+- **`BlockingAdapterPool`** (#256):
+  A bounded thread pool that isolates blocking I/O (LLM calls, database queries,
+  HTTP requests) from the cooperative scheduler thread. Default pool size is 10
+  threads with a queue depth of 100. Replaces direct `Thread.new` calls in core
+  agent and tool paths.
+
 - **`VectorStore#size` — document count for all backends, contract coverage for RedisSearch and Pgvector** (#240):
   `VectorStore::Base` gains `#size` as an abstract method; `InMemory`, `RedisSearch`,
   and `Pgvector` all implement it. `RedisSearch#size` queries `FT.INFO num_docs`;
@@ -128,9 +226,52 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   `dispatch_parallel` and `fan_out` accept `cancellation_token:` and automatically
   inject it into every worker task's config unless the task already supplies its own.
 
+### Removed
+
+- **BREAKING: `Agent::Base#run_as_child` drops `&result_writer` block parameter** (#265):
+  The optional block form `run_as_child(input, ctx: ctx) { |r| ctx.answer = r[:output] }`
+  is no longer supported. The result is now delivered **exclusively** as the
+  `:child_completed` event payload `{ output:, messages:, usage: }`. The parent
+  Workflow task is the sole owner of the `WorkflowContext`; no background thread
+  writes to it directly. Callers that were using the block to write back into the
+  context must update their workflow design (e.g. read the result in the target
+  state's entry action after the transition, or store output through an external
+  shared resource if needed).
+
+- **BREAKING (internal): `AgentFSM#initialize` drops `result_writer:` keyword** (#265):
+  Direct callers of `AgentFSM.new(result_writer: ...)` must remove that keyword.
+  This class is considered internal; gem consumers should use `run_as_child` instead.
+
 ### Changed
 
-- **`CancellationToken` checked at granular checkpoints** (#223):
+- **`AgentFSM`, `ParallelToolChat`, and `Orchestrator` use `Task`/`TaskGroup` instead of bare `Thread.new`** (#257, #258, #259):
+  All three components now spawn async work through the `Task` and `TaskGroup`
+  abstractions. This enables cancellation propagation, context threading, and
+  `BlockingAdapterPool` routing. No public API changes; behaviour is equivalent.
+
+- **`Thread.current[:phronomy_*]` context propagation replaced with explicit `InvocationContext`** (#260):
+  Thread-local keys `phronomy_event_loop_thread`, `phronomy_cancellation_token`,
+  and `phronomy_context_version_caches` are no longer used as the primary
+  propagation channel. `InvocationContext` is threaded explicitly through call
+  stacks. Importantly, `Tool::Base#call` no longer falls back to
+  `Thread.current[:phronomy_cancellation_token]`; cancellation is only observed
+  when the caller passes `cancellation_token:` explicitly (or when
+  `ParallelToolChat` injects it). Tools that relied on the thread-local fallback
+  must be updated.
+
+- **`Timeout.timeout` removed from core paths; replaced with `CancellationScope`** (#261):
+  `Agent::Base#invoke` and `McpTool::StdioTransport` no longer use `Timeout.timeout`
+  (which is unsafe with `Thread.new` and `ensure` blocks). A `CancellationScope`
+  with `deadline_in(seconds)` provides equivalent semantics without the thread-
+  interruption hazards. `ScopeTimeoutError < TimeoutError` is raised on expiry.
+
+- **RAG/VectorStore blocking I/O placed behind `BlockingAdapterPool` async boundary** (#267):
+  `KnowledgeSource#fetch` and all three `VectorStore` backends now execute their
+  blocking I/O through `Runtime#blocking_io` when a `Runtime` is present. Callers
+  in a synchronous context see no change; callers in an EventLoop context benefit
+  from non-blocking scheduler behaviour.
+
+
   The cancellation token (passed via `config: { cancellation_token: token }`) is
   now checked at multiple additional points beyond the initial LLM call boundary:
   before each `KnowledgeSource#fetch` in `build_context` (RAG phase); after each
@@ -195,6 +336,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Fixed
 
+- **`tool_name` preserved in `Orchestrator#prepare_tool_class` anonymous subclass wrapper**:
+  When `Orchestrator#prepare_tool_class` wrapped a subagent tool in an anonymous
+  subclass (`Class.new(prepared)`), the class-level instance variable `@tool_name`
+  was not inherited, causing the wrapper's `tool_name` to return `nil`. RubyLLM
+  then registered the tool under a `nil` key, making it unreachable when the LLM
+  called it by name. The fix captures the effective name before subclassing and
+  calls `tool_name effective_name` explicitly inside the anonymous class body —
+  the same pattern already used by the approval-gate wrapper.
+
 - **`EventLoop#start` is now idempotent; stale `:__stop__` sentinel race fixed** (#203):
   Calling `start` on an already-running `EventLoop` is now a no-op. Fixed a race condition
   where `stop` setting `@running = false` before the worker thread was scheduled left the

data/README.md

@@ -22,31 +22,80 @@ It provides composable building blocks — Workflows, Agents, Tools, Guardrails,
 > **Note**: The `main` branch contains unreleased development work. Pin to a released gem
 > version (`gem "phronomy", "~> 0.x"`) for stability in production.
 
+**Core building blocks**
+
 | Feature | Stability |
 |---|---|
 | **Workflow** — Stateful, branching workflows with wait_state/send_event | Stable |
-| **Workflow EventLoop Mode** — Opt-in event-driven execution: `Phronomy.configure { \|c\| c.event_loop = true }` | Experimental |
-| **Agent EventLoop Mode** — `Agent#invoke` (non-blocking via EventLoop), `Agent#run_as_child` (child-FSM pattern for Workflow integration), parallel tool dispatch via `ParallelToolChat` | Experimental |
-| **Workflow Parallel Node** — Concurrent branches via application-level threads | Beta |
+| **Workflow action_timeout** — Per-state `action_timeout:` keyword on `state` DSL; cancels Task-returning entry actions that exceed the limit and raises `Phronomy::ActionTimeoutError` | Beta |
 | **Agent** — ReAct-style tool-calling agents with guardrails and conversation history | Stable |
 | **Before-Completion Hook** — Three-tier LLM parameter injection | Stable |
 | **Context Management** — Token budget calculation, estimation, and pruning | Stable |
-| **Knowledge/RAG** — Retrieval sources with pluggable loaders, splitters, and vector stores; `static_knowledge_refresh!` for runtime cache invalidation | Beta |
-| **`VectorStore#size`** — Returns document count for all three backends (InMemory, RedisSearch, Pgvector) | Beta |
-| **Multi-agent** — Agent-as-Tool pattern and hub-and-spoke handoff routing | Beta |
-| **GeneratorVerifier** — Generator-Verifier loop with injectable prompt builders/parsers | Beta |
-| **Agent::Orchestrator** — Parallel subagent dispatch, fan-out, and `subagent` DSL | Beta |
-| **Agent::TeamCoordinator** — Agent teams pattern: LLM coordinator + stateful workers with sequential task assignment (worker-local message history persisted across tasks) | Beta |
-| **Agent::SharedState** — Shared state pattern: peer agents collaborate via a shared KnowledgeStore; `member` DSL with per-agent instructions and `coordination` team protocol | Experimental |
 | **Guardrails** — Input/output validation with custom `InputGuardrail`/`OutputGuardrail` | Beta |
+| **`PromptInjectionGuardrail`** — Built-in `InputGuardrail` subclass that detects prompt-injection patterns; usable standalone or as part of a guardrail chain | Beta |
+| **`Tool::Base.redact_params` / `.max_result_size`** — Class-level DSL: `redact_params` masks parameter values in log/trace output; `max_result_size` truncates oversized tool results before they reach the LLM | Beta |
 | **Output Parser** — JSON and Struct-mapped parsers for structured LLM responses | Stable |
 | **Eval Framework** — Dataset-driven evaluation with multiple scorer types | Beta |
 | **Tracing** — Pluggable span-based observability | Stable |
-| **MCP Tool** — Model Context Protocol server integration | Beta |
 | **Error Taxonomy** — `RateLimitError`, `AuthenticationError`, `ContextLengthError`, `TransportError` (subclasses of `Phronomy::Error`) raised at the agent retry boundary | Beta |
-| **`Phronomy.with_configuration` / `Phronomy.reset_runtime!`** — Scoped configuration override and full runtime reset for test isolation | Beta |
+
+**Knowledge and integration**
+
+| Feature | Stability |
+|---|---|
+| **Knowledge/RAG** — Retrieval sources with pluggable loaders, splitters, and vector stores; `static_knowledge_refresh!` for runtime cache invalidation | Beta |
+| **`VectorStore#size`** — Returns document count for all three backends (InMemory, RedisSearch, Pgvector) | Beta |
+| **`Agent::Context::Knowledge::VectorStore::AsyncBackend` mixin** — Pluggable async interface for `VectorStore`; default pool-backed implementations for `search_async`, `add_async`, `remove_async`, `clear_async`; backends with native async drivers override individual methods to bypass `BlockingAdapterPool` entirely; all existing backends remain unchanged | Beta |
+| **Parallel RAG multi-source fetch** — `Agent#build_context` fetches all `knowledge_sources` concurrently via `TaskGroup`; `config[:rag_failure_policy]` `:skip` (default) silently ignores failed sources so the agent answers with partial context, `:fail` surfaces the first error; per-source latency is emitted to `Phronomy.configuration.logger` at debug level | Beta |
+| **MCP Tool** — Model Context Protocol server integration | Beta |
+
+**Execution and reliability**
+
+| Feature | Stability |
+|---|---|
+| **Workflow EventLoop Mode** — Opt-in event-driven execution: `Phronomy.configure { \|c\| c.event_loop = true }` | Experimental |
+| **Agent EventLoop Mode** — `Agent#invoke` (non-blocking via EventLoop), `Agent#run_as_child` (child-FSM pattern for Workflow integration), parallel tool dispatch via `ParallelToolChat` | Experimental |
+| **`invoke_async` / `call_async`** — `Agent::Base#invoke_async` and `Workflow#invoke_async` return a `Task`; `Tool::Base#call_async` similarly; compatible with EventLoop and standalone contexts | Experimental |
 | **CancellationToken** — Cooperative cancellation via `cancel!`/`cancelled?`/`raise_if_cancelled!`; `timeout_after(seconds)` for monotonic-clock deadlines; optional `deadline:` (wall-clock) for backward compatibility; passed as `config: { cancellation_token: token }` to agents and `dispatch_parallel`; injected into `tool.execute` when the method declares a `cancellation_token:` keyword | Experimental |
 | **`dispatch_parallel` / `fan_out` `force_kill:` option** — `force_kill: false` (default) leaves timed-out workers running and raises `TimeoutError` immediately; `force_kill: true` restores the old `Thread#kill` behaviour with a `logger.warn` | Beta |
+| **`execution_mode` DSL on `Tool::Base`** — Declares how a tool's `execute` should be dispatched: `:cooperative` (same scheduler thread), `:blocking_io` (default; offloaded to `BlockingAdapterPool`), `:cpu_bound`, `:external_process` | Experimental |
+| **`invocation_context:` keyword on `Agent#invoke` / `Workflow#invoke`** — Pass a `Phronomy::InvocationContext` directly; `thread_id`, `cancellation_token`, and `deadline`-based timeout are derived from it; `task_id` / `parent_task_id` appear in trace spans automatically; `config:` keys remain supported as backward-compat aliases | Beta |
+| **`ConcurrencyGate` — unified backpressure** — Counting semaphore that enforces per-resource concurrency caps (`max_concurrent_agent_tasks`, `max_concurrent_tool_tasks`, `max_concurrent_workflow_tasks`, `max_concurrent_llm_calls`, `max_concurrent_rag_fetches`, `max_concurrent_vector_searches`); configured via `Phronomy.configure`; backpressure behaviour follows the global `backpressure` setting (`:wait`, `:raise`/`:reject`, `:timeout`); `nil` cap = unlimited (default) | Beta |
+| **Cooperative scheduler yield points** — `Runtime#yield` (cooperative yield; yields the current task's time slice); `Runtime#yield_if_needed(every: N)` (thread-local counter, yields every N calls); CPU-bound detection when `blocking_detect_threshold_ms` is set (warns and increments `non_yield_threshold_violation_count` when a task runs longer than the threshold without yielding); `starvation_threshold_ms` configuration field (default: 50ms) | Beta |
+| **`Phronomy::Metrics`** — `Phronomy::Metrics.snapshot` returns task-tree and pool counters; task-centric keys: `active_agent_tasks`, `active_tool_tasks`, `active_workflow_tasks`, `active_rag_tasks`, `active_llm_tasks`, `task_wait_time_p50_ms`, `task_wait_time_p95_ms`, `task_run_time_p50_ms`, `task_run_time_p95_ms`, `cancelled_tasks`, `failed_tasks`, `non_yield_threshold_violation_count`; pool/event-loop keys remain for backward compatibility; `Runtime#task_snapshot` exposes task-centric metrics directly | Beta |
+| **`Phronomy.with_configuration` / `Phronomy.reset_runtime!`** — Scoped configuration override and full runtime reset for test isolation | Beta |
+
+**Agent patterns**
+
+| Feature | Stability |
+|---|---|
+| **Workflow parallel pattern** — Concurrent branches via application-level threads (no built-in parallel primitive; see the Workflow section for the recommended pattern) | Beta |
+| **Multi-agent** — Agent-as-Tool pattern and hub-and-spoke handoff routing | Beta |
+| **GeneratorVerifier** — Generator-Verifier loop with injectable prompt builders/parsers | Beta |
+| **Agent::Orchestrator** — Parallel subagent dispatch, fan-out, and `subagent` DSL | Beta |
+| **Agent::TeamCoordinator** — Agent teams pattern: LLM coordinator + stateful workers with sequential task assignment (worker-local message history persisted across tasks) | Beta |
+| **Agent::SharedState** — Shared state pattern: peer agents collaborate via a shared KnowledgeStore; `member` DSL with per-agent instructions and `coordination` team protocol | Experimental |
+| **`ScopePolicy`** — Configurable policy callable that maps (tool, scope, agent) to `:allow`/`:approve`/`:reject`; default policy auto-routes high-risk scopes through the approval gate | Experimental |
+
+> **Public API boundary**: The tables above are the complete list of classes, modules, and features
+> intended for gem consumers. Every entry has an associated stability label.
+> All other classes, modules, and methods — including everything in the
+> [Advanced / Internal APIs](#advanced--internal-apis) section below — are
+> marked `@api private` in source and may change without notice. Do not
+> depend on internal APIs in application code.
+
+## Advanced / Internal APIs
+
+The APIs listed below are intended for advanced use cases, framework internals, and test infrastructure. Typical application code does not need to interact with them directly.
+
+> These APIs are subject to change without the same backwards-compatibility guarantees as the stable public API.
+
+| Feature | Stability |
+|---|---|
+| **`Phronomy::Diagnostics`** — Snapshot of scheduler internals for debug/monitoring; `SchedulerReentrancyError` raised on invalid re-entrant scheduler use; `Runtime.in_scheduler_context?` returns `true` when called from inside a scheduler task | Experimental |
+| **`Phronomy::Testing::FakeClock` / `FakeScheduler` / `SchedulerHelpers`** — Test helpers for deterministic concurrency specs: `FakeClock#advance(seconds)` controls time; `FakeScheduler` runs tasks synchronously and records `event_log`; `FakeScheduler#assert_order` / `#assert_cancelled` for ordering assertions; `FakeClock#advance_to_next_timer` fires the next pending callback; `Testing::SchedulerHelpers#with_fake_scheduler` replaces the global Runtime for the duration of a block | Beta |
+| **`Configuration#runtime_backend`** — `:thread` (default, one OS thread per task), `:immediate` (tests — tasks run synchronously, no extra threads), `:fiber` (**EXPERIMENTAL** — experimental validation backend only: runs tasks as Ruby Fibers on a cooperative scheduler to verify that framework components are truly non-blocking; **not for production use** and not a planned production replacement for `:thread`; no preemptive scheduling will be added). `:cooperative` is a **deprecated alias** for `:immediate` — do not use in new code | Beta |
+| **`Configuration#strict_runtime_guards`** — When `true`, calling `Agent#invoke` from inside a scheduler task raises `SchedulerReentrancyError`; when `false` (default) a warning is logged instead | Beta |
 
 ## Installation
 
@@ -82,8 +131,8 @@ Install additional gems only for the features you use:
 
 | Gem | Required for |
 |-----|-------------|
-| `pgvector` | `Phronomy::VectorStore::Pgvector` |
-| `redis` | `Phronomy::VectorStore::RedisSearch` |
+| `pgvector` | `Phronomy::Agent::Context::Knowledge::VectorStore::Pgvector` |
+| `redis` | `Phronomy::Agent::Context::Knowledge::VectorStore::RedisSearch` |
 | `opentelemetry-api` | `Phronomy::Tracing::OpenTelemetryTracer` |
 
 ## Quick Start
@@ -150,13 +199,16 @@ puts "Approved: #{final.approved}"  # => true
 ```
 
 In EventLoop mode (`c.event_loop = true`), `Agent#run_as_child` spawns a child agent
-asynchronously. When the child succeeds, `:child_completed` is dispatched; when it fails,
-`:child_failed` is dispatched. Always declare both transitions to avoid a stuck workflow:
+asynchronously. When the child succeeds, `:child_completed` is dispatched with the result
+`{ output:, messages:, usage: }` as its payload; when it fails, `:child_failed` is
+dispatched. Always declare both transitions to avoid a stuck workflow:
 
 ```ruby
-# EventLoop mode: workflow that runs an agent as a child FSM
+# EventLoop mode: workflow that runs an agent as a child FSM.
+# The result { output:, messages:, usage: } arrives as the :child_completed event
+# payload — write it back to the context in the target state's entry action.
 entry :run_agent, ->(ctx) {
-  MyAgent.new.run_as_child(ctx.query, ctx: ctx) { |r| ctx.answer = r[:output] }
+  MyAgent.new.run_as_child(ctx.query, ctx: ctx)
 }
 transition from: :run_agent, on: :child_completed, to: :done
 transition from: :run_agent, on: :child_failed,    to: :handle_error
@@ -222,24 +274,25 @@ rescue Phronomy::GuardrailError => e
 end
 ```
 
-> **Limitations:** Phronomy ships no built-in guardrail implementations. There is no
-> built-in prompt injection detector, PII scanner, or content classifier. All guardrail
-> logic must be implemented by the application. Reference implementations for common
-> patterns are available in `phronomy-examples` (example 06).
+> **Note:** Phronomy includes `PromptInjectionGuardrail`, a built-in pattern-based
+> input guardrail that detects common injection patterns (see the feature table above).
+> PII scanning and content classification are **not** provided by the framework;
+> that logic must be implemented by the application. Reference implementations for
+> common patterns are available in `phronomy-examples` (example 06).
 
 ### Knowledge/RAG — Context injection and vector retrieval
 
 ```ruby
 # Static knowledge (policy files, reference docs)
-policy = Phronomy::KnowledgeSource::StaticKnowledge.new(
+policy = Phronomy::Agent::Context::Knowledge::Source::StaticKnowledge.new(
   File.read("policy.md"),
   type:   :policy,
   source: "policy.md"   # exposed to LLM for citation
 )
 
 # RAG retrieval from a vector store
-store      = Phronomy::VectorStore::InMemory.new
-embeddings = Phronomy::Embeddings::RubyLLMEmbeddings.new(model: "text-embedding-3-small")
+store      = Phronomy::Agent::Context::Knowledge::VectorStore::InMemory.new
+embeddings = Phronomy::Agent::Context::Knowledge::Embeddings::RubyLLMEmbeddings.new(model: "text-embedding-3-small")
 
 # Add documents before querying
 text1 = "Refunds are processed within 5 business days."
@@ -247,7 +300,7 @@ text2 = "Contact support@example.com for refund requests."
 store.add(id: "doc-1", embedding: embeddings.embed(text1), metadata: { content: text1, source: "policy.md" })
 store.add(id: "doc-2", embedding: embeddings.embed(text2), metadata: { content: text2, source: "policy.md" })
 
-rag = Phronomy::KnowledgeSource::RAGKnowledge.new(store: store, embeddings: embeddings, k: 5)
+rag = Phronomy::Agent::Context::Knowledge::Source::RAGKnowledge.new(store: store, embeddings: embeddings, k: 5)
 
 # Inject at invocation time
 result = MyAgent.new.invoke("What is the refund policy?",
@@ -266,8 +319,8 @@ MyAgent.static_knowledge_refresh!
 Load and split documents with built-in loaders:
 
 ```ruby
-chunks = Phronomy::Loader::MarkdownLoader.new.load("docs/guide.md")
-         .then { |docs| Phronomy::Splitter::RecursiveSplitter.new(chunk_size: 512).split(docs) }
+chunks = Phronomy::Agent::Context::Knowledge::Loader::MarkdownLoader.new.load("docs/guide.md")
+         .then { |docs| Phronomy::Agent::Context::Knowledge::Splitter::RecursiveSplitter.new(chunk_size: 512).split(docs) }
 ```
 
 ### Multi-Agent Handoff — Hub-and-spoke routing
@@ -407,9 +460,11 @@ class MyOrchestrator < Phronomy::Agent::Orchestrator
 end
 ```
 
-### Workflow Parallel Node — Concurrent branches
+### Workflow parallel pattern — Concurrent branches
 
-Phronomy does not provide a built-in parallel abstraction. Use application-level Ruby threads inside a `state` action:
+Phronomy does not provide a dedicated parallel-node primitive. The recommended
+pattern for concurrent branches is to use application-level Ruby threads inside
+a `state` action:
 
 ```ruby
 class EnrichContext
@@ -426,9 +481,9 @@ app = Phronomy::Workflow.define(EnrichContext) do
       summary: Thread.new { Summarizer.call(s) },
       tags:    Thread.new { Tagger.call(s) }
     }
-    # For production use, wrap with Timeout.timeout to avoid unbounded waits:
-    #   require "timeout"
-    #   Timeout.timeout(30) { threads.each_value(&:join) }
+    # For bounded waits, use Thread#join(timeout_seconds); nil means timed out — handle explicitly.
+    # Do not use Timeout.timeout or Thread#kill — both inject async exceptions that bypass cleanup.
+    # Prefer CancellationToken for cooperative cancellation of Phronomy-managed tasks.
     threads.each_value(&:join)
     s.merge(summary: threads[:summary].value, tags: Array(threads[:tags].value))
   end
@@ -535,6 +590,8 @@ Phronomy.configure do |c|
   c.trace_pii                       = false # default; set to true only when trace data contains no PII
   c.logger                          = nil   # optional; any object responding to #warn (e.g. Rails.logger)
   c.event_loop_stop_grace_seconds   = 5     # seconds to wait for sessions to drain on EventLoop#stop(drain: true)
+  c.runtime_backend                 = :thread   # :thread (default); :immediate (tests, synchronous); :fiber (experimental validation only); :cooperative (deprecated alias for :immediate)
+  c.strict_runtime_guards           = false          # when true, raises on invoke-inside-task
 end
 ```
 
@@ -546,6 +603,66 @@ end
 > The default is `false` (PII protection enabled). Set to `true` only when
 > trace data does not contain sensitive information.
 
+## Sync vs Async API
+
+Phronomy provides both synchronous and asynchronous invocation APIs.
+Understanding when to use each prevents scheduler stalls and hidden deadlocks.
+
+| Context | Recommended API |
+|---------|----------------|
+| Top-level application code, Rails controller, background job | `agent.invoke(input)` — blocks the calling thread until done |
+| Inside a `Runtime#spawn` block, `TaskGroup`, Workflow action, Tool `execute` | `agent.invoke_async(input).await` — non-blocking within the scheduler |
+
+### Why this matters
+
+`invoke` is a synchronous wrapper that calls `invoke_async` and then _blocks_ the calling
+thread until the task completes. When called from **inside** an active scheduler task, the
+calling task blocks the scheduler thread, preventing other tasks from making progress — a
+hidden deadlock when all scheduler threads are occupied.
+
+### Runtime guard
+
+Phronomy detects this pattern automatically:
+
+```ruby
+# Default (soft mode): logs a warning and continues
+Phronomy.configure { |c| c.strict_runtime_guards = false }
+
+# Strict mode: raises SchedulerReentrancyError immediately
+Phronomy.configure { |c| c.strict_runtime_guards = true }
+```
+
+You can also query the current context directly:
+
+```ruby
+Phronomy::Runtime.in_scheduler_context?  # => true if called from inside a task
+```
+
+### Migration: invoke → invoke_async
+
+```ruby
+# Before (blocks scheduler if called from inside a task)
+result = my_agent.invoke("Hello")
+
+# After (safe inside tasks and TaskGroups)
+result = my_agent.invoke_async("Hello").await
+```
+
+### :immediate backend (synchronous / test mode)
+
+The `:immediate` backend runs tasks synchronously using `FakeScheduler`
+(backed by `Task::ImmediateBackend`).  Blocking I/O is isolated in `BlockingAdapterPool`.
+To switch back to the default thread-per-task backend:
+
+```ruby
+Phronomy.configure { |c| c.runtime_backend = :thread }
+# or per-example using SchedulerHelpers:
+include Phronomy::Testing::SchedulerHelpers
+with_fake_scheduler do |sched|
+  # all spawns run synchronously; sched.event_log records every lifecycle event
+end
+```
+
 ## Context Management
 
 Phronomy includes a context window management layer. When model metadata is
@@ -557,7 +674,7 @@ agents automatically stay within the configured token limit.
 Derives the effective token budget from RubyLLM's model registry:
 
 ```ruby
-budget = Phronomy::Context::TokenBudget.new(
+budget = Phronomy::LlmContextWindow::TokenBudget.new(
   model:    "claude-3-5-sonnet-20241022",  # looks up context_window + max_output_tokens
   overhead: 500                            # extra reservation for tool definitions
 )
@@ -569,7 +686,7 @@ budget.effective_input_limit # => 191_308
 Or supply explicit values (useful for local / unregistered models):
 
 ```ruby
-budget = Phronomy::Context::TokenBudget.new(
+budget = Phronomy::LlmContextWindow::TokenBudget.new(
   context_window:    32_768,
   max_output_tokens: 4_096
 )
@@ -583,7 +700,7 @@ class MyAgent < Phronomy::Agent::Base
   max_output_tokens 4096   # override max_output_tokens from registry
   context_overhead  600    # extra reservation for system prompt + tools
   invoke_timeout    30     # raise Phronomy::TimeoutError after 30 s (wait timeout, not cancellation)
-  max_parallel_tools 4     # cap concurrent tool-call threads (default: 10)
+  max_parallel_tools 4     # cap concurrent tool executions (default: 10)
 end
 ```
 
@@ -599,7 +716,7 @@ registry the budget is silently skipped.
 > ```ruby
 > require "tiktoken_ruby"
 > enc = Tiktoken.encoding_for_model("gpt-4o")
-> Phronomy::Context::TokenEstimator.tokenizer = ->(text) { enc.encode(text).length }
+> Phronomy::LlmContextWindow::TokenEstimator.tokenizer = ->(text) { enc.encode(text).length }
 > ```
 
 
@@ -624,12 +741,16 @@ blocks always execute.
 > - Any external I/O (database query, vector search, HTTP request) inside those calls
 >
 > For deep in-flight safety, complement `CancellationToken` with per-source or
-> per-tool timeouts (e.g. `Net::HTTP#read_timeout`, `Timeout.timeout`, connection
-> pool limits). Ruby's GVL prevents fully preemptive cancellation without
-> `Thread#kill`, which Phronomy avoids by default due to resource safety concerns.
+> per-tool timeouts. Prefer library-native timeouts such as `Net::HTTP#read_timeout`,
+> database `statement_timeout`, or Redis client timeout — these signal the I/O layer
+> to abort cleanly. Avoid `Timeout.timeout` unless you understand its async-exception
+> risks: it injects `Timeout::Error` at an arbitrary execution point (the same
+> mechanism as `Thread#kill`), which Phronomy avoids by default due to resource
+> safety concerns. Ruby's GVL prevents fully preemptive cancellation without such
+> risky interruption.
 
 ```ruby
-token = Phronomy::CancellationToken.new
+token = Phronomy::Concurrency::CancellationToken.new
 
 # Cancel from another thread after 5 s
 Thread.new { sleep 5; token.cancel! }
@@ -641,15 +762,15 @@ rescue Phronomy::CancellationError
 end
 
 # Hard deadline via monotonic clock (recommended — immune to NTP/DST changes)
-token = Phronomy::CancellationToken.timeout_after(30)
+token = Phronomy::Concurrency::CancellationToken.timeout_after(30)
 result = MyAgent.new.invoke("...", config: { cancellation_token: token })
 
 # Hard deadline via wall-clock (legacy — still supported)
-token = Phronomy::CancellationToken.new(deadline: Time.now + 30)
+token = Phronomy::Concurrency::CancellationToken.new(deadline: Time.now + 30)
 result = MyAgent.new.invoke("...", config: { cancellation_token: token })
 
 # Propagate to all parallel workers via dispatch_parallel / fan_out
-token = Phronomy::CancellationToken.new
+token = Phronomy::Concurrency::CancellationToken.new
 Thread.new { sleep 10; token.cancel! }
 
 orchestrator.dispatch_parallel(
@@ -740,9 +861,11 @@ span attributes by default (`trace_pii: false`). To include full content in trac
 Phronomy configuration. Evaluate whether your tracing backend (OTLP collector, Jaeger,
 Honeycomb, etc.) meets your data-retention and privacy requirements.
 
-**Prompt injection** — Phronomy provides no built-in prompt injection detection.
-Applications that process untrusted user input should implement their own input
-guardrails (see the Guardrails section above).
+**Prompt injection** — Phronomy provides `PromptInjectionGuardrail`, a built-in
+pattern-based input guardrail that detects common injection patterns (ignore/override
+instructions, role-switching phrases, etc.). It is a useful starting point, not a
+comprehensive defence; applications processing untrusted input should layer additional
+custom guardrails as needed (see the Guardrails section above).
 
 **Tool and MCP security** — Tools can perform real-world side effects (database
 writes, API calls, file deletion). Treat tool execution as a privileged operation:

data/Rakefile

@@ -7,4 +7,37 @@ RSpec::Core::RakeTask.new(:spec)
 
 require "standard/rake"
 
+# Verify that @api private classes do not leak into the public YARD output.
+# Any class or module without @api private that ends up in the public doc must
+# have a corresponding entry in the Features table in README.md.
+#
+# Usage: bundle exec rake yard_check
+desc "Build YARD docs excluding @api private items and check for undocumented public APIs"
+task :yard_check do
+  require "yard"
+  YARD::Registry.clear
+  YARD.parse(Dir["lib/**/*.rb"])
+
+  undocumented = []
+  YARD::Registry.all(:class, :module).each do |obj|
+    next if obj.visibility == :private
+    next if obj.tag(:api)&.name == "private"
+    next if obj.docstring.blank?
+
+    # Classes/modules with no docstring that are not @api private are worth
+    # noting, but only raise on truly undocumented public objects.
+    if obj.docstring.empty?
+      undocumented << obj.path
+    end
+  end
+
+  unless undocumented.empty?
+    warn "The following public classes/modules have no YARD documentation:\n" \
+         "  #{undocumented.join("\n  ")}\n" \
+         "Either add a docstring or mark them @api private."
+    exit 1
+  end
+  puts "yard_check passed — no undocumented public classes/modules found."
+end
+
 task default: %i[spec standard]

data/benchmark/baseline.json

@@ -2,7 +2,7 @@
   "workflow_context_merge": 124364.81010472385,
   "workflow_define": 2179.945274115319,
   "tool_params_schema_definition": 19534379.159046534,
-  "dispatch_parallel_10": 1483.2255243486482,
+  "dispatch_parallel_10": 886.0,
   "cancellation_token_cancelled": 4335060.97443425,
   "cancellation_token_raise_if_cancelled_noop": 3566903.189098373,
   "trim_context_remove_2000": 1761.5700678986254

data/benchmark/bench_context_assembler.rb

@@ -12,9 +12,9 @@ BenchAsmMessage = Struct.new(:content)
 
 def make_assembler(n_messages:, n_chunks:, with_budget: false)
   budget = if with_budget
-    Phronomy::Context::TokenBudget.new(context_window: 4096, max_output_tokens: 512)
+    Phronomy::LlmContextWindow::TokenBudget.new(context_window: 4096, max_output_tokens: 512)
   end
-  asm = Phronomy::Context::Assembler.new(budget: budget)
+  asm = Phronomy::LlmContextWindow::Assembler.new(budget: budget)
   asm.add_instruction("You are a helpful assistant. Answer the user's question.")
   n_chunks.times do |i|
     asm.add_knowledge("Fact #{i}: The capital of country #{i} is City #{i}.", type: :entity, trusted: true)