phronomy 0.15.1 → 0.17.0

data/README.md

@@ -27,9 +27,9 @@ It provides composable building blocks — Workflows, Agents, Tools, Filters, an
 | Feature | Stability |
 |---|---|
 | **Workflow** — Stateful, branching workflows with wait_state/send_event | Stable |
-| **Agent** — ReAct-style tool-calling agents with guardrails and conversation history | Stable |
-| **Before-Completion Hook** — Three-tier LLM parameter injection | Stable |
-| **Context Management** — Token budget calculation, estimation, and pruning; `Agent::Base` protected hooks: `build_context` (overridable), `trim_messages`, `trim_to_budget`, `compact_messages`, `budget_exceeded?`, `drop_messages_over` | Stable |
+| **Agent** — Stateful ReAct-style tool-calling agents with stable `agent_id`, persistence-backed execution state, canonical execution history, guardrails, and conversation context | Stable |
+| **Before-LLM-Input Hook** — Three-tier per-call LLM input customization via `before_llm_input` and `LLMInputPatch` | Stable |
+| **Context Management** — Canonical Journal + per-LLM-call Manifest architecture with token-budget-aware selection and protocol-safe Tool Call / Tool message dependencies. Context selection never deletes canonical execution history | Stable |
 | **Filters** — Input/output transformation and blocking via `Filter::Base`; call `block!(reason)` to reject and raise `FilterBlockError` | Beta |
 | **`PromptInjectionFilter`** — Built-in `Filter::Base` subclass that detects prompt-injection patterns; usable standalone or as part of a filter chain | Beta |
 | **`Agent::Context::Capability::Base.redact_params` / `.max_result_size`** — Class-level DSL: `redact_params` masks parameter values in log/trace output; `max_result_size` truncates oversized tool results before they reach the LLM | Beta |
@@ -42,7 +42,7 @@ It provides composable building blocks — Workflows, Agents, Tools, Filters, an
 
 | Feature | Stability |
 |---|---|
-| **Knowledge** — Static context injection with pluggable loaders, splitters, and vector stores; `static_knowledge_refresh!` for runtime cache invalidation | Beta |
+| **Knowledge** — Journal-backed persistent Agent context registered with `knowledge:` / `add_knowledge`; selected per LLM call by Context Policy; `clear_knowledge!` logically resets retained Knowledge without deleting Journal history | Beta |
 | **`VectorStore#size`** — Returns document count for all three backends (InMemory, RedisSearch, Pgvector) | Beta |
 | **`VectorStore::AsyncBackend` mixin** — Pluggable async interface for `VectorStore`; default pool-backed implementations for `search_async`, `add_async`, `remove_async`, `clear_async`; backends with native async drivers override individual methods to bypass `BlockingAdapterPool` entirely; all existing backends remain unchanged | Beta |
 | **MCP Tool** — `Phronomy::Tools::Mcp`: Model Context Protocol server integration via the official `mcp` gem; `Phronomy::Tools::Agent`: wraps an agent class as a callable tool via `from_agent` | Beta |
@@ -56,9 +56,8 @@ It provides composable building blocks — Workflows, Agents, Tools, Filters, an
 | **`stream` / `stream_async`** — callbacks execute on the EventLoop thread and must return quickly; the block form remains a compatibility alias for `on_event:` | Beta |
 | **`stream_callback_error_policy`** — Backward-compatible setting shared by `invoke_async` and `stream_async` terminal `on_event:` callbacks: `:report` (default) preserves the Agent result, while `:fail_task` fails the returned Task with `Phronomy::StreamCallbackError`; Agent execution errors are never replaced by callback errors | Beta |
 | **`invoke_async` / `call_async`** — `Agent::Base#invoke_async` and `Workflow#invoke_async` return a `Task`; `Agent::Context::Capability::Base#call_async` similarly; compatible with EventLoop and standalone contexts | Stable |
-| **`Task#map`** — transforms a Task's completed value and propagates failure/cancellation; Workflow entry and transition actions do not await mapped Tasks | Stable |
-| **CancellationToken** — Cooperative cancellation via `cancel!`/`cancelled?`/`raise_if_cancelled!`; `timeout_after(seconds)` for monotonic-clock deadlines; optional `deadline:` (wall-clock) for backward compatibility; passed as `config: { cancellation_token: token }` to agents and `dispatch_parallel`; injected into `tool.execute` when the method declares a `cancellation_token:` keyword; bridged to `MCP::Cancellation` in `Phronomy::Tools::Mcp#execute` | Experimental |
-| **`dispatch_parallel` / `fan_out` `force_kill:` option** — `force_kill: false` (default) leaves timed-out workers running and raises `TimeoutError` immediately; `force_kill: true` restores the old `Thread#kill` behaviour with a `logger.warn` | Beta |
+| **`Task#map`** — transforms a Task's completed value and propagates failure/cancellation. `Task#map` remains available for application-level Task composition, but Workflow entry and transition actions must not return a Task | Stable |
+| **CancellationToken** — Cooperative cancellation via `cancel!`/`cancelled?`/`raise_if_cancelled!`; `timeout_after(seconds)` for monotonic-clock deadlines; passed as `config: { cancellation_token: token }` to agents and `dispatch_parallel`; injected into `tool.execute` when the method declares a `cancellation_token:` keyword; bridged to `MCP::Cancellation` in `Phronomy::Tools::Mcp#execute` | Experimental |
 | **`execution_mode` DSL on `Agent::Context::Capability::Base`** — Declares how a tool's `execute` should be dispatched: `:cooperative` (same scheduler thread), `:blocking_io` (default; offloaded to `BlockingAdapterPool`), `:cpu_bound`, `:external_process`; Tool-specific timeout/retry belongs to the Tool implementation or its client | Experimental |
 | **`blocking_io_pool_size` / `blocking_io_queue_size`** — Configure the default `BlockingAdapterPool` via `Phronomy.configure { \|c\| c.blocking_io_pool_size = 20; c.blocking_io_queue_size = 200 }`; all LLM calls, MCP tool calls, and other blocking I/O share this pool; defaults: `pool_size: 10`, `queue_size: 100` | Beta |
 | **`invocation_context:` keyword on `Agent#invoke` / `Workflow#invoke`** — Pass a `Phronomy::InvocationContext` directly; `thread_id`, `cancellation_token`, and `deadline`-based timeout are derived from it; `task_id` / `parent_task_id` appear in trace spans automatically; `config:` keys remain supported as backward-compat aliases | Beta |
@@ -78,7 +77,7 @@ It provides composable building blocks — Workflows, Agents, Tools, Filters, an
 | **`Phronomy::MultiAgent::Orchestrator`** — Parallel subagent dispatch, fan-out, and `subagent` DSL | Beta |
 | **`Phronomy::MultiAgent::TeamCoordinator`** — Agent teams pattern: LLM coordinator + stateful workers with sequential task assignment (worker-local message history persisted across tasks) | Beta |
 | **Agent::SharedState** — Shared state pattern: peer agents collaborate via a shared KnowledgeStore; `member` DSL with per-agent instructions and `coordination` team protocol | Experimental |
-| **Human-in-the-loop approval** — `Agent::Base#invoke` returns `{ suspended: true, agent_invocation_id: String, approval_request: Phronomy::Agent::ToolApprovalRequest }` when a tool requiring approval is encountered; `Agent::Base#approve(id, approval_request_id:, approved:)` (synchronous) or `Agent::Base#approve_async(id, approval_request_id:, approved:)` (returns `Task`) resumes execution; approval state is in-process only — not persisted across process restarts or shared across pods | Beta |
+| **Human-in-the-loop approval** — `Agent::Base#invoke` returns `{ suspended: true, execution_id: String, approval_request: Phronomy::Agent::ToolApprovalRequest }` when approval is required. `#approve` / `#approve_async` resume that execution. Suspended execution state is stored in Persistence, but durable activation rehydration after a process restart is not yet supported | Beta |
 | **`tool_approval_policy`** — Instance-level callable that maps each `ToolApprovalRequest` to `:allow`, `:require_approval`, or `:reject`; set on the agent instance before invoking | Beta |
 | **`Filter::Base` — unified value filter interface** — `Phronomy::Filter::Base` with a single abstract method `call(value, **context)`; apply to user input (`add_input_filter` / `input_filter` DSL), final LLM output (`add_output_filter` / `output_filter` DSL), or individual tool return values (`add_tool_result_filter(tool_class?, filter)` / `tool_result_filter` DSL); filters transform values and return the result, or raise `Phronomy::FilterBlockError` to reject; filter chains are composable; the same filter instance can be reused across all three sites | Beta |
 
@@ -99,7 +98,7 @@ The APIs listed below are intended for advanced use cases, framework internals,
 |---|---|
 | **`Phronomy::Diagnostics`** — Snapshot of scheduler internals for debug/monitoring; `SchedulerReentrancyError` raised on invalid re-entrant scheduler use; `Runtime.in_scheduler_context?` returns `true` when called from inside a scheduler task | Experimental |
 | **`Phronomy::Testing::FakeClock` / `FakeScheduler` / `SchedulerHelpers`** — Test helpers for deterministic concurrency specs: `FakeClock#advance(seconds)` controls time; `FakeScheduler` runs tasks synchronously and records `event_log`; `FakeScheduler#assert_order` / `#assert_cancelled` for ordering assertions; `FakeClock#advance_to_next_timer` fires the next pending callback; `Testing::SchedulerHelpers#with_fake_scheduler` replaces the global Runtime for the duration of a block | Beta |
-| **`Configuration#runtime_backend`** — `:thread` (default, one OS thread per task), `:immediate` (tests — tasks run synchronously, no extra threads), `:fiber` (**EXPERIMENTAL** — experimental validation backend only: runs tasks as Ruby Fibers on a cooperative scheduler to verify that framework components are truly non-blocking; **not for production use** and not a planned production replacement for `:thread`; no preemptive scheduling will be added). `:cooperative` is a **deprecated alias** for `:immediate` — do not use in new code | Beta |
+| **`Configuration#runtime_backend`** — `:thread` (default, one OS thread per task), `:immediate` (tests — tasks run synchronously, no extra threads), `:fiber` (**EXPERIMENTAL** — validation backend only: runs tasks as Ruby Fibers on a cooperative scheduler to verify that framework components are truly non-blocking; **not for production use** and not a planned production replacement for `:thread`; no preemptive scheduling will be added) | Beta |
 | **`Configuration#strict_runtime_guards`** — When `true`, calling `Agent#invoke` from inside a scheduler task raises `SchedulerReentrancyError`; when `false` (default) a warning is logged instead | Beta |
 
 ## Installation
@@ -150,6 +149,28 @@ See the [RubyLLM documentation](https://rubyllm.com) for all supported providers
 | `InvocationContext#provider_limits` | Configure the provider client directly |
 | `stream_queue_max_size` | No replacement; the shared EventLoop queue is unbounded by design. Monitor `Metrics.snapshot[:event_loop_queue_depth]` instead |
 
+### 0.16 cleanup migration
+
+The following compatibility-only APIs have been removed from the active contract:
+
+| Removed API | Current contract |
+|---|---|
+| `context_overhead` | Manifest-first assembly budgets actual mandatory + selected content |
+| Tool `on_error :return_empty` | Use `:raise` or `:suppress` |
+| `dispatch_parallel(..., force_kill:)` / `fan_out(..., force_kill:)` | Cooperative cancellation; no force-kill switch |
+| `runtime_backend = :cooperative` | Use `:thread`, `:immediate`, or experimental `:fiber` explicitly |
+| `Runtime.instance = ...` | Runtime replacement is test/internal infrastructure, not a public setter |
+| `TaskGroup.new(runtime: nil)` | Runtime is required |
+| `tools ToolA, ToolB` | Use `tools(ToolA => nil, ToolB => nil)` |
+| `CancellationToken.new(deadline: Time...)` | Use `CancellationToken.timeout_after(seconds)` for token deadlines |
+| `StaticKnowledge` / `EntityKnowledge` / `Knowledge::Base` / `Phronomy::KnowledgeSource` | Register plain persistent Knowledge with `knowledge:` or `add_knowledge` |
+| `static_knowledge*` class APIs | Persistent Knowledge belongs to Agent instances and is Journal-backed |
+| `clear_memory!` | Use `clear_knowledge!`; conversation history is controlled independently with `clear_transcript!` |
+
+The legacy `build_context` / `LlmContextWindow::Assembler` extension path is no
+longer an active API. Stateful Agent input is assembled through the canonical
+Journal → Context Policy → LLM Input Manifest pipeline.
+
 ### Optional dependencies
 
 Install additional gems only for the features you use:
@@ -176,9 +197,10 @@ class WebSearch < Phronomy::Agent::Context::Capability::Base
 end
 
 class ResearchAgent < Phronomy::Agent::Base
+  agent_definition id: "research-agent", version: 1
   model "gpt-4o"
   instructions "You are a research assistant. Use tools to answer questions."
-  tools WebSearch
+  tools(WebSearch => nil)
   max_iterations 5
 end
 
@@ -433,7 +455,7 @@ end
 class OrchestratorAgent < Phronomy::Agent::Base
   model "gpt-4o"
   instructions "Use the research tool first, then the write tool to produce a blog post."
-  tools ResearchTool, WriteTool
+  tools(ResearchTool => nil, WriteTool => nil)
 end
 
 result = OrchestratorAgent.new.invoke("Write a blog post about Ruby 3.4 features")
@@ -469,39 +491,56 @@ end
 > that logic must be implemented by the application. Reference implementations for
 > common patterns are available in `phronomy-examples` (example 06).
 
-### Knowledge — Static context injection
+### Knowledge — Persistent Agent context
+
+Knowledge is plain logical content retained by one Agent and considered by
+Context Policy for future LLM calls. It is not a separate source-object type and
+it is not automatically mandatory.
+
+Register initial Knowledge when creating the Agent:
 
 ```ruby
-# Static knowledge (policy files, reference docs)
-policy = Phronomy::Agent::Context::Knowledge::StaticKnowledge.new(
-  File.read("policy.md"),
-  type:   :policy,
-  source: "policy.md"   # exposed to LLM for citation
+policy_text = File.read("policy.md")
+
+agent = ResearchAgent.new(
+  knowledge: [
+    policy_text,
+    "Customer tier: enterprise"
+  ]
 )
+```
 
-# Inject at invocation time via the agent DSL
-class MyAgent < Phronomy::Agent::Base
-  model "gpt-4o-mini"
-  knowledge policy
-end
+Add durable Knowledge later:
+
+```ruby
+agent.add_knowledge(
+  "Customer locale: ja-JP",
+  metadata: {"origin" => "customer_profile"}
+)
 ```
 
-`static_knowledge_refresh!` invalidates the class-level cache of static knowledge sources.
-Call it when the underlying file or content has changed:
+Persistent Knowledge is Journal-backed, survives `Agent.load`, and is excluded
+from the public conversation transcript. `clear_knowledge!` logically
+invalidates earlier Knowledge while keeping the append-only Journal intact.
 
 ```ruby
-# Static knowledge sources are cached at the class level after the first fetch.
-# Call refresh! when the underlying content changes (e.g. after reloading policy.md).
-MyAgent.static_knowledge_refresh!
+agent.clear_knowledge!
 ```
 
-Load and split documents with built-in loaders:
+For request-scoped retrieval results that should not be persisted, use
+`before_llm_input` `segment_candidates` instead.
+
+Load and split documents with built-in loaders when the application needs an
+acquisition/RAG pipeline:
 
 ```ruby
 chunks = Phronomy::VectorStore::Loader::MarkdownLoader.new.load("docs/guide.md")
          .then { |docs| Phronomy::VectorStore::Splitter::RecursiveSplitter.new(chunk_size: 512).split(docs) }
 ```
 
+The application decides whether retrieved/extracted information becomes durable
+Agent Knowledge (`add_knowledge`) or per-call Context (`before_llm_input`).
+
 ### Multi-Agent Handoff — Hub-and-spoke routing
 
 ```ruby
@@ -519,26 +558,106 @@ puts result[:output]           # final answer
 puts result[:agent].class      # => BillingAgent
 ```
 
-### Before-Completion Hook — Dynamic LLM parameter injection
+### Before-LLM-Input Hook — Per-call LLM input customization
+
+`before_llm_input` runs before every LLM call and allows an application to
+customize that call without mutating the Agent, RubyLLM chat, or canonical
+Journal state directly.
+
+Hooks can be configured at three levels:
+
+1. global — applies to every Agent
+2. class — applies to every instance of one Agent definition
+3. instance — applies only to one Agent instance
+
+They run in that order: global → class → instance.
+
+A hook receives an immutable `Phronomy::Agent::LLMInputBuildContext` containing
+metadata about the LLM call:
+
+- `agent_id`
+- `agent_definition_id`
+- `definition_version`
+- `config`
+- `call_sequence`
+
+The hook does not receive the mutable Agent instance, RubyLLM messages, or
+`RubyLLM::Chat`.
+
+Return either:
+
+- `nil` to leave the call unchanged, or
+- a `Phronomy::Agent::LLMInputPatch`
+
+### Class-level hook
 
 ```ruby
-# Class-level: applies to all instances
 class MyAgent < Phronomy::Agent::Base
+  agent_definition id: "my-agent", version: 1
+
   model "gpt-4o"
-  before_completion ->(ctx) { { temperature: ctx.config[:precise] ? 0.0 : 0.7 } }
+
+  before_llm_input ->(ctx) {
+    Phronomy::Agent::LLMInputPatch.new(
+      model_config_patch: {
+        temperature: ctx.config[:precise] ? 0.0 : 0.7
+      }
+    )
+  }
 end
+```
+
+### Instance-level hook
 
-# Instance-level: overrides class hook for this agent only
+```ruby
 agent = MyAgent.new
-agent.before_completion = ->(ctx) { { max_tokens: 512 } }
 
-# Global: applies to every agent across the app
-Phronomy.configure do |c|
-  c.before_completion = ->(ctx) { { temperature: 0.3 } }
+agent.before_llm_input = ->(_ctx) {
+  Phronomy::Agent::LLMInputPatch.new(
+    model_config_patch: {
+      max_output_tokens: 512
+    }
+  )
+}
+```
+
+### Global hook
+
+```ruby
+Phronomy.configure do |config|
+  config.before_llm_input = ->(_ctx) {
+    Phronomy::Agent::LLMInputPatch.new(
+      model_config_patch: {
+        temperature: 0.3
+      }
+    )
+  }
 end
 ```
 
-Hooks are called in order — global → class → instance — and shallow-merged (`Hash#merge`; last hook wins on key conflicts).
+When multiple hooks provide `model_config_patch`, patches are merged in hook
+order and later values win on key conflicts.
+
+`LLMInputPatch` can also supply `segment_candidates` for additional per-call
+context. Those candidates enter the same Context Policy selection path as
+persistent/history candidates and are not automatically mandatory. They are
+not persisted to the Journal.
+
+```ruby
+Phronomy::Agent::LLMInputPatch.new(
+  segment_candidates: [
+    {
+      content: "The customer is on the enterprise plan.",
+      category: :knowledge,
+      role: :user
+    }
+  ]
+)
+```
+
+The Journal remains the canonical record of observed execution history and
+persistent Knowledge. `before_llm_input` customizes only the logical candidate
+set for a particular LLM call.
 
 ### GeneratorVerifier — Generator-Verifier loop with custom prompt builders
 
@@ -590,8 +709,7 @@ end
 
 ### MultiAgent::Orchestrator — Parallel subagent dispatch
 
-> **Note:** `dispatch_parallel` and `fan_out` use plain Ruby threads. Use
-> `max_concurrency:` to cap the number of concurrent workers and `on_error:`
+> **Note:** Use `max_concurrency:` to cap concurrent workers and `on_error:`
 > to control failure handling (`:raise` re-raises the first error after all
 > tasks complete; `:skip` fills failed slots with `nil`). For very large
 > fan-outs consider additional rate-limiting at the application level.
@@ -617,7 +735,6 @@ class MyOrchestrator < Phronomy::MultiAgent::Orchestrator
   instructions "Orchestrate."
 
   def run(query)
-    # Heterogeneous agents in parallel (cap at 4 threads; skip failures; 30 s timeout)
     results = dispatch_parallel(
       {agent: SearchAgent,   input: "topic A"},
       {agent: AnalysisAgent, input: query},
@@ -626,7 +743,6 @@ class MyOrchestrator < Phronomy::MultiAgent::Orchestrator
       timeout: 30
     )
 
-    # Fan-out — same agent, multiple inputs
     translations = fan_out(
       agent: TranslationAgent,
       inputs: %w[Hello World],
@@ -655,14 +771,10 @@ end
 app = Phronomy::Workflow.define(EnrichContext) do
   initial :enrich
   state :enrich, action: ->(s) do
-    # Use Thread#value to collect results safely — avoids concurrent Hash writes
     threads = {
       summary: Thread.new { Summarizer.call(s) },
       tags:    Thread.new { Tagger.call(s) }
     }
-    # For bounded waits, use Thread#join(timeout_seconds); nil means timed out — handle explicitly.
-    # Do not use Timeout.timeout or Thread#kill — both inject async exceptions that bypass cleanup.
-    # Prefer CancellationToken for cooperative cancellation of Phronomy-managed tasks.
     threads.each_value(&:join)
     s.merge(summary: threads[:summary].value, tags: Array(threads[:tags].value))
   end
@@ -675,12 +787,10 @@ state = app.invoke({}, config: { thread_id: "t1" })
 ### Output Parser — Structured LLM responses
 
 ```ruby
-# Extract JSON from LLM output (handles Markdown code fences automatically)
 parser = Phronomy::OutputParser::JsonParser.new
 data   = parser.parse('```json\n{"name":"Alice","score":0.9}\n```')
 # => { name: "Alice", score: 0.9 }
 
-# Map JSON directly to a Struct
 PersonSchema = Struct.new(:name, :age, keyword_init: true)
 parser = Phronomy::OutputParser::StructuredParser.new(PersonSchema)
 person = parser.parse('{"name":"Alice","age":30}')
@@ -703,15 +813,15 @@ runner  = Phronomy::Eval::Runner.new(
 results = runner.run(dataset, ->(q) { agent.invoke(q) })
 metrics = Phronomy::Eval::Metrics.new(results)
 
-puts "Mean score: #{metrics.mean_score}"   # Float 0.0–1.0
-puts "Pass rate:  #{metrics.pass_rate}"    # fraction with score >= threshold
+puts "Mean score: #{metrics.mean_score}"
+puts "Pass rate:  #{metrics.pass_rate}"
 ```
 
 ### Tracing — Custom observability
 
 ```ruby
 Phronomy.configure do |c|
-  c.tracer = MyCustomTracer.new  # any Phronomy::Tracing::Base subclass
+  c.tracer = MyCustomTracer.new
 end
 ```
 
@@ -735,32 +845,79 @@ child process (stdio transport) or release the HTTP connection:
 search_tool.close
 ```
 
-### Conversation History — passing prior messages
+### Agent State and Conversation History
+
+Phronomy Agents are stateful objects. Each Agent has a stable `agent_id`, a persistent Agent root, an append-only execution Journal, and zero or more Agent executions.
+
+Every concrete Agent definition must declare a stable definition identity:
+
+```ruby
+class ResearchAgent < Phronomy::Agent::Base
+  agent_definition id: "research-agent", version: 1
+
+  model "gpt-4o"
+  instructions "You are a research assistant."
+end
+```
+
+The definition ID identifies the application-level Agent definition. The version is checked when a previously persisted Agent is loaded so that persisted state is not silently interpreted by an incompatible Agent definition.
+
+Create and continue using the same Agent instance normally:
+
+```ruby
+persistence = Phronomy::Persistence::InMemory.new
+
+agent = ResearchAgent.create(
+  agent_id: "research-session-42",
+  knowledge: ["Customer tier: enterprise"],
+  persistence: persistence
+)
+
+agent.invoke("My name is Alice.")
+agent.add_knowledge("Customer locale: ja-JP")
+result = agent.invoke("What is my name?")
+
+puts result[:output]
+```
+
+Conversation history does not need to be passed back through `messages:` on every invocation. The Agent's canonical history is retained in its Journal and selected automatically when later LLM Calls are assembled.
+
+A persisted Agent can be loaded again when the same Persistence backend is available:
+
+```ruby
+agent = ResearchAgent.load(
+  "research-session-42",
+  persistence: persistence
+)
+
+result = agent.invoke("Continue our previous discussion.")
+```
+
+`result[:messages]` remains available as a materialized transcript of the Agent's current conversation history. It is a projection of canonical Agent state, not the authoritative storage mechanism and does not need to be supplied to the next `invoke`.
 
-Phronomy does not manage conversation history internally. The application owns the
-message array and passes it in via the `messages:` keyword argument:
+Existing external conversation history can be supplied when a new Agent is created:
 
 ```ruby
-# First turn
-result1 = MyAgent.new.invoke("Hello! I'm Alice.", thread_id: "session-1")
-prior_messages = result1[:messages]   # Array<RubyLLM::Message>
-
-# Second turn — pass prior messages so the agent has context
-result2 = MyAgent.new.invoke(
-  "What is my name?",
-  messages: prior_messages,
-  thread_id: "session-1"
+agent = ResearchAgent.create(
+  context: existing_messages,
+  knowledge: initial_knowledge,
+  persistence: persistence
 )
-puts result2[:output]   # => "Your name is Alice."
 ```
 
-`result[:messages]` contains the complete message history after each invocation.
-Persist it however suits your application (in-memory hash, Redis, ActiveRecord, etc.).
+Imported history must satisfy Phronomy's Import contract. User, assistant, and Tool messages are journaled without destroying their logical message boundaries. System instructions are Agent configuration and are not imported as ordinary conversation messages.
+
+`thread_id` is an execution correlation identifier. It does not identify the persistent Agent and is not a substitute for `agent_id`.
 
-> **Note on `thread_id`**: `thread_id` is a correlation identifier used internally for
-> EventLoop session routing and compaction context. It does **not** automatically persist or
-> restore conversation history — you must pass `messages:` explicitly on each turn as shown above.
+The active conversation and Knowledge views can be advanced independently without deleting the canonical Journal:
+
+```ruby
+agent.clear_transcript!  # conversation only
+agent.clear_knowledge!   # persistent Knowledge only
+agent.reset_context!     # both
+```
 
+`purge!` is different: it permanently removes the Agent and its persisted execution history from the configured Persistence backend.
 
 ## Configuration
 
@@ -769,11 +926,11 @@ Phronomy.configure do |c|
   c.default_model                   = "gpt-4o-mini"
   c.recursion_limit                 = 25
   c.tracer                          = Phronomy::Tracing::NullTracer.new
-  c.before_completion               = nil   # optional; global hook lambda
+  c.before_llm_input                = nil   # optional global before_llm_input hook
   c.trace_pii                       = false # default; set to true only when trace data contains no PII
   c.logger                          = nil   # optional; any object responding to #warn (e.g. Rails.logger)
   c.event_loop_stop_grace_seconds   = 5     # seconds to wait for sessions to drain on shutdown
-  c.runtime_backend                 = :thread   # :thread (default); :immediate (tests, synchronous); :fiber (experimental validation only); :cooperative (deprecated alias for :immediate)
+  c.runtime_backend                 = :thread   # :thread (default); :immediate (tests, synchronous); :fiber (experimental validation only)
   c.strict_runtime_guards           = false     # when true, raises SchedulerReentrancyError on invoke-inside-task
   c.stream_callback_error_policy    = :report   # :report (default) preserves Agent result; :fail_task fails Task with StreamCallbackError
 end
@@ -796,7 +953,7 @@ Understanding when to use each prevents scheduler stalls and hidden deadlocks.
 | Context | Recommended API |
 |---------|----------------|
 | Top-level application code, Rails controller, background job | `agent.invoke(input)` — blocks the calling thread until done |
-| Workflow action / EventLoop callback | `agent.invoke_async(input).map { |r| ctx.merge(output: r[:output]) }` — returns a Task and resumes by state transition |
+| Workflow action | Start `invoke_async` and use `Workflow#signal` in the `on_event:` callback to deliver the result as a later Workflow event |
 | Top-level code that wants explicit async | `agent.invoke_async(input).wait_result` — blocks the calling thread until the Task completes |
 | Streaming from top-level code | `agent.stream(input) { |event| ... }` — blocks until done; callbacks run on the EventLoop thread |
 | Streaming non-blocking | `task = agent.stream_async(input) { |event| ... }` — returns Task immediately; callbacks run on the EventLoop thread |
@@ -805,51 +962,133 @@ Understanding when to use each prevents scheduler stalls and hidden deadlocks.
 
 ### Why this matters
 
-`invoke` is a synchronous wrapper that calls `invoke_async` and then _blocks_ the calling
-thread until the task completes.  It is intended for top-level application threads such as
-Rails controller actions, CLI scripts, or background jobs.  Inside EventLoop-driven
-workflow actions, return a Task and let `Task#map` / `Task#on_complete` drive the next
-state transition instead of waiting inside the EventLoop thread.
+`invoke` is a synchronous wrapper around asynchronous Agent execution and blocks
+the calling thread until the Agent finishes. It is appropriate for top-level
+application code such as CLI commands, controller actions, or background jobs.
+
+Workflow entry and transition actions have a different contract: they are
+synchronous Run-to-Completion callbacks and must finish promptly.
+
+If a Workflow action needs an Agent or another asynchronous operation, start the
+operation asynchronously, register its completion listener, return the Workflow
+context, and use `Workflow#signal` to deliver completion as a later Workflow
+event.
+
+Do not call blocking `Agent#invoke` from an EventLoop callback, and do not return
+the `Task` from `Agent#invoke_async` as the result of a Workflow entry or
+transition action.
 
 ### Runtime guard
 
 Phronomy detects this pattern automatically:
 
 ```ruby
-# Default (soft mode): logs a warning and continues
 Phronomy.configure { |c| c.strict_runtime_guards = false }
-
-# Strict mode: raises SchedulerReentrancyError immediately
 Phronomy.configure { |c| c.strict_runtime_guards = true }
 ```
 
 You can also query the current context directly:
 
 ```ruby
-Phronomy::Runtime.in_scheduler_context?  # => true if called from inside a task
+Phronomy::Runtime.in_scheduler_context?
 ```
 
 ### Migration: blocking wait → Task mapping
 
 ```ruby
-# Top-level synchronous use
 result = my_agent.invoke("Hello")
-
-# Explicit async from top-level code
 result = my_agent.invoke_async("Hello").wait_result
+```
+
+### Async work inside a Workflow
+
+Workflow entry and transition actions are synchronous Run-to-Completion
+callbacks.
+
+They may start asynchronous work, but they must return the Workflow context (or
+`nil`). Returning a `Phronomy::Task` from an entry or transition action is an
+error.
+
+When an asynchronous Agent finishes, deliver its result back to the live
+Workflow as a later event with `Workflow#signal`.
+
+```ruby
+class AnswerContext
+  include Phronomy::WorkflowContext
+
+  field :question, type: :replace, default: ""
+  field :answer,   type: :replace, default: nil
+end
+
+workflow = nil
+
+workflow = Phronomy::Workflow.define(AnswerContext) do
+  initial :asking
+
+  state :asking
+  state :done
+
+  entry :asking, ->(ctx) {
+    thread_id = ctx.thread_id
+
+    my_agent.invoke_async(
+      ctx.question,
+      on_event: ->(event) {
+        next unless event.type == :done
+
+        workflow.signal(
+          thread_id: thread_id,
+          event: :answer_ready,
+          payload: { answer: event.payload[:output] }
+        )
+      }
+    )
 
-# Workflow action / EventLoop-safe use
-NODE = ->(ctx) {
-  my_agent.invoke_async("Hello").map { |result|
-    ctx.merge(answer: result[:output])
+    ctx
   }
-}
+
+  transition(
+    from: :asking,
+    on: :answer_ready,
+    to: :done,
+    action: ->(ctx, event) {
+      ctx.merge(answer: event.payload[:answer])
+    }
+  )
+
+  transition from: :done, to: :__finish__
+end
+```
+
+The important separation is:
+
+```text
+Workflow action
+    │
+    ├─ starts asynchronous work
+    │
+    └─ returns context immediately
+             │
+             ▼
+       asynchronous Agent
+             │
+             ▼
+      on_event / callback
+             │
+             ▼
+       Workflow#signal
+             │
+             ▼
+      later Workflow event
 ```
 
+`Task#map` remains a valid Task API for transforming Task results, but a mapped
+Task must not be returned from a Workflow entry or transition action.
+
 ### :immediate backend (synchronous / test mode)
 
 The `:immediate` backend runs tasks synchronously using `FakeScheduler`
-(backed by `Task::ImmediateBackend`).  Blocking I/O is isolated in `BlockingAdapterPool`.
+(backed by `Task::ImmediateBackend`). Blocking I/O is isolated in `BlockingAdapterPool`.
 To switch back to the default thread-per-task backend:
 
 ```ruby
@@ -863,46 +1102,79 @@ end
 
 ## Context Management
 
-Phronomy includes a context window management layer. When model metadata is
-available (either from the built-in registry or via an explicit `context_window:` setting),
-agents automatically stay within the configured token limit.
+Phronomy uses a Manifest-first context architecture for stateful Agents.
 
-### TokenBudget
+```text
+Canonical Journal
+      ↓
+Context candidates
+      ↓
+Context Policy
+      ↓
+LLM Call Manifest
+      ↓
+Runtime Projection
+      ↓
+RubyLLM / Provider
+```
 
-Derives the effective token budget from RubyLLM's model registry:
+The **Journal** is the canonical append-only record of logical execution facts
+observed by Phronomy and persistent Knowledge explicitly registered by the
+application.
 
-```ruby
-budget = Phronomy::LlmContextWindow::TokenBudget.new(
-  model:    "claude-3-5-sonnet-20241022",  # looks up context_window + max_output_tokens
-  overhead: 500                            # extra reservation for tool definitions
-)
-budget.context_window       # => 200_000
-budget.max_output_tokens    # => 8_192
-budget.effective_input_limit # => 191_308
-```
+The **Manifest** is the canonical logical input fixed for one particular LLM
+Call.
 
-Or supply explicit values (useful for local / unregistered models):
+Context-window management therefore does not trim or rewrite the Agent's
+canonical history. Phronomy selects the subset of available optional Context
+needed for each LLM Call and records that selection in the Manifest.
 
-```ruby
-budget = Phronomy::LlmContextWindow::TokenBudget.new(
-  context_window:    32_768,
-  max_output_tokens: 4_096
-)
-```
+Persistent Knowledge is an ordinary `:knowledge` Context candidate. It is not
+concatenated into the mandatory system prompt. Conversation history and
+Knowledge share the same policy/budget selection path while remaining distinct
+in public transcript semantics.
+
+Per-call `before_llm_input` segment candidates also pass through Context Policy
+and are not written to the Journal.
+
+Tool protocol dependencies are preserved during selection. An assistant message
+containing Tool Calls and the corresponding Tool-role messages are selected as
+a protocol-safe unit rather than independently pruning messages in a way that
+would create an invalid LLM conversation.
+
+When the available budget is insufficient, optional history or Knowledge can be
+omitted from the current Manifest. Required context is never silently removed
+merely to satisfy the budget. If required input cannot fit, Phronomy raises
+`ContextBudgetExceededError`.
 
-### Agent DSL extensions
+### Context-window configuration
+
+Phronomy derives the effective context budget from RubyLLM model metadata when available.
+
+For local or otherwise unregistered models, the context window can be declared explicitly:
 
 ```ruby
-class MyAgent < Phronomy::Agent::Base
-  model "gpt-4o"
-  max_output_tokens 4096   # override max_output_tokens from registry
-  context_overhead  600    # extra reservation for system prompt + tools
-  # LLM timeout/retry is configured on RubyLLM, not on the Agent class.
+class LocalAgent < Phronomy::Agent::Base
+  agent_definition id: "local-agent", version: 1
+
+  model "local-model"
+  context_window 32_768
+  max_output_tokens 4_096
 end
 ```
 
-`Agent::Base#invoke` builds a `TokenBudget` automatically. When the model is not in the
-registry the budget is silently skipped.
+`context_window` determines the model's total context capacity.
+
+`max_output_tokens` reserves capacity for the model's output.
+
+Mandatory instructions, current input and Tool definitions are budgeted from
+their actual canonical values. `context_overhead` is not part of the current
+contract.
+
+The current default Context Policy is framework-managed. Public custom Context
+Policy APIs, deterministic persistent compaction, and other advanced policy
+extension points are still evolving and should not yet be treated as stable
+application APIs.
 
 > **Note on CJK languages**: The default `TokenEstimator` uses a character-ratio heuristic
 > calibrated for ASCII/Latin text (4 chars/token). For Chinese, Japanese, and Korean text,
@@ -916,15 +1188,14 @@ registry the budget is silently skipped.
 > Phronomy::LlmContextWindow::TokenEstimator.tokenizer = ->(text) { enc.encode(text).length }
 > ```
 
-
 ### CancellationToken — Cooperative cancellation
 
 Pass a `CancellationToken` to any agent via `config: { cancellation_token: token }`.
 Cancellation is checked at multiple granular checkpoints: before the LLM call,
-after each streaming chunk, before each parallel
-tool-call batch, and after each `before_completion` hook. `CancellationError` is
-raised immediately. Phronomy does not replay the complete Agent invocation. No threads are force-killed — `ensure`
-blocks always execute.
+after each streaming chunk, before each parallel tool-call batch, and after each
+`before_llm_input` hook. `CancellationError` is raised immediately. Phronomy
+does not replay the complete Agent invocation. No threads are force-killed —
+`ensure` blocks always execute.
 
 > **Cooperative cancellation — not preemptive**
 >
@@ -932,19 +1203,14 @@ blocks always execute.
 > checkpoints listed above; it is **not** injected as a signal into a running
 > operation. This means the following are **not** interrupted mid-execution:
 >
-> - A single `KnowledgeSource#fetch` that is already blocking (e.g. HTTP call)
+> - An application retrieval/load operation that is already blocking
 > - A single `chat.ask` call that is not streaming
 > - A single `tool.execute` call that is already running
 > - Any external I/O (database query, vector search, HTTP request) inside those calls
 >
-> For deep in-flight safety, complement `CancellationToken` with per-source or
-> per-tool timeouts. Prefer library-native timeouts such as `Net::HTTP#read_timeout`,
-> database `statement_timeout`, or Redis client timeout — these signal the I/O layer
-> to abort cleanly. Avoid `Timeout.timeout` unless you understand its async-exception
-> risks: it injects `Timeout::Error` at an arbitrary execution point (the same
-> mechanism as `Thread#kill`), which Phronomy avoids by default due to resource
-> safety concerns. Ruby's GVL prevents fully preemptive cancellation without such
-> risky interruption.
+> For deep in-flight safety, complement `CancellationToken` with operation-native
+> timeouts. Prefer library-native timeouts such as `Net::HTTP#read_timeout`,
+> database `statement_timeout`, or Redis client timeout.
 
 > **`timeout_after` vs `CancellationScope.deadline_in`**
 >
@@ -970,18 +1236,16 @@ blocks always execute.
 > Phronomy does not interpret `config[:llm_timeout]`, `config[:tool_timeout]`,
 > Agent `retry_policy`, or Tool `retry_on`. Configure LLM transport behavior on
 > RubyLLM (or another adapter) and configure Tool transport behavior on the Tool's
-> HTTP/DB/MCP client. This ensures the layer capable of safely aborting the I/O owns
-> the timeout and retry semantics.
+> HTTP/DB/MCP client.
 >
 > `InvocationContext#deadline` and `cancellation_token` remain available for a
 > caller-defined root-operation boundary. They provide cooperative cancellation
 > across the Phronomy execution tree; they do not replace provider-native socket,
 > request, statement, or session timeouts.
->
+
 ```ruby
 token = Phronomy::Concurrency::CancellationToken.new
 
-# Cancel from another thread after 5 s
 Thread.new { sleep 5; token.cancel! }
 
 begin
@@ -990,15 +1254,9 @@ rescue Phronomy::CancellationError
   puts "cancelled"
 end
 
-# Hard deadline via monotonic clock (recommended — immune to NTP/DST changes)
 token = Phronomy::Concurrency::CancellationToken.timeout_after(30)
 result = MyAgent.new.invoke("...", config: { cancellation_token: token })
 
-# Hard deadline via wall-clock (legacy — still supported)
-token = Phronomy::Concurrency::CancellationToken.new(deadline: Time.now + 30)
-result = MyAgent.new.invoke("...", config: { cancellation_token: token })
-
-# Propagate to all parallel workers via dispatch_parallel / fan_out
 token = Phronomy::Concurrency::CancellationToken.new
 Thread.new { sleep 10; token.cancel! }
 
@@ -1035,7 +1293,6 @@ bundle exec ruby NN_example_name/run.rb
 | 12 | `12_prompt_template/` | Advanced prompt templates |
 | 13 | `13_mcp_http_tool/` | HTTP-based MCP tool server |
 | 14 | `14_code_review/` | Automated code review agent |
-| 16 | `16_before_completion_hook/` | Global/class/instance before_completion hooks |
 | 17 | `17_multi_agent_handoff/` | Hub-and-spoke agent routing via Runner |
 
 The following examples are **app-level demos** (Rails apps or advanced pipelines)