phronomy 0.15.1 → 0.16.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (77) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +105 -28
  3. data/README.md +300 -75
  4. data/benchmark/bench_agent_invoke.rb +3 -0
  5. data/benchmark/bench_regression.rb +2 -18
  6. data/benchmark/bench_tool_schema.rb +1 -0
  7. data/docs/decisions/011-build-context-as-single-llm-input-authority.md +40 -1
  8. data/docs/decisions/012-canonical-execution-log-and-context-policy.md +69 -0
  9. data/lib/phronomy/agent/activation_registry.rb +28 -0
  10. data/lib/phronomy/agent/agent_execution.rb +97 -0
  11. data/lib/phronomy/agent/agent_execution_activation.rb +172 -0
  12. data/lib/phronomy/agent/agent_invocation.rb +42 -10
  13. data/lib/phronomy/agent/agent_invocation_session_builder.rb +50 -11
  14. data/lib/phronomy/agent/agent_root.rb +67 -0
  15. data/lib/phronomy/agent/async_event_api.rb +55 -475
  16. data/lib/phronomy/agent/base.rb +301 -285
  17. data/lib/phronomy/agent/concerns/before_llm_input.rb +66 -0
  18. data/lib/phronomy/agent/context_assembler.rb +321 -0
  19. data/lib/phronomy/agent/context_candidate.rb +47 -0
  20. data/lib/phronomy/agent/context_candidate_resolver.rb +65 -0
  21. data/lib/phronomy/agent/context_importer.rb +217 -0
  22. data/lib/phronomy/agent/context_parts/budget/token_budget_packer.rb +53 -0
  23. data/lib/phronomy/agent/context_parts/requirements/required_context_resolver.rb +56 -0
  24. data/lib/phronomy/agent/context_parts/selectors/recent_first_selector.rb +30 -0
  25. data/lib/phronomy/agent/context_parts/unit_builders/dependency_aware_unit_builder.rb +188 -0
  26. data/lib/phronomy/agent/context_parts/validators/final_budget_validator.rb +37 -0
  27. data/lib/phronomy/agent/context_plan.rb +25 -0
  28. data/lib/phronomy/agent/context_plan_validator.rb +167 -0
  29. data/lib/phronomy/agent/context_policies/default.rb +53 -0
  30. data/lib/phronomy/agent/context_policy.rb +15 -0
  31. data/lib/phronomy/agent/context_policy_descriptor.rb +49 -0
  32. data/lib/phronomy/agent/context_policy_registry.rb +46 -0
  33. data/lib/phronomy/agent/context_request.rb +35 -0
  34. data/lib/phronomy/agent/context_selection_unit.rb +38 -0
  35. data/lib/phronomy/agent/derived_content_spec.rb +34 -0
  36. data/lib/phronomy/agent/execution_coordinator.rb +1123 -0
  37. data/lib/phronomy/agent/fsm_runtime_adapter.rb +210 -0
  38. data/lib/phronomy/agent/immutable.rb +31 -0
  39. data/lib/phronomy/agent/journal_projection.rb +34 -0
  40. data/lib/phronomy/agent/journal_record.rb +67 -0
  41. data/lib/phronomy/agent/llm_call_record.rb +51 -0
  42. data/lib/phronomy/agent/llm_input_build_context.rb +17 -0
  43. data/lib/phronomy/agent/llm_input_manifest.rb +103 -0
  44. data/lib/phronomy/agent/llm_input_patch.rb +21 -0
  45. data/lib/phronomy/agent/phase_machine_builder.rb +12 -0
  46. data/lib/phronomy/agent/provider_call_outcome.rb +90 -0
  47. data/lib/phronomy/agent/ruby_llm_materializer.rb +298 -0
  48. data/lib/phronomy/agent/token_budget_resolver.rb +69 -0
  49. data/lib/phronomy/agent/tool_call_intercepted.rb +11 -4
  50. data/lib/phronomy/agent/tool_definition_set.rb +55 -0
  51. data/lib/phronomy/agent.rb +14 -16
  52. data/lib/phronomy/agent_busy_error.rb +5 -0
  53. data/lib/phronomy/canonical_json.rb +136 -0
  54. data/lib/phronomy/configuration.rb +9 -4
  55. data/lib/phronomy/content_store/base.rb +51 -0
  56. data/lib/phronomy/context_budget_exceeded_error.rb +8 -0
  57. data/lib/phronomy/engine/event_loop.rb +3 -0
  58. data/lib/phronomy/execution_rehydration_required_error.rb +5 -0
  59. data/lib/phronomy/invalid_context_budget_configuration_error.rb +8 -0
  60. data/lib/phronomy/llm_context_window/assembler.rb +8 -8
  61. data/lib/phronomy/multi_agent/orchestrator.rb +1 -0
  62. data/lib/phronomy/multi_agent/parallel_tool_chat.rb +7 -5
  63. data/lib/phronomy/multi_agent/team_coordinator.rb +6 -2
  64. data/lib/phronomy/persistence/in_memory.rb +247 -0
  65. data/lib/phronomy/persistence.rb +39 -0
  66. data/lib/phronomy/tools/agent.rb +14 -36
  67. data/lib/phronomy/version.rb +1 -1
  68. data/lib/phronomy.rb +11 -0
  69. data/scripts/add_to_h_to_token_doubles.rb +33 -0
  70. data/scripts/add_to_h_unnamed_doubles.rb +27 -0
  71. data/scripts/migrate_spec_agent_definition.rb +108 -0
  72. data/scripts/migrate_spec_agent_definition_pass2.rb +53 -0
  73. data/scripts/migrate_spec_inline_pass3.rb +24 -0
  74. metadata +54 -7
  75. data/lib/phronomy/agent/agent_invocation_registry.rb +0 -75
  76. data/lib/phronomy/agent/before_completion_context.rb +0 -47
  77. data/lib/phronomy/agent/concerns/before_completion.rb +0 -111
data/README.md CHANGED
@@ -27,9 +27,9 @@ It provides composable building blocks — Workflows, Agents, Tools, Filters, an
27
27
  | Feature | Stability |
28
28
  |---|---|
29
29
  | **Workflow** — Stateful, branching workflows with wait_state/send_event | Stable |
30
- | **Agent** — ReAct-style tool-calling agents with guardrails and conversation history | Stable |
31
- | **Before-Completion Hook** — Three-tier LLM parameter injection | Stable |
32
- | **Context Management** — Token budget calculation, estimation, and pruning; `Agent::Base` protected hooks: `build_context` (overridable), `trim_messages`, `trim_to_budget`, `compact_messages`, `budget_exceeded?`, `drop_messages_over` | Stable |
30
+ | **Agent** — Stateful ReAct-style tool-calling agents with stable `agent_id`, persistence-backed execution state, canonical execution history, guardrails, and conversation context | Stable |
31
+ | **Before-LLM-Input Hook** — Three-tier per-call LLM input customization via `before_llm_input` and `LLMInputPatch` | Stable |
32
+ | **Context Management** — Canonical Journal + per-LLM-call Manifest architecture with token-budget-aware selection and protocol-safe Tool Call / Tool message dependencies. Context selection never deletes canonical execution history | Stable |
33
33
  | **Filters** — Input/output transformation and blocking via `Filter::Base`; call `block!(reason)` to reject and raise `FilterBlockError` | Beta |
34
34
  | **`PromptInjectionFilter`** — Built-in `Filter::Base` subclass that detects prompt-injection patterns; usable standalone or as part of a filter chain | Beta |
35
35
  | **`Agent::Context::Capability::Base.redact_params` / `.max_result_size`** — Class-level DSL: `redact_params` masks parameter values in log/trace output; `max_result_size` truncates oversized tool results before they reach the LLM | Beta |
@@ -56,7 +56,7 @@ It provides composable building blocks — Workflows, Agents, Tools, Filters, an
56
56
  | **`stream` / `stream_async`** — callbacks execute on the EventLoop thread and must return quickly; the block form remains a compatibility alias for `on_event:` | Beta |
57
57
  | **`stream_callback_error_policy`** — Backward-compatible setting shared by `invoke_async` and `stream_async` terminal `on_event:` callbacks: `:report` (default) preserves the Agent result, while `:fail_task` fails the returned Task with `Phronomy::StreamCallbackError`; Agent execution errors are never replaced by callback errors | Beta |
58
58
  | **`invoke_async` / `call_async`** — `Agent::Base#invoke_async` and `Workflow#invoke_async` return a `Task`; `Agent::Context::Capability::Base#call_async` similarly; compatible with EventLoop and standalone contexts | Stable |
59
- | **`Task#map`** — transforms a Task's completed value and propagates failure/cancellation; Workflow entry and transition actions do not await mapped Tasks | Stable |
59
+ | **`Task#map`** — transforms a Task's completed value and propagates failure/cancellation. `Task#map` remains available for application-level Task composition, but Workflow entry and transition actions must not return a Task | Stable |
60
60
  | **CancellationToken** — Cooperative cancellation via `cancel!`/`cancelled?`/`raise_if_cancelled!`; `timeout_after(seconds)` for monotonic-clock deadlines; optional `deadline:` (wall-clock) for backward compatibility; passed as `config: { cancellation_token: token }` to agents and `dispatch_parallel`; injected into `tool.execute` when the method declares a `cancellation_token:` keyword; bridged to `MCP::Cancellation` in `Phronomy::Tools::Mcp#execute` | Experimental |
61
61
  | **`dispatch_parallel` / `fan_out` `force_kill:` option** — `force_kill: false` (default) leaves timed-out workers running and raises `TimeoutError` immediately; `force_kill: true` restores the old `Thread#kill` behaviour with a `logger.warn` | Beta |
62
62
  | **`execution_mode` DSL on `Agent::Context::Capability::Base`** — Declares how a tool's `execute` should be dispatched: `:cooperative` (same scheduler thread), `:blocking_io` (default; offloaded to `BlockingAdapterPool`), `:cpu_bound`, `:external_process`; Tool-specific timeout/retry belongs to the Tool implementation or its client | Experimental |
@@ -78,7 +78,7 @@ It provides composable building blocks — Workflows, Agents, Tools, Filters, an
78
78
  | **`Phronomy::MultiAgent::Orchestrator`** — Parallel subagent dispatch, fan-out, and `subagent` DSL | Beta |
79
79
  | **`Phronomy::MultiAgent::TeamCoordinator`** — Agent teams pattern: LLM coordinator + stateful workers with sequential task assignment (worker-local message history persisted across tasks) | Beta |
80
80
  | **Agent::SharedState** — Shared state pattern: peer agents collaborate via a shared KnowledgeStore; `member` DSL with per-agent instructions and `coordination` team protocol | Experimental |
81
- | **Human-in-the-loop approval** — `Agent::Base#invoke` returns `{ suspended: true, agent_invocation_id: String, approval_request: Phronomy::Agent::ToolApprovalRequest }` when a tool requiring approval is encountered; `Agent::Base#approve(id, approval_request_id:, approved:)` (synchronous) or `Agent::Base#approve_async(id, approval_request_id:, approved:)` (returns `Task`) resumes execution; approval state is in-process only not persisted across process restarts or shared across pods | Beta |
81
+ | **Human-in-the-loop approval** — `Agent::Base#invoke` returns `{ suspended: true, execution_id: String, approval_request: Phronomy::Agent::ToolApprovalRequest }` when approval is required. `#approve` / `#approve_async` resume that execution. Suspended execution state is stored in Persistence, but durable activation rehydration after a process restart is not yet supported | Beta |
82
82
  | **`tool_approval_policy`** — Instance-level callable that maps each `ToolApprovalRequest` to `:allow`, `:require_approval`, or `:reject`; set on the agent instance before invoking | Beta |
83
83
  | **`Filter::Base` — unified value filter interface** — `Phronomy::Filter::Base` with a single abstract method `call(value, **context)`; apply to user input (`add_input_filter` / `input_filter` DSL), final LLM output (`add_output_filter` / `output_filter` DSL), or individual tool return values (`add_tool_result_filter(tool_class?, filter)` / `tool_result_filter` DSL); filters transform values and return the result, or raise `Phronomy::FilterBlockError` to reject; filter chains are composable; the same filter instance can be reused across all three sites | Beta |
84
84
 
@@ -176,6 +176,7 @@ class WebSearch < Phronomy::Agent::Context::Capability::Base
176
176
  end
177
177
 
178
178
  class ResearchAgent < Phronomy::Agent::Base
179
+ agent_definition id: "research-agent", version: 1
179
180
  model "gpt-4o"
180
181
  instructions "You are a research assistant. Use tools to answer questions."
181
182
  tools WebSearch
@@ -519,26 +520,106 @@ puts result[:output] # final answer
519
520
  puts result[:agent].class # => BillingAgent
520
521
  ```
521
522
 
522
- ### Before-Completion Hook — Dynamic LLM parameter injection
523
+ ### Before-LLM-Input Hook — Per-call LLM input customization
524
+
525
+ `before_llm_input` runs before every LLM call and allows an application to
526
+ customize that call without mutating the Agent, RubyLLM chat, or canonical
527
+ Journal state directly.
528
+
529
+ Hooks can be configured at three levels:
530
+
531
+ 1. global — applies to every Agent
532
+ 2. class — applies to every instance of one Agent definition
533
+ 3. instance — applies only to one Agent instance
534
+
535
+ They run in that order: global → class → instance.
536
+
537
+ A hook receives an immutable `Phronomy::Agent::LLMInputBuildContext` containing
538
+ metadata about the LLM call:
539
+
540
+ - `agent_id`
541
+ - `agent_definition_id`
542
+ - `definition_version`
543
+ - `config`
544
+ - `call_sequence`
545
+
546
+ The hook does not receive the mutable Agent instance, RubyLLM messages, or
547
+ `RubyLLM::Chat`.
548
+
549
+ Return either:
550
+
551
+ - `nil` to leave the call unchanged, or
552
+ - a `Phronomy::Agent::LLMInputPatch`
553
+
554
+ ### Class-level hook
523
555
 
524
556
  ```ruby
525
- # Class-level: applies to all instances
526
557
  class MyAgent < Phronomy::Agent::Base
558
+ agent_definition id: "my-agent", version: 1
559
+
527
560
  model "gpt-4o"
528
- before_completion ->(ctx) { { temperature: ctx.config[:precise] ? 0.0 : 0.7 } }
561
+
562
+ before_llm_input ->(ctx) {
563
+ Phronomy::Agent::LLMInputPatch.new(
564
+ model_config_patch: {
565
+ temperature: ctx.config[:precise] ? 0.0 : 0.7
566
+ }
567
+ )
568
+ }
529
569
  end
570
+ ```
530
571
 
531
- # Instance-level: overrides class hook for this agent only
572
+ ### Instance-level hook
573
+
574
+ ```ruby
532
575
  agent = MyAgent.new
533
- agent.before_completion = ->(ctx) { { max_tokens: 512 } }
534
576
 
535
- # Global: applies to every agent across the app
536
- Phronomy.configure do |c|
537
- c.before_completion = ->(ctx) { { temperature: 0.3 } }
577
+ agent.before_llm_input = ->(_ctx) {
578
+ Phronomy::Agent::LLMInputPatch.new(
579
+ model_config_patch: {
580
+ max_output_tokens: 512
581
+ }
582
+ )
583
+ }
584
+ ```
585
+
586
+ ### Global hook
587
+
588
+ ```ruby
589
+ Phronomy.configure do |config|
590
+ config.before_llm_input = ->(_ctx) {
591
+ Phronomy::Agent::LLMInputPatch.new(
592
+ model_config_patch: {
593
+ temperature: 0.3
594
+ }
595
+ )
596
+ }
538
597
  end
539
598
  ```
540
599
 
541
- Hooks are called in order global → class → instance — and shallow-merged (`Hash#merge`; last hook wins on key conflicts).
600
+ When multiple hooks provide `model_config_patch`, patches are merged in hook
601
+ order and later values win on key conflicts.
602
+
603
+ `LLMInputPatch` can also supply `segment_candidates` for additional per-call
604
+ context. Those segments participate in Manifest-first input assembly. This is
605
+ intended for logical context supplied by the application; applications should
606
+ not mutate RubyLLM message history directly.
607
+
608
+ ```ruby
609
+ Phronomy::Agent::LLMInputPatch.new(
610
+ segment_candidates: [
611
+ {
612
+ content: "The customer is on the enterprise plan.",
613
+ category: :knowledge,
614
+ role: :user
615
+ }
616
+ ]
617
+ )
618
+ ```
619
+
620
+ The Journal remains the canonical record of observed execution history.
621
+ `before_llm_input` customizes the logical input assembled for a particular LLM
622
+ call; it does not rewrite previously recorded Journal history.
542
623
 
543
624
  ### GeneratorVerifier — Generator-Verifier loop with custom prompt builders
544
625
 
@@ -735,32 +816,78 @@ child process (stdio transport) or release the HTTP connection:
735
816
  search_tool.close
736
817
  ```
737
818
 
738
- ### Conversation History passing prior messages
819
+ ### Agent State and Conversation History
820
+
821
+ Phronomy Agents are stateful objects. Each Agent has a stable `agent_id`, a persistent Agent root, an append-only execution Journal, and zero or more Agent executions.
822
+
823
+ Every concrete Agent definition must declare a stable definition identity:
824
+
825
+ ```ruby
826
+ class ResearchAgent < Phronomy::Agent::Base
827
+ agent_definition id: "research-agent", version: 1
828
+
829
+ model "gpt-4o"
830
+ instructions "You are a research assistant."
831
+ end
832
+ ```
833
+
834
+ The definition ID identifies the application-level Agent definition. The version is checked when a previously persisted Agent is loaded so that persisted state is not silently interpreted by an incompatible Agent definition.
739
835
 
740
- Phronomy does not manage conversation history internally. The application owns the
741
- message array and passes it in via the `messages:` keyword argument:
836
+ Create and continue using the same Agent instance normally:
742
837
 
743
838
  ```ruby
744
- # First turn
745
- result1 = MyAgent.new.invoke("Hello! I'm Alice.", thread_id: "session-1")
746
- prior_messages = result1[:messages] # Array<RubyLLM::Message>
747
-
748
- # Second turn — pass prior messages so the agent has context
749
- result2 = MyAgent.new.invoke(
750
- "What is my name?",
751
- messages: prior_messages,
752
- thread_id: "session-1"
839
+ persistence = Phronomy::Persistence::InMemory.new
840
+
841
+ agent = ResearchAgent.create(
842
+ agent_id: "research-session-42",
843
+ persistence: persistence
753
844
  )
754
- puts result2[:output] # => "Your name is Alice."
845
+
846
+ agent.invoke("My name is Alice.")
847
+ result = agent.invoke("What is my name?")
848
+
849
+ puts result[:output]
755
850
  ```
756
851
 
757
- `result[:messages]` contains the complete message history after each invocation.
758
- Persist it however suits your application (in-memory hash, Redis, ActiveRecord, etc.).
852
+ Conversation history does not need to be passed back through `messages:` on every invocation. The Agent's canonical history is retained in its Journal and selected automatically when later LLM Calls are assembled.
853
+
854
+ A persisted Agent can be loaded again when the same Persistence backend is available:
855
+
856
+ ```ruby
857
+ agent = ResearchAgent.load(
858
+ "research-session-42",
859
+ persistence: persistence
860
+ )
861
+
862
+ result = agent.invoke("Continue our previous discussion.")
863
+ ```
864
+
865
+ `result[:messages]` remains available as a materialized transcript of the Agent's current conversation history. It is a projection of canonical Agent state, not the authoritative storage mechanism and does not need to be supplied to the next `invoke`.
866
+
867
+ Existing external conversation history can be supplied when a new Agent is created:
868
+
869
+ ```ruby
870
+ agent = ResearchAgent.create(
871
+ context: existing_messages,
872
+ persistence: persistence
873
+ )
874
+ ```
759
875
 
760
- > **Note on `thread_id`**: `thread_id` is a correlation identifier used internally for
761
- > EventLoop session routing and compaction context. It does **not** automatically persist or
762
- > restore conversation history — you must pass `messages:` explicitly on each turn as shown above.
876
+ Imported history must satisfy Phronomy's Import contract. User, assistant, and Tool messages are journaled without destroying their logical message boundaries. System instructions are Agent configuration and are not imported as ordinary conversation messages.
763
877
 
878
+ `thread_id` is an execution correlation identifier. It does not identify the persistent Agent and is not a substitute for `agent_id`.
879
+
880
+ The current conversation or memory view can be advanced without deleting the canonical Journal:
881
+
882
+ ```ruby
883
+ agent.clear_transcript!
884
+ agent.clear_memory!
885
+ agent.reset_context!
886
+ ```
887
+
888
+ These operations change which historical records belong to the active context generation. The underlying append-only Journal remains intact.
889
+
890
+ `purge!` is different: it permanently removes the Agent and its persisted execution history from the configured Persistence backend.
764
891
 
765
892
  ## Configuration
766
893
 
@@ -769,7 +896,7 @@ Phronomy.configure do |c|
769
896
  c.default_model = "gpt-4o-mini"
770
897
  c.recursion_limit = 25
771
898
  c.tracer = Phronomy::Tracing::NullTracer.new
772
- c.before_completion = nil # optional; global hook lambda
899
+ c.before_llm_input = nil # optional global before_llm_input hook
773
900
  c.trace_pii = false # default; set to true only when trace data contains no PII
774
901
  c.logger = nil # optional; any object responding to #warn (e.g. Rails.logger)
775
902
  c.event_loop_stop_grace_seconds = 5 # seconds to wait for sessions to drain on shutdown
@@ -796,7 +923,7 @@ Understanding when to use each prevents scheduler stalls and hidden deadlocks.
796
923
  | Context | Recommended API |
797
924
  |---------|----------------|
798
925
  | Top-level application code, Rails controller, background job | `agent.invoke(input)` — blocks the calling thread until done |
799
- | Workflow action / EventLoop callback | `agent.invoke_async(input).map { |r| ctx.merge(output: r[:output]) }` returns a Task and resumes by state transition |
926
+ | Workflow action | Start `invoke_async` and use `Workflow#signal` in the `on_event:` callback to deliver the result as a later Workflow event |
800
927
  | Top-level code that wants explicit async | `agent.invoke_async(input).wait_result` — blocks the calling thread until the Task completes |
801
928
  | Streaming from top-level code | `agent.stream(input) { |event| ... }` — blocks until done; callbacks run on the EventLoop thread |
802
929
  | Streaming non-blocking | `task = agent.stream_async(input) { |event| ... }` — returns Task immediately; callbacks run on the EventLoop thread |
@@ -805,11 +932,21 @@ Understanding when to use each prevents scheduler stalls and hidden deadlocks.
805
932
 
806
933
  ### Why this matters
807
934
 
808
- `invoke` is a synchronous wrapper that calls `invoke_async` and then _blocks_ the calling
809
- thread until the task completes. It is intended for top-level application threads such as
810
- Rails controller actions, CLI scripts, or background jobs. Inside EventLoop-driven
811
- workflow actions, return a Task and let `Task#map` / `Task#on_complete` drive the next
812
- state transition instead of waiting inside the EventLoop thread.
935
+ `invoke` is a synchronous wrapper around asynchronous Agent execution and blocks
936
+ the calling thread until the Agent finishes. It is appropriate for top-level
937
+ application code such as CLI commands, controller actions, or background jobs.
938
+
939
+ Workflow entry and transition actions have a different contract: they are
940
+ synchronous Run-to-Completion callbacks and must finish promptly.
941
+
942
+ If a Workflow action needs an Agent or another asynchronous operation, start the
943
+ operation asynchronously, register its completion listener, return the Workflow
944
+ context, and use `Workflow#signal` to deliver completion as a later Workflow
945
+ event.
946
+
947
+ Do not call blocking `Agent#invoke` from an EventLoop callback, and do not return
948
+ the `Task` from `Agent#invoke_async` as the result of a Workflow entry or
949
+ transition action.
813
950
 
814
951
  ### Runtime guard
815
952
 
@@ -837,15 +974,93 @@ result = my_agent.invoke("Hello")
837
974
 
838
975
  # Explicit async from top-level code
839
976
  result = my_agent.invoke_async("Hello").wait_result
977
+ ```
978
+
979
+ ### Async work inside a Workflow
980
+
981
+ Workflow entry and transition actions are synchronous Run-to-Completion
982
+ callbacks.
840
983
 
841
- # Workflow action / EventLoop-safe use
842
- NODE = ->(ctx) {
843
- my_agent.invoke_async("Hello").map { |result|
844
- ctx.merge(answer: result[:output])
984
+ They may start asynchronous work, but they must return the Workflow context (or
985
+ `nil`). Returning a `Phronomy::Task` from an entry or transition action is an
986
+ error.
987
+
988
+ When an asynchronous Agent finishes, deliver its result back to the live
989
+ Workflow as a later event with `Workflow#signal`.
990
+
991
+ ```ruby
992
+ class AnswerContext
993
+ include Phronomy::WorkflowContext
994
+
995
+ field :question, type: :replace, default: ""
996
+ field :answer, type: :replace, default: nil
997
+ end
998
+
999
+ workflow = nil
1000
+
1001
+ workflow = Phronomy::Workflow.define(AnswerContext) do
1002
+ initial :asking
1003
+
1004
+ state :asking
1005
+ state :done
1006
+
1007
+ entry :asking, ->(ctx) {
1008
+ thread_id = ctx.thread_id
1009
+
1010
+ my_agent.invoke_async(
1011
+ ctx.question,
1012
+ on_event: ->(event) {
1013
+ next unless event.type == :done
1014
+
1015
+ workflow.signal(
1016
+ thread_id: thread_id,
1017
+ event: :answer_ready,
1018
+ payload: { answer: event.payload[:output] }
1019
+ )
1020
+ }
1021
+ )
1022
+
1023
+ ctx
845
1024
  }
846
- }
1025
+
1026
+ transition(
1027
+ from: :asking,
1028
+ on: :answer_ready,
1029
+ to: :done,
1030
+ action: ->(ctx, event) {
1031
+ ctx.merge(answer: event.payload[:answer])
1032
+ }
1033
+ )
1034
+
1035
+ transition from: :done, to: :__finish__
1036
+ end
847
1037
  ```
848
1038
 
1039
+ The important separation is:
1040
+
1041
+ ```text
1042
+ Workflow action
1043
+
1044
+ ├─ starts asynchronous work
1045
+
1046
+ └─ returns context immediately
1047
+
1048
+
1049
+ asynchronous Agent
1050
+
1051
+
1052
+ on_event / callback
1053
+
1054
+
1055
+ Workflow#signal
1056
+
1057
+
1058
+ later Workflow event
1059
+ ```
1060
+
1061
+ `Task#map` remains a valid Task API for transforming Task results, but a mapped
1062
+ Task must not be returned from a Workflow entry or transition action.
1063
+
849
1064
  ### :immediate backend (synchronous / test mode)
850
1065
 
851
1066
  The `:immediate` backend runs tasks synchronously using `FakeScheduler`
@@ -863,46 +1078,57 @@ end
863
1078
 
864
1079
  ## Context Management
865
1080
 
866
- Phronomy includes a context window management layer. When model metadata is
867
- available (either from the built-in registry or via an explicit `context_window:` setting),
868
- agents automatically stay within the configured token limit.
869
-
870
- ### TokenBudget
1081
+ Phronomy uses a Manifest-first context architecture for stateful Agents.
871
1082
 
872
- Derives the effective token budget from RubyLLM's model registry:
1083
+ The main flow is:
873
1084
 
874
- ```ruby
875
- budget = Phronomy::LlmContextWindow::TokenBudget.new(
876
- model: "claude-3-5-sonnet-20241022", # looks up context_window + max_output_tokens
877
- overhead: 500 # extra reservation for tool definitions
878
- )
879
- budget.context_window # => 200_000
880
- budget.max_output_tokens # => 8_192
881
- budget.effective_input_limit # => 191_308
1085
+ ```text
1086
+ Canonical Journal
1087
+
1088
+ Context Policy
1089
+
1090
+ LLM Call Manifest
1091
+
1092
+ Runtime Projection
1093
+
1094
+ RubyLLM / Provider
882
1095
  ```
883
1096
 
884
- Or supply explicit values (useful for local / unregistered models):
1097
+ The **Journal** is the canonical append-only record of logical execution facts observed by Phronomy.
885
1098
 
886
- ```ruby
887
- budget = Phronomy::LlmContextWindow::TokenBudget.new(
888
- context_window: 32_768,
889
- max_output_tokens: 4_096
890
- )
891
- ```
1099
+ The **Manifest** is the canonical logical input fixed for one particular LLM Call.
1100
+
1101
+ Context-window management therefore does not trim or rewrite the Agent's canonical history. Instead, Phronomy selects the subset of available context needed for each LLM Call and records that selection in the Manifest.
1102
+
1103
+ This distinction allows old history to remain available even when it does not fit in the current model's context window.
1104
+
1105
+ Tool protocol dependencies are preserved during selection. For example, an assistant message containing Tool Calls and the corresponding Tool-role messages are selected as a protocol-safe unit rather than independently pruning messages in a way that would create an invalid LLM conversation.
1106
+
1107
+ When the available budget is insufficient, optional historical context can be omitted from the current Manifest. Required context is never silently removed merely to satisfy the budget. If the required input cannot fit, Phronomy raises `ContextBudgetExceededError`.
1108
+
1109
+ ### Context-window configuration
892
1110
 
893
- ### Agent DSL extensions
1111
+ Phronomy derives the effective context budget from RubyLLM model metadata when available.
1112
+
1113
+ For local or otherwise unregistered models, the context window can be declared explicitly:
894
1114
 
895
1115
  ```ruby
896
- class MyAgent < Phronomy::Agent::Base
897
- model "gpt-4o"
898
- max_output_tokens 4096 # override max_output_tokens from registry
899
- context_overhead 600 # extra reservation for system prompt + tools
900
- # LLM timeout/retry is configured on RubyLLM, not on the Agent class.
1116
+ class LocalAgent < Phronomy::Agent::Base
1117
+ agent_definition id: "local-agent", version: 1
1118
+
1119
+ model "local-model"
1120
+ context_window 32_768
1121
+ max_output_tokens 4_096
901
1122
  end
902
1123
  ```
903
1124
 
904
- `Agent::Base#invoke` builds a `TokenBudget` automatically. When the model is not in the
905
- registry the budget is silently skipped.
1125
+ `context_window` determines the model's total context capacity.
1126
+
1127
+ `max_output_tokens` reserves capacity for the model's output.
1128
+
1129
+ The legacy `context_overhead` setting remains for compatibility with the legacy `build_context` path, but it is not the mechanism used to reserve system-prompt or Tool-definition space in Manifest-first context assembly. New Agent implementations should not rely on `context_overhead` for that purpose.
1130
+
1131
+ The current default Context Policy is framework-managed. Public custom Context Policy APIs, deterministic persistent compaction, and other advanced policy extension points are still evolving and should not yet be treated as stable application APIs.
906
1132
 
907
1133
  > **Note on CJK languages**: The default `TokenEstimator` uses a character-ratio heuristic
908
1134
  > calibrated for ASCII/Latin text (4 chars/token). For Chinese, Japanese, and Korean text,
@@ -922,7 +1148,7 @@ registry the budget is silently skipped.
922
1148
  Pass a `CancellationToken` to any agent via `config: { cancellation_token: token }`.
923
1149
  Cancellation is checked at multiple granular checkpoints: before the LLM call,
924
1150
  after each streaming chunk, before each parallel
925
- tool-call batch, and after each `before_completion` hook. `CancellationError` is
1151
+ tool-call batch, and after each `before_llm_input` hook. `CancellationError` is
926
1152
  raised immediately. Phronomy does not replay the complete Agent invocation. No threads are force-killed — `ensure`
927
1153
  blocks always execute.
928
1154
 
@@ -1035,7 +1261,6 @@ bundle exec ruby NN_example_name/run.rb
1035
1261
  | 12 | `12_prompt_template/` | Advanced prompt templates |
1036
1262
  | 13 | `13_mcp_http_tool/` | HTTP-based MCP tool server |
1037
1263
  | 14 | `14_code_review/` | Automated code review agent |
1038
- | 16 | `16_before_completion_hook/` | Global/class/instance before_completion hooks |
1039
1264
  | 17 | `17_multi_agent_handoff/` | Hub-and-spoke agent routing via Runner |
1040
1265
 
1041
1266
  The following examples are **app-level demos** (Rails apps or advanced pipelines)
@@ -40,6 +40,7 @@ class BenchStubChat
40
40
  def with_cache_instructions(_) = self
41
41
  def with_output_schema(_) = self
42
42
  def on_tool_call(&) = self
43
+ def before_tool_call(&) = self
43
44
  def last_message = @response
44
45
 
45
46
  def ask(_)
@@ -71,12 +72,14 @@ BENCH_RESP = BenchAgentMessage.assistant("benchmark complete")
71
72
  BENCH_RESP_CHAT = BenchStubChat.new(BENCH_RESP)
72
73
 
73
74
  bench_minimal_class = Class.new(Phronomy::Agent::Base) do
75
+ agent_definition id: "bench-minimal", version: 1
74
76
  model "stub-model"
75
77
 
76
78
  define_method(:build_chat) { |*| BenchStubChat.new(BENCH_RESP) }
77
79
  end
78
80
 
79
81
  bench_tool_class = Class.new(Phronomy::Agent::Base) do
82
+ agent_definition id: "bench-tool", version: 1
80
83
  model "stub-model"
81
84
  tools BenchNullTool
82
85
 
@@ -88,6 +88,7 @@ end
88
88
  # Target 4: Orchestrator#dispatch_parallel overhead (10 stub agents, no LLM)
89
89
  # ---------------------------------------------------------------------------
90
90
  stub_agent_class = Class.new(Phronomy::Agent::Base) do
91
+ agent_definition id: "bench-stub", version: 1
91
92
  define_method(:invoke) do |_input, messages: [], thread_id: nil, config: {}|
92
93
  {output: "stub", messages: []}
93
94
  end
@@ -129,22 +130,6 @@ t6 = Benchmark.measure("CancellationToken#raise_if_cancelled! (no-op)") do
129
130
  RAISE_ITERATIONS.times { RAISE_TOKEN.raise_if_cancelled! }
130
131
  end
131
132
 
132
- # ---------------------------------------------------------------------------
133
- # Target 7: Agent::Base#trim_messages on a 2000-message history
134
- # ---------------------------------------------------------------------------
135
- BenchMsg = Struct.new(:content) unless defined?(BenchMsg)
136
-
137
- TRIM_MESSAGES = Array.new(2_000) { |i| BenchMsg.new("msg #{i}") }
138
- TRIM_ITERATIONS = 500
139
-
140
- bench_trim_agent = Class.new(Phronomy::Agent::Base).new
141
-
142
- t7 = Benchmark.measure("Agent::Base#trim_messages (2000-msg history)") do
143
- TRIM_ITERATIONS.times do
144
- bench_trim_agent.send(:trim_messages, TRIM_MESSAGES, keep: 1_800)
145
- end
146
- end
147
-
148
133
  # ---------------------------------------------------------------------------
149
134
  # Print results and store in REGRESSION_RESULTS
150
135
  # ---------------------------------------------------------------------------
@@ -158,8 +143,7 @@ metrics = {
158
143
  "tool_params_schema_definition" => [t3, REGRESSION_ITERATIONS],
159
144
  "dispatch_parallel_10" => [t4, PARALLEL_ITERATIONS],
160
145
  "cancellation_token_cancelled" => [t5, 8 * CANCEL_ITERATIONS],
161
- "cancellation_token_raise_if_cancelled_noop" => [t6, RAISE_ITERATIONS],
162
- "trim_messages_2000" => [t7, TRIM_ITERATIONS]
146
+ "cancellation_token_raise_if_cancelled_noop" => [t6, RAISE_ITERATIONS]
163
147
  }
164
148
 
165
149
  REGRESSION_RESULTS = {} # rubocop:disable Style/MutableConstant
@@ -54,6 +54,7 @@ class BenchKnowledgeSource < Phronomy::Agent::Context::Knowledge::Base
54
54
  end
55
55
 
56
56
  class BenchAgentWithKnowledge < Phronomy::Agent::Base
57
+ agent_definition id: "bench-knowledge", version: 1
57
58
  model "gpt-4o-mini"
58
59
  static_knowledge BenchKnowledgeSource.new
59
60
  end
@@ -2,7 +2,46 @@
2
2
 
3
3
  ## Status
4
4
 
5
- Proposed — 2026-05-31
5
+ Partially Superseded by ADR-012 — 2026-08-08
6
+
7
+ Originally proposed — 2026-05-31.
8
+
9
+ ## Supersession Note
10
+
11
+ ADR-012, **Canonical Complete Execution Log and Context Policy**, supersedes the architectural parts of this ADR that treat the legacy `build_context` / `LlmContextWindow::Assembler` path as the long-term single authority for LLM input.
12
+
13
+ In particular, the following parts of this ADR are no longer normative for the stateful Agent architecture:
14
+
15
+ * **D1 — `build_context` is the single authority for all LLM input**
16
+ * **D2 — Assembler handles all four regions including Capability**
17
+ * **D3 — `build_context` includes all tools**
18
+ * **D5 — Previous context stored as an instance variable**
19
+
20
+ The replacement authority model is:
21
+
22
+ ```text
23
+ Canonical Journal
24
+
25
+ Context Policy
26
+
27
+ LLM Call Manifest
28
+
29
+ Runtime Projection
30
+
31
+ RubyLLM / Provider
32
+ ```
33
+
34
+ Under ADR-012:
35
+
36
+ * the **Journal** is authoritative for logical execution facts observed by Phronomy;
37
+ * the **Manifest** is authoritative for the logical input fixed for one specific LLM Call;
38
+ * Context Policy determines which canonical history is selected for that Manifest;
39
+ * selection, pruning, and compaction do not delete or rewrite the canonical Journal;
40
+ * Tool protocol dependencies are preserved independently from semantic Context-selection policy.
41
+
42
+ The problem statements and historical analysis in this ADR remain useful as design history. Decisions or implementation notes that do not conflict with ADR-012 may still describe valid constraints, but ADR-012 is authoritative whenever the two documents differ.
43
+
44
+ See: `012-canonical-execution-log-and-context-policy.md`
6
45
 
7
46
  ## Context
8
47