phronomy 0.22.0 → 0.23.0

data/docs/architecture/knowledge-and-rag.md

@@ -0,0 +1,130 @@
+> **CURRENT explanatory architecture**
+>
+> This document describes the reconciled current Phronomy system. Normative
+> architecture decisions remain in the [ADR index](../decisions/README.md);
+> source/runtime behavior remains implementation reality and does not silently
+> amend an ADR.
+
+# Knowledge and RAG
+
+## 1. Boundary
+
+Phronomy separates information acquisition from Agent Context management.
+
+```text
+Application / Tool / retrieval pipeline
+        |
+        v
+plain logical Knowledge content
+        |
+        +-- persist: Agent Journal (`kind: :knowledge`)
+        |
+        +-- request-scoped: before_llm_input candidate
+        |
+        v
+ContextPolicyInput.knowledge
+        |
+        v
+ContextPolicy
+        |
+        v
+Manifest
+```
+
+Phronomy does not define a core hierarchy of `StaticKnowledge`,
+`EntityKnowledge`, `RAGKnowledge`, or `KnowledgeSource` objects. How information
+was obtained is not a Context-selection type.
+
+Persistent Knowledge authority is defined by
+[ADR-013](../decisions/013-journal-backed-knowledge-as-context-candidates.md).
+
+## 2. Persistent Knowledge
+
+Persistent Knowledge is registered on a live Agent:
+
+```ruby
+agent = ResearchAgent.new(
+  knowledge: [
+    "Account type: enterprise",
+    "Data residency: Japan"
+  ]
+)
+
+agent.add_knowledge(
+  "Customer locale: ja-JP",
+  metadata: {"origin" => "customer_profile"}
+)
+```
+
+Content is stored through the durable content/Journal path and remains available
+after Agent reload.
+
+`clear_knowledge!` logically invalidates prior Knowledge for future Context
+selection without deleting historical Journal records. `clear_transcript!` does
+not clear Knowledge. `reset_context!` resets both relevant eligibility domains
+without rewriting old Journal facts.
+
+## 3. Request-scoped Knowledge
+
+Information needed for only one LLM Call should not be persisted merely to make
+it available to Context Policy.
+
+`before_llm_input` may return an `LLMInputPatch` with a Knowledge
+`segment_candidate`. The candidate enters the same typed Policy path and is not
+automatically journaled.
+
+See [before_llm_input](before-llm-input.md).
+
+## 4. Retrieval paths
+
+RAG can enter Agent reasoning through more than one ordinary application path.
+
+### Tool-result retrieval
+
+A Tool may query a VectorStore/external search system and return results to the
+Agent. The Tool result participates in the normal Tool protocol and can influence
+later Context through normal Journal/Context rules.
+
+### Pre-Manifest retrieval
+
+Application code may retrieve/rank content before Context finalization and supply
+selected logical Knowledge through `before_llm_input` or an Application
+`ContextPolicy`.
+
+The core does not require one vector-retrieval lifecycle to be the universal RAG
+path.
+
+## 5. Vector/embedding responsibility
+
+Vector stores, loaders, splitters, embeddings, ranking, and external retrieval
+are acquisition/integration capabilities. They are not themselves durable Agent
+Knowledge state.
+
+An application may:
+
+1. retrieve content;
+2. rank/filter it;
+3. convert it to plain logical Knowledge;
+4. persist it with `add_knowledge`, or keep it request-scoped.
+
+## 6. Selection and budget
+
+Knowledge is optional by default. Context Policy decides whether it enters one
+LLM Call and may omit it under budget pressure.
+
+Information that is structurally required for the call must be represented
+through the corresponding required Context mechanism; registration as Knowledge
+does not make content mandatory merely by existence.
+
+## 7. Trust/security boundary
+
+Phronomy does not apply a universal semantic security Filter when Knowledge is
+added or retrieved. Source validation, retrieval policy, ingestion sanitization,
+and domain-specific trust decisions are Application/Tool responsibilities.
+
+At one-call Context selection time, Application `ContextPolicy` may inspect
+Knowledge provenance/metadata and decide whether to select, derive/sanitize, or
+fail the preparation. Framework validation still enforces structural authority
+and final Manifest invariants.
+
+See [Security Boundaries](security-boundaries.md).

data/docs/architecture/multi-agent-handoff.md

@@ -0,0 +1,152 @@
+> **CURRENT explanatory architecture**
+>
+> This document describes the reconciled current Phronomy system. Normative
+> architecture decisions remain in the [ADR index](../decisions/README.md);
+> source/runtime behavior remains implementation reality and does not silently
+> amend an ADR.
+
+# Semantic Multi-Agent Handoff
+
+## 1. Meaning
+
+Handoff transfers active responsibility from one live Source Agent to one live
+Target Agent together with sufficient policy-bounded Context for the Target to
+continue.
+
+It is a control-plane operation, not an ordinary Tool-result protocol.
+
+Handoff is distinct from Agent-as-Tool delegation. Delegation performs work and
+returns control to the caller. Handoff changes the active Agent for the current
+coordination lifetime.
+
+Normative Handoff intent is
+[ADR-016](../decisions/016-semantic-multi-agent-handoff.md).
+
+## 2. Public API
+
+```ruby
+handoff = Phronomy::MultiAgent::Handoff.new(
+  source_agent: triage,
+  target_agent: billing,
+  description: "Transfer billing responsibility",
+  policy: policy
+)
+
+runner = Phronomy::MultiAgent::Runner.new(
+  main_agent: triage,
+  handoffs: [handoff]
+)
+
+result = runner.invoke("My invoice is wrong")
+```
+
+The current public Runner facade is synchronous `#invoke`. Handoff architecture
+does not add async/stream APIs merely for symmetry.
+
+## 3. Private transport
+
+Outgoing Handoffs are represented to the Source LLM through generated Tool
+schemas. The generated Tool name is private transport encoding, not Handoff
+identity or public semantic contract.
+
+Phronomy intercepts the Provider Tool Call into a typed private
+`HandoffRequest`. It does not execute an ordinary `ToolInvocation` or emit a
+sentinel Tool result.
+
+## 4. Handoff Policy
+
+`HandoffPolicy` controls what material from the effective finalized Source
+Manifest may cross the Agent boundary.
+
+```text
+current_request
+history
+knowledge
+tool_exchanges
+```
+
+Each category is required, forbidden, or selectable. Handoff selection is bounded
+to the finalized Source Manifest.
+
+## 5. Context Policy relationship
+
+```text
+Source ContextPolicy
+  -> Source LLMInputManifest
+
+HandoffPolicy
+  -> what may cross Source -> Target
+
+immutable HandoffContext
+  -> request-scoped Target material
+
+Target ContextPolicy
+  -> what enters one Target LLM Call
+```
+
+Transfer does not automatically append content to Target Journal or persistent
+Knowledge.
+
+## 6. Conversation/Tool dependencies
+
+An assistant Tool Call and its corresponding Tool-role result message(s) remain
+an indivisible conversation group across the Handoff boundary.
+
+Framework-owned semantic/category/content-format metadata is carried only through
+the trusted typed Handoff boundary.
+
+## 7. Responsibility and provenance
+
+Transferred `responsibility` is the dynamic instruction for what the Target must
+continue. `description` is the static edge/capability description. No mandatory
+generic `handoff_reason` identity field is added.
+
+Handoff Context preserves origin Agent and, where available, Journal record,
+Agent execution, LLM call, and Tool call provenance. Multi-hop transfers extend
+the transfer path while retaining original provenance.
+
+## 8. Target state ownership
+
+Handoff does not give Source authority to mutate Target canonical state. Target
+state changes only through normal Target-owned execution/mutation paths.
+
+A Target Agent execution has its own `execution_id`; Source execution identity is
+provenance/audit context, not Target execution identity.
+
+## 9. Next-turn continuity and durability
+
+The same `main_agent` instance plus the same Runtime define one coordination
+lifetime.
+
+Within that lifetime, the active Target remains active on later turns and across
+Runner-facade recreation. Runtime admission rejects racing concurrent turns for
+the same coordination lifetime.
+
+Active routing is **not durably rehydrated**. Runtime/process reset starts a new
+coordination lifetime at `main_agent`. Historical Handoff audit facts do not
+reconstruct active Target ownership.
+
+## 10. Cancellation and tracing
+
+Handoff does not create a separate cancellation domain; active Agent execution
+uses normal Agent semantics.
+
+One Runner user turn is automatically observable as `multi_agent.turn`. Source
+and Target Agent/LLM/Tool logical operations keep their own automatic spans and
+semantic IDs. Handoff adds no generic correlation identity or cross-Runtime
+parent-span guarantee.
+
+See [Tracing](tracing.md).
+
+## 11. Safety and removed API
+
+`Phronomy::MultiAgent::Runner::MAX_HANDOFFS` bounds transfers in one user turn.
+
+Not current contracts:
+
+- sentinel Handoff Tool results;
+- `Phronomy::Agent::Runner`;
+- `agents:` / `routes:` Runner configuration;
+- Agent-owned Handoff Tool registration;
+- generated Tool-name identity;
+- blanket Source history/Knowledge copying.

data/docs/architecture/persistence.md

@@ -0,0 +1,175 @@
+> **CURRENT explanatory architecture**
+>
+> This document describes the reconciled current Phronomy system. Normative
+> architecture decisions remain in the [ADR index](../decisions/README.md);
+> source/runtime behavior remains implementation reality and does not silently
+> amend an ADR.
+
+# Persistence, Durability, and Recovery
+
+## 1. Durable boundary
+
+`Phronomy::Persistence` is the single durable-state backend abstraction for
+stateful Agents and durable Workflows.
+
+```text
+Persistence
+├─ contents
+├─ agents
+├─ journals
+├─ executions
+└─ workflow_states
+```
+
+Persistence stores defined durable logical state. It is not serialization of the
+currently running Runtime object graph.
+
+Detailed custom-backend method/codec contracts belong in
+[Persistence backends](../persistence-backends.md).
+
+Normative durability vocabulary is
+[ADR-018](../decisions/018-durability-guarantees-and-failure-model.md).
+
+## 2. Durable versus Runtime-only state
+
+Durable examples include AgentRoot, Journal records, AgentExecution records,
+content/Manifest references, and durable Workflow snapshots.
+
+Runtime-only examples include FSMSession objects/IDs, AgentInvocation objects,
+Task instances/callbacks, EventLoop entries, Runtime admission entries, and
+in-flight Provider/Tool operation objects.
+
+Runtime/process loss removes Runtime-only objects but does not imply confirmed
+durable state was lost.
+
+## 3. Agent live ownership
+
+After create/load, one live Agent instance owns current mutable logical Agent
+state for that `agent_id` within one Runtime/process.
+
+During execution, EventLoop is the single writer of Phronomy-managed live Agent
+execution state.
+
+```text
+EventLoop authoritative snapshot
+  -> OffloadPool command
+  -> Persistence transaction
+  -> immutable operation-specific result
+  -> EventLoop authority validation
+  -> live apply
+```
+
+See
+[ADR-024](../decisions/024-event-loop-single-writer-agent-runtime.md).
+
+## 4. Durable transitions and conflicts
+
+Defined semantic durable transitions are atomic according to the Persistence
+transaction contract and conforming backend.
+
+Revision/watermark/CAS checks reject stale durable transitions with
+`Persistence::ConflictError` rather than silently merging/reloading competing
+state.
+
+Conflict detection is not competing-execution exclusion and cannot undo an
+external side effect already performed.
+
+## 5. Agent and Workflow admission
+
+Within one Runtime/process, Agent admission prevents prohibited competing
+top-level execution for one logical Agent owner.
+
+The canonical logical/durable Workflow identity is `workflow_instance_id`, which
+is separate from one Runtime `fsm_session_id`.
+
+Workflow same-process admission is acquired before durable hydration and retained
+through the authoritative terminal/halted save barrier. The caller-facing Task
+settles after that authoritative durable barrier.
+
+See
+[ADR-025](../decisions/025-process-local-agent-ownership-and-runtime-admission.md),
+[ADR-020](../decisions/020-canonical-workflow-instance-identity.md), and
+[ADR-026](../decisions/026-workflow-runtime-admission-and-durable-terminal-barrier.md).
+
+## 6. Cross-process guarantee
+
+Cross-process competing-execution exclusion is conditional.
+
+A multi-process deployment must provide stable routing/partitioning by logical
+identity or another coordination mechanism that establishes exclusive authority.
+Persistence CAS/revision checks alone detect stale commits; they do not prevent
+duplicate semantic execution from starting.
+
+Phronomy does not claim arbitrary external exactly-once side effects.
+
+## 7. Durable codec
+
+Durable backend exchange uses immutable
+`Phronomy::Persistence::DurableRecord` values with:
+
+```text
+record_type
+format_version
+payload
+```
+
+`format_version` is a durable representation version, not Agent definition
+version, Workflow identity, or Runtime incarnation identity.
+
+Payloads follow the canonical durable value/codec contract. A backend may use its
+own physical storage representation but must return an equivalent logical
+DurableRecord.
+
+## 8. Recovery model
+
+Recovery reconstructs logical state from confirmed durable evidence; it does not
+restore the old Runtime object graph.
+
+The shared Recovery vocabulary classifies unfinished work as:
+
+```text
+resumable
+reconcilable
+resolution_required
+```
+
+Where outcome is uncertain, Phronomy does not infer "not performed" merely from
+connection/process failure and does not blindly re-dispatch an external
+operation.
+
+Application resolution, when required, records one of:
+
+```text
+succeeded
+failed
+not_performed
+```
+
+before dependent continuation may proceed.
+
+Agent Recovery is integrated into supported `Agent.load` lifecycle semantics.
+
+## 9. Manifest authority during Recovery
+
+A finalized historical `LLMInputManifest` is reused for the Provider input it
+represents. Recovery does not rerun historical ContextPolicy code to reconstruct
+that finalized input.
+
+## 10. External-effect boundary
+
+External Provider/Tool/Application effects are outside the ordinary Phronomy
+Persistence transaction boundary (`X0` in ADR-018).
+
+Therefore:
+
+- durable-transition atomicity is not external-effect atomicity;
+- semantic IDs alone do not prevent duplicate external effects;
+- retry eligibility depends on outcome certainty plus operation-specific
+  idempotency/reconciliation contracts; and
+- arbitrary exactly-once external side-effect execution is not an unconditional
+  Phronomy guarantee.
+
+## 11. Reference backend details are not architecture
+
+InMemory implementation mechanisms do not become requirements for external
+backends unless the Backend SPI states the corresponding semantic property.

data/docs/architecture/removed/agent-context.md

@@ -0,0 +1,72 @@
+> **CURRENT negative architecture guidance**
+>
+> This document records removed Agent Context/Memory designs that must not be
+> mistaken for current contracts. Current architecture starts at
+> [Agent Context](../agent-context.md) and
+> [Context Management](../context-management.md).
+
+# Removed Agent Context and Memory Architecture
+
+## Mutable Provider chat as canonical state
+
+RubyLLM/runtime message collections are projections/materializations, not
+canonical Agent history. Canonical facts live in the append-only Journal and
+each Provider input is fixed by an `LLMInputManifest`.
+
+## Generic Memory subsystem
+
+Phronomy does not use `Memory::ConversationManager`, `Memory::WindowMemory`,
+`SummaryMemory`, or another mutable Memory object as Agent Context authority.
+
+Long-context selection/derivation belongs to Context Policy.
+
+## LlmContextWindow / ContextVersionCache
+
+The removed window assembler/cache model is not a current extension boundary.
+Context selection uses typed `ContextPolicyInput` and `ContextPlan`, with final
+canonicalization/budget validation by `ContextAssembler`.
+
+## Automatic history rewriting / Memory Compression
+
+Phronomy does not delete or rewrite canonical Journal history to fit a model
+window.
+
+Application Policy may omit optional source items for one call and create
+derived/compacted current-call content. Canonical source facts remain unchanged.
+
+There is no separate current Memory Compression subsystem and no automatic
+Default-Policy summarization LLM call.
+
+## Static / Entity / RAG Knowledge hierarchy
+
+Persistent Knowledge is Journal-backed logical Context, not a hierarchy of
+source object classes. Retrieval/vector/entity extraction are acquisition
+strategies owned by Application/Tool integrations.
+
+## Removed Context selection DSLs
+
+Not current public contracts:
+
+```text
+ContextRequest
+ContextPolicyDescriptor
+ContextPolicyRegistry
+DerivedContentSpec
+Selection::Unit
+UnitBuilder / Selector / TokenBudgetPacker
+ContextPlan#derived_contents
+ContextPlan#ordering_hints
+```
+
+## Generic Agent session/thread identity
+
+Agent Context is not keyed by a generic Application conversation/thread/session
+identity. Phronomy uses purpose-specific logical/runtime operation identities.
+
+## Cross-Agent shared history as Handoff
+
+Semantic Handoff does not merge Journal histories or automatically copy all
+Source history/Knowledge into Target durable state. It transfers policy-bounded
+immutable Context and Target ContextPolicy still decides one-call input.
+
+See [Multi-Agent Handoff](../multi-agent-handoff.md).

data/docs/architecture/security-boundaries.md

@@ -0,0 +1,173 @@
+> **CURRENT explanatory architecture**
+>
+> This document describes the reconciled current Phronomy system. Normative
+> architecture decisions remain in the [ADR index](../decisions/README.md);
+> source/runtime behavior remains implementation reality and does not silently
+> amend an ADR.
+
+# Security Boundaries
+
+## 1. Responsibility model
+
+Phronomy deliberately keeps content-policy interception, Context semantic trust,
+Tool authorization, and execution isolation as separate mechanisms.
+
+```text
+raw invocation input
+    -> input_filter
+
+Tool result
+    -> tool_result_filter
+
+possible LLM Context material
+    -> ContextPolicyInput
+    -> Application ContextPolicy
+         semantic selection / omission / derivation /
+         source-aware trust/security policy
+    -> ContextPlan
+    -> Framework structural validation
+    -> LLMInputManifest
+
+final Agent output
+    -> output_filter
+```
+
+No mechanism above is an all-purpose security layer.
+
+The current Filter contract is
+[ADR-019](../decisions/019-filter-contract-and-security-boundaries.md).
+
+## 2. Explicit Filter call sites
+
+`Phronomy::Filter::Base` may transform a value or block processing through
+`FilterBlockError`.
+
+Framework Filter call sites are explicit:
+
+```text
+input
+output
+Tool result
+```
+
+A Filter registered at one call site is not silently applied to every value that
+may later reach an LLM.
+
+In particular, Phronomy does **not** add a fourth `context_filter` /
+`add_context_filter` Framework call site.
+
+## 3. Context semantic trust belongs to Application Policy
+
+LLM Context is domain-dependent: content that is suspicious in one application
+may be exactly the content another Agent is supposed to analyze.
+
+The Framework therefore owns structural/integrity validation, while
+Application-defined `ContextPolicy` owns semantic trust decisions for one LLM
+Call.
+
+An Application Policy may:
+
+- inspect typed category/provenance/metadata;
+- omit optional material;
+- fail preparation when required material is unacceptable;
+- derive sanitized/redacted current-call material;
+- retrieve/rerank content; or
+- call a `Filter::Base` instance internally as ordinary Ruby policy logic.
+
+There is no separate Framework invocation guarantee when a Policy chooses to
+compose a Filter internally.
+
+## 4. Non-destructive transformation
+
+Context trust policy must not rewrite canonical Journal history.
+
+For optional content, transformation follows the current Context model:
+
+```text
+canonical/source item remains intact
+        |
+Policy omits it for this call
+        +
+Policy creates derived current-call item
+        |
+ContextPlan -> Manifest
+```
+
+Required material is protected by Framework invariants. A Policy cannot silently
+drop a required source item; if the required input cannot be accepted, Context
+preparation fails.
+
+## 5. Framework validation is not semantic filtering
+
+After Policy selection, Phronomy validates:
+
+- typed category/item identity;
+- required material;
+- Framework-owned metadata/provenance authority;
+- conversation and Tool-call/result dependency structure;
+- effective Tool wiring;
+- canonical representation; and
+- final token budget.
+
+These checks establish integrity, not truth, safety, trustworthiness, or domain
+appropriateness of arbitrary content.
+
+## 6. Knowledge/RAG boundary
+
+Phronomy does not add a mandatory semantic security hook to `add_knowledge`.
+Retrieval-source validation and ingestion policy remain Application/Tool
+responsibilities.
+
+Per-call use of Knowledge is decided by Context Policy.
+
+See [Knowledge and RAG](knowledge-and-rag.md).
+
+## 7. Tool approval is authorization, not sanitization
+
+Tool approval controls whether a side-effecting/capability operation may execute.
+It does not sanitize arbitrary Tool arguments/results, establish prompt-injection
+safety, or create an OS sandbox.
+
+Authorization and content policy are separate concerns.
+
+## 8. PromptInjectionFilter
+
+`Phronomy::Filter::PromptInjectionFilter` is an optional bounded heuristic for
+common patterns.
+
+It does not guarantee detection of all prompt injection, automatically inspect
+all Context sources, or turn LLM execution into a security sandbox.
+
+Applications remain responsible for domain-specific controls.
+
+## 9. Execution placement is not isolation
+
+EventLoop, OffloadPool, cooperative cancellation, and Tool `execution_mode`
+describe execution/lifecycle mechanics.
+
+They do not provide:
+
+- OS process isolation;
+- container isolation;
+- filesystem or network sandboxing;
+- privilege separation; or
+- containment of arbitrary malicious Application/Tool code.
+
+Those properties are Application/deployment responsibilities unless a future
+explicit isolation subsystem is accepted.
+
+## 10. No new security-policy SPI
+
+This architecture does not add:
+
+```text
+context_filter
+SecurityContext
+TrustLevel
+security metadata authority API
+sandbox SPI
+```
+
+Arbitrary item `metadata` remains Application data except for
+Framework-reserved control keys. Applications must not treat arbitrary metadata
+as a Framework-certified trust assertion.