robot_lab 0.3.0 → 0.5.0

data/docs/architecture/index.md

@@ -1,252 +0,0 @@
-# Architecture Overview
-
-RobotLab is designed around a few core architectural principles that enable flexible, composable AI workflows.
-
-## Design Philosophy
-
-### 1. Separation of Concerns
-
-Each component has a single, well-defined responsibility:
-
-- **Robot**: LLM-powered agent (subclass of `RubyLLM::Agent`) with personality, tools, and memory
-- **Network**: Orchestrates robot execution as a DAG pipeline via SimpleFlow
-- **Memory**: Reactive key-value store for robot and network data
-- **Tool**: Provides external capabilities to robots (inherits from `RubyLLM::Tool`)
-- **Task**: Wraps a robot for pipeline execution with per-task configuration
-
-### 2. Composability
-
-Components are designed to be mixed and matched:
-
-- Robots can be used standalone or within networks
-- Tools can be shared across robots or scoped per-robot via `local_tools:`
-- Networks define DAG pipelines with sequential, parallel, and optional execution
-- Memory can be standalone (per-robot) or shared (per-network)
-- `with_*` methods return `self` for fluent chaining
-
-### 3. Provider Agnostic
-
-RobotLab abstracts away LLM provider differences through RubyLLM:
-
-- Unified interface across Anthropic, OpenAI, Gemini, DeepSeek, Mistral, and others
-- Consistent tool calling interface
-- Automatic provider detection from model names
-- Easy switching between providers via configuration
-
-## System Architecture
-
-```mermaid
-graph TB
-    subgraph "Application Layer"
-        A[Your Application]
-    end
-
-    subgraph "RobotLab Core"
-        B[Network]
-        C[Task]
-        D[Robot < RubyLLM::Agent]
-        E[Memory]
-        F[RobotResult]
-    end
-
-    subgraph "Configuration"
-        G[Config < MywayConfig::Base]
-        R[RunConfig]
-    end
-
-    subgraph "Cross-Cutting"
-        HK[Hooks<br/>HookRegistry x3]
-    end
-
-    subgraph "Integration Layer"
-        H[MCP Client]
-        I[Tools &lt; RubyLLM::Tool]
-        J[Templates / prompt_manager]
-        SK[AgentSkills<br/>+ optional Sandbox]
-    end
-
-    subgraph "Execution Layer"
-        K[SimpleFlow::Pipeline]
-        L[RubyLLM Chat]
-        TB[TypedBus + BusPoller]
-    end
-
-    subgraph "Provider Layer"
-        M[Anthropic]
-        N[OpenAI]
-        O[Gemini]
-        P[MCP Servers]
-    end
-
-    A --> B
-    A --> D
-    B --> C
-    C --> D
-    B --> K
-    B --> E
-    B --> TB
-    D --> E
-    D --> L
-    D --> TB
-    D --> H
-    D --> I
-    D --> J
-    D --> SK
-    D --> F
-    SK --> I
-    G --> R
-    R --> B
-    R --> C
-    R --> D
-    G --> D
-    G --> L
-    HK -.wraps.-> B
-    HK -.wraps.-> C
-    HK -.wraps.-> D
-    HK -.wraps.-> I
-    L --> M
-    L --> N
-    L --> O
-    H --> P
-```
-
-The dotted `wraps` edges are the hook system: `Hooks.run` brackets every network
-run, task, robot run, LLM generation, tool call, compaction, and `learn` call. It
-is the framework's extension seam — the extension gems (`robot_lab-audit`,
-`robot_lab-durable`, …) attach here rather than subclassing core objects. See the
-[Hooks API](../api/hooks.md).
-
-### Robot subsystems
-
-A `Robot` composes several small collaborators, each documented on the
-[Support](../api/support.md) and [Skills](../api/skills.md) API pages:
-
-```mermaid
-graph LR
-    D[Robot]
-
-    D --> BM[BusMessaging<br/>RobotMessage envelopes]
-    BM --> BP[BusPoller<br/>per-robot serialization]
-    D --> BD[Budget::Ledger<br/>token_budget / cost_budget]
-    D --> DL[DoomLoopDetector<br/>always installed per run]
-    D --> HC[HistoryCompressor<br/>compress_history / auto_compact]
-    D --> DF[DelegationFuture<br/>delegate async: true]
-    D --> HS[HistorySearch<br/>search_history]
-    D --> ASM[AgentSkillMatching<br/>prepended around run]
-
-    ASM --> AS[AgentSkill<br/>SKILL.md bundle]
-    AS --> CAT[AgentSkillCatalog]
-    AS --> CAP[Capabilities]
-    AS --> ST[ScriptTool]
-    ST --> EX{ScriptTool.executor}
-    EX -. "installed by<br/>robot_lab-sandbox" .-> SB[Sandbox<br/>Seatbelt or Null]
-    CAP -. "used if installed" .-> SB
-
-    HC --> TA[TextAnalysis<br/>TF / TF-IDF]
-    HS --> TA
-    CV[Convergence] --> TA
-    SD[MCP::ServerDiscovery] --> TA
-```
-
-`TextAnalysis` is the shared floor under every similarity feature — history
-compression, history search, convergence detection, and MCP server discovery all
-route through it, which is why they share the one optional `classifier`
-dependency.
-
-## Core Components
-
-| Component | Description | Documentation |
-|-----------|-------------|---------------|
-| **Robot** | LLM agent (subclass of `RubyLLM::Agent`) with template-based prompts, tools, and memory | [Core Concepts](core-concepts.md) |
-| **Network** | Orchestrates multiple robots as a SimpleFlow pipeline | [Network Orchestration](network-orchestration.md) |
-| **Memory** | Reactive key-value store with pub/sub and blocking reads | [Memory Management](state-management.md) |
-| **Task** | Wraps a robot for pipeline execution with per-task config | [Network Orchestration](network-orchestration.md) |
-| **RobotResult** | Captures LLM output, tool calls, and metadata from a run | [Message Flow](message-flow.md) |
-| **RunConfig** | Per-run settings object (LLM fields, `mcp`/`tools`, callbacks, infrastructure) that cascades global → network → task → robot | [Core Concepts](core-concepts.md) |
-| **Config** | MywayConfig-based global configuration with env var and file support | [Configuration](#configuration) |
-| **Hook** | Handler base class for the seven hook families — the framework's extension seam | [Hooks API](../api/hooks.md) |
-| **AgentSkill** | A `SKILL.md` bundle whose instructions and `scripts/` become prompt text and tools | [Skills API](../api/skills.md) |
-| **Sandbox** | Opt-in OS-level confinement (macOS Seatbelt) for skill scripts, derived from `Capabilities`; ships in the optional `robot_lab-sandbox` gem, not core | [Skills API](../api/skills.md) |
-| **RobotMessage** | Immutable envelope for TypedBus inter-robot messaging, serialized per robot by `BusPoller` | [Support API](../api/support.md) |
-| **Budget::Ledger** | Thread-safe reserve/reconcile ledger behind `token_budget` / `cost_budget` | [Support API](../api/support.md) |
-
-## Configuration
-
-Global configuration is a MywayConfig subclass (`Config < MywayConfig::Base`). It is loaded from multiple sources in priority order:
-
-1. **Bundled defaults** (`lib/robot_lab/config/defaults.yml`)
-2. **Environment overrides** (development, test, production)
-3. **XDG user config** (`~/.config/robot_lab/robot_lab.yml` — the filename repeats the app name; `config.yml` is never read)
-4. **Project config** (`./config/robot_lab.yml` — the only file with an ERB pass)
-5. **Environment variables** (`ROBOT_LAB_*` prefix, double underscore for nesting)
-6. **Constructor params**
-
-Top-level wrappers behave differently per file. In the **XDG user config**, a section named for the current environment is honoured — the loader looks for `parsed.key?(env)` (`Anyway::Settings.current_environment`, else `RAILS_ENV`, else `RACK_ENV`, else `"development"`) and falls back to the root when absent — so both `development:` and a flat file work there. A `defaults:` wrapper is not an environment name, so it is ignored. The **project config** must be flat outside Rails (all wrappers ignored); under Rails, `anyway_config` sets `current_environment` to `Rails.env` and the project file becomes environmental, so a flat file is ignored and keys must sit under `development:` / `test:` / `production:`. See [Core Concepts](core-concepts.md#configuration) for details.
-
-```ruby
-# Access configuration
-RobotLab.config.ruby_llm.model            #=> "claude-sonnet-4"
-RobotLab.config.ruby_llm.request_timeout  #=> 120
-
-# Block form (yields the same Config object)
-RobotLab.configure do |c|
-  c.logger = Logger.new($stdout)
-end
-```
-
-## Data Flow
-
-1. **Input**: User calls `robot.run("message")` or `network.run(message: "...")`
-2. **Memory**: Robot resolves active memory (standalone or network-shared)
-3. **MCP**: Robot resolves MCP servers from hierarchical config and connects clients. `run` defaults to `mcp: :none`, so a plain `run` connects nothing
-4. **Tools**: Robot resolves and filters tools from hierarchical config. `run` defaults to `tools: :none`, so a plain `run` sends the LLM zero tools; pass `tools: :inherit` to send the robot's attached tools
-5. **Execution**: Robot delegates to `Agent#ask` which calls `@chat.ask` on RubyLLM
-6. **Tool Loop**: LLM may invoke tools; RubyLLM handles the tool call/result loop
-7. **Result**: Robot builds and returns a `RobotResult`
-8. **Network**: If in a network, result flows to dependent tasks via SimpleFlow
-
-## Key Patterns
-
-### Factory Methods
-
-Robots and networks are created via factory methods on the `RobotLab` module:
-
-```ruby
-robot = RobotLab.build(
-  name: "assistant",
-  template: :assistant,
-  context: { tone: "friendly" }
-)
-
-network = RobotLab.create_network(name: "pipeline") do
-  task :analyst, analyst_robot, depends_on: :none
-  task :writer, writer_robot, depends_on: [:analyst]
-end
-```
-
-### Fluent Chaining
-
-`with_*` methods on Robot delegate to the underlying `@chat` and return `self`:
-
-```ruby
-robot = RobotLab.build(name: "bot")
-  .with_instructions("Be concise.")
-  .with_temperature(0.3)
-  .with_model("gpt-4o")
-```
-
-### Hierarchical Configuration
-
-Tools and MCP servers use hierarchical resolution: `run()/task > robot build-time > network > global config`. Values can be `:none`, `:inherit`, or an explicit array (which filters the already-attached tools by name — entries must match how the tool was attached, class or instance). `run()` defaults both to `:none`, so the runtime level is an explicit "send nothing" unless you override it.
-
-### SimpleFlow Pipeline
-
-Networks are thin wrappers around `SimpleFlow::Pipeline`. Each robot is wrapped in a `Task` that implements the `call(result)` interface. Tasks define dependencies (`:none`, `[:task_names]`, or `:optional`) to control execution order.
-
-## Next Steps
-
-- [Core Concepts](core-concepts.md) - Deep dive into robots and tools
-- [Robot Execution](robot-execution.md) - How robots process messages
-- [Network Orchestration](network-orchestration.md) - Multi-robot workflows
-- [Memory Management](state-management.md) - Managing memory and reactive features
-- [Message Flow](message-flow.md) - How messages move through the system

data/docs/architecture/message-flow.md

@@ -1,366 +0,0 @@
-# Message Flow
-
-This page explains how messages move through RobotLab, from user input to LLM response.
-
-## Message Types
-
-`RobotLab::Message` is the abstract base for three conversation message classes. `ToolMessage` is a plain value object that they reference, and two further message classes — `UserMessage` and `RobotMessage` — live outside the hierarchy entirely:
-
-```mermaid
-classDiagram
-    class Message {
-        <<abstract>>
-        +type: String
-        +role: String
-        +content
-        +stop_reason: String
-        +text?() bool
-        +tool_call?() bool
-        +tool_result?() bool
-        +system?() bool
-        +user?() bool
-        +assistant?() bool
-        +stopped?() bool
-        +tool_stop?() bool
-        +to_h() Hash
-    }
-
-    class TextMessage {
-        +content: String
-    }
-
-    class ToolMessage {
-        <<PORO>>
-        +id: String
-        +name: String
-        +input: Hash
-        +to_h() Hash
-        +to_json() String
-    }
-
-    class ToolCallMessage {
-        +tools: Array~ToolMessage~
-    }
-
-    class ToolResultMessage {
-        +tool: ToolMessage
-        +content: Hash
-        +success?() bool
-        +error?() bool
-        +data()
-        +error()
-    }
-
-    Message <|-- TextMessage
-    Message <|-- ToolCallMessage
-    Message <|-- ToolResultMessage
-    ToolMessage -- ToolCallMessage
-    ToolMessage -- ToolResultMessage
-```
-
-`ToolMessage` does **not** inherit from `Message` — its superclass is `Object`. It has no `type`, `role`, or predicate methods; only `id`, `name`, `input`, `to_h`, and `to_json`. The `tool_call?` predicate lives on `Message`, so it answers for `ToolCallMessage`, not for `ToolMessage`.
-
-The two classes outside the hierarchy:
-
-| Class | Role |
-|-------|------|
-| `UserMessage` | Envelope for user input carrying `session_id`, an extra `system_prompt`, `metadata`, `id`, `created_at`. `UserMessage.from` normalizes a String/Hash/TextMessage; `#to_message` converts to a `TextMessage` |
-| `RobotMessage` | Immutable `Data.define(:id, :from, :content, :in_reply_to)` envelope for robot-to-robot traffic on the message bus, with `#key` (`"from:id"`) and `#reply?` |
-
-### TextMessage
-
-Regular text content from users or assistants:
-
-```ruby
-TextMessage.new(
-  role: "user",
-  content: "What's the weather in Paris?"
-)
-
-TextMessage.new(
-  role: "assistant",
-  content: "The weather in Paris is sunny and 22 degrees C.",
-  stop_reason: "stop"
-)
-```
-
-### ToolMessage
-
-A standalone value object (not a `Message`) representing a tool invocation and its parameters:
-
-```ruby
-tool = ToolMessage.new(
-  id: "tool_123",
-  name: "get_weather",
-  input: { location: "Paris" }
-)
-
-tool.to_h  #=> { type: "tool", id: "tool_123", name: "get_weather", input: { location: "Paris" } }
-```
-
-Note that the `type: "tool"` key appears only in `to_h`; there is no `type` reader and no predicate methods.
-
-### ToolCallMessage
-
-LLM's request to execute one or more tools:
-
-```ruby
-ToolCallMessage.new(
-  role: "assistant",
-  tools: [
-    ToolMessage.new(id: "call_1", name: "get_weather", input: { location: "Paris" })
-  ],
-  stop_reason: "tool"
-)
-```
-
-The signature is `initialize(role:, tools:, stop_reason: nil)` — `role` and
-`tools` are required and there is **no** `content:` keyword. Passing one raises
-`ArgumentError`.
-
-### ToolResultMessage
-
-Result from tool execution:
-
-```ruby
-ToolResultMessage.new(
-  tool: tool_message,
-  content: { data: { temp: 22, condition: "sunny" } }
-)
-```
-
-## Message Flow: Standalone Robot
-
-The primary execution path is `robot.run("message")`:
-
-```mermaid
-sequenceDiagram
-    participant User
-    participant Robot
-    participant Memory
-    participant MCP
-    participant Tools
-    participant Agent
-    participant Chat
-    participant LLM
-
-    User->>Robot: robot.run("message")
-    Robot->>Memory: resolve_active_memory
-    Memory-->>Robot: active memory
-
-    Robot->>MCP: resolve_mcp_hierarchy
-    MCP-->>Robot: resolved MCP config
-    Robot->>Robot: ensure_mcp_clients
-
-    Robot->>Tools: resolve_tools_hierarchy
-    Tools-->>Robot: filtered + capped tools
-    Robot->>Chat: @chat.with_tools(*filtered, replace: true)
-
-    Robot->>Agent: ask("message")
-    Agent->>Chat: @chat.ask("message")
-    Chat->>LLM: Provider API call
-
-    loop Tool Loop (handled by RubyLLM)
-        LLM-->>Chat: Tool call response
-        Chat->>Tools: Execute tool
-        Tools-->>Chat: Tool result
-        Chat->>LLM: Send tool result
-    end
-
-    LLM-->>Chat: Final response
-    Chat-->>Agent: RubyLLM::Response
-    Agent-->>Robot: response
-
-    Robot->>Robot: build_result(response, memory)
-    Robot-->>User: RobotResult
-```
-
-### Step-by-Step
-
-1. **`robot.run("message")`**: Entry point. Accepts a positional string argument.
-
-2. **Resolve Memory**: Determines which memory to use:
-   - `network_memory` if provided (network execution)
-   - `network.memory` if in a network context
-   - `robot.memory` (standalone, the default)
-
-3. **Merge Runtime Memory**: If a `memory:` keyword argument is passed, it is merged into the active memory.
-
-4. **Set Current Writer**: Sets `memory.current_writer = robot.name` so subscription callbacks know which robot wrote a value.
-
-5. **Resolve MCP Hierarchy**: Resolves MCP server configuration through the hierarchy: `runtime (run/task) > robot build-time > network > global config`. `run` defaults `mcp:` to `:none`, so nothing is connected unless you pass `mcp: :inherit` or an explicit list.
-
-6. **Ensure MCP Clients**: Initializes or updates MCP client connections and discovers tools from them. Connection failures are logged and recorded in `failed_mcp_server_names`, never raised.
-
-7. **Resolve Tools Hierarchy**: Resolves which tools are available through the same hierarchy. `run` defaults `tools:` to `:none`, which means "send zero tools this turn" — pass `tools: :inherit` to send the robot's attached tools.
-
-8. **Filter Tools**: Filters by the resolved allowlist, clamps to `max_tools` (128 by default), and applies the set with `@chat.with_tools(*filtered, replace: true)` so the persistent chat holds exactly this turn's tools.
-
-9. **Agent#ask**: Delegates to the parent class `RubyLLM::Agent#ask`, which calls `@chat.ask(message)`.
-
-10. **LLM Interaction**: RubyLLM handles the provider-specific API call, including the tool call/result loop.
-
-11. **Build Result**: Wraps the LLM response in a `RobotResult` containing output messages, tool calls, and metadata.
-
-12. **Return**: Returns the `RobotResult` to the caller.
-
-## Message Flow: Network Execution
-
-When running through a network, the flow adds pipeline orchestration:
-
-```mermaid
-sequenceDiagram
-    participant User
-    participant Network
-    participant Pipeline
-    participant Task
-    participant Robot
-    participant LLM
-
-    User->>Network: network.run(message: "...")
-    Network->>Network: Inject network_memory into run_context
-    Network->>Pipeline: SimpleFlow::Pipeline.call_parallel(initial_result)
-
-    loop For each ready task
-        Pipeline->>Task: task.call(result)
-        Task->>Task: Deep merge task context with run_params
-        Task->>Robot: robot.call(enhanced_result)
-        Robot->>Robot: extract_run_context(result)
-        Robot->>Robot: run(message, network_memory: ...)
-        Robot->>LLM: Agent#ask -> @chat.ask
-        LLM-->>Robot: Response
-        Robot-->>Task: result.with_context(:name, robot_result).continue(robot_result)
-        Task-->>Pipeline: SimpleFlow::Result
-    end
-
-    Pipeline-->>Network: Final SimpleFlow::Result
-    Network-->>User: result
-```
-
-### Key Points
-
-- **Network creates initial result**: `SimpleFlow::Result.new(run_context, context: { run_params: run_context })`, after injecting `network_memory`, `network`, and (when non-empty) `network_config` into the run context, all wrapped in the `:network_run` hook
-- **Task wraps robot**: Each `Task` runs the `:task` hook and deep-merges its own context, `mcp`, `tools`, `memory`, and `config` into the run params before delegating to the robot
-- **Robot extracts context**: `extract_run_context(result)` pulls the message, MCP, tools, memory, network, and task out of the SimpleFlow result and passes them to `run` as keyword arguments
-- **Shared memory**: All robots use `network.memory` during network execution
-- **Result accumulation**: Each robot stores its `RobotResult` in `result.context[:robot_name]` — the key comes from `@name.to_sym` in `Robot#call`, so it is the **robot's** name, not the task name. They match only when the two are spelled the same way
-
-## RobotResult
-
-The return value of `robot.run("message")`:
-
-```ruby
-result = robot.run("What is Ruby?")
-
-result.last_text_content  #=> "Ruby is a dynamic programming language..."
-result.reply              #=> alias for last_text_content
-result.has_tool_calls?    #=> false
-result.robot_name         #=> "assistant"
-result.output             #=> [TextMessage(role: "assistant", content: "...")]
-result.tool_calls         #=> []
-result.stop_reason        #=> nil (always — see below)
-result.created_at         #=> Time
-result.id                 #=> "uuid"
-result.checksum           #=> "sha256-hex"
-result.input_tokens       #=> 42
-result.output_tokens      #=> 128
-result.duration           #=> Float, nil
-```
-
-`stop_reason` is always `nil` on a `Robot#run` result: `build_result` reads it as `response.respond_to?(:stop_reason) ? response.stop_reason : nil`, and `RubyLLM::Message` does not define the method. It is consequently dropped from `export`, and `result.stopped?` reduces to `!result.has_tool_calls?`.
-
-`output` always holds exactly one synthesized `TextMessage` built from the final response text — it is not a transcript of the turn, and it never contains `ToolCallMessage` or `ToolResultMessage` entries. `tool_calls` is read off the final assistant message, which carries no tool calls once RubyLLM's tool loop has finished, so it is effectively always empty. To observe tool activity, use the `on_tool_call` / `on_tool_result` callbacks or the tool hooks.
-
-### Result Serialization
-
-```ruby
-# Export for persistence (excludes debug fields)
-hash = result.export
-
-# Full hash including debug fields
-hash = result.to_h
-
-# JSON
-json = result.to_json
-
-# Reconstruct from hash
-result = RobotResult.from_hash(hash)
-```
-
-## Message Predicates
-
-These are defined on `Message`, so every `TextMessage`, `ToolCallMessage`, and `ToolResultMessage` responds to all of them. `ToolMessage` responds to none of them:
-
-```ruby
-message.text?         # Is it a TextMessage?
-message.tool_call?    # Is it a ToolCallMessage?
-message.tool_result?  # Is it a ToolResultMessage?
-
-message.user?         # Is role "user"?
-message.assistant?    # Is role "assistant"?
-message.system?       # Is role "system"?
-
-message.stopped?      # Is stop_reason "stop"?
-message.tool_stop?    # Is stop_reason "tool"?
-```
-
-Valid values are constrained at construction: `type` must be one of `text`, `tool_call`, `tool_result`; `role` one of `system`, `user`, `assistant`, `tool_result`; `stop_reason` one of `tool`, `stop`. Anything else raises `ArgumentError`.
-
-## Creating Messages
-
-### From Strings
-
-```ruby
-TextMessage.new(role: "user", content: "Hello")
-```
-
-### From Hashes
-
-```ruby
-Message.from_hash(
-  type: "text",
-  role: "user",
-  content: "Hello"
-)
-```
-
-## Serialization
-
-Messages can be serialized:
-
-```ruby
-# To hash
-hash = message.to_h
-#=> { type: "text", role: "user", content: "Hello" }
-
-# To JSON
-json = message.to_json
-
-# From hash
-message = Message.from_hash(hash)
-```
-
-## Template Resolution
-
-When a robot has a template, it is resolved at build time via prompt_manager:
-
-```ruby
-robot = RobotLab.build(
-  name: "helper",
-  template: :helper,
-  context: { tone: "friendly" }
-)
-```
-
-The template resolution process:
-1. `PM.parse(:helper)` loads the template file from the configured prompts directory
-2. Non-LLM front matter (`robot_name`, `description`, `tools`, `mcp`, `skills`) is applied to the robot, filling in only what the constructor did not provide
-3. The LLM front-matter keys become a `RunConfig` that the robot's own `@config` merges over — front matter is the base, constructor kwargs win — and `apply_to` dispatches `chat.with_<field>` for each field the chat supports. In practice only `model` and `temperature` reach the chat this way; `top_p`, `top_k`, `max_tokens`, `presence_penalty`, `frequency_penalty`, and `stop` are parsed and silently dropped because `RubyLLM::Chat` defines no `with_` method for them. Set those via constructor kwargs instead
-4. The template body is rendered with ERB (`<%= var %>`; `{{ var }}` is not interpolated) and set as system instructions via `@chat.with_instructions(rendered)`
-
-If both `template:` and `system_prompt:` are provided, the system prompt is appended to the rendered template, producing one combined system message — and it is re-appended on every template re-render, so supplying run-time context never silently drops it.
-
-## Next Steps
-
-- [Memory Management](state-management.md) - How memory stores conversation data
-- [Network Orchestration](network-orchestration.md) - Multi-robot pipeline execution