robot_lab 0.2.1 → 0.2.7

data/docs/examples/index.md

@@ -13,7 +13,7 @@ These examples show how to use RobotLab for common scenarios, from simple chatbo
 | [Basic Chat](basic-chat.md) | Simple conversational robot |
 | [Multi-Robot Network](multi-robot-network.md) | Customer service with routing |
 | [Tool Usage](tool-usage.md) | External API integration |
-| [MCP Server](mcp-server.md) | Creating an MCP tool server |
+| [MCP Server](mcp-server.md) | Connecting a robot to external MCP servers |
 | [Message Bus](#message-bus) | Bidirectional robot communication with convergence |
 | [Spawning Robots](#spawning-robots) | Dynamic specialist creation at runtime |
 
@@ -43,7 +43,8 @@ require "robot_lab"
 # Configuration is handled automatically via MywayConfig.
 # Set API keys via environment variables:
 #   ROBOT_LAB_RUBY_LLM__ANTHROPIC_API_KEY=sk-ant-...
-# Or via config files (~/.config/robot_lab/config.yml)
+# Or via config files (~/.config/robot_lab/robot_lab.yml — the filename
+# repeats the app name; ~/.config/robot_lab/config.yml is never read)
 
 robot = RobotLab.build(
   name: "greeter",
@@ -57,14 +58,25 @@ puts result.last_text_content
 
 ## Robot with Tools
 
+Give the LLM a fixed set of operations rather than an expression evaluator —
+never `eval` a string the model produced.
+
 ```ruby
 class CalculatorTool < RubyLLM::Tool
-  description "Perform a calculation"
-
-  param :expression, type: :string, desc: "Math expression to evaluate"
-
-  def execute(expression:)
-    eval(expression).to_s
+  description "Performs basic arithmetic operations"
+
+  param :operation, type: "string", desc: "add, subtract, multiply, or divide"
+  param :a,         type: "number", desc: "First operand"
+  param :b,         type: "number", desc: "Second operand"
+
+  def execute(operation:, a:, b:)
+    case operation
+    when "add"      then a + b
+    when "subtract" then a - b
+    when "multiply" then a * b
+    when "divide"   then a.to_f / b
+    else "Unknown operation: #{operation}"
+    end
   end
 end
 
@@ -74,41 +86,85 @@ robot = RobotLab.build(
   local_tools: [CalculatorTool]
 )
 
-result = robot.run("What's 25 * 4?")
+# tools: :inherit is REQUIRED — run() defaults to tools: :none
+result = robot.run("What's 25 * 4?", tools: :inherit)
 puts result.last_text_content
 ```
 
+> [!WARNING]
+> `Robot#run` defaults to `tools: :none` and `mcp: :none`. A plain
+> `robot.run("...")` sends the LLM **no tools at all**, even when `local_tools:`
+> were attached at build time. Pass `tools: :inherit` on the call that should be
+> able to use them.
+>
+> For a **standalone** robot like this one, do not pass `tools: :inherit` at
+> *build* time — there the parent level is the global default (`:none`), so it
+> produces an allowlist that matches nothing. Leave `tools:` unset in the
+> constructor. (Inside a network the opposite holds: build-time `:inherit` is how a
+> robot opts into the allowlist on the network's `config:`. See
+> [Configuration](../getting-started/configuration.md#hierarchical-mcp-and-tools).)
+
+See [`examples/02_tools.rb`](https://github.com/MadBomber/robot_lab/blob/main/examples/02_tools.rb)
+for a second tool definition (`FortuneCookie`) alongside the calculator.
+
 ## Network with Routing
 
+Routing is not configured declaratively — a robot performs it. Subclass
+`RobotLab::Robot`, override `#call`, and activate one of the `depends_on: :optional`
+tasks based on what the LLM returned.
+
 ```ruby
-classifier = RobotLab.build(
-  name: "classifier",
-  system_prompt: "Classify the request as BILLING or TECHNICAL. Respond with only the category."
-)
+class ClassifierRobot < RobotLab::Robot
+  def call(result)
+    run_context = extract_run_context(result)
+    message     = run_context.delete(:message)
+    robot_result = run(message, **run_context)
+
+    new_result = result
+      .with_context(@name.to_sym, robot_result)
+      .continue(robot_result)
+
+    case robot_result.reply.to_s.strip.downcase
+    when /billing/   then new_result.activate(:billing)
+    when /technical/ then new_result.activate(:technical)
+    else                  new_result.activate(:general)
+    end
+  end
+end
 
-billing = RobotLab.build(
-  name: "billing",
-  system_prompt: "You handle billing questions."
+classifier = ClassifierRobot.new(
+  name: "classifier",
+  system_prompt: "Classify the request as BILLING, TECHNICAL, or GENERAL. Respond with only the category."
 )
 
-tech = RobotLab.build(
-  name: "tech",
-  system_prompt: "You handle technical issues."
-)
+billing = RobotLab.build(name: "billing",   system_prompt: "You handle billing questions.")
+tech    = RobotLab.build(name: "technical", system_prompt: "You handle technical issues.")
+general = RobotLab.build(name: "general",   system_prompt: "You handle everything else.")
 
 network = RobotLab.create_network(name: "support") do
   task :classifier, classifier, depends_on: :none
-  task :billing, billing, depends_on: :optional
-  task :tech, tech, depends_on: :optional
+  task :billing,    billing,    depends_on: :optional
+  task :technical,  tech,       depends_on: :optional
+  task :general,    general,    depends_on: :optional
 end
 
 result = network.run(message: "I was charged twice for my subscription")
 
 # Access individual robot results via context
-classifier_result = result.context[:classifier]
-puts classifier_result.last_text_content
+puts result.context[:classifier].last_text_content
+puts result.value.last_text_content   # the specialist that was activated
 ```
 
+> [!WARNING]
+> `result.context` is keyed by the **robot's** name (`with_context(@name.to_sym, ...)`),
+> not the task name, and `activate(:name)` takes a **task** name. Keep the two
+> identical — activating a task name that was never declared **raises
+> `ArgumentError` and aborts the run** (`Step :classifier attempted to activate
+> unknown step :billing`). The same happens for a task that was declared without
+> `depends_on: :optional`.
+
+Full version: [`examples/03_network.rb`](https://github.com/MadBomber/robot_lab/blob/main/examples/03_network.rb).
+
 ## Chaining Configuration
 
 Robots support `with_*` methods that return `self` for chaining:
@@ -123,17 +179,27 @@ result = robot.run("Explain Ruby blocks.")
 puts result.last_text_content
 ```
 
+> [!NOTE]
+> The chainable set is exactly what `RubyLLM::Chat` exposes — `with_context`,
+> `with_headers`, `with_instructions`, `with_model`, `with_params`, `with_schema`,
+> `with_temperature`, `with_thinking`, `with_tool`, `with_tools` — plus RobotLab's
+> `with_template` and `with_bus`. There is no `with_max_tokens`, `with_top_p`, or
+> `with_top_k`; use a constructor kwarg or `with_params(max_tokens: 2000)`.
+
 ## Using Templates
 
-Templates are `.md` files with optional YAML front matter, managed by prompt_manager:
+Templates are `.md` files with optional YAML front matter, managed by prompt_manager.
+The body is rendered with **ERB** — interpolate with `<%= var %>`.
 
 ```ruby
 # Template file: prompts/support.md
 # ---
 # model: claude-sonnet-4
 # temperature: 0.5
+# parameters:
+#   company_name: null
 # ---
-# You are a support assistant for {{ company_name }}.
+# You are a support assistant for <%= company_name %>.
 
 robot = RobotLab.build(
   name: "support",
@@ -145,6 +211,13 @@ result = robot.run("How do I reset my password?")
 puts result.last_text_content
 ```
 
+> [!NOTE]
+> `{{ var }}` is **not** interpolated — it passes through to the LLM verbatim.
+> Of the LLM keys accepted in front matter, only `model:` and `temperature:` are
+> actually applied to the chat. `top_p`, `top_k`, `max_tokens`, `presence_penalty`,
+> `frequency_penalty`, and `stop` are parsed and then silently dropped; supply
+> those as constructor kwargs or via a `RunConfig` instead.
+
 ## Running Examples
 
 1. Install dependencies:
@@ -171,12 +244,16 @@ bundle exec rake examples:run[1]       # Run specific example by number
 
 ## Shared Example Setup (`examples/common.rb`)
 
-All numbered examples (`01_*.rb` through `34_*.rb`) begin with:
+Most numbered examples (`01_*.rb` through `35_*.rb`) pull in a shared setup file:
 
 ```ruby
 require_relative "common"
 ```
 
+The line sits below each example's header comment rather than at the very top of
+the file, and three examples do without it entirely —
+`32_newsletter_reader.rb`, `33_stock_generator.rb`, and `33_stock_predictor.rb`.
+
 `common.rb` handles the shared boilerplate so individual examples stay focused:
 
 - **`LLM` hash** — frozen lookup of provider/model pairs accessible as `LLM[:default]`, `LLM[:local]`, `LLM[:anthropic]`. Each entry is a `LlmConfig = Data.define(:provider, :model)` value, so you access the model string as `LLM[:default].model`.
@@ -224,7 +301,7 @@ class Comedian < RobotLab::Robot
       @attempts += 1
       temp = [TEMP_START + TEMP_STEP * (@attempts - 1), 1.0].min
       with_temperature(temp)
-      joke = run(message.content.to_s).last_text_content.strip
+      joke = run(message.content.to_s).reply.strip
       send_reply(to: message.from.to_sym, content: joke, in_reply_to: message.key)
     end
   end
@@ -236,10 +313,14 @@ class ComedyCritic < RobotLab::Robot
   def initialize(bus:)
     super(name: "alice", template: :comedy_critic, bus: bus)
     @accepted = false
+    @rounds   = 0
     on_message do |message|
-      verdict = run("Evaluate this joke:\n\n#{message.content}").last_text_content.strip
+      @rounds += 1
+      verdict = run("Evaluate this joke:\n\n#{message.content}").reply.strip
       @accepted = verdict.start_with?("FUNNY")
-      send_message(to: :bob, content: "Not funny enough. Try again.") unless @accepted
+      # The @rounds guard is what terminates the loop — without it the critic
+      # keeps sending Bob back forever.
+      send_message(to: :bob, content: "Not funny enough. Try again.") unless @accepted || @rounds >= MAX_ATTEMPTS
     end
   end
 
@@ -261,7 +342,12 @@ Key patterns demonstrated:
 - **Auto-ack** via 1-arg `on_message` blocks
 - **`send_reply(to:, content:, in_reply_to:)`** for correlated responses
 - **Temperature ramping** (0.2 &rarr; 1.0) for increasing creativity
-- **Convergence loop** that terminates when the critic approves
+- **Convergence loop** that terminates when the critic approves *or* `MAX_ATTEMPTS` is reached
+
+> [!WARNING]
+> `MAX_ATTEMPTS` only bounds the loop because the critic checks it before sending
+> Bob back. Declaring the constant without testing it leaves the two robots
+> messaging each other indefinitely.
 
 Run: `bundle exec ruby examples/12_message_bus.rb`
 

data/docs/examples/mcp-server.md

@@ -6,13 +6,36 @@ Connecting robots to Model Context Protocol servers for external tool access.
 
 This example demonstrates how to connect robots to external MCP servers. MCP servers expose tools that robots can discover and invoke automatically. RobotLab supports stdio, HTTP, WebSocket, and SSE transports.
 
+The runnable version of everything below is
+[`examples/04_mcp.rb`](https://github.com/MadBomber/robot_lab/blob/main/examples/04_mcp.rb)
+(direct `MCP::Client` usage in Part 1, robot integration in Part 2).
+
+> [!WARNING]
+> **Two things are required to actually use MCP tools in a run.** `Robot#run`
+> defaults to `mcp: :none, tools: :none`, so build-time `mcp:` servers are never
+> connected by a plain `robot.run("...")`. Pass both:
+>
+> ```ruby
+> robot.run("...", mcp: :inherit, tools: :inherit)
+> ```
+>
+> `mcp: :inherit` triggers the connection attempt; `tools: :inherit` is what sends
+> the discovered tools to the LLM.
+
+> [!WARNING]
+> `transport:` must be a **nested hash**. A flat `transport: stdio` with sibling
+> `command:` / `args:` keys raises
+> `NoMethodError: undefined method 'transform_keys' for an instance of String`
+> internally — and that error is swallowed, so the robot silently builds with zero
+> MCP tools. Connection failures are logged and recorded in
+> `robot.failed_mcp_server_names`; they are never raised.
+
 ## Using MCP with a Robot
 
 The primary pattern is to pass MCP server configurations via `mcp:` or `mcp_servers:` when building a robot:
 
 ```ruby
 #!/usr/bin/env ruby
-# examples/mcp_client.rb
 
 require "bundler/setup"
 require "robot_lab"
@@ -42,27 +65,34 @@ robot = RobotLab.build(
   model: "claude-sonnet-4"
 )
 
-# The robot auto-discovers MCP tools on first run
+# MCP clients are created lazily. connect_mcp! forces the connection now so the
+# counts below are meaningful -- without it both lines print empty results.
+robot.connect_mcp!
+
 puts "MCP Servers: #{robot.mcp_clients.keys.join(", ")}"
 puts "MCP Tools: #{robot.mcp_tools.size} discovered"
+puts "Failed servers: #{robot.failed_mcp_server_names.join(", ")}" if robot.failed_mcp_server_names.any?
 
-# Run the robot -- it can use any discovered MCP tools
-result = robot.run("What are the top 3 most starred Ruby web frameworks on GitHub?")
+# Run the robot -- both kwargs are required for it to reach the MCP tools
+result = robot.run(
+  "What are the top 3 most starred Ruby web frameworks on GitHub?",
+  mcp: :inherit,
+  tools: :inherit
+)
 puts result.last_text_content
 
-# Show tool calls if any were made
-if result.tool_calls.any?
-  puts "\nTool calls made:"
-  result.tool_calls.each do |tc|
-    tool_info = tc.respond_to?(:tool) ? tc.tool : tc
-    puts "  #{tool_info[:name] || tool_info}"
-  end
-end
+# Confirm which tools were sent this turn
+puts "Tools sent: #{robot.chat.tools.size}"
 
 # Always disconnect MCP clients when done
 robot.disconnect
 ```
 
+> [!NOTE]
+> `result.tool_calls` is effectively always empty — it reads the final assistant
+> message, which contains only text once ruby_llm's tool loop has finished. Use the
+> `on_tool_call:` callback or the Hook system to observe MCP tool invocations.
+
 ## Direct MCP Client Usage
 
 You can also use the MCP client directly without a robot:
@@ -81,6 +111,8 @@ github_server = {
   }
 }
 
+# NOTE: the config is a single POSITIONAL argument.
+# MCP::Client.new(name: ..., transport: ...) raises ArgumentError.
 client = RobotLab::MCP::Client.new(github_server)
 client.connect
 
@@ -134,7 +166,11 @@ robot = RobotLab.build(
   model: "claude-sonnet-4"
 )
 
-result = robot.run("Search for Ruby repos with CI configs and list their workflow files")
+result = robot.run(
+  "Search for Ruby repos with CI configs and list their workflow files",
+  mcp: :inherit,
+  tools: :inherit
+)
 puts result.last_text_content
 
 robot.disconnect
@@ -142,41 +178,77 @@ robot.disconnect
 
 ## MCP in Networks
 
-Networks pass MCP configuration through the hierarchical resolution system. Use `mcp: :inherit` on robots to use the network-level MCP config, or specify per-task MCP servers:
+`mcp` and `tools` are the only two fields a network passes down to its member
+robots. Declare them **per task** — the `task` DSL is what reliably reaches the
+robot's `run` call.
 
 ```ruby
-# Create robots
+# Create robots -- leave mcp:/tools: unset in the constructor
 data_analyst = RobotLab.build(
   name: "data_analyst",
-  system_prompt: "You analyze data.",
-  mcp: :inherit   # Will use whatever MCP config is resolved at runtime
+  system_prompt: "You analyze data."
 )
 
 file_manager = RobotLab.build(
   name: "file_manager",
-  system_prompt: "You manage files.",
-  mcp: :inherit,
-  tools: :none     # Only use inherited MCP tools, no local tools
+  system_prompt: "You manage files."
 )
 
-# Create network with per-task MCP configuration
+# Per-task MCP + tools configuration
 network = RobotLab.create_network(name: "support_with_mcp") do
-  task :analyst, data_analyst,
+  task :data_analyst, data_analyst,
        mcp: [github_server],
+       tools: :inherit,                       # send everything discovered
        depends_on: :none
 
-  task :files, file_manager,
+  task :file_manager, file_manager,
        mcp: [filesystem_server],
-       tools: %w[read_file list_directory],   # Whitelist only these MCP tools
+       tools: %w[read_file list_directory],   # allowlist only these tool names
        depends_on: :optional
 end
 
 result = network.run(message: "Analyze the project structure")
 ```
 
+> [!WARNING]
+> Build-time `:inherit` cuts both ways, so scope it deliberately.
+>
+> For a **standalone** robot it is a trap: `:inherit` resolves against the global
+> level (`:none`), yielding an allowlist of `["none"]` that matches nothing, so the
+> robot sends no tools no matter what you pass at run time.
+>
+> Inside a **network**, build-time `:inherit` is exactly how a robot opts into the
+> network's `config:` list — the parent is resolved at run time from
+> `network_config`. Verified with a network `config:` of `tools: %w[ReadFile]` on a
+> robot holding `ReadFile` and `ListDir`:
+>
+> | Robot constructor | Task line | Tools sent |
+> |---|---|---|
+> | `tools: :inherit` | `tools: :inherit` | `[:read_file]` — network allowlist applied |
+> | (unset) | `tools: :inherit` | `[:read_file, :list_dir]` — allowlist ignored |
+>
+> A network-level `config:` also propagates *only* `mcp` and `tools`. LLM fields
+> (`model`, `temperature`, `max_tokens`, ...) and callbacks (`on_content`) are read
+> from each robot's own config at construction time and are never inherited from
+> the network.
+
+## Transport Types
+
+The `type:` field must be one of `stdio`, `sse`, `ws`, `websocket`,
+`streamable-http`, or `http`. Anything else raises `ArgumentError` — note that
+`streamable_http` with an underscore is **not** valid.
+
+`timeout:` sits alongside `transport:` on the server config and defaults to 15
+seconds. It is normalized: `nil` becomes 15, any value of 1000 or more is treated
+as milliseconds (5000 → 5.0s), and anything under 1 second is floored to 1.
+
+> [!NOTE]
+> Only the stdio transport actually enforces `timeout`. The SSE, WebSocket, and
+> StreamableHTTP transports store the value and never reference it.
+
 ## HTTP Transport
 
-Connect to remote MCP servers over HTTP:
+Connect to remote MCP servers over HTTP. Requires the `async-http` gem.
 
 ```ruby
 robot = RobotLab.build(
@@ -185,22 +257,36 @@ robot = RobotLab.build(
   mcp: [
     {
       name: "remote_api",
+      timeout: 30,
       transport: {
         type: "http",
         url: "https://mcp.example.com/mcp",
-        headers: { "Authorization" => "Bearer #{ENV['MCP_TOKEN']}" }
+        # Auth is supplied by a callable, not a headers hash
+        auth_provider: -> { "Bearer #{ENV['MCP_TOKEN']}" }
       }
     }
   ]
 )
 
-result = robot.run("Use the remote tools to check system status")
+result = robot.run("Use the remote tools to check system status", mcp: :inherit, tools: :inherit)
 robot.disconnect
 ```
 
+> [!WARNING]
+> A `headers:` key in a streamable-http transport config is **silently discarded**.
+> The transport builds its own header hash on every request and only merges in
+> `Authorization` when `auth_provider` is set. Use `auth_provider:` (a proc
+> returning the full header value) for authentication.
+
+> [!WARNING]
+> The HTTP and SSE transports set `@connected = true` *before* the MCP
+> initialization handshake completes, so `connected?` returns `true` even against
+> an unreachable host. Treat it as "a connect was attempted", not "the server
+> answered", and check `robot.failed_mcp_server_names` as well.
+
 ## WebSocket Transport
 
-For real-time bidirectional communication:
+For real-time bidirectional communication. Requires the `async-websocket` gem.
 
 ```ruby
 robot = RobotLab.build(
@@ -217,13 +303,18 @@ robot = RobotLab.build(
   ]
 )
 
-result = robot.run("Subscribe to the events channel")
+result = robot.run("Subscribe to the events channel", mcp: :inherit, tools: :inherit)
 robot.disconnect
 ```
 
+> [!CAUTION]
+> The WebSocket transport currently raises `NameError` on
+> `Async::HTTP::Endpoint` inside an un-awaited `Async` block. It is not usable as
+> shipped — prefer stdio or HTTP.
+
 ## SSE Transport
 
-Server-Sent Events transport for streaming responses:
+Server-Sent Events transport. Requires the `async-http` gem.
 
 ```ruby
 robot = RobotLab.build(
@@ -240,13 +331,13 @@ robot = RobotLab.build(
   ]
 )
 
-result = robot.run("Stream the latest metrics")
+result = robot.run("Stream the latest metrics", mcp: :inherit, tools: :inherit)
 robot.disconnect
 ```
 
-## Runtime MCP Overrides
+## Runtime MCP Selection
 
-Override MCP configuration at runtime via `robot.run`:
+`mcp:` and `tools:` on `robot.run` decide, per call, what the LLM sees:
 
 ```ruby
 robot = RobotLab.build(
@@ -255,15 +346,28 @@ robot = RobotLab.build(
   mcp: [github_server]
 )
 
-# Use default MCP config
-result = robot.run("Search for Ruby repos")
+# Connect the configured MCP servers and send their tools
+result = robot.run("Search for Ruby repos", mcp: :inherit, tools: :inherit)
 
-# Override MCP at runtime -- disable all MCP
-result = robot.run("Just answer from your knowledge", mcp: :none)
+# The DEFAULT: no MCP connection, no tools sent. Writing these out is
+# redundant -- omitting both kwargs does exactly the same thing.
+result = robot.run("Just answer from your knowledge")
+
+# Connect MCP but send only two named tools
+result = robot.run(
+  "List the open issues",
+  mcp: :inherit,
+  tools: %w[list_issues get_issue]
+)
 
 robot.disconnect
 ```
 
+> [!NOTE]
+> `tools:` is a **name allowlist** across every available tool — local and MCP
+> alike. `tools: :none` means "zero tools of any kind", not "MCP only"; there is no
+> switch that selects local tools versus MCP tools.
+
 ## Running
 
 ```bash
@@ -274,18 +378,23 @@ export GITHUB_PERSONAL_ACCESS_TOKEN="your-token"
 # Install MCP server (example: GitHub)
 brew install github-mcp-server
 
-# Run client
-ruby examples/mcp_client.rb
+# Direct MCP::Client usage + robot integration
+ruby examples/04_mcp.rb
+
+# Semantic MCP server selection (no LLM calls)
+ruby examples/28_mcp_discovery.rb
 ```
 
 ## Key Concepts
 
-1. **MCP Configuration**: Pass server configs via `mcp:` parameter on `RobotLab.build` or `Robot.new`
-2. **Auto-Discovery**: Tools are automatically discovered when the robot connects to an MCP server
-3. **Transport Types**: stdio, http, websocket, sse
-4. **Hierarchical Config**: `runtime > robot > network > global`, using `:none`, `:inherit`, or explicit arrays
-5. **Tool Filtering**: Use `tools:` whitelist to limit which MCP tools are available
-6. **Cleanup**: Always call `robot.disconnect` when done to release MCP connections
+1. **MCP Configuration**: Pass server configs via `mcp:` / `mcp_servers:` on `RobotLab.build` or `Robot.new`. `transport:` must be a nested hash
+2. **Nothing is sent by default**: `run` defaults to `mcp: :none, tools: :none`. Pass `mcp: :inherit, tools: :inherit` to connect and send
+3. **Lazy connection**: MCP clients are created on the first qualifying `run`. Call `robot.connect_mcp!` to connect eagerly — before that, `mcp_clients` and `mcp_tools` are empty
+4. **Failures are silent**: connection errors are logged and recorded in `robot.failed_mcp_server_names`, never raised
+5. **Transport Types**: `stdio`, `sse`, `ws`, `websocket`, `streamable-http`, `http` (`streamable_http` is invalid)
+6. **Tool Filtering**: `tools:` is an allowlist over local *and* MCP tools combined; entries must match how each tool was attached (class-attached → `"ReadFile"`, instance-attached → `"read_file"`)
+7. **Networks**: an explicit list on the `task` line applies directly; to inherit the network's `config:` list instead, build the robot with `tools: :inherit` *and* pass `tools: :inherit` on the task
+8. **Cleanup**: Always call `robot.disconnect` when done to release MCP connections
 
 ## See Also