robot_lab 0.0.8 → 0.0.11

data/docs/api/mcp/server.md

@@ -13,31 +13,42 @@ server = RobotLab::MCP::Server.new(
   name: "filesystem",
   transport: { type: "stdio", command: "mcp-server-filesystem", args: ["--root", "/data"] }
 )
+
+# With custom timeout
+server = RobotLab::MCP::Server.new(
+  name: "slow_server",
+  transport: { type: "stdio", command: "heavy-mcp-server" },
+  timeout: 30
+)
 ```
 
 ## Constructor
 
 ```ruby
-Server.new(name:, transport:)
+Server.new(name:, transport:, timeout: nil, **_extra)
 ```
 
 **Parameters:**
 
-| Name | Type | Description |
-|------|------|-------------|
-| `name` | `String` | Unique server identifier |
-| `transport` | `Hash` | Transport configuration (must include `type`) |
+| Name | Type | Default | Description |
+|------|------|---------|-------------|
+| `name` | `String` | **required** | Unique server identifier |
+| `transport` | `Hash` | **required** | Transport configuration (must include `type`) |
+| `timeout` | `Numeric`, `nil` | `15` | Request timeout in seconds. Values >= 1000 are auto-converted from milliseconds. Minimum 1 second |
 
 **Raises:** `ArgumentError` if:
 - The transport type is not one of the valid types
 - A stdio transport is missing the `:command` key
 - A network transport (ws, websocket, sse, streamable-http, http) is missing the `:url` key
 
-## Valid Transport Types
+## Constants
 
 ```ruby
 RobotLab::MCP::Server::VALID_TRANSPORT_TYPES
 # => ["stdio", "sse", "ws", "websocket", "streamable-http", "http"]
+
+RobotLab::MCP::Server::DEFAULT_TIMEOUT
+# => 15  (seconds)
 ```
 
 ## Attributes
@@ -58,6 +69,14 @@ server.transport  # => Hash
 
 The normalized transport configuration hash (keys are symbols, type is downcased).
 
+### timeout
+
+```ruby
+server.timeout  # => Numeric
+```
+
+Request timeout in seconds. Defaults to `DEFAULT_TIMEOUT` (15). Values >= 1000 passed to the constructor are auto-converted from milliseconds to seconds. The minimum is 1 second.
+
 ## Methods
 
 ### transport_type
@@ -71,10 +90,10 @@ Returns the transport type string (e.g., `"stdio"`, `"ws"`, `"sse"`).
 ### to_h
 
 ```ruby
-server.to_h  # => { name: "...", transport: { ... } }
+server.to_h  # => { name: "...", transport: { ... }, timeout: 15 }
 ```
 
-Converts the server configuration to a hash representation.
+Converts the server configuration to a hash representation (includes `timeout`).
 
 ## Transport Configuration Options
 

data/docs/api/mcp/transports.md

@@ -21,7 +21,10 @@ All transports inherit from `RobotLab::MCP::Transports::Base` and implement:
 
 ```ruby
 class RobotLab::MCP::Transports::Base
-  attr_reader :config  # => Hash (symbolized keys)
+  DEFAULT_TIMEOUT = 15  # seconds
+
+  attr_reader :config   # => Hash (symbolized keys, :timeout removed)
+  attr_reader :timeout  # => Numeric (seconds, extracted from config)
 
   def connect        # Establish connection, returns self
   def send_request(message)  # Send JSON-RPC message, returns Hash response
@@ -30,11 +33,13 @@ class RobotLab::MCP::Transports::Base
 end
 ```
 
+The `timeout` is extracted from the config hash during initialization (and removed from `config`). If not provided, it defaults to `DEFAULT_TIMEOUT` (15 seconds). The timeout is propagated from `MCP::Server` through `MCP::Client` to the transport.
+
 ## Stdio Transport
 
 **Class:** `RobotLab::MCP::Transports::Stdio`
 
-Spawns a subprocess and communicates via stdin/stdout using JSON-RPC messages (one per line). Automatically sends MCP `initialize` and `notifications/initialized` on connect.
+Spawns a subprocess and communicates via stdin/stdout using JSON-RPC messages (one per line). Automatically sends MCP `initialize` and `notifications/initialized` on connect. All blocking I/O is wrapped with `Timeout.timeout` so a missing or hung server cannot block the caller forever.
 
 ### Configuration
 
@@ -43,7 +48,8 @@ Spawns a subprocess and communicates via stdin/stdout using JSON-RPC messages (o
   type: "stdio",
   command: "mcp-server-filesystem",      # Required: executable command
   args: ["--root", "/data"],             # Optional: command arguments
-  env: { "DEBUG" => "true" }             # Optional: environment variables
+  env: { "DEBUG" => "true" },            # Optional: environment variables
+  timeout: 10                            # Optional: request timeout in seconds (default: 15)
 }
 ```
 
@@ -52,14 +58,18 @@ Spawns a subprocess and communicates via stdin/stdout using JSON-RPC messages (o
 | `command` | `String` | Yes | Executable command to spawn |
 | `args` | `Array<String>` | No | Command arguments |
 | `env` | `Hash` | No | Environment variables (merged with current env) |
+| `timeout` | `Numeric` | No | Request timeout in seconds (default: 15) |
 
 ### Behavior
 
 - Uses `Open3.popen3` to spawn the subprocess
+- Verifies the process actually started (raises `MCPError` if it exits immediately)
 - Writes JSON-RPC messages to stdin (one per line)
 - Reads responses from stdout, skipping notifications (messages without `id`)
+- All blocking reads are wrapped with `Timeout.timeout` — raises `MCPError` if the server does not respond within the timeout period
 - `connected?` returns `true` when the subprocess is alive
-- `close` terminates stdin, stdout, stderr, and kills the subprocess
+- `close` calls `cleanup_process` to reliably close stdin, stdout, stderr and kill the subprocess
+- Handles `Errno::ENOENT` (command not found), `Errno::EPIPE` / `IOError` (broken pipe / connection lost), and `Timeout::Error` (hung server) with clear error messages
 
 ### Example
 
@@ -67,7 +77,8 @@ Spawns a subprocess and communicates via stdin/stdout using JSON-RPC messages (o
 transport = RobotLab::MCP::Transports::Stdio.new(
   command: "mcp-server-filesystem",
   args: ["--root", "/data"],
-  env: { "DEBUG" => "true" }
+  env: { "DEBUG" => "true" },
+  timeout: 10
 )
 
 transport.connect
@@ -243,7 +254,11 @@ end
 Specific error cases:
 - **Not connected** -- calling `send_request` before `connect` raises `MCPError`
 - **Missing gem** -- WebSocket, SSE, and HTTP transports raise `MCPError` with a `LoadError` message if required gems are not installed
-- **No response** -- Stdio transport raises `MCPError` if the subprocess produces no output
+- **No response** -- Stdio transport raises `MCPError` if the subprocess produces no output (EOF on stdout)
+- **Command not found** -- Stdio transport raises `MCPError` with the original `Errno::ENOENT` message
+- **Timeout** -- Stdio transport raises `MCPError` if the server does not respond within the configured timeout
+- **Broken pipe** -- Stdio transport raises `MCPError` and marks itself disconnected on `Errno::EPIPE` or `IOError`
+- **Immediate exit** -- Stdio transport raises `MCPError` if the server process exits immediately after spawn
 
 ## See Also
 

data/docs/architecture/core-concepts.md

@@ -9,7 +9,7 @@ A Robot is the primary unit of computation in RobotLab. It is a subclass of `Rub
 - A unique identity (name, description)
 - A personality (system prompt and/or template)
 - Capabilities (tools, MCP connections)
-- Model and inference configuration
+- Model, provider, and inference configuration
 - Inherent memory (key-value store)
 
 ### Robot Anatomy

data/docs/architecture/robot-execution.md

@@ -164,7 +164,8 @@ def build_result(response, _memory)
     robot_name: @name,
     output: output,
     tool_calls: normalize_tool_calls(tool_calls),
-    stop_reason: response.respond_to?(:stop_reason) ? response.stop_reason : nil
+    stop_reason: response.respond_to?(:stop_reason) ? response.stop_reason : nil,
+    raw: response
   )
 end
 ```
@@ -182,9 +183,12 @@ result.tool_calls       # => [ToolResultMessage, ...]
 result.stop_reason      # => "stop" or nil
 result.created_at       # => Time
 result.id               # => UUID string
+result.duration         # => Float or nil (elapsed seconds, set in pipeline execution)
+result.raw              # => raw LLM response object
 
 # Convenience methods
 result.last_text_content  # => "Hi there!" (last text message content)
+result.reply              # => alias for last_text_content
 result.has_tool_calls?    # => false
 result.stopped?           # => true
 ```
@@ -275,17 +279,31 @@ sequenceDiagram
     Robot-->>SF: result.continue(robot_result)
 ```
 
-The `Task` wrapper deep-merges per-task configuration (context, mcp, tools) before delegating to the robot's `call`. The base `Robot#call` extracts the message and calls `run`:
+The `Task` wrapper deep-merges per-task configuration (context, mcp, tools) before delegating to the robot's `call`. The base `Robot#call` extracts the message, calls `run`, and records the elapsed time in `RobotResult#duration`. If the robot raises any exception, the error is caught and wrapped in a `RobotResult` so one failing robot does not crash the entire pipeline:
 
 ```ruby
 def call(result)
   run_context = extract_run_context(result)
   message = run_context.delete(:message)
+
+  start_time = Process.clock_gettime(Process::CLOCK_MONOTONIC)
   robot_result = run(message, **run_context)
+  robot_result.duration = Process.clock_gettime(Process::CLOCK_MONOTONIC) - start_time
 
   result
     .with_context(@name.to_sym, robot_result)
     .continue(robot_result)
+rescue Exception => e
+  # Error is wrapped in a RobotResult with the elapsed duration
+  error_result = RobotResult.new(
+    robot_name: @name,
+    output: [TextMessage.new(role: 'assistant', content: "Error: #{e.class}: #{e.message}")]
+  )
+  error_result.duration = elapsed
+
+  result
+    .with_context(@name.to_sym, error_result)
+    .continue(error_result)
 end
 ```
 

data/docs/concepts.md

@@ -12,6 +12,7 @@ Each robot has:
 - **Template**: A `.md` file with YAML front matter managed by prompt_manager, referenced by symbol
 - **System Prompt**: Inline instructions (can be used alone or combined with a template)
 - **Model**: The LLM model to use (defaults to `RobotLab.config.ruby_llm.model`)
+- **Provider**: Optional LLM provider for local models (Ollama, GPUStack, etc.)
 - **Skills**: Composable template behaviors prepended before the main template
 - **Local Tools**: `RubyLLM::Tool` subclasses or `RobotLab::Tool` instances (with automatic error handling)
 - **Streaming**: Real-time content via stored `on_content` callback or per-call block
@@ -195,12 +196,15 @@ tool = RobotLab::Tool.create(
 result = robot.run("Hello!")
 
 result.last_text_content  # => "Hi there!" (String or nil)
+result.reply              # => alias for last_text_content
 result.output             # => [TextMessage, ...] array of output messages
 result.tool_calls         # => [] array of tool call results
 result.robot_name         # => "assistant"
 result.stop_reason        # => "end_turn" or nil
 result.has_tool_calls?    # => false
 result.checksum           # => "a1b2c3d4..." (for dedup)
+result.duration           # => Float or nil (elapsed seconds, set in pipeline execution)
+result.raw                # => raw LLM response object
 ```
 
 ## Memory

data/docs/guides/building-robots.md

@@ -50,6 +50,21 @@ robot = RobotLab.build(
 )
 ```
 
+### Provider
+
+For local LLM providers (Ollama, GPUStack, LM Studio, etc.), use the `provider:` parameter. This tells RubyLLM to skip model validation and connect directly:
+
+```ruby
+robot = RobotLab.build(
+  name: "local_bot",
+  model: "llama3.2",
+  provider: :ollama,
+  system_prompt: "You are a helpful assistant."
+)
+```
+
+When `provider:` is set, `assume_model_exists: true` is automatically applied. The provider is available via `robot.provider`.
+
 ### System Prompt
 
 An inline string that defines the robot's personality and behavior:
@@ -479,10 +494,13 @@ The `run` method returns a `RobotResult` with:
 
 ```ruby
 result.last_text_content  # => "Hi there! How can I help?"
+result.reply              # => alias for last_text_content
 result.output             # => Array of output messages
 result.tool_calls         # => Array of tool call results
 result.robot_name         # => "assistant"
 result.stop_reason        # => stop reason from the LLM
+result.duration           # => Float (elapsed seconds, set in pipeline execution)
+result.raw                # => raw LLM response object
 ```
 
 ### With Runtime Memory

data/docs/guides/creating-networks.md

@@ -124,6 +124,7 @@ end
 | `memory` | Task-specific memory |
 | `config` | Per-task `RunConfig` (merged on top of network's config) |
 | `depends_on` | `:none`, `[:task1]`, or `:optional` |
+| `poller_group` | Bus delivery group label (`:default`, `:slow`, etc.) |
 
 ## Conditional Routing
 
@@ -164,6 +165,26 @@ network = RobotLab.create_network(name: "support") do
 end
 ```
 
+## Poller Groups
+
+Each network maintains a shared `BusPoller` that serializes TypedBus deliveries on a per-robot basis: if a robot is already processing a message, new deliveries are queued and drained after the current one completes. This prevents re-entrancy without blocking other robots.
+
+Named **poller groups** let you label tasks so slow robots are identifiable in logs and monitoring without needing separate infrastructure:
+
+```ruby
+network = RobotLab.create_network(name: "mixed_speed") do
+  # Fast robots on the default group
+  task :fetcher,   fetcher_robot,   depends_on: :none
+  task :summarize, summarizer,      depends_on: [:fetcher]
+
+  # Slow robots with expensive LLM calls — label them :slow
+  task :analyst,   analyst_robot,   depends_on: [:fetcher],  poller_group: :slow
+  task :writer,    writer_robot,    depends_on: [:analyst],  poller_group: :slow
+end
+```
+
+Group labels are informational — there is no separate queue per group. In Async execution, robots naturally yield during LLM HTTP calls, so fast and slow robots interleave without explicit isolation.
+
 ## Running Networks
 
 ### Basic Run
@@ -291,6 +312,24 @@ network = RobotLab.create_network(name: "multi_analysis") do
 end
 ```
 
+### Pipeline Error Resilience
+
+When a robot raises an exception during pipeline execution, the error is caught and wrapped in a `RobotResult` with the error message as content. This ensures one failing robot does not crash the entire network:
+
+```ruby
+# If billing_robot raises an error, the network continues
+# The error is available in the result context:
+result = network.run(message: "Process this")
+billing_result = result.context[:billing]
+
+if billing_result&.last_text_content&.start_with?("Error:")
+  puts "Billing failed: #{billing_result.last_text_content}"
+  puts "Took: #{billing_result.duration}s"
+end
+```
+
+Each robot's `RobotResult` includes a `duration` field (elapsed seconds) that is set automatically during pipeline execution, even for errored results.
+
 ### Conditional Continuation
 
 A robot can halt execution early:

data/docs/guides/index.md

@@ -38,6 +38,14 @@ If you're new to RobotLab, start here:
 
     Share data between robots with the memory system
 
+-   [:octicons-pulse-24: **Observability & Safety**](observability.md)
+
+    Token tracking, circuit breakers, and learning accumulation
+
+-   [:material-cpu-64-bit: **Ractor Parallelism**](ractor-parallelism.md)
+
+    True CPU parallelism for tools and robot pipelines via Ruby Ractors
+
 </div>
 
 ## Framework Integration
@@ -61,3 +69,5 @@ If you're new to RobotLab, start here:
 | [Streaming](streaming.md) | Real-time responses | 5 min |
 | [Memory](memory.md) | Shared data store | 5 min |
 | [Rails Integration](rails-integration.md) | Rails application setup | 15 min |
+| [Observability & Safety](observability.md) | Token tracking, circuit breaker, learning loop | 10 min |
+| [Ractor Parallelism](ractor-parallelism.md) | CPU-parallel tools and robot pipelines | 15 min |

data/docs/guides/knowledge.md

@@ -0,0 +1,182 @@
+# Knowledge & Retrieval
+
+Facilities for searching and retrieving knowledge from a robot's history and from external documents:
+
+- **Chat History Search** — semantic search over accumulated conversation turns
+- **Embedding-Based Document Store** — lightweight RAG: store arbitrary text, search by meaning
+
+---
+
+## Chat History Search
+
+### The Problem
+
+Long-running robots accumulate many conversation turns. When you need to recall what was discussed earlier on a specific topic, re-sending the full history wastes tokens. `search_history` gives you a focused slice of the most relevant past messages without touching the LLM.
+
+### robot.search_history
+
+```ruby
+results = robot.search_history(query, limit: 5)
+```
+
+Scores every message in the robot's conversation history against `query` using stemmed term-frequency cosine similarity (via the `classifier` gem). Returns up to `limit` `HistoryResult` objects sorted by score descending.
+
+```ruby
+results = robot.search_history("quarterly revenue", limit: 3)
+
+results.each do |r|
+  puts "[#{r.role}] score=#{r.score.round(3)} idx=#{r.index}"
+  puts "  #{r.text}"
+end
+```
+
+### HistoryResult Fields
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `text` | String | The message text |
+| `role` | Symbol | `:user`, `:assistant`, or `:system` |
+| `score` | Float (0.0–1.0) | Cosine similarity with the query |
+| `index` | Integer | Position in `@chat.messages` |
+
+### Typical Scores
+
+| Relationship | Typical Score |
+|---|---|
+| Direct answer to the query | 0.50 – 0.80 |
+| Same topic, different phrasing | 0.20 – 0.50 |
+| Unrelated | < 0.10 |
+
+### Short Messages
+
+Messages shorter than 20 characters are skipped — they produce no meaningful term vector.
+
+### Full Example
+
+```ruby
+robot = RobotLab.build(name: "analyst", system_prompt: "You are a financial analyst.")
+
+# … after several robot.run() calls …
+
+hits = robot.search_history("customer acquisition cost")
+hits.each { |r| puts "#{r.role} (#{r.score.round(2)}): #{r.text}" }
+```
+
+### RAG Pattern — Retrieve Then Generate
+
+Use `search_history` to inject only the relevant past context into the next call:
+
+```ruby
+hits    = robot.search_history(user_query, limit: 3)
+context = hits.map(&:text).join("\n")
+
+robot.run("Recall context:\n#{context}\n\nNew question: #{user_query}")
+```
+
+### Optional Dependency
+
+`search_history` requires the `classifier` gem:
+
+```ruby
+gem "classifier", "~> 2.3"
+```
+
+Without it, calling `search_history` raises `RobotLab::DependencyError` with an install hint.
+
+---
+
+## Embedding-Based Document Store
+
+### The Problem
+
+Sometimes the knowledge you need isn't in the conversation history — it's in a README, a product spec, a changelog. `store_document` / `search_documents` embed arbitrary text with `fastembed` and retrieve the most relevant chunk at query time.
+
+### memory.store_document / memory.search_documents
+
+```ruby
+memory.store_document(:readme,    File.read("README.md"))
+memory.store_document(:changelog, File.read("CHANGELOG.md"))
+
+hits = memory.search_documents("how to configure redis", limit: 3)
+hits.each { |h| puts "#{h[:key]} (#{h[:score].round(3)}): #{h[:text][0..80]}" }
+```
+
+Each result hash contains:
+
+| Key | Type | Description |
+|-----|------|-------------|
+| `:key` | Symbol | The key the document was stored under |
+| `:text` | String | The full stored text |
+| `:score` | Float (0.0–1.0) | Cosine similarity with the query |
+
+### Standalone DocumentStore
+
+The `Memory` methods delegate to `RobotLab::DocumentStore`, which can also be used directly:
+
+```ruby
+store = RobotLab::DocumentStore.new
+store.store(:doc_a, "Ruby on Rails is a full-stack web framework.")
+store.store(:doc_b, "Postgres is an advanced relational database.")
+
+results = store.search("relational database SQL", limit: 2)
+puts results.first[:key]  # => :doc_b
+```
+
+Management methods:
+
+```ruby
+store.size          # => 2
+store.keys          # => [:doc_a, :doc_b]
+store.empty?        # => false
+store.delete(:doc_a)
+store.clear
+```
+
+### Embedding Model
+
+Default: `BAAI/bge-small-en-v1.5` (~23 MB, downloaded on first use, cached in `~/.cache/fastembed/`).
+
+Documents are embedded with a `"passage: "` prefix and queries with `"query: "` prefix — the standard retrieval convention for BGE models.
+
+Custom model:
+
+```ruby
+store = RobotLab::DocumentStore.new(model_name: "BAAI/bge-base-en-v1.5")
+```
+
+### RAG Pattern
+
+```ruby
+# 1. Index your knowledge base at startup
+memory.store_document(:readme,    File.read("README.md"))
+memory.store_document(:changelog, File.read("CHANGELOG.md"))
+memory.store_document(:api_docs,  File.read("docs/api.md"))
+
+# 2. At query time, retrieve the most relevant chunks
+hits    = memory.search_documents(user_query, limit: 3)
+context = hits.map { |h| h[:text] }.join("\n\n")
+
+# 3. Pass context to your robot
+result = robot.run("Use the following context:\n#{context}\n\nQuestion: #{user_query}")
+```
+
+### Memory API Summary
+
+| Method | Description |
+|--------|-------------|
+| `memory.store_document(key, text)` | Embed and store a document |
+| `memory.search_documents(query, limit: 5)` | Search by semantic similarity |
+| `memory.document_keys` | List stored keys |
+| `memory.delete_document(key)` | Remove a document |
+
+### Dependency
+
+`fastembed` is a core RobotLab dependency — no optional gem required. The ONNX model is downloaded on first use.
+
+---
+
+## See Also
+
+- [Observability Guide](observability.md)
+- [Example 25 — Chat History Search](../../examples/25_history_search.rb)
+- [Example 26 — Embedding-Based Document Store](../../examples/26_document_store.rb)