robot_lab 0.2.1 → 0.2.7

data/docs/guides/memory.md

@@ -86,7 +86,7 @@ Memory has reserved keys with special behavior:
 | `:results` | Array | Accumulated robot results |
 | `:messages` | Array | Conversation history |
 | `:session_id` | String | Session identifier for history persistence |
-| `:cache` | SemanticCache | Semantic cache (read-only after init) |
+| `:cache` | `RubyLLM::SemanticCache` module, or `nil` | Semantic cache (read-only after init). Set at construction time; `nil` when built with `enable_cache: false` |
 
 ### The Data Hash
 
@@ -143,6 +143,16 @@ You can also clear just the custom keys without resetting reserved keys:
 robot.memory.clear  # Clears non-reserved keys only
 ```
 
+> [!WARNING]
+> `reset_memory` resets **only the key-value store**. It does not touch the
+> robot's conversation history — the chat still holds every prior message, and
+> the LLM will still see them. Clearing the transcript is a separate call:
+>
+> ```ruby
+> robot.reset_memory                        # key-value store only
+> robot.clear_messages(keep_system: true)   # conversation history only
+> ```
+
 ## Network Shared Memory
 
 When robots run in a network, they share the network's memory instead of using their own inherent memory. This allows robots to communicate through shared state:
@@ -190,11 +200,40 @@ results = memory.get(:sentiment, :entities, :keywords, wait: 60)
 # => { sentiment: {...}, entities: [...], keywords: [...] }
 ```
 
-Each blocking wait is backed by an `IO.pipe` pair (`Waiter` class). Calling `signal` writes one byte per waiting caller, so all threads blocked on `IO.select` wake immediately. This design works cleanly with Ruby's Async fiber scheduler — no mutex contention or spurious wakeups.
+> [!WARNING]
+> A blocking `get` that expires **raises `RobotLab::AwaitTimeout`** — it does
+> not return `nil`. (A *non*-blocking `get`, the default, returns `nil` for a
+> missing key.) Wrap it if a missing value is survivable:
+>
+> ```ruby
+> begin
+>   memory.get(:sentiment, wait: 30)
+> rescue RobotLab::AwaitTimeout => e
+>   # => "Timeout waiting for :sentiment after 30 seconds"
+>   nil
+> end
+> ```
+
+> [!CAUTION]
+> With multiple keys the timeout is applied **per missing key**, not to the call
+> as a whole. `memory.get(:a, :b, :c, wait: 30)` awaits the missing keys
+> sequentially, each with its own fresh 30-second budget.
+>
+> It does not, however, spend the whole 90 seconds before reporting: the first
+> key whose wait expires **raises `AwaitTimeout` immediately**, aborting the call
+> — so the keys that were already resolved are lost along with the ones not yet
+> attempted. The 90 seconds is the worst case only for a call that *succeeds*
+> (each key arriving just under its own deadline).
+
+Each blocking wait is backed by an `IO.pipe` pair (`Waiter` class). The waiting
+side calls `@read_io.wait_readable(timeout)`; `signal` writes one byte per
+waiting caller so every blocked waiter wakes exactly once. `wait_readable`
+yields to Ruby's Async fiber scheduler when one is installed — no mutex
+contention or spurious wakeups.
 
 ### Subscriptions
 
-Subscribe to key changes with asynchronous callbacks:
+Subscribe to key changes:
 
 ```ruby
 # Subscribe to a single key
@@ -214,6 +253,24 @@ memory.subscribe_pattern("analysis:*") do |change|
 end
 ```
 
+> [!IMPORTANT]
+> Subscription callbacks are dispatched through `Async { }`. Inside a running
+> Async reactor that defers them; **outside one — which is the normal case for
+> plain Ruby, Rails request threads, and tests — the block runs synchronously
+> on the writer's thread**, before `memory[:key] = value` returns:
+>
+> ```ruby
+> order = []
+> memory.subscribe(:k) { |c| order << "callback" }
+> order << "before-set"
+> memory[:k] = 1
+> order << "after-set"
+> order   # => ["before-set", "callback", "after-set"]
+> ```
+>
+> Keep subscription callbacks fast, and never assume the writer has moved on by
+> the time your callback runs.
+
 ### Unsubscribe
 
 ```ruby
@@ -243,6 +300,8 @@ Memory can be exported and reconstructed:
 # Export to hash
 hash = robot.memory.to_h
 # => { data: {...}, results: [...], messages: [...], session_id: "...", custom: {...} }
+# to_h is compacted: nil entries are dropped, so an unset :session_id (and the
+# :cache key, which is never exported) simply will not appear.
 
 # Export to JSON
 json = robot.memory.to_json
@@ -279,34 +338,60 @@ memory[:stage] = "response"
 
 ### Caching Expensive Operations
 
+A tool reaches memory through its owning robot. Subclass `RobotLab::Tool` (which
+has a `robot` accessor) and attach an **instance constructed with `robot:`** —
+that is the only supported way for tool code to read and write robot memory:
+
 ```ruby
-class FetchUser < RubyLLM::Tool
+class FetchUser < RobotLab::Tool
   description "Fetch user details by ID"
   param :user_id, type: :string, desc: "User ID"
 
   def execute(user_id:)
-    cache_key = "cache:user:#{user_id}"
+    cache_key = :"cache:user:#{user_id}"
 
-    # Check robot's memory for cached value
-    # (In practice, you'd access memory through the robot's context)
-    cached = Thread.current[:robot_memory]&.[](cache_key.to_sym)
+    cached = robot&.memory&.[](cache_key)
     return cached if cached
 
-    # Fetch and cache
     user = User.find(user_id).to_h
-    Thread.current[:robot_memory]&.[]=(cache_key.to_sym, user)
+    robot&.memory&.[]=(cache_key, user)
     user
   end
 end
+
+robot = RobotLab.build(name: "support", system_prompt: "...")
+robot.local_tools << FetchUser.new(robot: robot)
+
+robot.run("Look up user 42", tools: :inherit)
 ```
 
+> [!WARNING]
+> There is **no thread-local memory handle** in RobotLab — nothing anywhere in
+> the codebase ever assigns `Thread.current[:robot_memory]`. A tool that reads
+> it will always see `nil` and silently cache nothing. Go through `robot.memory`
+> as above, and remember that `FetchUser.new` without `robot:` leaves `robot`
+> `nil`.
+
 ### Semantic Caching
 
-Memory includes a semantic cache for LLM response caching:
+Memory exposes a semantic cache for LLM response caching. It is on by default
+and becomes `nil` when you opt out with `enable_cache: false`:
+
+```ruby
+RobotLab.create_memory.cache                       # => RubyLLM::SemanticCache
+RobotLab.create_memory(enable_cache: false).cache  # => nil
+
+RobotLab.build(name: "x", system_prompt: "…").memory.cache
+# => RubyLLM::SemanticCache
+RobotLab.build(name: "x", system_prompt: "…", enable_cache: false).memory.cache
+# => nil
+```
+
+Guard for `nil` in any code that might run against a cache-disabled memory.
 
 ```ruby
 # Access the semantic cache
-cache = robot.memory.cache  # => RubyLLM::SemanticCache
+cache = robot.memory.cache
 
 # Use it to cache semantically similar queries
 response = cache.fetch("What is Ruby?") do
@@ -314,6 +399,12 @@ response = cache.fetch("What is Ruby?") do
 end
 ```
 
+> [!NOTE]
+> `memory.cache` is the `RubyLLM::SemanticCache` **module itself**, not a
+> per-memory instance. Its cache store, vector store, and configuration are
+> process-global — two memories with `enable_cache: true` share one cache. It
+> also embeds every query, so `fetch` costs an embedding call.
+
 ## Best Practices
 
 ### 1. Use Descriptive Keys

data/docs/guides/observability.md

@@ -10,6 +10,7 @@ Facilities that help you monitor, control, improve, and scale robot behaviour:
 - **Context Window Compression** — prune irrelevant history to stay within token budgets
 - **Convergence Detection** — detect when independent agents reach the same conclusion
 - **Structured Delegation** — synchronous inter-robot calls with duration and token metadata
+- **Live Narration** — an opt-in console feed of what a robot is doing as it happens
 
 ---
 
@@ -91,6 +92,41 @@ result = fresh.run("Explain memoization.")
 puts result.input_tokens  # smallest possible — no prior history
 ```
 
+### Budgets (Token & Cost)
+
+Where token/cost tracking above is purely observational, `token_budget:` and `cost_budget:` make it enforceable — a `Robot` refuses to keep spending once a configured limit is reached:
+
+```ruby
+robot = RobotLab.build(
+  name: "capped",
+  system_prompt: "You are a concise assistant.",
+  token_budget: 10_000,  # cumulative input + output tokens
+  cost_budget: 0.50      # cumulative $ across all runs
+)
+```
+
+Enforcement happens in two layers, backed by a thread-safe `RobotLab::Budget::Ledger`:
+
+- **Before the call** — `run()` reserves whatever remains of each configured dimension. If a *prior* call already exhausted the budget, the reservation raises `RobotLab::BudgetExceeded` immediately, refusing the call outright before spending anything on it.
+- **After the call** — actual usage (tokens from the result, cost from the response, when the provider reports pricing) replaces the reservation. If *this* call's actual usage pushes cumulative usage over budget, `RobotLab::InferenceError` is raised (unavoidable for the call that causes the overage, since totals aren't known until the response comes back) — the same error `token_budget` alone has always raised.
+
+```ruby
+begin
+  robot.run("Another expensive task")
+rescue RobotLab::BudgetExceeded => e
+  puts e.message  # "budget exceeded for cost: 0.51 > 0.5" — refused before spending
+rescue RobotLab::InferenceError => e
+  puts e.message
+  # "Cost budget exceeded: $0.523100 used, budget is $0.500000" — this call pushed it over
+  # (the cost message is formatted with %.6f; the token message is not formatted:
+  #  "Token budget exceeded: 10412 tokens used, budget is 10000")
+end
+```
+
+A dimension with no configured limit (e.g. `cost_budget` when only `token_budget` is set) is treated as unlimited and never raises. Both fields are also available on `RunConfig` and cascade through the same global → network → robot hierarchy as other infrastructure fields (see [RunConfig](../getting-started/configuration.md#runconfig-shared-operational-defaults)).
+
+This is a native alternative to the hand-rolled `BudgetHook` pattern in the [Hooks guide](hooks.md#cost-enforcement) for the common case of a per-robot token or dollar ceiling; reach for a hook instead when you need cross-robot session totals or custom accounting.
+
 ---
 
 ## Tool Loop Circuit Breaker
@@ -110,12 +146,20 @@ robot = RobotLab.build(
   local_tools: [StepTool],
   max_tool_rounds: 10
 )
+```
 
+> [!WARNING]
+> `run` defaults to `tools: :none`, so a plain `robot.run("...")` sends the LLM
+> **no** tools — even when `local_tools:` were attached at build time — and the
+> breaker can never fire. Pass `tools: :inherit` on the call to actually send the
+> attached tools.
+
+```ruby
 begin
-  robot.run("Run all steps.")
+  robot.run("Run all steps.", tools: :inherit)
 rescue RobotLab::ToolLoopError => e
-  puts "Circuit breaker fired: #{e.message}"
-  # => "Circuit breaker fired: Tool call limit of 10 exceeded"
+  puts e.message
+  # => "Circuit breaker triggered: 11 tool calls exceeded max_tool_rounds (10)"
 end
 ```
 
@@ -137,6 +181,8 @@ Error: tool_use ids were found without tool_result blocks immediately after
 Call `clear_messages` to flush the corrupted history before reusing the robot. The system prompt and all configuration (tools, `max_tool_rounds`, etc.) are preserved:
 
 ```ruby
+begin
+  robot.run("Keep calling the tool.", tools: :inherit)
 rescue RobotLab::ToolLoopError => e
   puts "Breaker fired: #{e.message}"
 end
@@ -158,7 +204,7 @@ unguarded = RobotLab.build(
   system_prompt: "Use the provided tool to answer questions.",
   local_tools: [DoubleTool]
 )
-result = unguarded.run("Double the number 21 using the tool.")
+result = unguarded.run("Double the number 21 using the tool.", tools: :inherit)
 puts result.reply  # "The result is 42."
 ```
 
@@ -172,16 +218,18 @@ puts result.reply  # "The result is 42."
 
 ### doom_loop_threshold
 
+Doom loop detection is **always on** — the detector is installed unconditionally on every `run()`. `doom_loop_threshold:` does not enable it; it only tunes the number of repetitions after which it fires. The default is `3`.
+
 ```ruby
 robot = RobotLab.build(
   name: "runner",
   system_prompt: "Execute all steps.",
   local_tools: [StepTool],
-  doom_loop_threshold: 3
+  doom_loop_threshold: 5   # tune the always-on detector; default is 3
 )
 ```
 
-Set `doom_loop_threshold:` to the number of repetitions after which the detector fires. It catches two patterns:
+The detector catches two patterns:
 
 - **Consecutive repetition** — `[A, A, A]` (same tool called N times in a row)
 - **Cyclic repetition** — `[A, B, C, A, B, C, A, B, C]` (same sequence repeated N times)
@@ -219,27 +267,38 @@ Long-running robots accumulate conversation history. Eventually, the cumulative
 
 ### auto_compact
 
-Set `auto_compact:` to have the robot compress its history automatically before each `run()`:
+`auto_compact` and `compact_threshold` are **`RunConfig` fields only** — they are not constructor keyword arguments. Build a `RunConfig` and pass it as `config:`:
 
 ```ruby
 # Compact when estimated token usage exceeds 80% of the model's context window
+config = RobotLab::RunConfig.new(auto_compact: :context_window)
+
 robot = RobotLab.build(
   name: "analyst",
   system_prompt: "You are a research analyst.",
-  auto_compact: :context_window
+  config: config
 )
 ```
 
+> [!WARNING]
+> `Robot#initialize` has a closed keyword list — it takes no `**rest`. Passing
+> `auto_compact:` or `compact_threshold:` directly to `RobotLab.build` raises
+> `ArgumentError: unknown keyword`.
+
 ### Tuning the Threshold
 
 `compact_threshold:` sets the fraction of the model's context window that triggers compaction. Defaults to `0.80` (80%):
 
 ```ruby
+config = RobotLab::RunConfig.new(
+  auto_compact:      :context_window,
+  compact_threshold: 0.70   # compact earlier, at 70%
+)
+
 robot = RobotLab.build(
   name: "analyst",
   system_prompt: "You are a research analyst.",
-  auto_compact:      :context_window,
-  compact_threshold: 0.70   # compact earlier, at 70%
+  config: config
 )
 ```
 
@@ -248,16 +307,20 @@ robot = RobotLab.build(
 Pass a `Proc` to take full control — the proc decides both when and how to compact:
 
 ```ruby
-robot = RobotLab.build(
-  name: "analyst",
-  system_prompt: "You are a research analyst.",
+config = RobotLab::RunConfig.new(
   auto_compact: ->(r) {
     r.compress_history(recent_turns: 5) if r.chat.messages.size > 40
   }
 )
+
+robot = RobotLab.build(
+  name: "analyst",
+  system_prompt: "You are a research analyst.",
+  config: config
+)
 ```
 
-The proc receives the robot instance and is called before every `run()` when messages are non-empty.
+The proc receives the robot instance and is called once per `run()` when messages are non-empty.
 
 ### Options
 
@@ -267,13 +330,6 @@ The proc receives the robot instance and is called before every `run()` when mes
 | `:context_window` | Compact when estimated token usage exceeds `compact_threshold` fraction of model's context window |
 | `Proc` | Called with the robot; application decides when and how to compact |
 
-Via `RunConfig`:
-
-```ruby
-config = RobotLab::RunConfig.new(auto_compact: :context_window, compact_threshold: 0.75)
-robot  = RobotLab.build(name: "analyst", system_prompt: "...", config: config)
-```
-
 Requires the `classifier` gem (`~> 2.3`) when using `:context_window`. Without it, a `RobotLab::DependencyError` is caught and logged rather than raised, so the robot continues running uncompressed.
 
 ---
@@ -354,20 +410,33 @@ end
 
 After all three runs, `reviewer.learnings` contains up to three insights (fewer if any are subsets of others).
 
-### Durable Learning (learn: Constructor Shorthand)
+### Durable Learning (the `:learn` hook family)
+
+Core RobotLab keeps learnings for the life of the process only. Cross-session persistence is supplied by the [`robot_lab-durable`](https://github.com/MadBomber/robot_lab-durable) gem, which registers a `RobotLab::Hook` on the `:learn` family — there is no `learn:` constructor shorthand.
 
-The `robot_lab-durable` gem adds automatic end-of-session learning promotion. Enable it with `learn: true` in the constructor:
+> [!WARNING]
+> `learn:` and `learn_domain:` are **not** constructor keyword arguments and do
+> not exist anywhere in the codebase. `RobotLab.build(learn: true)` raises
+> `ArgumentError: unknown keyword: :learn`.
+
+The wiring is the ordinary hook registration described in the [Hooks guide](hooks.md) — an `on_learn` handler receives each learning after session-level deduplication and decides whether to persist it:
 
 ```ruby
-reviewer = RobotLab.build(
-  name:         "reviewer",
-  system_prompt: "You are a Ruby code reviewer.",
-  learn:        true,
-  learn_domain: "ruby_review"   # optional namespace for the durable store
-)
+class DurableLearnHook < RobotLab::Hook
+  self.namespace = :durable
+
+  def self.on_learn(ctx)
+    return unless ctx.stored
+
+    DurableStore.promote(text: ctx.text, robot: ctx.robot.name, domain: ctx.local.domain)
+  end
+end
+
+reviewer = RobotLab.build(name: "reviewer", system_prompt: "You are a Ruby code reviewer.")
+reviewer.on(DurableLearnHook, context: { domain: "ruby_review" })
 ```
 
-At the end of each session, the robot reflects on its observations and promotes durable insights to a YAML-backed store that persists across process restarts. On the next run, those stored insights are automatically reloaded as learnings.
+The extension promotes durable insights to a YAML-backed store that persists across process restarts; see the gem's own README for its registration entry point.
 
 ### Memory Persistence
 
@@ -489,29 +558,41 @@ Texts shorter than 30 characters always return `0.0`.
 | Same topic, different emphasis | 0.45 – 0.60 |
 | Unrelated | < 0.15 |
 
-### Router Fast-Path Pattern
+### Reconciler Fast-Path Pattern
 
-Skip the reconciler when verifiers agree:
+Skip the reconciler when verifiers agree. RobotLab has no router object — routing is done by declaring the optional branch as a task with `depends_on: :optional` and having a preceding robot call `result.activate(:task_name)` on it. Subclass `RobotLab::Robot` and override `#call` to make the decision:
 
 ```ruby
-router = ->(args) do
-  a = args.context[:verifier_a]&.reply.to_s
-  b = args.context[:verifier_b]&.reply.to_s
+class ConvergenceGate < RobotLab::Robot
+  def call(result)
+    # result.context is keyed by ROBOT name, not task name
+    a = result.context[:verifier_a]&.reply.to_s
+    b = result.context[:verifier_b]&.reply.to_s
 
-  if RobotLab::Convergence.detected?(a, b)
-    nil                  # both agree — network halts, no reconciler call
-  else
-    ["reconciler"]       # diverged — send to reconciler
+    return result if RobotLab::Convergence.detected?(a, b)  # agree — reconciler stays dormant
+
+    result.activate(:reconciler)                            # diverged — activate the branch
   end
 end
 
-network = RobotLab.create_network(
-  name:   "fact_check",
-  robots: [verifier_a, verifier_b, reconciler],
-  router: router
-)
+network = RobotLab.create_network(name: "fact_check") do
+  task :verifier_a, verifier_a, depends_on: :none
+  task :verifier_b, verifier_b, depends_on: :none
+  task :gate,       ConvergenceGate.new(name: "gate"), depends_on: %i[verifier_a verifier_b]
+  task :reconciler, reconciler, depends_on: :optional
+end
+
+result = network.run(message: "Is the deployment healthy?")
+result.activated_steps  # => [] when they agreed, [:reconciler] when they diverged
 ```
 
+The gate robot never calls the LLM — overriding `#call` replaces the default "run and continue" behaviour entirely, so the decision costs nothing.
+
+> [!NOTE]
+> `result.context` is keyed by the **robot's** name (`@name`), not the task name.
+> The lookups above work because each verifier's `name:` matches its task label.
+> If they differ, index by the robot name.
+
 Tune `threshold:` to control how strictly "agreement" is defined. A lower threshold (e.g., `0.6`) accepts more variation between verifiers; a higher threshold (e.g., `0.9`) only fast-paths near-identical responses.
 
 ### Optional Dependency
@@ -606,13 +687,47 @@ puts "#{analysis.robot_name} (#{analysis.duration.round(2)}s): #{analysis.reply}
 
 ---
 
+## Live Narration (`RobotLab::Narrator`)
+
+`RobotLab::Narrator` is an opt-in [Hook](hooks.md) that narrates what a robot is doing as it happens, to `$stderr` (or any `IO`), so a run is never silent between events. It complements `RobotLab::Audit` (the `robot_lab-audit` gem, which records a persistent history for post-mortem analysis) with a live, human-facing console feed — the two are independent hooks and can both be registered at once.
+
+Enable it globally (applies to every robot run, including networks):
+
+```ruby
+RobotLab::Narrator.enable!                       # narrate to $stderr
+RobotLab::Narrator.enable!(output: $stdout)       # or any IO
+```
+
+Or register it like any hook for a narrower scope:
+
+```ruby
+robot.on(RobotLab::Narrator)
+network.on(RobotLab::Narrator)
+```
+
+Once registered, a run prints a line per event:
+
+```
+  · math_bot: thinking…
+  · → calculate operation="add"
+  · → calculate operation="multiply"
+```
+
+- Once per `run`: `"<robot name>: thinking…"` — Narrator hooks `before_llm_generation`, which fires exactly once per `robot.run`; the provider's tool loop happens *inside* that hook, so the line is not repeated per LLM API call
+- Before each tool call: `"→ <tool name> <first arg>=<value>"` — only the *first* argument is shown, truncated to 80 characters
+- After a tool call: `"  ✗ <error message>"` — printed only when the tool raised; silent on success
+
+Narrator uses `IO#puts` rather than `Kernel#warn`, since `warn` is silenced whenever Ruby warnings are disabled (`$VERBOSE` is `nil`, the common case under `bundle exec`). All three hooks rescue internally, so a narration failure never breaks the underlying run.
+
+---
+
 ## See Also
 
-- [Robot API](../api/core/robot.md#token--cost-tracking)
-- [Example 19 — Token & Cost Tracking](../../examples/19_token_tracking.rb)
-- [Example 20 — Tool Loop Circuit Breaker](../../examples/20_circuit_breaker.rb)
-- [Example 21 — Learning Accumulation Loop](../../examples/21_learning_loop.rb)
-- [Example 22 — Context Window Compression](../../examples/22_context_compression.rb)
-- [Example 23 — Convergence Detection](../../examples/23_convergence.rb)
-- [Example 24 — Structured Delegation](../../examples/24_structured_delegation.rb)
+- [Robot API](../api/core/robot.md#token-cost-tracking)
+- [examples/19_token_tracking.rb](https://github.com/MadBomber/robot_lab/blob/main/examples/19_token_tracking.rb) — Token & Cost Tracking
+- [examples/20_circuit_breaker.rb](https://github.com/MadBomber/robot_lab/blob/main/examples/20_circuit_breaker.rb) — Tool Loop Circuit Breaker
+- [examples/21_learning_loop.rb](https://github.com/MadBomber/robot_lab/blob/main/examples/21_learning_loop.rb) — Learning Accumulation Loop
+- [examples/22_context_compression.rb](https://github.com/MadBomber/robot_lab/blob/main/examples/22_context_compression.rb) — Context Window Compression
+- [examples/23_convergence.rb](https://github.com/MadBomber/robot_lab/blob/main/examples/23_convergence.rb) — Convergence Detection
+- [examples/24_structured_delegation.rb](https://github.com/MadBomber/robot_lab/blob/main/examples/24_structured_delegation.rb) — Structured Delegation
 - [RunConfig reference](../getting-started/configuration.md#runconfig-shared-operational-defaults)