claude_swarm 1.0.9 → 1.0.11

data/docs/v2/reference/ruby-dsl.md

@@ -32,6 +32,242 @@ result = swarm.execute("Task prompt")
 
 ## Top-Level Methods
 
+### SwarmSDK.agent
+
+Register an agent globally for reuse across multiple swarms.
+
+**Signature:**
+```ruby
+SwarmSDK.agent(name) {|builder| ... } → void
+```
+
+**Parameters:**
+- `name` (Symbol, required): Unique agent name
+- `block` (required): Agent configuration block (uses [Agent Builder DSL](#agent-builder-dsl))
+
+**Description:**
+Registers an agent definition in a global registry, allowing it to be referenced by name in any swarm or workflow without re-defining it. This promotes code reuse and separation of concerns.
+
+**Duplicate Registration:**
+Raises `ArgumentError` if an agent with the same name is already registered. Use `SwarmSDK.clear_agent_registry!` to reset.
+
+**Example - Define agents in separate files:**
+```ruby
+# agents/backend.rb
+SwarmSDK.agent :backend do
+  model "claude-sonnet-4"
+  description "Backend developer"
+  system_prompt "You build APIs and databases"
+  tools :Read, :Edit, :Bash
+  # Note: Don't set delegates_to here - it's swarm-specific!
+end
+
+# agents/database.rb
+SwarmSDK.agent :database do
+  model "claude-sonnet-4"
+  description "Database specialist"
+  system_prompt "You design schemas and write migrations"
+  tools :Read, :Edit
+end
+```
+
+**Example - Reference in swarms:**
+```ruby
+# Load agent definitions
+require_relative "agents/backend"
+require_relative "agents/database"
+
+# Reference by name and configure delegation (swarm-specific)
+swarm = SwarmSDK.build do
+  name "Dev Team"
+  lead :backend
+
+  agent :backend do
+    delegates_to :database  # Delegation is swarm-specific
+  end
+  agent :database    # Pulls from registry as-is
+end
+```
+
+**Example - Override registry settings:**
+```ruby
+swarm = SwarmSDK.build do
+  name "Extended Team"
+  lead :backend
+
+  # Registry config applied first, then override block
+  agent :backend do
+    delegates_to :database, :cache  # Set delegation for this swarm
+    tools :CustomTool               # Adds to registry tools
+    timeout 300                     # Overrides registry timeout
+  end
+end
+```
+
+**Workflow Auto-Resolution:**
+Agents referenced in workflow nodes are automatically resolved from the registry if not defined at workflow level:
+
+```ruby
+SwarmSDK.workflow do
+  name "Pipeline"
+  start_node :build
+
+  # No need to define :backend here - auto-resolved from registry!
+  node :build do
+    agent(:backend)  # Resolved from global registry
+  end
+
+  node :test do
+    agent(:backend).delegates_to(:database)  # Both resolved from registry
+  end
+end
+```
+
+**Use Cases:**
+- **Code reuse**: Define agents once, use in multiple swarms
+- **Separation of concerns**: Agent definitions in dedicated files
+- **Testing**: Clear registry between tests with `SwarmSDK.clear_agent_registry!`
+- **Organization**: Large projects with many agents
+
+---
+
+### SwarmSDK.clear_agent_registry!
+
+Clear all registered agents from the global registry.
+
+**Signature:**
+```ruby
+SwarmSDK.clear_agent_registry! → void
+```
+
+**Description:**
+Removes all agents from the global registry. Primarily used for testing to ensure isolation between test cases.
+
+**Example:**
+```ruby
+# In test setup
+def setup
+  SwarmSDK.clear_agent_registry!
+end
+
+# Or in RSpec
+before(:each) do
+  SwarmSDK.clear_agent_registry!
+end
+```
+
+---
+
+### SwarmSDK.build
+
+Build a simple multi-agent swarm.
+
+**Signature:**
+```ruby
+SwarmSDK.build(allow_filesystem_tools: nil) {|builder| ... } → Swarm
+```
+
+**Returns:**
+- `Swarm` - Always returns a Swarm instance (simple multi-agent collaboration)
+
+**Parameters:**
+- `allow_filesystem_tools` (Boolean, optional): Override global filesystem tools setting for this swarm
+- `block` (required): Configuration block using the DSL
+
+**Description:**
+Creates a simple multi-agent swarm where agents can collaborate through delegation. Use this for most use cases where you need multiple AI agents working together.
+
+**Example:**
+```ruby
+swarm = SwarmSDK.build do
+  name "Development Team"
+  lead :backend
+
+  agent :backend do
+    model "gpt-4"
+    description "Backend developer"
+    tools :Read, :Write, :Bash
+    delegates_to :tester
+  end
+
+  agent :tester do
+    model "gpt-4o-mini"
+    description "Test specialist"
+    tools :Read, :Bash
+  end
+end
+
+result = swarm.execute("Build authentication API")
+```
+
+**Notes:**
+- Cannot use `node` definitions (raises `ConfigurationError`)
+- Must specify `lead` agent
+- For multi-stage workflows, use `SwarmSDK.workflow` instead
+
+---
+
+### SwarmSDK.workflow
+
+Build a multi-stage workflow with nodes.
+
+**Signature:**
+```ruby
+SwarmSDK.workflow(allow_filesystem_tools: nil) {|builder| ... } → Workflow
+```
+
+**Returns:**
+- `Workflow` - Always returns a Workflow instance (multi-stage pipeline)
+
+**Parameters:**
+- `allow_filesystem_tools` (Boolean, optional): Override global filesystem tools setting for this workflow
+- `block` (required): Configuration block using the DSL
+
+**Description:**
+Creates a multi-stage workflow where different teams of agents execute in sequence. Each node is an independent swarm execution with its own agent configuration and delegation topology. Use this for complex pipelines with distinct stages (planning → implementation → testing).
+
+**Example:**
+```ruby
+workflow = SwarmSDK.workflow do
+  name "Build Pipeline"
+  start_node :planning
+
+  agent :architect do
+    model "gpt-4"
+    description "System architect"
+  end
+
+  agent :coder do
+    model "gpt-4"
+    description "Implementation specialist"
+  end
+
+  node :planning do
+    agent(:architect)
+  end
+
+  node :implementation do
+    agent(:coder)
+    depends_on :planning
+
+    # Transform input from planning node
+    input do |ctx|
+      "Implement this plan:\n#{ctx.content}"
+    end
+  end
+end
+
+result = workflow.execute("Build authentication system")
+```
+
+**Notes:**
+- Cannot use `lead` (raises `ConfigurationError`)
+- Must specify `start_node`
+- Nodes execute in topological order based on `depends_on`
+- Each node can have different agents and delegation topology
+
+---
+
 ### SwarmSDK.configure
 
 Configure global SwarmSDK settings.
@@ -51,6 +287,8 @@ SwarmSDK.configure {|config| ... } → void
 - `webfetch_max_tokens` (Integer): Maximum tokens for WebFetch LLM responses (default: 4096)
 - `allow_filesystem_tools` (Boolean): Enable/disable filesystem tools globally (default: true)
 
+**See [Configuration Reference](configuration_reference.md) for the complete list of 45+ configuration options including API keys, timeouts, limits, and more.**
+
 **Description:**
 Global configuration that applies to all swarms.
 
@@ -88,13 +326,220 @@ SwarmSDK.configure do |config|
 end
 
 # Can also set directly
-SwarmSDK.settings.allow_filesystem_tools = false
+SwarmSDK.config.allow_filesystem_tools = false
 
 # Or via environment variable
 ENV['SWARM_SDK_ALLOW_FILESYSTEM_TOOLS'] = 'false'
 
 # Reset to defaults (disables WebFetch LLM processing, enables filesystem tools)
-SwarmSDK.reset_settings!
+SwarmSDK.reset_config!
+```
+
+---
+
+### Custom Tool Registration
+
+Simple API for registering custom tools without creating full plugins.
+
+#### SwarmSDK.register_tool
+
+Register a custom RubyLLM::Tool for use in swarms.
+
+**Signatures:**
+```ruby
+SwarmSDK.register_tool(tool_class) → Symbol
+SwarmSDK.register_tool(name, tool_class) → Symbol
+```
+
+**Parameters:**
+- `tool_class` (Class): RubyLLM::Tool subclass to register
+- `name` (Symbol, optional): Explicit tool name (if not provided, name is inferred)
+
+**Returns:**
+- `Symbol`: The registered tool name
+
+**Description:**
+Registers a custom tool globally so it can be used in any swarm. Tools are registered by name and can be referenced in agent configurations.
+
+**Tool Lookup Order:**
+1. Plugin tools (from PluginRegistry)
+2. Custom tools (from CustomToolRegistry)
+3. Built-in tools (from Tools::Registry)
+
+**Name Inference:**
+If no name is provided, the tool name is inferred from the class name by:
+1. Taking the last component (e.g., `MyApp::Tools::WeatherTool` → `WeatherTool`)
+2. Removing the `Tool` suffix if present (e.g., `WeatherTool` → `Weather`)
+
+**Context Support:**
+Tools can declare `creation_requirements` to receive agent context:
+- `:agent_name` - Agent identifier (Symbol)
+- `:directory` - Agent's working directory (String)
+
+**Validation:**
+- Tool class must inherit from `RubyLLM::Tool`
+- Cannot override built-in tools (Read, Write, Edit, etc.)
+- Cannot override plugin tools (MemoryWrite, etc.)
+- Cannot register the same name twice
+
+**When to Use:**
+- Simple, stateless tools
+- No persistent storage needed
+- No lifecycle hooks required
+- Quick setup without plugin infrastructure
+
+**When to Use Plugins Instead:**
+- Need per-agent storage
+- Need lifecycle hooks (on_agent_initialized, on_user_message, etc.)
+- Want to contribute to system prompts
+- Building suite of related tools
+
+**Example - Simple Tool:**
+```ruby
+class WeatherTool < RubyLLM::Tool
+  description "Get weather for a city"
+  param :city, type: "string", required: true
+
+  def execute(city:)
+    # Call weather API
+    "Weather in #{city}: Sunny, 72°F"
+  end
+end
+
+# Register with inferred name (:Weather)
+SwarmSDK.register_tool(WeatherTool)
+
+# Use in swarm
+SwarmSDK.build do
+  agent :assistant do
+    model "claude-sonnet-4"
+    tools :Weather, :Read  # Mix custom and built-in tools
+  end
+end
+```
+
+**Example - Context-Aware Tool:**
+```ruby
+class AgentLogTool < RubyLLM::Tool
+  # Declare required context
+  def self.creation_requirements
+    [:agent_name, :directory]
+  end
+
+  description "Log a message for this agent"
+  param :message, type: "string", required: true
+
+  def initialize(agent_name:, directory:)
+    super()
+    @agent_name = agent_name
+    @log_file = File.join(directory, ".agent.log")
+  end
+
+  def execute(message:)
+    File.append(@log_file, "[#{@agent_name}] #{message}\n")
+    "Logged message to #{@log_file}"
+  end
+end
+
+SwarmSDK.register_tool(:AgentLog, AgentLogTool)
+```
+
+**Example - Explicit Name:**
+```ruby
+# Register with custom name
+SwarmSDK.register_tool(:GetWeather, WeatherTool)
+
+SwarmSDK.build do
+  agent :assistant do
+    tools :GetWeather  # Use the custom name
+  end
+end
+```
+
+#### SwarmSDK.custom_tool_registered?
+
+Check if a custom tool is registered.
+
+**Signature:**
+```ruby
+SwarmSDK.custom_tool_registered?(name) → Boolean
+```
+
+**Parameters:**
+- `name` (Symbol | String): Tool name to check
+
+**Returns:**
+- `true` if tool is registered, `false` otherwise
+
+**Example:**
+```ruby
+SwarmSDK.register_tool(WeatherTool)
+SwarmSDK.custom_tool_registered?(:Weather)  # => true
+SwarmSDK.custom_tool_registered?("Weather") # => true
+SwarmSDK.custom_tool_registered?(:Unknown)  # => false
+```
+
+#### SwarmSDK.custom_tools
+
+List all registered custom tool names.
+
+**Signature:**
+```ruby
+SwarmSDK.custom_tools → Array<Symbol>
+```
+
+**Returns:**
+- Array of registered tool names (as symbols)
+
+**Example:**
+```ruby
+SwarmSDK.register_tool(WeatherTool)
+SwarmSDK.register_tool(:Stock, StockPriceTool)
+
+SwarmSDK.custom_tools  # => [:Weather, :Stock]
+```
+
+#### SwarmSDK.unregister_tool
+
+Remove a registered custom tool.
+
+**Signature:**
+```ruby
+SwarmSDK.unregister_tool(name) → Class | nil
+```
+
+**Parameters:**
+- `name` (Symbol | String): Tool name to unregister
+
+**Returns:**
+- The unregistered tool class, or `nil` if not found
+
+**Example:**
+```ruby
+removed = SwarmSDK.unregister_tool(:Weather)
+removed  # => WeatherTool
+```
+
+#### SwarmSDK.clear_custom_tools!
+
+Clear all registered custom tools.
+
+**Signature:**
+```ruby
+SwarmSDK.clear_custom_tools! → void
+```
+
+**Description:**
+Removes all custom tool registrations. Primarily useful for testing to ensure clean state between test runs.
+
+**Example:**
+```ruby
+SwarmSDK.register_tool(WeatherTool)
+SwarmSDK.register_tool(StockPriceTool)
+
+SwarmSDK.clear_custom_tools!
+
+SwarmSDK.custom_tools  # => []
 ```
 
 ---
@@ -105,7 +550,7 @@ Build a swarm using the DSL.
 
 **Signature:**
 ```ruby
-SwarmSDK.build(allow_filesystem_tools: nil, &block) → Swarm | NodeOrchestrator
+SwarmSDK.build(allow_filesystem_tools: nil, &block) → Swarm | Workflow
 ```
 
 **Parameters:**
@@ -114,7 +559,7 @@ SwarmSDK.build(allow_filesystem_tools: nil, &block) → Swarm | NodeOrchestrator
 
 **Returns:**
 - `Swarm`: For single-swarm configurations
-- `NodeOrchestrator`: For multi-node workflow configurations
+- `Workflow`: For multi-node workflow configurations
 
 **Example:**
 ```ruby
@@ -400,7 +845,7 @@ scratchpad(mode) → void
 **Parameters:**
 - `mode` (Symbol, required): Scratchpad mode
   - For regular Swarms: `:enabled` or `:disabled`
-  - For NodeOrchestrator (workflows with nodes): `:enabled`, `:per_node`, or `:disabled`
+  - For Workflow (workflows with nodes): `:enabled`, `:per_node`, or `:disabled`
 
 **Default:** `:disabled`
 
@@ -409,8 +854,8 @@ Controls scratchpad availability and sharing behavior:
 
 - **`:enabled`**: Scratchpad tools available (ScratchpadWrite, ScratchpadRead, ScratchpadList)
   - Regular Swarm: All agents share one scratchpad
-  - NodeOrchestrator: All nodes share one scratchpad across the workflow
-- **`:per_node`**: (NodeOrchestrator only) Each node gets isolated scratchpad storage
+  - Workflow: All nodes share one scratchpad across the workflow
+- **`:per_node`**: (Workflow only) Each node gets isolated scratchpad storage
 - **`:disabled`**: No scratchpad tools available
 
 Scratchpad is volatile (in-memory only) and provides temporary storage for cross-agent or cross-node communication.
@@ -423,7 +868,7 @@ SwarmSDK.build do
   scratchpad :disabled # Disable scratchpad (default)
 end
 
-# NodeOrchestrator - shared across nodes
+# Workflow - shared across nodes
 SwarmSDK.build do
   scratchpad :enabled  # All nodes share one scratchpad
 
@@ -432,7 +877,7 @@ SwarmSDK.build do
   start_node :planning
 end
 
-# NodeOrchestrator - isolated per node
+# Workflow - isolated per node
 SwarmSDK.build do
   scratchpad :per_node  # Each node gets its own scratchpad
 
@@ -870,7 +1315,7 @@ tools(*tool_names, include_default: true) → void
 - `ScratchpadWrite`, `ScratchpadRead`, `ScratchpadList`
 
 **Memory tools** (added if agent has `memory` configured):
-- `MemoryWrite`, `MemoryRead`, `MemoryEdit`, `MemoryMultiEdit`, `MemoryGlob`, `MemoryGrep`, `MemoryDelete`
+- `MemoryWrite`, `MemoryRead`, `MemoryEdit`, `MemoryGlob`, `MemoryGrep`, `MemoryDelete`
 
 **Additional tools:**
 - `Write`, `Edit`, `MultiEdit`, `Bash`
@@ -910,7 +1355,7 @@ delegates_to(*agent_names) → void
 
 **Behavior:**
 - Multiple calls are cumulative
-- Creates a `DelegateTaskTo{Agent}` tool for each target (e.g., `DelegateTaskToDatabase`)
+- Creates a `WorkWith{Agent}` tool for each target (e.g., `WorkWithDatabase`)
 
 **Example:**
 ```ruby
@@ -1019,7 +1464,7 @@ memory(&block) → void
 - `directory(string)` - Directory where memory.json will be stored (required)
 
 **Description:**
-Enables persistent memory for the agent. When configured, the agent automatically gets all 7 memory tools (MemoryWrite, MemoryRead, MemoryEdit, MemoryMultiEdit, MemoryGlob, MemoryGrep, MemoryDelete) and a memory system prompt is appended to help the agent use memory effectively.
+Enables persistent memory for the agent. When configured, the agent automatically gets all 6 memory tools (MemoryWrite, MemoryRead, MemoryEdit, MemoryGlob, MemoryGrep, MemoryDelete) and a memory system prompt is appended to help the agent use memory effectively.
 
 Memory is per-agent (isolated) and persistent (survives across sessions).
 
@@ -1962,16 +2407,19 @@ Execute a task using the lead agent.
 
 **Signature:**
 ```ruby
-swarm.execute(prompt, &block) → Result
+swarm.execute(prompt, wait: true, &block) → Result | Async::Task
 ```
 
 **Parameters:**
 - `prompt` (String, required): Task prompt
+- `wait` (Boolean, optional): If true (default), blocks until execution completes. If false, returns Async::Task immediately for non-blocking execution.
 - `block` (optional): Log entry handler for streaming
 
-**Returns:** `Result` object
+**Returns:**
+- `Result` if `wait: true` (default)
+- `Async::Task` if `wait: false`
 
-**Example:**
+**Example - Blocking execution (default):**
 ```ruby
 # Basic execution
 result = swarm.execute("Build a REST API")
@@ -1992,6 +2440,36 @@ else
 end
 ```
 
+**Example - Non-blocking execution with cancellation:**
+```ruby
+# Start non-blocking execution
+task = swarm.execute("Build a REST API", wait: false) do |log_entry|
+  puts "#{log_entry[:type]}: #{log_entry[:agent]}"
+end
+
+# ... do other work ...
+
+# Cancel anytime
+task.stop
+
+# Wait for result (returns nil if cancelled)
+result = task.wait
+if result.nil?
+  puts "Execution was cancelled"
+elsif result.success?
+  puts result.content
+else
+  puts "Error: #{result.error.message}"
+end
+```
+
+**Non-blocking execution details:**
+- **`wait: false`**: Returns `Async::Task` immediately, enabling cancellation via `task.stop`
+- **Cooperative cancellation**: Stops when fiber yields (HTTP I/O, tool boundaries), not immediate
+- **Proper cleanup**: MCP clients, fiber storage, and logging cleaned in task's ensure block
+- **`task.wait` returns nil**: Cancelled tasks return `nil` from `wait` method
+- **Use cases**: Long-running tasks, user-initiated cancellation, timeout handling
+
 ---
 
 ## Result Object
@@ -2088,6 +2566,124 @@ result.to_json → String
 ```
 Convert to JSON string.
 
+**per_agent_usage**
+```ruby
+result.per_agent_usage → Hash<Symbol, Hash>
+```
+Returns per-agent usage breakdown extracted from execution logs.
+
+Each agent entry contains:
+- `input_tokens` - Input tokens for this agent
+- `output_tokens` - Output tokens for this agent
+- `total_tokens` - Total tokens (input + output)
+- `cached_tokens` - Cached tokens used
+- `context_limit` - Context window limit
+- `usage_percentage` - Percentage of context window used
+- `tokens_remaining` - Tokens remaining in context window
+- `input_cost` - Input cost in dollars
+- `output_cost` - Output cost in dollars
+- `total_cost` - Total cost in dollars
+
+**Example:**
+```ruby
+result = swarm.execute("Build authentication")
+usage = result.per_agent_usage
+
+usage[:backend]
+# => {
+#   input_tokens: 15000,
+#   output_tokens: 5000,
+#   total_tokens: 20000,
+#   cached_tokens: 2000,
+#   context_limit: 200000,
+#   usage_percentage: 10.0,
+#   tokens_remaining: 180000,
+#   input_cost: 0.045,
+#   output_cost: 0.075,
+#   total_cost: 0.12
+# }
+```
+
+---
+
+## Swarm Methods
+
+### context_breakdown
+
+Get real-time context usage breakdown for all agents.
+
+**Signature:**
+```ruby
+swarm.context_breakdown → Hash<Symbol, Hash>
+```
+
+**Returns:**
+Hash with agent names as keys and context usage info as values.
+
+**Description:**
+Returns live context metrics for all agents in the swarm, including both primary agents and delegation instances. Useful for monitoring token consumption and costs during execution.
+
+Each agent entry contains:
+- `input_tokens` - Cumulative input tokens
+- `output_tokens` - Cumulative output tokens
+- `total_tokens` - Total tokens (input + output)
+- `cached_tokens` - Cached tokens used
+- `cache_creation_tokens` - Tokens written to cache
+- `effective_input_tokens` - Actual input tokens charged (input - cached)
+- `context_limit` - Context window limit for the model
+- `usage_percentage` - Percentage of context window used
+- `tokens_remaining` - Tokens remaining in context window
+- `input_cost` - Cumulative input cost in dollars
+- `output_cost` - Cumulative output cost in dollars
+- `total_cost` - Cumulative total cost in dollars
+
+**Example:**
+```ruby
+swarm = SwarmSDK.build do
+  name "Dev Team"
+  lead :backend
+
+  agent :backend do
+    model "claude-sonnet-4"
+    delegates_to :tester
+  end
+
+  agent :tester do
+    model "gpt-4o-mini"
+  end
+end
+
+# During or after execution
+breakdown = swarm.context_breakdown
+
+# Primary agent
+breakdown[:backend]
+# => {
+#   input_tokens: 15000,
+#   output_tokens: 5000,
+#   total_tokens: 20000,
+#   cached_tokens: 2000,
+#   cache_creation_tokens: 1000,
+#   effective_input_tokens: 13000,
+#   context_limit: 200000,
+#   usage_percentage: 10.0,
+#   tokens_remaining: 180000,
+#   input_cost: 0.045,
+#   output_cost: 0.075,
+#   total_cost: 0.12
+# }
+
+# Delegation instance
+breakdown[:"tester@backend"]
+# => { ... same structure ... }
+```
+
+**Use Cases:**
+- Monitor token consumption during long-running tasks
+- Track costs per agent in real-time
+- Identify context-heavy agents approaching limits
+- Debug context window issues
+
 ---
 
 ## Context Management
@@ -2421,7 +3017,7 @@ Jump to a different node with custom content, bypassing normal dependency order.
 - `node_name` (Symbol, required): Target node name
 - `content` (String, required): Content to pass to target node (validated non-nil)
 
-**Returns:** Control hash (processed by NodeOrchestrator)
+**Returns:** Control hash (processed by Workflow)
 
 **Valid in:** Both input and output transformers
 
@@ -2452,7 +3048,7 @@ Stop entire workflow execution immediately and return content as final result.
 **Parameters:**
 - `content` (String, required): Final content to return (validated non-nil)
 
-**Returns:** Control hash (processed by NodeOrchestrator)
+**Returns:** Control hash (processed by Workflow)
 
 **Valid in:** Both input and output transformers
 
@@ -2483,7 +3079,7 @@ Skip LLM execution for current node and use provided content instead.
 **Parameters:**
 - `content` (String, required): Content to use instead of LLM execution (validated non-nil)
 
-**Returns:** Control hash (processed by NodeOrchestrator)
+**Returns:** Control hash (processed by Workflow)
 
 **Valid in:** Input transformers only