claude_swarm 1.0.5 → 1.0.7

data/docs/v2/README.md

@@ -1,8 +1,8 @@
-# SwarmSDK & SwarmCLI Documentation
+# SwarmSDK, SwarmCLI & SwarmMemory Documentation
 
-**Version 2.0**
+**Version 2.1**
 
-Welcome to the official documentation for SwarmSDK and SwarmCLI - a Ruby framework for orchestrating multiple AI agents as a collaborative team.
+Welcome to the official documentation for SwarmSDK, SwarmCLI, and SwarmMemory - a Ruby framework for orchestrating multiple AI agents as a collaborative team with persistent memory.
 
 ---
 
@@ -18,6 +18,10 @@ Welcome to the official documentation for SwarmSDK and SwarmCLI - a Ruby framewo
 - **[Getting Started with SwarmCLI](guides/quick-start-cli.md)** ⭐ START HERE
   Command-line interface: interactive REPL and automation modes
 
+### For Advanced Features
+- **[Composable Swarms Guide](guides/composable-swarms.md)** ⭐ NEW
+  Build reusable swarm components: file/YAML/inline registration, hierarchical composition, keep_context control
+
 ---
 
 ## 📖 Comprehensive Tutorial
@@ -41,21 +45,45 @@ Welcome to the official documentation for SwarmSDK and SwarmCLI - a Ruby framewo
 
 **Quick lookups and complete API reference:**
 
+### Architecture & Execution
+- **[Architecture Flow Diagram](reference/architecture-flow.md)** ⭐ NEW
+  Complete system architecture: components, dependencies, and relationships across SwarmSDK, SwarmCLI, and SwarmMemory
+
+- **[Execution Flow Diagram](reference/execution-flow.md)** ⭐ NEW
+  Runtime execution journey: what happens when you execute a prompt (21 detailed steps across 6 phases)
+
+- **[Event Payload Structures](reference/event_payload_structures.md)**
+  Complete reference for all log event types and their payloads
+
 ### Command-Line Interface
 - **[CLI Reference](reference/cli.md)**
   Complete reference for all swarm commands: `run`, `migrate`, `mcp serve`, `mcp tools`
 
 ### Ruby DSL API
 - **[Ruby DSL Reference](reference/ruby-dsl.md)**
-  Complete programmatic API: `SwarmSDK.build`, agent DSL, permissions DSL, node DSL, hooks
+  Complete programmatic API: `SwarmSDK.build`, `SwarmSDK.load`, agent DSL, permissions DSL, node DSL, hooks
 
 ### YAML Configuration
 - **[YAML Configuration Reference](reference/yaml.md)**
-  Complete YAML structure: agents, tools, permissions, hooks, MCP servers
+  Complete YAML structure: agents, tools, permissions, hooks, nodes, MCP servers
+
+### SwarmMemory
+- **[SwarmMemory Technical Details](reference/swarm_memory_technical_details.md)**
+  Deep dive: storage architecture, semantic search, FAISS indexing, adapter interface
 
 ---
 
-## 🛠️ Integration Guides
+## 🛠️ Feature Guides
+
+### Composable Swarms
+- **[Composable Swarms Guide](guides/composable-swarms.md)** ⭐ NEW
+  Build reusable swarm components that can be composed together:
+  - Three registration methods (file, YAML string, inline block)
+  - Hierarchical composition and deep nesting
+  - Context control (keep_context: true/false)
+  - Circular dependency detection
+  - Event tracking with swarm_id hierarchy
+  - Use cases: specialized teams, dynamic loading, gems, testing
 
 ### SwarmMemory
 - **[SwarmMemory Guide](guides/swarm-memory.md)** ⭐ NEW
@@ -114,6 +142,7 @@ Welcome to the official documentation for SwarmSDK and SwarmCLI - a Ruby framewo
 - **Agents**: [Getting Started](guides/getting-started.md#core-concepts) | [Tutorial Part 1](guides/complete-tutorial.md#part-1-fundamentals)
 - **Tools**: [Tutorial Part 2](guides/complete-tutorial.md#part-2-tools-and-permissions)
 - **Delegation**: [Tutorial Part 3](guides/complete-tutorial.md#part-3-agent-collaboration)
+- **Composable Swarms**: [Ruby DSL Ref](reference/ruby-dsl.md#swarms) | [YAML Ref](reference/yaml.md#swarms) ⭐ NEW
 - **Hooks**: [Tutorial Part 4](guides/complete-tutorial.md#part-4-hooks-system)
 - **Node Workflows**: [Tutorial Part 5](guides/complete-tutorial.md#part-5-node-workflows)
 
@@ -131,6 +160,7 @@ Welcome to the official documentation for SwarmSDK and SwarmCLI - a Ruby framewo
 - **Rate Limiting**: [Tutorial Part 6](guides/complete-tutorial.md#rate-limiting)
 
 ### Production
+- **Security**: [Filesystem Tools Control](reference/ruby-dsl.md#swarmsdk.configure) | [Tutorial Part 8](guides/complete-tutorial.md#security-considerations)
 - **Logging**: [Tutorial Part 7](guides/complete-tutorial.md#structured-logging)
 - **Error Handling**: [Tutorial Part 7](guides/complete-tutorial.md#error-handling-and-recovery)
 - **Testing**: [Tutorial Part 8](guides/complete-tutorial.md#testing-strategies) | [Rails Guide](guides/rails-integration.md#testing-strategies)
@@ -166,12 +196,24 @@ Welcome to the official documentation for SwarmSDK and SwarmCLI - a Ruby framewo
 **Add persistent memory to agents**
 → [SwarmMemory Guide](guides/swarm-memory.md)
 
+**Build composable/reusable swarm teams** ⭐ NEW
+→ [Composable Swarms Guide](guides/composable-swarms.md)
+
 **Build a SwarmSDK plugin**
 → [Plugin System Guide](guides/plugins.md)
 
 **Build a custom storage adapter**
 → [Memory Adapter Guide](guides/memory-adapters.md)
 
+**Secure swarms for production** ⭐ NEW
+→ [Filesystem Tools Control](reference/ruby-dsl.md#swarmsdk.configure) | [Security Considerations](guides/complete-tutorial.md#security-considerations)
+
+**Understand the system architecture**
+→ [Architecture Flow Diagram](reference/architecture-flow.md)
+
+**Understand how execution works**
+→ [Execution Flow Diagram](reference/execution-flow.md)
+
 ---
 
 ## 📊 Documentation Structure
@@ -184,20 +226,26 @@ docs/v2/
 │   ├── getting-started.md             # SDK quick start (YAML + Ruby DSL)
 │   ├── quick-start-cli.md             # CLI quick start
 │   ├── complete-tutorial.md           # 100% feature coverage tutorial
-│   ├── swarm-memory.md                # SwarmMemory guide ⭐ NEW
-│   ├── plugins.md                     # Plugin system guide ⭐ NEW
-│   ├── memory-adapters.md             # Adapter development ⭐ NEW
+│   ├── composable-swarms.md           # Composable swarms guide ⭐ NEW
+│   ├── swarm-memory.md                # SwarmMemory guide ⭐
+│   ├── plugins.md                     # Plugin system guide ⭐
+│   ├── memory-adapters.md             # Adapter development ⭐
 │   ├── rails-integration.md           # Rails integration guide
-│   └── claude-code-agents.md          # Claude Code compatibility
+│   ├── claude-code-agents.md          # Claude Code compatibility
+│   └── MEMORY_DEFRAG_GUIDE.md         # Memory defragmentation guide
 │
 ├── reference/                          # Complete API references
+│   ├── architecture-flow.md            # System architecture diagram ⭐ NEW
+│   ├── execution-flow.md               # Runtime execution flow ⭐ NEW
+│   ├── event_payload_structures.md     # Log event payloads
+│   ├── swarm_memory_technical_details.md  # SwarmMemory deep dive
 │   ├── cli.md                          # CLI command reference
 │   ├── ruby-dsl.md                     # Ruby DSL API reference
 │   └── yaml.md                         # YAML configuration reference
 │
 └── CHANGELOG.swarm_sdk.md              # SwarmSDK version history
     CHANGELOG.swarm_cli.md              # SwarmCLI version history
-    CHANGELOG.swarm_memory.md           # SwarmMemory version history ⭐ NEW
+    CHANGELOG.swarm_memory.md           # SwarmMemory version history ⭐
 ```
 
 ---
@@ -211,13 +259,22 @@ docs/v2/
 
 ### Path 2: Intermediate → Advanced
 1. [Complete Tutorial Parts 4-6](guides/complete-tutorial.md) - Hooks, workflows, advanced config
-2. [Rails Integration](guides/rails-integration.md) - Production integration patterns
-3. [Complete Tutorial Parts 7-8](guides/complete-tutorial.md) - Production features and best practices
+2. [Composable Swarms Guide](guides/composable-swarms.md) ⭐ NEW - Build reusable swarm teams
+3. [Rails Integration](guides/rails-integration.md) - Production integration patterns
+4. [Complete Tutorial Parts 7-8](guides/complete-tutorial.md) - Production features and best practices
+
+### Path 3: SwarmMemory Deep Dive
+1. [SwarmMemory Guide](guides/swarm-memory.md) - Installation, memory tools, usage patterns
+2. [Memory Defragmentation](guides/MEMORY_DEFRAG_GUIDE.md) - Relationship discovery and knowledge graphs
+3. [SwarmMemory Technical Details](reference/swarm_memory_technical_details.md) - Architecture and internals
+4. [Memory Adapter Development](guides/memory-adapters.md) - Build custom storage backends
 
-### Path 3: Reference & API
+### Path 4: Reference & API
 1. [Ruby DSL Reference](reference/ruby-dsl.md) - Complete programmatic API
 2. [YAML Reference](reference/yaml.md) - Complete configuration format
 3. [CLI Reference](reference/cli.md) - All command-line options
+4. [Architecture Flow](reference/architecture-flow.md) - System architecture diagram
+5. [Execution Flow](reference/execution-flow.md) - Runtime execution journey
 
 ---
 
@@ -227,7 +284,8 @@ docs/v2/
 A Ruby framework for orchestrating multiple AI agents that work together as a team. Each agent has:
 - **Role**: Specialized expertise (backend developer, code reviewer, etc.)
 - **Tools**: Capabilities (Read files, Write files, Run bash commands, etc.)
-- **Delegation**: Ability to delegate subtasks to other agents
+- **Delegation**: Ability to delegate subtasks to other agents or swarms
+- **Composable Swarms**: Build hierarchies by delegating to reusable swarm components ⭐ NEW
 - **Hooks**: Custom logic that runs at key points in execution
 
 ### SwarmCLI
@@ -235,6 +293,14 @@ A command-line interface for running SwarmSDK swarms with two modes:
 - **Interactive (REPL)**: Conversational interface for exploration and iteration
 - **Non-Interactive**: One-shot execution perfect for automation and scripting
 
+### SwarmMemory
+A persistent memory system for agents with semantic search capabilities:
+- **Storage**: Hierarchical knowledge organization (concept, fact, skill, experience)
+- **Semantic Search**: FAISS-based vector similarity with local ONNX embeddings
+- **Memory Tools**: 9 tools for writing, reading, editing, and searching knowledge
+- **LoadSkill**: Dynamic tool swapping based on semantic skill discovery
+- **Plugin Architecture**: Integrates seamlessly via SwarmSDK plugin system
+
 ### Configuration Formats
 - **YAML**: Declarative, easy to read, great for shell-based hooks
 - **Ruby DSL**: Programmatic, dynamic, full Ruby power, IDE support
@@ -247,14 +313,20 @@ A command-line interface for running SwarmSDK swarms with two modes:
 |-------|-------|-----------|
 | **Installation** | [SDK Guide](guides/getting-started.md#installation) | - |
 | **First Swarm** | [SDK Guide](guides/getting-started.md#your-first-swarm) | - |
+| **Architecture** | - | [Architecture Flow](reference/architecture-flow.md) |
+| **Execution Flow** | - | [Execution Flow](reference/execution-flow.md) |
 | **CLI Commands** | [CLI Guide](guides/quick-start-cli.md#commands-overview) | [CLI Ref](reference/cli.md) |
 | **REPL Mode** | [CLI Guide](guides/quick-start-cli.md#interactive-mode-repl) | [CLI Ref](reference/cli.md#interactive-mode) |
 | **Tools** | [Tutorial Part 2](guides/complete-tutorial.md#part-2-tools-and-permissions) | [YAML Ref](reference/yaml.md#tools) |
 | **Permissions** | [Tutorial Part 2](guides/complete-tutorial.md#permissions-system) | [YAML Ref](reference/yaml.md#permissions-configuration) |
 | **Delegation** | [Tutorial Part 3](guides/complete-tutorial.md#part-3-agent-collaboration) | [Ruby DSL Ref](reference/ruby-dsl.md#delegates_to) |
+| **Composable Swarms** ⭐ | [Composable Swarms Guide](guides/composable-swarms.md) | [Ruby DSL Ref](reference/ruby-dsl.md#swarms) · [YAML Ref](reference/yaml.md#swarms) |
 | **Hooks** | [Tutorial Part 4](guides/complete-tutorial.md#part-4-hooks-system) | [YAML Ref](reference/yaml.md#hooks-configuration) |
-| **Workflows** | [Tutorial Part 5](guides/complete-tutorial.md#part-5-node-workflows) | [Ruby DSL Ref](reference/ruby-dsl.md#node-builder-dsl) |
+| **Workflows** | [Tutorial Part 5](guides/complete-tutorial.md#part-5-node-workflows) | [YAML Ref](reference/yaml.md#nodes) · [Ruby DSL Ref](reference/ruby-dsl.md#node-builder-dsl) |
 | **MCP Servers** | [Tutorial Part 6](guides/complete-tutorial.md#mcp-server-integration) | [YAML Ref](reference/yaml.md#mcp_servers) |
+| **Memory** | [SwarmMemory Guide](guides/swarm-memory.md) | [Technical Details](reference/swarm_memory_technical_details.md) |
+| **Memory Adapters** | [Adapter Guide](guides/memory-adapters.md) | [Technical Details](reference/swarm_memory_technical_details.md) |
+| **Plugins** | [Plugin Guide](guides/plugins.md) | - |
 | **Rails** | [Rails Guide](guides/rails-integration.md) | - |
 | **Testing** | [Tutorial Part 8](guides/complete-tutorial.md#testing-strategies) | - |
 
@@ -285,18 +357,6 @@ Found an issue or want to improve the documentation?
 
 ---
 
-## 📄 Version History
-
-### v2.0 (January 2025)
-- Complete documentation rewrite
-- Consolidated from 261 files to 7 focused documents
-- 100% feature coverage
-- Added Rails integration guide
-- Added comprehensive tutorial
-- Complete CLI, Ruby DSL, and YAML references
-
----
-
 ## 📚 Additional Resources
 
 - **GitHub Repository**: [parruda/claude-swarm](https://github.com/parruda/claude-swarm)

data/docs/v2/guides/complete-tutorial.md

@@ -395,7 +395,7 @@ agent :planner do
   description "Task planner"
   model "gpt-4"
   system_prompt "Plan and track work"
-  # TodoWrite included by default
+  tools :Read, :Grep, :Glob, :TodoWrite  # Add TodoWrite explicitly
 end
 ```
 
@@ -538,7 +538,7 @@ agent :problem_solver do
   description "Problem solver"
   model "gpt-4"
   system_prompt "Use the Think tool frequently to reason through problems"
-  # Think included by default
+  tools :Read, :Grep, :Glob, :Think  # Add Think explicitly
 end
 ```
 
@@ -585,7 +585,7 @@ agent :researcher do
   description "Web researcher"
   model "gpt-4"
   system_prompt "Research information from web sources"
-  # WebFetch included by default
+  tools :Read, :Grep, :Glob, :WebFetch  # Add WebFetch explicitly
 end
 ```
 
@@ -631,8 +631,7 @@ WebFetch(url: "https://example.com/api-docs", prompt: "List all available API en
 agent :agent_name do
   description "..."
   model "gpt-4"
-  disable_default_tools [:Think, :WebFetch]  # Disable specific tools
-  # Or disable multiple: [:Think, :TodoWrite, :Grep]
+  disable_default_tools [:Read, :Grep]  # Disable specific tools
 end
 ```
 
@@ -678,8 +677,8 @@ end
 agent :selective_agent do
   description "Selective agent"
   model "gpt-4"
-  disable_default_tools [:Think, :TodoWrite]  # Disable these
-  # Still has: Read, Grep, Glob, Scratchpad tools
+  disable_default_tools [:Read, :Grep]  # Disable these
+  # Still has: Glob, Scratchpad tools
 end
 ```
 
@@ -1084,6 +1083,67 @@ Lead    ──┼──→ Security
 - **Hub and Spoke**: Parallel tasks coordinated by lead
 - **Specialist Pool**: Lead delegates to any specialist as needed
 
+#### Delegation Isolation Modes
+
+When multiple agents delegate to the same target agent, you can control whether they share conversation history or get isolated instances.
+
+**Problem this solves**: Without isolation, multiple agents delegating to the same target share conversation history, causing context mixing. For example, if both `frontend` and `backend` delegate to `tester`, the tester's conversation includes both delegations mixed together, making it confusing.
+
+**Solution: Isolated Instances (Default)**
+
+By default, each delegator gets its own isolated instance:
+
+```yaml
+agents:
+  tester:
+    description: "Testing agent"
+    # shared_across_delegations: false (default - isolated mode)
+
+  frontend:
+    description: "Frontend developer"
+    delegates_to: [tester]  # Gets tester@frontend
+
+  backend:
+    description: "Backend developer"
+    delegates_to: [tester]  # Gets tester@backend (separate instance)
+```
+
+**Result**:
+- `frontend` → `tester@frontend` (isolated conversation)
+- `backend` → `tester@backend` (separate isolated conversation)
+- No context mixing between frontend and backend's testing conversations
+
+**When to Use Shared Mode**
+
+For agents that benefit from seeing all delegation contexts:
+
+```yaml
+agents:
+  database:
+    description: "Database coordination agent"
+    shared_across_delegations: true  # Shared mode
+
+  frontend:
+    delegates_to: [database]
+
+  backend:
+    delegates_to: [database]
+```
+
+Both frontend and backend share the same database agent instance and conversation history.
+
+**Use shared mode for:**
+- Stateful coordination agents
+- Database agents maintaining transaction state
+- Agents that benefit from seeing all contexts
+
+**Memory Sharing (Always Enabled)**
+
+Regardless of isolation mode, **plugin storage is always shared** by base agent name:
+- `tester@frontend` and `tester@backend` share the same SwarmMemory storage
+- This allows delegation instances to share knowledge while maintaining separate conversations
+- Best of both worlds: isolated conversations + shared memory
+
 ### 3.5 Markdown Agent Files
 
 Define agents in separate markdown files for better organization:
@@ -1891,29 +1951,25 @@ end
 
 **Control Flow Methods**:
 
-NodeContext provides three methods for dynamic workflow control:
+NodeContext provides three methods for dynamic workflow control. Input and output blocks are automatically converted to lambdas, which means you can use `return` statements for clean early exits.
 
 **1. goto_node - Jump to any node**:
 ```ruby
 output do |ctx|
-  if needs_revision?(ctx.content)
-    # Jump back to revision node instead of continuing forward
-    ctx.goto_node(:revision, content: ctx.content)
-  else
-    ctx.content  # Continue to next node normally
-  end
+  # Using return for early exit (clean and natural)
+  return ctx.goto_node(:revision, content: ctx.content) if needs_revision?(ctx.content)
+
+  ctx.content  # Continue to next node normally
 end
 ```
 
 **2. halt_workflow - Stop entire workflow**:
 ```ruby
 output do |ctx|
-  if converged?(ctx.content)
-    # Stop workflow early with final result
-    ctx.halt_workflow(content: ctx.content)
-  else
-    ctx.content  # Continue to next node
-  end
+  # Using return for early exit
+  return ctx.halt_workflow(content: ctx.content) if converged?(ctx.content)
+
+  ctx.content  # Continue to next node
 end
 ```
 
@@ -1921,27 +1977,26 @@ end
 ```ruby
 input do |ctx|
   cached = check_cache(ctx.content)
-  if cached
-    # Skip expensive LLM call and use cached result
-    ctx.skip_execution(content: cached)
-  else
-    ctx.content  # Execute node normally
-  end
+
+  # Using return for early exit when cached
+  return ctx.skip_execution(content: cached) if cached
+
+  ctx.content  # Execute node normally
 end
 ```
 
+**Why `return` works safely**: Input and output blocks are automatically converted to lambdas, where `return` only exits the transformer block, not your entire program. This enables natural Ruby control flow patterns.
+
 **Creating loops with goto_node**:
 ```ruby
 node :reasoning do
   agent(:thinker, reset_context: false)  # Preserve context across iterations
 
   output do |ctx|
-    # Loop until convergence
-    if ctx.all_results.size > 10
-      ctx.halt_workflow(content: "Max iterations reached")
-    else
-      ctx.goto_node(:reflection, content: ctx.content)
-    end
+    # Using return for early exit when max iterations reached
+    return ctx.halt_workflow(content: "Max iterations reached") if ctx.all_results.size > 10
+
+    ctx.goto_node(:reflection, content: ctx.content)
   end
 end
 
@@ -1961,11 +2016,10 @@ start_node :reasoning
 
 ```ruby
 output do |ctx|
-  if ctx.error
-    ctx.halt_workflow(content: "Error: #{ctx.error.message}")
-  else
-    ctx.goto_node(:next_node, content: ctx.content)
-  end
+  # Using return for early exit on error
+  return ctx.halt_workflow(content: "Error: #{ctx.error.message}") if ctx.error
+
+  ctx.goto_node(:next_node, content: ctx.content)
 end
 ```
 
@@ -2635,7 +2689,7 @@ end
 SwarmSDK validates configurations and emits warnings:
 
 ```ruby
-swarm = SwarmSDK::Swarm.load("config.yml")
+swarm = SwarmSDK.load_file("config.yml")
 
 # Check for warnings
 warnings = swarm.validate
@@ -2835,7 +2889,7 @@ end
 **Integration test workflows**:
 ```ruby
 RSpec.describe "Development Pipeline" do
-  let(:swarm) { SwarmSDK::Swarm.load("swarm.yml") }
+  let(:swarm) { SwarmSDK.load_file("swarm.yml") }
 
   it "completes full workflow" do
     result = swarm.execute("Build auth system")
@@ -3003,6 +3057,48 @@ Staging: Restricted access, validated outputs
 Production: Minimal access, extensive hooks
 ```
 
+**6. Disable filesystem tools globally (system-wide security)**:
+```ruby
+# For multi-tenant platforms or sandboxed environments
+SwarmSDK.configure do |config|
+  config.allow_filesystem_tools = false
+end
+
+# Or via environment variable (recommended for production)
+ENV['SWARM_SDK_ALLOW_FILESYSTEM_TOOLS'] = 'false'
+
+# Agents can now only use non-filesystem tools
+swarm = SwarmSDK.build do
+  name "API Analyst"
+  lead :analyst
+
+  agent :analyst do
+    description "Analyzes data via APIs only"
+    model "gpt-5"
+    # These work: Think, WebFetch, Clock, TodoWrite, Scratchpad*, Memory*
+    tools :Think, :WebFetch
+    # These are blocked: Read, Write, Edit, MultiEdit, Grep, Glob, Bash
+  end
+end
+
+# Override per-swarm if needed
+restricted_swarm = SwarmSDK.build(allow_filesystem_tools: false) do
+  # ... specific swarm that needs extra restriction
+end
+```
+
+**Use cases:**
+- **Multi-tenant platforms**: Prevent user-provided swarms from accessing filesystem
+- **Containerized deployments**: Read-only filesystems or restricted environments
+- **Compliance requirements**: Data analysis workloads that forbid file operations
+- **CI/CD pipelines**: Agents should only interact via APIs
+
+**Key features:**
+- **Security boundary**: Cannot be overridden by swarm YAML/DSL configuration
+- **Validation**: Errors caught at build time with clear messages
+- **Priority**: Explicit parameter > Global setting > Environment variable > Default (true)
+- **Non-breaking**: Defaults to `true` for backward compatibility
+
 ### 8.6 Cost Management
 
 **Track costs in real-time**: