claude_swarm 0.1.11 → 0.1.13

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 6d943c577c58e38133c56a9852a175bb4167916784c2fcda3487740f70c9ff4e
-  data.tar.gz: 8aaebece866fd8f73f12a86b300d85ffb0a1ac3b160e3c3d70e4a3df48ba1594
+  metadata.gz: 57e09297e72fe54ef127b5f7b5789c9e9d825279404ffa96c29f2ff0e7fbc622
+  data.tar.gz: 6e1d719d97d2f8fefa4ec8d0e9396e5fba867012b94d88d915247aea18690523
 SHA512:
-  metadata.gz: 20cc61370c608454ec588cf5771290cb7e9c563ef6bbc13c3d02009d33292445d6da3b069a7dc4cc5c69ce916183b49adfbd86cbaf2df41185912fa7e73aab87
-  data.tar.gz: 2b2f63db094d495da5b76bede0b2c49e933a6a5adfd4b9fb9967753e51011302291ad0c47d721c120113b0d182b501650ffd4cd1eacd47fa8a5c21db08249475
+  metadata.gz: 8ad9a411c8bc503829927ea534fed74e9f525dd62d68fc6eb9978667afdb40435b9bac0c13320c097f1f41556a8d50215c723c8066dec7366cf7d847d07a7e9e
+  data.tar.gz: 490824249ac673b7904f05a084d2ff0190a1034529a9302f423d00e965cd7c793b519b1d8e318a36a1e58af80706cb1fda80dd507af525b45f7df26299abd3f9

data/CHANGELOG.md

@@ -1,3 +1,50 @@
+## [Unreleased]
+
+### Added
+- **Session restoration support (Experimental)**: Session management with the ability to resume previous Claude Swarm sessions. Note: This is an experimental feature with limitations - the main instance's conversation context is not fully restored
+  - New `--session-id` flag to resume a session by ID or path
+  - New `list-sessions` command to view available sessions with metadata
+  - Automatic capture and persistence of Claude session IDs for all instances
+  - Individual instance states stored in `state/` directory with instance ID as filename (e.g., `state/lead_abc123.json`)
+  - Swarm configuration copied to session directory as `config.yml` for restoration
+- **Instance ID tracking**: Each instance now gets a unique ID in the format `instance_name_<hex>` for better identification in logs
+- **Enhanced logging with instance IDs**: All log messages now include instance IDs when available (e.g., `lead (lead_1234abcd) -> backend (backend_5678efgh)`)
+- **Calling instance ID propagation**: When one instance calls another, both the calling instance name and ID are passed for complete tracking
+- Instance IDs are stored in MCP configuration files with `instance_id` and `instance_name` fields
+- New CLI options: `--instance-id` and `--calling-instance-id` for the `mcp-serve` command
+- ClaudeCodeExecutor now tracks and logs both instance and calling instance IDs
+- **Process tracking and cleanup**: Added automatic tracking and cleanup of child MCP server processes
+  - New `ProcessTracker` class creates individual PID files in a `pids/` directory within the session path
+  - Signal handlers (INT, TERM, QUIT) ensure all child processes are terminated when the main instance exits
+  - Prevents orphaned MCP server processes from continuing to run after swarm termination
+
+### Changed
+- Human-readable logs improved to show instance IDs in parentheses after instance names for easier tracking of multi-instance interactions
+- `log_request` method enhanced to include instance IDs in structured JSON logs
+- Configuration class now accepts optional `base_dir` parameter to support session restoration from different directories
+
+### Fixed
+- Fixed issue where child MCP server processes would continue running after the main instance exits
+
+## [0.1.12]
+### Added
+- **Circular dependency detection**: Configuration validation now detects and reports circular dependencies between instances
+- Clear error messages showing the dependency cycle (e.g., "Circular dependency detected: lead -> backend -> lead")
+- Comprehensive test coverage for various circular dependency scenarios
+- **Session management improvements**: Session files are now stored in `~/.claude-swarm/sessions/` organized by project path
+- Added `SessionPath` module to centralize session path management
+- Sessions are now organized by project directory for better multi-project support
+- Added `CLAUDE_SWARM_HOME` environment variable support for custom storage location
+- Log full JSON to `session.log.json` as JSONL
+
+### Changed
+- Session files moved from `./.claude-swarm/sessions/` to `~/.claude-swarm/sessions/[project]/[timestamp]/`
+- Replaced `CLAUDE_SWARM_SESSION_TIMESTAMP` with `CLAUDE_SWARM_SESSION_PATH` environment variable
+- MCP server configurations now use the new centralized session path
+
+### Fixed
+- Fixed circular dependency example in README documentation
+
 ## [0.1.11]
 ### Added
 - Main instance debug mode with `claude-swarm --debug`

data/README.md

@@ -1,6 +1,6 @@
 # Claude Swarm
 
-[![Gem Version](https://badge.fury.io/rb/claude_swarm.svg?cache_bust=0)](https://badge.fury.io/rb/claude_swarm)
+[![Gem Version](https://badge.fury.io/rb/claude_swarm.svg?cache_bust=0.1.12)](https://badge.fury.io/rb/claude_swarm)
 [![CI](https://github.com/parruda/claude-swarm/actions/workflows/ci.yml/badge.svg)](https://github.com/parruda/claude-swarm/actions/workflows/ci.yml)
 
 Claude Swarm orchestrates multiple Claude Code instances as a collaborative AI development team. It enables running AI agents with specialized roles, tools, and directory contexts, communicating via MCP (Model Context Protocol) in a tree-like hierarchy. Define your swarm topology in simple YAML and let Claude instances delegate tasks through connected instances. Perfect for complex projects requiring specialized AI agents for frontend, backend, testing, DevOps, or research tasks.
@@ -80,7 +80,7 @@ claude-swarm --vibe # That will allow ALL tools for all instances! Be Careful!
 This will:
 - Launch the main instance (lead) with connections to other instances
 - The lead instance can communicate with the other instances via MCP
-- All session files are stored in `.claude-swarm/sessions/{timestamp}/`
+- All session files are stored in `~/.claude-swarm/sessions/{project}/{timestamp}/` (customizable via `CLAUDE_SWARM_HOME`)
 
 #### Multi-Level Swarm Example
 
@@ -327,7 +327,7 @@ swarm:
       description: "Backend developer building APIs and services"
       directory: ./backend
       model: opus
-      connections: [architect, database]
+      connections: [database]
       allowed_tools:
         - Edit
         - Write
@@ -446,6 +446,14 @@ claude-swarm --vibe
 claude-swarm -p "Implement the new user authentication feature"
 claude-swarm --prompt "Fix the bug in the payment module"
 
+# Resume a previous session by ID
+claude-swarm --session-id 20241206_143022
+claude-swarm --session-id ~/path/to/session
+
+# List available sessions
+claude-swarm list-sessions
+claude-swarm list-sessions --limit 20
+
 # Show version
 claude-swarm version
 
@@ -457,6 +465,70 @@ claude-swarm tools-mcp --allowed-tools 'Read,Edit' --disallowed-tools 'Edit(*.lo
 claude-swarm mcp-serve INSTANCE_NAME --config CONFIG_FILE --session-timestamp TIMESTAMP
 ```
 
+### Session Management and Restoration (Experimental)
+
+Claude Swarm provides experimental session management with restoration capabilities. **Note: This feature is experimental and has limitations - the main instance's conversation context is not fully restored.**
+
+#### Session Structure
+All session files are organized in `~/.claude-swarm/sessions/{project}/{timestamp}/`:
+- `config.yml`: Copy of the original swarm configuration
+- `state/`: Directory containing individual instance states
+  - `{instance_id}.json`: Claude session ID and status for each instance (e.g., `lead_abc123.json`)
+- `{instance_name}.mcp.json`: MCP configuration files
+- `session.log`: Human-readable request/response tracking
+- `session.log.json`: All events in JSONL format (one JSON per line)
+- `permissions.log`: Permission checks and decisions
+
+#### Listing Sessions
+View your previous Claude Swarm sessions:
+
+```bash
+# List recent sessions (default: 10)
+claude-swarm list-sessions
+
+# List more sessions
+claude-swarm list-sessions --limit 20
+```
+
+Output shows:
+- Session ID (timestamp)
+- Creation time
+- Main instance name
+- Number of instances
+- Configuration file used
+- Full session path
+
+#### Resuming Sessions
+Resume a previous session with all instances restored to their Claude session states:
+
+```bash
+# Resume by session ID
+claude-swarm --session-id 20241206_143022
+
+# Resume by full path
+claude-swarm --session-id ~/.claude-swarm/sessions/my-project/20241206_143022
+```
+
+This will:
+1. Load the session manifest and instance states
+2. Restore the original swarm configuration
+3. Resume the main instance with its Claude session ID
+4. Restore all connected instances with their session IDs
+5. Maintain the same working directories and tool permissions
+
+#### How Session Restoration Works
+- Each instance's Claude session ID is automatically captured and persisted
+- Instance states are stored in separate files named by instance ID to prevent concurrency issues
+- MCP configurations are regenerated with the saved session IDs
+- The main instance uses Claude's `--resume` flag (limited effectiveness)
+- Connected instances receive their session IDs via `--claude-session-id`
+
+**Important Limitations:**
+- The main instance's conversation history and context are not fully restored
+- Only the session ID is preserved, not the actual conversation state
+- Connected instances restore more reliably than the main instance
+- This is an experimental feature and may not work as expected
+
 ## How It Works
 
 1. **Configuration Parsing**: Claude Swarm reads your YAML configuration and validates it
@@ -471,20 +543,17 @@ claude-swarm mcp-serve INSTANCE_NAME --config CONFIG_FILE --session-timestamp TI
    - Eliminates the need to manually accept each tool or use global `--vibe` mode
    - Per-instance `vibe: true` skips all permission checks for that specific instance
    - The permission MCP uses `--permission-prompt-tool` to check tool access
-   - Permission decisions are logged to `.claude-swarm/sessions/{timestamp}/permissions.log`
-4. **Session Management**: Claude Swarm maintains session continuity:
-   - Generates a shared session timestamp for all instances
-   - Each instance can maintain its own Claude session ID
-   - Sessions can be reset via the MCP server interface
+   - Permission decisions are logged to `~/.claude-swarm/sessions/{project}/{timestamp}/permissions.log`
+4. **Session Persistence**: Claude Swarm automatically tracks session state:
+   - Generates a shared session path for all instances
+   - Each instance's Claude session ID is captured and saved
+   - Instance states are stored using instance IDs as filenames to avoid conflicts
+   - Sessions can be fully restored with all instances reconnected
 5. **Main Instance Launch**: The main instance is launched with its MCP configuration, giving it access to all connected instances
 6. **Inter-Instance Communication**: Connected instances expose themselves as MCP servers with these tools:
    - **task**: Execute tasks using Claude Code with configurable tools and return results. The tool description includes the instance name and description (e.g., "Execute a task using Agent frontend_dev. Frontend developer specializing in React and TypeScript")
    - **session_info**: Get current Claude session information including ID and working directory
    - **reset_session**: Reset the Claude session for a fresh start
-7. **Session Management**: All session files are organized in `.claude-swarm/sessions/{timestamp}/`:
-   - MCP configuration files: `{instance_name}.mcp.json`
-   - Session log: `session.log` with detailed request/response tracking
-   - Permission log: `permissions.log` with all permission checks and decisions
 
 ## Troubleshooting
 
@@ -507,14 +576,15 @@ claude-swarm mcp-serve INSTANCE_NAME --config CONFIG_FILE --session-timestamp TI
 ### Debug Output
 
 The swarm will display:
-- Session directory location (`.claude-swarm/sessions/{timestamp}/`)
+- Session directory location (`~/.claude-swarm/sessions/{project}/{timestamp}/`)
 - Main instance details (model, directory, tools, connections)
 - The exact command being run
 
 ### Session Files
 
-Check the session directory `.claude-swarm/sessions/{timestamp}/` for:
-- `session.log`: Detailed logs with request/response tracking
+Check the session directory `~/.claude-swarm/sessions/{project}/{timestamp}/` for:
+- `session.log`: Human-readable logs with request/response tracking
+- `session.log.json`: All events in JSONL format (one JSON object per line)
 - `{instance}.mcp.json`: MCP configuration for each instance
 - All files for a session are kept together for easy review
 

data/claude-swarm.yml

@@ -0,0 +1,42 @@
+version: 1
+swarm:
+  name: "Swarm Name"
+  main: lead_developer
+  instances:
+    lead_developer:
+      description: "Lead developer who coordinates the team and makes architectural decisions"
+      directory: .
+      model: sonnet
+      prompt: "You are the lead developer coordinating the team"
+      allowed_tools: [Read, Edit, Bash, Write]
+      connections: [frontend_dev, backend_dev]
+
+    # Example instances (uncomment and modify as needed):
+
+    frontend_dev:
+      description: "Frontend developer specializing in React and modern web technologies"
+      directory: .
+      model: sonnet
+      prompt: "You specialize in frontend development with React, TypeScript, and modern web technologies"
+      allowed_tools: [Read, Edit, Write, "Bash(npm:*)", "Bash(yarn:*)", "Bash(pnpm:*)"]
+
+    backend_dev:
+      description: "Backend developer focusing on APIs, databases, and server architecture"
+      directory: .
+      model: sonnet
+      prompt: "You specialize in backend development, APIs, databases, and server architecture"
+      allowed_tools: [Read, Edit, Write, Bash]
+
+    # devops_engineer:
+    #   description: "DevOps engineer managing infrastructure, CI/CD, and deployments"
+    #   directory: .
+    #   model: sonnet
+    #   prompt: "You specialize in infrastructure, CI/CD, containerization, and deployment"
+    #   allowed_tools: [Read, Edit, Write, "Bash(docker:*)", "Bash(kubectl:*)", "Bash(terraform:*)"]
+
+    # qa_engineer:
+    #   description: "QA engineer ensuring quality through comprehensive testing"
+    #   directory: ./tests
+    #   model: sonnet
+    #   prompt: "You specialize in testing, quality assurance, and test automation"
+    #   allowed_tools: [Read, Edit, Write, Bash]

data/example/session-restoration-demo.yml

@@ -0,0 +1,19 @@
+version: 1
+swarm:
+  name: "Session Restoration Demo"
+  main: coordinator
+  instances:
+    coordinator:
+      description: "Main coordinator that manages the team"
+      directory: .
+      model: sonnet
+      prompt: "You are the team coordinator. You help manage tasks and coordinate between team members."
+      allowed_tools: [Read, Edit, Bash]
+      connections: [developer]
+    
+    developer:
+      description: "Developer who writes code"
+      directory: .
+      model: sonnet
+      prompt: "You are a skilled developer who writes clean, well-tested code."
+      allowed_tools: [Read, Edit, Write, Bash]

data/lib/claude_swarm/claude_code_executor.rb

@@ -4,23 +4,25 @@ require "json"
 require "open3"
 require "logger"
 require "fileutils"
+require_relative "session_path"
 
 module ClaudeSwarm
   class ClaudeCodeExecutor
-    SWARM_DIR = ".claude-swarm"
-    SESSIONS_DIR = "sessions"
+    attr_reader :session_id, :last_response, :working_directory, :logger, :session_path
 
-    attr_reader :session_id, :last_response, :working_directory, :logger, :session_timestamp
-
-    def initialize(working_directory: Dir.pwd, model: nil, mcp_config: nil, vibe: false, instance_name: nil, calling_instance: nil)
+    def initialize(working_directory: Dir.pwd, model: nil, mcp_config: nil, vibe: false,
+                   instance_name: nil, instance_id: nil, calling_instance: nil, calling_instance_id: nil,
+                   claude_session_id: nil)
       @working_directory = working_directory
       @model = model
       @mcp_config = mcp_config
       @vibe = vibe
-      @session_id = nil
+      @session_id = claude_session_id
       @last_response = nil
       @instance_name = instance_name
+      @instance_id = instance_id
       @calling_instance = calling_instance
+      @calling_instance_id = calling_instance_id
 
       # Setup logging
       setup_logging
@@ -53,7 +55,10 @@ module ClaudeSwarm
           log_streaming_event(json_data)
 
           # Capture session_id from system init
-          @session_id = json_data["session_id"] if json_data["type"] == "system" && json_data["subtype"] == "init"
+          if json_data["type"] == "system" && json_data["subtype"] == "init"
+            @session_id = json_data["session_id"]
+            write_instance_state
+          end
 
           # Capture the final result
           result_response = json_data if json_data["type"] == "result"
@@ -94,18 +99,35 @@ module ClaudeSwarm
 
     private
 
-    def setup_logging
-      # Use environment variable for session timestamp if available (set by orchestrator)
-      # Otherwise create a new timestamp
-      @session_timestamp = ENV["CLAUDE_SWARM_SESSION_TIMESTAMP"] || Time.now.strftime("%Y%m%d_%H%M%S")
+    def write_instance_state
+      return unless @instance_id && @session_id
+
+      state_dir = File.join(@session_path, "state")
+      FileUtils.mkdir_p(state_dir)
+
+      state_file = File.join(state_dir, "#{@instance_id}.json")
+      state_data = {
+        instance_name: @instance_name,
+        instance_id: @instance_id,
+        claude_session_id: @session_id,
+        status: "active",
+        updated_at: Time.now.iso8601
+      }
 
-      # Ensure the session directory exists
-      session_dir = File.join(Dir.pwd, SWARM_DIR, SESSIONS_DIR, @session_timestamp)
-      FileUtils.mkdir_p(session_dir)
+      File.write(state_file, JSON.pretty_generate(state_data))
+      @logger.info("Wrote instance state for #{@instance_name} (#{@instance_id}) with session ID: #{@session_id}")
+    rescue StandardError => e
+      @logger.error("Failed to write instance state for #{@instance_name} (#{@instance_id}): #{e.message}")
+    end
+
+    def setup_logging
+      # Use session path from environment (required)
+      @session_path = SessionPath.from_env
+      SessionPath.ensure_directory(@session_path)
 
       # Create logger with session.log filename
       log_filename = "session.log"
-      log_path = File.join(session_dir, log_filename)
+      log_path = File.join(@session_path, log_filename)
       @logger = Logger.new(log_path)
       @logger.level = Logger::INFO
 
@@ -114,20 +136,47 @@ module ClaudeSwarm
         "[#{datetime.strftime("%Y-%m-%d %H:%M:%S.%L")}] [#{severity}] #{msg}\n"
       end
 
-      @logger.info("Started Claude Code executor for instance: #{@instance_name}") if @instance_name
+      return unless @instance_name
+
+      instance_info = @instance_name
+      instance_info += " (#{@instance_id})" if @instance_id
+      @logger.info("Started Claude Code executor for instance: #{instance_info}")
     end
 
     def log_request(prompt)
-      @logger.info("#{@calling_instance} -> #{@instance_name}: \n---\n#{prompt}\n---")
+      caller_info = @calling_instance
+      caller_info += " (#{@calling_instance_id})" if @calling_instance_id
+      instance_info = @instance_name
+      instance_info += " (#{@instance_id})" if @instance_id
+      @logger.info("#{caller_info} -> #{instance_info}: \n---\n#{prompt}\n---")
+
+      # Build event hash for JSON logging
+      event = {
+        type: "request",
+        from_instance: @calling_instance,
+        from_instance_id: @calling_instance_id,
+        to_instance: @instance_name,
+        to_instance_id: @instance_id,
+        prompt: prompt,
+        timestamp: Time.now.iso8601
+      }
+
+      append_to_session_json(event)
     end
 
     def log_response(response)
+      caller_info = @calling_instance
+      caller_info += " (#{@calling_instance_id})" if @calling_instance_id
+      instance_info = @instance_name
+      instance_info += " (#{@instance_id})" if @instance_id
       @logger.info(
-        "($#{response["total_cost"]} - #{response["duration_ms"]}ms) #{@instance_name} -> #{@calling_instance}: \n---\n#{response["result"]}\n---"
+        "($#{response["total_cost"]} - #{response["duration_ms"]}ms) #{instance_info} -> #{caller_info}: \n---\n#{response["result"]}\n---"
       )
     end
 
     def log_streaming_event(event)
+      append_to_session_json(event)
+
       return log_system_message(event) if event["type"] == "system"
 
       # Add specific details based on event type
@@ -155,14 +204,18 @@ module ClaudeSwarm
         arguments = tool_call["input"].to_json
         arguments = "#{arguments[0..300]} ...}" if arguments.length > 300
 
+        instance_info = @instance_name
+        instance_info += " (#{@instance_id})" if @instance_id
         @logger.info(
-          "Tool call from #{@instance_name} -> Tool: #{tool_call["name"]}, ID: #{tool_call["id"]}, Arguments: #{arguments}"
+          "Tool call from #{instance_info} -> Tool: #{tool_call["name"]}, ID: #{tool_call["id"]}, Arguments: #{arguments}"
         )
       end
 
       text = content.select { |c| c["type"] == "text" }
       text.each do |t|
-        @logger.info("#{@instance_name} is thinking:\n---\n#{t["text"]}\n---")
+        instance_info = @instance_name
+        instance_info += " (#{@instance_id})" if @instance_id
+        @logger.info("#{instance_info} is thinking:\n---\n#{t["text"]}\n---")
       end
     end
 
@@ -170,6 +223,34 @@ module ClaudeSwarm
       @logger.debug("USER: #{JSON.pretty_generate(content)}")
     end
 
+    def append_to_session_json(event)
+      json_filename = "session.log.json"
+      json_path = File.join(@session_path, json_filename)
+
+      # Use file locking to ensure thread-safe writes
+      File.open(json_path, File::WRONLY | File::APPEND | File::CREAT) do |file|
+        file.flock(File::LOCK_EX)
+
+        # Create entry with metadata
+        entry = {
+          instance: @instance_name,
+          instance_id: @instance_id,
+          calling_instance: @calling_instance,
+          calling_instance_id: @calling_instance_id,
+          timestamp: Time.now.iso8601,
+          event: event
+        }
+
+        # Write as single line JSON (JSONL format)
+        file.puts(entry.to_json)
+
+        file.flock(File::LOCK_UN)
+      end
+    rescue StandardError => e
+      @logger.error("Failed to append to session JSON: #{e.message}")
+      raise
+    end
+
     def build_command_array(prompt, options)
       cmd_array = ["claude"]
 

data/lib/claude_swarm/claude_mcp_server.rb

@@ -6,35 +6,47 @@ require_relative "claude_code_executor"
 require_relative "task_tool"
 require_relative "session_info_tool"
 require_relative "reset_session_tool"
+require_relative "process_tracker"
 
 module ClaudeSwarm
   class ClaudeMcpServer
     # Class variables to share state with tool classes
     class << self
-      attr_accessor :executor, :instance_config, :logger, :session_timestamp, :calling_instance
+      attr_accessor :executor, :instance_config, :logger, :session_path, :calling_instance, :calling_instance_id
     end
 
-    def initialize(instance_config, calling_instance:)
+    def initialize(instance_config, calling_instance:, calling_instance_id: nil)
       @instance_config = instance_config
       @calling_instance = calling_instance
+      @calling_instance_id = calling_instance_id
       @executor = ClaudeCodeExecutor.new(
         working_directory: instance_config[:directory],
         model: instance_config[:model],
         mcp_config: instance_config[:mcp_config_path],
         vibe: instance_config[:vibe],
         instance_name: instance_config[:name],
-        calling_instance: calling_instance
+        instance_id: instance_config[:instance_id],
+        calling_instance: calling_instance,
+        calling_instance_id: calling_instance_id,
+        claude_session_id: instance_config[:claude_session_id]
       )
 
       # Set class variables so tools can access them
       self.class.executor = @executor
       self.class.instance_config = @instance_config
       self.class.logger = @executor.logger
-      self.class.session_timestamp = @executor.session_timestamp
+      self.class.session_path = @executor.session_path
       self.class.calling_instance = @calling_instance
+      self.class.calling_instance_id = @calling_instance_id
     end
 
     def start
+      # Track this process
+      if @executor.session_path && File.exist?(@executor.session_path)
+        tracker = ProcessTracker.new(@executor.session_path)
+        tracker.track_pid(Process.pid, "mcp_#{@instance_config[:name]}")
+      end
+
       server = FastMcp::Server.new(
         name: @instance_config[:name],
         version: "1.0.0"