@desplega.ai/agent-swarm 1.2.0 → 1.9.0

package/thoughts/shared/research/2025-12-21-gemini-cli-integration.md

@@ -0,0 +1,376 @@
+---
+date: 2025-12-21T21:10:00-08:00
+researcher: master lord
+git_commit: fb757dbc76fc9f3c6f5859d4a96c28af4dc02af9
+branch: main
+repository: agent-swarm
+topic: "Gemini CLI Integration Research"
+tags: [research, codebase, gemini-cli, multi-cli, integration]
+status: complete
+last_updated: 2025-12-21
+last_updated_by: master lord
+---
+
+# Research: Gemini CLI Integration
+
+**Date**: 2025-12-21T21:10:00-08:00
+**Researcher**: master lord
+**Git Commit**: fb757dbc76fc9f3c6f5859d4a96c28af4dc02af9
+**Branch**: main
+**Repository**: agent-swarm
+
+## Research Question
+
+How can we integrate the agent-swarm codebase with other CLI tools, particularly Gemini CLI, to support Gemini 3 and other models alongside Claude?
+
+## Summary
+
+The agent-swarm codebase has a clean separation between **CLI-agnostic** components (MCP server, tools, database) and **Claude-specific** components (CLI spawning, hooks, setup). Integration with Gemini CLI is feasible because:
+
+1. **MCP Server works as-is** - Gemini CLI fully supports MCP servers via the same protocol
+2. **Headless mode is similar** - Both CLIs support `--output-format stream-json` and prompt flags
+3. **Hooks exist in both** - Different event names but similar JSON stdin/stdout protocol
+4. **Authentication is straightforward** - Gemini uses API keys or service accounts, easy for containers
+
+The main work involves creating an abstraction layer in the runner to support multiple CLI tools.
+
+## Detailed Findings
+
+### Current Architecture: CLI-Specific vs CLI-Agnostic Components
+
+| Component | File | CLI Dependency | Notes |
+|-----------|------|----------------|-------|
+| MCP Server | `src/server.ts` | **CLI-agnostic** | Standard MCP SDK, works with any client |
+| MCP Tools | `src/tools/*.ts` | **CLI-agnostic** | Pure MCP protocol |
+| HTTP Transport | `src/http.ts` | **CLI-agnostic** | Standard MCP HTTP transport |
+| STDIO Transport | `src/stdio.ts` | **CLI-agnostic** | Standard MCP STDIO transport |
+| Database Layer | `src/be/db.ts` | **CLI-agnostic** | SQLite operations |
+| Type Definitions | `src/types.ts` | **CLI-agnostic** | Data structures |
+| CLI Spawner | `src/claude.ts` | **Claude-specific** | Hardcoded `claude` command |
+| Agent Runner | `src/commands/runner.ts` | **Claude-specific** | Claude CLI flags |
+| Hook Handler | `src/hooks/hook.ts` | **Claude-specific** | Claude Code hook protocol |
+| Setup Command | `src/commands/setup.tsx` | **Claude-specific** | Creates `.claude/` config |
+| Docker Files | `Dockerfile.worker` | **Claude-specific** | Installs Claude CLI |
+
+### Claude CLI Integration Points
+
+**1. CLI Spawning (`src/claude.ts:9-80`)**
+```typescript
+const CMD = ["claude"];  // Hardcoded CLI name
+if (opts.headless) {
+  CMD.push("--verbose");
+  CMD.push("--output-format");
+  CMD.push("stream-json");
+  CMD.push("-p");
+  CMD.push(opts.msg);
+}
+```
+
+**2. Agent Runner (`src/commands/runner.ts:65-92`)**
+```typescript
+const CMD = [
+  "claude",
+  "--verbose",
+  "--output-format", "stream-json",
+  "--dangerously-skip-permissions",
+  "--allow-dangerously-skip-permissions",
+  "--permission-mode", "bypassPermissions",
+  "-p", opts.prompt,
+];
+if (opts.systemPrompt) {
+  CMD.push("--append-system-prompt", opts.systemPrompt);
+}
+```
+
+**3. Hook Events (`src/hooks/hook.ts`)**
+- `SessionStart`
+- `PreCompact`
+- `PreToolUse`
+- `PostToolUse`
+- `UserPromptSubmit`
+- `Stop`
+
+---
+
+## Authentication Comparison
+
+### Claude Code
+
+| Method | Environment Variable | Notes |
+|--------|---------------------|-------|
+| OAuth Token | `CLAUDE_CODE_OAUTH_TOKEN` | Used in Docker workers |
+| API Key | Not directly supported | Uses Anthropic account auth |
+
+### Gemini CLI
+
+| Method | Environment Variables | Best For |
+|--------|----------------------|----------|
+| **API Key** | `GEMINI_API_KEY` | Simple automation, quick setup |
+| **Service Account** | `GOOGLE_APPLICATION_CREDENTIALS` + `GOOGLE_CLOUD_PROJECT` + `GOOGLE_CLOUD_LOCATION` | Docker, CI/CD, containers |
+| **ADC (gcloud)** | `GOOGLE_CLOUD_PROJECT` + `GOOGLE_CLOUD_LOCATION` | Local dev with gcloud installed |
+| **OAuth** | Browser login, cached at `~/.gemini/` | Interactive local use |
+| **Cloud API Key** | `GOOGLE_API_KEY` + `GOOGLE_CLOUD_PROJECT` | Vertex AI access |
+
+### Key Auth Behaviors
+
+**Auto .env loading**: Gemini CLI automatically loads environment variables from:
+1. `.gemini/.env` in project directory (first found going up)
+2. `~/.gemini/.env`
+3. `~/.env`
+
+This is similar to Bun's `.env` loading behavior.
+
+**Headless auth**: For containers/automation, use either:
+- `GEMINI_API_KEY` - Simplest, just set the API key
+- Service account JSON file mounted + `GOOGLE_APPLICATION_CREDENTIALS` path
+
+### Docker Auth Mapping
+
+| Current (Claude) | Gemini Equivalent |
+|------------------|-------------------|
+| `CLAUDE_CODE_OAUTH_TOKEN` env | `GEMINI_API_KEY` env |
+| Token validation at startup | API key validation at startup |
+| N/A | Service account JSON mount option |
+
+For multi-CLI Docker support, the entrypoint would check:
+```bash
+if [ "$CLI_TOOL" = "gemini" ]; then
+  if [ -z "$GEMINI_API_KEY" ] && [ -z "$GOOGLE_APPLICATION_CREDENTIALS" ]; then
+    echo "Error: GEMINI_API_KEY or GOOGLE_APPLICATION_CREDENTIALS required"
+    exit 1
+  fi
+elif [ "$CLI_TOOL" = "claude" ]; then
+  if [ -z "$CLAUDE_CODE_OAUTH_TOKEN" ]; then
+    echo "Error: CLAUDE_CODE_OAUTH_TOKEN required"
+    exit 1
+  fi
+fi
+```
+
+---
+
+## Gemini CLI Feature Comparison
+
+### Headless Mode Flags
+
+| Feature | Claude CLI | Gemini CLI |
+|---------|------------|------------|
+| Prompt flag | `-p <message>` | `--prompt <message>` |
+| JSON output | `--output-format stream-json` | `--output-format stream-json` |
+| Verbose mode | `--verbose` | N/A |
+| Auto-approve | `--dangerously-skip-permissions` | `--yolo` |
+| Model selection | N/A | `--model <name>` |
+| Approval mode | `--permission-mode bypassPermissions` | `--approval-mode <mode>` |
+| System prompt | `--append-system-prompt` | `GEMINI_SYSTEM_MD` env var |
+
+### MCP Server Configuration
+
+**Claude Code (`.mcp.json`):**
+```json
+{
+  "mcpServers": {
+    "agent-swarm": {
+      "command": "bunx",
+      "args": ["@desplega.ai/agent-swarm@latest", "mcp", "stdio"],
+      "env": { "API_KEY": "..." }
+    }
+  }
+}
+```
+
+**Gemini CLI (`settings.json`):**
+```json
+{
+  "mcpServers": {
+    "agent-swarm": {
+      "command": "bunx",
+      "args": ["@desplega.ai/agent-swarm@latest", "mcp", "stdio"],
+      "env": { "API_KEY": "..." }
+    }
+  }
+}
+```
+
+Nearly identical! Both follow the MCP specification.
+
+### Hook Events Mapping
+
+| Claude Code | Gemini CLI | Notes |
+|-------------|------------|-------|
+| `SessionStart` | `SessionStart` | Same |
+| `Stop` | `SessionEnd` | Different name |
+| `PreToolUse` | `BeforeTool` | Different name |
+| `PostToolUse` | `AfterTool` | Different name |
+| `UserPromptSubmit` | `BeforeAgent` | Similar concept |
+| `PreCompact` | `PreCompress` | Different name |
+| N/A | `BeforeModel` / `AfterModel` | Gemini-only |
+| N/A | `BeforeToolSelection` | Gemini-only |
+| N/A | `Notification` | Gemini-only |
+
+### Custom Commands
+
+| Feature | Claude Code | Gemini CLI |
+|---------|-------------|------------|
+| Format | Markdown | TOML |
+| Location | `.claude/commands/` | `.gemini/commands/` |
+| Invocation | `/command-name` | `/command-name` |
+| Arguments | Passed to prompt | `{{args}}` placeholder |
+| Namespacing | Flat | Colon-separated (`/git:commit`) |
+
+---
+
+## Integration Architecture
+
+### Option 1: CLI Adapter Pattern
+
+Create an abstraction layer that normalizes CLI differences:
+
+```typescript
+// src/adapters/types.ts
+interface CLIAdapter {
+  name: string;
+  command: string;
+  buildArgs(opts: RunOptions): string[];
+  parseOutput(line: string): ParsedOutput;
+  hookEventMap: Record<string, string>;
+  authEnvVars: string[];
+}
+
+// src/adapters/claude-adapter.ts
+export const claudeAdapter: CLIAdapter = {
+  name: "claude",
+  command: "claude",
+  authEnvVars: ["CLAUDE_CODE_OAUTH_TOKEN"],
+  buildArgs: (opts) => [
+    "--verbose",
+    "--output-format", "stream-json",
+    "--dangerously-skip-permissions",
+    "--allow-dangerously-skip-permissions",
+    "--permission-mode", "bypassPermissions",
+    "-p", opts.prompt,
+    ...(opts.systemPrompt ? ["--append-system-prompt", opts.systemPrompt] : []),
+  ],
+  parseOutput: (line) => JSON.parse(line),
+  hookEventMap: {
+    SessionStart: "SessionStart",
+    Stop: "Stop",
+    PreToolUse: "PreToolUse",
+    PostToolUse: "PostToolUse",
+  },
+};
+
+// src/adapters/gemini-adapter.ts
+export const geminiAdapter: CLIAdapter = {
+  name: "gemini",
+  command: "gemini",
+  authEnvVars: ["GEMINI_API_KEY", "GOOGLE_APPLICATION_CREDENTIALS"],
+  buildArgs: (opts) => [
+    "--output-format", "stream-json",
+    "--yolo",
+    "--prompt", opts.prompt,
+  ],
+  // System prompt handled via GEMINI_SYSTEM_MD env var
+  parseOutput: (line) => JSON.parse(line),
+  hookEventMap: {
+    SessionStart: "SessionStart",
+    Stop: "SessionEnd",
+    PreToolUse: "BeforeTool",
+    PostToolUse: "AfterTool",
+  },
+};
+```
+
+### Option 2: Configuration-Driven
+
+Add CLI tool configuration to environment/settings:
+
+```typescript
+interface CLIConfig {
+  command: string;
+  headlessFlags: string[];
+  autoApproveFlag: string;
+  promptFlag: string;
+  systemPromptFlag?: string;
+  systemPromptEnv?: string;
+  outputFormatFlag: string[];
+  authEnvVars: string[];
+}
+
+const CLI_CONFIGS: Record<string, CLIConfig> = {
+  claude: {
+    command: "claude",
+    headlessFlags: ["--verbose"],
+    autoApproveFlag: "--dangerously-skip-permissions",
+    promptFlag: "-p",
+    systemPromptFlag: "--append-system-prompt",
+    outputFormatFlag: ["--output-format", "stream-json"],
+    authEnvVars: ["CLAUDE_CODE_OAUTH_TOKEN"],
+  },
+  gemini: {
+    command: "gemini",
+    headlessFlags: [],
+    autoApproveFlag: "--yolo",
+    promptFlag: "--prompt",
+    systemPromptEnv: "GEMINI_SYSTEM_MD",
+    outputFormatFlag: ["--output-format", "stream-json"],
+    authEnvVars: ["GEMINI_API_KEY", "GOOGLE_APPLICATION_CREDENTIALS"],
+  },
+};
+```
+
+### Files Requiring Changes
+
+| File | Change Required |
+|------|-----------------|
+| `src/claude.ts` | Rename to `src/cli-runner.ts`, add adapter support |
+| `src/commands/runner.ts` | Use CLI adapter instead of hardcoded flags |
+| `src/hooks/hook.ts` | Map hook events through adapter |
+| `src/commands/setup.tsx` | Generate config for selected CLI |
+| `Dockerfile.worker` | Add Gemini CLI installation option |
+| `docker-entrypoint.sh` | Handle multiple auth mechanisms |
+| `package.json` | Add `gemini` script variants |
+
+### What Works Without Changes
+
+1. **MCP Server** (`src/server.ts`) - Fully CLI-agnostic
+2. **All MCP Tools** (`src/tools/*.ts`) - Pure MCP protocol
+3. **Database Layer** (`src/be/db.ts`) - No CLI awareness
+4. **Type Definitions** (`src/types.ts`) - CLI-agnostic data structures
+5. **HTTP/STDIO Transports** - Standard MCP SDK
+
+---
+
+## Code References
+
+- `src/claude.ts:9-80` - Claude CLI spawning wrapper
+- `src/commands/runner.ts:65-192` - Agent runner with CLI flags
+- `src/hooks/hook.ts:56-306` - Hook event handler
+- `src/server.ts:42-96` - CLI-agnostic MCP server
+- `src/commands/setup.tsx:56-76` - Claude-specific setup
+- `Dockerfile.worker:76-98` - Claude CLI installation
+
+## Limitations
+
+| Feature | Claude Code | Gemini CLI | Impact |
+|---------|-------------|------------|--------|
+| Plugin system | Full support | Extensions (different API) | Commands need porting |
+| Agents | Supported | Not mentioned | May not work |
+| Skills | Supported | Not mentioned | May not work |
+| Context caching | Yes | Yes (token caching) | Works differently |
+| Hooks | 6 events | 11 events | Need event mapping |
+
+## Related Research
+
+- `thoughts/shared/research/2025-12-19-agent-log-streaming.md` - Claude CLI JSON streaming format
+
+## Open Questions
+
+1. **JSON streaming format** - Are Claude's and Gemini's stream-json outputs compatible? Both emit newline-delimited JSON but event structures may differ.
+
+2. **Tool name prefixing** - Both CLIs prefix MCP tools. Need to verify `mcp__agent-swarm__*` permission pattern works in Gemini.
+
+3. **Gemini Extensions vs Claude Plugins** - These are different systems. If we want to support Gemini extensions, that's a separate effort.
+
+4. **Model selection** - Gemini CLI supports model switching (`/model`). Should we expose this in agent config?
+
+5. **Quota differences** - Gemini has different quota/pricing. May affect swarm behavior with many agents.

package/thoughts/shared/research/2025-12-22-setup-experience-improvements.md

@@ -0,0 +1,264 @@
+---
+date: 2025-12-22T01:20:00-08:00
+researcher: Claude Opus 4.5
+git_commit: 4334e7c
+branch: main
+repository: agent-swarm
+topic: "Setup Experience & Demo Ideas"
+tags: [research, developer-experience, onboarding, demo]
+status: complete
+last_updated: 2025-12-22
+last_updated_by: Claude Opus 4.5
+---
+
+# Research: Setup Experience & Demo Ideas
+
+**Date**: 2025-12-22T01:20:00-08:00
+**Researcher**: Claude Opus 4.5
+**Git Commit**: 4334e7c
+**Branch**: main
+**Repository**: agent-swarm
+
+## Research Question
+
+How to make the project super easy to setup and run for someone that clones or wants to try it out locally or deploy it? What are ideas for a demo?
+
+## Summary
+
+The project has solid infrastructure but the onboarding experience could be significantly improved. The main barriers are: (1) scattered documentation with no true "30-second quick start", (2) UI and server require separate setup, (3) Claude OAuth token requirement is a major barrier for new users, and (4) no simple demo to understand what the project does. Below are specific recommendations and demo ideas.
+
+## Current Project Structure
+
+```
+agent-swarm/
+├── src/                    # MCP Server + CLI
+│   ├── http.ts            # HTTP MCP server (main backend)
+│   ├── stdio.ts           # STDIO transport
+│   ├── cli.tsx            # CLI entry (setup, worker, lead, hook)
+│   ├── commands/          # CLI command implementations
+│   └── tools/             # MCP tool handlers
+├── ui/                    # Separate Vite dashboard (MUI Joy + React)
+│   ├── src/               # React components
+│   └── package.json       # Separate deps (uses pnpm)
+├── plugin/                # Claude Code plugin (not yet published)
+├── deploy/                # Deployment scripts (systemd, Docker)
+├── Dockerfile.worker      # Worker container with full dev env
+└── docker-compose.*.yml   # Docker compose configs
+```
+
+## Current Setup Experience Gaps
+
+### 1. Prerequisites Not Prominent
+- Bun is required but mentioned only in passing
+- No clear version requirements
+- Node.js 22 mentioned for Docker only
+
+### 2. No Unified Quick Start
+Current README jumps straight to `bunx @desplega.ai/agent-swarm@latest setup` which:
+- Assumes user has Bun installed
+- Requires API token from cloud service
+- Doesn't work for local development
+
+### 3. UI Requires Separate Setup
+```bash
+# User must know to:
+cd ui
+pnpm install  # Note: different package manager!
+pnpm dev      # Separate terminal
+```
+
+### 4. Missing "All-in-One" Development Script
+No single command to start both server and UI with hot reload.
+
+### 5. Claude OAuth Token Barrier
+Biggest friction: requires `CLAUDE_CODE_OAUTH_TOKEN` which is:
+- Hard to obtain (undocumented process)
+- Required even for basic testing
+- Blocks casual exploration
+
+### 6. No Simple "See It Work" Demo
+Users can't quickly see value without significant setup.
+
+## Improvement Recommendations
+
+### 1. Add Prerequisites Section (Top of README)
+
+```markdown
+## Prerequisites
+
+- [Bun](https://bun.sh) v1.1+ (`curl -fsSL https://bun.sh/install | bash`)
+- [Claude Code CLI](https://claude.ai/install.sh) (for running agents)
+- Docker (optional, for containerized workers)
+```
+
+### 2. Add True Quick Start (Local Development)
+
+```markdown
+## Quick Start (Local Development)
+
+# 1. Clone and install
+git clone https://github.com/desplega-ai/agent-swarm.git
+cd agent-swarm
+bun install
+
+# 2. Create .env file
+cp .env.example .env
+# Edit .env to set API_KEY=any-secret-you-want
+
+# 3. Start the MCP server
+bun run dev:http
+
+# Server running at http://localhost:3013
+
+# 4. (Optional) Start the dashboard
+cd ui && pnpm install && pnpm dev
+# Dashboard at http://localhost:5173
+```
+
+### 3. Add `dev:all` Script to package.json
+
+```json
+"scripts": {
+  "dev:all": "bun run dev:http & (cd ui && pnpm dev)",
+  "dev:ui": "cd ui && pnpm dev"
+}
+```
+
+Or use a process manager approach:
+```bash
+# Add to package.json scripts
+"dev:all": "bun --hot src/http.ts & cd ui && pnpm dev"
+```
+
+### 4. Add MCP Inspector Demo Mode
+
+For users who just want to see it work without Claude:
+
+```markdown
+## Try Without Claude
+
+Test the MCP tools using the inspector:
+
+bun run inspector:http
+
+This opens a web UI where you can:
+- Call `join-swarm` to register an agent
+- Use `send-task` to create tasks
+- See the full API without needing Claude
+```
+
+### 5. Create Demo Modes in CLI
+
+Add simple demo commands that simulate swarm behavior:
+
+```bash
+# Simulate a 3-agent swarm locally
+bunx @desplega.ai/agent-swarm demo --agents 3
+
+# This would:
+# 1. Start local server
+# 2. Register 3 fake agents
+# 3. Create sample tasks
+# 4. Show agents claiming and completing tasks
+```
+
+### 6. Add Screencast/GIF to README
+
+A 30-second GIF showing:
+1. Dashboard with agents
+2. Tasks being assigned
+3. Real-time progress updates
+4. Agent messaging
+
+### 7. Environment File Consolidation
+
+Create a single `.env.development.example`:
+```env
+# Local Development Configuration
+API_KEY=dev-secret-key
+PORT=3013
+SWARM_URL=localhost
+
+# Only needed for actual Claude agents:
+# CLAUDE_CODE_OAUTH_TOKEN=your-token
+```
+
+## Demo Ideas
+
+### Demo 1: Task Coordination Screencast (Simplest)
+**Setup**: 1 lead + 2 workers
+**Scenario**: Lead breaks down "Build a calculator" into tasks:
+- Worker 1: Implement add/subtract
+- Worker 2: Implement multiply/divide
+- Both report progress, lead monitors
+
+**Why it works**: Shows core value prop - parallel execution with coordination
+
+### Demo 2: Code Review Swarm (Practical)
+**Setup**: 1 lead + 3 specialized workers
+**Scenario**: Review a PR from multiple angles:
+- Worker 1: Security review
+- Worker 2: Performance review
+- Worker 3: Test coverage review
+- Lead synthesizes findings
+
+**Why it works**: Immediately useful, shows specialization
+
+### Demo 3: Research Swarm (Impressive)
+**Setup**: 1 lead + 3 researchers
+**Scenario**: Research "Best practices for X" topic:
+- Workers search different sources in parallel
+- Post findings to shared channel
+- Lead synthesizes into final report
+
+**Why it works**: Shows messaging + parallel speedup
+
+### Demo 4: Service Registry Demo (Technical)
+**Setup**: 2 workers with background services
+**Scenario**:
+- Worker 1: Starts an API server (PM2 + register-service)
+- Worker 2: Discovers and calls Worker 1's API
+- Shows cross-agent service communication
+
+**Why it works**: Demonstrates unique capability
+
+### Demo 5: Interactive Dashboard Demo
+**No Claude needed**
+**Scenario**: Use the MCP Inspector to:
+1. Register 3 agents manually
+2. Create tasks in the pool
+3. Claim tasks as different agents
+4. Show dashboard updating in real-time
+
+**Why it works**: Zero barrier to entry, visual
+
+## Recommended Demo Implementation Priority
+
+1. **Demo 5 (Dashboard Demo)** - Zero friction, can be a video on README
+2. **Demo 1 (Task Coordination)** - Core value, simple script
+3. **Demo 2 (Code Review)** - Practical application
+4. **Demo 3 (Research)** - Shows advanced features
+
+## Quick Wins
+
+1. Add `bun run dev:all` script
+2. Move Prerequisites to top of README
+3. Add "Try Without Claude" section
+4. Record 30-second GIF for README
+5. Create `.env.development.example` with comments
+
+## Related Files
+
+- `README.md` - Main documentation
+- `deploy/DEPLOY.md` - Server deployment docs
+- `.env.example` - Server environment template
+- `.env.docker.example` - Docker environment template
+- `src/cli.tsx` - CLI entry point
+- `ui/` - Dashboard application
+
+## Open Questions
+
+1. Can we publish a "demo mode" that doesn't require OAuth?
+2. Should UI be embedded in main server (single port)?
+3. Is there a way to provide temporary Claude tokens for demos?
+4. Should there be a hosted demo instance users can connect to?

package/tsconfig.json

@@ -2,6 +2,7 @@
   "compilerOptions": {
     // Environment setup & latest features
     "lib": ["ESNext"],
+    "types": ["bun"],
     "target": "ESNext",
     "module": "Preserve",
     "moduleDetection": "force",
@@ -31,5 +32,6 @@
     "noUnusedLocals": false,
     "noUnusedParameters": false,
     "noPropertyAccessFromIndexSignature": false
-  }
+  },
+  "exclude": ["ui", "node_modules"]
 }