agent-pool-mcp 1.6.0 → 1.7.1

package/README.md

@@ -1,3 +1,7 @@
+[![npm version](https://img.shields.io/npm/v/agent-pool-mcp)](https://www.npmjs.com/package/agent-pool-mcp)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![Node.js](https://img.shields.io/badge/Node.js-%3E%3D20-339933?logo=node.js&logoColor=white)](https://nodejs.org)
+
 # agent-pool-mcp
 
 **MCP server for multi-agent orchestration** — parallel task delegation, sequential pipelines, cron scheduling, and cross-model peer review via [Gemini CLI](https://github.com/google-gemini/gemini-cli).
@@ -6,13 +10,9 @@
 
 Compatible with [Antigravity](https://antigravity.dev), Cursor, Windsurf, Claude Code, and any MCP-enabled coding agent.
 
-## Why?
-
-AI coding assistants are powerful, but they work **sequentially** — one task at a time. Agent-pool turns your single Gemini subscription into a **parallel agent workforce**: your primary IDE agent delegates background tasks to Gemini CLI workers, all sharing the same authentication.
+Your primary IDE agent delegates background tasks to Gemini CLI workers in parallel — all sharing the same authentication from a single Gemini subscription.
 
-When the primary agent and Gemini workers are **different foundation models** (e.g. Claude + Gemini), `consult_peer` becomes a **cross-model reasoning amplifier** — two independent architectures reviewing each other eliminate systematic blind spots that plague single-model workflows.
-
-## How It Works
+When the primary agent and Gemini workers are **different foundation models** (e.g. Claude + Gemini), `consult_peer` gives you cross-model review — two models check each other's reasoning independently.
 
 ```
 ┌─────────────────────────────────┐
@@ -30,45 +30,18 @@ When the primary agent and Gemini workers are **different foundation models** (e
   (task1)   (task2)   (review)       (same auth, parallel)
 ```
 
-## Features
-
-### 🚀 Task Delegation
-- **`delegate_task`** — Non-blocking task delegation to Gemini CLI (full filesystem access).
-- **`delegate_task_readonly`** — Read-only analysis (plan mode). Supports `session_id` to resume previous analyses.
-- **`get_task_result`** — Poll task status, retrieve results, and see live progress (last 200 tool/message events).
-- **`cancel_task`** — Kill a running task and its entire process group immediately.
-
-### 🔗 Pipelines — Sequential Task Chains
-Define multi-step workflows where agents execute sequentially, with automatic handoff:
-
-```
-         ┌─ frontend ─┐
-research ─┤             ├── deploy
-         └─ backend  ─┘
-```
+> [!TIP]
+> A single $20/month Google AI Ultra subscription can power dozens of parallel workers — no additional API keys required.
 
-- **`create_pipeline`** — Define a pipeline with named steps, triggers, and timeouts.
-- **`run_pipeline`** — Start executing a pipeline. A detached daemon manages the lifecycle.
-- **`list_pipelines`** — See all definitions, active runs, and recent completions.
-- **`get_pipeline_status`** — Step-by-step status with emoji indicators.
-- **`cancel_pipeline`** — Stop a running pipeline and kill active step processes.
+### Task Delegation
 
-**Agent Signals** (called BY agents running inside pipeline steps):
-- **`signal_step_complete`** — Mark the current step as done. Accepts optional output and `run_id`.
-- **`bounce_back`** — Return task to a previous step with feedback (e.g. "data incomplete"). Supports `maxBounces` limit.
+Non-blocking task delegation to Gemini CLI workers. The primary agent fires off a task and continues working — polling for results when ready. Workers get full filesystem access (`delegate_task`) or read-only mode (`delegate_task_readonly`). Cancel anytime with `cancel_task`.
 
-**Triggers:**
+### Pipelines — Sequential Task Chains
 
-| Trigger | Description |
-|---------|-------------|
-| `on_complete` | Start when a specific step succeeds |
-| `on_complete_all` | Fan-in: start when ALL listed steps succeed |
-| `on_file` | Start when a file appears and the producing process exits |
-| Auto-fallback | Process death without signal → auto-complete/fail |
+Multi-step workflows with automatic handoff between steps:
 
-**Example — 3-step pipeline:**
 ```javascript
-// Agent creates the pipeline
 create_pipeline({
   name: "article-workflow",
   steps: [
@@ -77,86 +50,59 @@ create_pipeline({
     { name: "review", prompt: "Review the draft for accuracy and style" }
   ]
 })
-
-// Agent starts execution — daemon handles the rest
 run_pipeline({ pipeline_id: "article-workflow" })
 ```
 
-### ⏰ Cron Scheduler
-Schedule agents to run automatically on a cron schedule:
+Steps support triggers: `on_complete` (chain after one step), `on_complete_all` (fan-in after several), and `on_file` (start when a file appears). Agents can `bounce_back` to a previous step with feedback if data is incomplete.
 
-- **`schedule_task`** — Schedule a Gemini CLI agent with cron expression (e.g. `0 9 * * MON-FRI`).
-- **`list_schedules`** — See all schedules with next run times and daemon status.
-- **`cancel_schedule`** — Remove a schedule. Daemon auto-exits when no schedules remain.
-- **`get_scheduled_results`** — Retrieve results from past scheduled executions.
+### Cron Scheduler
 
-The scheduler runs as a **detached daemon** that survives IDE/CLI restarts. It uses atomic file locks to prevent duplicate execution when multiple clients are connected.
+Schedule agents on a cron expression — a detached daemon survives IDE/CLI restarts. Uses atomic file locks to prevent duplicate execution.
 
-### 📋 3-Tier Skill System
-Skills are Markdown files with YAML frontmatter that extend agent behavior. Agent-pool manages skills in three tiers:
-1.  **Project**: `.gemini/skills/` (local to repo, takes precedence).
-2.  **Global**: `~/.gemini/skills/` (available across all projects).
-3.  **Built-in**: Shipped with agent-pool (e.g., `code-reviewer`, `test-writer`, `doc-fixer`).
-
-**Skill Tools:**
-- **`list_skills`** — See all available skills and their tiers.
-- **`install_skill`** — Copy a global or built-in skill to the project tier for local customization.
-- **`create_skill` / `delete_skill`** — Manage skill files in project or global scope.
+```
+"0 9 * * MON-FRI"    — 9am weekdays
+"*/30 * * * *"       — every 30 minutes
+"0 */2 * * *"        — every 2 hours
+```
 
-*Note: When delegating with a skill, agent-pool uses "hybrid activation" — it ensures the skill is available in the project and instructs Gemini CLI to activate it natively.*
+Results are saved to `.agents/scheduled-results/` and retrievable via `get_scheduled_results`.
 
-### 🛡️ Per-Task Policies
-Restrict tool usage for specific tasks using YAML policies. Use built-in templates or custom paths:
-- `policy: "read-only"` — Disables all file-writing and destructive shell tools.
-- `policy: "safe-edit"` — Allows file modifications but blocks arbitrary shell execution.
-- `policy: "/path/to/my-policy.yaml"` — Use a custom security policy.
+### 3-Tier Skill System
 
-### 🤝 Cross-Model Peer Review
-- **`consult_peer`** — Architectural review with structured verdicts (AGREE / SUGGEST_CHANGES / DISAGREE).
-- Supports iterative rounds: propose → get feedback → revise → re-send until consensus.
+Skills are Markdown files with YAML frontmatter that extend agent behavior:
 
-### 📊 System Awareness & Management
-- **System Load Detection**: Automatically detects other running Gemini processes on the system and warns if the worker pool is saturated.
-- **Session Management**: `list_sessions` allows resuming previous Gemini CLI conversations by UUID.
+1. **Project** — `.gemini/skills/` (local to repo, takes precedence)
+2. **Global** — `~/.gemini/skills/` (available across all projects)
+3. **Built-in** — shipped with agent-pool (`code-reviewer`, `test-writer`, `doc-fixer`, `orchestrator`)
 
-## Remote Workers (SSH)
+Install a built-in or global skill into the project for local customization with `install_skill`.
 
-Run workers on remote servers via SSH — same interface, transparent stdio forwarding.
-Create `agent-pool.config.json` in your project root or `~/.config/agent-pool/config.json`:
+### Per-Task Policies
 
-```json
-{
-  "runners": [
-    { "id": "local", "type": "local" },
-    { "id": "gpu", "type": "ssh", "host": "gpu-server", "cwd": "/home/dev/project" }
-  ],
-  "defaultRunner": "local"
-}
-```
+Restrict tool usage for specific tasks using YAML policies:
+- `"read-only"` — disables all file-writing and destructive shell tools
+- `"safe-edit"` — allows file modifications but blocks arbitrary shell execution
+- Custom path — `"/path/to/my-policy.yaml"`
 
-## Nested Orchestration
+### Cross-Model Peer Review
 
-Install agent-pool inside Gemini CLI to enable **hierarchical delegation** — workers can spawn their own workers.
+`consult_peer` sends architectural proposals to a Gemini worker for structured review. The worker responds with a verdict: **AGREE**, **SUGGEST_CHANGES**, or **DISAGREE**. Supports iterative rounds until consensus.
 
-| Variable | Purpose | Default |
-|----------|---------|--------|
-| `AGENT_POOL_DEPTH` | Current nesting level (auto-incremented) | `0` |
-| `AGENT_POOL_MAX_DEPTH` | Max allowed depth | not set (no limit) |
+### Security
 
-See [parallel-work guide](examples/parallel-work.md) and built-in `orchestrator` skill for patterns.
+- **Path Traversal Protection** — all skill and policy operations are sanitized to prevent access outside designated directories
+- **Process Isolation** — tasks run as detached processes; `cancel_task` and server shutdown kill entire process groups
+- **Credential Safety** — uses your local Gemini CLI authentication; no keys are stored or transmitted
 
-## Prerequisites
+## Quick Start
 
-- **Node.js >= 20** — [Download](https://nodejs.org)
-- **[Gemini CLI](https://github.com/google-gemini/gemini-cli)** — installed and authenticated:
+**Prerequisites:** Node.js >= 20, [Gemini CLI](https://github.com/google-gemini/gemini-cli) installed and authenticated.
 
 ```bash
 npm install -g @google/gemini-cli
 gemini    # First run: opens browser for OAuth
 ```
 
-## Installation
-
 Add to your IDE's MCP configuration:
 
 ```json
@@ -173,7 +119,7 @@ Add to your IDE's MCP configuration:
 Restart your IDE — agent-pool-mcp will be downloaded and started automatically.
 
 <details>
-<summary>📍 Where is my MCP config file?</summary>
+<summary>Where is my MCP config file?</summary>
 
 | IDE | Config path |
 |-----|------------|
@@ -185,7 +131,7 @@ Restart your IDE — agent-pool-mcp will be downloaded and started automatically
 </details>
 
 <details>
-<summary>📦 Alternative: global install</summary>
+<summary>Alternative: global install</summary>
 
 ```bash
 npm install -g agent-pool-mcp
@@ -201,9 +147,9 @@ Then use `"command": "agent-pool-mcp"` in your MCP config (no npx needed).
 npx agent-pool-mcp --check
 ```
 
-This runs diagnostics: checks Node.js, Gemini CLI, authentication, and remote runner connectivity.
+Runs diagnostics: checks Node.js, Gemini CLI, authentication, and remote runner connectivity.
 
-### CLI Commands
+### CLI
 
 ```bash
 npx agent-pool-mcp --check      # Doctor mode: diagnose prerequisites
@@ -212,6 +158,31 @@ npx agent-pool-mcp --version    # Show version
 npx agent-pool-mcp --help       # Full help
 ```
 
+## Remote Workers (SSH)
+
+Run workers on remote servers via SSH — same interface, transparent stdio forwarding. Create `agent-pool.config.json` in your project root or `~/.config/agent-pool/config.json`:
+
+```json
+{
+  "runners": [
+    { "id": "local", "type": "local" },
+    { "id": "gpu", "type": "ssh", "host": "gpu-server", "cwd": "/home/dev/project" }
+  ],
+  "defaultRunner": "local"
+}
+```
+
+### Nested Orchestration
+
+Install agent-pool inside Gemini CLI to enable hierarchical delegation — workers can spawn their own workers.
+
+| Variable | Purpose | Default |
+|----------|---------|--------|
+| `AGENT_POOL_DEPTH` | Current nesting level (auto-incremented) | `0` |
+| `AGENT_POOL_MAX_DEPTH` | Max allowed depth | not set (no limit) |
+
+See [parallel-work guide](examples/parallel-work.md) and built-in `orchestrator` skill for patterns.
+
 ## MCP Ecosystem
 
 Best used together with [**project-graph-mcp**](https://www.npmjs.com/package/project-graph-mcp) — AST-based codebase analysis:
@@ -221,8 +192,6 @@ Best used together with [**project-graph-mcp**](https://www.npmjs.com/package/pr
 | **Primary IDE agent** | Delegates tasks, consults peer | Navigates codebase, runs analysis |
 | **Gemini CLI workers** | Executes delegated tasks | Available as MCP tool inside workers |
 
-Combined config for both:
-
 ```json
 {
   "mcpServers": {
@@ -238,53 +207,23 @@ Combined config for both:
 }
 ```
 
-## Security
-
-- **Path Traversal Protection**: All skill and policy operations are sanitized to prevent access outside designated directories.
-- **Process Isolation**: Tasks run as detached processes; `cancel_task` and server shutdown ensure no zombie processes remain by killing entire process groups.
-- **Credential Safety**: Uses your local Gemini CLI authentication; no keys are stored or transmitted by this server.
+> [!IMPORTANT]
+> Each Gemini CLI worker gets its own MCP server instance but shares pipeline state via filesystem — no coordination overhead.
 
-## Architecture
+## Documentation
 
-```
-index.js                    ← Entry point (stdio transport)
-policies/                   ← Tool restriction policies (YAML)
-├── read-only.yaml
-└── safe-edit.yaml
-skills/                     ← Built-in Gemini CLI skills (Markdown)
-├── code-reviewer.md
-├── doc-fixer.md
-├── orchestrator.md
-└── test-writer.md
-src/
-├── cli.js                  ← CLI commands (--check, --init, --help)
-├── server.js               ← MCP server setup + tool routing
-├── tool-definitions.js     ← Tool schemas (JSON Schema)
-├── tools/
-│   ├── consult.js          ← Peer review via Gemini CLI
-│   ├── results.js          ← Task store + result formatting (TTL cleanup, ring buffer)
-│   └── skills.js           ← 3-tier skill management (project/global/built-in)
-├── runner/
-│   ├── config.js           ← Runner config loader (local/SSH)
-│   ├── gemini-runner.js    ← Process spawning (streaming JSON, depth tracking)
-│   ├── process-manager.js  ← PID tracking, system load awareness, group kill
-│   └── ssh.js              ← Shell escaping, remote PID tracking
-└── scheduler/
-    ├── cron.js             ← Minimal cron expression parser (zero-dependency)
-    ├── daemon.js           ← Detached daemon: schedule ticks + pipeline lifecycle
-    ├── pipeline.js         ← Pipeline CRUD, run state, signals, bounce-back
-    └── scheduler.js        ← Schedule management + daemon spawning
-```
+- [ARCHITECTURE.md](ARCHITECTURE.md) — Source code structure and process management details
+- [examples/parallel-work.md](examples/parallel-work.md) — Delegation patterns and best practices
 
-**Process management:**
-- **Detached Spawn**: Workers are spawned in their own process groups.
-- **TTL Cleanup**: Completed task results are purged from memory after 10 minutes.
-- **Live Events**: Progress polling uses a ring buffer to show the latest activity without overwhelming context.
-- **Depth Tracking**: Nested orchestration support with optional `AGENT_POOL_MAX_DEPTH` limit.
-- **Adaptive Polling**: Pipeline daemon uses 3s intervals when active, 30s when idle.
-- **File-Based Communication**: Pipeline agents communicate through `.agents/runs/` JSON files — each Gemini process has its own MCP server instance but shares state via filesystem.
+## Related Projects
+- [project-graph-mcp](https://github.com/rnd-pro/project-graph-mcp) — AST-based codebase analysis for AI agents
+- [Symbiote.js](https://github.com/symbiotejs/symbiote.js) — Isomorphic Reactive Web Components framework
+- [JSDA-Kit](https://github.com/rnd-pro/jsda-kit) — SSG/SSR toolkit for modern web applications
 
 ## License
 
-MIT
+MIT © [RND-PRO.com](https://rnd-pro.com)
+
+---
 
+**Made with ❤️ by the RND-PRO team**

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "agent-pool-mcp",
-  "version": "1.6.0",
+  "version": "1.7.1",
   "type": "module",
   "description": "MCP Server for multi-agent task delegation and orchestration via Gemini CLI",
   "main": "index.js",
@@ -31,7 +31,8 @@
     "cron",
     "ai"
   ],
-  "author": "Vladislav Matiyasevich",
+  "author": "RND-PRO",
+  "homepage": "https://github.com/rnd-pro/agent-pool-mcp",
   "license": "MIT",
   "repository": {
     "type": "git",

package/src/scheduler/daemon.js

@@ -11,7 +11,7 @@
  * @module agent-pool/scheduler/daemon
  */
 
-import { readFileSync, writeFileSync, existsSync, mkdirSync, unlinkSync, readdirSync } from 'node:fs';
+import { readFileSync, writeFileSync, existsSync, mkdirSync, unlinkSync, readdirSync, renameSync } from 'node:fs';
 import { spawn } from 'node:child_process';
 import { join, dirname } from 'node:path';
 import { matchesCron } from './cron.js';
@@ -19,6 +19,7 @@ import { getGroup } from '../tools/groups.js';
 import { getRunner } from '../runner/config.js';
 import { buildSshSpawn } from '../runner/ssh.js';
 import { killGroup } from '../runner/process-manager.js';
+import { consumeSignals, deleteSignals } from './run-signals.js';
 
 const POLL_INTERVAL_MS = 30_000; // Check schedules every 30 seconds
 const PID_FILE = '.agents/scheduler.pid';
@@ -163,11 +164,125 @@ function executeSchedule(schedule) {
   console.error(`[scheduler] Started: ${schedule.id} → gemini pid ${child.pid}`);
 }
 
-// ─── Pipeline tick ──────────────────────────────────────────
+// ─── Pipeline tick ──────────────────────────────────────────────────
 
 const PIPELINES_DIR = '.agents/pipelines';
 const RUNS_DIR = '.agents/runs';
 
+/**
+ * In-memory pipeline state cache.
+ * Loaded from disk on startup, updated in-place during ticks.
+ * Written to disk on state transitions (write-through).
+ * @type {Map<string, object>}
+ */
+const runCache = new Map();
+
+/**
+ * Load all active runs from disk into the in-memory cache.
+ * Called once on daemon startup.
+ */
+function loadRunCache() {
+  const dir = join(cwd, RUNS_DIR);
+  if (!existsSync(dir)) return;
+  for (const f of readdirSync(dir).filter(f => f.endsWith('.json') && !f.includes('.signal-'))) {
+    try {
+      const run = JSON.parse(readFileSync(join(dir, f), 'utf-8'));
+      const runId = f.replace('.json', '');
+      runCache.set(runId, run);
+    } catch { /* skip corrupted */ }
+  }
+  console.error(`[pipeline] Loaded ${runCache.size} runs into memory cache`);
+}
+
+/**
+ * Persist a run to disk atomically (write-then-rename).
+ * Prevents corruption if daemon crashes mid-write.
+ * @param {string} runId
+ * @param {object} run
+ */
+function persistRun(runId, run) {
+  const dir = join(cwd, RUNS_DIR);
+  mkdirSync(dir, { recursive: true });
+  const target = join(dir, `${runId}.json`);
+  const tmp = join(dir, `${runId}.json.tmp`);
+  writeFileSync(tmp, JSON.stringify(run, null, 2));
+  // Atomic rename (same filesystem) — prevents corruption on crash
+  try { renameSync(tmp, target); }
+  catch { writeFileSync(target, JSON.stringify(run, null, 2)); }
+}
+
+/**
+ * Apply consumed signal files to a run's in-memory state.
+ * @param {object} run - Run state object (mutated in place)
+ * @param {Array} signals - Consumed signal objects
+ * @param {object} pipeline - Pipeline definition
+ * @returns {boolean} true if any signal was applied
+ */
+function applySignals(run, signals, pipeline) {
+  let modified = false;
+  for (const signal of signals) {
+    if (signal.type === 'STEP_COMPLETE') {
+      const step = run.steps[signal.stepName];
+      if (step && step.status === 'running') {
+        step.status = 'success';
+        step.signaled = true;
+        step.completedAt = new Date().toISOString();
+        if (signal.output) step.output = signal.output;
+        modified = true;
+        console.error(`[pipeline] Signal: step "${signal.stepName}" completed`);
+      }
+    } else if (signal.type === 'BOUNCE_BACK') {
+      const targetStep = run.steps[signal.stepName];
+      if (!targetStep) continue;
+
+      const stepDef = pipeline?.steps.find(s => s.name === signal.stepName);
+      const maxBounces = stepDef?.maxBounces ?? 2;
+
+      if (targetStep.bounces >= maxBounces) {
+        // Bounce limit reached
+        targetStep.status = 'failed';
+        targetStep.lastBounceReason = `Bounce limit (${maxBounces}) reached. Last: ${signal.reason}`;
+        run.status = 'failed';
+        run.completedAt = new Date().toISOString();
+        console.error(`[pipeline] Bounce limit reached for "${signal.stepName}"`);
+      } else {
+        // Reset target step
+        targetStep.status = 'bounce_pending';
+        targetStep.bounces = (targetStep.bounces || 0) + 1;
+        targetStep.lastBounceReason = signal.reason;
+
+        // Kill running processes for this step
+        const pidsToKill = [...(targetStep.pids || [])];
+        if (targetStep.pid && !pidsToKill.includes(targetStep.pid)) pidsToKill.push(targetStep.pid);
+        for (const pid of pidsToKill) killGroup(pid);
+
+        targetStep.pid = null;
+        targetStep.pids = [];
+        targetStep.exitCode = null;
+        targetStep.signaled = false;
+
+        // Reset calling step
+        if (signal.callingStepName && run.steps[signal.callingStepName]) {
+          run.steps[signal.callingStepName].status = 'waiting_bounce';
+        }
+        console.error(`[pipeline] Bounce: step "${signal.stepName}" reset (reason: ${signal.reason})`);
+      }
+      modified = true;
+    } else if (signal.type === 'CANCEL_RUN') {
+      // Cancel the entire run
+      for (const [name, step] of Object.entries(run.steps)) {
+        if (step.status === 'running') step.status = 'cancelled';
+        if (step.status === 'pending') step.status = 'skipped';
+      }
+      run.status = 'cancelled';
+      run.completedAt = new Date().toISOString();
+      console.error(`[pipeline] Signal: run cancelled`);
+      modified = true;
+    }
+  }
+  return modified;
+}
+
 /**
  * Spawn Gemini CLI agent(s) for a pipeline step.
  * @param {object} stepDef - Step definition from pipeline
@@ -251,19 +366,17 @@ function spawnStep(stepDef, run, runId, bounceReason) {
     const child = spawn(spawnCmd, spawnArgs, spawnOpts);
 
     child.on('close', (code) => {
-      // Update step exit code in run state
-      try {
-        const currentRun = JSON.parse(readFileSync(join(cwd, RUNS_DIR, `${runId}.json`), 'utf-8'));
-        if (currentRun.steps[stepDef.name]) {
-          // If any child fails, set exit code to non-zero
-          if (code !== 0) {
-             currentRun.steps[stepDef.name].exitCode = code;
-          } else if (currentRun.steps[stepDef.name].exitCode === null) {
-             currentRun.steps[stepDef.name].exitCode = 0;
-          }
+      // Update step exit code in in-memory state directly (same process)
+      const currentRun = runCache.get(runId);
+      if (currentRun?.steps[stepDef.name]) {
+        if (code !== 0) {
+          currentRun.steps[stepDef.name].exitCode = code;
+        } else if (currentRun.steps[stepDef.name].exitCode === null) {
+          currentRun.steps[stepDef.name].exitCode = 0;
         }
-        writeFileSync(join(cwd, RUNS_DIR, `${runId}.json`), JSON.stringify(currentRun, null, 2));
-      } catch { /* ignore */ }
+        // Write-through to disk
+        persistRun(runId, currentRun);
+      }
       console.error(`[pipeline] Step "${stepDef.name}" [pid ${child.pid}] exited (code: ${code}, run: ${runId})`);
     });
 
@@ -289,33 +402,69 @@ function isAlive(pid) {
 }
 
 /**
- * Process pipeline runs — check triggers, advance steps.
+ * Process pipeline runs — consume signals, check triggers, advance steps.
+ * Uses in-memory cache for state; persists to disk on changes.
  * @returns {boolean} true if any pipeline is actively running
  */
 function tickPipelines() {
+  // Pick up new runs added to disk since last tick (e.g., from runPipeline)
   const runsDir = join(cwd, RUNS_DIR);
-  if (!existsSync(runsDir)) return false;
+  if (existsSync(runsDir)) {
+    for (const f of readdirSync(runsDir).filter(f => f.endsWith('.json') && !f.includes('.signal-') && !f.endsWith('.tmp'))) {
+      const runId = f.replace('.json', '');
+      if (!runCache.has(runId)) {
+        try {
+          const run = JSON.parse(readFileSync(join(runsDir, f), 'utf-8'));
+          runCache.set(runId, run);
+          console.error(`[pipeline] Picked up new run: ${runId}`);
+        } catch { /* skip corrupted */ }
+      }
+    }
+  }
 
   const pipelinesDir = join(cwd, PIPELINES_DIR);
   let hasActive = false;
 
-  for (const file of readdirSync(runsDir).filter(f => f.endsWith('.json'))) {
-    let run;
-    try { run = JSON.parse(readFileSync(join(runsDir, file), 'utf-8')); }
-    catch { continue; }
-
-    if (run.status !== 'running') continue;
+  // Iterate over a copy of keys to allow modification of runCache during iteration
+  for (const runId of Array.from(runCache.keys())) {
+    const run = runCache.get(runId);
+
+    // Evict completed runs from cache (memory leak fix)
+    if (run.status !== 'running') {
+      // Clean up any orphaned/late signals for completed runs
+      const lateSignals = consumeSignals(cwd, runId);
+      if (lateSignals.length > 0) {
+        deleteSignals(cwd, lateSignals);
+        console.error(`[pipeline] Cleaned ${lateSignals.length} orphaned signal(s) for completed run ${runId}`);
+      }
+      runCache.delete(runId);
+      continue;
+    }
     hasActive = true;
 
     // Load pipeline definition
     let pipeline;
     try {
       pipeline = JSON.parse(readFileSync(join(pipelinesDir, `${run.pipeline}.json`), 'utf-8'));
-    } catch { continue; }
+    } catch {
+      console.error(`[pipeline] Could not load pipeline definition for run ${runId}: ${run.pipeline}.json`);
+      continue;
+    }
 
-    const runId = file.replace('.json', '');
+    // 1. Consume and apply signal files
+    const signals = consumeSignals(cwd, runId);
     let modified = false;
 
+    if (signals.length > 0) {
+      modified = applySignals(run, signals, pipeline);
+      if (modified) {
+        // Durability: persist state BEFORE deleting signals
+        persistRun(runId, run);
+        deleteSignals(cwd, signals);
+      }
+    }
+
+    // 2. Process each step
     for (const stepDef of pipeline.steps) {
       const step = run.steps[stepDef.name];
       if (!step) continue;
@@ -455,7 +604,7 @@ function tickPipelines() {
     }
 
     if (modified) {
-      writeFileSync(join(runsDir, file), JSON.stringify(run, null, 2));
+      persistRun(runId, run);
     }
   }
 
@@ -516,9 +665,10 @@ function tick() {
   setTimeout(tick, nextTickMs);
 }
 
-// ─── Startup ────────────────────────────────────────────────
+// ─── Startup ────────────────────────────────────────────────────
 
 acquireLock();
+loadRunCache();
 
 process.on('SIGINT', () => { releaseLock(); process.exit(0); });
 process.on('SIGTERM', () => { releaseLock(); process.exit(0); });

package/src/scheduler/pipeline.js

@@ -12,6 +12,7 @@ import { join, dirname } from 'node:path';
 import { randomUUID } from 'node:crypto';
 import { ensureDaemon } from './scheduler.js';
 import { killGroup } from '../runner/process-manager.js';
+import { writeSignal } from './run-signals.js';
 
 const PIPELINES_DIR = '.agents/pipelines';
 const RUNS_DIR = '.agents/runs';
@@ -202,7 +203,7 @@ export function listRuns(cwd, pipelineId) {
   const dir = join(cwd, RUNS_DIR);
   if (!existsSync(dir)) return [];
   return readdirSync(dir)
-    .filter(f => f.endsWith('.json'))
+    .filter(f => f.endsWith('.json') && !f.includes('.signal-'))
     .map(f => {
       try { return JSON.parse(readFileSync(join(dir, f), 'utf-8')); }
       catch { return null; }
@@ -212,7 +213,8 @@ export function listRuns(cwd, pipelineId) {
 }
 
 /**
- * Cancel a pipeline run.
+ * Cancel a pipeline run. Writes a signal file for the daemon.
+ * Kills running processes immediately for responsiveness.
  * @param {string} cwd
  * @param {string} runId
  * @returns {boolean}
@@ -221,24 +223,22 @@ export function cancelRun(cwd, runId) {
   const run = getRun(cwd, runId);
   if (!run || run.status !== 'running') return false;
 
-  // Kill any running step
+  // Kill running processes immediately (side-effect safe)
   for (const [name, step] of Object.entries(run.steps)) {
     if (step.status === 'running') {
       const pidsToKill = [...(step.pids || [])];
       if (step.pid && !pidsToKill.includes(step.pid)) pidsToKill.push(step.pid);
-
       for (const pid of pidsToKill) {
         killGroup(pid);
       }
-      step.status = 'cancelled';
-    }
-    if (step.status === 'pending') {
-      step.status = 'skipped';
     }
   }
-  run.status = 'cancelled';
-  run.completedAt = new Date().toISOString();
-  saveRun(cwd, runId, run);
+
+  // Write signal file — daemon will apply the state change
+  writeSignal(cwd, runId, {
+    type: 'CANCEL_RUN',
+  });
+
   return true;
 }
 
@@ -254,7 +254,7 @@ export function findActiveRunByStep(cwd, stepName) {
   const dir = join(cwd, RUNS_DIR);
   if (!existsSync(dir)) return null;
 
-  for (const f of readdirSync(dir).filter(f => f.endsWith('.json'))) {
+  for (const f of readdirSync(dir).filter(f => f.endsWith('.json') && !f.includes('.signal-'))) {
     try {
       const run = JSON.parse(readFileSync(join(dir, f), 'utf-8'));
       if (run.status === 'running' && run.steps[stepName]) {
@@ -267,42 +267,42 @@ export function findActiveRunByStep(cwd, stepName) {
 
 /**
  * Signal step completion. Called by agent via MCP tool.
+ * Writes a signal file instead of mutating run state directly.
+ * The daemon will consume this signal on its next tick.
  * @param {string} cwd
  * @param {string} stepName
  * @param {string} [output]
  * @param {string} [runId] - Specific run ID (recommended)
- * @returns {{ success: boolean, nextStep?: string }}
+ * @returns {{ success: boolean }}
  */
 export function signalStepComplete(cwd, stepName, output, runId) {
-  let run, resolvedRunId;
+  let resolvedRunId = runId;
 
-  if (runId) {
-    // Direct lookup by run ID
-    run = getRun(cwd, runId);
-    resolvedRunId = runId;
-  } else {
+  if (!resolvedRunId) {
     // Fallback: search by step name
     const found = findActiveRunByStep(cwd, stepName);
     if (!found) return { success: false };
-    run = found.run;
     resolvedRunId = found.runId;
   }
 
+  // Verify run exists and is active
+  const run = getRun(cwd, resolvedRunId);
   if (!run || run.status !== 'running') return { success: false };
-  const step = run.steps[stepName];
-  if (!step || step.status !== 'running') return { success: false };
+  if (!run.steps[stepName] || run.steps[stepName].status !== 'running') return { success: false };
 
-  step.status = 'success';
-  step.signaled = true;
-  step.completedAt = new Date().toISOString();
-  if (output) step.output = output;
+  // Write signal file — daemon will apply it
+  writeSignal(cwd, resolvedRunId, {
+    type: 'STEP_COMPLETE',
+    stepName,
+    output: output || null,
+  });
 
-  saveRun(cwd, resolvedRunId, run);
   return { success: true };
 }
 
 /**
  * Bounce back to a previous step. Called by agent via MCP tool.
+ * Writes a signal file instead of mutating run state directly.
  * @param {string} cwd
  * @param {string} targetStepName - Step to re-run
  * @param {string} reason - Why bouncing back
@@ -310,63 +310,53 @@ export function signalStepComplete(cwd, stepName, output, runId) {
  * @returns {{ success: boolean, bounceCount?: number, maxBounces?: number }}
  */
 export function bounceBack(cwd, targetStepName, reason, runId) {
-  // Find active run where the caller is running
-  const dir = join(cwd, RUNS_DIR);
-  if (!existsSync(dir)) return { success: false };
+  // Find the active run containing this step
+  let resolvedRunId = runId;
+  let run;
 
-  for (const f of readdirSync(dir).filter(f => f.endsWith('.json'))) {
-    try {
-      const run = JSON.parse(readFileSync(join(dir, f), 'utf-8'));
-      if (run.status !== 'running') continue;
-
-      const targetStep = run.steps[targetStepName];
-      if (!targetStep) continue;
-
-      // Find the pipeline definition for maxBounces
-      const pipeline = getPipeline(run.cwd || cwd, run.pipeline);
-      const stepDef = pipeline?.steps.find(s => s.name === targetStepName);
-      const maxBounces = stepDef?.maxBounces ?? 2;
-
-      if (targetStep.bounces >= maxBounces) {
-        // Bounce limit reached — fail pipeline
-        targetStep.status = 'failed';
-        targetStep.lastBounceReason = `Bounce limit (${maxBounces}) reached. Last: ${reason}`;
-        run.status = 'failed';
-        run.completedAt = new Date().toISOString();
-        saveRun(cwd, f.replace('.json', ''), run);
-        return { success: false, bounceCount: targetStep.bounces, maxBounces };
-      }
+  if (resolvedRunId) {
+    run = getRun(cwd, resolvedRunId);
+  } else {
+    const dir = join(cwd, RUNS_DIR);
+    if (!existsSync(dir)) return { success: false };
+    for (const f of readdirSync(dir).filter(f => f.endsWith('.json') && !f.includes('.signal-'))) {
+      try {
+        const r = JSON.parse(readFileSync(join(dir, f), 'utf-8'));
+        if (r.status === 'running' && r.steps[targetStepName]) {
+          run = r;
+          resolvedRunId = f.replace('.json', '');
+          break;
+        }
+      } catch { /* skip */ }
+    }
+  }
 
-      // Reset target step to pending with bounce info
-      targetStep.status = 'bounce_pending';
-      targetStep.bounces += 1;
-      targetStep.lastBounceReason = reason;
+  if (!run || run.status !== 'running') return { success: false };
 
-      // Fail-fast: kill any remaining running agents for this step
-      const pidsToKill = [...(targetStep.pids || [])];
-      if (targetStep.pid && !pidsToKill.includes(targetStep.pid)) pidsToKill.push(targetStep.pid);
-      for (const pid of pidsToKill) {
-        killGroup(pid);
-      }
+  const targetStep = run.steps[targetStepName];
+  if (!targetStep) return { success: false };
 
-      targetStep.pid = null;
-      targetStep.pids = [];
-      targetStep.exitCode = null;
-      targetStep.signaled = false;
-
-      // Reset the calling step too
-      const callingStepName = Object.keys(run.steps).find(name => {
-        const s = run.steps[name];
-        return s.status === 'running';
-      });
-      if (callingStepName) {
-        run.steps[callingStepName].status = 'waiting_bounce';
-      }
+  // Check bounce limit (read-only check — safe without lock)
+  const pipeline = getPipeline(run.cwd || cwd, run.pipeline);
+  const stepDef = pipeline?.steps.find(s => s.name === targetStepName);
+  const maxBounces = stepDef?.maxBounces ?? 2;
 
-      saveRun(cwd, f.replace('.json', ''), run);
-      return { success: true, bounceCount: targetStep.bounces, maxBounces };
-    } catch { /* skip */ }
+  if (targetStep.bounces >= maxBounces) {
+    return { success: false, bounceCount: targetStep.bounces, maxBounces };
   }
 
-  return { success: false };
+  // Find the calling step name (the step that's bouncing back)
+  const callingStepName = Object.keys(run.steps).find(name =>
+    run.steps[name].status === 'running' && name !== targetStepName,
+  );
+
+  // Write signal file — daemon will apply the state changes and kill processes
+  writeSignal(cwd, resolvedRunId, {
+    type: 'BOUNCE_BACK',
+    stepName: targetStepName,
+    callingStepName: callingStepName || null,
+    reason,
+  });
+
+  return { success: true, bounceCount: targetStep.bounces + 1, maxBounces };
 }

package/src/scheduler/run-signals.js

@@ -0,0 +1,81 @@
+/**
+ * Run signal files — atomic communication between MCP server and daemon.
+ *
+ * Instead of MCP tools writing directly to run JSON (race condition),
+ * they write small signal files that the daemon consumes on each tick.
+ *
+ * Signal types: STEP_COMPLETE, BOUNCE_BACK
+ *
+ * @module agent-pool/scheduler/run-signals
+ */
+
+import { writeFileSync, readFileSync, readdirSync, unlinkSync, existsSync, mkdirSync } from 'node:fs';
+import { join } from 'node:path';
+import { randomUUID } from 'node:crypto';
+
+const RUNS_DIR = '.agents/runs';
+
+/**
+ * Write a signal file for a specific run.
+ * Signal files are atomic — no concurrent read-modify-write.
+ * @param {string} cwd
+ * @param {string} runId
+ * @param {object} signal - { type, stepName, output?, reason?, targetStep? }
+ */
+export function writeSignal(cwd, runId, signal) {
+  const dir = join(cwd, RUNS_DIR);
+  mkdirSync(dir, { recursive: true });
+
+  const id = randomUUID().split('-')[0];
+  const fileName = `${runId}.signal-${id}.json`;
+  const payload = {
+    ...signal,
+    timestamp: new Date().toISOString(),
+  };
+
+  writeFileSync(join(dir, fileName), JSON.stringify(payload));
+}
+
+/**
+ * Consume all pending signal files for a run.
+ * Returns signals sorted by timestamp. Does NOT delete them —
+ * caller must call deleteSignals() after persisting state.
+ * @param {string} cwd
+ * @param {string} runId
+ * @returns {Array<{ type: string, stepName: string, fileName: string, [key: string]: any }>}
+ */
+export function consumeSignals(cwd, runId) {
+  const dir = join(cwd, RUNS_DIR);
+  if (!existsSync(dir)) return [];
+
+  const prefix = `${runId}.signal-`;
+  const signalFiles = readdirSync(dir).filter(f => f.startsWith(prefix) && f.endsWith('.json'));
+
+  const signals = [];
+  for (const f of signalFiles) {
+    try {
+      const data = JSON.parse(readFileSync(join(dir, f), 'utf-8'));
+      signals.push({ ...data, fileName: f });
+    } catch {
+      // Include corrupted files so they get cleaned up by deleteSignals
+      signals.push({ type: '_corrupted', fileName: f });
+    }
+  }
+
+  // Sort by timestamp for deterministic processing
+  signals.sort((a, b) => (a.timestamp || '').localeCompare(b.timestamp || ''));
+  return signals;
+}
+
+/**
+ * Delete signal files after state has been persisted to disk.
+ * @param {string} cwd
+ * @param {Array<{ fileName: string }>} signals
+ */
+export function deleteSignals(cwd, signals) {
+  const dir = join(cwd, RUNS_DIR);
+  for (const s of signals) {
+    try { unlinkSync(join(dir, s.fileName)); }
+    catch { /* ignore */ }
+  }
+}

package/src/server.js

@@ -23,6 +23,7 @@ import { consultPeer } from './tools/consult.js';
 import { addSchedule, listSchedules, removeSchedule, getScheduledResults, getDaemonStatus } from './scheduler/scheduler.js';
 import { createPipeline, listPipelines, runPipeline, getRun, listRuns, cancelRun, signalStepComplete, bounceBack } from './scheduler/pipeline.js';
 import { createGroup, listGroups, getGroup } from './tools/groups.js';
+import { sendMessage, getMessages } from './tools/messaging.js';
 
 import { TOOL_DEFINITIONS } from './tool-definitions.js';
 
@@ -112,7 +113,7 @@ export function createServer() {
   }
 
   const server = new Server(
-    { name: 'agent-pool', version: '1.6.0' },
+    { name: 'agent-pool', version: '1.7.0' },
     { capabilities: { tools: {}, resources: {} } },
   );
 
@@ -208,6 +209,10 @@ export function createServer() {
           response = handleListGroups(args); break;
         case 'delegate_to_group':
           response = handleDelegateToGroup(args); break;
+        case 'send_message':
+          response = handleSendMessage(args); break;
+        case 'get_messages':
+          response = handleGetMessages(args); break;
         default:
           response = { content: [{ type: 'text', text: `Unknown tool: ${name}` }], isError: true };
       }
@@ -803,3 +808,58 @@ function handleDelegateToGroup(args) {
     }],
   };
 }
+
+// ─── Messaging handlers ─────────────────────────────────────
+
+function handleSendMessage(args) {
+  const cwd = args.cwd ?? defaultCwd;
+  const result = sendMessage(cwd, {
+    channel: args.channel,
+    payload: args.payload,
+    from: args.from,
+  });
+
+  if (!result.success) {
+    return {
+      content: [{ type: 'text', text: `❌ Failed to send message: ${result.error || 'unknown error'}` }],
+      isError: true,
+    };
+  }
+
+  return {
+    content: [{ type: 'text', text: `📨 Message sent to channel \`${result.channel}\`.` }],
+  };
+}
+
+function handleGetMessages(args) {
+  const cwd = args.cwd ?? defaultCwd;
+  const result = getMessages(cwd, {
+    channel: args.channel,
+    clear: args.clear,
+  });
+
+  if (result.error) {
+    return {
+      content: [{ type: 'text', text: `❌ ${result.error}` }],
+      isError: true,
+    };
+  }
+
+  if (result.count === 0) {
+    return {
+      content: [{ type: 'text', text: `📭 No messages on channel \`${args.channel}\`.` }],
+    };
+  }
+
+  const lines = result.messages.map((m, i) =>
+    `**${i + 1}.** [${m.timestamp}] from \`${m.from}\`:\n\`\`\`json\n${JSON.stringify(m.payload, null, 2)}\n\`\`\``
+  );
+
+  return {
+    content: [{
+      type: 'text',
+      text: `📬 **${result.count}** message(s) on channel \`${args.channel}\`${args.clear ? ' (cleared)' : ''}:\n\n${lines.join('\n\n')}`,
+    }],
+  };
+}
+

package/src/tool-definitions.js

@@ -408,5 +408,49 @@ export const TOOL_DEFINITIONS = [
       required: ['group', 'prompt'],
     },
   },
+  {
+    name: 'send_message',
+    description: [
+      'Send a message to a channel for inter-agent communication.',
+      'Use this to pass structured data between pipeline steps or between any agents.',
+      '',
+      'Channel conventions:',
+      '  - {run_id} — broadcast to all steps in a pipeline run',
+      '  - {run_id}:{step_name} — targeted to a specific step',
+      '  - any string — ad-hoc channel for custom messaging',
+      '',
+      'Messages are persisted to disk (survives restarts). Uses JSONL format for concurrent-write safety.',
+    ].join('\n'),
+    inputSchema: {
+      type: 'object',
+      properties: {
+        channel: { type: 'string', description: 'Target channel. Use run_id for broadcast, run_id:step_name for targeted.' },
+        payload: { description: 'Message payload (any JSON-serializable value).' },
+        from: { type: 'string', description: 'Sender identifier (e.g., step name or task description).' },
+        cwd: { type: 'string', description: 'Working directory. Defaults to current working directory.' },
+      },
+      required: ['channel', 'payload'],
+    },
+  },
+  {
+    name: 'get_messages',
+    description: [
+      'Read messages from a channel. Returns all messages in chronological order.',
+      '',
+      'Channel conventions:',
+      '  - {run_id} — read broadcast messages for a pipeline run',
+      '  - {run_id}:{step_name} — read messages targeted to a specific step',
+      '',
+      'Use clear=true to consume messages (delete after reading).',
+    ].join('\n'),
+    inputSchema: {
+      type: 'object',
+      properties: {
+        channel: { type: 'string', description: 'Channel to read messages from.' },
+        clear: { type: 'boolean', description: 'If true, clear the channel after reading (consume mode). Default: false.' },
+        cwd: { type: 'string', description: 'Working directory. Defaults to current working directory.' },
+      },
+      required: ['channel'],
+    },
+  },
 ];
-

package/src/tools/messaging.js

@@ -0,0 +1,104 @@
+/**
+ * Inter-agent messaging — file-based JSONL mailboxes.
+ *
+ * Provides send_message / get_messages tools for agents
+ * to pass structured data between pipeline steps or tasks.
+ *
+ * Uses JSONL format (one JSON object per line) with appendFileSync()
+ * to avoid read-modify-write race conditions on concurrent writes.
+ *
+ * Channel addressing:
+ *   - {run_id}           → broadcast to all steps in a pipeline run
+ *   - {run_id}:{step}    → targeted to a specific step
+ *   - {custom_channel}   → any string for ad-hoc messaging
+ *
+ * @module agent-pool/tools/messaging
+ */
+
+import { appendFileSync, readFileSync, writeFileSync, existsSync, mkdirSync, renameSync, unlinkSync } from 'node:fs';
+import { join, dirname } from 'node:path';
+
+const MESSAGES_DIR = '.agents/messages';
+
+/**
+ * Sanitize channel name for use as filename.
+ * @param {string} channel
+ * @returns {string}
+ */
+function sanitizeChannel(channel) {
+  return channel.replace(/[^a-zA-Z0-9_:-]/g, '_');
+}
+
+/**
+ * Send a message to a channel.
+ * Uses appendFileSync for atomic writes (no read-modify-write).
+ * @param {string} cwd
+ * @param {object} opts
+ * @param {string} opts.channel - Target channel (e.g., "run_id:step_name")
+ * @param {*} opts.payload - Message payload (any JSON-serializable value)
+ * @param {string} [opts.from] - Sender identifier
+ * @returns {{ success: boolean, channel: string }}
+ */
+export function sendMessage(cwd, { channel, payload, from }) {
+  if (!channel) return { success: false, error: 'channel is required' };
+
+  const dir = join(cwd, MESSAGES_DIR);
+  mkdirSync(dir, { recursive: true });
+
+  const filePath = join(dir, `${sanitizeChannel(channel)}.jsonl`);
+  const message = {
+    timestamp: new Date().toISOString(),
+    from: from || 'unknown',
+    payload,
+  };
+
+  // JSONL: one JSON object per line, appended atomically
+  appendFileSync(filePath, JSON.stringify(message) + '\n');
+
+  return { success: true, channel };
+}
+
+/**
+ * Get messages from a channel.
+ * @param {string} cwd
+ * @param {object} opts
+ * @param {string} opts.channel - Channel to read from
+ * @param {boolean} [opts.clear] - If true, clear the channel after reading
+ * @returns {{ messages: Array<{ timestamp: string, from: string, payload: any }>, count: number }}
+ */
+export function getMessages(cwd, { channel, clear }) {
+  if (!channel) return { messages: [], count: 0, error: 'channel is required' };
+
+  const filePath = join(cwd, MESSAGES_DIR, `${sanitizeChannel(channel)}.jsonl`);
+  if (!existsSync(filePath)) return { messages: [], count: 0 };
+
+  let content;
+  if (clear) {
+    // Atomic consume: rename file first, then read. Any new messages
+    // appended after rename go to a NEW file (no data loss).
+    const tmpPath = filePath + '.consuming';
+    try {
+      renameSync(filePath, tmpPath);
+      content = readFileSync(tmpPath, 'utf-8').trim();
+      unlinkSync(tmpPath);
+    } catch {
+      // File was deleted or renamed between check and read
+      return { messages: [], count: 0 };
+    }
+  } else {
+    try {
+      content = readFileSync(filePath, 'utf-8').trim();
+    } catch {
+      return { messages: [], count: 0 };
+    }
+  }
+
+  if (!content) return { messages: [], count: 0 };
+
+  const messages = content.split('\n').map(line => {
+    try { return JSON.parse(line); }
+    catch { return null; }
+  }).filter(Boolean);
+
+  return { messages, count: messages.length };
+}