@yemi33/squad 0.1.7 → 0.1.9

package/README.md

@@ -229,6 +229,23 @@ Agents need MCP tools to interact with your repo host (create PRs, post review c
 
 Manually refresh with `squad mcp-sync`.
 
+### Azure DevOps Users
+
+For the best experience with ADO repos, install the [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli) and use the [Azure DevOps MCP server](https://github.com/microsoft/azure-devops-mcp). This gives agents full access to PRs, work items, repos, and pipelines via MCP tools — no `gh` CLI needed.
+
+```bash
+# Install Azure CLI
+winget install Microsoft.AzureCLI   # Windows
+brew install azure-cli               # macOS
+curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash  # Linux
+
+# Login and set defaults
+az login
+az devops configure --defaults organization=https://dev.azure.com/YOUR_ORG project=YOUR_PROJECT
+```
+
+Then add the ADO MCP server to your Claude Code settings (`~/.claude.json`). The engine will auto-sync it to all agents on next start.
+
 ## Work Items
 
 All work items use the shared `playbooks/work-item.md` template, which provides consistent branch naming, worktree workflow, PR creation steps, and status tracking.

package/docs/self-improvement.md

@@ -24,23 +24,62 @@ Agent completes task
 ```
 Agent finishes task
   → writes notes/inbox/<agent>-<date>.md
-  → engine checks inbox on each tick
-  → at 5+ files: consolidateInbox() runs
-  → items categorized (reviews, feedback, learnings, other)
-  → summary appended to notes.md
+  → engine checks inbox on next tick (~60s)
+  → consolidateInbox() fires (threshold: 1 file)
+  → spawns Claude Haiku for LLM-powered summarization
+  → Haiku reads all inbox notes + existing notes.md
+  → produces deduplicated, categorized digest
+  → digest appended to notes.md
   → originals moved to notes/archive/
   → notes.md injected into every future agent prompt
 ```
 
-### Smart Consolidation
+### LLM-Powered Consolidation (Primary)
 
-The engine doesn't just dump files — it categorizes them:
-- **Reviews** — files containing "review" or PR references
-- **Feedback** — review feedback files for authors
-- **Learnings** — build summaries, explorations, implementation notes
-- **Other** — everything else
+The engine spawns Claude Haiku to read all inbox notes and produce a smart digest. Haiku is chosen for speed (~10s) and low cost while being capable enough for summarization.
 
-Each category gets a header with item count and one-line summaries.
+**What Haiku does:**
+- Reads full content of every inbox note (up to 8KB each)
+- Cross-references against existing notes.md to avoid repeating known knowledge
+- Extracts only actionable insights: patterns, conventions, gotchas, build results, architectural decisions
+- Deduplicates — if multiple agents report the same finding, merges into one entry
+- Groups by category: Patterns & Conventions, Build & Test Results, PR Review Findings, Bugs & Gotchas, Architecture Notes, Action Items
+- Attributes each insight to the source agent(s)
+- Writes a descriptive title summarizing what was learned
+
+**Example output:**
+```markdown
+### 2026-03-13: PR-4964594 Build & Lint Analysis
+**By:** Engine (LLM-consolidated)
+
+#### Build & Test Results
+- **Build and tests passing** for progression UI feature _(rebecca)_
+
+#### Bugs & Gotchas
+- **Barrel file violation**: index.ts under src/features/progression/ violates no-barrel-files rule — use concrete paths _(rebecca)_
+- **11 lint errors**: unnecessary conditionals and missing curly braces in progressionAtoms.ts and ProgressionCard.tsx _(rebecca)_
+
+#### Patterns & Conventions
+- **Jotai for local state**: appropriate for synchronous UI state in Bebop features _(rebecca)_
+
+_Processed 1 note, 4 insights extracted, 1 duplicate removed._
+```
+
+**Async execution:** The LLM call doesn't block the engine tick loop. A `_consolidationInFlight` flag prevents concurrent runs.
+
+**Timeout:** 3 minutes max. If Haiku doesn't respond, the engine kills the process and falls back to regex.
+
+### Regex Fallback Consolidation
+
+If the LLM call fails (timeout, error, malformed output), the engine falls back to pattern-matching extraction:
+
+- **Numbered items**: `1. **Bold key**: explanation` — captured via regex
+- **Bullet items**: `- **Bold key**: explanation` — captured via regex
+- **Important lines**: standalone sentences containing must/never/always/convention/pattern/gotcha
+- **Deduplication**: fingerprint-based (lowercased, stripped, 70% word overlap threshold)
+- **Cross-reference**: checks against existing notes.md to skip known insights
+
+Fallback entries are labeled `**By:** Engine (regex fallback)` so you can tell which method was used.
 
 ### Auto-Pruning
 
@@ -284,7 +323,9 @@ When a git merge or rebase produces conflicts in yarn.lock.
 
 | Setting | Default | What it controls |
 |---------|---------|-----------------|
-| `engine.inboxConsolidateThreshold` | 5 | Files needed before consolidation triggers |
+| `engine.inboxConsolidateThreshold` | 1 | Files needed before consolidation triggers |
+| Consolidation model | Haiku | LLM used for summarization (fast, cheap) |
+| Consolidation timeout | 3 min | Max time for LLM call before fallback |
 | notes.md max size | 50KB | Auto-prunes old sections above this |
 | Agent history entries | 20 | Max entries kept in history.md |
 | Metrics file | `engine/metrics.json` | Auto-created on first completion |

package/engine.js

@@ -1966,10 +1966,14 @@ function updateMetrics(agentId, dispatchItem, result) {
 
 // ─── Inbox Consolidation ────────────────────────────────────────────────────
 
+// Track in-flight LLM consolidation to prevent concurrent runs
+let _consolidationInFlight = false;
+
 function consolidateInbox(config) {
-  const threshold = config.engine?.inboxConsolidateThreshold || 5;
+  const threshold = config.engine?.inboxConsolidateThreshold || 1;
   const files = getInboxFiles();
   if (files.length < threshold) return;
+  if (_consolidationInFlight) return;
 
   log('info', `Consolidating ${files.length} inbox items into notes.md`);
 
@@ -1978,91 +1982,286 @@ function consolidateInbox(config) {
     content: safeRead(path.join(INBOX_DIR, f))
   }));
 
-  // Categorize by type based on filename patterns
-  const categories = {
-    reviews: [],
-    feedback: [],
-    learnings: [],
-    other: []
-  };
+  const existingNotes = getNotes() || '';
+  consolidateWithLLM(items, existingNotes, files, config);
+}
 
-  for (const item of items) {
-    const name = item.name.toLowerCase();
-    if (name.includes('review') || name.includes('pr-') || name.includes('pr4')) {
-      categories.reviews.push(item);
-    } else if (name.includes('feedback')) {
-      categories.feedback.push(item);
-    } else if (name.includes('build') || name.includes('explore') || name.includes('m0') || name.includes('m1')) {
-      categories.learnings.push(item);
-    } else {
-      categories.other.push(item);
+// ─── LLM-Powered Consolidation ──────────────────────────────────────────────
+// Spawns Claude (Haiku) to read all inbox notes + existing notes.md and produce
+// a smart, deduplicated, categorized digest. Falls back to regex if LLM fails.
+
+function consolidateWithLLM(items, existingNotes, files, config) {
+  _consolidationInFlight = true;
+
+  // Build the prompt with all inbox notes
+  const notesBlock = items.map(item =>
+    `<note file="${item.name}">\n${(item.content || '').slice(0, 8000)}\n</note>`
+  ).join('\n\n');
+
+  // Include tail of existing notes.md for dedup context
+  const existingTail = existingNotes.length > 2000
+    ? '...\n' + existingNotes.slice(-2000)
+    : existingNotes;
+
+  const prompt = `You are a knowledge manager for a software engineering squad. Your job is to consolidate agent notes into team memory.
+
+## Inbox Notes to Process
+
+${notesBlock}
+
+## Existing Team Notes (for deduplication — do NOT repeat what's already here)
+
+<existing_notes>
+${existingTail}
+</existing_notes>
+
+## Instructions
+
+Read every inbox note carefully. Produce a consolidated digest following these rules:
+
+1. **Extract actionable knowledge only**: patterns, conventions, gotchas, warnings, build results, architectural decisions, review findings. Skip boilerplate (dates, filenames, task IDs).
+
+2. **Deduplicate aggressively**: If an insight already exists in the existing team notes, skip it entirely. If multiple agents report the same finding, merge into one entry and credit all agents.
+
+3. **Write concisely**: Each insight should be 1-2 sentences max. Use **bold key** at the start of each bullet.
+
+4. **Group by category**: Use these exact headers (only include categories that have content):
+   - \`#### Patterns & Conventions\`
+   - \`#### Build & Test Results\`
+   - \`#### PR Review Findings\`
+   - \`#### Bugs & Gotchas\`
+   - \`#### Architecture Notes\`
+   - \`#### Action Items\`
+
+5. **Attribute sources**: End each bullet with _(agentName)_ or _(agent1, agent2)_ if multiple.
+
+6. **Write a descriptive title**: First line must be a single-line title summarizing what was learned. Do NOT use generic text like "Consolidated from N items".
+
+## Output Format
+
+Respond with ONLY the markdown below — no preamble, no explanation, no code fences:
+
+### YYYY-MM-DD: <descriptive title>
+**By:** Engine (LLM-consolidated)
+
+#### Category Name
+- **Bold key**: insight text _(agent)_
+
+_Processed N notes, M insights extracted, K duplicates removed._
+
+Use today's date: ${dateStamp()}`;
+
+  // Write prompt to temp file
+  const promptPath = path.join(ENGINE_DIR, 'consolidate-prompt.md');
+  safeWrite(promptPath, prompt);
+
+  const sysPrompt = 'You are a concise knowledge manager. Output only markdown. No preamble. No code fences around your output.';
+  const sysPromptPath = path.join(ENGINE_DIR, 'consolidate-sysprompt.md');
+  safeWrite(sysPromptPath, sysPrompt);
+
+  // Build args — use Haiku for speed/cost, single turn, print mode
+  const spawnScript = path.join(ENGINE_DIR, 'spawn-agent.js');
+  const args = [
+    '--output-format', 'text',
+    '--max-turns', '1',
+    '--model', 'haiku',
+    '--permission-mode', 'bypassPermissions',
+    '--verbose',
+  ];
+
+  const childEnv = { ...process.env };
+  for (const key of Object.keys(childEnv)) {
+    if (key === 'CLAUDECODE' || key.startsWith('CLAUDE_CODE') || key.startsWith('CLAUDECODE_')) {
+      delete childEnv[key];
     }
   }
 
-  let entry = `\n\n---\n\n### ${dateStamp()}: Consolidated from ${items.length} inbox items\n`;
-  entry += `**By:** Engine (auto-consolidation)\n\n`;
+  log('info', 'Spawning Haiku for LLM consolidation...');
+
+  const proc = spawn(process.execPath, [spawnScript, promptPath, sysPromptPath, ...args], {
+    cwd: SQUAD_DIR,
+    stdio: ['pipe', 'pipe', 'pipe'],
+    env: childEnv
+  });
+
+  let stdout = '';
+  let stderr = '';
+  proc.stdout.on('data', d => { stdout += d.toString(); if (stdout.length > 100000) stdout = stdout.slice(-50000); });
+  proc.stderr.on('data', d => { stderr += d.toString(); if (stderr.length > 50000) stderr = stderr.slice(-25000); });
+
+  // Timeout: 3 minutes max
+  const timeout = setTimeout(() => {
+    log('warn', 'LLM consolidation timed out after 3m — killing and falling back to regex');
+    try { proc.kill('SIGTERM'); } catch {}
+  }, 180000);
+
+  proc.on('close', (code) => {
+    clearTimeout(timeout);
+    _consolidationInFlight = false;
+    try { fs.unlinkSync(promptPath); } catch {}
+    try { fs.unlinkSync(sysPromptPath); } catch {}
+
+    if (code === 0 && stdout.trim().length > 50) {
+      let digest = stdout.trim();
+      // Strip any code fences the model might add
+      digest = digest.replace(/^\`\`\`\w*\n?/gm, '').replace(/\n?\`\`\`$/gm, '').trim();
+
+      // Validate: must contain ### (our expected format)
+      if (!digest.startsWith('### ')) {
+        const sectionIdx = digest.indexOf('### ');
+        if (sectionIdx >= 0) {
+          digest = digest.slice(sectionIdx);
+        } else {
+          log('warn', 'LLM consolidation output missing expected format — falling back to regex');
+          consolidateWithRegex(items, files);
+          return;
+        }
+      }
+
+      const entry = '\n\n---\n\n' + digest;
+      const current = getNotes();
+      let newContent = current + entry;
+
+      // Prune if too large
+      if (newContent.length > 50000) {
+        const sections = newContent.split('\n---\n\n### ');
+        if (sections.length > 10) {
+          const header = sections[0];
+          const recent = sections.slice(-8);
+          newContent = header + '\n---\n\n### ' + recent.join('\n---\n\n### ');
+          log('info', `Pruned notes.md: removed ${sections.length - 9} old sections`);
+        }
+      }
 
-  if (categories.reviews.length > 0) {
-    entry += `#### Review Findings (${categories.reviews.length})\n`;
-    for (const item of categories.reviews) {
-      // Extract first meaningful line as summary
-      const firstLine = item.content.split('\n').find(l => l.trim() && !l.startsWith('#')) || item.name;
-      entry += `- **${item.name}**: ${firstLine.trim().slice(0, 150)}\n`;
+      safeWrite(NOTES_PATH, newContent);
+      archiveInboxFiles(files);
+      log('info', `LLM consolidation complete: ${files.length} notes processed by Haiku`);
+    } else {
+      log('warn', `LLM consolidation failed (code=${code}) — falling back to regex`);
+      if (stderr) log('debug', `LLM stderr: ${stderr.slice(0, 500)}`);
+      consolidateWithRegex(items, files);
     }
-    entry += '\n';
-  }
+  });
+
+  proc.on('error', (err) => {
+    clearTimeout(timeout);
+    _consolidationInFlight = false;
+    log('warn', `LLM consolidation spawn error: ${err.message} — falling back to regex`);
+    try { fs.unlinkSync(promptPath); } catch {}
+    try { fs.unlinkSync(sysPromptPath); } catch {}
+    consolidateWithRegex(items, files);
+  });
+}
+
+// ─── Regex Fallback Consolidation ────────────────────────────────────────────
 
-  if (categories.feedback.length > 0) {
-    entry += `#### Review Feedback (${categories.feedback.length})\n`;
-    for (const item of categories.feedback) {
-      const firstLine = item.content.split('\n').find(l => l.trim() && !l.startsWith('#')) || item.name;
-      entry += `- **${item.name}**: ${firstLine.trim().slice(0, 150)}\n`;
+function consolidateWithRegex(items, files) {
+  const allInsights = [];
+  for (const item of items) {
+    const content = item.content || '';
+    const agentMatch = item.name.match(/^(\w+)-/);
+    const agent = agentMatch ? agentMatch[1] : 'unknown';
+    const lines = content.split('\n');
+    const titleLine = lines.find(l => /^#\s/.test(l));
+    const noteTitle = titleLine ? titleLine.replace(/^#+\s*/, '').trim() : item.name;
+
+    const nameLower = item.name.toLowerCase();
+    const contentLower = content.toLowerCase();
+    let category = 'learnings';
+    if (nameLower.includes('review') || nameLower.includes('pr-') || nameLower.includes('pr4')) category = 'reviews';
+    else if (nameLower.includes('feedback')) category = 'feedback';
+    else if (nameLower.includes('build') || nameLower.includes('bt-')) category = 'build-results';
+    else if (nameLower.includes('explore')) category = 'exploration';
+    else if (contentLower.includes('bug') || contentLower.includes('fix')) category = 'bugs-fixes';
+
+    const numberedPattern = /^\d+\.\s+\*\*(.+?)\*\*\s*[\u2014\u2013:-]\s*(.+)/;
+    const bulletPattern = /^[-*]\s+\*\*(.+?)\*\*[:\s]+(.+)/;
+    const sectionPattern = /^###+\s+(.+)/;
+    const importantKeywords = /\b(must|never|always|convention|pattern|gotcha|warning|important|rule|tip|note that)\b/i;
+
+    for (const line of lines) {
+      const trimmed = line.trim();
+      if (!trimmed || sectionPattern.test(trimmed)) continue;
+      let insight = null;
+      const numMatch = trimmed.match(numberedPattern);
+      if (numMatch) insight = `**${numMatch[1].trim()}**: ${numMatch[2].trim()}`;
+      if (!insight) {
+        const bulMatch = trimmed.match(bulletPattern);
+        if (bulMatch) insight = `**${bulMatch[1].trim()}**: ${bulMatch[2].trim()}`;
+      }
+      if (!insight && importantKeywords.test(trimmed) && !trimmed.startsWith('#') && trimmed.length > 30 && trimmed.length < 500) {
+        insight = trimmed;
+      }
+      if (insight) {
+        if (insight.length > 300) insight = insight.slice(0, 297) + '...';
+        const fp = insight.toLowerCase().replace(/[^a-z0-9 ]/g, '').replace(/\s+/g, ' ').trim().slice(0, 80);
+        allInsights.push({ text: insight, source: item.name, noteTitle, category, agent, fingerprint: fp });
+      }
+    }
+    if (!allInsights.some(i => i.source === item.name)) {
+      allInsights.push({ text: `See full note: ${noteTitle}`, source: item.name, noteTitle, category, agent,
+        fingerprint: noteTitle.toLowerCase().replace(/[^a-z0-9 ]/g, '').replace(/\s+/g, ' ').trim().slice(0, 80) });
     }
-    entry += '\n';
   }
 
-  if (categories.learnings.length > 0) {
-    entry += `#### Learnings & Conventions (${categories.learnings.length})\n`;
-    for (const item of categories.learnings) {
-      const firstLine = item.content.split('\n').find(l => l.trim() && !l.startsWith('#')) || item.name;
-      entry += `- **${item.name}**: ${firstLine.trim().slice(0, 150)}\n`;
+  // Dedup
+  const existingNotes = (getNotes() || '').toLowerCase();
+  const seen = new Map();
+  const deduped = [];
+  for (const insight of allInsights) {
+    const fpWords = insight.fingerprint.split(' ').filter(w => w.length > 4).slice(0, 5);
+    if (fpWords.length >= 3 && fpWords.every(w => existingNotes.includes(w))) continue;
+    const existing = seen.get(insight.fingerprint);
+    if (existing) { if (!existing.sources.includes(insight.agent)) existing.sources.push(insight.agent); continue; }
+    let isDup = false;
+    for (const [fp, entry] of seen) {
+      const a = new Set(fp.split(' ')), b = new Set(insight.fingerprint.split(' '));
+      if ([...a].filter(w => b.has(w)).length / Math.max(a.size, b.size) > 0.7) {
+        if (!entry.sources.includes(insight.agent)) entry.sources.push(insight.agent); isDup = true; break;
+      }
     }
-    entry += '\n';
+    if (isDup) continue;
+    seen.set(insight.fingerprint, { insight, sources: [insight.agent] });
+    deduped.push({ ...insight, sources: seen.get(insight.fingerprint).sources });
   }
 
-  if (categories.other.length > 0) {
-    entry += `#### Other (${categories.other.length})\n`;
-    for (const item of categories.other) {
-      const firstLine = item.content.split('\n').find(l => l.trim() && !l.startsWith('#')) || item.name;
-      entry += `- **${item.name}**: ${firstLine.trim().slice(0, 150)}\n`;
+  const agents = [...new Set(items.map(i => { const m = i.name.match(/^(\w+)-/); return m ? m[1].charAt(0).toUpperCase() + m[1].slice(1) : 'Unknown'; }))];
+  const catLabels = { reviews: 'PR Review Findings', feedback: 'Review Feedback', 'build-results': 'Build & Test Results', exploration: 'Codebase Exploration', 'bugs-fixes': 'Bugs & Gotchas', learnings: 'Patterns & Conventions' };
+  const topicHints = [...new Set(deduped.map(i => i.category))].map(c => ({ reviews: 'PR reviews', feedback: 'review feedback', 'build-results': 'build/test results', exploration: 'codebase exploration', 'bugs-fixes': 'bug findings' }[c] || 'learnings'));
+  const title = `${agents.join(', ')}: ${topicHints.join(', ')} (${deduped.length} insights from ${items.length} notes)`;
+
+  const grouped = {};
+  for (const item of deduped) { if (!grouped[item.category]) grouped[item.category] = []; grouped[item.category].push(item); }
+
+  let entry = `\n\n---\n\n### ${dateStamp()}: ${title}\n`;
+  entry += '**By:** Engine (regex fallback)\n\n';
+  for (const [cat, catItems] of Object.entries(grouped)) {
+    entry += `#### ${catLabels[cat] || cat} (${catItems.length})\n`;
+    for (const item of catItems) {
+      const src = item.sources.length > 1 ? ` _(${item.sources.join(', ')})_` : ` _(${item.agent})_`;
+      entry += `- ${item.text}${src}\n`;
     }
     entry += '\n';
   }
+  const dupCount = allInsights.length - deduped.length;
+  if (dupCount > 0) entry += `_Deduplication: ${dupCount} duplicate(s) removed._\n`;
 
   const current = getNotes();
-
-  // Prune old consolidations if notes.md is getting too large (>50KB)
   let newContent = current + entry;
   if (newContent.length > 50000) {
     const sections = newContent.split('\n---\n\n### ');
-    if (sections.length > 10) {
-      // Keep header + last 8 consolidation sections
-      const header = sections[0];
-      const recent = sections.slice(-8);
-      newContent = header + '\n---\n\n### ' + recent.join('\n---\n\n### ');
-      log('info', `Pruned notes.md: removed ${sections.length - 9} old sections`);
-    }
+    if (sections.length > 10) { newContent = sections[0] + '\n---\n\n### ' + sections.slice(-8).join('\n---\n\n### '); }
   }
-
   safeWrite(NOTES_PATH, newContent);
+  archiveInboxFiles(files);
+  log('info', `Regex fallback: consolidated ${files.length} notes → ${deduped.length} insights into notes.md`);
+}
 
-  // Archive
+function archiveInboxFiles(files) {
   if (!fs.existsSync(ARCHIVE_DIR)) fs.mkdirSync(ARCHIVE_DIR, { recursive: true });
   for (const f of files) {
     try { fs.renameSync(path.join(INBOX_DIR, f), path.join(ARCHIVE_DIR, `${dateStamp()}-${f}`)); } catch {}
   }
-
-  log('info', `Consolidated ${files.length} items into notes.md (${Object.entries(categories).map(([k,v]) => `${v.length} ${k}`).join(', ')})`);
 }
 
 // ─── State Snapshot ─────────────────────────────────────────────────────────

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@yemi33/squad",
-  "version": "0.1.7",
+  "version": "0.1.9",
   "description": "Multi-agent AI dev team that runs from ~/.squad/ — five autonomous agents share a single engine, dashboard, and knowledge base",
   "bin": {
     "squad": "bin/squad.js"