compound-agent 1.5.0 → 1.6.1

package/CHANGELOG.md

@@ -9,6 +9,91 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [1.6.1] - 2026-03-05
+
+### Changed
+
+- **Renamed brainstorm phase to spec-dev**: The `/compound:brainstorm` slash command is now `/compound:spec-dev`. The phase focuses on structured specification development using EARS notation, Mermaid diagrams, and Socratic dialogue rather than open-ended brainstorming. Old `brainstorm.md` command files are auto-cleaned during upgrade.
+- **Integration test stability**: Reduced integration test parallelism (`maxForks: 1`) and increased timeouts to 60s to eliminate non-deterministic ETIMEDOUT failures under load.
+
+### Added
+
+- **Spec reference file**: `.claude/skills/compound/spec-dev/references/spec-guide.md` provides quick-reference material for EARS patterns, Mermaid diagram selection, NL ambiguity detection, and trade-off documentation frameworks. Installed automatically during `ca setup`.
+- **Hook error visibility**: Hook runners now log errors to stderr when `CA_DEBUG` environment variable is set, instead of silently swallowing all failures.
+- **check-plan stdin safety**: `ca check-plan` now enforces a 30-second timeout and 1MB size limit when reading from stdin, preventing hangs in CI/CD environments.
+- **Embed lock expiry**: Embedding lock files now expire after 1 hour as a safety valve against zombie processes holding locks indefinitely.
+- **Phase-state backward compatibility**: Legacy `lfg_active` field in phase state files is automatically migrated to `cookit_active` on read.
+- **clean-lessons scope messaging**: `ca clean-lessons` now reports when non-lesson items are excluded from analysis.
+
+### Fixed
+
+- **Missing spec-guide.md**: The reference file was declared in skill templates and CHANGELOG but never generated during setup. Now installed alongside phase skills.
+- **Upgrade cleanup for lfg.md**: Added `lfg.md` to deprecated commands list so `ca setup --update` removes stale lfg command files from upgraded repos.
+- **Docs template terminology**: WORKFLOW.md template now uses "Spec Dev" instead of "Brainstorm" for Phase 1.
+- **Test file naming**: Renamed `brainstorm-phase.test.ts` to `spec-dev-phase.test.ts` to match the refactored phase name.
+- **Library bundle cleanup**: Moved CLI-only re-exports (`registerWatchCommand`, `registerLoopCommands`) out of the library barrel to eliminate unused import warnings in `dist/index.js`.
+- **plan.test.ts embedding guard**: Added `skipIf(skipEmbedding)` to unguarded test that calls `retrieveForPlan` without mocking.
+- **Agent template test count**: Updated setup.test.ts to expect 9 agent templates (was 8), matching the actual AGENT_TEMPLATES count after `lessons-reviewer.md` was added.
+
+## [1.6.0] - 2026-03-02
+
+### Added
+
+- **`ca watch` command**: Live pretty-printer for infinity loop trace files. Tails `agent_logs/trace_*.jsonl` and formats stream-json events (tool calls, text deltas, token usage, epic markers) with colored output. Supports `--epic <id>` to watch a specific epic and `--no-follow` for one-shot reads.
+- **Stream-json micro logging**: Generated infinity loop scripts now use `--output-format stream-json --include-partial-messages` to capture structured JSONL event traces alongside the existing macro text log. Trace files written to `agent_logs/trace_<epic>-<ts>.jsonl` with `.latest` symlink for easy discovery.
+- **`/compound:learn-that` slash command**: Conversation-aware lesson capture with user confirmation before saving
+- **`/compound:check-that` slash command**: Search lessons and proactively apply them to current work
+- **Eager knowledge embedding**: Knowledge chunks from `docs/` are now embedded for semantic search when the model is available
+  - `ca index-docs --embed` embeds chunks after indexing
+  - `ca init` now downloads the embedding model (with `--skip-model` opt-out) and installs the post-commit hook
+  - Background embedding spawns after `ca init`/`ca setup` so users can start working immediately
+  - PID-based lock file prevents concurrent embedding processes
+  - Status file (`embed-status.json`) tracks background embedding progress
+- **New modules**: `embed-chunks.ts`, `embed-lock.ts`, `embed-status.ts`, `embed-background.ts` with full test coverage
+
+### Removed
+
+- **`/compound:learn` slash command**: Replaced by `/compound:learn-that` with conversation-aware capture and user confirmation
+- **`ca worktree` command family**: All five subcommands (`create`, `merge`, `list`, `cleanup`, `wire-deps`) removed. Claude Code now provides native `EnterWorktree` support. Running `ca worktree` prints a deprecation notice.
+- **`/compound:set-worktree` slash command**: Use Claude Code's native worktree workflow instead.
+- **Conditional Merge gate in `verify-gates`**: Only Review and Compound gates remain.
+- **`shortId` utility**: Dead code after worktree removal, cleaned up.
+
+### Changed
+
+- **Loop script uses piped stream splitting**: Claude invocation changed from `&>` capture to a `tee | extract_text` pipeline. Raw JSONL streams to trace file while extracted text feeds the macro log for marker detection. Backwards compatible — all existing markers (EPIC_COMPLETE, EPIC_FAILED, HUMAN_REQUIRED) still work.
+- **`ca setup --update` now cleans deprecated paths**: Automatically removes stale worktree skill/command files from `.claude/` and `.gemini/` directories.
+- **`ca setup` also cleans deprecated paths**: Fresh setup runs now remove stale files from prior versions.
+- **SKILLS.md template**: Command inventory now lists all 11 slash commands (was 7).
+
+### Fixed
+
+- **Eager embedding hardening** (production readiness fixes from triple review):
+  - **P0**: Background worker spawn now resolves `dist/cli.js` deterministically instead of relying on `npx ca` (which failed silently in dev/built contexts)
+  - **P0**: `embed-worker` command hidden from `ca --help` output
+  - **P1**: Stale lock recovery uses atomic delete-then-`wx` to prevent two processes both reclaiming
+  - **P1**: DB connection opened after lock acquisition to prevent leak on contention
+  - **P1**: `--embed` now throws when model unavailable (was silently returning 0)
+  - **P2**: Batch embedding (16 chunks per call) with per-batch SQLite transactions (was 1 fsync per row)
+  - **P2**: `EmbedStatus` rewritten as discriminated union; removed dead `chunksTotal` field
+  - **P2**: `readLock` validates JSON shape instead of blind `as` cast
+  - **P2**: Vector batch length assertion guards against short responses from embedding backend
+  - **P3**: Extracted `indexAndSpawnEmbed()` shared helper — `init.ts` and `all.ts` no longer duplicate logic
+  - **P3**: `ca setup` now prints feedback when background embedding spawns
+  - **P3**: `filesErrored` count shown in `ca index-docs` output
+  - **P3**: Barrel re-exports consolidated through `./memory/knowledge/index.js`
+- **`EPIC_ID_PATTERN` duplication**: `loop.ts` now uses distinctly named `LOOP_EPIC_ID_PATTERN` to avoid confusion with the canonical pattern in `cli-utils.ts`.
+- **Stale worktree lesson invalidated**: Memory item `Ld204372e` marked invalid to prevent irrelevant context injection.
+
+### Performance
+
+- **Eliminate double model initialization**: `ca search` now uses `isModelAvailable()` (fs.existsSync, zero cost) instead of `isModelUsable()` which loaded the 278MB native model just to probe availability, then loaded it again for actual embedding
+- **Bulk-read cached embeddings**: `getCachedEmbeddingsBulk()` replaces N individual `getCachedEmbedding()` SQLite queries with a single bulk read
+- **Eliminate redundant JSONL parsing**: `searchVector()` and `findSimilarLessons()` now use `readAllFromSqlite()` after `syncIfNeeded()` instead of re-parsing the JSONL file
+- **Float32Array consistency**: Lesson embedding path now keeps `Float32Array` from node-llama-cpp instead of converting via `Array.from()` (4x memory savings per vector)
+- **Pre-warm lesson embedding cache**: `ca init` now pre-computes embeddings for all lessons with missing or stale cache entries, eliminating cold-start latency on first search
+- **Graceful embedding fallback**: `ca search` falls back to keyword-only search on runtime embedding failures instead of crashing
+
 ## [1.5.0] - 2026-02-24
 
 ### Added
@@ -783,7 +868,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   - Vitest test suite
   - tsup build configuration
 
-[Unreleased]: https://github.com/Nathandela/compound-agent/compare/v1.4.4...HEAD
+[Unreleased]: https://github.com/Nathandela/compound-agent/compare/v1.5.0...HEAD
+[1.5.0]: https://github.com/Nathandela/compound-agent/compare/v1.4.4...v1.5.0
 [1.4.4]: https://github.com/Nathandela/compound-agent/compare/v1.4.3...v1.4.4
 [1.4.3]: https://github.com/Nathandela/compound-agent/compare/v1.4.2...v1.4.3
 [1.4.2]: https://github.com/Nathandela/compound-agent/compare/v1.4.1...v1.4.2

package/README.md

@@ -1,61 +1,73 @@
 # Compound Agent
 
-**Semantic memory for Claude Code -- capture mistakes once, never repeat them.**
+**Memory. Knowledge. Structure. Accountability. For AI coding agents.**
 
 [![npm version](https://img.shields.io/npm/v/compound-agent)](https://www.npmjs.com/package/compound-agent)
 [![license](https://img.shields.io/npm/l/compound-agent)](LICENSE)
 [![TypeScript](https://img.shields.io/badge/TypeScript-5.3+-blue)](https://www.typescriptlang.org/)
 
-## Overview
+- **Memory** -- capture mistakes once, surface them forever
+- **Knowledge** -- hybrid vector search over your project docs
+- **Structure** -- 5-phase workflows with 35+ specialized agents
+- **Accountability** -- git-tracked issues, multi-agent reviews, quality gates
+
+Fully local. Fully offline. Everything in git.
 
-Claude Code forgets everything between sessions. Compound Agent fixes this with a three-layer system: issue tracking (Beads) at the foundation, semantic memory with vector search in the middle, and structured workflow phases on top. It captures knowledge from corrections, discoveries, and completed work, then retrieves it precisely when relevant -- at session start, during planning, and before architectural decisions. Every cycle through the loop makes subsequent cycles smarter.
+## Overview
 
-## The Compound Loop
+AI coding agents forget everything between sessions. Compound Agent fixes this with a three-layer system: issue tracking at the foundation, semantic memory with vector search in the middle, and structured workflows with multi-agent review on top. It captures knowledge from corrections, discoveries, and completed work, then retrieves it precisely when relevant. Every cycle through the loop makes subsequent cycles smarter.
 
 ```mermaid
 graph LR
-    B[BRAINSTORM] --> P[PLAN]
+    B[SPEC DEV] --> P[PLAN]
     P --> W[WORK]
     W --> R[REVIEW]
     R --> C[COMPOUND]
     C --> M[(MEMORY)]
     M --> P
-```
-
-A bug found in review becomes a lesson. That lesson surfaces during planning of similar work. The plan accounts for the known issue. Work avoids the mistake.
 
-## Architecture
+    style M fill:#f9f,stroke:#333
+```
 
 ```mermaid
 block-beta
     columns 1
-    block:L3["Layer 3: Workflows"]
-        A["Slash commands"] B["Agent teams"] C["5-phase cycle"]
+    block:L3["Workflows"]
+        A["5-phase cycle"] B["35+ specialized agents"] C["Multi-model review"]
     end
-    block:L2["Layer 2: Semantic Memory"]
-        D["JSONL source of truth"] E["SQLite FTS5 index"] F["Vector embeddings"]
+    block:L2["Semantic Memory"]
+        D["Vector search"] E["Hybrid retrieval"] F["Cross-cutting patterns"]
     end
-    block:L1["Layer 1: Beads"]
-        G["Issue tracking"] H["Git-backed sync"] I["Dependency graph"]
+    block:L1["Foundation"]
+        G["Issue tracking"] H["Git-backed sync"] I["Quality gates"]
     end
 
     L3 --> L2
     L2 --> L1
 ```
 
-Four memory types -- `lesson`, `solution`, `pattern`, `preference` -- share one store, one schema, and one ranked retrieval mechanism combining vector similarity, severity, recency, and confirmation status.
+## Is this for you?
+
+**"It keeps making the same mistake every session."**
+Capture it once. Compound Agent surfaces it automatically before the agent repeats it.
+
+**"I explained our auth pattern three sessions ago. Now it's reimplementing from scratch."**
+Architectural decisions persist as searchable lessons. Next session, they inject into context before planning starts.
+
+**"My agent uses pandas when we standardized on Polars months ago."**
+Preferences survive across sessions and projects. Once captured, they appear at the right moment.
+
+**"Code reviews keep catching the same class of bugs."**
+35+ specialized review agents (security, performance, architecture, test coverage) run in parallel. Findings feed back as lessons that become test requirements in future work.
+
+**"I have no idea what my agent actually learned or if it's reliable."**
+`ca list` shows all captured knowledge. `ca stats` shows health. `ca wrong <id>` invalidates bad lessons. Everything is git-tracked JSONL -- you can read, diff, and audit it.
 
-## Why Not Just X?
+**"I want structured phases, not just 'go build this'."**
+Five workflow phases (spec-dev, plan, work, review, compound) with mandatory gates between them. Each phase searches memory and docs for relevant context before starting.
 
-| Feature | `.claude/CLAUDE.md` | Claude Reflect | mem0 | Compound Agent |
-|---------|---------------------|----------------|------|----------------|
-| Persists across sessions | Manual edits | Yes | Yes | Yes |
-| Semantic search | No | No (regex) | Yes (cloud) | Yes (local) |
-| Quality gate on capture | No | No | No | Yes (novelty + specificity) |
-| Runs fully offline | Yes | Yes | No (API) | Yes |
-| Git-tracked knowledge | Yes | No | No | Yes (JSONL) |
-| Structured workflow phases | No | No | No | Yes (5 phases) |
-| Claude Code native integration | N/A | Yes | No | Yes (hooks + commands) |
+**"My agent doesn't read the project docs before making decisions."**
+`ca knowledge "auth flow"` runs hybrid search (vector + keyword) over your indexed docs. Agents query it automatically during planning -- ADRs, specs, and standards surface before code gets written.
 
 ## Installation
 
@@ -97,7 +109,7 @@ Then run `pnpm install`.
 The five-phase workflow:
 
 ```
-1. /compound:brainstorm  -->  Explore the problem, clarify scope
+1. /compound:spec-dev    -->  Develop precise specifications
 2. /compound:plan        -->  Create tasks enriched by memory search
 3. /compound:work        -->  Execute with agent teams + TDD
 4. /compound:review      -->  Multi-agent review with inter-communication
@@ -107,7 +119,7 @@ The five-phase workflow:
 Or run all phases sequentially:
 
 ```
-/compound:lfg "Add auth to API"
+/compound:cook-it "Add auth to API"
 ```
 
 Each phase searches memory for relevant past knowledge and injects it into agent context. The compound phase captures new knowledge, closing the loop.
@@ -120,9 +132,9 @@ The CLI binary is `ca` (alias: `compound-agent`).
 
 | Command | Description |
 |---------|-------------|
-| `ca learn "<insight>"` | Capture a memory item manually |
+| `ca learn "<insight>"` | Capture a lesson manually |
 | `ca learn "<insight>" --trigger "<context>"` | Capture with trigger context |
-| `ca learn "<insight>" --severity high` | Set severity level |
+| `ca learn "<insight>" --severity high` | Set severity (low/medium/high) |
 | `ca learn "<insight>" --citation src/api.ts:42` | Attach file provenance |
 | `ca capture --input <file>` | Capture from structured input file |
 | `ca detect --input <file>` | Detect correction patterns in input |
@@ -154,21 +166,11 @@ The CLI binary is `ca` (alias: `compound-agent`).
 | `ca import <file>` | Import items from JSONL file |
 | `ca prime` | Load workflow context (used by hooks) |
 | `ca verify-gates <epic-id>` | Verify review + compound tasks exist and are closed |
-| `ca phase-check` | Manage LFG phase state (init/status/clean/gate) |
+| `ca phase-check` | Manage cook-it phase state (init/status/clean/gate) |
 | `ca audit` | Run audit checks against the codebase |
 | `ca rules check` | Run repository-defined rule checks |
 | `ca test-summary` | Run tests and output a compact summary |
 
-### Worktree
-
-| Command | Description |
-|---------|-------------|
-| `ca worktree create <epic-id>` | Create isolated worktree for an epic |
-| `ca worktree wire-deps <epic-id>` | Wire Review/Compound as merge blockers |
-| `ca worktree merge <epic-id>` | Two-phase merge back to main |
-| `ca worktree list` | List active worktrees with status |
-| `ca worktree cleanup <epic-id>` | Remove worktree and clean up (--force for dirty) |
-
 ### Automation
 
 | Command | Description |
@@ -179,6 +181,13 @@ The CLI binary is `ca` (alias: `compound-agent`).
 | `ca loop --max-retries <n>` | Max retries per epic on failure (default: 1) |
 | `ca loop --force` | Overwrite existing script |
 
+### Knowledge
+
+| Command | Description |
+|---------|-------------|
+| `ca knowledge "<query>"` | Hybrid search over indexed project docs |
+| `ca index-docs` | Index docs/ directory into knowledge base |
+
 ### Setup
 
 | Command | Description |
@@ -219,7 +228,7 @@ confirmation_boost: confirmed=1.3, unconfirmed=1.0
 ## FAQ
 
 **Q: How is this different from mem0?**
-A: mem0 is a cloud memory layer for general AI agents. Compound Agent is local-first, designed specifically for Claude Code, with git-tracked storage and local embeddings -- no API keys or cloud services needed.
+A: mem0 is a cloud memory layer for general AI agents. Compound Agent is local-first with git-tracked storage and local embeddings -- no API keys or cloud services needed. It also goes beyond memory with structured workflows, multi-agent review, and issue tracking.
 
 **Q: Does this work offline?**
 A: Yes, completely. Embeddings run locally via node-llama-cpp. No network requests after the initial model download.
@@ -228,10 +237,10 @@ A: Yes, completely. Embeddings run locally via node-llama-cpp. No network reques
 A: ~278MB for the embedding model (one-time download, shared across projects) plus negligible space for lessons.
 
 **Q: Can I use it with other AI coding tools?**
-A: The CLI (`ca`) works standalone, but hooks and slash commands are Claude Code specific. The TypeScript API can be integrated into other tools.
+A: The CLI (`ca`) works standalone with any tool. Full hook integration is available for Claude Code and Gemini CLI. The TypeScript API can be integrated into other tools.
 
 **Q: What happens if the embedding model isn't available?**
-A: Compound Agent hard-fails rather than silently degrading. Run `npx ca doctor` to diagnose issues.
+A: Search gracefully falls back to keyword-only mode. Other commands that require embeddings will tell you what's missing. Run `npx ca doctor` to diagnose issues.
 
 ## Development
 
@@ -285,6 +294,10 @@ Compound Agent builds on ideas and patterns from these projects:
 
 Also informed by research into [Reflexion](https://arxiv.org/abs/2303.11366) (verbal reinforcement learning), [Voyager](https://github.com/MineDojo/Voyager) (executable skill libraries), and production systems from mem0, Letta, and GitHub Copilot Memory.
 
+## Contributing
+
+Bug reports and feature requests are welcome via [Issues](https://github.com/Nathandela/compound-agent/issues). Pull requests are not accepted at this time -- see [CONTRIBUTING.md](CONTRIBUTING.md) for details.
+
 ## License
 
 MIT -- see [LICENSE](LICENSE) for details.