livepilot 1.21.3 → 1.22.0

package/CHANGELOG.md

@@ -1,5 +1,215 @@
 # Changelog
 
+## 1.22.0 — User atlas separation: ~/.livepilot/atlas/ (April 25 2026)
+
+First v1.22 release. Splits the device atlas into two files that serve
+different roles — a long-standing ambiguity that finally bit hard enough
+to be worth fixing at minor-version scope.
+
+### What changed
+
+**New: `~/.livepilot/atlas/device_atlas.json`** — the **user atlas**.
+Written by `scan_full_library` on your machine. Reflects YOUR installed
+packs, User Library, and plugins. Lives in the user-data directory
+(same convention as `~/.livepilot/memory/`) so `npm install livepilot`
+upgrades can't blow it away.
+
+**Unchanged: `mcp_server/atlas/device_atlas.json`** — the **bundled
+baseline**. Ships with the package (still 5264 devices, 120 enriched
+— stock Ableton 12 Suite inventory). Gives fresh installs a functional
+atlas before any personalized scan has run.
+
+**Resolution** at load time: user atlas wins if present, else bundled
+baseline falls through. Writes **always** go to the user path; the
+bundled path is read-only from the scanner's perspective.
+
+### Why this matters
+
+Prior to v1.22.0, the single `mcp_server/atlas/device_atlas.json` file
+served three conflicting roles — repo seed, personal scan cache, and
+runtime index. Three bugs resulted:
+
+1. **`npm install livepilot` wiped personal scans.** Every package
+   update overwrote the installed atlas with the bundled baseline. Users
+   who had carefully scanned their library (30-90 seconds per scan) lost
+   that work on every upgrade with no warning.
+2. **Dev installs polluted the repo.** Contributors running
+   LivePilot from a git checkout would scan their library (to test atlas
+   tools against live Ableton) and accidentally commit their personal
+   scan — pack names, user-library previews — to the public repo. This
+   happened once in the v1.21.x cycle and was the proximate trigger for
+   v1.22.
+3. **The "enriched" count ambiguity.** A single atlas file couldn't
+   honestly answer "how many devices are enriched?" — the right number
+   differed depending on whether you meant the bundled baseline (87 per
+   the scanner's last truncated pass), YAML files authored on disk (120),
+   or runtime coverage including native/M4L duplicates (137+ on a
+   fully-scanned install). The user/bundled split clarifies this by
+   letting each file carry its own honest count.
+
+### Migration for existing users
+
+Users with a personalized scan in `mcp_server/atlas/device_atlas.json`
+(most dev-install contributors) can migrate in one command:
+
+```bash
+mkdir -p ~/.livepilot/atlas
+cp "$(python3 -c 'import mcp_server.atlas; print(mcp_server.atlas.BUNDLED_ATLAS_PATH)')" \
+   ~/.livepilot/atlas/device_atlas.json
+# Then optionally restore the bundled baseline to its shipped state:
+git -C $(python3 -c 'import mcp_server.atlas, pathlib; print(pathlib.Path(mcp_server.atlas.__file__).parents[2])') \
+    checkout mcp_server/atlas/device_atlas.json
+```
+
+Users on the npm-installed path (no personalized scan yet) need nothing
+— the next `scan_full_library` call will write to the new user path
+automatically.
+
+### Code changes
+
+- `mcp_server/atlas/__init__.py` — new module-level constants
+  `BUNDLED_ATLAS_PATH`, `USER_ATLAS_DIR`, `USER_ATLAS_PATH`, and
+  function `_resolve_atlas_path()`. Existing `ATLAS_PATH` kept as a
+  backward-compat alias pointing at the resolved value.
+- `mcp_server/atlas/tools.py::scan_full_library` — writes to
+  `USER_ATLAS_PATH` and creates the user-data directory on demand.
+  Enrichments still read from the bundled package (they're authored
+  in-repo under `mcp_server/atlas/enrichments/`).
+
+### Tests
+
+8 new TDD regression tests in `tests/test_atlas_user_override.py`:
+
+- `test_bundled_path_is_in_package_dir`
+- `test_user_path_is_in_home_dir`
+- `test_resolver_returns_user_path_when_present`
+- `test_resolver_falls_back_to_bundled_when_user_atlas_missing`
+- `test_atlas_manager_loads_from_user_path_when_present`
+- `test_atlas_manager_falls_back_to_bundled_when_user_atlas_missing`
+- `test_scan_full_library_writes_to_user_path`
+- `test_user_atlas_dir_created_if_missing`
+
+Full suite: 3136 pass (3128 prior + 8 new), 1 skipped.
+
+### Documentation
+
+- `docs/manual/getting-started.md` — new "Step 5 (optional): Personalize
+  the device atlas" section.
+- `docs/manual/dev-install.md` — new "3a. User atlas vs bundled atlas"
+  subsection, critical for contributors.
+- `docs/manual/device-atlas.md` — header callout on the two-file split.
+- `CLAUDE.md` + `README.md` — short pointers to the new resolver.
+
+### Carried to future releases
+
+The atlas schema-canonicalization originally deferred from the v1.21.2
+audit is now **partially closed**: the user/bundled split makes the
+"which number counts as enriched?" question honest per-file, but the
+sync_metadata gate for `stats.enriched_devices` vs YAML file count
+hasn't been added. Deferred to a future patch because it's orthogonal
+to the user-atlas split.
+
+## 1.21.4 — v1.21.2 audit carry-over: slashed-compound filler + dev-install runbook (April 25 2026)
+
+Ships two items the v1.21.2 audit #2 deferred to v1.22 but that turned
+out to be safer to close in a patch cycle than to carry forward:
+
+1. **`check_prose_claim` regex widen for slashed/chained compound fillers** —
+   `"38 spectral/analyzer tools"` drift across 3 docs silently passed
+   `sync_metadata --check` because the filler group required trailing
+   whitespace. Now caught.
+2. **Dev-install runbook** (`docs/manual/dev-install.md`) — documented
+   path for contributors to run LivePilot from a local checkout without
+   publishing to npm. Previously every contributor had to reverse-engineer
+   this from `bin/livepilot.js` + `mcp_server/__main__.py`.
+
+No API changes. No user-observable behavior change at runtime. Four new
+TDD regression tests for the regex widening; ~180 lines of new
+documentation. `sync_metadata --check` now catches an entire class of
+compound-form drift that would otherwise have shipped untracked.
+
+### 1. `sync_metadata` slashed/chained compound filler
+
+All 4 regex patterns in `scripts/sync_metadata.py` (`check_tool_count`,
+`check_prose_claim`, `fix_tool_count`, `_fix_count`) had an optional
+filler group that required trailing whitespace —
+`[A-Za-z]+\s+` — and matched at most once (`?` quantifier). Slashed
+compounds like `"spectral/analyzer"` were rejected because `/` is not
+whitespace; chained compounds like `"spectral/MCP tools"` (two filler
+segments) were rejected because of the single-match quantifier.
+
+v1.21.4 widens the filler to two branches, iterable 0+ times:
+
+| Branch | Pattern | Matches |
+|---|---|---|
+| (a) Space-joined, uppercase-anchored | `[A-Z][A-Za-z\-]*\s+` | `"MCP "` in `"430 MCP Tools"` |
+| (b) Slash-joined, any-case | `[A-Za-z][A-Za-z\-]*/` | `"spectral/"` in `"38 spectral/analyzer tools"` |
+
+The uppercase anchor on branch (a) is preserved deliberately — without
+it, lowercase English articles ("the tool") would false-positive on
+prose like "released in 2020 the tool was first previewed." The slash
+on branch (b) is itself an unambiguous compound marker that doesn't
+appear in normal English, so any-case there is safe. The `*` quantifier
+lets branches chain: `"300 spectral/MCP tools"` now parses as
+`(b)"spectral/"` + `(a)"MCP "` + `"tools"`.
+
+For `check_prose_claim` + its paired `_fix_count`, the filler is a
+simpler single-branch `[A-Za-z][A-Za-z\-]*(?:/|\s+)` — these don't need
+the uppercase anchor (prose-claim nouns predate the v1.21.4 cycle and
+already allowed lowercase-start fillers).
+
+**Concrete effect**: `sync_metadata --check` now flags drift in all 3
+real surfaces that previously escaped:
+
+- `README.md` — `"38 spectral/analyzer tools"` × 2
+- `docs/manual/getting-started.md` — `"38 spectral/analyzer tools"` × 1
+- `livepilot/skills/livepilot-release/SKILL.md` — `"38 spectral/analyzer tools"` × 1
+
+(Today these say "38" and the count is 38, so no drift reported — but
+the next time the analyzer module gains or loses a `@mcp.tool`, CI will
+fail loudly instead of silently drifting.)
+
+Tests added to `tests/test_claim_consistency.py` (4):
+
+- `test_prose_claim_catches_slashed_compound` — drift detection
+- `test_fix_count_rewrites_slashed_compound` — `--fix` preserves the adjective
+- `test_tool_count_catches_slashed_compound` — chained filler (`"300 spectral/MCP tools"`)
+- `test_tool_count_no_false_positive_on_year_prose` — uppercase anchor guard
+
+### 2. Dev-install runbook
+
+New file `docs/manual/dev-install.md` (~180 lines) documents the
+bare-python local-checkout workflow for contributors:
+
+1. **venv setup** — `python3 -m venv .venv && source .venv/bin/activate && pip install -r requirements.txt`
+2. **Local Remote Script install** — `node bin/livepilot.js --install` (NOT `npx livepilot --install`, which resolves to the registry and silently discards local edits)
+3. **MCP client config pointing at `python -m mcp_server`** — per-client examples for Claude Desktop, Claude Code, Cursor/VS Code
+4. **Iterate loop** — client restart for `mcp_server/` edits; `node bin/livepilot.js --install` + `reload_handlers` tool for `remote_script/` edits
+5. **Test suite** — `python -m pytest tests/ -q` (3128 tests as of v1.21.4)
+6. **`sync_metadata` drift check** — `python scripts/sync_metadata.py --check` / `--fix`
+7. **Going back to published** — remove the dev MCP entry or stop using it
+
+Plus a troubleshooting section for the 4 most common first-run failures:
+`ModuleNotFoundError: No module named 'mcp_server'`, `Another client is
+already connected` on port 9878, stale `__pycache__` shadowing edits, and
+module-level constants not reloading via `reload_handlers` (full Ableton
+restart required).
+
+Cross-links added:
+
+- `docs/manual/getting-started.md` — callout at top routing contributors to dev-install before they commit to the npm path
+- `docs/manual/index.md` — new "Contributing" section in the Chapters TOC
+- `CLAUDE.md` — pointer in the Project section
+
+### Carried forward to v1.22
+
+Still deferred from the v1.21.2 audit:
+
+- Atlas `.enriched` schema canonicalization (`stats.enriched_devices = 87`
+  vs 120 YAML files vs 135 historical claim — pick one and derive the
+  others at build time; add a sync_metadata gate matching JSON stats to
+  YAML file count)
+
 ## 1.21.3 — Third audit-response: manual-docs drift + sync_metadata file-list widening + unicode-escape regex fix (April 24 2026)
 
 Fourth same-day patch in 48 hours, third audit-response in 24 hours.

package/README.md

@@ -105,7 +105,7 @@ Most MCP servers are tool collections — they execute commands. LivePilot is an
 
 **M4L Bridge** (`m4l_device/`) — Optional Max for Live Audio Effect on the master track. Provides deep LOM access through Max's LiveAPI that the ControlSurface API can't reach. UDP 9880 (M4L to server) carries spectral data and LiveAPI responses. OSC 9881 (server to M4L) sends commands. The 38 spectral/analyzer tools strictly require the bridge; device and sample tools that call the bridge also have graceful fallbacks, so core functionality works without it. Backed by 31 bridge commands for hidden parameters, Simpler internals, warp markers, display values, and Simpler warp / Compressor sidechain writes that live on child objects Python can't reach.
 
-**Device Atlas** (`mcp_server/atlas/`) — In-memory indexed JSON database. 5264 devices with browser URIs, 120 enriched with YAML sonic intelligence profiles (mood, genre, texture, recommended chains). 7 indexes: by_id, by_name, by_uri, by_category, by_tag, by_genre, by_pack. Reverse-index `device_techniques_index.json` powers `atlas_techniques_for_device` (146 cross-references across 58 devices). The AI never hallucinates a device name or preset — it always resolves against the atlas first.
+**Device Atlas** (`mcp_server/atlas/`) — In-memory indexed JSON database. 5264 devices with browser URIs (bundled baseline), 120 enriched with YAML sonic intelligence profiles (mood, genre, texture, recommended chains). 7 indexes: by_id, by_name, by_uri, by_category, by_tag, by_genre, by_pack. Reverse-index `device_techniques_index.json` powers `atlas_techniques_for_device` (146 cross-references across 58 devices). The AI never hallucinates a device name or preset — it always resolves against the atlas first. **v1.22.0+**: run `scan_full_library` after install to index YOUR packs + User Library + plugins into `~/.livepilot/atlas/device_atlas.json` — your personal atlas overrides the baseline and survives npm updates.
 
 **Sample Engine** (`mcp_server/sample_engine/`) — Searches three sources simultaneously: BrowserSource (Ableton's library), SpliceSource (local Splice catalog via SQLite), FilesystemSource (user directories). Every result passes through a 6-critic fitness battery (key, tempo, spectral, genre, mood, technical). 29 processing techniques (Surgeon precision vs. Alchemist experimentation). Builds complete sample processing plans with warp, slice, and effect recommendations.
 

package/mcp_server/__init__.py

@@ -1,2 +1,2 @@
 """LivePilot MCP Server — bridges MCP protocol to Ableton Live."""
-__version__ = "1.21.3"
+__version__ = "1.22.0"

package/mcp_server/atlas/__init__.py

@@ -519,28 +519,75 @@ class AtlasManager:
 from pathlib import Path
 from ..services.singletons import Singleton
 
-ATLAS_PATH = Path(__file__).parent / "device_atlas.json"
+# v1.22.0: the atlas now has TWO possible homes —
+#
+#   BUNDLED_ATLAS_PATH — mcp_server/atlas/device_atlas.json
+#     Ships with the package. Gives fresh installs a functional
+#     baseline device index before any personalized scan has run.
+#     Updated only when the repo's canonical baseline is regenerated.
+#
+#   USER_ATLAS_PATH — ~/.livepilot/atlas/device_atlas.json
+#     Written by scan_full_library on the user's machine. Reflects
+#     their actual installed packs, User Library, and plugins. Lives
+#     in the user-data directory (same convention as
+#     ~/.livepilot/memory/) so npm updates can't blow it away.
+#
+# Resolution order at load time: user atlas wins if present, else
+# bundled baseline. Written scans ALWAYS go to the user path — never
+# the bundled path — so the repo/npm package stays clean regardless of
+# where a user runs scan_full_library from. This split lands in v1.22.0
+# and solves three prior issues (see tests/test_atlas_user_override.py
+# for the full rationale).
+BUNDLED_ATLAS_PATH = Path(__file__).parent / "device_atlas.json"
+USER_ATLAS_DIR = Path.home() / ".livepilot" / "atlas"
+USER_ATLAS_PATH = USER_ATLAS_DIR / "device_atlas.json"
+
+
+def _resolve_atlas_path() -> Path:
+    """Return the effective atlas path — user if present, bundled if not.
+
+    Called from _build_atlas and get_atlas. Kept as a module-level
+    function (rather than inlined) so tests can monkeypatch HOME and
+    reimport the module to re-evaluate USER_ATLAS_PATH cleanly.
+    """
+    if USER_ATLAS_PATH.exists():
+        return USER_ATLAS_PATH
+    return BUNDLED_ATLAS_PATH
+
+
+# Backward-compat alias. External code that imported ATLAS_PATH before
+# v1.22.0 (e.g. pre-existing scripts outside this repo) gets the
+# resolved path. Internal code should call _resolve_atlas_path() so the
+# value is re-evaluated after the user first runs scan_full_library.
+ATLAS_PATH = _resolve_atlas_path()
 
 _atlas_instance: Optional[AtlasManager] = None  # kept for legacy imports
 
 
 def _build_atlas() -> AtlasManager:
-    return AtlasManager(str(ATLAS_PATH))
+    return AtlasManager(str(_resolve_atlas_path()))
 
 
 _atlas_holder = Singleton(_build_atlas)
 
 
 def get_atlas() -> AtlasManager:
-    """Thread-safe accessor. Re-reads device_atlas.json if its mtime advanced."""
+    """Thread-safe accessor. Re-reads the resolved atlas path if its
+    mtime advanced. Uses the resolver, so if the user's first
+    scan_full_library call runs mid-session, the next get_atlas()
+    picks up the new user path (via invalidate_atlas, which
+    scan_full_library already calls on success)."""
     global _atlas_instance
-    instance = _atlas_holder.get(reload_if_newer=ATLAS_PATH)
+    instance = _atlas_holder.get(reload_if_newer=_resolve_atlas_path())
     _atlas_instance = instance   # keep legacy attribute in sync
     return instance
 
 
 def invalidate_atlas() -> None:
-    """Force the next get_atlas() to re-read device_atlas.json from disk."""
+    """Force the next get_atlas() to re-read the resolved atlas path
+    from disk. Called by scan_full_library after writing a fresh
+    user atlas so subsequent tool calls see the new data without
+    an MCP server restart."""
     global _atlas_instance
     _atlas_holder.invalidate()
     _atlas_instance = None

package/mcp_server/atlas/tools.py

@@ -466,11 +466,17 @@ def scan_full_library(
     """
     from .scanner import normalize_scan_results
     from .enrichments import load_enrichments, merge_enrichments
-    from . import AtlasManager
+    from . import AtlasManager, USER_ATLAS_DIR, USER_ATLAS_PATH
 
+    # v1.22.0: scans always write to the user atlas path, never the
+    # bundled baseline. The user-data directory is created on demand
+    # so a brand-new install (no ~/.livepilot/ at all) still works.
+    # Enrichments are read from the bundled package (same as before —
+    # they're authored in-repo).
     atlas_dir = os.path.dirname(os.path.abspath(__file__))
-    atlas_path = os.path.join(atlas_dir, "device_atlas.json")
     enrichments_dir = os.path.join(atlas_dir, "enrichments")
+    USER_ATLAS_DIR.mkdir(parents=True, exist_ok=True)
+    atlas_path = str(USER_ATLAS_PATH)
 
     if not force and os.path.exists(atlas_path):
         age = time.time() - os.path.getmtime(atlas_path)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "livepilot",
-  "version": "1.21.3",
+  "version": "1.22.0",
   "mcpName": "io.github.dreamrec/livepilot",
   "description": "Agentic production system for Ableton Live 12 — 430 tools, 53 domains, 44 semantic moves. Device atlas (5264 devices, 120 enriched, 7 indexes), Splice intelligence (gRPC + GraphQL describe-a-sound + preview + collections + presets), 9-band spectral perception auto-loaded via ensure_analyzer_on_master, Creative Director skill, technique memory, 12 creative intelligence engines",
   "author": "Pilot Studio",

package/remote_script/LivePilot/__init__.py

@@ -5,7 +5,7 @@ Entry point for the ControlSurface. Ableton calls create_instance(c_instance)
 when this script is selected in Preferences > Link, Tempo & MIDI.
 """
 
-__version__ = "1.21.3"
+__version__ = "1.22.0"
 
 from _Framework.ControlSurface import ControlSurface
 from . import router

package/server.json

@@ -6,12 +6,12 @@
     "url": "https://github.com/dreamrec/LivePilot",
     "source": "github"
   },
-  "version": "1.21.3",
+  "version": "1.22.0",
   "packages": [
     {
       "registryType": "npm",
       "identifier": "livepilot",
-      "version": "1.21.3",
+      "version": "1.22.0",
       "transport": {
         "type": "stdio"
       }