livepilot 1.23.5 → 1.24.0

package/CHANGELOG.md

@@ -1,5 +1,66 @@
 # Changelog
 
+## v1.24.0 — 2026-05-02
+
+Compose framework rebuild — fast / full / develop modes share an Applier substrate; full mode is a clean-room rewrite around an LLM-creative two-phase brief flow (LLM provides FORM, framework provides VOCABULARY).
+
+### Added — compose framework
+
+- **`mcp_server/composer/framework/applier.py`** — shared pre-flight + post-flight skeleton. `preflight()` loads analyzer, reconnects bridge, retries handshake up to 3× with 200ms gaps (fixes the M4L UDP-bind race that previously left "bridge not connected" failures on the first instrument-load call). `postflight()` sets monitoring=Auto on every newly-created track and calls `back_to_arranger`. Functions are dependency-injected so each mode's apply.py wires its own analyzer/bridge funcs.
+- **`mcp_server/composer/framework/knowledge_pack.py`** — `event_lexicon` (42 structural events across 7 categories: drum_density, harmonic, texture, vocal, rhythm_feel, tension, fx_gesture), `genre_context` loader (parses `livepilot/skills/livepilot-core/references/genre-vocabularies.md`, 15 genres), `artist_context` loader (parses `artist-vocabularies.md`, ~25 producers). `atlas_candidates_per_role` is scaffolded but **left as empty stub** — see v1.25 plan.
+- **`mcp_server/composer/full/apply.py::apply_full_plan_v2`** — full-mode rebuild. Takes an LLM-authored plan with `form` (sections), `tracks` (instruments + variants + arrangement_clips), `events`. Per-section variants prevent the BUG-FULL-MODE-18 flat tile. Native arrangement clip flow via `create_native_arrangement_clip` + `add_arrangement_notes` + `set_clip_loop` produces a single arrangement clip per section instead of 32 tiny tiles (BUG-FULL-MODE-23).
+- **`mcp_server/composer/develop/`** — develop mode (extend an existing 8-bar loop). `seed_introspector.py` classifies tracks by name + content; `brief_builder.py` pulls artist references from the user prompt + research hooks; `apply.py` writes per-track variants without disturbing the seed.
+- **`mcp_server/composer/fast/brief_builder.py`** — fast-mode brief authoring with tier classification (Tier-A curated > Tier-B audible default > Tier-C never). Hunt order: `search_browser(path="sounds")` first to surface curated `.adg` chains, then atlas, then bare instruments only as last resort. Bans bare-melodic Tier-B when curated alternatives exist (§1 violation prevention).
+
+### Fixed — live-test wave
+
+- **BUG-FULL-MODE-14: bridge UDP race** — `apply_full_plan` returned success on `bridge.connect()` but the M4L JS listener takes 100-500ms to bind the UDP socket; next bridge call ("UDP bridge is not connected"). Fixed in `Applier.preflight` with handshake retry loop.
+- **BUG-FULL-MODE-17: monitoring=In on all tracks** — Phase 4 Task 4 set monitoring `state=0` ("In", always passes input) instead of `state=1` ("Auto", default). Live screenshot showed every track armed-red. Fixed: state=1 in `Applier.postflight`.
+- **BUG-FULL-MODE-18: flat tiling instead of per-section variants** — full mode reused one MIDI variant for the whole arrangement. Fixed: per-section `variant_id` in plan + per-section variant resolution in apply.
+- **BUG-FULL-MODE-19: `track_index` vs `index` field** — `apply_full_plan_v2` read `result["track_index"]` from `create_midi_track`, but the Remote Script returns `result["index"]`. Same bug class as the v1.23.x `parameter_name` vs `name_or_index` fix.
+- **BUG-FULL-MODE-20: zombie/leftover tracks from previous sessions** — postflight only deleted default-named tracks (`1-MIDI`, `2-Audio`). Now also deletes tracks with no clips AND no instrument device (true-empty zombies) regardless of name.
+- **BUG-FULL-MODE-21: drum-pitch super-low** — Simpler default root C3=60, drum clips firing at MIDI 36 = 24 semitones below native pitch. Drum-role repair (Volume=0, Snap=Off, Transpose=+24) was already in fast mode; now ported to full mode for parity. (`composer/fast/apply._apply_drum_role_repair` is imported by full mode.)
+- **BUG-FULL-MODE-22: arrangement clip length wrong** — fixed alongside BUG-FULL-MODE-23.
+- **BUG-FULL-MODE-23: 32-tile arrangement** — `create_arrangement_clip` duplicated the session clip every `loop_length` beats, producing a tile-grid arrangement. Switched to `create_native_arrangement_clip` + `add_arrangement_notes` + `set_clip_loop` for native arrangement clip authoring; one clip per section, looped to fill section length.
+
+### Known gaps (deferred to v1.25)
+
+- **`KnowledgePack.atlas_candidates_per_role` is empty stub** — agent still falls through to `search_browser` filename matching instead of consulting the indexed atlas. Ranking + truncation per role needs careful design (1-2 days). Documented as **BUG-FULL-MODE-24** and is the headline feature of v1.25.
+- **Reasoning loop (Scope B)** — full mode currently does best-effort `analyze_mix`/`analyze_loudness`/`analyze_sound_design` calls but doesn't iterate on findings. v1.25 wires an analyze→adjust→re-analyze loop with a budget.
+- **Drum craft + bass craft sophistication passes** — full mode produces correct but generic drum/bass programming. v1.25 lifts these to "poweruser" depth (per-bar variation, sidechain wiring, ghost notes, swing curves, sub/mid/click frequency separation).
+- **`verify_track_audible` MCP tool** — Phase 4 Task 18c, deferred.
+
+### Tool count
+- Net delta: **453 → 459** (+6: compose framework expansion).
+
+### Tests
+- `tests/composer/` — 184 passing across fast / full / develop / framework subdirectories.
+- `tests/test_tools_contract.py` — 459 tools verified.
+
+## v1.23.6 — 2026-04-30
+
+### Fixed
+
+- **`ensure_analyzer_on_master` cold-start disambiguation.** On a fresh Live boot the User Library browser cache is uncached, and `find_and_load_device("LivePilot_Analyzer")` can exceed the 20s recv timeout while BFS-ing the user library tree. The previous catch-block surfaced `status="install_required"` even though the .amxd is sitting at the canonical install path — sending the agent into a reinstall loop. New `_analyzer_amxd_installed_at_user_library()` filesystem check disambiguates: when the .amxd is present at `~/Music/Ableton/User Library/Presets/Audio Effects/Max Audio Effect/`, the tool now returns `status="cache_cold"` with a retry hint instead. Genuine missing-.amxd path still returns `install_required` correctly. (`mcp_server/tools/analyzer.py`)
+- **`set_compressor_sidechain` diagnostic depth.** When `_find_sidechain_surface` returns None on Compressor2 (Live 12.3.6+), the raised error now includes `class=...` + `canonical_parent.class=...` + a widened child-attribute walk (`input_routings`, `routing_inputs` added to the probe). Lets the next live failure leave a tighter trail so the actual Compressor2 LOM shape can be confirmed in one round trip rather than a separate probe session. No behavior change for legacy Compressor (I) or the two known Compressor2 hypotheses. (`remote_script/LivePilot/mixing.py`)
+- **`atlas_demo_story` production_decision class-name leak.** `harmonic-foundation` and `rhythmic-driver` role branches fell back to `primary_cls` when `user_name` was empty — producing useless prose like "InstrumentGroupDevice chosen as harmonic spine." Both branches now use `primary_uname or t_name or primary_cls`, matching the texture-role fallback that was already correct. (`mcp_server/atlas/demo_story.py`)
+- **`atlas_extract_chain` Macro N labeling on M4L devices.** The rack-device path already resolved producer-named macros via `resolve_preset_for_device`, but the M4L (.amxd) device path emitted raw `Macro N`. Now both paths cross-reference the preset sidecar — PitchLoop89's "Spectral Stretch" gets the proper name instead of "Macro 2" so `set_device_parameter` resolves at execution time. (`mcp_server/atlas/extract_chain.py`)
+- **Bundled atlas stats refresh.** `stats.enriched_devices` was stale at `87` (from a v1.21.x scan); the actual `enriched=True` flag count had grown to `135`. Recounted in place. No tool consumes the field with semantically-different behavior, but the stale stat had drifted past the soft-warn threshold's intent. (`mcp_server/atlas/device_atlas.json`)
+
+### Docs / drift
+
+- **`docs/M4L_BRIDGE.md`**: amxd ping reference and bridge-cmd count corrected — was `version: "1.23.1"` and "30 commands" from prior releases; now `1.23.6` and `32`.
+- **`livepilot/skills/livepilot-core/references/genre-vocabularies.md`**: added Synthwave / Retrowave / Outrun entry to match the 15-genre claim in CLAUDE.md/AGENTS.md (file had 14; the YAML packet at `concepts/genres/synthwave.yaml` had been the source-of-truth for v1.18+).
+
+### Tests
+
+- `tests/test_ensure_analyzer_on_master.py` — 2 new tests under `TestColdBrowserCacheDisambiguation` covering the cache-cold branch (`.amxd` present → cache_cold) and the install-required branch (`.amxd` absent → install_required). Existing `test_returns_install_required_when_device_not_in_browser` updated to monkeypatch the new path-check helper so it exercises the genuinely-not-installed path regardless of dev-machine state.
+- Total: **3409 passing**, 1 skipped, 0 failed (up from 3407 in v1.23.5).
+
+### Audit notes
+
+A full read-only audit pass against `BUGS.md` + `BUGS_TESTING_2026-04-30.md` confirmed that 14 of the bugs documented in those files were already fixed in v1.23.4's `bugfixes-2026-04-26` commit batch but never marked closed in the testing doc. No-ops here, but worth noting that the 38-bug list in the testing file is now ~5 genuinely-open items.
+
 ## v1.23.5 — 2026-04-30
 
 ### Fixed (Remote Script reliability — credit: PR #35 reporter)

package/README.md

@@ -17,7 +17,7 @@
 
 <p align="center">
   An agentic production system for Ableton Live 12.<br>
-  453 tools. 54 domains. Device atlas. Plan-aware Splice integration. Auto-composition. Spectral perception. Technique memory. Drum-rack pad builder. Live dead-device detection.
+  459 tools. 54 domains. Device atlas. Plan-aware Splice integration. Auto-composition. Spectral perception. Technique memory. Drum-rack pad builder. Live dead-device detection.
 </p>
 
 <br>
@@ -29,6 +29,11 @@
 > Side effects that touch state outside the Live project — Splice downloads, memory/ledger writes,
 > installer actions, atlas scans, filesystem writes — persist beyond undo.
 
+> [!WARNING]
+> LivePilot is actively in development. Tools, behavior, and APIs change frequently between versions.
+> Pin to a specific version for stable work. Known gaps and in-progress features are documented in
+> each release's CHANGELOG entry.
+
 <br>
 
 ---
@@ -54,7 +59,7 @@ Most MCP servers are tool collections — they execute commands. LivePilot is an
 
 ## Two Ways to Talk to LivePilot
 
-Pick whichever is faster for the idea in your head — both reach the same 453-tool surface.
+Pick whichever is faster for the idea in your head — both reach the same 459-tool surface.
 
 ### Route A — Artist / aesthetic shorthand
 
@@ -107,7 +112,7 @@ Most sessions do both. Lead with shorthand to anchor the aesthetic, then refine
 │         └─────────────────┼──────────────────┘                       │
 │                           ▼                                          │
 │                  ┌─────────────────┐                                  │
-│                  │   453 MCP Tools  │                                  │
+│                  │   459 MCP Tools  │                                  │
 │                  │   54 domains     │                                  │
 │                  └────────┬────────┘                                  │
 │                           │                                          │
@@ -148,7 +153,7 @@ Most sessions do both. Lead with shorthand to anchor the aesthetic, then refine
 
 ## The Intelligence Layer
 
-12 engines sit on top of the 453 tools. They give the AI musical judgment, not just musical execution.
+12 engines sit on top of the 459 tools. They give the AI musical judgment, not just musical execution.
 
 ### SongBrain — What the Song Is
 
@@ -200,7 +205,7 @@ Every engine follows: **measure before → act → measure after → compare**.
 
 ## Tools
 
-453 tools across 54 domains. Highlights below — [full catalog here](docs/manual/tool-catalog.md).
+459 tools across 54 domains. Highlights below — [full catalog here](docs/manual/tool-catalog.md).
 
 <br>
 
@@ -425,9 +430,9 @@ LivePilot reads Splice's local SQLite database to search your downloaded samples
 
 <br>
 
-### Composer — 3 tools
+### Composer — three modes (v1.24.0)
 
-Prompt-to-plan auto-composition engine.
+Prompt-to-plan auto-composition engine. Three modes share a common Applier substrate (preflight: bridge handshake retry + analyzer load; postflight: monitoring=Auto on new tracks + back_to_arranger). All modes return executable plans — the agent executes each step, it does not run autonomously.
 
 ```
 "dark minimal techno 128bpm with industrial textures and ghostly vocals"
@@ -451,10 +456,51 @@ Prompt-to-plan auto-composition engine.
 └─────────────────┘
 ```
 
-- 4 genre defaults: house, techno, trap, ambient (unknown genres fall back to a neutral plan)
-- Returns step-by-step plans — the agent executes each tool call in sequence
-- `compose` — plan a multi-layer composition from text prompt
-- `augment_with_samples` — plan sample-based layers for existing session
+#### fast mode — `compose_fast_apply`
+
+Quick loop sketch. Single scene in session view. Intended for roughing out ideas quickly.
+
+- Hunt order: curated `.adg` chains from the browser first, atlas devices second, bare instruments only as last resort
+- Drum-role pitch repair included (Simpler default root vs. MIDI 36 offset)
+- 4 genre defaults: house, techno, trap, ambient (unknown genres fall back to a neutral layer plan)
+- Invoke with: *"Make me a [genre] loop at [tempo] BPM"*
+
+#### full mode — `compose_full_apply`
+
+Full track with song form: intro, verse, hook, breakdown, outro. Uses a two-phase LLM-creative brief flow — the LLM authors the form (sections, track list, per-section variants); the framework supplies the vocabulary (device hunt order, MIDI generation rules, arrangement conventions).
+
+- Per-section MIDI variants prevent repeated tiles across the arrangement
+- Native arrangement clips via `create_native_arrangement_clip` (one clip per section, looped to fill section length)
+- Zombie-track cleanup in postflight (removes tracks with no clips and no instrument device)
+- Drum-role pitch repair ported from fast mode
+- **Known gap (v1.25):** `KnowledgePack.atlas_candidates_per_role` is an empty stub — the agent currently falls through to `search_browser` filename matching instead of consulting the indexed atlas. This is BUG-FULL-MODE-24 and is the headline feature of v1.25.
+- Invoke with: *"Write a full [genre] track at [tempo] BPM"* or *"Build a full arrangement"*
+
+#### develop mode — `develop_apply`
+
+Extends an existing 8-bar loop without disturbing the seed material.
+
+- Introspects the existing session (classifies tracks by name and content)
+- Pulls artist and stylistic references from the user prompt
+- Writes per-track variants and new supporting layers alongside the seed
+- Invoke with: *"Develop this loop"* or *"Extend what's here into a longer idea"*
+
+#### KnowledgePack scaffolding (v1.24.0)
+
+All three modes share a `KnowledgePack` that provides structured creative context at runtime:
+
+- `event_lexicon` — 42 structural events across 7 categories (drum density, harmonic, texture, vocal, rhythm feel, tension, fx gesture)
+- `genre_context` — parses the 15-genre `genre-vocabularies.md` at load time
+- `artist_context` — parses the ~25-producer `artist-vocabularies.md` at load time
+- `atlas_candidates_per_role` — **stubbed in v1.24.0**, will be populated in v1.25
+
+#### Core composer tools
+
+- `compose` — plan a multi-layer composition from text prompt (entry point, mode-agnostic)
+- `compose_fast_apply` — execute fast mode directly
+- `compose_full_apply` — execute full mode directly
+- `develop_apply` — execute develop mode directly
+- `augment_with_samples` — plan sample-based layers for an existing session
 - `get_composition_plan` — dry-run preview (see the plan without credit checks)
 
 <br>
@@ -494,7 +540,7 @@ The V2 intelligence layer. These tools analyze, diagnose, plan, evaluate, and le
 | Creative Constraints | 5 | constraint activation, reference-inspired variants |
 | Preview Studio | 5 | variant creation, preview rendering, comparison, commit |
 
-> **[View all 453 tools →](docs/manual/tool-catalog.md)**
+> **[View all 459 tools →](docs/manual/tool-catalog.md)**
 
 <br>
 
@@ -721,7 +767,7 @@ See [CONTRIBUTING.md](CONTRIBUTING.md) for architecture details, code guidelines
 
 | Document | What's inside |
 |----------|---------------|
-| [Manual](docs/manual/index.md) | Complete reference: architecture, all 453 tools, workflows |
+| [Manual](docs/manual/index.md) | Complete reference: architecture, all 459 tools, workflows |
 | [Intelligence Layer](docs/manual/intelligence.md) | How the 12 engines connect — conductor, moves, preview, evaluation |
 | [Device Atlas](docs/manual/device-atlas.md) | 5264 devices indexed — search, suggest, chain building |
 | [Samples & Slicing](docs/manual/samples.md) | 3-source search, fitness critics, slice workflows |

package/m4l_device/livepilot_bridge.js

@@ -34,7 +34,7 @@ outlets = 2; // 0: to udpsend (responses), 1: to buffer~/status
 // Single source of truth for the bridge version — bumped alongside the
 // rest of the release manifest. Surfaced in the UI via messnamed("livepilot_version", ...)
 // so the frozen .amxd visibly reports which build it was last exported from.
-var VERSION = "1.23.5";
+var VERSION = "1.23.6";
 
 // ── State ──────────────────────────────────────────────────────────────────
 

package/mcp_server/__init__.py

@@ -1,2 +1,2 @@
 """LivePilot MCP Server — bridges MCP protocol to Ableton Live."""
-__version__ = "1.23.5"
+__version__ = "1.24.0"

package/mcp_server/atlas/__init__.py

@@ -57,9 +57,23 @@ class AtlasManager:
             if dev_category:
                 self._by_category.setdefault(dev_category, []).append(dev)
 
-            # Tag index
-            for tag in dev.get("tags", []):
-                self._by_tag.setdefault(tag.lower(), []).append(dev)
+            # Tag index — pull from BOTH legacy "tags" AND enriched
+            # "character_tags". BUG-P (caught 2026-05-01 live demo): the
+            # bundled factory atlas uses character_tags exclusively, so
+            # _by_tag was empty across the board, breaking every tag-based
+            # role picker. Reading both fields makes the index actually
+            # populated for normal user atlases.
+            seen_tags = set()
+            for tag in dev.get("tags", []) or []:
+                tag_lower = str(tag).lower()
+                if tag_lower not in seen_tags:
+                    seen_tags.add(tag_lower)
+                    self._by_tag.setdefault(tag_lower, []).append(dev)
+            for tag in dev.get("character_tags", []) or []:
+                tag_lower = str(tag).lower()
+                if tag_lower not in seen_tags:
+                    seen_tags.add(tag_lower)
+                    self._by_tag.setdefault(tag_lower, []).append(dev)
 
             # Genre index (primary + secondary)
             for genre in dev.get("genres", {}).get("primary", []):

package/mcp_server/atlas/demo_story.py

@@ -608,19 +608,25 @@ def demo_story(
         primary_uname = primary_device.get("user_name") or ""
         nonzero = _count_nonzero_macros(devices)
 
+        # BUG-E5: when user_name is empty AND primary device is a Group/Rack
+        # device, primary_cls reads as "InstrumentGroupDevice" /
+        # "AudioEffectGroupDevice" — useless prose like "InstrumentGroupDevice
+        # chosen as harmonic spine". Fall back to track name first, then class.
+        primary_label = primary_uname or t_name or primary_cls or "<unknown>"
+
         if t_type == "ReturnTrack":
             prod_decision = (
-                f"{primary_cls} shared return — applies uniformly across all sends."
+                f"{primary_label} shared return — applies uniformly across all sends."
             )
         elif role == "harmonic-foundation":
             prod_decision = (
-                f"{primary_uname or primary_cls} chosen as harmonic spine; "
+                f"{primary_label} chosen as harmonic spine; "
                 f"{nonzero} macro(s) committed to specific values, "
                 "suggesting deliberate timbral targeting."
             )
         elif role == "rhythmic-driver":
             prod_decision = (
-                f"Drum rack '{primary_uname or primary_cls}' provides rhythmic drive. "
+                f"Drum rack '{primary_label}' provides rhythmic drive. "
                 f"{nonzero} non-default macro values indicate custom tuning."
             )
         elif role == "texture":

package/mcp_server/atlas/device_atlas.json

@@ -15,7 +15,7 @@
     "max_for_live": 523,
     "clips": 1000,
     "current_project": 60,
-    "enriched_devices": 87
+    "enriched_devices": 135
   },
   "devices": [
     {

package/mcp_server/atlas/extract_chain.py

@@ -492,18 +492,33 @@ def _emit_execution_steps(
             "device_index": device_index,
         })
         if fidelity != "structure-only" and macros:
+            # BUG-E2#1 (M4L path): producer-named macros on devices like
+            # PitchLoop89 ("Spectral Stretch") get raw "Macro N" labels here
+            # unless we cross-reference the preset sidecar — same as the rack
+            # path at line ~395. Without name resolution, set_device_parameter
+            # by name silently NOT-FOUNDs at execution time.
+            m4l_macro_names: dict[int, str] = {}
+            if uname and pack_name:
+                m4l_match = resolve_preset_for_device(pack_name, cls, uname)
+                m4l_macro_names = m4l_match.get("macro_names") or {}
             macro_subset = (
                 _get_nonzero_macros(macros)
                 if fidelity == "exact"
                 else _top_k_macros_by_deviation(macros, k=5)
             )
             for m in macro_subset:
+                resolved_name = m4l_macro_names.get(m["index"]) or f"Macro {m['index'] + 1}"
+                source_tag = (
+                    "als-parse+adg-parse"
+                    if resolved_name != f"Macro {m['index'] + 1}"
+                    else "als-parse"
+                )
                 steps.append({
                     "action": "set_device_parameter",
                     "device_index": device_index,
-                    "parameter_name": f"Macro {m['index'] + 1}",
+                    "parameter_name": resolved_name,
                     "value": round(_safe_float(m.get("value", "0")), 2),
-                    "comment": f"M4L macro {m['index'] + 1} [SOURCE: als-parse]",
+                    "comment": f"M4L macro {m['index'] + 1} [SOURCE: {source_tag}]",
                 })
         return steps, warnings
 

package/mcp_server/audit/__init__.py

@@ -0,0 +1,6 @@
+"""Layer + session audit aggregation.
+
+Replaces the §5 layer-precision and §9 end-of-session checklist (each a
+multi-call manual sequence) with single tool calls that run the checks
+server-side and return one structured report.
+"""