pixelweaver 0.1.0

package/todo/.done/server-multi-frame-design.md

@@ -0,0 +1,405 @@
+# Server-side multi-frame storage
+
+## Context
+
+The Python companion server (`server/src/pixelweaver/`) acts as the
+single source of truth for collaborative sessions and MCP tool calls. Its
+authoritative data model has **no concept of animation frames**: each
+canvas holds exactly one `pixel_data` dict keyed by layer id. Every path
+that touches "a frame" currently resolves to the canvas's only pixel
+buffer.
+
+The frontend (`src/lib/animation/frame-model.svelte.ts:15-37`) has full
+multi-frame support: an ordered list of `Frame` objects, per-frame
+`durationMs`, a current-frame pointer, global FPS, and an origin. When a
+project is saved locally (Tauri file dialog) the snapshot includes
+`frames` and `frameTags` (`src/lib/save/project-snapshot.ts:73-95`). When
+the same project flows through the collab/MCP server, those extra frames
+are silently dropped — only the layer slice of the currently-active
+frame survives.
+
+This matters for:
+
+- **MCP export**: `export_spritesheet` currently pretends each *canvas*
+  is an animation frame (`mcp_registry.py:946-958`); it cannot reach
+  real per-frame pixel data.
+- **MCP frame tools**: `add_frame`, `remove_frame`, `duplicate_frame`,
+  `reorder_frame`, `set_frame_duration`, `set_global_fps` are already
+  registered (`mcp_registry.py:442-493`) but they dispatch opaque
+  commands that the frontend executes — the server never models the
+  result, so `get_canvas_info` hard-codes `"frame_count": 1`
+  (`mcp_registry.py:1211`).
+- **Autosave / project save / load**: on-disk layout hardcodes
+  `frames/0/` (`storage.py:72-82, 145`) and the JSON carries no frame
+  list.
+- **State round-trip through the MCP bridge**: `serialize_state` /
+  `deserialize_into_state` only flatten layer pixel data
+  (`mcp_bridge.py:86-155`).
+
+The recent guardrail in `export_frame_png` at `storage.py:188-192`
+raises `ValueError` for any `frame != 0` rather than silently returning
+frame 0. That explicit failure is the scaffold this design builds on:
+once multi-frame support lands the guard becomes a normal bounds check.
+
+## Current data model
+
+### Python server
+
+`CanvasState` (`state.py:17-41`):
+
+- `name`, `width`, `height`
+- `layers: list[dict]` — layer tree (id, name, visible, opacity, blendMode, locked, …)
+- `pixel_data: dict[str, bytes]` — `layer_id -> raw RGBA bytes`
+
+`ProjectState` (`state.py:44-64`):
+
+- `name`, `width`, `height`
+- `canvases: dict[str, CanvasState]`
+- `command_history`, `redo_stack`
+
+No fields exist for frame list, current frame, global FPS, per-frame
+duration, onion skin, or frame tags.
+
+### On-disk layout (`storage.py:36-99`)
+
+```
+<base_dir>/<project>/
+  project.json
+  canvases/<canvas>/
+    canvas.json        -- name, width, height, layers
+    frames/0/
+      layer-<id>.png   -- raw RGBA per layer, written by PIL
+  history.json
+```
+
+`frames/0` is literally hard-coded in `_save_canvas` and `_load_canvas`.
+
+### Wire format (WS + REST)
+
+- `CanvasState.to_dict()` excludes pixel data (`state.py:28-35`).
+- `build_full_state_patch` base64-encodes the **one** pixel buffer per
+  layer into a `full_state` patch (`state.py:166-201`).
+- `PixelPatch` already carries `frame_index` (`protocol.py:142-150`) and
+  `FramePatch` models `add`/`remove`/`reorder`/`set_duration`
+  (`protocol.py:170-179`). The schemas are multi-frame-aware; the
+  *producers* on the server are not — nothing ever sets `frame_index` to
+  anything other than 0.
+- `get_sync_state` returns canvases without any frame list
+  (`state.py:136-163`).
+
+### Frontend wire expectations
+
+- `applyServerNativeState` in `patch-applicator.ts:118-156` drops any
+  incoming canvas state onto `getCurrentFrame()` — a silent collapse to
+  a single frame.
+- `project-manager.ts:80-102` reads the server `GET /api/projects/{name}`
+  shape (`{ canvases: {...} }`) and only extracts `{name, width, height}`.
+- `ws-client.ts:16-26` types the sync payload as `{canvases, layers,
+  history}` with no frames.
+
+## Target data model
+
+### `FrameState` dataclass
+
+Add to `state.py`:
+
+```python
+@dataclass
+class FrameState:
+    id: str                              # stable uuid, matches frontend Frame.id
+    duration_ms: int | None              # None -> derive from global_fps
+    # layer_id -> raw RGBA bytes for THIS frame
+    pixel_data: dict[str, bytes] = field(default_factory=dict)
+```
+
+### `CanvasState` changes
+
+- Remove `pixel_data: dict[str, bytes]`.
+- Add `frames: list[FrameState]`.
+- Add `current_frame_index: int = 0`.
+- Add `global_fps: float = 12.0`.
+- Add `origin_x: int = 0`, `origin_y: int = 0`.
+- Optionally `frame_tags: list[dict]` — named ranges from
+  `src/lib/animation/frame-tags.svelte.ts`. (Frontend already round-trips
+  these in its local save; parking them as an opaque
+  `list[dict[str, Any]]` on the server is cheapest.)
+
+`layers` stays shared across all frames (layer metadata like name,
+visibility, opacity is per-layer, not per-frame — matches
+`frame-model.svelte.ts:15-22` where a layer's *pixel data* is per frame
+but the layer tree lives in `layer-tree.svelte.ts`).
+
+### Convenience accessors
+
+- `CanvasState.current_frame() -> FrameState` — replaces `canvas.pixel_data[id]` usage.
+- `CanvasState.frame_at(index: int) -> FrameState` — bounds-checked.
+- `CanvasState.frame_count() -> int`.
+
+### Invariants
+
+- A canvas always has at least one frame (mirrors the frontend's
+  "cannot remove last frame" rule at `frame-model.svelte.ts:141`).
+- `0 <= current_frame_index < len(frames)`.
+- All frames share the same layer id set; a layer missing pixel data in
+  a frame is equivalent to a transparent buffer (matches the current
+  tolerance of missing `pixel_data[layer_id]`).
+
+## Affected files and changes
+
+| File | What changes |
+|---|---|
+| `state.py` | New `FrameState` dataclass; refactor `CanvasState` to hold `frames`, `current_frame_index`, `global_fps`, `origin_*`, `frame_tags`; update `create_project` to seed one `FrameState`; update `build_full_state_patch` to emit per-frame pixel data; update `get_sync_state` to include frames. |
+| `storage.py` | Update `_save_canvas` / `_load_canvas` to walk a real frame list; rewrite `frames/N/` layout (see format choice below); update `export_frame_png` to index into `frames[frame]` and drop the `frame != 0` guard. |
+| `main.py` | Update `GET /api/projects/{name}` response (via `to_dict`) so the frontend `project-manager.ts` sees frame metadata; export endpoint `/export/png/{canvas}/{frame}` already takes `frame` — wire it through. |
+| `mcp_bridge.py` | `serialize_state` / `deserialize_into_state` walk frames not `pixel_data`. |
+| `mcp_registry.py` | Make all drawing tools accept `frame_index` (default = current); implement real handlers for `add_frame`, `remove_frame`, `duplicate_frame`, `reorder_frame`, `set_frame_duration`, `set_global_fps` instead of dispatching opaque frontend commands; fix `get_canvas_info` frame count; fix `export_spritesheet` to iterate real frames (not canvases); add `set_current_frame`, `get_frame_count`, `get_current_frame` read tools. |
+| `protocol.py` | No shape changes needed for `PixelPatch` / `FramePatch` (already carry `frame_index` and frame actions). Add a `current_frame_index` field to the `state_sync` payload docs. |
+| `websocket.py` | `handle_message` is frame-agnostic today; no direct changes. `build_full_state_patch` change in `state.py` is enough. |
+| `autosave.py` | No structural change — still just calls `save_project`. Only a doc update if we want "dirty frame subset" granularity later. |
+| `tests/test_storage.py` | Update `TestExportFramePng` to cover multi-frame; update save/load round-trip to assert `frames/N/` for N > 0. |
+| `tests/test_api.py` | Update export endpoint test for N > 0. |
+| `tests/test_mcp_registry.py` | Add coverage for real frame tool handlers + `frame_index` on drawing tools. |
+| `tests/test_mcp_bridge.py` | Update `serialize_state` / `deserialize_into_state` round-trip to include frames. |
+| `src/lib/sync/ws-client.ts` | Extend `StateSnapshot` type with optional `frames` per canvas. |
+| `src/lib/sync/project-manager.ts` | Extend `ProjectInfo` / `CanvasInfo` with `frameCount`, `currentFrameIndex`. |
+| `src/lib/sync/patch-applicator.ts` | `applyServerNativeState` should rebuild the full frame list instead of collapsing to `getCurrentFrame()`. |
+
+## Wire protocol changes
+
+Good news: `PixelPatch.frame_index` (`protocol.py:147`) and `FramePatch`
+(`protocol.py:170-179`) **already exist**. The wire schema was designed
+multi-frame-ready; only the server-side producers need to catch up.
+
+Changes required:
+
+- `build_full_state_patch` must emit one `PixelPatch`-equivalent entry
+  per `(frame, layer)` pair, or (preferred) extend the `full_state`
+  snapshot shape so each canvas carries a `frames: [{id, durationMs,
+  layers: [{id, pixelData}]}]` array.
+- `get_sync_state` (`state.py:136-163`) needs to grow a `frames` field
+  per canvas and a `globalFps` / `currentFrameIndex` scalar pair.
+- No version bump needed **if** the new fields are additive and the
+  frontend treats missing fields as "legacy single frame". Because
+  server and frontend ship in lockstep and there is only one installed
+  version at a time (see Backward compatibility below), a hard swap
+  with no fallback is acceptable.
+
+Frontend currently does **not** send frame data on the WS. Commands flow
+one at a time; a full-state snapshot only moves server → client. So
+server-to-client is the only direction that changes.
+
+## MCP tool surface changes
+
+| Tool | Current state | Change |
+|---|---|---|
+| `add_frame`, `remove_frame`, `duplicate_frame`, `reorder_frame`, `set_frame_duration`, `set_global_fps` | Registered, but dispatched as opaque commands for the frontend to execute (`mcp_registry.py:442-493`). Server state never sees the mutation. | Implement as real state mutations; broadcast a `FramePatch` + full-state fallback. |
+| `draw_pixels`, `erase_pixels`, `flood_fill`, `draw_line`, `draw_rect`, `draw_ellipse`, `draw_diamond`, `draw_gradient` | Dispatched as commands on the active canvas, implicitly "current frame". | Accept an optional `frame_index` parameter; default = `canvas.current_frame_index`. |
+| `get_pixel`, `get_region`, `get_palette`, `get_canvas_thumbnail` | Read from the single `pixel_data` dict. | Read from `canvas.current_frame()` (or accept `frame_index`). |
+| `get_canvas_info` | Hardcodes `"frame_count": 1` (`mcp_registry.py:1211`). | Return `len(canvas.frames)` and `current_frame_index`. |
+| `get_layer_tree` | Reports `has_data` from `canvas.pixel_data` (`mcp_registry.py:1231`). | Check `canvas.current_frame().pixel_data`, or accept `frame_index`. |
+| `export_png` | Calls `export_frame_png(project, canvas.name)` (frame defaults to 0). | Accept `frame_index` parameter. |
+| `export_spritesheet` | Treats each *canvas* as a frame (`mcp_registry.py:946-958`). | Iterate real frames of one canvas, arrange into a strip. |
+
+New read-only tools to add:
+
+- `get_frame_count` — `{canvas_name?}` → `{count, current_index}`
+- `get_current_frame` — returns `{index, duration_ms}`
+- `set_current_frame` — `{frame_index}` (mutates `current_frame_index`)
+- `get_global_fps` / `set_global_fps` (the latter already registered; implement it)
+
+## On-disk format choice
+
+Three options for where per-frame pixel data lives on disk.
+
+| Aspect | A. `frames/N/layer-<id>.png` | B. Flat binary blob | C. Frame manifest + blobs |
+|---|---|---|---|
+| Layout | `frames/0/layer-a.png`, `frames/1/layer-a.png`, … | One `pixels.bin` per canvas with a header | `frames.json` + `frames/<id>/layer-<id>.png` |
+| Human-inspectable | Yes (PNGs per frame) | No | Yes |
+| Random-access per frame | Good (one dir per frame) | Excellent (seek by offset) | Good |
+| Per-frame metadata (duration, tag) | Must live in `canvas.json` (index-keyed) | Header must grow | Naturally in `frames.json` |
+| Migration cost from current layout | Near zero — already have `frames/0/` | High — new format, new reader/writer | Medium — add a manifest file |
+| Save granularity (dirty N frames only) | Easy | Harder (must rewrite the blob) | Easy |
+| Diff-friendliness (git, rsync) | Best (per-frame PNG) | Worst | Best |
+| Frame reorder cost | Rename dirs or update index in `canvas.json` | Rewrite the blob | Update `frames.json` |
+
+### Recommendation: **Option C (frame manifest + per-frame PNG blobs)**
+
+```
+canvases/<canvas>/
+  canvas.json        -- name, width, height, layers (unchanged)
+  frames.json        -- [{id, duration_ms, tag?}, ...], current_frame_index, global_fps, origin
+  frames/
+    <frame_id>/
+      layer-<layer_id>.png
+```
+
+- Frame identity is by uuid (`frame.id`) not by numeric index — reorders
+  touch only `frames.json`, no file renames.
+- `frames.json` is small and trivially diffable.
+- Per-frame subdirectory mirrors current `frames/0/` layout, so
+  `_save_layer_png` / `_load_layer_png` stay as-is.
+- Autosave can write only changed frame subdirs once a dirty-frame set
+  is tracked (future work; the format makes it possible).
+
+Option A is the path of least *change* but forces frame metadata into
+`canvas.json` with a brittle index-keyed shape and makes reorder awkward.
+Option B loses transparency and diffability.
+
+## Backward compatibility
+
+`package.json` reports `0.0.1` and `pyproject.toml` reports `0.1.0`.
+There is no release channel, no distributed build, no external consumer:
+this is pre-release.
+
+Per the CLAUDE.md rule ("Never add backward compatibility shims,
+wrapper methods, or legacy API surfaces to unreleased projects"):
+
+- **Delete the old single-frame path entirely.** No dual-read shim.
+- `frames/0/` style directories created by the current code will simply
+  fail to load after the change — that is acceptable for developer data.
+- The `_save_canvas` / `_load_canvas` functions are rewritten, not
+  extended with branches.
+- The `pixel_data` attribute on `CanvasState` is removed (not
+  deprecated), and every reader is updated in the same commit.
+
+If a dev has local test projects they want to keep, the recommended
+workflow is a one-shot `scripts/migrate-single-frame-to-multi.py` that
+walks `<data_dir>/*/canvases/*/frames/0/` and writes a new `frames.json`
++ renames `0` to a fresh uuid. This script lives in `scripts/` per the
+project convention, not in the server runtime.
+
+## Migration plan
+
+Each step is one commit; tests stay green at every step.
+
+1. **Add `FrameState` dataclass and `CanvasState.frames` alongside the
+   old `pixel_data`**, with `frames[0].pixel_data is canvas.pixel_data`
+   (shared reference). All existing code still works. Add unit tests for
+   the new accessors. *No behavior change.*
+2. **Rewrite all server-internal readers** (`mcp_registry.py`,
+   `state.py:build_full_state_patch`, `mcp_bridge.py`, `storage.py`) to
+   go through `canvas.current_frame().pixel_data` instead of
+   `canvas.pixel_data`. Existing tests still pass because there is still
+   only one frame. *No behavior change.*
+3. **Drop the `pixel_data` attribute from `CanvasState`.** Only the
+   `frames` list remains. Update `_init_` / `create_project` to seed
+   `frames=[FrameState(...)]`. Run full test suite.
+4. **Implement the new on-disk layout** (Option C). Rewrite
+   `_save_canvas` / `_load_canvas`. Update `test_storage.py`. This is
+   the breaking-change commit for on-disk projects.
+5. **Extend `get_sync_state` and `build_full_state_patch`** to carry
+   all frames, not just the current one. Update the frontend
+   `patch-applicator.ts:applyServerNativeState` to rebuild the full
+   frame list via `deserializeFrames`. Update `ws-client.ts` types.
+6. **Remove the `frame != 0` guardrail from `export_frame_png`**
+   (`storage.py:188-192`); replace with a proper `0 <= frame <
+   len(canvas.frames)` bounds check. Update the REST export endpoint
+   test to cover N > 0.
+7. **Implement real frame MCP tool handlers** (`add_frame`, `remove_frame`,
+   `duplicate_frame`, `reorder_frame`, `set_frame_duration`,
+   `set_global_fps`) — they mutate `canvas.frames` directly instead of
+   dispatching opaque commands, then broadcast a full-state patch. Fix
+   `get_canvas_info` to report real counts. Add
+   `get_frame_count` / `set_current_frame` / `get_current_frame` read
+   tools.
+8. **Add `frame_index` parameter to drawing tools.** Defaults to the
+   current frame; validated against `canvas.frames`. Update
+   `test_mcp_registry.py`.
+9. **Rewrite `export_spritesheet`** to iterate `canvas.frames` on a
+   single canvas instead of iterating canvases. Update its MCP schema
+   to accept `canvas_name` and optional `frame_range`.
+10. **One-shot migration script** at
+    `scripts/migrate-single-frame-to-multi.py` that walks legacy data
+    dirs and rewrites them. Not wired into the runtime.
+
+Rollback points: step 3 is the point of no return for in-memory code
+(the old attribute is gone). Step 4 is the point of no return for
+on-disk data. Everything before those can be reverted cleanly.
+
+## Test strategy
+
+New tests:
+
+- `test_storage.py::TestMultiFrameSaveLoad` — round-trip a canvas with
+  three frames, differing pixel data, differing durations, reorder,
+  reload, assert equality.
+- `test_storage.py::TestExportFramePng::test_frame_n` — export frame 0
+  and frame 2, assert they differ.
+- `test_mcp_registry.py::TestFrameTools` — add / remove / duplicate /
+  reorder / set_duration, each followed by a full-state patch
+  broadcast assertion.
+- `test_mcp_registry.py::TestDrawingWithFrameIndex` — draw on frame 1
+  and assert frame 0 is untouched.
+- `test_mcp_bridge.py::test_serialize_deserialize_multi_frame` —
+  round-trip serialization preserves frames, durations, and current index.
+
+Existing tests to update:
+
+- `test_storage.py` — `frame_dir` assertions currently use
+  `frames/0/` (line 46). Switch to resolving the frame id via
+  `frames.json`.
+- `test_api.py` — any export endpoint tests that implicitly assume
+  `frame=0`.
+- `test_mcp_registry.py::test_get_canvas_info` — expect
+  `frame_count == len(canvas.frames)`.
+- Any test that constructs `CanvasState(...)` directly with
+  `pixel_data=...` must switch to `frames=[FrameState(...)]`.
+
+Integration / e2e:
+
+- A Playwright scenario that creates a project, adds frames, closes
+  and reopens the app, and verifies all frames survive through the
+  autosave → load cycle.
+
+## Effort estimate
+
+**Large.** Rationale:
+
+- 11 files changed across four subsystems (state, storage, MCP, WS, frontend sync).
+- Six MCP tool handlers move from stub dispatch to real implementation.
+- On-disk format rewrite (even small) carries test-suite fallout.
+- Frontend `patch-applicator.ts` needs careful work to rebuild frames
+  without duplicating `deserializeFrames`.
+- Migration script is extra surface area even if one-shot.
+
+A disciplined 10-commit sequence per the migration plan lands this in
+roughly 1.5–2 focused sessions; skipping careful staging risks a broken
+test suite for a day or more.
+
+## Files that will change
+
+Data model and in-memory state:
+
+- `server/src/pixelweaver/state.py`
+
+Storage and on-disk format:
+
+- `server/src/pixelweaver/storage.py`
+- `scripts/migrate-single-frame-to-multi.py` (new, one-shot)
+
+REST API:
+
+- `server/src/pixelweaver/main.py`
+
+MCP surface:
+
+- `server/src/pixelweaver/mcp_registry.py`
+- `server/src/pixelweaver/mcp_bridge.py`
+
+WebSocket protocol (payload producers only; types already support frames):
+
+- `server/src/pixelweaver/websocket.py` (doc updates)
+- `server/src/pixelweaver/protocol.py` (doc updates)
+
+Frontend sync layer:
+
+- `src/lib/sync/ws-client.ts`
+- `src/lib/sync/project-manager.ts`
+- `src/lib/sync/patch-applicator.ts`
+
+Tests:
+
+- `server/tests/test_storage.py`
+- `server/tests/test_api.py`
+- `server/tests/test_mcp_registry.py`
+- `server/tests/test_mcp_bridge.py`