cozyclay 1.2.0 → 1.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (113) hide show
  1. package/CHANGELOG.md +193 -0
  2. package/LICENSE +67 -80
  3. package/LICENSES/GPL-3.0-or-later.txt +674 -0
  4. package/LICENSING.md +32 -0
  5. package/README.md +48 -6
  6. package/THIRD_PARTY_NOTICES.md +47 -1
  7. package/bin/cozyclay.mjs +120 -50
  8. package/bin/mcp-runtime.mjs +115 -0
  9. package/bin/open-browser.mjs +10 -0
  10. package/bin/update-check.mjs +158 -0
  11. package/dist/ai-camera-control/index.html +406 -0
  12. package/dist/app/index.html +5 -5
  13. package/dist/ardy/assembled/run-jump-15s.npz +0 -0
  14. package/dist/assets/app-CqOk49Bk.css +1 -0
  15. package/dist/assets/app-zu633VCI.js +4811 -0
  16. package/dist/assets/vision_bundle-jFkh-fIS.js +41 -0
  17. package/dist/fonts/InstrumentSerif-OFL.txt +93 -0
  18. package/dist/fonts/Inter-OFL.txt +92 -0
  19. package/dist/fonts/README.md +15 -0
  20. package/dist/index.html +62 -17
  21. package/dist/sitemap.xml +7 -1
  22. package/mcp/LIVE-PROTOCOL.md +144 -0
  23. package/mcp/README.md +160 -0
  24. package/mcp/ardy-prompts.mjs +188 -0
  25. package/mcp/live-hub.mjs +319 -0
  26. package/mcp/package.json +27 -0
  27. package/mcp/runtime/package-lock.json +1211 -0
  28. package/mcp/runtime/package.json +16 -0
  29. package/mcp/server.mjs +2008 -0
  30. package/package.json +142 -90
  31. package/src/App.jsx +5475 -1028
  32. package/src/ardy/cskel27.js +7 -2
  33. package/src/ardy/ik.js +25 -15
  34. package/src/ardy/motion-edit.js +172 -0
  35. package/src/ardy/npz.js +64 -3
  36. package/src/ardy/playback.js +31 -1
  37. package/src/ardy/pose-pin.js +97 -0
  38. package/src/ardy/prompt-clips.js +7 -2
  39. package/src/ardy/retime.js +211 -0
  40. package/src/ardy/root-drop.js +180 -0
  41. package/src/ardy/timeline-coordinates.js +13 -0
  42. package/src/ardy/timeline-resize.js +3 -1
  43. package/src/ardy/timeline.jsx +471 -63
  44. package/src/ardy/to-cskel27.js +34 -12
  45. package/src/ardy/trim.js +33 -0
  46. package/src/ardy/waypoints.js +4 -1
  47. package/src/asset-pane.jsx +354 -0
  48. package/src/asset-shelf.js +82 -0
  49. package/src/camera-block.js +42 -0
  50. package/src/camera-follow.js +149 -27
  51. package/src/camera-move.js +45 -19
  52. package/src/camera-rail-schedule.js +6 -0
  53. package/src/cuts.js +67 -54
  54. package/src/dualview.jsx +58 -12
  55. package/src/generation/generation-request.js +17 -0
  56. package/src/generation/use-generation.js +3 -14
  57. package/src/hierarchy-model.js +100 -16
  58. package/src/hierarchy-panel.jsx +139 -6
  59. package/src/live-control.js +142 -0
  60. package/src/matte-editor.js +543 -0
  61. package/src/matte.js +503 -0
  62. package/src/mp4-muxer.js +52 -0
  63. package/src/multimodel-ingest.js +344 -0
  64. package/src/object-gizmo.jsx +248 -25
  65. package/src/offscreen-export.js +166 -0
  66. package/src/otio.js +217 -0
  67. package/src/planview.jsx +57 -38
  68. package/src/pose-extract/detector.js +80 -0
  69. package/src/pose-extract/image-frame.js +40 -0
  70. package/src/pose-extract/index.js +4 -0
  71. package/src/pose-extract/take.js +114 -0
  72. package/src/pose-extract/video-frames.js +91 -0
  73. package/src/pose-thumbs.js +152 -0
  74. package/src/poses.js +19 -265
  75. package/src/posestudio.jsx +441 -17
  76. package/src/project-browser.jsx +135 -0
  77. package/src/project-poses.js +9 -0
  78. package/src/project.js +373 -0
  79. package/src/props.jsx +69 -3
  80. package/src/room.jsx +89 -37
  81. package/src/sample-at.js +136 -0
  82. package/src/scene-asset-cache.js +166 -0
  83. package/src/scene-assets.js +378 -0
  84. package/src/scene-objects.js +308 -7
  85. package/src/scenes.js +235 -26
  86. package/src/shot-authoring.js +120 -41
  87. package/src/shot.js +53 -13
  88. package/src/source-offer.jsx +12 -0
  89. package/src/stable-items.js +83 -0
  90. package/src/styles.css +2283 -349
  91. package/src/ui.jsx +32 -4
  92. package/src/usd-camera.js +145 -0
  93. package/tools/ardy/BRIDGE.md +11 -7
  94. package/tools/ardy/README.md +24 -5
  95. package/tools/ardy/artifacts.mjs +34 -0
  96. package/tools/ardy/bridge.mjs +89 -12
  97. package/tools/ardy/bvh-cskel27.mjs +1209 -0
  98. package/tools/ardy/cclay_constrained_generate.py +123 -11
  99. package/tools/ardy/cclay_sequence_generate.py +49 -0
  100. package/tools/ardy/extract.mjs +372 -0
  101. package/tools/ardy/footage.mjs +467 -0
  102. package/tools/ardy/npz.mjs +74 -9
  103. package/tools/ardy/run-on-box.sh +25 -0
  104. package/tools/ardy/run-sequence-on-box.sh +15 -0
  105. package/tools/ardy/runners/local.mjs +4 -1
  106. package/tools/ardy/runners/remote.mjs +8 -2
  107. package/tools/ardy/setup-text-encoder.py +14 -4
  108. package/tools/dev-full.mjs +63 -5
  109. package/tools/generation/bridge.mjs +14 -1
  110. package/tools/process-supervisor.mjs +97 -1
  111. package/tools/run-tests.mjs +169 -0
  112. package/dist/assets/app-Cgpk2hwX.js +0 -4803
  113. package/dist/assets/app-DgZvaAE1.css +0 -1
package/dist/index.html CHANGED
@@ -3,7 +3,7 @@
3
3
  <head>
4
4
  <meta charset="UTF-8" />
5
5
  <meta name="viewport" content="width=device-width, initial-scale=1.0, viewport-fit=cover" />
6
- <meta name="description" content="CozyClay is an open-source, browser-based 3D staging and previz studio. Block a scene, pose characters, author camera moves and cuts, and preview generated motion — no Blender, no Unity, no install." />
6
+ <meta name="description" content="Free, open source previs software that runs in the browser. Block a scene, pose characters, and author camera moves and cuts, then hand the same shots to an AI video model. No Blender, no Unity, no install." />
7
7
  <meta name="theme-color" content="#232323" />
8
8
  <meta name="apple-mobile-web-app-title" content="CozyClay" />
9
9
  <link rel="canonical" href="https://cozyclay.org/" />
@@ -14,19 +14,19 @@
14
14
 
15
15
  <meta property="og:type" content="website" />
16
16
  <meta property="og:site_name" content="CozyClay" />
17
- <meta property="og:title" content="CozyClay — browser-based 3D staging and previz studio" />
17
+ <meta property="og:title" content="CozyClay — open source 3D previs software, in your browser" />
18
18
  <meta property="og:description" content="Block a scene, pose characters, author camera moves and cuts, and preview generated motion. Open source, runs in your browser." />
19
19
  <meta property="og:url" content="https://cozyclay.org/" />
20
20
  <meta property="og:image" content="https://cozyclay.org/media/cozyclay-demo-poster.jpg" />
21
21
  <meta property="og:image:width" content="1920" />
22
22
  <meta property="og:image:height" content="1080" />
23
- <meta property="og:image:alt" content="A CozyClay shot mid-wipe: the greybox previz on one half, the generated result on the other." />
23
+ <meta property="og:image:alt" content="A CozyClay shot mid-wipe: the greybox previs on one half, the generated result on the other." />
24
24
  <meta name="twitter:card" content="summary_large_image" />
25
- <meta name="twitter:title" content="CozyClay — browser-based 3D staging and previz studio" />
25
+ <meta name="twitter:title" content="CozyClay — open source 3D previs software, in your browser" />
26
26
  <meta name="twitter:description" content="Block a scene, pose characters, author camera moves and cuts, and preview generated motion. Open source, runs in your browser." />
27
27
  <meta name="twitter:image" content="https://cozyclay.org/media/cozyclay-demo-poster.jpg" />
28
28
 
29
- <title>CozyClay — browser-based 3D staging and previz studio</title>
29
+ <title>CozyClay — open source 3D previs software, in your browser</title>
30
30
 
31
31
  <meta property="og:video" content="https://cozyclay.org/media/cozyclay-demo.mp4" />
32
32
  <meta property="og:video:type" content="video/mp4" />
@@ -67,7 +67,23 @@
67
67
  "name": "What is CozyClay?",
68
68
  "acceptedAnswer": {
69
69
  "@type": "Answer",
70
- "text": "CozyClay is an open-source 3D staging and previsualization studio that runs in a browser. You block a scene with primitives and set pieces, pose characters, draw root paths, author camera moves and cuts on a timeline, and preview generated motion."
70
+ "text": "CozyClay is open source previs software that runs in a browser. You block a scene with primitives and set pieces, pose characters, draw root paths, author camera moves and cuts on a timeline, and preview generated motion."
71
+ }
72
+ },
73
+ {
74
+ "@type": "Question",
75
+ "name": "What is previs, and is it spelled previz?",
76
+ "acceptedAnswer": {
77
+ "@type": "Answer",
78
+ "text": "Previs is short for previsualization: the rough 3D pass where a shot gets blocked — camera position, lens, character movement, cut points — before anyone commits to final assets or render budget. It is written both previs and previz; the industry mostly writes previs, and both mean the same thing."
79
+ }
80
+ },
81
+ {
82
+ "@type": "Question",
83
+ "name": "Is there free previs software?",
84
+ "acceptedAnswer": {
85
+ "@type": "Answer",
86
+ "text": "CozyClay is free and open source under AGPL-3.0-or-later, and it runs in the browser. Run it locally with npx cozyclay and nothing leaves your machine. Modified network services must offer their users the corresponding source."
71
87
  }
72
88
  },
73
89
  {
@@ -221,7 +237,7 @@
221
237
  <a href="#what">What it does</a>
222
238
  <a href="#start">Quick start</a>
223
239
  <a href="#controls">Controls</a>
224
- <a href="#faq">FAQ</a>
240
+ <a href="/ai-camera-control/">AI camera control</a>
225
241
  <a href="https://github.com/HaD0Yun/CozyClay">GitHub</a>
226
242
  </nav>
227
243
  </div>
@@ -235,9 +251,9 @@
235
251
 
236
252
  <h1>Block the shot before you pay to render it</h1>
237
253
  <p class="lede">
238
- CozyClay is a browser-based 3D staging studio. Block a scene, pose characters,
239
- author camera moves and cuts, then hand the exact same shots to a video model.
240
- No Blender, no Unity, no install.
254
+ CozyClay is free, open source previs software that runs in your browser. Block a
255
+ scene, pose characters, and author camera moves and cuts — then hand the same
256
+ shots to an AI video model. No Blender, no Unity, no install.
241
257
  </p>
242
258
 
243
259
  <div class="reel">
@@ -249,8 +265,9 @@
249
265
  ></video>
250
266
  </div>
251
267
  <p class="reel-note">
252
- Thirty seconds: greybox previz on the left of the wipe, the generated result on the
253
- right. Every cut came out of the same camera setup you see being built.
268
+ Thirty seconds: greybox previs on the left of the wipe, the generated result on the
269
+ right. Every cut came out of the same camera setup you see being built — more on
270
+ <a href="/ai-camera-control/">camera control for AI video</a>.
254
271
  </p>
255
272
 
256
273
  <div class="cta">
@@ -266,7 +283,7 @@
266
283
  <section id="what">
267
284
  <h2>What you can do</h2>
268
285
  <p class="sub">
269
- Previz is the part of animation where you decide what happens before anyone commits
286
+ Previs is the part of animation where you decide what happens before anyone commits
270
287
  to final assets. CozyClay keeps that whole loop in one page.
271
288
  </p>
272
289
  <div class="grid">
@@ -355,9 +372,36 @@ npm run dev</code></pre>
355
372
  <details open>
356
373
  <summary>What is CozyClay?</summary>
357
374
  <p>
358
- CozyClay is an open-source 3D staging and previsualization studio that runs in a
359
- browser. You block a scene with primitives and set pieces, pose characters, draw
360
- root paths, author camera moves and cuts on a timeline, and preview generated motion.
375
+ CozyClay is open source previs software that runs in a browser. You block a scene
376
+ with primitives and set pieces, pose characters, draw root paths, author camera
377
+ moves and cuts on a timeline, and preview generated motion.
378
+ </p>
379
+ </details>
380
+ <details>
381
+ <summary>What is previs, and is it spelled previz?</summary>
382
+ <p>
383
+ Previs is short for previsualization: the rough 3D pass where a shot gets blocked —
384
+ camera position, lens, character movement, cut points — before anyone commits to
385
+ final assets or render budget. It is written both <strong>previs</strong> and
386
+ <strong>previz</strong>; the industry mostly writes previs, and both mean the same
387
+ thing.
388
+ </p>
389
+ </details>
390
+ <details>
391
+ <summary>Is there free previs software?</summary>
392
+ <p>
393
+ CozyClay is free and open source under AGPL-3.0-or-later, and it runs in the browser.
394
+ Run it locally with <code>npx cozyclay</code> and nothing leaves your machine. Modified
395
+ network services must offer their users the corresponding source.
396
+ </p>
397
+ </details>
398
+ <details>
399
+ <summary>Can I use it for camera control in AI video?</summary>
400
+ <p>
401
+ That is the reason it exists. Block the shot in 3D — angle, lens, dolly, the frame
402
+ the cut lands on — then take those camera moves to a video model instead of
403
+ describing them in a prompt and hoping. See
404
+ <a href="/ai-camera-control/">camera control for AI video</a>.
361
405
  </p>
362
406
  </details>
363
407
  <details>
@@ -402,7 +446,8 @@ npm run dev</code></pre>
402
446
  <footer class="site">
403
447
  <div class="wrap">
404
448
  <p>
405
- <a href="https://github.com/HaD0Yun/CozyClay">GitHub</a> ·
449
+ <a href="https://github.com/HaD0Yun/CozyClay">Source code (AGPL)</a> ·
450
+ <a href="LICENSE">License</a> ·
406
451
  <a href="https://www.npmjs.com/package/cozyclay">npm</a>
407
452
  </p>
408
453
  <p>
package/dist/sitemap.xml CHANGED
@@ -2,8 +2,14 @@
2
2
  <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
3
3
  <url>
4
4
  <loc>https://cozyclay.org/</loc>
5
- <lastmod>2026-08-14</lastmod>
5
+ <lastmod>2026-08-15</lastmod>
6
6
  <changefreq>weekly</changefreq>
7
7
  <priority>1.0</priority>
8
8
  </url>
9
+ <url>
10
+ <loc>https://cozyclay.org/ai-camera-control/</loc>
11
+ <lastmod>2026-08-15</lastmod>
12
+ <changefreq>monthly</changefreq>
13
+ <priority>0.8</priority>
14
+ </url>
9
15
  </urlset>
@@ -0,0 +1,144 @@
1
+ # CozyClay live-control protocol v1
2
+
3
+ The MCP JSON-RPC endpoint deliberately negotiates only protocol version
4
+ `2025-11-25`. Older and draft tracks are rejected during `initialize`.
5
+
6
+ One WebSocket, JSON text frames. The **MCP server hosts** the socket
7
+ (`ws://127.0.0.1:5184/live`); the **editor is the client** and reconnects
8
+ every 3 s while the page is open. Either side may be absent: the editor works
9
+ exactly as before when nothing is listening, and the MCP server falls back to
10
+ its in-memory scene when no editor is connected.
11
+
12
+ ## Frames
13
+
14
+ editor -> server, once after connect:
15
+
16
+ { "type": "hello", "role": "editor", "version": 1 }
17
+
18
+ server -> editor, once after `hello`:
19
+
20
+ { "type": "workspace", "handle": "<opaque workspace handle>" }
21
+
22
+ The handle is newly issued for this socket connection. The editor surfaces it to
23
+ its operator. A disconnect invalidates it; a reconnect receives a fresh handle.
24
+ The editor also sends its stable per-tab `workspaceId` in `hello`; it is not an
25
+ MCP tool argument and lets the hub route retained terminal motion outcomes to
26
+ the same editor after that fresh handle is issued.
27
+
28
+ server -> editor, one per command:
29
+
30
+ { "type": "cmd", "id": "<opaque string>", "name": "<command>", "args": { ... } }
31
+
32
+ server -> editor, for an MCP-owned motion job state or terminal outcome (and again
33
+ on reconnect while retained). A completed event follows the editor's successful
34
+ `load_motion` acknowledgement:
35
+
36
+ { "type": "event", "name": "motion_job", "payload": { "taskId", "status", "createdAt", "lastUpdatedAt", "ttlMs", "pollIntervalMs", "outcome?" } }
37
+
38
+ editor -> server, to cancel its own active job before editor delivery:
39
+
40
+ { "type": "event", "name": "motion_job_cancel", "payload": { "taskId": "<opaque task id>" } }
41
+
42
+ editor -> server, one per command, echoing `id`:
43
+
44
+ { "type": "result", "id": "<same id>", "ok": true, "value": { ... } }
45
+ { "type": "result", "id": "<same id>", "ok": false, "error": "<human message>" }
46
+
47
+ Unknown `name` MUST answer `ok:false`, never silence. The server times a
48
+ command out after 5 s and treats it as failed, except `load_motion`, which
49
+ retains its dedicated 30 s editor decode and installation bound. Measurement
50
+ justifies the existing headroom: 12 real editor installs of one 94,672-byte
51
+ ARDY NPZ had nearest-rank p50 29.96 ms, p95 547.29 ms, and p99 547.29 ms;
52
+ 30 s remains appropriate for materially larger cold-cache production takes. A timeout or disconnect during
53
+ a mutation is ambiguous: it may already have applied, so callers MUST NOT retry
54
+ blindly and should `describe` before recovering.
55
+
56
+ ## Motion jobs
57
+
58
+ `generate_motion` returns immediately with exactly `{ taskId, status, createdAt,
59
+ lastUpdatedAt, ttlMs, pollIntervalMs }`; `pollIntervalMs` is `0` because the
60
+ model never polls it. The MCP server retains terminal outcomes for `ttlMs`
61
+ (currently 10 minutes) and pushes `motion_job` to the matching stable workspace
62
+ on completion, failure, cancellation, or reconnect. For a completed job, the
63
+ server first installs the take through the request/response `load_motion`
64
+ command, then publishes the terminal lifecycle event. The event never installs
65
+ the take a second time. A retained outcome that passes its TTL is explicitly
66
+ expired and is never replayed.
67
+
68
+ If the editor explicitly rejects `load_motion`, the job transitions to `failed`
69
+ and publishes that terminal event. If the socket disconnects or times out while
70
+ installation is in flight, the job also fails but is not retried: the mutation
71
+ may already have applied, so retrying could install the same take twice.
72
+
73
+ Cancellation is cooperative at the bridge HTTP stream: `motion_job_cancel`
74
+ aborts that request. The ARDY bridge detects the disconnected stream and kills
75
+ its request-owned child process groups. A cancelled job is terminal and is never
76
+ sent to editor installation. There are no `tasks/*` tools or start/status/list
77
+ polling tools in the MCP surface.
78
+
79
+ ## Commands (v1)
80
+
81
+ All coordinates are metres, rotations are degrees of yaw, ids are the ids the
82
+ scene document already uses.
83
+
84
+ | name | args | value | notes |
85
+ | --- | --- | --- | --- |
86
+ | `ping` | `{}` | `{ "pong": true }` | liveness |
87
+ | `describe` | `{}` | `{ document, sceneName, camera, stage, timeline, activeCharacterId, characters, objects }` | read the full live scene document plus active-scene convenience fields; the full `document` prevents one workspace's inactive scenes from leaking into another workspace's scene operations; `activeCharacterId` pins asynchronous work to the character selected when it started |
88
+ | `capture_frame` | `{}` | `{ width: 640, height: 360, mimeType: "image/png", encoding: "base64", byteSize, data, assertions: { renderable, blackFrame, nonBlackPixels, behindCameraPlane, fartherAlongCameraForward, distanceToFloor, occludedBy, visiblePixelCount, characters } }` | leaves the authored document untouched, but is classified open-world/non-idempotent because an oversized PNG creates a mode-0600 managed temporary artifact. Character visibility and occlusion are computed from mounted engine geometry with bounded ray samples. Missing camera, black frame and compressed payloads above 1 MB fail explicitly. Inline responses honor `max_inline_bytes`; managed artifacts are capped at 20 and expire after 10 minutes. |
89
+ | `set_camera` | `{ x?, y?, z?, focalMm? }` | `{ camera }` | omitted fields keep their value; the **viewport must visibly move** |
90
+ | `add_character` | `{ subject, x?, z?, rot?, model? }` | `{ id }` | `model` is one of the stable character model ids |
91
+ | `update_character` | `{ ref, x?, y?, z?, rot?, subject?, hidden? }` | `{ id }` | `ref` = id, letter (`"A"`) or 1-based slot |
92
+ | `remove_character` | `{ ref }` | `{ id }` | must refuse to empty the cast |
93
+ | `place_object` | `{ kind, x?, z?, y?, rot?, name?, parent? }` | `{ id }` | `kind` from OBJECT_LIBRARY; optional `name` labels the object and optional `parent` attaches it under another object |
94
+ | `update_object` | `{ id, x?, y?, z?, rot?, rotX?, rotZ?, scale?, scaleX?, scaleY?, scaleZ?, color?, name? }` | `{ id }` | `scale` sets all three axes; per-axis values override it; `name` renames the object |
95
+ | `remove_object` | `{ id }` | `{ id }` | |
96
+ | `group_objects` | `{ parent, children }` | `{ parent, children }` | attach every child under parent |
97
+ | `ungroup_objects` | `{ children }` | `{ children }` | detach every child |
98
+ | `apply_batch` | `{ ops, atomic?: false, stopOnError?: true, label?: "MCP batch" }` | `{ label, applied: number[], failed: [{ index, error }], rolledBack }` | executes at most 100 object mutations as one undo entry. `atomic` and `stopOnError` are independent; atomic failure restores the pre-batch objects and creates no undo entry. Nested batches are rejected. v1 rejects character mutations because cast history is a separate store. |
99
+ | `set_prompt_blocks` | `{ blocks: [{ startFrame, endFrame, text }] }` | `{ blocks }` | replace the active character's authored prompt clips after validating each frame range and text |
100
+ | `load_motion` | `{ url, prompt?, blocks?, drop? }` | `{ loaded, url, blocks }` | install an ARDY motion on the selected character. It has the dedicated 30-second editor-processing timeout; motion-job completion carries the character id captured when generation started |
101
+ | `load_scenes` | `{ document }` | `{ sceneName, activeSceneId, scenes: [{ id, name }] }` | replace the whole scene document (same shape `serializeSceneDocument` emits); the response attests the full scene list and active scene for `add_scene` and `switch_scene` parity |
102
+
103
+ ## Hard rules for the editor side
104
+
105
+ - Every mutation MUST go through the same React state paths the UI itself
106
+ uses (the setters/reducers the gizmo, inspector and panels call). Mutating
107
+ three.js objects directly is forbidden: anything outside React state is
108
+ overwritten on the next render.
109
+ - Every mutation MUST land in undo history exactly like the equivalent UI
110
+ action would, or be explicitly documented as not undoable.
111
+ - The socket client MUST be a no-op in production builds unless explicitly
112
+ enabled; in dev it may always try. A failed connection must never surface
113
+ an error to the user - silence and retry.
114
+
115
+ ## Workspace routing
116
+
117
+ The hub allows multiple editor instances at once; it never displaces an existing
118
+ editor for a newer connection. `live_status` lists every current workspace
119
+ handle. Every MCP tool that reads or mutates a live editor accepts `workspace_handle`.
120
+ When exactly one editor is connected, omitting it selects that editor. With two
121
+ or more editors, omitting it fails before dispatch and enumerates every candidate
122
+ handle. An unknown or disconnected handle fails as unknown or stale and is never
123
+ routed to another editor. The hub owns this resolution rule for every transport;
124
+ there is no last-active, heartbeat, focus, or recency fallback.
125
+
126
+ ## Hard rules for the server side
127
+
128
+ - When an editor is connected, live-capable tools forward to the workspace
129
+ selected by the rules above and answer from that workspace's `describe`; when
130
+ none is connected they fall back to the in-memory scene exactly as today.
131
+ - `add_scene` and `switch_scene` forward the complete scene document through
132
+ `load_scenes` and verify it through `describe` before reporting success. They
133
+ never succeed while the MCP server and selected editor have different scene
134
+ lists. Without an editor they retain the in-memory fallback.
135
+ - A live mutation may only target the explicitly selected workspace, except for
136
+ the exactly-one-editor auto-selection case. Ambiguity is an error, never a
137
+ guess.
138
+
139
+ Browser editor connections are accepted only from loopback HTTP origins, and a
140
+ stable workspace id may have only one live socket. Native loopback clients have
141
+ no Origin header and remain part of the trusted-local MCP boundary.
142
+ - Any bounded MCP read reports `total`, `returned`, `truncated`, and a
143
+ `revision`. Omitted entries must be reachable by an explicit selector or
144
+ cursor; `describe_scene` uses `character_cursor` and `object_cursor`.
package/mcp/README.md ADDED
@@ -0,0 +1,160 @@
1
+ # CozyClay MCP server
2
+
3
+ Lets an AI assistant (Claude Desktop, Cursor, any MCP client) block a scene, place the camera,
4
+ generate character motion and read the shot back as film vocabulary — then turn it into an AI
5
+ image/video prompt.
6
+
7
+ Works two ways, with one tool catalog:
8
+
9
+ - **Editor open** — tool calls drive the visible viewport live: camera, cast, set, motion,
10
+ prompt blocks on the timeline.
11
+ - **No editor** — scene, camera, prompt and project-file tools run in memory with no browser,
12
+ GPU or build step. `capture_frame`, `set_prompt_blocks`, `generate_motion`, and
13
+ `apply_batch` explicitly require a live editor.
14
+
15
+ ## Run it locally
16
+
17
+ ```sh
18
+ cd mcp
19
+ npm install
20
+ npm start # speaks MCP over stdio and hosts ws://127.0.0.1:5184/live
21
+ npm run verify # drives the no-editor fallback as a client and checks results
22
+ npm run verify:live # drives a fake editor over the live WebSocket protocol
23
+ ```
24
+
25
+ Then point a client at it. For Claude Desktop, in `claude_desktop_config.json`:
26
+
27
+ ```json
28
+ {
29
+ "mcpServers": {
30
+ "cozyclay": {
31
+ "command": "node",
32
+ "args": ["/absolute/path/to/CozyClay/mcp/server.mjs"]
33
+ }
34
+ }
35
+ }
36
+ ```
37
+
38
+ Restart the client; 24 tools appear.
39
+
40
+ Prefer a long-lived endpoint? `node server.mjs --http 5183` serves Streamable HTTP at
41
+ `http://127.0.0.1:5183/mcp` (one isolated session per client), with a plain status page at `/`.
42
+
43
+ ## What it does
44
+
45
+ Describe the shot you want. The server does the trigonometry and answers the way a crew would.
46
+
47
+ > "Put a detective and a courier in an alley, then give me a low wide profile shot of the courier."
48
+
49
+ ```
50
+ WIDE SHOT · RIGHT PROFILE · KNEE LEVEL · 24MM
51
+
52
+ size wide shot — the subject fills 40% of frame height
53
+ view a right-side profile view
54
+ level a low knee-level angle looking up at the subject
55
+ distance 4.39m
56
+ ```
57
+
58
+ `render_prompt` turns that same geometry into a prompt that carries the real framing, so the
59
+ generated frame matches the blocking instead of drifting off into a generic shot.
60
+
61
+ ## Tools
62
+
63
+ | tool | what it does |
64
+ | --- | --- |
65
+ | `describe_scene` | the whole state: camera, framing, cast, set |
66
+ | `live_status` | whether an editor tab is connected for live control |
67
+ | `describe_shot` | current camera geometry as film vocabulary |
68
+ | `capture_frame` | a compressed live render with camera-plane, visibility and occlusion assertions |
69
+ | `set_camera` | move the lens / change focal length directly |
70
+ | `frame_shot` | frame by intent — size, view, level, side |
71
+ | `add_character` / `place_character` / `remove_character` | the cast |
72
+ | `focus_character` | choose who the camera frames |
73
+ | `place_object` / `update_object` / `remove_object` | the set — `place_object` also accepts `name` and `parent`, so multi-part assets like "Building A" land as one named assembly |
74
+ | `group_objects` | attach children to a parent so they move as one; pass `parent: null` to detach |
75
+ | `apply_batch` | execute up to 100 object mutations as one user-visible undo transaction |
76
+ | `render_prompt` | the shot as an AI image or video prompt |
77
+ | `generate_motion` | multi-phase character motion through the ARDY bridge — phases land as prompt blocks |
78
+ | `mark_camera_move` / `describe_camera_move` | name a move between two camera positions |
79
+ | `add_scene` / `switch_scene` | multiple scenes per project |
80
+ | `open_project` / `save_project` | read and write `.cclayproject` files |
81
+
82
+ Project file tools are restricted to `COZYCLAY_PROJECT_ROOT` (the server
83
+ working directory by default), accept direct child `.cclayproject` files only,
84
+ anchor relative opens to the retained project-root working-directory inode, reject symlink escapes,
85
+ open the final name with `O_NOFOLLOW`, and require
86
+ `overwrite: true` before replacing an existing regular file.
87
+
88
+ Coordinates are metres (`x` right, `z` toward the default camera, `y` height above the floor).
89
+ Rotations are degrees of yaw. Characters are addressed by letter (`"A"`), slot (`"2"`) or id
90
+ (`"char-a"`).
91
+
92
+ ## Live editor control
93
+
94
+ The server hosts `ws://127.0.0.1:5184/live` by default (`COZYCLAY_LIVE_PORT` or
95
+ `--live-port <port>` to change it). Start the editor with that same `COZYCLAY_LIVE_PORT` so its
96
+ Vite build connects to the selected loopback endpoint; `npm run dev` preserves the environment
97
+ variable for the browser. A CozyClay editor tab then connects on its own; `live_status` tells you whether one is attached. With an editor connected, mutations
98
+ forward to it and reads report its real state — the screen you are looking at is the source of
99
+ truth. Without one, headless-capable tools keep their in-memory behaviour while `capture_frame`
100
+ returns an explicit connected-editor error. The other three live-only tools return a readable
101
+ ordinary response explaining that an editor is required. In `--http` mode each session
102
+ child attempts to bind the live port, so one session owns the editor and the others
103
+ intentionally run memory-only.
104
+
105
+ The wire protocol — one WebSocket, sixteen commands, editor-side rules — is specified in
106
+ [`LIVE-PROTOCOL.md`](LIVE-PROTOCOL.md).
107
+
108
+ ## Motion generation
109
+
110
+ `generate_motion` takes plain-language beats and a length:
111
+
112
+ ```
113
+ phases: ["seated on a chair, slowly stands up",
114
+ "breaks into a sprint",
115
+ "trips and falls hard to the ground"]
116
+ seconds: 10
117
+ ```
118
+
119
+ It tiles them into contiguous ARDY segments, streams the generation through the local bridge
120
+ (`127.0.0.1:5181`, started by `npm run dev`), and — when an editor is connected — loads the
121
+ result onto the active character with prompt blocks on the timeline. Beats longer than the
122
+ per-block limit are split into consecutive blocks, so one phase can produce multiple blocks.
123
+ Pass a previous `motion_url` to reload a clip without generating again.
124
+
125
+ ## Round-trips with the studio
126
+
127
+ `save_project` writes a real `.cclayproject` file, so a scene blocked here opens in the studio to
128
+ be posed, timed and generated — and a scene built in the studio opens here with `open_project`.
129
+ The configured project-root directory must only be writable by the trusted user running the MCP
130
+ server; project I/O rejects symlinks and hard-linked project files but relies on that directory
131
+ permission boundary to prevent an untrusted local process from adding links during an operation.
132
+
133
+ ## Design
134
+
135
+ This server owns no geometry, no film vocabulary and no prompt text. Every answer is computed by
136
+ the same modules the studio renders with, imported straight from the working tree:
137
+
138
+ | module | responsibility |
139
+ | --- | --- |
140
+ | `../src/shot.js` | geometry → film vocabulary → prompt |
141
+ | `../src/scenes.js` | the scene document and its stage envelope |
142
+ | `../src/scene-objects.js` | the set: create / update / remove, clamped and snapped |
143
+ | `../src/camera-move.js` | two framings → a named camera move |
144
+ | `../src/project.js` | the `.cclayproject` envelope |
145
+
146
+ That is the whole design. Because the imports are relative, the server always speaks the working
147
+ tree's vocabulary: retune a band in `shot.js` and this server reports the new answer on its next
148
+ start, with nothing to publish or reinstall. The studio and the MCP server cannot disagree about
149
+ what a 35mm medium shot is, because there is only one implementation of it.
150
+
151
+ `frame_shot` is the one place that inverts `shot.js` rather than calling it — it solves for the
152
+ camera position that produces a requested size. `npm run verify` checks all 420 combinations the
153
+ schema accepts, so a retune in `shot.js` fails loudly here instead of quietly mis-framing.
154
+
155
+ ### Framing conflicts
156
+
157
+ Shot size is treated as the stronger request. An extreme close-up from an overhead lens is
158
+ geometrically impossible on a wide lens — the camera would sit inside the subject — so the server
159
+ lengthens the lens until the requested angle fits and says so, the way a crew swaps glass rather
160
+ than abandoning the close-up.