cozyclay 1.2.0 → 1.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (72) hide show
  1. package/CHANGELOG.md +85 -0
  2. package/README.md +31 -0
  3. package/THIRD_PARTY_NOTICES.md +37 -1
  4. package/bin/cozyclay.mjs +51 -2
  5. package/dist/ai-camera-control/index.html +406 -0
  6. package/dist/app/index.html +5 -5
  7. package/dist/assets/app-B3U5aut1.js +4811 -0
  8. package/dist/assets/app-BrRF0wso.css +1 -0
  9. package/dist/assets/vision_bundle-jFkh-fIS.js +41 -0
  10. package/dist/fonts/InstrumentSerif-OFL.txt +93 -0
  11. package/dist/fonts/Inter-OFL.txt +92 -0
  12. package/dist/fonts/README.md +15 -0
  13. package/dist/index.html +60 -16
  14. package/dist/sitemap.xml +7 -1
  15. package/mcp/LIVE-PROTOCOL.md +63 -0
  16. package/mcp/README.md +142 -0
  17. package/mcp/ardy-prompts.mjs +170 -0
  18. package/mcp/live-hub.mjs +105 -0
  19. package/mcp/package.json +24 -0
  20. package/mcp/server.mjs +1394 -0
  21. package/package.json +122 -90
  22. package/src/App.jsx +2957 -514
  23. package/src/ardy/cskel27.js +7 -2
  24. package/src/ardy/ik.js +25 -15
  25. package/src/ardy/npz.js +64 -3
  26. package/src/ardy/playback.js +31 -1
  27. package/src/ardy/prompt-clips.js +7 -2
  28. package/src/ardy/retime.js +211 -0
  29. package/src/ardy/timeline-coordinates.js +13 -0
  30. package/src/ardy/timeline.jsx +124 -5
  31. package/src/ardy/to-cskel27.js +34 -12
  32. package/src/ardy/trim.js +33 -0
  33. package/src/asset-pane.jsx +36 -0
  34. package/src/dualview.jsx +14 -8
  35. package/src/hierarchy-model.js +95 -13
  36. package/src/hierarchy-panel.jsx +139 -6
  37. package/src/live-control.js +122 -0
  38. package/src/matte-editor.js +543 -0
  39. package/src/matte.js +503 -0
  40. package/src/multimodel-ingest.js +344 -0
  41. package/src/object-gizmo.jsx +43 -15
  42. package/src/planview.jsx +43 -32
  43. package/src/pose-extract/detector.js +75 -0
  44. package/src/pose-extract/index.js +3 -0
  45. package/src/pose-extract/take.js +87 -0
  46. package/src/pose-extract/video-frames.js +91 -0
  47. package/src/pose-thumbs.js +152 -0
  48. package/src/posestudio.jsx +361 -11
  49. package/src/project-browser.jsx +135 -0
  50. package/src/project.js +289 -0
  51. package/src/props.jsx +69 -3
  52. package/src/room.jsx +14 -35
  53. package/src/scene-asset-cache.js +125 -0
  54. package/src/scene-assets.js +288 -0
  55. package/src/scene-objects.js +245 -7
  56. package/src/scenes.js +207 -26
  57. package/src/shot-authoring.js +55 -13
  58. package/src/styles.css +1296 -129
  59. package/tools/ardy/BRIDGE.md +3 -2
  60. package/tools/ardy/README.md +9 -5
  61. package/tools/ardy/bridge.mjs +57 -1
  62. package/tools/ardy/bvh-cskel27.mjs +1209 -0
  63. package/tools/ardy/cclay_constrained_generate.py +123 -11
  64. package/tools/ardy/cclay_sequence_generate.py +49 -0
  65. package/tools/ardy/extract.mjs +367 -0
  66. package/tools/ardy/footage.mjs +462 -0
  67. package/tools/ardy/npz.mjs +74 -9
  68. package/tools/ardy/run-on-box.sh +25 -0
  69. package/tools/ardy/run-sequence-on-box.sh +15 -0
  70. package/tools/ardy/runners/remote.mjs +8 -2
  71. package/dist/assets/app-Cgpk2hwX.js +0 -4803
  72. package/dist/assets/app-DgZvaAE1.css +0 -1
@@ -0,0 +1,93 @@
1
+ Copyright 2022 The Instrument Serif Project Authors (https://github.com/Instrument/instrument-serif)
2
+
3
+ This Font Software is licensed under the SIL Open Font License, Version 1.1.
4
+ This license is copied below, and is also available with a FAQ at:
5
+ https://scripts.sil.org/OFL
6
+
7
+
8
+ -----------------------------------------------------------
9
+ SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007
10
+ -----------------------------------------------------------
11
+
12
+ PREAMBLE
13
+ The goals of the Open Font License (OFL) are to stimulate worldwide
14
+ development of collaborative font projects, to support the font creation
15
+ efforts of academic and linguistic communities, and to provide a free and
16
+ open framework in which fonts may be shared and improved in partnership
17
+ with others.
18
+
19
+ The OFL allows the licensed fonts to be used, studied, modified and
20
+ redistributed freely as long as they are not sold by themselves. The
21
+ fonts, including any derivative works, can be bundled, embedded,
22
+ redistributed and/or sold with any software provided that any reserved
23
+ names are not used by derivative works. The fonts and derivatives,
24
+ however, cannot be released under any other type of license. The
25
+ requirement for fonts to remain under this license does not apply
26
+ to any document created using the fonts or their derivatives.
27
+
28
+ DEFINITIONS
29
+ "Font Software" refers to the set of files released by the Copyright
30
+ Holder(s) under this license and clearly marked as such. This may
31
+ include source files, build scripts and documentation.
32
+
33
+ "Reserved Font Name" refers to any names specified as such after the
34
+ copyright statement(s).
35
+
36
+ "Original Version" refers to the collection of Font Software components as
37
+ distributed by the Copyright Holder(s).
38
+
39
+ "Modified Version" refers to any derivative made by adding to, deleting,
40
+ or substituting -- in part or in whole -- any of the components of the
41
+ Original Version, by changing formats or by porting the Font Software to a
42
+ new environment.
43
+
44
+ "Author" refers to any designer, engineer, programmer, technical
45
+ writer or other person who contributed to the Font Software.
46
+
47
+ PERMISSION & CONDITIONS
48
+ Permission is hereby granted, free of charge, to any person obtaining
49
+ a copy of the Font Software, to use, study, copy, merge, embed, modify,
50
+ redistribute, and sell modified and unmodified copies of the Font
51
+ Software, subject to the following conditions:
52
+
53
+ 1) Neither the Font Software nor any of its individual components,
54
+ in Original or Modified Versions, may be sold by itself.
55
+
56
+ 2) Original or Modified Versions of the Font Software may be bundled,
57
+ redistributed and/or sold with any software, provided that each copy
58
+ contains the above copyright notice and this license. These can be
59
+ included either as stand-alone text files, human-readable headers or
60
+ in the appropriate machine-readable metadata fields within text or
61
+ binary files as long as those fields can be easily viewed by the user.
62
+
63
+ 3) No Modified Version of the Font Software may use the Reserved Font
64
+ Name(s) unless explicit written permission is granted by the corresponding
65
+ Copyright Holder. This restriction only applies to the primary font name as
66
+ presented to the users.
67
+
68
+ 4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font
69
+ Software shall not be used to promote, endorse or advertise any
70
+ Modified Version, except to acknowledge the contribution(s) of the
71
+ Copyright Holder(s) and the Author(s) or with their explicit written
72
+ permission.
73
+
74
+ 5) The Font Software, modified or unmodified, in part or in whole,
75
+ must be distributed entirely under this license, and must not be
76
+ distributed under any other license. The requirement for fonts to
77
+ remain under this license does not apply to any document created
78
+ using the Font Software.
79
+
80
+ TERMINATION
81
+ This license becomes null and void if any of the above conditions are
82
+ not met.
83
+
84
+ DISCLAIMER
85
+ THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
86
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF
87
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
88
+ OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE
89
+ COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
90
+ INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL
91
+ DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
92
+ FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM
93
+ OTHER DEALINGS IN THE FONT SOFTWARE.
@@ -0,0 +1,92 @@
1
+ Copyright (c) 2016 The Inter Project Authors (https://github.com/rsms/inter)
2
+
3
+ This Font Software is licensed under the SIL Open Font License, Version 1.1.
4
+ This license is copied below, and is also available with a FAQ at:
5
+ http://scripts.sil.org/OFL
6
+
7
+ -----------------------------------------------------------
8
+ SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007
9
+ -----------------------------------------------------------
10
+
11
+ PREAMBLE
12
+ The goals of the Open Font License (OFL) are to stimulate worldwide
13
+ development of collaborative font projects, to support the font creation
14
+ efforts of academic and linguistic communities, and to provide a free and
15
+ open framework in which fonts may be shared and improved in partnership
16
+ with others.
17
+
18
+ The OFL allows the licensed fonts to be used, studied, modified and
19
+ redistributed freely as long as they are not sold by themselves. The
20
+ fonts, including any derivative works, can be bundled, embedded,
21
+ redistributed and/or sold with any software provided that any reserved
22
+ names are not used by derivative works. The fonts and derivatives,
23
+ however, cannot be released under any other type of license. The
24
+ requirement for fonts to remain under this license does not apply
25
+ to any document created using the fonts or their derivatives.
26
+
27
+ DEFINITIONS
28
+ "Font Software" refers to the set of files released by the Copyright
29
+ Holder(s) under this license and clearly marked as such. This may
30
+ include source files, build scripts and documentation.
31
+
32
+ "Reserved Font Name" refers to any names specified as such after the
33
+ copyright statement(s).
34
+
35
+ "Original Version" refers to the collection of Font Software components as
36
+ distributed by the Copyright Holder(s).
37
+
38
+ "Modified Version" refers to any derivative made by adding to, deleting,
39
+ or substituting -- in part or in whole -- any of the components of the
40
+ Original Version, by changing formats or by porting the Font Software to a
41
+ new environment.
42
+
43
+ "Author" refers to any designer, engineer, programmer, technical
44
+ writer or other person who contributed to the Font Software.
45
+
46
+ PERMISSION AND CONDITIONS
47
+ Permission is hereby granted, free of charge, to any person obtaining
48
+ a copy of the Font Software, to use, study, copy, merge, embed, modify,
49
+ redistribute, and sell modified and unmodified copies of the Font
50
+ Software, subject to the following conditions:
51
+
52
+ 1) Neither the Font Software nor any of its individual components,
53
+ in Original or Modified Versions, may be sold by itself.
54
+
55
+ 2) Original or Modified Versions of the Font Software may be bundled,
56
+ redistributed and/or sold with any software, provided that each copy
57
+ contains the above copyright notice and this license. These can be
58
+ included either as stand-alone text files, human-readable headers or
59
+ in the appropriate machine-readable metadata fields within text or
60
+ binary files as long as those fields can be easily viewed by the user.
61
+
62
+ 3) No Modified Version of the Font Software may use the Reserved Font
63
+ Name(s) unless explicit written permission is granted by the corresponding
64
+ Copyright Holder. This restriction only applies to the primary font name as
65
+ presented to the users.
66
+
67
+ 4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font
68
+ Software shall not be used to promote, endorse or advertise any
69
+ Modified Version, except to acknowledge the contribution(s) of the
70
+ Copyright Holder(s) and the Author(s) or with their explicit written
71
+ permission.
72
+
73
+ 5) The Font Software, modified or unmodified, in part or in whole,
74
+ must be distributed entirely under this license, and must not be
75
+ distributed under any other license. The requirement for fonts to
76
+ remain under this license does not apply to any document created
77
+ using the Font Software.
78
+
79
+ TERMINATION
80
+ This license becomes null and void if any of the above conditions are
81
+ not met.
82
+
83
+ DISCLAIMER
84
+ THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
85
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF
86
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
87
+ OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE
88
+ COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
89
+ INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL
90
+ DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
91
+ FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM
92
+ OTHER DEALINGS IN THE FONT SOFTWARE.
@@ -0,0 +1,15 @@
1
+ # Bundled fonts
2
+
3
+ The `.woff2` files in this directory are redistributed with CozyClay under the
4
+ SIL Open Font License 1.1, which permits bundling them with software provided
5
+ each copy carries the copyright notice and the licence. Those are the two
6
+ `-OFL.txt` files here; keep them alongside the fonts.
7
+
8
+ | File | Family | Copyright | Licence |
9
+ | --- | --- | --- | --- |
10
+ | `inter-latin.woff2` | [Inter](https://github.com/rsms/inter) | Copyright (c) 2016 The Inter Project Authors | [OFL-1.1](Inter-OFL.txt) |
11
+ | `instrument-serif-latin.woff2`, `instrument-serif-italic-latin.woff2` | [Instrument Serif](https://github.com/Instrument/instrument-serif) | Copyright 2022 The Instrument Serif Project Authors | [OFL-1.1](InstrumentSerif-OFL.txt) |
12
+
13
+ Both are subsets. Under OFL a modified copy may not use the Reserved Font Name,
14
+ so if a subset is ever renamed or edited beyond subsetting, check clause 3 before
15
+ shipping it.
package/dist/index.html CHANGED
@@ -3,7 +3,7 @@
3
3
  <head>
4
4
  <meta charset="UTF-8" />
5
5
  <meta name="viewport" content="width=device-width, initial-scale=1.0, viewport-fit=cover" />
6
- <meta name="description" content="CozyClay is an open-source, browser-based 3D staging and previz studio. Block a scene, pose characters, author camera moves and cuts, and preview generated motion — no Blender, no Unity, no install." />
6
+ <meta name="description" content="Free, open source previs software that runs in the browser. Block a scene, pose characters, and author camera moves and cuts, then hand the same shots to an AI video model. No Blender, no Unity, no install." />
7
7
  <meta name="theme-color" content="#232323" />
8
8
  <meta name="apple-mobile-web-app-title" content="CozyClay" />
9
9
  <link rel="canonical" href="https://cozyclay.org/" />
@@ -14,19 +14,19 @@
14
14
 
15
15
  <meta property="og:type" content="website" />
16
16
  <meta property="og:site_name" content="CozyClay" />
17
- <meta property="og:title" content="CozyClay — browser-based 3D staging and previz studio" />
17
+ <meta property="og:title" content="CozyClay — open source 3D previs software, in your browser" />
18
18
  <meta property="og:description" content="Block a scene, pose characters, author camera moves and cuts, and preview generated motion. Open source, runs in your browser." />
19
19
  <meta property="og:url" content="https://cozyclay.org/" />
20
20
  <meta property="og:image" content="https://cozyclay.org/media/cozyclay-demo-poster.jpg" />
21
21
  <meta property="og:image:width" content="1920" />
22
22
  <meta property="og:image:height" content="1080" />
23
- <meta property="og:image:alt" content="A CozyClay shot mid-wipe: the greybox previz on one half, the generated result on the other." />
23
+ <meta property="og:image:alt" content="A CozyClay shot mid-wipe: the greybox previs on one half, the generated result on the other." />
24
24
  <meta name="twitter:card" content="summary_large_image" />
25
- <meta name="twitter:title" content="CozyClay — browser-based 3D staging and previz studio" />
25
+ <meta name="twitter:title" content="CozyClay — open source 3D previs software, in your browser" />
26
26
  <meta name="twitter:description" content="Block a scene, pose characters, author camera moves and cuts, and preview generated motion. Open source, runs in your browser." />
27
27
  <meta name="twitter:image" content="https://cozyclay.org/media/cozyclay-demo-poster.jpg" />
28
28
 
29
- <title>CozyClay — browser-based 3D staging and previz studio</title>
29
+ <title>CozyClay — open source 3D previs software, in your browser</title>
30
30
 
31
31
  <meta property="og:video" content="https://cozyclay.org/media/cozyclay-demo.mp4" />
32
32
  <meta property="og:video:type" content="video/mp4" />
@@ -67,7 +67,23 @@
67
67
  "name": "What is CozyClay?",
68
68
  "acceptedAnswer": {
69
69
  "@type": "Answer",
70
- "text": "CozyClay is an open-source 3D staging and previsualization studio that runs in a browser. You block a scene with primitives and set pieces, pose characters, draw root paths, author camera moves and cuts on a timeline, and preview generated motion."
70
+ "text": "CozyClay is open source previs software that runs in a browser. You block a scene with primitives and set pieces, pose characters, draw root paths, author camera moves and cuts on a timeline, and preview generated motion."
71
+ }
72
+ },
73
+ {
74
+ "@type": "Question",
75
+ "name": "What is previs, and is it spelled previz?",
76
+ "acceptedAnswer": {
77
+ "@type": "Answer",
78
+ "text": "Previs is short for previsualization: the rough 3D pass where a shot gets blocked — camera position, lens, character movement, cut points — before anyone commits to final assets or render budget. It is written both previs and previz; the industry mostly writes previs, and both mean the same thing."
79
+ }
80
+ },
81
+ {
82
+ "@type": "Question",
83
+ "name": "Is there free previs software?",
84
+ "acceptedAnswer": {
85
+ "@type": "Answer",
86
+ "text": "CozyClay is free and open source under GPL-3.0, and it runs in the browser, so there is no licence and no install between you and a blocked shot. Run it locally with npx cozyclay and nothing leaves your machine."
71
87
  }
72
88
  },
73
89
  {
@@ -221,7 +237,7 @@
221
237
  <a href="#what">What it does</a>
222
238
  <a href="#start">Quick start</a>
223
239
  <a href="#controls">Controls</a>
224
- <a href="#faq">FAQ</a>
240
+ <a href="/ai-camera-control/">AI camera control</a>
225
241
  <a href="https://github.com/HaD0Yun/CozyClay">GitHub</a>
226
242
  </nav>
227
243
  </div>
@@ -235,9 +251,9 @@
235
251
 
236
252
  <h1>Block the shot before you pay to render it</h1>
237
253
  <p class="lede">
238
- CozyClay is a browser-based 3D staging studio. Block a scene, pose characters,
239
- author camera moves and cuts, then hand the exact same shots to a video model.
240
- No Blender, no Unity, no install.
254
+ CozyClay is free, open source previs software that runs in your browser. Block a
255
+ scene, pose characters, and author camera moves and cuts — then hand the same
256
+ shots to an AI video model. No Blender, no Unity, no install.
241
257
  </p>
242
258
 
243
259
  <div class="reel">
@@ -249,8 +265,9 @@
249
265
  ></video>
250
266
  </div>
251
267
  <p class="reel-note">
252
- Thirty seconds: greybox previz on the left of the wipe, the generated result on the
253
- right. Every cut came out of the same camera setup you see being built.
268
+ Thirty seconds: greybox previs on the left of the wipe, the generated result on the
269
+ right. Every cut came out of the same camera setup you see being built — more on
270
+ <a href="/ai-camera-control/">camera control for AI video</a>.
254
271
  </p>
255
272
 
256
273
  <div class="cta">
@@ -266,7 +283,7 @@
266
283
  <section id="what">
267
284
  <h2>What you can do</h2>
268
285
  <p class="sub">
269
- Previz is the part of animation where you decide what happens before anyone commits
286
+ Previs is the part of animation where you decide what happens before anyone commits
270
287
  to final assets. CozyClay keeps that whole loop in one page.
271
288
  </p>
272
289
  <div class="grid">
@@ -355,9 +372,36 @@ npm run dev</code></pre>
355
372
  <details open>
356
373
  <summary>What is CozyClay?</summary>
357
374
  <p>
358
- CozyClay is an open-source 3D staging and previsualization studio that runs in a
359
- browser. You block a scene with primitives and set pieces, pose characters, draw
360
- root paths, author camera moves and cuts on a timeline, and preview generated motion.
375
+ CozyClay is open source previs software that runs in a browser. You block a scene
376
+ with primitives and set pieces, pose characters, draw root paths, author camera
377
+ moves and cuts on a timeline, and preview generated motion.
378
+ </p>
379
+ </details>
380
+ <details>
381
+ <summary>What is previs, and is it spelled previz?</summary>
382
+ <p>
383
+ Previs is short for previsualization: the rough 3D pass where a shot gets blocked —
384
+ camera position, lens, character movement, cut points — before anyone commits to
385
+ final assets or render budget. It is written both <strong>previs</strong> and
386
+ <strong>previz</strong>; the industry mostly writes previs, and both mean the same
387
+ thing.
388
+ </p>
389
+ </details>
390
+ <details>
391
+ <summary>Is there free previs software?</summary>
392
+ <p>
393
+ CozyClay is free and open source under GPL-3.0, and it runs in the browser, so there
394
+ is no licence and no install between you and a blocked shot. Run it locally with
395
+ <code>npx cozyclay</code> and nothing leaves your machine.
396
+ </p>
397
+ </details>
398
+ <details>
399
+ <summary>Can I use it for camera control in AI video?</summary>
400
+ <p>
401
+ That is the reason it exists. Block the shot in 3D — angle, lens, dolly, the frame
402
+ the cut lands on — then take those camera moves to a video model instead of
403
+ describing them in a prompt and hoping. See
404
+ <a href="/ai-camera-control/">camera control for AI video</a>.
361
405
  </p>
362
406
  </details>
363
407
  <details>
package/dist/sitemap.xml CHANGED
@@ -2,8 +2,14 @@
2
2
  <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
3
3
  <url>
4
4
  <loc>https://cozyclay.org/</loc>
5
- <lastmod>2026-08-14</lastmod>
5
+ <lastmod>2026-08-15</lastmod>
6
6
  <changefreq>weekly</changefreq>
7
7
  <priority>1.0</priority>
8
8
  </url>
9
+ <url>
10
+ <loc>https://cozyclay.org/ai-camera-control/</loc>
11
+ <lastmod>2026-08-15</lastmod>
12
+ <changefreq>monthly</changefreq>
13
+ <priority>0.8</priority>
14
+ </url>
9
15
  </urlset>
@@ -0,0 +1,63 @@
1
+ # CozyClay live-control protocol v1
2
+
3
+ One WebSocket, JSON text frames. The **MCP server hosts** the socket
4
+ (`ws://127.0.0.1:5184/live`); the **editor is the client** and reconnects
5
+ every 3 s while the page is open. Either side may be absent: the editor works
6
+ exactly as before when nothing is listening, and the MCP server falls back to
7
+ its in-memory scene when no editor is connected.
8
+
9
+ ## Frames
10
+
11
+ editor -> server, once after connect:
12
+
13
+ { "type": "hello", "role": "editor", "version": 1 }
14
+
15
+ server -> editor, one per command:
16
+
17
+ { "type": "cmd", "id": "<opaque string>", "name": "<command>", "args": { ... } }
18
+
19
+ editor -> server, one per command, echoing `id`:
20
+
21
+ { "type": "result", "id": "<same id>", "ok": true, "value": { ... } }
22
+ { "type": "result", "id": "<same id>", "ok": false, "error": "<human message>" }
23
+
24
+ Unknown `name` MUST answer `ok:false`, never silence. The server times a
25
+ command out after 5 s and treats it as failed.
26
+
27
+ ## Commands (v1)
28
+
29
+ All coordinates are metres, rotations are degrees of yaw, ids are the ids the
30
+ scene document already uses.
31
+
32
+ | name | args | value | notes |
33
+ | --- | --- | --- | --- |
34
+ | `ping` | `{}` | `{ "pong": true }` | liveness |
35
+ | `describe` | `{}` | `{ sceneName, camera: { x, y, z, focalMm }, characters: [{ id, subject, x, z, rot, hidden }], objects: [{ id, name, x, y, z, rot, scaleX, scaleY, scaleZ, footprint, height }] }` | read the live scene; object scale and footprint travel too, so sizes are reported from what was actually built |
36
+ | `set_camera` | `{ x?, y?, z?, focalMm? }` | `{ camera }` | omitted fields keep their value; the **viewport must visibly move** |
37
+ | `add_character` | `{ subject, x?, z?, rot? }` | `{ id }` | |
38
+ | `update_character` | `{ ref, x?, z?, rot?, subject?, hidden? }` | `{ id }` | `ref` = id, letter (`"A"`) or 1-based slot |
39
+ | `remove_character` | `{ ref }` | `{ id }` | must refuse to empty the cast |
40
+ | `place_object` | `{ kind, x?, z?, y?, rot? }` | `{ id }` | `kind` from OBJECT_LIBRARY |
41
+ | `update_object` | `{ id, x?, y?, z?, rot?, rotX?, rotZ?, scale?, scaleX?, scaleY?, scaleZ?, color? }` | `{ id }` | `scale` sets all three axes; per-axis values override it |
42
+ | `remove_object` | `{ id }` | `{ id }` | |
43
+ | `load_scenes` | `{ document }` | `{ sceneName }` | replace the whole scene document (same shape `serializeSceneDocument` emits); the big hammer that guarantees parity |
44
+
45
+ ## Hard rules for the editor side
46
+
47
+ - Every mutation MUST go through the same React state paths the UI itself
48
+ uses (the setters/reducers the gizmo, inspector and panels call). Mutating
49
+ three.js objects directly is forbidden: anything outside React state is
50
+ overwritten on the next render.
51
+ - Every mutation MUST land in undo history exactly like the equivalent UI
52
+ action would, or be explicitly documented as not undoable.
53
+ - The socket client MUST be a no-op in production builds unless explicitly
54
+ enabled; in dev it may always try. A failed connection must never surface
55
+ an error to the user - silence and retry.
56
+
57
+ ## Hard rules for the server side
58
+
59
+ - When an editor is connected, live-capable tools forward to it and answer
60
+ from its `describe`; when none is connected they fall back to the in-memory
61
+ scene exactly as today. The tool surface does not change.
62
+ - One editor at a time: a second `hello` replaces the first (last write wins),
63
+ and the displaced socket is closed.
package/mcp/README.md ADDED
@@ -0,0 +1,142 @@
1
+ # CozyClay MCP server
2
+
3
+ Lets an AI assistant (Claude Desktop, Cursor, any MCP client) block a scene, place the camera,
4
+ generate character motion and read the shot back as film vocabulary — then turn it into an AI
5
+ image/video prompt.
6
+
7
+ Works two ways, with the same tools:
8
+
9
+ - **Editor open** — tool calls drive the visible viewport live: camera, cast, set, motion,
10
+ prompt blocks on the timeline.
11
+ - **No editor** — everything runs headless: no browser, no GPU, no build step.
12
+
13
+ ## Run it locally
14
+
15
+ ```sh
16
+ cd mcp
17
+ npm install
18
+ npm start # speaks MCP over stdio and hosts ws://127.0.0.1:5184/live
19
+ npm run verify # drives the no-editor fallback as a client and checks results
20
+ npm run verify:live # drives a fake editor over the live WebSocket protocol
21
+ ```
22
+
23
+ Then point a client at it. For Claude Desktop, in `claude_desktop_config.json`:
24
+
25
+ ```json
26
+ {
27
+ "mcpServers": {
28
+ "cozyclay": {
29
+ "command": "node",
30
+ "args": ["/absolute/path/to/CozyClay/mcp/server.mjs"]
31
+ }
32
+ }
33
+ }
34
+ ```
35
+
36
+ Restart the client; 20 tools appear.
37
+
38
+ Prefer a long-lived endpoint? `node server.mjs --http 5183` serves Streamable HTTP at
39
+ `http://127.0.0.1:5183/mcp` (one isolated session per client), with a plain status page at `/`.
40
+
41
+ ## What it does
42
+
43
+ Describe the shot you want. The server does the trigonometry and answers the way a crew would.
44
+
45
+ > "Put a detective and a courier in an alley, then give me a low wide profile shot of the courier."
46
+
47
+ ```
48
+ WIDE SHOT · RIGHT PROFILE · KNEE LEVEL · 24MM
49
+
50
+ size wide shot — the subject fills 40% of frame height
51
+ view a right-side profile view
52
+ level a low knee-level angle looking up at the subject
53
+ distance 4.39m
54
+ ```
55
+
56
+ `render_prompt` turns that same geometry into a prompt that carries the real framing, so the
57
+ generated frame matches the blocking instead of drifting off into a generic shot.
58
+
59
+ ## Tools
60
+
61
+ | tool | what it does |
62
+ | --- | --- |
63
+ | `describe_scene` | the whole state: camera, framing, cast, set |
64
+ | `live_status` | whether an editor tab is connected for live control |
65
+ | `describe_shot` | current camera geometry as film vocabulary |
66
+ | `set_camera` | move the lens / change focal length directly |
67
+ | `frame_shot` | frame by intent — size, view, level, side |
68
+ | `add_character` / `place_character` / `remove_character` | the cast |
69
+ | `focus_character` | choose who the camera frames |
70
+ | `place_object` / `update_object` / `remove_object` | the set |
71
+ | `render_prompt` | the shot as an AI image or video prompt |
72
+ | `generate_motion` | multi-phase character motion through the ARDY bridge — phases land as prompt blocks |
73
+ | `mark_camera_move` / `describe_camera_move` | name a move between two camera positions |
74
+ | `add_scene` / `switch_scene` | multiple scenes per project |
75
+ | `open_project` / `save_project` | read and write `.cclayproject` files |
76
+
77
+ Coordinates are metres (`x` right, `z` toward the default camera, `y` height above the floor).
78
+ Rotations are degrees of yaw. Characters are addressed by letter (`"A"`), slot (`"2"`) or id
79
+ (`"char-a"`).
80
+
81
+ ## Live editor control
82
+
83
+ The server hosts `ws://127.0.0.1:5184/live` by default (`COZYCLAY_LIVE_PORT` or
84
+ `--live-port <port>` to change it). A CozyClay editor tab served by `npm run dev` connects on
85
+ its own; `live_status` tells you whether one is attached. With an editor connected, mutations
86
+ forward to it and reads report its real state — the screen you are looking at is the source of
87
+ truth. Without one, every tool keeps its in-memory behaviour. In `--http` mode each session
88
+ child attempts to bind the live port, so one session owns the editor and the others
89
+ intentionally run memory-only.
90
+
91
+ The wire protocol — one WebSocket, ten commands, editor-side rules — is specified in
92
+ [`LIVE-PROTOCOL.md`](LIVE-PROTOCOL.md).
93
+
94
+ ## Motion generation
95
+
96
+ `generate_motion` takes plain-language beats and a length:
97
+
98
+ ```
99
+ phases: ["seated on a chair, slowly stands up",
100
+ "breaks into a sprint",
101
+ "trips and falls hard to the ground"]
102
+ seconds: 10
103
+ ```
104
+
105
+ It tiles them into contiguous ARDY segments, streams the generation through the local bridge
106
+ (`127.0.0.1:5181`, started by `npm run dev`), and — when an editor is connected — loads the
107
+ result onto the active character with one prompt block per phase on the timeline. Pass a
108
+ previous `motion_url` to reload a clip without generating again.
109
+
110
+ ## Round-trips with the studio
111
+
112
+ `save_project` writes a real `.cclayproject` file, so a scene blocked here opens in the studio to
113
+ be posed, timed and generated — and a scene built in the studio opens here with `open_project`.
114
+
115
+ ## Design
116
+
117
+ This server owns no geometry, no film vocabulary and no prompt text. Every answer is computed by
118
+ the same modules the studio renders with, imported straight from the working tree:
119
+
120
+ | module | responsibility |
121
+ | --- | --- |
122
+ | `../src/shot.js` | geometry → film vocabulary → prompt |
123
+ | `../src/scenes.js` | the scene document and its stage envelope |
124
+ | `../src/scene-objects.js` | the set: create / update / remove, clamped and snapped |
125
+ | `../src/camera-move.js` | two framings → a named camera move |
126
+ | `../src/project.js` | the `.cclayproject` envelope |
127
+
128
+ That is the whole design. Because the imports are relative, the server always speaks the working
129
+ tree's vocabulary: retune a band in `shot.js` and this server reports the new answer on its next
130
+ start, with nothing to publish or reinstall. The studio and the MCP server cannot disagree about
131
+ what a 35mm medium shot is, because there is only one implementation of it.
132
+
133
+ `frame_shot` is the one place that inverts `shot.js` rather than calling it — it solves for the
134
+ camera position that produces a requested size. `npm run verify` checks all 420 combinations the
135
+ schema accepts, so a retune in `shot.js` fails loudly here instead of quietly mis-framing.
136
+
137
+ ### Framing conflicts
138
+
139
+ Shot size is treated as the stronger request. An extreme close-up from an overhead lens is
140
+ geometrically impossible on a wide lens — the camera would sit inside the subject — so the server
141
+ lengthens the lens until the requested angle fits and says so, the way a crew swaps glass rather
142
+ than abandoning the close-up.