cozyclay 1.1.0 → 1.3.0

package/CHANGELOG.md

@@ -0,0 +1,85 @@
+# Changelog
+
+## 1.3.0
+
+The release where the studio stops being something only a person can drive.
+
+### Direct it with an AI
+
+CozyClay now ships an [MCP](https://modelcontextprotocol.io) server. Point Claude, Cursor, or any
+MCP client at it and ask for a shot in plain language — it places the cast, frames "a low wide
+profile", generates multi-phase motion, and **the viewport moves while you watch**.
+
+```json
+{ "mcpServers": { "cozyclay": { "command": "npx", "args": ["-y", "cozyclay", "mcp"] } } }
+```
+
+It computes nothing of its own. Every answer comes from the modules the studio already renders
+with — `shot.js` for film vocabulary, `scenes.js` for the stage, `camera-move.js` for moves — so
+the server and the UI cannot disagree about what a 35 mm medium shot is.
+
+- **Live or headless.** With a studio tab open, tool calls drive the real viewport over a local
+  socket. With no tab, the same tools run headless: block scenes, derive framing, render prompts,
+  and write `.cclayproject` files the studio opens.
+- **Motion from plain beats.** `generate_motion` takes ordinary sentences, rewrites them into the
+  shape ARDY was trained to read, splits compound beats, drops language a body cannot perform, and
+  lands each phase as a Prompt Block on the timeline.
+- **21 tools**, all documented in [`mcp/README.md`](mcp/README.md).
+
+### Cut a photograph out and stand it up
+
+Drop an image on Props — or straight onto the shot — and it becomes a card you can place, turn and
+block against. The background editor lives in the card's own inspector: paint or erase the
+selection, zoom and pan with the wheel, and the picture sits on a stage rather than being framed
+by one. Assets are stored once by content hash, so the same picture imported twice costs one copy.
+
+### An open stage
+
+The two-walled room corner is gone. The set is a 500 m open deck — far enough that no ordinary
+blocking meets its edge — with a two-tier grid (quiet 1 m cells, assertive 10 m lines) and a
+distance falloff that dissolves the floor into the background instead of ending at a horizon.
+Movement clamps moved from ±11 m to ±240 m, so a run or a chase finally has room.
+
+### Group props and move them as one
+
+Scene objects now carry a parent. Build a rocket from ten primitives, group them, and the whole
+thing drags as one body — through the gizmo, the inspector and the MCP tools alike. Deleting a
+parent promotes its children rather than taking them with it.
+
+### Everything else
+
+- **Gizmo** — the cast rides the same gizmo as scene objects, with rotate and scale; an XZ pad for
+  free ground drags; pick targets balanced so the pad no longer swallows every press.
+- **Mocap** — a video capture panel, takes assigned to cast members, and take trimming.
+- **Timeline** — authoring runs on a 24 fps production clock while ARDY is still spoken to at
+  20 fps, and exports record at true motion speed.
+- **Multi-character** — an unbounded cast with per-character motion layers.
+
+### Fixes
+
+- Props reported `1x1x1` regardless of how they were built: scale never crossed the live protocol,
+  and the server reset whatever did arrive. Sizes are now real, and per-axis scale is exposed.
+- A 6 m ceiling survived the walls it belonged to, silently decapitating anything tall.
+- Generating four Prompt Blocks produced one. IK keys from a discarded take made a fresh run look
+  like a local edit, and that path ships no schedule. Replacing a take now clears the corrections
+  authored against it.
+- A stale service worker served yesterday's bundle to the dev server, so edits appeared to do
+  nothing. Dev now serves a self-destructing worker.
+- Prompt blocks are capped at 4 s everywhere, not just in the UI — longer beats chain instead of
+  being authored into something the timeline would refuse.
+
+### Packaging
+
+- `mcp/` ships with the package, so `npx cozyclay mcp` works without a clone.
+- `exports` makes `cozyclay/src/*` a public contract instead of an accident.
+- `three` is declared as an optional peer: the studio bundles its own copy, but anything importing
+  `cozyclay/src/*` needs it, and now npm says so.
+
+## 1.2.0
+
+- `npx cozyclay` opens the studio again.
+- Stopped shipping 26 MB of scratch files.
+
+## 0.1.0
+
+First public release.

package/README.md

@@ -1,23 +1,57 @@
-# CozyClay
+<p align="center">
+  <img src="docs/images/cozyclay-logo.png" alt="CozyClay" width="340">
+</p>
+
+<p align="center">
+  <em>Block a scene, pose the cast, cut the camera — in a browser tab.</em>
+</p>
+
+<p align="center">
+  <a href="LICENSE"><img alt="License: GPL-3.0" src="https://img.shields.io/badge/license-GPL--3.0-blue"></a>
+  <a href="https://www.npmjs.com/package/cozyclay"><img alt="npm" src="https://img.shields.io/npm/v/cozyclay"></a>
+  <img alt="Node 22+" src="https://img.shields.io/badge/node-22%2B-brightgreen">
+  <a href="https://github.com/HaD0Yun/CozyClay/stargazers"><img alt="Stars" src="https://img.shields.io/github/stars/HaD0Yun/CozyClay?style=flat"></a>
+</p>
+
+<p align="center">
+  <a href="https://cozyclay.org/">Demo reel</a> ·
+  <a href="#quick-start">Quick start</a> ·
+  <a href="#what-you-can-do">Features</a> ·
+  <a href="#ai-control-mcp">AI control</a> ·
+  <a href="#controls">Controls</a> ·
+  <a href="https://github.com/HaD0Yun/CozyClay/issues">Issues</a>
+</p>
+
+---
+
+CozyClay is a browser-based 3D staging studio built with Three.js and React Three Fiber. Block a scene, pose characters, sequence motion prompts on a timeline, and preview generated motion — all in one local workspace.
 
-CozyClay is a browser-based 3D staging studio built with Three.js and React Three Fiber. Block a scene, pose characters, sequence motion prompts on a timeline, and preview generated motion — in one local workspace that handles like the Unity Editor.
-
-**[Open the live demo](https://had0yun.github.io/CozyClay/)** — the studio itself, running in your browser. Nothing to install. It comes seeded with a pre-generated motion clip so you can scrub the timeline, drive the cameras and draw a dolly rail straight away; generating new motion needs a local ARDY machine, so that part is off.
+```bash
+npx cozyclay
+```
 
-![CozyClay browser-based 3D staging studio](docs/images/cozyclay-studio.png)
+That is the whole install. **[cozyclay.org](https://cozyclay.org/)** has the demo reel and a walkthrough of what the studio does; to use it, run it on your own machine. It ships seeded with a pre-generated motion clip, so you can scrub the timeline, drive the cameras and draw a dolly rail straight away — generating *new* motion needs a local ARDY machine, so that part stays off until you point it at one.
 
 ## Demo
 
-![CozyClay motion, staging, and camera workflow](docs/images/cozyclay-demo.gif)
+https://github.com/user-attachments/assets/1d0113e5-6922-443d-affc-1bdabc666247
 
-CozyClay can connect to [NVIDIA ARDY](https://github.com/nv-tlabs/ardy) for motion generation. ARDY is a separate third-party project owned and maintained by NVIDIA; it is not included in this repository, and CozyClay is not affiliated with or endorsed by NVIDIA.
+## What you can do
+
+|  | |
+| --- | --- |
+| **Stage a scene** | Create primitives and set pieces, then move, rotate and scale them with a W/E/R gizmo. Grid snapping is a preference, not a law — hold `Ctrl` mid-drag to invert it. A bird's-eye plan view drives 2D root waypoints for character paths. |
+| **Fly the camera** | Right-drag flies (WASD walks, Q/E cranes), middle-drag pans, Alt+drag orbits the selection, click selects, `F` frames — the muscle memory you already have from a 3D editor. |
+| **Undo anything** | Every scene mutation goes through one history store: a drag, a scrub, an inspector edit is exactly one undo entry. `Esc` cancels an in-flight drag and restores the pre-drag transform. |
+| **Generate motion** | Pose characters and export poses, sequence multi-phase motion as Prompt Blocks on a resizable timeline, send them to ARDY, then play the result back with sparse IK correction where the generated motion needs fixing. |
+| **Direct it with an AI** | Connect Claude — or any MCP client — and ask for a shot in plain language. It places the cast, frames “a low wide profile”, generates multi-phase motion, and the viewport moves in front of you. See [AI control](#ai-control-mcp). |
 
 ## Requirements
 
 - Node.js 22 or newer
 - npm, or bun
 - A Chromium-based browser
-- Optional: an SSH-accessible NVIDIA machine running ARDY, for motion generation
+- *Optional:* an SSH-accessible NVIDIA machine running ARDY, for motion generation
 
 ## Quick start
 
@@ -27,17 +61,42 @@ npx cozyclay
 bunx cozyclay
 ```
 
-That downloads the built studio and opens it at `http://127.0.0.1:5180`. There is
-nothing to compile and no dependency tree to install. Useful flags:
-`--port 5200`, `--no-open`, `--no-ardy`.
+That downloads the built studio and opens it at `http://127.0.0.1:5180`. Nothing to compile, no dependency tree to install. Useful flags: `--port 5200`, `--no-open`, `--no-ardy`.
 
-Motion generation is off until you point it at a machine that can run it:
+Motion generation stays off until you point it at a machine that can run it:
 
 ```bash
 CCLAY_ARDY_HOST=user@your-gpu-box npx cozyclay
 ```
 
-Everything else, staging through camera work and playback, runs without it.
+Everything else — staging through camera work and playback — runs without it.
+
+## AI control (MCP)
+
+The studio ships an [MCP](https://modelcontextprotocol.io) server, so an AI assistant can drive it — the same scene, the same viewport, live:
+
+> “Put a detective and a courier in an alley, give me a low wide profile shot,
+> then make her stand up from the chair, sprint, and trip.”
+
+```json
+{
+  "mcpServers": {
+    "cozyclay": {
+      "command": "npx",
+      "args": ["-y", "cozyclay", "mcp"]
+    }
+  }
+}
+```
+
+Drop that into `claude_desktop_config.json` (or any MCP client config) and restart the client. The
+first run prints one `npm install` line: the MCP SDK is a 95-package tree, and opening the studio
+should never wait on it, so those dependencies are fetched only when you actually want the server.
+
+- **Editor open?** Tool calls move the visible viewport — camera, cast, set, generated motion, prompt blocks on the timeline.
+- **No editor?** The same tools run headless: block scenes, derive film vocabulary (“wide shot · right profile · knee level · 24mm”), render AI video prompts, and write `.cclayproject` files the studio opens.
+
+Tools, transports and the live-control protocol are documented in [`mcp/README.md`](mcp/README.md).
 
 ### From a clone
 
@@ -48,19 +107,20 @@ npm install
 npm run dev
 ```
 
-Open `http://127.0.0.1:5180`. `npm run dev` starts the studio together with its local ARDY bridge; `npm run dev:ui` starts the browser UI alone, without Block Generation.
-
-The bridge listens on loopback only. The environment variables that point it at a remote ARDY machine are documented in `tools/ardy/BRIDGE.md`.
+Open `http://127.0.0.1:5180`. `npm run dev` starts the studio together with its local ARDY bridge; `npm run dev:ui` starts the browser UI alone, without Block Generation. The bridge listens on loopback only; the environment variables that point it at a remote ARDY machine are documented in [`tools/ardy/BRIDGE.md`](tools/ardy/BRIDGE.md).
 
-## What you can do
+<details>
+<summary><b>Token-free ARDY text encoder</b> — skip the Hugging Face gate</summary>
 
-**Stage a scene.** Create primitives and set pieces, then move, rotate, and scale them with a W/E/R gizmo. Grid snapping is a preference, not a law — hold `Ctrl` during a drag to invert it. A bird's-eye plan view drives 2D root waypoints for character paths.
+ARDY's text encoder normally requires a Hugging Face account, gated-model approval, and an access token on the ARDY machine. CozyClay ships a token-free alternative — one command provisions the same encoder stack from public repositories, pinned by commit and SHA-256:
 
-**Edit like Unity.** Right-drag flies the camera (WASD walks, Q/E cranes), middle-drag pans, Alt+drag orbits the selection, left click selects, `F` frames. The full rule set, its source in Unity's manual, and every deliberate divergence are recorded in `docs/unity-reference.md`.
+```bash
+CCLAY_ARDY_HOST=user@your-gpu-box tools/ardy/setup-text-encoder-on-box.sh
+```
 
-**Undo anything.** Every scene mutation goes through a single history store: one interaction — a drag, a scrub, an inspector edit — is exactly one undo entry. `Esc` cancels an in-flight drag and restores the pre-drag transform.
+See [`tools/ardy/README.md`](tools/ardy/README.md) for details. This workflow is built with Meta Llama 3; the encoder's base weights are licensed under the Meta Llama 3 Community License.
 
-**Generate motion.** Pose characters and export poses, sequence multi-phase motion as Prompt Blocks on a resizable timeline, send them to ARDY, then play the result back with sparse IK correction where the generated motion needs fixing.
+</details>
 
 ## Controls
 
@@ -92,6 +152,8 @@ The bridge listens on loopback only. The environment variables that point it at
 | `npm run test:theme` / `test:appearance` / `test:layout` | UI theme, appearance, layout |
 | `npm run test:lifecycle` | Dev-server process lifecycle |
 | `npm run test:ardy` | ARDY conversion, playback, and IK pipeline |
+| `cd mcp && npm run verify` | MCP server over real stdio — all 420 framing combinations |
+| `cd mcp && npm run verify:live` | Live-control protocol against a fake editor |
 | `npm run build` | Production build |
 
 Ad-hoc browser QA, while a dev server is available:
@@ -100,10 +162,14 @@ Ad-hoc browser QA, while a dev server is available:
 npm run qa:browser -- <qa-script>
 ```
 
-## Repository hygiene
+## Contributing
+
+Found something broken, or want a feature? [Open an issue](https://github.com/HaD0Yun/CozyClay/issues) — bug reports with a repro are the most useful thing you can send.
 
-Generated motion archives, QA output, build output, logs, and local runtime artifacts are not source files and must not be committed. In particular, keep `tools/ardy/out/`, `artifacts/`, `dist/`, `.gjc/`, and `.npz` files local.
+**Repository hygiene.** Generated motion archives, QA output, build output, logs and local runtime artifacts are not source files and must not be committed. Keep `tools/ardy/out/`, `artifacts/`, `dist/`, `.gjc/` and `.npz` files local.
 
-## License
+## License & credits
 
-GNU General Public License v3.0 or later — see `LICENSE`. Third-party projects retain their own licenses and copyright; see `THIRD_PARTY_NOTICES.md`.
+GNU General Public License v3.0 or later — see [`LICENSE`](LICENSE). Third-party projects retain their own licenses and copyright; see [`THIRD_PARTY_NOTICES.md`](THIRD_PARTY_NOTICES.md).
+
+CozyClay can connect to [NVIDIA ARDY](https://github.com/nv-tlabs/ardy) for motion generation. ARDY is a separate third-party project owned and maintained by NVIDIA; it is not included in this repository, and CozyClay is not affiliated with or endorsed by NVIDIA.

package/THIRD_PARTY_NOTICES.md

@@ -24,6 +24,73 @@ ARDY is not bundled with CozyClay. Users must obtain, install, and operate ARDY
 
 ARDY model checkpoints and other model assets may be governed by separate terms, including the NVIDIA Open Model License identified by the ARDY project. Users are responsible for reviewing and complying with those terms before downloading or using the models.
 
+## Meta Llama 3
+
+ARDY's text encoder is based on Meta Llama 3 (`Meta-Llama-3-8B-Instruct`).
+CozyClay does not bundle or redistribute the model weights; the optional
+`tools/ardy/setup-text-encoder.py` script downloads them directly from a
+public repository to the user's own machine for local use, together with the
+model's LICENSE and USE_POLICY files.
+
+Built with Meta Llama 3.
+
+- Copyright © Meta Platforms, Inc. All Rights Reserved.
+- License: Meta Llama 3 Community License
+- License text: https://www.llama.com/llama3/license/
+- Acceptable Use Policy: https://www.llama.com/llama3/use-policy/
+
+Meta Llama 3 is licensed under the Meta Llama 3 Community License,
+Copyright © Meta Platforms, Inc. All Rights Reserved.
+
+## LLM2Vec
+
+ARDY's text encoder applies the LLM2Vec adapters from McGill NLP
+(`LLM2Vec-Meta-Llama-3-8B-Instruct-mntp` and `-mntp-supervised`). Like the
+base weights, they are downloaded by the setup script for local use, not
+bundled.
+
+- Copyright (c) 2024 McGill NLP
+- License: MIT (the adapters are derived from Meta Llama 3; the Meta Llama 3
+  Community License applies to that underlying model)
+- Source: https://github.com/McGill-NLP/llm2vec
+- License text: https://github.com/McGill-NLP/llm2vec/blob/main/LICENSE
+
+## Fonts
+
+CozyClay bundles subsets of two typefaces. Both are licensed under the SIL Open
+Font License 1.1, which allows them to be redistributed with software as long as
+the copyright notice and the licence travel with the files. The licence texts are
+in `public/fonts/` next to the fonts themselves.
+
+### Inter
+
+- Copyright (c) 2016 The Inter Project Authors
+- License: SIL Open Font License 1.1
+- Source: https://github.com/rsms/inter
+- License text: `public/fonts/Inter-OFL.txt`
+
+### Instrument Serif
+
+- Copyright 2022 The Instrument Serif Project Authors
+- License: SIL Open Font License 1.1
+- Source: https://github.com/Instrument/instrument-serif
+- License text: `public/fonts/InstrumentSerif-OFL.txt`
+
+## Character models
+
+The rigs in `public/models/` are Mixamo characters from Adobe. Adobe permits
+their use in projects but not the distribution of the raw character files, which
+is what shipping them in this repository, the npm package and the hosted site
+amounts to. They are also outside the scope of this repository's GPL-3.0 grant:
+nothing here relicenses them, and a fork does not acquire the right to
+redistribute them.
+
+They are being replaced with a CC0 rig. Until that lands, treat these two files
+as third-party content that this licence does not cover.
+
+- `x-bot-tpose.fbx`, `y-bot-tpose.fbx` — Adobe Mixamo
+- Terms: https://www.adobe.com/legal/terms.html
+
 ## CozyClay license scope
 
-The CozyClay source code in this repository is licensed under GPL-3.0-or-later. That license does not replace or relicense Three.js, ARDY, ARDY model checkpoints, or any other third-party component.
+The CozyClay source code in this repository is licensed under GPL-3.0-or-later. That license does not replace or relicense Three.js, ARDY, ARDY model checkpoints, the bundled fonts, the character rigs, or any other third-party component.

package/bin/cozyclay.mjs

@@ -79,6 +79,7 @@ const STAR_AFTER_MS = Number(process.env.COZYCLAY_STAR_AFTER_MS ?? 60_000);
 const HELP = `cozyclay - browser-based 3D staging studio
 
   npx cozyclay              start the studio and open it
+  npx cozyclay mcp          run the MCP server (for Claude, Cursor, any MCP client)
   npx cozyclay --port 5200  serve on another port
   npx cozyclay --no-ardy    skip the optional motion-generation sidecar
   npx cozyclay --no-open    do not open a browser
@@ -189,7 +190,50 @@ function openBrowser(url) {
 	child.unref();
 }
 
-const opts = parseArgs(process.argv.slice(2));
+/**
+ * `cozyclay mcp` hands over to the MCP server.
+ *
+ * Its dependencies are NOT the launcher's: the studio path stays dependency
+ * free on purpose, and the MCP SDK is a 95-package tree nobody who only wants
+ * a viewport should download. So the server ships with the package, its deps
+ * are installed on demand, and a missing one produces an exact instruction
+ * rather than a stack trace.
+ */
+async function runMcp(rest) {
+	const server = join(PKG_ROOT, "mcp", "server.mjs");
+	if (!existsSync(server)) {
+		console.error("cozyclay: this build does not include the MCP server.");
+		process.exit(1);
+	}
+	// The deps must land in the PACKAGE root, not in mcp/: server.mjs imports
+	// ../src/*.js, and those files resolve `three` from cozyclay/ itself.
+	try {
+		await import("@modelcontextprotocol/sdk/server/mcp.js");
+		await import("three");
+	} catch {
+		console.error(
+			[
+				"cozyclay: the MCP server needs four packages the studio does not.",
+				"",
+				`  npm install --prefix ${JSON.stringify(PKG_ROOT)} \\`,
+				"    @modelcontextprotocol/sdk ws zod three",
+				"",
+				"Then run `npx cozyclay mcp` again. They are kept out of the default",
+				"install so opening the studio never waits on a 95-package tree.",
+			].join("\n"),
+		);
+		process.exit(1);
+	}
+	const child = spawn(process.execPath, [server, ...rest], { stdio: "inherit" });
+	child.on("exit", (code, signal) => process.exit(signal ? 1 : (code ?? 0)));
+}
+
+const argv = process.argv.slice(2);
+if (argv[0] === "mcp") {
+	await runMcp(argv.slice(1));
+} else {
+
+const opts = parseArgs(argv);
 if (opts.help) {
 	console.log(HELP);
 	process.exit(0);
@@ -199,8 +243,8 @@ if (opts.version) {
 	console.log(pkg.version);
 	process.exit(0);
 }
-if (!existsSync(join(DIST, "index.html"))) {
-	console.error("cozyclay: this package is missing its build (dist/index.html).");
+if (!existsSync(join(DIST, "app", "index.html"))) {
+	console.error("cozyclay: this package is missing its build (dist/app/index.html).");
 	console.error("cozyclay: from a clone, run `npm install && npm run build` first.");
 	process.exit(1);
 }
@@ -219,14 +263,28 @@ if (opts.ardy && ardyHost && existsSync(BRIDGE)) {
 
 const server = createServer((req, res) => {
 	const url = new URL(req.url ?? "/", "http://localhost");
-	if (url.pathname.startsWith("/ardy/")) {
+	// Only the routes the bridge actually owns: /ardy/ is ALSO a public asset
+	// directory (cskel27-rest.json), and those files live in dist/, not behind
+	// the sidecar. Same rule as the Vite dev proxy bypass.
+	if (/^\/ardy\/(health|bases|generate|footage|extract|motions)(\/|$)/.test(url.pathname)) {
 		proxyToBridge(req, res);
 		return;
 	}
-	const rel = decodeURIComponent(url.pathname === "/" ? "/index.html" : url.pathname);
+	const rel = decodeURIComponent(url.pathname);
 	// normalize + prefix check: a request must not escape dist/.
-	const target = join(DIST, normalize(rel));
-	if (!target.startsWith(DIST) || !existsSync(target) || statSync(target).isDirectory()) {
+	let target = join(DIST, normalize(rel));
+	if (!target.startsWith(DIST)) {
+		res.writeHead(404, { "content-type": "text/plain; charset=utf-8" });
+		res.end("not found");
+		return;
+	}
+	// The build is multi-page: "/" is the landing and "/app/" is the studio, so a
+	// directory has to resolve to its index the way a static host would. Without
+	// this the studio 404s and the CLI only ever serves the landing page.
+	if (existsSync(target) && statSync(target).isDirectory()) {
+		target = join(target, "index.html");
+	}
+	if (!existsSync(target) || statSync(target).isDirectory()) {
 		res.writeHead(404, { "content-type": "text/plain; charset=utf-8" });
 		res.end("not found");
 		return;
@@ -261,7 +319,10 @@ server.on("error", (err) => {
 });
 
 server.listen(opts.port, opts.host, () => {
-	const url = `http://${opts.host}:${opts.port}/`;
+	// The package exists to open the studio, which the site serves from /app/.
+	// Landing on "/" would greet someone who just typed `npx cozyclay` with a
+	// marketing page.
+	const url = `http://${opts.host}:${opts.port}/app/`;
 	console.log(`CozyClay is running at ${url}`);
 	if (!opts.ardy) console.log("Motion generation: off (--no-ardy).");
 	else if (bridge) console.log(`Motion generation: sidecar running against ${ardyHost}.`);
@@ -272,3 +333,5 @@ server.listen(opts.port, opts.host, () => {
 		);
 	if (opts.open) openBrowser(url);
 });
+
+}

package/dist/CNAME

@@ -0,0 +1 @@
+cozyclay.org