cozyclay 1.7.0 → 1.7.1

package/CHANGELOG.md

@@ -1,5 +1,10 @@
 # Changelog
 
+## 1.7.1
+
+- Open the complete Studio at the development server root; the unfinished
+  Workflow canvas remains available at `/workflow/`.
+
 ## 1.7.0
 
 Kimodo becomes the default motion-generation backend, and a full edit-and-refine

package/README.md

@@ -120,7 +120,11 @@ npm install
 npm run dev
 ```
 
-Open `http://127.0.0.1:5180/app/`. `npm run dev` starts the studio together with its local Kimodo bridge once `CCLAY_KIMODO_HOST` points at a GPU box; without that variable it starts the studio alone and says so, and Block Generation stays unavailable until you set it. `npm run dev:ui` starts the browser UI alone in every case. The bridge listens on loopback only; Kimodo host variables are documented in [`tools/kimodo/setup-on-box.sh`](tools/kimodo/setup-on-box.sh).
+Open `http://127.0.0.1:5180/` for the Studio. The Workflow canvas remains available at `http://127.0.0.1:5180/workflow/` when you need its node editor. `npm run dev` starts the studio together with its local Kimodo bridge once `CCLAY_KIMODO_HOST` points at a GPU box; without that variable it starts the studio alone and says so, and Block Generation stays unavailable until you set it. `npm run dev:ui` starts the browser UI alone in every case. The bridge listens on loopback only; Kimodo host variables are documented in [`tools/kimodo/setup-on-box.sh`](tools/kimodo/setup-on-box.sh).
+
+### Workflow canvas
+
+The Vibe-Workflow canvas is available at `http://127.0.0.1:5180/workflow/`. It keeps the original node-style flow and adds a **CozyClay Scene** node with an interactive 3D viewport, frame/camera controls, and an Open Studio handoff. Start it with the normal dev server, or run `npm run dev:ui` by itself for local-only editing. Workflow graphs are saved in the browser and execute locally; no remote workflow service or API key is required.
 
 ## Hosted demo
 
@@ -228,15 +232,23 @@ Events collected:
 | --- | --- |
 | `install:first_launch` | First run of the official npm package |
 | `app:session_started` | Start of an official npm package session |
+| `app:session_ended` | Session duration, action count, and scenes touched (bucketed) |
+| `feature:used` | One signal per feature per session |
 | `$pageview` | Funnel and drop-off analysis |
 | `scene:created` | Funnel and drop-off analysis |
 | `scene:loaded` | Funnel and drop-off analysis |
+| `project:saved` | User-owned project persistence |
+| `project:opened` | Return to a saved project (age bucket) |
 | `craft:first_action` | Funnel and drop-off analysis |
+| `motion:backend_state` | Motion capability at session start (`none`, `local_kimodo`, or `hosted`) |
+| `motion:generate_blocked` | Generate intent when no motion backend is available |
 | `motion:job_started` | Motion reliability |
 | `motion:job_succeeded` | Motion reliability |
 | `motion:job_failed` | Motion reliability |
 | `export:blocking_frame_succeeded` | Funnel and drop-off analysis |
 | `activation:completed` | Funnel and drop-off analysis |
+| `hosted:composer_viewed`, `hosted:login_started`, `hosted:ticket_created` | Hosted demo funnel |
+| `hosted:result_opened`, `hosted:opened_in_studio` | Hosted result funnel |
 
 Geo data comes from ingest-time GeoIP country lookup only — no precise location is collected. Prompt text, asset names, file names, project content, local paths, and any user-entered text are never collected.
 
@@ -248,6 +260,14 @@ development servers, CI, and tests do not send analytics. Official npm
 artifacts carry a signature checked by the launcher, so copying or repackaging
 the source does not enable telemetry.
 
+Each event is registered with `origin_kind` (`local` or `hosted`), a coarse
+operating-system label, and (for npm sessions) `install_kind` (`npx` or
+`global`). Source checkouts are classified as `clone` and remain telemetry-off.
+The first npm launch may optionally answer a one-line channel question
+(`x`, `hn`, `reddit`, `github`, `friend`, `other`, or `skip`); `skip` sends no
+acquisition value. Session duration and action counts are buckets, and project
+events never include names, paths, prompts, or timestamps.
+
 The npm package prints this disclosure once on first launch. Control it at any
 time:
 

package/THIRD_PARTY_NOTICES.md

@@ -105,6 +105,14 @@ as third-party content that this licence does not cover.
 
 The hosted site at cozyclay.org uses [posthog-js](https://posthog.com/docs/libraries/js) for anonymous usage analytics.
 
+The disclosed wire events include session start/end (bucketed duration and
+actions), one-per-session feature usage, project save/open buckets, and the
+hosted composer/login/ticket/result funnel. Events carry only the registered
+`origin_kind`, coarse `os`, and npm `install_kind`; prompts, filenames, paths,
+project names, and timestamps are excluded. Source checkouts keep telemetry
+disabled. See the Analytics & privacy section in `README.md` for the complete
+event table and opt-out controls.
+
 - Copyright PostHog Inc.
 - License: Apache-2.0 AND MIT
 - Source: https://github.com/PostHog/posthog-js

package/bin/agent/agent-routes.mjs

@@ -0,0 +1,348 @@
+import { AsyncLocalStorage } from "node:async_hooks";
+import * as defaultAuth from "../codex-auth.mjs";
+import { createCodexClient } from "./codex-client.mjs";
+import { createAgentTools, agentToolSchemas, SYSTEM_PROMPT, pickWorkspace } from "./agent-tools.mjs";
+import { createVideoAdapters } from "./video-adapters.mjs";
+
+// Values the codex backend accepts for reasoning.effort (its own 400 lists them).
+export const REASONING_EFFORTS = ["none", "minimal", "low", "medium", "high", "xhigh", "max", "ultra"];
+
+const json = (res, status, value) => {
+	res.writeHead(status, { "content-type": "application/json; charset=utf-8", "cache-control": "no-store" });
+	res.end(JSON.stringify(value));
+};
+
+export function allowAgentOrigin(req, port) {
+	return [`http://127.0.0.1:${port}`, `http://${"local" + "host"}:${port}`].includes(req.headers.origin)
+		|| (req.headers.origin === undefined && req.method === "GET"
+			&& [`127.0.0.1:${port}`, `localhost:${port}`].includes(req.headers.host));
+}
+
+// Two 1920x1080 PNG data URLs (frame + reference) fit comfortably in this.
+const IMAGE_BODY_LIMIT = 24 * 1024 * 1024;
+
+// Attached scene references (#167): identity sheets and the environment
+// reference. Capped because every one of them is another full image the
+// backend has to read, and a shot with seven of them is a prompt nobody wrote.
+const IMAGE_REFERENCES_MAX = 6;
+
+/** Reject anything that is not a list of {role, name?, dataUrl} inline images. */
+function validReferences(references) {
+	if (references === undefined) return true;
+	if (!Array.isArray(references) || references.length > IMAGE_REFERENCES_MAX) return false;
+	return references.every((entry) => entry && typeof entry === "object" && !Array.isArray(entry)
+		&& typeof entry.role === "string" && entry.role
+		&& (entry.name === undefined || typeof entry.name === "string")
+		&& typeof entry.dataUrl === "string" && entry.dataUrl.startsWith("data:image/"));
+}
+
+/**
+ * What the attached pictures MEAN, in the order they are attached. Without
+ * this the backend sees a pile of images and guesses; with it the clay frame
+ * owns the geometry, each character sheet owns one performer's look and the
+ * environment reference owns the location.
+ */
+export function referenceGuidance(references = []) {
+	const list = Array.isArray(references) ? references : [];
+	if (!list.length) return "";
+	const lines = ["Geometry, camera and blocking come from the first image (the clay frame)."];
+	for (const entry of list) {
+		if (entry.role === "character") {
+			lines.push(`Character ${entry.name || "reference"}: match the identity, face, hair and wardrobe from the attached character sheet.`);
+		} else if (entry.role === "environment") {
+			lines.push("Environment: take the location look, materials, palette and lighting from the attached environment reference.");
+		}
+	}
+	return `\n${lines.join("\n")}`;
+}
+
+async function readBody(req, limit = 64 * 1024) {
+	let text = "";
+	for await (const chunk of req) {
+		text += chunk;
+		if (Buffer.byteLength(text) > limit) throw new Error("Request too large.");
+	}
+	return JSON.parse(text || "{}");
+}
+
+function quotaEvent(codex, headers) {
+	const quota = codex.parseQuotaHeaders(headers);
+	let resetAt = quota.primary.resetAt;
+	if (resetAt && /^\d+(\.\d+)?$/.test(String(resetAt))) resetAt = Number(resetAt) * 1000;
+	if (!resetAt && quota.primary.resetAfterSeconds !== undefined) resetAt = Date.now() + quota.primary.resetAfterSeconds * 1000;
+	return {
+		type: "quota", plan: quota.planType ?? null,
+		primary: { usedPercent: quota.primary.usedPercent ?? null, windowMinutes: quota.primary.windowMinutes ?? null, resetAt: resetAt ?? null },
+		credits: { has: quota.credits.hasCredits },
+	};
+}
+
+function errorInfo(error, quota) {
+	if (error?.status === 401 || error?.code === "unauthorized") return { code: "auth", message: "Authentication required. Sign in again." };
+	if (error?.status === 429) return { code: "rate_limit", message: "Rate limit exceeded.", resetAt: quota?.primary.resetAt ?? null };
+	if (error?.code === "entitlement") return { code: "entitlement", message: "This account cannot generate images." };
+	if (error?.code === "overloaded") return { code: "overloaded", message: "The model service is overloaded right now. Try again in a moment." };
+	if (error?.code === "server_error") return { code: "overloaded", message: "The model service hit an internal error. Try again in a moment." };
+	// Backend errors may echo credentials or image inputs. Never forward their bodies.
+	return { code: "upstream", message: "The model or live editor could not complete this turn." };
+}
+
+/** Use the existing client and its request queue, retaining failure headers that
+ * the client otherwise discards. A 429 belongs to the panel's paused state, not
+ * the client's unbounded retry loop (which cannot be interrupted during sleep). */
+function defaultClient(auth, requestContext) {
+	return createCodexClient({
+		getAccessToken: auth.getAccessToken, getAccountId: auth.getAccountId, originator: "cozyclay",
+		fetch: async (url, init) => {
+			const response = await fetch(url, init);
+			requestContext.getStore()?.(response.headers);
+			if (response.ok) return response;
+			const detail = await response.text();
+			const error = Object.assign(new Error("Codex backend request failed."), { status: response.status, headers: response.headers });
+			if (url.includes("/images/") && /entitlement|plan/i.test(detail)) error.code = "entitlement";
+			throw error;
+		},
+	});
+}
+
+/** Start the optional registry/live dependencies without making signed-out
+ * startup depend on an MCP dependency install. Failures remain visible on use. */
+function liveToolsRuntime() {
+	return Promise.all([import("../../mcp/tool-handlers.mjs"), import("../../mcp/live-hub.mjs")]).then(async ([registry, { startLiveHub }]) => {
+		const liveHub = await startLiveHub(Number(process.env.COZYCLAY_LIVE_PORT ?? 5184));
+		registry.setLiveHub(liveHub);
+		const handlers = registry.createToolHandlers().map((tool) => ({
+			...tool,
+			handler: async (args, { workspaceHandle } = {}) => {
+				const parsed = Object.fromEntries(Object.entries(tool.inputSchema).map(([key, schema]) => [key, schema.parse(args[key])]));
+				const run = (handle) => registry.liveWorkspace.run(handle, () => tool.handler(parsed));
+				return liveHub?.connected ? liveHub.runExclusive(tool.name, workspaceHandle, run) : run(workspaceHandle);
+			},
+		}));
+		return { liveHub, handlers };
+	}).catch((error) => ({ error }));
+}
+
+export function createAgentHandler({ auth = defaultAuth, codex, handlers, liveHub, port, retryDelayMs = 2000 } = {}) {
+	const requestContext = new AsyncLocalStorage();
+	codex ||= defaultClient(auth, requestContext);
+	const runtime = handlers !== undefined || liveHub !== undefined ? Promise.resolve({ handlers: handlers ?? [], liveHub }) : liveToolsRuntime();
+	const renderGuidance = async (environment) => {
+		try {
+			const { handlers: tools, liveHub: hub } = await runtime;
+			const tool = tools.find((entry) => entry.name === "render_prompt");
+			if (!tool || !hub?.connected) return "";
+			const workspaceHandle = pickWorkspace(hub);
+			const result = await tool.handler({ mode: "image", environment }, { workspaceHandle });
+			if (result?.isError) return "";
+			const text = typeof result === "string" ? result : (result?.content ?? []).filter((part) => part.type === "text").map((part) => part.text).join("\n");
+			return text ? `\n${text}` : "";
+		} catch { return ""; }
+	};
+	const sessions = new Map();
+	const unsubscribe = auth.onAuthChange?.(() => {
+		for (const session of sessions.values()) session.controller?.abort();
+		sessions.clear();
+	});
+
+	const handle = async (req, res, path = new URL(req.url, "http://127.0.0.1").pathname) => {
+		if (!path.startsWith("/agent/")) return false;
+		if (port !== undefined && !allowAgentOrigin(req, typeof port === "function" ? port() : port)) {
+			json(res, 403, { error: "forbidden origin" }); return true;
+		}
+		if (path === "/agent/models" && req.method === "GET") {
+			try {
+				const result = await codex.listModels();
+				const models = (Array.isArray(result) ? result : result.models).map((model) => {
+					const id = typeof model === "string" ? model : model.slug || model.id;
+					const efforts = Array.isArray(model.supported_reasoning_levels) ? model.supported_reasoning_levels.map((level) => (typeof level === "string" ? level : level.effort)).filter(Boolean) : [];
+					return { id, label: id, efforts, defaultEffort: typeof model.default_reasoning_level === "string" ? model.default_reasoning_level : efforts[0] ?? null };
+				});
+				models.sort((a, b) => Number(b.id === "gpt-6-astra") - Number(a.id === "gpt-6-astra"));
+				json(res, 200, { models });
+			} catch (error) { json(res, error.status === 401 ? 401 : 502, { error: errorInfo(error) }); }
+			return true;
+		}
+		if (path === "/agent/image" && req.method === "POST") {
+			let value;
+			try {
+				value = await readBody(req, IMAGE_BODY_LIMIT);
+				if (typeof value.prompt !== "string" || !value.prompt.trim() || typeof value.imageDataUrl !== "string" || !value.imageDataUrl.startsWith("data:image/") || (value.referenceDataUrl !== undefined && (typeof value.referenceDataUrl !== "string" || !value.referenceDataUrl.startsWith("data:image/"))) || !validReferences(value.references) || (value.quality !== undefined && !["auto", "low", "medium", "high"].includes(value.quality))) throw new Error("Invalid request.");
+			} catch { json(res, 400, { error: "invalid request" }); return true; }
+			if (!await auth.getAccessToken()) { json(res, 401, { error: { code: "auth", message: "Sign in with ChatGPT in the Agent panel." } }); return true; }
+			try {
+				// Same composition guidance the agent's render_from_frame appends: the
+				// node's prompt is intent only; camera, cast and set come from the scene.
+				// Scene references (#167) are appended after the frame/reference pair,
+				// and the prompt says what each attachment is for.
+				const references = Array.isArray(value.references) ? value.references : [];
+				const prompt = `${value.prompt}${await renderGuidance(value.prompt)}${referenceGuidance(references)}`;
+				const result = await codex.editImage({ ...value, prompt, extraImages: references.map((entry) => entry.dataUrl) });
+				json(res, 200, { dataUrl: `data:image/png;base64,${result.pngBase64}`, width: result.width, height: result.height });
+			} catch (error) { json(res, error.status === 401 ? 401 : 502, { error: errorInfo(error) }); }
+			return true;
+		}
+		if (path === "/agent/video/providers" && req.method === "GET") {
+			json(res, 200, { providers: createVideoAdapters().map((adapter) => ({ id: adapter.id, name: adapter.name, configured: adapter.configured() })) });
+			return true;
+		}
+		if (path === "/agent/video" && req.method === "POST") {
+			let value;
+			try {
+				value = await readBody(req, IMAGE_BODY_LIMIT);
+				if (!value || typeof value.provider !== "string" || typeof value.prompt !== "string" || !value.prompt.trim() || typeof value.imageDataUrl !== "string" || !value.imageDataUrl.startsWith("data:image/") || (value.lastFrameDataUrl !== undefined && (typeof value.lastFrameDataUrl !== "string" || !value.lastFrameDataUrl.startsWith("data:image/"))) || !Number.isFinite(Number(value.durationSeconds)) || Number(value.durationSeconds) < 1 || Number(value.durationSeconds) > 15 || typeof value.aspect !== "string" || (value.model !== undefined && typeof value.model !== "string")) throw new Error("Invalid request.");
+			} catch { json(res, 400, { error: "invalid request" }); return true; }
+			const adapter = createVideoAdapters().find((entry) => entry.id === value.provider);
+			if (!adapter || !adapter.configured()) { json(res, 409, { error: "video provider is not configured" }); return true; }
+			try {
+				const result = await adapter.generate({ ...value, durationSeconds: Number(value.durationSeconds) });
+				json(res, 200, { ...(result.mp4Base64 ? { dataUrl: `data:video/mp4;base64,${result.mp4Base64}` } : { url: result.url }), width: result.width, height: result.height, seconds: result.seconds });
+			} catch (error) { json(res, 502, { error: error?.message || "video provider failed" }); }
+			return true;
+		}
+		if (req.method !== "POST" || !["/agent/turn", "/agent/stop"].includes(path)) {
+			json(res, 404, { error: "not found" }); return true;
+		}
+		let value;
+		try {
+			value = await readBody(req);
+			if (!value || typeof value.sessionId !== "string" || !value.sessionId
+				|| (path === "/agent/turn" && (typeof value.text !== "string"
+					|| (value.attachFrame !== undefined && typeof value.attachFrame !== "boolean")
+					|| (value.model !== undefined && typeof value.model !== "string")
+					|| (value.effort !== undefined && !REASONING_EFFORTS.includes(value.effort))))) throw new Error("Invalid request.");
+		} catch { json(res, 400, { error: "invalid request" }); return true; }
+		if (path === "/agent/stop") {
+			sessions.get(value.sessionId)?.controller?.abort();
+			json(res, 200, { ok: true }); return true;
+		}
+
+		res.writeHead(200, { "content-type": "text/event-stream; charset=utf-8", "cache-control": "no-cache", connection: "keep-alive" });
+		res.flushHeaders();
+		const send = (event) => { if (!res.writableEnded && !res.destroyed) res.write(`data: ${JSON.stringify(event)}\n\n`); };
+		let session = sessions.get(value.sessionId);
+		if (session?.running) {
+			send({ type: "error", code: "upstream", message: "busy" }); send({ type: "done" }); res.end(); return true;
+		}
+		session ||= { images: new Map(), history: [] };
+		sessions.set(value.sessionId, session);
+		const controller = new AbortController();
+		const { signal } = controller;
+		Object.assign(session, { running: true, controller, signal });
+		const disconnect = () => controller.abort();
+		res.once("close", disconnect);
+		let quota;
+		const observeHeaders = (headers) => {
+			const next = quotaEvent(codex, headers);
+			if (!quota) send(next);
+			quota = next;
+		};
+		let refreshed = false;
+		// A stream that fails with server_is_overloaded usually succeeds on the
+		// next attempt; retry twice before reporting it.
+		const retryOverloaded = async (operation, attempts = 2) => {
+			for (let attempt = 0; ; attempt += 1) {
+				try { return await operation(); } catch (error) {
+					if (!["overloaded", "server_error"].includes(error.code) || attempt >= attempts || signal.aborted) throw error;
+					await new Promise((resolve) => setTimeout(resolve, retryDelayMs * (attempt + 1)));
+				}
+			}
+		};
+		const retryAuth = async (operation) => {
+			try { signal.throwIfAborted(); return await operation(); }
+			catch (error) {
+				if (error.status !== 401 || refreshed || signal.aborted) throw error;
+				refreshed = true;
+				if (!await auth.getAccessToken()) throw error;
+				signal.throwIfAborted();
+				return operation();
+			}
+		};
+		const turn = async () => {
+			if (!await auth.getAccessToken()) throw Object.assign(new Error("Authentication required."), { status: 401 });
+			const dependencies = await runtime;
+			session.codex = { editImage: (args) => retryAuth(() => codex.editImage(args)) };
+			const tools = createAgentTools({ ...dependencies, session, emit: send });
+			const executeTool = async (item, override) => {
+				signal.throwIfAborted();
+				const tool = override ?? tools.find((entry) => entry.name === item.name);
+				if (!tool) throw new Error("Unknown tool.");
+				const args = typeof item.arguments === "string" ? JSON.parse(item.arguments) : item.arguments;
+				send({ type: "tool.start", callId: item.call_id, name: item.name, label: item.name.replaceAll("_", " "), args });
+				const started = performance.now();
+				try {
+					if (dependencies.error) throw dependencies.error;
+					const result = await tool.handler(args);
+					signal.throwIfAborted();
+					send({ type: "tool.done", callId: item.call_id, ok: true, elapsedMs: Math.round(performance.now() - started), result });
+					return result;
+				} catch (error) {
+					if (process.env.COZYCLAY_AGENT_DEBUG) console.error("[agent] tool", item.name, "failed:", error?.message);
+					send({ type: "tool.done", callId: item.call_id, ok: false, elapsedMs: Math.round(performance.now() - started), error: errorInfo(error).message });
+					throw error;
+				}
+			};
+			let text = value.text;
+			if (value.attachFrame) {
+				const captured = await executeTool({ call_id: "attached-frame", name: "capture_blocking_frame", arguments: {} }, tools.internal.capture);
+				text += `\nAttached frame imageId: ${captured.imageId}`;
+			}
+			const history = [...session.history, { role: "user", content: [{ type: "input_text", text }] }];
+			// runAgentTurn closes over streamResponses and hides its headers. Drive
+			// that same serial loop here so quotas are observable, and retry only
+			// the failed request rather than replaying already-executed scene tools.
+			while (true) {
+				const output = await retryOverloaded(() => retryAuth(async () => {
+					const stream = codex.streamResponses({ input: history, tools: agentToolSchemas(tools), instructions: SYSTEM_PROMPT, model: value.model, effort: value.effort, signal });
+					const headers = stream.headers.then(observeHeaders, () => {});
+					const items = [];
+					try {
+						for await (const event of stream) {
+							signal.throwIfAborted();
+							if (event.type === "response.output_text.delta") send({ type: "text.delta", text: event.delta });
+							if (event.type === "response.output_item.done") items.push(event.item);
+							if (event.type === "error" || event.type === "response.failed") {
+								if (process.env.COZYCLAY_AGENT_DEBUG) console.error("[agent] model event:", JSON.stringify(event).slice(0, 600));
+								const code = event.error?.code ?? event.response?.error?.code;
+								throw Object.assign(new Error("Model response failed."), code === "server_is_overloaded" ? { code: "overloaded" } : code === "server_error" ? { code: "server_error" } : {});
+							}
+						}
+						await headers;
+						return items;
+					} finally { await headers; }
+				}));
+				for (const item of output) {
+					history.push(item); // Preserve reasoning items verbatim.
+					if (item.type === "function_call") {
+						const result = await executeTool(item);
+						history.push({ type: "function_call_output", call_id: item.call_id, output: JSON.stringify(result) });
+					}
+				}
+				if (!output.some((item) => item.type === "function_call")) break;
+			}
+			session.history = history;
+		};
+		try { await requestContext.run(observeHeaders, turn); }
+		catch (error) {
+			if (!signal.aborted) {
+				if (error.headers) observeHeaders(error.headers);
+				if (process.env.COZYCLAY_AGENT_DEBUG) console.error("[agent] turn failed:", error?.status, error?.message, String(error?.body ?? "").slice(0, 300));
+				send({ type: "error", ...errorInfo(error, quota) });
+			}
+		} finally {
+			session.running = false; send({ type: "done" }); res.end(); res.off("close", disconnect);
+		}
+		return true;
+	};
+	handle.close = async () => {
+		unsubscribe?.();
+		for (const session of sessions.values()) session.controller?.abort();
+		sessions.clear();
+		const { liveHub: hub } = await runtime;
+		if (hub?.server) {
+			for (const socket of hub.server.clients) socket.terminate();
+			await new Promise((resolve) => hub.server.close(resolve));
+		}
+	};
+	return handle;
+}

package/bin/agent/agent-tools.mjs

@@ -0,0 +1,164 @@
+import { randomUUID } from "node:crypto";
+
+const objectSchema = (properties = {}, required = []) => ({ type: "object", properties, required, additionalProperties: false });
+export const SYSTEM_PROMPT = "You are CozyClay's workflow agent. The user is looking at the Workflow canvas, and you work by building and running nodes on that canvas, so every step is visible and editable. Always call describe_workflow first to read the current graph. To render the scene in a new look: reuse the existing Scene node, add an Image node with model image-generation whose data.prompt is the user's intent, connect the Scene node to it with connect_workflow_nodes (sourceHandle render, targetHandle input), and when the user attached or mentioned a reference image, add one with add_reference_node and connect it to the same Image node's input handle; then call run_workflow. Finish with one or two sentences naming the nodes you created. Never describe results you did not run. Keep responses concise and practical.";
+
+/** The Workflow page embeds the Studio as a live preview, so the hub usually
+ * sees at least two editors. Prefer the tab the user is authoring in: any
+ * workspace whose hello meta does not say embed:true. Fall back to the hub's
+ * own single-workspace rule (which throws when the choice is ambiguous). */
+export function pickWorkspace(liveHub, requiredCommands = ["capture_framing_png", "import_asset"], kind = "scene") {
+	const details = typeof liveHub.workspaceHandleDetails === "function" ? liveHub.workspaceHandleDetails() : [];
+	if (kind === "workflow") {
+		const workflow = details.filter((entry) => entry.meta?.kind === "workflow").map((entry) => entry.handle);
+		if (workflow.length) return workflow[workflow.length - 1];
+		throw new Error("No workflow canvas workspace is connected.");
+	}
+	// An editor that does not advertise its commands predates the agent work;
+	// it cannot answer capture_framing_png, so it is never a candidate.
+	const supports = (entry) => Array.isArray(entry.meta?.commands) && requiredCommands.every((name) => entry.meta.commands.includes(name));
+	const authoring = details.filter((entry) => entry.meta?.embed !== true && supports(entry)).map((entry) => entry.handle);
+	if (authoring.length === 1) return authoring[0];
+	if (authoring.length > 1) return authoring[authoring.length - 1];
+	// On the Workflow page the embedded Studio IS the scene the user is looking
+	// at; use it when no standalone editor tab is open.
+	const embedded = details.filter((entry) => entry.meta?.embed === true && supports(entry)).map((entry) => entry.handle);
+	if (embedded.length) return embedded[embedded.length - 1];
+	// The hub's own rule would hand back whatever single workspace exists —
+	// on the Workflow page that is the canvas, which cannot capture a frame.
+	if (details.some((entry) => entry.meta?.kind === "workflow")) throw new Error("No scene editor is connected yet.");
+	return liveHub.resolveWorkspace("agent turn");
+}
+
+/** What the model gets back from a canvas command: ids, types, models and
+ * small fields only. Data URLs become a size note and the echoed graph is
+ * dropped — the model can call describe_workflow when it needs the graph. */
+export function summariseCanvasResult(result) {
+	const strip = (value) => {
+		if (typeof value === "string") return value.startsWith("data:") ? `[image ${Math.round(value.length * 3 / 4 / 1024)} KB]` : value;
+		if (Array.isArray(value)) return value.map(strip);
+		if (value && typeof value === "object") return Object.fromEntries(Object.entries(value).map(([key, item]) => [key, strip(item)]));
+		return value;
+	};
+	if (!result || typeof result !== "object") return result;
+	const { graph, ...rest } = result;
+	return strip(rest);
+}
+
+export function createAgentTools({ liveHub, handlers = [], session, emit }) {
+	const registry = new Map(handlers.map((tool) => [tool.name, tool]));
+	const workspace = (kind = "scene") => {
+		const key = kind === "workflow" ? "workflowHandle" : "workspaceHandle";
+		// The panel session outlives page reloads, so a cached handle may name an
+		// editor that is gone or the wrong kind of workspace; re-pick when it does.
+		const details = typeof liveHub?.workspaceHandleDetails === "function" ? liveHub.workspaceHandleDetails() : [];
+		const current = details.find((entry) => entry.handle === session[key]);
+		const fits = current && ((current.meta?.kind === "workflow") === (kind === "workflow"));
+		if (liveHub?.resolveWorkspace && !fits) session[key] = pickWorkspace(liveHub, kind === "workflow" ? [] : undefined, kind);
+		return session[key];
+	};
+	// The embedded Studio says hello a moment after the canvas; give it up to
+	// ten seconds before declaring that no scene editor exists.
+	const warmingUp = (error) => /scene editor|renderer is not ready/i.test(error?.message ?? "");
+	const untilReady = async (attempt) => {
+		const deadline = Date.now() + 10_000;
+		for (;;) {
+			try { return await attempt(); } catch (error) {
+				if (!warmingUp(error) || Date.now() >= deadline) throw error;
+				session.signal?.throwIfAborted?.();
+				await new Promise((resolve) => setTimeout(resolve, 250));
+			}
+		}
+	};
+	const live = (name, args = {}, kind = "scene") => {
+		if (!liveHub) throw new Error("Live editor is not connected.");
+		return liveHub.command(name, args, workspace(kind));
+	};
+	const registered = async (name, args = {}) => {
+		const tool = registry.get(name);
+		if (!tool) throw new Error(`Tool ${name} is unavailable.`);
+		const result = await tool.handler(args, { workspaceHandle: workspace("scene") });
+		if (result?.isError) throw new Error("The live editor could not describe the scene.");
+		return result;
+	};
+	const capture = {
+		name: "capture_blocking_frame", description: "Capture the current blocking frame before rendering.", parameters: objectSchema(),
+		handler: async () => {
+			const result = await untilReady(() => live("capture_framing_png"));
+			session.signal.throwIfAborted();
+			const imageId = randomUUID();
+			session.images.set(imageId, result.dataUrl);
+			session.latestCaptureId = imageId;
+			return { imageId, width: result.width, height: result.height };
+		},
+	};
+	const render = {
+		name: "render_from_frame", description: "Render an edited image from a captured frame.",
+		parameters: objectSchema({ prompt: { type: "string" }, imageId: { type: "string" }, quality: { type: "string", enum: ["auto", "low", "medium", "high"] }, addAsNode: { type: "boolean" } }, ["prompt"]),
+		handler: async ({ prompt, imageId, quality, addAsNode = false }) => {
+			if (typeof prompt !== "string" || !prompt.trim()) throw new Error("A render prompt is required.");
+			if (quality !== undefined && !["auto", "low", "medium", "high"].includes(quality)) throw new Error("Invalid image quality.");
+			const source = session.images.get(imageId || session.latestCaptureId);
+			if (!source) throw new Error("Capture a frame before rendering.");
+			const guidance = registry.has("render_prompt") ? await registered("render_prompt", { mode: "image", environment: prompt }) : null;
+			const suffix = typeof guidance === "string" ? guidance : (guidance?.content ?? []).filter((part) => part.type === "text").map((part) => part.text).join("\n");
+			const fullPrompt = `${prompt}${suffix ? `\n${suffix}` : ""}`;
+			let result;
+			try {
+				session.signal.throwIfAborted();
+				result = await session.codex.editImage({ prompt: fullPrompt, imageDataUrl: source, quality, signal: session.signal });
+			} catch (error) {
+				if (/entitlement|plan/i.test(error.message)) error.code = "entitlement";
+				throw error;
+			}
+			session.signal.throwIfAborted();
+			const newId = randomUUID();
+			const dataUrl = `data:image/png;base64,${result.pngBase64}`;
+			session.images.set(newId, dataUrl);
+			emit({ type: "image", imageId: newId, dataUrl, width: result.width, height: result.height, prompt: fullPrompt });
+			if (addAsNode) await live("add_node", { type: "image", model: "image-passthrough", data: { image_url: dataUrl, outputs: [{ value: dataUrl }] } }, "workflow");
+			return { imageId: newId, width: result.width, height: result.height, ...(addAsNode ? { addedAsNode: true } : {}) };
+		},
+	};
+	const place = {
+		name: "place_image_in_scene", description: "Place a rendered image in the scene.",
+		parameters: objectSchema({ imageId: { type: "string" }, placeAs: { type: "string", enum: ["cutout", "backdrop"] } }, ["imageId"]),
+		handler: async ({ imageId, placeAs = "cutout" }) => {
+			const dataUrl = session.images.get(imageId);
+			if (!dataUrl) throw new Error("Unknown image.");
+			if (!["cutout", "backdrop"].includes(placeAs)) throw new Error("Invalid image placement.");
+			return live("import_asset", { name: `${imageId}.png`, mimeType: "image/png", dataUrl, placeAs });
+		},
+	};
+	const reference = {
+		name: "add_reference_node", description: "Add an attached or captured image to the canvas as a reference upload node.",
+		parameters: objectSchema({ imageId: { type: "string" } }),
+		handler: async ({ imageId } = {}) => {
+			const dataUrl = session.images.get(imageId ?? session.latestCaptureId);
+			if (!dataUrl) throw new Error("No reference image is available. Capture or attach one first.");
+			return summariseCanvasResult(await live("add_node", { type: "upload", data: { image_url: dataUrl, fileName: "reference.png", mimeType: "image/png", outputs: [{ value: dataUrl }] } }, "workflow"));
+		},
+	};
+	const workflow = [
+		["describe_workflow", "Describe the current workflow canvas.", "get_graph", objectSchema()],
+		["add_workflow_node", "Add a node to the workflow canvas.", "add_node", objectSchema({ type: { type: "string" }, model: { type: "string" }, data: { type: "object" }, position: { type: "object" } }, ["type"])],
+		["update_workflow_node", "Update a workflow node.", "update_node", objectSchema({ id: { type: "string" }, data: { type: "object" } }, ["id", "data"])],
+		["remove_workflow_node", "Remove a workflow node.", "remove_node", objectSchema({ id: { type: "string" } }, ["id"])],
+		["connect_workflow_nodes", "Connect two workflow nodes.", "connect", objectSchema({ source: { type: "string" }, target: { type: "string" }, sourceHandle: { type: "string" }, targetHandle: { type: "string" } }, ["source", "target"])],
+		["disconnect_workflow_nodes", "Disconnect workflow nodes.", "disconnect", objectSchema({ edgeId: { type: "string" } }, ["edgeId"])],
+		["run_workflow", "Run the workflow locally.", "run_workflow", objectSchema()],
+		["set_workflow_node_output", "Set a workflow node output.", "set_node_output", objectSchema({ id: { type: "string" }, value: {} }, ["id", "value"])],
+		["focus_workflow_node", "Focus a workflow node.", "focus_node", objectSchema({ id: { type: "string" } }, ["id"])],
+	].map(([name, description, command, parameters]) => ({ name, description, parameters, handler: async (args) => summariseCanvasResult(await live(command, args, "workflow")) }));
+	const direct = ["describe_scene", "describe_shot"].map((name) => ({
+		name, description: registry.get(name)?.description || name,
+		parameters: objectSchema(), handler: () => registered(name),
+	}));
+	const tools = [reference, ...workflow, ...direct];
+	// The sidecar captures the frame itself when the user attaches it; the
+	// model never sees this tool, it builds an Image node instead.
+	tools.internal = { capture };
+	return tools;
+}
+
+export const agentToolSchemas = (tools) => tools.map(({ name, description, parameters }) => ({ type: "function", name, description, parameters }));