realtime-avatar-mcp 0.4.1 → 0.5.1

package/README.md

@@ -29,24 +29,28 @@ Working on it in this repo? `npm run build`, then point `command` at `node` with
 | `get_avatar` | Full detail for one |
 | `credit_balance` | Balance, and what in-flight calls have reserved |
 | `list_sessions` | The itemised bill — when each session ran, how long, what it cost |
+| `list_clips` | An avatar's clip library: every clip, its status, the revision and rest pose |
 
 With `REALTIME_AVATAR_ALLOW_WRITES=1`:
 
 | Tool | | Bills? |
 | --- | --- | --- |
-| `sync_clips` | Prepare an avatar's clip set. Idempotent | no |
+| `set_loop` | Re-direct her resting loop from a new description | no |
+| `set_clip_library` | Declare an avatar's full clip set as JSON — the platform renders it | no |
+| `sync_clips` | **Deprecated.** The external-URL tier. Use `set_clip_library` | no |
 | `upload_asset` | Upload a file **from this machine's disk**, get a public URL | no |
 | `create_remote_asset` | Register a file already on the internet — no local copy | no |
-| `create_avatar_from_video` | Build an avatar from a looping video URL | no |
+| `create_avatar_from_image` | Build an avatar from ONE still — the loop is generated | no |
+| `create_avatar_from_video` | **Deprecated.** Closed to new callers (422). Use the image tool | no |
 | `start_call` | Mint a live session to verify an integration | **yes, per second** |
 
 ## Spending money is opt-in, and twice gated
 
-The four tools above are **read-only**, and each carries `readOnlyHint` so a host can gate on
+The five tools above are **read-only**, and each carries `readOnlyHint` so a host can gate on
 the annotation rather than on a name it has to recognise. Pointed at a production key, this
 server is no more dangerous than a dashboard you left open.
 
-`REALTIME_AVATAR_ALLOW_WRITES=1` adds the five write tools. Only `start_call` costs credits,
+`REALTIME_AVATAR_ALLOW_WRITES=1` adds the eight write tools. Only `start_call` costs credits,
 and it **refuses a `tic_live_` key outright** — an operator who armed writes against a test
 key and later swapped in a production one should not discover it by being billed. Two
 independent gates, because one is a single mistake away from being none.
@@ -56,16 +60,19 @@ Two things to understand before arming writes:
 - **`upload_asset` reads local disk.** It opens whatever absolute path the agent names, on the
   machine running the server. Relative paths are refused rather than resolved against
   whatever directory the host spawned it in.
-- **`sync_clips` retires by omission.** Pass the complete set you want live; anything missing
-  from the list stops being served. The tool reports all three buckets — queued, ready,
-  retired — because "queued: 1" alone reads like nothing else changed.
+- **The clip tools retire by omission.** Both `set_clip_library` and `sync_clips` take the
+  COMPLETE set you want live; anything missing from it stops being served. Each reports all
+  three buckets — kept/queued/retired — because "queued: 1" alone reads like nothing else
+  changed. Declare with `set_clip_library`: clips are JSON (`motionPrompt` or an uploaded
+  `assetId`) and the platform renders and hosts them. `sync_clips` is the deprecated tier
+  that takes URLs on your own storage.
 
 ## Why an MCP server at all
 
 An agent that reads docs still guesses. `ava_…` ids cannot be inferred, and the failures that
-cost the most time here are invisible in a type signature: an image-sourced avatar reports
-`ready` and publishes a black video track; a reshaped connection payload is rejected by the
-browser client; a 429 on a call is the queue, not an error.
+cost the most time here are invisible in a type signature: a reshaped connection payload is
+rejected by the browser client; a 429 on a call is the queue, not an error; the resting loop
+is the avatar's source rather than a library clip, so no declaration changes it.
 
 `list_avatars` marks which avatars are actually usable. The server's `instructions` carry the
 relay rule and the per-second billing model. That is context an agent cannot derive, delivered

package/dist/server.d.ts

@@ -5,7 +5,7 @@ import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
  * `test/server.test.ts` asserts the two stay equal; the equivalent constant in
  * `realtime-avatar` had that guard and this one did not, which is how it drifted.
  */
-export declare const MCP_VERSION = "0.4.1";
+export declare const MCP_VERSION = "0.5.1";
 /**
  * The Realtime Avatar MCP server.
  *

package/dist/server.js

@@ -10,7 +10,7 @@ import { RealtimeAvatar, isQueued } from "realtime-avatar";
  * `test/server.test.ts` asserts the two stay equal; the equivalent constant in
  * `realtime-avatar` had that guard and this one did not, which is how it drifted.
  */
-export const MCP_VERSION = "0.4.1";
+export const MCP_VERSION = "0.5.1";
 const MICROS_PER_CREDIT = 1_000_000;
 /** Extension -> asset kind, so a caller does not have to restate what the filename says. */
 const KIND_BY_EXT = {
@@ -51,19 +51,26 @@ export function createServer(options) {
     server.registerTool("list_avatars", {
         title: "List avatars",
         description: "Every avatar on this account, with the id you pass to startCall. Call this before " +
-            "writing code — avatar ids cannot be guessed. `sourceKind: 'image'` avatars publish " +
-            "a BLACK video track on a live call; prefer 'video'.",
+            "writing code — avatar ids cannot be guessed. Callable means status 'ready' AND " +
+            "idleVideoStatus 'ready': the loop is attached. An avatar reads sourceKind 'video' " +
+            "once that happens, including the ones built from a single image — that is the " +
+            "normal end state, not a warning.",
         inputSchema: {},
         annotations: { readOnlyHint: true, openWorldHint: true },
     }, async () => {
         const avatars = await rta.listAvatars();
         if (avatars.length === 0) {
-            return text("No avatars yet. Create one from a video URL before starting a call.");
+            return text("No avatars yet. Create one from a single portrait image before starting a call.");
         }
-        const rows = avatars.map((a) => `${a.id}  ${a.status.padEnd(13)} ${a.sourceKind.padEnd(5)} ${a.displayName}`);
-        const usable = avatars.filter((a) => a.status === "ready" && a.sourceKind === "video");
-        return text(`${avatars.length} avatar(s). ${usable.length} ready + video-sourced (usable for a live call).\n\n` +
-            `id                                    status        kind  name\n${rows.join("\n")}`);
+        const rows = avatars.map((a) => `${a.id}  ${a.status.padEnd(13)} ${a.idleVideoStatus.padEnd(10)} ${a.displayName}`);
+        // Usable = ready WITH A LOOP ATTACHED. The old test was `sourceKind === "video"`,
+        // which happened to work only because an image-sourced avatar flips to `video` once
+        // its loop lands — it was reading the consequence, not the fact. `idleVideoStatus` is
+        // the fact, and it also distinguishes the three ways an avatar can be un-callable:
+        // still rendering (`queued`/`generating`), never had one (`none`), gave up (`failed`).
+        const usable = avatars.filter((a) => a.status === "ready" && a.idleVideoStatus === "ready");
+        return text(`${avatars.length} avatar(s). ${usable.length} ready with a loop attached (usable for a live call).\n\n` +
+            `id                                    status        loop        name\n${rows.join("\n")}`);
     });
     server.registerTool("get_avatar", {
         title: "Get one avatar",
@@ -109,6 +116,15 @@ export function createServer(options) {
             `total ${credits(total)}` +
             (page.nextCursor ? "\n(more pages available)" : ""));
     });
+    server.registerTool("list_clips", {
+        title: "List an avatar's clip library",
+        description: "The declared clip library: every non-retired clip with its render status, plus " +
+            "the library revision (pass it to set_clip_library as expectedRevision), the pose " +
+            "anchor and eligibility. status is the render JOB, not serveability — a clip " +
+            "re-rendering keeps serving its previous take.",
+        inputSchema: { avatarId: z.string() },
+        annotations: { readOnlyHint: true, openWorldHint: true },
+    }, async ({ avatarId }) => text(JSON.stringify(await rta.listClips(avatarId), null, 2)));
     if (!options.allowWrites)
         return server;
     // ── writes, opt-in only ────────────────────────────────────────────────────
@@ -153,9 +169,70 @@ export function createServer(options) {
             "Relay this payload to the browser byte-for-byte — the client SDK validates it " +
             "strictly and rejects a reshaped object.");
     });
+    server.registerTool("set_clip_library", {
+        title: "Declare an avatar's clip library",
+        description: "Declare the avatar's FULL desired clip library — a declaration, not a delta: " +
+            "unchanged clips are kept, new or changed ones are queued to render, omitted ones " +
+            "are retired. The 202 is acceptance, not readiness — poll list_clips until no row " +
+            "is queued or generating. Pass expectedRevision from list_clips so a concurrent " +
+            "writer surfaces as a 409 instead of a lost update. Does not spend credits.",
+        inputSchema: {
+            avatarId: z.string(),
+            clips: z.array(z.object({
+                clipId: z.string().describe("Stable id you choose; same id + same source = kept"),
+                role: z.enum(["idle", "listen", "gesture"]),
+                whenHint: z.string().optional().describe("Briefed to the character, like an actor"),
+                source: z.union([
+                    z.object({ motionPrompt: z.string() }),
+                    z.object({ assetId: z.string() }),
+                ]).describe("motionPrompt renders motion; assetId uploads a clip that must start AND end on the rest pose"),
+                durationSeconds: z.number().optional(),
+                reroll: z.boolean().optional().describe("Set true to force a re-render of the same prompt"),
+            })).max(12).describe("The COMPLETE library. Omitted clips are retired."),
+            expectedRevision: z.number().int().optional()
+                .describe("CAS: the revision you last read. Omit to declare unconditionally."),
+        },
+        annotations: { readOnlyHint: false, destructiveHint: false, idempotentHint: true, openWorldHint: true },
+    }, async ({ avatarId, clips, expectedRevision }) => {
+        const update = await rta.setClipLibrary(avatarId, { clips, expectedRevision });
+        const bucket = (label, ids) => `${label} (${ids.length})${ids.length ? `  ${ids.join(", ")}` : ""}`;
+        return text(`revision ${update.revision}\n` +
+            [
+                bucket("kept — still serving", update.plan.kept),
+                bucket("queued — rendering now", update.plan.queued),
+                bucket("retired — no longer live", update.plan.retired),
+            ].join("\n") +
+            (update.plan.queued.length
+                ? "\n\nQueued clips render in the background — poll list_clips until no row is queued or generating."
+                : ""));
+    });
+    server.registerTool("set_loop", {
+        title: "Re-direct the resting loop",
+        description: "Re-generate the RESTING LOOP — the video she plays when nothing else is happening — " +
+            "from a new one-sentence description. NOT a clip: a clip with role 'idle' is a " +
+            "variant spliced over the loop, and declaring one never changes what she rests in. " +
+            "Accepted immediately (202) and rendered over minutes; she stays ready and keeps " +
+            "serving her previous loop the whole time, and the clip library is untouched. Bills " +
+            "one video generation per call, so do not send it speculatively.",
+        inputSchema: {
+            avatarId: z.string(),
+            motionPrompt: z.string().max(1200)
+                .describe("How she should idle. Describe a CLOSED arc that ends where it began, or the " +
+                "loop snaps every time it wraps — e.g. 'leans in, listening, a slow blink, " +
+                "settles back'."),
+        },
+        annotations: { readOnlyHint: false, destructiveHint: false, openWorldHint: true },
+    }, async ({ avatarId, motionPrompt }) => {
+        const result = await rta.setLoop(avatarId, { motionPrompt });
+        return text(`Re-direct accepted (${result.loopStatus}).\n` +
+            `She is still playing the previous loop until the new one is ready:\n  ${result.servingUrl ?? "—"}\n\n` +
+            "Poll get_avatar; the swap publishes in one step. Her clips are unaffected.");
+    });
     server.registerTool("sync_clips", {
         title: "Sync an avatar's clips",
-        description: "Prepare an avatar's clip set. Clips are prepared once and cached by URL hash, and " +
+        description: "DEPRECATED — this serves the sunsetting external-URL clip tier; declare the " +
+            "library with set_clip_library instead. Clips are prepared once and cached by URL " +
+            "hash, and " +
             "the serve path only LOADS that cache — so a clip you added but never synced does " +
             "nothing at all on the next call, silently. Idempotent: call it after every clip " +
             "change. Pass the complete set you want live; anything omitted is retired. This does " +
@@ -226,10 +303,33 @@ export function createServer(options) {
         const asset = await rta.createRemoteAsset({ kind, remoteUrl });
         return text(`Created ${asset.id} (${kind})\n${asset.url}`);
     });
+    server.registerTool("create_avatar_from_image", {
+        title: "Create an avatar from one still image",
+        description: "Build an avatar from a SINGLE still image — the shortest path, and the one to reach " +
+            "for by default. The platform generates the resting loop she idles in and a starter " +
+            "motion library rendered against her rest pose; no footage and no clip URLs are " +
+            "involved. 'motionPrompt' directs the resting loop and is the ONLY chance to direct " +
+            "it — no endpoint re-generates a loop after creation. Returns while she is still " +
+            "'preprocessing'; poll get_avatar.",
+        inputSchema: {
+            displayName: z.string(),
+            imageUrl: z.string().url().describe("Publicly reachable still of the character, face in frame"),
+            motionPrompt: z.string().max(1200).optional()
+                .describe("Art direction for the generated resting loop. Describe a CLOSED arc that ends " +
+                "where it began, or the loop snaps every time it wraps — e.g. 'settles into " +
+                "frame, breathes gently, a slow blink'. Omit for the house default."),
+        },
+        annotations: { readOnlyHint: false, destructiveHint: false, openWorldHint: true },
+    }, async ({ displayName, imageUrl, motionPrompt }) => {
+        const avatar = await rta.createAvatarFromImage({ displayName, imageUrl, motionPrompt });
+        return text(`Created ${avatar.id} (${avatar.status}). Poll get_avatar until it is ready.`);
+    });
     server.registerTool("create_avatar_from_video", {
-        title: "Create an avatar from a video",
-        description: "Build an avatar from a looping video URL. Use a VIDEO source: an avatar built from " +
-            "a still image reaches 'ready' and then publishes a black track on every call.",
+        title: "Create an avatar from a video (deprecated)",
+        description: "DEPRECATED AND CLOSED — this answers 422 unless the tenant was already creating " +
+            "from video. Use create_avatar_from_image instead: one still, and the platform " +
+            "renders the loop and the motion library from it. Do not reach for this tool " +
+            "because an image is inconvenient to obtain; it will simply fail.",
         inputSchema: {
             displayName: z.string(),
             videoUrl: z.string().url().describe("Publicly reachable mp4, opening and closing on the same rest pose"),

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "realtime-avatar-mcp",
-  "version": "0.4.1",
+  "version": "0.5.1",
   "description": "MCP server for Realtime Avatar — let a coding agent read your avatars, balance and bill directly.",
   "license": "MIT",
   "author": "The Influence Company",
@@ -55,7 +55,7 @@
   },
   "dependencies": {
     "@modelcontextprotocol/sdk": "^1.30.0",
-    "realtime-avatar": "0.4.1",
+    "realtime-avatar": "0.5.1",
     "zod": "^4.4.3"
   }
 }