@shipvo/mcp 0.0.8 → 0.0.10

package/README.md

@@ -32,16 +32,25 @@ The result gets a line `Waited: matched | timed out after <ms> ms` and `structur
 | `stale_ref` | The ref is not on the current screen, or the screen changed since it was read. | Call `screen` and use a ref from the new tree. |
 | `unknown_app` | The app id is not one this server or key can open. | Call `list_apps` and use an id from it. |
 | `sim_timeout` | The simulator did not answer in time. The action may have run. | Call `screen` to see where the app is. Do not repeat the action blindly. |
-| `needs_confirm` | An owner command that cannot be undone did nothing and returned a plan. | Show the plan to the owner, then call again with `confirm: true`. |
+| `needs_confirm` | An owner command that cannot be undone did nothing and returned a plan and a one-time `challenge`. | Show the plan to the owner and ask them to approve it at `challenge.approval_url` in the admin. Then call again with the same args and `approval: <challenge id>`. `confirm: true` does nothing. |
 | `bad_url` | `open_url` got a URL the app cannot take. | Fix the URL and call again. |
 | `no_url_listener` | The app is not listening for URLs. | Use the app's screens instead. |
+| `phone_locked` | The phone is locked, so tap, type, scroll and back were not sent. | Call `unlock`, then try again. |
+| `already_locked` | `lock` was called on a locked phone. | Call `unlock` if you want to use the app. |
+| `not_locked` | `unlock` was called on a phone that is not locked. | Carry on; nothing to do. |
+| `app_not_foreground` | The home screen or app switcher is showing, so tap, type, scroll, back, `open_url` and `open_notification` were not sent. | Call `resume_app`. |
+| `recents_not_open` | `kill_app` was called without the app switcher open. | Call `open_recents` first. |
 | `mock_invalid` | A network mock rule is not valid. | Fix the rule. |
 
-Tools: `list_apps`, `open_app`, `screen`, `tap`, `type`, `scroll`, `back`, `screenshot`, `set_device`, `push_notification`, `notifications`, `open_notification`, `device_log`, `export_test`, `end_session`, `owner_describe`, `owner_command`.
+Tools: `list_apps`, `open_app`, `attach_session`, `screen`, `tap`, `type`, `scroll`, `back`, `screenshot`, `set_device`, `push_notification`, `notifications`, `open_notification`, `device_log`, `export_test`, `end_session`, `owner_describe`, `owner_command`, `approvals_list`, `approval_approve` (both need an owner token, and the MCP agent counts as an agent: it cannot approve an agent's hold).
 
 `export_test { session }` is read-only. It turns the taps, typing, scrolls, `back` and `set_device` calls recorded in a session into a `@shipvo/test` script and returns the source. Targets are named by visible text; an ambiguous target becomes a `// TODO` comment, and a password-like field uses `process.env.SHIPVO_TEST_<LABEL>` so the typed value is never in the output. Only sessions opened through this server are recorded; an unknown session returns `no_session`. Steps are dropped when the session ends. `payment.pin` is removed from recorded `set_device` calls. Only labels and roles that look like a password, passcode or PIN are treated as secret; other secret-like fields (card number, token) are exported as typed. `type` with `enter: true` and `scroll` with a `ref` are recorded without the Enter or the ref, so a replay can differ from the session.
 
-**Owner commands that cannot be undone need `confirm: true`.** `owner_command` takes an optional `confirm`. `publish`, price changes, offers, deletes and API keys first return `{ code: "needs_confirm", plan }` and do nothing; call again with the same `args` and `confirm: true` to run. `agentPolicy` (`ask` default, `deny`, `allow`) and `agentDeny` in `shipvo.json` (read from `PN_APP_DIR` or the current folder) can only tighten the control plane's rules. Details: `docs/agents.md`.
+## Record a demo video
+
+`record_start`, `record_marker` and `record_stop` write an MP4 of the phone to `.shipvo/recordings/`. `record_stop` takes `music: true` (a ducked music bed and a `.credits.json`) and `narration` (a path to a wav). It needs ffmpeg and `playwright`. Music works only from a repo checkout, or when `SHIPVO_BGMUSIC` points at `scripts/video/bgmusic.py`; otherwise `record_stop` returns `music_unavailable` (and `music_failed` if the mix fails). A runnable example is in [`docs/record-a-demo-video.md`](../../docs/record-a-demo-video.md).
+
+**Owner commands that cannot be undone need the owner's approval.** `publish`, price changes, offers, deletes and API keys first return `{ code: "needs_confirm", plan, challenge }` and change nothing. The owner approves the challenge in the admin (an agent cannot), then `owner_command` is called again with the same args and `approval: "<challenge id>"`. An approval works once, for 10 minutes, for exactly those arguments. `approvals_list` shows pending challenges. With `agentPolicy: "allow"` agents act without asking.
 
 Native features work with no phone: the simulated device (see `@shipvo/device`) shows permission dialogs, the camera, the fingerprint prompt, payment prompts, notification banners and the share sheet as real dialogs in the tree; `set_device` decides what the phone has (location, a barcode in view, whether the fingerprint matches, whether a payment is approved, network off).
 
@@ -76,7 +85,7 @@ The HTTP endpoint is stateless. There is no protocol session: each POST is handl
 ### Protocol versions
 
 - **2025-era clients** (`initialize`, then calls, with or without `Mcp-Session-Id`): work. `initialize` is answered, no session id is issued, and any `Mcp-Session-Id` a client sends is ignored.
-- **2026-07-28 clients**: `tools/list` and `tools/call` work with no session and no `initialize`. Not done yet: `server/discover`, the `Mcp-Method`/`Mcp-Name` headers, `ttlMs`/`cacheScope` on list results. `tools/list` order is fixed (registration order).
-- This package uses `@modelcontextprotocol/sdk` 1.x. Stdio is unchanged.
+- **2026-07-28 clients**: `tools/list` and `tools/call` work with no session and no `initialize`. `server/discover` is answered (supported versions, tools capability, server info). A request counts as 2026-era if it calls `server/discover`, sends `Mcp-Method`, or names a 2026 protocol version in `params._meta`; those must carry `Mcp-Method` (and `Mcp-Name` for `tools/call`, `prompts/get`, `resources/read`) matching the body, or get a 400 JSON-RPC error. Requests with no 2026 marker are not checked. Header mismatches return `-32020`; naming a protocol version we do not support returns `-32022`. 2026-era results carry `resultType: "complete"`, and `tools/list` results also carry `ttlMs` and `cacheScope: "private"` (the list depends on the per-request scope headers). 2025-era responses have none of these. `tools/list` is sorted by tool name on stdio and HTTP.
+- This package uses `@modelcontextprotocol/sdk` 1.x (the TypeScript SDK v2 is not stable on npm yet; 1.32.2 is `latest`). Tested by `test/http-stateless.test.ts`: discover without `initialize`, header/body mismatch, scope per request, 2025-era `initialize` clients. Stdio is unchanged.
 
  Requests without a valid key are refused. The hosted version at `sim.shipvo.dev` will issue keys per account with scopes (test, owner, read).

package/dist/attach.d.ts

@@ -0,0 +1,30 @@
+export type RunInfo = {
+    port: number;
+    platform: "android" | "ios";
+    app: string;
+    token: string;
+    pid: number;
+};
+export type DriverMessage = {
+    id: number;
+    type: string;
+    [key: string]: unknown;
+};
+export type ScreenMessage = {
+    id?: number;
+    type?: string;
+    ok?: boolean;
+    error?: string;
+    code?: string;
+    screen?: string | null;
+    [key: string]: unknown;
+};
+/** Finds `.shipvo/run.json` in `cwd` or a parent, stopping after the git root. Returns it only if the pid is alive and the app matches. */
+export declare function findLiveRun(app: string | undefined, cwd?: string): RunInfo | undefined;
+export type Driver = {
+    request(type: string, args?: Record<string, unknown>, ms?: number): Promise<ScreenMessage>;
+    close(): void;
+    closed: boolean;
+};
+/** Connects as `driver`. Rejects if the socket cannot open within `ms`. */
+export declare function connectDriver(info: RunInfo, ms?: number): Promise<Driver>;

package/dist/attach.js

@@ -0,0 +1,85 @@
+// Attach to a live `shipvo run` phone. `open_app` reads `.shipvo/run.json`, and if the run is alive and is the
+// same app, drives it over the agent channel as the `driver`. The types below mirror
+// packages/cli/src/run-protocol.ts: MCP cannot import from the CLI (rootDir is src, and the CLI is not a dependency).
+import { existsSync, readFileSync } from "node:fs";
+import { dirname, join, resolve } from "node:path";
+const alive = (pid) => { try {
+    process.kill(pid, 0);
+    return true;
+}
+catch (e) {
+    return e.code === "EPERM";
+} };
+// Safety: run.json may come from a parent directory, but we only ever connect to 127.0.0.1 and send its token there.
+/** Finds `.shipvo/run.json` in `cwd` or a parent, stopping after the git root. Returns it only if the pid is alive and the app matches. */
+export function findLiveRun(app, cwd = process.cwd()) {
+    let dir = resolve(cwd);
+    for (;;) {
+        const file = join(dir, ".shipvo", "run.json");
+        if (existsSync(file)) {
+            try {
+                const i = JSON.parse(readFileSync(file, "utf8"));
+                if (typeof i.port === "number" && typeof i.token === "string" && typeof i.pid === "number" && typeof i.app === "string" && (app === undefined || i.app === app) && alive(i.pid))
+                    return i;
+            }
+            catch { }
+            return undefined; // the nearest run.json decides
+        }
+        if (existsSync(join(dir, ".git")) || dirname(dir) === dir)
+            return undefined;
+        dir = dirname(dir);
+    }
+}
+/** Connects as `driver`. Rejects if the socket cannot open within `ms`. */
+export function connectDriver(info, ms = 3000) {
+    return new Promise((ok, fail) => {
+        if (typeof WebSocket === "undefined")
+            return fail(new Error("attach needs Node 22+ (no global WebSocket in this Node version)"));
+        const ws = new WebSocket(`ws://127.0.0.1:${info.port}/__shipvo/agent?role=driver&token=${encodeURIComponent(info.token)}`);
+        const pending = new Map();
+        let next = 1, opened = false;
+        const d = {
+            closed: false,
+            close: () => { try {
+                ws.close();
+            }
+            catch { } },
+            request: (type, args = {}, limit = 10000) => new Promise((res) => {
+                if (d.closed)
+                    return res({ ok: false, code: "attach_closed", error: "the shipvo run phone is gone: call open_app again" });
+                const id = next++;
+                const t = setTimeout(() => { pending.delete(id); res({ ok: false, code: "sim_timeout", error: `the live phone did not answer "${type}" within ${limit} ms. The action may still have run: call screen to see where it is.` }); }, limit);
+                pending.set(id, (m) => {
+                    clearTimeout(t);
+                    // The person using the phone paused the agent. Say what to do instead of retrying.
+                    res(m.code === "taken_over" ? { ...m, ok: false, error: "the person took over: they are using the live phone now. Stop acting and ask them to press Hand back, then call screen to see where the app is." } : m);
+                });
+                ws.send(JSON.stringify({ ...args, id, type }));
+            }),
+        };
+        const timer = setTimeout(() => { ws.close(); fail(new Error("timed out connecting to the shipvo run phone")); }, ms);
+        ws.onopen = () => { opened = true; clearTimeout(timer); ok(d); };
+        ws.onmessage = (e) => {
+            try {
+                const m = JSON.parse(String(e.data));
+                if (m.id !== undefined) {
+                    pending.get(m.id)?.(m);
+                    pending.delete(m.id);
+                }
+            }
+            catch { }
+        };
+        ws.onclose = () => {
+            d.closed = true;
+            for (const [id, f] of pending) {
+                f({ ok: false, code: "attach_closed", error: "the shipvo run phone closed the connection: call open_app again" });
+                pending.delete(id);
+            }
+            if (!opened) {
+                clearTimeout(timer);
+                fail(new Error("could not connect to the shipvo run phone"));
+            }
+        };
+        ws.onerror = () => { };
+    });
+}

package/dist/recorder.d.ts

@@ -0,0 +1,96 @@
+export type Format = "9:16" | "16:9" | "1:1";
+export declare const SIZES: Record<Format, {
+    width: number;
+    height: number;
+}>;
+export type Chapter = {
+    t: number;
+    caption: string;
+};
+export type Recording = {
+    file: string;
+    duration_s: number;
+    width: number;
+    height: number;
+    chapters: Chapter[];
+    credits?: string;
+    notes?: string[];
+    lumi?: boolean;
+    lumi_lines?: number;
+};
+export declare class RecorderError extends Error {
+    code: string;
+    constructor(code: string, message: string);
+}
+export declare const MAX_IDLE_S = 0.6;
+export type Background = string;
+export type Point = {
+    x: number;
+    y: number;
+};
+export type Step = {
+    at: number;
+    text: string;
+};
+/** Lines Lumi speaks: one per step caption, dropped when the previous line would still be playing (about 0.09 s a character plus a 0.6 s gap). Times are seconds on the final video. */
+export declare function voiceCues(steps: Step[], t0: number, shift: (ms: number) => number, total: number): [number, string, string][];
+export declare class Recorder {
+    private browser?;
+    private page?;
+    private loop?;
+    private t0;
+    private running;
+    private starting;
+    private stopping;
+    private ffErr;
+    private frames;
+    private framesDir;
+    private chapters;
+    private file;
+    private raw;
+    private size;
+    private fps;
+    private showTaps;
+    private lumi;
+    private lumiVoice;
+    private lumiOn;
+    private acts;
+    get active(): boolean;
+    start(o: {
+        url: string;
+        format?: Format;
+        fps?: 30 | 60;
+        title?: string;
+        dir?: string;
+        showTaps?: boolean;
+        background?: Background;
+        lumi?: boolean;
+        lumiVoice?: boolean;
+    }): Promise<{
+        file: string;
+        width: number;
+        height: number;
+    }>;
+    private grab;
+    /** True while a recording is running and taps should be drawn: callers use it to skip the extra locate call. */
+    get drawing(): boolean;
+    /** Soft finger dot and ripple at p (0..1 of the phone). No-op when not recording or show_taps is false. */
+    tap(p: Point): Promise<void>;
+    /** Finger trail from a to b (0..1 of the phone), for swipes and scrolls. */
+    trail(a: Point, b: Point): Promise<void>;
+    private draw;
+    /** The agent did something now. Gaps between actions longer than MAX_IDLE_S are cut from the video. */
+    action(): void;
+    /** An action that takes time. Its whole span is kept; call the returned function when it ends. */
+    actionSpan(): () => void;
+    marker(caption: string): Chapter;
+    stop(o?: {
+        music?: boolean;
+        narration?: string;
+    }): Promise<Recording>;
+    /** Encode the saved frames, each shown from the time it was taken until the next one. Resolves to ffmpeg's exit code (-1 if it did not start). */
+    private encode;
+    private cleanup;
+}
+/** Speak the cues with Lumi's voice: scripts/video/dialogue.py with the `energetic` preset. SHIPVO_DIALOGUE points at the script. Returns the wav path (next to the video). */
+export declare function lumiVoice(file: string, cues: [number, string, string][], total: number): string;