simulator-use 0.10.1 → 0.11.0

package/README.md

@@ -126,7 +126,7 @@ DemoTests/rejectsAnEmailWithoutAt(): Tests/SignInTests.swift:18: Expectation fai
 
 ## Several agents, one Mac
 
-- Each agent runs its own MCP server and holds its own Simulator through `device_use`.
+- Each agent runs its own MCP server and holds its own Simulator through `device_use`, and its own cloud phone: one agent's `cloud start` never moves another's tools, its `cloud stop` never ends another's phone, and the phone stops when the agent's server ends. A `cloud start` typed in a terminal is the command line's own.
 - Boots never overlap, across every simulator-use process on the Mac: two Simulators booting at once on an 8 GB Mac pushed it into swap for minutes.
 - `SIMULATOR_USE_MAX_BOOTED` caps how many run at once; asking for another shuts down the others first.
 - Steps on one Simulator run one at a time, across every simulator-use process on the Mac, so two agents' or two commands' taps never interleave.
@@ -184,13 +184,14 @@ From a shell: `npx simulator-use look`.
 Where this machine can't run the Simulator (Linux, Windows, a cloud agent), or for CI, the same tools run on a cloud iPhone Simulator on [NoMac](https://nomac.app/?utm_source=simulator-use), a Mac with Xcode, $0.80 an hour by the minute:
 
 ```
-simulator-use cloud start      # signs in through a link the first time (or NOMAC_API_KEY), starts a phone
+simulator-use cloud start      # signs in through a link the first time (or NOMAC_API_KEY), starts a phone (--device ipad for an iPad)
 simulator-use run              # every tool now runs there; run and test send the project first
+simulator-use cloud extend 30  # 30 minutes more (an hour by default), up to 8 hours in all
 simulator-use cloud stop       # ends it; the tools run on this machine again
 simulator-use cloud run . --tests   # CI in one line: start, build, run, test, a screenshot and a recording, stop
 ```
 
-Only the files the phone lacks are sent (a one-file change sends one file), into the account's own encrypted workspace, kept between sessions so builds stay incremental; build output and dependencies stay behind. The `cloud` tool does the same over MCP.
+Only the files the phone lacks are sent (a one-file change sends one file), into the account's own encrypted workspace, kept between sessions so builds stay incremental; build output and dependencies stay behind. Only files changed since the last send are read again (an index in simulator-use's state folder; `SIMULATOR_USE_SYNC_INDEX=0` reads every file). A CocoaPods project builds there too: `pod install` runs on the phone's host before the build when the pods are not the ones `Podfile` and `Podfile.lock` name, keeping the lock's versions, and the next build skips it. The `cloud` tool does the same over MCP.
 
 `cloud start` answers a link for a person to watch the phone in a browser and one to use it. `cloud stop` answers the session's replay: the screen as a video beside every call (typed text left out), kept for 7 days.
 
@@ -200,7 +201,7 @@ An app that talks to a backend on your machine reaches it through a tunnel (Node
 simulator-use cloud tunnel 8080    # until Ctrl-C: the phone reaches localhost:8080 here, each request printed
 ```
 
-The answer says the address the phone's apps use, such as `http://127.0.0.1:20199`; point the app there, for example with `run --env "API_URL=http://127.0.0.1:20199"`. Each request that passes is listed with its status and time (`cloud tunnel` without a port shows them over MCP). It lasts while the phone runs and comes back by itself after a dropped connection; `cloud tunnel` with port 0 closes it.
+The answer says the address the phone's apps use, such as `http://127.0.0.1:20199`; point the app there, for example with `run --env "API_URL=http://127.0.0.1:20199"`. Each request that passes is listed with its status and time (`cloud tunnel` without a port shows them over MCP). It lasts while the phone runs and comes back by itself after a dropped connection; `cloud tunnel` with port 0 closes it. While it is open, anything on the phone reaches that port: its apps, pages in its Safari and your project's build; nothing outside the phone does.
 
 On GitHub, from any runner, with the result posted on each pull request and the screenshot and recording kept as artifacts:
 
@@ -209,14 +210,16 @@ permissions:
   pull-requests: write
 steps:
   - uses: actions/checkout@v5
-  - uses: NoMac-app/simulator-use@v0.10
+  - uses: NoMac-app/simulator-use@v0.11
     with:
       api-key: ${{ secrets.NOMAC_API_KEY }}
 ```
 
+It runs the simulator-use of its own tag (`@v0.11` runs what that tag holds; `version` picks another). `path` is the project's folder, `scheme` the scheme when the project has several, and `tests: false` skips the tests (a project without a test target fails otherwise: zero tests is not a pass). A cancelled run stops its phone, and a pull request from a fork, which gets no secrets, fails with that reason. A job's log and its pull request comment can be public, so in CI (`CI` or `GITHUB_ACTIONS` set) `cloud run` prints no link that opens without signing in: files it links are kept in the run's artifacts, and the replay's link is read with the account's key (`GET https://nomac.app/api/v1/simulator/sessions/{id}`, `links.replay`).
+
 ## Privacy
 
-simulator-use sends nothing unless cloud mode is on. There is no telemetry and no network access at runtime: it runs `xcrun simctl`, the bundled AXe binary and `sips`, all on your Mac. With cloud on, it sends the project being run or tested, and the tools' calls, to NoMac, which keeps the session's replay for 7 days.
+simulator-use sends nothing unless cloud mode is on. There is no telemetry and no network access at runtime: it runs `xcrun simctl`, the bundled AXe binary and `sips`, all on your Mac. With cloud on, it sends the project being run or tested, and the tools' calls, to NoMac, which keeps the session's replay for 7 days. The project is the git repository it is in, as the files git tracks or would track (what `.gitignore` leaves out stays on your machine), or its folder when it is in none; either way never build output, dependencies, editor state or key and credential files (`.env` files but their examples, `.p8`, `.p12`, `.pfx`, `.pem`, `.key`, provisioning profiles, keystores, SSH keys, `.netrc`), even when git tracks them.
 
 ## How it works
 

package/dist/act/steps.js

@@ -186,3 +186,12 @@ export function describeTarget(t) {
         return `@${t.x},${t.y}`;
     return `${t.kind ? `${t.kind} ` : ""}${JSON.stringify(t.label)}`;
 }
+/** A step as an answer repeats it: a fill's or type's text left out, since the answer can reach logs and a replay. */
+export function maskedStep(line) {
+    const trimmed = line.trim();
+    const verb = (/^\S+/.exec(trimmed)?.[0] ?? "").toLowerCase();
+    if (verb !== "fill" && verb !== "type")
+        return trimmed;
+    const step = parseStep(trimmed);
+    return typeof step !== "string" && step.op === "fill" ? `fill ${describeTarget(step.target)} …` : `${verb} …`;
+}

package/dist/cli.js

@@ -5,9 +5,13 @@
 import { writeFile } from "node:fs/promises";
 import { tmpdir } from "node:os";
 import { join } from "node:path";
+import { setTimeout as delay } from "node:timers/promises";
+import { persistCloud } from "./cloud/account.js";
+import { onSignalExit } from "./engine/jobs.js";
 import { persistHeld } from "./engine/state.js";
 import { serve } from "./mcp.js";
 import { callTool, toContent, TOOLS, VERSION } from "./registry.js";
+import { stopServerPhone } from "./tools/cloud.js";
 import { relay, relayPorts } from "./tunnel-relay.js";
 const flag = (name) => `--${name.replace(/_/g, "-")}`;
 /** Parses `--name value` and `--switch` into arguments the tool's schema accepts. */
@@ -151,12 +155,17 @@ export async function main(argv) {
         process.exit(0);
     }
     if (command === "mcp") {
+        // The server's cloud phone is its agent's own: stopped when the server ends, by a signal or below.
+        onSignalExit(stopServerPhone);
         await serve();
-        // The client closed stdin: answer what was in flight, then go, even if a child process still holds a pipe.
+        // The client closed stdin: answer what was in flight, stop the server's phone, then go, even if a child process
+        // still holds a pipe.
+        await Promise.race([stopServerPhone().catch(() => undefined), delay(15_000, undefined, { ref: false })]);
         process.exit(0);
     }
-    // From here on each command is its own process: the Simulator chosen with `use` is kept in a file.
+    // From here on each command is its own process: the Simulator chosen with `use` and cloud mode are kept in files.
     persistHeld();
+    persistCloud();
     if (command === "call")
         return call(rest);
     const tool = TOOLS.find((t) => t.command === command || t.name === command);

package/dist/cloud/account.js

@@ -1,7 +1,8 @@
 /**
  * Cloud mode's account and phone: the NoMac account it signs in to (NOMAC_API_KEY, else the NoMac CLI's
  * ~/.nomac/credentials.json, else NoMac's device sign-in, whose key is saved there for both tools), and the phone
- * it uses, kept in simulator-use's state folder so every CLI call reaches the same one.
+ * it uses: an MCP server's own, in its memory; the CLI's in simulator-use's state folder, so every CLI call reaches
+ * the same one.
  */
 import { chmod, mkdir, readFile, writeFile } from "node:fs/promises";
 import { homedir, hostname } from "node:os";
@@ -34,7 +35,21 @@ async function saveKey(key, base) {
     await chmod(credentialsFile(), 0o600);
 }
 const stateFile = () => join(stateDir(), "cloud.json");
+/**
+ * An MCP server keeps cloud mode in its memory, as it keeps its held Simulator: each agent runs its own server, so
+ * one agent's cloud start never sends another agent's tools to its phone, and one's cloud stop never ends another's.
+ * A CLI call is its own process, so the CLI keeps it in a file (persistCloud), and cloud start carries over to the
+ * next command.
+ */
+let memory = { on: false };
+let persistent = false;
+/** The CLI keeps cloud mode between its calls. */
+export function persistCloud() {
+    persistent = true;
+}
 export async function cloudState() {
+    if (!persistent)
+        return { ...memory };
     try {
         return JSON.parse(await readFile(stateFile(), "utf8"));
     }
@@ -43,6 +58,10 @@ export async function cloudState() {
     }
 }
 export async function setCloudState(state) {
+    if (!persistent) {
+        memory = { ...state };
+        return;
+    }
     await mkdir(stateDir(), { recursive: true });
     await writeFile(stateFile(), JSON.stringify(state), { mode: 0o600 });
 }
@@ -57,6 +76,9 @@ export async function signIn() {
     const ready = await account();
     if (ready)
         return { api: ready };
+    // In CI nobody can open a sign-in link: no key is a missing secret (a fork's pull request gets none).
+    if (process.env.CI)
+        throw new Error("No NOMAC_API_KEY: in CI, set it from a secret (a pull request from a fork gets no secrets).");
     const state = await cloudState();
     const base = (process.env.NOMAC_API_URL || DEFAULT_API).replace(/\/+$/, "");
     let pending = state.signIn && state.signIn.expires_at > Date.now() && state.signIn.base === base ? state.signIn : undefined;

package/dist/cloud/api.js

@@ -6,10 +6,13 @@ import { VERSION } from "../registry.js";
 export class CloudError extends Error {
     code;
     status;
-    constructor(code, message, status) {
+    /** The rest of NoMac's error: starting (a phone not ready yet), retry_after_seconds, next_action ... */
+    details;
+    constructor(code, message, status, details = {}) {
         super(message);
         this.code = code;
         this.status = status;
+        this.details = details;
     }
 }
 const PASSING = new Set([429, 502, 503, 504]);
@@ -55,7 +58,8 @@ export class CloudApi {
                 await pause(attempt, Number(response.headers.get("retry-after")));
                 continue;
             }
-            throw new CloudError(error.code ?? `http_${response.status}`, error.message ?? `NoMac answered ${response.status}.`, response.status);
+            const { code: _code, message: _message, ...details } = error;
+            throw new CloudError(error.code ?? `http_${response.status}`, error.message ?? `NoMac answered ${response.status}.`, response.status, details);
         }
     }
 }

package/dist/cloud/forward.js

@@ -12,6 +12,10 @@ import { account, cloudState, setCloudState } from "./account.js";
 import { CloudError } from "./api.js";
 import { randomKey, syncFolder } from "./sync.js";
 const JOB = /Next: job_status \{job_id: "([^"]+)"\}/;
+// job_status's answer while the job still runs (tools/jobs.ts): the same job is read again.
+const STILL_RUNNING = /\nNext: job_status again, or job_cancel\./;
+/** How long a call waits for a phone that is still starting (queued for a free one, or being prepared). */
+const STARTING_WAIT_MS = { cli: 5 * 60_000, mcp: 30_000 };
 /** The phone cloud mode uses, or why there is none. */
 export async function cloudPhone() {
     const state = await cloudState();
@@ -48,8 +52,20 @@ async function projectRoot(path) {
     }
     return { folder, inside: relative(folder, realpathSync(local)).split(sep).join("/") };
 }
+/** Waits for a phone that is still starting, up to waitMs: its state as last read (ready, or not yet). */
+async function whenReady(api, session, waitMs) {
+    const deadline = Date.now() + waitMs;
+    for (;;) {
+        const phone = (await api.json("GET", `/api/v1/simulator/sessions/${session}`)).session;
+        if (phone.state !== "queued" && phone.state !== "provisioning")
+            return phone;
+        if (Date.now() >= deadline)
+            return phone;
+        await new Promise((r) => setTimeout(r, 2_000));
+    }
+}
 /** Runs one tool on the phone and answers as the local tool would. */
-export async function forward(name, args, context) {
+export async function forward(name, args, context, waited = false) {
     const { api, session } = await cloudPhone();
     let sent = args;
     let note = "";
@@ -70,16 +86,26 @@ export async function forward(name, args, context) {
             sent = { ...args, app: `${synced.root}/${basename(app)}` };
         }
         let result = await call(api, session, name, sent);
-        // The command line waits for a long build, as it does on this Mac: job_status until the job ends.
-        for (let rounds = 0; context.waitForJobs && rounds < 60; rounds++) {
-            const job = JOB.exec(result.text)?.[1];
-            if (!job)
-                break;
+        // The command line waits for a long build, as it does on this Mac: job_status until the job ends. A job named
+        // in an error (another build of the project running) is not this call's.
+        const job = context.waitForJobs && !result.isError ? JOB.exec(result.text)?.[1] : undefined;
+        for (let rounds = 0; job && rounds < 120; rounds++) {
             result = await call(api, session, "job_status", { job_id: job });
+            if (result.isError || !STILL_RUNNING.test(result.text))
+                break;
         }
         return note ? { ...result, text: `${note}${result.text}` } : result;
     }
     catch (error) {
+        // A phone still starting (waiting for a free one, or being prepared) is waited for, then the call goes again.
+        if (error instanceof CloudError && error.code === "session_not_ready" && error.details.starting === true) {
+            const phone = waited ? undefined : await whenReady(api, session, context.waitForJobs ? STARTING_WAIT_MS.cli : STARTING_WAIT_MS.mcp).catch(() => undefined);
+            if (phone?.state === "ready")
+                return forward(name, args, context, true);
+            if (phone && !["queued", "provisioning"].includes(phone.state))
+                return ended(session);
+            return { text: `The cloud phone ${session} is still starting${phone?.message ? `: ${phone.message}` : "."}\nNext: ${name} again in a moment (nothing ran); cloud status shows its place.`, isError: true };
+        }
         if (error instanceof CloudError && (error.code === "session_not_found" || error.code === "session_not_ready" || error.code === "session_expired"))
             return ended(session);
         if (error instanceof CloudError)
@@ -87,6 +113,19 @@ export async function forward(name, args, context) {
         throw error;
     }
 }
+/**
+ * NoMac's stop notice, in the phone's last 15 minutes before a limit or the account's credit's end, as cloud mode can
+ * act on it: it keeps no saved states, so a new phone after this one is the way on (its build stays in the workspace).
+ */
+export function stopNotice(answer) {
+    const notice = answer.stop_notice;
+    if (!notice?.message)
+        return "";
+    if (notice.reason === "credit")
+        return `\n${notice.message}`;
+    const when = (notice.minutes_left ?? 0) <= 1 ? "within a minute" : `in about ${notice.minutes_left} minutes`;
+    return `\nThe cloud phone stops ${when}, at its ${notice.reason === "spend_cap" ? "spend cap" : "time limit"}: cloud extend keeps it running an hour longer. Once it stops, its apps and data go with it; the project's build stays, so the next cloud start builds incrementally.`;
+}
 async function call(api, session, name, args) {
     let answer = await api.json("POST", `/api/v1/simulator/sessions/${session}/tools/${name}`, args, `cli-${name}-${randomKey()}`);
     while (answer.state === "running" && typeof answer.id === "string") {
@@ -102,7 +141,7 @@ async function call(api, session, name, args) {
     const picture = (result.content ?? []).find((c) => c.type === "image" && c.data);
     const notices = answer.notices?.includes("phone_restarted") ? "\nThe phone restarted; its apps and data are kept." : "";
     return {
-        text: `${text}${notices}`,
+        text: `${text}${notices}${stopNotice(answer)}`,
         ...(picture ? { image: { data: Buffer.from(picture.data, "base64"), mimeType: picture.mimeType === "image/png" ? "image/png" : "image/jpeg" } } : {}),
         ...(result.isError ? { isError: true } : {}),
     };

package/dist/cloud/sync.js

@@ -3,51 +3,190 @@
  * workspace lacks. The manifest (each file's path, size, mtime and SHA-256) goes in pages; the host answers the
  * paths it lacks; those go as one gzip tar, in 256 KiB chunks up to 8 MiB, else to an upload link; place writes
  * them and every mtime (new mtimes alone make Xcode compile again), and removes what the folder no longer has.
- * Build output and dependencies are left out, and left alone on the host.
+ * Build output and dependencies are left out, and left alone on the host. A file's hash is kept in an index on
+ * this machine, so only files that changed since the last sync are read again.
  */
 import { spawn } from "node:child_process";
 import { createHash, randomBytes } from "node:crypto";
-import { lstatSync, readdirSync, readFileSync, readlinkSync, realpathSync } from "node:fs";
-import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
+import { createReadStream, lstatSync, readdirSync, realpathSync } from "node:fs";
+import { mkdir, mkdtemp, readFile, readlink, rename, rm, writeFile } from "node:fs/promises";
 import { tmpdir } from "node:os";
 import { join, relative, sep } from "node:path";
 import { run } from "../engine/exec.js";
+import { stateDir } from "../engine/state.js";
+import { setting } from "../settings.js";
 /** Folders a build makes or a package manager fills: never sent, never removed on the host. */
 export const SKIP_DIRS = new Set([".git", ".hg", ".svn", "DerivedData", "build", ".build", "node_modules", "Pods", "xcuserdata", ".swiftpm", ".expo", ".gradle"]);
-const SKIP_FILES = (name) => name === ".DS_Store" || name.endsWith(".xcuserstate");
+const SKIP_FILES = (name) => name === ".DS_Store" || name.endsWith(".xcuserstate") || secretFile(name);
+/**
+ * Key and credential files, never sent even when git tracks them: environment files (but their examples), signing
+ * keys and certificates, provisioning profiles, keystores, SSH keys and .netrc. A Simulator build needs none of them.
+ */
+export const secretFile = (name) => (/^\.env(\..+)?$/.test(name) && !/\.(example|sample|template)$/.test(name)) ||
+    /\.(p8|p12|pfx|pem|key|mobileprovision|provisionprofile|keystore|jks)$/i.test(name) ||
+    /^id_(rsa|dsa|ecdsa|ed25519)(\.pub)?$/.test(name) ||
+    name === ".netrc";
 // The protocol's limits (NoMac's mac-protocol, simManifestEntrySchema and the sim route's body).
 const PAGE_ENTRIES = 1000;
 const PAGE_BYTES = 192 * 1024;
 const CHUNK = 256 * 1024;
 const INLINE_MAX = 8 * 1024 * 1024;
 const ARCHIVE_MAX = 2 * 1024 ** 3;
-/** The project's files as the host is to have them: build output, dependencies and editor state left out. */
-export function listProject(folder) {
-    const out = [];
-    const walk = (dir) => {
+/**
+ * The project's files as the host is to have them: in a git repository, the files git tracks or would track
+ * (.gitignore's rules kept), else the folder's; build output, dependencies, editor state and key or credential files
+ * left out either way. With only (a .app beside others in its folder), that one alone and all of it: a bundle's own
+ * folders (an app's node_modules or build) are part of the app. Files are hashed a few at a time, never blocking the process (a
+ * cloud tunnel runs in it), and a file whose size, mtime, ctime and inode match the index is not read again.
+ */
+export async function listProject(folder, only) {
+    const found = [];
+    const walk = (dir, whole) => {
         for (const name of readdirSync(dir).sort()) {
             if (SKIP_FILES(name))
                 continue;
             const path = join(dir, name);
             const st = lstatSync(path);
             if (st.isDirectory()) {
-                if (!SKIP_DIRS.has(name))
-                    walk(path);
+                if (whole || !SKIP_DIRS.has(name))
+                    walk(path, whole);
                 continue;
             }
-            const rel = relative(folder, path).split(sep).join("/");
-            if (st.isSymbolicLink()) {
-                const target = readlinkSync(path);
-                out.push({ path: rel, size: Buffer.byteLength(target), mtime_ms: Math.floor(st.mtimeMs), sha256: sha256(target), kind: "symlink" });
-            }
-            else if (st.isFile()) {
-                out.push({ path: rel, size: st.size, mtime_ms: Math.floor(st.mtimeMs), sha256: sha256(readFileSync(path)), kind: st.mode & 0o111 ? "executable" : "file" });
-            }
+            if (st.isSymbolicLink() || st.isFile())
+                found.push({ rel: relative(folder, path).split(sep).join("/"), path, st });
         }
     };
-    walk(folder);
+    if (only) {
+        const top = join(folder, only);
+        if (lstatSync(top, { throwIfNoEntry: false })?.isDirectory())
+            walk(top, true);
+    }
+    else {
+        const tracked = await filesInGit(folder);
+        if (!tracked)
+            walk(folder, false);
+        else {
+            for (const rel of tracked.sort()) {
+                const parts = rel.split("/");
+                if (SKIP_FILES(parts.at(-1)) || parts.slice(0, -1).some((part) => SKIP_DIRS.has(part)))
+                    continue;
+                const path = join(folder, ...parts);
+                const st = lstatSync(path, { throwIfNoEntry: false });
+                if (st && (st.isSymbolicLink() || st.isFile()))
+                    found.push({ rel, path, st });
+            }
+        }
+    }
+    // Windows keeps no executable bit: git's index has it for the files it tracks (a build phase's script).
+    const gitExecutable = process.platform === "win32" ? await executablesInGit(folder) : undefined;
+    const indexed = setting("SYNC_INDEX") !== "0";
+    const file = indexFile(folder, only);
+    const last = indexed ? await readIndex(file) : undefined;
+    const scanned = Date.now();
+    const next = {};
+    const out = new Array(found.length);
+    await eachLimit(found, HASHING, async ({ rel, path, st }, i) => {
+        if (st.isSymbolicLink()) {
+            const target = await readlink(path);
+            out[i] = { path: rel, size: Buffer.byteLength(target), mtime_ms: Math.floor(st.mtimeMs), sha256: sha256(target), kind: "symlink" };
+            return;
+        }
+        const known = last?.files[rel];
+        // Git's rule for a racy entry: a file changed within RACY_MS of the last scan may have changed again since,
+        // with the same mtime, so it is read again.
+        const same = known && known.size === st.size && known.mtime_ms === st.mtimeMs && known.ctime_ms === st.ctimeMs && known.ino === st.ino && st.mtimeMs < last.scanned_ms - RACY_MS;
+        const hash = same ? known.sha256 : await hashFile(path, st.size);
+        next[rel] = { size: st.size, mtime_ms: st.mtimeMs, ctime_ms: st.ctimeMs, ino: st.ino, sha256: hash };
+        const executable = gitExecutable ? gitExecutable.has(rel) : Boolean(st.mode & 0o111);
+        out[i] = { path: rel, size: st.size, mtime_ms: Math.floor(st.mtimeMs), sha256: hash, kind: executable ? "executable" : "file" };
+    });
+    if (indexed)
+        await writeIndex(file, { scanned_ms: scanned, files: next }).catch(() => undefined);
     return out;
 }
+/** The files git tracks as executable (mode 100755) under folder, by their path from it; none outside a repository. */
+/**
+ * The files git tracks in folder, or would (untracked but not ignored), from the folder, a submodule's too; undefined
+ * when folder is not a repository's top (or git is not there).
+ */
+export async function filesInGit(folder) {
+    try {
+        const top = (await run("git", ["-C", folder, "rev-parse", "--show-toplevel"], { timeoutMs: 5_000 })).stdout.trim();
+        if (realpathSync(top) !== realpathSync(folder))
+            return undefined;
+    }
+    catch {
+        return undefined;
+    }
+    const list = async (dir, prefix, depth) => {
+        const listed = (await run("git", ["-C", dir, "ls-files", "-co", "--exclude-standard", "-z"], { timeoutMs: 60_000 })).stdout.split("\0").filter(Boolean);
+        const out = [];
+        for (const rel of listed.map((path) => path.replace(/\/+$/, ""))) {
+            // A submodule (or a repository inside) is listed as its folder: its own files, by its own rules.
+            if (depth < 4 && lstatSync(join(dir, rel), { throwIfNoEntry: false })?.isDirectory())
+                out.push(...(await list(join(dir, rel), `${prefix}${rel}/`, depth + 1).catch(() => [])));
+            else
+                out.push(`${prefix}${rel}`);
+        }
+        return out;
+    };
+    try {
+        return await list(folder, "", 0);
+    }
+    catch {
+        return undefined;
+    }
+}
+export async function executablesInGit(folder) {
+    try {
+        const listed = (await run("git", ["-C", folder, "ls-files", "-s", "-z"], { timeoutMs: 30_000 })).stdout;
+        // Each entry: <mode> <object> <stage>\t<path>, NUL-separated; the paths are from the folder.
+        return new Set(listed.split("\0").filter((e) => e.startsWith("100755 ")).map((e) => e.slice(e.indexOf("\t") + 1)));
+    }
+    catch {
+        return new Set();
+    }
+}
+/** Files hashed at once. */
+const HASHING = 8;
+/** A file changed within this long of a scan is read again at the next one (coarse mtimes on some file systems). */
+const RACY_MS = 2_000;
+const indexFile = (folder, only) => join(stateDir(), "sync-index", `${sha256(`${realpathSync(folder)}\0${only ?? ""}`).slice(0, 32)}.json`);
+async function readIndex(file) {
+    try {
+        const index = JSON.parse(await readFile(file, "utf8"));
+        return typeof index.scanned_ms === "number" && index.files && typeof index.files === "object" ? index : undefined;
+    }
+    catch {
+        return undefined;
+    }
+}
+async function writeIndex(file, index) {
+    await mkdir(join(stateDir(), "sync-index"), { recursive: true });
+    const tmp = `${file}.${process.pid}.tmp`;
+    await writeFile(tmp, JSON.stringify(index), { mode: 0o600 });
+    await rename(tmp, file);
+}
+/** A file's SHA-256: small files read at once, larger ones streamed. */
+async function hashFile(path, size) {
+    if (size < 1024 * 1024)
+        return sha256(await readFile(path));
+    const hash = createHash("sha256");
+    for await (const chunk of createReadStream(path))
+        hash.update(chunk);
+    return hash.digest("hex");
+}
+/** Runs work on each item, at most limit at a time. */
+async function eachLimit(items, limit, work) {
+    let next = 0;
+    const worker = async () => {
+        while (next < items.length) {
+            const i = next++;
+            await work(items[i], i);
+        }
+    };
+    await Promise.all(Array.from({ length: Math.min(limit, items.length) }, worker));
+}
 const sha256 = (data) => createHash("sha256").update(data).digest("hex");
 /** Pages of at most 1000 entries and about 192 KiB each, in the manifest's order. */
 export function pages(entries) {
@@ -110,7 +249,7 @@ async function pack(folder, paths, file) {
 export async function syncFolder(api, session, folder, options = {}) {
     const { only, onStep } = options;
     const started = Date.now();
-    const entries = listProject(folder).filter((e) => !only || e.path === only || e.path.startsWith(`${only}/`));
+    const entries = await listProject(folder, only);
     if (!entries.length)
         throw new Error(`${folder} has no files to send.`);
     const project = options.project ?? (await projectId(folder));
@@ -119,11 +258,15 @@ export async function syncFolder(api, session, folder, options = {}) {
     if (book.length > 256)
         throw new Error(`${folder} has ${entries.length} files: more than NoMac's sync takes (256 pages).`);
     onStep?.(`listing ${entries.length} files`);
-    const missing = [];
-    for (let page = 0; page < book.length; page++) {
-        const answer = await api.json("POST", `/api/v1/simulator/sessions/${session}/sync`, { op: "manifest", sync, project, page, pages: book.length, entries: book[page] });
-        missing.push(...(answer.missing ?? []));
-    }
+    // Pages go 4 at a time: the host keeps each by its number, and places once it has them all.
+    const answered = new Array(book.length);
+    await eachLimit(book, 4, async (entriesOfPage, page) => {
+        const answer = await api.json("POST", `/api/v1/simulator/sessions/${session}/sync`, { op: "manifest", sync, project, page, pages: book.length, entries: entriesOfPage });
+        // Only paths this page listed are packed: an answer naming any other file of the folder sends nothing more.
+        const listed = new Set(entriesOfPage.map((e) => e.path));
+        answered[page] = (answer.missing ?? []).filter((path) => typeof path === "string" && listed.has(path));
+    });
+    const missing = answered.flat();
     let archive;
     let bytes = 0;
     if (missing.length) {

package/dist/cloud/tunnel.js

@@ -13,26 +13,32 @@ const RESPONSE = /^HTTP\/1\.[01] (\d{3})/;
 // The WebSocket may hold this much unsent before a local connection waits for it.
 const SEND_BUFFER = 4 * 1024 * 1024;
 export class Tunnel {
+    /** How long the phone's relay has to answer once the connection is up; tests shorten it. */
+    static helloWaitMs = 30_000;
     session;
     localPort;
     localHost;
     phonePort;
     state = "connecting";
     error;
+    /** Said once with the next answer: the phone's port changed when the tunnel came back. */
+    notice;
     requests = [];
     connections = 0;
     api;
     onRequest;
+    onNotice;
     ws;
     conns = new Map();
     stopped = false;
     drops = 0;
-    constructor(api, session, localPort, localHost = "127.0.0.1", onRequest) {
+    constructor(api, session, localPort, localHost = "127.0.0.1", onRequest, onNotice) {
         this.api = api;
         this.session = session;
         this.localPort = localPort;
         this.localHost = localHost;
         this.onRequest = onRequest;
+        this.onNotice = onNotice;
     }
     /** Opens the tunnel; answers the port the phone's apps reach once the phone's relay listens. */
     async start() {
@@ -58,6 +64,15 @@ export class Tunnel {
             const reader = new FrameReader();
             let opened = 0;
             let answered = false;
+            // A relay that never says which port it took would leave cloud tunnel waiting for ever.
+            const silent = setTimeout(() => {
+                if (answered)
+                    return;
+                answered = true;
+                this.stop();
+                this.error = "the phone's relay did not answer";
+                reject(new ToolError(`The phone's tunnel did not open within ${Math.max(1, Math.round(Tunnel.helloWaitMs / 1000))} s.`, "The phone must be ready; call cloud status, then cloud tunnel again."));
+            }, Tunnel.helloWaitMs);
             ws.onmessage = (event) => {
                 let frames;
                 try {
@@ -69,9 +84,16 @@ export class Tunnel {
                 }
                 for (const f of frames) {
                     if (f.type === HELLO) {
-                        this.phonePort = Number(JSON.parse(f.payload.toString("utf8")).port);
+                        const port = Number(JSON.parse(f.payload.toString("utf8")).port);
+                        // The host keeps the port when it can; when it could not, an app pointed at the old one fails.
+                        if (this.phonePort !== undefined && port !== this.phonePort) {
+                            this.notice = `The tunnel came back on another port of the phone, ${port} (it was ${this.phonePort}): app_run again with env "API_URL=http://127.0.0.1:${port}" for an app that used the old one.`;
+                            this.onNotice?.(this.notice);
+                        }
+                        this.phonePort = port;
                         this.state = "open";
                         opened = Date.now();
+                        clearTimeout(silent);
                         if (!answered) {
                             answered = true;
                             resolve(this.phonePort);
@@ -86,6 +108,7 @@ export class Tunnel {
                 }
             };
             ws.onclose = () => {
+                clearTimeout(silent);
                 for (const conn of this.conns.values())
                     conn.socket.destroy();
                 this.conns.clear();

package/dist/cloud-hint.js

@@ -12,3 +12,7 @@ export function cloudHint() {
 }
 /** A next step, with the hint after it when it is on. */
 export const withCloudHint = (next) => [next, cloudHint()].filter(Boolean).join(" ");
+/** The next step on a machine that can't run the iOS Simulator: a cloud phone first, when the hint is on. */
+export function cloudFirst(otherwise) {
+    return setting("NO_CLOUD_HINT") ? otherwise : `cloud start runs every tool on a NoMac cloud iPhone Simulator, $0.80 an hour by the minute (${NOMAC_URL}); then call this again. Or: ${otherwise.replace(/^./, (c) => c.toLowerCase())}`;
+}