@expo/serve-sim 0.2.3 → 0.3.0

package/README.md

@@ -38,6 +38,26 @@ Requires an Apple silicon (`arm64`) Mac with Xcode command line tools (`xcrun si
 
 The bundled native addon, simulator tools, and host helpers are arm64-only.
 
+The iPhone Duo simulator supports touch input, scrolling, and touch gestures on both the folded cover screen and the half-open or fully open inner screen. Frames and input follow the active display and its orientation; an app can still restrict its supported orientations.
+
+Three shortcuts below the phone select Fully folded, Partially open, and Fully open.
+At the top of Simulator settings in the Tools sidebar, the Fold pose dropdown
+adds Laptop and Tent. The next rows provide a live 0–180° hinge slider with
+decimal angle input and Table Mode, using the same controls as other settings.
+Partially open uses Book: it and Laptop use a 90° hinge angle with different
+physical orientations; Tent uses 80°.
+Duo defaults to 3D using Apple's model from the host's Xcode installation,
+with live displays and smooth folding, unfolding, and pose transitions.
+Display pixels stay attached to their
+physical panels as the model moves. Closed and Open face the viewer directly,
+with the open model following the screen's portrait or landscape layout. Laptop
+rests level and Tent presents the outside cover screen. The original `V68.usdz`
+is loaded locally, without bundling Apple's model or downloading it. The view
+respects the browser's reduced-motion preference. **Simulator → Preview mode**
+switches between 2D and 3D; AX inspection also uses the flat view. See [hinge controls and display
+selection](packages/serve-sim/docs/hinge-controls.md) for Device Hub's hidden
+controls and the simulator APIs behind them.
+
 ## CLI
 
 ```
@@ -50,6 +70,8 @@ serve-sim type <text> [-d udid]       Type text via the simulator keyboard
 serve-sim rotate <orientation> [-d udid]
                                       portrait | portrait_upside_down |
                                       landscape_left | landscape_right
+serve-sim hinge <fold|half|unfold|degrees> [-d udid]
+                                      Set the hinge angle (0° folded, 180° unfolded)
 serve-sim ca-debug <option> <on|off> [-d udid]
                                       Toggle a CoreAnimation debug flag
                                       (blended|copies|misaligned|offscreen|slow-animations)
@@ -95,6 +117,13 @@ Options:
                       H.264/WebRTC target bitrate
       --video-fps <fps>
                       H.264/WebRTC frame rate (1-140)
+      --launch-app-identifier <id>
+                      Bundle identifier of an installed app to launch once the
+                      simulator boots
+      --launch-arg <arg>
+                      Argument passed to the app when it launches (repeatable)
+      --open-url <url>
+                      URL to open in the app after it launches
       --list [device] List running streams
       --kill [device] Kill running stream(s)
 
@@ -171,6 +200,26 @@ Supervisors can probe `GET /healthz` to confirm that the preview server is
 listening and `GET /readyz` to wait until the selected simulator and native
 capture session are ready. Both endpoints return JSON and disable caching.
 
+### Launching an app
+
+`--launch-app-identifier <bundle-id>` launches an app that is already installed on the
+simulator, before the stream starts. Install it yourself first (`xcrun simctl install`);
+`serve-sim` only launches.
+
+```sh
+serve-sim --launch-app-identifier host.exp.Exponent \
+  --launch-arg -EXDevMenuIsOnboardingFinished --launch-arg 1 \
+  --open-url exp://127.0.0.1:8081
+```
+
+`--launch-arg` is repeatable and maps to `simctl launch` process arguments. `--open-url`
+runs `simctl openurl` after the app is up, and pre-approves custom URL schemes so the
+Simulator does not ask for confirmation. This matters for `exp://` deep links on a headless host.
+
+Launching from `serve-sim` rather than beforehand matters because a second `simctl launch`
+on a running app is a no-op: it neither restarts the app nor applies new arguments. Owning
+the launch is what lets `serve-sim` attach to the process from the start.
+
 ### Camera
 
 `serve-sim camera <bundle-id>` replaces the simulator's camera feed for a single app. A small host-side helper writes BGRA frames into a POSIX shared-memory region; an injected dylib (`DYLD_INSERT_LIBRARIES`) swizzles AVFoundation inside the simulator process so the app reads from that region instead of the simulator's stub camera.
@@ -259,7 +308,7 @@ server.onUpgrade((request, socket) => {
 });
 ```
 
-The middleware reads the helper's state from `$TMPDIR/serve-sim/` and points the browser at the helper's stream, interaction WebSocket, and WebKit DevTools endpoints. By default those URLs target the helper's own port directly (CORS is wide-open on the helper), so a plain `app.use(...)` mount works without touching your server's WebSocket handling.
+The middleware reads the helper's state from `$TMPDIR/serve-sim/` and points the browser at the helper's stream, interaction WebSocket, and WebKit DevTools endpoints. By default those URLs target the helper's own port directly (the helper answers loopback origins), so a plain `app.use(...)` mount works without touching your server's WebSocket handling.
 
 ### Single-port / remote proxying
 
@@ -296,6 +345,7 @@ The npm package ships the native capture addon and LiveKit WebRTC framework alon
 ```sh
 bun install
 bun run packages/serve-sim/build.ts                   # full production build
+bash packages/serve-sim/Sources/build-test-fixtures.sh # simulator test fixtures
 packages/serve-sim/Sources/SimNative/build.sh         # native addon only
 bun run --filter @expo/serve-sim dev                  # watch mode
 bun run --filter @expo/serve-sim tart-dev             # guest preview at localhost:3200

package/Sources/ServeSimCapabilityLoader/build.sh

@@ -0,0 +1,32 @@
+#!/bin/bash
+set -euo pipefail
+
+HERE="$(cd "$(dirname "$0")" && pwd)"
+OUT_DIR="${1:-$HERE/../../dist/capability-loader}"
+mkdir -p "$OUT_DIR"
+
+SDK="$(xcrun --sdk iphonesimulator --show-sdk-path)"
+DYLIB="$OUT_DIR/libServeSimCapabilityLoader.dylib"
+
+xcrun --sdk iphonesimulator clang \
+    -arch arm64 \
+    -mios-simulator-version-min=15.0 \
+    -isysroot "$SDK" \
+    -dynamiclib \
+    -O2 \
+    -Wall -Wextra -Werror -Wconversion -Wshadow \
+    -install_name "@rpath/libServeSimCapabilityLoader.dylib" \
+    -o "$DYLIB" \
+    "$HERE/serve-sim-capability-loader.c"
+
+# This image is inserted into every process in the simulator. Anything beyond
+# libSystem crash-loops system daemons, so fail the build rather than ship it.
+LINKED="$(otool -L "$DYLIB" | grep $'^\t' | awk '{print $1}' | sort -u)"
+UNEXPECTED="$(echo "$LINKED" | grep -v -e '^@rpath/libServeSimCapabilityLoader\.dylib$' -e '^/usr/lib/libSystem\.B\.dylib$' || true)"
+if [ -n "$UNEXPECTED" ]; then
+  echo "CapabilityLoader links more than libSystem:" >&2
+  echo "$UNEXPECTED" >&2
+  exit 1
+fi
+
+echo "$DYLIB"

package/Sources/ServeSimCapabilityLoader/serve-sim-capability-loader.c

@@ -0,0 +1,322 @@
+// Inserted into every simulator process via launchd DYLD_INSERT_LIBRARIES, so
+// it links libSystem only: a Foundation-linked insert crash-loops GSSCred, and
+// loading UIKit from the constructor crashes Safari on headless boots. Capability
+// dylibs load on the main queue after the constructor returns.
+//
+// Config format, one capability per line, written by the launch manager:
+//   <user|all>\t<dylib>\t[KEY=VALUE;KEY=VALUE]\t<delay-ms>
+//
+// The launch manager sets SERVE_SIM_CAPABILITIES_CONFIG alongside the insert,
+// per simulator, so the config can live with the rest of serve-sim's state
+// rather than inside the installed package.
+
+#include <dispatch/dispatch.h>
+#include <dlfcn.h>
+#include <errno.h>
+#include <fcntl.h>
+#include <mach-o/dyld.h>
+#include <sys/stat.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+
+#define CONFIG_VAR "SERVE_SIM_CAPABILITIES_CONFIG"
+#define MAX_CONFIG_BYTES (64 * 1024)
+// A capability that links UIKit asks to be loaded late; one that links only
+// libSystem does not have to wait for it.
+#define MAX_LOAD_DELAY_MS 10000
+#define MAX_CAPABILITIES 64
+
+// 0 read the whole file, 1 the file did not fit, -1 could not read it.
+static int read_config(const char *path, char *out, size_t cap) {
+  int fd;
+  do {
+    fd = open(path, O_RDONLY | O_NONBLOCK | O_NOFOLLOW);
+  } while (fd < 0 && errno == EINTR);
+  if (fd < 0) return -1;
+
+  // Only a real file. A fifo or device would give this thread a read that never ends.
+  struct stat info;
+  if (fstat(fd, &info) != 0 || !S_ISREG(info.st_mode)) {
+    close(fd);
+    return -1;
+  }
+
+  size_t used = 0;
+  int truncated = 0;
+  for (;;) {
+    ssize_t n = read(fd, out + used, cap - 1 - used);
+    if (n < 0) {
+      if (errno == EINTR) continue;
+      close(fd);
+      return -1;
+    }
+    if (n == 0) break;
+    used += (size_t)n;
+    if (used >= cap - 1) { truncated = 1; break; }
+  }
+  close(fd);
+  out[used] = '\0';
+  return truncated;
+}
+
+static int apply_env(char *pairs) {
+  int failed = 0;
+  char *pair, *rest = pairs;
+  while ((pair = strsep(&rest, ";")) != NULL) {
+    if (*pair == '\0') continue;
+    char *eq = strchr(pair, '=');
+    if (eq == NULL) continue;
+    *eq = '\0';
+    if (setenv(pair, eq + 1, 1) != 0) {
+      fprintf(stderr, "[serve-sim] could not set %s for a capability\n", pair);
+      failed = 1;
+    }
+  }
+  return failed;
+}
+
+// An app the user installed lives under the device's own Bundle container; an
+// Apple app ships inside the runtime, under RuntimeRoot.
+#define USER_APP_MARKER "/Containers/Bundle/Application/"
+
+static unsigned parse_delay_ms(const char *text) {
+  if (text == NULL || *text == '\0') return 0;
+  char *end = NULL;
+  errno = 0;
+  long value = strtol(text, &end, 10);
+  if (errno != 0 || end == text || *end != '\0' || value < 0) return 0;
+  if (value > MAX_LOAD_DELAY_MS) return MAX_LOAD_DELAY_MS;
+  return (unsigned)value;
+}
+
+static int capability_applies(char *line, const char *exec_path, char **dylib_out,
+                             char **env_out, unsigned *delay_ms_out) {
+  if (*line == '\0' || *line == '#') return 0;
+
+  char *fields = line;
+  char *scope = strsep(&fields, "\t");
+  char *dylib = strsep(&fields, "\t");
+  char *env = strsep(&fields, "\t");
+  char *delay = strsep(&fields, "\t");
+  if (dylib == NULL || *dylib == '\0') return 0;
+  if (*dylib != '/') {
+    fprintf(stderr, "[serve-sim] ignoring capability path that is not absolute: %s\n", dylib);
+    return 0;
+  }
+
+  // `all` needs no check of its own: the constructor already refused anything
+  // that is not an app. An unknown scope loads nowhere.
+  if (strcmp(scope, "user") == 0) {
+    if (strstr(exec_path, USER_APP_MARKER) == NULL) return 0;
+  } else if (strcmp(scope, "all") != 0) {
+    fprintf(stderr, "[serve-sim] ignoring capability with unknown scope '%s': %s\n", scope, dylib);
+    return 0;
+  }
+
+  *dylib_out = dylib;
+  *env_out = env;
+  *delay_ms_out = parse_delay_ms(delay);
+  return 1;
+}
+
+struct CapabilityLoad {
+  char *dylib;
+  char *env;
+  unsigned delay_ms;
+  uint64_t generation;
+  int loaded;
+  int pending;
+};
+
+struct Load {
+  char exec_path[1024];
+  char config_path[1024];
+  uint64_t next_generation;
+  struct CapabilityLoad capabilities[MAX_CAPABILITIES];
+};
+
+static void load_capabilities(struct Load *load);
+
+static void load_one(struct CapabilityLoad *capability) {
+  capability->pending = 0;
+  char *env = strdup(capability->env);
+  if (env == NULL) return;
+  int failed = apply_env(env);
+  free(env);
+  if (failed) {
+    fprintf(stderr, "[serve-sim] not loading %s: its environment is incomplete\n",
+            capability->dylib);
+    return;
+  }
+  if (dlopen(capability->dylib, RTLD_NOW | RTLD_LOCAL) == NULL) {
+    fprintf(stderr, "[serve-sim] could not load %s: %s\n", capability->dylib, dlerror());
+    return;
+  }
+  capability->loaded = 1;
+}
+
+static void load_capabilities(struct Load *load) {
+  char *config = malloc(MAX_CONFIG_BYTES);
+  if (config == NULL) return;
+  int status = read_config(load->config_path, config, MAX_CONFIG_BYTES);
+  if (status != 0) {
+    if (status > 0) {
+      fprintf(stderr, "[serve-sim] %s exceeds %d bytes; no capabilities loaded.\n",
+              load->config_path, MAX_CONFIG_BYTES);
+    }
+    config[0] = '\0';
+  }
+
+  struct CapabilityLoad desired[MAX_CAPABILITIES];
+  size_t count = 0;
+  char *line, *lines = config;
+  while ((line = strsep(&lines, "\n")) != NULL) {
+    char *dylib, *env;
+    unsigned delay_ms;
+    if (!capability_applies(line, load->exec_path, &dylib, &env, &delay_ms)) continue;
+    if (count == MAX_CAPABILITIES) {
+      fprintf(stderr, "[serve-sim] more than %d capabilities apply; ignoring the rest.\n",
+              MAX_CAPABILITIES);
+      break;
+    }
+    desired[count++] = (struct CapabilityLoad){
+      .dylib = dylib, .env = env ? env : "", .delay_ms = delay_ms,
+    };
+  }
+  for (size_t i = 0; i < MAX_CAPABILITIES; i++) {
+    struct CapabilityLoad *capability = &load->capabilities[i];
+    if (capability->dylib == NULL || capability->loaded) continue;
+    size_t j = 0;
+    while (j < count && strcmp(capability->dylib, desired[j].dylib) != 0) j++;
+    if (j == count) {
+      free(capability->dylib);
+      free(capability->env);
+      *capability = (struct CapabilityLoad){0};
+    }
+  }
+  for (size_t entry = 0; entry < count; entry++) {
+    char *dylib = desired[entry].dylib;
+    char *env = desired[entry].env;
+    unsigned delay_ms = desired[entry].delay_ms;
+    struct CapabilityLoad *capability = NULL;
+    for (size_t i = 0; i < MAX_CAPABILITIES; i++) {
+      if (load->capabilities[i].dylib && strcmp(load->capabilities[i].dylib, dylib) == 0) {
+        capability = &load->capabilities[i];
+        break;
+      }
+    }
+    if (capability && (capability->loaded ||
+        (capability->pending && capability->delay_ms == delay_ms && strcmp(capability->env, env) == 0))) {
+      continue;
+    }
+    if (capability == NULL) {
+      for (size_t i = 0; i < MAX_CAPABILITIES; i++) {
+        if (load->capabilities[i].dylib == NULL) {
+          capability = &load->capabilities[i];
+          break;
+        }
+      }
+    }
+    if (capability == NULL) {
+      fprintf(stderr, "[serve-sim] capability load limit reached; ignoring %s\n", dylib);
+      continue;
+    }
+    char *dylib_copy = strdup(dylib);
+    char *env_copy = strdup(env);
+    if (dylib_copy == NULL || env_copy == NULL) {
+      free(dylib_copy);
+      free(env_copy);
+      continue;
+    }
+    free(capability->dylib);
+    free(capability->env);
+    *capability = (struct CapabilityLoad){
+      .dylib = dylib_copy, .env = env_copy, .delay_ms = delay_ms,
+      .generation = ++load->next_generation,
+    };
+    if (delay_ms == 0) {
+      load_one(capability);
+    } else {
+      capability->pending = 1;
+      uint64_t generation = capability->generation;
+      dispatch_after(dispatch_time(DISPATCH_TIME_NOW, (int64_t)delay_ms * (int64_t)NSEC_PER_MSEC),
+                     dispatch_get_main_queue(), ^{
+        load_capabilities(load);
+        if (capability->generation == generation && capability->dylib && !capability->loaded)
+          load_one(capability);
+      });
+    }
+  }
+  free(config);
+}
+
+static int config_dir(const char *path, char *out, size_t cap) {
+  int n = snprintf(out, cap, "%s", path);
+  if (n < 0 || (size_t)n >= cap) return -1;
+  char *slash = strrchr(out, '/');
+  if (slash == NULL || slash == out) return -1;
+  *slash = '\0';
+  return 0;
+}
+
+// Watch the directory because config updates replace the file by rename.
+static void watch_config(struct Load *load) {
+  char dir[sizeof load->config_path];
+  if (config_dir(load->config_path, dir, sizeof dir) != 0) return;
+
+  int fd = open(dir, O_EVTONLY);
+  if (fd < 0) return;
+
+  dispatch_source_t source = dispatch_source_create(
+      DISPATCH_SOURCE_TYPE_VNODE, (uintptr_t)fd, DISPATCH_VNODE_WRITE, dispatch_get_main_queue());
+  if (source == NULL) {
+    close(fd);
+    return;
+  }
+  dispatch_source_set_event_handler(source, ^{ load_capabilities(load); });
+  dispatch_source_set_cancel_handler(source, ^{ close(fd); });
+  dispatch_resume(source);
+}
+
+// getenv and access are safe in a constructor; the dlopen is not.
+__attribute__((constructor))
+static void serve_sim_capability_loader_init(void) {
+  const char *tmp = getenv("TMPDIR");
+  if (tmp == NULL || strstr(tmp, "/Containers/Data/Application/") == NULL) return;
+
+  // Absolute only. A relative path would resolve against the app's working
+  // directory, which is not ours to guess.
+  const char *config = getenv(CONFIG_VAR);
+  if (config == NULL || *config != '/') return;
+
+  struct Load *load = calloc(1, sizeof *load);
+  if (load == NULL) return;
+  char dir[sizeof load->config_path];
+
+  int n = snprintf(load->config_path, sizeof load->config_path, "%s", config);
+  if (n < 0 || (size_t)n >= sizeof load->config_path) {
+    free(load);
+    return;
+  }
+  // The file need not exist yet: a capability turned on later creates it, and
+  // the watch is on the directory. Requiring the file here would mean an app
+  // started before the first capability could never receive one.
+  if (config_dir(load->config_path, dir, sizeof dir) != 0 || access(dir, R_OK | X_OK) != 0) {
+    free(load);
+    return;
+  }
+  uint32_t exec_size = sizeof load->exec_path;
+  if (_NSGetExecutablePath(load->exec_path, &exec_size) != 0) {
+    free(load);
+    return;
+  }
+
+  // Concurrent dlopen during construction can invert the dyld/ObjC load locks.
+  // Defer capability loads to the main queue after construction completes.
+  dispatch_async(dispatch_get_main_queue(), ^{
+    watch_config(load);
+    load_capabilities(load);
+  });
+}