@supermachine/core 0.5.3 → 0.7.1

package/README.md

@@ -182,6 +182,78 @@ const image = await Image.build({
 
 Host file contents are folded into the snapshot's input-hash, so editing the host file invalidates the cache and rebakes automatically.
 
+### Pattern 5: Live host directory mount (virtio-fs DAX)
+
+For dev loops, code-mounted-from-host workflows, and "run my project" sandboxes, you want a *live* host directory inside the guest — edits on the host show up in the guest immediately, and the guest can read/write back. This is what `mounts` does: a virtio-fs share with DAX (zero-copy reads through the host page cache, kqueue-driven cache invalidation when host files change).
+
+```ts
+const image = await Image.build({
+  ref: "node:20-alpine",
+  mounts: [
+    {
+      hostPath: "/Users/me/my-app",   // host source tree
+      guestTag: "workspace",          // guest mounts via this tag
+      symlinks: "opaque",             // see below
+    },
+  ],
+});
+
+const vm = await (await image.pool()).acquire();
+// Inside the guest:
+//   mount -t virtiofs workspace /work
+//   cd /work && npm install
+```
+
+**`symlinks` policy** picks the trust posture for the mount. Workspace tools (npm workspaces, pnpm, yarn berry) symlink prolifically — `node_modules/<member> → ../packages/<member>` — so symlink creation has to work. But symlinks are also the classic escape vector if you're running untrusted code. The policy lets integrators pick:
+
+| Value | Guest can create symlinks? | External host symlinks visible/traversable? | When |
+|---|---|---|---|
+| `"deny"` | no (`EPERM`) | no (`EACCES` at LOOKUP) | Paranoid mounts: pure file content, no metadata surprises |
+| `"opaque"` *(default)* | yes — targets stored verbatim, host never resolves them | no | **Safe multi-tenant default**: npm/pnpm/yarn all work; a hostile guest can plant `escape → /etc/passwd` but the host never follows it |
+| `"follow"` | yes | yes | Trusted single-tenant: dev trees that symlink into Homebrew/`~/.cache`/etc. |
+
+The `"opaque"` default exists because of how `symlink(2)` works: the target is opaque bytes stored on the symlink inode — POSIX never validates it. Resolution happens *in the guest's kernel* against the guest's own namespace, so a target like `/etc/passwd` resolves to the guest's `/etc/passwd`, not the host's. The host serves `readlink()` (returns those bytes) and refuses to follow them. That's the security model: guest can store arbitrary bytes, host never canonicalizes guest-supplied paths.
+
+A runnable example is in [`examples/06-mount-workspace.mjs`](./examples/06-mount-workspace.mjs).
+
+#### Performance: don't put dependency caches on the mount
+
+A mount is great for source code (host-editable, DAX-fast reads). It is **the wrong place** for `node_modules`, `__pycache__`, `target/`, `.next/`, or anything else that gets thousands of small files written by a build/install step.
+
+Why: every guest file operation on a virtio-fs mount becomes a FUSE message round-trip over vsock — open, write, close, fsync each cost ~µs of in-guest overhead plus ~µs of host-side handling. For one big file (a tarball, an image), that overhead is invisible. For an `npm install` that creates ~10–30 k tiny files, you're paying milliseconds × tens of thousands of round-trips. We've measured the same `npm install` at **3–6 minutes** on a virtio-fs mount vs **10–30 seconds** on a virtio-blk volume backing `node_modules`. Same workload, same data, same VM — the difference is purely FUSE round-trip cost × small-file count.
+
+Block devices (`virtio-blk`, exposed via `volumes`) bypass FUSE entirely — the guest's own filesystem driver writes to a backing file on the host with no per-syscall protocol overhead. Same speed as a "normal" disk inside the guest.
+
+### Pattern 6: npm / pnpm / cargo workflow (mount for source, volume for cache)
+
+The shape that gets you both live host edits and fast installs:
+
+```ts
+const image = await Image.build({
+  ref: "node:20-alpine",
+  // Source tree — live host edits, DAX reads
+  mounts: [{ hostPath: "/Users/me/my-app", guestTag: "src", symlinks: "opaque" }],
+  // Dependency cache — fast writes, persists between bakes via the snapshot
+  volumes: [
+    { name: "node_modules", sizeMib: 1024, mountPoint: "/work/node_modules" },
+  ],
+  // One-time install at bake. Once snapshotted, every restore has
+  // node_modules already populated — no install on the hot path.
+  warmup: async (vm) => {
+    await vm.exec({ argv: ["mount", "-t", "virtiofs", "src", "/work"] });
+    await vm.exec({ argv: ["cp", "-r", "/work/package*.json", "/work/node_modules/.."] });
+    await vm.exec({ argv: ["sh", "-c", "cd /work && npm install"], timeoutMs: 600_000 });
+  },
+});
+```
+
+Three layers, each playing to their strengths:
+- **`mounts`** for source: read-only-ish on the hot path, DAX zero-copy, host edits show up immediately (kqueue invalidation).
+- **`volumes`** for write-heavy caches: native-speed writes, no FUSE overhead, persists across restores.
+- **`warmup`** + bake to do the install once: every restore reuses the baked node_modules; cold install is amortized across all future runs of this snapshot.
+
+A runnable example is in [`examples/07-npm-workspace-fast.mjs`](./examples/07-npm-workspace-fast.mjs).
+
 ---
 
 ## Full API
@@ -217,6 +289,7 @@ class Image {
 | `guestPort` | `number` | 80 | Listener port (used with `listenerRequired`) |
 | `listenerRequired` | `boolean` | `false` | Wait for workload's listener before snapshotting |
 | `extraFiles` | `ExtraFile[]` | — | Stage host files into guest at fixed paths |
+| `mounts` | `MountSpec[]` | — | Live host dir mounts via virtio-fs DAX. See Pattern 5. |
 | `warmupTag` | `string` | `"default"` | Cache key for the warmup; bump to invalidate |
 | `warmup` | `(vm: WarmupVm) => Promise<void>` | — | Post-bake state pre-population |
 | `snapshotsDir` | `string` | `~/.local/supermachine-snapshots` | Where to write the snapshot |

package/examples/06-mount-workspace.mjs

@@ -0,0 +1,97 @@
+// 06-mount-workspace.mjs — mount a host directory into the guest via
+// virtio-fs DAX, then run an `npm install`-style symlink workflow
+// from inside the VM.
+//
+// Demonstrates:
+//   - `mounts: [{ hostPath, guestTag, symlinks }]` on Image.build
+//   - The per-mount `symlinks` policy: "deny" | "opaque" | "follow"
+//   - The npm-workspace symlink pattern (node_modules/<name> →
+//     ../packages/<name>) working end-to-end through virtio-fs
+//
+// The mount surfaces host files in the guest with zero-copy DAX
+// reads and kqueue-driven cache invalidation when the host file
+// changes — edit a file on the host, the guest sees the new bytes
+// on the next read with no manual re-mount.
+//
+// `symlinks: "opaque"` is the safe default for guest-writable mounts
+// in a multi-tenant setting: guest can create symlinks (workspace
+// tools need this), targets are stored verbatim (POSIX symlink(2)),
+// and the host never resolves them. A guest planting
+// `escape -> /etc/passwd` gets a symlink whose readlink returns
+// "/etc/passwd" bytes — but the host won't follow it. Path
+// resolution happens in the guest's kernel against the guest's
+// own namespace.
+//
+// Run:
+//   node examples/06-mount-workspace.mjs
+
+import { Image } from "@supermachine/core";
+import { mkdtempSync, mkdirSync, writeFileSync, rmSync, readlinkSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+// 1. Build a host-side workspace tree that mimics what npm creates:
+//      project/
+//        packages/alpha/index.js   "ALPHA"
+//        packages/beta/index.js    "BETA"
+//    Inside the guest we'll add node_modules/<name> symlinks
+//    pointing at ../packages/<name>.
+const project = mkdtempSync(join(tmpdir(), "sm-workspace-"));
+mkdirSync(join(project, "packages", "alpha"), { recursive: true });
+mkdirSync(join(project, "packages", "beta"), { recursive: true });
+writeFileSync(join(project, "packages", "alpha", "index.js"), "ALPHA");
+writeFileSync(join(project, "packages", "beta", "index.js"), "BETA");
+
+// 2. Bake an image with the project mounted at guest tag "workspace".
+//    `symlinks: "opaque"` is the default but spelled out here for
+//    documentation value.
+const img = await Image.build({
+  ref: "alpine:latest",
+  name: "mount-workspace-demo",
+  memoryMib: 256,
+  cmd: ["/bin/sh", "-c", "sleep infinity"],
+  mounts: [
+    {
+      hostPath: project,
+      guestTag: "workspace",
+      symlinks: "opaque",
+    },
+  ],
+});
+
+const pool = await img.pool({ min: 1, max: 1, restoreOnRelease: false });
+const vm = await pool.acquire();
+
+// 3. Inside the guest: mount the virtio-fs share, create the
+//    workspace symlinks, resolve through them.
+const result = await vm.exec({
+  argv: [
+    "sh",
+    "-c",
+    [
+      "mkdir -p /work && mount -t virtiofs workspace /work",
+      "mkdir -p /work/node_modules",
+      "ln -s ../packages/alpha /work/node_modules/alpha",
+      "ln -s ../packages/beta  /work/node_modules/beta",
+      "echo === readlink ===",
+      "readlink /work/node_modules/alpha",
+      "readlink /work/node_modules/beta",
+      "echo === resolve through ===",
+      "cat /work/node_modules/alpha/index.js",
+      "cat /work/node_modules/beta/index.js",
+    ].join(" && "),
+  ],
+  timeoutMs: 10_000,
+});
+console.log(result.stdout.toString());
+
+// 4. Host-side: confirm the symlinks landed on disk as real
+//    symlinks (not regular files), with the bytes the guest asked
+//    for. This is the "store-target-verbatim" guarantee in action.
+console.log("=== host-side symlink targets ===");
+console.log("alpha →", readlinkSync(join(project, "node_modules", "alpha")));
+console.log("beta  →", readlinkSync(join(project, "node_modules", "beta")));
+
+await vm.release();
+await pool.shutdown();
+rmSync(project, { recursive: true, force: true });

package/examples/07-npm-workspace-fast.mjs

@@ -0,0 +1,115 @@
+// 07-npm-workspace-fast.mjs — the integrator-recommended pattern
+// for "edit on host, run in VM, install dependencies once."
+//
+// Three layers, each playing to its strength:
+//
+//   1. virtio-fs MOUNT for the source tree — host edits are live
+//      in the guest (DAX-fast reads, kqueue-driven invalidation),
+//      `symlinks: 'opaque'` so npm workspace symlinks work without
+//      letting the host follow guest-supplied targets.
+//
+//   2. virtio-blk VOLUME for `node_modules` — block I/O is
+//      ~native disk speed; bypasses the FUSE per-file round-trip
+//      cost that makes `npm install` 3-6 min on a virtio-fs mount.
+//      Same install on a volume: ~10-30 s.
+//
+//   3. WARMUP runs `npm install` once at bake time. The result
+//      (node_modules populated on the volume) persists across
+//      restores — every subsequent restore has dependencies ready,
+//      no install on the hot path.
+//
+// Run:
+//   node examples/07-npm-workspace-fast.mjs
+
+import { Image } from "@supermachine/core";
+import { mkdtempSync, mkdirSync, writeFileSync, rmSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+// 1. Build a host-side project that mimics a tiny npm workspace.
+const project = mkdtempSync(join(tmpdir(), "sm-fast-"));
+mkdirSync(project, { recursive: true });
+writeFileSync(
+  join(project, "package.json"),
+  JSON.stringify(
+    {
+      name: "fast-demo",
+      version: "0.0.0",
+      dependencies: { "lodash.chunk": "^4.2.0" },
+    },
+    null,
+    2,
+  ),
+);
+
+// 2. Volume backing file — sparse, 1 GiB cap. Sits alongside the
+//    project for the demo; in production you'd put it under
+//    ~/.local/supermachine/volumes/ or wherever your tenancy boundary
+//    lives.
+const volumePath = join(project, "node_modules.img");
+
+// 3. Bake with a warmup that runs `npm install` once on the volume.
+//    The snapshot captures the *post-install* state; every restore
+//    has node_modules ready instantly.
+const t0 = Date.now();
+const img = await Image.build({
+  ref: "node:20-alpine",
+  name: "npm-workspace-fast-demo",
+  memoryMib: 512,
+  cmd: ["/bin/sh", "-c", "sleep infinity"],
+  mounts: [
+    { hostPath: project, guestTag: "src", symlinks: "opaque" },
+  ],
+  volumes: [
+    { hostPath: volumePath, guestPath: "/work/node_modules", sizeMib: 1024 },
+  ],
+  warmup: async (vm) => {
+    // Mount source RO-ish + format and mount the node_modules volume.
+    await vm.exec({
+      argv: [
+        "sh",
+        "-c",
+        [
+          "mkdir -p /src /work/node_modules",
+          "mount -t virtiofs src /src",
+          // The volume comes up as the next /dev/vdN; on first bake
+          // it's blank, format ext4. Idempotent: existing fs is left
+          // alone by mke2fs -F? No — use `blkid` to skip if already
+          // formatted. For a demo, the bake-time guest always sees
+          // an empty volume.
+          "mkfs.ext4 -F -q /dev/vdb",
+          "mount /dev/vdb /work/node_modules",
+          // Copy package*.json out of the read-mount into the
+          // volume-backed dir so `npm install` writes locally.
+          "cp /src/package.json /work/package.json",
+          "cd /work && npm install --silent --no-fund --no-audit",
+        ].join(" && "),
+      ],
+      timeoutMs: 120_000,
+    });
+  },
+  warmupTag: "v1",
+});
+console.log(`[fast] bake total: ${Date.now() - t0} ms`);
+
+// 4. Run a workload: every restore lands with node_modules already
+//    populated. Re-mount mounts and continue.
+const pool = await img.pool({ min: 1, max: 1, restoreOnRelease: false });
+const vm = await pool.acquire();
+const r = await vm.exec({
+  argv: [
+    "sh",
+    "-c",
+    [
+      "mount -t virtiofs src /src 2>/dev/null || true",
+      "mount /dev/vdb /work/node_modules 2>/dev/null || true",
+      "cd /work && node -e \"console.log(require('lodash.chunk')([1,2,3,4,5], 2))\"",
+    ].join("; "),
+  ],
+  timeoutMs: 10_000,
+});
+console.log(r.stdout.toString());
+
+await vm.release();
+await pool.shutdown();
+rmSync(project, { recursive: true, force: true });

package/examples/README.md

@@ -11,6 +11,8 @@ with the package installed.
 | `03-snapshot-then-restore.mjs` | `vm.writeFile` / `readFile`, mid-flight `vm.snapshot()`, `Image.fromSnapshot` |
 | `04-expose-tcp.mjs` | `listenerRequired: true` bakes, `vm.exposeTcp()`, `TcpForwarder` |
 | `05-spawn-streaming.mjs` | `vm.spawn()` → `ExecChild` (writeStdin / readStdout / wait) |
+| `06-mount-workspace.mjs` | `mounts` + `symlinks` policy: live host dir into guest, npm-workspace symlink pattern |
+| `07-npm-workspace-fast.mjs` | The recommended npm/pnpm/cargo pattern: virtio-fs mount for source + virtio-blk volume for node_modules + warmup-bake for one-time install |
 
 ## Requirements
 
@@ -33,7 +35,7 @@ node --input-type=module -e \
   "$(sed 's|@supermachine/core|./index.js|' examples/01-hello.mjs)"
 ```
 
-For a quick smoke test that all five work end-to-end:
+For a quick smoke test that all examples work end-to-end:
 
 ```sh
 for f in examples/*.mjs; do

package/index.d.ts

@@ -39,6 +39,49 @@ export interface MountSpec {
   hostPath: string
   /** virtiofs tag the guest mounts by. Max 35 bytes. */
   guestTag: string
+  /**
+   * Symlink policy: `"deny"` | `"opaque"` | `"follow"`. Default
+   * `"opaque"` — guest may create symlinks (npm/pnpm/yarn rely on
+   * them) but host-side symlinks pointing outside the mount root
+   * are rejected. Use `"deny"` for paranoid mounts (no symlinks or
+   * hard links at all) or `"follow"` for trusted single-tenant
+   * workloads (pre-0.5.5 `allowExternalSymlinks: true` behaviour).
+   */
+  symlinks?: string
+}
+
+/**
+ * A writable virtio-blk volume backed by a host file. Use for
+ * dependency caches (`node_modules`, `target/`, `__pycache__`,
+ * `.next/`) and other write-heavy paths — block I/O is ~native
+ * disk speed, no FUSE round-trip per file. Contrast with
+ * [`MountSpec`] which exposes a host directory via virtio-fs
+ * (right for source trees you want live-editable from the host,
+ * wrong for thousands-of-small-files write workloads).
+ *
+ * The host file is created sparse at `hostPath` if missing
+ * (actual disk usage tracks what the guest writes). Snapshots
+ * capture the mapping, not the volume contents — same backing
+ * file is reused across all restores of this snapshot.
+ */
+export interface VolumeSpec {
+  /**
+   * Host file backing this volume. Created sparse at `sizeMib`
+   * MiB if missing. Persists across restores; delete it to
+   * reset state.
+   */
+  hostPath: string
+  /**
+   * Mount point inside the guest, e.g. `"/var/lib/postgres"`
+   * or `"/work/node_modules"`. The guest needs to actually
+   * mount it (init can do this, or your warmup callback).
+   */
+  guestPath: string
+  /**
+   * Size cap in MiB. Default 1024 (1 GiB). Sparse — actual
+   * disk usage matches what the guest writes.
+   */
+  sizeMib?: number
 }
 
 /**
@@ -105,6 +148,25 @@ export interface BuildOptions {
    * re-bakes.
    */
   mounts?: Array<MountSpec>
+  /**
+   * Writable virtio-blk volumes. Each entry maps a sparse host
+   * file to a guest mount point. Use for dependency caches
+   * (`node_modules`, `target/`) — block I/O is ~native disk
+   * speed and bypasses the FUSE round-trip cost that makes
+   * `mounts` slow for thousands-of-small-files workloads.
+   * Volume bytes persist across restores; only the mapping is
+   * captured in the snapshot.
+   */
+  volumes?: Array<VolumeSpec>
+  /**
+   * Extra kernel cmdline tokens, appended after supermachine's
+   * own defaults (earlycon, console, tsi_hijack, etc.). Power-
+   * user / testing escape hatch — e.g. `["panic=1"]` to force
+   * an immediate panic on warning, or `["init=/nope"]` to
+   * deliberately trigger an early-init kernel panic in a test.
+   * Folded into the bake's input hash so changing this re-bakes.
+   */
+  extraKernelArgs?: Array<string>
   /**
    * Override where to fetch the image bytes. Default behavior
    * (when omitted) treats `ref` as a registry reference and pulls

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@supermachine/core",
-  "version": "0.5.3",
+  "version": "0.7.1",
   "description": "Run any OCI/Docker image as a hardware-isolated microVM. Node/Bun/Deno binding for the supermachine Rust crate.",
   "license": "Apache-2.0",
   "main": "index.js",
@@ -22,7 +22,7 @@
     "arm64"
   ],
   "optionalDependencies": {
-    "@supermachine/core-darwin-arm64": "0.5.3"
+    "@supermachine/core-darwin-arm64": "0.7.1"
   },
   "scripts": {
     "build:rust": "TYPE_DEF_TMP_PATH=$(pwd)/scripts/.napi_type_defs.tmp cargo build --release -p supermachine-napi && cargo build --release --bin supermachine-worker -p supermachine",