@inkandswitch/patchwork-bootloader 0.6.2 → 0.7.0

package/CHANGELOG.md

@@ -1,5 +1,49 @@
 # @inkandswitch/patchwork-bootloader
 
+## 0.7.0
+
+### Minor Changes
+
+- 2e71745: Every Repo on the origin is its own Subduction node. A tab holds this origin's IndexedDB, keeps its own WebSocket to the sync server, and meets the other tabs over a BroadcastChannel carrying Subduction (`siblingAdapters()` in `@inkandswitch/patchwork-bootloader/siblings`, passed to `new Repo({ subductionAdapters })`). The automerge SharedWorker no longer sits between tabs and storage — it is one more such node, kept only to resolve `automerge:` URLs for the service worker, which can't own a Repo itself. The websocket proxy worker is gone with it.
+
+  Benchmarked against the shared-worker arrangement (`sites/bench`): boot and memory are a wash or better, cross-tab propagation matches, and two shared-worker failures go away — a `find()` racing a sibling's `create()` settled as unavailable, and edits made just before a tab closed were lost, since a storageless tab had nothing to flush to. Each tab flushing its own IndexedDB closes both.
+
+  Keyhive sites use the subduction-backed hive in both the tab and the worker, each talking to the sync server directly.
+
+  `patchwork.sw.subscribeSyncState(documentId, listener)` stays, now a filter over the tab's own Repo's `subduction-remote-heads` event, replaying the server's current heads from `handle.getSyncInfo()` on subscribe; `SyncStateDocMessage` is exported from `@inkandswitch/patchwork`. Removed from `setupServiceWorker()`'s result: `subscribeToRepoChannel`, `getRepoChannel`, `subscribeSyncState`. The `@patchwork/syncstate` BroadcastChannel and the other `SyncState*` message types are gone too; a tab's own Repo has what they carried — `repo.isSubductionConnected()` and the `subduction-connection` event for the link, `repo.connectedSubductionPeerIds()` for which peers are the server, and `patchwork.signerIdentity` for this tab's peer id. `createRepo` in `@inkandswitch/patchwork` takes no arguments.
+
+  `@inkandswitch/patchwork-bootloader` depends on `@automerge/automerge-repo-network-broadcastchannel`, which is also on the importmap.
+
+  That worker is renamed for what it now does: `@inkandswitch/patchwork-bootloader/automerge-worker` is now `@inkandswitch/patchwork-bootloader/automerge-protocol-handler-worker`, emitted as `automerge-protocol-handler-worker.js` (the `workerPath` option on `setupServiceWorker` still overrides it), and `getAutomergeWorker()` is now `getAutomergeProtocolHandlerWorker()`.
+
+  Inter-tab sync is Subduction, not classic automerge sync. `connectSiblings(repo, hive)` is replaced by `siblingAdapters()`, which returns the `subductionAdapters` entries for a Repo rather than mutating one after the fact, so it is passed to the `Repo` constructor. The frames on the siblings BroadcastChannel are Subduction transport frames authenticated by each node's own signer; the keyhive network adapter no longer wraps that channel, since keyhive material reaches siblings the same way it reaches the sync server. The one remaining classic-sync path is the opt-in classic sync server the protocol handler worker connects to on request.
+
+  Subduction's handshake has an initiator and a responder, and a BroadcastChannel is a mesh, so the siblings adapter is passed with `role: "mesh"`, added to the automerge-repo fork's `subductionAdapters` by this repo's pnpm patch: for each pair of peers on the adapter, the one whose peer id sorts lower initiates the handshake and the other accepts.
+
+### Patch Changes
+
+- 47bc4cf: `@automerge/automerge` goes to `3.4.1`, and `@automerge/automerge-repo-network-broadcastchannel` joins the automerge-repo family at `2.6.0-subduction.48`.
+- Updated dependencies [47bc4cf]
+  - @inkandswitch/patchwork-filesystem@0.2.9
+  - @inkandswitch/patchwork-plugins@1.2.4
+
+## 0.6.3
+
+### Patch Changes
+
+- ebca53b: Move the automerge-repo subduction fork to 2.6.0-subduction.48, `@automerge/automerge-repo-keyhive` to 0.5.0-alpha.7, and `@keyhive/keyhive` to 0.1.0-alpha.8. These three are bumped together because the keyhive package pins its automerge-repo version exactly.
+
+  keyhive 0.5 renames the two hive flavours. The network-adapter hive is now `LegacyAutomergeRepoKeyhive`, built by `initializeLegacyAutomergeRepoKeyhive`; the subduction hive keeps the name `AutomergeRepoKeyhive` and is built by `initializeAutomergeRepoKeyhive`. Both extend `AutomergeRepoKeyhiveBase`, which is what Patchwork's `hive` fields are typed as, so a tool that only reads membership works against either.
+
+  `createKeyhiveNetworkAdapter` takes an options object instead of positional arguments, and `onlyShareWithHardcodedServerPeerId` is now `onlyShareWithSyncServer`.
+
+- Updated dependencies [ebca53b]
+- Updated dependencies [882eacd]
+  - @inkandswitch/patchwork-elements@6.0.2
+  - @inkandswitch/patchwork-filesystem@0.2.8
+  - @inkandswitch/patchwork-plugins@1.2.3
+  - @inkandswitch/patchwork-providers@0.5.2
+
 ## 0.6.2
 
 ### Patch Changes

package/dist/automerge-protocol-handler-worker.js

@@ -0,0 +1,315 @@
+// The Repo that resolves `automerge:` URLs for the service worker, in a
+// SharedWorker: one instance serves every tab and lives as long as any tab
+// does.
+//
+// It is a node like any tab's: the same IndexedDB, its own sync-server socket,
+// and the siblings channel to the tabs. Resolving requests is its whole job.
+// When the service worker misses the cache for a request that looks like a URL
+// encoded URL, it broadcasts a HandoffRequestMessage on HANDOFF_CHANNEL; we
+// resolve the automerge URL, write the response into the service worker's
+// cache (keyed by a Request reconstructed to match the one it's holding), and
+// reply on the same channel.
+import { initializeWasm, hasHeads } from "@automerge/automerge/slim";
+// eslint-disable-next-line
+// @ts-ignore — initSync is a wasm-bindgen runtime helper not in the .d.ts
+import { initSync as initSubductionSync } from "@automerge/automerge-subduction/slim";
+import { MemorySigner } from "@automerge/automerge-subduction/slim";
+import { Repo, isValidAutomergeUrl, parseAutomergeUrl, stringifyAutomergeUrl, } from "@automerge/automerge-repo/slim";
+import { resolvePath } from "@inkandswitch/patchwork-filesystem";
+import { IndexedDBWorkerStorageAdapter } from "@automerge/automerge-repo-storage-indexeddb/IndexedDBWorkerStorageAdapter";
+import { WebSocketWorkerClientAdapter } from "@automerge/automerge-repo-network-websocket";
+import { initializeAutomergeRepoKeyhive, initKeyhiveWasm, } from "@automerge/automerge-repo-keyhive";
+import { DEFAULT_CLASSIC_SYNC_SERVER } from "./sync-config.js";
+import { siblingAdapters } from "./siblings.js";
+import { keyhiveStorageName, storagePrefix } from "./storage.js";
+import { startWorkerControl } from "./worker-control.js";
+import { HANDOFF_CHANNEL, } from "./types.js";
+const syncServer = typeof __SYNC_SERVER__ !== "undefined"
+    ? __SYNC_SERVER__
+    : { url: "wss://subduction.sync.inkandswitch.com" };
+const RESOLVE_TIMEOUT_MS = 30_000;
+const CACHEABLE_STATUSES = [200, 203, 204];
+const control = startWorkerControl("automerge-protocol-handler-worker", {
+    onMessage: handleControlMessage,
+});
+const log = control.log;
+// ── The repo ───────────────────────────────────────────────────────────
+let repoPromise = null;
+function getRepo() {
+    if (!repoPromise) {
+        repoPromise = buildRepo();
+        // Don't cache a rejection (e.g. the wasm fetch failed): clear the slot so
+        // the next caller retries from scratch.
+        repoPromise.catch(() => {
+            repoPromise = null;
+        });
+    }
+    return repoPromise;
+}
+async function buildRepo() {
+    log("fetching wasm");
+    const [automergeWasm, subductionWasm] = await Promise.all([
+        fetch("/automerge.wasm").then((r) => r.arrayBuffer()),
+        fetch("/subduction.wasm").then((r) => r.arrayBuffer()),
+    ]);
+    initSubductionSync(new Uint8Array(subductionWasm));
+    await initializeWasm(new Uint8Array(automergeWasm));
+    log("wasm initialized");
+    const { repo, hive } = syncServer.keyhive
+        ? await buildKeyhiveRepo(syncServer.keyhive)
+        : { repo: buildPlainRepo() };
+    self.repo = repo;
+    if (hive)
+        self.hive = hive;
+    return repo;
+}
+function buildPlainRepo() {
+    return new Repo({
+        signer: new MemorySigner(),
+        storage: new IndexedDBWorkerStorageAdapter(),
+        peerId: `${storagePrefix}-resolver-${Math.random().toString(36).slice(2)}`,
+        subductionWebsocketEndpoints: [syncServer.url],
+        subductionAdapters: siblingAdapters(),
+        enableRemoteHeadsGossiping: true,
+    });
+}
+async function buildKeyhiveRepo(keyhiveSyncServer) {
+    initKeyhiveWasm();
+    const { hive, repo } = await initializeAutomergeRepoKeyhive({
+        createRepo: (config) => new Repo(config),
+        storage: new IndexedDBWorkerStorageAdapter(keyhiveStorageName),
+        peerIdSuffix: `${storagePrefix}-resolver` + Math.random().toString(36).slice(2),
+        automaticArchiveIngestion: true,
+        cachingMode: "periodic",
+        // ARK selects the relay via `syncServer`, which pairs the contact card with
+        // the matching peer id. Omitting it defaults to "subduction".
+        syncServer: keyhiveSyncServer,
+        repo: {
+            storage: new IndexedDBWorkerStorageAdapter(),
+            subductionWebsocketEndpoints: [syncServer.url],
+            subductionAdapters: siblingAdapters(),
+            enableRemoteHeadsGossiping: true,
+        },
+    });
+    hive.networkAdapter.whenReady().then(() => {
+        hive.networkAdapter.syncKeyhive();
+    });
+    return { repo, hive };
+}
+// ── Classic sync ───────────────────────────────────────────────────────
+let classicSyncServer = DEFAULT_CLASSIC_SYNC_SERVER;
+let classicSyncAdapter = null;
+let classicSyncConnect = null;
+function connectClassicSyncNetwork(server) {
+    const url = server.trim() || DEFAULT_CLASSIC_SYNC_SERVER;
+    if (classicSyncConnect && classicSyncServer === url)
+        return classicSyncConnect;
+    if (classicSyncAdapter && classicSyncServer !== url) {
+        classicSyncAdapter.disconnect();
+        classicSyncAdapter = null;
+    }
+    classicSyncServer = url;
+    const connecting = (async () => {
+        const repo = await getRepo();
+        if (!classicSyncAdapter) {
+            classicSyncAdapter = new WebSocketWorkerClientAdapter(url);
+            repo.networkSubsystem.addNetworkAdapter(classicSyncAdapter);
+        }
+        await classicSyncAdapter.whenReady();
+        log("classic sync connected", url);
+    })();
+    // Clear the memo on failure so a later attempt can retry, and swallow the
+    // rejection on this copy so it isn't reported as unhandled — callers get it
+    // from the promise we return.
+    classicSyncConnect = connecting;
+    connecting.catch(() => {
+        if (classicSyncConnect === connecting)
+            classicSyncConnect = null;
+    });
+    return connecting;
+}
+// ── Control protocol ───────────────────────────────────────────────────
+function handleControlMessage(data, controlPort, event) {
+    if (data?.type !== "connect-classic-sync")
+        return;
+    const [replyPort] = event.ports;
+    const server = typeof data.server === "string" ? data.server : DEFAULT_CLASSIC_SYNC_SERVER;
+    connectClassicSyncNetwork(server).then(() => {
+        replyPort?.postMessage({ type: "connect-classic-sync-ready" });
+        replyPort?.close();
+    }, (err) => {
+        console.error("connectClassicSyncNetwork failed", err);
+        replyPort?.postMessage({
+            type: "connect-classic-sync-failed",
+            error: String(err),
+        });
+        replyPort?.close();
+    });
+}
+// ── Resolving ──────────────────────────────────────────────────────────
+function waitForHeads(handle, hexHeads, signal) {
+    if (hasHeads(handle.doc(), hexHeads))
+        return Promise.resolve(true);
+    if (signal.aborted)
+        return Promise.resolve(false);
+    return new Promise((resolve) => {
+        const cleanup = () => {
+            handle.off("heads-changed", check);
+            signal.removeEventListener("abort", onAbort);
+        };
+        const check = () => {
+            if (!hasHeads(handle.doc(), hexHeads))
+                return;
+            cleanup();
+            resolve(true);
+        };
+        const onAbort = () => {
+            cleanup();
+            resolve(false);
+        };
+        handle.on("heads-changed", check);
+        signal.addEventListener("abort", onAbort);
+        // The heads may have landed between the check above and subscribing.
+        check();
+    });
+}
+/**
+ * Thrown instead of returning a Response when the request should fail as a
+ * network error rather than resolve to something the caller can memoize.
+ * See {@link HandoffAbortMessage}.
+ */
+class AbortHandoff extends Error {
+}
+async function resolveAutomergeUrl(automergeURL, signal) {
+    const repo = await getRepo();
+    const [maybeAutomergeUrl, ...path] = automergeURL.href.split("/");
+    if (!isValidAutomergeUrl(maybeAutomergeUrl)) {
+        return new Response("invalid automerge url", { status: 400 });
+    }
+    if (path.length && !path[path.length - 1])
+        path.pop();
+    const { heads, hexHeads, documentId } = parseAutomergeUrl(maybeAutomergeUrl);
+    // todo, maybe a bad idea? maybe we should throw instead of es-module-caching
+    // the headless req
+    if (!heads) {
+        const folder = await repo.find(maybeAutomergeUrl, { signal });
+        const url = stringifyAutomergeUrl({ documentId, heads: folder.heads() });
+        const location = `/${encodeURIComponent(url)}${path.length ? `/${path.join("/")}` : ""}`;
+        return Response.redirect(location, 307);
+    }
+    const baseHandle = await repo.find(stringifyAutomergeUrl({ documentId }), {
+        signal,
+    });
+    if (!(await waitForHeads(baseHandle, hexHeads ?? [], signal))) {
+        throw new AbortHandoff(`heads not found for ${maybeAutomergeUrl} within ${RESOLVE_TIMEOUT_MS}ms`);
+    }
+    const resolved = await resolvePath(repo, baseHandle.view(heads), path.map(decodeURIComponent));
+    if (!resolved) {
+        throw new Error(`couldn't resolve ${path.join("/")} in folder at ${maybeAutomergeUrl}`);
+    }
+    const body = resolved.content instanceof Uint8Array
+        ? new Uint8Array(resolved.content)
+        : resolved.content;
+    return new Response(body, {
+        status: 200,
+        headers: { "content-type": resolved.type },
+    });
+}
+const handoffChannel = new BroadcastChannel(HANDOFF_CHANNEL);
+function replyToHandoff(id, status, body) {
+    handoffChannel.postMessage({
+        id,
+        type: "response",
+        response: { status, body, headers: { "content-type": "text/plain" } },
+    });
+}
+function impatience(limit) {
+    return new Promise((_, reject) => setTimeout(() => reject(new Error(`resolve timeout after ${limit}ms`)), limit));
+}
+async function handleHandoffRequest(message) {
+    const { id, cachename, request } = message;
+    let handoff;
+    try {
+        handoff = new URL(request.handoffURL);
+    }
+    catch {
+        console.error("couldn't parse handoff url", request);
+        replyToHandoff(id, 400, `couldn't parse a special url out of ${request.url}`);
+        return;
+    }
+    // Other handlers may be listening on the channel for other schemes, so stay
+    // quiet rather than clobbering their reply with an error.
+    if (handoff.protocol !== "automerge:") {
+        log(`ignoring handoff ${id} for non-automerge url ${handoff}. not my circus, not my monkeys`);
+        return;
+    }
+    let response;
+    try {
+        log(`resolving handoff ${id} for ${handoff}`);
+        const signal = AbortSignal.timeout(RESOLVE_TIMEOUT_MS);
+        response = await Promise.race([
+            resolveAutomergeUrl(handoff, signal),
+            impatience(RESOLVE_TIMEOUT_MS),
+        ]);
+    }
+    catch (error) {
+        if (error instanceof AbortHandoff) {
+            handoffChannel.postMessage({
+                id,
+                type: "abort",
+                reason: error.message,
+            });
+            return;
+        }
+        console.error(`error resolving ${request.url}`, error);
+        replyToHandoff(id, 557, error instanceof Error
+            ? `${error.message}\n\n${error.stack}`
+            : String(error));
+        return;
+    }
+    try {
+        if (!CACHEABLE_STATUSES.includes(response.status)) {
+            // Errors, redirects and the like go back inline for the service worker to
+            // serve directly, so they aren't cached forever (still in esmodulecache,
+            // cleared after a refresh)
+            log(`responding inline to ${request.url} with ${response.status}`);
+            handoffChannel.postMessage({
+                id,
+                type: "response",
+                response: {
+                    status: response.status,
+                    headers: Object.fromEntries(response.headers.entries()),
+                    body: response.body ? await response.text() : undefined,
+                },
+            });
+            return;
+        }
+        // Reconstruct the request the service worker is holding so the entry matches
+        // its cache.match. `destination` isn't constructible but doesn't participate
+        // in cache matching.
+        const cacheKey = new Request(request.url, {
+            method: request.method,
+            headers: request.headers,
+            referrer: request.referrer,
+        });
+        const cache = await caches.open(cachename);
+        await cache.put(cacheKey, response);
+        log(`cached ${cacheKey.url} in ${cachename}`);
+        handoffChannel.postMessage({
+            id,
+            type: "cached",
+        });
+    }
+    catch (error) {
+        console.error(`failed to reply for ${request.url}`, error);
+        replyToHandoff(id, 558, String(error));
+    }
+}
+handoffChannel.addEventListener("message", (event) => {
+    if (event.data?.type === "request") {
+        void handleHandoffRequest(event.data);
+    }
+});
+// Announce ourselves so the service worker can re-broadcast handoff requests
+// sent while we were booting.
+handoffChannel.postMessage({ type: "online" });

package/dist/externals-list.js

@@ -6,11 +6,7 @@ const externals = [
     "@automerge/automerge/slim",
     "@automerge/automerge-repo",
     "@automerge/automerge-repo/slim",
-    // Port-donation plumbing for WorkerWebSocketEndpoint: tabs spawn the shared
-    // proxy entry and donate its port to the automerge worker (Chrome can't
-    // spawn workers from inside a SharedWorker). See setup.ts/automerge-worker.ts.
-    "@automerge/automerge-repo/worker-port",
-    "@automerge/automerge-repo/subduction-websocket-worker-shared",
+    "@automerge/automerge-repo-network-broadcastchannel",
     "@automerge/automerge-repo-network-messagechannel",
     "@automerge/automerge-repo-network-websocket",
     "@automerge/automerge-repo-storage-indexeddb",

package/dist/setup.d.ts

@@ -1,10 +1,7 @@
-import type { SetupServiceWorkerOptions, SetupServiceWorkerResult, SyncStateDocMessage } from "./types.js";
-import debug from "debug";
-export declare const lifecycleLog: debug.Debugger;
+import type { SetupServiceWorkerOptions, SetupServiceWorkerResult } from "./types.js";
+import { lifecycleLog } from "./shared-worker-lifecycle.js";
+export { lifecycleLog };
 export declare function bumpServiceWorkerCache(sw?: ServiceWorker | null): void;
-export declare function getAutomergeWorker(): SharedWorker;
-type SyncStateListener = (update: SyncStateDocMessage) => void;
-export declare function subscribeSyncState(documentId: string, listener: SyncStateListener): () => void;
+export declare function getAutomergeProtocolHandlerWorker(): SharedWorker;
 export declare function connectClassicSync(server?: string): Promise<void>;
 export default function setupServiceWorker(options?: SetupServiceWorkerOptions): Promise<SetupServiceWorkerResult>;
-export {};