@qvac/ggml-rpc-server 0.4.0 → 0.5.1

package/CHANGELOG.md

@@ -1,5 +1,34 @@
 # Changelog
 
+## [0.5.1] - 2026-10-15
+
+### Fixed
+
+- On Windows, the addon now binds the delay-loaded `@qvac/fabric` runtime
+  when it loads, so a first fabric call on a worker thread cannot fail with
+  `0xC06D007E` ([#5121](https://github.com/tetherto/qvac/pull/5121)). Loading throws an error naming the fabric
+  module if the bind fails.
+
+## [0.5.0] - 2026-10-15
+
+### Changed
+
+- Without a `port`, the native server binds a free port itself and reports it,
+  instead of this package picking a free port and releasing it before the
+  server bound it. Another process could take the port in between, which showed
+  up as a rare `RpcServerStartError`. `allocateFreePort()` is still exported.
+- `rdmaCapable` and `expectRdma` now use the capability that `@qvac/fabric`
+  reports, replacing a search of the RPC module for a log line. They mean the
+  server will try RDMA, which on Linux also needs `libibverbs.so.1` installed,
+  an RDMA device, and `GGML_RPC_NO_RDMA` unset. A host with the library but no
+  RDMA device now reports `rdmaCapable: false`, so `expectRdma: true` fails
+  closed there.
+- Requires `@qvac/fabric` 0.22.0, whose RPC backend exports
+  `ggml_backend_rpc_server_get_port` and `ggml_backend_rpc_rdma_supported`.
+  A mobile app that still installs `@qvac/fabric-android-arm64` or
+  `@qvac/fabric-ios` 0.21 fails to start the server with
+  `RpcServerStartError`.
+
 ## [0.4.0] - 2026-10-14
 
 ### Changed

package/README.md

@@ -24,6 +24,11 @@ try {
 }
 ```
 
+Without a `port`, the server binds a free port itself and reports it in
+`server.port`, so no other process can take the port before the server binds
+it. `allocateFreePort()` only finds a port that is free now; prefer leaving
+`port` unset.
+
 The listener accepts IPv4 addresses; `localhost` is normalized to `127.0.0.1`.
 The default host is `127.0.0.1`, but loopback is not private to the host
 application: other local processes (including apps with local TCP access on
@@ -48,9 +53,10 @@ Native server output is written to the host application's platform log.
 
 Errors are classes that can be matched by `name` or `instanceof`. Invalid
 options throw `RpcServerInvalidHostError`, `RpcServerNonLoopbackHostError`,
-`RangeError` (port) or `TypeError` (threads); a failed port lookup throws
-`RpcServerPortAllocationError`; and `expectRdma: true` on a backend without RDMA
-throws `RpcServerRdmaUnavailableError`. Failures reported by the native server
+`RangeError` (port) or `TypeError` (threads); `allocateFreePort()` throws
+`RpcServerPortAllocationError` when it cannot find a port; and
+`expectRdma: true` when the server would not try RDMA throws
+`RpcServerRdmaUnavailableError`. Failures reported by the native server
 extend `RpcServerNativeError`, which carries the native error as `cause` and a
 `code` equal to its `name`:
 
@@ -58,31 +64,31 @@ extend `RpcServerNativeError`, which carries the native error as `cause` and a
 |---|---|
 | `RpcServerDeviceError` | No requested device exists, or no device is available |
 | `RpcServerCacheError` | The cache directory cannot be resolved or created |
-| `RpcServerStartError` | The server cannot be created or bound, or the RPC backend is missing |
+| `RpcServerStartError` | The server cannot be created or bound, the RPC backend is missing, or the installed `@qvac/fabric` is too old |
 | `RpcServerBackendError` | The Fabric backends directory is invalid or cannot be inspected |
 | `RpcServerStopError` | The server does not stop cleanly |
 
 Anything else from the native addon, such as an out-of-memory failure, is
 rethrown unchanged.
 
-## Linux requirements
-
-The Linux RPC backend in `@qvac/fabric` links `libibverbs.so.1`, so Linux
-hosts must provide it (`libibverbs1` on Debian/Ubuntu) even when connections
-use TCP. Without it, `startRpcServer()` rejects with an error whose `code` is
-`RpcServerStartError`.
-RDMA also requires the provider package for the host's hardware.
+## RDMA on Linux
 
-## RDMA-capable builds
+The Linux RPC backend in `@qvac/fabric` is built with RDMA and loads
+`libibverbs.so.1` (`libibverbs1` on Debian/Ubuntu) at runtime. When the library
+is installed and the host has an RDMA device, the server tries RDMA with each
+client that also supports it and falls back to TCP otherwise. Without the
+library or a device, the server runs over TCP only. RDMA also requires the
+provider package for the host's hardware.
 
-On Linux (not Android), `rdmaCapable` reports whether the installed
-`@qvac/fabric` RPC backend was built with RDMA. Such a backend negotiates RDMA
-with each client that also supports it and falls back to TCP otherwise, so
-`rdmaCapable: true` does not guarantee that a given connection uses RDMA. Other
-platforms always report `false`.
+`rdmaCapable` reports whether the server will try RDMA: the library loaded, it
+lists an RDMA device, and `GGML_RPC_NO_RDMA` is not set. A host with libibverbs
+but no RDMA device, common where MPI or UCX installs the library, reports
+`false`. `rdmaCapable: true` does not guarantee that a given connection uses
+RDMA, since the client and the link must support it too. Android, iOS, macOS
+and Windows always report `false`.
 
 To fail closed when RDMA is required, pass `expectRdma: true`. Startup rejects
-with `RpcServerRdmaUnavailableError` when the installed backend lacks RDMA:
+with `RpcServerRdmaUnavailableError` when the server would not try RDMA:
 
 ```js
 const server = await startRpcServer({
@@ -95,8 +101,8 @@ const server = await startRpcServer({
 console.log(server.rdmaCapable)
 ```
 
-Without `expectRdma`, the server starts either way and reports the backend's
-capability in `rdmaCapable`.
+Without `expectRdma`, the server starts either way and reports in
+`rdmaCapable` whether it will try RDMA.
 
 ## Testing
 

package/index.d.ts

@@ -9,7 +9,7 @@ export declare class RpcServerInvalidHostError extends Error {
     constructor(host: string);
 }
 export declare class RpcServerRdmaUnavailableError extends Error {
-    constructor();
+    constructor(cause?: unknown);
 }
 /**
  * A failure reported by the native server. The addon raises plain errors with a
@@ -55,8 +55,10 @@ export interface RpcServer {
     readonly url: string;
     readonly device?: string;
     /**
-     * Whether the loaded Fabric RPC backend was built with RDMA. Such a backend
-     * negotiates RDMA with each RDMA-capable client and falls back to TCP otherwise.
+     * Whether new connections will try RDMA: the Fabric RPC backend was built
+     * with it, loaded libibverbs, found an RDMA device, and `GGML_RPC_NO_RDMA` is
+     * unset. Each connection still falls back to TCP when the client or the link
+     * cannot use RDMA.
      */
     readonly rdmaCapable: boolean;
     stop(): Promise<void>;
@@ -64,5 +66,9 @@ export interface RpcServer {
 export interface AllocateFreePortOptions {
     readonly allowNonLoopbackHost?: boolean;
 }
+/**
+ * Finds a port that is free now. Another process can take it before you bind
+ * it, so `startRpcServer()` without a `port` lets the server bind one itself.
+ */
 export declare function allocateFreePort(host?: string, options?: AllocateFreePortOptions): Promise<number>;
 export declare function startRpcServer(options?: StartRpcServerOptions): Promise<RpcServer>;

package/index.js

@@ -23,9 +23,6 @@ const TRUSTED_LAN_WARNING_CODE = "QVAC_GGML_RPC_SERVER_TRUSTED_LAN";
 // Keep native handles alive until an explicit stop finishes. Otherwise their
 // finalizer can synchronously stop and join a live server during garbage collection.
 const activeServerHandles = new Set();
-// Fabric never unloads its RPC module, so its RDMA build cannot change within a
-// process. Cache the native check, which reads the module from disk.
-let rdmaSupport;
 class RpcServerPortAllocationError extends Error {
   constructor(cause) {
     super("Failed to allocate a free port for ggml-rpc-server", { cause });
@@ -50,8 +47,12 @@ class RpcServerInvalidHostError extends Error {
 }
 exports.RpcServerInvalidHostError = RpcServerInvalidHostError;
 class RpcServerRdmaUnavailableError extends Error {
-  constructor() {
-    super("RDMA is not available in the installed @qvac/fabric RPC backend");
+  constructor(cause) {
+    super(
+      "RDMA is not available: the @qvac/fabric RPC backend lacks it, libibverbs.so.1 " +
+        "could not be loaded, the host has no RDMA device, or GGML_RPC_NO_RDMA is set",
+      { cause },
+    );
     this.name = "RpcServerRdmaUnavailableError";
   }
 }
@@ -115,6 +116,9 @@ const nativeErrorClasses = new Map([
 ]);
 function toTypedError(error) {
   const code = error?.code;
+  if (code === "RpcServerRdmaUnavailableError") {
+    return new RpcServerRdmaUnavailableError(error);
+  }
   const ErrorClass =
     typeof code === "string" ? nativeErrorClasses.get(code) : undefined;
   return ErrorClass === undefined
@@ -192,17 +196,10 @@ function validateThreads(threads) {
     throw new TypeError("threads must be a positive integer");
   }
 }
-// The RPC module @qvac/fabric loads, from wherever fabric ships its backends.
-function rpcBackendSupportsRdma() {
-  if (rdmaSupport === undefined) {
-    try {
-      rdmaSupport = binding.rpcBackendSupportsRdma({});
-    } catch (error) {
-      throw toTypedError(error);
-    }
-  }
-  return rdmaSupport;
-}
+/**
+ * Finds a port that is free now. Another process can take it before you bind
+ * it, so `startRpcServer()` without a `port` lets the server bind one itself.
+ */
 function allocateFreePort(
   host = exports.DEFAULT_RPC_SERVER_HOST,
   options = {},
@@ -231,23 +228,17 @@ async function startRpcServer(options = {}) {
   assertLoopbackHost(host, options.allowNonLoopbackHost);
   warnForTrustedLanHost(host, options.allowNonLoopbackHost);
   validateThreads(options.threads);
-  const rdmaCapable = rpcBackendSupportsRdma();
-  if (options.expectRdma === true && !rdmaCapable) {
-    throw new RpcServerRdmaUnavailableError();
-  }
-  const port =
-    options.port ??
-    (await allocateFreePort(host, {
-      allowNonLoopbackHost: options.allowNonLoopbackHost,
-    }));
-  validatePort(port);
+  if (options.port !== undefined) validatePort(options.port);
   const device = normalizeDevice(options.device);
-  const handle = await callNative(() =>
+  // Without a port the server binds port 0 itself and reports the port it got,
+  // so no other process can take the port between choosing and binding it.
+  const { handle, port, rdmaCapable } = await callNative(() =>
     binding.startServer({
-      endpoint: `${host}:${port}`,
+      endpoint: `${host}:${options.port ?? 0}`,
       device,
       cache: options.cache ?? false,
       threads: options.threads,
+      expectRdma: options.expectRdma === true,
     }),
   );
   activeServerHandles.add(handle);

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@qvac/ggml-rpc-server",
-  "version": "0.4.0",
+  "version": "0.5.1",
   "description": "Managed ggml RPC server for QVAC LLM inference",
   "addon": true,
   "scripts": {

package/prebuilds/darwin-arm64/qvac__ggml-rpc-server.bare.exports

@@ -7,14 +7,13 @@
     ],
     "current_versions": [
       {
-        "version": "0.4"
+        "version": "0.5.1"
       }
     ],
     "exported_symbols": [
       {
         "text": {
           "global": [
-            "__ZN10rpc_server23rpcBackendHasRdmaMarkerERKNSt3__14__fs10filesystem4pathE",
             "__ZN10rpc_server12ServerRunner4stopEv",
             "__ZN10rpc_server13selectDevicesERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE",
             "_bare_register_module_v0",

package/prebuilds/darwin-x64/qvac__ggml-rpc-server.bare.exports

@@ -7,14 +7,13 @@
     ],
     "current_versions": [
       {
-        "version": "0.4"
+        "version": "0.5.1"
       }
     ],
     "exported_symbols": [
       {
         "text": {
           "global": [
-            "__ZN10rpc_server23rpcBackendHasRdmaMarkerERKNSt3__14__fs10filesystem4pathE",
             "__ZN10rpc_server12ServerRunner4stopEv",
             "__ZN10rpc_server13selectDevicesERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE",
             "_bare_register_module_v0",

package/prebuilds/ios-arm64/qvac__ggml-rpc-server.bare.exports

@@ -7,14 +7,13 @@
     ],
     "current_versions": [
       {
-        "version": "0.4"
+        "version": "0.5.1"
       }
     ],
     "exported_symbols": [
       {
         "text": {
           "global": [
-            "__ZN10rpc_server23rpcBackendHasRdmaMarkerERKNSt3__14__fs10filesystem4pathE",
             "__ZN10rpc_server12ServerRunner4stopEv",
             "__ZN10rpc_server13selectDevicesERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE",
             "_bare_register_module_v0",

package/prebuilds/ios-arm64-simulator/qvac__ggml-rpc-server.bare.exports

@@ -7,14 +7,13 @@
     ],
     "current_versions": [
       {
-        "version": "0.4"
+        "version": "0.5.1"
       }
     ],
     "exported_symbols": [
       {
         "text": {
           "global": [
-            "__ZN10rpc_server23rpcBackendHasRdmaMarkerERKNSt3__14__fs10filesystem4pathE",
             "__ZN10rpc_server12ServerRunner4stopEv",
             "__ZN10rpc_server13selectDevicesERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE",
             "_bare_register_module_v0",

package/prebuilds/ios-x64-simulator/qvac__ggml-rpc-server.bare.exports

@@ -7,14 +7,13 @@
     ],
     "current_versions": [
       {
-        "version": "0.4"
+        "version": "0.5.1"
       }
     ],
     "exported_symbols": [
       {
         "text": {
           "global": [
-            "__ZN10rpc_server23rpcBackendHasRdmaMarkerERKNSt3__14__fs10filesystem4pathE",
             "__ZN10rpc_server12ServerRunner4stopEv",
             "__ZN10rpc_server13selectDevicesERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE",
             "_bare_register_module_v0",