@qvac/ggml-rpc-server 0.0.0 → 0.1.1

package/CHANGELOG.md

@@ -0,0 +1,24 @@
+# Changelog
+
+## [0.1.1] - 2026-09-30
+
+### Changed
+
+- `@qvac/fabric` dependency bumped `^0.18.1` -> `^0.19.0`, and the mobile
+  `@qvac/fabric-android-arm64` and `@qvac/fabric-ios` pins `0.18.1` ->
+  `0.19.0`. This carries `qvac-fabric` `10549.4.0` -> `10549.5.0`, whose RPC
+  change matters here: idle RDMA and tensor-dispatch threads now sleep instead
+  of spinning, so an idle server no longer keeps cores busy. A caret on a `0.x`
+  version locks the minor, so `^0.18.1` would not have resolved `0.19.0` on
+  its own. No API change.
+
+## 0.1.0
+
+- Add the managed `ggml-rpc-server` API as an in-process Bare addon backed by
+  the npm `@qvac/fabric` package.
+- Report through `rdmaCapable` whether the installed Fabric RPC backend was
+  built with RDMA, and reject `expectRdma: true` when it was not.
+- Report native start and stop failures as typed errors extending
+  `RpcServerNativeError` (`RpcServerDeviceError`, `RpcServerCacheError`,
+  `RpcServerStartError`, `RpcServerBackendError`, `RpcServerStopError`).
+- Build prebuild artifacts for macOS arm64/x64, Linux arm64/x64, Windows x64, Android arm64, and iOS arm64.

package/LICENSE

@@ -0,0 +1,179 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+Copyright 2026 Tether Data, S.A. de C.V.

package/NOTICE

@@ -0,0 +1,6 @@
+@qvac/ggml-rpc-server
+Copyright (c) Tether
+
+This product includes software developed by third parties under their
+respective licenses. @qvac/ggml-rpc-server itself is licensed under the Apache
+License, Version 2.0.

package/README.md

@@ -0,0 +1,99 @@
+# @qvac/ggml-rpc-server
+
+Managed in-process GGML RPC server for Bare applications.
+
+The package loads the RPC backend supplied by `@qvac/fabric` and starts it
+through a native addon. It does not bundle or launch llama.cpp's
+`ggml-rpc-server` CLI executable.
+
+Prebuild artifacts are produced for macOS arm64/x64, Linux arm64/x64, Windows
+x64, Android arm64, iOS arm64, and the iOS simulator on arm64/x64. Desktop
+prebuild jobs smoke-test the in-process lifecycle path; Android/iOS jobs
+cross-build the same addon for their targets.
+
+```js
+const { startRpcServer } = require('@qvac/ggml-rpc-server')
+
+const server = await startRpcServer({ device: 'Vulkan0' })
+
+try {
+  console.log(server.url)
+  console.log(server.rdmaCapable)
+} finally {
+  await server.stop()
+}
+```
+
+The listener accepts IPv4 addresses; `localhost` is normalized to `127.0.0.1`.
+The default host is `127.0.0.1`, but loopback is not private to the host
+application: other local processes (including apps with local TCP access on
+Android and iOS) can connect. The underlying RPC listener has no authentication
+and serves one client at a time, so another local client can occupy the server
+and block the intended client. Start it only when needed, stop it after use,
+and do not treat loopback binding as an access-control boundary. Non-loopback
+hosts are rejected unless `allowNonLoopbackHost: true` is passed; use them only
+on a trusted/private network with external access controls.
+
+With `cache: true`, the server stores tensor data sent by clients in a local
+cache directory, so any client that can connect can write to that directory.
+Combined with a non-loopback host, that is unauthenticated disk writes from the
+network; enable the cache only for clients you trust.
+
+Like a listening `net.Server`, a running server keeps the process alive until
+`stop()` resolves, so a standalone worker can start it and wait for clients.
+
+Native server output is written to the host application's platform log.
+
+## Errors
+
+Errors are classes that can be matched by `name` or `instanceof`. Invalid
+options throw `RpcServerInvalidHostError`, `RpcServerNonLoopbackHostError`,
+`RangeError` (port) or `TypeError` (threads); a failed port lookup throws
+`RpcServerPortAllocationError`; and `expectRdma: true` on a backend without RDMA
+throws `RpcServerRdmaUnavailableError`. Failures reported by the native server
+extend `RpcServerNativeError`, which carries the native error as `cause` and a
+`code` equal to its `name`:
+
+| Error | When |
+|---|---|
+| `RpcServerDeviceError` | No requested device exists, or no device is available |
+| `RpcServerCacheError` | The cache directory cannot be resolved or created |
+| `RpcServerStartError` | The server cannot be created or bound, or the RPC backend is missing |
+| `RpcServerBackendError` | The Fabric backends directory is invalid or cannot be inspected |
+| `RpcServerStopError` | The server does not stop cleanly |
+
+Anything else from the native addon, such as an out-of-memory failure, is
+rethrown unchanged.
+
+## Linux requirements
+
+The Linux RPC backend in `@qvac/fabric` links `libibverbs.so.1`, so Linux
+hosts must provide it (`libibverbs1` on Debian/Ubuntu) even when connections
+use TCP. Without it, `startRpcServer()` rejects with an error whose `code` is
+`RpcServerStartError`.
+RDMA also requires the provider package for the host's hardware.
+
+## RDMA-capable builds
+
+On Linux (not Android), `rdmaCapable` reports whether the installed
+`@qvac/fabric` RPC backend was built with RDMA. Such a backend negotiates RDMA
+with each client that also supports it and falls back to TCP otherwise, so
+`rdmaCapable: true` does not guarantee that a given connection uses RDMA. Other
+platforms always report `false`.
+
+To fail closed when RDMA is required, pass `expectRdma: true`. Startup rejects
+with `RpcServerRdmaUnavailableError` when the installed backend lacks RDMA:
+
+```js
+const server = await startRpcServer({
+  device: 'Vulkan0',
+  host: '10.10.10.2',
+  expectRdma: true,
+  allowNonLoopbackHost: true
+})
+
+console.log(server.rdmaCapable)
+```
+
+Without `expectRdma`, the server starts either way and reports the backend's
+capability in `rdmaCapable`.

package/binding.js

@@ -0,0 +1,6 @@
+// Pre-load @qvac/fabric so its shared .bare module (the llama.cpp + ggml
+// runtime) is registered with the bare runtime before our addon resolves its
+// DT_NEEDED dependency qvac__fabric@0.bare.
+require("@qvac/fabric");
+
+module.exports = require.addon();

package/index.d.ts

@@ -0,0 +1,68 @@
+export declare const DEFAULT_RPC_SERVER_HOST: string;
+export declare class RpcServerPortAllocationError extends Error {
+    constructor(cause?: unknown);
+}
+export declare class RpcServerNonLoopbackHostError extends Error {
+    constructor(host: string);
+}
+export declare class RpcServerInvalidHostError extends Error {
+    constructor(host: string);
+}
+export declare class RpcServerRdmaUnavailableError extends Error {
+    constructor();
+}
+/**
+ * A failure reported by the native server. The addon raises plain errors with a
+ * `code`; these classes let callers branch on `name` or `instanceof` as they do
+ * for the errors above. `code` equals `name`, and `cause` is the native error.
+ */
+export declare abstract class RpcServerNativeError extends Error {
+    readonly code: string;
+    constructor(name: string, message: string, cause: unknown);
+}
+/** No requested device exists, or no device is available. */
+export declare class RpcServerDeviceError extends RpcServerNativeError {
+    constructor(message: string, cause: unknown);
+}
+/** The RPC cache directory could not be resolved or created. */
+export declare class RpcServerCacheError extends RpcServerNativeError {
+    constructor(message: string, cause: unknown);
+}
+/** The server could not be created or bound, or the RPC backend is missing. */
+export declare class RpcServerStartError extends RpcServerNativeError {
+    constructor(message: string, cause: unknown);
+}
+/** The Fabric backends directory is invalid or could not be inspected. */
+export declare class RpcServerBackendError extends RpcServerNativeError {
+    constructor(message: string, cause: unknown);
+}
+/** The server did not stop cleanly. */
+export declare class RpcServerStopError extends RpcServerNativeError {
+    constructor(message: string, cause: unknown);
+}
+export interface StartRpcServerOptions {
+    readonly device?: string | readonly string[];
+    readonly host?: string;
+    readonly port?: number;
+    readonly cache?: boolean;
+    readonly threads?: number;
+    readonly expectRdma?: boolean;
+    readonly allowNonLoopbackHost?: boolean;
+}
+export interface RpcServer {
+    readonly host: string;
+    readonly port: number;
+    readonly url: string;
+    readonly device?: string;
+    /**
+     * Whether the loaded Fabric RPC backend was built with RDMA. Such a backend
+     * negotiates RDMA with each RDMA-capable client and falls back to TCP otherwise.
+     */
+    readonly rdmaCapable: boolean;
+    stop(): Promise<void>;
+}
+export interface AllocateFreePortOptions {
+    readonly allowNonLoopbackHost?: boolean;
+}
+export declare function allocateFreePort(host?: string, options?: AllocateFreePortOptions): Promise<number>;
+export declare function startRpcServer(options?: StartRpcServerOptions): Promise<RpcServer>;

package/index.js

@@ -0,0 +1,283 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.RpcServerStopError =
+  exports.RpcServerBackendError =
+  exports.RpcServerStartError =
+  exports.RpcServerCacheError =
+  exports.RpcServerDeviceError =
+  exports.RpcServerNativeError =
+  exports.RpcServerRdmaUnavailableError =
+  exports.RpcServerInvalidHostError =
+  exports.RpcServerNonLoopbackHostError =
+  exports.RpcServerPortAllocationError =
+  exports.DEFAULT_RPC_SERVER_HOST =
+    void 0;
+exports.allocateFreePort = allocateFreePort;
+exports.startRpcServer = startRpcServer;
+/* eslint-disable @typescript-eslint/no-require-imports -- Bare modules and native bindings expose CommonJS export shapes. */
+const net = require("bare-net");
+const path = require("bare-path");
+const fabricBackends = require("@qvac/fabric/backends");
+const binding = require("./binding");
+/* eslint-enable @typescript-eslint/no-require-imports */
+exports.DEFAULT_RPC_SERVER_HOST = "127.0.0.1";
+const TRUSTED_LAN_WARNING_CODE = "QVAC_GGML_RPC_SERVER_TRUSTED_LAN";
+// Keep native handles alive until an explicit stop finishes. Otherwise their
+// finalizer can synchronously stop and join a live server during garbage collection.
+const activeServerHandles = new Set();
+// Fabric never unloads its RPC module, so its RDMA build cannot change within a
+// process. Cache the native check, which reads the module from disk.
+const rdmaSupportByBackendsDir = new Map();
+class RpcServerPortAllocationError extends Error {
+  constructor(cause) {
+    super("Failed to allocate a free port for ggml-rpc-server", { cause });
+    this.name = "RpcServerPortAllocationError";
+  }
+}
+exports.RpcServerPortAllocationError = RpcServerPortAllocationError;
+class RpcServerNonLoopbackHostError extends Error {
+  constructor(host) {
+    super(
+      `${host} is not a loopback host; pass allowNonLoopbackHost: true to bind it on a trusted network`,
+    );
+    this.name = "RpcServerNonLoopbackHostError";
+  }
+}
+exports.RpcServerNonLoopbackHostError = RpcServerNonLoopbackHostError;
+class RpcServerInvalidHostError extends Error {
+  constructor(host) {
+    super(`ggml-rpc-server requires an IPv4 address or localhost: ${host}`);
+    this.name = "RpcServerInvalidHostError";
+  }
+}
+exports.RpcServerInvalidHostError = RpcServerInvalidHostError;
+class RpcServerRdmaUnavailableError extends Error {
+  constructor() {
+    super("RDMA is not available in the installed @qvac/fabric RPC backend");
+    this.name = "RpcServerRdmaUnavailableError";
+  }
+}
+exports.RpcServerRdmaUnavailableError = RpcServerRdmaUnavailableError;
+/**
+ * A failure reported by the native server. The addon raises plain errors with a
+ * `code`; these classes let callers branch on `name` or `instanceof` as they do
+ * for the errors above. `code` equals `name`, and `cause` is the native error.
+ */
+class RpcServerNativeError extends Error {
+  code;
+  // Names are literals, not new.target.name, so they survive minification.
+  constructor(name, message, cause) {
+    super(message, { cause });
+    this.name = name;
+    this.code = name;
+  }
+}
+exports.RpcServerNativeError = RpcServerNativeError;
+/** No requested device exists, or no device is available. */
+class RpcServerDeviceError extends RpcServerNativeError {
+  constructor(message, cause) {
+    super("RpcServerDeviceError", message, cause);
+  }
+}
+exports.RpcServerDeviceError = RpcServerDeviceError;
+/** The RPC cache directory could not be resolved or created. */
+class RpcServerCacheError extends RpcServerNativeError {
+  constructor(message, cause) {
+    super("RpcServerCacheError", message, cause);
+  }
+}
+exports.RpcServerCacheError = RpcServerCacheError;
+/** The server could not be created or bound, or the RPC backend is missing. */
+class RpcServerStartError extends RpcServerNativeError {
+  constructor(message, cause) {
+    super("RpcServerStartError", message, cause);
+  }
+}
+exports.RpcServerStartError = RpcServerStartError;
+/** The Fabric backends directory is invalid or could not be inspected. */
+class RpcServerBackendError extends RpcServerNativeError {
+  constructor(message, cause) {
+    super("RpcServerBackendError", message, cause);
+  }
+}
+exports.RpcServerBackendError = RpcServerBackendError;
+/** The server did not stop cleanly. */
+class RpcServerStopError extends RpcServerNativeError {
+  constructor(message, cause) {
+    super("RpcServerStopError", message, cause);
+  }
+}
+exports.RpcServerStopError = RpcServerStopError;
+const nativeErrorClasses = new Map([
+  ["RpcServerDeviceError", RpcServerDeviceError],
+  ["RpcServerCacheError", RpcServerCacheError],
+  ["RpcServerStartError", RpcServerStartError],
+  ["RpcServerBackendError", RpcServerBackendError],
+  ["RpcServerStopError", RpcServerStopError],
+]);
+function toTypedError(error) {
+  const code = error?.code;
+  const ErrorClass =
+    typeof code === "string" ? nativeErrorClasses.get(code) : undefined;
+  return ErrorClass === undefined
+    ? error
+    : new ErrorClass(error.message, error);
+}
+async function callNative(call) {
+  try {
+    return await call();
+  } catch (error) {
+    throw toTypedError(error);
+  }
+}
+function isLoopbackHost(host) {
+  const parts = host.split(".");
+  return (
+    parts.length === 4 &&
+    parts[0] === "127" &&
+    parts.every((part) => /^\d{1,3}$/.test(part) && Number(part) <= 255)
+  );
+}
+function isIpv4Host(host) {
+  const parts = host.split(".");
+  return (
+    parts.length === 4 &&
+    parts.every((part) => /^\d{1,3}$/.test(part) && Number(part) <= 255)
+  );
+}
+function normalizeHost(host) {
+  return host === "localhost" ? exports.DEFAULT_RPC_SERVER_HOST : host;
+}
+function assertSupportedHost(host) {
+  if (!isIpv4Host(host)) {
+    throw new RpcServerInvalidHostError(host);
+  }
+}
+function assertLoopbackHost(host, allowNonLoopbackHost = false) {
+  if (!allowNonLoopbackHost && !isLoopbackHost(host)) {
+    throw new RpcServerNonLoopbackHostError(host);
+  }
+}
+function warnForTrustedLanHost(host, allowNonLoopbackHost = false) {
+  if (!allowNonLoopbackHost || isLoopbackHost(host)) return;
+  console.warn(
+    `[${TRUSTED_LAN_WARNING_CODE}] ggml-rpc-server is binding to non-loopback host ${host}. ` +
+      "The ggml RPC transport has no authentication or encryption; use this only on a trusted private network with external access controls.",
+  );
+}
+function normalizeDevice(device) {
+  if (device === undefined) return undefined;
+  // The native side splits on ',' or '/' and matches names exactly, so trim
+  // each name: 'Vulkan0, CPU' would otherwise look up ' CPU'.
+  const names = typeof device === "string" ? device.split(/[,/]/) : device;
+  const trimmed = names.map((name) => name.trim());
+  // A blank-only value must not collapse to '', which means "default devices".
+  // Pass it through untouched so the native side rejects it as unknown.
+  if (
+    trimmed.every((name) => name === "") &&
+    names.some((name) => name !== "")
+  ) {
+    return names.join(",");
+  }
+  return trimmed.join(",");
+}
+function validatePort(port) {
+  if (!Number.isSafeInteger(port) || port <= 0 || port > 65535) {
+    throw new RangeError("port must be an integer between 1 and 65535");
+  }
+}
+function validateThreads(threads) {
+  if (
+    threads !== undefined &&
+    (!Number.isSafeInteger(threads) || threads <= 0)
+  ) {
+    throw new TypeError("threads must be a positive integer");
+  }
+}
+function rpcBackendSupportsRdma(backendsDir) {
+  let supported = rdmaSupportByBackendsDir.get(backendsDir);
+  if (supported === undefined) {
+    try {
+      supported = binding.rpcBackendSupportsRdma({ backendsDir });
+    } catch (error) {
+      throw toTypedError(error);
+    }
+    rdmaSupportByBackendsDir.set(backendsDir, supported);
+  }
+  return supported;
+}
+function allocateFreePort(
+  host = exports.DEFAULT_RPC_SERVER_HOST,
+  options = {},
+) {
+  const bindHost = normalizeHost(host);
+  assertSupportedHost(bindHost);
+  assertLoopbackHost(bindHost, options.allowNonLoopbackHost);
+  return new Promise((resolve, reject) => {
+    const server = net.createServer();
+    server.once("error", (error) =>
+      reject(new RpcServerPortAllocationError(error)),
+    );
+    server.listen(0, bindHost, () => {
+      const address = server.address();
+      if (address === null || typeof address === "string") {
+        server.close(() => reject(new RpcServerPortAllocationError()));
+        return;
+      }
+      server.close(() => resolve(address.port));
+    });
+  });
+}
+async function startRpcServer(options = {}) {
+  const host = normalizeHost(options.host ?? exports.DEFAULT_RPC_SERVER_HOST);
+  assertSupportedHost(host);
+  assertLoopbackHost(host, options.allowNonLoopbackHost);
+  warnForTrustedLanHost(host, options.allowNonLoopbackHost);
+  validateThreads(options.threads);
+  // Packed mobile bundles do not retain a resolvable node_modules tree. Their
+  // packagers stage Fabric's backends beside this addon instead.
+  const backendsDir =
+    fabricBackends.resolveBackendsDir() ?? path.join(__dirname, "prebuilds");
+  const rdmaCapable = rpcBackendSupportsRdma(backendsDir);
+  if (options.expectRdma === true && !rdmaCapable) {
+    throw new RpcServerRdmaUnavailableError();
+  }
+  const port =
+    options.port ??
+    (await allocateFreePort(host, {
+      allowNonLoopbackHost: options.allowNonLoopbackHost,
+    }));
+  validatePort(port);
+  const device = normalizeDevice(options.device);
+  const handle = await callNative(() =>
+    binding.startServer({
+      endpoint: `${host}:${port}`,
+      device,
+      cache: options.cache ?? false,
+      threads: options.threads,
+      backendsDir,
+    }),
+  );
+  activeServerHandles.add(handle);
+  let stopPromise;
+  return {
+    host,
+    port,
+    url: `${host}:${port}`,
+    device,
+    rdmaCapable,
+    stop: () => {
+      stopPromise ??= callNative(() => binding.stopServer(handle)).then(
+        () => {
+          activeServerHandles.delete(handle);
+        },
+        (error) => {
+          // Keep the handle pinned so the caller can retry a failed stop.
+          stopPromise = undefined;
+          throw error;
+        },
+      );
+      return stopPromise;
+    },
+  };
+}

package/package.json

@@ -1 +1,75 @@
-{"name":"@qvac/ggml-rpc-server","version":"0.0.0"}
+{
+  "name": "@qvac/ggml-rpc-server",
+  "version": "0.1.1",
+  "description": "Managed ggml RPC server for QVAC LLM inference",
+  "addon": true,
+  "scripts": {
+    "build": "npm run build:ts && npm run build:native",
+    "build:native": "bare-make generate && bare-make build && bare-make install",
+    "build:ts": "tsc -p tsconfig.build.json",
+    "check:generated": "node scripts/check-generated.mjs",
+    "typecheck": "tsc --noEmit -p tsconfig.json",
+    "lint": "npm run lint:js && npm run lint:ts && npm run test:types",
+    "lint:js": "prettier --check \"**/*.{js,cjs,mjs}\" && lunte",
+    "lint:ts": "eslint \"src/**/*.ts\" --max-warnings=0",
+    "lint:ts:fix": "eslint \"src/**/*.ts\" --fix",
+    "lint-cpp": "clang-tidy -p build addon/binding.cpp",
+    "format": "prettier --write \"**/*.{js,cjs,mjs}\"",
+    "test": "npm run build:ts && npm run test:node",
+    "test:node": "node --test test/node/*.test.cjs",
+    "test:unit": "npm run test:node",
+    "test:types": "npm run typecheck && npm run check:generated",
+    "test:dts": "npm run test:types"
+  },
+  "files": [
+    "index.js",
+    "index.d.ts",
+    "binding.js",
+    "prebuilds",
+    "CHANGELOG.md",
+    "LICENSE",
+    "NOTICE",
+    "README.md"
+  ],
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/tetherto/qvac.git",
+    "directory": "packages/ggml-rpc-server"
+  },
+  "author": "Tether",
+  "license": "Apache-2.0",
+  "bugs": "https://github.com/tetherto/qvac/issues",
+  "homepage": "https://github.com/tetherto/qvac/tree/main/packages/ggml-rpc-server#readme",
+  "devDependencies": {
+    "@qvac/fabric-android-arm64": "0.19.0",
+    "@qvac/fabric-ios": "0.19.0",
+    "@types/node": "^24.2.1",
+    "cmake-bare": "^1.7.5",
+    "cmake-npm": "^1.1.0",
+    "cmake-vcpkg": "^1.1.0",
+    "eslint": "^9.39.4",
+    "eslint-import-resolver-typescript": "^4.4.5",
+    "eslint-plugin-import": "^2.32.0",
+    "lunte": "^1.8.4",
+    "prettier": "^3.9.6",
+    "prettier-config-holepunch": "^2.0.0",
+    "typescript": "^5.9.2",
+    "typescript-eslint": "^8.63.0"
+  },
+  "dependencies": {
+    "@qvac/fabric": "^0.19.0",
+    "bare-net": "^2.3.2",
+    "bare-path": "^3.0.0"
+  },
+  "engines": {
+    "bare": ">=1.24.0"
+  },
+  "exports": {
+    "./package": "./package.json",
+    ".": {
+      "types": "./index.d.ts",
+      "default": "./index.js"
+    }
+  },
+  "types": "index.d.ts"
+}

package/prebuilds/darwin-arm64/qvac__ggml-rpc-server.bare.exports

@@ -0,0 +1,43 @@
+{
+  "main_library": {
+    "compatibility_versions": [
+      {
+        "version": "0"
+      }
+    ],
+    "current_versions": [
+      {
+        "version": "0.1.1"
+      }
+    ],
+    "exported_symbols": [
+      {
+        "text": {
+          "global": [
+            "_bare_register_module_v0",
+            "_bare_get_module_name_v0"
+          ]
+        }
+      }
+    ],
+    "flags": [
+      {
+        "attributes": [
+          "not_app_extension_safe"
+        ]
+      }
+    ],
+    "install_names": [
+      {
+        "name": "qvac__ggml-rpc-server@0.bare"
+      }
+    ],
+    "target_info": [
+      {
+        "min_deployment": "13",
+        "target": "arm64-macos"
+      }
+    ]
+  },
+  "tapi_tbd_version": 5
+}

package/prebuilds/darwin-x64/qvac__ggml-rpc-server.bare.exports

@@ -0,0 +1,43 @@
+{
+  "main_library": {
+    "compatibility_versions": [
+      {
+        "version": "0"
+      }
+    ],
+    "current_versions": [
+      {
+        "version": "0.1.1"
+      }
+    ],
+    "exported_symbols": [
+      {
+        "text": {
+          "global": [
+            "_bare_register_module_v0",
+            "_bare_get_module_name_v0"
+          ]
+        }
+      }
+    ],
+    "flags": [
+      {
+        "attributes": [
+          "not_app_extension_safe"
+        ]
+      }
+    ],
+    "install_names": [
+      {
+        "name": "qvac__ggml-rpc-server@0.bare"
+      }
+    ],
+    "target_info": [
+      {
+        "min_deployment": "13",
+        "target": "x86_64-macos"
+      }
+    ]
+  },
+  "tapi_tbd_version": 5
+}

package/prebuilds/ios-arm64/qvac__ggml-rpc-server.bare.exports

@@ -0,0 +1,43 @@
+{
+  "main_library": {
+    "compatibility_versions": [
+      {
+        "version": "0"
+      }
+    ],
+    "current_versions": [
+      {
+        "version": "0.1.1"
+      }
+    ],
+    "exported_symbols": [
+      {
+        "text": {
+          "global": [
+            "_bare_register_module_v0",
+            "_bare_get_module_name_v0"
+          ]
+        }
+      }
+    ],
+    "flags": [
+      {
+        "attributes": [
+          "not_app_extension_safe"
+        ]
+      }
+    ],
+    "install_names": [
+      {
+        "name": "qvac__ggml-rpc-server@0.bare"
+      }
+    ],
+    "target_info": [
+      {
+        "min_deployment": "14",
+        "target": "arm64-ios"
+      }
+    ]
+  },
+  "tapi_tbd_version": 5
+}

package/prebuilds/ios-arm64-simulator/qvac__ggml-rpc-server.bare.exports

@@ -0,0 +1,43 @@
+{
+  "main_library": {
+    "compatibility_versions": [
+      {
+        "version": "0"
+      }
+    ],
+    "current_versions": [
+      {
+        "version": "0.1.1"
+      }
+    ],
+    "exported_symbols": [
+      {
+        "text": {
+          "global": [
+            "_bare_register_module_v0",
+            "_bare_get_module_name_v0"
+          ]
+        }
+      }
+    ],
+    "flags": [
+      {
+        "attributes": [
+          "not_app_extension_safe"
+        ]
+      }
+    ],
+    "install_names": [
+      {
+        "name": "qvac__ggml-rpc-server@0.bare"
+      }
+    ],
+    "target_info": [
+      {
+        "min_deployment": "14",
+        "target": "arm64-ios-simulator"
+      }
+    ]
+  },
+  "tapi_tbd_version": 5
+}

package/prebuilds/ios-x64-simulator/qvac__ggml-rpc-server.bare.exports

@@ -0,0 +1,43 @@
+{
+  "main_library": {
+    "compatibility_versions": [
+      {
+        "version": "0"
+      }
+    ],
+    "current_versions": [
+      {
+        "version": "0.1.1"
+      }
+    ],
+    "exported_symbols": [
+      {
+        "text": {
+          "global": [
+            "_bare_register_module_v0",
+            "_bare_get_module_name_v0"
+          ]
+        }
+      }
+    ],
+    "flags": [
+      {
+        "attributes": [
+          "not_app_extension_safe"
+        ]
+      }
+    ],
+    "install_names": [
+      {
+        "name": "qvac__ggml-rpc-server@0.bare"
+      }
+    ],
+    "target_info": [
+      {
+        "min_deployment": "14",
+        "target": "x86_64-ios-simulator"
+      }
+    ]
+  },
+  "tapi_tbd_version": 5
+}