@ignex/nova 0.1.1

package/docs/wire-format.md

@@ -0,0 +1,170 @@
+# Wire format
+
+This document pins the on-the-wire contract so anyone can build an independent
+client from the generated FlatBuffers schema — no Bun, no Rust, no FFI needed.
+
+## Frame
+
+Every WebSocket **binary** frame is:
+
+```
+┌──────────┬───────────────────┬──────────────────────────────────┐
+│ version  │ event_id          │ size-prefixed FlatBuffer         │
+│ 1 byte   │ u32 (LE)          │ (flatc: 4-byte size prefix + buf)│
+└──────────┴───────────────────┴──────────────────────────────────┘
+    offset 0      1..5                 5..
+```
+
+- `version` = `WIRE_VERSION` (currently `1`). A peer MUST drop frames whose
+  version it doesn't recognize (the decoders return `null`).
+- `event_id` = **FNV-1a 32-bit hash of the event name** (e.g.
+  `fnv1a32("quote")`). Stable across schema reordering; collisions are rejected
+  at generate time. The id→name table is emitted in `src/generated/registry.ts`.
+- The payload is a **size-prefixed FlatBuffer** (built by flatc, both Rust
+  builders and the TS object API) with the root table for that event id.
+
+There is **no per-frame checksum and no authentication** — the envelope id byte
+is trusted. Integrity and AuthN are the application's job (see
+[Security](#security)).
+
+## Event ids
+
+Event ids are stable hashes, so adding events, reordering the registry, or
+adding fields does not renumber the wire format. Compute them with FNV-1a 32:
+
+```
+h = 2166136261
+for each byte b of the UTF-8 event name:
+    h ^= b
+    h = (h * 16777619) mod 2^32
+```
+
+## Control frames
+
+A fixed set of transport-internal events share the SAME frame format and the
+SAME codegen, but are routed internally (never delivered to app handlers):
+
+| name | direction | payload |
+| --- | --- | --- |
+| `hello` | both | `{ version, caps: string[], lastSeq }` — sent on connect; version mismatch → close(1002) |
+| `welcome` | server→client | `{ clientId, groups: string[] }` — identity assigned to this connection (auth metadata or UUID) + its server-side groups |
+| `subscribe` | client→server | `{ topic }` — join a room (server replies with replay history) |
+| `unsubscribe` | client→server | `{ topic }` |
+| `joinGroup` | client→server | `{ group }` — join a server-side group |
+| `leaveGroup` | client→server | `{ group }` |
+| `snapshotRequest` | client→server | `{ topic }` — reserved for future replay control |
+| `ping` | client→server | `{ ts }` — heartbeat |
+| `pong` | server→client | `{ ts }` |
+
+Control ids are `fnv1a32` of the control name, dispatched via
+`isControlId(id)` before user handlers.
+
+## Rooms / topics
+
+- `subscribe`/`unsubscribe` control frames manage server-side room membership.
+- The server keeps an optional per-topic history (`replay: { historySize }`).
+  On join it sends the recorded frames **oldest → newest** as a last-value
+  snapshot, then live traffic follows. History is bounded (`historySize`),
+  older frames are dropped.
+- `publish` = global broadcast; `publishToTopic` = room only.
+
+## Targeted delivery / groups
+
+Every connection gets a stable **client id**: an explicit id from the
+`authenticate` hook (`{ id, groups, meta }`) or an auto-generated UUID. The
+server assigns it during the upgrade and delivers it in the `welcome` control
+frame, so the client knows its own identity (`client.clientId`).
+
+- `publishToClient(id, name, payload)` — send to ONE client by id (not bridged
+to NATS).
+- **Groups** are a server-side targeting dimension (no replay):
+  - seeded from `authenticate` metadata (`{ groups: [...] }`),
+  - managed programmatically (`joinGroup(id, group)` / `leaveGroup(id, group)`),
+  - or joined by the client via the `joinGroup`/`leaveGroup` control frames.
+- `publishToGroup(group, name, payload)` fans out to every member.
+- Introspection: `getClient(id)` / `getClients()` / `groupMembers(group)` /
+  `groups()` — also exposed as `GET /clients` on the server.
+
+Rooms are client-joinable *subscriptions with optional replay*; groups are
+server-side *targeting sets without replay*. Both are independent dimensions.
+
+## NATS bridge
+
+When `createServer({ nats: {...} })` is set, every broadcast / topic / group
+publish is ALSO published to NATS as the **same wire frame** the WS clients
+receive (encoded once, copied for NATS). Best-effort: if NATS is down the
+frame is dropped and counted in `bridgeErrors` — the WS hot path never blocks.
+
+| publish API | NATS subject |
+| --- | --- |
+| `publish(name, …)` | `{prefix}.broadcast.{name}` |
+| `publishToTopic(topic, name, …)` | `{prefix}.topic.{topic}.{name}` |
+| `publishToGroup(group, name, …)` | `{prefix}.group.{group}.{name}` |
+
+`{prefix}` defaults to `ignex`. External apps push events INTO the hub by
+publishing on `{prefix}.inbound.>` (default; configurable via
+`inboundSubjects`); the server decodes them and forwards to all clients
+(allowlisted by `inboundEvents`, control frames dropped, and never re-bridged —
+no loops). `publishToClient` (single-socket) is intentionally not bridged.
+
+Bridge state is observable via `server.getMetrics()`
+(`natsStatus`, `bridged`, `bridgedBytes`, `bridgeErrors`, `bridgeInbound`).
+
+### Decoding bridged frames (external consumers)
+
+`bun run generate` emits `src/generated/wire-registry.json` — a machine-readable
+`{ version, events: { name: id } }` map of the FNV-1a ids. A consumer in any
+language:
+
+1. reads `version` (byte 0) and `event_id` (bytes 1..5) from the frame,
+2. maps `event_id` → name via `wire-registry.json`,
+3. decodes the size-prefixed FlatBuffer from `frame[5..]` with a flatc build of
+   `src/generated/fbs/backend.fbs`.
+
+See `examples/nats-consumer.ts` for a working reference.
+
+## Heartbeat
+
+Clients send `ping` every `heartbeatMs` (default 15000). The server replies
+`pong`. Bun's `idleTimeout` closes connections with no traffic, so pings keep
+long-idle sockets alive; a client that misses `heartbeatMisses` pongs force-
+closes and reconnects.
+
+## Sequence / replay
+
+The current model replays **recent history on subscribe** (bounded ring per
+topic, last-value snapshot). Full gap-based replay (per-frame sequence numbers
+in the envelope + `hello.lastSeq` negotiation) is a documented future extension;
+the envelope reserves no space for a per-frame seq today.
+
+## Security
+
+- **Trust model**: the envelope has no checksum/AuthN. A hostile peer can lie
+  about the event id; the decoder is fuzz-safe (never throws / OOB) but will
+  happily decode the payload as the claimed type. Add integrity at the
+  application layer if the wire crosses an untrusted boundary.
+- Server-side guards (all optional): `authenticate(req)` async hook, origin
+  allowlist, bearer `token`, `maxConnections`, `maxMessageSize`.
+- Slow consumers: `backpressure` policy (`drop-oldest`/`drop-newest`/
+  `disconnect`) bounds per-socket buffering; without it a hot publish loop can
+  balloon memory.
+- Injection: unknown object keys and prototype-pollution keys are dropped on
+  both encode paths; embedded NULs route to the JSON path so they round-trip
+  exactly (never silently truncated).
+
+## Building an independent client
+
+1. `bun run generate` produces `generated/fbs/backend.fbs` — the FlatBuffers
+   schema for every table (app + control).
+2. Compile that `.fbs` with flatc for your language: `flatc --python
+   --gen-object-api ...`, `--cpp`, `--go`, etc.
+3. Read `version` (byte 0) and `event_id` (bytes 1..5) from each binary frame.
+   Compute ids with FNV-1a 32 over the name (see above) or read the emitted
+   `src/generated/registry.ts` table.
+4. `flatbuffers.ByteBuffer(frame[5..])` + `getSizePrefixedRootAs<T>` (or the
+   language equivalent) gives you the payload.
+5. Implement the control frames (at minimum `hello` + `ping`/`pong`) to be a
+   good citizen; `subscribe` for rooms.
+
+Wire compatibility is guaranteed by flatc: the Rust builders (server) and the
+generated TS object API (browser client) both derive from the same `.fbs`.

package/index.ts

@@ -0,0 +1,61 @@
+/**
+ * ignex-nova — public package root.
+ *
+ * Re-exports the full typed pub/sub API (server + client + NATS bridge +
+ * schema types) so a single `import ... from "ignex-nova"` works in Bun
+ * projects.
+ *
+ * For leaner / target-specific imports use the subpath entrypoints — each
+ * resolves to its own source file and tree-shakes independently:
+ *
+ *   import { createServer } from "ignex-nova/server";   // Bun-only (Rust FFI)
+ *   import { createClient } from "ignex-nova/client";   // browser + Bun
+ *   import { createNatsBridge } from "ignex-nova/nats"; // standalone bridge
+ *
+ * Note: the root entry also pulls in the Bun-only server + FFI path, so
+ * browser bundles should import "ignex-nova/client" instead.
+ *
+ * The README (and docs/publishing.md) covers consuming this from an npm
+ * package: `bun add ignex-nova`, then use the subpaths above.
+ */
+export { createServer } from "./public/server";
+export type {
+  IgnServer,
+  ClientInfo,
+  IgnServerOptions,
+  IgnBackpressureOptions,
+  BackpressurePolicy,
+  WsData,
+  ClientMeta,
+  AuthResult,
+  MetricsSnapshot,
+  Int64GuardMode,
+} from "./public/server";
+
+export { createClient } from "./public/client";
+export type {
+  IgnClient,
+  IgnClientOptions,
+  IgnReconnectOptions,
+  ClientStatus,
+} from "./public/client";
+
+export { createNatsBridge, createSubjectBuilder } from "./public/nats";
+export type {
+  NatsBridge,
+  NatsBridgeOptions,
+  NatsBridgeStats,
+  NatsBridgeStatus,
+  NatsTransport,
+  SubjectBuilder,
+} from "./public/nats";
+
+// Plain-object payload types consumers type against (e.g. `Events["quote"]`).
+// Import as types only — the runtime `events` registry is transport-internal.
+export type {
+  Events,
+  EventName,
+  AnyEventName,
+  ControlEvents,
+  ControlEventName,
+} from "./src/schema";

package/package.json

@@ -0,0 +1,89 @@
+{
+  "name": "@ignex/nova",
+  "version": "0.1.1",
+  "type": "module",
+  "description": "TypeBox-driven FlatBuffer transport: Rust FFI serializer + Bun WebSocket + typed pub/sub API for server and FE.",
+  "license": "MIT",
+  "author": "ignex-nova contributors",
+  "keywords": [
+    "flatbuffers",
+    "websocket",
+    "pubsub",
+    "realtime",
+    "typebox",
+    "bun",
+    "ffi",
+    "rust",
+    "nats",
+    "typed"
+  ],
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/ignex-nova/ignex-nova.git"
+  },
+  "homepage": "https://github.com/ignex-nova/ignex-nova#readme",
+  "bugs": {
+    "url": "https://github.com/ignex-nova/ignex-nova/issues"
+  },
+  "sideEffects": false,
+  "engines": {
+    "bun": ">=1.4"
+  },
+  "files": [
+    "index.ts",
+    "public",
+    "src",
+    "rust",
+    "prebuilds",
+    "docs",
+    "README.md",
+    "LICENSE"
+  ],
+  "main": "./index.ts",
+  "module": "./index.ts",
+  "types": "./index.ts",
+  "browser": "./public/client.ts",
+  "exports": {
+    ".": "./index.ts",
+    "./server": "./public/server.ts",
+    "./client": "./public/client.ts",
+    "./nats": "./public/nats.ts",
+    "./package.json": "./package.json"
+  },
+  "publishConfig": {
+    "access": "public"
+  },
+  "scripts": {
+    "generate": "bun scripts/generate.ts",
+    "build:rust": "cargo build --release --manifest-path rust/Cargo.toml",
+    "build:client": "bun build ./client/main.ts --outdir ./client-dist --target=browser --splitting",
+    "build:dist": "bun build ./index.ts ./public/server.ts ./public/client.ts ./public/nats.ts --outdir ./dist --target=bun --splitting && bunx tsc --emitDeclarationOnly -p tsconfig.dist.json",
+    "build": "bun run generate && bun run build:rust && bun run build:client",
+    "prebuild": "bun scripts/build-prebuild.ts",
+    "test": "bun test",
+    "lint": "bunx oxlint",
+    "typecheck": "bunx tsc --noEmit -p tsconfig.json",
+    "verify": "bun run typecheck && bun run lint && bun test",
+    "pack:check": "bun scripts/check-pack.ts",
+    "serve": "bun run src/server.ts",
+    "bench:serialize": "bun run bench/serialize.ts",
+    "bench:throughput": "bun run bench/throughput.ts",
+    "release": "bun scripts/release.ts",
+    "release:dry": "bun scripts/release.ts --dry-run",
+    "prepublishOnly": "bun run generate && bun run verify",
+    "prepack": "bun run prebuild"
+  },
+  "dependencies": {
+    "@sinclair/typebox": "^0.34.0",
+    "flatbuffers": "^25.9.23",
+    "nats": "^2.29.3"
+  },
+  "devDependencies": {
+    "@types/bun": "latest",
+    "lefthook": "^2.1.10",
+    "oxlint": "^1.78.0"
+  },
+  "peerDependencies": {
+    "typescript": "^7"
+  }
+}

package/public/client.ts

@@ -0,0 +1,23 @@
+/**
+ * Public client API — thin re-export shim (keeps the npm entrypoint + `dist`
+ * build stable). The implementation lives in the functional modules under
+ * `src/core/`; this file just exposes the public surface.
+ *
+ *   import { createClient } from "ignex-nova/client";
+ *
+ *   const client = createClient("ws://localhost:3000/ws", { reconnect: true });
+ *   client.on("quote", (q) => { /* q: Events["quote"] — a plain object *\/ });
+ *   client.subscribe("equities");   // server-side room membership (+ last-value replay)
+ *   client.send("chat", {...});     // typed client→server (server must allow it)
+ *   client.connect();
+ *
+ * Works in the browser (bundle with `bun build --target=browser`) AND in Bun.
+ * Outgoing frames are encoded by the generated PURE-JS encoder — no Rust FFI
+ * needed, so the browser can send too.
+ */
+export { createClient, type IgnClient } from "../src/core/client";
+export type {
+  IgnClientOptions,
+  IgnReconnectOptions,
+  ClientStatus,
+} from "../src/core/client-state";

package/public/nats.ts

@@ -0,0 +1,19 @@
+/**
+ * Public NATS bridge API — standalone entrypoint (`ignex-nova/nats`).
+ *
+ *   import { createNatsBridge } from "ignex-nova/nats";
+ *   const bridge = createNatsBridge({ servers: ["nats://localhost:4222"] });
+ *   bridge.publish("ignex.broadcast.quote", frame);   // frame = wire bytes
+ *
+ * Most apps don't need this directly — pass `nats` to `createServer` and the
+ * server bridges broadcast / topic / group publishes automatically.
+ */
+export { createNatsBridge } from "../src/bridge/nats";
+export type {
+  NatsBridgeOptions,
+  NatsBridge,
+  NatsBridgeStats,
+  NatsBridgeStatus,
+  NatsTransport,
+} from "../src/bridge/nats";
+export { createSubjectBuilder, type SubjectBuilder } from "../src/bridge/subjects";

package/public/server.ts

@@ -0,0 +1,35 @@
+/**
+ * Public server API — thin re-export shim (keeps the npm entrypoint + `dist`
+ * build stable). The implementation lives in the functional modules under
+ * `src/core/`; this file just exposes the public surface.
+ *
+ *   import { createServer } from "ignex-nova/server";
+ *
+ *   const server = createServer({ port: 3000 });
+ *   server.publish("quote", { symbol: "AAPL", bid: 180.1, ask: 180.2, ... });
+ *   server.publishTo(ws, "trade", { ... });
+ *   server.join("equities", ws); server.publishToTopic("equities", "quote", {...});
+ *
+ * Bun-only (bun:ffi + Bun.serve).
+ */
+export { createServer, type IgnServer, type ClientInfo } from "../src/core/server";
+export type {
+  IgnServerOptions,
+  IgnBackpressureOptions,
+  BackpressurePolicy,
+  WsData,
+  ClientMeta,
+  AuthResult,
+} from "../src/core/state";
+export type { MetricsSnapshot } from "../src/core/metrics";
+export type { Int64GuardMode } from "../src/core/int64-guard";
+// NATS bridge — re-exported so `nats` options on `createServer` are typed
+// without a separate import (a standalone entrypoint is `ignex-nova/nats`).
+export { createNatsBridge } from "../src/bridge/nats";
+export type {
+  NatsBridgeOptions,
+  NatsBridge,
+  NatsBridgeStats,
+  NatsBridgeStatus,
+  NatsTransport,
+} from "../src/bridge/nats";

package/rust/Cargo.toml

@@ -0,0 +1,19 @@
+[package]
+name = "ignex-nova-ffi"
+version = "0.1.0"
+edition = "2021"
+
+[lib]
+name = "ignex_ffi"
+crate-type = ["cdylib", "rlib"]
+path = "src/lib.rs"
+
+[dependencies]
+flatbuffers = "25"
+serde = { version = "1", features = ["derive"] }
+serde_json = "1"
+
+[profile.release]
+opt-level = 3
+lto = true
+codegen-units = 1

package/rust/src/ffi.rs

@@ -0,0 +1,135 @@
+//! C-ABI surface for Bun `dlopen`. Conventions follow the castrum FFI guide:
+//!   - `panic_guard` every call (a panic unwinding through `extern "C"` kills Bun)
+//!   - null-check every pointer
+//!   - `cstring` ARG → `CStr::from_ptr` (Bun's engine transcodes the JS string;
+//!     zero JS-side text encoding — the "cstring zero text encoding" pattern)
+//!   - needed-size convention: `0` = error, `w > out_cap` = exact size required
+//!   - `thread_local` scratch only (no global `static mut`)
+//!
+//! Frame envelope (written by both `fb_serialize` and the generated direct
+//! `fb_*_serialize` exports):
+//!   `[WIRE_VERSION:1][event_id:u32 LE][size-prefixed FlatBuffer]`
+//!
+//! Bun binding (`src/native/ffi.ts`):
+//!   fb_serialize:     { args: ['u32','cstring','buffer','buffer_length'], returns: 'u64_fast' }
+//!   fb_probe:         { args: [], returns: 'u32' }
+//!   fb_wire_version:  { args: [], returns: 'u32' }
+use std::ffi::CStr;
+use std::os::raw::c_char;
+
+use crate::transcode::generated::{self, WIRE_HEADER_LEN, WIRE_VERSION};
+
+/// Bind-time self-test / capability probe magic ("IGNX").
+pub const FB_PROBE_MAGIC: u32 = 0x4947_4e58;
+
+/// Capability probe used by the Bun bind-time self-test.
+#[no_mangle]
+pub unsafe extern "C" fn fb_probe() -> u32 {
+    FB_PROBE_MAGIC
+}
+
+/// Wire-format version. Checked against the TS `WIRE_VERSION` at bind time so a
+/// stale cdylib (built from an older schema/envelope) fails loudly instead of
+/// silently producing frames the client can't decode.
+#[no_mangle]
+pub extern "C" fn fb_wire_version() -> u32 {
+    WIRE_VERSION as u32
+}
+
+// ── Diagnostic C-ABI probes (bench-only, NOT in the ffi.ts dlopen map) ─────
+//
+// Used ONLY by bench/ffi-margin.ts to isolate the fixed per-call FFI cost into
+// its components (mirrors castrum's `ffi_probe_*` set): the bare trampoline
+// (noop), scalar-arg/return conversion (echo_usize), TypedArray-view→pointer
+// resolution (echo_view), and `cstring`-ARG transcoding (echo_cstr — the engine
+// encodes the JS string to a call-scoped NUL-terminated buffer; the callee
+// borrows via `CStr::from_ptr`, never dereferencing beyond NUL). They are
+// trivial, non-fallible, allocate nothing, and are NOT part of the shipped
+// `src/native/ffi.ts` dlopen map.
+
+/// Bare C-ABI trampoline floor: does nothing, returns nothing.
+#[no_mangle]
+pub extern "C" fn ffi_probe_noop() {}
+
+/// Scalar pass-through: returns `v` unchanged. Measures scalar-arg + return
+/// conversion only (bind with `usize` vs `u64_fast` to isolate BigInt boxing).
+#[no_mangle]
+pub extern "C" fn ffi_probe_echo_usize(v: usize) -> usize {
+    v
+}
+
+/// View pass-through: returns the byte length of the `(ptr, len)` pair.
+/// Measures TypedArray-view→pointer resolution + (ptr,len) arg conversion.
+///
+/// # Safety
+/// `data` must be valid for reads of `len` bytes (the pointer is only used to
+/// form a slice whose length we return — never dereferenced).
+#[no_mangle]
+pub unsafe extern "C" fn ffi_probe_echo_view(data: *const u8, len: usize) -> usize {
+    if data.is_null() && len != 0 {
+        return 0;
+    }
+    let _ = std::slice::from_raw_parts(data, len);
+    len
+}
+
+/// `cstring`-ARG pass-through: returns the byte length of the NUL-terminated
+/// C string the engine produced by transcoding a JS string arg. Measures the
+/// engine-side JS-string→UTF-8 transcode + call-scoped buffer + callee
+/// `CStr::from_ptr` borrow — the cost a `'cstring'` input arg adds compared
+/// with a JS-side encode + `(ptr,len)` pair. The pointer is only used to
+/// form a slice whose length we return — never dereferenced beyond NUL.
+///
+/// # Safety
+/// `data` must be a valid NUL-terminated C string for reads up to its terminator.
+#[no_mangle]
+pub unsafe extern "C" fn ffi_probe_echo_cstr(data: *const c_char) -> usize {
+    if data.is_null() {
+        return 0;
+    }
+    CStr::from_ptr(data).to_bytes().len()
+}
+
+/// Serialize a JS object (JSON text) into the transport frame
+/// `[WIRE_VERSION][event_id:u32][size-prefixed FlatBuffer]`, writing into `out`.
+///
+/// Returns (`u64_fast` on the Bun side):
+///   - `0`             → hard error (bad JSON / unknown event id / panic)
+///   - `w <= out_cap`  → `w` bytes written (envelope + flatbuffer)
+///   - `w > out_cap`   → exactly `w` bytes required; nothing written
+///
+/// `json` is a NUL-terminated UTF-8 C string (Bun `cstring` arg).
+///
+/// # Safety
+/// `json` must point to a NUL-terminated UTF-8 buffer valid for the call.
+/// `out` must point to `out_cap` writable bytes (or be null when `out_cap == 0`).
+#[no_mangle]
+pub unsafe extern "C" fn fb_serialize(event_id: u32, json: *const c_char, out: *mut u8, out_cap: usize) -> usize {
+    std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
+        if json.is_null() || (out.is_null() && out_cap != 0) {
+            return 0;
+        }
+        let json_bytes = CStr::from_ptr(json).to_bytes();
+        // envelope = [WIRE_VERSION:1][event_id:u32 LE]; flatbuffer payload follows
+        let payload: &mut [u8] = if out_cap < WIRE_HEADER_LEN {
+            &mut []
+        } else {
+            std::slice::from_raw_parts_mut(out.add(WIRE_HEADER_LEN), out_cap - WIRE_HEADER_LEN)
+        };
+        match generated::serialize_event(event_id, json_bytes, payload) {
+            Ok(written) => {
+                let needed = written + WIRE_HEADER_LEN;
+                if needed <= out_cap {
+                    *out.add(0) = WIRE_VERSION;
+                    let id_bytes = event_id.to_le_bytes();
+                    std::ptr::copy_nonoverlapping(id_bytes.as_ptr(), out.add(1), 4);
+                }
+                needed
+            }
+            Err(_) => 0,
+        }
+    }))
+    .unwrap_or(0)
+}
+
+