@livequery/core 2.0.104 → 2.0.106

package/README.md

@@ -1,477 +1,734 @@
 # @livequery/core
 
-Reactive local-first data primitives for browser clients.
+`@livequery/core` is the framework-agnostic runtime layer for Livequery.
 
-This repository is the core library package, not an application. Changes here should preserve reusable public API behavior unless a task explicitly targets a breaking change.
+It provides the shared primitives used by HTTP adapters, data-source adapters, API gateway processes, service processes, and realtime synchronization layers. The package does not run database queries by itself and does not require a specific HTTP framework.
 
-This package provides the core building blocks behind Livequery collections: reactive document state, pluggable local storage, pluggable transporters, optimistic mutations, and typed inline filters.
+The framework-independent Livequery protocol is defined in [`LIVEQUERY_SPEC.md`](./LIVEQUERY_SPEC.md). Read that file for the canonical definitions of refs, collection/document paths, actions, custom actions, response envelopes, fake/non-database handlers, and realtime update emission.
 
-## AI Agent Guidance
+## What This Project Does
 
-Repository-specific agent guidance lives in `AGENTS.md` and `copilot-instructions.md`.
+Livequery treats an HTTP request path as a structured data reference.
 
-- `AGENTS.md` is the implementation-focused guide for coding agents modifying this package.
-- `copilot-instructions.md` provides repo-level instructions for Copilot when generating or reviewing code in this workspace.
-- Both documents assume this repo is a library package, so agent changes should avoid app-specific scaffolding and should preserve public API compatibility by default.
-- Agents generating consumer code should also follow the usage patterns documented below: create a shared `LivequeryCore`, initialize collections before querying, and subscribe to collection state instead of relying on one-time `.value` reads.
+Examples:
+
+- `/livequery/posts` points to the `posts` collection.
+- `/livequery/posts/p1` points to the `posts/p1` document.
+- `/livequery/users/u1/posts` points to the nested `users/u1/posts` collection.
+- `/livequery/users/u1/posts/p1` points to the nested `users/u1/posts/p1` document.
+
+This package handles the core infrastructure around that model:
+
+- Parse raw framework requests into normalized `LivequeryRequest` objects.
+- Pass request state through a shared `LivequeryContext`.
+- Define a handler interface for parser, middleware, datasource, and realtime handlers.
+- Discover gateway and service nodes over UDP.
+- Route HTTP requests through an API gateway to online service nodes.
+- Publish service metadata from service nodes.
+- Manage realtime WebSocket subscriptions and update forwarding.
+- Sanitize response objects by hiding private fields.
 
 ## Installation
 
-```bash
-bun add @livequery/core rxjs
+```sh
+bun add @livequery/core
 ```
 
-For React projects:
+For local development in this repository:
 
-```bash
-bun add @livequery/core @livequery/react rxjs
+```sh
+bun install
+bun run build
+bun test tests/
 ```
 
-The package is published as ESM and targets browser usage.
+Type-check tests:
 
-## Public Exports
+```sh
+bunx tsc -p tests/tsconfig.json --noEmit
+```
 
-The package re-exports:
+## Public Entry Point
 
 ```ts
-export * from "./LivequeryCollection"
-export * from "./LivequeryCore"
-export * from "./LivequeryMemoryStorage"
-export * from "./LivequeryStorge"
-export * from "./LivequeryTransporter"
-export * from "./types"
-export * from "./helpers/filterDocs"
-export * from "./LivequeryDocument"
+import {
+  ApiGatewayHandler,
+  ApiServiceLinker,
+  LivequeryRequestParser,
+  UdpDiscovery,
+  WebsocketGateway,
+  hidePrivateFields,
+} from '@livequery/core'
 ```
 
 ## Core Types
 
-### `Doc`
+### `CollectionResponse<T>`
+
+Response shape for collection queries.
+
+```ts
+type CollectionResponse<T> = {
+  items: T[]
+  paging: {
+    current: number
+    total: number
+  }
+  cursor: {
+    current: string
+    next: string
+    prev: string
+  }
+}
+```
+
+Use this when a handler returns a list of items with paging and cursor metadata.
+
+### `DocumentResponse<T>`
 
-Every record must have an `id`.
+Response shape for document queries.
 
 ```ts
-type Doc<T = {}> = T & {
-  id: string
+type DocumentResponse<T> = {
+  item: T
 }
 ```
 
-### `DocState`
+Use this when a handler returns one document.
 
-Collections and documents expose `DocState<T>`, which adds optimistic mutation metadata.
+### `RawRequest`
+
+The request shape expected from a framework adapter before Livequery parsing.
 
 ```ts
-type DocState<T extends Doc> = T & {
-  _deleting?: boolean
-  _deleting_error?: { code: string; message: string; transporter_id: string }
-  _updating?: boolean
-  _updating_error?: { code: string; message: string; transporter_id: string }
-  _adding?: boolean
-  _adding_error?: { code: string; message: string; transporter_id: string }
-  _remotes?: Record<string, string | number>
-  _prev?: Partial<T>
+type RawRequest = {
+  path: string
+  ref: string
+  method: string
+  body?: any
+  params: Record<string, any>
+  query: Record<string, any>
+  headers: Map<string, string>
 }
 ```
 
-### `DataChangeEvent`
+- `path` is the actual request path, for example `/livequery/posts/p1`.
+- `ref` is the route pattern, for example `/livequery/posts/:id`.
+- `params` contains framework route params.
+- `query` contains parsed query params.
+- `headers` is a string map used by handlers such as `WebsocketGateway`.
+
+### `LivequeryRequest<I>`
 
-Transporters stream incremental change events back into the core.
+The normalized request shape created by `LivequeryRequestParser`.
 
 ```ts
-type DataChangeEvent = {
+type LivequeryRequest<I> = {
+  keys: Record<string, any>
+  path: string
+  document_id?: string
   collection_ref: string
-  id: string
-  type: "added" | "removed" | "modified"
-  data?: Record<string, any>
+  schema_collection_ref: string
+  ref: string
+  method: string
+  body: I
+  query: Record<string, any>
 }
 ```
 
-## Architecture
+### `LivequeryContext<T>`
 
-```text
-LivequeryCollection / LivequeryDocument
-            |
-            v
-        LivequeryCore
-        /          \
-       v            v
-LivequeryStorge  LivequeryTransporter(s)
+The shared context passed through Livequery handlers.
+
+```ts
+type LivequeryContext<T = {}> = {
+  request: RawRequest
+  livequery?: LivequeryRequest<any>
+  response?: T
+}
 ```
 
-- `LivequeryCollection` owns the reactive state for one collection ref or one document ref.
-- `LivequeryDocument` wraps an item as a `BehaviorSubject` with convenience mutation methods.
-- `LivequeryCore` coordinates storage, transporters, optimistic writes, and fan-out to watchers.
-- `LivequeryStorge` is the local persistence contract.
-- `LivequeryTransporter` is the remote sync contract.
+### `LivequeryHandler<O>`
 
-## Quick Start
+The common handler contract.
 
 ```ts
-import {
-  LivequeryCollection,
-  LivequeryCore,
-  LivequeryMemoryStorage,
-  type Doc,
-  type LivequeryQueryResult,
-  type LivequeryTransporter,
-} from "@livequery/core"
-import { of } from "rxjs"
-
-type Todo = Doc<{
-  title: string
-  done: boolean
-  createdAt: number
-}>
-
-const storage = new LivequeryMemoryStorage()
-
-const transporter: LivequeryTransporter = {
-  query(_query) {
-    return of<Partial<LivequeryQueryResult>>({
-      changes: [],
-      summary: {},
-      paging: { total: 0, current: 0 },
-      metadata: {},
-      source: "query",
-    })
-  },
-  async add(_ref, doc) {
-    return { id: crypto.randomUUID(), ...doc } as Todo
-  },
-  async update(_ref, id, doc) {
-    return { id, ...doc } as Todo
-  },
-  async delete(_ref, id) {
-    return { id } as Todo
-  },
-  async trigger(_action) {
-    return { ok: true }
-  },
+type LivequeryHandler<O = {}> = {
+  handle(ctx: LivequeryContext<O>): any
 }
+```
 
-const core = new LivequeryCore({
-  storage,
-  transporters: {
-    primary: transporter,
-  },
-})
+Use this interface for parsers, middleware, datasource adapters, auth handlers, and realtime handlers.
 
-const todos = new LivequeryCollection<Todo>(core, {
-  filters: { "createdAt:sort": "desc" },
-  mode: "server-first",
-})
+## `LivequeryRequestParser`
 
-todos.initialize("todos")
+`LivequeryRequestParser` is the first handler in a typical request pipeline. It reads `ctx.request` and writes `ctx.livequery`.
 
-todos.items.subscribe((items) => {
-  console.log(items.map((doc) => doc.value))
-})
+### When To Use It
+
+Use it inside HTTP framework adapters before invoking datasource or business logic handlers. Downstream handlers should rely on `ctx.livequery` instead of reparsing paths.
 
-await todos.query({ ":limit": 20, "createdAt:sort": "desc" })
-await todos.add({ title: "Buy milk", done: false, createdAt: Date.now() })
-await todos.update("todo-1", { done: true })
-await todos.delete("todo-1")
+### Constructor
+
+```ts
+new LivequeryRequestParser()
 ```
 
-## `LivequeryCore`
+### `handle(ctx)`
+
+Parses a raw request into:
+
+- `ref`: actual data reference, for example `posts/p1`.
+- `collection_ref`: collection reference, for example `posts`.
+- `schema_collection_ref`: route-pattern-based collection reference, for example `users/uid/posts`.
+- `document_id`: document id when the request targets a document.
+- `method`: uppercased request method.
+- `keys`, `body`, `query`, and original `path`.
+
+It also removes:
 
-Create one core with a storage adapter and one or more transporters:
+- The route prefix before the data ref, such as `livequery`.
+- Query strings before parsing path segments.
+- Realtime suffixes after `~` in the pathname.
+
+Query values are preserved in `query`. A `~` inside the query string is not treated as a realtime suffix.
+
+### Example
 
 ```ts
-const core = new LivequeryCore({
-  storage,
-  transporters: {
-    primary: transporter,
+import { LivequeryRequestParser, type LivequeryContext } from '@livequery/core'
+
+const ctx: LivequeryContext = {
+  request: {
+    path: '/livequery/posts/p1',
+    ref: '/livequery/posts/:id',
+    method: 'get',
+    params: { id: 'p1' },
+    query: {},
+    headers: new Map(),
   },
-})
-```
+}
 
-### Mutation flow
+new LivequeryRequestParser().handle(ctx)
+
+console.log(ctx.livequery)
+// {
+//   ref: 'posts/p1',
+//   collection_ref: 'posts',
+//   schema_collection_ref: 'posts',
+//   document_id: 'p1',
+//   keys: { id: 'p1' },
+//   method: 'GET',
+//   ...
+// }
+```
 
-For `add`, `update`, and `delete`, the core:
+## `LivequeryDatasource`
 
-1. writes to local storage first
-2. broadcasts the optimistic change to active watchers
-3. pushes the mutation to each transporter
-4. clears optimistic flags or stores mutation errors after the remote call finishes
+`LivequeryDatasource<RouteConfig>` is a type for datasource adapters.
 
-Documents created locally receive ids prefixed with `local:` until a transporter returns a persisted id.
+```ts
+type LivequeryDatasourceInitConfig<Config> = Config & {
+  method: string
+  path: string
+}
 
-### Query modes
+type LivequeryDatasource<RouteConfig> = LivequeryHandler & {
+  init(routes: Array<LivequeryDatasourceInitConfig<RouteConfig>>): Promise<void> | void
+}
+```
 
-Collections support three modes through `LivequeryCollectionOptions.mode`:
+### When To Use It
 
-- `server-first`: queries are driven by transporters, and collection state is built from streamed change events.
-- `cache-first`: first query can hydrate from local storage, then transporters refresh the result.
-- `local-first`: queries resolve from local storage while remote sync runs in the background and rebroadcasts matching changes.
+Use this type when implementing an adapter that connects Livequery requests to a database, external API, or framework-specific route system.
 
-Implementation detail: in `local-first` mode, filters are applied by the storage adapter, while the remote query path is triggered with empty filters and matching is re-checked when added events are broadcast locally.
+A datasource should:
 
-## `LivequeryCollection`
+- Implement `handle(ctx)` to process a request.
+- Implement `init(routes)` to register route configuration.
 
-`LivequeryCollection<T>` manages one collection or one document ref.
+### Example
 
 ```ts
-type LivequeryCollectionOptions<T extends Doc> = {
-  core: LivequeryCore
-  filters: Partial<LivequeryFilters<T>>
-  lazy: boolean
-  debounce: number
-  mode: "server-first" | "local-first" | "cache-first"
+import type { LivequeryContext, LivequeryDatasource } from '@livequery/core'
+
+type RouteConfig = { table: string }
+
+class MemoryDatasource implements LivequeryDatasource<RouteConfig> {
+  #routes = new Map<string, RouteConfig>()
+
+  init(routes: Array<RouteConfig & { method: string; path: string }>) {
+    for (const route of routes) {
+      this.#routes.set(`${route.method.toUpperCase()} ${route.path}`, route)
+    }
+  }
+
+  handle(ctx: LivequeryContext) {
+    if (!ctx.livequery) return
+
+    ctx.response = {
+      item: {
+        id: ctx.livequery.document_id,
+        ref: ctx.livequery.ref,
+      },
+    }
+  }
 }
 ```
 
-### Create and initialize a collection
+## `ApiGatewayHandler`
+
+`ApiGatewayHandler` is an HTTP reverse proxy and route registry for Livequery service nodes.
+
+It can:
+
+- Receive service metadata from `UdpDiscovery`.
+- Register routes by method and path.
+- Forward HTTP requests to online service nodes.
+- Round-robin between multiple hosts for the same route.
+- Connect to service WebSocket gateways when service metadata includes `ws`.
+
+### When To Use It
 
-The current constructor takes `core` as the first argument and options as the second argument.
+Use this in a gateway process. Public HTTP requests enter the gateway and are forwarded to service nodes discovered at runtime.
+
+### Constructor
 
 ```ts
-const posts = new LivequeryCollection<Post>(core, {
-  filters: { "publishedAt:sort": "desc" },
-  lazy: false,
-  debounce: 250,
-  mode: "cache-first",
+new ApiGatewayHandler({
+  node_id?: string
+  discovery?: UdpDiscovery<ServiceApiMetadata>
+  ws?: WebsocketGateway
 })
+```
+
+- `node_id`: stable id for this gateway. A random id is used when omitted.
+- `discovery`: custom discovery instance, useful in tests or custom network setups.
+- `ws`: realtime gateway used for cross-gateway WebSocket forwarding.
+
+### `register(options)`
 
-posts.initialize("posts")
+Registers service routes manually.
+
+```ts
+gateway.register({
+  node_id: 'service-1',
+  hostname: '127.0.0.1',
+  port: 3001,
+  paths: [{ method: 'GET', path: 'livequery/posts' }],
+})
 ```
 
-`initialize(ref)` subscribes the collection to `LivequeryCore.watch(ref, id, mode)`. In the current implementation, it is browser-only and returns early when `window` is unavailable.
+### `deregister(node_id)`
+
+Removes all route hosts for a service node.
+
+Use this when a service goes offline or when a forwarded request fails.
 
-### Collection refs and document refs
+### `fetch(request)`
 
-If a ref has an even number of path segments, the last segment is treated as a document id.
+Accepts a Web `Request`, forwards it to the selected service, and returns a Web `Response`.
 
 ```ts
-posts.initialize("posts")
-singlePost.initialize("posts/post-1")
+const response = await gateway.fetch(
+  new Request('http://gateway/livequery/posts')
+)
 ```
 
-For collection mutations, `add`, `update`, and `delete` always target the collection portion of the ref.
+### `fetch(req, res, extraHeaders?)`
+
+Accepts Node.js `IncomingMessage` and `ServerResponse`.
+
+```ts
+import * as http from 'http'
+import { ApiGatewayHandler } from '@livequery/core'
+
+const gateway = new ApiGatewayHandler({})
+
+http.createServer((req, res) => {
+  gateway.fetch(req as any, res)
+}).listen(3000)
+```
+
+### `fetchRequest(request)`
+
+Alias for `fetch(request)`.
+
+### `close()`
+
+Unsubscribes from discovery, closes discovery sockets, disconnects service subscriptions, and clears service state.
+
+### Error Responses
+
+- Missing route: `404 { error: { status: 404, code: 'API_NOT_FOUND' } }`
+- Route exists but no host is online: `503 { error: { status: 503, code: 'API_OFFLINE' } }`
+- Forwarded service request fails: `502 { error: { status: 502, code: 'SERVICE_API_OFFLINE' } }`
+
+## `ApiServiceLinker`
+
+`ApiServiceLinker` publishes service metadata so gateways can discover and route to a service node.
 
-### Reactive state
+### When To Use It
 
-- `items`: `BehaviorSubject<LivequeryDocument<DocState<T>>[]>`
-- `summary`: `BehaviorSubject<Record<string, any>>`
-- `loading`: `BehaviorSubject<null | "all" | "next" | "prev">`
-- `filters`: `BehaviorSubject<Partial<LivequeryFilters<T>>>`
-- `paging`: `BehaviorSubject<LivequeryPaging>`
-- `error`: `BehaviorSubject<{ code: string; message: string } | null>`
+Use this inside each service process that should be discoverable by an `ApiGatewayHandler`.
 
-`items` is a `BehaviorSubject`, not a plain array. Reading `collection.items.value` gives the current snapshot only. If you need live updates, subscribe.
+### Constructor
 
 ```ts
-const subscription = posts.items.subscribe((items) => {
-  console.log("realtime items", items.map((doc) => doc.value))
+new ApiServiceLinker({
+  paths: [{ method: 'GET', path: 'livequery/posts' }],
+  node_id?: 'service-1',
+  discovery?: customDiscovery,
+  ws?: websocketGateway,
 })
-
-subscription.unsubscribe()
 ```
 
-In React, reading only `collection.items.value` during render will not trigger rerenders when new events arrive. Bridge the `BehaviorSubject` into component state.
+### `start(name, port)`
 
-```tsx
-function TodoList({ collection }: { collection: LivequeryCollection<Todo> }) {
-  const [items, setItems] = useState(() => collection.items.value)
+Broadcasts service metadata through UDP discovery.
 
-  useEffect(() => {
-    const subscription = collection.items.subscribe(setItems)
-    return () => subscription.unsubscribe()
-  }, [collection])
+```ts
+const linker = new ApiServiceLinker({
+  paths: [{ method: 'GET', path: 'livequery/posts' }],
+})
 
-  return (
-    <ul>
-      {items.map((item) => (
-        <li key={item.value.id}>{item.value.title}</li>
-      ))}
-    </ul>
-  )
-}
+linker.start('posts-service', 3001)
 ```
 
-### Main methods
+When the service sees a gateway in the same namespace, it refreshes its metadata version and broadcasts again.
+
+### `close()`
+
+Unsubscribes from discovery and closes the discovery instance.
+
+## `UdpDiscovery`
+
+`UdpDiscovery<T>` is an observable UDP discovery layer. It sends and receives msgpack packets signed with HMAC SHA-256.
+
+### When To Use It
+
+Use it when gateway and service nodes need to discover each other without a central registry. `ApiGatewayHandler` and `ApiServiceLinker` create a default instance when no custom discovery is provided.
+
+### Constructor
 
 ```ts
-query(filters: Partial<LivequeryFilters<T>>): Promise<void>
-debounceQuery(filters: Partial<LivequeryFilters<T>>): Promise<void>
-loadMore(): Promise<void>
-loadPrev(): Promise<void>
-loadAround(cursor: string): Promise<void>
-add(payload: Partial<T>): Promise<T>
-update(id: string, payload: Partial<T>): Promise<T | undefined>
-delete(id: string): Promise<void | T | undefined>
-trigger<R>(action: string, payload?: Record<string, any>): Observable<{ data: R; error?: Error }>
-resetError(): void
-watch(check: (prev: T, next: T) => boolean): Observable<[DocState<T>, DocState<T>]>
+const discovery = new UdpDiscovery<MyNode>({
+  key: 'shared-secret',
+  port: 11001,
+})
 ```
 
-Notes about current behavior:
+All trusted nodes must use the same key.
+
+### `status$`
+
+Observable lifecycle status:
+
+- `not_ready`
+- `ready`
+- `closed`
 
-- `query()` requires `initialize()` to have run first so the collection has a `ref` and watcher registration.
-- `debounceQuery()` only emits through the debounced path when `options.debounce` is truthy.
-- `loadMore()` uses `paging.next.cursor` as `:after`.
-- `loadPrev()` uses `paging.prev.cursor` as `:before`.
-- `loadAround()` currently sets both `:after` and `:before` to the provided cursor.
+### `broadcast(node, targetIp?)`
 
-## `LivequeryDocument`
+Broadcasts a node metadata packet.
 
-Each entry inside `collection.items` is a `LivequeryDocument`, which extends `BehaviorSubject<DocState<T>>`.
+If `targetIp` is omitted, the packet is sent to configured multicast peers and local multicast. If `targetIp` is provided, the packet is sent only to that address or list of addresses.
 
 ```ts
-class LivequeryDocument<T extends Doc> extends BehaviorSubject<DocState<T>> {
-  update(data: Partial<T>): Promise<T | undefined>
-  del(): Promise<void | T | undefined>
-  trigger<R>(action: string, payload: Record<string, any>): Observable<{ data: R; error?: Error }>
-}
+await discovery.broadcast({
+  node_id: 'service-1',
+  namespace: 'default',
+  version: Date.now(),
+  role: 'service',
+})
 ```
 
-Example:
+### `close()`
+
+Closes sockets and completes the observable streams.
+
+### Example
 
 ```ts
-const first = todos.items.value[0]
+import { UdpDiscovery, type UdpDiscoveryNode } from '@livequery/core'
 
-first.subscribe((doc) => {
-  console.log(doc.title, doc._updating)
+type Node = UdpDiscoveryNode & { role: 'service' | 'gateway' }
+
+const discovery = new UdpDiscovery<Node>({ key: 'livequery/' })
+
+discovery.subscribe(node => {
+  console.log('node online', node)
 })
 
-await first.update({ done: true })
-await first.del()
-first.trigger("archive", { reason: "completed" }).subscribe()
+await discovery.broadcast({
+  node_id: 'service-1',
+  namespace: 'default',
+  version: Date.now(),
+  role: 'service',
+})
 ```
 
-## `LivequeryStorge`
+## `WebsocketGateway`
+
+`WebsocketGateway` manages realtime subscriptions and forwards update events. It extends `Subject<UpdatedData>`, so callers can publish updates with `next(update)`.
+
+### When To Use It
 
-Local persistence adapters must implement:
+Use it when clients need realtime updates for Livequery refs.
+
+Typical flow:
+
+1. A client connects to the WebSocket endpoint.
+2. The client starts a socket session.
+3. HTTP requests register subscriptions by passing client/gateway headers.
+4. Services publish `UpdatedData`.
+5. The gateway sends `sync` events to subscribed clients.
+
+### Constructor
 
 ```ts
-type LivequeryStorge = {
-  query<T extends Doc>(
-    collection: string,
-    filters?: Record<string, any>
-  ): Promise<{
-    documents: T[]
-    paging: LivequeryPaging
-  }>
-  get<T extends Doc>(ref: string, id: string): Promise<T | null>
-  add<T extends Doc>(collection: string, document: T): Promise<T>
-  update<T extends Doc>(collection: string, id: string, document: Record<string, any>): Promise<T | null>
-  delete<T extends Doc>(collection: string, id: string): Promise<T | null>
-}
+new WebsocketGateway(serverOrPort)
 ```
 
-The package ships with `LivequeryMemoryStorage`, an in-memory adapter useful for tests, demos, and ephemeral state.
+- Pass an `http.Server` in Node.js.
+- Pass a port number in Bun runtime.
+
+### Properties
 
-### `LivequeryMemoryStorage`
+- `id`: unique gateway id.
+- `auth`: token used by trusted gateway-to-gateway connections.
 
-The built-in adapter:
+### `handle(ctx)`
 
-- stores documents in `Map<string, Map<string, Doc>>`
-- generates a local id with `local:${crypto.randomUUID()}` when `id` is missing
-- applies filters through the exported `filterDocs()` helper
-- supports nested sort keys such as `profile.createdAt:sort`
+Reads:
 
-## `LivequeryTransporter`
+- `ctx.livequery.ref`
+- `x-lcid` or `socket_id`
+- `x-lgid`
 
-Remote adapters must implement:
+Then registers a realtime subscription for the current request ref.
+
+### `listen(events)`
+
+Registers one or more realtime subscriptions.
 
 ```ts
-type LivequeryTransporter = {
-  query<T extends Doc>(query: LivequeryQueryParams<T>): Observable<Partial<LivequeryQueryResult>>
-  add<T extends Doc>(ref: string, doc: Omit<T, "id">): Promise<T>
-  update<T extends Doc>(ref: string, id: string, doc: Partial<T>): Promise<T>
-  delete<T extends Doc>(ref: string, id: string): Promise<T>
-  trigger<T>(action: LivequeryAction): Promise<T>
-}
+wsGateway.listen([{
+  ref: 'posts',
+  client_id: 'client-1',
+  gateway_id: wsGateway.id,
+  listener_node_id: wsGateway.id,
+}])
 ```
 
-### Query result shape
+### `unsubscribe_client(socket, body)`
+
+Removes subscriptions for a client by `ref` or `refs`.
+
+### `link(ref, handler)`
+
+Attaches an observable update stream for a ref that already has subscribers.
 
 ```ts
-type LivequeryQueryResult = {
-  error: { code: string; message: string }
-  changes: DataChangeEvent[]
-  summary: Record<string, any>
-  paging: LivequeryPaging
-  metadata: Record<string, any>
-  source: "query" | "action" | "realtime"
-  loading?: "all" | "next" | "prev" | null
-}
+import { Subject } from 'rxjs'
+
+const updates$ = new Subject<any>()
+
+await wsGateway.link('posts', () => updates$)
+
+updates$.next({
+  ref: 'posts',
+  type: 'modified',
+  data: { id: 'p1', title: 'New title' },
+})
 ```
 
-Transporters can emit partial results. In practice, the most useful fields are `changes`, `paging`, `summary`, `metadata`, and `error`.
+### `connect(url, auth, ondisconnect?)`
+
+Creates an outbound connection to another WebSocket gateway and forwards subscription/sync events.
+
+### `close()`
+
+Closes the WebSocket server, active sockets, subscriptions, update streams, and completes the subject.
 
-## Query Filters
+### Client Protocol
 
-Filters are flat keys derived from the document type.
+Client connects to `WEBSOCKET_PATH`, then sends:
 
-### Pagination keys
+```json
+{ "event": "start", "data": { "id": "client-1", "auth": "" } }
+```
+
+Gateway responds:
 
-- `:limit`
-- `:before`
-- `:after`
-- `:around`
-- `:page`
+```json
+{ "event": "hello", "gid": "...", "binary": true }
+```
 
-### Supported operators
+Server update sent to client:
 
-- `field:sort` with `"asc" | "desc"` for string and number fields
-- `field:gt`, `field:gte`, `field:lt`, `field:lte` for numeric fields
-- `field:eq-number` for numeric equality
-- `field` for string equality
-- `field:in`, `field:nin` for string or number membership
-- `field:include` for array containment
-- `field:boolean` with `"true" | "false" | "not-true" | "not-false"`
-- `field:like` for case-insensitive substring matching on strings
-- `field:null` with `"null-only" | "not-null"`
+```json
+{
+  "event": "sync",
+  "data": {
+    "changes": [
+      { "ref": "posts", "type": "modified", "data": { "id": "p1" } }
+    ]
+  }
+}
+```
 
-Nested field paths are supported, for example `"profile.createdAt:sort"`.
+## Helpers
+
+### `hidePrivateFieldsInItem(item)`
+
+Returns a new object with private fields removed. Fields beginning with `_` are removed, except `_id`, which is mapped to `id` when `id` is missing.
 
 ```ts
-await todos.query({
-  ":limit": 20,
-  "done:boolean": "false",
-  "title:like": "milk",
-  "createdAt:gte": 1714176000000,
-  "createdAt:sort": "desc",
-})
+hidePrivateFieldsInItem({ _id: '1', name: 'Alice', _secret: true })
+// { id: '1', name: 'Alice' }
 ```
 
-## Helper Exports
+### `hidePrivateFields(data)`
 
-### `filterDocs()`
+Sanitizes a plain item, a `DocumentResponse`, or a `CollectionResponse`.
 
 ```ts
-import { filterDocs } from "@livequery/core"
+hidePrivateFields({
+  item: { _id: 'p1', title: 'Hello', _internal: true },
+})
+// { item: { id: 'p1', title: 'Hello' } }
+```
+
+### `nodeRequestToWebRequest(req, extraHeaders?)`
+
+Converts a Node.js request with optional `rawBody` into a Web `Request`.
+
+- Uses the `host` header, or `127.0.0.1` as fallback.
+- Merges `extraHeaders`.
+- Omits body for `GET` and `HEAD`.
+- Uses `req.rawBody` for methods that support a body.
+
+### `writeWebResponse(res, response)`
 
-const visible = filterDocs(docs, {
-  "done:boolean": "false",
-  "title:like": "milk",
+Copies a Web `Response` into a Node.js `ServerResponse`.
+
+- Copies status.
+- Copies headers.
+- Writes the response body.
+
+## Constants
+
+| Constant | Meaning | Default |
+| --- | --- | --- |
+| `API_GATEWAY_NAMESPACE` | Namespace used by gateway and service metadata filtering | `default` |
+| `LIVEQUERY_MAGIC_KEY` | Livequery path prefix and default discovery key suffix | `livequery/` |
+| `API_GATEWAY_MULTICAST_PORT` | UDP discovery port | `11001` |
+| `API_GATEWAY_MULTICAST_ADDRESS` | UDP multicast address | `239.0.1.1` |
+| `API_GATEWAY_WHITELIST_ADDRESS` | Additional peer IPs or prefixes | empty |
+| `NODE_ID` | Runtime node id | random UUID |
+| `LIVEQUERY_API_GATEWAY_DEBUG` | Enables gateway logs | false |
+| `WEBSOCKET_PATH` | Realtime WebSocket path | `/livequery/realtime-updates` |
+
+## Example: Service Process
+
+```ts
+import * as http from 'http'
+import {
+  ApiServiceLinker,
+  LivequeryRequestParser,
+  hidePrivateFields,
+  type LivequeryContext,
+} from '@livequery/core'
+
+const parser = new LivequeryRequestParser()
+
+const server = http.createServer(async (req, res) => {
+  const url = new URL(req.url ?? '/', `http://${req.headers.host}`)
+  const id = url.pathname.split('/').at(-1)
+
+  const ctx: LivequeryContext = {
+    request: {
+      path: url.pathname,
+      ref: '/livequery/posts/:id',
+      method: req.method ?? 'GET',
+      params: { id },
+      query: Object.fromEntries(url.searchParams),
+      headers: new Map(Object.entries(req.headers).map(([k, v]) => [k, String(v)])),
+    },
+  }
+
+  parser.handle(ctx)
+
+  ctx.response = hidePrivateFields({
+    item: { _id: ctx.livequery?.document_id, title: 'Hello', _internal: true },
+  })
+
+  res.setHeader('content-type', 'application/json')
+  res.end(JSON.stringify(ctx.response))
 })
+
+server.listen(3001)
+
+new ApiServiceLinker({
+  paths: [{ method: 'GET', path: 'livequery/posts/:id' }],
+}).start('posts-service', 3001)
 ```
 
-### `matchesAllFilters()`
+## Example: Gateway Process
 
-The helper module also exports `matchesAllFilters(doc, filters)` for direct predicate checks.
+```ts
+import * as http from 'http'
+import {
+  ApiGatewayHandler,
+  WebsocketGateway,
+} from '@livequery/core'
 
-## Caveats
+const server = http.createServer()
+const ws = new WebsocketGateway(server)
+const gateway = new ApiGatewayHandler({ ws })
 
-- `initialize()` is browser-only because it exits early when `window` is unavailable.
-- The public storage interface name is intentionally spelled `LivequeryStorge`, matching the source.
-- Optimistic flags such as `_adding`, `_updating`, `_deleting`, and `_prev` are system-managed fields.
-- Transporter query streams are expected to emit incremental `changes`, not full snapshots.
-- `LivequeryCollection` declares a `metadata` subject but does not initialize it in the constructor, so transporter-emitted `metadata` is not safe to rely on yet.
-- `trigger()` is typed at the collection and document layer as `Observable<{ data, error? }>` but currently forwards raw transporter results from `LivequeryCore.trigger()`.
+server.on('request', (req, res) => {
+  gateway.fetch(req as any, res)
+})
 
-## Development
+server.listen(3000)
+```
 
-```bash
-bun run build
+## Environment Variables
+
+```sh
+API_GATEWAY_NAMESPACE=default
+LIVEQUERY_MAGIC_KEY=livequery
+UDP_PUBLIC_PORT=11001
+UDP_MULTICAST_ADDRESS=239.0.1.1
+UDP_WHITELIST_ADDRESS=192.168.1
+REALTIME_UPDATE_SOCKET_PATH=/livequery/realtime-updates
+LIVEQUERY_API_GATEWAY_DEBUG=1
+LIVEQUERY_UDP_DEBUG=1
 ```
 
-Available scripts:
+`UDP_WHITELIST_ADDRESS` accepts:
+
+- A full IP address, for example `192.168.1.10`.
+- A three-part prefix, for example `192.168.1`, expanded to `192.168.1.0` through `192.168.1.255`.
+
+## Tests
+
+```sh
+bun run build
+bun test tests/
+bunx tsc -p tests/tsconfig.json --noEmit
+```
 
-- `bun run clean`
-- `bun run build:js`
-- `bun run build:types`
-- `bun run build`
-- `bun run build:watch`
+The test suite covers:
+
+- Public entrypoint exports.
+- Request parsing, including nested params, realtime suffixes, and query strings containing `~`.
+- API gateway routing, metadata updates, header/body forwarding, error responses, and round-robin.
+- Service metadata publishing with `ApiServiceLinker`.
+- UDP discovery signatures, TTL, status, and close behavior.
+- WebSocket gateway lifecycle, subscriptions, observable links, and gateway-to-gateway forwarding.
+- Hono integration and multi-process gateway/service discovery flows.
+- Response field sanitization.
+- Node/Web HTTP helper conversion.