@rljson/server 0.0.4 → 0.0.5

package/dist/README.blog.md

@@ -8,4 +8,12 @@ found in the LICENSE file in the root of this package.
 
 # Blog
 
-Add latest posts at the end.
+Add posts as Markdown entries in this file (newest last). Keep each post small and link to source code or PRs when helpful. Template:
+
+```md
+## YYYY-MM-DD — Title
+
+- What changed (1–3 bullets)
+- Why it matters
+- Links: PRs, docs, demos
+```

package/dist/README.contributors.md

@@ -8,25 +8,59 @@ found in the LICENSE file in the root of this package.
 
 # Contributors Guide
 
-- [Prepare](#prepare)
-- [Develop](#develop)
-- [Administrate](#administrate)
-- [Fast Coding](#fast-coding)
+- [Prerequisites](#prerequisites)
+- [Setup](#setup)
+- [Everyday development](#everyday-development)
+- [Publishing](#publishing)
+- [More docs](#more-docs)
 
-## Prepare
+## Prerequisites
 
-Read [prepare.md](doc/prepare.md)
+- Node.js v22.14.0+
+- pnpm v10 (see [install-node-mac.md](doc/install-node-mac.md) or [install-node-win.md](doc/install-node-win.md))
+- Optional: sibling repos `rljson-io`, `rljson-bs`, `rljson-db`, `rljson` if you prefer linking for local development
 
-<!-- ........................................................................-->
+## Setup
 
-## Develop
+```sh
+pnpm install
+```
 
-Read [develop.md](doc/develop.md)
+By default we consume published npm versions. If you want to work against local packages instead, `pnpm link` them in sibling folders and add temporary overrides as needed.
 
-## Administrate
+## Everyday development
 
-Read [create-new-repo.md](doc/create-new-repo.md)
+- Run tests + lint (default CI path):
 
-## Fast Coding
+  ```sh
+  pnpm test
+  ```
 
-Read [fast-coding-guide.md](doc/fast-coding-guide.md)
+- Lint only:
+
+  ```sh
+  pnpm lint
+  ```
+
+- Build the package (emits dist, copies README):
+
+  ```sh
+  pnpm build
+  ```
+
+- Update goldens for snapshot-like tests:
+
+  ```sh
+  pnpm updateGoldens
+  ```
+
+## Publishing
+
+`pnpm build` runs `pnpm test` via `prebuild`. `pnpm publish` (or npm publish) will trigger `prepublishOnly` and uses the built `dist` folder. Keep the changelog and versioning in sync with repo guidelines.
+
+## More docs
+
+- [doc/prepare.md](doc/prepare.md)
+- [doc/develop.md](doc/develop.md)
+- [doc/create-new-repo.md](doc/create-new-repo.md)
+- [doc/fast-coding-guide.md](doc/fast-coding-guide.md)

package/dist/README.md

@@ -8,17 +8,76 @@ found in the LICENSE file in the root of this package.
 
 # @rljson/server
 
-## Users
+Local-first, pull-by-reference server layer for Rljson. Clients keep writes local, pull data on demand through multis, and let the server proxy references without duplicating client data.
 
-| File                                 | Purpose                     |
-| ------------------------------------ | --------------------------- |
-| [README.public.md](README.public.md) | Install and use the package |
+- Writes stay local; reads cascade: local ➜ server ➜ peers
+- References (hashes) flow; data is pulled on demand
+- Server aggregates sockets and multicasts refs, but only stores what you explicitly import
 
-## Contributors
+## Quick start
 
-| File                                             | Purpose                       |
-| ------------------------------------------------ | ----------------------------- |
-| [README.contributors.md](README.contributors.md) | Run, debug, build and publish |
-| [README.architecture.md](README.architecture.md) | Software architecture guide   |
-| [README.trouble.md](README.trouble.md)           | Errors & solutions            |
-| [README.blog.md](README.blog.md)                 | Blog                          |
+Install:
+
+```sh
+pnpm add @rljson/server
+```
+
+Minimal server:
+
+```ts
+import { BsMem } from '@rljson/bs';
+import { IoMem } from '@rljson/io';
+import { Route } from '@rljson/rljson';
+import { Server, SocketIoBridge } from '@rljson/server';
+
+const route = Route.fromFlat('my.app');
+const serverIo = new IoMem();
+await serverIo.init();
+await serverIo.isReady();
+
+const server = new Server(route, serverIo, new BsMem());
+await server.init();
+
+// When your runtime yields sockets, wrap them:
+// await server.addSocket(new SocketIoBridge(serverSocket));
+```
+
+Minimal client:
+
+```ts
+import { BsMem } from '@rljson/bs';
+import { IoMem } from '@rljson/io';
+import { Client, SocketIoBridge } from '@rljson/server';
+
+const client = new Client(new SocketIoBridge(clientSocket), new IoMem(), new BsMem());
+await client.init();
+
+const io = client.io; // IoMulti merged interface
+const bs = client.bs; // BsMulti merged interface
+```
+
+Run tests and lint:
+
+```sh
+pnpm test
+```
+
+Build distribution:
+
+```sh
+pnpm build
+```
+
+## Documentation map
+
+| Audience        | File                                             | Highlights                                        |
+| --------------- | ------------------------------------------------ | ------------------------------------------------- |
+| Users           | [README.public.md](README.public.md)             | Install, usage, networking model, examples        |
+| Contributors    | [README.contributors.md](README.contributors.md) | Setup, dev workflow, publishing, fast coding tips |
+| Architecture    | [README.architecture.md](README.architecture.md) | Deep dive into multis, peer bridges, data flows   |
+| Troubleshooting | [README.trouble.md](README.trouble.md)           | Known issues and fixes                            |
+| Blog            | [README.blog.md](README.blog.md)                 | Writing and collecting project blog entries       |
+
+## Example code
+
+See [src/example.ts](src/example.ts) for a runnable end-to-end demo and [test/server.spec.ts](test/server.spec.ts) for broader integration cases.

package/dist/README.public.md

@@ -10,6 +10,19 @@ found in the LICENSE file in the root of this package.
 
 @rljson/server provides a lightweight client/server layer for Rljson storage. It wires Io (row/table data) and Bs (blob storage) over sockets so clients can read from server storage while still keeping their own local storage.
 
+## Prerequisites
+
+- Node.js v22.14.0 or newer
+- A socket runtime (examples use Socket.IO)
+- `Io`/`Bs` implementations (in-memory examples use `IoMem` and `BsMem`)
+
+## Design pillars
+
+- **Local-first, read-through**: Writes stay on the caller; reads walk priorities (local first, then peers via server).
+- **Pull by reference**: Only references (hashes) travel over the wire; data is pulled on demand through `IoMulti`/`BsMulti`.
+- **Server as proxy**: The server aggregates and multicasts refs, but does not duplicate client data unless you intentionally store it there.
+- **Single abstraction surface**: `Client.io`/`Client.bs` and `Server.io`/`Server.bs` expose merged multis so you do not handle peers manually.
+
 ## What it does (quick overview)
 
 - **Server** hosts Io + Bs and exposes them over sockets.
@@ -22,6 +35,50 @@ found in the LICENSE file in the root of this package.
 pnpm add @rljson/server
 ```
 
+## Quick start (Socket.IO example)
+
+Server setup:
+
+```ts
+import { createServer } from 'node:http';
+import { Server as SocketIoServer } from 'socket.io';
+import { BsMem } from '@rljson/bs';
+import { IoMem } from '@rljson/io';
+import { Route } from '@rljson/rljson';
+import { Server, SocketIoBridge } from '@rljson/server';
+
+const httpServer = createServer();
+const socketIo = new SocketIoServer(httpServer);
+
+const route = Route.fromFlat('my.app');
+const server = new Server(route, new IoMem(), new BsMem());
+await server.init();
+
+socketIo.on('connection', async (socket) => {
+   await server.addSocket(new SocketIoBridge(socket));
+});
+
+httpServer.listen(0);
+```
+
+Client setup:
+
+```ts
+import { io as socketIoClient } from 'socket.io-client';
+import { BsMem } from '@rljson/bs';
+import { IoMem } from '@rljson/io';
+import { Db } from '@rljson/db';
+import { Client, SocketIoBridge } from '@rljson/server';
+
+const socket = socketIoClient('http://localhost:3000', { forceNew: true });
+
+const client = new Client(new SocketIoBridge(socket), new IoMem(), new BsMem());
+await client.init();
+
+const db = new Db(client.io!);
+// db.get/insert now cascade local ➜ server automatically
+```
+
 ## Basic usage
 
 ### Server
@@ -46,7 +103,7 @@ await server.init();
 // await server.addSocket(new SocketIoBridge(serverSocket));
 ```
 
-### Client
+### Client API
 
 ```ts
 import { BsMem } from '@rljson/bs';
@@ -87,7 +144,7 @@ This is implemented with `IoMulti` and `BsMulti` internally, but the public API
 - `io` – Io interface (multi-layer)
 - `bs` – Bs interface (multi-layer)
 
-### Server
+### Server API
 
 - `init()` – initializes server multis
 - `ready()` – resolves when Io is ready
@@ -127,3 +184,148 @@ The same pattern is used for Bs (blob storage).
 
 - `Client.io` and `Client.bs` are already merged interfaces. No need to access multis directly.
 - `Server.addSocket()` batches refreshes to reduce rebuild overhead when multiple sockets connect.
+- Multicast includes `__origin` markers plus a `_multicastedRefs` set to prevent ref echo loops.
+
+## Architecture Overview
+
+### Pull-Based Reference Architecture
+
+@rljson/server implements a **pull-based architecture** where data is retrieved on-demand using content-addressed references (hashes), not automatically pushed between clients. This fundamentally differs from traditional sync systems:
+
+### Key principle: references flow, data is pulled
+
+```text
+Reference Flow: Client A → Server → Client B (broadcast)
+Data Flow:      Client A ← Server ← Client B (pulled on query)
+```
+
+### How It Works
+
+1. **Client A stores data locally** (writes to priority 1 layer)
+
+   ```ts
+   const results = await db.insert(route, [data]);
+   const ref = results[0]._hash;
+   ```
+
+2. **Client A broadcasts reference** (not the data)
+
+   ```ts
+   socket.emit(route.flat, ref);
+   ```
+
+3. **Client B receives reference** via multicast
+
+   ```ts
+   socket.on(route.flat, (ref) => { /* ... */ });
+   ```
+
+4. **Client B queries by reference**
+
+   ```ts
+   const result = await db.get(route, { _hash: ref });
+   ```
+
+5. **Server automatically pulls from Client A**
+   - Client B's query goes to its IoMulti (priority 1: local, not found)
+   - Falls back to IoPeer → Server (priority 2)
+   - Server's IoMulti cascades: priority 1 (local cache), then priority 2 (IoPeer[Client A])
+   - Data flows back: Client A → Server → Client B
+
+**This cascade happens automatically** - no explicit pull operation needed.
+
+### Three Storage Types
+
+#### 1. Io Data (Tables)
+
+- **What**: Relational tables (Cake, Cell, custom content types)
+- **Storage**: IoMulti (local Io + IoPeer instances)
+- **Query**: `db.get(route, { _hash: ref })`
+- **Use Cases**: Structured data, records, metadata
+
+#### 2. Bs Data (Blobs)
+
+- **What**: Binary blobs (files, images, videos)
+- **Storage**: BsMulti (local Bs + BsPeer instances)
+- **Query**: `bs.get(blobHash)` after getting ref from Io table
+- **Use Cases**: Large files, media content
+- **Pattern**: Store blob → get hash → store hash in Io table → others query by hash
+
+#### 3. Tree Data (Hierarchical)
+
+- **What**: JSON objects converted to tree structures
+- **Storage**: In Io layer with special 'trees' content type
+- **Conversion**: `treeFromObject(jsObject)` creates Tree[] array
+- **Root**: Last element in array (`trees[trees.length - 1]._hash`)
+- **Query**: `db.get(route, { _hash: rootHash })` returns ALL related nodes
+- **Use Cases**: Configuration objects, nested data structures
+
+### Client-to-Client Communication
+
+### Pattern: Insert on Client A, Get on Client B
+
+```ts
+// Setup: All parties create table definitions
+await server.createTables({ withInsertHistory: [tableCfg] });
+await clientA.createTables({ withInsertHistory: [tableCfg] });
+await clientB.createTables({ withInsertHistory: [tableCfg] });
+
+// Client A: Insert data locally
+const result = await dbA.insert(route, [{ name: 'Tesla', model: 'Model S' }]);
+const ref = result[0]._hash;
+
+// Client A: Broadcast reference
+clientA.socket.emit(route.flat, ref);
+
+// Client B: Listen and pull data
+clientB.socket.on(route.flat, async (ref) => {
+  // This query automatically cascades through server to Client A
+  const data = await dbB.get(route, { _hash: ref });
+  console.log(data.rljson.cars._data[0]); // { name: 'Tesla', ... }
+});
+```
+
+**Server never stores the car data** - it only proxies the query from Client B to Client A.
+
+### Why Pull-Based?
+
+| Aspect              | Pull-Based (@rljson/server)   | Push-Based (Traditional)     |
+| ------------------- | ----------------------------- | ---------------------------- |
+| **Network Traffic** | Minimal (only refs)           | High (all data replicated)   |
+| **Data Freshness**  | Always latest (pull on query) | Can be stale (cached copies) |
+| **Storage**         | Single source of truth        | Multiple copies to sync      |
+| **Bandwidth**       | Low (on-demand only)          | High (push all changes)      |
+| **Offline**         | Works fully offline           | Needs sync when reconnected  |
+| **Conflicts**       | None (read from source)       | Requires resolution logic    |
+
+### When to Use @rljson/server
+
+✅ **Good fit:**
+
+- Local-first applications with occasional sharing
+- Collaborative tools where users own their data
+- Media sharing apps (store locally, share by reference)
+- Configuration management (pull config by root hash)
+- Document collaboration (pull latest version by ref)
+
+❌ **Not ideal for:**
+
+- Real-time collaborative editing (character-by-character)
+- Systems requiring strong consistency guarantees
+- Centralized storage where server must have all data
+- Automatic background sync without references
+
+### Key Design Principles
+
+1. **Local-First**: All writes go to local storage only
+2. **Content-Addressed**: Everything referenced by hash
+3. **Reference-Based Discovery**: Need a reference to query data
+4. **Automatic Cascade**: IoMulti/BsMulti handle priority traversal
+5. **Server as Proxy**: Server doesn't store client data, only routes queries
+6. **Pull on Demand**: Data retrieved only when explicitly queried
+
+### Next Steps
+
+- See [README.architecture.md](README.architecture.md) for detailed architecture documentation
+- See [test/server.spec.ts](test/server.spec.ts) for comprehensive integration examples
+- See [src/example.ts](src/example.ts) for a basic usage example

package/dist/README.trouble.md

@@ -11,6 +11,7 @@ found in the LICENSE file in the root of this package.
 ## Table of contents <!-- omit in toc -->
 
 - [Vscode Windows: Debugging is not working](#vscode-windows-debugging-is-not-working)
+- [Test Isolation: Socket.IO event listener accumulation](#test-isolation-socketio-event-listener-accumulation)
 
 ## Vscode Windows: Debugging is not working
 
@@ -21,3 +22,52 @@ in the VS Code Vitest extension (v1.14.4), which prevents test debugging from
 working: <https://github.com/vitest-dev/vscode/issues/548> Please check from
 time to time if the issue has been fixed and remove this note once it is
 resolved.
+
+## Test Isolation: Socket.IO event listener accumulation
+
+Date: 2025-01-28
+
+**Problem:**
+
+When running multiple tests that use Socket.IO connections, tests pass individually but fail when run together. This is caused by event listeners from previous tests remaining active on persistent socket instances.
+
+**Symptoms:**
+
+- Individual tests pass: ✅
+- All tests together fail: ❌
+- Error messages like "received 0 instead of expected number of nodes"
+- Unexpected behavior when sockets receive messages from previous tests
+
+**Root Cause:**
+
+Socket.IO sockets persist across tests in the `beforeAll` setup. When `SocketIoBridge` instances are created in `beforeEach`, old event listeners accumulate on the underlying sockets, causing interference between tests.
+
+**Solution:**
+
+Clear all event listeners in `beforeEach` before creating new bridges:
+
+```typescript
+beforeEach(async () => {
+  // Remove all event listeners from previous test to prevent interference
+  serverSockets.forEach((socket) => socket.removeAllListeners());
+  clientSockets.forEach((socket) => socket.removeAllListeners());
+
+  // Now proceed with test setup...
+  server = new Server(route, serverIo, serverBs);
+  await server.init();
+  // ... rest of setup
+});
+```
+
+**Why This Works:**
+
+- `removeAllListeners()` clears accumulated event handlers
+- Each test starts with clean sockets
+- No interference from previous test's `SocketIoBridge` instances
+- Maintains socket connections established in `beforeAll`
+
+**Alternative Approaches Considered:**
+
+1. ❌ `tearDown()` in `afterEach`: Caused hook timeouts
+2. ❌ Creating new socket connections per test: Too slow, defeats purpose of `beforeAll`
+3. ✅ Clear listeners while reusing connections: Fast and reliable

package/dist/client.d.ts

@@ -1,6 +1,7 @@
 import { Bs } from '@rljson/bs';
-import { Io, IoMulti, Socket } from '@rljson/io';
+import { Io, IoMulti } from '@rljson/io';
 import { BaseNode } from './base-node.ts';
+import { SocketLike } from './socket-bundle.ts';
 export declare class Client extends BaseNode {
     private _socketToServer;
     protected _localIo: Io;
@@ -11,11 +12,11 @@ export declare class Client extends BaseNode {
     private _bsMulti?;
     /**
      * Creates a Client instance
-     * @param _socketToServer - Socket to connect to server
+     * @param _socketToServer - Socket or namespace bundle to connect to server
      * @param _localIo - Local Io for local storage
      * @param _localBs - Local Bs for local blob storage
      */
-    constructor(_socketToServer: Socket, _localIo: Io, _localBs: Bs);
+    constructor(_socketToServer: SocketLike, _localIo: Io, _localBs: Bs);
     /**
      * Initializes Io and Bs multis and their peer bridges.
      * @returns The initialized Io implementation.
@@ -47,10 +48,12 @@ export declare class Client extends BaseNode {
     private _setupBs;
     /**
      * Creates and initializes a downstream Io peer.
+     * @param socket - Downstream socket to the server Io namespace.
      */
     private _createIoPeer;
     /**
      * Creates and initializes a downstream Bs peer.
+     * @param socket - Downstream socket to the server Bs namespace.
      */
     private _createBsPeer;
 }

package/dist/server.d.ts

@@ -2,6 +2,7 @@ import { Bs, BsPeer } from '@rljson/bs';
 import { Io, IoPeer, Socket } from '@rljson/io';
 import { Route } from '@rljson/rljson';
 import { BaseNode } from './base-node.ts';
+import { SocketLike } from './socket-bundle.ts';
 export type SocketWithClientId = Socket & {
     __clientId?: string;
 };
@@ -33,7 +34,7 @@ export declare class Server extends BaseNode {
      * @param socket - Client socket to register.
      * @returns The server instance.
      */
-    addSocket(socket: Socket): Promise<this>;
+    addSocket(socket: SocketLike): Promise<this>;
     /**
      * Removes all listeners from all connected clients.
      */
@@ -56,7 +57,10 @@ export declare class Server extends BaseNode {
      * Returns the connected clients map.
      */
     get clients(): Map<string, {
-        socket: SocketWithClientId;
+        ioUp: SocketWithClientId;
+        ioDown: SocketWithClientId;
+        bsUp: SocketWithClientId;
+        bsDown: SocketWithClientId;
         io: IoPeer;
         bs: BsPeer;
     }>;
@@ -73,7 +77,7 @@ export declare class Server extends BaseNode {
     /**
      * Registers the client socket and peers.
      * @param clientId - Stable client identifier.
-     * @param socket - Client socket to register.
+     * @param sockets - Directional sockets to register.
      * @param io - Io peer associated with the client.
      * @param bs - Bs peer associated with the client.
      */