lowlander 0.2.1 → 0.2.3

package/skill/SKILL.md

@@ -0,0 +1,430 @@
+---
+name: lowlander
+description: Expert guidance for building full-stack TypeScript apps with Lowlander. Covers type-safe RPCs, Edinburgh model streaming, reactive client sync, ServerProxy, Socket callbacks, and Connection setup.
+---
+
+# Lowlander
+
+An **experimental** TypeScript framework for data persistence and (partial) client synchronization.
+
+This project is still under heavy development. **DO NOT USE** for anything serious. Early feedback is very welcome though!
+
+To get an impression of what use of this framework currently looks like, check out the example project's...
+
+ - [server-side API](https://github.com/vanviegen/lowlander/blob/main/examples/helloworld/server/api.ts) and
+ - [client-side UI](https://github.com/vanviegen/lowlander/blob/main/examples/helloworld/client/js/base.ts) code.
+
+## Tech
+
+This library is built on top of a number of libraries by the same author:
+
+- [Edinburgh](https://github.com/vanviegen/edinburgh): use JavaScript objects as really fast ACID database records.
+- [OLMDB](https://github.com/vanviegen/olmdb): a very fast on-disk key/value store with MVCC and optimistic transactions, used by Edinburgh for persistence.
+- [WarpSocket](https://github.com/vanviegen/warpsocket): a high-performance WebSocket server written in Rust, that coordinates multiple JavaScript worker threads and provides an API for channel subscriptions.
+- [Aberdeen](https://github.com/vanviegen/aberdeen): a reactive UI library for JavaScript. It features fine-grained updates, needs no virtual DOM, and uses Proxy for reactivity.
+
+Lowlander glues these together and adds real-time partial data synchronization and type-safe RPCs to provide a framework for rapidly building performant full-stack (database included!) web applications.
+
+## Tutorial
+
+### Project Setup
+
+```bash
+bun init
+bun add lowlander aberdeen edinburgh
+```
+
+(npm should also work for all of this.)
+
+Create the project structure:
+
+```
+server/
+  main.ts     # starts the server
+  api.ts      # exported functions = RPC endpoints
+client/
+  app.ts      # UI using Aberdeen + Connection
+```
+
+If you use Claude Code, GitHub Copilot or another AI agent that supports Skills, Lowlander and its dependencies include `skill/` directories that provide specialized knowledge to the AI.
+
+Symlink them into your project's `.claude/skills` directory:
+
+```bash
+mkdir -p .claude/skills
+ln -s ../../node_modules/lowlander/skill .claude/skills/lowlander
+ln -s ../../node_modules/aberdeen/skill .claude/skills/aberdeen
+ln -s ../../node_modules/edinburgh/skill .claude/skills/edinburgh
+```
+
+### Server Entry Point
+
+The entry point starts the WarpSocket server and points it at the API file:
+
+```ts
+// server/main.ts
+import { start } from 'lowlander/server';
+import { fileURLToPath } from 'url';
+import { resolve, dirname } from 'path';
+
+const API_FILE = resolve(dirname(fileURLToPath(import.meta.url)), 'api.js');
+start(API_FILE, { bind: '0.0.0.0:8080' });
+```
+
+Options: `bind` (address:port), `threads` (worker count).
+
+### Defining RPC Endpoints
+
+Every exported function in the API file is callable from the client. No decorators or registration needed:
+
+```ts
+// server/api.ts
+export function add(a: number, b: number): number {
+    return a + b;
+}
+```
+
+Functions can be `async`. Thrown errors are sent to the client as error responses.
+
+### Edinburgh Models
+
+Define persistent data models using Edinburgh. See [Edinburgh docs](https://github.com/vanviegen/edinburgh) for full details.
+
+```ts
+import * as E from 'edinburgh';
+
+@E.registerModel
+class Person extends E.Model<Person> {
+    static byName = E.primary(Person, 'name');
+    name = E.field(E.string);
+    age = E.field(E.number);
+    friends = E.field(E.array(E.link(Person)));
+    password = E.field(E.string);
+}
+```
+
+Models are ACID, and RPC calls automatically run in transactions. When creating a `new Instance()` or updating props on an existing instance, changes are persisted to disk automatically. `E.link` objects are lazy-loaded.
+
+### Model Streaming with `createStreamType`
+
+Stream a subset of model fields to clients with real-time updates. Changes are pushed automatically. First you need to create a stream type, by doing this once:
+
+```ts
+import { createStreamType } from 'lowlander/server';
+
+// Exclude password; include friends' names and ages
+const PersonStream = createStreamType(Person, {
+    name: true,
+    age: true,
+    friends: {        // nested linked model: specify sub-selection
+        name: true,
+        age: true,
+    }
+});
+```
+
+Use `true` for plain fields. For linked model fields, provide a nested selection object. To return a stream instance from an API function:
+
+```ts
+export function streamPerson(name: string) {
+    const person = Person.byName.get(name)!;
+    return new PersonStream(person);
+}
+```
+
+On the client, this returns a reactive Aberdeen proxy that updates live when server data changes.
+
+### ServerProxy for Stateful APIs
+
+Wrap a class instance to expose per-connection stateful methods:
+
+```ts
+import { ServerProxy } from 'lowlander/server';
+
+class UserAPI {
+    constructor(public userName: string) {}
+    
+    get user(): Person {
+        return Person.byName.get(this.userName)!;
+    }
+
+    getBio() {
+        return `${this.user.name} is ${this.user.age} years old`;
+    }
+}
+
+export async function authenticate(token: string) {
+    const user = Person.byName.get(token);
+    if (!user) throw new Error('User not found');
+    return new ServerProxy(new UserAPI(token), 'secret-value');
+}
+```
+
+The client receives `'secret-value'` as `.value` and can call `UserAPI` methods via `.serverProxy`.
+
+### Socket Callbacks
+
+Use `Socket<T>` parameters for server-push streaming. On the client, these become callback functions:
+
+```ts
+import { Socket } from 'lowlander/server';
+
+export function streamNumbers(socket: Socket<number>) {
+    const interval = setInterval(() => {
+        if (!socket.send(Math.random())) clearInterval(interval);
+    }, 1000);
+}
+```
+
+`socket.send()` returns falsy when the client disconnects.
+
+### Client Connection
+
+Connect to the server with full type safety:
+
+```ts
+import { Connection } from 'lowlander/client';
+import type * as API from './server/api.js';
+
+const conn = new Connection<typeof API>('ws://localhost:8080/');
+const api = conn.api;
+```
+
+All server exports are available on `conn.api` with matching types, except `Socket<T>` params become callbacks.
+
+#### Simple RPC
+
+```ts
+const sum = api.add(1, 2);
+// sum is a PromiseProxy:
+// - sum.value starts out as undefined, and reactively updates to the result when available
+// - sum.error is an Error object if the call threw, or undefined otherwise
+// - sum.promise can be awaited: `const val = await sum.promise;` - this throws on error
+```
+
+#### Using ServerProxy
+
+```ts
+const auth = api.authenticate('Frank');
+// auth.value → 'secret-value' (after resolution)
+// auth.serverProxy → typed proxy to UserAPI methods
+
+const bio = auth.serverProxy.getBio();
+// bio.value → "Frank is 45 years old"
+```
+
+The server proxy is usable immediately—calls queue until authentication completes. If auth fails, queued calls fail too.
+
+#### Model Streaming
+
+```ts
+const person = api.streamPerson('Alice');
+// person.value is a reactive proxy that auto-updates
+```
+
+#### Socket Callbacks
+
+```ts
+api.streamNumbers(num => console.log(num));
+```
+
+On the server-side we should have a `export function streamNumbers(socket: Socket<number>)`.
+
+#### Reactive Integration with Aberdeen
+
+`PromiseProxy` results are reactive in Aberdeen scopes:
+
+```ts
+import A from 'aberdeen';
+
+const sum = api.add(1, 2);
+A(() => {
+    if (sum.busy) A('span#Loading...');
+    else if (sum.error) A('span#Error: ' + sum.error.message);
+    else A('span#Result: ' + sum.value);
+});
+```
+
+Model streams are also reactive—nested data updates trigger fine-grained UI updates:
+
+```ts
+const model = api.streamModel();
+A(() => {
+    if (!model.value) return;
+    A('h2#' + model.value.name);
+    A('p#Owner: ' + model.value.owner.name);
+});
+```
+
+#### Connection Status
+
+```ts
+A(() => {
+    A('span#' + (conn.isOnline() ? 'Connected' : 'Offline'));
+});
+```
+
+Reconnection is automatic with exponential backoff.
+
+#### Cleanup
+
+Aberdeen's `clean()` handles RPC lifecycle. When a reactive scope is destroyed, active requests and subscriptions are cancelled automatically.
+
+### Logging
+
+Set the `LOWLANDER_LOG_LEVEL` environment variable to a number from 0 to 3:
+
+- 0: no logging (default)
+- 1: connections & lifecycle
+- 2: RPC calls & responses
+- 3: model streaming & internals
+
+Set `EDINBURGH_LOG_LEVEL` similarly for Edinburgh internals.
+
+## Server API Reference
+
+The following is auto-generated from `server/server.ts`:
+
+### [createStreamType](createStreamType.md) · function
+
+Creates a stream type for reactive model streaming to clients with automatic updates.
+
+### [sendModel](sendModel.md) · function
+
+Sends (updated) data for `model` to `target`.
+`target` is a virtual socket with a requestId+'d' user prefix, or a channel that subscribes such virtual sockets.
+
+### [pushModel](pushModel.md) · function
+
+Subscribes `target` to this model, and sends initial data.
+`target` is a virtual socket with a requestId+'d' user prefix, or a channel that subscribes such virtual sockets.
+
+### [start](start.md) · function
+
+Starts the Lowlander WebSocket server.
+
+### logLevel · constant
+
+**Value:** `number`
+
+### [warpsocket](warpsocket.md) · class
+
+### StreamTypeBase · abstract class
+
+[object Object],[object Object],[object Object]
+
+**Type Parameters:**
+
+- `T`
+
+#### StreamTypeBase.fields · static property
+
+**Type:** `{ [key: string]: number | true; }`
+
+#### StreamTypeBase.id · static property
+
+**Type:** `number`
+
+#### streamTypeBase.toString · method
+
+**Signature:** `() => string`
+
+### [ServerProxy](ServerProxy.md) · class
+
+Wraps a server-side API object to create a stateful, type-safe proxy accessible from clients.
+Use for authentication, sessions, or any stateful context that persists across RPC calls.
+
+#### serverProxy.toString · method
+
+**Signature:** `() => string`
+
+### [Socket](Socket.md) · class
+
+Server-side socket for pushing data to a client. Server functions with `Socket<T>` parameters
+receive client callbacks on the client side.
+
+#### [socket.send](Socket_send.md) · method
+
+Sends data to the client.
+
+#### [socket.subscribe](Socket_subscribe.md) · method
+
+#### socket.toString · method
+
+**Signature:** `() => string`
+
+#### socket.[Symbol.for('nodejs.util.inspect.custom')] · method
+
+**Signature:** `() => string`
+
+## Client API Reference
+
+The following is auto-generated from `client/client.ts`:
+
+### setLogLevel · function
+
+Set to 0-3 for increasing verbosity.
+
+**Signature:** `(level: number) => void`
+
+**Parameters:**
+
+- `level: number`
+
+### ClientProxyObject · type
+
+Transforms server-side API objects to client-side proxy objects with type-safe RPC methods.
+
+**Type:** `{
+    [K in keyof T]: ClientProxyFunction<T[K]>
+}`
+
+### [Connection](Connection.md) · class
+
+WebSocket connection to a Lowlander server with type-safe RPC, automatic reconnection,
+and reactive updates.
+
+#### connection.ws · property
+
+**Type:** `WebSocket`
+
+#### connection.activeRequests · property
+
+**Type:** `Map<number, ActiveRequest>`
+
+#### connection.requestCounter · property
+
+**Type:** `number`
+
+#### connection.reconnectAttempts · property
+
+**Type:** `number`
+
+#### connection.onlineProxy · property
+
+**Type:** `ValueRef<boolean>`
+
+#### connection.api · property
+
+Type-safe proxy to the server-side API. Methods return `PromiseProxy` objects
+that work reactively in Aberdeen scopes. `ServerProxy` returns include a
+`.serverProxy` property for accessing stateful server APIs.
+
+**Type:** `ClientProxyObject<T>`
+
+#### connection.isOnline · method
+
+Returns the current connection status. Reactive in Aberdeen scopes.
+
+**Signature:** `() => boolean`
+
+#### connection.connect · method
+
+**Signature:** `() => void`
+
+#### connection.reconnect · method
+
+**Signature:** `() => void`
+
+#### [connection.pruneCommitIds](Connection_pruneCommitIds.md) · method
+

package/skill/ServerProxy.md

@@ -0,0 +1,30 @@
+### ServerProxy · class
+
+Wraps a server-side API object to create a stateful, type-safe proxy accessible from clients.
+Use for authentication, sessions, or any stateful context that persists across RPC calls.
+
+**Type Parameters:**
+
+- `API extends object`
+- `RETURN`
+
+**Examples:**
+
+```ts
+export class UserAPI {
+  constructor(public user: User) {}
+  getSecret() { return this.user.secret; }
+}
+
+export async function authenticate(token: string) {
+  const user = await validateToken(token);
+  return new ServerProxy(new UserAPI(user), user.name);
+}
+
+// Client: auth.value is user name, auth.serverProxy.getSecret() calls UserAPI method
+```
+
+**Constructor Parameters:**
+
+- `api`: - Server-side API object exposed to the client
+- `value`: - Value returned immediately to the client

package/skill/Socket.md

@@ -0,0 +1,22 @@
+### Socket · class
+
+Server-side socket for pushing data to a client. Server functions with `Socket<T>` parameters
+receive client callbacks on the client side.
+
+**Type Parameters:**
+
+- `T`
+
+**Examples:**
+
+```ts
+// Server
+export function streamNumbers(socket: Socket<number>) {
+  setInterval(() => {
+    if (!socket.send(Math.random())) clearInterval(interval);
+  }, 1000);
+}
+
+// Client
+api.streamNumbers(num => console.log(num));
+```

package/skill/Socket_send.md

@@ -0,0 +1,11 @@
+#### socket.send · method
+
+Sends data to the client.
+
+**Signature:** `(data: T) => number`
+
+**Parameters:**
+
+- `data: T` - - Data to send (automatically serialized)
+
+**Returns:** `true` if sent, `false` if socket is closed

package/skill/Socket_subscribe.md

@@ -0,0 +1,8 @@
+#### socket.subscribe · method
+
+**Signature:** `(channel: Uint8Array<ArrayBufferLike>, delta?: number) => void`
+
+**Parameters:**
+
+- `channel: Uint8Array`
+- `delta: any` (optional)

package/skill/createStreamType.md

@@ -0,0 +1,44 @@
+### createStreamType · function
+
+Creates a stream type for reactive model streaming to clients with automatic updates.
+
+Specify which fields to include; when they change, updates are pushed to subscribed clients.
+Supports nested linked models and type-safe field selection.
+
+**Signature:** `<T, S extends FieldSelection<T>>(Model: typeof E.Model<unknown> & (new (...args: any[]) => T), selection: S & ValidateSelection<T, S>) => typeof StreamType`
+
+**Type Parameters:**
+
+- `T`
+- `S extends FieldSelection<T>`
+
+**Parameters:**
+
+- `Model: ModelClass & (new (...args: any[]) => T)` - - The Edinburgh model class
+- `selection: S & ValidateSelection<T, S>` - - Field selection: `true` for simple fields, nested object for linked models
+
+**Returns:** Stream type class to instantiate in API functions
+
+**Examples:**
+
+```ts
+⁣@E.registerModel
+class Person extends Model {
+  name = field(string);
+  age = field(number);
+  password = field(string);
+  friends = field(array(link(Person)));
+}
+
+// Exclude password, include friends' names
+const PersonStream = createStreamType(Person, {
+  name: true,
+  age: true,
+  friends: { name: true }
+});
+
+export function streamPerson() {
+  const person = Person.byName.get('Alice')!;
+  return new PersonStream(person);
+}
+```

package/skill/pushModel.md

@@ -0,0 +1,14 @@
+### pushModel · function
+
+Subscribes `target` to this model, and sends initial data.
+`target` is a virtual socket with a requestId+'d' user prefix, or a channel that subscribes such virtual sockets.
+
+**Signature:** `(target: number | Uint8Array<ArrayBufferLike> | number[], model: Model<any>, commitId: number, SubStreamType: typeof StreamTypeBase<any>, delta: number) => void`
+
+**Parameters:**
+
+- `target: number | Uint8Array | number[]`
+- `model: E.Model<any>`
+- `commitId: number`
+- `SubStreamType: typeof StreamTypeBase<any>`
+- `delta: number`

package/skill/sendModel.md

@@ -0,0 +1,14 @@
+### sendModel · function
+
+Sends (updated) data for `model` to `target`.
+`target` is a virtual socket with a requestId+'d' user prefix, or a channel that subscribes such virtual sockets.
+
+**Signature:** `(target: number | Uint8Array<ArrayBufferLike> | number[], model: Model<any>, commitId: number, StreamType: typeof StreamTypeBase<any>, changed?: Change) => void`
+
+**Parameters:**
+
+- `target: Uint8Array | number | number[]`
+- `model: E.Model<any>`
+- `commitId: number`
+- `StreamType: typeof StreamTypeBase<any>`
+- `changed?: E.Change`

package/skill/start.md

@@ -0,0 +1,21 @@
+### start · function
+
+Starts the Lowlander WebSocket server.
+
+**Signature:** `(mainApiFile: string, opts?: { bind?: string; threads?: number; injectWarpSocket?: typeof import("/var/home/frank/projects/warpsocket/dist/src/index", { with: { "resolution-mode": "import" } }); }) => Promise<void>`
+
+**Parameters:**
+
+- `mainApiFile: string` - - Absolute path to the compiled API file exporting server functions
+- `opts: {bind?: string, threads?: number, injectWarpSocket?: typeof realWarpsocket}` (optional)
+
+**Examples:**
+
+```ts
+import { start } from 'lowlander/server';
+import { fileURLToPath } from 'url';
+import { resolve, dirname } from 'path';
+
+const API_FILE = resolve(dirname(fileURLToPath(import.meta.url)), 'api.js');
+start(API_FILE, { bind: '0.0.0.0:8080' });
+```

package/skill/warpsocket.md

@@ -0,0 +1,3 @@
+### warpsocket · class
+
+**Type:** `typeof import("/var/home/frank/projects/warpsocket/dist/src/index", { with: { "resolution-mode": "import" } })`

package/AGENTS.md

@@ -1,2 +0,0 @@
-- In all interactions and commit messages, be extremely concise and sacrifice grammar for the sake of concision.
-- Before touching any frontend code (involving the `A` symbol) study the Aberdeen skill.

package/ROADMAP.md

@@ -1,13 +0,0 @@
-Semi-prioritized Roadmap
-========================
-
-[ ] Integration with a component library
-[/] Integrate a WarpSocket state inspection tool?
-[ ] A simple built-in Sentry-like error logging system
-[ ] Model-streaming with edit permissions for specific fields
-[ ] Integration with an authentication library
-[ ] Form builder based on Edinburgh schemas
-[x] Database migrations
-[ ] Streaming replication and backups
-[ ] Federation primitives
-[ ] Merge WarpSocket, Edinburgh and OLMDB into Lowlander, to allow model streaming and serialization to be done in Rust/Neon