@hotfusion/modeller 0.0.2 → 0.0.3

package/README.md

@@ -1,29 +1,26 @@
 # @hotfusion/modeller
 
-A schema-first, in-memory data layer for Node.js with hooks, workers, methods, uploads, streams, and event subscriptions. Built around AJV validation, mingo queries, and a pluggable `Adapter` for persistence.
+A schema-first data layer for Node.js, built as a three-layer chain:
 
-```ts
-import { Model, Adapter } from '@hotfusion/modeller';
+```
+┌──────────────┐   data primitives — schema, validation, CRUD,
+│   core.ts    │   indexes, persistence adapter, trash
+└──────┬───────┘
+       │ extends
+┌──────▼───────┐   behavior — hooks, workers, methods, uploads,
+│   model.ts   │   streams, events, subscriptions, logging
+└──────┬───────┘
+       │ consumed by
+┌──────▼───────┐   transport — HTTP + WebSocket exposure of any model
+│  server.ts   │   (auth, sessions, socket fan-out)
+└──────────────┘
 ```
 
----
-
-## Table of Contents
+Each layer is usable on its own. Pick the layer that matches the problem you're solving.
 
-- [Quick Start](#quick-start)
-- [Schema](#schema)
-- [CRUD Methods](#crud-methods)
-- [Hooks](#hooks)
-- [Workers](#workers)
-- [Methods](#methods)
-- [Uploads](#uploads)
-- [Streams](#streams)
-- [Events & Subscriptions](#events--subscriptions)
-- [Internal Event Emitter](#internal-event-emitter)
-- [Extensions](#extensions)
-- [Adapter (Persistence)](#adapter-persistence)
-- [Logging](#logging)
-- [Trash](#trash)
+```ts
+import { Adapter, Model, Server, Bundler, KeyGen } from '@hotfusion/modeller';
+```
 
 ---
 
@@ -43,451 +40,32 @@ const users = new Model('users', {
     }
 });
 
-await users.insert({ email: 'alex@hotfusion.ca', name: 'Alex' });
-const found = await users.get({ email: 'alex@hotfusion.ca' });
-const list  = await users.list({}, { start: 0, count: 50 });
-```
-
----
-
-## Schema
-
-Schemas are JSON Schema (AJV-compatible) with a few Modeller-specific keywords:
-
-| Keyword     | Effect                                                                  |
-|-------------|-------------------------------------------------------------------------|
-| `unique`    | Enforces uniqueness across all documents on `insert` / `update`.        |
-| `private`   | Field is stripped from read responses unless `{ private: true }` is set.|
-| `index`     | Builds an in-memory index on the field for fast lookups.                |
-| `protected` | Reserved for transport-layer protection (consumed by `Server`).         |
-| `hidden`    | Hint for client-side rendering. Not enforced by the core.               |
-| `default`   | Standard JSON Schema default.                                            |
-| `model`     | (async) References another model by id for FK-style validation.         |
-
-`$ref` is resolved against `$defs` / `definitions`, and against any `schemes` passed to the constructor. A `$ref` of the form `someModel?key=_id&label=name` is rewritten into a model reference with display metadata.
-
----
-
-## CRUD Methods
-
-All CRUD methods are async, fire `before:*` / `after:*` hooks, validate against the schema, and emit an event on the internal emitter.
-
-### `insert(key?, ...patches, opts?)`
-
-Merges `key` and any number of patch objects into a single document, validates it, assigns an `_id` (BSON `ObjectId`), and stores it.
-
-```ts
-await users.insert({ email: 'alex@hotfusion.ca' }, { name: 'Alex' });
-await users.insert({ email: 'admin@hotfusion.ca' }, { role: 'admin' }, { private: true });
-```
-
-- Returns the inserted document. `private: true` includes `private` fields in the response.
-- Throws `{ code: 'VALIDATION_ERROR', errors }` or `{ code: 'UNIQUE_VIOLATION', field }`.
-
-### `update(key, updates, opts?)`
-
-Finds a document by `key` (mingo query), shallow-merges `updates`, re-validates, and saves.
-
-```ts
-await users.update({ _id: id }, { name: 'Alex H.' });
-```
-
-Returns the merged document, or `null` if no match. Same error codes as `insert`.
-
-### `delete(key)`
-
-Removes the document matching `key`. If `trash` is not disabled and an adapter is configured, the document is moved to the in-memory trash bin.
-
-```ts
-await users.delete({ _id: id });   // → { _id }  or  null
-```
-
-### `get(key, opts?)`
-
-Returns a single document or `null`. Pass `{ private: true }` to include private fields.
-
-```ts
-await users.get({ email: 'alex@hotfusion.ca' });
-```
-
-### `list(query?, opts?)`
-
-Paginated, **exact-match** query (mingo).
-
-```ts
-await users.list({ role: 'admin' }, { start: 0, count: 25 });
-```
-
-Default `count` is 10. Fires `before:list` and `after:list` with the query as payload `data`.
-
-### `find(query?, opts?)`
-
-Like `list`, but **strings match as case-insensitive substrings**. Non-string values match exactly.
-
-```ts
-await users.find({ name: 'al' }, { max: 10 });
-// matches 'Alex', 'Albert', ...
-```
-
-Default `max` is 10. Does **not** fire hooks.
-
----
-
-## Hooks
-
-Hooks run before or after a CRUD operation. They can short-circuit an operation by throwing, mutate the payload, or perform side effects. Hooks support both synchronous and asynchronous callbacks.
-
-### Registering
-
-```ts
-users.hook({
-    id       : 'normalize-email',
-    on       : 'before:insert',
-    callback : async ({ data }) => {
-        if (data?.email) data.email = data.email.toLowerCase();
-    }
-});
-```
-
-Multiple events can share a callback:
-
-```ts
-users.hook({
-    id       : 'audit',
-    on       : ['after:insert', 'after:update', 'after:delete'],
-    callback : ({ operation, key, data }) => audit.log(operation, key, data)
-});
-```
-
-### Removing
-
-```ts
-users.unhook('normalize-email');
-```
-
-### Hook events
-
-| Event             | Payload fields                                |
-|-------------------|-----------------------------------------------|
-| `before:insert`   | `operation`, `timing`, `data`                 |
-| `after:insert`    | `operation`, `timing`, `data`, `key: { _id }` |
-| `before:update`   | `operation`, `timing`, `key`, `data`          |
-| `after:update`    | `operation`, `timing`, `key`, `data`          |
-| `before:delete`   | `operation`, `timing`, `key`                  |
-| `after:delete`    | `operation`, `timing`, `key`, `data`          |
-| `before:list`     | `operation`, `timing`, `data` (= query)       |
-| `after:list`      | `operation`, `timing`, `data` (= query)       |
-
-Note: `find` does not fire hooks; `get` does not fire hooks.
-
-### Scheduled hooks
-
-Pass `schedule: <ms>` to defer execution via `setTimeout`. Scheduled hooks are fire-and-forget — they don't block the operation and their errors aren't propagated.
-
-```ts
 users.hook({
-    id       : 'welcome-email',
-    on       : 'after:insert',
-    schedule : 5_000,
-    callback : ({ data }) => mailer.send(data.email, 'welcome')
-});
-```
-
-### Throwing from a hook
-
-A non-scheduled hook that throws aborts the operation. The error is logged and re-thrown to the caller.
-
----
-
-## Workers
-
-Workers are interval-based background tasks bound to the model. They can be named (managed individually) or unsigned (managed in bulk).
-
-### Registering
-
-```ts
-users.worker({
-    id        : 'sync',
-    interval  : 30_000,
-    immediate : true,            // run once on start, default true
-    handler   : async (model) => {
-        await model.pull();
-    }
-});
-```
-
-Omit `id` for an unsigned worker:
-
-```ts
-users.worker({ interval: 60_000, handler: () => cleanup() });
-```
-
-### Lifecycle
-
-```ts
-users.start();           // start all workers (named + unsigned)
-users.start('sync');     // start a specific named worker
-users.stop();            // stop all
-users.stop('sync');      // stop a specific named worker
-users.isRunning('sync'); // → boolean
-```
-
-Errors thrown inside a worker handler are caught and logged at `error` level — they never crash the interval.
-
----
-
-## Methods
-
-Methods are named, schemaless RPC-style operations attached to the model. Use them for anything that doesn't fit the CRUD shape.
-
-```ts
-users.method({
-    id      : 'resetPassword',
-    handler : async ({ email, newPassword }, model, ctx) => {
-        const user = await model.get({ email });
-        if (!user) throw { code: 'NOT_FOUND' };
-        return model.update({ _id: user._id }, { password: newPassword });
-    }
+    id: 'normalize-email',
+    on: 'before:insert',
+    callback: ({ data }) => { data.email = data.email.toLowerCase(); }
 });
 
-await users.call('resetPassword', { email, newPassword }, ctx);
-users.getMethods(); // → ['resetPassword']
-```
-
-`call` throws `{ code: 'METHOD_NOT_FOUND', id }` if the method isn't registered.
-
----
-
-## Uploads
-
-Uploads receive a `Buffer` plus a `fields` map (typically multipart form fields).
-
-```ts
-users.upload({
-    id      : 'avatar',
-    handler : async (file, fields, model) => {
-        const path = await storage.put(file, fields.filename);
-        return model.update({ _id: fields.userId }, { avatar: path });
-    }
-});
-
-await users.callUpload('avatar', buffer, { userId, filename: 'me.png' });
-users.getUploads(); // → ['avatar']
-```
-
----
-
-## Streams
-
-Streams are similar to uploads but expect chunked `Buffer` data with arbitrary `meta`.
-
-```ts
-users.stream({
-    id      : 'import',
-    handler : async (chunk, meta, model) => {
-        for (const row of parseCsv(chunk)) await model.insert(row);
-    }
-});
-
-await users.callStream('import', chunk, { source: 'crm' });
-users.getStreams(); // → ['import']
-```
-
----
-
-## Events & Subscriptions
-
-Events are named, schema-validated handlers that clients subscribe to. Unlike `method` (one-shot RPC), events are designed for fan-out to multiple subscribers, scoped by query.
-
-### Defining an event
-
-```ts
-users.event({
-    id      : 'roleChanged',
-    schema  : {
-        type     : 'object',
-        required : ['_id', 'role'],
-        properties: {
-            _id  : { type: 'string' },
-            role : { type: 'string', enum: ['user', 'admin'] }
-        }
-    },
-    handler : async ({ _id, role }, model, ctx) => {
-        return model.update({ _id }, { role });
-    }
-});
-```
-
-If `schema` is set, args are validated before the handler runs. Validation errors throw `{ code: 'VALIDATION_ERROR', errors }`.
-
-### Subscribing
-
-A subscription is keyed by a normalized query — same query keys/values produce the same subscription instance.
-
-```ts
-const { subscription, subId } = users.subscribe({ role: 'admin' });
-
-subscription.on('publish', payload => {
-    console.log('admin update:', payload);
-});
-
-// shorthand — returns just the subscription
-const sub = users.subscription({ role: 'admin' });
-```
-
-### Dispatching an event from a subscription
-
-`dispatch` validates args (if a schema is set) and runs the matching event handler.
-
-```ts
-await sub.dispatch('roleChanged', { _id, role: 'admin' }, ctx);
-```
-
-Throws `{ code: 'EVENT_NOT_FOUND', id }` if the event isn't registered.
-
-### Publishing to subscribers
-
-`publish` is your fan-out hook — it emits to every listener attached via `on('publish', ...)` for that subscription.
-
-```ts
-sub.publish({ type: 'role-changed', _id, role: 'admin' });
-```
-
-A common pattern is calling `publish` from inside an `after:update` hook to notify everyone watching that slice of the model.
-
-### Inspection / cleanup
-
-```ts
-sub.getEvents();                 // → ['roleChanged']
-users.unsubscribe({ role: 'admin' });
-```
-
----
-
-## Internal Event Emitter
-
-Separate from the subscription system, every model is also an `EventEmitter`. CRUD operations emit on this emitter directly — useful for in-process listeners that don't need query scoping.
-
-```ts
-users.on('insert', doc => console.log('inserted', doc));
-users.on('update', ({ key, data }) => console.log('updated', key, data));
-users.on('delete', result => console.log('deleted', result));
-users.on('log',    entry => console.log(entry));   // log feed
-```
-
-`on`, `off`, `once`, `emit` are all available. Built-in events: `insert`, `update`, `delete`, `log`.
-
----
-
-## Extensions
-
-Pass `extensions: [...]` in the constructor options to mount sub-models as named properties on the parent.
-
-```ts
-const ssh   = new Model('ssh',   sshSchema);
-const users = new Model('users', userSchema, { extensions: [ssh] });
-
-users.ssh.insert({ host: '...', port: 22, username: 'root' });
-users.getExtensions(); // → [Model('ssh')]
-```
-
-The mount key is `extension.id`.
-
----
-
-## Adapter (Persistence)
-
-Models are in-memory by default. Provide an `Adapter` subclass to persist on every write and rehydrate on `pull()`.
-
-```ts
-import { Adapter, Model } from '@hotfusion/modeller';
-
-class FileAdapter extends Adapter {
-    constructor(id) {
-        super(id);
-        this.path = `./data/${id}.json`;
-    }
-
-    async sync(id, doc) {
-        // called on every insert / update / delete
-    }
-
-    async pull() {
-        // return the array of documents to load into memory
-        return JSON.parse(await fs.readFile(this.path, 'utf8'));
-    }
-}
-
-const users = new Model('users', schema, { adapter: FileAdapter });
-await users.pull();   // hydrate before serving traffic
-```
-
-When an adapter is configured, deletes are also moved to the trash bin (unless you pass `trash: false` in options).
-
----
-
-## Logging
-
-A built-in logger surfaces hook, worker, method, event, and CRUD activity. Configure it at construction time:
-
-```ts
-const users = new Model('users', schema, {
-    level     : 'debug',     // 'debug' | 'info' | 'warn' | 'error' | 'silent'
-    timestamp : true,
-    features  : { hook: true, worker: true, method: true, event: false, crud: true },
-    onEntry   : entry => myLogger.log(entry)
-});
-```
-
-Or supply a custom adapter:
-
-```ts
-{
-    logAdapter: { log: entry => myStore.append(entry) }
-}
-```
-
-You can also listen to `'log'` on the model itself:
-
-```ts
-users.on('log', entry => sink.write(entry));
-```
-
-Each entry includes `level`, `feature`, `modelId`, `message`, `callSite`, `timestamp`, and optional `data`.
-
----
-
-## Trash
-
-When an adapter is configured, deletes go to an in-memory trash bin keyed by `_id`.
-
-```ts
-await users.delete({ _id });
-
-// elsewhere — direct trash access isn't exposed on Model yet,
-// but Trash supports: restore, has, get, permanentDelete, empty,
-// size, list({ start, count }), find(query), findAll(query).
+await users.insert({ email: 'alex@hotfusion.ca', name: 'Alex' });
 ```
 
-Pass `{ trash: false }` in model options to disable.
-
 ---
 
-## Errors
+## Documentation
 
-All errors thrown by Modeller are plain objects with a `code`:
+| Section                              | Covers                                                                |
+|--------------------------------------|-----------------------------------------------------------------------|
+| [Core layer](./docs/CORE.md)         | Schema, validator, CRUD primitives, indexes, adapter, trash.          |
+| [Model layer](./docs/MODEL.md)       | Hooks, workers, methods, uploads, streams, events, subscriptions, logging, extensions. |
+| [Server layer](./docs/SERVER.md)     | HTTP + WebSocket transport, auth, server events, public API.          |
+| [Utilities](./docs/UTILITIES.md)     | `Bundler`, `KeyGen`, `Adapter`, encryption, JWT helpers.              |
+| [Errors](./docs/ERRORS.md)           | Error codes thrown by the framework.                                  |
+| [Common patterns](./docs/PATTERNS.md)| Recipes for publish-on-update, FK validation, extensions, workers.    |
 
-| Code                | Source                                          |
-|---------------------|-------------------------------------------------|
-| `VALIDATION_ERROR`  | AJV failed on `insert` / `update` / event args  |
-| `UNIQUE_VIOLATION`  | A `unique` field collided                       |
-| `METHOD_NOT_FOUND`  | `call('id', ...)` with unregistered id          |
-| `UPLOAD_NOT_FOUND`  | `callUpload('id', ...)` with unregistered id    |
-| `STREAM_NOT_FOUND`  | `callStream('id', ...)` with unregistered id    |
-| `EVENT_NOT_FOUND`   | `subscription.dispatch('id', ...)` mismatch     |
+Start with **Model** if you're building an app. Drop down to **Core** when you need to understand schema or persistence semantics. Read **Server** when you're ready to expose models over the network.
 
 ---
 
 ## License
 
-UNLICENSED — internal HotFusion project.
+UNLICENSED — internal HotFusion project.

package/docs/CORE.md

@@ -0,0 +1,191 @@
+# Core Layer
+
+`Core` is the in-memory data primitive. It holds documents, validates them against a schema, supports indexed lookups, mingo queries, and an optional persistence adapter. Everything in this doc is inherited by `Model`, so you can ignore Core if you only ever use `Model` — but understanding it explains where the behavior comes from.
+
+> [← Back to README](../README.md)
+
+---
+
+## Table of Contents
+
+- [Schema](#schema)
+- [Validator](#validator)
+- [CRUD primitives](#crud-primitives)
+- [Indexes](#indexes)
+- [Adapter](#adapter)
+- [Trash](#trash)
+
+---
+
+## Schema
+
+Schemas are JSON Schema (AJV-compatible) plus a few Modeller-specific keywords:
+
+| Keyword     | Effect                                                                  |
+|-------------|-------------------------------------------------------------------------|
+| `unique`    | Enforces uniqueness across all documents on `insert` / `update`.        |
+| `private`   | Field stripped from read responses unless `{ private: true }` is set.   |
+| `index`     | Builds an in-memory index on the field for fast lookups.                |
+| `protected` | Reserved for transport-layer protection (consumed by `Server`).         |
+| `hidden`    | Hint for client rendering. Not enforced.                                 |
+| `default`   | Standard JSON Schema default.                                            |
+| `model`     | Async FK-style validation against another model by id.                  |
+
+### `$ref` resolution
+
+`$ref` resolves against `$defs` / `definitions` first, then against any `schemes` passed to the constructor (via `$id` match).
+
+A `$ref` may also carry a query string:
+
+```
+"someModel?key=_id&label=name&search=name,email&columns=name,role&min=1&max=5&unique"
+```
+
+This rewrites the property into a model reference with display metadata (`key`, `label`, `search`, `columns`) — used by client tooling to render pickers. For `type: 'array'`, `min` / `max` / `unique` become `minItems` / `maxItems` / `uniqueItemProperties`.
+
+### Inspecting the schema
+
+```ts
+model.getSchema();                   // public — strips `private` fields
+model.getSchema({ private: true });  // full
+```
+
+---
+
+## Validator
+
+AJV with `allErrors`, `strict: false`, `ajv-formats`, and `ajv-errors`. Two custom formats are registered:
+
+- `phone` — `+?[0-9 \-()]{7,10}`
+- `password` — at least 8 chars, mixed case, digit, symbol
+
+And one custom keyword:
+
+- `model` (async) — value must reference an existing document in the named model. Used for FK-style references.
+
+```ts
+properties: {
+    ownerId: { type: 'string', model: 'users.email' }
+}
+```
+
+---
+
+## CRUD primitives
+
+All CRUD methods are async. They throw `{ code, ... }` on failure (see [Errors](./ERRORS.md)).
+
+### `insert(key?, ...patches, opts?)`
+
+Merges `key` and any patch objects into a single document, validates, assigns a BSON `ObjectId`, and stores it.
+
+```ts
+await users.insert({ email: 'a@b.com' }, { name: 'Alex' });
+await users.insert({ email: 'admin@b.com' }, { role: 'admin' }, { private: true });
+```
+
+Returns the inserted document. `private: true` includes `private` fields in the response.
+
+### `update(key, updates, opts?)`
+
+Finds a document by `key` (mingo query), shallow-merges `updates`, re-validates, and saves.
+
+```ts
+await users.update({ _id }, { name: 'Alex H.' });
+```
+
+Returns the merged document, or `null` if no match.
+
+### `delete(key)`
+
+Removes the matching document. If an adapter is configured and `trash` is not disabled, the document is moved to the trash bin.
+
+```ts
+await users.delete({ _id });   // → { _id }  or  null
+```
+
+### `get(key, opts?)`
+
+Returns a single document or `null`.
+
+```ts
+await users.get({ email: 'a@b.com' });
+await users.get({ _id }, { private: true });
+```
+
+### `list(query?, opts?)`
+
+Paginated, **exact-match** mingo query.
+
+```ts
+await users.list({ role: 'admin' }, { start: 0, count: 25, private: false });
+```
+
+Default `count` is 10.
+
+### `find(query?, opts?)`
+
+Like `list`, but **strings match as case-insensitive substrings**. Non-string values match exactly. Returns an array.
+
+```ts
+await users.find({ name: 'al' }, { max: 10 });
+// → [{ name: 'Alex' }, { name: 'Albert' }, ...]
+```
+
+Default `max` is 10.
+
+> **`list` vs `find`** — `list` is for paginated, exact-match queries (admin views, dashboards). `find` is for autocomplete and search. `list` fires hooks; `find` does not.
+
+---
+
+## Indexes
+
+Any property with `index: <number>` in its schema gets a per-field hash index built and maintained automatically. Indexes are used internally to keep large collections fast.
+
+```ts
+properties: {
+    email : { type: 'string', index: 1 }
+}
+```
+
+---
+
+## Adapter
+
+Models are in-memory by default. Provide an `Adapter` subclass to persist on every write and rehydrate on `pull()`.
+
+```ts
+import { Adapter, Model } from '@hotfusion/modeller';
+
+class FileAdapter extends Adapter {
+    async sync(id, doc) { /* called on every write */ }
+    async pull()        { /* return array of docs to load */ return []; }
+}
+
+const users = new Model('users', schema, { adapter: FileAdapter });
+await users.pull();   // hydrate before serving traffic
+```
+
+The Adapter contract:
+
+```ts
+class Adapter {
+    constructor(id: string) {}
+    async sync(id: string, doc: { _id: string, ... }): Promise<void>;
+    async pull(): Promise<any[]>;
+}
+```
+
+Modeller calls `sync` on every `insert`, `update`, and `delete` — including trash moves. `pull()` is called manually when you want to rehydrate.
+
+---
+
+## Trash
+
+When an adapter is configured, deleted documents go to an in-memory trash bin keyed by `_id`. The bin supports `restore`, `has`, `get`, `permanentDelete`, `empty`, `size`, `list({ start, count })`, `find(query)`, `findAll(query)`. Every trash operation also calls `adapter.sync('trash', doc)` so it can be persisted.
+
+Pass `{ trash: false }` in model options to disable.
+
+---
+
+> **Next:** [Model layer →](./MODEL.md)