@owlmeans/mongo-resource 0.1.15 → 0.1.16

package/README.md

@@ -4,9 +4,11 @@ MongoDB-backed `Resource<T>` implementation — the primary database resource fo
 
 ## Overview
 
-- `makeMongoResource<R, T>(alias, dbAlias?, serviceAlias?, maker?)` — factory for MongoDB resources
-- `MongoResource<T>` — extends `Resource<T>` with MongoDB collection, indexing, and field encryption
-- Supports CRUD, list/pagination, AJV schema validation, and field-level locking (encryption)
+- `makeMongoResource<R, T>(alias, dbAlias?, serviceAlias?, maker?, collectionName?)` — factory for MongoDB resources
+- `MongoResource<T>` — extends `Resource<T>` with MongoDB collection, indexing, field encryption, code migrations and ObjectId references
+- Supports CRUD, list/pagination, AJV schema validation ($jsonSchema collection validators), and field-level locking (encryption)
+- Declared references convert between the string ids records carry and the `ObjectId`s the collection stores — the same way `_id` already does
+- Code migrations run automatically at resource initialization, tracked in a per-database `_owlmeans_migrations` ledger
 - Used for all persistent data models in server applications
 
 ## Installation
@@ -53,19 +55,61 @@ const list = await projects.list({ criteria: { entityId } })
 
 ## API
 
-### `makeMongoResource<R, T>(alias, dbAlias?, serviceAlias?, maker?): T`
+### `makeMongoResource<R, T>(alias, dbAlias?, serviceAlias?, maker?, collectionName?): T`
 
 Creates a MongoDB resource. `dbAlias` defaults to `DEFAULT_DB_ALIAS` (`'mongo'`).
+`collectionName` overrides the physical collection name (otherwise `resourcePrefix + alias`).
+Pass the maker itself as the 4th argument so `schema`/`index()` survive context switches;
+`migration()`/`reference()` survive regardless (module-scope declarations keyed by alias).
 
 ### `MongoResource<T>`
 
-Extends `Resource<T>` with:
+Extends `Resource<T>` (and the shared `MigratableResource<MongoTx>` capability) with:
 - `collection: Collection` — MongoDB collection
-- `db(): Promise<Db>` — get the MongoDB database
+- `db(): Promise<Db>` / `client(): Promise<MongoClient>`
 - `index(name, spec, options?): this` — define a collection index
+- `reference(field, targetAlias?): this` / `references()` — declare that a field stores another record's id (see below)
+- `migration(name, apply, stage?): this` / `migrations()` — register a code migration (see below)
 - `lock(record, fields?)` / `unlock(record, fields?)` — encrypt/decrypt secure fields
 - `getDefaults(): Partial<T>` — default values derived from schema
 
+### ObjectId references
+
+`reference(field, targetAlias?)` declares that a record field references another record's id.
+The resource then treats the field exactly like `_id`:
+
+- Records and criteria carry **strings**; the collection stores **`ObjectId`s**. Conversion is
+  automatic on every read, write and lookup — including `$in`-style operator objects,
+  `$and`/`$or`/`$nor` branches and arrays of ids. `id` criteria are mapped onto `_id`.
+- Writes are strict (a non-24-hex value throws `MisshapedRecord`); reads and criteria are
+  tolerant (a non-id value simply matches nothing).
+- The field gets a mongo-level index (`ref_<field>`) automatically, unless an index with the
+  identical key pattern is already declared, and the collection validator declares it
+  `objectId`.
+- Declaring a reference registers the system migration `$ref:<field>@1` (pre stage) that
+  converts pre-existing string ids in place — idempotent and interrupt-safe. On every boot
+  the collection is additionally probed for convertible strings and repaired if the ledger
+  and the data disagree (the double check). Conversion bypasses document validation, which
+  requires the `bypassDocumentValidation` privilege (`dbOwner`/`root` hold it).
+- Only declare fields whose values really are another record's `id`. Business keys, composite
+  keys, external provider ids and slugs must stay strings — converting them corrupts data.
+- Raw `resource.collection.*` access bypasses the conversion: marshal filter values with
+  `marshalReference(field, value)` and convert read-back ids to strings yourself.
+
+### Migrations
+
+`migration(name, apply, stage?)` registers a code migration, applied once per database in
+declaration order and recorded in the `_owlmeans_migrations` collection (one ledger per
+database — an Entity-layer database tracks its own).
+
+- `MigrationStage.Pre` runs before the validator/index update, `Post` after. On a collection
+  created by this very boot, registered migrations are **baselined** (recorded, not run).
+- Bodies receive a `MongoTx` (`db`, `collection`, `use(alias)`, `ref(alias)`) and must be
+  **idempotent** — multi-document transactions are unavailable on a standalone `mongod`, so
+  the ledger claims-then-completes and an interrupted body may re-run.
+- The checksum fingerprints the body's source text: keep bodies at module scope; an edited
+  applied body raises `MigrationConflict` at boot.
+
 ### `Resource<T>` methods (all implemented)
 
 `get`, `load`, `create`, `update`, `save`, `delete`, `pick`, `list`
@@ -74,6 +118,7 @@ Extends `Resource<T>` with:
 
 - `DEFAULT_DB_ALIAS` — `'mongo'`
 - `DEFAULT_PAGE_SIZE` — `10`
+- `DEF_MIGRATIONS_COLLECTION` — `'_owlmeans_migrations'`
 
 ## Related Packages
 
@@ -83,10 +128,9 @@ Extends `Resource<T>` with:
 <!-- owlmeans:agent-guidance:start -->
 ## Agent guidance
 
-This package ships embedded Claude Code skills and GitHub Copilot instructions under
-`agent-meta/`. After installing your `@owlmeans/*` packages, run the OwlMeans
-agent-skills installer to place them into your project's native locations
-(`.claude/skills/` and `.github/instructions/`):
+This package ships embedded agent skills under `agent-meta/`. After installing your
+`@owlmeans/*` packages, run the OwlMeans agent-skills installer to place them into
+your project's skill store (`.agents/skills/`):
 
 ```sh
 npx @owlmeans/agent-skills

package/agent-meta/manifest.json

@@ -1,8 +1,8 @@
 {
-  "schemaVersion": 1,
+  "schemaVersion": 2,
   "package": "@owlmeans/mongo-resource",
-  "version": "0.1.15",
-  "generatedAt": "2026-08-07T17:06:32.896Z",
+  "version": "0.1.16",
+  "generatedAt": "2026-08-14T10:14:48.849Z",
   "canonicalRepo": "https://github.com/owlmeans/common",
   "entries": [
     {
@@ -10,14 +10,7 @@
       "name": "mongo-resource",
       "category": "package-specific",
       "file": "skills/mongo-resource/SKILL.md",
-      "canonicalPath": ".claude/skills/mongo-resource/SKILL.md"
-    },
-    {
-      "kind": "instruction",
-      "name": "mongo-resource",
-      "category": "package-specific",
-      "file": "instructions/mongo-resource.instructions.md",
-      "canonicalPath": ".github/instructions/mongo-resource.instructions.md"
+      "canonicalPath": ".agents/skills/mongo-resource/SKILL.md"
     }
   ]
 }

package/agent-meta/skills/mongo-resource/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: mongo-resource
-description: How to use @owlmeans/mongo-resource — MongoDB-backed Resource implementation. Builds a typed CRUD resource on top of @owlmeans/mongo. Auto-invoked when defining a resource backed by MongoDB.
+description: How to use @owlmeans/mongo-resource — MongoDB-backed Resource implementation with AJV-schema validators, code migrations and ObjectId reference conversion. Auto-invoked when defining a resource backed by MongoDB, declaring record references, or writing mongo migrations.
 user-invocable: false
 ---
 <!-- AUTO-GENERATED — do not edit. Regenerate via sync-agent-meta. -->
@@ -8,32 +8,165 @@ user-invocable: false
 # @owlmeans/mongo-resource
 
 **Layer:** Infra
-**Install:** `"@owlmeans/mongo-resource": "^0.1.15"` in `dependencies`
+**Install:** `"@owlmeans/mongo-resource": "^0.1.16"` in `dependencies` (peers `mongodb`, `ajv`)
+
+The Mongo counterpart of [[postgres-resource]]. A collection has no structure of its own, so
+here the resource layer owns the *validator* (`$jsonSchema` from the AJV schema), the indexes,
+the code migrations, and the string↔`ObjectId` conversion for `_id` **and every declared
+reference**.
 
 ## Key Exports
 
 | Export | Description |
 |--------|-------------|
-| `makeMongoResource<T>(options)` | Factory for a MongoDB-backed Resource<T> |
-| `MongoResource<T>` types | Resource interface |
-| Constants | Default collection prefix |
-| Helpers | Index creation, query helpers |
+| `makeMongoResource<R, T>(alias, dbAlias?, serviceAlias?, maker?, collectionName?)` | The resource factory. Aliases default to `DEFAULT_DB_ALIAS` (`'mongo'`). `collectionName` overrides the collection (else `resourcePrefix + alias`). |
+| `MongoResource<T>` | `Resource<T>` + `collection`, `index`, `reference`/`references`, `migration`/`migrations` (the shared `MigratableResource` capability), `lock`/`unlock`, `getDefaults`. |
+| `MongoDbService`, `MongoTx` | Service contract implemented by `@owlmeans/mongo`; the façade handed to migrations (`db`, `collection`, `use(alias)`, `ref(alias)`). |
+| `MongoReference`, `MongoRefOptions` | A declared ObjectId reference and the `reference()` options (`resource`, `noIndex`). |
+| `marshalReference`, `demarshalReference`, `marshalCriteria`, `identityCriteria`, `isObjectIdHex` | The conversion layer — reuse these wherever raw driver access bypasses the resource. |
+| `convertReferenceField`, `reconcileReferences`, `refMigrationName` | The system reference migration's machinery. |
+| `makeMongoTx`, `makeMongoMigrationStore` | The migration store (ledger) implementation. |
+| `getDeclaration`, `resetDeclarations` | Module-scope migration/reference declarations, keyed by alias. |
+| `schemaToMongoSchema`, `applyReferenceTypes`, `mongoCollectionName`, `updateIndexes` | Validator compilation and lifecycle helpers (deep import `utils/`). |
+| `DEFAULT_DB_ALIAS`, `DEFAULT_PAGE_SIZE`, `DEF_MIGRATIONS_COLLECTION` | Constants. |
 
-## Usage
+## Usage — the maker pattern
 
 ```typescript
-import { makeMongoResource } from '@owlmeans/mongo-resource'
-
-context.registerResource(makeMongoResource<Project>({
-  alias: 'projects',
-  collection: 'projects',
-}))
+export const makeProjectStoryResource: ResourceMaker<ProjectStoryRecord, ProjectStoryResource> =
+  (dbAlias, serviceAlias) => {
+    const resource = makeMongoResource<ProjectStoryRecord, ProjectStoryResource>(
+      RES_PROJECT_STORY, dbAlias, serviceAlias, makeProjectStoryResource
+    )
+    resource.schema = ProjectStorySchema
+    resource.reference('projectId', RES_PROJECT)
+    resource.index('code', { projectId: 1, code: 1 }, { sparse: true })
+    resource.migration('0001-backfill-code', async tx => { /* ... */ })
 
-// In a handler:
-const projects = ctx.getResource<Resource<Project>>('projects')
-await projects.create({ name: 'New', entityId })
+    return resource
+  }
+context.registerResource(makeProjectStoryResource())
 ```
 
+Pass the maker itself as the 4th argument — `reinitializeContext` re-runs it, which is what
+carries `schema` and `index()` calls across context switches. `migration()` and `reference()`
+survive regardless: they live in module-scope declarations keyed by alias, because losing one
+silently loses a data transformation.
+
+## ObjectId references
+
+A field that stores **another record's id** is declared with `reference(field, targetAlias?)`.
+The resource then behaves for that field exactly as it does for `_id`:
+
+- **Records and criteria carry strings; the collection stores `ObjectId`s.** Conversion is
+  automatic in `create`/`update`/`save` (write), `get`/`load`/`list`/`delete`/`pick` (read and
+  lookup), and in `list` criteria — including `$in`/`$ne`-style operator objects, `$and`/`$or`/
+  `$nor` branches, and arrays of ids (elementwise). `$regex`/`$type`-style operands are left
+  alone.
+- **Writes are strict**: storing a non-24-hex value in a declared reference throws
+  `MisshapedRecord('ref:<field>')` — a silent string would reintroduce the mixed-type state.
+  Reads and criteria are tolerant: an unconverted legacy string comes back as-is; a non-id
+  criteria value simply matches nothing (the auth `userId ?? profileId` fallback relies on
+  this).
+- **`id` criteria address `_id`.** Documents never store an `id` field, so `list({ id })` and
+  `load(x, 'id')` are mapped onto `_id` with conversion — before this mapping they silently
+  matched nothing.
+- **The field is indexed** automatically (`ref_<field>`), unless `noIndex: true` or the
+  resource already declares an index with the identical key pattern (mongo forbids two
+  indexes over the same keys; a declared unique index wins).
+- **The validator declares the field `objectId`** (nullable/array shapes carry over from the
+  AJV property) — after the switch a raw string write is rejected at the collection level.
+
+### The system migration and its double check
+
+Declaring a reference registers `$ref:<field>@1` at `Pre` stage: an idempotent, interrupt-safe
+`updateMany` that converts stored 24-hex strings (scalar or array elements) to `ObjectId`s and
+leaves everything else untouched. On restart the ledger says whether it ran; **independently**,
+after structure update the boot probes the collection for convertible strings and repairs any
+drift (restored backup, legacy writer), logging a warning. Both paths run with
+`bypassDocumentValidation` — the connection's user must hold that privilege (`dbOwner`/`root`
+do).
+
+The `@1` in the name is the body's version. The body is shared by every field, so its checksum
+never distinguishes them — **any semantic edit to `convertReferenceField` must bump the
+version suffix** in `refMigrationName`, or every already-applied ledger raises
+`MigrationConflict` at boot.
+
+### What is NOT a reference — do not declare these
+
+Only fields assigned from another record's `.id` qualify. Known traps from the live codebase:
+
+- `entityId` / `entity` — IAM entity slug (also a Keycloak realm and a k8s namespace label)
+- `profileId` — composite key `"{type}:{accountId}"`; `credentials.userId` — external
+  provider key `"{type}:{service}:{sub}"` (while `profile.userId` **is** a reference)
+- Stripe ids (`externalId`, `productId`, `taxId`), Cloudflare `providerId`, GitHub numeric ids
+- locally minted slugs/tokens (`linkId`, `alias`, `slug`, `code`, `credential`)
+
+Converting one of these corrupts the collection and breaks unique indexes. When in doubt,
+trace what the writer actually assigns.
+
+### Raw driver access bypasses all of this
+
+`resource.collection.find/aggregate/findOneAndUpdate` see `ObjectId`s. Marshal filter values
+with `marshalReference(field, value)` and convert read-back documents' reference fields (and
+`_id`) to strings by hand — or better, stay on the resource API.
+
+## Migrations
+
+`resource.migration(name, apply, stage?)` — the shared `MigratableResource` capability from
+[[resource]]. Applied once per database, in declaration order, ledgered in
+`_owlmeans_migrations` (one ledger per database, so an Entity-layer database tracks its own).
+
+- `Pre` runs **before** the validator is updated and indexes reconcile; `Post` after. A `Pre`
+  body writes shapes the *old* validator allows; a `Post` body the *new* one.
+- On a collection this boot just created, every registered migration is **baselined**
+  (recorded, not run) — a fresh collection is born at head.
+- **No transactions**: a standalone `mongod` (the dev/CI target) rejects multi-document
+  transactions, so the ledger claims-then-completes — the unique `(alias, name)` index is the
+  mutual exclusion; a replica losing the race waits for the winner; a failed body withdraws
+  the claim so the next boot retries. Consequence: **write migration bodies idempotent** —
+  they may be interrupted and re-run.
+- The checksum fingerprints the body's **source text**. Keep bodies at module scope; an edited
+  applied body raises `MigrationConflict`, a throwing one `MigrationError` and the boot
+  aborts. A body that closes over a loop variable fingerprints the wrapper — the trap
+  `createMigrationRegistry` documents.
+- Inside a body, `tx.use(alias)` / `tx.ref(alias)` address other registered resources'
+  collections by alias — resolved from config, so registration order does not matter (unlike
+  Postgres `{{alias}}`).
+
+## Lifecycle order at `init()`
+
+1. probe for the collection
+2. absent → baseline all migrations; present → run `Pre` (system `$ref:` first if declared before app migrations)
+3. create collection (validator + indexes) or update validator + reconcile indexes
+4. present → run `Post`
+5. reconcile declared references (probe + repair — the double check)
+
+## Method semantics worth remembering
+
+| Method | Semantics |
+|---|---|
+| `create` | refuses a caller-supplied id (`RecordExists`) |
+| `update` | **replaces** the whole record (no merge) |
+| `pick` | deletes the record it returns |
+| `load`/`get` | rejects `opts.ttl` (`UnsupportedArgumentError`); second arg selects the lookup field |
+| `list` | criteria go through reference conversion; documents never store `id` — use `id` criteria freely, they map to `_id` |
+| `lock`/`unlock` | encrypt/decrypt `secure: true` schema fields via the db service |
+
+## Tests
+
+Unit specs (conversion layer): `bun test ./tests` in this package — ungated. Integration
+specs that build a real `ServerContext` live in `@owlmeans/mongo` (`migration.spec.ts`,
+`references.spec.ts`), gated on `MONGO_URL` (see [[testing-integration]]); a dev port-forward
+to the cluster mongo satisfies the checked-in `.env`.
+
 ## Depends On
 
-- `@owlmeans/mongo`, `@owlmeans/resource`, `@owlmeans/server-context`
+- `@owlmeans/resource` · `@owlmeans/context` · `@owlmeans/server-context`
+- peer `mongodb`, `ajv`
+
+## Related
+
+- [[mongo]] — the connection service this resolves through
+- [[resource]] — `Resource<T>`, `MigratableResource`, the migration framework, errors
+- [[postgres-resource]] — the Postgres counterpart (structure reconciliation instead of validators)

package/build/consts.d.ts

@@ -1,3 +1,19 @@
 export declare const DEFAULT_DB_ALIAS = "mongo";
 export declare const DEFAULT_PAGE_SIZE = 10;
+/**
+ * Collection that records which code-registered migrations have already been applied.
+ *
+ * One ledger per database, which is the right boundary: `dbName()` already varies the
+ * database per Entity/User layer, so a tenant's migrations are tracked with the tenant's
+ * data and dropping the database drops the ledger with it.
+ */
+export declare const DEF_MIGRATIONS_COLLECTION = "_owlmeans_migrations";
+/**
+ * How long a replica waits for another replica's in-flight migration before giving up.
+ * Bounded because the alternative is a pod that hangs on boot with no diagnostic.
+ */
+export declare const DEF_MIGRATION_WAIT = 60000;
+export declare const DEF_MIGRATION_POLL = 250;
+/** `E11000` — the unique index on `(alias, name)` rejecting a second replica's claim. */
+export declare const MONGO_DUPLICATE_KEY = 11000;
 //# sourceMappingURL=consts.d.ts.map

package/build/consts.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"consts.d.ts","sourceRoot":"","sources":["../src/consts.ts"],"names":[],"mappings":"AACA,eAAO,MAAM,gBAAgB,UAAU,CAAA;AAEvC,eAAO,MAAM,iBAAiB,KAAK,CAAA"}
+{"version":3,"file":"consts.d.ts","sourceRoot":"","sources":["../src/consts.ts"],"names":[],"mappings":"AACA,eAAO,MAAM,gBAAgB,UAAU,CAAA;AAEvC,eAAO,MAAM,iBAAiB,KAAK,CAAA;AAEnC;;;;;;GAMG;AACH,eAAO,MAAM,yBAAyB,yBAAyB,CAAA;AAE/D;;;GAGG;AACH,eAAO,MAAM,kBAAkB,QAAQ,CAAA;AAEvC,eAAO,MAAM,kBAAkB,MAAM,CAAA;AAErC,yFAAyF;AACzF,eAAO,MAAM,mBAAmB,QAAQ,CAAA"}

package/build/consts.js

@@ -1,3 +1,19 @@
 export const DEFAULT_DB_ALIAS = 'mongo';
 export const DEFAULT_PAGE_SIZE = 10;
+/**
+ * Collection that records which code-registered migrations have already been applied.
+ *
+ * One ledger per database, which is the right boundary: `dbName()` already varies the
+ * database per Entity/User layer, so a tenant's migrations are tracked with the tenant's
+ * data and dropping the database drops the ledger with it.
+ */
+export const DEF_MIGRATIONS_COLLECTION = '_owlmeans_migrations';
+/**
+ * How long a replica waits for another replica's in-flight migration before giving up.
+ * Bounded because the alternative is a pod that hangs on boot with no diagnostic.
+ */
+export const DEF_MIGRATION_WAIT = 60000;
+export const DEF_MIGRATION_POLL = 250;
+/** `E11000` — the unique index on `(alias, name)` rejecting a second replica's claim. */
+export const MONGO_DUPLICATE_KEY = 11000;
 //# sourceMappingURL=consts.js.map

package/build/consts.js.map

@@ -1 +1 @@
-{"version":3,"file":"consts.js","sourceRoot":"","sources":["../src/consts.ts"],"names":[],"mappings":"AACA,MAAM,CAAC,MAAM,gBAAgB,GAAG,OAAO,CAAA;AAEvC,MAAM,CAAC,MAAM,iBAAiB,GAAG,EAAE,CAAA"}
+{"version":3,"file":"consts.js","sourceRoot":"","sources":["../src/consts.ts"],"names":[],"mappings":"AACA,MAAM,CAAC,MAAM,gBAAgB,GAAG,OAAO,CAAA;AAEvC,MAAM,CAAC,MAAM,iBAAiB,GAAG,EAAE,CAAA;AAEnC;;;;;;GAMG;AACH,MAAM,CAAC,MAAM,yBAAyB,GAAG,sBAAsB,CAAA;AAE/D;;;GAGG;AACH,MAAM,CAAC,MAAM,kBAAkB,GAAG,KAAK,CAAA;AAEvC,MAAM,CAAC,MAAM,kBAAkB,GAAG,GAAG,CAAA;AAErC,yFAAyF;AACzF,MAAM,CAAC,MAAM,mBAAmB,GAAG,KAAK,CAAA"}

package/build/declarations.d.ts

@@ -0,0 +1,11 @@
+import type { MigrationRegistry } from '@owlmeans/resource';
+import type { MongoReference, MongoTx } from './types.js';
+export interface MongoDeclaration {
+    migrations: MigrationRegistry<MongoTx>;
+    /** Declared ObjectId references, keyed by field. Registered via `resource.reference()`. */
+    references: Map<string, MongoReference>;
+}
+export declare const getDeclaration: (alias: string) => MongoDeclaration;
+/** Testing seam — drops every declaration so a spec can redeclare a resource from scratch. */
+export declare const resetDeclarations: (alias?: string) => void;
+//# sourceMappingURL=declarations.d.ts.map

package/build/declarations.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"declarations.d.ts","sourceRoot":"","sources":["../src/declarations.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EAAE,iBAAiB,EAAE,MAAM,oBAAoB,CAAA;AAE3D,OAAO,KAAK,EAAE,cAAc,EAAE,OAAO,EAAE,MAAM,YAAY,CAAA;AAEzD,MAAM,WAAW,gBAAgB;IAC/B,UAAU,EAAE,iBAAiB,CAAC,OAAO,CAAC,CAAA;IACtC,2FAA2F;IAC3F,UAAU,EAAE,GAAG,CAAC,MAAM,EAAE,cAAc,CAAC,CAAA;CACxC;AAcD,eAAO,MAAM,cAAc,GAAI,OAAO,MAAM,KAAG,gBAQ9C,CAAA;AAED,8FAA8F;AAC9F,eAAO,MAAM,iBAAiB,GAAI,QAAQ,MAAM,KAAG,IAOlD,CAAA"}

package/build/declarations.js

@@ -0,0 +1,29 @@
+import { createMigrationRegistry } from '@owlmeans/resource';
+/**
+ * Per-alias migration store, held at module scope rather than on the resource object.
+ *
+ * `reinitializeContext` rebuilds every resource from the maker, which drops anything a
+ * caller attached by chaining afterwards. For indexes that is survivable — they already
+ * exist in the database and `updateIndexes` only ever adds. Migrations are not: a layer
+ * switch points the resource at a *different* database, and a registry emptied by the
+ * rebuild would mean the entity database silently never gets the transformation. Keying
+ * by alias makes the declarations outlive any number of context switches.
+ */
+const declarations = new Map();
+export const getDeclaration = (alias) => {
+    let declaration = declarations.get(alias);
+    if (declaration == null) {
+        declaration = { migrations: createMigrationRegistry(), references: new Map() };
+        declarations.set(alias, declaration);
+    }
+    return declaration;
+};
+/** Testing seam — drops every declaration so a spec can redeclare a resource from scratch. */
+export const resetDeclarations = (alias) => {
+    if (alias == null) {
+        declarations.clear();
+        return;
+    }
+    declarations.delete(alias);
+};
+//# sourceMappingURL=declarations.js.map

package/build/declarations.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"declarations.js","sourceRoot":"","sources":["../src/declarations.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,uBAAuB,EAAE,MAAM,oBAAoB,CAAA;AAW5D;;;;;;;;;GASG;AACH,MAAM,YAAY,GAAkC,IAAI,GAAG,EAAE,CAAA;AAE7D,MAAM,CAAC,MAAM,cAAc,GAAG,CAAC,KAAa,EAAoB,EAAE;IAChE,IAAI,WAAW,GAAG,YAAY,CAAC,GAAG,CAAC,KAAK,CAAC,CAAA;IACzC,IAAI,WAAW,IAAI,IAAI,EAAE,CAAC;QACxB,WAAW,GAAG,EAAE,UAAU,EAAE,uBAAuB,EAAW,EAAE,UAAU,EAAE,IAAI,GAAG,EAAE,EAAE,CAAA;QACvF,YAAY,CAAC,GAAG,CAAC,KAAK,EAAE,WAAW,CAAC,CAAA;IACtC,CAAC;IAED,OAAO,WAAW,CAAA;AACpB,CAAC,CAAA;AAED,8FAA8F;AAC9F,MAAM,CAAC,MAAM,iBAAiB,GAAG,CAAC,KAAc,EAAQ,EAAE;IACxD,IAAI,KAAK,IAAI,IAAI,EAAE,CAAC;QAClB,YAAY,CAAC,KAAK,EAAE,CAAA;QAEpB,OAAM;IACR,CAAC;IACD,YAAY,CAAC,MAAM,CAAC,KAAK,CAAC,CAAA;AAC5B,CAAC,CAAA"}

package/build/index.d.ts

@@ -1,5 +1,8 @@
 export type * from './types.js';
 export * from './consts.js';
+export * from './declarations.js';
+export * from './utils/migrations.js';
+export * from './utils/refs.js';
 export * from './resource.js';
 export * from './helper.js';
 //# sourceMappingURL=index.d.ts.map

package/build/index.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AACA,mBAAmB,YAAY,CAAA;AAC/B,cAAc,aAAa,CAAA;AAC3B,cAAc,eAAe,CAAA;AAC7B,cAAc,aAAa,CAAA"}
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AACA,mBAAmB,YAAY,CAAA;AAC/B,cAAc,aAAa,CAAA;AAC3B,cAAc,mBAAmB,CAAA;AACjC,cAAc,uBAAuB,CAAA;AACrC,cAAc,iBAAiB,CAAA;AAC/B,cAAc,eAAe,CAAA;AAC7B,cAAc,aAAa,CAAA"}

package/build/index.js

@@ -1,4 +1,7 @@
 export * from './consts.js';
+export * from './declarations.js';
+export * from './utils/migrations.js';
+export * from './utils/refs.js';
 export * from './resource.js';
 export * from './helper.js';
 //# sourceMappingURL=index.js.map

package/build/index.js.map

@@ -1 +1 @@
-{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAEA,cAAc,aAAa,CAAA;AAC3B,cAAc,eAAe,CAAA;AAC7B,cAAc,aAAa,CAAA"}
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAEA,cAAc,aAAa,CAAA;AAC3B,cAAc,mBAAmB,CAAA;AACjC,cAAc,uBAAuB,CAAA;AACrC,cAAc,iBAAiB,CAAA;AAC/B,cAAc,eAAe,CAAA;AAC7B,cAAc,aAAa,CAAA"}

package/build/resource.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"resource.d.ts","sourceRoot":"","sources":["../src/resource.ts"],"names":[],"mappings":"AAGA,OAAO,KAAK,EAAgB,aAAa,EAAE,cAAc,EAAE,MAAM,oBAAoB,CAAA;AAErF,OAAO,KAAK,EAAkB,aAAa,EAAE,MAAM,YAAY,CAAA;AAU/D,eAAO,MAAM,iBAAiB,GAC5B,CAAC,SAAS,cAAc,EAAE,CAAC,SAAS,aAAa,CAAC,CAAC,CAAC,GAAG,aAAa,CAAC,CAAC,CAAC,EAEvE,OAAO,MAAM,EAAE,UAAS,MAAyB,EAAE,eAAc,MAAyB,EAC1F,qBAAqB,aAAa,CAAC,CAAC,EAAE,CAAC,CAAC,EAAE,iBAAiB,MAAM,KAChE,CAyRF,CAAA"}
+{"version":3,"file":"resource.d.ts","sourceRoot":"","sources":["../src/resource.ts"],"names":[],"mappings":"AAIA,OAAO,KAAK,EAAgB,aAAa,EAAE,cAAc,EAAE,MAAM,oBAAoB,CAAA;AAErF,OAAO,KAAK,EAAmD,aAAa,EAAW,MAAM,YAAY,CAAA;AAezG,eAAO,MAAM,iBAAiB,GAC5B,CAAC,SAAS,cAAc,EAAE,CAAC,SAAS,aAAa,CAAC,CAAC,CAAC,GAAG,aAAa,CAAC,CAAC,CAAC,EAEvE,OAAO,MAAM,EAAE,UAAS,MAAyB,EAAE,eAAc,MAAyB,EAC1F,qBAAqB,aAAa,CAAC,CAAC,EAAE,CAAC,CAAC,EAAE,iBAAiB,MAAM,KAChE,CAuTF,CAAA"}

package/build/resource.js

@@ -1,11 +1,24 @@
 import { appendContextual, assertContext } from '@owlmeans/context';
 import { DEFAULT_DB_ALIAS, DEFAULT_PAGE_SIZE } from './consts.js';
+import { MigrationStage } from '@owlmeans/resource';
 import { initializeCollection } from './utils/life-cycle.js';
+import { getDeclaration } from './declarations.js';
 import { ObjectId } from 'mongodb';
 import { MisshapedRecord, RecordExists, UnknownRecordError, UnsupportedArgumentError, RecordUpdateFailed, prepareListOptions } from '@owlmeans/resource';
 import { getSchemaSecureFeilds } from './helper.js';
+import { demarshalRefs, identityCriteria, makeRefMigration, marshalCriteria, marshalReference, refMigrationName } from './utils/refs.js';
 export const makeMongoResource = (alias, dbAlias = DEFAULT_DB_ALIAS, serviceAlias = DEFAULT_DB_ALIAS, makeCustomResource, collectionName) => {
     const location = `mongo-resource:${alias}`;
+    /**
+     * Live view — references may be declared after the resource is built, and the
+     * declarations are module scoped so they survive `reinitializeContext`.
+     */
+    const refs = () => getDeclaration(alias).references;
+    const demarshal = (record) => {
+        record.id = record._id instanceof ObjectId ? record._id.toString() : record._id;
+        delete record._id;
+        return demarshalRefs(record, refs());
+    };
     const resource = appendContextual(alias, {
         get: async (id, field, opts) => {
             const record = await resource.load(id, field, opts);
@@ -20,14 +33,12 @@ export const makeMongoResource = (alias, dbAlias = DEFAULT_DB_ALIAS, serviceAlia
                 field = field.field;
             }
             field = field ?? '_id';
-            const criteria = '_id' === field ? new ObjectId(id) : id;
             if (opts?.ttl != null) {
                 throw new UnsupportedArgumentError('ttl');
             }
-            const record = await resource.collection.findOne({ [field]: criteria });
+            const record = await resource.collection.findOne(identityCriteria(field, id, refs()));
             if (record != null) {
-                record.id = record._id instanceof ObjectId ? record._id.toString() : record._id;
-                delete record._id;
+                demarshal(record);
             }
             return record;
         },
@@ -47,12 +58,11 @@ export const makeMongoResource = (alias, dbAlias = DEFAULT_DB_ALIAS, serviceAlia
                 throw new MisshapedRecord('id');
             }
             const original = await resource.get(id, field);
-            const criteria = '_id' === field ? new ObjectId(id) : id;
             const replace = { ...record, _id: new ObjectId(original.id) };
             if (replace.id != null) {
                 delete replace.id;
             }
-            const result = await resource.collection.replaceOne({ [field]: criteria }, _prepareValues(replace, resource.schema));
+            const result = await resource.collection.replaceOne(identityCriteria(field, id, refs()), _prepareValues(replace, resource.schema, refs()));
             if (!result.acknowledged) {
                 throw new RecordUpdateFailed(`${field}:${id}`);
             }
@@ -90,7 +100,7 @@ export const makeMongoResource = (alias, dbAlias = DEFAULT_DB_ALIAS, serviceAlia
             }
             const result = await resource.collection.insertOne({
                 ...resource.getDefaults(),
-                ..._prepareValues(record, resource.schema)
+                ..._prepareValues(record, resource.schema, refs())
             });
             if (!result.acknowledged) {
                 throw new RecordUpdateFailed(`creation`);
@@ -124,8 +134,7 @@ export const makeMongoResource = (alias, dbAlias = DEFAULT_DB_ALIAS, serviceAlia
             if (id == null) {
                 throw new MisshapedRecord('id');
             }
-            const criteria = '_id' === field ? new ObjectId(_id) : _id;
-            const result = await resource.collection.deleteOne({ [field]: criteria });
+            const result = await resource.collection.deleteOne(identityCriteria(field, _id, refs()));
             if (!result.acknowledged || result.deletedCount === 0) {
                 return null;
             }
@@ -140,7 +149,7 @@ export const makeMongoResource = (alias, dbAlias = DEFAULT_DB_ALIAS, serviceAlia
         },
         list: async (criteria, opts) => {
             const options = prepareListOptions(DEFAULT_PAGE_SIZE, criteria, opts);
-            criteria = options.criteria;
+            criteria = marshalCriteria(options.criteria, refs()) ?? {};
             const pager = options.pager ?? {};
             const size = pager?.size ?? DEFAULT_PAGE_SIZE;
             const total = await resource.collection.countDocuments(criteria);
@@ -158,12 +167,7 @@ export const makeMongoResource = (alias, dbAlias = DEFAULT_DB_ALIAS, serviceAlia
             }
             const items = await cursor.toArray();
             return {
-                pager, items: items.map(item => {
-                    const _item = { ...item };
-                    _item.id = item._id.toString();
-                    delete _item._id;
-                    return _item;
-                })
+                pager, items: items.map(item => demarshal({ ...item }))
             };
         },
         lock: async (record, fields) => {
@@ -200,7 +204,30 @@ export const makeMongoResource = (alias, dbAlias = DEFAULT_DB_ALIAS, serviceAlia
             resource.indexes = resource.indexes ?? [];
             resource.indexes.push({ name, index, options });
             return resource;
-        }
+        },
+        /**
+         * `migration` and `reference` are `this`-returning in the interface, which an object
+         * literal can't express — hence the member level casts: the implementations return
+         * the closed over `resource`, which is that very object.
+         */
+        migration: ((name, apply, stage) => {
+            getDeclaration(alias).migrations.register(name, apply, stage);
+            return resource;
+        }),
+        migrations: () => getDeclaration(alias).migrations,
+        reference: ((field, opts) => {
+            const declaration = getDeclaration(alias);
+            const options = typeof opts === 'string' ? { resource: opts } : opts ?? {};
+            declaration.references.set(field, { field, resource: options.resource, noIndex: options.noIndex });
+            /**
+             * The system migration that converts the field's pre-existing string ids. Registered
+             * here rather than at init so it precedes migrations the app declares after its
+             * `reference()` calls — the field's type contract is the foundation those build on.
+             */
+            declaration.migrations.register(refMigrationName(field), makeRefMigration(field), MigrationStage.Pre);
+            return resource;
+        }),
+        references: () => [...getDeclaration(alias).references.values()]
     });
     // Explicit collection name override (decoupled from the registration alias, which may
     // contain characters that aren't valid in a collection name). Survives reinitializeContext
@@ -214,7 +241,7 @@ export const makeMongoResource = (alias, dbAlias = DEFAULT_DB_ALIAS, serviceAlia
         await mongo.ready();
         const db = await mongo.db(dbAlias);
         const config = mongo.config(dbAlias);
-        resource.collection = await initializeCollection(db, config, resource);
+        resource.collection = await initializeCollection(db, config, resource, context);
     };
     resource.reinitializeContext = (context) => {
         const resource = (makeCustomResource?.(dbAlias, serviceAlias)
@@ -224,15 +251,23 @@ export const makeMongoResource = (alias, dbAlias = DEFAULT_DB_ALIAS, serviceAlia
     };
     return resource;
 };
-const _prepareValues = (obj, schema) => {
+const _prepareValues = (obj, schema, refs) => {
+    /**
+     * Declared references convert independently of the schema — the schema is optional,
+     * and where it exists it declares these fields as strings, whose coercion below would
+     * undo the conversion.
+     */
+    if (refs != null && refs.size > 0) {
+        obj = Object.fromEntries(Object.entries(obj).map(([key, value]) => refs.has(key) ? [key, marshalReference(key, value)] : [key, value]));
+    }
     // @TODO Validate keys from additional properties in the root
     return schema != null ? Object.fromEntries(Object.entries(obj).map(([key, value]) => {
-        // @TODO What if we reference another table by object id in another field?
-        // How to properly transform them?
-        // What if _id isn't an Object Id?
-        if (key === '_id') {
+        if (key === '_id' && !(value instanceof ObjectId)) {
             return [key, new ObjectId(value)];
         }
+        if (key === '_id' || refs?.has(key)) {
+            return [key, value];
+        }
         // A null/undefined value has nothing to coerce — pass it through for any declared
         // type. Without this guard the object-map and array branches below call
         // `Object.entries`/`.map` on `undefined` and throw (and `new Date(null)` would