@rudderjs/orm 1.8.1 → 1.9.1

package/README.md

@@ -298,6 +298,45 @@ await user!.loadMissing('profile', 'posts')
 
 ---
 
+## Vector search
+
+`whereVectorSimilarTo(column, query, opts?)` runs a pgvector similarity query on the column. The terminal call (`get`/`first`/`all`) routes through raw SQL because pgvector ops (`<=>`, `<->`, `<#>`) can't be expressed in the fluent select APIs of either adapter, but everything else — `where(...)` chains, `limit`, `offset`, hydration, casts — composes normally.
+
+```ts
+const matches = await Document
+  .where('tenantId', tenantId)
+  .whereVectorSimilarTo('embedding', queryEmbedding, {
+    metric:        'cosine',  // 'cosine' (default) | 'euclidean' | 'inner_product'
+    minSimilarity: 0.75,       // optional — 1 - distance threshold
+    limit:         10,
+  })
+  .selectVectorDistance('embedding', queryEmbedding, 'distance')  // optional projected column
+  .get()
+
+// matches: Document[] with .distance set when selectVectorDistance() is used
+```
+
+**`query`** is either a `number[]` (pre-computed embedding) or a string — when a string, the adapter calls the registered embedder (see `@rudderjs/ai`'s `similaritySearch` lift) to produce the vector. Without a registered embedder, passing a string throws `MissingEmbedderError`.
+
+**Requirements** — Postgres + the pgvector extension, and a `vector(N)` column on the table:
+
+```sql
+CREATE EXTENSION IF NOT EXISTS vector;
+ALTER TABLE documents ADD COLUMN embedding vector(1536);
+```
+
+Helper: `pnpm rudder make:migration <name> --vector` scaffolds a migration with the `CREATE EXTENSION` + column add already filled in.
+
+**Limitations (v1):**
+
+- `whereGroup` / `orWhereGroup` / `whereHas` cannot wrap a `whereVectorSimilarTo` call — the raw-SQL path can't be nested inside a fluent subquery.
+- `.with(...)` and `withCount` / `withSum` / etc. don't compose with vector queries yet.
+- `orderBy` is redundant (vector queries always order by similarity) and throws if combined.
+- `count()` alongside `whereVectorSimilarTo` is not supported.
+- pgvector-only — SQLite/MySQL throw `VectorStorageUnsupportedError`.
+
+---
+
 ## Route model binding
 
 Models opt into route binding by exposing `static routeKey` (defaults to `'id'`) and `static findForRoute(value)`. The router's `router.bind(name, ModelClass)` API picks them up:
@@ -364,9 +403,23 @@ class Post extends Model {
 | `'encrypted'` | Decrypts string | Encrypts string |
 | `'encrypted:array'` | Decrypts + parses JSON | Encrypts JSON |
 | `'encrypted:object'` | Decrypts + parses JSON | Encrypts JSON |
+| `vector({ dimensions })` | `number[]` from pgvector text | `number[]` → pgvector text literal |
 
 Encrypted casts require `@rudderjs/crypt` to be installed.
 
+The `vector` cast is a factory — call it with the column's dimension so writes validate (`VectorDimensionMismatchError` on bad length, `NaN`, or `Infinity` before the SQL ever runs). Pair with `whereVectorSimilarTo()` for similarity search — see [Vector search](#vector-search) below.
+
+```ts
+import { Model, vector } from '@rudderjs/orm'
+
+class Document extends Model {
+  static override casts = {
+    embedding: vector({ dimensions: 1536 }),
+  }
+  declare embedding: number[]
+}
+```
+
 ### Custom cast classes
 
 ```ts

package/boost/skills/orm-models/SKILL.md

@@ -1,267 +1,40 @@
 ---
 name: orm-models
 description: Creating Eloquent-style models, queries, relationships, casts, factories, and API resources in RudderJS
+license: MIT
+appliesTo:
+  - '@rudderjs/orm'
+  - '@rudderjs/orm-prisma'
+trigger: creating or editing a `Model` class under `app/Models/`, writing queries / relationships, defining casts / accessors / factories, or building `JsonResource` API resources
+skip: a route handler that only reads a model — controller-views is enough
+metadata:
+  author: rudderjs
 ---
 
 # ORM Models
 
 ## When to use this skill
 
-Load this skill when you need to create or modify ORM models, write database queries, define casts/accessors/mutators, build model factories for testing, or create JSON API resources.
+Load when you're creating or editing a `Model` class under `app/Models/`, writing queries, defining casts / accessors / factories, or building `JsonResource` API resources. For depth, open the rule file matching your task.
 
-## Key concepts
+## Quick Reference
 
-- **Model base class**: All models extend `Model` from `@rudderjs/orm`. The ORM is adapter-based -- `ModelRegistry.set(adapter)` plugs in the actual DB driver (e.g. Prisma).
-- **Table naming**: Defaults to lowercase class name + `'s'` (e.g. `User` -> `users`). Override with `static table = 'my_table'`.
-- **Primary key**: Defaults to `'id'`. Override with `static primaryKey = 'uuid'`.
-- **Soft deletes**: Set `static softDeletes = true` to make `delete()` set `deletedAt` instead of removing the row.
-- **Decorators**: `@Hidden`, `@Visible`, `@Appends`, `@Cast` configure serialization on instance properties.
-- **Adapter pattern**: The ORM has no runtime DB dependency. The adapter (Prisma, Drizzle, etc.) is registered at boot time.
+| Task | Open |
+|---|---|
+| Define a model — `static table`, `fillable`, `hidden`, `casts`, soft deletes, decorators, custom casts | `rules/defining-models.md` |
+| Query the database — `find` / `where` / `paginate`, eager loading, scopes, soft-delete filters | `rules/querying.md` |
+| CRUD + observers — `create` / `update` / `delete`, observer lifecycle, atomic counters | `rules/crud-and-observers.md` |
+| Test data — `ModelFactory`, `sequence`, states, `.make()` vs `.create()` | `rules/factories.md` |
+| API output — `JsonResource`, `ResourceCollection`, `when` / `whenLoaded` / `mergeWhen` | `rules/resources.md` |
 
-## Step-by-step
+## Key concepts (load once)
 
-### 1. Define a model
-
-```ts
-import { Model, Hidden, Cast, Attribute } from '@rudderjs/orm'
-
-export class Post extends Model {
-  static table = 'posts'
-  static fillable = ['title', 'body', 'authorId']
-  static hidden = ['internalNotes']
-
-  static casts = {
-    isPublished: 'boolean' as const,
-    publishedAt: 'datetime' as const,
-    metadata:    'json' as const,
-  }
-
-  static attributes = {
-    excerpt: Attribute.make({
-      get: (_, attrs) => String(attrs['body'] ?? '').slice(0, 200),
-    }),
-  }
-
-  static appends = ['excerpt']
-}
-```
-
-### 2. Use decorator syntax (alternative)
-
-```ts
-import { Model, Hidden, Cast, Appends, Attribute } from '@rudderjs/orm'
-
-export class User extends Model {
-  static fillable = ['name', 'email', 'password']
-
-  @Hidden password = ''
-  @Cast('boolean') isAdmin = false
-  @Cast('date') createdAt = new Date()
-  @Appends fullName = ''
-
-  static attributes = {
-    fullName: Attribute.make({
-      get: (_, attrs) => `${attrs['firstName']} ${attrs['lastName']}`,
-    }),
-    password: Attribute.make({
-      set: async (v) => {
-        const { hash } = await import('bcrypt')
-        return hash(String(v), 10)
-      },
-    }),
-  }
-}
-```
-
-### 3. Query the database
-
-```ts
-// Basic queries
-const user = await User.find(1)
-const users = await User.all()
-const first = await User.first()
-const total = await User.count()
-
-// Filtered queries
-const admins = await User.where('isAdmin', true).all()
-const page = await User.paginate(1, 15) // { data, total, page, perPage, lastPage }
-
-// Chained query builder
-const results = await User.query()
-  .where('role', 'admin')
-  .where('active', true)
-  .orderBy('createdAt', 'desc')
-  .limit(10)
-  .all()
-
-// Eager loading
-const posts = await Post.with('author', 'comments').all()
-```
-
-### 4. CRUD operations
-
-```ts
-// Create -- triggers creating/created observers
-const post = await Post.create({ title: 'Hello', body: 'World', authorId: 1 })
-
-// Update -- triggers updating/updated observers
-const updated = await Post.update(post.id, { title: 'Updated' })
-
-// Delete -- triggers deleting/deleted observers
-await Post.delete(post.id)
-
-// Soft delete (when softDeletes = true)
-await Post.delete(post.id)          // sets deletedAt
-await Post.restore(post.id)         // clears deletedAt
-await Post.forceDelete(post.id)     // permanent removal
-```
-
-### 5. Scopes
-
-```ts
-export class Post extends Model {
-  static globalScopes = {
-    published: (q) => q.where('isPublished', true),
-  }
-
-  static scopes = {
-    byAuthor: (q, authorId: number) => q.where('authorId', authorId),
-    recent: (q) => q.orderBy('createdAt', 'desc').limit(10),
-  }
-}
-
-// Usage
-const posts = await Post.query().scope('byAuthor', 1).scope('recent').all()
-const allPosts = await Post.query().withoutGlobalScope('published').all()
-```
-
-### 6. Observers
-
-```ts
-import type { ModelObserver } from '@rudderjs/orm'
-
-class PostObserver implements ModelObserver {
-  creating(data: Record<string, unknown>) {
-    data['slug'] = slugify(String(data['title']))
-    return data
-  }
-
-  deleted(id: string | number) {
-    console.log(`Post ${id} was deleted`)
-  }
-}
-
-Post.observe(PostObserver)
-
-// Or inline listeners
-Post.on('creating', (data) => { data['slug'] = slugify(data['title']); return data })
-```
-
-### 7. Custom casts
-
-```ts
-import type { CastUsing } from '@rudderjs/orm'
-
-class MoneyCast implements CastUsing {
-  get(key: string, value: unknown): number {
-    return Number(value) / 100  // stored as cents
-  }
-  set(key: string, value: unknown): number {
-    return Math.round(Number(value) * 100)
-  }
-}
-
-class Product extends Model {
-  static casts = { price: MoneyCast }
-  // Or decorator: @Cast(MoneyCast) price = 0
-}
-```
-
-### 8. Model factories
-
-```ts
-import { ModelFactory, sequence } from '@rudderjs/orm'
-
-class UserFactory extends ModelFactory<{ name: string; email: string; role: string }> {
-  protected modelClass = User
-
-  definition() {
-    return {
-      name:  'Alice',
-      email: sequence(i => `user${i}@example.com`),
-      role:  'user',
-    }
-  }
-
-  protected states() {
-    return {
-      admin: () => ({ role: 'admin' }),
-    }
-  }
-}
-
-// Usage
-const user = await UserFactory.new().create()
-const admin = await UserFactory.new().state('admin').create()
-const users = await UserFactory.new().create(5)           // 5 persisted
-const dtos = await UserFactory.new().make(3)               // 3 in-memory only
-const custom = await UserFactory.new().with(() => ({ name: 'Bob' })).create()
-```
-
-### 9. API resources
-
-```ts
-import { JsonResource, ResourceCollection } from '@rudderjs/orm'
-
-class UserResource extends JsonResource<User> {
-  toArray() {
-    return {
-      id:    this.resource.id,
-      name:  this.resource.name,
-      email: this.resource.email,
-      admin: this.when(this.resource.role === 'admin', true),
-      posts: this.whenLoaded('posts', PostResource.collection(this.resource.posts)),
-      ...this.mergeWhen(this.resource.isAdmin, {
-        permissions: this.resource.permissions,
-      }),
-    }
-  }
-}
-
-// Single resource
-res.json(new UserResource(user).toArray())
-
-// Collection with pagination
-const collection = UserResource.collection(users, { total: 100, page: 1, perPage: 15 })
-res.json(await collection.toResponse())
-// -> { data: [...], meta: { total: 100, page: 1, perPage: 15 } }
-```
-
-### 10. Instance serialization controls
-
-```ts
-const user = await User.find(1)
-user.makeVisible('password')        // show normally-hidden field
-user.makeHidden('email')            // hide for this instance
-user.setVisible(['id', 'name'])     // allowlist override
-
-// On collections
-const collection = ModelCollection.wrap(await User.all())
-collection.makeHidden(['email'])
-collection.modelKeys()              // [1, 2, 3]
-collection.find(2)                  // user with id 2
-collection.except([1])              // all except id 1
-```
+- **Adapter pattern** — `ModelRegistry.set(adapter)` plugs in the DB driver. `@rudderjs/orm` has no runtime DB dependency.
+- **Hydration** — every read (`find`/`first`/`all`/`where(...).get()`/`paginate`) returns Model instances, not plain records. Use `Model.hydrate(record)` to wrap external data (cached JSON, fixtures).
+- **Mass assignment** — `static fillable` (allowlist) / `static guarded` (denylist; `['*']` locks all) drop keys outside policy on `create` / `update` / `fill`. `instance.forceFill` bypasses.
+- **Observers** — `Model.create/update/delete` fire lifecycle events. `query().create()` bypasses them. `increment` / `decrement` deliberately do **not** fire — pure data-plane operations.
+- **Built-in casts**: `'string'`, `'integer'`, `'float'`, `'boolean'`, `'date'`, `'datetime'`, `'json'`, `'array'`, `'collection'`, `'encrypted'`, `'encrypted:array'`, `'encrypted:object'`. Encrypted casts need `@rudderjs/crypt`.
 
 ## Examples
 
-See `playground/app/Models/User.ts` for a working model and `playground/routes/console.ts` for seeding with factories.
-
-## Common pitfalls
-
-- **No adapter registered**: `ModelRegistry.getAdapter()` throws if no database provider is in the provider list. Ensure `DatabaseServiceProvider` boots before any model usage.
-- **Observers vs query builder**: `Model.create()` / `Model.update()` / `Model.delete()` fire observer events. Using `Model.query().create()` directly bypasses observers.
-- **Built-in casts**: Available types are `'string'`, `'integer'`, `'float'`, `'boolean'`, `'date'`, `'datetime'`, `'json'`, `'array'`, `'collection'`, `'encrypted'`, `'encrypted:array'`, `'encrypted:object'`. Encrypted casts require `@rudderjs/crypt`.
-- **Visible vs hidden**: When `visible` is set (allowlist), `hidden` is ignored. Only one should be used per model.
-- **sequence() in factories**: The `sequence()` helper returns a callable. Inside `definition()`, return it directly -- the factory resolves callables automatically.
-- **Prisma schema**: Models map to Prisma models. Run `pnpm exec prisma generate` after schema changes and `pnpm exec prisma db push` to sync the DB.
+See `playground/app/Models/User.ts` for a working model and `playground/routes/console.ts` for factory-based seeding.

package/boost/skills/orm-models/rules/crud-and-observers.md

@@ -0,0 +1,130 @@
+# CRUD and Observers
+
+## Create / update / delete
+
+```ts
+// Create — triggers creating / created observer events
+const post = await Post.create({ title: 'Hello', body: 'World', authorId: 1 })
+
+// Update — triggers updating / updated
+const updated = await Post.update(post.id, { title: 'Hello v2' })
+
+// Delete — triggers deleting / deleted
+await Post.delete(post.id)
+```
+
+Mass assignment (`fillable` / `guarded`) filters keys on all three. To bypass: `instance.forceFill(data)` or set properties directly and call `instance.save()`.
+
+## firstOrCreate / updateOrCreate
+
+```ts
+const tag = await Tag.firstOrCreate({ name: 'rust' }, { description: 'systems language' })
+// SELECT, then INSERT if not found
+
+const user = await User.updateOrCreate({ email }, { name, lastSeenAt: new Date() })
+// SELECT, then INSERT or UPDATE
+```
+
+⚠️ Lookup keys (the first arg) go through `create()`, so they need to be **fillable** too — otherwise the lookup column isn't set on the new row.
+
+## Atomic counters
+
+For columns that change in concurrent writes (view counts, balances):
+
+```ts
+await Post.increment(postId, 'viewCount')           // viewCount = viewCount + 1
+await Post.decrement(postId, 'stock', 5)            // stock     = stock     - 5
+
+// Instance variant merges the resolved value back
+await post.increment('viewCount')                   // post.viewCount reflects the update
+```
+
+**Observers do NOT fire** on increment/decrement — they're pure data-plane. If you need observer hooks, read the row, set the resolved value, and call `Model.update()` instead.
+
+## Observers
+
+```ts
+import type { ModelObserver } from '@rudderjs/orm'
+
+class PostObserver implements ModelObserver {
+  creating(data: Record<string, unknown>) {
+    data['slug'] = slugify(String(data['title']))
+    return data
+  }
+
+  deleted(id: string | number) {
+    console.log(`Post ${id} was deleted`)
+  }
+}
+
+Post.observe(PostObserver)
+```
+
+Inline listeners are also supported:
+
+```ts
+Post.on('creating', (data) => {
+  data['slug'] = slugify(String(data['title']))
+  return data
+})
+```
+
+Events fired by lifecycle: `retrieved`, `creating` → `created`, `updating` → `updated`, `saving` → `saved` (on both create + update), `deleting` → `deleted`, `restoring` → `restored`.
+
+## Soft delete + restore
+
+```ts
+class Post extends Model { static softDeletes = true }
+
+await Post.delete(id)         // sets deletedAt
+await Post.restore(id)        // clears deletedAt — fires restoring / restored
+await Post.forceDelete(id)    // hard delete — fires deleting / deleted with no soft-delete
+```
+
+## Pitfalls
+
+❌ **Don't** rely on observers firing for `query().create()`:
+
+```ts
+await Post.query().create({ title })   // bypasses observers
+```
+
+✅ **Do** use the static method for observer-aware writes:
+
+```ts
+await Post.create({ title })           // fires creating / created
+```
+
+❌ **Don't** expect observers on counter updates:
+
+```ts
+class Post extends Model {
+  static observe = class {
+    updated(post: Post) { /* won't fire for increment() */ }
+  }
+}
+```
+
+✅ **Do** read + write through `update()` if you need hooks:
+
+```ts
+const post = await Post.find(id)
+await Post.update(id, { viewCount: post.viewCount + 1 })   // fires updating / updated
+```
+
+❌ **Don't** rely on lookup keys being set if they're not fillable:
+
+```ts
+class Tag extends Model {
+  static fillable = ['description']    // no 'name'
+}
+await Tag.firstOrCreate({ name: 'rust' }, { description: 'lang' })   // name dropped → new row missing name
+```
+
+✅ **Do** include lookup keys in `fillable`:
+
+```ts
+class Tag extends Model {
+  static fillable = ['name', 'description']
+}
+```

package/boost/skills/orm-models/rules/defining-models.md

@@ -0,0 +1,137 @@
+# Defining Models
+
+## Basic shape
+
+```ts
+import { Model } from '@rudderjs/orm'
+
+export class Post extends Model {
+  static table    = 'posts'
+  static fillable = ['title', 'body', 'authorId']
+  static hidden   = ['internalNotes']
+
+  static casts = {
+    isPublished: 'boolean'  as const,
+    publishedAt: 'datetime' as const,
+    metadata:    'json'     as const,
+  }
+}
+```
+
+Table defaults to lowercase class name + `'s'`. Override with `static table = 'my_table'`.
+Primary key defaults to `'id'`. Override with `static primaryKey = 'uuid'`.
+
+## Decorator syntax (alternative)
+
+```ts
+import { Model, Hidden, Cast, Appends } from '@rudderjs/orm'
+
+export class User extends Model {
+  static fillable = ['name', 'email', 'password']
+
+  @Hidden               password   = ''
+  @Cast('boolean')      isAdmin    = false
+  @Cast('date')         createdAt  = new Date()
+  @Appends              fullName   = ''
+}
+```
+
+Pick **one style per model** — mixing decorators and `static` config for the same fields is confusing.
+
+## Accessors and mutators
+
+```ts
+import { Model, Attribute } from '@rudderjs/orm'
+
+export class User extends Model {
+  static attributes = {
+    fullName: Attribute.make({
+      get: (_v, attrs) => `${attrs['firstName']} ${attrs['lastName']}`,
+    }),
+    password: Attribute.make({
+      set: async (v) => (await import('bcrypt')).hash(String(v), 10),
+    }),
+  }
+
+  static appends = ['fullName']   // include in toJSON output
+}
+```
+
+Mutators (`set`) run on `Model.create` / `Model.update` / `instance.fill`. Accessors (`get`) run on serialization. Appended attributes need a matching `get` definition, otherwise the field is missing from output.
+
+## Custom casts
+
+For domain types (money, vectors, custom serialization):
+
+```ts
+import type { CastUsing } from '@rudderjs/orm'
+
+class MoneyCast implements CastUsing {
+  get(_key: string, value: unknown): number { return Number(value) / 100 }
+  set(_key: string, value: unknown): number { return Math.round(Number(value) * 100) }
+}
+
+class Product extends Model {
+  static casts = { price: MoneyCast }
+  // or decorator: @Cast(MoneyCast) price = 0
+}
+```
+
+## Soft deletes
+
+```ts
+export class Post extends Model {
+  static softDeletes = true
+}
+
+await Post.delete(id)                                    // sets deletedAt
+await Post.restore(id)                                   // clears deletedAt
+await Post.forceDelete(id)                               // hard delete
+await Post.query().withTrashed().all()                   // include soft-deleted
+await Post.query().onlyTrashed().all()                   // only soft-deleted
+```
+
+## Pitfalls
+
+❌ **Don't** set `static visible` AND `static hidden` on the same model:
+
+```ts
+static visible = ['id', 'name']
+static hidden  = ['password']   // silently ignored when `visible` is set
+```
+
+✅ **Do** pick one. `visible` is a strict allowlist; `hidden` is a denylist.
+
+❌ **Don't** type a relation field that shadows the runtime-installed accessor:
+
+```ts
+class Post extends Model {
+  tags!: () => string[]   // class field shadows the prototype method
+}
+```
+
+✅ **Do** type the explicit override:
+
+```ts
+class Post extends Model {
+  tags() { return Model.morphToMany(this, 'tags') }
+}
+```
+
+❌ **Don't** assume the SQL table name is the Prisma delegate:
+
+```ts
+class OAuthClient extends Model {
+  static table = 'oauth_clients'   // wrong — that's the @@map'd SQL name
+}
+```
+
+✅ **Do** use the Prisma client delegate (camelCase of model name):
+
+```ts
+class OAuthClient extends Model {
+  static table = 'oAuthClient'     // the Prisma delegate
+}
+```
+
+Error: `[RudderJS ORM] Prisma has no delegate for table "oauth_clients"` means you used the SQL name by mistake.