@warlock.js/cascade 4.0.174 → 4.1.2

package/skills/paginate-results/SKILL.md

@@ -0,0 +1,122 @@
+---
+name: paginate-results
+description: 'Paginate query results — `.paginate({page, limit, filter?})` for offset (returns `data` + `pagination` total/page/limit/pages), `.cursorPaginate({limit, cursor})` for very large datasets, `.chunk(size, callback)` for streaming. Triggers: `.paginate`, `.cursorPaginate`, `.chunk`, `nextCursor`, `hasMore`, `pagination.total`; "paginate the list", "infinite scroll / load more", "stream a large table", "page 2 of users"; typical import `import { Model } from "@warlock.js/cascade"`. Skip: filter chain — `@warlock.js/cascade/query-data/SKILL.md`; eager loading on pages — `@warlock.js/cascade/define-relations/SKILL.md`; competing libs `mongoose-paginate-v2`, `prisma` cursor, `typeorm-pagination`.'
+---
+
+# Paginate results
+
+Three paginations. Pick by dataset size and access pattern.
+
+## Offset pagination — `.paginate({ page, limit })`
+
+The everyday case for listings with page numbers:
+
+```ts
+const page = await User.paginate({ page: 1, limit: 20 });
+
+page.data;        // User[]
+page.pagination;  // { total, page, limit, pages }
+```
+
+`PaginationOptions` is `{ page?, limit? }` — there is no `filter` field; filter by chaining `.where()` before `.paginate()` (below).
+
+Chain off `.where()` for filtered pagination:
+
+```ts
+const activePage = await User
+  .where("status", "active")
+  .paginate({ page: 2, limit: 20 });
+```
+
+**Cost characteristic.** Offset pagination scans `offset + limit` rows on every page — page 100 with limit 20 scans 2020 rows just to skip 2000. Fine for the first few pages; not great deep in the result set.
+
+## Cursor pagination — `.cursorPaginate({ limit, cursor? })`
+
+For very large datasets where deep pagination matters:
+
+```ts
+const first = await User.query().orderBy("created_at", "desc").cursorPaginate({ limit: 20 });
+
+first.data;                   // User[]
+first.pagination.nextCursor;  // opaque value — pass to the next call
+first.pagination.hasMore;     // boolean
+
+const next = await User.query()
+  .orderBy("created_at", "desc")
+  .cursorPaginate({ limit: 20, cursor: first.pagination.nextCursor });
+```
+
+The cursor fields live under `pagination` (`{ hasMore, nextCursor?, hasPrev?, prevCursor? }`), not at the top level. `cursorPaginate` and `orderBy` are query-builder methods, so start the chain with `User.query()` (or any static that returns a builder, like `User.where(...)`).
+
+**Cost characteristic.** Constant time per page regardless of how far in. The cursor encodes the last record's sort key — the next query is "give me records after this point," indexed.
+
+**Tradeoff.** No "total page count" — cursor pagination doesn't know how many records remain. If the UI shows "Page 3 of 50," reach for `.paginate()` instead. If it shows "Load more," cursor wins.
+
+## Chunked processing — `.chunk(size, callback)`
+
+For backfills, exports, and "process every record" loops:
+
+```ts
+await User.where("status", "active").chunk(500, async (users) => {
+  for (const user of users) {
+    await sendEmail(user);
+  }
+});
+```
+
+`.chunk(size, fn)` streams the table 500 records at a time, calling `fn` per batch. Constant memory regardless of total row count.
+
+Return `false` from the callback to stop early:
+
+```ts
+let processed = 0;
+await User.query().chunk(500, async (users) => {
+  for (const user of users) {
+    await process(user);
+    processed++;
+    if (processed >= 10_000) return false;
+  }
+});
+```
+
+`chunk` is a query-builder method — start from `User.query()` (or `User.where(...)`) before chaining it.
+
+## Pagination + relations
+
+Eager-load relations on a paginated page:
+
+```ts
+const page = await Post.with("author").paginate({ page: 1, limit: 20 });
+```
+
+See [`@warlock.js/cascade/define-relations/SKILL.md`](@warlock.js/cascade/define-relations/SKILL.md).
+
+## Pagination shape
+
+The default offset paginator returns:
+
+```ts
+{
+  data: T[],
+  pagination: {
+    total: number,    // total matching records (extra COUNT query)
+    page: number,     // current page
+    limit: number,    // page size
+    pages: number,    // total pages
+  },
+}
+```
+
+The total count requires an extra query. On very large filtered tables, this can dominate page-load time — switch to cursor pagination if the total isn't user-facing.
+
+## Things NOT to do
+
+- Don't use offset pagination for "Load more" infinite-scroll UIs. Cursor pagination is built for it; offset re-scans on every load.
+- Don't fetch `.all()` and slice in memory for pagination. Always page at the query layer.
+- Don't omit `.orderBy()` from `.cursorPaginate()`. The cursor encodes the sort key — without one, the cursor is meaningless and ordering is driver-dependent.
+- Don't keep cursor strings around past their stability window — schema changes that alter the sort key can invalidate stored cursors.
+
+## See also
+
+- [`@warlock.js/cascade/query-data/SKILL.md`](@warlock.js/cascade/query-data/SKILL.md) — `.where`, `.orderBy`, filter chains
+- [`@warlock.js/cascade/define-relations/SKILL.md`](@warlock.js/cascade/define-relations/SKILL.md) — eager loading on a paginated page

package/skills/perform-atomic-ops/SKILL.md

@@ -0,0 +1,98 @@
+---
+name: perform-atomic-ops
+description: 'Avoid races on concurrent writes — `Model.increase(filter, field, n)` / `Model.decrease` for atomic counters, `Model.atomic(filter, ops)` for arbitrary mutations (`$set` / `$inc` / `$push` / `$pull`), `Model.createMany` / `Model.findAndUpdate` / `Model.delete` for bulk. Triggers: `Model.increase`, `Model.decrease`, `Model.atomic`, `Model.createMany`, `Model.findAndUpdate`, `Model.delete`, `$inc`, `$set`; "increment counter under concurrency", "bulk insert without N+1", "atomic update without loading"; typical import `import { Model } from "@warlock.js/cascade"`. Skip: multi-row atomicity — `@warlock.js/cascade/manage-transactions/SKILL.md`; competing patterns `mongoose findOneAndUpdate`, `pg` `UPDATE ... SET x = x + 1`.'
+---
+
+# Use atomic operations
+
+When two requests want to change the same row at the same time, you need atomicity — a guarantee that one operation completes before the other reads. For multi-document atomicity use transactions; for single-document atomic mutations these are the right tools.
+
+## Counters — `Model.increase` / `Model.decrease`
+
+```ts
+await Post.increase({ id: postId }, "views", 1);
+await Product.decrease({ id: productId }, "inventory", 1);
+```
+
+Signature: `Model.increase(filter, field, amount)` / `Model.decrease(filter, field, amount)` → `Promise<number>` (matched count). Atomic at the storage layer — no read-modify-write race even under high concurrency.
+
+## Arbitrary atomic mutations — `Model.atomic`
+
+```ts
+await User.atomic({ id: userId }, {
+  $set: { last_seen: new Date() },
+  $inc: { login_count: 1 },
+});
+```
+
+`Model.atomic(filter, operations)` → `Promise<number>`. Driver-flavored atomic mutation — MongoDB has `$set` / `$inc` / `$push` / `$pull`; the Postgres driver translates the equivalents. Use when you need to combine multiple field changes atomically without loading the model first.
+
+## Bulk insert — `Model.createMany`
+
+```ts
+const created = await OrderItem.createMany([
+  { order_id, product_id: 1, quantity: 2 },
+  { order_id, product_id: 2, quantity: 1 },
+  { order_id, product_id: 3, quantity: 5 },
+]);
+// created: OrderItem[]
+```
+
+`Model.createMany(rows)` → `Promise<TModel[]>`. Validation runs per row; wrap in a transaction if you need strict all-or-nothing semantics.
+
+## Bulk update — `Model.findAndUpdate(filter, operations)`
+
+```ts
+const updated = await User.findAndUpdate(
+  { status: "pending" },
+  { $set: { status: "active" } },
+);
+// updated: User[] — the matched-and-updated models
+```
+
+`Model.findAndUpdate(filter, operations)` takes **update operators** (`$set` / `$inc` / `$unset`), not a plain data object, and returns the updated models. For a single record there's `Model.findOneAndUpdate(filter, operations)` → `TModel | null`, and to update strictly by id, `Model.update(id, data)` → `Promise<number>`.
+
+**Important.** Per-instance lifecycle `saved` events do NOT fire for each row on `findAndUpdate`. If you need `saved` per row, iterate with `.get()` and `.save()` instead — slower but event-correct.
+
+## Bulk delete — `Model.delete(filter)`
+
+```ts
+await User.delete({ status: "spam" });      // delete all matching → count
+await User.deleteOne({ status: "spam" });   // delete the first match → count
+```
+
+`Model.delete(filter?)` and `Model.deleteOne(filter?)` both return `Promise<number>`. These bypass the per-instance delete strategy and `deleted` events — they are raw driver deletes.
+
+For per-row event-aware (and delete-strategy-aware) bulk delete, iterate:
+
+```ts
+const targets = await User.where("status", "spam").get();
+for (const user of targets) {
+  await user.destroy();
+}
+```
+
+## When to reach for what
+
+| Task | Reach for |
+| --- | --- |
+| Increment a counter | `Model.increase(filter, field, n)` |
+| Atomically change multiple fields on one record | `Model.atomic(filter, ops)` |
+| Insert N records | `Model.createMany(rows)` |
+| Update many rows with operators | `Model.findAndUpdate(filter, { $set: {...} })` |
+| Update one record by id | `Model.update(id, data)` |
+| Delete many rows (raw) | `Model.delete(filter)` |
+| Multi-row read-modify-write | Wrap in a [transaction](@warlock.js/cascade/manage-transactions/SKILL.md) |
+| Need lifecycle events / delete strategy per row | `Model.where(...).get()` + iterate + `.save()` / `.destroy()` |
+
+## Things NOT to do
+
+- Don't `const post = await Post.find(id); post.set("views", post.get<number>("views") + 1); await post.save();` for a counter. That's a lost-update race under concurrency. Use `Post.increase(filter, "views", 1)`.
+- Don't reach for `insertMany` / `updateMany` / `deleteMany` — those names don't exist on the model. Use `createMany` / `findAndUpdate` / `delete`.
+- Don't expect `findAndUpdate` / `delete` to fire per-row `saved` / `deleted` events or honor the delete strategy. They don't. Iterate if you need that.
+- Don't bulk-insert a million rows in one `createMany` call — chunk it. Most drivers cap effectively at a few thousand per round-trip.
+
+## See also
+
+- [`@warlock.js/cascade/manage-transactions/SKILL.md`](@warlock.js/cascade/manage-transactions/SKILL.md) — multi-row atomicity
+- [`@warlock.js/cascade/paginate-results/SKILL.md`](@warlock.js/cascade/paginate-results/SKILL.md) — `.chunk` for bulk-processing iteration

package/skills/query-data/SKILL.md

@@ -0,0 +1,168 @@
+---
+name: query-data
+description: 'Query records via the model — `.where(field, value)` / `.where(field, op, value)`, `.find(id)` / `.first` / `.all`, `.orderBy`, `.count` / `.exists`, plus `.whereIn` / `.whereBetween` / `.whereLike` / `.pluck` / `.firstOrFail` / scopes via `addScope`. Triggers: `.where`, `.find`, `.first`, `.firstOrFail`, `.all`, `.get`, `.orderBy`, `.exists`, `.whereIn`, `.whereBetween`, `addScope`; "filter by status", "find by id", "fetch active users", "check existence"; typical import `import { Model } from "@warlock.js/cascade"`. Skip: pagination — `@warlock.js/cascade/paginate-results/SKILL.md`; aggregates — `@warlock.js/cascade/aggregate-data/SKILL.md`.'
+---
+
+# Query data
+
+The model is the query entry point. No `db.collection("users")`, no `prisma.user.findFirst()`, no repository to import — the class queries itself.
+
+## Filter — `.where()`
+
+### Equality — the shorthand
+
+```ts
+const activeUsers = await User.where("status", "active").get();
+```
+
+`User.where(field, value)` returns a query builder filtered to that condition. `.get()` runs the query and returns an array of `User` instances.
+
+### Operators
+
+```ts
+const adults     = await User.where("age", ">", 18).get();
+const recent     = await User.where("created_at", ">=", lastWeek).get();
+const nonAdmins  = await User.where("role", "!=", "admin").get();
+```
+
+3-argument form. Common operators: `=`, `!=`, `<`, `<=`, `>`, `>=`, `in`, `notIn`, `like`, `between`. Same syntax across MongoDB and Postgres.
+
+### Compound conditions
+
+```ts
+const activeAdmins = await User
+  .where("status", "active")
+  .where("role", "admin")
+  .get();
+```
+
+Chained `.where()` calls combine with `AND`.
+
+### Object form
+
+```ts
+const targets = await User.where({ status: "active", role: "admin" }).get();
+```
+
+Equivalent to chained equalities. Useful when the filter comes from a dynamic source. **Object form only supports equality** — use chained `.where()` for operators.
+
+## Get one record
+
+### By ID
+
+```ts
+const user = await User.find(id);  // User | null
+```
+
+### First match
+
+```ts
+const anyUser    = await User.first();                              // first user, any
+const firstAdmin = await User.first({ role: "admin" });             // first admin
+const filtered   = await User.where("status", "active").first();    // chain into .first()
+```
+
+`.first()` with no args returns the very first record (driver-dependent default order). With a filter object, the first match by equality. Chain off `.where()` when you need operators.
+
+### Throw if missing — `.firstOrFail()`
+
+```ts
+const user = await User.where("id", req.params.id).firstOrFail();
+```
+
+Throws when nothing matches — useful when you KNOW it should exist and want the error to surface loudly instead of an `undefined`-derived NPE downstream.
+
+**Always handle `null`** from `.find()` and `.first()` — use `?.` or a guard. Resist `!` on query results.
+
+## Order and paginate
+
+```ts
+const newest = await User
+  .where("status", "active")
+  .orderBy("created_at", "desc")
+  .get();
+```
+
+`.orderBy(field, "asc" | "desc")` sorts. Default direction is `"asc"`. Chain multiple `.orderBy()` for tiebreakers.
+
+For pagination see [`@warlock.js/cascade/paginate-results/SKILL.md`](@warlock.js/cascade/paginate-results/SKILL.md).
+
+## Count and existence
+
+```ts
+const total       = await User.count();
+const activeCount = await User.count({ status: "active" });
+const adminCount  = await User.where("role", "admin").count();
+
+const hasAdmin    = await User.where("role", "admin").exists();      // boolean, short-circuits
+const noneBlocked = await User.where("status", "blocked").notExists();
+```
+
+**Don't reach for `.count() > 0`** when you only need a boolean — `.exists()` short-circuits on the first matching row. The difference shows up immediately on tables with more than a few thousand rows.
+
+## Get many — `.all(filter?)`
+
+```ts
+const allUsers    = await User.all();
+const activeUsers = await User.all({ status: "active" });
+```
+
+`Model.all(filter?)` is the shortcut for "fetch all records matching a simple equality filter, or every record if no filter."
+
+**Caution.** `.all()` with no filter loads the entire table. Use [pagination](@warlock.js/cascade/paginate-results/SKILL.md) for tables larger than a few hundred rows.
+
+## The wider query vocabulary
+
+Cascade's query builder has around 60 methods. Reach for these as the need arises:
+
+| Reach for | When |
+| --- | --- |
+| `.whereIn(field, values)` / `.whereNotIn(field, values)` | Match against / exclude a list |
+| `.whereNull(field)` / `.whereNotNull(field)` | Nullability checks |
+| `.whereBetween(field, [a, b])` | Inclusive range |
+| `.whereDate(field, value)`, `.whereDateBetween`, `.whereDateBefore`, `.whereDateAfter` | Date helpers |
+| `.whereLike(field, pattern)` / `.whereStartsWith` / `.whereEndsWith` | Pattern matching |
+| `.whereHas(relation, callback)` | Filter by conditions on a related model |
+| `.sum(field)` / `.avg(field)` / `.min(field)` / `.max(field)` | Aggregates — [`use-aggregates`](@warlock.js/cascade/aggregate-data/SKILL.md) |
+| `.distinct(field)` / `.pluck(field)` | Single-field reads (distinct values, flat list) |
+| `.chunk(size, callback)` | Stream a large table in batches |
+| `.cursorPaginate({ limit, cursor })` | Cursor pagination — [`paginate-results`](@warlock.js/cascade/paginate-results/SKILL.md) |
+| `.similarTo(column, embedding)` | Vector similarity — [`use-vector-search`](@warlock.js/cascade/search-by-vector/SKILL.md) |
+
+Each chains off `User.where(...)` or `User.query()` and ends with the appropriate terminator. (`where`, `with`, `joinWith`, `first`, `count`, `find`, `all`, `paginate` are static shortcuts on the model; the rest live on the query builder, so reach them via `User.query()` or by chaining off a static `where`.)
+
+## Scopes — reusable query fragments
+
+When you write the same `.where("status", "active")` across multiple services, define a scope on the model:
+
+```ts
+@RegisterModel()
+export class User extends Model<UserSchema> {
+  public static table = "users";
+  public static schema = userSchema;
+
+  static {
+    this.addScope("active", (query) => {
+      query.where("status", "active");
+    });
+  }
+}
+
+const activeUsers = await User.query().scope("active").get();
+```
+
+**Local scopes** (`addScope`) — opt-in, only when you call `.scope("name")`.
+**Global scopes** (`addGlobalScope`) — run on every query for that model. Useful for multi-tenancy or default soft-delete filtering. Bypass per-query with `.withoutGlobalScope("name")` / `.withoutGlobalScopes()`.
+
+## Things NOT to do
+
+- Don't `.count() > 0` for existence — use `.exists()`.
+- Don't `Model.all()` without a filter on a production table — use pagination or chunking.
+- Don't `!` away the null from `.find()` / `.first()` — handle the missing case explicitly or use `.firstOrFail()` when absence is a real error.
+- Don't write the same filter chain across multiple services — promote it to a scope.
+
+## See also
+
+- [`@warlock.js/cascade/define-relations/SKILL.md`](@warlock.js/cascade/define-relations/SKILL.md) — `.with(...)`, `.whereHas(...)`, eager loading
+- [`@warlock.js/cascade/paginate-results/SKILL.md`](@warlock.js/cascade/paginate-results/SKILL.md) — pagination + cursor + chunk
+- [`@warlock.js/cascade/aggregate-data/SKILL.md`](@warlock.js/cascade/aggregate-data/SKILL.md) — `.sum`, `.avg`, `.groupBy`, `.having`

package/skills/run-cascade-cli/SKILL.md

@@ -0,0 +1,125 @@
+---
+name: run-cascade-cli
+description: 'Cascade''s standalone `cascade` binary + the Operations API it wraps — `cascade migrate` / `migrate:list` / `migrate:rollback` / `migrate:export-sql`, and `runMigrations` / `rollbackMigrations` / `freshMigrate` / `exportMigrationsSQL` / `listExecutedMigrations` / `createDatabase` / `dropAllTables` / `migrationRunner`. Triggers: `cascade migrate`, `migrate:list`, `migrate:rollback`, `migrate:export-sql`, `runMigrations`, `rollbackMigrations`, `freshMigrate`, `exportMigrationsSQL`, `listExecutedMigrations`, `migrationRunner`; "run migrations in deploy/CI", "reset DB for tests", "programmatic migration", "foreign key constraint cannot be implemented", `CASCADE_PRIMARY_KEY`; typical import `import { runMigrations, migrationRunner } from "@warlock.js/cascade"`. Skip: writing migration files — `@warlock.js/cascade/write-migration/SKILL.md`; competing tools `knex migrate:latest`, `prisma migrate deploy`, `typeorm migration:run`.'
+---
+
+# Run the cascade CLI / Operations API
+
+Cascade ships a standalone `cascade` binary plus a programmatic **Operations API** — named functions over the migration-runner singleton. The binary is a thin wrapper over those functions; warlock-core's CLI wraps the same code path. Use the binary from terminal / deploy scripts; use the Operations API from test setup, container init, or custom tooling.
+
+## CLI commands
+
+The standalone binary exposes four colon-keyed subcommands:
+
+```bash
+cascade migrate                 # run all pending migrations
+cascade migrate:list            # show executed migrations from _migrations
+cascade migrate:rollback        # undo the last batch
+cascade migrate:export-sql      # write .up.sql / .down.sql instead of executing
+```
+
+Flags:
+- `migrate` — `-f/--fresh` (drop everything and re-run), `-s/--sql` (export SQL instead of executing), `--pending-only`, `-c/--compact`, `-p/--path <glob>`
+- `migrate:rollback` — `-a/--all` (roll back everything), `--batches N`, `-p/--path <glob>`
+- `migrate:export-sql` — `--pending-only`, `-c/--compact`, `-p/--path <glob>`
+
+There is **no** `seed`, `db:create`, or `db:drop-tables` in the standalone binary — those need project context; use the warlock CLI when you have a Warlock app.
+
+## Configuration — env vars only
+
+No `cascade.config.{ts,js}` file. The CLI auto-loads `.env` from cwd at start.
+
+```bash
+DATABASE_URL=postgres://user:pass@host:5432/db    # one connection string …
+# … or discrete vars:
+DB_DIALECT=postgres        # or mongodb
+DB_HOST=localhost
+DB_PORT=5432
+DB_NAME=myapp
+DB_USER=postgres
+DB_PASSWORD=secret
+
+# Migration defaults — only set when overriding library defaults:
+CASCADE_PRIMARY_KEY=uuid   # uuid | int | bigInt
+CASCADE_UUID_STRATEGY=v7   # v4 | v7
+```
+
+Warlock-project aliases accepted: `DB_URL` ↔ `DATABASE_URL`, `DB_DRIVER` ↔ `DB_DIALECT`, `DB_USERNAME` ↔ `DB_USER`.
+
+## Diagnose first: "foreign key constraint … cannot be implemented"
+
+On a fresh Postgres run this is almost always a primary-key type mismatch: a migration declared a `uuid()` foreign key, but the referenced table's PK got created as `bigserial` because `migrationDefaults.primaryKey` defaulted to `"int"`. Fix by matching the project's PK convention via env:
+
+```bash
+CASCADE_PRIMARY_KEY=uuid
+CASCADE_UUID_STRATEGY=v7
+```
+
+Warlock projects set these in `src/config/database.ts`'s `migrationOptions`; the cascade CLI mirrors them via env — match the values exactly.
+
+## TS migrations — invoke through a TS runtime
+
+The cascade CLI ships **no TypeScript transpiler.** For `.ts` migrations, invoke through `tsx` / `ts-node` / any TS-aware runtime:
+
+```bash
+npx tsx node_modules/.bin/cascade migrate
+```
+
+If forgotten, cascade catches the import failure and prints a pointer to this pattern.
+
+## Migration file discovery
+
+Default glob: `./migrations/**/*.{ts,js,mjs,cjs}` from cwd. Override with `-p`, and **always quote the pattern** (the shell expands `**`/`*` before the binary sees it otherwise):
+
+```bash
+cascade migrate -p "src/app/**/migrations/*.ts"
+```
+
+Each file must `export default` a migration class; cascade infers the name from the filename and uses any leading `MM-DD-YYYY_HH-MM-SS` timestamp for ordering.
+
+## Operations API — programmatic equivalents
+
+```ts
+import {
+  runMigrations,
+  rollbackMigrations,
+  freshMigrate,
+  exportMigrationsSQL,
+  listExecutedMigrations,
+  createDatabase,
+  dropAllTables,
+  migrationRunner,
+} from "@warlock.js/cascade";
+import CreateUsersTable from "./migrations/create-users.migration";
+
+migrationRunner.registerMany([CreateUsersTable /* … */]);
+
+const results = await runMigrations();
+const failed = results.filter((r) => !r.success);
+```
+
+Reach for these when you need migrations inside test setup (`beforeAll(async () => { await runMigrations(); })`), container init scripts, custom CLI wrappers, or reading `_migrations` programmatically (`listExecutedMigrations()`). The Operations API returns structured data and **does not print** — the caller decides how to surface progress. (The runner still emits per-migration logs through `@warlock.js/logger`.)
+
+## Common task → command
+
+| You want to… | Command |
+|---|---|
+| Run all pending migrations | `cascade migrate` |
+| Drop everything and re-run | `cascade migrate -f` |
+| Roll back the last batch | `cascade migrate:rollback` |
+| Roll back everything | `cascade migrate:rollback --all` |
+| Generate SQL without executing | `cascade migrate --sql` |
+| Generate SQL for pending only | `cascade migrate --sql --pending-only` |
+| See what's been executed | `cascade migrate:list` |
+| Run from a non-default folder | `cascade migrate -p "<glob>"` |
+
+## Things NOT to do
+
+- Don't run `cascade migrate -f` (fresh) anywhere that could touch production — it drops everything first.
+- Don't run `migrate` from inside app code at boot. Migrations are a deploy step; coupling them to boot makes rolling restarts dangerous.
+- Don't forget to quote `-p` globs, or the shell expands them and cascade registers a single file.
+
+## See also
+
+- [`@warlock.js/cascade/write-migration/SKILL.md`](@warlock.js/cascade/write-migration/SKILL.md) — writing migration files
+- [`@warlock.js/cascade/manage-data-sources/SKILL.md`](@warlock.js/cascade/manage-data-sources/SKILL.md) — multi-database routing

package/skills/search-by-vector/SKILL.md

@@ -0,0 +1,127 @@
+---
+name: search-by-vector
+description: 'Vector similarity search via `.similarTo(column, embedding, alias?)` — adds a similarity `score` column and orders by vector distance so the index is used; cap results with `.limit()`. Postgres uses pgvector (IVFFlat index via `this.vectorIndex`); MongoDB needs Atlas. Schema: `this.vector(column, dimensions)` + `this.vectorIndex(column, { dimensions, similarity })`. Triggers: `.similarTo`, `this.vector`, `this.vectorIndex`, `.whereFullText`, pgvector; "semantic search", "RAG retrieval", "find similar articles", "hybrid vector + full-text"; typical import `import { Model } from "@warlock.js/cascade"`. Skip: query basics — `@warlock.js/cascade/query-data/SKILL.md`; semantic cache — `@warlock.js/cache/use-cache-similarity/SKILL.md`; competing libs `pgvector` directly, `chromadb`, `pinecone`, `weaviate`, `qdrant`.'
+---
+
+# Use vector search
+
+Query by vector distance for semantic search. Cascade gives you the column type, the index, and the similarity query method — generating embeddings is your AI provider's job.
+
+## Schema + migration
+
+Both the column and its index are builders on the migration `this`:
+
+```ts
+import { Migration } from "@warlock.js/cascade";
+
+export default class CreateArticles extends Migration {
+  public readonly table = "articles";
+
+  public up(): void {
+    this.createTable();
+    this.id();
+    this.string("title");
+    this.text("body");
+    this.vector("embedding", 1536); // pgvector column, 1536 dims
+    this.vectorIndex("embedding", { dimensions: 1536, similarity: "cosine" });
+  }
+
+  public down(): void {
+    this.dropTable();
+  }
+}
+```
+
+`vectorIndex(column, { dimensions, similarity?, lists?, name? })` — `similarity` is `"cosine" | "euclidean" | "dotProduct"` (maps to the pgvector operator class). On Postgres this builds an **IVFFlat** index (`lists` controls the cluster count, default 100). Requires `CREATE EXTENSION vector` on the database.
+
+On MongoDB the vector index is an Atlas Search index definition (Atlas-only).
+
+## Write — store an embedding
+
+```ts
+const embedding = await ai.embed(body);
+await Article.create({ title, body, embedding });
+```
+
+Cascade stores the vector and queries against it; it doesn't compute embeddings.
+
+## Read — similarity search
+
+`.similarTo(column, embedding, alias?)` does two things at once: it adds `1 - (column <=> embedding) AS score` to the SELECT so each row carries its similarity, and it adds `ORDER BY column <=> embedding` so the database uses the vector index instead of a sequential scan. Cap the result with `.limit()`:
+
+```ts
+const queryEmbedding = await ai.embed("how does cascade vector search work?");
+
+const hits = await Article.query()
+  .similarTo("embedding", queryEmbedding) // score column defaults to "score"
+  .limit(5)
+  .get<ArticleRow & { score: number }>();
+
+// hits[0].score → similarity of the closest match
+```
+
+There is no options object — `topK` is just `.limit(k)`, and the distance metric is fixed at index creation (the `similarity` you passed to `vectorIndex`). The third argument only renames the score column (`.similarTo("embedding", vec, "distance")`). Don't add your own `.orderBy()` on the score alias afterward — it would break index usage.
+
+## Filtered similarity
+
+Chain `.where()` before `.similarTo()`:
+
+```ts
+const tenantHits = await Article.query()
+  .where("tenant_id", tenantId)
+  .where("published", true)
+  .similarTo("embedding", queryEmbedding)
+  .limit(5)
+  .get<ArticleRow & { score: number }>();
+```
+
+The DB applies the filter first (regular index), then ranks the remaining candidates by similarity (vector index).
+
+## Hybrid search — vector + full-text
+
+Cascade has `.whereFullText(fields, query)` for the text side. For best retrieval quality on long-form text, run a vector search and a full-text search and combine the results in code (re-rank or reciprocal-rank-fusion).
+
+## RAG — retrieval-augmented generation
+
+```ts
+async function answer(question: string) {
+  const queryEmbedding = await ai.embed(question);
+
+  const context = await Document.query()
+    .where("tenant_id", currentTenant.id)
+    .similarTo("embedding", queryEmbedding)
+    .limit(8)
+    .get<DocumentRow & { score: number }>();
+
+  // optional: drop low-similarity hits in code
+  const relevant = context.filter((document) => document.score >= 0.75);
+
+  const prompt = buildPrompt(question, relevant.map((document) => document.body));
+  return ai.complete(prompt);
+}
+```
+
+A score threshold isn't a query option — filter on the `score` column in code after the fetch.
+
+## Driver support
+
+| Driver | Vector |
+| --- | --- |
+| Postgres (with `CREATE EXTENSION vector`) | ✅ pgvector + IVFFlat index |
+| MongoDB Atlas (paid tier + Atlas Search index) | ✅ `$vectorSearch` aggregation stage |
+| MongoDB community / self-hosted / local | ❌ Atlas-only |
+
+For local dev with MongoDB, develop the vector path against Postgres + pgvector.
+
+## Things NOT to do
+
+- Don't pass `{ topK, metric, threshold }` to `.similarTo()` — it takes `(column, embedding, alias?)`. Use `.limit()` for topK, set the metric at `vectorIndex` creation, and threshold in code on the `score` column.
+- Don't `.orderBy()` the score alias after `.similarTo()` — it already orders by distance for index usage.
+- Don't re-embed an entire corpus when changing embedding models — vectors aren't portable across models; plan the migration.
+- Don't ship the raw vector array to clients. Drop it from the public shape with `static toJsonColumns`.
+- Don't expect `.similarTo()` without a vector index to scale. Above a few thousand rows, sequential scans dominate.
+
+## See also
+
+- [`@warlock.js/cascade/query-data/SKILL.md`](@warlock.js/cascade/query-data/SKILL.md) — `.where`, `.whereFullText`, the broader query vocabulary
+- [`@warlock.js/cache/use-cache-similarity/SKILL.md`](@warlock.js/cache/use-cache-similarity/SKILL.md) — semantic cache of LLM output