@warlock.js/cascade 5.23.2 → 5.25.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (72) hide show
  1. package/CHANGELOG.md +352 -329
  2. package/README.md +101 -101
  3. package/bin/cascade.js +0 -0
  4. package/cjs/index.cjs +208 -69
  5. package/cjs/index.cjs.map +1 -1
  6. package/esm/contracts/database-driver.contract.d.mts +22 -1
  7. package/esm/contracts/database-driver.contract.d.mts.map +1 -1
  8. package/esm/contracts/index.d.mts +1 -1
  9. package/esm/data-source/data-source-registry.d.mts +8 -6
  10. package/esm/data-source/data-source-registry.d.mts.map +1 -1
  11. package/esm/data-source/data-source-registry.mjs +7 -1
  12. package/esm/data-source/data-source-registry.mjs.map +1 -1
  13. package/esm/drivers/mongodb/mongodb-driver.d.mts +13 -0
  14. package/esm/drivers/mongodb/mongodb-driver.d.mts.map +1 -1
  15. package/esm/drivers/mongodb/mongodb-driver.mjs +71 -2
  16. package/esm/drivers/mongodb/mongodb-driver.mjs.map +1 -1
  17. package/esm/drivers/mongodb/mongodb-query-builder.d.mts.map +1 -1
  18. package/esm/drivers/mongodb/mongodb-query-builder.mjs +2 -1
  19. package/esm/drivers/mongodb/mongodb-query-builder.mjs.map +1 -1
  20. package/esm/drivers/postgres/postgres-driver.d.mts +9 -0
  21. package/esm/drivers/postgres/postgres-driver.d.mts.map +1 -1
  22. package/esm/drivers/postgres/postgres-driver.mjs +61 -5
  23. package/esm/drivers/postgres/postgres-driver.mjs.map +1 -1
  24. package/esm/drivers/postgres/postgres-query-parser.d.mts +1 -4
  25. package/esm/drivers/postgres/postgres-query-parser.d.mts.map +1 -1
  26. package/esm/drivers/postgres/postgres-query-parser.mjs +4 -5
  27. package/esm/drivers/postgres/postgres-query-parser.mjs.map +1 -1
  28. package/esm/drivers/postgres/types.d.mts +5 -0
  29. package/esm/drivers/postgres/types.d.mts.map +1 -1
  30. package/esm/errors/unsafe-filter.error.mjs.map +1 -1
  31. package/esm/errors/unsafe-raw-expression.error.mjs.map +1 -1
  32. package/esm/errors/unsupported-lock-for-update.error.d.mts +11 -0
  33. package/esm/errors/unsupported-lock-for-update.error.d.mts.map +1 -0
  34. package/esm/errors/unsupported-lock-for-update.error.mjs +16 -0
  35. package/esm/errors/unsupported-lock-for-update.error.mjs.map +1 -0
  36. package/esm/events/model-events.mjs.map +1 -1
  37. package/esm/index.d.mts +3 -2
  38. package/esm/index.mjs +2 -1
  39. package/esm/migration/migration-runner.mjs.map +1 -1
  40. package/esm/migration/sql-grammar.mjs.map +1 -1
  41. package/esm/model/generate-model-id.mjs +23 -0
  42. package/esm/model/generate-model-id.mjs.map +1 -0
  43. package/esm/model/methods/delete-methods.mjs.map +1 -1
  44. package/esm/model/methods/meta-methods.mjs.map +1 -1
  45. package/esm/model/methods/query-methods.mjs +25 -2
  46. package/esm/model/methods/query-methods.mjs.map +1 -1
  47. package/esm/model/methods/write-methods.mjs.map +1 -1
  48. package/esm/query-builder/query-builder.mjs +1 -1
  49. package/esm/query-builder/query-builder.mjs.map +1 -1
  50. package/esm/utils/sanitize-filter.mjs.map +1 -1
  51. package/esm/validation/plugins/database-rules-plugin.mjs.map +1 -1
  52. package/esm/writer/database-writer.d.mts +0 -33
  53. package/esm/writer/database-writer.d.mts.map +1 -1
  54. package/esm/writer/database-writer.mjs +4 -52
  55. package/esm/writer/database-writer.mjs.map +1 -1
  56. package/llms-full.txt +2835 -2783
  57. package/package.json +5 -5
  58. package/skills/aggregate-data/SKILL.md +165 -165
  59. package/skills/alter-migration/SKILL.md +531 -531
  60. package/skills/configure-delete-strategy/SKILL.md +130 -130
  61. package/skills/define-model/SKILL.md +205 -205
  62. package/skills/define-relations/SKILL.md +171 -171
  63. package/skills/manage-data-sources/SKILL.md +177 -140
  64. package/skills/manage-transactions/SKILL.md +233 -208
  65. package/skills/paginate-results/SKILL.md +122 -122
  66. package/skills/perform-atomic-ops/SKILL.md +156 -156
  67. package/skills/query-data/SKILL.md +213 -213
  68. package/skills/run-cascade-cli/SKILL.md +125 -125
  69. package/skills/search-by-vector/SKILL.md +127 -127
  70. package/skills/subscribe-to-model-events/SKILL.md +148 -148
  71. package/skills/track-changes/SKILL.md +127 -127
  72. package/skills/write-migration/SKILL.md +206 -206
@@ -1,125 +1,125 @@
1
- ---
2
- name: run-cascade-cli
3
- description: 'Cascade''s standalone `cascade` binary + the Operations API it wraps — `cascade migrate` / `migrate:list` / `migrate:rollback` / `migrate:export-sql`, and `runMigrations` / `rollbackMigrations` / `freshMigrate` / `exportMigrationsSQL` / `listExecutedMigrations` / `createDatabase` / `dropAllTables` / `migrationRunner`. Triggers: `cascade migrate`, `migrate:list`, `migrate:rollback`, `migrate:export-sql`, `runMigrations`, `rollbackMigrations`, `freshMigrate`, `exportMigrationsSQL`, `listExecutedMigrations`, `migrationRunner`; "run migrations in deploy/CI", "reset DB for tests", "programmatic migration", "foreign key constraint cannot be implemented", `CASCADE_PRIMARY_KEY`; typical import `import { runMigrations, migrationRunner } from "@warlock.js/cascade"`. Skip: writing migration files — `@warlock.js/cascade/write-migration/SKILL.md`; competing tools `knex migrate:latest`, `prisma migrate deploy`, `typeorm migration:run`.'
4
- ---
5
-
6
- # Run the cascade CLI / Operations API
7
-
8
- Cascade ships a standalone `cascade` binary plus a programmatic **Operations API** — named functions over the migration-runner singleton. The binary is a thin wrapper over those functions; warlock-core's CLI wraps the same code path. Use the binary from terminal / deploy scripts; use the Operations API from test setup, container init, or custom tooling.
9
-
10
- ## CLI commands
11
-
12
- The standalone binary exposes four colon-keyed subcommands:
13
-
14
- ```bash
15
- cascade migrate # run all pending migrations
16
- cascade migrate:list # show executed migrations from _migrations
17
- cascade migrate:rollback # undo the last batch
18
- cascade migrate:export-sql # write .up.sql / .down.sql instead of executing
19
- ```
20
-
21
- Flags:
22
- - `migrate` — `-f/--fresh` (drop everything and re-run), `-s/--sql` (export SQL instead of executing), `--pending-only`, `-c/--compact`, `-p/--path <glob>`
23
- - `migrate:rollback` — `-a/--all` (roll back everything), `--batches N`, `-p/--path <glob>`
24
- - `migrate:export-sql` — `--pending-only`, `-c/--compact`, `-p/--path <glob>`
25
-
26
- There is **no** `seed`, `db:create`, or `db:drop-tables` in the standalone binary — those need project context; use the warlock CLI when you have a Warlock app.
27
-
28
- ## Configuration — env vars only
29
-
30
- No `cascade.config.{ts,js}` file. The CLI auto-loads `.env` from cwd at start.
31
-
32
- ```bash
33
- DATABASE_URL=postgres://user:pass@host:5432/db # one connection string …
34
- # … or discrete vars:
35
- DB_DIALECT=postgres # or mongodb
36
- DB_HOST=localhost
37
- DB_PORT=5432
38
- DB_NAME=myapp
39
- DB_USER=postgres
40
- DB_PASSWORD=secret
41
-
42
- # Migration defaults — only set when overriding library defaults:
43
- CASCADE_PRIMARY_KEY=uuid # uuid | int | bigInt
44
- CASCADE_UUID_STRATEGY=v7 # v4 | v7
45
- ```
46
-
47
- Warlock-project aliases accepted: `DB_URL` ↔ `DATABASE_URL`, `DB_DRIVER` ↔ `DB_DIALECT`, `DB_USERNAME` ↔ `DB_USER`.
48
-
49
- ## Diagnose first: "foreign key constraint … cannot be implemented"
50
-
51
- On a fresh Postgres run this is almost always a primary-key type mismatch: a migration declared a `uuid()` foreign key, but the referenced table's PK got created as `bigserial` because `migrationDefaults.primaryKey` defaulted to `"int"`. Fix by matching the project's PK convention via env:
52
-
53
- ```bash
54
- CASCADE_PRIMARY_KEY=uuid
55
- CASCADE_UUID_STRATEGY=v7
56
- ```
57
-
58
- Warlock projects set these in `src/config/database.ts`'s `migrationOptions`; the cascade CLI mirrors them via env — match the values exactly.
59
-
60
- ## TS migrations — invoke through a TS runtime
61
-
62
- The cascade CLI ships **no TypeScript transpiler.** For `.ts` migrations, invoke through `tsx` / `ts-node` / any TS-aware runtime:
63
-
64
- ```bash
65
- npx tsx node_modules/.bin/cascade migrate
66
- ```
67
-
68
- If forgotten, cascade catches the import failure and prints a pointer to this pattern.
69
-
70
- ## Migration file discovery
71
-
72
- Default glob: `./migrations/**/*.{ts,js,mjs,cjs}` from cwd. Override with `-p`, and **always quote the pattern** (the shell expands `**`/`*` before the binary sees it otherwise):
73
-
74
- ```bash
75
- cascade migrate -p "src/app/**/migrations/*.ts"
76
- ```
77
-
78
- Each file must `export default` a migration class; cascade infers the name from the filename and uses any leading `MM-DD-YYYY_HH-MM-SS` timestamp for ordering.
79
-
80
- ## Operations API — programmatic equivalents
81
-
82
- ```ts
83
- import {
84
- runMigrations,
85
- rollbackMigrations,
86
- freshMigrate,
87
- exportMigrationsSQL,
88
- listExecutedMigrations,
89
- createDatabase,
90
- dropAllTables,
91
- migrationRunner,
92
- } from "@warlock.js/cascade";
93
- import CreateUsersTable from "./migrations/create-users.migration";
94
-
95
- migrationRunner.registerMany([CreateUsersTable /* … */]);
96
-
97
- const results = await runMigrations();
98
- const failed = results.filter((r) => !r.success);
99
- ```
100
-
101
- Reach for these when you need migrations inside test setup (`beforeAll(async () => { await runMigrations(); })`), container init scripts, custom CLI wrappers, or reading `_migrations` programmatically (`listExecutedMigrations()`). The Operations API returns structured data and **does not print** — the caller decides how to surface progress. (The runner still emits per-migration logs through `@warlock.js/logger`.)
102
-
103
- ## Common task → command
104
-
105
- | You want to… | Command |
106
- |---|---|
107
- | Run all pending migrations | `cascade migrate` |
108
- | Drop everything and re-run | `cascade migrate -f` |
109
- | Roll back the last batch | `cascade migrate:rollback` |
110
- | Roll back everything | `cascade migrate:rollback --all` |
111
- | Generate SQL without executing | `cascade migrate --sql` |
112
- | Generate SQL for pending only | `cascade migrate --sql --pending-only` |
113
- | See what's been executed | `cascade migrate:list` |
114
- | Run from a non-default folder | `cascade migrate -p "<glob>"` |
115
-
116
- ## Things NOT to do
117
-
118
- - Don't run `cascade migrate -f` (fresh) anywhere that could touch production — it drops everything first.
119
- - Don't run `migrate` from inside app code at boot. Migrations are a deploy step; coupling them to boot makes rolling restarts dangerous.
120
- - Don't forget to quote `-p` globs, or the shell expands them and cascade registers a single file.
121
-
122
- ## See also
123
-
124
- - [`@warlock.js/cascade/write-migration/SKILL.md`](@warlock.js/cascade/write-migration/SKILL.md) — writing migration files
125
- - [`@warlock.js/cascade/manage-data-sources/SKILL.md`](@warlock.js/cascade/manage-data-sources/SKILL.md) — multi-database routing
1
+ ---
2
+ name: run-cascade-cli
3
+ description: 'Cascade''s standalone `cascade` binary + the Operations API it wraps — `cascade migrate` / `migrate:list` / `migrate:rollback` / `migrate:export-sql`, and `runMigrations` / `rollbackMigrations` / `freshMigrate` / `exportMigrationsSQL` / `listExecutedMigrations` / `createDatabase` / `dropAllTables` / `migrationRunner`. Triggers: `cascade migrate`, `migrate:list`, `migrate:rollback`, `migrate:export-sql`, `runMigrations`, `rollbackMigrations`, `freshMigrate`, `exportMigrationsSQL`, `listExecutedMigrations`, `migrationRunner`; "run migrations in deploy/CI", "reset DB for tests", "programmatic migration", "foreign key constraint cannot be implemented", `CASCADE_PRIMARY_KEY`; typical import `import { runMigrations, migrationRunner } from "@warlock.js/cascade"`. Skip: writing migration files — `@warlock.js/cascade/write-migration/SKILL.md`; competing tools `knex migrate:latest`, `prisma migrate deploy`, `typeorm migration:run`.'
4
+ ---
5
+
6
+ # Run the cascade CLI / Operations API
7
+
8
+ Cascade ships a standalone `cascade` binary plus a programmatic **Operations API** — named functions over the migration-runner singleton. The binary is a thin wrapper over those functions; warlock-core's CLI wraps the same code path. Use the binary from terminal / deploy scripts; use the Operations API from test setup, container init, or custom tooling.
9
+
10
+ ## CLI commands
11
+
12
+ The standalone binary exposes four colon-keyed subcommands:
13
+
14
+ ```bash
15
+ cascade migrate # run all pending migrations
16
+ cascade migrate:list # show executed migrations from _migrations
17
+ cascade migrate:rollback # undo the last batch
18
+ cascade migrate:export-sql # write .up.sql / .down.sql instead of executing
19
+ ```
20
+
21
+ Flags:
22
+ - `migrate` — `-f/--fresh` (drop everything and re-run), `-s/--sql` (export SQL instead of executing), `--pending-only`, `-c/--compact`, `-p/--path <glob>`
23
+ - `migrate:rollback` — `-a/--all` (roll back everything), `--batches N`, `-p/--path <glob>`
24
+ - `migrate:export-sql` — `--pending-only`, `-c/--compact`, `-p/--path <glob>`
25
+
26
+ There is **no** `seed`, `db:create`, or `db:drop-tables` in the standalone binary — those need project context; use the warlock CLI when you have a Warlock app.
27
+
28
+ ## Configuration — env vars only
29
+
30
+ No `cascade.config.{ts,js}` file. The CLI auto-loads `.env` from cwd at start.
31
+
32
+ ```bash
33
+ DATABASE_URL=postgres://user:pass@host:5432/db # one connection string …
34
+ # … or discrete vars:
35
+ DB_DIALECT=postgres # or mongodb
36
+ DB_HOST=localhost
37
+ DB_PORT=5432
38
+ DB_NAME=myapp
39
+ DB_USER=postgres
40
+ DB_PASSWORD=secret
41
+
42
+ # Migration defaults — only set when overriding library defaults:
43
+ CASCADE_PRIMARY_KEY=uuid # uuid | int | bigInt
44
+ CASCADE_UUID_STRATEGY=v7 # v4 | v7
45
+ ```
46
+
47
+ Warlock-project aliases accepted: `DB_URL` ↔ `DATABASE_URL`, `DB_DRIVER` ↔ `DB_DIALECT`, `DB_USERNAME` ↔ `DB_USER`.
48
+
49
+ ## Diagnose first: "foreign key constraint … cannot be implemented"
50
+
51
+ On a fresh Postgres run this is almost always a primary-key type mismatch: a migration declared a `uuid()` foreign key, but the referenced table's PK got created as `bigserial` because `migrationDefaults.primaryKey` defaulted to `"int"`. Fix by matching the project's PK convention via env:
52
+
53
+ ```bash
54
+ CASCADE_PRIMARY_KEY=uuid
55
+ CASCADE_UUID_STRATEGY=v7
56
+ ```
57
+
58
+ Warlock projects set these in `src/config/database.ts`'s `migrationOptions`; the cascade CLI mirrors them via env — match the values exactly.
59
+
60
+ ## TS migrations — invoke through a TS runtime
61
+
62
+ The cascade CLI ships **no TypeScript transpiler.** For `.ts` migrations, invoke through `tsx` / `ts-node` / any TS-aware runtime:
63
+
64
+ ```bash
65
+ npx tsx node_modules/.bin/cascade migrate
66
+ ```
67
+
68
+ If forgotten, cascade catches the import failure and prints a pointer to this pattern.
69
+
70
+ ## Migration file discovery
71
+
72
+ Default glob: `./migrations/**/*.{ts,js,mjs,cjs}` from cwd. Override with `-p`, and **always quote the pattern** (the shell expands `**`/`*` before the binary sees it otherwise):
73
+
74
+ ```bash
75
+ cascade migrate -p "src/app/**/migrations/*.ts"
76
+ ```
77
+
78
+ Each file must `export default` a migration class; cascade infers the name from the filename and uses any leading `MM-DD-YYYY_HH-MM-SS` timestamp for ordering.
79
+
80
+ ## Operations API — programmatic equivalents
81
+
82
+ ```ts
83
+ import {
84
+ runMigrations,
85
+ rollbackMigrations,
86
+ freshMigrate,
87
+ exportMigrationsSQL,
88
+ listExecutedMigrations,
89
+ createDatabase,
90
+ dropAllTables,
91
+ migrationRunner,
92
+ } from "@warlock.js/cascade";
93
+ import CreateUsersTable from "./migrations/create-users.migration";
94
+
95
+ migrationRunner.registerMany([CreateUsersTable /* … */]);
96
+
97
+ const results = await runMigrations();
98
+ const failed = results.filter((r) => !r.success);
99
+ ```
100
+
101
+ Reach for these when you need migrations inside test setup (`beforeAll(async () => { await runMigrations(); })`), container init scripts, custom CLI wrappers, or reading `_migrations` programmatically (`listExecutedMigrations()`). The Operations API returns structured data and **does not print** — the caller decides how to surface progress. (The runner still emits per-migration logs through `@warlock.js/logger`.)
102
+
103
+ ## Common task → command
104
+
105
+ | You want to… | Command |
106
+ |---|---|
107
+ | Run all pending migrations | `cascade migrate` |
108
+ | Drop everything and re-run | `cascade migrate -f` |
109
+ | Roll back the last batch | `cascade migrate:rollback` |
110
+ | Roll back everything | `cascade migrate:rollback --all` |
111
+ | Generate SQL without executing | `cascade migrate --sql` |
112
+ | Generate SQL for pending only | `cascade migrate --sql --pending-only` |
113
+ | See what's been executed | `cascade migrate:list` |
114
+ | Run from a non-default folder | `cascade migrate -p "<glob>"` |
115
+
116
+ ## Things NOT to do
117
+
118
+ - Don't run `cascade migrate -f` (fresh) anywhere that could touch production — it drops everything first.
119
+ - Don't run `migrate` from inside app code at boot. Migrations are a deploy step; coupling them to boot makes rolling restarts dangerous.
120
+ - Don't forget to quote `-p` globs, or the shell expands them and cascade registers a single file.
121
+
122
+ ## See also
123
+
124
+ - [`@warlock.js/cascade/write-migration/SKILL.md`](@warlock.js/cascade/write-migration/SKILL.md) — writing migration files
125
+ - [`@warlock.js/cascade/manage-data-sources/SKILL.md`](@warlock.js/cascade/manage-data-sources/SKILL.md) — multi-database routing
@@ -1,127 +1,127 @@
1
- ---
2
- name: search-by-vector
3
- description: 'Vector similarity search via `.similarTo(column, embedding, alias?)` — adds a similarity `score` column and orders by vector distance so the index is used; cap results with `.limit()`. Postgres uses pgvector (IVFFlat index via `this.vectorIndex`); MongoDB needs Atlas. Schema: `this.vector(column, dimensions)` + `this.vectorIndex(column, { dimensions, similarity })`. Triggers: `.similarTo`, `this.vector`, `this.vectorIndex`, `.whereFullText`, pgvector; "semantic search", "RAG retrieval", "find similar articles", "hybrid vector + full-text"; typical import `import { Model } from "@warlock.js/cascade"`. Skip: query basics — `@warlock.js/cascade/query-data/SKILL.md`; semantic cache — `@warlock.js/cache/use-cache-similarity/SKILL.md`; competing libs `pgvector` directly, `chromadb`, `pinecone`, `weaviate`, `qdrant`.'
4
- ---
5
-
6
- # Use vector search
7
-
8
- Query by vector distance for semantic search. Cascade gives you the column type, the index, and the similarity query method — generating embeddings is your AI provider's job.
9
-
10
- ## Schema + migration
11
-
12
- Both the column and its index are builders on the migration `this`:
13
-
14
- ```ts
15
- import { Migration } from "@warlock.js/cascade";
16
-
17
- export default class CreateArticles extends Migration {
18
- public readonly table = "articles";
19
-
20
- public up(): void {
21
- this.createTable();
22
- this.id();
23
- this.string("title");
24
- this.text("body");
25
- this.vector("embedding", 1536); // pgvector column, 1536 dims
26
- this.vectorIndex("embedding", { dimensions: 1536, similarity: "cosine" });
27
- }
28
-
29
- public down(): void {
30
- this.dropTable();
31
- }
32
- }
33
- ```
34
-
35
- `vectorIndex(column, { dimensions, similarity?, lists?, name? })` — `similarity` is `"cosine" | "euclidean" | "dotProduct"` (maps to the pgvector operator class). On Postgres this builds an **IVFFlat** index (`lists` controls the cluster count, default 100). Requires `CREATE EXTENSION vector` on the database.
36
-
37
- On MongoDB the vector index is an Atlas Search index definition (Atlas-only).
38
-
39
- ## Write — store an embedding
40
-
41
- ```ts
42
- const embedding = await ai.embed(body);
43
- await Article.create({ title, body, embedding });
44
- ```
45
-
46
- Cascade stores the vector and queries against it; it doesn't compute embeddings.
47
-
48
- ## Read — similarity search
49
-
50
- `.similarTo(column, embedding, alias?)` does two things at once: it adds `1 - (column <=> embedding) AS score` to the SELECT so each row carries its similarity, and it adds `ORDER BY column <=> embedding` so the database uses the vector index instead of a sequential scan. Cap the result with `.limit()`:
51
-
52
- ```ts
53
- const queryEmbedding = await ai.embed("how does cascade vector search work?");
54
-
55
- const hits = await Article.query()
56
- .similarTo("embedding", queryEmbedding) // score column defaults to "score"
57
- .limit(5)
58
- .get<ArticleRow & { score: number }>();
59
-
60
- // hits[0].score → similarity of the closest match
61
- ```
62
-
63
- There is no options object — `topK` is just `.limit(k)`, and the distance metric is fixed at index creation (the `similarity` you passed to `vectorIndex`). The third argument only renames the score column (`.similarTo("embedding", vec, "distance")`). Don't add your own `.orderBy()` on the score alias afterward — it would break index usage.
64
-
65
- ## Filtered similarity
66
-
67
- Chain `.where()` before `.similarTo()`:
68
-
69
- ```ts
70
- const tenantHits = await Article.query()
71
- .where("tenant_id", tenantId)
72
- .where("published", true)
73
- .similarTo("embedding", queryEmbedding)
74
- .limit(5)
75
- .get<ArticleRow & { score: number }>();
76
- ```
77
-
78
- The DB applies the filter first (regular index), then ranks the remaining candidates by similarity (vector index).
79
-
80
- ## Hybrid search — vector + full-text
81
-
82
- Cascade has `.whereFullText(fields, query)` for the text side. For best retrieval quality on long-form text, run a vector search and a full-text search and combine the results in code (re-rank or reciprocal-rank-fusion).
83
-
84
- ## RAG — retrieval-augmented generation
85
-
86
- ```ts
87
- async function answer(question: string) {
88
- const queryEmbedding = await ai.embed(question);
89
-
90
- const context = await Document.query()
91
- .where("tenant_id", currentTenant.id)
92
- .similarTo("embedding", queryEmbedding)
93
- .limit(8)
94
- .get<DocumentRow & { score: number }>();
95
-
96
- // optional: drop low-similarity hits in code
97
- const relevant = context.filter((document) => document.score >= 0.75);
98
-
99
- const prompt = buildPrompt(question, relevant.map((document) => document.body));
100
- return ai.complete(prompt);
101
- }
102
- ```
103
-
104
- A score threshold isn't a query option — filter on the `score` column in code after the fetch.
105
-
106
- ## Driver support
107
-
108
- | Driver | Vector |
109
- | --- | --- |
110
- | Postgres (with `CREATE EXTENSION vector`) | ✅ pgvector + IVFFlat index |
111
- | MongoDB Atlas (paid tier + Atlas Search index) | ✅ `$vectorSearch` aggregation stage |
112
- | MongoDB community / self-hosted / local | ❌ Atlas-only |
113
-
114
- For local dev with MongoDB, develop the vector path against Postgres + pgvector.
115
-
116
- ## Things NOT to do
117
-
118
- - Don't pass `{ topK, metric, threshold }` to `.similarTo()` — it takes `(column, embedding, alias?)`. Use `.limit()` for topK, set the metric at `vectorIndex` creation, and threshold in code on the `score` column.
119
- - Don't `.orderBy()` the score alias after `.similarTo()` — it already orders by distance for index usage.
120
- - Don't re-embed an entire corpus when changing embedding models — vectors aren't portable across models; plan the migration.
121
- - Don't ship the raw vector array to clients. Drop it from the public shape with `static toJsonColumns`.
122
- - Don't expect `.similarTo()` without a vector index to scale. Above a few thousand rows, sequential scans dominate.
123
-
124
- ## See also
125
-
126
- - [`@warlock.js/cascade/query-data/SKILL.md`](@warlock.js/cascade/query-data/SKILL.md) — `.where`, `.whereFullText`, the broader query vocabulary
127
- - [`@warlock.js/cache/use-cache-similarity/SKILL.md`](@warlock.js/cache/use-cache-similarity/SKILL.md) — semantic cache of LLM output
1
+ ---
2
+ name: search-by-vector
3
+ description: 'Vector similarity search via `.similarTo(column, embedding, alias?)` — adds a similarity `score` column and orders by vector distance so the index is used; cap results with `.limit()`. Postgres uses pgvector (IVFFlat index via `this.vectorIndex`); MongoDB needs Atlas. Schema: `this.vector(column, dimensions)` + `this.vectorIndex(column, { dimensions, similarity })`. Triggers: `.similarTo`, `this.vector`, `this.vectorIndex`, `.whereFullText`, pgvector; "semantic search", "RAG retrieval", "find similar articles", "hybrid vector + full-text"; typical import `import { Model } from "@warlock.js/cascade"`. Skip: query basics — `@warlock.js/cascade/query-data/SKILL.md`; semantic cache — `@warlock.js/cache/use-cache-similarity/SKILL.md`; competing libs `pgvector` directly, `chromadb`, `pinecone`, `weaviate`, `qdrant`.'
4
+ ---
5
+
6
+ # Use vector search
7
+
8
+ Query by vector distance for semantic search. Cascade gives you the column type, the index, and the similarity query method — generating embeddings is your AI provider's job.
9
+
10
+ ## Schema + migration
11
+
12
+ Both the column and its index are builders on the migration `this`:
13
+
14
+ ```ts
15
+ import { Migration } from "@warlock.js/cascade";
16
+
17
+ export default class CreateArticles extends Migration {
18
+ public readonly table = "articles";
19
+
20
+ public up(): void {
21
+ this.createTable();
22
+ this.id();
23
+ this.string("title");
24
+ this.text("body");
25
+ this.vector("embedding", 1536); // pgvector column, 1536 dims
26
+ this.vectorIndex("embedding", { dimensions: 1536, similarity: "cosine" });
27
+ }
28
+
29
+ public down(): void {
30
+ this.dropTable();
31
+ }
32
+ }
33
+ ```
34
+
35
+ `vectorIndex(column, { dimensions, similarity?, lists?, name? })` — `similarity` is `"cosine" | "euclidean" | "dotProduct"` (maps to the pgvector operator class). On Postgres this builds an **IVFFlat** index (`lists` controls the cluster count, default 100). Requires `CREATE EXTENSION vector` on the database.
36
+
37
+ On MongoDB the vector index is an Atlas Search index definition (Atlas-only).
38
+
39
+ ## Write — store an embedding
40
+
41
+ ```ts
42
+ const embedding = await ai.embed(body);
43
+ await Article.create({ title, body, embedding });
44
+ ```
45
+
46
+ Cascade stores the vector and queries against it; it doesn't compute embeddings.
47
+
48
+ ## Read — similarity search
49
+
50
+ `.similarTo(column, embedding, alias?)` does two things at once: it adds `1 - (column <=> embedding) AS score` to the SELECT so each row carries its similarity, and it adds `ORDER BY column <=> embedding` so the database uses the vector index instead of a sequential scan. Cap the result with `.limit()`:
51
+
52
+ ```ts
53
+ const queryEmbedding = await ai.embed("how does cascade vector search work?");
54
+
55
+ const hits = await Article.query()
56
+ .similarTo("embedding", queryEmbedding) // score column defaults to "score"
57
+ .limit(5)
58
+ .get<ArticleRow & { score: number }>();
59
+
60
+ // hits[0].score → similarity of the closest match
61
+ ```
62
+
63
+ There is no options object — `topK` is just `.limit(k)`, and the distance metric is fixed at index creation (the `similarity` you passed to `vectorIndex`). The third argument only renames the score column (`.similarTo("embedding", vec, "distance")`). Don't add your own `.orderBy()` on the score alias afterward — it would break index usage.
64
+
65
+ ## Filtered similarity
66
+
67
+ Chain `.where()` before `.similarTo()`:
68
+
69
+ ```ts
70
+ const tenantHits = await Article.query()
71
+ .where("tenant_id", tenantId)
72
+ .where("published", true)
73
+ .similarTo("embedding", queryEmbedding)
74
+ .limit(5)
75
+ .get<ArticleRow & { score: number }>();
76
+ ```
77
+
78
+ The DB applies the filter first (regular index), then ranks the remaining candidates by similarity (vector index).
79
+
80
+ ## Hybrid search — vector + full-text
81
+
82
+ Cascade has `.whereFullText(fields, query)` for the text side. For best retrieval quality on long-form text, run a vector search and a full-text search and combine the results in code (re-rank or reciprocal-rank-fusion).
83
+
84
+ ## RAG — retrieval-augmented generation
85
+
86
+ ```ts
87
+ async function answer(question: string) {
88
+ const queryEmbedding = await ai.embed(question);
89
+
90
+ const context = await Document.query()
91
+ .where("tenant_id", currentTenant.id)
92
+ .similarTo("embedding", queryEmbedding)
93
+ .limit(8)
94
+ .get<DocumentRow & { score: number }>();
95
+
96
+ // optional: drop low-similarity hits in code
97
+ const relevant = context.filter((document) => document.score >= 0.75);
98
+
99
+ const prompt = buildPrompt(question, relevant.map((document) => document.body));
100
+ return ai.complete(prompt);
101
+ }
102
+ ```
103
+
104
+ A score threshold isn't a query option — filter on the `score` column in code after the fetch.
105
+
106
+ ## Driver support
107
+
108
+ | Driver | Vector |
109
+ | --- | --- |
110
+ | Postgres (with `CREATE EXTENSION vector`) | ✅ pgvector + IVFFlat index |
111
+ | MongoDB Atlas (paid tier + Atlas Search index) | ✅ `$vectorSearch` aggregation stage |
112
+ | MongoDB community / self-hosted / local | ❌ Atlas-only |
113
+
114
+ For local dev with MongoDB, develop the vector path against Postgres + pgvector.
115
+
116
+ ## Things NOT to do
117
+
118
+ - Don't pass `{ topK, metric, threshold }` to `.similarTo()` — it takes `(column, embedding, alias?)`. Use `.limit()` for topK, set the metric at `vectorIndex` creation, and threshold in code on the `score` column.
119
+ - Don't `.orderBy()` the score alias after `.similarTo()` — it already orders by distance for index usage.
120
+ - Don't re-embed an entire corpus when changing embedding models — vectors aren't portable across models; plan the migration.
121
+ - Don't ship the raw vector array to clients. Drop it from the public shape with `static toJsonColumns`.
122
+ - Don't expect `.similarTo()` without a vector index to scale. Above a few thousand rows, sequential scans dominate.
123
+
124
+ ## See also
125
+
126
+ - [`@warlock.js/cascade/query-data/SKILL.md`](@warlock.js/cascade/query-data/SKILL.md) — `.where`, `.whereFullText`, the broader query vocabulary
127
+ - [`@warlock.js/cache/use-cache-similarity/SKILL.md`](@warlock.js/cache/use-cache-similarity/SKILL.md) — semantic cache of LLM output