@leonardovida-md/drizzle-neo-duckdb 1.5.4-15 → 2.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (72) hide show
  1. package/README.md +166 -306
  2. package/dist/bin/duckdb-introspect-args.d.ts +26 -0
  3. package/dist/client.d.ts +24 -2
  4. package/dist/columns.d.ts +49 -13
  5. package/dist/dialect.d.ts +44 -6
  6. package/dist/driver.d.ts +21 -4
  7. package/dist/duckdb-introspect.mjs +3987 -1217
  8. package/dist/ducklake.d.ts +52 -0
  9. package/dist/helpers.d.ts +1 -1
  10. package/dist/helpers.mjs +294 -50
  11. package/dist/index.d.ts +8 -2
  12. package/dist/index.mjs +5712 -2346
  13. package/dist/instance-keys.d.ts +3 -0
  14. package/dist/introspect.d.ts +14 -0
  15. package/dist/jev.d.ts +155 -0
  16. package/dist/motherduck.d.ts +183 -41
  17. package/dist/olap.d.ts +6 -0
  18. package/dist/pgduck.d.ts +5 -1
  19. package/dist/pool.d.ts +32 -5
  20. package/dist/prepared-statement-cache.d.ts +20 -2
  21. package/dist/prompt.d.ts +27 -0
  22. package/dist/select-builder.d.ts +37 -5
  23. package/dist/session.d.ts +60 -5
  24. package/dist/sql/array-bounds-text.d.ts +24 -0
  25. package/dist/sql/ast-transformer.d.ts +17 -5
  26. package/dist/sql/result-mapper.d.ts +9 -1
  27. package/dist/sql/selection.d.ts +11 -0
  28. package/dist/sql/visitors/array-bounds.d.ts +10 -0
  29. package/dist/sql/visitors/column-qualifier.d.ts +17 -0
  30. package/dist/sql/visitors/generate-series-alias.d.ts +2 -1
  31. package/dist/sql/visitors/union-with-hoister.d.ts +9 -0
  32. package/dist/sql-constants.d.ts +9 -0
  33. package/dist/time.d.ts +46 -1
  34. package/dist/value-wrappers.d.ts +21 -2
  35. package/package.json +5 -12
  36. package/dist/sql/visitors/array-operators.d.ts +0 -5
  37. package/dist/utils.d.ts +0 -2
  38. package/src/array-literals.ts +0 -38
  39. package/src/bin/duckdb-introspect.ts +0 -268
  40. package/src/client.ts +0 -901
  41. package/src/columns.ts +0 -608
  42. package/src/dialect.ts +0 -221
  43. package/src/driver.ts +0 -514
  44. package/src/ducklake.ts +0 -281
  45. package/src/helpers.ts +0 -18
  46. package/src/index.ts +0 -19
  47. package/src/introspect.ts +0 -1089
  48. package/src/migration-config.ts +0 -9
  49. package/src/migrator.ts +0 -22
  50. package/src/motherduck.ts +0 -994
  51. package/src/olap.ts +0 -315
  52. package/src/operators.ts +0 -8
  53. package/src/options.ts +0 -41
  54. package/src/own-property.ts +0 -17
  55. package/src/pgduck.ts +0 -57
  56. package/src/pool.ts +0 -368
  57. package/src/prepared-statement-cache.ts +0 -142
  58. package/src/select-builder.ts +0 -86
  59. package/src/session.ts +0 -562
  60. package/src/sql/ast-transformer.ts +0 -176
  61. package/src/sql/result-mapper.ts +0 -364
  62. package/src/sql/selection.ts +0 -107
  63. package/src/sql/split-top-level.ts +0 -40
  64. package/src/sql/visitors/array-operators.ts +0 -306
  65. package/src/sql/visitors/ast-helpers.ts +0 -39
  66. package/src/sql/visitors/column-qualifier.ts +0 -550
  67. package/src/sql/visitors/generate-series-alias.ts +0 -264
  68. package/src/sql/visitors/union-with-hoister.ts +0 -106
  69. package/src/time.ts +0 -5
  70. package/src/utils.ts +0 -2
  71. package/src/value-wrappers-core.ts +0 -168
  72. package/src/value-wrappers.ts +0 -165
package/README.md CHANGED
@@ -13,162 +13,122 @@
13
13
 
14
14
  <br>
15
15
 
16
- **Drizzle DuckDB** brings [Drizzle ORM](https://orm.drizzle.team/) to [DuckDB](https://duckdb.org/), an in-process analytical database. You get Drizzle's type-safe query builder, automatic migrations, and full TypeScript inference while working with DuckDB's analytics engine.
16
+ **Drizzle DuckDB** brings [Drizzle ORM](https://orm.drizzle.team/) to [DuckDB](https://duckdb.org/), the in-process analytical database. You get Drizzle's type-safe query builder, migrations and TypeScript inference on top of DuckDB. It works with in-memory databases, local files, [MotherDuck](https://motherduck.com/) and Postgres servers running [`pg_duckdb`](https://github.com/duckdb/pg_duckdb).
17
17
 
18
- Works with local DuckDB files, in-memory databases, and [MotherDuck](https://motherduck.com/) cloud.
18
+ > **Status:** Experimental. Query building, migrations and type inference are stable. Some DuckDB-specific types and edge cases are still being refined.
19
19
 
20
- > **Status:** Experimental. Core query building, migrations, and type inference work well. Some DuckDB-specific types and edge cases are still being refined.
21
-
22
- > **Note:** The npm package is `@leonardovida-md/drizzle-neo-duckdb`.
23
-
24
- Docs tip: every docs page has a **Markdown (raw)** button for LLM-friendly source.
20
+ Every docs page has a **Markdown (raw)** button with LLM-friendly source.
25
21
 
26
22
  ## Installation
27
23
 
28
24
  ```bash
29
- bun add @leonardovida-md/drizzle-neo-duckdb @duckdb/node-api
30
- ```
31
-
32
- ```bash
33
- npm install @leonardovida-md/drizzle-neo-duckdb @duckdb/node-api
34
- ```
35
-
36
- ```bash
37
- pnpm add @leonardovida-md/drizzle-neo-duckdb @duckdb/node-api
25
+ bun add @leonardovida-md/drizzle-neo-duckdb drizzle-orm @duckdb/node-api
26
+ # or: npm install / pnpm add with the same packages
38
27
  ```
39
28
 
40
- Supported client versions include `@duckdb/node-api@1.4.4-r.1` and `@duckdb/node-api@1.5.5-r.4`. The repo now develops against `1.5.5-r.4`, and `1.4.4-r.1` remains supported.
41
- Tested `drizzle-orm` versions currently span `0.40.1` through `0.45.x`.
29
+ | Requirement | Supported versions |
30
+ | ------------------ | ----------------------------------------------------------------------------------- |
31
+ | `drizzle-orm` | 0.40.1 or newer, below 0.46.0 |
32
+ | `@duckdb/node-api` | 1.4.4 or newer, below 1.6.0, including `-r.N` builds such as `1.5.6-r.1` |
33
+ | Runtime | Node.js 18.17 or newer (22 or 24 recommended), or Bun |
42
34
 
43
35
  ## Quick Start
44
36
 
45
37
  ```typescript
46
- import { DuckDBInstance } from '@duckdb/node-api';
47
38
  import { drizzle } from '@leonardovida-md/drizzle-neo-duckdb';
48
39
  import { sql } from 'drizzle-orm';
49
- import { integer, text, pgTable } from 'drizzle-orm/pg-core';
50
-
51
- // Connect to DuckDB
52
- const instance = await DuckDBInstance.create(':memory:');
53
- const connection = await instance.connect();
54
- const db = drizzle(connection);
40
+ import { integer, pgTable, text } from 'drizzle-orm/pg-core';
55
41
 
56
- // Define your schema
57
42
  const users = pgTable('users', {
58
43
  id: integer('id').primaryKey(),
59
44
  name: text('name').notNull(),
60
45
  email: text('email').notNull(),
61
46
  });
62
47
 
63
- // Create table
48
+ const db = await drizzle(':memory:');
49
+
64
50
  await db.execute(sql`
65
- CREATE TABLE IF NOT EXISTS users (
51
+ CREATE TABLE users (
66
52
  id INTEGER PRIMARY KEY,
67
53
  name TEXT NOT NULL,
68
54
  email TEXT NOT NULL
69
55
  )
70
56
  `);
71
57
 
72
- // Insert data
73
58
  await db.insert(users).values([
74
59
  { id: 1, name: 'Alice', email: 'alice@example.com' },
75
60
  { id: 2, name: 'Bob', email: 'bob@example.com' },
76
61
  ]);
77
62
 
78
- // Query with full type safety
79
63
  const allUsers = await db.select().from(users);
80
64
  // ^? { id: number; name: string; email: string }[]
81
65
 
82
- // Clean up
83
- connection.closeSync();
84
- ```
85
-
86
- ## Connecting to DuckDB
87
-
88
- ### In-Memory Database
89
-
90
- ```typescript
91
- const instance = await DuckDBInstance.create(':memory:');
92
- const connection = await instance.connect();
93
- const db = drizzle(connection);
66
+ await db.close();
94
67
  ```
95
68
 
96
- ### Local File
69
+ Schemas use the regular `drizzle-orm/pg-core` builders (`pgTable`, `pgSchema` and so on), and all standard Drizzle query methods work. DuckDB SQL is largely Postgres-compatible, and the dialect adapts queries where it is not.
97
70
 
98
- ```typescript
99
- const instance = await DuckDBInstance.create('./my-database.duckdb');
100
- const connection = await instance.connect();
101
- const db = drizzle(connection);
102
- ```
71
+ ## Connecting
103
72
 
104
- ### MotherDuck Cloud
73
+ Pass a path and the driver creates a connection pool for you. `await db.close()` closes it.
105
74
 
106
75
  ```typescript
107
- const instance = await DuckDBInstance.create('md:', {
108
- motherduck_token: process.env.MOTHERDUCK_TOKEN,
109
- });
110
- const connection = await instance.connect();
111
- const db = drizzle(connection);
112
- ```
113
-
114
- ### pg_duckdb
76
+ import { drizzle } from '@leonardovida-md/drizzle-neo-duckdb';
115
77
 
116
- For PostgreSQL servers with the
117
- [`pg_duckdb`](https://github.com/duckdb/pg_duckdb) extension installed, use a
118
- Postgres wire client such as `pg`:
78
+ // In-memory
79
+ const db = await drizzle(':memory:');
119
80
 
120
- ```typescript
121
- import pg from 'pg';
122
- import { sql } from 'drizzle-orm';
123
- import { drizzle } from '@leonardovida-md/drizzle-neo-duckdb';
81
+ // Local file (created if missing)
82
+ const db = await drizzle('./my-database.duckdb');
124
83
 
125
- const client = new pg.Client({
126
- connectionString: process.env.DATABASE_URL,
84
+ // MotherDuck
85
+ const db = await drizzle({
86
+ connection: {
87
+ path: 'md:my_database',
88
+ options: { motherduck_token: process.env.MOTHERDUCK_TOKEN },
89
+ },
127
90
  });
128
- await client.connect();
129
91
 
130
- const db = drizzle(client);
131
-
132
- await db.execute(sql`SET duckdb.force_execution = true`);
92
+ // DuckLake catalog
93
+ const db = await drizzle(':memory:', {
94
+ ducklake: {
95
+ catalog: './ducklake.duckdb',
96
+ attachOptions: { dataPath: './ducklake-data' },
97
+ },
98
+ });
133
99
  ```
134
100
 
135
- When using a `pg.Pool`, wrap it so transactions pin one backend connection:
101
+ Calls with the same file path share one DuckDB instance in a process, so they see each other's writes. `:memory:` is never shared.
102
+
103
+ You can also pass your own connection, pool or `pg_duckdb` client. These calls are synchronous:
136
104
 
137
105
  ```typescript
106
+ import { DuckDBInstance } from '@duckdb/node-api';
138
107
  import pg from 'pg';
139
- import {
140
- createPgDuckConnectionPool,
141
- drizzle,
142
- } from '@leonardovida-md/drizzle-neo-duckdb';
143
108
 
144
- const pool = new pg.Pool({
145
- connectionString: process.env.DATABASE_URL,
146
- });
109
+ // Existing DuckDB connection
110
+ const instance = await DuckDBInstance.create(':memory:');
111
+ const db = drizzle(await instance.connect());
147
112
 
148
- const db = drizzle(createPgDuckConnectionPool(pool));
113
+ // Postgres server with pg_duckdb. A pg.Client works too.
114
+ const db = drizzle(new pg.Pool({ connectionString: process.env.DATABASE_URL }));
149
115
  ```
150
116
 
151
- ### With Logging
152
-
153
- ```typescript
154
- import { DefaultLogger } from 'drizzle-orm';
155
-
156
- const db = drizzle(connection, {
157
- logger: new DefaultLogger(),
158
- });
159
- ```
117
+ A `pg.Pool` is wrapped with `createPgDuckConnectionPool()`, so each transaction runs on one pooled client and `await db.close()` ends the pool.
160
118
 
161
- > Tip: With connection strings (recommended), just pass the path: `const db = await drizzle(':memory:')`. Pooling is automatic.
119
+ See [Database Connection](https://leonardovida.github.io/drizzle-duckdb/core/connection) for singleton and serverless patterns.
162
120
 
163
121
  ## Connection Pooling
164
122
 
165
- DuckDB executes one query per connection. The async `drizzle()` entrypoints create a pool automatically (default size: 4). Options:
123
+ DuckDB runs one query per connection, so the path forms above create a pool (default size 4).
124
+
125
+ - Set the size or a MotherDuck preset: `pool: { size: 8 }`, `pool: 'jumbo'` or `pool: 'giga'`.
126
+ - Tune recycling: `pool: { size: 8, idleTimeoutMs: 60_000, maxLifetimeMs: 10 * 60_000 }`.
127
+ - Disable pooling: `pool: false`.
128
+ - Transactions pin one pooled connection for their whole lifetime.
129
+ - `pool` is ignored when you pass a connection or pool instance.
166
130
 
167
- - Set pool size or MotherDuck preset: `drizzle('md:', { pool: { size: 8 } })` or `pool: 'jumbo'` / `pool: 'giga'`.
168
- - Tune timeout and recycling behavior on the auto-created pool: `pool: { size: 8, idleTimeoutMs: 60_000, maxLifetimeMs: 10 * 60_000 }`.
169
- - Disable pooling for single-connection workloads: `pool: false`.
170
- - Transactions pin one pooled connection for their entire lifetime; non-transactional queries still use the pool.
171
- - Create the pool manually when you need the `setup` hook or want to reuse the same pool across multiple `drizzle()` instances:
131
+ Create the pool yourself when you need the `setup` hook or want to share it between `drizzle()` instances:
172
132
 
173
133
  ```typescript
174
134
  import { DuckDBInstance } from '@duckdb/node-api';
@@ -185,190 +145,78 @@ const pool = createDuckDBConnectionPool(instance, {
185
145
  idleTimeoutMs: 60_000,
186
146
  });
187
147
  const db = drizzle(pool);
148
+ console.log(pool.stats()); // Leases, queue depth, waits, recycling and timeouts
188
149
  ```
189
150
 
190
- ## Schema & Types
191
-
192
- - Use `drizzle-orm/pg-core` for schemas; DuckDB SQL is largely Postgres-compatible.
193
- - DuckDB-specific helpers: `duckDbList`, `duckDbArray`, `duckDbStruct`, `duckDbMap`, `duckDbJson`, `duckDbBlob`, `duckDbInet`, `duckDbInterval`, `duckDbTimestamp`, `duckDbDate`, `duckDbTime`.
194
- - `duckDbTimestamp` and `duckDbTime` can preserve DuckDB-specific storage variants such as `TIMESTAMP_NS`, `TIMESTAMP_MS`, `TIMESTAMP_S`, `TIME_NS`, and `TIME WITH TIME ZONE` via the `duckDbType` / `withTimezone` options.
195
- - Browser-safe imports live under `@leonardovida-md/drizzle-neo-duckdb/helpers` (introspection emits this path).
196
-
197
- See the [column types](https://leonardovida.github.io/drizzle-duckdb/api/columns) docs for full API.
151
+ `acquireTimeout` limits time in the pool queue. Connection creation and setup run after that wait and have no deadline from this option. Check `pool.stats()` before increasing pool size, since more connections can compete with DuckDB's own query threads.
198
152
 
199
- ## Postgres Schema Compatibility
153
+ ## Configuration
200
154
 
201
- Use `pgTable`, `pgSchema`, and other `drizzle-orm/pg-core` builders as you do with Postgres. The dialect keeps table definitions and relations intact while adapting queries to DuckDB.
155
+ ```typescript
156
+ import { DefaultLogger } from 'drizzle-orm';
202
157
 
203
- ## MotherDuck Helpers
158
+ const db = await drizzle(':memory:', {
159
+ // Log every query
160
+ logger: new DefaultLogger(),
204
161
 
205
- MotherDuck table function helpers are composable SQL fragments:
162
+ // Pool size, preset, or timeout and recycling options
163
+ pool: { size: 8, idleTimeoutMs: 60_000 },
206
164
 
207
- ```typescript
208
- import { sql } from 'drizzle-orm';
209
- import {
210
- mdAccessTokens,
211
- mdCreateDive,
212
- mdCreateFlight,
213
- mdGetDive,
214
- mdGetFlightRun,
215
- mdListDives,
216
- mdListFlights,
217
- mdRunFlight,
218
- } from '@leonardovida-md/drizzle-neo-duckdb';
165
+ // Per-connection prepared statement cache (default: disabled)
166
+ prepareCache: { size: 32 },
219
167
 
220
- const dives = await db.execute(sql`
221
- select id, title, owner_name, updated_at
222
- from ${mdListDives({ limit: 10, includeOrgShares: true })}
223
- order by updated_at desc
224
- `);
168
+ // Preserve DECIMAL values as strings in raw, streaming and nested reads
169
+ decimalMode: 'string', // Default: 'number'
225
170
 
226
- const [dive] = await db.execute(sql`
227
- select id, title, current_version
228
- from ${mdCreateDive({
229
- title: 'Revenue Trends',
230
- content: 'export default function Dive() { return null }',
231
- description: 'Monthly revenue dashboard',
232
- })}
233
- `);
171
+ // Legacy raw SQL join qualification heuristic, opt in only if needed
172
+ qualifyRawJoinColumns: false,
234
173
 
235
- const [diveContent] = await db.execute(sql`
236
- select title, content
237
- from ${mdGetDive(String(dive.id))}
238
- `);
174
+ // Throw on Postgres-style array literals like '{1,2,3}' (default: false)
175
+ rejectStringArrayLiterals: false,
239
176
 
240
- const flights = await db.execute(sql`
241
- select flight_id, flight_name, current_version
242
- from ${mdListFlights({ limit: 25 })}
243
- `);
177
+ // Map camelCase keys to snake_case column names
178
+ casing: 'snake_case',
244
179
 
245
- const activeTokens = await db.execute(sql`
246
- select token_name, token_type
247
- from ${mdAccessTokens({ activeOnly: true })}
248
- `);
180
+ // Schema for relational queries
181
+ schema: mySchema,
182
+ });
183
+ ```
249
184
 
250
- const [flight] = await db.execute(sql`
251
- select flight_id, status
252
- from ${mdCreateFlight({
253
- name: 'daily-refresh',
254
- sourceCode: 'print("hello")',
255
- scheduleCron: '0 0 * * *',
256
- maxRuntimeSec: 1_800,
257
- })}
258
- `);
185
+ `pool` and `ducklake` apply only to the path and `{ connection }` forms. The other options also work with `drizzle(connection, { ... })`. See [Configuration](https://leonardovida.github.io/drizzle-duckdb/reference/configuration) for the full list.
259
186
 
260
- const runs = await db.execute(sql`
261
- select run_number, status, created_at
262
- from ${mdRunFlight(String(flight.flight_id), {
263
- config: { region: 'eu-west-1' },
264
- })}
265
- `);
187
+ ## DuckDB Types
266
188
 
267
- const [run] = await db.execute(sql`
268
- select run_number, status, created_at, ended_at
269
- from ${mdGetFlightRun(String(flight.flight_id), 1)}
270
- `);
271
- ```
189
+ DuckDB-specific column helpers: `duckDbList`, `duckDbArray`, `duckDbStruct`, `duckDbUnion`, `duckDbMap`, `duckDbJson`, `duckDbBlob`, `duckDbInet`, `duckDbInterval`, `duckDbTimestamp`, `duckDbDate` and `duckDbTime`.
272
190
 
273
- The Dives helper family covers the public preview table functions for listing,
274
- reading, creating, updating, deleting, and versioning Dives: `mdListDives()`,
275
- `mdGetDive()`, `mdCreateDive()`, `mdUpdateDiveMetadata()`,
276
- `mdUpdateDiveContent()`, `mdDeleteDive()`, `mdListDiveVersions()`, and
277
- `mdGetDiveVersion()`. Dive listing and version rows include
278
- `required_resources` on supported MotherDuck deployments.
279
-
280
- The older `mdJobs()` helper family remains exported as deprecated compatibility
281
- aliases. Those helpers call the supported Flight table functions and preserve
282
- the older `job_*` result column names where the Flight result uses `flight_*`.
283
- The earlier `mdFlights()`, `mdFlightRuns()`, `mdFlightLogs()`, and
284
- `mdFlightVersions()` TypeScript helper names also remain as deprecated aliases,
285
- but new code should use the verb-style Flight helpers.
286
- `mdGetFlightLogs()` returns one `MotherDuckFlightLogLineRow` per log line with
287
- `line_number`, `reported_at`, and `line` fields. The old blob-shaped
288
- `MotherDuckFlightLogsRow` type remains exported for the deprecated
289
- `mdFlightLogs()` and `mdJobRunLogs()` compatibility views.
290
- Use `mdGetFlightRun()` to fetch one run by its Flight ID and run number. Pass
291
- `limit`, `offset`, and `order: 'asc' | 'desc'` to `mdGetFlightLogs()` when you
292
- only need a page of older or newer log lines.
293
- For optional Flight fields, `undefined` omits the named parameter and `null`
294
- emits an explicit SQL `NULL`. MotherDuck treats explicit `NULL` values as clear
295
- or empty values for nullable Flight options such as `requirementsTxt`, `config`,
296
- and `flightSecretNames`.
297
- Use `maxRuntimeSec` on `mdCreateFlight()` or `mdUpdateFlight()` to cap each run
298
- in seconds. Set it to `0` for no timeout, or omit it to use the plan default.
299
-
300
- `config` entries are exposed to Flight code as environment variables using the
301
- config key. Config keys must not be empty, cannot contain `=` or NULL bytes, and
302
- cannot use reserved runtime names such as `MOTHERDUCK_TOKEN` or
303
- `MOTHERDUCK_FLIGHTS_RUN`. Config values cannot contain NULL bytes.
304
- `flightSecretNames` references MotherDuck `TYPE flights` secrets; each secret
305
- param is exposed as `<SECRET_NAME>_<KEY>`, so a secret named `api_secret` with
306
- param `API_KEY` becomes `API_SECRET_API_KEY`.
307
-
308
- ## Querying
309
-
310
- All standard Drizzle query methods work:
191
+ - `duckDbTimestamp` and `duckDbTime` can keep storage variants such as `TIMESTAMP_NS`, `TIMESTAMP_MS`, `TIMESTAMP_S`, `TIME_NS` and `TIME WITH TIME ZONE` through the `duckDbType` and `withTimezone` options.
192
+ - Postgres `json` and `jsonb` columns are rejected. Use `duckDbJson()`.
193
+ - Browser-safe imports live under `@leonardovida-md/drizzle-neo-duckdb/helpers`. Introspection emits this path.
311
194
 
312
- ```typescript
313
- // Select
314
- const users = await db
315
- .select()
316
- .from(usersTable)
317
- .where(eq(usersTable.active, true));
318
-
319
- // Insert
320
- await db
321
- .insert(usersTable)
322
- .values({ name: 'Alice', email: 'alice@example.com' });
323
-
324
- // Insert with returning
325
- const inserted = await db
326
- .insert(usersTable)
327
- .values({ name: 'Bob' })
328
- .returning({ id: usersTable.id });
329
-
330
- // Update
331
- await db
332
- .update(usersTable)
333
- .set({ name: 'Updated' })
334
- .where(eq(usersTable.id, 1));
335
-
336
- // Delete
337
- await db.delete(usersTable).where(eq(usersTable.id, 1));
338
- ```
195
+ See [Column Types](https://leonardovida.github.io/drizzle-duckdb/api/columns) for the full API.
339
196
 
340
- ### Array Operations
197
+ ### Arrays
341
198
 
342
- For DuckDB array operations, use the custom helpers instead of Postgres operators:
199
+ DuckDB supports `@>`, `<@` and `&&` on lists and fixed-size arrays, so Drizzle's `arrayContains`, `arrayContained` and `arrayOverlaps` work unchanged. The DuckDB helpers emit `array_has_all` and `array_has_any`:
343
200
 
344
201
  ```typescript
345
202
  import {
346
- duckDbArrayContains,
347
203
  duckDbArrayContained,
204
+ duckDbArrayContains,
348
205
  duckDbArrayOverlaps,
349
206
  } from '@leonardovida-md/drizzle-neo-duckdb';
350
207
 
351
- // Check if array contains all values
352
- const results = await db
353
- .select()
354
- .from(products)
355
- .where(duckDbArrayContains(products.tags, ['electronics', 'sale']));
356
-
357
- // Check if array is contained by values
358
- const results = await db
359
- .select()
360
- .from(products)
361
- .where(
362
- duckDbArrayContained(products.tags, ['electronics', 'sale', 'featured'])
363
- );
364
-
365
- // Check if arrays overlap
366
- const results = await db
367
- .select()
368
- .from(products)
369
- .where(duckDbArrayOverlaps(products.tags, ['electronics', 'books']));
208
+ // Tags include every value
209
+ db.select().from(products).where(duckDbArrayContains(products.tags, ['electronics', 'sale']));
210
+
211
+ // Tags are a subset of the values
212
+ db.select().from(products).where(duckDbArrayContained(products.tags, ['electronics', 'sale', 'featured']));
213
+
214
+ // Tags share at least one value
215
+ db.select().from(products).where(duckDbArrayOverlaps(products.tags, ['electronics', 'books']));
370
216
  ```
371
217
 
218
+ First-dimension `array_lower(a, 1)` and `array_upper(a, 1)` calls are rewritten to `array_length` expressions, because DuckDB lacks those functions.
219
+
372
220
  ## Transactions
373
221
 
374
222
  ```typescript
@@ -378,11 +226,14 @@ await db.transaction(async (tx) => {
378
226
  });
379
227
  ```
380
228
 
381
- > **Note:** DuckDB doesn't support `SAVEPOINT`, so nested transactions reuse the outer transaction context. Inner rollbacks will abort the entire transaction.
229
+ DuckDB differs from Postgres here:
382
230
 
383
- ## Migrations
231
+ - There is no `SAVEPOINT`. Nested transactions reuse the outer one, so an inner rollback aborts everything.
232
+ - Without a pool, other `db` queries wait while a transaction is open on the shared connection, instead of joining it.
233
+ - Any failed statement aborts the whole transaction. Catching the error inside the callback does not keep earlier writes. `db.transaction()` rolls back and rejects.
234
+ - The `config` argument (`isolationLevel`, `accessMode`, `deferrable`) is deprecated and ignored with a one-time warning, because DuckDB has no `SET TRANSACTION`.
384
235
 
385
- Apply SQL migration files using the `migrate` function:
236
+ ## Migrations
386
237
 
387
238
  ```typescript
388
239
  import { migrate } from '@leonardovida-md/drizzle-neo-duckdb';
@@ -390,19 +241,17 @@ import { migrate } from '@leonardovida-md/drizzle-neo-duckdb';
390
241
  await migrate(db, { migrationsFolder: './drizzle' });
391
242
  ```
392
243
 
393
- Migration metadata is stored in `drizzle.__drizzle_migrations` by default. See [Migrations Documentation](https://leonardovida.github.io/drizzle-duckdb/guide/migrations) for configuration options.
244
+ Metadata goes to `drizzle.__drizzle_migrations` by default. Concurrent `migrate()` calls apply each migration once, and DuckLake works as the default catalog. See [Migrations](https://leonardovida.github.io/drizzle-duckdb/features/migrations) for configuration.
394
245
 
395
246
  ## Schema Introspection
396
247
 
397
- Generate Drizzle schema from an existing DuckDB database:
398
-
399
- ### CLI
248
+ Generate a Drizzle schema from an existing database with the CLI:
400
249
 
401
250
  ```bash
402
251
  bunx duckdb-introspect --url ./my-database.duckdb --out ./drizzle/schema.ts
403
252
  ```
404
253
 
405
- ### Programmatic
254
+ Or from code:
406
255
 
407
256
  ```typescript
408
257
  import { introspect } from '@leonardovida-md/drizzle-neo-duckdb';
@@ -415,80 +264,91 @@ const result = await introspect(db, {
415
264
  console.log(result.files.schemaTs);
416
265
  ```
417
266
 
418
- See [Introspection Documentation](https://leonardovida.github.io/drizzle-duckdb/guide/introspection) for all options.
267
+ See [Introspection](https://leonardovida.github.io/drizzle-duckdb/features/introspection) for all options.
268
+
269
+ ## Analytics and MotherDuck Helpers
419
270
 
420
- ## Configuration Options
271
+ The package ships composable SQL helpers for analytical work:
272
+
273
+ - OLAP helpers such as `sumN`, `percentileCont`, window helpers and the `olap()` grouped measures builder.
274
+ - Lance vector, full-text and hybrid search.
275
+ - MotherDuck table functions for Dives, Flights, file listings and access tokens, plus session identity scalars such as `mdCurrentEmail()` and `mdIsRoleInSession()`.
276
+ - Typed MotherDuck AI builders: `mdPromptJev()` classifies text or images and `mdPrompt()` generates text or structured output.
277
+ - `duckLakeMergeAdjacentFiles()` for DuckLake compaction.
278
+ - Batch and Arrow reads through `db.executeBatches()`, `db.executeBatchesRaw()` and `db.executeArrow()`.
421
279
 
422
280
  ```typescript
423
- const db = drizzle(connection, {
424
- // Enable query logging
425
- logger: new DefaultLogger(),
281
+ import { sql } from 'drizzle-orm';
282
+ import { mdCreateFlight, mdListDives, mdRunFlight } from '@leonardovida-md/drizzle-neo-duckdb';
426
283
 
427
- // Pool size, presets, and timeout/recycling when using connection strings.
428
- pool: { size: 8, idleTimeoutMs: 60_000 },
284
+ const dives = await db.execute(sql`
285
+ select id, title, updated_at
286
+ from ${mdListDives({ limit: 10, includeOrgShares: true })}
287
+ order by updated_at desc
288
+ `);
429
289
 
430
- // Throw on Postgres-style array literals like '{1,2,3}' (default: false)
431
- rejectStringArrayLiterals: false,
290
+ const [flight] = await db.execute(sql`
291
+ select flight_id, status
292
+ from ${mdCreateFlight({
293
+ name: 'daily-refresh',
294
+ sourceCode: 'print("hello")',
295
+ scheduleCron: '0 0 * * *',
296
+ maxRuntimeSec: 1_800,
297
+ instanceType: 'F16',
298
+ })}
299
+ `);
432
300
 
433
- // Pass your schema for relational queries
434
- schema: mySchema,
435
- });
301
+ await db.execute(sql`
302
+ select run_number, status
303
+ from ${mdRunFlight(String(flight.flight_id), { config: { region: 'eu-west-1' } })}
304
+ `);
436
305
  ```
437
306
 
438
- Postgres array operators (`@>`, `<@`, `&&`) are automatically rewritten to DuckDB's `array_has_*` functions via AST transformation. First-dimension `array_lower(a, 1)` and `array_upper(a, 1)` calls are also rewritten to DuckDB-compatible `array_length` expressions.
307
+ The older `mdJobs()` and `mdFlights()` helper names remain as deprecated aliases. See [OLAP Helpers](https://leonardovida.github.io/drizzle-duckdb/api/olap-helpers) for the full list, Flight options and log pagination, and [MotherDuck](https://leonardovida.github.io/drizzle-duckdb/integrations/motherduck) for the AI builders.
439
308
 
440
309
  ## Known Limitations
441
310
 
442
- This connector aims for compatibility with Drizzle's Postgres driver but has some differences:
443
-
444
- | Feature | Status |
445
- | --------------------- | ---------------------------------------------------------------------------- |
446
- | Basic CRUD operations | Full support |
447
- | Joins and subqueries | Full support |
448
- | Transactions | No savepoints (nested transactions reuse outer) |
449
- | JSON/JSONB columns | Use `duckDbJson()` instead |
450
- | Prepared statements | Optional per-connection cache via `prepareCache`; no named statements |
451
- | Streaming results | Chunked reads via `executeBatches()` / `executeArrow()`; no cursor streaming |
452
- | Concurrent queries | One query per connection; use pooling for parallelism |
311
+ | Feature | Status |
312
+ | --------------------- | ----------------------------------------------------------------------------- |
313
+ | Basic CRUD operations | Full support |
314
+ | Joins and subqueries | Full support |
315
+ | Transactions | No savepoints (nested transactions reuse the outer one). Config is ignored |
316
+ | JSON/JSONB columns | Use `duckDbJson()` instead |
317
+ | Prepared statements | Optional per-connection cache via `prepareCache`, no named statements |
318
+ | Streaming results | Chunked reads via `executeBatches()`, no cursor streaming |
319
+ | Concurrent queries | One query per connection. Use pooling for parallelism |
453
320
 
454
- See [Limitations Documentation](https://leonardovida.github.io/drizzle-duckdb/reference/limitations) for details.
321
+ See [Limitations](https://leonardovida.github.io/drizzle-duckdb/reference/limitations) for details.
455
322
 
456
323
  ## Examples
457
324
 
458
- - **[Parquet Analytics](./example/parquet-analytics.ts)**: Compose typed selections and grouped measures over evolving Parquet files without importing tables
325
+ - **[Parquet Analytics](./example/parquet-analytics.ts)**: typed selections and grouped measures over Parquet files without importing tables
326
+ - **[MotherDuck NYC Taxi](./example/motherduck-nyc-taxi.ts)**: query the MotherDuck sample NYC taxi dataset with a connection pool
327
+ - **[Analytics Dashboard](./example/analytics-dashboard.ts)**: local in-memory analytics with DuckDB types and Parquet loading
328
+ - **[DuckLake Local](./example/ducklake-local.ts)** and **[DuckLake on MotherDuck](./example/ducklake-motherduck.ts)**: attach a DuckLake catalog
459
329
 
460
- - **[MotherDuck NYC Taxi](./example/motherduck-nyc.ts)**: Query the built-in NYC taxi dataset from MotherDuck cloud
461
- - **[Analytics Dashboard](./example/analytics-dashboard.ts)**: Local in-memory analytics with DuckDB types and Parquet loading
462
-
463
- Run examples:
330
+ Run them from the repository root:
464
331
 
465
332
  ```bash
466
- MOTHERDUCK_TOKEN=your_token bun example/motherduck-nyc.ts
467
333
  bun example/analytics-dashboard.ts
334
+ MOTHERDUCK_TOKEN=your_token bun example/motherduck-nyc-taxi.ts
468
335
  ```
469
336
 
470
- ## Contributing
337
+ See [example/README.md](./example/README.md) for the full list.
471
338
 
472
- Contributions are welcome! Please:
339
+ ## Contributing
473
340
 
474
- 1. Include tests for new features (`test/<feature>.test.ts`)
475
- 2. Note any DuckDB-specific quirks you encounter
476
- 3. Use a clear, imperative commit message
341
+ Contributions are welcome. See [CONTRIBUTING.md](./CONTRIBUTING.md) for the guide and [SECURITY.md](./SECURITY.md) to report a vulnerability.
477
342
 
478
343
  ```bash
479
- # Install dependencies
480
- bun install
481
-
482
- # Run tests
483
- bun test
484
-
485
- # Run tests with UI
486
- bun t
487
-
488
- # Build
489
- bun run build
344
+ bun install # install dependencies
345
+ bun run test # run tests (Vitest, not `bun test`)
346
+ bun run t # watch mode with the UI
347
+ bun run build # build dist/
490
348
  ```
491
349
 
350
+ Include tests for new features in `test/<feature>.test.ts`, note any DuckDB quirks you hit, and use short imperative commit messages.
351
+
492
352
  ## License
493
353
 
494
354
  [Apache-2.0](./LICENSE)