@molecule/api-database-duckdb 0.0.0-stage → 1.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +202 -0
- package/README.md +772 -2
- package/dist/browser-guard.d.ts +2 -0
- package/dist/browser-guard.d.ts.map +1 -0
- package/dist/browser-guard.js +19 -0
- package/dist/browser-guard.js.map +1 -0
- package/dist/capabilities.d.ts +30 -0
- package/dist/capabilities.d.ts.map +1 -0
- package/dist/capabilities.js +48 -0
- package/dist/capabilities.js.map +1 -0
- package/dist/index.d.ts +115 -0
- package/dist/index.d.ts.map +1 -0
- package/dist/index.js +169 -0
- package/dist/index.js.map +1 -0
- package/dist/instances.d.ts +73 -0
- package/dist/instances.d.ts.map +1 -0
- package/dist/instances.js +117 -0
- package/dist/instances.js.map +1 -0
- package/dist/lifecycle.d.ts +22 -0
- package/dist/lifecycle.d.ts.map +1 -0
- package/dist/lifecycle.js +45 -0
- package/dist/lifecycle.js.map +1 -0
- package/dist/migrator.d.ts +74 -0
- package/dist/migrator.d.ts.map +1 -0
- package/dist/migrator.js +172 -0
- package/dist/migrator.js.map +1 -0
- package/dist/pool.d.ts +40 -0
- package/dist/pool.d.ts.map +1 -0
- package/dist/pool.js +273 -0
- package/dist/pool.js.map +1 -0
- package/dist/queue.d.ts +48 -0
- package/dist/queue.d.ts.map +1 -0
- package/dist/queue.js +98 -0
- package/dist/queue.js.map +1 -0
- package/dist/secrets.d.ts +16 -0
- package/dist/secrets.d.ts.map +1 -0
- package/dist/secrets.js +25 -0
- package/dist/secrets.js.map +1 -0
- package/dist/sql-dialect.d.ts +92 -0
- package/dist/sql-dialect.d.ts.map +1 -0
- package/dist/sql-dialect.js +309 -0
- package/dist/sql-dialect.js.map +1 -0
- package/dist/store.d.ts +18 -0
- package/dist/store.d.ts.map +1 -0
- package/dist/store.js +335 -0
- package/dist/store.js.map +1 -0
- package/dist/types.d.ts +57 -0
- package/dist/types.d.ts.map +1 -0
- package/dist/types.js +7 -0
- package/dist/types.js.map +1 -0
- package/dist/utilities.d.ts +119 -0
- package/dist/utilities.d.ts.map +1 -0
- package/dist/utilities.js +246 -0
- package/dist/utilities.js.map +1 -0
- package/package.json +87 -4
package/README.md
CHANGED
|
@@ -1,3 +1,773 @@
|
|
|
1
|
-
|
|
1
|
+
<!--
|
|
2
|
+
AUTO-GENERATED — DO NOT EDIT THIS FILE.
|
|
3
|
+
Generated by `mlcl sync-docs` from the package's src/index.ts JSDoc + mlcl/registry.json.
|
|
4
|
+
Edits here are overwritten on the next commit (molecule's pre-commit hook regenerates).
|
|
5
|
+
To change this document, edit the module-level JSDoc in src/index.ts.
|
|
6
|
+
Generated: 2026-10-24T17:47:56.117Z
|
|
7
|
+
-->
|
|
2
8
|
|
|
3
|
-
|
|
9
|
+
# @molecule/api-database-duckdb
|
|
10
|
+
|
|
11
|
+
DuckDB database provider for molecule.dev — an embedded analytical SQL
|
|
12
|
+
engine that queries Parquet, CSV and JSON files in-process.
|
|
13
|
+
|
|
14
|
+
Provides DatabasePool and DataStore implementations backed by
|
|
15
|
+
`@duckdb/node-api` (DuckDB 2.0). DuckDB is the SQLite shape, but
|
|
16
|
+
columnar/OLAP: it is the right bond for analytics over files
|
|
17
|
+
(`SELECT * FROM read_parquet('events.parquet')`), aggregations and ad-hoc
|
|
18
|
+
slices — not for a high-write-rate OLTP app (that is the postgresql bond's
|
|
19
|
+
job).
|
|
20
|
+
|
|
21
|
+
DuckDB cannot reproduce a foreign key's `ON DELETE`/`ON UPDATE` `CASCADE`,
|
|
22
|
+
`SET NULL` or `SET DEFAULT` action, and this bond refuses to silently drop
|
|
23
|
+
it: a resource whose setup SQL needs one is refused by the generator's
|
|
24
|
+
compatibility preflight ({@link sqlCapabilities}) before an app is built.
|
|
25
|
+
|
|
26
|
+
> **Auto-generated, AI-first package reference** for the [molecule.dev](https://molecule.dev) ecosystem.
|
|
27
|
+
> It is written to be read by coding agents as much as by people, and is generated from this
|
|
28
|
+
> package's source — edit `src/index.ts` JSDoc, not this file.
|
|
29
|
+
|
|
30
|
+
## Quick Start
|
|
31
|
+
|
|
32
|
+
```typescript
|
|
33
|
+
import { setPool, setStore } from '@molecule/api-database'
|
|
34
|
+
import { pool, store } from '@molecule/api-database-duckdb'
|
|
35
|
+
|
|
36
|
+
setPool(pool)
|
|
37
|
+
setStore(store)
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
## Type
|
|
41
|
+
|
|
42
|
+
`provider`
|
|
43
|
+
|
|
44
|
+
## Installation
|
|
45
|
+
|
|
46
|
+
```bash
|
|
47
|
+
npm install @molecule/api-database-duckdb @duckdb/node-api @molecule/api-bond @molecule/api-database @molecule/api-secrets
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
## API
|
|
51
|
+
|
|
52
|
+
### Interfaces
|
|
53
|
+
|
|
54
|
+
#### `DuckdbInstanceHandle`
|
|
55
|
+
|
|
56
|
+
One opened database: the DuckDB instance, the lock over its single pooled
|
|
57
|
+
connection, and how many pools currently use it.
|
|
58
|
+
|
|
59
|
+
```typescript
|
|
60
|
+
interface DuckdbInstanceHandle {
|
|
61
|
+
/** The DuckDB instance. */
|
|
62
|
+
instance: DuckDBInstance
|
|
63
|
+
/** FIFO lock over the pool's single connection. */
|
|
64
|
+
lock: ConnectionLock
|
|
65
|
+
/** The resolved path (`':memory:'` for in-memory). */
|
|
66
|
+
path: string
|
|
67
|
+
/** Pools currently using this instance. */
|
|
68
|
+
refs: number
|
|
69
|
+
}
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
#### `MigrationOptions`
|
|
73
|
+
|
|
74
|
+
Options for the migration runner.
|
|
75
|
+
|
|
76
|
+
```typescript
|
|
77
|
+
interface MigrationOptions {
|
|
78
|
+
/** Prefix for ledger keys when one database applies SQL from more than one directory. */
|
|
79
|
+
namespace?: string
|
|
80
|
+
}
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
#### `DuckdbConfig`
|
|
84
|
+
|
|
85
|
+
Configuration for the DuckDB database provider.
|
|
86
|
+
|
|
87
|
+
```typescript
|
|
88
|
+
interface DuckdbConfig {
|
|
89
|
+
/**
|
|
90
|
+
* Path of the DuckDB database file — created on first open — or `':memory:'`
|
|
91
|
+
* for a database that lives only for the process's lifetime. Defaults to the
|
|
92
|
+
* `DUCKDB_PATH` env var, then `'./data/app.duckdb'`.
|
|
93
|
+
*/
|
|
94
|
+
path?: string
|
|
95
|
+
/**
|
|
96
|
+
* DuckDB `memory_limit` for the instance. DuckDB's own default is 80% of
|
|
97
|
+
* system RAM, which on a shared VM lets one query evict everything else, so
|
|
98
|
+
* this bond caps it at `'1GB'` by default. Accepts DuckDB size strings
|
|
99
|
+
* (`'512MB'`, `'2GB'`) or a plain byte count as a string. Also settable via
|
|
100
|
+
* the `DUCKDB_MEMORY_LIMIT` env var.
|
|
101
|
+
*/
|
|
102
|
+
memoryLimit?: string
|
|
103
|
+
/**
|
|
104
|
+
* DuckDB `threads` for the instance. DuckDB's own default is every CPU core,
|
|
105
|
+
* which oversubscribes a shared host, so this bond caps it at `4` by default.
|
|
106
|
+
* Also settable via the `DUCKDB_THREADS` env var.
|
|
107
|
+
*/
|
|
108
|
+
threads?: number
|
|
109
|
+
/**
|
|
110
|
+
* Open the database read-only (`access_mode = 'READ_ONLY'`). Multiple
|
|
111
|
+
* processes may read one file concurrently; only read-write mode is
|
|
112
|
+
* exclusive, so a read-only pool never fights an app holding the write lock.
|
|
113
|
+
*/
|
|
114
|
+
readOnly?: boolean
|
|
115
|
+
/**
|
|
116
|
+
* Extra DuckDB settings handed to `DuckDBInstance.fromCache(path, options)`
|
|
117
|
+
* as-is (a flat `Record<string, string>` of setting name → value). Entries
|
|
118
|
+
* here win over the `memoryLimit`/`threads`/`readOnly` fields.
|
|
119
|
+
*/
|
|
120
|
+
config?: Record<string, string>
|
|
121
|
+
/**
|
|
122
|
+
* How long (ms) a query, `connect()` or `transaction()` waits for the single
|
|
123
|
+
* connection before failing with an error that says who is holding it.
|
|
124
|
+
* Defaults to 30 000.
|
|
125
|
+
*/
|
|
126
|
+
acquireTimeoutMillis?: number
|
|
127
|
+
}
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
#### `ProcessEnv`
|
|
131
|
+
|
|
132
|
+
Environment variables consumed by the DuckDB database provider.
|
|
133
|
+
|
|
134
|
+
```typescript
|
|
135
|
+
interface ProcessEnv {
|
|
136
|
+
DUCKDB_PATH?: string
|
|
137
|
+
DUCKDB_MEMORY_LIMIT?: string
|
|
138
|
+
DUCKDB_THREADS?: string
|
|
139
|
+
}
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
#### `TranslateDdlOptions`
|
|
143
|
+
|
|
144
|
+
Options for {@link translateDdlToDuckdb}.
|
|
145
|
+
|
|
146
|
+
```typescript
|
|
147
|
+
interface TranslateDdlOptions {
|
|
148
|
+
/** File name the SQL came from; copied into an unsupported-construct error. */
|
|
149
|
+
source?: string
|
|
150
|
+
}
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
### Types
|
|
154
|
+
|
|
155
|
+
#### `MigratablePool`
|
|
156
|
+
|
|
157
|
+
A pool that can run migrations — `DatabasePool` plus the interactive
|
|
158
|
+
transaction every bond in the `database` category implements (the interface
|
|
159
|
+
marks it optional; no bond ships without it, and the runner needs it).
|
|
160
|
+
|
|
161
|
+
```typescript
|
|
162
|
+
type MigratablePool = DatabasePool & {
|
|
163
|
+
/** Begins a transaction that holds the pool's connection until it ends. */
|
|
164
|
+
transaction(): Promise<DatabaseTransaction>
|
|
165
|
+
}
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
### Classes
|
|
169
|
+
|
|
170
|
+
#### `ConnectionLock`
|
|
171
|
+
|
|
172
|
+
Serializes every use of ONE DuckDB connection.
|
|
173
|
+
|
|
174
|
+
All connections from one instance share one database, and DuckDB detects
|
|
175
|
+
conflicting writes across connections of one instance as transaction
|
|
176
|
+
conflicts — an interactive `BEGIN` … `COMMIT` interleaved with writes from
|
|
177
|
+
another connection fails the transaction. So this pool uses ONE connection
|
|
178
|
+
(the sqlite bond's model): plain `query()` calls hold it for one statement;
|
|
179
|
+
`connect()` and `transaction()` hold it until they are released, so a
|
|
180
|
+
transaction's `BEGIN` → … → `COMMIT`/`ROLLBACK` runs alone.
|
|
181
|
+
|
|
182
|
+
Waiting is bounded: a waiter that cannot get the lock within its timeout is
|
|
183
|
+
removed from the queue and rejected with an error naming the likely cause
|
|
184
|
+
(code holding a connection while calling the pool), instead of hanging forever.
|
|
185
|
+
|
|
186
|
+
```typescript
|
|
187
|
+
class ConnectionLock {
|
|
188
|
+
private locked = false
|
|
189
|
+
private holder: string | null = null
|
|
190
|
+
private waiters: Array<{ grant: () => void }> = []
|
|
191
|
+
|
|
192
|
+
/**
|
|
193
|
+
* Number of callers currently waiting for the lock.
|
|
194
|
+
*
|
|
195
|
+
* @returns The queue length.
|
|
196
|
+
*/
|
|
197
|
+
get waiting(): number {
|
|
198
|
+
return this.waiters.length
|
|
199
|
+
}
|
|
200
|
+
|
|
201
|
+
/**
|
|
202
|
+
* Whether the lock is currently held.
|
|
203
|
+
*
|
|
204
|
+
* @returns `true` when held.
|
|
205
|
+
*/
|
|
206
|
+
get isLocked(): boolean {
|
|
207
|
+
return this.locked
|
|
208
|
+
}
|
|
209
|
+
|
|
210
|
+
/**
|
|
211
|
+
* Waits for the lock in FIFO order. The returned release function MUST be
|
|
212
|
+
* called exactly once; extra calls are ignored.
|
|
213
|
+
*
|
|
214
|
+
* @param purpose - What the caller will do with the lock (`'query'`,
|
|
215
|
+
* `'connect()'`, `'transaction()'`), used in the timeout message.
|
|
216
|
+
* @param timeoutMillis - Maximum wait; `0` or less waits forever.
|
|
217
|
+
* @returns A release function.
|
|
218
|
+
*/
|
|
219
|
+
acquire(purpose: string, timeoutMillis: number): Promise<() => void> {
|
|
220
|
+
let released = false
|
|
221
|
+
const release = (): void => {
|
|
222
|
+
if (released) return
|
|
223
|
+
released = true
|
|
224
|
+
this.holder = null
|
|
225
|
+
const next = this.waiters.shift()
|
|
226
|
+
if (next) {
|
|
227
|
+
next.grant()
|
|
228
|
+
} else {
|
|
229
|
+
this.locked = false
|
|
230
|
+
}
|
|
231
|
+
}
|
|
232
|
+
|
|
233
|
+
if (!this.locked) {
|
|
234
|
+
this.locked = true
|
|
235
|
+
this.holder = purpose
|
|
236
|
+
return Promise.resolve(release)
|
|
237
|
+
}
|
|
238
|
+
|
|
239
|
+
return new Promise<() => void>((resolve, reject) => {
|
|
240
|
+
let timer: ReturnType<typeof setTimeout> | undefined
|
|
241
|
+
const waiter = {
|
|
242
|
+
grant: (): void => {
|
|
243
|
+
if (timer !== undefined) clearTimeout(timer)
|
|
244
|
+
this.locked = true
|
|
245
|
+
this.holder = purpose
|
|
246
|
+
resolve(release)
|
|
247
|
+
},
|
|
248
|
+
}
|
|
249
|
+
this.waiters.push(waiter)
|
|
250
|
+
if (timeoutMillis > 0) {
|
|
251
|
+
timer = setTimeout(() => {
|
|
252
|
+
const index = this.waiters.indexOf(waiter)
|
|
253
|
+
if (index === -1) return
|
|
254
|
+
this.waiters.splice(index, 1)
|
|
255
|
+
reject(
|
|
256
|
+
new Error(
|
|
257
|
+
`DuckDB: a ${purpose} waited ${timeoutMillis} ms for the pool's single ` +
|
|
258
|
+
`connection, which is held by an open ${this.holder ?? 'operation'}. This pool ` +
|
|
259
|
+
`serializes everything on one connection: release() a connect()ed connection and ` +
|
|
260
|
+
`commit()/rollback() a transaction before calling the pool or the store again, and ` +
|
|
261
|
+
`run every statement of a transaction on its own query().`,
|
|
262
|
+
),
|
|
263
|
+
)
|
|
264
|
+
}, timeoutMillis)
|
|
265
|
+
}
|
|
266
|
+
})
|
|
267
|
+
}
|
|
268
|
+
}
|
|
269
|
+
```
|
|
270
|
+
|
|
271
|
+
#### `UnsupportedSqlFeatureError`
|
|
272
|
+
|
|
273
|
+
The error this bond's translator throws when a migration needs a construct
|
|
274
|
+
DuckDB cannot honour. Named and typed so a caller can tell a genuine dialect
|
|
275
|
+
limit apart from malformed SQL, and report it without a migration run.
|
|
276
|
+
|
|
277
|
+
The diagnostic itself is described by the core TYPE contract
|
|
278
|
+
({@link SqlCompatibilityIssue}); this class is the bond's own error
|
|
279
|
+
construction, next to the translation that raises it.
|
|
280
|
+
|
|
281
|
+
```typescript
|
|
282
|
+
class UnsupportedSqlFeatureError extends Error {
|
|
283
|
+
/** The construct's feature. */
|
|
284
|
+
readonly feature: SqlFeature
|
|
285
|
+
/** The exact construct found in the migration. */
|
|
286
|
+
readonly detail: string
|
|
287
|
+
/** Where the statement came from, when known. */
|
|
288
|
+
readonly source?: string
|
|
289
|
+
|
|
290
|
+
/**
|
|
291
|
+
* Build the error from a compatibility diagnostic, copying its message and fields.
|
|
292
|
+
*
|
|
293
|
+
* @param issue - The diagnostic being refused.
|
|
294
|
+
*/
|
|
295
|
+
constructor(issue: SqlCompatibilityIssue) {
|
|
296
|
+
super(issue.message)
|
|
297
|
+
this.name = 'UnsupportedSqlFeatureError'
|
|
298
|
+
this.feature = issue.feature
|
|
299
|
+
this.detail = issue.detail
|
|
300
|
+
this.source = issue.source
|
|
301
|
+
}
|
|
302
|
+
}
|
|
303
|
+
```
|
|
304
|
+
|
|
305
|
+
### Functions
|
|
306
|
+
|
|
307
|
+
#### `defaultPoolCreated()`
|
|
308
|
+
|
|
309
|
+
Whether the lazy default pool has been created.
|
|
310
|
+
|
|
311
|
+
Readiness and shutdown wiring must not FORCE a connection.
|
|
312
|
+
|
|
313
|
+
```typescript
|
|
314
|
+
function defaultPoolCreated(): boolean
|
|
315
|
+
```
|
|
316
|
+
|
|
317
|
+
**Returns:** `true` once the default pool exists.
|
|
318
|
+
|
|
319
|
+
#### `closeDefaultPool()`
|
|
320
|
+
|
|
321
|
+
Close the lazy default pool, if it was created.
|
|
322
|
+
|
|
323
|
+
```typescript
|
|
324
|
+
function closeDefaultPool(): Promise<void>
|
|
325
|
+
```
|
|
326
|
+
|
|
327
|
+
**Returns:** Resolves once the pool has ended.
|
|
328
|
+
|
|
329
|
+
#### `instanceOptionsFor(config)`
|
|
330
|
+
|
|
331
|
+
Resolve the effective instance options for a config: sane `memory_limit` and
|
|
332
|
+
`threads` defaults (DuckDB's own defaults are 80% of RAM and every core,
|
|
333
|
+
which let one in-process engine eat a shared VM whole), `access_mode` for
|
|
334
|
+
`readOnly`, env overrides, then the caller's raw `config` map on top.
|
|
335
|
+
|
|
336
|
+
```typescript
|
|
337
|
+
function instanceOptionsFor(config?: DuckdbConfig): Record<string, string>
|
|
338
|
+
```
|
|
339
|
+
|
|
340
|
+
- `config` — Pool configuration.
|
|
341
|
+
|
|
342
|
+
**Returns:** The options for `DuckDBInstance.fromCache(path, options)`.
|
|
343
|
+
|
|
344
|
+
#### `choosePath(config)`
|
|
345
|
+
|
|
346
|
+
Pick the database path from config, then `DUCKDB_PATH`, then the default.
|
|
347
|
+
|
|
348
|
+
```typescript
|
|
349
|
+
function choosePath(config?: DuckdbConfig): string
|
|
350
|
+
```
|
|
351
|
+
|
|
352
|
+
- `config` — Pool configuration.
|
|
353
|
+
|
|
354
|
+
**Returns:** The path as given (not yet resolved).
|
|
355
|
+
|
|
356
|
+
#### `acquireInstance(config)`
|
|
357
|
+
|
|
358
|
+
Open (or join) the instance for a pool.
|
|
359
|
+
|
|
360
|
+
A FILE path goes through `DuckDBInstance.fromCache`: the official answer to
|
|
361
|
+
"two instances must never open the same file" — every pool in this process
|
|
362
|
+
on one file shares one instance (and one write lock). In-memory databases
|
|
363
|
+
are always PRIVATE (a shared `:memory:` cache entry would hand two test
|
|
364
|
+
pools one database), so they use a fresh `create` and are closed when their
|
|
365
|
+
last pool ends. File-backed cached instances are NEVER closed by this
|
|
366
|
+
module: the client's cache has no eviction, and closing a cached instance
|
|
367
|
+
would hand the next `fromCache` caller a dead one — they live for the
|
|
368
|
+
process, which is also why a file path must not be reused across processes
|
|
369
|
+
in read-write mode.
|
|
370
|
+
|
|
371
|
+
```typescript
|
|
372
|
+
function acquireInstance(config?: DuckdbConfig): Promise<DuckdbInstanceHandle>
|
|
373
|
+
```
|
|
374
|
+
|
|
375
|
+
- `config` — Pool configuration.
|
|
376
|
+
|
|
377
|
+
**Returns:** The instance, with a reference taken for the caller.
|
|
378
|
+
|
|
379
|
+
#### `releaseInstance(handle)`
|
|
380
|
+
|
|
381
|
+
Drop a pool's reference; close the instance when no pool uses it and it is
|
|
382
|
+
one of this module's private in-memory instances.
|
|
383
|
+
|
|
384
|
+
```typescript
|
|
385
|
+
function releaseInstance(handle: DuckdbInstanceHandle): void
|
|
386
|
+
```
|
|
387
|
+
|
|
388
|
+
- `handle` — The instance to release.
|
|
389
|
+
|
|
390
|
+
#### `registerLifecycleIntegration(resources)`
|
|
391
|
+
|
|
392
|
+
Register the database readiness check and shutdown hook with the bonded
|
|
393
|
+
process-lifecycle provider.
|
|
394
|
+
|
|
395
|
+
```typescript
|
|
396
|
+
function registerLifecycleIntegration(resources: {
|
|
397
|
+
ping: () => Promise<void>
|
|
398
|
+
close: () => Promise<void>
|
|
399
|
+
created: () => boolean
|
|
400
|
+
}): void
|
|
401
|
+
```
|
|
402
|
+
|
|
403
|
+
- `resources` — The ping and close functions this bond owns.
|
|
404
|
+
|
|
405
|
+
#### `isIdempotencyError(err)`
|
|
406
|
+
|
|
407
|
+
Returns true when `err` looks like an "already exists" / duplicate-object
|
|
408
|
+
idempotency error — the class of error an `IF NOT EXISTS` migration is
|
|
409
|
+
EXPECTED to hit on a re-run. DuckDB errors carry no code, so this matches
|
|
410
|
+
the message text (the same approach the sqlite bond takes).
|
|
411
|
+
|
|
412
|
+
```typescript
|
|
413
|
+
function isIdempotencyError(err: unknown): boolean
|
|
414
|
+
```
|
|
415
|
+
|
|
416
|
+
- `err` — The error thrown while applying a migration.
|
|
417
|
+
|
|
418
|
+
**Returns:** True if it is safe to warn and continue.
|
|
419
|
+
|
|
420
|
+
#### `applyMigrations(pool, migrationsDir, options)`
|
|
421
|
+
|
|
422
|
+
Apply every migration file in `migrationsDir` to `pool`, in lexical order.
|
|
423
|
+
Exported so a sibling bond (motherduck) can run the same ledger logic on
|
|
424
|
+
its own pool.
|
|
425
|
+
|
|
426
|
+
```typescript
|
|
427
|
+
function applyMigrations(
|
|
428
|
+
pool: MigratablePool,
|
|
429
|
+
migrationsDir: string,
|
|
430
|
+
options?: MigrationOptions,
|
|
431
|
+
): Promise<void>
|
|
432
|
+
```
|
|
433
|
+
|
|
434
|
+
- `pool` — The pool to migrate (its database is created on first open).
|
|
435
|
+
- `migrationsDir` — Absolute path to the directory of ordered `*.sql` files.
|
|
436
|
+
- `options` — Ledger namespace for multi-directory apps.
|
|
437
|
+
|
|
438
|
+
**Returns:** Resolves when every file is applied or recorded.
|
|
439
|
+
|
|
440
|
+
#### `createMigrator(migrationsDir, options)`
|
|
441
|
+
|
|
442
|
+
Returns a `runMigrations()` bound to a migrations directory.
|
|
443
|
+
|
|
444
|
+
```typescript
|
|
445
|
+
function createMigrator(migrationsDir: string, options?: MigrationOptions): () => Promise<void>
|
|
446
|
+
```
|
|
447
|
+
|
|
448
|
+
- `migrationsDir` — Absolute path to the directory of ordered `*.sql` migration files, resolved from the API package directory.
|
|
449
|
+
- `options` — A stable namespace for a second SQL directory.
|
|
450
|
+
|
|
451
|
+
**Returns:** A no-arg `runMigrations()` that opens the DuckDB file (creating it and its parent directory if missing) and applies every migration file in lexical order. Reads the file path from the `DUCKDB_PATH` env var (default `./data/app.duckdb`), matching the pool.
|
|
452
|
+
|
|
453
|
+
#### `runDuckdbQuery(connection, text, values)`
|
|
454
|
+
|
|
455
|
+
Runs one statement on a connection and normalizes its result into the core
|
|
456
|
+
`QueryResult` shape. `$1`-style and `?` placeholders pass through to
|
|
457
|
+
DuckDB untranslated — both are native. With VALUES the client prepares the
|
|
458
|
+
statement (one statement only); without values a multi-statement string
|
|
459
|
+
runs in full and the LAST statement's result is returned (the pglite bond's
|
|
460
|
+
simple-query behavior).
|
|
461
|
+
|
|
462
|
+
`rowCount` follows node-postgres: rows returned for a statement that
|
|
463
|
+
produced rows (a SELECT, or `INSERT/UPDATE/DELETE … RETURNING` — DuckDB
|
|
464
|
+
reports 0 changed rows for a RETURNING statement, so the returned rows ARE
|
|
465
|
+
the affected count), `rowsChanged` for plain DML, `null` for DDL.
|
|
466
|
+
|
|
467
|
+
```typescript
|
|
468
|
+
function runDuckdbQuery(
|
|
469
|
+
connection: DuckDBConnection,
|
|
470
|
+
text: string,
|
|
471
|
+
values?: DuckDBValue[],
|
|
472
|
+
): Promise<QueryResult<T>>
|
|
473
|
+
```
|
|
474
|
+
|
|
475
|
+
- `connection` — The connection to run on.
|
|
476
|
+
- `text` — SQL with `$1, $2, …` or `?` placeholders.
|
|
477
|
+
- `values` — Positional parameter values (already coerced).
|
|
478
|
+
|
|
479
|
+
**Returns:** The normalized result.
|
|
480
|
+
|
|
481
|
+
#### `createPool(config)`
|
|
482
|
+
|
|
483
|
+
Creates a `DatabasePool` backed by DuckDB.
|
|
484
|
+
|
|
485
|
+
The database opens on first use, not here, so creating a pool is free and
|
|
486
|
+
synchronous. See {@link DuckdbConfig} for where the file lives and the
|
|
487
|
+
resource defaults the instance gets.
|
|
488
|
+
|
|
489
|
+
```typescript
|
|
490
|
+
function createPool(config?: DuckdbConfig): DatabasePool
|
|
491
|
+
```
|
|
492
|
+
|
|
493
|
+
- `config` — Database path, resource limits and wait timeout.
|
|
494
|
+
|
|
495
|
+
**Returns:** A pool over one DuckDB connection.
|
|
496
|
+
|
|
497
|
+
#### `createStore(pool)`
|
|
498
|
+
|
|
499
|
+
Creates a DataStore backed by a DuckDB DatabasePool.
|
|
500
|
+
|
|
501
|
+
```typescript
|
|
502
|
+
function createStore(pool: DatabasePool): DataStore
|
|
503
|
+
```
|
|
504
|
+
|
|
505
|
+
- `pool` — The DuckDB DatabasePool to use for queries.
|
|
506
|
+
|
|
507
|
+
**Returns:** A DataStore that translates CRUD operations to DuckDB-compatible SQL.
|
|
508
|
+
|
|
509
|
+
#### `translateDdlToDuckdb(sql, options)`
|
|
510
|
+
|
|
511
|
+
Translate PostgreSQL-dialect DDL to DuckDB-compatible DDL at migration time.
|
|
512
|
+
|
|
513
|
+
Resource/template setup migrations are authored in PostgreSQL dialect. DuckDB
|
|
514
|
+
2.0 already speaks most of it natively — `uuid`/`timestamptz`/`bytea`/`text[]`
|
|
515
|
+
type names, `gen_random_uuid()`, `now()`, `CURRENT_TIMESTAMP`, `::` casts,
|
|
516
|
+
`ADD COLUMN IF NOT EXISTS`, `DROP COLUMN IF EXISTS`, `ON CONFLICT`, `ILIKE` —
|
|
517
|
+
all verified against 2.0.0-r.1. The constructs it rejects are rewritten here,
|
|
518
|
+
and only where the rewrite preserves behaviour:
|
|
519
|
+
|
|
520
|
+
- `jsonb` → `json`, in both the column-type position and the `::jsonb` cast
|
|
521
|
+
(DuckDB has a `JSON` type, stored as VARCHAR; there is no `jsonb`).
|
|
522
|
+
- `USING <method>` on a `CREATE INDEX` (`gin`/`btree`/`hash`/`gist`) →
|
|
523
|
+
removed: DuckDB indexes have no access method.
|
|
524
|
+
- A partial index gets no `WHERE` support in DuckDB. A non-unique partial
|
|
525
|
+
index becomes a full index on the same columns — the predicate only narrows
|
|
526
|
+
which rows the index covers, never a query's result. A UNIQUE partial index
|
|
527
|
+
becomes a UNIQUE index over `CASE WHEN <predicate> THEN <column> END` per
|
|
528
|
+
column: rows outside the predicate get NULL keys (distinct in DuckDB), so
|
|
529
|
+
uniqueness applies to exactly the rows PostgreSQL would constrain.
|
|
530
|
+
|
|
531
|
+
What it does NOT rewrite is an `ON DELETE`/`ON UPDATE` referential action
|
|
532
|
+
(`CASCADE`, `SET NULL`, `SET DEFAULT`): DuckDB rejects the clause at parse
|
|
533
|
+
time and has no triggers, so dropping it would turn a cascading delete into a
|
|
534
|
+
|
|
535
|
+
_(description truncated — read the full text in `node_modules/@molecule/api-database-duckdb/dist/utilities.d.ts`.)_
|
|
536
|
+
|
|
537
|
+
```typescript
|
|
538
|
+
function translateDdlToDuckdb(sql: string, options?: TranslateDdlOptions): string
|
|
539
|
+
```
|
|
540
|
+
|
|
541
|
+
- `sql` — Raw migration SQL, possibly in Postgres dialect.
|
|
542
|
+
- `options` — The source file name used in an unsupported-construct error.
|
|
543
|
+
|
|
544
|
+
**Returns:** DuckDB-compatible SQL.
|
|
545
|
+
|
|
546
|
+
#### `coerceDuckdbParam(value)`
|
|
547
|
+
|
|
548
|
+
Coerce a JS value into something `@duckdb/node-api` can bind. The client
|
|
549
|
+
infers a parameter's type from the VALUE and only understands `null`,
|
|
550
|
+
booleans, numbers, bigints and strings — anything else fails at bind time
|
|
551
|
+
with `Cannot create values of type ANY. Specify a specific type.`, which
|
|
552
|
+
would crash every `create()`/`update()` that passes a `Date`, a boolean-ish
|
|
553
|
+
object or a jsonb value. Strings cast on the way in: DuckDB implicitly casts
|
|
554
|
+
a bound VARCHAR to UUID/TIMESTAMPTZ/DECIMAL/JSON/LIST parameters, verified
|
|
555
|
+
against 2.0.0-r.1.
|
|
556
|
+
|
|
557
|
+
```typescript
|
|
558
|
+
function coerceDuckdbParam(value: unknown): DuckDBValue
|
|
559
|
+
```
|
|
560
|
+
|
|
561
|
+
- `value` — The value to bind.
|
|
562
|
+
|
|
563
|
+
**Returns:** A `@duckdb/node-api`-bindable value.
|
|
564
|
+
|
|
565
|
+
#### `normalizeDuckdbRows(rows)`
|
|
566
|
+
|
|
567
|
+
Normalize DuckDB result rows to the JS shapes the other database bonds hand
|
|
568
|
+
back. DuckDB's JS conversion returns a `bigint` for every 64+-bit integer —
|
|
569
|
+
`COUNT(*)`, `BIGINT` and `HUGEINT` columns — and `JSON.stringify` THROWS on a
|
|
570
|
+
bigint, so a handler that serializes a row with `COUNT(*)` in it would 500.
|
|
571
|
+
A bigint within `Number.MAX_SAFE_INTEGER` becomes a JS number (matching the
|
|
572
|
+
sqlite bond); a larger one becomes its decimal string (matching node-postgres
|
|
573
|
+
`int8AsString`). Mutates rows in place for speed.
|
|
574
|
+
|
|
575
|
+
```typescript
|
|
576
|
+
function normalizeDuckdbRows(rows: Record<string, unknown>[]): T[]
|
|
577
|
+
```
|
|
578
|
+
|
|
579
|
+
- `rows` — Row objects from `result.getRowObjectsJS()`.
|
|
580
|
+
|
|
581
|
+
**Returns:** The normalized rows.
|
|
582
|
+
|
|
583
|
+
#### `parseJsonColumns(rows, jsonColumns)`
|
|
584
|
+
|
|
585
|
+
Parse the JSON-typed columns of a table's rows into values, matching what
|
|
586
|
+
the postgresql/sqlite bonds hand back from a `json`/`jsonb` column. DuckDB's
|
|
587
|
+
`JSON` type is an alias over VARCHAR, so a JSON column always READS as its
|
|
588
|
+
JSON text — the column's type is only visible in
|
|
589
|
+
`information_schema.columns.data_type`, never in the result metadata, which
|
|
590
|
+
is why this runs in the DataStore (it knows the table) and not in the pool.
|
|
591
|
+
A value that is not valid JSON text is left as the raw string.
|
|
592
|
+
|
|
593
|
+
```typescript
|
|
594
|
+
function parseJsonColumns(rows: Record<string, unknown>[], jsonColumns: Set<string>): T[]
|
|
595
|
+
```
|
|
596
|
+
|
|
597
|
+
- `rows` — Row objects.
|
|
598
|
+
- `jsonColumns` — The table's JSON-typed column names, lower-cased.
|
|
599
|
+
|
|
600
|
+
**Returns:** The rows with JSON columns parsed.
|
|
601
|
+
|
|
602
|
+
### Constants
|
|
603
|
+
|
|
604
|
+
#### `sqlCapabilities`
|
|
605
|
+
|
|
606
|
+
What the DuckDB bond supports from PostgreSQL-dialect migrations.
|
|
607
|
+
|
|
608
|
+
`fk-referential-action` is deliberately `unsupported`: applying the setup
|
|
609
|
+
library with the action dropped would make a parent delete fail while child
|
|
610
|
+
rows exist, where PostgreSQL cascades — a silently changed delete, which this
|
|
611
|
+
bond refuses rather than reproduces inexactly.
|
|
612
|
+
|
|
613
|
+
```typescript
|
|
614
|
+
const sqlCapabilities: Partial<Record<SqlFeature, SqlCapability>>
|
|
615
|
+
```
|
|
616
|
+
|
|
617
|
+
#### `pool`
|
|
618
|
+
|
|
619
|
+
The DuckDB connection pool instance.
|
|
620
|
+
|
|
621
|
+
```typescript
|
|
622
|
+
const pool: DatabasePool
|
|
623
|
+
```
|
|
624
|
+
|
|
625
|
+
#### `store`
|
|
626
|
+
|
|
627
|
+
Default DataStore proxy backed by the lazily-initialized default pool.
|
|
628
|
+
|
|
629
|
+
```typescript
|
|
630
|
+
const store: DataStore
|
|
631
|
+
```
|
|
632
|
+
|
|
633
|
+
#### `MEMORY_PATH`
|
|
634
|
+
|
|
635
|
+
The in-memory database path.
|
|
636
|
+
|
|
637
|
+
```typescript
|
|
638
|
+
const MEMORY_PATH: ':memory:'
|
|
639
|
+
```
|
|
640
|
+
|
|
641
|
+
#### `DEFAULT_DUCKDB_PATH`
|
|
642
|
+
|
|
643
|
+
Default on-disk database path.
|
|
644
|
+
|
|
645
|
+
```typescript
|
|
646
|
+
const DEFAULT_DUCKDB_PATH: './data/app.duckdb'
|
|
647
|
+
```
|
|
648
|
+
|
|
649
|
+
#### `DEFAULT_MEMORY_LIMIT`
|
|
650
|
+
|
|
651
|
+
Default `memory_limit`: DuckDB's own default is 80% of RAM — far too much on a shared host.
|
|
652
|
+
|
|
653
|
+
```typescript
|
|
654
|
+
const DEFAULT_MEMORY_LIMIT: '1GB'
|
|
655
|
+
```
|
|
656
|
+
|
|
657
|
+
#### `DEFAULT_THREADS`
|
|
658
|
+
|
|
659
|
+
Default `threads`: DuckDB's own default is every core — oversubscribes a shared host.
|
|
660
|
+
|
|
661
|
+
```typescript
|
|
662
|
+
const DEFAULT_THREADS: 4
|
|
663
|
+
```
|
|
664
|
+
|
|
665
|
+
#### `MIGRATION_LEDGER`
|
|
666
|
+
|
|
667
|
+
The table that records which migration files have run.
|
|
668
|
+
|
|
669
|
+
```typescript
|
|
670
|
+
const MIGRATION_LEDGER: '_molecule_migrations'
|
|
671
|
+
```
|
|
672
|
+
|
|
673
|
+
#### `databaseDuckdbSecretDefinitions`
|
|
674
|
+
|
|
675
|
+
Secret definitions required by the DuckDB database bond.
|
|
676
|
+
|
|
677
|
+
```typescript
|
|
678
|
+
const databaseDuckdbSecretDefinitions: SecretDefinition[]
|
|
679
|
+
```
|
|
680
|
+
|
|
681
|
+
## Core Interface
|
|
682
|
+
|
|
683
|
+
Implements `@molecule/api-database` interface.
|
|
684
|
+
|
|
685
|
+
## Bond Wiring
|
|
686
|
+
|
|
687
|
+
Setup function to register this provider with the core interface:
|
|
688
|
+
|
|
689
|
+
```typescript
|
|
690
|
+
import { setPool, setStore } from '@molecule/api-database'
|
|
691
|
+
import { pool, store } from '@molecule/api-database-duckdb'
|
|
692
|
+
|
|
693
|
+
export function setupDatabaseDuckdb(): void {
|
|
694
|
+
setPool(pool)
|
|
695
|
+
setStore(store)
|
|
696
|
+
}
|
|
697
|
+
```
|
|
698
|
+
|
|
699
|
+
## Injection Notes
|
|
700
|
+
|
|
701
|
+
### Requirements
|
|
702
|
+
|
|
703
|
+
Peer dependencies:
|
|
704
|
+
|
|
705
|
+
- `@molecule/api-bond` ^1.0.1
|
|
706
|
+
- `@molecule/api-database` ^1.2.0
|
|
707
|
+
- `@molecule/api-secrets` ^1.0.1
|
|
708
|
+
|
|
709
|
+
### Environment Variables
|
|
710
|
+
|
|
711
|
+
- `DUCKDB_PATH` _(optional)_ — DuckDB database path — default: `./data/app.duckdb`
|
|
712
|
+
- Setup: Filesystem path of the DuckDB database file (created on first run), or :memory:.
|
|
713
|
+
- Example: `./data/app.duckdb`
|
|
714
|
+
|
|
715
|
+
### Runtime Dependencies
|
|
716
|
+
|
|
717
|
+
- `@duckdb/node-api`
|
|
718
|
+
- `@molecule/api-bond`
|
|
719
|
+
- `@molecule/api-database`
|
|
720
|
+
- `@molecule/api-secrets`
|
|
721
|
+
|
|
722
|
+
Configure via the `DUCKDB_PATH` environment variable (default:
|
|
723
|
+
`./data/app.duckdb`; `:memory:` for a database that lives only for the
|
|
724
|
+
process's lifetime).
|
|
725
|
+
|
|
726
|
+
- **`pool.query(text, values)` takes Postgres-style `$1, $2` placeholders —
|
|
727
|
+
or `?` — natively, with no translation.** With VALUES a string must be
|
|
728
|
+
ONE statement (the client prepares it); without values a multi-statement
|
|
729
|
+
string runs in full and the LAST statement's result is returned.
|
|
730
|
+
`rowCount` follows node-postgres: rows returned for a SELECT or
|
|
731
|
+
`…RETURNING` statement, rows changed for plain INSERT/UPDATE/DELETE,
|
|
732
|
+
`null` for DDL.
|
|
733
|
+
- **One process per file, in read-write mode.** DuckDB holds a file lock:
|
|
734
|
+
the dev server and a CLI must never open the same `.duckdb` file
|
|
735
|
+
read-write at the same time (`Could not set lock on file`). This pool
|
|
736
|
+
shares one instance per file inside the process (`fromCache`) and
|
|
737
|
+
serializes every statement on ONE connection behind a FIFO queue, the
|
|
738
|
+
sqlite bond's model — a `transaction()` holds the queue for its whole
|
|
739
|
+
`BEGIN` → … → `COMMIT`/`ROLLBACK` lifetime. A second PROCESS wanting
|
|
740
|
+
concurrent reads should open `readOnly: true`.
|
|
741
|
+
- **Resource defaults differ from DuckDB's own.** DuckDB defaults to 80% of
|
|
742
|
+
RAM for `memory_limit` and one thread per core — enough to eat a shared
|
|
743
|
+
host. This bond defaults to `memory_limit: '1GB'` and `threads: 4`,
|
|
744
|
+
overridable via `DUCKDB_MEMORY_LIMIT` / `DUCKDB_THREADS` or the
|
|
745
|
+
`memoryLimit`/`threads` config fields.
|
|
746
|
+
- **Files are the point:** `read_parquet`, `read_csv`, `read_json` and the
|
|
747
|
+
`'file.parquet'` shorthand all work through `pool.query()` as plain SQL.
|
|
748
|
+
They read relative to the API process's working directory.
|
|
749
|
+
- **JSON columns read as JSON TEXT through raw `query()`** — DuckDB's
|
|
750
|
+
`JSON` type is an alias over VARCHAR, so the type is invisible in result
|
|
751
|
+
metadata. The DataStore CRUD helpers (`create`, `findMany`, …) parse
|
|
752
|
+
JSON-typed columns for you; in raw SQL use `data -> 'key'` /
|
|
753
|
+
`json_extract` or accept the string.
|
|
754
|
+
- **`COUNT(*)` and BIGINT come back as JS numbers** when they fit
|
|
755
|
+
`Number.MAX_SAFE_INTEGER` (raw DuckDB returns `bigint`, which
|
|
756
|
+
`JSON.stringify` throws on); a larger value becomes its decimal string,
|
|
757
|
+
matching node-postgres.
|
|
758
|
+
- **Schema changes are `*.sql` files in the app's `api/migrations/`**,
|
|
759
|
+
applied in lexical order on startup by `createMigrator`. Postgres-dialect
|
|
760
|
+
DDL applies after a translation that keeps behaviour: `jsonb` → `json`,
|
|
761
|
+
index `USING <method>` dropped, and a partial index applied as a full index
|
|
762
|
+
or (UNIQUE) as a `CASE WHEN` expression index. `gen_random_uuid()`,
|
|
763
|
+
`now()`, `ADD/DROP COLUMN IF [NOT] EXISTS`, `::` casts and `uuid`/
|
|
764
|
+
`timestamptz`/`bytea`/`text[]` types are native. An `ON DELETE`/
|
|
765
|
+
`ON UPDATE` referential action throws `UnsupportedSqlFeatureError` —
|
|
766
|
+
DuckDB cannot reproduce it; each file runs once (recorded in
|
|
767
|
+
`_molecule_migrations`) and in one transaction.
|
|
768
|
+
- **2.0 file format is a one-way break:** versions before DuckDB 2.0 cannot
|
|
769
|
+
read files this bond creates (2.0 still reads 1.x files). The engine is
|
|
770
|
+
OLAP — bulk scans and aggregations are its home; an app doing many small
|
|
771
|
+
row transactions belongs on the postgresql bond.
|
|
772
|
+
- `pool.end()` closes the pool's connection; a FILE-backed instance is
|
|
773
|
+
process-lifetime by design (the client's instance cache has no eviction).
|