@statewalker/db-duckdb-browser 0.1.6 → 0.1.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -7,6 +7,11 @@ A DuckDB driver for the browser that implements `Db` from
7
7
  Worker. The database lives in memory, or in an OPFS file that survives page
8
8
  reloads.
9
9
 
10
+ > **Known issue:** with `@duckdb/duckdb-wasm` 1.33.1-dev57.0, a new OPFS database is not
11
+ > actually written to OPFS, so it does not survive a reopen
12
+ > ([#74](https://github.com/statewalker/statewalker-db/issues/74)). The persistence tests in
13
+ > `src/opfs-persistence.browser.test.ts` are marked as expected failures until that is fixed.
14
+
10
15
  ## Why it exists
11
16
 
12
17
  DuckDB WASM has its own asynchronous API built around Arrow tables, worker
@@ -29,12 +34,56 @@ pnpm add @statewalker/db-duckdb-browser
29
34
  - `options.path`: an OPFS path such as `opfs://app.duckdb`. Omit it for an
30
35
  in-memory database. If the path cannot be opened, the call rejects with
31
36
  `Cannot open DuckDB database "<path>"`; it never falls back to memory.
32
- - `options.bundles`: same-origin `DuckDBBundles` URLs. Required for OPFS.
33
- Without it, the bundle is loaded from jsDelivr.
37
+ - `options.bundles`: same-origin `DuckDBBundles` URLs. Required for OPFS:
38
+ `options.path` without `options.bundles` rejects with
39
+ `Cannot open DuckDB database "<path>": OPFS needs self-hosted \`bundles\``
40
+ before any worker starts. Without it, the bundle is loaded from jsDelivr.
34
41
 
35
42
  The returned `Db` implements `query`, `exec`, `flush` and `close`. Placeholders
36
43
  are DuckDB's (`$1`, `$2`, ...).
37
44
 
45
+ ### Result values
46
+
47
+ Rows come back with the same JS values as `@statewalker/db-duckdb-node`:
48
+
49
+ | DuckDB type | JS value |
50
+ | --- | --- |
51
+ | `DECIMAL` | `number` (scaled; `1.25::DECIMAL(10,2)` is `1.25`) |
52
+ | `BIGINT`, `UBIGINT`, `HUGEINT`, `TIME` | `bigint` |
53
+ | `INTEGER`, `DOUBLE`, ... | `number` |
54
+ | `DATE`, `TIMESTAMP`, `TIMESTAMPTZ` | `Date` |
55
+ | `LIST`, fixed-size `ARRAY` | array, converted recursively |
56
+ | `STRUCT` | plain object, converted recursively |
57
+ | `MAP` | array of `{ key, value }` |
58
+ | `BLOB` | `Uint8Array` |
59
+
60
+ `INTERVAL` is returned as duckdb-wasm's Arrow layer decodes it and does not
61
+ match the node driver; cast it in SQL (for example to `VARCHAR`) if you need
62
+ it. A `HUGEINT` and a `DECIMAL(38,0)` look the same in Arrow, so both come
63
+ back as `bigint`.
64
+
65
+ ### Parameter limits
66
+
67
+ duckdb-wasm passes parameters through `JSON.stringify`, so the driver converts
68
+ them first:
69
+
70
+ | JS value | Bound as |
71
+ | --- | --- |
72
+ | `string`, `boolean`, `null` | itself (`undefined` binds as `null`) |
73
+ | finite `number` | `DOUBLE` |
74
+ | `bigint` | its exact decimal string, cast by DuckDB to the column's type |
75
+ | `Date` | its ISO 8601 string (`toISOString()`), cast to `TIMESTAMP` the same way |
76
+
77
+ The string forms work where the SQL fixes the parameter's type, as in
78
+ `WHERE id = $1` or `INSERT INTO t (id) VALUES ($1)`, so a `BIGINT` id read
79
+ from one query can be passed back to the next. Where nothing fixes it, cast
80
+ in SQL: `$1::BIGINT + 1`, `$1::TIMESTAMP`.
81
+
82
+ `Uint8Array` and other binary data, arrays, objects, `NaN`, `Infinity` and
83
+ invalid dates are rejected with a `TypeError` that names the parameter and its
84
+ type (`cannot bind parameter $1 of type Uint8Array`). Insert such values with
85
+ SQL literals, or through duckdb-wasm's own Arrow insert API.
86
+
38
87
  ## Examples
39
88
 
40
89
  In memory, bundle from jsDelivr:
@@ -82,24 +131,31 @@ await db.flush?.(); // CHECKPOINT: without it the insert can be lost on reload
82
131
  - **Same-origin worker for OPFS.** OPFS file I/O works only in a same-origin
83
132
  worker. With `bundles`, the worker is created directly from your URL. Without
84
133
  them, the jsDelivr worker is cross-origin and is started through a Blob that
85
- calls `importScripts`; that worker crashes on OPFS file I/O, so use it for
86
- in-memory databases only.
134
+ calls `importScripts`; that worker crashes on OPFS file I/O, so the driver
135
+ accepts it for in-memory databases only and rejects a `path` without
136
+ `bundles`.
137
+ - **A failed start leaves nothing running.** If loading the engine, opening
138
+ the path or connecting fails, the engine and its worker are terminated and
139
+ the jsDelivr Blob URL is revoked before the call rejects with the original
140
+ error (`src/start-duckdb.ts`).
87
141
  - **`flush` runs `CHECKPOINT`.** DuckDB WASM writes the OPFS database file only
88
142
  on `CHECKPOINT`. Writes not yet checkpointed can be lost on an abrupt
89
143
  teardown such as a page reload. On an in-memory database `flush` does
90
144
  nothing.
91
- - **No fallback to memory.** If `options.path` is set but `navigator.storage`
145
+ - **No fallback to memory.** If `options.path` is set without `bundles`, the
146
+ call rejects before starting a worker (see above). If `navigator.storage`
92
147
  is missing, the call rejects with
93
148
  `Cannot open DuckDB database "<path>": OPFS is unavailable`. If opening the
94
149
  path throws, the worker is terminated and the call rejects with
95
150
  `Cannot open DuckDB database "<path>"`, the engine's error as `cause`. An
96
151
  in-memory database is created only when `options.path` is omitted, so data
97
152
  is never silently lost on reload.
98
- - **Rows.** Query results are Arrow tables; the driver copies each row's
99
- fields into a plain object. Calls with parameters go through
100
- `conn.prepare(sql)` and `stmt.query(...params)`, and the statement is closed
101
- after each call, also when the query fails; calls without parameters use
102
- `conn.query(sql)`.
153
+ - **Rows.** Query results are Arrow tables; the driver copies each row into a
154
+ plain object, converting each value by its Arrow field type
155
+ (`src/arrow-values.ts`). Calls with parameters convert them
156
+ (`src/params.ts`), then go through `conn.prepare(sql)` and
157
+ `stmt.query(...params)`, and the statement is closed after each call, also
158
+ when the query fails; calls without parameters use `conn.query(sql)`.
103
159
  - **Under Node.js** the module imports, but `newBrowserDuckDb()` rejects with
104
160
  `Worker is not defined`. Use `@statewalker/db-duckdb-node` there.
105
161
  - **Dependencies:** `@duckdb/duckdb-wasm` (the engine), `@statewalker/db-api`
package/dist/index.d.mts CHANGED
@@ -8,7 +8,8 @@ interface BrowserDuckDbOptions extends DbOptions$1 {
8
8
  * `?url` import). Passing these makes the factory spawn a **same-origin**
9
9
  * worker, which OPFS database persistence requires — a cross-origin worker
10
10
  * (the jsDelivr fallback, loaded via a Blob `importScripts` shim) crashes on
11
- * OPFS file I/O. Omit to load from jsDelivr (in-memory use only).
11
+ * OPFS file I/O. Omit to load from jsDelivr (in-memory use only); a `path`
12
+ * without `bundles` is rejected.
12
13
  */
13
14
  bundles?: duckdb.DuckDBBundles;
14
15
  }
@@ -19,6 +20,7 @@ interface BrowserDuckDbOptions extends DbOptions$1 {
19
20
  * `options.bundles`, the database persists across reloads on OPFS. Without
20
21
  * `options.path` it runs in-memory. If `options.path` cannot be opened, the
21
22
  * returned promise rejects; it never falls back to an in-memory database.
23
+ * `options.path` without `options.bundles` rejects before any worker starts.
22
24
  *
23
25
  * @param options.path `opfs://` path for persistent storage. Omit for in-memory.
24
26
  * @param options.bundles Same-origin bundle URLs (required for OPFS).
@@ -1 +1 @@
1
- {"version":3,"file":"index.d.mts","names":["DbOptions","Db"],"sources":["../src/browser-duckdb.ts"],"mappings":";;;;UAoBiB,6BAA6BA;;;;;;;;EAQ5C,UAAU,OAAO;;;;;;;;;;;;;wBAcG,iBAAiB,UAAU,uBAAuB,QAAQC"}
1
+ {"version":3,"file":"index.d.mts","names":["DbOptions","Db"],"sources":["../src/browser-duckdb.ts"],"mappings":";;;;UAOiB,6BAA6BA;;;;;;;;;EAS5C,UAAU,OAAO;;;;;;;;;;;;;;wBAeG,iBAAiB,UAAU,uBAAuB,QAAQC"}
package/dist/index.mjs CHANGED
@@ -1,65 +1,252 @@
1
1
  import * as duckdb from "@duckdb/duckdb-wasm";
2
- //#region src/browser-duckdb.ts
2
+ //#region src/arrow-values.ts
3
+ /**
4
+ * Conversion of Arrow values (as duckdb-wasm returns them) into the plain JS
5
+ * values `@statewalker/db-duckdb-node` returns for the same SQL types.
6
+ *
7
+ * Typed structurally so this module does not import `apache-arrow` at runtime.
8
+ * Type ids are Arrow's `Type` enum (apache-arrow 17, the version duckdb-wasm
9
+ * depends on).
10
+ */
11
+ const TypeId = {
12
+ Dictionary: -1,
13
+ Decimal: 7,
14
+ Date: 8,
15
+ Timestamp: 10,
16
+ List: 12,
17
+ Struct: 13,
18
+ FixedSizeList: 16,
19
+ Map: 17
20
+ };
21
+ /** Arrow `TimeUnit`. */
22
+ const TimeUnit = {
23
+ SECOND: 0,
24
+ MILLISECOND: 1,
25
+ MICROSECOND: 2,
26
+ NANOSECOND: 3
27
+ };
3
28
  /**
4
- * Convert an Arrow Table (returned by duckdb-wasm queries) to plain JS objects.
29
+ * Convert an Arrow table into plain row objects, converting each value by its
30
+ * field's Arrow type ({@link arrowValueToJs}).
5
31
  */
6
- function arrowToObjects(table) {
7
- const fields = table.schema.fields.map((f) => f.name);
8
- return table.toArray().map((row) => {
32
+ function arrowTableToObjects(table) {
33
+ const fields = table.schema.fields;
34
+ return Array.from(table.toArray(), (row) => {
35
+ const source = row;
9
36
  const obj = {};
10
- for (const field of fields) obj[field] = row[field];
37
+ for (const field of fields) obj[field.name] = arrowValueToJs(source[field.name], field.type);
11
38
  return obj;
12
39
  });
13
40
  }
14
41
  /**
42
+ * Convert one Arrow value into the plain JS value the node driver returns:
43
+ *
44
+ * - DECIMAL: a `number` scaled by the type's scale (HUGEINT, which DuckDB sends
45
+ * as DECIMAL(38,0), becomes a `bigint`);
46
+ * - DATE, TIMESTAMP: a `Date`;
47
+ * - LIST, fixed-size ARRAY: an array, converted recursively;
48
+ * - STRUCT: a plain object, converted recursively;
49
+ * - MAP: an array of `{ key, value }`, converted recursively.
50
+ *
51
+ * Other values (numbers, `bigint` for BIGINT, strings, booleans, `Uint8Array`
52
+ * for BLOB, `null`) are returned as they are.
53
+ */
54
+ function arrowValueToJs(value, type) {
55
+ if (value === null || value === void 0) return null;
56
+ switch (type.typeId) {
57
+ case TypeId.Decimal: return decimalToJs(value, type);
58
+ case TypeId.Date: return typeof value === "number" ? new Date(value) : value;
59
+ case TypeId.Timestamp: return timestampToDate(value, type.unit);
60
+ case TypeId.List:
61
+ case TypeId.FixedSizeList: {
62
+ const childType = type.children?.[0]?.type;
63
+ const items = Array.from(value);
64
+ return childType ? items.map((item) => arrowValueToJs(item, childType)) : items;
65
+ }
66
+ case TypeId.Struct: {
67
+ const row = value;
68
+ const obj = {};
69
+ for (const child of type.children ?? []) obj[child.name] = arrowValueToJs(row[child.name], child.type);
70
+ return obj;
71
+ }
72
+ case TypeId.Map: {
73
+ const entryFields = type.children?.[0]?.type.children ?? [];
74
+ const keyType = entryFields[0]?.type;
75
+ const valueType = entryFields[1]?.type;
76
+ return Array.from(value, ([key, val]) => ({
77
+ key: keyType ? arrowValueToJs(key, keyType) : key,
78
+ value: valueType ? arrowValueToJs(val, valueType) : val
79
+ }));
80
+ }
81
+ case TypeId.Dictionary: return type.dictionary ? arrowValueToJs(value, type.dictionary) : value;
82
+ default: return value;
83
+ }
84
+ }
85
+ /**
86
+ * Arrow decimals are unscaled integers (`String(v)` gives the digits). Insert
87
+ * the decimal point by text and parse, so the result is the nearest double to
88
+ * the exact decimal value.
89
+ */
90
+ function decimalToJs(value, type) {
91
+ const digits = String(value);
92
+ const scale = type.scale ?? 0;
93
+ if (scale === 0) return type.precision === 38 ? BigInt(digits) : Number(digits);
94
+ const negative = digits.startsWith("-");
95
+ const abs = (negative ? digits.slice(1) : digits).padStart(scale + 1, "0");
96
+ const text = `${abs.slice(0, abs.length - scale)}.${abs.slice(abs.length - scale)}`;
97
+ return Number(negative ? `-${text}` : text);
98
+ }
99
+ function timestampToDate(value, unit) {
100
+ if (typeof value === "number") return new Date(value);
101
+ if (typeof value === "bigint") switch (unit) {
102
+ case TimeUnit.SECOND: return /* @__PURE__ */ new Date(Number(value) * 1e3);
103
+ case TimeUnit.MICROSECOND: return new Date(Number(value / 1000n));
104
+ case TimeUnit.NANOSECOND: return new Date(Number(value / 1000000n));
105
+ default: return new Date(Number(value));
106
+ }
107
+ return value;
108
+ }
109
+ //#endregion
110
+ //#region src/params.ts
111
+ /**
112
+ * Convert `Db.query` params into values duckdb-wasm can bind.
113
+ *
114
+ * duckdb-wasm passes prepared-statement params through `JSON.stringify`, so
115
+ * only strings, finite numbers, booleans and `null` survive. This maps:
116
+ *
117
+ * - `bigint` to its exact decimal string; DuckDB casts it to the column type
118
+ * where the context fixes one (`WHERE id = $1`, `INSERT ... VALUES ($1)`),
119
+ * elsewhere cast it in SQL (`$1::BIGINT`);
120
+ * - `Date` to its ISO 8601 string, which DuckDB casts to TIMESTAMP the same way;
121
+ * - `undefined` to `null`.
122
+ *
123
+ * Anything else (`Uint8Array` and other binary data, arrays, plain objects,
124
+ * `NaN`, `Infinity`, symbols, functions, invalid dates) throws a `TypeError`
125
+ * naming the parameter and its type instead of failing inside duckdb-wasm or
126
+ * binding the wrong value.
127
+ */
128
+ function toDuckDbParams(params) {
129
+ return params.map(toDuckDbParam);
130
+ }
131
+ function toDuckDbParam(value, index) {
132
+ switch (typeof value) {
133
+ case "string":
134
+ case "boolean": return value;
135
+ case "undefined": return null;
136
+ case "bigint": return value.toString();
137
+ case "number":
138
+ if (Number.isFinite(value)) return value;
139
+ throw unsupported(index, String(value));
140
+ case "object":
141
+ if (value === null) return null;
142
+ if (value instanceof Date) {
143
+ if (Number.isNaN(value.getTime())) throw unsupported(index, "invalid Date");
144
+ return value.toISOString();
145
+ }
146
+ if (Array.isArray(value)) throw unsupported(index, "array");
147
+ if (ArrayBuffer.isView(value) || value instanceof ArrayBuffer) throw unsupported(index, value.constructor.name);
148
+ throw unsupported(index, `object (${value.constructor?.name ?? "null prototype"})`);
149
+ default: throw unsupported(index, typeof value);
150
+ }
151
+ }
152
+ function unsupported(index, type) {
153
+ return /* @__PURE__ */ new TypeError(`db-duckdb-browser: cannot bind parameter $${index + 1} of type ${type}; duckdb-wasm binds only string, finite number, boolean, null, bigint and Date`);
154
+ }
155
+ //#endregion
156
+ //#region src/start-duckdb.ts
157
+ /** The real browser environment: a Web Worker and the page's `URL`. */
158
+ const browserStartEnv = {
159
+ newWorker: (url) => new Worker(url),
160
+ newDb: (worker) => new duckdb.AsyncDuckDB(new duckdb.ConsoleLogger(), worker),
161
+ createObjectURL: (blob) => URL.createObjectURL(blob),
162
+ revokeObjectURL: (url) => URL.revokeObjectURL(url)
163
+ };
164
+ /**
165
+ * Start a worker, instantiate DuckDB in it, open `path` if given, and connect.
166
+ *
167
+ * If any step fails, including the worker script failing to load, the engine
168
+ * and its worker are terminated before the error is rethrown, so a failed start leaves no worker running. The Blob URL
169
+ * of the jsDelivr shim is revoked whether the start succeeds or fails.
170
+ */
171
+ async function startDuckDb(bundle, options, env) {
172
+ const { selfHosted, path } = options;
173
+ let blobUrl;
174
+ let worker;
175
+ let db;
176
+ let onWorkerError;
177
+ const workerFailed = new Promise((_, reject) => {
178
+ onWorkerError = (event) => reject(/* @__PURE__ */ new Error(`DuckDB worker failed to load: ${event.message ?? "unknown error"}`));
179
+ });
180
+ workerFailed.catch(() => {});
181
+ try {
182
+ if (selfHosted) worker = env.newWorker(bundle.mainWorker);
183
+ else {
184
+ blobUrl = env.createObjectURL(new Blob([`importScripts("${bundle.mainWorker ?? ""}");`], { type: "text/javascript" }));
185
+ worker = env.newWorker(blobUrl);
186
+ }
187
+ worker.addEventListener("error", onWorkerError);
188
+ db = env.newDb(worker);
189
+ await Promise.race([db.instantiate(bundle.mainModule, bundle.pthreadWorker), workerFailed]);
190
+ if (path) try {
191
+ await db.open({
192
+ path,
193
+ accessMode: duckdb.DuckDBAccessMode.READ_WRITE
194
+ });
195
+ } catch (error) {
196
+ throw new Error(`Cannot open DuckDB database "${path}"`, { cause: error });
197
+ }
198
+ const conn = await db.connect();
199
+ return {
200
+ worker,
201
+ db,
202
+ conn
203
+ };
204
+ } catch (error) {
205
+ try {
206
+ await db?.terminate();
207
+ } catch {}
208
+ worker?.terminate();
209
+ throw error;
210
+ } finally {
211
+ worker?.removeEventListener("error", onWorkerError);
212
+ if (blobUrl) env.revokeObjectURL(blobUrl);
213
+ }
214
+ }
215
+ //#endregion
216
+ //#region src/browser-duckdb.ts
217
+ /**
15
218
  * Create a DuckDB-backed {@link Db} using WebAssembly in the browser.
16
219
  *
17
220
  * With `options.path` set to an `opfs://` URL **and** self-hosted
18
221
  * `options.bundles`, the database persists across reloads on OPFS. Without
19
222
  * `options.path` it runs in-memory. If `options.path` cannot be opened, the
20
223
  * returned promise rejects; it never falls back to an in-memory database.
224
+ * `options.path` without `options.bundles` rejects before any worker starts.
21
225
  *
22
226
  * @param options.path `opfs://` path for persistent storage. Omit for in-memory.
23
227
  * @param options.bundles Same-origin bundle URLs (required for OPFS).
24
228
  */
25
229
  async function newBrowserDuckDb(options) {
26
230
  const path = options?.path;
27
- if (path && (typeof navigator === "undefined" || !navigator.storage)) throw new Error(`Cannot open DuckDB database "${path}": OPFS is unavailable`);
28
231
  const selfHosted = options?.bundles;
29
- const bundle = await duckdb.selectBundle(selfHosted ?? duckdb.getJsDelivrBundles());
30
- let worker;
31
- let blobUrl;
32
- if (selfHosted) worker = new Worker(bundle.mainWorker);
33
- else {
34
- blobUrl = URL.createObjectURL(new Blob([`importScripts("${bundle.mainWorker ?? ""}");`], { type: "text/javascript" }));
35
- worker = new Worker(blobUrl);
36
- }
37
- const logger = new duckdb.ConsoleLogger();
38
- const db = new duckdb.AsyncDuckDB(logger, worker);
39
- await db.instantiate(bundle.mainModule, bundle.pthreadWorker);
40
- if (blobUrl) URL.revokeObjectURL(blobUrl);
41
- if (path) try {
42
- await db.open({
43
- path,
44
- accessMode: duckdb.DuckDBAccessMode.READ_WRITE
45
- });
46
- } catch (error) {
47
- await db.terminate();
48
- worker.terminate();
49
- throw new Error(`Cannot open DuckDB database "${path}"`, { cause: error });
50
- }
51
- const conn = await db.connect();
232
+ if (path && !selfHosted) throw new Error(`Cannot open DuckDB database "${path}": OPFS needs self-hosted \`bundles\` (a same-origin worker)`);
233
+ if (path && (typeof navigator === "undefined" || !navigator.storage)) throw new Error(`Cannot open DuckDB database "${path}": OPFS is unavailable`);
234
+ const { worker, db, conn } = await startDuckDb(await duckdb.selectBundle(selfHosted ?? duckdb.getJsDelivrBundles()), {
235
+ selfHosted: selfHosted !== void 0,
236
+ path
237
+ }, browserStartEnv);
52
238
  return {
53
239
  async query(sql, params) {
54
240
  if (params && params.length > 0) {
241
+ const bound = toDuckDbParams(params);
55
242
  const stmt = await conn.prepare(sql);
56
243
  try {
57
- return arrowToObjects(await stmt.query(...params));
244
+ return arrowTableToObjects(await stmt.query(...bound));
58
245
  } finally {
59
246
  await stmt.close();
60
247
  }
61
248
  }
62
- return arrowToObjects(await conn.query(sql));
249
+ return arrowTableToObjects(await conn.query(sql));
63
250
  },
64
251
  async exec(sql) {
65
252
  await conn.query(sql);
@@ -1 +1 @@
1
- {"version":3,"file":"index.mjs","names":[],"sources":["../src/browser-duckdb.ts"],"sourcesContent":["import * as duckdb from \"@duckdb/duckdb-wasm\";\nimport type { Db, DbEntry, DbOptions } from \"@statewalker/db-api\";\n\ntype Table = Awaited<ReturnType<duckdb.AsyncPreparedStatement[\"query\"]>>;\n\n/**\n * Convert an Arrow Table (returned by duckdb-wasm queries) to plain JS objects.\n */\nfunction arrowToObjects<T>(table: Table): T[] {\n const fields = table.schema.fields.map((f: { name: string }) => f.name);\n return table.toArray().map((row: Record<string, unknown>) => {\n const obj: Record<string, unknown> = {};\n for (const field of fields) {\n obj[field] = row[field];\n }\n return obj as T;\n });\n}\n\n/** Browser-specific options for {@link newBrowserDuckDb}. */\nexport interface BrowserDuckDbOptions extends DbOptions {\n /**\n * Self-hosted DuckDB bundle URLs (served same-origin, e.g. via a bundler's\n * `?url` import). Passing these makes the factory spawn a **same-origin**\n * worker, which OPFS database persistence requires — a cross-origin worker\n * (the jsDelivr fallback, loaded via a Blob `importScripts` shim) crashes on\n * OPFS file I/O. Omit to load from jsDelivr (in-memory use only).\n */\n bundles?: duckdb.DuckDBBundles;\n}\n\n/**\n * Create a DuckDB-backed {@link Db} using WebAssembly in the browser.\n *\n * With `options.path` set to an `opfs://` URL **and** self-hosted\n * `options.bundles`, the database persists across reloads on OPFS. Without\n * `options.path` it runs in-memory. If `options.path` cannot be opened, the\n * returned promise rejects; it never falls back to an in-memory database.\n *\n * @param options.path `opfs://` path for persistent storage. Omit for in-memory.\n * @param options.bundles Same-origin bundle URLs (required for OPFS).\n */\nexport async function newBrowserDuckDb(options?: BrowserDuckDbOptions): Promise<Db> {\n const path = options?.path;\n if (path && (typeof navigator === \"undefined\" || !navigator.storage)) {\n throw new Error(`Cannot open DuckDB database \"${path}\": OPFS is unavailable`);\n }\n const selfHosted = options?.bundles;\n const bundle = await duckdb.selectBundle(selfHosted ?? duckdb.getJsDelivrBundles());\n\n // OPFS file I/O needs a same-origin worker. Self-hosted bundles give a\n // same-origin URL we load directly; the jsDelivr fallback is cross-origin and\n // must be wrapped in a Blob that `importScripts` it.\n let worker: Worker;\n let blobUrl: string | undefined;\n if (selfHosted) {\n worker = new Worker(bundle.mainWorker as string);\n } else {\n blobUrl = URL.createObjectURL(\n new Blob([`importScripts(\"${bundle.mainWorker ?? \"\"}\");`], {\n type: \"text/javascript\",\n }),\n );\n worker = new Worker(blobUrl);\n }\n\n const logger = new duckdb.ConsoleLogger();\n const db = new duckdb.AsyncDuckDB(logger, worker);\n await db.instantiate(bundle.mainModule, bundle.pthreadWorker);\n if (blobUrl) URL.revokeObjectURL(blobUrl);\n\n if (path) {\n try {\n await db.open({\n path,\n accessMode: duckdb.DuckDBAccessMode.READ_WRITE,\n });\n } catch (error) {\n await db.terminate();\n worker.terminate();\n throw new Error(`Cannot open DuckDB database \"${path}\"`, { cause: error });\n }\n }\n\n const conn = await db.connect();\n\n return {\n async query<T = DbEntry>(sql: string, params?: unknown[]): Promise<T[]> {\n if (params && params.length > 0) {\n const stmt = await conn.prepare(sql);\n try {\n const result = await stmt.query(...params);\n return arrowToObjects<T>(result);\n } finally {\n await stmt.close();\n }\n }\n const result = await conn.query(sql);\n return arrowToObjects<T>(result);\n },\n\n async exec(sql: string): Promise<void> {\n await conn.query(sql);\n },\n\n async flush(): Promise<void> {\n // CHECKPOINT writes the WAL into the OPFS database file so committed\n // changes survive a reload. Crucial for OPFS persistence.\n if (path) await conn.query(\"CHECKPOINT\");\n },\n\n async close(): Promise<void> {\n await conn.close();\n await db.terminate();\n worker.terminate();\n },\n };\n}\n"],"mappings":";;;;;AAQA,SAAS,eAAkB,OAAmB;CAC5C,MAAM,SAAS,MAAM,OAAO,OAAO,KAAK,MAAwB,EAAE,IAAI;CACtE,OAAO,MAAM,QAAQ,CAAC,CAAC,KAAK,QAAiC;EAC3D,MAAM,MAA+B,CAAC;EACtC,KAAK,MAAM,SAAS,QAClB,IAAI,SAAS,IAAI;EAEnB,OAAO;CACT,CAAC;AACH;;;;;;;;;;;;AAyBA,eAAsB,iBAAiB,SAA6C;CAClF,MAAM,OAAO,SAAS;CACtB,IAAI,SAAS,OAAO,cAAc,eAAe,CAAC,UAAU,UAC1D,MAAM,IAAI,MAAM,gCAAgC,KAAK,uBAAuB;CAE9E,MAAM,aAAa,SAAS;CAC5B,MAAM,SAAS,MAAM,OAAO,aAAa,cAAc,OAAO,mBAAmB,CAAC;CAKlF,IAAI;CACJ,IAAI;CACJ,IAAI,YACF,SAAS,IAAI,OAAO,OAAO,UAAoB;MAC1C;EACL,UAAU,IAAI,gBACZ,IAAI,KAAK,CAAC,kBAAkB,OAAO,cAAc,GAAG,IAAI,GAAG,EACzD,MAAM,kBACR,CAAC,CACH;EACA,SAAS,IAAI,OAAO,OAAO;CAC7B;CAEA,MAAM,SAAS,IAAI,OAAO,cAAc;CACxC,MAAM,KAAK,IAAI,OAAO,YAAY,QAAQ,MAAM;CAChD,MAAM,GAAG,YAAY,OAAO,YAAY,OAAO,aAAa;CAC5D,IAAI,SAAS,IAAI,gBAAgB,OAAO;CAExC,IAAI,MACF,IAAI;EACF,MAAM,GAAG,KAAK;GACZ;GACA,YAAY,OAAO,iBAAiB;EACtC,CAAC;CACH,SAAS,OAAO;EACd,MAAM,GAAG,UAAU;EACnB,OAAO,UAAU;EACjB,MAAM,IAAI,MAAM,gCAAgC,KAAK,IAAI,EAAE,OAAO,MAAM,CAAC;CAC3E;CAGF,MAAM,OAAO,MAAM,GAAG,QAAQ;CAE9B,OAAO;EACL,MAAM,MAAmB,KAAa,QAAkC;GACtE,IAAI,UAAU,OAAO,SAAS,GAAG;IAC/B,MAAM,OAAO,MAAM,KAAK,QAAQ,GAAG;IACnC,IAAI;KAEF,OAAO,eAAkB,MADJ,KAAK,MAAM,GAAG,MAAM,CACV;IACjC,UAAU;KACR,MAAM,KAAK,MAAM;IACnB;GACF;GAEA,OAAO,eAAkB,MADJ,KAAK,MAAM,GAAG,CACJ;EACjC;EAEA,MAAM,KAAK,KAA4B;GACrC,MAAM,KAAK,MAAM,GAAG;EACtB;EAEA,MAAM,QAAuB;GAG3B,IAAI,MAAM,MAAM,KAAK,MAAM,YAAY;EACzC;EAEA,MAAM,QAAuB;GAC3B,MAAM,KAAK,MAAM;GACjB,MAAM,GAAG,UAAU;GACnB,OAAO,UAAU;EACnB;CACF;AACF"}
1
+ {"version":3,"file":"index.mjs","names":[],"sources":["../src/arrow-values.ts","../src/params.ts","../src/start-duckdb.ts","../src/browser-duckdb.ts"],"sourcesContent":["/**\n * Conversion of Arrow values (as duckdb-wasm returns them) into the plain JS\n * values `@statewalker/db-duckdb-node` returns for the same SQL types.\n *\n * Typed structurally so this module does not import `apache-arrow` at runtime.\n * Type ids are Arrow's `Type` enum (apache-arrow 17, the version duckdb-wasm\n * depends on).\n */\n\nconst TypeId = {\n Dictionary: -1,\n Decimal: 7,\n Date: 8,\n Timestamp: 10,\n List: 12,\n Struct: 13,\n FixedSizeList: 16,\n Map: 17,\n} as const;\n\n/** Arrow `TimeUnit`. */\nconst TimeUnit = { SECOND: 0, MILLISECOND: 1, MICROSECOND: 2, NANOSECOND: 3 } as const;\n\n/** The parts of an Arrow `Field` this module reads. */\nexport interface ArrowFieldLike {\n name: string;\n type: ArrowTypeLike;\n}\n\n/** The parts of an Arrow `DataType` this module reads. */\nexport interface ArrowTypeLike {\n typeId: number;\n scale?: number;\n precision?: number;\n unit?: number;\n children?: ArrowFieldLike[] | null;\n dictionary?: ArrowTypeLike;\n}\n\n/** The parts of an Arrow `Table` this module reads. */\nexport interface ArrowTableLike {\n schema: { fields: ArrowFieldLike[] };\n toArray(): ArrayLike<unknown>;\n}\n\n/**\n * Convert an Arrow table into plain row objects, converting each value by its\n * field's Arrow type ({@link arrowValueToJs}).\n */\nexport function arrowTableToObjects<T>(table: ArrowTableLike): T[] {\n const fields = table.schema.fields;\n return Array.from(table.toArray(), (row) => {\n const source = row as Record<string, unknown>;\n const obj: Record<string, unknown> = {};\n for (const field of fields) {\n obj[field.name] = arrowValueToJs(source[field.name], field.type);\n }\n return obj as T;\n });\n}\n\n/**\n * Convert one Arrow value into the plain JS value the node driver returns:\n *\n * - DECIMAL: a `number` scaled by the type's scale (HUGEINT, which DuckDB sends\n * as DECIMAL(38,0), becomes a `bigint`);\n * - DATE, TIMESTAMP: a `Date`;\n * - LIST, fixed-size ARRAY: an array, converted recursively;\n * - STRUCT: a plain object, converted recursively;\n * - MAP: an array of `{ key, value }`, converted recursively.\n *\n * Other values (numbers, `bigint` for BIGINT, strings, booleans, `Uint8Array`\n * for BLOB, `null`) are returned as they are.\n */\nexport function arrowValueToJs(value: unknown, type: ArrowTypeLike): unknown {\n if (value === null || value === undefined) return null;\n switch (type.typeId) {\n case TypeId.Decimal:\n return decimalToJs(value, type);\n case TypeId.Date:\n return typeof value === \"number\" ? new Date(value) : value;\n case TypeId.Timestamp:\n return timestampToDate(value, type.unit);\n case TypeId.List:\n case TypeId.FixedSizeList: {\n const childType = type.children?.[0]?.type;\n const items = Array.from(value as Iterable<unknown>);\n return childType ? items.map((item) => arrowValueToJs(item, childType)) : items;\n }\n case TypeId.Struct: {\n const row = value as Record<string, unknown>;\n const obj: Record<string, unknown> = {};\n for (const child of type.children ?? []) {\n obj[child.name] = arrowValueToJs(row[child.name], child.type);\n }\n return obj;\n }\n case TypeId.Map: {\n const entryFields = type.children?.[0]?.type.children ?? [];\n const keyType = entryFields[0]?.type;\n const valueType = entryFields[1]?.type;\n return Array.from(value as Iterable<[unknown, unknown]>, ([key, val]) => ({\n key: keyType ? arrowValueToJs(key, keyType) : key,\n value: valueType ? arrowValueToJs(val, valueType) : val,\n }));\n }\n case TypeId.Dictionary:\n return type.dictionary ? arrowValueToJs(value, type.dictionary) : value;\n default:\n return value;\n }\n}\n\n/**\n * Arrow decimals are unscaled integers (`String(v)` gives the digits). Insert\n * the decimal point by text and parse, so the result is the nearest double to\n * the exact decimal value.\n */\nfunction decimalToJs(value: unknown, type: ArrowTypeLike): number | bigint {\n const digits = String(value);\n const scale = type.scale ?? 0;\n if (scale === 0) {\n // DuckDB sends HUGEINT as DECIMAL(38,0); the node driver returns bigint.\n return type.precision === 38 ? BigInt(digits) : Number(digits);\n }\n const negative = digits.startsWith(\"-\");\n const abs = (negative ? digits.slice(1) : digits).padStart(scale + 1, \"0\");\n const text = `${abs.slice(0, abs.length - scale)}.${abs.slice(abs.length - scale)}`;\n return Number(negative ? `-${text}` : text);\n}\n\nfunction timestampToDate(value: unknown, unit: number | undefined): unknown {\n if (typeof value === \"number\") return new Date(value); // apache-arrow gives epoch ms\n if (typeof value === \"bigint\") {\n switch (unit) {\n case TimeUnit.SECOND:\n return new Date(Number(value) * 1000);\n case TimeUnit.MICROSECOND:\n return new Date(Number(value / 1000n));\n case TimeUnit.NANOSECOND:\n return new Date(Number(value / 1000000n));\n default:\n return new Date(Number(value));\n }\n }\n return value;\n}\n","/** A value duckdb-wasm's prepared statements can bind (they JSON-encode params). */\nexport type DuckDbWasmParam = string | number | boolean | null;\n\n/**\n * Convert `Db.query` params into values duckdb-wasm can bind.\n *\n * duckdb-wasm passes prepared-statement params through `JSON.stringify`, so\n * only strings, finite numbers, booleans and `null` survive. This maps:\n *\n * - `bigint` to its exact decimal string; DuckDB casts it to the column type\n * where the context fixes one (`WHERE id = $1`, `INSERT ... VALUES ($1)`),\n * elsewhere cast it in SQL (`$1::BIGINT`);\n * - `Date` to its ISO 8601 string, which DuckDB casts to TIMESTAMP the same way;\n * - `undefined` to `null`.\n *\n * Anything else (`Uint8Array` and other binary data, arrays, plain objects,\n * `NaN`, `Infinity`, symbols, functions, invalid dates) throws a `TypeError`\n * naming the parameter and its type instead of failing inside duckdb-wasm or\n * binding the wrong value.\n */\nexport function toDuckDbParams(params: readonly unknown[]): DuckDbWasmParam[] {\n return params.map(toDuckDbParam);\n}\n\nfunction toDuckDbParam(value: unknown, index: number): DuckDbWasmParam {\n switch (typeof value) {\n case \"string\":\n case \"boolean\":\n return value;\n case \"undefined\":\n return null;\n case \"bigint\":\n return value.toString();\n case \"number\":\n if (Number.isFinite(value)) return value;\n throw unsupported(index, String(value));\n case \"object\": {\n if (value === null) return null;\n if (value instanceof Date) {\n if (Number.isNaN(value.getTime())) throw unsupported(index, \"invalid Date\");\n return value.toISOString();\n }\n if (Array.isArray(value)) throw unsupported(index, \"array\");\n if (ArrayBuffer.isView(value) || value instanceof ArrayBuffer) {\n throw unsupported(index, value.constructor.name);\n }\n throw unsupported(index, `object (${value.constructor?.name ?? \"null prototype\"})`);\n }\n default:\n throw unsupported(index, typeof value);\n }\n}\n\nfunction unsupported(index: number, type: string): TypeError {\n return new TypeError(\n `db-duckdb-browser: cannot bind parameter $${index + 1} of type ${type}; ` +\n \"duckdb-wasm binds only string, finite number, boolean, null, bigint and Date\",\n );\n}\n","import * as duckdb from \"@duckdb/duckdb-wasm\";\n\n/**\n * What {@link startDuckDb} needs from the page. The browser defaults are\n * {@link browserStartEnv}; tests pass fakes to drive failures.\n */\nexport interface DuckDbStartEnv {\n newWorker(url: string): Worker;\n newDb(worker: Worker): duckdb.AsyncDuckDB;\n createObjectURL(blob: Blob): string;\n revokeObjectURL(url: string): void;\n}\n\n/** A started engine: its worker, the engine and one open connection. */\nexport interface StartedDuckDb {\n worker: Worker;\n db: duckdb.AsyncDuckDB;\n conn: duckdb.AsyncDuckDBConnection;\n}\n\n/** The real browser environment: a Web Worker and the page's `URL`. */\nexport const browserStartEnv: DuckDbStartEnv = {\n newWorker: (url) => new Worker(url),\n newDb: (worker) => new duckdb.AsyncDuckDB(new duckdb.ConsoleLogger(), worker),\n createObjectURL: (blob) => URL.createObjectURL(blob),\n revokeObjectURL: (url) => URL.revokeObjectURL(url),\n};\n\n/**\n * Start a worker, instantiate DuckDB in it, open `path` if given, and connect.\n *\n * If any step fails, including the worker script failing to load, the engine\n * and its worker are terminated before the error is rethrown, so a failed start leaves no worker running. The Blob URL\n * of the jsDelivr shim is revoked whether the start succeeds or fails.\n */\nexport async function startDuckDb(\n bundle: duckdb.DuckDBBundle,\n options: { selfHosted: boolean; path?: string },\n env: DuckDbStartEnv,\n): Promise<StartedDuckDb> {\n const { selfHosted, path } = options;\n let blobUrl: string | undefined;\n let worker: Worker | undefined;\n let db: duckdb.AsyncDuckDB | undefined;\n // duckdb-wasm only logs a worker error (a script that fails to load) and drops the pending\n // request, so `instantiate` would never settle: turn the error event into a rejection.\n let onWorkerError: ((event: ErrorEvent) => void) | undefined;\n const workerFailed = new Promise<never>((_, reject) => {\n onWorkerError = (event) =>\n reject(new Error(`DuckDB worker failed to load: ${event.message ?? \"unknown error\"}`));\n });\n workerFailed.catch(() => {}); // only observed through the race below\n try {\n // OPFS file I/O needs a same-origin worker. Self-hosted bundles give a\n // same-origin URL we load directly; the jsDelivr fallback is cross-origin\n // and must be wrapped in a Blob that `importScripts` it.\n if (selfHosted) {\n worker = env.newWorker(bundle.mainWorker as string);\n } else {\n blobUrl = env.createObjectURL(\n new Blob([`importScripts(\"${bundle.mainWorker ?? \"\"}\");`], {\n type: \"text/javascript\",\n }),\n );\n worker = env.newWorker(blobUrl);\n }\n worker.addEventListener(\"error\", onWorkerError as (event: ErrorEvent) => void);\n db = env.newDb(worker);\n await Promise.race([db.instantiate(bundle.mainModule, bundle.pthreadWorker), workerFailed]);\n if (path) {\n try {\n await db.open({ path, accessMode: duckdb.DuckDBAccessMode.READ_WRITE });\n } catch (error) {\n throw new Error(`Cannot open DuckDB database \"${path}\"`, { cause: error });\n }\n }\n const conn = await db.connect();\n return { worker, db, conn };\n } catch (error) {\n // Best effort: a failing teardown must not hide the start error.\n try {\n await db?.terminate();\n } catch {}\n worker?.terminate();\n throw error;\n } finally {\n worker?.removeEventListener(\"error\", onWorkerError as (event: ErrorEvent) => void);\n if (blobUrl) env.revokeObjectURL(blobUrl);\n }\n}\n","import * as duckdb from \"@duckdb/duckdb-wasm\";\nimport type { Db, DbEntry, DbOptions } from \"@statewalker/db-api\";\nimport { arrowTableToObjects } from \"./arrow-values.js\";\nimport { toDuckDbParams } from \"./params.js\";\nimport { browserStartEnv, startDuckDb } from \"./start-duckdb.js\";\n\n/** Browser-specific options for {@link newBrowserDuckDb}. */\nexport interface BrowserDuckDbOptions extends DbOptions {\n /**\n * Self-hosted DuckDB bundle URLs (served same-origin, e.g. via a bundler's\n * `?url` import). Passing these makes the factory spawn a **same-origin**\n * worker, which OPFS database persistence requires — a cross-origin worker\n * (the jsDelivr fallback, loaded via a Blob `importScripts` shim) crashes on\n * OPFS file I/O. Omit to load from jsDelivr (in-memory use only); a `path`\n * without `bundles` is rejected.\n */\n bundles?: duckdb.DuckDBBundles;\n}\n\n/**\n * Create a DuckDB-backed {@link Db} using WebAssembly in the browser.\n *\n * With `options.path` set to an `opfs://` URL **and** self-hosted\n * `options.bundles`, the database persists across reloads on OPFS. Without\n * `options.path` it runs in-memory. If `options.path` cannot be opened, the\n * returned promise rejects; it never falls back to an in-memory database.\n * `options.path` without `options.bundles` rejects before any worker starts.\n *\n * @param options.path `opfs://` path for persistent storage. Omit for in-memory.\n * @param options.bundles Same-origin bundle URLs (required for OPFS).\n */\nexport async function newBrowserDuckDb(options?: BrowserDuckDbOptions): Promise<Db> {\n const path = options?.path;\n const selfHosted = options?.bundles;\n if (path && !selfHosted) {\n // The jsDelivr worker is cross-origin and crashes on OPFS file I/O.\n throw new Error(\n `Cannot open DuckDB database \"${path}\": OPFS needs self-hosted \\`bundles\\` (a same-origin worker)`,\n );\n }\n if (path && (typeof navigator === \"undefined\" || !navigator.storage)) {\n throw new Error(`Cannot open DuckDB database \"${path}\": OPFS is unavailable`);\n }\n const bundle = await duckdb.selectBundle(selfHosted ?? duckdb.getJsDelivrBundles());\n const { worker, db, conn } = await startDuckDb(\n bundle,\n { selfHosted: selfHosted !== undefined, path },\n browserStartEnv,\n );\n\n return {\n async query<T = DbEntry>(sql: string, params?: unknown[]): Promise<T[]> {\n if (params && params.length > 0) {\n // Convert first: an unsupported param throws before a statement exists.\n const bound = toDuckDbParams(params);\n const stmt = await conn.prepare(sql);\n try {\n const result = await stmt.query(...bound);\n return arrowTableToObjects<T>(result);\n } finally {\n await stmt.close();\n }\n }\n const result = await conn.query(sql);\n return arrowTableToObjects<T>(result);\n },\n\n async exec(sql: string): Promise<void> {\n await conn.query(sql);\n },\n\n async flush(): Promise<void> {\n // CHECKPOINT writes the WAL into the OPFS database file so committed\n // changes survive a reload. Crucial for OPFS persistence.\n if (path) await conn.query(\"CHECKPOINT\");\n },\n\n async close(): Promise<void> {\n await conn.close();\n await db.terminate();\n worker.terminate();\n },\n };\n}\n"],"mappings":";;;;;;;;;;AASA,MAAM,SAAS;CACb,YAAY;CACZ,SAAS;CACT,MAAM;CACN,WAAW;CACX,MAAM;CACN,QAAQ;CACR,eAAe;CACf,KAAK;AACP;;AAGA,MAAM,WAAW;CAAE,QAAQ;CAAG,aAAa;CAAG,aAAa;CAAG,YAAY;AAAE;;;;;AA4B5E,SAAgB,oBAAuB,OAA4B;CACjE,MAAM,SAAS,MAAM,OAAO;CAC5B,OAAO,MAAM,KAAK,MAAM,QAAQ,IAAI,QAAQ;EAC1C,MAAM,SAAS;EACf,MAAM,MAA+B,CAAC;EACtC,KAAK,MAAM,SAAS,QAClB,IAAI,MAAM,QAAQ,eAAe,OAAO,MAAM,OAAO,MAAM,IAAI;EAEjE,OAAO;CACT,CAAC;AACH;;;;;;;;;;;;;;AAeA,SAAgB,eAAe,OAAgB,MAA8B;CAC3E,IAAI,UAAU,QAAQ,UAAU,KAAA,GAAW,OAAO;CAClD,QAAQ,KAAK,QAAb;EACE,KAAK,OAAO,SACV,OAAO,YAAY,OAAO,IAAI;EAChC,KAAK,OAAO,MACV,OAAO,OAAO,UAAU,WAAW,IAAI,KAAK,KAAK,IAAI;EACvD,KAAK,OAAO,WACV,OAAO,gBAAgB,OAAO,KAAK,IAAI;EACzC,KAAK,OAAO;EACZ,KAAK,OAAO,eAAe;GACzB,MAAM,YAAY,KAAK,WAAW,EAAE,EAAE;GACtC,MAAM,QAAQ,MAAM,KAAK,KAA0B;GACnD,OAAO,YAAY,MAAM,KAAK,SAAS,eAAe,MAAM,SAAS,CAAC,IAAI;EAC5E;EACA,KAAK,OAAO,QAAQ;GAClB,MAAM,MAAM;GACZ,MAAM,MAA+B,CAAC;GACtC,KAAK,MAAM,SAAS,KAAK,YAAY,CAAC,GACpC,IAAI,MAAM,QAAQ,eAAe,IAAI,MAAM,OAAO,MAAM,IAAI;GAE9D,OAAO;EACT;EACA,KAAK,OAAO,KAAK;GACf,MAAM,cAAc,KAAK,WAAW,EAAE,EAAE,KAAK,YAAY,CAAC;GAC1D,MAAM,UAAU,YAAY,EAAE,EAAE;GAChC,MAAM,YAAY,YAAY,EAAE,EAAE;GAClC,OAAO,MAAM,KAAK,QAAwC,CAAC,KAAK,UAAU;IACxE,KAAK,UAAU,eAAe,KAAK,OAAO,IAAI;IAC9C,OAAO,YAAY,eAAe,KAAK,SAAS,IAAI;GACtD,EAAE;EACJ;EACA,KAAK,OAAO,YACV,OAAO,KAAK,aAAa,eAAe,OAAO,KAAK,UAAU,IAAI;EACpE,SACE,OAAO;CACX;AACF;;;;;;AAOA,SAAS,YAAY,OAAgB,MAAsC;CACzE,MAAM,SAAS,OAAO,KAAK;CAC3B,MAAM,QAAQ,KAAK,SAAS;CAC5B,IAAI,UAAU,GAEZ,OAAO,KAAK,cAAc,KAAK,OAAO,MAAM,IAAI,OAAO,MAAM;CAE/D,MAAM,WAAW,OAAO,WAAW,GAAG;CACtC,MAAM,OAAO,WAAW,OAAO,MAAM,CAAC,IAAI,OAAA,CAAQ,SAAS,QAAQ,GAAG,GAAG;CACzE,MAAM,OAAO,GAAG,IAAI,MAAM,GAAG,IAAI,SAAS,KAAK,EAAE,GAAG,IAAI,MAAM,IAAI,SAAS,KAAK;CAChF,OAAO,OAAO,WAAW,IAAI,SAAS,IAAI;AAC5C;AAEA,SAAS,gBAAgB,OAAgB,MAAmC;CAC1E,IAAI,OAAO,UAAU,UAAU,OAAO,IAAI,KAAK,KAAK;CACpD,IAAI,OAAO,UAAU,UACnB,QAAQ,MAAR;EACE,KAAK,SAAS,QACZ,uBAAO,IAAI,KAAK,OAAO,KAAK,IAAI,GAAI;EACtC,KAAK,SAAS,aACZ,OAAO,IAAI,KAAK,OAAO,QAAQ,KAAK,CAAC;EACvC,KAAK,SAAS,YACZ,OAAO,IAAI,KAAK,OAAO,QAAQ,QAAQ,CAAC;EAC1C,SACE,OAAO,IAAI,KAAK,OAAO,KAAK,CAAC;CACjC;CAEF,OAAO;AACT;;;;;;;;;;;;;;;;;;;;AC9HA,SAAgB,eAAe,QAA+C;CAC5E,OAAO,OAAO,IAAI,aAAa;AACjC;AAEA,SAAS,cAAc,OAAgB,OAAgC;CACrE,QAAQ,OAAO,OAAf;EACE,KAAK;EACL,KAAK,WACH,OAAO;EACT,KAAK,aACH,OAAO;EACT,KAAK,UACH,OAAO,MAAM,SAAS;EACxB,KAAK;GACH,IAAI,OAAO,SAAS,KAAK,GAAG,OAAO;GACnC,MAAM,YAAY,OAAO,OAAO,KAAK,CAAC;EACxC,KAAK;GACH,IAAI,UAAU,MAAM,OAAO;GAC3B,IAAI,iBAAiB,MAAM;IACzB,IAAI,OAAO,MAAM,MAAM,QAAQ,CAAC,GAAG,MAAM,YAAY,OAAO,cAAc;IAC1E,OAAO,MAAM,YAAY;GAC3B;GACA,IAAI,MAAM,QAAQ,KAAK,GAAG,MAAM,YAAY,OAAO,OAAO;GAC1D,IAAI,YAAY,OAAO,KAAK,KAAK,iBAAiB,aAChD,MAAM,YAAY,OAAO,MAAM,YAAY,IAAI;GAEjD,MAAM,YAAY,OAAO,WAAW,MAAM,aAAa,QAAQ,iBAAiB,EAAE;EAEpF,SACE,MAAM,YAAY,OAAO,OAAO,KAAK;CACzC;AACF;AAEA,SAAS,YAAY,OAAe,MAAyB;CAC3D,uBAAO,IAAI,UACT,6CAA6C,QAAQ,EAAE,WAAW,KAAK,+EAEzE;AACF;;;;ACrCA,MAAa,kBAAkC;CAC7C,YAAY,QAAQ,IAAI,OAAO,GAAG;CAClC,QAAQ,WAAW,IAAI,OAAO,YAAY,IAAI,OAAO,cAAc,GAAG,MAAM;CAC5E,kBAAkB,SAAS,IAAI,gBAAgB,IAAI;CACnD,kBAAkB,QAAQ,IAAI,gBAAgB,GAAG;AACnD;;;;;;;;AASA,eAAsB,YACpB,QACA,SACA,KACwB;CACxB,MAAM,EAAE,YAAY,SAAS;CAC7B,IAAI;CACJ,IAAI;CACJ,IAAI;CAGJ,IAAI;CACJ,MAAM,eAAe,IAAI,SAAgB,GAAG,WAAW;EACrD,iBAAiB,UACf,uBAAO,IAAI,MAAM,iCAAiC,MAAM,WAAW,iBAAiB,CAAC;CACzF,CAAC;CACD,aAAa,YAAY,CAAC,CAAC;CAC3B,IAAI;EAIF,IAAI,YACF,SAAS,IAAI,UAAU,OAAO,UAAoB;OAC7C;GACL,UAAU,IAAI,gBACZ,IAAI,KAAK,CAAC,kBAAkB,OAAO,cAAc,GAAG,IAAI,GAAG,EACzD,MAAM,kBACR,CAAC,CACH;GACA,SAAS,IAAI,UAAU,OAAO;EAChC;EACA,OAAO,iBAAiB,SAAS,aAA4C;EAC7E,KAAK,IAAI,MAAM,MAAM;EACrB,MAAM,QAAQ,KAAK,CAAC,GAAG,YAAY,OAAO,YAAY,OAAO,aAAa,GAAG,YAAY,CAAC;EAC1F,IAAI,MACF,IAAI;GACF,MAAM,GAAG,KAAK;IAAE;IAAM,YAAY,OAAO,iBAAiB;GAAW,CAAC;EACxE,SAAS,OAAO;GACd,MAAM,IAAI,MAAM,gCAAgC,KAAK,IAAI,EAAE,OAAO,MAAM,CAAC;EAC3E;EAEF,MAAM,OAAO,MAAM,GAAG,QAAQ;EAC9B,OAAO;GAAE;GAAQ;GAAI;EAAK;CAC5B,SAAS,OAAO;EAEd,IAAI;GACF,MAAM,IAAI,UAAU;EACtB,QAAQ,CAAC;EACT,QAAQ,UAAU;EAClB,MAAM;CACR,UAAU;EACR,QAAQ,oBAAoB,SAAS,aAA4C;EACjF,IAAI,SAAS,IAAI,gBAAgB,OAAO;CAC1C;AACF;;;;;;;;;;;;;;;AC1DA,eAAsB,iBAAiB,SAA6C;CAClF,MAAM,OAAO,SAAS;CACtB,MAAM,aAAa,SAAS;CAC5B,IAAI,QAAQ,CAAC,YAEX,MAAM,IAAI,MACR,gCAAgC,KAAK,6DACvC;CAEF,IAAI,SAAS,OAAO,cAAc,eAAe,CAAC,UAAU,UAC1D,MAAM,IAAI,MAAM,gCAAgC,KAAK,uBAAuB;CAG9E,MAAM,EAAE,QAAQ,IAAI,SAAS,MAAM,YACjC,MAFmB,OAAO,aAAa,cAAc,OAAO,mBAAmB,CAAC,GAGhF;EAAE,YAAY,eAAe,KAAA;EAAW;CAAK,GAC7C,eACF;CAEA,OAAO;EACL,MAAM,MAAmB,KAAa,QAAkC;GACtE,IAAI,UAAU,OAAO,SAAS,GAAG;IAE/B,MAAM,QAAQ,eAAe,MAAM;IACnC,MAAM,OAAO,MAAM,KAAK,QAAQ,GAAG;IACnC,IAAI;KAEF,OAAO,oBAAuB,MADT,KAAK,MAAM,GAAG,KAAK,CACJ;IACtC,UAAU;KACR,MAAM,KAAK,MAAM;IACnB;GACF;GAEA,OAAO,oBAAuB,MADT,KAAK,MAAM,GAAG,CACC;EACtC;EAEA,MAAM,KAAK,KAA4B;GACrC,MAAM,KAAK,MAAM,GAAG;EACtB;EAEA,MAAM,QAAuB;GAG3B,IAAI,MAAM,MAAM,KAAK,MAAM,YAAY;EACzC;EAEA,MAAM,QAAuB;GAC3B,MAAM,KAAK,MAAM;GACjB,MAAM,GAAG,UAAU;GACnB,OAAO,UAAU;EACnB;CACF;AACF"}
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@statewalker/db-duckdb-browser",
3
- "version": "0.1.6",
3
+ "version": "0.1.7",
4
4
  "private": false,
5
5
  "type": "module",
6
6
  "description": "DuckDB WASM driver for @statewalker/db-api, targeting browser environments.",
@@ -22,16 +22,19 @@
22
22
  ".": {
23
23
  "source": "./src/index.ts",
24
24
  "types": "./dist/index.d.mts",
25
- "import": "./dist/index.mjs"
25
+ "import": "./dist/index.mjs",
26
+ "default": "./dist/index.mjs"
26
27
  }
27
28
  },
28
29
  "files": [
29
30
  "dist",
30
- "src"
31
+ "src",
32
+ "!src/**/*.test.ts",
33
+ "!src/**/test-wasm-node.ts"
31
34
  ],
32
35
  "dependencies": {
33
36
  "@duckdb/duckdb-wasm": "^1.33.1-dev57.0",
34
- "@statewalker/db-api": "^0.1.5"
37
+ "@statewalker/db-api": "^0.1.6"
35
38
  },
36
39
  "devDependencies": {
37
40
  "@playwright/test": "^1.63.0",
@@ -0,0 +1,147 @@
1
+ /**
2
+ * Conversion of Arrow values (as duckdb-wasm returns them) into the plain JS
3
+ * values `@statewalker/db-duckdb-node` returns for the same SQL types.
4
+ *
5
+ * Typed structurally so this module does not import `apache-arrow` at runtime.
6
+ * Type ids are Arrow's `Type` enum (apache-arrow 17, the version duckdb-wasm
7
+ * depends on).
8
+ */
9
+
10
+ const TypeId = {
11
+ Dictionary: -1,
12
+ Decimal: 7,
13
+ Date: 8,
14
+ Timestamp: 10,
15
+ List: 12,
16
+ Struct: 13,
17
+ FixedSizeList: 16,
18
+ Map: 17,
19
+ } as const;
20
+
21
+ /** Arrow `TimeUnit`. */
22
+ const TimeUnit = { SECOND: 0, MILLISECOND: 1, MICROSECOND: 2, NANOSECOND: 3 } as const;
23
+
24
+ /** The parts of an Arrow `Field` this module reads. */
25
+ export interface ArrowFieldLike {
26
+ name: string;
27
+ type: ArrowTypeLike;
28
+ }
29
+
30
+ /** The parts of an Arrow `DataType` this module reads. */
31
+ export interface ArrowTypeLike {
32
+ typeId: number;
33
+ scale?: number;
34
+ precision?: number;
35
+ unit?: number;
36
+ children?: ArrowFieldLike[] | null;
37
+ dictionary?: ArrowTypeLike;
38
+ }
39
+
40
+ /** The parts of an Arrow `Table` this module reads. */
41
+ export interface ArrowTableLike {
42
+ schema: { fields: ArrowFieldLike[] };
43
+ toArray(): ArrayLike<unknown>;
44
+ }
45
+
46
+ /**
47
+ * Convert an Arrow table into plain row objects, converting each value by its
48
+ * field's Arrow type ({@link arrowValueToJs}).
49
+ */
50
+ export function arrowTableToObjects<T>(table: ArrowTableLike): T[] {
51
+ const fields = table.schema.fields;
52
+ return Array.from(table.toArray(), (row) => {
53
+ const source = row as Record<string, unknown>;
54
+ const obj: Record<string, unknown> = {};
55
+ for (const field of fields) {
56
+ obj[field.name] = arrowValueToJs(source[field.name], field.type);
57
+ }
58
+ return obj as T;
59
+ });
60
+ }
61
+
62
+ /**
63
+ * Convert one Arrow value into the plain JS value the node driver returns:
64
+ *
65
+ * - DECIMAL: a `number` scaled by the type's scale (HUGEINT, which DuckDB sends
66
+ * as DECIMAL(38,0), becomes a `bigint`);
67
+ * - DATE, TIMESTAMP: a `Date`;
68
+ * - LIST, fixed-size ARRAY: an array, converted recursively;
69
+ * - STRUCT: a plain object, converted recursively;
70
+ * - MAP: an array of `{ key, value }`, converted recursively.
71
+ *
72
+ * Other values (numbers, `bigint` for BIGINT, strings, booleans, `Uint8Array`
73
+ * for BLOB, `null`) are returned as they are.
74
+ */
75
+ export function arrowValueToJs(value: unknown, type: ArrowTypeLike): unknown {
76
+ if (value === null || value === undefined) return null;
77
+ switch (type.typeId) {
78
+ case TypeId.Decimal:
79
+ return decimalToJs(value, type);
80
+ case TypeId.Date:
81
+ return typeof value === "number" ? new Date(value) : value;
82
+ case TypeId.Timestamp:
83
+ return timestampToDate(value, type.unit);
84
+ case TypeId.List:
85
+ case TypeId.FixedSizeList: {
86
+ const childType = type.children?.[0]?.type;
87
+ const items = Array.from(value as Iterable<unknown>);
88
+ return childType ? items.map((item) => arrowValueToJs(item, childType)) : items;
89
+ }
90
+ case TypeId.Struct: {
91
+ const row = value as Record<string, unknown>;
92
+ const obj: Record<string, unknown> = {};
93
+ for (const child of type.children ?? []) {
94
+ obj[child.name] = arrowValueToJs(row[child.name], child.type);
95
+ }
96
+ return obj;
97
+ }
98
+ case TypeId.Map: {
99
+ const entryFields = type.children?.[0]?.type.children ?? [];
100
+ const keyType = entryFields[0]?.type;
101
+ const valueType = entryFields[1]?.type;
102
+ return Array.from(value as Iterable<[unknown, unknown]>, ([key, val]) => ({
103
+ key: keyType ? arrowValueToJs(key, keyType) : key,
104
+ value: valueType ? arrowValueToJs(val, valueType) : val,
105
+ }));
106
+ }
107
+ case TypeId.Dictionary:
108
+ return type.dictionary ? arrowValueToJs(value, type.dictionary) : value;
109
+ default:
110
+ return value;
111
+ }
112
+ }
113
+
114
+ /**
115
+ * Arrow decimals are unscaled integers (`String(v)` gives the digits). Insert
116
+ * the decimal point by text and parse, so the result is the nearest double to
117
+ * the exact decimal value.
118
+ */
119
+ function decimalToJs(value: unknown, type: ArrowTypeLike): number | bigint {
120
+ const digits = String(value);
121
+ const scale = type.scale ?? 0;
122
+ if (scale === 0) {
123
+ // DuckDB sends HUGEINT as DECIMAL(38,0); the node driver returns bigint.
124
+ return type.precision === 38 ? BigInt(digits) : Number(digits);
125
+ }
126
+ const negative = digits.startsWith("-");
127
+ const abs = (negative ? digits.slice(1) : digits).padStart(scale + 1, "0");
128
+ const text = `${abs.slice(0, abs.length - scale)}.${abs.slice(abs.length - scale)}`;
129
+ return Number(negative ? `-${text}` : text);
130
+ }
131
+
132
+ function timestampToDate(value: unknown, unit: number | undefined): unknown {
133
+ if (typeof value === "number") return new Date(value); // apache-arrow gives epoch ms
134
+ if (typeof value === "bigint") {
135
+ switch (unit) {
136
+ case TimeUnit.SECOND:
137
+ return new Date(Number(value) * 1000);
138
+ case TimeUnit.MICROSECOND:
139
+ return new Date(Number(value / 1000n));
140
+ case TimeUnit.NANOSECOND:
141
+ return new Date(Number(value / 1000000n));
142
+ default:
143
+ return new Date(Number(value));
144
+ }
145
+ }
146
+ return value;
147
+ }
@@ -1,21 +1,8 @@
1
1
  import * as duckdb from "@duckdb/duckdb-wasm";
2
2
  import type { Db, DbEntry, DbOptions } from "@statewalker/db-api";
3
-
4
- type Table = Awaited<ReturnType<duckdb.AsyncPreparedStatement["query"]>>;
5
-
6
- /**
7
- * Convert an Arrow Table (returned by duckdb-wasm queries) to plain JS objects.
8
- */
9
- function arrowToObjects<T>(table: Table): T[] {
10
- const fields = table.schema.fields.map((f: { name: string }) => f.name);
11
- return table.toArray().map((row: Record<string, unknown>) => {
12
- const obj: Record<string, unknown> = {};
13
- for (const field of fields) {
14
- obj[field] = row[field];
15
- }
16
- return obj as T;
17
- });
18
- }
3
+ import { arrowTableToObjects } from "./arrow-values.js";
4
+ import { toDuckDbParams } from "./params.js";
5
+ import { browserStartEnv, startDuckDb } from "./start-duckdb.js";
19
6
 
20
7
  /** Browser-specific options for {@link newBrowserDuckDb}. */
21
8
  export interface BrowserDuckDbOptions extends DbOptions {
@@ -24,7 +11,8 @@ export interface BrowserDuckDbOptions extends DbOptions {
24
11
  * `?url` import). Passing these makes the factory spawn a **same-origin**
25
12
  * worker, which OPFS database persistence requires — a cross-origin worker
26
13
  * (the jsDelivr fallback, loaded via a Blob `importScripts` shim) crashes on
27
- * OPFS file I/O. Omit to load from jsDelivr (in-memory use only).
14
+ * OPFS file I/O. Omit to load from jsDelivr (in-memory use only); a `path`
15
+ * without `bundles` is rejected.
28
16
  */
29
17
  bundles?: duckdb.DuckDBBundles;
30
18
  }
@@ -36,67 +24,45 @@ export interface BrowserDuckDbOptions extends DbOptions {
36
24
  * `options.bundles`, the database persists across reloads on OPFS. Without
37
25
  * `options.path` it runs in-memory. If `options.path` cannot be opened, the
38
26
  * returned promise rejects; it never falls back to an in-memory database.
27
+ * `options.path` without `options.bundles` rejects before any worker starts.
39
28
  *
40
29
  * @param options.path `opfs://` path for persistent storage. Omit for in-memory.
41
30
  * @param options.bundles Same-origin bundle URLs (required for OPFS).
42
31
  */
43
32
  export async function newBrowserDuckDb(options?: BrowserDuckDbOptions): Promise<Db> {
44
33
  const path = options?.path;
45
- if (path && (typeof navigator === "undefined" || !navigator.storage)) {
46
- throw new Error(`Cannot open DuckDB database "${path}": OPFS is unavailable`);
47
- }
48
34
  const selfHosted = options?.bundles;
49
- const bundle = await duckdb.selectBundle(selfHosted ?? duckdb.getJsDelivrBundles());
50
-
51
- // OPFS file I/O needs a same-origin worker. Self-hosted bundles give a
52
- // same-origin URL we load directly; the jsDelivr fallback is cross-origin and
53
- // must be wrapped in a Blob that `importScripts` it.
54
- let worker: Worker;
55
- let blobUrl: string | undefined;
56
- if (selfHosted) {
57
- worker = new Worker(bundle.mainWorker as string);
58
- } else {
59
- blobUrl = URL.createObjectURL(
60
- new Blob([`importScripts("${bundle.mainWorker ?? ""}");`], {
61
- type: "text/javascript",
62
- }),
35
+ if (path && !selfHosted) {
36
+ // The jsDelivr worker is cross-origin and crashes on OPFS file I/O.
37
+ throw new Error(
38
+ `Cannot open DuckDB database "${path}": OPFS needs self-hosted \`bundles\` (a same-origin worker)`,
63
39
  );
64
- worker = new Worker(blobUrl);
65
40
  }
66
-
67
- const logger = new duckdb.ConsoleLogger();
68
- const db = new duckdb.AsyncDuckDB(logger, worker);
69
- await db.instantiate(bundle.mainModule, bundle.pthreadWorker);
70
- if (blobUrl) URL.revokeObjectURL(blobUrl);
71
-
72
- if (path) {
73
- try {
74
- await db.open({
75
- path,
76
- accessMode: duckdb.DuckDBAccessMode.READ_WRITE,
77
- });
78
- } catch (error) {
79
- await db.terminate();
80
- worker.terminate();
81
- throw new Error(`Cannot open DuckDB database "${path}"`, { cause: error });
82
- }
41
+ if (path && (typeof navigator === "undefined" || !navigator.storage)) {
42
+ throw new Error(`Cannot open DuckDB database "${path}": OPFS is unavailable`);
83
43
  }
84
-
85
- const conn = await db.connect();
44
+ const bundle = await duckdb.selectBundle(selfHosted ?? duckdb.getJsDelivrBundles());
45
+ const { worker, db, conn } = await startDuckDb(
46
+ bundle,
47
+ { selfHosted: selfHosted !== undefined, path },
48
+ browserStartEnv,
49
+ );
86
50
 
87
51
  return {
88
52
  async query<T = DbEntry>(sql: string, params?: unknown[]): Promise<T[]> {
89
53
  if (params && params.length > 0) {
54
+ // Convert first: an unsupported param throws before a statement exists.
55
+ const bound = toDuckDbParams(params);
90
56
  const stmt = await conn.prepare(sql);
91
57
  try {
92
- const result = await stmt.query(...params);
93
- return arrowToObjects<T>(result);
58
+ const result = await stmt.query(...bound);
59
+ return arrowTableToObjects<T>(result);
94
60
  } finally {
95
61
  await stmt.close();
96
62
  }
97
63
  }
98
64
  const result = await conn.query(sql);
99
- return arrowToObjects<T>(result);
65
+ return arrowTableToObjects<T>(result);
100
66
  },
101
67
 
102
68
  async exec(sql: string): Promise<void> {
package/src/params.ts ADDED
@@ -0,0 +1,59 @@
1
+ /** A value duckdb-wasm's prepared statements can bind (they JSON-encode params). */
2
+ export type DuckDbWasmParam = string | number | boolean | null;
3
+
4
+ /**
5
+ * Convert `Db.query` params into values duckdb-wasm can bind.
6
+ *
7
+ * duckdb-wasm passes prepared-statement params through `JSON.stringify`, so
8
+ * only strings, finite numbers, booleans and `null` survive. This maps:
9
+ *
10
+ * - `bigint` to its exact decimal string; DuckDB casts it to the column type
11
+ * where the context fixes one (`WHERE id = $1`, `INSERT ... VALUES ($1)`),
12
+ * elsewhere cast it in SQL (`$1::BIGINT`);
13
+ * - `Date` to its ISO 8601 string, which DuckDB casts to TIMESTAMP the same way;
14
+ * - `undefined` to `null`.
15
+ *
16
+ * Anything else (`Uint8Array` and other binary data, arrays, plain objects,
17
+ * `NaN`, `Infinity`, symbols, functions, invalid dates) throws a `TypeError`
18
+ * naming the parameter and its type instead of failing inside duckdb-wasm or
19
+ * binding the wrong value.
20
+ */
21
+ export function toDuckDbParams(params: readonly unknown[]): DuckDbWasmParam[] {
22
+ return params.map(toDuckDbParam);
23
+ }
24
+
25
+ function toDuckDbParam(value: unknown, index: number): DuckDbWasmParam {
26
+ switch (typeof value) {
27
+ case "string":
28
+ case "boolean":
29
+ return value;
30
+ case "undefined":
31
+ return null;
32
+ case "bigint":
33
+ return value.toString();
34
+ case "number":
35
+ if (Number.isFinite(value)) return value;
36
+ throw unsupported(index, String(value));
37
+ case "object": {
38
+ if (value === null) return null;
39
+ if (value instanceof Date) {
40
+ if (Number.isNaN(value.getTime())) throw unsupported(index, "invalid Date");
41
+ return value.toISOString();
42
+ }
43
+ if (Array.isArray(value)) throw unsupported(index, "array");
44
+ if (ArrayBuffer.isView(value) || value instanceof ArrayBuffer) {
45
+ throw unsupported(index, value.constructor.name);
46
+ }
47
+ throw unsupported(index, `object (${value.constructor?.name ?? "null prototype"})`);
48
+ }
49
+ default:
50
+ throw unsupported(index, typeof value);
51
+ }
52
+ }
53
+
54
+ function unsupported(index: number, type: string): TypeError {
55
+ return new TypeError(
56
+ `db-duckdb-browser: cannot bind parameter $${index + 1} of type ${type}; ` +
57
+ "duckdb-wasm binds only string, finite number, boolean, null, bigint and Date",
58
+ );
59
+ }
@@ -0,0 +1,90 @@
1
+ import * as duckdb from "@duckdb/duckdb-wasm";
2
+
3
+ /**
4
+ * What {@link startDuckDb} needs from the page. The browser defaults are
5
+ * {@link browserStartEnv}; tests pass fakes to drive failures.
6
+ */
7
+ export interface DuckDbStartEnv {
8
+ newWorker(url: string): Worker;
9
+ newDb(worker: Worker): duckdb.AsyncDuckDB;
10
+ createObjectURL(blob: Blob): string;
11
+ revokeObjectURL(url: string): void;
12
+ }
13
+
14
+ /** A started engine: its worker, the engine and one open connection. */
15
+ export interface StartedDuckDb {
16
+ worker: Worker;
17
+ db: duckdb.AsyncDuckDB;
18
+ conn: duckdb.AsyncDuckDBConnection;
19
+ }
20
+
21
+ /** The real browser environment: a Web Worker and the page's `URL`. */
22
+ export const browserStartEnv: DuckDbStartEnv = {
23
+ newWorker: (url) => new Worker(url),
24
+ newDb: (worker) => new duckdb.AsyncDuckDB(new duckdb.ConsoleLogger(), worker),
25
+ createObjectURL: (blob) => URL.createObjectURL(blob),
26
+ revokeObjectURL: (url) => URL.revokeObjectURL(url),
27
+ };
28
+
29
+ /**
30
+ * Start a worker, instantiate DuckDB in it, open `path` if given, and connect.
31
+ *
32
+ * If any step fails, including the worker script failing to load, the engine
33
+ * and its worker are terminated before the error is rethrown, so a failed start leaves no worker running. The Blob URL
34
+ * of the jsDelivr shim is revoked whether the start succeeds or fails.
35
+ */
36
+ export async function startDuckDb(
37
+ bundle: duckdb.DuckDBBundle,
38
+ options: { selfHosted: boolean; path?: string },
39
+ env: DuckDbStartEnv,
40
+ ): Promise<StartedDuckDb> {
41
+ const { selfHosted, path } = options;
42
+ let blobUrl: string | undefined;
43
+ let worker: Worker | undefined;
44
+ let db: duckdb.AsyncDuckDB | undefined;
45
+ // duckdb-wasm only logs a worker error (a script that fails to load) and drops the pending
46
+ // request, so `instantiate` would never settle: turn the error event into a rejection.
47
+ let onWorkerError: ((event: ErrorEvent) => void) | undefined;
48
+ const workerFailed = new Promise<never>((_, reject) => {
49
+ onWorkerError = (event) =>
50
+ reject(new Error(`DuckDB worker failed to load: ${event.message ?? "unknown error"}`));
51
+ });
52
+ workerFailed.catch(() => {}); // only observed through the race below
53
+ try {
54
+ // OPFS file I/O needs a same-origin worker. Self-hosted bundles give a
55
+ // same-origin URL we load directly; the jsDelivr fallback is cross-origin
56
+ // and must be wrapped in a Blob that `importScripts` it.
57
+ if (selfHosted) {
58
+ worker = env.newWorker(bundle.mainWorker as string);
59
+ } else {
60
+ blobUrl = env.createObjectURL(
61
+ new Blob([`importScripts("${bundle.mainWorker ?? ""}");`], {
62
+ type: "text/javascript",
63
+ }),
64
+ );
65
+ worker = env.newWorker(blobUrl);
66
+ }
67
+ worker.addEventListener("error", onWorkerError as (event: ErrorEvent) => void);
68
+ db = env.newDb(worker);
69
+ await Promise.race([db.instantiate(bundle.mainModule, bundle.pthreadWorker), workerFailed]);
70
+ if (path) {
71
+ try {
72
+ await db.open({ path, accessMode: duckdb.DuckDBAccessMode.READ_WRITE });
73
+ } catch (error) {
74
+ throw new Error(`Cannot open DuckDB database "${path}"`, { cause: error });
75
+ }
76
+ }
77
+ const conn = await db.connect();
78
+ return { worker, db, conn };
79
+ } catch (error) {
80
+ // Best effort: a failing teardown must not hide the start error.
81
+ try {
82
+ await db?.terminate();
83
+ } catch {}
84
+ worker?.terminate();
85
+ throw error;
86
+ } finally {
87
+ worker?.removeEventListener("error", onWorkerError as (event: ErrorEvent) => void);
88
+ if (blobUrl) env.revokeObjectURL(blobUrl);
89
+ }
90
+ }
@@ -1,208 +0,0 @@
1
- import * as duckdb from "@duckdb/duckdb-wasm";
2
- import type { Db } from "@statewalker/db-api";
3
- import { runDbConformance } from "@statewalker/db-tests";
4
- import { afterEach, describe, expect, it, vi } from "vitest";
5
- import { newBrowserDuckDb } from "./browser-duckdb.js";
6
-
7
- /**
8
- * The DuckDB-WASM browser adapter, exercised in a REAL browser (Vitest browser
9
- * mode + Playwright/Chromium). Launched only by `vitest.browser.config.ts` (the
10
- * `test:browser` script) — the Node config excludes `*.browser.test.ts`.
11
- *
12
- * With no `bundles`/`path`, the adapter loads the default DuckDB bundle from
13
- * jsDelivr and runs in-memory (a Worker inside the browser page), so the run
14
- * needs outbound network access from Chromium. The node-only file-persistence
15
- * scenario is skipped via `skipFilePersistence`.
16
- */
17
-
18
- // Shared contract: CRUD, parameter binding, injection safety, cardinality,
19
- // close, flush, error paths. File persistence is node-only, hence skipped here.
20
- runDbConformance(() => newBrowserDuckDb(), {
21
- // duckdb dialect: `$1` placeholder; rows are already plain objects, so the
22
- // spread normalizer is an identity-style copy.
23
- placeholder: "$1",
24
- normalizeRow: (row) => ({ ...row }),
25
- skipFilePersistence: true,
26
- });
27
-
28
- // Dialect-specific browser behavior the shared suite does not cover: DuckDB's
29
- // FTS extension and VSS/HNSW vector index.
30
- describe("newBrowserDuckDb", () => {
31
- let db: Db;
32
-
33
- afterEach(async () => {
34
- if (db) {
35
- await db.close();
36
- }
37
- });
38
-
39
- describe("FTS extension", () => {
40
- it("returns empty results for unmatched search terms", async () => {
41
- db = await newBrowserDuckDb();
42
- await db.exec("INSTALL fts; LOAD fts;");
43
- await db.exec("CREATE TABLE docs2 (id INTEGER, content VARCHAR)");
44
- await db.exec("INSERT INTO docs2 VALUES (1, 'hello world'), (2, 'goodbye world')");
45
- await db.exec("PRAGMA create_fts_index('docs2', 'id', 'content')");
46
-
47
- const rows = await db.query<{ id: number; score: number | null }>(`
48
- SELECT id, score FROM (
49
- SELECT *, fts_main_docs2.match_bm25(id, 'nonexistent') AS score FROM docs2
50
- ) sq WHERE score IS NOT NULL
51
- `);
52
- expect(rows).toHaveLength(0);
53
- });
54
-
55
- it("searches across multiple text columns", async () => {
56
- db = await newBrowserDuckDb();
57
- await db.exec("INSTALL fts; LOAD fts;");
58
- await db.exec("CREATE TABLE articles (id INTEGER, title VARCHAR, body VARCHAR)");
59
- await db.exec(`
60
- INSERT INTO articles VALUES
61
- (1, 'Travel Guide', 'Visit the beautiful lakes of Switzerland'),
62
- (2, 'Cooking Tips', 'How to make a perfect lake trout'),
63
- (3, 'Programming', 'Learn about SQL databases')
64
- `);
65
- await db.exec("PRAGMA create_fts_index('articles', 'id', 'title', 'body')");
66
-
67
- const rows = await db.query<{ id: number; score: number | null }>(`
68
- SELECT id, score FROM (
69
- SELECT *, fts_main_articles.match_bm25(id, 'lake') AS score FROM articles
70
- ) sq WHERE score IS NOT NULL
71
- `);
72
- expect(rows.length).toBe(2);
73
- const ids = rows.map((r) => r.id);
74
- expect(ids).toContain(1);
75
- expect(ids).toContain(2);
76
- });
77
-
78
- it("creates an FTS index and performs full-text search", async () => {
79
- db = await newBrowserDuckDb();
80
- await db.exec("INSTALL fts; LOAD fts;");
81
- await db.exec("CREATE TABLE docs (id INTEGER, content VARCHAR)");
82
- await db.exec(`
83
- INSERT INTO docs VALUES
84
- (1, 'the quick brown fox jumps over the lazy dog'),
85
- (2, 'a lazy cat sleeps on the mat'),
86
- (3, 'the fox and the hound are friends')
87
- `);
88
- await db.exec("PRAGMA create_fts_index('docs', 'id', 'content')");
89
-
90
- const rows = await db.query<{
91
- id: number;
92
- content: string;
93
- score: number | null;
94
- }>(`
95
- SELECT id, content, score
96
- FROM (
97
- SELECT *, fts_main_docs.match_bm25(id, 'fox') AS score
98
- FROM docs
99
- ) sq
100
- WHERE score IS NOT NULL
101
- ORDER BY score
102
- `);
103
-
104
- expect(rows.length).toBeGreaterThanOrEqual(1);
105
- const ids = rows.map((r) => r.id);
106
- expect(ids).toContain(1);
107
- expect(ids).toContain(3);
108
- });
109
- });
110
-
111
- describe("VSS extension", () => {
112
- it("creates HNSW index and performs vector similarity search", async () => {
113
- db = await newBrowserDuckDb();
114
- await db.exec("INSTALL vss; LOAD vss;");
115
- await db.exec("CREATE TABLE embeddings (id INTEGER, vec FLOAT[3])");
116
- await db.exec(`
117
- INSERT INTO embeddings VALUES
118
- (1, [1.0, 0.0, 0.0]),
119
- (2, [0.0, 1.0, 0.0]),
120
- (3, [0.0, 0.0, 1.0])
121
- `);
122
- await db.exec("CREATE INDEX vec_idx ON embeddings USING HNSW (vec)");
123
-
124
- const rows = await db.query<{ id: number }>(
125
- "SELECT id FROM embeddings ORDER BY array_distance(vec, [1.0, 0.1, 0.0]::FLOAT[3]) LIMIT 1",
126
- );
127
-
128
- expect(rows).toHaveLength(1);
129
- expect(rows[0]?.id).toBe(1);
130
- });
131
-
132
- it("returns k nearest neighbors ordered by distance", async () => {
133
- db = await newBrowserDuckDb();
134
- await db.exec("INSTALL vss; LOAD vss;");
135
- await db.exec("CREATE TABLE vectors (id INTEGER, vec FLOAT[3])");
136
- await db.exec(`
137
- INSERT INTO vectors VALUES
138
- (1, [1.0, 0.0, 0.0]),
139
- (2, [0.9, 0.1, 0.0]),
140
- (3, [0.0, 1.0, 0.0]),
141
- (4, [0.0, 0.0, 1.0])
142
- `);
143
- await db.exec("CREATE INDEX vec_idx2 ON vectors USING HNSW (vec)");
144
-
145
- const rows = await db.query<{ id: number }>(
146
- "SELECT id FROM vectors ORDER BY array_distance(vec, [1.0, 0.0, 0.0]::FLOAT[3]) LIMIT 2",
147
- );
148
-
149
- expect(rows).toHaveLength(2);
150
- expect(rows[0]?.id).toBe(1);
151
- expect(rows[1]?.id).toBe(2);
152
- });
153
-
154
- it("works without HNSW index (brute-force scan)", async () => {
155
- db = await newBrowserDuckDb();
156
- await db.exec("INSTALL vss; LOAD vss;");
157
- await db.exec("CREATE TABLE vecs_no_idx (id INTEGER, vec FLOAT[3])");
158
- await db.exec(`
159
- INSERT INTO vecs_no_idx VALUES
160
- (1, [1.0, 0.0, 0.0]),
161
- (2, [0.0, 1.0, 0.0])
162
- `);
163
-
164
- const rows = await db.query<{ id: number }>(
165
- "SELECT id FROM vecs_no_idx ORDER BY array_distance(vec, [0.0, 0.9, 0.1]::FLOAT[3]) LIMIT 1",
166
- );
167
-
168
- expect(rows).toHaveLength(1);
169
- expect(rows[0]?.id).toBe(2);
170
- });
171
- });
172
-
173
- describe("prepared statements", () => {
174
- afterEach(() => {
175
- vi.restoreAllMocks();
176
- });
177
-
178
- it("closes the statement of every parameterised query", async () => {
179
- db = await newBrowserDuckDb();
180
- const close = vi.spyOn(duckdb.AsyncPreparedStatement.prototype, "close");
181
- await db.query("SELECT $1::INTEGER AS x", [1]);
182
- await db.query("SELECT $1::INTEGER AS x", [2]);
183
- await expect(db.query("SELECT $1::INTEGER AS x", ["not a number"])).rejects.toThrow();
184
- expect(close).toHaveBeenCalledTimes(3);
185
- });
186
- });
187
-
188
- describe("persistent path", () => {
189
- afterEach(() => {
190
- vi.unstubAllGlobals();
191
- vi.restoreAllMocks();
192
- });
193
-
194
- it("rejects instead of falling back to memory when the path cannot be opened", async () => {
195
- vi.spyOn(duckdb.AsyncDuckDB.prototype, "open").mockRejectedValue(new Error("open failed"));
196
- await expect(newBrowserDuckDb({ path: "opfs://broken.duckdb" })).rejects.toThrow(
197
- /opfs:\/\/broken\.duckdb/,
198
- );
199
- });
200
-
201
- it("rejects instead of falling back to memory when OPFS is unavailable", async () => {
202
- vi.stubGlobal("navigator", {});
203
- await expect(newBrowserDuckDb({ path: "opfs://unavailable.duckdb" })).rejects.toThrow(
204
- /opfs:\/\/unavailable\.duckdb/,
205
- );
206
- });
207
- });
208
- });
@@ -1,14 +0,0 @@
1
- import { describe, expect, it } from "vitest";
2
-
3
- /**
4
- * Node-safe smoke test. The real browser coverage (shared conformance suite +
5
- * FTS/VSS) lives in `browser-duckdb.browser.test.ts`, run via Vitest browser
6
- * mode (`pnpm test:browser`); this file only checks the module loads and
7
- * exports its factory, since duckdb-wasm needs a browser Worker to run.
8
- */
9
- describe("browser-duckdb module", () => {
10
- it("exports newBrowserDuckDb function", async () => {
11
- const mod = await import("./browser-duckdb.js");
12
- expect(typeof mod.newBrowserDuckDb).toBe("function");
13
- });
14
- });