@statewalker/db-duckdb-browser 0.1.6 → 0.1.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +66 -10
- package/dist/index.d.mts +3 -1
- package/dist/index.d.mts.map +1 -1
- package/dist/index.mjs +219 -32
- package/dist/index.mjs.map +1 -1
- package/package.json +7 -4
- package/src/arrow-values.ts +147 -0
- package/src/browser-duckdb.ts +23 -57
- package/src/params.ts +59 -0
- package/src/start-duckdb.ts +90 -0
- package/src/browser-duckdb.browser.test.ts +0 -208
- package/src/browser-duckdb.test.ts +0 -14
package/README.md
CHANGED
|
@@ -7,6 +7,11 @@ A DuckDB driver for the browser that implements `Db` from
|
|
|
7
7
|
Worker. The database lives in memory, or in an OPFS file that survives page
|
|
8
8
|
reloads.
|
|
9
9
|
|
|
10
|
+
> **Known issue:** with `@duckdb/duckdb-wasm` 1.33.1-dev57.0, a new OPFS database is not
|
|
11
|
+
> actually written to OPFS, so it does not survive a reopen
|
|
12
|
+
> ([#74](https://github.com/statewalker/statewalker-db/issues/74)). The persistence tests in
|
|
13
|
+
> `src/opfs-persistence.browser.test.ts` are marked as expected failures until that is fixed.
|
|
14
|
+
|
|
10
15
|
## Why it exists
|
|
11
16
|
|
|
12
17
|
DuckDB WASM has its own asynchronous API built around Arrow tables, worker
|
|
@@ -29,12 +34,56 @@ pnpm add @statewalker/db-duckdb-browser
|
|
|
29
34
|
- `options.path`: an OPFS path such as `opfs://app.duckdb`. Omit it for an
|
|
30
35
|
in-memory database. If the path cannot be opened, the call rejects with
|
|
31
36
|
`Cannot open DuckDB database "<path>"`; it never falls back to memory.
|
|
32
|
-
- `options.bundles`: same-origin `DuckDBBundles` URLs. Required for OPFS
|
|
33
|
-
|
|
37
|
+
- `options.bundles`: same-origin `DuckDBBundles` URLs. Required for OPFS:
|
|
38
|
+
`options.path` without `options.bundles` rejects with
|
|
39
|
+
`Cannot open DuckDB database "<path>": OPFS needs self-hosted \`bundles\``
|
|
40
|
+
before any worker starts. Without it, the bundle is loaded from jsDelivr.
|
|
34
41
|
|
|
35
42
|
The returned `Db` implements `query`, `exec`, `flush` and `close`. Placeholders
|
|
36
43
|
are DuckDB's (`$1`, `$2`, ...).
|
|
37
44
|
|
|
45
|
+
### Result values
|
|
46
|
+
|
|
47
|
+
Rows come back with the same JS values as `@statewalker/db-duckdb-node`:
|
|
48
|
+
|
|
49
|
+
| DuckDB type | JS value |
|
|
50
|
+
| --- | --- |
|
|
51
|
+
| `DECIMAL` | `number` (scaled; `1.25::DECIMAL(10,2)` is `1.25`) |
|
|
52
|
+
| `BIGINT`, `UBIGINT`, `HUGEINT`, `TIME` | `bigint` |
|
|
53
|
+
| `INTEGER`, `DOUBLE`, ... | `number` |
|
|
54
|
+
| `DATE`, `TIMESTAMP`, `TIMESTAMPTZ` | `Date` |
|
|
55
|
+
| `LIST`, fixed-size `ARRAY` | array, converted recursively |
|
|
56
|
+
| `STRUCT` | plain object, converted recursively |
|
|
57
|
+
| `MAP` | array of `{ key, value }` |
|
|
58
|
+
| `BLOB` | `Uint8Array` |
|
|
59
|
+
|
|
60
|
+
`INTERVAL` is returned as duckdb-wasm's Arrow layer decodes it and does not
|
|
61
|
+
match the node driver; cast it in SQL (for example to `VARCHAR`) if you need
|
|
62
|
+
it. A `HUGEINT` and a `DECIMAL(38,0)` look the same in Arrow, so both come
|
|
63
|
+
back as `bigint`.
|
|
64
|
+
|
|
65
|
+
### Parameter limits
|
|
66
|
+
|
|
67
|
+
duckdb-wasm passes parameters through `JSON.stringify`, so the driver converts
|
|
68
|
+
them first:
|
|
69
|
+
|
|
70
|
+
| JS value | Bound as |
|
|
71
|
+
| --- | --- |
|
|
72
|
+
| `string`, `boolean`, `null` | itself (`undefined` binds as `null`) |
|
|
73
|
+
| finite `number` | `DOUBLE` |
|
|
74
|
+
| `bigint` | its exact decimal string, cast by DuckDB to the column's type |
|
|
75
|
+
| `Date` | its ISO 8601 string (`toISOString()`), cast to `TIMESTAMP` the same way |
|
|
76
|
+
|
|
77
|
+
The string forms work where the SQL fixes the parameter's type, as in
|
|
78
|
+
`WHERE id = $1` or `INSERT INTO t (id) VALUES ($1)`, so a `BIGINT` id read
|
|
79
|
+
from one query can be passed back to the next. Where nothing fixes it, cast
|
|
80
|
+
in SQL: `$1::BIGINT + 1`, `$1::TIMESTAMP`.
|
|
81
|
+
|
|
82
|
+
`Uint8Array` and other binary data, arrays, objects, `NaN`, `Infinity` and
|
|
83
|
+
invalid dates are rejected with a `TypeError` that names the parameter and its
|
|
84
|
+
type (`cannot bind parameter $1 of type Uint8Array`). Insert such values with
|
|
85
|
+
SQL literals, or through duckdb-wasm's own Arrow insert API.
|
|
86
|
+
|
|
38
87
|
## Examples
|
|
39
88
|
|
|
40
89
|
In memory, bundle from jsDelivr:
|
|
@@ -82,24 +131,31 @@ await db.flush?.(); // CHECKPOINT: without it the insert can be lost on reload
|
|
|
82
131
|
- **Same-origin worker for OPFS.** OPFS file I/O works only in a same-origin
|
|
83
132
|
worker. With `bundles`, the worker is created directly from your URL. Without
|
|
84
133
|
them, the jsDelivr worker is cross-origin and is started through a Blob that
|
|
85
|
-
calls `importScripts`; that worker crashes on OPFS file I/O, so
|
|
86
|
-
in-memory databases only
|
|
134
|
+
calls `importScripts`; that worker crashes on OPFS file I/O, so the driver
|
|
135
|
+
accepts it for in-memory databases only and rejects a `path` without
|
|
136
|
+
`bundles`.
|
|
137
|
+
- **A failed start leaves nothing running.** If loading the engine, opening
|
|
138
|
+
the path or connecting fails, the engine and its worker are terminated and
|
|
139
|
+
the jsDelivr Blob URL is revoked before the call rejects with the original
|
|
140
|
+
error (`src/start-duckdb.ts`).
|
|
87
141
|
- **`flush` runs `CHECKPOINT`.** DuckDB WASM writes the OPFS database file only
|
|
88
142
|
on `CHECKPOINT`. Writes not yet checkpointed can be lost on an abrupt
|
|
89
143
|
teardown such as a page reload. On an in-memory database `flush` does
|
|
90
144
|
nothing.
|
|
91
|
-
- **No fallback to memory.** If `options.path` is set
|
|
145
|
+
- **No fallback to memory.** If `options.path` is set without `bundles`, the
|
|
146
|
+
call rejects before starting a worker (see above). If `navigator.storage`
|
|
92
147
|
is missing, the call rejects with
|
|
93
148
|
`Cannot open DuckDB database "<path>": OPFS is unavailable`. If opening the
|
|
94
149
|
path throws, the worker is terminated and the call rejects with
|
|
95
150
|
`Cannot open DuckDB database "<path>"`, the engine's error as `cause`. An
|
|
96
151
|
in-memory database is created only when `options.path` is omitted, so data
|
|
97
152
|
is never silently lost on reload.
|
|
98
|
-
- **Rows.** Query results are Arrow tables; the driver copies each row
|
|
99
|
-
|
|
100
|
-
`
|
|
101
|
-
|
|
102
|
-
`
|
|
153
|
+
- **Rows.** Query results are Arrow tables; the driver copies each row into a
|
|
154
|
+
plain object, converting each value by its Arrow field type
|
|
155
|
+
(`src/arrow-values.ts`). Calls with parameters convert them
|
|
156
|
+
(`src/params.ts`), then go through `conn.prepare(sql)` and
|
|
157
|
+
`stmt.query(...params)`, and the statement is closed after each call, also
|
|
158
|
+
when the query fails; calls without parameters use `conn.query(sql)`.
|
|
103
159
|
- **Under Node.js** the module imports, but `newBrowserDuckDb()` rejects with
|
|
104
160
|
`Worker is not defined`. Use `@statewalker/db-duckdb-node` there.
|
|
105
161
|
- **Dependencies:** `@duckdb/duckdb-wasm` (the engine), `@statewalker/db-api`
|
package/dist/index.d.mts
CHANGED
|
@@ -8,7 +8,8 @@ interface BrowserDuckDbOptions extends DbOptions$1 {
|
|
|
8
8
|
* `?url` import). Passing these makes the factory spawn a **same-origin**
|
|
9
9
|
* worker, which OPFS database persistence requires — a cross-origin worker
|
|
10
10
|
* (the jsDelivr fallback, loaded via a Blob `importScripts` shim) crashes on
|
|
11
|
-
* OPFS file I/O. Omit to load from jsDelivr (in-memory use only)
|
|
11
|
+
* OPFS file I/O. Omit to load from jsDelivr (in-memory use only); a `path`
|
|
12
|
+
* without `bundles` is rejected.
|
|
12
13
|
*/
|
|
13
14
|
bundles?: duckdb.DuckDBBundles;
|
|
14
15
|
}
|
|
@@ -19,6 +20,7 @@ interface BrowserDuckDbOptions extends DbOptions$1 {
|
|
|
19
20
|
* `options.bundles`, the database persists across reloads on OPFS. Without
|
|
20
21
|
* `options.path` it runs in-memory. If `options.path` cannot be opened, the
|
|
21
22
|
* returned promise rejects; it never falls back to an in-memory database.
|
|
23
|
+
* `options.path` without `options.bundles` rejects before any worker starts.
|
|
22
24
|
*
|
|
23
25
|
* @param options.path `opfs://` path for persistent storage. Omit for in-memory.
|
|
24
26
|
* @param options.bundles Same-origin bundle URLs (required for OPFS).
|
package/dist/index.d.mts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.d.mts","names":["DbOptions","Db"],"sources":["../src/browser-duckdb.ts"],"mappings":";;;;
|
|
1
|
+
{"version":3,"file":"index.d.mts","names":["DbOptions","Db"],"sources":["../src/browser-duckdb.ts"],"mappings":";;;;UAOiB,6BAA6BA;;;;;;;;;EAS5C,UAAU,OAAO;;;;;;;;;;;;;;wBAeG,iBAAiB,UAAU,uBAAuB,QAAQC"}
|
package/dist/index.mjs
CHANGED
|
@@ -1,65 +1,252 @@
|
|
|
1
1
|
import * as duckdb from "@duckdb/duckdb-wasm";
|
|
2
|
-
//#region src/
|
|
2
|
+
//#region src/arrow-values.ts
|
|
3
|
+
/**
|
|
4
|
+
* Conversion of Arrow values (as duckdb-wasm returns them) into the plain JS
|
|
5
|
+
* values `@statewalker/db-duckdb-node` returns for the same SQL types.
|
|
6
|
+
*
|
|
7
|
+
* Typed structurally so this module does not import `apache-arrow` at runtime.
|
|
8
|
+
* Type ids are Arrow's `Type` enum (apache-arrow 17, the version duckdb-wasm
|
|
9
|
+
* depends on).
|
|
10
|
+
*/
|
|
11
|
+
const TypeId = {
|
|
12
|
+
Dictionary: -1,
|
|
13
|
+
Decimal: 7,
|
|
14
|
+
Date: 8,
|
|
15
|
+
Timestamp: 10,
|
|
16
|
+
List: 12,
|
|
17
|
+
Struct: 13,
|
|
18
|
+
FixedSizeList: 16,
|
|
19
|
+
Map: 17
|
|
20
|
+
};
|
|
21
|
+
/** Arrow `TimeUnit`. */
|
|
22
|
+
const TimeUnit = {
|
|
23
|
+
SECOND: 0,
|
|
24
|
+
MILLISECOND: 1,
|
|
25
|
+
MICROSECOND: 2,
|
|
26
|
+
NANOSECOND: 3
|
|
27
|
+
};
|
|
3
28
|
/**
|
|
4
|
-
* Convert an Arrow
|
|
29
|
+
* Convert an Arrow table into plain row objects, converting each value by its
|
|
30
|
+
* field's Arrow type ({@link arrowValueToJs}).
|
|
5
31
|
*/
|
|
6
|
-
function
|
|
7
|
-
const fields = table.schema.fields
|
|
8
|
-
return table.toArray()
|
|
32
|
+
function arrowTableToObjects(table) {
|
|
33
|
+
const fields = table.schema.fields;
|
|
34
|
+
return Array.from(table.toArray(), (row) => {
|
|
35
|
+
const source = row;
|
|
9
36
|
const obj = {};
|
|
10
|
-
for (const field of fields) obj[field] =
|
|
37
|
+
for (const field of fields) obj[field.name] = arrowValueToJs(source[field.name], field.type);
|
|
11
38
|
return obj;
|
|
12
39
|
});
|
|
13
40
|
}
|
|
14
41
|
/**
|
|
42
|
+
* Convert one Arrow value into the plain JS value the node driver returns:
|
|
43
|
+
*
|
|
44
|
+
* - DECIMAL: a `number` scaled by the type's scale (HUGEINT, which DuckDB sends
|
|
45
|
+
* as DECIMAL(38,0), becomes a `bigint`);
|
|
46
|
+
* - DATE, TIMESTAMP: a `Date`;
|
|
47
|
+
* - LIST, fixed-size ARRAY: an array, converted recursively;
|
|
48
|
+
* - STRUCT: a plain object, converted recursively;
|
|
49
|
+
* - MAP: an array of `{ key, value }`, converted recursively.
|
|
50
|
+
*
|
|
51
|
+
* Other values (numbers, `bigint` for BIGINT, strings, booleans, `Uint8Array`
|
|
52
|
+
* for BLOB, `null`) are returned as they are.
|
|
53
|
+
*/
|
|
54
|
+
function arrowValueToJs(value, type) {
|
|
55
|
+
if (value === null || value === void 0) return null;
|
|
56
|
+
switch (type.typeId) {
|
|
57
|
+
case TypeId.Decimal: return decimalToJs(value, type);
|
|
58
|
+
case TypeId.Date: return typeof value === "number" ? new Date(value) : value;
|
|
59
|
+
case TypeId.Timestamp: return timestampToDate(value, type.unit);
|
|
60
|
+
case TypeId.List:
|
|
61
|
+
case TypeId.FixedSizeList: {
|
|
62
|
+
const childType = type.children?.[0]?.type;
|
|
63
|
+
const items = Array.from(value);
|
|
64
|
+
return childType ? items.map((item) => arrowValueToJs(item, childType)) : items;
|
|
65
|
+
}
|
|
66
|
+
case TypeId.Struct: {
|
|
67
|
+
const row = value;
|
|
68
|
+
const obj = {};
|
|
69
|
+
for (const child of type.children ?? []) obj[child.name] = arrowValueToJs(row[child.name], child.type);
|
|
70
|
+
return obj;
|
|
71
|
+
}
|
|
72
|
+
case TypeId.Map: {
|
|
73
|
+
const entryFields = type.children?.[0]?.type.children ?? [];
|
|
74
|
+
const keyType = entryFields[0]?.type;
|
|
75
|
+
const valueType = entryFields[1]?.type;
|
|
76
|
+
return Array.from(value, ([key, val]) => ({
|
|
77
|
+
key: keyType ? arrowValueToJs(key, keyType) : key,
|
|
78
|
+
value: valueType ? arrowValueToJs(val, valueType) : val
|
|
79
|
+
}));
|
|
80
|
+
}
|
|
81
|
+
case TypeId.Dictionary: return type.dictionary ? arrowValueToJs(value, type.dictionary) : value;
|
|
82
|
+
default: return value;
|
|
83
|
+
}
|
|
84
|
+
}
|
|
85
|
+
/**
|
|
86
|
+
* Arrow decimals are unscaled integers (`String(v)` gives the digits). Insert
|
|
87
|
+
* the decimal point by text and parse, so the result is the nearest double to
|
|
88
|
+
* the exact decimal value.
|
|
89
|
+
*/
|
|
90
|
+
function decimalToJs(value, type) {
|
|
91
|
+
const digits = String(value);
|
|
92
|
+
const scale = type.scale ?? 0;
|
|
93
|
+
if (scale === 0) return type.precision === 38 ? BigInt(digits) : Number(digits);
|
|
94
|
+
const negative = digits.startsWith("-");
|
|
95
|
+
const abs = (negative ? digits.slice(1) : digits).padStart(scale + 1, "0");
|
|
96
|
+
const text = `${abs.slice(0, abs.length - scale)}.${abs.slice(abs.length - scale)}`;
|
|
97
|
+
return Number(negative ? `-${text}` : text);
|
|
98
|
+
}
|
|
99
|
+
function timestampToDate(value, unit) {
|
|
100
|
+
if (typeof value === "number") return new Date(value);
|
|
101
|
+
if (typeof value === "bigint") switch (unit) {
|
|
102
|
+
case TimeUnit.SECOND: return /* @__PURE__ */ new Date(Number(value) * 1e3);
|
|
103
|
+
case TimeUnit.MICROSECOND: return new Date(Number(value / 1000n));
|
|
104
|
+
case TimeUnit.NANOSECOND: return new Date(Number(value / 1000000n));
|
|
105
|
+
default: return new Date(Number(value));
|
|
106
|
+
}
|
|
107
|
+
return value;
|
|
108
|
+
}
|
|
109
|
+
//#endregion
|
|
110
|
+
//#region src/params.ts
|
|
111
|
+
/**
|
|
112
|
+
* Convert `Db.query` params into values duckdb-wasm can bind.
|
|
113
|
+
*
|
|
114
|
+
* duckdb-wasm passes prepared-statement params through `JSON.stringify`, so
|
|
115
|
+
* only strings, finite numbers, booleans and `null` survive. This maps:
|
|
116
|
+
*
|
|
117
|
+
* - `bigint` to its exact decimal string; DuckDB casts it to the column type
|
|
118
|
+
* where the context fixes one (`WHERE id = $1`, `INSERT ... VALUES ($1)`),
|
|
119
|
+
* elsewhere cast it in SQL (`$1::BIGINT`);
|
|
120
|
+
* - `Date` to its ISO 8601 string, which DuckDB casts to TIMESTAMP the same way;
|
|
121
|
+
* - `undefined` to `null`.
|
|
122
|
+
*
|
|
123
|
+
* Anything else (`Uint8Array` and other binary data, arrays, plain objects,
|
|
124
|
+
* `NaN`, `Infinity`, symbols, functions, invalid dates) throws a `TypeError`
|
|
125
|
+
* naming the parameter and its type instead of failing inside duckdb-wasm or
|
|
126
|
+
* binding the wrong value.
|
|
127
|
+
*/
|
|
128
|
+
function toDuckDbParams(params) {
|
|
129
|
+
return params.map(toDuckDbParam);
|
|
130
|
+
}
|
|
131
|
+
function toDuckDbParam(value, index) {
|
|
132
|
+
switch (typeof value) {
|
|
133
|
+
case "string":
|
|
134
|
+
case "boolean": return value;
|
|
135
|
+
case "undefined": return null;
|
|
136
|
+
case "bigint": return value.toString();
|
|
137
|
+
case "number":
|
|
138
|
+
if (Number.isFinite(value)) return value;
|
|
139
|
+
throw unsupported(index, String(value));
|
|
140
|
+
case "object":
|
|
141
|
+
if (value === null) return null;
|
|
142
|
+
if (value instanceof Date) {
|
|
143
|
+
if (Number.isNaN(value.getTime())) throw unsupported(index, "invalid Date");
|
|
144
|
+
return value.toISOString();
|
|
145
|
+
}
|
|
146
|
+
if (Array.isArray(value)) throw unsupported(index, "array");
|
|
147
|
+
if (ArrayBuffer.isView(value) || value instanceof ArrayBuffer) throw unsupported(index, value.constructor.name);
|
|
148
|
+
throw unsupported(index, `object (${value.constructor?.name ?? "null prototype"})`);
|
|
149
|
+
default: throw unsupported(index, typeof value);
|
|
150
|
+
}
|
|
151
|
+
}
|
|
152
|
+
function unsupported(index, type) {
|
|
153
|
+
return /* @__PURE__ */ new TypeError(`db-duckdb-browser: cannot bind parameter $${index + 1} of type ${type}; duckdb-wasm binds only string, finite number, boolean, null, bigint and Date`);
|
|
154
|
+
}
|
|
155
|
+
//#endregion
|
|
156
|
+
//#region src/start-duckdb.ts
|
|
157
|
+
/** The real browser environment: a Web Worker and the page's `URL`. */
|
|
158
|
+
const browserStartEnv = {
|
|
159
|
+
newWorker: (url) => new Worker(url),
|
|
160
|
+
newDb: (worker) => new duckdb.AsyncDuckDB(new duckdb.ConsoleLogger(), worker),
|
|
161
|
+
createObjectURL: (blob) => URL.createObjectURL(blob),
|
|
162
|
+
revokeObjectURL: (url) => URL.revokeObjectURL(url)
|
|
163
|
+
};
|
|
164
|
+
/**
|
|
165
|
+
* Start a worker, instantiate DuckDB in it, open `path` if given, and connect.
|
|
166
|
+
*
|
|
167
|
+
* If any step fails, including the worker script failing to load, the engine
|
|
168
|
+
* and its worker are terminated before the error is rethrown, so a failed start leaves no worker running. The Blob URL
|
|
169
|
+
* of the jsDelivr shim is revoked whether the start succeeds or fails.
|
|
170
|
+
*/
|
|
171
|
+
async function startDuckDb(bundle, options, env) {
|
|
172
|
+
const { selfHosted, path } = options;
|
|
173
|
+
let blobUrl;
|
|
174
|
+
let worker;
|
|
175
|
+
let db;
|
|
176
|
+
let onWorkerError;
|
|
177
|
+
const workerFailed = new Promise((_, reject) => {
|
|
178
|
+
onWorkerError = (event) => reject(/* @__PURE__ */ new Error(`DuckDB worker failed to load: ${event.message ?? "unknown error"}`));
|
|
179
|
+
});
|
|
180
|
+
workerFailed.catch(() => {});
|
|
181
|
+
try {
|
|
182
|
+
if (selfHosted) worker = env.newWorker(bundle.mainWorker);
|
|
183
|
+
else {
|
|
184
|
+
blobUrl = env.createObjectURL(new Blob([`importScripts("${bundle.mainWorker ?? ""}");`], { type: "text/javascript" }));
|
|
185
|
+
worker = env.newWorker(blobUrl);
|
|
186
|
+
}
|
|
187
|
+
worker.addEventListener("error", onWorkerError);
|
|
188
|
+
db = env.newDb(worker);
|
|
189
|
+
await Promise.race([db.instantiate(bundle.mainModule, bundle.pthreadWorker), workerFailed]);
|
|
190
|
+
if (path) try {
|
|
191
|
+
await db.open({
|
|
192
|
+
path,
|
|
193
|
+
accessMode: duckdb.DuckDBAccessMode.READ_WRITE
|
|
194
|
+
});
|
|
195
|
+
} catch (error) {
|
|
196
|
+
throw new Error(`Cannot open DuckDB database "${path}"`, { cause: error });
|
|
197
|
+
}
|
|
198
|
+
const conn = await db.connect();
|
|
199
|
+
return {
|
|
200
|
+
worker,
|
|
201
|
+
db,
|
|
202
|
+
conn
|
|
203
|
+
};
|
|
204
|
+
} catch (error) {
|
|
205
|
+
try {
|
|
206
|
+
await db?.terminate();
|
|
207
|
+
} catch {}
|
|
208
|
+
worker?.terminate();
|
|
209
|
+
throw error;
|
|
210
|
+
} finally {
|
|
211
|
+
worker?.removeEventListener("error", onWorkerError);
|
|
212
|
+
if (blobUrl) env.revokeObjectURL(blobUrl);
|
|
213
|
+
}
|
|
214
|
+
}
|
|
215
|
+
//#endregion
|
|
216
|
+
//#region src/browser-duckdb.ts
|
|
217
|
+
/**
|
|
15
218
|
* Create a DuckDB-backed {@link Db} using WebAssembly in the browser.
|
|
16
219
|
*
|
|
17
220
|
* With `options.path` set to an `opfs://` URL **and** self-hosted
|
|
18
221
|
* `options.bundles`, the database persists across reloads on OPFS. Without
|
|
19
222
|
* `options.path` it runs in-memory. If `options.path` cannot be opened, the
|
|
20
223
|
* returned promise rejects; it never falls back to an in-memory database.
|
|
224
|
+
* `options.path` without `options.bundles` rejects before any worker starts.
|
|
21
225
|
*
|
|
22
226
|
* @param options.path `opfs://` path for persistent storage. Omit for in-memory.
|
|
23
227
|
* @param options.bundles Same-origin bundle URLs (required for OPFS).
|
|
24
228
|
*/
|
|
25
229
|
async function newBrowserDuckDb(options) {
|
|
26
230
|
const path = options?.path;
|
|
27
|
-
if (path && (typeof navigator === "undefined" || !navigator.storage)) throw new Error(`Cannot open DuckDB database "${path}": OPFS is unavailable`);
|
|
28
231
|
const selfHosted = options?.bundles;
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
worker = new Worker(blobUrl);
|
|
36
|
-
}
|
|
37
|
-
const logger = new duckdb.ConsoleLogger();
|
|
38
|
-
const db = new duckdb.AsyncDuckDB(logger, worker);
|
|
39
|
-
await db.instantiate(bundle.mainModule, bundle.pthreadWorker);
|
|
40
|
-
if (blobUrl) URL.revokeObjectURL(blobUrl);
|
|
41
|
-
if (path) try {
|
|
42
|
-
await db.open({
|
|
43
|
-
path,
|
|
44
|
-
accessMode: duckdb.DuckDBAccessMode.READ_WRITE
|
|
45
|
-
});
|
|
46
|
-
} catch (error) {
|
|
47
|
-
await db.terminate();
|
|
48
|
-
worker.terminate();
|
|
49
|
-
throw new Error(`Cannot open DuckDB database "${path}"`, { cause: error });
|
|
50
|
-
}
|
|
51
|
-
const conn = await db.connect();
|
|
232
|
+
if (path && !selfHosted) throw new Error(`Cannot open DuckDB database "${path}": OPFS needs self-hosted \`bundles\` (a same-origin worker)`);
|
|
233
|
+
if (path && (typeof navigator === "undefined" || !navigator.storage)) throw new Error(`Cannot open DuckDB database "${path}": OPFS is unavailable`);
|
|
234
|
+
const { worker, db, conn } = await startDuckDb(await duckdb.selectBundle(selfHosted ?? duckdb.getJsDelivrBundles()), {
|
|
235
|
+
selfHosted: selfHosted !== void 0,
|
|
236
|
+
path
|
|
237
|
+
}, browserStartEnv);
|
|
52
238
|
return {
|
|
53
239
|
async query(sql, params) {
|
|
54
240
|
if (params && params.length > 0) {
|
|
241
|
+
const bound = toDuckDbParams(params);
|
|
55
242
|
const stmt = await conn.prepare(sql);
|
|
56
243
|
try {
|
|
57
|
-
return
|
|
244
|
+
return arrowTableToObjects(await stmt.query(...bound));
|
|
58
245
|
} finally {
|
|
59
246
|
await stmt.close();
|
|
60
247
|
}
|
|
61
248
|
}
|
|
62
|
-
return
|
|
249
|
+
return arrowTableToObjects(await conn.query(sql));
|
|
63
250
|
},
|
|
64
251
|
async exec(sql) {
|
|
65
252
|
await conn.query(sql);
|
package/dist/index.mjs.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.mjs","names":[],"sources":["../src/browser-duckdb.ts"],"sourcesContent":["import * as duckdb from \"@duckdb/duckdb-wasm\";\nimport type { Db, DbEntry, DbOptions } from \"@statewalker/db-api\";\n\ntype Table = Awaited<ReturnType<duckdb.AsyncPreparedStatement[\"query\"]>>;\n\n/**\n * Convert an Arrow Table (returned by duckdb-wasm queries) to plain JS objects.\n */\nfunction arrowToObjects<T>(table: Table): T[] {\n const fields = table.schema.fields.map((f: { name: string }) => f.name);\n return table.toArray().map((row: Record<string, unknown>) => {\n const obj: Record<string, unknown> = {};\n for (const field of fields) {\n obj[field] = row[field];\n }\n return obj as T;\n });\n}\n\n/** Browser-specific options for {@link newBrowserDuckDb}. */\nexport interface BrowserDuckDbOptions extends DbOptions {\n /**\n * Self-hosted DuckDB bundle URLs (served same-origin, e.g. via a bundler's\n * `?url` import). Passing these makes the factory spawn a **same-origin**\n * worker, which OPFS database persistence requires — a cross-origin worker\n * (the jsDelivr fallback, loaded via a Blob `importScripts` shim) crashes on\n * OPFS file I/O. Omit to load from jsDelivr (in-memory use only).\n */\n bundles?: duckdb.DuckDBBundles;\n}\n\n/**\n * Create a DuckDB-backed {@link Db} using WebAssembly in the browser.\n *\n * With `options.path` set to an `opfs://` URL **and** self-hosted\n * `options.bundles`, the database persists across reloads on OPFS. Without\n * `options.path` it runs in-memory. If `options.path` cannot be opened, the\n * returned promise rejects; it never falls back to an in-memory database.\n *\n * @param options.path `opfs://` path for persistent storage. Omit for in-memory.\n * @param options.bundles Same-origin bundle URLs (required for OPFS).\n */\nexport async function newBrowserDuckDb(options?: BrowserDuckDbOptions): Promise<Db> {\n const path = options?.path;\n if (path && (typeof navigator === \"undefined\" || !navigator.storage)) {\n throw new Error(`Cannot open DuckDB database \"${path}\": OPFS is unavailable`);\n }\n const selfHosted = options?.bundles;\n const bundle = await duckdb.selectBundle(selfHosted ?? duckdb.getJsDelivrBundles());\n\n // OPFS file I/O needs a same-origin worker. Self-hosted bundles give a\n // same-origin URL we load directly; the jsDelivr fallback is cross-origin and\n // must be wrapped in a Blob that `importScripts` it.\n let worker: Worker;\n let blobUrl: string | undefined;\n if (selfHosted) {\n worker = new Worker(bundle.mainWorker as string);\n } else {\n blobUrl = URL.createObjectURL(\n new Blob([`importScripts(\"${bundle.mainWorker ?? \"\"}\");`], {\n type: \"text/javascript\",\n }),\n );\n worker = new Worker(blobUrl);\n }\n\n const logger = new duckdb.ConsoleLogger();\n const db = new duckdb.AsyncDuckDB(logger, worker);\n await db.instantiate(bundle.mainModule, bundle.pthreadWorker);\n if (blobUrl) URL.revokeObjectURL(blobUrl);\n\n if (path) {\n try {\n await db.open({\n path,\n accessMode: duckdb.DuckDBAccessMode.READ_WRITE,\n });\n } catch (error) {\n await db.terminate();\n worker.terminate();\n throw new Error(`Cannot open DuckDB database \"${path}\"`, { cause: error });\n }\n }\n\n const conn = await db.connect();\n\n return {\n async query<T = DbEntry>(sql: string, params?: unknown[]): Promise<T[]> {\n if (params && params.length > 0) {\n const stmt = await conn.prepare(sql);\n try {\n const result = await stmt.query(...params);\n return arrowToObjects<T>(result);\n } finally {\n await stmt.close();\n }\n }\n const result = await conn.query(sql);\n return arrowToObjects<T>(result);\n },\n\n async exec(sql: string): Promise<void> {\n await conn.query(sql);\n },\n\n async flush(): Promise<void> {\n // CHECKPOINT writes the WAL into the OPFS database file so committed\n // changes survive a reload. Crucial for OPFS persistence.\n if (path) await conn.query(\"CHECKPOINT\");\n },\n\n async close(): Promise<void> {\n await conn.close();\n await db.terminate();\n worker.terminate();\n },\n };\n}\n"],"mappings":";;;;;AAQA,SAAS,eAAkB,OAAmB;CAC5C,MAAM,SAAS,MAAM,OAAO,OAAO,KAAK,MAAwB,EAAE,IAAI;CACtE,OAAO,MAAM,QAAQ,CAAC,CAAC,KAAK,QAAiC;EAC3D,MAAM,MAA+B,CAAC;EACtC,KAAK,MAAM,SAAS,QAClB,IAAI,SAAS,IAAI;EAEnB,OAAO;CACT,CAAC;AACH;;;;;;;;;;;;AAyBA,eAAsB,iBAAiB,SAA6C;CAClF,MAAM,OAAO,SAAS;CACtB,IAAI,SAAS,OAAO,cAAc,eAAe,CAAC,UAAU,UAC1D,MAAM,IAAI,MAAM,gCAAgC,KAAK,uBAAuB;CAE9E,MAAM,aAAa,SAAS;CAC5B,MAAM,SAAS,MAAM,OAAO,aAAa,cAAc,OAAO,mBAAmB,CAAC;CAKlF,IAAI;CACJ,IAAI;CACJ,IAAI,YACF,SAAS,IAAI,OAAO,OAAO,UAAoB;MAC1C;EACL,UAAU,IAAI,gBACZ,IAAI,KAAK,CAAC,kBAAkB,OAAO,cAAc,GAAG,IAAI,GAAG,EACzD,MAAM,kBACR,CAAC,CACH;EACA,SAAS,IAAI,OAAO,OAAO;CAC7B;CAEA,MAAM,SAAS,IAAI,OAAO,cAAc;CACxC,MAAM,KAAK,IAAI,OAAO,YAAY,QAAQ,MAAM;CAChD,MAAM,GAAG,YAAY,OAAO,YAAY,OAAO,aAAa;CAC5D,IAAI,SAAS,IAAI,gBAAgB,OAAO;CAExC,IAAI,MACF,IAAI;EACF,MAAM,GAAG,KAAK;GACZ;GACA,YAAY,OAAO,iBAAiB;EACtC,CAAC;CACH,SAAS,OAAO;EACd,MAAM,GAAG,UAAU;EACnB,OAAO,UAAU;EACjB,MAAM,IAAI,MAAM,gCAAgC,KAAK,IAAI,EAAE,OAAO,MAAM,CAAC;CAC3E;CAGF,MAAM,OAAO,MAAM,GAAG,QAAQ;CAE9B,OAAO;EACL,MAAM,MAAmB,KAAa,QAAkC;GACtE,IAAI,UAAU,OAAO,SAAS,GAAG;IAC/B,MAAM,OAAO,MAAM,KAAK,QAAQ,GAAG;IACnC,IAAI;KAEF,OAAO,eAAkB,MADJ,KAAK,MAAM,GAAG,MAAM,CACV;IACjC,UAAU;KACR,MAAM,KAAK,MAAM;IACnB;GACF;GAEA,OAAO,eAAkB,MADJ,KAAK,MAAM,GAAG,CACJ;EACjC;EAEA,MAAM,KAAK,KAA4B;GACrC,MAAM,KAAK,MAAM,GAAG;EACtB;EAEA,MAAM,QAAuB;GAG3B,IAAI,MAAM,MAAM,KAAK,MAAM,YAAY;EACzC;EAEA,MAAM,QAAuB;GAC3B,MAAM,KAAK,MAAM;GACjB,MAAM,GAAG,UAAU;GACnB,OAAO,UAAU;EACnB;CACF;AACF"}
|
|
1
|
+
{"version":3,"file":"index.mjs","names":[],"sources":["../src/arrow-values.ts","../src/params.ts","../src/start-duckdb.ts","../src/browser-duckdb.ts"],"sourcesContent":["/**\n * Conversion of Arrow values (as duckdb-wasm returns them) into the plain JS\n * values `@statewalker/db-duckdb-node` returns for the same SQL types.\n *\n * Typed structurally so this module does not import `apache-arrow` at runtime.\n * Type ids are Arrow's `Type` enum (apache-arrow 17, the version duckdb-wasm\n * depends on).\n */\n\nconst TypeId = {\n Dictionary: -1,\n Decimal: 7,\n Date: 8,\n Timestamp: 10,\n List: 12,\n Struct: 13,\n FixedSizeList: 16,\n Map: 17,\n} as const;\n\n/** Arrow `TimeUnit`. */\nconst TimeUnit = { SECOND: 0, MILLISECOND: 1, MICROSECOND: 2, NANOSECOND: 3 } as const;\n\n/** The parts of an Arrow `Field` this module reads. */\nexport interface ArrowFieldLike {\n name: string;\n type: ArrowTypeLike;\n}\n\n/** The parts of an Arrow `DataType` this module reads. */\nexport interface ArrowTypeLike {\n typeId: number;\n scale?: number;\n precision?: number;\n unit?: number;\n children?: ArrowFieldLike[] | null;\n dictionary?: ArrowTypeLike;\n}\n\n/** The parts of an Arrow `Table` this module reads. */\nexport interface ArrowTableLike {\n schema: { fields: ArrowFieldLike[] };\n toArray(): ArrayLike<unknown>;\n}\n\n/**\n * Convert an Arrow table into plain row objects, converting each value by its\n * field's Arrow type ({@link arrowValueToJs}).\n */\nexport function arrowTableToObjects<T>(table: ArrowTableLike): T[] {\n const fields = table.schema.fields;\n return Array.from(table.toArray(), (row) => {\n const source = row as Record<string, unknown>;\n const obj: Record<string, unknown> = {};\n for (const field of fields) {\n obj[field.name] = arrowValueToJs(source[field.name], field.type);\n }\n return obj as T;\n });\n}\n\n/**\n * Convert one Arrow value into the plain JS value the node driver returns:\n *\n * - DECIMAL: a `number` scaled by the type's scale (HUGEINT, which DuckDB sends\n * as DECIMAL(38,0), becomes a `bigint`);\n * - DATE, TIMESTAMP: a `Date`;\n * - LIST, fixed-size ARRAY: an array, converted recursively;\n * - STRUCT: a plain object, converted recursively;\n * - MAP: an array of `{ key, value }`, converted recursively.\n *\n * Other values (numbers, `bigint` for BIGINT, strings, booleans, `Uint8Array`\n * for BLOB, `null`) are returned as they are.\n */\nexport function arrowValueToJs(value: unknown, type: ArrowTypeLike): unknown {\n if (value === null || value === undefined) return null;\n switch (type.typeId) {\n case TypeId.Decimal:\n return decimalToJs(value, type);\n case TypeId.Date:\n return typeof value === \"number\" ? new Date(value) : value;\n case TypeId.Timestamp:\n return timestampToDate(value, type.unit);\n case TypeId.List:\n case TypeId.FixedSizeList: {\n const childType = type.children?.[0]?.type;\n const items = Array.from(value as Iterable<unknown>);\n return childType ? items.map((item) => arrowValueToJs(item, childType)) : items;\n }\n case TypeId.Struct: {\n const row = value as Record<string, unknown>;\n const obj: Record<string, unknown> = {};\n for (const child of type.children ?? []) {\n obj[child.name] = arrowValueToJs(row[child.name], child.type);\n }\n return obj;\n }\n case TypeId.Map: {\n const entryFields = type.children?.[0]?.type.children ?? [];\n const keyType = entryFields[0]?.type;\n const valueType = entryFields[1]?.type;\n return Array.from(value as Iterable<[unknown, unknown]>, ([key, val]) => ({\n key: keyType ? arrowValueToJs(key, keyType) : key,\n value: valueType ? arrowValueToJs(val, valueType) : val,\n }));\n }\n case TypeId.Dictionary:\n return type.dictionary ? arrowValueToJs(value, type.dictionary) : value;\n default:\n return value;\n }\n}\n\n/**\n * Arrow decimals are unscaled integers (`String(v)` gives the digits). Insert\n * the decimal point by text and parse, so the result is the nearest double to\n * the exact decimal value.\n */\nfunction decimalToJs(value: unknown, type: ArrowTypeLike): number | bigint {\n const digits = String(value);\n const scale = type.scale ?? 0;\n if (scale === 0) {\n // DuckDB sends HUGEINT as DECIMAL(38,0); the node driver returns bigint.\n return type.precision === 38 ? BigInt(digits) : Number(digits);\n }\n const negative = digits.startsWith(\"-\");\n const abs = (negative ? digits.slice(1) : digits).padStart(scale + 1, \"0\");\n const text = `${abs.slice(0, abs.length - scale)}.${abs.slice(abs.length - scale)}`;\n return Number(negative ? `-${text}` : text);\n}\n\nfunction timestampToDate(value: unknown, unit: number | undefined): unknown {\n if (typeof value === \"number\") return new Date(value); // apache-arrow gives epoch ms\n if (typeof value === \"bigint\") {\n switch (unit) {\n case TimeUnit.SECOND:\n return new Date(Number(value) * 1000);\n case TimeUnit.MICROSECOND:\n return new Date(Number(value / 1000n));\n case TimeUnit.NANOSECOND:\n return new Date(Number(value / 1000000n));\n default:\n return new Date(Number(value));\n }\n }\n return value;\n}\n","/** A value duckdb-wasm's prepared statements can bind (they JSON-encode params). */\nexport type DuckDbWasmParam = string | number | boolean | null;\n\n/**\n * Convert `Db.query` params into values duckdb-wasm can bind.\n *\n * duckdb-wasm passes prepared-statement params through `JSON.stringify`, so\n * only strings, finite numbers, booleans and `null` survive. This maps:\n *\n * - `bigint` to its exact decimal string; DuckDB casts it to the column type\n * where the context fixes one (`WHERE id = $1`, `INSERT ... VALUES ($1)`),\n * elsewhere cast it in SQL (`$1::BIGINT`);\n * - `Date` to its ISO 8601 string, which DuckDB casts to TIMESTAMP the same way;\n * - `undefined` to `null`.\n *\n * Anything else (`Uint8Array` and other binary data, arrays, plain objects,\n * `NaN`, `Infinity`, symbols, functions, invalid dates) throws a `TypeError`\n * naming the parameter and its type instead of failing inside duckdb-wasm or\n * binding the wrong value.\n */\nexport function toDuckDbParams(params: readonly unknown[]): DuckDbWasmParam[] {\n return params.map(toDuckDbParam);\n}\n\nfunction toDuckDbParam(value: unknown, index: number): DuckDbWasmParam {\n switch (typeof value) {\n case \"string\":\n case \"boolean\":\n return value;\n case \"undefined\":\n return null;\n case \"bigint\":\n return value.toString();\n case \"number\":\n if (Number.isFinite(value)) return value;\n throw unsupported(index, String(value));\n case \"object\": {\n if (value === null) return null;\n if (value instanceof Date) {\n if (Number.isNaN(value.getTime())) throw unsupported(index, \"invalid Date\");\n return value.toISOString();\n }\n if (Array.isArray(value)) throw unsupported(index, \"array\");\n if (ArrayBuffer.isView(value) || value instanceof ArrayBuffer) {\n throw unsupported(index, value.constructor.name);\n }\n throw unsupported(index, `object (${value.constructor?.name ?? \"null prototype\"})`);\n }\n default:\n throw unsupported(index, typeof value);\n }\n}\n\nfunction unsupported(index: number, type: string): TypeError {\n return new TypeError(\n `db-duckdb-browser: cannot bind parameter $${index + 1} of type ${type}; ` +\n \"duckdb-wasm binds only string, finite number, boolean, null, bigint and Date\",\n );\n}\n","import * as duckdb from \"@duckdb/duckdb-wasm\";\n\n/**\n * What {@link startDuckDb} needs from the page. The browser defaults are\n * {@link browserStartEnv}; tests pass fakes to drive failures.\n */\nexport interface DuckDbStartEnv {\n newWorker(url: string): Worker;\n newDb(worker: Worker): duckdb.AsyncDuckDB;\n createObjectURL(blob: Blob): string;\n revokeObjectURL(url: string): void;\n}\n\n/** A started engine: its worker, the engine and one open connection. */\nexport interface StartedDuckDb {\n worker: Worker;\n db: duckdb.AsyncDuckDB;\n conn: duckdb.AsyncDuckDBConnection;\n}\n\n/** The real browser environment: a Web Worker and the page's `URL`. */\nexport const browserStartEnv: DuckDbStartEnv = {\n newWorker: (url) => new Worker(url),\n newDb: (worker) => new duckdb.AsyncDuckDB(new duckdb.ConsoleLogger(), worker),\n createObjectURL: (blob) => URL.createObjectURL(blob),\n revokeObjectURL: (url) => URL.revokeObjectURL(url),\n};\n\n/**\n * Start a worker, instantiate DuckDB in it, open `path` if given, and connect.\n *\n * If any step fails, including the worker script failing to load, the engine\n * and its worker are terminated before the error is rethrown, so a failed start leaves no worker running. The Blob URL\n * of the jsDelivr shim is revoked whether the start succeeds or fails.\n */\nexport async function startDuckDb(\n bundle: duckdb.DuckDBBundle,\n options: { selfHosted: boolean; path?: string },\n env: DuckDbStartEnv,\n): Promise<StartedDuckDb> {\n const { selfHosted, path } = options;\n let blobUrl: string | undefined;\n let worker: Worker | undefined;\n let db: duckdb.AsyncDuckDB | undefined;\n // duckdb-wasm only logs a worker error (a script that fails to load) and drops the pending\n // request, so `instantiate` would never settle: turn the error event into a rejection.\n let onWorkerError: ((event: ErrorEvent) => void) | undefined;\n const workerFailed = new Promise<never>((_, reject) => {\n onWorkerError = (event) =>\n reject(new Error(`DuckDB worker failed to load: ${event.message ?? \"unknown error\"}`));\n });\n workerFailed.catch(() => {}); // only observed through the race below\n try {\n // OPFS file I/O needs a same-origin worker. Self-hosted bundles give a\n // same-origin URL we load directly; the jsDelivr fallback is cross-origin\n // and must be wrapped in a Blob that `importScripts` it.\n if (selfHosted) {\n worker = env.newWorker(bundle.mainWorker as string);\n } else {\n blobUrl = env.createObjectURL(\n new Blob([`importScripts(\"${bundle.mainWorker ?? \"\"}\");`], {\n type: \"text/javascript\",\n }),\n );\n worker = env.newWorker(blobUrl);\n }\n worker.addEventListener(\"error\", onWorkerError as (event: ErrorEvent) => void);\n db = env.newDb(worker);\n await Promise.race([db.instantiate(bundle.mainModule, bundle.pthreadWorker), workerFailed]);\n if (path) {\n try {\n await db.open({ path, accessMode: duckdb.DuckDBAccessMode.READ_WRITE });\n } catch (error) {\n throw new Error(`Cannot open DuckDB database \"${path}\"`, { cause: error });\n }\n }\n const conn = await db.connect();\n return { worker, db, conn };\n } catch (error) {\n // Best effort: a failing teardown must not hide the start error.\n try {\n await db?.terminate();\n } catch {}\n worker?.terminate();\n throw error;\n } finally {\n worker?.removeEventListener(\"error\", onWorkerError as (event: ErrorEvent) => void);\n if (blobUrl) env.revokeObjectURL(blobUrl);\n }\n}\n","import * as duckdb from \"@duckdb/duckdb-wasm\";\nimport type { Db, DbEntry, DbOptions } from \"@statewalker/db-api\";\nimport { arrowTableToObjects } from \"./arrow-values.js\";\nimport { toDuckDbParams } from \"./params.js\";\nimport { browserStartEnv, startDuckDb } from \"./start-duckdb.js\";\n\n/** Browser-specific options for {@link newBrowserDuckDb}. */\nexport interface BrowserDuckDbOptions extends DbOptions {\n /**\n * Self-hosted DuckDB bundle URLs (served same-origin, e.g. via a bundler's\n * `?url` import). Passing these makes the factory spawn a **same-origin**\n * worker, which OPFS database persistence requires — a cross-origin worker\n * (the jsDelivr fallback, loaded via a Blob `importScripts` shim) crashes on\n * OPFS file I/O. Omit to load from jsDelivr (in-memory use only); a `path`\n * without `bundles` is rejected.\n */\n bundles?: duckdb.DuckDBBundles;\n}\n\n/**\n * Create a DuckDB-backed {@link Db} using WebAssembly in the browser.\n *\n * With `options.path` set to an `opfs://` URL **and** self-hosted\n * `options.bundles`, the database persists across reloads on OPFS. Without\n * `options.path` it runs in-memory. If `options.path` cannot be opened, the\n * returned promise rejects; it never falls back to an in-memory database.\n * `options.path` without `options.bundles` rejects before any worker starts.\n *\n * @param options.path `opfs://` path for persistent storage. Omit for in-memory.\n * @param options.bundles Same-origin bundle URLs (required for OPFS).\n */\nexport async function newBrowserDuckDb(options?: BrowserDuckDbOptions): Promise<Db> {\n const path = options?.path;\n const selfHosted = options?.bundles;\n if (path && !selfHosted) {\n // The jsDelivr worker is cross-origin and crashes on OPFS file I/O.\n throw new Error(\n `Cannot open DuckDB database \"${path}\": OPFS needs self-hosted \\`bundles\\` (a same-origin worker)`,\n );\n }\n if (path && (typeof navigator === \"undefined\" || !navigator.storage)) {\n throw new Error(`Cannot open DuckDB database \"${path}\": OPFS is unavailable`);\n }\n const bundle = await duckdb.selectBundle(selfHosted ?? duckdb.getJsDelivrBundles());\n const { worker, db, conn } = await startDuckDb(\n bundle,\n { selfHosted: selfHosted !== undefined, path },\n browserStartEnv,\n );\n\n return {\n async query<T = DbEntry>(sql: string, params?: unknown[]): Promise<T[]> {\n if (params && params.length > 0) {\n // Convert first: an unsupported param throws before a statement exists.\n const bound = toDuckDbParams(params);\n const stmt = await conn.prepare(sql);\n try {\n const result = await stmt.query(...bound);\n return arrowTableToObjects<T>(result);\n } finally {\n await stmt.close();\n }\n }\n const result = await conn.query(sql);\n return arrowTableToObjects<T>(result);\n },\n\n async exec(sql: string): Promise<void> {\n await conn.query(sql);\n },\n\n async flush(): Promise<void> {\n // CHECKPOINT writes the WAL into the OPFS database file so committed\n // changes survive a reload. Crucial for OPFS persistence.\n if (path) await conn.query(\"CHECKPOINT\");\n },\n\n async close(): Promise<void> {\n await conn.close();\n await db.terminate();\n worker.terminate();\n },\n };\n}\n"],"mappings":";;;;;;;;;;AASA,MAAM,SAAS;CACb,YAAY;CACZ,SAAS;CACT,MAAM;CACN,WAAW;CACX,MAAM;CACN,QAAQ;CACR,eAAe;CACf,KAAK;AACP;;AAGA,MAAM,WAAW;CAAE,QAAQ;CAAG,aAAa;CAAG,aAAa;CAAG,YAAY;AAAE;;;;;AA4B5E,SAAgB,oBAAuB,OAA4B;CACjE,MAAM,SAAS,MAAM,OAAO;CAC5B,OAAO,MAAM,KAAK,MAAM,QAAQ,IAAI,QAAQ;EAC1C,MAAM,SAAS;EACf,MAAM,MAA+B,CAAC;EACtC,KAAK,MAAM,SAAS,QAClB,IAAI,MAAM,QAAQ,eAAe,OAAO,MAAM,OAAO,MAAM,IAAI;EAEjE,OAAO;CACT,CAAC;AACH;;;;;;;;;;;;;;AAeA,SAAgB,eAAe,OAAgB,MAA8B;CAC3E,IAAI,UAAU,QAAQ,UAAU,KAAA,GAAW,OAAO;CAClD,QAAQ,KAAK,QAAb;EACE,KAAK,OAAO,SACV,OAAO,YAAY,OAAO,IAAI;EAChC,KAAK,OAAO,MACV,OAAO,OAAO,UAAU,WAAW,IAAI,KAAK,KAAK,IAAI;EACvD,KAAK,OAAO,WACV,OAAO,gBAAgB,OAAO,KAAK,IAAI;EACzC,KAAK,OAAO;EACZ,KAAK,OAAO,eAAe;GACzB,MAAM,YAAY,KAAK,WAAW,EAAE,EAAE;GACtC,MAAM,QAAQ,MAAM,KAAK,KAA0B;GACnD,OAAO,YAAY,MAAM,KAAK,SAAS,eAAe,MAAM,SAAS,CAAC,IAAI;EAC5E;EACA,KAAK,OAAO,QAAQ;GAClB,MAAM,MAAM;GACZ,MAAM,MAA+B,CAAC;GACtC,KAAK,MAAM,SAAS,KAAK,YAAY,CAAC,GACpC,IAAI,MAAM,QAAQ,eAAe,IAAI,MAAM,OAAO,MAAM,IAAI;GAE9D,OAAO;EACT;EACA,KAAK,OAAO,KAAK;GACf,MAAM,cAAc,KAAK,WAAW,EAAE,EAAE,KAAK,YAAY,CAAC;GAC1D,MAAM,UAAU,YAAY,EAAE,EAAE;GAChC,MAAM,YAAY,YAAY,EAAE,EAAE;GAClC,OAAO,MAAM,KAAK,QAAwC,CAAC,KAAK,UAAU;IACxE,KAAK,UAAU,eAAe,KAAK,OAAO,IAAI;IAC9C,OAAO,YAAY,eAAe,KAAK,SAAS,IAAI;GACtD,EAAE;EACJ;EACA,KAAK,OAAO,YACV,OAAO,KAAK,aAAa,eAAe,OAAO,KAAK,UAAU,IAAI;EACpE,SACE,OAAO;CACX;AACF;;;;;;AAOA,SAAS,YAAY,OAAgB,MAAsC;CACzE,MAAM,SAAS,OAAO,KAAK;CAC3B,MAAM,QAAQ,KAAK,SAAS;CAC5B,IAAI,UAAU,GAEZ,OAAO,KAAK,cAAc,KAAK,OAAO,MAAM,IAAI,OAAO,MAAM;CAE/D,MAAM,WAAW,OAAO,WAAW,GAAG;CACtC,MAAM,OAAO,WAAW,OAAO,MAAM,CAAC,IAAI,OAAA,CAAQ,SAAS,QAAQ,GAAG,GAAG;CACzE,MAAM,OAAO,GAAG,IAAI,MAAM,GAAG,IAAI,SAAS,KAAK,EAAE,GAAG,IAAI,MAAM,IAAI,SAAS,KAAK;CAChF,OAAO,OAAO,WAAW,IAAI,SAAS,IAAI;AAC5C;AAEA,SAAS,gBAAgB,OAAgB,MAAmC;CAC1E,IAAI,OAAO,UAAU,UAAU,OAAO,IAAI,KAAK,KAAK;CACpD,IAAI,OAAO,UAAU,UACnB,QAAQ,MAAR;EACE,KAAK,SAAS,QACZ,uBAAO,IAAI,KAAK,OAAO,KAAK,IAAI,GAAI;EACtC,KAAK,SAAS,aACZ,OAAO,IAAI,KAAK,OAAO,QAAQ,KAAK,CAAC;EACvC,KAAK,SAAS,YACZ,OAAO,IAAI,KAAK,OAAO,QAAQ,QAAQ,CAAC;EAC1C,SACE,OAAO,IAAI,KAAK,OAAO,KAAK,CAAC;CACjC;CAEF,OAAO;AACT;;;;;;;;;;;;;;;;;;;;AC9HA,SAAgB,eAAe,QAA+C;CAC5E,OAAO,OAAO,IAAI,aAAa;AACjC;AAEA,SAAS,cAAc,OAAgB,OAAgC;CACrE,QAAQ,OAAO,OAAf;EACE,KAAK;EACL,KAAK,WACH,OAAO;EACT,KAAK,aACH,OAAO;EACT,KAAK,UACH,OAAO,MAAM,SAAS;EACxB,KAAK;GACH,IAAI,OAAO,SAAS,KAAK,GAAG,OAAO;GACnC,MAAM,YAAY,OAAO,OAAO,KAAK,CAAC;EACxC,KAAK;GACH,IAAI,UAAU,MAAM,OAAO;GAC3B,IAAI,iBAAiB,MAAM;IACzB,IAAI,OAAO,MAAM,MAAM,QAAQ,CAAC,GAAG,MAAM,YAAY,OAAO,cAAc;IAC1E,OAAO,MAAM,YAAY;GAC3B;GACA,IAAI,MAAM,QAAQ,KAAK,GAAG,MAAM,YAAY,OAAO,OAAO;GAC1D,IAAI,YAAY,OAAO,KAAK,KAAK,iBAAiB,aAChD,MAAM,YAAY,OAAO,MAAM,YAAY,IAAI;GAEjD,MAAM,YAAY,OAAO,WAAW,MAAM,aAAa,QAAQ,iBAAiB,EAAE;EAEpF,SACE,MAAM,YAAY,OAAO,OAAO,KAAK;CACzC;AACF;AAEA,SAAS,YAAY,OAAe,MAAyB;CAC3D,uBAAO,IAAI,UACT,6CAA6C,QAAQ,EAAE,WAAW,KAAK,+EAEzE;AACF;;;;ACrCA,MAAa,kBAAkC;CAC7C,YAAY,QAAQ,IAAI,OAAO,GAAG;CAClC,QAAQ,WAAW,IAAI,OAAO,YAAY,IAAI,OAAO,cAAc,GAAG,MAAM;CAC5E,kBAAkB,SAAS,IAAI,gBAAgB,IAAI;CACnD,kBAAkB,QAAQ,IAAI,gBAAgB,GAAG;AACnD;;;;;;;;AASA,eAAsB,YACpB,QACA,SACA,KACwB;CACxB,MAAM,EAAE,YAAY,SAAS;CAC7B,IAAI;CACJ,IAAI;CACJ,IAAI;CAGJ,IAAI;CACJ,MAAM,eAAe,IAAI,SAAgB,GAAG,WAAW;EACrD,iBAAiB,UACf,uBAAO,IAAI,MAAM,iCAAiC,MAAM,WAAW,iBAAiB,CAAC;CACzF,CAAC;CACD,aAAa,YAAY,CAAC,CAAC;CAC3B,IAAI;EAIF,IAAI,YACF,SAAS,IAAI,UAAU,OAAO,UAAoB;OAC7C;GACL,UAAU,IAAI,gBACZ,IAAI,KAAK,CAAC,kBAAkB,OAAO,cAAc,GAAG,IAAI,GAAG,EACzD,MAAM,kBACR,CAAC,CACH;GACA,SAAS,IAAI,UAAU,OAAO;EAChC;EACA,OAAO,iBAAiB,SAAS,aAA4C;EAC7E,KAAK,IAAI,MAAM,MAAM;EACrB,MAAM,QAAQ,KAAK,CAAC,GAAG,YAAY,OAAO,YAAY,OAAO,aAAa,GAAG,YAAY,CAAC;EAC1F,IAAI,MACF,IAAI;GACF,MAAM,GAAG,KAAK;IAAE;IAAM,YAAY,OAAO,iBAAiB;GAAW,CAAC;EACxE,SAAS,OAAO;GACd,MAAM,IAAI,MAAM,gCAAgC,KAAK,IAAI,EAAE,OAAO,MAAM,CAAC;EAC3E;EAEF,MAAM,OAAO,MAAM,GAAG,QAAQ;EAC9B,OAAO;GAAE;GAAQ;GAAI;EAAK;CAC5B,SAAS,OAAO;EAEd,IAAI;GACF,MAAM,IAAI,UAAU;EACtB,QAAQ,CAAC;EACT,QAAQ,UAAU;EAClB,MAAM;CACR,UAAU;EACR,QAAQ,oBAAoB,SAAS,aAA4C;EACjF,IAAI,SAAS,IAAI,gBAAgB,OAAO;CAC1C;AACF;;;;;;;;;;;;;;;AC1DA,eAAsB,iBAAiB,SAA6C;CAClF,MAAM,OAAO,SAAS;CACtB,MAAM,aAAa,SAAS;CAC5B,IAAI,QAAQ,CAAC,YAEX,MAAM,IAAI,MACR,gCAAgC,KAAK,6DACvC;CAEF,IAAI,SAAS,OAAO,cAAc,eAAe,CAAC,UAAU,UAC1D,MAAM,IAAI,MAAM,gCAAgC,KAAK,uBAAuB;CAG9E,MAAM,EAAE,QAAQ,IAAI,SAAS,MAAM,YACjC,MAFmB,OAAO,aAAa,cAAc,OAAO,mBAAmB,CAAC,GAGhF;EAAE,YAAY,eAAe,KAAA;EAAW;CAAK,GAC7C,eACF;CAEA,OAAO;EACL,MAAM,MAAmB,KAAa,QAAkC;GACtE,IAAI,UAAU,OAAO,SAAS,GAAG;IAE/B,MAAM,QAAQ,eAAe,MAAM;IACnC,MAAM,OAAO,MAAM,KAAK,QAAQ,GAAG;IACnC,IAAI;KAEF,OAAO,oBAAuB,MADT,KAAK,MAAM,GAAG,KAAK,CACJ;IACtC,UAAU;KACR,MAAM,KAAK,MAAM;IACnB;GACF;GAEA,OAAO,oBAAuB,MADT,KAAK,MAAM,GAAG,CACC;EACtC;EAEA,MAAM,KAAK,KAA4B;GACrC,MAAM,KAAK,MAAM,GAAG;EACtB;EAEA,MAAM,QAAuB;GAG3B,IAAI,MAAM,MAAM,KAAK,MAAM,YAAY;EACzC;EAEA,MAAM,QAAuB;GAC3B,MAAM,KAAK,MAAM;GACjB,MAAM,GAAG,UAAU;GACnB,OAAO,UAAU;EACnB;CACF;AACF"}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@statewalker/db-duckdb-browser",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.7",
|
|
4
4
|
"private": false,
|
|
5
5
|
"type": "module",
|
|
6
6
|
"description": "DuckDB WASM driver for @statewalker/db-api, targeting browser environments.",
|
|
@@ -22,16 +22,19 @@
|
|
|
22
22
|
".": {
|
|
23
23
|
"source": "./src/index.ts",
|
|
24
24
|
"types": "./dist/index.d.mts",
|
|
25
|
-
"import": "./dist/index.mjs"
|
|
25
|
+
"import": "./dist/index.mjs",
|
|
26
|
+
"default": "./dist/index.mjs"
|
|
26
27
|
}
|
|
27
28
|
},
|
|
28
29
|
"files": [
|
|
29
30
|
"dist",
|
|
30
|
-
"src"
|
|
31
|
+
"src",
|
|
32
|
+
"!src/**/*.test.ts",
|
|
33
|
+
"!src/**/test-wasm-node.ts"
|
|
31
34
|
],
|
|
32
35
|
"dependencies": {
|
|
33
36
|
"@duckdb/duckdb-wasm": "^1.33.1-dev57.0",
|
|
34
|
-
"@statewalker/db-api": "^0.1.
|
|
37
|
+
"@statewalker/db-api": "^0.1.6"
|
|
35
38
|
},
|
|
36
39
|
"devDependencies": {
|
|
37
40
|
"@playwright/test": "^1.63.0",
|
|
@@ -0,0 +1,147 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Conversion of Arrow values (as duckdb-wasm returns them) into the plain JS
|
|
3
|
+
* values `@statewalker/db-duckdb-node` returns for the same SQL types.
|
|
4
|
+
*
|
|
5
|
+
* Typed structurally so this module does not import `apache-arrow` at runtime.
|
|
6
|
+
* Type ids are Arrow's `Type` enum (apache-arrow 17, the version duckdb-wasm
|
|
7
|
+
* depends on).
|
|
8
|
+
*/
|
|
9
|
+
|
|
10
|
+
const TypeId = {
|
|
11
|
+
Dictionary: -1,
|
|
12
|
+
Decimal: 7,
|
|
13
|
+
Date: 8,
|
|
14
|
+
Timestamp: 10,
|
|
15
|
+
List: 12,
|
|
16
|
+
Struct: 13,
|
|
17
|
+
FixedSizeList: 16,
|
|
18
|
+
Map: 17,
|
|
19
|
+
} as const;
|
|
20
|
+
|
|
21
|
+
/** Arrow `TimeUnit`. */
|
|
22
|
+
const TimeUnit = { SECOND: 0, MILLISECOND: 1, MICROSECOND: 2, NANOSECOND: 3 } as const;
|
|
23
|
+
|
|
24
|
+
/** The parts of an Arrow `Field` this module reads. */
|
|
25
|
+
export interface ArrowFieldLike {
|
|
26
|
+
name: string;
|
|
27
|
+
type: ArrowTypeLike;
|
|
28
|
+
}
|
|
29
|
+
|
|
30
|
+
/** The parts of an Arrow `DataType` this module reads. */
|
|
31
|
+
export interface ArrowTypeLike {
|
|
32
|
+
typeId: number;
|
|
33
|
+
scale?: number;
|
|
34
|
+
precision?: number;
|
|
35
|
+
unit?: number;
|
|
36
|
+
children?: ArrowFieldLike[] | null;
|
|
37
|
+
dictionary?: ArrowTypeLike;
|
|
38
|
+
}
|
|
39
|
+
|
|
40
|
+
/** The parts of an Arrow `Table` this module reads. */
|
|
41
|
+
export interface ArrowTableLike {
|
|
42
|
+
schema: { fields: ArrowFieldLike[] };
|
|
43
|
+
toArray(): ArrayLike<unknown>;
|
|
44
|
+
}
|
|
45
|
+
|
|
46
|
+
/**
|
|
47
|
+
* Convert an Arrow table into plain row objects, converting each value by its
|
|
48
|
+
* field's Arrow type ({@link arrowValueToJs}).
|
|
49
|
+
*/
|
|
50
|
+
export function arrowTableToObjects<T>(table: ArrowTableLike): T[] {
|
|
51
|
+
const fields = table.schema.fields;
|
|
52
|
+
return Array.from(table.toArray(), (row) => {
|
|
53
|
+
const source = row as Record<string, unknown>;
|
|
54
|
+
const obj: Record<string, unknown> = {};
|
|
55
|
+
for (const field of fields) {
|
|
56
|
+
obj[field.name] = arrowValueToJs(source[field.name], field.type);
|
|
57
|
+
}
|
|
58
|
+
return obj as T;
|
|
59
|
+
});
|
|
60
|
+
}
|
|
61
|
+
|
|
62
|
+
/**
|
|
63
|
+
* Convert one Arrow value into the plain JS value the node driver returns:
|
|
64
|
+
*
|
|
65
|
+
* - DECIMAL: a `number` scaled by the type's scale (HUGEINT, which DuckDB sends
|
|
66
|
+
* as DECIMAL(38,0), becomes a `bigint`);
|
|
67
|
+
* - DATE, TIMESTAMP: a `Date`;
|
|
68
|
+
* - LIST, fixed-size ARRAY: an array, converted recursively;
|
|
69
|
+
* - STRUCT: a plain object, converted recursively;
|
|
70
|
+
* - MAP: an array of `{ key, value }`, converted recursively.
|
|
71
|
+
*
|
|
72
|
+
* Other values (numbers, `bigint` for BIGINT, strings, booleans, `Uint8Array`
|
|
73
|
+
* for BLOB, `null`) are returned as they are.
|
|
74
|
+
*/
|
|
75
|
+
export function arrowValueToJs(value: unknown, type: ArrowTypeLike): unknown {
|
|
76
|
+
if (value === null || value === undefined) return null;
|
|
77
|
+
switch (type.typeId) {
|
|
78
|
+
case TypeId.Decimal:
|
|
79
|
+
return decimalToJs(value, type);
|
|
80
|
+
case TypeId.Date:
|
|
81
|
+
return typeof value === "number" ? new Date(value) : value;
|
|
82
|
+
case TypeId.Timestamp:
|
|
83
|
+
return timestampToDate(value, type.unit);
|
|
84
|
+
case TypeId.List:
|
|
85
|
+
case TypeId.FixedSizeList: {
|
|
86
|
+
const childType = type.children?.[0]?.type;
|
|
87
|
+
const items = Array.from(value as Iterable<unknown>);
|
|
88
|
+
return childType ? items.map((item) => arrowValueToJs(item, childType)) : items;
|
|
89
|
+
}
|
|
90
|
+
case TypeId.Struct: {
|
|
91
|
+
const row = value as Record<string, unknown>;
|
|
92
|
+
const obj: Record<string, unknown> = {};
|
|
93
|
+
for (const child of type.children ?? []) {
|
|
94
|
+
obj[child.name] = arrowValueToJs(row[child.name], child.type);
|
|
95
|
+
}
|
|
96
|
+
return obj;
|
|
97
|
+
}
|
|
98
|
+
case TypeId.Map: {
|
|
99
|
+
const entryFields = type.children?.[0]?.type.children ?? [];
|
|
100
|
+
const keyType = entryFields[0]?.type;
|
|
101
|
+
const valueType = entryFields[1]?.type;
|
|
102
|
+
return Array.from(value as Iterable<[unknown, unknown]>, ([key, val]) => ({
|
|
103
|
+
key: keyType ? arrowValueToJs(key, keyType) : key,
|
|
104
|
+
value: valueType ? arrowValueToJs(val, valueType) : val,
|
|
105
|
+
}));
|
|
106
|
+
}
|
|
107
|
+
case TypeId.Dictionary:
|
|
108
|
+
return type.dictionary ? arrowValueToJs(value, type.dictionary) : value;
|
|
109
|
+
default:
|
|
110
|
+
return value;
|
|
111
|
+
}
|
|
112
|
+
}
|
|
113
|
+
|
|
114
|
+
/**
|
|
115
|
+
* Arrow decimals are unscaled integers (`String(v)` gives the digits). Insert
|
|
116
|
+
* the decimal point by text and parse, so the result is the nearest double to
|
|
117
|
+
* the exact decimal value.
|
|
118
|
+
*/
|
|
119
|
+
function decimalToJs(value: unknown, type: ArrowTypeLike): number | bigint {
|
|
120
|
+
const digits = String(value);
|
|
121
|
+
const scale = type.scale ?? 0;
|
|
122
|
+
if (scale === 0) {
|
|
123
|
+
// DuckDB sends HUGEINT as DECIMAL(38,0); the node driver returns bigint.
|
|
124
|
+
return type.precision === 38 ? BigInt(digits) : Number(digits);
|
|
125
|
+
}
|
|
126
|
+
const negative = digits.startsWith("-");
|
|
127
|
+
const abs = (negative ? digits.slice(1) : digits).padStart(scale + 1, "0");
|
|
128
|
+
const text = `${abs.slice(0, abs.length - scale)}.${abs.slice(abs.length - scale)}`;
|
|
129
|
+
return Number(negative ? `-${text}` : text);
|
|
130
|
+
}
|
|
131
|
+
|
|
132
|
+
function timestampToDate(value: unknown, unit: number | undefined): unknown {
|
|
133
|
+
if (typeof value === "number") return new Date(value); // apache-arrow gives epoch ms
|
|
134
|
+
if (typeof value === "bigint") {
|
|
135
|
+
switch (unit) {
|
|
136
|
+
case TimeUnit.SECOND:
|
|
137
|
+
return new Date(Number(value) * 1000);
|
|
138
|
+
case TimeUnit.MICROSECOND:
|
|
139
|
+
return new Date(Number(value / 1000n));
|
|
140
|
+
case TimeUnit.NANOSECOND:
|
|
141
|
+
return new Date(Number(value / 1000000n));
|
|
142
|
+
default:
|
|
143
|
+
return new Date(Number(value));
|
|
144
|
+
}
|
|
145
|
+
}
|
|
146
|
+
return value;
|
|
147
|
+
}
|
package/src/browser-duckdb.ts
CHANGED
|
@@ -1,21 +1,8 @@
|
|
|
1
1
|
import * as duckdb from "@duckdb/duckdb-wasm";
|
|
2
2
|
import type { Db, DbEntry, DbOptions } from "@statewalker/db-api";
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
/**
|
|
7
|
-
* Convert an Arrow Table (returned by duckdb-wasm queries) to plain JS objects.
|
|
8
|
-
*/
|
|
9
|
-
function arrowToObjects<T>(table: Table): T[] {
|
|
10
|
-
const fields = table.schema.fields.map((f: { name: string }) => f.name);
|
|
11
|
-
return table.toArray().map((row: Record<string, unknown>) => {
|
|
12
|
-
const obj: Record<string, unknown> = {};
|
|
13
|
-
for (const field of fields) {
|
|
14
|
-
obj[field] = row[field];
|
|
15
|
-
}
|
|
16
|
-
return obj as T;
|
|
17
|
-
});
|
|
18
|
-
}
|
|
3
|
+
import { arrowTableToObjects } from "./arrow-values.js";
|
|
4
|
+
import { toDuckDbParams } from "./params.js";
|
|
5
|
+
import { browserStartEnv, startDuckDb } from "./start-duckdb.js";
|
|
19
6
|
|
|
20
7
|
/** Browser-specific options for {@link newBrowserDuckDb}. */
|
|
21
8
|
export interface BrowserDuckDbOptions extends DbOptions {
|
|
@@ -24,7 +11,8 @@ export interface BrowserDuckDbOptions extends DbOptions {
|
|
|
24
11
|
* `?url` import). Passing these makes the factory spawn a **same-origin**
|
|
25
12
|
* worker, which OPFS database persistence requires — a cross-origin worker
|
|
26
13
|
* (the jsDelivr fallback, loaded via a Blob `importScripts` shim) crashes on
|
|
27
|
-
* OPFS file I/O. Omit to load from jsDelivr (in-memory use only)
|
|
14
|
+
* OPFS file I/O. Omit to load from jsDelivr (in-memory use only); a `path`
|
|
15
|
+
* without `bundles` is rejected.
|
|
28
16
|
*/
|
|
29
17
|
bundles?: duckdb.DuckDBBundles;
|
|
30
18
|
}
|
|
@@ -36,67 +24,45 @@ export interface BrowserDuckDbOptions extends DbOptions {
|
|
|
36
24
|
* `options.bundles`, the database persists across reloads on OPFS. Without
|
|
37
25
|
* `options.path` it runs in-memory. If `options.path` cannot be opened, the
|
|
38
26
|
* returned promise rejects; it never falls back to an in-memory database.
|
|
27
|
+
* `options.path` without `options.bundles` rejects before any worker starts.
|
|
39
28
|
*
|
|
40
29
|
* @param options.path `opfs://` path for persistent storage. Omit for in-memory.
|
|
41
30
|
* @param options.bundles Same-origin bundle URLs (required for OPFS).
|
|
42
31
|
*/
|
|
43
32
|
export async function newBrowserDuckDb(options?: BrowserDuckDbOptions): Promise<Db> {
|
|
44
33
|
const path = options?.path;
|
|
45
|
-
if (path && (typeof navigator === "undefined" || !navigator.storage)) {
|
|
46
|
-
throw new Error(`Cannot open DuckDB database "${path}": OPFS is unavailable`);
|
|
47
|
-
}
|
|
48
34
|
const selfHosted = options?.bundles;
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
// must be wrapped in a Blob that `importScripts` it.
|
|
54
|
-
let worker: Worker;
|
|
55
|
-
let blobUrl: string | undefined;
|
|
56
|
-
if (selfHosted) {
|
|
57
|
-
worker = new Worker(bundle.mainWorker as string);
|
|
58
|
-
} else {
|
|
59
|
-
blobUrl = URL.createObjectURL(
|
|
60
|
-
new Blob([`importScripts("${bundle.mainWorker ?? ""}");`], {
|
|
61
|
-
type: "text/javascript",
|
|
62
|
-
}),
|
|
35
|
+
if (path && !selfHosted) {
|
|
36
|
+
// The jsDelivr worker is cross-origin and crashes on OPFS file I/O.
|
|
37
|
+
throw new Error(
|
|
38
|
+
`Cannot open DuckDB database "${path}": OPFS needs self-hosted \`bundles\` (a same-origin worker)`,
|
|
63
39
|
);
|
|
64
|
-
worker = new Worker(blobUrl);
|
|
65
40
|
}
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
const db = new duckdb.AsyncDuckDB(logger, worker);
|
|
69
|
-
await db.instantiate(bundle.mainModule, bundle.pthreadWorker);
|
|
70
|
-
if (blobUrl) URL.revokeObjectURL(blobUrl);
|
|
71
|
-
|
|
72
|
-
if (path) {
|
|
73
|
-
try {
|
|
74
|
-
await db.open({
|
|
75
|
-
path,
|
|
76
|
-
accessMode: duckdb.DuckDBAccessMode.READ_WRITE,
|
|
77
|
-
});
|
|
78
|
-
} catch (error) {
|
|
79
|
-
await db.terminate();
|
|
80
|
-
worker.terminate();
|
|
81
|
-
throw new Error(`Cannot open DuckDB database "${path}"`, { cause: error });
|
|
82
|
-
}
|
|
41
|
+
if (path && (typeof navigator === "undefined" || !navigator.storage)) {
|
|
42
|
+
throw new Error(`Cannot open DuckDB database "${path}": OPFS is unavailable`);
|
|
83
43
|
}
|
|
84
|
-
|
|
85
|
-
const conn = await
|
|
44
|
+
const bundle = await duckdb.selectBundle(selfHosted ?? duckdb.getJsDelivrBundles());
|
|
45
|
+
const { worker, db, conn } = await startDuckDb(
|
|
46
|
+
bundle,
|
|
47
|
+
{ selfHosted: selfHosted !== undefined, path },
|
|
48
|
+
browserStartEnv,
|
|
49
|
+
);
|
|
86
50
|
|
|
87
51
|
return {
|
|
88
52
|
async query<T = DbEntry>(sql: string, params?: unknown[]): Promise<T[]> {
|
|
89
53
|
if (params && params.length > 0) {
|
|
54
|
+
// Convert first: an unsupported param throws before a statement exists.
|
|
55
|
+
const bound = toDuckDbParams(params);
|
|
90
56
|
const stmt = await conn.prepare(sql);
|
|
91
57
|
try {
|
|
92
|
-
const result = await stmt.query(...
|
|
93
|
-
return
|
|
58
|
+
const result = await stmt.query(...bound);
|
|
59
|
+
return arrowTableToObjects<T>(result);
|
|
94
60
|
} finally {
|
|
95
61
|
await stmt.close();
|
|
96
62
|
}
|
|
97
63
|
}
|
|
98
64
|
const result = await conn.query(sql);
|
|
99
|
-
return
|
|
65
|
+
return arrowTableToObjects<T>(result);
|
|
100
66
|
},
|
|
101
67
|
|
|
102
68
|
async exec(sql: string): Promise<void> {
|
package/src/params.ts
ADDED
|
@@ -0,0 +1,59 @@
|
|
|
1
|
+
/** A value duckdb-wasm's prepared statements can bind (they JSON-encode params). */
|
|
2
|
+
export type DuckDbWasmParam = string | number | boolean | null;
|
|
3
|
+
|
|
4
|
+
/**
|
|
5
|
+
* Convert `Db.query` params into values duckdb-wasm can bind.
|
|
6
|
+
*
|
|
7
|
+
* duckdb-wasm passes prepared-statement params through `JSON.stringify`, so
|
|
8
|
+
* only strings, finite numbers, booleans and `null` survive. This maps:
|
|
9
|
+
*
|
|
10
|
+
* - `bigint` to its exact decimal string; DuckDB casts it to the column type
|
|
11
|
+
* where the context fixes one (`WHERE id = $1`, `INSERT ... VALUES ($1)`),
|
|
12
|
+
* elsewhere cast it in SQL (`$1::BIGINT`);
|
|
13
|
+
* - `Date` to its ISO 8601 string, which DuckDB casts to TIMESTAMP the same way;
|
|
14
|
+
* - `undefined` to `null`.
|
|
15
|
+
*
|
|
16
|
+
* Anything else (`Uint8Array` and other binary data, arrays, plain objects,
|
|
17
|
+
* `NaN`, `Infinity`, symbols, functions, invalid dates) throws a `TypeError`
|
|
18
|
+
* naming the parameter and its type instead of failing inside duckdb-wasm or
|
|
19
|
+
* binding the wrong value.
|
|
20
|
+
*/
|
|
21
|
+
export function toDuckDbParams(params: readonly unknown[]): DuckDbWasmParam[] {
|
|
22
|
+
return params.map(toDuckDbParam);
|
|
23
|
+
}
|
|
24
|
+
|
|
25
|
+
function toDuckDbParam(value: unknown, index: number): DuckDbWasmParam {
|
|
26
|
+
switch (typeof value) {
|
|
27
|
+
case "string":
|
|
28
|
+
case "boolean":
|
|
29
|
+
return value;
|
|
30
|
+
case "undefined":
|
|
31
|
+
return null;
|
|
32
|
+
case "bigint":
|
|
33
|
+
return value.toString();
|
|
34
|
+
case "number":
|
|
35
|
+
if (Number.isFinite(value)) return value;
|
|
36
|
+
throw unsupported(index, String(value));
|
|
37
|
+
case "object": {
|
|
38
|
+
if (value === null) return null;
|
|
39
|
+
if (value instanceof Date) {
|
|
40
|
+
if (Number.isNaN(value.getTime())) throw unsupported(index, "invalid Date");
|
|
41
|
+
return value.toISOString();
|
|
42
|
+
}
|
|
43
|
+
if (Array.isArray(value)) throw unsupported(index, "array");
|
|
44
|
+
if (ArrayBuffer.isView(value) || value instanceof ArrayBuffer) {
|
|
45
|
+
throw unsupported(index, value.constructor.name);
|
|
46
|
+
}
|
|
47
|
+
throw unsupported(index, `object (${value.constructor?.name ?? "null prototype"})`);
|
|
48
|
+
}
|
|
49
|
+
default:
|
|
50
|
+
throw unsupported(index, typeof value);
|
|
51
|
+
}
|
|
52
|
+
}
|
|
53
|
+
|
|
54
|
+
function unsupported(index: number, type: string): TypeError {
|
|
55
|
+
return new TypeError(
|
|
56
|
+
`db-duckdb-browser: cannot bind parameter $${index + 1} of type ${type}; ` +
|
|
57
|
+
"duckdb-wasm binds only string, finite number, boolean, null, bigint and Date",
|
|
58
|
+
);
|
|
59
|
+
}
|
|
@@ -0,0 +1,90 @@
|
|
|
1
|
+
import * as duckdb from "@duckdb/duckdb-wasm";
|
|
2
|
+
|
|
3
|
+
/**
|
|
4
|
+
* What {@link startDuckDb} needs from the page. The browser defaults are
|
|
5
|
+
* {@link browserStartEnv}; tests pass fakes to drive failures.
|
|
6
|
+
*/
|
|
7
|
+
export interface DuckDbStartEnv {
|
|
8
|
+
newWorker(url: string): Worker;
|
|
9
|
+
newDb(worker: Worker): duckdb.AsyncDuckDB;
|
|
10
|
+
createObjectURL(blob: Blob): string;
|
|
11
|
+
revokeObjectURL(url: string): void;
|
|
12
|
+
}
|
|
13
|
+
|
|
14
|
+
/** A started engine: its worker, the engine and one open connection. */
|
|
15
|
+
export interface StartedDuckDb {
|
|
16
|
+
worker: Worker;
|
|
17
|
+
db: duckdb.AsyncDuckDB;
|
|
18
|
+
conn: duckdb.AsyncDuckDBConnection;
|
|
19
|
+
}
|
|
20
|
+
|
|
21
|
+
/** The real browser environment: a Web Worker and the page's `URL`. */
|
|
22
|
+
export const browserStartEnv: DuckDbStartEnv = {
|
|
23
|
+
newWorker: (url) => new Worker(url),
|
|
24
|
+
newDb: (worker) => new duckdb.AsyncDuckDB(new duckdb.ConsoleLogger(), worker),
|
|
25
|
+
createObjectURL: (blob) => URL.createObjectURL(blob),
|
|
26
|
+
revokeObjectURL: (url) => URL.revokeObjectURL(url),
|
|
27
|
+
};
|
|
28
|
+
|
|
29
|
+
/**
|
|
30
|
+
* Start a worker, instantiate DuckDB in it, open `path` if given, and connect.
|
|
31
|
+
*
|
|
32
|
+
* If any step fails, including the worker script failing to load, the engine
|
|
33
|
+
* and its worker are terminated before the error is rethrown, so a failed start leaves no worker running. The Blob URL
|
|
34
|
+
* of the jsDelivr shim is revoked whether the start succeeds or fails.
|
|
35
|
+
*/
|
|
36
|
+
export async function startDuckDb(
|
|
37
|
+
bundle: duckdb.DuckDBBundle,
|
|
38
|
+
options: { selfHosted: boolean; path?: string },
|
|
39
|
+
env: DuckDbStartEnv,
|
|
40
|
+
): Promise<StartedDuckDb> {
|
|
41
|
+
const { selfHosted, path } = options;
|
|
42
|
+
let blobUrl: string | undefined;
|
|
43
|
+
let worker: Worker | undefined;
|
|
44
|
+
let db: duckdb.AsyncDuckDB | undefined;
|
|
45
|
+
// duckdb-wasm only logs a worker error (a script that fails to load) and drops the pending
|
|
46
|
+
// request, so `instantiate` would never settle: turn the error event into a rejection.
|
|
47
|
+
let onWorkerError: ((event: ErrorEvent) => void) | undefined;
|
|
48
|
+
const workerFailed = new Promise<never>((_, reject) => {
|
|
49
|
+
onWorkerError = (event) =>
|
|
50
|
+
reject(new Error(`DuckDB worker failed to load: ${event.message ?? "unknown error"}`));
|
|
51
|
+
});
|
|
52
|
+
workerFailed.catch(() => {}); // only observed through the race below
|
|
53
|
+
try {
|
|
54
|
+
// OPFS file I/O needs a same-origin worker. Self-hosted bundles give a
|
|
55
|
+
// same-origin URL we load directly; the jsDelivr fallback is cross-origin
|
|
56
|
+
// and must be wrapped in a Blob that `importScripts` it.
|
|
57
|
+
if (selfHosted) {
|
|
58
|
+
worker = env.newWorker(bundle.mainWorker as string);
|
|
59
|
+
} else {
|
|
60
|
+
blobUrl = env.createObjectURL(
|
|
61
|
+
new Blob([`importScripts("${bundle.mainWorker ?? ""}");`], {
|
|
62
|
+
type: "text/javascript",
|
|
63
|
+
}),
|
|
64
|
+
);
|
|
65
|
+
worker = env.newWorker(blobUrl);
|
|
66
|
+
}
|
|
67
|
+
worker.addEventListener("error", onWorkerError as (event: ErrorEvent) => void);
|
|
68
|
+
db = env.newDb(worker);
|
|
69
|
+
await Promise.race([db.instantiate(bundle.mainModule, bundle.pthreadWorker), workerFailed]);
|
|
70
|
+
if (path) {
|
|
71
|
+
try {
|
|
72
|
+
await db.open({ path, accessMode: duckdb.DuckDBAccessMode.READ_WRITE });
|
|
73
|
+
} catch (error) {
|
|
74
|
+
throw new Error(`Cannot open DuckDB database "${path}"`, { cause: error });
|
|
75
|
+
}
|
|
76
|
+
}
|
|
77
|
+
const conn = await db.connect();
|
|
78
|
+
return { worker, db, conn };
|
|
79
|
+
} catch (error) {
|
|
80
|
+
// Best effort: a failing teardown must not hide the start error.
|
|
81
|
+
try {
|
|
82
|
+
await db?.terminate();
|
|
83
|
+
} catch {}
|
|
84
|
+
worker?.terminate();
|
|
85
|
+
throw error;
|
|
86
|
+
} finally {
|
|
87
|
+
worker?.removeEventListener("error", onWorkerError as (event: ErrorEvent) => void);
|
|
88
|
+
if (blobUrl) env.revokeObjectURL(blobUrl);
|
|
89
|
+
}
|
|
90
|
+
}
|
|
@@ -1,208 +0,0 @@
|
|
|
1
|
-
import * as duckdb from "@duckdb/duckdb-wasm";
|
|
2
|
-
import type { Db } from "@statewalker/db-api";
|
|
3
|
-
import { runDbConformance } from "@statewalker/db-tests";
|
|
4
|
-
import { afterEach, describe, expect, it, vi } from "vitest";
|
|
5
|
-
import { newBrowserDuckDb } from "./browser-duckdb.js";
|
|
6
|
-
|
|
7
|
-
/**
|
|
8
|
-
* The DuckDB-WASM browser adapter, exercised in a REAL browser (Vitest browser
|
|
9
|
-
* mode + Playwright/Chromium). Launched only by `vitest.browser.config.ts` (the
|
|
10
|
-
* `test:browser` script) — the Node config excludes `*.browser.test.ts`.
|
|
11
|
-
*
|
|
12
|
-
* With no `bundles`/`path`, the adapter loads the default DuckDB bundle from
|
|
13
|
-
* jsDelivr and runs in-memory (a Worker inside the browser page), so the run
|
|
14
|
-
* needs outbound network access from Chromium. The node-only file-persistence
|
|
15
|
-
* scenario is skipped via `skipFilePersistence`.
|
|
16
|
-
*/
|
|
17
|
-
|
|
18
|
-
// Shared contract: CRUD, parameter binding, injection safety, cardinality,
|
|
19
|
-
// close, flush, error paths. File persistence is node-only, hence skipped here.
|
|
20
|
-
runDbConformance(() => newBrowserDuckDb(), {
|
|
21
|
-
// duckdb dialect: `$1` placeholder; rows are already plain objects, so the
|
|
22
|
-
// spread normalizer is an identity-style copy.
|
|
23
|
-
placeholder: "$1",
|
|
24
|
-
normalizeRow: (row) => ({ ...row }),
|
|
25
|
-
skipFilePersistence: true,
|
|
26
|
-
});
|
|
27
|
-
|
|
28
|
-
// Dialect-specific browser behavior the shared suite does not cover: DuckDB's
|
|
29
|
-
// FTS extension and VSS/HNSW vector index.
|
|
30
|
-
describe("newBrowserDuckDb", () => {
|
|
31
|
-
let db: Db;
|
|
32
|
-
|
|
33
|
-
afterEach(async () => {
|
|
34
|
-
if (db) {
|
|
35
|
-
await db.close();
|
|
36
|
-
}
|
|
37
|
-
});
|
|
38
|
-
|
|
39
|
-
describe("FTS extension", () => {
|
|
40
|
-
it("returns empty results for unmatched search terms", async () => {
|
|
41
|
-
db = await newBrowserDuckDb();
|
|
42
|
-
await db.exec("INSTALL fts; LOAD fts;");
|
|
43
|
-
await db.exec("CREATE TABLE docs2 (id INTEGER, content VARCHAR)");
|
|
44
|
-
await db.exec("INSERT INTO docs2 VALUES (1, 'hello world'), (2, 'goodbye world')");
|
|
45
|
-
await db.exec("PRAGMA create_fts_index('docs2', 'id', 'content')");
|
|
46
|
-
|
|
47
|
-
const rows = await db.query<{ id: number; score: number | null }>(`
|
|
48
|
-
SELECT id, score FROM (
|
|
49
|
-
SELECT *, fts_main_docs2.match_bm25(id, 'nonexistent') AS score FROM docs2
|
|
50
|
-
) sq WHERE score IS NOT NULL
|
|
51
|
-
`);
|
|
52
|
-
expect(rows).toHaveLength(0);
|
|
53
|
-
});
|
|
54
|
-
|
|
55
|
-
it("searches across multiple text columns", async () => {
|
|
56
|
-
db = await newBrowserDuckDb();
|
|
57
|
-
await db.exec("INSTALL fts; LOAD fts;");
|
|
58
|
-
await db.exec("CREATE TABLE articles (id INTEGER, title VARCHAR, body VARCHAR)");
|
|
59
|
-
await db.exec(`
|
|
60
|
-
INSERT INTO articles VALUES
|
|
61
|
-
(1, 'Travel Guide', 'Visit the beautiful lakes of Switzerland'),
|
|
62
|
-
(2, 'Cooking Tips', 'How to make a perfect lake trout'),
|
|
63
|
-
(3, 'Programming', 'Learn about SQL databases')
|
|
64
|
-
`);
|
|
65
|
-
await db.exec("PRAGMA create_fts_index('articles', 'id', 'title', 'body')");
|
|
66
|
-
|
|
67
|
-
const rows = await db.query<{ id: number; score: number | null }>(`
|
|
68
|
-
SELECT id, score FROM (
|
|
69
|
-
SELECT *, fts_main_articles.match_bm25(id, 'lake') AS score FROM articles
|
|
70
|
-
) sq WHERE score IS NOT NULL
|
|
71
|
-
`);
|
|
72
|
-
expect(rows.length).toBe(2);
|
|
73
|
-
const ids = rows.map((r) => r.id);
|
|
74
|
-
expect(ids).toContain(1);
|
|
75
|
-
expect(ids).toContain(2);
|
|
76
|
-
});
|
|
77
|
-
|
|
78
|
-
it("creates an FTS index and performs full-text search", async () => {
|
|
79
|
-
db = await newBrowserDuckDb();
|
|
80
|
-
await db.exec("INSTALL fts; LOAD fts;");
|
|
81
|
-
await db.exec("CREATE TABLE docs (id INTEGER, content VARCHAR)");
|
|
82
|
-
await db.exec(`
|
|
83
|
-
INSERT INTO docs VALUES
|
|
84
|
-
(1, 'the quick brown fox jumps over the lazy dog'),
|
|
85
|
-
(2, 'a lazy cat sleeps on the mat'),
|
|
86
|
-
(3, 'the fox and the hound are friends')
|
|
87
|
-
`);
|
|
88
|
-
await db.exec("PRAGMA create_fts_index('docs', 'id', 'content')");
|
|
89
|
-
|
|
90
|
-
const rows = await db.query<{
|
|
91
|
-
id: number;
|
|
92
|
-
content: string;
|
|
93
|
-
score: number | null;
|
|
94
|
-
}>(`
|
|
95
|
-
SELECT id, content, score
|
|
96
|
-
FROM (
|
|
97
|
-
SELECT *, fts_main_docs.match_bm25(id, 'fox') AS score
|
|
98
|
-
FROM docs
|
|
99
|
-
) sq
|
|
100
|
-
WHERE score IS NOT NULL
|
|
101
|
-
ORDER BY score
|
|
102
|
-
`);
|
|
103
|
-
|
|
104
|
-
expect(rows.length).toBeGreaterThanOrEqual(1);
|
|
105
|
-
const ids = rows.map((r) => r.id);
|
|
106
|
-
expect(ids).toContain(1);
|
|
107
|
-
expect(ids).toContain(3);
|
|
108
|
-
});
|
|
109
|
-
});
|
|
110
|
-
|
|
111
|
-
describe("VSS extension", () => {
|
|
112
|
-
it("creates HNSW index and performs vector similarity search", async () => {
|
|
113
|
-
db = await newBrowserDuckDb();
|
|
114
|
-
await db.exec("INSTALL vss; LOAD vss;");
|
|
115
|
-
await db.exec("CREATE TABLE embeddings (id INTEGER, vec FLOAT[3])");
|
|
116
|
-
await db.exec(`
|
|
117
|
-
INSERT INTO embeddings VALUES
|
|
118
|
-
(1, [1.0, 0.0, 0.0]),
|
|
119
|
-
(2, [0.0, 1.0, 0.0]),
|
|
120
|
-
(3, [0.0, 0.0, 1.0])
|
|
121
|
-
`);
|
|
122
|
-
await db.exec("CREATE INDEX vec_idx ON embeddings USING HNSW (vec)");
|
|
123
|
-
|
|
124
|
-
const rows = await db.query<{ id: number }>(
|
|
125
|
-
"SELECT id FROM embeddings ORDER BY array_distance(vec, [1.0, 0.1, 0.0]::FLOAT[3]) LIMIT 1",
|
|
126
|
-
);
|
|
127
|
-
|
|
128
|
-
expect(rows).toHaveLength(1);
|
|
129
|
-
expect(rows[0]?.id).toBe(1);
|
|
130
|
-
});
|
|
131
|
-
|
|
132
|
-
it("returns k nearest neighbors ordered by distance", async () => {
|
|
133
|
-
db = await newBrowserDuckDb();
|
|
134
|
-
await db.exec("INSTALL vss; LOAD vss;");
|
|
135
|
-
await db.exec("CREATE TABLE vectors (id INTEGER, vec FLOAT[3])");
|
|
136
|
-
await db.exec(`
|
|
137
|
-
INSERT INTO vectors VALUES
|
|
138
|
-
(1, [1.0, 0.0, 0.0]),
|
|
139
|
-
(2, [0.9, 0.1, 0.0]),
|
|
140
|
-
(3, [0.0, 1.0, 0.0]),
|
|
141
|
-
(4, [0.0, 0.0, 1.0])
|
|
142
|
-
`);
|
|
143
|
-
await db.exec("CREATE INDEX vec_idx2 ON vectors USING HNSW (vec)");
|
|
144
|
-
|
|
145
|
-
const rows = await db.query<{ id: number }>(
|
|
146
|
-
"SELECT id FROM vectors ORDER BY array_distance(vec, [1.0, 0.0, 0.0]::FLOAT[3]) LIMIT 2",
|
|
147
|
-
);
|
|
148
|
-
|
|
149
|
-
expect(rows).toHaveLength(2);
|
|
150
|
-
expect(rows[0]?.id).toBe(1);
|
|
151
|
-
expect(rows[1]?.id).toBe(2);
|
|
152
|
-
});
|
|
153
|
-
|
|
154
|
-
it("works without HNSW index (brute-force scan)", async () => {
|
|
155
|
-
db = await newBrowserDuckDb();
|
|
156
|
-
await db.exec("INSTALL vss; LOAD vss;");
|
|
157
|
-
await db.exec("CREATE TABLE vecs_no_idx (id INTEGER, vec FLOAT[3])");
|
|
158
|
-
await db.exec(`
|
|
159
|
-
INSERT INTO vecs_no_idx VALUES
|
|
160
|
-
(1, [1.0, 0.0, 0.0]),
|
|
161
|
-
(2, [0.0, 1.0, 0.0])
|
|
162
|
-
`);
|
|
163
|
-
|
|
164
|
-
const rows = await db.query<{ id: number }>(
|
|
165
|
-
"SELECT id FROM vecs_no_idx ORDER BY array_distance(vec, [0.0, 0.9, 0.1]::FLOAT[3]) LIMIT 1",
|
|
166
|
-
);
|
|
167
|
-
|
|
168
|
-
expect(rows).toHaveLength(1);
|
|
169
|
-
expect(rows[0]?.id).toBe(2);
|
|
170
|
-
});
|
|
171
|
-
});
|
|
172
|
-
|
|
173
|
-
describe("prepared statements", () => {
|
|
174
|
-
afterEach(() => {
|
|
175
|
-
vi.restoreAllMocks();
|
|
176
|
-
});
|
|
177
|
-
|
|
178
|
-
it("closes the statement of every parameterised query", async () => {
|
|
179
|
-
db = await newBrowserDuckDb();
|
|
180
|
-
const close = vi.spyOn(duckdb.AsyncPreparedStatement.prototype, "close");
|
|
181
|
-
await db.query("SELECT $1::INTEGER AS x", [1]);
|
|
182
|
-
await db.query("SELECT $1::INTEGER AS x", [2]);
|
|
183
|
-
await expect(db.query("SELECT $1::INTEGER AS x", ["not a number"])).rejects.toThrow();
|
|
184
|
-
expect(close).toHaveBeenCalledTimes(3);
|
|
185
|
-
});
|
|
186
|
-
});
|
|
187
|
-
|
|
188
|
-
describe("persistent path", () => {
|
|
189
|
-
afterEach(() => {
|
|
190
|
-
vi.unstubAllGlobals();
|
|
191
|
-
vi.restoreAllMocks();
|
|
192
|
-
});
|
|
193
|
-
|
|
194
|
-
it("rejects instead of falling back to memory when the path cannot be opened", async () => {
|
|
195
|
-
vi.spyOn(duckdb.AsyncDuckDB.prototype, "open").mockRejectedValue(new Error("open failed"));
|
|
196
|
-
await expect(newBrowserDuckDb({ path: "opfs://broken.duckdb" })).rejects.toThrow(
|
|
197
|
-
/opfs:\/\/broken\.duckdb/,
|
|
198
|
-
);
|
|
199
|
-
});
|
|
200
|
-
|
|
201
|
-
it("rejects instead of falling back to memory when OPFS is unavailable", async () => {
|
|
202
|
-
vi.stubGlobal("navigator", {});
|
|
203
|
-
await expect(newBrowserDuckDb({ path: "opfs://unavailable.duckdb" })).rejects.toThrow(
|
|
204
|
-
/opfs:\/\/unavailable\.duckdb/,
|
|
205
|
-
);
|
|
206
|
-
});
|
|
207
|
-
});
|
|
208
|
-
});
|
|
@@ -1,14 +0,0 @@
|
|
|
1
|
-
import { describe, expect, it } from "vitest";
|
|
2
|
-
|
|
3
|
-
/**
|
|
4
|
-
* Node-safe smoke test. The real browser coverage (shared conformance suite +
|
|
5
|
-
* FTS/VSS) lives in `browser-duckdb.browser.test.ts`, run via Vitest browser
|
|
6
|
-
* mode (`pnpm test:browser`); this file only checks the module loads and
|
|
7
|
-
* exports its factory, since duckdb-wasm needs a browser Worker to run.
|
|
8
|
-
*/
|
|
9
|
-
describe("browser-duckdb module", () => {
|
|
10
|
-
it("exports newBrowserDuckDb function", async () => {
|
|
11
|
-
const mod = await import("./browser-duckdb.js");
|
|
12
|
-
expect(typeof mod.newBrowserDuckDb).toBe("function");
|
|
13
|
-
});
|
|
14
|
-
});
|