@romedb/node 0.2.12 → 0.2.13
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +849 -433
- package/THIRD_PARTY_CRATE_NOTICES.md +4 -4
- package/client.cjs +107 -21
- package/client.d.ts +15 -1
- package/client.mjs +107 -21
- package/embedded-client.cjs +71 -13
- package/embedded-client.d.ts +11 -1
- package/embedded-worker.cjs +38 -19
- package/index.cjs +18 -7
- package/index.d.ts +3 -0
- package/native.cjs +2 -2
- package/package.json +11 -11
- package/romedb.node +2 -2
- package/worker.cjs +21 -3
package/README.md
CHANGED
|
@@ -1,170 +1,547 @@
|
|
|
1
|
-
# RomeDB Node.js
|
|
1
|
+
# RomeDB for Node.js
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
RomeDB is an embedded database. You query it with RomeQL, a document query
|
|
4
|
+
language that also accepts SQL. This package has two parts:
|
|
4
5
|
|
|
5
|
-
|
|
6
|
+
- A native addon that runs RomeDB inside your Node.js process. It is built for
|
|
7
|
+
Linux x64 and needs Node.js 22 or later. Node 22 and 24 are the tested versions.
|
|
8
|
+
- A client that connects to `romedb serve` or RomeDB Cloud by URL. It uses only Web
|
|
9
|
+
APIs, so it also runs in Cloudflare Workers and in browsers.
|
|
10
|
+
|
|
11
|
+
Other operating systems and architectures are outside this release. The addon does
|
|
12
|
+
not expose the raw Turso connection. One process owns a database file at a time.
|
|
13
|
+
The [deployment contract](https://github.com/romedb/romedb/blob/main/romedb/docs/deployment.md) describes this rule.
|
|
14
|
+
|
|
15
|
+
## Install
|
|
6
16
|
|
|
7
17
|
```sh
|
|
8
18
|
npm install @romedb/node
|
|
9
19
|
```
|
|
10
20
|
|
|
11
|
-
|
|
12
|
-
|
|
21
|
+
Each release publishes its Node package asset to npm unchanged. The first npm
|
|
22
|
+
version, 0.2.12, differs from its asset only in `package.json` and this README. A
|
|
23
|
+
project can also depend on the asset of a GitHub release by URL:
|
|
13
24
|
|
|
14
25
|
```sh
|
|
15
|
-
pnpm add https://github.com/romedb/romedb/releases/download/romedb-v0.2.
|
|
26
|
+
pnpm add https://github.com/romedb/romedb/releases/download/romedb-v0.2.13/romedb-node-0.2.13.tgz
|
|
16
27
|
```
|
|
17
28
|
|
|
18
|
-
|
|
19
|
-
RomeDB runs inside the server process and stores the database there. For request
|
|
20
|
-
handlers, prefer [AsyncDatabase](#asynchronous-database) to keep database work off
|
|
21
|
-
the main JavaScript thread. The synchronous Database API remains useful for
|
|
22
|
-
scripts and other blocking workflows. Browser/WASM support is not part of V2.
|
|
23
|
-
|
|
24
|
-
From the repository root, install the declaration checker and run the scoped checks:
|
|
29
|
+
## A first database
|
|
25
30
|
|
|
26
|
-
```
|
|
27
|
-
|
|
28
|
-
romedb
|
|
31
|
+
```js
|
|
32
|
+
const { Database, Record } = require('@romedb/node');
|
|
33
|
+
const rql = new Database('app.romedb');
|
|
34
|
+
try {
|
|
35
|
+
// The first document write creates the posts collection.
|
|
36
|
+
rql.execute('INSERT INTO posts DOCUMENT $doc', {
|
|
37
|
+
$doc: { id: new Record('posts', 'p1'), title: 'Hello', visits: 1n },
|
|
38
|
+
});
|
|
39
|
+
rql.execute('UPDATE posts:p1 MERGE {visits: 2}');
|
|
40
|
+
const [post] = rql.exactlyOne('SELECT * FROM posts:p1');
|
|
41
|
+
console.log(post.title, post.visits); // Hello 2n
|
|
42
|
+
} finally {
|
|
43
|
+
rql.close();
|
|
44
|
+
}
|
|
29
45
|
```
|
|
30
46
|
|
|
31
|
-
|
|
47
|
+
`posts:p1` is a record ID: the collection `posts` and the key `p1`. In
|
|
48
|
+
JavaScript, `new Record('posts', 'p1')` is the same ID. `$doc` is a parameter, so
|
|
49
|
+
the values never become part of the statement text. Integer results are always
|
|
50
|
+
`bigint`.
|
|
51
|
+
|
|
52
|
+
Use this package in your Node.js backend. A browser calls your backend API, and
|
|
53
|
+
RomeDB runs and stores the database inside the server process. For request
|
|
54
|
+
handlers, use [AsyncDatabase](#asynchronous-database) or
|
|
55
|
+
[openEmbedded](#embedded-client), which keep database work off the main JavaScript
|
|
56
|
+
thread. The synchronous `Database` suits scripts and other blocking work.
|
|
57
|
+
|
|
58
|
+
`Database` and `AsyncDatabase` open MVCC files and WAL files, and they create new
|
|
59
|
+
files in WAL mode. `openEmbedded`, `romedb serve` and RomeDB Cloud use only MVCC
|
|
60
|
+
files. To use a WAL file with them, convert it with `romedb migrate-to-mvcc`, or
|
|
61
|
+
create the file with `openEmbedded` from the start.
|
|
62
|
+
|
|
63
|
+
## RomeQL at a glance
|
|
64
|
+
|
|
65
|
+
| Statement | What it does |
|
|
66
|
+
| --- | --- |
|
|
67
|
+
| `INSERT INTO posts {id: posts:hello, title: 'Hello', tags: ['news']}` | Adds a document. If the collection does not exist, the write creates it. |
|
|
68
|
+
| `UPSERT people:alice {name: 'Alice', role: 'editor'}` | Creates the record, or merges these fields into it. |
|
|
69
|
+
| `SELECT * FROM posts:hello` | Reads one record by its ID. |
|
|
70
|
+
| `UPDATE posts:hello MERGE {visits: 2}` | Changes the given fields and keeps the others. |
|
|
71
|
+
| `SELECT id, title FROM posts WHERE array::contains(tags, 'news')` | Finds documents by a value in an array. |
|
|
72
|
+
| `INSERT INTO posts {id: posts:second, title: 'Second', author: people:alice}` | Stores a link to another record. |
|
|
73
|
+
| `SELECT * FROM posts WHERE id = posts:second FETCH author` | Returns the post with the linked `people:alice` document in place of the link. |
|
|
74
|
+
| `DELETE posts:hello` | Deletes one record. `DELETE FROM posts:hello` is the same statement. |
|
|
75
|
+
| `DEFINE FIELD title ON posts TYPE string` | Adds validation. After it, a write with a number in `title` fails with `RDB_VALIDATION`. |
|
|
76
|
+
| `CREATE UNIQUE INDEX posts_title ON posts(title)` | Adds a unique index on a field. |
|
|
77
|
+
| `CREATE TABLE visits(at INTEGER, path TEXT)` | Makes an ordinary SQL table. SQL statements work on it, and collections can join it. |
|
|
78
|
+
|
|
79
|
+
The [language guide](https://github.com/romedb/romedb/blob/main/romedb/docs/v2.2-language.md) describes the full language. The
|
|
80
|
+
[document SDK guide](https://github.com/romedb/romedb/blob/main/romedb/docs/node-sdk.md) describes the typed `collection(name)`
|
|
81
|
+
helpers.
|
|
82
|
+
|
|
83
|
+
## Choose an API
|
|
84
|
+
|
|
85
|
+
| API | Entry | Use it for |
|
|
86
|
+
| --- | --- | --- |
|
|
87
|
+
| `Database` | `@romedb/node` | Scripts and tools. Each call blocks the JavaScript thread. |
|
|
88
|
+
| `AsyncDatabase` | `@romedb/node` | Servers. Queries run on a worker thread, in the order you send them. |
|
|
89
|
+
| `openEmbedded` | `@romedb/node/embedded-client` | A local MVCC file with the same request interface as the URL client: atomic batches, sequences and write recovery. |
|
|
90
|
+
| `connect` | `@romedb/node/client` | A database behind `romedb serve` or RomeDB Cloud. Runs in Node.js, Workers and browsers. |
|
|
91
|
+
|
|
92
|
+
## Database
|
|
93
|
+
|
|
94
|
+
### Results
|
|
95
|
+
|
|
96
|
+
`execute(sql, parameters)` returns an object with these fields:
|
|
97
|
+
|
|
98
|
+
- `columns`: the column names.
|
|
99
|
+
- `rows`: positional rows of typed values.
|
|
100
|
+
- `affected`: the count of changed rows, as a `bigint`.
|
|
101
|
+
- `transaction`: the transaction state before and after the statement.
|
|
102
|
+
|
|
103
|
+
`all`, `first` and `exactlyOne` return only rows. `first` returns `undefined` when
|
|
104
|
+
there is no row. `exactlyOne` throws a `RangeError` with the code `RDB_CARDINALITY`
|
|
105
|
+
unless the result has exactly one row. That error has the `transaction` field of
|
|
106
|
+
the statement. The statement already ran, so the helper does not undo writes or
|
|
107
|
+
roll back an outer transaction.
|
|
108
|
+
|
|
109
|
+
### The connection
|
|
110
|
+
|
|
111
|
+
`Database` methods are synchronous and block the calling JavaScript thread. Each
|
|
112
|
+
instance owns one frontend connection and the lifetime of its database. You can
|
|
113
|
+
call `close()` more than once. Calls after `close()` fail. Closing during an active
|
|
114
|
+
transaction uses the normal connection-drop behavior of the engine. Do not share
|
|
115
|
+
native handles across workers. Use `AsyncDatabase` to give a dedicated worker its
|
|
116
|
+
own connection.
|
|
117
|
+
|
|
118
|
+
### Values
|
|
119
|
+
|
|
120
|
+
| JavaScript | RomeDB |
|
|
121
|
+
| --- | --- |
|
|
122
|
+
| `bigint`, or a `Number` that is a safe integer | int64. Integer results are always `bigint`. |
|
|
123
|
+
| A `Number` with a fraction, and `-0` | binary64, with the exact bits kept |
|
|
124
|
+
| A `Number` that is an unsafe integer | Refused. Use a `bigint`. |
|
|
125
|
+
| `boolean`, `null`, `string` | The same type |
|
|
126
|
+
| Array, plain object | Array, object |
|
|
127
|
+
| `Buffer`, `Uint8Array` | Binary |
|
|
128
|
+
| `Record(table, key)` | A record ID. The key is a string or a `bigint`. |
|
|
129
|
+
| `Vector` | A vector of validated native-format bytes. See [typed vectors](#typed-vectors). |
|
|
130
|
+
|
|
131
|
+
RomeDB never makes a reference from a string. Unsupported JavaScript types,
|
|
132
|
+
non-finite numbers, int64 values out of range and too much nesting fail. The bridge
|
|
133
|
+
reads only the own entries of an object, so a field named like a prototype property
|
|
134
|
+
stays data. Values travel in the transfer-v1 tagged encoding.
|
|
135
|
+
|
|
136
|
+
Standalone projections such as `SELECT $object AS value` keep the types of
|
|
137
|
+
composite, record, vector and boolean parameters. Typed helper expressions and
|
|
138
|
+
DOCUMENT writes keep them too. Type propagation through other source queries is
|
|
139
|
+
not finished. Prepared statements and streamed results are not available.
|
|
140
|
+
|
|
141
|
+
### Errors
|
|
142
|
+
|
|
143
|
+
Engine and frontend failures throw an `Error` with a `code` and a `transaction`
|
|
144
|
+
field. A call on a closed `Database` or `AsyncDatabase`, or on a worker that is
|
|
145
|
+
closing, fails with `RDB_CLOSED`. That error has no `transaction` field, because no
|
|
146
|
+
statement ran. Constructor errors and JavaScript argument errors are ordinary
|
|
147
|
+
errors without these fields. The error contract across clients is not final.
|
|
148
|
+
|
|
149
|
+
| Code | Meaning |
|
|
150
|
+
| --- | --- |
|
|
151
|
+
| `RDB_BUSY` | Native lock contention. |
|
|
152
|
+
| `RDB_BUSY_SNAPSHOT` | A stale read transaction cannot become a writer. The work needs a fresh transaction. |
|
|
153
|
+
| `RDB_CONSTRAINT` | A native constraint, foreign-key or trigger-raise error, with the engine message. |
|
|
154
|
+
| `RDB_VALIDATION` | Frontend validation of a document failed. |
|
|
155
|
+
| `RDB_CLOSED` | The database or its worker is closed. |
|
|
156
|
+
|
|
157
|
+
Before you recover from an error, read its `transaction` field. The client does not
|
|
158
|
+
retry by itself. `RDB_CONSTRAINT` also covers some runtime validation failures, and
|
|
159
|
+
it does not tell the rollback scope: ordinary SQL `OR FAIL` can keep earlier rows.
|
|
160
|
+
Apply the conflict policy of the statement when you recover. A test with two
|
|
161
|
+
`Database` instances on one file covers both busy codes. Contention and recovery
|
|
162
|
+
across processes need more qualification.
|
|
163
|
+
|
|
164
|
+
`isRomeDBError(error)` narrows a caught `unknown` value to the exported
|
|
165
|
+
`RomeDBError` interface. It accepts an `Error` with an `RDB_*` code. It also
|
|
166
|
+
validates the transaction and migration details when they are present. The
|
|
167
|
+
`transaction` field is optional, because closed-handle, worker and queue errors can
|
|
168
|
+
happen before execution. Constructor and argument errors can fail this test.
|
|
169
|
+
|
|
170
|
+
### Migrations
|
|
171
|
+
|
|
172
|
+
`migrate(migrations)` takes the complete ordered history as
|
|
173
|
+
`{ version: bigint, name: string, sql: string }` entries. It returns
|
|
174
|
+
`{ alreadyApplied: number, applied: bigint[], transaction }`. The rules of the Rust
|
|
175
|
+
runner apply unchanged. It needs autocommit and compares applied scripts with their
|
|
176
|
+
exact source. It has limits, and it runs all pending scripts atomically. The Node
|
|
177
|
+
client does not read migration folders, so your code supplies the scripts.
|
|
178
|
+
|
|
179
|
+
| Migration failure | Error and meaning |
|
|
180
|
+
| --- | --- |
|
|
181
|
+
| Script splitting or tokenization | `RDB_SYNTAX`, with the migration version and the byte offset in that script. The plan did not start. |
|
|
182
|
+
| Edited or incomplete applied history | `RDB_VALIDATION`. The message tells whether the version sequence, the name or the exact SQL differs. Restore the original applied scripts before you retry. |
|
|
183
|
+
| Size limit of the input or the stored history | `RDB_LIMIT`. The message names the limit. |
|
|
184
|
+
| Incompatible history schema or invalid stored types | `RDB_STORAGE`. The runner does not repair the history. |
|
|
185
|
+
| A pending statement fails | Usually `RDB_MIGRATION`. `error.migration.version` and `.offset` are `bigint` values, and `.cause.code` and `.cause.message` give the underlying failure. The offset counts UTF-8 bytes in that script. The runner rolls back all pending scripts and their history together. |
|
|
186
|
+
| Cooperative cancellation | `RDB_CANCELLED`. The message of a pending statement keeps its version and offset. Completion can win the race with the abort. |
|
|
187
|
+
| Rollback failure | `RDB_ROLLBACK`. Cleanup did not reach the normal rollback outcome. |
|
|
188
|
+
|
|
189
|
+
Migration errors keep `code`, `message` and the transaction observations of the
|
|
190
|
+
runner in both clients. The version and the UTF-8 byte offset are in the message.
|
|
191
|
+
There are no separate JavaScript `version` or `offset` properties on the error.
|
|
192
|
+
|
|
193
|
+
The optional `migration` property has the exported `MigrationFailure` shape. Wrapped
|
|
194
|
+
statement failures have it, interrupted statements included. Plan parsing, history
|
|
195
|
+
comparison and cancellation before execution do not always provide it.
|
|
196
|
+
|
|
197
|
+
Before you retry, read `error.transaction` and the actual result. A worker
|
|
198
|
+
transport failure (`RDB_WORKER`) has no transaction observation, so it does not tell
|
|
199
|
+
whether accepted migration work committed. JavaScript argument validation can fail
|
|
200
|
+
before the runner starts. Keep applied scripts unchanged, comments and whitespace
|
|
201
|
+
included. After a confirmed failed run, you can correct a pending script that is not
|
|
202
|
+
recorded yet.
|
|
203
|
+
|
|
204
|
+
### Document import and export
|
|
205
|
+
|
|
206
|
+
`exportDocuments(table, format = 'json')` returns the versioned typed transfer
|
|
207
|
+
string. `importDocuments(table, input, format = 'json')` returns
|
|
208
|
+
`{ imported: number, transaction }`. The format is `json` or `ndjson`. Imports go
|
|
209
|
+
into an existing collection and keep its validation, indexes and rollback behavior.
|
|
210
|
+
|
|
211
|
+
These methods hold the whole string in memory and use the transfer limits of the
|
|
212
|
+
frontend. They do not back up the schema, stream data or read or write files. To
|
|
213
|
+
copy a whole database into a file, use [`exportFile`](#copying-a-database-into-a-file).
|
|
214
|
+
Failures use the same code and transaction fields as `execute`. Closed-handle calls
|
|
215
|
+
fail with `RDB_CLOSED` and no transaction field.
|
|
216
|
+
|
|
217
|
+
## Asynchronous database
|
|
32
218
|
|
|
33
219
|
```js
|
|
34
|
-
const {
|
|
35
|
-
const
|
|
220
|
+
const { AsyncDatabase } = require('@romedb/node');
|
|
221
|
+
const rql = await AsyncDatabase.open('app.romedb');
|
|
36
222
|
try {
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
$doc: { id: new Record('posts', 'p1'), title: 'Hello', visits: 1n }
|
|
40
|
-
});
|
|
41
|
-
const row = db.exactlyOne('SELECT * FROM posts WHERE id = $id', {
|
|
42
|
-
$id: new Record('posts', 'p1')
|
|
43
|
-
});
|
|
44
|
-
console.log(row[0]);
|
|
223
|
+
await rql.execute("UPSERT people:alice {name: 'Alice', active: true}");
|
|
224
|
+
console.log(await rql.all('SELECT name FROM people WHERE active = true')); // [ [ 'Alice' ] ]
|
|
45
225
|
} finally {
|
|
46
|
-
|
|
226
|
+
await rql.close();
|
|
47
227
|
}
|
|
48
228
|
```
|
|
49
229
|
|
|
50
|
-
`
|
|
230
|
+
`AsyncDatabase` has the methods of `Database` and returns Promises with the same
|
|
231
|
+
values and results. It also has `exportFile(destination, options)`, which copies an
|
|
232
|
+
MVCC database into a new file while queries continue and resolves to `{ bytes }`.
|
|
51
233
|
|
|
52
|
-
|
|
234
|
+
### How it runs
|
|
53
235
|
|
|
236
|
+
`AsyncDatabase.open()` waits for the native database to open and rejects if it
|
|
237
|
+
fails. One dedicated [Node worker](https://nodejs.org/api/worker_threads.html) owns
|
|
238
|
+
the native connection. Requests run one at a time, in the order you send them.
|
|
239
|
+
Each `AsyncDatabase` instance has its own worker and connection. The database work
|
|
240
|
+
runs off the event loop of the caller. Argument encoding, message copies and result
|
|
241
|
+
decoding still use time and memory in the caller.
|
|
54
242
|
|
|
55
|
-
|
|
243
|
+
### Queue and closing
|
|
56
244
|
|
|
57
|
-
|
|
245
|
+
The queue accepts 256 outstanding operations and 128 MiB of encoded argument
|
|
246
|
+
strings, the active operation included. A request over these limits rejects with
|
|
247
|
+
`RDB_LIMIT` before it is sent. These limits do not bound total memory or query time.
|
|
58
248
|
|
|
59
|
-
|
|
249
|
+
`close()` stops new requests, runs the requests already accepted, closes the native
|
|
250
|
+
connection and waits for the worker to exit. Repeated `close()` calls share the same
|
|
251
|
+
Promise. Closing during an active transaction rolls it back, through the native
|
|
252
|
+
drop behavior that the reopen tests cover. You must await `close()` to release the
|
|
253
|
+
worker. There is no automatic idle shutdown and no force-termination API.
|
|
60
254
|
|
|
255
|
+
### Transactions and errors
|
|
61
256
|
|
|
62
|
-
|
|
257
|
+
A query error rejects only its own Promise. Later queued requests still run. An
|
|
258
|
+
explicit transaction belongs to the whole connection, not to one caller or one
|
|
259
|
+
Promise chain. Your application must decide who owns a transaction, and it must
|
|
260
|
+
await each step before it sends dependent work. The API has no transaction callback
|
|
261
|
+
that isolates callers.
|
|
63
262
|
|
|
64
|
-
|
|
263
|
+
An unexpected worker error or exit rejects every outstanding request. Recovery
|
|
264
|
+
after a worker crash, accounting of native memory, more cancellation tests and
|
|
265
|
+
long-running lifecycle stress are still release gates.
|
|
65
266
|
|
|
66
|
-
|
|
67
|
-
|---|---|
|
|
68
|
-
| Script splitting/tokenization | `FDB_SYNTAX`, with migration version and byte offset within that script; the plan has not begun executing. |
|
|
69
|
-
| Edited or incomplete applied history | `FDB_VALIDATION`; a mismatch message distinguishes the version sequence, name or exact SQL source. Restore the original applied prefix before retrying. |
|
|
70
|
-
| Input or stored-history size limit | `FDB_LIMIT`; inspect the message to identify the rejected limit. |
|
|
71
|
-
| Incompatible ledger schema or invalid stored types | `FDB_STORAGE`; the runner does not repair the ledger. |
|
|
72
|
-
| Pending statement failure | Usually `FDB_MIGRATION`, with `error.migration.version` and `.offset` as bigint values and `.cause.code`/`.message` identifying the underlying failure. The offset counts UTF-8 bytes within that script. The runner attempts to roll back all pending scripts and history together. |
|
|
73
|
-
| Cooperative cancellation | `FDB_CANCELLED`; a pending statement's version and offset remain in its message. Completion can win the abort race. |
|
|
74
|
-
| Rollback failure | `FDB_ROLLBACK`; cleanup did not establish the normal rollback outcome. |
|
|
267
|
+
### Interrupting a statement
|
|
75
268
|
|
|
76
|
-
|
|
269
|
+
`interrupt()` asks the engine to stop the active statement. It runs at once and does
|
|
270
|
+
not wait in the worker queue. It returns `false` after native close, and `true` while
|
|
271
|
+
the connection is live. `true` does not confirm that the statement stopped.
|
|
77
272
|
|
|
78
|
-
|
|
273
|
+
A cancelled `execute` rejects with `RDB_CANCELLED` and the observed transaction
|
|
274
|
+
state, and the worker can run later operations. The interruption applies to the
|
|
275
|
+
whole connection: it does not target one Promise and does not remove queued
|
|
276
|
+
requests. Stop repeated interrupt requests before you use the connection again.
|
|
277
|
+
Parsing, work outside the engine and a running bundled function do not stop by
|
|
278
|
+
interruption, and there is no hard deadline.
|
|
79
279
|
|
|
80
|
-
|
|
280
|
+
The addon keeps a process-local registry of weak interrupt handles and removes an
|
|
281
|
+
entry on native close or drop. No raw engine pointer crosses into JavaScript.
|
|
81
282
|
|
|
283
|
+
### Worker failures
|
|
82
284
|
|
|
83
|
-
|
|
285
|
+
A fatal worker transport error uses `RDB_WORKER`, keeps the first cause and rejects
|
|
286
|
+
every outstanding request. It has no transaction observation: a missing response
|
|
287
|
+
does not tell whether an accepted write committed. Later requests reject with the
|
|
288
|
+
same failure. Do not retry a write with an unknown outcome automatically.
|
|
84
289
|
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
290
|
+
After such a failure, `close()` waits for the worker to exit. If the failure happens
|
|
291
|
+
during a `close()` that is already running, that `close()` rejects after cleanup
|
|
292
|
+
ends. When a message cannot be deserialized, the client asks for a graceful native
|
|
293
|
+
close through the remaining channel. If that send fails too, cleanup terminates the
|
|
294
|
+
worker. This fallback is separate from query interruption, and it does not prove
|
|
295
|
+
database recovery or the outcome of a write.
|
|
296
|
+
|
|
297
|
+
Isolated fault-injection tests cover the state changes of the worker transport.
|
|
298
|
+
Native crashes, interrupted commit or recovery and the resources of a forced
|
|
299
|
+
termination need separate release qualification.
|
|
300
|
+
|
|
301
|
+
### Script batches
|
|
95
302
|
|
|
96
|
-
|
|
303
|
+
`executeBatch(script)` runs a semicolon-separated script and returns one entry per
|
|
304
|
+
statement. `AsyncDatabase` sends the whole batch as one worker operation. Each entry
|
|
305
|
+
is one of these:
|
|
97
306
|
|
|
98
|
-
|
|
307
|
+
- `{ offset, transaction, result }`, where `result` has columns, typed positional
|
|
308
|
+
rows and a `bigint` affected count.
|
|
309
|
+
- `{ offset, transaction, error: { code, message } }`.
|
|
99
310
|
|
|
100
|
-
|
|
311
|
+
Offsets are UTF-8 byte positions. A statement error is returned as an entry and
|
|
312
|
+
stops the batch, so you must read every entry. Lexical errors and other errors
|
|
313
|
+
before execution throw or reject in the normal error format, and no statement runs.
|
|
101
314
|
|
|
315
|
+
Batches use the same semicolon-aware parser as Rust and the CLI, trigger bodies and
|
|
316
|
+
multiline documents included. A batch has no implicit transaction and takes no
|
|
317
|
+
parameters. Put explicit transaction statements in the script, and read the
|
|
318
|
+
transaction state of the last entry after a failure. Each result is encoded before
|
|
319
|
+
the next statement runs, so a result that cannot be encoded also stops the batch.
|
|
320
|
+
Neither a stop nor an encoding error undoes earlier work. Results stay in memory,
|
|
321
|
+
and limits for scripts and result memory need more qualification.
|
|
102
322
|
|
|
103
|
-
|
|
323
|
+
### Cancellation with AbortSignal
|
|
104
324
|
|
|
325
|
+
Pass `{ signal: controller.signal }` in the options position. If you skip an earlier
|
|
326
|
+
optional argument, pass `undefined` in its place.
|
|
105
327
|
|
|
106
|
-
|
|
328
|
+
| Async method | Call with options |
|
|
329
|
+
| --- | --- |
|
|
330
|
+
| Queries and row helpers | `execute(sql, parameters, options)`, `all(sql, parameters, options)`, `first(sql, parameters, options)`, `exactlyOne(sql, parameters, options)` |
|
|
331
|
+
| Query profiling | `profileSelect(sql, parameters, options)` |
|
|
332
|
+
| Collection integrity | `checkCollectionIntegrity(table, limits, options)` |
|
|
333
|
+
| Script batch | `executeBatch(script, options)` |
|
|
334
|
+
| Document export | `exportDocuments(table, format, options)` |
|
|
335
|
+
| Document import | `importDocuments(table, input, format, options)` |
|
|
336
|
+
| Database file copy | `exportFile(destination, options)` |
|
|
337
|
+
| Migrations | `migrate(plan, options)` |
|
|
107
338
|
|
|
108
|
-
|
|
339
|
+
For example, `await rql.exportDocuments('posts', undefined, { signal })` uses the
|
|
340
|
+
default JSON format. `open()` and `close()` do not take a signal.
|
|
341
|
+
|
|
342
|
+
Each signalled request gets its own native cancellation token. A queued request that
|
|
343
|
+
is aborted runs no SQL when its turn comes. Accepted requests keep their queue order
|
|
344
|
+
and their queue slot until the worker responds. Cancellation does not close the
|
|
345
|
+
database or cancel other requests.
|
|
346
|
+
|
|
347
|
+
Active cancellation is cooperative: the engine reacts at its progress points.
|
|
348
|
+
Completion can win the race with the abort, so use the returned result or error. An
|
|
349
|
+
abort alone does not prove a rollback. A cancelled query rejects with
|
|
350
|
+
`RDB_CANCELLED` and the usual transaction fields, not with `signal.reason`. There is
|
|
351
|
+
no fixed cancellation delay for compilation or work outside the engine. The pinned
|
|
352
|
+
defect in trigger interruption is still a release gate.
|
|
353
|
+
|
|
354
|
+
Listeners and token entries are released on a response, a send failure or a worker
|
|
355
|
+
failure. Tokens do not keep connections, so a late abort cannot reach another
|
|
356
|
+
request. The process-local registry allows 16,384 outstanding signalled operations
|
|
357
|
+
across all clients, in addition to the queue limits of each client. Requests without
|
|
358
|
+
a signal use the normal path. Synchronous `Database` methods do not take signals.
|
|
359
|
+
|
|
360
|
+
How each operation ends after cancellation:
|
|
361
|
+
|
|
362
|
+
- `executeBatch`: a request aborted before it starts rejects before the script is
|
|
363
|
+
split. Cancellation during the batch appears as a final `RDB_CANCELLED` entry with
|
|
364
|
+
the byte offset and transaction state of that statement. Earlier entries stay,
|
|
365
|
+
and later statements do not run. Earlier writes keep their effects, and an explicit
|
|
366
|
+
transaction can stay active. Coordinate any rollback yourself, and use a fresh
|
|
367
|
+
signal for a retry.
|
|
368
|
+
- `importDocuments` and `exportDocuments`: rejects with `RDB_CANCELLED` and
|
|
369
|
+
transaction fields. An import rolls back its own work and keeps earlier outer work.
|
|
370
|
+
An export returns a complete string or an error.
|
|
371
|
+
- `migrate`: interrupted statements report `RDB_CANCELLED`, and the message keeps the
|
|
372
|
+
migration version and byte offset. All pending scripts and history rows roll back
|
|
373
|
+
together, and applied migrations stay. A request aborted before it starts does no
|
|
374
|
+
database work, but JavaScript argument validation still runs.
|
|
375
|
+
- `close()`: runs the accepted requests and does not abort them. Signals can still
|
|
376
|
+
cancel accepted operations while `close()` waits. Later requests reject with
|
|
377
|
+
`RDB_CLOSED`.
|
|
378
|
+
|
|
379
|
+
Parsing, conversion and serialization have no fixed cancellation delay. In every
|
|
380
|
+
case, a successful completion can win the race, so read the outcome before a retry.
|
|
381
|
+
|
|
382
|
+
### Timeouts
|
|
383
|
+
|
|
384
|
+
Async operations that take options also take `timeoutMs`, an integer from 0 to
|
|
385
|
+
4,294,967,295:
|
|
109
386
|
|
|
387
|
+
```js
|
|
388
|
+
await rql.execute('SELECT * FROM posts', {}, { timeoutMs: 500 });
|
|
389
|
+
```
|
|
110
390
|
|
|
111
|
-
|
|
391
|
+
The deadline starts when the request token is made: after the parameters are
|
|
392
|
+
encoded and before the request enters the worker queue. Time in the queue counts,
|
|
393
|
+
and 0 has already expired. An expired request returns `RDB_CANCELLED` through the
|
|
394
|
+
cancellation path, with the normal rollback and transaction fields. You can pass
|
|
395
|
+
`signal` with it. An expired request does not cancel later requests. `close()` has no
|
|
396
|
+
timeout.
|
|
112
397
|
|
|
113
|
-
|
|
398
|
+
This is not a hard deadline for the Promise. The engine notices the deadline at its
|
|
399
|
+
progress points. Parsing, work outside the engine, cleanup and the response can take
|
|
400
|
+
longer, and completion can win the race. Synchronous clients do not have this
|
|
401
|
+
option.
|
|
114
402
|
|
|
403
|
+
## Result and memory limits
|
|
115
404
|
|
|
116
|
-
|
|
405
|
+
### Bounded SELECT results
|
|
117
406
|
|
|
407
|
+
Both clients have `selectWithLimits(sql, limits, parameters?)` and
|
|
408
|
+
`profileSelectWithLimits(sql, limits, parameters?)`. The `AsyncDatabase` versions
|
|
409
|
+
take `{ signal }` as a fourth argument. Both limits are required, nonnegative
|
|
410
|
+
`bigint` values:
|
|
118
411
|
|
|
119
|
-
|
|
412
|
+
```js
|
|
413
|
+
const result = await rql.selectWithLimits(
|
|
414
|
+
'SELECT title FROM posts LIMIT 100',
|
|
415
|
+
{ maxRows: 100n, maxPayloadBytes: 65536n },
|
|
416
|
+
);
|
|
417
|
+
```
|
|
120
418
|
|
|
419
|
+
If the result passes a limit, the call fails with `RDB_LIMIT` and the transaction
|
|
420
|
+
fields, and it returns no partial rows or metrics. An empty result still counts its
|
|
421
|
+
column names. These calls accept one SQL SELECT, one-hop FETCH included. Fetched
|
|
422
|
+
documents and their duplicates count toward the payload limit, and missing targets
|
|
423
|
+
count as nulls. A found target is counted with its output multiplicity before the
|
|
424
|
+
cache keeps it, so the call can fail early. Reference placeholders do not count.
|
|
425
|
+
`execute` and `profileSelect` keep their own result policy.
|
|
426
|
+
|
|
427
|
+
Payload bytes count UTF-8 column names, strings, object keys, record table names and
|
|
428
|
+
string keys. Integer keys and numbers count eight bytes, `null` and booleans one byte,
|
|
429
|
+
and binary values and encoded vectors their byte length. Objects and arrays count
|
|
430
|
+
the sum of their contents. These are budgets for the logical result. They do not
|
|
431
|
+
count containers, allocator overhead, temporary decoding, copies in Node transport
|
|
432
|
+
or engine working memory. FETCH keeps its own reference and tagged-JSON limits, and
|
|
433
|
+
the final payload limit does not cover temporary references or the target cache.
|
|
434
|
+
These limits are not process memory limits or deadlines.
|
|
121
435
|
|
|
122
|
-
|
|
436
|
+
### Limits on write results
|
|
123
437
|
|
|
124
|
-
|
|
438
|
+
Both clients have `writeWithResultLimits(sql, limits, parameters?)`. The
|
|
439
|
+
`AsyncDatabase` version takes `{ signal }` as a fourth argument. It uses the same
|
|
440
|
+
required `bigint` fields `maxRows` and `maxPayloadBytes`:
|
|
125
441
|
|
|
126
|
-
```
|
|
127
|
-
|
|
128
|
-
|
|
442
|
+
```js
|
|
443
|
+
const result = await rql.writeWithResultLimits(
|
|
444
|
+
'UPDATE posts SET published = true RETURNING id',
|
|
445
|
+
{ maxRows: 100n, maxPayloadBytes: 65536n },
|
|
446
|
+
);
|
|
129
447
|
```
|
|
130
448
|
|
|
131
|
-
|
|
449
|
+
Supported SQL writes and document writes run in an operation savepoint. Native SQL
|
|
450
|
+
counts each result row before the frontend keeps it. Collection SQL and document
|
|
451
|
+
RETURNING count each projected row before they keep it. Native destinations with
|
|
452
|
+
logical sources also count rows during the final collection. In all cases, a write
|
|
453
|
+
over a limit returns `RDB_LIMIT` and rolls back, index and trigger effects included.
|
|
132
454
|
|
|
133
|
-
|
|
455
|
+
This policy accepts or refuses a result. It does not bound engine RETURNING buffers,
|
|
456
|
+
write candidates, document snapshots or the working memory of one projection.
|
|
457
|
+
Transaction control, DDL, reads and multiple statements are refused. A write without
|
|
458
|
+
RETURNING can succeed with limits of zero, because affected rows are not returned
|
|
459
|
+
rows. An empty RETURNING still counts its column names. Cooperative cancellation
|
|
460
|
+
uses the usual error fields and the savepoint recovery, and the count of completed
|
|
461
|
+
results has no fixed cancellation delay.
|
|
134
462
|
|
|
135
|
-
|
|
463
|
+
### Collection write buffers
|
|
136
464
|
|
|
137
|
-
|
|
465
|
+
Both clients take an optional second argument when they open:
|
|
138
466
|
|
|
139
|
-
|
|
467
|
+
```js
|
|
468
|
+
const options = { writeBufferLimits: { maxRows: 10_000n, maxPayloadBytes: 64n * 1024n * 1024n } };
|
|
469
|
+
const rql = new Database(':memory:', options);
|
|
470
|
+
const worker = await AsyncDatabase.open(':memory:', options);
|
|
471
|
+
```
|
|
472
|
+
|
|
473
|
+
The limits apply to each write-candidate buffer and each document-snapshot buffer of
|
|
474
|
+
the frontend. If a buffer passes a limit, the operation fails with `RDB_LIMIT` and
|
|
475
|
+
rolls back, and earlier work in the transaction stays. Ordinary reads and the result
|
|
476
|
+
limits above have their own policies. The limits are off by default and cannot
|
|
477
|
+
change while the connection is open. Both fields are required, nonnegative uint64
|
|
478
|
+
`bigint` values. Unknown options and invalid values fail before the database opens.
|
|
479
|
+
|
|
480
|
+
These are not total memory limits. Native engine buffers, parsing, temporary values
|
|
481
|
+
and overlapping buffers are outside them, and transfer APIs keep their own limits.
|
|
482
|
+
See [result budgets](https://github.com/romedb/romedb/blob/main/romedb/docs/result-budgets.md) for the accounting and the open
|
|
483
|
+
resource gates.
|
|
484
|
+
|
|
485
|
+
## Profiling and audits
|
|
486
|
+
|
|
487
|
+
### SELECT profiling
|
|
488
|
+
|
|
489
|
+
Both clients have `profileSelect(sql, parameters?)`. It returns `{ result, metrics }`.
|
|
490
|
+
`result` has the same typed rows, `bigint` affected count and transaction fields as
|
|
491
|
+
`execute`.
|
|
140
492
|
|
|
141
493
|
```js
|
|
142
|
-
const profile = await
|
|
143
|
-
'SELECT id FROM posts WHERE author
|
|
144
|
-
{ $author: new Record('
|
|
494
|
+
const profile = await rql.profileSelect(
|
|
495
|
+
'SELECT id FROM posts WHERE author = $author',
|
|
496
|
+
{ $author: new Record('people', 'alice') },
|
|
145
497
|
);
|
|
146
498
|
console.log(profile.result.rows, profile.metrics.rowsRead);
|
|
147
499
|
```
|
|
148
500
|
|
|
149
|
-
|
|
501
|
+
Every metric is a `bigint`: `rowsRead`, `rowsWritten`, `fullscanSteps`,
|
|
502
|
+
`indexSteps`, `vmSteps`, `sortOperations`, `btreeSeeks`, `fetchBatches`,
|
|
503
|
+
`fetchRowsRead` and `fetchVmSteps`, and the graph and spatial counters of
|
|
504
|
+
`QueryMetrics`. The addon sends counters as decimal strings, so no JavaScript
|
|
505
|
+
`Number` conversion loses digits.
|
|
150
506
|
|
|
151
|
-
|
|
507
|
+
The `fetch` counters report the target SELECT batches, physical row reads and VM
|
|
508
|
+
instructions of FETCH, and they are zero without target reads. The other counters
|
|
509
|
+
cover the main statement. References that point to the same target share its reads.
|
|
510
|
+
Physical row reads are not logical document counts. Metadata and lowering queries,
|
|
511
|
+
decoding and transport are not counted.
|
|
152
512
|
|
|
153
|
-
`
|
|
513
|
+
`profileSelect` accepts one SQL SELECT. Writes, multiple statements, EXPLAIN and
|
|
514
|
+
direct-record shorthand are refused. A failure throws or rejects with the usual
|
|
515
|
+
error fields and no partial counters. The `AsyncDatabase` version takes
|
|
516
|
+
`{ signal }` as a third argument, which cancels cooperatively and is not a deadline.
|
|
517
|
+
|
|
518
|
+
### Collection integrity audit
|
|
519
|
+
|
|
520
|
+
Both clients have `checkCollectionIntegrity(table, limits?)`. It returns the
|
|
521
|
+
`bigint` counts `documents`, `indexes`, `indexEntries` and `encodedBytes`, and the
|
|
522
|
+
transaction fields:
|
|
154
523
|
|
|
155
524
|
```js
|
|
156
|
-
const audit = await
|
|
525
|
+
const audit = await rql.checkCollectionIntegrity('posts', {
|
|
157
526
|
maxDocuments: 100_000n,
|
|
158
527
|
maxEncodedBytes: 64n * 1024n * 1024n,
|
|
159
528
|
});
|
|
160
529
|
console.log(audit.documents, audit.indexEntries);
|
|
161
530
|
```
|
|
162
531
|
|
|
163
|
-
The
|
|
532
|
+
The optional limits are nonnegative `bigint` values that fit in uint64. Without them,
|
|
533
|
+
the defaults are 100,000 documents and 64 MiB of encoded IDs and documents. Zero is
|
|
534
|
+
valid for an empty collection. The `AsyncDatabase` version takes `{ signal }` as a
|
|
535
|
+
third argument.
|
|
536
|
+
|
|
537
|
+
The audit tests typed IDs, validation and the consistency of index entries in one
|
|
538
|
+
snapshot, and it repairs nothing. `RDB_LIMIT` returns no partial report. The limits
|
|
539
|
+
do not bound engine memory or time. This is not a test of physical pages or B-trees,
|
|
540
|
+
and it is not a tool to recover from corruption.
|
|
164
541
|
|
|
165
|
-
##
|
|
542
|
+
## Typed vectors
|
|
166
543
|
|
|
167
|
-
|
|
544
|
+
The vector factories work without a database, and both clients accept their values:
|
|
168
545
|
|
|
169
546
|
```js
|
|
170
547
|
const { Vector } = require('@romedb/node');
|
|
@@ -176,431 +553,470 @@ const bits = Vector.bit1([1, 0, -1]);
|
|
|
176
553
|
const sparseEntries = Vector.sparse32Entries(65536, [[0, 1], [65535, -1]]);
|
|
177
554
|
```
|
|
178
555
|
|
|
179
|
-
The five dense
|
|
556
|
+
The five dense factories take a number array (a readonly array in TypeScript), a
|
|
557
|
+
`Float32Array` or a `Float64Array` of 1 to 65,536 finite numbers. `float64` keeps
|
|
558
|
+
binary64 values. The other factories first convert to float32 and refuse an overflow
|
|
559
|
+
to infinity. Quantized and bit conversion lose precision. Inputs are copied, and the
|
|
560
|
+
factories run synchronously. The installed-package smoke uses the five dense
|
|
561
|
+
factories and the sparse-entry factory through both clients.
|
|
562
|
+
|
|
563
|
+
`Vector.sparse32Entries(dimensions, entries)` takes `[index, value]` pairs. The
|
|
564
|
+
exported `SparseVectorEntry` type supports readonly tuples. Dimensions are 1 to
|
|
565
|
+
65,536. Indices must be unique, strictly increasing and in range, zero-valued entries
|
|
566
|
+
included. Values must be finite and fit in float32. Zero entries are left out after
|
|
567
|
+
the float32 conversion, and no entries make an all-zero vector of the given
|
|
568
|
+
dimensions. Memory grows with the count of entries, with no dense copy.
|
|
569
|
+
|
|
570
|
+
Invalid indices throw a `RangeError`, and malformed pairs throw a `TypeError`. Invalid
|
|
571
|
+
container or component types throw a `TypeError`, and invalid dimensions or a float32
|
|
572
|
+
overflow throw a `RangeError`. Native conversion errors are ordinary errors with no
|
|
573
|
+
transaction fields, because no connection is open. `new Vector(encodedBytes)` keeps
|
|
574
|
+
the encoded-byte path, and binding that value still validates its encoding.
|
|
575
|
+
|
|
576
|
+
The native adapter uses the Rust value constructors and validates their output.
|
|
577
|
+
JavaScript passes a bounded binary64 buffer, so no JSON number conversion changes a
|
|
578
|
+
value. Tests cover float32 halfway rounding, subnormal underflow and signed zero with
|
|
579
|
+
known IEEE-754 bits. `float64` keeps the smallest positive and negative subnormals, a
|
|
580
|
+
value next to one and the largest finite binary64 value. Quantized conversion can fail
|
|
581
|
+
for finite inputs: the pinned scale calculation overflows for a range from the
|
|
582
|
+
negative to the positive float32 maximum. These errors happen locally and do not
|
|
583
|
+
change an open transaction.
|
|
584
|
+
|
|
585
|
+
## Embedded client
|
|
586
|
+
|
|
587
|
+
`@romedb/node/embedded-client` opens a local MVCC file. It has the same request
|
|
588
|
+
methods as the URL client (`info`, `read`, `prepareWrite`, `submit`, `recover` and
|
|
589
|
+
`writeAtomic`) and the same value types. It needs the native addon. CommonJS
|
|
590
|
+
and ESM imports both work.
|
|
180
591
|
|
|
181
|
-
|
|
592
|
+
```js
|
|
593
|
+
const { openEmbedded, Record } = require('@romedb/node/embedded-client');
|
|
594
|
+
const rql = await openEmbedded({ path: 'app.romedb', create: true });
|
|
595
|
+
try {
|
|
596
|
+
await rql.writeAtomic([
|
|
597
|
+
{ sql: "INSERT INTO posts {id: posts:hello, title: 'Hello', visits: 1}" },
|
|
598
|
+
{ sql: 'UPDATE posts:hello MERGE {visits: $visits}', params: { $visits: 2n } },
|
|
599
|
+
]);
|
|
600
|
+
const { results } = await rql.read([
|
|
601
|
+
{ sql: 'SELECT title, visits FROM posts WHERE id = $id', params: { $id: new Record('posts', 'hello') } },
|
|
602
|
+
]);
|
|
603
|
+
console.log(results[0].rows); // [ [ 'Hello', 2n ] ]
|
|
604
|
+
} finally {
|
|
605
|
+
await rql.close();
|
|
606
|
+
}
|
|
607
|
+
```
|
|
182
608
|
|
|
183
|
-
|
|
609
|
+
### Opening and creating files
|
|
184
610
|
|
|
185
|
-
|
|
611
|
+
`openEmbedded({ path })` opens an existing regular file with one link. To create a
|
|
612
|
+
new file, pass `create: true`. Creation refuses an existing file and leftover
|
|
613
|
+
journal files. Files use MVCC, the journal mode of `romedb serve` and RomeDB Cloud.
|
|
614
|
+
A file moves between these without conversion and gets the same write conflicts.
|
|
186
615
|
|
|
187
|
-
|
|
616
|
+
Opening a WAL file fails with `RDB_UNSUPPORTED` and leaves the file unchanged. Stop
|
|
617
|
+
its owner and convert it with `romedb migrate-to-mvcc`. The managed connection turns
|
|
618
|
+
on SQL foreign keys. The legacy APIs keep their own setting.
|
|
188
619
|
|
|
189
|
-
|
|
620
|
+
### Private files
|
|
190
621
|
|
|
191
|
-
|
|
622
|
+
`create: true` makes the database file with mode 0600. The WAL file (`NAME-wal`) and
|
|
623
|
+
the MVCC log (`STEM.db-log`, which holds row data until a checkpoint) get the
|
|
624
|
+
permissions of the database file, as in SQLite. RomeDB creates them before the
|
|
625
|
+
engine does. On each open, it removes permission bits that the database file does
|
|
626
|
+
not have from existing ones. So every file of a new database is 0600, and a database
|
|
627
|
+
made by 0.2.11 or earlier gets private journal files at its next open.
|
|
192
628
|
|
|
193
|
-
|
|
194
|
-
|---|---|
|
|
195
|
-
| Query and row helpers | `execute(sql, parameters, options)`, `all(sql, parameters, options)`, `first(sql, parameters, options)`, `exactlyOne(sql, parameters, options)` |
|
|
196
|
-
| Query profiling | `profileSelect(sql, parameters, options)` |
|
|
197
|
-
| Collection integrity | `checkCollectionIntegrity(table, limits, options)` |
|
|
198
|
-
| Script batch | `executeBatch(script, options)` |
|
|
199
|
-
| Document export | `exportDocuments(table, format, options)` |
|
|
200
|
-
| Document import | `importDocuments(table, input, format, options)` |
|
|
201
|
-
| Migrations | `migrate(plan, options)` |
|
|
629
|
+
### Open files
|
|
202
630
|
|
|
203
|
-
|
|
631
|
+
One process keeps at most `maxOpen` different files open: 1 to 1024, 16 by default.
|
|
632
|
+
An open over this number fails with `RDB_LIMIT`. Clients of a file that is already
|
|
633
|
+
open do not count again. To keep many databases, such as one per site, open them
|
|
634
|
+
with the same `maxOpen` and close the least recently used one yourself.
|
|
204
635
|
|
|
205
|
-
`
|
|
636
|
+
`close()` of the last client checkpoints the MVCC log into the database file, so the
|
|
637
|
+
next open has no log to replay. Then it releases the OS lock of the file. Reopening a
|
|
638
|
+
closed file takes about 75 to 90 ms on Linux x64 (release build, 2026-11-02), nearly
|
|
639
|
+
independent of its size. Each open file has its own worker thread, and starting it
|
|
640
|
+
and loading the addon in it takes about 60 ms of that. The engine opens in about
|
|
641
|
+
9 ms, and closing with the checkpoint takes about 10 ms.
|
|
206
642
|
|
|
207
|
-
|
|
643
|
+
### Batches and sequences
|
|
208
644
|
|
|
209
|
-
|
|
645
|
+
Clients of the same canonical file in one Node module share one native worker. Each
|
|
646
|
+
batch runs as one worker operation inside a managed transaction. All statements are
|
|
647
|
+
validated before any runs, and user transaction control is refused. A failure or an
|
|
648
|
+
exceeded result or work limit rolls back the whole batch. The `executeBatch`
|
|
649
|
+
behavior of the legacy API does not change.
|
|
210
650
|
|
|
651
|
+
Sequence conditions cover writes through this shared adapter. Its capabilities
|
|
652
|
+
report `sequenceScope: 'adapter'` and `replayDurability: 'adapter-lifetime'`. Other
|
|
653
|
+
native or legacy handles do not advance its sequence. An application must use one
|
|
654
|
+
managed adapter for each canonical file. To share a file between processes, use the
|
|
655
|
+
service URL.
|
|
211
656
|
|
|
212
|
-
|
|
657
|
+
`info()` also returns `engineVersion`, the RomeDB release of the native engine.
|
|
658
|
+
Database URLs report it from local services 2.7.0 and later and from Cloud. Older
|
|
659
|
+
services do not report it.
|
|
213
660
|
|
|
661
|
+
### Write recovery
|
|
214
662
|
|
|
215
|
-
|
|
663
|
+
The adapter keeps 64 exact write receipts in memory. A saved journal can recover a
|
|
664
|
+
write while that adapter stays open. After its last client closes, reopening makes a
|
|
665
|
+
new identity and refuses old requests. After a reopen, reconcile uncertain writes
|
|
666
|
+
from the stored data. Service and Cloud connections recover writes across process
|
|
667
|
+
restarts.
|
|
216
668
|
|
|
669
|
+
### Limits and closing
|
|
217
670
|
|
|
218
|
-
|
|
671
|
+
Each adapter accepts 32 outstanding requests and 2 MiB of queued request bodies, and
|
|
672
|
+
a process has 16 open file adapters by default. The request, result and work limits
|
|
673
|
+
match the common contract. Calls time out after 20 seconds by default and accept a
|
|
674
|
+
`timeoutMs` of 1 to 120,000. Opening also has a deadline. If opening expires, cleanup
|
|
675
|
+
continues until the native startup returns. Filesystem work and native cleanup do
|
|
676
|
+
not stop early.
|
|
219
677
|
|
|
678
|
+
`close()` stops requests from that client and cancels its outstanding work. The last
|
|
679
|
+
client runs the remaining worker queue, closes the native handle and waits for the
|
|
680
|
+
worker to exit. Await `close()` of every client to release the file.
|
|
220
681
|
|
|
221
|
-
|
|
682
|
+
### Copying a database into a file
|
|
222
683
|
|
|
684
|
+
`exportFile` copies the open database into a new `.romedb` file while reads and
|
|
685
|
+
writes continue. The file is one snapshot, and it opens by itself in `openEmbedded`
|
|
686
|
+
and `romedb serve FILE`.
|
|
223
687
|
|
|
224
|
-
|
|
688
|
+
```js
|
|
689
|
+
const { sequence, bytes } = await rql.exportFile('backups/site-2026-11-03.romedb');
|
|
690
|
+
// Or stream the copy, for example as a download. Nothing stays on disk afterwards.
|
|
691
|
+
const { size, body } = await rql.exportFile({ signal });
|
|
692
|
+
```
|
|
225
693
|
|
|
226
|
-
|
|
694
|
+
The file holds every write up to and including `sequence`, and none after it. The
|
|
695
|
+
destination and its `-wal`, `-shm`, `-log`, `-journal` and `.db-log` files must not
|
|
696
|
+
exist, or the call fails with `RDB_ALREADY_EXISTS`. The new file has mode 0600 and
|
|
697
|
+
appears only when it is complete.
|
|
227
698
|
|
|
699
|
+
The streaming form builds its copy in a private folder beside the database file. It
|
|
700
|
+
removes the name of the copy before it returns, so read `body` to the end or cancel
|
|
701
|
+
it. Exports have no deadline unless you pass `timeoutMs`. `signal`, `timeoutMs` and
|
|
702
|
+
`close()` cancel a running export. It then rejects with `RDB_CANCELLED`, unless it
|
|
703
|
+
finished first.
|
|
228
704
|
|
|
229
|
-
The
|
|
705
|
+
The URL client has the streaming form, `rql.exportFile({ signal, timeoutMs })`, for
|
|
706
|
+
`romedb serve` and `serve --directory`. Cloud URLs do not support it yet. See
|
|
707
|
+
[database files](https://github.com/romedb/romedb/blob/main/romedb/docs/service-protocol.md#database-files) for how the copy is
|
|
708
|
+
made.
|
|
230
709
|
|
|
231
|
-
##
|
|
710
|
+
## URL client
|
|
232
711
|
|
|
233
|
-
|
|
712
|
+
`@romedb/node/client` connects to a database behind `romedb serve`, `romedb serve
|
|
713
|
+
--directory` or RomeDB Cloud. Importing it never loads the native addon.
|
|
234
714
|
|
|
235
|
-
```
|
|
236
|
-
|
|
237
|
-
|
|
715
|
+
```js
|
|
716
|
+
const { connect, Float64, saveRequest, loadRequest } = require('@romedb/node/client');
|
|
717
|
+
const rql = await connect({ url: process.env.ROMEDB_URL, token: process.env.ROMEDB_TOKEN });
|
|
718
|
+
const snapshot = await rql.read([{ sql: 'SELECT * FROM posts LIMIT 20' }]);
|
|
719
|
+
const request = rql.prepareWrite([
|
|
720
|
+
{ sql: 'UPDATE posts:p1 MERGE {score: $score}', params: { $score: new Float64(1) } },
|
|
721
|
+
], { ifSequence: snapshot.sequence });
|
|
722
|
+
await saveRequest('/private/new-request.json', request);
|
|
723
|
+
try {
|
|
724
|
+
await rql.submit(request);
|
|
725
|
+
} catch (error) {
|
|
726
|
+
if (error.outcome === 'unknown') {
|
|
727
|
+
// Keep this journal. Recovery sends its original identity and exact body.
|
|
728
|
+
await rql.recover(await loadRequest('/private/new-request.json'));
|
|
729
|
+
} else throw error;
|
|
730
|
+
}
|
|
731
|
+
await rql.close();
|
|
238
732
|
```
|
|
239
733
|
|
|
240
|
-
|
|
734
|
+
### Saved requests
|
|
241
735
|
|
|
242
|
-
|
|
736
|
+
Use a new journal file name for each prepared write. `saveRequest` refuses to
|
|
737
|
+
overwrite an existing file, and it flushes a mode-0600 file and its folder. A journal
|
|
738
|
+
holds SQL and parameter values, so keep it private, although it holds no token. A
|
|
739
|
+
refused recovery does not tell the outcome of the earlier write. Do not replace an
|
|
740
|
+
expired or uncertain request with a new write automatically.
|
|
243
741
|
|
|
244
|
-
|
|
245
|
-
python3 romedb/scripts/audit-crate-notices.py romedb/docs/node-dependencies-linux-x64.json romedb/docs/node-crate-notices-linux-x64.json
|
|
246
|
-
python3 romedb/scripts/audit-crate-notices.py romedb/docs/node-dependencies-linux-x64.json romedb/docs/node-crate-notices-linux-x64.json --check
|
|
247
|
-
```
|
|
742
|
+
### URLs and tokens
|
|
248
743
|
|
|
249
|
-
|
|
744
|
+
A local service URL is `http://127.0.0.1:4321`. A Cloud URL names the organization
|
|
745
|
+
and database by stable UUIDs, and a dashboard URL that ends in `/query` also works.
|
|
746
|
+
Use HTTPS for every host that is not local.
|
|
250
747
|
|
|
251
|
-
|
|
748
|
+
`romedb://` is another name for `https://`. The client replaces the scheme and applies
|
|
749
|
+
the same rules, so `romedb://cloud.romedb.com/v1/organizations/{org}/databases/{db}`
|
|
750
|
+
and the `https://` form are the same database. Requests use `redirect: 'manual'`, and
|
|
751
|
+
any redirect fails with `RDB_PROTOCOL`, so the token never goes to another URL. Pass
|
|
752
|
+
the bearer token separately. The client refuses a token in the URL.
|
|
252
753
|
|
|
253
|
-
|
|
754
|
+
### Cloud URLs
|
|
254
755
|
|
|
255
|
-
|
|
256
|
-
|
|
257
|
-
|
|
258
|
-
```
|
|
756
|
+
For a Cloud URL, `connect()` returns without a network round trip. The database ID
|
|
757
|
+
comes from the URL, and the Cloud capabilities are documented. Reads start from
|
|
758
|
+
sequence 0 as their lower bound and update the sequence from each reply.
|
|
259
759
|
|
|
260
|
-
|
|
760
|
+
A write needs a current sequence. If none is known yet, `writeAtomic` first fetches
|
|
761
|
+
the database details, and `prepareWrite` throws `RDB_SEQUENCE_UNKNOWN` (outcome
|
|
762
|
+
`not-sent`) until a read, `info()` or an explicit `afterSequence` supplies one. To
|
|
763
|
+
fetch the details at connect, pass `deferInfo: false`. Other URLs always fetch the
|
|
764
|
+
details at connect and refuse `deferInfo: true`.
|
|
261
765
|
|
|
262
|
-
|
|
766
|
+
`connect()` on a Cloud URL also sends a warm request in the background and does not
|
|
767
|
+
wait for it. Cloud then starts the container of the database and opens it, so a query
|
|
768
|
+
a second or more later does not wait for that start. Errors of the background request
|
|
769
|
+
are ignored. To skip it, pass `warm: false`.
|
|
263
770
|
|
|
264
|
-
|
|
265
|
-
|
|
266
|
-
|
|
771
|
+
`await rql.warm()` sends the same request and returns `{ state }`. The state is
|
|
772
|
+
`ready` when the database is already open, `starting` when Cloud began to open it, or
|
|
773
|
+
`skipped` when Cloud declined because another database is busy. Call it when your
|
|
774
|
+
application knows that a query is coming, for example when a page or a function
|
|
775
|
+
starts. Local URLs refuse `warm()`.
|
|
776
|
+
|
|
777
|
+
### Local directories
|
|
267
778
|
|
|
268
|
-
|
|
779
|
+
An organization URL on a loopback host, such as a database of `romedb serve
|
|
780
|
+
--directory` (`http://127.0.0.1:4321/v1/organizations/{org}/databases/{db}`), always
|
|
781
|
+
fetches the details at connect. A reply with `serviceVersion: 1` is a local service.
|
|
782
|
+
The client then negotiates its capabilities and uses the 64 KiB service request
|
|
783
|
+
format, with no warm request. Any other reply is Cloud running locally, which keeps
|
|
784
|
+
the Cloud request format. So an application can use the same organization URLs for a
|
|
785
|
+
local directory and for Cloud.
|
|
269
786
|
|
|
270
|
-
|
|
787
|
+
### Values and results
|
|
271
788
|
|
|
272
|
-
|
|
789
|
+
Results use positional rows and keep duplicate columns. Integer results are `bigint`.
|
|
790
|
+
Number results are `Float64` instances with a `.value` number. They keep their exact
|
|
791
|
+
float type when you bind them again, integral floats and negative zero included.
|
|
273
792
|
|
|
274
|
-
|
|
275
|
-
|
|
276
|
-
|
|
277
|
-
|
|
793
|
+
This entry exports its own `Record`, `Vector` and value codec. Use those types for its
|
|
794
|
+
parameters. Encoded vectors and the dense float32, float64 and sparse-entry factories
|
|
795
|
+
work without native code. `null` is a present field. `undefined` and array holes are
|
|
796
|
+
refused, so no data disappears without an error.
|
|
278
797
|
|
|
279
|
-
|
|
798
|
+
### Errors and timeouts
|
|
280
799
|
|
|
800
|
+
Requests and responses have finite byte and row limits. Each call times out after 20
|
|
801
|
+
seconds by default and accepts `signal` and `timeoutMs`. An HTTP error reply with a
|
|
802
|
+
`Retry-After` header sets `ClientError.retryAfterMs`. A transport or response failure
|
|
803
|
+
without a database reply keeps the underlying error as `ClientError.cause`, for
|
|
804
|
+
example a `fetch failed` TypeError with `ECONNREFUSED`.
|
|
281
805
|
|
|
282
|
-
|
|
806
|
+
`rql.sequenceKnown` is `true` once a reply has given the current sequence. `rql.cloud`
|
|
807
|
+
tells a Cloud database from a local service. A write cancelled after it was sent stays
|
|
808
|
+
unknown until a receipt confirms it. Cloud capabilities are marked as documented,
|
|
809
|
+
because its API does not negotiate the local capability object. See the
|
|
810
|
+
[service contract](https://github.com/romedb/romedb/blob/main/romedb/docs/service-protocol.md) for limits, receipt rules and
|
|
811
|
+
compatibility gaps.
|
|
283
812
|
|
|
284
|
-
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
|
|
813
|
+
### Query versions
|
|
814
|
+
|
|
815
|
+
From RomeDB 2.9.0 the client asks Cloud for query version 3
|
|
816
|
+
(`x-romedb-query-version: 3` on `/query` and `/read`). In version 3, Binary and
|
|
817
|
+
Vector values travel as canonical base64, and request bodies can be up to 1 MiB.
|
|
818
|
+
Local services and embedded databases keep version 2, with byte arrays and 64 KiB
|
|
819
|
+
bodies.
|
|
820
|
+
|
|
821
|
+
Version 2 covers tagged Record keys and typed query errors. Resource URLs stay `/v1`,
|
|
822
|
+
and `readVersion: 2` separately defines fresh-read execution. The client accepts the
|
|
823
|
+
version it asked for, version 2, and older responses without the header, and it
|
|
824
|
+
decodes both byte forms. An unknown version fails with `RDB_PROTOCOL`. A write that
|
|
825
|
+
was sent then stays `unknown` and keeps its exact saved request for recovery.
|
|
826
|
+
|
|
827
|
+
Saved writes are version 2 and record their query version. Version 1 saved writes
|
|
828
|
+
from 2.8.0 replay at version 2. Older receipts with enum keys still decode.
|
|
829
|
+
|
|
830
|
+
### Workers, browsers and other runtimes
|
|
831
|
+
|
|
832
|
+
From 0.2.12, `@romedb/node/client` and `@romedb/node/values` use only Web APIs:
|
|
833
|
+
`fetch`, `crypto.randomUUID`, `TextEncoder`, `AbortSignal`, `atob` and `btoa`. Node.js
|
|
834
|
+
loads the CommonJS files through the `node` export condition. Every other runtime and
|
|
835
|
+
bundler gets the ES modules `client.mjs` and `values.mjs`, for example Cloudflare
|
|
836
|
+
Workers, browsers and Deno with a bundler. They run the same code: `build-esm.cjs`
|
|
837
|
+
writes them from `client.cjs` and `values.cjs`, and the package tests fail when they
|
|
838
|
+
are out of date.
|
|
839
|
+
|
|
840
|
+
Workers need no `nodejs_compat`. The client calls `fetch` without a `this` value, so
|
|
841
|
+
`globalThis.fetch` and your own `fetch` both work in Workers.
|
|
288
842
|
|
|
289
843
|
```js
|
|
290
|
-
|
|
291
|
-
|
|
292
|
-
|
|
293
|
-
)
|
|
844
|
+
import { connect } from '@romedb/node/client';
|
|
845
|
+
|
|
846
|
+
export default {
|
|
847
|
+
async fetch(request, env) {
|
|
848
|
+
const rql = await connect({ url: env.ROMEDB_URL, token: env.ROMEDB_TOKEN });
|
|
849
|
+
const { results } = await rql.read([{ sql: "SELECT title FROM posts WHERE array::contains(tags, 'news')" }]);
|
|
850
|
+
return Response.json({ posts: results[0].rows.length });
|
|
851
|
+
},
|
|
852
|
+
};
|
|
294
853
|
```
|
|
295
854
|
|
|
296
|
-
|
|
297
|
-
|
|
298
|
-
|
|
299
|
-
and duplicate occurrences count toward the final payload budget; missing targets
|
|
300
|
-
count as nulls. Found targets are charged with their output multiplicity before
|
|
301
|
-
cache retention, allowing early rejection during target reads. Reference
|
|
302
|
-
placeholders are excluded. Existing execute/profile
|
|
303
|
-
methods keep their current result policy.
|
|
304
|
-
|
|
305
|
-
Payload bytes count UTF-8 column names, strings, object keys, record table names
|
|
306
|
-
and string keys; integer keys and numbers cost eight bytes, null and booleans one
|
|
307
|
-
byte, and binary/encoded vectors their byte length. Objects and arrays sum their
|
|
308
|
-
contents. These are logical result budgets, excluding container/allocator
|
|
309
|
-
costs, temporary decoding, Node transport copies and engine working memory.
|
|
310
|
-
FETCH retains its independent reference and tagged-JSON limits; temporary
|
|
311
|
-
references and the target cache are not covered by the final payload budget.
|
|
312
|
-
These limits are not process memory limits or deadlines.
|
|
855
|
+
`@romedb/node/values` exports only the value types and the codec (`Float64`, `Record`,
|
|
856
|
+
`Vector`, `encodeValue`, `decodeValue`). Use it for code that never connects, such as
|
|
857
|
+
a browser bundle. Both entries share one set of value classes.
|
|
313
858
|
|
|
859
|
+
Binary values and `Vector.bytes` are plain `Uint8Array` objects in every runtime.
|
|
860
|
+
Before 0.2.12, Node returned `Buffer`. Use `Buffer.from(bytes)` where you need Buffer
|
|
861
|
+
methods. `saveRequest` and `loadRequest` need Node.js file access and reject in other
|
|
862
|
+
runtimes. There, store the `PreparedWrite` object yourself, for example in a Durable
|
|
863
|
+
Object.
|
|
314
864
|
|
|
315
|
-
|
|
865
|
+
The package also contains the native addon. So installing it for a Workers app also
|
|
866
|
+
downloads the addon, but the Workers bundle never includes it.
|
|
316
867
|
|
|
317
|
-
|
|
318
|
-
version accepts a fourth `{ signal }` argument. Use the same required bigint
|
|
319
|
-
`maxRows` and `maxPayloadBytes` fields as bounded SELECT. Supported SQL data
|
|
320
|
-
writes and object writes run in an operation savepoint. Native SQL checks each
|
|
321
|
-
result row before frontend retention; collection SQL and object RETURNING check
|
|
322
|
-
each projected row before retention. Native destinations with logical sources
|
|
323
|
-
also check rows during final statement collection. In all cases, overflow returns `FDB_LIMIT` and rolls back that write,
|
|
324
|
-
including index and trigger effects, using the existing recovery path.
|
|
868
|
+
## Load errors
|
|
325
869
|
|
|
326
|
-
|
|
327
|
-
|
|
328
|
-
|
|
329
|
-
|
|
330
|
-
|
|
870
|
+
On Node.js versions before 22, importing the native entries throws
|
|
871
|
+
`RDB_RUNTIME_VERSION` before native code loads.
|
|
872
|
+
|
|
873
|
+
If the native addon is missing or cannot load, importing the package throws
|
|
874
|
+
`RDB_NATIVE_LOAD`. The message names the platform, the architecture and the Node
|
|
875
|
+
version, and it points to the source build instructions. `error.cause` keeps the
|
|
876
|
+
original loader error, for example a missing library or an invalid binary. The package
|
|
877
|
+
does not select, download or rebuild an addon automatically.
|
|
878
|
+
|
|
879
|
+
## For maintainers
|
|
880
|
+
|
|
881
|
+
### Build and test from the checkout
|
|
882
|
+
|
|
883
|
+
From the repository root, install the declaration checker and run the checks:
|
|
884
|
+
|
|
885
|
+
```sh
|
|
886
|
+
pnpm --dir romedb/bindings/node install --ignore-scripts
|
|
887
|
+
romedb/scripts/check.sh
|
|
331
888
|
```
|
|
332
889
|
|
|
333
|
-
|
|
334
|
-
|
|
335
|
-
|
|
336
|
-
|
|
337
|
-
affected rows do not count as returned rows. Empty RETURNING still charges column
|
|
338
|
-
names. Cooperative cancellation uses the usual error/transaction envelope and
|
|
339
|
-
savepoint recovery; completed-result accounting has no fixed cancellation latency.
|
|
890
|
+
`check-node.sh` builds the addon and copies it to the ignored `romedb.node` path. Then
|
|
891
|
+
it runs the Node tests and type-checks the declarations. If you need the temporary
|
|
892
|
+
toolchain, use the documented Rust environment of the repository. The package has no
|
|
893
|
+
npm runtime dependencies. TypeScript 5.8.3 is pinned only for the declaration checks.
|
|
340
894
|
|
|
341
|
-
###
|
|
895
|
+
### Local package smoke
|
|
342
896
|
|
|
343
|
-
|
|
897
|
+
Normal npm packing first makes sure that the native addon loads. A missing or
|
|
898
|
+
incompatible build fails before the archive exists. The npm file list holds the native
|
|
899
|
+
addon, the synchronous and worker JavaScript entries, the declarations, the README,
|
|
900
|
+
the MIT license and the third-party notices. Build the addon for the current
|
|
901
|
+
platform, then test the actual tarball from the repository root:
|
|
344
902
|
|
|
345
|
-
```
|
|
346
|
-
|
|
347
|
-
|
|
348
|
-
const workerDb = await AsyncDatabase.open(':memory:', options);
|
|
903
|
+
```sh
|
|
904
|
+
romedb/scripts/check-node.sh
|
|
905
|
+
node romedb/scripts/check-node-package.cjs
|
|
349
906
|
```
|
|
350
907
|
|
|
351
|
-
The
|
|
352
|
-
|
|
353
|
-
|
|
354
|
-
|
|
355
|
-
|
|
356
|
-
|
|
357
|
-
|
|
908
|
+
The smoke packs into a temporary folder and compares the exact file list. It installs
|
|
909
|
+
the tarball into a separate consumer with npm offline and lifecycle scripts off. Then
|
|
910
|
+
it tests public synchronous and worker queries, typed int64 and record values,
|
|
911
|
+
rollback and reopen. It also covers profiling and audits in both clients, cancellation
|
|
912
|
+
before dispatch for every signalled API, and listener and token cleanup. It compiles a
|
|
913
|
+
consumer TypeScript import against the installed declarations. It removes its
|
|
914
|
+
temporary files afterwards.
|
|
358
915
|
|
|
359
|
-
|
|
360
|
-
|
|
361
|
-
|
|
362
|
-
|
|
916
|
+
For an optimized local addon, run `romedb/scripts/check-node.sh --release` before the
|
|
917
|
+
smoke. That build uses the pinned release profile of the workspace and runs the same
|
|
918
|
+
tests and TypeScript type check. It does not build universal prebuilds or publish a package.
|
|
919
|
+
On Linux, `--release-stripped` also runs `strip --strip-debug` on the package copy. It
|
|
920
|
+
keeps the full release artifact in the Cargo target folder for diagnostics, and it
|
|
921
|
+
needs `strip`.
|
|
363
922
|
|
|
364
|
-
|
|
923
|
+
The smoke is a maintainer test, separate from routine CI, because the debug addon is
|
|
924
|
+
large. The tested local artifact is Linux x64 with Node 24.19.0, not a universal
|
|
925
|
+
binary. Prebuilds for other platforms, more Node versions and platforms, and complete
|
|
926
|
+
distribution notices are release work. Releases reach npm through the publishing
|
|
927
|
+
workflow, which [publishing to npm](https://github.com/romedb/romedb/blob/main/romedb/docs/npm-publishing.md) describes.
|
|
365
928
|
|
|
366
|
-
|
|
367
|
-
number from 0 through 4,294,967,295. For example:
|
|
929
|
+
### Dependency declaration inventory
|
|
368
930
|
|
|
369
|
-
|
|
370
|
-
|
|
931
|
+
From the checkout root, with the pinned Rust toolchain:
|
|
932
|
+
|
|
933
|
+
```sh
|
|
934
|
+
node romedb/scripts/inventory-node-dependencies.cjs x86_64-unknown-linux-gnu romedb/docs/node-dependencies-linux-x64.json
|
|
935
|
+
node romedb/scripts/inventory-node-dependencies.cjs x86_64-unknown-linux-gnu romedb/docs/node-dependencies-linux-x64.json --check
|
|
371
936
|
```
|
|
372
937
|
|
|
373
|
-
|
|
374
|
-
|
|
375
|
-
|
|
376
|
-
path and returns `FDB_CANCELLED`, with normal operation rollback and transaction
|
|
377
|
-
observations. `signal` can be supplied alongside it. No JavaScript timeout timer
|
|
378
|
-
is needed; token cleanup follows request completion/failure. Timeout expiry does
|
|
379
|
-
not cancel later requests. `close()` has no timeout option.
|
|
380
|
-
|
|
381
|
-
This is not a hard promise-return deadline. Native progress checks observe expiry;
|
|
382
|
-
parsing, non-engine work, cleanup and response transport can exceed the requested
|
|
383
|
-
time, and completion can win a race. Synchronous clients do not expose this option.
|
|
384
|
-
|
|
385
|
-
### URL client for the Studio service work
|
|
386
|
-
|
|
387
|
-
The new `@romedb/node/client` entry supports the current Cloud read-v2/write
|
|
388
|
-
envelopes and the version-1 local service contract. The URL client, native service,
|
|
389
|
-
and embedded adapter are implemented in development source. They are not released.
|
|
390
|
-
The hosted common fixture and production Studio browser workflow pass. Existing
|
|
391
|
-
native imports are unchanged. Final release-candidate packaging remains pending.
|
|
392
|
-
|
|
393
|
-
Since 2.9.0 the SDK asks Cloud for query version 3 (`x-romedb-query-version: 3`
|
|
394
|
-
on `/query` and `/read`): Binary and Vector values travel as canonical base64 and
|
|
395
|
-
request bodies may be up to 1 MiB. Local services and embedded databases keep
|
|
396
|
-
version 2 (byte arrays, 64 KiB). Version 2 covers tagged Record keys and typed
|
|
397
|
-
query errors; resource URLs remain `/v1`, and `readVersion: 2` separately defines
|
|
398
|
-
fresh-read execution. The SDK accepts the version it asked for, version 2, and
|
|
399
|
-
historical responses without the header, and decodes both byte forms. An unknown
|
|
400
|
-
advertised version fails with `FDB_PROTOCOL`; a dispatched write remains `unknown`
|
|
401
|
-
and retains its exact saved request for recovery. Saved writes are version 2 and
|
|
402
|
-
record their query version; version 1 saved writes from 2.8.0 replay at version 2.
|
|
403
|
-
Historical enum-key receipts still decode.
|
|
938
|
+
This offline, locked Cargo query records normal and build dependencies, their declared
|
|
939
|
+
license expressions, the target and the lockfile hash. The Linux inventory has 192
|
|
940
|
+
package versions. Build tools are included, and development dependencies are not.
|
|
404
941
|
|
|
405
|
-
|
|
406
|
-
|
|
407
|
-
|
|
408
|
-
|
|
409
|
-
|
|
410
|
-
|
|
411
|
-
|
|
412
|
-
|
|
413
|
-
|
|
414
|
-
|
|
415
|
-
|
|
416
|
-
|
|
417
|
-
|
|
418
|
-
|
|
419
|
-
|
|
420
|
-
|
|
421
|
-
await db.close();
|
|
942
|
+
`cfg_block` 0.1.1 has no license expression in the query output. Its pinned manifest
|
|
943
|
+
declares `license-file = "LICENSE"` instead, and that file contains an Apache 2.0
|
|
944
|
+
notice. The package notice file includes it and the Apache 2.0 text. The inventory
|
|
945
|
+
keeps the missing expression rather than an inferred value.
|
|
946
|
+
|
|
947
|
+
The inventory is an input for an audit. It does not prove that every listed crate is
|
|
948
|
+
linked into the addon, and it is not a complete component inventory or a replacement
|
|
949
|
+
for license texts. Bundled C and C++ sources and other vendored components need their
|
|
950
|
+
own inspection. Regenerate the inventory when dependencies change, and qualify other
|
|
951
|
+
targets separately.
|
|
952
|
+
|
|
953
|
+
With Python 3.11 or later and the same Cargo cache, inspect the cached source archives:
|
|
954
|
+
|
|
955
|
+
```sh
|
|
956
|
+
python3 romedb/scripts/audit-crate-notices.py romedb/docs/node-dependencies-linux-x64.json romedb/docs/node-crate-notices-linux-x64.json
|
|
957
|
+
python3 romedb/scripts/audit-crate-notices.py romedb/docs/node-dependencies-linux-x64.json romedb/docs/node-crate-notices-linux-x64.json --check
|
|
422
958
|
```
|
|
423
959
|
|
|
424
|
-
|
|
425
|
-
|
|
426
|
-
|
|
427
|
-
|
|
428
|
-
|
|
429
|
-
|
|
430
|
-
|
|
431
|
-
|
|
432
|
-
|
|
433
|
-
|
|
434
|
-
|
|
435
|
-
|
|
436
|
-
|
|
437
|
-
|
|
438
|
-
|
|
439
|
-
|
|
440
|
-
|
|
441
|
-
|
|
442
|
-
from sequence 0 as their lower bound and update the sequence from each reply. A
|
|
443
|
-
write needs a current sequence: `writeAtomic` fetches the database details first
|
|
444
|
-
when none is known yet, and `prepareWrite` throws `FDB_SEQUENCE_UNKNOWN` (outcome
|
|
445
|
-
`not-sent`) until a read, `info()` or an explicit `afterSequence` supplies one.
|
|
446
|
-
Pass `deferInfo: false` to request the details at connect, as 2.5 did by default.
|
|
447
|
-
Other URLs always request the details at connect, and `deferInfo: true` is
|
|
448
|
-
rejected for them.
|
|
449
|
-
|
|
450
|
-
An organization URL on a loopback host, such as a database of
|
|
451
|
-
`romedb serve --directory` (`http://127.0.0.1:4321/v1/organizations/{org}/databases/{db}`),
|
|
452
|
-
always requests the details at connect. A reply with `serviceVersion: 1` is a local
|
|
453
|
-
service: the client negotiates its capabilities and uses the 64 KiB service request
|
|
454
|
-
format, with no warm request. Any other reply is Cloud running locally, which keeps
|
|
455
|
-
the Cloud request format. Application code can therefore use the same organization
|
|
456
|
-
URLs for a local directory and for Cloud.
|
|
457
|
-
|
|
458
|
-
`connect()` on a Cloud URL also sends a warm request in the background and does
|
|
459
|
-
not wait for it. Cloud then starts the database's container and opens the
|
|
460
|
-
database, so a query sent a second or more later skips that start-up time. Errors
|
|
461
|
-
from the background request are ignored; pass `warm: false` to skip it.
|
|
462
|
-
`await db.warm()` sends the same request and returns `{ state }`: `ready` when the
|
|
463
|
-
database is already open, `starting` when Cloud began opening it, or `skipped`
|
|
464
|
-
when Cloud declined because another database is busy. Call it when your
|
|
465
|
-
application knows a query is coming, for example when a page or a function
|
|
466
|
-
starts. `warm()` is rejected for local URLs.
|
|
467
|
-
|
|
468
|
-
Results use positional rows, preserving duplicate columns. Integer results are
|
|
469
|
-
bigint. Number results are explicit `Float64` instances with a `.value` number;
|
|
470
|
-
they retain their exact float type when rebound, including integral floats and
|
|
471
|
-
negative zero. This new entry exports its own `Record`, `Vector` and value codec.
|
|
472
|
-
Use those types for its parameters. Encoded vectors and dense float32/float64 or
|
|
473
|
-
sparse-entry factories work without native code. Null is a present field;
|
|
474
|
-
undefined and array holes are rejected instead of silently removing data.
|
|
475
|
-
|
|
476
|
-
Requests and responses have finite byte/row limits. Each call defaults to a
|
|
477
|
-
20-second timeout and accepts `signal`/`timeoutMs`. An HTTP error reply with a
|
|
478
|
-
`Retry-After` header sets `ClientError.retryAfterMs`. A transport or response
|
|
479
|
-
failure without a database reply keeps the underlying error as `ClientError.cause`
|
|
480
|
-
(for example a `fetch failed` TypeError with `ECONNREFUSED`). `db.sequenceKnown`
|
|
481
|
-
is true once a reply has given the current sequence, and `db.cloud` tells a Cloud
|
|
482
|
-
database from a local service. Cancellation after dispatch
|
|
483
|
-
leaves a write unknown until a receipt confirms it. Cloud capabilities are marked
|
|
484
|
-
as documented, because its API currently does not negotiate the local capability
|
|
485
|
-
object. See the [service contract](../../docs/service-protocol.md) for limits,
|
|
486
|
-
receipt rules and compatibility gaps.
|
|
960
|
+
The audit compares archive checksums with `Cargo.lock`. It records declared license-file
|
|
961
|
+
paths and hashes of files whose names look like licenses or notices, nested bundled
|
|
962
|
+
sources included. It reads the archives without extracting them. Missing or ambiguous
|
|
963
|
+
archives, checksum mismatches and stale reports fail. The current report has 185
|
|
964
|
+
archives with 316 candidate files. 13 archives have no candidates, and seven workspace
|
|
965
|
+
packages need separate inspection. A name search can miss inline notices and unusual
|
|
966
|
+
names, and a found file does not prove that the package has every required notice.
|
|
967
|
+
The report stays outside the npm package.
|
|
968
|
+
|
|
969
|
+
For archives without candidates, the report also searches selected code and text files
|
|
970
|
+
of up to 200,000 bytes for copyright, SPDX and license phrases. It records source
|
|
971
|
+
hashes, line numbers and short excerpts. The current report finds four files in
|
|
972
|
+
crc32c, rapidhash and rquickjs-core: crc32c refers to zlib, and rquickjs-core has an
|
|
973
|
+
attribution to the futures authors. These references need review. Ten of the thirteen
|
|
974
|
+
archives have no hits in this bounded search, and no hit is not proof that no
|
|
975
|
+
attribution is needed.
|
|
976
|
+
|
|
977
|
+
Generate and test the collected crate notice texts:
|
|
487
978
|
|
|
488
|
-
|
|
979
|
+
```sh
|
|
980
|
+
python3 romedb/scripts/bundle-crate-notices.py romedb/docs/node-dependencies-linux-x64.json romedb/docs/node-crate-notices-linux-x64.json romedb/bindings/node/THIRD_PARTY_CRATE_NOTICES.md
|
|
981
|
+
python3 romedb/scripts/bundle-crate-notices.py romedb/docs/node-dependencies-linux-x64.json romedb/docs/node-crate-notices-linux-x64.json romedb/bindings/node/THIRD_PARTY_CRATE_NOTICES.md --check
|
|
982
|
+
```
|
|
489
983
|
|
|
490
|
-
|
|
491
|
-
|
|
492
|
-
|
|
493
|
-
|
|
494
|
-
|
|
495
|
-
|
|
496
|
-
|
|
497
|
-
The client calls `fetch` without a `this` value, so `globalThis.fetch` and your
|
|
498
|
-
own `fetch` both work in Workers.
|
|
984
|
+
The generator compares the lockfile, inventory, archive and notice-file hashes again
|
|
985
|
+
before it writes. It writes identical texts once, with source links for each package.
|
|
986
|
+
The npm package includes this partial collection: 125 distinct texts for 316 candidate
|
|
987
|
+
files. The offline package smoke compares both installed notice files exactly. The
|
|
988
|
+
twenty packages without collected candidates and any other inline or bundled
|
|
989
|
+
attribution are still review items, so this is not a complete set of distribution
|
|
990
|
+
notices.
|
|
499
991
|
|
|
500
|
-
|
|
501
|
-
import { connect, Float64 } from '@romedb/node/client';
|
|
992
|
+
Run the synthetic tests of the notice generator without Cargo or registry access:
|
|
502
993
|
|
|
503
|
-
|
|
504
|
-
|
|
505
|
-
const db = await connect({ url: env.ROMEDB_URL, token: env.ROMEDB_TOKEN });
|
|
506
|
-
const { results } = await db.read([{ sql: 'SELECT $x AS x', params: { $x: new Float64(1) } }]);
|
|
507
|
-
return Response.json({ rows: results[0].rows.length });
|
|
508
|
-
},
|
|
509
|
-
};
|
|
994
|
+
```sh
|
|
995
|
+
python3 romedb/scripts/test-notice-bundle.py
|
|
510
996
|
```
|
|
511
997
|
|
|
512
|
-
|
|
513
|
-
`Vector`, `encodeValue`, `decodeValue`) for code that never connects, such as a
|
|
514
|
-
browser bundle. Both entries share one set of value classes.
|
|
998
|
+
They test these cases:
|
|
515
999
|
|
|
516
|
-
|
|
517
|
-
|
|
518
|
-
|
|
519
|
-
|
|
520
|
-
a Durable Object.
|
|
1000
|
+
- Reproducible output.
|
|
1001
|
+
- Identical texts written once, with their source references.
|
|
1002
|
+
- Detection of a stale inventory or a stale output.
|
|
1003
|
+
- A package identity mismatch, a corrupt archive and changed notice hashes.
|
|
521
1004
|
|
|
522
|
-
|
|
523
|
-
also downloads the addon, which the Workers bundle never includes.
|
|
1005
|
+
A failure case must leave an existing output file unchanged.
|
|
524
1006
|
|
|
525
|
-
###
|
|
1007
|
+
### Linux artifact requirements
|
|
526
1008
|
|
|
527
|
-
|
|
528
|
-
interface. This entry requires the native addon. It exports the same value types
|
|
529
|
-
and `info`, `read`, `prepareWrite`, `submit`, `recover`, and `writeAtomic` methods.
|
|
530
|
-
CommonJS and ESM imports are supported.
|
|
1009
|
+
Inspect the package addon without loading it (Python 3.11 or later and `readelf`):
|
|
531
1010
|
|
|
532
|
-
```
|
|
533
|
-
|
|
534
|
-
|
|
535
|
-
try {
|
|
536
|
-
await db.writeAtomic([
|
|
537
|
-
{ sql: 'UPSERT posts:p1 { count: $count }', params: { $count: 9223372036854775807n } },
|
|
538
|
-
]);
|
|
539
|
-
const result = await db.read([
|
|
540
|
-
{ sql: 'SELECT * FROM posts WHERE id=$id', params: { $id: new Record('posts', 'p1') } },
|
|
541
|
-
]);
|
|
542
|
-
console.log(result.results[0].rows);
|
|
543
|
-
} finally {
|
|
544
|
-
await db.close();
|
|
545
|
-
}
|
|
1011
|
+
```sh
|
|
1012
|
+
python3 romedb/scripts/inspect-node-elf.py romedb/bindings/node/romedb.node romedb/docs/node-elf-linux-x64.json
|
|
1013
|
+
python3 romedb/scripts/inspect-node-elf.py romedb/bindings/node/romedb.node romedb/docs/node-elf-linux-x64.json --check
|
|
546
1014
|
```
|
|
547
1015
|
|
|
548
|
-
|
|
549
|
-
|
|
550
|
-
|
|
551
|
-
|
|
552
|
-
|
|
553
|
-
|
|
554
|
-
|
|
555
|
-
|
|
556
|
-
`create: true` makes the database file with mode 0600. The WAL file
|
|
557
|
-
(`NAME-wal`) and the MVCC log (`STEM.db-log`, which holds row data until a
|
|
558
|
-
checkpoint) get the database file's permissions, as in SQLite: RomeDB creates
|
|
559
|
-
them before the engine does, and on each open it removes permission bits that the
|
|
560
|
-
database file does not have from existing ones. So every file of a new database is
|
|
561
|
-
0600, and a database made by 0.2.11 or earlier gets private journal files at its
|
|
562
|
-
next open.
|
|
563
|
-
|
|
564
|
-
One process keeps at most `maxOpen` different files open (1 to 1024, default 16).
|
|
565
|
-
An open that would pass this number fails with `FDB_LIMIT`; clients of a file that
|
|
566
|
-
is already open do not count again. `close()` of the last client checkpoints the
|
|
567
|
-
MVCC log into the database file, so the next open has no log to replay, and then
|
|
568
|
-
releases the file's OS lock. To keep many databases, such as one per site, open
|
|
569
|
-
them with the same `maxOpen` and close the least recently used one yourself.
|
|
570
|
-
Reopening a closed file takes about 75 to 90 ms on Linux x64 (release build,
|
|
571
|
-
2026-11-02), nearly independent of its size: each open file has its own worker
|
|
572
|
-
thread, and starting it and loading the addon in it is about 60 ms of that; the
|
|
573
|
-
engine open is about 9 ms and closing with the checkpoint about 10 ms.
|
|
574
|
-
|
|
575
|
-
Clients for the same canonical file in one Node module share one native worker.
|
|
576
|
-
Each batch runs as one worker operation inside a managed transaction. All
|
|
577
|
-
statements are validated before execution. User transaction control is rejected.
|
|
578
|
-
Failures and exceeded result/work limits restore the batch. Existing
|
|
579
|
-
`executeBatch` behavior remains unchanged.
|
|
580
|
-
|
|
581
|
-
Sequence conditions cover writes through this shared adapter. Its capabilities
|
|
582
|
-
report `sequenceScope: 'adapter'` and `replayDurability: 'adapter-lifetime'`.
|
|
583
|
-
`info()` also returns `engineVersion`, the RomeDB release of the native engine.
|
|
584
|
-
Database URLs report it from local services 2.7.0 and later and from Cloud; it is
|
|
585
|
-
absent for older services.
|
|
586
|
-
Other native or legacy handles do not advance this revision. Studio must reuse
|
|
587
|
-
the managed adapter for each canonical file. Use the service URL for sharing
|
|
588
|
-
between application processes.
|
|
589
|
-
|
|
590
|
-
The adapter retains 64 exact write receipts in memory. Saved journals can recover
|
|
591
|
-
a write while that adapter remains open. After its last client closes, reopening
|
|
592
|
-
creates a new identity and refuses old requests. Reconcile uncertain writes from
|
|
593
|
-
the stored data after a reopen. Service and Cloud connections provide recovery
|
|
594
|
-
across process restarts.
|
|
595
|
-
|
|
596
|
-
The initial limits are 16 open file adapters, 32 outstanding requests per adapter,
|
|
597
|
-
and 2 MiB of queued request bodies. The request, result, and work limits match the
|
|
598
|
-
common contract. Calls default to 20 seconds and accept a 1–120,000 ms timeout.
|
|
599
|
-
Opening also has a deadline. If opening expires, cleanup continues until native
|
|
600
|
-
startup returns. Filesystem work and native cleanup are not hard preempted.
|
|
601
|
-
|
|
602
|
-
Close stops requests from that client and cancels its outstanding work. The last
|
|
603
|
-
client drains the worker queue, closes the native handle, and waits for worker
|
|
604
|
-
exit. Await every client's `close()` to release ownership.
|
|
605
|
-
|
|
606
|
-
For typed collection CRUD helpers, see the [document SDK guide](https://github.com/romedb/romedb/blob/main/romedb/docs/node-sdk.md).
|
|
1016
|
+
The report names the tested stripped artifact by size and SHA-256. It records the ELF
|
|
1017
|
+
class and machine, linked libraries, search paths and required symbol versions. This
|
|
1018
|
+
artifact uses GLIBC symbols up to 2.35 and has no RPATH or RUNPATH. A passing local
|
|
1019
|
+
Linux test does not prove compatibility with older glibc systems, musl distributions,
|
|
1020
|
+
another architecture or another Node version. A release baseline needs an explicit
|
|
1021
|
+
build environment and tests on the advertised targets. Regenerate the report whenever
|
|
1022
|
+
the addon changes.
|