@optimystic/db-p2p-storage-ns 1.24.0 → 1.26.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +51 -17
- package/dist/src/connection-mutex.d.ts +1 -1
- package/dist/src/connection-mutex.js +1 -1
- package/dist/src/db.d.ts +16 -19
- package/dist/src/db.d.ts.map +1 -1
- package/dist/src/db.js +13 -15
- package/dist/src/db.js.map +1 -1
- package/dist/src/ns-opener.d.ts +27 -25
- package/dist/src/ns-opener.d.ts.map +1 -1
- package/dist/src/ns-opener.js +48 -19
- package/dist/src/ns-opener.js.map +1 -1
- package/package.json +4 -4
- package/src/connection-mutex.ts +1 -1
- package/src/db.ts +19 -24
- package/src/ns-opener.ts +62 -28
package/README.md
CHANGED
|
@@ -22,7 +22,7 @@ plugin). It cannot run in plain Node, browsers, or React Native — use the
|
|
|
22
22
|
sibling adapter for each of those:
|
|
23
23
|
|
|
24
24
|
- `@optimystic/db-p2p-storage-fs` — Node filesystem
|
|
25
|
-
- `@optimystic/db-p2p-storage-rn` — React Native (
|
|
25
|
+
- `@optimystic/db-p2p-storage-rn` — React Native (LevelDB)
|
|
26
26
|
- `@optimystic/db-p2p-storage-web` — Browser (IndexedDB)
|
|
27
27
|
|
|
28
28
|
## Install
|
|
@@ -32,8 +32,12 @@ yarn add @optimystic/db-p2p-storage-ns @optimystic/db-p2p @optimystic/db-core \
|
|
|
32
32
|
@nativescript-community/sqlite
|
|
33
33
|
```
|
|
34
34
|
|
|
35
|
-
`@nativescript-community/sqlite` is a peer dependency — the host app
|
|
36
|
-
version it needs (same approach as `
|
|
35
|
+
`@nativescript-community/sqlite` is a peer dependency at `^4.1.0` — the host app
|
|
36
|
+
pins the version it needs (same approach as `rn-leveldb` for the RN package).
|
|
37
|
+
It brings `sqlite-core`, both bindings (`sqlite-napi` for NativeScript 9.1+,
|
|
38
|
+
`sqlite-bridge` for any version) and `sqlite-default-engine`, so NativeScript 8
|
|
39
|
+
needs nothing extra. This package does not use the plugin's `*Sync` methods,
|
|
40
|
+
which throw on the bridge.
|
|
37
41
|
|
|
38
42
|
Use the Node-free `/rn` entry point of `@optimystic/db-p2p` so the Node-only
|
|
39
43
|
TCP transport doesn't get bundled into the NativeScript app:
|
|
@@ -65,13 +69,18 @@ const libp2p = await createLibp2pNode({
|
|
|
65
69
|
});
|
|
66
70
|
```
|
|
67
71
|
|
|
68
|
-
The same handle is shared by all three consumers.
|
|
69
|
-
|
|
72
|
+
The same handle is shared by all three consumers. The plugin runs every call on
|
|
73
|
+
one native thread per database, in the order made, which orders single calls
|
|
74
|
+
but does not keep a sequence of calls together, and SQLite allows at most one
|
|
75
|
+
open transaction per connection. So the wrapper serializes every mutating
|
|
70
76
|
operation — `exec`, statement `run`, and whole `transaction` bodies — on a
|
|
71
77
|
per-connection FIFO mutex. Without it, two concurrent `transaction` bodies
|
|
72
|
-
would each `BEGIN` on the shared connection; the second
|
|
73
|
-
|
|
74
|
-
|
|
78
|
+
would each `BEGIN` on the shared connection; the second fails, and its rollback
|
|
79
|
+
discards the first's still-open writes. A plain write between another body's
|
|
80
|
+
`BEGIN` and `COMMIT` would land inside that transaction and share its fate.
|
|
81
|
+
Reads (`get`/`all`) stay off the mutex: a read waits for the statements queued
|
|
82
|
+
ahead of it, not for a whole transaction body, so it can land between a body's
|
|
83
|
+
statements and see its uncommitted rows.
|
|
75
84
|
|
|
76
85
|
Close the storage before the handle, and the handle last:
|
|
77
86
|
|
|
@@ -87,8 +96,10 @@ still has armed would fire on a closed connection.
|
|
|
87
96
|
|
|
88
97
|
## Persistence model
|
|
89
98
|
|
|
90
|
-
The package opens a single SQLite database (`optimystic.sqlite` by default
|
|
91
|
-
in the
|
|
99
|
+
The package opens a single SQLite database (`optimystic.sqlite` by default; a
|
|
100
|
+
bare name goes in the app's databases folder on Android and its documents folder
|
|
101
|
+
on iOS, and a full path is used as given) with eight tables. On Android the
|
|
102
|
+
engine is the SQLite the plugin bundles; on iOS it is the process's. The six
|
|
92
103
|
block-storage tables keep their original columns and keys, but their value
|
|
93
104
|
columns are **BLOB**: `SqliteBatchBackend` binds/reads the kernel's raw
|
|
94
105
|
`Uint8Array` bytes and the shared `KvRawStorage` kernel owns the JSON/UTF-8
|
|
@@ -182,10 +193,33 @@ underlying SQLite file.
|
|
|
182
193
|
|
|
183
194
|
## Tests
|
|
184
195
|
|
|
185
|
-
`yarn test` runs the spec suite
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
|
|
196
|
+
`yarn test` runs the spec suite through the production wrapper, its mutex and
|
|
197
|
+
its transaction protocol, over a stand-in for the plugin built on Node's built-in
|
|
198
|
+
`node:sqlite` (Node 22+ — default-enabled on Node 23+;
|
|
199
|
+
`packages/db-p2p-storage-ns/test/node-sqlite-driver.ts`). As on the plugin's
|
|
200
|
+
native thread, every call of the stand-in resolves later, strictly in the order
|
|
201
|
+
made, so other callers run between a transaction body's statements. The suite
|
|
202
|
+
never imports `@nativescript-community/sqlite`, so the plugin's native bindings
|
|
203
|
+
don't need to be installed to run tests, and it says nothing about how the plugin
|
|
204
|
+
behaves on a device. No test cuts the power: that a committed group survives one
|
|
205
|
+
rests on SQLite's write-ahead log, as stated above.
|
|
206
|
+
|
|
207
|
+
## What has run on a device
|
|
208
|
+
|
|
209
|
+
Nothing in this repository runs NativeScript on a device. The evidence for this
|
|
210
|
+
package on the plugin is one reporter's run (GitHub #38): this package at 1.20.1,
|
|
211
|
+
unmodified, with the plugin forced to 4.1.0, on a Galaxy S7 with Android 8.0 and
|
|
212
|
+
NativeScript 8.9, so the plugin's JNI bridge.
|
|
213
|
+
|
|
214
|
+
| Check | Result |
|
|
215
|
+
|---|---|
|
|
216
|
+
| Opens through `openOptimysticNSDb` | works, `journal_mode` = `wal`, SQLite 3.50.1 (bundled by the plugin) |
|
|
217
|
+
| `ON CONFLICT … DO UPDATE` | works |
|
|
218
|
+
| `SqliteRawStorage` save, `flush()`, read back | works |
|
|
219
|
+
| Write, then `am force-stop` within milliseconds of the commit resolving | 10 of 10 marker rows survived 5 kills: one autocommit row and one `synced: true` row per cycle |
|
|
220
|
+
| 2,003 point reads interleaved with a 2,000-row transaction | no errors; the reads see the transaction's uncommitted rows |
|
|
221
|
+
| App boot on databases written by 3.5.9 | opened and ran |
|
|
222
|
+
|
|
223
|
+
Not run by anyone yet: a long session with real network traffic; iOS; the
|
|
224
|
+
Node-API binding NativeScript 9.1+ selects; a power cut (a force-stop kills the
|
|
225
|
+
process only).
|
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
*
|
|
4
4
|
* SQLite allows at most one open transaction per connection, so every mutating
|
|
5
5
|
* operation on a shared connection (plain writes AND transaction bodies) must be
|
|
6
|
-
* serialized or a concurrent `BEGIN`
|
|
6
|
+
* serialized, or a concurrent `BEGIN` fails and its rollback discards another
|
|
7
7
|
* operation's still-open writes.
|
|
8
8
|
*
|
|
9
9
|
* This is deliberately NOT the global `Latches` keyed map from
|
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
*
|
|
4
4
|
* SQLite allows at most one open transaction per connection, so every mutating
|
|
5
5
|
* operation on a shared connection (plain writes AND transaction bodies) must be
|
|
6
|
-
* serialized or a concurrent `BEGIN`
|
|
6
|
+
* serialized, or a concurrent `BEGIN` fails and its rollback discards another
|
|
7
7
|
* operation's still-open writes.
|
|
8
8
|
*
|
|
9
9
|
* This is deliberately NOT the global `Latches` keyed map from
|
package/dist/src/db.d.ts
CHANGED
|
@@ -3,23 +3,21 @@
|
|
|
3
3
|
*
|
|
4
4
|
* The storage classes (`SqliteRawStorage`, `SqliteKVStore`) and the identity
|
|
5
5
|
* helper (`loadOrCreateNSPeerKey`) only depend on the `SqliteDb` interface
|
|
6
|
-
* defined here — never on `@nativescript-community/sqlite` directly.
|
|
7
|
-
*
|
|
8
|
-
*
|
|
6
|
+
* defined here — never on `@nativescript-community/sqlite` directly.
|
|
7
|
+
* `ns-opener.ts` implements it over the plugin, and the suite runs that same
|
|
8
|
+
* wrapper under Node mocha over a stand-in for the plugin built on `node:sqlite`,
|
|
9
|
+
* matching the pattern `db-p2p-storage-web` uses with `fake-indexeddb`.
|
|
9
10
|
*
|
|
10
|
-
*
|
|
11
|
-
*
|
|
11
|
+
* Consumers normally need only `openOptimysticNSDb`; `wrapNSPluginDb` is for a host
|
|
12
|
+
* that already holds an open plugin handle.
|
|
12
13
|
*/
|
|
13
14
|
/** SQL parameter value — covers everything `IRawStorage` and `IKVStore` write. */
|
|
14
15
|
export type SqliteParam = string | number | Uint8Array | null;
|
|
15
16
|
/** A single row returned from a SQL query, keyed by column name. */
|
|
16
17
|
export type SqliteRow = Record<string, SqliteParam>;
|
|
17
18
|
/**
|
|
18
|
-
* A
|
|
19
|
-
*
|
|
20
|
-
* Both `node:sqlite` and `@nativescript-community/sqlite` cache parsed SQL
|
|
21
|
-
* internally, so re-binding a prepared statement is cheaper than re-parsing a
|
|
22
|
-
* raw `execute`. The storage classes prepare each query once per instance.
|
|
19
|
+
* A statement bound to a single SQL string. The storage classes prepare each
|
|
20
|
+
* query once per instance.
|
|
23
21
|
*/
|
|
24
22
|
export interface SqliteStatement {
|
|
25
23
|
/** Execute with the given bind parameters, ignoring any returned rows. */
|
|
@@ -58,9 +56,9 @@ export interface SqliteTransactionOptions {
|
|
|
58
56
|
/**
|
|
59
57
|
* Minimal SQLite driver surface used by this package.
|
|
60
58
|
*
|
|
61
|
-
*
|
|
62
|
-
*
|
|
63
|
-
*
|
|
59
|
+
* Implemented once, over the NativeScript plugin (`wrapNSPluginDb`); in tests the
|
|
60
|
+
* plugin is a stand-in over `node:sqlite`. Async on every method because every
|
|
61
|
+
* plugin call runs on the database's native thread and resolves later.
|
|
64
62
|
*
|
|
65
63
|
* A single connection is shared across the storage classes, so every mutating
|
|
66
64
|
* operation (`exec`, statement `run`, and `transaction` bodies) is serialized on
|
|
@@ -93,14 +91,13 @@ export interface SqliteDb {
|
|
|
93
91
|
}
|
|
94
92
|
/** The raw connection surface {@link runTransaction} drives: one statement at a time, no rows back. */
|
|
95
93
|
export interface TransactionConnection {
|
|
96
|
-
execute(sql: string): Promise<void
|
|
94
|
+
execute(sql: string): Promise<void>;
|
|
97
95
|
}
|
|
98
96
|
/**
|
|
99
|
-
* The transaction protocol
|
|
100
|
-
* `
|
|
101
|
-
*
|
|
102
|
-
*
|
|
103
|
-
* released. `BEGIN IMMEDIATE` takes the write lock up front rather than deferring it to the first
|
|
97
|
+
* The transaction protocol `SqliteDb.transaction` runs inside its mutex slot: with `synced`,
|
|
98
|
+
* `PRAGMA synchronous = FULL` first; then `BEGIN IMMEDIATE`, the body, and `COMMIT`, or `ROLLBACK`
|
|
99
|
+
* when the body or the commit throws; then, synced, `PRAGMA synchronous = NORMAL` whichever way the
|
|
100
|
+
* transaction ended, so the connection's default is restored before the slot is released. `BEGIN IMMEDIATE` takes the write lock up front rather than deferring it to the first
|
|
104
101
|
* write, so contention surfaces here and not mid-body.
|
|
105
102
|
*
|
|
106
103
|
* A failure of the restore after a failed body is logged rather than thrown, so the body's error,
|
package/dist/src/db.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"db.d.ts","sourceRoot":"","sources":["../../src/db.ts"],"names":[],"mappings":"AAAA
|
|
1
|
+
{"version":3,"file":"db.d.ts","sourceRoot":"","sources":["../../src/db.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;GAYG;AAMH,kFAAkF;AAClF,MAAM,MAAM,WAAW,GAAG,MAAM,GAAG,MAAM,GAAG,UAAU,GAAG,IAAI,CAAC;AAE9D,oEAAoE;AACpE,MAAM,MAAM,SAAS,GAAG,MAAM,CAAC,MAAM,EAAE,WAAW,CAAC,CAAC;AAEpD;;;GAGG;AACH,MAAM,WAAW,eAAe;IAC/B,0EAA0E;IAC1E,GAAG,CAAC,GAAG,MAAM,EAAE,WAAW,EAAE,GAAG,OAAO,CAAC,IAAI,CAAC,CAAC;IAC7C,gEAAgE;IAChE,GAAG,CAAC,GAAG,MAAM,EAAE,WAAW,EAAE,GAAG,OAAO,CAAC,SAAS,GAAG,SAAS,CAAC,CAAC;IAC9D,oFAAoF;IACpF,GAAG,CAAC,GAAG,MAAM,EAAE,WAAW,EAAE,GAAG,OAAO,CAAC,SAAS,EAAE,CAAC,CAAC;CACpD;AAED;;;;;;;;;;GAUG;AACH,MAAM,WAAW,iBAAiB;IACjC,6EAA6E;IAC7E,OAAO,CAAC,GAAG,EAAE,MAAM,GAAG,eAAe,CAAC;CACtC;AAED,+CAA+C;AAC/C,MAAM,WAAW,wBAAwB;IACxC;;;;;;OAMG;IACH,QAAQ,CAAC,MAAM,CAAC,EAAE,OAAO,CAAC;CAC1B;AAED;;;;;;;;;;;GAWG;AACH,MAAM,WAAW,QAAQ;IACxB,yFAAyF;IACzF,IAAI,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC,IAAI,CAAC,CAAC;IACjC,2FAA2F;IAC3F,OAAO,CAAC,GAAG,EAAE,MAAM,GAAG,eAAe,CAAC;IACtC;;;;;;;;;;;;;;OAcG;IACH,WAAW,CAAC,CAAC,EAAE,EAAE,EAAE,CAAC,EAAE,EAAE,iBAAiB,KAAK,OAAO,CAAC,CAAC,CAAC,EAAE,OAAO,CAAC,EAAE,wBAAwB,GAAG,OAAO,CAAC,CAAC,CAAC,CAAC;IAC1G,qCAAqC;IACrC,KAAK,IAAI,OAAO,CAAC,IAAI,CAAC,CAAC;CACvB;AAED,uGAAuG;AACvG,MAAM,WAAW,qBAAqB;IACrC,OAAO,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC,IAAI,CAAC,CAAC;CACpC;AAED;;;;;;;;;;GAUG;AACH,wBAAsB,cAAc,CAAC,CAAC,EACrC,IAAI,EAAE,qBAAqB,EAC3B,EAAE,EAAE,iBAAiB,EACrB,EAAE,EAAE,CAAC,EAAE,EAAE,iBAAiB,KAAK,OAAO,CAAC,CAAC,CAAC,EACzC,OAAO,CAAC,EAAE,wBAAwB,GAChC,OAAO,CAAC,CAAC,CAAC,CAkBZ;AAmBD,eAAO,MAAM,eAAe,sBAAsB,CAAC;AACnD,eAAO,MAAM,kBAAkB,IAAI,CAAC;AAEpC;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GAqCG;AACH,eAAO,MAAM,UAAU,woCAmDtB,CAAC;AAQF;;;GAGG;AACH,wBAAsB,WAAW,CAAC,EAAE,EAAE,QAAQ,EAAE,OAAO,GAAE,MAA2B,GAAG,OAAO,CAAC,IAAI,CAAC,CAQnG;AAED;;;GAGG;AACH,MAAM,MAAM,oBAAoB,GAAG,QAAQ,CAAC"}
|
package/dist/src/db.js
CHANGED
|
@@ -3,21 +3,21 @@
|
|
|
3
3
|
*
|
|
4
4
|
* The storage classes (`SqliteRawStorage`, `SqliteKVStore`) and the identity
|
|
5
5
|
* helper (`loadOrCreateNSPeerKey`) only depend on the `SqliteDb` interface
|
|
6
|
-
* defined here — never on `@nativescript-community/sqlite` directly.
|
|
7
|
-
*
|
|
8
|
-
*
|
|
6
|
+
* defined here — never on `@nativescript-community/sqlite` directly.
|
|
7
|
+
* `ns-opener.ts` implements it over the plugin, and the suite runs that same
|
|
8
|
+
* wrapper under Node mocha over a stand-in for the plugin built on `node:sqlite`,
|
|
9
|
+
* matching the pattern `db-p2p-storage-web` uses with `fake-indexeddb`.
|
|
9
10
|
*
|
|
10
|
-
*
|
|
11
|
-
*
|
|
11
|
+
* Consumers normally need only `openOptimysticNSDb`; `wrapNSPluginDb` is for a host
|
|
12
|
+
* that already holds an open plugin handle.
|
|
12
13
|
*/
|
|
13
14
|
import { createLogger } from './logger.js';
|
|
14
15
|
const log = createLogger('db');
|
|
15
16
|
/**
|
|
16
|
-
* The transaction protocol
|
|
17
|
-
* `
|
|
18
|
-
*
|
|
19
|
-
*
|
|
20
|
-
* released. `BEGIN IMMEDIATE` takes the write lock up front rather than deferring it to the first
|
|
17
|
+
* The transaction protocol `SqliteDb.transaction` runs inside its mutex slot: with `synced`,
|
|
18
|
+
* `PRAGMA synchronous = FULL` first; then `BEGIN IMMEDIATE`, the body, and `COMMIT`, or `ROLLBACK`
|
|
19
|
+
* when the body or the commit throws; then, synced, `PRAGMA synchronous = NORMAL` whichever way the
|
|
20
|
+
* transaction ended, so the connection's default is restored before the slot is released. `BEGIN IMMEDIATE` takes the write lock up front rather than deferring it to the first
|
|
21
21
|
* write, so contention surfaces here and not mid-body.
|
|
22
22
|
*
|
|
23
23
|
* A failure of the restore after a failed body is logged rather than thrown, so the body's error,
|
|
@@ -167,11 +167,9 @@ PRAGMA foreign_keys = OFF;
|
|
|
167
167
|
* migrations can branch on it. Idempotent — safe to call on every open.
|
|
168
168
|
*/
|
|
169
169
|
export async function applySchema(db, version = DEFAULT_DB_VERSION) {
|
|
170
|
-
// `PRAGMA journal_mode = WAL` returns the resulting mode as a row
|
|
171
|
-
//
|
|
172
|
-
//
|
|
173
|
-
// this pragma must run through a query method. prepare().get() does that on
|
|
174
|
-
// every driver (NS plugin → get()/rawQuery; node:sqlite/better-sqlite3 → get()).
|
|
170
|
+
// `PRAGMA journal_mode = WAL` returns the resulting mode as a row, so it goes through a
|
|
171
|
+
// query method. It stays although the plugin turns WAL on by default on Android: iOS has
|
|
172
|
+
// no such default and relies on this pragma.
|
|
175
173
|
await db.prepare('PRAGMA journal_mode = WAL').get();
|
|
176
174
|
await db.exec(PRAGMAS_SQL);
|
|
177
175
|
await db.exec(SCHEMA_SQL);
|
package/dist/src/db.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"db.js","sourceRoot":"","sources":["../../src/db.ts"],"names":[],"mappings":"AAAA
|
|
1
|
+
{"version":3,"file":"db.js","sourceRoot":"","sources":["../../src/db.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;GAYG;AAEH,OAAO,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAE3C,MAAM,GAAG,GAAG,YAAY,CAAC,IAAI,CAAC,CAAC;AA2F/B;;;;;;;;;;GAUG;AACH,MAAM,CAAC,KAAK,UAAU,cAAc,CACnC,IAA2B,EAC3B,EAAqB,EACrB,EAAyC,EACzC,OAAkC;IAElC,MAAM,MAAM,GAAG,OAAO,EAAE,MAAM,KAAK,IAAI,CAAC;IACxC,IAAI,MAAM;QAAE,MAAM,IAAI,CAAC,OAAO,CAAC,2BAA2B,CAAC,CAAC;IAC5D,IAAI,MAAS,CAAC;IACd,IAAI,CAAC;QACJ,MAAM,GAAG,MAAM,gBAAgB,CAAC,IAAI,EAAE,EAAE,EAAE,EAAE,CAAC,CAAC;IAC/C,CAAC;IAAC,OAAO,GAAG,EAAE,CAAC;QACd,IAAI,MAAM,EAAE,CAAC;YACZ,IAAI,CAAC;gBACJ,MAAM,IAAI,CAAC,OAAO,CAAC,6BAA6B,CAAC,CAAC;YACnD,CAAC;YAAC,OAAO,UAAU,EAAE,CAAC;gBACrB,GAAG,CAAC,sEAAsE,EAAE,UAAU,CAAC,CAAC;YACzF,CAAC;QACF,CAAC;QACD,MAAM,GAAG,CAAC;IACX,CAAC;IACD,IAAI,MAAM;QAAE,MAAM,IAAI,CAAC,OAAO,CAAC,6BAA6B,CAAC,CAAC;IAC9D,OAAO,MAAM,CAAC;AACf,CAAC;AAED,KAAK,UAAU,gBAAgB,CAAI,IAA2B,EAAE,EAAqB,EAAE,EAAyC;IAC/H,MAAM,IAAI,CAAC,OAAO,CAAC,iBAAiB,CAAC,CAAC;IACtC,IAAI,CAAC;QACJ,MAAM,MAAM,GAAG,MAAM,EAAE,CAAC,EAAE,CAAC,CAAC;QAC5B,MAAM,IAAI,CAAC,OAAO,CAAC,QAAQ,CAAC,CAAC;QAC7B,OAAO,MAAM,CAAC;IACf,CAAC;IAAC,OAAO,GAAG,EAAE,CAAC;QACd,IAAI,CAAC;YACJ,MAAM,IAAI,CAAC,OAAO,CAAC,UAAU,CAAC,CAAC;QAChC,CAAC;QAAC,OAAO,WAAW,EAAE,CAAC;YACtB,2EAA2E;YAC3E,GAAG,CAAC,gDAAgD,EAAE,WAAW,CAAC,CAAC;QACpE,CAAC;QACD,MAAM,GAAG,CAAC;IACX,CAAC;AACF,CAAC;AAED,MAAM,CAAC,MAAM,eAAe,GAAG,mBAAmB,CAAC;AACnD,MAAM,CAAC,MAAM,kBAAkB,GAAG,CAAC,CAAC;AAEpC;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GAqCG;AACH,MAAM,CAAC,MAAM,UAAU,GAAG;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;CAmDzB,CAAC;AAEF,+EAA+E;AAC/E,MAAM,WAAW,GAAG;;;CAGnB,CAAC;AAEF;;;GAGG;AACH,MAAM,CAAC,KAAK,UAAU,WAAW,CAAC,EAAY,EAAE,UAAkB,kBAAkB;IACnF,wFAAwF;IACxF,yFAAyF;IACzF,6CAA6C;IAC7C,MAAM,EAAE,CAAC,OAAO,CAAC,2BAA2B,CAAC,CAAC,GAAG,EAAE,CAAC;IACpD,MAAM,EAAE,CAAC,IAAI,CAAC,WAAW,CAAC,CAAC;IAC3B,MAAM,EAAE,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC;IAC1B,MAAM,EAAE,CAAC,IAAI,CAAC,yBAAyB,OAAO,EAAE,CAAC,CAAC;AACnD,CAAC"}
|
package/dist/src/ns-opener.d.ts
CHANGED
|
@@ -1,40 +1,43 @@
|
|
|
1
|
-
import { type OptimysticNSDBHandle, type SqliteDb } from './db.js';
|
|
1
|
+
import { type OptimysticNSDBHandle, type SqliteDb, type SqliteParam } from './db.js';
|
|
2
2
|
/**
|
|
3
|
-
* Minimal subset of `@nativescript-community/sqlite`
|
|
3
|
+
* Minimal subset of a `@nativescript-community/sqlite` 4.x database we depend on.
|
|
4
4
|
* Re-declared locally so this module's *types* don't pull in the plugin's
|
|
5
5
|
* declarations (the plugin is a peer dependency that may not be installed
|
|
6
|
-
* at typecheck time on non-NativeScript consumers).
|
|
6
|
+
* at typecheck time on non-NativeScript consumers). Exported for the test
|
|
7
|
+
* driver, not from `index.ts`.
|
|
7
8
|
*/
|
|
8
|
-
interface NSPluginDb {
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
9
|
+
export interface NSPluginDb {
|
|
10
|
+
/** Settles when the open does; a failed open rejects it. */
|
|
11
|
+
readonly ready: Promise<void>;
|
|
12
|
+
execute(sql: string, params?: SqliteParam[]): Promise<unknown>;
|
|
13
|
+
get(sql: string, params?: SqliteParam[]): Promise<Record<string, unknown> | null | undefined>;
|
|
14
|
+
select(sql: string, params?: SqliteParam[]): Promise<Array<Record<string, unknown>>>;
|
|
15
|
+
close(): Promise<void>;
|
|
16
|
+
}
|
|
17
|
+
export interface NSPluginModule {
|
|
18
|
+
/** Returns the handle at once; the open itself runs on the database's native thread. */
|
|
19
|
+
openOrCreate(path: string): NSPluginDb;
|
|
13
20
|
}
|
|
14
21
|
/**
|
|
15
|
-
* Opens (creating if needed) the Optimystic SQLite database at `name
|
|
16
|
-
* the
|
|
17
|
-
*
|
|
22
|
+
* Opens (creating if needed) the Optimystic SQLite database at `name`, applies
|
|
23
|
+
* WAL pragmas, and runs the migration to `version`. A bare `name` goes in the
|
|
24
|
+
* app's databases folder on Android and its documents folder on iOS, and needs
|
|
25
|
+
* no foreground activity; a full path is used as given.
|
|
18
26
|
*
|
|
19
27
|
* The returned `SqliteDb` handle is safe to share across `SqliteRawStorage`,
|
|
20
|
-
* `SqliteKVStore`, and `loadOrCreateNSPeerKey
|
|
21
|
-
*
|
|
22
|
-
*
|
|
23
|
-
* a per-connection FIFO mutex. Without it, two concurrent `transaction` bodies
|
|
24
|
-
* would each `BEGIN` on the shared connection; the second would nest and its
|
|
25
|
-
* rollback would silently discard the first's still-open writes. Reads
|
|
26
|
-
* (`get`/`all`) stay off the mutex to preserve read concurrency.
|
|
27
|
-
*
|
|
28
|
-
* `path` may be passed in as the full filesystem path if the caller wants
|
|
29
|
-
* to control file placement; otherwise the plugin's documents-directory
|
|
30
|
-
* default is used.
|
|
28
|
+
* `SqliteKVStore`, and `loadOrCreateNSPeerKey`: the wrapper serializes every
|
|
29
|
+
* mutating operation through a per-connection FIFO mutex (`NSPluginDbWrapper`
|
|
30
|
+
* says why).
|
|
31
31
|
*/
|
|
32
32
|
export declare function openOptimysticNSDb(name?: string, version?: number): Promise<OptimysticNSDBHandle>;
|
|
33
|
+
/** `openOptimysticNSDb` once the plugin is loaded. */
|
|
34
|
+
export declare function openOverPlugin(plugin: NSPluginModule, name: string, version: number): Promise<OptimysticNSDBHandle>;
|
|
33
35
|
/**
|
|
34
|
-
* Wraps a `@nativescript-community/sqlite`
|
|
36
|
+
* Wraps a `@nativescript-community/sqlite` database to satisfy the
|
|
35
37
|
* internal `SqliteDb` interface used by the storage classes. Exported only
|
|
36
38
|
* for callers that already hold an open NS-plugin handle and want to skip
|
|
37
|
-
* the opener (rare — typically users just call `openOptimysticNSDb`).
|
|
39
|
+
* the opener (rare — typically users just call `openOptimysticNSDb`). The
|
|
40
|
+
* plugin returns a handle before its open has finished, so await `ready` first.
|
|
38
41
|
*
|
|
39
42
|
* NOTE: the serialization mutex lives on the wrapper, not the raw handle. Wrap
|
|
40
43
|
* a given raw connection exactly once — two wrappers over one handle each hold
|
|
@@ -42,5 +45,4 @@ export declare function openOptimysticNSDb(name?: string, version?: number): Pro
|
|
|
42
45
|
* cross-rollback this fix closes.
|
|
43
46
|
*/
|
|
44
47
|
export declare function wrapNSPluginDb(raw: NSPluginDb): SqliteDb;
|
|
45
|
-
export {};
|
|
46
48
|
//# sourceMappingURL=ns-opener.d.ts.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"ns-opener.d.ts","sourceRoot":"","sources":["../../src/ns-opener.ts"],"names":[],"mappings":"AACA,OAAO,EAAoE,KAAK,oBAAoB,EAAE,KAAK,QAAQ,
|
|
1
|
+
{"version":3,"file":"ns-opener.d.ts","sourceRoot":"","sources":["../../src/ns-opener.ts"],"names":[],"mappings":"AACA,OAAO,EAAoE,KAAK,oBAAoB,EAAE,KAAK,QAAQ,EAAE,KAAK,WAAW,EAA+F,MAAM,SAAS,CAAC;AAEpP;;;;;;GAMG;AACH,MAAM,WAAW,UAAU;IAC1B,4DAA4D;IAC5D,QAAQ,CAAC,KAAK,EAAE,OAAO,CAAC,IAAI,CAAC,CAAC;IAC9B,OAAO,CAAC,GAAG,EAAE,MAAM,EAAE,MAAM,CAAC,EAAE,WAAW,EAAE,GAAG,OAAO,CAAC,OAAO,CAAC,CAAC;IAC/D,GAAG,CAAC,GAAG,EAAE,MAAM,EAAE,MAAM,CAAC,EAAE,WAAW,EAAE,GAAG,OAAO,CAAC,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,GAAG,IAAI,GAAG,SAAS,CAAC,CAAC;IAC9F,MAAM,CAAC,GAAG,EAAE,MAAM,EAAE,MAAM,CAAC,EAAE,WAAW,EAAE,GAAG,OAAO,CAAC,KAAK,CAAC,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC,CAAC,CAAC;IACrF,KAAK,IAAI,OAAO,CAAC,IAAI,CAAC,CAAC;CACvB;AAED,MAAM,WAAW,cAAc;IAC9B,wFAAwF;IACxF,YAAY,CAAC,IAAI,EAAE,MAAM,GAAG,UAAU,CAAC;CACvC;AAED;;;;;;;;;;GAUG;AACH,wBAAsB,kBAAkB,CACvC,IAAI,GAAE,MAAwB,EAC9B,OAAO,GAAE,MAA2B,GAClC,OAAO,CAAC,oBAAoB,CAAC,CAG/B;AAED,sDAAsD;AACtD,wBAAsB,cAAc,CAAC,MAAM,EAAE,cAAc,EAAE,IAAI,EAAE,MAAM,EAAE,OAAO,EAAE,MAAM,GAAG,OAAO,CAAC,oBAAoB,CAAC,CAOzH;AAED;;;;;;;;;;;GAWG;AACH,wBAAgB,cAAc,CAAC,GAAG,EAAE,UAAU,GAAG,QAAQ,CAExD"}
|
package/dist/src/ns-opener.js
CHANGED
|
@@ -1,35 +1,35 @@
|
|
|
1
1
|
import { ConnectionMutex } from './connection-mutex.js';
|
|
2
2
|
import { applySchema, runTransaction, DEFAULT_DB_NAME, DEFAULT_DB_VERSION } from './db.js';
|
|
3
3
|
/**
|
|
4
|
-
* Opens (creating if needed) the Optimystic SQLite database at `name
|
|
5
|
-
* the
|
|
6
|
-
*
|
|
4
|
+
* Opens (creating if needed) the Optimystic SQLite database at `name`, applies
|
|
5
|
+
* WAL pragmas, and runs the migration to `version`. A bare `name` goes in the
|
|
6
|
+
* app's databases folder on Android and its documents folder on iOS, and needs
|
|
7
|
+
* no foreground activity; a full path is used as given.
|
|
7
8
|
*
|
|
8
9
|
* The returned `SqliteDb` handle is safe to share across `SqliteRawStorage`,
|
|
9
|
-
* `SqliteKVStore`, and `loadOrCreateNSPeerKey
|
|
10
|
-
*
|
|
11
|
-
*
|
|
12
|
-
* a per-connection FIFO mutex. Without it, two concurrent `transaction` bodies
|
|
13
|
-
* would each `BEGIN` on the shared connection; the second would nest and its
|
|
14
|
-
* rollback would silently discard the first's still-open writes. Reads
|
|
15
|
-
* (`get`/`all`) stay off the mutex to preserve read concurrency.
|
|
16
|
-
*
|
|
17
|
-
* `path` may be passed in as the full filesystem path if the caller wants
|
|
18
|
-
* to control file placement; otherwise the plugin's documents-directory
|
|
19
|
-
* default is used.
|
|
10
|
+
* `SqliteKVStore`, and `loadOrCreateNSPeerKey`: the wrapper serializes every
|
|
11
|
+
* mutating operation through a per-connection FIFO mutex (`NSPluginDbWrapper`
|
|
12
|
+
* says why).
|
|
20
13
|
*/
|
|
21
14
|
export async function openOptimysticNSDb(name = DEFAULT_DB_NAME, version = DEFAULT_DB_VERSION) {
|
|
22
15
|
const plugin = (await import('@nativescript-community/sqlite'));
|
|
23
|
-
|
|
16
|
+
return openOverPlugin(plugin, name, version);
|
|
17
|
+
}
|
|
18
|
+
/** `openOptimysticNSDb` once the plugin is loaded. */
|
|
19
|
+
export async function openOverPlugin(plugin, name, version) {
|
|
20
|
+
const raw = plugin.openOrCreate(name);
|
|
21
|
+
await raw.ready;
|
|
24
22
|
const wrapped = wrapNSPluginDb(raw);
|
|
23
|
+
// NOTE: a failed schema leaves this connection open; if a host ever retries the open in a loop, close it here.
|
|
25
24
|
await applySchema(wrapped, version);
|
|
26
25
|
return wrapped;
|
|
27
26
|
}
|
|
28
27
|
/**
|
|
29
|
-
* Wraps a `@nativescript-community/sqlite`
|
|
28
|
+
* Wraps a `@nativescript-community/sqlite` database to satisfy the
|
|
30
29
|
* internal `SqliteDb` interface used by the storage classes. Exported only
|
|
31
30
|
* for callers that already hold an open NS-plugin handle and want to skip
|
|
32
|
-
* the opener (rare — typically users just call `openOptimysticNSDb`).
|
|
31
|
+
* the opener (rare — typically users just call `openOptimysticNSDb`). The
|
|
32
|
+
* plugin returns a handle before its open has finished, so await `ready` first.
|
|
33
33
|
*
|
|
34
34
|
* NOTE: the serialization mutex lives on the wrapper, not the raw handle. Wrap
|
|
35
35
|
* a given raw connection exactly once — two wrappers over one handle each hold
|
|
@@ -39,6 +39,27 @@ export async function openOptimysticNSDb(name = DEFAULT_DB_NAME, version = DEFAU
|
|
|
39
39
|
export function wrapNSPluginDb(raw) {
|
|
40
40
|
return new NSPluginDbWrapper(raw);
|
|
41
41
|
}
|
|
42
|
+
/**
|
|
43
|
+
* `SqliteDb` over a plugin handle.
|
|
44
|
+
*
|
|
45
|
+
* The plugin runs every call on the database's one native thread, in the order
|
|
46
|
+
* made. That orders single calls but does not keep a sequence of calls together,
|
|
47
|
+
* and every awaited statement yields the JavaScript thread, so other callers run
|
|
48
|
+
* between a transaction body's statements. So the connection mutex stays: two
|
|
49
|
+
* bodies would otherwise each send `BEGIN`, the second would fail, and its
|
|
50
|
+
* `ROLLBACK` would discard the first's open writes; and a plain write between
|
|
51
|
+
* another body's `BEGIN` and `COMMIT` would land inside that transaction and
|
|
52
|
+
* share its fate.
|
|
53
|
+
*
|
|
54
|
+
* The plugin's own `db.transaction` is not used, and would not remove the mutex:
|
|
55
|
+
* it joins a second concurrent `transaction` call into the first, so one body's
|
|
56
|
+
* failure would roll the other back. It would also have to take the place of
|
|
57
|
+
* `runTransaction`, whose `PRAGMA synchronous = FULL` must run outside the
|
|
58
|
+
* transaction (SQLite refuses to change it inside one) and be restored after a
|
|
59
|
+
* rollback. Because the wrapper sends `BEGIN IMMEDIATE` and `COMMIT` as ordinary
|
|
60
|
+
* `execute` calls, the plugin does not know a transaction is open, and runs every
|
|
61
|
+
* request in arrival order.
|
|
62
|
+
*/
|
|
42
63
|
class NSPluginDbWrapper {
|
|
43
64
|
raw;
|
|
44
65
|
mutex = new ConnectionMutex();
|
|
@@ -97,8 +118,10 @@ class NSPluginStatement {
|
|
|
97
118
|
}
|
|
98
119
|
async get(...params) {
|
|
99
120
|
// NOTE: reads run directly on the connection, unserialized, to preserve read
|
|
100
|
-
// concurrency
|
|
101
|
-
//
|
|
121
|
+
// concurrency: a read does not wait for a whole transaction body, only for the
|
|
122
|
+
// statements queued ahead of it on the plugin's native thread. So a read issued
|
|
123
|
+
// while a write transaction is open on this same connection lands between that
|
|
124
|
+
// body's statements and observes its UNCOMMITTED rows (read-your-connection
|
|
102
125
|
// semantics). Fine today: the one transaction writer of the block tables is
|
|
103
126
|
// `SqliteBatchBackend.commit`, and every row it is writing is held by the
|
|
104
127
|
// `WriteGroupDriver` overlay above it until the batch is durable. A point read of
|
|
@@ -116,6 +139,12 @@ class NSPluginStatement {
|
|
|
116
139
|
}
|
|
117
140
|
async all(...params) {
|
|
118
141
|
// NOTE: unserialized read — see the note on `get` above re: uncommitted reads.
|
|
142
|
+
// NOTE: on NativeScript 8's JNI bridge a result over 512 KiB arrives through a fresh
|
|
143
|
+
// direct ByteBuffer, which holds a JNI global reference until it is collected, and
|
|
144
|
+
// Android allows 51,200 of those. This storage reaches that size with one block near
|
|
145
|
+
// or over 512 KiB per read (`get`, like this method) or `listBlockIds` over a large
|
|
146
|
+
// table. Fine now; if a node on NativeScript 8 ever reads thousands of such results
|
|
147
|
+
// between garbage collections, read them in smaller pieces.
|
|
119
148
|
const rows = await this.raw.select(this.sql, params);
|
|
120
149
|
return rows;
|
|
121
150
|
}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"ns-opener.js","sourceRoot":"","sources":["../../src/ns-opener.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,eAAe,EAAE,MAAM,uBAAuB,CAAC;AACxD,OAAO,EAAE,WAAW,EAAE,cAAc,EAAE,eAAe,EAAE,kBAAkB,EAA2J,MAAM,SAAS,CAAC;
|
|
1
|
+
{"version":3,"file":"ns-opener.js","sourceRoot":"","sources":["../../src/ns-opener.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,eAAe,EAAE,MAAM,uBAAuB,CAAC;AACxD,OAAO,EAAE,WAAW,EAAE,cAAc,EAAE,eAAe,EAAE,kBAAkB,EAA2J,MAAM,SAAS,CAAC;AAuBpP;;;;;;;;;;GAUG;AACH,MAAM,CAAC,KAAK,UAAU,kBAAkB,CACvC,OAAe,eAAe,EAC9B,UAAkB,kBAAkB;IAEpC,MAAM,MAAM,GAAG,CAAC,MAAM,MAAM,CAAC,gCAA0C,CAAC,CAA8B,CAAC;IACvG,OAAO,cAAc,CAAC,MAAM,EAAE,IAAI,EAAE,OAAO,CAAC,CAAC;AAC9C,CAAC;AAED,sDAAsD;AACtD,MAAM,CAAC,KAAK,UAAU,cAAc,CAAC,MAAsB,EAAE,IAAY,EAAE,OAAe;IACzF,MAAM,GAAG,GAAG,MAAM,CAAC,YAAY,CAAC,IAAI,CAAC,CAAC;IACtC,MAAM,GAAG,CAAC,KAAK,CAAC;IAChB,MAAM,OAAO,GAAG,cAAc,CAAC,GAAG,CAAC,CAAC;IACpC,+GAA+G;IAC/G,MAAM,WAAW,CAAC,OAAO,EAAE,OAAO,CAAC,CAAC;IACpC,OAAO,OAAO,CAAC;AAChB,CAAC;AAED;;;;;;;;;;;GAWG;AACH,MAAM,UAAU,cAAc,CAAC,GAAe;IAC7C,OAAO,IAAI,iBAAiB,CAAC,GAAG,CAAC,CAAC;AACnC,CAAC;AAED;;;;;;;;;;;;;;;;;;;;GAoBG;AACH,MAAM,iBAAiB;IAGO;IAFZ,KAAK,GAAG,IAAI,eAAe,EAAE,CAAC;IAE/C,YAA6B,GAAe;QAAf,QAAG,GAAH,GAAG,CAAY;IAAG,CAAC;IAEhD,KAAK,CAAC,IAAI,CAAC,GAAW;QACrB,oEAAoE;QACpE,+DAA+D;QAC/D,MAAM,UAAU,GAAG,GAAG;aACpB,KAAK,CAAC,GAAG,CAAC;aACV,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;aAClB,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC;QAC5B,MAAM,IAAI,CAAC,KAAK,CAAC,SAAS,CAAC,KAAK,IAAI,EAAE;YACrC,KAAK,MAAM,SAAS,IAAI,UAAU,EAAE,CAAC;gBACpC,MAAM,IAAI,CAAC,GAAG,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC;YACnC,CAAC;QACF,CAAC,CAAC,CAAC;IACJ,CAAC;IAED,OAAO,CAAC,GAAW;QAClB,8DAA8D;QAC9D,OAAO,IAAI,iBAAiB,CAAC,IAAI,CAAC,GAAG,EAAE,GAAG,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC;IACzD,CAAC;IAED,KAAK,CAAC,WAAW,CAAI,EAAyC,EAAE,OAAkC;QACjG,OAAO,IAAI,CAAC,KAAK,CAAC,SAAS,CAAC,GAAG,EAAE,CAAC,cAAc,CAC/C,EAAE,OAAO,EAAE,KAAK,EAAE,GAAG,EAAE,EAAE,GAAG,MAAM,IAAI,CAAC,GAAG,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,EAAE;QAC5D,iEAAiE;QACjE,oDAAoD;QACpD,EAAE,OAAO,EAAE,CAAC,GAAW,EAAE,EAAE,CAAC,IAAI,iBAAiB,CAAC,IAAI,CAAC,GAAG,EAAE,GAAG,CAAC,EAAE,EAClE,EAAE,EACF,OAAO,CACP,CAAC,CAAC;IACJ,CAAC;IAED,KAAK,CAAC,KAAK;QACV,MAAM,IAAI,CAAC,GAAG,CAAC,KAAK,EAAE,CAAC;IACxB,CAAC;CACD;AAED,MAAM,iBAAiB;IAQJ;IACA;IACA;IATlB;;;;;OAKG;IACH,YACkB,GAAe,EACf,GAAW,EACX,KAAuB;QAFvB,QAAG,GAAH,GAAG,CAAY;QACf,QAAG,GAAH,GAAG,CAAQ;QACX,UAAK,GAAL,KAAK,CAAkB;IACtC,CAAC;IAEJ,KAAK,CAAC,GAAG,CAAC,GAAG,MAAqB;QACjC,IAAI,IAAI,CAAC,KAAK,EAAE,CAAC;YAChB,MAAM,IAAI,CAAC,KAAK,CAAC,SAAS,CAAC,GAAG,EAAE,CAAC,IAAI,CAAC,GAAG,CAAC,OAAO,CAAC,IAAI,CAAC,GAAG,EAAE,MAAM,CAAC,CAAC,CAAC;QACtE,CAAC;aAAM,CAAC;YACP,MAAM,IAAI,CAAC,GAAG,CAAC,OAAO,CAAC,IAAI,CAAC,GAAG,EAAE,MAAM,CAAC,CAAC;QAC1C,CAAC;IACF,CAAC;IAED,KAAK,CAAC,GAAG,CAAC,GAAG,MAAqB;QACjC,6EAA6E;QAC7E,+EAA+E;QAC/E,gFAAgF;QAChF,+EAA+E;QAC/E,4EAA4E;QAC5E,4EAA4E;QAC5E,0EAA0E;QAC1E,kFAAkF;QAClF,6EAA6E;QAC7E,kFAAkF;QAClF,kFAAkF;QAClF,kFAAkF;QAClF,+EAA+E;QAC/E,8EAA8E;QAC9E,6CAA6C;QAC7C,MAAM,GAAG,GAAG,MAAM,IAAI,CAAC,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,GAAG,EAAE,MAAM,CAAC,CAAC;QACjD,IAAI,GAAG,KAAK,IAAI,IAAI,GAAG,KAAK,SAAS;YAAE,OAAO,SAAS,CAAC;QACxD,OAAO,GAAgB,CAAC;IACzB,CAAC;IAED,KAAK,CAAC,GAAG,CAAC,GAAG,MAAqB;QACjC,+EAA+E;QAC/E,qFAAqF;QACrF,mFAAmF;QACnF,qFAAqF;QACrF,oFAAoF;QACpF,oFAAoF;QACpF,4DAA4D;QAC5D,MAAM,IAAI,GAAG,MAAM,IAAI,CAAC,GAAG,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,EAAE,MAAM,CAAC,CAAC;QACrD,OAAO,IAAmB,CAAC;IAC5B,CAAC;CACD"}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@optimystic/db-p2p-storage-ns",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.26.0",
|
|
4
4
|
"type": "module",
|
|
5
5
|
"description": "NativeScript SQLite storage backend for @optimystic/db-p2p",
|
|
6
6
|
"main": "dist/src/index.js",
|
|
@@ -53,14 +53,14 @@
|
|
|
53
53
|
"typescript": "^5.9.3"
|
|
54
54
|
},
|
|
55
55
|
"dependencies": {
|
|
56
|
-
"@optimystic/db-core": "^1.
|
|
57
|
-
"@optimystic/db-p2p": "^1.
|
|
56
|
+
"@optimystic/db-core": "^1.26.0",
|
|
57
|
+
"@optimystic/db-p2p": "^1.26.0",
|
|
58
58
|
"debug": "^4.4.3"
|
|
59
59
|
},
|
|
60
60
|
"peerDependencies": {
|
|
61
61
|
"@libp2p/crypto": "^5.1.23",
|
|
62
62
|
"@libp2p/interface": "^3.3.0",
|
|
63
|
-
"@nativescript-community/sqlite": "^
|
|
63
|
+
"@nativescript-community/sqlite": "^4.1.0"
|
|
64
64
|
},
|
|
65
65
|
"peerDependenciesMeta": {
|
|
66
66
|
"@nativescript-community/sqlite": {
|
package/src/connection-mutex.ts
CHANGED
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
*
|
|
4
4
|
* SQLite allows at most one open transaction per connection, so every mutating
|
|
5
5
|
* operation on a shared connection (plain writes AND transaction bodies) must be
|
|
6
|
-
* serialized or a concurrent `BEGIN`
|
|
6
|
+
* serialized, or a concurrent `BEGIN` fails and its rollback discards another
|
|
7
7
|
* operation's still-open writes.
|
|
8
8
|
*
|
|
9
9
|
* This is deliberately NOT the global `Latches` keyed map from
|
package/src/db.ts
CHANGED
|
@@ -3,12 +3,13 @@
|
|
|
3
3
|
*
|
|
4
4
|
* The storage classes (`SqliteRawStorage`, `SqliteKVStore`) and the identity
|
|
5
5
|
* helper (`loadOrCreateNSPeerKey`) only depend on the `SqliteDb` interface
|
|
6
|
-
* defined here — never on `@nativescript-community/sqlite` directly.
|
|
7
|
-
*
|
|
8
|
-
*
|
|
6
|
+
* defined here — never on `@nativescript-community/sqlite` directly.
|
|
7
|
+
* `ns-opener.ts` implements it over the plugin, and the suite runs that same
|
|
8
|
+
* wrapper under Node mocha over a stand-in for the plugin built on `node:sqlite`,
|
|
9
|
+
* matching the pattern `db-p2p-storage-web` uses with `fake-indexeddb`.
|
|
9
10
|
*
|
|
10
|
-
*
|
|
11
|
-
*
|
|
11
|
+
* Consumers normally need only `openOptimysticNSDb`; `wrapNSPluginDb` is for a host
|
|
12
|
+
* that already holds an open plugin handle.
|
|
12
13
|
*/
|
|
13
14
|
|
|
14
15
|
import { createLogger } from './logger.js';
|
|
@@ -22,11 +23,8 @@ export type SqliteParam = string | number | Uint8Array | null;
|
|
|
22
23
|
export type SqliteRow = Record<string, SqliteParam>;
|
|
23
24
|
|
|
24
25
|
/**
|
|
25
|
-
* A
|
|
26
|
-
*
|
|
27
|
-
* Both `node:sqlite` and `@nativescript-community/sqlite` cache parsed SQL
|
|
28
|
-
* internally, so re-binding a prepared statement is cheaper than re-parsing a
|
|
29
|
-
* raw `execute`. The storage classes prepare each query once per instance.
|
|
26
|
+
* A statement bound to a single SQL string. The storage classes prepare each
|
|
27
|
+
* query once per instance.
|
|
30
28
|
*/
|
|
31
29
|
export interface SqliteStatement {
|
|
32
30
|
/** Execute with the given bind parameters, ignoring any returned rows. */
|
|
@@ -68,9 +66,9 @@ export interface SqliteTransactionOptions {
|
|
|
68
66
|
/**
|
|
69
67
|
* Minimal SQLite driver surface used by this package.
|
|
70
68
|
*
|
|
71
|
-
*
|
|
72
|
-
*
|
|
73
|
-
*
|
|
69
|
+
* Implemented once, over the NativeScript plugin (`wrapNSPluginDb`); in tests the
|
|
70
|
+
* plugin is a stand-in over `node:sqlite`. Async on every method because every
|
|
71
|
+
* plugin call runs on the database's native thread and resolves later.
|
|
74
72
|
*
|
|
75
73
|
* A single connection is shared across the storage classes, so every mutating
|
|
76
74
|
* operation (`exec`, statement `run`, and `transaction` bodies) is serialized on
|
|
@@ -104,15 +102,14 @@ export interface SqliteDb {
|
|
|
104
102
|
|
|
105
103
|
/** The raw connection surface {@link runTransaction} drives: one statement at a time, no rows back. */
|
|
106
104
|
export interface TransactionConnection {
|
|
107
|
-
execute(sql: string): Promise<void
|
|
105
|
+
execute(sql: string): Promise<void>;
|
|
108
106
|
}
|
|
109
107
|
|
|
110
108
|
/**
|
|
111
|
-
* The transaction protocol
|
|
112
|
-
* `
|
|
113
|
-
*
|
|
114
|
-
*
|
|
115
|
-
* released. `BEGIN IMMEDIATE` takes the write lock up front rather than deferring it to the first
|
|
109
|
+
* The transaction protocol `SqliteDb.transaction` runs inside its mutex slot: with `synced`,
|
|
110
|
+
* `PRAGMA synchronous = FULL` first; then `BEGIN IMMEDIATE`, the body, and `COMMIT`, or `ROLLBACK`
|
|
111
|
+
* when the body or the commit throws; then, synced, `PRAGMA synchronous = NORMAL` whichever way the
|
|
112
|
+
* transaction ended, so the connection's default is restored before the slot is released. `BEGIN IMMEDIATE` takes the write lock up front rather than deferring it to the first
|
|
116
113
|
* write, so contention surfaces here and not mid-body.
|
|
117
114
|
*
|
|
118
115
|
* A failure of the restore after a failed body is logged rather than thrown, so the body's error,
|
|
@@ -266,11 +263,9 @@ PRAGMA foreign_keys = OFF;
|
|
|
266
263
|
* migrations can branch on it. Idempotent — safe to call on every open.
|
|
267
264
|
*/
|
|
268
265
|
export async function applySchema(db: SqliteDb, version: number = DEFAULT_DB_VERSION): Promise<void> {
|
|
269
|
-
// `PRAGMA journal_mode = WAL` returns the resulting mode as a row
|
|
270
|
-
//
|
|
271
|
-
//
|
|
272
|
-
// this pragma must run through a query method. prepare().get() does that on
|
|
273
|
-
// every driver (NS plugin → get()/rawQuery; node:sqlite/better-sqlite3 → get()).
|
|
266
|
+
// `PRAGMA journal_mode = WAL` returns the resulting mode as a row, so it goes through a
|
|
267
|
+
// query method. It stays although the plugin turns WAL on by default on Android: iOS has
|
|
268
|
+
// no such default and relies on this pragma.
|
|
274
269
|
await db.prepare('PRAGMA journal_mode = WAL').get();
|
|
275
270
|
await db.exec(PRAGMAS_SQL);
|
|
276
271
|
await db.exec(SCHEMA_SQL);
|
package/src/ns-opener.ts
CHANGED
|
@@ -2,56 +2,61 @@ import { ConnectionMutex } from './connection-mutex.js';
|
|
|
2
2
|
import { applySchema, runTransaction, DEFAULT_DB_NAME, DEFAULT_DB_VERSION, type OptimysticNSDBHandle, type SqliteDb, type SqliteParam, type SqliteRow, type SqliteStatement, type SqliteTransaction, type SqliteTransactionOptions } from './db.js';
|
|
3
3
|
|
|
4
4
|
/**
|
|
5
|
-
* Minimal subset of `@nativescript-community/sqlite`
|
|
5
|
+
* Minimal subset of a `@nativescript-community/sqlite` 4.x database we depend on.
|
|
6
6
|
* Re-declared locally so this module's *types* don't pull in the plugin's
|
|
7
7
|
* declarations (the plugin is a peer dependency that may not be installed
|
|
8
|
-
* at typecheck time on non-NativeScript consumers).
|
|
8
|
+
* at typecheck time on non-NativeScript consumers). Exported for the test
|
|
9
|
+
* driver, not from `index.ts`.
|
|
9
10
|
*/
|
|
10
|
-
interface NSPluginDb {
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
11
|
+
export interface NSPluginDb {
|
|
12
|
+
/** Settles when the open does; a failed open rejects it. */
|
|
13
|
+
readonly ready: Promise<void>;
|
|
14
|
+
execute(sql: string, params?: SqliteParam[]): Promise<unknown>;
|
|
15
|
+
get(sql: string, params?: SqliteParam[]): Promise<Record<string, unknown> | null | undefined>;
|
|
16
|
+
select(sql: string, params?: SqliteParam[]): Promise<Array<Record<string, unknown>>>;
|
|
17
|
+
close(): Promise<void>;
|
|
15
18
|
}
|
|
16
19
|
|
|
17
|
-
interface NSPluginModule {
|
|
18
|
-
|
|
20
|
+
export interface NSPluginModule {
|
|
21
|
+
/** Returns the handle at once; the open itself runs on the database's native thread. */
|
|
22
|
+
openOrCreate(path: string): NSPluginDb;
|
|
19
23
|
}
|
|
20
24
|
|
|
21
25
|
/**
|
|
22
|
-
* Opens (creating if needed) the Optimystic SQLite database at `name
|
|
23
|
-
* the
|
|
24
|
-
*
|
|
26
|
+
* Opens (creating if needed) the Optimystic SQLite database at `name`, applies
|
|
27
|
+
* WAL pragmas, and runs the migration to `version`. A bare `name` goes in the
|
|
28
|
+
* app's databases folder on Android and its documents folder on iOS, and needs
|
|
29
|
+
* no foreground activity; a full path is used as given.
|
|
25
30
|
*
|
|
26
31
|
* The returned `SqliteDb` handle is safe to share across `SqliteRawStorage`,
|
|
27
|
-
* `SqliteKVStore`, and `loadOrCreateNSPeerKey
|
|
28
|
-
*
|
|
29
|
-
*
|
|
30
|
-
* a per-connection FIFO mutex. Without it, two concurrent `transaction` bodies
|
|
31
|
-
* would each `BEGIN` on the shared connection; the second would nest and its
|
|
32
|
-
* rollback would silently discard the first's still-open writes. Reads
|
|
33
|
-
* (`get`/`all`) stay off the mutex to preserve read concurrency.
|
|
34
|
-
*
|
|
35
|
-
* `path` may be passed in as the full filesystem path if the caller wants
|
|
36
|
-
* to control file placement; otherwise the plugin's documents-directory
|
|
37
|
-
* default is used.
|
|
32
|
+
* `SqliteKVStore`, and `loadOrCreateNSPeerKey`: the wrapper serializes every
|
|
33
|
+
* mutating operation through a per-connection FIFO mutex (`NSPluginDbWrapper`
|
|
34
|
+
* says why).
|
|
38
35
|
*/
|
|
39
36
|
export async function openOptimysticNSDb(
|
|
40
37
|
name: string = DEFAULT_DB_NAME,
|
|
41
38
|
version: number = DEFAULT_DB_VERSION,
|
|
42
39
|
): Promise<OptimysticNSDBHandle> {
|
|
43
40
|
const plugin = (await import('@nativescript-community/sqlite' as string)) as unknown as NSPluginModule;
|
|
44
|
-
|
|
41
|
+
return openOverPlugin(plugin, name, version);
|
|
42
|
+
}
|
|
43
|
+
|
|
44
|
+
/** `openOptimysticNSDb` once the plugin is loaded. */
|
|
45
|
+
export async function openOverPlugin(plugin: NSPluginModule, name: string, version: number): Promise<OptimysticNSDBHandle> {
|
|
46
|
+
const raw = plugin.openOrCreate(name);
|
|
47
|
+
await raw.ready;
|
|
45
48
|
const wrapped = wrapNSPluginDb(raw);
|
|
49
|
+
// NOTE: a failed schema leaves this connection open; if a host ever retries the open in a loop, close it here.
|
|
46
50
|
await applySchema(wrapped, version);
|
|
47
51
|
return wrapped;
|
|
48
52
|
}
|
|
49
53
|
|
|
50
54
|
/**
|
|
51
|
-
* Wraps a `@nativescript-community/sqlite`
|
|
55
|
+
* Wraps a `@nativescript-community/sqlite` database to satisfy the
|
|
52
56
|
* internal `SqliteDb` interface used by the storage classes. Exported only
|
|
53
57
|
* for callers that already hold an open NS-plugin handle and want to skip
|
|
54
|
-
* the opener (rare — typically users just call `openOptimysticNSDb`).
|
|
58
|
+
* the opener (rare — typically users just call `openOptimysticNSDb`). The
|
|
59
|
+
* plugin returns a handle before its open has finished, so await `ready` first.
|
|
55
60
|
*
|
|
56
61
|
* NOTE: the serialization mutex lives on the wrapper, not the raw handle. Wrap
|
|
57
62
|
* a given raw connection exactly once — two wrappers over one handle each hold
|
|
@@ -62,6 +67,27 @@ export function wrapNSPluginDb(raw: NSPluginDb): SqliteDb {
|
|
|
62
67
|
return new NSPluginDbWrapper(raw);
|
|
63
68
|
}
|
|
64
69
|
|
|
70
|
+
/**
|
|
71
|
+
* `SqliteDb` over a plugin handle.
|
|
72
|
+
*
|
|
73
|
+
* The plugin runs every call on the database's one native thread, in the order
|
|
74
|
+
* made. That orders single calls but does not keep a sequence of calls together,
|
|
75
|
+
* and every awaited statement yields the JavaScript thread, so other callers run
|
|
76
|
+
* between a transaction body's statements. So the connection mutex stays: two
|
|
77
|
+
* bodies would otherwise each send `BEGIN`, the second would fail, and its
|
|
78
|
+
* `ROLLBACK` would discard the first's open writes; and a plain write between
|
|
79
|
+
* another body's `BEGIN` and `COMMIT` would land inside that transaction and
|
|
80
|
+
* share its fate.
|
|
81
|
+
*
|
|
82
|
+
* The plugin's own `db.transaction` is not used, and would not remove the mutex:
|
|
83
|
+
* it joins a second concurrent `transaction` call into the first, so one body's
|
|
84
|
+
* failure would roll the other back. It would also have to take the place of
|
|
85
|
+
* `runTransaction`, whose `PRAGMA synchronous = FULL` must run outside the
|
|
86
|
+
* transaction (SQLite refuses to change it inside one) and be restored after a
|
|
87
|
+
* rollback. Because the wrapper sends `BEGIN IMMEDIATE` and `COMMIT` as ordinary
|
|
88
|
+
* `execute` calls, the plugin does not know a transaction is open, and runs every
|
|
89
|
+
* request in arrival order.
|
|
90
|
+
*/
|
|
65
91
|
class NSPluginDbWrapper implements SqliteDb {
|
|
66
92
|
private readonly mutex = new ConnectionMutex();
|
|
67
93
|
|
|
@@ -125,8 +151,10 @@ class NSPluginStatement implements SqliteStatement {
|
|
|
125
151
|
|
|
126
152
|
async get(...params: SqliteParam[]): Promise<SqliteRow | undefined> {
|
|
127
153
|
// NOTE: reads run directly on the connection, unserialized, to preserve read
|
|
128
|
-
// concurrency
|
|
129
|
-
//
|
|
154
|
+
// concurrency: a read does not wait for a whole transaction body, only for the
|
|
155
|
+
// statements queued ahead of it on the plugin's native thread. So a read issued
|
|
156
|
+
// while a write transaction is open on this same connection lands between that
|
|
157
|
+
// body's statements and observes its UNCOMMITTED rows (read-your-connection
|
|
130
158
|
// semantics). Fine today: the one transaction writer of the block tables is
|
|
131
159
|
// `SqliteBatchBackend.commit`, and every row it is writing is held by the
|
|
132
160
|
// `WriteGroupDriver` overlay above it until the batch is durable. A point read of
|
|
@@ -144,6 +172,12 @@ class NSPluginStatement implements SqliteStatement {
|
|
|
144
172
|
|
|
145
173
|
async all(...params: SqliteParam[]): Promise<SqliteRow[]> {
|
|
146
174
|
// NOTE: unserialized read — see the note on `get` above re: uncommitted reads.
|
|
175
|
+
// NOTE: on NativeScript 8's JNI bridge a result over 512 KiB arrives through a fresh
|
|
176
|
+
// direct ByteBuffer, which holds a JNI global reference until it is collected, and
|
|
177
|
+
// Android allows 51,200 of those. This storage reaches that size with one block near
|
|
178
|
+
// or over 512 KiB per read (`get`, like this method) or `listBlockIds` over a large
|
|
179
|
+
// table. Fine now; if a node on NativeScript 8 ever reads thousands of such results
|
|
180
|
+
// between garbage collections, read them in smaller pieces.
|
|
147
181
|
const rows = await this.raw.select(this.sql, params);
|
|
148
182
|
return rows as SqliteRow[];
|
|
149
183
|
}
|