@quereus/store 3.2.1 → 4.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (48) hide show
  1. package/README.md +436 -317
  2. package/dist/src/common/backing-host.d.ts +198 -0
  3. package/dist/src/common/backing-host.d.ts.map +1 -0
  4. package/dist/src/common/backing-host.js +452 -0
  5. package/dist/src/common/backing-host.js.map +1 -0
  6. package/dist/src/common/bytes.d.ts +16 -0
  7. package/dist/src/common/bytes.d.ts.map +1 -0
  8. package/dist/src/common/bytes.js +33 -0
  9. package/dist/src/common/bytes.js.map +1 -0
  10. package/dist/src/common/cached-kv-store.d.ts +3 -3
  11. package/dist/src/common/cached-kv-store.d.ts.map +1 -1
  12. package/dist/src/common/cached-kv-store.js +6 -4
  13. package/dist/src/common/cached-kv-store.js.map +1 -1
  14. package/dist/src/common/encoding.d.ts +9 -1
  15. package/dist/src/common/encoding.d.ts.map +1 -1
  16. package/dist/src/common/encoding.js +12 -2
  17. package/dist/src/common/encoding.js.map +1 -1
  18. package/dist/src/common/index.d.ts +8 -6
  19. package/dist/src/common/index.d.ts.map +1 -1
  20. package/dist/src/common/index.js +7 -3
  21. package/dist/src/common/index.js.map +1 -1
  22. package/dist/src/common/key-builder.d.ts +125 -4
  23. package/dist/src/common/key-builder.d.ts.map +1 -1
  24. package/dist/src/common/key-builder.js +186 -9
  25. package/dist/src/common/key-builder.js.map +1 -1
  26. package/dist/src/common/kv-store.d.ts +75 -4
  27. package/dist/src/common/kv-store.d.ts.map +1 -1
  28. package/dist/src/common/memory-store.d.ts +3 -3
  29. package/dist/src/common/memory-store.d.ts.map +1 -1
  30. package/dist/src/common/memory-store.js +4 -2
  31. package/dist/src/common/memory-store.js.map +1 -1
  32. package/dist/src/common/store-connection.d.ts +11 -4
  33. package/dist/src/common/store-connection.d.ts.map +1 -1
  34. package/dist/src/common/store-connection.js +12 -6
  35. package/dist/src/common/store-connection.js.map +1 -1
  36. package/dist/src/common/store-module.d.ts +580 -20
  37. package/dist/src/common/store-module.d.ts.map +1 -1
  38. package/dist/src/common/store-module.js +1590 -135
  39. package/dist/src/common/store-module.js.map +1 -1
  40. package/dist/src/common/store-table.d.ts +347 -14
  41. package/dist/src/common/store-table.d.ts.map +1 -1
  42. package/dist/src/common/store-table.js +751 -98
  43. package/dist/src/common/store-table.js.map +1 -1
  44. package/dist/src/common/transaction.d.ts +135 -27
  45. package/dist/src/common/transaction.d.ts.map +1 -1
  46. package/dist/src/common/transaction.js +216 -55
  47. package/dist/src/common/transaction.js.map +1 -1
  48. package/package.json +5 -5
package/README.md CHANGED
@@ -1,317 +1,436 @@
1
- # @quereus/store
2
-
3
- Abstract key-value storage module for [Quereus](https://github.com/gotchoices/quereus). Provides platform-agnostic interfaces and a generic `StoreModule` virtual table implementation.
4
-
5
- ## Architecture
6
-
7
- This package provides the **abstract layer** that separates virtual table logic from platform-specific storage:
8
-
9
- ```
10
- @quereus/store (this package)
11
- ├── KVStore interface - Abstract key-value store
12
- ├── KVStoreProvider interface - Store factory/management
13
- ├── StoreModule - Generic VirtualTableModule
14
- ├── StoreTable - Generic virtual table implementation
15
- ├── StoreConnection - Generic transaction support
16
- └── Common utilities - Encoding, serialization, events
17
-
18
- @quereus/plugin-leveldb (Node.js) @quereus/plugin-indexeddb (Browser)
19
- ├── LevelDBStore ├── IndexedDBStore
20
- ├── LevelDBProvider ├── IndexedDBProvider
21
- └── Plugin registration ├── IndexedDBManager
22
- └── CrossTabSync
23
- ```
24
-
25
- This architecture enables:
26
- - **Platform portability** - Same SQL tables work across Node.js, browsers, and mobile
27
- - **Custom storage backends** - Implement `KVStore` for IndexedDB, LevelDB, LMDB, or other "NoSQL" stores
28
- - **Dependency injection** - Use `KVStoreProvider` for store management
29
-
30
- ## Storage Architecture
31
-
32
- The store module uses separate logical stores for different data types:
33
-
34
- **Store Naming Convention:**
35
- - `{schema}.{table}` - Data store (row data)
36
- - `{schema}.{table}_idx_{indexName}` - Index stores (one per secondary index)
37
- - `{prefix}.__stats__` - Unified stats store (row counts for all tables)
38
- - `__catalog__` - Catalog store (DDL metadata)
39
-
40
- **Key Formats:**
41
- - **Data keys**: Encoded primary key (no prefix)
42
- - **Index keys**: Encoded index columns + encoded PK
43
- - **Catalog keys**: `{schema}.{table}` as string
44
-
45
- This design eliminates redundant prefixes and groups related stores together by table name.
46
-
47
- ## Installation
48
-
49
- ```bash
50
- npm install @quereus/store
51
- ```
52
-
53
- For platform-specific implementations:
54
- ```bash
55
- # Node.js
56
- npm install @quereus/plugin-leveldb
57
-
58
- # Browser
59
- npm install @quereus/plugin-indexeddb
60
- ```
61
-
62
- ## Usage
63
-
64
- ### With a Provider
65
-
66
- ```typescript
67
- import { Database } from '@quereus/quereus';
68
- import { StoreModule } from '@quereus/store';
69
- import { createLevelDBProvider } from '@quereus/plugin-leveldb';
70
- // OR: import { createIndexedDBProvider } from '@quereus/plugin-indexeddb';
71
-
72
- const db = new Database();
73
-
74
- // Create provider for your platform
75
- const provider = createLevelDBProvider({ basePath: './data' });
76
-
77
- // Create the generic store module with your provider
78
- const storeModule = new StoreModule(provider);
79
- db.registerModule('store', storeModule);
80
-
81
- // Use it in SQL
82
- await db.exec(`
83
- create table users (id integer primary key, name text)
84
- using store
85
- `);
86
- ```
87
-
88
- ### Custom Storage Backend
89
-
90
- Implement `KVStore` and `KVStoreProvider` to create custom storage backends:
91
-
92
- ```typescript
93
- import type { KVStore, KVStoreProvider } from '@quereus/store';
94
-
95
- class MyCustomStore implements KVStore {
96
- async get(key: Uint8Array) { /* ... */ }
97
- async put(key: Uint8Array, value: Uint8Array) { /* ... */ }
98
- async delete(key: Uint8Array) { /* ... */ }
99
- async has(key: Uint8Array) { /* ... */ }
100
- iterate(options?: IterateOptions) { /* ... */ }
101
- batch() { /* ... */ }
102
- async close() { /* ... */ }
103
- async approximateCount(options?: IterateOptions) { /* ... */ }
104
- }
105
-
106
- class MyCustomProvider implements KVStoreProvider {
107
- async getStore(schemaName: string, tableName: string) {
108
- return new MyCustomStore(/* ... */);
109
- }
110
- async getIndexStore(schemaName: string, tableName: string, indexName: string) {
111
- return new MyCustomStore(/* ... */);
112
- }
113
- async getStatsStore(schemaName: string, tableName: string) {
114
- return new MyCustomStore(/* ... */);
115
- }
116
- async getCatalogStore() { /* ... */ }
117
- async closeStore(schemaName: string, tableName: string) { /* ... */ }
118
- async closeIndexStore(schemaName: string, tableName: string, indexName: string) { /* ... */ }
119
- async closeAll() { /* ... */ }
120
- }
121
-
122
- // Use it with StoreModule
123
- const provider = new MyCustomProvider();
124
- const module = new StoreModule(provider);
125
- db.registerModule('store', module);
126
- ```
127
-
128
- ## KVStore Interface
129
-
130
- The `KVStore` interface is the foundation for all storage backends:
131
-
132
- ```typescript
133
- interface KVStore {
134
- get(key: Uint8Array): Promise<Uint8Array | undefined>;
135
- put(key: Uint8Array, value: Uint8Array): Promise<void>;
136
- delete(key: Uint8Array): Promise<void>;
137
- has(key: Uint8Array): Promise<boolean>;
138
- iterate(options?: IterateOptions): AsyncIterable<KVEntry>;
139
- batch(): WriteBatch;
140
- close(): Promise<void>;
141
- approximateCount(options?: IterateOptions): Promise<number>;
142
- }
143
-
144
- interface KVStoreProvider {
145
- // Get data store for a table
146
- getStore(schemaName: string, tableName: string): Promise<KVStore>;
147
-
148
- // Get index store for a secondary index
149
- getIndexStore(schemaName: string, tableName: string, indexName: string): Promise<KVStore>;
150
-
151
- // Get stats store for table statistics
152
- getStatsStore(schemaName: string, tableName: string): Promise<KVStore>;
153
-
154
- // Get catalog store for DDL metadata
155
- getCatalogStore(): Promise<KVStore>;
156
-
157
- // Close specific stores
158
- closeStore(schemaName: string, tableName: string): Promise<void>;
159
- closeIndexStore(schemaName: string, tableName: string, indexName: string): Promise<void>;
160
- closeAll(): Promise<void>;
161
-
162
- // Optional: Delete stores
163
- deleteIndexStore?(schemaName: string, tableName: string, indexName: string): Promise<void>;
164
- deleteTableStores?(schemaName: string, tableName: string): Promise<void>;
165
- }
166
- ```
167
-
168
- ## Module Capabilities
169
-
170
- The `StoreModule` reports its capabilities via `getCapabilities()`:
171
-
172
- ```typescript
173
- const storeModule = new StoreModule(provider);
174
- const caps = storeModule.getCapabilities();
175
-
176
- // {
177
- // isolation: false, // Store module does NOT provide transaction isolation
178
- // savepoints: false, // No savepoint support without isolation layer
179
- // persistent: true, // Data persists across restarts
180
- // secondaryIndexes: true,// Supports secondary indexes
181
- // rangeScans: true // Supports range scans
182
- // }
183
- ```
184
-
185
- **Important:** The base `StoreModule` does not provide transaction isolation. Without isolation:
186
- - Reads see only committed data (no read-your-own-writes within a transaction)
187
- - Concurrent readers may see partial writes
188
- - Savepoints are not supported
189
-
190
- ## Transaction Isolation
191
-
192
- To add full ACID transaction semantics with read-your-own-writes, wrap the store module with the `IsolationModule`:
193
-
194
- ```typescript
195
- import { Database, MemoryTableModule } from '@quereus/quereus';
196
- import { IsolationModule } from '@quereus/isolation';
197
- import { StoreModule, createIsolatedStoreModule } from '@quereus/store';
198
- import { createLevelDBProvider } from '@quereus/plugin-leveldb';
199
-
200
- const db = new Database();
201
- const provider = createLevelDBProvider({ basePath: './data' });
202
-
203
- // Option 1: Use the convenience function
204
- const isolatedModule = createIsolatedStoreModule({ provider });
205
- db.registerModule('store', isolatedModule);
206
-
207
- // Option 2: Manual wrapping for more control
208
- const storeModule = new StoreModule(provider);
209
- const isolatedModule = new IsolationModule({
210
- underlying: storeModule,
211
- overlay: new MemoryTableModule(),
212
- });
213
- db.registerModule('store', isolatedModule);
214
-
215
- // Now transactions have full isolation
216
- await db.exec('BEGIN');
217
- await db.exec(`INSERT INTO users VALUES (1, 'Alice')`);
218
-
219
- // Read-your-own-writes: sees uncommitted insert
220
- const user = await db.get('SELECT * FROM users WHERE id = 1');
221
- console.log(user.name); // 'Alice'
222
-
223
- await db.exec('COMMIT'); // Or ROLLBACK to discard
224
- ```
225
-
226
- The isolation layer provides:
227
- - **Read-your-own-writes** within transactions
228
- - **Snapshot isolation** for consistent reads
229
- - **Savepoint support** via the overlay module
230
-
231
- ### Checking for Isolation Support
232
-
233
- ```typescript
234
- import { hasIsolation } from '@quereus/store';
235
-
236
- const storeModule = new StoreModule(provider);
237
- console.log(hasIsolation(storeModule)); // false
238
-
239
- const isolatedModule = createIsolatedStoreModule({ provider });
240
- console.log(hasIsolation(isolatedModule)); // true
241
- ```
242
-
243
- ## API
244
-
245
- ### Core Exports
246
-
247
- | Export | Description |
248
- |--------|-------------|
249
- | `KVStore` | Key-value store interface (type) |
250
- | `KVStoreProvider` | Store factory interface (type) |
251
- | `WriteBatch` | Batch write interface (type) |
252
- | `IterateOptions` | Iteration options (type) |
253
- | `StoreModule` | Generic VirtualTableModule |
254
- | `StoreTable` | Virtual table implementation |
255
- | `StoreConnection` | Transaction connection |
256
- | `TransactionCoordinator` | Transaction management |
257
- | `StoreEventEmitter` | Event system for data/schema changes |
258
-
259
- ### Isolation Layer Utilities
260
-
261
- | Export | Description |
262
- |--------|-------------|
263
- | `createIsolatedStoreModule` | Create store module with isolation layer |
264
- | `hasIsolation` | Check if a module has isolation capability |
265
- | `IsolatedStoreModuleConfig` | Configuration for isolated store module |
266
-
267
- ### Caching
268
-
269
- | Export | Description |
270
- |--------|-------------|
271
- | `CachedKVStore` | Read-through LRU cache wrapper for any `KVStore` |
272
- | `CacheOptions` | Configuration for cache (maxEntries, maxBytes, enabled) |
273
-
274
- ### Encoding Utilities
275
-
276
- | Export | Description |
277
- |--------|-------------|
278
- | `encodeValue` | Encode a SQL value to sortable bytes |
279
- | `decodeValue` | Decode bytes back to SQL value |
280
- | `encodeCompositeKey` | Encode multiple values as composite key |
281
- | `decodeCompositeKey` | Decode composite key to values |
282
- | `registerCollationEncoder` | Register custom collation |
283
-
284
- ### Serialization Utilities
285
-
286
- | Export | Description |
287
- |--------|-------------|
288
- | `serializeRow` | Serialize a row to bytes |
289
- | `deserializeRow` | Deserialize bytes to row |
290
- | `serializeValue` | Serialize a single value |
291
- | `deserializeValue` | Deserialize a single value |
292
-
293
- ### Key Building
294
-
295
- | Export | Description |
296
- |--------|-------------|
297
- | `buildDataStoreName` | Build store name for table data |
298
- | `buildIndexStoreName` | Build store name for an index |
299
- | `buildStatsStoreName` | Build store name for table stats |
300
- | `buildDataKey` | Build key for row data (encoded PK) |
301
- | `buildIndexKey` | Build key for index entry |
302
- | `buildCatalogKey` | Build key for catalog metadata |
303
- | `buildFullScanBounds` | Build bounds for full table scan |
304
- | `buildIndexPrefixBounds` | Build bounds for index prefix scan |
305
- | `buildCatalogScanBounds` | Build bounds for catalog scan |
306
- | `CATALOG_STORE_NAME` | Reserved catalog store name constant |
307
- | `STORE_SUFFIX` | Store name suffixes (INDEX, STATS) |
308
-
309
- ## Related Packages
310
-
311
- - [`@quereus/plugin-leveldb`](../quereus-plugin-leveldb/) - LevelDB implementation for Node.js
312
- - [`@quereus/plugin-indexeddb`](../quereus-plugin-indexeddb/) - IndexedDB implementation for browsers
313
- - [`@quereus/sync`](../quereus-sync/) - CRDT sync layer
314
-
315
- ## License
316
-
317
- MIT
1
+ # @quereus/store
2
+
3
+ Abstract key-value storage module for [Quereus](https://github.com/gotchoices/quereus). Provides platform-agnostic interfaces and a generic `StoreModule` virtual table implementation.
4
+
5
+ ## Architecture
6
+
7
+ This package provides the **abstract layer** that separates virtual table logic from platform-specific storage:
8
+
9
+ ```
10
+ @quereus/store (this package)
11
+ ├── KVStore interface - Abstract key-value store
12
+ ├── KVStoreProvider interface - Store factory/management
13
+ ├── StoreModule - Generic VirtualTableModule
14
+ ├── StoreTable - Generic virtual table implementation
15
+ ├── StoreConnection - Generic transaction support
16
+ └── Common utilities - Encoding, serialization, events
17
+
18
+ @quereus/plugin-leveldb (Node.js) @quereus/plugin-indexeddb (Browser)
19
+ ├── LevelDBStore ├── IndexedDBStore
20
+ ├── LevelDBProvider ├── IndexedDBProvider
21
+ └── Plugin registration ├── IndexedDBManager
22
+ └── CrossTabSync
23
+ ```
24
+
25
+ This architecture enables:
26
+ - **Platform portability** - Same SQL tables work across Node.js, browsers, and mobile
27
+ - **Custom storage backends** - Implement `KVStore` for IndexedDB, LevelDB, LMDB, or other "NoSQL" stores
28
+ - **Dependency injection** - Use `KVStoreProvider` for store management
29
+
30
+ ## Storage Architecture
31
+
32
+ The store module uses separate logical stores for different data types:
33
+
34
+ **Store Naming Convention:**
35
+ - `{schema}.{table}` - Data store (row data)
36
+ - `{schema}.{table}_idx_{indexName}` - Index stores (one per secondary index)
37
+ - `{prefix}.__stats__` - Unified stats store (row counts for all tables)
38
+ - `__catalog__` - Catalog store (DDL metadata)
39
+
40
+ **Key Formats:**
41
+ - **Data keys**: Encoded primary key (no prefix)
42
+ - **Index keys**: Encoded index columns + encoded PK
43
+ - **Catalog keys**:
44
+ - Tables: `{schema}.{table}` as a string (the `CREATE TABLE` bundle, with its index DDL and any exposed-implicit-index tag DDL)
45
+ - Views: `\x00view\x00{schema}.{view}` (reserved-prefix; `generateViewDDL`)
46
+ - Materialized views: `\x00mview\x00{schema}.{mv}` (reserved-prefix; `generateMaterializedViewDDL`)
47
+
48
+ This design eliminates redundant prefixes and groups related stores together by table name. The leading-`0x00` view/MV prefixes never collide with an unprefixed table key, so a view/MV may safely share a name with a table; a full catalog scan returns all three kinds intermixed and rehydrate classifies each by its key prefix.
49
+
50
+ **Catalog DDL is re-persisted on catalog-only mutations.** `ALTER … SET TAGS` (and the programmatic `setTableTags` / `setColumnTags` / `setConstraintTags` / `setViewTags` / `setMaterializedViewTags`), plus `CREATE`/`DROP VIEW` and `CREATE`/`DROP MATERIALIZED VIEW`, never reach `module.alterTable`/`module.destroy`. The module subscribes to the engine's schema-change events (`table_modified`, the `view_*` events, and the `materialized_view_*` events) and writes the matching `__catalog__` entry when its `generate*DDL` output changes — table / column / constraint / **index** / **view** / **materialized-view** tags, and view/MV lifecycle, all survive close → reopen. A table's bundle is its `CREATE TABLE` DDL, one `CREATE [UNIQUE] INDEX` line per secondary index, and one trailing `alter index … set tags (…)` line per *exposed implicit index* carrying user tags (an exposed implicit index is never materialized in store mode, so its `UniqueConstraintSchema.exposedIndexTags` has no `CREATE INDEX` line to ride; the alter line re-applies silently on import). These async writes are serialized and drained by `closeAll()` (or the `whenCatalogPersisted()` barrier) before the provider closes. On reopen, `rehydrateCatalog` classifies entries by key prefix and imports them in phases — tables → views → materialized views, all through the engine's `importCatalog` (MVs re-materialize silently via the shared create core, dependency-ordered for MV-over-MV by fixpoint retry). See [`docs/schema.md`](../../docs/schema.md#view-and-materialized-view-persistence) for the full design.
51
+
52
+ ## Installation
53
+
54
+ ```bash
55
+ npm install @quereus/store
56
+ ```
57
+
58
+ For platform-specific implementations:
59
+ ```bash
60
+ # Node.js
61
+ npm install @quereus/plugin-leveldb
62
+
63
+ # Browser
64
+ npm install @quereus/plugin-indexeddb
65
+ ```
66
+
67
+ ## Usage
68
+
69
+ ### With a Provider
70
+
71
+ ```typescript
72
+ import { Database } from '@quereus/quereus';
73
+ import { StoreModule } from '@quereus/store';
74
+ import { createLevelDBProvider } from '@quereus/plugin-leveldb';
75
+ // OR: import { createIndexedDBProvider } from '@quereus/plugin-indexeddb';
76
+
77
+ const db = new Database();
78
+
79
+ // Create provider for your platform
80
+ const provider = createLevelDBProvider({ basePath: './data' });
81
+
82
+ // Create the generic store module with your provider
83
+ const storeModule = new StoreModule(provider);
84
+ db.registerModule('store', storeModule);
85
+
86
+ // Use it in SQL
87
+ await db.exec(`
88
+ create table users (id integer primary key, name text)
89
+ using store
90
+ `);
91
+ ```
92
+
93
+ ### Custom Storage Backend
94
+
95
+ Implement `KVStore` and `KVStoreProvider` to create custom storage backends:
96
+
97
+ ```typescript
98
+ import type { KVStore, KVStoreProvider } from '@quereus/store';
99
+
100
+ class MyCustomStore implements KVStore {
101
+ async get(key: Uint8Array) { /* ... */ }
102
+ async put(key: Uint8Array, value: Uint8Array) { /* ... */ }
103
+ async delete(key: Uint8Array) { /* ... */ }
104
+ async has(key: Uint8Array) { /* ... */ }
105
+ iterate(options?: IterateOptions) { /* ... */ }
106
+ batch() { /* ... */ }
107
+ async close() { /* ... */ }
108
+ async approximateCount(options?: IterateOptions) { /* ... */ }
109
+ }
110
+
111
+ class MyCustomProvider implements KVStoreProvider {
112
+ async getStore(schemaName: string, tableName: string) {
113
+ return new MyCustomStore(/* ... */);
114
+ }
115
+ async getIndexStore(schemaName: string, tableName: string, indexName: string) {
116
+ return new MyCustomStore(/* ... */);
117
+ }
118
+ async getStatsStore(schemaName: string, tableName: string) {
119
+ return new MyCustomStore(/* ... */);
120
+ }
121
+ async getCatalogStore() { /* ... */ }
122
+ async closeStore(schemaName: string, tableName: string) { /* ... */ }
123
+ async closeIndexStore(schemaName: string, tableName: string, indexName: string) { /* ... */ }
124
+ async closeAll() { /* ... */ }
125
+ }
126
+
127
+ // Use it with StoreModule
128
+ const provider = new MyCustomProvider();
129
+ const module = new StoreModule(provider);
130
+ db.registerModule('store', module);
131
+ ```
132
+
133
+ ## KVStore Interface
134
+
135
+ The `KVStore` interface is the foundation for all storage backends:
136
+
137
+ ```typescript
138
+ interface KVStore {
139
+ get(key: Uint8Array): Promise<Uint8Array | undefined>;
140
+ put(key: Uint8Array, value: Uint8Array): Promise<void>;
141
+ delete(key: Uint8Array): Promise<void>;
142
+ has(key: Uint8Array): Promise<boolean>;
143
+ iterate(options?: IterateOptions): AsyncIterable<KVEntry>;
144
+ batch(): WriteBatch;
145
+ close(): Promise<void>;
146
+ approximateCount(options?: IterateOptions): Promise<number>;
147
+ }
148
+
149
+ interface KVStoreProvider {
150
+ // Get data store for a table
151
+ getStore(schemaName: string, tableName: string): Promise<KVStore>;
152
+
153
+ // Get index store for a secondary index
154
+ getIndexStore(schemaName: string, tableName: string, indexName: string): Promise<KVStore>;
155
+
156
+ // Get stats store for table statistics
157
+ getStatsStore(schemaName: string, tableName: string): Promise<KVStore>;
158
+
159
+ // Get catalog store for DDL metadata
160
+ getCatalogStore(): Promise<KVStore>;
161
+
162
+ // Close specific stores
163
+ closeStore(schemaName: string, tableName: string): Promise<void>;
164
+ closeIndexStore(schemaName: string, tableName: string, indexName: string): Promise<void>;
165
+ closeAll(): Promise<void>;
166
+
167
+ // Optional: Delete stores. `indexNames` is the table's exact secondary-index
168
+ // names (from the schema) — build index store names from it via
169
+ // buildIndexStoreName instead of prefix-scanning `{table}_idx_`, which would
170
+ // also match a sibling table literally named `{table}_idx_<x>`.
171
+ deleteIndexStore?(schemaName: string, tableName: string, indexName: string): Promise<void>;
172
+ deleteTableStores?(schemaName: string, tableName: string, indexNames: readonly string[]): Promise<void>;
173
+
174
+ // Optional: Relocate a table's data + index stores for ALTER TABLE ... RENAME TO
175
+ // (`indexNames` carries the same authoritative, exact index list).
176
+ renameTableStores?(schemaName: string, oldName: string, newName: string, indexNames: readonly string[]): Promise<void>;
177
+ }
178
+ ```
179
+
180
+ ## Module Capabilities
181
+
182
+ The `StoreModule` reports its capabilities via `getCapabilities()`:
183
+
184
+ ```typescript
185
+ const storeModule = new StoreModule(provider);
186
+ const caps = storeModule.getCapabilities();
187
+
188
+ // {
189
+ // isolation: false, // Store module does NOT provide transaction isolation
190
+ // savepoints: true, // Coordinator-buffered ops support savepoints within a transaction
191
+ // persistent: true, // Data persists across restarts
192
+ // secondaryIndexes: true,// Supports secondary indexes
193
+ // rangeScans: true // Supports range scans
194
+ // }
195
+ ```
196
+
197
+ **Important:** The base `StoreModule` does not provide transaction isolation:
198
+ - No snapshot isolation: between connections, reads see only committed data, and concurrent readers may observe partial writes
199
+ - Within a transaction, reads through the table's shared coordinator DO see that transaction's own pending writes (read-your-own-writes). This extends to DML's own internal reads: the insert PK-conflict probe, the update/delete old-image reads, and the update PK-change conflict probe all read through the pending merge, so an INSERT/UPDATE/DELETE against a row written earlier in the same transaction sees that pending row — it raises a UNIQUE conflict (or evicts under `OR REPLACE`), cleans up secondary-index entries, tracks the correct row-count delta, and emits events carrying the pending `oldRow`
200
+ - Savepoints (create / release / rollback-to) work within a transaction via the coordinator's buffered op log
201
+ - Caveat: a DDL-commit operation (`replaceContents` / `renameTable`, e.g. `refresh materialized view` or `alter table … rename`) commits the coordinator mid-transaction, clearing the savepoint stack. A later `rollback to` / `release` targeting a now-vanished savepoint degrades to a no-op (warn-and-return) rather than throwing; the committed DDL and everything before it stays committed
202
+
203
+ ## Atomic multi-store commit (module-wide, cross-table)
204
+
205
+ A single `TransactionCoordinator` is shared by **every table of one storage
206
+ module** — it is the unit of cross-table atomicity. Every buffered op is
207
+ addressed by its explicit target `KVStore` handle (data ops, secondary-index
208
+ ops, and backing-host writes alike), so a transaction touching tables A and B
209
+ accumulates all of their stores' ops in one coordinator. Because the engine
210
+ commits virtual-table connections **sequentially** and the coordinator's
211
+ `commit()`/`rollback()` are **idempotent**, the first connection to commit
212
+ flushes **every** touched store of **every** table the transaction wrote; the
213
+ remaining connections no-op.
214
+
215
+ `TransactionCoordinator.commit()` thus writes each table's data store and each of
216
+ its secondary-index stores. By default it writes **one `KVStore.batch()` per
217
+ store, sequentially** a crash between those batches can leave tables/indexes
218
+ divergent on disk, with no automatic healing (no worse than the prior per-table
219
+ commits, which were already non-atomic across tables).
220
+
221
+ A provider whose stores share a single durable commit domain can close that
222
+ window by implementing the optional `KVStoreProvider.beginAtomicBatch()`:
223
+
224
+ ```typescript
225
+ interface AtomicBatch {
226
+ put(store: KVStore, key: Uint8Array, value: Uint8Array): void;
227
+ delete(store: KVStore, key: Uint8Array): void;
228
+ write(): Promise<void>; // one durable, all-or-nothing physical commit
229
+ clear(): void;
230
+ }
231
+
232
+ interface KVStoreProvider {
233
+ // ...
234
+ // Open a batch spanning this provider's stores, or undefined when the provider
235
+ // has no shared commit domain (the coordinator then falls back to per-store batch()).
236
+ beginAtomicBatch?(): AtomicBatch | undefined;
237
+ }
238
+ ```
239
+
240
+ The batch addresses stores by **`KVStore` handle**, so it composes with the
241
+ coordinator's existing per-store bucketing without a name lookup. When present,
242
+ `commit()` queues every pending op — every store of **every table** in the
243
+ transaction — into one `AtomicBatch` and issues a single `write()`; all of those
244
+ tables commit or roll back together. When absent — or when the factory returns
245
+ `undefined` behavior is byte-identical to the per-store loop, so providers
246
+ without a shared domain are unaffected.
247
+
248
+ The capability surface spans **multiple stores of one provider** (every store of
249
+ every table the module owns), giving full module-wide cross-table atomicity with
250
+ no interface change. The
251
+ [`@quereus/plugin-indexeddb`](../quereus-plugin-indexeddb) provider implements it
252
+ over its single IndexedDB database (multiple object stores, one
253
+ `db.transaction(...,'readwrite')`), invalidating each touched store's read cache
254
+ after a successful write so read-your-own-writes survives the cache.
255
+
256
+ ## Materialized-View Backing Host
257
+
258
+ The store module implements the engine's backing-host capability
259
+ (`StoreBackingHost`), so `create materialized view mv using store as <body>`
260
+ places the MV's backing table in persistent storage. Maintenance writes ride
261
+ the module's shared `TransactionCoordinator`'s pending state (committing/rolling
262
+ back in lockstep with the source write — and, since the coordinator is
263
+ module-wide, in the same all-or-nothing batch as a write to a same-module
264
+ source), mid-transaction reads of the MV see pending
265
+ maintenance through the read-your-own-writes merge, and the backing's text
266
+ primary-key columns are keyed under the store's `collation` arg (default
267
+ `NOCASE` — pass `using store(collation = 'BINARY')` for byte-exact keys). The
268
+ isolation wrapper forwards the capability automatically. See
269
+ [`docs/materialized-views.md` § The store host](../../docs/materialized-views.md#the-store-host-using-store).
270
+
271
+ ## External Row-Write Entry Point
272
+
273
+ `StoreTable.applyExternalRowChanges(ops)` applies trusted, externally-originated
274
+ row writes (e.g. inbound replication) directly to a **source** table's committed
275
+ storage — table-owned data-key put/delete, **secondary-index maintenance**, and
276
+ stats tracking and returns the effective `BackingRowChange[]` (the shape
277
+ `Database.ingestExternalRowChanges` consumes). It is the index-maintaining
278
+ sibling of the backing host (whose MV backing tables carry no indexes): a caller
279
+ writing the data `KVStore` directly would silently skip index and stats upkeep.
280
+
281
+ Resolve the table with `StoreModule.getTableForExternalWrite(db, schema, table)`
282
+ (same ownership/wrapper resolution as `getBackingHost`), read a row's current
283
+ image with `StoreTable.readRowByPk(pk)`, then apply one `ExternalRowOp` per row:
284
+
285
+ ```typescript
286
+ const table = storeModule.getTableForExternalWrite(db, 'main', 'users');
287
+ if (table) {
288
+ const changes = await table.applyExternalRowChanges([
289
+ { op: 'upsert', row: [1, 'alice'] }, // full row, schema column order
290
+ { op: 'delete', pk: [2] }, // PK values, PK-definition order
291
+ ]);
292
+ }
293
+ ```
294
+
295
+ Deliberately emits **no** module data events (the caller owns emission and the
296
+ `remote` flag), opens **no** coordinator transaction (writes commit at once,
297
+ last-writer-wins against any pending local transaction), and runs **no**
298
+ constraint validation (the origin is trusted). No-ops are suppressed: a delete
299
+ of an absent key and a value-identical upsert (byte-faithful) write nothing and
300
+ report nothing.
301
+
302
+ ## Transaction Isolation
303
+
304
+ To add full ACID transaction semantics with snapshot isolation, wrap the store module with the `IsolationModule`:
305
+
306
+ ```typescript
307
+ import { Database, MemoryTableModule } from '@quereus/quereus';
308
+ import { IsolationModule } from '@quereus/isolation';
309
+ import { StoreModule, createIsolatedStoreModule } from '@quereus/store';
310
+ import { createLevelDBProvider } from '@quereus/plugin-leveldb';
311
+
312
+ const db = new Database();
313
+ const provider = createLevelDBProvider({ basePath: './data' });
314
+
315
+ // Option 1: Use the convenience function
316
+ const isolatedModule = createIsolatedStoreModule({ provider });
317
+ db.registerModule('store', isolatedModule);
318
+
319
+ // Option 2: Manual wrapping for more control
320
+ const storeModule = new StoreModule(provider);
321
+ const isolatedModule = new IsolationModule({
322
+ underlying: storeModule,
323
+ overlay: new MemoryTableModule(),
324
+ });
325
+ db.registerModule('store', isolatedModule);
326
+
327
+ // Now transactions have full isolation
328
+ await db.exec('BEGIN');
329
+ await db.exec(`INSERT INTO users VALUES (1, 'Alice')`);
330
+
331
+ // Read-your-own-writes: sees uncommitted insert
332
+ const user = await db.get('SELECT * FROM users WHERE id = 1');
333
+ console.log(user.name); // 'Alice'
334
+
335
+ await db.exec('COMMIT'); // Or ROLLBACK to discard
336
+ ```
337
+
338
+ The isolation layer provides:
339
+ - **Read-your-own-writes** within transactions
340
+ - **Snapshot isolation** for consistent reads
341
+ - **Savepoint support** via the overlay module
342
+
343
+ ### Checking for Isolation Support
344
+
345
+ ```typescript
346
+ import { hasIsolation } from '@quereus/store';
347
+
348
+ const storeModule = new StoreModule(provider);
349
+ console.log(hasIsolation(storeModule)); // false
350
+
351
+ const isolatedModule = createIsolatedStoreModule({ provider });
352
+ console.log(hasIsolation(isolatedModule)); // true
353
+ ```
354
+
355
+ ## API
356
+
357
+ ### Core Exports
358
+
359
+ | Export | Description |
360
+ |--------|-------------|
361
+ | `KVStore` | Key-value store interface (type) |
362
+ | `KVStoreProvider` | Store factory interface (type) |
363
+ | `WriteBatch` | Batch write interface (type) |
364
+ | `AtomicBatch` | Cross-store all-or-nothing batch from `KVStoreProvider.beginAtomicBatch` (type) |
365
+ | `IterateOptions` | Iteration options (type) |
366
+ | `StoreModule` | Generic VirtualTableModule |
367
+ | `StoreTable` | Virtual table implementation (incl. `applyExternalRowChanges` / `readRowByPk` for externally-applied source writes) |
368
+ | `ExternalRowOp` | One externally-applied row op (`upsert`/`delete`) for `StoreTable.applyExternalRowChanges` (type) |
369
+ | `resolvePkKeyCollations` | Per-PK-column key collations (pass to `buildDataKey`/`buildIndexKey` to match `StoreTable`'s key bytes) |
370
+ | `StoreConnection` | Transaction connection |
371
+ | `TransactionCoordinator` | Transaction management |
372
+ | `StoreEventEmitter` | Event system for data/schema changes |
373
+
374
+ ### Isolation Layer Utilities
375
+
376
+ | Export | Description |
377
+ |--------|-------------|
378
+ | `createIsolatedStoreModule` | Create store module with isolation layer |
379
+ | `hasIsolation` | Check if a module has isolation capability |
380
+ | `IsolatedStoreModuleConfig` | Configuration for isolated store module |
381
+
382
+ ### Caching
383
+
384
+ | Export | Description |
385
+ |--------|-------------|
386
+ | `CachedKVStore` | Read-through LRU cache wrapper for any `KVStore` |
387
+ | `CacheOptions` | Configuration for cache (maxEntries, maxBytes, enabled) |
388
+
389
+ ### Encoding Utilities
390
+
391
+ | Export | Description |
392
+ |--------|-------------|
393
+ | `encodeValue` | Encode a SQL value to sortable bytes |
394
+ | `decodeValue` | Decode bytes back to SQL value |
395
+ | `encodeCompositeKey` | Encode multiple values as composite key |
396
+ | `decodeCompositeKey` | Decode composite key to values |
397
+ | `registerCollationEncoder` | Register custom collation |
398
+
399
+ ### Serialization Utilities
400
+
401
+ | Export | Description |
402
+ |--------|-------------|
403
+ | `serializeRow` | Serialize a row to bytes |
404
+ | `deserializeRow` | Deserialize bytes to row |
405
+ | `serializeValue` | Serialize a single value |
406
+ | `deserializeValue` | Deserialize a single value |
407
+
408
+ ### Key Building
409
+
410
+ | Export | Description |
411
+ |--------|-------------|
412
+ | `buildDataStoreName` | Build store name for table data |
413
+ | `buildIndexStoreName` | Build store name for an index |
414
+ | `buildStatsStoreName` | Build store name for table stats |
415
+ | `buildDataKey` | Build key for row data (encoded PK) |
416
+ | `buildIndexKey` | Build key for index entry |
417
+ | `buildCatalogKey` | Build key for a table's catalog entry (`{schema}.{table}`) |
418
+ | `buildViewCatalogKey` | Build key for a view's catalog entry (reserved `\x00view\x00` prefix) |
419
+ | `buildMaterializedViewCatalogKey` | Build key for an MV's catalog entry (reserved `\x00mview\x00` prefix) |
420
+ | `classifyCatalogKey` | Classify a loaded catalog key as `'table'` / `'view'` / `'materializedView'` |
421
+ | `buildFullScanBounds` | Build bounds for full table scan |
422
+ | `buildIndexPrefixBounds` | Build bounds for index prefix scan |
423
+ | `buildPkPrefixBounds` | Build bounds for a data-store PK prefix range (per-column DESC + key collations) |
424
+ | `buildCatalogScanBounds` | Build bounds for catalog scan |
425
+ | `CATALOG_STORE_NAME` | Reserved catalog store name constant |
426
+ | `STORE_SUFFIX` | Store name suffixes (INDEX, STATS) |
427
+
428
+ ## Related Packages
429
+
430
+ - [`@quereus/plugin-leveldb`](../quereus-plugin-leveldb/) - LevelDB implementation for Node.js
431
+ - [`@quereus/plugin-indexeddb`](../quereus-plugin-indexeddb/) - IndexedDB implementation for browsers
432
+ - [`@quereus/sync`](../quereus-sync/) - CRDT sync layer
433
+
434
+ ## License
435
+
436
+ MIT