@frostpillar/frostpillar-storage-engine 0.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README-JA.md +1205 -0
- package/README.md +1204 -0
- package/dist/drivers/file.cjs +960 -0
- package/dist/drivers/file.d.ts +3 -0
- package/dist/drivers/file.js +18 -0
- package/dist/drivers/indexedDB.cjs +570 -0
- package/dist/drivers/indexedDB.d.ts +3 -0
- package/dist/drivers/indexedDB.js +18 -0
- package/dist/drivers/localStorage.cjs +668 -0
- package/dist/drivers/localStorage.d.ts +3 -0
- package/dist/drivers/localStorage.js +23 -0
- package/dist/drivers/opfs.cjs +550 -0
- package/dist/drivers/opfs.d.ts +3 -0
- package/dist/drivers/opfs.js +18 -0
- package/dist/drivers/syncStorage.cjs +898 -0
- package/dist/drivers/syncStorage.d.ts +3 -0
- package/dist/drivers/syncStorage.js +22 -0
- package/dist/drivers/validation.d.ts +1 -0
- package/dist/drivers/validation.js +8 -0
- package/dist/errors/index.d.ts +32 -0
- package/dist/errors/index.js +48 -0
- package/dist/frostpillar-storage-engine.min.js +1 -0
- package/dist/index.cjs +2957 -0
- package/dist/index.d.ts +7 -0
- package/dist/index.js +6 -0
- package/dist/storage/backend/asyncDurableAutoCommitController.d.ts +26 -0
- package/dist/storage/backend/asyncDurableAutoCommitController.js +188 -0
- package/dist/storage/backend/asyncMutex.d.ts +7 -0
- package/dist/storage/backend/asyncMutex.js +38 -0
- package/dist/storage/backend/autoCommit.d.ts +2 -0
- package/dist/storage/backend/autoCommit.js +22 -0
- package/dist/storage/backend/capacity.d.ts +2 -0
- package/dist/storage/backend/capacity.js +27 -0
- package/dist/storage/backend/capacityResolver.d.ts +3 -0
- package/dist/storage/backend/capacityResolver.js +25 -0
- package/dist/storage/backend/encoding.d.ts +17 -0
- package/dist/storage/backend/encoding.js +148 -0
- package/dist/storage/backend/types.d.ts +184 -0
- package/dist/storage/backend/types.js +1 -0
- package/dist/storage/btree/recordKeyIndexBTree.d.ts +39 -0
- package/dist/storage/btree/recordKeyIndexBTree.js +104 -0
- package/dist/storage/config/config.browser.d.ts +4 -0
- package/dist/storage/config/config.browser.js +8 -0
- package/dist/storage/config/config.d.ts +1 -0
- package/dist/storage/config/config.js +1 -0
- package/dist/storage/config/config.node.d.ts +4 -0
- package/dist/storage/config/config.node.js +74 -0
- package/dist/storage/config/config.shared.d.ts +6 -0
- package/dist/storage/config/config.shared.js +105 -0
- package/dist/storage/datastore/Datastore.d.ts +47 -0
- package/dist/storage/datastore/Datastore.js +525 -0
- package/dist/storage/datastore/datastoreClose.d.ts +12 -0
- package/dist/storage/datastore/datastoreClose.js +60 -0
- package/dist/storage/datastore/datastoreKeyDefinition.d.ts +7 -0
- package/dist/storage/datastore/datastoreKeyDefinition.js +60 -0
- package/dist/storage/datastore/datastoreLifecycle.d.ts +18 -0
- package/dist/storage/datastore/datastoreLifecycle.js +63 -0
- package/dist/storage/datastore/mutationById.d.ts +29 -0
- package/dist/storage/datastore/mutationById.js +71 -0
- package/dist/storage/drivers/IndexedDB/indexedDBBackend.d.ts +11 -0
- package/dist/storage/drivers/IndexedDB/indexedDBBackend.js +109 -0
- package/dist/storage/drivers/IndexedDB/indexedDBBackendController.d.ts +27 -0
- package/dist/storage/drivers/IndexedDB/indexedDBBackendController.js +60 -0
- package/dist/storage/drivers/IndexedDB/indexedDBConfig.d.ts +7 -0
- package/dist/storage/drivers/IndexedDB/indexedDBConfig.js +24 -0
- package/dist/storage/drivers/file/fileBackend.d.ts +5 -0
- package/dist/storage/drivers/file/fileBackend.js +168 -0
- package/dist/storage/drivers/file/fileBackendController.d.ts +31 -0
- package/dist/storage/drivers/file/fileBackendController.js +72 -0
- package/dist/storage/drivers/file/fileBackendSnapshot.d.ts +10 -0
- package/dist/storage/drivers/file/fileBackendSnapshot.js +166 -0
- package/dist/storage/drivers/localStorage/localStorageBackend.d.ts +10 -0
- package/dist/storage/drivers/localStorage/localStorageBackend.js +156 -0
- package/dist/storage/drivers/localStorage/localStorageBackendController.d.ts +24 -0
- package/dist/storage/drivers/localStorage/localStorageBackendController.js +35 -0
- package/dist/storage/drivers/localStorage/localStorageConfig.d.ts +10 -0
- package/dist/storage/drivers/localStorage/localStorageConfig.js +16 -0
- package/dist/storage/drivers/localStorage/localStorageLayout.d.ts +5 -0
- package/dist/storage/drivers/localStorage/localStorageLayout.js +29 -0
- package/dist/storage/drivers/opfs/opfsBackend.d.ts +12 -0
- package/dist/storage/drivers/opfs/opfsBackend.js +142 -0
- package/dist/storage/drivers/opfs/opfsBackendController.d.ts +26 -0
- package/dist/storage/drivers/opfs/opfsBackendController.js +44 -0
- package/dist/storage/drivers/syncStorage/syncStorageAdapter.d.ts +2 -0
- package/dist/storage/drivers/syncStorage/syncStorageAdapter.js +123 -0
- package/dist/storage/drivers/syncStorage/syncStorageBackend.d.ts +11 -0
- package/dist/storage/drivers/syncStorage/syncStorageBackend.js +169 -0
- package/dist/storage/drivers/syncStorage/syncStorageBackendController.d.ts +24 -0
- package/dist/storage/drivers/syncStorage/syncStorageBackendController.js +34 -0
- package/dist/storage/drivers/syncStorage/syncStorageChunkMaintenance.d.ts +2 -0
- package/dist/storage/drivers/syncStorage/syncStorageChunkMaintenance.js +28 -0
- package/dist/storage/drivers/syncStorage/syncStorageConfig.d.ts +13 -0
- package/dist/storage/drivers/syncStorage/syncStorageConfig.js +42 -0
- package/dist/storage/drivers/syncStorage/syncStorageQuota.d.ts +3 -0
- package/dist/storage/drivers/syncStorage/syncStorageQuota.js +45 -0
- package/dist/storage/record/ordering.d.ts +3 -0
- package/dist/storage/record/ordering.js +7 -0
- package/dist/types.d.ts +125 -0
- package/dist/types.js +1 -0
- package/dist/validation/metadata.d.ts +1 -0
- package/dist/validation/metadata.js +7 -0
- package/dist/validation/payload.d.ts +7 -0
- package/dist/validation/payload.js +135 -0
- package/dist/validation/typeGuards.d.ts +1 -0
- package/dist/validation/typeGuards.js +7 -0
- package/package.json +110 -0
package/README.md
ADDED
|
@@ -0,0 +1,1204 @@
|
|
|
1
|
+
# Frostpillar Storage Engine
|
|
2
|
+
|
|
3
|
+
[English/英語](./README.md) | [Japanese/日本語](./README-JA.md)
|
|
4
|
+
|
|
5
|
+
[](https://www.npmjs.com/package/@frostpillar/frostpillar-storage-engine)
|
|
6
|
+
[](https://nodejs.org/)
|
|
7
|
+
[](https://github.com/hjmsano/frostpillar-storage-engine/actions/workflows/ci.yml)
|
|
8
|
+
[](https://opensource.org/licenses/MIT)
|
|
9
|
+
|
|
10
|
+
A lightweight embedded key-value database for JavaScript. Store and retrieve structured records in Node.js, browsers, or browser extensions — no server required.
|
|
11
|
+
|
|
12
|
+
Under the hood it is a chunk-based storage engine that packs many small entries into a single backing store with pluggable drivers, capacity control, and auto-commit support. It is part of the Frostpillar ecosystem:
|
|
13
|
+
|
|
14
|
+
```
|
|
15
|
+
frostpillar-db — Database management and orchestration, also provide native query interface
|
|
16
|
+
├── frostpillar-query-interface — SQL-like / Lucene-like query API
|
|
17
|
+
├── frostpillar-storage-engine — Core storage and chunk handling (this package)
|
|
18
|
+
│ └── frostpillar-btree — B+ tree indexing
|
|
19
|
+
frostpillar-http-api — RESTful API layer
|
|
20
|
+
frostpillar-mcp — MCP interface for AI agent integration
|
|
21
|
+
frostpillar-cli — Command-line interface
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
## Features
|
|
25
|
+
|
|
26
|
+
- **Multi-runtime** — works in Node.js, browsers, and browser extensions
|
|
27
|
+
- **Pluggable drivers** — in-memory, file, localStorage, IndexedDB, OPFS, and browser extension sync storage
|
|
28
|
+
- **Capacity control** — strict quota enforcement or automatic turnover eviction
|
|
29
|
+
- **Auto-commit** — configurable interval and byte-threshold based background persistence
|
|
30
|
+
- **Custom keys** — bring your own key type with normalize/compare/serialize/deserialize
|
|
31
|
+
- **Tree-shakable** — ESM with `sideEffects: false`; unused drivers are eliminated by bundlers
|
|
32
|
+
- **Zero third-party runtime dependencies** — only Frostpillar family packages
|
|
33
|
+
|
|
34
|
+
## Quick Example
|
|
35
|
+
|
|
36
|
+
**Node.js / TypeScript:**
|
|
37
|
+
|
|
38
|
+
```ts
|
|
39
|
+
import { Datastore } from '@frostpillar/frostpillar-storage-engine';
|
|
40
|
+
|
|
41
|
+
const db = new Datastore({});
|
|
42
|
+
|
|
43
|
+
await db.put({
|
|
44
|
+
key: 'tenant-001',
|
|
45
|
+
payload: { event: 'login', userId: 'u-001' },
|
|
46
|
+
});
|
|
47
|
+
|
|
48
|
+
const rows = await db.get('tenant-001');
|
|
49
|
+
console.log(rows[0].payload.event); // login
|
|
50
|
+
|
|
51
|
+
await db.close();
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
**Browser (ESM):**
|
|
55
|
+
|
|
56
|
+
```js
|
|
57
|
+
import { Datastore } from '@frostpillar/frostpillar-storage-engine';
|
|
58
|
+
|
|
59
|
+
const db = new Datastore({});
|
|
60
|
+
|
|
61
|
+
await db.put({
|
|
62
|
+
key: 'user-001',
|
|
63
|
+
payload: { event: 'open' },
|
|
64
|
+
});
|
|
65
|
+
|
|
66
|
+
const rows = await db.get('user-001');
|
|
67
|
+
console.log(rows[0].payload.event); // open
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
**Browser (Bundle):**
|
|
71
|
+
|
|
72
|
+
```js
|
|
73
|
+
const { Datastore } = window.FrostpillarStorageEngine;
|
|
74
|
+
|
|
75
|
+
const db = new Datastore({});
|
|
76
|
+
|
|
77
|
+
await db.put({
|
|
78
|
+
key: 'user-001',
|
|
79
|
+
payload: { event: 'open' },
|
|
80
|
+
});
|
|
81
|
+
|
|
82
|
+
const rows = await db.get('user-001');
|
|
83
|
+
console.log(rows[0].payload.event); // open
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
> **Note:** The IIFE bundle includes `Datastore`, error classes, and browser storage drivers (`localStorageDriver`, `indexedDBDriver`, `opfsDriver`, `syncStorageDriver`). `fileDriver` is Node.js-only and is not included — import it from the subpath `@frostpillar/frostpillar-storage-engine/drivers/file`.
|
|
87
|
+
|
|
88
|
+
---
|
|
89
|
+
|
|
90
|
+
## Table of Contents
|
|
91
|
+
|
|
92
|
+
- [Getting Started](#getting-started)
|
|
93
|
+
- [User Manual](#user-manual)
|
|
94
|
+
- [Core Concepts](#core-concepts)
|
|
95
|
+
- [CRUD Operations](#crud-operations)
|
|
96
|
+
- [Record ID (`_id`)](#record-id-_id)
|
|
97
|
+
- [Storage Drivers](#storage-drivers)
|
|
98
|
+
- [Auto-Commit](#auto-commit)
|
|
99
|
+
- [Capacity Control](#capacity-control)
|
|
100
|
+
- [Custom Key Definition](#custom-key-definition)
|
|
101
|
+
- [Error Handling](#error-handling)
|
|
102
|
+
- [API Reference](#api-reference)
|
|
103
|
+
- [How to Contribute](#how-to-contribute)
|
|
104
|
+
- [License](#license)
|
|
105
|
+
|
|
106
|
+
---
|
|
107
|
+
|
|
108
|
+
## Getting Started
|
|
109
|
+
|
|
110
|
+
### Installation (Node.js / TypeScript)
|
|
111
|
+
|
|
112
|
+
```bash
|
|
113
|
+
pnpm add @frostpillar/frostpillar-storage-engine
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
This package is published to [npm](https://www.npmjs.com/package/@frostpillar/frostpillar-storage-engine).
|
|
117
|
+
|
|
118
|
+
### Installation (Browser)
|
|
119
|
+
|
|
120
|
+
Download the minified IIFE bundle from [GitHub Releases](https://github.com/hjmsano/frostpillar-storage-engine/releases) and load it with a `<script>` tag. Replace `<TAG>` with a released tag (e.g. `v0.2.1`).
|
|
121
|
+
|
|
122
|
+
```html
|
|
123
|
+
<script src="https://github.com/hjmsano/frostpillar-storage-engine/releases/download/<TAG>/frostpillar-storage-engine.min.js"></script>
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
`Datastore`, error classes, and browser storage drivers (`localStorageDriver`, `indexedDBDriver`, `opfsDriver`, `syncStorageDriver`) are available on `window.FrostpillarStorageEngine`. No `type="module"` is required.
|
|
127
|
+
|
|
128
|
+
### Compatibility
|
|
129
|
+
|
|
130
|
+
| Environment | Requirement |
|
|
131
|
+
| ----------- | ----------------------------------------------------------------- |
|
|
132
|
+
| Node.js | >= 24.0.0 (ESM and CJS) |
|
|
133
|
+
| Browser | ES2020-compatible (Chrome 80+, Firefox 74+, Safari 14+, Edge 80+) |
|
|
134
|
+
| TypeScript | >= 5.0 |
|
|
135
|
+
|
|
136
|
+
> **Pre-1.0 notice:** This package follows [SemVer](https://semver.org/). While the major version is `0`, minor version bumps may include breaking changes. Pin your dependency version and review changelogs before upgrading.
|
|
137
|
+
|
|
138
|
+
---
|
|
139
|
+
|
|
140
|
+
## User Manual
|
|
141
|
+
|
|
142
|
+
### Core Concepts
|
|
143
|
+
|
|
144
|
+
**Datastore** is the single entry point. The basic lifecycle is:
|
|
145
|
+
|
|
146
|
+
1. **Create** — `new Datastore(config)` (in-memory by default, or pass a `driver`)
|
|
147
|
+
2. **Write** — `put()` / `putMany()` to insert records
|
|
148
|
+
3. **Read** — `get()`, `getFirst()`, `getLast()`, `getAll()`, etc.
|
|
149
|
+
4. **Persist** — `commit()` flushes to durable storage (or use `autoCommit`)
|
|
150
|
+
5. **Close** — `close()` releases resources and locks
|
|
151
|
+
|
|
152
|
+
Each record has:
|
|
153
|
+
|
|
154
|
+
| Field | Description |
|
|
155
|
+
| --------- | ----------------------------------------------- |
|
|
156
|
+
| `key` | User-provided lookup key (string by default) |
|
|
157
|
+
| `payload` | JSON-compatible data object |
|
|
158
|
+
| `_id` | Ephemeral system-generated `EntryId`, read-only |
|
|
159
|
+
|
|
160
|
+
> **Defensive cloning:** Payloads are defensively cloned at insertion time but are **not** frozen. Read APIs return internal references without cloning. Mutating a returned payload will **not** throw, but may corrupt internal state — treat returned payloads as read-only. If you need a mutable copy, clone it yourself (e.g. `structuredClone(record.payload)`).
|
|
161
|
+
|
|
162
|
+
Records are ordered by `key` ascending (lexicographic by default), then by insertion order ascending for ties.
|
|
163
|
+
|
|
164
|
+
#### Duplicate Key Policy
|
|
165
|
+
|
|
166
|
+
By default, duplicate keys are allowed (multiple records can share the same key). Configure the policy at construction:
|
|
167
|
+
|
|
168
|
+
**Node.js / TypeScript:**
|
|
169
|
+
|
|
170
|
+
```ts
|
|
171
|
+
const db = new Datastore({
|
|
172
|
+
duplicateKeys: 'allow', // default — multiple records per key
|
|
173
|
+
// duplicateKeys: 'replace', // one record per key, last-write-wins
|
|
174
|
+
// duplicateKeys: 'reject', // one record per key, throws on duplicate
|
|
175
|
+
});
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
**Browser (ESM):**
|
|
179
|
+
|
|
180
|
+
```js
|
|
181
|
+
import { Datastore } from '@frostpillar/frostpillar-storage-engine';
|
|
182
|
+
|
|
183
|
+
const db = new Datastore({
|
|
184
|
+
duplicateKeys: 'allow', // default — multiple records per key
|
|
185
|
+
// duplicateKeys: 'replace', // one record per key, last-write-wins
|
|
186
|
+
// duplicateKeys: 'reject', // one record per key, throws on duplicate
|
|
187
|
+
});
|
|
188
|
+
```
|
|
189
|
+
|
|
190
|
+
**Browser (Bundle):**
|
|
191
|
+
|
|
192
|
+
```js
|
|
193
|
+
const { Datastore } = window.FrostpillarStorageEngine;
|
|
194
|
+
|
|
195
|
+
const db = new Datastore({
|
|
196
|
+
duplicateKeys: 'allow', // default — multiple records per key
|
|
197
|
+
// duplicateKeys: 'replace', // one record per key, last-write-wins
|
|
198
|
+
// duplicateKeys: 'reject', // one record per key, throws on duplicate
|
|
199
|
+
});
|
|
200
|
+
```
|
|
201
|
+
|
|
202
|
+
| Policy | Behavior | Use case |
|
|
203
|
+
| ----------- | ------------------------------------- | ------------------------- |
|
|
204
|
+
| `'allow'` | Multiple records per key | Logs, events, time-series |
|
|
205
|
+
| `'replace'` | Last-write-wins overwrite | Config, settings, cache |
|
|
206
|
+
| `'reject'` | Throws `ValidationError` on duplicate | Unique constraints |
|
|
207
|
+
|
|
208
|
+
#### Payload Validation
|
|
209
|
+
|
|
210
|
+
Payloads are validated on every `put()`, `putMany()`, and `updateById()` call. The following limits apply:
|
|
211
|
+
|
|
212
|
+
| Constraint | Limit |
|
|
213
|
+
|------------|-------|
|
|
214
|
+
| Total payload bytes | 1,048,576 (1 MB) |
|
|
215
|
+
| Max nesting depth | 64 object levels |
|
|
216
|
+
| Max total keys | 4,096 |
|
|
217
|
+
| Max keys per object | 256 |
|
|
218
|
+
| Max key size | 1,024 bytes (UTF-8) |
|
|
219
|
+
| Max string value | 65,535 bytes (UTF-8) |
|
|
220
|
+
|
|
221
|
+
Additional rules:
|
|
222
|
+
- Payload must be a plain object (no arrays, functions, or `BigInt` at top level).
|
|
223
|
+
- Keys must be non-empty, non-whitespace strings.
|
|
224
|
+
- Reserved keys (`__proto__`, `constructor`, `prototype`) are forbidden.
|
|
225
|
+
- Circular references are forbidden.
|
|
226
|
+
- Violations throw `ValidationError`.
|
|
227
|
+
|
|
228
|
+
For trusted input where you control the shape, you can skip validation:
|
|
229
|
+
|
|
230
|
+
```ts
|
|
231
|
+
const db = new Datastore({ skipPayloadValidation: true });
|
|
232
|
+
```
|
|
233
|
+
|
|
234
|
+
> **Warning:** Skipping validation disables all payload safety checks. Only use this when you are certain the input is well-formed.
|
|
235
|
+
|
|
236
|
+
---
|
|
237
|
+
|
|
238
|
+
### CRUD Operations
|
|
239
|
+
|
|
240
|
+
#### Write
|
|
241
|
+
|
|
242
|
+
**`put(record)`** — insert a single record.
|
|
243
|
+
|
|
244
|
+
```ts
|
|
245
|
+
await db.put({ key: 'k1', payload: { name: 'Alice' } });
|
|
246
|
+
```
|
|
247
|
+
|
|
248
|
+
**`putMany(records)`** — insert multiple records (non-atomic, left-to-right).
|
|
249
|
+
|
|
250
|
+
```ts
|
|
251
|
+
await db.putMany([
|
|
252
|
+
{ key: 'k1', payload: { name: 'Alice' } },
|
|
253
|
+
{ key: 'k2', payload: { name: 'Bob' } },
|
|
254
|
+
]);
|
|
255
|
+
```
|
|
256
|
+
|
|
257
|
+
`put()` inserts a record. Duplicate key behavior depends on the `duplicateKeys` policy (default: `'allow'`).
|
|
258
|
+
|
|
259
|
+
#### Read
|
|
260
|
+
|
|
261
|
+
**`get(key)`** — all records matching `key`.
|
|
262
|
+
|
|
263
|
+
```ts
|
|
264
|
+
const rows = await db.get('k1');
|
|
265
|
+
```
|
|
266
|
+
|
|
267
|
+
**`getFirst(key)`** — first record matching `key`, or `null`.
|
|
268
|
+
|
|
269
|
+
```ts
|
|
270
|
+
const row = await db.getFirst('k1');
|
|
271
|
+
```
|
|
272
|
+
|
|
273
|
+
**`getLast(key)`** — last record matching `key`, or `null`. When `duplicateKeys` is `'replace'` or `'reject'`, behaves identically to `getFirst()`.
|
|
274
|
+
|
|
275
|
+
```ts
|
|
276
|
+
const row = await db.getLast('k1');
|
|
277
|
+
```
|
|
278
|
+
|
|
279
|
+
**`getById(id)`** — single record by `_id`, or `null`.
|
|
280
|
+
|
|
281
|
+
```ts
|
|
282
|
+
const row = await db.getById(id);
|
|
283
|
+
```
|
|
284
|
+
|
|
285
|
+
**`getAll()`** — all records, ordered by key then insertion order.
|
|
286
|
+
|
|
287
|
+
```ts
|
|
288
|
+
const all = await db.getAll();
|
|
289
|
+
```
|
|
290
|
+
|
|
291
|
+
**`getRange(start, end)`** — records where `start <= key <= end` (inclusive).
|
|
292
|
+
|
|
293
|
+
```ts
|
|
294
|
+
const range = await db.getRange('a', 'f');
|
|
295
|
+
```
|
|
296
|
+
|
|
297
|
+
**`getMany(keys)`** — records for a set of discrete keys.
|
|
298
|
+
|
|
299
|
+
```ts
|
|
300
|
+
const rows = await db.getMany(['k1', 'k3', 'k5']);
|
|
301
|
+
```
|
|
302
|
+
|
|
303
|
+
**`has(key)`** — check if any record exists with the given key.
|
|
304
|
+
|
|
305
|
+
```ts
|
|
306
|
+
const exists = await db.has('k1');
|
|
307
|
+
```
|
|
308
|
+
|
|
309
|
+
All record-returning APIs include the `_id` field in the result.
|
|
310
|
+
|
|
311
|
+
#### Update
|
|
312
|
+
|
|
313
|
+
**`updateById(id, patch)`** — shallow-merge `patch` into the existing payload. Returns `true` if found, `false` otherwise. Does not change `key` or `_id`.
|
|
314
|
+
|
|
315
|
+
```ts
|
|
316
|
+
const updated = await db.updateById(id, { name: 'Alice V2' });
|
|
317
|
+
```
|
|
318
|
+
|
|
319
|
+
#### Delete
|
|
320
|
+
|
|
321
|
+
**`delete(key)`** — remove all records with `key`. Returns the number of records removed.
|
|
322
|
+
|
|
323
|
+
```ts
|
|
324
|
+
const count = await db.delete('k1');
|
|
325
|
+
```
|
|
326
|
+
|
|
327
|
+
**`deleteById(id)`** — remove a single record by `_id`. Returns `true` if found.
|
|
328
|
+
|
|
329
|
+
```ts
|
|
330
|
+
const removed = await db.deleteById(id);
|
|
331
|
+
```
|
|
332
|
+
|
|
333
|
+
**`deleteMany(keys)`** — remove records across multiple keys (non-atomic). Returns total removed.
|
|
334
|
+
|
|
335
|
+
```ts
|
|
336
|
+
const count = await db.deleteMany(['k1', 'k2']);
|
|
337
|
+
```
|
|
338
|
+
|
|
339
|
+
**`clear()`** — remove all records.
|
|
340
|
+
|
|
341
|
+
```ts
|
|
342
|
+
await db.clear();
|
|
343
|
+
```
|
|
344
|
+
|
|
345
|
+
#### Metadata
|
|
346
|
+
|
|
347
|
+
**`count()`** — total number of records.
|
|
348
|
+
|
|
349
|
+
```ts
|
|
350
|
+
const n = await db.count();
|
|
351
|
+
```
|
|
352
|
+
|
|
353
|
+
**`keys()`** — distinct keys in ascending order (no duplicates, no payload loaded).
|
|
354
|
+
|
|
355
|
+
```ts
|
|
356
|
+
const allKeys = await db.keys();
|
|
357
|
+
```
|
|
358
|
+
|
|
359
|
+
---
|
|
360
|
+
|
|
361
|
+
### Record ID (`_id`)
|
|
362
|
+
|
|
363
|
+
`_id` is a system-generated `EntryId` (a branded number) included in every record returned by read APIs. It is ephemeral — re-issued when the datastore is restored from persistent storage.
|
|
364
|
+
|
|
365
|
+
- You do not get an `_id` back from `put()` — discover it by reading records.
|
|
366
|
+
- After restart or `fromJSON()` restoration, previously obtained `_id` values become invalid. Re-query to obtain new ones.
|
|
367
|
+
- `EntryId` is re-exported from the package for type annotations:
|
|
368
|
+
|
|
369
|
+
**Node.js / TypeScript:**
|
|
370
|
+
|
|
371
|
+
```ts
|
|
372
|
+
import type { EntryId } from '@frostpillar/frostpillar-storage-engine';
|
|
373
|
+
```
|
|
374
|
+
|
|
375
|
+
**Browser (ESM / Bundle):**
|
|
376
|
+
|
|
377
|
+
```js
|
|
378
|
+
// EntryId is a plain number at runtime — no import needed.
|
|
379
|
+
// Use it directly from record results:
|
|
380
|
+
const record = await db.getFirst('k1');
|
|
381
|
+
const id = record._id; // EntryId
|
|
382
|
+
```
|
|
383
|
+
|
|
384
|
+
---
|
|
385
|
+
|
|
386
|
+
### Storage Drivers
|
|
387
|
+
|
|
388
|
+
#### Driver Comparison
|
|
389
|
+
|
|
390
|
+
| Driver | Environment | Persistence | Typical Use Case |
|
|
391
|
+
| -------------------- | ----------------- | ---------------------------------------------- | ------------------------------- |
|
|
392
|
+
| _(none)_ | Node.js / Browser | In-memory only | Caches, tests, ephemeral data |
|
|
393
|
+
| `fileDriver` | Node.js | File system | Server-side durable storage |
|
|
394
|
+
| `localStorageDriver` | Browser | localStorage | Small browser-side persistence |
|
|
395
|
+
| `indexedDBDriver` | Browser | IndexedDB | Larger browser-side storage |
|
|
396
|
+
| `opfsDriver` | Browser | Origin Private File System | High-throughput browser storage |
|
|
397
|
+
| `syncStorageDriver` | Browser Extension | `browser.storage.sync` / `chrome.storage.sync` | Cross-device extension data |
|
|
398
|
+
|
|
399
|
+
#### In-Memory (default)
|
|
400
|
+
|
|
401
|
+
No driver needed. Data lives only in memory.
|
|
402
|
+
|
|
403
|
+
**Node.js / TypeScript:**
|
|
404
|
+
|
|
405
|
+
```ts
|
|
406
|
+
const db = new Datastore({});
|
|
407
|
+
```
|
|
408
|
+
|
|
409
|
+
**Browser (ESM):**
|
|
410
|
+
|
|
411
|
+
```js
|
|
412
|
+
import { Datastore } from '@frostpillar/frostpillar-storage-engine';
|
|
413
|
+
|
|
414
|
+
const db = new Datastore({});
|
|
415
|
+
```
|
|
416
|
+
|
|
417
|
+
**Browser (Bundle):**
|
|
418
|
+
|
|
419
|
+
```js
|
|
420
|
+
const { Datastore } = window.FrostpillarStorageEngine;
|
|
421
|
+
|
|
422
|
+
const db = new Datastore({});
|
|
423
|
+
```
|
|
424
|
+
|
|
425
|
+
#### File Driver (Node.js)
|
|
426
|
+
|
|
427
|
+
**Node.js / TypeScript:**
|
|
428
|
+
|
|
429
|
+
```ts
|
|
430
|
+
import { Datastore } from '@frostpillar/frostpillar-storage-engine';
|
|
431
|
+
import { fileDriver } from '@frostpillar/frostpillar-storage-engine/drivers/file';
|
|
432
|
+
|
|
433
|
+
const db = new Datastore({
|
|
434
|
+
autoCommit: {
|
|
435
|
+
frequency: '5s',
|
|
436
|
+
maxPendingBytes: 1024 * 1024,
|
|
437
|
+
},
|
|
438
|
+
driver: fileDriver({
|
|
439
|
+
filePath: './data/events.fpdb',
|
|
440
|
+
}),
|
|
441
|
+
});
|
|
442
|
+
|
|
443
|
+
await db.put({
|
|
444
|
+
key: 'tenant-001',
|
|
445
|
+
payload: { event: 'purchase', amount: 1200 },
|
|
446
|
+
});
|
|
447
|
+
|
|
448
|
+
await db.commit();
|
|
449
|
+
await db.close();
|
|
450
|
+
```
|
|
451
|
+
|
|
452
|
+
| Option | Type | Description |
|
|
453
|
+
|--------|------|-------------|
|
|
454
|
+
| `filePath` | `string` | Direct path to the data file (e.g. `'./data/events.fpdb'`) |
|
|
455
|
+
|
|
456
|
+
Alternatively, use directory-based targeting via the `target` option:
|
|
457
|
+
|
|
458
|
+
| Option | Type | Description |
|
|
459
|
+
|--------|------|-------------|
|
|
460
|
+
| `target.kind` | `'directory'` | Use directory-based file resolution |
|
|
461
|
+
| `target.directory` | `string` | Directory containing the data file |
|
|
462
|
+
| `target.fileName` | `string` | Optional file name (default: auto-generated) |
|
|
463
|
+
| `target.filePrefix` | `string` | Optional file name prefix |
|
|
464
|
+
|
|
465
|
+
**Lock file behavior:**
|
|
466
|
+
|
|
467
|
+
`fileDriver` uses `${filePath}.lock` to enforce a single writer. If a process exits without calling `close()`, subsequent opens fail with `DatabaseLockedError`.
|
|
468
|
+
|
|
469
|
+
Recovery steps:
|
|
470
|
+
|
|
471
|
+
1. Verify no active writer process is using the same datastore file.
|
|
472
|
+
2. Remove the stale lock file manually (`<resolved-data-file>.lock`).
|
|
473
|
+
3. Reopen the datastore.
|
|
474
|
+
|
|
475
|
+
> **Note:** `fileDriver` is Node.js-only and is **not** included in the browser IIFE bundle. Import it from the subpath `@frostpillar/frostpillar-storage-engine/drivers/file`.
|
|
476
|
+
|
|
477
|
+
#### localStorage Driver
|
|
478
|
+
|
|
479
|
+
**Node.js / TypeScript:**
|
|
480
|
+
|
|
481
|
+
```ts
|
|
482
|
+
import { Datastore } from '@frostpillar/frostpillar-storage-engine';
|
|
483
|
+
import { localStorageDriver } from '@frostpillar/frostpillar-storage-engine/drivers/localStorage';
|
|
484
|
+
|
|
485
|
+
const db = new Datastore({
|
|
486
|
+
driver: localStorageDriver({
|
|
487
|
+
databaseKey: 'app-events',
|
|
488
|
+
keyPrefix: 'frostpillar',
|
|
489
|
+
maxChunkChars: 32768,
|
|
490
|
+
maxChunks: 64,
|
|
491
|
+
}),
|
|
492
|
+
});
|
|
493
|
+
```
|
|
494
|
+
|
|
495
|
+
**Browser (ESM):**
|
|
496
|
+
|
|
497
|
+
```js
|
|
498
|
+
import { Datastore } from '@frostpillar/frostpillar-storage-engine';
|
|
499
|
+
import { localStorageDriver } from '@frostpillar/frostpillar-storage-engine/drivers/localStorage';
|
|
500
|
+
|
|
501
|
+
const db = new Datastore({
|
|
502
|
+
driver: localStorageDriver({
|
|
503
|
+
databaseKey: 'app-events',
|
|
504
|
+
keyPrefix: 'frostpillar',
|
|
505
|
+
maxChunkChars: 32768,
|
|
506
|
+
maxChunks: 64,
|
|
507
|
+
}),
|
|
508
|
+
});
|
|
509
|
+
```
|
|
510
|
+
|
|
511
|
+
**Browser (Bundle):**
|
|
512
|
+
|
|
513
|
+
```js
|
|
514
|
+
const { Datastore, localStorageDriver } = window.FrostpillarStorageEngine;
|
|
515
|
+
|
|
516
|
+
const db = new Datastore({
|
|
517
|
+
driver: localStorageDriver({
|
|
518
|
+
databaseKey: 'app-events',
|
|
519
|
+
keyPrefix: 'frostpillar',
|
|
520
|
+
maxChunkChars: 32768,
|
|
521
|
+
maxChunks: 64,
|
|
522
|
+
}),
|
|
523
|
+
});
|
|
524
|
+
```
|
|
525
|
+
|
|
526
|
+
| Option | Type | Description |
|
|
527
|
+
| --------------- | -------- | ------------------------------------------------------ |
|
|
528
|
+
| `databaseKey` | `string` | Logical database name within localStorage |
|
|
529
|
+
| `keyPrefix` | `string` | Prefix for all localStorage keys (namespace isolation) |
|
|
530
|
+
| `maxChunkChars` | `number` | Maximum characters per chunk |
|
|
531
|
+
| `maxChunks` | `number` | Maximum number of chunks |
|
|
532
|
+
|
|
533
|
+
#### IndexedDB Driver
|
|
534
|
+
|
|
535
|
+
**Node.js / TypeScript:**
|
|
536
|
+
|
|
537
|
+
```ts
|
|
538
|
+
import { Datastore } from '@frostpillar/frostpillar-storage-engine';
|
|
539
|
+
import { indexedDBDriver } from '@frostpillar/frostpillar-storage-engine/drivers/indexedDB';
|
|
540
|
+
|
|
541
|
+
const db = new Datastore({
|
|
542
|
+
autoCommit: { frequency: 'immediate' },
|
|
543
|
+
driver: indexedDBDriver({
|
|
544
|
+
databaseName: 'frostpillar-demo',
|
|
545
|
+
objectStoreName: 'records',
|
|
546
|
+
version: 1,
|
|
547
|
+
}),
|
|
548
|
+
});
|
|
549
|
+
```
|
|
550
|
+
|
|
551
|
+
**Browser (ESM):**
|
|
552
|
+
|
|
553
|
+
```js
|
|
554
|
+
import { Datastore } from '@frostpillar/frostpillar-storage-engine';
|
|
555
|
+
import { indexedDBDriver } from '@frostpillar/frostpillar-storage-engine/drivers/indexedDB';
|
|
556
|
+
|
|
557
|
+
const db = new Datastore({
|
|
558
|
+
autoCommit: { frequency: 'immediate' },
|
|
559
|
+
driver: indexedDBDriver({
|
|
560
|
+
databaseName: 'frostpillar-demo',
|
|
561
|
+
objectStoreName: 'records',
|
|
562
|
+
version: 1,
|
|
563
|
+
}),
|
|
564
|
+
});
|
|
565
|
+
```
|
|
566
|
+
|
|
567
|
+
**Browser (Bundle):**
|
|
568
|
+
|
|
569
|
+
```js
|
|
570
|
+
const { Datastore, indexedDBDriver } = window.FrostpillarStorageEngine;
|
|
571
|
+
|
|
572
|
+
const db = new Datastore({
|
|
573
|
+
autoCommit: { frequency: 'immediate' },
|
|
574
|
+
driver: indexedDBDriver({
|
|
575
|
+
databaseName: 'frostpillar-demo',
|
|
576
|
+
objectStoreName: 'records',
|
|
577
|
+
version: 1,
|
|
578
|
+
}),
|
|
579
|
+
});
|
|
580
|
+
```
|
|
581
|
+
|
|
582
|
+
| Option | Type | Description |
|
|
583
|
+
| ----------------- | -------- | ------------------------------------- |
|
|
584
|
+
| `databaseName` | `string` | IndexedDB database name |
|
|
585
|
+
| `objectStoreName` | `string` | Object store name within the database |
|
|
586
|
+
| `version` | `number` | Database schema version |
|
|
587
|
+
|
|
588
|
+
#### OPFS Driver
|
|
589
|
+
|
|
590
|
+
**Node.js / TypeScript:**
|
|
591
|
+
|
|
592
|
+
```ts
|
|
593
|
+
import { Datastore } from '@frostpillar/frostpillar-storage-engine';
|
|
594
|
+
import { opfsDriver } from '@frostpillar/frostpillar-storage-engine/drivers/opfs';
|
|
595
|
+
|
|
596
|
+
const db = new Datastore({
|
|
597
|
+
autoCommit: { frequency: 'immediate' },
|
|
598
|
+
driver: opfsDriver({
|
|
599
|
+
directoryName: 'frostpillar-opfs',
|
|
600
|
+
}),
|
|
601
|
+
});
|
|
602
|
+
```
|
|
603
|
+
|
|
604
|
+
**Browser (ESM):**
|
|
605
|
+
|
|
606
|
+
```js
|
|
607
|
+
import { Datastore } from '@frostpillar/frostpillar-storage-engine';
|
|
608
|
+
import { opfsDriver } from '@frostpillar/frostpillar-storage-engine/drivers/opfs';
|
|
609
|
+
|
|
610
|
+
const db = new Datastore({
|
|
611
|
+
autoCommit: { frequency: 'immediate' },
|
|
612
|
+
driver: opfsDriver({
|
|
613
|
+
directoryName: 'frostpillar-opfs',
|
|
614
|
+
}),
|
|
615
|
+
});
|
|
616
|
+
```
|
|
617
|
+
|
|
618
|
+
**Browser (Bundle):**
|
|
619
|
+
|
|
620
|
+
```js
|
|
621
|
+
const { Datastore, opfsDriver } = window.FrostpillarStorageEngine;
|
|
622
|
+
|
|
623
|
+
const db = new Datastore({
|
|
624
|
+
autoCommit: { frequency: 'immediate' },
|
|
625
|
+
driver: opfsDriver({
|
|
626
|
+
directoryName: 'frostpillar-opfs',
|
|
627
|
+
}),
|
|
628
|
+
});
|
|
629
|
+
```
|
|
630
|
+
|
|
631
|
+
| Option | Type | Description |
|
|
632
|
+
| --------------- | -------- | ------------------- |
|
|
633
|
+
| `directoryName` | `string` | OPFS directory name |
|
|
634
|
+
|
|
635
|
+
#### Sync Storage Driver (Browser Extensions)
|
|
636
|
+
|
|
637
|
+
**Node.js / TypeScript:**
|
|
638
|
+
|
|
639
|
+
```ts
|
|
640
|
+
import { Datastore } from '@frostpillar/frostpillar-storage-engine';
|
|
641
|
+
import { syncStorageDriver } from '@frostpillar/frostpillar-storage-engine/drivers/syncStorage';
|
|
642
|
+
|
|
643
|
+
const db = new Datastore({
|
|
644
|
+
capacity: {
|
|
645
|
+
maxSize: 'backendLimit',
|
|
646
|
+
policy: 'strict',
|
|
647
|
+
},
|
|
648
|
+
autoCommit: {
|
|
649
|
+
frequency: '10s',
|
|
650
|
+
maxPendingBytes: 32768,
|
|
651
|
+
},
|
|
652
|
+
driver: syncStorageDriver({
|
|
653
|
+
databaseKey: 'extension-events',
|
|
654
|
+
keyPrefix: 'frostpillar-ext',
|
|
655
|
+
maxChunkChars: 6000,
|
|
656
|
+
maxChunks: 128,
|
|
657
|
+
maxItemBytes: 8192,
|
|
658
|
+
maxTotalBytes: 102400,
|
|
659
|
+
maxItems: 256,
|
|
660
|
+
}),
|
|
661
|
+
});
|
|
662
|
+
```
|
|
663
|
+
|
|
664
|
+
**Browser (ESM):**
|
|
665
|
+
|
|
666
|
+
```js
|
|
667
|
+
import { Datastore } from '@frostpillar/frostpillar-storage-engine';
|
|
668
|
+
import { syncStorageDriver } from '@frostpillar/frostpillar-storage-engine/drivers/syncStorage';
|
|
669
|
+
|
|
670
|
+
const db = new Datastore({
|
|
671
|
+
capacity: {
|
|
672
|
+
maxSize: 'backendLimit',
|
|
673
|
+
policy: 'strict',
|
|
674
|
+
},
|
|
675
|
+
autoCommit: {
|
|
676
|
+
frequency: '10s',
|
|
677
|
+
maxPendingBytes: 32768,
|
|
678
|
+
},
|
|
679
|
+
driver: syncStorageDriver({
|
|
680
|
+
databaseKey: 'extension-events',
|
|
681
|
+
keyPrefix: 'frostpillar-ext',
|
|
682
|
+
maxChunkChars: 6000,
|
|
683
|
+
maxChunks: 128,
|
|
684
|
+
maxItemBytes: 8192,
|
|
685
|
+
maxTotalBytes: 102400,
|
|
686
|
+
maxItems: 256,
|
|
687
|
+
}),
|
|
688
|
+
});
|
|
689
|
+
```
|
|
690
|
+
|
|
691
|
+
**Browser (Bundle):**
|
|
692
|
+
|
|
693
|
+
```js
|
|
694
|
+
const { Datastore, syncStorageDriver } = window.FrostpillarStorageEngine;
|
|
695
|
+
|
|
696
|
+
const db = new Datastore({
|
|
697
|
+
capacity: {
|
|
698
|
+
maxSize: 'backendLimit',
|
|
699
|
+
policy: 'strict',
|
|
700
|
+
},
|
|
701
|
+
autoCommit: {
|
|
702
|
+
frequency: '10s',
|
|
703
|
+
maxPendingBytes: 32768,
|
|
704
|
+
},
|
|
705
|
+
driver: syncStorageDriver({
|
|
706
|
+
databaseKey: 'extension-events',
|
|
707
|
+
keyPrefix: 'frostpillar-ext',
|
|
708
|
+
maxChunkChars: 6000,
|
|
709
|
+
maxChunks: 128,
|
|
710
|
+
maxItemBytes: 8192,
|
|
711
|
+
maxTotalBytes: 102400,
|
|
712
|
+
maxItems: 256,
|
|
713
|
+
}),
|
|
714
|
+
});
|
|
715
|
+
```
|
|
716
|
+
|
|
717
|
+
| Option | Type | Description |
|
|
718
|
+
| --------------- | -------- | --------------------------------------------- |
|
|
719
|
+
| `databaseKey` | `string` | Logical database name |
|
|
720
|
+
| `keyPrefix` | `string` | Prefix for storage keys (namespace isolation) |
|
|
721
|
+
| `maxChunkChars` | `number` | Maximum characters per chunk |
|
|
722
|
+
| `maxChunks` | `number` | Maximum number of chunks |
|
|
723
|
+
| `maxItemBytes` | `number` | Maximum bytes per storage item |
|
|
724
|
+
| `maxTotalBytes` | `number` | Maximum total bytes across all items |
|
|
725
|
+
| `maxItems` | `number` | Maximum number of storage items |
|
|
726
|
+
|
|
727
|
+
When both APIs are available, the driver prefers the `browser.storage.sync` Promise API and falls back to `chrome.storage.sync` callback API.
|
|
728
|
+
|
|
729
|
+
---
|
|
730
|
+
|
|
731
|
+
### Auto-Commit
|
|
732
|
+
|
|
733
|
+
With durable drivers, you can configure automatic background persistence instead of calling `commit()` manually.
|
|
734
|
+
|
|
735
|
+
**Node.js / TypeScript:**
|
|
736
|
+
|
|
737
|
+
```ts
|
|
738
|
+
import { Datastore } from '@frostpillar/frostpillar-storage-engine';
|
|
739
|
+
import { fileDriver } from '@frostpillar/frostpillar-storage-engine/drivers/file';
|
|
740
|
+
|
|
741
|
+
const db = new Datastore({
|
|
742
|
+
autoCommit: {
|
|
743
|
+
frequency: '5s', // commit every 5 seconds
|
|
744
|
+
maxPendingBytes: 1024 * 1024, // or when 1 MB of writes are pending
|
|
745
|
+
},
|
|
746
|
+
driver: fileDriver({ filePath: './data/events.fpdb' }),
|
|
747
|
+
});
|
|
748
|
+
```
|
|
749
|
+
|
|
750
|
+
**Browser (ESM):**
|
|
751
|
+
|
|
752
|
+
```js
|
|
753
|
+
import { Datastore } from '@frostpillar/frostpillar-storage-engine';
|
|
754
|
+
import { indexedDBDriver } from '@frostpillar/frostpillar-storage-engine/drivers/indexedDB';
|
|
755
|
+
|
|
756
|
+
const db = new Datastore({
|
|
757
|
+
autoCommit: {
|
|
758
|
+
frequency: '5s', // commit every 5 seconds
|
|
759
|
+
maxPendingBytes: 1024 * 1024, // or when 1 MB of writes are pending
|
|
760
|
+
},
|
|
761
|
+
driver: indexedDBDriver({
|
|
762
|
+
databaseName: 'my-app',
|
|
763
|
+
objectStoreName: 'records',
|
|
764
|
+
version: 1,
|
|
765
|
+
}),
|
|
766
|
+
});
|
|
767
|
+
```
|
|
768
|
+
|
|
769
|
+
**Browser (Bundle):**
|
|
770
|
+
|
|
771
|
+
```js
|
|
772
|
+
const { Datastore, indexedDBDriver } = window.FrostpillarStorageEngine;
|
|
773
|
+
|
|
774
|
+
const db = new Datastore({
|
|
775
|
+
autoCommit: {
|
|
776
|
+
frequency: '5s', // commit every 5 seconds
|
|
777
|
+
maxPendingBytes: 1024 * 1024, // or when 1 MB of writes are pending
|
|
778
|
+
},
|
|
779
|
+
driver: indexedDBDriver({
|
|
780
|
+
databaseName: 'my-app',
|
|
781
|
+
objectStoreName: 'records',
|
|
782
|
+
version: 1,
|
|
783
|
+
}),
|
|
784
|
+
});
|
|
785
|
+
```
|
|
786
|
+
|
|
787
|
+
| Option | Type | Description |
|
|
788
|
+
| ----------------- | ------------------------------------------------------------------------------ | ------------------------------------------------ |
|
|
789
|
+
| `frequency` | `'immediate'` \| `number` \| `'${n}ms'` \| `'${n}s'` \| `'${n}m'` \| `'${n}h'` | How often to auto-commit |
|
|
790
|
+
| `maxPendingBytes` | `number` | Byte threshold that triggers an immediate commit |
|
|
791
|
+
|
|
792
|
+
`autoCommit` requires a durable `driver`. Configuring `autoCommit` without a `driver` fails with `ConfigurationError`.
|
|
793
|
+
|
|
794
|
+
You can always call `commit()` manually for an explicit flush, even when `autoCommit` is configured.
|
|
795
|
+
|
|
796
|
+
#### Monitoring auto-commit errors
|
|
797
|
+
|
|
798
|
+
Auto-commit failures are delivered asynchronously and do not reject the triggering `put()` call. Use `on('error')` to monitor them:
|
|
799
|
+
|
|
800
|
+
**Node.js / TypeScript:**
|
|
801
|
+
|
|
802
|
+
```ts
|
|
803
|
+
const unsubscribe = db.on('error', (event) => {
|
|
804
|
+
console.error('autoCommit error:', event.error);
|
|
805
|
+
});
|
|
806
|
+
|
|
807
|
+
// Stop listening:
|
|
808
|
+
unsubscribe();
|
|
809
|
+
|
|
810
|
+
// Or explicitly:
|
|
811
|
+
// db.off('error', listener);
|
|
812
|
+
```
|
|
813
|
+
|
|
814
|
+
**Browser (ESM):**
|
|
815
|
+
|
|
816
|
+
```js
|
|
817
|
+
const unsubscribe = db.on('error', (event) => {
|
|
818
|
+
console.error('autoCommit error:', event.error);
|
|
819
|
+
});
|
|
820
|
+
|
|
821
|
+
// Stop listening:
|
|
822
|
+
unsubscribe();
|
|
823
|
+
```
|
|
824
|
+
|
|
825
|
+
**Browser (Bundle):**
|
|
826
|
+
|
|
827
|
+
```js
|
|
828
|
+
const unsubscribe = db.on('error', (event) => {
|
|
829
|
+
console.error('autoCommit error:', event.error);
|
|
830
|
+
});
|
|
831
|
+
|
|
832
|
+
// Stop listening:
|
|
833
|
+
unsubscribe();
|
|
834
|
+
```
|
|
835
|
+
|
|
836
|
+
---
|
|
837
|
+
|
|
838
|
+
### Capacity Control
|
|
839
|
+
|
|
840
|
+
Limit datastore size with the `capacity` config.
|
|
841
|
+
|
|
842
|
+
**Node.js / TypeScript:**
|
|
843
|
+
|
|
844
|
+
```ts
|
|
845
|
+
import { Datastore } from '@frostpillar/frostpillar-storage-engine';
|
|
846
|
+
import { fileDriver } from '@frostpillar/frostpillar-storage-engine/drivers/file';
|
|
847
|
+
|
|
848
|
+
const db = new Datastore({
|
|
849
|
+
capacity: {
|
|
850
|
+
maxSize: '10MB',
|
|
851
|
+
policy: 'strict',
|
|
852
|
+
},
|
|
853
|
+
driver: fileDriver({ filePath: './data/events.fpdb' }),
|
|
854
|
+
});
|
|
855
|
+
```
|
|
856
|
+
|
|
857
|
+
**Browser (ESM):**
|
|
858
|
+
|
|
859
|
+
```js
|
|
860
|
+
import { Datastore } from '@frostpillar/frostpillar-storage-engine';
|
|
861
|
+
import { localStorageDriver } from '@frostpillar/frostpillar-storage-engine/drivers/localStorage';
|
|
862
|
+
|
|
863
|
+
const db = new Datastore({
|
|
864
|
+
capacity: {
|
|
865
|
+
maxSize: '10MB',
|
|
866
|
+
policy: 'strict',
|
|
867
|
+
},
|
|
868
|
+
driver: localStorageDriver({
|
|
869
|
+
databaseKey: 'my-app',
|
|
870
|
+
}),
|
|
871
|
+
});
|
|
872
|
+
```
|
|
873
|
+
|
|
874
|
+
**Browser (Bundle):**
|
|
875
|
+
|
|
876
|
+
```js
|
|
877
|
+
const { Datastore, localStorageDriver } = window.FrostpillarStorageEngine;
|
|
878
|
+
|
|
879
|
+
const db = new Datastore({
|
|
880
|
+
capacity: {
|
|
881
|
+
maxSize: '10MB',
|
|
882
|
+
policy: 'strict',
|
|
883
|
+
},
|
|
884
|
+
driver: localStorageDriver({
|
|
885
|
+
databaseKey: 'my-app',
|
|
886
|
+
}),
|
|
887
|
+
});
|
|
888
|
+
```
|
|
889
|
+
|
|
890
|
+
| Option | Type | Description |
|
|
891
|
+
| --------- | ----------------------------------------------------------------------------------- | ---------------------------------- |
|
|
892
|
+
| `maxSize` | `number` \| `'${n}B'` \| `'${n}KB'` \| `'${n}MB'` \| `'${n}GB'` \| `'backendLimit'` | Maximum datastore size |
|
|
893
|
+
| `policy` | `'strict'` \| `'turnover'` | Behavior when capacity is exceeded |
|
|
894
|
+
|
|
895
|
+
**Policies:**
|
|
896
|
+
|
|
897
|
+
- **`strict`** (default) — rejects writes that exceed the limit with `QuotaExceededError`.
|
|
898
|
+
- **`turnover`** — evicts the oldest records (by insertion order) until the new record fits.
|
|
899
|
+
|
|
900
|
+
**`backendLimit` sentinel:**
|
|
901
|
+
|
|
902
|
+
Set `maxSize: 'backendLimit'` to use the driver's own limit (e.g. `maxChunkChars * maxChunks` for `localStorageDriver`, `maxTotalBytes` for `syncStorageDriver`). Requires a durable driver that supports backend-limit resolution.
|
|
903
|
+
|
|
904
|
+
---
|
|
905
|
+
|
|
906
|
+
### Custom Key Definition
|
|
907
|
+
|
|
908
|
+
By default, keys are non-empty strings with lexicographic ordering. You can define a custom key type by providing all four callbacks:
|
|
909
|
+
|
|
910
|
+
**Node.js / TypeScript:**
|
|
911
|
+
|
|
912
|
+
```ts
|
|
913
|
+
const db = new Datastore({
|
|
914
|
+
key: {
|
|
915
|
+
normalize: (value, fieldName) => {
|
|
916
|
+
if (typeof value === 'number' && Number.isSafeInteger(value)) {
|
|
917
|
+
return value;
|
|
918
|
+
}
|
|
919
|
+
throw new TypeError(`${fieldName} must be a safe integer.`);
|
|
920
|
+
},
|
|
921
|
+
compare: (left, right) => left - right,
|
|
922
|
+
serialize: (key) => key.toString(10),
|
|
923
|
+
deserialize: (serialized) => {
|
|
924
|
+
const parsed = Number(serialized);
|
|
925
|
+
if (!Number.isSafeInteger(parsed)) {
|
|
926
|
+
throw new TypeError('serialized key must be a safe integer.');
|
|
927
|
+
}
|
|
928
|
+
return parsed;
|
|
929
|
+
},
|
|
930
|
+
},
|
|
931
|
+
});
|
|
932
|
+
```
|
|
933
|
+
|
|
934
|
+
**Browser (ESM):**
|
|
935
|
+
|
|
936
|
+
```js
|
|
937
|
+
import { Datastore } from '@frostpillar/frostpillar-storage-engine';
|
|
938
|
+
|
|
939
|
+
const db = new Datastore({
|
|
940
|
+
key: {
|
|
941
|
+
normalize: (value, fieldName) => {
|
|
942
|
+
if (typeof value === 'number' && Number.isSafeInteger(value)) {
|
|
943
|
+
return value;
|
|
944
|
+
}
|
|
945
|
+
throw new TypeError(fieldName + ' must be a safe integer.');
|
|
946
|
+
},
|
|
947
|
+
compare: (left, right) => left - right,
|
|
948
|
+
serialize: (key) => key.toString(10),
|
|
949
|
+
deserialize: (serialized) => {
|
|
950
|
+
const parsed = Number(serialized);
|
|
951
|
+
if (!Number.isSafeInteger(parsed)) {
|
|
952
|
+
throw new TypeError('serialized key must be a safe integer.');
|
|
953
|
+
}
|
|
954
|
+
return parsed;
|
|
955
|
+
},
|
|
956
|
+
},
|
|
957
|
+
});
|
|
958
|
+
```
|
|
959
|
+
|
|
960
|
+
**Browser (Bundle):**
|
|
961
|
+
|
|
962
|
+
```js
|
|
963
|
+
const { Datastore } = window.FrostpillarStorageEngine;
|
|
964
|
+
|
|
965
|
+
const db = new Datastore({
|
|
966
|
+
key: {
|
|
967
|
+
normalize: (value, fieldName) => {
|
|
968
|
+
if (typeof value === 'number' && Number.isSafeInteger(value)) {
|
|
969
|
+
return value;
|
|
970
|
+
}
|
|
971
|
+
throw new TypeError(fieldName + ' must be a safe integer.');
|
|
972
|
+
},
|
|
973
|
+
compare: (left, right) => left - right,
|
|
974
|
+
serialize: (key) => key.toString(10),
|
|
975
|
+
deserialize: (serialized) => {
|
|
976
|
+
const parsed = Number(serialized);
|
|
977
|
+
if (!Number.isSafeInteger(parsed)) {
|
|
978
|
+
throw new TypeError('serialized key must be a safe integer.');
|
|
979
|
+
}
|
|
980
|
+
return parsed;
|
|
981
|
+
},
|
|
982
|
+
},
|
|
983
|
+
});
|
|
984
|
+
```
|
|
985
|
+
|
|
986
|
+
| Callback | Description |
|
|
987
|
+
| ----------------------------- | -------------------------------------------------------- |
|
|
988
|
+
| `normalize(value, fieldName)` | Validate and normalize input to your key type |
|
|
989
|
+
| `compare(left, right)` | Return a finite integer for ordering (`< 0`, `0`, `> 0`) |
|
|
990
|
+
| `serialize(key)` | Convert key to a string for storage |
|
|
991
|
+
| `deserialize(serialized)` | Restore key from stored string |
|
|
992
|
+
|
|
993
|
+
All four are required when `config.key` is provided. `compare` must return a finite integer — `NaN`, `Infinity`, or non-integer values fail with `IndexCorruptionError`.
|
|
994
|
+
|
|
995
|
+
---
|
|
996
|
+
|
|
997
|
+
### Error Handling
|
|
998
|
+
|
|
999
|
+
All public errors extend `FrostpillarError` (which extends `Error`).
|
|
1000
|
+
|
|
1001
|
+
**Node.js / TypeScript:**
|
|
1002
|
+
|
|
1003
|
+
```ts
|
|
1004
|
+
import {
|
|
1005
|
+
Datastore,
|
|
1006
|
+
FrostpillarError,
|
|
1007
|
+
} from '@frostpillar/frostpillar-storage-engine';
|
|
1008
|
+
|
|
1009
|
+
try {
|
|
1010
|
+
await db.put({ key: 'k1', payload: { event: 'login' } });
|
|
1011
|
+
} catch (error) {
|
|
1012
|
+
if (error instanceof FrostpillarError) {
|
|
1013
|
+
console.error(error.name, error.message);
|
|
1014
|
+
} else {
|
|
1015
|
+
throw error;
|
|
1016
|
+
}
|
|
1017
|
+
}
|
|
1018
|
+
```
|
|
1019
|
+
|
|
1020
|
+
**Browser (ESM):**
|
|
1021
|
+
|
|
1022
|
+
```js
|
|
1023
|
+
import {
|
|
1024
|
+
Datastore,
|
|
1025
|
+
FrostpillarError,
|
|
1026
|
+
} from '@frostpillar/frostpillar-storage-engine';
|
|
1027
|
+
|
|
1028
|
+
try {
|
|
1029
|
+
await db.put({ key: 'k1', payload: { event: 'login' } });
|
|
1030
|
+
} catch (error) {
|
|
1031
|
+
if (error instanceof FrostpillarError) {
|
|
1032
|
+
console.error(error.name, error.message);
|
|
1033
|
+
} else {
|
|
1034
|
+
throw error;
|
|
1035
|
+
}
|
|
1036
|
+
}
|
|
1037
|
+
```
|
|
1038
|
+
|
|
1039
|
+
**Browser (Bundle):**
|
|
1040
|
+
|
|
1041
|
+
```js
|
|
1042
|
+
const { Datastore, FrostpillarError } = window.FrostpillarStorageEngine;
|
|
1043
|
+
|
|
1044
|
+
try {
|
|
1045
|
+
await db.put({ key: 'k1', payload: { event: 'login' } });
|
|
1046
|
+
} catch (error) {
|
|
1047
|
+
if (error instanceof FrostpillarError) {
|
|
1048
|
+
console.error(error.name, error.message);
|
|
1049
|
+
} else {
|
|
1050
|
+
throw error;
|
|
1051
|
+
}
|
|
1052
|
+
}
|
|
1053
|
+
```
|
|
1054
|
+
|
|
1055
|
+
#### Error Types
|
|
1056
|
+
|
|
1057
|
+
| Error | Description |
|
|
1058
|
+
| ------------------------- | ---------------------------------------------------------------------- |
|
|
1059
|
+
| `FrostpillarError` | Root class for all Frostpillar errors |
|
|
1060
|
+
| `ValidationError` | Invalid input (payload keys, nesting depth, etc.) |
|
|
1061
|
+
| `ConfigurationError` | Invalid datastore configuration |
|
|
1062
|
+
| `InvalidQueryRangeError` | `start > end` in `getRange()` |
|
|
1063
|
+
| `ClosedDatastoreError` | Operation on a closed datastore |
|
|
1064
|
+
| `QuotaExceededError` | Capacity exceeded under `strict` policy |
|
|
1065
|
+
| `StorageEngineError` | Storage-layer I/O or internal error |
|
|
1066
|
+
| `DatabaseLockedError` | File lock conflict (extends `StorageEngineError`) |
|
|
1067
|
+
| `BinaryFormatError` | Corrupt binary data (extends `StorageEngineError`) |
|
|
1068
|
+
| `PageCorruptionError` | Corrupt page/generation data (extends `StorageEngineError`) |
|
|
1069
|
+
| `IndexCorruptionError` | Corrupt index or invalid internal state (extends `StorageEngineError`) |
|
|
1070
|
+
| `UnsupportedBackendError` | Backend not available in current environment |
|
|
1071
|
+
|
|
1072
|
+
#### `close()` Error Aggregation
|
|
1073
|
+
|
|
1074
|
+
If both a deferred backend initialization failure and a backend close failure occur in the same `close()` call, `close()` throws a native `AggregateError` containing both errors (initialization error first, close error second).
|
|
1075
|
+
|
|
1076
|
+
---
|
|
1077
|
+
|
|
1078
|
+
## API Reference
|
|
1079
|
+
|
|
1080
|
+
### Key-Based Operations
|
|
1081
|
+
|
|
1082
|
+
| Method | Parameters | Returns | Description |
|
|
1083
|
+
| --------------- | ------------------ | ------------------------------ | -------------------------- |
|
|
1084
|
+
| `put(record)` | `{ key, payload }` | `Promise<void>` | Insert a record |
|
|
1085
|
+
| `get(key)` | key | `Promise<KeyedRecord[]>` | All records for key |
|
|
1086
|
+
| `getFirst(key)` | key | `Promise<KeyedRecord \| null>` | First record for key |
|
|
1087
|
+
| `getLast(key)` | key | `Promise<KeyedRecord \| null>` | Last record for key |
|
|
1088
|
+
| `has(key)` | key | `Promise<boolean>` | Check key existence |
|
|
1089
|
+
| `delete(key)` | key | `Promise<number>` | Delete all records for key |
|
|
1090
|
+
|
|
1091
|
+
### ID-Based Operations
|
|
1092
|
+
|
|
1093
|
+
| Method | Parameters | Returns | Description |
|
|
1094
|
+
| ----------------------- | ------------------------ | ------------------------------ | -------------------- |
|
|
1095
|
+
| `getById(id)` | `EntryId` | `Promise<KeyedRecord \| null>` | Get by record ID |
|
|
1096
|
+
| `updateById(id, patch)` | `EntryId`, payload patch | `Promise<boolean>` | Shallow-merge update |
|
|
1097
|
+
| `deleteById(id)` | `EntryId` | `Promise<boolean>` | Delete by record ID |
|
|
1098
|
+
|
|
1099
|
+
### Bulk Operations
|
|
1100
|
+
|
|
1101
|
+
| Method | Parameters | Returns | Description |
|
|
1102
|
+
| ---------------------- | ------------------ | ------------------------ | --------------------------- |
|
|
1103
|
+
| `getAll()` | — | `Promise<KeyedRecord[]>` | All records |
|
|
1104
|
+
| `getRange(start, end)` | start key, end key | `Promise<KeyedRecord[]>` | Inclusive range query |
|
|
1105
|
+
| `getMany(keys)` | key array | `Promise<KeyedRecord[]>` | Records for multiple keys |
|
|
1106
|
+
| `putMany(records)` | record array | `Promise<void>` | Insert multiple records |
|
|
1107
|
+
| `deleteMany(keys)` | key array | `Promise<number>` | Delete across multiple keys |
|
|
1108
|
+
| `clear()` | — | `Promise<void>` | Remove all records |
|
|
1109
|
+
|
|
1110
|
+
### Metadata
|
|
1111
|
+
|
|
1112
|
+
| Method | Returns | Description |
|
|
1113
|
+
| --------- | -------------------- | ------------------------- |
|
|
1114
|
+
| `count()` | `Promise<number>` | Total record count |
|
|
1115
|
+
| `keys()` | `Promise<unknown[]>` | Distinct keys (ascending) |
|
|
1116
|
+
|
|
1117
|
+
### Lifecycle
|
|
1118
|
+
|
|
1119
|
+
| Method | Returns | Description |
|
|
1120
|
+
| ------------------------ | -------------------------- | ------------------------------------------ |
|
|
1121
|
+
| `commit()` | `Promise<void>` | Flush to durable storage (no-op without a driver) |
|
|
1122
|
+
| `close()` | `Promise<void>` | Release resources and locks |
|
|
1123
|
+
| `on('error', listener)` | `() => void` (unsubscribe) | Monitor async errors |
|
|
1124
|
+
| `off('error', listener)` | `void` | Remove error listener |
|
|
1125
|
+
|
|
1126
|
+
### Exported Types
|
|
1127
|
+
|
|
1128
|
+
| Type | Description |
|
|
1129
|
+
|------|-------------|
|
|
1130
|
+
| `DatastoreConfig` | Constructor configuration object |
|
|
1131
|
+
| `DatastoreKeyDefinition` | Custom key normalize/compare/serialize/deserialize callbacks |
|
|
1132
|
+
| `InputRecord` | Record shape accepted by `put()` and `putMany()` |
|
|
1133
|
+
| `KeyedRecord` | Record object with `key`, `payload`, and `_id` fields |
|
|
1134
|
+
| `PersistedRecord` | Internal record format with `payload` and `sizeBytes` |
|
|
1135
|
+
| `RecordPayload` | Payload value type (nested record of strings, numbers, booleans, nulls, and arrays) |
|
|
1136
|
+
| `EntryId` | Branded `number` identifying a specific record (ephemeral, re-issued on restore) |
|
|
1137
|
+
| `DuplicateKeyPolicy` | `'allow' \| 'reject' \| 'replace'` |
|
|
1138
|
+
| `CapacityConfig` | Capacity control configuration (`maxSize` + `policy`) |
|
|
1139
|
+
| `CapacityPolicy` | `'strict' \| 'turnover'` |
|
|
1140
|
+
| `AutoCommitConfig` | Auto-commit configuration (`frequency` + `maxPendingBytes`) |
|
|
1141
|
+
| `AutoCommitFrequencyInput` | Frequency value (`'immediate'` \| number \| time string) |
|
|
1142
|
+
| `DatastoreDriver` | Driver interface for pluggable backends |
|
|
1143
|
+
| `DatastoreDriverController` | Driver controller lifecycle interface |
|
|
1144
|
+
| `DatastoreDriverInitContext` | Context passed to driver during initialization |
|
|
1145
|
+
| `DatastoreDriverInitResult` | Result returned from driver initialization |
|
|
1146
|
+
| `DatastoreDriverSnapshot` | Snapshot payload for persistence |
|
|
1147
|
+
| `DatastoreErrorEvent` | Error event shape emitted by `on('error')` |
|
|
1148
|
+
| `DatastoreErrorListener` | Listener callback type for error events |
|
|
1149
|
+
| `FileBackendConfig` | File driver configuration |
|
|
1150
|
+
| `FileTargetConfig` | File target (path or directory) union type |
|
|
1151
|
+
| `FileTargetByPathConfig` | File target with direct `filePath` |
|
|
1152
|
+
| `FileTargetByDirectoryConfig` | File target with directory-based resolution |
|
|
1153
|
+
| `IndexedDBConfig` | IndexedDB driver configuration |
|
|
1154
|
+
| `LocalStorageConfig` | localStorage driver configuration |
|
|
1155
|
+
| `OpfsConfig` | OPFS driver configuration |
|
|
1156
|
+
| `SyncStorageConfig` | Sync storage driver configuration |
|
|
1157
|
+
| `FrostpillarError` | Root error class for all Frostpillar errors |
|
|
1158
|
+
| `ValidationError` | Invalid input error |
|
|
1159
|
+
| `ConfigurationError` | Invalid configuration error |
|
|
1160
|
+
| `QuotaExceededError` | Capacity exceeded error |
|
|
1161
|
+
| `StorageEngineError` | Storage-layer error |
|
|
1162
|
+
|
|
1163
|
+
For full behavioral details, see the [Datastore API spec](docs/specs/01_DatastoreAPI.md) and [Durable Backends spec](docs/specs/02_DurableBackends.md).
|
|
1164
|
+
|
|
1165
|
+
---
|
|
1166
|
+
|
|
1167
|
+
## How to Contribute
|
|
1168
|
+
|
|
1169
|
+
### Requirements
|
|
1170
|
+
|
|
1171
|
+
- Node.js `>=24.0.0`
|
|
1172
|
+
- pnpm `>=10.0.0`
|
|
1173
|
+
|
|
1174
|
+
### Development Commands
|
|
1175
|
+
|
|
1176
|
+
| Command | Description |
|
|
1177
|
+
| ------------------- | -------------------------------------------- |
|
|
1178
|
+
| `pnpm check` | Run type checking, lint, tests, and textlint |
|
|
1179
|
+
| `pnpm test` | Run tests |
|
|
1180
|
+
| `pnpm build` | Build the package |
|
|
1181
|
+
| `pnpm build:bundle` | Build the browser IIFE bundle |
|
|
1182
|
+
|
|
1183
|
+
### Development Workflow
|
|
1184
|
+
|
|
1185
|
+
This project follows a strict SDD/TDD workflow:
|
|
1186
|
+
|
|
1187
|
+
1. **Spec** — update or create a spec in `docs/specs/` before implementation.
|
|
1188
|
+
2. **Test** — write tests before code.
|
|
1189
|
+
3. **Code** — implement minimal logic to pass the tests.
|
|
1190
|
+
4. **Verify** — run `pnpm check` to ensure everything passes.
|
|
1191
|
+
|
|
1192
|
+
### Documentation
|
|
1193
|
+
|
|
1194
|
+
- [Architecture overview](docs/architecture/overview.md)
|
|
1195
|
+
- [Vision and principles](docs/architecture/vision-and-principles.md)
|
|
1196
|
+
- [Testing strategy](docs/architecture/testing-strategy.md)
|
|
1197
|
+
- [Specs index](docs/specs/README.md)
|
|
1198
|
+
- [ADRs](docs/adr)
|
|
1199
|
+
|
|
1200
|
+
---
|
|
1201
|
+
|
|
1202
|
+
## License
|
|
1203
|
+
|
|
1204
|
+
[MIT](LICENSE)
|