picovolt 1.9.0 → 2.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +34 -16
- package/browser.js +5 -0
- package/package.json +1 -1
- package/picovolt.d.ts +4 -0
- package/picovolt_bg.js +26 -0
- package/picovolt_bg.wasm +0 -0
- package/sqlite.js +5 -0
- package/worker.js +4 -0
package/README.md
CHANGED
|
@@ -3,13 +3,19 @@
|
|
|
3
3
|
[](https://github.com/MiniJe/picovolt/actions/workflows/ci.yml)
|
|
4
4
|
[](https://crates.io/crates/picovolt)
|
|
5
5
|
[](LICENSE)
|
|
6
|
-

|
|
7
7
|
[](https://github.com/MiniJe/picovolt)
|
|
8
8
|
|
|
9
|
-
PicoVolt is an embedded database engine written in Rust.
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
9
|
+
PicoVolt is an embedded database engine written in Rust. **2.0** provides
|
|
10
|
+
concurrent snapshot readers, bounded writer scheduling, and an incremental
|
|
11
|
+
durable commit log. Independent security review and external application trials
|
|
12
|
+
have not been completed. See the [2.0 release ledger](docs/RELEASE_2_0.md)
|
|
13
|
+
for qualification evidence and publication status.
|
|
14
|
+
|
|
15
|
+
Start with the [2.0 guide for every maintained interface](docs/QUICKSTART_2_0.md)
|
|
16
|
+
for atomic batches, persistence choices, log diagnostics and error recovery.
|
|
17
|
+
The [standalone review prompt](docs/INDEPENDENT_REVIEW_PROMPT.md) defines an
|
|
18
|
+
independent assessment and external trials deferred beyond the 2.0 release.
|
|
13
19
|
|
|
14
20
|
If PicoVolt is useful to you, consider starring the repository on GitHub. It is
|
|
15
21
|
the simplest way to help others discover the project.
|
|
@@ -28,7 +34,7 @@ MVCC-preserving columnar layout with packed decimal encoding.
|
|
|
28
34
|
|
|
29
35
|
## Status
|
|
30
36
|
|
|
31
|
-
The
|
|
37
|
+
The engine is exercised by Rust unit and integration suites plus doctests
|
|
32
38
|
and maintained-binding integration tests. CI also enforces formatting and
|
|
33
39
|
warning-free Clippy builds on Linux and Windows. Shipped changes are tracked in
|
|
34
40
|
[CHANGELOG.md](CHANGELOG.md), and the remaining work toward 2.0 is tracked in
|
|
@@ -73,8 +79,10 @@ warning-free Clippy builds on Linux and Windows. Shipped changes are tracked in
|
|
|
73
79
|
and are rejected rather than mis-run.
|
|
74
80
|
- **Page-backed engine.** Tables are append-only chains of hot row pages and
|
|
75
81
|
optional packed cold pages, each header linking to the next. Inserts append to
|
|
76
|
-
a row tail
|
|
77
|
-
|
|
82
|
+
a row tail. Commit cost also includes catalog/index maintenance, retained-log
|
|
83
|
+
accounting and the selected durability protocol; it is not uniformly O(1).
|
|
84
|
+
Logged 2.0 workspaces persist index definitions to reduce catalog rewrites.
|
|
85
|
+
Reads stream through a
|
|
78
86
|
bounded buffer pool ([`storage/cache.rs`](src/storage/cache.rs)), so datasets
|
|
79
87
|
need not fit in RAM, and opt-in ordered indexes
|
|
80
88
|
([`storage/index.rs`](src/storage/index.rs)) turn `WHERE col = value` into a
|
|
@@ -84,10 +92,12 @@ warning-free Clippy builds on Linux and Windows. Shipped changes are tracked in
|
|
|
84
92
|
each flush `fsync` the data and commit the manifest atomically (write to a temp
|
|
85
93
|
file, `fsync`, then rename). The default `Fast` mode uses the OS cache only:
|
|
86
94
|
fast and durable on a clean exit, but not power-loss-safe.
|
|
87
|
-
- **
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
95
|
+
- **Concurrent transactions.** Native `SharedDatabase` exposes independent
|
|
96
|
+
snapshot readers and bounded FIFO writers. Logged workspaces sync original
|
|
97
|
+
pages before overwriting them and publish an ordered physical change stream.
|
|
98
|
+
Reopening rolls back incomplete writes. Format 6 prevents old binaries from
|
|
99
|
+
bypassing recovery. Existing 1.x images remain readable and migratable.
|
|
100
|
+
See [the concurrency contract](docs/CONCURRENCY.md) for limits and costs.
|
|
91
101
|
- **Hardened against untrusted input.** Opening a `.pvdb` or workspace, or running
|
|
92
102
|
a WASM module, validates manifest hashes (no path traversal), bounds-checks CAS
|
|
93
103
|
offsets and page chains (no out-of-bounds reads or infinite loops on a crafted
|
|
@@ -150,6 +160,15 @@ databases.
|
|
|
150
160
|
Durability is selectable via `Database::set_durability` (`Fast` OS-cache default,
|
|
151
161
|
or crash-safe `Sync` with fsync and an atomic manifest).
|
|
152
162
|
|
|
163
|
+
Native Rust applications can begin adopting the 2.0 concurrency surface through
|
|
164
|
+
`SharedDatabase`. It is a cloneable, bounded worker-thread coordinator with
|
|
165
|
+
explicit read and write transaction handles, FIFO admission, cooperative
|
|
166
|
+
cancellation, and rollback before failed or abandoned writes release the queue.
|
|
167
|
+
The first slice serializes execution and preserves format v5; use clones of one
|
|
168
|
+
coordinator rather than independently opening the same development workspace.
|
|
169
|
+
See [Shared database concurrency](docs/CONCURRENCY.md) for the contract and
|
|
170
|
+
current limits.
|
|
171
|
+
|
|
153
172
|
Measured results and the methodology are in [BENCHMARKS.md](BENCHMARKS.md). In
|
|
154
173
|
short, PicoVolt is a page-backed engine with O(1) filesystem appends (autocommit
|
|
155
174
|
around 33k rows/s, linear), larger-than-RAM reads through a bounded buffer pool (a
|
|
@@ -175,8 +194,8 @@ pv inspect ./data.pv --json
|
|
|
175
194
|
`Database::compact_step(max_pages)` preserves record addresses, indexes, and
|
|
176
195
|
complete MVCC history; it never compacts the mutable tail and leaves a page in
|
|
177
196
|
row form when transposition would not save space. Each pass uses the
|
|
178
|
-
crash-recoverable
|
|
179
|
-
|
|
197
|
+
crash-recoverable transaction protocol: allow bounded journal space for logged
|
|
198
|
+
workspaces, or a full rollback image for unlogged workspaces. Baked-image migration is
|
|
180
199
|
out-of-place and deeply verified before publication:
|
|
181
200
|
|
|
182
201
|
```sh
|
|
@@ -194,7 +213,7 @@ See [Migration and compaction](docs/MIGRATION.md).
|
|
|
194
213
|
| **Rust** (crates.io) | `cargo add picovolt` |
|
|
195
214
|
| **JavaScript / npm** (WebAssembly, browser and Node) | `npm install picovolt` |
|
|
196
215
|
| **Python** (native wheels) | `python -m pip install picovolt` |
|
|
197
|
-
| **Go** (`database/sql` and direct API) | `go get github.com/MiniJe/picovolt/bindings/go@
|
|
216
|
+
| **Go** (`database/sql` and direct API) | `go get github.com/MiniJe/picovolt/bindings/go/v2@v2.0.0`, then provide the matching native C ABI library described in [`bindings/go/`](bindings/go) |
|
|
198
217
|
| **C** | Download the matching `picovolt-capi-*` bundle from the [latest release](https://github.com/MiniJe/picovolt/releases/latest), or run `cargo build --release --features capi` |
|
|
199
218
|
| **In-memory** (native, no filesystem) | `Database::open_memory()`, export with `bake_to_bytes()` |
|
|
200
219
|
|
|
@@ -263,7 +282,6 @@ native modules built on the public API. Both are documented in
|
|
|
263
282
|
| | |
|
|
264
283
|
|--|--|
|
|
265
284
|
| Roadmap | [ROADMAP.md](ROADMAP.md) |
|
|
266
|
-
| One-million-download plan | [docs/ROADMAP_1M_DOWNLOADS.md](docs/ROADMAP_1M_DOWNLOADS.md) |
|
|
267
285
|
| Monetization thesis | [docs/MONETIZATION.md](docs/MONETIZATION.md) |
|
|
268
286
|
| Enterprise integration foundation | [docs/ENTERPRISE.md](docs/ENTERPRISE.md) |
|
|
269
287
|
| Platform and file support | [docs/SUPPORT.md](docs/SUPPORT.md) |
|
package/browser.js
CHANGED
|
@@ -62,6 +62,11 @@ export class PersistentDb {
|
|
|
62
62
|
return statement;
|
|
63
63
|
}
|
|
64
64
|
|
|
65
|
+
executeMany(sql, rows) {
|
|
66
|
+
this._assertOpen();
|
|
67
|
+
return JSON.parse(this.db.executeMany(sql, rows)).mutated;
|
|
68
|
+
}
|
|
69
|
+
|
|
65
70
|
async save() {
|
|
66
71
|
this._assertOpen();
|
|
67
72
|
const root = await navigator.storage.getDirectory();
|
package/package.json
CHANGED
package/picovolt.d.ts
CHANGED
|
@@ -20,6 +20,10 @@ export class Db {
|
|
|
20
20
|
* `... BEFORE tx` time-travel query.
|
|
21
21
|
*/
|
|
22
22
|
currentTx(): number;
|
|
23
|
+
/**
|
|
24
|
+
* Atomically execute an INSERT/UPDATE/DELETE for an array of parameter arrays.
|
|
25
|
+
*/
|
|
26
|
+
executeMany(sql: string, rows: any): string;
|
|
23
27
|
/**
|
|
24
28
|
* Export the whole database as a `.pvdb` byte image (a `Uint8Array` in JS).
|
|
25
29
|
*/
|
package/picovolt_bg.js
CHANGED
|
@@ -45,6 +45,32 @@ export class Db {
|
|
|
45
45
|
const ret = wasm.db_currentTx(this.__wbg_ptr);
|
|
46
46
|
return ret >>> 0;
|
|
47
47
|
}
|
|
48
|
+
/**
|
|
49
|
+
* Atomically execute an INSERT/UPDATE/DELETE for an array of parameter arrays.
|
|
50
|
+
* @param {string} sql
|
|
51
|
+
* @param {any} rows
|
|
52
|
+
* @returns {string}
|
|
53
|
+
*/
|
|
54
|
+
executeMany(sql, rows) {
|
|
55
|
+
let deferred3_0;
|
|
56
|
+
let deferred3_1;
|
|
57
|
+
try {
|
|
58
|
+
const ptr0 = passStringToWasm0(sql, wasm.__wbindgen_malloc, wasm.__wbindgen_realloc);
|
|
59
|
+
const len0 = WASM_VECTOR_LEN;
|
|
60
|
+
const ret = wasm.db_executeMany(this.__wbg_ptr, ptr0, len0, rows);
|
|
61
|
+
var ptr2 = ret[0];
|
|
62
|
+
var len2 = ret[1];
|
|
63
|
+
if (ret[3]) {
|
|
64
|
+
ptr2 = 0; len2 = 0;
|
|
65
|
+
throw takeFromExternrefTable0(ret[2]);
|
|
66
|
+
}
|
|
67
|
+
deferred3_0 = ptr2;
|
|
68
|
+
deferred3_1 = len2;
|
|
69
|
+
return getStringFromWasm0(ptr2, len2);
|
|
70
|
+
} finally {
|
|
71
|
+
wasm.__wbindgen_free(deferred3_0, deferred3_1, 1);
|
|
72
|
+
}
|
|
73
|
+
}
|
|
48
74
|
/**
|
|
49
75
|
* Export the whole database as a `.pvdb` byte image (a `Uint8Array` in JS).
|
|
50
76
|
* @returns {Uint8Array}
|
package/picovolt_bg.wasm
CHANGED
|
Binary file
|
package/sqlite.js
CHANGED
|
@@ -94,6 +94,11 @@ class Database {
|
|
|
94
94
|
return new Statement(this, sql);
|
|
95
95
|
}
|
|
96
96
|
|
|
97
|
+
executeMany(sql, rows) {
|
|
98
|
+
this._assertOpen();
|
|
99
|
+
return JSON.parse(this._db.executeMany(sql, rows)).mutated;
|
|
100
|
+
}
|
|
101
|
+
|
|
97
102
|
// Run one or more `;`-separated statements with no bound parameters.
|
|
98
103
|
exec(sql) {
|
|
99
104
|
this._assertOpen();
|
package/worker.js
CHANGED
|
@@ -30,6 +30,10 @@ self.addEventListener("message", async ({ data }) => {
|
|
|
30
30
|
result = { statementId, parameterCount: statement.parameterCount };
|
|
31
31
|
break;
|
|
32
32
|
}
|
|
33
|
+
case "executeMany":
|
|
34
|
+
if (!database) throw new Error("open the database first");
|
|
35
|
+
result = database.executeMany(data.sql, data.rows);
|
|
36
|
+
break;
|
|
33
37
|
case "execute": {
|
|
34
38
|
const statement = statements.get(data.statementId);
|
|
35
39
|
if (!statement) throw new Error("unknown PicoVolt prepared statement");
|