@libredb/libredb 0.0.3 → 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,326 +1,309 @@
1
1
  # LibreDB
2
2
 
3
- > Status: pre-alpha. The kernel (`core.ts`) and all three lenses — `lens/kv.ts` (key-value), `lens/document.ts` (JSON documents), and `lens/relational.ts` (typed tables) — are implemented and tested.
3
+ <picture>
4
+ <source media="(prefers-color-scheme: dark)" srcset="docs/img/01-hero-dark.png">
5
+ <img alt="LibreDB - Multi-model without the magic. One core, three lenses, every line tested." src="docs/img/01-hero-light.png" width="100%">
6
+ </picture>
7
+
8
+ **Multi-model without the magic. One core, three lenses, every line tested.**
9
+
10
+ [![npm version](https://img.shields.io/npm/v/@libredb/libredb.svg)](https://www.npmjs.com/package/@libredb/libredb)
11
+ [![CI](https://github.com/libredb/libredb-database/actions/workflows/ci.yml/badge.svg)](https://github.com/libredb/libredb-database/actions/workflows/ci.yml)
12
+ [![Quality Gate](https://sonarcloud.io/api/project_badges/measure?project=libredb_libredb-database&metric=alert_status)](https://sonarcloud.io/summary/new_code?id=libredb_libredb-database)
13
+ [![Coverage](https://sonarcloud.io/api/project_badges/measure?project=libredb_libredb-database&metric=coverage)](https://sonarcloud.io/summary/new_code?id=libredb_libredb-database)
14
+ [![license](https://img.shields.io/npm/l/@libredb/libredb.svg)](./LICENSE)
15
+ [![types: included](https://img.shields.io/badge/types-included-blue.svg)](https://www.typescriptlang.org/)
16
+ [![dependencies: 0](https://img.shields.io/badge/dependencies-0-brightgreen.svg)](./package.json)
17
+ [![bundle size](https://img.shields.io/bundlephobia/minzip/@libredb/libredb)](https://bundlephobia.com/package/@libredb/libredb)
18
+ [![status: pre-alpha](https://img.shields.io/badge/status-pre--alpha-orange.svg)](#project-status--roadmap)
19
+
20
+ LibreDB is a small, readable, embeddable, multi-model database written in TypeScript. It is built on
21
+ one idea: a database can be powerful and still be understood by opening its source. A single ordered
22
+ key-value core handles durability and transactions; key-value, document, and relational APIs are thin
23
+ *lenses* over that one core — not three separate engines. It runs in-memory for tests or file-backed
24
+ for durability, ships **zero runtime dependencies**, and proves its crash recovery with deterministic
25
+ simulation testing. Today it is pre-alpha, aimed at test and development environments — small enough to
26
+ learn how a database actually works, and serious enough to grow into more.
27
+
28
+ ## Highlights
29
+
30
+ - **One small core, three lenses** — key-value, document, and relational over a single ordered
31
+ key-value engine (FoundationDB-style), not three engines bolted together.
32
+ - **Multi-model** — raw strings, JSON documents, and schema-validated typed tables in the same
33
+ database, even the same file.
34
+ - **Readable by design** — the kernel is under 600 lines; open the source and learn how a database
35
+ actually works.
36
+ - **Embeddable, zero dependencies** — `bun add @libredb/libredb` and go; nothing else to install or
37
+ run.
38
+ - **In-memory or durable** — `open()` for tests, `open({ path })` for a crash-safe, WAL-backed,
39
+ fsync-on-commit file.
40
+ - **TypeScript-native** — full types shipped, ESM-only, tree-shakeable, under 4 kB min+brotli.
41
+ - **Crash recovery you can trust** — 100% line coverage on the core, plus deterministic simulation
42
+ testing that tortures the write-ahead log under a simulated crashing filesystem.
43
+ - **Nothing hidden** — queries are plain in-engine scans, errors surface, and costs are obvious (O(n)
44
+ scans, no secret indexes).
45
+
46
+ ## Quick start
4
47
 
5
- **LibreDB is a small, readable, embeddable database.**
6
-
7
- It is built on one idea: a database can be powerful and still remain understandable. You should be able to open its source, learn how a database actually works, and embed it into your own product in an afternoon.
8
-
9
- - A single small storage core (`core.ts`), with relational, document, and key-value as thin *lenses* on top — not three separate engines (FoundationDB-style architecture).
10
- - Written in TypeScript, shipped on npm.
11
- - Open at the edges, guarded at the durability core. Every line of the core is tested.
12
-
13
- LibreDB is the database in a three-product family that shares one spine:
14
-
15
- - **LibreDB** — the database (this repository).
16
- - **LibreDB Studio** — the open-source IDE for every database (Postgres, MySQL, MongoDB, Redis, and more). LibreDB is one database it supports natively, not a requirement.
17
- - **LibreDB Platform** — the managed, team-oriented form of data.
18
-
19
- ## Usage — the key-value lens
48
+ ```sh
49
+ bun add @libredb/libredb
50
+ # or: npm install @libredb/libredb
51
+ ```
20
52
 
21
- The first lens is `kv`: a durable, ordered, string-keyed map over the kernel. You `open` a kernel database and put a `kv` lens over it.
53
+ LibreDB is ESM-only, ships zero runtime dependencies, and targets Bun (the development runtime) and
54
+ Node 22+. The same database speaks all three lenses — here they are in one file:
22
55
 
23
56
  ```ts
24
- import { open, kv } from "libredb";
57
+ import { open, kv, doc, table } from "@libredb/libredb";
25
58
 
26
- // In-memory: the natural fit for tests and ephemeral use.
59
+ // In-memory for tests, or open({ path: "data.libredb" }) for a durable, crash-safe file.
27
60
  const db = open();
28
61
 
29
- // Or file-backed and durable. Every write is appended to a write-ahead
30
- // log and fsync'd before it returns, so a committed write survives a
31
- // crash; reopening the same path reconstructs exactly the committed state:
32
- //
33
- // const db = open({ path: "data.libredb" });
34
-
35
- const store = kv(db);
62
+ // 1. Key-value: a durable, ordered, string-keyed map.
63
+ const cache = kv(db);
64
+ cache.set("user:1", "Ada");
65
+ cache.get("user:1"); // "Ada"
36
66
 
37
- store.set("user:1", "Ada");
38
- store.set("user:2", "Grace");
67
+ // 2. Document: a collection of JSON documents under string ids.
68
+ const logs = doc(db, "logs");
69
+ logs.put("l1", { level: "info", message: "started", at: 1 });
70
+ logs.find({ level: "info" }).toArray(); // [{ id: "l1", doc: { ... } }]
39
71
 
40
- store.get("user:1"); // "Ada"
41
- store.get("missing"); // undefined
72
+ // 3. Relational: a schema-validated, typed table with where / select / join.
73
+ const users = table(db, "users", {
74
+ primaryKey: "id",
75
+ columns: { id: "string", name: "string", age: "number" },
76
+ });
77
+ users.insert({ id: "1", name: "Ada", age: 36 });
78
+ users.where({ name: "Ada" }).select("id", "age").toArray(); // [{ id: "1", age: 36 }]
42
79
 
43
80
  db.close();
44
81
  ```
45
82
 
46
- ### Writes report what they changed
83
+ Each lens has its own guide: [key-value](./docs/guides/key-value.md) ·
84
+ [document](./docs/guides/document.md) · [relational](./docs/guides/relational.md) ·
85
+ [catalog](./docs/guides/catalog.md).
47
86
 
48
- Every write returns a `WriteResult` (`{ changed }`), so a caller can always see what the write actually did:
87
+ ## Install elsewhere: JSR, CDN, and the browser
49
88
 
50
- ```ts
51
- store.set("user:1", "Ada Lovelace").changed; // 1 (overwrote the existing value)
52
-
53
- store.delete("user:2").changed; // 1 (it existed)
54
- store.delete("user:2").changed; // 0 (already gone)
55
- ```
56
-
57
- ### Ordered scans
58
-
59
- Keys are ordered by byte order, so reads come back in ascending key order. `range(start, end)` scans the half-open interval `[start, end)` — `start` included, `end` excluded:
60
-
61
- ```ts
62
- store.set("a", "1");
63
- store.set("b", "2");
64
- store.set("c", "3");
65
-
66
- store.range("a", "c").toArray();
67
- // [{ key: "a", value: "1" }, { key: "b", value: "2" }] — "c" is excluded
68
- ```
89
+ LibreDB is the same ESM-only package everywhere; only how you reach it changes.
69
90
 
70
- `prefix(p)` scans every key beginning with `p` — the canonical ordered-KV query:
71
-
72
- ```ts
73
- store.prefix("user:").toArray();
74
- // [{ key: "user:1", value: "Ada Lovelace" }]
75
- ```
91
+ **JSR** published to [jsr.io](https://jsr.io/@libredb/libredb) alongside npm:
76
92
 
77
- A read returns a `Result`, which is **lazy and re-iterable**: nothing runs until you iterate it (or call `toArray()`), and each pass re-runs the scan against the current state. Iterate it directly or materialize it:
78
-
79
- ```ts
80
- for (const { key, value } of store.prefix("user:")) {
81
- console.log(key, value);
82
- }
83
- ```
84
-
85
- ### Multi-key atomicity
86
-
87
- Each `kv` operation auto-commits in its own transaction. When you need several writes to apply atomically, drop to the kernel's `transact` directly — it commits all of the body's writes together, or nothing if the body throws:
88
-
89
- ```ts
90
- const encode = new TextEncoder();
91
-
92
- db.transact((tx) => {
93
- tx.set(encode.encode("from"), encode.encode("90"));
94
- tx.set(encode.encode("to"), encode.encode("110"));
95
- }); // both writes commit together, or neither does
93
+ ```sh
94
+ bunx jsr add @libredb/libredb
95
+ # or: npx jsr add @libredb/libredb / deno add jsr:@libredb/libredb
96
96
  ```
97
97
 
98
- The kernel speaks raw bytes on purpose; the `kv` lens is the string-ergonomic face over it.
99
-
100
- ## Usage — the document lens
101
-
102
- The `doc` lens is the JSON face over the same kernel: a *collection* of JSON documents, each stored under a string `id`. Where `kv` stores strings, `doc` stores objects — numbers stay numbers, and nested objects and arrays survive the round-trip.
98
+ **CDN** every release is served from the npm registry by the usual CDNs. Pin a version:
103
99
 
104
100
  ```ts
105
- import { open, doc } from "libredb";
106
-
107
- const db = open(); // or open({ path: "data.libredb" }) for durability
108
-
109
- const users = doc(db, "users");
110
-
111
- users.put("1", { name: "Ada", age: 36, active: true });
112
- users.put("2", { name: "Grace", age: 45, active: false });
113
-
114
- users.get("1"); // { name: "Ada", age: 36, active: true }
115
- users.get("missing"); // undefined
116
-
117
- db.close();
101
+ import { open, kv } from "https://esm.sh/@libredb/libredb@0.1.0";
118
102
  ```
119
103
 
120
- Documents are scoped to their collection: `doc(db, "users")` never sees a `doc(db, "orders")` document and the boundary is exact, so `users` and `users2` stay fully separate.
121
-
122
- ### Writes report what they changed
123
-
124
- Like the kv lens, every write returns a `WriteResult` (`{ changed }`):
104
+ **Browser** a dedicated entry that imports nothing from `node:`, so it bundles for the browser
105
+ cleanly. Its `open` carries no default filesystem: an in-memory database works anywhere, and a
106
+ path-backed open takes a filesystem you inject (e.g. the OPFS adapter shown below).
125
107
 
126
108
  ```ts
127
- users.put("1", { name: "Ada Lovelace", age: 36 }).changed; // 1 (overwrote the existing document)
109
+ import { open, kv } from "@libredb/libredb/browser";
128
110
 
129
- users.delete("2").changed; // 1 (it existed)
130
- users.delete("2").changed; // 0 (already gone)
111
+ const db = open(); // in-memory
112
+ kv(db).set("greeting", "hello");
131
113
  ```
132
114
 
133
- ### Scanning and finding
134
-
135
- `all()` returns every document in the collection as `{ id, doc }` rows, in ascending `id` (byte) order:
115
+ For durable storage in the browser, run inside a Web Worker and back the database with an OPFS sync
116
+ access handle (the kernel stays synchronous — no async core):
136
117
 
137
118
  ```ts
138
- const people = doc(open(), "people");
139
-
140
- people.put("1", { name: "Ada", team: "research", active: true });
141
- people.put("2", { name: "Grace", team: "research", active: false });
142
- people.put("3", { name: "Edsger", team: "ops", active: true });
143
-
144
- people.all().toArray();
145
- // [
146
- // { id: "1", doc: { name: "Ada", team: "research", active: true } },
147
- // { id: "2", doc: { name: "Grace", team: "research", active: false } },
148
- // { id: "3", doc: { name: "Edsger", team: "ops", active: true } },
149
- // ]
150
- ```
119
+ import { open, opfsFileSystem } from "@libredb/libredb/browser";
151
120
 
152
- `find(predicate)` returns the documents whose top-level fields all equal the predicate's. Multiple fields are AND'd, and equality is deep and type-sensitive — `1` does not match `"1"`:
153
-
154
- ```ts
155
- people.find({ team: "research", active: true }).toArray();
156
- // [{ id: "1", doc: { name: "Ada", team: "research", active: true } }]
121
+ const root = await navigator.storage.getDirectory();
122
+ const file = await root.getFileHandle("app.libredb", { create: true });
123
+ const handle = await file.createSyncAccessHandle();
124
+ const db = open({ path: "app.libredb", fs: opfsFileSystem(handle) });
157
125
  ```
158
126
 
159
- `find` is an O(n) full collection scan with an in-engine predicate there are no secondary indexes (a deliberate v1 omission, so you can read exactly what a query costs). Like every read it returns a lazy, re-iterable `Result`, so you can iterate it directly or call `toArray()`.
160
-
161
- ## Usage — the relational lens
127
+ A browser-targeting bundler resolves the browser build automatically via the package's `browser`
128
+ export condition, so importing the main `@libredb/libredb` entry works too.
162
129
 
163
- The `table` lens is the *typed* face over the same kernel: a table is a schema-validated collection of rows. Where `doc` accepts any JSON object, `table` declares its columns and their types up front and enforces them at insert. A row is stored under `<table>:<pk>`, so a table is literally a schema-validated document collection — it reuses the document codec and the same collection-isolation boundary.
130
+ ## Command-line tool
164
131
 
165
- You declare a schema (the columns, their types, and which one is the primary key) when you open the table:
132
+ The package ships a `libredb` bin for inspecting and editing `.libredb` files no code required:
166
133
 
167
- ```ts
168
- import { open, table } from "libredb";
169
-
170
- const db = open(); // or open({ path: "data.libredb" }) for durability
171
-
172
- const users = table(db, "users", {
173
- primaryKey: "id",
174
- columns: { id: "string", name: "string", age: "number", active: "boolean" },
175
- });
176
-
177
- users.insert({ id: "1", name: "Ada", age: 36, active: true });
178
- users.insert({ id: "2", name: "Grace", age: 45, active: false });
179
-
180
- users.get("1"); // { id: "1", name: "Ada", age: 36, active: true }
181
- users.get("missing"); // undefined
182
-
183
- db.close();
134
+ ```sh
135
+ npx libredb inspect data.libredb # namespaces, kinds, and table schemas
136
+ npx libredb stats data.libredb # file size and namespace counts
137
+ npx libredb get data.libredb user:1 # print one value
138
+ npx libredb scan data.libredb user: # print key=value under a prefix
139
+ npx libredb set data.libredb user:1 Ada # set a key
140
+ npx libredb delete data.libredb user:1 # remove a key
141
+ npx libredb import data.libredb seed.json # bulk-set from a JSON object, atomically
184
142
  ```
185
143
 
186
- A column type is one of `"string"`, `"number"`, `"boolean"`, or `"object"` (a plain JSON object). The `primaryKey` must name a declared `"string"` column — it becomes the kernel key.
187
-
188
- ### Validation is strict at insert
144
+ Read commands open the file read-only, so inspection never mutates it. Write commands take an
145
+ advisory `<path>.lock` to refuse a second concurrent writer; pass `--force` to override a stale lock.
189
146
 
190
- Insert rejects a row that is missing a declared column, has a wrong-typed value, or carries a field the schema does not declare — there is no silent coercion or field-dropping:
147
+ Prefer a standalone binary with no Node or Bun installed? Each release attaches self-contained
148
+ executables (Linux, macOS, Windows; x64 and arm64) with `.sha256` checksums on its
149
+ [GitHub Release](https://github.com/libredb/libredb-database/releases). Or build one locally with
150
+ `bun run compile`.
191
151
 
192
- ```ts
193
- users.insert({ id: "3", name: "Edsger" });
194
- // throws: missing required column "age"
152
+ Or run the CLI from a container (multi-arch, published to GHCR) — mount your data and pass a command:
195
153
 
196
- users.insert({ id: "3", name: "Edsger", age: "old", active: true });
197
- // throws: column "age" expected number, got string
198
-
199
- users.insert({ id: "3", name: "Edsger", age: 40, active: true, role: "ops" });
200
- // throws: unknown column "role" (not declared in the table schema)
154
+ ```sh
155
+ docker run --rm -v "$PWD:/data" ghcr.io/libredb/libredb inspect /data/app.libredb
201
156
  ```
202
157
 
203
- ### Writes report what they changed
204
-
205
- Like the other lenses, every write returns a `WriteResult` (`{ changed }`):
206
-
207
- ```ts
208
- users.insert({ id: "1", name: "Ada Lovelace", age: 36, active: true }).changed; // 1 (overwrote)
209
-
210
- users.delete("2").changed; // 1 (it existed)
211
- users.delete("2").changed; // 0 (already gone)
158
+ The image is a CLI shell, not a server: LibreDB stays an embedded, in-process database.
159
+
160
+ ## How it works: one core, three lenses
161
+
162
+ <picture>
163
+ <source media="(prefers-color-scheme: dark)" srcset="docs/img/02-lenses-dark.png">
164
+ <img alt="One core, three lenses: kv, document, and relational APIs over a single ordered key-value core (core.ts), reaching disk through one FileSystem seam." src="docs/img/02-lenses-light.png" width="100%">
165
+ </picture>
166
+
167
+ LibreDB has a single ordered byte key-value kernel (`src/core.ts`). Key-value, document, and relational
168
+ are thin typed *lenses* over it — three faces of the same store. A relational table is physically a
169
+ JSON document collection, which is physically ordered key-value entries built from composite keys like
170
+ `users:42`. The kernel reaches the disk through one injectable filesystem seam, which is also what
171
+ makes deterministic crash testing possible.
172
+
173
+ ```mermaid
174
+ flowchart TB
175
+ subgraph consumers [Consumers]
176
+ App[Your app]
177
+ Studio[LibreDB Studio]
178
+ Platform[LibreDB Platform]
179
+ end
180
+
181
+ API["Public API · index.ts<br/>open · kv · doc · table · catalog"]
182
+
183
+ subgraph lenses [Lenses and shared edges - open, fast to contribute]
184
+ KV[kv<br/>strings]
185
+ DOC[document<br/>JSON]
186
+ REL[relational<br/>typed tables]
187
+ CAT[catalog<br/>self-describing registry]
188
+ end
189
+
190
+ CORE["core.ts - THE KERNEL<br/>ordered byte KV · serializable txns · WAL · crash recovery"]
191
+ FS["FileSystem seam<br/>node:fs adapter - or SimFS for crash tests"]
192
+
193
+ App --> API
194
+ Studio --> API
195
+ Platform --> API
196
+ API --> KV
197
+ API --> DOC
198
+ API --> REL
199
+ API --> CAT
200
+ REL --> DOC
201
+ KV --> CORE
202
+ DOC --> CORE
203
+ REL --> CORE
204
+ CAT --> CORE
205
+ CORE --> FS
212
206
  ```
213
207
 
214
- ### Querying — where, select, join
208
+ <picture>
209
+ <source media="(prefers-color-scheme: dark)" srcset="docs/img/03-trust-dark.png">
210
+ <img alt="Open at the edges, guarded at the core: lenses, query surface, catalog, adapters, Studio, and docs are open to contribute; core.ts is guarded with 100% line coverage, deterministic crash tests, and heavy review - everything reaches the store through one narrow transact() port." src="docs/img/03-trust-light.png" width="100%">
211
+ </picture>
215
212
 
216
- Reads return a chainable `Query` (which is itself a lazy, re-iterable `Result`). `where(predicate)` keeps the rows whose top-level fields all equal the predicate's — deep and type-sensitive, just like document `find`, so `36` does not match `"36"`. `select(...columns)` projects each row down to the named columns. They compose in any order:
213
+ **The file boundary is the trust boundary.** Below the line (`core.ts`) is guarded: heavy review and
214
+ deterministic crash tests, because a bug there corrupts data. Above the line (lenses, query, catalog)
215
+ is open and fast to contribute to, because the worst a bug can do is present a bad *view* — it reaches
216
+ the store only through one narrow `transact` port. For the full tour, read
217
+ [`ARCHITECTURE.md`](./ARCHITECTURE.md).
217
218
 
218
- ```ts
219
- const people = table(open(), "people", {
220
- primaryKey: "id",
221
- columns: { id: "string", name: "string", team: "string", active: "boolean" },
222
- });
219
+ ## When to use LibreDB
223
220
 
224
- people.insert({ id: "1", name: "Ada", team: "research", active: true });
225
- people.insert({ id: "2", name: "Grace", team: "research", active: false });
226
- people.insert({ id: "3", name: "Edsger", team: "ops", active: true });
221
+ **Reach for it when you want to:**
227
222
 
228
- people.where({ team: "research", active: true }).select("name").toArray();
229
- // [{ name: "Ada" }]
230
- ```
223
+ - Back tests and local development with a real, durable, multi-model store instead of mocks.
224
+ - Embed a small database directly in a TypeScript / Bun / Node app with zero infrastructure.
225
+ - Learn how a database works by reading — and hacking — a small, honest codebase.
226
+ - Prototype across key-value, document, and relational shapes without standing up three systems.
231
227
 
232
- `join(other, leftField, rightField)` is an inner equi-join via nested loop: it pairs each left row with every right row whose `rightField` equals the left row's `leftField`. The result rows carry every column of both sides, qualified as `table.column`, so the two sides never collide — and `select`/`where` name a qualified column the same way as any other:
228
+ **Do not use it (yet) when you need:**
233
229
 
234
- ```ts
235
- const orders = table(db, "orders", {
236
- primaryKey: "id",
237
- columns: { id: "string", userId: "string", total: "number" },
238
- });
239
-
240
- orders.insert({ id: "o1", userId: "1", total: 42 });
241
- orders.insert({ id: "o2", userId: "1", total: 7 });
242
-
243
- users.join(orders, "id", "userId").select("users.name", "orders.total").toArray();
244
- // [
245
- // { "users.name": "Ada Lovelace", "orders.total": 42 },
246
- // { "users.name": "Ada Lovelace", "orders.total": 7 },
247
- // ]
248
- ```
249
-
250
- `where` is O(n) in the table size and `join` is O(n*m) — nested-loop, no indexes (the same deliberate v1 omission as the document lens). Unmatched rows on either side are dropped (inner join), and a left key matching several right rows fans out to several result rows.
251
-
252
- ## The catalog — what a database holds
230
+ - A hardened production datastore at scale — it is **pre-alpha**; today's beachhead is test/dev.
231
+ - Secondary indexes or a query planner — queries are O(n) scans by design in v1 (on the roadmap).
232
+ - Concurrent multi-process access, replication, or a networked client/server — it is embedded and
233
+ in-process.
234
+ - SQL wire compatibility or an existing-driver ecosystem.
253
235
 
254
- Open a LibreDB file cold and the bytes alone don't say which lens each namespace belongs to. The catalog records that as you write: a `table` registers `{ kind: "relational", schema }` when its handle is built, and a `doc` collection registers `{ kind: "document" }` on its first write. `catalog(db)` reads the whole registry — a `Map` from namespace name to its entry — so a tool can render faithful per-kind views without guessing:
236
+ These limits are deliberate v1 scope, not hidden gaps LibreDB's strength comes from what it refuses.
237
+ See the [Manifesto](./MANIFESTO.md).
255
238
 
256
- ```ts
257
- import { open, table, doc, catalog } from "libredb";
239
+ ## Reliability
258
240
 
259
- const db = open();
241
+ <picture>
242
+ <source media="(prefers-color-scheme: dark)" srcset="docs/img/04-reliability-dark.png">
243
+ <img alt="Crash recovery you can trust: a length-framed, CRC-32-checksummed write-ahead log fsync'd before commit; recovery truncates the last un-fsync'd record so what remains is always a valid committed prefix - proven by deterministic simulation testing." src="docs/img/04-reliability-light.png" width="100%">
244
+ </picture>
260
245
 
261
- table(db, "users", { primaryKey: "id", columns: { id: "string", age: "number" } });
262
- doc(db, "logs").put("l1", { message: "hello" });
246
+ A transaction that returns has been written to a length-framed, CRC-32-checksummed write-ahead log and
247
+ `fsync`'d *before* the commit becomes visible — so a committed write survives a crash, and a crash can
248
+ only ever damage the last, un-fsync'd record (which recovery detects and truncates). This is not just
249
+ asserted: the kernel's crash/recovery path is proven by **deterministic simulation testing**, running
250
+ the real engine against a seeded in-memory filesystem that tears, corrupts, and crashes the log on
251
+ command, then checking that recovery is always a valid committed prefix.
263
252
 
264
- const registry = catalog(db);
265
- registry.get("users");
266
- // { kind: "relational", schema: { primaryKey: "id", columns: { id: "string", age: "number" } } }
267
- registry.get("logs");
268
- // { kind: "document" }
269
- [...registry.keys()].sort();
270
- // ["logs", "users"]
253
+ ```sh
254
+ bun run test # includes a bounded 50-seed DST run
271
255
  ```
272
256
 
273
- The catalog lives under a reserved key prefix that sorts below all user data, so its entries never appear in a `kv`, `doc`, or `table` scan — and no user row leaks into the registry. `kv` namespaces are deliberately not cataloged: `kv` is the raw layer with full keyspace access.
274
-
275
- A tool that renders the raw `kv` layer (so it sees everything, including the catalog) should hide those engine-internal keys. Rather than hardcode the byte layout, import the contract: `isReservedKey(key)` is true for any key in the reserved namespace, and `RESERVED_MARKER` / `CATALOG_PREFIX` are the underlying constants if you need to build a range.
276
-
277
- ```ts
278
- import { open, kv, isReservedKey } from "libredb";
279
-
280
- const db = open();
281
- // ... user writes through doc/table, which also write catalog entries ...
282
-
283
- const visible = kv(db).range("", "￿").toArray().filter((e) => !isReservedKey(e.key));
284
- // only user keys; catalog entries (under the reserved marker) are filtered out
285
- ```
257
+ The full durability and DST walkthrough is in [`docs/RELIABILITY.md`](./docs/RELIABILITY.md).
286
258
 
287
- ## Reliability — deterministic simulation testing
259
+ ## Documentation
288
260
 
289
- Durability is the trust-critical promise: a transaction that returns has been fsync'd, and a crash can only ever damage the last, un-fsync'd record. LibreDB proves this with **deterministic simulation testing (DST)** — the WAL's crash/recovery path is tortured under a seeded, in-memory simulated filesystem. The kernel reaches the disk only through an injectable FS seam (`open({ path, fs })`), so a test can run the real engine on a `SimFS` that crashes on command, tears the un-fsync'd tail at a seeded point, and injects CRC corruption or short reads.
261
+ | Topic | Where |
262
+ |-------|-------|
263
+ | Lens guides (kv, document, relational, catalog) | [`docs/guides/`](./docs/guides/) |
264
+ | Architecture — the guided tour under the hood | [`ARCHITECTURE.md`](./ARCHITECTURE.md) |
265
+ | Design — the locked engineering decisions | [`docs/DESIGN.md`](./docs/DESIGN.md) |
266
+ | Reliability — durability and crash recovery | [`docs/RELIABILITY.md`](./docs/RELIABILITY.md) |
267
+ | Manifesto — what LibreDB is and refuses to be | [`MANIFESTO.md`](./MANIFESTO.md) |
268
+ | LibreDB Studio integration | [`docs/STUDIO.md`](./docs/STUDIO.md) |
290
269
 
291
- Every run is driven by one integer seed and a seeded workload of `set`/`delete`/`transact` operations. After the crash and reopen, recovery must reproduce **a valid committed prefix** of the workload — every transaction that returned successfully, and never a torn or un-committed one. An independent committed-map model (sharing no code with the engine) is the oracle the recovered state is compared against.
270
+ ## Project status & roadmap
292
271
 
293
- The DST suite runs as part of the normal gate:
272
+ LibreDB is **pre-alpha** (`0.0.x`). The architecture is in place and every line of the core is tested,
273
+ but the API may still change and it is not yet meant for production data.
294
274
 
295
- ```sh
296
- bun run test # runs a bounded 50 seeds (fast, CI-friendly)
297
- ```
275
+ - **Done:** the ordered key-value kernel (transactions, WAL, crash recovery); the key-value, document,
276
+ and relational lenses; the self-describing catalog; the deterministic simulation testing harness;
277
+ 100% line/function/statement coverage on the core.
278
+ - **Next:** secondary indexes and a richer query surface; more query operators; additional lenses;
279
+ production-hardening milestones (directory fsync on first create, WAL compaction/checkpointing).
298
280
 
299
- Run a longer soak by raising the seed count (and optionally the base seed):
281
+ ## The LibreDB family
300
282
 
301
- ```sh
302
- LIBREDB_DST_SEEDS=5000 bun test src/sim/dst.test.ts
303
- LIBREDB_DST_BASE=1000000 LIBREDB_DST_SEEDS=5000 bun test src/sim/dst.test.ts
304
- ```
283
+ <picture>
284
+ <source media="(prefers-color-scheme: dark)" srcset="docs/img/05-family-dark.png">
285
+ <img alt="The LibreDB family: LibreDB (the database, this repo), LibreDB Studio (the open-source IDE for every database), and LibreDB Platform (the managed, team-oriented form of data) - three products, one access-model spine." src="docs/img/05-family-light.png" width="100%">
286
+ </picture>
305
287
 
306
- When a seed fails, the suite prints the exact replay hint `Replay with runSeed(<seed>)`. Because everything is seed-driven, one seed reproduces the whole run byte-for-byte:
288
+ LibreDB is the database in a three-product family that shares one access-model spine:
307
289
 
308
- ```ts
309
- import { runSeed } from "./src/sim/dst.ts";
290
+ - **LibreDB** — the database (this repository).
291
+ - **LibreDB Studio** the open-source IDE for *every* database (Postgres, MySQL, MongoDB, Redis, and
292
+ more). LibreDB is one database it supports natively, not a requirement.
293
+ - **LibreDB Platform** — the managed, team-oriented form of data.
310
294
 
311
- const result = runSeed(42);
312
- result.passed; // true — recovered state is a valid committed prefix
313
- result.recovered; // Map of the recovered committed key-value state
314
- result.expected; // the model oracle's committed state
315
- ```
295
+ ## Contributing
316
296
 
317
- The DST harness lives in `src/sim/` and is excluded from the published package it is test machinery, not shipped code.
297
+ Contributions are welcome. LibreDB is open at the edges and guarded at the durability core see
298
+ [`CONTRIBUTING.md`](./CONTRIBUTING.md) for how to get set up, the `bun run gate` bar every change must
299
+ pass, and where contributions land fastest. Please also read the
300
+ [`CODE_OF_CONDUCT.md`](./CODE_OF_CONDUCT.md).
318
301
 
319
- ## Read first
302
+ ## Security
320
303
 
321
- - [`MANIFESTO.md`](./MANIFESTO.md) — what LibreDB is and what it refuses to be.
322
- - [`DESIGN.md`](./DESIGN.md) — the engineering decision record behind the manifesto.
304
+ To report a security vulnerability, see [`SECURITY.md`](./SECURITY.md) — please do not open a public
305
+ issue.
323
306
 
324
307
  ## License
325
308
 
326
- Open source and free under the MIT License.
309
+ Open source and free under the [MIT License](./LICENSE).
@@ -0,0 +1,3 @@
1
+ import type { FileSystem } from "../core.ts";
2
+ /** Build the default node:fs-backed {@link FileSystem}. */
3
+ export declare function nodeFileSystem(): FileSystem;
@@ -0,0 +1,49 @@
1
+ /**
2
+ * adapter/node-fs.ts — the default {@link FileSystem} for Node and Bun.
3
+ *
4
+ * This is an edge, not the kernel (DESIGN.md section 5): it holds the one place
5
+ * LibreDB touches `node:fs`. The kernel (`core.ts`) is runtime-agnostic and
6
+ * imports nothing from `node:`; it reaches the disk only through the injected
7
+ * {@link FileSystem} seam. Keeping the node dependency HERE — and out of the
8
+ * kernel — is what lets the browser entry (`browser.ts`) ship without dragging
9
+ * `node:fs` into its import graph. The default Node entry (`index.ts`) wires this
10
+ * adapter in as the default `fs`, so production behaviour is unchanged.
11
+ *
12
+ * Each method is the obvious synchronous syscall, so the adapter adds an
13
+ * interface boundary, not behaviour. Appends go through one append-mode
14
+ * descriptor (creating the file if missing); reads, size and truncate work by
15
+ * path, matching how the WAL has always reached the disk.
16
+ */
17
+ import { closeSync, fsyncSync, openSync, readFileSync, statSync, truncateSync, writeSync } from "node:fs";
18
+ /** Build the default node:fs-backed {@link FileSystem}. */
19
+ export function nodeFileSystem() {
20
+ return {
21
+ open(path) {
22
+ const fd = openSync(path, "a"); // append-only; creates the file if missing
23
+ return {
24
+ size() {
25
+ return statSync(path).size;
26
+ },
27
+ read(offset, length) {
28
+ // A fresh Uint8Array so the returned slice is an independent copy, not
29
+ // a view aliasing a shared Buffer pool.
30
+ return new Uint8Array(readFileSync(path)).subarray(offset, offset + length);
31
+ },
32
+ append(bytes) {
33
+ for (let written = 0; written < bytes.length;) {
34
+ written += writeSync(fd, bytes, written);
35
+ }
36
+ },
37
+ fsync() {
38
+ fsyncSync(fd);
39
+ },
40
+ truncate(length) {
41
+ truncateSync(path, length);
42
+ },
43
+ close() {
44
+ closeSync(fd);
45
+ },
46
+ };
47
+ },
48
+ };
49
+ }