@crvouga/sqlite-mem 0.1.0 → 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (149) hide show
  1. package/AGENTS.md +154 -0
  2. package/COMPATIBILITY-AUDIT.md +7 -9
  3. package/COMPATIBILITY.md +5 -4
  4. package/README.md +166 -16
  5. package/compat/coverage.json +797 -512
  6. package/compat/requirements.json +1 -1
  7. package/dist/api/database.d.ts +97 -12
  8. package/dist/api/statement.d.ts +65 -18
  9. package/dist/ast/nodes.d.ts +11 -2
  10. package/dist/constraints/check.d.ts +5 -6
  11. package/dist/errors/index.d.ts +36 -3
  12. package/dist/executor/attach.d.ts +3 -4
  13. package/dist/executor/ddl.d.ts +3 -4
  14. package/dist/executor/dml.d.ts +4 -4
  15. package/dist/executor/env.d.ts +9 -11
  16. package/dist/executor/execute.d.ts +3 -4
  17. package/dist/executor/pragma.d.ts +3 -4
  18. package/dist/executor/result.d.ts +10 -6
  19. package/dist/executor/select.d.ts +4 -5
  20. package/dist/executor/simple-select.d.ts +3 -4
  21. package/dist/executor/triggers.d.ts +6 -7
  22. package/dist/executor/vtable.d.ts +5 -6
  23. package/dist/expressions/context.d.ts +5 -5
  24. package/dist/expressions/equals.d.ts +3 -0
  25. package/dist/expressions/eval.d.ts +9 -4
  26. package/dist/expressions/index.d.ts +3 -4
  27. package/dist/expressions/like.d.ts +13 -3
  28. package/dist/functions/aggregate.d.ts +1 -2
  29. package/dist/functions/datetime.d.ts +1 -2
  30. package/dist/functions/extensions.d.ts +1 -2
  31. package/dist/functions/index.d.ts +8 -9
  32. package/dist/functions/json.d.ts +3 -4
  33. package/dist/functions/math.d.ts +1 -2
  34. package/dist/functions/registry.d.ts +5 -6
  35. package/dist/functions/scalar.d.ts +2 -3
  36. package/dist/functions/table-valued.d.ts +3 -4
  37. package/dist/functions/window.d.ts +1 -2
  38. package/dist/index.d.ts +27 -12
  39. package/dist/index.js +986 -203
  40. package/dist/index.js.map +4 -4
  41. package/dist/indexes/index.d.ts +17 -4
  42. package/dist/json/index.d.ts +7 -8
  43. package/dist/json/jsonb.d.ts +1 -2
  44. package/dist/json/ops.d.ts +8 -3
  45. package/dist/json/parse.d.ts +2 -3
  46. package/dist/json/path.d.ts +1 -2
  47. package/dist/json/stringify.d.ts +1 -2
  48. package/dist/json/tvf.d.ts +1 -2
  49. package/dist/json/types.d.ts +0 -1
  50. package/dist/lexer/tokenize.d.ts +7 -1
  51. package/dist/parser/index.d.ts +11 -5
  52. package/dist/parser/parser.d.ts +3 -3
  53. package/dist/planner/access.d.ts +13 -6
  54. package/dist/planner/index.d.ts +1 -2
  55. package/dist/runtime/clock.d.ts +8 -2
  56. package/dist/runtime/index.d.ts +3 -4
  57. package/dist/runtime/options.d.ts +4 -6
  58. package/dist/runtime/prng.d.ts +12 -1
  59. package/dist/schema/catalog.d.ts +2 -3
  60. package/dist/serialization/codec.d.ts +17 -2
  61. package/dist/serialization/index.d.ts +1 -2
  62. package/dist/storage/database-state.d.ts +16 -8
  63. package/dist/storage/index.d.ts +6 -7
  64. package/dist/storage/row.d.ts +1 -2
  65. package/dist/storage/table.d.ts +8 -4
  66. package/dist/transactions/manager.d.ts +2 -3
  67. package/dist/types/collation.d.ts +1 -2
  68. package/dist/types/strict.d.ts +6 -0
  69. package/dist/types/value.d.ts +30 -2
  70. package/dist/unstable.d.ts +20 -0
  71. package/dist/unstable.js +8734 -0
  72. package/dist/unstable.js.map +7 -0
  73. package/dist/vtable/fts/options.d.ts +0 -1
  74. package/dist/vtable/fts/porter.d.ts +0 -1
  75. package/dist/vtable/fts/query.d.ts +0 -1
  76. package/dist/vtable/fts/table.d.ts +5 -6
  77. package/dist/vtable/fts/tokenize.d.ts +1 -2
  78. package/dist/vtable/fts5.d.ts +1 -2
  79. package/dist/vtable/index.d.ts +2 -3
  80. package/dist/vtable/modules.d.ts +2 -3
  81. package/package.json +19 -10
  82. package/compat/gate-report.json +0 -424
  83. package/dist/api/database.d.ts.map +0 -1
  84. package/dist/api/statement.d.ts.map +0 -1
  85. package/dist/ast/nodes.d.ts.map +0 -1
  86. package/dist/constraints/check.d.ts.map +0 -1
  87. package/dist/errors/index.d.ts.map +0 -1
  88. package/dist/executor/attach.d.ts.map +0 -1
  89. package/dist/executor/ddl.d.ts.map +0 -1
  90. package/dist/executor/dml.d.ts.map +0 -1
  91. package/dist/executor/env.d.ts.map +0 -1
  92. package/dist/executor/execute.d.ts.map +0 -1
  93. package/dist/executor/pragma.d.ts.map +0 -1
  94. package/dist/executor/result.d.ts.map +0 -1
  95. package/dist/executor/select.d.ts.map +0 -1
  96. package/dist/executor/simple-select.d.ts.map +0 -1
  97. package/dist/executor/triggers.d.ts.map +0 -1
  98. package/dist/executor/vtable.d.ts.map +0 -1
  99. package/dist/expressions/context.d.ts.map +0 -1
  100. package/dist/expressions/eval.d.ts.map +0 -1
  101. package/dist/expressions/index.d.ts.map +0 -1
  102. package/dist/expressions/like.d.ts.map +0 -1
  103. package/dist/functions/aggregate.d.ts.map +0 -1
  104. package/dist/functions/datetime.d.ts.map +0 -1
  105. package/dist/functions/extensions.d.ts.map +0 -1
  106. package/dist/functions/index.d.ts.map +0 -1
  107. package/dist/functions/json.d.ts.map +0 -1
  108. package/dist/functions/math.d.ts.map +0 -1
  109. package/dist/functions/registry.d.ts.map +0 -1
  110. package/dist/functions/scalar.d.ts.map +0 -1
  111. package/dist/functions/table-valued.d.ts.map +0 -1
  112. package/dist/functions/window.d.ts.map +0 -1
  113. package/dist/index.d.ts.map +0 -1
  114. package/dist/indexes/index.d.ts.map +0 -1
  115. package/dist/json/index.d.ts.map +0 -1
  116. package/dist/json/jsonb.d.ts.map +0 -1
  117. package/dist/json/ops.d.ts.map +0 -1
  118. package/dist/json/parse.d.ts.map +0 -1
  119. package/dist/json/path.d.ts.map +0 -1
  120. package/dist/json/stringify.d.ts.map +0 -1
  121. package/dist/json/tvf.d.ts.map +0 -1
  122. package/dist/json/types.d.ts.map +0 -1
  123. package/dist/lexer/tokenize.d.ts.map +0 -1
  124. package/dist/parser/index.d.ts.map +0 -1
  125. package/dist/parser/parser.d.ts.map +0 -1
  126. package/dist/planner/access.d.ts.map +0 -1
  127. package/dist/planner/index.d.ts.map +0 -1
  128. package/dist/runtime/clock.d.ts.map +0 -1
  129. package/dist/runtime/index.d.ts.map +0 -1
  130. package/dist/runtime/options.d.ts.map +0 -1
  131. package/dist/runtime/prng.d.ts.map +0 -1
  132. package/dist/schema/catalog.d.ts.map +0 -1
  133. package/dist/serialization/codec.d.ts.map +0 -1
  134. package/dist/serialization/index.d.ts.map +0 -1
  135. package/dist/storage/database-state.d.ts.map +0 -1
  136. package/dist/storage/index.d.ts.map +0 -1
  137. package/dist/storage/row.d.ts.map +0 -1
  138. package/dist/storage/table.d.ts.map +0 -1
  139. package/dist/transactions/manager.d.ts.map +0 -1
  140. package/dist/types/collation.d.ts.map +0 -1
  141. package/dist/types/value.d.ts.map +0 -1
  142. package/dist/vtable/fts/options.d.ts.map +0 -1
  143. package/dist/vtable/fts/porter.d.ts.map +0 -1
  144. package/dist/vtable/fts/query.d.ts.map +0 -1
  145. package/dist/vtable/fts/table.d.ts.map +0 -1
  146. package/dist/vtable/fts/tokenize.d.ts.map +0 -1
  147. package/dist/vtable/fts5.d.ts.map +0 -1
  148. package/dist/vtable/index.d.ts.map +0 -1
  149. package/dist/vtable/modules.d.ts.map +0 -1
package/AGENTS.md ADDED
@@ -0,0 +1,154 @@
1
+ # AGENTS.md — contributing to sqlite-mem
2
+
3
+ Guidance for humans and coding agents editing this repository. For install and consumer API, see [README.md](README.md). For the feature matrix, see [COMPATIBILITY.md](COMPATIBILITY.md).
4
+
5
+ ## Mission and non-goals
6
+
7
+ **Mission:** full SQLite3 **SQL dialect** behavioral parity vs **SQLite 3.51.0** (`bun:sqlite`). Same statements → same observable results (rows, errors, counters), proven by differential contracts and a fail-closed gate.
8
+
9
+ **Allowed intentional differences:**
10
+
11
+ 1. Custom snapshot codec (`SQLM`), not on-disk `.sqlite` files
12
+ 2. Deterministic `random()` / `randomblob()` and fixed `'now'` by default (injectable)
13
+ 3. `NOT APPLICABLE` items: C API, VFS/pager/WAL, file locking, URI filenames
14
+
15
+ **Non-goals:** shipping a SQLite file format, native/WASM bindings, or matching better-sqlite3’s full Node API surface.
16
+
17
+ ## SQL pipeline
18
+
19
+ ```
20
+ SQL string
21
+ → tokenize() src/lexer/tokenize.ts
22
+ → parse() src/parser/index.ts → parser.ts
23
+ → Statement[] AST src/ast/nodes.ts
24
+ → Statement.execute() src/api/statement.ts
25
+ → executeStatement() src/executor/execute.ts
26
+ → per-stmt executor select / dml / ddl / …
27
+ ```
28
+
29
+ Public entry points: `Database.exec` / `query` / `prepare` in [`src/api/database.ts`](src/api/database.ts).
30
+
31
+ **Planner note:** [`src/planner/index.ts`](src/planner/index.ts) `preparePlan` is an **identity stub** (`Plan === Statement`) and is unused. Runtime index/PK access paths live in [`src/planner/access.ts`](src/planner/access.ts) (`tryIndexedTableRows`, `lookupTableRows`, `tryJoinProbe`), called from the executor.
32
+
33
+ SELECT has a **fast path** ([`src/executor/simple-select.ts`](src/executor/simple-select.ts)) and a **full path** ([`src/executor/select.ts`](src/executor/select.ts)). Fast paths return `null` when the shape is unsupported; then the full path runs. Changing join/WHERE/expression semantics in only one path will silently diverge.
34
+
35
+ ## `src/` map
36
+
37
+ | Directory | Role |
38
+ | --- | --- |
39
+ | `api/` | Public `Database` / `Statement` facade |
40
+ | `ast/` | Discriminated-union AST (`nodes.ts`) |
41
+ | `lexer/` | Tokenizer |
42
+ | `parser/` | Recursive-descent parser |
43
+ | `planner/` | Access-path helpers (not a logical plan IR) |
44
+ | `executor/` | Statement dispatch, SELECT, DML, DDL, PRAGMA, triggers, ATTACH |
45
+ | `expressions/` | `evalExpr`, LIKE/GLOB, eval context |
46
+ | `functions/` | Scalar / aggregate / window / datetime / JSON / math / TVF registries |
47
+ | `types/` | Engine `SqlValue`, affinity, collation |
48
+ | `storage/` | In-memory tables, rows, `DatabaseState` |
49
+ | `indexes/` | Equality index store |
50
+ | `schema/` | `sqlite_master` / catalog |
51
+ | `constraints/` | NOT NULL / PK / UNIQUE / CHECK |
52
+ | `transactions/` | BEGIN / COMMIT / SAVEPOINT (clones state + PRNG) |
53
+ | `runtime/` | Clock, PRNG, `DatabaseOptions` |
54
+ | `serialization/` | `SQLM` snapshot codec |
55
+ | `json/` | JSON1 / JSONB internals |
56
+ | `vtable/` | FTS5 and other virtual-table modules |
57
+ | `errors/` | `SqliteError`, `unsupported()` |
58
+
59
+ Hot / large files: `parser/parser.ts`, `executor/select.ts`, `executor/dml.ts`.
60
+
61
+ ## Critical conventions
62
+
63
+ - **AST `type` tags** are snake_case (`"create_table"`, `"drop_index"`). TypeScript interfaces are PascalCase (`CreateTableStmt`).
64
+ - **Identifiers** are case-folded (`toLowerCase`) as Map keys in storage and function registries.
65
+ - **Engine `SqlValue`** ([`src/types/value.ts`](src/types/value.ts)) may include `SqlReal` / `SqlJsonText`. **Harness `SqlValue`** ([`tests/harness/types.ts`](tests/harness/types.ts)) is the normalized compare type — do not confuse them.
66
+ - **API `Statement`** vs **AST `Statement`**: the API class aliases the AST union as `AstStatement`.
67
+ - **`Database`** (API) vs **`DatabaseState`** (engine storage).
68
+ - Throw **`SqliteError`** with an `ErrorCategory`. Missing SQL must fail loud via `unsupported()` — the inventory gate fails if the oracle exposes an unimplemented builtin/module.
69
+ - Fast-path helpers (`tryExecuteSimpleSelect`, `tryFastInsert`, `tryIndexedTableRows`, …): return `null` → fall through. Update **both** paths when semantics change.
70
+ - TypeScript: `strict` + `noUncheckedIndexedAccess`. Imports use `.ts` extensions. Biome: 2-space, double quotes, 120 columns.
71
+ - IEEE `-0` is canonicalized to `+0` on bind, affinity, and arithmetic. Keep determinism invariants (see README).
72
+
73
+ ## Change checklists
74
+
75
+ ### New SQL statement
76
+
77
+ 1. Add union member + interface in [`src/ast/nodes.ts`](src/ast/nodes.ts)
78
+ 2. Parse in [`src/parser/parser.ts`](src/parser/parser.ts) (`parseStatement` dispatch)
79
+ 3. Handle in [`src/executor/execute.ts`](src/executor/execute.ts) (+ `ddl.ts` / `dml.ts` / `vtable.ts` as needed)
80
+ 4. Mutate `DatabaseState` if schema changes
81
+ 5. Add differential contract under `tests/contract/<area>/`
82
+ 6. Optionally add evidence paths in `SOURCE_SEED` in [`scripts/sqlite-requirements.ts`](scripts/sqlite-requirements.ts), then `bun run requirements`
83
+
84
+ ### New SQL function
85
+
86
+ 1. Implement and register in the right map under `src/functions/*` (`getScalarFunctions()`, datetime, JSON, etc.) so [`scripts/sqlite-inventory.ts`](scripts/sqlite-inventory.ts) sees it
87
+ 2. Contract tests under `tests/contract/functions/` (and related areas)
88
+ 3. Run `bun run inventory` / `bun run test:sqlite-compat` — oracle builtins must not be missing
89
+
90
+ ### New contract test
91
+
92
+ 1. Prefer helpers in [`tests/contract/helpers.ts`](tests/contract/helpers.ts):
93
+ - `parity` — query both engines
94
+ - `execParity` — writes
95
+ - `sequenceParity` — multi-step (optional final-state compare)
96
+ - `errorParity` / `queryErrorParity` — both must fail
97
+ - `ftsRankParity` — REAL epsilon compare
98
+ 2. Or `matrixBoth` + `expectParity` from `tests/harness/`
99
+ 3. **Do not** treat isolated internal unit tests as SQLite proof. The differential suite is authoritative.
100
+ 4. Gate: `bun run test:sqlite-compat`
101
+
102
+ ## Test layout
103
+
104
+ | Path | Role |
105
+ | --- | --- |
106
+ | `tests/contract/` | Differential SQL vs `bun:sqlite` (**authoritative**) |
107
+ | `tests/fuzz/` | fast-check property tests (seeded); same two backends |
108
+ | `tests/harness/` | Compare/normalize helpers + harness unit tests |
109
+ | `tests/adapters/` | Wrappers for sqlite-mem and `bun:sqlite` |
110
+
111
+ Examples of public API usage: `tests/contract/api/`, `tests/contract/parameters/`, `tests/contract/determinism/`, `examples/react-vite`.
112
+
113
+ ### Fuzz replay
114
+
115
+ Default seed `0x5a17e0e1`. On failure the seed is printed:
116
+
117
+ ```bash
118
+ bun test tests/fuzz
119
+ SQLITE_MEM_FUZZ_SEED=12345 bun test tests/fuzz
120
+ SQLITE_MEM_FUZZ_SEED=12345 SQLITE_MEM_FUZZ_PATH='0:1' bun test tests/fuzz
121
+ ```
122
+
123
+ ## Compat system
124
+
125
+ | Command | Role |
126
+ | --- | --- |
127
+ | `bun run test:sqlite-compat` | Requirements + fail-closed gate + contract/fuzz/harness |
128
+ | `bun run inventory` | Oracle `pragma_function_list` / modules vs memory registries |
129
+ | `bun run requirements` | Refresh sqlite.org requirements → `compat/requirements.json` + `compat/coverage.json` |
130
+ | `bun run fts-surface` | FTS oracle surface → `compat/fts-oracle-surface.json` |
131
+
132
+ Statuses: **VERIFIED** / **PARTIALLY VERIFIED** / **UNSUPPORTED** / **NOT APPLICABLE**. Do not market PARTIAL as complete. Coverage evidence is directory paths (e.g. `tests/contract/joins/`), not automatic from test filenames.
133
+
134
+ Details: [COMPATIBILITY.md](COMPATIBILITY.md), audit: [COMPATIBILITY-AUDIT.md](COMPATIBILITY-AUDIT.md).
135
+
136
+ ## Local gates
137
+
138
+ Requires [Bun](https://bun.sh).
139
+
140
+ ```bash
141
+ bun install
142
+ bun run ci:local # same gates as GitHub Actions CI (except publish)
143
+ bun run check # format + lint + typecheck + sqlite-compat suite
144
+ bun run format
145
+ bun run lint
146
+ bun run typecheck
147
+ bun run test:sqlite-compat
148
+ bun test # contract + fuzz + harness
149
+ bun run build
150
+ ```
151
+
152
+ ## PR and commits
153
+
154
+ Use [Conventional Commits](https://www.conventionalcommits.org/) for commits and PR titles (enforced in CI). Prefer squash merges with a conventional title. Releasing is automated via semantic-release — see [README.md](README.md#releasing).
@@ -44,14 +44,13 @@ JSON / JSONB:
44
44
  VERIFIED
45
45
 
46
46
  Aggregates / windows:
47
- VERIFIED — includes string_agg, ntile, cume_dist, percent_rank
48
- (window EXCLUDE still thinner)
47
+ VERIFIED — includes string_agg, ntile, cume_dist, percent_rank, EXCLUDE
49
48
 
50
49
  CTEs / transactions / savepoints / constraints / FK / triggers:
51
- VERIFIED (deferred FK thinner PARTIAL edges)
50
+ VERIFIED (deferred FK + composite FK)
52
51
 
53
52
  Indexes / views / generated / STRICT / WITHOUT ROWID:
54
- VERIFIED / PARTIAL per COMPATIBILITY.md
53
+ VERIFIED — partial/expression indexes, STRICT tables, leftmost prefix
55
54
 
56
55
  PRAGMAs:
57
56
  PARTIALLY VERIFIED — schema/FK; storage pragmas N/A or :memory: no-op
@@ -67,11 +66,11 @@ ANALYZE / REINDEX / VACUUM:
67
66
  VERIFIED (:memory: observable parity)
68
67
 
69
68
  Prepared statements / errors / snapshot:
70
- PARTIAL / VERIFIED — schema invalidation thin; SQLM logical round-trip VERIFIED
69
+ VERIFIED — schema invalidation re-prepares; SQLM logical round-trip VERIFIED
71
70
 
72
- Differential tests:
73
- Total: 656 under `bun test` (contract + fuzz + harness)
74
- Passed: 656
71
+ Differential tests:
72
+ Total: 727 under `bun test` (contract + fuzz + harness)
73
+ Passed: 727
75
74
  Failed: 0
76
75
 
77
76
  Stateful / fuzz:
@@ -91,7 +90,6 @@ New incompatibilities found & fixed (this pass):
91
90
  Remaining known differences:
92
91
  Custom SQLM snapshots; deterministic random()/'now';
93
92
  EXPLAIN/INDEXED BY stubs/no-ops; some PRAGMA storage no-ops;
94
- Window EXCLUDE / deferred FK / STRICT / prepared invalidation thinner;
95
93
  BigInt beyond Number.MAX_SAFE_INTEGER without bun safeIntegers;
96
94
  NOT APPLICABLE C API / on-disk / VFS surfaces.
97
95
 
package/COMPATIBILITY.md CHANGED
@@ -34,10 +34,11 @@ Oracle builtins (math, string extras, uuid, ieee754, …) and modules (FTS3/4/5,
34
34
  | Area | Status | Notes |
35
35
  | --- | --- | --- |
36
36
  | Core DML / SELECT / joins / CTE / UPSERT / RETURNING | VERIFIED | Contract + fuzz |
37
+ | STRICT tables / indexes | VERIFIED | STRICT types; partial + expression indexes; leftmost prefix |
37
38
  | Expressions / operators / `->` `->>` / row values | VERIFIED | Row-value + precedence contracts |
38
39
  | Affinity / NULL / COLLATE | VERIFIED | |
39
- | Constraints / FK / triggers / views / ATTACH | VERIFIED | Deferred FK thinner PARTIAL edges |
40
- | Windows (incl. ntile/cume_dist/percent_rank) | VERIFIED | EXCLUDE still thinner |
40
+ | Constraints / FK / triggers / views / ATTACH | VERIFIED | Deferred FK, composite FK |
41
+ | Windows (incl. ntile/cume_dist/percent_rank) | VERIFIED | EXCLUDE NO OTHERS/CURRENT ROW/GROUP/TIES |
41
42
  | JSON1 / JSONB / TVFs | VERIFIED | |
42
43
  | Math / string / date extras / uuid / ieee754 | VERIFIED | Scope-3 inventory |
43
44
  | FTS3 / FTS4 / FTS5 + MATCH | PARTIALLY VERIFIED | Differential FTS suite + fuzz vs 3.51.0; see FTS matrix below. Shadow-table change counters intentionally diverge. |
@@ -76,7 +77,7 @@ Reference: **SQLite 3.51.0** (`bun:sqlite`). Inventory: `bun run scripts/fts-ora
76
77
  | RTREE / dbstat / bytecode / tables_used | VERIFIED | dbstat synthetic pages; bytecode empty cursor |
77
78
  | ANALYZE / REINDEX / VACUUM | VERIFIED | `:memory:` observable parity |
78
79
  | EXPLAIN / INDEXED BY | PARTIALLY VERIFIED | Stub shapes / no-ops |
79
- | Prepared stmt schema invalidation | PARTIALLY VERIFIED | Thin |
80
+ | Prepared stmt schema invalidation | VERIFIED | Re-prepare after ALTER/DROP; `tests/contract/api/schema-invalidation.test.ts` |
80
81
  | On-disk file format / C API | NOT APPLICABLE | |
81
82
 
82
83
  ## How to verify
@@ -85,7 +86,7 @@ Reference: **SQLite 3.51.0** (`bun:sqlite`). Inventory: `bun run scripts/fts-ora
85
86
  bun run test:sqlite-compat # requirements + gate + contract/fuzz/harness
86
87
  bun run inventory # oracle function/module inventory
87
88
  bun run requirements # refresh SQLite.org requirements + coverage
88
- bun run test:browser # Playwright smoke (not the SQL oracle)
89
+ bun run build && bun run verify-package # ESM browser build + isomorphic pack gates
89
90
  ```
90
91
 
91
92
  Do not treat isolated unit tests of internal modules as proof of SQLite compatibility. The differential matrix runner is authoritative for SQL behavior; `test:sqlite-compat` is the release gate.
package/README.md CHANGED
@@ -5,10 +5,22 @@ Pure TypeScript, completely in-memory SQLite implementation aiming for **full SQ
5
5
  - Runs in modern browsers and Node.js / Bun
6
6
  - **Zero** WASM, native bindings, workers, or filesystem dependencies
7
7
  - Entire database stored in memory
8
+ - **Synchronous** ESM-only API (no Promises, no `require`)
8
9
  - **Verified against SQLite 3.51.0** (`bun:sqlite`) via differential contracts + fail-closed gate
9
10
  - Intentional differences: deterministic `random()` / `'now'` by default, and a custom snapshot format (not `.sqlite` files)
10
11
 
11
- See [COMPATIBILITY.md](COMPATIBILITY.md) for the matrix and [COMPATIBILITY-AUDIT.md](COMPATIBILITY-AUDIT.md) for the audit report.
12
+ See [COMPATIBILITY.md](COMPATIBILITY.md) for the matrix and [COMPATIBILITY-AUDIT.md](COMPATIBILITY-AUDIT.md) for the audit report. Agents contributing to this repo: start with [AGENTS.md](AGENTS.md).
13
+
14
+ ## Documentation
15
+
16
+ | Doc | For |
17
+ | --- | --- |
18
+ | [README.md](README.md) | Install, API, pitfalls (this file) |
19
+ | [AGENTS.md](AGENTS.md) | Architecture, how to change code, test/compat gates |
20
+ | [COMPATIBILITY.md](COMPATIBILITY.md) | Feature matrix + verify commands |
21
+ | [COMPATIBILITY-AUDIT.md](COMPATIBILITY-AUDIT.md) | Audit evidence |
22
+ | [docs/SECRETS.md](docs/SECRETS.md) | npm / CI publish setup |
23
+ | [benchmarks/PERFORMANCE.md](benchmarks/PERFORMANCE.md) | Performance notes |
12
24
 
13
25
  ## Install
14
26
 
@@ -18,6 +30,8 @@ bun add @crvouga/sqlite-mem
18
30
  npm install @crvouga/sqlite-mem
19
31
  ```
20
32
 
33
+ Requires Node.js ≥ 20 or Bun ≥ 1.1. The published package is **ESM only** (`import` from `@crvouga/sqlite-mem`).
34
+
21
35
  ## Usage
22
36
 
23
37
  ```ts
@@ -32,7 +46,7 @@ db.exec(`
32
46
  )
33
47
  `);
34
48
 
35
- db.exec(`INSERT INTO users (name) VALUES (?)`, ["Alice"]);
49
+ db.prepare(`INSERT INTO users (name) VALUES (?)`).run("Alice");
36
50
 
37
51
  const users = db.query<{ id: number; name: string }>(`SELECT * FROM users`);
38
52
  console.log(users);
@@ -42,36 +56,132 @@ const db2 = new Database();
42
56
  db2.restore(snap);
43
57
  ```
44
58
 
59
+ All methods are **synchronous** — do not `await` them. Browser and Node/Bun share the same in-memory JS surface (no filesystem; `ATTACH` opens a new empty in-memory schema, not a file).
60
+
61
+ ## Example
62
+
63
+ A React + Vite SQL playground lives in [`examples/react-vite`](examples/react-vite):
64
+
65
+ ```bash
66
+ cd examples/react-vite
67
+ bun install
68
+ bun run dev
69
+ ```
70
+
71
+ From the repo root after that install: `bun run example`.
72
+
45
73
  ## API
46
74
 
47
75
  ```ts
76
+ import { Database, SqliteError } from "@crvouga/sqlite-mem";
77
+
78
+ interface DatabaseOptions {
79
+ seed?: number | bigint; // default 1 — PRNG for random() / randomblob()
80
+ now?: Date | (() => Date); // default 2000-01-01T00:00:00.000Z
81
+ }
82
+
48
83
  interface Database {
49
- // seed default 1; now default 2000-01-01T00:00:00.000Z
50
- constructor(options?: { seed?: number | bigint; now?: Date | (() => Date) });
51
- exec(sql: string, params?: unknown[]): void;
52
- query<T = Record<string, unknown>>(sql: string, params?: unknown[]): T[];
84
+ constructor(options?: DatabaseOptions);
85
+ exec(sql: string): void;
86
+ query<T = QueryRow>(sql: string, params?: BindValue[]): T[];
53
87
  prepare(sql: string): Statement;
54
88
  transaction<T>(fn: () => T): T;
55
89
  snapshot(): Uint8Array;
56
90
  restore(snapshot: Uint8Array): void;
57
91
  close(): void;
92
+ [Symbol.dispose]?(): void; // alias for close() when Symbol.dispose exists
93
+ readonly changes: number;
94
+ readonly lastInsertRowid: number | bigint;
58
95
  }
59
96
 
60
97
  interface Statement {
61
- bind(...params: unknown[]): Statement;
62
- run(...params: unknown[]): { changes: number; lastInsertRowid: number | bigint };
63
- all<T = Record<string, unknown>>(...params: unknown[]): T[];
64
- get<T = Record<string, unknown>>(...params: unknown[]): T | undefined;
98
+ run(...params: BindValue[]): RunResult;
99
+ all<T = QueryRow>(...params: BindValue[]): T[];
100
+ get<T = QueryRow>(...params: BindValue[]): T | undefined;
101
+ result(...params: BindValue[]): ResultSet; // includes columns + values when zero rows
102
+ }
103
+
104
+ interface RunResult {
105
+ changes: number;
106
+ lastInsertRowid: number | bigint;
107
+ }
108
+
109
+ interface ResultSet {
110
+ columns: string[];
111
+ rows: QueryRow[];
112
+ values: QueryValue[][]; // always present (empty array for zero rows)
113
+ changes: number;
114
+ lastInsertRowid: number | bigint;
65
115
  }
116
+
117
+ class SqliteError extends Error {
118
+ readonly category: ErrorCategory; // syntax, no_such_table, constraint_unique, misuse, …
119
+ readonly sqliteCode: string; // always set; default "SQLITE_ERROR"
120
+ readonly code: string; // === sqliteCode (Node err.code convention)
121
+ }
122
+ ```
123
+
124
+ Stick to `Database`, `Statement`, and `SqliteError` for application code. Advanced internals (`parse`, `tokenize`, `evalExpr`, snapshot codec pieces, `SqlValue` utilities, `Prng`, …) are available only from `@crvouga/sqlite-mem/unstable` and are **exempt from semver**.
125
+
126
+ ### Method semantics
127
+
128
+ | Method | Behavior |
129
+ | --- | --- |
130
+ | `exec(sql)` | Runs all semicolon-separated statements; **discards** row results (`void`). Does **not** accept bind parameters. Read `db.changes` / `db.lastInsertRowid` afterward if needed (counters reflect the **most recent** completed statement, matching SQLite). |
131
+ | `query(sql, params?)` | **Single statement only** (trailing `;` is fine). Returns all rows. Multi-statement scripts throw `misuse`. |
132
+ | `prepare(sql)` | **Single statement only**. Parses immediately; AST is reused. Pass binds as rest args to `run` / `all` / `get` / `result` on each call. |
133
+ | `transaction(fn)` | If idle: `BEGIN` → `fn()` → `COMMIT`, or `ROLLBACK` + rethrow. If already in a transaction: nested savepoint. Nested SQL `BEGIN` still errors. `close()` inside `fn` throws `misuse`. |
134
+ | `snapshot` / `restore` | Custom binary format (see below). |
135
+ | `close()` | Idempotent; rolls back an open SQL transaction; further ops throw `misuse`. Also available as `[Symbol.dispose]` when supported. |
136
+
137
+ SQL `BEGIN` / `COMMIT` / `ROLLBACK` / `SAVEPOINT` / `RELEASE` are first-class. Empty / comment-only SQL on `prepare` / `query` throws `misuse` (`empty statement`), matching SQLite prepare failure.
138
+
139
+ ### Parameter binding
140
+
141
+ Supported styles: `?`, `?NNN`, `:name`, `@name`, `$name`.
142
+
143
+ - The JS API takes **rest args** (or a positional array into `query`) only — there is **no** sticky `bind()` and **no** `bind({ name: value })`.
144
+ - Named parameters occupy slots in **first-occurrence order**; repeated names share one slot.
145
+ - Prefixes are part of the name: `@x`, `$x`, and `:x` are **three different** parameters.
146
+ - Names are lowercased for lookup (`:Left` ≡ `:left`).
147
+ - Bindable: `null`, `string`, finite `number`, `bigint`, `boolean` → `0`/`1`, `Uint8Array` / `ArrayBuffer`.
148
+ - Rejected (`misuse`): `DataView`, typed-array views other than `Uint8Array`, `SharedArrayBuffer` / SAB-backed buffers.
149
+ - Rejected (`datatype_mismatch`): `undefined`, `Date`, plain objects, `NaN` / `Infinity`.
150
+
151
+ ```ts
152
+ db.query(`SELECT ? AS a, :name AS b`, [1, "Alice"]);
153
+ db.prepare(`SELECT @id AS id`).get(42);
66
154
  ```
67
155
 
156
+ ### Returned JavaScript types
157
+
158
+ | SQL storage | JS value | Notes |
159
+ | --- | --- | --- |
160
+ | NULL | `null` | Never `undefined` |
161
+ | INTEGER | `number` or `bigint` | `bigint` when outside `Number.MAX_SAFE_INTEGER` |
162
+ | REAL | `number` | Including integer-valued reals (`1.0` → `1`); use SQL `typeof()` to distinguish from INTEGER |
163
+ | TEXT | `string` | JSON subtype unwrapped to string |
164
+ | BLOB | `Uint8Array` | |
165
+
166
+ Duplicate column names collapse in row objects (last write wins). Use `stmt.result().values` for positional cells.
167
+
168
+ ### Snapshots
169
+
170
+ - Format magic `SQLM` followed by an explicit little-endian format-version `u32` — **not** a portable `.sqlite` file and not loadable by the SQLite CLI.
171
+ - Round-trips ordinary tables, views, indexes, change counters, PRNG state, and clock.
172
+ - **Not** encoded: triggers, ATTACH’d schemas, virtual tables (FTS / RTREE / …), `userVersion`.
173
+ - Cannot `restore()` while a transaction is open.
174
+ - `restore()` replaces `now` with a fixed clock from the snapshot (a live `() => Date` is overwritten).
175
+ - Equivalent databases produce byte-identical snapshots (schema/rows sorted) **within a single library version**.
176
+ - **Compatibility policy:** newer library versions can always restore older snapshots; older libraries cannot restore newer format versions (`snapshot_version` / `SQLITE_FORMAT`). Corrupt magic yields a distinct error.
177
+
68
178
  ## Determinism
69
179
 
70
180
  The engine is deterministic by default. Invariants:
71
181
 
72
182
  | Source | Default | Override / notes |
73
183
  | --- | --- | --- |
74
- | `random()` / `randomblob()` | Seeded xorshift64* (`seed: 1`) | `new Database({ seed })` or `{ prng }` |
184
+ | `random()` / `randomblob()` | Seeded xorshift64* (`seed: 1`) | `new Database({ seed })` |
75
185
  | `date('now')` / friends | Fixed `2000-01-01T00:00:00.000Z` | `new Database({ now: Date \| (() => Date) })` |
76
186
  | Table scans | Rowid order | Same order after `snapshot`/`restore` |
77
187
  | Snapshots | Sorted schema/rows + PRNG state + clock | Restored into PRNG and `now` |
@@ -86,12 +196,53 @@ SQLITE_MEM_FUZZ_SEED=12345 bun test tests/fuzz
86
196
  SQLITE_MEM_FUZZ_SEED=12345 SQLITE_MEM_FUZZ_PATH='0:1' bun test tests/fuzz # exact replay
87
197
  ```
88
198
 
199
+ ## Stability policy
200
+
201
+ The exports of the main entry (`@crvouga/sqlite-mem`) are **frozen**:
202
+
203
+ - **Never** outside a major: removals, renames, signature changes, or changes to documented behavior of the stable surface.
204
+ - **Allowed in minors:** additions (new methods, new optional `DatabaseOptions` fields, new `ErrorCategory` values). Consumers that `switch` on `category` must include a default case — new categories may appear without a major bump.
205
+ - **`@crvouga/sqlite-mem/unstable`** is exempt from semver and may change or disappear in any release.
206
+ - **Snapshots:** newer library → can restore older blobs; older library → cannot restore newer format versions; byte-identical snapshot guarantee holds only within one library version.
207
+
208
+ ## Compatibility notes for integrators
209
+
210
+ Goal: drop-in SQL behavior vs SQLite **3.51.0**. Full matrix: [COMPATIBILITY.md](COMPATIBILITY.md).
211
+
212
+ **Intentional differences:** custom `SQLM` snapshots; seeded `random()` / fixed `'now'`; no C API / on-disk DB / VFS.
213
+
214
+ **Know these thin or partial areas** (do not assume full oracle fidelity):
215
+
216
+ - FTS3/4/5 — largely implemented; shadow-table change counters intentionally diverge; some edges partial
217
+ - `EXPLAIN` / `EXPLAIN QUERY PLAN` — stub shapes, not real bytecode
218
+ - `INDEXED BY` / `NOT INDEXED` — parsed and discarded
219
+ - Unknown `PRAGMA` succeeds with an empty result (SQLite-like); storage/journal/WAL pragmas N/A or no-op
220
+
221
+ ## Common pitfalls
222
+
223
+ 1. **Do not `await`** — the API is sync.
224
+ 2. **No named-object binds and no sticky `bind()`** — pass positional rest args / arrays in declaration order to `query` / `run` / `all` / `get` / `result`.
225
+ 3. **`query` / `prepare` are single-statement only** — multi-statement scripts belong in `exec()` (which does not take bind parameters).
226
+ 4. **`exec` returns `void` and takes no params** — use `db.prepare(…).run(…)` or `db.query(…)` for binds; use `db.changes` / `stmt.run()` for counters.
227
+ 5. **`'now'` is not wall-clock** unless you pass `{ now: () => new Date() }`. Default is year 2000.
228
+ 6. **`random()` is seeded**, not OS entropy; snapshots restore the PRNG.
229
+ 7. **Snapshots are not `.sqlite` files** and do not round-trip FTS / triggers / ATTACH.
230
+ 8. **No better-sqlite3 extras** — no `iterate`, `pluck`/`raw`, `safeIntegers` option, `pragma()` helper, `loadExtension`, or SQLite-file `serialize()`.
231
+ 9. **Do not bind `Date` objects** — store unixepoch integers or ISO text. Do not bind `DataView` / non-`Uint8Array` typed arrays.
232
+ 10. **Do not use `Number.isInteger` for SQL REAL vs INTEGER** — use SQL `typeof()`.
233
+ 11. **Do not import `@crvouga/sqlite-mem/unstable` in application code** unless you accept breakage in any release.
234
+
235
+ Working examples beyond this README: `examples/react-vite`, `tests/contract/api/`, `tests/contract/parameters/`, `tests/browser/run.ts`.
236
+
89
237
  ## Development
90
238
 
91
- Requires [Bun](https://bun.sh).
239
+ Requires [Bun](https://bun.sh). For architecture, change checklists, and how to add contract tests, see **[AGENTS.md](AGENTS.md)**.
240
+
241
+ Parity is proven only by differential contracts against real SQLite (`bun:sqlite`). Isolated internal unit tests are not SQLite compatibility proof.
92
242
 
93
243
  ```bash
94
244
  bun install
245
+ bun run ci:local # same gates as GitHub Actions CI (except publish)
95
246
  bun run check # format + lint + typecheck + sqlite-compat suite
96
247
  bun run format # write Biome formatting
97
248
  bun run lint # Biome lint
@@ -99,10 +250,10 @@ bun run typecheck
99
250
  bun run test:sqlite-compat # requirements + inventory gate + differential suite
100
251
  bun test # contract + fuzz + harness
101
252
  bun run build
102
- bun run test:browser # Playwright smoke (Chrome/Firefox/Safari)
253
+ bun run test:browser # Playwright smoke (Chrome/Firefox/Safari; CI uses Chromium)
103
254
  ```
104
255
 
105
- Contract tests compare the pure TypeScript engine against real SQLite (`bun:sqlite`). See [COMPATIBILITY.md](./COMPATIBILITY.md).
256
+ See [COMPATIBILITY.md](./COMPATIBILITY.md).
106
257
 
107
258
  ## Releasing
108
259
 
@@ -137,8 +288,7 @@ PR titles must also follow Conventional Commits (enforced in CI). Prefer squash
137
288
  Local checks:
138
289
 
139
290
  ```bash
140
- bunx commitlint --last --verbose
141
- bun run build && bun run verify-package
291
+ bun run ci:local # commitlint + quality + tests + browser + benchmarks
142
292
  # dry-run needs a GitHub token for API calls; CI publish uses Trusted Publishing (no NPM_TOKEN)
143
293
  bun run release:dry-run
144
294
  ```