@crvouga/mockingbird-service-sqlite 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +170 -0
- package/COMPATIBILITY-AUDIT.md +236 -0
- package/COMPATIBILITY.md +102 -0
- package/LICENSE +21 -0
- package/README.md +405 -0
- package/compat/coverage.json +22231 -0
- package/compat/divergences.json +145 -0
- package/compat/fts-oracle-surface.json +198 -0
- package/compat/requirements.json +20929 -0
- package/compat/scenario-types.ts +63 -0
- package/compat/scenarios.ts +871 -0
- package/compat/smoke-baseline.json +3 -0
- package/compat/sqllogictest-skip.json +5 -0
- package/dist/api/database.d.ts +112 -0
- package/dist/api/snapshot.d.ts +21 -0
- package/dist/api/statement.d.ts +75 -0
- package/dist/ast/nodes.d.ts +476 -0
- package/dist/constraints/check.d.ts +11 -0
- package/dist/errors/index.d.ts +45 -0
- package/dist/executor/attach.d.ts +5 -0
- package/dist/executor/ddl.d.ts +10 -0
- package/dist/executor/dml.d.ts +7 -0
- package/dist/executor/env.d.ts +66 -0
- package/dist/executor/execute.d.ts +4 -0
- package/dist/executor/pragma-engine.d.ts +31 -0
- package/dist/executor/pragma.d.ts +4 -0
- package/dist/executor/result.d.ts +22 -0
- package/dist/executor/select.d.ts +7 -0
- package/dist/executor/simple-select.d.ts +9 -0
- package/dist/executor/triggers.d.ts +11 -0
- package/dist/executor/vtable.d.ts +12 -0
- package/dist/expressions/context.d.ts +30 -0
- package/dist/expressions/equals.d.ts +3 -0
- package/dist/expressions/eval.d.ts +12 -0
- package/dist/expressions/index.d.ts +3 -0
- package/dist/expressions/like.d.ts +16 -0
- package/dist/functions/aggregate.d.ts +8 -0
- package/dist/functions/datetime.d.ts +2 -0
- package/dist/functions/extensions.d.ts +7 -0
- package/dist/functions/index.d.ts +8 -0
- package/dist/functions/json.d.ts +6 -0
- package/dist/functions/math.d.ts +2 -0
- package/dist/functions/pragma-tvf.d.ts +8 -0
- package/dist/functions/registry.d.ts +39 -0
- package/dist/functions/scalar.d.ts +9 -0
- package/dist/functions/table-valued-registry.d.ts +11 -0
- package/dist/functions/table-valued.d.ts +13 -0
- package/dist/functions/window.d.ts +23 -0
- package/dist/index.d.ts +28 -0
- package/dist/index.js +16010 -0
- package/dist/index.js.map +7 -0
- package/dist/indexes/index.d.ts +48 -0
- package/dist/indexes/keys.d.ts +15 -0
- package/dist/json/index.d.ts +7 -0
- package/dist/json/jsonb.d.ts +42 -0
- package/dist/json/ops.d.ts +24 -0
- package/dist/json/parse.d.ts +13 -0
- package/dist/json/path.d.ts +26 -0
- package/dist/json/stringify.d.ts +4 -0
- package/dist/json/tvf.d.ts +13 -0
- package/dist/json/types.d.ts +29 -0
- package/dist/lexer/tokenize.d.ts +23 -0
- package/dist/parser/index.d.ts +17 -0
- package/dist/parser/parser.d.ts +115 -0
- package/dist/planner/access.d.ts +50 -0
- package/dist/planner/index.d.ts +3 -0
- package/dist/runtime/assert.d.ts +11 -0
- package/dist/runtime/catch.d.ts +3 -0
- package/dist/runtime/clock.d.ts +14 -0
- package/dist/runtime/index.d.ts +3 -0
- package/dist/runtime/options.d.ts +25 -0
- package/dist/runtime/prng.d.ts +42 -0
- package/dist/schema/catalog.d.ts +14 -0
- package/dist/schema/master-sql.d.ts +10 -0
- package/dist/serialization/codec.d.ts +19 -0
- package/dist/serialization/index.d.ts +1 -0
- package/dist/serialization/wire.d.ts +61 -0
- package/dist/storage/columnar-slab.d.ts +58 -0
- package/dist/storage/database-state.d.ts +96 -0
- package/dist/storage/index.d.ts +6 -0
- package/dist/storage/row.d.ts +13 -0
- package/dist/storage/table.d.ts +119 -0
- package/dist/transactions/manager.d.ts +19 -0
- package/dist/types/collation.d.ts +4 -0
- package/dist/types/sqlite-atof.d.ts +4 -0
- package/dist/types/sqlite-real-format.d.ts +5 -0
- package/dist/types/strict.d.ts +6 -0
- package/dist/types/value.d.ts +88 -0
- package/dist/unstable.d.ts +20 -0
- package/dist/unstable.js +10343 -0
- package/dist/unstable.js.map +7 -0
- package/dist/vtable/fts/options.d.ts +34 -0
- package/dist/vtable/fts/porter.d.ts +2 -0
- package/dist/vtable/fts/query.d.ts +39 -0
- package/dist/vtable/fts/table.d.ts +78 -0
- package/dist/vtable/fts/tokenize.d.ts +20 -0
- package/dist/vtable/fts5.d.ts +1 -0
- package/dist/vtable/index.d.ts +2 -0
- package/dist/vtable/modules.d.ts +65 -0
- package/package.json +122 -0
package/AGENTS.md
ADDED
|
@@ -0,0 +1,170 @@
|
|
|
1
|
+
# AGENTS.md β contributing to sqlite-mem
|
|
2
|
+
|
|
3
|
+
Guidance for humans and coding agents editing this repository. For install and consumer API, see [README.md](README.md). For the feature matrix, see [COMPATIBILITY.md](COMPATIBILITY.md).
|
|
4
|
+
|
|
5
|
+
## Mission and non-goals
|
|
6
|
+
|
|
7
|
+
**Mission:** full SQLite3 **SQL dialect** behavioral parity vs **SQLite 3.51.0** (`bun:sqlite`). Same statements β same observable results (rows, errors, counters), proven by differential contracts and a fail-closed gate.
|
|
8
|
+
|
|
9
|
+
**Allowed intentional differences:**
|
|
10
|
+
|
|
11
|
+
1. Custom snapshot codec (`SQLM`), not on-disk `.sqlite` files
|
|
12
|
+
2. Deterministic `random()` / `randomblob()` and fixed `'now'` by default (injectable)
|
|
13
|
+
3. `NOT APPLICABLE` items: C API, VFS/pager/WAL, file locking, URI filenames
|
|
14
|
+
|
|
15
|
+
**Non-goals:** shipping a SQLite file format, native/WASM bindings, or matching better-sqlite3βs full Node API surface.
|
|
16
|
+
|
|
17
|
+
## SQL pipeline
|
|
18
|
+
|
|
19
|
+
```
|
|
20
|
+
SQL string
|
|
21
|
+
β tokenize() src/lexer/tokenize.ts
|
|
22
|
+
β parse() src/parser/index.ts β parser.ts
|
|
23
|
+
β Statement[] AST src/ast/nodes.ts
|
|
24
|
+
β Statement.execute() src/api/statement.ts
|
|
25
|
+
β executeStatement() src/executor/execute.ts
|
|
26
|
+
β per-stmt executor select / dml / ddl / β¦
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
Public entry points: `Database.exec` / `query` / `prepare` in [`src/api/database.ts`](src/api/database.ts).
|
|
30
|
+
|
|
31
|
+
**Planner note:** [`src/planner/index.ts`](src/planner/index.ts) `preparePlan` is an **identity stub** (`Plan === Statement`) and is unused. Runtime index/PK access paths live in [`src/planner/access.ts`](src/planner/access.ts) (`tryIndexedTableRows`, `lookupTableRows`, `tryJoinProbe`), called from the executor.
|
|
32
|
+
|
|
33
|
+
SELECT has a **fast path** ([`src/executor/simple-select.ts`](src/executor/simple-select.ts)) and a **full path** ([`src/executor/select.ts`](src/executor/select.ts)). Fast paths return `null` when the shape is unsupported; then the full path runs. Changing join/WHERE/expression semantics in only one path will silently diverge.
|
|
34
|
+
|
|
35
|
+
## `src/` map
|
|
36
|
+
|
|
37
|
+
| Directory | Role |
|
|
38
|
+
| --- | --- |
|
|
39
|
+
| `api/` | Public `Database` / `Statement` facade |
|
|
40
|
+
| `ast/` | Discriminated-union AST (`nodes.ts`) |
|
|
41
|
+
| `lexer/` | Tokenizer |
|
|
42
|
+
| `parser/` | Recursive-descent parser |
|
|
43
|
+
| `planner/` | Access-path helpers (not a logical plan IR) |
|
|
44
|
+
| `executor/` | Statement dispatch, SELECT, DML, DDL, PRAGMA, triggers, ATTACH |
|
|
45
|
+
| `expressions/` | `evalExpr`, LIKE/GLOB, eval context |
|
|
46
|
+
| `functions/` | Scalar / aggregate / window / datetime / JSON / math / TVF registries |
|
|
47
|
+
| `types/` | Engine `SqlValue`, affinity, collation |
|
|
48
|
+
| `storage/` | In-memory tables, rows, `DatabaseState` |
|
|
49
|
+
| `indexes/` | Equality index store |
|
|
50
|
+
| `schema/` | `sqlite_master` / catalog |
|
|
51
|
+
| `constraints/` | NOT NULL / PK / UNIQUE / CHECK |
|
|
52
|
+
| `transactions/` | BEGIN / COMMIT / SAVEPOINT (clones state + PRNG) |
|
|
53
|
+
| `runtime/` | Clock, PRNG, `DatabaseOptions` |
|
|
54
|
+
| `serialization/` | `SQLM` snapshot codec |
|
|
55
|
+
| `json/` | JSON1 / JSONB internals |
|
|
56
|
+
| `vtable/` | FTS5 and other virtual-table modules |
|
|
57
|
+
| `errors/` | `SqliteError`, `unsupported()` |
|
|
58
|
+
|
|
59
|
+
Hot / large files: `parser/parser.ts`, `executor/select.ts`, `executor/dml.ts`.
|
|
60
|
+
|
|
61
|
+
## Critical conventions
|
|
62
|
+
|
|
63
|
+
- **AST `type` tags** are snake_case (`"create_table"`, `"drop_index"`). TypeScript interfaces are PascalCase (`CreateTableStmt`).
|
|
64
|
+
- **Identifiers** are case-folded (`toLowerCase`) as Map keys in storage and function registries.
|
|
65
|
+
- **Engine `SqlValue`** ([`src/types/value.ts`](src/types/value.ts)) may include `SqlReal` / `SqlJsonText`. **Harness `SqlValue`** ([`tests/harness/types.ts`](tests/harness/types.ts)) is the normalized compare type β do not confuse them.
|
|
66
|
+
- **API `Statement`** vs **AST `Statement`**: the API class aliases the AST union as `AstStatement`.
|
|
67
|
+
- **`Database`** (API) vs **`DatabaseState`** (engine storage).
|
|
68
|
+
- Throw **`SqliteError`** with an `ErrorCategory`. Missing SQL must fail loud via `unsupported()` β the inventory gate fails if the oracle exposes an unimplemented builtin/module.
|
|
69
|
+
- Fast-path helpers (`tryExecuteSimpleSelect`, `tryFastInsert`, `tryIndexedTableRows`, β¦): return `null` β fall through. Update **both** paths when semantics change.
|
|
70
|
+
- **Dual-path invariant:** residual `WHERE` filtering is skipped on an index/hash access path only when `whereFullyCovered(where, coveredColumns, β¦)` is true (see [`src/planner/access.ts`](src/planner/access.ts)). Any change to WHERE/join/insert semantics requires updating both fast and full paths **and** the fastβ‘full property tests under `tests/fuzz/fast-path.test.ts`.
|
|
71
|
+
- Internal invariants use always-on asserts in [`src/runtime/assert.ts`](src/runtime/assert.ts) (`assert`, `assertUnreachable`, `assertRowShape`).
|
|
72
|
+
- TypeScript: `strict` + `noUncheckedIndexedAccess`. Imports use `.ts` extensions. Biome: 2-space, double quotes, 120 columns.
|
|
73
|
+
- IEEE `-0` is canonicalized to `+0` on bind, affinity, and arithmetic. Keep determinism invariants (see README).
|
|
74
|
+
|
|
75
|
+
## Change checklists
|
|
76
|
+
|
|
77
|
+
### New SQL statement
|
|
78
|
+
|
|
79
|
+
1. Add union member + interface in [`src/ast/nodes.ts`](src/ast/nodes.ts)
|
|
80
|
+
2. Parse in [`src/parser/parser.ts`](src/parser/parser.ts) (`parseStatement` dispatch)
|
|
81
|
+
3. Handle in [`src/executor/execute.ts`](src/executor/execute.ts) (+ `ddl.ts` / `dml.ts` / `vtable.ts` as needed)
|
|
82
|
+
4. Mutate `DatabaseState` if schema changes
|
|
83
|
+
5. Add differential contract under `tests/contract/<area>/`
|
|
84
|
+
6. Optionally add evidence paths in `SOURCE_SEED` in [`scripts/sqlite-requirements.ts`](scripts/sqlite-requirements.ts), then `bun run requirements`
|
|
85
|
+
|
|
86
|
+
### New SQL function
|
|
87
|
+
|
|
88
|
+
1. Implement and register in the right map under `src/functions/*` (`getScalarFunctions()`, datetime, JSON, etc.) so [`scripts/sqlite-inventory.ts`](scripts/sqlite-inventory.ts) sees it
|
|
89
|
+
2. Contract tests under `tests/contract/functions/` (and related areas)
|
|
90
|
+
3. Run `bun run inventory` / `bun run test:sqlite-compat` β oracle builtins must not be missing
|
|
91
|
+
|
|
92
|
+
### New contract test
|
|
93
|
+
|
|
94
|
+
1. Prefer helpers in [`tests/contract/helpers.ts`](tests/contract/helpers.ts):
|
|
95
|
+
- `parity` β query both engines
|
|
96
|
+
- `execParity` β writes
|
|
97
|
+
- `sequenceParity` β multi-step (optional final-state compare)
|
|
98
|
+
- `errorParity` / `queryErrorParity` β both must fail
|
|
99
|
+
- `ftsRankParity` β REAL epsilon compare
|
|
100
|
+
2. Or `matrixBoth` + `expectParity` from `tests/harness/`
|
|
101
|
+
3. **Do not** treat isolated internal unit tests as SQLite proof. The differential suite is authoritative.
|
|
102
|
+
4. Gate: `bun run test:sqlite-compat`
|
|
103
|
+
|
|
104
|
+
## Test layout
|
|
105
|
+
|
|
106
|
+
| Path | Role |
|
|
107
|
+
| --- | --- |
|
|
108
|
+
| `tests/contract/` | Differential SQL vs `bun:sqlite` (**authoritative**) |
|
|
109
|
+
| `tests/fuzz/` | fast-check property tests (seeded); same two backends |
|
|
110
|
+
| `tests/harness/` | Compare/normalize helpers + harness unit tests |
|
|
111
|
+
| `tests/adapters/` | Wrappers for sqlite-mem and `bun:sqlite` |
|
|
112
|
+
|
|
113
|
+
Examples of public API usage: `tests/contract/api/`, `tests/contract/parameters/`, `tests/contract/determinism/`, `examples/react-vite`.
|
|
114
|
+
|
|
115
|
+
### Fuzz replay
|
|
116
|
+
|
|
117
|
+
Default seed `0x5a17e0e1`. On failure the seed is printed:
|
|
118
|
+
|
|
119
|
+
```bash
|
|
120
|
+
bun test tests/fuzz
|
|
121
|
+
SQLITE_MEM_FUZZ_SEED=12345 bun test tests/fuzz
|
|
122
|
+
SQLITE_MEM_FUZZ_SEED=12345 SQLITE_MEM_FUZZ_PATH='0:1' bun test tests/fuzz
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
### Random walk (state-dependent DST)
|
|
126
|
+
|
|
127
|
+
`tests/fuzz/random-walk.test.ts` walks the SQL state space one enabled action at
|
|
128
|
+
a time and asserts result + logical-state parity after every step.
|
|
129
|
+
|
|
130
|
+
```bash
|
|
131
|
+
bun run test:walk
|
|
132
|
+
SQLITE_MEM_WALK_STEPS=80 bun run test:walk
|
|
133
|
+
bun run test:walk:soak -- --depth 200 --runs 20
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
## Compat system
|
|
137
|
+
|
|
138
|
+
| Command | Role |
|
|
139
|
+
| --- | --- |
|
|
140
|
+
| `bun run test:sqlite-compat` | Requirements + fail-closed gate + construct catalog + π + smoke ratchet + contract/fuzz/harness |
|
|
141
|
+
| `bun run inventory` | Oracle `pragma_function_list` / modules vs memory registries |
|
|
142
|
+
| `bun run scenarios` | Construct-level scenario catalog (`compat/scenarios.ts`) + π / smoke gates |
|
|
143
|
+
| `bun run requirements` | Refresh sqlite.org requirements β `compat/requirements.json` + `compat/coverage.json` |
|
|
144
|
+
| `bun run fts-surface` | FTS oracle surface β `compat/fts-oracle-surface.json` |
|
|
145
|
+
|
|
146
|
+
Statuses: **VERIFIED** / **PARTIALLY VERIFIED** / **UNSUPPORTED** / **NOT APPLICABLE**. Do not market PARTIAL as complete. Coverage evidence is directory paths (e.g. `tests/contract/joins/`), not automatic from test filenames.
|
|
147
|
+
|
|
148
|
+
**Catalog vs proof:** `tests/contract/catalog/` IDs must execute; smoke (`SELECT 1 AS v`) is tracked in `compat/smoke-baseline.json`. Documented divergences bind to `compat/divergences.json`. Generated operator/CAST matrices: `tests/contract/matrices/`. Stateful dump-after-each fuzz: `tests/fuzz/stateful.test.ts`. Oracle `sqlite_version()` must be 3.51.0 or 3.53.0.
|
|
149
|
+
|
|
150
|
+
Details: [COMPATIBILITY.md](COMPATIBILITY.md), audit: [COMPATIBILITY-AUDIT.md](COMPATIBILITY-AUDIT.md).
|
|
151
|
+
|
|
152
|
+
## Local gates
|
|
153
|
+
|
|
154
|
+
Requires [Bun](https://bun.sh).
|
|
155
|
+
|
|
156
|
+
```bash
|
|
157
|
+
bun install
|
|
158
|
+
bun run check:full # same gates as GitHub Actions CI (except publish)
|
|
159
|
+
bun run check # format + lint + typecheck + sqlite-compat suite
|
|
160
|
+
bun run format
|
|
161
|
+
bun run lint
|
|
162
|
+
bun run typecheck
|
|
163
|
+
bun run test:sqlite-compat
|
|
164
|
+
bun test # contract + fuzz + harness
|
|
165
|
+
bun run build
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
## PR and commits
|
|
169
|
+
|
|
170
|
+
Use [Conventional Commits](https://www.conventionalcommits.org/) for commits and PR titles (enforced on PRs). Prefer squash merges with a conventional title so the npm bump is `feat` β minor / `fix` β patch. Direct pushes to `main` with any other subject still publish a patch. See [README.md](README.md#releasing).
|
|
@@ -0,0 +1,236 @@
|
|
|
1
|
+
# SQLite3 Compatibility Audit
|
|
2
|
+
|
|
3
|
+
```text
|
|
4
|
+
SQLite3 Compatibility Audit
|
|
5
|
+
===========================
|
|
6
|
+
|
|
7
|
+
Reference SQLite version:
|
|
8
|
+
bun:sqlite (SQLite 3.51.0)
|
|
9
|
+
|
|
10
|
+
Reference SQLite compile options:
|
|
11
|
+
ENABLE_MATH_FUNCTIONS, ENABLE_FTS3/4/5, ENABLE_RTREE, ENABLE_DBSTAT_VTAB,
|
|
12
|
+
ENABLE_BYTECODE_VTAB, JSON builtin (not OMIT_JSON), plus Bun/SEE codec flags.
|
|
13
|
+
Full list: `bun run inventory`
|
|
14
|
+
|
|
15
|
+
sqlite-mem version:
|
|
16
|
+
0.0.0-development (package.json; released from the monorepo via `<name>@<version>` tags)
|
|
17
|
+
|
|
18
|
+
Scope:
|
|
19
|
+
Scope 3 β every oracle-exposed SQL builtin/module is in-scope except
|
|
20
|
+
NOT APPLICABLE (C API, on-disk format, VFS/pager/WAL).
|
|
21
|
+
|
|
22
|
+
SQLite requirements reviewed:
|
|
23
|
+
3487 (from sqlite.org requirements matrix β compat/requirements.json)
|
|
24
|
+
|
|
25
|
+
SQLite requirements classification:
|
|
26
|
+
NOT APPLICABLE: see compat/coverage.json counts.notApplicable
|
|
27
|
+
SQL_BEHAVIOR: see counts.sqlBehavior
|
|
28
|
+
unknown: 0 (gate fails if non-zero)
|
|
29
|
+
|
|
30
|
+
Coverage statuses (SQL_BEHAVIOR):
|
|
31
|
+
VERIFIED / PARTIALLY_VERIFIED / UNSUPPORTED β see `bun run requirements`
|
|
32
|
+
Regenerated via scripts/sqlite-requirements.ts + coverage upgrades.
|
|
33
|
+
|
|
34
|
+
SQL grammar / operators / expressions:
|
|
35
|
+
VERIFIED β contracts + row-value comparisons + ->/->> precedence
|
|
36
|
+
|
|
37
|
+
Types / affinity / NULL:
|
|
38
|
+
VERIFIED β affinity matrix, typeof/REAL (incl. unary Β± float literals),
|
|
39
|
+
truthiness/3VL, JS-leak bind probes (differential + documented divergences)
|
|
40
|
+
|
|
41
|
+
Functions (oracle surface):
|
|
42
|
+
VERIFIED β inventory missingOracleFunctions = 0 (163 names covered)
|
|
43
|
+
|
|
44
|
+
JSON / JSONB:
|
|
45
|
+
VERIFIED
|
|
46
|
+
|
|
47
|
+
Aggregates / windows:
|
|
48
|
+
VERIFIED β includes string_agg, ntile, cume_dist, percent_rank, EXCLUDE;
|
|
49
|
+
unaliased aggregate column names (`sum(v)`, `count(*)`) match oracle
|
|
50
|
+
|
|
51
|
+
CTEs / transactions / savepoints / constraints / FK / triggers:
|
|
52
|
+
VERIFIED (deferred FK + composite FK + MATCH SIMPLE/FULL + statement ABORT)
|
|
53
|
+
|
|
54
|
+
Indexes / views / generated / STRICT / WITHOUT ROWID:
|
|
55
|
+
VERIFIED β partial/expression indexes, STRICT tables, leftmost prefix;
|
|
56
|
+
WITHOUT ROWID UPSERT; generated STORED/VIRTUAL WHERE/ORDER BY
|
|
57
|
+
|
|
58
|
+
PRAGMAs:
|
|
59
|
+
PARTIALLY VERIFIED β schema/FK; storage pragmas N/A or :memory: no-op
|
|
60
|
+
|
|
61
|
+
ATTACH/DETACH:
|
|
62
|
+
VERIFIED (in-memory schemas; temp.schema = main state)
|
|
63
|
+
|
|
64
|
+
Virtual tables / extensions:
|
|
65
|
+
VERIFIED for oracle modules: fts3/4/5, fts5vocab, rtree/rtree_i32,
|
|
66
|
+
dbstat, bytecode, tables_used (bytecode/tables_used empty cursors)
|
|
67
|
+
|
|
68
|
+
ANALYZE / REINDEX / VACUUM:
|
|
69
|
+
VERIFIED (:memory: observable parity)
|
|
70
|
+
|
|
71
|
+
Prepared statements / errors / snapshot:
|
|
72
|
+
VERIFIED β schema invalidation re-prepares; SQLM logical round-trip VERIFIED;
|
|
73
|
+
post-restore A/B lockstep + dumpLogicalState vs oracle
|
|
74
|
+
|
|
75
|
+
Differential tests (2026-08-21 audit refresh):
|
|
76
|
+
Total: 1912 under `bun test` / `bun run test:sqlite-compat`
|
|
77
|
+
(contract + fuzz + harness; Bun 1.4.0, SQLite 3.51.0)
|
|
78
|
+
Passed: 1912
|
|
79
|
+
Failed: 0
|
|
80
|
+
expect() calls: 7314
|
|
81
|
+
|
|
82
|
+
Stateful / fuzz:
|
|
83
|
+
Seeds: 0x5a17e0e1 (+ SQLITE_MEM_FUZZ_SEED override)
|
|
84
|
+
Combination fuzz: tests/fuzz/combinations-scope3.test.ts
|
|
85
|
+
FTS fuzz: tests/fuzz/fts.test.ts
|
|
86
|
+
Long deterministic scripts: tests/contract/transactions/stateful.test.ts
|
|
87
|
+
Collate / window / CTE+DML / JSON subtype / affinity / conflicts fuzz
|
|
88
|
+
Mismatches: 0
|
|
89
|
+
|
|
90
|
+
Harness integrity (Phase 1):
|
|
91
|
+
RealSqliteAdapter DML via prepare().run() with live changes/lastInsertRowid
|
|
92
|
+
sequenceParity compares DML counters; DDL/txn/PRAGMA allowlist neutralized
|
|
93
|
+
errorParity via deepCompareResults; dumpLogicalState + compareFinalState
|
|
94
|
+
Comparator meta-tests: tests/harness/normalize.test.ts
|
|
95
|
+
|
|
96
|
+
Hardening pass (2026-08-19) β incompatibilities found & fixed:
|
|
97
|
+
1. Comparison affinity (INTEGER/TEXT/NUMERIC column vs literal) missing
|
|
98
|
+
2. WITH on UPDATE/DELETE/INSERT VALUES dispatched as SELECT only
|
|
99
|
+
3. Plain INTEGER PRIMARY KEY never reused deleted max rowid
|
|
100
|
+
4. date() `weekday N` modifier unimplemented
|
|
101
|
+
5. REGEXP operator not parsed (now calls missing regexp() like oracle)
|
|
102
|
+
6. last_insert_rowid() stale inside AFTER INSERT triggers
|
|
103
|
+
7. INSTEAD OF triggers never fired; view DML rejected
|
|
104
|
+
8. INSERT/UPDATE OR ROLLBACK / OR FAIL not distinguished
|
|
105
|
+
9. Window GROUPS/RANGE frames, window FILTER, recursive CTE queue edges
|
|
106
|
+
10. Collation-aware GROUP BY; FTS3 matchinfo format variants; external-content delete
|
|
107
|
+
|
|
108
|
+
Audit refresh (2026-08-21) β additional findings & fixes:
|
|
109
|
+
11. sequenceParity previously ignored all DML write counters (hid divergences)
|
|
110
|
+
12. Unary +/- on float literals dropped REAL storage class (`typeof(-0.0)`)
|
|
111
|
+
13. UPDATE OR REPLACE counted conflict deletions in changes() (oracle = 1)
|
|
112
|
+
14. Empty-result column names + unaliased aggregate headers asserted vs oracle
|
|
113
|
+
15. JS bind edges: empty TEXT/BLOB, BigInt@MAX_SAFE, intentional Number/BigInt
|
|
114
|
+
bind-class divergences pinned via divergence()
|
|
115
|
+
16. Snapshot post-restore A/B + state dump vs live oracle
|
|
116
|
+
|
|
117
|
+
Remaining known differences / intentional:
|
|
118
|
+
Custom SQLM snapshots; deterministic random()/'now' by default
|
|
119
|
+
(`random: "os"` / `now: "system"` match SQLite entropy and wall clock);
|
|
120
|
+
EXPLAIN stubs (shape contracts only); INDEXED BY no-op (documented);
|
|
121
|
+
ATTACH file path records filename but opens empty in-memory schema;
|
|
122
|
+
some PRAGMA storage no-ops; FTS shadow-table changes()/total_changes diverge
|
|
123
|
+
(sequenceParity neutralizeCounters on FTS maintenance paths);
|
|
124
|
+
MATERIALIZED/NOT MATERIALIZED stored but both materialize today;
|
|
125
|
+
compile_options / function_list content is sqlite-mem's;
|
|
126
|
+
generate_series is sqlite-mem extension (not in bun:sqlite default);
|
|
127
|
+
Number.MAX_SAFE_INTEGER JS bind typeof (mem integer vs bun real);
|
|
128
|
+
BigInt beyond Number.MAX_SAFE_INTEGER without bun safeIntegers;
|
|
129
|
+
NOT APPLICABLE C API / on-disk / VFS surfaces; uri.html remapped N/A.
|
|
130
|
+
|
|
131
|
+
Final assessment:
|
|
132
|
+
Verified against SQLite 3.51.0 (bun:sqlite on Bun 1.4.0). Oracle
|
|
133
|
+
function/module inventory is closed (0 missing). Requirements matrix
|
|
134
|
+
ingested with zero unknown statuses. Gate: `bun run test:sqlite-compat`.
|
|
135
|
+
Harness now compares live DML counters; green means stronger evidence than
|
|
136
|
+
pre-2026-08-21. FTS/EXPLAIN remain PARTIALLY VERIFIED honestly β not
|
|
137
|
+
βfully compatible because green.β
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
---
|
|
141
|
+
|
|
142
|
+
## SQLite Full-Text Search Compatibility Audit
|
|
143
|
+
|
|
144
|
+
```text
|
|
145
|
+
SQLite Full-Text Search Compatibility Audit
|
|
146
|
+
==========================================
|
|
147
|
+
|
|
148
|
+
Reference SQLite:
|
|
149
|
+
version: 3.51.0 (bun:sqlite)
|
|
150
|
+
source_id: 2025-06-12 13:14:41 f0ca7bba1c5e232e5d279fad6338121ab55af0c8c68c84cdfb18ba5114dcaapl
|
|
151
|
+
compile options: ENABLE_FTS3, ENABLE_FTS3_PARENTHESIS, ENABLE_FTS3_TOKENIZER,
|
|
152
|
+
ENABLE_FTS4, ENABLE_FTS5
|
|
153
|
+
inventory: compat/fts-oracle-surface.json (bun run scripts/fts-oracle-surface.ts)
|
|
154
|
+
|
|
155
|
+
FTS3: PARTIALLY VERIFIED (MATCH, snippet, offsets; matchinfo formats thinner)
|
|
156
|
+
FTS4: PARTIALLY VERIFIED (same surface as FTS3 for tested paths)
|
|
157
|
+
FTS5: PARTIALLY VERIFIED overall β core MATCH/tokenizers/ranking/aux VERIFIED;
|
|
158
|
+
external-content + full matchinfo format strings still thinner
|
|
159
|
+
|
|
160
|
+
Tokenizers verified:
|
|
161
|
+
unicode61 (incl. remove_diacritics 0/1/2), ascii, porter,
|
|
162
|
+
porter unicode61, porter ascii, trigram
|
|
163
|
+
|
|
164
|
+
MATCH grammar verified:
|
|
165
|
+
terms, AND/OR/NOT, implicit AND, phrases, prefix *, column filters
|
|
166
|
+
(col : term / {cols} :), NEAR / NEAR(β¦, N), parentheses, NEAR-as-term
|
|
167
|
+
|
|
168
|
+
Ranking verified:
|
|
169
|
+
bm25() / rank column β order + scores vs oracle (1e-15 abs epsilon for ULP)
|
|
170
|
+
|
|
171
|
+
Auxiliary functions verified:
|
|
172
|
+
bm25, highlight, snippet (FTS5); snippet, offsets (FTS3/4);
|
|
173
|
+
matchinfo present (default format PARTIAL)
|
|
174
|
+
|
|
175
|
+
Content modes verified:
|
|
176
|
+
normal content tables: VERIFIED
|
|
177
|
+
contentless (content=''): VERIFIED
|
|
178
|
+
external content: PARTIALLY VERIFIED (CREATE accepted; sync/triggers thinner)
|
|
179
|
+
|
|
180
|
+
Special commands verified:
|
|
181
|
+
optimize, rebuild, integrity-check (success + post-command MATCH)
|
|
182
|
+
delete-all / merge / automerge: error parity with oracle where probed
|
|
183
|
+
|
|
184
|
+
Differential tests:
|
|
185
|
+
Passed: 872 (contract + fuzz + harness)
|
|
186
|
+
Failed: 0
|
|
187
|
+
FTS contract: tests/contract/fts/basic.test.ts,
|
|
188
|
+
tests/contract/fts/comprehensive.test.ts
|
|
189
|
+
(+ matchinfo formats, external-content, trigger maintenance)
|
|
190
|
+
|
|
191
|
+
Fuzz cases:
|
|
192
|
+
Generated: fast-check seed 0x5a17e0e1 (override SQLITE_MEM_FUZZ_SEED)
|
|
193
|
+
Files: tests/fuzz/fts.test.ts (MATCH queries, tokenizers, stateful DML)
|
|
194
|
+
Mismatches: 0 (after trigram phrase expansion + NEAR-as-term fixes)
|
|
195
|
+
|
|
196
|
+
Stateful cases:
|
|
197
|
+
Operations: INSERT/UPDATE/DELETE/MATCH sequences + txn/savepoint contracts
|
|
198
|
+
Mismatches: 0
|
|
199
|
+
|
|
200
|
+
Compatibility gaps found:
|
|
201
|
+
1. Toy AND-token matcher (replaced)
|
|
202
|
+
2. Aux functions always threw (wired MATCH cursor + real bm25/highlight/snippet)
|
|
203
|
+
3. No CREATE option parsing (tokenize=/content=/prefix=/UNINDEXED)
|
|
204
|
+
4. No FTS5 query language / NEAR / phrases / column filters
|
|
205
|
+
5. False VERIFIED status on fts5.html / fts3.html (downgraded then re-evidenced)
|
|
206
|
+
|
|
207
|
+
Compatibility gaps fixed:
|
|
208
|
+
Positional inverted index; unicode61/ascii/porter/trigram tokenizers;
|
|
209
|
+
FTS5 query parser; BM25 with SQLite IDF floor 1e-6; highlight/snippet;
|
|
210
|
+
contentless mode; special commands; FTS3/4 MATCH + snippet/offsets;
|
|
211
|
+
comparator realEpsilon for ranking ULP only; FTS fuzz suites
|
|
212
|
+
|
|
213
|
+
Regression tests added:
|
|
214
|
+
tests/contract/fts/comprehensive.test.ts
|
|
215
|
+
tests/fuzz/fts.test.ts
|
|
216
|
+
tests/harness/fts-compare.ts
|
|
217
|
+
scripts/fts-oracle-surface.ts
|
|
218
|
+
|
|
219
|
+
Remaining unsupported / thinner functionality:
|
|
220
|
+
- SQLite FTS shadow tables not mirrored (change counters diverge; documented)
|
|
221
|
+
- External-content sync/stale-index edge cases thinner
|
|
222
|
+
- FTS3 matchinfo format-string variants not exhaustively verified
|
|
223
|
+
- fts3tokenize / fts4aux still thin CREATE stubs
|
|
224
|
+
- locale / tokendata / detail=none advanced options accepted but lightly tested
|
|
225
|
+
```
|
|
226
|
+
|
|
227
|
+
Verification commands:
|
|
228
|
+
|
|
229
|
+
```bash
|
|
230
|
+
bun run test:sqlite-compat
|
|
231
|
+
bun run scripts/fts-oracle-surface.ts
|
|
232
|
+
bun test tests/contract/fts tests/fuzz/fts.test.ts
|
|
233
|
+
bun run inventory
|
|
234
|
+
bun run requirements
|
|
235
|
+
bun run typecheck
|
|
236
|
+
```
|
package/COMPATIBILITY.md
ADDED
|
@@ -0,0 +1,102 @@
|
|
|
1
|
+
# Compatibility
|
|
2
|
+
|
|
3
|
+
Goal: **full SQLite3 SQL dialect behavioral parity** as a drop-in for the same statements against the reference oracle. Compatibility is proven by the differential contract suite and the fail-closed gate:
|
|
4
|
+
|
|
5
|
+
```bash
|
|
6
|
+
bun run test:sqlite-compat
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
See [COMPATIBILITY-AUDIT.md](COMPATIBILITY-AUDIT.md) for the latest evidence-based audit report.
|
|
10
|
+
|
|
11
|
+
Reference oracle: **SQLite 3.51.0** (`bun:sqlite`; Linux/Windows bun may report **3.53.0** β see π `oracle-platform-sqlite-version`). Inventory: `bun run inventory`. Construct catalog: `bun run scenarios` β [`compat/scenarios.ts`](compat/scenarios.ts). Divergences: [`compat/divergences.json`](compat/divergences.json). Requirements matrix: `bun run requirements` β `compat/requirements.json` + `compat/coverage.json`.
|
|
12
|
+
|
|
13
|
+
## Proof surface (Phase 1)
|
|
14
|
+
|
|
15
|
+
Differential tests compare a **B-tuple**: rows (plus `typeof` where requested), column names, error category / sqliteCode / message (Tier A exact or Tier B prefix-normalized), `changes`, `total_changes`, `lastInsertRowid`, and autocommit, plus a **logical Dump** (`sqlite_master` names, `table_info`, row payloads with per-column `typeof`, `sqlite_sequence`, selected pragmas).
|
|
16
|
+
|
|
17
|
+
A catalog ID appearing in a test file is **not** proof by itself. Cases whose SQL is `SELECT 1 AS v` are **smoke**; [`compat/smoke-baseline.json`](compat/smoke-baseline.json) ratchets that list downward. Generated matrices live under [`tests/contract/matrices/`](tests/contract/matrices/). Observed memβ oracle diffs must be `known-divergence(id)` from π or **FAILURE** β unexplained diffs are not allowed.
|
|
18
|
+
|
|
19
|
+
Intentional differences are finite and machine-readable in `compat/divergences.json` (SQLM snapshots, seeded `random()`/`now`, ATTACH empty schema, EXPLAIN stubs, INDEXED BY discarded, MATERIALIZED hint ignored, FTS shadow counters, compile_options/function_list, `-0`, JS API extras, snapshot exclusions).
|
|
20
|
+
|
|
21
|
+
## Status vocabulary
|
|
22
|
+
|
|
23
|
+
| Status | Meaning |
|
|
24
|
+
| --- | --- |
|
|
25
|
+
| **VERIFIED** | Differential contracts (+ fuzz where applicable) cover happy path **and** meaningful edges vs oracle |
|
|
26
|
+
| **PARTIALLY VERIFIED** | Implemented; coverage thin or known edges remain |
|
|
27
|
+
| **UNSUPPORTED** | Missing from SQL surface (must fail loud; gate fails if oracle-exposed) |
|
|
28
|
+
| **NOT APPLICABLE** | Outside the in-memory dialect surface (C API, on-disk `.sqlite`, VFS/pager/WAL) |
|
|
29
|
+
|
|
30
|
+
## Scope 3 bound
|
|
31
|
+
|
|
32
|
+
Anything a SQLite application can invoke through SQL against this Bun/SQLite **3.51.0** build must match oracle observable behavior, except:
|
|
33
|
+
|
|
34
|
+
1. **Snapshot format** β custom binary codec (`SQLM`), not the on-disk SQLite database file format (logical state still round-trips).
|
|
35
|
+
2. **Deterministic `random()` / `'now'`** β seeded PRNG and fixed clock by default (injectable).
|
|
36
|
+
3. **NOT APPLICABLE** rows in `compat/coverage.json` (C API, VFS, pager, file locking, etc.).
|
|
37
|
+
|
|
38
|
+
Oracle builtins (math, string extras, uuid, ieee754, β¦) and modules (FTS3/4/5, RTREE, dbstat, bytecode, tables_used) are **in scope**.
|
|
39
|
+
|
|
40
|
+
## Feature matrix (summary)
|
|
41
|
+
|
|
42
|
+
| Area | Status | Notes |
|
|
43
|
+
| --- | --- | --- |
|
|
44
|
+
| Core DML / SELECT / joins / CTE / UPSERT / RETURNING | VERIFIED | Contract + fuzz |
|
|
45
|
+
| STRICT tables / indexes | VERIFIED | STRICT types; partial + expression indexes; leftmost prefix |
|
|
46
|
+
| Expressions / operators / `->` `->>` / row values | VERIFIED | Row-value + precedence contracts |
|
|
47
|
+
| Affinity / NULL / COLLATE | VERIFIED | Comparison affinity + collation on GROUP BY/JOIN |
|
|
48
|
+
| Constraints / FK / triggers / views / ATTACH | VERIFIED | Deferred FK, composite FK, INSTEAD OF, OR ROLLBACK/FAIL |
|
|
49
|
+
| Windows (incl. ntile/cume_dist/percent_rank) | VERIFIED | EXCLUDE; GROUPS/RANGE frames; window FILTER |
|
|
50
|
+
| JSON1 / JSONB / TVFs | VERIFIED | |
|
|
51
|
+
| Math / string / date extras / uuid / ieee754 | VERIFIED | Scope-3 inventory |
|
|
52
|
+
| FTS3 / FTS4 / FTS5 + MATCH | PARTIALLY VERIFIED | Differential FTS suite + fuzz vs 3.51.0; see FTS matrix below. Shadow-table change counters intentionally diverge. |
|
|
53
|
+
|
|
54
|
+
## FTS compatibility matrix
|
|
55
|
+
|
|
56
|
+
Reference: **SQLite 3.51.0** (`bun:sqlite`). Inventory: `bun run scripts/fts-oracle-surface.ts` β `compat/fts-oracle-surface.json`.
|
|
57
|
+
|
|
58
|
+
| Feature | Status |
|
|
59
|
+
| --- | --- |
|
|
60
|
+
| FTS3 | PARTIALLY VERIFIED |
|
|
61
|
+
| FTS4 | PARTIALLY VERIFIED |
|
|
62
|
+
| FTS5 | PARTIALLY VERIFIED |
|
|
63
|
+
| Virtual table creation (options/tokenizers) | VERIFIED |
|
|
64
|
+
| Tokenizers (unicode61/ascii/porter/trigram) | VERIFIED |
|
|
65
|
+
| MATCH grammar (AND/OR/NOT/phrase/prefix/NEAR/columns) | VERIFIED |
|
|
66
|
+
| Boolean operators | VERIFIED |
|
|
67
|
+
| Phrases | VERIFIED |
|
|
68
|
+
| Prefix queries | VERIFIED |
|
|
69
|
+
| NEAR | VERIFIED |
|
|
70
|
+
| Column filters | VERIFIED |
|
|
71
|
+
| Ranking / bm25 / rank | VERIFIED |
|
|
72
|
+
| highlight / snippet | VERIFIED |
|
|
73
|
+
| matchinfo / offsets (FTS3/4) | PARTIALLY VERIFIED | Default + common format strings verified; some FTS4-only formats thinner |
|
|
74
|
+
| Contentless tables | VERIFIED |
|
|
75
|
+
| External content | PARTIALLY VERIFIED | Canonical delete/sync covered; backfill/projection edges thinner |
|
|
76
|
+
| Content tables | VERIFIED |
|
|
77
|
+
| Triggers + FTS | PARTIALLY VERIFIED | Maintenance sequences covered; advanced edges thinner |
|
|
78
|
+
| Special commands (optimize/rebuild/integrity-check) | VERIFIED |
|
|
79
|
+
| Prefix indexes | VERIFIED |
|
|
80
|
+
| Unicode / adversarial corpus | VERIFIED |
|
|
81
|
+
| Transactions / savepoints | VERIFIED |
|
|
82
|
+
| Error behavior | VERIFIED |
|
|
83
|
+
| FTS differential fuzz | VERIFIED |
|
|
84
|
+
| FTS stateful fuzz | VERIFIED |
|
|
85
|
+
| RTREE / dbstat / bytecode / tables_used | VERIFIED | dbstat synthetic pages; bytecode empty cursor |
|
|
86
|
+
| ANALYZE / REINDEX / VACUUM | VERIFIED | `:memory:` observable parity |
|
|
87
|
+
| EXPLAIN / INDEXED BY | PARTIALLY VERIFIED | Stub shapes / no-ops (missing INDEXED BY errors documented) |
|
|
88
|
+
| Prepared stmt schema invalidation | VERIFIED | Re-prepare after ALTER/DROP; `tests/contract/api/schema-invalidation.test.ts` |
|
|
89
|
+
| On-disk file format / C API | NOT APPLICABLE | |
|
|
90
|
+
|
|
91
|
+
## How to verify
|
|
92
|
+
|
|
93
|
+
```bash
|
|
94
|
+
bun run test:sqlite-compat # requirements + gate + contract/fuzz/harness
|
|
95
|
+
bun run inventory # oracle function/module inventory
|
|
96
|
+
bun run requirements # refresh SQLite.org requirements + coverage
|
|
97
|
+
bun run build && bun run verify-package # ESM browser build + isomorphic pack gates
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
Do not treat isolated unit tests of internal modules as proof of SQLite compatibility. The differential matrix runner is authoritative for SQL behavior; `test:sqlite-compat` is the release gate.
|
|
101
|
+
|
|
102
|
+
**Parity claim:** Verified against **SQLite 3.51.0** (`bun:sqlite`). Features marked **VERIFIED** are oracle-proven. **PARTIALLY VERIFIED** rows must not be marketed as complete. **NOT APPLICABLE** is the only allowed permanent omission from the SQL drop-in claim.
|
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Chris Vouga
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|