@crvouga/mockingbird-service-sqlite 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (100) hide show
  1. package/AGENTS.md +170 -0
  2. package/COMPATIBILITY-AUDIT.md +236 -0
  3. package/COMPATIBILITY.md +102 -0
  4. package/LICENSE +21 -0
  5. package/README.md +405 -0
  6. package/compat/coverage.json +22231 -0
  7. package/compat/divergences.json +145 -0
  8. package/compat/fts-oracle-surface.json +198 -0
  9. package/compat/requirements.json +20929 -0
  10. package/compat/scenario-types.ts +63 -0
  11. package/compat/scenarios.ts +871 -0
  12. package/compat/smoke-baseline.json +3 -0
  13. package/compat/sqllogictest-skip.json +5 -0
  14. package/dist/api/database.d.ts +112 -0
  15. package/dist/api/snapshot.d.ts +21 -0
  16. package/dist/api/statement.d.ts +75 -0
  17. package/dist/ast/nodes.d.ts +476 -0
  18. package/dist/constraints/check.d.ts +11 -0
  19. package/dist/errors/index.d.ts +45 -0
  20. package/dist/executor/attach.d.ts +5 -0
  21. package/dist/executor/ddl.d.ts +10 -0
  22. package/dist/executor/dml.d.ts +7 -0
  23. package/dist/executor/env.d.ts +66 -0
  24. package/dist/executor/execute.d.ts +4 -0
  25. package/dist/executor/pragma-engine.d.ts +31 -0
  26. package/dist/executor/pragma.d.ts +4 -0
  27. package/dist/executor/result.d.ts +22 -0
  28. package/dist/executor/select.d.ts +7 -0
  29. package/dist/executor/simple-select.d.ts +9 -0
  30. package/dist/executor/triggers.d.ts +11 -0
  31. package/dist/executor/vtable.d.ts +12 -0
  32. package/dist/expressions/context.d.ts +30 -0
  33. package/dist/expressions/equals.d.ts +3 -0
  34. package/dist/expressions/eval.d.ts +12 -0
  35. package/dist/expressions/index.d.ts +3 -0
  36. package/dist/expressions/like.d.ts +16 -0
  37. package/dist/functions/aggregate.d.ts +8 -0
  38. package/dist/functions/datetime.d.ts +2 -0
  39. package/dist/functions/extensions.d.ts +7 -0
  40. package/dist/functions/index.d.ts +8 -0
  41. package/dist/functions/json.d.ts +6 -0
  42. package/dist/functions/math.d.ts +2 -0
  43. package/dist/functions/pragma-tvf.d.ts +8 -0
  44. package/dist/functions/registry.d.ts +39 -0
  45. package/dist/functions/scalar.d.ts +9 -0
  46. package/dist/functions/table-valued-registry.d.ts +11 -0
  47. package/dist/functions/table-valued.d.ts +13 -0
  48. package/dist/functions/window.d.ts +23 -0
  49. package/dist/index.d.ts +28 -0
  50. package/dist/index.js +16010 -0
  51. package/dist/index.js.map +7 -0
  52. package/dist/indexes/index.d.ts +48 -0
  53. package/dist/indexes/keys.d.ts +15 -0
  54. package/dist/json/index.d.ts +7 -0
  55. package/dist/json/jsonb.d.ts +42 -0
  56. package/dist/json/ops.d.ts +24 -0
  57. package/dist/json/parse.d.ts +13 -0
  58. package/dist/json/path.d.ts +26 -0
  59. package/dist/json/stringify.d.ts +4 -0
  60. package/dist/json/tvf.d.ts +13 -0
  61. package/dist/json/types.d.ts +29 -0
  62. package/dist/lexer/tokenize.d.ts +23 -0
  63. package/dist/parser/index.d.ts +17 -0
  64. package/dist/parser/parser.d.ts +115 -0
  65. package/dist/planner/access.d.ts +50 -0
  66. package/dist/planner/index.d.ts +3 -0
  67. package/dist/runtime/assert.d.ts +11 -0
  68. package/dist/runtime/catch.d.ts +3 -0
  69. package/dist/runtime/clock.d.ts +14 -0
  70. package/dist/runtime/index.d.ts +3 -0
  71. package/dist/runtime/options.d.ts +25 -0
  72. package/dist/runtime/prng.d.ts +42 -0
  73. package/dist/schema/catalog.d.ts +14 -0
  74. package/dist/schema/master-sql.d.ts +10 -0
  75. package/dist/serialization/codec.d.ts +19 -0
  76. package/dist/serialization/index.d.ts +1 -0
  77. package/dist/serialization/wire.d.ts +61 -0
  78. package/dist/storage/columnar-slab.d.ts +58 -0
  79. package/dist/storage/database-state.d.ts +96 -0
  80. package/dist/storage/index.d.ts +6 -0
  81. package/dist/storage/row.d.ts +13 -0
  82. package/dist/storage/table.d.ts +119 -0
  83. package/dist/transactions/manager.d.ts +19 -0
  84. package/dist/types/collation.d.ts +4 -0
  85. package/dist/types/sqlite-atof.d.ts +4 -0
  86. package/dist/types/sqlite-real-format.d.ts +5 -0
  87. package/dist/types/strict.d.ts +6 -0
  88. package/dist/types/value.d.ts +88 -0
  89. package/dist/unstable.d.ts +20 -0
  90. package/dist/unstable.js +10343 -0
  91. package/dist/unstable.js.map +7 -0
  92. package/dist/vtable/fts/options.d.ts +34 -0
  93. package/dist/vtable/fts/porter.d.ts +2 -0
  94. package/dist/vtable/fts/query.d.ts +39 -0
  95. package/dist/vtable/fts/table.d.ts +78 -0
  96. package/dist/vtable/fts/tokenize.d.ts +20 -0
  97. package/dist/vtable/fts5.d.ts +1 -0
  98. package/dist/vtable/index.d.ts +2 -0
  99. package/dist/vtable/modules.d.ts +65 -0
  100. package/package.json +122 -0
package/AGENTS.md ADDED
@@ -0,0 +1,170 @@
1
+ # AGENTS.md β€” contributing to sqlite-mem
2
+
3
+ Guidance for humans and coding agents editing this repository. For install and consumer API, see [README.md](README.md). For the feature matrix, see [COMPATIBILITY.md](COMPATIBILITY.md).
4
+
5
+ ## Mission and non-goals
6
+
7
+ **Mission:** full SQLite3 **SQL dialect** behavioral parity vs **SQLite 3.51.0** (`bun:sqlite`). Same statements β†’ same observable results (rows, errors, counters), proven by differential contracts and a fail-closed gate.
8
+
9
+ **Allowed intentional differences:**
10
+
11
+ 1. Custom snapshot codec (`SQLM`), not on-disk `.sqlite` files
12
+ 2. Deterministic `random()` / `randomblob()` and fixed `'now'` by default (injectable)
13
+ 3. `NOT APPLICABLE` items: C API, VFS/pager/WAL, file locking, URI filenames
14
+
15
+ **Non-goals:** shipping a SQLite file format, native/WASM bindings, or matching better-sqlite3’s full Node API surface.
16
+
17
+ ## SQL pipeline
18
+
19
+ ```
20
+ SQL string
21
+ β†’ tokenize() src/lexer/tokenize.ts
22
+ β†’ parse() src/parser/index.ts β†’ parser.ts
23
+ β†’ Statement[] AST src/ast/nodes.ts
24
+ β†’ Statement.execute() src/api/statement.ts
25
+ β†’ executeStatement() src/executor/execute.ts
26
+ β†’ per-stmt executor select / dml / ddl / …
27
+ ```
28
+
29
+ Public entry points: `Database.exec` / `query` / `prepare` in [`src/api/database.ts`](src/api/database.ts).
30
+
31
+ **Planner note:** [`src/planner/index.ts`](src/planner/index.ts) `preparePlan` is an **identity stub** (`Plan === Statement`) and is unused. Runtime index/PK access paths live in [`src/planner/access.ts`](src/planner/access.ts) (`tryIndexedTableRows`, `lookupTableRows`, `tryJoinProbe`), called from the executor.
32
+
33
+ SELECT has a **fast path** ([`src/executor/simple-select.ts`](src/executor/simple-select.ts)) and a **full path** ([`src/executor/select.ts`](src/executor/select.ts)). Fast paths return `null` when the shape is unsupported; then the full path runs. Changing join/WHERE/expression semantics in only one path will silently diverge.
34
+
35
+ ## `src/` map
36
+
37
+ | Directory | Role |
38
+ | --- | --- |
39
+ | `api/` | Public `Database` / `Statement` facade |
40
+ | `ast/` | Discriminated-union AST (`nodes.ts`) |
41
+ | `lexer/` | Tokenizer |
42
+ | `parser/` | Recursive-descent parser |
43
+ | `planner/` | Access-path helpers (not a logical plan IR) |
44
+ | `executor/` | Statement dispatch, SELECT, DML, DDL, PRAGMA, triggers, ATTACH |
45
+ | `expressions/` | `evalExpr`, LIKE/GLOB, eval context |
46
+ | `functions/` | Scalar / aggregate / window / datetime / JSON / math / TVF registries |
47
+ | `types/` | Engine `SqlValue`, affinity, collation |
48
+ | `storage/` | In-memory tables, rows, `DatabaseState` |
49
+ | `indexes/` | Equality index store |
50
+ | `schema/` | `sqlite_master` / catalog |
51
+ | `constraints/` | NOT NULL / PK / UNIQUE / CHECK |
52
+ | `transactions/` | BEGIN / COMMIT / SAVEPOINT (clones state + PRNG) |
53
+ | `runtime/` | Clock, PRNG, `DatabaseOptions` |
54
+ | `serialization/` | `SQLM` snapshot codec |
55
+ | `json/` | JSON1 / JSONB internals |
56
+ | `vtable/` | FTS5 and other virtual-table modules |
57
+ | `errors/` | `SqliteError`, `unsupported()` |
58
+
59
+ Hot / large files: `parser/parser.ts`, `executor/select.ts`, `executor/dml.ts`.
60
+
61
+ ## Critical conventions
62
+
63
+ - **AST `type` tags** are snake_case (`"create_table"`, `"drop_index"`). TypeScript interfaces are PascalCase (`CreateTableStmt`).
64
+ - **Identifiers** are case-folded (`toLowerCase`) as Map keys in storage and function registries.
65
+ - **Engine `SqlValue`** ([`src/types/value.ts`](src/types/value.ts)) may include `SqlReal` / `SqlJsonText`. **Harness `SqlValue`** ([`tests/harness/types.ts`](tests/harness/types.ts)) is the normalized compare type β€” do not confuse them.
66
+ - **API `Statement`** vs **AST `Statement`**: the API class aliases the AST union as `AstStatement`.
67
+ - **`Database`** (API) vs **`DatabaseState`** (engine storage).
68
+ - Throw **`SqliteError`** with an `ErrorCategory`. Missing SQL must fail loud via `unsupported()` β€” the inventory gate fails if the oracle exposes an unimplemented builtin/module.
69
+ - Fast-path helpers (`tryExecuteSimpleSelect`, `tryFastInsert`, `tryIndexedTableRows`, …): return `null` β†’ fall through. Update **both** paths when semantics change.
70
+ - **Dual-path invariant:** residual `WHERE` filtering is skipped on an index/hash access path only when `whereFullyCovered(where, coveredColumns, …)` is true (see [`src/planner/access.ts`](src/planner/access.ts)). Any change to WHERE/join/insert semantics requires updating both fast and full paths **and** the fast≑full property tests under `tests/fuzz/fast-path.test.ts`.
71
+ - Internal invariants use always-on asserts in [`src/runtime/assert.ts`](src/runtime/assert.ts) (`assert`, `assertUnreachable`, `assertRowShape`).
72
+ - TypeScript: `strict` + `noUncheckedIndexedAccess`. Imports use `.ts` extensions. Biome: 2-space, double quotes, 120 columns.
73
+ - IEEE `-0` is canonicalized to `+0` on bind, affinity, and arithmetic. Keep determinism invariants (see README).
74
+
75
+ ## Change checklists
76
+
77
+ ### New SQL statement
78
+
79
+ 1. Add union member + interface in [`src/ast/nodes.ts`](src/ast/nodes.ts)
80
+ 2. Parse in [`src/parser/parser.ts`](src/parser/parser.ts) (`parseStatement` dispatch)
81
+ 3. Handle in [`src/executor/execute.ts`](src/executor/execute.ts) (+ `ddl.ts` / `dml.ts` / `vtable.ts` as needed)
82
+ 4. Mutate `DatabaseState` if schema changes
83
+ 5. Add differential contract under `tests/contract/<area>/`
84
+ 6. Optionally add evidence paths in `SOURCE_SEED` in [`scripts/sqlite-requirements.ts`](scripts/sqlite-requirements.ts), then `bun run requirements`
85
+
86
+ ### New SQL function
87
+
88
+ 1. Implement and register in the right map under `src/functions/*` (`getScalarFunctions()`, datetime, JSON, etc.) so [`scripts/sqlite-inventory.ts`](scripts/sqlite-inventory.ts) sees it
89
+ 2. Contract tests under `tests/contract/functions/` (and related areas)
90
+ 3. Run `bun run inventory` / `bun run test:sqlite-compat` β€” oracle builtins must not be missing
91
+
92
+ ### New contract test
93
+
94
+ 1. Prefer helpers in [`tests/contract/helpers.ts`](tests/contract/helpers.ts):
95
+ - `parity` β€” query both engines
96
+ - `execParity` β€” writes
97
+ - `sequenceParity` β€” multi-step (optional final-state compare)
98
+ - `errorParity` / `queryErrorParity` β€” both must fail
99
+ - `ftsRankParity` β€” REAL epsilon compare
100
+ 2. Or `matrixBoth` + `expectParity` from `tests/harness/`
101
+ 3. **Do not** treat isolated internal unit tests as SQLite proof. The differential suite is authoritative.
102
+ 4. Gate: `bun run test:sqlite-compat`
103
+
104
+ ## Test layout
105
+
106
+ | Path | Role |
107
+ | --- | --- |
108
+ | `tests/contract/` | Differential SQL vs `bun:sqlite` (**authoritative**) |
109
+ | `tests/fuzz/` | fast-check property tests (seeded); same two backends |
110
+ | `tests/harness/` | Compare/normalize helpers + harness unit tests |
111
+ | `tests/adapters/` | Wrappers for sqlite-mem and `bun:sqlite` |
112
+
113
+ Examples of public API usage: `tests/contract/api/`, `tests/contract/parameters/`, `tests/contract/determinism/`, `examples/react-vite`.
114
+
115
+ ### Fuzz replay
116
+
117
+ Default seed `0x5a17e0e1`. On failure the seed is printed:
118
+
119
+ ```bash
120
+ bun test tests/fuzz
121
+ SQLITE_MEM_FUZZ_SEED=12345 bun test tests/fuzz
122
+ SQLITE_MEM_FUZZ_SEED=12345 SQLITE_MEM_FUZZ_PATH='0:1' bun test tests/fuzz
123
+ ```
124
+
125
+ ### Random walk (state-dependent DST)
126
+
127
+ `tests/fuzz/random-walk.test.ts` walks the SQL state space one enabled action at
128
+ a time and asserts result + logical-state parity after every step.
129
+
130
+ ```bash
131
+ bun run test:walk
132
+ SQLITE_MEM_WALK_STEPS=80 bun run test:walk
133
+ bun run test:walk:soak -- --depth 200 --runs 20
134
+ ```
135
+
136
+ ## Compat system
137
+
138
+ | Command | Role |
139
+ | --- | --- |
140
+ | `bun run test:sqlite-compat` | Requirements + fail-closed gate + construct catalog + 𝔇 + smoke ratchet + contract/fuzz/harness |
141
+ | `bun run inventory` | Oracle `pragma_function_list` / modules vs memory registries |
142
+ | `bun run scenarios` | Construct-level scenario catalog (`compat/scenarios.ts`) + 𝔇 / smoke gates |
143
+ | `bun run requirements` | Refresh sqlite.org requirements β†’ `compat/requirements.json` + `compat/coverage.json` |
144
+ | `bun run fts-surface` | FTS oracle surface β†’ `compat/fts-oracle-surface.json` |
145
+
146
+ Statuses: **VERIFIED** / **PARTIALLY VERIFIED** / **UNSUPPORTED** / **NOT APPLICABLE**. Do not market PARTIAL as complete. Coverage evidence is directory paths (e.g. `tests/contract/joins/`), not automatic from test filenames.
147
+
148
+ **Catalog vs proof:** `tests/contract/catalog/` IDs must execute; smoke (`SELECT 1 AS v`) is tracked in `compat/smoke-baseline.json`. Documented divergences bind to `compat/divergences.json`. Generated operator/CAST matrices: `tests/contract/matrices/`. Stateful dump-after-each fuzz: `tests/fuzz/stateful.test.ts`. Oracle `sqlite_version()` must be 3.51.0 or 3.53.0.
149
+
150
+ Details: [COMPATIBILITY.md](COMPATIBILITY.md), audit: [COMPATIBILITY-AUDIT.md](COMPATIBILITY-AUDIT.md).
151
+
152
+ ## Local gates
153
+
154
+ Requires [Bun](https://bun.sh).
155
+
156
+ ```bash
157
+ bun install
158
+ bun run check:full # same gates as GitHub Actions CI (except publish)
159
+ bun run check # format + lint + typecheck + sqlite-compat suite
160
+ bun run format
161
+ bun run lint
162
+ bun run typecheck
163
+ bun run test:sqlite-compat
164
+ bun test # contract + fuzz + harness
165
+ bun run build
166
+ ```
167
+
168
+ ## PR and commits
169
+
170
+ Use [Conventional Commits](https://www.conventionalcommits.org/) for commits and PR titles (enforced on PRs). Prefer squash merges with a conventional title so the npm bump is `feat` β†’ minor / `fix` β†’ patch. Direct pushes to `main` with any other subject still publish a patch. See [README.md](README.md#releasing).
@@ -0,0 +1,236 @@
1
+ # SQLite3 Compatibility Audit
2
+
3
+ ```text
4
+ SQLite3 Compatibility Audit
5
+ ===========================
6
+
7
+ Reference SQLite version:
8
+ bun:sqlite (SQLite 3.51.0)
9
+
10
+ Reference SQLite compile options:
11
+ ENABLE_MATH_FUNCTIONS, ENABLE_FTS3/4/5, ENABLE_RTREE, ENABLE_DBSTAT_VTAB,
12
+ ENABLE_BYTECODE_VTAB, JSON builtin (not OMIT_JSON), plus Bun/SEE codec flags.
13
+ Full list: `bun run inventory`
14
+
15
+ sqlite-mem version:
16
+ 0.0.0-development (package.json; released from the monorepo via `<name>@<version>` tags)
17
+
18
+ Scope:
19
+ Scope 3 β€” every oracle-exposed SQL builtin/module is in-scope except
20
+ NOT APPLICABLE (C API, on-disk format, VFS/pager/WAL).
21
+
22
+ SQLite requirements reviewed:
23
+ 3487 (from sqlite.org requirements matrix β†’ compat/requirements.json)
24
+
25
+ SQLite requirements classification:
26
+ NOT APPLICABLE: see compat/coverage.json counts.notApplicable
27
+ SQL_BEHAVIOR: see counts.sqlBehavior
28
+ unknown: 0 (gate fails if non-zero)
29
+
30
+ Coverage statuses (SQL_BEHAVIOR):
31
+ VERIFIED / PARTIALLY_VERIFIED / UNSUPPORTED β€” see `bun run requirements`
32
+ Regenerated via scripts/sqlite-requirements.ts + coverage upgrades.
33
+
34
+ SQL grammar / operators / expressions:
35
+ VERIFIED β€” contracts + row-value comparisons + ->/->> precedence
36
+
37
+ Types / affinity / NULL:
38
+ VERIFIED β€” affinity matrix, typeof/REAL (incl. unary Β± float literals),
39
+ truthiness/3VL, JS-leak bind probes (differential + documented divergences)
40
+
41
+ Functions (oracle surface):
42
+ VERIFIED β€” inventory missingOracleFunctions = 0 (163 names covered)
43
+
44
+ JSON / JSONB:
45
+ VERIFIED
46
+
47
+ Aggregates / windows:
48
+ VERIFIED β€” includes string_agg, ntile, cume_dist, percent_rank, EXCLUDE;
49
+ unaliased aggregate column names (`sum(v)`, `count(*)`) match oracle
50
+
51
+ CTEs / transactions / savepoints / constraints / FK / triggers:
52
+ VERIFIED (deferred FK + composite FK + MATCH SIMPLE/FULL + statement ABORT)
53
+
54
+ Indexes / views / generated / STRICT / WITHOUT ROWID:
55
+ VERIFIED β€” partial/expression indexes, STRICT tables, leftmost prefix;
56
+ WITHOUT ROWID UPSERT; generated STORED/VIRTUAL WHERE/ORDER BY
57
+
58
+ PRAGMAs:
59
+ PARTIALLY VERIFIED β€” schema/FK; storage pragmas N/A or :memory: no-op
60
+
61
+ ATTACH/DETACH:
62
+ VERIFIED (in-memory schemas; temp.schema = main state)
63
+
64
+ Virtual tables / extensions:
65
+ VERIFIED for oracle modules: fts3/4/5, fts5vocab, rtree/rtree_i32,
66
+ dbstat, bytecode, tables_used (bytecode/tables_used empty cursors)
67
+
68
+ ANALYZE / REINDEX / VACUUM:
69
+ VERIFIED (:memory: observable parity)
70
+
71
+ Prepared statements / errors / snapshot:
72
+ VERIFIED β€” schema invalidation re-prepares; SQLM logical round-trip VERIFIED;
73
+ post-restore A/B lockstep + dumpLogicalState vs oracle
74
+
75
+ Differential tests (2026-08-21 audit refresh):
76
+ Total: 1912 under `bun test` / `bun run test:sqlite-compat`
77
+ (contract + fuzz + harness; Bun 1.4.0, SQLite 3.51.0)
78
+ Passed: 1912
79
+ Failed: 0
80
+ expect() calls: 7314
81
+
82
+ Stateful / fuzz:
83
+ Seeds: 0x5a17e0e1 (+ SQLITE_MEM_FUZZ_SEED override)
84
+ Combination fuzz: tests/fuzz/combinations-scope3.test.ts
85
+ FTS fuzz: tests/fuzz/fts.test.ts
86
+ Long deterministic scripts: tests/contract/transactions/stateful.test.ts
87
+ Collate / window / CTE+DML / JSON subtype / affinity / conflicts fuzz
88
+ Mismatches: 0
89
+
90
+ Harness integrity (Phase 1):
91
+ RealSqliteAdapter DML via prepare().run() with live changes/lastInsertRowid
92
+ sequenceParity compares DML counters; DDL/txn/PRAGMA allowlist neutralized
93
+ errorParity via deepCompareResults; dumpLogicalState + compareFinalState
94
+ Comparator meta-tests: tests/harness/normalize.test.ts
95
+
96
+ Hardening pass (2026-08-19) β€” incompatibilities found & fixed:
97
+ 1. Comparison affinity (INTEGER/TEXT/NUMERIC column vs literal) missing
98
+ 2. WITH on UPDATE/DELETE/INSERT VALUES dispatched as SELECT only
99
+ 3. Plain INTEGER PRIMARY KEY never reused deleted max rowid
100
+ 4. date() `weekday N` modifier unimplemented
101
+ 5. REGEXP operator not parsed (now calls missing regexp() like oracle)
102
+ 6. last_insert_rowid() stale inside AFTER INSERT triggers
103
+ 7. INSTEAD OF triggers never fired; view DML rejected
104
+ 8. INSERT/UPDATE OR ROLLBACK / OR FAIL not distinguished
105
+ 9. Window GROUPS/RANGE frames, window FILTER, recursive CTE queue edges
106
+ 10. Collation-aware GROUP BY; FTS3 matchinfo format variants; external-content delete
107
+
108
+ Audit refresh (2026-08-21) β€” additional findings & fixes:
109
+ 11. sequenceParity previously ignored all DML write counters (hid divergences)
110
+ 12. Unary +/- on float literals dropped REAL storage class (`typeof(-0.0)`)
111
+ 13. UPDATE OR REPLACE counted conflict deletions in changes() (oracle = 1)
112
+ 14. Empty-result column names + unaliased aggregate headers asserted vs oracle
113
+ 15. JS bind edges: empty TEXT/BLOB, BigInt@MAX_SAFE, intentional Number/BigInt
114
+ bind-class divergences pinned via divergence()
115
+ 16. Snapshot post-restore A/B + state dump vs live oracle
116
+
117
+ Remaining known differences / intentional:
118
+ Custom SQLM snapshots; deterministic random()/'now' by default
119
+ (`random: "os"` / `now: "system"` match SQLite entropy and wall clock);
120
+ EXPLAIN stubs (shape contracts only); INDEXED BY no-op (documented);
121
+ ATTACH file path records filename but opens empty in-memory schema;
122
+ some PRAGMA storage no-ops; FTS shadow-table changes()/total_changes diverge
123
+ (sequenceParity neutralizeCounters on FTS maintenance paths);
124
+ MATERIALIZED/NOT MATERIALIZED stored but both materialize today;
125
+ compile_options / function_list content is sqlite-mem's;
126
+ generate_series is sqlite-mem extension (not in bun:sqlite default);
127
+ Number.MAX_SAFE_INTEGER JS bind typeof (mem integer vs bun real);
128
+ BigInt beyond Number.MAX_SAFE_INTEGER without bun safeIntegers;
129
+ NOT APPLICABLE C API / on-disk / VFS surfaces; uri.html remapped N/A.
130
+
131
+ Final assessment:
132
+ Verified against SQLite 3.51.0 (bun:sqlite on Bun 1.4.0). Oracle
133
+ function/module inventory is closed (0 missing). Requirements matrix
134
+ ingested with zero unknown statuses. Gate: `bun run test:sqlite-compat`.
135
+ Harness now compares live DML counters; green means stronger evidence than
136
+ pre-2026-08-21. FTS/EXPLAIN remain PARTIALLY VERIFIED honestly β€” not
137
+ β€œfully compatible because green.”
138
+ ```
139
+
140
+ ---
141
+
142
+ ## SQLite Full-Text Search Compatibility Audit
143
+
144
+ ```text
145
+ SQLite Full-Text Search Compatibility Audit
146
+ ==========================================
147
+
148
+ Reference SQLite:
149
+ version: 3.51.0 (bun:sqlite)
150
+ source_id: 2025-06-12 13:14:41 f0ca7bba1c5e232e5d279fad6338121ab55af0c8c68c84cdfb18ba5114dcaapl
151
+ compile options: ENABLE_FTS3, ENABLE_FTS3_PARENTHESIS, ENABLE_FTS3_TOKENIZER,
152
+ ENABLE_FTS4, ENABLE_FTS5
153
+ inventory: compat/fts-oracle-surface.json (bun run scripts/fts-oracle-surface.ts)
154
+
155
+ FTS3: PARTIALLY VERIFIED (MATCH, snippet, offsets; matchinfo formats thinner)
156
+ FTS4: PARTIALLY VERIFIED (same surface as FTS3 for tested paths)
157
+ FTS5: PARTIALLY VERIFIED overall β€” core MATCH/tokenizers/ranking/aux VERIFIED;
158
+ external-content + full matchinfo format strings still thinner
159
+
160
+ Tokenizers verified:
161
+ unicode61 (incl. remove_diacritics 0/1/2), ascii, porter,
162
+ porter unicode61, porter ascii, trigram
163
+
164
+ MATCH grammar verified:
165
+ terms, AND/OR/NOT, implicit AND, phrases, prefix *, column filters
166
+ (col : term / {cols} :), NEAR / NEAR(…, N), parentheses, NEAR-as-term
167
+
168
+ Ranking verified:
169
+ bm25() / rank column β€” order + scores vs oracle (1e-15 abs epsilon for ULP)
170
+
171
+ Auxiliary functions verified:
172
+ bm25, highlight, snippet (FTS5); snippet, offsets (FTS3/4);
173
+ matchinfo present (default format PARTIAL)
174
+
175
+ Content modes verified:
176
+ normal content tables: VERIFIED
177
+ contentless (content=''): VERIFIED
178
+ external content: PARTIALLY VERIFIED (CREATE accepted; sync/triggers thinner)
179
+
180
+ Special commands verified:
181
+ optimize, rebuild, integrity-check (success + post-command MATCH)
182
+ delete-all / merge / automerge: error parity with oracle where probed
183
+
184
+ Differential tests:
185
+ Passed: 872 (contract + fuzz + harness)
186
+ Failed: 0
187
+ FTS contract: tests/contract/fts/basic.test.ts,
188
+ tests/contract/fts/comprehensive.test.ts
189
+ (+ matchinfo formats, external-content, trigger maintenance)
190
+
191
+ Fuzz cases:
192
+ Generated: fast-check seed 0x5a17e0e1 (override SQLITE_MEM_FUZZ_SEED)
193
+ Files: tests/fuzz/fts.test.ts (MATCH queries, tokenizers, stateful DML)
194
+ Mismatches: 0 (after trigram phrase expansion + NEAR-as-term fixes)
195
+
196
+ Stateful cases:
197
+ Operations: INSERT/UPDATE/DELETE/MATCH sequences + txn/savepoint contracts
198
+ Mismatches: 0
199
+
200
+ Compatibility gaps found:
201
+ 1. Toy AND-token matcher (replaced)
202
+ 2. Aux functions always threw (wired MATCH cursor + real bm25/highlight/snippet)
203
+ 3. No CREATE option parsing (tokenize=/content=/prefix=/UNINDEXED)
204
+ 4. No FTS5 query language / NEAR / phrases / column filters
205
+ 5. False VERIFIED status on fts5.html / fts3.html (downgraded then re-evidenced)
206
+
207
+ Compatibility gaps fixed:
208
+ Positional inverted index; unicode61/ascii/porter/trigram tokenizers;
209
+ FTS5 query parser; BM25 with SQLite IDF floor 1e-6; highlight/snippet;
210
+ contentless mode; special commands; FTS3/4 MATCH + snippet/offsets;
211
+ comparator realEpsilon for ranking ULP only; FTS fuzz suites
212
+
213
+ Regression tests added:
214
+ tests/contract/fts/comprehensive.test.ts
215
+ tests/fuzz/fts.test.ts
216
+ tests/harness/fts-compare.ts
217
+ scripts/fts-oracle-surface.ts
218
+
219
+ Remaining unsupported / thinner functionality:
220
+ - SQLite FTS shadow tables not mirrored (change counters diverge; documented)
221
+ - External-content sync/stale-index edge cases thinner
222
+ - FTS3 matchinfo format-string variants not exhaustively verified
223
+ - fts3tokenize / fts4aux still thin CREATE stubs
224
+ - locale / tokendata / detail=none advanced options accepted but lightly tested
225
+ ```
226
+
227
+ Verification commands:
228
+
229
+ ```bash
230
+ bun run test:sqlite-compat
231
+ bun run scripts/fts-oracle-surface.ts
232
+ bun test tests/contract/fts tests/fuzz/fts.test.ts
233
+ bun run inventory
234
+ bun run requirements
235
+ bun run typecheck
236
+ ```
@@ -0,0 +1,102 @@
1
+ # Compatibility
2
+
3
+ Goal: **full SQLite3 SQL dialect behavioral parity** as a drop-in for the same statements against the reference oracle. Compatibility is proven by the differential contract suite and the fail-closed gate:
4
+
5
+ ```bash
6
+ bun run test:sqlite-compat
7
+ ```
8
+
9
+ See [COMPATIBILITY-AUDIT.md](COMPATIBILITY-AUDIT.md) for the latest evidence-based audit report.
10
+
11
+ Reference oracle: **SQLite 3.51.0** (`bun:sqlite`; Linux/Windows bun may report **3.53.0** β€” see 𝔇 `oracle-platform-sqlite-version`). Inventory: `bun run inventory`. Construct catalog: `bun run scenarios` β†’ [`compat/scenarios.ts`](compat/scenarios.ts). Divergences: [`compat/divergences.json`](compat/divergences.json). Requirements matrix: `bun run requirements` β†’ `compat/requirements.json` + `compat/coverage.json`.
12
+
13
+ ## Proof surface (Phase 1)
14
+
15
+ Differential tests compare a **B-tuple**: rows (plus `typeof` where requested), column names, error category / sqliteCode / message (Tier A exact or Tier B prefix-normalized), `changes`, `total_changes`, `lastInsertRowid`, and autocommit, plus a **logical Dump** (`sqlite_master` names, `table_info`, row payloads with per-column `typeof`, `sqlite_sequence`, selected pragmas).
16
+
17
+ A catalog ID appearing in a test file is **not** proof by itself. Cases whose SQL is `SELECT 1 AS v` are **smoke**; [`compat/smoke-baseline.json`](compat/smoke-baseline.json) ratchets that list downward. Generated matrices live under [`tests/contract/matrices/`](tests/contract/matrices/). Observed memβ‰ oracle diffs must be `known-divergence(id)` from 𝔇 or **FAILURE** β€” unexplained diffs are not allowed.
18
+
19
+ Intentional differences are finite and machine-readable in `compat/divergences.json` (SQLM snapshots, seeded `random()`/`now`, ATTACH empty schema, EXPLAIN stubs, INDEXED BY discarded, MATERIALIZED hint ignored, FTS shadow counters, compile_options/function_list, `-0`, JS API extras, snapshot exclusions).
20
+
21
+ ## Status vocabulary
22
+
23
+ | Status | Meaning |
24
+ | --- | --- |
25
+ | **VERIFIED** | Differential contracts (+ fuzz where applicable) cover happy path **and** meaningful edges vs oracle |
26
+ | **PARTIALLY VERIFIED** | Implemented; coverage thin or known edges remain |
27
+ | **UNSUPPORTED** | Missing from SQL surface (must fail loud; gate fails if oracle-exposed) |
28
+ | **NOT APPLICABLE** | Outside the in-memory dialect surface (C API, on-disk `.sqlite`, VFS/pager/WAL) |
29
+
30
+ ## Scope 3 bound
31
+
32
+ Anything a SQLite application can invoke through SQL against this Bun/SQLite **3.51.0** build must match oracle observable behavior, except:
33
+
34
+ 1. **Snapshot format** β€” custom binary codec (`SQLM`), not the on-disk SQLite database file format (logical state still round-trips).
35
+ 2. **Deterministic `random()` / `'now'`** β€” seeded PRNG and fixed clock by default (injectable).
36
+ 3. **NOT APPLICABLE** rows in `compat/coverage.json` (C API, VFS, pager, file locking, etc.).
37
+
38
+ Oracle builtins (math, string extras, uuid, ieee754, …) and modules (FTS3/4/5, RTREE, dbstat, bytecode, tables_used) are **in scope**.
39
+
40
+ ## Feature matrix (summary)
41
+
42
+ | Area | Status | Notes |
43
+ | --- | --- | --- |
44
+ | Core DML / SELECT / joins / CTE / UPSERT / RETURNING | VERIFIED | Contract + fuzz |
45
+ | STRICT tables / indexes | VERIFIED | STRICT types; partial + expression indexes; leftmost prefix |
46
+ | Expressions / operators / `->` `->>` / row values | VERIFIED | Row-value + precedence contracts |
47
+ | Affinity / NULL / COLLATE | VERIFIED | Comparison affinity + collation on GROUP BY/JOIN |
48
+ | Constraints / FK / triggers / views / ATTACH | VERIFIED | Deferred FK, composite FK, INSTEAD OF, OR ROLLBACK/FAIL |
49
+ | Windows (incl. ntile/cume_dist/percent_rank) | VERIFIED | EXCLUDE; GROUPS/RANGE frames; window FILTER |
50
+ | JSON1 / JSONB / TVFs | VERIFIED | |
51
+ | Math / string / date extras / uuid / ieee754 | VERIFIED | Scope-3 inventory |
52
+ | FTS3 / FTS4 / FTS5 + MATCH | PARTIALLY VERIFIED | Differential FTS suite + fuzz vs 3.51.0; see FTS matrix below. Shadow-table change counters intentionally diverge. |
53
+
54
+ ## FTS compatibility matrix
55
+
56
+ Reference: **SQLite 3.51.0** (`bun:sqlite`). Inventory: `bun run scripts/fts-oracle-surface.ts` β†’ `compat/fts-oracle-surface.json`.
57
+
58
+ | Feature | Status |
59
+ | --- | --- |
60
+ | FTS3 | PARTIALLY VERIFIED |
61
+ | FTS4 | PARTIALLY VERIFIED |
62
+ | FTS5 | PARTIALLY VERIFIED |
63
+ | Virtual table creation (options/tokenizers) | VERIFIED |
64
+ | Tokenizers (unicode61/ascii/porter/trigram) | VERIFIED |
65
+ | MATCH grammar (AND/OR/NOT/phrase/prefix/NEAR/columns) | VERIFIED |
66
+ | Boolean operators | VERIFIED |
67
+ | Phrases | VERIFIED |
68
+ | Prefix queries | VERIFIED |
69
+ | NEAR | VERIFIED |
70
+ | Column filters | VERIFIED |
71
+ | Ranking / bm25 / rank | VERIFIED |
72
+ | highlight / snippet | VERIFIED |
73
+ | matchinfo / offsets (FTS3/4) | PARTIALLY VERIFIED | Default + common format strings verified; some FTS4-only formats thinner |
74
+ | Contentless tables | VERIFIED |
75
+ | External content | PARTIALLY VERIFIED | Canonical delete/sync covered; backfill/projection edges thinner |
76
+ | Content tables | VERIFIED |
77
+ | Triggers + FTS | PARTIALLY VERIFIED | Maintenance sequences covered; advanced edges thinner |
78
+ | Special commands (optimize/rebuild/integrity-check) | VERIFIED |
79
+ | Prefix indexes | VERIFIED |
80
+ | Unicode / adversarial corpus | VERIFIED |
81
+ | Transactions / savepoints | VERIFIED |
82
+ | Error behavior | VERIFIED |
83
+ | FTS differential fuzz | VERIFIED |
84
+ | FTS stateful fuzz | VERIFIED |
85
+ | RTREE / dbstat / bytecode / tables_used | VERIFIED | dbstat synthetic pages; bytecode empty cursor |
86
+ | ANALYZE / REINDEX / VACUUM | VERIFIED | `:memory:` observable parity |
87
+ | EXPLAIN / INDEXED BY | PARTIALLY VERIFIED | Stub shapes / no-ops (missing INDEXED BY errors documented) |
88
+ | Prepared stmt schema invalidation | VERIFIED | Re-prepare after ALTER/DROP; `tests/contract/api/schema-invalidation.test.ts` |
89
+ | On-disk file format / C API | NOT APPLICABLE | |
90
+
91
+ ## How to verify
92
+
93
+ ```bash
94
+ bun run test:sqlite-compat # requirements + gate + contract/fuzz/harness
95
+ bun run inventory # oracle function/module inventory
96
+ bun run requirements # refresh SQLite.org requirements + coverage
97
+ bun run build && bun run verify-package # ESM browser build + isomorphic pack gates
98
+ ```
99
+
100
+ Do not treat isolated unit tests of internal modules as proof of SQLite compatibility. The differential matrix runner is authoritative for SQL behavior; `test:sqlite-compat` is the release gate.
101
+
102
+ **Parity claim:** Verified against **SQLite 3.51.0** (`bun:sqlite`). Features marked **VERIFIED** are oracle-proven. **PARTIALLY VERIFIED** rows must not be marketed as complete. **NOT APPLICABLE** is the only allowed permanent omission from the SQL drop-in claim.
package/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Chris Vouga
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.