sqlrite 0.9.1__tar.gz → 0.10.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {sqlrite-0.9.1 → sqlrite-0.10.0}/CLAUDE.md +1 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/Cargo.lock +7 -7
- {sqlrite-0.9.1 → sqlrite-0.10.0}/Cargo.toml +11 -2
- {sqlrite-0.9.1 → sqlrite-0.10.0}/PKG-INFO +1 -1
- {sqlrite-0.9.1 → sqlrite-0.10.0}/README.md +2 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/desktop/package.json +1 -1
- {sqlrite-0.9.1 → sqlrite-0.10.0}/docs/_index.md +5 -1
- {sqlrite-0.9.1 → sqlrite-0.10.0}/docs/benchmarks-plan.md +23 -4
- {sqlrite-0.9.1 → sqlrite-0.10.0}/docs/concurrent-writes-plan.md +20 -8
- sqlrite-0.10.0/docs/concurrent-writes.md +340 -0
- sqlrite-0.10.0/docs/design-decisions.md +537 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/docs/embedding.md +71 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/docs/file-format.md +89 -2
- {sqlrite-0.9.1 → sqlrite-0.10.0}/docs/roadmap.md +228 -6
- {sqlrite-0.9.1 → sqlrite-0.10.0}/docs/smoke-test.md +5 -3
- {sqlrite-0.9.1 → sqlrite-0.10.0}/docs/supported-sql.md +78 -11
- {sqlrite-0.9.1 → sqlrite-0.10.0}/docs/usage.md +31 -1
- {sqlrite-0.9.1 → sqlrite-0.10.0}/examples/README.md +8 -0
- sqlrite-0.10.0/examples/rust/concurrent_writers.rs +85 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/pyproject.toml +1 -1
- {sqlrite-0.9.1 → sqlrite-0.10.0}/sdk/go/README.md +53 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/sdk/go/conn.go +57 -15
- sqlrite-0.10.0/sdk/go/sqlrite.go +444 -0
- sqlrite-0.10.0/sdk/go/sqlrite_test.go +558 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/sdk/python/Cargo.toml +1 -1
- {sqlrite-0.9.1 → sqlrite-0.10.0}/sdk/python/src/lib.rs +109 -11
- {sqlrite-0.9.1 → sqlrite-0.10.0}/sdk/python/tests/test_sqlrite.py +135 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/sqlrite-ask/Cargo.toml +1 -1
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/ask/mod.rs +4 -2
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/ask/schema.rs +2 -1
- sqlrite-0.10.0/src/connection.rs +3238 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/error.rs +31 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/lib.rs +6 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/main.rs +51 -41
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/meta_command/mod.rs +275 -45
- sqlrite-0.10.0/src/mvcc/clock.rs +182 -0
- sqlrite-0.10.0/src/mvcc/log.rs +624 -0
- sqlrite-0.10.0/src/mvcc/mod.rs +134 -0
- sqlrite-0.10.0/src/mvcc/registry.rs +322 -0
- sqlrite-0.10.0/src/mvcc/store.rs +984 -0
- sqlrite-0.10.0/src/mvcc/transaction.rs +262 -0
- sqlrite-0.10.0/src/repl/mod.rs +281 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/db/database.rs +77 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/pager/mod.rs +55 -1
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/pager/pager.rs +64 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/pager/wal.rs +290 -12
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/pragma.rs +49 -1
- sqlrite-0.10.0/web/.gitignore +40 -0
- sqlrite-0.10.0/web/README.md +235 -0
- sqlrite-0.10.0/web/components.json +21 -0
- sqlrite-0.10.0/web/content/blog/adding-vector-search-with-hnsw.mdx +235 -0
- sqlrite-0.10.0/web/content/blog/how-sqlrite-stores-rows-on-disk.mdx +275 -0
- sqlrite-0.10.0/web/content/blog/shipping-sqlrite-tauri-mcp-sdks.mdx +351 -0
- sqlrite-0.10.0/web/content/blog/sqlrite-vs-sqlite-benchmarks.mdx +248 -0
- sqlrite-0.10.0/web/content/blog/why-im-building-sqlrite.mdx +222 -0
- sqlrite-0.10.0/web/eslint.config.mjs +16 -0
- sqlrite-0.10.0/web/next.config.ts +7 -0
- sqlrite-0.10.0/web/package-lock.json +8370 -0
- sqlrite-0.10.0/web/package.json +36 -0
- sqlrite-0.10.0/web/postcss.config.mjs +7 -0
- sqlrite-0.10.0/web/src/app/blog/[slug]/opengraph-image.tsx +32 -0
- sqlrite-0.10.0/web/src/app/blog/[slug]/page.tsx +232 -0
- sqlrite-0.10.0/web/src/app/blog/[slug]/twitter-image.tsx +29 -0
- sqlrite-0.10.0/web/src/app/blog/page.tsx +143 -0
- sqlrite-0.10.0/web/src/app/blog/rss.xml/route.ts +63 -0
- sqlrite-0.10.0/web/src/app/blog/tags/[tag]/page.tsx +109 -0
- sqlrite-0.10.0/web/src/app/docs/opengraph-image.tsx +20 -0
- sqlrite-0.10.0/web/src/app/docs/page.tsx +843 -0
- sqlrite-0.10.0/web/src/app/docs/twitter-image.tsx +20 -0
- sqlrite-0.10.0/web/src/app/globals.css +2377 -0
- sqlrite-0.10.0/web/src/app/layout.tsx +106 -0
- sqlrite-0.10.0/web/src/app/opengraph-image.tsx +20 -0
- sqlrite-0.10.0/web/src/app/page.tsx +99 -0
- sqlrite-0.10.0/web/src/app/robots.ts +15 -0
- sqlrite-0.10.0/web/src/app/sitemap.ts +48 -0
- sqlrite-0.10.0/web/src/app/twitter-image.tsx +20 -0
- sqlrite-0.10.0/web/src/components/architecture.tsx +111 -0
- sqlrite-0.10.0/web/src/components/benchmarks.tsx +158 -0
- sqlrite-0.10.0/web/src/components/blog-mdx.tsx +64 -0
- sqlrite-0.10.0/web/src/components/blog.tsx +81 -0
- sqlrite-0.10.0/web/src/components/cta-strip.tsx +26 -0
- sqlrite-0.10.0/web/src/components/desktop.tsx +117 -0
- sqlrite-0.10.0/web/src/components/features.tsx +133 -0
- sqlrite-0.10.0/web/src/components/footer.tsx +97 -0
- sqlrite-0.10.0/web/src/components/hero.tsx +57 -0
- sqlrite-0.10.0/web/src/components/icons.tsx +47 -0
- sqlrite-0.10.0/web/src/components/install-bar.tsx +54 -0
- sqlrite-0.10.0/web/src/components/nav.tsx +164 -0
- sqlrite-0.10.0/web/src/components/roadmap.tsx +228 -0
- sqlrite-0.10.0/web/src/components/sdk-showcase-client.tsx +91 -0
- sqlrite-0.10.0/web/src/components/sdk-showcase.tsx +186 -0
- sqlrite-0.10.0/web/src/components/sql-ref.tsx +254 -0
- sqlrite-0.10.0/web/src/components/terminal.tsx +193 -0
- sqlrite-0.10.0/web/src/lib/benchmarks.ts +263 -0
- sqlrite-0.10.0/web/src/lib/blog.ts +129 -0
- sqlrite-0.10.0/web/src/lib/highlight.ts +34 -0
- sqlrite-0.10.0/web/src/lib/og.tsx +144 -0
- sqlrite-0.10.0/web/src/lib/site.ts +26 -0
- sqlrite-0.10.0/web/src/lib/utils.ts +6 -0
- sqlrite-0.10.0/web/tsconfig.json +21 -0
- sqlrite-0.9.1/docs/design-decisions.md +0 -289
- sqlrite-0.9.1/sdk/go/sqlrite.go +0 -226
- sqlrite-0.9.1/sdk/go/sqlrite_test.go +0 -297
- sqlrite-0.9.1/src/connection.rs +0 -1292
- sqlrite-0.9.1/src/repl/mod.rs +0 -144
- {sqlrite-0.9.1 → sqlrite-0.10.0}/.github/workflows/ci.yml +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/.github/workflows/release-pr.yml +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/.github/workflows/release.yml +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/.github/workflows/rust.yml +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/.gitignore +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/CODE_OF_CONDUCT.md +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/LICENSE +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/MAINTAINERS +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/Makefile +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/desktop/index.html +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/desktop/package-lock.json +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/desktop/src/App.svelte +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/desktop/src/app.css +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/desktop/src/main.ts +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/desktop/src/vite-env.d.ts +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/desktop/svelte.config.js +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/desktop/tsconfig.json +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/desktop/vite.config.ts +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/docs/architecture.md +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/docs/ask-backend-examples.md +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/docs/ask.md +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/docs/benchmarks.md +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/docs/desktop.md +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/docs/fts.md +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/docs/getting-started.md +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/docs/mcp.md +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/docs/pager.md +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/docs/phase-7-plan.md +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/docs/phase-8-plan.md +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/docs/release-plan.md +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/docs/release-secrets.md +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/docs/sql-engine.md +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/docs/storage-model.md +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/examples/c/Makefile +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/examples/c/hello.c +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/examples/go/go.mod +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/examples/go/hello.go +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/examples/hybrid-retrieval/README.md +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/examples/hybrid-retrieval/hybrid_retrieval.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/examples/nodejs/hello.mjs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/examples/python/hello.py +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/examples/rust/quickstart.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/examples/wasm/Makefile +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/examples/wasm/index.html +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/examples/wasm/server.mjs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/images/SQLRite - Desktop.png +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/images/SQLRite Data Structures.png +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/images/SQLRite Simple SQL Execution High Level Diagram.png +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/images/SQLRite Simple SQL INSERT Execution High Level Diagram (Insert Row).png +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/images/SQLRite Simple SQL INSERT Execution High Level Diagram.png +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/images/SQLRite_logo.png +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/images/architecture.png +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/rust-toolchain.toml +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/samples/AST.delete.example +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/samples/AST.insert.exemple +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/samples/AST.select.example +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/samples/AST.update.example +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/samples/CREATE TABLE sqlrite_schema.sql +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/samples/CREATE_TABLE with duplicate.sql +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/samples/CREATE_TABLE.sql +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/samples/INSERT.sql +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/scripts/bump-version.sh +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/sdk/go/ask.go +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/sdk/go/ask_test.go +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/sdk/go/go.mod +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/sdk/go/rows.go +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/sdk/go/stmt.go +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/sdk/python/README.md +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/sdk/python/tests/test_ask.py +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/sqlrite-ask/README.md +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/sqlrite-ask/src/lib.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/sqlrite-ask/src/prompt.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/sqlrite-ask/src/provider/anthropic.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/sqlrite-ask/src/provider/mock.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/sqlrite-ask/src/provider/mod.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/sqlrite-ask/tests/anthropic_http.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/agg.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/db/mod.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/db/secondary_index.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/db/table.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/dialect.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/executor.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/fts/bm25.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/fts/mod.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/fts/posting_list.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/fts/tokenizer.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/hnsw.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/mod.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/pager/allocator.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/pager/cell.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/pager/file.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/pager/freelist.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/pager/fts_cell.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/pager/header.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/pager/hnsw_cell.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/pager/index_cell.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/pager/interior_page.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/pager/overflow.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/pager/page.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/pager/table_page.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/pager/varint.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/params.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/parser/create.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/parser/insert.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/parser/mod.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/parser/select.rs +0 -0
- {sqlrite-0.9.1 → sqlrite-0.10.0}/src/sql/tokenizer.rs +0 -0
|
@@ -16,6 +16,7 @@ SQLRite is a from-scratch SQLite-style embedded database written in Rust. It's p
|
|
|
16
16
|
- `sqlrite-ffi/` — C ABI cdylib + generated `sqlrite.h` header. Backs the Go SDK and any C consumer.
|
|
17
17
|
- `desktop/` — Tauri 2 + Svelte 5 GUI. Embeds the engine directly (no FFI hop).
|
|
18
18
|
- `benchmarks/` — SQLR-4 / SQLR-16 bench harness. `Driver` trait + SQLRite + SQLite (rusqlite-bundled) drivers + criterion-driven workloads. Excluded from the default CI build/test/clippy/doc commands; run locally with `make bench` (or `make bench-duckdb`). See [docs/benchmarks-plan.md](docs/benchmarks-plan.md).
|
|
19
|
+
- `web/` — marketing + docs site (Next.js 15 + Tailwind v4). Independent of the Cargo workspace; lives in-repo for now but is structured to lift into its own repository later. See [web/README.md](web/README.md).
|
|
19
20
|
|
|
20
21
|
Architecture deep-dive: [docs/architecture.md](docs/architecture.md). The full doc index is [docs/_index.md](docs/_index.md).
|
|
21
22
|
|
|
@@ -4799,7 +4799,7 @@ dependencies = [
|
|
|
4799
4799
|
|
|
4800
4800
|
[[package]]
|
|
4801
4801
|
name = "sqlrite-ask"
|
|
4802
|
-
version = "0.
|
|
4802
|
+
version = "0.10.0"
|
|
4803
4803
|
dependencies = [
|
|
4804
4804
|
"serde",
|
|
4805
4805
|
"serde_json",
|
|
@@ -4827,7 +4827,7 @@ dependencies = [
|
|
|
4827
4827
|
|
|
4828
4828
|
[[package]]
|
|
4829
4829
|
name = "sqlrite-desktop"
|
|
4830
|
-
version = "0.
|
|
4830
|
+
version = "0.10.0"
|
|
4831
4831
|
dependencies = [
|
|
4832
4832
|
"serde",
|
|
4833
4833
|
"serde_json",
|
|
@@ -4839,7 +4839,7 @@ dependencies = [
|
|
|
4839
4839
|
|
|
4840
4840
|
[[package]]
|
|
4841
4841
|
name = "sqlrite-engine"
|
|
4842
|
-
version = "0.
|
|
4842
|
+
version = "0.10.0"
|
|
4843
4843
|
dependencies = [
|
|
4844
4844
|
"clap",
|
|
4845
4845
|
"env_logger",
|
|
@@ -4856,7 +4856,7 @@ dependencies = [
|
|
|
4856
4856
|
|
|
4857
4857
|
[[package]]
|
|
4858
4858
|
name = "sqlrite-ffi"
|
|
4859
|
-
version = "0.
|
|
4859
|
+
version = "0.10.0"
|
|
4860
4860
|
dependencies = [
|
|
4861
4861
|
"cbindgen",
|
|
4862
4862
|
"serde",
|
|
@@ -4866,7 +4866,7 @@ dependencies = [
|
|
|
4866
4866
|
|
|
4867
4867
|
[[package]]
|
|
4868
4868
|
name = "sqlrite-mcp"
|
|
4869
|
-
version = "0.
|
|
4869
|
+
version = "0.10.0"
|
|
4870
4870
|
dependencies = [
|
|
4871
4871
|
"clap",
|
|
4872
4872
|
"libc",
|
|
@@ -4877,7 +4877,7 @@ dependencies = [
|
|
|
4877
4877
|
|
|
4878
4878
|
[[package]]
|
|
4879
4879
|
name = "sqlrite-nodejs"
|
|
4880
|
-
version = "0.
|
|
4880
|
+
version = "0.10.0"
|
|
4881
4881
|
dependencies = [
|
|
4882
4882
|
"napi",
|
|
4883
4883
|
"napi-build",
|
|
@@ -4887,7 +4887,7 @@ dependencies = [
|
|
|
4887
4887
|
|
|
4888
4888
|
[[package]]
|
|
4889
4889
|
name = "sqlrite-python"
|
|
4890
|
-
version = "0.
|
|
4890
|
+
version = "0.10.0"
|
|
4891
4891
|
dependencies = [
|
|
4892
4892
|
"pyo3",
|
|
4893
4893
|
"sqlrite-engine",
|
|
@@ -27,7 +27,7 @@ resolver = "3"
|
|
|
27
27
|
# `package =` key so the import name stays `sqlrite` internally:
|
|
28
28
|
# sqlrite = { package = "sqlrite-engine", path = "…" }
|
|
29
29
|
name = "sqlrite-engine"
|
|
30
|
-
version = "0.
|
|
30
|
+
version = "0.10.0"
|
|
31
31
|
authors = ["Joao Henrique Machado Silva <joaoh82@gmail.com>"]
|
|
32
32
|
edition = "2024"
|
|
33
33
|
rust-version = "1.85"
|
|
@@ -73,6 +73,15 @@ path = "examples/rust/quickstart.rs"
|
|
|
73
73
|
name = "hybrid-retrieval"
|
|
74
74
|
path = "examples/hybrid-retrieval/hybrid_retrieval.rs"
|
|
75
75
|
|
|
76
|
+
# Phase 11.12 — BEGIN CONCURRENT retry-loop demo. Mints a sibling
|
|
77
|
+
# Connection via `Connection::connect`, runs two concurrent
|
|
78
|
+
# transactions (first disjoint, then same-row), and surfaces the
|
|
79
|
+
# Busy / retry path. Run with `cargo run --example concurrent_writers`.
|
|
80
|
+
# See `docs/concurrent-writes.md` for the conceptual walkthrough.
|
|
81
|
+
[[example]]
|
|
82
|
+
name = "concurrent_writers"
|
|
83
|
+
path = "examples/rust/concurrent_writers.rs"
|
|
84
|
+
|
|
76
85
|
[features]
|
|
77
86
|
# Default build includes everything: the REPL binary (cli) and
|
|
78
87
|
# POSIX/Windows advisory file locks on the Pager (file-locks).
|
|
@@ -141,4 +150,4 @@ fs2 = { version = "0.4", optional = true }
|
|
|
141
150
|
# crate publishes to crates.io, and a path-only dep without a
|
|
142
151
|
# version field fails the manifest verification step. See PR #58
|
|
143
152
|
# retrospective in docs/roadmap.md.
|
|
144
|
-
sqlrite-ask = { version = "0.
|
|
153
|
+
sqlrite-ask = { version = "0.10.0", path = "sqlrite-ask", optional = true }
|
|
@@ -11,6 +11,8 @@ Rust-SQLite (SQLRite)
|
|
|
11
11
|
> What I cannot create, I do not understand.
|
|
12
12
|
> — Richard Feynman
|
|
13
13
|
|
|
14
|
+
📖 **Project website + docs:** the marketing + getting-started site is in [`web/`](web/) (Next.js 15 + Tailwind v4). Run `cd web && npm install && npm run dev` to preview locally; deploys to Vercel out of the box.
|
|
15
|
+
|
|
14
16
|
|
|
15
17
|
<table style="width:100%">
|
|
16
18
|
<tr>
|
|
@@ -16,7 +16,8 @@ A small, hand-written guide to the SQLRite codebase — how it's structured, how
|
|
|
16
16
|
## Using SQLRite as a library
|
|
17
17
|
|
|
18
18
|
- [Embedding](embedding.md) — the public `Connection` / `Statement` / `Rows` API (Phase 5a) and where the non-Rust SDKs plug in (Phase 5b – 5g)
|
|
19
|
-
- [`
|
|
19
|
+
- [Concurrent writes — MVCC + `BEGIN CONCURRENT`](concurrent-writes.md) — Phase 11 canonical reference: SQL surface, embedding API, SDK error mapping, REPL meta-commands, durability story, limitations. Design rationale lives in the [historical plan-doc](concurrent-writes-plan.md).
|
|
20
|
+
- [`examples/`](../examples/) — runnable Rust quickstart (`cargo run --example quickstart`) + concurrent-writers retry-loop demo (`cargo run --example concurrent_writers`); language-specific subdirectories fill in as each 5x sub-phase lands
|
|
20
21
|
|
|
21
22
|
## Phase 7 — AI-era extensions
|
|
22
23
|
|
|
@@ -52,6 +53,9 @@ As of May 2026, SQLRite has:
|
|
|
52
53
|
- A Tauri 2.0 + Svelte desktop app (Phase 2.5 complete)
|
|
53
54
|
- AI-era extensions across the product surface (Phase 7 complete): VECTOR columns + HNSW indexes (7a-7d), JSON columns (7e), the `ask()` natural-language → SQL family across the REPL / desktop / Rust / Python / Node / Go / WASM (7g.1-7g.7), and the [`sqlrite-mcp`](mcp.md) Model Context Protocol server (7h + 7g.8)
|
|
54
55
|
- Full-text search + hybrid retrieval (Phase 8 complete): FTS5-style inverted index with BM25 ranking + `fts_match` / `bm25_score` scalar functions + `try_fts_probe` optimizer hook + on-disk persistence with on-demand v4 → v5 file-format bump (8a-8c), a worked hybrid-retrieval example combining BM25 with vector cosine via raw arithmetic (8d), and a `bm25_search` MCP tool symmetric with `vector_search` (8e). See [`docs/fts.md`](fts.md).
|
|
56
|
+
- SQL surface + DX follow-ups (Phase 9 complete, v0.2.0 → v0.9.1): DDL completeness — `DEFAULT`, `DROP TABLE` / `DROP INDEX`, `ALTER TABLE` (9a); free-list + manual `VACUUM` (9b) + auto-VACUUM (9c); `IS NULL` / `IS NOT NULL` (9d); `GROUP BY` + aggregates + `DISTINCT` + `LIKE` + `IN` (9e); four flavors of `JOIN` — INNER, LEFT, RIGHT, FULL OUTER (9f); prepared statements + `?` parameter binding with a per-connection LRU plan cache (9g); HNSW probe widened to cosine + dot via `WITH (metric = …)` (9h); `PRAGMA` dispatcher with the `auto_vacuum` knob (9i)
|
|
57
|
+
- Benchmarks against SQLite + DuckDB (Phase 10 complete, SQLR-4 / SQLR-16): twelve-workload bench harness with a pluggable `Driver` trait, criterion-driven, pinned-host runs published. See [`docs/benchmarks.md`](benchmarks.md).
|
|
58
|
+
- Phase 11 (concurrent writes via MVCC + `BEGIN CONCURRENT`, SQLR-22) is **shipped end-to-end** — `Connection` is `Send + Sync`; `Connection::connect()` mints sibling handles. `sqlrite::mvcc` exposes `MvccClock`, `ActiveTxRegistry`, `MvStore`, `ConcurrentTx`, and the `MvccCommitBatch` / `MvccLogRecord` WAL codec. WAL header v1 → v2 persisted the clock high-water mark; v2 → v3 added typed MVCC log-record frames. `PRAGMA journal_mode = mvcc;` opts a database into MVCC. `BEGIN CONCURRENT` writes commit-validate against `MvStore`, abort with `SQLRiteError::Busy`, and append a typed MVCC log-record frame to the WAL — covered by the same fsync as the legacy page commit. Reopen replays those frames into `MvStore` and seeds `MvccClock` past the highest committed `commit_ts`. Reads via `Statement::query` see the BEGIN-time snapshot. Per-commit GC + `vacuum_mvcc()` bound version-chain growth. C FFI / Python / Node / Go all propagate `Busy` / `BusySnapshot` as typed retryable errors *and* mint sibling handles that share backing state — Go's process-level path registry (Phase 11.11c) handles cross-`*sql.DB` sharing too. The `sqlrite` REPL ships `.spawn` / `.use` / `.conns` for interactive demos; the SQLR-16 benchmark suite adds `W13` (concurrent writers, mostly disjoint rows) as the Phase-11 differentiator workload. The only remaining items are deferred-by-design or foundation work: indexes under MVCC (11.10) and the checkpoint-drain follow-up (parked half of 11.9). **User-facing reference:** [`docs/concurrent-writes.md`](concurrent-writes.md); runnable example at [`examples/rust/concurrent_writers.rs`](../examples/rust/concurrent_writers.rs). Original design proposal: [`docs/concurrent-writes-plan.md`](concurrent-writes-plan.md).
|
|
55
59
|
- A fully-automated release pipeline that ships every product to its registry on every release with one human action — Rust engine + `sqlrite-ask` + `sqlrite-mcp` to crates.io, Python wheels to PyPI (`sqlrite`), Node.js + WASM to npm (`@joaoh82/sqlrite` + `@joaoh82/sqlrite-wasm`), Go module via `sdk/go/v*` git tag, plus C FFI tarballs, MCP binary tarballs, and unsigned desktop installers as GitHub Release assets (Phase 6 complete)
|
|
56
60
|
|
|
57
61
|
See the [Roadmap](roadmap.md) for the full phase plan.
|
|
@@ -111,6 +111,16 @@ For W10/W11, the goal isn't to beat the comparators — they're battle-hardened
|
|
|
111
111
|
|
|
112
112
|
For W12, no off-the-shelf comparator exists in a single embedded engine; the number stands on its own.
|
|
113
113
|
|
|
114
|
+
### Group D — Concurrent writes (Phase 11.11b, the Phase-11 MVCC differentiator)
|
|
115
|
+
|
|
116
|
+
| ID | Name | Shape | Comparator |
|
|
117
|
+
|----|------|-------|------------|
|
|
118
|
+
| W13 | Concurrent writers | 4 worker threads × 50 BEGIN/UPDATE/COMMIT cycles each, random rowid in `1..=1000` (≈ 0.4% collision per op), `UPDATE counters SET n = n + 1 WHERE id = ?` | SQLite (`BEGIN IMMEDIATE` + `busy_timeout = 5s` per-connection) |
|
|
119
|
+
|
|
120
|
+
The headline workload Phase 11's MVCC machinery was designed for. SQLRite drives `BEGIN CONCURRENT` across sibling [`Connection::connect`](../docs/concurrent-writes.md) handles minted from the same process; SQLite drives `BEGIN IMMEDIATE` across separate `rusqlite::Connection` handles serializing through the WAL write lock. Both engines run the same retry-on-busy outer loop ([`is_retryable_busy`](../benchmarks/src/lib.rs) is engine-dispatched); only SQLRite actually exercises the retry path under this workload's shape — the contrast *is* the measurement.
|
|
121
|
+
|
|
122
|
+
Workload parameters live in [`benchmarks/src/workloads/concurrent_writers.rs`](../benchmarks/src/workloads/concurrent_writers.rs) as named constants (`W13_PRELOAD_ROWS`, `W13_N_WORKERS`, `W13_TXS_PER_WORKER`). Bumping any of them is a workload-version bump under Q8.
|
|
123
|
+
|
|
114
124
|
---
|
|
115
125
|
|
|
116
126
|
## Metrics
|
|
@@ -127,9 +137,10 @@ Keep tight. The task brief lists many candidates; the suite measures these:
|
|
|
127
137
|
Explicitly **not** measured in v1:
|
|
128
138
|
|
|
129
139
|
- **CPU%.** Noisy on a shared machine, redundant with wall-clock for single-threaded workloads.
|
|
130
|
-
- **Concurrency curves.** Engine is single-writer by design (Phase 4e). No concurrent-writer workload is meaningful until that changes.
|
|
131
140
|
- **Network I/O.** All targets are in-process.
|
|
132
141
|
|
|
142
|
+
**Updated post-Phase 11.11b:** Group D's `W13` (concurrent writers) lifts the single-writer caveat — SQLRite now has a real multi-writer story via `BEGIN CONCURRENT`, and W13 measures it directly against SQLite's single-writer baseline. Full concurrency-curve sweeps (varying `N` workers and collision rate) are a clean follow-up; v1 reports a single representative point.
|
|
143
|
+
|
|
133
144
|
---
|
|
134
145
|
|
|
135
146
|
## Methodology
|
|
@@ -190,7 +201,8 @@ benchmarks/
|
|
|
190
201
|
│ ├── join.rs — W9
|
|
191
202
|
│ ├── vector.rs — W10
|
|
192
203
|
│ ├── fts.rs — W11
|
|
193
|
-
│
|
|
204
|
+
│ ├── hybrid.rs — W12
|
|
205
|
+
│ └── concurrent_writers.rs — W13 (Phase 11.11b)
|
|
194
206
|
├── benches/
|
|
195
207
|
│ └── suite.rs — single criterion entry point that fans out
|
|
196
208
|
├── scripts/
|
|
@@ -264,11 +276,18 @@ Add the `duckdb-rs` driver under a `--features duckdb` flag. Wire only into Grou
|
|
|
264
276
|
|
|
265
277
|
**Exit criterion:** `docs/benchmarks.md` exists, the README has a "Benchmarks" section pointing at it, the first dated results JSON is committed.
|
|
266
278
|
|
|
267
|
-
###
|
|
279
|
+
### 9.7 — Group D concurrent writers (Phase 11.11b, shipped)
|
|
280
|
+
|
|
281
|
+
Adds `W13` (concurrent writers, mostly-disjoint rows) under a new Group D. The `Driver` trait grows three optional methods (`connect_sibling`, `concurrent_begin_sql`, `is_retryable_busy`) with defaults that make sense for engines without an MVCC story; SQLRite overrides all three. SQLite gains a `busy_timeout = 5s` pragma at open so its `BEGIN IMMEDIATE` blocks rather than fails on contention. The workload lives in [`benchmarks/src/workloads/concurrent_writers.rs`](../benchmarks/src/workloads/concurrent_writers.rs).
|
|
282
|
+
|
|
283
|
+
**Exit criterion:** W13 runs under both drivers, correctness gate passes (`SUM(n) == n_workers * txs_per_worker` after a sample), and the JSON envelope picks up `W13.v1` rows for both drivers.
|
|
284
|
+
|
|
285
|
+
### Post-9.7 ideas (parked)
|
|
268
286
|
|
|
269
287
|
- **libSQL driver** if/when we want a non-extension vector competitor for W10.
|
|
270
288
|
- **Per-PR regression detector.** A GitHub Action that runs the bench on a self-hosted runner and posts a comment if any workload regresses >20% from the last `main` baseline.
|
|
271
|
-
- **Concurrency
|
|
289
|
+
- **Concurrency curves for W13.** Sweep `N` workers (1, 2, 4, 8, 16) and `K` rows (10, 100, 1k, 10k) to chart SQLRite-MVCC's scaling envelope vs SQLite's serial baseline. v1 reports a single representative point; the sweep is a clean follow-up.
|
|
290
|
+
- **W13b hot-row contention.** Same workload, `K = 10` rows instead of 1000 — collision probability climbs to ~40% per op, exercising the retry loop hard. Useful for stressing the GC + retry path under adversarial contention.
|
|
272
291
|
- **Larger datasets (10M, 100M).** v1 is sized for fast iteration on a laptop. A "release-blocker run" config could 100× the row counts.
|
|
273
292
|
|
|
274
293
|
### Total scope estimate
|
|
@@ -1,8 +1,16 @@
|
|
|
1
1
|
# Concurrent writes plan — MVCC + `BEGIN CONCURRENT`
|
|
2
2
|
|
|
3
|
-
**
|
|
3
|
+
> 📘 **Looking for the user-facing reference?** This is the original
|
|
4
|
+
> design proposal, kept as the historical record of the decisions
|
|
5
|
+
> that shaped Phase 11. For the shipped surface — SQL, embedding API,
|
|
6
|
+
> SDK error mapping, REPL meta-commands, durability story,
|
|
7
|
+
> limitations — read [**`concurrent-writes.md`**](concurrent-writes.md)
|
|
8
|
+
> first; come back here when you want the *why* and the
|
|
9
|
+
> sequencing discussion.
|
|
10
|
+
|
|
11
|
+
**Status:** shipped end-to-end through Phase 11.11a (May 2026); a small set of follow-ups remain explicitly parked — see the [roadmap](roadmap.md#phase-11--concurrent-writes-via-mvcc--begin-concurrent-sqlr-22-in-flight--see-concurrent-writes-planmd). Drafted 2026-05-07.
|
|
4
12
|
**Inspiration:** [Turso](https://turso.tech) — a SQLite-compatible engine, written in Rust, that implements multi-version concurrency control to lift SQLite's single-writer ceiling. See [`turso/core/mvcc/`](https://github.com/tursodatabase/turso/tree/main/core/mvcc) and the [Turso concurrent-writes docs](https://docs.turso.tech/tursodb/concurrent-writes).
|
|
5
|
-
**Tracks:** SQLR
|
|
13
|
+
**Tracks:** [SQLR-22](https://app.marvinapp.io/) (Marvin).
|
|
6
14
|
|
|
7
15
|
This document proposes adding **multi-version concurrency control (MVCC)** and a **`BEGIN CONCURRENT`** transaction mode to SQLRite, enabling multiple writers in the same process to make progress in parallel under snapshot isolation, with row-level write-write conflict detection at commit. It is intentionally a *plan* — there is no code yet.
|
|
8
16
|
|
|
@@ -270,9 +278,11 @@ Goal: more than one `Connection` can target the same `Database` within a process
|
|
|
270
278
|
|
|
271
279
|
### Phase 10.5 — Checkpoint + crash recovery
|
|
272
280
|
|
|
273
|
-
-
|
|
274
|
-
|
|
275
|
-
-
|
|
281
|
+
> **Status (roadmap 11.9 — May 2026):** The crash-recovery half landed in roadmap Phase 11.9. WAL format is bumped to v3; commits append a typed `MvccCommitBatch` frame before the legacy save's fsync; reopen replays those frames into `MvStore` and seeds `MvccClock` past the highest `commit_ts`. The checkpoint-drain half — folding MVCC log records into pager-level updates and re-enabling the `Mvcc → Wal` journal-mode downgrade — is the remaining slice and stays parked for a follow-up.
|
|
282
|
+
|
|
283
|
+
- ~~Extend the checkpointer to drain MVCC log records into pager-level updates before folding the WAL into the main file.~~ *Deferred — see status note above.*
|
|
284
|
+
- Crash recovery: on open, replay WAL log records into `MvStore`, then replay pager-level commit frames as today. **(Shipped — 11.9.)**
|
|
285
|
+
- Tests: kill the process mid-MVCC-commit (between log-record append and version-chain push), reopen, verify the committed transaction is visible and the half-written one is not. **(Shipped — 11.9 covers the clean-drop case which exercises the same recovery codepath; a real OS-kill test is parked with the checkpoint-drain follow-up.)**
|
|
276
286
|
|
|
277
287
|
### Phase 10.6 — Garbage collection
|
|
278
288
|
|
|
@@ -294,9 +304,11 @@ Index maintenance under MVCC is hard enough that Turso explicitly punted on it.
|
|
|
294
304
|
|
|
295
305
|
### Phase 10.9 — Docs
|
|
296
306
|
|
|
297
|
-
|
|
298
|
-
|
|
299
|
-
-
|
|
307
|
+
> **Status (roadmap 11.12 — May 2026):** Shipped. The canonical user-facing reference at [`docs/concurrent-writes.md`](concurrent-writes.md) covers the SQL surface, embedding API, SDK error mapping, REPL meta-commands, durability story, and limitations as of Phase 11.11a. This plan-doc is now the historical record. Cross-references in `_index.md`, `supported-sql.md`, `embedding.md`, and `design-decisions.md` point at the canonical doc; a runnable example lives at [`examples/rust/concurrent_writers.rs`](../examples/rust/concurrent_writers.rs).
|
|
308
|
+
|
|
309
|
+
- Promote this plan to `docs/concurrent-writes.md` (the canonical user-facing reference), keeping `concurrent-writes-plan.md` as the historical design document. **(Shipped — 11.12.)**
|
|
310
|
+
- Update [roadmap.md](roadmap.md), [`docs/_index.md`](_index.md), [supported-sql.md](supported-sql.md), [embedding.md](embedding.md), [design-decisions.md](design-decisions.md). **(Shipped — 11.12.)**
|
|
311
|
+
- Add a worked example under `examples/rust/concurrent_writers.rs`. **(Shipped — 11.12.)**
|
|
300
312
|
|
|
301
313
|
---
|
|
302
314
|
|
|
@@ -0,0 +1,340 @@
|
|
|
1
|
+
# Concurrent writes — MVCC + `BEGIN CONCURRENT`
|
|
2
|
+
|
|
3
|
+
User-facing reference for SQLRite's multi-version concurrency control. For the original design discussion + sequencing decisions, see [`concurrent-writes-plan.md`](concurrent-writes-plan.md); this doc covers the *shipped* surface as of Phase 11.11a (May 2026).
|
|
4
|
+
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
## TL;DR
|
|
8
|
+
|
|
9
|
+
```sql
|
|
10
|
+
PRAGMA journal_mode = mvcc; -- once per database
|
|
11
|
+
BEGIN CONCURRENT;
|
|
12
|
+
UPDATE accounts SET balance = balance - 50 WHERE id = 1;
|
|
13
|
+
UPDATE accounts SET balance = balance + 50 WHERE id = 2;
|
|
14
|
+
COMMIT; -- may return Busy → caller retries
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
Two writers on *disjoint* rows now make progress in parallel; two writers on the *same* row see the second commit fail fast with [`SQLRiteError::Busy`](../src/error.rs), which the caller retries. The data structure backing this is a per-row in-memory version chain ([`MvStore`](../src/mvcc/store.rs)) sitting in front of the existing pager; the on-disk format is unchanged — durability piggybacks on the WAL via a new `MvccCommitBatch` frame (Phase 11.9). Reads inside a `BEGIN CONCURRENT` transaction see a stable BEGIN-time snapshot.
|
|
18
|
+
|
|
19
|
+
The story is the same one Turso ships with `--experimental-mvcc`, narrowed for SQLRite's single-process scope.
|
|
20
|
+
|
|
21
|
+
---
|
|
22
|
+
|
|
23
|
+
## Why MVCC
|
|
24
|
+
|
|
25
|
+
SQLite (and pre-Phase-11 SQLRite) serializes every writer through a single exclusive lock. Two writers touching *unrelated rows* still wait on each other — the lock is page- or file-granularity, not row-granularity. For workloads where most writes don't actually conflict, that's throughput left on the table.
|
|
26
|
+
|
|
27
|
+
Phase 11 replaces the lock-for-the-whole-transaction model with **optimistic concurrency control**: writes run against a per-transaction snapshot; the engine only checks for conflicts at `COMMIT`, and only on the row IDs the transaction actually touched. The shape is straight out of [Hekaton (Larson et al., VLDB 2011)](https://www.microsoft.com/en-us/research/wp-content/uploads/2011/01/main-mem-cc-techreport.pdf).
|
|
28
|
+
|
|
29
|
+
What you get:
|
|
30
|
+
|
|
31
|
+
- **Disjoint-row writers run in parallel.** A `BEGIN CONCURRENT` on connection A and one on connection B can both progress; commit ordering is decided by a process-wide logical clock, not by lock acquisition.
|
|
32
|
+
- **Snapshot-isolated reads.** A reader inside `BEGIN CONCURRENT` sees the database as it was at BEGIN time, regardless of what other writers commit in the meantime.
|
|
33
|
+
- **Row-level conflict detection.** The unit of conflict is `(table, rowid)`, not a page or a table.
|
|
34
|
+
- **Same on-disk format.** Existing `.sqlrite` files open unchanged. The toggle is `PRAGMA journal_mode = mvcc;`.
|
|
35
|
+
|
|
36
|
+
What you don't get (v0; see [Limitations](#limitations)):
|
|
37
|
+
|
|
38
|
+
- Cross-process MVCC. The version index is in-memory only; multi-process writers still serialize through the pager's `flock`.
|
|
39
|
+
- `CREATE INDEX` while `journal_mode = mvcc`. Index maintenance under MVCC is Phase 11.10 (deferred-by-design).
|
|
40
|
+
- DDL inside `BEGIN CONCURRENT`. Rejected with a typed error; commit your DDL outside the concurrent transaction.
|
|
41
|
+
|
|
42
|
+
---
|
|
43
|
+
|
|
44
|
+
## Quick start
|
|
45
|
+
|
|
46
|
+
```sql
|
|
47
|
+
-- 1. Opt the database into MVCC. Per-database setting; survives reopens.
|
|
48
|
+
PRAGMA journal_mode = mvcc;
|
|
49
|
+
|
|
50
|
+
-- 2. Multi-row update inside a concurrent transaction.
|
|
51
|
+
BEGIN CONCURRENT;
|
|
52
|
+
INSERT INTO orders (id, customer, total) VALUES (1, 'alice', 100);
|
|
53
|
+
UPDATE inventory SET stock = stock - 1 WHERE sku = 'WIDGET-A';
|
|
54
|
+
COMMIT;
|
|
55
|
+
|
|
56
|
+
-- 3. If two concurrent transactions touch the same row, the second
|
|
57
|
+
-- commit fails with Busy. Retry with a fresh BEGIN CONCURRENT.
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
The same end-to-end thing from Rust:
|
|
61
|
+
|
|
62
|
+
```rust
|
|
63
|
+
use sqlrite::{Connection, SQLRiteError};
|
|
64
|
+
|
|
65
|
+
let mut conn = Connection::open("orders.sqlrite")?;
|
|
66
|
+
conn.execute("PRAGMA journal_mode = mvcc")?;
|
|
67
|
+
|
|
68
|
+
loop {
|
|
69
|
+
conn.execute("BEGIN CONCURRENT")?;
|
|
70
|
+
conn.execute("INSERT INTO orders (id, customer, total) VALUES (1, 'alice', 100)")?;
|
|
71
|
+
conn.execute("UPDATE inventory SET stock = stock - 1 WHERE sku = 'WIDGET-A'")?;
|
|
72
|
+
match conn.execute("COMMIT") {
|
|
73
|
+
Ok(_) => break,
|
|
74
|
+
Err(e) if e.is_retryable() => {
|
|
75
|
+
conn.execute("ROLLBACK").ok();
|
|
76
|
+
continue;
|
|
77
|
+
}
|
|
78
|
+
Err(e) => return Err(e.into()),
|
|
79
|
+
}
|
|
80
|
+
}
|
|
81
|
+
# Ok::<(), sqlrite::SQLRiteError>(())
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
[`SQLRiteError::is_retryable`](../src/error.rs) covers both `Busy` (write-write conflict at commit) and `BusySnapshot` (the snapshot the read path expected has been GC'd) — see [Error semantics](#error-semantics).
|
|
85
|
+
|
|
86
|
+
A complete runnable version of this loop lives in [`examples/rust/concurrent_writers.rs`](../examples/rust/concurrent_writers.rs).
|
|
87
|
+
|
|
88
|
+
---
|
|
89
|
+
|
|
90
|
+
## Conceptual model
|
|
91
|
+
|
|
92
|
+
### The version chain
|
|
93
|
+
|
|
94
|
+
For every `(table, rowid)` SQLRite has touched under `BEGIN CONCURRENT`, the [`MvStore`](../src/mvcc/store.rs) holds an ordered chain of `RowVersion`s:
|
|
95
|
+
|
|
96
|
+
```
|
|
97
|
+
begin=ts1 begin=ts3 begin=ts7
|
|
98
|
+
end=Some(ts3) end=Some(ts7) end=None
|
|
99
|
+
┌────────────┐ ┌────────────┐ ┌────────────┐
|
|
100
|
+
rowid 42 ─→ │ Present { │ ──next──→ │ Present { │ ──next────→ │ Tombstone │
|
|
101
|
+
│ balance: │ │ balance: │ │ (DELETE) │
|
|
102
|
+
│ 100 │ │ 150 │ │ │
|
|
103
|
+
│ } │ │ } │ │ │
|
|
104
|
+
└────────────┘ └────────────┘ └────────────┘
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
A version is **visible** to a transaction with begin-timestamp `T` when `begin <= T < end` (the textbook snapshot-isolation rule). New writes push a new head onto the chain at commit time, capping the previous latest version's `end` to the new `commit_ts`.
|
|
108
|
+
|
|
109
|
+
### Timestamps come from a process-wide logical clock
|
|
110
|
+
|
|
111
|
+
[`MvccClock`](../src/mvcc/clock.rs) is an `AtomicU64` that hands out `begin_ts` at `BEGIN CONCURRENT` and `commit_ts` at the start of validation. The clock's high-water mark is persisted in the WAL header (Phase 11.2's WAL v2) and seeded past the highest replayed `commit_ts` on reopen (Phase 11.9), so timestamps don't reuse the same value across restarts.
|
|
112
|
+
|
|
113
|
+
### Commit-time validation
|
|
114
|
+
|
|
115
|
+
When a `BEGIN CONCURRENT` transaction commits, the engine:
|
|
116
|
+
|
|
117
|
+
1. Allocates a `commit_ts` from the clock.
|
|
118
|
+
2. Walks the write-set. For each `(table, rowid)`, if any committed version's `begin > tx.begin_ts`, somebody else superseded us → return `SQLRiteError::Busy`.
|
|
119
|
+
3. Otherwise, for each row in the write-set, push a new `RowVersion` onto the chain at `commit_ts`, capping the previous latest's `end`.
|
|
120
|
+
4. Append an `MvccCommitBatch` frame to the WAL; the legacy page-commit's fsync covers it (Phase 11.9).
|
|
121
|
+
5. Mirror the writes into `Database::tables` so the legacy read path stays correct after commit.
|
|
122
|
+
6. Drop the transaction's `TxHandle` and run a per-commit GC sweep over the write-set's chains.
|
|
123
|
+
|
|
124
|
+
### Reads
|
|
125
|
+
|
|
126
|
+
Reads via [`Statement::query`](../src/connection.rs) (Phase 11.5) consult `MvStore` first when a `BEGIN CONCURRENT` is open on the connection. If the row has a version visible at the transaction's `begin_ts`, that's the answer; otherwise the read falls through to the legacy table → pager path. This means a reader inside `BEGIN CONCURRENT` sees a consistent BEGIN-time snapshot for as long as the transaction is open.
|
|
127
|
+
|
|
128
|
+
Reads *outside* `BEGIN CONCURRENT` still go through the legacy path — they see the latest committed state, exactly as before Phase 11. That's the keystone of the design: nothing about the existing non-concurrent codepath changed; MVCC is layered on top, opt-in.
|
|
129
|
+
|
|
130
|
+
---
|
|
131
|
+
|
|
132
|
+
## SQL surface
|
|
133
|
+
|
|
134
|
+
### `PRAGMA journal_mode`
|
|
135
|
+
|
|
136
|
+
| Form | Effect |
|
|
137
|
+
|---|---|
|
|
138
|
+
| `PRAGMA journal_mode;` | Read — returns the current mode as a single-row `wal` / `mvcc` result |
|
|
139
|
+
| `PRAGMA journal_mode = mvcc;` | Switch this database into MVCC mode |
|
|
140
|
+
| `PRAGMA journal_mode = wal;` | Switch back to the legacy WAL-backed pager |
|
|
141
|
+
|
|
142
|
+
Case-insensitive on both the pragma name and the value. Quoted values (`'mvcc'`) work; numeric values are rejected. Unknown modes return a typed error.
|
|
143
|
+
|
|
144
|
+
The setting is **per-database**, not per-connection — every [`Connection::connect`](#sibling-handles) sibling sees the same value. Switching `Mvcc → Wal` is rejected if `MvStore` carries committed versions; call [`Connection::vacuum_mvcc`](#vacuum_mvcc) first to drain the store.
|
|
145
|
+
|
|
146
|
+
### `BEGIN CONCURRENT`
|
|
147
|
+
|
|
148
|
+
Opens a concurrent transaction. Requires `PRAGMA journal_mode = mvcc;` first.
|
|
149
|
+
|
|
150
|
+
```sql
|
|
151
|
+
BEGIN CONCURRENT;
|
|
152
|
+
-- DML against the per-tx snapshot
|
|
153
|
+
SELECT … ; -- sees BEGIN-time state
|
|
154
|
+
INSERT … ;
|
|
155
|
+
UPDATE … ;
|
|
156
|
+
DELETE … ;
|
|
157
|
+
COMMIT; -- or ROLLBACK
|
|
158
|
+
```
|
|
159
|
+
|
|
160
|
+
Rules (each surfaces as a typed error):
|
|
161
|
+
|
|
162
|
+
- Plain `BEGIN CONCURRENT` against a `Wal`-mode database is rejected.
|
|
163
|
+
- Nested transactions (`BEGIN CONCURRENT` inside an open one, or `BEGIN` inside one) are rejected.
|
|
164
|
+
- DDL inside `BEGIN CONCURRENT` is rejected — `CREATE TABLE`, `CREATE INDEX`, `DROP TABLE`, `DROP INDEX`, `ALTER TABLE`, `VACUUM` all bounce, the transaction stays open so the caller can `ROLLBACK`.
|
|
165
|
+
- Read-only databases reject `BEGIN CONCURRENT`.
|
|
166
|
+
|
|
167
|
+
`COMMIT` may surface `SQLRiteError::Busy` or `SQLRiteError::BusySnapshot`. The transaction is dropped on either; the caller's loop should `continue` after a `ROLLBACK`.
|
|
168
|
+
|
|
169
|
+
### `COMMIT` / `ROLLBACK`
|
|
170
|
+
|
|
171
|
+
Inside an open `BEGIN CONCURRENT`, plain `COMMIT` validates the write-set and either commits or returns `Busy`. Plain `ROLLBACK` drops the per-tx state and returns control. Both also work outside `BEGIN CONCURRENT` (they fall through to the legacy single-writer transaction control).
|
|
172
|
+
|
|
173
|
+
---
|
|
174
|
+
|
|
175
|
+
## Embedding API
|
|
176
|
+
|
|
177
|
+
### Sibling handles
|
|
178
|
+
|
|
179
|
+
A single `Connection::open` is the only path that touches the file. Mint additional handles with [`Connection::connect`](../src/connection.rs):
|
|
180
|
+
|
|
181
|
+
```rust
|
|
182
|
+
let primary = Connection::open("orders.sqlrite")?;
|
|
183
|
+
let secondary = primary.connect();
|
|
184
|
+
let tertiary = primary.connect();
|
|
185
|
+
```
|
|
186
|
+
|
|
187
|
+
Every sibling shares the same `Arc<Mutex<Database>>`. Each sibling can hold its own independent `BEGIN CONCURRENT` — that's the whole point of multi-handle MVCC. Sibling handles are `Send + Sync`, so it's safe to send them across threads.
|
|
188
|
+
|
|
189
|
+
Sibling propagation across each SDK (Phase 11.7 + 11.8):
|
|
190
|
+
|
|
191
|
+
| SDK | Sibling API | Retryable-error type |
|
|
192
|
+
|---|---|---|
|
|
193
|
+
| C FFI | `sqlrite_connect_sibling(existing, out)` | `SqlriteStatus::Busy` / `BusySnapshot`; `sqlrite_status_is_retryable` |
|
|
194
|
+
| Python | `conn.connect()` | `sqlrite.BusyError` / `sqlrite.BusySnapshotError` (both subclass `SQLRiteError`) |
|
|
195
|
+
| Node.js | `db.connect()` | `errorKind(message)` returns `'Busy'` / `'BusySnapshot'` / `'Other'` |
|
|
196
|
+
| Go | `database/sql` pool + cross-pool path registry (Phase 11.11c) | `errors.Is(err, sqlrite.ErrBusy)` / `ErrBusySnapshot`; `sqlrite.IsRetryable(err)` |
|
|
197
|
+
| WASM | *(deferred — single-threaded runtime)* | *(deferred)* |
|
|
198
|
+
|
|
199
|
+
For Go, every `sql.Open("sqlrite", path)` against a file-backed read-write DB routes through a process-level path registry (Phase 11.11c) — multiple `sql.Open` calls for the same canonical path mint sibling handles off a shared primary, so each `*sql.DB`'s pool can issue its own `BEGIN CONCURRENT` against the same backing engine. `:memory:` opens stay isolated by design; read-only opens (via `sqlrite.OpenReadOnly`) take a shared lock and bypass the registry. See [`sdk/go/README.md`](../sdk/go/README.md#multi-handle-reads--writes-phase-1111c) for the runnable cross-pool example.
|
|
200
|
+
|
|
201
|
+
### The retry loop
|
|
202
|
+
|
|
203
|
+
The canonical shape is the same in every language:
|
|
204
|
+
|
|
205
|
+
```rust
|
|
206
|
+
loop {
|
|
207
|
+
conn.execute("BEGIN CONCURRENT")?;
|
|
208
|
+
conn.execute(/* writes */)?;
|
|
209
|
+
match conn.execute("COMMIT") {
|
|
210
|
+
Ok(_) => break,
|
|
211
|
+
Err(e) if e.is_retryable() => {
|
|
212
|
+
conn.execute("ROLLBACK").ok();
|
|
213
|
+
continue;
|
|
214
|
+
}
|
|
215
|
+
Err(e) => return Err(e.into()),
|
|
216
|
+
}
|
|
217
|
+
}
|
|
218
|
+
```
|
|
219
|
+
|
|
220
|
+
SQLRite intentionally **does not** ship an automatic-backoff retry helper — the right policy (immediate retry, exponential backoff, capped attempts, jittered, etc.) depends on the workload. The retryable-error classification is the only piece the SDK guarantees.
|
|
221
|
+
|
|
222
|
+
### `Connection::vacuum_mvcc` <a id="vacuum_mvcc"></a>
|
|
223
|
+
|
|
224
|
+
Per-commit GC sweeps the write-set's chains automatically. For a deterministic full drain (memory-pressure testing, debug snapshots, `Mvcc → Wal` downgrade prep), call [`conn.vacuum_mvcc()`](../src/connection.rs) — returns the count of versions reclaimed across the whole store. Both paths are safe against in-flight readers: a reader inside `BEGIN CONCURRENT` keeps every version its `begin_ts` snapshot still needs visible.
|
|
225
|
+
|
|
226
|
+
---
|
|
227
|
+
|
|
228
|
+
## REPL multi-handle demo (Phase 11.11a)
|
|
229
|
+
|
|
230
|
+
The `sqlrite` REPL ships with three meta-commands for interactive MVCC demos. The prompt always shows the active handle (`sqlrite[A]>`, `sqlrite[B]>`):
|
|
231
|
+
|
|
232
|
+
| Command | Effect |
|
|
233
|
+
|---|---|
|
|
234
|
+
| `.spawn` | Mint a sibling handle off the active one and switch to it |
|
|
235
|
+
| `.use NAME` | Switch the active handle (case-insensitive); errors with the list of valid names on miss |
|
|
236
|
+
| `.conns` | List every handle, mark the active one with `*`, tag handles in an open `BEGIN CONCURRENT` |
|
|
237
|
+
|
|
238
|
+
End-to-end demo:
|
|
239
|
+
|
|
240
|
+
```text
|
|
241
|
+
sqlrite[A]> PRAGMA journal_mode = mvcc;
|
|
242
|
+
sqlrite[A]> CREATE TABLE t (id INTEGER PRIMARY KEY, v INTEGER);
|
|
243
|
+
sqlrite[A]> INSERT INTO t (id, v) VALUES (1, 0);
|
|
244
|
+
sqlrite[A]> .spawn
|
|
245
|
+
Spawned sibling handle 'B' and switched to it. 2 handles open.
|
|
246
|
+
sqlrite[B]> .use A
|
|
247
|
+
sqlrite[A]> BEGIN CONCURRENT;
|
|
248
|
+
sqlrite[A]> UPDATE t SET v = 100 WHERE id = 1;
|
|
249
|
+
sqlrite[A]> .conns
|
|
250
|
+
2 handle(s):
|
|
251
|
+
* A (BEGIN CONCURRENT)
|
|
252
|
+
B
|
|
253
|
+
sqlrite[A]> .use B
|
|
254
|
+
sqlrite[B]> BEGIN CONCURRENT;
|
|
255
|
+
sqlrite[B]> UPDATE t SET v = 200 WHERE id = 1;
|
|
256
|
+
sqlrite[B]> COMMIT;
|
|
257
|
+
sqlrite[B]> .use A
|
|
258
|
+
sqlrite[A]> COMMIT;
|
|
259
|
+
An error occured: Busy: write-write conflict on t/1: another transaction
|
|
260
|
+
committed this row at ts=3 (after our begin_ts=1); transaction rolled
|
|
261
|
+
back, retry with a fresh BEGIN CONCURRENT
|
|
262
|
+
sqlrite[A]> .use B
|
|
263
|
+
sqlrite[B]> SELECT * FROM t;
|
|
264
|
+
+----+-----+
|
|
265
|
+
| id | v |
|
|
266
|
+
+----+-----+
|
|
267
|
+
| 1 | 200 |
|
|
268
|
+
+----+-----+
|
|
269
|
+
```
|
|
270
|
+
|
|
271
|
+
---
|
|
272
|
+
|
|
273
|
+
## Error semantics
|
|
274
|
+
|
|
275
|
+
| Variant | When | Retryable |
|
|
276
|
+
|---|---|---|
|
|
277
|
+
| `SQLRiteError::Busy` | A `BEGIN CONCURRENT` `COMMIT` lost the validation race — some other transaction superseded one of our row writes after our `begin_ts` | yes |
|
|
278
|
+
| `SQLRiteError::BusySnapshot` | A snapshot the read path expected has been GC'd; surfaces from `Statement::query` when a long-lived reader's `begin_ts` predates the GC watermark | yes |
|
|
279
|
+
| Any other variant | Programming error or storage failure — not retryable | no |
|
|
280
|
+
|
|
281
|
+
`SQLRiteError::is_retryable()` is the single classifier — every SDK's retryable-error helper is a wrapper over the same predicate.
|
|
282
|
+
|
|
283
|
+
---
|
|
284
|
+
|
|
285
|
+
## Durability and recovery
|
|
286
|
+
|
|
287
|
+
### WAL log records (Phase 11.9)
|
|
288
|
+
|
|
289
|
+
Every successful `BEGIN CONCURRENT` commit writes **two** WAL records: the legacy per-page commit frames *and* a new typed `MvccCommitBatch` frame distinguished by the sentinel `page_num = u32::MAX`. The MVCC frame is appended buffered; the legacy save's commit-frame fsync covers both — so a crash between commits either keeps both writes or loses both.
|
|
290
|
+
|
|
291
|
+
The MVCC frame body encodes `commit_ts + record stream` (per-record: op tag, table name, rowid, optional column-value pairs). The encoder caps each batch at 4 KiB (the frame body size); multi-frame batches for very large transactions are a deferred follow-up.
|
|
292
|
+
|
|
293
|
+
### Reopen replay
|
|
294
|
+
|
|
295
|
+
`pager::open_database` walks every recovered MVCC frame and re-pushes the row versions into `MvStore` via `MvStore::push_committed`. The `MvccClock` is seeded past `max(WAL header's clock_high_water, max(commit_ts among replayed batches))` so post-restart transactions can never hand out a regressed `begin_ts`.
|
|
296
|
+
|
|
297
|
+
### What's parked
|
|
298
|
+
|
|
299
|
+
The checkpoint half of plan-doc §10.5 — folding MVCC log records back into pager-level updates so a WAL truncate doesn't lose them, and re-enabling the `Mvcc → Wal` journal-mode downgrade once the store is drainable — is the remaining slice. The legacy save mirror still covers durability of the visible row state on the read path, so the gap is foundation work, not a correctness regression.
|
|
300
|
+
|
|
301
|
+
### WAL format version
|
|
302
|
+
|
|
303
|
+
| Version | Adds |
|
|
304
|
+
|---|---|
|
|
305
|
+
| v1 | Pre-Phase-11 baseline. Reads cleanly today. |
|
|
306
|
+
| v2 (Phase 11.2) | `clock_high_water: u64` in the WAL header (bytes 24..32) |
|
|
307
|
+
| v3 (Phase 11.9) | MVCC log-record frames (`page_num = u32::MAX`) |
|
|
308
|
+
|
|
309
|
+
Decoders accept v1..=v3. A v2 reader on a v3 WAL emits a clean "unsupported WAL format version" diagnostic instead of silently dropping MVCC frames.
|
|
310
|
+
|
|
311
|
+
---
|
|
312
|
+
|
|
313
|
+
## Limitations
|
|
314
|
+
|
|
315
|
+
- **`CREATE INDEX` is rejected while `journal_mode = mvcc`.** Index maintenance under MVCC is Phase 11.10 (deferred-by-design — Turso explicitly punted on the same problem).
|
|
316
|
+
- **DDL inside `BEGIN CONCURRENT` is rejected.** Run DDL outside the concurrent transaction, then begin a fresh one.
|
|
317
|
+
- **Cross-process MVCC is out of scope.** The version index is in-memory only; multi-process writers still serialize through the pager's `flock(LOCK_EX)`. SQLRite has no shared-memory coordination file.
|
|
318
|
+
- **No automatic backoff in retry helpers.** Callers pick the policy.
|
|
319
|
+
- **FTS / HNSW indexes are not maintained inside `BEGIN CONCURRENT`.** The per-row commit-apply path covers B-tree secondary indexes only; tables under MVCC writers shouldn't have FTS or HNSW indexes attached if you need the search index to stay current.
|
|
320
|
+
- **`AUTOINCREMENT` is not specifically guarded** — two concurrent INSERTs that each allocate the same rowid surface as `Busy` at the second commit. The plan's "reject AUTOINCREMENT under MVCC" gate is a clean follow-up.
|
|
321
|
+
- **Memory growth is bounded only via GC.** Per-commit sweeps + `vacuum_mvcc()` cover most cases; for adversarial workloads where readers hold long-lived `begin_ts` snapshots, the chains can grow until the longest-lived reader closes.
|
|
322
|
+
- **Bottom-up B-tree rebuild on every save.** The architectural mismatch flagged in the plan-doc still applies. MVCC amortizes the rebuild to checkpoint time only once the checkpoint-drain follow-up lands; until then, every concurrent commit's mirror write to `Database::tables` triggers the legacy `save_database` rebuild path. Fine for v0 workloads; will matter at scale.
|
|
323
|
+
|
|
324
|
+
---
|
|
325
|
+
|
|
326
|
+
## See also
|
|
327
|
+
|
|
328
|
+
- [`docs/concurrent-writes-plan.md`](concurrent-writes-plan.md) — original design proposal + sequencing decisions. Historical; the current doc reflects shipped reality.
|
|
329
|
+
- [`docs/supported-sql.md`](supported-sql.md) — full SQL reference; the `PRAGMA journal_mode` and `BEGIN CONCURRENT` sections cross-link here.
|
|
330
|
+
- [`docs/embedding.md`](embedding.md) — embedding API + multi-handle examples.
|
|
331
|
+
- [`docs/file-format.md`](file-format.md) — WAL frame layout, MVCC log-record body, clock-high-water field.
|
|
332
|
+
- [`docs/design-decisions.md`](design-decisions.md) §12a–§12h — the design notes accumulated across Phase 11 sub-phases.
|
|
333
|
+
- [`docs/roadmap.md`](roadmap.md#phase-11--concurrent-writes-via-mvcc--begin-concurrent-sqlr-22-in-flight--see-concurrent-writes-planmd) — phase-by-phase shipped vs deferred status.
|
|
334
|
+
- [`examples/rust/concurrent_writers.rs`](../examples/rust/concurrent_writers.rs) — runnable retry-loop example.
|
|
335
|
+
|
|
336
|
+
External:
|
|
337
|
+
|
|
338
|
+
- [Turso concurrent writes](https://docs.turso.tech/tursodb/concurrent-writes) — the direct inspiration; we cite their issues throughout the plan-doc.
|
|
339
|
+
- [Hekaton (Larson et al., VLDB 2011)](https://www.microsoft.com/en-us/research/wp-content/uploads/2011/01/main-mem-cc-techreport.pdf) — the optimistic MVCC paper Turso (and now SQLRite) builds on.
|
|
340
|
+
- [Hermitage anomaly test suite](https://github.com/ept/hermitage) — snapshot-isolation conformance bar; SQLRite has not yet ported these (a clean follow-up).
|