sqlrite 0.1.10__tar.gz → 0.1.12__tar.gz

{sqlrite-0.1.10 → sqlrite-0.1.12}/Cargo.lock

@@ -3736,7 +3736,7 @@ dependencies = [
 
 [[package]]
 name = "sqlrite-desktop"
-version = "0.1.10"
+version = "0.1.12"
 dependencies = [
  "serde",
  "serde_json",
@@ -3748,7 +3748,7 @@ dependencies = [
 
 [[package]]
 name = "sqlrite-engine"
-version = "0.1.10"
+version = "0.1.12"
 dependencies = [
  "clap",
  "env_logger",
@@ -3763,7 +3763,7 @@ dependencies = [
 
 [[package]]
 name = "sqlrite-ffi"
-version = "0.1.10"
+version = "0.1.12"
 dependencies = [
  "cbindgen",
  "sqlrite-engine",
@@ -3771,7 +3771,7 @@ dependencies = [
 
 [[package]]
 name = "sqlrite-nodejs"
-version = "0.1.10"
+version = "0.1.12"
 dependencies = [
  "napi",
  "napi-build",
@@ -3781,7 +3781,7 @@ dependencies = [
 
 [[package]]
 name = "sqlrite-python"
-version = "0.1.10"
+version = "0.1.12"
 dependencies = [
  "pyo3",
  "sqlrite-engine",

{sqlrite-0.1.10 → sqlrite-0.1.12}/Cargo.toml

@@ -27,7 +27,7 @@ resolver = "3"
 # `package =` key so the import name stays `sqlrite` internally:
 #     sqlrite = { package = "sqlrite-engine", path = "…" }
 name = "sqlrite-engine"
-version = "0.1.10"
+version = "0.1.12"
 authors = ["Joao Henrique Machado Silva <joaoh82@gmail.com>"]
 edition = "2024"
 rust-version = "1.85"

{sqlrite-0.1.10 → sqlrite-0.1.12}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: sqlrite
-Version: 0.1.10
+Version: 0.1.12
 Classifier: Development Status :: 3 - Alpha
 Classifier: Intended Audience :: Developers
 Classifier: License :: OSI Approved :: MIT License

{sqlrite-0.1.10 → sqlrite-0.1.12}/desktop/package.json

@@ -1,7 +1,7 @@
 {
   "name": "sqlrite-desktop-frontend",
   "private": true,
-  "version": "0.1.10",
+  "version": "0.1.12",
   "type": "module",
   "scripts": {
     "dev": "vite",

{sqlrite-0.1.10 → sqlrite-0.1.12}/docs/phase-7-plan.md

@@ -106,24 +106,33 @@ SELECT id, title FROM docs ORDER BY embedding <-> [0.1, ...] LIMIT 10;
 - `<=>`  → `vec_distance_cosine`
 - `<#>`  → `vec_distance_dot`
 
+> **Scope correction (2026-04-27, during 7b implementation):** Operators turned out to be a much bigger parser change than Q6 anticipated. sqlparser-rs (current pinned version) **fails outright** on `<->` and `<#>` ("Expected: an expression, found: ->"). Only `<=>` parses, as MySQL's `Spaceship` (null-safe equality). Supporting all three operators requires either a fork of sqlparser to extend the SQLite dialect, or a string-preprocessing pass that rewrites operators to function calls before handing SQL to the parser — neither is the "tiny parser change" Q6 estimated.
+>
+> **Decision:** ship 7b with **functions only**. Operators are deferred to a follow-up sub-phase **7b.1**. The KNN use case (`ORDER BY vec_distance_l2(col, [...]) LIMIT k`) still works — just verbose. When 7b.1 lands, queries can switch from function-call form to operator form without any other behavior change.
+
 **Decisions:**
 
 - **Dispatch in the existing expression evaluator.** No new function-registration framework — these are built-in functions like `||` is.
-- **Operators land in the parser as new infix tokens.** sqlparser's SQLite dialect doesn't have these; we either extend the dialect or post-process the AST. Either is fine.
+- **Operators land in 7b.1, not 7b.** See scope-correction note above.
+- **`ORDER BY` widened to accept arbitrary expressions** as part of 7b. Pre-7b, the parser restricted ORDER BY to bare column refs; without expression support, KNN queries would have been impossible. New shape: `eval_expr` is called per-row to produce sort keys. This is a strict superset — `ORDER BY col` still works because `Expr::Identifier` takes the same path.
 
-**LOC estimate:** ~250 lines.
+**LOC estimate:** ~250 lines for the functions; another ~50 for the ORDER BY parser extension. Total ~300 LOC, slightly over Q-time estimate.
 
 **Tests:** all three distance metrics against hand-computed values; operator parsing; KNN result ordering.
 
 ---
 
-### 7c — Brute-force KNN executor optimization
+### ✅ 7c — Brute-force KNN executor optimization
+
+**What shipped.** The SELECT executor now branches on `(ORDER BY, LIMIT k)` shape. When both are present and `k < N`, the new `select_topk` function maintains a bounded `BinaryHeap` of size k instead of full-sorting all N rowids. O(N log k) instead of O(N log N).
+
+**Implementation note: max-heap with direction-aware Ord.** A single `HeapEntry { key: Value, rowid: i64, asc: bool }` wrapper handles both `ORDER BY ASC LIMIT k` (k smallest) and `ORDER BY DESC LIMIT k` (k largest) without separate code paths. The `asc` flag inverts the natural Ord, so the displacement test reduces to "new entry < heap top" in both cases. After the scan, `into_sorted_vec` returns the right caller-facing order (ascending for ASC, descending for DESC).
 
-**What.** Recognize the pattern `ORDER BY <distance-expr> LIMIT k` and execute it with a bounded min-heap (size k) instead of a full sort. O(N log k) instead of O(N log N).
+**Measured speedup** (N=10k, k=10, single REAL column sort key, release build): ~1.8×. The advantage scales with N and with per-row work — KNN queries where the sort key is `vec_distance_l2(col, [...])` benefit much more because each key evaluation is itself O(dim).
 
-**Why a separate sub-phase.** 7b makes it work; 7c makes it fast enough to be useful on millions of rows. Worth shipping as its own commit so the perf delta is visible in benchmarks.
+**LOC**: ~120 implementation + ~180 tests/benchmark = ~300 total. Slightly over the ~150 estimate because the test surface (correctness + bench + edge cases for k=0, k>N, empty input, distance-function integration) ended up larger than initially projected.
 
-**LOC estimate:** ~150 lines including a tiny benchmark to prove the speedup.
+**Pre-existing bug surfaced.** The seed function for the benchmark needed positive scores because the INSERT parser doesn't currently handle `Expr::UnaryOp(Minus, Number(...))` for negative literals. Worked around with a Knuth-hash scrambler that stays positive; the underlying parser bug is documented as a follow-up.
 
 ---
 
@@ -296,10 +305,11 @@ let rows = conn.execute(&resp.sql)?;
 ## Implementation order + dependencies
 
 ```
-7a (VECTOR type)           — independent, foundational
-  └── 7b (distances)       — needs 7a
-        └── 7c (KNN exec)  — needs 7b
-              └── 7d (HNSW)— needs 7b/7c
+7a (VECTOR type)              — independent, foundational
+  └── 7b (distance functions) — needs 7a
+        └── 7b.1 (operators)  — sugar over 7b; deferred from 7b per scope correction
+        └── 7c (KNN exec opt) — needs 7b (operators not required)
+              └── 7d (HNSW)   — needs 7b/7c
 
 7e (JSON)                  — independent, can interleave anywhere
 

{sqlrite-0.1.10 → sqlrite-0.1.12}/docs/roadmap.md

@@ -470,9 +470,9 @@ The full plan + recorded design decisions live in [`docs/phase-7-plan.md`](phase
 
 Approved sub-phases (Q1–Q10 resolved):
 
-- **7a — `VECTOR(N)` column type** — dense fixed-dimension f32 storage via the existing cell encoding; bump file format to v4. Bracket-array literal syntax `[0.1, 0.2, …]` (Q7).
-- **7b — Distance functions + KNN operators** — `vec_distance_l2/cosine/dot` plus pgvector-style `<->` `<=>` `<#>` operators (Q6).
-- **7c — Brute-force KNN executor optimization** — recognize `ORDER BY <distance> LIMIT k`, use bounded min-heap.
+- **✅ 7a — `VECTOR(N)` column type** *(v0.1.10)* — dense fixed-dimension f32 storage via the existing cell encoding; format bumped to v4. Bracket-array literal syntax `[0.1, 0.2, …]` (Q7).
+- **✅ 7b — Distance functions** *(v0.1.11)* — `vec_distance_l2/cosine/dot`, plus the ORDER BY-expressions parser change so KNN queries work end-to-end. Operators (`<->` `<=>` `<#>`) deferred to **7b.1** — sqlparser doesn't parse them natively, contradicting Q6's "tiny parser change" assumption.
+- **✅ 7c — Brute-force KNN executor optimization** — bounded `BinaryHeap` of size k for `ORDER BY <expr> LIMIT k`. ~1.8× faster than full-sort at N=10k for cheap keys; bigger gains on expensive keys like `vec_distance_l2`.
 - **7d — HNSW ANN index** — `CREATE INDEX … USING hnsw (col)`; persisted as cell-encoded graph. Fixed defaults `M=16, ef_construction=200, ef_search=50` (Q2).
 - **7e — JSON column type + path queries** — `JSON` data type stored as bincoded `serde_json::Value` (Q3); `json_extract` / `json_array_length` / `json_object_keys` / `json_type`.
 - **7f — ~~Full-text search with BM25~~** — **deferred to Phase 8** (Q1).

{sqlrite-0.1.10 → sqlrite-0.1.12}/docs/supported-sql.md

@@ -125,7 +125,7 @@ FROM <table>
 
 - **Projection**: `*` (all columns in declaration order) or a bare column list. Columns not declared on the table are rejected.
 - **`WHERE`**: any [expression](#expressions). Evaluated per row; NULL-as-false in WHERE context (three-valued logic collapsed to two-valued for filtering).
-- **`ORDER BY`**: single column, `ASC` (default) or `DESC`. Sort key types must match; mixing `INTEGER` and `TEXT` across rows under a single `ORDER BY` is a runtime error.
+- **`ORDER BY`**: single sort key, `ASC` (default) or `DESC`. The sort key can be a bare column reference OR any expression — including function calls — so KNN queries like `ORDER BY vec_distance_l2(embedding, [...]) LIMIT k` work end-to-end *(Phase 7b)*. Sort key types must match; mixing `INTEGER` and `TEXT` across rows under a single `ORDER BY` is a runtime error.
 - **`LIMIT`**: non-negative integer literal. `LIMIT 0` is valid (returns zero rows).
 
 ### Index probing
@@ -140,7 +140,7 @@ The executor includes a tiny optimizer: if the `WHERE` is exactly `<indexed_col>
 - **`DISTINCT`**
 - **`LIKE`**, **`IN`**, **`IS NULL`** / **`IS NOT NULL`**, `BETWEEN`
 - **Expressions in the projection list** (`SELECT age + 1 FROM users`) — projection is bare column references only
-- **Multi-column `ORDER BY`**, `NULLS FIRST/LAST`
+- **Multi-column `ORDER BY`**, `NULLS FIRST/LAST` (single sort key only; the sort key itself can be an expression as of Phase 7b)
 - **`OFFSET`**
 - **Column aliases** (`SELECT name AS n FROM users`)
 
@@ -196,6 +196,26 @@ Expressions work inside `WHERE` (both in `SELECT`, `UPDATE`, `DELETE`) and on th
 
 Same set accepted by `INSERT` (see [Value literals accepted](#value-literals-accepted)).
 
+### Built-in functions
+
+| Function | Returns | Notes |
+|---|---|---|
+| `vec_distance_l2(a, b)` | Real (f64) | Euclidean distance √Σ(aᵢ−bᵢ)². Smaller is closer. *(Phase 7b)* |
+| `vec_distance_cosine(a, b)` | Real (f64) | Cosine distance `1 − (a·b) / (‖a‖·‖b‖)`. Errors on zero-magnitude vectors (cosine is undefined). Smaller is closer; identical vectors return 0.0, orthogonal vectors return 1.0. *(Phase 7b)* |
+| `vec_distance_dot(a, b)` | Real (f64) | Negated dot product `−(a·b)`. Negation makes "smaller is closer" consistent with the others. For unit-norm vectors equals `vec_distance_cosine(a, b) - 1`. *(Phase 7b)* |
+
+All three vector-distance functions take exactly two arguments, both of which must be vectors of the same dimension. Either argument can be a column reference (`embedding`), a bracket-array literal (`[0.1, 0.2, 0.3]`), or any sub-expression that evaluates to a vector. Mismatched dimensions error with `vector dimensions don't match (lhs=N, rhs=M)`.
+
+The KNN ranking pattern that motivates this set:
+
+```sql
+SELECT id, title FROM docs
+ORDER BY vec_distance_l2(embedding, [0.1, 0.2, ..., 0.0])
+LIMIT 10;
+```
+
+> **Operator forms (`<->` `<=>` `<#>`) are not supported yet.** They're the de facto pgvector convention but blocked on a sqlparser limitation — will land as a Phase 7b.1 follow-up. Use the function-call form for now.
+
 ### Type coercion in arithmetic
 
 - **Integer-only ops stay integer.** `1 + 2` → `3` (Integer).

{sqlrite-0.1.10 → sqlrite-0.1.12}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "maturin"
 
 [project]
 name = "sqlrite"
-version = "0.1.10"
+version = "0.1.12"
 description = "Python bindings for SQLRite — a small, embeddable SQLite clone written in Rust."
 authors = [{ name = "Joao Henrique Machado Silva", email = "joaoh82@gmail.com" }]
 license = { text = "MIT" }

{sqlrite-0.1.10 → sqlrite-0.1.12}/sdk/python/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "sqlrite-python"
-version = "0.1.10"
+version = "0.1.12"
 authors = ["Joao Henrique Machado Silva <joaoh82@gmail.com>"]
 edition = "2024"
 rust-version = "1.85"