nedb-engine 3.2.2__tar.gz → 3.3.1__tar.gz

{nedb_engine-3.2.2 → nedb_engine-3.3.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: nedb-engine
-Version: 3.2.2
+Version: 3.3.1
 Summary: NEDB — a versioned, self-compressing, time-traveling embedded database (replay-protected, idempotent, relational, searchable) with durable AOF persistence and a server daemon (nedbd).
 Project-URL: Homepage, https://github.com/aiassistsecure/nedb
 Project-URL: Repository, https://github.com/aiassistsecure/nedb
@@ -47,6 +47,78 @@ One Rust core → ships to **PyPI** and **npm** from a single source.
 
 ---
 
+## New in 3.3.0 — the query language grew up
+
+`WHERE` was six operators wide (`= != > < >= <=`) joined by an implicit `AND`.
+It now takes a full boolean expression, in **both** engines, and the clauses
+around it run in SQL's order.
+
+```sql
+FROM jobs
+WHERE (status IN ("open", "pending") OR fee > 100)
+  AND miner IS NOT NULL
+  AND NOT (region LIKE "eu-%")
+GROUP BY region SUM fee
+HAVING sum_fee > 10000
+ORDER BY sum_fee DESC
+LIMIT 20 OFFSET 40
+```
+
+| added | |
+| --- | --- |
+| `IN (…)` / `NOT IN (…)` | set membership |
+| `BETWEEN a AND b` / `NOT BETWEEN` | inclusive both ends, as in SQL |
+| `LIKE` / `NOT LIKE` / `ILIKE` | `%` any run, `_` any one char |
+| `IS NULL` / `IS NOT NULL` | matches absent **and** explicitly-null |
+| `AND` / `OR` / `NOT` / `(…)` | `AND` binds tighter; parens nest to any depth |
+| `OFFSET n` | pagination; pairs with `LIMIT` |
+| `ORDER BY a, b DESC` | multi-key, per-key direction |
+| `HAVING <predicate>` | filters the aggregated rows |
+| `COUNT` / `SUM f` / `AVG f` / `MIN f` / `MAX f` | whole-result aggregate, one row |
+
+**Indexed range scans.** `=`, `IN`, `BETWEEN` and the inequalities are served
+from a sorted index when one covers the field — a point lookup on 20,000 rows
+goes from 137 ms to 0.01 ms, a 1%-selective `BETWEEN` from 186 ms to 1.1 ms.
+See [**Indexes**](#indexes) for the measured table and the three cases that
+deliberately decline the index.
+
+**Nine silent defects fixed.** None of them crashed; they all returned a
+confident wrong answer with HTTP 200. The worst:
+
+- `GROUP BY status MAX fee` aggregated the *group* field, not the target —
+  answering `1.0` where the real maxima were 40 and 30.
+- `LIMIT 2 GROUP BY status COUNT` truncated the aggregate's **input**, so
+  twelve rows across three statuses reported counts summing to 2.
+- `ORDER BY count DESC` on grouped rows sorted the raw documents on a field
+  that only exists after grouping — silently inert.
+- `FROM jobs OFFSET 2` and a misspelled `ORDRE BY fee` were silently **dropped**
+  and a different query answered. Unknown clauses are now a parse error.
+- `SUM` ran through `f64`, losing integer precision above 2^53 — a real problem
+  for satoshi amounts and block heights. Integer inputs now stay in `i64`.
+- An ordering comparison against a missing field was true in the Rust engine
+  (`WHERE fee < 5` returned rows with no `fee` at all) and false in the Python
+  reference. Now false in both, matching SQL.
+- A field named like a keyword (`count`, `min`, `value`, `status`) was
+  unaddressable, because both lexers canonicalised case at field positions.
+
+**Compatibility.** Verified by building a `nedbd` from the released v3.2.2 tag
+and diffing every answer: **40 of 45 legacy queries byte-identical, zero
+regressions.** The differences are the three bug fixes above, each documented
+in `tests/test_backcompat.py`. `scripts/compare_engine_answers.py` reproduces
+the comparison against any released binary.
+
+**The DAG is untouched.** `tests/test_dag_preserved.py` runs the entire new
+query surface against a live chain and asserts head, seq and `verify()` are
+unchanged afterwards — and that `verify()` still returns *false* when the log
+is tampered with. Reads are reads.
+
+**Cross-engine parity is now gated.** Nothing previously checked that the
+Python reference and the Rust core agreed, which is how they had drifted apart
+in five places. Two suites now run the same battery through both and assert
+identical answers.
+
+---
+
 ## New in 3.2.0 — wrap the databases you already run
 
 NEDB adds **tamper-evident causal provenance to a database you already have**, in one line, without
@@ -292,6 +364,12 @@ db.query('FROM users WHERE status = "active" ORDER BY age ASC')
 db.query('FROM users SEARCH "rust"')
 db.query('FROM users GROUP BY status COUNT')
 
+# Full boolean predicates — IN, BETWEEN, LIKE, IS NULL, OR, NOT, parentheses
+db.query('FROM users WHERE status IN ("active", "trialing")')
+db.query('FROM users WHERE age BETWEEN 25 AND 40')
+db.query('FROM users WHERE bio LIKE "%rust%" AND NOT (status = "retired")')
+db.query('FROM users WHERE (age < 25 OR age > 60) AND bio IS NOT NULL')
+
 # Time-travel — AS OF any past sequence
 snap = db.seq
 db.put("users", "alice", {"name": "Alice", "age": 32, "status": "retired"})
@@ -516,13 +594,185 @@ redis-cli -p 6380 SELECT shop EVAL 'FROM beliefs TRACE caused_by' 0
 FROM <collection>
   [ AS OF <seq> ]                            transaction time (when was it written?)
   [ VALID AS OF "<date>" ]                   valid time (when was it true in the world?)
-  [ WHERE <field> <op> <value> (AND ...) ]   op: = != < <= > >=
+  [ WHERE <predicate> ]                      full boolean predicate, see below
   [ SEARCH "<text>" ]                        full-text search
-  [ ORDER BY <field> [ASC|DESC] ]
   [ TRAVERSE <relation> ]                    graph traversal
   [ TRACE caused_by [REVERSE] ]              causal provenance (why? / what did this cause?)
-  [ LIMIT <n> ]
   [ GROUP BY <field> [COUNT|SUM f|AVG f|MIN f|MAX f] ]
+  [ COUNT | SUM f | AVG f | MIN f | MAX f ]  whole-result aggregate, one row
+  [ HAVING <predicate> ]                     filters the AGGREGATED rows
+  [ ORDER BY <field> [ASC|DESC] (, ...) ]
+  [ LIMIT <n> ] [ OFFSET <n> ]
+```
+
+Clauses are evaluated in SQL's order, whatever order you write them in:
+
+```
+FROM → WHERE → GROUP BY → HAVING → ORDER BY → OFFSET → LIMIT
+```
+
+That matters, and before 3.3.0 it was wrong. `LIMIT` truncated the *input* to
+an aggregate rather than the result, so `LIMIT 5 GROUP BY status COUNT` over
+twelve rows reported counts summing to 5 — it said only five rows existed when
+twelve did. `ORDER BY` ran before grouping, so sorting on `count` or `sum_fee`
+silently did nothing. `VALID AS OF` was applied after `LIMIT` in the Python
+engine, so a limited bi-temporal query returned fewer valid rows than exist.
+
+### Predicates
+
+`WHERE` takes a full boolean expression. `AND` binds tighter than `OR`;
+parentheses override, and nest to any depth.
+
+```
+<predicate> := <or>
+<or>        := <and> [OR <and>]*
+<and>       := <not> [AND <not>]*
+<not>       := [NOT] <primary>
+<primary>   := "(" <predicate> ")" | <comparison>
+```
+
+| Comparison | Example |
+| --- | --- |
+| `= != < <= > >=` | `WHERE height > 600000` |
+| `IN (…)` / `NOT IN (…)` | `WHERE status IN ("open", "pending")` |
+| `BETWEEN a AND b` | `WHERE height BETWEEN 100 AND 200` — inclusive, as in SQL |
+| `NOT BETWEEN a AND b` | `WHERE fee NOT BETWEEN 10 AND 20` |
+| `LIKE` / `NOT LIKE` | `WHERE miner LIKE "Acme%"` — `%` any run, `_` any one char |
+| `ILIKE` | `WHERE miner ILIKE "acme%"` — case-insensitive |
+| `IS NULL` / `IS NOT NULL` | `WHERE miner IS NULL` — matches absent **and** explicitly-null |
+
+```python
+db.query('''FROM jobs
+            WHERE (status IN ("open", "pending") OR fee > 100)
+              AND miner IS NOT NULL
+              AND NOT (region LIKE "eu-%")
+            ORDER BY fee DESC LIMIT 20''')
+```
+
+Filterable metadata fields: `_id`, `_coll`, `_hash`, `_seq`.
+
+`_id = "x"` is an O(1) index lookup rather than a scan — but only when it is a
+genuine conjunct. Under an `OR` it cannot constrain the result set, so the
+planner correctly declines the fast path there.
+
+### Indexes
+
+`=`, `IN (...)`, `BETWEEN` and the one-sided inequalities are served from a
+sorted index when one covers the field, turning a full collection scan into a
+bounded range walk:
+
+```python
+db.create_index("blocks", "height", "sorted")
+db.query("FROM blocks WHERE height BETWEEN 600000 AND 600100")
+```
+
+Measured on 20,000 rows with `scripts/bench_index_range.py` — two identical
+databases, one indexed, one not:
+
+| Query | Scan | Indexed | Speedup |
+| --- | --- | --- | --- |
+| `WHERE fee = 10000` | 137 ms | 0.01 ms | 17,000× |
+| `WHERE fee IN (a, b, c)` | 185 ms | 0.02 ms | 9,700× |
+| `WHERE fee BETWEEN …` (1% of rows) | 186 ms | 1.1 ms | 170× |
+| `WHERE fee BETWEEN …` (10% of rows) | 188 ms | 13 ms | 14× |
+| unindexed field (control) | 188 ms | 187 ms | 1.0× |
+
+The planner asks the index how many rows each candidate range covers and takes
+the narrowest, so `WHERE region = "eu" AND height BETWEEN 600000 AND 600001`
+uses the height index rather than whichever field it saw first. Same-field
+bounds are merged, so `height > 100 AND height < 200` is one walk.
+
+An index is only ever used to NARROW candidates — the full predicate is
+re-evaluated on whatever comes back, so the answer never depends on whether an
+index exists. Three cases deliberately decline it:
+
+- **`IS NULL`.** A document whose field is absent is not in that field's
+  index, so an index scan would return the exact complement of the answer.
+- **Anything under `OR` or `NOT`.** A disjunct does not constrain the result
+  set; narrowing on one arm would silently drop the rows the other arm matched.
+- **`AS OF`.** The sorted index holds current versions only (a superseded hash
+  is dropped on overwrite), so it cannot answer a historical query.
+
+> **Predicates over NULL follow SQL's three-valued logic.** An ordering
+> comparison (`<` `<=` `>` `>=`, and therefore `BETWEEN`) against a missing or
+> null field is never true — `WHERE fee < 5` will not return a row that has no
+> `fee` at all. `LIKE` is false in *both* polarities, so a null row appears in
+> neither `LIKE` nor `NOT LIKE`. `=` and `!=` do operate on null, so
+> `WHERE fee = NULL` selects rows where the field is absent or null, and
+> `WHERE fee != 5` includes them. Use `IS NULL` / `IS NOT NULL` to test
+> presence explicitly.
+
+### Unknown clauses are errors
+
+A query containing a clause the engine does not implement is **rejected**, not
+silently reinterpreted. Before 3.3.0 the parser skipped tokens it did not
+recognise, so `FROM jobs OFFSET 2` returned un-offset rows with HTTP 200 and a
+misspelled `ORDRE BY fee` returned unsorted rows — the engine answered a
+*different query* than the one asked, and said nothing. Both now return
+HTTP 400 with the offending token.
+
+### Aggregates
+
+`SUM`/`AVG`/`MIN`/`MAX` take the field to aggregate; `COUNT` (the default when
+no aggregate is given) takes none.
+
+```python
+db.query('FROM items GROUP BY cat MAX price')
+# → [{"cat": "x", "count": 3, "max_price": 10.0, "value": 10.0}, …]
+```
+
+`count` is the group size. The aggregate only considers rows whose target field
+is numeric, so a group of 5 where 2 carry a numeric `price` reports `count: 5`
+and averages over 2. An aggregate with no numeric input is `null`, never `0`.
+The `value` key is a back-compatible alias for the aggregate result.
+
+Integer inputs give integer results — `SUM`/`MIN`/`MAX` stay in 64-bit
+integers rather than passing through a float, so a sum over satoshi amounts or
+block heights above 2^53 is exact. `AVG` is always fractional.
+
+Groups come back sorted by key unless you say otherwise, so results are stable
+run to run and identical across engines.
+
+Drop the `GROUP BY` for a whole-result aggregate, which returns exactly one row:
+
+```python
+db.query('FROM orders COUNT')                    # → [{"count": 1049, "value": 1049}]
+db.query('FROM orders WHERE status = "paid" COUNT')
+db.query('FROM orders SUM total')                # → [{"count": 1049, "sum_total": 88123, …}]
+```
+
+`COUNT` of an empty result is one row holding `0` — a caller asking "how many?"
+always gets a number. `SUM` of an empty result is `null`.
+
+### HAVING
+
+`WHERE` filters rows before they are grouped; `HAVING` filters the groups.
+
+```python
+db.query('FROM orders GROUP BY region SUM total HAVING sum_total > 10000')
+db.query('FROM orders GROUP BY region COUNT HAVING count BETWEEN 5 AND 50')
+```
+
+`HAVING` runs through the same evaluator as `WHERE`, so it gets the whole
+predicate surface — `IN`, `BETWEEN`, `LIKE`, `OR`, `NOT`, parentheses — rather
+than a poorer second copy.
+
+### Sorting and paging
+
+```python
+db.query('FROM orders ORDER BY region, total DESC')   # ties broken by the next key
+db.query('FROM orders ORDER BY total DESC LIMIT 20 OFFSET 40')
+```
+
+`OFFSET` skips rows of the result and pairs with `LIMIT` for pagination. An
+offset past the end is an empty page, not an error.
+
+A field whose name collides with a reserved word is still addressable — a
+document may legitimately have a `count`, `min`, `value` or `status` field, and
+`WHERE count > 3` reads that field rather than the aggregate:
+
+```python
+db.query('FROM metrics WHERE count > 3 ORDER BY count DESC')
 ```
 
 Combine both time axes:

{nedb_engine-3.2.2 → nedb_engine-3.3.1}/README.md

@@ -26,6 +26,78 @@ One Rust core → ships to **PyPI** and **npm** from a single source.
 
 ---
 
+## New in 3.3.0 — the query language grew up
+
+`WHERE` was six operators wide (`= != > < >= <=`) joined by an implicit `AND`.
+It now takes a full boolean expression, in **both** engines, and the clauses
+around it run in SQL's order.
+
+```sql
+FROM jobs
+WHERE (status IN ("open", "pending") OR fee > 100)
+  AND miner IS NOT NULL
+  AND NOT (region LIKE "eu-%")
+GROUP BY region SUM fee
+HAVING sum_fee > 10000
+ORDER BY sum_fee DESC
+LIMIT 20 OFFSET 40
+```
+
+| added | |
+| --- | --- |
+| `IN (…)` / `NOT IN (…)` | set membership |
+| `BETWEEN a AND b` / `NOT BETWEEN` | inclusive both ends, as in SQL |
+| `LIKE` / `NOT LIKE` / `ILIKE` | `%` any run, `_` any one char |
+| `IS NULL` / `IS NOT NULL` | matches absent **and** explicitly-null |
+| `AND` / `OR` / `NOT` / `(…)` | `AND` binds tighter; parens nest to any depth |
+| `OFFSET n` | pagination; pairs with `LIMIT` |
+| `ORDER BY a, b DESC` | multi-key, per-key direction |
+| `HAVING <predicate>` | filters the aggregated rows |
+| `COUNT` / `SUM f` / `AVG f` / `MIN f` / `MAX f` | whole-result aggregate, one row |
+
+**Indexed range scans.** `=`, `IN`, `BETWEEN` and the inequalities are served
+from a sorted index when one covers the field — a point lookup on 20,000 rows
+goes from 137 ms to 0.01 ms, a 1%-selective `BETWEEN` from 186 ms to 1.1 ms.
+See [**Indexes**](#indexes) for the measured table and the three cases that
+deliberately decline the index.
+
+**Nine silent defects fixed.** None of them crashed; they all returned a
+confident wrong answer with HTTP 200. The worst:
+
+- `GROUP BY status MAX fee` aggregated the *group* field, not the target —
+  answering `1.0` where the real maxima were 40 and 30.
+- `LIMIT 2 GROUP BY status COUNT` truncated the aggregate's **input**, so
+  twelve rows across three statuses reported counts summing to 2.
+- `ORDER BY count DESC` on grouped rows sorted the raw documents on a field
+  that only exists after grouping — silently inert.
+- `FROM jobs OFFSET 2` and a misspelled `ORDRE BY fee` were silently **dropped**
+  and a different query answered. Unknown clauses are now a parse error.
+- `SUM` ran through `f64`, losing integer precision above 2^53 — a real problem
+  for satoshi amounts and block heights. Integer inputs now stay in `i64`.
+- An ordering comparison against a missing field was true in the Rust engine
+  (`WHERE fee < 5` returned rows with no `fee` at all) and false in the Python
+  reference. Now false in both, matching SQL.
+- A field named like a keyword (`count`, `min`, `value`, `status`) was
+  unaddressable, because both lexers canonicalised case at field positions.
+
+**Compatibility.** Verified by building a `nedbd` from the released v3.2.2 tag
+and diffing every answer: **40 of 45 legacy queries byte-identical, zero
+regressions.** The differences are the three bug fixes above, each documented
+in `tests/test_backcompat.py`. `scripts/compare_engine_answers.py` reproduces
+the comparison against any released binary.
+
+**The DAG is untouched.** `tests/test_dag_preserved.py` runs the entire new
+query surface against a live chain and asserts head, seq and `verify()` are
+unchanged afterwards — and that `verify()` still returns *false* when the log
+is tampered with. Reads are reads.
+
+**Cross-engine parity is now gated.** Nothing previously checked that the
+Python reference and the Rust core agreed, which is how they had drifted apart
+in five places. Two suites now run the same battery through both and assert
+identical answers.
+
+---
+
 ## New in 3.2.0 — wrap the databases you already run
 
 NEDB adds **tamper-evident causal provenance to a database you already have**, in one line, without
@@ -271,6 +343,12 @@ db.query('FROM users WHERE status = "active" ORDER BY age ASC')
 db.query('FROM users SEARCH "rust"')
 db.query('FROM users GROUP BY status COUNT')
 
+# Full boolean predicates — IN, BETWEEN, LIKE, IS NULL, OR, NOT, parentheses
+db.query('FROM users WHERE status IN ("active", "trialing")')
+db.query('FROM users WHERE age BETWEEN 25 AND 40')
+db.query('FROM users WHERE bio LIKE "%rust%" AND NOT (status = "retired")')
+db.query('FROM users WHERE (age < 25 OR age > 60) AND bio IS NOT NULL')
+
 # Time-travel — AS OF any past sequence
 snap = db.seq
 db.put("users", "alice", {"name": "Alice", "age": 32, "status": "retired"})
@@ -495,13 +573,185 @@ redis-cli -p 6380 SELECT shop EVAL 'FROM beliefs TRACE caused_by' 0
 FROM <collection>
   [ AS OF <seq> ]                            transaction time (when was it written?)
   [ VALID AS OF "<date>" ]                   valid time (when was it true in the world?)
-  [ WHERE <field> <op> <value> (AND ...) ]   op: = != < <= > >=
+  [ WHERE <predicate> ]                      full boolean predicate, see below
   [ SEARCH "<text>" ]                        full-text search
-  [ ORDER BY <field> [ASC|DESC] ]
   [ TRAVERSE <relation> ]                    graph traversal
   [ TRACE caused_by [REVERSE] ]              causal provenance (why? / what did this cause?)
-  [ LIMIT <n> ]
   [ GROUP BY <field> [COUNT|SUM f|AVG f|MIN f|MAX f] ]
+  [ COUNT | SUM f | AVG f | MIN f | MAX f ]  whole-result aggregate, one row
+  [ HAVING <predicate> ]                     filters the AGGREGATED rows
+  [ ORDER BY <field> [ASC|DESC] (, ...) ]
+  [ LIMIT <n> ] [ OFFSET <n> ]
+```
+
+Clauses are evaluated in SQL's order, whatever order you write them in:
+
+```
+FROM → WHERE → GROUP BY → HAVING → ORDER BY → OFFSET → LIMIT
+```
+
+That matters, and before 3.3.0 it was wrong. `LIMIT` truncated the *input* to
+an aggregate rather than the result, so `LIMIT 5 GROUP BY status COUNT` over
+twelve rows reported counts summing to 5 — it said only five rows existed when
+twelve did. `ORDER BY` ran before grouping, so sorting on `count` or `sum_fee`
+silently did nothing. `VALID AS OF` was applied after `LIMIT` in the Python
+engine, so a limited bi-temporal query returned fewer valid rows than exist.
+
+### Predicates
+
+`WHERE` takes a full boolean expression. `AND` binds tighter than `OR`;
+parentheses override, and nest to any depth.
+
+```
+<predicate> := <or>
+<or>        := <and> [OR <and>]*
+<and>       := <not> [AND <not>]*
+<not>       := [NOT] <primary>
+<primary>   := "(" <predicate> ")" | <comparison>
+```
+
+| Comparison | Example |
+| --- | --- |
+| `= != < <= > >=` | `WHERE height > 600000` |
+| `IN (…)` / `NOT IN (…)` | `WHERE status IN ("open", "pending")` |
+| `BETWEEN a AND b` | `WHERE height BETWEEN 100 AND 200` — inclusive, as in SQL |
+| `NOT BETWEEN a AND b` | `WHERE fee NOT BETWEEN 10 AND 20` |
+| `LIKE` / `NOT LIKE` | `WHERE miner LIKE "Acme%"` — `%` any run, `_` any one char |
+| `ILIKE` | `WHERE miner ILIKE "acme%"` — case-insensitive |
+| `IS NULL` / `IS NOT NULL` | `WHERE miner IS NULL` — matches absent **and** explicitly-null |
+
+```python
+db.query('''FROM jobs
+            WHERE (status IN ("open", "pending") OR fee > 100)
+              AND miner IS NOT NULL
+              AND NOT (region LIKE "eu-%")
+            ORDER BY fee DESC LIMIT 20''')
+```
+
+Filterable metadata fields: `_id`, `_coll`, `_hash`, `_seq`.
+
+`_id = "x"` is an O(1) index lookup rather than a scan — but only when it is a
+genuine conjunct. Under an `OR` it cannot constrain the result set, so the
+planner correctly declines the fast path there.
+
+### Indexes
+
+`=`, `IN (...)`, `BETWEEN` and the one-sided inequalities are served from a
+sorted index when one covers the field, turning a full collection scan into a
+bounded range walk:
+
+```python
+db.create_index("blocks", "height", "sorted")
+db.query("FROM blocks WHERE height BETWEEN 600000 AND 600100")
+```
+
+Measured on 20,000 rows with `scripts/bench_index_range.py` — two identical
+databases, one indexed, one not:
+
+| Query | Scan | Indexed | Speedup |
+| --- | --- | --- | --- |
+| `WHERE fee = 10000` | 137 ms | 0.01 ms | 17,000× |
+| `WHERE fee IN (a, b, c)` | 185 ms | 0.02 ms | 9,700× |
+| `WHERE fee BETWEEN …` (1% of rows) | 186 ms | 1.1 ms | 170× |
+| `WHERE fee BETWEEN …` (10% of rows) | 188 ms | 13 ms | 14× |
+| unindexed field (control) | 188 ms | 187 ms | 1.0× |
+
+The planner asks the index how many rows each candidate range covers and takes
+the narrowest, so `WHERE region = "eu" AND height BETWEEN 600000 AND 600001`
+uses the height index rather than whichever field it saw first. Same-field
+bounds are merged, so `height > 100 AND height < 200` is one walk.
+
+An index is only ever used to NARROW candidates — the full predicate is
+re-evaluated on whatever comes back, so the answer never depends on whether an
+index exists. Three cases deliberately decline it:
+
+- **`IS NULL`.** A document whose field is absent is not in that field's
+  index, so an index scan would return the exact complement of the answer.
+- **Anything under `OR` or `NOT`.** A disjunct does not constrain the result
+  set; narrowing on one arm would silently drop the rows the other arm matched.
+- **`AS OF`.** The sorted index holds current versions only (a superseded hash
+  is dropped on overwrite), so it cannot answer a historical query.
+
+> **Predicates over NULL follow SQL's three-valued logic.** An ordering
+> comparison (`<` `<=` `>` `>=`, and therefore `BETWEEN`) against a missing or
+> null field is never true — `WHERE fee < 5` will not return a row that has no
+> `fee` at all. `LIKE` is false in *both* polarities, so a null row appears in
+> neither `LIKE` nor `NOT LIKE`. `=` and `!=` do operate on null, so
+> `WHERE fee = NULL` selects rows where the field is absent or null, and
+> `WHERE fee != 5` includes them. Use `IS NULL` / `IS NOT NULL` to test
+> presence explicitly.
+
+### Unknown clauses are errors
+
+A query containing a clause the engine does not implement is **rejected**, not
+silently reinterpreted. Before 3.3.0 the parser skipped tokens it did not
+recognise, so `FROM jobs OFFSET 2` returned un-offset rows with HTTP 200 and a
+misspelled `ORDRE BY fee` returned unsorted rows — the engine answered a
+*different query* than the one asked, and said nothing. Both now return
+HTTP 400 with the offending token.
+
+### Aggregates
+
+`SUM`/`AVG`/`MIN`/`MAX` take the field to aggregate; `COUNT` (the default when
+no aggregate is given) takes none.
+
+```python
+db.query('FROM items GROUP BY cat MAX price')
+# → [{"cat": "x", "count": 3, "max_price": 10.0, "value": 10.0}, …]
+```
+
+`count` is the group size. The aggregate only considers rows whose target field
+is numeric, so a group of 5 where 2 carry a numeric `price` reports `count: 5`
+and averages over 2. An aggregate with no numeric input is `null`, never `0`.
+The `value` key is a back-compatible alias for the aggregate result.
+
+Integer inputs give integer results — `SUM`/`MIN`/`MAX` stay in 64-bit
+integers rather than passing through a float, so a sum over satoshi amounts or
+block heights above 2^53 is exact. `AVG` is always fractional.
+
+Groups come back sorted by key unless you say otherwise, so results are stable
+run to run and identical across engines.
+
+Drop the `GROUP BY` for a whole-result aggregate, which returns exactly one row:
+
+```python
+db.query('FROM orders COUNT')                    # → [{"count": 1049, "value": 1049}]
+db.query('FROM orders WHERE status = "paid" COUNT')
+db.query('FROM orders SUM total')                # → [{"count": 1049, "sum_total": 88123, …}]
+```
+
+`COUNT` of an empty result is one row holding `0` — a caller asking "how many?"
+always gets a number. `SUM` of an empty result is `null`.
+
+### HAVING
+
+`WHERE` filters rows before they are grouped; `HAVING` filters the groups.
+
+```python
+db.query('FROM orders GROUP BY region SUM total HAVING sum_total > 10000')
+db.query('FROM orders GROUP BY region COUNT HAVING count BETWEEN 5 AND 50')
+```
+
+`HAVING` runs through the same evaluator as `WHERE`, so it gets the whole
+predicate surface — `IN`, `BETWEEN`, `LIKE`, `OR`, `NOT`, parentheses — rather
+than a poorer second copy.
+
+### Sorting and paging
+
+```python
+db.query('FROM orders ORDER BY region, total DESC')   # ties broken by the next key
+db.query('FROM orders ORDER BY total DESC LIMIT 20 OFFSET 40')
+```
+
+`OFFSET` skips rows of the result and pairs with `LIMIT` for pagination. An
+offset past the end is an empty page, not an error.
+
+A field whose name collides with a reserved word is still addressable — a
+document may legitimately have a `count`, `min`, `value` or `status` field, and
+`WHERE count > 3` reads that field rather than the aggregate:
+
+```python
+db.query('FROM metrics WHERE count > 3 ORDER BY count DESC')
 ```
 
 Combine both time axes:

{nedb_engine-3.2.2 → nedb_engine-3.3.1}/pyproject.toml

@@ -10,7 +10,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "nedb-engine"
-version = "3.2.2"
+version = "3.3.1"
 description = "NEDB — a versioned, self-compressing, time-traveling embedded database (replay-protected, idempotent, relational, searchable) with durable AOF persistence and a server daemon (nedbd)."
 readme = "README.md"
 requires-python = ">=3.8"

{nedb_engine-3.2.2 → nedb_engine-3.3.1}/python/nedb/__init__.py

@@ -102,4 +102,4 @@ __all__ = [
     "verify_proof", "fold_head",
     "_native", "__has_native__",
 ]
-__version__ = "3.2.2"
+__version__ = "3.3.1"

{nedb_engine-3.2.2 → nedb_engine-3.3.1}/python/nedb/client.py

@@ -53,6 +53,7 @@ import json
 import os
 import time
 import urllib.error
+import urllib.parse
 import urllib.request
 from typing import Any, Callable, Dict, Iterable, List, Optional, Tuple, TypeVar
 
@@ -75,6 +76,16 @@ CAS_BACKOFF_CAP_S = 0.2    # … capped here
 
 # ── errors ───────────────────────────────────────────────────────────────────
 
+def _seg(value: str) -> str:
+    """Percent-encode one URL path segment.
+
+    `/` MUST be encoded (hence ``safe=""``): an id containing a slash used to
+    split the path so the route matched a different id, and ``delete_doc``
+    returned "did not exist" for a document that did.
+    """
+    return urllib.parse.quote(str(value), safe="")
+
+
 class NedbError(RuntimeError):
     """Base error for daemon interactions. ``status`` is the HTTP status
     (``None`` for transport failures); ``payload`` is the parsed error body."""
@@ -283,12 +294,47 @@ class NedbClient:
         """Run NQL, return the full envelope ``{rows, count, seq, head}``."""
         return self.request("POST", self._dbp("query"), {"nql": nql})
 
-    def get_doc(self, coll: str, doc_id: str) -> Optional[dict]:
+    def get_doc(self, coll: str, doc_id: str,
+                as_of: Optional[int] = None) -> Optional[dict]:
         """Fetch one doc by id (returns ``None`` when absent). Docs carry
-        ``_id`` and ``_seq`` — feed ``_seq`` to ``op_put(if_seq=…)``."""
+        ``_id`` and ``_seq`` — feed ``_seq`` to ``op_put(if_seq=…)``.
+
+        With ``as_of``, returns the version at or before that sequence number.
+
+        Uses ``GET …/rows/{coll}/{id}``, which takes the id from the URL path.
+        This used to build ``FROM coll WHERE _id = "…"`` and interpolate the id
+        into it — so an id containing a double quote was REJECTED outright
+        (``unquotable identifier``) even though ``put`` accepts it, ``FROM
+        coll`` returns it and the engine stores it fine. The guard was there
+        because the interpolation was unsafe; taking the id from the path
+        removes the interpolation, so the id no longer needs guarding and
+        legitimate ids with quotes now work.
+
+        Injection is impossible by construction here: nothing is concatenated
+        into a query, so a crafted id can only ever name a document.
+
+        Falls back to the query path against a server predating the route
+        (nedb-engine < 3.3.0 answers 405 there, having registered only DELETE),
+        where the historical ``unquotable identifier`` guard still applies
+        because the interpolation is back.
+        """
+        path = self._dbp(f"rows/{_seg(coll)}/{_seg(doc_id)}")
+        if as_of is not None:
+            path = f"{path}?as_of={int(as_of)}"
+        try:
+            # The route answers 200 with `row: null` for a missing document,
+            # precisely so this cannot be confused with "no such route".
+            return self.request("GET", path).get("row")
+        except NedbError as e:
+            # 404/405 => the server predates this route (the Rust daemon
+            # registered only DELETE on this path; the Python AOF server had
+            # no such path at all). Anything else is a real error.
+            if getattr(e, "status", None) not in (404, 405):
+                raise
         if '"' in coll or '"' in str(doc_id):
             raise NedbBadRequest(f"unquotable identifier: {coll}:{doc_id}")
-        rows = self.query(f'FROM {coll} WHERE _id = "{doc_id}"')
+        as_of_clause = f" AS OF {int(as_of)}" if as_of is not None else ""
+        rows = self.query(f'FROM {coll}{as_of_clause} WHERE _id = "{doc_id}"')
         return rows[0] if rows else None
 
     def count(self, coll: str) -> int:
@@ -326,7 +372,7 @@ class NedbClient:
     def delete(self, coll: str, doc_id: str) -> dict:
         """``DELETE …/rows/{coll}/{id}`` → ``{ok, seq, head}``."""
         return self.request(
-            "DELETE", self._dbp(f"rows/{coll}/{doc_id}"))
+            "DELETE", self._dbp(f"rows/{_seg(coll)}/{_seg(doc_id)}"))
 
     def tx(self, ops: List[Dict[str, Any]], *,
            client: Optional[str] = None) -> dict: