nedb-engine 2.2.28__tar.gz → 2.2.31__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {nedb_engine-2.2.28 → nedb_engine-2.2.31}/PKG-INFO +11 -11
- {nedb_engine-2.2.28 → nedb_engine-2.2.31}/README.md +10 -10
- {nedb_engine-2.2.28 → nedb_engine-2.2.31}/pyproject.toml +1 -1
- {nedb_engine-2.2.28 → nedb_engine-2.2.31}/python/nedb/__init__.py +27 -13
- {nedb_engine-2.2.28 → nedb_engine-2.2.31}/python/nedb/_native.abi3.so +0 -0
- nedb_engine-2.2.31/python/nedb/nedbd-v2 +0 -0
- {nedb_engine-2.2.28 → nedb_engine-2.2.31}/python/nedb/nedbd-v2-darwin-arm64 +0 -0
- {nedb_engine-2.2.28 → nedb_engine-2.2.31}/python/nedb/nedbd-v2-darwin-x64 +0 -0
- nedb_engine-2.2.31/python/nedb/nedbd-v2.exe +0 -0
- {nedb_engine-2.2.28 → nedb_engine-2.2.31}/python/nedb/server.py +26 -2
- nedb_engine-2.2.31/rust/nedb-v2/README.md +179 -0
- nedb_engine-2.2.28/python/nedb/nedbd-v2 +0 -0
- nedb_engine-2.2.28/python/nedb/nedbd-v2.exe +0 -0
- {nedb_engine-2.2.28 → nedb_engine-2.2.31}/.gitignore +0 -0
- {nedb_engine-2.2.28 → nedb_engine-2.2.31}/LICENSE +0 -0
- {nedb_engine-2.2.28 → nedb_engine-2.2.31}/client/node/README.md +0 -0
- {nedb_engine-2.2.28 → nedb_engine-2.2.31}/client/python/README.md +0 -0
- {nedb_engine-2.2.28 → nedb_engine-2.2.31}/python/nedb/autoindex.py +0 -0
- {nedb_engine-2.2.28 → nedb_engine-2.2.31}/python/nedb/backends/__init__.py +0 -0
- {nedb_engine-2.2.28 → nedb_engine-2.2.31}/python/nedb/backends/redis_backend.py +0 -0
- {nedb_engine-2.2.28 → nedb_engine-2.2.31}/python/nedb/cascade.py +0 -0
- {nedb_engine-2.2.28 → nedb_engine-2.2.31}/python/nedb/concurrent.py +0 -0
- {nedb_engine-2.2.28 → nedb_engine-2.2.31}/python/nedb/crypto.py +0 -0
- {nedb_engine-2.2.28 → nedb_engine-2.2.31}/python/nedb/engine.py +0 -0
- {nedb_engine-2.2.28 → nedb_engine-2.2.31}/python/nedb/index.py +0 -0
- {nedb_engine-2.2.28 → nedb_engine-2.2.31}/python/nedb/log.py +0 -0
- {nedb_engine-2.2.28 → nedb_engine-2.2.31}/python/nedb/merkle.py +0 -0
- {nedb_engine-2.2.28 → nedb_engine-2.2.31}/python/nedb/mongo.py +0 -0
- {nedb_engine-2.2.28 → nedb_engine-2.2.31}/python/nedb/query.py +0 -0
- {nedb_engine-2.2.28 → nedb_engine-2.2.31}/python/nedb/redis_compat.py +0 -0
- {nedb_engine-2.2.28 → nedb_engine-2.2.31}/python/nedb/relations.py +0 -0
- {nedb_engine-2.2.28 → nedb_engine-2.2.31}/python/nedb/resp2.py +0 -0
- {nedb_engine-2.2.28 → nedb_engine-2.2.31}/python/nedb/snapshot.py +0 -0
- {nedb_engine-2.2.28 → nedb_engine-2.2.31}/python/nedb/sql.py +0 -0
- {nedb_engine-2.2.28 → nedb_engine-2.2.31}/python/nedb/store.py +0 -0
- {nedb_engine-2.2.28 → nedb_engine-2.2.31}/python/nedb/wrap_redis.py +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: nedb-engine
|
|
3
|
-
Version: 2.2.
|
|
3
|
+
Version: 2.2.31
|
|
4
4
|
Summary: NEDB — a versioned, self-compressing, time-traveling embedded database (replay-protected, idempotent, relational, searchable) with durable AOF persistence and a server daemon (nedbd).
|
|
5
5
|
Project-URL: Homepage, https://github.com/aiassistsecure/nedb
|
|
6
6
|
Project-URL: Repository, https://github.com/aiassistsecure/nedb
|
|
@@ -40,11 +40,11 @@ One Rust core → ships to **PyPI** and **npm** from a single source.
|
|
|
40
40
|
|
|
41
41
|
---
|
|
42
42
|
|
|
43
|
-
## NEDB v2.
|
|
43
|
+
## NEDB v2.2.31 — Production Stable
|
|
44
44
|
|
|
45
|
-
**Current stable: 2.
|
|
45
|
+
**Current stable: 2.2.31** — Cross-platform native wheels shipping `nedbd-v2` binary inside `pip install nedb-engine`. Linux + Windows wheels built on GitHub Actions; macOS arm64 + x86_64 wheels built on Codemagic (M2 Mac Minis). All four platforms publish from one `v*` tag.
|
|
46
46
|
|
|
47
|
-
NEDB v2 replaces the append-only log (AOF) with a **content-addressed Merkle DAG**. Every document version is an immutable, BLAKE2b-verified object. Nothing is ever overwritten. As of **v2.
|
|
47
|
+
NEDB v2 replaces the append-only log (AOF) with a **content-addressed Merkle DAG**. Every document version is an immutable, BLAKE2b-verified object. Nothing is ever overwritten. As of **v2.2.31**, restarts after the first open are **O(1) warm starts** (driven by a `MANIFEST` of `seq` + Merkle head), the **cold scan is deferred** so the daemon accepts connections immediately, and a new **`GET /events` SSE endpoint** streams scan progress + per-write events live.
|
|
48
48
|
|
|
49
49
|
```bash
|
|
50
50
|
# Run the v2 DAG engine — ships inside pip install nedb-engine
|
|
@@ -53,9 +53,9 @@ nedbd --dag --data ./data
|
|
|
53
53
|
NEDBD_DAG=1 NEDB_TMK=<32-byte-hex> nedbd --data ./data
|
|
54
54
|
|
|
55
55
|
curl http://127.0.0.1:7070/health
|
|
56
|
-
# {"ok":true,"version":"2.
|
|
56
|
+
# {"ok":true,"version":"2.2.31","service":"nedbd","engine":"dag","startup_ready":true,"encrypted":true}
|
|
57
57
|
|
|
58
|
-
# Tail the live event stream (new in v2.
|
|
58
|
+
# Tail the live event stream (new in v2.2.31)
|
|
59
59
|
curl http://127.0.0.1:7070/events
|
|
60
60
|
# event: scan data: {"objects":730000,"of":1310703,"rate":21043,"eta_s":28}
|
|
61
61
|
# event: ready data: {"seq":1310703,"head":"b2:9c14e07a…"}
|
|
@@ -79,7 +79,7 @@ curl http://127.0.0.1:7070/events
|
|
|
79
79
|
|
|
80
80
|
**v1 AOF engine is still shipped and unchanged** — `nedbd` (no flag) runs v1.
|
|
81
81
|
|
|
82
|
-
**Production status:** [vision.interchained.org](https://vision.interchained.org) is live on v2.
|
|
82
|
+
**Production status:** [vision.interchained.org](https://vision.interchained.org) is live on v2.2.31 — **1,310,703 sequences** indexed in the Vision database, AES-256-GCM encrypted at rest, at block height **620,989**.
|
|
83
83
|
|
|
84
84
|
---
|
|
85
85
|
|
|
@@ -244,11 +244,11 @@ nedbd --dag --data ./data # v2 DAG engine (or NEDBD_DAG=1)
|
|
|
244
244
|
NEDBD_RESP2_PORT=6380 nedbd # also speak RESP2 (redis-cli compatible)
|
|
245
245
|
nedbd --log-level 2 # 0=errors 1=requests 2=deploy 3=verbose
|
|
246
246
|
|
|
247
|
-
# Live event stream (new in v2.
|
|
247
|
+
# Live event stream (new in v2.2.31) — SSE: scan progress, ready, per-write head
|
|
248
248
|
curl http://127.0.0.1:7070/events
|
|
249
249
|
```
|
|
250
250
|
|
|
251
|
-
### Startup modes (v2.
|
|
251
|
+
### Startup modes (v2.2.31)
|
|
252
252
|
|
|
253
253
|
- **Warm start** — every restart after the first open reads the `MANIFEST` file and restores `seq` + Merkle `head` in **O(1)**. No scan, no replay, independent of dataset size. Boots in milliseconds.
|
|
254
254
|
- **Cold start** — first open of an existing dataset spawns the integrity scan in a background thread *and accepts connections immediately*. Reads serve instantly from the content-addressed DAG; writes return `HTTP 503 startup in progress` until the `startup_ready` gate flips. Progress (objects, rate, ETA) streams over `GET /events`.
|
|
@@ -258,7 +258,7 @@ curl http://127.0.0.1:7070/events
|
|
|
258
258
|
| Variable | Default | Description |
|
|
259
259
|
|---|---|---|
|
|
260
260
|
| `NEDBD_DAG` | `0` | Set `1` to launch the v2 DAG engine (`nedbd-v2`). Same as `--dag`. |
|
|
261
|
-
| `NEDBD_HOST` | `127.0.0.1` | Bind address. **v2.
|
|
261
|
+
| `NEDBD_HOST` | `127.0.0.1` | Bind address. **v2.2.31** defaults to loopback (was `0.0.0.0`) — security hardening fix. Set explicitly to `0.0.0.0` to expose. |
|
|
262
262
|
| `NEDBD_PORT` | `7070` | HTTP bind port. |
|
|
263
263
|
| `NEDBD_TOKEN` | unset | Optional bearer token; required on every `/v1/*` request when set. |
|
|
264
264
|
| `NEDB_TMK` | unset | 32-byte hex AES-256-GCM at-rest encryption key. |
|
|
@@ -321,7 +321,7 @@ db.query('FROM policy AS OF 200 VALID AS OF "2024-02-15"')
|
|
|
321
321
|
|
|
322
322
|
## Performance
|
|
323
323
|
|
|
324
|
-
**v2 DAG Rust server (v2.
|
|
324
|
+
**v2 DAG Rust server (v2.2.31, Intel iMac — 10k writes / 100k reads / 30k objects, AES-256-GCM on):**
|
|
325
325
|
|
|
326
326
|
| Operation | Throughput | p50 | p99 |
|
|
327
327
|
|---|---|---|---|
|
|
@@ -19,11 +19,11 @@ One Rust core → ships to **PyPI** and **npm** from a single source.
|
|
|
19
19
|
|
|
20
20
|
---
|
|
21
21
|
|
|
22
|
-
## NEDB v2.
|
|
22
|
+
## NEDB v2.2.31 — Production Stable
|
|
23
23
|
|
|
24
|
-
**Current stable: 2.
|
|
24
|
+
**Current stable: 2.2.31** — Cross-platform native wheels shipping `nedbd-v2` binary inside `pip install nedb-engine`. Linux + Windows wheels built on GitHub Actions; macOS arm64 + x86_64 wheels built on Codemagic (M2 Mac Minis). All four platforms publish from one `v*` tag.
|
|
25
25
|
|
|
26
|
-
NEDB v2 replaces the append-only log (AOF) with a **content-addressed Merkle DAG**. Every document version is an immutable, BLAKE2b-verified object. Nothing is ever overwritten. As of **v2.
|
|
26
|
+
NEDB v2 replaces the append-only log (AOF) with a **content-addressed Merkle DAG**. Every document version is an immutable, BLAKE2b-verified object. Nothing is ever overwritten. As of **v2.2.31**, restarts after the first open are **O(1) warm starts** (driven by a `MANIFEST` of `seq` + Merkle head), the **cold scan is deferred** so the daemon accepts connections immediately, and a new **`GET /events` SSE endpoint** streams scan progress + per-write events live.
|
|
27
27
|
|
|
28
28
|
```bash
|
|
29
29
|
# Run the v2 DAG engine — ships inside pip install nedb-engine
|
|
@@ -32,9 +32,9 @@ nedbd --dag --data ./data
|
|
|
32
32
|
NEDBD_DAG=1 NEDB_TMK=<32-byte-hex> nedbd --data ./data
|
|
33
33
|
|
|
34
34
|
curl http://127.0.0.1:7070/health
|
|
35
|
-
# {"ok":true,"version":"2.
|
|
35
|
+
# {"ok":true,"version":"2.2.31","service":"nedbd","engine":"dag","startup_ready":true,"encrypted":true}
|
|
36
36
|
|
|
37
|
-
# Tail the live event stream (new in v2.
|
|
37
|
+
# Tail the live event stream (new in v2.2.31)
|
|
38
38
|
curl http://127.0.0.1:7070/events
|
|
39
39
|
# event: scan data: {"objects":730000,"of":1310703,"rate":21043,"eta_s":28}
|
|
40
40
|
# event: ready data: {"seq":1310703,"head":"b2:9c14e07a…"}
|
|
@@ -58,7 +58,7 @@ curl http://127.0.0.1:7070/events
|
|
|
58
58
|
|
|
59
59
|
**v1 AOF engine is still shipped and unchanged** — `nedbd` (no flag) runs v1.
|
|
60
60
|
|
|
61
|
-
**Production status:** [vision.interchained.org](https://vision.interchained.org) is live on v2.
|
|
61
|
+
**Production status:** [vision.interchained.org](https://vision.interchained.org) is live on v2.2.31 — **1,310,703 sequences** indexed in the Vision database, AES-256-GCM encrypted at rest, at block height **620,989**.
|
|
62
62
|
|
|
63
63
|
---
|
|
64
64
|
|
|
@@ -223,11 +223,11 @@ nedbd --dag --data ./data # v2 DAG engine (or NEDBD_DAG=1)
|
|
|
223
223
|
NEDBD_RESP2_PORT=6380 nedbd # also speak RESP2 (redis-cli compatible)
|
|
224
224
|
nedbd --log-level 2 # 0=errors 1=requests 2=deploy 3=verbose
|
|
225
225
|
|
|
226
|
-
# Live event stream (new in v2.
|
|
226
|
+
# Live event stream (new in v2.2.31) — SSE: scan progress, ready, per-write head
|
|
227
227
|
curl http://127.0.0.1:7070/events
|
|
228
228
|
```
|
|
229
229
|
|
|
230
|
-
### Startup modes (v2.
|
|
230
|
+
### Startup modes (v2.2.31)
|
|
231
231
|
|
|
232
232
|
- **Warm start** — every restart after the first open reads the `MANIFEST` file and restores `seq` + Merkle `head` in **O(1)**. No scan, no replay, independent of dataset size. Boots in milliseconds.
|
|
233
233
|
- **Cold start** — first open of an existing dataset spawns the integrity scan in a background thread *and accepts connections immediately*. Reads serve instantly from the content-addressed DAG; writes return `HTTP 503 startup in progress` until the `startup_ready` gate flips. Progress (objects, rate, ETA) streams over `GET /events`.
|
|
@@ -237,7 +237,7 @@ curl http://127.0.0.1:7070/events
|
|
|
237
237
|
| Variable | Default | Description |
|
|
238
238
|
|---|---|---|
|
|
239
239
|
| `NEDBD_DAG` | `0` | Set `1` to launch the v2 DAG engine (`nedbd-v2`). Same as `--dag`. |
|
|
240
|
-
| `NEDBD_HOST` | `127.0.0.1` | Bind address. **v2.
|
|
240
|
+
| `NEDBD_HOST` | `127.0.0.1` | Bind address. **v2.2.31** defaults to loopback (was `0.0.0.0`) — security hardening fix. Set explicitly to `0.0.0.0` to expose. |
|
|
241
241
|
| `NEDBD_PORT` | `7070` | HTTP bind port. |
|
|
242
242
|
| `NEDBD_TOKEN` | unset | Optional bearer token; required on every `/v1/*` request when set. |
|
|
243
243
|
| `NEDB_TMK` | unset | 32-byte hex AES-256-GCM at-rest encryption key. |
|
|
@@ -300,7 +300,7 @@ db.query('FROM policy AS OF 200 VALID AS OF "2024-02-15"')
|
|
|
300
300
|
|
|
301
301
|
## Performance
|
|
302
302
|
|
|
303
|
-
**v2 DAG Rust server (v2.
|
|
303
|
+
**v2 DAG Rust server (v2.2.31, Intel iMac — 10k writes / 100k reads / 30k objects, AES-256-GCM on):**
|
|
304
304
|
|
|
305
305
|
| Operation | Throughput | p50 | p99 |
|
|
306
306
|
|---|---|---|---|
|
|
@@ -10,7 +10,7 @@ build-backend = "hatchling.build"
|
|
|
10
10
|
|
|
11
11
|
[project]
|
|
12
12
|
name = "nedb-engine"
|
|
13
|
-
version = "2.2.
|
|
13
|
+
version = "2.2.31"
|
|
14
14
|
description = "NEDB — a versioned, self-compressing, time-traveling embedded database (replay-protected, idempotent, relational, searchable) with durable AOF persistence and a server daemon (nedbd)."
|
|
15
15
|
readme = "README.md"
|
|
16
16
|
requires-python = ">=3.8"
|
|
@@ -38,21 +38,35 @@ except ImportError: # pure-Python install (sdist / unsupported platform)
|
|
|
38
38
|
# informative error instead of a bare ImportError with no guidance.
|
|
39
39
|
import types as _types, sys as _sys
|
|
40
40
|
|
|
41
|
+
import sys as _sys_tmp, os as _os_tmp
|
|
42
|
+
_is_msys2 = bool(_os_tmp.environ.get("MSYSTEM")) or "mingw" in _sys_tmp.executable.lower()
|
|
43
|
+
del _sys_tmp, _os_tmp
|
|
44
|
+
|
|
41
45
|
class _NativeStub(_types.ModuleType):
|
|
42
|
-
|
|
46
|
+
# Primary fix: install the Rust crate → get the nedbd server → use HTTP mode.
|
|
47
|
+
# Secondary fix (CPython only): pip reinstall to get the platform wheel with _native embedded.
|
|
48
|
+
_MSG_MSYS2 = (
|
|
49
|
+
"\n\n"
|
|
50
|
+
" nedb._native (embedded v2 DAG core) is not available on MSYS2/MinGW Python.\n\n"
|
|
51
|
+
" To use NEDB v2 features, install the server binary and use HTTP mode:\n\n"
|
|
52
|
+
" cargo install nedb-engine # install nedbd v2 server\n"
|
|
53
|
+
" nedbd --dag ./data # start DAG server\n"
|
|
54
|
+
" NEDB_URL=http://localhost:7070 python3 your_script.py\n\n"
|
|
55
|
+
" Run 'nedbd --doctor' for a full diagnosis.\n"
|
|
56
|
+
)
|
|
57
|
+
_MSG_OTHER = (
|
|
43
58
|
"\n\n"
|
|
44
|
-
" nedb._native is not available
|
|
45
|
-
"
|
|
46
|
-
"
|
|
47
|
-
" the
|
|
48
|
-
"
|
|
49
|
-
"
|
|
50
|
-
"
|
|
51
|
-
"
|
|
52
|
-
" 2. Re-install on a platform that has a native wheel:\n"
|
|
53
|
-
" pip install --force-reinstall --no-cache-dir nedb-engine\n\n"
|
|
54
|
-
" 3. Run 'nedbd --doctor' for a full diagnosis.\n"
|
|
59
|
+
" nedb._native (embedded v2 DAG core) is not available.\n"
|
|
60
|
+
" You have the universal wheel — reinstall to get the platform wheel:\n\n"
|
|
61
|
+
" pip install --force-reinstall --no-cache-dir nedb-engine\n\n"
|
|
62
|
+
" Or install the server binary and use HTTP mode (works everywhere):\n\n"
|
|
63
|
+
" cargo install nedb-engine # install nedbd v2 server\n"
|
|
64
|
+
" nedbd --dag ./data # start DAG server\n"
|
|
65
|
+
" NEDB_URL=http://localhost:7070 python3 your_script.py\n\n"
|
|
66
|
+
" Run 'nedbd --doctor' for a full diagnosis.\n"
|
|
55
67
|
)
|
|
68
|
+
_MSG = _MSG_MSYS2 if _is_msys2 else _MSG_OTHER
|
|
69
|
+
|
|
56
70
|
def __getattr__(self, name: str):
|
|
57
71
|
raise ImportError(f"nedb._native.{name} is not available.{self._MSG}")
|
|
58
72
|
|
|
@@ -73,4 +87,4 @@ __all__ = [
|
|
|
73
87
|
"wrap_redis", "WrappedRedis",
|
|
74
88
|
"_native", "__has_native__",
|
|
75
89
|
]
|
|
76
|
-
__version__ = "2.2.
|
|
90
|
+
__version__ = "2.2.31"
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
@@ -676,7 +676,7 @@ def _run_doctor() -> None: # noqa: C901
|
|
|
676
676
|
step += 1
|
|
677
677
|
if has_cargo:
|
|
678
678
|
_cmd("install from crates.io (recommended)",
|
|
679
|
-
"cargo install nedb-
|
|
679
|
+
"cargo install nedb-engine",
|
|
680
680
|
f"binary → {os.path.join(_cargo_bin, 'nedbd')}")
|
|
681
681
|
print()
|
|
682
682
|
_cmd("OR reinstall pip wheel (also bundles the binary)",
|
|
@@ -810,7 +810,7 @@ def main() -> None:
|
|
|
810
810
|
print("", file=_sys.stderr)
|
|
811
811
|
print(" Quick fixes:", file=_sys.stderr)
|
|
812
812
|
print(" pip install --force-reinstall --no-cache-dir nedb-engine # re-install with binary", file=_sys.stderr)
|
|
813
|
-
print(" cargo install nedb-
|
|
813
|
+
print(" cargo install nedb-engine # build from source (any platform)", file=_sys.stderr)
|
|
814
814
|
print("", file=_sys.stderr)
|
|
815
815
|
_sys.exit(1)
|
|
816
816
|
# Rust binary: nedbd-v2 [data_dir]
|
|
@@ -861,6 +861,30 @@ def main() -> None:
|
|
|
861
861
|
print(BANNER)
|
|
862
862
|
level_label = {0: "errors-only", 1: "requests", 2: "deploy", 3: "verbose"}.get(_log_level, str(_log_level))
|
|
863
863
|
print(f" nedbd {__version__} — http://{host}:{port} data={os.path.abspath(data)} auth={auth} log={level_label}")
|
|
864
|
+
|
|
865
|
+
# ── v2 DAG health notice ──────────────────────────────────────────────────
|
|
866
|
+
# Check once at startup whether the v2 binary and _native extension are available.
|
|
867
|
+
# Print a single actionable line if either is missing — non-blocking, no exit.
|
|
868
|
+
_has_native = False
|
|
869
|
+
try:
|
|
870
|
+
from nedb import __has_native__ as _has_native # type: ignore[attr-defined]
|
|
871
|
+
except Exception:
|
|
872
|
+
pass
|
|
873
|
+
_dag_bin_available = shutil.which("nedbd-v2") or shutil.which("nedbd_v2") or \
|
|
874
|
+
any(os.path.isfile(os.path.join(os.path.dirname(os.path.abspath(__file__)), n))
|
|
875
|
+
for n in ["nedbd-v2", "nedbd-v2.exe", "nedbd_v2", "nedbd_v2.exe"])
|
|
876
|
+
if not _has_native or not _dag_bin_available:
|
|
877
|
+
print()
|
|
878
|
+
if not _dag_bin_available:
|
|
879
|
+
print(" ⚡ v2 DAG engine not found — for content-addressed, instant-cold-start mode:")
|
|
880
|
+
print(" cargo install nedb-engine → nedbd --dag ./data")
|
|
881
|
+
if not _has_native:
|
|
882
|
+
print(" ⚡ nedb._native not available — embedded v2 API requires the platform wheel")
|
|
883
|
+
print(" pip install --force-reinstall --no-cache-dir nedb-engine")
|
|
884
|
+
print(" OR use HTTP mode: NEDB_URL=http://localhost:7070 python3 your_script.py")
|
|
885
|
+
if not _has_native or not _dag_bin_available:
|
|
886
|
+
print(" ℹ Run 'nedbd --doctor' for a full diagnosis and exact fix commands.")
|
|
887
|
+
print()
|
|
864
888
|
# Eagerly open all known databases so backfill-encrypt (if needed) runs
|
|
865
889
|
# immediately and is visible in the boot log, not hidden on first request.
|
|
866
890
|
names = manager.names()
|
|
@@ -0,0 +1,179 @@
|
|
|
1
|
+
# nedb-engine
|
|
2
|
+
|
|
3
|
+
**NEDB v2 — content-addressed DAG storage engine with NQL and HTTP server**
|
|
4
|
+
|
|
5
|
+
[](https://crates.io/crates/nedb-engine)
|
|
6
|
+
[](https://github.com/Eth-Interchained/nedb/blob/master/LICENSE)
|
|
7
|
+
[](https://github.com/Eth-Interchained/nedb)
|
|
8
|
+
|
|
9
|
+
This crate ships the **`nedbd`** binary — the NEDB v2 DAG HTTP server. Install it, point it at a data directory, and any language can speak to it over HTTP/JSON.
|
|
10
|
+
|
|
11
|
+
```bash
|
|
12
|
+
cargo install nedb-engine
|
|
13
|
+
nedbd ./data # AOF engine (pure Rust, v1-compatible)
|
|
14
|
+
nedbd --dag ./data # DAG engine (v2, content-addressed, recommended)
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
---
|
|
18
|
+
|
|
19
|
+
## What is NEDB v2?
|
|
20
|
+
|
|
21
|
+
NEDB v2 replaces the append-only log (AOF) with a **content-addressed Merkle DAG**:
|
|
22
|
+
|
|
23
|
+
- Every document version is an **immutable, BLAKE2b-hashed object**. Nothing is ever overwritten.
|
|
24
|
+
- Every write produces a new **chain head** — a BLAKE2b commitment over the entire database history.
|
|
25
|
+
- **Time-travel**: read any document `AS OF seq N` to see exactly what it contained at that point.
|
|
26
|
+
- **Causal provenance**: documents link to their causal parents via `caused_by` hashes. `TRACE caused_by` walks the full causal graph backward.
|
|
27
|
+
- **TRAVERSE**: named graph relations via `__links__`. `FROM person WHERE _id = "robert" TRAVERSE parent_of` returns linked nodes.
|
|
28
|
+
- **O(1) warm start**: a `MANIFEST` file stores `seq` + Merkle head so restarts never re-scan the entire object store.
|
|
29
|
+
- **Instant cold start**: the daemon accepts connections immediately; background scan loads objects incrementally.
|
|
30
|
+
- **AES-256-GCM at rest**: optional symmetric encryption with a double-envelope key structure (TMK wraps DEK).
|
|
31
|
+
|
|
32
|
+
---
|
|
33
|
+
|
|
34
|
+
## Install
|
|
35
|
+
|
|
36
|
+
```bash
|
|
37
|
+
cargo install nedb-engine
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
The `nedbd` binary lands in `~/.cargo/bin/`. Make sure that's on your `PATH`.
|
|
41
|
+
|
|
42
|
+
To verify:
|
|
43
|
+
|
|
44
|
+
```bash
|
|
45
|
+
nedbd --doctor # diagnose your NEDB environment
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
---
|
|
49
|
+
|
|
50
|
+
## Usage
|
|
51
|
+
|
|
52
|
+
```
|
|
53
|
+
nedbd [OPTIONS] [data_dir]
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
| Argument | Default | Description |
|
|
57
|
+
|---|---|---|
|
|
58
|
+
| `data_dir` | `./nedb-data` | Directory for database files |
|
|
59
|
+
| `--dag` | off | Use the v2 content-addressed DAG engine |
|
|
60
|
+
| `--doctor` | — | Diagnose environment, print fix commands |
|
|
61
|
+
|
|
62
|
+
### Environment variables
|
|
63
|
+
|
|
64
|
+
| Variable | Default | Description |
|
|
65
|
+
|---|---|---|
|
|
66
|
+
| `NEDBD_HOST` | `127.0.0.1` | Bind address |
|
|
67
|
+
| `NEDBD_PORT` | `7070` | HTTP port |
|
|
68
|
+
| `NEDBD_TOKEN` | — | Bearer token for auth (optional) |
|
|
69
|
+
| `NEDB_TMK` | — | 64-char hex master key for AES-256-GCM encryption at rest |
|
|
70
|
+
| `NEDBD_MEMORY` | — | `1` = pure in-memory mode (no disk I/O) |
|
|
71
|
+
| `NEDBD_DAG` | — | `1` = same as `--dag` flag |
|
|
72
|
+
|
|
73
|
+
---
|
|
74
|
+
|
|
75
|
+
## HTTP API
|
|
76
|
+
|
|
77
|
+
All endpoints return JSON. Auth: `Authorization: Bearer <token>` if `NEDBD_TOKEN` is set.
|
|
78
|
+
|
|
79
|
+
```
|
|
80
|
+
GET /health
|
|
81
|
+
GET /v1/databases
|
|
82
|
+
POST /v1/databases {name, init?}
|
|
83
|
+
GET /v1/databases/<name>
|
|
84
|
+
DELETE /v1/databases/<name>
|
|
85
|
+
POST /v1/databases/<name>/put {coll, id, doc, caused_by?}
|
|
86
|
+
POST /v1/databases/<name>/query {nql}
|
|
87
|
+
POST /v1/databases/<name>/link {frm, rel, to}
|
|
88
|
+
POST /v1/databases/<name>/neighbors {node, rel}
|
|
89
|
+
GET /v1/databases/<name>/verify
|
|
90
|
+
POST /v1/databases/<name>/checkpoint
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
---
|
|
94
|
+
|
|
95
|
+
## NQL — NEDB Query Language
|
|
96
|
+
|
|
97
|
+
```sql
|
|
98
|
+
-- Basic queries
|
|
99
|
+
FROM person LIMIT 10
|
|
100
|
+
FROM driver WHERE status = "active"
|
|
101
|
+
FROM driver WHERE rating >= 4.5 ORDER BY rating DESC
|
|
102
|
+
|
|
103
|
+
-- Time-travel
|
|
104
|
+
FROM will WHERE _id = "evans_will_2019" AS OF 5
|
|
105
|
+
|
|
106
|
+
-- Causal trace (walks caused_by links backward)
|
|
107
|
+
FROM event WHERE _id = "probate_filing" TRACE caused_by
|
|
108
|
+
|
|
109
|
+
-- Graph traversal
|
|
110
|
+
FROM person WHERE _id = "robert" TRAVERSE parent_of
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
---
|
|
114
|
+
|
|
115
|
+
## Example: The Will — causal DAG in action
|
|
116
|
+
|
|
117
|
+
```python
|
|
118
|
+
import json, os, urllib.request
|
|
119
|
+
|
|
120
|
+
def put(db_url, coll, id_, doc):
|
|
121
|
+
body = json.dumps({"coll": coll, "id": id_, "doc": doc}).encode()
|
|
122
|
+
req = urllib.request.Request(f"{db_url}/put", body,
|
|
123
|
+
{"Content-Type": "application/json"}, method="POST")
|
|
124
|
+
return json.loads(urllib.request.urlopen(req).read())["doc"]
|
|
125
|
+
|
|
126
|
+
BASE = "http://localhost:7070/v1/databases/will"
|
|
127
|
+
|
|
128
|
+
# Robert writes his will
|
|
129
|
+
robert = put(BASE, "person", "robert", {"name": "Robert Evans", "role": "testator"})
|
|
130
|
+
will_v1 = put(BASE, "will", "evans_will", {
|
|
131
|
+
"house": "mark", "business": "lisa",
|
|
132
|
+
"caused_by": [robert["_hash"]], # causal link to testator record
|
|
133
|
+
})
|
|
134
|
+
|
|
135
|
+
# Amendment — chains off v1
|
|
136
|
+
will_v2 = put(BASE, "will", "evans_will", {
|
|
137
|
+
"house": "mark", "business": "lisa", "vintage_car": "mark",
|
|
138
|
+
"caused_by": [will_v1["_hash"]],
|
|
139
|
+
})
|
|
140
|
+
|
|
141
|
+
# TRACE proves the amendment links back to the original
|
|
142
|
+
query = json.dumps({"nql": 'FROM will WHERE _id = "evans_will" TRACE caused_by'}).encode()
|
|
143
|
+
req = urllib.request.Request(f"{BASE}/query", query,
|
|
144
|
+
{"Content-Type": "application/json"}, method="POST")
|
|
145
|
+
trace = json.loads(urllib.request.urlopen(req).read())["rows"]
|
|
146
|
+
# → both versions + Robert's testator record, in causal order
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
---
|
|
150
|
+
|
|
151
|
+
## Python companion
|
|
152
|
+
|
|
153
|
+
The `nedb-engine` PyPI package ships the same server binary bundled in the wheel, plus:
|
|
154
|
+
- Pure-Python AOF engine (`NEDB` class)
|
|
155
|
+
- Embedded v2 DAG API (`nedb._native.NedbCore`) on supported platforms
|
|
156
|
+
- `nedbd` console script
|
|
157
|
+
|
|
158
|
+
```bash
|
|
159
|
+
pip install nedb-engine
|
|
160
|
+
|
|
161
|
+
# Use embedded API (Linux/macOS/Windows CPython)
|
|
162
|
+
python3 -c "from nedb._native import NedbCore; db = NedbCore(); ..."
|
|
163
|
+
|
|
164
|
+
# Use HTTP mode (any platform, including MSYS2/MinGW)
|
|
165
|
+
NEDB_URL=http://localhost:7070 python3 your_script.py
|
|
166
|
+
|
|
167
|
+
# Diagnose
|
|
168
|
+
nedbd --doctor
|
|
169
|
+
```
|
|
170
|
+
|
|
171
|
+
---
|
|
172
|
+
|
|
173
|
+
## License
|
|
174
|
+
|
|
175
|
+
[BUSL-1.1](https://github.com/Eth-Interchained/nedb/blob/master/LICENSE) — Business Source License. Free for development and evaluation; production use requires a commercial licence from INTERCHAINED, LLC.
|
|
176
|
+
|
|
177
|
+
---
|
|
178
|
+
|
|
179
|
+
*Built by [INTERCHAINED, LLC](https://interchained.org) × Claude Sonnet 4.6*
|
|
Binary file
|
|
Binary file
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|