@xdarkicex/openclaw-memory-libravdb 1.3.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (80) hide show
  1. package/README.md +46 -0
  2. package/docs/README.md +14 -0
  3. package/docs/architecture-decisions/README.md +6 -0
  4. package/docs/architecture-decisions/adr-001-onnx-over-ollama.md +21 -0
  5. package/docs/architecture-decisions/adr-002-libravdb-over-lancedb.md +19 -0
  6. package/docs/architecture-decisions/adr-003-convex-gating-over-threshold.md +27 -0
  7. package/docs/architecture-decisions/adr-004-sidecar-over-native-ts.md +21 -0
  8. package/docs/architecture.md +188 -0
  9. package/docs/contributing.md +76 -0
  10. package/docs/dependencies.md +38 -0
  11. package/docs/embedding-profiles.md +42 -0
  12. package/docs/gating.md +329 -0
  13. package/docs/implementation.md +381 -0
  14. package/docs/installation.md +272 -0
  15. package/docs/mathematics.md +695 -0
  16. package/docs/models.md +63 -0
  17. package/docs/problem.md +64 -0
  18. package/docs/security.md +86 -0
  19. package/openclaw.plugin.json +84 -0
  20. package/package.json +41 -0
  21. package/scripts/build-sidecar.sh +30 -0
  22. package/scripts/postinstall.js +169 -0
  23. package/scripts/setup.sh +20 -0
  24. package/scripts/setup.ts +505 -0
  25. package/scripts/sidecar-release.d.ts +4 -0
  26. package/scripts/sidecar-release.js +17 -0
  27. package/sidecar/cmd/inspect_onnx/main.go +105 -0
  28. package/sidecar/compact/gate.go +273 -0
  29. package/sidecar/compact/gate_test.go +85 -0
  30. package/sidecar/compact/summarize.go +345 -0
  31. package/sidecar/compact/summarize_test.go +319 -0
  32. package/sidecar/compact/tokens.go +11 -0
  33. package/sidecar/config/config.go +119 -0
  34. package/sidecar/config/config_test.go +75 -0
  35. package/sidecar/embed/engine.go +696 -0
  36. package/sidecar/embed/engine_test.go +349 -0
  37. package/sidecar/embed/matryoshka.go +93 -0
  38. package/sidecar/embed/matryoshka_test.go +150 -0
  39. package/sidecar/embed/onnx_local.go +319 -0
  40. package/sidecar/embed/onnx_local_test.go +159 -0
  41. package/sidecar/embed/profile_contract_test.go +71 -0
  42. package/sidecar/embed/profile_eval_test.go +923 -0
  43. package/sidecar/embed/profiles.go +39 -0
  44. package/sidecar/go.mod +21 -0
  45. package/sidecar/go.sum +30 -0
  46. package/sidecar/health/check.go +33 -0
  47. package/sidecar/health/check_test.go +55 -0
  48. package/sidecar/main.go +151 -0
  49. package/sidecar/model/encoder.go +222 -0
  50. package/sidecar/model/registry.go +262 -0
  51. package/sidecar/model/registry_test.go +102 -0
  52. package/sidecar/model/seq2seq.go +133 -0
  53. package/sidecar/server/rpc.go +343 -0
  54. package/sidecar/server/rpc_test.go +350 -0
  55. package/sidecar/server/transport.go +160 -0
  56. package/sidecar/store/libravdb.go +676 -0
  57. package/sidecar/store/libravdb_test.go +472 -0
  58. package/sidecar/summarize/engine.go +360 -0
  59. package/sidecar/summarize/engine_test.go +148 -0
  60. package/sidecar/summarize/onnx_local.go +494 -0
  61. package/sidecar/summarize/onnx_local_test.go +48 -0
  62. package/sidecar/summarize/profiles.go +52 -0
  63. package/sidecar/summarize/tokenizer.go +13 -0
  64. package/sidecar/summarize/tokenizer_hf.go +76 -0
  65. package/sidecar/summarize/util.go +13 -0
  66. package/src/cli.ts +205 -0
  67. package/src/context-engine.ts +195 -0
  68. package/src/index.ts +27 -0
  69. package/src/memory-provider.ts +24 -0
  70. package/src/openclaw-plugin-sdk.d.ts +53 -0
  71. package/src/plugin-runtime.ts +67 -0
  72. package/src/recall-cache.ts +34 -0
  73. package/src/recall-utils.ts +22 -0
  74. package/src/rpc.ts +84 -0
  75. package/src/scoring.ts +58 -0
  76. package/src/sidecar.ts +506 -0
  77. package/src/tokens.ts +36 -0
  78. package/src/types.ts +146 -0
  79. package/tsconfig.json +20 -0
  80. package/tsconfig.tests.json +12 -0
@@ -0,0 +1,272 @@
1
+ # Installation Reference
2
+
3
+ This document is the full installation reference for `@xdarkicex/openclaw-memory-libravdb`. For the short path, use the root [README.md](../README.md).
4
+
5
+ ## System Requirements
6
+
7
+ | Requirement | Minimum | Recommended | Notes |
8
+ |---|---|---|---|
9
+ | Node.js | `22.0.0` | Latest LTS | Enforced in [`package.json`](../package.json) `engines.node` |
10
+ | OpenClaw | `2026.3.22` | Current stable | Pinned by [`package.json`](../package.json) `peerDependencies.openclaw`; this is the earliest local tag confirmed to expose `definePluginEntry`, `registerContextEngine`, `registerMemoryPromptSection`, and the plugin API shape this repo uses |
11
+ | Go | `1.22` | Latest stable | Dev/fallback build only; not required when prebuilt release assets exist |
12
+ | Disk | about `1 GB` free for default Nomic install | `2 GB+` if provisioning optional T5 and leaving room for DB growth | See Resource Requirements below |
13
+ | RAM | about `512 MB` for embed-only runtime | `1 GB+` if optional T5 summarizer is provisioned | Based on local RSS measurements below |
14
+ | OS | macOS, Linux, Windows | Current stable releases | Windows uses TCP loopback instead of Unix sockets |
15
+ | Architecture | `arm64`, `x64` | Match published release assets | Current release matrix builds five sidecar targets |
16
+
17
+ The published install path is prebuilt-first. End users should not normally need Go.
18
+
19
+ ## Resource Requirements
20
+
21
+ The numbers in this section are either directly measured from the current local
22
+ build on `2026-03-29` or explicitly labeled as estimates.
23
+
24
+ ### Disk
25
+
26
+ Measured locally from this checkout:
27
+
28
+ - sidecar binary: `7.7M`
29
+ - bundled Nomic model directory: `523M`
30
+ - bundled MiniLM fallback model directory: `87M`
31
+ - optional T5 summarizer directory: `371M`
32
+ - unpacked ONNX Runtime directory on macOS arm64: `44M`
33
+ - ONNX Runtime archive download on macOS arm64: `9.5M`
34
+
35
+ Practical footprints:
36
+
37
+ - default quality-first install without optional T5:
38
+ about `575 MB` (`7.7M + 523M + 44M`)
39
+ - install with optional T5 summarizer:
40
+ about `946 MB`
41
+
42
+ Vector payload lower bounds for stored turns, derived from embedding dimension:
43
+
44
+ - MiniLM `384d`: `384 * 4 = 1536 bytes` per vector
45
+ - Nomic `768d`: `768 * 4 = 3072 bytes` per vector
46
+
47
+ Estimated lower-bound vector payload for `10,000` stored turns:
48
+
49
+ - MiniLM: about `15.4 MB`
50
+ - Nomic: about `30.7 MB`
51
+
52
+ These are lower bounds for vector payload only. Actual on-disk LibraVDB usage is
53
+ higher because text, metadata, collection structure, and index state are stored
54
+ as well.
55
+
56
+ ### Memory
57
+
58
+ Measured locally on Apple M2, `2026-03-29`, by starting the sidecar and reading
59
+ RSS after startup:
60
+
61
+ - idle RSS with Nomic embedding path loaded and no optional T5 summarizer:
62
+ about `271,872 KB` (`~266 MB`)
63
+ - idle RSS with Nomic plus local ONNX T5 summarizer loaded:
64
+ about `515,312 KB` (`~503 MB`)
65
+
66
+ Not yet bench-measured in the repo:
67
+
68
+ - RSS during active inference
69
+ - peak RSS during compaction of large clusters
70
+
71
+ Current operational estimate:
72
+
73
+ - embedding inference should remain close to the embed-only idle baseline plus
74
+ transient ONNX workspace allocation
75
+ - optional T5 provisioning roughly doubles steady-state RSS
76
+
77
+ ### CPU
78
+
79
+ Measured locally from the existing Go benchmark harness on Apple M2,
80
+ `2026-03-29`:
81
+
82
+ - MiniLM bundled query embedding: about `22.6 ms/op`
83
+ - MiniLM onnx-local query embedding: about `16.3 ms/op`
84
+ - Nomic onnx-local query embedding: about `43.7 ms/op`
85
+
86
+ Measured locally from a one-off 40-query timing sample on Apple M2,
87
+ `2026-03-29`:
88
+
89
+ - Nomic query embedding `p50`: about `18.61 ms`
90
+ - Nomic query embedding `p95`: about `24.19 ms`
91
+
92
+ Measured locally from a one-off synthetic 50-turn compaction run using the
93
+ current extractive summarizer and Nomic embeddings:
94
+
95
+ - `50`-turn extractive compaction wall time: about `3175 ms`
96
+
97
+ Not yet bench-measured in the repo:
98
+
99
+ - equivalent Linux x64 embedding latency on a reference machine
100
+ - `50`-turn compaction wall time through the optional ONNX T5 abstractive path
101
+
102
+ ### Network
103
+
104
+ Setup downloads are front-loaded. After installation, the plugin is local-first.
105
+
106
+ Current setup assets:
107
+
108
+ - Nomic model: about `522 MB`
109
+ - T5-small encoder: about `135 MB`
110
+ - T5-small decoder: about `222 MB`
111
+ - ONNX Runtime macOS arm64 archive: about `9.5 MB`
112
+
113
+ After install, the plugin makes no required network calls for embedding or
114
+ extractive compaction. The only optional runtime network path is:
115
+
116
+ - `summarizerBackend = "ollama-local"` or another custom summarizer endpoint
117
+
118
+ ## Standard Install
119
+
120
+ ```bash
121
+ openclaw plugins install @xdarkicex/openclaw-memory-libravdb
122
+ ```
123
+
124
+ Expected successful install shape on a published release:
125
+
126
+ ```text
127
+ [openclaw-memory-libravdb] Sidecar installed (prebuilt clawdb-sidecar-<platform>)
128
+ [openclaw-memory-libravdb] Provisioning embedding model...
129
+ [openclaw-memory-libravdb] Provisioning ONNX runtime...
130
+ [openclaw-memory-libravdb] Provisioning summarizer model... (optional)
131
+ [openclaw-memory-libravdb] Verifying sidecar health...
132
+ [openclaw-memory-libravdb] Setup complete.
133
+ Installed plugin: libravdb-memory
134
+ ```
135
+
136
+ If the host also activates the plugin into the exclusive memory slot during the
137
+ same flow, output should additionally include a line like:
138
+
139
+ ```text
140
+ Exclusive slot "memory" switched from "memory-core" to "libravdb-memory".
141
+ ```
142
+
143
+ That slot-takeover line is the proof that OpenClaw is no longer using the stock
144
+ memory provider.
145
+
146
+ Development fallback shape when a prebuilt sidecar asset is not available:
147
+
148
+ ```text
149
+ [openclaw-memory-libravdb] Prebuilt binary unavailable. Attempting local go build...
150
+ [openclaw-memory-libravdb] This requires Go >= 1.22: https://go.dev/dl/
151
+ [openclaw-memory-libravdb] Sidecar installed (local build)
152
+ ...
153
+ ```
154
+
155
+ Published users should rarely see the fallback path. If they do, the plugin
156
+ version likely has not published sidecar release assets for that platform yet.
157
+
158
+ ## Activation
159
+
160
+ The plugin declares `kind: "memory"` and is intended to occupy the `memory` slot. If your OpenClaw build also exposes legacy context-engine slotting, keep the memory slot authoritative and use the context-engine slot only for compatibility testing.
161
+
162
+ Add this to `~/.openclaw/openclaw.json`:
163
+
164
+ ```json
165
+ {
166
+ "plugins": {
167
+ "slots": {
168
+ "memory": "libravdb-memory",
169
+ "contextEngine": "legacy"
170
+ }
171
+ }
172
+ }
173
+ ```
174
+
175
+ Notes:
176
+
177
+ - `memory: "libravdb-memory"` is the actual activation step.
178
+ - `contextEngine: "legacy"` keeps the legacy engine explicit when the host still exposes that slot.
179
+ - If you instead point `contextEngine` at another plugin, you are changing a separate slot from the memory replacement.
180
+ - The plugin id is `libravdb-memory`. The npm package name used at install time is `@xdarkicex/openclaw-memory-libravdb`.
181
+
182
+ Without the `memory` slot entry, OpenClaw's default memory can continue to run in parallel.
183
+
184
+ ## Verification
185
+
186
+ Run:
187
+
188
+ ```bash
189
+ openclaw memory status
190
+ ```
191
+
192
+ Expected output shape:
193
+
194
+ ```text
195
+ ┌────────────────────┬──────────────────────────────┐
196
+ │ Sidecar │ running │
197
+ │ Turns stored │ 0 │
198
+ │ Memories stored │ 0 │
199
+ │ Gate threshold │ 0.35 │
200
+ │ Abstractive model │ ready | not provisioned │
201
+ │ Embedding profile │ nomic-embed-text-v1.5 │
202
+ │ Message │ ok │
203
+ └────────────────────┴──────────────────────────────┘
204
+ ```
205
+
206
+ Interpretation:
207
+
208
+ - `Sidecar=running` means the Go sidecar booted and answered JSON-RPC `health`.
209
+ - `Gate threshold=0.35` confirms the default gating scalar boundary is active.
210
+ - `Abstractive model=not provisioned` is acceptable. The system degrades to extractive compaction.
211
+
212
+ ## Contributor Install
213
+
214
+ For contributors working from a clone:
215
+
216
+ ```bash
217
+ pnpm check
218
+ cd sidecar && env GOCACHE=/tmp/openclaw-memory-libravdb-gocache go test -race ./... && cd ..
219
+ node scripts/setup.ts
220
+ ```
221
+
222
+ Optional direct dev build:
223
+
224
+ ```bash
225
+ bash scripts/build-sidecar.sh
226
+ ```
227
+
228
+ This produces a local sidecar in `.sidecar-bin/` and copies any locally available model/runtime assets there for testing.
229
+
230
+ ## Troubleshooting
231
+
232
+ ### Sidecar fails to start
233
+
234
+ Common causes:
235
+
236
+ - ONNX Runtime library missing or unpacked in the wrong place
237
+ - downloaded model file hash mismatch
238
+ - local Go fallback unavailable and no prebuilt asset for the requested version
239
+
240
+ Check:
241
+
242
+ ```bash
243
+ openclaw memory status
244
+ ```
245
+
246
+ If the sidecar is down, rerun:
247
+
248
+ ```bash
249
+ node scripts/setup.ts
250
+ ```
251
+
252
+ ### Model download fails
253
+
254
+ The setup script verifies hashes for required assets. A failed or partial download is deleted and retried on the next run. This is intentional. A model file that exists but fails hash verification is treated as corrupt.
255
+
256
+ ### Hash mismatch
257
+
258
+ Hash mismatch means one of:
259
+
260
+ - the release asset is corrupt
261
+ - the local cache is stale
262
+ - the expected checksum is wrong
263
+
264
+ Do not bypass this. Delete the asset and rerun setup, or republish the release with corrected checksums.
265
+
266
+ ### Windows behavior
267
+
268
+ On Windows the sidecar advertises a loopback TCP endpoint instead of a Unix socket. This is expected. The plugin’s transport layer already handles the fallback.
269
+
270
+ ### Local fallback path
271
+
272
+ If the installer logs that it is attempting a local `go build`, the prebuilt release asset was not available for the plugin version being installed. For published tags this should be unusual; for branch or unreleased work it is expected.