@xdarkicex/openclaw-memory-libravdb 1.3.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +46 -0
- package/docs/README.md +14 -0
- package/docs/architecture-decisions/README.md +6 -0
- package/docs/architecture-decisions/adr-001-onnx-over-ollama.md +21 -0
- package/docs/architecture-decisions/adr-002-libravdb-over-lancedb.md +19 -0
- package/docs/architecture-decisions/adr-003-convex-gating-over-threshold.md +27 -0
- package/docs/architecture-decisions/adr-004-sidecar-over-native-ts.md +21 -0
- package/docs/architecture.md +188 -0
- package/docs/contributing.md +76 -0
- package/docs/dependencies.md +38 -0
- package/docs/embedding-profiles.md +42 -0
- package/docs/gating.md +329 -0
- package/docs/implementation.md +381 -0
- package/docs/installation.md +272 -0
- package/docs/mathematics.md +695 -0
- package/docs/models.md +63 -0
- package/docs/problem.md +64 -0
- package/docs/security.md +86 -0
- package/openclaw.plugin.json +84 -0
- package/package.json +41 -0
- package/scripts/build-sidecar.sh +30 -0
- package/scripts/postinstall.js +169 -0
- package/scripts/setup.sh +20 -0
- package/scripts/setup.ts +505 -0
- package/scripts/sidecar-release.d.ts +4 -0
- package/scripts/sidecar-release.js +17 -0
- package/sidecar/cmd/inspect_onnx/main.go +105 -0
- package/sidecar/compact/gate.go +273 -0
- package/sidecar/compact/gate_test.go +85 -0
- package/sidecar/compact/summarize.go +345 -0
- package/sidecar/compact/summarize_test.go +319 -0
- package/sidecar/compact/tokens.go +11 -0
- package/sidecar/config/config.go +119 -0
- package/sidecar/config/config_test.go +75 -0
- package/sidecar/embed/engine.go +696 -0
- package/sidecar/embed/engine_test.go +349 -0
- package/sidecar/embed/matryoshka.go +93 -0
- package/sidecar/embed/matryoshka_test.go +150 -0
- package/sidecar/embed/onnx_local.go +319 -0
- package/sidecar/embed/onnx_local_test.go +159 -0
- package/sidecar/embed/profile_contract_test.go +71 -0
- package/sidecar/embed/profile_eval_test.go +923 -0
- package/sidecar/embed/profiles.go +39 -0
- package/sidecar/go.mod +21 -0
- package/sidecar/go.sum +30 -0
- package/sidecar/health/check.go +33 -0
- package/sidecar/health/check_test.go +55 -0
- package/sidecar/main.go +151 -0
- package/sidecar/model/encoder.go +222 -0
- package/sidecar/model/registry.go +262 -0
- package/sidecar/model/registry_test.go +102 -0
- package/sidecar/model/seq2seq.go +133 -0
- package/sidecar/server/rpc.go +343 -0
- package/sidecar/server/rpc_test.go +350 -0
- package/sidecar/server/transport.go +160 -0
- package/sidecar/store/libravdb.go +676 -0
- package/sidecar/store/libravdb_test.go +472 -0
- package/sidecar/summarize/engine.go +360 -0
- package/sidecar/summarize/engine_test.go +148 -0
- package/sidecar/summarize/onnx_local.go +494 -0
- package/sidecar/summarize/onnx_local_test.go +48 -0
- package/sidecar/summarize/profiles.go +52 -0
- package/sidecar/summarize/tokenizer.go +13 -0
- package/sidecar/summarize/tokenizer_hf.go +76 -0
- package/sidecar/summarize/util.go +13 -0
- package/src/cli.ts +205 -0
- package/src/context-engine.ts +195 -0
- package/src/index.ts +27 -0
- package/src/memory-provider.ts +24 -0
- package/src/openclaw-plugin-sdk.d.ts +53 -0
- package/src/plugin-runtime.ts +67 -0
- package/src/recall-cache.ts +34 -0
- package/src/recall-utils.ts +22 -0
- package/src/rpc.ts +84 -0
- package/src/scoring.ts +58 -0
- package/src/sidecar.ts +506 -0
- package/src/tokens.ts +36 -0
- package/src/types.ts +146 -0
- package/tsconfig.json +20 -0
- package/tsconfig.tests.json +12 -0
|
@@ -0,0 +1,272 @@
|
|
|
1
|
+
# Installation Reference
|
|
2
|
+
|
|
3
|
+
This document is the full installation reference for `@xdarkicex/openclaw-memory-libravdb`. For the short path, use the root [README.md](../README.md).
|
|
4
|
+
|
|
5
|
+
## System Requirements
|
|
6
|
+
|
|
7
|
+
| Requirement | Minimum | Recommended | Notes |
|
|
8
|
+
|---|---|---|---|
|
|
9
|
+
| Node.js | `22.0.0` | Latest LTS | Enforced in [`package.json`](../package.json) `engines.node` |
|
|
10
|
+
| OpenClaw | `2026.3.22` | Current stable | Pinned by [`package.json`](../package.json) `peerDependencies.openclaw`; this is the earliest local tag confirmed to expose `definePluginEntry`, `registerContextEngine`, `registerMemoryPromptSection`, and the plugin API shape this repo uses |
|
|
11
|
+
| Go | `1.22` | Latest stable | Dev/fallback build only; not required when prebuilt release assets exist |
|
|
12
|
+
| Disk | about `1 GB` free for default Nomic install | `2 GB+` if provisioning optional T5 and leaving room for DB growth | See Resource Requirements below |
|
|
13
|
+
| RAM | about `512 MB` for embed-only runtime | `1 GB+` if optional T5 summarizer is provisioned | Based on local RSS measurements below |
|
|
14
|
+
| OS | macOS, Linux, Windows | Current stable releases | Windows uses TCP loopback instead of Unix sockets |
|
|
15
|
+
| Architecture | `arm64`, `x64` | Match published release assets | Current release matrix builds five sidecar targets |
|
|
16
|
+
|
|
17
|
+
The published install path is prebuilt-first. End users should not normally need Go.
|
|
18
|
+
|
|
19
|
+
## Resource Requirements
|
|
20
|
+
|
|
21
|
+
The numbers in this section are either directly measured from the current local
|
|
22
|
+
build on `2026-03-29` or explicitly labeled as estimates.
|
|
23
|
+
|
|
24
|
+
### Disk
|
|
25
|
+
|
|
26
|
+
Measured locally from this checkout:
|
|
27
|
+
|
|
28
|
+
- sidecar binary: `7.7M`
|
|
29
|
+
- bundled Nomic model directory: `523M`
|
|
30
|
+
- bundled MiniLM fallback model directory: `87M`
|
|
31
|
+
- optional T5 summarizer directory: `371M`
|
|
32
|
+
- unpacked ONNX Runtime directory on macOS arm64: `44M`
|
|
33
|
+
- ONNX Runtime archive download on macOS arm64: `9.5M`
|
|
34
|
+
|
|
35
|
+
Practical footprints:
|
|
36
|
+
|
|
37
|
+
- default quality-first install without optional T5:
|
|
38
|
+
about `575 MB` (`7.7M + 523M + 44M`)
|
|
39
|
+
- install with optional T5 summarizer:
|
|
40
|
+
about `946 MB`
|
|
41
|
+
|
|
42
|
+
Vector payload lower bounds for stored turns, derived from embedding dimension:
|
|
43
|
+
|
|
44
|
+
- MiniLM `384d`: `384 * 4 = 1536 bytes` per vector
|
|
45
|
+
- Nomic `768d`: `768 * 4 = 3072 bytes` per vector
|
|
46
|
+
|
|
47
|
+
Estimated lower-bound vector payload for `10,000` stored turns:
|
|
48
|
+
|
|
49
|
+
- MiniLM: about `15.4 MB`
|
|
50
|
+
- Nomic: about `30.7 MB`
|
|
51
|
+
|
|
52
|
+
These are lower bounds for vector payload only. Actual on-disk LibraVDB usage is
|
|
53
|
+
higher because text, metadata, collection structure, and index state are stored
|
|
54
|
+
as well.
|
|
55
|
+
|
|
56
|
+
### Memory
|
|
57
|
+
|
|
58
|
+
Measured locally on Apple M2, `2026-03-29`, by starting the sidecar and reading
|
|
59
|
+
RSS after startup:
|
|
60
|
+
|
|
61
|
+
- idle RSS with Nomic embedding path loaded and no optional T5 summarizer:
|
|
62
|
+
about `271,872 KB` (`~266 MB`)
|
|
63
|
+
- idle RSS with Nomic plus local ONNX T5 summarizer loaded:
|
|
64
|
+
about `515,312 KB` (`~503 MB`)
|
|
65
|
+
|
|
66
|
+
Not yet bench-measured in the repo:
|
|
67
|
+
|
|
68
|
+
- RSS during active inference
|
|
69
|
+
- peak RSS during compaction of large clusters
|
|
70
|
+
|
|
71
|
+
Current operational estimate:
|
|
72
|
+
|
|
73
|
+
- embedding inference should remain close to the embed-only idle baseline plus
|
|
74
|
+
transient ONNX workspace allocation
|
|
75
|
+
- optional T5 provisioning roughly doubles steady-state RSS
|
|
76
|
+
|
|
77
|
+
### CPU
|
|
78
|
+
|
|
79
|
+
Measured locally from the existing Go benchmark harness on Apple M2,
|
|
80
|
+
`2026-03-29`:
|
|
81
|
+
|
|
82
|
+
- MiniLM bundled query embedding: about `22.6 ms/op`
|
|
83
|
+
- MiniLM onnx-local query embedding: about `16.3 ms/op`
|
|
84
|
+
- Nomic onnx-local query embedding: about `43.7 ms/op`
|
|
85
|
+
|
|
86
|
+
Measured locally from a one-off 40-query timing sample on Apple M2,
|
|
87
|
+
`2026-03-29`:
|
|
88
|
+
|
|
89
|
+
- Nomic query embedding `p50`: about `18.61 ms`
|
|
90
|
+
- Nomic query embedding `p95`: about `24.19 ms`
|
|
91
|
+
|
|
92
|
+
Measured locally from a one-off synthetic 50-turn compaction run using the
|
|
93
|
+
current extractive summarizer and Nomic embeddings:
|
|
94
|
+
|
|
95
|
+
- `50`-turn extractive compaction wall time: about `3175 ms`
|
|
96
|
+
|
|
97
|
+
Not yet bench-measured in the repo:
|
|
98
|
+
|
|
99
|
+
- equivalent Linux x64 embedding latency on a reference machine
|
|
100
|
+
- `50`-turn compaction wall time through the optional ONNX T5 abstractive path
|
|
101
|
+
|
|
102
|
+
### Network
|
|
103
|
+
|
|
104
|
+
Setup downloads are front-loaded. After installation, the plugin is local-first.
|
|
105
|
+
|
|
106
|
+
Current setup assets:
|
|
107
|
+
|
|
108
|
+
- Nomic model: about `522 MB`
|
|
109
|
+
- T5-small encoder: about `135 MB`
|
|
110
|
+
- T5-small decoder: about `222 MB`
|
|
111
|
+
- ONNX Runtime macOS arm64 archive: about `9.5 MB`
|
|
112
|
+
|
|
113
|
+
After install, the plugin makes no required network calls for embedding or
|
|
114
|
+
extractive compaction. The only optional runtime network path is:
|
|
115
|
+
|
|
116
|
+
- `summarizerBackend = "ollama-local"` or another custom summarizer endpoint
|
|
117
|
+
|
|
118
|
+
## Standard Install
|
|
119
|
+
|
|
120
|
+
```bash
|
|
121
|
+
openclaw plugins install @xdarkicex/openclaw-memory-libravdb
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
Expected successful install shape on a published release:
|
|
125
|
+
|
|
126
|
+
```text
|
|
127
|
+
[openclaw-memory-libravdb] Sidecar installed (prebuilt clawdb-sidecar-<platform>)
|
|
128
|
+
[openclaw-memory-libravdb] Provisioning embedding model...
|
|
129
|
+
[openclaw-memory-libravdb] Provisioning ONNX runtime...
|
|
130
|
+
[openclaw-memory-libravdb] Provisioning summarizer model... (optional)
|
|
131
|
+
[openclaw-memory-libravdb] Verifying sidecar health...
|
|
132
|
+
[openclaw-memory-libravdb] Setup complete.
|
|
133
|
+
Installed plugin: libravdb-memory
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
If the host also activates the plugin into the exclusive memory slot during the
|
|
137
|
+
same flow, output should additionally include a line like:
|
|
138
|
+
|
|
139
|
+
```text
|
|
140
|
+
Exclusive slot "memory" switched from "memory-core" to "libravdb-memory".
|
|
141
|
+
```
|
|
142
|
+
|
|
143
|
+
That slot-takeover line is the proof that OpenClaw is no longer using the stock
|
|
144
|
+
memory provider.
|
|
145
|
+
|
|
146
|
+
Development fallback shape when a prebuilt sidecar asset is not available:
|
|
147
|
+
|
|
148
|
+
```text
|
|
149
|
+
[openclaw-memory-libravdb] Prebuilt binary unavailable. Attempting local go build...
|
|
150
|
+
[openclaw-memory-libravdb] This requires Go >= 1.22: https://go.dev/dl/
|
|
151
|
+
[openclaw-memory-libravdb] Sidecar installed (local build)
|
|
152
|
+
...
|
|
153
|
+
```
|
|
154
|
+
|
|
155
|
+
Published users should rarely see the fallback path. If they do, the plugin
|
|
156
|
+
version likely has not published sidecar release assets for that platform yet.
|
|
157
|
+
|
|
158
|
+
## Activation
|
|
159
|
+
|
|
160
|
+
The plugin declares `kind: "memory"` and is intended to occupy the `memory` slot. If your OpenClaw build also exposes legacy context-engine slotting, keep the memory slot authoritative and use the context-engine slot only for compatibility testing.
|
|
161
|
+
|
|
162
|
+
Add this to `~/.openclaw/openclaw.json`:
|
|
163
|
+
|
|
164
|
+
```json
|
|
165
|
+
{
|
|
166
|
+
"plugins": {
|
|
167
|
+
"slots": {
|
|
168
|
+
"memory": "libravdb-memory",
|
|
169
|
+
"contextEngine": "legacy"
|
|
170
|
+
}
|
|
171
|
+
}
|
|
172
|
+
}
|
|
173
|
+
```
|
|
174
|
+
|
|
175
|
+
Notes:
|
|
176
|
+
|
|
177
|
+
- `memory: "libravdb-memory"` is the actual activation step.
|
|
178
|
+
- `contextEngine: "legacy"` keeps the legacy engine explicit when the host still exposes that slot.
|
|
179
|
+
- If you instead point `contextEngine` at another plugin, you are changing a separate slot from the memory replacement.
|
|
180
|
+
- The plugin id is `libravdb-memory`. The npm package name used at install time is `@xdarkicex/openclaw-memory-libravdb`.
|
|
181
|
+
|
|
182
|
+
Without the `memory` slot entry, OpenClaw's default memory can continue to run in parallel.
|
|
183
|
+
|
|
184
|
+
## Verification
|
|
185
|
+
|
|
186
|
+
Run:
|
|
187
|
+
|
|
188
|
+
```bash
|
|
189
|
+
openclaw memory status
|
|
190
|
+
```
|
|
191
|
+
|
|
192
|
+
Expected output shape:
|
|
193
|
+
|
|
194
|
+
```text
|
|
195
|
+
┌────────────────────┬──────────────────────────────┐
|
|
196
|
+
│ Sidecar │ running │
|
|
197
|
+
│ Turns stored │ 0 │
|
|
198
|
+
│ Memories stored │ 0 │
|
|
199
|
+
│ Gate threshold │ 0.35 │
|
|
200
|
+
│ Abstractive model │ ready | not provisioned │
|
|
201
|
+
│ Embedding profile │ nomic-embed-text-v1.5 │
|
|
202
|
+
│ Message │ ok │
|
|
203
|
+
└────────────────────┴──────────────────────────────┘
|
|
204
|
+
```
|
|
205
|
+
|
|
206
|
+
Interpretation:
|
|
207
|
+
|
|
208
|
+
- `Sidecar=running` means the Go sidecar booted and answered JSON-RPC `health`.
|
|
209
|
+
- `Gate threshold=0.35` confirms the default gating scalar boundary is active.
|
|
210
|
+
- `Abstractive model=not provisioned` is acceptable. The system degrades to extractive compaction.
|
|
211
|
+
|
|
212
|
+
## Contributor Install
|
|
213
|
+
|
|
214
|
+
For contributors working from a clone:
|
|
215
|
+
|
|
216
|
+
```bash
|
|
217
|
+
pnpm check
|
|
218
|
+
cd sidecar && env GOCACHE=/tmp/openclaw-memory-libravdb-gocache go test -race ./... && cd ..
|
|
219
|
+
node scripts/setup.ts
|
|
220
|
+
```
|
|
221
|
+
|
|
222
|
+
Optional direct dev build:
|
|
223
|
+
|
|
224
|
+
```bash
|
|
225
|
+
bash scripts/build-sidecar.sh
|
|
226
|
+
```
|
|
227
|
+
|
|
228
|
+
This produces a local sidecar in `.sidecar-bin/` and copies any locally available model/runtime assets there for testing.
|
|
229
|
+
|
|
230
|
+
## Troubleshooting
|
|
231
|
+
|
|
232
|
+
### Sidecar fails to start
|
|
233
|
+
|
|
234
|
+
Common causes:
|
|
235
|
+
|
|
236
|
+
- ONNX Runtime library missing or unpacked in the wrong place
|
|
237
|
+
- downloaded model file hash mismatch
|
|
238
|
+
- local Go fallback unavailable and no prebuilt asset for the requested version
|
|
239
|
+
|
|
240
|
+
Check:
|
|
241
|
+
|
|
242
|
+
```bash
|
|
243
|
+
openclaw memory status
|
|
244
|
+
```
|
|
245
|
+
|
|
246
|
+
If the sidecar is down, rerun:
|
|
247
|
+
|
|
248
|
+
```bash
|
|
249
|
+
node scripts/setup.ts
|
|
250
|
+
```
|
|
251
|
+
|
|
252
|
+
### Model download fails
|
|
253
|
+
|
|
254
|
+
The setup script verifies hashes for required assets. A failed or partial download is deleted and retried on the next run. This is intentional. A model file that exists but fails hash verification is treated as corrupt.
|
|
255
|
+
|
|
256
|
+
### Hash mismatch
|
|
257
|
+
|
|
258
|
+
Hash mismatch means one of:
|
|
259
|
+
|
|
260
|
+
- the release asset is corrupt
|
|
261
|
+
- the local cache is stale
|
|
262
|
+
- the expected checksum is wrong
|
|
263
|
+
|
|
264
|
+
Do not bypass this. Delete the asset and rerun setup, or republish the release with corrected checksums.
|
|
265
|
+
|
|
266
|
+
### Windows behavior
|
|
267
|
+
|
|
268
|
+
On Windows the sidecar advertises a loopback TCP endpoint instead of a Unix socket. This is expected. The plugin’s transport layer already handles the fallback.
|
|
269
|
+
|
|
270
|
+
### Local fallback path
|
|
271
|
+
|
|
272
|
+
If the installer logs that it is attempting a local `go build`, the prebuilt release asset was not available for the plugin version being installed. For published tags this should be unusual; for branch or unreleased work it is expected.
|