@xdarkicex/openclaw-memory-libravdb 1.3.11 → 1.3.13

package/docs/gating.md

@@ -3,327 +3,125 @@
 This document describes the ingestion gate used to decide whether a user turn should be promoted into durable `user:` memory. It is the most novel scoring component in the repository.
 
 Implemented in:
-
-- [`sidecar/compact/gate.go`](../sidecar/compact/gate.go)
-- [`sidecar/compact/tokens.go`](../sidecar/compact/tokens.go)
-- [`sidecar/compact/summarize.go`](../sidecar/compact/summarize.go) for the
-  downstream abstractive-routing threshold
+- `sidecar/compact/gate.go`
+- `sidecar/compact/tokens.go`
+- `sidecar/compact/summarize.go` for the downstream abstractive-routing threshold
 
 ## 1. Why the Original Scalar Failed
 
 The original scalar assumed conversational memory semantics:
-
 - low novelty meant "already known"
 - repetition meant "probably redundant"
 - low natural-language structure meant "probably noise"
 
-That logic breaks for technical sessions.
-
-Repeated workflow context is often exactly what should be remembered:
-
-- file paths
-- APIs
-- failure signatures
-- configuration changes
-- architectural decisions
-
-In technical work, repetition can indicate persistent work context rather than low value.
+That logic breaks for technical sessions. Repeated workflow context is often exactly what should be remembered: file paths, APIs, failure signatures, configuration changes, and architectural decisions. In technical work, repetition can indicate persistent work context rather than low value.
 
 ## 2. The Convex Mixture
 
 The corrected gate is:
-
-$$
-G(t) = (1 - T(t)) \cdot G_{\mathrm{conv}}(t) + T(t) \cdot G_{\mathrm{tech}}(t)
-$$
+\[ G(t) = (1 - T(t)) \cdot G_{\mathrm{conv}}(t) + T(t) \cdot G_{\mathrm{tech}}(t) \]
 
 where:
+\[ G_{\mathrm{conv}}(t) = w_1^c H(t) + w_2^c R(t) + w_3^c D_{nl}(t) \]
+\[ G_{\mathrm{tech}}(t) = w_1^t P(t) + w_2^t A(t) + w_3^t D_{\mathrm{tech}}(t) \]
 
-$$
-G_{\mathrm{conv}}(t) = w_1^c H(t) + w_2^c R(t) + w_3^c D_{nl}(t)
-$$
-
-$$
-G_{\mathrm{tech}}(t) = w_1^t P(t) + w_2^t A(t) + w_3^t D_{\mathrm{tech}}(t)
-$$
-
-and:
-
-$$
-T(t) \in [0,1]
-$$
+and the domain indicator is bounded:
+\[ T(t) \in [0,1] \]
 
-is the technical-density signal.
-
-Current default weights from
-[`DefaultGatingConfig()`](../sidecar/compact/gate.go):
+### Weight Invariants
+To guarantee that the sub-branch scores remain strictly bounded to $[0,1]$, the configuration must satisfy:
+\[ \sum_{i=1}^3 w_i^c = 1 \quad \text{and} \quad \sum_{i=1}^3 w_i^t = 1 \]
 
+Current default weights from `DefaultGatingConfig()`:
 - conversational branch: $w_1^c = 0.35$, $w_2^c = 0.40$, $w_3^c = 0.25$
 - technical branch: $w_1^t = 0.40$, $w_2^t = 0.35$, $w_3^t = 0.25$
 
-### Boundedness
-
-If:
-
-- $T(t) \in [0,1]$
-- $G_{\mathrm{conv}}(t) \in [0,1]$
-- $G_{\mathrm{tech}}(t) \in [0,1]$
-
-then:
-
-$$
-G(t) \in [0,1]
-$$
-
-because $G$ is a convex combination of two values in $[0,1]$.
-
-### Continuity
+### Boundedness and Continuity
+Because $T(t) \in [0,1]$, $G_{\mathrm{conv}}(t) \in [0,1]$, and $G_{\mathrm{tech}}(t) \in [0,1]$, $G(t)$ is a true convex combination bounded to $[0,1]$.
 
 The gate is continuous in $T$:
-
-$$
-\frac{\partial G}{\partial T} = G_{\mathrm{tech}} - G_{\mathrm{conv}}
-$$
-
-There is no discontinuous jump at a domain boundary. A mixed technical/conversational turn interpolates smoothly between the two sub-formulas.
+\[ \frac{\partial G}{\partial T} = G_{\mathrm{tech}} - G_{\mathrm{conv}} \]
+There is no discontinuous jump at a domain boundary. A mixed technical/conversational turn interpolates smoothly.
 
 ## 3. Domain Detection $T(t)$
 
 Technical density is a weighted sum of technical patterns with saturation:
+\[ T(t) = \min\left(\frac{\sum_i s_i \cdot \mathbf{1}[\mathrm{pattern}_i(t)]}{\theta_{\mathrm{norm}}}, 1\right) \]
 
-$$
-T(t) = \min\left(\frac{\sum_i s_i \cdot \mathbf{1}[\mathrm{pattern}_i(t)]}{\theta_{\mathrm{norm}}}, 1\right)
-$$
-
-The shipped patterns include:
-
-- code fences
-- file paths
-- function definitions
-- shell commands
-- URLs or endpoints
-- stack traces
-- hashes or hex identifiers
-
-Default normalization:
-
-$$
-\theta_{\mathrm{norm}} = 1.5
-$$
+The shipped patterns include code fences, file paths, function definitions, shell commands, URLs, stack traces, and hashes.
 
-This means two strong technical signals are enough to saturate the branch weight.
-
-Saturation at `1.0` is correct because the gate does not need "how technical beyond fully technical"; it only needs the branch mixture weight.
+Default normalization is $\theta_{\mathrm{norm}} = 1.5$. This means two strong technical signals are enough to saturate the branch weight. Saturation at `1.0` is correct because the gate only needs the branch mixture weight, not a unbounded "technical magnitude."
 
 ## 4. Conversational Branch
 
 ### Novelty $H(t)$
 
-Novelty is:
+In the live implementation (`sidecar/compact/gate.go`), retrieval scores reaching the gate use the public higher-is-better cosine-style relevance contract from the retrieval layer, spanning $[-1, 1]$ for cosine collections. To ensure the novelty term remains in $[0,1]$ for the convex mixture, the mathematical model applies a zero-clamp:
 
-$$
-H(t) = 1 - \frac{1}{|K|} \sum_{k \in K} \cos(\vec{v}_t, \vec{v}_k)
-$$
+\[ H(t) = \begin{cases} 
+1.0 & \text{if } |K| = 0 \\ 
+1 - \frac{1}{|K|} \sum_{k \in K} \max(0, \cos(\vec{v}_t, \vec{v}_k)) & \text{otherwise}
+\end{cases} \]
 
 where $K$ is the retrieved nearest-neighbor set from durable `user:` memory.
 
 Properties:
-
-- empty memory gives $H=1.0$
-- highly similar existing memories drive $H$ toward `0`
-
-The implementation deliberately uses top-k mean similarity rather than centroid distance because user memory is often multimodal.
+- An empty memory (cold start) safely returns $H=1.0$ instead of a division-by-zero.
+- Highly similar existing memories ($\cos \to 1$) drive $H \to 0$.
+- Negative or orthogonal neighbors are clamped to prevent $H(t) > 1$.
 
 ### Repetition Gate $R(t)$
 
-The repetition term is:
-
-$$
-R(t) = F(t) \cdot (1 - S(t))
-$$
+The repetition term is a product, not a sum:
+\[ R(t) = F(t) \cdot (1 - S(t)) \]
 
 with:
+\[ F(t) = \min\left(\frac{\mathrm{hitsAbove}(\mathrm{turns:userId}, 0.80, k=10)}{5}, 1\right) \]
+\[ S(t) = \min\left(\frac{\mathrm{hitsAbove}(\mathrm{user:userId}, 0.85, k=5)}{3}, 1\right) \]
 
-$$
-F(t) = \min\left(\frac{\mathrm{hitsAbove}(\mathrm{turns:userId}, 0.80, k=10)}{5}, 1\right)
-$$
-
-$$
-S(t) = \min\left(\frac{\mathrm{hitsAbove}(\mathrm{user:userId}, 0.85, k=5)}{3}, 1\right)
-$$
-
-This is intentionally a product, not a sum.
-
-Why:
-
-- high input frequency should help only if durable memory is not already saturated
-- high saturation should veto the repetition term regardless of frequency
-
-The veto property is structural:
-
-$$
-S(t) = 1 \Rightarrow R(t) = 0
-$$
+Why a product? High input frequency should help only if durable memory is not already saturated. High saturation must veto the repetition term regardless of frequency. The veto property is structural: $S(t) = 1 \Rightarrow R(t) = 0$.
 
 ### Natural-Language Structural Load $D_{nl}(t)$
-
-The conversational branch adds heuristic structure for turns that look like:
-
-- preferences
-- human-name references
-- dates
-- quantities
-- fact assertions
-
-This is intentionally narrow. It excludes general proper-noun detection so technical identifiers do not inflate the conversational signal.
+Detects heuristics like preferences, human-name references, dates, and fact assertions.
 
 ## 5. Technical Branch
 
 ### Specificity $P(t)$
 
-Specificity measures concrete artifact density:
-
-$$
-P(t) = \min\left(
-\frac{
-\sum_j p_j \cdot \mathrm{count}_j(t)
-}{
-\max(\mathrm{EstimateTokens}(t)/100, 1)
-},
-1
-\right)
-$$
-
-The numerator counts things like:
-
-- file paths
-- function references
-- error codes
-- git references
-- API endpoints
-
-The normalization denominator is implemented in
-[`sidecar/compact/tokens.go`](../sidecar/compact/tokens.go):
+Specificity measures concrete artifact density normalized by turn length:
 
-$$
-L(t)=\max\left(\left\lfloor \frac{\mathrm{len}(t)}{4} \right\rfloor, 1\right)
-$$
+\[ P(t) = \min\left( \frac{\sum_j p_j \cdot \mathrm{count}_j(t)}{\max(L(t)/100.0, 1.0)}, 1 \right) \]
 
-This bytes-per-token heuristic is the token estimator used by the gating
-subsystem. It is intentionally cheap and deterministic. It is not the same as
-the separate host-side prompt-budget estimator in [`src/tokens.ts`](../src/tokens.ts).
+The numerator counts things like file paths, error codes, and API endpoints.
+The normalization denominator is the token estimator used by the gating subsystem (`sidecar/compact/tokens.go`):
+\[ L(t) = \max\left(\left\lfloor \frac{\mathrm{RuneCount}(t)}{4} \right\rfloor, 1\right) \]
 
-Length normalization matters. Without it, any long technical turn would score
-high simply because it contains more surface area, not because it is more
-memory-worthy.
+Length normalization matters. Without it, any long technical turn would score high simply because it contains more surface area, not because it is more memory-worthy.
 
 ### Actionability $A(t)$
-
-Actionability captures decision and outcome content:
-
-- architectural decisions
-- fixes or resolutions
-- deployment or merge milestones
-- configuration changes
-
-These are the kinds of technical turns that are expensive to reconstruct later and therefore worth persisting.
+Captures architectural decisions, fixes, merge milestones, and configuration changes.
 
 ### Technical Structural Load $D_{\mathrm{tech}}(t)$
-
-This branch detects structural technical content such as:
-
-- function definitions
-- data structures
-- dependencies
-- tests
-- documentation comments
-
-It is the technical analogue to $D_{nl}$, not a replacement for it.
+Detects function definitions, dependencies, and tests. It is the technical analogue to $D_{nl}$.
 
 ## 6. Calibration
 
-Stored metadata includes:
-
-- `gating_score`
-- `gating_t`
-- `gating_h`
-- `gating_r`
-- `gating_d`
-- `gating_p`
-- `gating_a`
-- `gating_dtech`
-- `gating_gconv`
-- `gating_gtech`
-
-The first calibration pass should inspect the empirical score distribution after real traffic arrives.
-
-What to look for:
-
-- bimodality in `gating_score`
-- sensible spread in `gating_t`
-- non-degenerate contributions from both `gconv` and `gtech`
-
 For threshold tuning, isotonic regression is the correct calibration method once usefulness labels exist:
-
-$$
-P(\mathrm{useful} \mid G) = \mathrm{IsotonicRegression}(G, y)
-$$
-
-It preserves the monotonic design of the gate without assuming a sigmoid link function.
+\[ P(\mathrm{useful} \mid G) = \mathrm{IsotonicRegression}(G, y) \]
 
 Current thresholds implemented in code:
-
-- durable promotion threshold:
-  [`DefaultGatingConfig().Threshold = 0.35`](../sidecar/compact/gate.go)
-- abstractive compaction routing threshold:
-  [`AbstractiveRoutingThreshold = 0.60`](../sidecar/compact/summarize.go)
+- durable promotion: `DefaultGatingConfig().Threshold = 0.35`
+- abstractive routing: `AbstractiveRoutingThreshold = 0.60`
 
 ## 7. Invariants
 
-The gate has six mathematical invariants in `gate_test.go`.
-
-### 1. Empty memory implies full novelty
-
-$$
-\mathrm{memHits} = \emptyset \Rightarrow H = 1.0
-$$
-
-This prevents a cold start from suppressing every first durable insertion.
-
-### 2. Saturation vetoes repetition
-
-$$
-\mathrm{MemSaturation} = 1 \Rightarrow R = 0
-$$
-
-This is what makes the repetition term a true gate instead of an accumulation bonus.
-
-### 3. The convex blend stays in bounds
-
-$$
-G \in [0,1]
-$$
-
-and:
-
-$$
-G \in [\min(G_{\mathrm{conv}}, G_{\mathrm{tech}}), \max(G_{\mathrm{conv}}, G_{\mathrm{tech}})]
-$$
-
-### 4. Purely conversational turns collapse to the conversational branch
-
-$$
-T = 0 \Rightarrow G = G_{\mathrm{conv}}
-$$
-
-### 5. Purely technical turns collapse to the technical branch
-
-$$
-T = 1 \Rightarrow G = G_{\mathrm{tech}}
-$$
-
-### 6. Conversational structure should not overfire on pure code
+The gate preserves six mathematical invariants mapped to `gate_test.go`:
 
-This guards against a common failure mode where technical identifiers masquerade as conversational entities.
+1. **Empty memory implies full novelty:** $\mathrm{memHits} = \emptyset \Rightarrow H = 1.0$
+2. **Saturation vetoes repetition:** $\mathrm{MemSaturation} = 1 \Rightarrow R = 0$
+3. **The convex blend stays in bounds:** $G \in [0,1]$
+4. **Monotonic Interpolation:** $G \in [\min(G_{\mathrm{conv}}, G_{\mathrm{tech}}), \max(G_{\mathrm{conv}}, G_{\mathrm{tech}})]$
+5. **Purely conversational turns collapse:** $T = 0 \Rightarrow G = G_{\mathrm{conv}}$
+6. **Purely technical turns collapse:** $T = 1 \Rightarrow G = G_{\mathrm{tech}}$
 
-Together these invariants make the scalar interpretable, stable, and safe to tune later from real traffic rather than intuition.
+Conversational structure must not overfire on pure code. Together these invariants make the scalar interpretable, stable, and safe to tune.

package/docs/install.md

@@ -0,0 +1,179 @@
+# Install Guide
+
+LibraVDB Memory is a connect-only OpenClaw plugin. Install the plugin as a
+normal package, install `libravdbd` separately, and point the plugin at the
+daemon endpoint when you need a non-default location.
+
+For deeper operational detail, use the full
+[installation reference](./installation.md).
+
+## Recommended Path: Homebrew + OpenClaw Plugin
+
+On macOS, the shortest supported path is:
+
+```bash
+brew tap xDarkicex/openclaw-libravdb-memory
+brew install libravdbd
+brew services start libravdbd
+openclaw plugins install @xdarkicex/openclaw-memory-libravdb
+```
+
+This gives you:
+
+- a managed `libravdbd` service
+- a scanner-clean plugin install
+- a clean separation between plugin lifecycle and daemon lifecycle
+
+## Plugin Install
+
+Install the plugin package with the OpenClaw CLI:
+
+```bash
+openclaw plugins install @xdarkicex/openclaw-memory-libravdb
+```
+
+If you use the OpenClaw.ai plugin UI instead of the CLI, install the same
+package and then assign the plugin id `libravdb-memory` to the `memory` slot or
+the `contextEngine` slot.
+
+Activate the plugin in `~/.openclaw/openclaw.json`:
+
+```json
+{
+  "plugins": {
+    "slots": {
+      "memory": "libravdb-memory"
+    }
+  }
+}
+```
+
+If you run the daemon on a non-default endpoint, add a plugin config:
+
+```json
+{
+  "plugins": {
+    "slots": {
+      "memory": "libravdb-memory"
+    },
+    "configs": {
+      "libravdb-memory": {
+        "sidecarPath": "unix:/Users/<you>/.clawdb/run/libravdb.sock"
+      }
+    }
+  }
+}
+```
+
+## Sidecar Daemon Install
+
+The daemon owns the local database, embeddings, and JSON-RPC endpoint.
+
+Default endpoints:
+
+- macOS/Linux: `unix:$HOME/.clawdb/run/libravdb.sock`
+- Windows: `tcp:127.0.0.1:37421`
+
+Default data path:
+
+- macOS/Linux/Windows user installs: `$HOME/.clawdb/data.libravdb`
+
+### Homebrew
+
+Homebrew is the preferred daemon lifecycle on macOS:
+
+```bash
+brew tap xDarkicex/openclaw-libravdb-memory
+brew install libravdbd
+brew services start libravdbd
+```
+
+Useful lifecycle commands:
+
+```bash
+brew services restart libravdbd
+brew services stop libravdbd
+brew info libravdbd
+```
+
+### Manual Service Management
+
+If you are not using Homebrew, manage the daemon explicitly.
+
+Linux user service from the repo template:
+
+```bash
+# Replace vX.Y.Z with the latest tag from:
+# https://github.com/xDarkicex/openclaw-memory-libravdb/releases
+mkdir -p ~/.local/bin ~/.config/systemd/user
+curl -L -o ~/.local/bin/libravdbd https://github.com/xDarkicex/openclaw-memory-libravdb/releases/download/vX.Y.Z/libravdbd-linux-amd64
+chmod +x ~/.local/bin/libravdbd
+curl -L -o ~/.config/systemd/user/libravdbd.service \
+  https://raw.githubusercontent.com/xDarkicex/openclaw-memory-libravdb/main/packaging/systemd/libravdbd.service
+systemctl --user enable --now libravdbd.service
+```
+
+macOS LaunchAgent from the repo template:
+
+1. Download `com.xdarkicex.libravdbd.plist` from:
+   `https://raw.githubusercontent.com/xDarkicex/openclaw-memory-libravdb/main/packaging/launchd/com.xdarkicex.libravdbd.plist`
+2. Replace `__HOME__` with your home directory.
+3. Save it to `~/Library/LaunchAgents/com.xdarkicex.libravdbd.plist`.
+4. Load it with `launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.xdarkicex.libravdbd.plist`.
+
+### Windows
+
+Windows uses a loopback TCP endpoint by default:
+
+- `tcp:127.0.0.1:37421`
+
+This guide does not yet include a full Windows service-install walkthrough.
+For now, use the published Windows daemon asset from the GitHub releases page
+and run it under your preferred process supervisor or a manual terminal session.
+
+Foreground manual run:
+
+```bash
+libravdbd serve
+```
+
+That mode is useful for debugging or validating a local release asset before
+you wrap it in `brew services`, `systemd`, or `launchd`.
+
+## Lifecycle Management
+
+### Plugin Lifecycle
+
+- Install the package with `openclaw plugins install`.
+- Activate it by assigning `libravdb-memory` to `memory` or `contextEngine`.
+- Update it with your normal OpenClaw plugin update flow.
+- Disable it by removing the slot assignment from `~/.openclaw/openclaw.json`.
+
+The plugin does not manage the daemon process. Treat plugin activation and
+daemon supervision as separate lifecycle decisions.
+
+### Daemon Lifecycle
+
+- Start it with `brew services`, `systemd --user`, `launchctl bootstrap`, or a manual `libravdbd serve`.
+- Restart it when you change daemon-level environment variables or replace the binary.
+- Stop it before uninstalling or deleting on-disk data.
+- Point the plugin at the correct endpoint with `sidecarPath` if you do not use the default location.
+
+## Verification
+
+After the plugin and daemon are both in place, run:
+
+```bash
+openclaw memory status
+```
+
+Healthy output should show that:
+
+- the daemon answered the local health check
+- the memory slot is active
+- the plugin can read stored counts and runtime settings
+
+If OpenClaw cannot reach the daemon, verify the endpoint first:
+
+- macOS/Linux default: `unix:$HOME/.clawdb/run/libravdb.sock`
+- Windows default: `tcp:127.0.0.1:37421`

package/docs/installation.md

@@ -117,6 +117,19 @@ extractive compaction. The only optional runtime network path is:
 
 ## Standard Install
 
+### Fastest Path on macOS
+
+```bash
+brew tap xDarkicex/openclaw-libravdb-memory
+brew install libravdbd
+brew services start libravdbd
+openclaw plugins install @xdarkicex/openclaw-memory-libravdb
+```
+
+This is the preferred install flow for macOS users. It gives you a managed `libravdbd` service and a scanner-clean OpenClaw plugin package.
+
+### Plugin Package
+
 ```bash
 openclaw plugins install @xdarkicex/openclaw-memory-libravdb
 ```
@@ -155,7 +168,15 @@ openclaw memory status
 
 ### Homebrew / macOS
 
-The release workflow now generates a publish-ready `libravdbd.rb` formula asset from [`packaging/homebrew/libravdbd.rb.tmpl`](../packaging/homebrew/libravdbd.rb.tmpl). It is designed for GitHub release assets named:
+Homebrew users should normally install from the published tap:
+
+```bash
+brew tap xDarkicex/openclaw-libravdb-memory
+brew install libravdbd
+brew services start libravdbd
+```
+
+The release workflow generates a publish-ready `libravdbd.rb` formula asset from [`packaging/homebrew/libravdbd.rb.tmpl`](../packaging/homebrew/libravdbd.rb.tmpl). It is designed for GitHub release assets named:
 
 - `libravdbd-darwin-arm64`
 - `libravdbd-darwin-amd64`
@@ -169,7 +190,7 @@ If your GitHub Actions configuration includes:
 
 then tagged releases also push the generated formula into `Formula/libravdbd.rb` in that tap repository automatically.
 
-Example:
+Example plugin config:
 
 ```json
 {
@@ -196,7 +217,7 @@ Installed plugin: libravdb-memory
 
 ## Activation
 
-The plugin declares `kind: "memory"` and is intended to occupy the `memory` slot. If your OpenClaw build also exposes legacy context-engine slotting, keep the memory slot authoritative and use the context-engine slot only for compatibility testing.
+The plugin declares `kind: ["memory", "context-engine"]` and registers for both the `memory` and `context-engine` slots. Either slot assignment activates the plugin.
 
 Add this to `~/.openclaw/openclaw.json`:
 
@@ -204,8 +225,19 @@ Add this to `~/.openclaw/openclaw.json`:
 {
   "plugins": {
     "slots": {
-      "memory": "libravdb-memory",
-      "contextEngine": "legacy"
+      "memory": "libravdb-memory"
+    }
+  }
+}
+```
+
+If your OpenClaw build uses the `contextEngine` slot instead, you can assign it there:
+
+```json
+{
+  "plugins": {
+    "slots": {
+      "contextEngine": "libravdb-memory"
     }
   }
 }
@@ -213,12 +245,10 @@ Add this to `~/.openclaw/openclaw.json`:
 
 Notes:
 
-- `memory: "libravdb-memory"` is the actual activation step.
-- `contextEngine: "legacy"` keeps the legacy engine explicit when the host still exposes that slot.
-- If you instead point `contextEngine` at another plugin, you are changing a separate slot from the memory replacement.
+- Either `memory` or `contextEngine` slot assignment activates the plugin. You do not need both.
 - The plugin id is `libravdb-memory`. The npm package name used at install time is `@xdarkicex/openclaw-memory-libravdb`.
 
-Without the `memory` slot entry, OpenClaw's default memory can continue to run in parallel.
+Without a slot entry, OpenClaw's default memory can continue to run in parallel.
 
 ## Verification
 
@@ -301,6 +331,12 @@ openclaw memory status
 
 If the daemon is down, start it and verify the configured endpoint:
 
+```bash
+brew services start libravdbd
+```
+
+Or, without Homebrew:
+
 ```bash
 libravdbd serve
 ```