memory-lancedb-pro 1.0.7 → 1.0.8

package/CHANGELOG.md

@@ -1,5 +1,10 @@
 # Changelog
 
+## 1.0.8
+
+- Add: JSONL distill extractor supports optional agent allowlist via env var `OPENCLAW_JSONL_DISTILL_ALLOWED_AGENT_IDS` (default off / compatible).
+
+
 ## 1.0.7
 
 - Fix: resolve `agentId` from hook context (`ctx?.agentId`) for `before_agent_start` and `agent_end`, restoring per-agent scope isolation when using multi-agent setups.

package/README.md

@@ -161,6 +161,32 @@ Filters out low-quality content at both auto-capture and tool-store stages:
 
 ## Installation
 
+### AI-safe install notes (anti-hallucination)
+
+If you are following this README using an AI assistant, **do not assume defaults**. Always run these commands first and use the real output:
+
+```bash
+openclaw config get agents.defaults.workspace
+openclaw config get plugins.load.paths
+openclaw config get plugins.slots.memory
+openclaw config get plugins.entries.memory-lancedb-pro
+```
+
+Recommendations:
+- Prefer **absolute paths** in `plugins.load.paths` unless you have confirmed the active workspace.
+- If you use `${JINA_API_KEY}` (or any `${...}` variable) in config, ensure the **Gateway service process** has that environment variable (system services often do **not** inherit your interactive shell env).
+- After changing plugin config, run `openclaw gateway restart`.
+
+### Jina API keys (embedding + rerank)
+
+- **Embedding**: set `embedding.apiKey` to your Jina key (recommended: use an env var like `${JINA_API_KEY}`).
+- **Rerank** (when `retrieval.rerankProvider: "jina"`): you can typically use the **same** Jina key for `retrieval.rerankApiKey`.
+- If you use a different rerank provider (`siliconflow`, `pinecone`, etc.), `retrieval.rerankApiKey` should be that provider’s key.
+
+Key storage guidance:
+- Avoid committing secrets into git.
+- Using `${...}` env vars is fine, but make sure the **Gateway service process** has those env vars (system services often do not inherit your interactive shell environment).
+
 ### What is the “OpenClaw workspace”?
 
 In OpenClaw, the **agent workspace** is the agent’s working directory (default: `~/.openclaw/workspace`).
@@ -168,7 +194,9 @@ According to the docs, the workspace is the **default cwd**, and **relative path
 
 > Note: OpenClaw configuration typically lives under `~/.openclaw/openclaw.json` (separate from the workspace).
 
-**Common mistake:** cloning the plugin somewhere else, while keeping `plugins.load.paths: ["plugins/memory-lancedb-pro"]` (a **relative path**). In that case OpenClaw will look for `plugins/memory-lancedb-pro` under your **workspace** and fail to load it.
+**Common mistake:** cloning the plugin somewhere else, while keeping a **relative path** like `plugins.load.paths: ["plugins/memory-lancedb-pro"]`. Relative paths can be resolved against different working directories depending on how the Gateway is started.
+
+To avoid ambiguity, use an **absolute path** (Option B) or clone into `<workspace>/plugins/` (Option A) and keep your config consistent.
 
 ### Option A (recommended): clone into `plugins/` under your workspace
 
@@ -285,7 +313,7 @@ openclaw config get plugins.slots.memory
     "bm25Weight": 0.3,
     "minScore": 0.3,
     "rerank": "cross-encoder",
-    "rerankApiKey": "jina_xxx",
+    "rerankApiKey": "${JINA_API_KEY}",
     "rerankModel": "jina-reranker-v2-base-multilingual",
     "rerankEndpoint": "https://api.jina.ai/v1/rerank",
     "rerankProvider": "jina",
@@ -414,6 +442,19 @@ The script is **safe**: it never modifies session logs.
 
 By default it skips historical reset snapshots (`*.reset.*`) and excludes the distiller agent itself (`memory-distiller`) to prevent self-ingestion loops.
 
+### Optional: restrict distillation sources (allowlist)
+
+By default, the extractor scans **all agents** (except `memory-distiller`).
+
+If you want higher signal (e.g., only distill from your main assistant + coding bot), set:
+
+```bash
+export OPENCLAW_JSONL_DISTILL_ALLOWED_AGENT_IDS="main,code-agent"
+```
+
+- Unset / empty / `*` / `all` → allow all agents (default)
+- Comma-separated list → only those agents are scanned
+
 ### Recommended setup (dedicated distiller agent)
 
 #### 1) Create a dedicated agent

package/README_CN.md

@@ -162,6 +162,32 @@ Query → BM25 FTS ─────┘
 
 ## 安装
 
+### AI 安装指引（防幻觉版）
+
+如果你是用 AI 按 README 操作，**不要假设任何默认值**。请先运行以下命令，并以真实输出为准：
+
+```bash
+openclaw config get agents.defaults.workspace
+openclaw config get plugins.load.paths
+openclaw config get plugins.slots.memory
+openclaw config get plugins.entries.memory-lancedb-pro
+```
+
+建议：
+- `plugins.load.paths` 建议优先用**绝对路径**（除非你已确认当前 workspace）。
+- 如果配置里使用 `${JINA_API_KEY}`（或任何 `${...}` 变量），务必确保运行 Gateway 的**服务进程环境**里真的有这些变量（systemd/launchd/docker 通常不会继承你终端的 export）。
+- 修改插件配置后，运行 `openclaw gateway restart` 使其生效。
+
+### Jina API Key（Embedding + Rerank）如何填写
+
+- **Embedding**：将 `embedding.apiKey` 设置为你的 Jina key（推荐用环境变量 `${JINA_API_KEY}`）。
+- **Rerank**（当 `retrieval.rerankProvider: "jina"`）：通常可以直接复用同一个 Jina key，填到 `retrieval.rerankApiKey`。
+- 如果你选择了其它 rerank provider（如 `siliconflow` / `pinecone`），则 `retrieval.rerankApiKey` 应填写对应提供商的 key。
+
+Key 存储建议：
+- 不要把 key 提交到 git。
+- 使用 `${...}` 环境变量没问题，但务必确保运行 Gateway 的**服务进程环境**里真的有该变量（systemd/launchd/docker 往往不会继承你终端的 export）。
+
 ### 什么是 “OpenClaw workspace”？
 
 在 OpenClaw 中，**agent workspace（工作区）** 是 Agent 的工作目录（默认：`~/.openclaw/workspace`）。
@@ -169,7 +195,9 @@ Query → BM25 FTS ─────┘
 
 > 说明：OpenClaw 的配置文件通常在 `~/.openclaw/openclaw.json`，与 workspace 是分开的。
 
-**最常见的安装错误：** 把插件 clone 到别的目录，但在配置里仍然写 `"paths": ["plugins/memory-lancedb-pro"]`（这是**相对路径**）。OpenClaw 会去 workspace 下找 `plugins/memory-lancedb-pro`，导致加载失败，于是出现“安装位置不对”的反馈。
+**最常见的安装错误：** 把插件 clone 到别的目录，但在配置里仍然写类似 `"paths": ["plugins/memory-lancedb-pro"]` 的**相对路径**。相对路径的解析基准会受 Gateway 启动方式/工作目录影响，容易指向错误位置。
+
+为避免歧义：建议用**绝对路径**（方案 B），或把插件放在 `<workspace>/plugins/`（方案 A）并保持配置一致。
 
 ### 方案 A（推荐）：克隆到 workspace 的 `plugins/` 目录下
 
@@ -286,7 +314,7 @@ openclaw config get plugins.slots.memory
     "bm25Weight": 0.3,
     "minScore": 0.3,
     "rerank": "cross-encoder",
-    "rerankApiKey": "jina_xxx",
+    "rerankApiKey": "${JINA_API_KEY}",
     "rerankModel": "jina-reranker-v2-base-multilingual",
     "candidatePoolSize": 20,
     "recencyHalfLifeDays": 14,
@@ -357,6 +385,19 @@ OpenClaw 会把每个 Agent 的完整会话自动落盘为 JSONL：
 
 > 脚本只读 session JSONL，不会修改原始日志。
 
+### （可选）启用 Agent 来源白名单（提高信噪比）
+
+默认情况下，extractor 会扫描 **所有 Agent**（但会排除 `memory-distiller` 自身，防止自我吞噬）。
+
+如果你只想从某些 Agent 蒸馏（例如只蒸馏 `main` + `code-agent`），可以设置环境变量：
+
+```bash
+export OPENCLAW_JSONL_DISTILL_ALLOWED_AGENT_IDS="main,code-agent"
+```
+
+- 不设置 / 空 / `*` / `all`：扫描全部（默认）
+- 逗号分隔列表：只扫描列表内 agentId
+
 ### 推荐部署（独立 distiller agent）
 
 #### 1）创建 distiller agent（示例用 gpt-5.2）

package/openclaw.plugin.json

@@ -2,7 +2,7 @@
   "id": "memory-lancedb-pro",
   "name": "Memory (LanceDB Pro)",
   "description": "Enhanced LanceDB-backed long-term memory with hybrid retrieval, multi-scope isolation, and management CLI",
-  "version": "1.0.7",
+  "version": "1.0.8",
   "kind": "memory",
   "configSchema": {
     "type": "object",
@@ -118,7 +118,11 @@
           },
           "rerankProvider": {
             "type": "string",
-            "enum": ["jina", "siliconflow", "pinecone"],
+            "enum": [
+              "jina",
+              "siliconflow",
+              "pinecone"
+            ],
             "default": "jina",
             "description": "Reranker provider format. Determines request/response shape and auth header."
           },

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "memory-lancedb-pro",
-  "version": "1.0.7",
+  "version": "1.0.8",
   "description": "OpenClaw enhanced LanceDB memory plugin with hybrid retrieval (Vector + BM25), cross-encoder rerank, multi-scope isolation, and management CLI",
   "type": "module",
   "main": "index.ts",

package/scripts/jsonl_distill.py

@@ -36,6 +36,22 @@ EXCLUDED_AGENT_IDS = {
     "memory-distiller",
 }
 
+# Source allowlist (optional quality control).
+# Default (env unset): allow all agents (except EXCLUDED_AGENT_IDS).
+# If set: only distill from the listed agent IDs.
+# Example:
+#   OPENCLAW_JSONL_DISTILL_ALLOWED_AGENT_IDS=main,code-agent
+ENV_ALLOWED_AGENT_IDS = "OPENCLAW_JSONL_DISTILL_ALLOWED_AGENT_IDS"
+
+
+def _get_allowed_agent_ids() -> Optional[set[str]]:
+    raw = os.environ.get(ENV_ALLOWED_AGENT_IDS, "").strip()
+    if not raw or raw in ("*", "all"):
+        return None
+    parts = [p.strip() for p in raw.split(",") if p.strip()]
+    return set(parts) if parts else None
+
+
 
 NOISE_PREFIXES = (
     "✅ New session started",
@@ -175,12 +191,16 @@ def _list_session_files(agents_dir: Path) -> List[Tuple[str, Path]]:
     if not agents_dir.exists():
         return results
 
+    allowed_agent_ids = _get_allowed_agent_ids()
+
     for agent_dir in sorted(agents_dir.iterdir()):
         if not agent_dir.is_dir():
             continue
         agent_id = agent_dir.name
         if agent_id in EXCLUDED_AGENT_IDS:
             continue
+        if allowed_agent_ids is not None and agent_id not in allowed_agent_ids:
+            continue
         sessions_dir = agent_dir / "sessions"
         if not sessions_dir.exists():
             continue