@nowledge/openclaw-nowledge-mem 0.6.15 → 0.7.1

package/CHANGELOG.md

@@ -2,6 +2,38 @@
 
 All notable changes to the Nowledge Mem OpenClaw plugin will be documented in this file.
 
+## [Unreleased]
+
+### Fixed
+
+- **Thread sync no longer depends on argv-sized CLI payloads.** OpenClaw conversation capture now creates and appends threads through the Mem HTTP API instead of passing whole message arrays through `nmem ... -m '<json>'`. This removes the transport limit that caused repeated append failures on long or repetitive sessions.
+- **Session capture now syncs only the unsynced tail.** The plugin preserves the real transcript, asks Mem how many messages are already stored, and appends only the new tail instead of replaying the whole session on every hook or Context Engine turn.
+- **Remote config is still unified after the transport change.** The same resolved `apiUrl` and `apiKey` from OpenClaw settings / `~/.nowledge-mem/config.json` now drive both CLI-backed memory tools and API-backed thread sync.
+- **Removed lossy repetitive-session collapse.** The temporary content-based dedup workaround for cron-style sessions has been removed so conversation structure is preserved faithfully.
+
+## [0.7.1] - 2026-03-23
+
+### Fixed
+
+- **Heartbeat sessions no longer trigger thread capture.** Sessions with `ctx.trigger === "heartbeat"` are now skipped in hook handlers. For cron-triggered heartbeat sessions (which use `trigger: "cron"`), a content-based dedup detects repetitive patterns: when >50% of messages in a session are duplicates, only unique messages are kept. This collapses 20 repetitive heartbeat messages down to 2, eliminating the CLI timeout caused by oversized payloads.
+
+## [0.7.0] - 2026-03-23
+
+### Added
+
+- **Context Engine support.** The plugin now registers a full OpenClaw Context Engine alongside its memory slot. When you set `plugins.slots.contextEngine: "nowledge-mem"` in your OpenClaw config, the engine takes over context assembly, capturing, and compaction — replacing the hook-based approach with richer lifecycle integration:
+  - **`assemble()`** — behavioral guidance and recalled memories injected via `systemPromptAddition` (system-prompt space, cache-friendly). Replaces the behavioral and recall hooks.
+  - **`afterTurn()`** — continuous thread capture and triage/distillation after every turn, not just session end. More granular than the `agent_end` hook.
+  - **`compact()`** — memory-aware compaction. When compacting old messages, the compactor is told which key decisions and learnings are already saved in your knowledge graph, so it can reference them concisely rather than losing them in summarization.
+  - **`prepareSubagentSpawn()`** — when OpenClaw spawns parallel research agents, child sessions inherit your Working Memory and recently recalled memories automatically.
+  - **`bootstrap()`** — pre-warms Working Memory on session start for instant first-turn context.
+  - **`dispose()`** — clean session teardown.
+- **Backward compatible.** When the CE slot is not activated, hooks continue working exactly as before. No config changes required for existing users.
+
+### Fixed
+
+- **Recalled memories no longer hurt prompt cache.** The recall hook now injects context via `appendSystemContext` (system-prompt space) instead of `prependContext` (user-message space). This preserves OpenClaw's prompt cache across turns. The fix applies to both the hook path and the new CE path.
+
 ## [0.6.15] - 2026-03-18
 
 ### Changed
@@ -62,7 +94,7 @@ All notable changes to the Nowledge Mem OpenClaw plugin will be documented in th
 
 ### Added
 
-- **`recallMinScore` config option** (0-100, default 0): Minimum relevance score threshold for auto-recalled memories. Set to e.g. 30 to filter out low-confidence results. Configurable via OpenClaw Config UI, config file, or `NMEM_RECALL_MIN_SCORE` env var.
+- `recallMinScore` config option (0-100, default 0): Minimum relevance score threshold for auto-recalled memories. Set to e.g. 30 to filter out low-confidence results. Configurable via OpenClaw Config UI, config file, or `NMEM_RECALL_MIN_SCORE` env var.
 
 ## [0.6.8] - 2026-02-27
 

package/CLAUDE.md

@@ -4,11 +4,12 @@ Continuation guide for `community/nowledge-mem-openclaw-plugin`.
 
 ## Scope
 
-- Plugin target: OpenClaw plugin runtime (memory slot provider)
+- Plugin target: OpenClaw plugin runtime (memory slot + context engine)
 - Runtime: JS ESM modules under `src/`, no TS build pipeline
 - Memory backend: `nmem` CLI (fallback: `uvx --from nmem-cli nmem`)
 - OpenClaw minimum: `2026.3.7` (`appendSystemContext` / system-context guidance required)
 - Architecture: **CLI-first via OpenClaw runtime** - all CLI execution goes through `api.runtime.system.runCommandWithTimeout`, not direct `child_process`
+- Context engine: registered via `api.registerContextEngine("nowledge-mem", factory)`. Activated when user sets `plugins.slots.contextEngine: "nowledge-mem"`. Falls back to hooks when CE is not active.
 - Remote mode: `~/.nowledge-mem/config.json` (shared) or OpenClaw dashboard. Legacy `openclaw.json` still honored.
 
 ## Design Philosophy
@@ -27,14 +28,16 @@ Reflects Nowledge Mem's genuine v0.6 strengths:
 
 ```
 src/
-  index.js          - plugin registration (tools, hooks, commands, CLI)
+  index.js          - plugin registration (tools, hooks, CE, commands, CLI)
+  context-engine.js - Context Engine factory: assemble, afterTurn, compact, subagent hooks
+  ce-state.js       - shared { active } flag for CE/hook coordination
   client.js         - CLI wrapper with API fallback; async runtime command execution; credential handling
   spawn-env.js      - env-only credential injection for the nmem runner
   config.js         - config cascade: openclaw.json (legacy) > pluginConfig > config.json (credentials) > env > defaults
   hooks/
-    behavioral.js   - always-on behavioral guidance (~50 tokens/turn)
-    recall.js       - before_prompt_build: inject Working Memory + recalled memories
-    capture.js      - thread capture + LLM triage/distillation at session lifecycle events
+    behavioral.js   - always-on behavioral guidance (~50 tokens/turn); no-ops when CE active
+    recall.js       - before_prompt_build: inject Working Memory + recalled memories; no-ops when CE active
+    capture.js      - thread capture + LLM triage/distillation; shared functions used by both hooks and CE
   tools/
     memory-search.js    - OpenClaw compat; multi-signal; bi-temporal; relevance_reason; sourceThreadId
     memory-get.js       - OpenClaw compat; supports MEMORY.md alias; sourceThreadId
@@ -53,6 +56,50 @@ openclaw.plugin.json - manifest + config schema (version, uiHints, configSchema,
 ~/.nowledge-mem/config.json   - shared credentials (apiUrl/apiKey) read by all Nowledge Mem tools
 ```
 
+## Context Engine (CE) Architecture
+
+The plugin registers both a **memory slot** (`kind: "memory"`) and a **context engine** (`api.registerContextEngine`). These are independent registrations:
+
+- **Memory slot**: provides `memory_search` + `memory_get`, activates OpenClaw's "Memory Recall" system prompt section. Always active.
+- **Context engine**: activated when user sets `plugins.slots.contextEngine: "nowledge-mem"`. Replaces hooks with richer CE lifecycle.
+
+### CE vs Hooks (dual-path design)
+
+A shared `ceState.active` flag (in `ce-state.js`) coordinates the two paths:
+
+| Lifecycle | CE active | CE inactive (hooks) |
+|-----------|-----------|---------------------|
+| Behavioral guidance | `assemble()` → `systemPromptAddition` | `before_prompt_build` → `appendSystemContext` |
+| Memory recall | `assemble()` → `systemPromptAddition` | `before_prompt_build` → `appendSystemContext` |
+| Thread capture | `afterTurn()` (every turn) | `agent_end` / `after_compaction` / `before_reset` |
+| Triage + distill | `afterTurn()` | `agent_end` only |
+| Compaction | `compact()` with memory-aware instructions | None (OpenClaw legacy) |
+| Subagent context | `prepareSubagentSpawn()` + `onSubagentEnded()` | None |
+| Session init | `bootstrap()` pre-warms WM | None |
+
+### Key design decisions
+
+- **`ownsCompaction: false`**: we enhance compaction instructions with memory context, but delegate the actual compaction to OpenClaw's runtime via `delegateCompactionToRuntime()`.
+- **Messages pass through unchanged**: `assemble()` returns the same messages it receives. We only add `systemPromptAddition`. We never own message selection.
+- **Per-session state**: `_sessions` map (bounded at 100 entries) caches Working Memory and recalled memories per session. `_childContext` map caches subagent context.
+- **Cache-friendly injection**: both CE (`systemPromptAddition`) and hooks (`appendSystemContext`) inject into system-prompt space. Never use `prependContext` (user-message space, breaks cache).
+
+### Activation
+
+```json
+// openclaw.json
+{
+  "plugins": {
+    "slots": {
+      "memory": "openclaw-nowledge-mem",
+      "contextEngine": "nowledge-mem"
+    }
+  }
+}
+```
+
+When `contextEngine` points elsewhere (or is absent), hooks handle everything. No config change needed for existing users.
+
 ## Tool Surface (10 tools)
 
 ### OpenClaw Memory Slot (required for system prompt activation)
@@ -178,6 +225,13 @@ After bumping, commit inside the `community/` submodule, then stage the updated
 4. **`unit_type` requires rebuilt backend** - `MemoryCreateRequest` includes `unit_type` (fixed). Restart backend after rebuild.
 5. **Working Memory full-overwrite only via API** - the API (`PUT /agent/working-memory`) still takes full content. The section-level patch is implemented purely client-side. This is acceptable; the Knowledge Agent regenerates WM each morning anyway.
 
+## Cache Safety Rules
+
+- **Hooks**: always return `{ appendSystemContext }` — never `{ prependContext }`. `prependContext` injects into user-message space and breaks Anthropic's system prompt cache prefix on every turn.
+- **CE assemble()**: return `systemPromptAddition` — same cache-safe position as `appendSystemContext`.
+- **Never** embed dynamic content (timestamps, per-turn IDs) in system-prompt-level injection. Static behavioral guidance is fine; recalled memories are fine (they append after the cached prefix).
+
+
 ## Non-Goals
 
 - Do NOT add `nowledge_mem_search` - `memory_search` covers it.

package/README.md

@@ -70,7 +70,7 @@ Connect to a Nowledge Mem server running elsewhere (a VPS, a home server, or a s
 
 Enable the plugin the same way as local mode, then set `apiUrl` and `apiKey` in the OpenClaw plugin settings.
 
-Or use a config file at `~/.nowledge-mem/openclaw.json`:
+Or use the shared config file at `~/.nowledge-mem/config.json` so OpenClaw, `nmem`, Bub, Claude Code, and other integrations all point at the same remote Mem:
 
 ```json
 {
@@ -79,7 +79,9 @@ Or use a config file at `~/.nowledge-mem/openclaw.json`:
 }
 ```
 
-The `apiKey` is injected as `NMEM_API_KEY` into the nmem CLI process. Never passed as a CLI argument, never logged.
+Legacy `~/.nowledge-mem/openclaw.json` is still honored first for backward compatibility.
+
+The resolved `apiUrl` and `apiKey` are reused across the plugin: CLI-backed memory tools and API-backed thread sync both talk to the same backend. The `apiKey` is never passed as a CLI argument and is never logged.
 
 ### Configure via WebUI of OpenClaw
 
@@ -154,6 +156,7 @@ flowchart TD
 
 **Key points:**
 - Thread capture is unconditional: every conversation is saved and searchable via `nowledge_mem_thread_search`
+- Thread sync is incremental: the plugin preserves the real transcript but appends only the unsynced tail instead of replaying the whole session
 - LLM distillation only runs at `agent_end`, not during compaction/reset checkpoints
 - Distilled memories carry `sourceThreadId`, linking them back to the source conversation
 - Cooldown (`digestMinInterval`, default 300s) prevents burst distillation

package/openclaw.plugin.json

@@ -1,6 +1,6 @@
 {
 	"id": "openclaw-nowledge-mem",
-	"version": "0.6.15",
+	"version": "0.7.1",
 	"kind": "memory",
 	"skills": ["skills/memory-guide"],
 	"uiHints": {

package/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "@nowledge/openclaw-nowledge-mem",
-	"version": "0.6.15",
+	"version": "0.7.1",
 	"type": "module",
 	"description": "Nowledge Mem memory plugin for OpenClaw, local-first personal knowledge base",
 	"author": {

package/src/ce-state.js

@@ -0,0 +1,8 @@
+/**
+ * Shared mutable state flag for context engine activation.
+ *
+ * When the CE is bootstrapped, `active` is set to true. Event hooks
+ * check this flag to avoid duplicate work — the CE's lifecycle methods
+ * (assemble, afterTurn, etc.) replace hook behavior when active.
+ */
+export const ceState = { active: false };

package/src/client.js

@@ -54,15 +54,11 @@ function patchWmSection(currentContent, heading, { content, append } = {}) {
 }
 
 /**
- * Nowledge Mem client. Wraps the nmem CLI for local-first and remote operations.
+ * Nowledge Mem client. Wraps the nmem CLI plus direct API calls.
  *
- * All operations go through the CLI first. This means:
- * - Local mode: CLI uses http://127.0.0.1:14242 automatically
- * - Remote mode: configure via apiUrl + apiKey (plugin config or env vars)
- *   (see: https://docs.nowledge.co/docs/remote-access)
- *
- * Falls back to direct API calls when a CLI command is too new for the installed
- * version. The fallback path uses the same apiUrl / apiKey.
+ * Most interactive reads/writes stay CLI-first for local-first ergonomics.
+ * Thread create/append are API-first because large message batches should
+ * travel in an HTTP body, not as a single argv-sized JSON argument.
  *
  * Credential rules:
  * - apiUrl: passed to CLI via --api-url flag (not a secret)
@@ -82,8 +78,9 @@ export class NowledgeMemClient {
 		this.nmemCmd = null;
 		this.nmemCmdPromise = null;
 		// Resolved once from config + env (config wins over env, both win over default)
-		this._apiUrl =
-			((credentials.apiUrl || "").trim() || "http://127.0.0.1:14242").replace(/\/+$/, "");
+		this._apiUrl = (
+			(credentials.apiUrl || "").trim() || "http://127.0.0.1:14242"
+		).replace(/\/+$/, "");
 		this._apiKey = (credentials.apiKey || "").trim();
 	}
 
@@ -558,42 +555,12 @@ export class NowledgeMemClient {
 			throw new Error("createThread requires at least one message");
 		}
 
-		let data;
-		try {
-			const args = [
-				"--json",
-				"t",
-				"create",
-				"-t",
-				normalizedTitle,
-				"-m",
-				JSON.stringify(messages),
-				"-s",
-				String(source),
-			];
-			if (threadId) {
-				args.push("--id", String(threadId));
-			}
-			data = await this.execJson(args);
-		} catch (err) {
-			const message = err instanceof Error ? err.message : String(err);
-			const needsApiFallback =
-				Boolean(threadId) &&
-				(message.includes("unrecognized arguments: --id") ||
-					message.includes("invalid choice"));
-			if (!needsApiFallback) {
-				throw err;
-			}
-			this.logger.warn(
-				"createThread: CLI missing --id support, falling back to API",
-			);
-			data = await this.apiJson("POST", "/threads", {
-				thread_id: String(threadId),
-				title: normalizedTitle,
-				source: String(source),
-				messages,
-			});
-		}
+		const data = await this.apiJson("POST", "/threads", {
+			...(threadId ? { thread_id: String(threadId) } : {}),
+			title: normalizedTitle,
+			source: String(source),
+			messages,
+		});
 
 		return String(
 			data.id ?? data.thread?.thread_id ?? data.thread_id ?? "created",
@@ -614,50 +581,46 @@ export class NowledgeMemClient {
 			return { messagesAdded: 0, totalMessages: 0 };
 		}
 
+		const data = await this.apiJson(
+			"POST",
+			`/threads/${encodeURIComponent(normalizedThreadId)}/append`,
+			{
+				messages,
+				deduplicate,
+				...(idempotencyKey
+					? { idempotency_key: String(idempotencyKey) }
+					: {}),
+			},
+		);
+		return {
+			messagesAdded: Number(data.messages_added ?? 0),
+			totalMessages: Number(data.total_messages ?? 0),
+		};
+	}
+
+	async getThreadMessageCount(threadId) {
+		const normalizedThreadId = String(threadId || "").trim();
+		if (!normalizedThreadId) {
+			throw new Error("getThreadMessageCount requires threadId");
+		}
+
 		try {
-			const args = [
-				"--json",
-				"t",
-				"append",
-				normalizedThreadId,
-				"-m",
-				JSON.stringify(messages),
-				...(deduplicate ? [] : ["--no-deduplicate"]),
-			];
-			if (idempotencyKey) {
-				args.push("--idempotency-key", String(idempotencyKey));
-			}
-			const data = await this.execJson(args);
-			return {
-				messagesAdded: Number(data.messages_added ?? 0),
-				totalMessages: Number(data.total_messages ?? 0),
-			};
-		} catch (err) {
-			const message = err instanceof Error ? err.message : String(err);
-			const needsApiFallback =
-				message.includes("invalid choice") ||
-				message.includes("unrecognized arguments");
-			if (!needsApiFallback) {
-				throw err;
-			}
-			this.logger.warn(
-				"appendThread: CLI missing append support, falling back to API",
-			);
 			const data = await this.apiJson(
-				"POST",
-				`/threads/${encodeURIComponent(normalizedThreadId)}/append`,
-				{
-					messages,
-					deduplicate,
-					...(idempotencyKey
-						? { idempotency_key: String(idempotencyKey) }
-						: {}),
-				},
+				"GET",
+				`/threads/${encodeURIComponent(normalizedThreadId)}?limit=1`,
+				undefined,
+				15_000,
 			);
-			return {
-				messagesAdded: Number(data.messages_added ?? 0),
-				totalMessages: Number(data.total_messages ?? 0),
-			};
+			return Number(
+				data.message_count ??
+					data.total_messages ??
+					(Array.isArray(data.messages) ? data.messages.length : 0),
+			);
+		} catch (err) {
+			if (this.isThreadNotFoundError(err)) {
+				return null;
+			}
+			throw err;
 		}
 	}