@etzelm/openclaw-memory-hindsight 0.5.0 → 0.7.0

package/CHANGELOG.md

@@ -1,5 +1,67 @@
 # Changelog
 
+## 0.7.0 (2026-11-11)
+
+### Upgrade notes
+
+- No config change is needed, and no key is added or removed.
+- A 404 whose body cannot be read (request aborted, connection lost) is now an error, not "no
+  bank yet". Auto-recall logs it at warn and injects nothing, as before. The retain queue is
+  unchanged: it drops a queued retain only on an explicit `Bank '<id>' not found` body, so such a
+  retain stayed queued before and still does.
+- With `recallCacheTtlMs` set, a `hindsight_retain` call now drops that session's cached recall,
+  so the next turn of the session recalls fresh. Other sessions keep their cache.
+
+### Fixed
+
+- Retain queue: a copy that replay delivered but whose batched save failed stays in the file until
+  the next save. When a newer copy of the same document then could not be queued (too large, or
+  the file could not be written) and went out live, a restart replayed the old copy over it. The
+  queue now writes a tombstone for that copy.
+- Auto-retain: the host does not await `agent_end`, so run N's handler can run after run N+1
+  started. It used and deleted run N+1's start marker, so run N retained the wrong slice and run
+  N+1 lost its start. A marker of another run is now left alone.
+- A 404 whose body read failed counted as a missing bank, so recall reported "no memories"
+  instead of an error. The read error is now thrown. Bodies of non-404 errors are cancelled
+  instead of left unread.
+- Auto-recall cache (`recallCacheTtlMs`): a `hindsight_retain` call drops the session's cached
+  recall. Auto-retain leaves the cache alone; that turn is already in the transcript.
+- Compatibility proof (not in the package): the owner fact is a boat name instead of a "launch
+  code word", which Hindsight's retain model sometimes stored without its value. B1 and B2 judge
+  the chat.send session, so a host heartbeat turn in the same window no longer fails them, and
+  B2 still fails on a code hit from any session. The probe imports the memory host SDK while it
+  registers, because upstream main now closes a plugin's invocation scope after the run.
+- Compatibility matrix (not in the package): G2 counts with the audience checks for the
+  expected `claude-cli` failure. With `CLI_AUDIENCE_FIXED_IN` set, a host built from source must
+  pass, and `CLI_AUDIENCE_FIXED_IN=main` covers a fix that is only on upstream main.
+- Compatibility nightly (not in the package): an unwritable `WORK_ROOT` exits 4 with "cannot
+  write WORK_ROOT" instead of "another nightly run took the lock". The example plist starts bash
+  through a node binary with Full Disk Access, because a LaunchAgent bootstrapped from cron after
+  a macOS reboot gets no access to an external disk.
+
+### Added
+
+- Compatibility nightly (not in the package): `HS_COMPAT_REF=latest-release` tests the highest
+  `vX.Y.Z` tag, and `HS_COMPAT_STATUS_FILE` writes a Prometheus textfile (last attempt, last run,
+  last success, exit code) for a missed-nightly alert.
+
+## 0.6.0 (2026-11-11)
+
+### Upgrade notes
+
+- Needs an OpenClaw build with openclaw/openclaw#197274 (merged 2026-11-11) for `hindsight_*`
+  tools on claude-cli sessions. That host change passes the turn's memory audience when it lists
+  claude-cli MCP tools, so the tools list through the normal audience route. On an older
+  2026.11.x host, claude-cli sessions keep auto-recall but lose the `hindsight_*` tools; stay on
+  0.5.0 there.
+- `listToolsWithoutAudience` is deprecated and ignored. The key still validates, so an existing
+  config keeps loading; remove it. A later major release rejects it.
+
+### Removed
+
+- The `listToolsWithoutAudience` workaround from 0.5.0: a tool context without a memory audience
+  gets no `hindsight_*` tools again, whatever the config says.
+
 ## 0.5.0 (2026-11-10)
 
 ### Upgrade notes

package/README.md

@@ -129,7 +129,6 @@ tool conflict.
 | `recallCacheTtlMs`          | `0` (off)            | Auto-recall cache per session and bank, 0 to 3600000 ms; a repeated query skips the API call          |
 | `recallFollowupMaxWords`    | `0` (off)            | At most N words, no URL or path: skip after a recent recall with hits, else add the prior user turn   |
 | `knowledgeReadTools`        | `false`              | Offer the read-only knowledge-base tools (see below); turning it on needs a gateway restart           |
-| `listToolsWithoutAudience`  | `false`              | List the `hindsight_*` tools when the host builds the tool list without an audience (see below)       |
 
 Session patterns: `*` matches any run of characters except `:`, `**` matches anything including
 `:`, everything else is literal. Patterns match the trimmed session key. `ignoreSessionPatterns`
@@ -149,14 +148,10 @@ prompt, so the cached prefix stays stable). Knowledge-base tool caps: a response
 the tree shows 500 entries, then `(truncated, N more entries)`; search shows at most 50 hits
 with snippets cut at 500 characters and names and ids at 200.
 
-`listToolsWithoutAudience`: the claude-cli runtime on OpenClaw 2026.11.x builds a turn's MCP tool
-list before it attaches the turn's memory audience, so every tool factory sees no audience, returns
-null, and `hindsight_*` never reach claude-cli sessions (auto-recall still works; it reads the
-audience from the hook context). With this key on, an agent session with a session key gets the
-tools listed anyway. The host then builds the callable copy with the audience from the run's grant,
-and that copy alone reaches Hindsight; a call that still has no audience fails with an error before
-any request. The session filters (`memoryMode`, incognito, `excludeProviders`,
-`skipOperationalSessions`) apply as before, and an audience for another agent is still refused.
+`hindsight_*` tools on claude-cli sessions need an OpenClaw build that passes the turn's memory
+audience when it lists MCP tools (openclaw/openclaw#197274). On older 2026.11.x builds those
+sessions get auto-recall but no `hindsight_*` tools. `listToolsWithoutAudience` (0.5.0) is
+deprecated and ignored since 0.6.0; remove it from your config.
 
 Config changes apply without a restart: the plugin re-reads its entry from the host's current
 config snapshot (`api.runtime.config.current()`), falling back to the register-time

package/dist/client.js

@@ -120,10 +120,24 @@ export class HindsightClient {
             ...(init.signal ? { signal: init.signal } : {}),
         });
         if (!response.ok) {
-            const body = response.status === 404
-                ? // An oversized body is not "empty", so it never reads as a missing bank.
-                    await readBoundedText(response, ERROR_BODY_MAX_BYTES).catch((error) => error instanceof BodyTooLargeError ? "<oversized>" : "")
-                : "";
+            let body = "";
+            if (response.status === 404) {
+                try {
+                    body = await readBoundedText(response, ERROR_BODY_MAX_BYTES);
+                }
+                catch (error) {
+                    // An oversized body is not "empty", so it never reads as a missing bank. A read that
+                    // failed (aborted, connection lost) says nothing about the bank: throw that error.
+                    if (!(error instanceof BodyTooLargeError)) {
+                        throw error;
+                    }
+                    body = "<oversized>";
+                }
+            }
+            else {
+                // Never read, so release the connection instead of leaving the body to the GC.
+                await response.body?.cancel().catch(() => undefined);
+            }
             const bankMissing = response.status === 404 && isBankMissingBody(body);
             const bankMissingExplicit = response.status === 404 && isExplicitBankMissingBody(body);
             const pageMissing = response.status === 404 && isExplicitPageMissingBody(body);

package/dist/config.js

@@ -40,6 +40,7 @@ const CONFIG_KEYS = [
     "retainQueueMaxItems",
     "retainQueueMaxAgeMs",
     "knowledgeReadTools",
+    // Deprecated in 0.6.0 and ignored: accepted so a config that still sets it keeps validating.
     "listToolsWithoutAudience",
     "recallCacheTtlMs",
     "recallFollowupMaxWords",
@@ -227,7 +228,6 @@ export const hindsightConfigSchema = {
             retainQueueMaxItems: readPositiveInt(value, "retainQueueMaxItems", DEFAULT_RETAIN_QUEUE_MAX_ITEMS, MAX_RETAIN_QUEUE_MAX_ITEMS),
             retainQueueMaxAgeMs: readPositiveInt(value, "retainQueueMaxAgeMs", DEFAULT_RETAIN_QUEUE_MAX_AGE_MS),
             knowledgeReadTools: readBool(value, "knowledgeReadTools", false),
-            listToolsWithoutAudience: readBool(value, "listToolsWithoutAudience", false),
             recallCacheTtlMs: readNonNegativeInt(value, "recallCacheTtlMs", 0, MAX_RECALL_CACHE_TTL_MS),
             recallFollowupMaxWords: readNonNegativeInt(value, "recallFollowupMaxWords", 0, MAX_RECALL_FOLLOWUP_WORDS),
         };

package/dist/index.js

@@ -134,7 +134,13 @@ export default definePluginEntry({
                 return body ? { name, ...body } : null;
             }, { name });
         };
-        registerTool(RETAIN_TOOL, (ctx) => createRetainTool(service, flushDeduper, ctx));
+        // A hindsight_retain makes the session's cached recall out of date. Auto-retain does not: its
+        // turn is already in the transcript, and dropping the entry every turn would void the cache.
+        registerTool(RETAIN_TOOL, (ctx) => createRetainTool(service, flushDeduper, ctx, (key) => {
+            if (key) {
+                recallCache.delete(key);
+            }
+        }));
         registerTool(RECALL_TOOL, (ctx) => createRecallTool(service, ctx));
         registerTool(REFLECT_TOOL, (ctx) => createReflectTool(service, ctx));
         // Opt-in: registered only when knowledgeReadTools is on at startup, so the default tool set is
@@ -281,10 +287,18 @@ export default definePluginEntry({
                 trigger: ctx.trigger,
                 sessionKey,
             });
-            const start = sessionKey && !maintenance ? runStartBySession.get(sessionKey) : undefined;
+            const stored = sessionKey && !maintenance ? runStartBySession.get(sessionKey) : undefined;
+            // The host does not await agent_end, so run N's handler can run after run N+1's
+            // before_prompt_build replaced the marker. A marker of another run is neither used nor
+            // deleted; this run then falls back to its last user message.
+            const otherRun = stored !== undefined &&
+                runId !== undefined &&
+                stored.runId !== undefined &&
+                stored.runId !== runId;
+            const start = otherRun ? undefined : stored;
             const runStart = start?.marker;
             // Taken before any skip. The marker stays while the run can still continue; without a
-            // runId it is dropped so the next run marks its own start.
+            // runId on either side it is dropped so the next run marks its own start.
             if (sessionKey && start && (runId === undefined || start.runId !== runId)) {
                 runStartBySession.delete(sessionKey);
             }

package/dist/retain-queue.js

@@ -127,6 +127,12 @@ export class RetainQueue {
     dropWarned = false;
     /** Removals not yet written (replay batches its writes). */
     dirty = false;
+    /**
+     * Copies removed from memory but still in the file until the next successful save, by bank and
+     * document id. A newer copy that cannot be queued gets a tombstone for it, so a restart before
+     * that save cannot replay the old copy over the newer one the caller sent live.
+     */
+    unsaved = new Map();
     /** Bytes in the file as this queue last read or wrote it; appends grow it, rewrites reset it. */
     fileBytes = 0;
     limits;
@@ -158,6 +164,8 @@ export class RetainQueue {
      * skipped with one warning. Any other read error throws, so the file is never overwritten.
      */
     load() {
+        // The file is the source of truth again; any copy it still holds is loaded below.
+        this.unsaved.clear();
         let text;
         try {
             text = readFileSync(this.filePath, "utf8");
@@ -258,6 +266,7 @@ export class RetainQueue {
             queuedAt: this.now(),
         });
         const index = this.stored.findIndex((s) => s.entry.bankId === entry.bankId && s.entry.documentId === entry.documentId);
+        const unsavedCopy = index < 0 ? this.unsaved.get(queueKey(entry)) : undefined;
         if (next.size > this.maxBytes()) {
             this.logger.warn?.(`memory-hindsight: retain too large to queue (${next.size} bytes, bound ${this.maxBytes()}); not queued`);
             // An older copy of the same document is out of date now; do not replay it.
@@ -270,6 +279,9 @@ export class RetainQueue {
                     this.markStale(stale.entry);
                 }
             }
+            else if (unsavedCopy) {
+                this.markStale(unsavedCopy);
+            }
             return false;
         }
         // A failed write never leaves memory holding an item the file lacks; see the catch below.
@@ -338,6 +350,9 @@ export class RetainQueue {
         if (index >= 0) {
             this.markStale(before[index].entry);
         }
+        else if (unsavedCopy) {
+            this.markStale(unsavedCopy);
+        }
         throw rewriteError;
     }
     /**
@@ -353,6 +368,7 @@ export class RetainQueue {
         this.stored.splice(index, 1);
         if (options.persist === false) {
             this.dirty = true;
+            this.unsaved.set(queueKey(entry), entry);
         }
         else {
             this.persist();
@@ -439,6 +455,7 @@ export class RetainQueue {
         this.rewrite();
         // Cleared only once written, so a failed replay save is retried by the next save().
         this.dirty = false;
+        this.unsaved.clear();
     }
     rewrite() {
         if (this.stored.length === 0) {

package/dist/tools.js

@@ -24,19 +24,6 @@ function requiredString(params, name) {
     }
     return value.trim();
 }
-export const NO_AUDIENCE_AT_CALL = "This session has no host memory audience for this call, so Hindsight memory is unavailable.";
-/**
- * A tool listed for a session whose tool list the host built without a memory audience. Reading
- * `bankId` throws, so `execute` fails before any request; the host builds the callable tool again
- * with the session's audience, and that copy gets a real bank.
- */
-function deferredBank() {
-    return {
-        get bankId() {
-            throw new Error(NO_AUDIENCE_AT_CALL);
-        },
-    };
-}
 /**
  * Resolves the bank for one tool context, or null when the session must not get the tool.
  * The bank comes only from the host-minted audience; the model never names a bank.
@@ -52,15 +39,6 @@ function resolveToolBank(service, ctx, purpose) {
         })) {
         return null;
     }
-    // Some hosts list a session's tools before they attach its audience (claude-cli MCP tool
-    // projection on OpenClaw 2026.11.x). Opt-in: list the tool for an agent session with a key, and
-    // let only the audience-bearing copy built for the call reach Hindsight.
-    if (config.listToolsWithoutAudience &&
-        !ctx.memoryAudience &&
-        ctx.agentId?.trim() &&
-        ctx.sessionKey?.trim()) {
-        return deferredBank();
-    }
     const route = routeAudience(ctx.agentId, ctx.memoryAudience, config.agentBankMap);
     return route.ok ? { bankId: route.bankId } : null;
 }
@@ -94,7 +72,9 @@ export function flushDocumentId(flushId, content) {
     const digest = createHash("sha256").update(`${flushId}\u0000${content}`).digest("hex");
     return `openclaw-flush:${digest.slice(0, 32)}`;
 }
-export function createRetainTool(service, deduper, ctx) {
+export function createRetainTool(service, deduper, ctx, 
+/** Called after a retain Hindsight accepted, so the session's cached recall is dropped. */
+onRetained) {
     const bank = resolveToolBank(service, ctx, "retain");
     if (!bank) {
         return null;
@@ -148,6 +128,7 @@ export function createRetainTool(service, deduper, ctx) {
                         ...(flushId ? { retention_scope: "pre_compaction_flush" } : {}),
                     },
                 }, { async: true, signal: bounded });
+                onRetained?.(ctx.sessionKey);
             }
             catch (error) {
                 if (flushId) {

package/openclaw.plugin.json

@@ -96,11 +96,6 @@
       "label": "Knowledge Base Read Tools",
       "help": "Offer hindsight_get_knowledge_base_tree, hindsight_search_knowledge_base and hindsight_get_knowledge_page on the same bank as hindsight_recall",
       "advanced": true
-    },
-    "listToolsWithoutAudience": {
-      "label": "List Tools Without Audience",
-      "help": "List the hindsight_* tools when the host builds a session's tool list before it attaches the memory audience (claude-cli sessions on OpenClaw 2026.11.x). Calls still need the audience",
-      "advanced": true
     }
   },
   "configSchema": {
@@ -223,7 +218,7 @@
       "listToolsWithoutAudience": {
         "type": "boolean",
         "default": false,
-        "description": "List the hindsight_* tools for an agent session whose tool list the host builds without its memory audience (claude-cli MCP tool projection on OpenClaw 2026.11.x). The bank still comes only from the audience: a call without one fails before any request"
+        "description": "Deprecated since 0.6.0 and ignored. The host now passes the memory audience when it lists claude-cli tools (openclaw/openclaw#197274). Remove this key; a later major release rejects it"
       },
       "agentBankMap": {
         "type": "object",

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@etzelm/openclaw-memory-hindsight",
-  "version": "0.5.0",
+  "version": "0.7.0",
   "description": "Native Hindsight memory engine for OpenClaw: memory slot provider runtime, auto-recall, auto-retain, pre-compaction flush and hindsight_* agent tools.",
   "keywords": [
     "hindsight",
@@ -37,7 +37,7 @@
     "typecheck": "tsc -p tsconfig.json --noEmit",
     "test": "vitest run",
     "test:coverage": "vitest run --coverage",
-    "lint": "oxlint src",
+    "lint": "oxlint --deny-warnings src",
     "format": "oxfmt --write .",
     "format:check": "oxfmt --check .",
     "prepack": "npm run build"