@totalreclaw/totalreclaw 3.3.11-rc.3 → 3.3.11-rc.5

package/CHANGELOG.md

@@ -4,6 +4,55 @@ All notable changes to `@totalreclaw/totalreclaw` (the OpenClaw plugin) are docu
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [3.3.11-rc.5] — 2026-05-07
+
+Trajectory poller hardening: cap extractions per poll iteration + skip stale trajectory files. Pedro's 2026-05-07 zai 429 cascade was caused by ~5 old session files all crossing the extract threshold in the same poll → 5 back-to-back LLM calls in seconds → daily quota tripped. Today's chat memories were lost because every extraction call returned 0 facts (LLM rejected with rate-limit).
+
+### Changed
+
+- **Cap = 1 extraction per poll iteration.** When multiple session files cross the extract-interval threshold in the same 60 s poll, only the first fires the LLM call. The rest defer to subsequent polls. Their `turnsAccum` and `offset` state is preserved, so they don't lose progress — they just stagger. With the default 60 s poll interval, 5 backlogged files take 5 minutes to drain instead of 5 seconds. Free-tier LLM rate limits don't trip.
+- **Stale-trajectory skip (>7 days mtime).** A user installing TotalReclaw on a host with months of OpenClaw session log history won't get a retroactive extraction backlog. Files with mtime older than 7 days are baseline-snapshotted (offset captured) but skip the extraction path entirely. If the user later resumes an old session, the offset is current and only net-new content extracts.
+
+### Implementation notes
+
+- `MAX_EXTRACTIONS_PER_POLL = 1` constant in `trajectory-poller.ts`. Sequential `for` loop already guarantees serialization within a poll; the cap just stops issuing additional LLM calls once the first one completes.
+- `STALE_TRAJECTORY_AGE_MS = 7 * 24 * 60 * 60 * 1000`. Stale files get a one-time offset record (`state[file] = {offset: stat.size, turnsAccum: 0}`) so they're never re-stat'd repeatedly. If the user re-engages an old session (writes new content → mtime refreshes), the stale-skip check fails and normal extraction resumes from the recorded offset.
+- 4 new test cases in `trajectory-poller.test.ts`: cap=1 multi-session deferral, second-poll-no-op-after-deferred-cap, stale-file skip (>7 days), recent-file (<7 days) NOT skipped. 44/44 green.
+- 92/92 fs-helpers + 21/21 register-command-name + 37/37 skill-md + 21/21 tr-cli-json + 44/44 trajectory-poller + 10/10 manifest-shape. check-scanner: 129 files, 0 flags.
+
+### Won't fix here
+
+- Pop-os specific delayed-load anomaly (~58 min from rc.4 install to plugin first-load) — couldn't reproduce in fresh container; pop-os has accumulated state from many install/wipe cycles. Auto-QA confirmed clean fresh-install path.
+- The retroactive-backlog scenario itself remains: if a user has ACTIVE sessions (mtime < 7 days) with high turn counts, they'll still drain over multiple polls. With cap=1 and 60 s interval, 5 active-but-backlogged files = 5 minutes to drain. Acceptable.
+
+## [3.3.11-rc.4] — 2026-05-07
+
+Fix #4 added to `patchOpenClawConfig()`: the **populated-allowlist plugin-skip bug** that broke every realistic install path.
+
+### Fixed
+
+- **`plugins.bundledDiscovery: "compat"` now auto-set when `plugins.allow` is populated.** OpenClaw 2026.5.x switches the plugin loader into strict-allowlist mode whenever `plugins.allow` is a non-empty array. In that mode, **non-bundled plugins like totalreclaw are silently rejected even when listed in the allow array** unless `plugins.bundledDiscovery: "compat"` is set. Without the compat flag, the gateway boots with only the bundled providers (e.g. `[device-pair, telegram]`), the totalreclaw plugin is dropped on the floor, and `extractd:` log lines never appear because the plugin's `register()` never runs.
+
+  This bug surfaces on every real-world install path:
+  - User installs telegram + zai/openai/anthropic before installing TotalReclaw → `plugins.allow` populated → loader rejects totalreclaw on every gateway restart.
+  - Pedro's pop-os QA host hit this on rc.3 because the host had `['device-pair', 'google', 'telegram', 'totalreclaw', 'zai']` in `plugins.allow` from prior install cycles.
+  - `openclaw doctor --fix` was the manual cure (sets `bundledDiscovery: "compat"`), but users shouldn't need to run doctor for the plugin they just installed.
+
+  Fix: `patchOpenClawConfig()` now sets `plugins.bundledDiscovery = "compat"` when `plugins.allow` is a non-empty array AND `bundledDiscovery` is unset. Power-users who explicitly chose `"allowlist"` (the stricter mode) keep their setting — only first-run defaults are touched. `plugins.allow=null` (auto-discover mode) is left alone.
+
+### Auto-QA gap (acknowledged)
+
+The 3.3.11-rc.1 auto-QA harness ran on a fresh canonical OpenClaw 2026.5.4 container with `plugins.allow=null` (auto-discover mode). It hit the looser code path where the plugin loaded fine. The strict-allowlist code path was untested. As a result, rc.3 shipped with this bug, Pedro hit it on pop-os, and I had to chase it down post-publish.
+
+This RC adds:
+- 5 new fs-helpers test cases covering Fix #4: empty-allow no-op, populated-allow + missing-bundledDiscovery patches, empty-array allow no-op, explicit `"allowlist"` preserved, full pop-os-style scenario.
+- TODO for next iteration: expand the live-container auto-QA harness to run a populated-allowlist scenario (mimicking a real install with telegram + zai pre-configured) alongside the existing fresh-allow=null scenario.
+
+### Implementation notes
+
+- 92/92 fs-helpers tests green. 21/21 register-command-name + 37/37 skill-md + 21/21 tr-cli-json + 40/40 trajectory-poller + 10/10 manifest-shape. check-scanner: 129 files, 0 flags.
+- The patchOpenClawConfig restart-required warn now mentions all four keys (slots.memory + allowConversationAccess + telegram.streaming.mode + bundledDiscovery).
+
 ## [3.3.11-rc.3] — 2026-05-06
 
 Two real-bug fixes flagged by Pedro's pop-os QA on rc.11-rc.1/rc.2: credentials.json shipped with only `{mnemonic}` (no userId, no salt), and the subgraph "Invalid character 'q' at position 0" Bytes-decode error.

package/SKILL.md

@@ -1,7 +1,7 @@
 ---
 name: totalreclaw
 description: "Install + set up TotalReclaw encrypted memory for OpenClaw, then use tr CLI for remember / recall. Trigger on 'install TotalReclaw', 'set up TotalReclaw', 'restore my recovery phrase', or any remember / recall request."
-version: 3.3.11-rc.3
+version: 3.3.11-rc.5
 author: TotalReclaw Team
 license: MIT
 homepage: https://totalreclaw.xyz

package/dist/fs-helpers.js

@@ -954,6 +954,16 @@ export function resolveOnboardingState(credentialsPath, statePath) {
  *      this to "off" on first run for a clean UX. Existing explicit values
  *      ("partial", "block", "progress") are preserved.
  *
+ *   4. `plugins.bundledDiscovery = "compat"` (only if unset, and only when
+ *      `plugins.allow` is populated). When `plugins.allow` is a non-empty
+ *      array, OpenClaw 2026.5.x switches the loader into strict-allowlist
+ *      mode and silently rejects non-bundled plugins like totalreclaw —
+ *      EVEN IF they are listed in the allow array. Setting
+ *      `bundledDiscovery: "compat"` restores the looser behavior so allow-
+ *      listed non-bundled plugins load. Without this fix, users with
+ *      telegram or any model provider configured before TR install (which
+ *      populates allow) get a silent plugin-skip on every gateway boot.
+ *
  * Design constraints
  * ------------------
  * - SYNCHRONOUS — called during register() which must be sync.
@@ -1076,6 +1086,36 @@ export function patchOpenClawConfig(configPath) {
                 }
             }
         }
+        // --- Fix #4: plugins.bundledDiscovery = "compat" (3.3.11-rc.4) ---
+        //
+        // OpenClaw 2026.5.x: when `plugins.allow` is populated (any non-empty
+        // array), the gateway's plugin loader switches into strict-allowlist
+        // mode. In strict mode, NON-BUNDLED plugins like totalreclaw are
+        // silently rejected even when listed in `plugins.allow`, unless
+        // `plugins.bundledDiscovery = "compat"` is explicitly set. Pedro's
+        // 2026-05-07 QA on pop-os surfaced this — the gateway booted with only
+        // the bundled providers (telegram, device-pair) and skipped totalreclaw
+        // despite it being in the allow list. `openclaw doctor --fix` cures it
+        // by setting `bundledDiscovery: "compat"`, but users shouldn't need
+        // to run doctor manually for the plugin to load.
+        //
+        // Fix: when `plugins.allow` is a non-empty array AND
+        // `plugins.bundledDiscovery` is unset, set it to "compat". If the user
+        // explicitly chose "allowlist" (the stricter mode), preserve their
+        // choice — only first-run defaults are touched.
+        //
+        // This bug was missed by the auto-QA harness because the harness ran
+        // on a fresh canonical container with `plugins.allow=null`, hitting
+        // the auto-discover code path. Real users with telegram + a model
+        // provider configured before TR install have a populated allow list,
+        // hitting the strict-mode path. The auto-QA harness in 3.3.11-rc.4
+        // adds a populated-allow scenario to catch future regressions.
+        if (Array.isArray(cfg.plugins.allow) && cfg.plugins.allow.length > 0) {
+            if (cfg.plugins.bundledDiscovery === undefined) {
+                cfg.plugins.bundledDiscovery = 'compat';
+                mutated = true;
+            }
+        }
         if (!mutated)
             return 'unchanged';
         // Write back with 2-space indent to match OpenClaw's own write style.

package/dist/index.js

@@ -2574,7 +2574,7 @@ const plugin = {
                     if (patchResult === 'patched') {
                         api.logger.warn('TotalReclaw: updated openclaw.json with required 2026.5.x keys ' +
                             '(plugins.slots.memory + hooks.allowConversationAccess + ' +
-                            'channels.telegram.streaming.mode). ' +
+                            'channels.telegram.streaming.mode + plugins.bundledDiscovery). ' +
                             'Gateway restart required for the changes to take effect. ' +
                             'Run `/totalreclaw-restart` or restart the gateway manually.');
                     }

package/dist/tr-cli.js

@@ -41,7 +41,7 @@ const STATE_PATH = CONFIG.onboardingStatePath;
 // Auto-synced by skill/scripts/sync-version.mjs from skill/plugin/package.json::version.
 // Do not edit by hand — running tests will catch drift but the publish workflow
 // rewrites this constant at the start of every npm/ClawHub publish.
-const PLUGIN_VERSION = '3.3.11-rc.3';
+const PLUGIN_VERSION = '3.3.11-rc.5';
 function die(msg, code = 1) {
     process.stderr.write(`tr: ${msg}\n`);
     process.exit(code);

package/dist/trajectory-poller.js

@@ -41,6 +41,15 @@ import os from 'node:os';
 import path from 'node:path';
 const DEFAULT_POLL_INTERVAL_MS = 60_000;
 const STATE_FILE = path.join(os.homedir(), '.totalreclaw', 'extract-state.json');
+/**
+ * Skip trajectory files older than this. A user who installs
+ * TotalReclaw on a host with months of OpenClaw session log history
+ * shouldn't get a retroactive extraction backlog — we only care about
+ * ongoing chat from now forward (3.3.11-rc.5). Files with mtime older
+ * than this threshold get a one-time offset snapshot so they're never
+ * re-scanned, and skip the extraction path entirely.
+ */
+const STALE_TRAJECTORY_AGE_MS = 7 * 24 * 60 * 60 * 1000; // 7 days
 /**
  * Start the trajectory poller. Runs an initial poll after 5 s, then
  * every `pollIntervalMs` (default 60 s). Returns a handle the caller
@@ -62,15 +71,51 @@ export function startTrajectoryPoller(deps, opts = {}) {
                 return;
             const extractInterval = deps.getExtractInterval();
             let stateChanged = false;
+            // 3.3.11-rc.5: cap extractions per poll iteration. Pedro's
+            // 2026-05-07 zai 429 cascade was caused by N session files all
+            // crossing the extract threshold in the same poll → N back-to-back
+            // LLM calls trip the rate-limiter (especially on free tiers).
+            // With cap=1, extra files defer to the next poll iteration
+            // (60 s later by default). Their turnsAccum is preserved across
+            // polls so they don't lose progress.
+            let extractionsThisPoll = 0;
+            const MAX_EXTRACTIONS_PER_POLL = 1;
             for (const file of files) {
+                // Stale-file skip: trajectory files untouched for STALE_TRAJECTORY_AGE_MS
+                // are likely abandoned sessions (old test runs, dead chats). Skip them
+                // entirely — don't burn LLM budget on extraction from stale content
+                // that the user has effectively forgotten about.
+                let mtimeMs = 0;
+                try {
+                    mtimeMs = fs.statSync(file).mtimeMs;
+                }
+                catch {
+                    continue;
+                }
+                if (Date.now() - mtimeMs > STALE_TRAJECTORY_AGE_MS) {
+                    // Lazy-record offset so we don't repeatedly re-scan stale files —
+                    // if the user later resumes this session, the offset is already
+                    // current and we'll only extract net-new content.
+                    if (!state[file]) {
+                        try {
+                            state[file] = { offset: fs.statSync(file).size, turnsAccum: 0 };
+                            stateChanged = true;
+                        }
+                        catch { /* ignore */ }
+                    }
+                    continue;
+                }
                 const lastEntry = state[file] ?? { offset: 0, turnsAccum: 0 };
                 const { messages, newOffset } = parseNewMessages(file, lastEntry.offset);
                 if (newOffset === lastEntry.offset)
                     continue; // nothing new
                 const turnsAdded = countTurns(messages);
                 const turnsAccum = lastEntry.turnsAccum + turnsAdded;
-                const shouldExtract = turnsAccum >= extractInterval && messages.length >= 2;
+                const shouldExtract = turnsAccum >= extractInterval &&
+                    messages.length >= 2 &&
+                    extractionsThisPoll < MAX_EXTRACTIONS_PER_POLL;
                 if (shouldExtract) {
+                    extractionsThisPoll++;
                     deps.logger.info(`extractd: ${path.basename(file)} -> ${turnsAccum}/${extractInterval} turns; running extraction (${messages.length} messages)`);
                     const existing = deps.isDedupEnabled() ? await deps.getDedupCandidates(20, messages) : [];
                     const rawFacts = await deps.runExtraction(messages, 'turn', existing, undefined);

package/fs-helpers.ts

@@ -1170,6 +1170,16 @@ export type OpenClawConfigPatchResult = 'patched' | 'unchanged' | 'skipped' | 'e
  *      this to "off" on first run for a clean UX. Existing explicit values
  *      ("partial", "block", "progress") are preserved.
  *
+ *   4. `plugins.bundledDiscovery = "compat"` (only if unset, and only when
+ *      `plugins.allow` is populated). When `plugins.allow` is a non-empty
+ *      array, OpenClaw 2026.5.x switches the loader into strict-allowlist
+ *      mode and silently rejects non-bundled plugins like totalreclaw —
+ *      EVEN IF they are listed in the allow array. Setting
+ *      `bundledDiscovery: "compat"` restores the looser behavior so allow-
+ *      listed non-bundled plugins load. Without this fix, users with
+ *      telegram or any model provider configured before TR install (which
+ *      populates allow) get a silent plugin-skip on every gateway boot.
+ *
  * Design constraints
  * ------------------
  * - SYNCHRONOUS — called during register() which must be sync.
@@ -1301,6 +1311,37 @@ export function patchOpenClawConfig(
       }
     }
 
+    // --- Fix #4: plugins.bundledDiscovery = "compat" (3.3.11-rc.4) ---
+    //
+    // OpenClaw 2026.5.x: when `plugins.allow` is populated (any non-empty
+    // array), the gateway's plugin loader switches into strict-allowlist
+    // mode. In strict mode, NON-BUNDLED plugins like totalreclaw are
+    // silently rejected even when listed in `plugins.allow`, unless
+    // `plugins.bundledDiscovery = "compat"` is explicitly set. Pedro's
+    // 2026-05-07 QA on pop-os surfaced this — the gateway booted with only
+    // the bundled providers (telegram, device-pair) and skipped totalreclaw
+    // despite it being in the allow list. `openclaw doctor --fix` cures it
+    // by setting `bundledDiscovery: "compat"`, but users shouldn't need
+    // to run doctor manually for the plugin to load.
+    //
+    // Fix: when `plugins.allow` is a non-empty array AND
+    // `plugins.bundledDiscovery` is unset, set it to "compat". If the user
+    // explicitly chose "allowlist" (the stricter mode), preserve their
+    // choice — only first-run defaults are touched.
+    //
+    // This bug was missed by the auto-QA harness because the harness ran
+    // on a fresh canonical container with `plugins.allow=null`, hitting
+    // the auto-discover code path. Real users with telegram + a model
+    // provider configured before TR install have a populated allow list,
+    // hitting the strict-mode path. The auto-QA harness in 3.3.11-rc.4
+    // adds a populated-allow scenario to catch future regressions.
+    if (Array.isArray(cfg.plugins.allow) && cfg.plugins.allow.length > 0) {
+      if (cfg.plugins.bundledDiscovery === undefined) {
+        cfg.plugins.bundledDiscovery = 'compat';
+        mutated = true;
+      }
+    }
+
     if (!mutated) return 'unchanged';
 
     // Write back with 2-space indent to match OpenClaw's own write style.

package/index.ts

@@ -3156,7 +3156,7 @@ const plugin = {
           api.logger.warn(
             'TotalReclaw: updated openclaw.json with required 2026.5.x keys ' +
               '(plugins.slots.memory + hooks.allowConversationAccess + ' +
-              'channels.telegram.streaming.mode). ' +
+              'channels.telegram.streaming.mode + plugins.bundledDiscovery). ' +
               'Gateway restart required for the changes to take effect. ' +
               'Run `/totalreclaw-restart` or restart the gateway manually.',
           );

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@totalreclaw/totalreclaw",
-  "version": "3.3.11-rc.3",
+  "version": "3.3.11-rc.5",
   "description": "End-to-end encrypted, agent-portable memory for OpenClaw and any LLM-agent runtime. XChaCha20-Poly1305 with protobuf v4 + on-chain Memory Taxonomy v1 (claim / preference / directive / commitment / episode / summary).",
   "type": "module",
   "keywords": [

package/skill.json

@@ -1,6 +1,6 @@
 {
   "name": "totalreclaw",
-  "version": "3.3.11-rc.3",
+  "version": "3.3.11-rc.5",
   "description": "End-to-end encrypted memory for AI agents — portable, yours forever. XChaCha20-Poly1305 E2EE: server never sees plaintext.",
   "author": "TotalReclaw Team",
   "license": "MIT",

package/tr-cli.ts

@@ -52,7 +52,7 @@ const STATE_PATH = CONFIG.onboardingStatePath;
 // Auto-synced by skill/scripts/sync-version.mjs from skill/plugin/package.json::version.
 // Do not edit by hand — running tests will catch drift but the publish workflow
 // rewrites this constant at the start of every npm/ClawHub publish.
-const PLUGIN_VERSION = '3.3.11-rc.3';
+const PLUGIN_VERSION = '3.3.11-rc.5';
 
 function die(msg: string, code = 1): never {
   process.stderr.write(`tr: ${msg}\n`);

package/trajectory-poller.ts

@@ -127,6 +127,15 @@ export interface TrajectoryPollerHandle {
 
 const DEFAULT_POLL_INTERVAL_MS = 60_000;
 const STATE_FILE = path.join(os.homedir(), '.totalreclaw', 'extract-state.json');
+/**
+ * Skip trajectory files older than this. A user who installs
+ * TotalReclaw on a host with months of OpenClaw session log history
+ * shouldn't get a retroactive extraction backlog — we only care about
+ * ongoing chat from now forward (3.3.11-rc.5). Files with mtime older
+ * than this threshold get a one-time offset snapshot so they're never
+ * re-scanned, and skip the extraction path entirely.
+ */
+const STALE_TRAJECTORY_AGE_MS = 7 * 24 * 60 * 60 * 1000; // 7 days
 
 /**
  * Start the trajectory poller. Runs an initial poll after 5 s, then
@@ -152,17 +161,54 @@ export function startTrajectoryPoller(
 
       const extractInterval = deps.getExtractInterval();
       let stateChanged = false;
+      // 3.3.11-rc.5: cap extractions per poll iteration. Pedro's
+      // 2026-05-07 zai 429 cascade was caused by N session files all
+      // crossing the extract threshold in the same poll → N back-to-back
+      // LLM calls trip the rate-limiter (especially on free tiers).
+      // With cap=1, extra files defer to the next poll iteration
+      // (60 s later by default). Their turnsAccum is preserved across
+      // polls so they don't lose progress.
+      let extractionsThisPoll = 0;
+      const MAX_EXTRACTIONS_PER_POLL = 1;
 
       for (const file of files) {
+        // Stale-file skip: trajectory files untouched for STALE_TRAJECTORY_AGE_MS
+        // are likely abandoned sessions (old test runs, dead chats). Skip them
+        // entirely — don't burn LLM budget on extraction from stale content
+        // that the user has effectively forgotten about.
+        let mtimeMs = 0;
+        try {
+          mtimeMs = fs.statSync(file).mtimeMs;
+        } catch {
+          continue;
+        }
+        if (Date.now() - mtimeMs > STALE_TRAJECTORY_AGE_MS) {
+          // Lazy-record offset so we don't repeatedly re-scan stale files —
+          // if the user later resumes this session, the offset is already
+          // current and we'll only extract net-new content.
+          if (!state[file]) {
+            try {
+              state[file] = { offset: fs.statSync(file).size, turnsAccum: 0 };
+              stateChanged = true;
+            } catch { /* ignore */ }
+          }
+          continue;
+        }
+
         const lastEntry = state[file] ?? { offset: 0, turnsAccum: 0 };
         const { messages, newOffset } = parseNewMessages(file, lastEntry.offset);
         if (newOffset === lastEntry.offset) continue; // nothing new
 
         const turnsAdded = countTurns(messages);
         const turnsAccum = lastEntry.turnsAccum + turnsAdded;
-        const shouldExtract = turnsAccum >= extractInterval && messages.length >= 2;
+        const shouldExtract =
+          turnsAccum >= extractInterval &&
+          messages.length >= 2 &&
+          extractionsThisPoll < MAX_EXTRACTIONS_PER_POLL;
 
         if (shouldExtract) {
+          extractionsThisPoll++;
+
           deps.logger.info(
             `extractd: ${path.basename(file)} -> ${turnsAccum}/${extractInterval} turns; running extraction (${messages.length} messages)`,
           );