@totalreclaw/totalreclaw 3.3.1-rc.16 → 3.3.1-rc.17

package/CHANGELOG.md

@@ -4,38 +4,6 @@ All notable changes to `@totalreclaw/totalreclaw` (the OpenClaw plugin) are docu
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## [3.3.1-rc.16] — 2026-04-24
-
-Fixes #92 — slow-host install times out during ONNX-runtime / embedding-model
-download. ONNX stays mandatory (no opt-in flag); first-call download is now
-wrapped with timeout, progress, and retry UX so slow connections succeed
-instead of silently hanging until OpenClaw SIGTERMs.
-
-### Embedding-model download UX
-
-- New `download-ux.ts` module — pure stdlib, no third-party imports — exposes
-  `downloadWithUX(label, fn, opts)`. Wraps a download promise with:
-  - **Per-attempt timeout**, default 600s (covers ~290 KB/s for the 344 MB
-    Harrier model). Configurable via env `TOTALRECLAW_ONNX_INSTALL_TIMEOUT`
-    (in seconds). Per-attempt timeout grows 1x/2x/4x across retries.
-  - **60s keep-alive log** during long downloads so users on slow networks
-    see "still downloading… (Ns elapsed)" rather than a frozen prompt.
-  - **3-attempt exponential-backoff retry** (5s/10s backoff between attempts)
-    to absorb transient network blips.
-  - **Loud actionable error** on exhaustion: names the env var to extend the
-    timeout and the exact `openclaw plugins install totalreclaw` command to
-    rerun.
-- `embedding.ts` now wraps `AutoTokenizer.from_pretrained`,
-  `AutoModel.from_pretrained`, and the `pipeline()` call with
-  `downloadWithUX`. Prints a user-visible "Downloading embedding model
-  (~344MB) — this may take a few minutes on slower connections. Please wait."
-  message before the first download starts.
-- ONNX remains a mandatory hard `dependency` (no `[embedding]`-style opt-in
-  extra). Recall accuracy is unchanged.
-- Regression: `test_issue_92_onnx_download_ux.test.ts` exercises happy path,
-  transient failure → retry, full exhaustion, per-attempt timeout, and
-  keep-alive cadence. Wired into the plugin `npm test` chain.
-
 ## [3.3.1-rc.14] — 2026-04-24
 
 Coordinated version bump with Python `2.3.1rc14`. Two narrow bug fixes

package/SKILL.md

@@ -66,7 +66,7 @@ The QR encodes ONLY the pair URL. The 6-digit PIN is a separate out-of-band conf
 
 ## Phrase safety (HARD — never break)
 
-NEVER echo / generate / ask the user to paste a recovery phrase in chat. NEVER invoke `openclaw totalreclaw onboard`, `totalreclaw setup`, or any phrase-touching CLI via your shell tool (stdout enters LLM context). The ONLY agent-facilitated setup path is `totalreclaw_pair`. If the user pastes a phrase anyway: tell them it is compromised, call `totalreclaw_pair` with `mode=generate` for a fresh wallet.
+NEVER echo / generate / ask the user to paste a recovery phrase in chat. NEVER invoke `openclaw totalreclaw onboard`, `totalreclaw setup`, or any phrase-touching CLI via your shell tool (stdout enters LLM context). The ONLY agent-facilitated setup paths are (in order of preference): (1) `totalreclaw_pair` tool; (2) if the tool is missing from your tool list — a known gap for some container-based agents, issue #87 — shell out to `openclaw totalreclaw pair generate --url-pin-only`, which emits a single-line JSON `{v,url,pin,expires_at_ms}` with zero phrase material on stdout (pair-crypto is x25519-only, not BIP-39). If the user pastes a phrase anyway: tell them it is compromised, call `totalreclaw_pair` with `mode=generate` for a fresh wallet.
 
 ## Usage (post-setup)
 
@@ -78,7 +78,7 @@ NEVER echo / generate / ask the user to paste a recovery phrase in chat. NEVER i
 
 ## Diagnostics
 
-- `totalreclaw_*` tools not visible → plugin not installed or gateway not restarted. Verify via `openclaw plugins list | grep totalreclaw`. If listed, ask the user to restart the gateway.
+- `totalreclaw_*` tools not visible → plugin not installed or gateway not restarted. Verify via `openclaw plugins list | grep totalreclaw`. If listed but tools still absent (container-agent gap, issue #87), shell out to `openclaw totalreclaw pair generate --url-pin-only` on the gateway host and hand the user the `url` + `pin` from the JSON payload — do NOT suggest `openclaw totalreclaw onboard` (leaks phrase on stdout).
 - User says done but `credentials.json` missing → PIN expired or entered wrong phrase; call `totalreclaw_pair` again.
 - `onboarding required` → credentials missing; redo from the pair step.
 - `quota exceeded` → `totalreclaw_status`, then offer `totalreclaw_upgrade`.

package/embedding.ts

@@ -9,17 +9,10 @@
  * `TOTALRECLAW_EMBEDDING_MODEL` user-facing env var was removed in v1.
  *
  * Dependencies: @huggingface/transformers
- *
- * Download UX (rc.16, fixes #92):
- *   First-call download is wrapped via `downloadWithUX` from `download-ux.ts`
- *   — configurable timeout (`TOTALRECLAW_ONNX_INSTALL_TIMEOUT`, default 600s),
- *   60s keep-alive, 3-attempt exponential-backoff retry, loud actionable
- *   failure. Slow-bandwidth hosts no longer see a silent freeze.
  */
 
 // @ts-ignore - @huggingface/transformers types may not be perfect
 import { AutoTokenizer, AutoModel, pipeline, type FeatureExtractionPipeline } from '@huggingface/transformers';
-import { downloadWithUX, getDownloadTimeoutMs } from './download-ux.js';
 
 interface ModelConfig {
   id: string;
@@ -61,36 +54,20 @@ export async function generateEmbedding(
 ): Promise<number[]> {
   if (!activeModel) {
     activeModel = getModelConfig();
-    const timeoutSec = Math.floor(getDownloadTimeoutMs() / 1000);
-    console.error(
-      `[TotalReclaw] Downloading embedding model (${activeModel.size}) — this may take a few minutes on slower connections. Please wait.`,
-    );
-    console.error(
-      `[TotalReclaw] One-time setup. Per-attempt timeout: ${timeoutSec}s (configurable via TOTALRECLAW_ONNX_INSTALL_TIMEOUT). Cached after first download.`,
-    );
+    console.error(`[TotalReclaw] Downloading embedding model (${activeModel.size}, one-time setup)...`);
+    console.error('[TotalReclaw] This enables semantic search across your encrypted memories.');
 
     if (activeModel.pooling === 'sentence_embedding') {
       // Harrier: use AutoModel (pipeline doesn't support sentence_embedding output)
-      autoTokenizer = await downloadWithUX(
-        'tokenizer',
-        () => AutoTokenizer.from_pretrained(activeModel!.id),
-      );
-      autoModel = await downloadWithUX(
-        'embedding model',
-        () =>
-          AutoModel.from_pretrained(activeModel!.id, {
-            dtype: activeModel!.dtype as any,
-          }),
-      );
+      autoTokenizer = await AutoTokenizer.from_pretrained(activeModel.id);
+      autoModel = await AutoModel.from_pretrained(activeModel.id, {
+        dtype: activeModel.dtype as any,
+      });
     } else {
       // e5-small / Qwen: use pipeline
-      pipelineExtractor = await downloadWithUX(
-        'embedding pipeline',
-        () =>
-          pipeline('feature-extraction', activeModel!.id, {
-            dtype: activeModel!.dtype as any,
-          }),
-      );
+      pipelineExtractor = await pipeline('feature-extraction', activeModel.id, {
+        dtype: activeModel.dtype as any,
+      });
     }
     console.error('[TotalReclaw] Embedding model ready. Future startups will be instant.');
   }

package/index.ts

@@ -5412,11 +5412,18 @@ const plugin = {
                 '## TotalReclaw setup pending\n\n' +
                 'TotalReclaw encrypted memory is installed but not yet set up on this machine. ' +
                 'If the user asks about memory features or wants to configure TotalReclaw, ' +
-                'call the `totalreclaw_onboarding_start` tool for instructions.\n\n' +
+                'call the `totalreclaw_pair` tool — it returns a pairing URL + 6-digit PIN ' +
+                'the user opens in a browser (phrase is generated and encrypted in-browser, ' +
+                'never leaves their device in plaintext). If `totalreclaw_pair` is not in ' +
+                'your tool list (container-based agents sometimes miss it), shell out to ' +
+                '`openclaw totalreclaw pair generate --url-pin-only` on the gateway host — ' +
+                'it emits a single JSON line `{v,url,pin,expires_at_ms}` with no phrase ' +
+                'material; hand the user the `url` and `pin`.\n\n' +
                 '**Do NOT** attempt to generate, display, or relay a recovery phrase in chat. ' +
-                'All phrase handling happens on the user\'s local terminal via ' +
-                '`openclaw totalreclaw onboard` — this keeps the phrase out of the LLM ' +
-                'provider\'s logs.',
+                '**Do NOT** run `openclaw totalreclaw onboard` — that CLI emits the recovery ' +
+                'phrase on stdout and would leak it into the LLM transcript. Use `pair` ' +
+                '(tool or `--url-pin-only` CLI) instead; `onboard` is reserved for users ' +
+                'running it directly in their own local terminal.',
             };
           }
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@totalreclaw/totalreclaw",
-  "version": "3.3.1-rc.16",
+  "version": "3.3.1-rc.17",
   "description": "End-to-end encrypted, agent-portable memory for OpenClaw and any LLM-agent runtime. XChaCha20-Poly1305 with protobuf v4 + on-chain Memory Taxonomy v1 (claim / preference / directive / commitment / episode / summary).",
   "type": "module",
   "keywords": [
@@ -54,7 +54,7 @@
     "skill.json"
   ],
   "scripts": {
-    "test": "npx tsx manifest-shape.test.ts && npx tsx config-schema.test.ts && npx tsx llm-profile-reader.test.ts && npx tsx llm-client.test.ts && npx tsx llm-client-retry.test.ts && npx tsx gateway-url.test.ts && npx tsx retype-setscope.test.ts && npx tsx tool-gating.test.ts && npx tsx onboarding-noninteractive.test.ts && npx tsx pair-cli-json.test.ts && npx tsx pair-qr.test.ts && npx tsx pair-remote-client.test.ts && npx tsx qa-bug-report.test.ts && npx tsx nonce-serialization.test.ts && npx tsx phrase-safety-registry.test.ts && npx tsx test_issue_92_onnx_download_ux.test.ts",
+    "test": "npx tsx manifest-shape.test.ts && npx tsx config-schema.test.ts && npx tsx llm-profile-reader.test.ts && npx tsx llm-client.test.ts && npx tsx llm-client-retry.test.ts && npx tsx gateway-url.test.ts && npx tsx retype-setscope.test.ts && npx tsx tool-gating.test.ts && npx tsx onboarding-noninteractive.test.ts && npx tsx pair-cli-json.test.ts && npx tsx pair-qr.test.ts && npx tsx pair-remote-client.test.ts && npx tsx qa-bug-report.test.ts && npx tsx nonce-serialization.test.ts && npx tsx phrase-safety-registry.test.ts",
     "check-scanner": "node ../scripts/check-scanner.mjs",
     "prepublishOnly": "node ../scripts/check-scanner.mjs"
   },

package/pair-cli.ts

@@ -85,8 +85,15 @@ export interface PairCliOutcome {
  *     as the session reaches a terminal state — same status-code
  *     semantics as 'human' (0 on completed, 1 on expired/rejected/error,
  *     130 on canceled).
+ *   - 'url-pin': (3.3.1-rc.15, issue #87) headless container-agent fallback.
+ *     Emits ONLY `{ v, url, pin, expires_at_ms }` — no QR ASCII, no SID,
+ *     no mode echo. Use when a container-based agent cannot see the
+ *     `totalreclaw_pair` tool (OpenClaw gateway-to-container tool-injection
+ *     gap) and must shell out to the CLI. Guarantees zero phrase material
+ *     on stdout by construction — pair-crypto is x25519-only and the slim
+ *     payload carries nothing BIP-39-adjacent.
  */
-export type PairCliOutputMode = 'human' | 'json';
+export type PairCliOutputMode = 'human' | 'json' | 'url-pin';
 
 /**
  * JSON payload emitted by runPairCli when outputMode === 'json'. Printed
@@ -103,6 +110,17 @@ export interface PairCliJsonPayload {
   qr_ascii: string;
 }
 
+/**
+ * Slim payload for outputMode === 'url-pin'. Intentionally a subset of
+ * `PairCliJsonPayload` with no QR ASCII, SID, or mode echo. Issue #87.
+ */
+export interface PairCliUrlPinPayload {
+  v: 1;
+  url: string;
+  pin: string;
+  expires_at_ms: number;
+}
+
 // ---------------------------------------------------------------------------
 // Default stdout IO
 // ---------------------------------------------------------------------------
@@ -213,9 +231,11 @@ export async function runPairCli(
     return { status: 'error', error: msg };
   }
 
-  // 2. Render the QR — promise-based so both human + json modes share it.
+  // 2. Build the URL unconditionally, but only render the QR for modes
+  //    that actually emit it. url-pin mode skips the renderer entirely —
+  //    no CPU cost, no qrcode-terminal import, no ASCII on stdout.
   const url = deps.renderPairingUrl(session);
-  const qrAscii = await new Promise<string>((resolve) => {
+  const qrAscii = outputMode === 'url-pin' ? '' : await new Promise<string>((resolve) => {
     // Guard against QR renderers that never fire their callback (shouldn't
     // happen with qrcode-terminal, but defensive): a 10-second timeout
     // returns an empty string so we never hang the pairing flow.
@@ -241,8 +261,16 @@ export async function runPairCli(
     }
   });
 
-  // 3. Emit the visible surface (JSON first — single line — or human copy).
-  if (outputMode === 'json') {
+  // 3. Emit the visible surface (JSON/url-pin first — single line — or human copy).
+  if (outputMode === 'url-pin') {
+    const payload: PairCliUrlPinPayload = {
+      v: 1,
+      url,
+      pin: session.secondaryCode,
+      expires_at_ms: session.expiresAtMs,
+    };
+    stdout.write(JSON.stringify(payload) + '\n');
+  } else if (outputMode === 'json') {
     const payload: PairCliJsonPayload = {
       v: 1,
       sid: session.sid,
@@ -276,7 +304,9 @@ export async function runPairCli(
     canceled = true;
   });
 
-  // 5. Poll
+  // 5. Poll — status transitions only surface in human mode; json/url-pin
+  //    modes stay silent after the single payload line so agents parsing
+  //    stdout get one JSON line and an exit code, nothing else.
   const emitStatus = (text: string): void => {
     if (outputMode === 'human') stdout.write(text);
   };
@@ -399,14 +429,19 @@ export function registerPairCli(
       'Pair a remote browser device to this gateway (mode = generate | import; default generate)',
     )
     .option('--json', 'Emit a single JSON payload (url/pin/sid/qr_ascii) instead of the human-readable banner. Enables agent-driven pairing.')
+    .option('--url-pin-only', 'Emit ONLY {v,url,pin,expires_at_ms} — no QR ASCII, no SID, no mode echo. Headless fallback for container-based agents where the totalreclaw_pair tool is not injected (issue #87). Zero phrase exposure on stdout.')
     .option('--timeout <sec>', 'Session TTL in seconds (default: 900 = 15 min, matches pair-session-store default)')
     .action(async (...args: unknown[]) => {
       // commander passes: [modeArg, options, cmd]
       const modeRaw = typeof args[0] === 'string' ? args[0] : undefined;
-      const opts = (args[1] ?? {}) as { json?: boolean; timeout?: string | number };
+      const opts = (args[1] ?? {}) as { json?: boolean; urlPinOnly?: boolean; timeout?: string | number };
       const mode: PairCliMode =
         modeRaw === 'import' || modeRaw === 'imp' ? 'import' : 'generate';
-      const outputMode: PairCliOutputMode = opts.json ? 'json' : 'human';
+      // --url-pin-only wins over --json when both are passed, since it is
+      // strictly the tighter surface (no QR, no SID). The flag is a subset.
+      const outputMode: PairCliOutputMode = opts.urlPinOnly
+        ? 'url-pin'
+        : opts.json ? 'json' : 'human';
       let ttlSeconds: number | undefined;
       if (typeof opts.timeout === 'number' && Number.isFinite(opts.timeout)) {
         ttlSeconds = opts.timeout;

package/download-ux.ts

@@ -1,91 +0,0 @@
-/**
- * download-ux.ts — Wrapper for heavy first-call downloads (rc.16, fixes #92).
- *
- * Wraps a download promise with:
- *   - per-attempt timeout (default 600s, override via TOTALRECLAW_ONNX_INSTALL_TIMEOUT in seconds)
- *   - 60s keep-alive log so slow-bandwidth users don't think it's frozen
- *   - 3-attempt exponential-backoff retry (per-attempt timeout grows 1x/2x/4x)
- *   - loud actionable error after exhaustion
- *
- * No third-party imports here — pure stdlib so the unit test can exercise it
- * without pulling the heavy `@huggingface/transformers` chain.
- */
-
-const DEFAULT_DOWNLOAD_TIMEOUT_MS = 600_000;
-const KEEPALIVE_INTERVAL_MS = 60_000;
-const MAX_DOWNLOAD_ATTEMPTS = 3;
-
-export function getDownloadTimeoutMs(): number {
-  const raw = process.env.TOTALRECLAW_ONNX_INSTALL_TIMEOUT;
-  if (!raw) return DEFAULT_DOWNLOAD_TIMEOUT_MS;
-  const parsed = Number(raw);
-  if (!Number.isFinite(parsed) || parsed <= 0) return DEFAULT_DOWNLOAD_TIMEOUT_MS;
-  // Spec accepts seconds; convert to ms.
-  return Math.floor(parsed * 1000);
-}
-
-export interface DownloadWithUXOpts {
-  /** Override the per-attempt base timeout in ms (env var takes precedence by default). */
-  timeoutMs?: number;
-  /** Override the keep-alive cadence in ms. */
-  keepaliveMs?: number;
-  /** Override the max attempts. */
-  maxAttempts?: number;
-  /** Logger override (defaults to console.error). */
-  log?: (msg: string) => void;
-  /** Sleep override for tests; defaults to setTimeout. */
-  sleep?: (ms: number) => Promise<void>;
-}
-
-export async function downloadWithUX<T>(
-  label: string,
-  download: () => Promise<T>,
-  opts?: DownloadWithUXOpts,
-): Promise<T> {
-  const baseTimeoutMs = opts?.timeoutMs ?? getDownloadTimeoutMs();
-  const keepaliveMs = opts?.keepaliveMs ?? KEEPALIVE_INTERVAL_MS;
-  const maxAttempts = opts?.maxAttempts ?? MAX_DOWNLOAD_ATTEMPTS;
-  const log = opts?.log ?? ((msg: string) => console.error(msg));
-  const sleep = opts?.sleep ?? ((ms: number) => new Promise(r => setTimeout(r, ms)));
-
-  let lastErr: unknown = null;
-
-  for (let attempt = 1; attempt <= maxAttempts; attempt++) {
-    const attemptTimeoutMs = baseTimeoutMs * Math.pow(2, attempt - 1);
-    const startedAt = Date.now();
-    const keepaliveTimer = setInterval(() => {
-      const elapsedSec = Math.floor((Date.now() - startedAt) / 1000);
-      log(`[TotalReclaw] ${label}: still downloading… (${elapsedSec}s elapsed, attempt ${attempt}/${maxAttempts})`);
-    }, keepaliveMs);
-
-    try {
-      const result = await Promise.race([
-        download(),
-        new Promise<never>((_, reject) =>
-          setTimeout(
-            () => reject(new Error(`Download timeout after ${Math.floor(attemptTimeoutMs / 1000)}s (attempt ${attempt}/${maxAttempts})`)),
-            attemptTimeoutMs,
-          ),
-        ),
-      ]);
-      clearInterval(keepaliveTimer);
-      return result;
-    } catch (err) {
-      clearInterval(keepaliveTimer);
-      lastErr = err;
-      const msg = err instanceof Error ? err.message : String(err);
-      if (attempt < maxAttempts) {
-        const backoffMs = Math.min(5_000 * Math.pow(2, attempt - 1), 30_000);
-        log(`[TotalReclaw] ${label}: attempt ${attempt} failed (${msg}). Retrying in ${Math.floor(backoffMs / 1000)}s…`);
-        await sleep(backoffMs);
-      }
-    }
-  }
-
-  const finalMsg = lastErr instanceof Error ? lastErr.message : String(lastErr);
-  throw new Error(
-    `[TotalReclaw] Embedding model download failed after ${maxAttempts} attempts (last error: ${finalMsg}). ` +
-      `Check your network connection and retry: \`openclaw plugins install totalreclaw\`. ` +
-      `On slow connections, set TOTALRECLAW_ONNX_INSTALL_TIMEOUT=1200 (in seconds) to extend the per-attempt timeout.`,
-  );
-}