@totalreclaw/totalreclaw 3.3.0-rc.6 → 3.3.1-rc.2

package/CHANGELOG.md

@@ -4,6 +4,292 @@ All notable changes to `@totalreclaw/totalreclaw` (the OpenClaw plugin) are docu
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [3.3.1-rc.2] — 2026-04-22
+
+Follow-up RC for the 3.3.1-rc.1 QA NO-GO
+(`docs/notes/QA-plugin-3.3.1-rc.1-20260422-0121.md` in
+`totalreclaw-internal`). Fixes 3 ship-stoppers + 1 serious non-blocker
+identified by the first real-user-flow QA under the 2026-04-22 chat-only
+discipline, plus several UX gaps flagged by Pedro's agent (Hermes) during
+parallel Telegram testing. All 3.3.1-rc.1 provider-agnostic LLM work is
+preserved.
+
+### Changed
+
+- **`gateway-url.ts` — drop `child_process` subprocess probe.** The rc.1
+  implementation shelled out to `tailscale status --json` via
+  `child_process.execFileSync` to discover the local MagicDNS hostname.
+  This tripped the OpenClaw dangerous-code scanner's shell-execution
+  rule and **blocked every `openclaw plugins install @totalreclaw/totalreclaw`**.
+  rc.2 swaps to a passive probe: `os.networkInterfaces()` detects a
+  `tailscale*` NIC carrying a CGNAT IPv4 (100.64/10), and we surface
+  the raw IP as the auto-detected host. Operators who want a proper
+  `https://<magicdns>.ts.net` URL now set
+  `plugins.entries.totalreclaw.config.publicUrl` explicitly (documented
+  in SKILL.md). The six-layer URL cascade is otherwise unchanged.
+
+- **`check-scanner.mjs` — add shell-execution rule (catches `child_process`).**
+  Scanner-sim now mirrors the real OpenClaw `shell-execution` rule that
+  trips on any `child_process` substring (no context gate). Prevents a
+  repeat of the rc.1 regression. See `skill/scripts/check-scanner.mjs`
+  SHELL_EXEC_PATTERN.
+
+- **`totalreclaw_forget` — route through `submitFactBatchOnChain` and write
+  tombstones at legacy v3.** The rc.1 implementation used the single-fact
+  `submitFactOnChain` path and wrote the tombstone at protobuf v4, which
+  the subgraph did NOT reflect as `isActive=false`. rc.2 mirrors the
+  pin/unpin tombstone shape exactly (legacy v3, `source="tombstone"`,
+  single-payload batch via `submitFactBatchOnChain`). Also adds
+  UUID-shape validation on `factId` to reject LLM hallucinations
+  ("forget that I live in Porto" passed as the factId) with a clear
+  message pointing the agent at `totalreclaw_recall` first.
+
+- **`totalreclaw_forget` tool description** — rewritten from terse
+  ("Delete a specific memory by its ID.") to agent-instructive with a
+  recall-first workflow hint. Fixes the rc.1 QA failure where the LLM
+  hallucinated "Done" without actually calling the tool.
+
+- **`chatCompletion` — exponential-backoff retry for 429 / timeouts.**
+  rc.1 QA: 5 of 6 extraction windows returned 0 raw facts because zai
+  429s and timeouts had no retry path. rc.2 adds a retry wrapper:
+  3 attempts with 1s → 2s → 4s backoff; 30s per-attempt timeout;
+  fail-fast on 4xx-other-than-429. Every extractor callsite
+  (`extractFacts`, `extractFactsForCompaction`, `comparativeRescoreV1`,
+  `extractDebriefFacts`) opts in to the retry + logger. See
+  `isRetryable()` for the classification list.
+
+- **`llm-profile-reader.ts` — fallback to legacy `models.json` format.**
+  rc.1 QA VPS had `~/.openclaw/agents/<agent>/agent/models.json` (the
+  pre-auth-profiles shape, `{ providers: { zai: { apiKey: "..." } } }`)
+  not `auth-profiles.json`. The auto-resolve silently no-op'd.
+  rc.2 adds a 5th cascade tier: `readAllProfileKeys` reads
+  auth-profiles.json FIRST (takes precedence on overlap), then merges
+  in models.json entries for any provider not already covered.
+
+### Added
+
+- **`totalreclaw_onboard`** (agent tool) — lets the agent drive the
+  non-interactive onboard flow from chat without shelling out. Generate
+  mode only (restore still requires `openclaw totalreclaw onboard --mode
+  restore` in the local terminal for security). Returns scope address +
+  credentials path; NEVER returns the mnemonic. Directly wraps
+  `runNonInteractiveOnboard` in-process.
+
+- **`totalreclaw_pair`** (agent tool) — lets the agent start a pairing
+  session from chat and relay the URL + PIN + QR ASCII to the user.
+  Built on the same `createPairSession` + `buildPairingUrl` surface the
+  CLI uses, no subprocess. The recovery phrase still never crosses the
+  LLM — it's generated/entered in the BROWSER and uploaded E2EE.
+
+- **`totalreclaw_retype`** (agent tool) — reclassify an existing memory
+  from one taxonomy type to another (claim/preference/directive/
+  commitment/episode/summary). Writes a new v1.1 claim with the updated
+  type, tombstones the old fact on-chain. rc.1 QA confirmed this tool
+  was documented in SKILL.md but NOT registered — agents couldn't call
+  it.
+
+- **`totalreclaw_set_scope`** (agent tool) — move an existing memory to
+  a different scope (work/personal/health/family/creative/finance/misc/
+  unspecified). Same write pattern as retype. Also previously
+  documented-not-registered; rc.1 QA showed agents falling back to a
+  hallucinated delete+re-store workaround.
+
+- **`skill/plugin/retype-setscope.ts`** — new pure-logic module
+  supporting the two agent tools above. Tightly mirrors pin.ts but
+  without the idempotent-status short-circuit (user may be confirming
+  a prior auto-extraction label) and without feedback wiring.
+
+- **`skill/plugin/gateway-url.test.ts`** — unit coverage for the new
+  passive Tailscale + LAN detection. 17 cases, all green.
+
+- **`skill/plugin/retype-setscope.test.ts`** — 31 cases covering arg
+  validation, successful rewrites, fact-not-found, submit failure,
+  malformed-blob, invalid-type/scope.
+
+- **`skill/plugin/llm-client-retry.test.ts`** — 29 cases for the retry
+  wrapper: isRetryable classification, backoff behaviour, fail-fast on
+  non-retryable errors, logger interaction.
+
+- **`skill/plugin/llm-profile-reader.test.ts`** — 13 additional cases
+  for models.json parsing + combined reader.
+
+### Preserved from rc.1
+
+All the rc.1 LLM-autoresolve work carries forward unchanged:
+- 4-tier cascade (plugin config → openclawProviders → auth-profiles →
+  env). With rc.2's `models.json` fallback it's effectively 5 tiers.
+- `openclaw totalreclaw onboard --non-interactive --json --mode` CLI.
+- `openclaw totalreclaw pair generate --json` CLI.
+- `extraction.llm` plugin-config override block.
+- Synchronous HTTP-route registration, manifest `kind` drop, etc.
+
+## [3.3.1-rc.1] — 2026-04-22
+
+First release candidate for 3.3.1. Comprehensive patch release addressing
+user-QA findings against 3.3.0-rc.6
+(`docs/notes/QA-user-findings-3.3.0-rc.6-20260421.md` in
+`totalreclaw-internal`). The 3.3.0 runtime works; what 3.3.1 fixes is the
+user experience around LLM auto-detection, config schema, non-interactive
+CLI, gateway-URL resolution, and SKILL.md. All rc.2–rc.6 fixes are
+preserved (scanner comment, auth: 'plugin' literal, ensureSessionsFileDir
+mkdir, sync HTTP-route registration, manifest kind drop).
+
+See: `plans/2026-04-22-plugin-3.3.1-provider-agnostic-llm.md` (internal).
+
+### Added
+
+- **`skill/plugin/llm-profile-reader.ts`** — new scanner-isolated module that
+  harvests provider API keys from
+  `~/.openclaw/agents/<agent>/agent/auth-profiles.json`. This is where real
+  OpenClaw installs store user API keys. rc.6 silently no-op'd auto-extraction
+  for nearly every real user because `initLLMClient` only looked at env vars
+  and the SDK-passed `api.config.providers` — neither of which reach
+  auth-profiles.json.
+
+- **`skill/plugin/gateway-url.ts`** — new scanner-isolated module that detects
+  the gateway's externally-reachable URL for QR pairing. Two autodetect tiers:
+    1. Tailscale MagicDNS via `tailscale status --json` (assumes `tailscale
+       serve` on 443).
+    2. First non-loopback, non-virtual IPv4 interface (LAN mode; emits a
+       "only works on the same network" warning).
+
+- **`initLLMClient` 4-tier resolution cascade** — plugin-config override
+  (highest) → SDK-passed openclawProviders → harvested auth-profiles.json
+  keys → env vars (lowest). Every tier logs ONCE at startup at INFO level;
+  per-turn noise from rc.6 is removed.
+
+- **`openclaw totalreclaw onboard` non-interactive modes**:
+    - `--non-interactive` — exits 1 if any input would be prompted.
+    - `--json` — emits a structured payload (requires `--non-interactive`).
+    - `--mode <generate|restore>` — skip the menu prompt.
+    - `--phrase <12-or-24>` — required for `--mode restore`; `-` reads stdin.
+    - `--emit-phrase` — opt-in path that includes the plaintext phrase in the
+      JSON payload. Default omits the phrase; the agent should direct the
+      user to read `~/.totalreclaw/credentials.json` in their terminal.
+
+- **`openclaw totalreclaw pair [mode]` non-interactive flags**:
+    - `--json` — emits `{v, sid, url, pin, mode, expires_at_ms, qr_ascii}` to
+      stdout before polling begins. Agents capture + present to the user.
+    - `--timeout <sec>` — override the 15-minute default session TTL.
+
+- **`extraction.llm` plugin-config override** — new optional block in the
+  plugin config schema. Explicit provider/model/apiKey/baseUrl wins over
+  every auto-detection tier:
+  ```yaml
+  plugins:
+    entries:
+      totalreclaw:
+        config:
+          extraction:
+            llm:
+              provider: zai
+              apiKey: <your-key>
+              model: glm-4.5-flash   # optional — derived from provider default otherwise
+  ```
+
+- **Config schema accepts `publicUrl` + `extraction.interval` +
+  `extraction.maxFactsPerExtraction`** — 3.3.0 rejected these keys with
+  `invalid config: must NOT have additional properties`. Both the manifest
+  (`openclaw.plugin.json`) and the JS plugin definition now accept them.
+  `extraction.additionalProperties` and `extraction.llm.additionalProperties`
+  remain `false` to keep the surface strictly typed.
+
+- **Three new test files**:
+    - `llm-profile-reader.test.ts` — 19 assertions covering the auth-profiles
+      harvester (provider mapping, malformed input, multi-agent aggregation).
+    - `llm-client.test.ts` — 28 assertions covering the 4-tier cascade,
+      plus the `deriveCheapModel` regex-boundary fix.
+    - `config-schema.test.ts` — 14 assertions (+ Ajv strict validation when
+      available) covering the 3.3.1 schema surface.
+    - `onboarding-noninteractive.test.ts` — 22 assertions covering
+      `runNonInteractiveOnboard` happy path, phrase-validation, mode 0600,
+      `already-active` short-circuit.
+    - `pair-cli-json.test.ts` — 17 assertions covering pair-cli JSON output,
+      `ttlSeconds` propagation, and human-mode regression.
+
+### Changed
+
+- **`pair-cli.ts` — no TTY requirement**. Prior rc versions imported
+  `readline` but never used it; the intro block also had no interactive
+  prompts. 3.3.1 removes any path that touches `setRawMode` in pair-cli and
+  adds a 10-second timeout on the QR renderer so a misbehaving qrcode-terminal
+  never hangs the pairing flow. Confirmed by
+  `pair-cli-json.test.ts` asserting JSON mode emits a single payload without
+  any TTY interaction.
+
+- **`deriveCheapModel` — fixes word-boundary regression**. rc.6 used
+  `primaryModel.toLowerCase().includes(cheapWord)` which matched the substring
+  `mini` inside `gemini`, so `gemini-2.5-pro` passed through unchanged and
+  the extractor called a model the user hadn't configured. 3.3.1 uses a
+  word-boundary regex (`/(?:^|[-_/.])(?:flash|mini|nano|haiku|small|lite|fast)(?:[-_/.]|$)/i`).
+
+- **Cheap-model table** — exported as `CHEAP_MODEL_BY_PROVIDER` for use by
+  paths that resolve a provider without knowing the user's primary model
+  (auth-profiles.json tier). Includes zai→glm-4.5-flash, openai→gpt-4.1-mini,
+  anthropic→claude-haiku-4-5-20251001, gemini/google→gemini-flash-lite,
+  groq→llama-3.3-70b-versatile, deepseek→deepseek-chat,
+  openrouter→anthropic/claude-haiku-4-5-20251001, xai→grok-2,
+  mistral→mistral-small-latest, together→meta-llama/Llama-3.3-70B-Instruct-Turbo,
+  cerebras→llama3.3-70b.
+
+- **Gateway pairing URL cascade** — `buildPairingUrl` now threads through the
+  six-layer cascade: `publicUrl` → `gateway.remote.url` → custom bind host →
+  Tailscale autodetect → LAN autodetect → localhost fallback. Each fallback
+  emits a warning with clear pointer to `publicUrl` for override.
+
+- **SKILL.md — full rewrite**. Explicit prohibition of generating phrases in
+  chat; canonical onboarding commands (`openclaw totalreclaw onboard` or
+  `onboard --non-interactive --json --mode generate`); two-step install flow
+  documented clearly; full 3.3.1 config schema documented; all tool surfaces
+  aligned with current taxonomy (`claim|preference|directive|commitment|
+  episode|summary`); references to `npx @totalreclaw/mcp-server setup`
+  removed.
+
+### Fixed
+
+- **LLM auto-resolve silent no-op** — the root user-facing bug from
+  `QA-user-findings-3.3.0-rc.6-20260421.md`. Users store their provider key
+  in `~/.openclaw/agents/<agent>/agent/auth-profiles.json`; rc.6 never looked
+  there, so every turn logged `No LLM available for auto-extraction` and
+  zero facts were extracted. 3.3.1 adds auth-profiles as tier 3 of the
+  cascade.
+
+- **`plugins.entries.totalreclaw.config.publicUrl` rejected** — user-documented
+  config key errored out with `invalid config: must NOT have additional
+  properties`. Schema was missing the property. Fixed in both `openclaw.plugin.json`
+  and the in-JS `configSchema`.
+
+- **`No LLM available` fires every turn** — downgraded to a single INFO log
+  at startup. Never per-turn unless the resolvable state changes. The
+  `extraction.enabled=false` path also moved from warn to info (it's a user
+  choice, not a diagnostic signal).
+
+- **Recovery-phrase-in-chat in SKILL.md** — the prior SKILL.md told the
+  agent to "run `npx @totalreclaw/mcp-server setup` to generate a
+  cryptographically valid recovery phrase… display it prominently". Any
+  compliant agent following this leaked the phrase to the LLM provider's
+  logging path. Removed entirely and replaced with an explicit prohibition
+  + pointer to CLI flows.
+
+### Preserved from rc.2–rc.6
+
+- rc.2 scanner-comment isolation (fetch-word in comments rewrapped)
+- rc.4 `auth: 'plugin'` literal on HTTP routes
+- rc.4 `ensureSessionsFileDir` mkdir before lock acquire
+- rc.5 synchronous `registerHttpRoute` calls (no async IIFE)
+- rc.6 `openclaw.plugin.json` drop of `"kind": "memory"` (startup registry
+  fix; JS plugin definition still returns `kind: 'memory' as const` for
+  memory-slot matching)
+
+### Unchanged
+
+No protocol / on-chain changes vs 3.3.0. Memory Taxonomy v1 unchanged.
+Protobuf v4 unchanged. Subgraph schema unchanged. Billing cache unchanged.
+Relay API surface unchanged. No breaking changes to any public tool
+contract.
+
+---
+
 ## [3.3.0-rc.6] — 2026-04-20
 
 Sixth release candidate for 3.3.0. Single manifest-only fix for the