@totalreclaw/totalreclaw 1.6.0 → 3.0.7-rc.1

package/CLAWHUB.md

@@ -0,0 +1,134 @@
+# ClawHub Publishing
+
+Internal notes for publishing the TotalReclaw skill on [clawhub.ai](https://clawhub.ai).
+
+## Published
+
+- **Published:** 2026-03-26
+- **Version:** 1.4.0
+- **Package ID:** k978pv4b9nyrx4bt968vqyzpa983mbx3
+- **URL:** https://clawhub.ai/skills/totalreclaw
+
+---
+
+## How ClawHub Works
+
+ClawHub is the official skill registry for OpenClaw agents. Key facts:
+
+- **SKILL.md IS the listing page** — the full markdown body is rendered as the listing content
+- **Publishing is instant via CLI** — no manual review queue
+- **Automated security scan runs post-publish** — flags suspicious patterns, undeclared env vars, dynamic code execution
+- **No screenshots/video hosted on ClawHub** — embed as markdown image links in SKILL.md if desired
+- **Discovery via vector search** — embedding similarity + slug/name boosts + popularity prior
+
+---
+
+## Readiness Status
+
+### Published (v1.4.0, 2026-03-26)
+
+- [x] `skill.json` — Metadata, hooks, tools, config, and ClawHub fields populated
+- [x] `SKILL.md` — YAML frontmatter with full metadata; tools, hooks, prompts, and LLM instructions documented
+- [x] `README.md` — Public-facing documentation with quick start, benchmarks, configuration, and architecture
+- [x] Hooks defined: `before_agent_start`, `agent_end`, `pre_compaction`
+- [x] Tools defined: `totalreclaw_remember`, `totalreclaw_recall`, `totalreclaw_forget`, `totalreclaw_export`, `totalreclaw_status`, `totalreclaw_generate_recovery_phrase`
+- [x] Environment variables documented (`TOTALRECLAW_SERVER_URL`, `TOTALRECLAW_RECOVERY_PHRASE`)
+- [x] Benchmark comparison table (98.1% recall@8 with 100% privacy)
+- [x] License declared (MIT)
+- [x] Keywords and OS compatibility specified
+- [x] E2E onboarding tests passing (4/4)
+- [x] E2E subgraph tests passing (9/9)
+
+### Not Yet Done
+
+- [ ] **Screenshots** (optional — linked from external hosting as markdown images in SKILL.md)
+  - Suggested:
+    1. Agent remembering a user preference (tool call + response)
+    2. Agent recalling memories at conversation start (context injection)
+    3. Memory export in JSON format
+    4. Encryption in action (showing encrypted vs plaintext data)
+- [ ] **Demo video** (optional, 30-90 seconds)
+  - Show a full cycle: store a memory, start a new conversation, recall it automatically
+  - Highlight that the server never sees plaintext
+
+---
+
+## SKILL.md Frontmatter
+
+Ensure the SKILL.md starts with this frontmatter for ClawHub:
+
+```yaml
+---
+name: totalreclaw
+description: "End-to-end encrypted memory for AI agents — portable, yours forever. XChaCha20-Poly1305 E2EE: server never sees plaintext."
+version: 1.5.1
+author: TotalReclaw Team
+license: MIT
+homepage: https://totalreclaw.xyz
+metadata:
+  openclaw:
+    requires:
+      env: []
+      bins: []
+    emoji: "\U0001F9E0"
+    os: ["macos", "linux", "windows"]
+    keywords:
+      - memory
+      - e2ee
+      - e2e-encryption
+      - encryption
+      - privacy
+      - agent-memory
+      - persistent-context
+---
+```
+
+**Note:** `env: []` is intentional. The skill installs without any pre-configured env vars. The agent guides the user through onboarding (generating a recovery phrase) during first use.
+
+---
+
+## Publish Command
+
+```bash
+# Login
+clawhub login
+
+# Publish (update version as needed)
+clawhub publish ./skill \
+  --slug totalreclaw \
+  --name "TotalReclaw" \
+  --version 1.4.0 \
+  --tags latest,memory,encryption,e2ee,e2e-encryption,privacy,agent-memory,persistent-context \
+  --changelog "v1.4.0: ClawHub as primary install method, LLM-guided dedup, dual-chain billing, Harrier-OSS-v1-270M."
+```
+
+---
+
+## Security Scan Notes
+
+The `TOTALRECLAW_RECOVERY_PHRASE` env var will likely trigger extra scrutiny from the automated security scanner. The E2EE architecture explanation in SKILL.md should help it pass as `clean`:
+
+- The password is a 12-word BIP-39 mnemonic used to derive encryption keys
+- It never leaves the client device
+- The server only ever receives encrypted blobs
+- All crypto code is open-source and auditable
+
+---
+
+## Competitor Context
+
+ClawHub has one similar listing: **Everclaw** (also XChaCha20-Poly1305 encrypted cloud memory, ~2,959 downloads). It is flagged as `suspicious` by the moderation system. TotalReclaw differentiates by:
+
+- Fully open-source (server + client)
+- On-chain data anchoring (Gnosis Chain + The Graph)
+- Seed-phrase portability (no accounts)
+- Competitive benchmark data (98.1% recall@8)
+- Clean security scan (no suspicious patterns)
+
+---
+
+## Post-Publishing
+
+- **Version updates**: Bump version in `skill.json` and SKILL.md frontmatter, then `clawhub publish` again
+- **Monitor**: Check downloads, stars, and comments on the listing
+- **Respond to feedback**: Monitor the ClawHub listing for user comments

package/README.md

@@ -1,103 +1,446 @@
-<p align="center">
-  <img src="../../docs/assets/logo.png" alt="TotalReclaw" width="80" />
-</p>
+# TotalReclaw Skill for OpenClaw
 
-<h1 align="center">@totalreclaw/totalreclaw</h1>
+> **End-to-end encrypted memory + knowledge graph for AI agents -- portable, yours forever.**
+>
+> Your AI remembers everything. Your server sees nothing.
 
-<p align="center">
-  <strong>End-to-end encrypted memory for OpenClaw -- fully automatic, yours forever</strong>
-</p>
+TotalReclaw gives any [OpenClaw](https://github.com/openclaw/openclaw) agent persistent, encrypted long-term memory. Preferences, decisions, commitments, rules, and context carry across every conversation -- fully end-to-end encrypted so the server **never** sees plaintext.
 
-<p align="center">
-  <a href="https://totalreclaw.xyz">Website</a> &middot;
-  <a href="https://www.npmjs.com/package/@totalreclaw/totalreclaw">npm</a> &middot;
-  <a href="../../docs/guides/beta-tester-guide.md">Getting Started</a>
-</p>
+**v3.0.0 ships Memory Taxonomy v1**: every memory is typed (`claim` / `preference` / `directive` / `commitment` / `episode` / `summary`) and tagged with source, scope, and volatility. Recall uses source-weighted reranking so user-authored claims consistently rank above assistant-regurgitated noise. See [`docs/guides/memory-types-guide.md`](../../docs/guides/memory-types-guide.md).
 
-<p align="center">
-  <a href="https://www.npmjs.com/package/@totalreclaw/totalreclaw"><img src="https://img.shields.io/npm/v/@totalreclaw/totalreclaw?color=7B5CFF" alt="npm version"></a>
-  <a href="https://www.npmjs.com/package/@totalreclaw/totalreclaw"><img src="https://img.shields.io/npm/dm/@totalreclaw/totalreclaw" alt="npm downloads"></a>
-  <a href="../../LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue" alt="License"></a>
-</p>
+## Installation
+
+### ClawHub (recommended)
+
+Tell your OpenClaw agent:
+
+> "Install the TotalReclaw skill from ClawHub"
+
+Or via terminal:
+
+```bash
+openclaw skills install totalreclaw
+```
+
+Then set one environment variable:
+
+```bash
+export TOTALRECLAW_RECOVERY_PHRASE="your twelve word recovery phrase here"
+```
+
+That's it. TotalReclaw hooks into your agent automatically. The server URL defaults to `https://api.totalreclaw.xyz` (managed service) -- only set `TOTALRECLAW_SERVER_URL` if you are self-hosting. See the [env vars reference](../../docs/guides/env-vars-reference.md) for the full (short) list.
+
+### Alternative: npm
+
+```bash
+openclaw plugins install @totalreclaw/totalreclaw
+```
+
+---
+
+## Why TotalReclaw?
+
+Most AI memory solutions force a tradeoff: **good recall OR privacy**. TotalReclaw eliminates that tradeoff.
+
+| | Recall@8 | Privacy | Encryption | Portable Export |
+|---|:---:|:---:|:---:|:---:|
+| **TotalReclaw (E2EE)** | **98.1%** | **100%** | XChaCha20-Poly1305 | Yes |
+| Plaintext vector search | 99.2% | 0% | None | Varies |
+| Mem0 (hosted) | ~95% | 0% | At-rest only | No |
+| Native OpenClaw QMD | ~90% | 50% | Partial | No |
+
+**98.1% recall with 100% privacy** -- tested against 8,727 real-world memories. The server never sees your data, yet search quality is within 1.1% of plaintext alternatives.
+
+### Key Differentiators
+
+- **True end-to-end encryption**: XChaCha20-Poly1305 encryption, Argon2id key derivation, HKDF-SHA256 auth. The server is cryptographically unable to read your memories.
+- **Near-plaintext recall**: LSH blind indices with client-side BM25 + cosine + RRF reranking achieve 98.1% recall@8.
+- **No vendor lock-in**: One-click plaintext export in JSON or Markdown. Your data is always yours.
+- **Works everywhere**: Any MCP-compatible AI agent, not just OpenClaw.
+
+---
+
+## Features
+
+- **End-to-End Encryption**: XChaCha20-Poly1305 ensures the server never sees plaintext memories
+- **Memory Taxonomy v1**: 6 speech-act types + source / scope / volatility axes on every memory. [Learn more](../../docs/guides/memory-types-guide.md)
+- **Intelligent Extraction**: G-pipeline — single merged-topic LLM call, provenance filter, comparative rescoring, volatility heuristic. v1 is the only write path.
+- **Semantic Search**: LSH blind indices with client-side BM25 + cosine + RRF fusion reranking
+- **Retrieval v2 Tier 1**: Source-weighted reranking — user=1.0, user-inferred=0.9, derived/external=0.7, assistant=0.55
+- **Lifecycle Hooks**: Seamlessly integrates with OpenClaw's agent lifecycle (before_agent_start, agent_end, pre_compaction, before_reset)
+- **Natural-language overrides**: "pin that", "that was actually a rule, not a preference", "file that under work" — agent calls the right tool automatically
+- **Portable Export**: One-click plaintext export -- no vendor lock-in
+- **Decay Management**: Automatic memory decay with configurable thresholds
 
 ---
 
-Your AI agent remembers everything -- preferences, decisions, facts -- encrypted so only you can read it. Built for [OpenClaw](https://openclaw.ai) with fully automatic memory extraction and recall.
+## Quick Start
 
-## Install
+### 1. Install
 
-Ask your OpenClaw agent:
+Tell your OpenClaw agent:
 
-> "Install the @totalreclaw/totalreclaw plugin"
+> "Install the TotalReclaw skill from ClawHub"
 
-Or from the terminal:
+Or via terminal:
+
+```bash
+openclaw skills install totalreclaw
+```
+
+Alternative (npm):
 
 ```bash
 openclaw plugins install @totalreclaw/totalreclaw
 ```
 
-The agent handles setup: generates your encryption keys, asks you to save a 12-word recovery phrase, and registers you. After that, memory is fully automatic.
+### 2. Configure
 
-## How It Works
+Set one environment variable:
 
-After setup, everything happens in the background:
+```bash
+export TOTALRECLAW_RECOVERY_PHRASE="your twelve word recovery phrase here"
+```
 
-- **Start of conversation** -- loads relevant memories from your encrypted vault
-- **During conversation** -- extracts facts, preferences, and decisions automatically
-- **Before context compaction** -- saves important context before the window is trimmed
+**That's it.** v1 is the default extraction and write path. Extraction cadence, importance floor, candidate pool size, and dedup thresholds are all server-tuned via the relay's billing response -- no client env vars to set. See [env vars reference](../../docs/guides/env-vars-reference.md).
 
-All encryption happens client-side using AES-256-GCM. The server never sees your plaintext data.
+For self-hosted deployments:
+
+```bash
+export TOTALRECLAW_SERVER_URL="http://your-totalreclaw-server:8080"
+export TOTALRECLAW_SELF_HOSTED=true
+```
+
+### 3. Use
+
+Once installed, TotalReclaw hooks into your agent lifecycle automatically. No code changes needed.
+
+Your agent will:
+- **Load relevant memories** before processing each message (`before_agent_start`)
+- **Extract and store facts** after each turn (`agent_end`)
+- **Flush all memories** before context compaction (`pre_compaction`)
+
+You can also use the tools directly in conversation:
+
+```
+"Remember that I prefer dark mode in all editors"
+"What do you know about my programming preferences?"
+"Forget the memory about my old email address"
+"Export all my memories as JSON"
+```
+
+---
 
 ## Tools
 
-Your agent gets these tools automatically:
+The plugin exposes these tools to your OpenClaw agent. Most invocations happen via natural language -- the agent picks the right tool from context.
 
-| Tool | Description |
-|------|-------------|
-| `totalreclaw_remember` | Manually store a fact |
-| `totalreclaw_recall` | Search memories by natural language |
-| `totalreclaw_forget` | Delete a specific memory |
-| `totalreclaw_export` | Export all memories as plaintext |
-| `totalreclaw_status` | Check billing status and quota |
-| `totalreclaw_consolidate` | Merge duplicate memories |
-| `totalreclaw_import_from` | Import from Mem0 or MCP Memory Server |
+### totalreclaw_remember
 
-Most of the time you won't use these directly -- the automatic hooks handle memory for you.
+Explicitly store a memory. Accepts v1 taxonomy fields.
 
-## Features
+```typescript
+const result = await skill.remember({
+  text: 'User prefers dark mode',
+  type: 'preference',  // v1 types: claim, preference, directive, commitment, episode, summary
+  source: 'user',      // v1 sources: user, user-inferred, assistant, external, derived
+  scope: 'personal',   // v1 scopes: work, personal, health, family, creative, finance, misc, unspecified
+  importance: 7,       // 1-10 (see importance rubric)
+});
+
+console.log(result); // "Memory stored successfully with ID: fact-123"
+```
+
+### totalreclaw_recall
+
+Search for relevant memories.
+
+```typescript
+const memories = await skill.recall({
+  query: 'programming language preferences',
+  k: 5,  // optional: number of results (default: 8, max: 20)
+});
+
+// Each memory has:
+// - fact: The fact object with text, metadata, etc.
+// - score: Combined relevance score
+// - vectorScore: Vector similarity score
+// - textScore: BM25 text score
+// - decayAdjustedScore: Score adjusted for decay
+```
+
+### totalreclaw_forget
+
+Delete a specific memory.
+
+```typescript
+await skill.forget({
+  factId: 'fact-123',
+});
+```
+
+### totalreclaw_export
+
+Export all memories for portability.
+
+```typescript
+const jsonExport = await skill.export({
+  format: 'json',  // or 'markdown'
+});
+
+console.log(jsonExport);
+```
+
+---
+
+## Lifecycle Hooks
+
+TotalReclaw integrates with OpenClaw through three lifecycle hooks:
+
+| Hook | Priority | Description |
+|------|----------|-------------|
+| `before_agent_start` | 10 | Retrieve relevant memories before agent processes message |
+| `agent_end` | 90 | Extract and store facts after agent completes turn |
+| `pre_compaction` | 5 | Full memory flush before context compaction |
+
+### before_agent_start
 
-- **End-to-end encrypted** -- AES-256-GCM encryption, blind index search, HKDF auth
-- **Automatic extraction** -- LLM extracts facts from conversations, no manual input needed
-- **Semantic search** -- Local embeddings + BM25 + cosine reranking with RRF fusion
-- **Smart dedup** -- Cosine similarity catches paraphrases; LLM-guided dedup catches contradictions (Pro)
-- **On-chain storage** -- Encrypted data stored on Gnosis Chain, indexed by The Graph
-- **Portable** -- One 12-word phrase. Any device, same memories, no lock-in
-- **Import** -- Migrate from Mem0 or MCP Memory Server
+Runs before the agent processes a user message. Retrieves relevant memories and formats them for context injection.
 
-## Free Tier & Pricing
+```typescript
+const result = await skill.onBeforeAgentStart(context);
 
-| Tier | Memories | Reads | Storage | Price |
-|------|----------|-------|---------|-------|
-| **Free** | 500/month | Unlimited | Testnet (trial) | $0 |
-| **Pro** | Unlimited | Unlimited | Permanent on-chain (Gnosis) | See `totalreclaw_status` |
+// result.memories - Array of retrieved memories
+// result.contextString - Formatted string for injection
+// result.latencyMs - Search latency in milliseconds
+```
+
+### agent_end
+
+Runs after the agent completes its turn. Extracts facts from the conversation and stores them.
+
+```typescript
+const result = await skill.onAgentEnd(context);
+
+// result.factsExtracted - Number of facts extracted
+// result.factsStored - Number of facts stored
+// result.processingTimeMs - Processing time
+```
+
+### pre_compaction
+
+Runs before conversation history is compacted. Performs comprehensive extraction of the full history.
+
+```typescript
+const result = await skill.onPreCompaction(context);
+
+// result.factsExtracted - Number of facts extracted
+// result.factsStored - Number of facts stored
+// result.duplicatesSkipped - Duplicates skipped
+// result.processingTimeMs - Processing time
+```
+
+---
+
+## Configuration
+
+### Environment Variables
+
+See [`docs/guides/env-vars-reference.md`](../../docs/guides/env-vars-reference.md)
+for the complete, authoritative list. The v1-launch cleanup reduced the
+user-facing surface to 5 vars plus LLM provider keys. The short version:
+
+| Variable | Required | Default | Description |
+|----------|:---:|---------|-------------|
+| `TOTALRECLAW_RECOVERY_PHRASE` | **Yes** | -- | 12-word BIP-39 recovery phrase (never sent to server) |
+| `TOTALRECLAW_SERVER_URL` | No | `https://api.totalreclaw.xyz` | Relay URL (override for self-hosted / staging) |
+| `TOTALRECLAW_SELF_HOSTED` | No | `false` | Set `true` if running against a self-hosted PostgreSQL server |
+| `TOTALRECLAW_CREDENTIALS_PATH` | No | `~/.totalreclaw/credentials.json` | Credential file location |
+| `TOTALRECLAW_CACHE_PATH` | No | `~/.totalreclaw/cache.enc` | Encrypted cache file location |
+
+Tuning knobs (extraction interval, importance threshold, cosine thresholds)
+now come from the relay billing response. Self-hosted operators can still
+set the env-var equivalents as fallbacks — see the env vars reference.
+
+### Configuration Sources (Priority Order)
+
+Configuration is loaded from multiple sources. Higher priority overrides lower:
+
+1. **Default values** -- Built-in defaults
+2. **OpenClaw config** -- `agents.defaults.totalreclaw.*`
+3. **Environment variables** -- `TOTALRECLAW_*`
+4. **Explicit overrides** -- Passed to constructor
+
+### OpenClaw Configuration
+
+Add to your OpenClaw configuration file:
+
+```json
+{
+  "agents": {
+    "defaults": {
+      "totalreclaw": {
+        "serverUrl": "http://your-server:8080",
+        "autoExtractEveryTurns": 3,
+        "minImportanceForAutoStore": 6,
+        "maxMemoriesInContext": 8,
+        "forgetThreshold": 0.3
+      }
+    }
+  }
+}
+```
+
+---
+
+## Memory Types
 
-Pay with card via Stripe. Use `totalreclaw_status` to check current pricing. Counter resets monthly.
+TotalReclaw categorizes memories into five types:
 
-## Using with Other Agents
+| Type | Description | Example |
+|------|-------------|---------|
+| `fact` | Objective information | "User works at Acme Corp" |
+| `preference` | User likes/dislikes | "User prefers dark mode" |
+| `decision` | Choices made | "User decided to use PostgreSQL" |
+| `episodic` | Events and experiences | "User attended PyCon 2024" |
+| `goal` | Objectives and targets | "User wants to learn Rust" |
 
-TotalReclaw also works outside OpenClaw:
+## Importance Scoring
 
-- **Claude Desktop / Cursor / Windsurf** -- Use [@totalreclaw/mcp-server](https://www.npmjs.com/package/@totalreclaw/mcp-server)
-- **NanoClaw** -- Built-in support via MCP bridge
+Memories are scored on a 1-10 scale:
 
-Same encryption, same recovery phrase, same memories across all agents.
+| Score | Level | Description |
+|-------|-------|-------------|
+| 1-3 | Trivial | Small talk, pleasantries |
+| 4-6 | Useful | Tool preferences, working style |
+| 7-8 | Important | Key decisions, major preferences |
+| 9-10 | Critical | Core values, safety info |
 
-## Learn More
+---
+
+## Encryption Details
+
+All cryptographic operations are powered by [`@totalreclaw/core`](https://www.npmjs.com/package/@totalreclaw/core) -- a unified Rust/WASM module that ensures byte-for-byte consistency across all TotalReclaw clients.
+
+TotalReclaw uses end-to-end encryption:
+
+1. **Key Derivation**: Recovery phrase is processed through Argon2id to derive encryption keys. The phrase is never sent to the server.
+2. **Encryption**: All memories are encrypted client-side using XChaCha20-Poly1305 before transmission.
+3. **Search**: LSH blind indices (SHA-256 hashed) enable server-side search without exposing plaintext.
+4. **Decryption**: Memories are decrypted client-side after retrieval.
+5. **Authentication**: HKDF-SHA256 for authentication tokens.
+
+The server is cryptographically unable to read your memories, embeddings, or search queries.
+
+---
+
+## Performance Targets
+
+| Metric | Target |
+|--------|--------|
+| Search latency (p95) | < 140ms for 1M memories |
+| Recall accuracy | >= 93% of true top-250 |
+| Storage overhead | <= 2.2x vs plaintext |
+| Extraction latency | < 500ms |
+
+---
+
+## Architecture
+
+```
++-------------------+     +-------------------+     +-------------------+
+|   OpenClaw Agent  |     |  TotalReclaw Skill |     | TotalReclaw Server |
++-------------------+     +-------------------+     +-------------------+
+        |                         |                         |
+        | onBeforeAgentStart()    |                         |
+        |------------------------>| recall()                |
+        |                         |------------------------>|
+        |                         |<------------------------|
+        |<------------------------|                         |
+        |                         |                         |
+        | [Agent processes]       |                         |
+        |                         |                         |
+        | onAgentEnd()            |                         |
+        |------------------------>| extract + store()       |
+        |                         |------------------------>|
+        |<------------------------|                         |
+```
+
+---
 
-- [Getting Started Guide](../../docs/guides/beta-tester-guide.md)
-- [totalreclaw.xyz](https://totalreclaw.xyz)
-- [Main Repository](https://github.com/p-diogo/totalreclaw)
+## Troubleshooting
+
+### "Skill not initialized"
+
+Call `await skill.init()` before using any methods.
+
+### "Failed to load reranker model"
+
+The reranker model is optional. If not found, vector scores are used as fallback.
+
+### "Memory not found"
+
+The fact ID may be incorrect, or the memory may have been evicted due to decay.
+
+### Slow searches
+
+- Ensure the TotalReclaw server is properly indexed
+- Check network latency to the server
+- Consider increasing `maxMemoriesInContext` for better recall
+
+---
+
+## Development
+
+### Setup
+
+```bash
+git clone https://github.com/p-diogo/totalreclaw
+cd totalreclaw/skill
+npm install
+```
+
+### Build
+
+```bash
+npm run build
+```
+
+### Test
+
+```bash
+npm test
+
+# With coverage
+npm test -- --coverage
+
+# Watch mode
+npm run test:watch
+```
+
+### Lint
+
+```bash
+npm run lint
+```
+
+---
+
+## Contributing
+
+1. Fork the repository
+2. Create a feature branch: `git checkout -b feature/my-feature`
+3. Commit changes: `git commit -am 'Add my feature'`
+4. Push to the branch: `git push origin feature/my-feature`
+5. Submit a pull request
 
 ## License
 
-MIT
+MIT License - see [LICENSE](LICENSE) for details.
+
+## Links
+
+- [TotalReclaw Documentation](https://github.com/p-diogo/totalreclaw)
+- [Claw Hub Listing](https://clawhub.ai/skills/totalreclaw)
+- [OpenClaw](https://github.com/openclaw/openclaw)
+- [Issue Tracker](https://github.com/p-diogo/totalreclaw/issues)