clawvault 3.4.1 → 3.5.0

package/docs/getting-started/installation.md

@@ -0,0 +1,99 @@
+# Installation
+
+This guide covers the recommended install flow for ClawVault on Linux (Ubuntu), macOS, Windows, and WSL.
+
+## System requirements
+
+- Node.js 18+ (Node.js 22+ recommended)
+- npm 9+
+- Supported OS: Linux, macOS, Windows, WSL
+
+## Install the CLI
+
+```bash
+npm install -g clawvault
+```
+
+`qmd` is optional. ClawVault ships with an in-process BM25 search engine by default. Install `qmd` only if you want qmd fallback behavior:
+
+```bash
+bun install -g github:tobi/qmd
+```
+
+## OpenClaw plugin setup (new architecture)
+
+ClawVault is loaded as a plugin package (not the deprecated hook install/enable flow).
+
+1. Add ClawVault's package path to `plugins.load.paths` in `openclaw.json`.
+2. Enable the plugin entry and memory slot:
+   - `plugins.slots.memory: "clawvault"`
+   - `plugins.entries.clawvault.enabled: true`
+3. Configure plugin behavior under `plugins.entries.clawvault.config` (for example `vaultPath`, context/observation toggles, and protocol settings).
+
+For complete configuration and hook behavior details, see [OpenClaw Plugin Usage](../openclaw-plugin-usage.md).
+
+## Quick verification
+
+After installation, run:
+
+```bash
+clawvault doctor
+```
+
+This checks your Node/npm environment, vault/config health, search setup, OpenClaw integration, and common Linux permission issues.
+
+## Linux (Ubuntu 22.04 / 24.04) setup
+
+### 1) Install Node.js and npm
+
+Use your preferred version manager (nvm/fnm/asdf) and install Node.js 22 LTS. Validate:
+
+```bash
+node -v
+npm -v
+```
+
+### 2) Configure npm global prefix (if you hit EACCES)
+
+If `npm install -g clawvault` fails with permissions errors:
+
+```bash
+npm config set prefix ~/.npm-global
+```
+
+### 3) Add npm global bin directory to PATH
+
+Append this to `~/.bashrc` (or `~/.zshrc`):
+
+```bash
+export PATH="$HOME/.npm-global/bin:$PATH"
+```
+
+Reload your shell:
+
+```bash
+source ~/.bashrc
+```
+
+### 4) Re-run install and verify
+
+```bash
+npm install -g clawvault
+clawvault doctor
+```
+
+## Troubleshooting quick fixes
+
+- `clawvault: command not found`
+  - Ensure your npm global bin directory is in PATH.
+  - Run `npm config get prefix` and confirm `<prefix>/bin` is exported.
+- Global install fails with `EACCES`
+  - Set user-owned npm prefix: `npm config set prefix ~/.npm-global`
+  - Re-open terminal and retry install.
+- `qmd` not found
+  - This is optional. In-process BM25 search still works.
+  - Install qmd only if you need fallback compatibility paths.
+- OpenClaw plugin not registered
+  - Verify `plugins.load.paths` includes the ClawVault package path.
+  - Verify `plugins.slots.memory` is `clawvault`.
+  - Verify `plugins.entries.clawvault.enabled` is `true`.

package/docs/openclaw-plugin-usage.md

@@ -0,0 +1,152 @@
+# OpenClaw Plugin Usage Guide
+
+ClawVault now runs as a proper OpenClaw plugin via `plugins.load.paths` and `plugins.entries.clawvault`.
+The old `openclaw hooks install/enable` flow is deprecated.
+
+## Installation (New Plugin System)
+
+```bash
+# Install ClawVault globally
+npm install -g clawvault
+
+# ClawVault auto-registers as an OpenClaw plugin.
+# Add to plugins.load.paths in openclaw.json:
+# /path/to/node_modules/clawvault
+
+# Enable in config:
+# plugins.slots.memory: "clawvault"
+# plugins.entries.clawvault.enabled: true
+```
+
+Tip: on most setups, you can discover the install path with `npm root -g`.
+
+## Plugin Configuration
+
+All plugin configuration lives under `plugins.entries.clawvault.config`:
+
+```json
+{
+  "plugins": {
+    "slots": { "memory": "clawvault" },
+    "entries": {
+      "clawvault": {
+        "enabled": true,
+        "config": {
+          "vaultPath": "/path/to/memory",
+          "allowClawvaultExec": true,
+          "allowEnvAccess": true,
+          "enableStartupRecovery": true,
+          "enableSessionContextInjection": true,
+          "enableAutoCheckpoint": true,
+          "enableObserveOnNew": true,
+          "enableHeartbeatObservation": true,
+          "enableCompactionObservation": true,
+          "enableBeforePromptRecall": true,
+          "enforceCommunicationProtocol": true,
+          "enableMessageSendingFilter": true,
+          "enableFactExtraction": true,
+          "contextProfile": "auto",
+          "maxContextResults": 5
+        }
+      }
+    }
+  }
+}
+```
+
+## What the Plugin Does (Hook by Hook)
+
+1. **`before_prompt_build`**
+   - Injects ClawVault context and recent session recap.
+   - Adds a memory recall mandate so the agent uses `memory_search`/`memory_get` before memory-sensitive answers.
+   - Appends communication protocol rules.
+   - Runs every turn.
+
+2. **`message_sending`**
+   - Filters outbound phrasing and removes banned patterns (for example: "good catch", "great question").
+   - Rewrites rabbit-hole offers (for example: "if you'd like I can ...").
+   - If the response is a question and memory already has the answer, rewrites toward evidence-grounded statements.
+
+3. **`gateway_start`**
+   - Runs startup recovery.
+   - Detects context death and stores a recovery notice for next prompt injection.
+
+4. **`session_start`**
+   - Fetches session recap entries for context injection.
+   - Triggers weekly reflection when due.
+
+5. **`session_end`**
+   - Clears session-specific runtime state.
+
+6. **`before_reset`**
+   - Auto-checkpoints before `/new` when enabled.
+   - Runs observer flow when enabled.
+   - Extracts facts when enabled.
+
+7. **`before_compaction`**
+   - Runs observation before context compaction when enabled.
+   - Optionally extracts facts.
+
+8. **`agent_end`**
+   - Runs heartbeat observation after agent turns when enabled.
+
+## Registered Tools
+
+The plugin registers two tools:
+
+- **`memory_search`**
+  - Search the vault using:
+  - `query` (required)
+  - `maxResults` (optional)
+  - `minScore` (optional)
+  - `sessionKey` (optional)
+
+- **`memory_get`**
+  - Read vault files using:
+  - `path` or `relPath` (either accepted)
+  - `from` (optional starting line)
+  - `lines` (optional number of lines)
+
+## Communication Protocol
+
+When enabled, ClawVault enforces protocol-safe messaging:
+
+- Removes banned phrases:
+  - "good catch"
+  - "great question"
+  - "you're right to call that out"
+- Removes rabbit-hole offers like:
+  - "if you'd like I can ..."
+  - "let me know if you'd like ..."
+- Avoids asking questions when memory already contains the answer by rewriting to evidence-backed output.
+
+## MEMORY.md vs Vault: Understanding the Relationship
+
+You may have both a workspace `MEMORY.md` and a ClawVault vault. They are complementary, not competing.
+
+| Layer | Purpose | Access Pattern |
+|-------|---------|----------------|
+| **MEMORY.md** | Boot-time executive summary | Immediate, no tool calls |
+| **Vault** | Full structured memory system | Retrieved via tools/CLI/injection |
+
+Recommended pattern:
+
+1. Keep `MEMORY.md` short and curated (identity, key decisions, active focus).
+2. Treat the vault as source of truth for detailed records and history.
+3. Reference vault notes from `MEMORY.md` instead of duplicating full reasoning.
+4. Periodically refresh `MEMORY.md` from vault state to prevent drift.
+
+## Troubleshooting
+
+- Plugin not loading:
+  - Verify `plugins.load.paths` includes the ClawVault package path.
+  - Verify `plugins.entries.clawvault.enabled` is `true`.
+  - Verify `plugins.slots.memory` is set to `"clawvault"`.
+- Context/tool behavior not appearing:
+  - Confirm feature flags under `plugins.entries.clawvault.config`.
+  - Run `clawvault compat` to validate runtime assumptions.
+
+## Related Documentation
+
+- [README: OpenClaw Integration](../README.md#openclaw-integration)
+- [Installation Guide](./getting-started/installation.md)

package/openclaw.plugin.json

@@ -1,7 +1,7 @@
 {
   "id": "clawvault",
   "name": "ClawVault",
-  "version": "2.7.0",
+  "version": "3.5.0",
   "kind": "memory",
   "description": "Structured memory system for AI agents with context death resilience",
   "configSchema": {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "clawvault",
-  "version": "3.4.1",
+  "version": "3.5.0",
   "description": "Structured memory system for AI agents — typed storage, knowledge graph, context profiles, canvas dashboards, neural graph themes, and Obsidian-native task views. An elephant never forgets. 🐘",
   "type": "module",
   "main": "dist/index.cjs",
@@ -18,19 +18,37 @@
   },
   "files": [
     "dist",
-    "bin",
-    "dashboard",
+    "bin/clawvault.js",
+    "bin/command-runtime.js",
+    "bin/register-core-commands.js",
+    "bin/register-query-commands.js",
+    "bin/register-session-lifecycle-commands.js",
+    "bin/register-template-commands.js",
+    "bin/register-maintenance-commands.js",
+    "bin/register-resilience-commands.js",
+    "bin/register-vault-operations-commands.js",
+    "bin/register-config-commands.js",
+    "bin/register-route-commands.js",
+    "bin/register-kanban-commands.js",
+    "bin/register-project-commands.js",
+    "bin/register-task-commands.js",
+    "bin/register-tailscale-commands.js",
+    "docs",
+    "dashboard/server.js",
+    "dashboard/public",
+    "dashboard/lib/vault-parser.js",
+    "dashboard/lib/graph-diff.js",
     "templates",
-    "hooks",
-    "openclaw.plugin.json"
+    "openclaw.plugin.json",
+    "SKILL.md",
+    "CHANGELOG.md",
+    "README.md",
+    "LICENSE"
   ],
   "openclaw": {
     "plugin": "./openclaw.plugin.json",
     "extensions": [
       "./dist/openclaw-plugin.js"
-    ],
-    "hooks": [
-      "./hooks/clawvault"
     ]
   },
   "scripts": {

package/bin/command-registration.test.js

@@ -1,179 +0,0 @@
-import { describe, expect, it } from 'vitest';
-import { Command } from 'commander';
-import * as fs from 'fs';
-import * as path from 'path';
-import { registerCoreCommands } from './register-core-commands.js';
-import { registerMaintenanceCommands } from './register-maintenance-commands.js';
-import { registerQueryCommands } from './register-query-commands.js';
-import { registerResilienceCommands } from './register-resilience-commands.js';
-import { registerSessionLifecycleCommands } from './register-session-lifecycle-commands.js';
-import { registerTemplateCommands } from './register-template-commands.js';
-import { registerVaultOperationsCommands } from './register-vault-operations-commands.js';
-import { registerConfigCommands } from './register-config-commands.js';
-import { registerRouteCommands } from './register-route-commands.js';
-import {
-  chalkStub,
-  createGetVaultStub,
-  registerAllCommandModules,
-  stubResolveVaultPath
-} from './test-helpers/cli-command-fixtures.js';
-
-function listCommandNames(program) {
-  return program.commands.map((command) => command.name()).sort((a, b) => a.localeCompare(b));
-}
-
-describe('CLI command registration modules', () => {
-  it('registers core lifecycle commands', () => {
-    const program = new Command();
-    registerCoreCommands(program, {
-      chalk: chalkStub,
-      path,
-      fs,
-      createVault: async () => ({ getCategories: () => [], getQmdRoot: () => '', getQmdCollection: () => '' }),
-      getVault: createGetVaultStub({ store: async () => ({}), capture: async () => ({}) }),
-      runQmd: async () => {}
-    });
-
-    const names = listCommandNames(program);
-    expect(names).toEqual(expect.arrayContaining(['init', 'setup', 'store', 'patch', 'capture', 'inbox']));
-  });
-
-  it('registers query commands with profile option', () => {
-    const program = new Command();
-    registerQueryCommands(program, {
-      chalk: chalkStub,
-      getVault: createGetVaultStub({ find: async () => [], vsearch: async () => [] }),
-      resolveVaultPath: stubResolveVaultPath,
-      QmdUnavailableError: class extends Error {},
-      printQmdMissing: () => {}
-    });
-
-    const names = listCommandNames(program);
-    expect(names).toEqual(expect.arrayContaining(['search', 'vsearch', 'context', 'recall', 'inject', 'observe', 'reflect', 'session-recap']));
-
-    const contextCommand = program.commands.find((command) => command.name() === 'context');
-    const profileOption = contextCommand?.options.find((option) => option.flags.includes('--profile <profile>'));
-    expect(profileOption?.description).toContain('auto');
-  });
-
-  it('registers vault operation commands', () => {
-    const program = new Command();
-    registerVaultOperationsCommands(program, {
-      chalk: chalkStub,
-      fs,
-      getVault: createGetVaultStub({
-        list: async () => [],
-        get: async () => null,
-        stats: async () => ({ tags: [], categories: {} }),
-        sync: async () => ({ copied: [], deleted: [], unchanged: [], errors: [] }),
-        reindex: async () => 0,
-        remember: async () => ({ id: '' }),
-        getQmdCollection: () => ''
-      }),
-      runQmd: async () => {},
-      resolveVaultPath: stubResolveVaultPath,
-      path
-    });
-
-    const names = listCommandNames(program);
-    expect(names).toEqual(expect.arrayContaining([
-      'list',
-      'get',
-      'stats',
-      'sync',
-      'reindex',
-      'remember',
-      'shell-init',
-      'dashboard'
-    ]));
-  });
-
-  it('registers maintenance, resilience, session-lifecycle and template commands', () => {
-    const program = new Command();
-    registerMaintenanceCommands(program, { chalk: chalkStub });
-    registerResilienceCommands(program, {
-      chalk: chalkStub,
-      resolveVaultPath: stubResolveVaultPath
-    });
-    registerSessionLifecycleCommands(program, {
-      chalk: chalkStub,
-      resolveVaultPath: stubResolveVaultPath,
-      QmdUnavailableError: class extends Error {},
-      printQmdMissing: () => {},
-      getVault: createGetVaultStub({
-        createHandoff: async () => ({ id: '', path: '' }),
-        getQmdCollection: () => '',
-        generateRecap: async () => ({}),
-        formatRecap: () => ''
-      }),
-      runQmd: async () => {}
-    });
-    registerTemplateCommands(program, { chalk: chalkStub });
-    registerConfigCommands(program, {
-      chalk: chalkStub,
-      resolveVaultPath: stubResolveVaultPath
-    });
-    registerRouteCommands(program, {
-      chalk: chalkStub,
-      resolveVaultPath: stubResolveVaultPath
-    });
-
-    const names = listCommandNames(program);
-    expect(names).toEqual(expect.arrayContaining([
-      'doctor',
-      'benchmark',
-      'maintain',
-      'embed',
-      'compat',
-      'graph',
-      'entities',
-      'entity',
-      'link',
-      'rebuild',
-      'archive',
-      'migrate-observations',
-      'replay',
-      'sync-bd',
-      'checkpoint',
-      'recover',
-      'status',
-      'clean-exit',
-      'repair-session',
-      'wake',
-      'sleep',
-      'handoff',
-      'recap',
-      'template',
-      'config',
-      'route'
-    ]));
-
-    const templateCommand = program.commands.find((command) => command.name() === 'template');
-    const templateSubcommands = templateCommand?.commands.map((command) => command.name()) ?? [];
-    expect(templateSubcommands).toEqual(expect.arrayContaining(['list', 'create', 'add']));
-
-    const compatCommand = program.commands.find((command) => command.name() === 'compat');
-    const strictOption = compatCommand?.options.find((option) => option.flags.includes('--strict'));
-    const baseDirOption = compatCommand?.options.find((option) => option.flags.includes('--base-dir <path>'));
-    expect(strictOption).toBeTruthy();
-    expect(baseDirOption).toBeTruthy();
-  });
-
-  it('keeps top-level command names unique when modules are combined', () => {
-    const program = registerAllCommandModules(new Command());
-
-    const names = program.commands.map((command) => command.name());
-    const unique = new Set(names);
-    expect(unique.size).toBe(names.length);
-  });
-
-  it('does not register removed workgraph commands', () => {
-    const program = registerAllCommandModules(new Command());
-    const names = listCommandNames(program);
-
-    expect(names).not.toContain('wg');
-    expect(names).not.toContain('thread');
-    expect(names).not.toContain('primitive');
-    expect(names).not.toContain('ledger');
-  });
-});

package/bin/command-runtime.test.js

@@ -1,154 +0,0 @@
-import { beforeEach, describe, expect, it, vi } from 'vitest';
-import path from 'path';
-
-const {
-  spawnMock,
-  resolveConfiguredVaultPathMock,
-  clawvaultCtorMock,
-  loadMock
-} = vi.hoisted(() => ({
-  spawnMock: vi.fn(),
-  resolveConfiguredVaultPathMock: vi.fn(),
-  clawvaultCtorMock: vi.fn(),
-  loadMock: vi.fn()
-}));
-
-vi.mock('child_process', () => ({
-  spawn: spawnMock
-}));
-
-vi.mock('../dist/index.js', () => ({
-  ClawVault: clawvaultCtorMock,
-  resolveVaultPath: resolveConfiguredVaultPathMock,
-  QmdUnavailableError: class QmdUnavailableError extends Error {},
-  QMD_INSTALL_COMMAND: 'install-qmd'
-}));
-
-async function loadRuntimeModule() {
-  vi.resetModules();
-  return await import('./command-runtime.js');
-}
-
-beforeEach(() => {
-  vi.clearAllMocks();
-  clawvaultCtorMock.mockImplementation(() => ({
-    load: loadMock
-  }));
-});
-
-describe('command runtime helpers', () => {
-  it('delegates vault path resolution and loads vaults', async () => {
-    resolveConfiguredVaultPathMock.mockReturnValue('/resolved/vault');
-    loadMock.mockResolvedValue(undefined);
-    const { getVault, resolveVaultPath } = await loadRuntimeModule();
-
-    const resolved = resolveVaultPath('/explicit');
-    expect(resolveConfiguredVaultPathMock).toHaveBeenCalledWith({ explicitPath: '/explicit' });
-    expect(resolved).toBe('/resolved/vault');
-
-    await getVault('/explicit');
-    expect(clawvaultCtorMock).toHaveBeenCalledWith('/resolved/vault');
-    expect(loadMock).toHaveBeenCalled();
-  });
-
-  it('maps qmd ENOENT failures to QmdUnavailableError', async () => {
-    const { runQmd, QmdUnavailableError } = await loadRuntimeModule();
-    spawnMock.mockImplementation(() => {
-      const handlers = {};
-      const proc = {
-        on: (event, handler) => {
-          handlers[event] = handler;
-        }
-      };
-      queueMicrotask(() => {
-        handlers.error?.({ code: 'ENOENT' });
-      });
-      return proc;
-    });
-
-    await expect(runQmd(['update'])).rejects.toBeInstanceOf(QmdUnavailableError);
-  });
-
-  it('injects qmd index from environment when configured', async () => {
-    const previous = process.env.CLAWVAULT_QMD_INDEX;
-    process.env.CLAWVAULT_QMD_INDEX = 'clawvault-test';
-
-    try {
-      const { runQmd } = await loadRuntimeModule();
-      spawnMock.mockImplementation((_command, _args) => {
-        const handlers = {};
-        const proc = {
-          on: (event, handler) => {
-            handlers[event] = handler;
-          }
-        };
-        queueMicrotask(() => {
-          handlers.close?.(0);
-        });
-        return proc;
-      });
-
-      await runQmd(['update']);
-      expect(spawnMock).toHaveBeenCalledWith(
-        'qmd',
-        ['--index', 'clawvault-test', 'update'],
-        { stdio: 'inherit' }
-      );
-    } finally {
-      if (previous === undefined) {
-        delete process.env.CLAWVAULT_QMD_INDEX;
-      } else {
-        process.env.CLAWVAULT_QMD_INDEX = previous;
-      }
-    }
-  });
-
-  it('surfaces qmd non-zero exit codes as errors', async () => {
-    const { runQmd } = await loadRuntimeModule();
-    spawnMock.mockImplementation(() => {
-      const handlers = {};
-      const proc = {
-        on: (event, handler) => {
-          handlers[event] = handler;
-        }
-      };
-      queueMicrotask(() => {
-        handlers.close?.(2);
-      });
-      return proc;
-    });
-
-    await expect(runQmd(['update'])).rejects.toThrow('qmd exited with code 2');
-  });
-
-  it('exports and enforces argument/path sanitization helpers', async () => {
-    const { sanitizeQmdArg, validatePathWithinBase } = await loadRuntimeModule();
-    expect(sanitizeQmdArg('update')).toBe('update');
-    expect(sanitizeQmdArg(42)).toBe('42');
-    expect(() => sanitizeQmdArg('bad\0arg')).toThrow('contains null byte');
-
-    const safePath = validatePathWithinBase('notes/today.md', '/tmp/vault');
-    expect(safePath).toBe(path.resolve('/tmp/vault', 'notes/today.md'));
-    expect(() => validatePathWithinBase('../etc/passwd', '/tmp/vault')).toThrow('Path traversal detected');
-  });
-
-  it('rejects qmd args with null-byte injection attempts', async () => {
-    const { runQmd } = await loadRuntimeModule();
-    await expect(runQmd(['up\0date'])).rejects.toThrow('contains null byte');
-    expect(spawnMock).not.toHaveBeenCalled();
-  });
-
-  it('prints consistent qmd missing guidance', async () => {
-    const errorSpy = vi.spyOn(console, 'error').mockImplementation(() => {});
-    const logSpy = vi.spyOn(console, 'log').mockImplementation(() => {});
-    try {
-      const { printQmdMissing } = await loadRuntimeModule();
-      printQmdMissing();
-      expect(errorSpy).toHaveBeenCalledWith(expect.stringContaining('ClawVault requires qmd.'));
-      expect(logSpy).toHaveBeenCalledWith(expect.stringContaining('install-qmd'));
-    } finally {
-      errorSpy.mockRestore();
-      logSpy.mockRestore();
-    }
-  });
-});