@memorycrystal/crystal-memory 0.7.4

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Illumin8 Inc.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,117 @@
+# crystal-memory — OpenClaw Plugin
+
+Persistent memory for AI agents. Captures conversations, extracts durable memories, and injects relevant context before every response.
+
+## Install
+
+```bash
+curl -fsSL https://memorycrystal.ai/crystal | bash
+```
+
+Or install manually from this repo:
+
+```bash
+mkdir -p ~/.openclaw/extensions/crystal-memory
+rsync -a \
+  --exclude node_modules \
+  --exclude '*.test.js' \
+  plugin/ ~/.openclaw/extensions/crystal-memory/
+
+cd ~/.openclaw/extensions/crystal-memory && npm install
+```
+
+Then enable the plugin in `~/.openclaw/openclaw.json` under `plugins.slots.memory`.
+
+## Configuration
+
+All schema-backed config is defined in `openclaw.plugin.json` under `configSchema.properties`:
+
+| Key | Type | Default | Description |
+|-----|------|---------|-------------|
+| `apiKey` | string | — | Memory Crystal API key |
+| `convexUrl` | string | `https://rightful-mockingbird-389.convex.site` | Convex backend URL |
+| `defaultRecallMode` | string | `general` | Default recall mode (`general`, `decision`, `project`, `people`, `workflow`, `conversation`) |
+| `defaultRecallLimit` | number | `8` | Memories to recall per query (`1`-`20`) |
+| `channelScope` | string | — | Namespace prefix for tenant, client, or agent isolation |
+| `localSummaryInjection` | boolean | `true` | Enable local summary injection |
+| `localSummaryMaxTokens` | number | `2000` | Max tokens for local summaries |
+
+## Files
+
+| File | Purpose |
+|------|---------|
+| `index.js` | Main plugin entry point for the modern OpenClaw plugin API |
+| `context-budget.js` | Model-aware context budget calculator |
+| `openclaw.plugin.json` | Plugin manifest and config schema |
+| `package.json` | npm metadata and optional dependencies |
+| `compaction/` | Context compaction and summarization helpers |
+| `tools/` | Local tool implementations |
+| `utils/` | Shared plugin utilities |
+| `store/` | Local SQLite-backed storage files |
+
+## Hooks
+
+The plugin registers hooks for these OpenClaw lifecycle events:
+
+- `before_agent_start` — inject wake context and relevant recall
+- `before_tool_call` — surface action-trigger warnings before risky tools
+- `before_dispatch` — rate limiting, proactive recall, and reinforcement injection
+- `message_received` — capture incoming user messages
+- `llm_output` — capture assistant responses and extract durable memories
+- `message_sent` — fallback assistant capture
+- `session_end` — clear per-session state
+
+It also watches `/new` and `/reset` command flows to trigger reflection behavior.
+
+## Knowledge Bases
+
+The plugin benefits from Knowledge Bases automatically through the same Memory Crystal backend used for recall. Use KBs for stable reference material like runbooks, policies, docs, and imported datasets while conversational memory continues to capture learned context.
+
+- Scoped knowledge bases respect the same tenant and channel boundaries as the rest of Memory Crystal.
+- KB management and direct query/import flows live on the MCP and HTTP API surfaces.
+- Plugin recall can combine durable memory with scoped KB-backed reference material when relevant.
+
+## Compaction Lifecycle
+
+Memory Crystal owns the OpenClaw context-engine compaction path and preserves context across compaction boundaries:
+
+- `before_compaction` — snapshot and checkpoint the source conversation before raw turns are condensed
+- `after_compaction` — refresh local summary state so recall remains usable after compaction completes
+
+## Procedural vs Skills
+
+- **Procedural memories** are quiet execution patterns: repeated workflows, troubleshooting loops, and operator habits that help recall without needing explicit approval.
+- **Skills** are curated artifacts promoted for deliberate agent use. Treat them as reviewed playbooks, not just ambient pattern extraction.
+
+## Tools
+
+`plugin/index.js` registers these tools directly via `api.registerTool()`:
+
+- `crystal_set_scope` — override Memory Crystal channel scope for the current session
+- `memory_search` — legacy compatibility search returning `crystal/<id>.md` paths
+- `crystal_search_messages` — search short-term conversation logs
+- `memory_get` — legacy compatibility read by memory ID or `crystal/<id>.md` path
+- `crystal_recall` — semantic search across long-term memory
+- `crystal_remember` — store a durable memory manually
+- `crystal_what_do_i_know` — topic knowledge snapshot
+- `crystal_why_did_we` — decision archaeology
+- `crystal_checkpoint` — milestone memory snapshot
+- `crystal_preflight` — pre-flight check returning relevant rules and lessons
+- `crystal_recent` — fetch recent memory-backed messages
+- `crystal_stats` — memory and usage statistics
+- `crystal_forget` — archive or delete a memory
+- `crystal_trace` — trace a memory back to its source conversation
+- `crystal_wake` — session startup briefing
+- `crystal_who_owns` — find ownership context for files, modules, or areas
+- `crystal_explain_connection` — explain relationships between concepts
+- `crystal_dependency_chain` — trace dependency chains
+
+When the local store is available, the plugin also lazily registers:
+
+- `crystal_grep` — search in-session local history and summaries
+- `crystal_describe` — inspect a local summary node
+- `crystal_expand` — expand a local summary into underlying context
+
+## Version
+
+Current: `v0.7.1`

package/capture-hook.js

@@ -0,0 +1,166 @@
+/**
+ * DEPRECATED / LEGACY
+ * -------------------
+ * This file is a legacy duplicate of capture logic now consolidated into `index.js`.
+ * It was previously used by handler.js (via child_process.spawnSync) and as a
+ * standalone hook in older OpenClaw configurations.
+ *
+ * Canonical capture logic now lives in: `index.js`
+ * Do NOT delete — may be referenced by legacy configurations or handler.js.
+ *
+ * Crystal Capture plugin — captures conversation turns via MCP API
+ * Writes to crystalMemories (sensory store) with proper userId via API key auth
+ */
+
+const DEFAULT_CONVEX_URL = "https://rightful-mockingbird-389.convex.site";
+const pendingUserMessages = new Map();
+
+function firstString(...values) {
+  for (const value of values) {
+    if (typeof value === "string" && value.trim().length > 0) {
+      return value.trim();
+    }
+  }
+  return "";
+}
+
+function joinStringArray(values) {
+  if (!Array.isArray(values)) return "";
+  return values
+    .filter((value) => typeof value === "string" && value.trim().length > 0)
+    .join("\n")
+    .trim();
+}
+
+function extractAssistantText(event) {
+  const direct = firstString(
+    joinStringArray(event?.assistantTexts),
+    joinStringArray(event?.texts),
+    joinStringArray(event?.outputs),
+    event?.lastAssistant,
+    event?.outputText,
+    event?.content,
+    event?.text,
+    event?.message?.content,
+    event?.message?.text,
+    event?.response?.content,
+    event?.response?.text,
+    event?.result?.content,
+    event?.result?.text
+  );
+
+  if (direct) {
+    return direct;
+  }
+
+  const candidates = [
+    event?.response?.messages,
+    event?.result?.messages,
+    event?.messages,
+    event?.response?.parts,
+    event?.result?.parts,
+    event?.parts,
+  ];
+
+  for (const candidate of candidates) {
+    if (!Array.isArray(candidate)) continue;
+    const text = candidate
+      .map((item) =>
+        firstString(
+          item?.content,
+          item?.text,
+          item?.message?.content,
+          item?.message?.text
+        )
+      )
+      .filter(Boolean)
+      .join("\n")
+      .trim();
+    if (text) {
+      return text;
+    }
+  }
+
+  return "";
+}
+
+function extractUserText(event) {
+  return firstString(
+    event?.context?.content,
+    event?.content,
+    event?.text,
+    event?.message?.content,
+    event?.message?.text,
+    event?.input,
+    event?.prompt
+  );
+}
+
+async function captureToMCP(apiKey, convexUrl, payload) {
+  try {
+    const res = await fetch(`${convexUrl}/api/mcp/capture`, {
+      method: "POST",
+      headers: {
+        "Authorization": `Bearer ${apiKey}`,
+        "Content-Type": "application/json",
+      },
+      body: JSON.stringify(payload),
+    });
+    return res.ok;
+  } catch {
+    return false;
+  }
+}
+
+module.exports = (api) => {
+  // Get API key from config or env
+  const getConfig = (ctx) => {
+    const apiKey =
+      ctx?.config?.apiKey ||
+      process.env.CRYSTAL_API_KEY ||
+      api.config?.apiKey;
+    const convexUrl =
+      ctx?.config?.convexUrl ||
+      process.env.CRYSTAL_CONVEX_URL ||
+      DEFAULT_CONVEX_URL;
+    return { apiKey, convexUrl };
+  };
+
+  // Capture user message before each turn
+  api.on("message_received", (event, ctx) => {
+    const text = extractUserText(event);
+    if (text && ctx?.sessionKey) {
+      pendingUserMessages.set(ctx.sessionKey, String(text));
+    }
+  });
+
+  // Fire capture after each LLM response
+  api.on("llm_output", async (event, ctx) => {
+    const assistantText = extractAssistantText(event);
+    if (!assistantText) return;
+
+    const { apiKey, convexUrl } = getConfig(ctx);
+    if (!apiKey) return;
+
+    const sessionKey = ctx?.sessionKey || "";
+    const channel = ctx?.messageProvider || "openclaw";
+    const userMessage = sessionKey ? (pendingUserMessages.get(sessionKey) || "") : "";
+    if (sessionKey) pendingUserMessages.delete(sessionKey);
+
+    const content = [
+      userMessage ? `User: ${userMessage}` : null,
+      `Assistant: ${assistantText}`,
+    ].filter(Boolean).join("\n\n");
+
+    await captureToMCP(apiKey, convexUrl, {
+      title: `Conversation — ${new Date().toISOString().slice(0, 16).replace("T", " ")}`,
+      content,
+      store: "sensory",
+      category: "conversation",
+      tags: ["openclaw", "auto-capture", channel],
+      channel,
+    });
+  });
+
+  api.logger?.info?.("[crystal] capture hooks registered (message_received + llm_output)");
+};

package/context-budget.js

@@ -0,0 +1,71 @@
+// context-budget.js — Model-aware injection budget calculator
+//
+// Memory Crystal injects context (recall results, recent messages, etc.) into
+// the agent's system prompt. This module ensures we don't blow past the model's
+// effective context capacity. Research shows effective capacity is ~60-70% of
+// advertised max, and past that hallucination climbs.
+
+const MODEL_EFFECTIVE_CAPACITY = {
+  "claude-opus": { maxTokens: 1000000, effectiveTokens: 600000, safeInjectionPct: 0.15 },
+  "claude-sonnet": { maxTokens: 1000000, effectiveTokens: 500000, safeInjectionPct: 0.15 },
+  "claude-haiku": { maxTokens: 200000, effectiveTokens: 120000, safeInjectionPct: 0.12 },
+  "gpt-5": { maxTokens: 1000000, effectiveTokens: 500000, safeInjectionPct: 0.15 },
+  "gpt-4.1": { maxTokens: 1000000, effectiveTokens: 500000, safeInjectionPct: 0.15 },
+  "gpt-4o": { maxTokens: 128000, effectiveTokens: 80000, safeInjectionPct: 0.12 },
+  "gemini-2.5-pro": { maxTokens: 1000000, effectiveTokens: 500000, safeInjectionPct: 0.15 },
+  "gemini-2.5-flash": { maxTokens: 1000000, effectiveTokens: 400000, safeInjectionPct: 0.12 },
+  "gemini-3-pro": { maxTokens: 2000000, effectiveTokens: 800000, safeInjectionPct: 0.15 },
+  "gemini-3-flash": { maxTokens: 1000000, effectiveTokens: 400000, safeInjectionPct: 0.12 },
+  codex: { maxTokens: 1000000, effectiveTokens: 500000, safeInjectionPct: 0.15 },
+  default: { maxTokens: 128000, effectiveTokens: 75000, safeInjectionPct: 0.10 },
+};
+
+function getModelCapacity(modelName) {
+  const normalized = String(modelName || "").toLowerCase();
+  for (const [key, capacity] of Object.entries(MODEL_EFFECTIVE_CAPACITY)) {
+    if (key === "default") continue;
+    if (normalized.includes(key)) return capacity;
+  }
+  return MODEL_EFFECTIVE_CAPACITY.default;
+}
+
+function getInjectionBudget(modelName) {
+  const cap = getModelCapacity(modelName);
+  const maxTokens = Math.floor(cap.effectiveTokens * cap.safeInjectionPct);
+  return {
+    maxChars: maxTokens * 4,
+    maxTokens,
+    model: modelName,
+    effectiveCapacity: cap.effectiveTokens,
+  };
+}
+
+/**
+ * Trims an array of labeled sections to fit within a character budget.
+ * Drops lowest-priority sections first.
+ *
+ * @param {Array<{label: string, text: string}>} sections - Sections to trim
+ * @param {number} maxChars - Maximum total characters
+ * @param {string[]} dropOrder - Labels ordered from lowest to highest priority
+ * @returns {Array<{label: string, text: string}>} Trimmed sections
+ */
+function trimSections(sections, maxChars, dropOrder) {
+  let totalChars = sections.reduce((sum, s) => sum + s.text.length, 0);
+  if (totalChars <= maxChars) return sections;
+
+  const result = [...sections];
+  for (const label of dropOrder) {
+    // Drop ALL sections matching this label (handles duplicate labels)
+    for (let i = result.length - 1; i >= 0; i--) {
+      if (totalChars <= maxChars) break;
+      if (result[i].label === label) {
+        totalChars -= result[i].text.length;
+        result.splice(i, 1);
+      }
+    }
+    if (totalChars <= maxChars) break;
+  }
+  return result;
+}
+
+module.exports = { MODEL_EFFECTIVE_CAPACITY, getModelCapacity, getInjectionBudget, trimSections };

package/context-budget.test.js

@@ -0,0 +1,92 @@
+const { getModelCapacity, getInjectionBudget, trimSections } = require("./context-budget");
+
+let passed = 0;
+let failed = 0;
+
+function test(name, fn) {
+  try {
+    fn();
+    passed++;
+    console.log(`  PASS: ${name}`);
+  } catch (err) {
+    failed++;
+    console.error(`  FAIL: ${name} — ${err.message}`);
+  }
+}
+
+function assert(condition, msg) {
+  if (!condition) throw new Error(msg || "assertion failed");
+}
+
+console.log("context-budget tests:");
+
+test("getInjectionBudget for claude-opus returns opus capacity", () => {
+  const budget = getInjectionBudget("claude-opus-4-6");
+  assert(budget.maxTokens === Math.floor(600000 * 0.15), `expected ${Math.floor(600000 * 0.15)}, got ${budget.maxTokens}`);
+  assert(budget.effectiveCapacity === 600000, `expected 600000, got ${budget.effectiveCapacity}`);
+});
+
+test("getInjectionBudget for gpt-4o returns gpt-4o capacity", () => {
+  const budget = getInjectionBudget("gpt-4o-mini");
+  assert(budget.maxTokens === Math.floor(80000 * 0.12), `expected ${Math.floor(80000 * 0.12)}, got ${budget.maxTokens}`);
+  assert(budget.effectiveCapacity === 80000);
+});
+
+test("getInjectionBudget for unknown model returns default", () => {
+  const budget = getInjectionBudget("unknown-model-xyz");
+  assert(budget.effectiveCapacity === 75000, `expected 75000, got ${budget.effectiveCapacity}`);
+  assert(budget.maxTokens === Math.floor(75000 * 0.10));
+});
+
+test("getInjectionBudget for empty string returns default", () => {
+  const budget = getInjectionBudget("");
+  assert(budget.effectiveCapacity === 75000);
+});
+
+test("128K model budget is smaller than 1M model budget", () => {
+  const small = getInjectionBudget("gpt-4o");
+  const large = getInjectionBudget("claude-opus-4");
+  assert(small.maxChars < large.maxChars, `${small.maxChars} should be < ${large.maxChars}`);
+});
+
+test("getModelCapacity matches partial model names", () => {
+  const opus = getModelCapacity("anthropic/claude-opus-4-6");
+  assert(opus.effectiveTokens === 600000, `expected 600000, got ${opus.effectiveTokens}`);
+
+  const codex = getModelCapacity("openai-codex/gpt-5.3-codex");
+  // Should match gpt-5 or codex
+  assert(codex.effectiveTokens >= 500000, `expected >= 500000, got ${codex.effectiveTokens}`);
+});
+
+test("trimSections returns all sections when under budget", () => {
+  const sections = [
+    { label: "A", text: "hello" },
+    { label: "B", text: "world" },
+  ];
+  const result = trimSections(sections, 1000, ["A", "B"]);
+  assert(result.length === 2);
+});
+
+test("trimSections drops lowest-priority first", () => {
+  const sections = [
+    { label: "Recent Context", text: "x".repeat(500) },
+    { label: "Relevant Recall", text: "y".repeat(500) },
+  ];
+  const result = trimSections(sections, 600, ["Recent Context", "Relevant Recall"]);
+  assert(result.length === 1, `expected 1, got ${result.length}`);
+  assert(result[0].label === "Relevant Recall", `expected Relevant Recall, got ${result[0].label}`);
+});
+
+test("trimSections drops multiple sections if needed", () => {
+  const sections = [
+    { label: "A", text: "x".repeat(300) },
+    { label: "B", text: "y".repeat(300) },
+    { label: "C", text: "z".repeat(300) },
+  ];
+  const result = trimSections(sections, 350, ["A", "B", "C"]);
+  assert(result.length === 1, `expected 1, got ${result.length}`);
+  assert(result[0].label === "C");
+});
+
+console.log(`\n${passed} passed, ${failed} failed`);
+if (failed > 0) process.exit(1);