@unblocklabs/unblock-memory 0.3.10 → 0.3.11

package/README.md

@@ -2,7 +2,7 @@
 
 Workspace-native memory for OpenClaw, powered internally by `@unblocklabs/qmd`.
 It keeps one warm QMD store per agent and exposes the standard `memory_search`
-and `memory_get` tools. Search uses semantic chunking v2 and direct QMD vector
+and `memory_get` tools. Search uses semantic chunking and direct QMD vector
 search without query expansion or a reranker, so only the embedding model loads.
 
 Optional memory analysis uses those same stored vectors in the same SQLite
@@ -55,6 +55,7 @@ directories, or globs into named corpora:
               name: "sessions",
               kind: "sessions",
               chatTypes: ["channel", "group"],
+              maxExpandedTokens: 500,
             },
             {
               name: "knowledge",
@@ -237,17 +238,22 @@ eligible.
 
 The optional `sessions` corpus reads the current agent's normal OpenClaw SQLite
 store and indexes its active user/assistant transcript branch. It defaults to
-channel and group conversations; add `direct` explicitly to include DMs. Run
+channel and group conversations; add `direct` explicitly to include DMs. A
+session vector hit expands to its complete user/assistant turn when the turn
+fits `maxExpandedTokens`, or to its complete enclosing message when only that
+fits. The default is `500`; the original semantic chunk is preserved when
+neither complete context fits, so expansion never clips the matched evidence.
+Run
 `memory_sync_sessions` to start a refresh, then use `memory_sync_status` to
 check its progress or result. The read-only adapter explicitly supports OpenClaw
 agent database schemas 17, 18, and 19 and validates its required columns before
 reading. Projections are private derived Markdown under the
 agent's `unblock-memory/sessions` state directory and can be rebuilt from
 OpenClaw at any time. Their embedded text contains only `# Transcript` and
-timestamped speaker messages; filtering metadata remains in the session
-manifest. The projected file modification time matches the session start time
-for meaningful chronological cluster reads. Session results include provider,
-chat type, conversation identity, and start time as an ISO 8601 timestamp. They
+role-labeled, timestamped speaker messages; filtering metadata remains in the
+session manifest. The projected file modification time matches the session
+start time for meaningful chronological cluster reads. Session results include
+provider, chat type, conversation identity, and start time as an ISO 8601 timestamp. They
 participate in the same search and clustering index as file memory. This phase
 does not sync sessions at startup or on a schedule; refreshes are manual through
 `memory_sync_sessions`.
@@ -262,7 +268,7 @@ Markdown filesystem changes queue a debounced, serialized background refresh.
 
 Analysis is opt-in. Core indexing, `memory_search`, and `memory_get` need only
 Unblock Memory and its automatically installed QMD dependency. To enable
-clustering, install the
+clustering, install the public
 [`unblock-cluster`](https://github.com/unblocklabs-ai/unblock-cluster) worker once
 on the same host:
 
@@ -270,7 +276,7 @@ on the same host:
 git clone https://github.com/unblocklabs-ai/unblock-cluster.git
 cd unblock-cluster
 python3 -m venv .venv
-.venv/bin/python -m pip install -r requirements.txt
+.venv/bin/python -m pip install -r requirements-analysis.txt
 ```
 
 Set `analysis.executable` to the absolute path of
@@ -286,11 +292,12 @@ analyzed and `memory_recluster` reports that analysis is unavailable. Ordinary
 memory search and reads continue to work.
 
 The analysis worker reads QMD's existing semantic vectors and writes only
-derived results into three namespaced tables in that same `index.sqlite`:
+derived results into four namespaced tables in that same `index.sqlite`:
 
 - `memory_analysis_runs`
 - `memory_analysis_clusters`
 - `memory_analysis_memberships`
+- `memory_analysis_duplicate_occurrences`
 
 Unblock Memory exposes:
 

package/dist/src/config.d.ts

@@ -14,6 +14,7 @@ type SessionCorpusConfig = {
     name: "sessions";
     kind: "sessions";
     chatTypes: readonly ChatType[];
+    maxExpandedTokens: number;
 };
 export type CorpusConfig = FileCorpusConfig | SkillCorpusConfig | SessionCorpusConfig;
 export declare const DEFAULT_CORPORA: readonly FileCorpusConfig[];

package/dist/src/config.js

@@ -1,5 +1,7 @@
 import { isAbsolute } from "node:path";
 const DEFAULT_PATHS = ["MEMORY.md", "USER.md", "memory/**/*.md"];
+const DEFAULT_SESSION_MAX_EXPANDED_TOKENS = 500;
+const MAX_SESSION_MAX_EXPANDED_TOKENS = 10_000;
 const CHAT_TYPES = ["channel", "group", "direct"];
 export const DEFAULT_CORPORA = [
     {
@@ -59,7 +61,7 @@ function resolveCorpora(value) {
             return { name: "skills", kind: "skills", paths: corpus.paths.map((path) => path.trim()) };
         }
         if (corpus.kind === "sessions") {
-            assertOnlyKeys(corpus, ["name", "kind", "chatTypes"], `corpora[${index}]`);
+            assertOnlyKeys(corpus, ["name", "kind", "chatTypes", "maxExpandedTokens"], `corpora[${index}]`);
             if (name !== "sessions") {
                 throw new Error('unblock-memory session corpus must be named "sessions"');
             }
@@ -73,6 +75,7 @@ function resolveCorpora(value) {
                 name: "sessions",
                 kind: "sessions",
                 chatTypes: [...new Set(chatTypes)],
+                maxExpandedTokens: positiveInteger(corpus.maxExpandedTokens, DEFAULT_SESSION_MAX_EXPANDED_TOKENS, "corpus sessions maxExpandedTokens", MAX_SESSION_MAX_EXPANDED_TOKENS),
             };
         }
         assertOnlyKeys(corpus, ["name", "kind", "paths"], `corpora[${index}]`);

package/dist/src/manager.d.ts

@@ -1,4 +1,4 @@
-import type { QMDStore } from "@unblocklabs/qmd";
+import type { QMDStore, VectorSearchResult } from "@unblocklabs/qmd";
 import { type AnalysisRunner, type MemoryAnalysisSummary, type MemoryClusterDetail, type MemoryClusterList, type MemoryClusterSort, type MemoryReclusterOptions } from "./analysis.js";
 import type { CorpusMemorySearchResult, CorpusSearchOptions, MemoryEmbeddingProbeResult, MemoryProviderStatus, MemoryReadResult, MemoryRequestContext, MemorySearchManagerContract, MemorySyncParams } from "./contracts.js";
 import type { ChatType } from "./config.js";
@@ -10,6 +10,7 @@ export type ManagerSessionConfig = {
     agentId: string;
     agentName: string;
     chatTypes: readonly ChatType[];
+    maxExpandedTokens: number;
     collection: string;
     databasePath: string;
     manifestPath: string;
@@ -30,6 +31,10 @@ export declare function buildReadResult(params: {
     from?: number;
     lines?: number;
 }): MemoryReadResult;
+export declare function expandSessionSearchHit(result: Pick<VectorSearchResult, "body" | "bestChunk" | "chunkPos" | "chunkLen">, maxTokens: number, countTokens: (text: string) => Promise<number>): Promise<{
+    text: string;
+    position: number;
+}>;
 export declare class QmdMemoryManager implements MemorySearchManagerContract {
     #private;
     constructor(params: {

package/dist/src/manager.js

@@ -6,6 +6,7 @@ import picomatch from "picomatch";
 import { ensureMemoryAnalysisSchema, latestAnalysisCollections, latestAnalysisRunId, markMemoryAnalysisStale, readAnalysisSummary, readCluster, readClusters, runAnalysisWorker, } from "./analysis.js";
 import { CurationStore, chunkFingerprint, } from "./curation.js";
 import { readSessionManifest, sessionMetadataByPath, syncSessionProjections, } from "./session-sync.js";
+import { sessionContextSpans } from "./session-projector.js";
 import { parseSafeVirtualPath, sourceMatchesPath } from "./sources.js";
 const DEFAULT_READ_LINES = 120;
 const MAX_READ_CHARS = 12_000;
@@ -174,12 +175,27 @@ export function buildReadResult(params) {
         ...(nextFrom ? { nextFrom } : {}),
     };
 }
-function lineSpan(result) {
-    const before = result.body.slice(0, result.chunkPos);
+function lineSpan(body, position, text) {
+    const before = body.slice(0, position);
     const startLine = before.split("\n").length;
-    const endLine = startLine + Math.max(0, result.bestChunk.split("\n").length - 1);
+    const endLine = startLine + Math.max(0, text.split("\n").length - 1);
     return { startLine, endLine };
 }
+export async function expandSessionSearchHit(result, maxTokens, countTokens) {
+    const leaf = { text: result.bestChunk, position: result.chunkPos };
+    const spans = sessionContextSpans(result.body, result.chunkPos);
+    if (!spans)
+        return leaf;
+    const leafEnd = result.chunkPos + result.chunkLen;
+    for (const span of [spans.turn, spans.message]) {
+        if (span.start > result.chunkPos || span.end < leafEnd)
+            continue;
+        const text = result.body.slice(span.start, span.end).trimEnd();
+        if (await countTokens(text) <= maxTokens)
+            return { text, position: span.start };
+    }
+    return leaf;
+}
 function lexicalResult(hit, corpus, session) {
     const body = hit.body ?? hit.title;
     const endLine = Math.max(1, body.split("\n").length);
@@ -703,30 +719,36 @@ export class QmdMemoryManager {
             allowedPaths,
             expand: false,
         });
-        return hits.flatMap((hit) => {
+        const tokenizer = store.internal?.llm;
+        const results = [];
+        for (const hit of hits) {
             const collection = /^qmd:\/\/([^/]+)\//.exec(hit.file)?.[1];
             const corpus = collection ? this.#sources.get(collection)?.corpus : undefined;
             if (!corpus)
-                return [];
-            const span = lineSpan(hit);
+                continue;
             const relativePath = collection && hit.file.startsWith(`qmd://${collection}/`)
                 ? hit.file.slice(`qmd://${collection}/`.length)
                 : undefined;
             const session = corpus === "sessions" && relativePath
                 ? this.#sessionMetadata.get(relativePath)
                 : undefined;
-            return [{
-                    path: hit.file,
-                    ...span,
-                    score: hit.score,
-                    vectorScore: hit.score,
-                    snippet: hit.bestChunk,
-                    source: "memory",
-                    corpus,
-                    ...(session ? { session } : {}),
-                    citation: `${hit.displayPath}#L${span.startLine}-L${span.endLine}`,
-                }];
-        });
+            const selected = corpus === "sessions" && this.#sessions && tokenizer
+                ? await expandSessionSearchHit(hit, this.#sessions.maxExpandedTokens, (text) => tokenizer.countTokens(text))
+                : { text: hit.bestChunk, position: hit.chunkPos };
+            const span = lineSpan(hit.body, selected.position, selected.text);
+            results.push({
+                path: hit.file,
+                ...span,
+                score: hit.score,
+                vectorScore: hit.score,
+                snippet: selected.text,
+                source: "memory",
+                corpus,
+                ...(session ? { session } : {}),
+                citation: `${hit.displayPath}#L${span.startLine}-L${span.endLine}`,
+            });
+        }
+        return results;
     }
     async searchSkills(query, minScore, limit) {
         const collections = this.#skillCollectionNames();

package/dist/src/runtime.js

@@ -224,6 +224,7 @@ export class QmdMemoryRuntime {
                     agentId,
                     agentName: resolveAgentIdentity(cfg, agentId)?.name?.trim() || agentId,
                     chatTypes: sessionCorpus.chatTypes,
+                    maxExpandedTokens: sessionCorpus.maxExpandedTokens,
                     collection: sessionSource.collection,
                     databasePath: join(resolveAgentDir(cfg, agentId), "openclaw-agent.sqlite"),
                     manifestPath: join(stateDir, "sessions-manifest.json"),

package/dist/src/session-projector.d.ts

@@ -16,6 +16,17 @@ export type SessionProjectionInput = SessionMetadata & {
         createdAt: number;
     }[];
 };
+export type SessionContextSpans = {
+    message: {
+        start: number;
+        end: number;
+    };
+    turn: {
+        start: number;
+        end: number;
+    };
+};
 export declare function projectSession(input: SessionProjectionInput): string | undefined;
+export declare function sessionContextSpans(content: string, position: number): SessionContextSpans | undefined;
 export declare function sessionDocumentPath(metadata: SessionMetadata): string;
 export declare function resolveTimezone(configured?: string): string;

package/dist/src/session-projector.js

@@ -1,4 +1,5 @@
 import { createHash } from "node:crypto";
+const MESSAGE_HEADING = /^## (User|Assistant) — .* — \d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2} \S.*$/gmu;
 function record(value) {
     return value !== null && typeof value === "object" && !Array.isArray(value)
         ? value
@@ -79,6 +80,7 @@ function projectMessage(row, input) {
     if (!text)
         return undefined;
     return {
+        role,
         speaker: speaker.replace(/[\r\n]+/gu, " "),
         text,
         timestamp: timestamp(eventRecord.timestamp) ?? row.createdAt ?? timestamp(message.timestamp) ?? input.startedAt,
@@ -107,9 +109,36 @@ export function projectSession(input) {
     });
     if (messages.length === 0)
         return undefined;
-    const transcript = messages.map((message) => `${formatTimestamp(message.timestamp, input.timezone)} — ${message.speaker}: ${message.text}`);
+    const transcript = messages.map((message) => `## ${message.role === "user" ? "User" : "Assistant"} — ${message.speaker} — ` +
+        `${formatTimestamp(message.timestamp, input.timezone)}\n\n${message.text}`);
     return `# Transcript\n\n${transcript.join("\n\n")}\n`;
 }
+export function sessionContextSpans(content, position) {
+    const markers = [...content.matchAll(MESSAGE_HEADING)].map((match) => ({
+        start: match.index,
+        role: match[1] === "User" ? "user" : "assistant",
+    }));
+    const containing = markers.findLastIndex((marker) => marker.start <= position);
+    if (containing < 0)
+        return undefined;
+    const message = {
+        start: markers[containing].start,
+        end: markers[containing + 1]?.start ?? content.length,
+    };
+    let turnStart = containing;
+    while (turnStart > 0 && markers[turnStart].role !== "user")
+        turnStart -= 1;
+    if (markers[turnStart].role !== "user")
+        turnStart = containing;
+    const nextUser = markers.findIndex((marker, index) => index > turnStart && marker.role === "user");
+    return {
+        message,
+        turn: {
+            start: markers[turnStart].start,
+            end: nextUser < 0 ? content.length : markers[nextUser].start,
+        },
+    };
+}
 function hash(value) {
     return createHash("sha256").update(value).digest("hex").slice(0, 16);
 }

package/dist/src/session-sync.js

@@ -5,7 +5,7 @@ import { dirname, isAbsolute, join, relative, resolve, sep } from "node:path";
 import { DatabaseSync } from "node:sqlite";
 import { projectSession, sessionDocumentPath, } from "./session-projector.js";
 const MANIFEST_VERSION = 1;
-const PROJECTOR_VERSION = 2;
+const PROJECTOR_VERSION = 3;
 const SUPPORTED_SCHEMA_VERSIONS = new Set([17, 18, 19]);
 const REQUIRED_COLUMNS = {
     schema_meta: ["meta_key", "role", "schema_version", "agent_id", "app_version"],

package/openclaw.plugin.json

@@ -1,7 +1,7 @@
 {
   "id": "unblock-memory",
   "name": "Unblock Memory",
-  "version": "0.3.10",
+  "version": "0.3.11",
   "description": "Indexes, retrieves, and analyzes configured workspace memory with existing QMD vectors.",
   "kind": "memory",
   "activation": { "onStartup": false },
@@ -41,7 +41,7 @@
     },
     "corpora": {
       "label": "Memory Corpora",
-      "help": "Named groups of exact Markdown files, directories, or globs. Relative paths resolve from each agent workspace. Skill files use the isolated skills corpus."
+      "help": "Named groups of exact Markdown files, directories, or globs. Relative paths resolve from each agent workspace. Skill files use the isolated skills corpus. Session corpora can expand matching semantic chunks up to maxExpandedTokens."
     },
     "skillWhisperer.enabled": {
       "label": "Skill Whisperer",
@@ -99,6 +99,12 @@
                   "minItems": 1,
                   "items": { "enum": ["channel", "group", "direct"] },
                   "default": ["channel", "group"]
+                },
+                "maxExpandedTokens": {
+                  "type": "integer",
+                  "minimum": 1,
+                  "maximum": 10000,
+                  "default": 500
                 }
               }
             },

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@unblocklabs/unblock-memory",
-  "version": "0.3.10",
+  "version": "0.3.11",
   "description": "Workspace-native memory for OpenClaw, powered by QMD",
   "type": "module",
   "license": "MIT",
@@ -35,7 +35,7 @@
     "preflight": "npm run knip && npm run build && npm run typecheck && npm test && npm run plugin:inspect && npm run plugin:inspect:runtime && npm pack --dry-run"
   },
   "dependencies": {
-    "@unblocklabs/qmd": "https://github.com/unblocklabs-ai/qmd/releases/download/v2.9.3/unblocklabs-qmd-2.9.3.tgz",
+    "@unblocklabs/qmd": "https://github.com/unblocklabs-ai/qmd/releases/download/v2.9.4/unblocklabs-qmd-2.9.4.tgz",
     "chokidar": "5.0.0",
     "picomatch": "^4.0.5",
     "typebox": "1.3.6"

package/skills/memory-curator/SKILL.md

@@ -10,6 +10,20 @@ understanding of its world. A cluster shows similarity, not a complete timeline,
 truth, or consensus. Do not write from a cluster alone, and prefer no write over
 weak, duplicative, or easily looked-up knowledge.
 
+## Clusters and the knowledge corpus
+
+Clustering and knowledge are separate. `memory_recluster` analyzes every
+configured non-skill corpus, so a cluster may connect raw memory, sessions,
+meeting notes, other source material, and previously maintained knowledge.
+
+`knowledge` is not a special cluster type. It is the dedicated indexed corpus
+for the agent's maintained synthesis, normally backed by
+`knowledge/**/*.md`. Source corpora preserve what happened or was recorded;
+the knowledge corpus preserves the agent's supported current understanding
+when that understanding would be expensive to reconstruct. A cluster is only
+an invitation to investigate, not evidence that something belongs in
+knowledge.
+
 ## Investigate
 
 1. Call `memory_list_clusters`. If analysis is missing or stale, call
@@ -21,7 +35,7 @@ weak, duplicative, or easily looked-up knowledge.
    fallback when `eventTime` is unresolved.
 3. State the question the cluster raises: what may be repeated, contradictory,
    changing, or worth understanding?
-4. Search existing knowledge with `memory_search`, using
+4. Search existing maintained knowledge with `memory_search`, using
    `corpora: ["knowledge"]`. If that corpus is not configured, report that and
    do not create an unindexed file.
 5. Investigate the evidence needed to answer the question. Follow important