@cairnkeep/cli 2.22.5 → 2.23.1

package/docs/memory-retrieval.md

@@ -0,0 +1,174 @@
+# Memory retrieval and the Markdown baseline
+
+Status: available since Cairnkeep 2.23.0 as an opt-in strategy. Existing search
+defaults are unchanged; the developer study commands require a source checkout.
+
+Cairnkeep keeps authoritative memory text readable. Ordinary memory lives in
+AgentFS-backed SQLite; derived note address spaces and wiki pages use files.
+Optional embeddings are a rebuildable search aid, not fine-tuning, encrypted
+storage, or a replacement for the original text. Semantic search computes
+missing vectors lazily and caches them outside the authoritative records.
+Configured providers receive memory text and queries; an endpoint can run
+locally or remotely. See [storage](storage.md).
+
+## Opt-in hybrid search
+
+Existing `memory_search` calls retain legacy behavior and response shapes.
+Clients can explicitly select the new strategy through MCP:
+
+```json
+{
+  "scope": "project",
+  "query": "office tunnel access",
+  "strategy": "hybrid",
+  "top_k": 5,
+  "explain": true
+}
+```
+
+The result has `mode: "hybrid"`, ordinary original-text `results`, and a
+`retrieval` diagnostic. Empty results are valid: `empty_reason` distinguishes
+an empty scope from `no_relevant_memory`. `embedding_status` distinguishes
+unconfigured, available, failed and not-used providers. Ranking scores are
+labelled `ranking-not-confidence`; neither a cosine nor a lexical score is a
+probability that a statement is true.
+
+The new strategy currently supports the `memory` address space. File-backed
+note searches retain their existing exact behavior; requesting hybrid there
+fails explicitly. Existing `memory_read` key/query semantics are unchanged.
+
+### Selection and ranking
+
+1. Resolve explicit scopes and apply enabled typed-node hard filters before
+   ranking or embedding. Neither aliases nor semantic similarity widens scope.
+2. Tokenize Unicode words, ignore a small source-defined set of common
+   English/Italian stop words, and rank with BM25 (key terms appear twice).
+   Query terms are capped at 64; tokenization is capped at 4,096 terms per field.
+3. Admit lexical matches covering at least 60% of significant query terms,
+   with at least two matched terms for multi-term queries or one for a single
+   term. An exact key is admitted directly. Compound identifiers and numeric
+   identifiers must match exactly within at least one explicit query variant.
+   Match complete identifiers, not prefixes: `ember-42` does not match
+   `ember-42-backup`, and `4.2.1` does not match `4.2.1-rc1`. Sentence-ending
+   punctuation is not part of an identifier.
+4. With embeddings available, semantic scores contribute to ranking. A match
+   based only on semantics additionally requires an **explicit positive**
+   `min_score`. Omitted or zero `min_score` never admits semantic-only matches.
+   Choose that threshold using examples from the configured model and corpus;
+   no universal cosine cutoff is supplied or claimed as calibrated confidence.
+5. Put exact keys first, then sort by 80% normalized BM25 contribution plus
+   20% nonnegative cosine contribution. Break ties by scope/key deterministically.
+   Collapse identical key/value pairs; keep different values separate.
+
+`top_k` is a maximum, not a quota to fill. Provider failure, incomplete vectors,
+zero vectors or invalid dimensions fall back to lexical admission. Failure
+diagnostics include no provider response bodies. This does not change the
+legacy strategy's existing fallback semantics.
+
+With `explain: true`, each result includes `match.reasons`, matched terms,
+lexical coverage, the optional semantic score, and source scope/key locators.
+Identical records retain all source locators. `possible_conflicts` identifies
+admitted records sharing a key but having different live values across scopes.
+That is a review aid, not a finding that one value is false or an instruction
+to merge them. Explanations exist only in the response; no new recall log is kept.
+
+### Explicit aliases
+
+Aliases can be curated into ordinary memory text, or supplied as bounded query
+alternatives:
+
+```json
+{
+  "scope": "project",
+  "query": "home connection",
+  "strategy": "hybrid",
+  "query_aliases": ["office tunnel access", "relaybox"],
+  "explain": true
+}
+```
+
+At most eight aliases of 256 UTF-8 bytes each are accepted. The original hybrid
+query is bounded to 4 KiB. Invalid bounds fail before opening scopes or making
+provider requests. Aliases affect lexical selection; the original query is
+embedded once. They are caller-supplied hints and may explicitly substitute an
+identifier, not automatically inferred identities or additional authority.
+`query_aliases` and `explain` require `strategy: "hybrid"`.
+
+## Current values, history and authority
+
+Search reads live records and excludes hidden supersession history. Updating a
+fact through the existing supersession workflow makes the new value current
+and preserves the displaced value for explicit history inspection. A new MCP
+process can retrieve the live value after the previous local process exits.
+Simultaneous clients should use one shared server: independent direct local
+AgentFS processes can contend for native database locks.
+
+Current means current in the store, not independently verified or still true
+in the real world. No automatic expiry, contradiction resolution, history
+deletion, knowledge promotion or durable memory mutation is introduced. Verify
+maintained sources and supersede stale facts only through an authorized workflow.
+Personal scopes in a trusted deployment are not a multi-user ACL mechanism.
+The separate team preview is unchanged.
+
+## A fair Markdown comparison
+
+A curated Markdown folder provides readable durable knowledge, human editing,
+navigation and Git history. Adding the same retrieval policy can reproduce the
+same retrieval results. Cairnkeep provides a maintained shared MCP interface,
+scope routing, lifecycle operations and harness integrations; SQLite itself is
+not evidence of better recall.
+
+The reproducible study builds real temporary Markdown files and equivalent
+memory candidates with identical keys, values, scopes and explicit aliases.
+It compares a searchable-folder BM25 baseline, the legacy substring policy,
+the new admission policy and the same new policy applied to Markdown. Both
+folder variants receive the same tokenizer/BM25 implementation; the baseline
+is deliberately strong, and the Markdown parity control distinguishes search
+policy gains from storage format. The substring comparator unions explicit
+query variants, equivalent to separate legacy requests.
+
+Thirteen declared scenarios include a changed tunnel configuration, aliases,
+Italian queries, continuation conventions, release policy, unrelated queries,
+nearby/obsolete identifiers and wrong-project content, with 0/100/1,000
+distractors. The [frozen report](benchmarks/memory-retrieval-v1.json) shows:
+
+| Adapter, 1,000 distractors | Correct first result, 9 positive cases | Negative queries returning results, 4 cases | Irrelevant value bytes |
+|---|---:|---:|---:|
+| Legacy substring | 3/9 | 0/4 | 0 |
+| Markdown + BM25 | 8/9 | 3/4 | 3,359 |
+| Hybrid lexical admission | 8/9 | 0/4 | 0 |
+| Markdown + same admission | 8/9 | 0/4 | 0 |
+
+These are synthetic regression cases, not general retrieval accuracy, an
+independent holdout, autonomous task quality or proof of human consent. The
+semantic-only paraphrase is deliberately missed by lexical modes; the report
+does not claim perfect recall. A real MCP control separately verifies actual
+supersession, new-process handoff, scope filtering before provider requests and
+provider-failure behavior. No live user memory is read by these checks.
+
+From a source checkout:
+
+```sh
+npm --prefix mcp-memory-server run build
+npm --prefix mcp-memory-server run check:memory-retrieval
+npm --prefix mcp-memory-server run study:memory-retrieval
+# Explicit provider evaluation, using synthetic fixtures only:
+node mcp-memory-server/scripts/run-memory-retrieval-study.mjs --live --summary
+```
+
+The live study reports a model-ID digest, not credentials or endpoint addresses.
+Its sample threshold of 0.75 is an evaluation arm, not a recommended default.
+An attempted live run during development accepted short probes but failed with
+HTTP 502 on some fixture inputs. No successful live-model result is claimed.
+The offline suite never requires a model, API key, paid service or network beyond
+its own loopback fixture. An optional learned reranker remains deferred until
+provider-specific evidence demonstrates a benefit beyond this simpler policy.
+
+## Research connection
+
+[Alex Wang's associative-memory article](https://alexwang.ai/posts/associative-memory/)
+provides a useful perspective on recall interference. Its neural-layer capacity
+experiments do not establish limits for a text database. This implementation
+applies selective retrieval at the application layer without changing model
+weights. Merely applying softmax to an existing score list preserves ordering
+and would not improve fixed-top-k selection.

package/docs/operating.md

@@ -32,6 +32,11 @@ sync command but never runs it automatically. This guide covers all three.
 - Optional: an OpenAI-compatible LLM endpoint for memory extraction and
   embedding-ranked search. Without it, memory search degrades to substring
   matching — everything else still works.
+  Explicit `memory_search` calls with `strategy: "hybrid"` additionally support
+  lexical/identifier admission without a provider, bounded `query_aliases`,
+  optional explanations and empty results. Semantic-only admission requires an
+  explicit positive `min_score`; see [memory retrieval](memory-retrieval.md)
+  for model-specific threshold limits and the Markdown comparison.
 - Optional: rootless Podman for the containerized memory server and isolated
   workspace base described in [Containers](containers.md).
 

package/docs/releasing.md

@@ -66,6 +66,13 @@ or moved. If the tree-addressed candidate has expired, rerun CI for the exact
 release commit before retrying publication; the workflow never substitutes a
 different version or tree.
 
+CI job retries can also regenerate the same tree-addressed candidate. Its
+upload explicitly replaces an existing artifact of that name within the same
+workflow run, only after all platform/runtime dependencies succeed. This avoids
+an artifact-name conflict after a completed job is retried; it does not move
+versioned image tags, republish npm versions or replace GitHub Release assets.
+Candidate checksums and tree/version binding are verified as usual.
+
 ## Repository configuration
 
 ### Repeatable post-publication verification

package/docs/storage.md

@@ -48,6 +48,11 @@ services used to process/search memory. They do not change where the SQLite
 databases are stored. Git-provider and routing configuration do not change
 storage either.
 
+`memory_search` can optionally select [hybrid retrieval](memory-retrieval.md).
+This changes read-time ranking only: aliases are request hints, explanations
+are returned to the caller without a new log, and original memory text remains
+authoritative. No new store, migration or automatic memory capture is introduced.
+
 Trajectory storage is deliberately separate from MCP memory. The harness hook
 writes `.agentfs/trajectory.db` on the client machine where Claude Code,
 OpenCode, or Pi is running; it is never redirected to the remote HTTP memory server.

package/examples/eval/bundled-fake.json

@@ -1 +1 @@
-{"identifier":"cairn-offline-fake-v1","package_version":"2.22.5","schema_version":1,"task_set_digest":"13e95379f35839593047ec34521a65a8fa3ab92bd3993bca9dc8bf5c4a9ff36f"}
+{"identifier":"cairn-offline-fake-v1","package_version":"2.23.1","schema_version":1,"task_set_digest":"13e95379f35839593047ec34521a65a8fa3ab92bd3993bca9dc8bf5c4a9ff36f"}

package/mcp-memory-server/dist/anythingllm-workspace.js

@@ -0,0 +1,109 @@
+import { existsSync, realpathSync } from "node:fs";
+import { homedir } from "node:os";
+import { isAbsolute, join, relative, resolve } from "node:path";
+import { readStableJson } from "./stable-file.js";
+/** Shared workspace that is never a project default; it holds cross-project patterns. */
+const SHARED_WORKSPACE = "engineering-patterns";
+const WORKSPACE_PATTERN = /^[A-Za-z0-9][A-Za-z0-9._-]{0,127}$/;
+const MAX_MAPPING_BYTES = 1024 * 1024;
+const MAX_PROJECTS = 4096;
+function expandHome(value) {
+    if (value === "~")
+        return homedir();
+    if (value.startsWith("~/"))
+        return join(homedir(), value.slice(2));
+    return value;
+}
+/**
+ * Locate `anythingllm-projects.json`.
+ *
+ * Mirrors the precedence in `examples/anythingllm/sync_to_anythingllm.py` so the
+ * resolver and the sync script can never disagree about which mapping is live.
+ */
+export function anythingllmProjectsFile(env = process.env, legacyDir) {
+    const override = env.CAIRN_ANYTHINGLLM_PROJECTS_FILE;
+    if (override) {
+        const explicit = resolve(expandHome(override));
+        return existsSync(explicit) ? explicit : undefined;
+    }
+    if (legacyDir) {
+        const legacy = join(legacyDir, "anythingllm-projects.json");
+        if (existsSync(legacy))
+            return legacy;
+    }
+    const configHome = env.XDG_CONFIG_HOME
+        ? resolve(expandHome(env.XDG_CONFIG_HOME))
+        : join(homedir(), ".config");
+    const standard = join(configHome, "cairnkeep", "anythingllm-projects.json");
+    if (existsSync(standard))
+        return standard;
+    return undefined;
+}
+/**
+ * Workspace slug declared for the project containing `cwd`, if any.
+ *
+ * The deepest matching `path` wins so a checkout nested inside another project
+ * resolves to its own workspace rather than its parent's.
+ */
+export function workspaceForDirectory(cwd, env = process.env, legacyDir) {
+    const file = anythingllmProjectsFile(env, legacyDir);
+    if (!file)
+        return undefined;
+    let entries;
+    try {
+        const parsed = readStableJson(file, { label: "Workspace mapping", maxBytes: MAX_MAPPING_BYTES });
+        if (!Array.isArray(parsed?.projects) || parsed.projects.length > MAX_PROJECTS)
+            return undefined;
+        entries = parsed.projects;
+    }
+    catch {
+        // A malformed or unreadable mapping must not break workspace resolution;
+        // fall back to the configured list rather than failing the tool call.
+        return undefined;
+    }
+    let here;
+    try {
+        here = realpathSync.native(cwd);
+    }
+    catch {
+        return undefined;
+    }
+    let bestSlug;
+    let bestDepth = -1;
+    let ambiguous = false;
+    for (const entry of entries) {
+        const slug = typeof entry?.slug === "string" ? entry.slug : "";
+        const path = typeof entry?.path === "string" ? entry.path : "";
+        if (!WORKSPACE_PATTERN.test(slug) || slug === SHARED_WORKSPACE || !isAbsolute(path))
+            continue;
+        let root;
+        try {
+            root = realpathSync.native(path);
+        }
+        catch {
+            continue;
+        }
+        const rel = relative(root, here);
+        const contains = rel === "" || (rel !== ".." && !rel.startsWith("../") && !rel.startsWith("..\\") && !isAbsolute(rel));
+        if (contains && root.length > bestDepth) {
+            bestSlug = slug;
+            bestDepth = root.length;
+            ambiguous = false;
+        }
+        else if (contains && root.length === bestDepth && slug !== bestSlug) {
+            ambiguous = true;
+        }
+    }
+    if (ambiguous)
+        throw new Error("Ambiguous project workspace mapping; specify workspace explicitly.");
+    return bestSlug;
+}
+/**
+ * Default workspace for a tool call: the project containing `cwd` when it is
+ * declared, otherwise the previous behaviour so existing single-project setups
+ * keep working unchanged.
+ */
+export function defaultAnythingLLMWorkspace(config, cwd = process.cwd(), env = process.env, legacyDir) {
+    return (workspaceForDirectory(cwd, env, legacyDir) ??
+        config.anythingllm_workspaces?.find((workspace) => workspace !== SHARED_WORKSPACE));
+}

package/mcp-memory-server/dist/index.js

@@ -11,6 +11,7 @@ import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
 import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
 import { WebStandardStreamableHTTPServerTransport } from "@modelcontextprotocol/sdk/server/webStandardStreamableHttp.js";
 import { z } from "zod";
+import { defaultAnythingLLMWorkspace as resolveDefaultAnythingLLMWorkspace } from "./anythingllm-workspace.js";
 import { artifactIdSchema, artifactKindSchema, artifactMediaTypeSchema, artifactNodeRefSchema, artifactProvenanceSchema, artifactSessionRefSchema, compactionSummaryInputContentSchema, diffContentSchema, generatedFileInputContentSchema, isArtifactHttpEnabled, isArtifactStoreEnabled, testOutputContentSchema, } from "./artifact-schema.js";
 import { isCapabilityContractEnabled, resolveCapabilityStatus, } from "./capability-config.js";
 import { EmbeddingCache, cosineSimilarity, embedTexts, getEmbeddingConfig, hashText, } from "./embeddings.js";
@@ -25,6 +26,7 @@ import { profileAllowsTool, resolveMcpToolProfile } from "./mcp-tool-profile.js"
 import { isWorkEvidenceEnabled } from "./work-evidence-schema.js";
 import { extractMemoryCandidates as extractMemoryCandidatesFromProvider } from "./memory-extraction.js";
 import { applyReviewedMemory, invalidateReviewedMemory } from "./reviewed-memory-store.js";
+import { rankHybridMemory } from "./memory-retrieval.js";
 const MINIMUM_NODE_MAJOR = 22;
 const nodeMajor = Number.parseInt(process.versions.node, 10);
 if (!Number.isInteger(nodeMajor) || nodeMajor < MINIMUM_NODE_MAJOR) {
@@ -359,8 +361,14 @@ async function readStdin() {
     }
     return Buffer.concat(chunks).toString("utf8");
 }
-function defaultAnythingLLMWorkspace(config) {
-    return config.anythingllm_workspaces?.find((workspace) => workspace !== "engineering-patterns");
+function defaultAnythingLLMWorkspace(config, remote = false) {
+    // HTTP callers route by session metadata, never by the server's checkout.
+    if (remote || resolveDomainRetrievalProvider() !== "anythingllm") {
+        return config.anythingllm_workspaces?.find((workspace) => workspace !== "engineering-patterns");
+    }
+    // `anythingllm-projects.json` may also sit beside the bundled sync script,
+    // which is the legacy location the script still honours.
+    return resolveDefaultAnythingLLMWorkspace(config, process.cwd(), process.env, dirname(anythingllmSyncScript));
 }
 function entryText(entry) {
     const metadata = [entry.node_type, ...(entry.tags ?? [])].filter(Boolean).join("\n");
@@ -377,7 +385,14 @@ async function semanticSearch(scope, query, topK, minScore, config = getMemoryCo
         entries: filterEntries(await listEntries(candidate, "", { projectId, typed: options.typed }), options.filters ?? {}),
     })));
     const allEntries = perScopeEntries.flatMap((group) => group.entries);
+    const hybrid = (embeddingStatus, semanticScores) => ({
+        ...rankHybridMemory(allEntries, query, { topK, minScore, aliases: options.aliases, explain: options.explain, embeddingStatus, semanticScores }),
+        mode: "hybrid",
+        ...(embeddingConfig ? { model: embeddingConfig.model } : {}),
+    });
     if (allEntries.length === 0) {
+        if (options.strategy === "hybrid")
+            return hybrid(embeddingConfig ? "not_used" : "unconfigured");
         return { results: [], mode: embeddingConfig ? "semantic" : "substring" };
     }
     const substringFallback = () => {
@@ -390,6 +405,8 @@ async function semanticSearch(scope, query, topK, minScore, config = getMemoryCo
         };
     };
     if (!embeddingConfig) {
+        if (options.strategy === "hybrid")
+            return hybrid("unconfigured");
         return substringFallback();
     }
     try {
@@ -422,9 +439,23 @@ async function semanticSearch(scope, query, topK, minScore, config = getMemoryCo
             }
         }
         const [queryVector] = await embedTexts(embeddingConfig, [query]);
+        if (options.strategy === "hybrid") {
+            const validVector = (vector) => {
+                if (!vector?.length || vector.length > 65536 || vector.some(value => !Number.isFinite(value)))
+                    return false;
+                const squaredNorm = vector.reduce((total, value) => total + value * value, 0);
+                return squaredNorm > 0 && Number.isFinite(squaredNorm);
+            };
+            if (!validVector(queryVector) || allEntries.some(entry => !validVector(vectors.get(entry)) || vectors.get(entry).length !== queryVector.length)) {
+                throw new Error("Embedding vectors are incomplete or invalid.");
+            }
+        }
         for (const cache of caches) {
             cache.save();
         }
+        if (options.strategy === "hybrid") {
+            return hybrid("available", new Map(allEntries.map(entry => [entry, Math.max(-1, Math.min(1, cosineSimilarity(queryVector, vectors.get(entry))))])));
+        }
         const exact = options.typed
             ? allEntries.filter((entry) => isExactTextHit(entry, query)).sort(compareMemoryEntries).map((entry) => ({ ...entry, score: 1 }))
             : [];
@@ -444,6 +475,8 @@ async function semanticSearch(scope, query, topK, minScore, config = getMemoryCo
     }
     catch {
         // Embedding endpoint failure — degrade gracefully to substring matching.
+        if (options.strategy === "hybrid")
+            return hybrid("failed");
         return substringFallback();
     }
 }
@@ -1112,6 +1145,9 @@ function buildMemoryServer(context, capabilitySnapshot, withCapability, toolProf
             query: z.string().min(1),
             top_k: z.number().int().min(1).max(50).optional(),
             min_score: z.number().min(0).max(1).optional(),
+            strategy: z.enum(["legacy", "hybrid"]).optional().describe("Default legacy preserves current retrieval. Hybrid admits lexical/identifier matches and semantic-only matches above an explicit positive min_score."),
+            query_aliases: z.array(z.string().trim().min(1).max(256)).max(8).optional().describe("Explicit alternative query phrases for hybrid lexical retrieval; never widens scopes."),
+            explain: z.boolean().optional().describe("Include hybrid match reasons, provenance locators and possible conflicting live values."),
             ...(typedNodesEnabled ? {
                 address_space: NODE_ADDRESS_SPACE_SCHEMA.optional(),
                 node_types: z.array(nodeTypeSchema).min(1).optional(),
@@ -1124,8 +1160,17 @@ function buildMemoryServer(context, capabilitySnapshot, withCapability, toolProf
         },
     }, async (input) => {
         const { scope, query, top_k, min_score } = input;
+        if (input.strategy !== "hybrid" && (input.query_aliases !== undefined || input.explain !== undefined)) {
+            throw new Error("query_aliases and explain require strategy='hybrid'.");
+        }
+        if (input.strategy === "hybrid") {
+            // Validate bounds before opening scopes or contacting an embedding service.
+            rankHybridMemory([], query, { topK: top_k ?? 8, minScore: min_score ?? 0, aliases: input.query_aliases });
+        }
         const addressSpace = "address_space" in input && input.address_space !== undefined ? NODE_ADDRESS_SPACE_SCHEMA.parse(input.address_space) : undefined;
         if (addressSpace && addressSpace !== "memory") {
+            if (input.strategy === "hybrid")
+                throw new Error("Hybrid retrieval currently supports the memory address space only.");
             const noteStore = await import("./note-store.js");
             const entries = noteStore.searchAddressedNotes({ ...noteTargetOptions(scope, addressSpace), query, ...memoryFiltersFromInput(input) }).slice(0, top_k ?? 8);
             const results = entries.map((entry) => ({ ...entry, score: 1 }));
@@ -1135,11 +1180,14 @@ function buildMemoryServer(context, capabilitySnapshot, withCapability, toolProf
                 structuredContent: payload,
             };
         }
-        const { results, mode, model } = await semanticSearch(scope, query, top_k ?? 8, min_score ?? 0, memoryConfig(), context.projectId, {
+        const { results, mode, model, retrieval } = await semanticSearch(scope, query, top_k ?? 8, min_score ?? 0, memoryConfig(), context.projectId, {
             typed: typedNodesEnabled,
             filters: typedNodesEnabled ? memoryFiltersFromInput(input) : {},
+            strategy: input.strategy,
+            aliases: input.query_aliases,
+            explain: input.explain,
         });
-        const payload = { mode, model, count: results.length, results };
+        const payload = { mode, model, count: results.length, results, ...(retrieval ? { retrieval } : {}) };
         return {
             content: [{ type: "text", text: asToolText(payload) }],
             structuredContent: payload,
@@ -1411,7 +1459,7 @@ function buildMemoryServer(context, capabilitySnapshot, withCapability, toolProf
             readOnlyHint: true,
         },
     }, async ({ workspace, query }) => {
-        const workspaceSlug = workspace ?? defaultAnythingLLMWorkspace(memoryConfig());
+        const workspaceSlug = workspace ?? defaultAnythingLLMWorkspace(memoryConfig(), context.remote);
         if (!workspaceSlug) {
             throw new Error("No AnythingLLM workspace provided and no project workspace found in memory config.");
         }
@@ -1439,7 +1487,7 @@ function buildMemoryServer(context, capabilitySnapshot, withCapability, toolProf
         }
         const syncMode = mode ?? "incremental";
         const config = memoryConfig();
-        const workspaceSlug = workspace ?? defaultAnythingLLMWorkspace(config);
+        const workspaceSlug = workspace ?? defaultAnythingLLMWorkspace(config, context.remote);
         if (!workspaceSlug) {
             throw new Error("No AnythingLLM workspace provided and no project workspace found in memory config.");
         }

package/mcp-memory-server/dist/memory-retrieval.js

@@ -0,0 +1,128 @@
+const STOP = new Set('a an and are as at be by can do does for from how i in is it me my of on or our please that the their this to use we what when where which with would you your un una uno il lo la gli le di da del della e che con come per posso mio mia sul nel dei'.split(' '));
+const normalize = (value) => value.normalize('NFKC').toLowerCase();
+const compare = (left, right) => left.scope.localeCompare(right.scope, 'en') || left.key.localeCompare(right.key, 'en');
+const rounded = (value) => Math.round(value * 1e6) / 1e6;
+export function retrievalTerms(text) {
+    const result = [];
+    for (const match of normalize(text).matchAll(/[\p{L}\p{N}]+/gu)) {
+        if (match[0].length > 1 && !STOP.has(match[0]))
+            result.push(match[0]);
+        if (result.length === 4096)
+            break;
+    }
+    return result;
+}
+function identifiers(text) {
+    // Match ordinary words too, then select identifiers. Requiring a separator
+    // in an unanchored greedy expression repeatedly rescans long plain words.
+    const result = new Set();
+    for (const match of text.matchAll(/[\p{L}\p{N}]+(?:[._:/-][\p{L}\p{N}]+)*/gu)) {
+        if (/[._:/\d-]/u.test(match[0]))
+            result.add(match[0]);
+    }
+    return result;
+}
+function variant(text) {
+    const normalized = normalize(text).trim();
+    return { normalized, terms: [...new Set(retrievalTerms(text))].slice(0, 64), identifiers: identifiers(normalized) };
+}
+/** BM25 is also used by the transparent Markdown-folder baseline. */
+export function lexicalMemoryScores(entries, query) {
+    const queryTerms = [...new Set(retrievalTerms(query))].slice(0, 64);
+    const docs = entries.map(entry => [
+        ...retrievalTerms(entry.key), ...retrievalTerms(entry.key),
+        ...retrievalTerms(entry.value), ...retrievalTerms([entry.node_type, ...(entry.tags ?? [])].filter(Boolean).join(' ')),
+    ]);
+    const frequencies = docs.map(terms => {
+        const counts = new Map();
+        for (const term of terms)
+            counts.set(term, (counts.get(term) ?? 0) + 1);
+        return counts;
+    });
+    const averageLength = docs.reduce((total, terms) => total + terms.length, 0) / Math.max(1, docs.length) || 1;
+    const documentFrequencies = new Map(queryTerms.map(term => [term, frequencies.filter(counts => counts.has(term)).length]));
+    return docs.map((terms, index) => queryTerms.reduce((score, term) => {
+        const frequency = frequencies[index].get(term) ?? 0;
+        const df = documentFrequencies.get(term) ?? 0;
+        const idf = Math.log(1 + (entries.length - df + 0.5) / (df + 0.5));
+        return score + idf * frequency * 2.2 / (frequency + 1.2 * (0.25 + 0.75 * terms.length / averageLength));
+    }, 0));
+}
+export function rankHybridMemory(entries, query, options) {
+    const aliases = options.aliases ?? [];
+    if (Buffer.byteLength(query, 'utf8') > 4096 || !query.trim() || aliases.length > 8
+        || aliases.some(alias => !alias.trim() || Buffer.byteLength(alias, 'utf8') > 256)
+        || !Number.isInteger(options.topK) || options.topK < 1 || options.topK > 50
+        || !Number.isFinite(options.minScore) || options.minScore < 0 || options.minScore > 1) {
+        throw new Error('Hybrid memory search requires bounded query, aliases, result count and similarity threshold.');
+    }
+    const variants = [...new Set([query, ...aliases].map(text => normalize(text).trim()))].map(variant);
+    const lexicalScores = variants.map(item => lexicalMemoryScores(entries, item.normalized));
+    const candidates = entries.map((entry, index) => {
+        const text = normalize([entry.key, entry.value, entry.node_type, ...(entry.tags ?? [])].filter(Boolean).join('\n'));
+        const words = new Set(retrievalTerms(text));
+        const entryIdentifiers = identifiers(text);
+        const matches = variants.map((item, v) => {
+            const matched = item.terms.filter(term => words.has(term));
+            const coverage = item.terms.length ? matched.length / item.terms.length : 0;
+            const identifierMatch = [...item.identifiers].every(id => entryIdentifiers.has(id));
+            const exactKey = normalize(entry.key) === item.normalized;
+            return {
+                matched, coverage, identifierMatch, exactKey,
+                eligible: identifierMatch && (exactKey || (coverage >= 0.6 && matched.length >= Math.min(2, item.terms.length) && matched.length > 0)),
+                lexical: lexicalScores[v][index],
+            };
+        });
+        const best = matches.filter(match => match.eligible).sort((a, b) => Number(b.exactKey) - Number(a.exactKey) || b.lexical - a.lexical)[0];
+        const rawSemantic = options.semanticScores?.get(entry);
+        const semantic = typeof rawSemantic === 'number' && Number.isFinite(rawSemantic) && rawSemantic >= -1 && rawSemantic <= 1 ? rawSemantic : null;
+        const semanticAdmission = options.minScore > 0 && semantic !== null && semantic >= options.minScore && matches.some(match => match.identifierMatch);
+        const lexical = best?.lexical ?? 0;
+        const score = 0.8 * lexical / (1 + lexical) + 0.2 * Math.max(0, semantic ?? 0);
+        return { entry, best, semantic, semanticAdmission, score };
+    }).filter(item => item.best || item.semanticAdmission)
+        .sort((a, b) => Number(Boolean(b.best?.exactKey)) - Number(Boolean(a.best?.exactKey)) || b.score - a.score || compare(a.entry, b.entry));
+    const grouped = new Map();
+    for (const candidate of candidates) {
+        const key = JSON.stringify([candidate.entry.key, candidate.entry.value]);
+        const group = grouped.get(key) ?? [];
+        group.push(candidate);
+        grouped.set(key, group);
+    }
+    const results = [...grouped.values()].slice(0, options.topK).map(group => {
+        const item = group[0];
+        const reasons = [
+            ...(item.best?.exactKey ? ['exact-key'] : []),
+            ...(item.best ? ['lexical-coverage'] : []),
+            ...(item.semanticAdmission ? ['semantic-threshold'] : []),
+            ...(group.length > 1 ? ['identical-records-collapsed'] : []),
+        ];
+        return {
+            ...item.entry, score: rounded(item.score),
+            ...(options.explain ? { match: {
+                    reasons, matched_terms: item.best?.matched ?? [], lexical_coverage: rounded(item.best?.coverage ?? 0),
+                    semantic_score: item.semantic === null ? null : rounded(item.semantic),
+                    sources: group.map(({ entry }) => ({ scope: entry.scope, key: entry.key })).sort(compare),
+                } } : {}),
+        };
+    });
+    const conflicts = [...new Set(results.map(item => item.key))].flatMap(key => {
+        const relevant = candidates.filter(item => item.entry.key === key);
+        return new Set(relevant.map(item => item.entry.value)).size > 1
+            ? [{ key, scopes: [...new Set(relevant.map(item => item.entry.scope))].sort() }] : [];
+    });
+    return {
+        results,
+        retrieval: {
+            strategy: 'hybrid',
+            score_kind: 'ranking-not-confidence',
+            embedding_status: options.embeddingStatus ?? (options.semanticScores ? 'available' : 'unconfigured'),
+            semantic_min_score: options.minScore > 0 ? options.minScore : null,
+            alias_count: aliases.length,
+            candidates: entries.length,
+            admitted: grouped.size,
+            ...(results.length === 0 ? { empty_reason: entries.length ? 'no_relevant_memory' : 'empty_scope' } : {}),
+            ...(options.explain ? { possible_conflicts: conflicts } : {}),
+        },
+    };
+}

package/mcp-memory-server/package.json

@@ -15,9 +15,11 @@
         "check:embeddings": "node scripts/smoke-embeddings.mjs",
         "check:extract": "node scripts/smoke-extract-cli.mjs",
         "check:search": "node scripts/smoke-search-e2e.mjs",
+        "check:memory-retrieval": "node scripts/smoke-memory-retrieval.mjs && node scripts/smoke-memory-retrieval-mcp.mjs && node scripts/run-memory-retrieval-study.mjs --check --summary",
+        "study:memory-retrieval": "npm run build && node scripts/run-memory-retrieval-study.mjs --summary",
         "check:scope-guard": "node scripts/smoke-scope-guard.mjs",
         "check:http-guard": "node scripts/smoke-http-guard.mjs",
-        "check:remote-context": "node scripts/smoke-remote-context.mjs",
+        "check:remote-context": "node scripts/smoke-remote-context.mjs && node scripts/smoke-remote-context-ports.mjs",
         "check:explore-guard": "node scripts/smoke-explore-guard.mjs",
         "check:explore-cache": "node scripts/smoke-explore-cache.mjs",
         "check:explore-crossref": "node scripts/smoke-explore-crossref.mjs",
@@ -56,6 +58,7 @@
         "check:context-pack-v216-compat": "node scripts/smoke-context-pack-v216-compat.mjs",
         "check:context-usage": "node scripts/smoke-context-usage.mjs",
         "check:domain-retrieval": "node scripts/smoke-domain-retrieval.mjs",
+        "check:anythingllm-workspace": "node scripts/smoke-anythingllm-workspace.mjs && node scripts/smoke-anythingllm-workspace-mcp.mjs",
         "check:retrieval-benchmark": "node scripts/smoke-retrieval-benchmark.mjs && node scripts/run-retrieval-benchmark.mjs --check",
         "benchmark:retrieval": "npm run build && node scripts/run-retrieval-benchmark.mjs",
         "check:work-evidence": "node scripts/smoke-work-evidence.mjs",
@@ -75,7 +78,7 @@
         "@modelcontextprotocol/sdk": "^1.32.1",
         "agentfs-sdk": "^0.6.4",
         "yaml": "2.9.1",
-        "zod": "^4.6.5"
+        "zod": "^4.6.7"
     },
     "devDependencies": {
         "@types/node": "^22.20.5",

package/opencode/command/recall.md

@@ -25,7 +25,14 @@ Read order (canonical sources still win over derived layers):
 
 <process>
 ## 1. AgentFS
-`cairn-memory_memory_search` scope `project` with $ARGUMENTS. If nothing relevant, retry scope `all`.
+`cairn-memory_memory_search` scope `project` with $ARGUMENTS. When the discovered
+schema supports it, use `strategy: "hybrid"` for identifier-sensitive or
+mixed-wording queries and `explain: true` to inspect matches. Supply only aliases
+already known from the task. Semantic-only hybrid matches require an explicitly
+chosen positive `min_score`; retain the existing semantic strategy for paraphrases
+when no model-specific threshold is available. Empty results are valid.
+Use broader configured scopes only when the question genuinely spans them;
+do not automatically widen a search merely to force a hit.
 
 ## 2. Wiki
 Read `.planning/wiki/index.md` if present and follow links relevant to $ARGUMENTS.