@cairnkeep/cli 2.22.5 → 2.23.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +21 -0
- package/claude/commands/recall.md +8 -1
- package/docs/README.md +3 -0
- package/docs/agents.md +9 -0
- package/docs/benchmarks/memory-retrieval-v1.json +115 -0
- package/docs/learning/CURRICULUM-MAP.md +1 -1
- package/docs/learning/FEATURE-GUIDE.md +7 -1
- package/docs/learning/lessons/L00-why-cairnkeep.md +1 -1
- package/docs/learning/lessons/L01-safe-trial.md +3 -3
- package/docs/learning/lessons/L02-installation.md +2 -2
- package/docs/learning/lessons/L03-first-project.md +1 -1
- package/docs/learning/lessons/L04-memory-fundamentals.md +20 -0
- package/docs/learning/lessons/L19-validated-skill-improvement.md +1 -1
- package/docs/learning/lessons/L20-mcp-tool-profiles.md +1 -1
- package/docs/learning/lessons/L21-context-packs.md +1 -1
- package/docs/learning/lessons/L22-native-windows.md +2 -2
- package/docs/learning/lessons/L23-guided-setup.md +1 -1
- package/docs/learning/lessons/L24-okf-exchange.md +1 -1
- package/docs/learning/lessons/L25-playbooks.md +1 -1
- package/docs/learning/lessons/L26-context-intelligence.md +1 -1
- package/docs/learning/lessons/L27-ecosystem-evidence.md +1 -1
- package/docs/learning/lessons/L28-security-assurance.md +2 -2
- package/docs/learning/video-scripts/V04-memory-retrieval-supplement.md +34 -0
- package/docs/memory-retrieval.md +174 -0
- package/docs/operating.md +5 -0
- package/docs/releasing.md +7 -0
- package/docs/storage.md +5 -0
- package/examples/eval/bundled-fake.json +1 -1
- package/mcp-memory-server/dist/index.js +43 -2
- package/mcp-memory-server/dist/memory-retrieval.js +128 -0
- package/mcp-memory-server/package.json +2 -0
- package/opencode/command/recall.md +8 -1
- package/package.json +1 -1
- package/scripts/test-memory-retrieval-docs.sh +19 -0
- package/scripts/test-release-sbom.sh +32 -0
- package/templates/playbook-agent-instructions.md.template +5 -0
package/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,27 @@ All notable user-facing changes are documented here. This project follows
|
|
|
5
5
|
|
|
6
6
|
## [Unreleased]
|
|
7
7
|
|
|
8
|
+
## [2.23.0] - 2026-10-21
|
|
9
|
+
|
|
10
|
+
### Memory retrieval
|
|
11
|
+
|
|
12
|
+
- Add opt-in `memory_search` hybrid lexical/identifier admission, bounded query
|
|
13
|
+
aliases, explicit semantic-only thresholds, deterministic ranking, duplicate
|
|
14
|
+
source locators and optional conflict/match explanations. Preserve existing
|
|
15
|
+
default responses, history, scope filters and storage layouts. Fall back to
|
|
16
|
+
lexical admission when optional embedding results are invalid or unavailable.
|
|
17
|
+
- Add a frozen continuity study against real curated Markdown files, including
|
|
18
|
+
the same-policy Markdown parity control and 0/100/1,000 distractors. Document
|
|
19
|
+
synthetic limitations and unsuccessful live-provider evaluation; update the
|
|
20
|
+
agent guidance, learning lesson and recording supplement.
|
|
21
|
+
|
|
22
|
+
### CI
|
|
23
|
+
|
|
24
|
+
- Allow a retried, fully gated CI job to replace its run-local tree-addressed
|
|
25
|
+
release candidate instead of failing on the existing artifact name. Keep all
|
|
26
|
+
platform/runtime dependencies and immutable published versions unchanged; add
|
|
27
|
+
parsed-workflow positive and mutation controls.
|
|
28
|
+
|
|
8
29
|
## [2.22.5] - 2026-10-08
|
|
9
30
|
|
|
10
31
|
### Operations
|
|
@@ -19,7 +19,14 @@ Read order (canonical sources still win over derived layers):
|
|
|
19
19
|
|
|
20
20
|
<process>
|
|
21
21
|
## 1. AgentFS
|
|
22
|
-
`memory_search` scope `project` with $ARGUMENTS.
|
|
22
|
+
`memory_search` scope `project` with $ARGUMENTS. When the discovered schema
|
|
23
|
+
supports it, use `strategy: "hybrid"` for identifier-sensitive or mixed-wording
|
|
24
|
+
queries and `explain: true` to inspect matches. Supply only aliases already known
|
|
25
|
+
from the task. Semantic-only hybrid matches require an explicitly chosen
|
|
26
|
+
positive `min_score`; retain the existing semantic strategy for paraphrases
|
|
27
|
+
when no model-specific threshold is available. Empty results are valid.
|
|
28
|
+
Use broader configured scopes only when the question genuinely spans them;
|
|
29
|
+
do not automatically widen a search merely to force a hit.
|
|
23
30
|
|
|
24
31
|
## 2. Wiki
|
|
25
32
|
Read `.planning/wiki/index.md` if present and follow links relevant to $ARGUMENTS.
|
package/docs/README.md
CHANGED
|
@@ -31,6 +31,9 @@ linked guides rather than the project landing page.
|
|
|
31
31
|
|
|
32
32
|
## Context intelligence
|
|
33
33
|
|
|
34
|
+
- [Memory retrieval and the Markdown baseline](memory-retrieval.md) explains
|
|
35
|
+
optional hybrid admission, aliases, relevance diagnostics and reproducible
|
|
36
|
+
continuity evidence against searchable Markdown.
|
|
34
37
|
- [Context intelligence](context-intelligence.md) connects progressive context-pack
|
|
35
38
|
retrieval, the frozen benchmark, usage receipts, review-gated proposals, and
|
|
36
39
|
the optional read-only OpenViking provider.
|
package/docs/agents.md
CHANGED
|
@@ -27,6 +27,15 @@ constraints, recurring failures, or prior work:
|
|
|
27
27
|
This protocol improves continuity without allowing stale memory to outrank
|
|
28
28
|
code, tests, current documentation, or the user's request.
|
|
29
29
|
|
|
30
|
+
When the discovered tool schema supports it, `strategy: "hybrid"` is useful for
|
|
31
|
+
identifier-sensitive and mixed-wording queries. `explain: true` exposes selection
|
|
32
|
+
reasons; explicit `query_aliases` can express names already known from the task.
|
|
33
|
+
Do not invent aliases or broaden scopes to obtain a match. Semantic-only
|
|
34
|
+
paraphrase retrieval needs an explicitly chosen positive `min_score` in hybrid
|
|
35
|
+
mode; without a model-specific threshold, the existing semantic strategy remains
|
|
36
|
+
available. Empty results are valid, and ranking scores are not factual confidence.
|
|
37
|
+
See [memory retrieval and evidence](memory-retrieval.md).
|
|
38
|
+
|
|
30
39
|
## What setup installs
|
|
31
40
|
|
|
32
41
|
`cairn setup` reconciles a delimited Cairnkeep block in the project's
|
|
@@ -0,0 +1,115 @@
|
|
|
1
|
+
{
|
|
2
|
+
"schema_version": 1,
|
|
3
|
+
"study": "synthetic-memory-continuity-v1",
|
|
4
|
+
"fixture_digest": "051a1ee175fc20d9e38c43d838b234c759133805ab335f74ef5603ee0871fce8",
|
|
5
|
+
"embedding": {
|
|
6
|
+
"mode": "none"
|
|
7
|
+
},
|
|
8
|
+
"limitations": [
|
|
9
|
+
"Synthetic retrieval scenarios, not autonomous task-quality or human-usability evidence.",
|
|
10
|
+
"Markdown baseline shares BM25/tokenization and receives identical alias information.",
|
|
11
|
+
"Markdown with the same admission policy is included to distinguish retrieval-policy gains from storage format.",
|
|
12
|
+
"Offline cases do not evaluate learned semantic paraphrase retrieval.",
|
|
13
|
+
"Explicit live threshold is an example, not universal calibration or a confidence probability.",
|
|
14
|
+
"Current-state fixture excludes archived history; actual supersession is covered by the MCP regression.",
|
|
15
|
+
"No automatic contradiction reconciliation, human consent or concurrent local-store access is claimed."
|
|
16
|
+
],
|
|
17
|
+
"measurements": [
|
|
18
|
+
{
|
|
19
|
+
"adapter": "markdown-bm25",
|
|
20
|
+
"distractor_count": 0,
|
|
21
|
+
"hit_at_1": 0.8888888888888888,
|
|
22
|
+
"recall_at_5": 0.8888888888888888,
|
|
23
|
+
"negative_false_positive_rate": 0.75,
|
|
24
|
+
"irrelevant_bytes": 794
|
|
25
|
+
},
|
|
26
|
+
{
|
|
27
|
+
"adapter": "markdown-hybrid",
|
|
28
|
+
"distractor_count": 0,
|
|
29
|
+
"hit_at_1": 0.8888888888888888,
|
|
30
|
+
"recall_at_5": 0.8888888888888888,
|
|
31
|
+
"negative_false_positive_rate": 0,
|
|
32
|
+
"irrelevant_bytes": 0
|
|
33
|
+
},
|
|
34
|
+
{
|
|
35
|
+
"adapter": "legacy-substring",
|
|
36
|
+
"distractor_count": 0,
|
|
37
|
+
"hit_at_1": 0.3333333333333333,
|
|
38
|
+
"recall_at_5": 0.3333333333333333,
|
|
39
|
+
"negative_false_positive_rate": 0,
|
|
40
|
+
"irrelevant_bytes": 0
|
|
41
|
+
},
|
|
42
|
+
{
|
|
43
|
+
"adapter": "hybrid-lexical",
|
|
44
|
+
"distractor_count": 0,
|
|
45
|
+
"hit_at_1": 0.8888888888888888,
|
|
46
|
+
"recall_at_5": 0.8888888888888888,
|
|
47
|
+
"negative_false_positive_rate": 0,
|
|
48
|
+
"irrelevant_bytes": 0
|
|
49
|
+
},
|
|
50
|
+
{
|
|
51
|
+
"adapter": "markdown-bm25",
|
|
52
|
+
"distractor_count": 100,
|
|
53
|
+
"hit_at_1": 0.8888888888888888,
|
|
54
|
+
"recall_at_5": 0.8888888888888888,
|
|
55
|
+
"negative_false_positive_rate": 0.75,
|
|
56
|
+
"irrelevant_bytes": 3359
|
|
57
|
+
},
|
|
58
|
+
{
|
|
59
|
+
"adapter": "markdown-hybrid",
|
|
60
|
+
"distractor_count": 100,
|
|
61
|
+
"hit_at_1": 0.8888888888888888,
|
|
62
|
+
"recall_at_5": 0.8888888888888888,
|
|
63
|
+
"negative_false_positive_rate": 0,
|
|
64
|
+
"irrelevant_bytes": 0
|
|
65
|
+
},
|
|
66
|
+
{
|
|
67
|
+
"adapter": "legacy-substring",
|
|
68
|
+
"distractor_count": 100,
|
|
69
|
+
"hit_at_1": 0.3333333333333333,
|
|
70
|
+
"recall_at_5": 0.3333333333333333,
|
|
71
|
+
"negative_false_positive_rate": 0,
|
|
72
|
+
"irrelevant_bytes": 0
|
|
73
|
+
},
|
|
74
|
+
{
|
|
75
|
+
"adapter": "hybrid-lexical",
|
|
76
|
+
"distractor_count": 100,
|
|
77
|
+
"hit_at_1": 0.8888888888888888,
|
|
78
|
+
"recall_at_5": 0.8888888888888888,
|
|
79
|
+
"negative_false_positive_rate": 0,
|
|
80
|
+
"irrelevant_bytes": 0
|
|
81
|
+
},
|
|
82
|
+
{
|
|
83
|
+
"adapter": "markdown-bm25",
|
|
84
|
+
"distractor_count": 1000,
|
|
85
|
+
"hit_at_1": 0.8888888888888888,
|
|
86
|
+
"recall_at_5": 0.8888888888888888,
|
|
87
|
+
"negative_false_positive_rate": 0.75,
|
|
88
|
+
"irrelevant_bytes": 3359
|
|
89
|
+
},
|
|
90
|
+
{
|
|
91
|
+
"adapter": "markdown-hybrid",
|
|
92
|
+
"distractor_count": 1000,
|
|
93
|
+
"hit_at_1": 0.8888888888888888,
|
|
94
|
+
"recall_at_5": 0.8888888888888888,
|
|
95
|
+
"negative_false_positive_rate": 0,
|
|
96
|
+
"irrelevant_bytes": 0
|
|
97
|
+
},
|
|
98
|
+
{
|
|
99
|
+
"adapter": "legacy-substring",
|
|
100
|
+
"distractor_count": 1000,
|
|
101
|
+
"hit_at_1": 0.3333333333333333,
|
|
102
|
+
"recall_at_5": 0.3333333333333333,
|
|
103
|
+
"negative_false_positive_rate": 0,
|
|
104
|
+
"irrelevant_bytes": 0
|
|
105
|
+
},
|
|
106
|
+
{
|
|
107
|
+
"adapter": "hybrid-lexical",
|
|
108
|
+
"distractor_count": 1000,
|
|
109
|
+
"hit_at_1": 0.8888888888888888,
|
|
110
|
+
"recall_at_5": 0.8888888888888888,
|
|
111
|
+
"negative_false_positive_rate": 0,
|
|
112
|
+
"irrelevant_bytes": 0
|
|
113
|
+
}
|
|
114
|
+
]
|
|
115
|
+
}
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Cairnkeep feature guide
|
|
2
2
|
|
|
3
|
-
**Baseline:** Cairnkeep 2.
|
|
3
|
+
**Baseline:** Cairnkeep 2.23.0
|
|
4
4
|
|
|
5
5
|
This is the step-back map for choosing and operating Cairnkeep features. The
|
|
6
6
|
[course labs](https://github.com/cairnkeep/cairnkeep-course-labs) provide one
|
|
@@ -36,6 +36,12 @@ explicit native Codex exec exports. See the L16 memory-protocol supplement;
|
|
|
36
36
|
protocol observations and independently verified task correctness remain
|
|
37
37
|
separate evidence, not a claim of model improvement.
|
|
38
38
|
|
|
39
|
+
Version 2.23 adds opt-in hybrid memory retrieval, explicit query aliases and
|
|
40
|
+
match/source/conflict explanations without changing legacy defaults. See
|
|
41
|
+
[memory retrieval](../memory-retrieval.md), [L04](lessons/L04-memory-fundamentals.md)
|
|
42
|
+
and its recording supplement. The synthetic Markdown comparison measures
|
|
43
|
+
retrieval policy, not general task quality or a storage-format advantage.
|
|
44
|
+
|
|
39
45
|
## The three-layer model
|
|
40
46
|
|
|
41
47
|
1. **Memory server:** an MCP process that owns scoped durable storage. The
|
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
**Status:** Ready
|
|
4
4
|
**Track:** Quickstart
|
|
5
5
|
**Time:** 15 minutes
|
|
6
|
-
**Tested with:** Cairnkeep 2.
|
|
6
|
+
**Tested with:** Cairnkeep 2.23.0
|
|
7
7
|
|
|
8
8
|
## Outcome
|
|
9
9
|
|
|
@@ -26,7 +26,7 @@ ephemeral; the named course volume is persistent.
|
|
|
26
26
|
1. Pull the public image:
|
|
27
27
|
|
|
28
28
|
```bash
|
|
29
|
-
podman pull ghcr.io/cairnkeep/cairnkeep:2.
|
|
29
|
+
podman pull ghcr.io/cairnkeep/cairnkeep:2.23.0
|
|
30
30
|
```
|
|
31
31
|
|
|
32
32
|
2. Start the stdio server with a read-only root filesystem and no Linux
|
|
@@ -38,7 +38,7 @@ ephemeral; the named course volume is persistent.
|
|
|
38
38
|
--read-only --cap-drop=all --security-opt=no-new-privileges \
|
|
39
39
|
--tmpfs=/tmp:rw,noexec,nosuid,size=64m,mode=1777 \
|
|
40
40
|
--volume cairnkeep-course-data:/data:Z,U \
|
|
41
|
-
ghcr.io/cairnkeep/cairnkeep:2.
|
|
41
|
+
ghcr.io/cairnkeep/cairnkeep:2.23.0 stdio
|
|
42
42
|
```
|
|
43
43
|
|
|
44
44
|
3. The process waits quietly for MCP input. Press `Ctrl-C` after confirming it
|
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
**Status:** Ready
|
|
4
4
|
**Track:** Quickstart
|
|
5
5
|
**Time:** 25 minutes
|
|
6
|
-
**Tested with:** Cairnkeep 2.
|
|
6
|
+
**Tested with:** Cairnkeep 2.23.0 and Node.js 22 or newer
|
|
7
7
|
|
|
8
8
|
## Outcome
|
|
9
9
|
|
|
@@ -47,7 +47,7 @@ sync. Codex project trust remains an explicit operator decision.
|
|
|
47
47
|
2. Install Cairnkeep:
|
|
48
48
|
|
|
49
49
|
```bash
|
|
50
|
-
npm install --global @cairnkeep/cli@2.
|
|
50
|
+
npm install --global @cairnkeep/cli@2.23.0
|
|
51
51
|
cairn --version
|
|
52
52
|
```
|
|
53
53
|
|
|
@@ -15,6 +15,10 @@ review extracted candidates before they become durable memory.
|
|
|
15
15
|
- Key families: `decisions/`, `pitfalls/`, `patterns/`, `bugs/`,
|
|
16
16
|
`constraints/`, and `conventions/`.
|
|
17
17
|
- Exact read versus semantic search and substring fallback.
|
|
18
|
+
- Optional hybrid lexical/identifier admission, explicit aliases, match reasons
|
|
19
|
+
and valid empty results; similarity scores are not factual confidence.
|
|
20
|
+
- Curated Markdown as a credible baseline; indexing policy and storage format
|
|
21
|
+
are separate decisions.
|
|
18
22
|
- Superseding a fact without losing history.
|
|
19
23
|
- Session-end candidate extraction and `/memory-review` as the acceptance gate.
|
|
20
24
|
- Typed node metadata, tags, and hard filters as an independent opt-in.
|
|
@@ -27,10 +31,24 @@ Store three synthetic facts in different categories, retrieve them by an
|
|
|
27
31
|
imprecise query, supersede one, inspect its history, and reject one staged
|
|
28
32
|
candidate.
|
|
29
33
|
|
|
34
|
+
From a source checkout, run:
|
|
35
|
+
|
|
36
|
+
```sh
|
|
37
|
+
npm --prefix mcp-memory-server run study:memory-retrieval
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
Compare the 0/100/1,000-distractor results and explain
|
|
41
|
+
why Markdown with the same admission policy matches Cairnkeep. Inspect the
|
|
42
|
+
semantic-only case that lexical modes miss; do not describe the synthetic
|
|
43
|
+
report as general accuracy. Repeat retrieval after superseding a synthetic
|
|
44
|
+
fact and starting a fresh client, verifying the new value and explicit history.
|
|
45
|
+
|
|
30
46
|
## Acceptance criteria
|
|
31
47
|
|
|
32
48
|
- The learner can justify the scope and category for each fact.
|
|
33
49
|
- Search finds a fact without requiring its exact key.
|
|
50
|
+
- A nearby but incorrect identifier and an unrelated query return no hybrid
|
|
51
|
+
match; aliases never widen scope or grant authority.
|
|
34
52
|
- Superseding preserves the prior value in history.
|
|
35
53
|
- No extracted candidate becomes durable without an explicit review action.
|
|
36
54
|
- Enabling typed nodes changes discovery structure, not the authority of a fact.
|
|
@@ -38,5 +56,7 @@ candidate.
|
|
|
38
56
|
## Source material
|
|
39
57
|
|
|
40
58
|
- [Operating guide](../../operating.md#the-workflow)
|
|
59
|
+
- [Memory retrieval and the Markdown baseline](../../memory-retrieval.md)
|
|
60
|
+
- [Recording supplement](../video-scripts/V04-memory-retrieval-supplement.md)
|
|
41
61
|
- [Storage and deployment](../../storage.md)
|
|
42
62
|
- [L14 - Typed memory and controlled import](L14-typed-memory.md)
|
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
**Status:** Ready
|
|
4
4
|
**Track:** Operator
|
|
5
5
|
**Time:** 30 minutes
|
|
6
|
-
**Tested with:** Cairnkeep 2.
|
|
6
|
+
**Tested with:** Cairnkeep 2.23.0, Windows x64, and Node.js 22 or newer
|
|
7
7
|
|
|
8
8
|
## Outcome
|
|
9
9
|
|
|
@@ -16,7 +16,7 @@ perform a reversible uninstall without relying on WSL or Git Bash.
|
|
|
16
16
|
From PowerShell:
|
|
17
17
|
|
|
18
18
|
```powershell
|
|
19
|
-
npm install --global @cairnkeep/cli@2.
|
|
19
|
+
npm install --global @cairnkeep/cli@2.23.0
|
|
20
20
|
cairn version
|
|
21
21
|
|
|
22
22
|
$Lab = Join-Path $env:TEMP 'Cairnkeep Windows Lab'
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# L28 - Verify Cairnkeep's security posture
|
|
2
2
|
|
|
3
3
|
**Status:** Ready
|
|
4
|
-
**Tested with:** Cairnkeep 2.
|
|
4
|
+
**Tested with:** Cairnkeep 2.23.0 and Node.js 22 or newer
|
|
5
5
|
|
|
6
6
|
## Outcome
|
|
7
7
|
|
|
@@ -11,7 +11,7 @@ and choose least authority for a network-facing MCP client.
|
|
|
11
11
|
|
|
12
12
|
## Prerequisites
|
|
13
13
|
|
|
14
|
-
- Install Cairnkeep 2.
|
|
14
|
+
- Install Cairnkeep 2.23.0 or use a disposable source checkout.
|
|
15
15
|
- Complete [L20 - Least-authority MCP tool profiles](L20-mcp-tool-profiles.md).
|
|
16
16
|
- Use synthetic values only; never paste a real bearer token into a recording,
|
|
17
17
|
terminal transcript, or issue.
|
|
@@ -0,0 +1,34 @@
|
|
|
1
|
+
# V04 supplement — Finding the current fact without flooding context
|
|
2
|
+
|
|
3
|
+
Technical recording brief; no video is claimed as recorded. Use the source
|
|
4
|
+
checkout containing hybrid memory search and only invented fixture content.
|
|
5
|
+
|
|
6
|
+
## Real terminal demonstration
|
|
7
|
+
|
|
8
|
+
Type at readable human speed, leaving pauses after each command:
|
|
9
|
+
|
|
10
|
+
```sh
|
|
11
|
+
npm --prefix mcp-memory-server run build
|
|
12
|
+
npm --prefix mcp-memory-server run check:memory-retrieval
|
|
13
|
+
npm --prefix mcp-memory-server run study:memory-retrieval
|
|
14
|
+
```
|
|
15
|
+
|
|
16
|
+
Zoom into the real MCP test's new-process handoff, superseded-history exclusion,
|
|
17
|
+
wrong-identifier refusal and provider-failure result. Demonstrate an MCP call
|
|
18
|
+
with `strategy: "hybrid"` and `explain: true`; show matched terms and source
|
|
19
|
+
locators, then show an unrelated query returning zero results. Explain that
|
|
20
|
+
agents still verify maintained sources and do not automatically correct memory.
|
|
21
|
+
|
|
22
|
+
## Evidence and pacing
|
|
23
|
+
|
|
24
|
+
Show the frozen 1,000-distractor table: 3/9 first hits for substring, 8/9 for
|
|
25
|
+
BM25 and hybrid. Pause on the difference in irrelevant bytes and negative hits.
|
|
26
|
+
Then highlight that Markdown with the same admission policy also achieves 8/9
|
|
27
|
+
and zero negative hits. State that storage format did not cause the gain.
|
|
28
|
+
|
|
29
|
+
Finish with the semantic-only paraphrase that lexical modes miss. Explain that
|
|
30
|
+
provider-specific thresholds need their own evidence, that a ranking score is
|
|
31
|
+
not confidence, and that the attempted live study failed with HTTP 502. Do not
|
|
32
|
+
invent a successful live result or claim general task-quality improvements.
|
|
33
|
+
Use short terminal zooms and side-by-side result comparisons instead of static
|
|
34
|
+
slides; keep credentials and real memory off screen.
|
|
@@ -0,0 +1,174 @@
|
|
|
1
|
+
# Memory retrieval and the Markdown baseline
|
|
2
|
+
|
|
3
|
+
Status: available since Cairnkeep 2.23.0 as an opt-in strategy. Existing search
|
|
4
|
+
defaults are unchanged; the developer study commands require a source checkout.
|
|
5
|
+
|
|
6
|
+
Cairnkeep keeps authoritative memory text readable. Ordinary memory lives in
|
|
7
|
+
AgentFS-backed SQLite; derived note address spaces and wiki pages use files.
|
|
8
|
+
Optional embeddings are a rebuildable search aid, not fine-tuning, encrypted
|
|
9
|
+
storage, or a replacement for the original text. Semantic search computes
|
|
10
|
+
missing vectors lazily and caches them outside the authoritative records.
|
|
11
|
+
Configured providers receive memory text and queries; an endpoint can run
|
|
12
|
+
locally or remotely. See [storage](storage.md).
|
|
13
|
+
|
|
14
|
+
## Opt-in hybrid search
|
|
15
|
+
|
|
16
|
+
Existing `memory_search` calls retain legacy behavior and response shapes.
|
|
17
|
+
Clients can explicitly select the new strategy through MCP:
|
|
18
|
+
|
|
19
|
+
```json
|
|
20
|
+
{
|
|
21
|
+
"scope": "project",
|
|
22
|
+
"query": "office tunnel access",
|
|
23
|
+
"strategy": "hybrid",
|
|
24
|
+
"top_k": 5,
|
|
25
|
+
"explain": true
|
|
26
|
+
}
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
The result has `mode: "hybrid"`, ordinary original-text `results`, and a
|
|
30
|
+
`retrieval` diagnostic. Empty results are valid: `empty_reason` distinguishes
|
|
31
|
+
an empty scope from `no_relevant_memory`. `embedding_status` distinguishes
|
|
32
|
+
unconfigured, available, failed and not-used providers. Ranking scores are
|
|
33
|
+
labelled `ranking-not-confidence`; neither a cosine nor a lexical score is a
|
|
34
|
+
probability that a statement is true.
|
|
35
|
+
|
|
36
|
+
The new strategy currently supports the `memory` address space. File-backed
|
|
37
|
+
note searches retain their existing exact behavior; requesting hybrid there
|
|
38
|
+
fails explicitly. Existing `memory_read` key/query semantics are unchanged.
|
|
39
|
+
|
|
40
|
+
### Selection and ranking
|
|
41
|
+
|
|
42
|
+
1. Resolve explicit scopes and apply enabled typed-node hard filters before
|
|
43
|
+
ranking or embedding. Neither aliases nor semantic similarity widens scope.
|
|
44
|
+
2. Tokenize Unicode words, ignore a small source-defined set of common
|
|
45
|
+
English/Italian stop words, and rank with BM25 (key terms appear twice).
|
|
46
|
+
Query terms are capped at 64; tokenization is capped at 4,096 terms per field.
|
|
47
|
+
3. Admit lexical matches covering at least 60% of significant query terms,
|
|
48
|
+
with at least two matched terms for multi-term queries or one for a single
|
|
49
|
+
term. An exact key is admitted directly. Compound identifiers and numeric
|
|
50
|
+
identifiers must match exactly within at least one explicit query variant.
|
|
51
|
+
Match complete identifiers, not prefixes: `ember-42` does not match
|
|
52
|
+
`ember-42-backup`, and `4.2.1` does not match `4.2.1-rc1`. Sentence-ending
|
|
53
|
+
punctuation is not part of an identifier.
|
|
54
|
+
4. With embeddings available, semantic scores contribute to ranking. A match
|
|
55
|
+
based only on semantics additionally requires an **explicit positive**
|
|
56
|
+
`min_score`. Omitted or zero `min_score` never admits semantic-only matches.
|
|
57
|
+
Choose that threshold using examples from the configured model and corpus;
|
|
58
|
+
no universal cosine cutoff is supplied or claimed as calibrated confidence.
|
|
59
|
+
5. Put exact keys first, then sort by 80% normalized BM25 contribution plus
|
|
60
|
+
20% nonnegative cosine contribution. Break ties by scope/key deterministically.
|
|
61
|
+
Collapse identical key/value pairs; keep different values separate.
|
|
62
|
+
|
|
63
|
+
`top_k` is a maximum, not a quota to fill. Provider failure, incomplete vectors,
|
|
64
|
+
zero vectors or invalid dimensions fall back to lexical admission. Failure
|
|
65
|
+
diagnostics include no provider response bodies. This does not change the
|
|
66
|
+
legacy strategy's existing fallback semantics.
|
|
67
|
+
|
|
68
|
+
With `explain: true`, each result includes `match.reasons`, matched terms,
|
|
69
|
+
lexical coverage, the optional semantic score, and source scope/key locators.
|
|
70
|
+
Identical records retain all source locators. `possible_conflicts` identifies
|
|
71
|
+
admitted records sharing a key but having different live values across scopes.
|
|
72
|
+
That is a review aid, not a finding that one value is false or an instruction
|
|
73
|
+
to merge them. Explanations exist only in the response; no new recall log is kept.
|
|
74
|
+
|
|
75
|
+
### Explicit aliases
|
|
76
|
+
|
|
77
|
+
Aliases can be curated into ordinary memory text, or supplied as bounded query
|
|
78
|
+
alternatives:
|
|
79
|
+
|
|
80
|
+
```json
|
|
81
|
+
{
|
|
82
|
+
"scope": "project",
|
|
83
|
+
"query": "home connection",
|
|
84
|
+
"strategy": "hybrid",
|
|
85
|
+
"query_aliases": ["office tunnel access", "relaybox"],
|
|
86
|
+
"explain": true
|
|
87
|
+
}
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
At most eight aliases of 256 UTF-8 bytes each are accepted. The original hybrid
|
|
91
|
+
query is bounded to 4 KiB. Invalid bounds fail before opening scopes or making
|
|
92
|
+
provider requests. Aliases affect lexical selection; the original query is
|
|
93
|
+
embedded once. They are caller-supplied hints and may explicitly substitute an
|
|
94
|
+
identifier, not automatically inferred identities or additional authority.
|
|
95
|
+
`query_aliases` and `explain` require `strategy: "hybrid"`.
|
|
96
|
+
|
|
97
|
+
## Current values, history and authority
|
|
98
|
+
|
|
99
|
+
Search reads live records and excludes hidden supersession history. Updating a
|
|
100
|
+
fact through the existing supersession workflow makes the new value current
|
|
101
|
+
and preserves the displaced value for explicit history inspection. A new MCP
|
|
102
|
+
process can retrieve the live value after the previous local process exits.
|
|
103
|
+
Simultaneous clients should use one shared server: independent direct local
|
|
104
|
+
AgentFS processes can contend for native database locks.
|
|
105
|
+
|
|
106
|
+
Current means current in the store, not independently verified or still true
|
|
107
|
+
in the real world. No automatic expiry, contradiction resolution, history
|
|
108
|
+
deletion, knowledge promotion or durable memory mutation is introduced. Verify
|
|
109
|
+
maintained sources and supersede stale facts only through an authorized workflow.
|
|
110
|
+
Personal scopes in a trusted deployment are not a multi-user ACL mechanism.
|
|
111
|
+
The separate team preview is unchanged.
|
|
112
|
+
|
|
113
|
+
## A fair Markdown comparison
|
|
114
|
+
|
|
115
|
+
A curated Markdown folder provides readable durable knowledge, human editing,
|
|
116
|
+
navigation and Git history. Adding the same retrieval policy can reproduce the
|
|
117
|
+
same retrieval results. Cairnkeep provides a maintained shared MCP interface,
|
|
118
|
+
scope routing, lifecycle operations and harness integrations; SQLite itself is
|
|
119
|
+
not evidence of better recall.
|
|
120
|
+
|
|
121
|
+
The reproducible study builds real temporary Markdown files and equivalent
|
|
122
|
+
memory candidates with identical keys, values, scopes and explicit aliases.
|
|
123
|
+
It compares a searchable-folder BM25 baseline, the legacy substring policy,
|
|
124
|
+
the new admission policy and the same new policy applied to Markdown. Both
|
|
125
|
+
folder variants receive the same tokenizer/BM25 implementation; the baseline
|
|
126
|
+
is deliberately strong, and the Markdown parity control distinguishes search
|
|
127
|
+
policy gains from storage format. The substring comparator unions explicit
|
|
128
|
+
query variants, equivalent to separate legacy requests.
|
|
129
|
+
|
|
130
|
+
Thirteen declared scenarios include a changed tunnel configuration, aliases,
|
|
131
|
+
Italian queries, continuation conventions, release policy, unrelated queries,
|
|
132
|
+
nearby/obsolete identifiers and wrong-project content, with 0/100/1,000
|
|
133
|
+
distractors. The [frozen report](benchmarks/memory-retrieval-v1.json) shows:
|
|
134
|
+
|
|
135
|
+
| Adapter, 1,000 distractors | Correct first result, 9 positive cases | Negative queries returning results, 4 cases | Irrelevant value bytes |
|
|
136
|
+
|---|---:|---:|---:|
|
|
137
|
+
| Legacy substring | 3/9 | 0/4 | 0 |
|
|
138
|
+
| Markdown + BM25 | 8/9 | 3/4 | 3,359 |
|
|
139
|
+
| Hybrid lexical admission | 8/9 | 0/4 | 0 |
|
|
140
|
+
| Markdown + same admission | 8/9 | 0/4 | 0 |
|
|
141
|
+
|
|
142
|
+
These are synthetic regression cases, not general retrieval accuracy, an
|
|
143
|
+
independent holdout, autonomous task quality or proof of human consent. The
|
|
144
|
+
semantic-only paraphrase is deliberately missed by lexical modes; the report
|
|
145
|
+
does not claim perfect recall. A real MCP control separately verifies actual
|
|
146
|
+
supersession, new-process handoff, scope filtering before provider requests and
|
|
147
|
+
provider-failure behavior. No live user memory is read by these checks.
|
|
148
|
+
|
|
149
|
+
From a source checkout:
|
|
150
|
+
|
|
151
|
+
```sh
|
|
152
|
+
npm --prefix mcp-memory-server run build
|
|
153
|
+
npm --prefix mcp-memory-server run check:memory-retrieval
|
|
154
|
+
npm --prefix mcp-memory-server run study:memory-retrieval
|
|
155
|
+
# Explicit provider evaluation, using synthetic fixtures only:
|
|
156
|
+
node mcp-memory-server/scripts/run-memory-retrieval-study.mjs --live --summary
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
The live study reports a model-ID digest, not credentials or endpoint addresses.
|
|
160
|
+
Its sample threshold of 0.75 is an evaluation arm, not a recommended default.
|
|
161
|
+
An attempted live run during development accepted short probes but failed with
|
|
162
|
+
HTTP 502 on some fixture inputs. No successful live-model result is claimed.
|
|
163
|
+
The offline suite never requires a model, API key, paid service or network beyond
|
|
164
|
+
its own loopback fixture. An optional learned reranker remains deferred until
|
|
165
|
+
provider-specific evidence demonstrates a benefit beyond this simpler policy.
|
|
166
|
+
|
|
167
|
+
## Research connection
|
|
168
|
+
|
|
169
|
+
[Alex Wang's associative-memory article](https://alexwang.ai/posts/associative-memory/)
|
|
170
|
+
provides a useful perspective on recall interference. Its neural-layer capacity
|
|
171
|
+
experiments do not establish limits for a text database. This implementation
|
|
172
|
+
applies selective retrieval at the application layer without changing model
|
|
173
|
+
weights. Merely applying softmax to an existing score list preserves ordering
|
|
174
|
+
and would not improve fixed-top-k selection.
|
package/docs/operating.md
CHANGED
|
@@ -32,6 +32,11 @@ sync command but never runs it automatically. This guide covers all three.
|
|
|
32
32
|
- Optional: an OpenAI-compatible LLM endpoint for memory extraction and
|
|
33
33
|
embedding-ranked search. Without it, memory search degrades to substring
|
|
34
34
|
matching — everything else still works.
|
|
35
|
+
Explicit `memory_search` calls with `strategy: "hybrid"` additionally support
|
|
36
|
+
lexical/identifier admission without a provider, bounded `query_aliases`,
|
|
37
|
+
optional explanations and empty results. Semantic-only admission requires an
|
|
38
|
+
explicit positive `min_score`; see [memory retrieval](memory-retrieval.md)
|
|
39
|
+
for model-specific threshold limits and the Markdown comparison.
|
|
35
40
|
- Optional: rootless Podman for the containerized memory server and isolated
|
|
36
41
|
workspace base described in [Containers](containers.md).
|
|
37
42
|
|
package/docs/releasing.md
CHANGED
|
@@ -66,6 +66,13 @@ or moved. If the tree-addressed candidate has expired, rerun CI for the exact
|
|
|
66
66
|
release commit before retrying publication; the workflow never substitutes a
|
|
67
67
|
different version or tree.
|
|
68
68
|
|
|
69
|
+
CI job retries can also regenerate the same tree-addressed candidate. Its
|
|
70
|
+
upload explicitly replaces an existing artifact of that name within the same
|
|
71
|
+
workflow run, only after all platform/runtime dependencies succeed. This avoids
|
|
72
|
+
an artifact-name conflict after a completed job is retried; it does not move
|
|
73
|
+
versioned image tags, republish npm versions or replace GitHub Release assets.
|
|
74
|
+
Candidate checksums and tree/version binding are verified as usual.
|
|
75
|
+
|
|
69
76
|
## Repository configuration
|
|
70
77
|
|
|
71
78
|
### Repeatable post-publication verification
|
package/docs/storage.md
CHANGED
|
@@ -48,6 +48,11 @@ services used to process/search memory. They do not change where the SQLite
|
|
|
48
48
|
databases are stored. Git-provider and routing configuration do not change
|
|
49
49
|
storage either.
|
|
50
50
|
|
|
51
|
+
`memory_search` can optionally select [hybrid retrieval](memory-retrieval.md).
|
|
52
|
+
This changes read-time ranking only: aliases are request hints, explanations
|
|
53
|
+
are returned to the caller without a new log, and original memory text remains
|
|
54
|
+
authoritative. No new store, migration or automatic memory capture is introduced.
|
|
55
|
+
|
|
51
56
|
Trajectory storage is deliberately separate from MCP memory. The harness hook
|
|
52
57
|
writes `.agentfs/trajectory.db` on the client machine where Claude Code,
|
|
53
58
|
OpenCode, or Pi is running; it is never redirected to the remote HTTP memory server.
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"identifier":"cairn-offline-fake-v1","package_version":"2.
|
|
1
|
+
{"identifier":"cairn-offline-fake-v1","package_version":"2.23.0","schema_version":1,"task_set_digest":"13e95379f35839593047ec34521a65a8fa3ab92bd3993bca9dc8bf5c4a9ff36f"}
|
|
@@ -25,6 +25,7 @@ import { profileAllowsTool, resolveMcpToolProfile } from "./mcp-tool-profile.js"
|
|
|
25
25
|
import { isWorkEvidenceEnabled } from "./work-evidence-schema.js";
|
|
26
26
|
import { extractMemoryCandidates as extractMemoryCandidatesFromProvider } from "./memory-extraction.js";
|
|
27
27
|
import { applyReviewedMemory, invalidateReviewedMemory } from "./reviewed-memory-store.js";
|
|
28
|
+
import { rankHybridMemory } from "./memory-retrieval.js";
|
|
28
29
|
const MINIMUM_NODE_MAJOR = 22;
|
|
29
30
|
const nodeMajor = Number.parseInt(process.versions.node, 10);
|
|
30
31
|
if (!Number.isInteger(nodeMajor) || nodeMajor < MINIMUM_NODE_MAJOR) {
|
|
@@ -377,7 +378,14 @@ async function semanticSearch(scope, query, topK, minScore, config = getMemoryCo
|
|
|
377
378
|
entries: filterEntries(await listEntries(candidate, "", { projectId, typed: options.typed }), options.filters ?? {}),
|
|
378
379
|
})));
|
|
379
380
|
const allEntries = perScopeEntries.flatMap((group) => group.entries);
|
|
381
|
+
const hybrid = (embeddingStatus, semanticScores) => ({
|
|
382
|
+
...rankHybridMemory(allEntries, query, { topK, minScore, aliases: options.aliases, explain: options.explain, embeddingStatus, semanticScores }),
|
|
383
|
+
mode: "hybrid",
|
|
384
|
+
...(embeddingConfig ? { model: embeddingConfig.model } : {}),
|
|
385
|
+
});
|
|
380
386
|
if (allEntries.length === 0) {
|
|
387
|
+
if (options.strategy === "hybrid")
|
|
388
|
+
return hybrid(embeddingConfig ? "not_used" : "unconfigured");
|
|
381
389
|
return { results: [], mode: embeddingConfig ? "semantic" : "substring" };
|
|
382
390
|
}
|
|
383
391
|
const substringFallback = () => {
|
|
@@ -390,6 +398,8 @@ async function semanticSearch(scope, query, topK, minScore, config = getMemoryCo
|
|
|
390
398
|
};
|
|
391
399
|
};
|
|
392
400
|
if (!embeddingConfig) {
|
|
401
|
+
if (options.strategy === "hybrid")
|
|
402
|
+
return hybrid("unconfigured");
|
|
393
403
|
return substringFallback();
|
|
394
404
|
}
|
|
395
405
|
try {
|
|
@@ -422,9 +432,23 @@ async function semanticSearch(scope, query, topK, minScore, config = getMemoryCo
|
|
|
422
432
|
}
|
|
423
433
|
}
|
|
424
434
|
const [queryVector] = await embedTexts(embeddingConfig, [query]);
|
|
435
|
+
if (options.strategy === "hybrid") {
|
|
436
|
+
const validVector = (vector) => {
|
|
437
|
+
if (!vector?.length || vector.length > 65536 || vector.some(value => !Number.isFinite(value)))
|
|
438
|
+
return false;
|
|
439
|
+
const squaredNorm = vector.reduce((total, value) => total + value * value, 0);
|
|
440
|
+
return squaredNorm > 0 && Number.isFinite(squaredNorm);
|
|
441
|
+
};
|
|
442
|
+
if (!validVector(queryVector) || allEntries.some(entry => !validVector(vectors.get(entry)) || vectors.get(entry).length !== queryVector.length)) {
|
|
443
|
+
throw new Error("Embedding vectors are incomplete or invalid.");
|
|
444
|
+
}
|
|
445
|
+
}
|
|
425
446
|
for (const cache of caches) {
|
|
426
447
|
cache.save();
|
|
427
448
|
}
|
|
449
|
+
if (options.strategy === "hybrid") {
|
|
450
|
+
return hybrid("available", new Map(allEntries.map(entry => [entry, Math.max(-1, Math.min(1, cosineSimilarity(queryVector, vectors.get(entry))))])));
|
|
451
|
+
}
|
|
428
452
|
const exact = options.typed
|
|
429
453
|
? allEntries.filter((entry) => isExactTextHit(entry, query)).sort(compareMemoryEntries).map((entry) => ({ ...entry, score: 1 }))
|
|
430
454
|
: [];
|
|
@@ -444,6 +468,8 @@ async function semanticSearch(scope, query, topK, minScore, config = getMemoryCo
|
|
|
444
468
|
}
|
|
445
469
|
catch {
|
|
446
470
|
// Embedding endpoint failure — degrade gracefully to substring matching.
|
|
471
|
+
if (options.strategy === "hybrid")
|
|
472
|
+
return hybrid("failed");
|
|
447
473
|
return substringFallback();
|
|
448
474
|
}
|
|
449
475
|
}
|
|
@@ -1112,6 +1138,9 @@ function buildMemoryServer(context, capabilitySnapshot, withCapability, toolProf
|
|
|
1112
1138
|
query: z.string().min(1),
|
|
1113
1139
|
top_k: z.number().int().min(1).max(50).optional(),
|
|
1114
1140
|
min_score: z.number().min(0).max(1).optional(),
|
|
1141
|
+
strategy: z.enum(["legacy", "hybrid"]).optional().describe("Default legacy preserves current retrieval. Hybrid admits lexical/identifier matches and semantic-only matches above an explicit positive min_score."),
|
|
1142
|
+
query_aliases: z.array(z.string().trim().min(1).max(256)).max(8).optional().describe("Explicit alternative query phrases for hybrid lexical retrieval; never widens scopes."),
|
|
1143
|
+
explain: z.boolean().optional().describe("Include hybrid match reasons, provenance locators and possible conflicting live values."),
|
|
1115
1144
|
...(typedNodesEnabled ? {
|
|
1116
1145
|
address_space: NODE_ADDRESS_SPACE_SCHEMA.optional(),
|
|
1117
1146
|
node_types: z.array(nodeTypeSchema).min(1).optional(),
|
|
@@ -1124,8 +1153,17 @@ function buildMemoryServer(context, capabilitySnapshot, withCapability, toolProf
|
|
|
1124
1153
|
},
|
|
1125
1154
|
}, async (input) => {
|
|
1126
1155
|
const { scope, query, top_k, min_score } = input;
|
|
1156
|
+
if (input.strategy !== "hybrid" && (input.query_aliases !== undefined || input.explain !== undefined)) {
|
|
1157
|
+
throw new Error("query_aliases and explain require strategy='hybrid'.");
|
|
1158
|
+
}
|
|
1159
|
+
if (input.strategy === "hybrid") {
|
|
1160
|
+
// Validate bounds before opening scopes or contacting an embedding service.
|
|
1161
|
+
rankHybridMemory([], query, { topK: top_k ?? 8, minScore: min_score ?? 0, aliases: input.query_aliases });
|
|
1162
|
+
}
|
|
1127
1163
|
const addressSpace = "address_space" in input && input.address_space !== undefined ? NODE_ADDRESS_SPACE_SCHEMA.parse(input.address_space) : undefined;
|
|
1128
1164
|
if (addressSpace && addressSpace !== "memory") {
|
|
1165
|
+
if (input.strategy === "hybrid")
|
|
1166
|
+
throw new Error("Hybrid retrieval currently supports the memory address space only.");
|
|
1129
1167
|
const noteStore = await import("./note-store.js");
|
|
1130
1168
|
const entries = noteStore.searchAddressedNotes({ ...noteTargetOptions(scope, addressSpace), query, ...memoryFiltersFromInput(input) }).slice(0, top_k ?? 8);
|
|
1131
1169
|
const results = entries.map((entry) => ({ ...entry, score: 1 }));
|
|
@@ -1135,11 +1173,14 @@ function buildMemoryServer(context, capabilitySnapshot, withCapability, toolProf
|
|
|
1135
1173
|
structuredContent: payload,
|
|
1136
1174
|
};
|
|
1137
1175
|
}
|
|
1138
|
-
const { results, mode, model } = await semanticSearch(scope, query, top_k ?? 8, min_score ?? 0, memoryConfig(), context.projectId, {
|
|
1176
|
+
const { results, mode, model, retrieval } = await semanticSearch(scope, query, top_k ?? 8, min_score ?? 0, memoryConfig(), context.projectId, {
|
|
1139
1177
|
typed: typedNodesEnabled,
|
|
1140
1178
|
filters: typedNodesEnabled ? memoryFiltersFromInput(input) : {},
|
|
1179
|
+
strategy: input.strategy,
|
|
1180
|
+
aliases: input.query_aliases,
|
|
1181
|
+
explain: input.explain,
|
|
1141
1182
|
});
|
|
1142
|
-
const payload = { mode, model, count: results.length, results };
|
|
1183
|
+
const payload = { mode, model, count: results.length, results, ...(retrieval ? { retrieval } : {}) };
|
|
1143
1184
|
return {
|
|
1144
1185
|
content: [{ type: "text", text: asToolText(payload) }],
|
|
1145
1186
|
structuredContent: payload,
|
|
@@ -0,0 +1,128 @@
|
|
|
1
|
+
const STOP = new Set('a an and are as at be by can do does for from how i in is it me my of on or our please that the their this to use we what when where which with would you your un una uno il lo la gli le di da del della e che con come per posso mio mia sul nel dei'.split(' '));
|
|
2
|
+
const normalize = (value) => value.normalize('NFKC').toLowerCase();
|
|
3
|
+
const compare = (left, right) => left.scope.localeCompare(right.scope, 'en') || left.key.localeCompare(right.key, 'en');
|
|
4
|
+
const rounded = (value) => Math.round(value * 1e6) / 1e6;
|
|
5
|
+
export function retrievalTerms(text) {
|
|
6
|
+
const result = [];
|
|
7
|
+
for (const match of normalize(text).matchAll(/[\p{L}\p{N}]+/gu)) {
|
|
8
|
+
if (match[0].length > 1 && !STOP.has(match[0]))
|
|
9
|
+
result.push(match[0]);
|
|
10
|
+
if (result.length === 4096)
|
|
11
|
+
break;
|
|
12
|
+
}
|
|
13
|
+
return result;
|
|
14
|
+
}
|
|
15
|
+
function identifiers(text) {
|
|
16
|
+
// Match ordinary words too, then select identifiers. Requiring a separator
|
|
17
|
+
// in an unanchored greedy expression repeatedly rescans long plain words.
|
|
18
|
+
const result = new Set();
|
|
19
|
+
for (const match of text.matchAll(/[\p{L}\p{N}]+(?:[._:/-][\p{L}\p{N}]+)*/gu)) {
|
|
20
|
+
if (/[._:/\d-]/u.test(match[0]))
|
|
21
|
+
result.add(match[0]);
|
|
22
|
+
}
|
|
23
|
+
return result;
|
|
24
|
+
}
|
|
25
|
+
function variant(text) {
|
|
26
|
+
const normalized = normalize(text).trim();
|
|
27
|
+
return { normalized, terms: [...new Set(retrievalTerms(text))].slice(0, 64), identifiers: identifiers(normalized) };
|
|
28
|
+
}
|
|
29
|
+
/** BM25 is also used by the transparent Markdown-folder baseline. */
|
|
30
|
+
export function lexicalMemoryScores(entries, query) {
|
|
31
|
+
const queryTerms = [...new Set(retrievalTerms(query))].slice(0, 64);
|
|
32
|
+
const docs = entries.map(entry => [
|
|
33
|
+
...retrievalTerms(entry.key), ...retrievalTerms(entry.key),
|
|
34
|
+
...retrievalTerms(entry.value), ...retrievalTerms([entry.node_type, ...(entry.tags ?? [])].filter(Boolean).join(' ')),
|
|
35
|
+
]);
|
|
36
|
+
const frequencies = docs.map(terms => {
|
|
37
|
+
const counts = new Map();
|
|
38
|
+
for (const term of terms)
|
|
39
|
+
counts.set(term, (counts.get(term) ?? 0) + 1);
|
|
40
|
+
return counts;
|
|
41
|
+
});
|
|
42
|
+
const averageLength = docs.reduce((total, terms) => total + terms.length, 0) / Math.max(1, docs.length) || 1;
|
|
43
|
+
const documentFrequencies = new Map(queryTerms.map(term => [term, frequencies.filter(counts => counts.has(term)).length]));
|
|
44
|
+
return docs.map((terms, index) => queryTerms.reduce((score, term) => {
|
|
45
|
+
const frequency = frequencies[index].get(term) ?? 0;
|
|
46
|
+
const df = documentFrequencies.get(term) ?? 0;
|
|
47
|
+
const idf = Math.log(1 + (entries.length - df + 0.5) / (df + 0.5));
|
|
48
|
+
return score + idf * frequency * 2.2 / (frequency + 1.2 * (0.25 + 0.75 * terms.length / averageLength));
|
|
49
|
+
}, 0));
|
|
50
|
+
}
|
|
51
|
+
export function rankHybridMemory(entries, query, options) {
|
|
52
|
+
const aliases = options.aliases ?? [];
|
|
53
|
+
if (Buffer.byteLength(query, 'utf8') > 4096 || !query.trim() || aliases.length > 8
|
|
54
|
+
|| aliases.some(alias => !alias.trim() || Buffer.byteLength(alias, 'utf8') > 256)
|
|
55
|
+
|| !Number.isInteger(options.topK) || options.topK < 1 || options.topK > 50
|
|
56
|
+
|| !Number.isFinite(options.minScore) || options.minScore < 0 || options.minScore > 1) {
|
|
57
|
+
throw new Error('Hybrid memory search requires bounded query, aliases, result count and similarity threshold.');
|
|
58
|
+
}
|
|
59
|
+
const variants = [...new Set([query, ...aliases].map(text => normalize(text).trim()))].map(variant);
|
|
60
|
+
const lexicalScores = variants.map(item => lexicalMemoryScores(entries, item.normalized));
|
|
61
|
+
const candidates = entries.map((entry, index) => {
|
|
62
|
+
const text = normalize([entry.key, entry.value, entry.node_type, ...(entry.tags ?? [])].filter(Boolean).join('\n'));
|
|
63
|
+
const words = new Set(retrievalTerms(text));
|
|
64
|
+
const entryIdentifiers = identifiers(text);
|
|
65
|
+
const matches = variants.map((item, v) => {
|
|
66
|
+
const matched = item.terms.filter(term => words.has(term));
|
|
67
|
+
const coverage = item.terms.length ? matched.length / item.terms.length : 0;
|
|
68
|
+
const identifierMatch = [...item.identifiers].every(id => entryIdentifiers.has(id));
|
|
69
|
+
const exactKey = normalize(entry.key) === item.normalized;
|
|
70
|
+
return {
|
|
71
|
+
matched, coverage, identifierMatch, exactKey,
|
|
72
|
+
eligible: identifierMatch && (exactKey || (coverage >= 0.6 && matched.length >= Math.min(2, item.terms.length) && matched.length > 0)),
|
|
73
|
+
lexical: lexicalScores[v][index],
|
|
74
|
+
};
|
|
75
|
+
});
|
|
76
|
+
const best = matches.filter(match => match.eligible).sort((a, b) => Number(b.exactKey) - Number(a.exactKey) || b.lexical - a.lexical)[0];
|
|
77
|
+
const rawSemantic = options.semanticScores?.get(entry);
|
|
78
|
+
const semantic = typeof rawSemantic === 'number' && Number.isFinite(rawSemantic) && rawSemantic >= -1 && rawSemantic <= 1 ? rawSemantic : null;
|
|
79
|
+
const semanticAdmission = options.minScore > 0 && semantic !== null && semantic >= options.minScore && matches.some(match => match.identifierMatch);
|
|
80
|
+
const lexical = best?.lexical ?? 0;
|
|
81
|
+
const score = 0.8 * lexical / (1 + lexical) + 0.2 * Math.max(0, semantic ?? 0);
|
|
82
|
+
return { entry, best, semantic, semanticAdmission, score };
|
|
83
|
+
}).filter(item => item.best || item.semanticAdmission)
|
|
84
|
+
.sort((a, b) => Number(Boolean(b.best?.exactKey)) - Number(Boolean(a.best?.exactKey)) || b.score - a.score || compare(a.entry, b.entry));
|
|
85
|
+
const grouped = new Map();
|
|
86
|
+
for (const candidate of candidates) {
|
|
87
|
+
const key = JSON.stringify([candidate.entry.key, candidate.entry.value]);
|
|
88
|
+
const group = grouped.get(key) ?? [];
|
|
89
|
+
group.push(candidate);
|
|
90
|
+
grouped.set(key, group);
|
|
91
|
+
}
|
|
92
|
+
const results = [...grouped.values()].slice(0, options.topK).map(group => {
|
|
93
|
+
const item = group[0];
|
|
94
|
+
const reasons = [
|
|
95
|
+
...(item.best?.exactKey ? ['exact-key'] : []),
|
|
96
|
+
...(item.best ? ['lexical-coverage'] : []),
|
|
97
|
+
...(item.semanticAdmission ? ['semantic-threshold'] : []),
|
|
98
|
+
...(group.length > 1 ? ['identical-records-collapsed'] : []),
|
|
99
|
+
];
|
|
100
|
+
return {
|
|
101
|
+
...item.entry, score: rounded(item.score),
|
|
102
|
+
...(options.explain ? { match: {
|
|
103
|
+
reasons, matched_terms: item.best?.matched ?? [], lexical_coverage: rounded(item.best?.coverage ?? 0),
|
|
104
|
+
semantic_score: item.semantic === null ? null : rounded(item.semantic),
|
|
105
|
+
sources: group.map(({ entry }) => ({ scope: entry.scope, key: entry.key })).sort(compare),
|
|
106
|
+
} } : {}),
|
|
107
|
+
};
|
|
108
|
+
});
|
|
109
|
+
const conflicts = [...new Set(results.map(item => item.key))].flatMap(key => {
|
|
110
|
+
const relevant = candidates.filter(item => item.entry.key === key);
|
|
111
|
+
return new Set(relevant.map(item => item.entry.value)).size > 1
|
|
112
|
+
? [{ key, scopes: [...new Set(relevant.map(item => item.entry.scope))].sort() }] : [];
|
|
113
|
+
});
|
|
114
|
+
return {
|
|
115
|
+
results,
|
|
116
|
+
retrieval: {
|
|
117
|
+
strategy: 'hybrid',
|
|
118
|
+
score_kind: 'ranking-not-confidence',
|
|
119
|
+
embedding_status: options.embeddingStatus ?? (options.semanticScores ? 'available' : 'unconfigured'),
|
|
120
|
+
semantic_min_score: options.minScore > 0 ? options.minScore : null,
|
|
121
|
+
alias_count: aliases.length,
|
|
122
|
+
candidates: entries.length,
|
|
123
|
+
admitted: grouped.size,
|
|
124
|
+
...(results.length === 0 ? { empty_reason: entries.length ? 'no_relevant_memory' : 'empty_scope' } : {}),
|
|
125
|
+
...(options.explain ? { possible_conflicts: conflicts } : {}),
|
|
126
|
+
},
|
|
127
|
+
};
|
|
128
|
+
}
|
|
@@ -15,6 +15,8 @@
|
|
|
15
15
|
"check:embeddings": "node scripts/smoke-embeddings.mjs",
|
|
16
16
|
"check:extract": "node scripts/smoke-extract-cli.mjs",
|
|
17
17
|
"check:search": "node scripts/smoke-search-e2e.mjs",
|
|
18
|
+
"check:memory-retrieval": "node scripts/smoke-memory-retrieval.mjs && node scripts/smoke-memory-retrieval-mcp.mjs && node scripts/run-memory-retrieval-study.mjs --check --summary",
|
|
19
|
+
"study:memory-retrieval": "npm run build && node scripts/run-memory-retrieval-study.mjs --summary",
|
|
18
20
|
"check:scope-guard": "node scripts/smoke-scope-guard.mjs",
|
|
19
21
|
"check:http-guard": "node scripts/smoke-http-guard.mjs",
|
|
20
22
|
"check:remote-context": "node scripts/smoke-remote-context.mjs",
|
|
@@ -25,7 +25,14 @@ Read order (canonical sources still win over derived layers):
|
|
|
25
25
|
|
|
26
26
|
<process>
|
|
27
27
|
## 1. AgentFS
|
|
28
|
-
`cairn-memory_memory_search` scope `project` with $ARGUMENTS.
|
|
28
|
+
`cairn-memory_memory_search` scope `project` with $ARGUMENTS. When the discovered
|
|
29
|
+
schema supports it, use `strategy: "hybrid"` for identifier-sensitive or
|
|
30
|
+
mixed-wording queries and `explain: true` to inspect matches. Supply only aliases
|
|
31
|
+
already known from the task. Semantic-only hybrid matches require an explicitly
|
|
32
|
+
chosen positive `min_score`; retain the existing semantic strategy for paraphrases
|
|
33
|
+
when no model-specific threshold is available. Empty results are valid.
|
|
34
|
+
Use broader configured scopes only when the question genuinely spans them;
|
|
35
|
+
do not automatically widen a search merely to force a hit.
|
|
29
36
|
|
|
30
37
|
## 2. Wiki
|
|
31
38
|
Read `.planning/wiki/index.md` if present and follow links relevant to $ARGUMENTS.
|
package/package.json
CHANGED
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
set -euo pipefail
|
|
3
|
+
ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)
|
|
4
|
+
cd "$ROOT"
|
|
5
|
+
for path in docs/memory-retrieval.md docs/benchmarks/memory-retrieval-v1.json docs/learning/video-scripts/V04-memory-retrieval-supplement.md; do
|
|
6
|
+
[[ -f "$path" ]]
|
|
7
|
+
done
|
|
8
|
+
for path in docs/README.md docs/operating.md docs/storage.md docs/agents.md docs/learning/lessons/L04-memory-fundamentals.md; do
|
|
9
|
+
grep -qF 'memory-retrieval.md' "$path"
|
|
10
|
+
done
|
|
11
|
+
for path in claude/commands/recall.md opencode/command/recall.md; do
|
|
12
|
+
grep -qF 'do not automatically widen' "$path"
|
|
13
|
+
grep -qF 'strategy: "hybrid"' "$path"
|
|
14
|
+
done
|
|
15
|
+
grep -qF 'same admission' docs/memory-retrieval.md
|
|
16
|
+
grep -qF 'HTTP 502' docs/memory-retrieval.md
|
|
17
|
+
grep -qF 'ranking-not-confidence' docs/memory-retrieval.md
|
|
18
|
+
grep -qF 'Status: available since Cairnkeep 2.23.0' docs/memory-retrieval.md
|
|
19
|
+
echo 'PASS: hybrid retrieval, Markdown parity, provider limitations and learning/recording references'
|
|
@@ -14,6 +14,8 @@ fail() {
|
|
|
14
14
|
|
|
15
15
|
node - "$ROOT/package.json" "$ROOT/package-lock.json" "$CANDIDATE_WORKFLOW" "$PUBLISH_WORKFLOW" <<'NODE'
|
|
16
16
|
const fs = require("node:fs");
|
|
17
|
+
const assert = require("node:assert/strict");
|
|
18
|
+
const { parse } = require("yaml");
|
|
17
19
|
|
|
18
20
|
const manifest = JSON.parse(fs.readFileSync(process.argv[2], "utf8"));
|
|
19
21
|
const lock = JSON.parse(fs.readFileSync(process.argv[3], "utf8"));
|
|
@@ -36,6 +38,36 @@ if (!candidateWorkflow.includes("release-candidate:")
|
|
|
36
38
|
|| !candidateWorkflow.includes("name: release-candidate-${{ steps.candidate.outputs.tree }}")) {
|
|
37
39
|
throw new Error("CI must build and upload a tree-addressed release candidate after its required jobs");
|
|
38
40
|
}
|
|
41
|
+
function validateCandidateRetry(workflow) {
|
|
42
|
+
const job = workflow.jobs["release-candidate"];
|
|
43
|
+
assert.deepEqual(new Set(job.needs), new Set([
|
|
44
|
+
"memory-server", "native-windows", "repository", "shell-portability",
|
|
45
|
+
"server-boot", "container", "fresh-install-macos",
|
|
46
|
+
]), "candidate retry must retain every platform/runtime gate");
|
|
47
|
+
assert.equal(job.if, undefined, "candidate retry must not bypass failed or pending gates");
|
|
48
|
+
const uploads = job.steps.filter(step => step.uses?.startsWith("actions/upload-artifact@"));
|
|
49
|
+
assert.equal(uploads.length, 1, "candidate retry must have one artifact publication");
|
|
50
|
+
assert.equal(uploads[0].with.name, "release-candidate-${{ steps.candidate.outputs.tree }}");
|
|
51
|
+
assert.equal(uploads[0].with.overwrite, true, "candidate upload must support same-run retries");
|
|
52
|
+
assert.equal(uploads[0].with["if-no-files-found"], "error");
|
|
53
|
+
}
|
|
54
|
+
const candidate = parse(candidateWorkflow);
|
|
55
|
+
const positive = structuredClone(candidate);
|
|
56
|
+
positive.jobs["release-candidate"].steps.find(step => step.uses?.startsWith("actions/upload-artifact@")).with.overwrite = true;
|
|
57
|
+
validateCandidateRetry(positive);
|
|
58
|
+
for (const overwrite of [undefined, false]) {
|
|
59
|
+
const invalid = structuredClone(positive);
|
|
60
|
+
invalid.jobs["release-candidate"].steps.find(step => step.uses?.startsWith("actions/upload-artifact@")).with.overwrite = overwrite;
|
|
61
|
+
assert.throws(() => validateCandidateRetry(invalid), /same-run retries/);
|
|
62
|
+
}
|
|
63
|
+
const missingGate = structuredClone(positive);
|
|
64
|
+
missingGate.jobs["release-candidate"].needs = missingGate.jobs["release-candidate"].needs.filter(name => name !== "native-windows");
|
|
65
|
+
assert.throws(() => validateCandidateRetry(missingGate), /every platform\/runtime gate/);
|
|
66
|
+
const bypass = structuredClone(positive);
|
|
67
|
+
bypass.jobs["release-candidate"].if = "always()";
|
|
68
|
+
assert.throws(() => validateCandidateRetry(bypass), /must not bypass/);
|
|
69
|
+
validateCandidateRetry(candidate);
|
|
70
|
+
console.log("PASS: retry-safe candidate upload and four gate/replacement mutation controls");
|
|
39
71
|
if (!publishWorkflow.includes("- name: Download the CI-verified release candidate")
|
|
40
72
|
|| !publishWorkflow.includes("jq -r '.conclusion'")
|
|
41
73
|
|| !publishWorkflow.includes("== success")
|
|
@@ -10,6 +10,11 @@ inventory, or search commands such as `pwd`, `ls`, `rg --files`, `find`,
|
|
|
10
10
|
- If `memory_search` is available, derive one short query from the task and
|
|
11
11
|
search `scope: project` once. Derive it from the user's task; do not inspect
|
|
12
12
|
the repository first merely to formulate the query.
|
|
13
|
+
- When the discovered schema supports hybrid search, use it for explicit
|
|
14
|
+
identifiers or mixed wording; request explanations when needed. Supply only
|
|
15
|
+
aliases already known from the task. Hybrid semantic-only matches require an
|
|
16
|
+
explicit model-specific positive similarity threshold; otherwise keep the
|
|
17
|
+
existing semantic strategy for paraphrases. Empty results are acceptable.
|
|
13
18
|
- Treat returned memory as a locator, not authority. Read and verify the
|
|
14
19
|
maintained repository sources it references before implementing a result.
|
|
15
20
|
- If the tool is unavailable or no relevant result exists, continue with
|