@cairnkeep/cli 2.23.0 → 2.24.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (44) hide show
  1. package/CHANGELOG.md +49 -0
  2. package/docs/containers.md +4 -0
  3. package/docs/domain-knowledge.md +42 -4
  4. package/docs/learning/CURRICULUM-MAP.md +3 -3
  5. package/docs/learning/FEATURE-GUIDE.md +8 -1
  6. package/docs/learning/lessons/L00-why-cairnkeep.md +1 -1
  7. package/docs/learning/lessons/L01-safe-trial.md +3 -3
  8. package/docs/learning/lessons/L02-installation.md +2 -2
  9. package/docs/learning/lessons/L03-first-project.md +1 -1
  10. package/docs/learning/lessons/L04-memory-fundamentals.md +12 -0
  11. package/docs/learning/lessons/L09-domain-knowledge.md +11 -0
  12. package/docs/learning/lessons/L19-validated-skill-improvement.md +1 -1
  13. package/docs/learning/lessons/L20-mcp-tool-profiles.md +1 -1
  14. package/docs/learning/lessons/L21-context-packs.md +1 -1
  15. package/docs/learning/lessons/L22-native-windows.md +2 -2
  16. package/docs/learning/lessons/L23-guided-setup.md +1 -1
  17. package/docs/learning/lessons/L24-okf-exchange.md +1 -1
  18. package/docs/learning/lessons/L25-playbooks.md +1 -1
  19. package/docs/learning/lessons/L26-context-intelligence.md +1 -1
  20. package/docs/learning/lessons/L27-ecosystem-evidence.md +1 -1
  21. package/docs/learning/lessons/L28-security-assurance.md +2 -2
  22. package/docs/learning/video-scripts/V04-memory-retrieval-supplement.md +13 -0
  23. package/docs/learning/video-scripts/V09-workspace-routing-supplement.md +36 -0
  24. package/docs/mcp-tool-profiles.md +7 -0
  25. package/docs/memory-retrieval.md +84 -2
  26. package/docs/operating.md +5 -0
  27. package/docs/privacy-and-data-flow.md +13 -4
  28. package/examples/eval/bundled-fake.json +1 -1
  29. package/mcp-memory-server/dist/anythingllm-workspace.js +109 -0
  30. package/mcp-memory-server/dist/index.js +41 -6
  31. package/mcp-memory-server/dist/mcp-tool-catalog.js +4 -1
  32. package/mcp-memory-server/dist/memory-reranker.js +109 -0
  33. package/mcp-memory-server/package.json +4 -2
  34. package/package.json +2 -2
  35. package/scripts/doctor.sh +17 -0
  36. package/scripts/probe-memory-reranker.mjs +22 -0
  37. package/scripts/test-container-health.mjs +2 -0
  38. package/scripts/test-container.sh +21 -1
  39. package/scripts/test-dependency-security.mjs +50 -0
  40. package/scripts/test-dependency-security.sh +4 -0
  41. package/scripts/test-doctor.sh +28 -0
  42. package/scripts/test-windows-native.mjs +5 -0
  43. package/scripts/verify-dependency-security.mjs +22 -2
  44. package/scripts/windows-platform.mjs +8 -0
package/CHANGELOG.md CHANGED
@@ -5,6 +5,55 @@ All notable user-facing changes are documented here. This project follows
5
5
 
6
6
  ## [Unreleased]
7
7
 
8
+ ## [2.24.0] - 2026-10-24
9
+
10
+ ### Memory retrieval
11
+
12
+ - Add optional provider-neutral learned reranking for ordinary memory search,
13
+ with explicit endpoint/model consent and a separate optional credential.
14
+ Preserve existing scope, current-value, typed and relevance admission;
15
+ bound candidate text, streamed responses and deadlines. Provider failures
16
+ preserve the original ranking. Exact keys, including explicit aliases,
17
+ retain priority. Default search behavior stays unchanged.
18
+ - Add functional POSIX/native-Windows diagnostics, real MCP isolation/fallback
19
+ controls and a frozen synthetic provider comparison. Document both improved
20
+ first-result ordering and the regression; do not claim better general recall
21
+ or agent productivity. Update retrieval/privacy guidance and L04/V04 material.
22
+ - Preserve frozen fixture bytes on Windows and verify linked-install probes.
23
+ No new runtime dependency, storage migration, automatic provider discovery
24
+ or team-feature activation is included.
25
+
26
+ ### Security
27
+
28
+ - Refresh compatible transitive HTTP parser, URI and IP-address dependencies
29
+ in both lock graphs. Enforce the patched minimum versions, including nested
30
+ copies, with dependency-policy regression controls.
31
+ - Normalize the container health-check directory's traversal permissions after
32
+ copying its probe. Verify the legacy entrypoint imports as the runtime UID;
33
+ do not count module-load failures as successful credential rejection.
34
+
35
+ ## [2.23.1] - 2026-10-22
36
+
37
+ ### Domain knowledge
38
+
39
+ - Resolve local AnythingLLM defaults from the existing absolute project-path
40
+ mapping, with deepest-project matching and the sync script's discovery order.
41
+ Preserve explicit workspace overrides, legacy fallback, remote HTTP session
42
+ routing, OpenViking selection and synchronization approval requirements.
43
+ - Bound mapping reads and reject unsafe files, invalid slugs and ambiguous
44
+ canonical project declarations. Add pure, real stdio, HTTP and synchronization
45
+ regression controls; update operating guidance and the recording supplement.
46
+
47
+ ### Maintenance
48
+
49
+ - Allocate the remote-routing test provider's loopback port dynamically; retain
50
+ the formerly failing PID as a regression instead of hitting fetch's blocked
51
+ port 10080. Production routing and transport policy are unchanged.
52
+ - Refresh pinned setup, artifact-upload and code-scanning actions; align the
53
+ packaged CLI and server on Zod 4.6.7 with dependency-parity mutation controls.
54
+ - Update the isolated historical Python MCP integration experiment to 2.2.0;
55
+ this does not add Python or that experiment to the shipped Node runtime.
56
+
8
57
  ## [2.23.0] - 2026-10-21
9
58
 
10
59
  ### Memory retrieval
@@ -86,6 +86,10 @@ configuration fails without issuing a request; probes target only
86
86
  `127.0.0.1` and refuse redirects. The existing
87
87
  `/usr/local/lib/cairnkeep/container-healthcheck.mjs` command remains available
88
88
  as an alias to the packaged probe. Stdio mode requires no HTTP probe.
89
+ The image explicitly makes the probe directory searchable by its unprivileged
90
+ runtime user after copying the file. The OCI regression imports that legacy
91
+ entrypoint as UID 10001 before testing credentials, so a missing or inaccessible
92
+ module cannot be mistaken for a successful authentication rejection.
89
93
  File-based probes require a private token readable by the container user;
90
94
  world-readable secret mounts fail closed. The Quadlet example explicitly mounts
91
95
  its secret as UID/GID 10001 with mode `0400`. For other engines, verify the
@@ -29,7 +29,7 @@ work exactly the same without them.
29
29
  | `ANYTHINGLLM_API_KEY` | **Required** for the domain-knowledge tools. Unset → the tools error at call time (everything else works). |
30
30
  | `ANYTHINGLLM_BASE_URL` | AnythingLLM base URL. Default `http://localhost:3001`. |
31
31
  | `CAIRN_ANYTHINGLLM_SYNC_SCRIPT` | Optional path to a custom document-sync script. Unset -> `domain_knowledge_sync` uses the bundled deployment-neutral example. |
32
- | `CAIRN_ANYTHINGLLM_PROJECTS_FILE` | Optional config path for the bundled sync script. Default `${XDG_CONFIG_HOME:-~/.config}/cairnkeep/anythingllm-projects.json`. |
32
+ | `CAIRN_ANYTHINGLLM_PROJECTS_FILE` | Optional config path for the bundled sync script, also used to pick the default workspace per repository. Default `${XDG_CONFIG_HOME:-~/.config}/cairnkeep/anythingllm-projects.json`. |
33
33
  | `CAIRN_ANYTHINGLLM_STATE_FILE` | Optional state path for the bundled sync script. Default `${XDG_STATE_HOME:-~/.local/state}/cairnkeep/anythingllm-sync.json`. |
34
34
 
35
35
  The query security follow-up validates HTTP(S) URLs and rejects embedded
@@ -54,9 +54,43 @@ first of `.agent/memory.json`, `.opencode/memory.json`, `.claude/memory.json`, o
54
54
  }
55
55
  ```
56
56
 
57
- `domain_knowledge_query` takes an explicit `workspace`, or falls back to the first
58
- configured workspace that is not `engineering-patterns` (the conventional shared
59
- workspace). With no workspace given and none configured, it errors.
57
+ ### Which workspace a call targets
58
+
59
+ A local stdio tool call without an explicit `workspace` resolves one in this order:
60
+
61
+ 1. The project in `anythingllm-projects.json` whose absolute `path` contains the current
62
+ working directory. The deepest match wins, so a checkout nested inside
63
+ another project resolves to its own workspace.
64
+ 2. Otherwise the first entry in `anythingllm_workspaces` that is not
65
+ `engineering-patterns`, which is reserved for cross-project patterns and is
66
+ never chosen as a project default.
67
+
68
+ Step 1 matters when one machine holds several checkouts that share a single
69
+ global memory config. Without it every repository resolves to the same
70
+ workspace, and a query made from a different repository reports "there is no
71
+ relevant information" instead of revealing that it searched somewhere
72
+ unrelated. Because the mapping is the same file the sync script uses, the
73
+ workspace a repository reads from is the one it embeds into.
74
+
75
+ If the mapping is absent, unreadable, or malformed, resolution falls back to
76
+ step 2, so existing single-project setups are unaffected.
77
+
78
+ Mapping discovery follows the bundled sync script exactly:
79
+ `CAIRN_ANYTHINGLLM_PROJECTS_FILE`, then a legacy file beside the selected sync
80
+ script, then `${XDG_CONFIG_HOME:-~/.config}/cairnkeep/anythingllm-projects.json`.
81
+ A missing explicit override does not select another mapping implicitly. Custom
82
+ sync implementations must honor the same mapping or configure an explicit
83
+ workspace; Cairnkeep cannot infer their private lookup rules.
84
+
85
+ Mapping reads are descriptor-bound UTF-8 JSON, limited to 1 MiB and 4,096
86
+ entries. Symlinked, special, changing and oversized mapping files are not read
87
+ as trusted content. Project paths must be absolute, without tilde expansion,
88
+ matching the sync script. They are canonicalized; relative declarations,
89
+ invalid workspace identifiers and the shared `engineering-patterns` slug are
90
+ skipped. Conflicting slugs for the same deepest canonical project raise a
91
+ controlled ambiguity error: specify `workspace` or correct the declarations.
92
+ Explicit tool arguments always win. With no resolved default, the tool errors.
93
+ This lookup selects a destination; it does not grant approval to synchronize.
60
94
 
61
95
  In local stdio mode, that config is read from the client project's working
62
96
  directory. In remote HTTP mode, the server cannot read files from the client
@@ -64,6 +98,10 @@ PC. Configure `ANYTHINGLLM_BASE_URL`, `ANYTHINGLLM_API_KEY`, and any sync script
64
98
  on the HTTP server host, then send the project workspaces in the
65
99
  `X-Cairn-AnythingLLM-Workspaces` session header. See
66
100
  [Memory storage and deployment](storage.md#per-project-remote-sessions).
101
+ HTTP calls never consult the server checkout's project mapping to choose a
102
+ client's default, including on the synchronization write path. The optional
103
+ OpenViking provider also retains its configured workspace selection rather
104
+ than consulting an AnythingLLM-specific mapping.
67
105
 
68
106
  The HTTP server performs queries and runs the sync script on its own host. A
69
107
  remote sync script must therefore obtain project documents from a server-side
@@ -1,7 +1,7 @@
1
1
  # Curriculum Coverage Map
2
2
 
3
- **Baseline:** Cairnkeep 2.23.0
4
- **Last reviewed:** 2026-10-07
3
+ **Baseline:** Cairnkeep 2.24.0
4
+ **Last reviewed:** 2026-10-22
5
5
 
6
6
  This map prevents the course from drifting behind the product. It assigns each
7
7
  user-facing surface to a lesson and records whether the lesson is complete or
@@ -20,7 +20,7 @@ been rehearsed.
20
20
  | Wiki, alignment, and repository quality workflows | L05-L06 | L05-L06 | Brief |
21
21
  | Optional local code graph, graph inspection, and harness delegates | L18 | L18 | Brief; default off |
22
22
  | Storage placement, export/import, and authenticated remote memory | L07 | L07-L08 | Brief |
23
- | Document RAG | L00 | L09 | Brief; optional |
23
+ | Document RAG | L00 | L09 | Brief; optional; [V09 routing supplement](video-scripts/V09-workspace-routing-supplement.md) covers local mappings and HTTP isolation |
24
24
  | token-miser exploration and routing | L00 | L10 | Brief; optional |
25
25
  | Rootless containers and persistent service operation | L01 | L11 | L01 Ready; L11 Brief |
26
26
  | Private managed distributions | L12 | L12 | Brief; optional |
@@ -1,6 +1,6 @@
1
1
  # Cairnkeep feature guide
2
2
 
3
- **Baseline:** Cairnkeep 2.23.0
3
+ **Baseline:** Cairnkeep 2.24.0
4
4
 
5
5
  This is the step-back map for choosing and operating Cairnkeep features. The
6
6
  [course labs](https://github.com/cairnkeep/cairnkeep-course-labs) provide one
@@ -42,6 +42,13 @@ match/source/conflict explanations without changing legacy defaults. See
42
42
  and its recording supplement. The synthetic Markdown comparison measures
43
43
  retrieval policy, not general task quality or a storage-format advantage.
44
44
 
45
+ Version 2.24 adds an optional, separately configured ranking
46
+ service, not a chat agent or memory authority. It remains default-off. See
47
+ [its bounded provider comparison and regression](../memory-retrieval.md#optional-learned-reranking)
48
+ before enabling it; L04 and its recording supplement cover the privacy,
49
+ doctor and fallback controls. The default-off team preview is separate and is
50
+ not included in this release.
51
+
45
52
  ## The three-layer model
46
53
 
47
54
  1. **Memory server:** an MCP process that owns scoped durable storage. The
@@ -3,7 +3,7 @@
3
3
  **Status:** Ready
4
4
  **Track:** Quickstart
5
5
  **Time:** 10 minutes
6
- **Tested with:** Cairnkeep 2.23.0
6
+ **Tested with:** Cairnkeep 2.24.0
7
7
 
8
8
  ## Outcome
9
9
 
@@ -3,7 +3,7 @@
3
3
  **Status:** Ready
4
4
  **Track:** Quickstart
5
5
  **Time:** 15 minutes
6
- **Tested with:** Cairnkeep 2.23.0
6
+ **Tested with:** Cairnkeep 2.24.0
7
7
 
8
8
  ## Outcome
9
9
 
@@ -26,7 +26,7 @@ ephemeral; the named course volume is persistent.
26
26
  1. Pull the public image:
27
27
 
28
28
  ```bash
29
- podman pull ghcr.io/cairnkeep/cairnkeep:2.23.0
29
+ podman pull ghcr.io/cairnkeep/cairnkeep:2.24.0
30
30
  ```
31
31
 
32
32
  2. Start the stdio server with a read-only root filesystem and no Linux
@@ -38,7 +38,7 @@ ephemeral; the named course volume is persistent.
38
38
  --read-only --cap-drop=all --security-opt=no-new-privileges \
39
39
  --tmpfs=/tmp:rw,noexec,nosuid,size=64m,mode=1777 \
40
40
  --volume cairnkeep-course-data:/data:Z,U \
41
- ghcr.io/cairnkeep/cairnkeep:2.23.0 stdio
41
+ ghcr.io/cairnkeep/cairnkeep:2.24.0 stdio
42
42
  ```
43
43
 
44
44
  3. The process waits quietly for MCP input. Press `Ctrl-C` after confirming it
@@ -3,7 +3,7 @@
3
3
  **Status:** Ready
4
4
  **Track:** Quickstart
5
5
  **Time:** 25 minutes
6
- **Tested with:** Cairnkeep 2.23.0 and Node.js 22 or newer
6
+ **Tested with:** Cairnkeep 2.24.0 and Node.js 22 or newer
7
7
 
8
8
  ## Outcome
9
9
 
@@ -47,7 +47,7 @@ sync. Codex project trust remains an explicit operator decision.
47
47
  2. Install Cairnkeep:
48
48
 
49
49
  ```bash
50
- npm install --global @cairnkeep/cli@2.23.0
50
+ npm install --global @cairnkeep/cli@2.24.0
51
51
  cairn --version
52
52
  ```
53
53
 
@@ -3,7 +3,7 @@
3
3
  **Status:** Ready
4
4
  **Track:** Quickstart
5
5
  **Time:** 25 minutes
6
- **Tested with:** Cairnkeep 2.23.0
6
+ **Tested with:** Cairnkeep 2.24.0
7
7
 
8
8
  ## Outcome
9
9
 
@@ -53,6 +53,18 @@ fact and starting a fresh client, verifying the new value and explicit history.
53
53
  - No extracted candidate becomes durable without an explicit review action.
54
54
  - Enabling typed nodes changes discovery structure, not the authority of a fact.
55
55
 
56
+ ## Optional reranker extension
57
+
58
+ From a source checkout of 2.24.0 or newer, run `npm --prefix mcp-memory-server run check:memory-reranker`
59
+ without a provider. Then, only with explicit synthetic-data/provider consent,
60
+ compare the frozen live study in the retrieval guide. Explain both the 6/16
61
+ to 15/16 first-result improvement and the single regression; this is not a
62
+ general accuracy or agent-productivity claim. Show that reranking is off by
63
+ default, has a separate credential, preserves admission and exact-key priority,
64
+ and falls back without mutating stored facts. A failed provider must make
65
+ `cairn doctor` fail even though search still works. Read-only is not a promise
66
+ of zero network traffic: configured search advertises open-world behavior.
67
+
56
68
  ## Source material
57
69
 
58
70
  - [Operating guide](../../operating.md#the-workflow)
@@ -13,6 +13,8 @@ without affecting core Cairnkeep memory.
13
13
 
14
14
  - Memory versus document RAG responsibilities.
15
15
  - Workspace selection and project configuration.
16
+ - Explicit directory-to-workspace mapping for local stdio; HTTP session headers
17
+ remain authoritative for remote routing, not the server's current directory.
16
18
  - Query-only setup before document synchronization.
17
19
  - Incremental, full, and replacement synchronization boundaries.
18
20
  - Endpoint, credential, and document-selection safety.
@@ -24,12 +26,21 @@ Index a synthetic three-document corpus into a disposable workspace, ask a
24
26
  question requiring citations to two documents, update one document, resync, and
25
27
  verify the new answer. Then unset the integration and prove memory still works.
26
28
 
29
+ Use two disposable checkouts sharing a global memory configuration. Declare
30
+ their absolute paths and distinct slugs in `anythingllm-projects.json`; verify
31
+ root and subdirectory queries report the expected workspace. Check an explicit
32
+ workspace override and an undeclared-directory fallback. Keep synchronization
33
+ synthetic and separately approved; a destination mapping is not write consent.
34
+ See the [recording supplement](../video-scripts/V09-workspace-routing-supplement.md).
35
+
27
36
  ## Acceptance criteria
28
37
 
29
38
  - Only allowlisted synthetic documents are uploaded.
30
39
  - The learner can identify the selected workspace and endpoint without printing
31
40
  its credential.
32
41
  - Updated content replaces stale retrieval results.
42
+ - The learner verifies the returned workspace before trusting an empty answer;
43
+ conflicting deepest declarations require an explicit destination or repair.
33
44
  - Disabling RAG leaves `/remember` and `/recall` healthy.
34
45
 
35
46
  ## Source material
@@ -3,7 +3,7 @@
3
3
  **Status:** Ready
4
4
  **Track:** Evidence and Evaluation
5
5
  **Time:** 50 minutes
6
- **Tested with:** Cairnkeep 2.23.0
6
+ **Tested with:** Cairnkeep 2.24.0
7
7
 
8
8
  ## Outcome
9
9
 
@@ -1,7 +1,7 @@
1
1
  # L20 - Least-authority MCP tool profiles
2
2
 
3
3
  **Status:** Ready
4
- **Tested with:** Cairnkeep 2.23.0 and Node.js 22 or newer
4
+ **Tested with:** Cairnkeep 2.24.0 and Node.js 22 or newer
5
5
  **Time:** 25 minutes
6
6
 
7
7
  ## Outcome
@@ -1,7 +1,7 @@
1
1
  # L21 - Immutable context packs and skill approval
2
2
 
3
3
  **Status:** Ready
4
- **Tested with:** Cairnkeep 2.23.0 and Node.js 22 or newer
4
+ **Tested with:** Cairnkeep 2.24.0 and Node.js 22 or newer
5
5
  **Time:** 40 minutes
6
6
 
7
7
  ## Outcome
@@ -3,7 +3,7 @@
3
3
  **Status:** Ready
4
4
  **Track:** Operator
5
5
  **Time:** 30 minutes
6
- **Tested with:** Cairnkeep 2.23.0, Windows x64, and Node.js 22 or newer
6
+ **Tested with:** Cairnkeep 2.24.0, Windows x64, and Node.js 22 or newer
7
7
 
8
8
  ## Outcome
9
9
 
@@ -16,7 +16,7 @@ perform a reversible uninstall without relying on WSL or Git Bash.
16
16
  From PowerShell:
17
17
 
18
18
  ```powershell
19
- npm install --global @cairnkeep/cli@2.23.0
19
+ npm install --global @cairnkeep/cli@2.24.0
20
20
  cairn version
21
21
 
22
22
  $Lab = Join-Path $env:TEMP 'Cairnkeep Windows Lab'
@@ -3,7 +3,7 @@
3
3
  **Status:** Ready
4
4
  **Track:** Quickstart and Operator
5
5
  **Time:** 35 minutes
6
- **Tested with:** Cairnkeep 2.23.0; Pi 0.84.1; Node.js 22, 24, and 26
6
+ **Tested with:** Cairnkeep 2.24.0; Pi 0.84.1; Node.js 22, 24, and 26
7
7
 
8
8
  ## Outcome
9
9
 
@@ -1,7 +1,7 @@
1
1
  # L24 - Reviewed knowledge exchange
2
2
 
3
3
  **Status:** Ready
4
- **Tested with:** Cairnkeep 2.23.0 and Node.js 22 or newer
4
+ **Tested with:** Cairnkeep 2.24.0 and Node.js 22 or newer
5
5
  **Time:** 35 minutes
6
6
 
7
7
  ## Outcome
@@ -1,7 +1,7 @@
1
1
  # L25 - Bounded workflow playbooks
2
2
 
3
3
  **Status:** Ready
4
- **Tested with:** Cairnkeep 2.23.0 and Node.js 22 or newer
4
+ **Tested with:** Cairnkeep 2.24.0 and Node.js 22 or newer
5
5
  **Time:** 35 minutes
6
6
 
7
7
  ## Outcome
@@ -1,7 +1,7 @@
1
1
  # L26 - Context intelligence with explicit authority
2
2
 
3
3
  **Status:** Ready
4
- **Tested with:** Cairnkeep 2.23.0 and Node.js 22 or newer
4
+ **Tested with:** Cairnkeep 2.24.0 and Node.js 22 or newer
5
5
 
6
6
  ## Outcome
7
7
 
@@ -1,6 +1,6 @@
1
1
  # L27 - Evaluate context companions without moving authority
2
2
 
3
- **Tested with:** Cairnkeep 2.23.0 and Node.js 22 or newer
3
+ **Tested with:** Cairnkeep 2.24.0 and Node.js 22 or newer
4
4
 
5
5
  ## Outcome
6
6
 
@@ -1,7 +1,7 @@
1
1
  # L28 - Verify Cairnkeep's security posture
2
2
 
3
3
  **Status:** Ready
4
- **Tested with:** Cairnkeep 2.23.0 and Node.js 22 or newer
4
+ **Tested with:** Cairnkeep 2.24.0 and Node.js 22 or newer
5
5
 
6
6
  ## Outcome
7
7
 
@@ -11,7 +11,7 @@ and choose least authority for a network-facing MCP client.
11
11
 
12
12
  ## Prerequisites
13
13
 
14
- - Install Cairnkeep 2.23.0 or use a disposable source checkout.
14
+ - Install Cairnkeep 2.24.0 or use a disposable source checkout.
15
15
  - Complete [L20 - Least-authority MCP tool profiles](L20-mcp-tool-profiles.md).
16
16
  - Use synthetic values only; never paste a real bearer token into a recording,
17
17
  terminal transcript, or issue.
@@ -32,3 +32,16 @@ not confidence, and that the attempted live study failed with HTTP 502. Do not
32
32
  invent a successful live result or claim general task-quality improvements.
33
33
  Use short terminal zooms and side-by-side result comparisons instead of static
34
34
  slides; keep credentials and real memory off screen.
35
+
36
+ ## Optional reranker segment (2.24.0 or newer source checkout)
37
+
38
+ Record real `check:memory-reranker` output, pausing on the no-network default,
39
+ scope/history canaries and provider-failure fallback. With explicit synthetic
40
+ provider configuration, run `run-memory-reranker-study.mjs --live --summary`.
41
+ Do not show credentials or endpoint addresses. Compare the two arms using the
42
+ same embeddings: the first frozen run improved first hits from 6/16 to 15/16,
43
+ but `current-not-history` regressed. Zoom into that failure rather than hiding
44
+ it. Explain that the earlier HTTP-502 study is separate and remains a failed
45
+ run. Demonstrate the doctor's synthetic request, then disable reranking and
46
+ show unchanged memory text and provenance. This is a recording brief, not a
47
+ claim that this segment has been filmed or published.
@@ -0,0 +1,36 @@
1
+ # V09 supplement — The right documents for the right repository
2
+
3
+ Technical recording brief; no completed video or full Ready L09 lab is claimed.
4
+ Use invented repository names and a loopback mock, never private mappings,
5
+ provider keys, real documents or production synchronization.
6
+
7
+ ## Real terminal demonstration
8
+
9
+ Leave a readable pause after each command:
10
+
11
+ ```sh
12
+ npm --prefix mcp-memory-server run build
13
+ npm --prefix mcp-memory-server run check:anythingllm-workspace
14
+ npm --prefix mcp-memory-server run check:remote-context
15
+ ```
16
+
17
+ Show two synthetic checkouts sharing one global configuration, then highlight
18
+ the distinct workspace reported by each real stdio MCP query. Zoom into a
19
+ subdirectory call and an explicit override. Explain that an empty answer from
20
+ the wrong corpus is not evidence that the requested information does not exist.
21
+
22
+ The mock synchronization prints arguments only; it uploads nothing. Contrast
23
+ destination selection with write consent and show replacement being refused
24
+ without confirmation. For HTTP, show the conflicting server-directory mapping
25
+ fixture and the client header still winning. Do not run a live provider or
26
+ silently activate RAG merely to record this supplement.
27
+
28
+ ## Boundaries and recovery
29
+
30
+ Explain the lookup order: explicit mapping file, legacy mapping beside the
31
+ selected sync script, then XDG. Missing or invalid mappings keep the legacy
32
+ configured default; inspect the returned workspace. Ambiguous deepest entries
33
+ require an explicit workspace or a corrected mapping. Mapping files are bounded
34
+ and unsafe file types are rejected. Directory matching is routing, not an ACL
35
+ or publisher-authenticity guarantee. Keep the full L09 lesson marked Brief until
36
+ its separate live synthetic-corpus lab has been verified.
@@ -41,3 +41,10 @@ The default `full` profile preserves the same tool names, schemas, responses,
41
41
  feature gates, and ordering as an unconfigured server. The annotations help
42
42
  conservative clients distinguish observations from actions; they do not grant
43
43
  permission or replace the client's approval policy.
44
+
45
+ With a valid, explicitly enabled memory reranker, `memory_search` advertises
46
+ `openWorldHint: true` because query and bounded candidate excerpts can reach
47
+ the configured service. It remains `readOnlyHint: true`, so a read-only
48
+ profile still exposes it. Tool restriction is not network consent: omit it
49
+ from a custom profile or disable reranking if that data flow is unwanted.
50
+ With reranking off, existing discovery metadata is unchanged.
@@ -161,8 +161,90 @@ Its sample threshold of 0.75 is an evaluation arm, not a recommended default.
161
161
  An attempted live run during development accepted short probes but failed with
162
162
  HTTP 502 on some fixture inputs. No successful live-model result is claimed.
163
163
  The offline suite never requires a model, API key, paid service or network beyond
164
- its own loopback fixture. An optional learned reranker remains deferred until
165
- provider-specific evidence demonstrates a benefit beyond this simpler policy.
164
+ its own loopback fixture. A separate optional reranker comparison below does
165
+ not replace this older study or retroactively make that failed run successful.
166
+
167
+ ## Optional learned reranking
168
+
169
+ Available since Cairnkeep 2.24.0. This feature improves ordering, not guaranteed
170
+ recall: the small live comparison below retained the same Recall@3 in both arms.
171
+
172
+ The optional reranker is default-off and independent of extraction and
173
+ embeddings. Set `CAIRN_MEMORY_RERANK=1`, `CAIRN_MEMORY_RERANK_URL` (a base URL,
174
+ normally ending in `/v1`) and `CAIRN_MEMORY_RERANK_MODEL` on the memory-server
175
+ process. The client appends `/rerank`. If authentication is needed, set the
176
+ separate `CAIRN_MEMORY_RERANK_API_KEY`; extraction/embedding credentials are
177
+ never reused. `CAIRN_MEMORY_RERANK_TIMEOUT_MS` defaults to 2,000 ms and accepts
178
+ 100–10,000 ms. Restart the server after configuration changes.
179
+
180
+ Only `memory_search` in the `memory` address space is reranked, for either
181
+ legacy or hybrid strategy. Notes, context packs, documents and the separate
182
+ team store are unchanged. Existing scope, current-value, typed and relevance
183
+ filters admit candidates first. The provider receives the original query
184
+ (at most 4 KiB) and at most 50 admitted key/value excerpts (at most 4 KiB each),
185
+ not the whole store. Candidate depth is `min(50, max(32, 4 * top_k))`.
186
+ Exact-key priority in hybrid search (including explicit aliases) and typed legacy exact-hit priority remain
187
+ ahead of model scores. The provider only reorders admitted records; it cannot
188
+ add facts, change authority, write memory or recover a candidate already
189
+ rejected by admission. A small final `top_k` can include a candidate outside
190
+ the original top-k prefix after reranking.
191
+
192
+ The endpoint must return every candidate index exactly once in a `results`
193
+ array with finite `relevance_score` numbers. Negative raw scores are accepted;
194
+ these are ranking values, not calibrated confidence. Returned memory text,
195
+ provenance and existing retrieval scores are unchanged; those scores need not
196
+ be descending after learned reranking. Ties preserve the
197
+ existing order. Timeout, HTTP errors, redirects, oversized responses, invalid
198
+ configuration or malformed/partial indices preserve the original order.
199
+ No retries or alternate provider fallback are made. Unset/disabled configuration
200
+ performs no reranker request and preserves existing discovery and responses.
201
+ When configured, `memory_search` remains read-only but advertises
202
+ `openWorldHint: true` because it can export candidate text.
203
+
204
+ `cairn doctor` uses synthetic text to check configured reranking and fails if
205
+ the configuration or response is invalid. Ordinary searches still fall back
206
+ safely; doctor does not mistake fallback for a healthy provider.
207
+
208
+ ### Bounded provider comparison
209
+
210
+ The frozen `memory-reranker-study-v1.json` fixture contains 20 synthetic
211
+ continuity cases, including 16 positive and four negative cases. Both arms use
212
+ the same embedding vectors, admission rules, aliases and final top three.
213
+ Labels and the fixture digest were frozen before the first live ranking run.
214
+
215
+ | Metric | Hybrid + embeddings | Same candidates + reranker |
216
+ |---|---:|---:|
217
+ | Relevant first result, 16 positive cases | 6/16 | 15/16 |
218
+ | MRR@3 | 0.635417 | 0.968750 |
219
+ | NDCG@3 | 0.728416 | 0.976933 |
220
+ | Recall@3 | 1.0 | 1.0 |
221
+ | Negative queries returning results, 4 cases | 0/4 | 0/4 |
222
+
223
+ Ten cases improved and **one regressed**: `current-not-history` ranked a
224
+ topical interface distractor above the correct current fact. Hidden history
225
+ was still excluded. No scope/history violation or provider failure occurred.
226
+ Observed additional ranking latency was 9 ms median and 46 ms p95 in this
227
+ small local run; this is not a deployment latency guarantee.
228
+
229
+ This is an engineering-labeled synthetic comparison with deliberately
230
+ ambiguous topical distractors, not an independent holdout, a human study or
231
+ proof of general agent-quality gains. It justifies an optional experiment,
232
+ not default enablement. Evaluate your provider and corpus before opting in;
233
+ disable `CAIRN_MEMORY_RERANK` to reverse the change.
234
+
235
+ From a source checkout (these developer study scripts are not installed CLI
236
+ commands):
237
+
238
+ ```sh
239
+ npm --prefix mcp-memory-server run check:memory-reranker
240
+ # Requires explicit existing embedding and reranker configuration:
241
+ node mcp-memory-server/scripts/run-memory-reranker-study.mjs --live --summary
242
+ ```
243
+
244
+ Without `--live`, the study makes no provider calls even when configured.
245
+ Reports include fixture, implementation and model-ID digests, no URLs or keys.
246
+ The frozen fixture SHA-256 is
247
+ `f6a5d2de8d2846dd5a8de764096d92255716592805e8abfdc771181eac23beb1`.
166
248
 
167
249
  ## Research connection
168
250
 
package/docs/operating.md CHANGED
@@ -495,6 +495,11 @@ vendor or host.
495
495
  | `CAIRN_MEMORY_EMBEDDING_URL` | Embeddings endpoint (falls back to `CAIRN_LLM_API_URL`) |
496
496
  | `CAIRN_MEMORY_EMBEDDING_MODEL` | Embedding model name (required for semantic search) |
497
497
  | `CAIRN_MEMORY_EMBEDDING_TIMEOUT_MS` | Embedding request timeout before substring fallback (default `15000`) |
498
+ | `CAIRN_MEMORY_RERANK` | Explicit `1` enables optional learned memory-search reranking; off by default |
499
+ | `CAIRN_MEMORY_RERANK_URL` | Explicit HTTP(S) base URL; client appends `/rerank`; no embedded credentials, query or fragment |
500
+ | `CAIRN_MEMORY_RERANK_MODEL` | Required model identifier when reranking is enabled |
501
+ | `CAIRN_MEMORY_RERANK_API_KEY` | Optional separate reranker credential; no extraction/embedding key fallback |
502
+ | `CAIRN_MEMORY_RERANK_TIMEOUT_MS` | Reranking deadline, default `2000`, allowed `100`–`10000`; failure preserves existing search order |
498
503
  | `CAIRN_AGENTFS_BASE_DIR` | Server-side base dir for named/global memory scopes (default `~/.cairnkeep`); it does not affect `project` scope |
499
504
  | `CAIRN_TRAJECTORY_CAPTURE` | Opt in to local structured session capture (`1`, `true`, `yes`, or `on`; unset/default → no capture work) |
500
505
  | `CAIRN_TRAJECTORY_SESSION_MAX_BYTES` | Maximum serialized bytes per session (default `5242880`, 5 MiB; minimum 1024) |
@@ -3,7 +3,8 @@
3
3
  Cairnkeep has no telemetry, analytics collector, hosted control plane, or
4
4
  automatic remote-service discovery. A default local stdio installation stores
5
5
  memory in SQLite on the machine running the MCP server and makes no model or RAG
6
- request unless the corresponding endpoint and credential are configured.
6
+ request unless the corresponding feature is explicitly configured as described
7
+ below (some operator-owned endpoints do not require a credential).
7
8
 
8
9
  ## Data flows
9
10
 
@@ -11,6 +12,7 @@ request unless the corresponding endpoint and credential are configured.
11
12
  |---|---|---|
12
13
  | Local substring search | None | Local SQLite only |
13
14
  | Embedding-ranked search | Stored memory values that are not cached, plus the search query | `CAIRN_MEMORY_EMBEDDING_URL` or `CAIRN_LLM_API_URL` |
15
+ | Separately opted-in memory reranking | Query and at most 50 scope/current/filter-admitted key/value excerpts, each at most 4 KiB | Only the explicit `CAIRN_MEMORY_RERANK_URL`; separate optional credential, no retries/redirects/alternate hosts |
14
16
  | `memory_extract` | The transcript supplied to the tool | `CAIRN_LLM_API_URL` |
15
17
  | `domain_knowledge_query` | Workspace slug and query | `ANYTHINGLLM_BASE_URL` |
16
18
  | `domain_knowledge_sync` | Files selected by the sync configuration | `ANYTHINGLLM_BASE_URL` |
@@ -50,6 +52,10 @@ Model endpoints may be local or remote. Cairnkeep cannot determine a provider's
50
52
  retention, training, or logging policy; verify it before sending confidential
51
53
  material. Disabling `CAIRN_LLM_API_KEY`, `ANYTHINGLLM_API_KEY`, remote HTTP
52
54
  registration, and delegated tools keeps the core memory workflow local.
55
+ Also leave `CAIRN_MEMORY_RERANK` unset to disable its independent data flow;
56
+ it does not require or inherit `CAIRN_LLM_API_KEY`. A local provider is still a
57
+ separate process that receives these excerpts. Neither a chat agent nor a
58
+ research assistant is part of reranking or authorization.
53
59
 
54
60
  ## Playbook decision and receipt flow
55
61
 
@@ -91,11 +97,14 @@ backups/journals are sensitive local data at rest. Dry-run and commit responses
91
97
  contain scopes/address spaces, digests, keys, actions, and counts—not supplied
92
98
  or displaced values. No model, API key, endpoint, or network is required.
93
99
 
94
- The existing explicitly configured embedding search is the sole related egress:
95
- eligible key/value/type/tag text and the query may be sent to
100
+ Explicitly configured embedding search can send eligible key/value/type/tag
101
+ text and the query to
96
102
  `CAIRN_MEMORY_EMBEDDING_URL` (or its documented fallback). Hard filters run
97
103
  before that request; endpoint failure returns to local substring search. Note
98
- search is always local and never uses embeddings.
104
+ search is always local and never uses embeddings or reranking. Separately
105
+ opting into the [bounded reranker](memory-retrieval.md#optional-learned-reranking)
106
+ can also send admitted memory excerpts and the query. It only changes ordering,
107
+ never scope, history admission, approval, factual authority or durable memory.
99
108
 
100
109
  Canonical note storage includes manifest records, Markdown, indexes, history,
101
110
  import replay state, staged bytes, backups, and journals. Prepared/committing
@@ -1 +1 @@
1
- {"identifier":"cairn-offline-fake-v1","package_version":"2.23.0","schema_version":1,"task_set_digest":"13e95379f35839593047ec34521a65a8fa3ab92bd3993bca9dc8bf5c4a9ff36f"}
1
+ {"identifier":"cairn-offline-fake-v1","package_version":"2.24.0","schema_version":1,"task_set_digest":"13e95379f35839593047ec34521a65a8fa3ab92bd3993bca9dc8bf5c4a9ff36f"}