@cairnkeep/cli 2.4.0 → 2.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (42) hide show
  1. package/CHANGELOG.md +27 -1
  2. package/README.md +21 -3
  3. package/docs/building-an-overlay.md +4 -3
  4. package/docs/harness-compatibility.md +195 -0
  5. package/docs/learning/CURRICULUM-MAP.md +91 -0
  6. package/docs/learning/FEATURE-GUIDE.md +86 -0
  7. package/docs/learning/PRODUCTION-PLAN.md +45 -14
  8. package/docs/learning/README.md +20 -1
  9. package/docs/learning/instructor-guide.md +6 -0
  10. package/docs/learning/lessons/L00-why-cairnkeep.md +20 -8
  11. package/docs/learning/lessons/L01-safe-trial.md +3 -3
  12. package/docs/learning/lessons/L02-installation.md +31 -5
  13. package/docs/learning/lessons/L03-first-project.md +7 -2
  14. package/docs/learning/lessons/L04-memory-fundamentals.md +4 -0
  15. package/docs/learning/lessons/L05-daily-workflow.md +4 -0
  16. package/docs/learning/lessons/L07-privacy-and-storage.md +3 -0
  17. package/docs/learning/lessons/L10-context-exploration.md +4 -2
  18. package/docs/learning/lessons/L12-managed-overlays.md +2 -0
  19. package/docs/learning/lessons/L13-session-evidence.md +71 -0
  20. package/docs/learning/lessons/L14-typed-memory.md +51 -0
  21. package/docs/learning/lessons/L15-capability-governance.md +56 -0
  22. package/docs/learning/lessons/L16-evaluation.md +67 -0
  23. package/docs/learning/lessons/L17-agent-boundary.md +55 -0
  24. package/docs/learning/templates/lesson-template.md +4 -1
  25. package/docs/learning/tracks/evidence-and-evaluation.md +20 -0
  26. package/docs/learning/tracks/operator.md +8 -2
  27. package/docs/learning/tracks/practitioner.md +6 -1
  28. package/docs/learning/tracks/quickstart.md +3 -0
  29. package/docs/learning/video-scripts/L00-why-cairnkeep.md +21 -17
  30. package/docs/learning/video-scripts/L02-installation.md +9 -2
  31. package/docs/learning/video-scripts/L03-first-project.md +3 -0
  32. package/docs/operating.md +70 -5
  33. package/examples/eval/bundled-fake.json +1 -1
  34. package/package.json +3 -2
  35. package/scripts/bootstrap.sh +4 -0
  36. package/scripts/test-bootstrap-untracked.sh +3 -1
  37. package/scripts/test-launcher-hooks.sh +57 -1
  38. package/scripts/test-learning-docs.sh +20 -4
  39. package/scripts/test-package-install.sh +6 -0
  40. package/scripts/uninstall.sh +2 -0
  41. package/templates/start-kimi.sh.template +39 -0
  42. package/templates/start-qwen.sh.template +32 -0
@@ -3,7 +3,7 @@
3
3
  **Status:** Ready
4
4
  **Track:** Quickstart
5
5
  **Time:** 10 minutes
6
- **Tested with:** Cairnkeep 2.4.0
6
+ **Tested with:** Cairnkeep 2.6.0
7
7
 
8
8
  ## Outcome
9
9
 
@@ -18,22 +18,27 @@ entire transcript into the next prompt is expensive and preserves noise along
18
18
  with useful facts.
19
19
 
20
20
  Cairnkeep stores small, reviewed operational facts and retrieves them when they
21
- are relevant. It also provides a derived wiki and governed review workflows,
22
- without replacing source code, tests, issue trackers, or canonical documents.
21
+ are relevant. It also provides a derived wiki and governed review workflows.
22
+ Optional local evidence features can retain redacted session structure,
23
+ artifacts, and deterministic hindsight notes, but they remain distinct from
24
+ reviewed memory and are disabled by default. None of these layers replaces
25
+ source code, tests, issue trackers, or canonical documents.
23
26
 
24
27
  ## Mental model
25
28
 
26
- Keep four layers distinct:
29
+ Follow information through a trust ladder:
27
30
 
28
31
  | Layer | Example | Role |
29
32
  |---|---|---|
30
33
  | Harness history | The current conversation | Temporary working context |
34
+ | Captured evidence (optional) | A redacted tool result or compaction summary | Bounded local evidence, not trusted instruction |
35
+ | Hindsight note (optional) | A repeated failure signature and resolution | Deterministic derived knowledge, not automatically shared |
31
36
  | Cairnkeep memory | “Use transactional migrations” | Durable operational fact |
32
37
  | Project wiki | A cited architecture summary | Derived, reviewable knowledge |
33
38
  | Canonical source | Code, tests, ADR, issue | Authority when layers disagree |
34
39
 
35
- Document RAG is an optional fifth layer for retrieving larger documents. It is
36
- not required for Cairnkeep memory.
40
+ Document RAG and token-miser context exploration are optional retrieval paths,
41
+ not additional authorities. Neither is required for Cairnkeep memory.
37
42
 
38
43
  ## Exercise
39
44
 
@@ -43,6 +48,7 @@ For each statement, choose the appropriate destination:
43
48
  2. “Here is the complete API specification.”
44
49
  3. “The current debugging hypothesis is a race in the cache.”
45
50
  4. “Never run the production migration from a developer laptop.”
51
+ 5. “This synthetic stack trace appeared and was resolved twice.”
46
52
 
47
53
  Suggested answer:
48
54
 
@@ -51,6 +57,8 @@ Suggested answer:
51
57
  - 2 stays a canonical document and may optionally be indexed for RAG.
52
58
  - 3 remains session context until confirmed.
53
59
  - 4 is a durable constraint after it is verified against policy.
60
+ - 5 may become local session evidence and a hindsight note. It becomes reviewed
61
+ memory only through a separate explicit workflow.
54
62
 
55
63
  ## Verify
56
64
 
@@ -59,6 +67,7 @@ You are ready for L01 if you can answer these questions:
59
67
  - Does Cairnkeep replace the repository or its documentation? **No.**
60
68
  - Does the default installation discover a remote memory server? **No.**
61
69
  - Should every sentence in a session become durable memory? **No.**
70
+ - Does enabling one optional evidence feature enable the others? **No.**
62
71
 
63
72
  ## Common failures
64
73
 
@@ -66,17 +75,20 @@ You are ready for L01 if you can answer these questions:
66
75
  |---|---|
67
76
  | Memory is another transcript store | Store concise, durable facts rather than complete prompts |
68
77
  | RAG and memory are the same | RAG retrieves documents; memory preserves reviewed operational facts |
78
+ | Captured evidence is trusted memory | Evidence and notes remain derived until explicitly reviewed or promoted |
69
79
  | Derived knowledge is authoritative | Code, tests, policies, and canonical documents win conflicts |
70
80
 
71
81
  ## Privacy and trust boundary
72
82
 
73
83
  The default stdio topology stores memory on the local machine. Remote storage,
74
- model-backed extraction, embeddings, and document RAG require explicit
75
- configuration and are taught separately.
84
+ structured capture, note distillation/enrichment, artifacts, evaluation,
85
+ model-backed extraction, embeddings, and document RAG require separate explicit
86
+ configuration and are taught later.
76
87
 
77
88
  ## Recap
78
89
 
79
90
  - Memory is selective and durable.
91
+ - Optional evidence and notes are distinct from reviewed memory.
80
92
  - Derived knowledge remains subordinate to canonical sources.
81
93
  - Optional services are not prerequisites.
82
94
 
@@ -3,7 +3,7 @@
3
3
  **Status:** Ready
4
4
  **Track:** Quickstart
5
5
  **Time:** 15 minutes
6
- **Tested with:** Cairnkeep 2.4.0
6
+ **Tested with:** Cairnkeep 2.6.0
7
7
 
8
8
  ## Outcome
9
9
 
@@ -26,7 +26,7 @@ ephemeral; the named course volume is persistent.
26
26
  1. Pull the public image:
27
27
 
28
28
  ```bash
29
- podman pull ghcr.io/cairnkeep/cairnkeep:2.4.0
29
+ podman pull ghcr.io/cairnkeep/cairnkeep:2.6.0
30
30
  ```
31
31
 
32
32
  2. Start the stdio server with a read-only root filesystem and no Linux
@@ -38,7 +38,7 @@ ephemeral; the named course volume is persistent.
38
38
  --read-only --cap-drop=all --security-opt=no-new-privileges \
39
39
  --tmpfs=/tmp:rw,noexec,nosuid,size=64m,mode=1777 \
40
40
  --volume cairnkeep-course-data:/data:Z,U \
41
- ghcr.io/cairnkeep/cairnkeep:2.4.0 stdio
41
+ ghcr.io/cairnkeep/cairnkeep:2.6.0 stdio
42
42
  ```
43
43
 
44
44
  3. The process waits quietly for MCP input. Press `Ctrl-C` after confirming it
@@ -3,7 +3,7 @@
3
3
  **Status:** Ready
4
4
  **Track:** Quickstart
5
5
  **Time:** 25 minutes
6
- **Tested with:** Cairnkeep 2.4.0 and Node.js 22 or newer
6
+ **Tested with:** Cairnkeep 2.6.0 and Node.js 22 or newer
7
7
 
8
8
  ## Outcome
9
9
 
@@ -13,8 +13,9 @@ layer, and distinguish a complete installation from a partial one.
13
13
  ## Prerequisites
14
14
 
15
15
  - Node.js 22 or newer and npm.
16
- - Claude Code installed for the commands below. OpenCode users should follow
17
- the equivalent setup in [the operating guide](../../operating.md).
16
+ - Claude Code installed for the commands below. OpenCode, Kimi Code, and Qwen Code users
17
+ should follow the equivalent setup in
18
+ [the operating guide](../../operating.md).
18
19
  - Permission to install an npm package globally for your user.
19
20
 
20
21
  ## Mental model
@@ -40,7 +41,7 @@ machine-level steps.
40
41
  2. Install Cairnkeep:
41
42
 
42
43
  ```bash
43
- npm install --global @cairnkeep/cli@2.4.0
44
+ npm install --global @cairnkeep/cli@2.6.0
44
45
  cairn --version
45
46
  ```
46
47
 
@@ -62,6 +63,15 @@ machine-level steps.
62
63
  cairn sync --check
63
64
  ```
64
65
 
66
+ 6. Optionally enable completion in the current Bash shell:
67
+
68
+ ```bash
69
+ source <(cairn completion bash)
70
+ ```
71
+
72
+ Use `cairn completion zsh` or `cairn completion fish` for those shells.
73
+ Completion is a convenience and is not another runtime component.
74
+
65
75
  ## Verify
66
76
 
67
77
  Run both checks:
@@ -88,7 +98,23 @@ installation.
88
98
 
89
99
  This registration starts `cairn memory-server` locally as a stdio child. It
90
100
  does not discover a server, send memory to a remote host, configure embeddings,
91
- or enable an optional document-RAG integration.
101
+ enable session capture, or enable an optional document-RAG integration. Pi uses
102
+ the separate `cairn sync-pi` trajectory adapter and still needs a user-selected
103
+ bridge if Pi should expose MCP memory tools.
104
+
105
+ Kimi Code can use the same memory server through its generated
106
+ `.ai/start-kimi.sh` launcher, but its HTTP MCP configuration requires a literal
107
+ URL rather than `${CAIRN_MEMORY_REMOTE_URL}`. Keep the resolved URL in a private
108
+ `.kimi-code/mcp.json` and keep the bearer token in the environment. See
109
+ [Harness compatibility](../../harness-compatibility.md#kimi-code).
110
+
111
+ Qwen Code can use the generated `.ai/start-qwen.sh` launcher. Its project
112
+ `.qwen/settings.json` supports environment references for both the remote URL
113
+ and authorization header, so the file can contain names such as
114
+ `${CAIRN_MEMORY_REMOTE_URL}` and `${CAIRN_MEMORY_HTTP_TOKEN}` instead of secret
115
+ values. Keep this project settings file untracked and approve the generated MCP
116
+ entry with `qwen mcp approve cairn-memory`. See
117
+ [Harness compatibility](../../harness-compatibility.md#qwen-code).
92
118
 
93
119
  ## Clean up if you stop the evaluation
94
120
 
@@ -3,7 +3,7 @@
3
3
  **Status:** Ready
4
4
  **Track:** Quickstart
5
5
  **Time:** 25 minutes
6
- **Tested with:** Cairnkeep 2.4.0
6
+ **Tested with:** Cairnkeep 2.6.0
7
7
 
8
8
  ## Outcome
9
9
 
@@ -41,6 +41,10 @@ fact, and recall it from a new harness session.
41
41
  cairn doctor
42
42
  ```
43
43
 
44
+ Also leave trajectory, note, compaction, artifact, typed-memory, capability,
45
+ and evaluation flags unset. The quickstart proves core memory without
46
+ silently changing the amount of retained data.
47
+
44
48
  4. Launch Claude Code through the generated project launcher:
45
49
 
46
50
  ```bash
@@ -85,7 +89,8 @@ The lesson is complete only if:
85
89
 
86
90
  Use synthetic memory in the course. The local stdio server writes to local
87
91
  AgentFS/SQLite storage. Project wiki and planning files are derived artifacts;
88
- the repository remains canonical.
92
+ the repository remains canonical. Optional session evidence is off in this
93
+ lesson and is introduced in L13.
89
94
 
90
95
  ## Clean up
91
96
 
@@ -17,6 +17,8 @@ review extracted candidates before they become durable memory.
17
17
  - Exact read versus semantic search and substring fallback.
18
18
  - Superseding a fact without losing history.
19
19
  - Session-end candidate extraction and `/memory-review` as the acceptance gate.
20
+ - Typed node metadata, tags, and hard filters as an independent opt-in.
21
+ - Why hindsight notes and logical note address spaces are not reviewed memory.
20
22
  - Deleting synthetic course memory safely.
21
23
 
22
24
  ## Hands-on lab
@@ -31,6 +33,7 @@ candidate.
31
33
  - Search finds a fact without requiring its exact key.
32
34
  - Superseding preserves the prior value in history.
33
35
  - No extracted candidate becomes durable without an explicit review action.
36
+ - Enabling typed nodes changes discovery structure, not the authority of a fact.
34
37
 
35
38
  ## Planned video
36
39
 
@@ -42,3 +45,4 @@ Wave 2 after the lab fixture is executable.
42
45
 
43
46
  - [Operating guide](../../operating.md#the-workflow)
44
47
  - [Storage and deployment](../../storage.md)
48
+ - [L14 - Typed memory and controlled import](L14-typed-memory.md)
@@ -12,6 +12,8 @@ confusing derived knowledge with canonical project sources.
12
12
  ## Planned lesson
13
13
 
14
14
  - SessionStart recall, pre-edit recall, and SessionEnd candidate staging.
15
+ - Optional structured capture, hindsight distillation, and compaction continuity
16
+ as separate post-session paths that do not bypass memory review.
15
17
  - `/wiki-ingest`, `/wiki-query`, and `/wiki-lint`.
16
18
  - Sparse, citation-heavy wiki pages rather than copied documentation.
17
19
  - Alignment gaps, contradictions, and project planning artifacts.
@@ -29,6 +31,7 @@ queue before exiting.
29
31
  - Every derived claim links to a canonical source.
30
32
  - A stale or contradictory page is detected rather than silently trusted.
31
33
  - Automatic hooks are observable and do not bypass human memory review.
34
+ - Captured evidence and notes are never described as canonical project state.
32
35
  - The learner can describe what to do at session start, before editing, and at
33
36
  session end.
34
37
 
@@ -41,3 +44,4 @@ minutes and split wiki maintenance into a second video if rehearsal runs long.
41
44
 
42
45
  - [Operating guide](../../operating.md#the-workflow)
43
46
  - [Git provider configuration](../../git-providers.md)
47
+ - [L13 - Session evidence and hindsight](L13-session-evidence.md)
@@ -14,6 +14,8 @@ will go before starting a session.
14
14
  - The placement rule: the server process owns the store it opens.
15
15
  - Local stdio versus explicitly configured remote HTTP.
16
16
  - AgentFS/SQLite data at rest and project-derived artifacts.
17
+ - Separate local stores and retention for trajectories, notes, immutable
18
+ artifacts, capability callbacks, and evaluation experiments.
17
19
  - Credential boundaries and environment-variable precedence.
18
20
  - Embedding/extraction endpoints and what content they receive.
19
21
  - `cairn memory path` and a preflight data-flow checklist.
@@ -30,6 +32,7 @@ names, and prove that no remote destination was discovered.
30
32
  - Secrets are never printed, committed, or included in screenshots.
31
33
  - Optional model and RAG flows are identified separately from memory storage.
32
34
  - The learner can stop when an effective destination is ambiguous.
35
+ - Independent feature flags are not treated as one blanket consent switch.
33
36
 
34
37
  ## Planned video
35
38
 
@@ -6,12 +6,14 @@
6
6
 
7
7
  ## Outcome
8
8
 
9
- Use optional token-miser/FastContext exploration with compact citations and
10
- understand caching, routing, and fallback behavior.
9
+ Use optional token-miser exploration with compact citations and understand
10
+ caching, routing, and fallback behavior.
11
11
 
12
12
  ## Planned lesson
13
13
 
14
14
  - `context_explore` as a thin delegate rather than a second memory store.
15
+ - token-miser as the public sibling that owns routing and context exploration;
16
+ Cairnkeep retains only the integration seam.
15
17
  - Repository root, query, citations, and cache keys.
16
18
  - Cross-referencing citations with memory and the wiki.
17
19
  - Optional pre-task invocation and its privacy implication.
@@ -17,6 +17,8 @@ through supported seams, delivers no secrets, and upgrades fleets safely.
17
17
  - Machine-private configuration versus distributable defaults.
18
18
  - Bootstrap, diagnostics, fleet status/update, release gates, and rollback.
19
19
  - Compatibility guarantees for existing clients and stored memory.
20
+ - Explicit decisions for every new default-off evidence, capability, and
21
+ evaluation flag rather than enabling the entire advanced stack by policy.
20
22
 
21
23
  ## Hands-on lab
22
24
 
@@ -0,0 +1,71 @@
1
+ # L13 - Session evidence and hindsight
2
+
3
+ **Status:** Brief
4
+ **Track:** Practitioner; Evidence and Evaluation
5
+ **Planned time:** 45 minutes
6
+
7
+ ## Outcome
8
+
9
+ Enable one optional capture path in a disposable project, inspect what was
10
+ retained, distil a local hindsight note, and remove the course evidence without
11
+ confusing it with reviewed durable memory.
12
+
13
+ ## Planned lesson
14
+
15
+ - The lifecycle: transient harness context, redacted trajectory, deterministic
16
+ hindsight note, explicit shared promotion, and reviewed memory.
17
+ - Independent consent flags for trajectory capture, note distillation, note
18
+ enrichment, compaction capture, artifact tools, and artifact HTTP access.
19
+ - Claude Code and OpenCode hooks versus Pi's native `session_shutdown` adapter.
20
+ - Pre-write redaction, reasoning omission, size limits, 30-day defaults, and
21
+ fail-open capture behavior.
22
+ - Compaction summaries as harness-produced continuity evidence, never trusted
23
+ instructions and never automatic fresh-session context.
24
+ - `cairn trajectory`, `cairn notes`, `cairn artifact`, and `cairn doctor
25
+ --repair` inspection and recovery paths.
26
+
27
+ ## Hands-on lab
28
+
29
+ Use a disposable repository and synthetic error. Enable only trajectory capture
30
+ and note distillation in its `.ai/.env`; point `CAIRN_AGENTFS_BASE_DIR` at a
31
+ course-only directory inside that disposable root. Start a supported harness
32
+ through the project launcher, reproduce and resolve the error, then close the
33
+ session.
34
+ Inspect the trajectory, distil the exact session, search its failure signature,
35
+ and preview pruning before removing the disposable project and isolated course
36
+ note root.
37
+
38
+ Core inspection commands:
39
+
40
+ ```bash
41
+ cairn trajectory list --json
42
+ cairn trajectory show SESSION-ID --json
43
+ cairn notes distill --session SESSION-ID --json
44
+ printf '%s\n' 'TypeError: course fixture' \
45
+ | cairn notes search-error --project "$PWD" --json
46
+ cairn notes doctor --json
47
+ cairn trajectory prune --dry-run --json
48
+ cairn artifact list --json
49
+ ```
50
+
51
+ ## Acceptance criteria
52
+
53
+ - Nothing is captured before explicit opt-in.
54
+ - The retained trajectory contains no reasoning field or planted secret.
55
+ - Distillation is local and deterministic when enrichment remains disabled.
56
+ - The learner can distinguish a trajectory, note, artifact, and reviewed
57
+ memory by purpose, store, lifecycle, and trust level.
58
+ - Cleanup targets only the disposable course evidence.
59
+
60
+ ## Planned video
61
+
62
+ Draw the evidence lifecycle, capture one synthetic session, inspect the redacted
63
+ record, and find the resulting failure note. Target 14 minutes. Treat compaction
64
+ and Pi as short boundary chapters rather than requiring three harness demos.
65
+
66
+ ## Source material
67
+
68
+ - [Operating guide: structured trajectories](../../operating.md#structured-session-trajectories-opt-in)
69
+ - [Operating guide: hindsight notes](../../operating.md#hindsight-notes-opt-in)
70
+ - [Privacy and data flow](../../privacy-and-data-flow.md#structured-trajectory-capture)
71
+ - [Storage and deployment](../../storage.md)
@@ -0,0 +1,51 @@
1
+ # L14 - Typed memory and controlled import
2
+
3
+ **Status:** Brief
4
+ **Track:** Evidence and Evaluation
5
+ **Planned time:** 40 minutes
6
+
7
+ ## Outcome
8
+
9
+ Use typed metadata and hard filters, preview a bounded structured import, and
10
+ explain how project-note and shared-note address spaces differ from filesystem
11
+ paths and ordinary memory scopes.
12
+
13
+ ## Planned lesson
14
+
15
+ - `CAIRN_TYPED_MEMORY_NODES` as a default-off server-start contract.
16
+ - Core node types, canonical tags, exact-match ordering, and hard filters before
17
+ semantic ranking.
18
+ - Logical `memory`, `project-notes`, and `shared-notes` address spaces.
19
+ - Bounded schema-v1 `memory_import`, stable `import_id`, dry run, reject by
20
+ default, and explicit supersession with preserved history.
21
+ - Journaled note-address transactions, replay, and `cairn doctor --repair`.
22
+ - Why typed metadata adds discovery structure but does not make a fact true.
23
+
24
+ ## Hands-on lab
25
+
26
+ Start an isolated local server with typed nodes enabled. Write synthetic nodes
27
+ with two types and several normalized tags, compare unfiltered and hard-filtered
28
+ searches, then dry-run a two-item import. Apply it once with reject semantics,
29
+ replay the same `import_id`, and inspect history after one explicit superseding
30
+ import.
31
+
32
+ ## Acceptance criteria
33
+
34
+ - Restarting the server after changing the feature flag is part of the lab.
35
+ - Type and tag filters exclude nonmatching nodes before ranking.
36
+ - The dry run writes nothing, the replay is idempotent, and the default policy
37
+ cannot overwrite an existing key.
38
+ - Imported values never appear in command diagnostics or course screenshots.
39
+ - The learner does not present a logical note address as a host path.
40
+
41
+ ## Planned video
42
+
43
+ Use a small type-and-tag diagram, then demonstrate filtered discovery and the
44
+ dry-run/replay sequence. Target 12 minutes. Keep the full import limits in the
45
+ written lesson rather than narrating a table of numbers.
46
+
47
+ ## Source material
48
+
49
+ - [Operating guide: typed memory nodes](../../operating.md#typed-memory-nodes-and-note-address-spaces-opt-in)
50
+ - [Storage and deployment](../../storage.md#typed-metadata-import-replay-and-history)
51
+ - [Typed memory schema](../../../schemas/memory-node.schema.json)
@@ -0,0 +1,56 @@
1
+ # L15 - Capability governance
2
+
3
+ **Status:** Brief
4
+ **Track:** Evidence and Evaluation; Operator
5
+ **Planned time:** 35 minutes
6
+
7
+ ## Outcome
8
+
9
+ Inspect effective capability state, apply and reset a project override, and
10
+ predict when a harness or memory-server restart is required.
11
+
12
+ ## Planned lesson
13
+
14
+ - The default-off managed contract around eight existing capability owners.
15
+ - Compatibility defaults, project overrides, process overrides, and precedence.
16
+ - `list`, `status`, `enable`, `disable`, `reset`, and payload-free callback
17
+ logging.
18
+ - Tool omission for disabled MCP capabilities versus invocation-time checks for
19
+ offline jobs and operating workflows.
20
+ - Configuration digests as state identity, not approval or quality evidence.
21
+ - Why enabling the master contract does not enable graph or note distillation.
22
+
23
+ ## Hands-on lab
24
+
25
+ Use a disposable project with the master contract enabled. Save JSON status,
26
+ disable `context.explore`, restart the memory server, verify effective state,
27
+ then reset the override and restart again. Toggle callback logging separately
28
+ and confirm that payload-free evidence still requires local trajectory capture.
29
+
30
+ ```bash
31
+ CAIRN_CAPABILITY_CONTRACT=1 cairn capabilities list
32
+ CAIRN_CAPABILITY_CONTRACT=1 cairn capabilities status --json
33
+ CAIRN_CAPABILITY_CONTRACT=1 cairn capabilities disable context.explore
34
+ CAIRN_CAPABILITY_CONTRACT=1 cairn capabilities reset context.explore
35
+ CAIRN_CAPABILITY_CONTRACT=1 cairn capabilities logging enable
36
+ ```
37
+
38
+ ## Acceptance criteria
39
+
40
+ - The learner records the before and after configuration digests.
41
+ - A disabled MCP capability disappears only after a server restart.
42
+ - `reset` restores inherited state; it is not described as `enable`.
43
+ - No callback record contains prompts, tool arguments, results, or memory values.
44
+ - Disabling an optional capability does not break standalone memory.
45
+
46
+ ## Planned video
47
+
48
+ Show precedence as three stacked layers, then perform one disable/reset cycle.
49
+ Target 11 minutes. Use JSON status for evidence and avoid presenting a digest as
50
+ a security signature or performance result.
51
+
52
+ ## Source material
53
+
54
+ - [Operating guide: managed capability contract](../../operating.md#managed-capability-contract-opt-in)
55
+ - [Privacy and data flow](../../privacy-and-data-flow.md#capability-callback-flow)
56
+ - [Capability contract schema](../../../schemas/capability-contract.schema.json)
@@ -0,0 +1,67 @@
1
+ # L16 - Evaluation and ablation
2
+
3
+ **Status:** Brief
4
+ **Track:** Evidence and Evaluation; Operator
5
+ **Planned time:** 50 minutes
6
+
7
+ ## Outcome
8
+
9
+ Validate and run the bundled offline fixture, inspect a canonical two-pass
10
+ report, and explain what a one-capability ablation can and cannot establish.
11
+
12
+ ## Planned lesson
13
+
14
+ - The default-off coordinator and operator-owned adapter boundary.
15
+ - Immutable task sets, paired seeds, fresh isolated workspaces, independent
16
+ verification, and deterministic serial schedules.
17
+ - Run 1 versus fresh Run 2 with a task-contained immutable note snapshot.
18
+ - An all-enabled baseline versus exactly one disabled capability.
19
+ - Eligible pairs, missingness, compatible turn semantics, reported token/cost
20
+ fields, bootstrap intervals, and bounded reports.
21
+ - `offline-framework` evidence versus separately validated live evidence.
22
+ - Dry validation, invocation estimates, explicit `--yes`, cancellation, report
23
+ retention, dry-run pruning, and deletion.
24
+
25
+ ## Hands-on lab
26
+
27
+ Build the server, enable evaluation only for the commands below, and use a
28
+ temporary output directory with the committed fake task set and adapter.
29
+
30
+ ```bash
31
+ npm run build:server
32
+ output=$(mktemp -d)
33
+ CAIRN_EVAL=1 bin/cairn eval validate \
34
+ --task-set examples/eval/task-set.json \
35
+ --adapter examples/eval/adapter.json --output "$output" --json
36
+ CAIRN_EVAL=1 bin/cairn eval run \
37
+ --task-set examples/eval/task-set.json \
38
+ --adapter examples/eval/adapter.json --output "$output" --yes --json
39
+ ```
40
+
41
+ Inspect the reported experiment ID with `cairn eval report`, then run one
42
+ `memory.search` ablation only after reviewing the printed invocation estimate.
43
+ Finish with a dry-run prune. The complete lesson will pin exact expected fields
44
+ after the lab is rehearsed from the packaged CLI rather than a source clone.
45
+
46
+ ## Acceptance criteria
47
+
48
+ - `validate` invokes no adapter and writes no experiment.
49
+ - Every run uses isolated task workspaces and an independent verifier result.
50
+ - Unknown or infrastructure outcomes are not counted as pass or fail.
51
+ - The learner reports estimates, intervals, missingness, and evidence scope
52
+ without claiming causality, significance, or product improvement.
53
+ - Course cleanup is limited to its temporary contained experiment root.
54
+
55
+ ## Planned video
56
+
57
+ Start with the claim boundary, validate before running, inspect one two-pass
58
+ report, and close by marking the fixture `offline-framework`. Target 16 minutes.
59
+ Publish ablation as a second short video if the complete schedule obscures the
60
+ measurement concepts.
61
+
62
+ ## Source material
63
+
64
+ - [Operating guide: evaluation harness](../../operating.md#evaluation-harness-opt-in)
65
+ - [Privacy and data flow](../../privacy-and-data-flow.md#evaluation-adapter-and-report-flow)
66
+ - [Bundled offline evaluation fixture](../../../examples/eval/README.md)
67
+ - [Evaluation task schema](../../../schemas/eval-task-set.schema.json)
@@ -0,0 +1,55 @@
1
+ # L17 - The agent boundary
2
+
3
+ **Status:** Brief
4
+ **Track:** Evidence and Evaluation
5
+ **Planned time:** 25 minutes
6
+
7
+ ## Outcome
8
+
9
+ Separate Cairnkeep's shipped evidence and evaluation substrates from the
10
+ design-only bounded meta-agent proposal, and identify which authority always
11
+ remains with the operator and external harness.
12
+
13
+ ## Planned lesson
14
+
15
+ - Current facts: typed capability state, local evidence, immutable evaluation
16
+ inputs, isolated schedules, independent verification, and canonical reports.
17
+ - Design only: natural-language form compilation, candidate generation,
18
+ candidate injection, comparison, proposal bundles, patch application, and
19
+ rollback ledgers.
20
+ - Cairnkeep coordinates evidence; it does not own a harness inference loop.
21
+ - Natural language is untrusted proposal data, not executable authorization.
22
+ - Separate human gates for form, evaluation, application, confirmation, and
23
+ rollback.
24
+ - Why no CLI command, schema, runtime default, or quality claim ships for the
25
+ proposed meta-agent loop.
26
+
27
+ ## Hands-on lab
28
+
29
+ Give learners a fictional automation proposal. Ask them to mark each operation
30
+ as a current Cairnkeep capability, an operator/external-adapter responsibility,
31
+ or future design-only work. Then identify the minimum approval, immutable input,
32
+ path boundary, resource ceiling, evidence, and rollback information that would
33
+ be required before any future implementation could act.
34
+
35
+ ## Acceptance criteria
36
+
37
+ - No learner attempts to invoke a meta-agent command or treats an illustrative
38
+ block as a supported schema.
39
+ - Every inference execution remains owned by an operator-configured adapter.
40
+ - A capability digest is not treated as approval, identity proof, or evidence
41
+ of improvement.
42
+ - Held-out confirmation data never becomes candidate-generation feedback.
43
+ - The learner can name at least one explicit future prerequisite and one
44
+ unconditional human stop point.
45
+
46
+ ## Planned video
47
+
48
+ Use one ownership diagram and one fictional proposal. Target 9 minutes. Put a
49
+ persistent `DESIGN ONLY` label on every future component and end on the exact
50
+ current-versus-future boundary.
51
+
52
+ ## Source material
53
+
54
+ - [Meta-agent design contract](../../design/meta-agent.md)
55
+ - [Evaluation and ablation](L16-evaluation.md)
@@ -1,7 +1,7 @@
1
1
  # LXX - Lesson title
2
2
 
3
3
  **Status:** Draft
4
- **Track:** Quickstart | Practitioner | Operator
4
+ **Track:** Quickstart | Practitioner | Evidence and Evaluation | Operator
5
5
  **Time:** NN minutes
6
6
  **Tested with:** Cairnkeep X.Y.Z
7
7
 
@@ -44,6 +44,9 @@ failed setup.
44
44
  State where data and credentials go, what leaves the machine, and what remains
45
45
  derived rather than canonical.
46
46
 
47
+ For measurements, also state which claims the retained evidence can and cannot
48
+ support.
49
+
47
50
  ## Clean up
48
51
 
49
52
  Provide reversible cleanup for every resource created by the lesson.
@@ -0,0 +1,20 @@
1
+ # Evidence and Evaluation Track
2
+
3
+ **Audience:** maintainers and advanced practitioners evaluating optional layers
4
+ **Prerequisite:** Practitioner track
5
+ **Total time:** about 3.25 hours
6
+ **Infrastructure:** local fixtures first; a live harness is operator-provided
7
+
8
+ 1. [L13 - Session evidence and hindsight](../lessons/L13-session-evidence.md)
9
+ 2. [L14 - Typed memory and controlled import](../lessons/L14-typed-memory.md)
10
+ 3. [L15 - Capability governance](../lessons/L15-capability-governance.md)
11
+ 4. [L16 - Evaluation and ablation](../lessons/L16-evaluation.md)
12
+ 5. [L17 - The agent boundary](../lessons/L17-agent-boundary.md)
13
+
14
+ The track is complete when the learner can explain each consent boundary,
15
+ inspect and manage every local evidence store, compare baseline and ablation
16
+ reports without making unsupported claims, and distinguish current runtime
17
+ features from the design-only meta-agent contract.
18
+
19
+ Hands-on spine: `course-05-evidence` through `course-07-evaluation` in the
20
+ public [course labs](https://github.com/cairnkeep/cairnkeep-course-labs).