@cairnkeep/cli 2.3.1 → 2.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (151) hide show
  1. package/CHANGELOG.md +85 -0
  2. package/README.md +232 -6
  3. package/bin/cairn +43 -2
  4. package/claude/capability-contract/hooks/capability-command-finish.sh +81 -0
  5. package/claude/capability-contract/hooks/capability-command-start.sh +76 -0
  6. package/claude/hooks/compaction-capture.sh +34 -0
  7. package/claude/hooks/memory-capture.sh +47 -5
  8. package/claude/hooks/memory-wakeup.sh +55 -2
  9. package/docs/building-an-overlay.md +4 -3
  10. package/docs/design/meta-agent.md +439 -0
  11. package/docs/harness-compatibility.md +117 -0
  12. package/docs/learning/CURRICULUM-MAP.md +91 -0
  13. package/docs/learning/FEATURE-GUIDE.md +86 -0
  14. package/docs/learning/PRODUCTION-PLAN.md +45 -14
  15. package/docs/learning/README.md +20 -1
  16. package/docs/learning/instructor-guide.md +6 -0
  17. package/docs/learning/lessons/L00-why-cairnkeep.md +20 -8
  18. package/docs/learning/lessons/L01-safe-trial.md +3 -3
  19. package/docs/learning/lessons/L02-installation.md +23 -5
  20. package/docs/learning/lessons/L03-first-project.md +7 -2
  21. package/docs/learning/lessons/L04-memory-fundamentals.md +4 -0
  22. package/docs/learning/lessons/L05-daily-workflow.md +4 -0
  23. package/docs/learning/lessons/L07-privacy-and-storage.md +3 -0
  24. package/docs/learning/lessons/L10-context-exploration.md +4 -2
  25. package/docs/learning/lessons/L12-managed-overlays.md +2 -0
  26. package/docs/learning/lessons/L13-session-evidence.md +71 -0
  27. package/docs/learning/lessons/L14-typed-memory.md +51 -0
  28. package/docs/learning/lessons/L15-capability-governance.md +56 -0
  29. package/docs/learning/lessons/L16-evaluation.md +67 -0
  30. package/docs/learning/lessons/L17-agent-boundary.md +55 -0
  31. package/docs/learning/templates/lesson-template.md +4 -1
  32. package/docs/learning/tracks/evidence-and-evaluation.md +20 -0
  33. package/docs/learning/tracks/operator.md +8 -2
  34. package/docs/learning/tracks/practitioner.md +6 -1
  35. package/docs/learning/tracks/quickstart.md +3 -0
  36. package/docs/learning/video-scripts/L00-why-cairnkeep.md +21 -17
  37. package/docs/learning/video-scripts/L02-installation.md +9 -2
  38. package/docs/learning/video-scripts/L03-first-project.md +3 -0
  39. package/docs/operating.md +567 -7
  40. package/docs/privacy-and-data-flow.md +417 -0
  41. package/docs/storage.md +252 -0
  42. package/examples/eval/README.md +42 -0
  43. package/examples/eval/adapter.json +12 -0
  44. package/examples/eval/bundled-fake.json +1 -0
  45. package/examples/eval/task-set.json +1 -0
  46. package/mcp-memory-server/dist/artifact-cli.js +184 -0
  47. package/mcp-memory-server/dist/artifact-schema.js +218 -0
  48. package/mcp-memory-server/dist/artifact-store.js +812 -0
  49. package/mcp-memory-server/dist/capability-adapter.js +286 -0
  50. package/mcp-memory-server/dist/capability-cli.js +361 -0
  51. package/mcp-memory-server/dist/capability-config.js +310 -0
  52. package/mcp-memory-server/dist/capability-harness.js +496 -0
  53. package/mcp-memory-server/dist/capability-registry.js +123 -0
  54. package/mcp-memory-server/dist/capability-schema.js +65 -0
  55. package/mcp-memory-server/dist/capability-store.js +431 -0
  56. package/mcp-memory-server/dist/compaction-normalize.js +267 -0
  57. package/mcp-memory-server/dist/eval-cli.js +523 -0
  58. package/mcp-memory-server/dist/eval-plan.js +346 -0
  59. package/mcp-memory-server/dist/eval-process.js +273 -0
  60. package/mcp-memory-server/dist/eval-report.js +609 -0
  61. package/mcp-memory-server/dist/eval-runner.js +827 -0
  62. package/mcp-memory-server/dist/eval-schema.js +368 -0
  63. package/mcp-memory-server/dist/eval-statistics.js +161 -0
  64. package/mcp-memory-server/dist/eval-workspace.js +313 -0
  65. package/mcp-memory-server/dist/failure-signature.js +141 -0
  66. package/mcp-memory-server/dist/index.js +600 -55
  67. package/mcp-memory-server/dist/node-cli.js +84 -0
  68. package/mcp-memory-server/dist/node-schema.js +170 -0
  69. package/mcp-memory-server/dist/node-store.js +455 -0
  70. package/mcp-memory-server/dist/note-cli.js +235 -0
  71. package/mcp-memory-server/dist/note-distiller.js +177 -0
  72. package/mcp-memory-server/dist/note-enrichment.js +93 -0
  73. package/mcp-memory-server/dist/note-schema.js +4 -0
  74. package/mcp-memory-server/dist/note-store.js +985 -0
  75. package/mcp-memory-server/dist/trajectory-cli.js +141 -0
  76. package/mcp-memory-server/dist/trajectory-normalize.js +448 -0
  77. package/mcp-memory-server/dist/trajectory-redaction.js +155 -0
  78. package/mcp-memory-server/dist/trajectory-schema.js +73 -0
  79. package/mcp-memory-server/dist/trajectory-store.js +339 -0
  80. package/mcp-memory-server/package.json +24 -2
  81. package/mcp-memory-server/scripts/fixtures/compaction-claude-code-2.1.219.json +9 -0
  82. package/mcp-memory-server/scripts/fixtures/compaction-claude-code-2.1.220.json +9 -0
  83. package/mcp-memory-server/scripts/fixtures/compaction-opencode-1.17.20-event.json +6 -0
  84. package/mcp-memory-server/scripts/fixtures/compaction-opencode-1.17.20-messages.json +119 -0
  85. package/mcp-memory-server/scripts/fixtures/compaction-unknown-version.json +17 -0
  86. package/opencode/capability-contract/plugins/capability-command.ts +319 -0
  87. package/opencode/command/graphify.md +25 -14
  88. package/opencode/plugins/memory-capture.ts +76 -10
  89. package/opencode/plugins/memory-wakeup.ts +58 -6
  90. package/package.json +8 -4
  91. package/pi/extensions/cairnkeep-trajectory.ts +55 -0
  92. package/schemas/artifact.schema.json +133 -0
  93. package/schemas/capability-callback.schema.json +68 -0
  94. package/schemas/capability-contract.schema.json +126 -0
  95. package/schemas/eval-adapter.schema.json +43 -0
  96. package/schemas/eval-protocol.schema.json +114 -0
  97. package/schemas/eval-report.schema.json +423 -0
  98. package/schemas/eval-task-set.schema.json +114 -0
  99. package/schemas/memory-node.schema.json +90 -0
  100. package/schemas/note.schema.json +112 -0
  101. package/schemas/trajectory-redaction.schema.json +25 -0
  102. package/schemas/trajectory.schema.json +82 -0
  103. package/scripts/bootstrap.sh +6 -0
  104. package/scripts/completion.sh +119 -5
  105. package/scripts/doctor.sh +284 -0
  106. package/scripts/fake-eval-adapter.mjs +120 -0
  107. package/scripts/fixtures/capability-harness-contracts.json +408 -0
  108. package/scripts/install-audit-timer.sh +3 -1
  109. package/scripts/lib/capability-opencode-plugin-harness.mjs +291 -0
  110. package/scripts/lib/trajectory-opencode-plugin-harness.mjs +25 -0
  111. package/scripts/lib/trajectory-pi-extension-harness.mjs +42 -0
  112. package/scripts/memory-wiki-audit.sh +49 -0
  113. package/scripts/smoke-bash32.sh +9 -0
  114. package/scripts/sync-claude-assets.sh +99 -6
  115. package/scripts/sync-opencode-graphify-assets.sh +24 -3
  116. package/scripts/sync-opencode-plugin-assets.sh +31 -5
  117. package/scripts/sync-opencode-security-assets.sh +24 -3
  118. package/scripts/sync-opencode-wiki-assets.sh +24 -3
  119. package/scripts/sync-pi-assets.sh +72 -0
  120. package/scripts/test-audit-timer.sh +98 -0
  121. package/scripts/test-bootstrap-untracked.sh +49 -0
  122. package/scripts/test-cli-dispatch.sh +41 -2
  123. package/scripts/test-compaction-hooks.sh +495 -0
  124. package/scripts/test-completion.sh +77 -0
  125. package/scripts/test-doctor.sh +348 -1
  126. package/scripts/test-eval-cli.sh +175 -0
  127. package/scripts/test-launcher-hooks.sh +630 -3
  128. package/scripts/test-learning-docs.sh +20 -4
  129. package/scripts/test-notes-cli.sh +114 -0
  130. package/scripts/test-package-install.sh +254 -0
  131. package/scripts/test-phase16-node-lifecycle.sh +50 -0
  132. package/scripts/test-phase17-artifact-lifecycle.sh +370 -0
  133. package/scripts/test-phase18-capability-lifecycle.sh +504 -0
  134. package/scripts/test-phase18-harness-boundary.sh +1060 -0
  135. package/scripts/test-phase19-eval-lifecycle.sh +1015 -0
  136. package/scripts/test-trajectory-cli.sh +87 -0
  137. package/scripts/test-trajectory-hooks.sh +142 -0
  138. package/scripts/test-uninstall.sh +231 -5
  139. package/scripts/uninstall.sh +38 -5
  140. package/scripts/verify-dependency-security.mjs +121 -0
  141. package/scripts/verify-docs-parity.sh +527 -1
  142. package/scripts/verify-phase18-runtime-evidence.mjs +895 -0
  143. package/scripts/verify-phase19-runtime-evidence.mjs +947 -0
  144. package/templates/capabilities.json.template +7 -0
  145. package/templates/env.example.template +51 -0
  146. package/templates/memory-wiki-audit.service.template +1 -1
  147. package/templates/start-claude.sh.template +36 -0
  148. package/templates/start-kimi.sh.template +39 -0
  149. package/templates/start-opencode.sh.template +47 -5
  150. package/templates/start-pi.sh.template +30 -0
  151. package/templates/trajectory-redaction.json.template +4 -0
@@ -3,7 +3,7 @@
3
3
  **Status:** Ready
4
4
  **Track:** Quickstart
5
5
  **Time:** 25 minutes
6
- **Tested with:** Cairnkeep 2.3.1
6
+ **Tested with:** Cairnkeep 2.5.0
7
7
 
8
8
  ## Outcome
9
9
 
@@ -41,6 +41,10 @@ fact, and recall it from a new harness session.
41
41
  cairn doctor
42
42
  ```
43
43
 
44
+ Also leave trajectory, note, compaction, artifact, typed-memory, capability,
45
+ and evaluation flags unset. The quickstart proves core memory without
46
+ silently changing the amount of retained data.
47
+
44
48
  4. Launch Claude Code through the generated project launcher:
45
49
 
46
50
  ```bash
@@ -85,7 +89,8 @@ The lesson is complete only if:
85
89
 
86
90
  Use synthetic memory in the course. The local stdio server writes to local
87
91
  AgentFS/SQLite storage. Project wiki and planning files are derived artifacts;
88
- the repository remains canonical.
92
+ the repository remains canonical. Optional session evidence is off in this
93
+ lesson and is introduced in L13.
89
94
 
90
95
  ## Clean up
91
96
 
@@ -17,6 +17,8 @@ review extracted candidates before they become durable memory.
17
17
  - Exact read versus semantic search and substring fallback.
18
18
  - Superseding a fact without losing history.
19
19
  - Session-end candidate extraction and `/memory-review` as the acceptance gate.
20
+ - Typed node metadata, tags, and hard filters as an independent opt-in.
21
+ - Why hindsight notes and logical note address spaces are not reviewed memory.
20
22
  - Deleting synthetic course memory safely.
21
23
 
22
24
  ## Hands-on lab
@@ -31,6 +33,7 @@ candidate.
31
33
  - Search finds a fact without requiring its exact key.
32
34
  - Superseding preserves the prior value in history.
33
35
  - No extracted candidate becomes durable without an explicit review action.
36
+ - Enabling typed nodes changes discovery structure, not the authority of a fact.
34
37
 
35
38
  ## Planned video
36
39
 
@@ -42,3 +45,4 @@ Wave 2 after the lab fixture is executable.
42
45
 
43
46
  - [Operating guide](../../operating.md#the-workflow)
44
47
  - [Storage and deployment](../../storage.md)
48
+ - [L14 - Typed memory and controlled import](L14-typed-memory.md)
@@ -12,6 +12,8 @@ confusing derived knowledge with canonical project sources.
12
12
  ## Planned lesson
13
13
 
14
14
  - SessionStart recall, pre-edit recall, and SessionEnd candidate staging.
15
+ - Optional structured capture, hindsight distillation, and compaction continuity
16
+ as separate post-session paths that do not bypass memory review.
15
17
  - `/wiki-ingest`, `/wiki-query`, and `/wiki-lint`.
16
18
  - Sparse, citation-heavy wiki pages rather than copied documentation.
17
19
  - Alignment gaps, contradictions, and project planning artifacts.
@@ -29,6 +31,7 @@ queue before exiting.
29
31
  - Every derived claim links to a canonical source.
30
32
  - A stale or contradictory page is detected rather than silently trusted.
31
33
  - Automatic hooks are observable and do not bypass human memory review.
34
+ - Captured evidence and notes are never described as canonical project state.
32
35
  - The learner can describe what to do at session start, before editing, and at
33
36
  session end.
34
37
 
@@ -41,3 +44,4 @@ minutes and split wiki maintenance into a second video if rehearsal runs long.
41
44
 
42
45
  - [Operating guide](../../operating.md#the-workflow)
43
46
  - [Git provider configuration](../../git-providers.md)
47
+ - [L13 - Session evidence and hindsight](L13-session-evidence.md)
@@ -14,6 +14,8 @@ will go before starting a session.
14
14
  - The placement rule: the server process owns the store it opens.
15
15
  - Local stdio versus explicitly configured remote HTTP.
16
16
  - AgentFS/SQLite data at rest and project-derived artifacts.
17
+ - Separate local stores and retention for trajectories, notes, immutable
18
+ artifacts, capability callbacks, and evaluation experiments.
17
19
  - Credential boundaries and environment-variable precedence.
18
20
  - Embedding/extraction endpoints and what content they receive.
19
21
  - `cairn memory path` and a preflight data-flow checklist.
@@ -30,6 +32,7 @@ names, and prove that no remote destination was discovered.
30
32
  - Secrets are never printed, committed, or included in screenshots.
31
33
  - Optional model and RAG flows are identified separately from memory storage.
32
34
  - The learner can stop when an effective destination is ambiguous.
35
+ - Independent feature flags are not treated as one blanket consent switch.
33
36
 
34
37
  ## Planned video
35
38
 
@@ -6,12 +6,14 @@
6
6
 
7
7
  ## Outcome
8
8
 
9
- Use optional token-miser/FastContext exploration with compact citations and
10
- understand caching, routing, and fallback behavior.
9
+ Use optional token-miser exploration with compact citations and understand
10
+ caching, routing, and fallback behavior.
11
11
 
12
12
  ## Planned lesson
13
13
 
14
14
  - `context_explore` as a thin delegate rather than a second memory store.
15
+ - token-miser as the public sibling that owns routing and context exploration;
16
+ Cairnkeep retains only the integration seam.
15
17
  - Repository root, query, citations, and cache keys.
16
18
  - Cross-referencing citations with memory and the wiki.
17
19
  - Optional pre-task invocation and its privacy implication.
@@ -17,6 +17,8 @@ through supported seams, delivers no secrets, and upgrades fleets safely.
17
17
  - Machine-private configuration versus distributable defaults.
18
18
  - Bootstrap, diagnostics, fleet status/update, release gates, and rollback.
19
19
  - Compatibility guarantees for existing clients and stored memory.
20
+ - Explicit decisions for every new default-off evidence, capability, and
21
+ evaluation flag rather than enabling the entire advanced stack by policy.
20
22
 
21
23
  ## Hands-on lab
22
24
 
@@ -0,0 +1,71 @@
1
+ # L13 - Session evidence and hindsight
2
+
3
+ **Status:** Brief
4
+ **Track:** Practitioner; Evidence and Evaluation
5
+ **Planned time:** 45 minutes
6
+
7
+ ## Outcome
8
+
9
+ Enable one optional capture path in a disposable project, inspect what was
10
+ retained, distil a local hindsight note, and remove the course evidence without
11
+ confusing it with reviewed durable memory.
12
+
13
+ ## Planned lesson
14
+
15
+ - The lifecycle: transient harness context, redacted trajectory, deterministic
16
+ hindsight note, explicit shared promotion, and reviewed memory.
17
+ - Independent consent flags for trajectory capture, note distillation, note
18
+ enrichment, compaction capture, artifact tools, and artifact HTTP access.
19
+ - Claude Code and OpenCode hooks versus Pi's native `session_shutdown` adapter.
20
+ - Pre-write redaction, reasoning omission, size limits, 30-day defaults, and
21
+ fail-open capture behavior.
22
+ - Compaction summaries as harness-produced continuity evidence, never trusted
23
+ instructions and never automatic fresh-session context.
24
+ - `cairn trajectory`, `cairn notes`, `cairn artifact`, and `cairn doctor
25
+ --repair` inspection and recovery paths.
26
+
27
+ ## Hands-on lab
28
+
29
+ Use a disposable repository and synthetic error. Enable only trajectory capture
30
+ and note distillation in its `.ai/.env`; point `CAIRN_AGENTFS_BASE_DIR` at a
31
+ course-only directory inside that disposable root. Start a supported harness
32
+ through the project launcher, reproduce and resolve the error, then close the
33
+ session.
34
+ Inspect the trajectory, distil the exact session, search its failure signature,
35
+ and preview pruning before removing the disposable project and isolated course
36
+ note root.
37
+
38
+ Core inspection commands:
39
+
40
+ ```bash
41
+ cairn trajectory list --json
42
+ cairn trajectory show SESSION-ID --json
43
+ cairn notes distill --session SESSION-ID --json
44
+ printf '%s\n' 'TypeError: course fixture' \
45
+ | cairn notes search-error --project "$PWD" --json
46
+ cairn notes doctor --json
47
+ cairn trajectory prune --dry-run --json
48
+ cairn artifact list --json
49
+ ```
50
+
51
+ ## Acceptance criteria
52
+
53
+ - Nothing is captured before explicit opt-in.
54
+ - The retained trajectory contains no reasoning field or planted secret.
55
+ - Distillation is local and deterministic when enrichment remains disabled.
56
+ - The learner can distinguish a trajectory, note, artifact, and reviewed
57
+ memory by purpose, store, lifecycle, and trust level.
58
+ - Cleanup targets only the disposable course evidence.
59
+
60
+ ## Planned video
61
+
62
+ Draw the evidence lifecycle, capture one synthetic session, inspect the redacted
63
+ record, and find the resulting failure note. Target 14 minutes. Treat compaction
64
+ and Pi as short boundary chapters rather than requiring three harness demos.
65
+
66
+ ## Source material
67
+
68
+ - [Operating guide: structured trajectories](../../operating.md#structured-session-trajectories-opt-in)
69
+ - [Operating guide: hindsight notes](../../operating.md#hindsight-notes-opt-in)
70
+ - [Privacy and data flow](../../privacy-and-data-flow.md#structured-trajectory-capture)
71
+ - [Storage and deployment](../../storage.md)
@@ -0,0 +1,51 @@
1
+ # L14 - Typed memory and controlled import
2
+
3
+ **Status:** Brief
4
+ **Track:** Evidence and Evaluation
5
+ **Planned time:** 40 minutes
6
+
7
+ ## Outcome
8
+
9
+ Use typed metadata and hard filters, preview a bounded structured import, and
10
+ explain how project-note and shared-note address spaces differ from filesystem
11
+ paths and ordinary memory scopes.
12
+
13
+ ## Planned lesson
14
+
15
+ - `CAIRN_TYPED_MEMORY_NODES` as a default-off server-start contract.
16
+ - Core node types, canonical tags, exact-match ordering, and hard filters before
17
+ semantic ranking.
18
+ - Logical `memory`, `project-notes`, and `shared-notes` address spaces.
19
+ - Bounded schema-v1 `memory_import`, stable `import_id`, dry run, reject by
20
+ default, and explicit supersession with preserved history.
21
+ - Journaled note-address transactions, replay, and `cairn doctor --repair`.
22
+ - Why typed metadata adds discovery structure but does not make a fact true.
23
+
24
+ ## Hands-on lab
25
+
26
+ Start an isolated local server with typed nodes enabled. Write synthetic nodes
27
+ with two types and several normalized tags, compare unfiltered and hard-filtered
28
+ searches, then dry-run a two-item import. Apply it once with reject semantics,
29
+ replay the same `import_id`, and inspect history after one explicit superseding
30
+ import.
31
+
32
+ ## Acceptance criteria
33
+
34
+ - Restarting the server after changing the feature flag is part of the lab.
35
+ - Type and tag filters exclude nonmatching nodes before ranking.
36
+ - The dry run writes nothing, the replay is idempotent, and the default policy
37
+ cannot overwrite an existing key.
38
+ - Imported values never appear in command diagnostics or course screenshots.
39
+ - The learner does not present a logical note address as a host path.
40
+
41
+ ## Planned video
42
+
43
+ Use a small type-and-tag diagram, then demonstrate filtered discovery and the
44
+ dry-run/replay sequence. Target 12 minutes. Keep the full import limits in the
45
+ written lesson rather than narrating a table of numbers.
46
+
47
+ ## Source material
48
+
49
+ - [Operating guide: typed memory nodes](../../operating.md#typed-memory-nodes-and-note-address-spaces-opt-in)
50
+ - [Storage and deployment](../../storage.md#typed-metadata-import-replay-and-history)
51
+ - [Typed memory schema](../../../schemas/memory-node.schema.json)
@@ -0,0 +1,56 @@
1
+ # L15 - Capability governance
2
+
3
+ **Status:** Brief
4
+ **Track:** Evidence and Evaluation; Operator
5
+ **Planned time:** 35 minutes
6
+
7
+ ## Outcome
8
+
9
+ Inspect effective capability state, apply and reset a project override, and
10
+ predict when a harness or memory-server restart is required.
11
+
12
+ ## Planned lesson
13
+
14
+ - The default-off managed contract around eight existing capability owners.
15
+ - Compatibility defaults, project overrides, process overrides, and precedence.
16
+ - `list`, `status`, `enable`, `disable`, `reset`, and payload-free callback
17
+ logging.
18
+ - Tool omission for disabled MCP capabilities versus invocation-time checks for
19
+ offline jobs and operating workflows.
20
+ - Configuration digests as state identity, not approval or quality evidence.
21
+ - Why enabling the master contract does not enable graph or note distillation.
22
+
23
+ ## Hands-on lab
24
+
25
+ Use a disposable project with the master contract enabled. Save JSON status,
26
+ disable `context.explore`, restart the memory server, verify effective state,
27
+ then reset the override and restart again. Toggle callback logging separately
28
+ and confirm that payload-free evidence still requires local trajectory capture.
29
+
30
+ ```bash
31
+ CAIRN_CAPABILITY_CONTRACT=1 cairn capabilities list
32
+ CAIRN_CAPABILITY_CONTRACT=1 cairn capabilities status --json
33
+ CAIRN_CAPABILITY_CONTRACT=1 cairn capabilities disable context.explore
34
+ CAIRN_CAPABILITY_CONTRACT=1 cairn capabilities reset context.explore
35
+ CAIRN_CAPABILITY_CONTRACT=1 cairn capabilities logging enable
36
+ ```
37
+
38
+ ## Acceptance criteria
39
+
40
+ - The learner records the before and after configuration digests.
41
+ - A disabled MCP capability disappears only after a server restart.
42
+ - `reset` restores inherited state; it is not described as `enable`.
43
+ - No callback record contains prompts, tool arguments, results, or memory values.
44
+ - Disabling an optional capability does not break standalone memory.
45
+
46
+ ## Planned video
47
+
48
+ Show precedence as three stacked layers, then perform one disable/reset cycle.
49
+ Target 11 minutes. Use JSON status for evidence and avoid presenting a digest as
50
+ a security signature or performance result.
51
+
52
+ ## Source material
53
+
54
+ - [Operating guide: managed capability contract](../../operating.md#managed-capability-contract-opt-in)
55
+ - [Privacy and data flow](../../privacy-and-data-flow.md#capability-callback-flow)
56
+ - [Capability contract schema](../../../schemas/capability-contract.schema.json)
@@ -0,0 +1,67 @@
1
+ # L16 - Evaluation and ablation
2
+
3
+ **Status:** Brief
4
+ **Track:** Evidence and Evaluation; Operator
5
+ **Planned time:** 50 minutes
6
+
7
+ ## Outcome
8
+
9
+ Validate and run the bundled offline fixture, inspect a canonical two-pass
10
+ report, and explain what a one-capability ablation can and cannot establish.
11
+
12
+ ## Planned lesson
13
+
14
+ - The default-off coordinator and operator-owned adapter boundary.
15
+ - Immutable task sets, paired seeds, fresh isolated workspaces, independent
16
+ verification, and deterministic serial schedules.
17
+ - Run 1 versus fresh Run 2 with a task-contained immutable note snapshot.
18
+ - An all-enabled baseline versus exactly one disabled capability.
19
+ - Eligible pairs, missingness, compatible turn semantics, reported token/cost
20
+ fields, bootstrap intervals, and bounded reports.
21
+ - `offline-framework` evidence versus separately validated live evidence.
22
+ - Dry validation, invocation estimates, explicit `--yes`, cancellation, report
23
+ retention, dry-run pruning, and deletion.
24
+
25
+ ## Hands-on lab
26
+
27
+ Build the server, enable evaluation only for the commands below, and use a
28
+ temporary output directory with the committed fake task set and adapter.
29
+
30
+ ```bash
31
+ npm run build:server
32
+ output=$(mktemp -d)
33
+ CAIRN_EVAL=1 bin/cairn eval validate \
34
+ --task-set examples/eval/task-set.json \
35
+ --adapter examples/eval/adapter.json --output "$output" --json
36
+ CAIRN_EVAL=1 bin/cairn eval run \
37
+ --task-set examples/eval/task-set.json \
38
+ --adapter examples/eval/adapter.json --output "$output" --yes --json
39
+ ```
40
+
41
+ Inspect the reported experiment ID with `cairn eval report`, then run one
42
+ `memory.search` ablation only after reviewing the printed invocation estimate.
43
+ Finish with a dry-run prune. The complete lesson will pin exact expected fields
44
+ after the lab is rehearsed from the packaged CLI rather than a source clone.
45
+
46
+ ## Acceptance criteria
47
+
48
+ - `validate` invokes no adapter and writes no experiment.
49
+ - Every run uses isolated task workspaces and an independent verifier result.
50
+ - Unknown or infrastructure outcomes are not counted as pass or fail.
51
+ - The learner reports estimates, intervals, missingness, and evidence scope
52
+ without claiming causality, significance, or product improvement.
53
+ - Course cleanup is limited to its temporary contained experiment root.
54
+
55
+ ## Planned video
56
+
57
+ Start with the claim boundary, validate before running, inspect one two-pass
58
+ report, and close by marking the fixture `offline-framework`. Target 16 minutes.
59
+ Publish ablation as a second short video if the complete schedule obscures the
60
+ measurement concepts.
61
+
62
+ ## Source material
63
+
64
+ - [Operating guide: evaluation harness](../../operating.md#evaluation-harness-opt-in)
65
+ - [Privacy and data flow](../../privacy-and-data-flow.md#evaluation-adapter-and-report-flow)
66
+ - [Bundled offline evaluation fixture](../../../examples/eval/README.md)
67
+ - [Evaluation task schema](../../../schemas/eval-task-set.schema.json)
@@ -0,0 +1,55 @@
1
+ # L17 - The agent boundary
2
+
3
+ **Status:** Brief
4
+ **Track:** Evidence and Evaluation
5
+ **Planned time:** 25 minutes
6
+
7
+ ## Outcome
8
+
9
+ Separate Cairnkeep's shipped evidence and evaluation substrates from the
10
+ design-only bounded meta-agent proposal, and identify which authority always
11
+ remains with the operator and external harness.
12
+
13
+ ## Planned lesson
14
+
15
+ - Current facts: typed capability state, local evidence, immutable evaluation
16
+ inputs, isolated schedules, independent verification, and canonical reports.
17
+ - Design only: natural-language form compilation, candidate generation,
18
+ candidate injection, comparison, proposal bundles, patch application, and
19
+ rollback ledgers.
20
+ - Cairnkeep coordinates evidence; it does not own a harness inference loop.
21
+ - Natural language is untrusted proposal data, not executable authorization.
22
+ - Separate human gates for form, evaluation, application, confirmation, and
23
+ rollback.
24
+ - Why no CLI command, schema, runtime default, or quality claim ships for the
25
+ proposed meta-agent loop.
26
+
27
+ ## Hands-on lab
28
+
29
+ Give learners a fictional automation proposal. Ask them to mark each operation
30
+ as a current Cairnkeep capability, an operator/external-adapter responsibility,
31
+ or future design-only work. Then identify the minimum approval, immutable input,
32
+ path boundary, resource ceiling, evidence, and rollback information that would
33
+ be required before any future implementation could act.
34
+
35
+ ## Acceptance criteria
36
+
37
+ - No learner attempts to invoke a meta-agent command or treats an illustrative
38
+ block as a supported schema.
39
+ - Every inference execution remains owned by an operator-configured adapter.
40
+ - A capability digest is not treated as approval, identity proof, or evidence
41
+ of improvement.
42
+ - Held-out confirmation data never becomes candidate-generation feedback.
43
+ - The learner can name at least one explicit future prerequisite and one
44
+ unconditional human stop point.
45
+
46
+ ## Planned video
47
+
48
+ Use one ownership diagram and one fictional proposal. Target 9 minutes. Put a
49
+ persistent `DESIGN ONLY` label on every future component and end on the exact
50
+ current-versus-future boundary.
51
+
52
+ ## Source material
53
+
54
+ - [Meta-agent design contract](../../design/meta-agent.md)
55
+ - [Evaluation and ablation](L16-evaluation.md)
@@ -1,7 +1,7 @@
1
1
  # LXX - Lesson title
2
2
 
3
3
  **Status:** Draft
4
- **Track:** Quickstart | Practitioner | Operator
4
+ **Track:** Quickstart | Practitioner | Evidence and Evaluation | Operator
5
5
  **Time:** NN minutes
6
6
  **Tested with:** Cairnkeep X.Y.Z
7
7
 
@@ -44,6 +44,9 @@ failed setup.
44
44
  State where data and credentials go, what leaves the machine, and what remains
45
45
  derived rather than canonical.
46
46
 
47
+ For measurements, also state which claims the retained evidence can and cannot
48
+ support.
49
+
47
50
  ## Clean up
48
51
 
49
52
  Provide reversible cleanup for every resource created by the lesson.
@@ -0,0 +1,20 @@
1
+ # Evidence and Evaluation Track
2
+
3
+ **Audience:** maintainers and advanced practitioners evaluating optional layers
4
+ **Prerequisite:** Practitioner track
5
+ **Total time:** about 3.25 hours
6
+ **Infrastructure:** local fixtures first; a live harness is operator-provided
7
+
8
+ 1. [L13 - Session evidence and hindsight](../lessons/L13-session-evidence.md)
9
+ 2. [L14 - Typed memory and controlled import](../lessons/L14-typed-memory.md)
10
+ 3. [L15 - Capability governance](../lessons/L15-capability-governance.md)
11
+ 4. [L16 - Evaluation and ablation](../lessons/L16-evaluation.md)
12
+ 5. [L17 - The agent boundary](../lessons/L17-agent-boundary.md)
13
+
14
+ The track is complete when the learner can explain each consent boundary,
15
+ inspect and manage every local evidence store, compare baseline and ablation
16
+ reports without making unsupported claims, and distinguish current runtime
17
+ features from the design-only meta-agent contract.
18
+
19
+ Hands-on spine: `course-05-evidence` through `course-07-evaluation` in the
20
+ public [course labs](https://github.com/cairnkeep/cairnkeep-course-labs).
@@ -1,14 +1,20 @@
1
1
  # Operator Track
2
2
 
3
3
  **Audience:** advanced users and platform maintainers
4
- **Prerequisite:** Practitioner track
5
- **Total time:** about 3 hours
4
+ **Prerequisite:** Practitioner track; L13 before operating captured evidence
5
+ **Total time:** about 4 hours
6
6
 
7
7
  1. [L09 - Domain knowledge with RAG](../lessons/L09-domain-knowledge.md)
8
8
  2. [L10 - Faster context exploration](../lessons/L10-context-exploration.md)
9
9
  3. [L11 - Containers and isolation](../lessons/L11-containers.md)
10
10
  4. [L12 - Managed overlays](../lessons/L12-managed-overlays.md)
11
+ 5. [L15 - Capability governance](../lessons/L15-capability-governance.md)
12
+ 6. [L16 - Evaluation and ablation](../lessons/L16-evaluation.md)
11
13
 
12
14
  All modules are optional. The track is complete when the operator can add and
13
15
  remove each integration without breaking the standalone local workflow and can
14
16
  explain its storage, network, credential, and rollback boundaries.
17
+
18
+ Hands-on spine: `course-04-operation`, `course-06-governance`, and
19
+ `course-07-evaluation` in the public
20
+ [course labs](https://github.com/cairnkeep/cairnkeep-course-labs).
@@ -2,14 +2,19 @@
2
2
 
3
3
  **Audience:** developers adopting Cairnkeep in daily work
4
4
  **Prerequisite:** Quickstart track
5
- **Total time:** about 3.5 hours
5
+ **Total time:** about 4 hours
6
6
 
7
7
  1. [L04 - Memory fundamentals](../lessons/L04-memory-fundamentals.md)
8
8
  2. [L05 - The daily workflow](../lessons/L05-daily-workflow.md)
9
9
  3. [L06 - Repository review and security](../lessons/L06-repository-quality.md)
10
10
  4. [L07 - Privacy and storage](../lessons/L07-privacy-and-storage.md)
11
11
  5. [L08 - Multiple machines](../lessons/L08-multiple-machines.md)
12
+ 6. [L13 - Session evidence and hindsight](../lessons/L13-session-evidence.md)
12
13
 
13
14
  The track is complete when the learner can decide what should become memory,
14
15
  review derived knowledge, run governed repository checks, locate the store, and
15
16
  perform a reversible backup or migration.
17
+
18
+ Hands-on spine: `course-02-memory`, `course-03-quality`, and
19
+ `course-05-evidence` in the public
20
+ [course labs](https://github.com/cairnkeep/cairnkeep-course-labs).
@@ -12,3 +12,6 @@
12
12
  The track is complete when a learner can start a new harness session, retrieve
13
13
  a previously accepted fact, and explain that default stdio memory is stored on
14
14
  the local machine rather than a discovered remote service.
15
+
16
+ Hands-on spine: `course-00-app` through `course-01-bootstrap` in the public
17
+ [course labs](https://github.com/cairnkeep/cairnkeep-course-labs).
@@ -5,7 +5,7 @@
5
5
 
6
6
  ## Recording setup
7
7
 
8
- - Show only the four-layer table from the lesson.
8
+ - Show only the trust-ladder table from the lesson.
9
9
  - Prepare one slide containing the question: “What should survive this session?”
10
10
  - No terminal or private project is required.
11
11
 
@@ -24,12 +24,14 @@ remember, and what must remain in the repository or another canonical system.”
24
24
 
25
25
  **Show:** The lesson outcome.
26
26
 
27
- ## 01:00 - Four layers
27
+ ## 01:00 - Trust ladder
28
28
 
29
- **Say:** “Start with four different layers. The current conversation is working
30
- context. Cairnkeep memory is a set of concise operational facts. The project
31
- wiki is derived, cited knowledge. Code, tests, specifications, policies, and
32
- issue trackers remain canonical.”
29
+ **Say:** “Follow information through a trust ladder. The current conversation is
30
+ working context. Optional captured evidence and hindsight notes preserve bounded
31
+ local observations, but they are not automatically trusted memory. Cairnkeep
32
+ memory holds concise operational facts. The project wiki is derived, cited
33
+ knowledge. Code, tests, specifications, policies, and issue trackers remain
34
+ canonical.”
33
35
 
34
36
  **Show:** Reveal one table row at a time.
35
37
 
@@ -38,10 +40,11 @@ agent find and apply knowledge; it does not redefine truth.”
38
40
 
39
41
  ## 02:40 - Memory versus RAG
40
42
 
41
- **Say:** “Document RAG answers a different question: which parts of a larger
42
- document are relevant right now? Cairnkeep can integrate with a RAG system, but
43
- it does not require one. A decision such as ‘use transactional migrations’ is a
44
- memory. The complete migration policy remains a document.”
43
+ **Say:** “Document RAG and codebase exploration answer a different question:
44
+ which source material is relevant right now? Cairnkeep can integrate with those
45
+ retrieval paths, but it does not require them. A decision such as ‘use
46
+ transactional migrations’ is a memory. The complete migration policy remains a
47
+ document.”
45
48
 
46
49
  **Show:** Memory on the left, document retrieval on the right.
47
50
 
@@ -57,20 +60,21 @@ until confirmed. A verified operational prohibition is a durable constraint.”
57
60
  ## 05:10 - Boundary
58
61
 
59
62
  **Say:** “Cairnkeep does not store every sentence from a session. The default
60
- stdio setup stores memory on this computer. Remote storage, model extraction,
61
- embeddings, and document RAG are all explicit, optional choices.”
63
+ stdio setup stores memory on this computer. Structured capture, notes,
64
+ artifacts, evaluation, remote storage, model extraction, embeddings, and
65
+ document RAG are separate explicit choices.”
62
66
 
63
67
  **Show:** “Local by default. Optional means opt-in.”
64
68
 
65
69
  ## 06:10 - Recap
66
70
 
67
- **Say:** “Remember three things: durable memory is selective, derived knowledge
68
- does not overrule canonical sources, and optional services are not prerequisites.
69
- In L01 we will inspect the published container without installing the complete
70
- workflow.”
71
+ **Say:** “Remember three things: durable memory is selective, evidence is not
72
+ automatically trusted memory, and canonical sources still win. Optional layers
73
+ are not prerequisites. In L01 we will inspect the published container without
74
+ installing the complete workflow.”
71
75
 
72
76
  ## Editing notes
73
77
 
74
- - Chapters: Problem, Four layers, Memory vs RAG, Exercise, Boundary.
78
+ - Chapters: Problem, Trust ladder, Memory vs retrieval, Exercise, Boundary.
75
79
  - Link L00 and L01 in the description.
76
80
  - Re-record the version-independent slide only if the conceptual model changes.
@@ -71,13 +71,20 @@ the harness after applying assets.”
71
71
 
72
72
  **Show:** The two verification commands together.
73
73
 
74
- ## 09:40 - Boundary
74
+ ## 09:30 - Optional shell completion
75
+
76
+ **Do:** Run `source <(cairn completion bash)` and complete `cairn tra`.
77
+
78
+ **Say:** “Completion is optional convenience. It does not start another server
79
+ or enable any feature.”
80
+
81
+ ## 10:00 - Boundary
75
82
 
76
83
  **Say:** “This default server is local. We did not configure embeddings,
77
84
  document RAG, or remote HTTP memory. Those choices have separate lessons
78
85
  because they change data flow.”
79
86
 
80
- ## 10:30 - Recap
87
+ ## 11:00 - Recap
81
88
 
82
89
  **Say:** “The machine setup has three pieces: CLI, MCP registration, and the
83
90
  operating layer. In L03 we add the fourth piece, a bootstrapped project, and
@@ -43,6 +43,9 @@ does not reinstall the machine-level MCP server or operating assets.”
43
43
  **Say:** “Optional unconfigured integrations may be skipped. A configured but
44
44
  unreachable dependency is a failure and should be fixed before relying on it.”
45
45
 
46
+ **Point out:** Session capture, notes, compaction, artifacts, typed memory,
47
+ capability governance, and evaluation remain off for the core-memory proof.
48
+
46
49
  ## 05:00 - Launch and remember
47
50
 
48
51
  **Do:** Start `./.ai/start-claude.sh`.