daftari 3.10.0 → 3.12.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (229) hide show
  1. package/CHANGELOG.md +59 -0
  2. package/README.md +211 -865
  3. package/dist/access/rbac.d.ts +2 -0
  4. package/dist/access/rbac.d.ts.map +1 -1
  5. package/dist/access/rbac.js +11 -0
  6. package/dist/access/rbac.js.map +1 -1
  7. package/dist/board/board.test.js +8 -7
  8. package/dist/board/board.test.js.map +1 -1
  9. package/dist/board/ledger.test.js +24 -23
  10. package/dist/board/ledger.test.js.map +1 -1
  11. package/dist/board/reconcile.test.js +79 -78
  12. package/dist/board/reconcile.test.js.map +1 -1
  13. package/dist/board/sources/lint.d.ts.map +1 -1
  14. package/dist/board/sources/lint.js +6 -2
  15. package/dist/board/sources/lint.js.map +1 -1
  16. package/dist/board/sources/lint.test.js +62 -21
  17. package/dist/board/sources/lint.test.js.map +1 -1
  18. package/dist/board/sources/staged.test.js +15 -14
  19. package/dist/board/sources/staged.test.js.map +1 -1
  20. package/dist/board/sources/staleness.test.js +43 -42
  21. package/dist/board/sources/staleness.test.js.map +1 -1
  22. package/dist/board/sources/tension.test.js +15 -14
  23. package/dist/board/sources/tension.test.js.map +1 -1
  24. package/dist/board/sources/tier2.test.js +16 -15
  25. package/dist/board/sources/tier2.test.js.map +1 -1
  26. package/dist/cli.d.ts.map +1 -1
  27. package/dist/cli.js +6 -0
  28. package/dist/cli.js.map +1 -1
  29. package/dist/consolidate/index.d.ts.map +1 -1
  30. package/dist/consolidate/index.js +40 -9
  31. package/dist/consolidate/index.js.map +1 -1
  32. package/dist/curation/consumes.d.ts +8 -0
  33. package/dist/curation/consumes.d.ts.map +1 -1
  34. package/dist/curation/consumes.js +27 -0
  35. package/dist/curation/consumes.js.map +1 -1
  36. package/dist/curation/lint-voice.d.ts.map +1 -1
  37. package/dist/curation/lint-voice.js +3 -1
  38. package/dist/curation/lint-voice.js.map +1 -1
  39. package/dist/curation/lint.d.ts +4 -1
  40. package/dist/curation/lint.d.ts.map +1 -1
  41. package/dist/curation/lint.js +20 -2
  42. package/dist/curation/lint.js.map +1 -1
  43. package/dist/curation/source-refs.d.ts +42 -0
  44. package/dist/curation/source-refs.d.ts.map +1 -0
  45. package/dist/curation/source-refs.js +168 -0
  46. package/dist/curation/source-refs.js.map +1 -0
  47. package/dist/curation/tension-blast.d.ts.map +1 -1
  48. package/dist/curation/tension-blast.js +7 -6
  49. package/dist/curation/tension-blast.js.map +1 -1
  50. package/dist/curation/tier0.d.ts +0 -1
  51. package/dist/curation/tier0.d.ts.map +1 -1
  52. package/dist/curation/tier0.js +24 -23
  53. package/dist/curation/tier0.js.map +1 -1
  54. package/dist/curation/vault-docs.d.ts +3 -3
  55. package/dist/curation/vault-docs.d.ts.map +1 -1
  56. package/dist/curation/vault-docs.js +12 -5
  57. package/dist/curation/vault-docs.js.map +1 -1
  58. package/dist/distill/extract.d.ts.map +1 -1
  59. package/dist/distill/extract.js +8 -1
  60. package/dist/distill/extract.js.map +1 -1
  61. package/dist/distill/propose.d.ts.map +1 -1
  62. package/dist/distill/propose.js +2 -7
  63. package/dist/distill/propose.js.map +1 -1
  64. package/dist/distill/state.d.ts +9 -1
  65. package/dist/distill/state.d.ts.map +1 -1
  66. package/dist/distill/state.js +58 -7
  67. package/dist/distill/state.js.map +1 -1
  68. package/dist/eval/llm-openrouter.d.ts.map +1 -1
  69. package/dist/eval/llm-openrouter.js +5 -22
  70. package/dist/eval/llm-openrouter.js.map +1 -1
  71. package/dist/eval/subgraph.d.ts.map +1 -1
  72. package/dist/eval/subgraph.js +19 -15
  73. package/dist/eval/subgraph.js.map +1 -1
  74. package/dist/frontmatter/source-ref.d.ts +24 -0
  75. package/dist/frontmatter/source-ref.d.ts.map +1 -0
  76. package/dist/frontmatter/source-ref.js +20 -0
  77. package/dist/frontmatter/source-ref.js.map +1 -0
  78. package/dist/index.d.ts +1 -1
  79. package/dist/index.d.ts.map +1 -1
  80. package/dist/index.js +47 -14
  81. package/dist/index.js.map +1 -1
  82. package/dist/integrations/distill.d.ts +16 -0
  83. package/dist/integrations/distill.d.ts.map +1 -0
  84. package/dist/integrations/distill.js +164 -0
  85. package/dist/integrations/distill.js.map +1 -0
  86. package/dist/integrations/engine.d.ts +134 -0
  87. package/dist/integrations/engine.d.ts.map +1 -0
  88. package/dist/integrations/engine.js +571 -0
  89. package/dist/integrations/engine.js.map +1 -0
  90. package/dist/integrations/google.d.ts +13 -0
  91. package/dist/integrations/google.d.ts.map +1 -0
  92. package/dist/integrations/google.js +551 -0
  93. package/dist/integrations/google.js.map +1 -0
  94. package/dist/integrations/notion.d.ts +16 -0
  95. package/dist/integrations/notion.d.ts.map +1 -0
  96. package/dist/integrations/notion.js +485 -0
  97. package/dist/integrations/notion.js.map +1 -0
  98. package/dist/integrations/oauth.d.ts +14 -0
  99. package/dist/integrations/oauth.d.ts.map +1 -0
  100. package/dist/integrations/oauth.js +141 -0
  101. package/dist/integrations/oauth.js.map +1 -0
  102. package/dist/integrations/queue.d.ts +30 -0
  103. package/dist/integrations/queue.d.ts.map +1 -0
  104. package/dist/integrations/queue.js +255 -0
  105. package/dist/integrations/queue.js.map +1 -0
  106. package/dist/integrations/review.d.ts +5 -0
  107. package/dist/integrations/review.d.ts.map +1 -0
  108. package/dist/integrations/review.js +36 -0
  109. package/dist/integrations/review.js.map +1 -0
  110. package/dist/integrations/routes.d.ts +26 -0
  111. package/dist/integrations/routes.d.ts.map +1 -0
  112. package/dist/integrations/routes.js +258 -0
  113. package/dist/integrations/routes.js.map +1 -0
  114. package/dist/integrations/runtime.d.ts +33 -0
  115. package/dist/integrations/runtime.d.ts.map +1 -0
  116. package/dist/integrations/runtime.js +280 -0
  117. package/dist/integrations/runtime.js.map +1 -0
  118. package/dist/integrations/state.d.ts +10 -0
  119. package/dist/integrations/state.d.ts.map +1 -0
  120. package/dist/integrations/state.js +238 -0
  121. package/dist/integrations/state.js.map +1 -0
  122. package/dist/integrations/types.d.ts +45 -0
  123. package/dist/integrations/types.d.ts.map +1 -0
  124. package/dist/integrations/types.js +4 -0
  125. package/dist/integrations/types.js.map +1 -0
  126. package/dist/lifecycle/lock.d.ts.map +1 -1
  127. package/dist/lifecycle/lock.js +15 -1
  128. package/dist/lifecycle/lock.js.map +1 -1
  129. package/dist/schema/diff.d.ts +44 -0
  130. package/dist/schema/diff.d.ts.map +1 -0
  131. package/dist/schema/diff.js +143 -0
  132. package/dist/schema/diff.js.map +1 -0
  133. package/dist/schema/index.d.ts +2 -0
  134. package/dist/schema/index.d.ts.map +1 -0
  135. package/dist/schema/index.js +164 -0
  136. package/dist/schema/index.js.map +1 -0
  137. package/dist/schema/infer.d.ts +23 -0
  138. package/dist/schema/infer.d.ts.map +1 -0
  139. package/dist/schema/infer.js +167 -0
  140. package/dist/schema/infer.js.map +1 -0
  141. package/dist/schema/render.d.ts +12 -0
  142. package/dist/schema/render.d.ts.map +1 -0
  143. package/dist/schema/render.js +103 -0
  144. package/dist/schema/render.js.map +1 -0
  145. package/dist/schema/scan.d.ts +14 -0
  146. package/dist/schema/scan.d.ts.map +1 -0
  147. package/dist/schema/scan.js +92 -0
  148. package/dist/schema/scan.js.map +1 -0
  149. package/dist/search/coverage.d.ts.map +1 -1
  150. package/dist/search/coverage.js +5 -5
  151. package/dist/search/coverage.js.map +1 -1
  152. package/dist/search/graph-expansion.d.ts +55 -0
  153. package/dist/search/graph-expansion.d.ts.map +1 -0
  154. package/dist/search/graph-expansion.js +160 -0
  155. package/dist/search/graph-expansion.js.map +1 -0
  156. package/dist/search/hybrid.d.ts +4 -0
  157. package/dist/search/hybrid.d.ts.map +1 -1
  158. package/dist/search/hybrid.js.map +1 -1
  159. package/dist/serve/index.d.ts +5 -0
  160. package/dist/serve/index.d.ts.map +1 -1
  161. package/dist/serve/index.js +251 -20
  162. package/dist/serve/index.js.map +1 -1
  163. package/dist/serve/limits.d.ts +13 -0
  164. package/dist/serve/limits.d.ts.map +1 -1
  165. package/dist/serve/limits.js +36 -0
  166. package/dist/serve/limits.js.map +1 -1
  167. package/dist/sleep/cycle.d.ts +2 -0
  168. package/dist/sleep/cycle.d.ts.map +1 -1
  169. package/dist/sleep/cycle.js +95 -16
  170. package/dist/sleep/cycle.js.map +1 -1
  171. package/dist/sleep/index.d.ts.map +1 -1
  172. package/dist/sleep/index.js +8 -7
  173. package/dist/sleep/index.js.map +1 -1
  174. package/dist/sleep/report.d.ts.map +1 -1
  175. package/dist/sleep/report.js +16 -9
  176. package/dist/sleep/report.js.map +1 -1
  177. package/dist/sleep/run-ledger.d.ts.map +1 -1
  178. package/dist/sleep/run-ledger.js +1 -0
  179. package/dist/sleep/run-ledger.js.map +1 -1
  180. package/dist/test-utils/require-defined.d.ts +2 -0
  181. package/dist/test-utils/require-defined.d.ts.map +1 -0
  182. package/dist/test-utils/require-defined.js +7 -0
  183. package/dist/test-utils/require-defined.js.map +1 -0
  184. package/dist/test-utils/require-defined.test.d.ts +2 -0
  185. package/dist/test-utils/require-defined.test.d.ts.map +1 -0
  186. package/dist/test-utils/require-defined.test.js +13 -0
  187. package/dist/test-utils/require-defined.test.js.map +1 -0
  188. package/dist/tools/board.test.js +21 -17
  189. package/dist/tools/board.test.js.map +1 -1
  190. package/dist/tools/curation.d.ts +2 -0
  191. package/dist/tools/curation.d.ts.map +1 -1
  192. package/dist/tools/curation.js +32 -3
  193. package/dist/tools/curation.js.map +1 -1
  194. package/dist/tools/edge-staleness.d.ts +2 -0
  195. package/dist/tools/edge-staleness.d.ts.map +1 -1
  196. package/dist/tools/edge-staleness.js +46 -4
  197. package/dist/tools/edge-staleness.js.map +1 -1
  198. package/dist/tools/erase.d.ts +24 -0
  199. package/dist/tools/erase.d.ts.map +1 -1
  200. package/dist/tools/erase.js +125 -20
  201. package/dist/tools/erase.js.map +1 -1
  202. package/dist/tools/read.d.ts +2 -0
  203. package/dist/tools/read.d.ts.map +1 -1
  204. package/dist/tools/read.js +18 -0
  205. package/dist/tools/read.js.map +1 -1
  206. package/dist/tools/search.d.ts.map +1 -1
  207. package/dist/tools/search.js +31 -3
  208. package/dist/tools/search.js.map +1 -1
  209. package/dist/tools/write.d.ts.map +1 -1
  210. package/dist/tools/write.js +44 -4
  211. package/dist/tools/write.js.map +1 -1
  212. package/dist/utils/config.d.ts +17 -0
  213. package/dist/utils/config.d.ts.map +1 -1
  214. package/dist/utils/config.js +189 -4
  215. package/dist/utils/config.js.map +1 -1
  216. package/dist/utils/git-erase.d.ts +2 -0
  217. package/dist/utils/git-erase.d.ts.map +1 -1
  218. package/dist/utils/git-erase.js +3 -0
  219. package/dist/utils/git-erase.js.map +1 -1
  220. package/dist/utils/git.d.ts +1 -0
  221. package/dist/utils/git.d.ts.map +1 -1
  222. package/dist/utils/git.js +28 -0
  223. package/dist/utils/git.js.map +1 -1
  224. package/dist/utils/vault-gitignore.d.ts +1 -1
  225. package/dist/utils/vault-gitignore.d.ts.map +1 -1
  226. package/dist/utils/vault-gitignore.js +23 -3
  227. package/dist/utils/vault-gitignore.js.map +1 -1
  228. package/package.json +3 -2
  229. package/templates/config.yaml +3 -0
package/README.md CHANGED
@@ -2,157 +2,182 @@
2
2
 
3
3
  [![CI](https://github.com/mavaali/daftari/actions/workflows/ci.yml/badge.svg)](https://github.com/mavaali/daftari/actions/workflows/ci.yml) [![npm version](https://img.shields.io/npm/v/daftari.svg)](https://www.npmjs.com/package/daftari) [![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
4
4
 
5
- *Daftari* (دفتری) is the Urdu word for a ledger-keeper: the person in a
6
- trading house who maintained the *daftar*, the bound register where every
7
- transaction was recorded, cross-referenced, and preserved. The daftar was not a
8
- filing cabinet. It was a living document. Entries referenced earlier entries.
9
- Corrections were noted, not erased. The ledger got more valuable the longer it
10
- was kept, because the accumulated record revealed patterns no single entry
11
- could.
5
+ **Durable, inspectable memory for AI agents.** Daftari exposes a markdown vault
6
+ over the Model Context Protocol (MCP), then adds the controls an agent needs to
7
+ use that vault without flattening its history: provenance, supersession,
8
+ staleness, open contradictions, access control, and Git-backed writes.
12
9
 
13
- Daftari is the **long-term memory cortex for your LLM agents** — a persistent,
14
- structured vault they read, write, and curate over time, **portable across any
15
- model.** A cortex, not a clipboard.
10
+ The model is replaceable. The memory stays yours: plain files on disk, readable
11
+ without Daftari and portable across MCP clients.
16
12
 
17
- ## Contents
13
+ *Daftari* (دفتری) is Urdu for a ledger-keeper. The model is deliberate: a
14
+ ledger records corrections instead of erasing them, and becomes more useful as
15
+ its cross-referenced history accumulates.
18
16
 
19
- **Why Daftari** — [Rent the brain, own the memory](#rent-the-brain-own-the-memory) · [Not a second brain](#not-a-second-brain) · [It remembers, it doesn't resolve](#it-remembers--it-doesnt-resolve-for-you) · [How it compares](#how-it-compares)
20
-
21
- **Concepts** — [What it is](#what-it-is) · [The four layers](#the-four-layers) · [Two kinds of knowledge](#two-kinds-of-knowledge) · [File format](#file-format)
22
-
23
- **The rituals** — [The tools](#the-tools) · [Circadian memory](#circadian-memory) · [The viewer](#the-viewer) · [Tension Court](#tension-court) · [The principal interview](#the-principal-interview) · [Belief archaeology](#belief-archaeology) · [The vault as witness](#the-vault-as-witness--and-the-wager-layer) · [Coherence audit](#coherence-audit)
24
-
25
- **Running it** — [Access control](#access-control) · [Cross-vault federation](#cross-vault-federation) · [Server mode](#server-mode-self-hosted) · [Storage backing](#storage-backing) · [Adopting a vault + OKF](#adopting-an-existing-vault)
26
-
27
- **Reference** — [What's not in v1](#whats-not-in-v1) · [Development](#development) · [Documentation](#documentation) · [Integrations](#integrations) · [Privacy](#privacy) · [License](#license)
28
-
29
- <!-- Why Daftari -->
30
-
31
- ## Rent the brain, own the memory
32
-
33
- LLMs are stateless; they forget. So memory is being bolted on everywhere — but
34
- *inside the model*: ChatGPT memory, Claude projects, Copilot, Gemini. Memory is
35
- becoming a feature of the provider, and whoever holds your memory holds you.
36
-
37
- Daftari takes the other path. The model is the rented part — swappable, obsolete
38
- in six months. Your memory is the durable asset, so it should belong to you and
39
- travel with you: plain markdown on your disk, under git, readable in any editor,
40
- queryable by any agent. **Compilation over retrieval** — the agent compiles an
41
- answer once and writes it back, so every later read starts from the compiled
42
- result instead of re-stitching chunks from zero.
43
-
44
- ## Not a second brain
45
-
46
- Second brains are memory for a *human* to think with — you catch the stale fact,
47
- you notice the contradiction. Daftari is memory for an *agent* to reason with:
48
- the persistence layer for a consumer that acts on what it is handed and cannot
49
- sanity-check it first. Same substrate (markdown, links); reversed purpose — and
50
- a higher bar, because the reliability has to live in the memory, not the reader.
51
-
52
- ## It remembers — it doesn't resolve for you
17
+ ```bash
18
+ npx daftari --init ./my-vault
19
+ ```
53
20
 
54
- Because the agent can't infer them, the vault carries three things and collapses
55
- none:
21
+ ## Choose your path
56
22
 
57
- - **what's current** — supersession follows a real edge to the latest source
58
- - **what's grounded** — provenance on every entry; the vault never mints a value
59
- - **what's contested** — contradictions surface as *tensions*, held open, not
60
- flattened into a false answer
23
+ | I want to… | Start here |
24
+ |---|---|
25
+ | Create a vault and connect an MCP client | [Five-minute quickstart](#five-minute-quickstart) |
26
+ | Adopt an Obsidian vault or existing markdown wiki | [Adopt existing notes](docs/adoption.md) |
27
+ | See how knowledge compounds instead of being repeatedly retrieved | [Worked example](docs/worked-example.md) |
28
+ | Run the curation loop | [Curation workflow](docs/curation-workflow.md) |
29
+ | Review tensions, stale beliefs, or historical state | [Operator workflows](docs/operator-workflows.md) |
30
+ | Configure access, HTTP serving, federation, or storage | [Deployment and access](docs/deployment.md) |
31
+ | Continuously distill Google Docs or Notion | [Source integrations](docs/integrations.md) |
32
+ | Understand the design and its boundaries | [Architecture](docs/architecture.md) |
33
+ | Look up frontmatter fields | [File format](docs/file-format.md) |
34
+ | Find the rest of the documentation | [Documentation map](docs/README.md) |
61
35
 
62
- The daftar noted corrections rather than erasing them. Daftari keeps that as a
63
- law:
36
+ ## Five-minute quickstart
64
37
 
65
- > **A tension may never masquerade as a supersession.**
38
+ **Prerequisite:** Node.js 20 or newer.
66
39
 
67
- It resolves only by discovery — a real edge — never by invention. The agent
68
- compiles; the vault preserves; *you* keep the judgment. See
69
- [the manifesto](docs/manifesto.md) for the full argument.
40
+ ### 1. Create a vault
70
41
 
71
- ```mermaid
72
- flowchart TD
73
- A[Two docs disagree] --> B{Is there a real edge?<br/>source / supersession link}
74
- B -- yes --> C[Supersede: one current,<br/>lineage preserved]
75
- B -- no --> D[Tension: both held open, live]
42
+ ```bash
43
+ npx daftari --init ./my-vault
76
44
  ```
77
45
 
78
- ## How it compares
79
-
80
- The real question a memory layer has to answer: **what happens when two facts
81
- contradict?**
46
+ Daftari creates a Git-backed vault with a config file, four starter
47
+ collections, and three fictional example documents. The markdown is the source
48
+ of truth; `.daftari/index.db` is a rebuildable search index.
82
49
 
83
- | What happens to a contradiction? | Systems |
84
- |---|---|
85
- | Invisible | AGENTS.md, RAG |
86
- | Synthesis-overwrite (association only) | Mem0 (OSS), ChatGPT / Claude memory, Glean |
87
- | New wins, old tagged-invalid | Zep / Graphiti, Sentra |
88
- | Resolved by graph / majority-vote | Cognee, ElephantBroker |
89
- | **Held open, live & queryable** | **Daftari** |
90
-
91
- And the honest cost, since compounding isn't free:
50
+ ### 2. Connect an MCP client
92
51
 
93
- | Who has to do the curation work? | |
94
- |---|---|
95
- | RAG | Nobody — retrieval only, zero authoring cost |
96
- | AGENTS.md | One file, hand-maintained |
97
- | **Daftari** | **An agent (or human) curates — the heaviest of the three, and the reason knowledge compounds** |
52
+ Add Daftari to your MCP client configuration. For Claude Desktop on macOS,
53
+ edit `~/Library/Application Support/Claude/claude_desktop_config.json`:
98
54
 
99
- > The nearest formal prior art is **TOKI** ([arXiv 2606.06240](https://arxiv.org/abs/2606.06240),
100
- > with a [reference implementation](https://github.com/ZenAlexa/toki-bitemporal-memory)):
101
- > a bitemporal operator algebra whose opt-in `await-confirmation` operator can hold a
102
- > contradiction open. The line: TOKI resolves at write time (hold-open is one of four
103
- > operators) and keeps the loser in an *archival* audit row; Daftari makes non-resolution
104
- > the *default* and keeps both facts *live and queryable* in retrieval.
55
+ ```json
56
+ {
57
+ "mcpServers": {
58
+ "daftari": {
59
+ "command": "npx",
60
+ "args": [
61
+ "-y",
62
+ "daftari@latest",
63
+ "--vault",
64
+ "/absolute/path/to/my-vault",
65
+ "--user",
66
+ "me",
67
+ "--role",
68
+ "admin"
69
+ ]
70
+ }
71
+ }
72
+ }
73
+ ```
105
74
 
106
- <!-- Concepts -->
75
+ Use an absolute vault path, then restart the client. The scaffolded config
76
+ includes an `admin` role. Omitting `--role`, or naming a role that does not
77
+ exist, starts Daftari as a deny-all guest.
107
78
 
108
- ## What it is
79
+ ### 3. Ask the vault a question
109
80
 
110
- A directory of markdown files with YAML frontmatter, exposed to agents as 38
111
- MCP tools over stdio. The vault is plain text: you can read it in any editor,
112
- `git log` it, grep it. Daftari adds the machinery agents need to treat it as a
113
- shared workspace.
81
+ Try a request that forces the agent to search before answering:
114
82
 
115
- ```
116
- npx daftari --init ./my-vault
117
- npx daftari --vault ./my-vault --user me --role admin
118
- ```
83
+ > Search my Daftari vault for the current Helios pricing model. Cite the source
84
+ > document and tell me whether it is stale or contested.
119
85
 
120
- Point any MCP client (Claude Desktop, Claude Code, an agent SDK) at it.
86
+ The agent should search the example vault, read the matching markdown, and
87
+ report the document's standing rather than returning an unqualified snippet.
121
88
 
122
- ## The four layers
89
+ Next, ask it to write a draft:
123
90
 
124
- All four implement one idea — *resolve only by discovery, never by invention.*
125
- Read [**The core idea**](docs/architecture.md#the-core-idea) first and the layers
126
- become its consequences. Storage and access control are table stakes; the moat is
127
- layers 3 and 4.
91
+ > Create a low-confidence draft that compares the Helios and Aurora examples.
92
+ > Preserve the source links and list the questions the draft cannot answer.
128
93
 
129
- |Layer |What it does |Why it matters |
130
- |---------------------|------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------|
131
- |**Storage** |Markdown + frontmatter on disk, git history, rebuildable SQLite index for hybrid BM25 + vector search.|Plain text is the source of truth. Delete every `.db` file and rebuild.|
132
- |**Access control** |Config-driven RBAC. Roles and per-collection read/write/promote permissions in `.daftari/config.yaml`.|Multiple agents, scoped access, no user-management system. |
133
- |**Write arbitration**|File-level locks (60s TTL), auto-commit to git, structured provenance log. |Concurrent agents write safely. Every mutation is attributable. |
134
- |**Curation** |Draft-to-canonical lifecycle, TTL-based staleness, tension logging, advisory linter. |Knowledge that stops being true gets surfaced, not silently trusted. |
94
+ Every mutation is written to markdown, recorded in provenance, indexed, and
95
+ committed to Git. Continue through promotion, retirement, and tension handling
96
+ in the [full getting-started walkthrough](docs/getting-started.md).
135
97
 
136
- ## Two kinds of knowledge
98
+ ## The problem Daftari solves
137
99
 
138
- Every document declares a `domain`. The distinction drives how the curation
139
- layer treats it.
100
+ Agents do not merely need stored text. They need to know what they can trust
101
+ when the text changes, conflicts, or loses its grounding.
140
102
 
141
- **Accumulation** documents compile and compound. A competitive-intel note, a
142
- pricing breakdown, a researched comparison. Each write builds on the last.
143
- Going stale is a problem to fix.
103
+ Daftari keeps three judgments separate:
144
104
 
145
- **Generative** documents speculate. A moonshot sketch, a brainstorm, a “what
146
- if.” Going stale is expected, not a defect.
105
+ - **What is current.** A supersession follows an explicit edge to a successor;
106
+ recency alone does not make a claim true.
107
+ - **What is grounded.** Sources and provenance remain attached to the document;
108
+ the vault does not manufacture evidence.
109
+ - **What is contested.** When live claims disagree and neither replaces the
110
+ other, the contradiction remains visible as a tension.
147
111
 
148
- Applied uniformly, the same curation rules would either nag about every
149
- brainstorm or wave through every stale fact. The domain split lets the system
150
- hold each to the right standard.
112
+ That gives the system one governing rule:
151
113
 
152
- ## File format
114
+ > **A tension may never masquerade as a supersession.**
153
115
 
154
- Markdown with YAML frontmatter. Frontmatter is the metadata layer; there is no
155
- separate database.
116
+ ```mermaid
117
+ flowchart TD
118
+ A[Two live claims disagree] --> B{Does one explicitly<br/>supersede the other?}
119
+ B -- yes --> C[Follow the successor<br/>preserve the lineage]
120
+ B -- no --> D[Keep both live<br/>surface an open tension]
121
+ ```
122
+
123
+ The longer argument—including why memory should outlive the model renting
124
+ it—lives in the [manifesto](docs/manifesto.md). The implementation boundaries
125
+ live in the [architecture guide](docs/architecture.md).
126
+
127
+ ## What using Daftari looks like
128
+
129
+ The basic loop is small:
130
+
131
+ 1. **Search before writing.** Find the current documents, their sources, and
132
+ any unresolved tensions.
133
+ 2. **Write a draft.** State confidence, provenance, sources, and open questions
134
+ in YAML frontmatter.
135
+ 3. **Curate deliberately.** Lint reports staleness, weak grounding, abandoned
136
+ drafts, unanswered questions, and broken relationships. It does not fix
137
+ them.
138
+ 4. **Ratify or retire.** Promote trustworthy drafts, supersede replaced
139
+ knowledge, and record why a claim changed.
140
+ 5. **Revisit.** Sleep runs, interviews, court dockets, and archaeology reports
141
+ turn accumulated history into a review queue.
142
+
143
+ This is **compilation over retrieval**: an agent writes a considered result
144
+ back into the vault so the next agent begins with the accumulated record, not
145
+ with the same pile of fragments. See the [worked example](docs/worked-example.md)
146
+ for the document lifecycle across three writes.
147
+
148
+ ## Capabilities by outcome
149
+
150
+ Configure the MCP registry in `core`, `standard`, or `full` tiers. The tier
151
+ changes what clients see in `tools/list`; it does not change the vault's data
152
+ model.
153
+
154
+ | Outcome | Main surfaces | What they provide |
155
+ |---|---|---|
156
+ | Find relevant knowledge | `vault_search`, `vault_search_related`, `vault_themes` | Hybrid lexical/vector retrieval, related documents, and thematic clusters |
157
+ | Read with context | `vault_read`, `vault_backlinks`, `vault_consumes` | Document content, inbound references, and compiled dependencies |
158
+ | Write and maintain documents | `vault_write`, `vault_append`, `vault_merge`, `vault_supersede` | Structured writes with locking, provenance, indexing, and Git history |
159
+ | Control lifecycle and confidence | `vault_promote`, `vault_deprecate`, `vault_set_confidence`, `vault_set_tier` | Explicit gates between draft, canonical, source, and retired knowledge |
160
+ | Keep contradictions visible | `vault_tension_log`, `vault_tension_triage`, `vault_positions`, `vault_canon` | Open tensions, attributed positions, and settled-versus-contested belief |
161
+ | Require human judgment | `vault_stage_action`, `vault_ratify`, `vault_consolidate` | Proposed actions and ratified organizational positions |
162
+ | Inspect trust and history | `vault_receipt`, `vault_provenance`, `vault_witness`, `daftari asof` | Evidence receipts, write history, principal track records, and past belief state |
163
+ | Operate the vault | `daftari sleep`, `court`, `interview`, `view`, `audit` | Review queues, rulings, elicited evidence, a read-only portal, and coherence checks |
164
+
165
+ Use `daftari --help` for the current CLI surface. MCP clients obtain the current
166
+ tool names and schemas directly from `tools/list`; the README does not duplicate
167
+ the complete registry.
168
+
169
+ ## How the vault is built
170
+
171
+ | Layer | Responsibility | Boundary |
172
+ |---|---|---|
173
+ | **Storage** | Markdown with YAML frontmatter, Git history, SQLite search index | Markdown is canonical; the index is disposable |
174
+ | **Access** | Config-driven roles and collection permissions | No separate user-management database |
175
+ | **Write safety** | Process lock, file locks, attributable writes, automatic commits | Concurrency protection does not resolve semantic conflicts |
176
+ | **Curation** | Lifecycle, staleness, tensions, provenance, staged actions | Advisory by default; judgment is never silently automated |
177
+
178
+ Every document remains readable in an editor and inspectable with ordinary Git
179
+ commands. Frontmatter is the metadata layer; Daftari does not introduce a
180
+ second document format. A typical accumulation document begins like this:
156
181
 
157
182
  ```yaml
158
183
  ---
@@ -166,792 +191,113 @@ updated: 2026-05-17
166
191
  updated_by: agent:claude-code
167
192
  provenance: synthesized
168
193
  sources:
169
- - aurora-product-page
194
+ - https://example.com/aurora-product-page
170
195
  ttl_days: 120
171
- valid_from: 2026-05-01
172
- valid_until: null
173
196
  tags: [aurora, ingestion, competitive]
174
197
  questions_answered:
175
- - "How does Aurora frame the ingestion/transformation boundary?"
198
+ - "How does Aurora frame the ingestion boundary?"
176
199
  questions_raised:
177
- - "Does an authored-pipeline model slow teams down at small scale?"
200
+ - "Does an authored pipeline slow small teams down?"
178
201
  ---
179
202
  ```
180
203
 
181
- Documents can make their epistemic edges explicit: `questions_answered` is what
182
- later agents can take as settled, `questions_raised` is where to build next.
183
- `vault_lint` turns the open questions across the vault into a coverage map.
184
-
185
- `valid_from` / `valid_until` are the second temporal axis. Everything else in
186
- the file records **transaction time** — when the vault came to believe
187
- something; these record **valid time** — when the claim was true in the world. A
188
- document edited this morning can describe a price that stopped applying in
189
- March, and only the second axis can tell you so. That makes supersession
190
- checkable rather than merely asserted:
191
-
192
- ```bash
193
- daftari asof 2026-04-01 --valid 2026-01-15
194
- ```
195
-
196
- "On April 1st, what did the vault believe was true on January 15th?"
197
-
198
- <details>
199
- <summary><b>Half-open valid-time intervals and the authoring rules (▸ expand for full reference)</b></summary>
200
-
201
- The window is half-open — `[valid_from, valid_until)` — so `valid_until` is the
202
- first day the claim no longer held, and a successor's `valid_from` is exactly
203
- its predecessor's `valid_until`. Handoffs share no day and leave no gap. The
204
- fields are optional and always authored: nothing derives them from git dates or
205
- mtime, because that would manufacture a claim nobody made. Both null means
206
- unknown, which is never read as "always true".
207
-
208
- Full field reference in <docs/file-format.md>.
209
-
210
- </details>
211
-
212
- <!-- The rituals -->
213
-
214
- ## The tools
215
-
216
- **Read:** `vault_read`, `vault_index`, `vault_status`
217
-
218
- **Attest:** `vault_receipt` — compile an epistemic receipt for the documents
219
- an answer cites: per-source status, confidence, provenance, freshness, exact
220
- content-version hash, supersession-chain resolution, and open tensions, plus
221
- deterministic summary flags, the vault's git HEAD as an as-of anchor, and a
222
- recomputable hash over the whole receipt. Attach it to the answer so any
223
- consumer can see what the answer stands on.
224
-
225
- **Believe:** `vault_canon` — compute settled vs. contested belief across
226
- holders over an emergent topic (params: `seed`, `holders?`, `as_of?`,
227
- `depth?`). Returns a per-claim settled/contested classification, integrity
228
- flags (`graph_completeness: "curated"`, `partial_visibility`, `unindexed`,
229
- `ghost_holder_warning`), and an attached `vault_receipt` as the epistemic
230
- anchor. Recorded, not omniscient: settled means no contradiction is on record,
231
- not that none exists. Never auto-resolves.
232
-
233
- **Witness:** `vault_witness` — per-principal track records from the vault's
234
- own ledgers, priced by the wager schedule: write volume, live claims with
235
- open exposure, contested claims with stake at risk, the settled book
236
- (burned vs credited), proposal outcomes, tensions logged. Advisory and
237
- deterministic; includes the flat-curve monitor so a single-author vault is
238
- reported as uninformative rather than as signal.
239
-
240
- <details>
241
- <summary><b>The full per-category tool registry and the tool-tier config (▸ expand for full reference)</b></summary>
242
-
243
- **Search:** `vault_search` (hybrid BM25 + vector; hits carry their unresolved
244
- tensions inline and foreground the current source of superseded documents),
245
- `vault_search_related`, `vault_themes` (thematic clustering), `vault_reindex`
246
-
247
- **Write:** `vault_write`, `vault_append`, `vault_promote`, `vault_deprecate`, `vault_supersede`, `vault_merge`, `vault_set_confidence`, `vault_set_tier`
248
-
249
- **Curate:** `vault_tension_log`, `vault_tension_resolve`, `vault_tension_clusters`, `vault_tension_blast`, `vault_tension_triage` (the unranked triage card / `court --triage`), `vault_lint`, `vault_provenance`, `vault_staleness` (edge-staleness: pending upstream changes / broken-read rate)
250
-
251
- **Edges:** `vault_edge_observe`, `vault_edge_contest`, `vault_edges`, `vault_consumes` (query the compiled dependency graph), `vault_backlinks` (the reverse knowledge-graph query — which documents reference a target; a code-path target lists the docs whose `describes` binds that file, and `verify:true` classifies each pin against its code repo)
252
-
253
- **Dispatch:** `vault_tier1` (type-directed change dispatch), `vault_tier2_queue`, `vault_tier2_verdict` (semantic-review queue + verdict)
254
-
255
- **Ratify:** `vault_stage_action`, `vault_ratify`
256
-
257
- **Positions:** `vault_assert` (a principal's assert/dispute/qualify stance on a claim doc; conflicting live stances mark it contested and cap confidence), `vault_positions` (query positions by doc or principal), `vault_consolidate` (ratify-gated org stance; clears the contested confidence cap, dissent computed server-side)
258
-
259
- The curation engine is advisory: `vault_lint` reports problems and
260
- `vault_tension_log` records contradictions. Neither auto-fixes anything. Every
261
- change is a deliberate, attributable act. Retraction is surfaced, not enforced:
262
- the `retiredStillLinked` lint check / `retired_still_linked` read-time advisory
263
- flags canonical docs still citing a deprecated or superseded source, and
264
- `vault_deprecate`/`vault_supersede` return a `dependents` advisory (the
265
- downstream blast) at retraction time.
266
-
267
- ### Tool tiers
268
-
269
- Every advertised tool costs the agent context tokens and a decision branch.
270
- The `tools` block in `.daftari/config.yaml` picks how much of the registry
271
- `tools/list` advertises:
272
-
273
- ```yaml
274
- tools:
275
- tier: standard # core | standard | full (default: full)
276
- include: # add tools the tier omits
277
- - vault_tension_log
278
- exclude: # remove tools the tier includes — exclude wins
279
- - vault_status
280
- ```
281
-
282
- `core` is the search-before-derive loop (`vault_search`, `vault_read`,
283
- `vault_write`, `vault_index`, `vault_lint`, `vault_status`); `standard` adds
284
- the full document lifecycle (append/promote/deprecate/supersede/merge,
285
- confidence and tier setters, propose/ratify) plus `vault_search_related`,
286
- `vault_provenance`, and diagnostics; `full` is everything. `--tools <tier>` overrides the tier for one invocation. Filtering
287
- only changes what is *advertised* — calls to any registered tool still work,
288
- so an agent holding a cached tool name keeps working across a tier change.
289
- Unknown names in `include`/`exclude` warn at startup and are ignored, so a
290
- config written for a newer daftari still loads.
291
-
292
- </details>
293
-
294
- **Evaluate (opt-in, requires an Anthropic API key):** `daftari eval` — scores how
295
- well an LLM can use the curation surface to answer multi-hop questions about the
296
- vault. See the [design spec](docs/superpowers/specs/2026-05-31-cortex-quality-metric-design.md)
297
- for the rationale and the cortex framing.
298
-
299
- ## Circadian memory
300
-
301
- The vault sleeps. `daftari sleep` is the nightly metabolic pass — deterministic,
302
- LLM-free, write-free (documents are never touched):
303
-
304
- ```bash
305
- # In cron (any scheduler works; daftari ships the cycle, not a daemon):
306
- 0 3 * * * cd /path/to/vault && npx daftari sleep --output .daftari/morning-report.md
307
- ```
308
-
309
- The cycle sweeps expired staged actions, scores every document's decay, and
310
- builds the **wake list**: canonical accumulation documents past their TTL
311
- with downstream dependents, ranked by blast radius. The list is written to
312
- `.daftari/wake-queue.jsonl` for an external agent to consume — re-verify each
313
- document against its sources, stage the diff — because the vault never
314
- re-verifies on its own. Generative documents going stale are expected, not a
315
- defect: counted, never woken.
316
-
317
- The **Morning Report** ends where the human begins: tension aging and the
318
- court docket head, the ratification queue with soon-to-expire proposals, and
319
- the rubber-stamp monitor — zero rejections over a long decision history is
320
- printed as a warning, not a compliment. The agent proposes overnight; you
321
- ratify over coffee.
322
-
323
- Each completed pass appends a content-light summary (counts only, no document
324
- bodies) to a self-pruning `.daftari/runs.jsonl`; `daftari runs [list|show <id>]`
325
- reads that history back, so the loop's own liveness is legible over time.
326
-
327
- ## The viewer
328
-
329
- `daftari view` serves a read-only web portal over the vault — loopback only,
330
- no editing — that makes the vault's *epistemic state and knowledge graph*
331
- legible, not just its prose:
332
-
333
- - **Dashboard home** — vault metrics, a freshness distribution (fresh/aging/
334
- stale), open-tension and ratification-queue counts, and the recent sleep-run
335
- trend.
336
- - **Document pages** lead with a **standing strip** — status, a confidence
337
- meter, decay, and a contested flag — over rendered (sanitized) markdown, with
338
- epistemic banners (decay / structural / upstream / pins), a **Contested**
339
- panel, **backlinks**, a table of contents, and in-vault links resolved to
340
- their documents.
341
- - **Knowledge graph** (`/graph`) — an interactive map of the vault: nodes are
342
- documents (colored by status, flagged when decayed or contested), edges are
343
- source / link / derives-from / contested relations. Click a node to open it;
344
- every document links to its own neighborhood.
345
- - **Search** over the hybrid index, on every page.
346
-
347
- Read-only by construction (only GET, no mutation routes, a loopback Host
348
- allow-list against DNS-rebinding), built on `node:http` with one lazily loaded
349
- graph library — no web framework.
350
-
351
- ```bash
352
- daftari view --vault ./my-vault # then open http://127.0.0.1:8788
353
- ```
354
-
355
- ## Tension Court
356
-
357
- Tensions wait in a log; the court turns them into decidable cases. `daftari
358
- court` compiles a **docket** — every open tension briefed and ranked (stale
359
- first, then by blast radius): both sides' claims, the present state of their
360
- documents, the downstream stakes, cluster membership, and **precedents** —
361
- past rulings on disputes that shared a document, a collection pair, or a
362
- kind.
363
-
364
- The **triage card** (`daftari court --triage` / `vault_tension_triage`) is the
365
- deliberately **unranked**, cluster-grouped companion view — legibility, not a
366
- severity score. Each contested side surfaces `criticality`, `provenance`, and
367
- `updated_by`, so a resolver can judge trust without opening the documents.
368
-
369
- ```bash
370
- # The docket — a 5-minute weekly ritual
371
- daftari court --vault ./my-vault
372
-
373
- # One case's full brief (verbatim rationales from cited precedents)
374
- daftari court --tension tension-abc123
375
-
376
- # Rule. The rationale is recorded verbatim and cited by future dockets.
377
- daftari court rule tension-abc123 --kind corrected \
378
- --rationale "Vendor pricing page confirmed the entry tier on 2026-07-10."
379
- ```
380
-
381
- Rulings go through the same `resolveTension` write path as
382
- `vault_tension_resolve` — a ruling records the closure, it never edits the
383
- disputed documents. Precedent retrieval is deterministic (shared-document >
384
- collection-pair > same-kind; no LLM): the court retrieves how this house has
385
- resolved similar disputes before, and whether a precedent applies stays the
386
- human's judgment. Memory grows case law.
387
-
388
- ## The principal interview
389
-
390
- The vault interrogates you. `daftari interview` assembles a question sheet
391
- from what the vault already knows is unclear — open tensions (contested
392
- first, longest-carried first), canonical accumulation docs past their TTL,
393
- and `questions_raised` entries no document answers — deterministic and
394
- LLM-free, like the circadian pass:
395
-
396
- ```bash
397
- # The sheet — read-only, priority-ordered
398
- daftari interview --vault ./my-vault
399
-
400
- # Sit for it. Answers are recorded VERBATIM; empty skips, 'q' ends.
401
- daftari interview ask --vault ./my-vault
402
- ```
403
-
404
- Answered questions become a transcript document in an `interviews/`
405
- collection — `tier: source` (your words, body immutable), `provenance:
406
- direct`, no TTL (testimony doesn't expire) — committed like any other write,
407
- with `sources` tracing each question back to the tension or document that
408
- prompted it. The pattern is the companion interview on a living-summary
409
- page: the compiler's highest-value move is not more compilation, it is
410
- asking the principal about the unclear spots and folding the verbatim
411
- answers back into the corpus.
412
-
413
- Recording testimony resolves nothing. The transcript is *evidence*: the CLI
414
- ends by pointing at `daftari court rule <id> --references <transcript>` for
415
- tensions it touched, and at re-verified writes for stale docs. Operator-only,
416
- like the court — no MCP tool, no access context.
417
-
418
- ## Belief archaeology
419
-
420
- Git is the version layer, so the vault can answer **"what did we believe on
421
- March 3?"** `daftari asof` is a read-only report over the repo's history — no
422
- checkout, no index, no API key:
423
-
424
- ```bash
425
- # The vault's belief state at a date (or any git ref), plus the drift since
426
- daftari asof 2026-03-03 --vault ./my-vault
427
-
428
- # One document's trajectory: frontmatter then vs now, commits in between
429
- daftari asof HEAD~20 --doc pricing/helios-consumption-pricing.md
430
-
431
- # Counterfactual replay: this fact turned out wrong — who had inherited it
432
- # at the time, and where are they now?
433
- daftari asof 2026-03-03 --blast pricing/helios-consumption-pricing.md
434
- ```
435
-
436
- The default report shows the document and tension state at that point and
437
- the drift since: documents added/removed, `status`/`confidence` transitions,
438
- and tensions opened or resolved. `--blast` computes the blast radius of a
439
- document over the tree *as of the commit* (same source/link edge semantics as
440
- `vault_tension_blast`), annotating each downstream document with its status
441
- today — the post-mortem view for a fact that was later corrected. Markdown to
442
- stdout by default; `--output` / `--output-json` write files. Pairs with
443
- `vault_receipt`: a receipt's `vaultHead` is exactly the anchor to hand back
444
- to `asof`.
445
-
446
- ## The vault as witness — and the wager layer
447
-
448
- Every write already carries an identity, every proposal an outcome, every
449
- tension a logger and a ruling. `vault_witness` aggregates that ledger into a
450
- **track record per principal** — and prices it. Confidence is free to claim,
451
- so the wager schedule makes it cost something: writing at `high` stakes 3
452
- points, `medium` 1, `low` 0 (hedged claims are the honest default and are
453
- never taxed). A claim later corrected by a ruling or retired by someone else
454
- burns the stake; a claim maintained through a full TTL cycle earns credit.
455
- The balance is arithmetic on recorded facts — advisory, provisional
456
- constants, nothing enforced: routing a high-stakes write to the agent with
457
- the earned balance is your policy, not the vault's.
458
-
459
- <details>
460
- <summary><b>The two kill conditions that travel with the tool (▸ expand for full reference)</b></summary>
461
-
462
- Both kill conditions from the design travel with the tool: the flat-curve
463
- monitor (one author ≥95% of writes → curves declared uninformative) and the
464
- longitudinal write-volume series (if stake-fear suppresses honest claims, it
465
- shows up here first).
466
-
467
- </details>
468
-
469
- ## Coherence audit
470
-
471
- `daftari audit` is a read-only, deterministic check across one or more
472
- markdown repos for **broken cross-repo references** and **link-graph
473
- transitive staleness**. It works against any markdown tree — daftari-managed
474
- or not. The audit creates no `.daftari/` directory and writes nothing to the
475
- audited repos.
476
-
477
- ### Multi-repo (the headline use case)
478
-
479
- When two or more repos link to each other, the audit detects broken
480
- references that neither repo's own lint could see — because each repo only
481
- knows about itself.
482
-
483
- ```bash
484
- daftari audit \
485
- --repo ~/repos/service-a \
486
- --repo ~/repos/service-b
487
- ```
488
-
489
- That works for relative-path links (`../service-b/docs/api.md`). For
490
- GitHub-style URL links between repos (`https://github.com/org/service-b/...`),
491
- declare each repo's URL patterns in an `audit.yaml` so the resolver can map
492
- them back to the local repo:
493
-
494
- ```yaml
495
- # audit.yaml
496
- repos:
497
- - name: service-a
498
- path: ~/repos/service-a
499
- urls: ["github.com/org/service-a"]
500
- - name: service-b
501
- path: ~/repos/service-b
502
- urls: ["github.com/org/service-b"]
503
- ```
504
-
505
- ```bash
506
- daftari audit --config audit.yaml
507
- ```
508
-
509
- ### Single repo
510
-
511
- The same command, one `--repo`:
512
-
513
- ```bash
514
- daftari audit --repo ./docs
515
- ```
516
-
517
- In single-repo mode the cross-repo check trivially has no work, but the
518
- staleness check still runs over the in-repo link graph.
519
-
520
- ### What gets detected
521
-
522
- - **Missing files.** A link from `service-a/intro.md` to `../service-b/api.md`
523
- or `https://github.com/org/service-b/blob/main/api.md` — flagged if `api.md`
524
- doesn't exist in `service-b`.
525
- - **Missing anchors.** Same link with `#run` — flagged if `api.md` has no
526
- `## Run` heading.
527
- - **Direct staleness.** Any doc whose git mtime is older than
528
- `staleness.threshold_days` (default 540, ~18 months).
529
- - **Transitive staleness.** A fresh doc that links — directly or through a
530
- chain — to a stale doc is itself flagged, with the shortest chain reported.
531
- Catches the case where you keep touching an index page while the docs it
532
- links to are rotting.
533
-
534
- <details>
535
- <summary><b>Sample output, CI integration, exit codes, CLI flags, and the full <code>audit.yaml</code> schema (▸ expand for full reference)</b></summary>
536
-
537
- ### Sample output
538
-
539
- ```markdown
540
- # Coherence Audit Report
541
-
542
- ## Totals
543
- - repos scanned: **2**
544
- - docs scanned: **47**
545
- - broken cross-repo refs: **2**
546
- - directly stale docs: **3**
547
- - transitively stale docs: **5**
548
-
549
- ## Broken cross-repo references
550
- | kind | source | target | href |
551
- |----------------|---------------------------|-------------------------|------|
552
- | missing_anchor | service-a/intro.md | service-b/api.md#run | `https://github.com/org/service-b/blob/main/api.md#run` |
553
- | missing_file | service-a/architecture.md | service-b/deleted.md | `../service-b/deleted.md` |
554
-
555
- ## Staleness
556
- | kind | doc | mtime | chain |
557
- |------------|--------------------------|------------|-------|
558
- | transitive | service-a/onboarding.md | 2026-04-01 | service-a/onboarding.md → service-b/legacy-flow.md |
559
- ```
560
-
561
- JSON output (`--output-json` or `output.json` in config) carries the same
562
- structure with full detail in `brokenRefs[]` and `staleness[]` arrays plus a
563
- `totals` summary block for compact downstream rendering.
564
-
565
- ### CI integration
566
-
567
- The audit's exit code is designed to gate CI:
568
-
569
- ```yaml
570
- # .github/workflows/docs-audit.yml
571
- name: Docs audit
572
- on: [pull_request]
573
- jobs:
574
- audit:
575
- runs-on: ubuntu-latest
576
- steps:
577
- - uses: actions/checkout@v4
578
- with: { fetch-depth: 0 } # full history so git mtime works
579
- - run: npx daftari@latest audit --config audit.yaml
580
- ```
581
-
582
- Exit codes:
583
-
584
- | code | meaning |
585
- |------|---------|
586
- | `0` | clean run, all findings within `fail_on` thresholds |
587
- | `1` | clean run but a threshold was exceeded — CI fails |
588
- | `2` | config error (missing required fields, bad paths, malformed YAML) |
589
- | `3` | runtime error (IO failure during collection) |
590
-
591
- ### CLI flags
592
-
593
- `audit.yaml` and CLI flags overlap; CLI wins. A warning is printed to stderr
594
- when `--output` or `--output-json` displaces a value from the config.
595
-
596
- - `--repo <path>` — add a repo. May be repeated. Anonymous CLI repos get no
597
- URL patterns; URL-based cross-refs into them won't be detected. Use
598
- `--config` for URL-aware repos.
599
- - `--config <path>` — load an `audit.yaml`.
600
- - `--output <md>` — markdown report destination (default: stdout).
601
- - `--output-json <json>` — JSON report destination (default: not written).
602
- - `--help` — full help text.
603
-
604
- ### Full `audit.yaml` schema
605
-
606
- ```yaml
607
- repos:
608
- - name: service-a
609
- path: ~/repos/service-a
610
- docs_glob: "docs/**/*.md" # default: "**/*.md"
611
- urls: # optional; enables URL-pattern matching
612
- - "github.com/org/service-a"
613
-
614
- - name: service-b
615
- path: ~/repos/service-b
616
- urls:
617
- - "github.com/org/service-b"
618
-
619
- output:
620
- markdown: coherence-report.md # default: stdout
621
- json: coherence-report.json # default: not emitted
622
-
623
- staleness:
624
- threshold_days: 540 # default: 540 (18 months)
625
-
626
- fail_on:
627
- broken_refs: 1 # default: fail on any broken ref
628
- transitive_staleness: 100 # default: generous; teams tune
629
- ```
630
-
631
- </details>
632
-
633
- <!-- Running it -->
634
-
635
- ## Access control
636
-
637
- No user-management system. Roles live in config, the server starts with one.
638
- No `--role` or an unknown name falls back to deny-all. An agent identity is
639
- just a role too — e.g. a `curation-loop` role that reads and writes but leaves
640
- `ratify` off: the agent proposes, humans ratify.
641
-
642
- <details>
643
- <summary><b>Roles config example (▸ expand for full reference)</b></summary>
644
-
645
- ```yaml
646
- roles:
647
- analyst:
648
- read: [competitive-intel, pricing]
649
- write: [competitive-intel, _drafts]
650
- researcher:
651
- read: ["*"]
652
- write: [moonshot, _drafts]
653
- admin:
654
- read: ["*"]
655
- write: ["*"]
656
- promote: true
657
- ratify: true # may approve/reject staged actions and contest edges
658
- ```
659
-
660
- </details>
661
-
662
- ## Cross-vault federation
663
-
664
- One process, one *writable* vault — but it can mount other vaults read-only
665
- and search across them. Reads and search compose over every mounted vault;
666
- writes, locks, git, and curation stay bound to the local one. The boundary in
667
- one line: **a mount exposes documents, not vault state** — you can read and
668
- search a mounted vault's markdown, but its tensions, edges, provenance, and
669
- lint never cross.
670
-
671
- Declare mounts in your own config:
672
-
673
- ```yaml
674
- # your vault's .daftari/config.yaml
675
- federation:
676
- mounts:
677
- - alias: research
678
- path: ../research-vault
679
- index: lexical # skip embeddings for this mount (default: full)
680
- ```
681
-
682
- Access is granted by the *other* vault, in its own config, keyed by your
683
- authenticated identity — deny-all until its owner says otherwise:
684
-
685
- ```yaml
686
- # the referenced vault's .daftari/config.yaml
687
- federation:
688
- principals:
689
- "human:mihir": { role: researcher } # a role defined in THAT vault
690
- ```
204
+ Read the [file-format reference](docs/file-format.md) for validity intervals,
205
+ typed source references, lifecycle fields, positions, and extension rules.
691
206
 
692
- Federated documents address as `research:notes/pricing.md`, every result
693
- carries a `vault` label, and `vault_search` fuses ranked hits across vaults
694
- (rank fusion, so differing corpus statistics can't skew scores). Nothing is
695
- ever written under a mounted vault — its search index lives inside *your*
696
- `.daftari/`, rebuilt from its markdown like any other derived cache. Mount
697
- freshness is checked at startup; `vault_reindex {mount: research}` refreshes
698
- on demand. stdio-only for now: `daftari serve` refuses a config with mounts.
207
+ ## Run it where the work happens
699
208
 
700
- ## Server mode (self-hosted)
209
+ ### Local MCP server
701
210
 
702
- By default daftari is a per-user stdio process. `daftari serve` starts the
703
- same vault as **one always-on instance over Streamable HTTP** — for a team, or
704
- for clients that can't spawn a local process (claude.ai web/mobile, Cowork):
211
+ The default mode is one stdio process serving one writable vault:
705
212
 
706
213
  ```bash
707
- daftari serve --vault ./my-vault # http://127.0.0.1:8787/mcp
708
- daftari serve --vault ./my-vault --bind 0.0.0.0 --port 9000
709
- daftari serve --vault ./my-vault --legacy-http # also answer 2025-era MCP clients
710
- ```
711
-
712
- Identity is **per request**, resolved from the bearer on every request (MCP
713
- 2026-07-28 is stateless — there are no sessions), so every RBAC and
714
- existence-disclosure rule applies per caller with zero tool changes. Two
715
- composable auth schemes, both declared in config. `--legacy-http` is a
716
- temporary, opt-in migration flag for clients still on 2025-era MCP: same
717
- process, same per-request auth, no session table (removal tracked in #371).
718
-
719
- <details>
720
- <summary><b>Auth config (bearer tokens + OAuth 2.1) and the fail-loud posture (▸ expand for full reference)</b></summary>
721
-
722
- ```yaml
723
- server:
724
- transport_security: external # required off-loopback: TLS terminates upstream
725
- auth:
726
- tokens: # static bearer tokens (agents, ETL)
727
- - env: DAFTARI_TOKEN_ETL # the VALUE lives in this env var, never in config
728
- user: agent:etl
729
- role: admin
730
- oauth: # OAuth 2.1 resource server (humans via your IdP)
731
- issuer: https://idp.example.com
732
- audience: daftari
733
- jwks_uri: https://idp.example.com/.well-known/jwks.json
734
- subjects:
735
- "alice@example.com": { user: human:alice, role: analyst }
214
+ npx daftari --vault ./my-vault --user me --role admin
736
215
  ```
737
216
 
738
- The posture is fail-loud, never silent-downgrade: a non-loopback bind refuses
739
- to start without auth **and** the explicit `transport_security: external`
740
- acknowledgment (daftari never terminates TLS itself — put Caddy/nginx/a load
741
- balancer in front); with any auth configured, a bad or missing credential is
742
- a 401 at session open and a valid JWT whose subject isn't mapped is a 403 —
743
- neither ever becomes the guest. Guest sessions exist only in the no-auth
744
- loopback configuration. A running server also refuses to be silently killed:
745
- stray stdio invocations against the same vault refuse to start, and replacing
746
- a live holder requires a deliberate `daftari serve --takeover`.
747
-
748
- </details>
217
+ This is the normal setup for Claude Desktop, Claude Code, and local agent SDKs.
749
218
 
750
- ## Storage backing
219
+ ### Shared self-hosted server
751
220
 
752
- A self-hosted vault can push to durable object storage (#6). The local git
753
- working copy stays canonical — reads, writes, auto-commits, locks, and the
754
- index all stay exactly as above — and the backing is a dumb sync target.
755
-
756
- <details>
757
- <summary><b>Storage config, <code>daftari sync</code> commands, and sync semantics (▸ expand for full reference)</b></summary>
758
-
759
- ```yaml
760
- # .daftari/config.yaml
761
- storage:
762
- backend: s3 # fs | s3 | azure
763
- bucket: team-vault
764
- region: us-east-1
765
- # endpoint: https://… # MinIO / R2 / GCS S3-interop endpoints
766
- # sync_interval_minutes: 15 # daftari serve pushes on this cadence
767
- ```
221
+ For multiple clients, `daftari serve` exposes the same vault over Streamable
222
+ HTTP:
768
223
 
769
224
  ```bash
770
- daftari sync --vault ./my-vault # incremental push
771
- daftari sync --vault ./my-vault --dry-run # show the diff only
772
- daftari sync --vault ./empty-dir --restore --backend s3 --bucket team-vault
225
+ daftari serve --vault ./my-vault
773
226
  ```
774
227
 
775
- The push covers the markdown tree, the `.git` directory, and durable
776
- `.daftari` journals; the rebuildable SQLite index and lock files never sync.
777
- Neither do `.git/config` and `.git/hooks` — git *executes* what those
778
- declare, and a backup channel must not deliver code (`git clone` refuses to
779
- transmit them for the same reason), so re-add remotes and local git config
780
- by hand after a restore.
781
- Cloud backends load their SDKs (`@aws-sdk/client-s3`, `@azure/storage-blob`)
782
- as optional dependencies — install the one you use; credentials come from the
783
- SDK's standard environment chain, never from vault config. GCS is reached via
784
- its S3-interoperability endpoint. Restore refuses non-empty directories and
785
- reindexes when done.
786
-
787
- </details>
228
+ Non-loopback deployments fail closed unless authentication and external
229
+ transport security are configured. See [deployment and access](docs/deployment.md)
230
+ for bearer tokens, OAuth 2.1, process takeover, federation, and storage backing.
788
231
 
789
- ## Adopting an existing vault
232
+ ### Existing markdown
790
233
 
791
- Already have a wiki or an Obsidian vault? Daftari adopts it **in place** — it
792
- indexes and curates the same markdown files you already edit, not a separate copy.
234
+ Daftari can inspect and adopt an Obsidian vault or other markdown wiki in
235
+ place. Schema inference and drift checks are read-only; import fills missing
236
+ frontmatter without replacing existing content.
793
237
 
794
238
  ```bash
795
- # Dry run — see what would change, write nothing
239
+ daftari schema infer --vault ~/my-vault
240
+ daftari schema diff --vault ~/my-vault
796
241
  daftari import obsidian ~/my-vault --plan
797
-
798
- # Adopt one folder at a time (per-folder ratification; --yes to skip the prompt)
799
- daftari import obsidian ~/my-vault --apply --scope notes
800
- ```
801
-
802
- The import is non-destructive to your content. It fills only the *missing*
803
- Daftari frontmatter — collection from the folder, dates from git history (or file
804
- mtime), sensible defaults for the rest — and preserves everything you already
805
- had, including custom frontmatter fields. Obsidian specifics: inline `#tags` are
806
- merged into `tags`, a Web Clipper `source` is mapped into `sources`, and
807
- `[[wikilinks]]` are left untouched (Daftari resolves them as written). Filling
808
- frontmatter is deliberate, not automatic — re-run the import to pick up newly
809
- added notes.
810
-
811
- ### Cloud-synced vaults (iCloud, Dropbox, …)
812
-
813
- Daftari versions every change with git, and a `.git/` directory churning inside a
814
- cloud-synced folder can corrupt. For a synced vault, keep git's data outside it:
815
-
816
- ```bash
817
- daftari import obsidian "~/Library/Mobile Documents/.../my-vault" \
818
- --apply --scope notes --external-git-dir
819
242
  ```
820
243
 
821
- This writes `git_dir: external` to `.daftari/config.yaml`, so Daftari uses
822
- `git init --separate-git-dir`: only a tiny static `.git` *file* stays in the vault
823
- (syncs harmlessly) while the repo data lives under `~/.local/share/daftari/git/`.
824
- Pass `--external-git-dir=/path` for an explicit location, or set `git_dir`
825
- directly in config to apply the same to any vault. History is per-device; your
826
- notes still sync everywhere.
244
+ Cloud-synced folders need an external Git directory so sync software never
245
+ copies a live `.git/` database. Follow the safeguards in
246
+ [adopting existing notes](docs/adoption.md) before applying an import.
827
247
 
828
- ### Lower-level: `backfill`
248
+ ## What Daftari does not do
829
249
 
830
- `daftari import obsidian` is a thin, Obsidian-aware wrapper over `daftari
831
- backfill` — the git-driven frontmatter migration that works on any markdown wiki.
832
- Use it directly for non-Obsidian trees:
250
+ - It does not resolve contradictions by generating a compromise.
251
+ - It does not auto-fix lint findings or promote agent output on its own.
252
+ - It does not hide the canonical files behind a proprietary database.
253
+ - It does not provide a hosted multi-tenant service; server mode is
254
+ self-hosted.
255
+ - It does not replace the model or agent framework. It gives them a durable
256
+ memory substrate over MCP.
833
257
 
834
- ```bash
835
- daftari backfill --plan
836
- daftari backfill --apply --scope specs
837
- ```
838
-
839
- ## Open Knowledge Format (OKF)
840
-
841
- [OKF](https://cloud.google.com/blog/products/data-analytics/how-the-open-knowledge-format-can-improve-data-sharing/)
842
- is Google Cloud's vendor-neutral spec for the LLM-wiki pattern: a directory of
843
- markdown files with YAML frontmatter that any producer can emit and any consumer
844
- can read without translation. A Daftari vault *is* that pattern, so `daftari
845
- okf` bridges the two directions.
846
- [v0.2](https://cloud.google.com/blog/products/data-analytics/okf-v0-2-adds-trust-signals/)
847
- adds opt-in **trust signals** — raw credibility indicators (provenance,
848
- verification, lifecycle, freshness), never computed scores — and Daftari maps
849
- them from/to its native metadata.
850
-
851
- <details>
852
- <summary><b>OKF export / import field-mapping and the <code>daftari okf</code> commands (▸ expand for full reference)</b></summary>
853
-
854
- **Export** renders the vault as a portable OKF bundle — every doc becomes an OKF
855
- concept doc (the core `type` / `title` / `description` / `resource` / `tags`
856
- fields, plus a verbatim `daftari` sidecar for lossless round-trip), with
857
- generated `index.md` (progressive-disclosure listing) and `log.md`
858
- (chronological history). Trust signals derive from native metadata:
859
- `updated`/`updated_by` become `generated`, the lifecycle maps to
860
- `draft`/`stable`/`deprecated`, `ttl_days` becomes an absolute `stale_after`
861
- date, and `sources` become structured entries. `verified` is never fabricated —
862
- Daftari records authorship, not independent confirmations, so exported docs
863
- honestly read as unverified. The source vault is never mutated.
864
-
865
- ```bash
866
- daftari okf export ./my-vault --out ./okf-bundle
867
- daftari okf export ./my-vault --out ./okf-bundle --collection pricing
868
- ```
869
-
870
- **Import** adopts an OKF bundle into a vault. A bundle produced by `okf export`
871
- round-trips exactly via its sidecar; a foreign bundle is mapped conservatively
872
- (docs land as `draft` in the `accumulation` domain, the original OKF `type` is
873
- preserved in an `okf_type` field). Trust signals inform the mapping: a
874
- human-reviewed doc (a `human:*` entry in `verified`) imports with `confidence:
875
- high`, `status: deprecated` stays `deprecated` (retired knowledge must not
876
- resurface as a fresh draft), and `stale_after` converts to `ttl_days`. An
877
- `Attested Computation` is surfaced in an import warning but **not**
878
- auto-write-protected: `tier: source` is enforcement, and its only sanctioned
879
- grant path is `vault_set_tier` (reason required, provenance-logged) — a foreign
880
- bundle's self-declared `type` doesn't buy it. Review the doc, then elevate
881
- deliberately. OKF fields with no lossless Daftari slot (the trust record,
882
- attestation machinery) are preserved under `okf_*` keys. Writes auto-commit and
883
- the index is rebuilt.
884
-
885
- ```bash
886
- daftari okf import ./okf-bundle --into ./my-vault --dry-run
887
- daftari okf import ./okf-bundle --into ./my-vault
888
- ```
889
-
890
- </details>
891
-
892
- <!-- Reference / meta -->
258
+ For a longer comparison with adjacent memory patterns, see
259
+ [positioning](docs/positioning-2026-07.md). That analysis is kept outside the
260
+ onboarding path because competitor claims age faster than the product contract.
893
261
 
894
- ## What’s not in v1
262
+ ## Documentation
895
263
 
896
- Deliberately deferred to keep the surface tight:
264
+ The [documentation map](docs/README.md) organizes the full set by task. The
265
+ main paths are:
897
266
 
898
- - **Cloud-hosted multi-tenant SaaS** (self-hosted server mode HAS shipped —
899
- see [Server mode](#server-mode-self-hosted) and
900
- [Storage backing](#storage-backing); the project itself hosts nothing)
901
- - **Conflict resolution beyond file-level locks** (CRDTs, semantic merge)
902
- - **Background curation agent** running lint on a cadence
903
- - **Enforced domain separation** (v1 documents the convention; v2 enforces it —
904
- advisory boundary warnings shipped in the meantime: `vault_lint`'s
905
- `domainLeaks` check and write-time `domain_warnings`)
267
+ - [Getting started](docs/getting-started.md)
268
+ - [Worked example](docs/worked-example.md)
269
+ - [Curation workflow](docs/curation-workflow.md)
270
+ - [Operator workflows](docs/operator-workflows.md)
271
+ - [Deployment and access](docs/deployment.md)
272
+ - [Google Docs and Notion integrations](docs/integrations.md)
273
+ - [Adopting existing notes](docs/adoption.md)
274
+ - [Architecture](docs/architecture.md)
275
+ - [File format](docs/file-format.md)
276
+ - [Privacy](PRIVACY.md)
906
277
 
907
- (LLM reranking, deferred here originally, has since shipped as the opt-in
908
- agent-as-judge `rerank_candidates` on `vault_search`: the server prepares the
909
- fused candidate pool and the protocol; the calling agent is the judge.)
278
+ Integrations:
910
279
 
911
- Each is a clean increment on a surface that already works.
280
+ - [`integrations/langchain/`](integrations/langchain/) exposes Daftari tools as
281
+ LangChain `BaseTool`s for LangGraph and `create_react_agent`.
282
+ - [`packages/router/`](packages/router/) routes one MCP connection across
283
+ multiple writable Daftari vaults.
912
284
 
913
285
  ## Development
914
286
 
915
- ```
287
+ ```bash
916
288
  npm install
917
289
  npm run build
918
290
  npm test
919
291
  ```
920
292
 
921
- Design tenets: functions and types, no classes; tool handlers return
922
- `Result<T, Error>` rather than throwing; tests mirror the `src/` structure.
923
-
924
- The build board: the chain currently under construction — the derived-content
925
- critical path — is tracked with live progress in
926
- [#428](https://github.com/mavaali/daftari/issues/428).
927
-
928
- ## Documentation
929
-
930
- - <docs/getting-started.md> — scaffold, write, search, lint, promote, deprecate
931
- - <docs/architecture.md> — layered design, request path, accumulation vs. generative domains
932
- - <docs/file-format.md> — complete frontmatter reference
933
-
934
- ## Integrations
935
-
936
- - [`integrations/langchain/`](integrations/langchain/) — `langchain-daftari`, a
937
- Python package that exposes all 38 daftari tools as LangChain `BaseTool`s
938
- for use with LangGraph / `create_react_agent`. Sync + async, schemas pulled
939
- live from `tools/list`.
940
- - [`packages/router`](packages/router) — multi-vault MCP router that fans out across N Daftari vaults
941
-
942
- ## Privacy
943
-
944
- Daftari is a local MCP server. It runs on your machine, against vault files on
945
- your machine. The default configuration makes no network calls — vault content
946
- stays on your local filesystem and is read or written only through tools the
947
- MCP client invokes. The only optional egress is the OpenAI embedding provider,
948
- which the user must explicitly opt into per vault.
949
-
950
- Full policy: [PRIVACY.md](./PRIVACY.md) — covers data collection (none),
951
- storage (local-only), the OpenAI opt-in, third-party integrations (none),
952
- retention, and contact.
293
+ The codebase is TypeScript and Node.js. Functions and types are preferred over
294
+ classes; tool handlers return `Result<T, Error>` rather than throwing; tests
295
+ mirror the `src/` structure.
953
296
 
954
- ## License
297
+ ## Privacy and license
955
298
 
956
- MIT.
299
+ Daftari runs locally by default and makes no network calls unless a vault opts
300
+ into an external provider or integration. Read the [privacy policy](PRIVACY.md)
301
+ for the complete boundary.
957
302
 
303
+ Daftari is available under the [MIT License](LICENSE).