@dreamforgestudiollc/emet 0.0.0-stage → 0.2.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (114) hide show
  1. package/.env.example +189 -0
  2. package/CHANGELOG.md +360 -0
  3. package/CHARTER.md +137 -0
  4. package/LICENSE +22 -0
  5. package/README.md +868 -2
  6. package/docs/AUTHORSHIP.md +26 -0
  7. package/docs/BENCHMARKS.md +261 -0
  8. package/docs/BOOTSTRAP.md +127 -0
  9. package/docs/BOTS.md +83 -0
  10. package/docs/COVERAGE.md +64 -0
  11. package/docs/DEPLOY.md +251 -0
  12. package/docs/DISCIPLINES.md +29 -0
  13. package/docs/GUARDRAILS-AND-MONORAILS.md +108 -0
  14. package/docs/GUIDED-SETUP.md +301 -0
  15. package/docs/PERMISSIONS.md +121 -0
  16. package/docs/ROADMAP.md +62 -0
  17. package/docs/SETUP.md +265 -0
  18. package/docs/SUBKEYS.md +59 -0
  19. package/gems/artifact-index.json +12 -0
  20. package/gems/bot-team-backup/floor.json +15 -0
  21. package/gems/bot-team-backup/gem.json +43 -0
  22. package/gems/bot-team-backup/guides/BOT_TEAM_BACKUP.md +27 -0
  23. package/gems/bot-team-backup/guides/BOT_TEAM_RECOVERY.md +23 -0
  24. package/gems/fieldnote/floor.json +14 -0
  25. package/gems/fieldnote/gem.json +70 -0
  26. package/gems/fieldnote/runes/fieldnote.js +4 -0
  27. package/gems/lean-six-sigma/CHANGELOG.md +10 -0
  28. package/gems/lean-six-sigma/README.md +92 -0
  29. package/gems/lean-six-sigma/floor.json +31 -0
  30. package/gems/lean-six-sigma/gem.json +1296 -0
  31. package/gems/lean-six-sigma/guides/ASSUMPTIONS.md +45 -0
  32. package/gems/lean-six-sigma/guides/BELTS.md +52 -0
  33. package/gems/lean-six-sigma/guides/DMADV.md +56 -0
  34. package/gems/lean-six-sigma/guides/DMAIC.md +127 -0
  35. package/gems/lean-six-sigma/guides/FORMULAS.md +330 -0
  36. package/gems/lean-six-sigma/guides/LEAN.md +97 -0
  37. package/gems/lean-six-sigma/guides/TEMPLATES.md +279 -0
  38. package/gems/lean-six-sigma/runes/lss_capability.js +132 -0
  39. package/gems/lean-six-sigma/runes/lss_control_limits.js +240 -0
  40. package/gems/lean-six-sigma/runes/lss_dpmo_sigma.js +81 -0
  41. package/gems/lean-six-sigma/runes/lss_hypothesis.js +298 -0
  42. package/gems/lean-six-sigma/runes/lss_sample_size.js +81 -0
  43. package/gems/neurendipity/floor.json +65 -0
  44. package/gems/neurendipity/gem.json +42 -0
  45. package/gems/neurendipity/guides/AUTISTIC_BURNOUT.md +81 -0
  46. package/gems/neurendipity/guides/NEURODIVERGENT_SUPPORT.md +287 -0
  47. package/guides/README.md +3 -0
  48. package/package.json +76 -4
  49. package/server.json +79 -0
  50. package/src/aleph.js +395 -0
  51. package/src/annotations.js +170 -0
  52. package/src/approve-throttle.js +54 -0
  53. package/src/atlas-role.js +58 -0
  54. package/src/attribution.js +555 -0
  55. package/src/conformance.js +688 -0
  56. package/src/content_analyzer.js +450 -0
  57. package/src/database.js +812 -0
  58. package/src/disciplines.js +26 -0
  59. package/src/dispatch.js +382 -0
  60. package/src/env.js +97 -0
  61. package/src/floor.js +228 -0
  62. package/src/gaps.js +509 -0
  63. package/src/gem-check.js +1359 -0
  64. package/src/guides.js +75 -0
  65. package/src/host-reach.js +23 -0
  66. package/src/http-remote.js +313 -0
  67. package/src/index.js +397 -0
  68. package/src/invites.js +167 -0
  69. package/src/layers.js +449 -0
  70. package/src/lessons-to-checks.js +366 -0
  71. package/src/logger.js +300 -0
  72. package/src/metadata.js +61 -0
  73. package/src/module-path.js +32 -0
  74. package/src/mongo-connect.js +91 -0
  75. package/src/named-items.js +208 -0
  76. package/src/oauth-provider.js +838 -0
  77. package/src/owner-approvals.js +281 -0
  78. package/src/persons.js +136 -0
  79. package/src/provenance.js +257 -0
  80. package/src/redact.js +237 -0
  81. package/src/request-log.js +124 -0
  82. package/src/rune-dispatch.js +360 -0
  83. package/src/rune-registry.js +71 -0
  84. package/src/rune-scope.js +42 -0
  85. package/src/scopes.js +100 -0
  86. package/src/security-headers.js +26 -0
  87. package/src/session-graph.js +947 -0
  88. package/src/session.js +1097 -0
  89. package/src/setup.js +1580 -0
  90. package/src/socket.js +545 -0
  91. package/src/startup-pages.js +371 -0
  92. package/src/startup-payload.js +129 -0
  93. package/src/subkey-store.js +357 -0
  94. package/src/subkeys.js +947 -0
  95. package/src/tools.js +4508 -0
  96. package/src/transcript-session.js +106 -0
  97. package/src/validation.js +1187 -0
  98. package/src/version.js +2 -0
  99. package/templates/BOARD.md +72 -0
  100. package/templates/BOTHANDOFF.md +57 -0
  101. package/templates/CHARACTER.md +30 -0
  102. package/templates/HANDOFF.md +63 -0
  103. package/templates/IDENTITY.md +29 -0
  104. package/templates/INDEX.md +82 -0
  105. package/templates/INSTALL.md +29 -0
  106. package/templates/PROCEDURE.md +39 -0
  107. package/templates/ROSTER.md +43 -0
  108. package/templates/SEED.md +65 -0
  109. package/templates/STAGING.md +37 -0
  110. package/templates/STANDARD.md +99 -0
  111. package/templates/THREAD.md +43 -0
  112. package/templates/USER.md +32 -0
  113. package/verification/load-gate/corpus.json +1765 -0
  114. package/verification/load-gate/load-gate.mjs +1246 -0
package/README.md CHANGED
@@ -1,3 +1,869 @@
1
- # Temporary Holding Version
1
+ # EMET
2
2
 
3
- This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
3
+ EMET is an MCP (Model Context Protocol) memory server, persistent shared memory for AI agents, bot teams and chats.
4
+
5
+ [![M8ven Score](https://m8ven.ai/badge/mcp/dreamforgestudiollc-emet-z9o7e8?v=4d037e6ca2c2131c77968f135f4681da)](https://m8ven.ai/mcp/dreamforgestudiollc-emet-z9o7e8?s=readme)
6
+
7
+ <img src="site/assets/portrait-560.webp" width="120" height="120" alt="Close view of a sepia sculpture’s face and shoulders. The Hebrew word אמת is set across her headband.">
8
+
9
+ אמת — Hebrew for truth. Also **Enduring Memory & Epistemic Tiers.**
10
+
11
+ **Website:** <https://dreamforgestudiollc.github.io/EMET/>
12
+
13
+ See also: [CHANGELOG](CHANGELOG.md) · [CONTRIBUTING](CONTRIBUTING.md) · [SECURITY](SECURITY.md) · [Charter](CHARTER.md)
14
+
15
+ ## What it does
16
+
17
+ - **Layered memory.** Six layers — identity, semantic, episodic, procedural, meta, and working — hold curated claims, split by what kind of thing an entry is.
18
+ - **Verified writes.** Every write is read back and compared before it is reported as saved.
19
+ - **Word-for-word transcript tier.** A separate tier stores the conversation as it was written, so a summary cannot overwrite the record it came from.
20
+ - **stdio and remote HTTP with OAuth.** The same server runs locally over stdio, and remotely over HTTP with OAuth. The transport changes. The store does not.
21
+ - **MongoDB.** Atlas, or MongoDB you run yourself. A one-machine trial can use MongoDB 6 or newer. Only `semantic_recall` needs Atlas.
22
+ - **Golem Expertise Module (GEM) socket.** Add-on behavior ships as GEMs, seated at startup and checked by `gem-check` against the GEM standard in `gem-library/`. A GEM ranks below the core floor and below the user's current word, and cannot remove a protected floor line. Two work-in-progress examples ship bundled and stay off unless you ask: Neurendipity (neurodivergent engagement and autistic burnout support; support only, not counseling or treatment; `EMET_ACTIVE_GEMS=neurendipity`) and Lean Six Sigma (DMAIC and DMADV guidance, templates, and client-side calculators; `EMET_ACTIVE_GEMS=lean-six-sigma`). Fieldnote stays in the tree as a developer case-study of a runed GEM. It is not a user-facing third GEM, and it is off by default.
23
+ - **Bot-team backup (optional).** Off unless the first-run interview records yes. Yes is the recommended choice. The recipe and the recovery template ship as the bundled bot-team-backup GEM. EMET does not copy prompts or settings itself, and secret values are never stored.
24
+ - **Client-side GEM execution.** A GEM may declare a tool. EMET does not run that computation. EMET verifies the record, not the computation. The client runs the tool, and what is written back is checked the same way as any other write.
25
+ - **Released flag.** A GEM is seated only when its released flag is the boolean true. Any other value is refused. A missing flag is refused except for a legacy manifest pinned by id, version, and file digests.
26
+ - **Per-client attribution and approvals.** Every write is attributed to the client that made it. A new client is read-limited until it is approved. The approval is recorded.
27
+ - **MIT.** Version 0.2.6. EMET is still 0.x: stored document shapes may change before 1.0. Pin a version if you depend on them.
28
+
29
+ The store has one owner, and other people can join only through the optional Guest
30
+ (read-only) and Member (read and write) access, both off by default.
31
+
32
+ ## Document control (ISO 9001 7.5.3 / ISO 13485 4.2.5)
33
+
34
+ EMET is built for document control in environments that work under ISO 9001 and ISO 13485. It supports that work; it holds no certification and claims no conformity. An organization that uses it stays responsible for its own quality system, including validating EMET for its intended use.
35
+
36
+ - **No delete, on any store, by design.** No tool removes a memory entry, a document or a transcript. The database role EMET runs under is not granted remove on the record either.
37
+ - **Corrections are made by adding.** A memory entry is superseded by a revision that stands alone. A document is revised or retired. A transcript part is revised. Events are never rewritten.
38
+ - **Retired means obsolete, not gone.** `retire_doc` marks a document obsolete and keeps its id, content, version and digest. `read_doc` still returns it, flagged retired, with the reason.
39
+ - **Prior versions are retained.** Every document write archives the copy it replaces, and `read_doc_history` reads any retained revision back, checked against its stored digest.
40
+ - **Records are superseded whole, never patched.** The handoff and any other record document cannot be patched. A new revision is written in full and names the version it supersedes.
41
+ - **Templates are controlled documents.** Base templates ship with the release, and the shipped copy governs. Every templated write is checked against its template, and a write that fails is kept and marked for review, never silently accepted or discarded.
42
+ - **A test holds the line.** `write-path.test.js` fails if anything in `src/` removes from a layer, document or transcript collection, and `atlas-role.test.js` fails if the database role grants remove.
43
+
44
+ ## 30-second start
45
+
46
+ > **npm lags the repo.** `npx -y @dreamforgestudiollc/emet` installs 0.2.4 until 0.2.6 is
47
+ > published. For 0.2.6, clone this repo and run `npm ci && npm start`, or deploy the
48
+ > `Dockerfile`. See [docs/SETUP.md](docs/SETUP.md).
49
+
50
+ - **Guided setup:** [docs/GUIDED-SETUP.md](docs/GUIDED-SETUP.md) — never deployed anything. Your assistant installs, configures, deploys, and verifies. You sign in when asked and paste the connector URL.
51
+ - **Setup:** [docs/SETUP.md](docs/SETUP.md) — the manual walkthrough: MongoDB, both transports, and the first-run interview.
52
+ - **Deploy:** [docs/DEPLOY.md](docs/DEPLOY.md) — remote HTTP with OAuth, so a phone, a browser, or another vendor's agent reaches the same store.
53
+
54
+ > **Status: 0.x. The stored document shapes are not frozen yet.** Breaking changes to the
55
+ > schema may land before 1.0. Pin a version if you depend on it.
56
+
57
+ > **Set `EMET_SOURCE_TAGS` before you let more than one host or bot write.** The source-tag
58
+ > guards (unregistered tags refused on the main records, and on another bot's `bots/` folder)
59
+ > are **off unless `EMET_SOURCE_TAGS` is set**. With it unset, any caller can write under any
60
+ > tag. See `.env.example` and [docs/DEPLOY.md](docs/DEPLOY.md).
61
+
62
+ ## Install as a Cursor / Grok Bot plugin
63
+
64
+ EMET ships a plugin for Cursor and Grok Bot (this repo's `.cursor-plugin/plugin.json` and
65
+ `mcp.json`). The Cursor Marketplace listing is submitted and pending review. The plugin does **not** host memory for you: it connects to **your own** EMET
66
+ server, so your records stay in your MongoDB.
67
+
68
+ 1. Deploy EMET remotely: [docs/DEPLOY.md](docs/DEPLOY.md). Set `PUBLIC_URL` and a strong
69
+ `EMET_OAUTH_PASSPHRASE`. Check `https://<your-host>/health` returns `{"ok":true,...}`.
70
+ 2. Install the **EMET** plugin and set the variable `EMET_URL` to `https://<host>/mcp`.
71
+ 3. Connect. The OAuth sign-in opens EMET's passphrase page; enter your passphrase once per device.
72
+ 4. Verify: call `emet_status` with `{"probe": true}` (it writes, reads back and removes a row
73
+ from `_emet_verify` only), then save with `save_to_layer` and `recall` it from another client.
74
+
75
+ The plugin also ships an `emet-memory` skill that tells the agent when to recall, save and close
76
+ a session.
77
+
78
+ ## Shared memory across hosts
79
+
80
+ EMET stores agent memory in MongoDB (Atlas or self-hosted) across six layers, keeps a separate word-for-word
81
+ transcript tier, and verifies every write by reading it back. It runs as a local stdio server
82
+ and as a remote HTTP server from the same source, so it works as a bridge: AI models on several
83
+ hosts — desktop, CLI, browser, phone, another vendor's agent — read and write one shared
84
+ memory, and the work, checks, and decisions one leaves behind are there for the next. That
85
+ includes Grok Bot teams and single-session Grok chats connected to the same server: what the
86
+ bots save is there for a fresh Grok chat to find once it initializes, and the reverse. To
87
+ connect a single-session chat (a Grok project, or any chat host that takes a remote MCP
88
+ connector and supports its OAuth sign-in), add your deployed server's URL,
89
+ `https://<your-host>/mcp`, as a remote MCP connector there and complete the sign-in with the
90
+ passphrase you set (`EMET_OAUTH_PASSPHRASE`; see
91
+ [Connecting clients](docs/DEPLOY.md#connecting-clients)). A connector that only takes a fixed
92
+ header or key will not work. Then put the instruction block from
93
+ [docs/BOOTSTRAP.md](docs/BOOTSTRAP.md) in the project instructions: the chat reaches for that
94
+ memory most reliably, unprompted from the first turn, with the block there, or, as a fallback
95
+ where there is no such field, with it pasted as the first message. The store has one
96
+ owner, and other people can join only through the optional Guest (read-only) and
97
+ Member (read and write) access, both off by default.
98
+
99
+ ---
100
+
101
+ ## Two ways in
102
+
103
+ A session that has not initialized can read everything and can write only a new drop (`drops/<date>-<slug>.md`): its items, wrap-up and handoff. A session with a registered source and the full startup has full access. Every initialized session sees unhandled drops first. A board edit is refused only for a drop that was listed at that session's boot. New drops are capped at 20 per New York day, in one bucket: the whole local process on stdio, one shared visitor bucket for HTTP with no signed-in client, the source tag bound at approval when there is one, or that OAuth client otherwise. A source the caller types does not move the bucket. Drops are never deleted.
104
+
105
+ ## Editing working state
106
+
107
+ The server provisions a blank board at setup. No session can write the board. It is edited with `patch_doc` and a reason; an added row names its origin. Once any working document exists, it is edited, not rewritten. A change that leaves a document under half the size it had when this session first touched it, or under half of its largest size in the last 24 hours, is refused. `retire_doc` refuses a working document. `rename_doc` issues a new id and marks the old id retired; the text stays readable and stays in the list. `restore_doc` is the one whole-document recovery, for an owner-listed tag, and only a stored conforming revision whose digest matches.
108
+
109
+ ## The name
110
+
111
+ **EMET** is אמת — Hebrew for *truth*. It is also an acronym for what the server does:
112
+ **Enduring Memory & Epistemic Tiers.**
113
+
114
+ The choice is not decoration. In the golem legend the word *emet* is inscribed to animate the
115
+ body, and erasing its first letter leaves מת — *met*, **dead**. One character is the whole
116
+ difference.
117
+
118
+ That is why **the verification layer is called `aleph`.** Every write in EMET is read back and
119
+ compared before it is reported as saved. That read-back is the aleph: a memory system without
120
+ it still returns answers, still looks healthy, and is quietly a corpse — confidently serving
121
+ things it never actually stored. Aleph is also the *first* letter, which matches the order the
122
+ protocol insists on: verify before retrying, verify the premise before applying the remedy,
123
+ read back before trusting.
124
+
125
+ The same image explains what setup is for. A golem is inert until the word is written on it;
126
+ a fresh model instance is unformed until the record is written into it. `emet_initialize` is
127
+ that inscription — it hands the model an identity, the person it serves, where things stood,
128
+ and the disciplines it is bound by. Nothing mystical is claimed here. It is a literal
129
+ description of what the first tool call does.
130
+
131
+ Both halves of the acronym are load-bearing.
132
+
133
+ **Enduring** is not a modest claim, and it is not a feature competing with other memory
134
+ products. It is measured against what a language model is by default: an etch-a-sketch. Every
135
+ session starts from nothing, and ends by forgetting everything. You explain yourself again,
136
+ re-establish what you are working on again, re-derive decisions you already made. **Enduring
137
+ names the difference between an assistant that knows you and one meeting you for the first
138
+ time, again.** That is the character of the thing, not a marketing adjective attached to it.
139
+
140
+ **Epistemic Tiers** is how that endurance stays trustworthy. Tier 1 holds curated claims;
141
+ tier 2 holds the verbatim record that justifies them — justified belief and its evidence, kept
142
+ structurally separate so neither can quietly overwrite the other. Memory that persists but
143
+ cannot be checked is just confident noise that survives longer.
144
+
145
+ ---
146
+
147
+ ## One memory, every device
148
+
149
+ This is the part worth understanding before anything else.
150
+
151
+ Most MCP memory servers run as a local process: your desktop launches it over stdio, and the
152
+ memory lives wherever that process can reach. That works until you pick up your phone.
153
+
154
+ EMET ships **two entry points over one source tree**:
155
+
156
+ | | | |
157
+ |---|---|---|
158
+ | `src/index.js` | stdio | desktop and CLI, launched by the client |
159
+ | `src/http-remote.js` | streamable HTTP + OAuth | phone, tablet, browser, another vendor's agent |
160
+
161
+ **They are not two projects.** Both call the same `dispatchTool` in `dispatch.js`, which calls
162
+ `tools.js` and `database.js`. `src/index.js` also imports `validation.js`. `src/http-remote.js`
163
+ does not import `validation.js` or `content_analyzer.js`; `tools.js` imports the analyser. The
164
+ transport is the only thing that differs. There is no fork to keep in sync, no second copy
165
+ drifting out of date, no "mobile version" lagging the real one. A fix lands once and both
166
+ transports have it.
167
+
168
+ Deploy the HTTP entry point to any host that can run a container — the `Dockerfile` is
169
+ included, and [docs/DEPLOY.md](docs/DEPLOY.md) walks through Railway and Fly — and every device
170
+ you own reaches **the same store**. Write a decision from your laptop, ask about it from your
171
+ phone an hour later, and it is simply there. Add a third host and it joins the same memory
172
+ rather than starting its own.
173
+
174
+ The remote endpoint is OAuth-protected: a device is authorized once with a passphrase you set,
175
+ and the token persists across restarts and redeploys. Leave `EMET_SOURCE_TAG` unset on a
176
+ server several hosts share. Over HTTP, document, transcript, and memory writes use the source
177
+ tag bound at approval. A local stdio host still passes an exact tag from `EMET_SOURCE_TAGS`
178
+ as `source` on its writes (see `.env.example`). Every entry records which machine produced
179
+ it — with several hosts writing to one store, provenance stops being optional bookkeeping and becomes the thing that lets you trust
180
+ what you read.
181
+
182
+ Running cost for a single user is a small MongoDB cluster plus a hobby-tier container
183
+ host. Both have free tiers adequate to start.
184
+
185
+ ---
186
+
187
+ ## Best practices
188
+
189
+ **Split the work by what each host does best.** A single-session chat (for example, a Grok
190
+ project chat, connected as described at the top) only works when you send it a message. It can't watch EMET, and bots can't
191
+ message it. So when a bot team needs the chat's help:
192
+
193
+ 1. The bots leave a task in EMET with everything needed to do it, in a document whose id they
194
+ give you.
195
+ 2. You send the chat one line that names that document, for example: "Initialize EMET, read
196
+ threads/open-tasks.md, do the task, and save the result to EMET."
197
+ 3. The chat saves its draft to EMET.
198
+ 4. The bots check it.
199
+
200
+ Good fits for the chat: long drafting and review done in one sitting. Good fits for the bots:
201
+ work in the repository, on the server, on a schedule, or in accounts. If the chat runs on a
202
+ separate plan, its work draws on that plan's allowance, not the bots'.
203
+
204
+ **Fetch every startup page, then open the session with that startup's `load_id`.** Call
205
+ `emet_initialize {}` first and fetch every page it names; ask for `{"page": "all"}` only on a
206
+ host known to read long results whole. Pass the `load_id` from those pages to
207
+ `emet_session_open`. An open with no `load_id`, with a page missing, or with a `load_id` the
208
+ server has no record of (one from before a restart, or 60 minutes past its last page) is
209
+ refused, and the reply says what to call. The page check runs on your own startup, and a
210
+ skipped page is refused before anything is written. See [docs/BOOTSTRAP.md](docs/BOOTSTRAP.md)
211
+ and [docs/BOTS.md](docs/BOTS.md).
212
+
213
+ **Set the source-tag list before more than one host or bot writes.** The guards that refuse
214
+ an unregistered tag are off unless `EMET_SOURCE_TAGS` is set; with it unset, a stdio caller can
215
+ write under any tag. Over HTTP the tag is the one bound when that host was approved. Add each
216
+ host's tag to the list before that host's setup interview. With several hosts writing one store,
217
+ the tag is what lets you trust who wrote what. See [docs/SETUP.md](docs/SETUP.md).
218
+
219
+ **Put the instruction block in the project's instructions.** Connecting the server gives the
220
+ assistant the ability to remember; it does not make it use that ability. A host with an
221
+ instructions field is the strongest integration: paste the block from
222
+ [docs/BOOTSTRAP.md](docs/BOOTSTRAP.md) there. Where there is no such field, paste it as the
223
+ first message of a session. With the block in place, the assistant reaches for memory most
224
+ reliably from the first turn, instead of answering from what it can infer and saving nothing.
225
+
226
+ **Approve reads standing, and keep an eye on the three writes that replace a document.** Reads
227
+ cost nothing and cannot be wrong destructively, so a reasonable default for most users is
228
+ standing approval for reads. Additive writes are recoverable, because a correction supersedes what it corrects rather than
229
+ overwriting it. The three replacing writes archive the prior version, but a wrong one is
230
+ silently a different document rather than an error. Do not carry this posture to a filesystem
231
+ or shell server. See [docs/PERMISSIONS.md](docs/PERMISSIONS.md).
232
+
233
+ **Reconnect each host after an upgrade that changes the tool list.** Hosts cache the tool
234
+ list: after a deploy that adds or renames tools, each host keeps offering the old set until
235
+ you reconnect the connector there. New tools stay invisible until the host sees the new list,
236
+ and the host may ask again for approvals you had already given. See
237
+ [docs/DEPLOY.md](docs/DEPLOY.md).
238
+
239
+ **Register a bot before its first write, and set both lists in one change.** One bot, one tag.
240
+ In the same deployment, add the tag to `EMET_SOURCE_TAGS` and append `bots/<tag>/HANDOFF.md`
241
+ to `EMET_RECORD_DOCS`. A registered tag that is not listed as a bot can still write the main
242
+ records. The hub's tag is registered as a host and is never listed as a bot. See
243
+ [docs/BOTS.md](docs/BOTS.md).
244
+
245
+ **Prove the write path before you trust a new deploy.** Run `emet_status` with `probe: true`.
246
+ It writes a test document, reads the same value back and removes it; a read alone cannot show
247
+ a broken write path. A role missing a permission fails there, loudly, instead of in the middle
248
+ of a session. Then save something from one device and recall it from another. See
249
+ [docs/SETUP.md](docs/SETUP.md).
250
+
251
+ **Keep house style in the store, not pasted into every project.** A document written with
252
+ `write_doc` is read by every host and changed in one place; text pasted into three projects is
253
+ three copies that will disagree. Delivery preferences belong in `bootstrap/USER.md`, not in one
254
+ vendor's own memory, which does not travel to the next host. See
255
+ [docs/BOOTSTRAP.md](docs/BOOTSTRAP.md).
256
+
257
+ ---
258
+
259
+ ## Attribution
260
+
261
+ EMET is a derivative of **CASCADE Enterprise RAM** by Jason Glass / C.I.P.S. LLC, published
262
+ at `github.com/For-Sunny/cascade-memory-enterprise`, used and redistributed under the MIT
263
+ license the project adopted at commit `acdb5c8c99cdfc29fb44db2cd828d7752be744db` (v2.2.2,
264
+ 2026-02-19). Both copyright lines are retained in `LICENSE`.
265
+
266
+ *Link unavailable:* as of 2026-10-03 that repository no longer resolves (GitHub answers 404),
267
+ and no moved copy, public mirror or archive of it was found. The address and commit are kept
268
+ here as the record of where this code came from.
269
+
270
+ **How much is upstream, measured rather than estimated.** Line-level comparison of this tree
271
+ against upstream v2.2.2 at the commit named above.
272
+
273
+ **Method**, stated so it can be reproduced: exact, case-sensitive comparison of non-blank
274
+ lines; an upstream line counts as retained if the same line is present in this tree's file.
275
+ "How much of upstream survives here" and "how much of this file is upstream's" are different
276
+ questions with different answers, so both are given. **`scripts/measure-upstream.mjs`
277
+ regenerates this table** against a clone of the commit above (while the upstream repository is unavailable, that needs a copy kept from before) — run it after any change to
278
+ `src/`, because a figure nobody can cheaply re-derive is a figure that goes quietly stale.
279
+
280
+ | File | Lines here | Upstream lines retained | of upstream | of this file |
281
+ |---|---:|---|---:|---:|
282
+ | `src/content_analyzer.js` | 450 | not recomputed | — | — |
283
+ | `src/validation.js` | 1163 | not recomputed | — | — |
284
+ | `src/index.js` | 390 | not recomputed | — | — |
285
+ | `src/tools.js` | 4273 | not recomputed | — | — |
286
+ | `src/database.js` | 811 | not recomputed | — | — |
287
+ | `src/http-remote.js` | 269 | — | — | no upstream counterpart |
288
+ | `src/oauth-provider.js` | 702 | — | — | no upstream counterpart |
289
+ | `src/named-items.js` | 206 | — | — | no upstream counterpart |
290
+ | `src/security-headers.js` | 26 | — | — | no upstream counterpart |
291
+ | `src/layers.js`, `floor.js`, `aleph.js`, `session.js`, `setup.js`, `gaps.js`, `provenance.js`, `redact.js`, `annotations.js`, `request-log.js`, `disciplines.js`, `metadata.js`, `env.js` | 4803 | — | — | no upstream counterpart |
292
+
293
+ **Lines here** were counted on 2026-10-07 (every line, including blanks, with one trailing empty line dropped). The retained percentages were not recomputed: the upstream repository no longer resolves, and no local copy of that upstream commit was available. The last measurement, against an earlier tree, was content analyser 361 of 394 (91.6% of upstream, 93.5% of that file), validation 765 of 798 (95.9%, 78.2%), index 541 of 631 (85.7%, 74.0%), tools 798 of 1007 (79.2%, 33.2%), and database 353 of 651 (54.2%, 40.2%). Other files under `src/` also have no upstream counterpart and are not in this table.
294
+
295
+ Upstream's `server/decay.js` and `server/healthcheck.js` have no counterpart here and are not
296
+ carried at all. Of the five files that do have one, the content analyser and the validation
297
+ layer are substantially Jason's work, and most of his `tools.js` and `index.js` survives here
298
+ as well — the routing patterns, the validators, and the error classes are his. What changed is
299
+ described below; what did not change is his.
300
+
301
+ **What diverged from upstream**
302
+
303
+ - **Storage:** SQLite on a RAM disk → MongoDB. Upstream's `ram_disk_manager` and
304
+ `server/decay.js` temporal-decay engine are not carried forward, and the SQLite/RAM code
305
+ paths have been removed rather than left dormant.
306
+ - **Transport:** stdio only → stdio *and* streamable HTTP with OAuth, from one source tree.
307
+ - **Tools:** 7 → 29. Three are carried from upstream (`recall`, `query_layer`,
308
+ `save_to_layer`); `remember`, `get_stats`, `get_echo_stats` and `get_status` are dropped as
309
+ duplicates or folded into `emet_status`. Twenty-six have no upstream equivalent — the document store (`read_doc`/
310
+ `read_doc_history`/`write_doc`/`list_docs`/`patch_doc`/`retire_doc`/`rename_doc`/`restore_doc`/
311
+ `validate_doc`/`accept_nonconformance`), the transcript tier (`save_transcript`/
312
+ `revise_transcript`/`query_transcripts` — which also reads the pre-2026-07 exchange archive),
313
+ `semantic_recall`, `revise_memory`, `corpus_recall`, `emet_gaps`, `emet_floor`, the setup and
314
+ session family (`emet_initialize`, `emet_status`, `emet_setup_complete`, `emet_session_open`,
315
+ `emet_transcript_append`, `emet_session_close`), and the two access doors
316
+ (`emet_invite_create`, `emet_member_access_create`).
317
+ - **The write path:** upstream validates and inserts. Here every layer write,
318
+ transcript write and document write passes through one door that scrubs secrets, decides the
319
+ source tag, fills missing accountability fields with visible markers, digests content and
320
+ metadata into their own columns, reads the row back, and returns one receipt shape. A test
321
+ fails the build on any second door. See *The door* below.
322
+ - **Metadata vocabulary:** upstream's whitelist relocated any field it did not know under
323
+ `custom`. EMET's own fields — `supersedes`, `superseded_by`, `superseded_at`, `corrects`,
324
+ `corrected_by`, `valid_until`, `valid_from`, `routing`, `assertion_origin`, and the marker booleans — are first-class, and
325
+ anything that is still relocated is named in the receipt.
326
+ - **Charter as data:** `layers.js` carries, beside each layer's prose, its numeric importance
327
+ band, whether it may be revised, whether provenance is required, its aliases and its expected
328
+ share, and the write path reads those rather than hardcoding them.
329
+
330
+ No claim is made that layered memory, entry supersession, or a split curated/verbatim store
331
+ originate here. Comparable ideas ship in other projects. EMET's aim is that the substrate is
332
+ not something you have to think about.
333
+
334
+ ---
335
+
336
+ ## Install
337
+
338
+ **Never deployed anything? Let your AI do it: [docs/GUIDED-SETUP.md](docs/GUIDED-SETUP.md).**
339
+ You sign in to two services when asked and paste one URL; the assistant installs, configures,
340
+ deploys and verifies the rest. It needs a shell — Claude Code and Cowork have one; Claude
341
+ Desktop gets one from an MCP server such as Desktop Commander (one command, in the guide).
342
+
343
+ **Full walkthrough: [docs/SETUP.md](docs/SETUP.md)** — Atlas, both transports, and the
344
+ first-run interview. **Deploying remotely: [docs/DEPLOY.md](docs/DEPLOY.md)**.
345
+ **Making your assistant actually use it: [docs/BOOTSTRAP.md](docs/BOOTSTRAP.md)**.
346
+ **Running bots alongside your assistant** — registering their tags, their own handoffs, the
347
+ session loop: [docs/BOTS.md](docs/BOTS.md). For designing a bot team, see
348
+ [docs/SUBKEYS.md](docs/SUBKEYS.md).
349
+
350
+ **Continuous chats.** A bot in a long-running chat saves its transcripts to EMET
351
+ word for word at least nightly (a checkpoint), under the open session or a daily
352
+ `YYYY-MM-DD_chat-daily` session. Prefer event triggers over time-based polling.
353
+ When polling is unavoidable, use the longest interval the job allows. Never
354
+ reconstruct turns. Mark gaps as gaps. Session close cites those saved parts.
355
+ Context compression erases older turns, so a close days later cannot invent the
356
+ transcript.
357
+
358
+ Short version:
359
+
360
+ ```
361
+ npm install
362
+ cp .env.example .env # fill in MONGODB_URI and MONGODB_DB
363
+ npm start
364
+ ```
365
+
366
+ Requires Node 22+ and a MongoDB connection. `.env` is gitignored — never commit it.
367
+
368
+ **Local (stdio)** — as an MCP server entry:
369
+
370
+ ```json
371
+ { "mcpServers": { "emet": { "command": "node", "args": ["/path/to/emet/src/index.js"] } } }
372
+ ```
373
+
374
+ **Remote (HTTP)** — `npm run start:http`, or deploy the included `Dockerfile` to any container
375
+ host. Set `PORT`, `PUBLIC_URL` (the OAuth issuer — it must match the deployed URL exactly) and
376
+ `EMET_OAUTH_PASSPHRASE`. Then add `https://<your-host>/mcp` as a custom connector in each
377
+ client. See [docs/DEPLOY.md](docs/DEPLOY.md).
378
+
379
+ Give every host its own source tag. Over HTTP, document, transcript, and memory writes use the
380
+ tag bound at approval; the caller does not pass `source` on those writes. A local stdio host
381
+ still passes an exact registered tag as `source`. `EMET_SOURCE_TAGS` registers the tags. Leave
382
+ **`EMET_SOURCE_TAG`** unset on a server several hosts share, since a default tag makes every
383
+ host look the same; set it only where one host uses the server (a local stdio entry, for
384
+ example). Every write records the host that produced it; a shared default tag is ambiguity
385
+ rather than attribution.
386
+
387
+ ### First run
388
+
389
+ Say **"initialize"**. That is the whole interface.
390
+
391
+ `emet_initialize` is the front door and the only thing anyone needs to know about. On a new
392
+ store it returns a short interview — what the assistant should be called, how it should carry
393
+ itself, your name, what you are using this for, and whether more than one bot or AI will use
394
+ this memory — which your assistant runs conversationally,
395
+ then records with `emet_setup_complete`, which also creates the owner person. Whether more than one bot or AI will use this memory is optional; leaving it out leaves delegated client sub-keys off. A yes writes them only when this server reads `.env`, and they take effect on restart. A hosted server refuses and tells you the three variables to set, then redeploy. See [docs/SUBKEYS.md](docs/SUBKEYS.md). On an established store the same call returns the
396
+ assistant's identity, your profile, and where things were left, so the session opens already
397
+ knowing you. A read-scoped `emet_initialize` writes nothing; a write-scoped one may provision a missing board. First-time setup is the owner's:
398
+ `emet_setup_complete` is refused under a bot tag, and on a store that is already set up it is
399
+ refused unless `reissue: true` is passed on the owner's explicit word.
400
+
401
+ Requiring a setup command would have been backwards: the one time you need it is the first
402
+ session, before you have read anything. So the branch happens inside the tool, and you never
403
+ have to know which case you were in.
404
+
405
+ Connecting the server gives your assistant the ability to remember; it does not make it use
406
+ that ability unprompted. [docs/BOOTSTRAP.md](docs/BOOTSTRAP.md) has a short block to paste
407
+ into your project instructions, and explains what still works on hosts that have no
408
+ instructions field at all.
409
+
410
+ Two behaviours worth knowing. Setup **refuses to finish on partial answers** rather than
411
+ inventing an identity for itself. And if the store cannot be reached, initialize says so
412
+ plainly instead of proceeding as though memory were available — fabricated continuity is
413
+ worse than an honest gap.
414
+
415
+ ### Every session after that: how to start, how to end
416
+
417
+ Memory is only as good as the two moments at the edges of a conversation. Nothing your
418
+ assistant learned survives unless it was written to the store, and the moment to write it
419
+ arrives without warning — a closed tab, a context limit, a phone call. So every session has
420
+ one word at the start and one at the end.
421
+
422
+ **Start: say "initialize".** Your assistant calls `emet_initialize`, which returns who it is,
423
+ who you are, where things were left and the operating disciplines, and picks up from there.
424
+ Before it asks you anything, it names your items from the handoff and state documents: anything
425
+ left half-finished, what is dated in the next seven days, each open item you own with its next
426
+ action, and whatever was left undecided. An item not named at the start of a session is easy
427
+ to lose, so a count or a bare "where do you want to start?" is not treated as a startup.
428
+ Do this in every new conversation, on every device — the first message, before the question
429
+ you actually came to ask. A session that skips it is a session that answers from guesswork
430
+ and saves nothing.
431
+
432
+ **End: say "wrap up"** (or goodbye, or that you are opening a new session). Your assistant
433
+ then writes four things — the handoff, the state document, an episodic entry for the session,
434
+ and the **verbatim transcript** — and calls `emet_session_close`, which checks that each one
435
+ actually landed and returns `state: "complete"`. **Wait for that word before you close the
436
+ window.** If it comes back anything else, something did not save, and the assistant will say
437
+ which.
438
+
439
+ Why both, every time: the layers are curated meaning and the transcript is the word-for-word
440
+ record; the next session reads the first and can reach into the second by timestamp. Close
441
+ without the ritual and the next session inherits a hole — it cannot see what happened, only
442
+ that something did, and `emet_gaps` will count it. Start without it and the assistant is a
443
+ stranger with your tools. The instruction block in [docs/BOOTSTRAP.md](docs/BOOTSTRAP.md)
444
+ makes both of these automatic on hosts that take standing instructions; on hosts that do not,
445
+ the two words are what you supply.
446
+
447
+ ---
448
+
449
+ ## Design notes
450
+
451
+ ### The six layers
452
+
453
+ Memory is split by *what kind of thing* an entry is, not by topic. Each entry carries an
454
+ importance score and optional temporal validity.
455
+
456
+ | Layer | Holds | Importance | Written |
457
+ |---|---|---|---|
458
+ | **identity** | Who the assistant is and who it serves. Durable self-definition and the user's standing requirements. | 0.9 – 1.0 | Rarely, by design |
459
+ | **semantic** | Facts that stay true regardless of when they were learned. People, systems, definitions, established conclusions. | 0.6 – 0.9 | When a durable fact is established or corrected |
460
+ | **episodic** | What happened, and when. Session records, problems solved, progress through long work. | 0.4 – 0.7 | At session close and at milestones |
461
+ | **procedural** | How something is done, and decisions that govern future action — *with the reasoning*. | 0.5 – 0.8 | When a decision constrains later work |
462
+ | **meta** | Everything visible only from **outside** the exchange: how the memory system behaves, and how the assistant behaves toward the user. | 0.6 – 0.9 | On retrieval failures and wins, protocol drift, and corrections to how the user is worked with |
463
+ | **working** | Short-lived context, in-flight state, notes for whoever picks the work up. | 0.3 – 0.6 | When parking state between sessions |
464
+
465
+ The distinction that does most of the work: **episodic, semantic and procedural are about the
466
+ world** — what happened, what is true, how something is done. **Meta is about the system** —
467
+ looking back at the exchange from outside it. Material belonging in meta feels like nothing else
468
+ fits it, which is precisely why it gets discarded when meta has no definition.
469
+
470
+ ### Every layer works from the first install
471
+
472
+ Creating six collections is not the same as having six layers. A layer with no stated charter has
473
+ no answer to *"does this belong here?"*, so nothing routes to it — and an empty collection throws
474
+ no error and fails no test, so it can stay empty indefinitely without anyone noticing.
475
+
476
+ In the deployment this design came from, **the meta layer sat nearly unused for about a year**.
477
+ The collection existed from day one and the tools accepted writes to it; it was simply never the
478
+ obvious place to put anything. When it was finally given a definition it filled immediately —
479
+ which is the proof that the material had been there all along and was being thrown away.
480
+
481
+ Two things in this repo exist because of that:
482
+
483
+ - **`src/layers.js` ships the charter for all six layers in code**, and `emet_initialize` returns
484
+ it — on a brand-new store as well as an established one. Same reasoning as the operating
485
+ disciplines: documentation is read by people, and this has to reach the model at the start of
486
+ every session. What belongs, what does *not* belong, when to write, when to read, and what the
487
+ layer's absence looks like.
488
+ - **`assessLayerHealth()` reports a layer that has gone dark.** A store cannot know what *should*
489
+ have been written, but it can see that one layer took almost nothing while the others filled,
490
+ and say so in the first payload of the next session. Run against the real history above, it
491
+ flags meta at 10 entries / 1.8% of the store — and goes quiet once the layer is in genuine use.
492
+ It reports; it never blocks, and it stays silent on a store too new to have a pattern.
493
+
494
+
495
+ **Two tiers, deliberately independent.** Layer entries are curated meaning; a separate transcript
496
+ store holds the word-for-word conversation. When a layer entry is relevant but thin, the
497
+ transcript holds the depth that was compressed out of it. Summarising at transcript-write time
498
+ collapses the two tiers into one and destroys the second line of inquiry — so the transcript path
499
+ stores exactly what it is given. If a payload will not transmit, split it across parts; do not
500
+ compress it.
501
+
502
+ **How tier 1 reaches tier 2 — two links, one precedence rule.**
503
+
504
+ An entry records the transcript span it came from: `session_id` plus a range of exchange indices,
505
+ written at save time as `derived_from`. Entries consolidated from other entries carry
506
+ `{entries: [id, ...]}` instead, because an entry with no transcript origin must be able to say so
507
+ rather than carry a nearest-timestamp guess dressed as a citation. Both are stored as top-level
508
+ indexed columns rather than inside the metadata subdocument, so the reverse question — every
509
+ entry drawn from this transcript — is entirely a query.
510
+
511
+ Every entry also has a timestamp, so a window around it can always be searched. The two are not
512
+ redundant copies: the pointer is exact but conditional, the timestamp universal but approximate —
513
+ it depends on choosing the window, degrades when sessions overlap, and when it is wrong it returns
514
+ the *neighbouring* span rather than failing.
515
+
516
+ **Precedence is what keeps them from conflicting: the pointer wins whenever present, the
517
+ timestamp is the fallback when it is null.** Never averaged, never combined, and the timestamp is
518
+ never preferred merely because it returned something. `resolveProvenance()` applies the rule so no
519
+ caller has to decide it.
520
+
521
+ **Keeping both buys something neither gives alone — they can check each other.** If an entry's
522
+ pointer names a session whose span sits nowhere near that entry's own timestamp, something is
523
+ wrong: a mis-written pointer, a field copied from another entry, an entry attributed to the wrong
524
+ session. `crossCheckProvenance()` reports the disagreement. Two independently derived answers that
525
+ can disagree is redundancy in the engineering sense; one link that can only be trusted is not.
526
+
527
+ **The reverse direction — *what did we take from this session?* — is not a second structure.**
528
+ `source_session_id` is indexed, so it is the same data read the other way. A stored reverse index
529
+ would be a mirror of data that already exists, and mirrors are how forks are born.
530
+
531
+ **Entries written before pointers existed keep a null pointer and resolve by timestamp, and that
532
+ remains correct for them — superseded, not wrong.** They are not backfilled: the only way to
533
+ invent a pointer for them is to infer it from the timestamp, which is exactly the weaker method
534
+ the pointer replaces. Old records are not rewritten to look like new ones. Note also that an empty
535
+ timestamp-window search is not proof that nothing was recorded — a period may hold no transcript
536
+ at all.
537
+
538
+ **Supersession, not mutation.** `revise_memory` writes a *new* entry carrying `supersedes` and
539
+ marks the parent with `superseded_by` and `superseded_at`. Nothing is edited or deleted, and the
540
+ parent is not hidden. Recall ranks by how many of the query's words match, then by importance.
541
+ Current entries come first in every order, including time order. The store applies that same order before it keeps the window, so a stronger match is not dropped for a lower importance or an older time. A word matches as written, so `$gt` is that text, and capital and lower case match, including `École` and `école`. `İ` matches only itself. Full-width letters fold only with their full-width counterparts. `ẞ` and `ß` fold together. The store and the page use that same match. Time order is what you get
542
+ from `order: 'latest'`. A standalone "latest", "newest", or "most recent", together with other
543
+ words, reorders those matches and does not change which rows match. A hyphenated word such as
544
+ `latest-budget`, "not the latest", and a bare "latest" are ordinary keywords.
545
+ A superseded or expired entry stays in the same result, marked
546
+ with its end date and the id of what replaced it. Pass `history: true` when current matches fill
547
+ the limit and you still need the ended ones. Episodic entries
548
+ are immutable — events do not get revised.
549
+
550
+ **Corrections, not supersession, for partial fixes.** When a later entry corrects only part of an
551
+ earlier one (one item of an episode, say), superseding puts an end date on the whole parent,
552
+ including the parts that still stand. The parent stays readable, marked ended. Instead the new entry carries `metadata.corrects: <parent id>` (same layer; a missing
553
+ parent is refused before anything is written), and the parent is marked: its `corrected_by` list
554
+ gains the new id, its content, digest and liveness are untouched, and its metadata digest is
555
+ re-attested in the same write. Reads return `corrected_by`, so a reader of the parent sees that a
556
+ correction exists. `corrected_by` is a door marker; a caller cannot set it. Entries corrected
557
+ before this existed (e.g. episodic ref 604 correcting ref 603) carry no link; the fix is a new correction
558
+ entry naming corrects: 603, not an edit or a backfill.
559
+
560
+ **Verified writes - `src/aleph.js`.** Every write is read back and compared before it is
561
+ reported as saved. Document writes archive the prior version, then confirm both the sha256 and
562
+ the version; layer writes confirm the stored content at the id that was allocated. The id is
563
+ checked by content rather than existence on purpose: ids come from a counters collection, and a
564
+ counter read from the wrong collection restarts at 1 and silently overwrites real entries -
565
+ which looks exactly like a successful save.
566
+
567
+ A verification failure raises `AlephError`, deliberately distinct from a database error. A
568
+ database error means the store could not be reached and the write probably did not happen. An
569
+ `AlephError` means the call **succeeded** and the store does not contain what it should. That is
570
+ the more serious of the two and the one that must never be retried blindly, so it is never
571
+ flattened into "the save failed".
572
+
573
+ The aleph answers two questions, at two moments. **At the write: was it recorded accurately?**
574
+ The read-back above, before any receipt. **At every later read: is it still what was
575
+ recorded?** Each row carries its content and metadata digests, an HMAC signature over both
576
+ (when `EMET_SIGNING_KEY` is set; the key stays on the service), and a link in its layer's hash chain (each entry's chain
577
+ digest covers its own content digest and its predecessor's). Every read recomputes the digests
578
+ and reports `integrity`; `emet_gaps` walks the whole chain for any host and counts a row altered
579
+ after writing (`broken_chain`) or removed from mid-chain (`chain_discontinuities`). Expected
580
+ count for both: zero.
581
+
582
+ ### The door
583
+
584
+ Truth integrity, accountability and validation are meant to be a guaranteed property of the
585
+ store, not an option a model can decline to use. A file a model can be told about is a file it
586
+ can ignore. So the guarantee comes from a property, not a module: **exactly one way to write
587
+ each store, with the checks inside it.** `saveMemory` for the six layers, `writeDoc` for
588
+ documents (`patch_doc` delegates to it), `saveTranscript`
589
+ and `reviseTranscript` for the verbatim tier. `tests/write-path.test.js` scans `src/` for every
590
+ MongoDB write verb and fails the build on any that sits outside those functions — it found and
591
+ removed a second document door in `setup.js`.
592
+
593
+ Inside the door, in order:
594
+
595
+ - **Secrets are masked before anything is stored or hashed** (`src/redact.js`): the literal
596
+ values of secret-bearing environment variables (including `EMET_SUBKEY_PEPPER`), credentials inside URIs, bearer tokens, JWTs,
597
+ and `key: value` pairs whose key names a secret become `*****`. The mask is visible on
598
+ purpose, the receipt's `redacted` reports the number of spans **this write changed** (text
599
+ already carrying the mask is left alone and not counted, so a read-modify-write of a document
600
+ reports zero for the masks it already held), and the same scrubber wraps every rendered log line.
601
+ - **The source tag is decided here**, not by the caller over HTTP. Document, transcript, and
602
+ memory writes use the tag bound at approval. On stdio it is the caller's exact registered
603
+ tag, else the environment default marked `source_defaulted: true`, else `null` marked
604
+ `source_missing: true`.
605
+ - **Every missing accountability field becomes a visible marker** rather than an absence —
606
+ `provenance_missing`, `assertion_origin_missing`, `importance_defaulted`, `source_unregistered`
607
+ — because absence cannot be queried and "missing" can. A missing field is flagged, not
608
+ refused, with one exception (2026-09-29): a **semantic or procedural** entry with no source
609
+ is refused, because a durable claim that cannot point to where it was said is exactly what
610
+ the record exists to prevent. The door stamps the source from the session's transcript
611
+ position when the caller names its session. Importance defaults to the layer's charter midpoint;
612
+ a value outside the band is kept and warned about. The door never infers an assertion's origin.
613
+ - **The routing decision is kept** in `metadata.routing` — whether the layer was chosen by the
614
+ caller or fell back, and what both routers thought — so routing can be measured later instead
615
+ of remembered. **An unlabelled write is never placed by a router:** it lands in `working`,
616
+ flagged `routing.method: 'fallback'`, with a receipt warning. Naming the layer is the norm.
617
+ - **Content and metadata are digested into their own columns** (`sha256`, `metadata_sha256`,
618
+ the latter over canonical JSON), read back and compared before the write is reported. Every
619
+ read path — `recall`, `query_layer`, `semantic_recall` — recomputes both on the way out and
620
+ attaches `integrity` and `metadata_integrity`: `true`, `false` (the row is returned and
621
+ flagged, never dropped), or `'unattested'` for rows written before the column existed.
622
+ Supersession and the correction mark are the only sanctioned mutations of stored metadata, and
623
+ both re-attest the parent.
624
+ - **One receipt shape on both transports:** `layer, id, timestamp, verified, sha256,
625
+ digest_stored, metadata_sha256, metadata_digest_stored, redacted, provenance, source,
626
+ assertion_origin, importance, routing, warnings[]` plus whichever markers apply.
627
+
628
+ And the record's honesty about its own holes is a query: **`emet_gaps`** counts, with the ids
629
+ behind each count, unsourced assertions, unattributed inferences, ambiguous attribution by tag
630
+ and by session, self-routed writes, unattested rows, digest mismatches, pointer disagreements
631
+ against the transcript store, broken supersession, broken correction links, expired-but-live rows, consolidation
632
+ candidates, and redacted spans by source. It identifies and never fills — where a source is
633
+ knowable the fix is `revise_memory` with a real pointer; where it is not, the gap stands as a
634
+ fact and the count stops growing from that date. `emet_initialize` carries the counts beside
635
+ `layer_health`.
636
+
637
+ What can be guaranteed this way: every write attested, attributed, scrubbed, and every absence
638
+ visible. What cannot be guaranteed without inviting fabrication: every write complete. The
639
+ store does not promise the latter.
640
+
641
+ ### Guardrails & Monorails
642
+
643
+ The door guards each write. **Guardrails & Monorails** guards the order of a session's writes.
644
+ **Guardrails** (the floor, the character, the served instructions) bound a field where the AI
645
+ moves freely and does its judgment work. **The monorail** (the session graph,
646
+ `src/session-graph.js`) carries it from one field to the next without asking it to choose:
647
+ open, work, record, board, handoff, close. The car can't derail or leave the beam. Where the
648
+ flow must fork, a **switch** at the junction (a small logic unit in EMET's code, tripped only by
649
+ its condition, like a transistor; no hosted model, no API) routes the session to the right
650
+ guardrail space. The track throws the switch, never the AI. The stations are where the
651
+ processing happens, and each station's door opens only when every stop before it has been made.
652
+ Switches are designed, not yet built. The server checks the track
653
+ on every governed write, so it is the same for any AI on any device. No host hook is needed.
654
+ Every skipped step is refused, and the refusal says how to pass. The full practice, the gates and the honest limit
655
+ are in [`docs/GUARDRAILS-AND-MONORAILS.md`](docs/GUARDRAILS-AND-MONORAILS.md).
656
+
657
+ ---
658
+
659
+ ## Tools
660
+
661
+ Thirty tools are registered with sub-keys off. With `EMET_SUBKEYS=1`, four more (`emet_subkey_request`, `emet_subkey_claim`, `emet_subkey_revoke`, `emet_subkey_end_session`) and the `subkey` and `session_handle` arguments. The names below are every tool the registry registers, both sets together.
662
+
663
+ <!-- emet-tools -->
664
+ ```
665
+ accept_nonconformance
666
+ corpus_recall
667
+ emet_client_approve
668
+ emet_floor
669
+ emet_gaps
670
+ emet_initialize
671
+ emet_invite_create
672
+ emet_member_access_create
673
+ emet_session_close
674
+ emet_session_open
675
+ emet_setup_complete
676
+ emet_status
677
+ emet_subkey_claim
678
+ emet_subkey_end_session
679
+ emet_subkey_request
680
+ emet_subkey_revoke
681
+ emet_transcript_append
682
+ list_docs
683
+ patch_doc
684
+ query_layer
685
+ query_transcripts
686
+ read_doc
687
+ read_doc_history
688
+ recall
689
+ rename_doc
690
+ restore_doc
691
+ retire_doc
692
+ revise_memory
693
+ revise_transcript
694
+ save_to_layer
695
+ save_transcript
696
+ semantic_recall
697
+ validate_doc
698
+ write_doc
699
+ ```
700
+ <!-- /emet-tools -->
701
+
702
+ **Retained revisions** — every document write archives the prior copy; `read_doc_history` lists
703
+ what is retained for a document and returns any retained revision in full, re-hashed against its
704
+ stored digest. Retention that nothing could read back was a promise without a witness.
705
+
706
+ **One reader for the second tier** — `query_transcripts` reads the live transcript store and the
707
+ legacy exchange archive (rows written by earlier hosts, before the transcript store existed)
708
+ together, in one session shape, each result naming the collection it came from. Nothing is
709
+ migrated between them.
710
+
711
+ **Document control** — `validate_doc` checks a controlled document against the template that
712
+ governs it, and the same check runs inside the write path on every document write, where no
713
+ caller can skip it. A record that fails is **marked, never rejected**: it lands, carries a
714
+ nonconformance marker naming what failed, and stays counted at session start until it is
715
+ corrected by a later revision or dispositioned with `accept_nonconformance`. The checks are form
716
+ and continuity — sections present, in order and filled; no placeholder left unfilled; every item
717
+ on the previous revision still present or carrying a disposition. Whether an answer is *true* is
718
+ not something a machine can check, and this one does not pretend to.
719
+
720
+ **Session start** — `emet_initialize` (start here): fetch every page it names, or pass `page: "all"`
721
+ on a host with no size limit; then `emet_session_open` with that startup's `load_id`. Session ids are
722
+ dated in the local time zone (`EMET_TIMEZONE`, default `America/New_York`); a closed session is
723
+ never reopened, so a slug whose id is already closed opens as `<id>-2`, `-3`, and the result says so.
724
+ After a complete close a session takes exactly one more append, its closing exchange, within
725
+ `EMET_CLOSE_GRACE_MINUTES` (default 5, at most 60) of the close, written into the session's last
726
+ part, and is then locked; a retry of it writes nothing. Corrections with `revise_transcript` stay
727
+ allowed but must keep exactly the same exchange numbers.
728
+
729
+ **During the session** — `emet_transcript_append` after every reply, one exchange at a time.
730
+ Every write tool also takes `session_id` and `exchange`, and returns `session_graph.next`, the
731
+ next stop on the monorail ([Guardrails & Monorails](docs/GUARDRAILS-AND-MONORAILS.md)).
732
+
733
+ **Session end** — `emet_session_close`. Checks whether the session's work actually reached the
734
+ store — transcript saved and intact, continuity documents updated, an episodic entry written —
735
+ and reports exactly what is missing. It does **not** write the artifacts: only the model holds the
736
+ session's content, and a tool that generated a plausible-looking summary would be manufacturing
737
+ the false continuity the disciplines exist to prevent. `emet_initialize` returns the same protocol
738
+ at session *start*, which is the only moment early enough to matter.
739
+
740
+ **Setup internals and diagnostics** — `emet_status` (environment, layers, health; `probe: true` writes and reads back a probe) · `emet_setup_complete`
741
+
742
+ ### Approvals
743
+
744
+ Every tool declares MCP annotations — `readOnlyHint`, `destructiveHint`, `idempotentHint`,
745
+ `openWorldHint` — in one table in `src/annotations.js`, which is also the complete statement of
746
+ what EMET does to a store. **11 of the 30 tools cannot modify anything**; four can replace the
747
+ visible content of a document; `openWorldHint` is false everywhere. The tools reach this
748
+ server's own database and nothing else: no filesystem and no shell. The sign-in server fetches
749
+ a client's metadata document, and only from a public address it has already checked.
750
+
751
+ A server that declares nothing is treated as if every call might do anything, so a host that asks
752
+ before acting asks before *every* call, including the reads that make up most of a session. These
753
+ hints let a host stop asking about reads without being told to trust writes. They are hints, not
754
+ enforcement: what is actually auto-approved is a host-side setting. See
755
+ [docs/PERMISSIONS.md](docs/PERMISSIONS.md).
756
+
757
+ ---
758
+
759
+ ## Tests
760
+
761
+ ```
762
+ npm test
763
+ ```
764
+
765
+ 1,852 checks across 64 suites (the PASS lines `npm test` prints, counted 2026-10-07). No framework, no database, no
766
+ credentials — each suite is a plain node script on one shared harness (`tests/harness.js`), and
767
+ the runner discovers every `*.test.js`, so `npm test` runs on a fresh clone. Per-suite counts
768
+ are not kept here because they go stale; `npm test` prints them.
769
+
770
+ `validation.test.js` and `content_analyzer.test.js` began as **upstream's own
771
+ suites**, ported unchanged apart from the import path; every one of upstream's assertions still
772
+ stands, and the additions cover EMET's metadata vocabulary and the shared layer list. They are
773
+ the original author's assertions about the original author's code, and they are kept passing.
774
+
775
+ The rest cover this tree. Some are fixtures rather than unit tests — they read the source and
776
+ fail on a class of drift: `write-path.test.js` allows no MongoDB write outside the
777
+ sanctioned doors, requires each door to scrub before it hashes, and requires that **nothing in
778
+ the tree removes from a layer, document or transcript collection**; `dispatch.test.js`
779
+ requires the declared tool list and both transports' dispatch switches to be the same set, and
780
+ that a list response can carry a caller-facing warning; `annotations.test.js` requires
781
+ every tool to declare its approval hints; `atlas-role.test.js` requires the database role to
782
+ grant no remove on the record. The others test behaviour — among them `aleph.test.js`
783
+ (read-back, digests, signatures, the chain walk), `session-graph.test.js` (the monorail's
784
+ gates and lab results), `receipt.test.js`, `provenance-stamp.test.js`, `session.test.js`,
785
+ `gaps.test.js` and `redact.test.js`. Their failure
786
+ cases matter more than their success cases — a verifier that passes something it should have
787
+ caught converts an undetected problem into a confident assurance. `layers.test.js` asserts
788
+ against the store shape that produced the year-long dormant layer, so the check is tested
789
+ against the failure it was written for rather than a hypothetical one.
790
+
791
+ Upstream's `index.test.js`, `decay.test.js` and `integration.test.js` are **not** carried: they
792
+ target the SQLite backend and the temporal decay engine, neither of which exists here. Running
793
+ them would produce failures that say nothing about this code.
794
+
795
+ ### Benchmarks
796
+
797
+ EMET's implementation language was evaluated rather than assumed. The same specification was
798
+ implemented in **JavaScript, Python, Go and Rust**, deployed identically, and run against live
799
+ Atlas — with a digest equivalence check proving all four did the same work before any timing
800
+ was read. **JavaScript won**, on serial in-process cost, on throughput under concurrency, and on
801
+ standards provenance as the only candidate carrying an international language standard
802
+ (ECMA-262 and ISO/IEC 16262).
803
+
804
+ The exercise's most useful output was not a language choice but a function to optimise: the
805
+ write-path scrubber was paying nearly full cost on documents containing no secrets, and fixing
806
+ that saved roughly thirty times what the best available language change could have.
807
+
808
+ Method, full results, and an explicit account of what was *not* measured:
809
+ [docs/BENCHMARKS.md](docs/BENCHMARKS.md). `scripts/redact-bench.mjs` measures the scrubber
810
+ alone — run it before and after any change to `src/redact.js`.
811
+
812
+ ---
813
+
814
+ ## Known gaps
815
+
816
+ Honest list, because the point of this project is that it behaves predictably:
817
+
818
+ - Rows written before 2026-09-05 carry no content digest and rows written before 2026-09-07
819
+ carry no metadata digest; reads report them as `'unattested'`, and they are not backfilled —
820
+ a digest computed today would attest to today's copy, not to the write. `emet_gaps` counts
821
+ the boundary.
822
+ - Entries written before provenance existed keep a null pointer and resolve by timestamp
823
+ window; `emet_gaps` counts them as unsourced. They are not backfilled, for the reason given
824
+ under *Provenance*.
825
+ - Two content routers are carried from upstream (a substring keyword router and a content
826
+ analyser). Measured against a 1,175-row corpus both agree with the stored layer well under
827
+ half the time (41% and 36%), so neither places entries: an unlabelled write lands in
828
+ `working`, flagged, with both opinions recorded in `routing` (`keyword_layer`,
829
+ `analyzer_layer`). Retuning them into something fit to route is possible with those
830
+ numbers as the baseline; it is not scheduled.
831
+ - `consolidation_candidates` in `emet_gaps` is tag overlap, not semantic similarity.
832
+ - The monorail (session graph) refuses every skipped step (enforce only since 2026-10-03; there
833
+ is no report mode or off switch). Its turn coverage is only as strong as the exchange numbers hosts report, and it never sees
834
+ the reply text. See [Guardrails & Monorails](docs/GUARDRAILS-AND-MONORAILS.md).
835
+ - The scheduled weekly intake writes without opening a session, so the session-order check
836
+ refuses those writes (gate `no_session`) until the routine opens a session with its startup's
837
+ `load_id` and passes the `session_id`. Its old source tag, `cloud-scheduled-intake`, was retired on
838
+ 2026-10-02. A `cloud-scheduled-` prefix is not a registered tag, and those writes are refused.
839
+ The weekly intake runs under `intake-bot`. `emet_gaps` keeps counting the rows already written
840
+ under the old tag as ambiguous attribution instead of rewriting history.
841
+ - `save_transcript`'s description does not state that it stores content verbatim; the
842
+ implementation does.
843
+ - No export / import / restore-and-compare yet.
844
+ - The document store is not prefixed, so one database holds one instance's identity. Run
845
+ separate databases rather than separate prefixes if you want separate memories.
846
+ - Dynamic Client Registration is still served beside Client ID Metadata Documents; the MCP
847
+ specification's deprecation window closes around 2027-07. It is scheduled to come off on
848
+ 2027-06-01.
849
+ - Delegated client sub-keys stay off unless setup is told that more than one bot or AI will use the memory. A hosted server does not write a pepper into its container. A stamp shows that an approved key was presented, not which program held it. See [docs/SUBKEYS.md](docs/SUBKEYS.md).
850
+
851
+ ---
852
+
853
+ ## Origin
854
+
855
+ EMET was built by Dreamforge Studio LLC from a neurodivergent user’s own need for an AI assistant that remembers. The word coined for that is neurendipity: neurodivergent and serendipity.
856
+
857
+ It is designed first for neurodivergent users, and it works for anyone.
858
+
859
+ ---
860
+
861
+ ## License and access
862
+
863
+ EMET is free and open source under the MIT License (see LICENSE). No account, key or payment is required to run it.
864
+
865
+ Authorship of the commit history, including a correction recorded as an addition
866
+ rather than a rewrite: `docs/AUTHORSHIP.md`.
867
+
868
+ MIT. See `LICENSE` — both the original C.I.P.S. LLC notice and the Dreamforge Studio LLC notice
869
+ must travel with any copy.