memhtml 0.3.0 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,28 +1,19 @@
1
1
  # memhtml
2
2
 
3
- [![check](https://github.com/memhtml/memhtml/actions/workflows/check.yml/badge.svg)](https://github.com/memhtml/memhtml/actions/workflows/check.yml)
4
- [![security](https://github.com/memhtml/memhtml/actions/workflows/security.yml/badge.svg)](https://github.com/memhtml/memhtml/actions/workflows/security.yml)
5
- [![OpenSSF Scorecard](https://api.scorecard.dev/projects/github.com/memhtml/memhtml/badge)](https://scorecard.dev/viewer/?uri=github.com/memhtml/memhtml)
6
- [![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
3
+ [![check](https://github.com/memhtml/memhtml/actions/workflows/check.yml/badge.svg)](https://github.com/memhtml/memhtml/actions/workflows/check.yml) [![security](https://github.com/memhtml/memhtml/actions/workflows/security.yml/badge.svg)](https://github.com/memhtml/memhtml/actions/workflows/security.yml) [![OpenSSF Scorecard](https://api.scorecard.dev/projects/github.com/memhtml/memhtml/badge)](https://scorecard.dev/viewer/?uri=github.com/memhtml/memhtml) [![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
7
4
 
8
- memhtml stores an agent's long-term memory as a git repository of semantic HTML5 files, one fact per
9
- file. A rebuildable SQLite index sits over that tree, retrieval fuses four ranking arms, and a nightly
10
- curation pipeline commits its work to a branch a human reviews before it lands.
5
+ memhtml stores an agent's long-term memory as a git repository of semantic HTML5 files, one fact per file. A rebuildable SQLite index sits over that tree, retrieval fuses four ranking arms, and a nightly curation pipeline commits its work to a branch a human reviews before it lands.
11
6
 
12
7
  ## Install
13
8
 
14
- One package carries the whole system — the CLI, code mode, the sleep cycle, the trace indexer, and the
15
- MCP server — and installs two binaries, `memhtml` and `memhtml-mcp`. Node 24 or newer.
9
+ One package carries the whole system — the CLI, code mode, the sleep cycle, the trace indexer, and the MCP server — and installs two binaries, `memhtml` and `memhtml-mcp`. Node 24 or newer.
16
10
 
17
11
  ```bash
18
12
  npm i -g memhtml # or: pnpm add -g memhtml, bun add -g memhtml
19
13
  npx memhtml manifest # every command, flag, and error code, without installing anything
20
14
  ```
21
15
 
22
- Point it at a corpus with `MEMHTML_ROOT` (default `~/memhtml`), and at your transcripts with
23
- `MEMHTML_TRACE_ROOT` (default `~/.claude`). Reading and writing memories needs no credentials;
24
- embeddings and the sleep cycle's model calls use Bedrock through the default AWS credential chain, and
25
- `MEMHTML_EMBED=off` / `MEMHTML_LLM=off` turn both off.
16
+ Point it at a corpus with `MEMHTML_ROOT` (default `~/memhtml`), and at your transcripts with `MEMHTML_TRACE_ROOT` (default `~/.claude`). Reading and writing memories needs no credentials; embeddings and the sleep cycle's model calls use Bedrock through the default AWS credential chain, and `MEMHTML_EMBED=off` / `MEMHTML_LLM=off` turn both off.
26
17
 
27
18
  To register the MCP server with a client, the command is `memhtml-mcp` over stdio.
28
19
 
@@ -34,24 +25,15 @@ memhtml search "one writer many readers" # FTS + vector + recency + salienc
34
25
  memhtml serve mcp # the same store over stdio: 14 tools, 2 resources
35
26
  ```
36
27
 
37
- `memhtml manifest` (or a bare `memhtml`) answers with every command, flag, response type, and error code
38
- the binary accepts, and it answers on a machine with no repo, no database, and no credentials. Every
39
- command writes exactly one JSON envelope to stdout, logs go to stderr, and the exit code is 0 for
40
- success, 2 for a usage error, 1 for a runtime failure. `AGENTS.md` is generated from the same table that
41
- drives parsing, so the doc cannot drift from the binary.
28
+ `memhtml manifest` (or a bare `memhtml`) answers with every command, flag, response type, and error code the binary accepts, and it answers on a machine with no repo, no database, and no credentials. Every command writes exactly one JSON envelope to stdout, logs go to stderr, and the exit code is 0 for success, 2 for a usage error, 1 for a runtime failure. `AGENTS.md` is generated from the same table that drives parsing, so the doc cannot drift from the binary.
42
29
 
43
30
  ## The design in three sentences
44
31
 
45
- The git tree is the system of record, and `.memhtml/index.db` is a projection of it that can be deleted
46
- and rebuilt without loss. Anything that must survive `rm index.db` lives in the files, where authored
47
- links are `<link>` elements and metadata is `<meta>` elements, while re-derivable artifacts such as
48
- embeddings and mined edges live only in the index. Every removal is a `git mv` into `archive/<YYYY>/`
49
- that mirrors the original path, so the file stays in the tree and `git log --follow` reads straight
50
- through a memory's whole life.
32
+ The git tree is the system of record, and `.memhtml/index.db` is a projection of it that can be deleted and rebuilt without loss. Anything that must survive `rm index.db` lives in the files, where authored links are `<link>` elements and metadata is `<meta>` elements, while re-derivable artifacts such as embeddings and mined edges live only in the index. Every removal is a `git mv` into `archive/<YYYY>/` that mirrors the original path, so the file stays in the tree and `git log --follow` reads straight through a memory's whole life.
51
33
 
52
- Figure 1 draws that. It is built from monospace box characters, which a screen reader reads as noise, so
53
- skip to the paragraph below the figure: it says the same thing in words.
34
+ Figure 1 draws that. It is built from monospace box characters, which a screen reader reads as noise, so skip to the paragraph below the figure: it says the same thing in words.
54
35
 
36
+ <!-- dprint-ignore-start -->
55
37
  <!-- figure:system-topology -->
56
38
  ```text
57
39
  +--------+ +---------------+ +----------------+
@@ -101,41 +83,25 @@ skip to the paragraph below the figure: it says the same thing in words.
101
83
  +------------+
102
84
  ```
103
85
  <!-- /figure:system-topology -->
86
+ <!-- dprint-ignore-end -->
104
87
 
105
- **Figure 1: every write door lands in the git tree, and every read is served from projections of it.**
106
- Three doors reach in from outside: `memhtml write` and `memhtml apply`, the MCP server's 14 tools, and
107
- your own file tools. All three commit into one git tree, and they differ only in who owns the commit.
108
- Eviction moves a file to `archive/YYYY/` and leaves it in the tree. From the tree, a git-driven indexer
109
- derives `index.db`, which supplies three of the four ranking arms. The fourth, salience, comes from
110
- `state.db`, the one plane git cannot reproduce. A query enters the ranker at the bottom, RRF fuses the
111
- four arms, MMR diversifies the result, and ranked hits come out. On the docs site the same drawing puts
112
- into its borders what this caption has to spell out: a heavy border is a door, a double border is the
113
- system of record, a dashed border is a projection that can be deleted and rebuilt, and a cylinder is a
114
- database on disk.
88
+ **Figure 1: every write door lands in the git tree, and every read is served from projections of it.** Three doors reach in from outside: `memhtml write` and `memhtml apply`, the MCP server's 14 tools, and your own file tools. All three commit into one git tree, and they differ only in who owns the commit. Eviction moves a file to `archive/YYYY/` and leaves it in the tree. From the tree, a git-driven indexer derives `index.db`, which supplies three of the four ranking arms. The fourth, salience, comes from `state.db`, the one plane git cannot reproduce. A query enters the ranker at the bottom, RRF fuses the four arms, MMR diversifies the result, and ranked hits come out. On the docs site the same drawing puts into its borders what this caption has to spell out: a heavy border is a door, a double border is the system of record, a dashed border is a projection that can be deleted and rebuilt, and a cylinder is a database on disk.
115
89
 
116
90
  ## Why files
117
91
 
118
- A memory an agent can be trusted with has to be reviewable, diffable, and recoverable. git supplies all
119
- three directly, and that is why every write below lands as a commit:
92
+ A memory an agent can be trusted with has to be reviewable, diffable, and recoverable. git supplies all three directly, and that is why every write below lands as a commit:
120
93
 
121
- - A correction is a commit. `memhtml correct` writes the new file and archives the old one in one
122
- commit, so an interrupted run cannot leave two live memories contradicting each other.
123
- - A batch is a commit. `memhtml apply` (JSONL ops) and `memory_write_batch` (MCP) stage N files, make
124
- one commit, and reindex once. The batch is atomic by default, per-op results come back in input
125
- order, and a duplicate succeeds with `deduped: true` and the existing path.
126
- - A nightly curation run is a branch. `memhtml sleep run` walks fifteen phases and commits each one's
127
- work on its own, so a human reads the curation one phase-shaped diff at a time, and
128
- `memhtml sleep merge` fast-forwards `main` only after a quality gate that can refuse.
94
+ - A correction is a commit. `memhtml correct` writes the new file and archives the old one in one commit, so an interrupted run cannot leave two live memories contradicting each other.
95
+ - A batch is a commit. `memhtml apply` (JSONL ops) and `memory_write_batch` (MCP) stage N files, make one commit, and reindex once. The batch is atomic by default, per-op results come back in input order, and a duplicate succeeds with `deduped: true` and the existing path.
96
+ - A nightly curation run is a branch. `memhtml sleep run` walks sixteen phases and commits each one's work on its own, so a human reads the curation one phase-shaped diff at a time, and `memhtml sleep merge` fast-forwards `main` only after a quality gate that can refuse.
129
97
 
130
98
  ## Who does what
131
99
 
132
- Three actors share one tree. The agent writes facts, and it resolves only the conflicts it found itself.
133
- Sleep curates nightly on a branch, and it detects conflicts without resolving them. The human owns the
134
- gate and every one-way door.
100
+ Three actors share one tree. The agent writes facts, and it resolves only the conflicts it found itself. Sleep curates nightly on a branch, and it detects conflicts without resolving them. The human owns the gate and every one-way door.
135
101
 
136
- Figure 2 draws the cycle they form. A screen reader reads its box characters as noise, so the paragraph
137
- beneath the figure carries the same content in words.
102
+ Figure 2 draws the cycle they form. A screen reader reads its box characters as noise, so the paragraph beneath the figure carries the same content in words.
138
103
 
104
+ <!-- dprint-ignore-start -->
139
105
  <!-- figure:three-actors -->
140
106
  ```text
141
107
  +----------+
@@ -181,20 +147,13 @@ beneath the figure carries the same content in words.
181
147
  +------------+
182
148
  ```
183
149
  <!-- /figure:three-actors -->
150
+ <!-- dprint-ignore-end -->
184
151
 
185
- **Figure 2: the three actors form a cycle through `main`, and only one of them may settle a
186
- contradiction.** Reading top to bottom: the agent writes to `main` at any hour, one fact per file.
187
- Sleep reads `main` nightly and puts its fifteen commits on a `sleep/<date>` branch, leaving `main`
188
- untouched. Those phases deduplicate, resolve entities, decay confidence, compress, and synthesize arcs,
189
- and they flag a contradiction without choosing a winner. The human reviews that branch and merges, which
190
- returns the cycle to `main` and to the agent. The two heavy-bordered boxes are the actors outside the
191
- system, and `main` and the branch are double-bordered because they are the system of record.
152
+ **Figure 2: the three actors form a cycle through `main`, and only one of them may settle a contradiction.** Reading top to bottom: the agent writes to `main` at any hour, one fact per file. Sleep reads `main` nightly and puts its sixteen commits on a `sleep/<date>` branch, leaving `main` untouched. Those phases deduplicate, resolve entities, decay confidence, compress, and synthesize arcs, and they flag a contradiction without choosing a winner. The human reviews that branch and merges, which returns the cycle to `main` and to the agent. The two heavy-bordered boxes are the actors outside the system, and `main` and the branch are double-bordered because they are the system of record.
192
153
 
193
154
  ## The file format
194
155
 
195
- One fact per file, in standard HTML5 that a browser displays and a person can read in view-source. The
196
- element vocabulary is closed, and `memhtml doctor` warns on anything outside it. Each element carries
197
- meaning the indexer reads, which is the structure Markdown gives you no way to express:
156
+ One fact per file, in standard HTML5 that a browser displays and a person can read in view-source. The element vocabulary is closed, and `memhtml doctor` warns on anything outside it. Each element carries meaning the indexer reads, which is the structure Markdown gives you no way to express:
198
157
 
199
158
  ```html
200
159
  <article>
@@ -207,37 +166,23 @@ meaning the indexer reads, which is the structure Markdown gives you no way to e
207
166
  </article>
208
167
  ```
209
168
 
210
- The single `<mark>` is the claim. It becomes the gist every listing shows, and it is the span a
211
- correction targets. `<time datetime>` records when the fact happened in the world, so an episodic memory
212
- ranks by that date instead of by its write time. `<dl>` pairs index as facets and `<cite>` as citations.
213
- `<details>` folds elaboration behind a summary, and recall always discloses that a fold is there.
214
- `docs/format.md` is the full vocabulary. `docs/tasks.md` covers the task type (`memhtml-task-status`,
215
- `memhtml-due`), which rides the same format.
169
+ The single `<mark>` is the claim. It becomes the gist every listing shows, and it is the span a correction targets. `<time datetime>` records when the fact happened in the world, so an episodic memory ranks by that date instead of by its write time. `<dl>` pairs index as facets and `<cite>` as citations. `<details>` folds elaboration behind a summary, and recall always discloses that a fold is there. `docs/format.md` is the full vocabulary. `docs/tasks.md` covers the task type (`memhtml-task-status`, `memhtml-due`), which rides the same format.
216
170
 
217
171
  ## Writing
218
172
 
219
173
  Three doors, all supported, all landing in the same tree:
220
174
 
221
- 1. The CLI. `memhtml write` takes one memory. Give it `--claim` plus `--body` and the template owns the
222
- markup; give it `--article-html` and you own the markup, with the format check refusing violations
223
- before anything is written. `memhtml apply` takes many: one JSONL op per line, every op validated for
224
- shape before any of them executes, then one commit and one index pass.
225
- 2. The MCP server. `memhtml serve mcp` speaks stdio and exposes 14 tools and 2 resources over the same
226
- repo: write, read, search, recall, correct, link, archive, batch writes, and trace search. A CLI
227
- command and a running server share one store, because WAL admits one writer and any number of
228
- readers, and a contended write retries on `SQLITE_BUSY` (see `RUNBOOK.md`, section 4).
229
- 3. Your file tools. The tree is the system of record, so a hand-written file is as real as one the CLI
230
- wrote. You take on what the write path would have done: format validity (`memhtml doctor`), path
231
- choice, dedup, and the commit. Sleep refuses to start on a dirty tree.
232
-
233
- Dedup is enforced by the schema: a partial unique index over active files makes a duplicate write
234
- impossible to index, so the write returns the existing path with `deduped: true` and creates nothing.
175
+ 1. The CLI. `memhtml write` takes one memory. Give it `--claim` plus `--body` and the template owns the markup; give it `--article-html` and you own the markup, with the format check refusing violations before anything is written. `memhtml apply` takes many: one JSONL op per line, every op validated for shape before any of them executes, then one commit and one index pass.
176
+ 2. The MCP server. `memhtml serve mcp` speaks stdio and exposes 14 tools and 2 resources over the same repo: write, read, search, recall, correct, link, archive, batch writes, and trace search. A CLI command and a running server share one store, because WAL admits one writer and any number of readers, and a contended write retries on `SQLITE_BUSY` (see `RUNBOOK.md`, section 4).
177
+ 3. Your file tools. The tree is the system of record, so a hand-written file is as real as one the CLI wrote. You take on what the write path would have done: format validity (`memhtml doctor`), path choice, dedup, and the commit. Sleep refuses to start on a dirty tree.
178
+
179
+ Dedup is enforced by the schema: a partial unique index over active files makes a duplicate write impossible to index, so the write returns the existing path with `deduped: true` and creates nothing.
235
180
 
236
181
  A memory's whole life is commits in one tree.
237
182
 
238
- Figure 3 draws that life. A screen reader sounds out its box characters, so read the paragraph beneath
239
- the figure, which states the same four transitions in words.
183
+ Figure 3 draws that life. A screen reader sounds out its box characters, so read the paragraph beneath the figure, which states the same four transitions in words.
240
184
 
185
+ <!-- dprint-ignore-start -->
241
186
  <!-- figure:memory-lifecycle -->
242
187
  ```text
243
188
  +--------+
@@ -265,77 +210,44 @@ the figure, which states the same four transitions in words.
265
210
  +-----------+ +---------+ +-----------+
266
211
  ```
267
212
  <!-- /figure:memory-lifecycle -->
213
+ <!-- dprint-ignore-end -->
268
214
 
269
- **Figure 3: a memory has one entry and three exits, and every one of them is a commit.** A write enters
270
- the corpus as a single dedup-checked commit, and the file is then active. It stays active while sleep
271
- reinforces or decays its confidence in place. Three things can end that state: `memhtml correct` writes a
272
- replacement and archives the original in one commit, which makes it superseded; retention triage scores
273
- it into the EVICT band and archives it, which makes it archived; or compress folds it into a synthesized
274
- canonical memory and archives it with a `supersedes` link, which makes it compressed. Each of the three
275
- exits is a `git mv` into `archive/YYYY/` mirroring the original path, so the file survives all of them
276
- and `git log --follow` reads straight through the whole life.
215
+ **Figure 3: a memory has one entry and three exits, and every one of them is a commit.** A write enters the corpus as a single dedup-checked commit, and the file is then active. It stays active while sleep reinforces or decays its confidence in place. Three things can end that state: `memhtml correct` writes a replacement and archives the original in one commit, which makes it superseded; retention triage scores it into the EVICT band and archives it, which makes it archived; or compress folds it into a synthesized canonical memory and archives it with a `supersedes` link, which makes it compressed. Each of the three exits is a `git mv` into `archive/YYYY/` mirroring the original path, so the file survives all of them and `git log --follow` reads straight through the whole life.
277
216
 
278
217
  ## Retrieval
279
218
 
280
- Four ranking arms are fused by reciprocal rank fusion (RRF, k=60) inside one SQL statement, and maximal
281
- marginal relevance (MMR) diversifies the result afterwards in TypeScript:
219
+ Four ranking arms are fused by reciprocal rank fusion (RRF, k=60) inside one SQL statement, and maximal marginal relevance (MMR) diversifies the result afterwards in TypeScript:
282
220
 
283
- | arm | weight | what it ranks |
284
- |---|---|---|
285
- | fts | 1.0 | one denormalized title+gist+body column |
286
- | vector | 1.0 | exact brute force over Cohere Embed v4 (1024-dim), grouped by path so a long memory does not outrank a relevant one |
287
- | recency | 0.5 | `coalesce(event_at, updated_at)`, so an episodic memory sorts by when the fact happened |
288
- | salience | 0.4 | the durable access plane, attached in the same statement, with tasks and `resources/people/` excluded |
221
+ | arm | weight | what it ranks |
222
+ | -------- | ------ | ------------------------------------------------------------------------------------------------------------------- |
223
+ | fts | 1.0 | one denormalized title+gist+body column |
224
+ | vector | 1.0 | exact brute force over Cohere Embed v4 (1024-dim), grouped by path so a long memory does not outrank a relevant one |
225
+ | recency | 0.5 | `coalesce(event_at, updated_at)`, so an episodic memory sorts by when the fact happened |
226
+ | salience | 0.4 | the durable access plane, attached in the same statement, with tasks and `resources/people/` excluded |
289
227
 
290
- A Bedrock outage narrows retrieval instead of stopping it. Arms that need a query vector are dropped
291
- before the statement is assembled, the response carries `degraded: true`, and the remaining arms answer.
228
+ A Bedrock outage narrows retrieval instead of stopping it. Arms that need a query vector are dropped before the statement is assembled, the response carries `degraded: true`, and the remaining arms answer.
292
229
 
293
- Salience counts the opens a caller chose. `memhtml read` and `memory_read` of a named path bump the access
294
- plane; a path that `memhtml search` or `memhtml recall` merely returned does not, and neither does a sleep
295
- phase. Bumping on a hit would make today's top five rank higher tomorrow for having been listed, while
296
- the memory that should displace them never gets a first bump. `memhtml reinforce` is the explicit outcome
297
- channel, and it moves the same exponentially weighted moving average. The arm also stays out of the way
298
- of a `task` row and a `resources/people/` reference record: both are reached by predicate and by key, and
299
- salience there would reward a stale task and decay a person's identity.
230
+ Salience counts the opens a caller chose. `memhtml read` and `memory_read` of a named path bump the access plane; a path that `memhtml search` or `memhtml recall` merely returned does not, and neither does a sleep phase. Bumping on a hit would make today's top five rank higher tomorrow for having been listed, while the memory that should displace them never gets a first bump. `memhtml reinforce` is the explicit outcome channel, and it moves the same exponentially weighted moving average. The arm also stays out of the way of a `task` row and a `resources/people/` reference record: both are reached by predicate and by key, and salience there would reward a stale task and decay a person's identity.
300
231
 
301
- `memhtml recall` adds a disclosure fold on top. Arcs get their own character envelope, so a summary does
302
- not compete with the memories it summarizes. Each fold quotes at most 2 memories per entity name, and
303
- everything past the budget collapses to one index line plus a path to drill into.
232
+ `memhtml recall` adds a disclosure fold on top. Arcs get their own character envelope, so a summary does not compete with the memories it summarizes. Each fold quotes at most 2 memories per entity name, and everything past the budget collapses to one index line plus a path to drill into.
304
233
 
305
234
  ## The discrimination gate
306
235
 
307
- `memhtml eval discriminate` reports the number that says whether retrieval can tell two similar facts
308
- apart.
309
- Embeddings are weakest on the tokens that carry a fact's polarity: "drain the VIP before reverting" and
310
- "do not drain the VIP before reverting" sit above 0.99 cosine similarity while asserting opposite things.
311
- So the gate derives every control from the probe's own target by flipping a negation, a number, or a
312
- qualifier, which makes each control a high-cosine wrong answer by construction. Every target has to
313
- strictly outrank all of its own controls, mean reciprocal rank has to clear 0.85, and one inversion fails
314
- the run.
236
+ `memhtml eval discriminate` reports the number that says whether retrieval can tell two similar facts apart. Embeddings are weakest on the tokens that carry a fact's polarity: "drain the VIP before reverting" and "do not drain the VIP before reverting" sit above 0.99 cosine similarity while asserting opposite things. So the gate derives every control from the probe's own target by flipping a negation, a number, or a qualifier, which makes each control a high-cosine wrong answer by construction. Every target has to strictly outrank all of its own controls, mean reciprocal rank has to clear 0.85, and one inversion fails the run.
315
237
 
316
- Two places run it. `pnpm check` runs it, and CI runs `pnpm check`. `memhtml sleep merge` runs it a second
317
- time, so a sleep run that degrades retrieval cannot land. Fake-embedder mode is deterministic and needs
318
- no credentials. `live` mode is an operator diagnostic, and it reports `skipped: true` when it cannot
319
- reach the model, so a skipped gate reads as skipped rather than as green.
238
+ Two places run it. `pnpm check` runs it, and CI runs `pnpm check`. `memhtml sleep merge` runs it a second time, so a sleep run that degrades retrieval cannot land. Fake-embedder mode is deterministic and needs no credentials. `live` mode is an operator diagnostic, and it reports `skipped: true` when it cannot reach the model, so a skipped gate reads as skipped rather than as green.
320
239
 
321
240
  ## Sleep
322
241
 
323
- `memhtml sleep run` executes fifteen curation phases on a `sleep/<date>` branch: dedup-merge, entity
324
- resolution, edge typing, confidence decay, arc synthesis, retention triage, compress, integrity,
325
- and the rest. Each committing phase makes its own isolated commit with a machine-readable trailer, so
326
- `memhtml sleep resume` re-runs only what is missing. Two phases commit nothing by design. `preflight`
327
- refreshes the index, and `relationship-mining` writes derived edges to the index alone, because thousands
328
- of re-derivable edges would bury every real diff. `trace-consolidation` hands unread session transcripts
329
- to an agent and lands each distilled memory as its own commit, one per memory, so a reviewer reads one
330
- claim at a time. A failed phase leaves the phases before it committed.
242
+ `memhtml sleep run` executes sixteen curation phases on a `sleep/<date>` branch: dedup-merge, entity resolution, edge typing, confidence decay, arc synthesis, retention triage, compress, integrity, and the rest. Each committing phase makes its own isolated commit with a machine-readable trailer, so `memhtml sleep resume` re-runs only what is missing. Two phases commit nothing by design. `preflight` refreshes the index, and `relationship-mining` writes derived edges to the index alone, because thousands of re-derivable edges would bury every real diff. `trace-consolidation` hands unread session transcripts to an agent and lands each distilled memory as its own commit, one per memory, so a reviewer reads one claim at a time. A failed phase leaves the phases before it committed.
243
+
244
+ A run also opens TASKS, for work the corpus records and nobody opened. `task-detection` reads the recent memories in batches and asks which of them carry a commitment nobody closed, quoting the sentence it found; three other phases do the same for the decisions they decline to make — an alias pair too close to ignore and too far to merge, a near-duplicate pair the divergence veto refused, a contradiction seen only once. Every detected task is authored `agent:sleep`, cites its evidence verbatim, is capped at ten a night across all four detectors, and closes itself when its finding stops appearing. A detection is a proposal for a human, never a fact the corpus asserts.
331
245
 
332
- `memhtml sleep review` classifies every touched file. `memhtml sleep merge` re-runs the discrimination gate
333
- and refuses to move `main` on a regression. Detecting a conflict is nightly and automatic; resolving one
334
- stays with the writer or a human, because choosing a winner is a one-way door.
246
+ `memhtml sleep review` classifies every touched file. `memhtml sleep merge` re-runs the discrimination gate and refuses to move `main` on a regression. Detecting a conflict is nightly and automatic; resolving one stays with the writer or a human, because choosing a winner is a one-way door.
335
247
 
336
- Figure 4 draws the branch and the gate. A screen reader sounds out its box characters, so read the
337
- paragraph beneath the figure, which carries the same content in words.
248
+ Figure 4 draws the branch and the gate. A screen reader sounds out its box characters, so read the paragraph beneath the figure, which carries the same content in words.
338
249
 
250
+ <!-- dprint-ignore-start -->
339
251
  <!-- figure:sleep-branch -->
340
252
  ```text
341
253
  +-------+
@@ -352,10 +264,10 @@ paragraph beneath the figure, which carries the same content in words.
352
264
  +-------------+
353
265
  |
354
266
  v
355
- +---------------+
356
- |fifteen phases |
357
- | |
358
- +---------------+
267
+ +---------------+
268
+ |sixteen phases |
269
+ | |
270
+ +---------------+
359
271
  |
360
272
  review
361
273
  |
@@ -376,45 +288,26 @@ paragraph beneath the figure, which carries the same content in words.
376
288
  +-----------+ +-------------+
377
289
  ```
378
290
  <!-- /figure:sleep-branch -->
291
+ <!-- dprint-ignore-end -->
379
292
 
380
- **Figure 4: `main` moves only after a gate that can refuse says so.** A run branches `main` into
381
- `sleep/<date>` before any phase executes and walks its fifteen phases there, thirteen of them committing,
382
- each on its own, with `preflight` and `relationship-mining` committing nothing by design. Then it submits
383
- the branch for review. That review re-runs the discrimination gate and has two outcomes, both drawn: it
384
- passes and `main` moves, or it refuses and `main` stays exactly where it was. Those are the only two
385
- outcomes, and neither needs a rollback, because nothing on `main` ever moved. The abort is
386
- `git branch -D`.
293
+ **Figure 4: `main` moves only after a gate that can refuse says so.** A run branches `main` into `sleep/<date>` before any phase executes and walks its sixteen phases there, fourteen of them committing, each on its own, with `preflight` and `relationship-mining` committing nothing by design. Then it submits the branch for review. That review re-runs the discrimination gate and has two outcomes, both drawn: it passes and `main` moves, or it refuses and `main` stays exactly where it was. Those are the only two outcomes, and neither needs a rollback, because nothing on `main` ever moved. The abort is `git branch -D`.
387
294
 
388
295
  ## Code-mode
389
296
 
390
- The closed vocabulary makes the corpus a queryable API with no new surface: `article mark` is always the
391
- claim, `link[rel^="memhtml-"]` is always an authored edge, and `dl` pairs are always facets. Use the
392
- descendant selector. The markup is `<article><p><mark>`, so `article > mark` matches nothing and a helper
393
- written from that spelling reports zero claims while looking correct. An agent that writes parser code
394
- against `$MEMHTML_ROOT` composes multi-hop traversals in one execution and answers corpus-shaped
395
- questions no tool enumerates: live contradiction pairs, an orphan census, a walk up a supersedence chain.
396
- The contract is read-only, and writes stay behind the three doors above.
397
-
398
- `memhtml exec` ships this as a command. It runs your script in a QuickJS sandbox with the corpus mounted
399
- read-only at `/mnt/memhtml` and a helper preloaded at `/workspace/lib/corpus.mjs`, and it answers one
400
- `exec.report` envelope. Measured on the 305-file fixture corpus: 305/305 claims parsed and 410/410 edges
401
- resolved in one execution. The script reads a pinned commit, so an answer is reproducible and an
402
- uncommitted edit stays invisible to it. The sandbox reaches the structural and lexical planes and holds
403
- no index handle, so a script that wants ranked retrieval shells out to `memhtml search` and consumes its
404
- envelope. `docs/code-mode.md` is the cookbook, with a measured helper and five recipes.
297
+ The closed vocabulary makes the corpus a queryable API with no new surface: `article mark` is always the claim, `link[rel^="memhtml-"]` is always an authored edge, and `dl` pairs are always facets. Use the descendant selector. The markup is `<article><p><mark>`, so `article > mark` matches nothing and a helper written from that spelling reports zero claims while looking correct. An agent that writes parser code against `$MEMHTML_ROOT` composes multi-hop traversals in one execution and answers corpus-shaped questions no tool enumerates: live contradiction pairs, an orphan census, a walk up a supersedence chain. The contract is read-only, and writes stay behind the three doors above.
298
+
299
+ `memhtml exec` ships this as a command. It runs your script in a QuickJS sandbox with the corpus mounted read-only at `/mnt/memhtml` and a helper preloaded at `/workspace/lib/corpus.mjs`, and it answers one `exec.report` envelope. Measured on the 305-file fixture corpus: 305/305 claims parsed and 410/410 edges resolved in one execution. The script reads a pinned commit, so an answer is reproducible and an uncommitted edit stays invisible to it. The sandbox reaches the structural and lexical planes and holds no index handle, so a script that wants ranked retrieval shells out to `memhtml search` and consumes its envelope. `docs/code-mode.md` is the cookbook, with a measured helper and five recipes.
405
300
 
406
301
  ## Measured
407
302
 
408
- | benchmark | memhtml | published reference |
409
- |---|---|---|
410
- | MemoryAgentBench FactConsolidation single-hop (26KB to 1.1MB stores) | 92% to 97% | ~60% at 26KB only |
411
- | MemoryAgentBench FactConsolidation multi-hop | 37% to 49% | ≤7% all methods |
412
- | BEAM Contradiction Resolution (100K split, 40 probes) | 43.8% mean | 0% to 5% all systems |
413
- | LongMemEval-S (full 500, judged 2026-08-07) | 67.0% | ~55% to 65% typical agent baselines |
303
+ | benchmark | memhtml | published reference |
304
+ | -------------------------------------------------------------------- | ---------- | ----------------------------------- |
305
+ | MemoryAgentBench FactConsolidation single-hop (26KB to 1.1MB stores) | 92% to 97% | ~60% at 26KB only |
306
+ | MemoryAgentBench FactConsolidation multi-hop | 37% to 49% | ≤7% all methods |
307
+ | BEAM Contradiction Resolution (100K split, 40 probes) | 43.8% mean | 0% to 5% all systems |
308
+ | LongMemEval-S (full 500, judged 2026-08-07) | 67.0% | ~55% to 65% typical agent baselines |
414
309
 
415
- Read the cross-judge numbers as reference points rather than as a ranking: the judges here are verbatim
416
- prompt ports running haiku-4.5, where the papers used gpt-4o and gpt-4.1-mini. `ROADMAP.md` carries these
417
- numbers and the horizons they rank.
310
+ Read the cross-judge numbers as reference points rather than as a ranking: the judges here are verbatim prompt ports running haiku-4.5, where the papers used gpt-4o and gpt-4.1-mini. `ROADMAP.md` carries these numbers and the horizons they rank.
418
311
 
419
312
  ## Layout
420
313
 
@@ -437,38 +330,29 @@ $MEMHTML_ROOT/ # its own git repo, one global memory stor
437
330
 
438
331
  ## Packages
439
332
 
440
- The layering is strict and TypeScript project references enforce it. `@memhtml/contracts` and
441
- `@memhtml/domain` import `effect` and nothing else, and a test reads `domain`'s own `dist` to confirm it
442
- names no database driver, no SDK, and no `node:fs`.
443
-
444
- None of them is published. Every workspace package is `private`, and `mise run package:assemble`
445
- bundles the libraries and the binary-bearing apps into the single `memhtml` package that carries the
446
- two binaries the docs site and the integration-test harness stay outside the bundle
447
- (`tsdown.config.ts` names the exact set). The table below is a map of the source, not a list of
448
- things to install. `RELEASING.md` covers how the artifact is built and what must stay outside the
449
- bundle.
450
-
451
- | Package | What it owns |
452
- |---|---|
453
- | `@memhtml/contracts` | Schemas, the closed vocabularies, errors, path algebra. Zero I/O. |
454
- | `@memhtml/domain` | Pure math: retention, decay, RRF, MMR, PageRank, the anti-merge guards. |
455
- | `@memhtml/html` | The memory file format: parse, serialize, hash, surgical head editors. |
456
- | `@memhtml/store` | The git-backed file store. One commit per operation, typed conflicts. |
457
- | `@memhtml/index` | SQLite schema, the git-driven indexer, four-arm RRF retrieval, the state plane. |
458
- | `@memhtml/traces` | Streaming JSONL parser over `~/.claude`, with a size+mtime+offset watermark. |
459
- | `@memhtml/sleep` | The fifteen curation phases, each an isolated commit. |
460
- | `@memhtml/llm` | Bedrock: Cohere embeddings and forced-tool structured output. |
461
- | `@memhtml/eval` | The fixture corpus generator and the refusable discrimination gate. |
462
- | `@memhtml/cli` | The `memhtml` binary, the envelope contract, and the one composition root. |
463
- | `@memhtml/mcp` | The `memhtml-mcp` stdio server: 14 tools, 2 resources. |
464
- | `@memhtml/consolidator` | The sandboxed eve agent that distills candidate memories from raw transcripts. |
465
- | `@memhtml/docs` | The documentation site. |
333
+ The layering is strict and TypeScript project references enforce it. `@memhtml/contracts` and `@memhtml/domain` import `effect` and nothing else, and a test reads `domain`'s own `dist` to confirm it names no database driver, no SDK, and no `node:fs`.
334
+
335
+ None of them is published. Every workspace package is `private`, and `mise run package:assemble` bundles the libraries and the binary-bearing apps into the single `memhtml` package that carries the two binaries — the docs site and the integration-test harness stay outside the bundle (`tsdown.config.ts` names the exact set). The table below is a map of the source, not a list of things to install. `RELEASING.md` covers how the artifact is built and what must stay outside the bundle.
336
+
337
+ | Package | What it owns |
338
+ | ----------------------- | ------------------------------------------------------------------------------- |
339
+ | `@memhtml/contracts` | Schemas, the closed vocabularies, errors, path algebra. Zero I/O. |
340
+ | `@memhtml/domain` | Pure math: retention, decay, RRF, MMR, PageRank, the anti-merge guards. |
341
+ | `@memhtml/html` | The memory file format: parse, serialize, hash, surgical head editors. |
342
+ | `@memhtml/store` | The git-backed file store. One commit per operation, typed conflicts. |
343
+ | `@memhtml/index` | SQLite schema, the git-driven indexer, four-arm RRF retrieval, the state plane. |
344
+ | `@memhtml/traces` | Streaming JSONL parser over `~/.claude`, with a size+mtime+offset watermark. |
345
+ | `@memhtml/sleep` | The sixteen curation phases, each an isolated commit. |
346
+ | `@memhtml/llm` | Bedrock: Cohere embeddings and forced-tool structured output. |
347
+ | `@memhtml/eval` | The fixture corpus generator and the refusable discrimination gate. |
348
+ | `@memhtml/cli` | The `memhtml` binary, the envelope contract, and the one composition root. |
349
+ | `@memhtml/mcp` | The `memhtml-mcp` stdio server: 14 tools, 2 resources. |
350
+ | `@memhtml/consolidator` | The sandboxed eve agent that distills candidate memories from raw transcripts. |
351
+ | `@memhtml/docs` | The documentation site. |
466
352
 
467
353
  ## Development
468
354
 
469
- [`mise`](https://mise.jdx.dev) is the command surface. It installs the toolchain from `mise.toml`, which
470
- declares node, pnpm, lefthook, and the scanners, each pinned by checksum and provenance in the committed
471
- `mise.lock`, so a clone resolves the same binaries CI does:
355
+ [`mise`](https://mise.jdx.dev) is the command surface. It installs the toolchain from `mise.toml`, which declares node, pnpm, lefthook, and the scanners, each pinned by checksum and provenance in the committed `mise.lock`, so a clone resolves the same binaries CI does:
472
356
 
473
357
  ```bash
474
358
  mise install # node 24, pnpm 11.21.0, lefthook, scanners, from mise.lock
@@ -476,60 +360,45 @@ mise run install # dependencies from the lockfile + the git hooks
476
360
  mise run check # the definition of done: lint, typecheck, tests, integration, eval, a11y, budget
477
361
  ```
478
362
 
479
- CI runs that same `mise run check`, so the gate cannot drift from the one you run locally. Every task
480
- delegates to the pnpm script underneath it, and turbo owns the task graph and the cache. No mise task
481
- declares `sources` or `outputs`, because mise decides freshness by mtime and turbo by content hash, so a
482
- mise-level skip would preempt turbo's per-package hashing.
483
-
484
- `check` includes the discrimination gate in fake mode, so a change that degrades retrieval fails the
485
- build. Tests run against a real temp-dir git repo and a real SQLite database with the shipped migrations.
486
- Fakes are limited to the two edges that reach the network, the embedder and the model, because a stateless
487
- fake verifies the shape of a call and misses the state semantics behind it, which is where the defects in
488
- this system have actually lived.
489
-
490
- | Command | Delegates to | What it runs |
491
- |---|---|---|
492
- | `mise run build` | `pnpm build` | `tsc -b` across the project graph |
493
- | `mise run lint` | `pnpm lint` | biome |
494
- | `mise run typecheck` | `pnpm typecheck` | strict `tsc --noEmit`, tests included |
495
- | `mise run test` | `pnpm test` | every package's unit and property suites |
363
+ CI runs that same `mise run check`, so the gate cannot drift from the one you run locally. Every task delegates to the pnpm script underneath it, and turbo owns the task graph and the cache. No mise task declares `sources` or `outputs`, because mise decides freshness by mtime and turbo by content hash, so a mise-level skip would preempt turbo's per-package hashing.
364
+
365
+ `check` includes the discrimination gate in fake mode, so a change that degrades retrieval fails the build. Tests run against a real temp-dir git repo and a real SQLite database with the shipped migrations. Fakes are limited to the two edges that reach the network, the embedder and the model, because a stateless fake verifies the shape of a call and misses the state semantics behind it, which is where the defects in this system have actually lived.
366
+
367
+ | Command | Delegates to | What it runs |
368
+ | --------------------------- | ----------------------- | ---------------------------------------------------------------- |
369
+ | `mise run build` | `pnpm build` | `tsc -b` across the project graph |
370
+ | `mise run lint` | `pnpm lint` | biome |
371
+ | `mise run typecheck` | `pnpm typecheck` | strict `tsc --noEmit`, tests included |
372
+ | `mise run test` | `pnpm test` | every package's unit and property suites |
496
373
  | `mise run test:integration` | `pnpm test:integration` | the cross-package contracts over a real repo and a real database |
497
- | `mise run test:eval` | `pnpm test:eval` | the discrimination gate (fake mode) |
498
- | `mise run test:a11y` | `pnpm test:a11y` | WCAG 2.2 AA over the built docs site, in a real browser |
499
- | `mise run test:budget` | `pnpm test:budget` | Lighthouse category floors and the byte budget for that site |
500
- | `mise run gen:fixture` | `pnpm gen:fixture` | write a browsable fixture corpus (pure function of a seed) |
501
- | `mise run agents-doc` | none | regenerate `AGENTS.md` from the built CLI's own table |
502
- | `mise run security` | none | osv-scanner + semgrep + betterleaks, SARIF into `.sarif/` |
503
- | `mise run tools:bump` | none | re-resolve every `latest` tool in `mise.lock` |
374
+ | `mise run test:eval` | `pnpm test:eval` | the discrimination gate (fake mode) |
375
+ | `mise run test:a11y` | `pnpm test:a11y` | WCAG 2.2 AA over the built docs site, in a real browser |
376
+ | `mise run test:budget` | `pnpm test:budget` | Lighthouse category floors and the byte budget for that site |
377
+ | `mise run gen:fixture` | `pnpm gen:fixture` | write a browsable fixture corpus (pure function of a seed) |
378
+ | `mise run agents-doc` | none | regenerate `AGENTS.md` from the built CLI's own table |
379
+ | `mise run security` | none | osv-scanner + semgrep + betterleaks, SARIF into `.sarif/` |
380
+ | `mise run tools:bump` | none | re-resolve every `latest` tool in `mise.lock` |
504
381
 
505
- To narrow a run to one package, use `mise run test-pkg <package> [vitest args]`. The package name takes
506
- either spelling, and everything after it goes to vitest:
382
+ To narrow a run to one package, use `mise run test-pkg <package> [vitest args]`. The package name takes either spelling, and everything after it goes to vitest:
507
383
 
508
384
  ```bash
509
385
  mise run test-pkg domain rrf -t "strictly" # one test
510
386
  mise run test-pkg index retrieval # one file
511
387
  ```
512
388
 
513
- That path goes straight to the package's vitest, so it skips turbo and builds nothing first. Every
514
- `@memhtml/*` package's exports resolve only to `./dist`, so run `mise run build` after editing another
515
- package's `src/`.
389
+ That path goes straight to the package's vitest, so it skips turbo and builds nothing first. Every `@memhtml/*` package's exports resolve only to `./dist`, so run `mise run build` after editing another package's `src/`.
516
390
 
517
- `mise.toml`'s `[tools] pnpm` and `package.json`'s `packageManager` both declare the pnpm that runs, and
518
- neither can be derived from the other. `mise run tools:verify` fails when they disagree, and
519
- `mise run install` depends on it, so the check runs on every install. Declaration order matters too:
520
- `pnpm` sits above `node` because node's own bin holds a `pnpm` symlink into corepack wherever
521
- `corepack enable` has run, and with node first every pnpm call would resolve to corepack rather than to
522
- the pinned binary.
391
+ `mise.toml`'s `[tools] pnpm` and `package.json`'s `packageManager` both declare the pnpm that runs, and neither can be derived from the other. `mise run tools:verify` fails when they disagree, and `mise run install` depends on it, so the check runs on every install. Declaration order matters too: `pnpm` sits above `node` because node's own bin holds a `pnpm` symlink into corepack wherever `corepack enable` has run, and with node first every pnpm call would resolve to corepack rather than to the pinned binary.
523
392
 
524
393
  ## Docs
525
394
 
526
- | File | What it holds |
527
- |---|---|
528
- | `AGENTS.md` | The full command surface, generated from the binary's own table |
529
- | `RUNBOOK.md` | Operating the store day to day |
530
- | `ROADMAP.md` | The system-level view: measured standing, ranked horizons |
531
- | `docs/design.md` | Every architectural decision with its evidence |
532
- | `docs/format.md` | The file format and the closed vocabulary |
533
- | `docs/tasks.md` | The task memory type |
534
- | `docs/code-mode.md` | Navigating the corpus with code |
535
- | `docs/backlog.md` | The fine-grained ledger |
395
+ | File | What it holds |
396
+ | ------------------- | --------------------------------------------------------------- |
397
+ | `AGENTS.md` | The full command surface, generated from the binary's own table |
398
+ | `RUNBOOK.md` | Operating the store day to day |
399
+ | `ROADMAP.md` | The system-level view: measured standing, ranked horizons |
400
+ | `docs/design.md` | Every architectural decision with its evidence |
401
+ | `docs/format.md` | The file format and the closed vocabulary |
402
+ | `docs/tasks.md` | The task memory type |
403
+ | `docs/code-mode.md` | Navigating the corpus with code |
404
+ | `docs/backlog.md` | The fine-grained ledger |