trajectories 0.0.0 → 0.1.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (205) hide show
  1. package/CHANGELOG.md +36 -0
  2. package/LICENSE +201 -0
  3. package/README.md +288 -2
  4. package/dist/cli.js +30210 -0
  5. package/dist/convert.d.ts +31 -0
  6. package/dist/convert.js +431 -0
  7. package/dist/deepagents_checkpoint.py +362 -0
  8. package/dist/index.d.ts +7 -0
  9. package/dist/index.js +8402 -0
  10. package/dist/log-baseline.d.ts +26 -0
  11. package/dist/log-baseline.js +104 -0
  12. package/dist/log-content.d.ts +16 -0
  13. package/dist/log-content.js +80 -0
  14. package/dist/log-io.d.ts +10 -0
  15. package/dist/log-io.js +121 -0
  16. package/dist/log-read.d.ts +24 -0
  17. package/dist/log-read.js +117 -0
  18. package/dist/log-reconcile.d.ts +5 -0
  19. package/dist/log-reconcile.js +38 -0
  20. package/dist/log-records.d.ts +35 -0
  21. package/dist/log-records.js +85 -0
  22. package/dist/log-recovery.d.ts +18 -0
  23. package/dist/log-recovery.js +123 -0
  24. package/dist/log-stage.d.ts +14 -0
  25. package/dist/log-stage.js +138 -0
  26. package/dist/log-state.d.ts +89 -0
  27. package/dist/log-state.js +92 -0
  28. package/dist/log-switch.d.ts +3 -0
  29. package/dist/log-switch.js +98 -0
  30. package/dist/log-types.d.ts +129 -0
  31. package/dist/log-types.js +8 -0
  32. package/dist/log.d.ts +8 -0
  33. package/dist/log.js +62 -0
  34. package/dist/migrate.d.ts +29 -0
  35. package/dist/migrate.js +173 -0
  36. package/dist/normalization-runtime.d.ts +1 -0
  37. package/dist/normalization-runtime.js +3150 -0
  38. package/dist/portable.d.ts +9 -0
  39. package/dist/portable.js +26 -0
  40. package/dist/readers/canonical.d.ts +13 -0
  41. package/dist/readers/canonical.js +54 -0
  42. package/dist/readers/cline.d.ts +10 -0
  43. package/dist/readers/cline.js +107 -0
  44. package/dist/readers/cursor.d.ts +20 -0
  45. package/dist/readers/cursor.js +374 -0
  46. package/dist/readers/deepagents.d.ts +41 -0
  47. package/dist/readers/deepagents.js +163 -0
  48. package/dist/readers/fx.d.ts +11 -0
  49. package/dist/readers/fx.js +573 -0
  50. package/dist/readers/github-copilot.d.ts +8 -0
  51. package/dist/readers/github-copilot.js +216 -0
  52. package/dist/readers/grok-build.d.ts +12 -0
  53. package/dist/readers/grok-build.js +391 -0
  54. package/dist/readers/guards.d.ts +4 -0
  55. package/dist/readers/guards.js +121 -0
  56. package/dist/readers/pi-compaction.d.ts +7 -0
  57. package/dist/readers/pi-compaction.js +112 -0
  58. package/dist/readers/pi-context.d.ts +13 -0
  59. package/dist/readers/pi-context.js +64 -0
  60. package/dist/store.d.ts +25 -0
  61. package/dist/store.js +218 -0
  62. package/dist/types.d.ts +91 -0
  63. package/dist/types.js +1 -0
  64. package/dist/writers/claude-code.d.ts +5 -0
  65. package/dist/writers/claude-code.js +51 -0
  66. package/dist/writers/cline.d.ts +3 -0
  67. package/dist/writers/cline.js +77 -0
  68. package/dist/writers/codex.d.ts +3 -0
  69. package/dist/writers/codex.js +32 -0
  70. package/dist/writers/cursor.d.ts +3 -0
  71. package/dist/writers/cursor.js +197 -0
  72. package/dist/writers/deepagents.d.ts +3 -0
  73. package/dist/writers/deepagents.js +116 -0
  74. package/dist/writers/entropy.d.ts +5 -0
  75. package/dist/writers/entropy.js +33 -0
  76. package/dist/writers/fx.d.ts +3 -0
  77. package/dist/writers/fx.js +135 -0
  78. package/dist/writers/github-copilot.d.ts +3 -0
  79. package/dist/writers/github-copilot.js +71 -0
  80. package/dist/writers/grok-build.d.ts +3 -0
  81. package/dist/writers/grok-build.js +43 -0
  82. package/dist/writers/helpers.d.ts +17 -0
  83. package/dist/writers/helpers.js +75 -0
  84. package/dist/writers/omp.d.ts +3 -0
  85. package/dist/writers/omp.js +7 -0
  86. package/dist/writers/opencode.d.ts +3 -0
  87. package/dist/writers/opencode.js +65 -0
  88. package/dist/writers/pi.d.ts +3 -0
  89. package/dist/writers/pi.js +45 -0
  90. package/docs/cline-native-format.md +43 -0
  91. package/docs/copilot-native-format.md +78 -0
  92. package/docs/cursor-native-format.md +103 -0
  93. package/docs/deepagents-native-format.md +98 -0
  94. package/docs/evidence-api.md +170 -0
  95. package/docs/fx-native-format.md +87 -0
  96. package/docs/harness-parity-bootstrap.md +78 -0
  97. package/docs/harness-support-parity.md +107 -0
  98. package/docs/inspirations.md +103 -0
  99. package/docs/ledger-migration.md +141 -0
  100. package/docs/pi-compaction-context.md +75 -0
  101. package/docs/pi-provider-context.md +76 -0
  102. package/docs/research/2026-10-06-huihua-and-oar.md +176 -0
  103. package/docs/research/ledger/README.md +96 -0
  104. package/docs/research/ledger/confer/2026-09-13-round-10/README.md +61 -0
  105. package/docs/research/ledger/confer/2026-09-13-round-2/cass.md +72 -0
  106. package/docs/research/ledger/confer/2026-09-13-round-3/README.md +70 -0
  107. package/docs/research/ledger/confer/2026-09-13-round-5/README.md +61 -0
  108. package/docs/research/ledger/confer/2026-09-13-round-6/README.md +68 -0
  109. package/docs/research/ledger/confer/2026-09-13-round-7/README.md +64 -0
  110. package/docs/research/ledger/confer/2026-09-13-round-9/README.md +59 -0
  111. package/docs/research/ledger/confer/2026-09-14-wake/wake.md +146 -0
  112. package/docs/research/ledger/docs/inspirations.md +320 -0
  113. package/docs/research/ledger/docs/pr2-pr6-integration.md +53 -0
  114. package/docs/research/ledger/docs/research/2026-09-07-confer-inspirations.md +163 -0
  115. package/docs/research/ledger/docs/research/2026-09-07-letta-continuous-service.md +104 -0
  116. package/docs/research/ledger/docs/research/2026-09-08-claude-history-retrieval.md +117 -0
  117. package/docs/research/ledger/docs/research/2026-09-09-agentsview-native-discovery.md +87 -0
  118. package/docs/research/ledger/docs/research/2026-09-09-daemon-observation.md +231 -0
  119. package/docs/research/ledger/docs/research/2026-09-09-discovery-pagination.md +114 -0
  120. package/docs/research/ledger/docs/research/2026-09-09-harness-transcript-contract.md +145 -0
  121. package/docs/research/ledger/docs/research/2026-09-09-native-history-discovery.md +180 -0
  122. package/docs/research/ledger/docs/research/2026-09-09-native-history-implementation.md +137 -0
  123. package/docs/research/ledger/docs/research/2026-09-09-other-inspirations.md +279 -0
  124. package/docs/research/ledger/docs/research/2026-09-09-trajectory-and-letta-code.md +128 -0
  125. package/docs/research/ledger/docs/research/2026-09-13-diagnostics-coverage.md +116 -0
  126. package/docs/research/ledger/docs/research/2026-09-13-fts-rebuild.md +83 -0
  127. package/docs/research/ledger/docs/research/2026-09-13-read-path-split.md +74 -0
  128. package/docs/research/ledger/docs/research/2026-09-13-runtime-rollout.md +102 -0
  129. package/docs/research/ledger/docs/research/2026-09-13-scan-outcomes.md +80 -0
  130. package/docs/research/ledger/docs/research/2026-09-13-search-quality.md +105 -0
  131. package/docs/research/ledger/docs/research/2026-09-13-source-coverage.md +64 -0
  132. package/docs/research/ledger/docs/research/2026-09-13-sync-wall-budget.md +82 -0
  133. package/docs/research/ledger/docs/research/2026-09-13-typed-read-gate.md +81 -0
  134. package/docs/research/ledger/docs/research/2026-09-13-writer-lease.md +84 -0
  135. package/docs/research/ledger/docs/research/2026-09-16-c1-copy-fallback.md +227 -0
  136. package/docs/research/ledger/docs/research/2026-09-16-c10-kimi.md +414 -0
  137. package/docs/research/ledger/docs/research/2026-09-16-c2-derive-counter.md +216 -0
  138. package/docs/research/ledger/docs/research/2026-09-16-c3-echo-filter.md +305 -0
  139. package/docs/research/ledger/docs/research/2026-09-16-c4-title-index.md +349 -0
  140. package/docs/research/ledger/docs/research/2026-09-16-c5-trigram.md +268 -0
  141. package/docs/research/ledger/docs/research/2026-09-16-c6-recency-prior.md +438 -0
  142. package/docs/research/ledger/docs/research/2026-09-16-c7-tombstones.md +467 -0
  143. package/docs/research/ledger/docs/research/2026-09-16-c8-remote-truth.md +390 -0
  144. package/docs/research/ledger/docs/research/2026-09-16-c9-grok.md +380 -0
  145. package/docs/research/ledger/docs/research/2026-09-16-wake-port-program.md +38 -0
  146. package/docs/research/ledger/docs/research/2026-09-17-cursor-blobstore.md +157 -0
  147. package/docs/research/ledger/manifest.json +540 -0
  148. package/legacy/NOTICE.md +50 -0
  149. package/legacy/bin/ledger-core.ts +7 -0
  150. package/legacy/bin/ledger.ts +7 -0
  151. package/legacy/bin/traj-bridge.ts +7 -0
  152. package/legacy/scripts/import-claude-history.ts +130 -0
  153. package/legacy/scripts/native-history-smoke.ts +52 -0
  154. package/legacy/scripts/observe-daemon.ts +570 -0
  155. package/legacy/scripts/retitle-sessions.md +19 -0
  156. package/legacy/scripts/verify-runtime-rollout.ts +151 -0
  157. package/legacy/src/canonical.ts +108 -0
  158. package/legacy/src/cli.ts +1293 -0
  159. package/legacy/src/db.ts +2254 -0
  160. package/legacy/src/diagnostics.ts +87 -0
  161. package/legacy/src/discovery.ts +56 -0
  162. package/legacy/src/embeddings.ts +382 -0
  163. package/legacy/src/filter.ts +602 -0
  164. package/legacy/src/gemini_embeddings.ts +103 -0
  165. package/legacy/src/identity.ts +9 -0
  166. package/legacy/src/index.ts +10 -0
  167. package/legacy/src/ingest.ts +791 -0
  168. package/legacy/src/managed.ts +178 -0
  169. package/legacy/src/native/common.ts +68 -0
  170. package/legacy/src/native/cursor.ts +647 -0
  171. package/legacy/src/native/grok.ts +540 -0
  172. package/legacy/src/native/kimi.ts +706 -0
  173. package/legacy/src/native/opencode.ts +470 -0
  174. package/legacy/src/native/sqlite-ro.ts +281 -0
  175. package/legacy/src/native.ts +97 -0
  176. package/legacy/src/normalizer.ts +457 -0
  177. package/legacy/src/parentage.ts +562 -0
  178. package/legacy/src/pressure.ts +162 -0
  179. package/legacy/src/private-storage.ts +74 -0
  180. package/legacy/src/reader.ts +1255 -0
  181. package/legacy/src/scanner.ts +591 -0
  182. package/legacy/src/schema-gate.ts +263 -0
  183. package/legacy/src/schema.ts +829 -0
  184. package/legacy/src/semantic-top-k.ts +71 -0
  185. package/legacy/src/session-hash.ts +49 -0
  186. package/legacy/src/store-status.ts +218 -0
  187. package/legacy/src/sync-preparation.ts +45 -0
  188. package/legacy/src/sync-truth.ts +843 -0
  189. package/legacy/src/sync.ts +1233 -0
  190. package/legacy/src/types.ts +436 -0
  191. package/legacy/src/vocab.txt +30522 -0
  192. package/legacy/src/watch.ts +673 -0
  193. package/legacy/src/writer-lease.ts +310 -0
  194. package/ops/README.md +150 -0
  195. package/ops/install-cli.sh +63 -0
  196. package/ops/install-native.sh +35 -0
  197. package/ops/install-release.sh +49 -0
  198. package/ops/linux/install-systemd-user.sh +103 -0
  199. package/ops/macos/install-launchd.sh +72 -0
  200. package/ops/sync-settings.sh +37 -0
  201. package/package.json +21 -5
  202. package/skills/trajectories/SKILL.md +138 -0
  203. package/skills/trajectories/scripts/trajectories_recent.sh +26 -0
  204. package/skills/trajectories/scripts/trajectories_search.sh +26 -0
  205. package/index.js +0 -1
@@ -0,0 +1,467 @@
1
+ # C7: tombstone double-track, 2026-09-16
2
+
3
+ ## Outcome
4
+
5
+ Deletion is now defined, and it is the user's to make. `ledger delete
6
+ <session-id> --reason <text>` writes one row — the store's audit of that
7
+ deletion — and the session leaves every serving lane and every admission path in
8
+ the same moment; `--hard` carries the same deletion through to the records
9
+ themselves; `--list` reads the deletions back; `ledger status` discloses both the
10
+ count and what those deletions still hold. Nothing else in the system can write
11
+ one: not a scan, not a sync pass, not a source that vanished, not a source that
12
+ was truncated, and not a copy of a source at another path.
13
+
14
+ Measured on a byte-consistent copy of the production store (1,142,136,832 bytes;
15
+ 2,617 sessions; 231,553 records; 231,327 indexed rows and 226 echo rows; schema
16
+ v13 at the start, v14 after the writer open — measured while this cycle's step
17
+ still carried the number 14, before the collision below renamed it to 15; the
18
+ statement is the same one, so what follows stands as the cost of the step):
19
+
20
+ - **A soft deletion is one row and nothing else.** 0.4–12.5 ms across samples,
21
+ with 0 canonical or derived rows removed or rewritten: the probe session's
22
+ 1,572 `records_fts` rows and its `titles_fts` row were all still there
23
+ afterwards, and the session row with them.
24
+ - **Serving drops it everywhere.** The rare term that returned exactly that
25
+ session's record (1 hit, 3.4 ms) returns 0 of its rows afterwards; three
26
+ queries — that term, a zero-hit query that took the widening recovery (11
27
+ widened hits, 5 sessions), and a 50-hit broad query whose hits were all title
28
+ hits — returned **0 hits from the deleted sessions** (leaked = 0), while the
29
+ raw index still held their 1,572 + 86 rows and both title rows.
30
+ - **A `--hard` deletion is not cheap and it is complete.** Escalating a
31
+ 32,583-record session took **7.6 s** and removed that many records, leaving the
32
+ store at 2,616 sessions / 198,970 records; a forced re-derivation over the
33
+ result took **23.8 s** with both rebuild parity gates
34
+ (`verifyRecordsFtsRebuild`, `verifyTitlesFtsRebuild`) passing.
35
+ - **A re-scan is refused before it reads anything.** For a claude-code listing
36
+ whose id *is* the session's native id, and for a codex listing whose id is the
37
+ rollout filename while the session inside carries a bare uuid — both refused
38
+ with `sessionNeedsIngest = false` and a direct ingest returning `tombstoned`
39
+ with **0 source bytes read**. The second case matters: on this store **1,724 of
40
+ 2,681 cursors (64%)** name a listing id that differs from the session's native
41
+ part, so the naive native-id check would have read most files before refusing
42
+ them.
43
+ - **The migration increment is invisible.** Opening the v13 copy as a writer
44
+ (which adds the table) measured 8.7 ms; the same open on the already-migrated
45
+ file measured 3.4 ms. The one 5.2 s open in this cycle was the *first* open of the
46
+ copy — the open that had to build the WAL sidecars for a 1.06 GiB file — not
47
+ the migration.
48
+ - `ledger status` on that copy: `Tombstones: 2 sessions deleted by the user; 1658
49
+ of the records above are held back from search and reads`; robot payload
50
+ `"tombstones":2,"tombstoned_records":1658` beside the unchanged holdings counts
51
+ (`"sessions":2617,"records":231754`).
52
+ - **The production store itself migrated, and kept working.** This host's
53
+ scheduled `ledger sync` runs this checkout (`~/.local/bin/ledger` is a wrapper
54
+ for `packages/core/bin/ledger.ts`), so one of its passes during the cycle
55
+ opened the 1.06 GiB store with this build: `user_version` moved 13 → 14 (the
56
+ number this cycle claimed at the time; see *Version collision* for why the
57
+ shipped step is 15, which that store — at 14, table present — is exactly the
58
+ shape the ladder must accept), the
59
+ `session_tombstones` table is present and empty, `fts_format` is still `"3"`
60
+ (the counter did not move — the argument below), and the daemon's own summaries
61
+ stayed normal (25-session `pending_budget` passes, no error results). That is
62
+ an unplanned but real end-to-end check of the migration and of the serving
63
+ filter on the largest store this project has. The flip side, for the owner: a
64
+ store at v15 is refused by an older build (`SCHEMA_MISMATCH`), so this cycle
65
+ landing is what keeps the machine's daemon and its store compatible.
66
+
67
+ The tree's own suite grew by this cycle's cases — `bun test` 601 pass / 0 fail
68
+ across 42 files, including fourteen new cases in
69
+ `packages/core/tests/tombstones.test.ts` and the version-collision case in
70
+ `packages/core/tests/integration-migrations.test.ts`, which pin every claim above
71
+ except the production-scale numbers.
72
+
73
+ ## What the store does — the whole rule
74
+
75
+ A tombstone closes a session to **serving** and to **admission**, and — in
76
+ `suppress` mode, the default — changes nothing else:
77
+
78
+ | Surface | What the deletion does | Where |
79
+ | --- | --- | --- |
80
+ | Record lane (`search`) | filtered out of `records_fts` hits | `reader.ts`, `executeFtsQuery` |
81
+ | Title lane (`search`) | filtered out of `titles_fts` hits | `reader.ts`, `titlePrepass` |
82
+ | Widening recovery | same predicate, same lane, so a zero-hit rewrite cannot find it | `reader.ts`, `widenZeroHitQuery` → `executeFtsQuery` |
83
+ | Vector lane (`--semantic`, `--hybrid`) | filtered out of `records_embeddings` scoring | `reader.ts`, `vectorSearch` |
84
+ | `recent` / `last`, session lookup, session records, child sessions, lineage | not listed, not found, no records | `reader.ts` (`sessionById`, `allSessions`, `lastSessions`, `sessionRecords`), `db.ts` (`getChildSessions`) |
85
+ | Ingest admission (pre-read) | the listing is refused before the source is stat-ed | `ingest.ts` (`sessionNeedsIngest`, `ingestDiscoveredSession`) → `db.ts` (`tombstoneForListing`) |
86
+ | Ingest admission (store boundary) | a write for the session id is refused, inside the transaction that would have committed it | `db.ts` (`assertTombstoneFree` in `saveSession`, `appendSessionRecords`, `commitStagedIngest`, `insertRecordEmbeddings`) |
87
+ | Vector backfill | never asks for a tombstoned session's records | `db.ts`, `backfillEmbeddings` |
88
+ | Pending rebuild / queue | cleared: neither can be admitted, and a staged replacement holds the session's records | `db.ts`, `clearAdmissionState` |
89
+ | `status` | **not** subtracted: the holdings counts stay what the store holds, and the tombstone count and the records it withholds are disclosed beside them | `store-status.ts`, `cli.ts` |
90
+ | `build` / `watch` | counted as `tombstoned`, never as `failed` or `imported` | `cli.ts` (`countTombstoned`), `watch.ts`, `sync.ts` |
91
+
92
+ ## The double-track, and why the path is not one of its axes
93
+
94
+ Wake's pair is `file_path` and the logical session key
95
+ (`crates/wake-core/src/db.rs#L76-L80`: `tombstones(file_path PK, key, deleted_at)`;
96
+ `#L1002`/`#L1014`: both are probed; `crates/wake-core/src/scanner.rs#L224`: both
97
+ filter enumeration). Ledger's pair is `(source, native id)` — as a direct lookup
98
+ and through the store's own cursors — and the Ledger session id:
99
+
100
+ 1. **Listing axis, before the read.** `tombstoneForListing` answers "was the
101
+ listing this discovery pass is holding deleted?" Two lookups: the direct
102
+ `(source, native_id)` (the listing id *is* the session's native id — hermes and
103
+ other listing-keyed sources, and any source that names a session after its
104
+ file), and a join through `ingest_cursors`, which is the store's own record of
105
+ *which listing was ingested into which session*. The second lookup is not a
106
+ nicety: without it a source that keys files by rollout name while the session
107
+ inside carries its own id would be read in full before being refused — measured
108
+ at 64% of this store's cursors.
109
+ 2. **Session axis, at the store boundary.** Every write path refuses the session
110
+ id (`assertTombstoneFree`), inside the caller's transaction. This is the axis
111
+ that catches a copy whose listing looks new — another path, another listing id
112
+ — because the copy's *content* still normalizes to the same session id. It is
113
+ also the axis serving uses, so "hidden" and "cannot come back" are the same
114
+ identity, not two.
115
+
116
+ **The path deliberately is not an admission key**, though Wake's is its primary
117
+ one. In this store a path is not listing identity: shared stores (hermes,
118
+ deepagents) hold many distinct listings in one file, and this repository already
119
+ has that rule written down and enforced where it matters — `getLiveIngestSession`
120
+ in `db.ts` refuses to treat a path match as authority to delete *another*
121
+ listing's session. Tombstoning a path would therefore silently block every other
122
+ listing in that file, which is a worse failure than the one it would prevent. The
123
+ cursor is the honest Ledger analog of Wake's `file_path`: it is listing identity
124
+ *by construction* (`(source, listing_id, source_path)` is its key), and it is
125
+ already the evidence this store uses to decide who owns a generation.
126
+
127
+ Wake's other half — absence is not deletion — is structurally true here as well,
128
+ and this cycle pins it rather than changing it: Ledger never prunes a session
129
+ because a file disappeared, and Wake's two non-tombstoning removals
130
+ (`scanner.rs#L272`, owner change; `scanner.rs#L341-L349`, "in the store but gone
131
+ from disk") remove rows *without* a tombstone because they are re-ingestions, not
132
+ deletions. Measurement and tests: a truncated source in place yields
133
+ `deferred` — the repo's existing bounded-rebuild verdict — with the live history
134
+ preserved and the session still served, and a vanished file changes nothing at
135
+ all; neither writes a tombstone, and neither hides the session.
136
+
137
+ ## The identity axis: why not the content hash, and what evades this
138
+
139
+ `identity.ts` keys a session as `source:native-id`, and the native id is derived
140
+ from the transcript's own content (`normalizer.ts`: `canonicalRecords[0].
141
+ source_group_id || item.id`), not from the path. That is what makes an id-keyed
142
+ tombstone hold where a content-keyed one would not:
143
+
144
+ - **A copy at another path** carries the same embedded id. Content-keyed or
145
+ path-keyed suppression misses it; the committed-id axis refuses it. Measured
146
+ and pinned in `tombstones.test.ts` ("a copy at another path — even under
147
+ another listing id — is refused by the committed id"): the same file copied
148
+ under a different name, discovered under a different listing id, is read,
149
+ normalized, and then refused by the store, leaving no rows, no cursor, and no
150
+ session behind.
151
+ - **An appended source** (the normal case for these harnesses) changes the
152
+ content hash — `session-hash.ts`'s v2 digest is explicitly *extendable*, which
153
+ is a feature for dedup and poison for a tombstone: a content-keyed tombstone
154
+ would be evaded by the next chunk. The id does not move, so the refusal holds.
155
+ - **A source that was rewritten** keeps the same id when it grows, and changes it
156
+ when the harness writes a genuinely different session — at which point this
157
+ store's own identity rules say it *is* a different session, and no
158
+ content-keyed rule could have said otherwise without inventing an identity the
159
+ source does not carry (and `AGENTS.md` forbids inferring identity from
160
+ convenience).
161
+
162
+ **What evades the axis** (stated, not hidden): a copy whose *embedded* id was
163
+ rewritten — a re-export with a fresh uuid, or a merge tool that renumbers
164
+ sessions — re-enters as a new session, with the deleted one still hidden. Nothing
165
+ in this store's identity model can call those the same session, and a
166
+ content-keyed rule would trade this narrow evasion for a wide one (any append
167
+ resurrects). A path-keyed rule would trade it for silent over-blocking of shared
168
+ files. Both trades are worse; the note records the boundary instead of pretending
169
+ there is none.
170
+
171
+ ## The transaction, and the invariants it leaves
172
+
173
+ `tombstoneSession` is one `BEGIN IMMEDIATE` transaction:
174
+
175
+ - `suppress` (default): insert one row (`session_id` PK, `source`, `native_id`,
176
+ `source_path` for the audit, `mode`, `reason`, `record_count`, `deleted_at`),
177
+ clear the queued admissions of the listings that fed the session (matched by the
178
+ tombstone's own native id and by the listings its cursors name) and any staged
179
+ replacement of it, commit. Nothing else is removed; a crash leaves the row
180
+ present or absent.
181
+ - `hard`: the same insert, then the session's content in the same transaction —
182
+ `records_fts`, `records_echo`, `records_embeddings`, `ingest_diagnostics`,
183
+ staged replacements, `records`, `sessions` (whose delete trigger removes the
184
+ `titles_fts` row). The deletes are explicit rather than left to the foreign
185
+ keys, the same stance `trg_sessions_diagnostics_delete` takes: a writer that
186
+ never enabled foreign keys must still leave nothing behind.
187
+ - Repeat calls are idempotent and never rewrite the audit: a second `suppress`,
188
+ or a `--hard` on an already-hard row, reports the existing deletion and changes
189
+ nothing (`created: false`). A `--hard` on a `suppress` row is the one
190
+ escalation, and it keeps the original `deleted_at` and `reason` — the same
191
+ deletion carried further, not a second one.
192
+ - **Two ordered writes, never one**: Wake writes its tombstone in the same
193
+ transaction as its deletes (`db.rs#L537-L571`, insert at `#L565`), and this does
194
+ the same. Neither store can be caught with the content gone and the source free
195
+ to re-admit it.
196
+
197
+ The invariants the derived-index gates name still hold after either mode, because
198
+ soft deletion touches no derived row and hard deletion removes a record and its
199
+ index row in the same count: the rebuild's arithmetic (`records − excluded =
200
+ indexed`) and the title parity (`sessions with a title = titles_fts rows`) both
201
+ pass on a store that was force-re-derived after a hard delete (measured: 23.8 s
202
+ over the production-scale copy). **`FTS_FORMAT` does not move — it stays `"3"`**,
203
+ and that is a decision with a reason:
204
+
205
+ - The projection `records → records_fts` is unchanged. A tombstone is not a rule
206
+ over canonical record fields; it is user-authored state about a *session*. Rule
207
+ version counters exist so a *rule change* costs one re-derivation (C2's note),
208
+ and there is no rule change here.
209
+ - Hiding inside the index would be erased by the next rule change anyway: the
210
+ rebuild is a whole-table re-projection from `records` (db.ts, `rebuildRecordsFts`),
211
+ so an index-only deletion survives only until someone bumps the format — at
212
+ which point a "deleted" session would quietly return to search unless the
213
+ rebuild also learned the tombstone rule, which would make `records_echo` (the
214
+ echo filter's derived set) carry two unrelated derivations and make the status
215
+ line `echo_filtered` mean two different things.
216
+ - The gates would have to count policy state as if it were data: the parity
217
+ assertion is `records − excluded = indexed`, so a suppression subtraction would
218
+ need its own term, and the index would stop being equal to its derivation.
219
+
220
+ ## Review fixes (2026-09-16)
221
+
222
+ Two P3s from the adversarial review, both fixed on this tree with regression
223
+ tests:
224
+
225
+ 1. **The verb's open could still create a store.** The read-gate classification
226
+ and the writer open are two statements, and a store that vanished (or was
227
+ replaced by an empty file) between them was *created* by the open —
228
+ `delete --list` would even report an empty readout from a store it had just
229
+ conjured, against this verb's documented "nothing is created or adopted".
230
+ Fixed by making the open itself refuse to initialize anything:
231
+ `new TrajectoryDb(path, { requireExisting: true })` opens with file creation
232
+ disabled (`{ readwrite: true, create: false }`, which is the only bun:sqlite
233
+ spelling for "writable, never create") and checks the shape *before* any
234
+ PRAGMA or DDL — `PRAGMA journal_mode = WAL` alone writes to a file, so the
235
+ check has to come first. A missing, empty, foreign, or newly-vanished store
236
+ raises `StoreNotInitializedError`, which the verb maps to the read verbs'
237
+ `STORE_NOT_INITIALIZED`. The option is opt-in: every other writer verb still
238
+ creates and migrates its store.
239
+ *Tests:* `a writer told not to initialize one refuses missing, empty, and
240
+ foreign files without an artifact` (missing → no file; empty → still zero
241
+ bytes; foreign → its own tables intact; classification-passes-then-file-
242
+ vanishes → refused, nothing created; a plain open still creates).
243
+ 2. **The guards ran outside the transactions they protect.** `saveSession`,
244
+ `appendSessionRecords` and `insertRecordEmbeddings` checked the tombstone and
245
+ then opened their write transactions, so a deletion committed by another
246
+ writer in between was lost: the write landed *under* the tombstone and
247
+ rewrote the session's records while the deletion's `record_count` named a
248
+ generation that no longer existed. Each of those methods is now one
249
+ `BEGIN IMMEDIATE` transaction with its check inside it (nested calls join it
250
+ as a savepoint, exactly as `commitIngest`'s transaction already did), so the
251
+ write lock is the serialization point and the check is a verdict rather than a
252
+ guess.
253
+ *Test:* `a tombstone from a second writer is honored by a write already in
254
+ flight` — a second process (`tests/tombstone-race-holder.ts`) holds the write
255
+ lock with an uncommitted tombstone and announces it; the parent's
256
+ `saveSession` then runs inside that window and must refuse, with the session's
257
+ rows untouched. Verified to discriminate: emulating the pre-fix ordering in a
258
+ throwaway probe (guard as a plain read outside the transaction) shows the
259
+ guard seeing *nothing* and the write landing anyway
260
+ (`content_hash = "pre-fix-write"`), which is the failure the test now pins.
261
+
262
+ Round two (2026-09-16) added: the **version collision with PR #52** — the step was
263
+ renumbered to 15 with a ladder that enters from 13 and from 14 (see *Version
264
+ collision* below, and the reciprocal obligation in *Still open*); this note's
265
+ suite count, which lagged the tree by two cases (597 → 599, and 601 once this
266
+ round's own two cases landed); and a **lineage
267
+ honesty** case at the serving level — a deleted parent leaves serving while its
268
+ children keep `parent_session_id` and lose only the `parent_title` that has to
269
+ resolve through it, in both modes, with the child's row untouched either way.
270
+ That last case is also why the escalation's completeness is asserted where it is:
271
+ `hard` removes the deleted session's own row (the tombstone's snapshot of source,
272
+ native id, path, and record count is what keeps the audit meaningful without it)
273
+ and nothing else.
274
+
275
+ ## Surface
276
+
277
+ - **CLI.** `ledger delete <session-id> --reason <text> [--hard] [--db <path>]
278
+ [--robot|--json]`, and `ledger delete --list`. It takes the writer lease, like
279
+ every other verb that writes (`holder: "delete"`), so a deletion cannot
280
+ interleave with a pass that is re-admitting the same session and a refused
281
+ deletion leaves the store as it found it. `--reason` is required: the row is
282
+ the mechanism's account of why, and a tombstone that cannot explain itself is a
283
+ hole in the record the audit exists to be. Text output says which mode the
284
+ deletion took and what it left behind; robot output is one
285
+ `{"type":"tombstone", …}` object (JSONL for `--list`). An id the store does not
286
+ hold is refused with `SESSION_NOT_FOUND` — a tombstone for a session that was
287
+ never ingested would be an *exclusion rule*, a different mechanism with
288
+ different consequences, deliberately not shipped as a side effect of this verb.
289
+ - **Status.** `tombstones` (sessions) and `tombstoned_records` (records still
290
+ held behind a `suppress` deletion) in `status --robot`/`--json`, and a text
291
+ line: `Tombstones: N sessions deleted by the user; M of the records above are
292
+ held back from search and reads`. Both are `0`, never null, for a store that
293
+ predates the table — a store no build could have written a tombstone into has
294
+ an empty set, not an unknown one.
295
+ - **Library.** `TrajectoryDb.tombstoneSession(id, {reason, hard?, now?})` →
296
+ `TombstoneOutcome` (`tombstone`, `created`, `escalated`, `recordsRemoved`),
297
+ `getTombstone(id)`, `tombstoneForListing({source, id})`, `listTombstones()`;
298
+ `SessionNotFoundError` and `TombstonedSessionError` (both with a `code`); the
299
+ `TombstoneRecord`/`TombstoneMode` row types. `TrajectoryReader` gains no write
300
+ method and no tombstone method: it serves, and it serves filtered.
301
+ - **Version collision with PR #52.** The step was designed as 13 → 14 and had to
302
+ be renumbered before this landed: the owner's public-alpha branch (#52) claims
303
+ 14 for `source_discovery`, and one store carries one `user_version`, so
304
+ whichever branch's code ran second would have found the number already at its
305
+ target — and skipped the other's `CREATE_TABLES_SQL` upgrade, because nothing
306
+ in the ladder re-runs DDL for a version it is already at. The boring correct
307
+ fix is not to number a diff but to say what the step does: this build claims
308
+ **15** and *enters the ladder from both shapes* — 13 (neither table) and 14
309
+ (#52's table, ours missing) — adding what it knows through the same idempotent
310
+ `CREATE_TABLES_SQL` and leaving unknown tables alone. A store migrated by
311
+ either branch therefore converges from either direction, and #52's ladder needs
312
+ the same two-sided treatment (recorded here for the owner; the note's
313
+ *Still open* list carries it). 14 is reserved, not skipped.
314
+ - **Schema.** `SCHEMA_VERSION` 13 → 15; `session_tombstones` plus an index on
315
+ `(source, native_id)`; no dedicated migration constant — like v12, the table
316
+ arrives from the same `CREATE_TABLES_SQL` a writer open already runs, and no row
317
+ is fabricated. The table is deliberately **not** in `REQUIRED_READ_SCHEMA`: the
318
+ read path probes for it and skips the predicate when it is absent, which is
319
+ exact rather than tolerant — a store without the table is a store no build
320
+ could have written a deletion into, and a store *with* tombstones can never be
321
+ read unfiltered by an older build, because the read gate refuses a store whose
322
+ `user_version` is newer than the build supports. One `sqlite_master` lookup per
323
+ serving call (the same shape the title lane already pays), not per row.
324
+
325
+ ## Decisions
326
+
327
+ 1. **Suppress by default, `--hard` behind an explicit flag.** `AGENTS.md`'s
328
+ "preserve raw evidence" and "source truncation is not authorization to erase
329
+ captured evidence" both point one way: the mechanism must not make deletion the
330
+ one write that destroys what an adapter already attributed, and the audit row is
331
+ more useful beside the rows it names than instead of them. Wake's own shape is
332
+ the opposite (its removal is the index's removal) because its store *is* the
333
+ index; Ledger is a system of record, so its deletion removes a session from
334
+ service, not from history. The privacy posture is still real: the user asked for
335
+ it to be gone from search and from re-import, and it is — everywhere, in every
336
+ mode. `--hard` exists for the case where the request is "not on this disk
337
+ either", and it says what it costs.
338
+ 2. **A tombstone is not an exclusion rule, and the verb never creates a
339
+ store.** Only an existing session can be deleted; an unknown id is refused,
340
+ a `--db` path with no store (or one that is not a Ledger store at all) is
341
+ classified by the read gate and refused with the read verbs' own codes and
342
+ wording instead of a writer open initializing or adopting the file — creating
343
+ stores is `sync`'s and `build`'s job. The canonical executable resolves its
344
+ XDG default and refuses there too, because `delete` is deliberately not one of
345
+ the commands that create the data directory. "Never ingest this" is a policy artifact
346
+ (a deny list) with different lifetime, scope, and failure semantics, and
347
+ smuggling it into the deletion verb would make an operator's typo silently
348
+ permanent.
349
+ 3. **Status reports holdings and discloses the deletion.**
350
+ `countSessions`/`countRecords` are what the store *holds*, tombstones
351
+ included; subtracting them would claim the store holds less than it does and
352
+ would hide exactly the data the audit decision keeps. The disclosure is two
353
+ fields and a sentence. This is the enforcement point that is *not* a filter,
354
+ and the reason is the decision above it: a soft deletion retains, so a count
355
+ that "looks deleted" would be a lie the rest of the mechanism doesn't tell.
356
+ 4. **`--reason` is required.** The row exists to explain the deletion months
357
+ later; the flag costs one sentence. This is a mechanism default, not a policy
358
+ position (see the non-goal below): if the owner's policy says a deletion may be
359
+ unexplained, relaxing it is a one-line change and the row keeps a timestamp.
360
+ 5. **No restore verb this cycle.** The tombstone is plain rows in one table, so
361
+ restoring is one `DELETE FROM session_tombstones` — but *who may undelete* is
362
+ the same policy question as who may delete, and shipping a one-flag escape from
363
+ an audit row would undercut the audit. Recorded as a boundary, not an oversight.
364
+ 6. **A hard deletion keeps no copy of what it removed** — no staged replacement,
365
+ no queue entry with the session's title, no diagnostics — because "irreversibly
366
+ removed" has to mean something. What it keeps is the tombstone: the axis that
367
+ stops the source from putting it back, and the audit that says it happened.
368
+
369
+ ## Policy: an explicit non-goal
370
+
371
+ This cycle ships the **mechanism** and says so in the verb's own help and its
372
+ `docs/api-surface.md` section: who may delete, how long a record may be kept,
373
+ whether a deletion must be explained, what a deletion means for a *published*
374
+ derivative, and whether a restore exists are the retention/privacy policy this
375
+ repository has not decided. Wake's product answers some of that outside its store
376
+ — its delete is a desktop action that moves the session files to the system Trash
377
+ and then tombstones them (`crates/wake/src/workbench.rs#L4594-L4609`) — and the
378
+ program's own row for C7 names "explicit delete semantics before anyone trusts us
379
+ with their history" as the buy, not a retention schedule. The mechanism is
380
+ deliberately narrower than the policy so the policy can still be chosen; what it
381
+ must not be is *undefined*, which is what "deletion today is undefined" meant.
382
+
383
+ ## The roadmap row
384
+
385
+ > **C7** Tombstone double-track (path + logical key, same transaction as removal)
386
+ > — *Fixes/buys:* deletion today is undefined; source truncation is not
387
+ > authorization to erase, and user deletion must survive re-scans. *Open-source
388
+ > case:* privacy posture: explicit delete semantics before anyone trusts us with
389
+ > their history; Wake's tombstones + system Trash is the reference UX.
390
+ > (`docs/research/2026-09-16-wake-port-program.md`)
391
+
392
+ Ported: the tombstones table with its axis pair and its same-transaction rule, the
393
+ double-track admission filter at enumeration, `absence ≠ deletion`, and the tested
394
+ pair itself (`crates/wake-core/tests/db_roundtrip.rs#L82-L113` pins
395
+ `is_tombstoned` by path and `is_key_tombstoned` by key). Adapted: the path axis
396
+ becomes the store's cursor link (see above — Ledger's paths are not listing
397
+ identity), the logical key becomes the committed session id, and the removal is
398
+ split: suppression by default, record removal behind `--hard`. Not ported: the
399
+ system-Trash half of Wake's UX (a CLI has no Trash), and the UI affordances
400
+ (confirmation prose, per-platform wording).
401
+
402
+ ## Files
403
+
404
+ - `packages/core/src/schema.ts` — `SCHEMA_VERSION` 15 (the ladder's hole at 14 is
405
+ PR #52's `source_discovery`; both 13 and 14 enter here); the `session_tombstones`
406
+ DDL and its index, with the axes, the mode split, and the `FTS_FORMAT`
407
+ argument written where the table is.
408
+ - `packages/core/src/db.ts` — `tombstoneSession`, `getTombstone`,
409
+ `tombstoneForListing`, `listTombstones`, `TombstoneOutcome`,
410
+ `IngestCommitOutcome`, `SessionNotFoundError`, `TombstonedSessionError`; the
411
+ write-boundary guards; the admission-state cleanup; the raw-row view the writer
412
+ needs (`sessionRowById`) separated from the serving view (`getSession`).
413
+ - `packages/core/src/reader.ts` — `tombstonePredicate` and its five call sites
414
+ (record lane, title lane, vector lane, session reads, record reads), plus the
415
+ holdings note on `countSessions`/`countRecords`.
416
+ - `packages/core/src/ingest.ts` — `IngestKind` grows `tombstoned`; the listing
417
+ admission check in `sessionNeedsIngest` and `ingestDiscoveredSession`; the
418
+ commit refusal converted to the same verdict (`tombstonedIngestResult`).
419
+ - `packages/core/src/cli.ts` — `handleDelete` (verb, `--hard`, `--list`,
420
+ `--reason`), the typed `SESSION_NOT_FOUND` refusal, `status` fields and text,
421
+ `build`'s `tombstoned` accounting, help text.
422
+ - `packages/core/src/store-status.ts` — `tombstones` / `tombstonedRecords`.
423
+ - `packages/core/src/sync.ts` — a listing the store refused leaves the queue
424
+ (instead of returning as a retry every pass).
425
+ - `packages/core/src/watch.ts` — the `tombstoned` event kind, counted with skips.
426
+ - `packages/core/src/types.ts`, `index.ts`, `docs/api-surface.md`, `README.md`,
427
+ `packages/core/README.md` — the row types, the exported refusals, the documented
428
+ surface (verb, flags, robot shapes, table, ADR entry, `status` fields). The
429
+ agent-facing skill (`skills/ledger/SKILL.md`) is deliberately *not* told that a
430
+ delete verb exists: whether an agent may delete is exactly the policy question
431
+ this cycle leaves open.
432
+ - `packages/core/tests/tombstones.test.ts` (new, 14 cases) — serving, holdings,
433
+ the audit and its idempotence/escalation, the two admission axes, the copy,
434
+ absence and truncation, `--hard` completeness and the index gates after it, the
435
+ staged-replacement cleanup, the pre-v15 shape, the verb end-to-end (including a
436
+ `--db` path that holds no store and one that holds a foreign SQLite file: both
437
+ refused, nothing created or adopted), the `requireExisting` open's artifact-free
438
+ refusals (missing, empty, foreign, vanished), the two-writer race whose outcome
439
+ the guards' placement decides, and lineage honesty for a deleted parent in both
440
+ modes (children keep `parent_session_id`, the parent's `parent_title` stops
441
+ resolving, and nothing reaches the child's row).
442
+ - `packages/core/tests/tombstone-race-holder.ts` (new) — the second writer that
443
+ race test needs: it holds the write lock with an uncommitted tombstone and
444
+ announces it on stdout.
445
+ - `packages/core/tests/integration-migrations.test.ts` — the final version is 15,
446
+ plus the collision case in test form: a store shaped at 14 (PR #52's
447
+ `source_discovery` table, no tombstone table) migrates to 15, gains the
448
+ tombstone table, and leaves the other branch's table alone.
449
+
450
+ ## Still open (recorded, not hidden)
451
+
452
+ - **PR #52's inverted ladder is closed on this combined build.** C7 claimed 15
453
+ and accepts a store at 13 or 14; the stabilization branch now writes 15 as
454
+ well, accepts an already-v15 store that has `session_tombstones`, and still
455
+ acquires `source_discovery` from the same idempotent DDL (pinned by
456
+ `integration-migrations.test.ts` in both directions). Remaining items below
457
+ are unchanged C7 leftovers.
458
+ - A member listing of a deleted *grouped* session is refused at the store
459
+ boundary, which means one read of that file on the first pass that discovers it
460
+ (and none after: the pass drops the listing). Closing it needs the group
461
+ mapping at admission time, i.e. reading the file — the current split (cheap axis
462
+ first, authoritative axis at the write) is the honest ordering, and the
463
+ alternative would move a deletion verdict into the ingest state machine.
464
+ - Rewriting a source's embedded id evades the axis (above). A deny list is the
465
+ mechanism that would cover it, and it is policy.
466
+ - `--hard` on a session with a very large record count is not fast (7.6 s for
467
+ 32,583 records); it is a single transaction, so it is atomic, not incremental.