@smthrs/journal 0.0.0-stage → 1.0.0-rc.13

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (257) hide show
  1. package/CHANGELOG.md +149 -0
  2. package/LICENSE +21 -0
  3. package/README.md +208 -2
  4. package/dist/cjs/CallFact.d.ts +129 -0
  5. package/dist/cjs/CallFact.d.ts.map +1 -0
  6. package/dist/cjs/CallFact.js +106 -0
  7. package/dist/cjs/CallFact.js.map +7 -0
  8. package/dist/cjs/Consensus.d.ts +492 -0
  9. package/dist/cjs/Consensus.d.ts.map +1 -0
  10. package/dist/cjs/Consensus.js +314 -0
  11. package/dist/cjs/Consensus.js.map +7 -0
  12. package/dist/cjs/EngineEvent.d.ts +890 -0
  13. package/dist/cjs/EngineEvent.d.ts.map +1 -0
  14. package/dist/cjs/EngineEvent.js +383 -0
  15. package/dist/cjs/EngineEvent.js.map +7 -0
  16. package/dist/cjs/ExecutionFact.d.ts +216 -0
  17. package/dist/cjs/ExecutionFact.d.ts.map +1 -0
  18. package/dist/cjs/ExecutionFact.js +303 -0
  19. package/dist/cjs/ExecutionFact.js.map +7 -0
  20. package/dist/cjs/Journal.d.ts +617 -0
  21. package/dist/cjs/Journal.d.ts.map +1 -0
  22. package/dist/cjs/Journal.js +190 -0
  23. package/dist/cjs/Journal.js.map +7 -0
  24. package/dist/cjs/JournalEvent.d.ts +421 -0
  25. package/dist/cjs/JournalEvent.d.ts.map +1 -0
  26. package/dist/cjs/JournalEvent.js +281 -0
  27. package/dist/cjs/JournalEvent.js.map +7 -0
  28. package/dist/cjs/JournalGeneration.d.ts +34 -0
  29. package/dist/cjs/JournalGeneration.d.ts.map +1 -0
  30. package/dist/cjs/JournalGeneration.js +73 -0
  31. package/dist/cjs/JournalGeneration.js.map +7 -0
  32. package/dist/cjs/JournalMetrics.d.ts +72 -0
  33. package/dist/cjs/JournalMetrics.d.ts.map +1 -0
  34. package/dist/cjs/JournalMetrics.js +55 -0
  35. package/dist/cjs/JournalMetrics.js.map +7 -0
  36. package/dist/cjs/Migrations.d.ts +42 -0
  37. package/dist/cjs/Migrations.d.ts.map +1 -0
  38. package/dist/cjs/Migrations.js +58 -0
  39. package/dist/cjs/Migrations.js.map +7 -0
  40. package/dist/cjs/OwnerId.d.ts +40 -0
  41. package/dist/cjs/OwnerId.d.ts.map +1 -0
  42. package/dist/cjs/OwnerId.js +40 -0
  43. package/dist/cjs/OwnerId.js.map +7 -0
  44. package/dist/cjs/Projection.d.ts +29 -0
  45. package/dist/cjs/Projection.d.ts.map +1 -0
  46. package/dist/cjs/Projection.js +25 -0
  47. package/dist/cjs/Projection.js.map +7 -0
  48. package/dist/cjs/RedactedLogger.d.ts +121 -0
  49. package/dist/cjs/RedactedLogger.d.ts.map +1 -0
  50. package/dist/cjs/RedactedLogger.js +235 -0
  51. package/dist/cjs/RedactedLogger.js.map +7 -0
  52. package/dist/cjs/Redaction.d.ts +350 -0
  53. package/dist/cjs/Redaction.d.ts.map +1 -0
  54. package/dist/cjs/Redaction.js +962 -0
  55. package/dist/cjs/Redaction.js.map +7 -0
  56. package/dist/cjs/SqlConsensus.d.ts +44 -0
  57. package/dist/cjs/SqlConsensus.d.ts.map +1 -0
  58. package/dist/cjs/SqlConsensus.js +273 -0
  59. package/dist/cjs/SqlConsensus.js.map +7 -0
  60. package/dist/cjs/SqlJournal.d.ts +173 -0
  61. package/dist/cjs/SqlJournal.d.ts.map +1 -0
  62. package/dist/cjs/SqlJournal.js +1568 -0
  63. package/dist/cjs/SqlJournal.js.map +7 -0
  64. package/dist/cjs/StepFact.d.ts +66 -0
  65. package/dist/cjs/StepFact.d.ts.map +1 -0
  66. package/dist/cjs/StepFact.js +68 -0
  67. package/dist/cjs/StepFact.js.map +7 -0
  68. package/dist/cjs/index.d.ts +95 -0
  69. package/dist/cjs/index.d.ts.map +1 -0
  70. package/dist/cjs/index.js +63 -0
  71. package/dist/cjs/index.js.map +7 -0
  72. package/dist/cjs/internal/JournalInternalFault.d.ts +37 -0
  73. package/dist/cjs/internal/JournalInternalFault.d.ts.map +1 -0
  74. package/dist/cjs/internal/JournalInternalFault.js +43 -0
  75. package/dist/cjs/internal/JournalInternalFault.js.map +7 -0
  76. package/dist/cjs/migrations/0001_initial.d.ts +15 -0
  77. package/dist/cjs/migrations/0001_initial.d.ts.map +1 -0
  78. package/dist/cjs/migrations/0001_initial.js +54 -0
  79. package/dist/cjs/migrations/0001_initial.js.map +7 -0
  80. package/dist/cjs/migrations/0002_checkpoints.d.ts +27 -0
  81. package/dist/cjs/migrations/0002_checkpoints.d.ts.map +1 -0
  82. package/dist/cjs/migrations/0002_checkpoints.js +48 -0
  83. package/dist/cjs/migrations/0002_checkpoints.js.map +7 -0
  84. package/dist/cjs/migrations/0003_startup_index.d.ts +15 -0
  85. package/dist/cjs/migrations/0003_startup_index.d.ts.map +1 -0
  86. package/dist/cjs/migrations/0003_startup_index.js +40 -0
  87. package/dist/cjs/migrations/0003_startup_index.js.map +7 -0
  88. package/dist/cjs/migrations/0004_dedup.d.ts +15 -0
  89. package/dist/cjs/migrations/0004_dedup.d.ts.map +1 -0
  90. package/dist/cjs/migrations/0004_dedup.js +57 -0
  91. package/dist/cjs/migrations/0004_dedup.js.map +7 -0
  92. package/dist/cjs/migrations/0005_run_event_type.d.ts +15 -0
  93. package/dist/cjs/migrations/0005_run_event_type.d.ts.map +1 -0
  94. package/dist/cjs/migrations/0005_run_event_type.js +40 -0
  95. package/dist/cjs/migrations/0005_run_event_type.js.map +7 -0
  96. package/dist/cjs/migrations/0006_consensus.d.ts +22 -0
  97. package/dist/cjs/migrations/0006_consensus.d.ts.map +1 -0
  98. package/dist/cjs/migrations/0006_consensus.js +84 -0
  99. package/dist/cjs/migrations/0006_consensus.js.map +7 -0
  100. package/dist/cjs/package.json +1 -0
  101. package/dist/cjs/test/Notifying.d.ts +65 -0
  102. package/dist/cjs/test/Notifying.d.ts.map +1 -0
  103. package/dist/cjs/test/Notifying.js +62 -0
  104. package/dist/cjs/test/Notifying.js.map +7 -0
  105. package/dist/cjs/test/TestJournal.d.ts +43 -0
  106. package/dist/cjs/test/TestJournal.d.ts.map +1 -0
  107. package/dist/cjs/test/TestJournal.js +43 -0
  108. package/dist/cjs/test/TestJournal.js.map +7 -0
  109. package/dist/esm/CallFact.d.ts +129 -0
  110. package/dist/esm/CallFact.d.ts.map +1 -0
  111. package/dist/esm/CallFact.js +96 -0
  112. package/dist/esm/CallFact.js.map +1 -0
  113. package/dist/esm/Consensus.d.ts +492 -0
  114. package/dist/esm/Consensus.d.ts.map +1 -0
  115. package/dist/esm/Consensus.js +548 -0
  116. package/dist/esm/Consensus.js.map +1 -0
  117. package/dist/esm/EngineEvent.d.ts +890 -0
  118. package/dist/esm/EngineEvent.d.ts.map +1 -0
  119. package/dist/esm/EngineEvent.js +614 -0
  120. package/dist/esm/EngineEvent.js.map +1 -0
  121. package/dist/esm/ExecutionFact.d.ts +216 -0
  122. package/dist/esm/ExecutionFact.d.ts.map +1 -0
  123. package/dist/esm/ExecutionFact.js +374 -0
  124. package/dist/esm/ExecutionFact.js.map +1 -0
  125. package/dist/esm/Journal.d.ts +617 -0
  126. package/dist/esm/Journal.d.ts.map +1 -0
  127. package/dist/esm/Journal.js +323 -0
  128. package/dist/esm/Journal.js.map +1 -0
  129. package/dist/esm/JournalEvent.d.ts +421 -0
  130. package/dist/esm/JournalEvent.d.ts.map +1 -0
  131. package/dist/esm/JournalEvent.js +517 -0
  132. package/dist/esm/JournalEvent.js.map +1 -0
  133. package/dist/esm/JournalGeneration.d.ts +34 -0
  134. package/dist/esm/JournalGeneration.d.ts.map +1 -0
  135. package/dist/esm/JournalGeneration.js +71 -0
  136. package/dist/esm/JournalGeneration.js.map +1 -0
  137. package/dist/esm/JournalMetrics.d.ts +72 -0
  138. package/dist/esm/JournalMetrics.d.ts.map +1 -0
  139. package/dist/esm/JournalMetrics.js +79 -0
  140. package/dist/esm/JournalMetrics.js.map +1 -0
  141. package/dist/esm/Migrations.d.ts +42 -0
  142. package/dist/esm/Migrations.d.ts.map +1 -0
  143. package/dist/esm/Migrations.js +59 -0
  144. package/dist/esm/Migrations.js.map +1 -0
  145. package/dist/esm/OwnerId.d.ts +40 -0
  146. package/dist/esm/OwnerId.d.ts.map +1 -0
  147. package/dist/esm/OwnerId.js +33 -0
  148. package/dist/esm/OwnerId.js.map +1 -0
  149. package/dist/esm/Projection.d.ts +29 -0
  150. package/dist/esm/Projection.d.ts.map +1 -0
  151. package/dist/esm/Projection.js +13 -0
  152. package/dist/esm/Projection.js.map +1 -0
  153. package/dist/esm/RedactedLogger.d.ts +121 -0
  154. package/dist/esm/RedactedLogger.d.ts.map +1 -0
  155. package/dist/esm/RedactedLogger.js +445 -0
  156. package/dist/esm/RedactedLogger.js.map +1 -0
  157. package/dist/esm/Redaction.d.ts +350 -0
  158. package/dist/esm/Redaction.d.ts.map +1 -0
  159. package/dist/esm/Redaction.js +1628 -0
  160. package/dist/esm/Redaction.js.map +1 -0
  161. package/dist/esm/SqlConsensus.d.ts +44 -0
  162. package/dist/esm/SqlConsensus.d.ts.map +1 -0
  163. package/dist/esm/SqlConsensus.js +273 -0
  164. package/dist/esm/SqlConsensus.js.map +1 -0
  165. package/dist/esm/SqlJournal.d.ts +173 -0
  166. package/dist/esm/SqlJournal.d.ts.map +1 -0
  167. package/dist/esm/SqlJournal.js +1858 -0
  168. package/dist/esm/SqlJournal.js.map +1 -0
  169. package/dist/esm/StepFact.d.ts +66 -0
  170. package/dist/esm/StepFact.d.ts.map +1 -0
  171. package/dist/esm/StepFact.js +50 -0
  172. package/dist/esm/StepFact.js.map +1 -0
  173. package/dist/esm/index.d.ts +95 -0
  174. package/dist/esm/index.d.ts.map +1 -0
  175. package/dist/esm/index.js +95 -0
  176. package/dist/esm/index.js.map +1 -0
  177. package/dist/esm/internal/JournalInternalFault.d.ts +37 -0
  178. package/dist/esm/internal/JournalInternalFault.d.ts.map +1 -0
  179. package/dist/esm/internal/JournalInternalFault.js +28 -0
  180. package/dist/esm/internal/JournalInternalFault.js.map +1 -0
  181. package/dist/esm/migrations/0001_initial.d.ts +15 -0
  182. package/dist/esm/migrations/0001_initial.d.ts.map +1 -0
  183. package/dist/esm/migrations/0001_initial.js +32 -0
  184. package/dist/esm/migrations/0001_initial.js.map +1 -0
  185. package/dist/esm/migrations/0002_checkpoints.d.ts +27 -0
  186. package/dist/esm/migrations/0002_checkpoints.d.ts.map +1 -0
  187. package/dist/esm/migrations/0002_checkpoints.js +38 -0
  188. package/dist/esm/migrations/0002_checkpoints.js.map +1 -0
  189. package/dist/esm/migrations/0003_startup_index.d.ts +15 -0
  190. package/dist/esm/migrations/0003_startup_index.d.ts.map +1 -0
  191. package/dist/esm/migrations/0003_startup_index.js +18 -0
  192. package/dist/esm/migrations/0003_startup_index.js.map +1 -0
  193. package/dist/esm/migrations/0004_dedup.d.ts +15 -0
  194. package/dist/esm/migrations/0004_dedup.d.ts.map +1 -0
  195. package/dist/esm/migrations/0004_dedup.js +38 -0
  196. package/dist/esm/migrations/0004_dedup.js.map +1 -0
  197. package/dist/esm/migrations/0005_run_event_type.d.ts +15 -0
  198. package/dist/esm/migrations/0005_run_event_type.d.ts.map +1 -0
  199. package/dist/esm/migrations/0005_run_event_type.js +18 -0
  200. package/dist/esm/migrations/0005_run_event_type.js.map +1 -0
  201. package/dist/esm/migrations/0006_consensus.d.ts +22 -0
  202. package/dist/esm/migrations/0006_consensus.d.ts.map +1 -0
  203. package/dist/esm/migrations/0006_consensus.js +69 -0
  204. package/dist/esm/migrations/0006_consensus.js.map +1 -0
  205. package/dist/esm/test/Notifying.d.ts +65 -0
  206. package/dist/esm/test/Notifying.d.ts.map +1 -0
  207. package/dist/esm/test/Notifying.js +62 -0
  208. package/dist/esm/test/Notifying.js.map +1 -0
  209. package/dist/esm/test/TestJournal.d.ts +43 -0
  210. package/dist/esm/test/TestJournal.d.ts.map +1 -0
  211. package/dist/esm/test/TestJournal.js +27 -0
  212. package/dist/esm/test/TestJournal.js.map +1 -0
  213. package/docs/README.md +166 -0
  214. package/docs/api.md +650 -0
  215. package/docs/concepts/compaction.md +157 -0
  216. package/docs/concepts/idempotency.md +127 -0
  217. package/docs/concepts/owner-fence.md +128 -0
  218. package/docs/concepts/redaction.md +222 -0
  219. package/docs/concepts/state-event-authority.md +159 -0
  220. package/docs/concepts/two-channels.md +167 -0
  221. package/docs/guides/commit-state-and-entry.md +114 -0
  222. package/docs/guides/compact-a-run.md +134 -0
  223. package/docs/guides/fold-a-projection.md +106 -0
  224. package/docs/guides/read-a-run.md +139 -0
  225. package/docs/guides/redact-log-output.md +94 -0
  226. package/docs/guides/testing.md +124 -0
  227. package/docs/guides/write-lifecycle-events.md +110 -0
  228. package/docs/installation.md +137 -0
  229. package/docs/quickstart.md +146 -0
  230. package/docs/troubleshooting.md +214 -0
  231. package/package.json +263 -3
  232. package/src/CallFact.ts +117 -0
  233. package/src/Consensus.ts +814 -0
  234. package/src/EngineEvent.ts +823 -0
  235. package/src/ExecutionFact.ts +433 -0
  236. package/src/Journal.ts +688 -0
  237. package/src/JournalEvent.ts +654 -0
  238. package/src/JournalGeneration.ts +83 -0
  239. package/src/JournalMetrics.ts +89 -0
  240. package/src/Migrations.ts +62 -0
  241. package/src/OwnerId.ts +42 -0
  242. package/src/Projection.ts +31 -0
  243. package/src/RedactedLogger.ts +483 -0
  244. package/src/Redaction.ts +1828 -0
  245. package/src/SqlConsensus.ts +384 -0
  246. package/src/SqlJournal.ts +2732 -0
  247. package/src/StepFact.ts +69 -0
  248. package/src/index.ts +109 -0
  249. package/src/internal/JournalInternalFault.ts +40 -0
  250. package/src/migrations/0001_initial.ts +41 -0
  251. package/src/migrations/0002_checkpoints.ts +46 -0
  252. package/src/migrations/0003_startup_index.ts +19 -0
  253. package/src/migrations/0004_dedup.ts +43 -0
  254. package/src/migrations/0005_run_event_type.ts +19 -0
  255. package/src/migrations/0006_consensus.ts +83 -0
  256. package/src/test/Notifying.ts +101 -0
  257. package/src/test/TestJournal.ts +52 -0
package/docs/api.md ADDED
@@ -0,0 +1,650 @@
1
+ ---
2
+ title: "API reference"
3
+ description: "Every public export of @smthrs/journal: the Journal service and its operations, the event envelope, the SQL layer options, redaction, projections, metrics, migrations, and the two test entry points."
4
+ ---
5
+
6
+ ```ts
7
+ import * as DurableWriter from "@smthrs/database/DurableWriter"
8
+ import * as NodeDatabase from "@smthrs/database/node/NodeDatabase"
9
+ import { Journal, JournalEvent, Migrations, SqlJournal } from "@smthrs/journal"
10
+ import * as Layer from "effect/Layer"
11
+
12
+ const database = Layer.provideMerge(
13
+ DurableWriter.layer(),
14
+ NodeDatabase.layer({ filename: "flows.db" })
15
+ )
16
+
17
+ const journal = SqlJournal.layer({ capacity: 1024, overflow: "reject" }).pipe(
18
+ Layer.provide(Layer.provideMerge(Migrations.layer, database))
19
+ )
20
+ ```
21
+
22
+ ## Entry points
23
+
24
+ | Import | Source | Platform |
25
+ | ---------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | -------- |
26
+ | `@smthrs/journal` | [src/index.ts](https://github.com/smithersai/smithers/blob/main/packages/smithers/flows/journal/src/index.ts) | any |
27
+ | `@smthrs/journal/test/TestJournal` | [src/test/TestJournal.ts](https://github.com/smithersai/smithers/blob/main/packages/smithers/flows/journal/src/test/TestJournal.ts) | Node |
28
+ | `@smthrs/journal/test/Notifying` | [src/test/Notifying.ts](https://github.com/smithersai/smithers/blob/main/packages/smithers/flows/journal/src/test/Notifying.ts) | any |
29
+
30
+ Every module in the root is also importable from `@smthrs/journal/<Module>`.
31
+ The root holds the journal and its contracts, written against the
32
+ driver-neutral `@smthrs/database` service, and it bundles for the browser. The
33
+ test doubles are kept out of it and live under explicit subpaths; of the two,
34
+ only `TestJournal` binds a Node SQLite database, which is what the platform
35
+ column above records. See
36
+ [platform support](/docs/reference/api/#platform-support).
37
+
38
+ ## Journal
39
+
40
+ `Journal.Journal` is the service tag; `Journal.Service` is its interface.
41
+ Fallible journal operations use `JournalError` with a stable `code`.
42
+
43
+ ### Operations
44
+
45
+ | Method | Signature | Behavior |
46
+ | --------------------- | -------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
47
+ | `emitLossy` | `(input: Input) => Effect<EmitReceipt, JournalError>` | bounded non-blocking queue; may return `Dropped` |
48
+ | `emitDurable` | `(input: Input, owner: OwnerId) => Effect<DurableReceipt, JournalError>` | allocates and commits inside the write transaction, fenced on `owner` |
49
+ | `emitDurableUnfenced` | `(input: Input) => Effect<DurableReceipt, JournalError>` | the same commit with no fence, for a genuinely ownerless admission |
50
+ | `transact` | `<A, E, R>(effect: Effect<A, E, R>) => Effect<A, E \| JournalError, R>` | runs a state projection and its `emitDurable` calls in one transaction |
51
+ | `whenCommitted` | `(update: Effect<void>) => Effect<boolean>` | accepts a short, non-failing update after managed commit, or runs it immediately outside a transaction; returns `false` without publishing when commit ownership is unknown |
52
+ | `stream` | `(options: StreamOptions) => Stream<Entry, JournalError>` | durable history, then committed changes; never completes |
53
+ | `entries` | `(options: EntriesOptions) => Effect<EntriesPage, JournalError>` | paged read |
54
+ | `changes` | `Effect<PubSub.Subscription<Entry>, never, Scope>` | post-commit publication, bounded and sliding |
55
+ | `project` | `<S, E, R>(projection: Projection<S, E, R>, options: StreamOptions) => Stream<S, JournalError, R>` | folds `stream` through a deterministic reducer |
56
+ | `flush` | `Effect<void, JournalError>` | barrier for the lossy queue |
57
+ | `checkpoint` | `(options: CheckpointOptions, owner: OwnerId) => Effect<Checkpoint, JournalError>` | durably captures replay state at a committed sequence, in `transact`'s discipline |
58
+ | `latestCheckpoint` | `(runId: RunId) => Effect<Option<Checkpoint>, JournalError>` | the resync point for a compacted run |
59
+ | `compact` | `(options: CompactOptions, owner: OwnerId) => Effect<Compacted, JournalError>` | truncates strictly below a checkpoint, atomically with the floor advance |
60
+ | `generation?` | `(runId: RunId) => Effect<{ generation: number; afterSeq: number }, JournalError>` | optional durable rewind generation and archive boundary; omission means generation zero |
61
+
62
+ `whenCommitted` returning `true` means accepted, possibly deferred. Failed
63
+ savepoints and transaction retries discard their registrations. See
64
+ [commit state and its entry together](./guides/commit-state-and-entry.md).
65
+
66
+ `owner` is mandatory on `emitDurable`, `checkpoint`, and `compact`. The insert,
67
+ the checkpoint replacement, and the truncation join the `Consensus` strategy's
68
+ `guard` inside their write transaction and land only while the strategy still
69
+ records that owner as holding the run, and otherwise fail `fence_lost`. An owner that is missing, null, or not an `OwnerId` at all fails
70
+ `invalid_event` instead: that is a caller contract violation, and reporting it
71
+ as `fence_lost` would send the caller hunting a race that never happened.
72
+ `emitDurableUnfenced` is the one sanctioned ownerless path, for an import or a
73
+ repair tool that owns no run. Reaching for it to dodge `fence_lost` writes
74
+ exactly the zombie entry the fence exists to reject.
75
+
76
+ `SqlJournal.layer` fences through [`SqlConsensus`](#sqlconsensus), whose lease
77
+ table this package's own migrations create; `SqlJournal.layerWith` fences
78
+ through any injected [`Consensus`](#consensus).
79
+
80
+ ### Errors
81
+
82
+ `JournalError` is a `Schema.TaggedError` with the tag
83
+ `"@smthrs/journal/JournalError"` and the fields `code`, `message`, optional
84
+ `cause`, and optional `checkpointSeq`.
85
+
86
+ | Member | Type | Meaning |
87
+ | --------------- | -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
88
+ | `code` | `JournalErrorCode` | the stable classification |
89
+ | `message` | `string` | a human-readable description |
90
+ | `cause` | `unknown` (optional) | the underlying failure, for storage codes |
91
+ | `checkpointSeq` | `Seq` (optional) | on `compacted`, the run's floor; on `reader_behind`, the checkpoint a compaction refused to truncate below; on `checkpoint_invalid`, the floor that refused the checkpoint |
92
+
93
+ `JournalErrorCode` is one of `invalid_event`, `idempotency_conflict`,
94
+ `sequence_conflict`, `fence_lost`, `queue_overflow`, `journal_closed`,
95
+ `sink_failed`, `read_failed`, `decode_failed`, `projection_failed`,
96
+ `checkpoint_invalid`, `reader_behind`, `compacted`, and `unknown`. Each one is
97
+ diagnosed in [Troubleshooting](./troubleshooting.md).
98
+
99
+ ### Receipts
100
+
101
+ | Export | Shape | Returned when |
102
+ | ---------------- | ------------------------------------ | -------------------------------------------------------------------------------- |
103
+ | `Accepted` | `{ _tag, seq, sourceSeq, evicted? }` | a new event was admitted; `evicted` counts what a `drop-oldest` policy displaced |
104
+ | `Duplicate` | `{ _tag, seq, sourceSeq, status }` | an exact producer retry; `status` is `"pending"` or `"committed"` |
105
+ | `Dropped` | `{ _tag, seq, sourceSeq, policy }` | a `drop-newest` policy discarded the event |
106
+ | `EmitReceipt` | `Accepted \| Duplicate \| Dropped` | the lossy channel's union |
107
+ | `DurableReceipt` | `Accepted \| Duplicate` | the durable channel's union; a dropped lifecycle event is unrepresentable |
108
+
109
+ ### Options and models
110
+
111
+ | Export | Shape |
112
+ | ------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
113
+ | `OverflowPolicy` | `"reject" \| "drop-newest" \| "drop-oldest"` |
114
+ | `StreamOptions` | `{ runId, afterSequence? }` |
115
+ | `EntriesOptions` | `{ runId, after?, eventTypes?, limit }`, `limit` in `1..maxEntriesLimit`; optional nonempty list of up to 64 exact types is filtered before pagination. Cursors stay canonical; compaction checks still apply. |
116
+ | `EntriesPage` | `{ entries: ReadonlyArray<Entry>, hasMore: boolean }` |
117
+ | `CheckpointOptions` | `{ runId, seq, state }` |
118
+ | `Checkpoint` | `{ runId, seq, state, createdAtMs, compactedAtMs }`, the last null until a compaction truncated below it |
119
+ | `CompactOptions` | `{ runId, upTo? }`, defaulting to the run's latest checkpoint |
120
+ | `Compacted` | `{ runId, checkpointSeq, deleted }`, `deleted: 0` for a retried compaction |
121
+ | `maxEntriesLimit` | `10_000`, the largest page `entries` reads |
122
+
123
+ ### Constructors and layers
124
+
125
+ | Export | Signature |
126
+ | ----------- | ------------------------------------------------------------------------------------------------ |
127
+ | `make` | `(implementation: Service) => Service` |
128
+ | `makeNoop` | `(overrides?: Partial<Service>) => Service`, every un-overridden method failing `journal_closed` |
129
+ | `layerNoop` | `(overrides?: Partial<Service>) => Layer<Journal>` |
130
+
131
+ The closed stub's `transact` runs its effect directly, because a stub with no
132
+ sink has no transaction to open. A double that models rollback overrides it.
133
+
134
+ ## JournalEvent
135
+
136
+ The durable event envelope. Event types and values remain an open envelope on
137
+ purpose: the durable core never closes them into an interpreter-specific union.
138
+
139
+ | Export | Kind | Meaning |
140
+ | --------------------- | ---------------------------------------- | -------------------------------------------------------------------------------------------------------- |
141
+ | `RunId` | branded schema and type | one durable run |
142
+ | `SourceId` | branded schema and type | one event producer |
143
+ | `Seq` | branded schema and type | canonical per-run replay order |
144
+ | `SourceSeq` | branded schema and type | producer-local sequence, the idempotency key |
145
+ | `Dedupe` | `"content" \| "identity"` | what a re-emitted producer identity means |
146
+ | `Input` | `Schema.Class` | `{ runId, sourceId, sourceSeq?, dedupe?, eventType, payload, meta? }` |
147
+ | `Entry` | `Schema.Class` | `{ runId, seq, eventId, sourceId, sourceSeq, emittedAtMs, eventType, payload, meta }` |
148
+ | `makeEventId` | `(runId, sourceId, sourceSeq) => string` | the deterministic durable id, length-prefixed so a separator in an identifier cannot forge another tuple |
149
+ | `maxIdentifierLength` | `1024` | the identifier ceiling |
150
+ | `companionRunId` | `(stream, runId) => RunId` | the id of the run's companion stream named `stream` |
151
+ | `companionPrefix` | `"flows.companion/"` | the reserved prefix of every companion stream id |
152
+ | `isCompanionRunId` | `(runId) => boolean` | whether a run id names a companion stream |
153
+
154
+ `RunId`, `SourceId`, and both an `Input`'s and a committed `Entry`'s
155
+ `eventType` are non-empty, at most `maxIdentifierLength` UTF-16 code units,
156
+ free of unpaired UTF-16 surrogates, and free of NUL. SQLite binds a lone
157
+ surrogate as U+FFFD, so two ill-formed identifiers that differ only in their
158
+ surrogates would land on one persisted key: the second run's first event would
159
+ dedupe into the first run's row, and a read by either id would return the same
160
+ history. SQLite's `length()` stops at the first NUL, so a NUL-bearing
161
+ identifier fails the column's own non-empty check and the caller is told the
162
+ sink failed about an identifier it had just supplied. Valid astral text is
163
+ ordinary text and round-trips exactly.
164
+
165
+ `Seq` and `SourceSeq` stop below `Number.MAX_SAFE_INTEGER`, so the journal can
166
+ always allocate the next sequence.
167
+
168
+ A companion stream carries one store's facts about a run beside the run's own
169
+ stream: an ordinary stream with its own sequence clock, named
170
+ `flows.companion/<stream>/<sha-256 of the run id>` so the id fits the
171
+ identifier ceiling for every run id. A consumer that reads the run's stream by
172
+ position never sees a companion fact, and a rewind or compaction of the run's
173
+ stream never truncates one. Each fact names its run in its payload.
174
+ `@smthrs/run-store` records ownership transitions on the `run-store` companion
175
+ stream, and refuses to create a run whose id begins with `companionPrefix`.
176
+
177
+ `entries`, `stream`, `checkpoint`, `latestCheckpoint`, and `compact` decode the
178
+ same schemas, so an identifier the writer refuses is refused on every read with
179
+ `invalid_event` rather than answered with an empty page.
180
+
181
+ ## SqlJournal
182
+
183
+ `SqlJournal.layer(options)` provides the bounded telemetry writer and the
184
+ inline durable writer over `DurableWriter`.
185
+
186
+ ```ts
187
+ declare const layer: (
188
+ options: SqlJournalOptions
189
+ ) => Layer.Layer<Journal, JournalError, DurableWriter | SqlClient.SqlClient>
190
+ ```
191
+
192
+ | Option | Type | Default | Meaning |
193
+ | ------------------ | -------------------- | ------------------- | ------------------------------------------------------------------------------------------------------------ |
194
+ | `capacity` | `number` | required | entries held in the lossy admission queue and in the sliding `changes` buffer |
195
+ | `overflow` | `OverflowPolicy` | required | what a full admission queue does |
196
+ | `batchSize` | `number` | `min(capacity, 64)` | entries the queued writer commits per transaction; `stream` reads pages of `min(batchSize, maxEntriesLimit)` |
197
+ | `sourceEventCache` | `number` | `4096` | upper bound on the in-process producer-idempotency index |
198
+ | `maxEntryBytes` | `number` | unset | largest single entry, in UTF-8 bytes of encoded `payload` plus `meta` |
199
+ | `redact` | `Redaction.Redactor` | `Redaction.make()` | scrub applied before encoding |
200
+ | `compaction` | `CompactionPolicy` | unset | automatic checkpoint-and-compact policy |
201
+
202
+ `CompactionPolicy` is `{ entryThreshold: number, capture: (runId: RunId, upTo: Seq) => Effect<unknown, unknown> }`.
203
+ Once a run's committed entry count reaches `entryThreshold`, the journal asks
204
+ `capture` for the caller's replay state at the run's durable tail, writes it as
205
+ a checkpoint at that sequence, and compacts below it. A capture is interrupted
206
+ after 30 seconds, and a failed or refused attempt is logged at warning, damped
207
+ for `entryThreshold` further committed entries, and never surfaced to the emit
208
+ that triggered it. The policy drives the internal unfenced channel, so it needs
209
+ no owner.
210
+
211
+ `capacity` bounds entries, never bytes. `maxEntryBytes` is checked after
212
+ encoding and before any sequence is allocated, so an entry over the bound fails
213
+ `invalid_event` and leaves no gap. It is unset by default because a running
214
+ engine may legitimately write large payloads and a cap introduced under one
215
+ would refuse writes that used to succeed.
216
+
217
+ `sourceEventCache` bounds a cache, never the authority: the database unique
218
+ constraint on `(run_id, source_id, source_seq)` stays authoritative, so a miss
219
+ changes the receipt, not the durable answer. Resident memory and startup decode
220
+ are proportional to this bound rather than to total history. See
221
+ [Producer identity and idempotency](./concepts/idempotency.md).
222
+
223
+ ## OwnerId
224
+
225
+ `OwnerId.OwnerId` is the fencing token the durable channel accepts:
226
+ `{ hostId: string, pid: number, nonce: string }`, where `pid` is a
227
+ non-negative integer.
228
+
229
+ It lives here rather than with the ownership arbitration in
230
+ [`@smthrs/run-store`](/api/run-store) because the journal is what it fences.
231
+ That package's `Ownership` re-exports it alongside `LivenessEvidence`,
232
+ `LivenessProbe`, and the heartbeat constants.
233
+
234
+ ## Consensus
235
+
236
+ The injectable strategy that arbitrates who holds a run. The journal owns the
237
+ rules (one writer per run, fenced and unfenced appends, commit-time
238
+ admission, the generation fence, steal only on staleness plus liveness
239
+ evidence or on a same-host probe that proves the owner's pid gone, and
240
+ ownership transitions as events), and a strategy chooses where
241
+ the lease lives and how the fence is checked at commit time.
242
+
243
+ | Export | Signature |
244
+ | ----------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
245
+ | `Consensus` | `Context.Service<Consensus, Service>` |
246
+ | `Service.claim` | `(runId, claimant: OwnerId, nowMs) => Effect<ClaimOutcome, ConsensusError>` |
247
+ | `Service.activate` | `(runId, owner: OwnerId, grantedAtMs, nowMs) => Effect<ActivateOutcome, ConsensusError>` |
248
+ | `Service.heartbeat` | `(runId, owner: OwnerId, nowMs) => Effect<HeartbeatOutcome, ConsensusError>` |
249
+ | `Service.reconfirm` | `(runId, owner: OwnerId, nowMs) => Effect<HeartbeatOutcome, ConsensusError>`; renews only while the owner matches and no claim is pending |
250
+ | `Service.release` | `(runId, owner: OwnerId) => Effect<void, ConsensusError>` |
251
+ | `Service.steal` | `(runId, claimant: OwnerId, nowMs, evidence: LivenessEvidence) => Effect<ClaimOutcome, ConsensusError>` |
252
+ | `Service.recover` | `(runId, staleClaimant: OwnerId, grantedAtMs, observer: OwnerId, nowMs, evidence: LivenessEvidence) => Effect<RecoverOutcome, ConsensusError>` |
253
+ | `Service.guard` | `(runId, owner: OwnerId) => Effect<void, ConsensusError>`; fails `fence_lost` unless `owner` holds the run |
254
+ | `ClaimOutcome` | `Claimed { grantedAtMs }` or `Rejected { reason }` |
255
+ | `ActivateOutcome` | `Activated` or `Lost` |
256
+ | `HeartbeatOutcome` | `Renewed { heartbeatAtMs }` or `Lost` |
257
+ | `RecoverOutcome` | `Recovered` or `Rejected { reason }` |
258
+ | `RejectionReason` | `already_claimed`, `owner_held`, `owner_live`, `evidence_invalid`, `claim_fresh`, `claim_changed`, `unavailable` |
259
+ | `ConsensusError` | `{ code: "fence_lost" \| "persistence_failed", message, cause? }` |
260
+ | `LivenessEvidence` | `{ expectedOwner: OwnerId, checkedAtMs, kind }` with `kind` one of `same-host-pid-dead`, `cross-host-unreachable-stale`, `lease-expired` |
261
+ | `matchesEvidence`, `sameOwner` | The evidence and identity predicates every strategy applies |
262
+ | `provesDeath` | `(evidence, claimant: OwnerId) => boolean`; `true` for `same-host-pid-dead` about a pid other than the claimant's, which `steal` admits at once |
263
+ | `heartbeatInterval`, `heartbeatStaleAfter`, `heartbeatSkewAllowance`, `heartbeatWriteTolerance` | The lease durations every strategy judges staleness against |
264
+ | `make`, `makeNoop`, `makeLocal`, `layerNoop`, `layerLocal` | Constructors and layers; `layerLocal` is the in-memory single-process strategy |
265
+
266
+ The claim lifecycle is two-phase: `claim` (or `steal`, its evidence-gated
267
+ form) reserves the run and returns the grant timestamp, `activate` presents
268
+ that grant to take ownership, and `release` clears whichever of the caller's
269
+ claim or ownership still stands. `recover` clears a third party's stale claim
270
+ after its claimant was proven dead. Every operation takes the caller's clock
271
+ reading, so staleness is judged against the reading the caller observed.
272
+ Heartbeats and lease state never enter the journal.
273
+
274
+ `layerLocal` keeps its leases per layer build and outside any database
275
+ transaction: provide one instance to every service that must agree, and
276
+ prefer `SqlConsensus` wherever a database is present.
277
+
278
+ `Service.withPermit(key, owner, isAlive, effect)` holds a named resource through
279
+ claim, activation, supervised execution and release. SQL strategies share the
280
+ permit across processes; local strategies share it within their instance.
281
+ Waiters retry admission and then execute their effect. Heartbeats renew once
282
+ per second; lost ownership interrupts work with `ConsensusError` (`fence_lost`).
283
+ Failure and interruption release the permit. Stale claims and owners require a
284
+ negative `isAlive` observation before recovery. Resource keys share the lease
285
+ namespace with run IDs; callers must reserve their prefix.
286
+
287
+ ## SqlConsensus
288
+
289
+ The default database-backed strategy. Its lease is the `flows_consensus_leases`
290
+ row this package's migration `0006_consensus` creates, holding the owner tuple,
291
+ the two-phase claim columns, the grant timestamp, and the heartbeat. `guard`
292
+ is a plain SELECT inside the caller's write transaction, and every mutating
293
+ operation runs through `DurableWriter.write`, joining an enclosing transaction
294
+ as a savepoint, so a rolled-back ownership write rolls its lease back too.
295
+
296
+ | Export | Signature |
297
+ | ------- | -------------------------------------------------------------- |
298
+ | `make` | `Effect<Consensus.Service, never, DurableWriter \| SqlClient>` |
299
+ | `layer` | `Layer<Consensus, never, DurableWriter \| SqlClient>` |
300
+
301
+ The migration backfills a lease for every `flows_runs` row that was running
302
+ or claimed when the table is already present, so an owner driving a run
303
+ before the upgrade keeps its fence afterwards.
304
+
305
+ ## Redaction
306
+
307
+ The scrub applied to every `payload` and `meta` before persistence.
308
+
309
+ | Export | Signature | Meaning |
310
+ | --------------------- | ------------------------------------------------ | ----------------------------------------------------------------------- |
311
+ | `make` | `(options?: Options) => Redactor` | builds a redactor over a rule set |
312
+ | `makeNoop` | `() => Redactor` | the identity redactor, for persisting verbatim by choice |
313
+ | `redact` | `(value: unknown, options?: Options) => unknown` | the scrub itself |
314
+ | `redactJsonString` | `(json: string, redactor: Redactor) => string` | scrubs an already-encoded column at a display surface |
315
+ | `isSensitiveKey` | `(key: string) => boolean` | whether a field name names a credential |
316
+ | `Redactor` | `(value: unknown) => unknown` | the function type the journal consumes |
317
+ | `Rule` | `{ id, pattern, replace?, rewrite? }` | one textual rule; `rewrite` for a value a regex cannot bound |
318
+ | `Options` | `{ rules?, onTooDeep? }` | `onTooDeep` is `"throw"` (default) or `"name"` |
319
+ | `defaultRules` | `ReadonlyArray<Rule>` | the built-in credential shapes |
320
+ | `diagnosticRules` | `ReadonlyArray<Rule>` | `defaultRules` plus the spellings too broad for a durable row |
321
+ | `redactDiagnostic` | `(value: unknown) => unknown` | `redact` with `diagnosticRules`, naming a too-deep value |
322
+ | `lineRedactor` | `(rules?: ReadonlyArray<Rule>) => LineRedactor` | redacts a stream line by line, holding a value open across lines |
323
+ | `LineRedactor` | `{ line, flush }` | `line` returns the lines safe to emit; `flush` ends the stream |
324
+ | `maxHeldLines` | `256` | lines a `LineRedactor` holds for one open value before withholding |
325
+ | `placeholder` | `"[REDACTED]"` | the substitution |
326
+ | `maxDepth` | `256` | container edges traversed before a payload is refused |
327
+ | `RedactionDepthError` | `{ code: "depth_exceeded", maxDepth, message }` | what `onTooDeep: "throw"` throws past `maxDepth`; a tagged error |
328
+ | `binaryWalkLimit` | `65_536` | bytes, and own members, before a binary view is named instead of walked |
329
+ | `binaryMarker` | `"[Binary]"` | a named binary view |
330
+ | `functionMarker` | `"[Function]"` | a named function |
331
+ | `symbolMarker` | `"[Symbol]"` | a named symbol |
332
+ | `depthMarker` | `"[Deep]"` | a value past `maxDepth`, under `onTooDeep: "name"` |
333
+
334
+ Built-in text rules cover credential assignments around `:`, `=` or `=>`, in
335
+ every value shape `util.inspect` and escaped JSON write, credential flags such
336
+ as `--password value`, Basic and Bearer authorization, bare JWTs, private-key
337
+ PEM and PGP blocks (including OpenSSH, and a block with no footer), and
338
+ cookie/session assignments. Structured
339
+ credential suffixes also include `secretAccessKey`, `passphrase`, `sshKey`,
340
+ `session`, `signature`, and `auth`. See the full
341
+ [redaction rules](./concepts/redaction.md).
342
+
343
+ Rows are permanent and are replayed verbatim to sync subscribers and
344
+ time-travel consumers, so redaction on write is the only place it can be
345
+ enforced once. The rule set is a best-effort textual net over shapes seen in
346
+ real reports, not a proof: a value that must never persist belongs in a
347
+ `Redacted` field of the caller's own schema.
348
+
349
+ Redaction stops at the journal. It is an observability concern, and journal
350
+ rows exist to be read. The stores in [`@smthrs/run-store`](/api/run-store) and
351
+ [`@smthrs/step-cache`](/api/step-cache) hold executable state and are
352
+ deliberately not redacted; neither takes a `redact` option at all. A journal
353
+ `Checkpoint`'s `state` is executable state for the same reason and is not
354
+ redacted either.
355
+
356
+ `redactJsonString` returns a string that does not parse untouched, because
357
+ validation is the caller's. Once parsing succeeds it fails closed: a throwing
358
+ redactor or an encoding failure returns the JSON string `"[REDACTED]"`, never
359
+ the original text.
360
+
361
+ Full behavior, including the bounds on hostile input, is in
362
+ [Redaction](./concepts/redaction.md).
363
+
364
+ ## RedactedLogger
365
+
366
+ The same rules applied to log output.
367
+
368
+ | Export | Signature |
369
+ | ------------------ | ------------------------------------------------------------------------------------------------------------ |
370
+ | `layer` | `(options?: Redaction.Options) => Layer<never>` |
371
+ | `wrap` | `<Message, Output>(logger: Logger<Message, Output>, options?: Redaction.Options) => Logger<Message, Output>` |
372
+ | `isRedacted` | `(logger: Logger<any, any>) => boolean` |
373
+ | `redactArgument` | `(value: unknown, redactor: Redaction.Redactor) => unknown` |
374
+ | `redactingConsole` | `(target: Console, redactor: Redaction.Redactor) => Console` |
375
+ | `TypeId` | `unique symbol`, the marker `wrap` sets |
376
+
377
+ `layer` replaces the active logger set with redacting wrappers of the same
378
+ loggers, so an operator keeps the format they had. `wrap` redacts the log event
379
+ itself, the message, the cause, and the annotations, and hands the wrapped
380
+ logger a fiber whose `Console` is redacting. Both halves are load-bearing:
381
+ `Logger.tracerLogger` reads the event and never touches `Console`.
382
+
383
+ `wrap` defaults `onTooDeep` to `"name"`, because a throw in a logger costs the
384
+ operator the whole line. A caller's own `onTooDeep` still wins. Wrapping is
385
+ idempotent.
386
+
387
+ See [Keep credentials out of log output](./guides/redact-log-output.md).
388
+
389
+ ## Projection
390
+
391
+ | Export | Signature |
392
+ | --------------------- | ----------------------------------------------------------------------------------- |
393
+ | `Projection<S, E, R>` | `{ name: string, initial: S, reduce: (state: S, entry: Entry) => Effect<S, E, R> }` |
394
+ | `make` | `<S, E, R>(projection: Projection<S, E, R>) => Projection<S, E, R>` |
395
+
396
+ Projections have no independent durable state. Replaying the same entries
397
+ through the same reducer must reproduce the same result. `Journal.project`
398
+ emits `initial` first, then one state per entry, and follows the run's tail
399
+ without completing. See
400
+ [Fold a run into a projection](./guides/fold-a-projection.md).
401
+
402
+ ## JournalMetrics
403
+
404
+ | Export | Type |
405
+ | ------------- | --------------------------------------------------------------------------- |
406
+ | `writes` | the `flows_journal_writes` counter, dimensioned by `channel` and `receipt` |
407
+ | `durable` | `writes` views keyed by `"Accepted" \| "Duplicate"` |
408
+ | `lossy` | `writes` views keyed by `"Accepted" \| "Duplicate" \| "Dropped"` |
409
+ | `lostEntries` | the `flows_journal_lost` counter, dimensioned by `channel` and error `code` |
410
+ | `lost(code)` | the `lostEntries` view for one `JournalError` code |
411
+
412
+ `SqlJournal` updates these once per emission receipt, so they measure
413
+ admissions on the hot path rather than rows read back. A durable emission
414
+ counts when its receipt is produced; under `transact` that is still inside the
415
+ caller's transaction, so a receipt that later rolls back has already counted.
416
+ The counter is throughput evidence, not commit evidence.
417
+
418
+ `lostEntries` moves by the number of admitted lossy entries the writer failed
419
+ to commit, when it gives them up, whether or not a `flush` is waiting. The
420
+ writer also logs a warning naming the error code, the count, and the runs. A
421
+ final flush that fails while the journal closes logs a warning too.
422
+
423
+ No exporter ships in this package. Provide one, for example
424
+ [`@smthrs/observability`](/api/observability), and these counters appear in it.
425
+
426
+ ## Migrations
427
+
428
+ | Export | Signature |
429
+ | ------- | -------------------------------------------------------------------------- |
430
+ | `set` | `DatabaseMigrations.MigrationSet`, namespace `"journal"`, id block `0` |
431
+ | `run` | `Effect<ReadonlyArray<[id, name]>, MigrationError \| SqlError, SqlClient>` |
432
+ | `layer` | `Layer<never, MigrationError \| SqlError, SqlClient>` |
433
+
434
+ `0001_initial` creates `flows_journal_events` and its event-type index;
435
+ `0002_checkpoints` creates `flows_journal_checkpoints`;
436
+ `0003_startup_index` indexes the timestamp/run/sequence ordering used by the
437
+ bounded startup query. `run` and `layer` install this package's full set.
438
+ The migration composer also applies the new index to an installed journal
439
+ block when another package has already advanced the global migration cursor.
440
+
441
+ Every other durable table belongs to the package that reads it.
442
+ `@smthrs/database`'s `Migrations` composes several sets over one
443
+ `flows_migrations` table, namespacing each package's ids into a reserved block
444
+ so two packages' `0001_initial` cannot collide.
445
+ `@smthrs/engine-store/Migrations` exports `sets`, the composed list a durable
446
+ engine installs. No version shipped before `1.0.0-rc.0`, so each package
447
+ carries one authoritative initial schema rather than compatibility migrations
448
+ for schemas no published release ever used.
449
+
450
+ ## Test entry points
451
+
452
+ `@smthrs/journal/test/TestJournal` provides the production SQL journal over a
453
+ migrated in-memory database:
454
+
455
+ | Export | Signature |
456
+ | -------------------- | ----------------------------------------------------------- |
457
+ | `layer` | `(options?: TestJournalOptions) => Layer<Journal, ...>` |
458
+ | `TestJournalOptions` | `SqlJournalOptions` with `capacity` and `overflow` optional |
459
+
460
+ `TestJournalOptions` is the production option type, so every field forwards to
461
+ `SqlJournal.layer` unchanged and a new production option is reachable here the
462
+ day it is added. The defaults are `capacity: 1024` and `overflow: "reject"`.
463
+ This bundle creates the journal's tables, the consensus lease table included,
464
+ so a suite exercising a fenced call takes the run through `Consensus.claim`
465
+ and `activate`, or takes `@smthrs/engine-store/test/TestStores`, which
466
+ provides the journal, run, attempt, and cache services over one database.
467
+
468
+ `@smthrs/journal/test/Notifying` wraps a record-of-Effect-methods service so a
469
+ hook fires around every operation:
470
+
471
+ | Export | Signature |
472
+ | ------- | ------------------------------------------------------------------------------------- |
473
+ | `Order` | `"before" \| "after"` |
474
+ | `Hook` | `(op: string, order: Order, args: ReadonlyArray<unknown>) => Effect<void>` |
475
+ | `wrap` | `<S extends object>(service: S, hook: Hook) => S` |
476
+ | `layer` | `<Id, S extends object>(tag: Context.Key<Id, S>, hook: Hook) => Layer<Id, never, Id>` |
477
+
478
+ The hook runs in the calling fiber, so a hook that dies, interrupts, or awaits
479
+ a `Latch` injects crashes, fence loss, and exact sequencing at any durable
480
+ transition point. The `after` firing only happens when the operation succeeds.
481
+ Effect-returning methods and plain `Effect` properties are hooked; non-Effect
482
+ members, such as `stream` and `project`, delegate untouched.
483
+
484
+ See [Test against a real journal](./guides/testing.md).
485
+
486
+ ## Sequence allocation
487
+
488
+ `emitDurable` allocates both sequences inside the writer's transaction
489
+ (`MAX(seq) + 1`, taking the in-memory clock as a floor) and inserts the row
490
+ before returning, so the returned `seq` is already committed. Use it wherever a
491
+ caller acts on the returned sequence: lifecycle finalization, cross-process
492
+ supervisors, or any deployment where a second writer may open the same run. A
493
+ durable boundary must not advance the run or expose its result until this
494
+ commit returns.
495
+
496
+ `Seq` is canonical per-run replay order and `SourceSeq` identifies producer
497
+ retries. Rejected and dropped admissions may consume either sequence, so gaps
498
+ are valid: allocation is `MAX(seq) + 1` and replay is `ORDER BY seq`, so
499
+ neither reads a gap as anything.
500
+
501
+ ## Idempotency equality
502
+
503
+ `(runId, sourceId, sourceSeq)` is the producer identity, unique in the database
504
+ and addressed by the deterministic `makeEventId`. Two emissions under one
505
+ identity are the same event when their event type and their canonically
506
+ encoded, redacted `payload` and `meta` match. The encoding sorts object keys,
507
+ so key order does not change the answer. An exact producer retry returns
508
+ `Duplicate` with `status: "committed"`; a reused producer sequence carrying
509
+ different content fails `idempotency_conflict` unless the input declares
510
+ `dedupe: "identity"`.
511
+
512
+ The persisted bytes are `JSON.stringify` semantics with object keys sorted: a
513
+ `Date` is its ISO string, an `undefined` member is dropped, `NaN` is `null`.
514
+ Sorting happens over the encoded JSON, never over the raw value, so
515
+ canonicalization cannot destroy anything the encoder would have kept.
516
+
517
+ Two consequences follow from comparing the persisted, redacted bytes. Two
518
+ different secrets that redact to the same placeholder are the same event to the
519
+ journal, and `NaN` and `null` are the same value. Comparing the pre-redaction
520
+ value instead would keep unredacted secrets resident in the in-process index,
521
+ which is the leak this package exists to prevent, and the durable re-check at
522
+ insert can only read the persisted bytes.
523
+
524
+ ## transact
525
+
526
+ Committing an entry makes that entry durable; it does not by itself make a
527
+ host's whole view crash-consistent, because the executable state lives in the
528
+ neighboring stores: `RunStore` and `AttemptStore` in
529
+ [`@smthrs/run-store`](/api/run-store), `CacheStore` in
530
+ [`@smthrs/step-cache`](/api/step-cache), and `DurableEngineState` in
531
+ [`@smthrs/engine-store`](/api/engine-store). `transact` closes that seam: those stores write through the same `DurableWriter`, so their
532
+ writes join this transaction as savepoints and the row and its lifecycle entry
533
+ either both commit or both roll back.
534
+
535
+ Three properties matter to callers:
536
+
537
+ - **Publication follows COMMIT.** Inside a transaction, `emitDurable` returning
538
+ means a savepoint was released. The `changes` and `stream` publish and the
539
+ in-process producer index update are parked until the outermost transaction
540
+ commits, so a subscriber never sees an entry that later rolls back, and a
541
+ rolled-back producer identity stays re-emittable instead of deduplicating
542
+ against a sequence that does not exist.
543
+ - **Only storage work belongs inside.** The transaction is held for its whole
544
+ body: no flow bodies, host calls, or `flush`, which waits on the lossy writer
545
+ and would deadlock against the open transaction.
546
+ - **Nesting is a savepoint.** An inner `transact` defers its settlements to the
547
+ outermost commit.
548
+
549
+ A crash before COMMIT still loses the whole unit, so work that had already run,
550
+ an action body for instance, re-executes on the next drive. And no local
551
+ transaction makes a remote effect atomic, so external effects still need
552
+ idempotency keys, fencing tokens, or compensation. The transaction can also be
553
+ replayed by the database package's write retry, which is a second reason a body
554
+ must tolerate running more than once; see
555
+ [Commit state and its entry together](./guides/commit-state-and-entry.md#a-note-on-retries).
556
+
557
+ ## Failure and loss
558
+
559
+ The two channels fail independently, and neither failure is permanent. A batch
560
+ the optimistic writer cannot persist is lost and reported: to the `flush`
561
+ waiters that covered it, to live `stream` consumers that were following when it
562
+ happened, and, if nobody was waiting, to the next `flush`. The writer fiber
563
+ survives it. Each loss is reported once; a later `flush` with nothing
564
+ outstanding succeeds, while entries queued behind the lost batch stay
565
+ outstanding, so a subsequent `flush` still waits for them instead of vouching
566
+ for unpersisted work. `emitDurable` was never gated by the queue: it opens its
567
+ own transaction inline, so the lossless lifecycle channel keeps working as soon
568
+ as the database is healthy again.
569
+
570
+ `changes` is a bounded sliding buffer sized by the layer's `capacity`: a slow
571
+ subscriber loses entries with no error and no gap signal. `stream` is the
572
+ lossless follower. Entries published to `changes` are frozen, so one subscriber
573
+ cannot mutate another's view.
574
+
575
+ Reads below a run's compaction floor fail with `compacted`. See
576
+ [Checkpoints and compaction](./concepts/compaction.md) and the
577
+ [`@smthrs/journal` error codes](/docs/reference/errors/#smthrsjournal).
578
+
579
+ ## Resource limits
580
+
581
+ | Limit | Value |
582
+ | ------------------------------------ | ---------------------------------------------------------------------- |
583
+ | admission queue and `changes` buffer | `capacity` entries |
584
+ | one entry | `maxEntryBytes` UTF-8 bytes of `payload` plus `meta`, unset by default |
585
+ | run ids, source ids, event types | 1,024 UTF-16 code units |
586
+ | `Seq`, `SourceSeq` | below `Number.MAX_SAFE_INTEGER` |
587
+ | one `entries` page | `maxEntriesLimit`, 10,000 entries |
588
+ | producer-idempotency index | `sourceEventCache` entries, default 4,096 |
589
+ | redaction traversal | `maxDepth`, 256 container edges |
590
+ | binary view walk | `binaryWalkLimit`, 65,536 bytes or own members |
591
+
592
+ A payload deeper than `Redaction.maxDepth` fails `invalid_event` rather than
593
+ overflowing the stack, and the canonical encoder carries the same ceiling for a
594
+ caller who disables redaction.
595
+
596
+ ## See also
597
+
598
+ - [State and event authority](./concepts/state-event-authority.md): shared
599
+ bounded identities, `EngineEvent` schemas, constructors and consumer decoders.
600
+ - [Durable execution](/docs/concepts/durable-execution/) and
601
+ [Execution IDs and ownership](/docs/concepts/ownership/).
602
+ - [`@smthrs/engine-store`](/api/engine-store), which composes this package with
603
+ the run store and the step cache.
604
+ - [`@smthrs/chain`](/api/chain) has a journal of its own with a different
605
+ contract; see [the chain journal](/pkg/chain/concepts/journal).
606
+
607
+ ### Journal generations
608
+
609
+ `Journal.Service.generation?(runId)` returns an Effect of
610
+ `{ generation: number, afterSeq: number }` with `JournalError` failures.
611
+ Append-only adapters may omit it (generation zero). A truncating adapter must
612
+ advance it atomically with truncation so followers detect reused sequences.
613
+ `SqlJournal.layer` persists it in `flows_journal_generations`, installed through
614
+ `JournalGeneration.initialize`; a fresh run reports `{ generation: 0, afterSeq: -1 }`.
615
+
616
+ `JournalGeneration.forget(runIds)` invalidates live journal source identities and
617
+ allocation floors after a committed truncation, for every journal sharing the
618
+ SQL client. It requires `SqlClient`; it changes no durable rows. Producers must
619
+ be quiescent and pending lossy admissions flushed before truncation.
620
+ `JournalGeneration.onTruncate(callback)` registers an invalidator for the current
621
+ scope. `SqlJournal` registers automatically, and `SqlTimeTravelStore.archiveAndTruncate`
622
+ notifies after commit for the parent and attached descendants. Re-emitting an
623
+ archived lossy source identity is then admitted against the retained history.
624
+
625
+ ## Native execution facts and human waits
626
+
627
+ `ExecutionFact.fold` verifies native run-local sequence/generation evidence and
628
+ reconstructs root identity, current trampoline lifecycle, and attached descendant
629
+ human waits. Current observations include the additive `treeVersion: 1`,
630
+ `parentPolicy`, and redacted `waiting.request`/`waiting.point` metadata. Detached
631
+ subtrees do not roll their questions into an ancestor's status. A settled wait
632
+ leaves the derived open-wait set.
633
+
634
+ `Provenance.humanWaits` distinguishes covered events from legacy or unverified
635
+ observations. Old version-1 facts lacking these fields cannot establish tree-wait
636
+ coverage. `humanWaitSources` records separate generation and baseline sequence
637
+ bounds for each native execution in the derived tree, so upgrading an old stream
638
+ does not claim question history before the extension first appeared. A missing
639
+ child stream, explicit gap, incompatible metadata, or question that differs from
640
+ the coherent observed tree refuses that coverage. The wait set compares by
641
+ execution identity and exact semantic values; adapter enumeration order does not
642
+ change coverage. The derived list orders by depth, creation time, and lexical ID.
643
+ The root/current semantic provenance also declines a complete event claim when
644
+ the requested tree comparison fails.
645
+
646
+ The fact is display evidence. Its token digest does not authorize a signal or
647
+ reconstruct an opaque resolver token. The gateway joins a verified question to
648
+ a currently observed wait with the same execution identity and digest, then
649
+ uses that current wait's existing submission envelope. A historical question
650
+ alone cannot make a gate answerable after the live wait has ended.