@smthrs/journal 0.0.0-stage → 1.0.0-rc.4

package/docs/concepts/state-event-authority.md

@@ -0,0 +1,159 @@
+---
+title: "State and event authority"
+description: "Bounded identities, versioned engine history, explicit consumer admission, and the limits of projection recovery."
+sidebar:
+  order: 7
+---
+
+The journal owns committed history. Engine stores still own executable run,
+attempt, deferred and clock state. The additive `EngineEvent` contracts make
+history safer to consume; they do not move recovery into the journal.
+
+| Durable concern                           | Authority                        | History relationship                                           |
+| ----------------------------------------- | -------------------------------- | -------------------------------------------------------------- |
+| Control admission, approvals and signals  | Control records and local events | Separate outbox and receipt contract.                          |
+| Execution, attempts, deferreds and clocks | Engine-owned stores              | State and required events share a `DurableWriter` transaction. |
+| Ownership leases and fences               | Arbitration store                | Never restored from an old projection.                         |
+| Engine history and time-travel evidence   | Typed journal families           | Consumers validate family, version, source and lineage.        |
+| Cache and artifacts                       | Validated content and provenance | Cache loss does not erase authoritative completion.            |
+| UI and listing summaries                  | Read projections                 | Rebuildability must be demonstrated for each projection.       |
+
+## Shared boundary primitives
+
+`JournalEvent` exports `RunId`, `LineageId`, `WaitId`, `CommandId`, `PlanId`,
+`DispatchId` and `ArtifactId`. Each offers a validating `.make(value)`
+constructor and a named Effect decoder, such as `decodeRunId(unknown)`.
+Identifiers preserve bytes, require 1 through 1,024 UTF-16 code units, and
+refuse NUL and unpaired surrogates. Plan, dispatch and artifact brands are
+distinct, even when their encoded strings happen to match.
+
+`NonNegativeQuantity` and `TimestampMs` admit safe integers from zero through
+`Number.MAX_SAFE_INTEGER`. `PositiveQuantity` starts at one. Fractions,
+infinity, string coercion and missing-value defaults are refused. Existing
+journal sequences retain their stricter exclusive upper bound.
+
+## Engine families and consumer admission
+
+`@smthrs/journal/EngineEvent` keeps the journal's generic envelope open while
+defining two version-2 families:
+
+| Event type                          | Typed payload                                                                              |
+| ----------------------------------- | ------------------------------------------------------------------------------------------ |
+| `flows.engine.v2.attempt-lifecycle` | Execution, dispatch, attempt number and running, suspended, succeeded or failed lifecycle. |
+| `flows.engine.v2.state-event`       | Execution lifecycle, deferred completion or an absolute clock schedule.                    |
+
+Both payloads require version, run, lineage id, root run, round and parent
+where applicable. The committed journal envelope supplies global sequence,
+source identity, source sequence and emission time. Root runs have round zero
+and no parent; derived runs name a distinct parent and root. A continuation
+requires a positive round. Diagnostic `meta` is encoded JSON and grants no
+semantic authority.
+
+`decodeEntry(input, consumer)` requires an explicit expected run, lineage,
+root, round, parent and source allowlist. It returns a typed `Attempt` or
+`State`. Conflicting identity fails `EventError` with code `foreign`.
+Malformed supported families fail `malformed`, preserving the original
+schema or accessor error in `cause`. Unsupported families and versions in
+`flows.engine.*` fail `unsupported`. Other namespaces follow the consumer's
+explicit `unknown: "ignore" | "surface"` policy. Source and run admission
+still applies to those extensions.
+
+Running and suspended attempts cannot contain completion fields. A terminal
+attempt requires its timestamp and a matching encoded success or failure.
+Failure reasons distinguish typed error, defect, interruption and encoding
+failure. Values must be JSON after the value's own codec has run; a class
+instance accepted by `Schema.Unknown` is not an encoded result. Wall clocks
+can move backwards, so completion timestamps need not exceed start time.
+Suspended executions require at least one typed wait; completed executions
+require a result. Ownership and cancellation remain separate facts.
+
+## Node and plan records
+
+Two executors drive a graph: the plan scheduler and the flow interpreter. Both
+write the same six records, and `EngineEvent` publishes a schema for each so a
+reader decodes one shape rather than guessing per writer.
+
+| Event type                       | Typed payload                                                    |
+| -------------------------------- | ---------------------------------------------------------------- |
+| `flows.engine.plan-recorded`     | `PlanRecordedPayload`: the flow, the generation, the node count. |
+| `flows.engine.subgraph-appended` | `SubgraphAppendedPayload`: the ids a generation or a page added. |
+| `flows.engine.node-scheduled`    | `NodeScheduledPayload`: the node, its kind, attempt and tag.     |
+| `flows.engine.node-settled`      | `NodeSettledPayload`: the outcome, the attempts, the join.       |
+| `flows.engine.node-invalidated`  | `NodeInvalidatedPayload`: the re-keying and the reason for it.   |
+| `flows.engine.node-reconciled`   | `NodeReconciledPayload`: the verdict for one deviation.          |
+
+`nodes` on a plan record is the node COUNT it has always been. The node list
+rides beside it as `graph`, optional and paged, because a journal entry has a
+byte bound and a record that clipped its list would lose nodes silently.
+`graph.edges` is optional for the same reason a fact is optional anywhere here:
+a plan names its edges through each node's `dependsOn` and knows no reason for
+them, while an interpreted graph knows the reason it drew each one.
+
+A node's `declaredAt` is where its action was written. It is repo-relative by
+contract and the schema refuses an absolute path: a journal is read on machines
+that did not write it, so an absolute path is at best noise and at worst an
+operator's home directory published into a run's history. A writer that cannot
+make a path relative omits the field. `EngineEvent.relativePath(root, path)` is
+how a writer obeys that rule: it answers the path relative to the root, or
+nothing at all for a path outside the root, for no root, and for a root that
+names the whole filesystem. Every writer of a declaration site uses it, the
+engine's node records and a control plan card alike.
+
+The five settlement words are `built`, `clean`, `failed`, `skipped` and
+`deferred`. `clean` means a recorded result served the node and no executor
+ran, which covers same-run durable replay as well as a cross-run cache hit.
+
+A settlement's `attempts` is the executor's own count. For the plan scheduler
+it is the node's; for the interpreter it is the highest durable attempt any of
+the node's dispatches ran as, because the interpreter settles each node once
+and a retry happens underneath it inside one dispatch.
+
+`stepKeyDigests` is the join. An attempt record carries a step key digest and
+no node id, so before it an attempt belonged to no node and "attempt 2" could
+not be attributed. A settlement now names the distinct step keys its node
+dispatched under, and an attempt record belongs to the node of the same
+execution whose settlement claims its digest. A retried dispatch keeps ONE
+digest, because the attempt is folded into no key, so the list counts
+dispatches and not attempts. A node that dispatched nothing records an empty
+list, which is a different statement from a writer that derives no digests and
+omits the field.
+
+`result` is what the node settled with: the success value for `built` and
+`clean`, the typed failure for `failed`. It is a bounded, redacted `preview`
+of the value's JSON, the `bytes` of the encoding that preview was cut from,
+and a `truncated` flag that is also the warning that the text is a prefix of
+JSON and no longer parses. The writer redacts before it truncates, because
+cutting first can split a credential across the boundary and the textual rules
+would no longer recognise what is left; a value too large to redact at all is
+named by the size of its own encoding, with an empty preview. A writer that
+kept no summary omits the field.
+
+## Additive cutover and retained history
+
+Current engine writers are unchanged. `decodeCurrentAttempt` validates their
+actual started/finished markers, using the recorded metadata lineage. It does
+not invent a root, round, timestamp or result absent from an old row. That
+adapter is explicitly separate from version-2 admission. Generic store
+extension points remain generic.
+
+A writer cutover must persist complete lineage and completion evidence,
+introduce the new family identity, and write state plus event in the same
+transaction. Old histories retain their original bytes and decoder. Backfill
+requires authoritative state evidence and an explicit migration; an old
+finished marker alone cannot backfill a result. No database column or journal
+migration is needed for the additive schemas themselves.
+
+The engine's additive attempt projection demonstrates full-history and
+snapshot-plus-suffix equality against real attempt rows at 2 and 70 attempts.
+Its tests drop the projection in a subprocess, kill that process, reopen
+SQLite with a fresh connection and rebuild it. A version-2 snapshot binds its
+rows to lineage and a covering sequence; duplicate identities, foreign runs
+and rows past the covering sequence are refused. Compaction requires that
+validated snapshot before reading the surviving suffix.
+
+This proof covers the disclosed attempt projection. Journal redaction can
+change result content, so it does not prove executable recovery of arbitrary
+private values. Retain the authoritative stores, their backup, and operational
+fences. A concrete sync consumer still owns atomic snapshot application and
+its applied cursor. Extending retention to a public sync projection requires
+its own authorization and retention policy.

package/docs/concepts/two-channels.md

@@ -0,0 +1,167 @@
+---
+title: "The two channels"
+description: "Why the journal has a lossless lifecycle channel and a lossy telemetry channel, what each receipt means, and why sequence gaps are valid."
+sidebar:
+  order: 1
+---
+
+The journal has one table and two ways into it. The split is deliberate: a
+lifecycle fact and a telemetry sample have different costs when they are lost,
+so they get different guarantees and the caller picks.
+
+## The durable channel
+
+`emitDurable(input, owner)` and `emitDurableUnfenced(input)` open a write
+transaction inline, allocate the sequence inside it, insert the row, and return
+only after the transaction commits. The returned `seq` is therefore already on
+disk.
+
+The receipt type says the rest. `DurableReceipt` is `Accepted | Duplicate`, so
+a dropped lifecycle event is unrepresentable: the write commits, collapses onto
+an identical earlier write, or fails with a typed `JournalError`.
+
+Use this channel for anything a run's correctness depends on. A durable
+boundary must not advance a run or expose its result until the lifecycle entry
+that describes the move has committed.
+
+## The lossy channel
+
+`emitLossy(input)` validates the event, allocates its sequences from an
+in-memory clock, and hands it to a bounded queue that one scoped writer drains
+in batches. The call returns before SQL commits. Its `seq` is provisional until
+commit; `sourceSeq` remains the stable producer identity.
+
+`EmitReceipt` is `Accepted | Duplicate | Dropped`, and which of those you can
+see depends on the layer's `overflow` policy:
+
+| `overflow`    | A full queue produces                                                |
+| ------------- | -------------------------------------------------------------------- |
+| `reject`      | a `queue_overflow` failure                                           |
+| `drop-newest` | a `Dropped` receipt for the event just offered                       |
+| `drop-oldest` | an `Accepted` receipt whose `evicted` field counts what it displaced |
+
+Use this channel for telemetry: spans, progress, per-frame samples. A process
+crash can lose an accepted but unwritten entry, and that is the trade the
+channel exists to make.
+
+`emitLossy` opens no transaction and performs no deduplication lookup in SQL.
+New allocation is read-free only when its allocation floors are warmed. On a cold
+run, it reads `MAX(seq) + 1` from the run's durable entries. If `sourceSeq` is
+omitted and the producer floor is uncached, it also reads
+`MAX(source_seq) + 1` for `(runId, sourceId)`. An explicit `sourceSeq` avoids
+the producer-floor read, but does not warm the run floor by itself.
+
+A retained duplicate needs no allocation floors and returns from the identity
+cache even when those floors are cold. For a new admission, cold floor reads
+run outside the run permit. Admission then checks the compaction gate,
+rechecks the producer identity, and takes the maximum of each read floor and
+its current in-memory floor before reserving sequences. A batch holding the
+database transaction can therefore drain while a new producer waits for SQL.
+Top-level compaction drains accepted work with admission open, then acquires
+its SQL transaction before its run locks. It checks the drain again before closing the
+admission gate and reading the checkpoint. If work arrived while it waited for
+SQL, compaction releases the transaction and drains again. It never holds a run
+permit or a closed gate while waiting for a database connection. Producers in
+an enclosing transaction can therefore finish admission and release SQL before
+compaction closes the gate. Continuous admissions can postpone compaction until
+the run drains; accepted work takes precedence over truncation.
+
+Compaction inside an enclosing managed or raw SQL transaction already owns the
+connection. It cannot wait for the queued writer. Instead, it inserts this run's
+pending lossy entries inline before truncation, including the batch waiting for
+SQL. It uses the same transactional insert path as the batch writer. Queue
+settlement stays with that writer: an outer rollback leaves the entries queued,
+and an outer commit makes their later batch attempts collapse by identity.
+Managed transactions publish only after the outer commit; raw SQL transactions
+leave replay authoritative, as with durable emits.
+
+A pending reservation below the chosen checkpoint moves above the durable tail
+when inserted inline. It survives replay after the checkpoint, with unchanged
+producer identity and content checks. Reservations below a previously compacted
+floor retain the existing loss behavior. Durable writers register as active
+only after acquiring SQL; a writer waiting for the compactor's connection has
+allocated no canonical sequence and inserts above the surviving tail later.
+
+Inside `Journal.transact` or `DurableWriter.write` using the same SQL client,
+these floor reads join the caller's transaction. A raw `sql.withTransaction`
+on that client also supplies the read transaction, but lossy admission remains
+a separate queued write: it is not atomic with the caller's commit or rollback.
+For atomic state and event writes, use
+[`transact` with `emitDurable`](../guides/commit-state-and-entry.md).
+
+If another transaction holds the shared connection without passing its SQL
+transaction context to the admission, a cold floor read can wait on that
+connection while its holder waits on the caller. Only warmed allocations avoid
+that read dependency. Admit outside that transaction when the required floors
+may be cold, including after truncation invalidates them. Do not wait for
+`flush` while holding a transaction the queued writer needs.
+
+## flush is the lossy barrier only
+
+`flush` waits for everything currently queued to reach disk. It says nothing
+about the durable channel, which was already committed when its receipt
+arrived.
+
+A producer still waiting for a cold allocation-floor read has not queued an
+entry. Await its `emitLossy` receipt before flushing to cover that entry. If
+the journal closes during this admission wait, the emit fails with
+`journal_closed` after the wait instead of offering to the closed queue.
+
+A batch the writer cannot persist is lost and reported once: to the `flush`
+waiters that covered it, to live `stream` consumers that were following at the
+time, and, if nobody was waiting, to the next `flush`. The writer fiber
+survives the failure. Entries queued behind the lost batch stay outstanding, so
+a later `flush` still waits for them rather than vouching for unpersisted work.
+
+Never call `flush` inside `transact`. It waits on the queued writer while your
+transaction holds the write lock the writer needs.
+
+## Two sequence domains
+
+Every admitted event gets two numbers, and they answer different questions.
+
+- `seq` is the run's canonical durable order. Replay, paging, streams, and
+  projections all use it, and it is shared by both channels: a durable entry
+  and a lossy entry from the same run interleave in one history.
+- `sourceSeq` is per `(runId, sourceId)`. It identifies a producer's retries
+  and is the idempotency key. A producer may supply its own; otherwise the
+  journal allocates the next one.
+
+## Commit ordering
+
+Both channels allocate canonical `seq` inside the write transaction, above the
+run's committed tail. The lossy writer uses its admission reservation as a
+minimum. If a durable commit or another journal instance overtakes that
+reservation, the queued entry moves forward when it commits. For example,
+lossy admission `0`, followed by durable commit `1`, commits the lossy entry at
+`2`. Replay and followers then deliver `1, 2`.
+
+The durable tail is therefore a committed-prefix watermark: a pending entry
+cannot later commit below it. Readers can advance to the last returned row
+without skipping later commits, including commits from independent connections.
+No persisted reservation or local pending-writer registry is needed.
+
+Use committed entries or committed duplicate receipts for paging and checkpoint
+cursors. An `Accepted` lossy receipt or a pending `Duplicate` carries the
+provisional reservation, not a durable cursor. `sourceSeq` and event identity do
+not change when the canonical sequence moves. The compaction guard still drops
+queued reservations at or below an already compacted boundary and reports the
+loss through `flush`.
+
+## Gaps are valid
+
+A rejected or dropped admission still consumes both allocations. Allocation is
+`MAX(seq) + 1` and replay is `ORDER BY seq`, so nothing reads a gap as
+anything: a missing number is not a missing entry, and no reader should treat
+it as one.
+
+The one receipt that consumes neither allocation is a `Duplicate` the
+in-process producer index still holds. It returns the original event's
+`seq`, provisional while pending and canonical after commit.
+
+## Where the durable channel is fenced
+
+`emitDurable` requires an `OwnerId` and refuses to write behind a live
+successor. That is the subject of [the owner fence](./owner-fence.md).
+`emitDurableUnfenced` is the same durability with no fence, for an admission
+that is genuinely ownerless.

package/docs/guides/commit-state-and-entry.md

@@ -0,0 +1,114 @@
+---
+title: "Commit state and its entry together"
+description: "Use Journal.transact to append a lifecycle entry in the same write transaction as the state transition it describes, and learn the three rules that keeps."
+sidebar:
+  order: 2
+---
+
+Committing an entry makes that entry durable. It does not by itself make a
+host's whole view crash-consistent, because the executable state lives in other
+stores: `RunStore` and `AttemptStore` in
+[`@smthrs/run-store`](/api/run-store), `CacheStore` in
+[`@smthrs/step-cache`](/api/step-cache), and `DurableEngineState` in
+[`@smthrs/engine-store`](/api/engine-store).
+
+`transact` closes that seam. It runs a state transition and the `emitDurable`
+calls describing it in one write transaction, so either both land or neither
+does.
+
+## Wrap the transition and the entry
+
+```ts
+import { Journal, JournalEvent } from "@smthrs/journal"
+import { RunStore } from "@smthrs/run-store"
+import * as Effect from "effect/Effect"
+
+const finish = Effect.gen(function*() {
+  const journal = yield* Journal.Journal
+  const runs = yield* RunStore.RunStore
+
+  return yield* journal.transact(Effect.gen(function*() {
+    const receipt = yield* journal.emitDurable({
+      runId,
+      sourceId,
+      sourceSeq: 7 as JournalEvent.SourceSeq,
+      eventType: "run.finished",
+      payload: { outcome: "succeeded" }
+    }, owner)
+    const result = yield* runs.transitionOwned(runId, owner, "completed")
+    if (result._tag !== "Transitioned") return yield* Effect.fail(result)
+    return receipt
+  }))
+})
+```
+
+The fenced append runs while the owned row is still running. Transitioning to
+`completed` first invalidates the journal fence. Both stores write through
+the same `DurableWriter`, so the entry and terminal transition join this
+transaction as savepoints and commit or roll back together. If the transition
+is rejected, the entry rolls back too.
+
+## Three rules the transaction keeps
+
+**Publication follows commit.** Inside a transaction, `emitDurable` returning
+means a savepoint was released, not that anything is visible. The `changes` and
+`stream` publish and the in-process producer index update are parked until the
+outermost transaction commits. A subscriber therefore never sees an entry that
+later rolls back, and a rolled-back producer identity stays re-emittable
+instead of deduplicating against a sequence that does not exist.
+
+**Only storage work belongs inside.** The transaction is held for the whole
+body. No flow bodies, no host calls, no waits. In particular, never call
+`flush` inside `transact`: it waits on the lossy writer, which is waiting on
+the transaction you are holding.
+
+**Nesting is a savepoint.** An inner `transact` becomes a savepoint of the
+outer one and defers its publication to the outermost commit. If its failure
+is caught by the outer transaction, its publication is discarded even when
+the outer transaction succeeds. This also holds when another store opens the
+outer transaction through the same `DurableWriter` and SQL client.
+
+Projections can register short, non-failing cache updates with the service's
+`journal.whenCommitted(update)`. It uses the shared database commit owner and
+is bound to this journal's database: an unrelated database's nested commit
+cannot publish its updates. Outside a transaction the update runs immediately;
+inside a managed transaction it waits for the outer commit. Its `true` result
+means the update was accepted, not necessarily already executed. A `false`
+result means ownership is unknown; skip optional cache updates then. Raw
+`sql.withTransaction` and raw savepoints are not owned by this hook. Journal
+writes in them remain durable on commit, but skip process-local publication;
+`entries` and the polling `stream` still read the authoritative SQL history.
+Use `transact` or `DurableWriter.write` when immediate `changes` notifications
+are required.
+
+The lower-level `Journal.afterCommit` re-export has the database helper's
+different outside-transaction contract: it returns `false` without publishing.
+Test doubles implementing transactions must preserve commit/rollback semantics
+for `whenCommitted`, not run updates immediately from transaction bodies.
+
+## What a transaction does not buy
+
+A crash before COMMIT loses the whole unit, so work that had already run, an
+action body for instance, re-executes on the next drive. Design the body to
+tolerate that.
+
+And no local transaction makes a remote effect atomic. External effects still
+need idempotency keys, fencing tokens, or compensation.
+
+## A note on retries
+
+The pinned Node SQLite driver uses `BEGIN IMMEDIATE`, acquiring its writer
+lock before reading allocation floors. A caller-supplied SQL client may use
+DEFERRED transactions; under WAL its lock upgrade can fail with
+`SQLITE_BUSY_SNAPSHOT`. The shared writer retries transient contention at the
+whole-transaction boundary in either case, floor read included.
+
+A `transact` body must therefore be safe to run more than once. Process-local
+publication belongs in `afterCommit`, outside that retry loop.
+
+## Next steps
+
+- [The two channels](../concepts/two-channels.md) for why the durable channel
+  returns only after commit.
+- [Durable execution](/docs/concepts/durable-execution/) for the execution
+  model this transaction supports.

package/docs/guides/compact-a-run.md

@@ -0,0 +1,134 @@
+---
+title: "Compact a long-running run"
+description: "Checkpoint a run's replay state, truncate the entries below it, resync a reader that fails with compacted, and hand the work to the automatic policy."
+sidebar:
+  order: 5
+---
+
+Compaction drops the entries a run no longer needs to replay. It is off by
+default, so nothing here happens unless you ask for it.
+
+Every step is fenced: `checkpoint` and `compact` both take an `OwnerId` and
+both need the run claimed and activated through the journal's `Consensus`
+strategy, which [`@smthrs/run-store`](/api/run-store) drives. See
+[Installation](../installation.md#what-a-fenced-write-needs).
+
+## Checkpoint the replay state
+
+Capture whatever your reader needs to resume, and pin it to a committed
+sequence:
+
+```ts
+import { Journal } from "@smthrs/journal"
+import * as Effect from "effect/Effect"
+
+const checkpointTail = Effect.gen(function*() {
+  const journal = yield* Journal.Journal
+
+  const page = yield* journal.entries({ runId, limit: 1000 })
+  const tip = page.entries.at(-1)
+  if (tip === undefined) return
+
+  yield* journal.checkpoint({ runId, seq: tip.seq, state: { started: 3, finished: 2 } }, owner)
+})
+```
+
+`state` round-trips verbatim; the journal never interprets it and never redacts
+it. A value that must not persist belongs in a `Redacted` field of your own
+state schema.
+
+The sequence must name a committed entry and must lie above the run's current
+floor, or the write fails `checkpoint_invalid`. Re-checkpointing an uncompacted
+sequence replaces its state.
+
+## Truncate below it
+
+```ts
+const truncate = Effect.gen(function*() {
+  const journal = yield* Journal.Journal
+  const compacted = yield* journal.compact({ runId }, owner)
+  console.log(compacted.deleted, "entries removed below", compacted.checkpointSeq)
+})
+```
+
+Omit `upTo` to compact to the run's latest checkpoint, or pass a specific
+sequence. The deletion and the floor advance commit atomically. The checkpointed
+entry itself survives: it holds the run's allocation floor, so a restarted
+process can never re-allocate a truncated sequence.
+
+A retried compaction returns `deleted: 0` rather than failing.
+
+## Handle a refusal
+
+`compact` refuses in three typed ways, and each asks for something different:
+
+- `checkpoint_invalid`: the run has no checkpoint to truncate below, or none at
+  the sequence you named. Write one first.
+- `reader_behind`: a live in-process `stream` still needs a sequence the
+  truncation would delete. `checkpointSeq` carries the checkpoint it declined
+  to truncate below. Wait for the reader, or drop it.
+- `fence_lost`: another process owns the run. Stop; a zombie owner must not
+  truncate history behind a live successor.
+
+## Resync a reader
+
+A reader whose cursor is below the floor fails with `compacted`, carrying the
+floor in `checkpointSeq`. Recovery is the checkpoint plus the tail after it:
+
+```ts
+import * as Option from "effect/Option"
+import * as Stream from "effect/Stream"
+
+const resync = Effect.gen(function*() {
+  const journal = yield* Journal.Journal
+  return yield* journal.stream({ runId }).pipe(
+    Stream.catchTag("@smthrs/journal/JournalError", (failure) =>
+      failure.code === "compacted"
+        ? Stream.unwrap(Effect.gen(function*() {
+          const checkpoint = yield* journal.latestCheckpoint(runId)
+          if (Option.isNone(checkpoint)) return Stream.fail(failure)
+          return journal.stream({ runId, afterSequence: checkpoint.value.seq })
+        }))
+        : Stream.fail(failure)),
+    Stream.take(100),
+    Stream.runCollect
+  )
+})
+```
+
+`latestCheckpoint` returns the run's most recent checkpoint whether or not it
+has been compacted, so the same call serves both cases.
+
+## Hand it to the policy
+
+Set `compaction` on the layer to make this automatic:
+
+```ts
+import { SqlJournal } from "@smthrs/journal"
+
+const layer = SqlJournal.layer({
+  capacity: 1024,
+  overflow: "reject",
+  compaction: {
+    entryThreshold: 5_000,
+    capture: (runId, upTo) => captureReplayState(runId, upTo)
+  }
+})
+```
+
+Once a run's committed entry count reaches `entryThreshold`, the journal
+captures your replay state at the run's durable tail, checkpoints it, and
+compacts below it. The policy holds no fence, so it needs no owner and no
+lease.
+
+Write `capture` to finish quickly and to tolerate being abandoned: it is
+interrupted after 30 seconds, and a failed or refused attempt is logged at
+warning, damped for `entryThreshold` further entries, and never surfaced to the
+emit that triggered it.
+
+## Next steps
+
+- [Checkpoints and compaction](../concepts/compaction.md) for the model behind
+  these calls.
+- [Run a read-only follower](/docs/guides/sync-followers/) for a follower that
+  meets the floor.

package/docs/guides/fold-a-projection.md

@@ -0,0 +1,106 @@
+---
+title: "Fold a run into a projection"
+description: "Write a reproducible reducer over journal entries, run it with project, and bound a live projection so it terminates."
+sidebar:
+  order: 4
+---
+
+A projection is a fold over a run's entries: `{ name, initial, reduce }`. It has
+no durable state of its own, so replaying the same entries through the same
+reducer must reproduce the same result. That is the whole contract, and it is
+what lets a served view be recomputed rather than patched.
+
+## Write the reducer
+
+```ts
+import { Projection } from "@smthrs/journal"
+import * as Effect from "effect/Effect"
+
+interface StepCounts {
+  readonly started: number
+  readonly finished: number
+}
+
+const counts: Projection.Projection<StepCounts> = Projection.make({
+  name: "docs/step-counts",
+  initial: { started: 0, finished: 0 },
+  reduce: (state, entry) =>
+    Effect.succeed(
+      entry.eventType === "step.started"
+        ? { ...state, started: state.started + 1 }
+        : entry.eventType === "step.finished"
+        ? { ...state, finished: state.finished + 1 }
+        : state
+    )
+})
+```
+
+`reduce` returns an `Effect`, so a projection may read a service. Keep it
+deterministic anyway: an entry that is not yours returns the state unchanged,
+as the fall-through above does.
+
+## Run it
+
+`project` folds `stream` through the reducer and emits a state per step. It
+emits `initial` first, then one state for each entry:
+
+```ts
+import { Journal } from "@smthrs/journal"
+import * as Stream from "effect/Stream"
+
+const snapshot = Effect.gen(function*() {
+  const journal = yield* Journal.Journal
+  return yield* journal.project(counts, { runId }).pipe(
+    Stream.take(4),
+    Stream.runCollect
+  )
+})
+```
+
+For a run with three entries, that collects:
+
+```text
+[
+  { started: 0, finished: 0 },
+  { started: 1, finished: 0 },
+  { started: 1, finished: 1 },
+  { started: 2, finished: 1 }
+]
+```
+
+`afterSequence` resumes a fold from a cursor, and then only the entries after
+it are folded.
+
+## Bound the stream
+
+`project` inherits `stream`'s behavior: it replays and then follows, so it
+never completes on its own. A `Stream.runCollect` or `Stream.runLast` with no
+bound hangs.
+
+Bound it one of three ways, by what you actually want:
+
+- A **snapshot**: `Stream.take(n)` where `n` is `1` plus the entry count you
+  read from `entries`.
+- A **live view**: fork the stream and interrupt the fiber when the view is torn
+  down. Interruption is clean.
+- A **condition**: `Stream.takeUntil` on the state you were waiting for.
+
+## Handle a failing reducer
+
+A reducer that fails ends the stream with `projection_failed`. The journal is
+unaffected: later emissions still succeed, and a fresh `project` call folds the
+same history again from the start.
+
+## Where projections are used
+
+[`@smthrs/notifications`](/api/notifications) derives its pending-notification
+queue this way, and [`@smthrs/gateway`](/api/gateway) recomputes a selector's
+rows from accumulated events rather than patching them, precisely because a
+reproducible fold is the only delta that cannot disagree with a fresh
+snapshot.
+
+## Next steps
+
+- [Read and follow a run](./read-a-run.md) for the reads underneath `project`.
+- [Checkpoints and compaction](../concepts/compaction.md) for what a projection
+  does when its cursor falls below the floor.