@smthrs/journal 0.0.0-stage → 1.0.0-rc.3

package/docs/concepts/compaction.md

@@ -0,0 +1,157 @@
+---
+title: "Checkpoints and compaction"
+description: "How a checkpoint pins replay state to a sequence, what the compaction floor guarantees, and why a reader below the floor gets a typed compacted failure instead of a shortened history."
+sidebar:
+  order: 5
+---
+
+A run that never stops growing eventually costs more to replay than it is worth.
+Compaction is how the journal drops old entries without ever handing a reader a
+history that is quietly missing its beginning.
+
+Compaction is off by default. Without a `compaction` policy on the layer, the
+journal never deletes an entry, and checkpointing stays a call you make.
+
+## A checkpoint is replay state pinned to a sequence
+
+`checkpoint({ runId, seq, state }, owner)` durably records the caller's own
+replay state at a committed sequence. The journal never interprets `state`; it
+round-trips verbatim, and redaction deliberately does not apply to it for the
+same reason it does not apply to any other executable state.
+
+The contract on `seq` is what makes replay work: a checkpoint at `seq` must
+subsume every entry at or below `seq`, so replay is `state` plus
+`stream({ runId, afterSequence: seq })`.
+
+`seq` must also name a committed entry of the run, and it must lie above the
+run's compaction floor. Otherwise the write fails `checkpoint_invalid`.
+Re-checkpointing an uncompacted `seq` replaces its state: last writer wins.
+
+The committed entry at `seq` is what survives compaction, and it survives for a
+mechanical reason. It holds the run's durable `MAX(seq)` allocation floor at or
+above the compaction boundary, so a process restarted after a compaction can
+never re-allocate a truncated sequence.
+
+## The floor is what a reader sees
+
+`compact({ runId, upTo }, owner)` deletes the run's entries strictly below a
+checkpoint and advances the run's compaction floor, atomically. Omit `upTo` to
+use the run's latest checkpoint.
+
+`compacted_at_ms` on a checkpoint row is null until a compaction has truncated
+below it. The largest compacted `seq` for a run is that run's compaction floor.
+
+Every read whose cursor starts below the floor fails with `compacted`, carrying
+the floor in `checkpointSeq`. `entries` and `stream` behave identically here, so
+a poller and a follower cannot disagree about what history exists. A reader
+never receives a silently shortened history.
+
+The read order is part of the guarantee. `entries` reads the page first and the
+floor second, because truncation and the floor advance commit together: any
+deletion that could have shortened the page is visible in the later floor read.
+Reading the floor first would let a compaction commit between the two reads and
+hand back a gapped history.
+
+## Two kinds of reader, two kinds of protection
+
+- **Readers this process can see.** A live in-process `stream` registers its
+  durable cursor, and `compact` refuses to truncate a sequence one of them
+  still needs, failing `reader_behind` with the checkpoint it declined to
+  truncate below.
+- **Readers this process cannot see.** A poller of `entries`, or a follower in
+  another process, is covered by the read-side `compacted` guard instead. There
+  is no way to register their cursors, so the answer is a typed failure and a
+  resync point rather than a refusal to compact.
+
+## Resyncing
+
+A reader that fails with `compacted` recovers in two calls: read
+`latestCheckpoint(runId)`, apply `checkpoint.state`, then continue from
+`stream({ runId, afterSequence: checkpoint.seq })`.
+
+`latestCheckpoint` returns the run's most recent checkpoint whether or not it
+has been compacted, so it is the resync point in both cases.
+
+## The automatic policy
+
+`SqlJournalOptions.compaction` turns the manual calls into a policy:
+
+```ts
+import { SqlJournal } from "@smthrs/journal"
+
+const layer = SqlJournal.layer({
+  capacity: 1024,
+  overflow: "reject",
+  compaction: {
+    entryThreshold: 5_000,
+    capture: (runId, upTo) => captureReplayState(runId, upTo)
+  }
+})
+```
+
+Once a run's committed entry count reaches `entryThreshold`, the journal asks
+`capture` for the caller's replay state at the run's durable tail, writes it as
+a checkpoint at that sequence, and compacts below it.
+
+The policy is the journal's own post-commit maintenance rather than a caller's
+entrypoint, so it holds no fence and needs no lease. It only ever
+truncates below a tail the run's own commits produced, and a retry after a
+reclaim compacts the same committed prefix the live owner sees.
+
+Three properties of the hook matter when you write `capture`:
+
+- It runs after the triggering commit's allocation permit is free. Lossy
+  commits schedule one scoped maintenance fiber per compacting run, so capture
+  and compaction never occupy the queue-draining fiber. Capture may read or
+  emit through the journal while other runs continue committing.
+- It is interrupted after 30 seconds, so caller code cannot wedge journal
+  admission indefinitely.
+- A failed or refused attempt, whether a live stream behind the boundary or a
+  `capture` failure, is logged at warning, damped for `entryThreshold` further
+  committed entries while the run's policy counter is retained, and never
+  surfaced to the emit that triggered it.
+
+`flush` waits for queued writes and registered automatic maintenance attempts.
+The attempt is registered before its triggering batch settles, so flush cannot
+miss it. Scope closure flushes before interrupting the maintenance scope.
+Durable emits still await their own policy attempt after commit.
+
+Flush also retires per-run barriers and policy counters when there are no
+admissions, writes, queued entries, barrier waiters, live readers, or maintenance
+using the run. A later commit seeds its policy counter from durable history
+again, including after a previously damped failure. Sequence allocation floors
+remain in memory: dropped entries and rolled-back transactions can consume
+sequences that cannot be recovered from durable rows. Retirement does not
+permit those sequences to be reused.
+
+## Retries are idempotent
+
+A `compact` whose checkpoint is already the floor returns `deleted: 0` rather
+than failing. The rows below it are already gone.
+
+## Producer identities survive compaction
+
+Compaction retains each deleted event's producer identity, event ID, original
+canonical sequence, and SHA-256 fingerprint of its encoded type, payload, and
+metadata in `flows_journal_dedup`. These records commit in the same transaction
+as event deletion and the floor advance. They contain no replay payload.
+
+An exact producer retry returns `Duplicate` with the original sequence, even
+after reopening the database. Changed content fails `idempotency_conflict`;
+`dedupe: "identity"` keeps its identity-only behavior. Queued retries cannot
+append another row or enter the checkpoint suffix. Owner fencing still applies.
+The retained identities also preserve each producer's automatic `sourceSeq`
+allocation floor when all its event rows have been compacted.
+
+Identity records have no expiry and survive later compactions. Storage therefore
+remains proportional to the number of compacted identities, while event payloads
+can be reclaimed. Migration `0004_dedup` adds this retention for subsequent
+compactions; identities deleted before the upgrade cannot be reconstructed.
+
+## Related reading
+
+- [Compact a long-running run](../guides/compact-a-run.md) is the task-shaped
+  version of this page.
+- [Run a read-only follower](/docs/guides/sync-followers/) covers a follower
+  that meets the floor.
+- [Retention](/docs/guides/retention/) covers what a deployment keeps overall.

package/docs/concepts/idempotency.md

@@ -0,0 +1,127 @@
+---
+title: "Producer identity and idempotency"
+description: "What makes two emissions the same event, why the comparison runs on canonical redacted bytes, what the two dedupe modes mean, and why the in-process index is a cache rather than the authority."
+sidebar:
+  order: 3
+---
+
+A producer that retries must not double an entry. The journal answers that with
+one identity and one comparison, both of which are persisted facts rather than
+in-memory bookkeeping.
+
+## The identity
+
+`(runId, sourceId, sourceSeq)` is the producer identity. It is unique in the
+database, and `JournalEvent.makeEventId` derives the durable `eventId` from it
+deterministically, with length prefixes so an identifier containing the
+separator cannot forge another tuple's id.
+
+Deterministic is the point: retrying the same source event must produce the
+same durable id, so the unique index can collapse the retry onto the committed
+row instead of storing it twice.
+
+## What makes two emissions the same event
+
+Two emissions under one identity are the same event when their `eventType` and
+their canonically encoded, redacted `payload` and `meta` match.
+
+Canonical encoding is `JSON.stringify` semantics with object keys sorted, so
+key order does not change the answer. A `Date` is its ISO string, an
+`undefined` member is dropped, and `NaN` is `null`. The sort runs over the
+encoded JSON rather than over the raw value, so canonicalization cannot destroy
+anything the encoder would have kept.
+
+Two consequences follow from comparing the persisted, redacted bytes, and both
+are deliberate:
+
+- Two different secrets that redact to the same placeholder are the same event
+  to the journal. Comparing the pre-redaction value instead would keep
+  unredacted secrets resident in the in-process index, which is the leak this
+  package exists to prevent.
+- `NaN` and `null` are the same value, because JSON says so.
+
+## The two dedupe modes
+
+`Input.dedupe` states what a re-emitted identity means. It defaults to
+`content`.
+
+`content` is the strict reading: the identity names one set of bytes, so a
+producer that re-emits it with different bytes has a bug, and the journal says
+so with `idempotency_conflict`.
+
+`identity` is for a producer that derives the sequence from the event itself.
+There a collision is the same event observed twice, and the bytes that differ
+between the two observations are metadata about the observation rather than the
+event: whether a replayed frame was re-recorded, how long a step took the
+second time a durable engine served it from its record. The first admitted row
+stands and the re-emission settles as `Duplicate`.
+
+Declare `identity` only with a sequence derived from the event's own content.
+Otherwise the journal has nothing left with which to notice two different
+events wearing one identity.
+
+## The index is a cache, not the authority
+
+`SqlJournalOptions.sourceEventCache` (default 4096) bounds the in-process index
+that answers producer idempotency from memory. The database unique constraint
+on `(run_id, source_id, source_seq)` stays authoritative, so a miss changes the
+receipt, not the durable answer.
+
+On a miss for an explicit producer sequence, the entry is admitted
+optimistically without a SQL deduplication lookup:
+
+- `emitLossy` returns `Accepted` where a resident entry would have returned
+  `Duplicate`.
+- The insert collapses onto the committed row through the unique index rather
+  than doubling it.
+- A changed retry behind an evicted entry surfaces its `idempotency_conflict`
+  from `flush` instead of from the emit.
+
+The deduplication lookup is deliberately deferred to the writer. Allocation
+can still read before admission: a cold run needs `MAX(seq) + 1`, and an
+omitted `sourceSeq` with an uncached producer floor also needs
+`MAX(source_seq) + 1` for `(runId, sourceId)`. Only warmed allocation floors
+make admission read-free; an explicit producer sequence avoids only the
+producer-floor read.
+
+These reads use the caller's SQL transaction context when the client is shared.
+A caller holding the connection in another transaction without propagating
+that context can deadlock on a cold admission. See
+[the two channels](./two-channels.md) for supported transaction compositions
+and the separate commit of queued lossy entries.
+
+Bounding the index also bounds startup. Only the most recent
+`sourceEventCache` events are decoded when the layer is built, so resident
+memory and startup cost are proportional to the bound rather than to the total
+history ever written.
+
+## Identifiers are the persisted key
+
+`RunId`, `SourceId`, and both an `Input`'s and an `Entry`'s `eventType` are
+non-empty, at most 1,024 UTF-16 code units, free of unpaired UTF-16 surrogates,
+and free of NUL. Each rule closes a way for the identifier the caller decoded
+and the identifier the database stores to stop being the same value:
+
+- SQLite binds a lone surrogate as U+FFFD, so two ill-formed identifiers that
+  differ only in their surrogates land on one persisted key. The second run's
+  first event would dedupe into the first run's row, and a read by either id
+  would return the same history.
+- SQLite's `length()` stops at the first NUL, so a NUL-bearing identifier fails
+  the column's own non-empty check and the caller is told the sink failed about
+  an identifier it had just supplied.
+- The length bound exists because every identifier occupies permanent index
+  space and layer-lifetime heap.
+
+Valid astral text is ordinary text and round-trips exactly. `entries`,
+`stream`, `checkpoint`, `latestCheckpoint`, and `compact` decode the same
+schemas, so an identifier the writer refuses is refused on every read with
+`invalid_event` rather than answered with an empty page.
+
+## Reading a receipt
+
+`Duplicate` carries a `status`, which is the one piece of information the tag
+alone does not give you:
+
+- `pending` means the original event is in the lossy queue and not yet
+  committed.
+- `committed` means the original is already durable.

package/docs/concepts/owner-fence.md

@@ -0,0 +1,124 @@
+---
+title: "The owner fence"
+description: "Why the durable channel takes an OwnerId, what makes a write fail with fence_lost, and when the unfenced channel is the correct one."
+sidebar:
+  order: 2
+---
+
+A run can outlive the process that claimed it. A machine pauses, a supervisor
+declares the run abandoned, another process takes it over, and then the first
+process wakes up holding a half-finished lifecycle write. Without a fence, that
+zombie write lands behind the live successor and the run's history says two
+processes were driving it.
+
+The journal refuses that write. `emitDurable`, `checkpoint`, and `compact` each
+take an `OwnerId`, and each lands only while the consensus strategy the
+journal was built with still records that owner as holding the run.
+
+## The token
+
+`OwnerId.OwnerId` is three fields:
+
+```ts
+import type * as OwnerId from "@smthrs/journal/OwnerId"
+
+const owner: OwnerId.OwnerId = {
+  hostId: "host-1",
+  pid: process.pid,
+  nonce: "run-1-claim"
+}
+```
+
+`hostId` and `nonce` are plain strings. `pid` is a real operating-system
+process id, so the schema states that: a non-negative integer.
+
+The token lives in this package rather than with the ownership arbitration in
+[`@smthrs/run-store`](/api/run-store) because the journal is what it fences.
+The run store stores the token on runs and decides who holds it, and its
+`Ownership` module re-exports `OwnerId` alongside that arbitration.
+
+## What the fence actually checks
+
+A fenced write joins the injected `Consensus` strategy's `guard` inside its
+own write transaction. The guard succeeds while the strategy records the
+supplied token as the run's owner and fails `fence_lost` otherwise, and
+because the durable writer serializes write transactions no reclaim can
+commit between the guard and the statements beside it.
+
+Two consequences follow:
+
+- A run that another process reclaimed fails the write with `fence_lost`.
+- A run whose owner released it, because it suspended or finished, fails the
+  same way. A finished run does not accept late lifecycle writes.
+
+The strategy is the journal's own service. `SqlJournal.layer` fences through
+`SqlConsensus`, whose lease lives in `flows_consensus_leases`, a table this
+package's migrations create; `SqlJournal.layerWith` fences through whatever
+strategy the composition provides, such as `Consensus.layerLocal` for a
+single process. The same instance must arbitrate the run store's claims, which
+is why `RunStore.layerWith` takes the strategy too.
+
+## Who arbitrates
+
+The journal owns the rules: one writer per run, fenced and unfenced appends,
+commit-time admission, the generation fence, steal only on staleness plus
+liveness evidence, and ownership transitions as events. A strategy chooses
+where the lease lives and how the fence is checked. The claim lifecycle is
+two-phase — `claim` or `steal` reserves the run and returns a grant
+timestamp, `activate` presents that grant to take ownership, `release` gives
+it back, and `recover` clears a dead claimant's stale claim — and
+[`@smthrs/run-store`](/api/run-store) drives it for a durable run, mirroring
+the outcome on the run row in the same transaction.
+
+The run store appends every transition the strategy grants — `claimed`,
+`activated`, `released`, `stolen`, `expired` — as a `flows.consensus.*` fact on
+the run's `run-store` companion stream (`JournalEvent.companionRunId`), in the
+transaction that made it. The facts live beside the run's own stream, so a
+consumer reading that stream by position never sees one. Heartbeats renew the
+lease and never enter the journal.
+
+## Reconfirm after a host stall
+
+`Consensus.reconfirm(runId, owner, nowMs)` returns `Renewed` only while the
+same owner holds the lease and no claim is pending. Its compare-and-swap
+renews the heartbeat monotonically. A missing lease, foreign owner, or
+pending claim returns `Lost` without modifying the lease.
+
+`SqlConsensus` performs that check in one update; `layerLocal` performs it
+atomically in memory. If reconfirm wins a race with `steal`, the refreshed
+heartbeat prevents the steal. If steal wins, its pending claim prevents
+reconfirm even before the new owner activates. Reconfirm preserves the
+owner token and generation. The engine records a `lease-reconfirmed`
+decision when supervision uses it to recover from an unconfirmed interval.
+
+## A bad token is not a lost fence
+
+An owner that is missing, null, or not an `OwnerId` at all fails
+`invalid_event`. That is a caller contract violation, and reporting it as
+`fence_lost` would send the caller hunting a race that never happened. A
+fractional or negative `pid` is the common case.
+
+Read the two codes as different questions:
+
+- `fence_lost` asks who owns this run now. Stop writing.
+- `invalid_event` asks what you passed. Fix the argument.
+
+## The unfenced escape
+
+`emitDurableUnfenced` is the same durability and the same receipt contract with
+no fence. It exists for admissions that are genuinely ownerless, where
+first-writer-wins is the design rather than an accident. The canonical case is
+an external trigger: a deferred completion or a clock-schedule record delivered
+by a sweeper that owns no run, where the producer dedup index rather than the
+fence is the idempotency mechanism.
+
+A caller that holds an `OwnerId` uses `emitDurable`. Reaching for the unfenced
+channel to get past a `fence_lost` writes exactly the zombie entry the fence
+exists to reject.
+
+## Related reading
+
+- [Write a fenced lifecycle event](../guides/write-lifecycle-events.md) is the
+  task-shaped version of this page.
+- [Execution IDs and ownership](/docs/concepts/ownership/) covers how a run
+  acquires and loses the token in the first place.

package/docs/concepts/redaction.md

@@ -0,0 +1,218 @@
+---
+title: "Redaction"
+description: "Why the journal scrubs payloads on the write path, what the default rules catch, where redaction deliberately stops, and the bounds that keep the scrub safe on hostile input."
+sidebar:
+  order: 4
+---
+
+A journal row is permanent and broadly readable. Every entry is replayed
+verbatim to sync followers, to time-travel consumers, and into a support
+bundle. A credential that reaches `payload_json` is therefore not a transient
+leak, and no reader can undo it.
+
+So the journal redacts once, on the write path. Every write funnels through one
+preparation step, and that step scrubs `payload` and `meta` before they are
+encoded. No channel can persist a credential.
+
+## What the default rules catch
+
+Redaction has two halves.
+
+**Structural.** A field whose name reads as a credential has its value replaced
+wholesale, whatever the value is: `authorization`, `cookie`, `apiKey`, `token`,
+`password`, `passphrase`, `secret`, `secretAccessKey`, `credential`, `dsn`,
+`connectionString`, `privateKey`, `sshKey`, `session`, `signature`, `auth`, and
+the rest of that family, plus a trailing `key` that is a word of its own (`key`,
+`api_key`, `x-api-key`). `Redaction.isSensitiveKey` is the predicate, and
+`Redaction.placeholder` is the string that lands, `"[REDACTED]"`.
+
+The match is a suffix test on the separator-free lowercase form of the name,
+not a substring test, so `tokenizer`, `secretary`, `monkey`, and
+`idempotencyKey` stay intact. Redacting them would destroy data in a permanent
+row without protecting anything, and `idempotencyKey` in particular is the
+durable identity an effect boundary replays on. Token counters are kept too: a
+non-negative safe integer under `inputTokens`, `outputTokens`,
+`cachedInputTokens`, `reasoningTokens`, or `totalTokens` is accounting, not
+bearer material.
+
+**Textual.** `Redaction.defaultRules` scans inside any string for credential
+shapes seen in real bug reports: provider keys, bearer tokens, GitHub, AWS,
+Slack, and Google credentials, URL passwords, Basic authorization, bare JWTs,
+PEM private-key blocks, INI/YAML credential assignments such as
+`SECRET = value` and `password: value`, credential flags such as
+`--password value`, cookie and session assignments, and embedded JSON
+credential members.
+
+An assignment's value is read by a scanner rather than a regular expression,
+because where a value ends depends on quotes, escapes, and brackets. Each shape
+below once leaked the part a simpler rule stopped short of:
+
+| Shape                                      | Example                                           |
+| ------------------------------------------ | ------------------------------------------------- |
+| Quoted, in any quote `util.inspect` writes | `` password: `it's "a b"` ``                      |
+| Split by `util.inspect` over several lines | `privateKey: '…\n' +` then `'…'`                  |
+| Escaped JSON, at any depth                 | `\"password\":\"a b\"`                            |
+| A buffer, a cycle, a null prototype        | `token: <Buffer 73 65>`, `apiKey: <ref *1> { … }` |
+| A container, brackets in its strings too   | `apiKey: { note: '} ', v: '…' }`                  |
+| A `,` or `;` inside the value              | `DB_PASSWORD=Zq7;Syn,thetic`                      |
+
+A quoted value keeps its quotes around the placeholder
+(`PASSWORD="[REDACTED]"`). A name ending in `key` counts only where
+`isSensitiveKey` agrees, so `sortKey` and `idempotencyKey` keep their values,
+and a count under a plural `tokens` name (`max_tokens: 4096`) is kept.
+A credential name that is a URL's user
+(`https://x-access-token:…@github.com/acme/app.git`) takes only the password
+before `@`, so the host and path a failed clone is diagnosed by are kept.
+
+In a journal row a bare value is one token, and a string or container the text
+never closes stops at the end of its line: a stray backtick in an agent's
+Markdown must not erase the rest of a permanent row. A PEM or PGP private-key
+block with no footer is redacted to the next header or to the end of the text.
+In an argv array, the element after a credential flag is redacted.
+
+The rule set is a best-effort net, not a proof. A value that must never persist
+belongs in a `Redacted` field of the caller's own schema, where the type
+system, rather than a name-suffix guess made at the storage seam, keeps it out.
+
+Objects and arrays are rebuilt, a cycle collapses to `"[Circular]"` so the
+result always encodes, and a number, a boolean, `null`, or `undefined` is
+returned untouched. The rules are re-applied to a string until it stops
+changing, so replaying or exporting an entry cannot mutate it again; a string
+that has not settled after eight passes is replaced whole.
+
+## Diagnostics
+
+A log line, a stderr message, and an error handed to a remote caller are read
+once, so a false positive there costs a word of context rather than data in a
+permanent row. `Redaction.diagnosticRules` reads further than the journal: a
+bare value runs over several words to the end of its line
+(`password=correct horse battery`), an unclosed string or container runs to the
+end of the text (a value cut by a bound), and a value may sit on the line after
+its name. It adds an authorization or cookie header in any scheme, written as
+`name: value` or as an array pair (`[['Authorization', 'Token …']]`). Whitespace
+`Authorization Token value` and `Proxy-Authorization Token value` forms also
+match, with Token, Bearer, or Basic schemes and the value on the scheme's line.
+It adds bare URL userinfo, a signed query parameter, `curl -u user:pass` (or `-uuser:pass`),
+`zip -P`, `redis-cli -a`, and a `-p` flag's value, a password to `mysql` and
+`sshpass`. A path after `-p` and the single-dash words that are not passwords
+(`-print`, `-pthread`) are kept.
+`Redaction.redactDiagnostic` applies them, and every diagnostic path in
+Smithers shares it: `RedactedLogger`, the engine's failure rendering, and the
+CLI's error output.
+
+Redact a diagnostic before rendering or bounding it. Redacting a rendering line
+by line, or after a bound, is how a key's body escaped. A stream that must be
+emitted line by line uses `Redaction.lineRedactor()`: it holds lines while the
+rules say a value is still open and redacts the block whole once it closes. A
+value still open after `Redaction.maxHeldLines` lines or 64 KiB withholds the
+rest of the stream, and `flush` emits one placeholder for it.
+
+```ts
+import { Redaction } from "@smthrs/journal"
+
+const redactor = Redaction.lineRedactor()
+const emit = (line: string) => redactor.line(line).forEach((safe) => process.stderr.write(`${safe}\n`))
+// at the end of the stream
+redactor.flush().forEach((safe) => process.stderr.write(`${safe}\n`))
+```
+
+A custom rule whose extent is not regular sets `rewrite: (text) => string`
+instead of `replace`; `pattern` then names what it looks for.
+
+## Where redaction stops
+
+Redaction is an observability concern, and it is confined to journal events and
+to display surfaces.
+
+It deliberately does not apply to executable state: `flows_runs.state_json`,
+attempt checkpoints, errors, outcomes, cache results, and a journal
+`Checkpoint`'s own `state`. Those values are decoded and re-entered on resume.
+A placeholder there resumes the flow with the wrong data, and replacing a
+non-string value with a placeholder string makes the persisted state fail
+schema decode outright, which leaves the run undrivable.
+
+That is why [`@smthrs/run-store`](/api/run-store) and
+[`@smthrs/step-cache`](/api/step-cache) take no `redact` option at all.
+
+## Choosing a redactor
+
+`SqlJournalOptions.redact` defaults to `Redaction.make()`. Two other choices
+are supported:
+
+```ts
+import { Redaction, SqlJournal } from "@smthrs/journal"
+
+const verbatim = SqlJournal.layer({
+  capacity: 1024,
+  overflow: "reject",
+  redact: Redaction.makeNoop()
+})
+
+const extended = SqlJournal.layer({
+  capacity: 1024,
+  overflow: "reject",
+  redact: Redaction.make({
+    rules: [...Redaction.defaultRules, { id: "internal-ticket", pattern: /\bTKT-\d{6}\b/g }]
+  })
+})
+```
+
+`Redaction.makeNoop()` persists payloads verbatim by choice, for a trusted
+single-tenant store or a suite asserting on raw input. A custom rule set
+replaces the defaults, so spread `Redaction.defaultRules` when you mean to add
+to them rather than to swap them out. A non-global custom pattern is normalized
+to global once, and a sticky one keeps its stickiness.
+
+## Bounds on the scrub
+
+Redaction runs on every write, so it is bounded against hostile input rather
+than trusted to terminate:
+
+- **Depth.** The walk traverses at most `Redaction.maxDepth` (256) container
+  edges and fails a deeper payload as `invalid_event` rather than overflowing
+  the stack. The canonical encoder carries the same ceiling, so a value that
+  reaches it through `Redaction.makeNoop()` is refused the same way.
+- **Binary views.** Enumerating a typed array's own properties materializes one
+  pair per byte, so a view over `Redaction.binaryWalkLimit` (65,536) bytes, or
+  one carrying that many own members, is named rather than walked. The size is
+  read from the value's own internal slot, where nothing a caller writes can
+  answer for it.
+- **Rule cost.** Every alternation inside a repetition starts its branches
+  with disjoint characters, so each rule is linear in the length of the value
+  and no input backtracks catastrophically.
+
+A redaction that throws, from the redactor itself or from a getter or proxy
+trap on the value, fails the write as `invalid_event` with no `cause`. The
+thrown value can quote the unredacted input, so the error carries only which
+field, `payload` or `meta`, could not be redacted.
+
+Values a JSON row cannot hold are named rather than dropped:
+`Redaction.binaryMarker`, `Redaction.functionMarker`,
+`Redaction.symbolMarker`, and `Redaction.depthMarker`.
+
+## Rendering a stored column
+
+`Redaction.redactJsonString` scrubs an already-encoded JSON string at a display
+surface, leaving the durable row untouched. Use it for a rendered `state_json`
+or a support bundle, where the value is already encoded and must not be decoded
+back into the executable path.
+
+```ts
+import { Redaction } from "@smthrs/journal"
+
+const redactor = Redaction.make()
+const forDisplay = (stateJson: string): string => Redaction.redactJsonString(stateJson, redactor)
+```
+
+A string that does not parse is returned untouched: validation belongs to the
+caller, and rejecting here would turn a redaction concern into a schema error.
+Once parsing succeeds the function fails closed, returning the JSON string
+`"[REDACTED]"` rather than the original text, if the redactor throws or the
+result cannot be re-encoded.
+
+## The other half: log output
+
+The journal closes the durable half. The operator's terminal is the other place
+the same credential surfaces, and `RedactedLogger` closes it with the same
+rules. See
+[Keep credentials out of log output](../guides/redact-log-output.md).