@smthrs/artifacts 0.0.0-stage → 1.0.0-rc.12

package/docs/concepts/content-addressing.md

@@ -0,0 +1,174 @@
+---
+title: "Content addressing"
+description: "Why an artifact's address is a measurement of its bytes, what that buys, and the four invariants the store enforces to keep an address and its content from ever disagreeing."
+sidebar:
+  order: 1
+---
+
+An artifact has no name. Its address is the SHA-256 digest of its own bytes,
+spelled as 64 lowercase hexadecimal characters, and the store computed it by
+hashing what the caller handed over. That single decision is where every other
+property in this package comes from.
+
+Three things follow immediately. Two callers who produce the same bytes produce
+the same address, so storing the second copy is free. An address is a proof
+obligation the store can check at any moment, so a read can verify rather than
+trust. And a digest is safe to write into a durable row, ship over a network,
+or print in a log, because it reveals nothing about where the bytes live or who
+produced them.
+
+Smithers uses the same idea one layer up: a step key is derived from canonical
+content rather than from an activity's name, which is what
+[content addressing on smithers.sh](/docs/concepts/content-addressing/)
+describes. This package is the byte-level half of that story.
+
+## A digest is a value, and an untrusted one
+
+`ArtifactStore.Digest` is both the branded type and the schema for the
+canonical representation. `put` returns the brand, because it measured the
+bytes itself and can vouch for the form.
+
+Every read operation accepts a plain `string` instead. That is deliberate: a
+digest read back out of a durable row, a cache entry, or an HTTP body is
+untrusted input, and asking every caller to re-brand a persisted column would
+only move the trust question somewhere less careful. The store validates
+instead. `ArtifactStore.validateDigest` refuses anything that is not exactly 64
+lowercase hexadecimal characters, and it runs before any implementation logs a
+digest or interpolates one into a path or a URL:
+
+```ts
+import * as ArtifactStore from "@smthrs/artifacts/ArtifactStore"
+import * as Effect from "effect/Effect"
+
+/** Prints `invalid_digest` rather than reaching for `../../etc/passwd`. */
+const checked = ArtifactStore.validateDigest("../../etc/passwd").pipe(
+  Effect.catchTag("@smthrs/artifacts/ArtifactStoreError", (failure) => Effect.succeed(failure.code))
+)
+
+console.log(await Effect.runPromise(checked))
+```
+
+The failure message is a constant. A hostile multi-megabyte value cannot copy
+itself into a log line or a durable error by being rejected.
+
+## The four invariants
+
+### Every read is digest verified
+
+`get` measures the bytes it found and compares them against the address it was
+asked for. A mismatch is `ArtifactCorruption`, carrying both the recorded and
+the measured digest, and the bytes are never returned. This is what makes a
+truncated blob from a crashing writer, a failing disk, or a shared tier serving
+the wrong object into a loud failure rather than a silently wrong replay.
+
+The in-memory store is the deliberate exception. Its address space is a private
+`Map` keyed by the digest it measured on the way in, `put` stores a copy of the
+caller's array, and `get` hands out a copy of the stored one. No reference a
+caller can still mutate aliases the stored content, so there is no window in
+which the address and the content can disagree. The filesystem and HTTP stores
+verify because their address spaces are genuinely shared.
+
+### Publication is atomic
+
+A plain write to the canonical path could be observed, or survive a crash, as a
+partial file that every later read of that digest would trust. The filesystem
+store instead writes to a temp path inside the destination fanout directory,
+fsyncs it, and renames it into place. The rename never crosses a filesystem
+because the temp path is a sibling of the blob.
+
+Temp names carry a random per-store token drawn from Effect's `Random`, so two
+processes publishing the same digest into one workspace never share a scratch
+path and never clobber each other into torn bytes at the content address. The
+token enters no persisted identity, so its randomness is invisible to replay.
+
+`FileSystemOptions.durability` controls the fsync. `required`, the default,
+reports success only after syncing the blob, its fanout directory, the objects
+directory, and each ancestor through the relative or filesystem root. This
+persists new fanout and lock directory entries and nested configured directories.
+Every put repeats these barriers, including a deduplicated retry after an
+interrupted publication. A sync refusal fails with `ArtifactStoreError` code
+`unavailable`. `best-effort` is the explicit weaker capability for a
+host that cannot sync file or directory handles. Both modes require exclusive
+writable handles and symlink inspection; unsupported hosts fail as `unavailable`.
+
+### An existing blob is verified on every put
+
+The store does not treat "the path exists" as "the bytes are correct", and it
+does not remember a verification it already did. The objects directory is
+shared by the whole workspace, so a blob can change behind one store's back. A
+remembered proof would let a later put report success over corrupt bytes
+without repairing them, and `get` would then refuse that digest forever even
+though every put held the cure.
+
+So every put reads the existing blob, measures it, and only skips the write on
+a verified match. A mismatch, or a read the host refuses, falls through to the
+atomic rewrite and heals the address. Re-verifying costs a constant factor, not
+a new asymptote: a put already pays one hash over its own input.
+
+### A deduplicated put freshens the blob
+
+When a put deduplicates, the store stamps the blob's modification time. This is
+git's loose-object freshening, and the touch Bazel's disk cache performs on a
+hit. It matters because mtime is the age evidence a retention sweep fences its
+deletions on. Re-publishing old bytes has to read as a recent reference, or the
+grace period cannot protect the cache entry recorded moments later.
+
+The freshen is best effort. On a host with no `utimes`, a failure over a blob
+that still exists keeps the dedupe skip and accepts git's own
+freshen-versus-prune race. A failure over a blob that vanished, because a sweep
+won, falls through to the atomic rewrite instead.
+
+## What an address does not tell you
+
+An address says what the bytes are. It says nothing about whether anything
+still wants them. This package therefore never reclaims a published artifact as
+a side effect of a store call: `ArtifactSweep` is the deletion surface, and the
+mark phase that decides which digests are live belongs to
+[`@smthrs/engine-store`](/api/engine-store)'s `ArtifactGc`, the only place the
+durable roots are visible. See
+[Reclaim disk space](../guides/reclaim-disk-space.md).
+
+The one thing a store does clean up on its own is its own scratch: `.tmp-*`
+payloads and `.locks/` files older than an hour, swept once per store instance
+on its first publication. Those are crash orphans that nothing else observes.
+
+## The layout on disk
+
+The filesystem store publishes at `<directory>/<first two hex>/<digest>`, with
+the directory defaulting to `ArtifactStore.defaultDirectory`, `.flows/objects`.
+
+The two-hex fanout is Bazel's `DiskCacheClient` layout, taken "to bypass
+possible folder file count limits". The directory is workspace-relative rather
+than absolute so a workspace can be moved or copied whole and still resolve its
+own artifacts, and so a sandbox that mounts the workspace inherits them.
+
+Addresses published under the pre-1.0 flat `<directory>/<digest>` layout are
+cache misses that re-publish. There is no compatibility shim.
+
+## Prior art
+
+The contract's ergonomics follow Effect's own `KeyValueStore`
+(`effect/persistence/KeyValueStore`): one small set of total
+operations over one address space, so a memory, filesystem, or network
+implementation is the same shape. `findMissing` is Bazel's
+`MissingDigestsFinder`, one batched round trip whose result is guaranteed to be
+a subset of its input, because a per-digest existence probe over a network tier
+is the wrong shape entirely.
+
+The mechanics come from Bazel's remote-cache classes:
+
+| Taken from                                                                                                                                                                | What                                                                                     |
+| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- |
+| [`disk/DiskCacheClient.java`](https://github.com/bazelbuild/bazel/blob/master/src/main/java/com/google/devtools/build/lib/remote/disk/DiskCacheClient.java)               | The two-hex fanout layout and the fsync of the temp file before the rename               |
+| [`http/HttpCacheClient.java`](https://github.com/bazelbuild/bazel/blob/master/src/main/java/com/google/devtools/build/lib/remote/http/HttpCacheClient.java)               | The wire protocol: CAS blobs under `/cas/base16-key`, `PUT` to upload, `GET` to download |
+| [`common/MissingDigestsFinder.java`](https://github.com/bazelbuild/bazel/blob/master/src/main/java/com/google/devtools/build/lib/remote/common/MissingDigestsFinder.java) | `findMissing` as one batched probe whose result is a subset of its input                 |
+| [`CombinedCache.java`](https://github.com/bazelbuild/bazel/blob/master/src/main/java/com/google/devtools/build/lib/remote/CombinedCache.java)                             | Local first, remote second, write back what the remote returned                          |
+
+Where this package deviates is documented in [The three tiers](./tiers.md).
+
+## Related
+
+- [The three tiers](./tiers.md): how local, shared, and combined stores compose.
+- [Coordination between processes](./coordination.md): the locks and fences
+  that keep two processes out of each other's way in one objects directory.
+- [API reference](../api.md): the exact signatures and error shapes.

package/docs/concepts/coordination.md

@@ -0,0 +1,141 @@
+---
+title: "Coordination between processes"
+description: "How one objects directory stays safe when several processes write and sweep it: per-digest lock files, heartbeats, the stale-reclaim window, the backup lease, and their limits."
+sidebar:
+  order: 3
+---
+
+An objects directory belongs to a workspace, not to a process. A CLI run, a
+gateway, a sandboxed step, and a garbage collector can all be holding the same
+`.flows/objects` open at once. Publication is already atomic, so two writers
+racing on one digest is harmless. The dangerous overlap is a writer and a
+sweeper: one is deduplicating against a blob the other is deleting.
+
+`FileSystemOptions.coordination` and `SweepOptions.coordination` are the dial
+for that overlap.
+
+| Mode                 | What it does                                                                                                   |
+| -------------------- | -------------------------------------------------------------------------------------------------------------- |
+| `required` (default) | An in-process semaphore plus heartbeat-backed lock files under `.locks/`, shared by every cooperating process. |
+| `process`            | The in-process semaphore only. The explicit weaker browser and test mode.                                      |
+
+## The lock protocol
+
+Under `required`, a put or a sweep deletion takes the lock file for its digest
+before touching the blob:
+
+| Bound                          | Value                          |
+| ------------------------------ | ------------------------------ |
+| Heartbeat interval             | 10 seconds                     |
+| Stale after                    | 60 seconds without a heartbeat |
+| Acquisition deadline           | 2 minutes                      |
+| Retry interval while contended | 25 milliseconds                |
+
+A holder that crashes stops heartbeating, and the next contender reclaims its
+lock a minute later. A contender that cannot acquire within two minutes fails
+rather than waiting forever. This deadline includes the in-process semaphore
+wait, directory creation, and lockfile acquisition. It ends before the protected
+operation starts. Uninterruptible host completion and release finalizers can
+extend cancellation latency. Semaphores are scoped by filesystem service,
+objects directory, and digest; unrelated objects roots do not share permits.
+
+## The fence is bounded, not absolute
+
+Reclaiming a stale lock is serialized per lock generation. A contender reads
+the owner token, then measures the age. If the lock is stale, the contender
+races for a `wx` claim file named after that token. Only the claim winner
+renames the lock away, and only after re-reading that the path still holds the
+same stale token. Two contenders that measured the same lock as stale
+therefore move it at most once, and neither moves the fresh lock the other
+just took.
+
+The fence still assumes a holder that stops heartbeating is dead. A holder
+whose host stalls past 60 seconds is reaped while it is still running, and its
+heartbeat only logs the loss. Neither the mtime check nor the backup gate
+independently protects against that loss of mutual exclusion. The mtime check
+is followed by a separate delete, and the backup gate uses the same lock
+protocol.
+
+## Both sides must agree
+
+A lock fences only the parties that take it. Build a store and its sweep with
+the same `directory` and the same `coordination`:
+
+```ts
+import * as ArtifactStore from "@smthrs/artifacts/ArtifactStore"
+import * as ArtifactSweep from "@smthrs/artifacts/ArtifactSweep"
+import * as Layer from "effect/Layer"
+
+const directory = ".flows/objects"
+
+export const artifacts = Layer.merge(
+  ArtifactStore.layerFileSystem({ directory, coordination: "required" }),
+  ArtifactSweep.layerFileSystem({ directory, coordination: "required" })
+)
+```
+
+Nothing checks the pairing, and a mismatch is silent. A sweep on `required`
+beside a store on `process` takes lock files no writer ever observes: the fence
+reads as armed and protects nothing.
+
+A store on `process` also gives up cross-process exclusion between writers and
+sweep deletion, and a sweep on `process` skips the backup lease entirely.
+
+## The backup lease
+
+A filesystem backup freezes a database and then copies the blobs that database
+references. Publication during that window is fine, because a new blob cannot
+already be referenced by a frozen snapshot. Deletion is not: a sweep that
+removes a blob the frozen database names leaves a backup that cannot be
+restored.
+
+`ArtifactBackupLease` is the exclusion for exactly that. The backup holds one
+heartbeat-backed marker, `.backup-lease`, for the whole copy, with the same 10
+second heartbeat and 60 second staleness bound as the digest locks. Sweep
+deletion checks that marker while holding a short-lived workspace-global gate,
+`.backup-lease-gate`, so the check and the delete cannot be separated. A
+crashed lease becomes reclaimable once its heartbeat goes stale. Marker cleanup
+is registered before acquisition; interruption during marker creation or gate
+release still removes the marker owned by that acquisition. Failed gate release
+also runs marker cleanup, which may wait for the gate to become reclaimable.
+
+`ArtifactBackupLease.unlessActive` answers `None` when a live backup fenced the
+operation, and `ArtifactSweep.remove` turns that into `false`. See
+[Fence a backup against the sweep](../guides/fence-a-backup.md).
+
+## What coordination costs
+
+Under `required`, one sweep deletion takes two locks, the per-digest lock and
+then the workspace-global backup-lease gate. That is roughly ten filesystem
+operations and two forked heartbeat fibers per blob. The gate is one file for
+the whole workspace, so concurrent deletions serialize through it.
+
+That is the price of never deleting a blob a running backup already recorded.
+Size a collection window with it in mind: a sweep of ten thousand blobs is
+tens of thousands of filesystem operations, not ten thousand unlinks.
+
+## Crash orphans
+
+Lock files and temp payloads outlive a hard-killed process. Nothing else
+observes either, because reads resolve only canonical paths and a lock is
+reclaimed on contention alone, so a digest nobody publishes again would keep
+its lock file forever.
+
+Each filesystem store therefore runs one conservative sweep of its own scratch
+on its first publication: `.tmp-*` payloads and `.locks/` entries whose
+modification time is more than an hour old. An hour is far beyond any live
+writer's publication window and sixty times the lock staleness bound, so a file
+that old belongs to no living holder. A file whose age the host cannot report
+says nothing about its owner and survives.
+
+This is a sweep of scratch, not garbage collection. Reclaiming published
+artifacts is always an explicit `ArtifactSweep` call.
+
+## Related
+
+- [Reclaim disk space](../guides/reclaim-disk-space.md): the sweep, its
+  inventory, and the mtime fence.
+- [Fence a backup against the sweep](../guides/fence-a-backup.md): holding the
+  lease from a backup tool.
+- [Troubleshooting](../troubleshooting.md): what a lock timeout looks like and
+  what to change.

package/docs/concepts/tiers.md

@@ -0,0 +1,147 @@
+---
+title: "The three tiers"
+description: "One store contract with a filesystem, memory, and HTTP implementation, how CombinedArtifacts composes a local and a shared tier, and what the download policy changes about a read."
+sidebar:
+  order: 2
+---
+
+`ArtifactStore.Service` is four operations over one address space:
+
+| Operation              | Meaning                                                                                  |
+| ---------------------- | ---------------------------------------------------------------------------------------- |
+| `put(bytes)`           | Store the bytes under their own digest and return that address. Idempotent.              |
+| `get(digest)`          | Return the bytes at the address, verified against it.                                    |
+| `has(digest)`          | Whether this tier holds an artifact at the address.                                      |
+| `findMissing(digests)` | Which of these addresses this tier does not hold, as a deduplicated subset of the input. |
+
+Every implementation in this package is that same shape, which is the whole
+reason a composition can swap one for another without touching a caller.
+
+## The implementations
+
+| Constructor                                       | Backing                                             | Notes                                                                                         |
+| ------------------------------------------------- | --------------------------------------------------- | --------------------------------------------------------------------------------------------- |
+| `ArtifactStore.makeFileSystem(fs, path, options)` | A directory reached through Effect's `FileSystem`   | The durable host tier. Atomic publication, digest-verified reads, cross-process coordination. |
+| `ArtifactStore.makeMemory()`                      | A private `Map`                                     | Tests and browser hosts with no durable filesystem. Copies on both boundaries.                |
+| `ArtifactStore.makeNoop(overrides)`               | Nothing                                             | Every operation fails as `unavailable`, with per-method overrides. The honest absence.        |
+| `RemoteArtifacts.make(options)`                   | An HTTP cache reached through Effect's `HttpClient` | The shared tier. Bounded, digest-verified downloads.                                          |
+| `CombinedArtifacts.make(options)`                 | Two of the above                                    | Local first, shared second, with write-back.                                                  |
+
+`RemoteArtifacts.Service` extends the store contract with one field,
+`downloadPolicy`, so a composition can read the shared tier's own preference
+instead of being told it twice. `RemoteArtifacts.downloadPolicyOf` reads that
+field off any store and answers `undefined` for one that declares none, which
+every local store does.
+
+## What a combined read does
+
+`CombinedArtifacts.get` asks the local tier first and treats its three possible
+answers differently:
+
+| Local answer         | What happens                                                                                 |
+| -------------------- | -------------------------------------------------------------------------------------------- |
+| Bytes                | Returned. The shared tier is not contacted.                                                  |
+| `ArtifactMissing`    | Falls through to the shared tier.                                                            |
+| `ArtifactCorruption` | Falls through to the shared tier, and the fetched bytes are written back under every policy. |
+| `ArtifactStoreError` | The read fails.                                                                              |
+
+That last row is the one worth pausing on. A host that refused the read has not
+answered the question, and quietly paying for a network round trip instead
+would hide a broken local tier behind a working shared one. A miss is an
+answer; a refusal is not.
+
+The write-back is opportunistic in the same way the upload is. The bytes are
+already in hand, so a full disk, a read-only mount, or a refused sync costs the
+next read a round trip rather than failing this one. `writeBackTimeout` bounds
+the write-back, defaulting to 60 seconds. At expiry the write-back is interrupted;
+the read returns the verified remote bytes after interruption cleanup finishes.
+
+## What a combined write does
+
+`put` publishes to the local tier first and returns the local digest. The local
+tier is the one this machine's replays resolve against, so an unreachable
+shared tier must not stop an artifact from being recorded.
+
+The upload that follows is opportunistic, and its refusal is dropped rather
+than propagated. Failing here would fail whatever produced the bytes because a
+cache was unreachable, which is the opposite of the line above. Two properties
+keep it contained:
+
+- **Deduplicated in flight by digest.** Two callers in one process that publish
+  the same artifact join one upload instead of both pushing the same bytes.
+  The entry is removed before its deferred is completed, so a later put starts
+  a fresh upload rather than replaying a stale outcome, and an interrupted
+  upload resolves waiters with a typed failure instead of tearing them down.
+- **Bounded by `uploadTimeout`,** 60 seconds by default. A shared tier that
+  stalls instead of refusing must not hold the local answer hostage.
+
+Nothing depends on that upload. What actually guarantees a shared cache entry's
+blobs are durable is the publication protocol that runs before the entry is
+published: `findMissing`, then upload, then confirm. A dropped upload here
+costs that protocol one re-upload, never correctness.
+
+`has` asks local, then shared. `findMissing` asks the shared tier only about
+what the local tier could not answer, which is one network round trip over the
+smallest possible set, and the result stays a subset of the input because each
+stage filters the previous stage's output.
+
+## The download policy
+
+The dial is Bazel's `--remote_download_{all,toplevel,minimal}`, and it lives on
+the shared tier because the shared tier is the thing being conserved. Set it
+with `RemoteArtifacts.Options.downloadPolicy`, or override it for one
+composition with `CombinedArtifacts.Options.downloadPolicy`. With neither, the
+policy is `all`.
+
+| Policy     | Prefetch on replay admission | Write-back on a read                       |
+| ---------- | ---------------------------- | ------------------------------------------ |
+| `all`      | Every referenced blob        | Yes                                        |
+| `toplevel` | Nothing                      | Yes, on the first read that needs the blob |
+| `minimal`  | Nothing                      | No                                         |
+
+Two different modules honor different halves. `CombinedArtifacts.get` honors
+the write-back column. [`@smthrs/engine-store`](/api/engine-store)'s
+`ArtifactSync.hydrate` honors the prefetch column.
+
+`minimal` still writes back after local corruption. An address the local tier
+already claims, one that `has` and `findMissing` both report as present, must
+be one it can serve, or the publication protocol has been told a lie no later
+read can correct. Replacing such an address is repair, not growth.
+
+Choosing between them is a job, not a concept:
+[Share artifacts across machines](../guides/share-artifacts-across-machines.md)
+walks the composition.
+
+## What the metrics count
+
+`ArtifactStoreMetrics` defines two counters, `flows_artifact_puts` and
+`flows_artifact_gets`. Only the filesystem and memory implementations update
+them; `RemoteArtifacts` is deliberately uninstrumented, and the counters carry
+no tier attribute. Read them as local artifact store traffic:
+
+- A combined read the local tier serves counts one get.
+- A combined read the shared tier serves counts no get, because no local store
+  answered it.
+- The write-back that materializes such a read counts a put, indistinguishable
+  from a producer publishing new bytes.
+- A deduplicated put still counts: the caller stored bytes and received an
+  address either way.
+- Missing and corrupt reads count nothing. They are error evidence, not
+  throughput.
+
+A third counter, `flows_artifact_remote_failures`, counts the shared uploads
+(`operation: put`) and local write-backs (`operation: write_back`) a combined
+store dropped. Neither fails the caller, so this counter and its warning are
+the only evidence that the shared tier stopped filling.
+
+No exporter ships in this package; provide one, for example
+[`@smthrs/observability`](/api/observability), and the counters appear in it.
+
+## Related
+
+- [Content addressing](./content-addressing.md): why a read can verify rather
+  than trust.
+- [Coordination between processes](./coordination.md): what happens when two
+  processes share one objects directory.
+- [Serve the artifact protocol](../guides/serve-the-artifact-protocol.md): the
+  HTTP surface a shared tier owes.

package/docs/guides/fence-a-backup.md

@@ -0,0 +1,123 @@
+---
+title: "Fence a backup against the sweep"
+description: "Hold ArtifactBackupLease.withLease around a filesystem backup so a concurrent sweep cannot delete a blob the frozen database references, and use unlessActive when you are the one deleting."
+sidebar:
+  order: 4
+---
+
+A filesystem backup freezes a database and then copies the blobs that database
+references. Between the freeze and the last copy, a garbage collector running
+in another process can delete a blob the frozen snapshot names, leaving a
+backup that cannot be restored.
+
+`ArtifactBackupLease` is the cross-process exclusion for exactly that window.
+
+## Before you start
+
+- The objects directory the store publishes into. The lease marker lives inside
+  it, so a backup and a sweep pointed at different directories fence nothing.
+- Both sides on `coordination: "required"`. A sweep built with `process` skips
+  the lease entirely.
+
+## Hold the lease around the copy
+
+`withLease` acquires the marker, heartbeats it while your effect runs, and
+releases it afterwards. Pass a `failure` function that maps an unknown host
+cause into your own error type:
+
+```ts
+import * as ArtifactBackupLease from "@smthrs/artifacts/ArtifactBackupLease"
+import * as ArtifactStore from "@smthrs/artifacts/ArtifactStore"
+import * as Effect from "effect/Effect"
+import * as FileSystem from "effect/FileSystem"
+
+const leaseFailure = (cause: unknown): ArtifactStore.ArtifactStoreError =>
+  new ArtifactStore.ArtifactStoreError({
+    code: "unavailable",
+    message: "the artifact backup lease could not be held",
+    cause
+  })
+
+/** Runs `copy` while no sweep in any process can delete a referenced blob. */
+export const underLease = <A, E, R>(copy: Effect.Effect<A, E, R>) =>
+  Effect.gen(function*() {
+    const fs = yield* FileSystem.FileSystem
+    return yield* ArtifactBackupLease.withLease(fs, ArtifactStore.defaultDirectory, copy, leaseFailure)
+  })
+```
+
+Wrap the whole backup, from the database freeze through the last blob copy. A
+lease held only around the copy leaves the freeze unprotected, which is where
+the reference list came from.
+
+## What the lease stops, and what it does not
+
+It stops deletion. It does not stop publication: new blobs may appear
+throughout a backup, and that is harmless, because a blob published after the
+freeze cannot be referenced by the frozen database.
+
+The bounds are the same ones the per-digest locks use:
+
+| Bound                | Value                          |
+| -------------------- | ------------------------------ |
+| Heartbeat interval   | 10 seconds                     |
+| Stale after          | 60 seconds without a heartbeat |
+| Acquisition deadline | 2 minutes                      |
+
+A crashed backup stops heartbeating, and its marker becomes reclaimable a
+minute later, so a hard-killed process cannot fence collection forever. The
+flip side is that a backup whose host stalls past 60 seconds can have its lease
+reaped while it is still copying.
+
+Releasing is tolerant of that. A lease whose heartbeat lapsed has its marker
+reaped by whoever noticed, so finding nothing to release is the ordinary end of
+a slow lease rather than a fault.
+
+Acquisition is bounded rather than infinite: if another backup holds the lease
+for more than two minutes, `withLease` fails through your `failure` function
+instead of waiting.
+
+## When you are the one deleting
+
+`ArtifactSweep.remove` already consults the lease under
+`coordination: "required"`, so a collector built on that surface needs nothing
+here. Use `ArtifactBackupLease.unlessActive` only when you are writing a
+different deletion path over the same objects directory:
+
+`unlessActive` runs your effect only when no live lease exists, and it holds
+the workspace-global gate across both the check and the effect, so the two
+cannot be separated by a backup starting in between. It answers `Option.none()`
+when a backup deliberately fenced the operation, which is what
+`ArtifactSweep.remove` turns into `false`:
+
+```ts
+import * as ArtifactBackupLease from "@smthrs/artifacts/ArtifactBackupLease"
+import * as Effect from "effect/Effect"
+import * as FileSystem from "effect/FileSystem"
+import * as Option from "effect/Option"
+
+/** Deletes one path unless a live backup fenced the deletion. */
+export const removeUnlessBackingUp = (directory: string, path: string) =>
+  Effect.gen(function*() {
+    const fs = yield* FileSystem.FileSystem
+    const outcome = yield* ArtifactBackupLease.unlessActive(
+      fs,
+      directory,
+      fs.remove(path).pipe(Effect.as(true), Effect.mapError(leaseFailure)),
+      leaseFailure
+    )
+    return Option.getOrElse(outcome, () => false)
+  })
+```
+
+Treat `none` as "nothing was reclaimed, retry after the backup finishes", never
+as "the blob is gone".
+
+## Related
+
+- [Reclaim disk space](./reclaim-disk-space.md): the collector side, and why
+  its `false` is ambiguous on purpose.
+- [Coordination between processes](../concepts/coordination.md): the gate, the
+  markers, and the stale-reclaim window they share.
+- [Retention and cleanup](/docs/guides/retention/): backing up a project before
+  collecting it, on smithers.sh.