ductwork 1.1.0 → 1.1.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 95a205cfc0a928333a96512317753b11a461512ae58e38a4320969108d1b7431
-  data.tar.gz: 96a4ebef590088f0b44c71345b190934493ef44ec137be136804ecf8cad7c6d3
+  metadata.gz: bd349349c33899f1c8cdcf97339176dd72fb6397df4dbccba7445946337810b1
+  data.tar.gz: 39560169d62cd46193221f6d74f00524b39314bc470787213f891260bcc5a3db
 SHA512:
-  metadata.gz: 5785b7e08bfd89bb7f94303babc18fc8b528f26e8b16cbe0bb338b072faee51057f89fdc8d4bcc9f551bd83a81766a9d07045cb246e2cdd5c1bb8ab8e0fb6094
-  data.tar.gz: f1887e429b87d44c3e90dddd2acb5106c4e39b100689b95f1570a8c240e3e7ea08555b0efb3fce1341a9f72c66e961196e5468999390f034a1147ae0e5f1cbf7
+  metadata.gz: 62b5cc64d92c9b5caf225624bb51a252d550f48bbcbc4f5a73ffc19e464964477f41bceec74a654c783859890fcfab84305327abdf4685fa460ffbeb6e35fdff
+  data.tar.gz: 0d4483bcd3dce63092438c392cc6089f9b8ad9a8e5730cb133fbc7bacd22d0a82ff6afdabb285c447593afcc13979d3f74df3d57801861eb13815d91d5744a46

data/.claude/skills/audit-clock-drift/SKILL.md

@@ -1,11 +1,173 @@
 ---
 name: audit-clock-drift
-description: Audit ductwork for time comparisons that are prone to clock drift
-allowed-tools: Read, Grep, Glob
+description: Audit the OSS ductwork codebase for unsafe time handling across hosts — comparing a database timestamp against a Ruby in-memory clock read, measuring durations with a wall clock instead of a monotonic one, timestamp precision mismatches between Ruby and column types, and timezone-naive comparisons. Use when the user asks about clock drift, NTP skew, time comparisons, heartbeat staleness, deadline or timeout correctness, multi-host clock safety, or DatabaseClock usage.
+allowed-tools: Read, Grep, Glob, Bash
 ---
 
-# Audit Clock Drift
+# Clock Drift Audit
 
-Audit the entire OSS ductwork codebase for places where we are open to clock drift issues. Specifically, if ductwork is running across multiple hosts, where are we prone to comparing a database timestamp with an in-memory OS clock read generated with Ruby. Only look for comparisons that gate safety or visibility. For example: heartbeat, enqueueing, claiming, possibly ordering.
+Audit the OSS ductwork codebase for time handling that breaks when Ductwork
+runs across multiple hosts whose clocks disagree.
 
-For each finding: file:line, severity, why it matters, suggested fix. Do not modify files.
+Bash is available for **read-only** inspection. Do not modify files.
+
+## Step 0 — Load shared context
+
+- `.claude/skills/audit-common/method.md`
+- `.claude/skills/audit-common/severity.md`
+- `.claude/skills/audit-common/scope-boundaries.md`
+- `.claude/skills/audit-common/accepted-tradeoffs.md`
+
+## The core hazard
+
+Ductwork runs as multiple processes, potentially on multiple hosts, all
+against one database. Each host's wall clock drifts independently. Any
+comparison that mixes **a timestamp stored by the database** with **a clock
+read taken in a Ruby process** is only as correct as the skew between those
+two machines.
+
+Under drift these comparisons make healthy work look stale (premature reap,
+double execution) or stale work look healthy (never reaped, permanent stall).
+
+`Ductwork::DatabaseClock` exists to resolve exactly this, and it is the
+canonical fix for category 1 below. Findings should name it rather than
+inventing a remedy.
+
+## Step 1 — Inventory
+
+Enumerate and count before analyzing:
+
+1. Every `Time.current`, `Time.now`, `Time.zone.now`, `Date.today`, and
+   `DateTime.now` in `lib/` and `app/`.
+2. Every `Ductwork::DatabaseClock` call site (`.now`, `.ago_sql`, `.now_sql`).
+3. Every `Process.clock_gettime` call site.
+4. Every timestamp column in the migration templates, with its declared
+   precision.
+5. Every timeout/interval in `lib/ductwork/configuration.rb` and each place
+   the value is consumed.
+
+## Step 2 — Classify every time read
+
+Not every `Time.current` is a bug. Most are fine. Sort each call site into
+one of these, and only the first three are reportable:
+
+| Class | Reportable? |
+|---|---|
+| **A. Cross-host comparison** — Ruby clock read compared against a DB-stored timestamp, gating a safety or visibility decision | **Yes** — usually High/Critical |
+| **B. Wall-clock duration** — elapsed time or a deadline measured by subtracting/adding wall-clock reads | **Yes** — usually Medium/High |
+| **C. Precision or timezone mismatch** — read and column disagree on resolution or zone | **Yes** — usually Medium |
+| D. Same-process comparison — both reads from one process's clock, used only for that process's own bookkeeping | No, unless it's also class B |
+| E. Recorded value — a timestamp written for display, audit, or metrics, never compared to gate a decision | No |
+| F. Test/factory/dashboard code | No |
+
+Class D is the common false positive: comparing two in-memory reads inside
+one process is drift-safe by construction, because there is only one clock.
+It may still be a class B bug if the interval matters. Check for that, then
+move on — do not file it as drift.
+
+## Step 3 — Checklist
+
+### 3.1 Cross-host comparison (class A)
+
+The critical category. For each, ask: does one side of this comparison come
+from the database and the other from Ruby?
+
+Highest-risk surfaces — check each explicitly:
+
+- **Heartbeat staleness.** Comparing `last_heartbeat_at` (written by process
+  A) against a clock read in process B decides whether to reap A. Drift here
+  reaps live processes or leaves dead ones running.
+- **Claim eligibility.** Any `WHERE ... <= ?` where the bind is a Ruby time
+  and the column is DB-written — including retry-after / backoff gates and
+  availability windows.
+- **Reap sweeps and global timeouts.** "Started more than N seconds ago"
+  predicates.
+- **Ordering.** `ORDER BY` on a column written by many hosts is already
+  approximate; a finding here needs a real consequence, not just imprecision.
+
+**Fix:** push the comparison into SQL so both sides resolve on the database
+server — `DatabaseClock.ago_sql(column, interval)` for "older than N
+seconds", `DatabaseClock.now_sql(column)` for "at or before now". Where a
+materialized value must be written, use `DatabaseClock.now` so the value
+originates from the same clock everything compares against.
+
+**Also check the writes.** Storing `Time.current` into a column that another
+host later compares against the DB clock reintroduces the skew from the write
+side. Both ends must agree on which clock is authoritative.
+
+### 3.2 Monotonic vs wall clock (class B)
+
+A distinct bug from drift, and easy to miss because it is single-host. Wall
+clocks step — NTP corrections, DST, manual sets, VM resume. A duration or
+deadline computed from wall-clock reads can jump backward or forward
+arbitrarily.
+
+Check every:
+
+- `deadline = Time.current + timeout`, then `while Time.current < deadline`
+- `(Time.current - some_earlier_read) > threshold`
+- Shutdown budgets, kill budgets, poll loops, backoff computation
+
+**Fix:** `Process.clock_gettime(Process::CLOCK_MONOTONIC)` for anything
+measuring *elapsed* time. Wall clock is correct only for timestamps that must
+be meaningful to a human or comparable across processes.
+
+Note the asymmetry when reporting: for a shutdown budget a backward clock
+step means the loop waits far too long; for a staleness threshold it means
+the check never fires. Say which.
+
+The codebase already uses `CLOCK_MONOTONIC` in at least one place — cite it
+as the in-repo precedent so the fix reads as consistency, not novelty.
+
+### 3.3 Precision and truncation (class C)
+
+- Do timestamp columns declare a precision, and does it match what the code
+  compares against? `DatabaseClock` emits `CURRENT_TIMESTAMP(6)` on
+  MySQL/Trilogy — microseconds. A MySQL `DATETIME` declared with no precision
+  stores **whole seconds** and truncates on write. A stored value can then
+  appear up to a second *earlier* than it was, making `<=` comparisons fire
+  early or late near the boundary.
+- SQLite stores timestamps as strings; `julianday()` comparison and string
+  comparison do not order identically for mixed formats.
+- Does Ruby write sub-second precision the column cannot hold?
+- Are two columns compared against each other stored at different precisions?
+
+Grade these by whether the truncation can cross a decision boundary. If the
+threshold is 60 seconds, a one-second truncation is Low. If a claim gate
+compares near-simultaneous timestamps, it is not.
+
+### 3.4 Timezone handling
+
+- `Time.now` (system zone) rather than `Time.current` (app zone) — a real bug
+  when hosts have different `TZ`.
+- Comparing a zone-aware value to a naive one.
+- `Date.today` in any gating logic — it's the system zone and it rolls over
+  at different instants per host.
+
+### 3.5 The single-clock assumption
+
+`DatabaseClock` is only safe if every process reaches the *same* clock.
+Verify and report where it does not hold:
+
+- A read replica serves a different `clock_timestamp()` than the primary. Are
+  any of these comparisons on a connection that could be routed to a replica?
+- CockroachDB is multi-node; its clock guarantees differ from single-primary
+  Postgres. Does anything assume tighter ordering than Cockroach provides?
+- Does `DatabaseClock` handle every supported adapter, and does the `else`
+  branch raise rather than silently falling back? An adapter that reaches the
+  fallback would be comparing against nothing.
+
+## Step 4 — Verify and report
+
+Follow `method.md`. Additionally, for each finding state **which two clocks
+are being compared** and **which direction of skew causes which failure**.
+A clock finding without that is not actionable:
+
+> `process.rb:NN` compares `last_heartbeat_at` (written by the worker host)
+> against `Time.current` on the supervisor host. If the supervisor's clock
+> runs fast by more than the timeout, live workers are reaped and their
+> in-flight work is re-claimed while still running.
+
+Per `severity.md`, cross-host comparisons that gate reaping or claiming are
+High or Critical because the failure is silent double execution or a
+permanent stall. Recorded-but-uncompared timestamps are not findings.

data/.claude/skills/audit-common/accepted-tradeoffs.md

@@ -0,0 +1,118 @@
+# Accepted Tradeoffs — Do Not Report
+
+Each item below has been investigated and consciously decided. Re-reporting
+them wastes the reader's attention and buries real findings.
+
+**Do not file these as findings.** If the audit surfaces one, mention it in a
+single line under a "Known, previously accepted" heading at the end of the
+report — or omit it entirely.
+
+**Exception:** report it *only* if you have genuinely new information — a
+concrete interleaving, call site, or consequence not covered by the reasoning
+recorded here. If so, lead the finding with what is new. Do not re-argue the
+original decision.
+
+---
+
+## 1. Branch claiming does not use `SKIP LOCKED`
+
+Candidate selection (`BranchClaim#find_candidate_branch_id`) joins `Branch` to
+`Step`, and SQLite must be supported uniformly (no adapter split like job
+claiming's `RowLocking` / `OptimisticLocking` pair). That forced branch
+claiming into a universal CAS: `SELECT` a candidate, then a guarded `UPDATE`,
+then check rows-updated. Locking across the Branch↔Step join would also
+contend with the job-claiming path's own `Step` writes.
+
+Acknowledged 2026-07-22 as a data-model corner, acceptable for now.
+
+**Also accepted:** the resulting thundering herd. With no `SKIP LOCKED` and no
+randomization, every advancer thread converges on the same
+`ORDER BY last_advanced_at LIMIT 1` row. At ~1M branches this makes the
+select→update gap a routine stale window. It costs **wasted claim attempts,
+not corruption**. Do not file the contention as a correctness bug.
+
+**Still in force — this rule is auditable:** whatever the candidate `SELECT`
+filters on, the claiming `UPDATE` must re-assert. A *new* divergence between
+those two predicates is a real finding.
+
+## 2. Layer 3 per-claim heartbeats — rejected for OSS
+
+The plan for `last_progress_at` on Advancement/Execution plus a reaper sweep,
+to detect threads that are alive but stuck, was rejected.
+
+Acting on "thread stuck mid-job" means killing the thread, which requires
+bounding user-code runtime — that is the step-timeout feature, which is Pro
+by design. You cannot distinguish "hung" from "legitimately slow" without it.
+The remaining case, a thread hung in *framework* code between claims with no
+execution claimed, is rare, short-lived, and holds nothing worth reaping.
+
+Do not re-propose per-claim heartbeats. The OSS durability story is: ensure
+blocks plus restart cleanup for thread crashes, process-heartbeat reaper for
+process crashes, at-least-once documented for the rest.
+
+## 3. Reaper race 2 (process record drift) — deferred
+
+A reaped-then-resumed process has worker threads holding in-memory ownership
+of records that were already released and possibly re-claimed. The worker can
+overwrite reaper state and the job can run twice.
+
+Deliberately deferred. Acceptable while Ductwork assumes idempotent jobs. The
+fix — an ownership check at the worker's commit boundary — lands when
+non-idempotent work does, or if drift is observed in the wild.
+
+Do not file this as Critical double-execution. **At-least-once under process
+death is the documented contract.**
+
+Race 1 (the reaper stomping a legitimate concurrent claim) is *not* on this
+list — it is an active concern. Note that the design memo proposing
+`Advancement#abandon!` / `Availability#abandon!` was superseded: the shipped
+mechanism is a claim fence token (`Branch#claim_fence_token`, checked in
+`Branch#release!` at `lib/ductwork/models/branch.rb:183`) plus
+`BranchClaim#fail_abandoned_advancement`. Audit what is in the tree, not the
+memo.
+
+## 4. MySQL durability-spec segfault — deferred
+
+`spec/integration/durability/*` intermittently segfaults on mysql2 only. A
+fault-injection `kill` terminates a thread holding a mysql2 connection
+mid-query; the C client's fiber-ownership flag is never cleared, the poisoned
+connection returns to the pool, and teardown double-frees in libmysqlclient.
+
+Diagnosed as **test isolation, not a production-code bug**. Fix deferred by
+explicit decision (2026-06-23) until it shows up often enough in CI to matter.
+
+## 5. No `status` column on transitions or advancements
+
+State is derived from timestamps and error columns by design — in progress is
+`completed_at IS NULL`, succeeded adds `error_klass IS NULL`, failed is
+`error_klass IS NOT NULL`. Do not propose adding a status enum. Do not report
+the derivation as a missing-column problem.
+
+## 6. No `stuck` pipeline state
+
+Considered and rejected. Revival, hooks, dashboards, and transitions do not
+diverge between "stuck" and "halted by failure", and a state enum must earn
+its place through divergent behavior rather than labeling. Cause metadata
+lives on `Branch#halt_reason` instead. Pipeline and Run states stay
+`in_progress`, `completed`, `halted`.
+
+## 7. `Ductwork.validate!` is not run at boot
+
+It runs in host-application specs, deliberately, for developer experience. Do
+not recommend moving it to boot-time or engine initialization.
+
+---
+
+# Known Open — report only with new information
+
+Not settled, but already on record. Same rule: lead with what is new.
+
+- **`PipelineAdvancer#kill` uses `thread.kill`**, which could poison a mysql2
+  connection the same way the spec segfault does, if it ever fires mid-query.
+  Flagged as worth a later look; production impact unconfirmed. A finding
+  here needs a concrete path showing it firing mid-query.
+- **`Ductwork::UserJobError` wrapper.** The rescue layering
+  (`Job#execute` owns user code, `JobWorker#work_loop` owns everything else,
+  `Branch.with_latest_claimed` owns token-conditional release) would be
+  narrower and safer if each layer could rescue by class. Known shape to aim
+  for, not implemented. Report only a *new* concrete misrouting it causes.

data/.claude/skills/audit-common/method.md

@@ -0,0 +1,86 @@
+# Shared Audit Method
+
+Every `audit-*` skill in this repo follows this procedure. The topic-specific
+SKILL.md supplies the *what*; this file supplies the *how*.
+
+## Step 1 — Build the inventory before analyzing
+
+Do not start grepping for problems. First enumerate the surface in scope and
+write the list down. The audit is only as complete as this list.
+
+- Use Glob/Grep to enumerate every file in the topic's surface.
+- Record the count. State it in the report.
+- Work the list. A file that was never opened is not a file that passed.
+
+"Audit the entire codebase" without an inventory produces a handful of
+findings from whatever grep happened to surface first. That is the single
+biggest cause of a thin audit.
+
+## Step 2 — Analyze against the topic checklist
+
+Follow the topic skill's categories in order. For each item on the inventory,
+ask the checklist questions explicitly rather than scanning for anything that
+looks wrong.
+
+Prefer reading whole files over grepping for patterns. Durability,
+concurrency, and adapter bugs live in the *relationship* between lines — the
+order of two writes, the extent of a transaction, what a rescue does and does
+not cover. Grep finds none of that.
+
+## Step 3 — Verification pass (required)
+
+Before writing the report, re-read every cited `file:line` and confirm the
+code still says what the finding claims.
+
+Drop any finding where:
+- The line number no longer matches the quoted code.
+- The concern is already handled somewhere the first pass missed (a guard in
+  the caller, a CAS predicate, an outer `ensure`, a DB constraint).
+- It appears in `accepted-tradeoffs.md`.
+- It is out of scope per `scope-boundaries.md` (notably: Pro features).
+
+State how many candidate findings were dropped in verification. A pass that
+drops nothing usually means the verification did not actually happen.
+
+## Step 4 — Deduplicate
+
+A single line can legitimately trip several audits — a `Time.current`
+comparison inside a claim query is both a clock-drift and a durability
+finding. Within one audit, report each root cause **once**, at the deepest
+layer that owns it, and list the other affected call sites underneath it.
+Do not file the same root cause once per call site.
+
+## Output contract
+
+Report inline in the response. Do not write a report file unless asked, and
+do not modify any source file.
+
+Order findings by severity (see `severity.md`), highest first. Group by
+category only when there are enough findings that grouping helps.
+
+Each finding uses this shape:
+
+```
+### [SEVERITY] Short title
+`path/to/file.rb:123`
+
+**What:** The specific code and what it does.
+**Why it matters:** The concrete failure — the sequence of events that
+produces data loss, a stall, double execution, or a wrong result. Not
+"this is risky."
+**Fix:** A specific change. Name the existing helper or pattern in this
+codebase that it should use, if one exists.
+**Confidence:** high | medium — and for medium, what you could not verify.
+```
+
+Close with a short summary: files inventoried, files read, findings by
+severity, candidates dropped in verification.
+
+## When there are no findings
+
+Say so plainly. State what was inventoried and what checklist categories were
+checked and came back clean. A clean audit is a valid and useful result.
+
+Do not pad a report with speculative or stylistic findings to make it look
+thorough. A "Low" that is really a preference is noise, and it trains the
+reader to skim.

data/.claude/skills/audit-common/scope-boundaries.md

@@ -0,0 +1,101 @@
+# Audit Scope & Boundaries
+
+## The OSS / Pro line
+
+This repo is the OSS `ductwork` gem (LGPL v3). A paid `ductwork-pro` gem
+extends it via `prepend`. **Pro features are not gaps in OSS.** They are
+deliberately absent.
+
+An audit that recommends a Pro feature as the fix for an OSS finding is
+producing noise, and it is the single most common failure mode of the
+durability and reliability audits in particular.
+
+### In OSS — fair game to audit
+
+- Core transitions: `chain`, `expand`, `divide`, `divert`, `combine`,
+  `converge`, `collapse`
+- Core pipeline DSL
+- Two-phase commit (transition + advancement records) for advancement
+- Supervisor / advancer / worker process hierarchy
+- Forking and threaded concurrency modes
+- Configurable advancer thread pool
+- Heartbeat-based orphan detection
+- SKIP LOCKED claiming with atomic `UPDATE...WHERE` fallback
+- Reaper with global-timeout sweeps
+- Restart of worker threads stuck in *framework* code (no execution claimed)
+- `Ductwork::Pipeline#revive!`
+- UUID v7 primary keys across PG / MySQL / SQLite
+- Rails engine-mountable web dashboard
+
+### In Pro — do NOT recommend, do NOT report as missing
+
+- Human-in-the-loop / the `dampen` transition
+- **Step timeouts** defined in the DSL
+- **Step delays** defined in the DSL
+- Restart of worker threads stuck *inside job execution* (a claimed execution
+  that will not return) — this requires bounding user-code runtime, which is
+  the step-timeout feature
+- Large payload support
+- Resumable batched fan-out / fan-in
+- Interruptible pipeline advancement
+- StatsD metric reporting
+
+### Hard rules
+
+- Never reference `Ductwork::Pro::*` constants from OSS code, and never
+  suggest that OSS code do so.
+- OSS must remain fully functional standalone. A finding whose fix requires
+  Pro is not a valid OSS finding.
+- If the correct fix genuinely lies in Pro, say so in one line and move on.
+  Do not file it.
+
+## Supported databases
+
+Claimed support, per `CLAUDE.md` and the install migration templates:
+
+| Adapter | Notes |
+|---|---|
+| PostgreSQL | Primary target |
+| CockroachDB | PG wire protocol, divergent semantics |
+| MySQL 8+ | Both the technology and the `mysql2` adapter |
+| Trilogy | MySQL-compatible adapter |
+| SQLite | No partial-index-free workarounds; drives "uniformity" constraints |
+| Oracle | |
+
+**CI covers only postgres, mysql, trilogy, and sqlite**
+(`.github/workflows/main.yml`). Oracle and CockroachDB are claimed as
+supported with no automated coverage. Claimed-but-untested support is itself
+a legitimate finding for `audit-database-support` — flag it there, not in
+every audit.
+
+## Code layout
+
+- `lib/ductwork/` — core: claiming, clock, config, context, fault injection
+- `lib/ductwork/models/` — ActiveRecord models (the durability surface)
+- `lib/ductwork/processes/` — supervisor / advancer / worker hierarchy and
+  their runners
+- `lib/ductwork/dsl/` — pipeline definition DSL
+- `lib/generators/ductwork/install/templates/db/` — install migrations
+- `lib/generators/ductwork/update/templates/db/` — **upgrade migrations**
+- `app/` — the mountable dashboard engine
+- `spec/integration/durability/` — fault-injection crash-window specs
+
+**Schema changes land in two places.** A new index or column must be added to
+the install template *and* as a new upgrade migration, or existing
+installations never receive it. An audit that recommends a schema change
+without noting both is incomplete.
+
+## Key abstractions to prefer in fixes
+
+When recommending a fix, name the existing primitive rather than inventing a
+new one:
+
+- `Ductwork::DatabaseClock` — `.now`, `.ago_sql`, `.now_sql`. The canonical
+  answer to any cross-host time comparison.
+- `Ductwork::BranchClaim` — branch claiming CAS.
+- `Ductwork::RowLockingExecutionClaim` / `OptimisticLockingExecutionClaim` —
+  the adapter-split job claim strategies.
+- `Ductwork::FaultInjection.checkpoint(:name)` — named crash points, driven
+  by the `DUCTWORK_FAULT` env var.
+- `Ductwork::MigrationHelper` — adapter capability predicates for migrations.
+- `Ductwork::Step::ADVANCEABLE_STATUSES` — the shared advanceable predicate.

data/.claude/skills/audit-common/severity.md

@@ -0,0 +1,77 @@
+# Severity Rubric
+
+Ductwork is a job pipeline framework. Severity is graded by **what the user's
+pipeline does wrong**, not by how unusual the code looks. A framework bug
+silently corrupts every host application that hits it, so correctness
+outranks performance at every level.
+
+## Critical
+
+Silent incorrectness or permanent loss. The host application cannot detect it
+and cannot recover without manual intervention.
+
+- Work is lost: a step reports success but its result is never durably
+  written, or a branch is dropped and never advanced.
+- Double execution of a non-idempotent unit under a realistic interleaving,
+  where at-least-once is not the documented contract for that path.
+- A pipeline stalls forever with no reaper, timeout, or revive path that can
+  recover it.
+- Data corruption: a run reaches a terminal state that contradicts its
+  branches, or a completed run overwritten as halted (or the reverse).
+- A migration that can lose or corrupt existing rows.
+
+## High
+
+Recoverable but requires operator action, or a correctness bug that needs an
+uncommon-but-real interleaving.
+
+- Stuck work that the reaper recovers only after a global timeout, when a
+  targeted mechanism should have caught it promptly.
+- A crash window that leaks a claim, requiring a reap sweep to clear.
+- A race that needs specific timing to trigger but produces incorrect state
+  when it does.
+- Missing index on a hot claim path that degrades throughput non-linearly
+  with table size.
+- An adapter in the supported list where a code path raises or silently
+  misbehaves.
+
+## Medium
+
+Degraded behavior with a clear operational signal, or a latent bug that today
+is masked by an assumption that holds but is not enforced.
+
+- Correctness that depends on an invariant no constraint or guard enforces.
+- Error routing that sends a failure to the wrong recovery path, where the
+  outcome is still eventually correct but the retry accounting is wrong.
+- Missing index on a warm path.
+- Adapter support that works but relies on undocumented or version-specific
+  behavior.
+- A crash window that is real but leaks only a recoverable record with no
+  correctness consequence.
+
+## Low
+
+Correct today and correct under the interleavings that matter, but fragile to
+future change.
+
+- Duplicated logic where one copy could drift from another (notably: a
+  predicate expressed in two places that must stay in sync).
+- Missing test coverage for a durability window that is otherwise sound.
+- Naming or structure that invites a future contributor to introduce a real
+  bug.
+
+Do not file style, formatting, or preference items at any severity. Rubocop
+owns those.
+
+## Grading rules
+
+- **Grade by outcome, not by mechanism.** "No lock here" is not a finding.
+  "Two advancers both pass this check and both create an advancement, so the
+  step runs twice" is a finding.
+- **If you cannot describe the interleaving, it is not High or Critical.**
+  A finding whose "why it matters" is theoretical caps at Medium.
+- **A documented, accepted contract is not a bug.** At-least-once execution
+  under process death is Ductwork's documented contract. Do not file it as
+  Critical double-execution. See `accepted-tradeoffs.md`.
+- **Frequency does not raise severity; consequence does.** A rare path that
+  silently loses data outranks a common path that wastes a claim attempt.