rubydb 0.1.4 → 0.1.6

data/docs/getting-started/quickstart.md

@@ -1,17 +1,17 @@
-# Quick start
-
-Create a local database in one owning Ruby process:
-
-```ruby
-require "rubydb"
-
-engine = RubyDB::Storage::Engine.new("tmp/quickstart.rdb")
-engine.execute("CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT NOT NULL)")
-engine.execute("INSERT INTO users (id, name) VALUES (1, 'Aldane')")
-p engine.execute("SELECT * FROM users")
-engine.close
-```
-
-For a full walkthrough see [first database](first-database.md) and
-[first query](first-query.md). For multiple processes, use the server/client
-topology and do not share an embedded path.
+# Quick start
+
+Create a local database in one owning Ruby process:
+
+```ruby
+require "rubydb"
+
+engine = RubyDB::Storage::Engine.new("tmp/quickstart.rdb")
+engine.execute("CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT NOT NULL)")
+engine.execute("INSERT INTO users (id, name) VALUES (1, 'Aldane')")
+p engine.execute("SELECT * FROM users")
+engine.close
+```
+
+For a full walkthrough see [first database](first-database.md) and
+[first query](first-query.md). For multiple processes, use the server/client
+topology and do not share an embedded path.

data/docs/getting-started/rails.md

@@ -1,16 +1,16 @@
-# Rails quick start
-
-The runnable application under `examples/rails_app` demonstrates a real Rails
-migration, model, query, and server:
-
-```sh
-cd examples/rails_app
-bundle install
-bundle exec ruby bin/rails db:migrate
-bundle exec ruby bin/rails server -b 127.0.0.1 -p 3001
-```
-
-Use the ActiveRecord adapter only within its tested feature surface. For a
-production Rails deployment, use server mode for multiple processes, protect
-TLS/authentication secrets, back up before migrations, and run the exact Rails
-matrix and populated-schema checks described in `docs/rails/production.md`.
+# Rails quick start
+
+The runnable application under `examples/rails_app` demonstrates a real Rails
+migration, model, query, and server:
+
+```sh
+cd examples/rails_app
+bundle install
+bundle exec ruby bin/rails db:migrate
+bundle exec ruby bin/rails server -b 127.0.0.1 -p 3001
+```
+
+Use the ActiveRecord adapter only within its tested feature surface. For a
+production Rails deployment, use server mode for multiple processes, protect
+TLS/authentication secrets, back up before migrations, and run the exact Rails
+matrix and populated-schema checks described in `docs/rails/production.md`.

data/docs/hardening_backlog.md

@@ -1,93 +1,93 @@
-# Remaining production work
-
-Completed checkpoints: exclusive embedded engine ownership, duplicate-open
-rejection, release after process exit, initialization failure cleanup, fail-closed
-metadata and recovery startup, atomic metadata replacement, and deterministic
-background-maintenance shutdown.
-
-The following work remains open; passing the regression suite does not certify
-these capabilities.
-
-1. Server concurrency: cancellation and sustained load with latency percentiles.
-   Client deadlines are checked before execution and propagated into the query
-   executor, which checks during long read phases. Engine transaction state is
-   now scoped per client connection thread, and concurrent commit/rollback
-   behavior is covered. Multi-process durability is covered; wire-level
-   cancellation is now request-scoped and cooperative: a client can send
-   a cancel frame while the connection reader remains active, and the server
-   acknowledges it only for the active request. Mixed transaction reference-log
-   and restart validation remain open.
-2. Persistence: fault-test disk-full and interrupted checkpoints/schema changes;
-   index metadata load and write errors now fail visibly, and failed schema
-   publications roll back in-memory state. Metadata and index catalogs are
-   published through unique temporary files with flush/fsync/atomic rename.
-   Commit acknowledgements now expose durable versus uncertain WAL state and
-   recovery-required post-WAL flush failures. Storage accepts an explicit
-   `io_fault_injector` hook for page writes, file extension/truncation, and
-   sync operations; those paths now verify typed failures and descriptor
-   cleanup. Compaction now reads the engine's actual record-header layout,
-   observes dirty buffer-pool pages, and is covered through reopen validation.
-   Real filesystem quota and power-loss tests remain environment work.
-3. SQL correctness: ambiguous identifiers and broader dialect-specific edge
-   cases remain open. Boolean false values, `IS NULL`, NULL comparison behavior, and
-   outer-join NULL extension now have regression coverage. Non-recursive CTEs,
-   subqueries, set operations, targeted `ON CONFLICT DO UPDATE`, and
-   ranking/partition window functions, explicit `ROWS` window frames, bounded
-   recursive CTEs, dependency-aware inner-join reordering, and targetless
-   conflict updates using primary/unique definitions are implemented; multi-row
-   `VALUES` sources are now atomic when executed outside a caller transaction.
-   Broader dialect upsert forms and statistics-driven plan costing remain open.
-4. Replication: synchronized acknowledgements and promotion with an explicit
-   recovery point. Engine commits now package all committed row changes into a
-   single replication envelope after the local WAL commit point. Primary
-   connections bootstrap the catalog before row replay, including empty replicas. TCP input now
-   uses bounded newline framing and replay positions are fsynced before ack;
-   replica engine mutation entry points are read-only except for internal replay.
-   Promotion now rejects any received/replayed LSN gap and can require a
-   caller-supplied recovery point. Active primary engine mutations validate the
-   fencing lease before writing. Replication shutdown now closes established
-   replica sockets, and a reconnect/catch-up partition test verifies that the
-   replica resumes from the durable log. A process-level failover drill now
-   kills and replaces an independent primary, advances the fencing epoch from
-   another process, rejects the stale writer, and verifies replica catch-up.
-   A reproducible TCP fault-proxy drill now exercises live partition, healing,
-   catch-up, fencing, and promotion while the primary remains alive. True
-   multi-host partitions, independent fencing infrastructure, and split-brain
-   recovery still require deployment validation before adding automatic election.
-5. Rails: populated migration round trips, eager loading, nested associations,
-   and live adapter coverage are present. CI now exercises Rails 7.1, 7.2, and
-   8.0 against the adapter; connection-pool behavior and each version's
-   compatibility result still require hosted-run evidence. Migration
-   tracking now uses stable content checksums and fails closed for changed or
-   missing applied migrations. Native and ActiveRecord schema dumps now
-   round-trip automatic/custom primary-key modes, defaults, and indexes through
-   live engines.
-6. Operations: backup manifest writes now use durable atomic publication, live
-   engines flush WAL/storage before physical backup, and a scheduled restore
-   drill reopens restored files. Upgrade tests, measured resource limits,
-   alerting, and full security review remain open. Replication peers now
-   support constant-time shared-token authentication when configured; TLS and
-   credential rotation procedures still require deployment validation.
-
-7. Release engineering: cross-platform Ruby 3.3/3.4 CI, Rails 7.1/7.2/8.0
-   adapter jobs, and a deterministic bounded fuzz safety workflow are wired
-   into GitHub Actions. CI enforces 25% line and 20% branch coverage (the
-   current audit measured 62.0% line and 32.84% branch). Release provenance
-   signing is enabled, RubyGems gem-level signing requires protected
-   key/certificate paths, tag/changelog preflight fails closed, dependency
-   audit runs weekly, and tag releases can publish generated GitHub release
-   notes. A maintainer must still provision the RubyGems signing secrets and
-   review generated notes before publication.
-
-Deployment tests must record the commit, platform, workload and measured
-results. Keep untested features marked as unvalidated.
-
-## Latest documentation checkpoint
-
-The current repository also ships a tested common SQLite-style compatibility
-profile, a durability release gate, Prometheus alert rules, a multi-node Docker
-topology manifest, a production runbook, and a release checklist. The lessons
-from these hardening changes are collected in
-[`docs/lessons-learned.md`](lessons-learned.md). These artifacts improve
-repeatability but do not replace hosted multi-host, physical-filesystem, or
-independent security validation.
+# Remaining production work
+
+Completed checkpoints: exclusive embedded engine ownership, duplicate-open
+rejection, release after process exit, initialization failure cleanup, fail-closed
+metadata and recovery startup, atomic metadata replacement, and deterministic
+background-maintenance shutdown.
+
+The following work remains open; passing the regression suite does not certify
+these capabilities.
+
+1. Server concurrency: cancellation and sustained load with latency percentiles.
+   Client deadlines are checked before execution and propagated into the query
+   executor, which checks during long read phases. Engine transaction state is
+   now scoped per client connection thread, and concurrent commit/rollback
+   behavior is covered. Multi-process durability is covered; wire-level
+   cancellation is now request-scoped and cooperative: a client can send
+   a cancel frame while the connection reader remains active, and the server
+   acknowledges it only for the active request. Mixed transaction reference-log
+   and restart validation remain open.
+2. Persistence: fault-test disk-full and interrupted checkpoints/schema changes;
+   index metadata load and write errors now fail visibly, and failed schema
+   publications roll back in-memory state. Metadata and index catalogs are
+   published through unique temporary files with flush/fsync/atomic rename.
+   Commit acknowledgements now expose durable versus uncertain WAL state and
+   recovery-required post-WAL flush failures. Storage accepts an explicit
+   `io_fault_injector` hook for page writes, file extension/truncation, and
+   sync operations; those paths now verify typed failures and descriptor
+   cleanup. Compaction now reads the engine's actual record-header layout,
+   observes dirty buffer-pool pages, and is covered through reopen validation.
+   Real filesystem quota and power-loss tests remain environment work.
+3. SQL correctness: ambiguous identifiers and broader dialect-specific edge
+   cases remain open. Boolean false values, `IS NULL`, NULL comparison behavior, and
+   outer-join NULL extension now have regression coverage. Non-recursive CTEs,
+   subqueries, set operations, targeted `ON CONFLICT DO UPDATE`, and
+   ranking/partition window functions, explicit `ROWS` window frames, bounded
+   recursive CTEs, dependency-aware inner-join reordering, and targetless
+   conflict updates using primary/unique definitions are implemented; multi-row
+   `VALUES` sources are now atomic when executed outside a caller transaction.
+   Broader dialect upsert forms and statistics-driven plan costing remain open.
+4. Replication: synchronized acknowledgements and promotion with an explicit
+   recovery point. Engine commits now package all committed row changes into a
+   single replication envelope after the local WAL commit point. Primary
+   connections bootstrap the catalog before row replay, including empty replicas. TCP input now
+   uses bounded newline framing and replay positions are fsynced before ack;
+   replica engine mutation entry points are read-only except for internal replay.
+   Promotion now rejects any received/replayed LSN gap and can require a
+   caller-supplied recovery point. Active primary engine mutations validate the
+   fencing lease before writing. Replication shutdown now closes established
+   replica sockets, and a reconnect/catch-up partition test verifies that the
+   replica resumes from the durable log. A process-level failover drill now
+   kills and replaces an independent primary, advances the fencing epoch from
+   another process, rejects the stale writer, and verifies replica catch-up.
+   A reproducible TCP fault-proxy drill now exercises live partition, healing,
+   catch-up, fencing, and promotion while the primary remains alive. True
+   multi-host partitions, independent fencing infrastructure, and split-brain
+   recovery still require deployment validation before adding automatic election.
+5. Rails: populated migration round trips, eager loading, nested associations,
+   and live adapter coverage are present. CI now exercises Rails 7.1, 7.2, and
+   8.0 against the adapter; connection-pool behavior and each version's
+   compatibility result still require hosted-run evidence. Migration
+   tracking now uses stable content checksums and fails closed for changed or
+   missing applied migrations. Native and ActiveRecord schema dumps now
+   round-trip automatic/custom primary-key modes, defaults, and indexes through
+   live engines.
+6. Operations: backup manifest writes now use durable atomic publication, live
+   engines flush WAL/storage before physical backup, and a scheduled restore
+   drill reopens restored files. Upgrade tests, measured resource limits,
+   alerting, and full security review remain open. Replication peers now
+   support constant-time shared-token authentication when configured; TLS and
+   credential rotation procedures still require deployment validation.
+
+7. Release engineering: cross-platform Ruby 3.3/3.4 CI, Rails 7.1/7.2/8.0
+   adapter jobs, and a deterministic bounded fuzz safety workflow are wired
+   into GitHub Actions. CI enforces 25% line and 20% branch coverage (the
+   current audit measured 62.0% line and 32.84% branch). Release provenance
+   signing is enabled, RubyGems gem-level signing requires protected
+   key/certificate paths, tag/changelog preflight fails closed, dependency
+   audit runs weekly, and tag releases can publish generated GitHub release
+   notes. A maintainer must still provision the RubyGems signing secrets and
+   review generated notes before publication.
+
+Deployment tests must record the commit, platform, workload and measured
+results. Keep untested features marked as unvalidated.
+
+## Latest documentation checkpoint
+
+The current repository also ships a tested common SQLite-style compatibility
+profile, a durability release gate, Prometheus alert rules, a multi-node Docker
+topology manifest, a production runbook, and a release checklist. The lessons
+from these hardening changes are collected in
+[`docs/lessons-learned.md`](lessons-learned.md). These artifacts improve
+repeatability but do not replace hosted multi-host, physical-filesystem, or
+independent security validation.

data/docs/lessons-learned.md

@@ -1,112 +1,112 @@
-# RubyDB lessons learned
-
-This page records the engineering lessons from hardening RubyDB. Each lesson is
-paired with the rule it creates and the validation that protects it. New
-features should update this page when they change an operational guarantee.
-
-## Correctness before breadth
-
-An implemented parser branch, adapter method, or replication class is not a
-production feature until it is exercised end to end. A narrow, explicit
-compatibility contract is safer than accepting syntax with incomplete
-semantics.
-
-Rule: document supported behavior and fail closed for unsupported behavior.
-Validation: integration specs and the SQL compatibility documents must agree.
-
-## Durability is a sequence, not a boolean
-
-“Write succeeded” and “data is durable” are different outcomes. WAL ordering,
-flush errors, atomic metadata publication, checksums, recovery, and restore must
-be tested together. A successful in-memory mutation cannot hide a failed
-publication or sync.
-
-Rule: never report a durable acknowledgement before the required WAL and file
-operations succeed. Preserve uncertain state for recovery instead of guessing.
-Validation: filesystem fault injection, crash recovery, compaction/reopen, and
-backup/restore drills.
-
-## Ownership must be explicit
-
-An embedded file cannot safely be opened by multiple independent application
-processes merely because each process has a mutex. The operating-system lock and
-the server boundary are part of the data-safety model.
-
-Rule: one embedded owner; multiple processes use the server/client topology.
-Validation: duplicate-open rejection, process workloads, and connection-limit
-tests.
-
-## Replication needs fencing before election
-
-Replica catch-up alone does not prevent two writers. Promotion requires a
-known recovery point, durable fencing, stale-writer rejection, and operator
-evidence. Automatic election without an independent fencing authority can turn
-a network partition into split-brain writes.
-
-Rule: keep automatic election disabled until multi-host fencing is independently
-validated in the deployment environment.
-Validation: process replacement, TCP partition/healing, stale-primary rejection,
-and a real multi-host drill.
-
-## Cancellation must reach the wire and the executor
-
-A client-side timeout that only stops waiting does not stop database work. The
-server must keep reading control frames, mark the active request cancelled, and
-the executor must check cooperatively during long operations.
-
-Rule: test cancellation as an in-flight protocol event, not as a mocked timeout.
-Validation: the production soak sends a real cancel frame and checks the server
-request lifecycle.
-
-## Deadlock recovery belongs to transaction ownership
-
-Detecting a cycle is insufficient if the chosen victim is not rolled back and
-its locks are not released. The lock manager and transaction manager must share
-the same victim identity and lifecycle.
-
-Rule: every resolved deadlock has a victim, rollback, lock release, and a
-retryable error for the victim.
-Validation: the two-transaction cycle specs and the production soak.
-
-## Adapters are compatibility products
-
-ActiveRecord compatibility includes schema introspection, quoted identifiers,
-binds, associations, eager loading, migrations, schema dumps, connection pools,
-and error behavior. Passing one CRUD example is not Rails compatibility.
-
-Rule: test the adapter against every supported Rails version and a populated
-schema, not only an empty database.
-Validation: the Rails matrix and adapter integration suite for Rails 7.1, 7.2,
-and 8.0.
-
-## Operations are part of the implementation
-
-Backups, restore destinations, monitoring thresholds, certificate rotation,
-secret storage, upgrade rollback, incident preservation, and release evidence
-are not afterthoughts. Operators need safe commands and clear stop conditions.
-
-Rule: every destructive or irreversible operation must have a dry run, a backup
-boundary, and a documented recovery path.
-Validation: the production runbook, restore drill, alert rules, and release
-preflight.
-
-## Evidence must be reproducible
-
-Test counts without the commit, runtime, platform, workload, seed, and result
-artifact are difficult to trust or reproduce.
-
-Rule: archive machine-readable workload output and record commit, Ruby/Rails
-versions, OS, resources, seed, latency percentiles, recovery time, and RPO/RTO.
-Validation: CI artifacts, bounded fuzz seeds, benchmark output, and release
-checklists.
-
-## Security claims require a separate review
-
-Passing authentication and TLS tests does not review deployment identity,
-filesystem permissions, secret lifecycle, dependency exposure, or incident
-response.
-
-Rule: treat the repository security suite as a baseline and require an
-independent review before sensitive or regulated workloads.
-Validation: dependency audit, CodeQL, TLS/auth tests, deployment review, and
-credential/certificate rotation drills.
+# RubyDB lessons learned
+
+This page records the engineering lessons from hardening RubyDB. Each lesson is
+paired with the rule it creates and the validation that protects it. New
+features should update this page when they change an operational guarantee.
+
+## Correctness before breadth
+
+An implemented parser branch, adapter method, or replication class is not a
+production feature until it is exercised end to end. A narrow, explicit
+compatibility contract is safer than accepting syntax with incomplete
+semantics.
+
+Rule: document supported behavior and fail closed for unsupported behavior.
+Validation: integration specs and the SQL compatibility documents must agree.
+
+## Durability is a sequence, not a boolean
+
+“Write succeeded” and “data is durable” are different outcomes. WAL ordering,
+flush errors, atomic metadata publication, checksums, recovery, and restore must
+be tested together. A successful in-memory mutation cannot hide a failed
+publication or sync.
+
+Rule: never report a durable acknowledgement before the required WAL and file
+operations succeed. Preserve uncertain state for recovery instead of guessing.
+Validation: filesystem fault injection, crash recovery, compaction/reopen, and
+backup/restore drills.
+
+## Ownership must be explicit
+
+An embedded file cannot safely be opened by multiple independent application
+processes merely because each process has a mutex. The operating-system lock and
+the server boundary are part of the data-safety model.
+
+Rule: one embedded owner; multiple processes use the server/client topology.
+Validation: duplicate-open rejection, process workloads, and connection-limit
+tests.
+
+## Replication needs fencing before election
+
+Replica catch-up alone does not prevent two writers. Promotion requires a
+known recovery point, durable fencing, stale-writer rejection, and operator
+evidence. Automatic election without an independent fencing authority can turn
+a network partition into split-brain writes.
+
+Rule: keep automatic election disabled until multi-host fencing is independently
+validated in the deployment environment.
+Validation: process replacement, TCP partition/healing, stale-primary rejection,
+and a real multi-host drill.
+
+## Cancellation must reach the wire and the executor
+
+A client-side timeout that only stops waiting does not stop database work. The
+server must keep reading control frames, mark the active request cancelled, and
+the executor must check cooperatively during long operations.
+
+Rule: test cancellation as an in-flight protocol event, not as a mocked timeout.
+Validation: the production soak sends a real cancel frame and checks the server
+request lifecycle.
+
+## Deadlock recovery belongs to transaction ownership
+
+Detecting a cycle is insufficient if the chosen victim is not rolled back and
+its locks are not released. The lock manager and transaction manager must share
+the same victim identity and lifecycle.
+
+Rule: every resolved deadlock has a victim, rollback, lock release, and a
+retryable error for the victim.
+Validation: the two-transaction cycle specs and the production soak.
+
+## Adapters are compatibility products
+
+ActiveRecord compatibility includes schema introspection, quoted identifiers,
+binds, associations, eager loading, migrations, schema dumps, connection pools,
+and error behavior. Passing one CRUD example is not Rails compatibility.
+
+Rule: test the adapter against every supported Rails version and a populated
+schema, not only an empty database.
+Validation: the Rails matrix and adapter integration suite for Rails 7.1, 7.2,
+and 8.0.
+
+## Operations are part of the implementation
+
+Backups, restore destinations, monitoring thresholds, certificate rotation,
+secret storage, upgrade rollback, incident preservation, and release evidence
+are not afterthoughts. Operators need safe commands and clear stop conditions.
+
+Rule: every destructive or irreversible operation must have a dry run, a backup
+boundary, and a documented recovery path.
+Validation: the production runbook, restore drill, alert rules, and release
+preflight.
+
+## Evidence must be reproducible
+
+Test counts without the commit, runtime, platform, workload, seed, and result
+artifact are difficult to trust or reproduce.
+
+Rule: archive machine-readable workload output and record commit, Ruby/Rails
+versions, OS, resources, seed, latency percentiles, recovery time, and RPO/RTO.
+Validation: CI artifacts, bounded fuzz seeds, benchmark output, and release
+checklists.
+
+## Security claims require a separate review
+
+Passing authentication and TLS tests does not review deployment identity,
+filesystem permissions, secret lifecycle, dependency exposure, or incident
+response.
+
+Rule: treat the repository security suite as a baseline and require an
+independent review before sensitive or regulated workloads.
+Validation: dependency audit, CodeQL, TLS/auth tests, deployment review, and
+credential/certificate rotation drills.

data/docs/operations/backups.md

@@ -1,33 +1,33 @@
-# Backup policy
-
-Backups are only useful after they have been verified and restored. Keep the
-database file, WAL chain, backup manifest, and checksum together. Store a copy
-outside the database host and encrypt it with the organization’s approved
-archive tooling.
-
-## Create and verify
-
-Stop writes for a maintenance backup when possible, then run:
-
-```sh
-rubydb backup --dir /var/lib/rubydb/backups
-rubydb restore --latest --dry-run --dir /var/lib/rubydb/backups
-```
-
-The backup implementation flushes the WAL and storage before copying files and
-publishes its manifest atomically. A successful command is not a substitute for
-the scheduled restore drill in `scripts/restore_drill`.
-
-## Retention and deletion
-
-Use a documented retention schedule such as daily backups for 14 days, weekly
-backups for 12 weeks, and monthly backups for the required compliance period.
-Delete only verified, expired backup sets; never delete a base backup while an
-incremental or differential chain depends on it.
-
-## Recovery point checks
-
-Record the backup name, manifest checksum, database version, page size, WAL
-position, and verification result in the deployment record. Before restoring,
-confirm that the selected backup is complete and that its destination is a new,
-inactive directory.
+# Backup policy
+
+Backups are only useful after they have been verified and restored. Keep the
+database file, WAL chain, backup manifest, and checksum together. Store a copy
+outside the database host and encrypt it with the organization’s approved
+archive tooling.
+
+## Create and verify
+
+Stop writes for a maintenance backup when possible, then run:
+
+```sh
+rubydb backup --dir /var/lib/rubydb/backups
+rubydb restore --latest --dry-run --dir /var/lib/rubydb/backups
+```
+
+The backup implementation flushes the WAL and storage before copying files and
+publishes its manifest atomically. A successful command is not a substitute for
+the scheduled restore drill in `scripts/restore_drill`.
+
+## Retention and deletion
+
+Use a documented retention schedule such as daily backups for 14 days, weekly
+backups for 12 weeks, and monthly backups for the required compliance period.
+Delete only verified, expired backup sets; never delete a base backup while an
+incremental or differential chain depends on it.
+
+## Recovery point checks
+
+Record the backup name, manifest checksum, database version, page size, WAL
+position, and verification result in the deployment record. Before restoring,
+confirm that the selected backup is complete and that its destination is a new,
+inactive directory.

data/docs/operations/disaster-recovery.md

@@ -1,31 +1,31 @@
-# Disaster recovery
-
-Define application-specific recovery objectives before deployment:
-
-- RPO: the maximum acceptable committed data loss.
-- RTO: the maximum time to restore service.
-- retention: how long backups and incident copies must be kept.
-
-RubyDB’s verified recovery path is restore to a new directory followed by an
-engine reopen and row/catalog verification. It does not silently repair
-unknown corruption or perform automatic cross-region failover.
-
-## Recovery procedure
-
-1. Declare the incident and stop application writes to the affected database.
-2. Preserve the original data directory, WAL, logs, configuration, and process
-   information. Work from a copied incident set.
-3. Select the newest backup whose manifest and checksum verify within the RPO.
-4. Restore into a new directory and run `rubydb restore --dry-run` before
-   opening it.
-5. Open the restored database, run migrations/status checks, and execute the
-   application smoke and integrity queries.
-6. Switch the application to the restored directory only after validation.
-7. Preserve the failed directory until the incident review is complete.
-
-Run `ruby scripts/restore_drill` in staging on the same filesystem and RubyDB
-release family used in production. Record measured restore time, recovered WAL
-position, row counts, and any gaps against RTO/RPO.
-
-Do not remove WAL, run vacuum, force promotion, or overwrite the source during
-an active corruption investigation.
+# Disaster recovery
+
+Define application-specific recovery objectives before deployment:
+
+- RPO: the maximum acceptable committed data loss.
+- RTO: the maximum time to restore service.
+- retention: how long backups and incident copies must be kept.
+
+RubyDB’s verified recovery path is restore to a new directory followed by an
+engine reopen and row/catalog verification. It does not silently repair
+unknown corruption or perform automatic cross-region failover.
+
+## Recovery procedure
+
+1. Declare the incident and stop application writes to the affected database.
+2. Preserve the original data directory, WAL, logs, configuration, and process
+   information. Work from a copied incident set.
+3. Select the newest backup whose manifest and checksum verify within the RPO.
+4. Restore into a new directory and run `rubydb restore --dry-run` before
+   opening it.
+5. Open the restored database, run migrations/status checks, and execute the
+   application smoke and integrity queries.
+6. Switch the application to the restored directory only after validation.
+7. Preserve the failed directory until the incident review is complete.
+
+Run `ruby scripts/restore_drill` in staging on the same filesystem and RubyDB
+release family used in production. Record measured restore time, recovered WAL
+position, row counts, and any gaps against RTO/RPO.
+
+Do not remove WAL, run vacuum, force promotion, or overwrite the source during
+an active corruption investigation.