rubydb 0.1.3 → 0.1.5

data/docs/production_validation.md

@@ -1,150 +1,150 @@
-# Production validation checkpoint
-
-This checkpoint validates the embedded RubyDB engine and ActiveRecord adapter
-against a focused, repeatable set of production-relevant paths. Passing it is
-evidence for these paths; it is not a claim of universal SQL or Rails
-compatibility.
-
-## Validated paths
-
-- ActiveRecord 7.2 embedded CRUD, Arel bind compilation, qualified columns,
-  association-generated `INNER JOIN`, and `LEFT OUTER JOIN` SQL execution.
-- Reversible Rails migrations covering `create_table`, automatic integer `id`,
-  `add_column` with a default, unique `add_index`, and their `down` operations.
-- Repeated threaded insert workloads with row-count and close/reopen durability
-  verification.
-- Four independent network clients concurrently inserting through the live
-  server, with request metrics and post-restart durable-row verification.
-- Independent concurrent client transactions are isolated by connection:
-  committed work remains visible while a sibling transaction's work is rolled
-  back.
-- Two-engine logical replication of an insert followed by explicit, manual
-  promotion of the synchronized replica. Promotion retains the replicated row
-  and starts a fenced primary listener. Replicas reject local engine mutations
-  while allowing the internal logical replay path; explicit promotion restores
-  local writes. Replication TCP input is newline-frame buffered, rejects
-  oversized incomplete frames, bootstraps a new replica's catalog before row
-  replay, and persists the replay position before ack.
-- A stopped primary closes established replication sockets; a replica detects
-  the interruption, reconnects after the listener returns, and catches up from
-  the durable replication log. A caught-up disconnected replica may be
-  explicitly promoted, while lagging or never-synchronized replicas remain
-  ineligible.
-- The process-level failover drill runs an independent primary and replica,
-  kills and replaces the primary process, uses a separate process to advance
-  the fencing epoch, verifies the stale writer is rejected, and confirms a
-  fresh primary continues the log and the replica reaches both committed rows.
-- The network partition drill routes the live replication stream through a
-  fault-injecting TCP proxy, drops and heals the stream while the primary stays
-  running, verifies replica catch-up, fences the stale primary, and promotes a
-  synchronized replica.
-- Engine transaction integration: a committed transaction containing multiple
-  row mutations is emitted as one replication envelope only after its local
-  WAL commit and flush complete.
-- Persistence safety at the engine boundary: malformed metadata and failed WAL
-  recovery abort startup, metadata publishes are fsynced before atomic rename,
-  and the maintenance worker is joined before storage closes.
-- SQL window ranking (`ROW_NUMBER`, `RANK`, `DENSE_RANK`) and partition-wide
-  aggregate windows have focused regression coverage.
-- The SQLite compatibility profile covers common schema, CRUD, transaction,
-  grouped-join, aggregate, and targeted-upsert application paths.
-
-## Run before a release
-
-```powershell
-bundle exec rspec
-
-# Short CI-style repeatability check
-bundle exec rspec spec/concurrent_soak_harness_spec.rb
-
-# Deployment-sized threaded durability soak (adjust to the target hardware)
-$env:RUBYDB_SOAK_ROUNDS = "10"
-$env:RUBYDB_SOAK_THREADS = "16"
-$env:RUBYDB_SOAK_OPERATIONS = "10000"
-$env:RUBYDB_SOAK_PAYLOAD_BYTES = "512"
-ruby benchmarks/concurrent_soak.rb
-
-# Network server/client durability and latency smoke. Increase these values on
-# target hardware and archive the JSON p50/p95/p99 result with the release.
-$env:RUBYDB_SERVER_WORKLOAD_CLIENTS = "16"
-$env:RUBYDB_SERVER_WORKLOAD_OPERATIONS = "1000"
-ruby -Ilib benchmarks/server_workload.rb
-
-# Independent client processes through the server. This validates process
-# isolation and durable rows; scale processes/operations for the deployment.
-$env:RUBYDB_SERVER_WORKLOAD_PROCESSES = "8"
-$env:RUBYDB_SERVER_WORKLOAD_OPERATIONS = "1000"
-$env:RUBYDB_SERVER_WORKLOAD_CHILD_TIMEOUT = "120"
-ruby benchmarks/multiprocess_server_workload.rb
-
-# The parent supervises and reaps every child; a timed-out worker fails the run
-# instead of leaving orphaned workload processes behind.
-
-# Combined production concurrency/resilience gate: traffic latency, deadline,
-# real wire cancellation, connection capacity rejection, and deadlock detection
-$env:RUBYDB_PRODUCTION_SOAK_CLIENTS = "16"
-$env:RUBYDB_PRODUCTION_SOAK_OPERATIONS = "2000"
-$env:RUBYDB_PRODUCTION_SOAK_CANCEL_ROWS = "250000"
-ruby benchmarks/production_soak.rb
-
-# Independent primary/replica processes, crash replacement, and stale-writer fencing
-ruby scripts/replication_failover_drill
-
-# Live TCP partition, catch-up, fencing, and promotion
-ruby scripts/replication_network_failover_drill
-
-# Real two-engine replication and promotion validation
-bundle exec rspec spec/replication_failover_integration_spec.rb
-
-# SQLite-style application compatibility profile
-bundle exec rspec spec/sqlite_compatibility_spec.rb
-
-# Durability, crash, corruption, compaction, and restore release gate
-ruby scripts/durability_drill
-```
-
-The scheduled/manual GitHub Actions workflow `.github/workflows/workload.yml`
-also runs the threaded durability, server latency, and multi-process client
-workloads and uploads their JSON results as an artifact. Treat those results as
-environment-specific evidence, not a universal capacity guarantee.
-
-Archive the JSON output from the soak run with the Ruby version, RubyDB commit,
-host resources, and elapsed time. The harness creates a fresh temporary
-database for every round and fails if any round loses durable rows.
-
-## Current boundaries
-
-- Embedded databases now require exclusive ownership by one engine. A second
-  engine or process opening the same path receives an error. Multiple application
-  processes should connect through the server. The adjacent `.lock` file is
-  intentionally retained after close; the operating system releases ownership
-  on close or process exit. Never delete it while the database is open. This
-  requires a filesystem that implements file locking correctly. Hard-linked
-  database aliases and shared custom WAL/metadata paths are unsupported.
-
-- Failed metadata publication rolls back the in-memory schema and leaves the
-  durable catalog unchanged; callers receive an error and may retry the
-  mutation. Exercise disk-full and interrupted-rename fault injection on the
-  target filesystem before release.
-
-- Join support currently covers qualified `INNER`, `LEFT [OUTER]`, `RIGHT`, and
-  `FULL [OUTER] JOIN` with `ON` predicates. Join reordering is limited to safe
-  inner-join plans; correlated subqueries, advanced set-operation ordering,
-  and broader dialect-specific SQL still require dedicated compatibility tests.
-- The ActiveRecord migration test is intentionally scoped. Complex table
-  rebuilds, `change_column`, polymorphic references, generated columns, and
-  adapter-specific schema dumps require dedicated compatibility tests before
-  relying on them.
-- The soak harness and multi-process server workload cover distinct concurrency
-  paths, but they do not provide a universal capacity certification; perform
-  environment-specific load, crash, and operational recovery testing.
-- Failover is manual and requires an operator to confirm the replica is caught
-  up and that the old primary is fenced. Automatic leader election is not
-  enabled.
-- The replica fence covers engine schema, row, branch, vacuum, and compaction
-  mutation entry points. Empty-replica catalog bootstrap, transaction-integrated
-  replication, and reconnect catch-up are covered; true multi-host partition,
-  split-brain, and automated-election validation remain deployment work.
-
-RubyDB reports unsupported features as unsupported rather than advertising CTE
-or bulk-alter capability to ActiveRecord.
+# Production validation checkpoint
+
+This checkpoint validates the embedded RubyDB engine and ActiveRecord adapter
+against a focused, repeatable set of production-relevant paths. Passing it is
+evidence for these paths; it is not a claim of universal SQL or Rails
+compatibility.
+
+## Validated paths
+
+- ActiveRecord 7.2 embedded CRUD, Arel bind compilation, qualified columns,
+  association-generated `INNER JOIN`, and `LEFT OUTER JOIN` SQL execution.
+- Reversible Rails migrations covering `create_table`, automatic integer `id`,
+  `add_column` with a default, unique `add_index`, and their `down` operations.
+- Repeated threaded insert workloads with row-count and close/reopen durability
+  verification.
+- Four independent network clients concurrently inserting through the live
+  server, with request metrics and post-restart durable-row verification.
+- Independent concurrent client transactions are isolated by connection:
+  committed work remains visible while a sibling transaction's work is rolled
+  back.
+- Two-engine logical replication of an insert followed by explicit, manual
+  promotion of the synchronized replica. Promotion retains the replicated row
+  and starts a fenced primary listener. Replicas reject local engine mutations
+  while allowing the internal logical replay path; explicit promotion restores
+  local writes. Replication TCP input is newline-frame buffered, rejects
+  oversized incomplete frames, bootstraps a new replica's catalog before row
+  replay, and persists the replay position before ack.
+- A stopped primary closes established replication sockets; a replica detects
+  the interruption, reconnects after the listener returns, and catches up from
+  the durable replication log. A caught-up disconnected replica may be
+  explicitly promoted, while lagging or never-synchronized replicas remain
+  ineligible.
+- The process-level failover drill runs an independent primary and replica,
+  kills and replaces the primary process, uses a separate process to advance
+  the fencing epoch, verifies the stale writer is rejected, and confirms a
+  fresh primary continues the log and the replica reaches both committed rows.
+- The network partition drill routes the live replication stream through a
+  fault-injecting TCP proxy, drops and heals the stream while the primary stays
+  running, verifies replica catch-up, fences the stale primary, and promotes a
+  synchronized replica.
+- Engine transaction integration: a committed transaction containing multiple
+  row mutations is emitted as one replication envelope only after its local
+  WAL commit and flush complete.
+- Persistence safety at the engine boundary: malformed metadata and failed WAL
+  recovery abort startup, metadata publishes are fsynced before atomic rename,
+  and the maintenance worker is joined before storage closes.
+- SQL window ranking (`ROW_NUMBER`, `RANK`, `DENSE_RANK`) and partition-wide
+  aggregate windows have focused regression coverage.
+- The SQLite compatibility profile covers common schema, CRUD, transaction,
+  grouped-join, aggregate, and targeted-upsert application paths.
+
+## Run before a release
+
+```powershell
+bundle exec rspec
+
+# Short CI-style repeatability check
+bundle exec rspec spec/concurrent_soak_harness_spec.rb
+
+# Deployment-sized threaded durability soak (adjust to the target hardware)
+$env:RUBYDB_SOAK_ROUNDS = "10"
+$env:RUBYDB_SOAK_THREADS = "16"
+$env:RUBYDB_SOAK_OPERATIONS = "10000"
+$env:RUBYDB_SOAK_PAYLOAD_BYTES = "512"
+ruby benchmarks/concurrent_soak.rb
+
+# Network server/client durability and latency smoke. Increase these values on
+# target hardware and archive the JSON p50/p95/p99 result with the release.
+$env:RUBYDB_SERVER_WORKLOAD_CLIENTS = "16"
+$env:RUBYDB_SERVER_WORKLOAD_OPERATIONS = "1000"
+ruby -Ilib benchmarks/server_workload.rb
+
+# Independent client processes through the server. This validates process
+# isolation and durable rows; scale processes/operations for the deployment.
+$env:RUBYDB_SERVER_WORKLOAD_PROCESSES = "8"
+$env:RUBYDB_SERVER_WORKLOAD_OPERATIONS = "1000"
+$env:RUBYDB_SERVER_WORKLOAD_CHILD_TIMEOUT = "120"
+ruby benchmarks/multiprocess_server_workload.rb
+
+# The parent supervises and reaps every child; a timed-out worker fails the run
+# instead of leaving orphaned workload processes behind.
+
+# Combined production concurrency/resilience gate: traffic latency, deadline,
+# real wire cancellation, connection capacity rejection, and deadlock detection
+$env:RUBYDB_PRODUCTION_SOAK_CLIENTS = "16"
+$env:RUBYDB_PRODUCTION_SOAK_OPERATIONS = "2000"
+$env:RUBYDB_PRODUCTION_SOAK_CANCEL_ROWS = "250000"
+ruby benchmarks/production_soak.rb
+
+# Independent primary/replica processes, crash replacement, and stale-writer fencing
+ruby scripts/replication_failover_drill
+
+# Live TCP partition, catch-up, fencing, and promotion
+ruby scripts/replication_network_failover_drill
+
+# Real two-engine replication and promotion validation
+bundle exec rspec spec/replication_failover_integration_spec.rb
+
+# SQLite-style application compatibility profile
+bundle exec rspec spec/sqlite_compatibility_spec.rb
+
+# Durability, crash, corruption, compaction, and restore release gate
+ruby scripts/durability_drill
+```
+
+The scheduled/manual GitHub Actions workflow `.github/workflows/workload.yml`
+also runs the threaded durability, server latency, and multi-process client
+workloads and uploads their JSON results as an artifact. Treat those results as
+environment-specific evidence, not a universal capacity guarantee.
+
+Archive the JSON output from the soak run with the Ruby version, RubyDB commit,
+host resources, and elapsed time. The harness creates a fresh temporary
+database for every round and fails if any round loses durable rows.
+
+## Current boundaries
+
+- Embedded databases now require exclusive ownership by one engine. A second
+  engine or process opening the same path receives an error. Multiple application
+  processes should connect through the server. The adjacent `.lock` file is
+  intentionally retained after close; the operating system releases ownership
+  on close or process exit. Never delete it while the database is open. This
+  requires a filesystem that implements file locking correctly. Hard-linked
+  database aliases and shared custom WAL/metadata paths are unsupported.
+
+- Failed metadata publication rolls back the in-memory schema and leaves the
+  durable catalog unchanged; callers receive an error and may retry the
+  mutation. Exercise disk-full and interrupted-rename fault injection on the
+  target filesystem before release.
+
+- Join support currently covers qualified `INNER`, `LEFT [OUTER]`, `RIGHT`, and
+  `FULL [OUTER] JOIN` with `ON` predicates. Join reordering is limited to safe
+  inner-join plans; correlated subqueries, advanced set-operation ordering,
+  and broader dialect-specific SQL still require dedicated compatibility tests.
+- The ActiveRecord migration test is intentionally scoped. Complex table
+  rebuilds, `change_column`, polymorphic references, generated columns, and
+  adapter-specific schema dumps require dedicated compatibility tests before
+  relying on them.
+- The soak harness and multi-process server workload cover distinct concurrency
+  paths, but they do not provide a universal capacity certification; perform
+  environment-specific load, crash, and operational recovery testing.
+- Failover is manual and requires an operator to confirm the replica is caught
+  up and that the old primary is fenced. Automatic leader election is not
+  enabled.
+- The replica fence covers engine schema, row, branch, vacuum, and compaction
+  mutation entry points. Empty-replica catalog bootstrap, transaction-integrated
+  replication, and reconnect catch-up are covered; true multi-host partition,
+  split-brain, and automated-election validation remain deployment work.
+
+RubyDB reports unsupported features as unsupported rather than advertising CTE
+or bulk-alter capability to ActiveRecord.

data/docs/rails/active-record.md

@@ -1,11 +1,11 @@
-# ActiveRecord adapter
-
-The `rubydb-activerecord` adapter connects ActiveRecord models to RubyDB. The
-tested surface includes CRUD, quoted identifiers, binds, associations, joins,
-eager loading, nested associations, schema inspection, transactions, indexes,
-defaults, schema dumps, and populated-table migration paths.
-
-Run the adapter suite from `adapters/activerecord` for the target Rails version.
-Use server mode when multiple Rails processes share a database. The adapter is
-not a complete PostgreSQL, MySQL, or SQLite compatibility layer; validate any
-application-specific Arel, extension, callback, migration, or SQL behavior.
+# ActiveRecord adapter
+
+The `rubydb-activerecord` adapter connects ActiveRecord models to RubyDB. The
+tested surface includes CRUD, quoted identifiers, binds, associations, joins,
+eager loading, nested associations, schema inspection, transactions, indexes,
+defaults, schema dumps, and populated-table migration paths.
+
+Run the adapter suite from `adapters/activerecord` for the target Rails version.
+Use server mode when multiple Rails processes share a database. The adapter is
+not a complete PostgreSQL, MySQL, or SQLite compatibility layer; validate any
+application-specific Arel, extension, callback, migration, or SQL behavior.

data/docs/rails/compatibility-guide.md

@@ -1,90 +1,90 @@
-# Rails compatibility and integration guide
-
-RubyDB’s ActiveRecord adapter is intended for applications that stay within
-the documented RubyDB SQL and schema surface. It is not a claim of complete
-Rails compatibility or complete SQLite compatibility. Validate the exact
-Rails/Ruby versions, gems, queries, migrations, and deployment topology used
-by your application.
-
-## Choose the ownership mode
-
-Embedded mode is appropriate for a single-owner development process or a
-deliberately single-process application. Rails web servers, job workers,
-console sessions, and migration commands must not independently open the same
-embedded path.
-
-For multiple processes, start one RubyDB server and configure every Rails
-process to use the client/server adapter. Size the Rails connection pool below
-the server’s connection limit and leave headroom for deploys and health
-checks.
-
-## First application
-
-Use the maintained example as a smoke harness:
-
-```sh
-cd examples/rails_app
-bundle install
-bundle exec ruby bin/rails db:migrate
-bundle exec ruby bin/rails runner 'puts User.count'
-bundle exec ruby bin/rails server -b 127.0.0.1 -p 3001
-```
-
-Set `RUBYDB_DATABASE` to choose the database path in the embedded example.
-Before production, replace the local path with the managed server configuration
-and test the same migration and query flow through the network adapter.
-
-## Supported application patterns to validate
-
-Test the application’s real use of:
-
-* model creation, updates, deletes, validations, and transactions;
-* `where`, scopes, ordering, limits, offsets, projections, and bind values;
-* inner/left/multi-table joins, aliases, grouped aggregates, and `HAVING`;
-* eager loading, nested associations, and inverse association behavior;
-* connection pools, checkout timeouts, reconnects, and shutdown;
-* savepoints, rollback, retry, and deadlock handling;
-* schema dump/load, defaults, indexes, foreign keys, and constraints; and
-* fresh and populated-table migrations with rollback plans.
-
-Generated SQL is part of the compatibility surface. Capture representative
-queries and compare results, affected rows, exceptions, and transaction state.
-
-## Migration discipline
-
-Treat migrations as deployment code. Review generated SQL, run against a fresh
-database and a realistic populated copy, measure locks and duration, and
-define the rollback/backfill plan. Do not assume that a migration that creates
-a table on an empty database is safe on a large live table.
-
-Record the migration version and checksum. A changed migration should be
-treated as a new migration or an explicit controlled repair, not silently
-accepted. Take a verified backup before destructive schema changes.
-
-## Pooling and concurrency
-
-The pool must return connections with no open transaction, savepoint, lock, or
-pending cancellation. Test pool exhaustion and a server restart while Rails
-threads are active. Set bounded checkout and request timeouts. A client
-timeout does not automatically prove that a write rolled back; use an
-idempotency key or query the outcome before retrying.
-
-Run a multi-process workload with web-like reads, writes, jobs, migrations, and
-connection churn. Track p95/p99 latency, lock waits, deadlocks, cancellations,
-pool utilization, WAL growth, and error classes.
-
-## Version matrix
-
-The repository must run its Rails compatibility suite against every supported
-combination, for example Rails 7.1, 7.2, and 8.0 with the Ruby versions declared
-by the project. A local pass on one version is not evidence for the matrix.
-Record unsupported combinations explicitly in release documentation.
-
-## Debugging adapter failures
-
-Start with [Rails troubleshooting](troubleshooting.md) and
-[Debugging RubyDB](../debugging.md). Capture sanitized generated SQL, bind
-count/order, transaction boundaries, pool state, Ruby/Rails versions, and the
-smallest model/migration that reproduces the failure. Test direct SQL to
-separate Rails query generation from engine behavior.
-
+# Rails compatibility and integration guide
+
+RubyDB’s ActiveRecord adapter is intended for applications that stay within
+the documented RubyDB SQL and schema surface. It is not a claim of complete
+Rails compatibility or complete SQLite compatibility. Validate the exact
+Rails/Ruby versions, gems, queries, migrations, and deployment topology used
+by your application.
+
+## Choose the ownership mode
+
+Embedded mode is appropriate for a single-owner development process or a
+deliberately single-process application. Rails web servers, job workers,
+console sessions, and migration commands must not independently open the same
+embedded path.
+
+For multiple processes, start one RubyDB server and configure every Rails
+process to use the client/server adapter. Size the Rails connection pool below
+the server’s connection limit and leave headroom for deploys and health
+checks.
+
+## First application
+
+Use the maintained example as a smoke harness:
+
+```sh
+cd examples/rails_app
+bundle install
+bundle exec ruby bin/rails db:migrate
+bundle exec ruby bin/rails runner 'puts User.count'
+bundle exec ruby bin/rails server -b 127.0.0.1 -p 3001
+```
+
+Set `RUBYDB_DATABASE` to choose the database path in the embedded example.
+Before production, replace the local path with the managed server configuration
+and test the same migration and query flow through the network adapter.
+
+## Supported application patterns to validate
+
+Test the application’s real use of:
+
+* model creation, updates, deletes, validations, and transactions;
+* `where`, scopes, ordering, limits, offsets, projections, and bind values;
+* inner/left/multi-table joins, aliases, grouped aggregates, and `HAVING`;
+* eager loading, nested associations, and inverse association behavior;
+* connection pools, checkout timeouts, reconnects, and shutdown;
+* savepoints, rollback, retry, and deadlock handling;
+* schema dump/load, defaults, indexes, foreign keys, and constraints; and
+* fresh and populated-table migrations with rollback plans.
+
+Generated SQL is part of the compatibility surface. Capture representative
+queries and compare results, affected rows, exceptions, and transaction state.
+
+## Migration discipline
+
+Treat migrations as deployment code. Review generated SQL, run against a fresh
+database and a realistic populated copy, measure locks and duration, and
+define the rollback/backfill plan. Do not assume that a migration that creates
+a table on an empty database is safe on a large live table.
+
+Record the migration version and checksum. A changed migration should be
+treated as a new migration or an explicit controlled repair, not silently
+accepted. Take a verified backup before destructive schema changes.
+
+## Pooling and concurrency
+
+The pool must return connections with no open transaction, savepoint, lock, or
+pending cancellation. Test pool exhaustion and a server restart while Rails
+threads are active. Set bounded checkout and request timeouts. A client
+timeout does not automatically prove that a write rolled back; use an
+idempotency key or query the outcome before retrying.
+
+Run a multi-process workload with web-like reads, writes, jobs, migrations, and
+connection churn. Track p95/p99 latency, lock waits, deadlocks, cancellations,
+pool utilization, WAL growth, and error classes.
+
+## Version matrix
+
+The repository must run its Rails compatibility suite against every supported
+combination, for example Rails 7.1, 7.2, and 8.0 with the Ruby versions declared
+by the project. A local pass on one version is not evidence for the matrix.
+Record unsupported combinations explicitly in release documentation.
+
+## Debugging adapter failures
+
+Start with [Rails troubleshooting](troubleshooting.md) and
+[Debugging RubyDB](../debugging.md). Capture sanitized generated SQL, bind
+count/order, transaction boundaries, pool state, Ruby/Rails versions, and the
+smallest model/migration that reproduces the failure. Test direct SQL to
+separate Rails query generation from engine behavior.
+