rubydb 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/.gitattributes +20 -0
- data/.github/ISSUE_TEMPLATE/bug.yml +0 -0
- data/.github/ISSUE_TEMPLATE/feature.yml +0 -0
- data/.github/ISSUE_TEMPLATE/performance.yml +0 -0
- data/.github/ISSUE_TEMPLATE/security.yml +0 -0
- data/.github/PULL_REQUEST_TEMPLATE.md +15 -0
- data/.github/dependabot.yml +0 -0
- data/.github/workflows/benchmark.yml +26 -0
- data/.github/workflows/compatibility.yml +63 -0
- data/.github/workflows/fuzz.yml +33 -0
- data/.github/workflows/lint.yml +21 -0
- data/.github/workflows/operations.yml +24 -0
- data/.github/workflows/production-validation.yml +111 -0
- data/.github/workflows/release.yml +77 -0
- data/.github/workflows/security.yml +37 -0
- data/.github/workflows/test.yml +26 -0
- data/.github/workflows/workload.yml +58 -0
- data/.gitignore +43 -0
- data/.rspec +4 -0
- data/.rubocop.yml +44 -0
- data/.ruby-version +1 -0
- data/.standard.yml +14 -0
- data/ARCHITECTURE.md +21 -0
- data/CHANGELOG.md +31 -0
- data/CODE_OF_CONDUCT.md +13 -0
- data/CONTRIBUTING.md +29 -0
- data/GOVERNANCE.md +16 -0
- data/Gemfile +17 -0
- data/Gemfile.lock +71 -0
- data/LICENSE +60 -0
- data/README.md +162 -0
- data/ROADMAP.md +27 -0
- data/Rakefile +71 -0
- data/SECURITY.md +54 -0
- data/SUPPORT.md +14 -0
- data/adapters/activerecord/Gemfile +11 -0
- data/adapters/activerecord/README.md +9 -0
- data/adapters/activerecord/lib/active_record/connection_adapters/rubydb_adapter.rb +887 -0
- data/adapters/activerecord/rubydb-activerecord.gemspec +27 -0
- data/adapters/activerecord/spec/rubydb_adapter_integration_spec.rb +151 -0
- data/adapters/ruby/README.md +18 -0
- data/adapters/sequel/README.md +11 -0
- data/bin/rubydb +19 -0
- data/bin/rubydb-server +126 -0
- data/config/development.yml +0 -0
- data/config/monitoring/prometheus-alerts.yml +39 -0
- data/config/production.yml +36 -0
- data/config/test.yml +0 -0
- data/docs/README.md +71 -0
- data/docs/architecture/concurrency.md +14 -0
- data/docs/architecture/current-state.md +125 -0
- data/docs/architecture/execution-engine.md +25 -0
- data/docs/architecture/indexes.md +19 -0
- data/docs/architecture/mvcc.md +19 -0
- data/docs/architecture/overview.md +13 -0
- data/docs/architecture/pages.md +11 -0
- data/docs/architecture/production-roadmap.md +82 -0
- data/docs/architecture/query-planner.md +20 -0
- data/docs/architecture/recovery.md +18 -0
- data/docs/architecture/sql-engine.md +12 -0
- data/docs/architecture/storage-engine.md +14 -0
- data/docs/architecture/transactions.md +10 -0
- data/docs/architecture/wal.md +28 -0
- data/docs/cli-cheatsheet.md +98 -0
- data/docs/cli.md +275 -0
- data/docs/contributing/architecture.md +9 -0
- data/docs/contributing/benchmarking.md +14 -0
- data/docs/contributing/development.md +16 -0
- data/docs/contributing/release-process.md +49 -0
- data/docs/contributing/testing.md +16 -0
- data/docs/debugging.md +229 -0
- data/docs/developer/branching.md +10 -0
- data/docs/developer/database-diff.md +10 -0
- data/docs/developer/local-development.md +17 -0
- data/docs/developer/snapshots.md +9 -0
- data/docs/developer/temporal-data.md +10 -0
- data/docs/developer-guide.md +297 -0
- data/docs/getting-started/first-database.md +16 -0
- data/docs/getting-started/first-query.md +13 -0
- data/docs/getting-started/installation.md +19 -0
- data/docs/getting-started/local-to-production.md +300 -0
- data/docs/getting-started/quickstart.md +17 -0
- data/docs/getting-started/rails.md +16 -0
- data/docs/hardening_backlog.md +93 -0
- data/docs/lessons-learned.md +112 -0
- data/docs/operations/backups.md +33 -0
- data/docs/operations/disaster-recovery.md +31 -0
- data/docs/operations/failover.md +30 -0
- data/docs/operations/monitoring.md +25 -0
- data/docs/operations/production-guide.md +295 -0
- data/docs/operations/production-runbook.md +45 -0
- data/docs/operations/replication.md +33 -0
- data/docs/operations/restore.md +6 -0
- data/docs/operations/runbook.md +34 -0
- data/docs/operations/upgrades.md +14 -0
- data/docs/operations/workload-testing.md +17 -0
- data/docs/production-readiness.md +118 -0
- data/docs/production_validation.md +150 -0
- data/docs/rails/active-record.md +11 -0
- data/docs/rails/compatibility-guide.md +90 -0
- data/docs/rails/database-yml.md +92 -0
- data/docs/rails/installation.md +17 -0
- data/docs/rails/migrations.md +17 -0
- data/docs/rails/production.md +82 -0
- data/docs/rails/troubleshooting.md +18 -0
- data/docs/release.md +59 -0
- data/docs/server/architecture.md +10 -0
- data/docs/server/authentication.md +10 -0
- data/docs/server/configuration.md +16 -0
- data/docs/server/connection-pooling.md +10 -0
- data/docs/server/deployment.md +10 -0
- data/docs/server/protocol.md +12 -0
- data/docs/sql/compatibility-guide.md +82 -0
- data/docs/sql/compatibility.md +39 -0
- data/docs/sql/data-types.md +10 -0
- data/docs/sql/functions.md +9 -0
- data/docs/sql/joins.md +9 -0
- data/docs/sql/operators.md +9 -0
- data/docs/sql/sqlite-compatibility.md +21 -0
- data/docs/sql/syntax.md +10 -0
- data/docs/sql/transactions.md +10 -0
- data/docs/troubleshooting.md +244 -0
- data/exe/rubydb +23 -0
- data/lib/rubydb/backup/archive.rb +334 -0
- data/lib/rubydb/backup/backup.rb +401 -0
- data/lib/rubydb/backup/incremental.rb +353 -0
- data/lib/rubydb/backup/restore.rb +290 -0
- data/lib/rubydb/backup/snapshot.rb +267 -0
- data/lib/rubydb/backup/verification.rb +279 -0
- data/lib/rubydb/branching/branch.rb +181 -0
- data/lib/rubydb/branching/branch_manager.rb +311 -0
- data/lib/rubydb/branching/branch_metadata.rb +140 -0
- data/lib/rubydb/branching/checkout.rb +166 -0
- data/lib/rubydb/branching/copy_on_write.rb +272 -0
- data/lib/rubydb/branching/diff.rb +138 -0
- data/lib/rubydb/branching/merge.rb +285 -0
- data/lib/rubydb/build_info.rb +15 -0
- data/lib/rubydb/catalog/catalog.rb +391 -0
- data/lib/rubydb/catalog/column.rb +112 -0
- data/lib/rubydb/catalog/constraint.rb +180 -0
- data/lib/rubydb/catalog/database.rb +184 -0
- data/lib/rubydb/catalog/index.rb +97 -0
- data/lib/rubydb/catalog/schema.rb +103 -0
- data/lib/rubydb/catalog/sequence.rb +90 -0
- data/lib/rubydb/catalog/system_catalog.rb +698 -0
- data/lib/rubydb/catalog/table.rb +178 -0
- data/lib/rubydb/catalog/trigger.rb +102 -0
- data/lib/rubydb/catalog/view.rb +66 -0
- data/lib/rubydb/cli/application.rb +163 -0
- data/lib/rubydb/cli/commands/backup.rb +81 -0
- data/lib/rubydb/cli/commands/branch.rb +72 -0
- data/lib/rubydb/cli/commands/checkout.rb +54 -0
- data/lib/rubydb/cli/commands/create.rb +58 -0
- data/lib/rubydb/cli/commands/diff.rb +77 -0
- data/lib/rubydb/cli/commands/doctor.rb +74 -0
- data/lib/rubydb/cli/commands/drop.rb +57 -0
- data/lib/rubydb/cli/commands/init.rb +102 -0
- data/lib/rubydb/cli/commands/inspect.rb +95 -0
- data/lib/rubydb/cli/commands/merge.rb +63 -0
- data/lib/rubydb/cli/commands/migrate.rb +62 -0
- data/lib/rubydb/cli/commands/restart.rb +39 -0
- data/lib/rubydb/cli/commands/restore.rb +121 -0
- data/lib/rubydb/cli/commands/shell.rb +365 -0
- data/lib/rubydb/cli/commands/snapshot.rb +79 -0
- data/lib/rubydb/cli/commands/start.rb +82 -0
- data/lib/rubydb/cli/commands/status.rb +92 -0
- data/lib/rubydb/cli/commands/stop.rb +47 -0
- data/lib/rubydb/cli/commands/vacuum.rb +58 -0
- data/lib/rubydb/cli/formatter.rb +221 -0
- data/lib/rubydb/cli/output.rb +168 -0
- data/lib/rubydb/client/client.rb +304 -0
- data/lib/rubydb/client/connection.rb +415 -0
- data/lib/rubydb/client/connection_pool.rb +168 -0
- data/lib/rubydb/client/connection_url.rb +96 -0
- data/lib/rubydb/client/prepared_statement.rb +60 -0
- data/lib/rubydb/client/result.rb +123 -0
- data/lib/rubydb/client/statement.rb +52 -0
- data/lib/rubydb/client/transaction.rb +130 -0
- data/lib/rubydb/concurrency/concurrency.rb +19 -0
- data/lib/rubydb/concurrency/deadlock_detector.rb +150 -0
- data/lib/rubydb/concurrency/latch.rb +101 -0
- data/lib/rubydb/concurrency/lock_graph.rb +165 -0
- data/lib/rubydb/concurrency/mutex.rb +183 -0
- data/lib/rubydb/concurrency/rw_lock.rb +180 -0
- data/lib/rubydb/concurrency/scheduler.rb +250 -0
- data/lib/rubydb/concurrency/worker_pool.rb +143 -0
- data/lib/rubydb/configuration/config.rb +170 -0
- data/lib/rubydb/configuration/defaults.rb +179 -0
- data/lib/rubydb/configuration/environment.rb +152 -0
- data/lib/rubydb/configuration/parser.rb +185 -0
- data/lib/rubydb/configuration/validation.rb +221 -0
- data/lib/rubydb/constants.rb +74 -0
- data/lib/rubydb/constraints/check.rb +181 -0
- data/lib/rubydb/constraints/constraint.rb +101 -0
- data/lib/rubydb/constraints/foreign_key.rb +130 -0
- data/lib/rubydb/constraints/not_null.rb +64 -0
- data/lib/rubydb/constraints/primary_key.rb +99 -0
- data/lib/rubydb/constraints/unique.rb +108 -0
- data/lib/rubydb/constraints/validator.rb +350 -0
- data/lib/rubydb/errors/authentication_error.rb +10 -0
- data/lib/rubydb/errors/authorization_error.rb +23 -0
- data/lib/rubydb/errors/client_error.rb +10 -0
- data/lib/rubydb/errors/configuration_error.rb +10 -0
- data/lib/rubydb/errors/connection_error.rb +10 -0
- data/lib/rubydb/errors/constraint_error.rb +23 -0
- data/lib/rubydb/errors/corruption_error.rb +10 -0
- data/lib/rubydb/errors/database_error.rb +10 -0
- data/lib/rubydb/errors/error.rb +20 -0
- data/lib/rubydb/errors/execution_error.rb +10 -0
- data/lib/rubydb/errors/parser_error.rb +10 -0
- data/lib/rubydb/errors/recovery_error.rb +10 -0
- data/lib/rubydb/errors/replication_error.rb +10 -0
- data/lib/rubydb/errors/server_error.rb +6 -0
- data/lib/rubydb/errors/storage_error.rb +10 -0
- data/lib/rubydb/errors/transaction_error.rb +10 -0
- data/lib/rubydb/execution/aggregate_executor.rb +138 -0
- data/lib/rubydb/execution/delete_executor.rb +112 -0
- data/lib/rubydb/execution/distinct_executor.rb +135 -0
- data/lib/rubydb/execution/executor.rb +1192 -0
- data/lib/rubydb/execution/expression.rb +193 -0
- data/lib/rubydb/execution/index_scan.rb +142 -0
- data/lib/rubydb/execution/insert_executor.rb +217 -0
- data/lib/rubydb/execution/join_executor.rb +249 -0
- data/lib/rubydb/execution/limit_executor.rb +85 -0
- data/lib/rubydb/execution/optimizer.rb +215 -0
- data/lib/rubydb/execution/plan.rb +361 -0
- data/lib/rubydb/execution/planner.rb +546 -0
- data/lib/rubydb/execution/predicate.rb +235 -0
- data/lib/rubydb/execution/scan.rb +49 -0
- data/lib/rubydb/execution/sequential_scan.rb +63 -0
- data/lib/rubydb/execution/sort_executor.rb +185 -0
- data/lib/rubydb/execution/update_executor.rb +162 -0
- data/lib/rubydb/functions/aggregate.rb +70 -0
- data/lib/rubydb/functions/date_functions.rb +278 -0
- data/lib/rubydb/functions/function.rb +85 -0
- data/lib/rubydb/functions/json_functions.rb +215 -0
- data/lib/rubydb/functions/numeric_functions.rb +346 -0
- data/lib/rubydb/functions/scalar.rb +52 -0
- data/lib/rubydb/functions/string_functions.rb +383 -0
- data/lib/rubydb/functions/system_functions.rb +246 -0
- data/lib/rubydb/history/as_of.rb +238 -0
- data/lib/rubydb/history/change.rb +105 -0
- data/lib/rubydb/history/history.rb +131 -0
- data/lib/rubydb/history/history_manager.rb +229 -0
- data/lib/rubydb/history/temporal_query.rb +202 -0
- data/lib/rubydb/history/timeline.rb +144 -0
- data/lib/rubydb/indexes/btree.rb +186 -0
- data/lib/rubydb/indexes/btree_cursor.rb +258 -0
- data/lib/rubydb/indexes/btree_node.rb +385 -0
- data/lib/rubydb/indexes/hash_index.rb +150 -0
- data/lib/rubydb/indexes/index.rb +71 -0
- data/lib/rubydb/indexes/index_manager.rb +406 -0
- data/lib/rubydb/indexes/index_scan.rb +470 -0
- data/lib/rubydb/migrations/migration.rb +254 -0
- data/lib/rubydb/migrations/migration_lock.rb +146 -0
- data/lib/rubydb/migrations/migration_manager.rb +176 -0
- data/lib/rubydb/migrations/migration_version.rb +71 -0
- data/lib/rubydb/migrations/schema_diff.rb +211 -0
- data/lib/rubydb/migrations/schema_version.rb +64 -0
- data/lib/rubydb/monitoring/events.rb +160 -0
- data/lib/rubydb/monitoring/health.rb +222 -0
- data/lib/rubydb/monitoring/logger.rb +193 -0
- data/lib/rubydb/monitoring/metrics.rb +359 -0
- data/lib/rubydb/monitoring/performance.rb +176 -0
- data/lib/rubydb/monitoring/statistics.rb +170 -0
- data/lib/rubydb/mvcc/garbage_collector.rb +199 -0
- data/lib/rubydb/mvcc/mvcc.rb +16 -0
- data/lib/rubydb/mvcc/snapshot.rb +147 -0
- data/lib/rubydb/mvcc/vacuum.rb +180 -0
- data/lib/rubydb/mvcc/version.rb +106 -0
- data/lib/rubydb/mvcc/version_store.rb +398 -0
- data/lib/rubydb/mvcc/visibility.rb +109 -0
- data/lib/rubydb/protocol/capabilities.rb +125 -0
- data/lib/rubydb/protocol/decoder.rb +145 -0
- data/lib/rubydb/protocol/encoder.rb +136 -0
- data/lib/rubydb/protocol/handshake.rb +305 -0
- data/lib/rubydb/protocol/message.rb +121 -0
- data/lib/rubydb/protocol/protocol.rb +277 -0
- data/lib/rubydb/protocol/version.rb +54 -0
- data/lib/rubydb/rails/adapter.rb +239 -0
- data/lib/rubydb/rails/connection.rb +314 -0
- data/lib/rubydb/rails/database_statements.rb +122 -0
- data/lib/rubydb/rails/migration.rb +131 -0
- data/lib/rubydb/rails/quoting.rb +109 -0
- data/lib/rubydb/rails/result.rb +120 -0
- data/lib/rubydb/rails/schema_statements.rb +339 -0
- data/lib/rubydb/rails/transaction.rb +105 -0
- data/lib/rubydb/rails/type.rb +126 -0
- data/lib/rubydb/recovery/checkpoint.rb +257 -0
- data/lib/rubydb/recovery/consistency.rb +467 -0
- data/lib/rubydb/recovery/corruption_detector.rb +5 -0
- data/lib/rubydb/recovery/crash_recovery.rb +387 -0
- data/lib/rubydb/recovery/recovery_manager.rb +206 -0
- data/lib/rubydb/recovery/redo.rb +237 -0
- data/lib/rubydb/recovery/undo.rb +206 -0
- data/lib/rubydb/replication/failover.rb +5 -0
- data/lib/rubydb/replication/fencing.rb +63 -0
- data/lib/rubydb/replication/primary.rb +450 -0
- data/lib/rubydb/replication/replica.rb +384 -0
- data/lib/rubydb/replication/replication_log.rb +200 -0
- data/lib/rubydb/replication/replication_manager.rb +308 -0
- data/lib/rubydb/replication/replication_slot.rb +295 -0
- data/lib/rubydb/replication/replication_stream.rb +201 -0
- data/lib/rubydb/rubydb.rb +560 -0
- data/lib/rubydb/security/access_control.rb +254 -0
- data/lib/rubydb/security/audit_log.rb +213 -0
- data/lib/rubydb/security/authentication.rb +302 -0
- data/lib/rubydb/security/authorization.rb +282 -0
- data/lib/rubydb/security/credentials.rb +196 -0
- data/lib/rubydb/security/password.rb +215 -0
- data/lib/rubydb/security/permissions.rb +74 -0
- data/lib/rubydb/security/role.rb +101 -0
- data/lib/rubydb/security/user.rb +86 -0
- data/lib/rubydb/server/connection.rb +366 -0
- data/lib/rubydb/server/connection_pool.rb +193 -0
- data/lib/rubydb/server/lifecycle.rb +228 -0
- data/lib/rubydb/server/listener.rb +136 -0
- data/lib/rubydb/server/request_handler.rb +276 -0
- data/lib/rubydb/server/server.rb +364 -0
- data/lib/rubydb/server/session.rb +369 -0
- data/lib/rubydb/server/worker.rb +210 -0
- data/lib/rubydb/server/worker_pool.rb +168 -0
- data/lib/rubydb/sql/ast/alter_table.rb +169 -0
- data/lib/rubydb/sql/ast/begin_transaction.rb +47 -0
- data/lib/rubydb/sql/ast/commit.rb +37 -0
- data/lib/rubydb/sql/ast/constraint.rb +83 -0
- data/lib/rubydb/sql/ast/create_database.rb +41 -0
- data/lib/rubydb/sql/ast/create_index.rb +61 -0
- data/lib/rubydb/sql/ast/create_schema.rb +52 -0
- data/lib/rubydb/sql/ast/create_table.rb +187 -0
- data/lib/rubydb/sql/ast/delete.rb +54 -0
- data/lib/rubydb/sql/ast/drop_database.rb +41 -0
- data/lib/rubydb/sql/ast/drop_index.rb +41 -0
- data/lib/rubydb/sql/ast/drop_schema.rb +49 -0
- data/lib/rubydb/sql/ast/drop_table.rb +49 -0
- data/lib/rubydb/sql/ast/explain.rb +64 -0
- data/lib/rubydb/sql/ast/expression.rb +604 -0
- data/lib/rubydb/sql/ast/insert.rb +82 -0
- data/lib/rubydb/sql/ast/node.rb +42 -0
- data/lib/rubydb/sql/ast/rollback.rb +63 -0
- data/lib/rubydb/sql/ast/savepoint.rb +59 -0
- data/lib/rubydb/sql/ast/select.rb +88 -0
- data/lib/rubydb/sql/ast/set_operation.rb +20 -0
- data/lib/rubydb/sql/ast/trigger.rb +29 -0
- data/lib/rubydb/sql/ast/update.rb +88 -0
- data/lib/rubydb/sql/ast/vacuum.rb +19 -0
- data/lib/rubydb/sql/ast/view.rb +32 -0
- data/lib/rubydb/sql/ast/with.rb +32 -0
- data/lib/rubydb/sql/grammar.rb +86 -0
- data/lib/rubydb/sql/keywords.rb +156 -0
- data/lib/rubydb/sql/lexer.rb +214 -0
- data/lib/rubydb/sql/operators.rb +100 -0
- data/lib/rubydb/sql/parser.rb +1185 -0
- data/lib/rubydb/sql/planner/analyzer.rb +302 -0
- data/lib/rubydb/sql/planner/binder.rb +550 -0
- data/lib/rubydb/sql/planner/type_checker.rb +431 -0
- data/lib/rubydb/sql/token.rb +210 -0
- data/lib/rubydb/storage/buffer_frame.rb +44 -0
- data/lib/rubydb/storage/buffer_pool.rb +155 -0
- data/lib/rubydb/storage/database_lock.rb +74 -0
- data/lib/rubydb/storage/deserializer.rb +342 -0
- data/lib/rubydb/storage/engine.rb +2338 -0
- data/lib/rubydb/storage/file_manager.rb +187 -0
- data/lib/rubydb/storage/free_space_map.rb +81 -0
- data/lib/rubydb/storage/page.rb +94 -0
- data/lib/rubydb/storage/page_allocator.rb +855 -0
- data/lib/rubydb/storage/page_header.rb +67 -0
- data/lib/rubydb/storage/page_manager.rb +131 -0
- data/lib/rubydb/storage/record.rb +58 -0
- data/lib/rubydb/storage/row.rb +78 -0
- data/lib/rubydb/storage/serializer.rb +51 -0
- data/lib/rubydb/storage/storage_layout.rb +151 -0
- data/lib/rubydb/storage/storage_manager.rb +114 -0
- data/lib/rubydb/storage/tuple.rb +461 -0
- data/lib/rubydb/storage/visibility_map.rb +984 -0
- data/lib/rubydb/transactions/commit_manager.rb +220 -0
- data/lib/rubydb/transactions/isolation.rb +98 -0
- data/lib/rubydb/transactions/lock.rb +76 -0
- data/lib/rubydb/transactions/lock_manager.rb +362 -0
- data/lib/rubydb/transactions/savepoint.rb +143 -0
- data/lib/rubydb/transactions/transaction.rb +215 -0
- data/lib/rubydb/transactions/transaction_id.rb +84 -0
- data/lib/rubydb/transactions/transaction_log.rb +257 -0
- data/lib/rubydb/transactions/transaction_manager.rb +435 -0
- data/lib/rubydb/types/bigint.rb +36 -0
- data/lib/rubydb/types/blob.rb +37 -0
- data/lib/rubydb/types/boolean.rb +34 -0
- data/lib/rubydb/types/date.rb +39 -0
- data/lib/rubydb/types/decimal.rb +48 -0
- data/lib/rubydb/types/float.rb +34 -0
- data/lib/rubydb/types/integer.rb +36 -0
- data/lib/rubydb/types/json.rb +41 -0
- data/lib/rubydb/types/null.rb +34 -0
- data/lib/rubydb/types/smallint.rb +36 -0
- data/lib/rubydb/types/text.rb +37 -0
- data/lib/rubydb/types/time.rb +46 -0
- data/lib/rubydb/types/timestamp.rb +39 -0
- data/lib/rubydb/types/type.rb +119 -0
- data/lib/rubydb/types/uuid.rb +47 -0
- data/lib/rubydb/types/varchar.rb +37 -0
- data/lib/rubydb/version.rb +33 -0
- data/lib/rubydb/wal/archive.rb +193 -0
- data/lib/rubydb/wal/checkpoint.rb +183 -0
- data/lib/rubydb/wal/lsn.rb +94 -0
- data/lib/rubydb/wal/reader.rb +260 -0
- data/lib/rubydb/wal/record.rb +105 -0
- data/lib/rubydb/wal/segment.rb +193 -0
- data/lib/rubydb/wal/wal.rb +452 -0
- data/lib/rubydb/wal/writer.rb +236 -0
- data/lib/rubydb.rb +7 -0
- data/packaging/docker/Dockerfile +36 -0
- data/packaging/docker/docker-compose.failover.yml +43 -0
- data/packaging/docker/entrypoint.sh +15 -0
- data/packaging/homebrew/rubydb.rb +19 -0
- data/packaging/systemd/rubydb.service +22 -0
- data/rubydb.gemspec +57 -0
- data/scripts/benchmark +7 -0
- data/scripts/durability_drill +37 -0
- data/scripts/format +0 -0
- data/scripts/fuzz +63 -0
- data/scripts/lint +0 -0
- data/scripts/release +50 -0
- data/scripts/release_check +43 -0
- data/scripts/replication_failover_drill +250 -0
- data/scripts/replication_network_failover_drill +255 -0
- data/scripts/restore_drill +45 -0
- data/scripts/security +0 -0
- data/scripts/setup +0 -0
- data/scripts/test +0 -0
- data/scripts/test-chaos +0 -0
- data/scripts/test-crash +0 -0
- data/scripts/test-integration +0 -0
- data/scripts/test-unit +0 -0
- data/visibility_map.json +1 -0
- metadata +538 -0
|
@@ -0,0 +1,150 @@
|
|
|
1
|
+
# Production validation checkpoint
|
|
2
|
+
|
|
3
|
+
This checkpoint validates the embedded RubyDB engine and ActiveRecord adapter
|
|
4
|
+
against a focused, repeatable set of production-relevant paths. Passing it is
|
|
5
|
+
evidence for these paths; it is not a claim of universal SQL or Rails
|
|
6
|
+
compatibility.
|
|
7
|
+
|
|
8
|
+
## Validated paths
|
|
9
|
+
|
|
10
|
+
- ActiveRecord 7.2 embedded CRUD, Arel bind compilation, qualified columns,
|
|
11
|
+
association-generated `INNER JOIN`, and `LEFT OUTER JOIN` SQL execution.
|
|
12
|
+
- Reversible Rails migrations covering `create_table`, automatic integer `id`,
|
|
13
|
+
`add_column` with a default, unique `add_index`, and their `down` operations.
|
|
14
|
+
- Repeated threaded insert workloads with row-count and close/reopen durability
|
|
15
|
+
verification.
|
|
16
|
+
- Four independent network clients concurrently inserting through the live
|
|
17
|
+
server, with request metrics and post-restart durable-row verification.
|
|
18
|
+
- Independent concurrent client transactions are isolated by connection:
|
|
19
|
+
committed work remains visible while a sibling transaction's work is rolled
|
|
20
|
+
back.
|
|
21
|
+
- Two-engine logical replication of an insert followed by explicit, manual
|
|
22
|
+
promotion of the synchronized replica. Promotion retains the replicated row
|
|
23
|
+
and starts a fenced primary listener. Replicas reject local engine mutations
|
|
24
|
+
while allowing the internal logical replay path; explicit promotion restores
|
|
25
|
+
local writes. Replication TCP input is newline-frame buffered, rejects
|
|
26
|
+
oversized incomplete frames, bootstraps a new replica's catalog before row
|
|
27
|
+
replay, and persists the replay position before ack.
|
|
28
|
+
- A stopped primary closes established replication sockets; a replica detects
|
|
29
|
+
the interruption, reconnects after the listener returns, and catches up from
|
|
30
|
+
the durable replication log. A caught-up disconnected replica may be
|
|
31
|
+
explicitly promoted, while lagging or never-synchronized replicas remain
|
|
32
|
+
ineligible.
|
|
33
|
+
- The process-level failover drill runs an independent primary and replica,
|
|
34
|
+
kills and replaces the primary process, uses a separate process to advance
|
|
35
|
+
the fencing epoch, verifies the stale writer is rejected, and confirms a
|
|
36
|
+
fresh primary continues the log and the replica reaches both committed rows.
|
|
37
|
+
- The network partition drill routes the live replication stream through a
|
|
38
|
+
fault-injecting TCP proxy, drops and heals the stream while the primary stays
|
|
39
|
+
running, verifies replica catch-up, fences the stale primary, and promotes a
|
|
40
|
+
synchronized replica.
|
|
41
|
+
- Engine transaction integration: a committed transaction containing multiple
|
|
42
|
+
row mutations is emitted as one replication envelope only after its local
|
|
43
|
+
WAL commit and flush complete.
|
|
44
|
+
- Persistence safety at the engine boundary: malformed metadata and failed WAL
|
|
45
|
+
recovery abort startup, metadata publishes are fsynced before atomic rename,
|
|
46
|
+
and the maintenance worker is joined before storage closes.
|
|
47
|
+
- SQL window ranking (`ROW_NUMBER`, `RANK`, `DENSE_RANK`) and partition-wide
|
|
48
|
+
aggregate windows have focused regression coverage.
|
|
49
|
+
- The SQLite compatibility profile covers common schema, CRUD, transaction,
|
|
50
|
+
grouped-join, aggregate, and targeted-upsert application paths.
|
|
51
|
+
|
|
52
|
+
## Run before a release
|
|
53
|
+
|
|
54
|
+
```powershell
|
|
55
|
+
bundle exec rspec
|
|
56
|
+
|
|
57
|
+
# Short CI-style repeatability check
|
|
58
|
+
bundle exec rspec spec/concurrent_soak_harness_spec.rb
|
|
59
|
+
|
|
60
|
+
# Deployment-sized threaded durability soak (adjust to the target hardware)
|
|
61
|
+
$env:RUBYDB_SOAK_ROUNDS = "10"
|
|
62
|
+
$env:RUBYDB_SOAK_THREADS = "16"
|
|
63
|
+
$env:RUBYDB_SOAK_OPERATIONS = "10000"
|
|
64
|
+
$env:RUBYDB_SOAK_PAYLOAD_BYTES = "512"
|
|
65
|
+
ruby benchmarks/concurrent_soak.rb
|
|
66
|
+
|
|
67
|
+
# Network server/client durability and latency smoke. Increase these values on
|
|
68
|
+
# target hardware and archive the JSON p50/p95/p99 result with the release.
|
|
69
|
+
$env:RUBYDB_SERVER_WORKLOAD_CLIENTS = "16"
|
|
70
|
+
$env:RUBYDB_SERVER_WORKLOAD_OPERATIONS = "1000"
|
|
71
|
+
ruby -Ilib benchmarks/server_workload.rb
|
|
72
|
+
|
|
73
|
+
# Independent client processes through the server. This validates process
|
|
74
|
+
# isolation and durable rows; scale processes/operations for the deployment.
|
|
75
|
+
$env:RUBYDB_SERVER_WORKLOAD_PROCESSES = "8"
|
|
76
|
+
$env:RUBYDB_SERVER_WORKLOAD_OPERATIONS = "1000"
|
|
77
|
+
$env:RUBYDB_SERVER_WORKLOAD_CHILD_TIMEOUT = "120"
|
|
78
|
+
ruby benchmarks/multiprocess_server_workload.rb
|
|
79
|
+
|
|
80
|
+
# The parent supervises and reaps every child; a timed-out worker fails the run
|
|
81
|
+
# instead of leaving orphaned workload processes behind.
|
|
82
|
+
|
|
83
|
+
# Combined production concurrency/resilience gate: traffic latency, deadline,
|
|
84
|
+
# real wire cancellation, connection capacity rejection, and deadlock detection
|
|
85
|
+
$env:RUBYDB_PRODUCTION_SOAK_CLIENTS = "16"
|
|
86
|
+
$env:RUBYDB_PRODUCTION_SOAK_OPERATIONS = "2000"
|
|
87
|
+
$env:RUBYDB_PRODUCTION_SOAK_CANCEL_ROWS = "250000"
|
|
88
|
+
ruby benchmarks/production_soak.rb
|
|
89
|
+
|
|
90
|
+
# Independent primary/replica processes, crash replacement, and stale-writer fencing
|
|
91
|
+
ruby scripts/replication_failover_drill
|
|
92
|
+
|
|
93
|
+
# Live TCP partition, catch-up, fencing, and promotion
|
|
94
|
+
ruby scripts/replication_network_failover_drill
|
|
95
|
+
|
|
96
|
+
# Real two-engine replication and promotion validation
|
|
97
|
+
bundle exec rspec spec/replication_failover_integration_spec.rb
|
|
98
|
+
|
|
99
|
+
# SQLite-style application compatibility profile
|
|
100
|
+
bundle exec rspec spec/sqlite_compatibility_spec.rb
|
|
101
|
+
|
|
102
|
+
# Durability, crash, corruption, compaction, and restore release gate
|
|
103
|
+
ruby scripts/durability_drill
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
The scheduled/manual GitHub Actions workflow `.github/workflows/workload.yml`
|
|
107
|
+
also runs the threaded durability, server latency, and multi-process client
|
|
108
|
+
workloads and uploads their JSON results as an artifact. Treat those results as
|
|
109
|
+
environment-specific evidence, not a universal capacity guarantee.
|
|
110
|
+
|
|
111
|
+
Archive the JSON output from the soak run with the Ruby version, RubyDB commit,
|
|
112
|
+
host resources, and elapsed time. The harness creates a fresh temporary
|
|
113
|
+
database for every round and fails if any round loses durable rows.
|
|
114
|
+
|
|
115
|
+
## Current boundaries
|
|
116
|
+
|
|
117
|
+
- Embedded databases now require exclusive ownership by one engine. A second
|
|
118
|
+
engine or process opening the same path receives an error. Multiple application
|
|
119
|
+
processes should connect through the server. The adjacent `.lock` file is
|
|
120
|
+
intentionally retained after close; the operating system releases ownership
|
|
121
|
+
on close or process exit. Never delete it while the database is open. This
|
|
122
|
+
requires a filesystem that implements file locking correctly. Hard-linked
|
|
123
|
+
database aliases and shared custom WAL/metadata paths are unsupported.
|
|
124
|
+
|
|
125
|
+
- Failed metadata publication rolls back the in-memory schema and leaves the
|
|
126
|
+
durable catalog unchanged; callers receive an error and may retry the
|
|
127
|
+
mutation. Exercise disk-full and interrupted-rename fault injection on the
|
|
128
|
+
target filesystem before release.
|
|
129
|
+
|
|
130
|
+
- Join support currently covers qualified `INNER`, `LEFT [OUTER]`, `RIGHT`, and
|
|
131
|
+
`FULL [OUTER] JOIN` with `ON` predicates. Join reordering is limited to safe
|
|
132
|
+
inner-join plans; correlated subqueries, advanced set-operation ordering,
|
|
133
|
+
and broader dialect-specific SQL still require dedicated compatibility tests.
|
|
134
|
+
- The ActiveRecord migration test is intentionally scoped. Complex table
|
|
135
|
+
rebuilds, `change_column`, polymorphic references, generated columns, and
|
|
136
|
+
adapter-specific schema dumps require dedicated compatibility tests before
|
|
137
|
+
relying on them.
|
|
138
|
+
- The soak harness and multi-process server workload cover distinct concurrency
|
|
139
|
+
paths, but they do not provide a universal capacity certification; perform
|
|
140
|
+
environment-specific load, crash, and operational recovery testing.
|
|
141
|
+
- Failover is manual and requires an operator to confirm the replica is caught
|
|
142
|
+
up and that the old primary is fenced. Automatic leader election is not
|
|
143
|
+
enabled.
|
|
144
|
+
- The replica fence covers engine schema, row, branch, vacuum, and compaction
|
|
145
|
+
mutation entry points. Empty-replica catalog bootstrap, transaction-integrated
|
|
146
|
+
replication, and reconnect catch-up are covered; true multi-host partition,
|
|
147
|
+
split-brain, and automated-election validation remain deployment work.
|
|
148
|
+
|
|
149
|
+
RubyDB reports unsupported features as unsupported rather than advertising CTE
|
|
150
|
+
or bulk-alter capability to ActiveRecord.
|
|
@@ -0,0 +1,11 @@
|
|
|
1
|
+
# ActiveRecord adapter
|
|
2
|
+
|
|
3
|
+
The `rubydb-activerecord` adapter connects ActiveRecord models to RubyDB. The
|
|
4
|
+
tested surface includes CRUD, quoted identifiers, binds, associations, joins,
|
|
5
|
+
eager loading, nested associations, schema inspection, transactions, indexes,
|
|
6
|
+
defaults, schema dumps, and populated-table migration paths.
|
|
7
|
+
|
|
8
|
+
Run the adapter suite from `adapters/activerecord` for the target Rails version.
|
|
9
|
+
Use server mode when multiple Rails processes share a database. The adapter is
|
|
10
|
+
not a complete PostgreSQL, MySQL, or SQLite compatibility layer; validate any
|
|
11
|
+
application-specific Arel, extension, callback, migration, or SQL behavior.
|
|
@@ -0,0 +1,90 @@
|
|
|
1
|
+
# Rails compatibility and integration guide
|
|
2
|
+
|
|
3
|
+
RubyDB’s ActiveRecord adapter is intended for applications that stay within
|
|
4
|
+
the documented RubyDB SQL and schema surface. It is not a claim of complete
|
|
5
|
+
Rails compatibility or complete SQLite compatibility. Validate the exact
|
|
6
|
+
Rails/Ruby versions, gems, queries, migrations, and deployment topology used
|
|
7
|
+
by your application.
|
|
8
|
+
|
|
9
|
+
## Choose the ownership mode
|
|
10
|
+
|
|
11
|
+
Embedded mode is appropriate for a single-owner development process or a
|
|
12
|
+
deliberately single-process application. Rails web servers, job workers,
|
|
13
|
+
console sessions, and migration commands must not independently open the same
|
|
14
|
+
embedded path.
|
|
15
|
+
|
|
16
|
+
For multiple processes, start one RubyDB server and configure every Rails
|
|
17
|
+
process to use the client/server adapter. Size the Rails connection pool below
|
|
18
|
+
the server’s connection limit and leave headroom for deploys and health
|
|
19
|
+
checks.
|
|
20
|
+
|
|
21
|
+
## First application
|
|
22
|
+
|
|
23
|
+
Use the maintained example as a smoke harness:
|
|
24
|
+
|
|
25
|
+
```sh
|
|
26
|
+
cd examples/rails_app
|
|
27
|
+
bundle install
|
|
28
|
+
bundle exec ruby bin/rails db:migrate
|
|
29
|
+
bundle exec ruby bin/rails runner 'puts User.count'
|
|
30
|
+
bundle exec ruby bin/rails server -b 127.0.0.1 -p 3001
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
Set `RUBYDB_DATABASE` to choose the database path in the embedded example.
|
|
34
|
+
Before production, replace the local path with the managed server configuration
|
|
35
|
+
and test the same migration and query flow through the network adapter.
|
|
36
|
+
|
|
37
|
+
## Supported application patterns to validate
|
|
38
|
+
|
|
39
|
+
Test the application’s real use of:
|
|
40
|
+
|
|
41
|
+
* model creation, updates, deletes, validations, and transactions;
|
|
42
|
+
* `where`, scopes, ordering, limits, offsets, projections, and bind values;
|
|
43
|
+
* inner/left/multi-table joins, aliases, grouped aggregates, and `HAVING`;
|
|
44
|
+
* eager loading, nested associations, and inverse association behavior;
|
|
45
|
+
* connection pools, checkout timeouts, reconnects, and shutdown;
|
|
46
|
+
* savepoints, rollback, retry, and deadlock handling;
|
|
47
|
+
* schema dump/load, defaults, indexes, foreign keys, and constraints; and
|
|
48
|
+
* fresh and populated-table migrations with rollback plans.
|
|
49
|
+
|
|
50
|
+
Generated SQL is part of the compatibility surface. Capture representative
|
|
51
|
+
queries and compare results, affected rows, exceptions, and transaction state.
|
|
52
|
+
|
|
53
|
+
## Migration discipline
|
|
54
|
+
|
|
55
|
+
Treat migrations as deployment code. Review generated SQL, run against a fresh
|
|
56
|
+
database and a realistic populated copy, measure locks and duration, and
|
|
57
|
+
define the rollback/backfill plan. Do not assume that a migration that creates
|
|
58
|
+
a table on an empty database is safe on a large live table.
|
|
59
|
+
|
|
60
|
+
Record the migration version and checksum. A changed migration should be
|
|
61
|
+
treated as a new migration or an explicit controlled repair, not silently
|
|
62
|
+
accepted. Take a verified backup before destructive schema changes.
|
|
63
|
+
|
|
64
|
+
## Pooling and concurrency
|
|
65
|
+
|
|
66
|
+
The pool must return connections with no open transaction, savepoint, lock, or
|
|
67
|
+
pending cancellation. Test pool exhaustion and a server restart while Rails
|
|
68
|
+
threads are active. Set bounded checkout and request timeouts. A client
|
|
69
|
+
timeout does not automatically prove that a write rolled back; use an
|
|
70
|
+
idempotency key or query the outcome before retrying.
|
|
71
|
+
|
|
72
|
+
Run a multi-process workload with web-like reads, writes, jobs, migrations, and
|
|
73
|
+
connection churn. Track p95/p99 latency, lock waits, deadlocks, cancellations,
|
|
74
|
+
pool utilization, WAL growth, and error classes.
|
|
75
|
+
|
|
76
|
+
## Version matrix
|
|
77
|
+
|
|
78
|
+
The repository must run its Rails compatibility suite against every supported
|
|
79
|
+
combination, for example Rails 7.1, 7.2, and 8.0 with the Ruby versions declared
|
|
80
|
+
by the project. A local pass on one version is not evidence for the matrix.
|
|
81
|
+
Record unsupported combinations explicitly in release documentation.
|
|
82
|
+
|
|
83
|
+
## Debugging adapter failures
|
|
84
|
+
|
|
85
|
+
Start with [Rails troubleshooting](troubleshooting.md) and
|
|
86
|
+
[Debugging RubyDB](../debugging.md). Capture sanitized generated SQL, bind
|
|
87
|
+
count/order, transaction boundaries, pool state, Ruby/Rails versions, and the
|
|
88
|
+
smallest model/migration that reproduces the failure. Test direct SQL to
|
|
89
|
+
separate Rails query generation from engine behavior.
|
|
90
|
+
|
|
@@ -0,0 +1,92 @@
|
|
|
1
|
+
# Rails database configuration
|
|
2
|
+
|
|
3
|
+
Embedded single-owner configuration:
|
|
4
|
+
|
|
5
|
+
```yaml
|
|
6
|
+
development:
|
|
7
|
+
adapter: rubydb
|
|
8
|
+
database: tmp/development.rdb
|
|
9
|
+
embedded: true
|
|
10
|
+
```
|
|
11
|
+
|
|
12
|
+
For multiple web or worker processes, use a managed RubyDB server and configure
|
|
13
|
+
the host, port, credentials, timeout, pool size, TLS, and database name through
|
|
14
|
+
protected deployment configuration. Do not put passwords, replication tokens,
|
|
15
|
+
or private keys in `database.yml` committed to source control.
|
|
16
|
+
|
|
17
|
+
Match the total Rails pool size to the server connection limit and leave headroom
|
|
18
|
+
for migrations, monitoring, and replication. Test the exact configuration with
|
|
19
|
+
the Rails matrix before release.
|
|
20
|
+
|
|
21
|
+
## Production server/client configuration
|
|
22
|
+
|
|
23
|
+
For a multi-process Rails deployment, keep the server private and put only
|
|
24
|
+
non-secret connection settings in source control. This is a complete example;
|
|
25
|
+
the `RUBYDB_*` values are supplied by the deployment environment or secret
|
|
26
|
+
manager:
|
|
27
|
+
|
|
28
|
+
```yaml
|
|
29
|
+
production:
|
|
30
|
+
adapter: rubydb
|
|
31
|
+
embedded: false
|
|
32
|
+
host: <%= ENV.fetch("RUBYDB_HOST") %>
|
|
33
|
+
port: <%= ENV.fetch("RUBYDB_PORT", "7432") %>
|
|
34
|
+
database: <%= ENV.fetch("RUBYDB_DATABASE", "rubydb") %>
|
|
35
|
+
username: <%= ENV.fetch("RUBYDB_USERNAME") %>
|
|
36
|
+
password: <%= ENV.fetch("RUBYDB_PASSWORD") %>
|
|
37
|
+
timeout: <%= ENV.fetch("RUBYDB_TIMEOUT", "30") %>
|
|
38
|
+
pool: <%= ENV.fetch("RAILS_MAX_THREADS", "5") %>
|
|
39
|
+
ssl:
|
|
40
|
+
enabled: <%= ENV.fetch("RUBYDB_SSL_ENABLED", "true") == "true" %>
|
|
41
|
+
ca_file: <%= ENV.fetch("RUBYDB_SSL_CA_FILE") %>
|
|
42
|
+
verify_peer: <%= ENV.fetch("RUBYDB_SSL_VERIFY_PEER", "true") == "true" %>
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
Deploy the application with values similar to:
|
|
46
|
+
|
|
47
|
+
```sh
|
|
48
|
+
RUBYDB_HOST=db.internal.example
|
|
49
|
+
RUBYDB_PORT=7432
|
|
50
|
+
RUBYDB_DATABASE=app
|
|
51
|
+
RUBYDB_USERNAME=app_rw
|
|
52
|
+
RUBYDB_PASSWORD='provided-by-secret-manager'
|
|
53
|
+
RUBYDB_SSL_ENABLED=true
|
|
54
|
+
RUBYDB_SSL_CA_FILE=/etc/rubydb/tls/ca.crt
|
|
55
|
+
RUBYDB_SSL_VERIFY_PEER=true
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
The exact variable names are application conventions; Rails reads them because
|
|
59
|
+
`database.yml` maps them into adapter settings. Do not put the password in a
|
|
60
|
+
committed YAML file, Docker image, URL, shell history, or log. The server must
|
|
61
|
+
be configured separately with its own data directory, authentication, TLS
|
|
62
|
+
certificate/key, and resource limits; see [server configuration](../server/configuration.md).
|
|
63
|
+
|
|
64
|
+
## One connection URL
|
|
65
|
+
|
|
66
|
+
The adapter also accepts a RubyDB-specific URL through `url`. This is useful
|
|
67
|
+
when a hosting provider gives the application one connection string:
|
|
68
|
+
|
|
69
|
+
```yaml
|
|
70
|
+
production:
|
|
71
|
+
adapter: rubydb
|
|
72
|
+
embedded: false
|
|
73
|
+
url: <%= ENV.fetch("RUBYDB_URL") %>
|
|
74
|
+
pool: <%= ENV.fetch("RAILS_MAX_THREADS", "5") %>
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
Example URL with TLS verification:
|
|
78
|
+
|
|
79
|
+
```text
|
|
80
|
+
rubydbs://app_user:password@db.example.com:7432/app?verify_peer=true&ca_file=%2Fetc%2Frubydb%2Fca.crt
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
Supported schemes are `rubydb://` and `rubydbs://`; `rubydbs` enables TLS.
|
|
84
|
+
The URL supports percent-encoded username/password/database values and query
|
|
85
|
+
options including `ssl`, `sslmode`, `verify_peer`, `ca_file`, `cert_file`,
|
|
86
|
+
`key_file`, `min_version`, `timeout`, `pool_size`, `compress`, and `format`.
|
|
87
|
+
Use a secret manager to inject `RUBYDB_URL`; URLs containing passwords must not
|
|
88
|
+
be committed, printed, or placed in public issue reports. This is RubyDB’s
|
|
89
|
+
URL format, not a PostgreSQL wire-protocol or generic `DATABASE_URL` promise.
|
|
90
|
+
|
|
91
|
+
Use the documented mapping above and run a real migration plus smoke query
|
|
92
|
+
through the network path before production.
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
# Rails adapter installation
|
|
2
|
+
|
|
3
|
+
From the repository adapter directory:
|
|
4
|
+
|
|
5
|
+
```sh
|
|
6
|
+
cd adapters/activerecord
|
|
7
|
+
bundle install
|
|
8
|
+
bundle exec rspec
|
|
9
|
+
```
|
|
10
|
+
|
|
11
|
+
Add the adapter gem to the application bundle according to the release being
|
|
12
|
+
tested. Configure `adapter: rubydb` and a database path or server endpoint. Run
|
|
13
|
+
`db:migrate`, schema dump/load, representative joins, eager loading, nested
|
|
14
|
+
associations, and rollback tests before deployment.
|
|
15
|
+
|
|
16
|
+
The CI matrix currently exercises Rails 7.1, 7.2, and 8.0 with the supported
|
|
17
|
+
Ruby versions. Hosted results are the release evidence for a combination.
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
# Rails migrations
|
|
2
|
+
|
|
3
|
+
RubyDB supports the documented ActiveRecord migration surface, including table
|
|
4
|
+
creation, integer primary keys, columns, defaults, indexes, constraints, and
|
|
5
|
+
populated-table round trips covered by the adapter suite.
|
|
6
|
+
|
|
7
|
+
Before production migration:
|
|
8
|
+
|
|
9
|
+
1. verify a full backup and WAL chain;
|
|
10
|
+
2. restore into a new staging directory with representative data;
|
|
11
|
+
3. run the migration, schema dump/load, application queries, and rollback path;
|
|
12
|
+
4. confirm the migration checksum and schema status;
|
|
13
|
+
5. retain the pre-migration directory until rollback is no longer needed.
|
|
14
|
+
|
|
15
|
+
RubyDB fails closed when an applied migration changes or disappears. Unsupported
|
|
16
|
+
table rebuilds, generated columns, polymorphic references, and dialect-specific
|
|
17
|
+
extensions require explicit validation.
|
|
@@ -0,0 +1,82 @@
|
|
|
1
|
+
# Rails production guidance
|
|
2
|
+
|
|
3
|
+
RubyDB can serve Rails applications that stay within the documented RubyDB SQL
|
|
4
|
+
and ActiveRecord adapter surface. Validate the exact Rails, Ruby, operating
|
|
5
|
+
system, schema, workload, and deployment topology before handling important
|
|
6
|
+
data. The compatibility workflow currently exercises Rails 7.1, 7.2, and 8.0
|
|
7
|
+
with Ruby 3.3; hosted workflow results are required before declaring a release
|
|
8
|
+
combination supported.
|
|
9
|
+
|
|
10
|
+
## Choose the topology
|
|
11
|
+
|
|
12
|
+
An embedded database path has exclusive ownership. Use it only when one Rails
|
|
13
|
+
process owns the database, such as a single-process development or controlled
|
|
14
|
+
worker deployment. Do not point multiple independent Rails processes at the
|
|
15
|
+
same embedded path.
|
|
16
|
+
|
|
17
|
+
For multiple web/worker processes, run a separately managed RubyDB server and
|
|
18
|
+
configure clients to use its network endpoint. Size the server connection
|
|
19
|
+
limit for the Rails pool total, keep application and replication traffic on
|
|
20
|
+
private networks, and enable TLS and authentication. A Rails connection pool
|
|
21
|
+
does not make an embedded file safe for concurrent process ownership.
|
|
22
|
+
|
|
23
|
+
## Migrations and releases
|
|
24
|
+
|
|
25
|
+
Before a migration:
|
|
26
|
+
|
|
27
|
+
1. Run the complete test suite and the adapter suite for the target Rails
|
|
28
|
+
version.
|
|
29
|
+
2. Create and verify a full backup, including the WAL chain when applicable.
|
|
30
|
+
3. Restore the backup into a separate staging directory and run the migration
|
|
31
|
+
against the restored copy with a representative populated dataset.
|
|
32
|
+
4. Test schema dump/load, indexes, defaults, foreign keys, eager loading,
|
|
33
|
+
nested associations, joins, and application write/read smoke queries.
|
|
34
|
+
|
|
35
|
+
Stop writes or use the deployment's migration lock. RubyDB fails closed when an
|
|
36
|
+
already-applied migration changes checksum or disappears. Keep the pre-release
|
|
37
|
+
backup and old data directory until the rollback window closes; do not delete
|
|
38
|
+
or overwrite the live directory during rollback.
|
|
39
|
+
|
|
40
|
+
## Operations
|
|
41
|
+
|
|
42
|
+
Collect request rate/error rate/latency, active connections, WAL and checkpoint
|
|
43
|
+
health, filesystem space, recovery-required acknowledgements, and replication
|
|
44
|
+
received/replayed LSNs. Alert on readiness failure, repeated errors, failed
|
|
45
|
+
flush/checkpoint, low disk space, excessive pool wait, authentication failures,
|
|
46
|
+
and replication lag beyond the application RPO.
|
|
47
|
+
|
|
48
|
+
Run `scripts/restore_drill`, the threaded and multi-process workloads, and
|
|
49
|
+
`scripts/replication_failover_drill` on target-like staging infrastructure.
|
|
50
|
+
Archive the Ruby version, RubyDB commit, Rails version, host resources, result
|
|
51
|
+
JSON, and recovery time with the release record.
|
|
52
|
+
|
|
53
|
+
## Current boundaries
|
|
54
|
+
|
|
55
|
+
The adapter is not a complete PostgreSQL, MySQL, or SQLite compatibility layer.
|
|
56
|
+
The tested common SQLite-style profile is documented in
|
|
57
|
+
[`docs/sql/sqlite-compatibility.md`](../sql/sqlite-compatibility.md). Complex
|
|
58
|
+
table rebuilds, generated columns, polymorphic references, adapter-specific
|
|
59
|
+
schema features, and dialect-specific SQL require explicit validation before
|
|
60
|
+
use. Automatic high-availability election is disabled; failover is an
|
|
61
|
+
operator-controlled, fenced procedure described in
|
|
62
|
+
[`docs/operations/failover.md`](../operations/failover.md).
|
|
63
|
+
|
|
64
|
+
## Environment-based deployment
|
|
65
|
+
|
|
66
|
+
The Rails application normally points at RubyDB through `config/database.yml`.
|
|
67
|
+
For a production server, set `embedded: false` and map protected environment
|
|
68
|
+
values to `host`, `port`, `database`, `username`, `password`, `timeout`, and
|
|
69
|
+
the `ssl` hash. See the copy-paste example in
|
|
70
|
+
[Rails database configuration](database-yml.md). The application connects to
|
|
71
|
+
the RubyDB server endpoint; it does not open the server’s data directory.
|
|
72
|
+
|
|
73
|
+
If the deployment platform provides one connection string, set
|
|
74
|
+
`RUBYDB_URL` to the documented `rubydb://` or `rubydbs://` format and use the
|
|
75
|
+
`url:` form in `database.yml`. Do not assume a PostgreSQL `DATABASE_URL` will
|
|
76
|
+
work; RubyDB uses its own protocol and URL scheme.
|
|
77
|
+
|
|
78
|
+
Deploy the RubyDB server separately with its own persistent volume and
|
|
79
|
+
`config/production.yml`. Verify TLS hostname/CA validation, authentication,
|
|
80
|
+
readiness, migration status, and a read/write smoke query from the same network
|
|
81
|
+
path as the Rails application. Use a secret manager for credentials and never
|
|
82
|
+
commit or print them.
|
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
# Rails troubleshooting
|
|
2
|
+
|
|
3
|
+
## Connection or ownership errors
|
|
4
|
+
|
|
5
|
+
Confirm the database path exists, the process has permission, and no second
|
|
6
|
+
embedded owner is using it. Multiple Rails processes must use server mode.
|
|
7
|
+
|
|
8
|
+
## Migration failures
|
|
9
|
+
|
|
10
|
+
Stop writes, preserve the database and WAL, inspect migration status and
|
|
11
|
+
checksums, and retry only after restoring a verified staging copy. Do not delete
|
|
12
|
+
WAL or overwrite the original directory.
|
|
13
|
+
|
|
14
|
+
## Query failures
|
|
15
|
+
|
|
16
|
+
Capture the RubyDB version, Rails/Ruby versions, generated SQL without secrets,
|
|
17
|
+
and a minimal schema/query reproduction. Check the documented SQL and adapter
|
|
18
|
+
compatibility surface before changing application behavior.
|
data/docs/release.md
ADDED
|
@@ -0,0 +1,59 @@
|
|
|
1
|
+
# RubyDB release checklist
|
|
2
|
+
|
|
3
|
+
The release workflow is intentionally fail-closed. A maintainer must configure
|
|
4
|
+
these protected GitHub Actions secrets:
|
|
5
|
+
|
|
6
|
+
- `RUBYGEMS_API_KEY`: a RubyGems API key scoped to the gem
|
|
7
|
+
- `RUBYDB_GEM_SIGNING_KEY_B64`: base64-encoded private signing key
|
|
8
|
+
- `RUBYDB_GEM_CERT_B64`: base64-encoded certificate chain
|
|
9
|
+
|
|
10
|
+
Create the signing key and certificate outside the repository, store them in a
|
|
11
|
+
secret manager, and rotate them according to the organization's key policy.
|
|
12
|
+
Never commit the private key or write it to a persistent workspace.
|
|
13
|
+
|
|
14
|
+
To publish, review `CHANGELOG.md`, commit the version, create a matching tag,
|
|
15
|
+
and let `.github/workflows/release.yml` run the full suite, build the gem,
|
|
16
|
+
verify its SHA-512 checksum, attest provenance, sign it, and publish it. A
|
|
17
|
+
local artifact check is:
|
|
18
|
+
|
|
19
|
+
```sh
|
|
20
|
+
ruby scripts/release_check
|
|
21
|
+
ruby scripts/release
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
The local command does not publish unless `RUBYDB_PUBLISH=1` and
|
|
25
|
+
`GEM_HOST_API_KEY` are explicitly set. Publishing also requires the release
|
|
26
|
+
version to be supplied when you are not running from a matching Git tag.
|
|
27
|
+
|
|
28
|
+
On Windows PowerShell, run this from the repository root:
|
|
29
|
+
|
|
30
|
+
```powershell
|
|
31
|
+
$env:RUBYDB_RELEASE_VERSION = "0.1.0"
|
|
32
|
+
$env:RUBYDB_PUBLISH = "1"
|
|
33
|
+
$env:GEM_HOST_API_KEY = "YOUR_RUBYGEMS_API_KEY"
|
|
34
|
+
ruby scripts/release
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
Instead of `GEM_HOST_API_KEY`, RubyGems can read a credentials file. With the
|
|
38
|
+
RubyGems version bundled with Ruby 4 on Windows, create this file:
|
|
39
|
+
`C:\Users\<your-user>\.local\share\gem\credentials`
|
|
40
|
+
|
|
41
|
+
```yaml
|
|
42
|
+
---
|
|
43
|
+
:rubygems_api_key: YOUR_RUBYGEMS_API_KEY
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
Then omit `GEM_HOST_API_KEY`; `scripts/release` and `gem push` will use the
|
|
47
|
+
file automatically. Keep the file private and never commit it.
|
|
48
|
+
|
|
49
|
+
On macOS/Linux, use:
|
|
50
|
+
|
|
51
|
+
```sh
|
|
52
|
+
RUBYDB_RELEASE_VERSION=0.1.0 \
|
|
53
|
+
RUBYDB_PUBLISH=1 \
|
|
54
|
+
GEM_HOST_API_KEY="YOUR_RUBYGEMS_API_KEY" \
|
|
55
|
+
ruby scripts/release
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
Never commit the API key or put it in a tracked file. Review the generated
|
|
59
|
+
GitHub release notes and uploaded checksum before announcing a version.
|
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
# Server architecture
|
|
2
|
+
|
|
3
|
+
The server owns one embedded engine and exposes client sessions over the RubyDB
|
|
4
|
+
protocol. A listener accepts bounded connections, a connection pool tracks
|
|
5
|
+
active sessions, and a worker pool handles requests under configured limits.
|
|
6
|
+
|
|
7
|
+
Lifecycle shutdown stops acceptance, closes connections, joins workers, flushes
|
|
8
|
+
the engine, and removes the PID file. Multiple application processes should
|
|
9
|
+
share the server, not the embedded path. Use health/readiness and Prometheus
|
|
10
|
+
metrics to decide whether to route traffic.
|
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
# Server authentication
|
|
2
|
+
|
|
3
|
+
Configure authentication explicitly for production. Password/SCRAM-SHA-256,
|
|
4
|
+
authorization, peer replication tokens, bounded frames, TLS, and server
|
|
5
|
+
signature verification are covered by the security suite.
|
|
6
|
+
|
|
7
|
+
Use high-entropy credentials from a secret manager. Bind to a private address,
|
|
8
|
+
enable TLS 1.2 or newer, validate the CA and hostname, and rotate credentials
|
|
9
|
+
through a drain/reconnect procedure. Missing or incomplete authentication
|
|
10
|
+
configuration must fail startup rather than silently downgrade security.
|
|
@@ -0,0 +1,16 @@
|
|
|
1
|
+
# Server configuration
|
|
2
|
+
|
|
3
|
+
Run with an explicit database and resource limits:
|
|
4
|
+
|
|
5
|
+
```sh
|
|
6
|
+
rubydb-server --host 127.0.0.1 --port 7432 --database /var/lib/rubydb/app.rdb
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
Production configuration should set host/port, data and log directories, PID
|
|
10
|
+
file, max connections, worker and queue limits, request/frame size, read/write
|
|
11
|
+
and idle timeouts, authentication, and TLS. Keep configuration outside source
|
|
12
|
+
control when it contains secrets.
|
|
13
|
+
|
|
14
|
+
Validate configuration before startup and monitor readiness, active/rejected
|
|
15
|
+
connections, request errors, disk space, WAL/checkpoint state, and recovery-
|
|
16
|
+
required acknowledgements.
|
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
# Connection pooling
|
|
2
|
+
|
|
3
|
+
The server limits active connections and queues work through its connection and
|
|
4
|
+
worker pools. Rails/client pool totals must fit below the server limit with
|
|
5
|
+
headroom for administrative and replication traffic.
|
|
6
|
+
|
|
7
|
+
Set finite acquisition, read, write, idle, and query timeouts. Alert on pool
|
|
8
|
+
wait, rejected connections, request failures, and saturation. Load-test pool
|
|
9
|
+
behavior with the multi-process and production soak harness; a successful local
|
|
10
|
+
connection does not prove capacity under application traffic.
|
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
# Server deployment
|
|
2
|
+
|
|
3
|
+
Run RubyDB under a dedicated least-privilege service account with protected
|
|
4
|
+
data, WAL, logs, TLS, and backup paths. Use the packaged systemd unit or the
|
|
5
|
+
non-root Docker image, private networking, TLS, authentication, resource
|
|
6
|
+
limits, health checks, and persistent storage.
|
|
7
|
+
|
|
8
|
+
Before accepting traffic, run status/readiness, smoke queries, backup/restore,
|
|
9
|
+
workload, and recovery checks. Keep one writable primary, fence before
|
|
10
|
+
promotion, and retain the previous data directory during upgrades.
|
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
# RubyDB server protocol
|
|
2
|
+
|
|
3
|
+
The client/server protocol uses bounded framed requests and responses with
|
|
4
|
+
handshake, authentication, query, prepared statement, transaction, health, and
|
|
5
|
+
metrics operations. Frames must be size-limited and malformed input must be
|
|
6
|
+
rejected without taking down the server.
|
|
7
|
+
|
|
8
|
+
Query deadlines and in-flight cancellation are request-scoped. Cancellation is
|
|
9
|
+
cooperative and must be checked by long-running execution paths. Do not expose
|
|
10
|
+
the listener publicly; use TLS/private networking and rotate credentials through
|
|
11
|
+
the documented operations procedure. The normative notes are in
|
|
12
|
+
`spec/wire/protocol.md` and `spec/protocol/protocol.md`.
|