rubydb 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/.gitattributes +20 -0
- data/.github/ISSUE_TEMPLATE/bug.yml +0 -0
- data/.github/ISSUE_TEMPLATE/feature.yml +0 -0
- data/.github/ISSUE_TEMPLATE/performance.yml +0 -0
- data/.github/ISSUE_TEMPLATE/security.yml +0 -0
- data/.github/PULL_REQUEST_TEMPLATE.md +15 -0
- data/.github/dependabot.yml +0 -0
- data/.github/workflows/benchmark.yml +26 -0
- data/.github/workflows/compatibility.yml +63 -0
- data/.github/workflows/fuzz.yml +33 -0
- data/.github/workflows/lint.yml +21 -0
- data/.github/workflows/operations.yml +24 -0
- data/.github/workflows/production-validation.yml +111 -0
- data/.github/workflows/release.yml +77 -0
- data/.github/workflows/security.yml +37 -0
- data/.github/workflows/test.yml +26 -0
- data/.github/workflows/workload.yml +58 -0
- data/.gitignore +43 -0
- data/.rspec +4 -0
- data/.rubocop.yml +44 -0
- data/.ruby-version +1 -0
- data/.standard.yml +14 -0
- data/ARCHITECTURE.md +21 -0
- data/CHANGELOG.md +31 -0
- data/CODE_OF_CONDUCT.md +13 -0
- data/CONTRIBUTING.md +29 -0
- data/GOVERNANCE.md +16 -0
- data/Gemfile +17 -0
- data/Gemfile.lock +71 -0
- data/LICENSE +60 -0
- data/README.md +162 -0
- data/ROADMAP.md +27 -0
- data/Rakefile +71 -0
- data/SECURITY.md +54 -0
- data/SUPPORT.md +14 -0
- data/adapters/activerecord/Gemfile +11 -0
- data/adapters/activerecord/README.md +9 -0
- data/adapters/activerecord/lib/active_record/connection_adapters/rubydb_adapter.rb +887 -0
- data/adapters/activerecord/rubydb-activerecord.gemspec +27 -0
- data/adapters/activerecord/spec/rubydb_adapter_integration_spec.rb +151 -0
- data/adapters/ruby/README.md +18 -0
- data/adapters/sequel/README.md +11 -0
- data/bin/rubydb +19 -0
- data/bin/rubydb-server +126 -0
- data/config/development.yml +0 -0
- data/config/monitoring/prometheus-alerts.yml +39 -0
- data/config/production.yml +36 -0
- data/config/test.yml +0 -0
- data/docs/README.md +71 -0
- data/docs/architecture/concurrency.md +14 -0
- data/docs/architecture/current-state.md +125 -0
- data/docs/architecture/execution-engine.md +25 -0
- data/docs/architecture/indexes.md +19 -0
- data/docs/architecture/mvcc.md +19 -0
- data/docs/architecture/overview.md +13 -0
- data/docs/architecture/pages.md +11 -0
- data/docs/architecture/production-roadmap.md +82 -0
- data/docs/architecture/query-planner.md +20 -0
- data/docs/architecture/recovery.md +18 -0
- data/docs/architecture/sql-engine.md +12 -0
- data/docs/architecture/storage-engine.md +14 -0
- data/docs/architecture/transactions.md +10 -0
- data/docs/architecture/wal.md +28 -0
- data/docs/cli-cheatsheet.md +98 -0
- data/docs/cli.md +275 -0
- data/docs/contributing/architecture.md +9 -0
- data/docs/contributing/benchmarking.md +14 -0
- data/docs/contributing/development.md +16 -0
- data/docs/contributing/release-process.md +49 -0
- data/docs/contributing/testing.md +16 -0
- data/docs/debugging.md +229 -0
- data/docs/developer/branching.md +10 -0
- data/docs/developer/database-diff.md +10 -0
- data/docs/developer/local-development.md +17 -0
- data/docs/developer/snapshots.md +9 -0
- data/docs/developer/temporal-data.md +10 -0
- data/docs/developer-guide.md +297 -0
- data/docs/getting-started/first-database.md +16 -0
- data/docs/getting-started/first-query.md +13 -0
- data/docs/getting-started/installation.md +19 -0
- data/docs/getting-started/local-to-production.md +300 -0
- data/docs/getting-started/quickstart.md +17 -0
- data/docs/getting-started/rails.md +16 -0
- data/docs/hardening_backlog.md +93 -0
- data/docs/lessons-learned.md +112 -0
- data/docs/operations/backups.md +33 -0
- data/docs/operations/disaster-recovery.md +31 -0
- data/docs/operations/failover.md +30 -0
- data/docs/operations/monitoring.md +25 -0
- data/docs/operations/production-guide.md +295 -0
- data/docs/operations/production-runbook.md +45 -0
- data/docs/operations/replication.md +33 -0
- data/docs/operations/restore.md +6 -0
- data/docs/operations/runbook.md +34 -0
- data/docs/operations/upgrades.md +14 -0
- data/docs/operations/workload-testing.md +17 -0
- data/docs/production-readiness.md +118 -0
- data/docs/production_validation.md +150 -0
- data/docs/rails/active-record.md +11 -0
- data/docs/rails/compatibility-guide.md +90 -0
- data/docs/rails/database-yml.md +92 -0
- data/docs/rails/installation.md +17 -0
- data/docs/rails/migrations.md +17 -0
- data/docs/rails/production.md +82 -0
- data/docs/rails/troubleshooting.md +18 -0
- data/docs/release.md +59 -0
- data/docs/server/architecture.md +10 -0
- data/docs/server/authentication.md +10 -0
- data/docs/server/configuration.md +16 -0
- data/docs/server/connection-pooling.md +10 -0
- data/docs/server/deployment.md +10 -0
- data/docs/server/protocol.md +12 -0
- data/docs/sql/compatibility-guide.md +82 -0
- data/docs/sql/compatibility.md +39 -0
- data/docs/sql/data-types.md +10 -0
- data/docs/sql/functions.md +9 -0
- data/docs/sql/joins.md +9 -0
- data/docs/sql/operators.md +9 -0
- data/docs/sql/sqlite-compatibility.md +21 -0
- data/docs/sql/syntax.md +10 -0
- data/docs/sql/transactions.md +10 -0
- data/docs/troubleshooting.md +244 -0
- data/exe/rubydb +23 -0
- data/lib/rubydb/backup/archive.rb +334 -0
- data/lib/rubydb/backup/backup.rb +401 -0
- data/lib/rubydb/backup/incremental.rb +353 -0
- data/lib/rubydb/backup/restore.rb +290 -0
- data/lib/rubydb/backup/snapshot.rb +267 -0
- data/lib/rubydb/backup/verification.rb +279 -0
- data/lib/rubydb/branching/branch.rb +181 -0
- data/lib/rubydb/branching/branch_manager.rb +311 -0
- data/lib/rubydb/branching/branch_metadata.rb +140 -0
- data/lib/rubydb/branching/checkout.rb +166 -0
- data/lib/rubydb/branching/copy_on_write.rb +272 -0
- data/lib/rubydb/branching/diff.rb +138 -0
- data/lib/rubydb/branching/merge.rb +285 -0
- data/lib/rubydb/build_info.rb +15 -0
- data/lib/rubydb/catalog/catalog.rb +391 -0
- data/lib/rubydb/catalog/column.rb +112 -0
- data/lib/rubydb/catalog/constraint.rb +180 -0
- data/lib/rubydb/catalog/database.rb +184 -0
- data/lib/rubydb/catalog/index.rb +97 -0
- data/lib/rubydb/catalog/schema.rb +103 -0
- data/lib/rubydb/catalog/sequence.rb +90 -0
- data/lib/rubydb/catalog/system_catalog.rb +698 -0
- data/lib/rubydb/catalog/table.rb +178 -0
- data/lib/rubydb/catalog/trigger.rb +102 -0
- data/lib/rubydb/catalog/view.rb +66 -0
- data/lib/rubydb/cli/application.rb +163 -0
- data/lib/rubydb/cli/commands/backup.rb +81 -0
- data/lib/rubydb/cli/commands/branch.rb +72 -0
- data/lib/rubydb/cli/commands/checkout.rb +54 -0
- data/lib/rubydb/cli/commands/create.rb +58 -0
- data/lib/rubydb/cli/commands/diff.rb +77 -0
- data/lib/rubydb/cli/commands/doctor.rb +74 -0
- data/lib/rubydb/cli/commands/drop.rb +57 -0
- data/lib/rubydb/cli/commands/init.rb +102 -0
- data/lib/rubydb/cli/commands/inspect.rb +95 -0
- data/lib/rubydb/cli/commands/merge.rb +63 -0
- data/lib/rubydb/cli/commands/migrate.rb +62 -0
- data/lib/rubydb/cli/commands/restart.rb +39 -0
- data/lib/rubydb/cli/commands/restore.rb +121 -0
- data/lib/rubydb/cli/commands/shell.rb +365 -0
- data/lib/rubydb/cli/commands/snapshot.rb +79 -0
- data/lib/rubydb/cli/commands/start.rb +82 -0
- data/lib/rubydb/cli/commands/status.rb +92 -0
- data/lib/rubydb/cli/commands/stop.rb +47 -0
- data/lib/rubydb/cli/commands/vacuum.rb +58 -0
- data/lib/rubydb/cli/formatter.rb +221 -0
- data/lib/rubydb/cli/output.rb +168 -0
- data/lib/rubydb/client/client.rb +304 -0
- data/lib/rubydb/client/connection.rb +415 -0
- data/lib/rubydb/client/connection_pool.rb +168 -0
- data/lib/rubydb/client/connection_url.rb +96 -0
- data/lib/rubydb/client/prepared_statement.rb +60 -0
- data/lib/rubydb/client/result.rb +123 -0
- data/lib/rubydb/client/statement.rb +52 -0
- data/lib/rubydb/client/transaction.rb +130 -0
- data/lib/rubydb/concurrency/concurrency.rb +19 -0
- data/lib/rubydb/concurrency/deadlock_detector.rb +150 -0
- data/lib/rubydb/concurrency/latch.rb +101 -0
- data/lib/rubydb/concurrency/lock_graph.rb +165 -0
- data/lib/rubydb/concurrency/mutex.rb +183 -0
- data/lib/rubydb/concurrency/rw_lock.rb +180 -0
- data/lib/rubydb/concurrency/scheduler.rb +250 -0
- data/lib/rubydb/concurrency/worker_pool.rb +143 -0
- data/lib/rubydb/configuration/config.rb +170 -0
- data/lib/rubydb/configuration/defaults.rb +179 -0
- data/lib/rubydb/configuration/environment.rb +152 -0
- data/lib/rubydb/configuration/parser.rb +185 -0
- data/lib/rubydb/configuration/validation.rb +221 -0
- data/lib/rubydb/constants.rb +74 -0
- data/lib/rubydb/constraints/check.rb +181 -0
- data/lib/rubydb/constraints/constraint.rb +101 -0
- data/lib/rubydb/constraints/foreign_key.rb +130 -0
- data/lib/rubydb/constraints/not_null.rb +64 -0
- data/lib/rubydb/constraints/primary_key.rb +99 -0
- data/lib/rubydb/constraints/unique.rb +108 -0
- data/lib/rubydb/constraints/validator.rb +350 -0
- data/lib/rubydb/errors/authentication_error.rb +10 -0
- data/lib/rubydb/errors/authorization_error.rb +23 -0
- data/lib/rubydb/errors/client_error.rb +10 -0
- data/lib/rubydb/errors/configuration_error.rb +10 -0
- data/lib/rubydb/errors/connection_error.rb +10 -0
- data/lib/rubydb/errors/constraint_error.rb +23 -0
- data/lib/rubydb/errors/corruption_error.rb +10 -0
- data/lib/rubydb/errors/database_error.rb +10 -0
- data/lib/rubydb/errors/error.rb +20 -0
- data/lib/rubydb/errors/execution_error.rb +10 -0
- data/lib/rubydb/errors/parser_error.rb +10 -0
- data/lib/rubydb/errors/recovery_error.rb +10 -0
- data/lib/rubydb/errors/replication_error.rb +10 -0
- data/lib/rubydb/errors/server_error.rb +6 -0
- data/lib/rubydb/errors/storage_error.rb +10 -0
- data/lib/rubydb/errors/transaction_error.rb +10 -0
- data/lib/rubydb/execution/aggregate_executor.rb +138 -0
- data/lib/rubydb/execution/delete_executor.rb +112 -0
- data/lib/rubydb/execution/distinct_executor.rb +135 -0
- data/lib/rubydb/execution/executor.rb +1192 -0
- data/lib/rubydb/execution/expression.rb +193 -0
- data/lib/rubydb/execution/index_scan.rb +142 -0
- data/lib/rubydb/execution/insert_executor.rb +217 -0
- data/lib/rubydb/execution/join_executor.rb +249 -0
- data/lib/rubydb/execution/limit_executor.rb +85 -0
- data/lib/rubydb/execution/optimizer.rb +215 -0
- data/lib/rubydb/execution/plan.rb +361 -0
- data/lib/rubydb/execution/planner.rb +546 -0
- data/lib/rubydb/execution/predicate.rb +235 -0
- data/lib/rubydb/execution/scan.rb +49 -0
- data/lib/rubydb/execution/sequential_scan.rb +63 -0
- data/lib/rubydb/execution/sort_executor.rb +185 -0
- data/lib/rubydb/execution/update_executor.rb +162 -0
- data/lib/rubydb/functions/aggregate.rb +70 -0
- data/lib/rubydb/functions/date_functions.rb +278 -0
- data/lib/rubydb/functions/function.rb +85 -0
- data/lib/rubydb/functions/json_functions.rb +215 -0
- data/lib/rubydb/functions/numeric_functions.rb +346 -0
- data/lib/rubydb/functions/scalar.rb +52 -0
- data/lib/rubydb/functions/string_functions.rb +383 -0
- data/lib/rubydb/functions/system_functions.rb +246 -0
- data/lib/rubydb/history/as_of.rb +238 -0
- data/lib/rubydb/history/change.rb +105 -0
- data/lib/rubydb/history/history.rb +131 -0
- data/lib/rubydb/history/history_manager.rb +229 -0
- data/lib/rubydb/history/temporal_query.rb +202 -0
- data/lib/rubydb/history/timeline.rb +144 -0
- data/lib/rubydb/indexes/btree.rb +186 -0
- data/lib/rubydb/indexes/btree_cursor.rb +258 -0
- data/lib/rubydb/indexes/btree_node.rb +385 -0
- data/lib/rubydb/indexes/hash_index.rb +150 -0
- data/lib/rubydb/indexes/index.rb +71 -0
- data/lib/rubydb/indexes/index_manager.rb +406 -0
- data/lib/rubydb/indexes/index_scan.rb +470 -0
- data/lib/rubydb/migrations/migration.rb +254 -0
- data/lib/rubydb/migrations/migration_lock.rb +146 -0
- data/lib/rubydb/migrations/migration_manager.rb +176 -0
- data/lib/rubydb/migrations/migration_version.rb +71 -0
- data/lib/rubydb/migrations/schema_diff.rb +211 -0
- data/lib/rubydb/migrations/schema_version.rb +64 -0
- data/lib/rubydb/monitoring/events.rb +160 -0
- data/lib/rubydb/monitoring/health.rb +222 -0
- data/lib/rubydb/monitoring/logger.rb +193 -0
- data/lib/rubydb/monitoring/metrics.rb +359 -0
- data/lib/rubydb/monitoring/performance.rb +176 -0
- data/lib/rubydb/monitoring/statistics.rb +170 -0
- data/lib/rubydb/mvcc/garbage_collector.rb +199 -0
- data/lib/rubydb/mvcc/mvcc.rb +16 -0
- data/lib/rubydb/mvcc/snapshot.rb +147 -0
- data/lib/rubydb/mvcc/vacuum.rb +180 -0
- data/lib/rubydb/mvcc/version.rb +106 -0
- data/lib/rubydb/mvcc/version_store.rb +398 -0
- data/lib/rubydb/mvcc/visibility.rb +109 -0
- data/lib/rubydb/protocol/capabilities.rb +125 -0
- data/lib/rubydb/protocol/decoder.rb +145 -0
- data/lib/rubydb/protocol/encoder.rb +136 -0
- data/lib/rubydb/protocol/handshake.rb +305 -0
- data/lib/rubydb/protocol/message.rb +121 -0
- data/lib/rubydb/protocol/protocol.rb +277 -0
- data/lib/rubydb/protocol/version.rb +54 -0
- data/lib/rubydb/rails/adapter.rb +239 -0
- data/lib/rubydb/rails/connection.rb +314 -0
- data/lib/rubydb/rails/database_statements.rb +122 -0
- data/lib/rubydb/rails/migration.rb +131 -0
- data/lib/rubydb/rails/quoting.rb +109 -0
- data/lib/rubydb/rails/result.rb +120 -0
- data/lib/rubydb/rails/schema_statements.rb +339 -0
- data/lib/rubydb/rails/transaction.rb +105 -0
- data/lib/rubydb/rails/type.rb +126 -0
- data/lib/rubydb/recovery/checkpoint.rb +257 -0
- data/lib/rubydb/recovery/consistency.rb +467 -0
- data/lib/rubydb/recovery/corruption_detector.rb +5 -0
- data/lib/rubydb/recovery/crash_recovery.rb +387 -0
- data/lib/rubydb/recovery/recovery_manager.rb +206 -0
- data/lib/rubydb/recovery/redo.rb +237 -0
- data/lib/rubydb/recovery/undo.rb +206 -0
- data/lib/rubydb/replication/failover.rb +5 -0
- data/lib/rubydb/replication/fencing.rb +63 -0
- data/lib/rubydb/replication/primary.rb +450 -0
- data/lib/rubydb/replication/replica.rb +384 -0
- data/lib/rubydb/replication/replication_log.rb +200 -0
- data/lib/rubydb/replication/replication_manager.rb +308 -0
- data/lib/rubydb/replication/replication_slot.rb +295 -0
- data/lib/rubydb/replication/replication_stream.rb +201 -0
- data/lib/rubydb/rubydb.rb +560 -0
- data/lib/rubydb/security/access_control.rb +254 -0
- data/lib/rubydb/security/audit_log.rb +213 -0
- data/lib/rubydb/security/authentication.rb +302 -0
- data/lib/rubydb/security/authorization.rb +282 -0
- data/lib/rubydb/security/credentials.rb +196 -0
- data/lib/rubydb/security/password.rb +215 -0
- data/lib/rubydb/security/permissions.rb +74 -0
- data/lib/rubydb/security/role.rb +101 -0
- data/lib/rubydb/security/user.rb +86 -0
- data/lib/rubydb/server/connection.rb +366 -0
- data/lib/rubydb/server/connection_pool.rb +193 -0
- data/lib/rubydb/server/lifecycle.rb +228 -0
- data/lib/rubydb/server/listener.rb +136 -0
- data/lib/rubydb/server/request_handler.rb +276 -0
- data/lib/rubydb/server/server.rb +364 -0
- data/lib/rubydb/server/session.rb +369 -0
- data/lib/rubydb/server/worker.rb +210 -0
- data/lib/rubydb/server/worker_pool.rb +168 -0
- data/lib/rubydb/sql/ast/alter_table.rb +169 -0
- data/lib/rubydb/sql/ast/begin_transaction.rb +47 -0
- data/lib/rubydb/sql/ast/commit.rb +37 -0
- data/lib/rubydb/sql/ast/constraint.rb +83 -0
- data/lib/rubydb/sql/ast/create_database.rb +41 -0
- data/lib/rubydb/sql/ast/create_index.rb +61 -0
- data/lib/rubydb/sql/ast/create_schema.rb +52 -0
- data/lib/rubydb/sql/ast/create_table.rb +187 -0
- data/lib/rubydb/sql/ast/delete.rb +54 -0
- data/lib/rubydb/sql/ast/drop_database.rb +41 -0
- data/lib/rubydb/sql/ast/drop_index.rb +41 -0
- data/lib/rubydb/sql/ast/drop_schema.rb +49 -0
- data/lib/rubydb/sql/ast/drop_table.rb +49 -0
- data/lib/rubydb/sql/ast/explain.rb +64 -0
- data/lib/rubydb/sql/ast/expression.rb +604 -0
- data/lib/rubydb/sql/ast/insert.rb +82 -0
- data/lib/rubydb/sql/ast/node.rb +42 -0
- data/lib/rubydb/sql/ast/rollback.rb +63 -0
- data/lib/rubydb/sql/ast/savepoint.rb +59 -0
- data/lib/rubydb/sql/ast/select.rb +88 -0
- data/lib/rubydb/sql/ast/set_operation.rb +20 -0
- data/lib/rubydb/sql/ast/trigger.rb +29 -0
- data/lib/rubydb/sql/ast/update.rb +88 -0
- data/lib/rubydb/sql/ast/vacuum.rb +19 -0
- data/lib/rubydb/sql/ast/view.rb +32 -0
- data/lib/rubydb/sql/ast/with.rb +32 -0
- data/lib/rubydb/sql/grammar.rb +86 -0
- data/lib/rubydb/sql/keywords.rb +156 -0
- data/lib/rubydb/sql/lexer.rb +214 -0
- data/lib/rubydb/sql/operators.rb +100 -0
- data/lib/rubydb/sql/parser.rb +1185 -0
- data/lib/rubydb/sql/planner/analyzer.rb +302 -0
- data/lib/rubydb/sql/planner/binder.rb +550 -0
- data/lib/rubydb/sql/planner/type_checker.rb +431 -0
- data/lib/rubydb/sql/token.rb +210 -0
- data/lib/rubydb/storage/buffer_frame.rb +44 -0
- data/lib/rubydb/storage/buffer_pool.rb +155 -0
- data/lib/rubydb/storage/database_lock.rb +74 -0
- data/lib/rubydb/storage/deserializer.rb +342 -0
- data/lib/rubydb/storage/engine.rb +2338 -0
- data/lib/rubydb/storage/file_manager.rb +187 -0
- data/lib/rubydb/storage/free_space_map.rb +81 -0
- data/lib/rubydb/storage/page.rb +94 -0
- data/lib/rubydb/storage/page_allocator.rb +855 -0
- data/lib/rubydb/storage/page_header.rb +67 -0
- data/lib/rubydb/storage/page_manager.rb +131 -0
- data/lib/rubydb/storage/record.rb +58 -0
- data/lib/rubydb/storage/row.rb +78 -0
- data/lib/rubydb/storage/serializer.rb +51 -0
- data/lib/rubydb/storage/storage_layout.rb +151 -0
- data/lib/rubydb/storage/storage_manager.rb +114 -0
- data/lib/rubydb/storage/tuple.rb +461 -0
- data/lib/rubydb/storage/visibility_map.rb +984 -0
- data/lib/rubydb/transactions/commit_manager.rb +220 -0
- data/lib/rubydb/transactions/isolation.rb +98 -0
- data/lib/rubydb/transactions/lock.rb +76 -0
- data/lib/rubydb/transactions/lock_manager.rb +362 -0
- data/lib/rubydb/transactions/savepoint.rb +143 -0
- data/lib/rubydb/transactions/transaction.rb +215 -0
- data/lib/rubydb/transactions/transaction_id.rb +84 -0
- data/lib/rubydb/transactions/transaction_log.rb +257 -0
- data/lib/rubydb/transactions/transaction_manager.rb +435 -0
- data/lib/rubydb/types/bigint.rb +36 -0
- data/lib/rubydb/types/blob.rb +37 -0
- data/lib/rubydb/types/boolean.rb +34 -0
- data/lib/rubydb/types/date.rb +39 -0
- data/lib/rubydb/types/decimal.rb +48 -0
- data/lib/rubydb/types/float.rb +34 -0
- data/lib/rubydb/types/integer.rb +36 -0
- data/lib/rubydb/types/json.rb +41 -0
- data/lib/rubydb/types/null.rb +34 -0
- data/lib/rubydb/types/smallint.rb +36 -0
- data/lib/rubydb/types/text.rb +37 -0
- data/lib/rubydb/types/time.rb +46 -0
- data/lib/rubydb/types/timestamp.rb +39 -0
- data/lib/rubydb/types/type.rb +119 -0
- data/lib/rubydb/types/uuid.rb +47 -0
- data/lib/rubydb/types/varchar.rb +37 -0
- data/lib/rubydb/version.rb +33 -0
- data/lib/rubydb/wal/archive.rb +193 -0
- data/lib/rubydb/wal/checkpoint.rb +183 -0
- data/lib/rubydb/wal/lsn.rb +94 -0
- data/lib/rubydb/wal/reader.rb +260 -0
- data/lib/rubydb/wal/record.rb +105 -0
- data/lib/rubydb/wal/segment.rb +193 -0
- data/lib/rubydb/wal/wal.rb +452 -0
- data/lib/rubydb/wal/writer.rb +236 -0
- data/lib/rubydb.rb +7 -0
- data/packaging/docker/Dockerfile +36 -0
- data/packaging/docker/docker-compose.failover.yml +43 -0
- data/packaging/docker/entrypoint.sh +15 -0
- data/packaging/homebrew/rubydb.rb +19 -0
- data/packaging/systemd/rubydb.service +22 -0
- data/rubydb.gemspec +57 -0
- data/scripts/benchmark +7 -0
- data/scripts/durability_drill +37 -0
- data/scripts/format +0 -0
- data/scripts/fuzz +63 -0
- data/scripts/lint +0 -0
- data/scripts/release +50 -0
- data/scripts/release_check +43 -0
- data/scripts/replication_failover_drill +250 -0
- data/scripts/replication_network_failover_drill +255 -0
- data/scripts/restore_drill +45 -0
- data/scripts/security +0 -0
- data/scripts/setup +0 -0
- data/scripts/test +0 -0
- data/scripts/test-chaos +0 -0
- data/scripts/test-crash +0 -0
- data/scripts/test-integration +0 -0
- data/scripts/test-unit +0 -0
- data/visibility_map.json +1 -0
- metadata +538 -0
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
# Monitoring and alerting
|
|
2
|
+
|
|
3
|
+
Expose the RubyDB health and Prometheus endpoints through the deployment’s
|
|
4
|
+
authenticated monitoring path. Use liveness to decide whether a process
|
|
5
|
+
exists; use readiness to decide whether it is safe to route traffic.
|
|
6
|
+
|
|
7
|
+
At minimum, collect:
|
|
8
|
+
|
|
9
|
+
- request count, error count, latency, and active connections;
|
|
10
|
+
- WAL bytes/segments, checkpoint failures, flush failures, and recovery-required
|
|
11
|
+
commit acknowledgements;
|
|
12
|
+
- table/index counts, row counts, storage bytes, and vacuum/compaction duration;
|
|
13
|
+
- replication role, state, received/replayed LSN, lag, authentication failures,
|
|
14
|
+
and fencing epoch;
|
|
15
|
+
- process memory, CPU, open files, and filesystem free space.
|
|
16
|
+
|
|
17
|
+
Alert on readiness failure, repeated request errors, checkpoint or WAL flush
|
|
18
|
+
failures, any recovery-required acknowledgement, filesystem space below the
|
|
19
|
+
service threshold, replication lag beyond the application RPO, authentication
|
|
20
|
+
failures, and a fencing epoch change.
|
|
21
|
+
|
|
22
|
+
Dashboards must show rates and percentiles over time, not only current gauges.
|
|
23
|
+
Set thresholds from staging workload measurements and review them after every
|
|
24
|
+
capacity or schema change. A green liveness check alone is not evidence that
|
|
25
|
+
the database can accept traffic.
|
|
@@ -0,0 +1,295 @@
|
|
|
1
|
+
# RubyDB production operations guide
|
|
2
|
+
|
|
3
|
+
This guide describes a controlled production deployment for applications that
|
|
4
|
+
fit RubyDB’s documented feature surface. It is an operational companion to
|
|
5
|
+
the [production runbook](production-runbook.md), [disaster recovery guide](disaster-recovery.md),
|
|
6
|
+
and [monitoring guide](monitoring.md). It does not turn the current project
|
|
7
|
+
into a universal PostgreSQL, MySQL, or SQLite replacement.
|
|
8
|
+
|
|
9
|
+
## Deployment decision
|
|
10
|
+
|
|
11
|
+
Choose embedded mode only when one process owns the database directory and the
|
|
12
|
+
application accepts process-local availability. Use server mode when web,
|
|
13
|
+
worker, migration, or administrative processes need concurrent access. Put
|
|
14
|
+
the database directory on storage with documented durability and rename/
|
|
15
|
+
flush semantics. Do not place it on an untested shared filesystem.
|
|
16
|
+
|
|
17
|
+
Before launch, validate the application’s schema, generated SQL, migrations,
|
|
18
|
+
backup/restore path, concurrency profile, and failure behavior against the
|
|
19
|
+
exact RubyDB version and configuration that will be deployed.
|
|
20
|
+
|
|
21
|
+
## Reference topology
|
|
22
|
+
|
|
23
|
+
```text
|
|
24
|
+
clients/web/workers -> TLS -> RubyDB server -> private database volume
|
|
25
|
+
|-> WAL/checkpoints
|
|
26
|
+
|-> verified backup destination
|
|
27
|
+
|-> metrics/log sink
|
|
28
|
+
optional replica --------------------^
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
A replica is not a backup. A backup is not a fencing system. An application
|
|
32
|
+
load balancer health check is not proof that a primary is safe to write. Keep
|
|
33
|
+
these responsibilities separate.
|
|
34
|
+
|
|
35
|
+
## Complete first deployment
|
|
36
|
+
|
|
37
|
+
The following is a reference deployment for a Rails or regular Ruby
|
|
38
|
+
application. Replace paths, hostnames, users, and limits with values approved
|
|
39
|
+
by your infrastructure team.
|
|
40
|
+
|
|
41
|
+
### 1. Install and provision the server
|
|
42
|
+
|
|
43
|
+
Pin the RubyDB gem version on the database host. Do not use an unpinned
|
|
44
|
+
prerelease in a production deployment:
|
|
45
|
+
|
|
46
|
+
```sh
|
|
47
|
+
gem install rubydb -v 0.1.0
|
|
48
|
+
useradd --system --home-dir /var/lib/rubydb --shell /usr/sbin/nologin rubydb
|
|
49
|
+
install -d -o rubydb -g rubydb -m 0700 /var/lib/rubydb/data
|
|
50
|
+
install -d -o rubydb -g rubydb -m 0750 /var/log/rubydb
|
|
51
|
+
install -d -o root -g rubydb -m 0750 /etc/rubydb
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
Use a persistent local volume for `/var/lib/rubydb/data`. Keep backups on a
|
|
55
|
+
separate failure domain. The server account should not own application source,
|
|
56
|
+
certificate private keys, or unrelated host data.
|
|
57
|
+
|
|
58
|
+
### 2. Configure the server
|
|
59
|
+
|
|
60
|
+
Start from the repository’s `config/production.yml` and place a reviewed copy
|
|
61
|
+
at `/etc/rubydb/production.yml`. Supply required secrets through the service
|
|
62
|
+
manager or secret store. At minimum, configure:
|
|
63
|
+
|
|
64
|
+
```text
|
|
65
|
+
RUBYDB_HOST=0.0.0.0
|
|
66
|
+
RUBYDB_PORT=7432
|
|
67
|
+
RUBYDB_DATA_DIR=/var/lib/rubydb/data
|
|
68
|
+
RUBYDB_LOG_DIR=/var/log/rubydb
|
|
69
|
+
RUBYDB_USERNAME=app_rw
|
|
70
|
+
RUBYDB_PASSWORD=<secret-manager-value>
|
|
71
|
+
RUBYDB_SSL_ENABLED=true
|
|
72
|
+
RUBYDB_SSL_CERT_FILE=/etc/rubydb/tls/server.crt
|
|
73
|
+
RUBYDB_SSL_KEY_FILE=/etc/rubydb/tls/server.key
|
|
74
|
+
RUBYDB_SSL_CA_FILE=/etc/rubydb/tls/ca.crt
|
|
75
|
+
RUBYDB_SSL_VERIFY_PEER=true
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
The server’s TLS private key and application password must be readable only by
|
|
79
|
+
the service or secret-management mechanism. Restrict port `7432` to the Rails
|
|
80
|
+
and worker network; do not expose it to the public internet.
|
|
81
|
+
|
|
82
|
+
### 3. Start and verify the server
|
|
83
|
+
|
|
84
|
+
Run it under systemd, a supervised container, or an equivalent process
|
|
85
|
+
manager. The CLI configuration options must appear before the command:
|
|
86
|
+
|
|
87
|
+
```sh
|
|
88
|
+
sudo -u rubydb env RUBYDB_USERNAME=app_rw RUBYDB_PASSWORD='from-secret-store' \
|
|
89
|
+
rubydb --config /etc/rubydb/production.yml --env production start
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
In a systemd unit, use an `EnvironmentFile` protected with mode `0600` or a
|
|
93
|
+
native secret integration, then use:
|
|
94
|
+
|
|
95
|
+
```ini
|
|
96
|
+
ExecStart=/usr/local/bin/rubydb --config /etc/rubydb/production.yml --env production start
|
|
97
|
+
Restart=on-failure
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
Verify from the application network, not only from the database host:
|
|
101
|
+
|
|
102
|
+
```sh
|
|
103
|
+
rubydb --config /etc/rubydb/production.yml --env production status --json
|
|
104
|
+
rubydb --config /etc/rubydb/production.yml --env production doctor --json
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
Then run an authenticated TLS smoke query using the Ruby client and the same
|
|
108
|
+
URL the application will use:
|
|
109
|
+
|
|
110
|
+
```sh
|
|
111
|
+
RUBYDB_URL='rubydbs://app_rw:URL_ENCODED_PASSWORD@db.example.com:7432/app?verify_peer=true&ca_file=%2Fetc%2Frubydb%2Ftls%2Fca.crt' \
|
|
112
|
+
ruby -rrubydb -e 'c=RubyDB::Client::Client.new(url: ENV.fetch("RUBYDB_URL")); p c.query("SELECT 1").to_hash; c.disconnect'
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
### 4. Configure a Rails application
|
|
116
|
+
|
|
117
|
+
In the Rails application’s `config/database.yml`, select server mode and read
|
|
118
|
+
the connection string from the deployment environment:
|
|
119
|
+
|
|
120
|
+
```yaml
|
|
121
|
+
production:
|
|
122
|
+
adapter: rubydb
|
|
123
|
+
embedded: false
|
|
124
|
+
url: <%= ENV.fetch("RUBYDB_URL") %>
|
|
125
|
+
pool: <%= ENV.fetch("RAILS_MAX_THREADS", "5") %>
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
Set `RUBYDB_URL` through the platform secret store:
|
|
129
|
+
|
|
130
|
+
```text
|
|
131
|
+
RUBYDB_URL=rubydbs://app_rw:URL_ENCODED_PASSWORD@db.example.com:7432/app?verify_peer=true&ca_file=%2Fetc%2Frubydb%2Ftls%2Fca.crt
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
Deploy the Rails application, then run the migration once using a controlled
|
|
135
|
+
release job, not concurrently from every web process:
|
|
136
|
+
|
|
137
|
+
```sh
|
|
138
|
+
RAILS_ENV=production bundle exec rails db:migrate
|
|
139
|
+
RAILS_ENV=production bundle exec rails runner 'puts User.count'
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
The web and job processes connect to the server endpoint. They never mount or
|
|
143
|
+
open `/var/lib/rubydb/data`.
|
|
144
|
+
|
|
145
|
+
### 5. Configure a regular Ruby application
|
|
146
|
+
|
|
147
|
+
The regular client uses the same environment value:
|
|
148
|
+
|
|
149
|
+
```ruby
|
|
150
|
+
require "rubydb"
|
|
151
|
+
|
|
152
|
+
client = RubyDB::Client::Client.new(url: ENV.fetch("RUBYDB_URL"))
|
|
153
|
+
begin
|
|
154
|
+
client.query("SELECT 1")
|
|
155
|
+
client.query("INSERT INTO audit_events (event_name) VALUES (?)", ["boot"])
|
|
156
|
+
ensure
|
|
157
|
+
client.disconnect
|
|
158
|
+
end
|
|
159
|
+
```
|
|
160
|
+
|
|
161
|
+
Use application-level idempotency for retried writes. A network timeout does
|
|
162
|
+
not by itself prove that a commit was rolled back.
|
|
163
|
+
|
|
164
|
+
### 6. Accept traffic gradually
|
|
165
|
+
|
|
166
|
+
Run a read/write smoke test, migration status check, backup, and representative
|
|
167
|
+
workload before routing all traffic. Watch p95/p99 latency, errors, lock waits,
|
|
168
|
+
active connections, WAL growth, checkpoint age, disk space, and memory during a
|
|
169
|
+
canary period. Keep the previous application version and verified database
|
|
170
|
+
backup available until the rollback window closes.
|
|
171
|
+
|
|
172
|
+
The Rails URL example and URL option reference are also maintained in [Rails
|
|
173
|
+
database configuration](../rails/database-yml.md).
|
|
174
|
+
|
|
175
|
+
## Configuration and service identity
|
|
176
|
+
|
|
177
|
+
Run the server as a dedicated least-privilege account. Give it access only to
|
|
178
|
+
the database, WAL, temporary, certificate, and backup paths it needs. Store
|
|
179
|
+
passwords, peer tokens, private keys, and API credentials in a secret manager.
|
|
180
|
+
Do not put secrets in YAML committed to the repository, process arguments, or
|
|
181
|
+
logs.
|
|
182
|
+
|
|
183
|
+
Pin the RubyDB version and configuration for each deployment. Review changes to
|
|
184
|
+
durability mode, WAL retention, checkpoint thresholds, memory, connection
|
|
185
|
+
limits, request deadlines, lock timeouts, and TLS/authentication as production
|
|
186
|
+
changes. Keep a configuration checksum in the deployment record.
|
|
187
|
+
|
|
188
|
+
## Readiness checklist
|
|
189
|
+
|
|
190
|
+
Before accepting traffic:
|
|
191
|
+
|
|
192
|
+
* database directory is on approved storage with sufficient space and inodes;
|
|
193
|
+
* service account and file permissions are verified;
|
|
194
|
+
* TLS certificate, key, CA, hostname, and expiration are checked;
|
|
195
|
+
* authentication and authorization deny an unauthenticated test client;
|
|
196
|
+
* health and readiness checks exercise a real request path;
|
|
197
|
+
* connection, request, lock, and shutdown timeouts are bounded;
|
|
198
|
+
* schema/migrations have completed and version/checksum is recorded;
|
|
199
|
+
* full backup has been created and restored into a fresh directory;
|
|
200
|
+
* monitoring, alert routing, and log retention are active;
|
|
201
|
+
* rollback and restore owners are named; and
|
|
202
|
+
* a representative smoke query and write have passed.
|
|
203
|
+
|
|
204
|
+
## Capacity and resource limits
|
|
205
|
+
|
|
206
|
+
Set explicit limits for connections, request duration, lock waits, result size,
|
|
207
|
+
memory, worker count, WAL size, backup space, and file descriptors. Size the
|
|
208
|
+
application pool below the server limit, leaving room for migrations,
|
|
209
|
+
replication, health checks, and administration. A pool that equals the server
|
|
210
|
+
limit can starve the control plane.
|
|
211
|
+
|
|
212
|
+
Alert before exhaustion, not after it. Watch CPU, RSS, open files, disk bytes,
|
|
213
|
+
free inodes, WAL bytes, checkpoint age/duration, active transactions, lock
|
|
214
|
+
waits, pool utilization, errors, cancellations, and p95/p99 latency.
|
|
215
|
+
|
|
216
|
+
## Backup policy
|
|
217
|
+
|
|
218
|
+
Define RPO and RTO with the application owner. At minimum, maintain verified
|
|
219
|
+
full backups, protect them from the database host, encrypt them at rest and in
|
|
220
|
+
transit, retain multiple generations, and record manifests/checksums. If using
|
|
221
|
+
incremental or differential backups, retain their verified base and ordered
|
|
222
|
+
chain.
|
|
223
|
+
|
|
224
|
+
A successful backup command is not proof of recoverability. Regularly restore
|
|
225
|
+
to an isolated directory, validate checksums and schema, run representative
|
|
226
|
+
queries, compare critical row counts, and record elapsed restore time. Run a
|
|
227
|
+
restore drill after format, backup, storage, or release changes.
|
|
228
|
+
|
|
229
|
+
## Upgrade procedure
|
|
230
|
+
|
|
231
|
+
1. Read the release notes, format compatibility, and migration notes.
|
|
232
|
+
2. Create and verify a new full backup.
|
|
233
|
+
3. Test the new version against a restored production-like copy.
|
|
234
|
+
4. Run schema and application smoke tests, including populated-table writes.
|
|
235
|
+
5. Drain or fence writes according to the deployment topology.
|
|
236
|
+
6. Upgrade one controlled instance and verify health, WAL, and metrics.
|
|
237
|
+
7. Re-enable traffic gradually and watch errors and latency.
|
|
238
|
+
8. Keep the rollback binary and backup available until the validation window
|
|
239
|
+
closes.
|
|
240
|
+
|
|
241
|
+
Never roll back by pointing an older binary at a directory whose format or
|
|
242
|
+
metadata it cannot read. Use the documented restore/rollback path.
|
|
243
|
+
|
|
244
|
+
## Failover procedure
|
|
245
|
+
|
|
246
|
+
RubyDB’s safe failover model requires a synchronized candidate and a durable
|
|
247
|
+
fencing decision. A manual operator sequence is:
|
|
248
|
+
|
|
249
|
+
1. declare the incident and stop or isolate application writes;
|
|
250
|
+
2. verify the primary’s last acknowledged LSN and fence epoch;
|
|
251
|
+
3. confirm the candidate’s applied LSN and integrity;
|
|
252
|
+
4. fence the old primary at the process, host, storage, or network layer;
|
|
253
|
+
5. promote only after fencing is observable and durable;
|
|
254
|
+
6. point clients at the new primary and run smoke writes/reads;
|
|
255
|
+
7. monitor replication and stale-writer rejection; and
|
|
256
|
+
8. recover the old primary as a replica only after its state is understood.
|
|
257
|
+
|
|
258
|
+
Automatic election requires an independently validated quorum, fencing,
|
|
259
|
+
partition behavior, stale-primary rejection, and recovery procedure. Do not
|
|
260
|
+
enable election based solely on a successful same-host test.
|
|
261
|
+
|
|
262
|
+
## Incident response
|
|
263
|
+
|
|
264
|
+
Contain first: stop unsafe writes, protect the database directory, and record
|
|
265
|
+
the timeline. Preserve logs, metrics, WAL, metadata, configuration, process
|
|
266
|
+
state, and backup manifests. Use [troubleshooting](../troubleshooting.md) and
|
|
267
|
+
[debugging](../debugging.md) for evidence collection.
|
|
268
|
+
|
|
269
|
+
Classify the event as availability, durability, correctness, security, or
|
|
270
|
+
capacity. Assign an incident owner and a recovery owner. Communicate whether
|
|
271
|
+
commit outcomes are known, unknown, or confirmed rolled back. After recovery,
|
|
272
|
+
verify application invariants rather than relying only on process health.
|
|
273
|
+
|
|
274
|
+
## TLS and secret rotation
|
|
275
|
+
|
|
276
|
+
Stage new certificates and CA material, validate the chain and hostname with a
|
|
277
|
+
test client, then switch through an atomic deployment/configuration change.
|
|
278
|
+
Maintain an overlap window only if clients support it. Confirm old material is
|
|
279
|
+
no longer accepted before revoking it. Rotate database passwords and peer
|
|
280
|
+
tokens through the secret manager; audit access and avoid printing values.
|
|
281
|
+
|
|
282
|
+
## Maintenance
|
|
283
|
+
|
|
284
|
+
Schedule vacuum, compaction, checkpoint, index maintenance, and backups with
|
|
285
|
+
awareness of active readers and write load. Measure before and after. Use a
|
|
286
|
+
copy for repair or compaction experiments. Verify reopen, checksums, row counts,
|
|
287
|
+
indexes, and application smoke queries after maintenance.
|
|
288
|
+
|
|
289
|
+
## Production evidence
|
|
290
|
+
|
|
291
|
+
The release record should contain the RubyDB commit, Ruby/Rails versions,
|
|
292
|
+
configuration checksum, schema/migration version, backup manifest, restore
|
|
293
|
+
drill result, benchmark/soak result, monitoring link, security review status,
|
|
294
|
+
and known limitations. See [production readiness](../production-readiness.md)
|
|
295
|
+
for the project-level boundaries.
|
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
# RubyDB production runbook
|
|
2
|
+
|
|
3
|
+
## Before deployment
|
|
4
|
+
|
|
5
|
+
1. Pin the RubyDB commit and Ruby version. Run `bundle exec rspec`, the
|
|
6
|
+
production soak, the durability drill, and the network failover drill.
|
|
7
|
+
2. Place the database, WAL, lock file, replication state, and backups on
|
|
8
|
+
protected storage owned by the service account. Keep backups on a separate
|
|
9
|
+
failure domain.
|
|
10
|
+
3. Enable TLS, use a private bind address, configure authentication and a
|
|
11
|
+
replication token from a secret manager, and set connection, frame, query,
|
|
12
|
+
worker, and disk-space limits.
|
|
13
|
+
4. Install the alert rules in
|
|
14
|
+
`config/monitoring/prometheus-alerts.yml`. Test readiness, metrics, backup,
|
|
15
|
+
and restore from the deployment environment.
|
|
16
|
+
|
|
17
|
+
## Backup and restore
|
|
18
|
+
|
|
19
|
+
Run a full backup before upgrades and retain the matching WAL chain and
|
|
20
|
+
manifest. Restore into a new inactive directory, run the restore drill, reopen
|
|
21
|
+
the database, compare row counts/checksums, and only then switch traffic.
|
|
22
|
+
Never overwrite the only copy during a restore.
|
|
23
|
+
|
|
24
|
+
## Failover
|
|
25
|
+
|
|
26
|
+
Stop writes, confirm the replica is synchronized, fence the old primary using
|
|
27
|
+
the shared durable fence path, promote exactly one replica, and verify client
|
|
28
|
+
writes and replication status. Automatic election is not enabled until the
|
|
29
|
+
deployment provides an independently tested fencing lease. A network partition
|
|
30
|
+
must never result in two writable primaries.
|
|
31
|
+
|
|
32
|
+
## TLS and secret rotation
|
|
33
|
+
|
|
34
|
+
Stage a new certificate/key and replication token in the secret manager,
|
|
35
|
+
validate them on an inactive instance, restart or drain one peer at a time,
|
|
36
|
+
verify TLS and replication authentication, then revoke the old credentials
|
|
37
|
+
after the overlap window. Do not place secrets in repository files, command
|
|
38
|
+
arguments, URLs, or logs.
|
|
39
|
+
|
|
40
|
+
## Upgrade and incident response
|
|
41
|
+
|
|
42
|
+
Run the upgrade-compatibility suite against a copy of production backups.
|
|
43
|
+
Preserve the database, WAL, replication state, logs, metrics, and timestamps
|
|
44
|
+
before repair, vacuum, restore, or promotion. If checksums, fencing, or
|
|
45
|
+
replication diverge, stop writes and escalate; do not guess at a repair.
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
# Replication operations
|
|
2
|
+
|
|
3
|
+
RubyDB replication is logical WAL row-mutation replication. Configure the
|
|
4
|
+
primary and replica with separate data directories, explicit network addresses,
|
|
5
|
+
and the same high-entropy `replication_auth_token`. Restrict the listener with
|
|
6
|
+
firewall rules and use TLS or a trusted private network; the token is peer
|
|
7
|
+
authentication, not a replacement for transport encryption.
|
|
8
|
+
|
|
9
|
+
## Bootstrap and catch-up
|
|
10
|
+
|
|
11
|
+
Start the primary first, then start the replica with its replica identifier and
|
|
12
|
+
primary address. The primary sends catalog schema before row replay. Verify the
|
|
13
|
+
replica reaches `SYNCED`, and compare received and replayed LSNs before routing
|
|
14
|
+
reads to it.
|
|
15
|
+
|
|
16
|
+
Monitor replication status continuously. A replica with a received/replayed
|
|
17
|
+
gap is not eligible for promotion. Preserve the replication log and replica
|
|
18
|
+
state files during an incident; do not manually edit LSN state.
|
|
19
|
+
|
|
20
|
+
## Token rotation
|
|
21
|
+
|
|
22
|
+
Drain replication, stop or disconnect the replica, update the protected token
|
|
23
|
+
on both sides, restart the primary listener if required, and reconnect the
|
|
24
|
+
replica. Confirm accepted handshakes and zero unexpected authentication
|
|
25
|
+
failures before returning the replica to service. Never place tokens in source
|
|
26
|
+
control or command histories.
|
|
27
|
+
|
|
28
|
+
## Failure handling
|
|
29
|
+
|
|
30
|
+
On a network partition, assume the old primary may still be reachable until it
|
|
31
|
+
is fenced. Do not promote a lagging replica and do not run two writable primaries
|
|
32
|
+
without a verified fencing lease. Re-bootstrap a divergent replica rather than
|
|
33
|
+
guessing which WAL records are safe to apply.
|
|
@@ -0,0 +1,6 @@
|
|
|
1
|
+
## Restore verification
|
|
2
|
+
|
|
3
|
+
Before treating a backup as recoverable, run the verification path that extracts
|
|
4
|
+
it into an isolated temporary directory, opens the database with RubyDB, and
|
|
5
|
+
compares the restored catalog and table row counts with the manifest. The
|
|
6
|
+
verification removes its temporary database after the check.
|
|
@@ -0,0 +1,34 @@
|
|
|
1
|
+
# RubyDB operator runbook
|
|
2
|
+
|
|
3
|
+
## Start and health check
|
|
4
|
+
|
|
5
|
+
Run the server with an explicit production configuration:
|
|
6
|
+
|
|
7
|
+
```sh
|
|
8
|
+
RUBYDB_ENV=production rubydb -c config/production.yml start
|
|
9
|
+
```
|
|
10
|
+
|
|
11
|
+
Confirm the process is listening, then run `rubydb status --json`. Treat a missing PID file, an invalid configuration, or a failed health check as a failed deployment; do not bypass startup validation.
|
|
12
|
+
|
|
13
|
+
## Backup and restore
|
|
14
|
+
|
|
15
|
+
Create a verified full backup before upgrades or maintenance. Retain the manifest and WAL chain together. Test restoration into a separate directory before replacing a live data directory. Never restore over an active database.
|
|
16
|
+
|
|
17
|
+
## Upgrade and rollback
|
|
18
|
+
|
|
19
|
+
Stop writes, create and verify a full backup, restore it into a staging directory, and run migration status plus application smoke queries. Promote the staged directory only after validation. Roll back by stopping RubyDB and switching back to the retained pre-upgrade directory; preserve the failed directory for diagnosis.
|
|
20
|
+
|
|
21
|
+
## Incident boundaries
|
|
22
|
+
|
|
23
|
+
On corruption, checksum failure, repeated crash recovery, or replication divergence, stop writes and preserve the data directory and logs. Do not delete WAL, run vacuum, or force promotion until a verified backup and incident copy exist. Automatic failover is intentionally disabled and promotion requires an operator decision.
|
|
24
|
+
|
|
25
|
+
## Security operations
|
|
26
|
+
|
|
27
|
+
Keep production credentials and TLS private keys outside the repository with restrictive permissions. Rotate certificates by staging the replacement, validating the chain, and restarting during a planned window. Keep TLS enabled and use peer verification when a trusted CA is available.
|
|
28
|
+
|
|
29
|
+
For replication, configure the same high-entropy `replication_auth_token` on
|
|
30
|
+
each primary and replica. A peer with a missing or incorrect token is rejected;
|
|
31
|
+
rotate the token by draining replicas, updating the protected configuration on
|
|
32
|
+
both sides, and reconnecting them. The replication listener is not a substitute
|
|
33
|
+
for a private network or TLS termination, so restrict its port at the firewall
|
|
34
|
+
and validate certificate/private-network controls during deployment.
|
|
@@ -0,0 +1,14 @@
|
|
|
1
|
+
# Upgrades
|
|
2
|
+
|
|
3
|
+
RubyDB upgrades must be treated as storage migrations, not as an in-place binary replacement.
|
|
4
|
+
|
|
5
|
+
Before upgrading:
|
|
6
|
+
|
|
7
|
+
1. Stop writes and confirm the server is healthy.
|
|
8
|
+
2. Create and verify a full backup, including WAL when available.
|
|
9
|
+
3. Record the RubyDB version, storage page size, and backup manifest checksum.
|
|
10
|
+
4. Test the new release against a restored copy of the backup.
|
|
11
|
+
|
|
12
|
+
The current storage format is version 1. RubyDB refuses to open an unknown page format or a database whose page size differs from the configured page size. There is no automatic on-disk format conversion yet; a future incompatible format must provide an explicit export/import migration.
|
|
13
|
+
|
|
14
|
+
After upgrading, reopen the restored or migrated database, run the migration status command, verify application queries, and retain the pre-upgrade backup until the rollback window closes. Never delete the original data directory as part of an upgrade.
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
# Workload testing
|
|
2
|
+
|
|
3
|
+
Run the deterministic concurrent workload from the repository root:
|
|
4
|
+
|
|
5
|
+
```sh
|
|
6
|
+
RUBYDB_WORKLOAD_THREADS=4 RUBYDB_WORKLOAD_OPERATIONS=250 ruby benchmarks/concurrent_workload.rb
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
It performs concurrent inserts against one engine, verifies the final row count, closes the database, reopens it, and verifies durable reads. It prints one JSON result and exits non-zero on any operation or durability failure.
|
|
10
|
+
|
|
11
|
+
Use a temporary database by default. To retain an artifact for inspection, provide an explicit path:
|
|
12
|
+
|
|
13
|
+
```sh
|
|
14
|
+
RUBYDB_WORKLOAD_PATH=/var/lib/rubydb/workload.rdb RUBYDB_WORKLOAD_THREADS=16 RUBYDB_WORKLOAD_OPERATIONS=10000 ruby benchmarks/concurrent_workload.rb
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
Run this in a staging environment on the target filesystem before deployment. Establish acceptable latency, throughput, WAL growth, disk-space, recovery-time, and backup/restore targets for the expected workload; do not treat a successful smoke workload as capacity certification.
|
|
@@ -0,0 +1,118 @@
|
|
|
1
|
+
# Production-readiness audit
|
|
2
|
+
|
|
3
|
+
## Status
|
|
4
|
+
|
|
5
|
+
RubyDB has verified production-oriented foundations, but it is not yet a general-purpose production database. Supported behavior is backed by the RSpec suite in `spec/`.
|
|
6
|
+
|
|
7
|
+
Latest local audit: 270 examples, 0 failures. Hosted CI and deployment-specific
|
|
8
|
+
evidence remain separate release gates.
|
|
9
|
+
|
|
10
|
+
The feature inventory below was written during an earlier checkpoint and its
|
|
11
|
+
parenthetical historical count is not the current total. Use the latest audit
|
|
12
|
+
line above and rerun the suite for release evidence.
|
|
13
|
+
|
|
14
|
+
## Implemented features
|
|
15
|
+
|
|
16
|
+
The repository contains substantial scaffolding for:
|
|
17
|
+
|
|
18
|
+
- page-based storage
|
|
19
|
+
- buffer pool caching
|
|
20
|
+
- file and page management
|
|
21
|
+
- WAL, recovery, and checkpoint modules
|
|
22
|
+
- transactions and MVCC abstractions
|
|
23
|
+
- SQL parser/planner infrastructure
|
|
24
|
+
- server, protocol, and connection components
|
|
25
|
+
- authentication and authorization modules
|
|
26
|
+
- backup and replication APIs
|
|
27
|
+
- monitoring and metrics interfaces
|
|
28
|
+
- Rails integration adapters
|
|
29
|
+
- ordered, locked migrations with durable version/checksum tracking
|
|
30
|
+
|
|
31
|
+
## Tested features
|
|
32
|
+
|
|
33
|
+
The current suite verifies storage reopen, subprocess crash recovery, MVCC isolation/vacuum, durable visibility version-history traversal, constraints including ON DELETE/ON UPDATE referential actions and nullable values, indexes including deep B-tree splits, SQL execution including idempotent database/table/index DDL and SQL foreign-key actions, schema DDL, views, trigger DDL and dispatch, CREATE TABLE, ALTER TABLE column and constraint changes, transaction before-image rollback and SQL savepoints, and VACUUM, live TCP sessions, TLS transport, password/SCRAM authentication and authorization, metrics updates including Prometheus export, liveness/readiness health reporting through the server request router, live CLI doctor checks with safe repair behavior, truthful CLI status reporting, live CLI branch diff/merge/checkout operations, live CLI inspection values, live CLI snapshot creation/listing and vacuum reporting, live CLI backup creation and restore dry-run validation, live CLI database creation and deletion, the documented SQL compatibility contract, full/snapshot/incremental/differential backup validation, logical replication, durable replica state, guarded failover promotion, durable fencing epochs, replication peer-token authentication, recovery resource checks, migration/schema-diff behavior including executable migration SQL serialization and changed-migration detection, atomic branch checkout, branch state/diff/merge behavior, foreign-key integrity lookup behavior, compound/null/boolean check-constraint evaluation, lock conflict/wait/timeout behavior, complete standalone and transaction-manager deadlock cycle detection with victim rollback, concurrency mutex initialization, Rails schema-builder SQL generation including defaults and foreign-key conventions, engine block-based table creation, adapter schema dumps preserving primary keys and defaults, ActiveRecord hash-form ordering and populated-table migration round trips, bounded multi-process workload supervision with child reaping, combined production traffic/deadline/cancellation/capacity/deadlock soak validation, release configuration, release tag/changelog preflight, upgrade guards, benchmark execution, executor query deadlines, wire-level cancellation of in-flight requests, standalone SCRAM verification, actual WAL checkpoint sizing, typed deserialization-corruption detection, preservation of stored false values when defaults are present, filesystem fault injection with descriptor cleanup, compaction/reopen validation, process-level replication failover with stale-writer fencing, and live TCP network-partition catch-up and promotion (268 examples, 0 failures at the latest audit).
|
|
34
|
+
|
|
35
|
+
## Known limitations
|
|
36
|
+
|
|
37
|
+
- The concurrent workload and regression suite verify parallel appends, point
|
|
38
|
+
reads, scans, durable reopen reads, multi-process server clients, and a
|
|
39
|
+
process-level replication failover drill. This does not certify universal
|
|
40
|
+
capacity; embedded ownership remains exclusive and multi-process clients must
|
|
41
|
+
use the server.
|
|
42
|
+
|
|
43
|
+
- multi-row SQL `VALUES` inserts execute through the parser, binder, planner,
|
|
44
|
+
and executor, including per-row conflict handling and Rails insert IDs
|
|
45
|
+
- incremental backups capture WAL mutations after a verified base LSN; differential backups capture the verified base-relative WAL delta and restore through the same validated delta path
|
|
46
|
+
- replication is limited to the explicit logical row-mutation envelope API;
|
|
47
|
+
process replacement, live TCP partition/catch-up, and same-host fencing are
|
|
48
|
+
tested, while true multi-host partitions and split-brain recovery remain
|
|
49
|
+
deployment work
|
|
50
|
+
- automatic failover is limited to synchronized candidates; fencing requires a shared durable fence path and still needs multi-host split-brain validation
|
|
51
|
+
- the SQLite compatibility profile covers the documented common application
|
|
52
|
+
surface, but it is not SQLite file-format or extension compatibility
|
|
53
|
+
- production CA lifecycle, independent security review, performance targets,
|
|
54
|
+
and large-scale workload behavior still require dedicated validation
|
|
55
|
+
- branch state application and live target-branch merges are supported through the engine reconciliation hook
|
|
56
|
+
|
|
57
|
+
## Unsupported SQL
|
|
58
|
+
|
|
59
|
+
The supported dialect is defined in [docs/sql/compatibility.md](sql/compatibility.md) and enforced by parser/execution integration coverage. RubyDB provides a tested common SQLite-style profile, not complete SQLite file-format, extension, pragma, or error compatibility.
|
|
60
|
+
|
|
61
|
+
## Durability guarantees
|
|
62
|
+
|
|
63
|
+
WAL framing, reopen behavior, subprocess crash-recovery scenarios, actual checkpoint sizing, fail-closed visibility-map loading, filesystem fault injection, compaction/reopen behavior, networked replication failover, and verified restore drills are tested. This is not a substitute for real quota, power-loss, and filesystem-specific validation.
|
|
64
|
+
|
|
65
|
+
## Transaction guarantees
|
|
66
|
+
|
|
67
|
+
Transaction commit/rollback, MVCC visibility, snapshot isolation behavior, and vacuum safe-point handling are covered by focused tests. High-contention workload validation remains outstanding.
|
|
68
|
+
|
|
69
|
+
## Isolation guarantees
|
|
70
|
+
|
|
71
|
+
Read committed, repeatable read, and serializable conflict behavior are covered by focused tests; broader workload and distributed-isolation validation remains outstanding.
|
|
72
|
+
|
|
73
|
+
## Backup guarantees
|
|
74
|
+
|
|
75
|
+
Full compressed backups restore into a fresh directory and checksum tampering is detected. Incremental and differential backups validate the base, checksum the change set, and apply WAL row mutations during restore. Production-scale backup-chain and filesystem fault-injection testing remains outstanding.
|
|
76
|
+
|
|
77
|
+
## Replication guarantees
|
|
78
|
+
|
|
79
|
+
Logical row-mutation streaming, LSN deduplication, acknowledgments, acknowledged-LSN reporting, synchronized-candidate promotion verification, process replacement, live TCP partition/catch-up, and stale-primary rejection through durable fencing epochs are tested. Multi-host fencing and split-brain recovery still require dedicated validation.
|
|
80
|
+
|
|
81
|
+
## TLS guarantees
|
|
82
|
+
|
|
83
|
+
TLS 1.2 or newer is enforced when enabled. Certificate/key parsing, CA-file validation, TLS client connections, and the end-to-end SSL transport path are tested. Mutual TLS is configurable through client certificates; certificate rotation and production CA lifecycle procedures remain operational work.
|
|
84
|
+
|
|
85
|
+
## Security model
|
|
86
|
+
|
|
87
|
+
Password and SCRAM-SHA-256 authentication, server-signature verification, read/write authorization, and startup validation of incomplete auth configuration are tested through the server/session path.
|
|
88
|
+
|
|
89
|
+
## Performance characteristics
|
|
90
|
+
|
|
91
|
+
The repository includes a deterministic storage benchmark (`RUBYDB_BENCHMARK_ITERATIONS=100 ruby -Ilib benchmarks/basic_workload.rb`) and a concurrent write/durability workload (`ruby benchmarks/concurrent_workload.rb`). Both emit machine-readable JSON. Production-scale throughput, WAL, checkpoint, and concurrency targets still require workload-specific baselines.
|
|
92
|
+
|
|
93
|
+
## Deployment requirements
|
|
94
|
+
|
|
95
|
+
The project currently needs:
|
|
96
|
+
|
|
97
|
+
- a validated Ruby support matrix
|
|
98
|
+
- RubyGems publication credentials (`RUBYGEMS_API_KEY`) configured as a protected
|
|
99
|
+
GitHub Actions secret before pushing a release tag
|
|
100
|
+
- a documented operator runbook for backups, upgrades, rollback, and incident preservation
|
|
101
|
+
- explicit durability and crash-recovery validation
|
|
102
|
+
- secure-by-default configuration development
|
|
103
|
+
- a narrower set of supported features documented honestly
|
|
104
|
+
|
|
105
|
+
## Remaining roadmap items
|
|
106
|
+
|
|
107
|
+
Main remaining work is to complete the sequence laid out in the repository design:
|
|
108
|
+
|
|
109
|
+
1. production CA lifecycle and certificate rotation
|
|
110
|
+
2. broader SQL compatibility contract and workload benchmarks
|
|
111
|
+
3. production-grade failover fencing and broader SQL compatibility
|
|
112
|
+
4. operator runbooks and deployment validation
|
|
113
|
+
|
|
114
|
+
## Bottom line
|
|
115
|
+
|
|
116
|
+
RubyDB has a validated production-oriented foundation, but it does not yet meet
|
|
117
|
+
the bar for unrestricted production use with real data outside its documented
|
|
118
|
+
feature set and deployment-specific validation.
|