rubydb 0.1.4 → 0.1.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.github/PULL_REQUEST_TEMPLATE.md +15 -15
- data/.github/workflows/benchmark.yml +26 -26
- data/.github/workflows/compatibility.yml +63 -63
- data/.github/workflows/fuzz.yml +33 -33
- data/.github/workflows/lint.yml +21 -21
- data/.github/workflows/operations.yml +24 -24
- data/.github/workflows/production-validation.yml +111 -111
- data/.github/workflows/release.yml +77 -77
- data/.github/workflows/security.yml +39 -37
- data/.github/workflows/test.yml +26 -26
- data/.github/workflows/workload.yml +58 -58
- data/.gitignore +16 -5
- data/.rubocop.yml +50 -44
- data/.standard.yml +9 -14
- data/ARCHITECTURE.md +21 -21
- data/CHANGELOG.md +57 -27
- data/CODE_OF_CONDUCT.md +13 -13
- data/CONTRIBUTING.md +29 -29
- data/GOVERNANCE.md +16 -16
- data/Gemfile +18 -17
- data/Gemfile.lock +125 -71
- data/README.md +168 -12
- data/ROADMAP.md +27 -27
- data/Rakefile +76 -71
- data/SECURITY.md +54 -54
- data/SUPPORT.md +14 -14
- data/accelerator/bin/SHA256SUMS +6 -0
- data/accelerator/bin/rubydb-accelerator-darwin-amd64 +0 -0
- data/accelerator/bin/rubydb-accelerator-darwin-arm64 +0 -0
- data/accelerator/bin/rubydb-accelerator-linux-amd64 +0 -0
- data/accelerator/bin/rubydb-accelerator-linux-arm64 +0 -0
- data/accelerator/bin/rubydb-accelerator-windows-amd64.exe +0 -0
- data/accelerator/bin/rubydb-accelerator-windows-arm64.exe +0 -0
- data/accelerator/cmd/rubydb-accelerator/main.go +11 -0
- data/accelerator/go.mod +3 -0
- data/accelerator/internal/execution/aggregate.go +94 -0
- data/accelerator/internal/execution/distinct.go +22 -0
- data/accelerator/internal/execution/filter.go +73 -0
- data/accelerator/internal/execution/join.go +79 -0
- data/accelerator/internal/execution/operators.go +167 -0
- data/accelerator/internal/execution/scan.go +20 -0
- data/accelerator/internal/execution/sort.go +62 -0
- data/accelerator/internal/execution/types.go +136 -0
- data/accelerator/internal/execution/value.go +67 -0
- data/accelerator/internal/memory/arena.go +47 -0
- data/accelerator/internal/memory/reuse.go +22 -0
- data/accelerator/internal/metrics/registry.go +67 -0
- data/accelerator/internal/parallel/bounded_queue.go +56 -0
- data/accelerator/internal/parallel/scheduler.go +47 -0
- data/accelerator/internal/parallel/worker_pool.go +53 -0
- data/accelerator/internal/protocol/cancellation.go +48 -0
- data/accelerator/internal/protocol/columnar.go +263 -0
- data/accelerator/internal/protocol/frame.go +187 -0
- data/accelerator/internal/runtime/worker.go +521 -0
- data/accelerator/internal/storage/page_reader.go +81 -0
- data/accelerator/internal/storage/snapshot_scan.go +539 -0
- data/accelerator/internal/wal/checksum.go +13 -0
- data/accelerator/internal/wal/compression.go +41 -0
- data/accelerator/internal/wal/group_commit.go +24 -0
- data/accelerator/internal/wal/record_encoder.go +40 -0
- data/adapters/activerecord/Gemfile +11 -11
- data/adapters/activerecord/README.md +8 -3
- data/adapters/activerecord/lib/active_record/connection_adapters/rubydb_adapter.rb +881 -879
- data/adapters/activerecord/rubydb-activerecord.gemspec +21 -21
- data/adapters/activerecord/spec/rubydb_adapter_integration_spec.rb +143 -143
- data/adapters/ruby/README.md +18 -18
- data/adapters/sequel/README.md +11 -11
- data/config/monitoring/prometheus-alerts.yml +39 -39
- data/config/production.yml +36 -36
- data/docs/README.md +77 -71
- data/docs/architecture/concurrency.md +14 -14
- data/docs/architecture/current-state.md +125 -125
- data/docs/architecture/execution-engine.md +25 -25
- data/docs/architecture/go-accelerator.md +179 -0
- data/docs/architecture/indexes.md +19 -19
- data/docs/architecture/mvcc.md +19 -19
- data/docs/architecture/overview.md +13 -13
- data/docs/architecture/pages.md +11 -11
- data/docs/architecture/production-roadmap.md +82 -82
- data/docs/architecture/query-planner.md +20 -20
- data/docs/architecture/recovery.md +18 -18
- data/docs/architecture/sql-engine.md +12 -12
- data/docs/architecture/storage-engine.md +14 -14
- data/docs/architecture/transactions.md +10 -10
- data/docs/architecture/wal.md +28 -28
- data/docs/cli-cheatsheet.md +98 -98
- data/docs/cli.md +299 -275
- data/docs/contributing/architecture.md +9 -9
- data/docs/contributing/benchmarking.md +30 -14
- data/docs/contributing/development.md +16 -16
- data/docs/contributing/release-process.md +49 -49
- data/docs/contributing/testing.md +16 -16
- data/docs/debugging.md +229 -229
- data/docs/developer/branching.md +10 -10
- data/docs/developer/database-diff.md +10 -10
- data/docs/developer/local-development.md +49 -17
- data/docs/developer/snapshots.md +9 -9
- data/docs/developer/temporal-data.md +10 -10
- data/docs/developer-guide.md +297 -297
- data/docs/getting-started/first-database.md +16 -16
- data/docs/getting-started/first-query.md +13 -13
- data/docs/getting-started/installation.md +19 -19
- data/docs/getting-started/local-to-production.md +300 -300
- data/docs/getting-started/quickstart.md +17 -17
- data/docs/getting-started/rails.md +16 -16
- data/docs/hardening_backlog.md +93 -93
- data/docs/lessons-learned.md +112 -112
- data/docs/operations/backups.md +33 -33
- data/docs/operations/disaster-recovery.md +31 -31
- data/docs/operations/failover.md +30 -30
- data/docs/operations/monitoring.md +25 -25
- data/docs/operations/production-guide.md +295 -295
- data/docs/operations/production-runbook.md +45 -45
- data/docs/operations/replication.md +33 -33
- data/docs/operations/restore.md +6 -6
- data/docs/operations/runbook.md +34 -34
- data/docs/operations/upgrades.md +14 -14
- data/docs/operations/workload-testing.md +17 -17
- data/docs/production-readiness.md +118 -118
- data/docs/production_validation.md +150 -150
- data/docs/rails/active-record.md +11 -11
- data/docs/rails/compatibility-guide.md +90 -90
- data/docs/rails/database-yml.md +92 -92
- data/docs/rails/installation.md +17 -17
- data/docs/rails/migrations.md +17 -17
- data/docs/rails/production.md +82 -82
- data/docs/rails/troubleshooting.md +18 -18
- data/docs/release.md +25 -25
- data/docs/server/architecture.md +10 -10
- data/docs/server/authentication.md +10 -10
- data/docs/server/configuration.md +16 -16
- data/docs/server/connection-pooling.md +10 -10
- data/docs/server/deployment.md +10 -10
- data/docs/server/protocol.md +12 -12
- data/docs/sql/compatibility-guide.md +82 -82
- data/docs/sql/compatibility.md +39 -39
- data/docs/sql/data-types.md +10 -10
- data/docs/sql/functions.md +9 -9
- data/docs/sql/joins.md +9 -9
- data/docs/sql/operators.md +9 -9
- data/docs/sql/sqlite-compatibility.md +21 -21
- data/docs/sql/syntax.md +10 -10
- data/docs/sql/transactions.md +10 -10
- data/docs/troubleshooting.md +244 -244
- data/lessons/01-foundations.md +73 -0
- data/lessons/02-local-development.md +121 -0
- data/lessons/03-embedded-rubydb.md +99 -0
- data/lessons/04-rails-complex-apps.md +138 -0
- data/lessons/05-rubydb-production-server.md +237 -0
- data/lessons/06-postgresql-massive-apps.md +96 -0
- data/lessons/07-hybrid-microservices.md +179 -0
- data/lessons/08-migrations-backups-recovery.md +86 -0
- data/lessons/09-observability-security-scale.md +87 -0
- data/lessons/10-release-readiness.md +192 -0
- data/lessons/11-community-adapter.md +323 -0
- data/lessons/12-rails-ecommerce-pressure.md +263 -0
- data/lib/rubydb/accelerator/client.rb +451 -0
- data/lib/rubydb/accelerator/error.rb +22 -0
- data/lib/rubydb/accelerator/manager.rb +606 -0
- data/lib/rubydb/accelerator.rb +13 -0
- data/lib/rubydb/backup/archive.rb +332 -334
- data/lib/rubydb/backup/backup.rb +400 -401
- data/lib/rubydb/backup/incremental.rb +349 -353
- data/lib/rubydb/backup/restore.rb +289 -290
- data/lib/rubydb/backup/snapshot.rb +265 -267
- data/lib/rubydb/backup/verification.rb +276 -279
- data/lib/rubydb/branching/branch.rb +181 -181
- data/lib/rubydb/branching/branch_manager.rb +307 -311
- data/lib/rubydb/branching/branch_metadata.rb +140 -140
- data/lib/rubydb/branching/checkout.rb +165 -166
- data/lib/rubydb/branching/copy_on_write.rb +272 -272
- data/lib/rubydb/branching/diff.rb +137 -138
- data/lib/rubydb/branching/merge.rb +282 -285
- data/lib/rubydb/build_info.rb +15 -15
- data/lib/rubydb/catalog/catalog.rb +391 -391
- data/lib/rubydb/catalog/column.rb +112 -112
- data/lib/rubydb/catalog/constraint.rb +180 -180
- data/lib/rubydb/catalog/database.rb +184 -184
- data/lib/rubydb/catalog/index.rb +97 -97
- data/lib/rubydb/catalog/schema.rb +103 -103
- data/lib/rubydb/catalog/sequence.rb +90 -90
- data/lib/rubydb/catalog/system_catalog.rb +698 -698
- data/lib/rubydb/catalog/table.rb +178 -178
- data/lib/rubydb/catalog/trigger.rb +102 -102
- data/lib/rubydb/catalog/view.rb +66 -66
- data/lib/rubydb/cli/application.rb +168 -163
- data/lib/rubydb/cli/commands/accelerator.rb +72 -0
- data/lib/rubydb/cli/commands/backup.rb +80 -81
- data/lib/rubydb/cli/commands/branch.rb +72 -72
- data/lib/rubydb/cli/commands/checkout.rb +54 -54
- data/lib/rubydb/cli/commands/create.rb +58 -58
- data/lib/rubydb/cli/commands/diff.rb +76 -77
- data/lib/rubydb/cli/commands/doctor.rb +77 -74
- data/lib/rubydb/cli/commands/drop.rb +57 -57
- data/lib/rubydb/cli/commands/init.rb +101 -102
- data/lib/rubydb/cli/commands/inspect.rb +95 -95
- data/lib/rubydb/cli/commands/merge.rb +63 -63
- data/lib/rubydb/cli/commands/migrate.rb +62 -62
- data/lib/rubydb/cli/commands/restart.rb +42 -39
- data/lib/rubydb/cli/commands/restore.rb +121 -121
- data/lib/rubydb/cli/commands/shell.rb +365 -365
- data/lib/rubydb/cli/commands/snapshot.rb +79 -79
- data/lib/rubydb/cli/commands/start.rb +88 -82
- data/lib/rubydb/cli/commands/status.rb +96 -92
- data/lib/rubydb/cli/commands/stop.rb +47 -47
- data/lib/rubydb/cli/commands/vacuum.rb +58 -58
- data/lib/rubydb/cli/formatter.rb +221 -221
- data/lib/rubydb/cli/output.rb +168 -168
- data/lib/rubydb/client/client.rb +309 -304
- data/lib/rubydb/client/connection.rb +429 -415
- data/lib/rubydb/client/connection_pool.rb +168 -168
- data/lib/rubydb/client/connection_url.rb +96 -96
- data/lib/rubydb/client/prepared_statement.rb +60 -60
- data/lib/rubydb/client/result.rb +127 -123
- data/lib/rubydb/client/statement.rb +52 -52
- data/lib/rubydb/client/transaction.rb +130 -130
- data/lib/rubydb/concurrency/concurrency.rb +19 -19
- data/lib/rubydb/concurrency/deadlock_detector.rb +148 -150
- data/lib/rubydb/concurrency/latch.rb +101 -101
- data/lib/rubydb/concurrency/lock_graph.rb +163 -165
- data/lib/rubydb/concurrency/mutex.rb +181 -183
- data/lib/rubydb/concurrency/rw_lock.rb +180 -180
- data/lib/rubydb/concurrency/scheduler.rb +248 -250
- data/lib/rubydb/concurrency/worker_pool.rb +145 -143
- data/lib/rubydb/configuration/config.rb +170 -170
- data/lib/rubydb/configuration/defaults.rb +191 -179
- data/lib/rubydb/configuration/environment.rb +152 -152
- data/lib/rubydb/configuration/parser.rb +185 -185
- data/lib/rubydb/configuration/validation.rb +228 -221
- data/lib/rubydb/constants.rb +74 -74
- data/lib/rubydb/constraints/check.rb +181 -181
- data/lib/rubydb/constraints/constraint.rb +101 -101
- data/lib/rubydb/constraints/foreign_key.rb +130 -130
- data/lib/rubydb/constraints/not_null.rb +64 -64
- data/lib/rubydb/constraints/primary_key.rb +99 -99
- data/lib/rubydb/constraints/unique.rb +106 -108
- data/lib/rubydb/constraints/validator.rb +349 -350
- data/lib/rubydb/errors/authentication_error.rb +10 -10
- data/lib/rubydb/errors/authorization_error.rb +23 -23
- data/lib/rubydb/errors/client_error.rb +10 -10
- data/lib/rubydb/errors/configuration_error.rb +10 -10
- data/lib/rubydb/errors/connection_error.rb +10 -10
- data/lib/rubydb/errors/constraint_error.rb +23 -23
- data/lib/rubydb/errors/corruption_error.rb +10 -10
- data/lib/rubydb/errors/database_error.rb +10 -10
- data/lib/rubydb/errors/error.rb +20 -20
- data/lib/rubydb/errors/execution_error.rb +10 -10
- data/lib/rubydb/errors/parser_error.rb +10 -10
- data/lib/rubydb/errors/recovery_error.rb +10 -10
- data/lib/rubydb/errors/replication_error.rb +10 -10
- data/lib/rubydb/errors/server_error.rb +6 -6
- data/lib/rubydb/errors/storage_error.rb +10 -10
- data/lib/rubydb/errors/transaction_error.rb +10 -10
- data/lib/rubydb/execution/accelerator_dispatch.rb +30 -0
- data/lib/rubydb/execution/aggregate_executor.rb +134 -138
- data/lib/rubydb/execution/cost_model.rb +72 -0
- data/lib/rubydb/execution/delete_executor.rb +110 -112
- data/lib/rubydb/execution/distinct_executor.rb +131 -135
- data/lib/rubydb/execution/executor.rb +1544 -1188
- data/lib/rubydb/execution/expression.rb +191 -193
- data/lib/rubydb/execution/index_scan.rb +142 -142
- data/lib/rubydb/execution/insert_executor.rb +215 -217
- data/lib/rubydb/execution/join_executor.rb +243 -249
- data/lib/rubydb/execution/limit_executor.rb +83 -85
- data/lib/rubydb/execution/operator_selection.rb +57 -0
- data/lib/rubydb/execution/optimizer.rb +227 -215
- data/lib/rubydb/execution/physical_plan.rb +47 -0
- data/lib/rubydb/execution/plan.rb +355 -353
- data/lib/rubydb/execution/planner.rb +508 -536
- data/lib/rubydb/execution/predicate.rb +235 -235
- data/lib/rubydb/execution/scan.rb +49 -49
- data/lib/rubydb/execution/sequential_scan.rb +63 -63
- data/lib/rubydb/execution/sort_executor.rb +194 -185
- data/lib/rubydb/execution/update_executor.rb +160 -162
- data/lib/rubydb/functions/aggregate.rb +70 -70
- data/lib/rubydb/functions/date_functions.rb +274 -278
- data/lib/rubydb/functions/function.rb +85 -85
- data/lib/rubydb/functions/json_functions.rb +231 -215
- data/lib/rubydb/functions/numeric_functions.rb +346 -346
- data/lib/rubydb/functions/scalar.rb +52 -52
- data/lib/rubydb/functions/string_functions.rb +383 -383
- data/lib/rubydb/functions/system_functions.rb +258 -246
- data/lib/rubydb/history/as_of.rb +238 -238
- data/lib/rubydb/history/change.rb +105 -105
- data/lib/rubydb/history/history.rb +131 -131
- data/lib/rubydb/history/history_manager.rb +228 -229
- data/lib/rubydb/history/temporal_query.rb +202 -202
- data/lib/rubydb/history/timeline.rb +144 -144
- data/lib/rubydb/indexes/btree.rb +215 -186
- data/lib/rubydb/indexes/btree_cursor.rb +258 -258
- data/lib/rubydb/indexes/btree_node.rb +384 -385
- data/lib/rubydb/indexes/hash_index.rb +150 -150
- data/lib/rubydb/indexes/index.rb +71 -71
- data/lib/rubydb/indexes/index_manager.rb +408 -406
- data/lib/rubydb/indexes/index_scan.rb +466 -470
- data/lib/rubydb/migrations/migration.rb +253 -254
- data/lib/rubydb/migrations/migration_lock.rb +146 -146
- data/lib/rubydb/migrations/migration_manager.rb +187 -176
- data/lib/rubydb/migrations/migration_version.rb +71 -71
- data/lib/rubydb/migrations/schema_diff.rb +211 -211
- data/lib/rubydb/migrations/schema_version.rb +64 -64
- data/lib/rubydb/monitoring/events.rb +155 -160
- data/lib/rubydb/monitoring/health.rb +216 -222
- data/lib/rubydb/monitoring/logger.rb +188 -193
- data/lib/rubydb/monitoring/metrics.rb +363 -359
- data/lib/rubydb/monitoring/performance.rb +176 -176
- data/lib/rubydb/monitoring/statistics.rb +168 -170
- data/lib/rubydb/mvcc/garbage_collector.rb +199 -199
- data/lib/rubydb/mvcc/mvcc.rb +16 -16
- data/lib/rubydb/mvcc/snapshot.rb +146 -147
- data/lib/rubydb/mvcc/vacuum.rb +180 -180
- data/lib/rubydb/mvcc/version.rb +106 -106
- data/lib/rubydb/mvcc/version_store.rb +396 -398
- data/lib/rubydb/mvcc/visibility.rb +107 -109
- data/lib/rubydb/protocol/capabilities.rb +125 -125
- data/lib/rubydb/protocol/decoder.rb +142 -145
- data/lib/rubydb/protocol/encoder.rb +131 -136
- data/lib/rubydb/protocol/handshake.rb +306 -305
- data/lib/rubydb/protocol/message.rb +121 -121
- data/lib/rubydb/protocol/parameter_binder.rb +101 -0
- data/lib/rubydb/protocol/protocol.rb +276 -277
- data/lib/rubydb/protocol/version.rb +54 -54
- data/lib/rubydb/rails/adapter.rb +245 -239
- data/lib/rubydb/rails/connection.rb +312 -314
- data/lib/rubydb/rails/database_statements.rb +122 -122
- data/lib/rubydb/rails/migration.rb +131 -131
- data/lib/rubydb/rails/quoting.rb +109 -109
- data/lib/rubydb/rails/result.rb +117 -117
- data/lib/rubydb/rails/schema_statements.rb +339 -339
- data/lib/rubydb/rails/transaction.rb +105 -105
- data/lib/rubydb/rails/type.rb +126 -126
- data/lib/rubydb/recovery/checkpoint.rb +261 -257
- data/lib/rubydb/recovery/consistency.rb +457 -467
- data/lib/rubydb/recovery/corruption_detector.rb +5 -5
- data/lib/rubydb/recovery/crash_recovery.rb +381 -387
- data/lib/rubydb/recovery/recovery_manager.rb +204 -206
- data/lib/rubydb/recovery/redo.rb +235 -237
- data/lib/rubydb/recovery/undo.rb +204 -206
- data/lib/rubydb/replication/failover.rb +5 -5
- data/lib/rubydb/replication/fencing.rb +63 -63
- data/lib/rubydb/replication/primary.rb +461 -450
- data/lib/rubydb/replication/replica.rb +382 -384
- data/lib/rubydb/replication/replication_log.rb +194 -200
- data/lib/rubydb/replication/replication_manager.rb +307 -308
- data/lib/rubydb/replication/replication_slot.rb +293 -295
- data/lib/rubydb/replication/replication_stream.rb +198 -201
- data/lib/rubydb/rubydb.rb +570 -560
- data/lib/rubydb/security/access_control.rb +252 -254
- data/lib/rubydb/security/audit_log.rb +209 -213
- data/lib/rubydb/security/authentication.rb +302 -302
- data/lib/rubydb/security/authorization.rb +282 -282
- data/lib/rubydb/security/credentials.rb +192 -196
- data/lib/rubydb/security/password.rb +205 -215
- data/lib/rubydb/security/permissions.rb +74 -74
- data/lib/rubydb/security/role.rb +99 -101
- data/lib/rubydb/security/user.rb +86 -86
- data/lib/rubydb/server/connection.rb +383 -366
- data/lib/rubydb/server/connection_pool.rb +193 -193
- data/lib/rubydb/server/lifecycle.rb +227 -228
- data/lib/rubydb/server/listener.rb +139 -136
- data/lib/rubydb/server/request_handler.rb +277 -276
- data/lib/rubydb/server/server.rb +363 -364
- data/lib/rubydb/server/session.rb +416 -369
- data/lib/rubydb/server/worker.rb +206 -210
- data/lib/rubydb/server/worker_pool.rb +168 -168
- data/lib/rubydb/sql/ast/alter_table.rb +169 -169
- data/lib/rubydb/sql/ast/begin_transaction.rb +47 -47
- data/lib/rubydb/sql/ast/commit.rb +37 -37
- data/lib/rubydb/sql/ast/constraint.rb +92 -83
- data/lib/rubydb/sql/ast/create_database.rb +41 -41
- data/lib/rubydb/sql/ast/create_index.rb +61 -61
- data/lib/rubydb/sql/ast/create_schema.rb +52 -52
- data/lib/rubydb/sql/ast/create_table.rb +187 -187
- data/lib/rubydb/sql/ast/delete.rb +54 -54
- data/lib/rubydb/sql/ast/drop_database.rb +41 -41
- data/lib/rubydb/sql/ast/drop_index.rb +41 -41
- data/lib/rubydb/sql/ast/drop_schema.rb +49 -49
- data/lib/rubydb/sql/ast/drop_table.rb +49 -49
- data/lib/rubydb/sql/ast/explain.rb +64 -64
- data/lib/rubydb/sql/ast/expression.rb +617 -604
- data/lib/rubydb/sql/ast/insert.rb +66 -66
- data/lib/rubydb/sql/ast/node.rb +42 -42
- data/lib/rubydb/sql/ast/rollback.rb +63 -63
- data/lib/rubydb/sql/ast/savepoint.rb +59 -59
- data/lib/rubydb/sql/ast/select.rb +88 -88
- data/lib/rubydb/sql/ast/set_operation.rb +22 -20
- data/lib/rubydb/sql/ast/trigger.rb +35 -29
- data/lib/rubydb/sql/ast/update.rb +88 -88
- data/lib/rubydb/sql/ast/vacuum.rb +19 -19
- data/lib/rubydb/sql/ast/view.rb +38 -32
- data/lib/rubydb/sql/ast/with.rb +32 -32
- data/lib/rubydb/sql/grammar.rb +86 -86
- data/lib/rubydb/sql/keywords.rb +156 -156
- data/lib/rubydb/sql/lexer.rb +209 -214
- data/lib/rubydb/sql/operators.rb +100 -100
- data/lib/rubydb/sql/parser.rb +1167 -1170
- data/lib/rubydb/sql/planner/analyzer.rb +283 -302
- data/lib/rubydb/sql/planner/binder.rb +537 -550
- data/lib/rubydb/sql/planner/type_checker.rb +427 -431
- data/lib/rubydb/sql/token.rb +210 -210
- data/lib/rubydb/storage/buffer_frame.rb +44 -44
- data/lib/rubydb/storage/buffer_pool.rb +155 -155
- data/lib/rubydb/storage/database_lock.rb +74 -74
- data/lib/rubydb/storage/deserializer.rb +332 -342
- data/lib/rubydb/storage/engine.rb +2409 -2330
- data/lib/rubydb/storage/file_manager.rb +191 -187
- data/lib/rubydb/storage/free_space_map.rb +79 -81
- data/lib/rubydb/storage/page.rb +92 -94
- data/lib/rubydb/storage/page_allocator.rb +852 -855
- data/lib/rubydb/storage/page_header.rb +63 -67
- data/lib/rubydb/storage/page_manager.rb +127 -131
- data/lib/rubydb/storage/record.rb +58 -58
- data/lib/rubydb/storage/row.rb +78 -78
- data/lib/rubydb/storage/serializer.rb +51 -51
- data/lib/rubydb/storage/snapshot_reader.rb +167 -0
- data/lib/rubydb/storage/storage_layout.rb +151 -151
- data/lib/rubydb/storage/storage_manager.rb +114 -114
- data/lib/rubydb/storage/tuple.rb +458 -461
- data/lib/rubydb/storage/visibility_map.rb +964 -973
- data/lib/rubydb/transactions/commit_manager.rb +219 -220
- data/lib/rubydb/transactions/isolation.rb +98 -98
- data/lib/rubydb/transactions/lock.rb +76 -76
- data/lib/rubydb/transactions/lock_manager.rb +359 -362
- data/lib/rubydb/transactions/savepoint.rb +142 -143
- data/lib/rubydb/transactions/transaction.rb +214 -215
- data/lib/rubydb/transactions/transaction_id.rb +84 -84
- data/lib/rubydb/transactions/transaction_log.rb +256 -257
- data/lib/rubydb/transactions/transaction_manager.rb +434 -435
- data/lib/rubydb/types/bigint.rb +36 -36
- data/lib/rubydb/types/blob.rb +37 -37
- data/lib/rubydb/types/boolean.rb +34 -34
- data/lib/rubydb/types/date.rb +39 -39
- data/lib/rubydb/types/decimal.rb +48 -48
- data/lib/rubydb/types/float.rb +34 -34
- data/lib/rubydb/types/integer.rb +36 -36
- data/lib/rubydb/types/json.rb +41 -41
- data/lib/rubydb/types/null.rb +34 -34
- data/lib/rubydb/types/smallint.rb +36 -36
- data/lib/rubydb/types/text.rb +37 -37
- data/lib/rubydb/types/time.rb +46 -46
- data/lib/rubydb/types/timestamp.rb +39 -39
- data/lib/rubydb/types/type.rb +119 -119
- data/lib/rubydb/types/uuid.rb +47 -47
- data/lib/rubydb/types/varchar.rb +37 -37
- data/lib/rubydb/version.rb +32 -32
- data/lib/rubydb/wal/archive.rb +207 -193
- data/lib/rubydb/wal/checkpoint.rb +181 -183
- data/lib/rubydb/wal/lsn.rb +94 -94
- data/lib/rubydb/wal/reader.rb +259 -260
- data/lib/rubydb/wal/record.rb +105 -105
- data/lib/rubydb/wal/segment.rb +193 -193
- data/lib/rubydb/wal/wal.rb +481 -452
- data/lib/rubydb/wal/writer.rb +236 -236
- data/lib/rubydb.rb +7 -7
- data/packaging/docker/docker-compose.failover.yml +43 -43
- data/packaging/homebrew/rubydb.rb +19 -19
- data/rubydb.gemspec +70 -57
- data/scripts/benchmark +7 -7
- data/scripts/build_accelerator +49 -0
- data/scripts/durability_drill +37 -37
- data/scripts/fuzz +63 -63
- data/scripts/release +77 -42
- data/scripts/release_check +43 -43
- data/scripts/replication_failover_drill +268 -250
- data/scripts/replication_network_failover_drill +287 -255
- data/scripts/restore_drill +45 -45
- data/scripts/security +45 -0
- metadata +102 -1
|
@@ -1,45 +1,45 @@
|
|
|
1
|
-
# RubyDB production runbook
|
|
2
|
-
|
|
3
|
-
## Before deployment
|
|
4
|
-
|
|
5
|
-
1. Pin the RubyDB commit and Ruby version. Run `bundle exec rspec`, the
|
|
6
|
-
production soak, the durability drill, and the network failover drill.
|
|
7
|
-
2. Place the database, WAL, lock file, replication state, and backups on
|
|
8
|
-
protected storage owned by the service account. Keep backups on a separate
|
|
9
|
-
failure domain.
|
|
10
|
-
3. Enable TLS, use a private bind address, configure authentication and a
|
|
11
|
-
replication token from a secret manager, and set connection, frame, query,
|
|
12
|
-
worker, and disk-space limits.
|
|
13
|
-
4. Install the alert rules in
|
|
14
|
-
`config/monitoring/prometheus-alerts.yml`. Test readiness, metrics, backup,
|
|
15
|
-
and restore from the deployment environment.
|
|
16
|
-
|
|
17
|
-
## Backup and restore
|
|
18
|
-
|
|
19
|
-
Run a full backup before upgrades and retain the matching WAL chain and
|
|
20
|
-
manifest. Restore into a new inactive directory, run the restore drill, reopen
|
|
21
|
-
the database, compare row counts/checksums, and only then switch traffic.
|
|
22
|
-
Never overwrite the only copy during a restore.
|
|
23
|
-
|
|
24
|
-
## Failover
|
|
25
|
-
|
|
26
|
-
Stop writes, confirm the replica is synchronized, fence the old primary using
|
|
27
|
-
the shared durable fence path, promote exactly one replica, and verify client
|
|
28
|
-
writes and replication status. Automatic election is not enabled until the
|
|
29
|
-
deployment provides an independently tested fencing lease. A network partition
|
|
30
|
-
must never result in two writable primaries.
|
|
31
|
-
|
|
32
|
-
## TLS and secret rotation
|
|
33
|
-
|
|
34
|
-
Stage a new certificate/key and replication token in the secret manager,
|
|
35
|
-
validate them on an inactive instance, restart or drain one peer at a time,
|
|
36
|
-
verify TLS and replication authentication, then revoke the old credentials
|
|
37
|
-
after the overlap window. Do not place secrets in repository files, command
|
|
38
|
-
arguments, URLs, or logs.
|
|
39
|
-
|
|
40
|
-
## Upgrade and incident response
|
|
41
|
-
|
|
42
|
-
Run the upgrade-compatibility suite against a copy of production backups.
|
|
43
|
-
Preserve the database, WAL, replication state, logs, metrics, and timestamps
|
|
44
|
-
before repair, vacuum, restore, or promotion. If checksums, fencing, or
|
|
45
|
-
replication diverge, stop writes and escalate; do not guess at a repair.
|
|
1
|
+
# RubyDB production runbook
|
|
2
|
+
|
|
3
|
+
## Before deployment
|
|
4
|
+
|
|
5
|
+
1. Pin the RubyDB commit and Ruby version. Run `bundle exec rspec`, the
|
|
6
|
+
production soak, the durability drill, and the network failover drill.
|
|
7
|
+
2. Place the database, WAL, lock file, replication state, and backups on
|
|
8
|
+
protected storage owned by the service account. Keep backups on a separate
|
|
9
|
+
failure domain.
|
|
10
|
+
3. Enable TLS, use a private bind address, configure authentication and a
|
|
11
|
+
replication token from a secret manager, and set connection, frame, query,
|
|
12
|
+
worker, and disk-space limits.
|
|
13
|
+
4. Install the alert rules in
|
|
14
|
+
`config/monitoring/prometheus-alerts.yml`. Test readiness, metrics, backup,
|
|
15
|
+
and restore from the deployment environment.
|
|
16
|
+
|
|
17
|
+
## Backup and restore
|
|
18
|
+
|
|
19
|
+
Run a full backup before upgrades and retain the matching WAL chain and
|
|
20
|
+
manifest. Restore into a new inactive directory, run the restore drill, reopen
|
|
21
|
+
the database, compare row counts/checksums, and only then switch traffic.
|
|
22
|
+
Never overwrite the only copy during a restore.
|
|
23
|
+
|
|
24
|
+
## Failover
|
|
25
|
+
|
|
26
|
+
Stop writes, confirm the replica is synchronized, fence the old primary using
|
|
27
|
+
the shared durable fence path, promote exactly one replica, and verify client
|
|
28
|
+
writes and replication status. Automatic election is not enabled until the
|
|
29
|
+
deployment provides an independently tested fencing lease. A network partition
|
|
30
|
+
must never result in two writable primaries.
|
|
31
|
+
|
|
32
|
+
## TLS and secret rotation
|
|
33
|
+
|
|
34
|
+
Stage a new certificate/key and replication token in the secret manager,
|
|
35
|
+
validate them on an inactive instance, restart or drain one peer at a time,
|
|
36
|
+
verify TLS and replication authentication, then revoke the old credentials
|
|
37
|
+
after the overlap window. Do not place secrets in repository files, command
|
|
38
|
+
arguments, URLs, or logs.
|
|
39
|
+
|
|
40
|
+
## Upgrade and incident response
|
|
41
|
+
|
|
42
|
+
Run the upgrade-compatibility suite against a copy of production backups.
|
|
43
|
+
Preserve the database, WAL, replication state, logs, metrics, and timestamps
|
|
44
|
+
before repair, vacuum, restore, or promotion. If checksums, fencing, or
|
|
45
|
+
replication diverge, stop writes and escalate; do not guess at a repair.
|
|
@@ -1,33 +1,33 @@
|
|
|
1
|
-
# Replication operations
|
|
2
|
-
|
|
3
|
-
RubyDB replication is logical WAL row-mutation replication. Configure the
|
|
4
|
-
primary and replica with separate data directories, explicit network addresses,
|
|
5
|
-
and the same high-entropy `replication_auth_token`. Restrict the listener with
|
|
6
|
-
firewall rules and use TLS or a trusted private network; the token is peer
|
|
7
|
-
authentication, not a replacement for transport encryption.
|
|
8
|
-
|
|
9
|
-
## Bootstrap and catch-up
|
|
10
|
-
|
|
11
|
-
Start the primary first, then start the replica with its replica identifier and
|
|
12
|
-
primary address. The primary sends catalog schema before row replay. Verify the
|
|
13
|
-
replica reaches `SYNCED`, and compare received and replayed LSNs before routing
|
|
14
|
-
reads to it.
|
|
15
|
-
|
|
16
|
-
Monitor replication status continuously. A replica with a received/replayed
|
|
17
|
-
gap is not eligible for promotion. Preserve the replication log and replica
|
|
18
|
-
state files during an incident; do not manually edit LSN state.
|
|
19
|
-
|
|
20
|
-
## Token rotation
|
|
21
|
-
|
|
22
|
-
Drain replication, stop or disconnect the replica, update the protected token
|
|
23
|
-
on both sides, restart the primary listener if required, and reconnect the
|
|
24
|
-
replica. Confirm accepted handshakes and zero unexpected authentication
|
|
25
|
-
failures before returning the replica to service. Never place tokens in source
|
|
26
|
-
control or command histories.
|
|
27
|
-
|
|
28
|
-
## Failure handling
|
|
29
|
-
|
|
30
|
-
On a network partition, assume the old primary may still be reachable until it
|
|
31
|
-
is fenced. Do not promote a lagging replica and do not run two writable primaries
|
|
32
|
-
without a verified fencing lease. Re-bootstrap a divergent replica rather than
|
|
33
|
-
guessing which WAL records are safe to apply.
|
|
1
|
+
# Replication operations
|
|
2
|
+
|
|
3
|
+
RubyDB replication is logical WAL row-mutation replication. Configure the
|
|
4
|
+
primary and replica with separate data directories, explicit network addresses,
|
|
5
|
+
and the same high-entropy `replication_auth_token`. Restrict the listener with
|
|
6
|
+
firewall rules and use TLS or a trusted private network; the token is peer
|
|
7
|
+
authentication, not a replacement for transport encryption.
|
|
8
|
+
|
|
9
|
+
## Bootstrap and catch-up
|
|
10
|
+
|
|
11
|
+
Start the primary first, then start the replica with its replica identifier and
|
|
12
|
+
primary address. The primary sends catalog schema before row replay. Verify the
|
|
13
|
+
replica reaches `SYNCED`, and compare received and replayed LSNs before routing
|
|
14
|
+
reads to it.
|
|
15
|
+
|
|
16
|
+
Monitor replication status continuously. A replica with a received/replayed
|
|
17
|
+
gap is not eligible for promotion. Preserve the replication log and replica
|
|
18
|
+
state files during an incident; do not manually edit LSN state.
|
|
19
|
+
|
|
20
|
+
## Token rotation
|
|
21
|
+
|
|
22
|
+
Drain replication, stop or disconnect the replica, update the protected token
|
|
23
|
+
on both sides, restart the primary listener if required, and reconnect the
|
|
24
|
+
replica. Confirm accepted handshakes and zero unexpected authentication
|
|
25
|
+
failures before returning the replica to service. Never place tokens in source
|
|
26
|
+
control or command histories.
|
|
27
|
+
|
|
28
|
+
## Failure handling
|
|
29
|
+
|
|
30
|
+
On a network partition, assume the old primary may still be reachable until it
|
|
31
|
+
is fenced. Do not promote a lagging replica and do not run two writable primaries
|
|
32
|
+
without a verified fencing lease. Re-bootstrap a divergent replica rather than
|
|
33
|
+
guessing which WAL records are safe to apply.
|
data/docs/operations/restore.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
|
-
## Restore verification
|
|
2
|
-
|
|
3
|
-
Before treating a backup as recoverable, run the verification path that extracts
|
|
4
|
-
it into an isolated temporary directory, opens the database with RubyDB, and
|
|
5
|
-
compares the restored catalog and table row counts with the manifest. The
|
|
6
|
-
verification removes its temporary database after the check.
|
|
1
|
+
## Restore verification
|
|
2
|
+
|
|
3
|
+
Before treating a backup as recoverable, run the verification path that extracts
|
|
4
|
+
it into an isolated temporary directory, opens the database with RubyDB, and
|
|
5
|
+
compares the restored catalog and table row counts with the manifest. The
|
|
6
|
+
verification removes its temporary database after the check.
|
data/docs/operations/runbook.md
CHANGED
|
@@ -1,34 +1,34 @@
|
|
|
1
|
-
# RubyDB operator runbook
|
|
2
|
-
|
|
3
|
-
## Start and health check
|
|
4
|
-
|
|
5
|
-
Run the server with an explicit production configuration:
|
|
6
|
-
|
|
7
|
-
```sh
|
|
8
|
-
RUBYDB_ENV=production rubydb -c config/production.yml start
|
|
9
|
-
```
|
|
10
|
-
|
|
11
|
-
Confirm the process is listening, then run `rubydb status --json`. Treat a missing PID file, an invalid configuration, or a failed health check as a failed deployment; do not bypass startup validation.
|
|
12
|
-
|
|
13
|
-
## Backup and restore
|
|
14
|
-
|
|
15
|
-
Create a verified full backup before upgrades or maintenance. Retain the manifest and WAL chain together. Test restoration into a separate directory before replacing a live data directory. Never restore over an active database.
|
|
16
|
-
|
|
17
|
-
## Upgrade and rollback
|
|
18
|
-
|
|
19
|
-
Stop writes, create and verify a full backup, restore it into a staging directory, and run migration status plus application smoke queries. Promote the staged directory only after validation. Roll back by stopping RubyDB and switching back to the retained pre-upgrade directory; preserve the failed directory for diagnosis.
|
|
20
|
-
|
|
21
|
-
## Incident boundaries
|
|
22
|
-
|
|
23
|
-
On corruption, checksum failure, repeated crash recovery, or replication divergence, stop writes and preserve the data directory and logs. Do not delete WAL, run vacuum, or force promotion until a verified backup and incident copy exist. Automatic failover is intentionally disabled and promotion requires an operator decision.
|
|
24
|
-
|
|
25
|
-
## Security operations
|
|
26
|
-
|
|
27
|
-
Keep production credentials and TLS private keys outside the repository with restrictive permissions. Rotate certificates by staging the replacement, validating the chain, and restarting during a planned window. Keep TLS enabled and use peer verification when a trusted CA is available.
|
|
28
|
-
|
|
29
|
-
For replication, configure the same high-entropy `replication_auth_token` on
|
|
30
|
-
each primary and replica. A peer with a missing or incorrect token is rejected;
|
|
31
|
-
rotate the token by draining replicas, updating the protected configuration on
|
|
32
|
-
both sides, and reconnecting them. The replication listener is not a substitute
|
|
33
|
-
for a private network or TLS termination, so restrict its port at the firewall
|
|
34
|
-
and validate certificate/private-network controls during deployment.
|
|
1
|
+
# RubyDB operator runbook
|
|
2
|
+
|
|
3
|
+
## Start and health check
|
|
4
|
+
|
|
5
|
+
Run the server with an explicit production configuration:
|
|
6
|
+
|
|
7
|
+
```sh
|
|
8
|
+
RUBYDB_ENV=production rubydb -c config/production.yml start
|
|
9
|
+
```
|
|
10
|
+
|
|
11
|
+
Confirm the process is listening, then run `rubydb status --json`. Treat a missing PID file, an invalid configuration, or a failed health check as a failed deployment; do not bypass startup validation.
|
|
12
|
+
|
|
13
|
+
## Backup and restore
|
|
14
|
+
|
|
15
|
+
Create a verified full backup before upgrades or maintenance. Retain the manifest and WAL chain together. Test restoration into a separate directory before replacing a live data directory. Never restore over an active database.
|
|
16
|
+
|
|
17
|
+
## Upgrade and rollback
|
|
18
|
+
|
|
19
|
+
Stop writes, create and verify a full backup, restore it into a staging directory, and run migration status plus application smoke queries. Promote the staged directory only after validation. Roll back by stopping RubyDB and switching back to the retained pre-upgrade directory; preserve the failed directory for diagnosis.
|
|
20
|
+
|
|
21
|
+
## Incident boundaries
|
|
22
|
+
|
|
23
|
+
On corruption, checksum failure, repeated crash recovery, or replication divergence, stop writes and preserve the data directory and logs. Do not delete WAL, run vacuum, or force promotion until a verified backup and incident copy exist. Automatic failover is intentionally disabled and promotion requires an operator decision.
|
|
24
|
+
|
|
25
|
+
## Security operations
|
|
26
|
+
|
|
27
|
+
Keep production credentials and TLS private keys outside the repository with restrictive permissions. Rotate certificates by staging the replacement, validating the chain, and restarting during a planned window. Keep TLS enabled and use peer verification when a trusted CA is available.
|
|
28
|
+
|
|
29
|
+
For replication, configure the same high-entropy `replication_auth_token` on
|
|
30
|
+
each primary and replica. A peer with a missing or incorrect token is rejected;
|
|
31
|
+
rotate the token by draining replicas, updating the protected configuration on
|
|
32
|
+
both sides, and reconnecting them. The replication listener is not a substitute
|
|
33
|
+
for a private network or TLS termination, so restrict its port at the firewall
|
|
34
|
+
and validate certificate/private-network controls during deployment.
|
data/docs/operations/upgrades.md
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
|
-
# Upgrades
|
|
2
|
-
|
|
3
|
-
RubyDB upgrades must be treated as storage migrations, not as an in-place binary replacement.
|
|
4
|
-
|
|
5
|
-
Before upgrading:
|
|
6
|
-
|
|
7
|
-
1. Stop writes and confirm the server is healthy.
|
|
8
|
-
2. Create and verify a full backup, including WAL when available.
|
|
9
|
-
3. Record the RubyDB version, storage page size, and backup manifest checksum.
|
|
10
|
-
4. Test the new release against a restored copy of the backup.
|
|
11
|
-
|
|
12
|
-
The current storage format is version 1. RubyDB refuses to open an unknown page format or a database whose page size differs from the configured page size. There is no automatic on-disk format conversion yet; a future incompatible format must provide an explicit export/import migration.
|
|
13
|
-
|
|
14
|
-
After upgrading, reopen the restored or migrated database, run the migration status command, verify application queries, and retain the pre-upgrade backup until the rollback window closes. Never delete the original data directory as part of an upgrade.
|
|
1
|
+
# Upgrades
|
|
2
|
+
|
|
3
|
+
RubyDB upgrades must be treated as storage migrations, not as an in-place binary replacement.
|
|
4
|
+
|
|
5
|
+
Before upgrading:
|
|
6
|
+
|
|
7
|
+
1. Stop writes and confirm the server is healthy.
|
|
8
|
+
2. Create and verify a full backup, including WAL when available.
|
|
9
|
+
3. Record the RubyDB version, storage page size, and backup manifest checksum.
|
|
10
|
+
4. Test the new release against a restored copy of the backup.
|
|
11
|
+
|
|
12
|
+
The current storage format is version 1. RubyDB refuses to open an unknown page format or a database whose page size differs from the configured page size. There is no automatic on-disk format conversion yet; a future incompatible format must provide an explicit export/import migration.
|
|
13
|
+
|
|
14
|
+
After upgrading, reopen the restored or migrated database, run the migration status command, verify application queries, and retain the pre-upgrade backup until the rollback window closes. Never delete the original data directory as part of an upgrade.
|
|
@@ -1,17 +1,17 @@
|
|
|
1
|
-
# Workload testing
|
|
2
|
-
|
|
3
|
-
Run the deterministic concurrent workload from the repository root:
|
|
4
|
-
|
|
5
|
-
```sh
|
|
6
|
-
RUBYDB_WORKLOAD_THREADS=4 RUBYDB_WORKLOAD_OPERATIONS=250 ruby benchmarks/concurrent_workload.rb
|
|
7
|
-
```
|
|
8
|
-
|
|
9
|
-
It performs concurrent inserts against one engine, verifies the final row count, closes the database, reopens it, and verifies durable reads. It prints one JSON result and exits non-zero on any operation or durability failure.
|
|
10
|
-
|
|
11
|
-
Use a temporary database by default. To retain an artifact for inspection, provide an explicit path:
|
|
12
|
-
|
|
13
|
-
```sh
|
|
14
|
-
RUBYDB_WORKLOAD_PATH=/var/lib/rubydb/workload.rdb RUBYDB_WORKLOAD_THREADS=16 RUBYDB_WORKLOAD_OPERATIONS=10000 ruby benchmarks/concurrent_workload.rb
|
|
15
|
-
```
|
|
16
|
-
|
|
17
|
-
Run this in a staging environment on the target filesystem before deployment. Establish acceptable latency, throughput, WAL growth, disk-space, recovery-time, and backup/restore targets for the expected workload; do not treat a successful smoke workload as capacity certification.
|
|
1
|
+
# Workload testing
|
|
2
|
+
|
|
3
|
+
Run the deterministic concurrent workload from the repository root:
|
|
4
|
+
|
|
5
|
+
```sh
|
|
6
|
+
RUBYDB_WORKLOAD_THREADS=4 RUBYDB_WORKLOAD_OPERATIONS=250 ruby benchmarks/concurrent_workload.rb
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
It performs concurrent inserts against one engine, verifies the final row count, closes the database, reopens it, and verifies durable reads. It prints one JSON result and exits non-zero on any operation or durability failure.
|
|
10
|
+
|
|
11
|
+
Use a temporary database by default. To retain an artifact for inspection, provide an explicit path:
|
|
12
|
+
|
|
13
|
+
```sh
|
|
14
|
+
RUBYDB_WORKLOAD_PATH=/var/lib/rubydb/workload.rdb RUBYDB_WORKLOAD_THREADS=16 RUBYDB_WORKLOAD_OPERATIONS=10000 ruby benchmarks/concurrent_workload.rb
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
Run this in a staging environment on the target filesystem before deployment. Establish acceptable latency, throughput, WAL growth, disk-space, recovery-time, and backup/restore targets for the expected workload; do not treat a successful smoke workload as capacity certification.
|
|
@@ -1,118 +1,118 @@
|
|
|
1
|
-
# Production-readiness audit
|
|
2
|
-
|
|
3
|
-
## Status
|
|
4
|
-
|
|
5
|
-
RubyDB has verified production-oriented foundations, but it is not yet a general-purpose production database. Supported behavior is backed by the RSpec suite in `spec/`.
|
|
6
|
-
|
|
7
|
-
Latest local audit: 270 examples, 0 failures. Hosted CI and deployment-specific
|
|
8
|
-
evidence remain separate release gates.
|
|
9
|
-
|
|
10
|
-
The feature inventory below was written during an earlier checkpoint and its
|
|
11
|
-
parenthetical historical count is not the current total. Use the latest audit
|
|
12
|
-
line above and rerun the suite for release evidence.
|
|
13
|
-
|
|
14
|
-
## Implemented features
|
|
15
|
-
|
|
16
|
-
The repository contains substantial scaffolding for:
|
|
17
|
-
|
|
18
|
-
- page-based storage
|
|
19
|
-
- buffer pool caching
|
|
20
|
-
- file and page management
|
|
21
|
-
- WAL, recovery, and checkpoint modules
|
|
22
|
-
- transactions and MVCC abstractions
|
|
23
|
-
- SQL parser/planner infrastructure
|
|
24
|
-
- server, protocol, and connection components
|
|
25
|
-
- authentication and authorization modules
|
|
26
|
-
- backup and replication APIs
|
|
27
|
-
- monitoring and metrics interfaces
|
|
28
|
-
- Rails integration adapters
|
|
29
|
-
- ordered, locked migrations with durable version/checksum tracking
|
|
30
|
-
|
|
31
|
-
## Tested features
|
|
32
|
-
|
|
33
|
-
The current suite verifies storage reopen, subprocess crash recovery, MVCC isolation/vacuum, durable visibility version-history traversal, constraints including ON DELETE/ON UPDATE referential actions and nullable values, indexes including deep B-tree splits, SQL execution including idempotent database/table/index DDL and SQL foreign-key actions, schema DDL, views, trigger DDL and dispatch, CREATE TABLE, ALTER TABLE column and constraint changes, transaction before-image rollback and SQL savepoints, and VACUUM, live TCP sessions, TLS transport, password/SCRAM authentication and authorization, metrics updates including Prometheus export, liveness/readiness health reporting through the server request router, live CLI doctor checks with safe repair behavior, truthful CLI status reporting, live CLI branch diff/merge/checkout operations, live CLI inspection values, live CLI snapshot creation/listing and vacuum reporting, live CLI backup creation and restore dry-run validation, live CLI database creation and deletion, the documented SQL compatibility contract, full/snapshot/incremental/differential backup validation, logical replication, durable replica state, guarded failover promotion, durable fencing epochs, replication peer-token authentication, recovery resource checks, migration/schema-diff behavior including executable migration SQL serialization and changed-migration detection, atomic branch checkout, branch state/diff/merge behavior, foreign-key integrity lookup behavior, compound/null/boolean check-constraint evaluation, lock conflict/wait/timeout behavior, complete standalone and transaction-manager deadlock cycle detection with victim rollback, concurrency mutex initialization, Rails schema-builder SQL generation including defaults and foreign-key conventions, engine block-based table creation, adapter schema dumps preserving primary keys and defaults, ActiveRecord hash-form ordering and populated-table migration round trips, bounded multi-process workload supervision with child reaping, combined production traffic/deadline/cancellation/capacity/deadlock soak validation, release configuration, release tag/changelog preflight, upgrade guards, benchmark execution, executor query deadlines, wire-level cancellation of in-flight requests, standalone SCRAM verification, actual WAL checkpoint sizing, typed deserialization-corruption detection, preservation of stored false values when defaults are present, filesystem fault injection with descriptor cleanup, compaction/reopen validation, process-level replication failover with stale-writer fencing, and live TCP network-partition catch-up and promotion (268 examples, 0 failures at the latest audit).
|
|
34
|
-
|
|
35
|
-
## Known limitations
|
|
36
|
-
|
|
37
|
-
- The concurrent workload and regression suite verify parallel appends, point
|
|
38
|
-
reads, scans, durable reopen reads, multi-process server clients, and a
|
|
39
|
-
process-level replication failover drill. This does not certify universal
|
|
40
|
-
capacity; embedded ownership remains exclusive and multi-process clients must
|
|
41
|
-
use the server.
|
|
42
|
-
|
|
43
|
-
- multi-row SQL `VALUES` inserts execute through the parser, binder, planner,
|
|
44
|
-
and executor, including per-row conflict handling and Rails insert IDs
|
|
45
|
-
- incremental backups capture WAL mutations after a verified base LSN; differential backups capture the verified base-relative WAL delta and restore through the same validated delta path
|
|
46
|
-
- replication is limited to the explicit logical row-mutation envelope API;
|
|
47
|
-
process replacement, live TCP partition/catch-up, and same-host fencing are
|
|
48
|
-
tested, while true multi-host partitions and split-brain recovery remain
|
|
49
|
-
deployment work
|
|
50
|
-
- automatic failover is limited to synchronized candidates; fencing requires a shared durable fence path and still needs multi-host split-brain validation
|
|
51
|
-
- the SQLite compatibility profile covers the documented common application
|
|
52
|
-
surface, but it is not SQLite file-format or extension compatibility
|
|
53
|
-
- production CA lifecycle, independent security review, performance targets,
|
|
54
|
-
and large-scale workload behavior still require dedicated validation
|
|
55
|
-
- branch state application and live target-branch merges are supported through the engine reconciliation hook
|
|
56
|
-
|
|
57
|
-
## Unsupported SQL
|
|
58
|
-
|
|
59
|
-
The supported dialect is defined in [docs/sql/compatibility.md](sql/compatibility.md) and enforced by parser/execution integration coverage. RubyDB provides a tested common SQLite-style profile, not complete SQLite file-format, extension, pragma, or error compatibility.
|
|
60
|
-
|
|
61
|
-
## Durability guarantees
|
|
62
|
-
|
|
63
|
-
WAL framing, reopen behavior, subprocess crash-recovery scenarios, actual checkpoint sizing, fail-closed visibility-map loading, filesystem fault injection, compaction/reopen behavior, networked replication failover, and verified restore drills are tested. This is not a substitute for real quota, power-loss, and filesystem-specific validation.
|
|
64
|
-
|
|
65
|
-
## Transaction guarantees
|
|
66
|
-
|
|
67
|
-
Transaction commit/rollback, MVCC visibility, snapshot isolation behavior, and vacuum safe-point handling are covered by focused tests. High-contention workload validation remains outstanding.
|
|
68
|
-
|
|
69
|
-
## Isolation guarantees
|
|
70
|
-
|
|
71
|
-
Read committed, repeatable read, and serializable conflict behavior are covered by focused tests; broader workload and distributed-isolation validation remains outstanding.
|
|
72
|
-
|
|
73
|
-
## Backup guarantees
|
|
74
|
-
|
|
75
|
-
Full compressed backups restore into a fresh directory and checksum tampering is detected. Incremental and differential backups validate the base, checksum the change set, and apply WAL row mutations during restore. Production-scale backup-chain and filesystem fault-injection testing remains outstanding.
|
|
76
|
-
|
|
77
|
-
## Replication guarantees
|
|
78
|
-
|
|
79
|
-
Logical row-mutation streaming, LSN deduplication, acknowledgments, acknowledged-LSN reporting, synchronized-candidate promotion verification, process replacement, live TCP partition/catch-up, and stale-primary rejection through durable fencing epochs are tested. Multi-host fencing and split-brain recovery still require dedicated validation.
|
|
80
|
-
|
|
81
|
-
## TLS guarantees
|
|
82
|
-
|
|
83
|
-
TLS 1.2 or newer is enforced when enabled. Certificate/key parsing, CA-file validation, TLS client connections, and the end-to-end SSL transport path are tested. Mutual TLS is configurable through client certificates; certificate rotation and production CA lifecycle procedures remain operational work.
|
|
84
|
-
|
|
85
|
-
## Security model
|
|
86
|
-
|
|
87
|
-
Password and SCRAM-SHA-256 authentication, server-signature verification, read/write authorization, and startup validation of incomplete auth configuration are tested through the server/session path.
|
|
88
|
-
|
|
89
|
-
## Performance characteristics
|
|
90
|
-
|
|
91
|
-
The repository includes a deterministic storage benchmark (`RUBYDB_BENCHMARK_ITERATIONS=100 ruby -Ilib benchmarks/basic_workload.rb`) and a concurrent write/durability workload (`ruby benchmarks/concurrent_workload.rb`). Both emit machine-readable JSON. Production-scale throughput, WAL, checkpoint, and concurrency targets still require workload-specific baselines.
|
|
92
|
-
|
|
93
|
-
## Deployment requirements
|
|
94
|
-
|
|
95
|
-
The project currently needs:
|
|
96
|
-
|
|
97
|
-
- a validated Ruby support matrix
|
|
98
|
-
- RubyGems publication credentials (`RUBYGEMS_API_KEY`) configured as a protected
|
|
99
|
-
GitHub Actions secret before pushing a release tag
|
|
100
|
-
- a documented operator runbook for backups, upgrades, rollback, and incident preservation
|
|
101
|
-
- explicit durability and crash-recovery validation
|
|
102
|
-
- secure-by-default configuration development
|
|
103
|
-
- a narrower set of supported features documented honestly
|
|
104
|
-
|
|
105
|
-
## Remaining roadmap items
|
|
106
|
-
|
|
107
|
-
Main remaining work is to complete the sequence laid out in the repository design:
|
|
108
|
-
|
|
109
|
-
1. production CA lifecycle and certificate rotation
|
|
110
|
-
2. broader SQL compatibility contract and workload benchmarks
|
|
111
|
-
3. production-grade failover fencing and broader SQL compatibility
|
|
112
|
-
4. operator runbooks and deployment validation
|
|
113
|
-
|
|
114
|
-
## Bottom line
|
|
115
|
-
|
|
116
|
-
RubyDB has a validated production-oriented foundation, but it does not yet meet
|
|
117
|
-
the bar for unrestricted production use with real data outside its documented
|
|
118
|
-
feature set and deployment-specific validation.
|
|
1
|
+
# Production-readiness audit
|
|
2
|
+
|
|
3
|
+
## Status
|
|
4
|
+
|
|
5
|
+
RubyDB has verified production-oriented foundations, but it is not yet a general-purpose production database. Supported behavior is backed by the RSpec suite in `spec/`.
|
|
6
|
+
|
|
7
|
+
Latest local audit: 270 examples, 0 failures. Hosted CI and deployment-specific
|
|
8
|
+
evidence remain separate release gates.
|
|
9
|
+
|
|
10
|
+
The feature inventory below was written during an earlier checkpoint and its
|
|
11
|
+
parenthetical historical count is not the current total. Use the latest audit
|
|
12
|
+
line above and rerun the suite for release evidence.
|
|
13
|
+
|
|
14
|
+
## Implemented features
|
|
15
|
+
|
|
16
|
+
The repository contains substantial scaffolding for:
|
|
17
|
+
|
|
18
|
+
- page-based storage
|
|
19
|
+
- buffer pool caching
|
|
20
|
+
- file and page management
|
|
21
|
+
- WAL, recovery, and checkpoint modules
|
|
22
|
+
- transactions and MVCC abstractions
|
|
23
|
+
- SQL parser/planner infrastructure
|
|
24
|
+
- server, protocol, and connection components
|
|
25
|
+
- authentication and authorization modules
|
|
26
|
+
- backup and replication APIs
|
|
27
|
+
- monitoring and metrics interfaces
|
|
28
|
+
- Rails integration adapters
|
|
29
|
+
- ordered, locked migrations with durable version/checksum tracking
|
|
30
|
+
|
|
31
|
+
## Tested features
|
|
32
|
+
|
|
33
|
+
The current suite verifies storage reopen, subprocess crash recovery, MVCC isolation/vacuum, durable visibility version-history traversal, constraints including ON DELETE/ON UPDATE referential actions and nullable values, indexes including deep B-tree splits, SQL execution including idempotent database/table/index DDL and SQL foreign-key actions, schema DDL, views, trigger DDL and dispatch, CREATE TABLE, ALTER TABLE column and constraint changes, transaction before-image rollback and SQL savepoints, and VACUUM, live TCP sessions, TLS transport, password/SCRAM authentication and authorization, metrics updates including Prometheus export, liveness/readiness health reporting through the server request router, live CLI doctor checks with safe repair behavior, truthful CLI status reporting, live CLI branch diff/merge/checkout operations, live CLI inspection values, live CLI snapshot creation/listing and vacuum reporting, live CLI backup creation and restore dry-run validation, live CLI database creation and deletion, the documented SQL compatibility contract, full/snapshot/incremental/differential backup validation, logical replication, durable replica state, guarded failover promotion, durable fencing epochs, replication peer-token authentication, recovery resource checks, migration/schema-diff behavior including executable migration SQL serialization and changed-migration detection, atomic branch checkout, branch state/diff/merge behavior, foreign-key integrity lookup behavior, compound/null/boolean check-constraint evaluation, lock conflict/wait/timeout behavior, complete standalone and transaction-manager deadlock cycle detection with victim rollback, concurrency mutex initialization, Rails schema-builder SQL generation including defaults and foreign-key conventions, engine block-based table creation, adapter schema dumps preserving primary keys and defaults, ActiveRecord hash-form ordering and populated-table migration round trips, bounded multi-process workload supervision with child reaping, combined production traffic/deadline/cancellation/capacity/deadlock soak validation, release configuration, release tag/changelog preflight, upgrade guards, benchmark execution, executor query deadlines, wire-level cancellation of in-flight requests, standalone SCRAM verification, actual WAL checkpoint sizing, typed deserialization-corruption detection, preservation of stored false values when defaults are present, filesystem fault injection with descriptor cleanup, compaction/reopen validation, process-level replication failover with stale-writer fencing, and live TCP network-partition catch-up and promotion (268 examples, 0 failures at the latest audit).
|
|
34
|
+
|
|
35
|
+
## Known limitations
|
|
36
|
+
|
|
37
|
+
- The concurrent workload and regression suite verify parallel appends, point
|
|
38
|
+
reads, scans, durable reopen reads, multi-process server clients, and a
|
|
39
|
+
process-level replication failover drill. This does not certify universal
|
|
40
|
+
capacity; embedded ownership remains exclusive and multi-process clients must
|
|
41
|
+
use the server.
|
|
42
|
+
|
|
43
|
+
- multi-row SQL `VALUES` inserts execute through the parser, binder, planner,
|
|
44
|
+
and executor, including per-row conflict handling and Rails insert IDs
|
|
45
|
+
- incremental backups capture WAL mutations after a verified base LSN; differential backups capture the verified base-relative WAL delta and restore through the same validated delta path
|
|
46
|
+
- replication is limited to the explicit logical row-mutation envelope API;
|
|
47
|
+
process replacement, live TCP partition/catch-up, and same-host fencing are
|
|
48
|
+
tested, while true multi-host partitions and split-brain recovery remain
|
|
49
|
+
deployment work
|
|
50
|
+
- automatic failover is limited to synchronized candidates; fencing requires a shared durable fence path and still needs multi-host split-brain validation
|
|
51
|
+
- the SQLite compatibility profile covers the documented common application
|
|
52
|
+
surface, but it is not SQLite file-format or extension compatibility
|
|
53
|
+
- production CA lifecycle, independent security review, performance targets,
|
|
54
|
+
and large-scale workload behavior still require dedicated validation
|
|
55
|
+
- branch state application and live target-branch merges are supported through the engine reconciliation hook
|
|
56
|
+
|
|
57
|
+
## Unsupported SQL
|
|
58
|
+
|
|
59
|
+
The supported dialect is defined in [docs/sql/compatibility.md](sql/compatibility.md) and enforced by parser/execution integration coverage. RubyDB provides a tested common SQLite-style profile, not complete SQLite file-format, extension, pragma, or error compatibility.
|
|
60
|
+
|
|
61
|
+
## Durability guarantees
|
|
62
|
+
|
|
63
|
+
WAL framing, reopen behavior, subprocess crash-recovery scenarios, actual checkpoint sizing, fail-closed visibility-map loading, filesystem fault injection, compaction/reopen behavior, networked replication failover, and verified restore drills are tested. This is not a substitute for real quota, power-loss, and filesystem-specific validation.
|
|
64
|
+
|
|
65
|
+
## Transaction guarantees
|
|
66
|
+
|
|
67
|
+
Transaction commit/rollback, MVCC visibility, snapshot isolation behavior, and vacuum safe-point handling are covered by focused tests. High-contention workload validation remains outstanding.
|
|
68
|
+
|
|
69
|
+
## Isolation guarantees
|
|
70
|
+
|
|
71
|
+
Read committed, repeatable read, and serializable conflict behavior are covered by focused tests; broader workload and distributed-isolation validation remains outstanding.
|
|
72
|
+
|
|
73
|
+
## Backup guarantees
|
|
74
|
+
|
|
75
|
+
Full compressed backups restore into a fresh directory and checksum tampering is detected. Incremental and differential backups validate the base, checksum the change set, and apply WAL row mutations during restore. Production-scale backup-chain and filesystem fault-injection testing remains outstanding.
|
|
76
|
+
|
|
77
|
+
## Replication guarantees
|
|
78
|
+
|
|
79
|
+
Logical row-mutation streaming, LSN deduplication, acknowledgments, acknowledged-LSN reporting, synchronized-candidate promotion verification, process replacement, live TCP partition/catch-up, and stale-primary rejection through durable fencing epochs are tested. Multi-host fencing and split-brain recovery still require dedicated validation.
|
|
80
|
+
|
|
81
|
+
## TLS guarantees
|
|
82
|
+
|
|
83
|
+
TLS 1.2 or newer is enforced when enabled. Certificate/key parsing, CA-file validation, TLS client connections, and the end-to-end SSL transport path are tested. Mutual TLS is configurable through client certificates; certificate rotation and production CA lifecycle procedures remain operational work.
|
|
84
|
+
|
|
85
|
+
## Security model
|
|
86
|
+
|
|
87
|
+
Password and SCRAM-SHA-256 authentication, server-signature verification, read/write authorization, and startup validation of incomplete auth configuration are tested through the server/session path.
|
|
88
|
+
|
|
89
|
+
## Performance characteristics
|
|
90
|
+
|
|
91
|
+
The repository includes a deterministic storage benchmark (`RUBYDB_BENCHMARK_ITERATIONS=100 ruby -Ilib benchmarks/basic_workload.rb`) and a concurrent write/durability workload (`ruby benchmarks/concurrent_workload.rb`). Both emit machine-readable JSON. Production-scale throughput, WAL, checkpoint, and concurrency targets still require workload-specific baselines.
|
|
92
|
+
|
|
93
|
+
## Deployment requirements
|
|
94
|
+
|
|
95
|
+
The project currently needs:
|
|
96
|
+
|
|
97
|
+
- a validated Ruby support matrix
|
|
98
|
+
- RubyGems publication credentials (`RUBYGEMS_API_KEY`) configured as a protected
|
|
99
|
+
GitHub Actions secret before pushing a release tag
|
|
100
|
+
- a documented operator runbook for backups, upgrades, rollback, and incident preservation
|
|
101
|
+
- explicit durability and crash-recovery validation
|
|
102
|
+
- secure-by-default configuration development
|
|
103
|
+
- a narrower set of supported features documented honestly
|
|
104
|
+
|
|
105
|
+
## Remaining roadmap items
|
|
106
|
+
|
|
107
|
+
Main remaining work is to complete the sequence laid out in the repository design:
|
|
108
|
+
|
|
109
|
+
1. production CA lifecycle and certificate rotation
|
|
110
|
+
2. broader SQL compatibility contract and workload benchmarks
|
|
111
|
+
3. production-grade failover fencing and broader SQL compatibility
|
|
112
|
+
4. operator runbooks and deployment validation
|
|
113
|
+
|
|
114
|
+
## Bottom line
|
|
115
|
+
|
|
116
|
+
RubyDB has a validated production-oriented foundation, but it does not yet meet
|
|
117
|
+
the bar for unrestricted production use with real data outside its documented
|
|
118
|
+
feature set and deployment-specific validation.
|