rubydb 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/.gitattributes +20 -0
- data/.github/ISSUE_TEMPLATE/bug.yml +0 -0
- data/.github/ISSUE_TEMPLATE/feature.yml +0 -0
- data/.github/ISSUE_TEMPLATE/performance.yml +0 -0
- data/.github/ISSUE_TEMPLATE/security.yml +0 -0
- data/.github/PULL_REQUEST_TEMPLATE.md +15 -0
- data/.github/dependabot.yml +0 -0
- data/.github/workflows/benchmark.yml +26 -0
- data/.github/workflows/compatibility.yml +63 -0
- data/.github/workflows/fuzz.yml +33 -0
- data/.github/workflows/lint.yml +21 -0
- data/.github/workflows/operations.yml +24 -0
- data/.github/workflows/production-validation.yml +111 -0
- data/.github/workflows/release.yml +77 -0
- data/.github/workflows/security.yml +37 -0
- data/.github/workflows/test.yml +26 -0
- data/.github/workflows/workload.yml +58 -0
- data/.gitignore +43 -0
- data/.rspec +4 -0
- data/.rubocop.yml +44 -0
- data/.ruby-version +1 -0
- data/.standard.yml +14 -0
- data/ARCHITECTURE.md +21 -0
- data/CHANGELOG.md +31 -0
- data/CODE_OF_CONDUCT.md +13 -0
- data/CONTRIBUTING.md +29 -0
- data/GOVERNANCE.md +16 -0
- data/Gemfile +17 -0
- data/Gemfile.lock +71 -0
- data/LICENSE +60 -0
- data/README.md +162 -0
- data/ROADMAP.md +27 -0
- data/Rakefile +71 -0
- data/SECURITY.md +54 -0
- data/SUPPORT.md +14 -0
- data/adapters/activerecord/Gemfile +11 -0
- data/adapters/activerecord/README.md +9 -0
- data/adapters/activerecord/lib/active_record/connection_adapters/rubydb_adapter.rb +887 -0
- data/adapters/activerecord/rubydb-activerecord.gemspec +27 -0
- data/adapters/activerecord/spec/rubydb_adapter_integration_spec.rb +151 -0
- data/adapters/ruby/README.md +18 -0
- data/adapters/sequel/README.md +11 -0
- data/bin/rubydb +19 -0
- data/bin/rubydb-server +126 -0
- data/config/development.yml +0 -0
- data/config/monitoring/prometheus-alerts.yml +39 -0
- data/config/production.yml +36 -0
- data/config/test.yml +0 -0
- data/docs/README.md +71 -0
- data/docs/architecture/concurrency.md +14 -0
- data/docs/architecture/current-state.md +125 -0
- data/docs/architecture/execution-engine.md +25 -0
- data/docs/architecture/indexes.md +19 -0
- data/docs/architecture/mvcc.md +19 -0
- data/docs/architecture/overview.md +13 -0
- data/docs/architecture/pages.md +11 -0
- data/docs/architecture/production-roadmap.md +82 -0
- data/docs/architecture/query-planner.md +20 -0
- data/docs/architecture/recovery.md +18 -0
- data/docs/architecture/sql-engine.md +12 -0
- data/docs/architecture/storage-engine.md +14 -0
- data/docs/architecture/transactions.md +10 -0
- data/docs/architecture/wal.md +28 -0
- data/docs/cli-cheatsheet.md +98 -0
- data/docs/cli.md +275 -0
- data/docs/contributing/architecture.md +9 -0
- data/docs/contributing/benchmarking.md +14 -0
- data/docs/contributing/development.md +16 -0
- data/docs/contributing/release-process.md +49 -0
- data/docs/contributing/testing.md +16 -0
- data/docs/debugging.md +229 -0
- data/docs/developer/branching.md +10 -0
- data/docs/developer/database-diff.md +10 -0
- data/docs/developer/local-development.md +17 -0
- data/docs/developer/snapshots.md +9 -0
- data/docs/developer/temporal-data.md +10 -0
- data/docs/developer-guide.md +297 -0
- data/docs/getting-started/first-database.md +16 -0
- data/docs/getting-started/first-query.md +13 -0
- data/docs/getting-started/installation.md +19 -0
- data/docs/getting-started/local-to-production.md +300 -0
- data/docs/getting-started/quickstart.md +17 -0
- data/docs/getting-started/rails.md +16 -0
- data/docs/hardening_backlog.md +93 -0
- data/docs/lessons-learned.md +112 -0
- data/docs/operations/backups.md +33 -0
- data/docs/operations/disaster-recovery.md +31 -0
- data/docs/operations/failover.md +30 -0
- data/docs/operations/monitoring.md +25 -0
- data/docs/operations/production-guide.md +295 -0
- data/docs/operations/production-runbook.md +45 -0
- data/docs/operations/replication.md +33 -0
- data/docs/operations/restore.md +6 -0
- data/docs/operations/runbook.md +34 -0
- data/docs/operations/upgrades.md +14 -0
- data/docs/operations/workload-testing.md +17 -0
- data/docs/production-readiness.md +118 -0
- data/docs/production_validation.md +150 -0
- data/docs/rails/active-record.md +11 -0
- data/docs/rails/compatibility-guide.md +90 -0
- data/docs/rails/database-yml.md +92 -0
- data/docs/rails/installation.md +17 -0
- data/docs/rails/migrations.md +17 -0
- data/docs/rails/production.md +82 -0
- data/docs/rails/troubleshooting.md +18 -0
- data/docs/release.md +59 -0
- data/docs/server/architecture.md +10 -0
- data/docs/server/authentication.md +10 -0
- data/docs/server/configuration.md +16 -0
- data/docs/server/connection-pooling.md +10 -0
- data/docs/server/deployment.md +10 -0
- data/docs/server/protocol.md +12 -0
- data/docs/sql/compatibility-guide.md +82 -0
- data/docs/sql/compatibility.md +39 -0
- data/docs/sql/data-types.md +10 -0
- data/docs/sql/functions.md +9 -0
- data/docs/sql/joins.md +9 -0
- data/docs/sql/operators.md +9 -0
- data/docs/sql/sqlite-compatibility.md +21 -0
- data/docs/sql/syntax.md +10 -0
- data/docs/sql/transactions.md +10 -0
- data/docs/troubleshooting.md +244 -0
- data/exe/rubydb +23 -0
- data/lib/rubydb/backup/archive.rb +334 -0
- data/lib/rubydb/backup/backup.rb +401 -0
- data/lib/rubydb/backup/incremental.rb +353 -0
- data/lib/rubydb/backup/restore.rb +290 -0
- data/lib/rubydb/backup/snapshot.rb +267 -0
- data/lib/rubydb/backup/verification.rb +279 -0
- data/lib/rubydb/branching/branch.rb +181 -0
- data/lib/rubydb/branching/branch_manager.rb +311 -0
- data/lib/rubydb/branching/branch_metadata.rb +140 -0
- data/lib/rubydb/branching/checkout.rb +166 -0
- data/lib/rubydb/branching/copy_on_write.rb +272 -0
- data/lib/rubydb/branching/diff.rb +138 -0
- data/lib/rubydb/branching/merge.rb +285 -0
- data/lib/rubydb/build_info.rb +15 -0
- data/lib/rubydb/catalog/catalog.rb +391 -0
- data/lib/rubydb/catalog/column.rb +112 -0
- data/lib/rubydb/catalog/constraint.rb +180 -0
- data/lib/rubydb/catalog/database.rb +184 -0
- data/lib/rubydb/catalog/index.rb +97 -0
- data/lib/rubydb/catalog/schema.rb +103 -0
- data/lib/rubydb/catalog/sequence.rb +90 -0
- data/lib/rubydb/catalog/system_catalog.rb +698 -0
- data/lib/rubydb/catalog/table.rb +178 -0
- data/lib/rubydb/catalog/trigger.rb +102 -0
- data/lib/rubydb/catalog/view.rb +66 -0
- data/lib/rubydb/cli/application.rb +163 -0
- data/lib/rubydb/cli/commands/backup.rb +81 -0
- data/lib/rubydb/cli/commands/branch.rb +72 -0
- data/lib/rubydb/cli/commands/checkout.rb +54 -0
- data/lib/rubydb/cli/commands/create.rb +58 -0
- data/lib/rubydb/cli/commands/diff.rb +77 -0
- data/lib/rubydb/cli/commands/doctor.rb +74 -0
- data/lib/rubydb/cli/commands/drop.rb +57 -0
- data/lib/rubydb/cli/commands/init.rb +102 -0
- data/lib/rubydb/cli/commands/inspect.rb +95 -0
- data/lib/rubydb/cli/commands/merge.rb +63 -0
- data/lib/rubydb/cli/commands/migrate.rb +62 -0
- data/lib/rubydb/cli/commands/restart.rb +39 -0
- data/lib/rubydb/cli/commands/restore.rb +121 -0
- data/lib/rubydb/cli/commands/shell.rb +365 -0
- data/lib/rubydb/cli/commands/snapshot.rb +79 -0
- data/lib/rubydb/cli/commands/start.rb +82 -0
- data/lib/rubydb/cli/commands/status.rb +92 -0
- data/lib/rubydb/cli/commands/stop.rb +47 -0
- data/lib/rubydb/cli/commands/vacuum.rb +58 -0
- data/lib/rubydb/cli/formatter.rb +221 -0
- data/lib/rubydb/cli/output.rb +168 -0
- data/lib/rubydb/client/client.rb +304 -0
- data/lib/rubydb/client/connection.rb +415 -0
- data/lib/rubydb/client/connection_pool.rb +168 -0
- data/lib/rubydb/client/connection_url.rb +96 -0
- data/lib/rubydb/client/prepared_statement.rb +60 -0
- data/lib/rubydb/client/result.rb +123 -0
- data/lib/rubydb/client/statement.rb +52 -0
- data/lib/rubydb/client/transaction.rb +130 -0
- data/lib/rubydb/concurrency/concurrency.rb +19 -0
- data/lib/rubydb/concurrency/deadlock_detector.rb +150 -0
- data/lib/rubydb/concurrency/latch.rb +101 -0
- data/lib/rubydb/concurrency/lock_graph.rb +165 -0
- data/lib/rubydb/concurrency/mutex.rb +183 -0
- data/lib/rubydb/concurrency/rw_lock.rb +180 -0
- data/lib/rubydb/concurrency/scheduler.rb +250 -0
- data/lib/rubydb/concurrency/worker_pool.rb +143 -0
- data/lib/rubydb/configuration/config.rb +170 -0
- data/lib/rubydb/configuration/defaults.rb +179 -0
- data/lib/rubydb/configuration/environment.rb +152 -0
- data/lib/rubydb/configuration/parser.rb +185 -0
- data/lib/rubydb/configuration/validation.rb +221 -0
- data/lib/rubydb/constants.rb +74 -0
- data/lib/rubydb/constraints/check.rb +181 -0
- data/lib/rubydb/constraints/constraint.rb +101 -0
- data/lib/rubydb/constraints/foreign_key.rb +130 -0
- data/lib/rubydb/constraints/not_null.rb +64 -0
- data/lib/rubydb/constraints/primary_key.rb +99 -0
- data/lib/rubydb/constraints/unique.rb +108 -0
- data/lib/rubydb/constraints/validator.rb +350 -0
- data/lib/rubydb/errors/authentication_error.rb +10 -0
- data/lib/rubydb/errors/authorization_error.rb +23 -0
- data/lib/rubydb/errors/client_error.rb +10 -0
- data/lib/rubydb/errors/configuration_error.rb +10 -0
- data/lib/rubydb/errors/connection_error.rb +10 -0
- data/lib/rubydb/errors/constraint_error.rb +23 -0
- data/lib/rubydb/errors/corruption_error.rb +10 -0
- data/lib/rubydb/errors/database_error.rb +10 -0
- data/lib/rubydb/errors/error.rb +20 -0
- data/lib/rubydb/errors/execution_error.rb +10 -0
- data/lib/rubydb/errors/parser_error.rb +10 -0
- data/lib/rubydb/errors/recovery_error.rb +10 -0
- data/lib/rubydb/errors/replication_error.rb +10 -0
- data/lib/rubydb/errors/server_error.rb +6 -0
- data/lib/rubydb/errors/storage_error.rb +10 -0
- data/lib/rubydb/errors/transaction_error.rb +10 -0
- data/lib/rubydb/execution/aggregate_executor.rb +138 -0
- data/lib/rubydb/execution/delete_executor.rb +112 -0
- data/lib/rubydb/execution/distinct_executor.rb +135 -0
- data/lib/rubydb/execution/executor.rb +1192 -0
- data/lib/rubydb/execution/expression.rb +193 -0
- data/lib/rubydb/execution/index_scan.rb +142 -0
- data/lib/rubydb/execution/insert_executor.rb +217 -0
- data/lib/rubydb/execution/join_executor.rb +249 -0
- data/lib/rubydb/execution/limit_executor.rb +85 -0
- data/lib/rubydb/execution/optimizer.rb +215 -0
- data/lib/rubydb/execution/plan.rb +361 -0
- data/lib/rubydb/execution/planner.rb +546 -0
- data/lib/rubydb/execution/predicate.rb +235 -0
- data/lib/rubydb/execution/scan.rb +49 -0
- data/lib/rubydb/execution/sequential_scan.rb +63 -0
- data/lib/rubydb/execution/sort_executor.rb +185 -0
- data/lib/rubydb/execution/update_executor.rb +162 -0
- data/lib/rubydb/functions/aggregate.rb +70 -0
- data/lib/rubydb/functions/date_functions.rb +278 -0
- data/lib/rubydb/functions/function.rb +85 -0
- data/lib/rubydb/functions/json_functions.rb +215 -0
- data/lib/rubydb/functions/numeric_functions.rb +346 -0
- data/lib/rubydb/functions/scalar.rb +52 -0
- data/lib/rubydb/functions/string_functions.rb +383 -0
- data/lib/rubydb/functions/system_functions.rb +246 -0
- data/lib/rubydb/history/as_of.rb +238 -0
- data/lib/rubydb/history/change.rb +105 -0
- data/lib/rubydb/history/history.rb +131 -0
- data/lib/rubydb/history/history_manager.rb +229 -0
- data/lib/rubydb/history/temporal_query.rb +202 -0
- data/lib/rubydb/history/timeline.rb +144 -0
- data/lib/rubydb/indexes/btree.rb +186 -0
- data/lib/rubydb/indexes/btree_cursor.rb +258 -0
- data/lib/rubydb/indexes/btree_node.rb +385 -0
- data/lib/rubydb/indexes/hash_index.rb +150 -0
- data/lib/rubydb/indexes/index.rb +71 -0
- data/lib/rubydb/indexes/index_manager.rb +406 -0
- data/lib/rubydb/indexes/index_scan.rb +470 -0
- data/lib/rubydb/migrations/migration.rb +254 -0
- data/lib/rubydb/migrations/migration_lock.rb +146 -0
- data/lib/rubydb/migrations/migration_manager.rb +176 -0
- data/lib/rubydb/migrations/migration_version.rb +71 -0
- data/lib/rubydb/migrations/schema_diff.rb +211 -0
- data/lib/rubydb/migrations/schema_version.rb +64 -0
- data/lib/rubydb/monitoring/events.rb +160 -0
- data/lib/rubydb/monitoring/health.rb +222 -0
- data/lib/rubydb/monitoring/logger.rb +193 -0
- data/lib/rubydb/monitoring/metrics.rb +359 -0
- data/lib/rubydb/monitoring/performance.rb +176 -0
- data/lib/rubydb/monitoring/statistics.rb +170 -0
- data/lib/rubydb/mvcc/garbage_collector.rb +199 -0
- data/lib/rubydb/mvcc/mvcc.rb +16 -0
- data/lib/rubydb/mvcc/snapshot.rb +147 -0
- data/lib/rubydb/mvcc/vacuum.rb +180 -0
- data/lib/rubydb/mvcc/version.rb +106 -0
- data/lib/rubydb/mvcc/version_store.rb +398 -0
- data/lib/rubydb/mvcc/visibility.rb +109 -0
- data/lib/rubydb/protocol/capabilities.rb +125 -0
- data/lib/rubydb/protocol/decoder.rb +145 -0
- data/lib/rubydb/protocol/encoder.rb +136 -0
- data/lib/rubydb/protocol/handshake.rb +305 -0
- data/lib/rubydb/protocol/message.rb +121 -0
- data/lib/rubydb/protocol/protocol.rb +277 -0
- data/lib/rubydb/protocol/version.rb +54 -0
- data/lib/rubydb/rails/adapter.rb +239 -0
- data/lib/rubydb/rails/connection.rb +314 -0
- data/lib/rubydb/rails/database_statements.rb +122 -0
- data/lib/rubydb/rails/migration.rb +131 -0
- data/lib/rubydb/rails/quoting.rb +109 -0
- data/lib/rubydb/rails/result.rb +120 -0
- data/lib/rubydb/rails/schema_statements.rb +339 -0
- data/lib/rubydb/rails/transaction.rb +105 -0
- data/lib/rubydb/rails/type.rb +126 -0
- data/lib/rubydb/recovery/checkpoint.rb +257 -0
- data/lib/rubydb/recovery/consistency.rb +467 -0
- data/lib/rubydb/recovery/corruption_detector.rb +5 -0
- data/lib/rubydb/recovery/crash_recovery.rb +387 -0
- data/lib/rubydb/recovery/recovery_manager.rb +206 -0
- data/lib/rubydb/recovery/redo.rb +237 -0
- data/lib/rubydb/recovery/undo.rb +206 -0
- data/lib/rubydb/replication/failover.rb +5 -0
- data/lib/rubydb/replication/fencing.rb +63 -0
- data/lib/rubydb/replication/primary.rb +450 -0
- data/lib/rubydb/replication/replica.rb +384 -0
- data/lib/rubydb/replication/replication_log.rb +200 -0
- data/lib/rubydb/replication/replication_manager.rb +308 -0
- data/lib/rubydb/replication/replication_slot.rb +295 -0
- data/lib/rubydb/replication/replication_stream.rb +201 -0
- data/lib/rubydb/rubydb.rb +560 -0
- data/lib/rubydb/security/access_control.rb +254 -0
- data/lib/rubydb/security/audit_log.rb +213 -0
- data/lib/rubydb/security/authentication.rb +302 -0
- data/lib/rubydb/security/authorization.rb +282 -0
- data/lib/rubydb/security/credentials.rb +196 -0
- data/lib/rubydb/security/password.rb +215 -0
- data/lib/rubydb/security/permissions.rb +74 -0
- data/lib/rubydb/security/role.rb +101 -0
- data/lib/rubydb/security/user.rb +86 -0
- data/lib/rubydb/server/connection.rb +366 -0
- data/lib/rubydb/server/connection_pool.rb +193 -0
- data/lib/rubydb/server/lifecycle.rb +228 -0
- data/lib/rubydb/server/listener.rb +136 -0
- data/lib/rubydb/server/request_handler.rb +276 -0
- data/lib/rubydb/server/server.rb +364 -0
- data/lib/rubydb/server/session.rb +369 -0
- data/lib/rubydb/server/worker.rb +210 -0
- data/lib/rubydb/server/worker_pool.rb +168 -0
- data/lib/rubydb/sql/ast/alter_table.rb +169 -0
- data/lib/rubydb/sql/ast/begin_transaction.rb +47 -0
- data/lib/rubydb/sql/ast/commit.rb +37 -0
- data/lib/rubydb/sql/ast/constraint.rb +83 -0
- data/lib/rubydb/sql/ast/create_database.rb +41 -0
- data/lib/rubydb/sql/ast/create_index.rb +61 -0
- data/lib/rubydb/sql/ast/create_schema.rb +52 -0
- data/lib/rubydb/sql/ast/create_table.rb +187 -0
- data/lib/rubydb/sql/ast/delete.rb +54 -0
- data/lib/rubydb/sql/ast/drop_database.rb +41 -0
- data/lib/rubydb/sql/ast/drop_index.rb +41 -0
- data/lib/rubydb/sql/ast/drop_schema.rb +49 -0
- data/lib/rubydb/sql/ast/drop_table.rb +49 -0
- data/lib/rubydb/sql/ast/explain.rb +64 -0
- data/lib/rubydb/sql/ast/expression.rb +604 -0
- data/lib/rubydb/sql/ast/insert.rb +82 -0
- data/lib/rubydb/sql/ast/node.rb +42 -0
- data/lib/rubydb/sql/ast/rollback.rb +63 -0
- data/lib/rubydb/sql/ast/savepoint.rb +59 -0
- data/lib/rubydb/sql/ast/select.rb +88 -0
- data/lib/rubydb/sql/ast/set_operation.rb +20 -0
- data/lib/rubydb/sql/ast/trigger.rb +29 -0
- data/lib/rubydb/sql/ast/update.rb +88 -0
- data/lib/rubydb/sql/ast/vacuum.rb +19 -0
- data/lib/rubydb/sql/ast/view.rb +32 -0
- data/lib/rubydb/sql/ast/with.rb +32 -0
- data/lib/rubydb/sql/grammar.rb +86 -0
- data/lib/rubydb/sql/keywords.rb +156 -0
- data/lib/rubydb/sql/lexer.rb +214 -0
- data/lib/rubydb/sql/operators.rb +100 -0
- data/lib/rubydb/sql/parser.rb +1185 -0
- data/lib/rubydb/sql/planner/analyzer.rb +302 -0
- data/lib/rubydb/sql/planner/binder.rb +550 -0
- data/lib/rubydb/sql/planner/type_checker.rb +431 -0
- data/lib/rubydb/sql/token.rb +210 -0
- data/lib/rubydb/storage/buffer_frame.rb +44 -0
- data/lib/rubydb/storage/buffer_pool.rb +155 -0
- data/lib/rubydb/storage/database_lock.rb +74 -0
- data/lib/rubydb/storage/deserializer.rb +342 -0
- data/lib/rubydb/storage/engine.rb +2338 -0
- data/lib/rubydb/storage/file_manager.rb +187 -0
- data/lib/rubydb/storage/free_space_map.rb +81 -0
- data/lib/rubydb/storage/page.rb +94 -0
- data/lib/rubydb/storage/page_allocator.rb +855 -0
- data/lib/rubydb/storage/page_header.rb +67 -0
- data/lib/rubydb/storage/page_manager.rb +131 -0
- data/lib/rubydb/storage/record.rb +58 -0
- data/lib/rubydb/storage/row.rb +78 -0
- data/lib/rubydb/storage/serializer.rb +51 -0
- data/lib/rubydb/storage/storage_layout.rb +151 -0
- data/lib/rubydb/storage/storage_manager.rb +114 -0
- data/lib/rubydb/storage/tuple.rb +461 -0
- data/lib/rubydb/storage/visibility_map.rb +984 -0
- data/lib/rubydb/transactions/commit_manager.rb +220 -0
- data/lib/rubydb/transactions/isolation.rb +98 -0
- data/lib/rubydb/transactions/lock.rb +76 -0
- data/lib/rubydb/transactions/lock_manager.rb +362 -0
- data/lib/rubydb/transactions/savepoint.rb +143 -0
- data/lib/rubydb/transactions/transaction.rb +215 -0
- data/lib/rubydb/transactions/transaction_id.rb +84 -0
- data/lib/rubydb/transactions/transaction_log.rb +257 -0
- data/lib/rubydb/transactions/transaction_manager.rb +435 -0
- data/lib/rubydb/types/bigint.rb +36 -0
- data/lib/rubydb/types/blob.rb +37 -0
- data/lib/rubydb/types/boolean.rb +34 -0
- data/lib/rubydb/types/date.rb +39 -0
- data/lib/rubydb/types/decimal.rb +48 -0
- data/lib/rubydb/types/float.rb +34 -0
- data/lib/rubydb/types/integer.rb +36 -0
- data/lib/rubydb/types/json.rb +41 -0
- data/lib/rubydb/types/null.rb +34 -0
- data/lib/rubydb/types/smallint.rb +36 -0
- data/lib/rubydb/types/text.rb +37 -0
- data/lib/rubydb/types/time.rb +46 -0
- data/lib/rubydb/types/timestamp.rb +39 -0
- data/lib/rubydb/types/type.rb +119 -0
- data/lib/rubydb/types/uuid.rb +47 -0
- data/lib/rubydb/types/varchar.rb +37 -0
- data/lib/rubydb/version.rb +33 -0
- data/lib/rubydb/wal/archive.rb +193 -0
- data/lib/rubydb/wal/checkpoint.rb +183 -0
- data/lib/rubydb/wal/lsn.rb +94 -0
- data/lib/rubydb/wal/reader.rb +260 -0
- data/lib/rubydb/wal/record.rb +105 -0
- data/lib/rubydb/wal/segment.rb +193 -0
- data/lib/rubydb/wal/wal.rb +452 -0
- data/lib/rubydb/wal/writer.rb +236 -0
- data/lib/rubydb.rb +7 -0
- data/packaging/docker/Dockerfile +36 -0
- data/packaging/docker/docker-compose.failover.yml +43 -0
- data/packaging/docker/entrypoint.sh +15 -0
- data/packaging/homebrew/rubydb.rb +19 -0
- data/packaging/systemd/rubydb.service +22 -0
- data/rubydb.gemspec +57 -0
- data/scripts/benchmark +7 -0
- data/scripts/durability_drill +37 -0
- data/scripts/format +0 -0
- data/scripts/fuzz +63 -0
- data/scripts/lint +0 -0
- data/scripts/release +50 -0
- data/scripts/release_check +43 -0
- data/scripts/replication_failover_drill +250 -0
- data/scripts/replication_network_failover_drill +255 -0
- data/scripts/restore_drill +45 -0
- data/scripts/security +0 -0
- data/scripts/setup +0 -0
- data/scripts/test +0 -0
- data/scripts/test-chaos +0 -0
- data/scripts/test-crash +0 -0
- data/scripts/test-integration +0 -0
- data/scripts/test-unit +0 -0
- data/visibility_map.json +1 -0
- metadata +538 -0
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
# Indexes
|
|
2
|
+
|
|
3
|
+
RubyDB provides B-tree indexes for supported table columns, including unique
|
|
4
|
+
indexes. The executor can use indexes for eligible lookups while preserving
|
|
5
|
+
correctness through table visibility and transaction rules.
|
|
6
|
+
|
|
7
|
+
Index metadata is persisted with the catalog. Index creation, DML maintenance,
|
|
8
|
+
rollback, deep splits, reopen, and failure handling must remain consistent. If
|
|
9
|
+
index recovery fails, startup must fail visibly; never silently rebuild or claim
|
|
10
|
+
an index is healthy without verification.
|
|
11
|
+
|
|
12
|
+
## Change checklist
|
|
13
|
+
|
|
14
|
+
Index changes require coverage for empty and populated tables, duplicate and
|
|
15
|
+
null keys, deep splits, deletes, rollback, crash/replay, compaction, and
|
|
16
|
+
concurrent readers. Compare an index lookup with a table scan before and after
|
|
17
|
+
reopen. A required-index persistence error must abort the owning operation and
|
|
18
|
+
be observable by the caller. See [troubleshooting](../troubleshooting.md) for
|
|
19
|
+
the safe response to suspected index corruption.
|
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
# MVCC support status
|
|
2
|
+
|
|
3
|
+
RubyDB currently provides safe READ COMMITTED behavior for the storage engine.
|
|
4
|
+
Uncommitted row versions are visible to their owning transaction and hidden
|
|
5
|
+
from other transactions. Committed versions become visible after the durable
|
|
6
|
+
COMMIT WAL record is written. Deleted physical records remain on pages until
|
|
7
|
+
vacuum/compaction and are excluded from ordinary scans.
|
|
8
|
+
|
|
9
|
+
REPEATABLE READ uses persisted historical row versions and transaction
|
|
10
|
+
snapshots. SERIALIZABLE uses snapshot validation over tracked row read/write
|
|
11
|
+
keys and conservative table predicates. A newer committed version intersecting
|
|
12
|
+
the dependency set aborts commit with a serialization failure. Table-level
|
|
13
|
+
predicate tracking prevents phantoms at the cost of false-positive conflicts;
|
|
14
|
+
exact index-range predicate locking is a future optimization.
|
|
15
|
+
|
|
16
|
+
The version store is persisted atomically and supports safe-point vacuuming.
|
|
17
|
+
Vacuum never removes active versions and retains the newest committed base
|
|
18
|
+
version needed by an active reader; older committed history is removable only
|
|
19
|
+
when its commit ID precedes the oldest active transaction.
|
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
# Architecture overview
|
|
2
|
+
|
|
3
|
+
RubyDB separates logical query processing from durable state changes. SQL is
|
|
4
|
+
lexed into tokens, parsed into an AST, bound against catalog metadata, planned,
|
|
5
|
+
and executed against the storage engine. DML is coordinated with transactions
|
|
6
|
+
and WAL before durable acknowledgement.
|
|
7
|
+
|
|
8
|
+
The catalog defines tables, columns, constraints, views, and indexes. The page
|
|
9
|
+
manager and buffer pool provide durable storage. Recovery replays valid WAL and
|
|
10
|
+
rejects malformed or uncertain state. Server/client mode isolates application
|
|
11
|
+
processes from the exclusive embedded owner.
|
|
12
|
+
|
|
13
|
+
Read the [current-state audit](current-state.md) before relying on any feature.
|
|
@@ -0,0 +1,11 @@
|
|
|
1
|
+
# Pages and file layout
|
|
2
|
+
|
|
3
|
+
RubyDB stores fixed-size pages containing validated headers, checksums, and
|
|
4
|
+
typed records. Page allocation, metadata, table data, and index pages are
|
|
5
|
+
managed by the storage layer; callers must not edit database files directly.
|
|
6
|
+
|
|
7
|
+
The page format is versioned. Unknown formats, invalid page sizes, malformed
|
|
8
|
+
headers, and checksum failures fail closed during open or recovery. Keep the
|
|
9
|
+
database, WAL, catalog metadata, and lock files together when copying or
|
|
10
|
+
restoring a database. See [storage format](../../spec/storage/format.md) and
|
|
11
|
+
[upgrade guidance](../operations/upgrades.md).
|
|
@@ -0,0 +1,82 @@
|
|
|
1
|
+
# RubyDB production roadmap
|
|
2
|
+
|
|
3
|
+
## Current baseline
|
|
4
|
+
|
|
5
|
+
The project has a meaningful repository skeleton and substantial design intent. It already includes modules for storage, indexing, transactions, MVCC, SQL parts, server, replication, and backup flows. However, the implementation still lacks enough end-to-end proof for durable, crash-safe operation.
|
|
6
|
+
|
|
7
|
+
## Prioritized roadmap
|
|
8
|
+
|
|
9
|
+
### Phase 0 — Repository stabilization
|
|
10
|
+
|
|
11
|
+
- fix library loading and Ruby compatibility metadata
|
|
12
|
+
- verify the project loads via `require "rubydb"`
|
|
13
|
+
- establish a truthful project health baseline
|
|
14
|
+
- add characterization tests for current public behavior
|
|
15
|
+
|
|
16
|
+
### Phase 1 — Storage durability
|
|
17
|
+
|
|
18
|
+
- define a stable on-disk page format and file format
|
|
19
|
+
- add checksums, header validation, and corruption detection
|
|
20
|
+
- ensure WAL durability precedes page flush semantics
|
|
21
|
+
- add real restart tests and crash tests
|
|
22
|
+
|
|
23
|
+
### Phase 2 — Transaction correctness
|
|
24
|
+
|
|
25
|
+
- complete transaction lifecycle handling
|
|
26
|
+
- prove commit/rollback semantics under restart
|
|
27
|
+
- validate isolation semantics and visibility rules
|
|
28
|
+
- add deadlock and lock manager tests
|
|
29
|
+
|
|
30
|
+
### Phase 3 — Catalog and indexes
|
|
31
|
+
|
|
32
|
+
- persist catalog metadata consistently
|
|
33
|
+
- maintain indexes correctly during DML
|
|
34
|
+
- verify index consistency after crash and rollback
|
|
35
|
+
- document supported SQL compatibility targets honestly
|
|
36
|
+
|
|
37
|
+
### Phase 4 — SQL execution and optimizer
|
|
38
|
+
|
|
39
|
+
- validate end-to-end SQL execution
|
|
40
|
+
- ensure binder, planner, and executor are integrated
|
|
41
|
+
- add integration tests for basic DML and queries
|
|
42
|
+
- implement `EXPLAIN` only with real, honest plan output
|
|
43
|
+
|
|
44
|
+
### Phase 5 — Server and security
|
|
45
|
+
|
|
46
|
+
- harden server lifecycle and malformed-client handling
|
|
47
|
+
- enforce connection limits and timeouts
|
|
48
|
+
- check authorization before execution
|
|
49
|
+
- validate auth flows and access controls
|
|
50
|
+
|
|
51
|
+
### Phase 6 — Backup, recovery, and replication
|
|
52
|
+
|
|
53
|
+
- prove full backup + restore works from a clean environment
|
|
54
|
+
- validate point-in-time or incremental recovery only when implemented correctly
|
|
55
|
+
- only add replication once WAL recovery is demonstrably correct
|
|
56
|
+
|
|
57
|
+
### Phase 7 — Monitoring and production readiness
|
|
58
|
+
|
|
59
|
+
- add structured logging and health/readiness checks
|
|
60
|
+
- expose operational metrics
|
|
61
|
+
- create a final production-readiness audit with limitations and unsupported features
|
|
62
|
+
|
|
63
|
+
## Hard gates before declaring production readiness
|
|
64
|
+
|
|
65
|
+
RubyDB should not claim production readiness until it demonstrates all of the following:
|
|
66
|
+
|
|
67
|
+
- updated data survives restart
|
|
68
|
+
- crash recovery is validated with subprocess termination tests
|
|
69
|
+
- transactions preserve isolation semantics
|
|
70
|
+
- WAL ordering is durable and replayed correctly
|
|
71
|
+
- index and table data stay consistent
|
|
72
|
+
- authZ/authN is enforced over real operations
|
|
73
|
+
- backup and restore work in a clean environment
|
|
74
|
+
- server operations handle failures safely without whole-process crashes
|
|
75
|
+
|
|
76
|
+
## Decision rule
|
|
77
|
+
|
|
78
|
+
The repository must prefer correctness over feature breadth. A smaller well-tested engine is more valuable than a broad but unsafe one.
|
|
79
|
+
|
|
80
|
+
## Long-term target
|
|
81
|
+
|
|
82
|
+
The final target is not a fake "PostgreSQL clone" or a marketing demo. The target is a Ruby-first, SQLite-like development experience paired with durable, crash-safe relational database semantics that can be validated under real workloads.
|
|
@@ -0,0 +1,20 @@
|
|
|
1
|
+
# Query planner
|
|
2
|
+
|
|
3
|
+
The planner binds identifiers to catalog columns, validates expressions, and
|
|
4
|
+
builds executable plans. It supports sequential/index scans, safe inner-join
|
|
5
|
+
reordering, outer joins, filters, grouping, sorting, limits, set operations,
|
|
6
|
+
subqueries, CTEs, and window operations in the documented SQL surface.
|
|
7
|
+
|
|
8
|
+
Plans must preserve SQL null, ordering, grouping, and transaction visibility
|
|
9
|
+
semantics. `EXPLAIN` reports the actual selected plan; it is not a performance
|
|
10
|
+
promise. Use the benchmark and workload harnesses to establish capacity on the
|
|
11
|
+
target hardware.
|
|
12
|
+
|
|
13
|
+
## Plan review
|
|
14
|
+
|
|
15
|
+
For every rewrite, compare the optimized and baseline results on empty,
|
|
16
|
+
duplicate, null, and concurrent data. Include joins with unmatched rows,
|
|
17
|
+
grouping, limits, and repeated execution with bound parameters. Measure plan
|
|
18
|
+
selection, execution, lock wait, and I/O separately. A plan that is faster but
|
|
19
|
+
changes cardinality or ordering is incorrect; add the minimized query as a
|
|
20
|
+
regression spec before merging.
|
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
# Recovery architecture
|
|
2
|
+
|
|
3
|
+
Recovery validates the database format, page headers, checksums, metadata, and
|
|
4
|
+
WAL before making state available. Valid committed WAL records may be replayed;
|
|
5
|
+
malformed or ambiguous records fail closed and require operator investigation.
|
|
6
|
+
|
|
7
|
+
Recovery procedures work from a preserved source or verified backup. Restore to
|
|
8
|
+
a new directory, run dry-run and checksum checks, reopen the restored engine,
|
|
9
|
+
compare schema/row evidence, and only then switch application traffic. See the
|
|
10
|
+
[disaster recovery](../operations/disaster-recovery.md) procedure.
|
|
11
|
+
|
|
12
|
+
## Failure handling
|
|
13
|
+
|
|
14
|
+
Recovery is allowed to replay only validated, committed records. Truncated
|
|
15
|
+
frames, invalid checksums, impossible LSNs, and incomplete metadata must stop
|
|
16
|
+
startup with an actionable error. Preserve the original directory and WAL;
|
|
17
|
+
repair or compaction experiments belong on a copy. The [debugging playbook](../debugging.md)
|
|
18
|
+
lists the evidence to collect.
|
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
# SQL engine
|
|
2
|
+
|
|
3
|
+
RubyDB implements a documented RubyDB SQL subset with a tested common
|
|
4
|
+
SQLite-style application profile. Covered features include CRUD, constraints,
|
|
5
|
+
joins, grouping/aggregates, transactions/savepoints, CTEs, subqueries, set
|
|
6
|
+
operations, upserts, windows, indexes, views, and maintenance statements listed
|
|
7
|
+
in [SQL compatibility](../sql/compatibility.md).
|
|
8
|
+
|
|
9
|
+
This is not complete PostgreSQL, MySQL, or SQLite dialect/file-format
|
|
10
|
+
compatibility. Unsupported syntax must fail explicitly. Applications migrating
|
|
11
|
+
from another engine must run their own schema, query, migration, and error
|
|
12
|
+
behavior suite.
|
|
@@ -0,0 +1,14 @@
|
|
|
1
|
+
# Storage engine
|
|
2
|
+
|
|
3
|
+
The storage engine owns the database path, page manager, buffer pool, catalog,
|
|
4
|
+
indexes, WAL, and recovery lifecycle. It serializes durable changes and exposes
|
|
5
|
+
Ruby-native operations used by the SQL executor and adapters.
|
|
6
|
+
|
|
7
|
+
Embedded ownership is exclusive and enforced with an operating-system lock.
|
|
8
|
+
Multiple processes must use the server. On failure, callers should preserve the
|
|
9
|
+
original directory and recover into a new destination; do not delete WAL or
|
|
10
|
+
overwrite the source during investigation.
|
|
11
|
+
|
|
12
|
+
Storage changes require reopen, crash, fault-injection, corruption, compaction,
|
|
13
|
+
and backup/restore coverage. The [lessons learned](../lessons-learned.md) page
|
|
14
|
+
explains why these are separate guarantees.
|
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
# Transactions
|
|
2
|
+
|
|
3
|
+
Transactions group mutations into a commit or rollback boundary. WAL commit and
|
|
4
|
+
flush ordering precede durable acknowledgement. MVCC determines visibility,
|
|
5
|
+
while lock management protects conflicting writes and resolves wait-for cycles.
|
|
6
|
+
|
|
7
|
+
Supported controls include `BEGIN`, `COMMIT`, `ROLLBACK`, savepoints, and
|
|
8
|
+
rollback to savepoint. A deadlock victim is rolled back and must retry the whole
|
|
9
|
+
application unit as appropriate. Test isolation and recovery together; a green
|
|
10
|
+
single-thread transaction test does not prove concurrent correctness.
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
# RubyDB WAL format
|
|
2
|
+
|
|
3
|
+
Each WAL segment is a sequence of independently framed records:
|
|
4
|
+
|
|
5
|
+
```text
|
|
6
|
+
4-byte unsigned big-endian payload length
|
|
7
|
+
16-byte ASCII SHA-256 checksum prefix
|
|
8
|
+
UTF-8 JSON payload
|
|
9
|
+
```
|
|
10
|
+
|
|
11
|
+
The checksum covers the JSON payload only. A record's LSN offset is the byte
|
|
12
|
+
offset of its frame header within the segment, so readers can advance exactly
|
|
13
|
+
one frame at a time. If the final frame is incomplete, recovery stops at that
|
|
14
|
+
tail and retains all complete preceding records.
|
|
15
|
+
|
|
16
|
+
WAL data is flushed and fsynced before the storage engine flushes data pages
|
|
17
|
+
when synchronous WAL mode is enabled (the default). A successful `write` is
|
|
18
|
+
therefore durable only after the WAL sync completes; callers must still use
|
|
19
|
+
the engine's transaction/close APIs to coordinate higher-level durability.
|
|
20
|
+
|
|
21
|
+
## Transaction recovery records
|
|
22
|
+
|
|
23
|
+
Mutation records include the transaction ID. UPDATE records include the
|
|
24
|
+
previous column values, and DELETE records include the previous row values and
|
|
25
|
+
columns. Recovery replays mutations for transactions with a durable COMMIT
|
|
26
|
+
record. Transactions that have a BEGIN but no COMMIT are undone in reverse WAL
|
|
27
|
+
order. Recovery uses the existing row version for DELETE undo and does not
|
|
28
|
+
create a replacement row with a new identifier.
|
|
@@ -0,0 +1,98 @@
|
|
|
1
|
+
# RubyDB CLI cheat sheet
|
|
2
|
+
|
|
3
|
+
Replace paths and names before running. Put global options before the command.
|
|
4
|
+
|
|
5
|
+
## Global
|
|
6
|
+
|
|
7
|
+
```sh
|
|
8
|
+
rubydb --help
|
|
9
|
+
rubydb --version
|
|
10
|
+
rubydb --config config/production.yml --env production --no-color status --json
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
## Database lifecycle
|
|
14
|
+
|
|
15
|
+
```sh
|
|
16
|
+
rubydb init --name app --dir data
|
|
17
|
+
rubydb create --database data/app.rdb
|
|
18
|
+
rubydb status --json
|
|
19
|
+
rubydb doctor --quick --json
|
|
20
|
+
rubydb inspect --database data/app.rdb --stats --wal
|
|
21
|
+
rubydb drop --database data/app.rdb # prompts
|
|
22
|
+
rubydb drop --database data/app.rdb --force # destructive
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
## Server
|
|
26
|
+
|
|
27
|
+
```sh
|
|
28
|
+
rubydb --env production start --host 127.0.0.1 --port 7432 \
|
|
29
|
+
--data-dir /var/lib/rubydb/data --log-dir /var/log/rubydb \
|
|
30
|
+
--pid-file /run/rubydb.pid
|
|
31
|
+
rubydb stop --pid-file /run/rubydb.pid
|
|
32
|
+
rubydb restart --pid-file /run/rubydb.pid
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
## SQL shell
|
|
36
|
+
|
|
37
|
+
```sh
|
|
38
|
+
rubydb shell --database app --table
|
|
39
|
+
rubydb shell --database app --json
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
Inside shell: `.help` `.tables` `.table users` `.schema users` `.status`
|
|
43
|
+
`.begin` `.commit` `.rollback` `.explain SELECT ...` `.exit`.
|
|
44
|
+
|
|
45
|
+
## Migrations
|
|
46
|
+
|
|
47
|
+
```sh
|
|
48
|
+
rubydb migrate --database data/app.rdb --path db/migrate --dry-run
|
|
49
|
+
rubydb migrate --database data/app.rdb --path db/migrate
|
|
50
|
+
rubydb migrate --database data/app.rdb --path db/migrate --down --steps 1
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
## Backups and restore
|
|
54
|
+
|
|
55
|
+
```sh
|
|
56
|
+
rubydb backup --database data/app.rdb --dir backups --type full --compress
|
|
57
|
+
rubydb backup --database data/app.rdb --dir backups --type incremental
|
|
58
|
+
rubydb backup --database data/app.rdb --dir backups --type differential
|
|
59
|
+
rubydb restore --dir backups --latest --dry-run
|
|
60
|
+
rubydb restore --dir backups --latest
|
|
61
|
+
rubydb restore --dir backups --backup NAME --point-in-time 2026-09-09T12:00:00Z
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
## Snapshots and branches
|
|
65
|
+
|
|
66
|
+
```sh
|
|
67
|
+
rubydb snapshot --database data/app.rdb --dir snapshots --name before-change
|
|
68
|
+
rubydb snapshot --database data/app.rdb --dir snapshots --list
|
|
69
|
+
rubydb branch --database data/app.rdb --branch-dir branches --list
|
|
70
|
+
rubydb branch --database data/app.rdb --branch-dir branches --create feature --from main
|
|
71
|
+
rubydb checkout --database data/app.rdb --branch-dir branches feature
|
|
72
|
+
rubydb diff --database data/app.rdb --branch-dir branches main feature --summary
|
|
73
|
+
rubydb merge --database data/app.rdb --branch-dir branches feature --into main
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
## Maintenance and release checks
|
|
77
|
+
|
|
78
|
+
```sh
|
|
79
|
+
rubydb vacuum --database data/app.rdb --dry-run
|
|
80
|
+
rubydb vacuum --database data/app.rdb --full --analyze
|
|
81
|
+
ruby scripts/durability_drill
|
|
82
|
+
ruby scripts/replication_failover_drill
|
|
83
|
+
ruby scripts/replication_network_failover_drill
|
|
84
|
+
ruby scripts/release
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
## Safety rules
|
|
88
|
+
|
|
89
|
+
- Back up and verify before `migrate`, `merge`, `checkout`, `vacuum --full`, or
|
|
90
|
+
any restore.
|
|
91
|
+
- Treat `drop --force`, `create --force`, restore `--force`, and process `--force`
|
|
92
|
+
stop as destructive operations.
|
|
93
|
+
- Preserve database, WAL, lock, metadata, logs, and configuration together.
|
|
94
|
+
- Use one embedded owner; use server/client mode for multiple processes.
|
|
95
|
+
- Never put passwords, tokens, private keys, or API keys in commands committed
|
|
96
|
+
to source control or in captured output.
|
|
97
|
+
- A successful CLI exit is not a universal production certification; retain the
|
|
98
|
+
workload, restore, failover, CI, and security evidence.
|
data/docs/cli.md
ADDED
|
@@ -0,0 +1,275 @@
|
|
|
1
|
+
# RubyDB CLI guide
|
|
2
|
+
|
|
3
|
+
The `rubydb` command manages local databases, server processes, migrations,
|
|
4
|
+
backups, snapshots, branches, inspection, maintenance, and health checks. Run
|
|
5
|
+
`rubydb --help` or `rubydb <command> --help` for the installed command's live
|
|
6
|
+
options.
|
|
7
|
+
|
|
8
|
+
## Command shape
|
|
9
|
+
|
|
10
|
+
```text
|
|
11
|
+
rubydb [global options] <command> [command options]
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
Global options must appear before the command:
|
|
15
|
+
|
|
16
|
+
```sh
|
|
17
|
+
rubydb --version
|
|
18
|
+
rubydb --no-color --env production status --json
|
|
19
|
+
rubydb --config config/production.yml --env production doctor --json
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
Global options are `--config FILE`, `--env ENV`, `--verbose`, `--quiet`,
|
|
23
|
+
`--no-color`, `--help`, and `--version`. Commands return zero on success and a
|
|
24
|
+
non-zero status on failure. Keep `RUBYDB_DEBUG=1` available when a failed
|
|
25
|
+
command needs a Ruby backtrace; never include secrets in captured logs.
|
|
26
|
+
|
|
27
|
+
## Initialize or create a database
|
|
28
|
+
|
|
29
|
+
`init` creates a named database under a data directory and installs the default
|
|
30
|
+
RubyDB metadata/users/sessions tables:
|
|
31
|
+
|
|
32
|
+
```sh
|
|
33
|
+
rubydb init --name app --dir data
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
`create` creates an empty database file:
|
|
37
|
+
|
|
38
|
+
```sh
|
|
39
|
+
rubydb create --name app --dir data
|
|
40
|
+
rubydb create --database /var/lib/rubydb/app.rdb
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
`--force` is destructive for an existing database. Use it only for a disposable
|
|
44
|
+
development directory after verifying the path. It is not a production reset
|
|
45
|
+
procedure.
|
|
46
|
+
|
|
47
|
+
## Start, stop, restart, and status
|
|
48
|
+
|
|
49
|
+
Start a foreground server for development or a service manager:
|
|
50
|
+
|
|
51
|
+
```sh
|
|
52
|
+
rubydb --env production start \
|
|
53
|
+
--host 127.0.0.1 \
|
|
54
|
+
--port 7432 \
|
|
55
|
+
--data-dir /var/lib/rubydb/data \
|
|
56
|
+
--log-dir /var/log/rubydb \
|
|
57
|
+
--pid-file /run/rubydb.pid
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
Use `--daemon` only when an external service manager is not supervising the
|
|
61
|
+
process. Prefer systemd, a container supervisor, or an equivalent process
|
|
62
|
+
manager for production. Stop gracefully first:
|
|
63
|
+
|
|
64
|
+
```sh
|
|
65
|
+
rubydb stop --pid-file /run/rubydb.pid
|
|
66
|
+
rubydb restart --pid-file /run/rubydb.pid
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
`--force` sends a hard termination and can interrupt active work; reserve it
|
|
70
|
+
for a confirmed hung process after preserving logs and checking recovery needs.
|
|
71
|
+
|
|
72
|
+
Inspect local status:
|
|
73
|
+
|
|
74
|
+
```sh
|
|
75
|
+
rubydb --env production status
|
|
76
|
+
rubydb --env production status --json > status.json
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
Use JSON status in monitoring, but combine it with the server readiness endpoint,
|
|
80
|
+
Prometheus metrics, process state, disk-space checks, and application probes.
|
|
81
|
+
|
|
82
|
+
## Doctor and inspect
|
|
83
|
+
|
|
84
|
+
`doctor` runs health/integrity checks. Start with a read-only quick check:
|
|
85
|
+
|
|
86
|
+
```sh
|
|
87
|
+
rubydb doctor --quick --json
|
|
88
|
+
rubydb doctor --json > doctor.json
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
`doctor --fix` may perform supported repairs. Take and verify a backup first,
|
|
92
|
+
stop application writes, record the output, and review every proposed action:
|
|
93
|
+
|
|
94
|
+
```sh
|
|
95
|
+
rubydb doctor --fix
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
`inspect` reports storage internals without replacing a backup or integrity
|
|
99
|
+
verification:
|
|
100
|
+
|
|
101
|
+
```sh
|
|
102
|
+
rubydb inspect --database data/app.rdb --stats --wal
|
|
103
|
+
rubydb inspect --database data/app.rdb --pages --indexes
|
|
104
|
+
rubydb inspect --database data/app.rdb --table users
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
Use `inspect --wal` to correlate WAL growth with checkpoint/backup incidents.
|
|
108
|
+
Do not edit files based on an inspection result; use the documented recovery
|
|
109
|
+
procedure.
|
|
110
|
+
|
|
111
|
+
## SQL shell
|
|
112
|
+
|
|
113
|
+
Open the configured local database shell:
|
|
114
|
+
|
|
115
|
+
```sh
|
|
116
|
+
rubydb shell --database app
|
|
117
|
+
```
|
|
118
|
+
|
|
119
|
+
Output modes include `--table`, `--json`, and `--csv`. The shell supports these
|
|
120
|
+
dot-commands:
|
|
121
|
+
|
|
122
|
+
| Command | Purpose |
|
|
123
|
+
| --- | --- |
|
|
124
|
+
| `.help`, `.?` | Show shell help |
|
|
125
|
+
| `.exit`, `.quit` | Exit safely |
|
|
126
|
+
| `.status` | Show connection/database status |
|
|
127
|
+
| `.tables` | List tables |
|
|
128
|
+
| `.table NAME` | Describe columns and constraints |
|
|
129
|
+
| `.schema NAME` | Show a table definition |
|
|
130
|
+
| `.db NAME` | Switch configured local database |
|
|
131
|
+
| `.begin`, `.txn` | Begin a transaction |
|
|
132
|
+
| `.commit` | Commit the active transaction |
|
|
133
|
+
| `.rollback` | Roll back the active transaction |
|
|
134
|
+
| `.explain SQL` | Show the selected query plan |
|
|
135
|
+
| `.clear`, `.cls` | Clear the terminal |
|
|
136
|
+
|
|
137
|
+
Example:
|
|
138
|
+
|
|
139
|
+
```text
|
|
140
|
+
users> .tables
|
|
141
|
+
users> SELECT id, email FROM users ORDER BY id LIMIT 20;
|
|
142
|
+
users> .begin
|
|
143
|
+
users> UPDATE users SET active = TRUE WHERE id = 1;
|
|
144
|
+
users> .rollback
|
|
145
|
+
users> .exit
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
Use bound parameters from application/client APIs for untrusted values. The
|
|
149
|
+
shell is an administrative tool, not a connection-pool substitute. For
|
|
150
|
+
multiple application processes, use the managed server and client libraries;
|
|
151
|
+
never open one embedded path concurrently from independent processes.
|
|
152
|
+
|
|
153
|
+
## Migrations
|
|
154
|
+
|
|
155
|
+
Run migrations from the default `db/migrate` directory:
|
|
156
|
+
|
|
157
|
+
```sh
|
|
158
|
+
rubydb migrate --database data/app.rdb --path db/migrate
|
|
159
|
+
rubydb migrate --database data/app.rdb --path db/migrate --dry-run
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
To roll back, specify the number of steps and use a verified backup boundary:
|
|
163
|
+
|
|
164
|
+
```sh
|
|
165
|
+
rubydb migrate --database data/app.rdb --path db/migrate --down --steps 1
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
`--version VERSION` targets a migration version. Before applying migrations in
|
|
169
|
+
production, stop or coordinate writes, create a verified backup, test against a
|
|
170
|
+
populated restore, run schema dump/load and application smoke queries, and keep
|
|
171
|
+
the pre-migration directory until rollback closes. Applied migration checksum
|
|
172
|
+
changes fail closed.
|
|
173
|
+
|
|
174
|
+
## Backups and restore
|
|
175
|
+
|
|
176
|
+
Create a verified full backup:
|
|
177
|
+
|
|
178
|
+
```sh
|
|
179
|
+
rubydb backup --database data/app.rdb --dir backups --type full --compress
|
|
180
|
+
```
|
|
181
|
+
|
|
182
|
+
The supported types are `full`, `incremental`, and `differential`. Use
|
|
183
|
+
`--no-verify` only for controlled diagnostics; a release or production backup
|
|
184
|
+
should verify. Retain the manifest, checksum, database version, and required
|
|
185
|
+
WAL chain together.
|
|
186
|
+
|
|
187
|
+
List or dry-run a restore before changing a live path:
|
|
188
|
+
|
|
189
|
+
```sh
|
|
190
|
+
rubydb restore --dir backups --latest --dry-run
|
|
191
|
+
rubydb restore --database data/app.rdb --dir backups --backup BACKUP_NAME --dry-run
|
|
192
|
+
```
|
|
193
|
+
|
|
194
|
+
Restore to a new inactive directory whenever possible. `--force` is destructive
|
|
195
|
+
and must not overwrite the only source. After restore, reopen the engine, verify
|
|
196
|
+
schema and row evidence, run application smoke queries, and record RPO/RTO.
|
|
197
|
+
Point-in-time restore uses `--point-in-time TIME` only when the required WAL
|
|
198
|
+
chain is present and verified.
|
|
199
|
+
|
|
200
|
+
## Snapshots
|
|
201
|
+
|
|
202
|
+
Snapshots are useful for staging, inspection, and branch workflows:
|
|
203
|
+
|
|
204
|
+
```sh
|
|
205
|
+
rubydb snapshot --database data/app.rdb --dir snapshots --name before-migration
|
|
206
|
+
rubydb snapshot --database data/app.rdb --dir snapshots --list
|
|
207
|
+
rubydb snapshot --database data/app.rdb --dir snapshots --restore before-migration
|
|
208
|
+
rubydb snapshot --database data/app.rdb --dir snapshots --delete old-snapshot
|
|
209
|
+
```
|
|
210
|
+
|
|
211
|
+
Keep snapshot metadata/checksums and do not treat snapshots as the only backup.
|
|
212
|
+
Restore into a separate destination before routing traffic.
|
|
213
|
+
|
|
214
|
+
## Branches and database diff
|
|
215
|
+
|
|
216
|
+
List or create a branch:
|
|
217
|
+
|
|
218
|
+
```sh
|
|
219
|
+
rubydb branch --database data/app.rdb --branch-dir branches --list
|
|
220
|
+
rubydb branch --database data/app.rdb --branch-dir branches --create feature-x --from main
|
|
221
|
+
rubydb checkout --database data/app.rdb --branch-dir branches feature-x
|
|
222
|
+
rubydb checkout --database data/app.rdb --branch-dir branches --create experiment
|
|
223
|
+
```
|
|
224
|
+
|
|
225
|
+
Review changes before merging:
|
|
226
|
+
|
|
227
|
+
```sh
|
|
228
|
+
rubydb diff --database data/app.rdb --branch-dir branches main feature-x
|
|
229
|
+
rubydb diff --database data/app.rdb --branch-dir branches --summary main feature-x
|
|
230
|
+
rubydb diff --database data/app.rdb --branch-dir branches --table users main feature-x
|
|
231
|
+
rubydb merge --database data/app.rdb --branch-dir branches feature-x --into main
|
|
232
|
+
```
|
|
233
|
+
|
|
234
|
+
`merge` supports the configured strategy and `--no-commit`. Take a verified
|
|
235
|
+
backup, review the diff, test the target with representative data, and preserve
|
|
236
|
+
the pre-merge state. Branches are not replication or backup replacements.
|
|
237
|
+
|
|
238
|
+
## Vacuum and maintenance
|
|
239
|
+
|
|
240
|
+
Preview maintenance first:
|
|
241
|
+
|
|
242
|
+
```sh
|
|
243
|
+
rubydb vacuum --database data/app.rdb --dry-run
|
|
244
|
+
rubydb vacuum --database data/app.rdb --table users
|
|
245
|
+
rubydb vacuum --database data/app.rdb --full --analyze
|
|
246
|
+
```
|
|
247
|
+
|
|
248
|
+
Run full vacuum only during an approved maintenance window. Long-running
|
|
249
|
+
transactions may retain versions and prevent space reclamation. Monitor free
|
|
250
|
+
space, WAL/checkpoint behavior, query latency, and active transaction age.
|
|
251
|
+
|
|
252
|
+
## Production command sequences
|
|
253
|
+
|
|
254
|
+
### Pre-deployment
|
|
255
|
+
|
|
256
|
+
```sh
|
|
257
|
+
rubydb --config config/production.yml --env production doctor --quick --json
|
|
258
|
+
rubydb --config config/production.yml --env production status --json
|
|
259
|
+
rubydb backup --database /var/lib/rubydb/app.rdb --dir /var/backups/rubydb --type full --compress
|
|
260
|
+
rubydb migrate --database /var/lib/rubydb/app.rdb --path db/migrate --dry-run
|
|
261
|
+
```
|
|
262
|
+
|
|
263
|
+
### Incident preservation
|
|
264
|
+
|
|
265
|
+
Stop writes, preserve the database/WAL/log/configuration set, copy it to an
|
|
266
|
+
incident directory, and then use `doctor --quick`, `inspect --stats --wal`, and
|
|
267
|
+
the restore procedure. Do not use `drop --force`, `create --force`, or vacuum as
|
|
268
|
+
an attempted corruption repair.
|
|
269
|
+
|
|
270
|
+
### Release evidence
|
|
271
|
+
|
|
272
|
+
Archive command output with the RubyDB commit/version, Ruby version, OS,
|
|
273
|
+
database path class, workload, backup name/checksum, restore time, and operator.
|
|
274
|
+
The CLI is one part of the production evidence; CI, workload, failover, and
|
|
275
|
+
security review records are also required.
|
|
@@ -0,0 +1,9 @@
|
|
|
1
|
+
# Architecture contribution guide
|
|
2
|
+
|
|
3
|
+
Before changing a core subsystem, identify its ownership, durability boundary,
|
|
4
|
+
failure behavior, and public compatibility contract. Changes to storage, WAL,
|
|
5
|
+
recovery, transactions, replication, protocol, or adapters need focused tests
|
|
6
|
+
and an end-to-end regression.
|
|
7
|
+
|
|
8
|
+
Document invariants in the relevant architecture page and add a lesson when a
|
|
9
|
+
new failure mode is discovered. Prefer explicit errors over silent fallback.
|
|
@@ -0,0 +1,14 @@
|
|
|
1
|
+
# Benchmarking guide
|
|
2
|
+
|
|
3
|
+
Run deterministic and concurrent workloads from the repository root:
|
|
4
|
+
|
|
5
|
+
```sh
|
|
6
|
+
RUBYDB_BENCHMARK_ITERATIONS=100 ruby -Ilib benchmarks/basic_workload.rb
|
|
7
|
+
RUBYDB_SOAK_THREADS=16 RUBYDB_SOAK_OPERATIONS=2000 ruby benchmarks/concurrent_soak.rb
|
|
8
|
+
RUBYDB_PRODUCTION_SOAK_CLIENTS=16 RUBYDB_PRODUCTION_SOAK_OPERATIONS=2000 ruby benchmarks/production_soak.rb
|
|
9
|
+
```
|
|
10
|
+
|
|
11
|
+
Record commit, Ruby version, OS, CPU/memory/storage, dataset size, seed,
|
|
12
|
+
throughput, p50/p95/p99 latency, WAL growth, and recovery time. Benchmarks are
|
|
13
|
+
not universal capacity certification; compare against application-specific
|
|
14
|
+
limits and repeat after schema or runtime changes.
|
|
@@ -0,0 +1,16 @@
|
|
|
1
|
+
# Development guide
|
|
2
|
+
|
|
3
|
+
Install dependencies with `bundle install`. Run focused specs while developing,
|
|
4
|
+
then run `bundle exec rspec` and `bundle exec rubocop` before opening a pull
|
|
5
|
+
request. Keep test databases in temporary directories and close engines in
|
|
6
|
+
`ensure` blocks.
|
|
7
|
+
|
|
8
|
+
Never use production data or secrets in local tests. Changes that affect a
|
|
9
|
+
stored format, SQL behavior, protocol, migration, or release process must update
|
|
10
|
+
the corresponding documentation and changelog.
|
|
11
|
+
|
|
12
|
+
For the complete change workflow, read the [developer guide](../developer-guide.md),
|
|
13
|
+
[debugging playbook](../debugging.md), and [testing guide](testing.md). Every
|
|
14
|
+
bug fix should include a regression test and an explanation of its invariant.
|
|
15
|
+
Run fault, concurrency, or recovery tests when the change crosses a durable
|
|
16
|
+
boundary; a unit test alone is not sufficient evidence.
|