rubydb 0.1.4 → 0.1.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.github/PULL_REQUEST_TEMPLATE.md +15 -15
- data/.github/workflows/benchmark.yml +26 -26
- data/.github/workflows/compatibility.yml +63 -63
- data/.github/workflows/fuzz.yml +33 -33
- data/.github/workflows/lint.yml +21 -21
- data/.github/workflows/operations.yml +24 -24
- data/.github/workflows/production-validation.yml +111 -111
- data/.github/workflows/release.yml +77 -77
- data/.github/workflows/security.yml +39 -37
- data/.github/workflows/test.yml +26 -26
- data/.github/workflows/workload.yml +58 -58
- data/.gitignore +12 -5
- data/.rubocop.yml +50 -44
- data/.standard.yml +9 -14
- data/ARCHITECTURE.md +21 -21
- data/CHANGELOG.md +43 -27
- data/CODE_OF_CONDUCT.md +13 -13
- data/CONTRIBUTING.md +29 -29
- data/GOVERNANCE.md +16 -16
- data/Gemfile +18 -17
- data/Gemfile.lock +125 -71
- data/README.md +249 -162
- data/ROADMAP.md +27 -27
- data/Rakefile +71 -71
- data/SECURITY.md +54 -54
- data/SUPPORT.md +14 -14
- data/adapters/activerecord/Gemfile +11 -11
- data/adapters/activerecord/lib/active_record/connection_adapters/rubydb_adapter.rb +873 -879
- data/adapters/activerecord/rubydb-activerecord.gemspec +20 -20
- data/adapters/activerecord/spec/rubydb_adapter_integration_spec.rb +143 -143
- data/adapters/ruby/README.md +18 -18
- data/adapters/sequel/README.md +11 -11
- data/config/monitoring/prometheus-alerts.yml +39 -39
- data/config/production.yml +36 -36
- data/docs/README.md +75 -71
- data/docs/architecture/concurrency.md +14 -14
- data/docs/architecture/current-state.md +125 -125
- data/docs/architecture/execution-engine.md +25 -25
- data/docs/architecture/indexes.md +19 -19
- data/docs/architecture/mvcc.md +19 -19
- data/docs/architecture/overview.md +13 -13
- data/docs/architecture/pages.md +11 -11
- data/docs/architecture/production-roadmap.md +82 -82
- data/docs/architecture/query-planner.md +20 -20
- data/docs/architecture/recovery.md +18 -18
- data/docs/architecture/sql-engine.md +12 -12
- data/docs/architecture/storage-engine.md +14 -14
- data/docs/architecture/transactions.md +10 -10
- data/docs/architecture/wal.md +28 -28
- data/docs/cli-cheatsheet.md +98 -98
- data/docs/cli.md +275 -275
- data/docs/contributing/architecture.md +9 -9
- data/docs/contributing/benchmarking.md +14 -14
- data/docs/contributing/development.md +16 -16
- data/docs/contributing/release-process.md +49 -49
- data/docs/contributing/testing.md +16 -16
- data/docs/debugging.md +229 -229
- data/docs/developer/branching.md +10 -10
- data/docs/developer/database-diff.md +10 -10
- data/docs/developer/local-development.md +17 -17
- data/docs/developer/snapshots.md +9 -9
- data/docs/developer/temporal-data.md +10 -10
- data/docs/developer-guide.md +297 -297
- data/docs/getting-started/first-database.md +16 -16
- data/docs/getting-started/first-query.md +13 -13
- data/docs/getting-started/installation.md +19 -19
- data/docs/getting-started/local-to-production.md +300 -300
- data/docs/getting-started/quickstart.md +17 -17
- data/docs/getting-started/rails.md +16 -16
- data/docs/hardening_backlog.md +93 -93
- data/docs/lessons-learned.md +112 -112
- data/docs/operations/backups.md +33 -33
- data/docs/operations/disaster-recovery.md +31 -31
- data/docs/operations/failover.md +30 -30
- data/docs/operations/monitoring.md +25 -25
- data/docs/operations/production-guide.md +295 -295
- data/docs/operations/production-runbook.md +45 -45
- data/docs/operations/replication.md +33 -33
- data/docs/operations/restore.md +6 -6
- data/docs/operations/runbook.md +34 -34
- data/docs/operations/upgrades.md +14 -14
- data/docs/operations/workload-testing.md +17 -17
- data/docs/production-readiness.md +118 -118
- data/docs/production_validation.md +150 -150
- data/docs/rails/active-record.md +11 -11
- data/docs/rails/compatibility-guide.md +90 -90
- data/docs/rails/database-yml.md +92 -92
- data/docs/rails/installation.md +17 -17
- data/docs/rails/migrations.md +17 -17
- data/docs/rails/production.md +82 -82
- data/docs/rails/troubleshooting.md +18 -18
- data/docs/release.md +25 -25
- data/docs/server/architecture.md +10 -10
- data/docs/server/authentication.md +10 -10
- data/docs/server/configuration.md +16 -16
- data/docs/server/connection-pooling.md +10 -10
- data/docs/server/deployment.md +10 -10
- data/docs/server/protocol.md +12 -12
- data/docs/sql/compatibility-guide.md +82 -82
- data/docs/sql/compatibility.md +39 -39
- data/docs/sql/data-types.md +10 -10
- data/docs/sql/functions.md +9 -9
- data/docs/sql/joins.md +9 -9
- data/docs/sql/operators.md +9 -9
- data/docs/sql/sqlite-compatibility.md +21 -21
- data/docs/sql/syntax.md +10 -10
- data/docs/sql/transactions.md +10 -10
- data/docs/troubleshooting.md +244 -244
- data/lessons/01-foundations.md +73 -0
- data/lessons/02-local-development.md +121 -0
- data/lessons/03-embedded-rubydb.md +99 -0
- data/lessons/04-rails-complex-apps.md +138 -0
- data/lessons/05-rubydb-production-server.md +237 -0
- data/lessons/06-postgresql-massive-apps.md +96 -0
- data/lessons/07-hybrid-microservices.md +94 -0
- data/lessons/08-migrations-backups-recovery.md +86 -0
- data/lessons/09-observability-security-scale.md +87 -0
- data/lessons/10-release-readiness.md +125 -0
- data/lib/rubydb/backup/archive.rb +332 -334
- data/lib/rubydb/backup/backup.rb +400 -401
- data/lib/rubydb/backup/incremental.rb +349 -353
- data/lib/rubydb/backup/restore.rb +289 -290
- data/lib/rubydb/backup/snapshot.rb +265 -267
- data/lib/rubydb/backup/verification.rb +276 -279
- data/lib/rubydb/branching/branch.rb +181 -181
- data/lib/rubydb/branching/branch_manager.rb +307 -311
- data/lib/rubydb/branching/branch_metadata.rb +140 -140
- data/lib/rubydb/branching/checkout.rb +165 -166
- data/lib/rubydb/branching/copy_on_write.rb +272 -272
- data/lib/rubydb/branching/diff.rb +137 -138
- data/lib/rubydb/branching/merge.rb +282 -285
- data/lib/rubydb/build_info.rb +15 -15
- data/lib/rubydb/catalog/catalog.rb +391 -391
- data/lib/rubydb/catalog/column.rb +112 -112
- data/lib/rubydb/catalog/constraint.rb +180 -180
- data/lib/rubydb/catalog/database.rb +184 -184
- data/lib/rubydb/catalog/index.rb +97 -97
- data/lib/rubydb/catalog/schema.rb +103 -103
- data/lib/rubydb/catalog/sequence.rb +90 -90
- data/lib/rubydb/catalog/system_catalog.rb +698 -698
- data/lib/rubydb/catalog/table.rb +178 -178
- data/lib/rubydb/catalog/trigger.rb +102 -102
- data/lib/rubydb/catalog/view.rb +66 -66
- data/lib/rubydb/cli/application.rb +163 -163
- data/lib/rubydb/cli/commands/backup.rb +80 -81
- data/lib/rubydb/cli/commands/branch.rb +72 -72
- data/lib/rubydb/cli/commands/checkout.rb +54 -54
- data/lib/rubydb/cli/commands/create.rb +58 -58
- data/lib/rubydb/cli/commands/diff.rb +76 -77
- data/lib/rubydb/cli/commands/doctor.rb +74 -74
- data/lib/rubydb/cli/commands/drop.rb +57 -57
- data/lib/rubydb/cli/commands/init.rb +101 -102
- data/lib/rubydb/cli/commands/inspect.rb +95 -95
- data/lib/rubydb/cli/commands/merge.rb +63 -63
- data/lib/rubydb/cli/commands/migrate.rb +62 -62
- data/lib/rubydb/cli/commands/restart.rb +42 -39
- data/lib/rubydb/cli/commands/restore.rb +121 -121
- data/lib/rubydb/cli/commands/shell.rb +365 -365
- data/lib/rubydb/cli/commands/snapshot.rb +79 -79
- data/lib/rubydb/cli/commands/start.rb +88 -82
- data/lib/rubydb/cli/commands/status.rb +96 -92
- data/lib/rubydb/cli/commands/stop.rb +47 -47
- data/lib/rubydb/cli/commands/vacuum.rb +58 -58
- data/lib/rubydb/cli/formatter.rb +221 -221
- data/lib/rubydb/cli/output.rb +168 -168
- data/lib/rubydb/client/client.rb +302 -304
- data/lib/rubydb/client/connection.rb +414 -415
- data/lib/rubydb/client/connection_pool.rb +168 -168
- data/lib/rubydb/client/connection_url.rb +96 -96
- data/lib/rubydb/client/prepared_statement.rb +60 -60
- data/lib/rubydb/client/result.rb +123 -123
- data/lib/rubydb/client/statement.rb +52 -52
- data/lib/rubydb/client/transaction.rb +130 -130
- data/lib/rubydb/concurrency/concurrency.rb +19 -19
- data/lib/rubydb/concurrency/deadlock_detector.rb +148 -150
- data/lib/rubydb/concurrency/latch.rb +101 -101
- data/lib/rubydb/concurrency/lock_graph.rb +163 -165
- data/lib/rubydb/concurrency/mutex.rb +181 -183
- data/lib/rubydb/concurrency/rw_lock.rb +180 -180
- data/lib/rubydb/concurrency/scheduler.rb +248 -250
- data/lib/rubydb/concurrency/worker_pool.rb +145 -143
- data/lib/rubydb/configuration/config.rb +170 -170
- data/lib/rubydb/configuration/defaults.rb +179 -179
- data/lib/rubydb/configuration/environment.rb +152 -152
- data/lib/rubydb/configuration/parser.rb +185 -185
- data/lib/rubydb/configuration/validation.rb +221 -221
- data/lib/rubydb/constants.rb +74 -74
- data/lib/rubydb/constraints/check.rb +181 -181
- data/lib/rubydb/constraints/constraint.rb +101 -101
- data/lib/rubydb/constraints/foreign_key.rb +130 -130
- data/lib/rubydb/constraints/not_null.rb +64 -64
- data/lib/rubydb/constraints/primary_key.rb +99 -99
- data/lib/rubydb/constraints/unique.rb +106 -108
- data/lib/rubydb/constraints/validator.rb +349 -350
- data/lib/rubydb/errors/authentication_error.rb +10 -10
- data/lib/rubydb/errors/authorization_error.rb +23 -23
- data/lib/rubydb/errors/client_error.rb +10 -10
- data/lib/rubydb/errors/configuration_error.rb +10 -10
- data/lib/rubydb/errors/connection_error.rb +10 -10
- data/lib/rubydb/errors/constraint_error.rb +23 -23
- data/lib/rubydb/errors/corruption_error.rb +10 -10
- data/lib/rubydb/errors/database_error.rb +10 -10
- data/lib/rubydb/errors/error.rb +20 -20
- data/lib/rubydb/errors/execution_error.rb +10 -10
- data/lib/rubydb/errors/parser_error.rb +10 -10
- data/lib/rubydb/errors/recovery_error.rb +10 -10
- data/lib/rubydb/errors/replication_error.rb +10 -10
- data/lib/rubydb/errors/server_error.rb +6 -6
- data/lib/rubydb/errors/storage_error.rb +10 -10
- data/lib/rubydb/errors/transaction_error.rb +10 -10
- data/lib/rubydb/execution/aggregate_executor.rb +134 -138
- data/lib/rubydb/execution/delete_executor.rb +110 -112
- data/lib/rubydb/execution/distinct_executor.rb +131 -135
- data/lib/rubydb/execution/executor.rb +1182 -1188
- data/lib/rubydb/execution/expression.rb +191 -193
- data/lib/rubydb/execution/index_scan.rb +142 -142
- data/lib/rubydb/execution/insert_executor.rb +215 -217
- data/lib/rubydb/execution/join_executor.rb +243 -249
- data/lib/rubydb/execution/limit_executor.rb +83 -85
- data/lib/rubydb/execution/optimizer.rb +227 -215
- data/lib/rubydb/execution/plan.rb +355 -353
- data/lib/rubydb/execution/planner.rb +542 -536
- data/lib/rubydb/execution/predicate.rb +235 -235
- data/lib/rubydb/execution/scan.rb +49 -49
- data/lib/rubydb/execution/sequential_scan.rb +63 -63
- data/lib/rubydb/execution/sort_executor.rb +180 -185
- data/lib/rubydb/execution/update_executor.rb +160 -162
- data/lib/rubydb/functions/aggregate.rb +70 -70
- data/lib/rubydb/functions/date_functions.rb +274 -278
- data/lib/rubydb/functions/function.rb +85 -85
- data/lib/rubydb/functions/json_functions.rb +231 -215
- data/lib/rubydb/functions/numeric_functions.rb +346 -346
- data/lib/rubydb/functions/scalar.rb +52 -52
- data/lib/rubydb/functions/string_functions.rb +383 -383
- data/lib/rubydb/functions/system_functions.rb +258 -246
- data/lib/rubydb/history/as_of.rb +238 -238
- data/lib/rubydb/history/change.rb +105 -105
- data/lib/rubydb/history/history.rb +131 -131
- data/lib/rubydb/history/history_manager.rb +228 -229
- data/lib/rubydb/history/temporal_query.rb +202 -202
- data/lib/rubydb/history/timeline.rb +144 -144
- data/lib/rubydb/indexes/btree.rb +186 -186
- data/lib/rubydb/indexes/btree_cursor.rb +258 -258
- data/lib/rubydb/indexes/btree_node.rb +384 -385
- data/lib/rubydb/indexes/hash_index.rb +150 -150
- data/lib/rubydb/indexes/index.rb +71 -71
- data/lib/rubydb/indexes/index_manager.rb +408 -406
- data/lib/rubydb/indexes/index_scan.rb +466 -470
- data/lib/rubydb/migrations/migration.rb +253 -254
- data/lib/rubydb/migrations/migration_lock.rb +146 -146
- data/lib/rubydb/migrations/migration_manager.rb +187 -176
- data/lib/rubydb/migrations/migration_version.rb +71 -71
- data/lib/rubydb/migrations/schema_diff.rb +211 -211
- data/lib/rubydb/migrations/schema_version.rb +64 -64
- data/lib/rubydb/monitoring/events.rb +155 -160
- data/lib/rubydb/monitoring/health.rb +216 -222
- data/lib/rubydb/monitoring/logger.rb +188 -193
- data/lib/rubydb/monitoring/metrics.rb +363 -359
- data/lib/rubydb/monitoring/performance.rb +176 -176
- data/lib/rubydb/monitoring/statistics.rb +168 -170
- data/lib/rubydb/mvcc/garbage_collector.rb +199 -199
- data/lib/rubydb/mvcc/mvcc.rb +16 -16
- data/lib/rubydb/mvcc/snapshot.rb +146 -147
- data/lib/rubydb/mvcc/vacuum.rb +180 -180
- data/lib/rubydb/mvcc/version.rb +106 -106
- data/lib/rubydb/mvcc/version_store.rb +396 -398
- data/lib/rubydb/mvcc/visibility.rb +107 -109
- data/lib/rubydb/protocol/capabilities.rb +125 -125
- data/lib/rubydb/protocol/decoder.rb +142 -145
- data/lib/rubydb/protocol/encoder.rb +131 -136
- data/lib/rubydb/protocol/handshake.rb +306 -305
- data/lib/rubydb/protocol/message.rb +121 -121
- data/lib/rubydb/protocol/parameter_binder.rb +101 -0
- data/lib/rubydb/protocol/protocol.rb +276 -277
- data/lib/rubydb/protocol/version.rb +54 -54
- data/lib/rubydb/rails/adapter.rb +245 -239
- data/lib/rubydb/rails/connection.rb +312 -314
- data/lib/rubydb/rails/database_statements.rb +122 -122
- data/lib/rubydb/rails/migration.rb +131 -131
- data/lib/rubydb/rails/quoting.rb +109 -109
- data/lib/rubydb/rails/result.rb +117 -117
- data/lib/rubydb/rails/schema_statements.rb +339 -339
- data/lib/rubydb/rails/transaction.rb +105 -105
- data/lib/rubydb/rails/type.rb +126 -126
- data/lib/rubydb/recovery/checkpoint.rb +261 -257
- data/lib/rubydb/recovery/consistency.rb +457 -467
- data/lib/rubydb/recovery/corruption_detector.rb +5 -5
- data/lib/rubydb/recovery/crash_recovery.rb +381 -387
- data/lib/rubydb/recovery/recovery_manager.rb +204 -206
- data/lib/rubydb/recovery/redo.rb +235 -237
- data/lib/rubydb/recovery/undo.rb +204 -206
- data/lib/rubydb/replication/failover.rb +5 -5
- data/lib/rubydb/replication/fencing.rb +63 -63
- data/lib/rubydb/replication/primary.rb +461 -450
- data/lib/rubydb/replication/replica.rb +382 -384
- data/lib/rubydb/replication/replication_log.rb +194 -200
- data/lib/rubydb/replication/replication_manager.rb +307 -308
- data/lib/rubydb/replication/replication_slot.rb +293 -295
- data/lib/rubydb/replication/replication_stream.rb +198 -201
- data/lib/rubydb/rubydb.rb +564 -560
- data/lib/rubydb/security/access_control.rb +252 -254
- data/lib/rubydb/security/audit_log.rb +209 -213
- data/lib/rubydb/security/authentication.rb +302 -302
- data/lib/rubydb/security/authorization.rb +282 -282
- data/lib/rubydb/security/credentials.rb +192 -196
- data/lib/rubydb/security/password.rb +205 -215
- data/lib/rubydb/security/permissions.rb +74 -74
- data/lib/rubydb/security/role.rb +99 -101
- data/lib/rubydb/security/user.rb +86 -86
- data/lib/rubydb/server/connection.rb +383 -366
- data/lib/rubydb/server/connection_pool.rb +193 -193
- data/lib/rubydb/server/lifecycle.rb +227 -228
- data/lib/rubydb/server/listener.rb +139 -136
- data/lib/rubydb/server/request_handler.rb +277 -276
- data/lib/rubydb/server/server.rb +363 -364
- data/lib/rubydb/server/session.rb +371 -369
- data/lib/rubydb/server/worker.rb +206 -210
- data/lib/rubydb/server/worker_pool.rb +168 -168
- data/lib/rubydb/sql/ast/alter_table.rb +169 -169
- data/lib/rubydb/sql/ast/begin_transaction.rb +47 -47
- data/lib/rubydb/sql/ast/commit.rb +37 -37
- data/lib/rubydb/sql/ast/constraint.rb +92 -83
- data/lib/rubydb/sql/ast/create_database.rb +41 -41
- data/lib/rubydb/sql/ast/create_index.rb +61 -61
- data/lib/rubydb/sql/ast/create_schema.rb +52 -52
- data/lib/rubydb/sql/ast/create_table.rb +187 -187
- data/lib/rubydb/sql/ast/delete.rb +54 -54
- data/lib/rubydb/sql/ast/drop_database.rb +41 -41
- data/lib/rubydb/sql/ast/drop_index.rb +41 -41
- data/lib/rubydb/sql/ast/drop_schema.rb +49 -49
- data/lib/rubydb/sql/ast/drop_table.rb +49 -49
- data/lib/rubydb/sql/ast/explain.rb +64 -64
- data/lib/rubydb/sql/ast/expression.rb +617 -604
- data/lib/rubydb/sql/ast/insert.rb +66 -66
- data/lib/rubydb/sql/ast/node.rb +42 -42
- data/lib/rubydb/sql/ast/rollback.rb +63 -63
- data/lib/rubydb/sql/ast/savepoint.rb +59 -59
- data/lib/rubydb/sql/ast/select.rb +88 -88
- data/lib/rubydb/sql/ast/set_operation.rb +22 -20
- data/lib/rubydb/sql/ast/trigger.rb +35 -29
- data/lib/rubydb/sql/ast/update.rb +88 -88
- data/lib/rubydb/sql/ast/vacuum.rb +19 -19
- data/lib/rubydb/sql/ast/view.rb +38 -32
- data/lib/rubydb/sql/ast/with.rb +32 -32
- data/lib/rubydb/sql/grammar.rb +86 -86
- data/lib/rubydb/sql/keywords.rb +156 -156
- data/lib/rubydb/sql/lexer.rb +209 -214
- data/lib/rubydb/sql/operators.rb +100 -100
- data/lib/rubydb/sql/parser.rb +1167 -1170
- data/lib/rubydb/sql/planner/analyzer.rb +283 -302
- data/lib/rubydb/sql/planner/binder.rb +537 -550
- data/lib/rubydb/sql/planner/type_checker.rb +427 -431
- data/lib/rubydb/sql/token.rb +210 -210
- data/lib/rubydb/storage/buffer_frame.rb +44 -44
- data/lib/rubydb/storage/buffer_pool.rb +155 -155
- data/lib/rubydb/storage/database_lock.rb +74 -74
- data/lib/rubydb/storage/deserializer.rb +332 -342
- data/lib/rubydb/storage/engine.rb +2347 -2330
- data/lib/rubydb/storage/file_manager.rb +191 -187
- data/lib/rubydb/storage/free_space_map.rb +79 -81
- data/lib/rubydb/storage/page.rb +92 -94
- data/lib/rubydb/storage/page_allocator.rb +852 -855
- data/lib/rubydb/storage/page_header.rb +63 -67
- data/lib/rubydb/storage/page_manager.rb +127 -131
- data/lib/rubydb/storage/record.rb +58 -58
- data/lib/rubydb/storage/row.rb +78 -78
- data/lib/rubydb/storage/serializer.rb +51 -51
- data/lib/rubydb/storage/storage_layout.rb +151 -151
- data/lib/rubydb/storage/storage_manager.rb +114 -114
- data/lib/rubydb/storage/tuple.rb +458 -461
- data/lib/rubydb/storage/visibility_map.rb +964 -973
- data/lib/rubydb/transactions/commit_manager.rb +219 -220
- data/lib/rubydb/transactions/isolation.rb +98 -98
- data/lib/rubydb/transactions/lock.rb +76 -76
- data/lib/rubydb/transactions/lock_manager.rb +359 -362
- data/lib/rubydb/transactions/savepoint.rb +142 -143
- data/lib/rubydb/transactions/transaction.rb +214 -215
- data/lib/rubydb/transactions/transaction_id.rb +84 -84
- data/lib/rubydb/transactions/transaction_log.rb +256 -257
- data/lib/rubydb/transactions/transaction_manager.rb +434 -435
- data/lib/rubydb/types/bigint.rb +36 -36
- data/lib/rubydb/types/blob.rb +37 -37
- data/lib/rubydb/types/boolean.rb +34 -34
- data/lib/rubydb/types/date.rb +39 -39
- data/lib/rubydb/types/decimal.rb +48 -48
- data/lib/rubydb/types/float.rb +34 -34
- data/lib/rubydb/types/integer.rb +36 -36
- data/lib/rubydb/types/json.rb +41 -41
- data/lib/rubydb/types/null.rb +34 -34
- data/lib/rubydb/types/smallint.rb +36 -36
- data/lib/rubydb/types/text.rb +37 -37
- data/lib/rubydb/types/time.rb +46 -46
- data/lib/rubydb/types/timestamp.rb +39 -39
- data/lib/rubydb/types/type.rb +119 -119
- data/lib/rubydb/types/uuid.rb +47 -47
- data/lib/rubydb/types/varchar.rb +37 -37
- data/lib/rubydb/version.rb +32 -32
- data/lib/rubydb/wal/archive.rb +190 -193
- data/lib/rubydb/wal/checkpoint.rb +181 -183
- data/lib/rubydb/wal/lsn.rb +94 -94
- data/lib/rubydb/wal/reader.rb +259 -260
- data/lib/rubydb/wal/record.rb +105 -105
- data/lib/rubydb/wal/segment.rb +193 -193
- data/lib/rubydb/wal/wal.rb +480 -452
- data/lib/rubydb/wal/writer.rb +236 -236
- data/lib/rubydb.rb +7 -7
- data/packaging/docker/docker-compose.failover.yml +43 -43
- data/packaging/homebrew/rubydb.rb +19 -19
- data/rubydb.gemspec +60 -57
- data/scripts/benchmark +7 -7
- data/scripts/durability_drill +37 -37
- data/scripts/fuzz +63 -63
- data/scripts/release +45 -40
- data/scripts/release_check +43 -43
- data/scripts/replication_failover_drill +268 -250
- data/scripts/replication_network_failover_drill +287 -255
- data/scripts/restore_drill +45 -45
- data/scripts/security +45 -0
- metadata +54 -1
data/docs/README.md
CHANGED
|
@@ -1,71 +1,75 @@
|
|
|
1
|
-
# RubyDB documentation index
|
|
2
|
-
|
|
3
|
-
Start here:
|
|
4
|
-
|
|
5
|
-
- [Developer guide](developer-guide.md)
|
|
6
|
-
- [Troubleshooting](troubleshooting.md)
|
|
7
|
-
- [Debugging playbook](debugging.md)
|
|
8
|
-
- [Production operations guide](operations/production-guide.md)
|
|
9
|
-
- [Quick start](getting-started/quickstart.md)
|
|
10
|
-
- [Local development to production](getting-started/local-to-production.md)
|
|
11
|
-
- [First database](getting-started/first-database.md)
|
|
12
|
-
- [First query](getting-started/first-query.md)
|
|
13
|
-
- [Rails installation](rails/installation.md)
|
|
14
|
-
- [Rails production guidance](rails/production.md)
|
|
15
|
-
- [SQLite compatibility profile](sql/sqlite-compatibility.md)
|
|
16
|
-
- [SQL compatibility contract](sql/compatibility.md)
|
|
17
|
-
- [SQL compatibility guide](sql/compatibility-guide.md)
|
|
18
|
-
- [Rails compatibility guide](rails/compatibility-guide.md)
|
|
19
|
-
- [
|
|
20
|
-
- [Production
|
|
21
|
-
- [Production
|
|
22
|
-
- [
|
|
23
|
-
- [CLI
|
|
24
|
-
- [
|
|
25
|
-
- [
|
|
26
|
-
- [
|
|
27
|
-
- [
|
|
28
|
-
- [
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
- [
|
|
35
|
-
- [
|
|
36
|
-
- [
|
|
37
|
-
- [
|
|
38
|
-
- [
|
|
39
|
-
- [
|
|
40
|
-
- [
|
|
41
|
-
- [
|
|
42
|
-
- [
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
* `
|
|
57
|
-
|
|
58
|
-
* `
|
|
59
|
-
|
|
60
|
-
* `
|
|
61
|
-
|
|
62
|
-
* `
|
|
63
|
-
|
|
64
|
-
* `
|
|
65
|
-
|
|
66
|
-
* `
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
1
|
+
# RubyDB documentation index
|
|
2
|
+
|
|
3
|
+
Start here:
|
|
4
|
+
|
|
5
|
+
- [Developer guide](developer-guide.md)
|
|
6
|
+
- [Troubleshooting](troubleshooting.md)
|
|
7
|
+
- [Debugging playbook](debugging.md)
|
|
8
|
+
- [Production operations guide](operations/production-guide.md)
|
|
9
|
+
- [Quick start](getting-started/quickstart.md)
|
|
10
|
+
- [Local development to production](getting-started/local-to-production.md)
|
|
11
|
+
- [First database](getting-started/first-database.md)
|
|
12
|
+
- [First query](getting-started/first-query.md)
|
|
13
|
+
- [Rails installation](rails/installation.md)
|
|
14
|
+
- [Rails production guidance](rails/production.md)
|
|
15
|
+
- [SQLite compatibility profile](sql/sqlite-compatibility.md)
|
|
16
|
+
- [SQL compatibility contract](sql/compatibility.md)
|
|
17
|
+
- [SQL compatibility guide](sql/compatibility-guide.md)
|
|
18
|
+
- [Rails compatibility guide](rails/compatibility-guide.md)
|
|
19
|
+
- [Python adapter](../adapters/python/README.md)
|
|
20
|
+
- [Production validation](production_validation.md)
|
|
21
|
+
- [Production-readiness audit](production-readiness.md)
|
|
22
|
+
- [Production runbook](operations/production-runbook.md)
|
|
23
|
+
- [CLI guide](cli.md)
|
|
24
|
+
- [CLI cheat sheet](cli-cheatsheet.md)
|
|
25
|
+
- [Disaster recovery](operations/disaster-recovery.md)
|
|
26
|
+
- [Monitoring and alerting](operations/monitoring.md)
|
|
27
|
+
- [Workload testing](operations/workload-testing.md)
|
|
28
|
+
- [Release checklist](release.md)
|
|
29
|
+
- [Lessons learned](lessons-learned.md)
|
|
30
|
+
- [Ten-lesson production journey](../lessons/01-foundations.md)
|
|
31
|
+
|
|
32
|
+
Architecture and development:
|
|
33
|
+
|
|
34
|
+
- [Current-state audit](architecture/current-state.md)
|
|
35
|
+
- [Production roadmap](architecture/production-roadmap.md)
|
|
36
|
+
- [Architecture overview](architecture/overview.md)
|
|
37
|
+
- [Storage engine](architecture/storage-engine.md)
|
|
38
|
+
- [Transactions](architecture/transactions.md)
|
|
39
|
+
- [MVCC](architecture/mvcc.md)
|
|
40
|
+
- [WAL](architecture/wal.md)
|
|
41
|
+
- [Query planner](architecture/query-planner.md)
|
|
42
|
+
- [Testing guide](contributing/testing.md)
|
|
43
|
+
- [Release process](contributing/release-process.md)
|
|
44
|
+
- [Branching](developer/branching.md)
|
|
45
|
+
|
|
46
|
+
The root [README](../README.md) is the public project overview. The current
|
|
47
|
+
production claim is intentionally bounded by the tested features and the
|
|
48
|
+
deployment-specific validation described above.
|
|
49
|
+
|
|
50
|
+
## Documentation map
|
|
51
|
+
|
|
52
|
+
Use the guides for the complete workflow and the smaller pages for focused
|
|
53
|
+
reference:
|
|
54
|
+
|
|
55
|
+
* `getting-started/` gets a new Ruby or Rails application running.
|
|
56
|
+
* `sql/` defines syntax, types, expressions, transactions, joins, functions,
|
|
57
|
+
and compatibility boundaries.
|
|
58
|
+
* `architecture/` explains storage, WAL, recovery, MVCC, indexes, planning,
|
|
59
|
+
execution, server ownership, and replication.
|
|
60
|
+
* `developer/` covers local development, snapshots, branches, temporal data,
|
|
61
|
+
database diffs, and implementation workflows.
|
|
62
|
+
* `rails/` covers installation, configuration, migrations, production use,
|
|
63
|
+
compatibility, and troubleshooting.
|
|
64
|
+
* `server/` covers deployment, authentication, TLS, pooling, protocol, and
|
|
65
|
+
operational behavior.
|
|
66
|
+
* `operations/` covers backups, restore drills, monitoring, workload tests,
|
|
67
|
+
production procedures, and incident response.
|
|
68
|
+
* `../lessons/` is a guided beginner-to-production journey that combines local
|
|
69
|
+
RubyDB, Rails, PostgreSQL, and hybrid microservice decisions.
|
|
70
|
+
* `contributing/` and the root policy files cover testing, release, governance,
|
|
71
|
+
support, security, and contribution requirements.
|
|
72
|
+
|
|
73
|
+
If a topic page and an implementation disagree, treat executable tests and the
|
|
74
|
+
documented compatibility contract as the source of truth, then open an issue
|
|
75
|
+
to reconcile the documentation.
|
|
@@ -1,14 +1,14 @@
|
|
|
1
|
-
# Concurrency architecture
|
|
2
|
-
|
|
3
|
-
Threads coordinate through locks, transaction ownership, the lock manager,
|
|
4
|
-
buffer-pool synchronization, and server worker/connection limits. Transactions
|
|
5
|
-
have connection-scoped state in server mode; the embedded owner remains one
|
|
6
|
-
process.
|
|
7
|
-
|
|
8
|
-
The lock manager detects wait-for cycles and selects a victim. The transaction
|
|
9
|
-
manager rolls the victim back, releases its locks, and returns a retryable
|
|
10
|
-
deadlock error. Timeouts and cancellation are distinct: a timeout stops a
|
|
11
|
-
request deadline, while wire cancellation actively marks an in-flight request.
|
|
12
|
-
|
|
13
|
-
Validate concurrency with the production soak, multi-process workload, latency
|
|
14
|
-
percentiles, deadlock checks, resource limits, and durable reopen verification.
|
|
1
|
+
# Concurrency architecture
|
|
2
|
+
|
|
3
|
+
Threads coordinate through locks, transaction ownership, the lock manager,
|
|
4
|
+
buffer-pool synchronization, and server worker/connection limits. Transactions
|
|
5
|
+
have connection-scoped state in server mode; the embedded owner remains one
|
|
6
|
+
process.
|
|
7
|
+
|
|
8
|
+
The lock manager detects wait-for cycles and selects a victim. The transaction
|
|
9
|
+
manager rolls the victim back, releases its locks, and returns a retryable
|
|
10
|
+
deadlock error. Timeouts and cancellation are distinct: a timeout stops a
|
|
11
|
+
request deadline, while wire cancellation actively marks an in-flight request.
|
|
12
|
+
|
|
13
|
+
Validate concurrency with the production soak, multi-process workload, latency
|
|
14
|
+
percentiles, deadlock checks, resource limits, and durable reopen verification.
|
|
@@ -1,125 +1,125 @@
|
|
|
1
|
-
# RubyDB current-state audit
|
|
2
|
-
|
|
3
|
-
> This document is a dated architecture audit, not a statement that every
|
|
4
|
-
> finding remains open. For current validation evidence, see
|
|
5
|
-
> [production validation](../production_validation.md) and the latest CI runs.
|
|
6
|
-
|
|
7
|
-
## Executive summary
|
|
8
|
-
|
|
9
|
-
RubyDB is a broad database project with integrated storage, WAL, MVCC,
|
|
10
|
-
transactions, catalog, security, server, replication, backup, Rails, and
|
|
11
|
-
monitoring paths. The implementation now has substantial focused validation,
|
|
12
|
-
but several deployment-level capabilities remain environment-specific or
|
|
13
|
-
intentionally bounded.
|
|
14
|
-
|
|
15
|
-
The primary risk is not that the code is absent; it is that multiple components exist without being integrated into a consistent, correct, durable engine. Existing code needs additional hardening before it can safely store user data.
|
|
16
|
-
|
|
17
|
-
## Repository audit
|
|
18
|
-
|
|
19
|
-
### Runtime and package state
|
|
20
|
-
|
|
21
|
-
- Ruby runtime in the development container: Ruby 3.4.7
|
|
22
|
-
- Package metadata targets Ruby >= 3.3.0; CI covers Ruby 3.3 and 3.4 on Linux,
|
|
23
|
-
macOS, and Windows.
|
|
24
|
-
- The top-level library entrypoint and gem packaging are covered by
|
|
25
|
-
`require "rubydb"` and release-artifact tests.
|
|
26
|
-
- The project uses Bundler and RSpec as the base test stack.
|
|
27
|
-
- The repository is organized around a large `lib/rubydb` module tree and includes docs, examples, chaos, and fuzz directories.
|
|
28
|
-
|
|
29
|
-
### Architectural reality
|
|
30
|
-
|
|
31
|
-
The project already contains modules for:
|
|
32
|
-
|
|
33
|
-
- storage engine and page manager
|
|
34
|
-
- buffer pool and file management
|
|
35
|
-
- WAL and recovery components
|
|
36
|
-
- transactions and MVCC
|
|
37
|
-
- catalog, indexes, and constraints
|
|
38
|
-
- SQL parser and planner
|
|
39
|
-
- server and protocol layers
|
|
40
|
-
- security and authorization
|
|
41
|
-
- replication, backups, and monitoring
|
|
42
|
-
- Rails integration
|
|
43
|
-
|
|
44
|
-
The remaining risk is uneven validation breadth: many capabilities are proven
|
|
45
|
-
by focused tests, but multi-host operations, capacity limits, and complete
|
|
46
|
-
external-database dialect compatibility are not yet certified.
|
|
47
|
-
|
|
48
|
-
## High-risk findings
|
|
49
|
-
|
|
50
|
-
### 1. Library loading (resolved)
|
|
51
|
-
|
|
52
|
-
The standard Ruby consumer path is now tested:
|
|
53
|
-
|
|
54
|
-
```ruby
|
|
55
|
-
require "rubydb"
|
|
56
|
-
```
|
|
57
|
-
|
|
58
|
-
Release metadata and the local gem build verify that this loads correctly.
|
|
59
|
-
|
|
60
|
-
### 2. Storage layer (partially resolved)
|
|
61
|
-
|
|
62
|
-
The storage engine now has page validation, corruption detection, WAL/recovery,
|
|
63
|
-
fsync paths, restart tests, subprocess crash tests, filesystem fault injection,
|
|
64
|
-
and compaction/reopen coverage. Real disk-quota and power-loss validation on
|
|
65
|
-
deployment filesystems remains open.
|
|
66
|
-
|
|
67
|
-
### 3. Transaction and MVCC semantics (validated foundation)
|
|
68
|
-
|
|
69
|
-
The transaction manager and visibility map exist, transaction rollback restores
|
|
70
|
-
update/delete before-images, and read-committed, repeatable-read, serializable
|
|
71
|
-
conflict, deadlock, and concurrency workload paths have regression coverage.
|
|
72
|
-
Distributed isolation and deployment-specific contention limits remain open.
|
|
73
|
-
|
|
74
|
-
### 4. SQL engine is a documented subset
|
|
75
|
-
|
|
76
|
-
The parser, planner, and executor are covered through end-to-end tests for the
|
|
77
|
-
documented RubyDB SQL subset, including joins, aggregates, CTEs, subqueries,
|
|
78
|
-
set operations, upserts, and window functions. A common SQLite-style profile
|
|
79
|
-
is tested for new Ruby/Rails applications; full PostgreSQL/MySQL/SQLite dialect
|
|
80
|
-
and file-format compatibility remains intentionally out of scope for this
|
|
81
|
-
phase.
|
|
82
|
-
|
|
83
|
-
### 5. Security (validated foundation)
|
|
84
|
-
|
|
85
|
-
Authentication, TLS, SCRAM, authorization, framing limits, deadlines, and
|
|
86
|
-
resource-limit startup checks are tied to server/query execution and tested.
|
|
87
|
-
An independent security review and production certificate/secret lifecycle
|
|
88
|
-
validation remain open.
|
|
89
|
-
|
|
90
|
-
### 6. Production posture is intentionally bounded
|
|
91
|
-
|
|
92
|
-
The README and production documents now distinguish tested behavior from
|
|
93
|
-
deployment work. RubyDB still needs environment-specific capacity, filesystem,
|
|
94
|
-
multi-host failover, certificate lifecycle, and independent security validation
|
|
95
|
-
before broader production claims are appropriate.
|
|
96
|
-
|
|
97
|
-
## Current production posture
|
|
98
|
-
|
|
99
|
-
RubyDB is best understood as:
|
|
100
|
-
|
|
101
|
-
- a serious Ruby database foundation with repeatable local validation,
|
|
102
|
-
- suitable for controlled workloads within the documented feature set,
|
|
103
|
-
- not a drop-in general-purpose database or automatic high-availability service.
|
|
104
|
-
|
|
105
|
-
The correct short-term operating posture is to treat the project as a pre-production, high-potential codebase that needs disciplined validation and narrower, correctness-first milestones.
|
|
106
|
-
|
|
107
|
-
## Immediate action items
|
|
108
|
-
|
|
109
|
-
1. Run the hosted Ruby/Rails/OS compatibility matrix and retain its evidence.
|
|
110
|
-
2. Validate real disk quota, power-loss, and filesystem behavior on deployment targets.
|
|
111
|
-
3. Validate multi-host partition, split-brain, fencing, and operator failover procedures.
|
|
112
|
-
4. Complete an independent security review and certificate/secret rotation drill.
|
|
113
|
-
5. Implement or intentionally reject unsupported features with explicit errors.
|
|
114
|
-
6. Establish workload-specific capacity baselines and release sign-off records.
|
|
115
|
-
|
|
116
|
-
The latest local audit passed 270 examples with zero failures. Hosted matrix,
|
|
117
|
-
container, filesystem quota/power-loss, multi-host fencing, and independent
|
|
118
|
-
security-review evidence must still be retained for a production deployment.
|
|
119
|
-
|
|
120
|
-
## Conclusion
|
|
121
|
-
|
|
122
|
-
RubyDB has a substantial production-oriented foundation and repeatable local
|
|
123
|
-
validation. Operators must stay within the documented feature set and complete
|
|
124
|
-
deployment-specific capacity, filesystem, security, and multi-host failover
|
|
125
|
-
validation before entrusting critical data to it.
|
|
1
|
+
# RubyDB current-state audit
|
|
2
|
+
|
|
3
|
+
> This document is a dated architecture audit, not a statement that every
|
|
4
|
+
> finding remains open. For current validation evidence, see
|
|
5
|
+
> [production validation](../production_validation.md) and the latest CI runs.
|
|
6
|
+
|
|
7
|
+
## Executive summary
|
|
8
|
+
|
|
9
|
+
RubyDB is a broad database project with integrated storage, WAL, MVCC,
|
|
10
|
+
transactions, catalog, security, server, replication, backup, Rails, and
|
|
11
|
+
monitoring paths. The implementation now has substantial focused validation,
|
|
12
|
+
but several deployment-level capabilities remain environment-specific or
|
|
13
|
+
intentionally bounded.
|
|
14
|
+
|
|
15
|
+
The primary risk is not that the code is absent; it is that multiple components exist without being integrated into a consistent, correct, durable engine. Existing code needs additional hardening before it can safely store user data.
|
|
16
|
+
|
|
17
|
+
## Repository audit
|
|
18
|
+
|
|
19
|
+
### Runtime and package state
|
|
20
|
+
|
|
21
|
+
- Ruby runtime in the development container: Ruby 3.4.7
|
|
22
|
+
- Package metadata targets Ruby >= 3.3.0; CI covers Ruby 3.3 and 3.4 on Linux,
|
|
23
|
+
macOS, and Windows.
|
|
24
|
+
- The top-level library entrypoint and gem packaging are covered by
|
|
25
|
+
`require "rubydb"` and release-artifact tests.
|
|
26
|
+
- The project uses Bundler and RSpec as the base test stack.
|
|
27
|
+
- The repository is organized around a large `lib/rubydb` module tree and includes docs, examples, chaos, and fuzz directories.
|
|
28
|
+
|
|
29
|
+
### Architectural reality
|
|
30
|
+
|
|
31
|
+
The project already contains modules for:
|
|
32
|
+
|
|
33
|
+
- storage engine and page manager
|
|
34
|
+
- buffer pool and file management
|
|
35
|
+
- WAL and recovery components
|
|
36
|
+
- transactions and MVCC
|
|
37
|
+
- catalog, indexes, and constraints
|
|
38
|
+
- SQL parser and planner
|
|
39
|
+
- server and protocol layers
|
|
40
|
+
- security and authorization
|
|
41
|
+
- replication, backups, and monitoring
|
|
42
|
+
- Rails integration
|
|
43
|
+
|
|
44
|
+
The remaining risk is uneven validation breadth: many capabilities are proven
|
|
45
|
+
by focused tests, but multi-host operations, capacity limits, and complete
|
|
46
|
+
external-database dialect compatibility are not yet certified.
|
|
47
|
+
|
|
48
|
+
## High-risk findings
|
|
49
|
+
|
|
50
|
+
### 1. Library loading (resolved)
|
|
51
|
+
|
|
52
|
+
The standard Ruby consumer path is now tested:
|
|
53
|
+
|
|
54
|
+
```ruby
|
|
55
|
+
require "rubydb"
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
Release metadata and the local gem build verify that this loads correctly.
|
|
59
|
+
|
|
60
|
+
### 2. Storage layer (partially resolved)
|
|
61
|
+
|
|
62
|
+
The storage engine now has page validation, corruption detection, WAL/recovery,
|
|
63
|
+
fsync paths, restart tests, subprocess crash tests, filesystem fault injection,
|
|
64
|
+
and compaction/reopen coverage. Real disk-quota and power-loss validation on
|
|
65
|
+
deployment filesystems remains open.
|
|
66
|
+
|
|
67
|
+
### 3. Transaction and MVCC semantics (validated foundation)
|
|
68
|
+
|
|
69
|
+
The transaction manager and visibility map exist, transaction rollback restores
|
|
70
|
+
update/delete before-images, and read-committed, repeatable-read, serializable
|
|
71
|
+
conflict, deadlock, and concurrency workload paths have regression coverage.
|
|
72
|
+
Distributed isolation and deployment-specific contention limits remain open.
|
|
73
|
+
|
|
74
|
+
### 4. SQL engine is a documented subset
|
|
75
|
+
|
|
76
|
+
The parser, planner, and executor are covered through end-to-end tests for the
|
|
77
|
+
documented RubyDB SQL subset, including joins, aggregates, CTEs, subqueries,
|
|
78
|
+
set operations, upserts, and window functions. A common SQLite-style profile
|
|
79
|
+
is tested for new Ruby/Rails applications; full PostgreSQL/MySQL/SQLite dialect
|
|
80
|
+
and file-format compatibility remains intentionally out of scope for this
|
|
81
|
+
phase.
|
|
82
|
+
|
|
83
|
+
### 5. Security (validated foundation)
|
|
84
|
+
|
|
85
|
+
Authentication, TLS, SCRAM, authorization, framing limits, deadlines, and
|
|
86
|
+
resource-limit startup checks are tied to server/query execution and tested.
|
|
87
|
+
An independent security review and production certificate/secret lifecycle
|
|
88
|
+
validation remain open.
|
|
89
|
+
|
|
90
|
+
### 6. Production posture is intentionally bounded
|
|
91
|
+
|
|
92
|
+
The README and production documents now distinguish tested behavior from
|
|
93
|
+
deployment work. RubyDB still needs environment-specific capacity, filesystem,
|
|
94
|
+
multi-host failover, certificate lifecycle, and independent security validation
|
|
95
|
+
before broader production claims are appropriate.
|
|
96
|
+
|
|
97
|
+
## Current production posture
|
|
98
|
+
|
|
99
|
+
RubyDB is best understood as:
|
|
100
|
+
|
|
101
|
+
- a serious Ruby database foundation with repeatable local validation,
|
|
102
|
+
- suitable for controlled workloads within the documented feature set,
|
|
103
|
+
- not a drop-in general-purpose database or automatic high-availability service.
|
|
104
|
+
|
|
105
|
+
The correct short-term operating posture is to treat the project as a pre-production, high-potential codebase that needs disciplined validation and narrower, correctness-first milestones.
|
|
106
|
+
|
|
107
|
+
## Immediate action items
|
|
108
|
+
|
|
109
|
+
1. Run the hosted Ruby/Rails/OS compatibility matrix and retain its evidence.
|
|
110
|
+
2. Validate real disk quota, power-loss, and filesystem behavior on deployment targets.
|
|
111
|
+
3. Validate multi-host partition, split-brain, fencing, and operator failover procedures.
|
|
112
|
+
4. Complete an independent security review and certificate/secret rotation drill.
|
|
113
|
+
5. Implement or intentionally reject unsupported features with explicit errors.
|
|
114
|
+
6. Establish workload-specific capacity baselines and release sign-off records.
|
|
115
|
+
|
|
116
|
+
The latest local audit passed 270 examples with zero failures. Hosted matrix,
|
|
117
|
+
container, filesystem quota/power-loss, multi-host fencing, and independent
|
|
118
|
+
security-review evidence must still be retained for a production deployment.
|
|
119
|
+
|
|
120
|
+
## Conclusion
|
|
121
|
+
|
|
122
|
+
RubyDB has a substantial production-oriented foundation and repeatable local
|
|
123
|
+
validation. Operators must stay within the documented feature set and complete
|
|
124
|
+
deployment-specific capacity, filesystem, security, and multi-host failover
|
|
125
|
+
validation before entrusting critical data to it.
|
|
@@ -1,25 +1,25 @@
|
|
|
1
|
-
# Execution engine
|
|
2
|
-
|
|
3
|
-
The execution pipeline is:
|
|
4
|
-
|
|
5
|
-
`SQL text -> lexer -> parser/AST -> binder -> planner -> executor -> result`
|
|
6
|
-
|
|
7
|
-
The planner selects scans, index scans, joins, filters, aggregates, sorting,
|
|
8
|
-
limits, set operations, CTEs, subqueries, and window operations supported by
|
|
9
|
-
the documented dialect. DML is routed through transaction-aware executors and
|
|
10
|
-
the engine's durable mutation path.
|
|
11
|
-
|
|
12
|
-
Unsupported syntax must produce an explicit parser or execution error. New
|
|
13
|
-
operators and functions require type-checking, null behavior, transaction
|
|
14
|
-
coverage, and compatibility documentation.
|
|
15
|
-
|
|
16
|
-
## Execution invariants
|
|
17
|
-
|
|
18
|
-
The executor must consume a bound plan, check cancellation and deadlines at
|
|
19
|
-
bounded points, and return either a complete result or an error that leaves the
|
|
20
|
-
transaction in a documented state. DML goes through the same transaction and
|
|
21
|
-
WAL path as the Ruby API; a server response must never bypass durable commit.
|
|
22
|
-
|
|
23
|
-
When diagnosing a result, compare parser output, bound parameters, selected
|
|
24
|
-
plan, row visibility, and final serialization. See the [developer guide](../developer-guide.md)
|
|
25
|
-
and [debugging playbook](../debugging.md).
|
|
1
|
+
# Execution engine
|
|
2
|
+
|
|
3
|
+
The execution pipeline is:
|
|
4
|
+
|
|
5
|
+
`SQL text -> lexer -> parser/AST -> binder -> planner -> executor -> result`
|
|
6
|
+
|
|
7
|
+
The planner selects scans, index scans, joins, filters, aggregates, sorting,
|
|
8
|
+
limits, set operations, CTEs, subqueries, and window operations supported by
|
|
9
|
+
the documented dialect. DML is routed through transaction-aware executors and
|
|
10
|
+
the engine's durable mutation path.
|
|
11
|
+
|
|
12
|
+
Unsupported syntax must produce an explicit parser or execution error. New
|
|
13
|
+
operators and functions require type-checking, null behavior, transaction
|
|
14
|
+
coverage, and compatibility documentation.
|
|
15
|
+
|
|
16
|
+
## Execution invariants
|
|
17
|
+
|
|
18
|
+
The executor must consume a bound plan, check cancellation and deadlines at
|
|
19
|
+
bounded points, and return either a complete result or an error that leaves the
|
|
20
|
+
transaction in a documented state. DML goes through the same transaction and
|
|
21
|
+
WAL path as the Ruby API; a server response must never bypass durable commit.
|
|
22
|
+
|
|
23
|
+
When diagnosing a result, compare parser output, bound parameters, selected
|
|
24
|
+
plan, row visibility, and final serialization. See the [developer guide](../developer-guide.md)
|
|
25
|
+
and [debugging playbook](../debugging.md).
|
|
@@ -1,19 +1,19 @@
|
|
|
1
|
-
# Indexes
|
|
2
|
-
|
|
3
|
-
RubyDB provides B-tree indexes for supported table columns, including unique
|
|
4
|
-
indexes. The executor can use indexes for eligible lookups while preserving
|
|
5
|
-
correctness through table visibility and transaction rules.
|
|
6
|
-
|
|
7
|
-
Index metadata is persisted with the catalog. Index creation, DML maintenance,
|
|
8
|
-
rollback, deep splits, reopen, and failure handling must remain consistent. If
|
|
9
|
-
index recovery fails, startup must fail visibly; never silently rebuild or claim
|
|
10
|
-
an index is healthy without verification.
|
|
11
|
-
|
|
12
|
-
## Change checklist
|
|
13
|
-
|
|
14
|
-
Index changes require coverage for empty and populated tables, duplicate and
|
|
15
|
-
null keys, deep splits, deletes, rollback, crash/replay, compaction, and
|
|
16
|
-
concurrent readers. Compare an index lookup with a table scan before and after
|
|
17
|
-
reopen. A required-index persistence error must abort the owning operation and
|
|
18
|
-
be observable by the caller. See [troubleshooting](../troubleshooting.md) for
|
|
19
|
-
the safe response to suspected index corruption.
|
|
1
|
+
# Indexes
|
|
2
|
+
|
|
3
|
+
RubyDB provides B-tree indexes for supported table columns, including unique
|
|
4
|
+
indexes. The executor can use indexes for eligible lookups while preserving
|
|
5
|
+
correctness through table visibility and transaction rules.
|
|
6
|
+
|
|
7
|
+
Index metadata is persisted with the catalog. Index creation, DML maintenance,
|
|
8
|
+
rollback, deep splits, reopen, and failure handling must remain consistent. If
|
|
9
|
+
index recovery fails, startup must fail visibly; never silently rebuild or claim
|
|
10
|
+
an index is healthy without verification.
|
|
11
|
+
|
|
12
|
+
## Change checklist
|
|
13
|
+
|
|
14
|
+
Index changes require coverage for empty and populated tables, duplicate and
|
|
15
|
+
null keys, deep splits, deletes, rollback, crash/replay, compaction, and
|
|
16
|
+
concurrent readers. Compare an index lookup with a table scan before and after
|
|
17
|
+
reopen. A required-index persistence error must abort the owning operation and
|
|
18
|
+
be observable by the caller. See [troubleshooting](../troubleshooting.md) for
|
|
19
|
+
the safe response to suspected index corruption.
|
data/docs/architecture/mvcc.md
CHANGED
|
@@ -1,19 +1,19 @@
|
|
|
1
|
-
# MVCC support status
|
|
2
|
-
|
|
3
|
-
RubyDB currently provides safe READ COMMITTED behavior for the storage engine.
|
|
4
|
-
Uncommitted row versions are visible to their owning transaction and hidden
|
|
5
|
-
from other transactions. Committed versions become visible after the durable
|
|
6
|
-
COMMIT WAL record is written. Deleted physical records remain on pages until
|
|
7
|
-
vacuum/compaction and are excluded from ordinary scans.
|
|
8
|
-
|
|
9
|
-
REPEATABLE READ uses persisted historical row versions and transaction
|
|
10
|
-
snapshots. SERIALIZABLE uses snapshot validation over tracked row read/write
|
|
11
|
-
keys and conservative table predicates. A newer committed version intersecting
|
|
12
|
-
the dependency set aborts commit with a serialization failure. Table-level
|
|
13
|
-
predicate tracking prevents phantoms at the cost of false-positive conflicts;
|
|
14
|
-
exact index-range predicate locking is a future optimization.
|
|
15
|
-
|
|
16
|
-
The version store is persisted atomically and supports safe-point vacuuming.
|
|
17
|
-
Vacuum never removes active versions and retains the newest committed base
|
|
18
|
-
version needed by an active reader; older committed history is removable only
|
|
19
|
-
when its commit ID precedes the oldest active transaction.
|
|
1
|
+
# MVCC support status
|
|
2
|
+
|
|
3
|
+
RubyDB currently provides safe READ COMMITTED behavior for the storage engine.
|
|
4
|
+
Uncommitted row versions are visible to their owning transaction and hidden
|
|
5
|
+
from other transactions. Committed versions become visible after the durable
|
|
6
|
+
COMMIT WAL record is written. Deleted physical records remain on pages until
|
|
7
|
+
vacuum/compaction and are excluded from ordinary scans.
|
|
8
|
+
|
|
9
|
+
REPEATABLE READ uses persisted historical row versions and transaction
|
|
10
|
+
snapshots. SERIALIZABLE uses snapshot validation over tracked row read/write
|
|
11
|
+
keys and conservative table predicates. A newer committed version intersecting
|
|
12
|
+
the dependency set aborts commit with a serialization failure. Table-level
|
|
13
|
+
predicate tracking prevents phantoms at the cost of false-positive conflicts;
|
|
14
|
+
exact index-range predicate locking is a future optimization.
|
|
15
|
+
|
|
16
|
+
The version store is persisted atomically and supports safe-point vacuuming.
|
|
17
|
+
Vacuum never removes active versions and retains the newest committed base
|
|
18
|
+
version needed by an active reader; older committed history is removable only
|
|
19
|
+
when its commit ID precedes the oldest active transaction.
|
|
@@ -1,13 +1,13 @@
|
|
|
1
|
-
# Architecture overview
|
|
2
|
-
|
|
3
|
-
RubyDB separates logical query processing from durable state changes. SQL is
|
|
4
|
-
lexed into tokens, parsed into an AST, bound against catalog metadata, planned,
|
|
5
|
-
and executed against the storage engine. DML is coordinated with transactions
|
|
6
|
-
and WAL before durable acknowledgement.
|
|
7
|
-
|
|
8
|
-
The catalog defines tables, columns, constraints, views, and indexes. The page
|
|
9
|
-
manager and buffer pool provide durable storage. Recovery replays valid WAL and
|
|
10
|
-
rejects malformed or uncertain state. Server/client mode isolates application
|
|
11
|
-
processes from the exclusive embedded owner.
|
|
12
|
-
|
|
13
|
-
Read the [current-state audit](current-state.md) before relying on any feature.
|
|
1
|
+
# Architecture overview
|
|
2
|
+
|
|
3
|
+
RubyDB separates logical query processing from durable state changes. SQL is
|
|
4
|
+
lexed into tokens, parsed into an AST, bound against catalog metadata, planned,
|
|
5
|
+
and executed against the storage engine. DML is coordinated with transactions
|
|
6
|
+
and WAL before durable acknowledgement.
|
|
7
|
+
|
|
8
|
+
The catalog defines tables, columns, constraints, views, and indexes. The page
|
|
9
|
+
manager and buffer pool provide durable storage. Recovery replays valid WAL and
|
|
10
|
+
rejects malformed or uncertain state. Server/client mode isolates application
|
|
11
|
+
processes from the exclusive embedded owner.
|
|
12
|
+
|
|
13
|
+
Read the [current-state audit](current-state.md) before relying on any feature.
|
data/docs/architecture/pages.md
CHANGED
|
@@ -1,11 +1,11 @@
|
|
|
1
|
-
# Pages and file layout
|
|
2
|
-
|
|
3
|
-
RubyDB stores fixed-size pages containing validated headers, checksums, and
|
|
4
|
-
typed records. Page allocation, metadata, table data, and index pages are
|
|
5
|
-
managed by the storage layer; callers must not edit database files directly.
|
|
6
|
-
|
|
7
|
-
The page format is versioned. Unknown formats, invalid page sizes, malformed
|
|
8
|
-
headers, and checksum failures fail closed during open or recovery. Keep the
|
|
9
|
-
database, WAL, catalog metadata, and lock files together when copying or
|
|
10
|
-
restoring a database. See [storage format](../../spec/storage/format.md) and
|
|
11
|
-
[upgrade guidance](../operations/upgrades.md).
|
|
1
|
+
# Pages and file layout
|
|
2
|
+
|
|
3
|
+
RubyDB stores fixed-size pages containing validated headers, checksums, and
|
|
4
|
+
typed records. Page allocation, metadata, table data, and index pages are
|
|
5
|
+
managed by the storage layer; callers must not edit database files directly.
|
|
6
|
+
|
|
7
|
+
The page format is versioned. Unknown formats, invalid page sizes, malformed
|
|
8
|
+
headers, and checksum failures fail closed during open or recovery. Keep the
|
|
9
|
+
database, WAL, catalog metadata, and lock files together when copying or
|
|
10
|
+
restoring a database. See [storage format](../../spec/storage/format.md) and
|
|
11
|
+
[upgrade guidance](../operations/upgrades.md).
|