rubydb 0.1.4 → 0.1.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (469) hide show
  1. checksums.yaml +4 -4
  2. data/.github/PULL_REQUEST_TEMPLATE.md +15 -15
  3. data/.github/workflows/benchmark.yml +26 -26
  4. data/.github/workflows/compatibility.yml +63 -63
  5. data/.github/workflows/fuzz.yml +33 -33
  6. data/.github/workflows/lint.yml +21 -21
  7. data/.github/workflows/operations.yml +24 -24
  8. data/.github/workflows/production-validation.yml +111 -111
  9. data/.github/workflows/release.yml +77 -77
  10. data/.github/workflows/security.yml +39 -37
  11. data/.github/workflows/test.yml +26 -26
  12. data/.github/workflows/workload.yml +58 -58
  13. data/.gitignore +16 -5
  14. data/.rubocop.yml +50 -44
  15. data/.standard.yml +9 -14
  16. data/ARCHITECTURE.md +21 -21
  17. data/CHANGELOG.md +57 -27
  18. data/CODE_OF_CONDUCT.md +13 -13
  19. data/CONTRIBUTING.md +29 -29
  20. data/GOVERNANCE.md +16 -16
  21. data/Gemfile +18 -17
  22. data/Gemfile.lock +125 -71
  23. data/README.md +168 -12
  24. data/ROADMAP.md +27 -27
  25. data/Rakefile +76 -71
  26. data/SECURITY.md +54 -54
  27. data/SUPPORT.md +14 -14
  28. data/accelerator/bin/SHA256SUMS +6 -0
  29. data/accelerator/bin/rubydb-accelerator-darwin-amd64 +0 -0
  30. data/accelerator/bin/rubydb-accelerator-darwin-arm64 +0 -0
  31. data/accelerator/bin/rubydb-accelerator-linux-amd64 +0 -0
  32. data/accelerator/bin/rubydb-accelerator-linux-arm64 +0 -0
  33. data/accelerator/bin/rubydb-accelerator-windows-amd64.exe +0 -0
  34. data/accelerator/bin/rubydb-accelerator-windows-arm64.exe +0 -0
  35. data/accelerator/cmd/rubydb-accelerator/main.go +11 -0
  36. data/accelerator/go.mod +3 -0
  37. data/accelerator/internal/execution/aggregate.go +94 -0
  38. data/accelerator/internal/execution/distinct.go +22 -0
  39. data/accelerator/internal/execution/filter.go +73 -0
  40. data/accelerator/internal/execution/join.go +79 -0
  41. data/accelerator/internal/execution/operators.go +167 -0
  42. data/accelerator/internal/execution/scan.go +20 -0
  43. data/accelerator/internal/execution/sort.go +62 -0
  44. data/accelerator/internal/execution/types.go +136 -0
  45. data/accelerator/internal/execution/value.go +67 -0
  46. data/accelerator/internal/memory/arena.go +47 -0
  47. data/accelerator/internal/memory/reuse.go +22 -0
  48. data/accelerator/internal/metrics/registry.go +67 -0
  49. data/accelerator/internal/parallel/bounded_queue.go +56 -0
  50. data/accelerator/internal/parallel/scheduler.go +47 -0
  51. data/accelerator/internal/parallel/worker_pool.go +53 -0
  52. data/accelerator/internal/protocol/cancellation.go +48 -0
  53. data/accelerator/internal/protocol/columnar.go +263 -0
  54. data/accelerator/internal/protocol/frame.go +187 -0
  55. data/accelerator/internal/runtime/worker.go +521 -0
  56. data/accelerator/internal/storage/page_reader.go +81 -0
  57. data/accelerator/internal/storage/snapshot_scan.go +539 -0
  58. data/accelerator/internal/wal/checksum.go +13 -0
  59. data/accelerator/internal/wal/compression.go +41 -0
  60. data/accelerator/internal/wal/group_commit.go +24 -0
  61. data/accelerator/internal/wal/record_encoder.go +40 -0
  62. data/adapters/activerecord/Gemfile +11 -11
  63. data/adapters/activerecord/README.md +8 -3
  64. data/adapters/activerecord/lib/active_record/connection_adapters/rubydb_adapter.rb +881 -879
  65. data/adapters/activerecord/rubydb-activerecord.gemspec +21 -21
  66. data/adapters/activerecord/spec/rubydb_adapter_integration_spec.rb +143 -143
  67. data/adapters/ruby/README.md +18 -18
  68. data/adapters/sequel/README.md +11 -11
  69. data/config/monitoring/prometheus-alerts.yml +39 -39
  70. data/config/production.yml +36 -36
  71. data/docs/README.md +77 -71
  72. data/docs/architecture/concurrency.md +14 -14
  73. data/docs/architecture/current-state.md +125 -125
  74. data/docs/architecture/execution-engine.md +25 -25
  75. data/docs/architecture/go-accelerator.md +179 -0
  76. data/docs/architecture/indexes.md +19 -19
  77. data/docs/architecture/mvcc.md +19 -19
  78. data/docs/architecture/overview.md +13 -13
  79. data/docs/architecture/pages.md +11 -11
  80. data/docs/architecture/production-roadmap.md +82 -82
  81. data/docs/architecture/query-planner.md +20 -20
  82. data/docs/architecture/recovery.md +18 -18
  83. data/docs/architecture/sql-engine.md +12 -12
  84. data/docs/architecture/storage-engine.md +14 -14
  85. data/docs/architecture/transactions.md +10 -10
  86. data/docs/architecture/wal.md +28 -28
  87. data/docs/cli-cheatsheet.md +98 -98
  88. data/docs/cli.md +299 -275
  89. data/docs/contributing/architecture.md +9 -9
  90. data/docs/contributing/benchmarking.md +30 -14
  91. data/docs/contributing/development.md +16 -16
  92. data/docs/contributing/release-process.md +49 -49
  93. data/docs/contributing/testing.md +16 -16
  94. data/docs/debugging.md +229 -229
  95. data/docs/developer/branching.md +10 -10
  96. data/docs/developer/database-diff.md +10 -10
  97. data/docs/developer/local-development.md +49 -17
  98. data/docs/developer/snapshots.md +9 -9
  99. data/docs/developer/temporal-data.md +10 -10
  100. data/docs/developer-guide.md +297 -297
  101. data/docs/getting-started/first-database.md +16 -16
  102. data/docs/getting-started/first-query.md +13 -13
  103. data/docs/getting-started/installation.md +19 -19
  104. data/docs/getting-started/local-to-production.md +300 -300
  105. data/docs/getting-started/quickstart.md +17 -17
  106. data/docs/getting-started/rails.md +16 -16
  107. data/docs/hardening_backlog.md +93 -93
  108. data/docs/lessons-learned.md +112 -112
  109. data/docs/operations/backups.md +33 -33
  110. data/docs/operations/disaster-recovery.md +31 -31
  111. data/docs/operations/failover.md +30 -30
  112. data/docs/operations/monitoring.md +25 -25
  113. data/docs/operations/production-guide.md +295 -295
  114. data/docs/operations/production-runbook.md +45 -45
  115. data/docs/operations/replication.md +33 -33
  116. data/docs/operations/restore.md +6 -6
  117. data/docs/operations/runbook.md +34 -34
  118. data/docs/operations/upgrades.md +14 -14
  119. data/docs/operations/workload-testing.md +17 -17
  120. data/docs/production-readiness.md +118 -118
  121. data/docs/production_validation.md +150 -150
  122. data/docs/rails/active-record.md +11 -11
  123. data/docs/rails/compatibility-guide.md +90 -90
  124. data/docs/rails/database-yml.md +92 -92
  125. data/docs/rails/installation.md +17 -17
  126. data/docs/rails/migrations.md +17 -17
  127. data/docs/rails/production.md +82 -82
  128. data/docs/rails/troubleshooting.md +18 -18
  129. data/docs/release.md +25 -25
  130. data/docs/server/architecture.md +10 -10
  131. data/docs/server/authentication.md +10 -10
  132. data/docs/server/configuration.md +16 -16
  133. data/docs/server/connection-pooling.md +10 -10
  134. data/docs/server/deployment.md +10 -10
  135. data/docs/server/protocol.md +12 -12
  136. data/docs/sql/compatibility-guide.md +82 -82
  137. data/docs/sql/compatibility.md +39 -39
  138. data/docs/sql/data-types.md +10 -10
  139. data/docs/sql/functions.md +9 -9
  140. data/docs/sql/joins.md +9 -9
  141. data/docs/sql/operators.md +9 -9
  142. data/docs/sql/sqlite-compatibility.md +21 -21
  143. data/docs/sql/syntax.md +10 -10
  144. data/docs/sql/transactions.md +10 -10
  145. data/docs/troubleshooting.md +244 -244
  146. data/lessons/01-foundations.md +73 -0
  147. data/lessons/02-local-development.md +121 -0
  148. data/lessons/03-embedded-rubydb.md +99 -0
  149. data/lessons/04-rails-complex-apps.md +138 -0
  150. data/lessons/05-rubydb-production-server.md +237 -0
  151. data/lessons/06-postgresql-massive-apps.md +96 -0
  152. data/lessons/07-hybrid-microservices.md +179 -0
  153. data/lessons/08-migrations-backups-recovery.md +86 -0
  154. data/lessons/09-observability-security-scale.md +87 -0
  155. data/lessons/10-release-readiness.md +192 -0
  156. data/lessons/11-community-adapter.md +323 -0
  157. data/lessons/12-rails-ecommerce-pressure.md +263 -0
  158. data/lib/rubydb/accelerator/client.rb +451 -0
  159. data/lib/rubydb/accelerator/error.rb +22 -0
  160. data/lib/rubydb/accelerator/manager.rb +606 -0
  161. data/lib/rubydb/accelerator.rb +13 -0
  162. data/lib/rubydb/backup/archive.rb +332 -334
  163. data/lib/rubydb/backup/backup.rb +400 -401
  164. data/lib/rubydb/backup/incremental.rb +349 -353
  165. data/lib/rubydb/backup/restore.rb +289 -290
  166. data/lib/rubydb/backup/snapshot.rb +265 -267
  167. data/lib/rubydb/backup/verification.rb +276 -279
  168. data/lib/rubydb/branching/branch.rb +181 -181
  169. data/lib/rubydb/branching/branch_manager.rb +307 -311
  170. data/lib/rubydb/branching/branch_metadata.rb +140 -140
  171. data/lib/rubydb/branching/checkout.rb +165 -166
  172. data/lib/rubydb/branching/copy_on_write.rb +272 -272
  173. data/lib/rubydb/branching/diff.rb +137 -138
  174. data/lib/rubydb/branching/merge.rb +282 -285
  175. data/lib/rubydb/build_info.rb +15 -15
  176. data/lib/rubydb/catalog/catalog.rb +391 -391
  177. data/lib/rubydb/catalog/column.rb +112 -112
  178. data/lib/rubydb/catalog/constraint.rb +180 -180
  179. data/lib/rubydb/catalog/database.rb +184 -184
  180. data/lib/rubydb/catalog/index.rb +97 -97
  181. data/lib/rubydb/catalog/schema.rb +103 -103
  182. data/lib/rubydb/catalog/sequence.rb +90 -90
  183. data/lib/rubydb/catalog/system_catalog.rb +698 -698
  184. data/lib/rubydb/catalog/table.rb +178 -178
  185. data/lib/rubydb/catalog/trigger.rb +102 -102
  186. data/lib/rubydb/catalog/view.rb +66 -66
  187. data/lib/rubydb/cli/application.rb +168 -163
  188. data/lib/rubydb/cli/commands/accelerator.rb +72 -0
  189. data/lib/rubydb/cli/commands/backup.rb +80 -81
  190. data/lib/rubydb/cli/commands/branch.rb +72 -72
  191. data/lib/rubydb/cli/commands/checkout.rb +54 -54
  192. data/lib/rubydb/cli/commands/create.rb +58 -58
  193. data/lib/rubydb/cli/commands/diff.rb +76 -77
  194. data/lib/rubydb/cli/commands/doctor.rb +77 -74
  195. data/lib/rubydb/cli/commands/drop.rb +57 -57
  196. data/lib/rubydb/cli/commands/init.rb +101 -102
  197. data/lib/rubydb/cli/commands/inspect.rb +95 -95
  198. data/lib/rubydb/cli/commands/merge.rb +63 -63
  199. data/lib/rubydb/cli/commands/migrate.rb +62 -62
  200. data/lib/rubydb/cli/commands/restart.rb +42 -39
  201. data/lib/rubydb/cli/commands/restore.rb +121 -121
  202. data/lib/rubydb/cli/commands/shell.rb +365 -365
  203. data/lib/rubydb/cli/commands/snapshot.rb +79 -79
  204. data/lib/rubydb/cli/commands/start.rb +88 -82
  205. data/lib/rubydb/cli/commands/status.rb +96 -92
  206. data/lib/rubydb/cli/commands/stop.rb +47 -47
  207. data/lib/rubydb/cli/commands/vacuum.rb +58 -58
  208. data/lib/rubydb/cli/formatter.rb +221 -221
  209. data/lib/rubydb/cli/output.rb +168 -168
  210. data/lib/rubydb/client/client.rb +309 -304
  211. data/lib/rubydb/client/connection.rb +429 -415
  212. data/lib/rubydb/client/connection_pool.rb +168 -168
  213. data/lib/rubydb/client/connection_url.rb +96 -96
  214. data/lib/rubydb/client/prepared_statement.rb +60 -60
  215. data/lib/rubydb/client/result.rb +127 -123
  216. data/lib/rubydb/client/statement.rb +52 -52
  217. data/lib/rubydb/client/transaction.rb +130 -130
  218. data/lib/rubydb/concurrency/concurrency.rb +19 -19
  219. data/lib/rubydb/concurrency/deadlock_detector.rb +148 -150
  220. data/lib/rubydb/concurrency/latch.rb +101 -101
  221. data/lib/rubydb/concurrency/lock_graph.rb +163 -165
  222. data/lib/rubydb/concurrency/mutex.rb +181 -183
  223. data/lib/rubydb/concurrency/rw_lock.rb +180 -180
  224. data/lib/rubydb/concurrency/scheduler.rb +248 -250
  225. data/lib/rubydb/concurrency/worker_pool.rb +145 -143
  226. data/lib/rubydb/configuration/config.rb +170 -170
  227. data/lib/rubydb/configuration/defaults.rb +191 -179
  228. data/lib/rubydb/configuration/environment.rb +152 -152
  229. data/lib/rubydb/configuration/parser.rb +185 -185
  230. data/lib/rubydb/configuration/validation.rb +228 -221
  231. data/lib/rubydb/constants.rb +74 -74
  232. data/lib/rubydb/constraints/check.rb +181 -181
  233. data/lib/rubydb/constraints/constraint.rb +101 -101
  234. data/lib/rubydb/constraints/foreign_key.rb +130 -130
  235. data/lib/rubydb/constraints/not_null.rb +64 -64
  236. data/lib/rubydb/constraints/primary_key.rb +99 -99
  237. data/lib/rubydb/constraints/unique.rb +106 -108
  238. data/lib/rubydb/constraints/validator.rb +349 -350
  239. data/lib/rubydb/errors/authentication_error.rb +10 -10
  240. data/lib/rubydb/errors/authorization_error.rb +23 -23
  241. data/lib/rubydb/errors/client_error.rb +10 -10
  242. data/lib/rubydb/errors/configuration_error.rb +10 -10
  243. data/lib/rubydb/errors/connection_error.rb +10 -10
  244. data/lib/rubydb/errors/constraint_error.rb +23 -23
  245. data/lib/rubydb/errors/corruption_error.rb +10 -10
  246. data/lib/rubydb/errors/database_error.rb +10 -10
  247. data/lib/rubydb/errors/error.rb +20 -20
  248. data/lib/rubydb/errors/execution_error.rb +10 -10
  249. data/lib/rubydb/errors/parser_error.rb +10 -10
  250. data/lib/rubydb/errors/recovery_error.rb +10 -10
  251. data/lib/rubydb/errors/replication_error.rb +10 -10
  252. data/lib/rubydb/errors/server_error.rb +6 -6
  253. data/lib/rubydb/errors/storage_error.rb +10 -10
  254. data/lib/rubydb/errors/transaction_error.rb +10 -10
  255. data/lib/rubydb/execution/accelerator_dispatch.rb +30 -0
  256. data/lib/rubydb/execution/aggregate_executor.rb +134 -138
  257. data/lib/rubydb/execution/cost_model.rb +72 -0
  258. data/lib/rubydb/execution/delete_executor.rb +110 -112
  259. data/lib/rubydb/execution/distinct_executor.rb +131 -135
  260. data/lib/rubydb/execution/executor.rb +1544 -1188
  261. data/lib/rubydb/execution/expression.rb +191 -193
  262. data/lib/rubydb/execution/index_scan.rb +142 -142
  263. data/lib/rubydb/execution/insert_executor.rb +215 -217
  264. data/lib/rubydb/execution/join_executor.rb +243 -249
  265. data/lib/rubydb/execution/limit_executor.rb +83 -85
  266. data/lib/rubydb/execution/operator_selection.rb +57 -0
  267. data/lib/rubydb/execution/optimizer.rb +227 -215
  268. data/lib/rubydb/execution/physical_plan.rb +47 -0
  269. data/lib/rubydb/execution/plan.rb +355 -353
  270. data/lib/rubydb/execution/planner.rb +508 -536
  271. data/lib/rubydb/execution/predicate.rb +235 -235
  272. data/lib/rubydb/execution/scan.rb +49 -49
  273. data/lib/rubydb/execution/sequential_scan.rb +63 -63
  274. data/lib/rubydb/execution/sort_executor.rb +194 -185
  275. data/lib/rubydb/execution/update_executor.rb +160 -162
  276. data/lib/rubydb/functions/aggregate.rb +70 -70
  277. data/lib/rubydb/functions/date_functions.rb +274 -278
  278. data/lib/rubydb/functions/function.rb +85 -85
  279. data/lib/rubydb/functions/json_functions.rb +231 -215
  280. data/lib/rubydb/functions/numeric_functions.rb +346 -346
  281. data/lib/rubydb/functions/scalar.rb +52 -52
  282. data/lib/rubydb/functions/string_functions.rb +383 -383
  283. data/lib/rubydb/functions/system_functions.rb +258 -246
  284. data/lib/rubydb/history/as_of.rb +238 -238
  285. data/lib/rubydb/history/change.rb +105 -105
  286. data/lib/rubydb/history/history.rb +131 -131
  287. data/lib/rubydb/history/history_manager.rb +228 -229
  288. data/lib/rubydb/history/temporal_query.rb +202 -202
  289. data/lib/rubydb/history/timeline.rb +144 -144
  290. data/lib/rubydb/indexes/btree.rb +215 -186
  291. data/lib/rubydb/indexes/btree_cursor.rb +258 -258
  292. data/lib/rubydb/indexes/btree_node.rb +384 -385
  293. data/lib/rubydb/indexes/hash_index.rb +150 -150
  294. data/lib/rubydb/indexes/index.rb +71 -71
  295. data/lib/rubydb/indexes/index_manager.rb +408 -406
  296. data/lib/rubydb/indexes/index_scan.rb +466 -470
  297. data/lib/rubydb/migrations/migration.rb +253 -254
  298. data/lib/rubydb/migrations/migration_lock.rb +146 -146
  299. data/lib/rubydb/migrations/migration_manager.rb +187 -176
  300. data/lib/rubydb/migrations/migration_version.rb +71 -71
  301. data/lib/rubydb/migrations/schema_diff.rb +211 -211
  302. data/lib/rubydb/migrations/schema_version.rb +64 -64
  303. data/lib/rubydb/monitoring/events.rb +155 -160
  304. data/lib/rubydb/monitoring/health.rb +216 -222
  305. data/lib/rubydb/monitoring/logger.rb +188 -193
  306. data/lib/rubydb/monitoring/metrics.rb +363 -359
  307. data/lib/rubydb/monitoring/performance.rb +176 -176
  308. data/lib/rubydb/monitoring/statistics.rb +168 -170
  309. data/lib/rubydb/mvcc/garbage_collector.rb +199 -199
  310. data/lib/rubydb/mvcc/mvcc.rb +16 -16
  311. data/lib/rubydb/mvcc/snapshot.rb +146 -147
  312. data/lib/rubydb/mvcc/vacuum.rb +180 -180
  313. data/lib/rubydb/mvcc/version.rb +106 -106
  314. data/lib/rubydb/mvcc/version_store.rb +396 -398
  315. data/lib/rubydb/mvcc/visibility.rb +107 -109
  316. data/lib/rubydb/protocol/capabilities.rb +125 -125
  317. data/lib/rubydb/protocol/decoder.rb +142 -145
  318. data/lib/rubydb/protocol/encoder.rb +131 -136
  319. data/lib/rubydb/protocol/handshake.rb +306 -305
  320. data/lib/rubydb/protocol/message.rb +121 -121
  321. data/lib/rubydb/protocol/parameter_binder.rb +101 -0
  322. data/lib/rubydb/protocol/protocol.rb +276 -277
  323. data/lib/rubydb/protocol/version.rb +54 -54
  324. data/lib/rubydb/rails/adapter.rb +245 -239
  325. data/lib/rubydb/rails/connection.rb +312 -314
  326. data/lib/rubydb/rails/database_statements.rb +122 -122
  327. data/lib/rubydb/rails/migration.rb +131 -131
  328. data/lib/rubydb/rails/quoting.rb +109 -109
  329. data/lib/rubydb/rails/result.rb +117 -117
  330. data/lib/rubydb/rails/schema_statements.rb +339 -339
  331. data/lib/rubydb/rails/transaction.rb +105 -105
  332. data/lib/rubydb/rails/type.rb +126 -126
  333. data/lib/rubydb/recovery/checkpoint.rb +261 -257
  334. data/lib/rubydb/recovery/consistency.rb +457 -467
  335. data/lib/rubydb/recovery/corruption_detector.rb +5 -5
  336. data/lib/rubydb/recovery/crash_recovery.rb +381 -387
  337. data/lib/rubydb/recovery/recovery_manager.rb +204 -206
  338. data/lib/rubydb/recovery/redo.rb +235 -237
  339. data/lib/rubydb/recovery/undo.rb +204 -206
  340. data/lib/rubydb/replication/failover.rb +5 -5
  341. data/lib/rubydb/replication/fencing.rb +63 -63
  342. data/lib/rubydb/replication/primary.rb +461 -450
  343. data/lib/rubydb/replication/replica.rb +382 -384
  344. data/lib/rubydb/replication/replication_log.rb +194 -200
  345. data/lib/rubydb/replication/replication_manager.rb +307 -308
  346. data/lib/rubydb/replication/replication_slot.rb +293 -295
  347. data/lib/rubydb/replication/replication_stream.rb +198 -201
  348. data/lib/rubydb/rubydb.rb +570 -560
  349. data/lib/rubydb/security/access_control.rb +252 -254
  350. data/lib/rubydb/security/audit_log.rb +209 -213
  351. data/lib/rubydb/security/authentication.rb +302 -302
  352. data/lib/rubydb/security/authorization.rb +282 -282
  353. data/lib/rubydb/security/credentials.rb +192 -196
  354. data/lib/rubydb/security/password.rb +205 -215
  355. data/lib/rubydb/security/permissions.rb +74 -74
  356. data/lib/rubydb/security/role.rb +99 -101
  357. data/lib/rubydb/security/user.rb +86 -86
  358. data/lib/rubydb/server/connection.rb +383 -366
  359. data/lib/rubydb/server/connection_pool.rb +193 -193
  360. data/lib/rubydb/server/lifecycle.rb +227 -228
  361. data/lib/rubydb/server/listener.rb +139 -136
  362. data/lib/rubydb/server/request_handler.rb +277 -276
  363. data/lib/rubydb/server/server.rb +363 -364
  364. data/lib/rubydb/server/session.rb +416 -369
  365. data/lib/rubydb/server/worker.rb +206 -210
  366. data/lib/rubydb/server/worker_pool.rb +168 -168
  367. data/lib/rubydb/sql/ast/alter_table.rb +169 -169
  368. data/lib/rubydb/sql/ast/begin_transaction.rb +47 -47
  369. data/lib/rubydb/sql/ast/commit.rb +37 -37
  370. data/lib/rubydb/sql/ast/constraint.rb +92 -83
  371. data/lib/rubydb/sql/ast/create_database.rb +41 -41
  372. data/lib/rubydb/sql/ast/create_index.rb +61 -61
  373. data/lib/rubydb/sql/ast/create_schema.rb +52 -52
  374. data/lib/rubydb/sql/ast/create_table.rb +187 -187
  375. data/lib/rubydb/sql/ast/delete.rb +54 -54
  376. data/lib/rubydb/sql/ast/drop_database.rb +41 -41
  377. data/lib/rubydb/sql/ast/drop_index.rb +41 -41
  378. data/lib/rubydb/sql/ast/drop_schema.rb +49 -49
  379. data/lib/rubydb/sql/ast/drop_table.rb +49 -49
  380. data/lib/rubydb/sql/ast/explain.rb +64 -64
  381. data/lib/rubydb/sql/ast/expression.rb +617 -604
  382. data/lib/rubydb/sql/ast/insert.rb +66 -66
  383. data/lib/rubydb/sql/ast/node.rb +42 -42
  384. data/lib/rubydb/sql/ast/rollback.rb +63 -63
  385. data/lib/rubydb/sql/ast/savepoint.rb +59 -59
  386. data/lib/rubydb/sql/ast/select.rb +88 -88
  387. data/lib/rubydb/sql/ast/set_operation.rb +22 -20
  388. data/lib/rubydb/sql/ast/trigger.rb +35 -29
  389. data/lib/rubydb/sql/ast/update.rb +88 -88
  390. data/lib/rubydb/sql/ast/vacuum.rb +19 -19
  391. data/lib/rubydb/sql/ast/view.rb +38 -32
  392. data/lib/rubydb/sql/ast/with.rb +32 -32
  393. data/lib/rubydb/sql/grammar.rb +86 -86
  394. data/lib/rubydb/sql/keywords.rb +156 -156
  395. data/lib/rubydb/sql/lexer.rb +209 -214
  396. data/lib/rubydb/sql/operators.rb +100 -100
  397. data/lib/rubydb/sql/parser.rb +1167 -1170
  398. data/lib/rubydb/sql/planner/analyzer.rb +283 -302
  399. data/lib/rubydb/sql/planner/binder.rb +537 -550
  400. data/lib/rubydb/sql/planner/type_checker.rb +427 -431
  401. data/lib/rubydb/sql/token.rb +210 -210
  402. data/lib/rubydb/storage/buffer_frame.rb +44 -44
  403. data/lib/rubydb/storage/buffer_pool.rb +155 -155
  404. data/lib/rubydb/storage/database_lock.rb +74 -74
  405. data/lib/rubydb/storage/deserializer.rb +332 -342
  406. data/lib/rubydb/storage/engine.rb +2409 -2330
  407. data/lib/rubydb/storage/file_manager.rb +191 -187
  408. data/lib/rubydb/storage/free_space_map.rb +79 -81
  409. data/lib/rubydb/storage/page.rb +92 -94
  410. data/lib/rubydb/storage/page_allocator.rb +852 -855
  411. data/lib/rubydb/storage/page_header.rb +63 -67
  412. data/lib/rubydb/storage/page_manager.rb +127 -131
  413. data/lib/rubydb/storage/record.rb +58 -58
  414. data/lib/rubydb/storage/row.rb +78 -78
  415. data/lib/rubydb/storage/serializer.rb +51 -51
  416. data/lib/rubydb/storage/snapshot_reader.rb +167 -0
  417. data/lib/rubydb/storage/storage_layout.rb +151 -151
  418. data/lib/rubydb/storage/storage_manager.rb +114 -114
  419. data/lib/rubydb/storage/tuple.rb +458 -461
  420. data/lib/rubydb/storage/visibility_map.rb +964 -973
  421. data/lib/rubydb/transactions/commit_manager.rb +219 -220
  422. data/lib/rubydb/transactions/isolation.rb +98 -98
  423. data/lib/rubydb/transactions/lock.rb +76 -76
  424. data/lib/rubydb/transactions/lock_manager.rb +359 -362
  425. data/lib/rubydb/transactions/savepoint.rb +142 -143
  426. data/lib/rubydb/transactions/transaction.rb +214 -215
  427. data/lib/rubydb/transactions/transaction_id.rb +84 -84
  428. data/lib/rubydb/transactions/transaction_log.rb +256 -257
  429. data/lib/rubydb/transactions/transaction_manager.rb +434 -435
  430. data/lib/rubydb/types/bigint.rb +36 -36
  431. data/lib/rubydb/types/blob.rb +37 -37
  432. data/lib/rubydb/types/boolean.rb +34 -34
  433. data/lib/rubydb/types/date.rb +39 -39
  434. data/lib/rubydb/types/decimal.rb +48 -48
  435. data/lib/rubydb/types/float.rb +34 -34
  436. data/lib/rubydb/types/integer.rb +36 -36
  437. data/lib/rubydb/types/json.rb +41 -41
  438. data/lib/rubydb/types/null.rb +34 -34
  439. data/lib/rubydb/types/smallint.rb +36 -36
  440. data/lib/rubydb/types/text.rb +37 -37
  441. data/lib/rubydb/types/time.rb +46 -46
  442. data/lib/rubydb/types/timestamp.rb +39 -39
  443. data/lib/rubydb/types/type.rb +119 -119
  444. data/lib/rubydb/types/uuid.rb +47 -47
  445. data/lib/rubydb/types/varchar.rb +37 -37
  446. data/lib/rubydb/version.rb +32 -32
  447. data/lib/rubydb/wal/archive.rb +207 -193
  448. data/lib/rubydb/wal/checkpoint.rb +181 -183
  449. data/lib/rubydb/wal/lsn.rb +94 -94
  450. data/lib/rubydb/wal/reader.rb +259 -260
  451. data/lib/rubydb/wal/record.rb +105 -105
  452. data/lib/rubydb/wal/segment.rb +193 -193
  453. data/lib/rubydb/wal/wal.rb +481 -452
  454. data/lib/rubydb/wal/writer.rb +236 -236
  455. data/lib/rubydb.rb +7 -7
  456. data/packaging/docker/docker-compose.failover.yml +43 -43
  457. data/packaging/homebrew/rubydb.rb +19 -19
  458. data/rubydb.gemspec +70 -57
  459. data/scripts/benchmark +7 -7
  460. data/scripts/build_accelerator +49 -0
  461. data/scripts/durability_drill +37 -37
  462. data/scripts/fuzz +63 -63
  463. data/scripts/release +77 -42
  464. data/scripts/release_check +43 -43
  465. data/scripts/replication_failover_drill +268 -250
  466. data/scripts/replication_network_failover_drill +287 -255
  467. data/scripts/restore_drill +45 -45
  468. data/scripts/security +45 -0
  469. metadata +102 -1
@@ -1,45 +1,45 @@
1
- # RubyDB production runbook
2
-
3
- ## Before deployment
4
-
5
- 1. Pin the RubyDB commit and Ruby version. Run `bundle exec rspec`, the
6
- production soak, the durability drill, and the network failover drill.
7
- 2. Place the database, WAL, lock file, replication state, and backups on
8
- protected storage owned by the service account. Keep backups on a separate
9
- failure domain.
10
- 3. Enable TLS, use a private bind address, configure authentication and a
11
- replication token from a secret manager, and set connection, frame, query,
12
- worker, and disk-space limits.
13
- 4. Install the alert rules in
14
- `config/monitoring/prometheus-alerts.yml`. Test readiness, metrics, backup,
15
- and restore from the deployment environment.
16
-
17
- ## Backup and restore
18
-
19
- Run a full backup before upgrades and retain the matching WAL chain and
20
- manifest. Restore into a new inactive directory, run the restore drill, reopen
21
- the database, compare row counts/checksums, and only then switch traffic.
22
- Never overwrite the only copy during a restore.
23
-
24
- ## Failover
25
-
26
- Stop writes, confirm the replica is synchronized, fence the old primary using
27
- the shared durable fence path, promote exactly one replica, and verify client
28
- writes and replication status. Automatic election is not enabled until the
29
- deployment provides an independently tested fencing lease. A network partition
30
- must never result in two writable primaries.
31
-
32
- ## TLS and secret rotation
33
-
34
- Stage a new certificate/key and replication token in the secret manager,
35
- validate them on an inactive instance, restart or drain one peer at a time,
36
- verify TLS and replication authentication, then revoke the old credentials
37
- after the overlap window. Do not place secrets in repository files, command
38
- arguments, URLs, or logs.
39
-
40
- ## Upgrade and incident response
41
-
42
- Run the upgrade-compatibility suite against a copy of production backups.
43
- Preserve the database, WAL, replication state, logs, metrics, and timestamps
44
- before repair, vacuum, restore, or promotion. If checksums, fencing, or
45
- replication diverge, stop writes and escalate; do not guess at a repair.
1
+ # RubyDB production runbook
2
+
3
+ ## Before deployment
4
+
5
+ 1. Pin the RubyDB commit and Ruby version. Run `bundle exec rspec`, the
6
+ production soak, the durability drill, and the network failover drill.
7
+ 2. Place the database, WAL, lock file, replication state, and backups on
8
+ protected storage owned by the service account. Keep backups on a separate
9
+ failure domain.
10
+ 3. Enable TLS, use a private bind address, configure authentication and a
11
+ replication token from a secret manager, and set connection, frame, query,
12
+ worker, and disk-space limits.
13
+ 4. Install the alert rules in
14
+ `config/monitoring/prometheus-alerts.yml`. Test readiness, metrics, backup,
15
+ and restore from the deployment environment.
16
+
17
+ ## Backup and restore
18
+
19
+ Run a full backup before upgrades and retain the matching WAL chain and
20
+ manifest. Restore into a new inactive directory, run the restore drill, reopen
21
+ the database, compare row counts/checksums, and only then switch traffic.
22
+ Never overwrite the only copy during a restore.
23
+
24
+ ## Failover
25
+
26
+ Stop writes, confirm the replica is synchronized, fence the old primary using
27
+ the shared durable fence path, promote exactly one replica, and verify client
28
+ writes and replication status. Automatic election is not enabled until the
29
+ deployment provides an independently tested fencing lease. A network partition
30
+ must never result in two writable primaries.
31
+
32
+ ## TLS and secret rotation
33
+
34
+ Stage a new certificate/key and replication token in the secret manager,
35
+ validate them on an inactive instance, restart or drain one peer at a time,
36
+ verify TLS and replication authentication, then revoke the old credentials
37
+ after the overlap window. Do not place secrets in repository files, command
38
+ arguments, URLs, or logs.
39
+
40
+ ## Upgrade and incident response
41
+
42
+ Run the upgrade-compatibility suite against a copy of production backups.
43
+ Preserve the database, WAL, replication state, logs, metrics, and timestamps
44
+ before repair, vacuum, restore, or promotion. If checksums, fencing, or
45
+ replication diverge, stop writes and escalate; do not guess at a repair.
@@ -1,33 +1,33 @@
1
- # Replication operations
2
-
3
- RubyDB replication is logical WAL row-mutation replication. Configure the
4
- primary and replica with separate data directories, explicit network addresses,
5
- and the same high-entropy `replication_auth_token`. Restrict the listener with
6
- firewall rules and use TLS or a trusted private network; the token is peer
7
- authentication, not a replacement for transport encryption.
8
-
9
- ## Bootstrap and catch-up
10
-
11
- Start the primary first, then start the replica with its replica identifier and
12
- primary address. The primary sends catalog schema before row replay. Verify the
13
- replica reaches `SYNCED`, and compare received and replayed LSNs before routing
14
- reads to it.
15
-
16
- Monitor replication status continuously. A replica with a received/replayed
17
- gap is not eligible for promotion. Preserve the replication log and replica
18
- state files during an incident; do not manually edit LSN state.
19
-
20
- ## Token rotation
21
-
22
- Drain replication, stop or disconnect the replica, update the protected token
23
- on both sides, restart the primary listener if required, and reconnect the
24
- replica. Confirm accepted handshakes and zero unexpected authentication
25
- failures before returning the replica to service. Never place tokens in source
26
- control or command histories.
27
-
28
- ## Failure handling
29
-
30
- On a network partition, assume the old primary may still be reachable until it
31
- is fenced. Do not promote a lagging replica and do not run two writable primaries
32
- without a verified fencing lease. Re-bootstrap a divergent replica rather than
33
- guessing which WAL records are safe to apply.
1
+ # Replication operations
2
+
3
+ RubyDB replication is logical WAL row-mutation replication. Configure the
4
+ primary and replica with separate data directories, explicit network addresses,
5
+ and the same high-entropy `replication_auth_token`. Restrict the listener with
6
+ firewall rules and use TLS or a trusted private network; the token is peer
7
+ authentication, not a replacement for transport encryption.
8
+
9
+ ## Bootstrap and catch-up
10
+
11
+ Start the primary first, then start the replica with its replica identifier and
12
+ primary address. The primary sends catalog schema before row replay. Verify the
13
+ replica reaches `SYNCED`, and compare received and replayed LSNs before routing
14
+ reads to it.
15
+
16
+ Monitor replication status continuously. A replica with a received/replayed
17
+ gap is not eligible for promotion. Preserve the replication log and replica
18
+ state files during an incident; do not manually edit LSN state.
19
+
20
+ ## Token rotation
21
+
22
+ Drain replication, stop or disconnect the replica, update the protected token
23
+ on both sides, restart the primary listener if required, and reconnect the
24
+ replica. Confirm accepted handshakes and zero unexpected authentication
25
+ failures before returning the replica to service. Never place tokens in source
26
+ control or command histories.
27
+
28
+ ## Failure handling
29
+
30
+ On a network partition, assume the old primary may still be reachable until it
31
+ is fenced. Do not promote a lagging replica and do not run two writable primaries
32
+ without a verified fencing lease. Re-bootstrap a divergent replica rather than
33
+ guessing which WAL records are safe to apply.
@@ -1,6 +1,6 @@
1
- ## Restore verification
2
-
3
- Before treating a backup as recoverable, run the verification path that extracts
4
- it into an isolated temporary directory, opens the database with RubyDB, and
5
- compares the restored catalog and table row counts with the manifest. The
6
- verification removes its temporary database after the check.
1
+ ## Restore verification
2
+
3
+ Before treating a backup as recoverable, run the verification path that extracts
4
+ it into an isolated temporary directory, opens the database with RubyDB, and
5
+ compares the restored catalog and table row counts with the manifest. The
6
+ verification removes its temporary database after the check.
@@ -1,34 +1,34 @@
1
- # RubyDB operator runbook
2
-
3
- ## Start and health check
4
-
5
- Run the server with an explicit production configuration:
6
-
7
- ```sh
8
- RUBYDB_ENV=production rubydb -c config/production.yml start
9
- ```
10
-
11
- Confirm the process is listening, then run `rubydb status --json`. Treat a missing PID file, an invalid configuration, or a failed health check as a failed deployment; do not bypass startup validation.
12
-
13
- ## Backup and restore
14
-
15
- Create a verified full backup before upgrades or maintenance. Retain the manifest and WAL chain together. Test restoration into a separate directory before replacing a live data directory. Never restore over an active database.
16
-
17
- ## Upgrade and rollback
18
-
19
- Stop writes, create and verify a full backup, restore it into a staging directory, and run migration status plus application smoke queries. Promote the staged directory only after validation. Roll back by stopping RubyDB and switching back to the retained pre-upgrade directory; preserve the failed directory for diagnosis.
20
-
21
- ## Incident boundaries
22
-
23
- On corruption, checksum failure, repeated crash recovery, or replication divergence, stop writes and preserve the data directory and logs. Do not delete WAL, run vacuum, or force promotion until a verified backup and incident copy exist. Automatic failover is intentionally disabled and promotion requires an operator decision.
24
-
25
- ## Security operations
26
-
27
- Keep production credentials and TLS private keys outside the repository with restrictive permissions. Rotate certificates by staging the replacement, validating the chain, and restarting during a planned window. Keep TLS enabled and use peer verification when a trusted CA is available.
28
-
29
- For replication, configure the same high-entropy `replication_auth_token` on
30
- each primary and replica. A peer with a missing or incorrect token is rejected;
31
- rotate the token by draining replicas, updating the protected configuration on
32
- both sides, and reconnecting them. The replication listener is not a substitute
33
- for a private network or TLS termination, so restrict its port at the firewall
34
- and validate certificate/private-network controls during deployment.
1
+ # RubyDB operator runbook
2
+
3
+ ## Start and health check
4
+
5
+ Run the server with an explicit production configuration:
6
+
7
+ ```sh
8
+ RUBYDB_ENV=production rubydb -c config/production.yml start
9
+ ```
10
+
11
+ Confirm the process is listening, then run `rubydb status --json`. Treat a missing PID file, an invalid configuration, or a failed health check as a failed deployment; do not bypass startup validation.
12
+
13
+ ## Backup and restore
14
+
15
+ Create a verified full backup before upgrades or maintenance. Retain the manifest and WAL chain together. Test restoration into a separate directory before replacing a live data directory. Never restore over an active database.
16
+
17
+ ## Upgrade and rollback
18
+
19
+ Stop writes, create and verify a full backup, restore it into a staging directory, and run migration status plus application smoke queries. Promote the staged directory only after validation. Roll back by stopping RubyDB and switching back to the retained pre-upgrade directory; preserve the failed directory for diagnosis.
20
+
21
+ ## Incident boundaries
22
+
23
+ On corruption, checksum failure, repeated crash recovery, or replication divergence, stop writes and preserve the data directory and logs. Do not delete WAL, run vacuum, or force promotion until a verified backup and incident copy exist. Automatic failover is intentionally disabled and promotion requires an operator decision.
24
+
25
+ ## Security operations
26
+
27
+ Keep production credentials and TLS private keys outside the repository with restrictive permissions. Rotate certificates by staging the replacement, validating the chain, and restarting during a planned window. Keep TLS enabled and use peer verification when a trusted CA is available.
28
+
29
+ For replication, configure the same high-entropy `replication_auth_token` on
30
+ each primary and replica. A peer with a missing or incorrect token is rejected;
31
+ rotate the token by draining replicas, updating the protected configuration on
32
+ both sides, and reconnecting them. The replication listener is not a substitute
33
+ for a private network or TLS termination, so restrict its port at the firewall
34
+ and validate certificate/private-network controls during deployment.
@@ -1,14 +1,14 @@
1
- # Upgrades
2
-
3
- RubyDB upgrades must be treated as storage migrations, not as an in-place binary replacement.
4
-
5
- Before upgrading:
6
-
7
- 1. Stop writes and confirm the server is healthy.
8
- 2. Create and verify a full backup, including WAL when available.
9
- 3. Record the RubyDB version, storage page size, and backup manifest checksum.
10
- 4. Test the new release against a restored copy of the backup.
11
-
12
- The current storage format is version 1. RubyDB refuses to open an unknown page format or a database whose page size differs from the configured page size. There is no automatic on-disk format conversion yet; a future incompatible format must provide an explicit export/import migration.
13
-
14
- After upgrading, reopen the restored or migrated database, run the migration status command, verify application queries, and retain the pre-upgrade backup until the rollback window closes. Never delete the original data directory as part of an upgrade.
1
+ # Upgrades
2
+
3
+ RubyDB upgrades must be treated as storage migrations, not as an in-place binary replacement.
4
+
5
+ Before upgrading:
6
+
7
+ 1. Stop writes and confirm the server is healthy.
8
+ 2. Create and verify a full backup, including WAL when available.
9
+ 3. Record the RubyDB version, storage page size, and backup manifest checksum.
10
+ 4. Test the new release against a restored copy of the backup.
11
+
12
+ The current storage format is version 1. RubyDB refuses to open an unknown page format or a database whose page size differs from the configured page size. There is no automatic on-disk format conversion yet; a future incompatible format must provide an explicit export/import migration.
13
+
14
+ After upgrading, reopen the restored or migrated database, run the migration status command, verify application queries, and retain the pre-upgrade backup until the rollback window closes. Never delete the original data directory as part of an upgrade.
@@ -1,17 +1,17 @@
1
- # Workload testing
2
-
3
- Run the deterministic concurrent workload from the repository root:
4
-
5
- ```sh
6
- RUBYDB_WORKLOAD_THREADS=4 RUBYDB_WORKLOAD_OPERATIONS=250 ruby benchmarks/concurrent_workload.rb
7
- ```
8
-
9
- It performs concurrent inserts against one engine, verifies the final row count, closes the database, reopens it, and verifies durable reads. It prints one JSON result and exits non-zero on any operation or durability failure.
10
-
11
- Use a temporary database by default. To retain an artifact for inspection, provide an explicit path:
12
-
13
- ```sh
14
- RUBYDB_WORKLOAD_PATH=/var/lib/rubydb/workload.rdb RUBYDB_WORKLOAD_THREADS=16 RUBYDB_WORKLOAD_OPERATIONS=10000 ruby benchmarks/concurrent_workload.rb
15
- ```
16
-
17
- Run this in a staging environment on the target filesystem before deployment. Establish acceptable latency, throughput, WAL growth, disk-space, recovery-time, and backup/restore targets for the expected workload; do not treat a successful smoke workload as capacity certification.
1
+ # Workload testing
2
+
3
+ Run the deterministic concurrent workload from the repository root:
4
+
5
+ ```sh
6
+ RUBYDB_WORKLOAD_THREADS=4 RUBYDB_WORKLOAD_OPERATIONS=250 ruby benchmarks/concurrent_workload.rb
7
+ ```
8
+
9
+ It performs concurrent inserts against one engine, verifies the final row count, closes the database, reopens it, and verifies durable reads. It prints one JSON result and exits non-zero on any operation or durability failure.
10
+
11
+ Use a temporary database by default. To retain an artifact for inspection, provide an explicit path:
12
+
13
+ ```sh
14
+ RUBYDB_WORKLOAD_PATH=/var/lib/rubydb/workload.rdb RUBYDB_WORKLOAD_THREADS=16 RUBYDB_WORKLOAD_OPERATIONS=10000 ruby benchmarks/concurrent_workload.rb
15
+ ```
16
+
17
+ Run this in a staging environment on the target filesystem before deployment. Establish acceptable latency, throughput, WAL growth, disk-space, recovery-time, and backup/restore targets for the expected workload; do not treat a successful smoke workload as capacity certification.
@@ -1,118 +1,118 @@
1
- # Production-readiness audit
2
-
3
- ## Status
4
-
5
- RubyDB has verified production-oriented foundations, but it is not yet a general-purpose production database. Supported behavior is backed by the RSpec suite in `spec/`.
6
-
7
- Latest local audit: 270 examples, 0 failures. Hosted CI and deployment-specific
8
- evidence remain separate release gates.
9
-
10
- The feature inventory below was written during an earlier checkpoint and its
11
- parenthetical historical count is not the current total. Use the latest audit
12
- line above and rerun the suite for release evidence.
13
-
14
- ## Implemented features
15
-
16
- The repository contains substantial scaffolding for:
17
-
18
- - page-based storage
19
- - buffer pool caching
20
- - file and page management
21
- - WAL, recovery, and checkpoint modules
22
- - transactions and MVCC abstractions
23
- - SQL parser/planner infrastructure
24
- - server, protocol, and connection components
25
- - authentication and authorization modules
26
- - backup and replication APIs
27
- - monitoring and metrics interfaces
28
- - Rails integration adapters
29
- - ordered, locked migrations with durable version/checksum tracking
30
-
31
- ## Tested features
32
-
33
- The current suite verifies storage reopen, subprocess crash recovery, MVCC isolation/vacuum, durable visibility version-history traversal, constraints including ON DELETE/ON UPDATE referential actions and nullable values, indexes including deep B-tree splits, SQL execution including idempotent database/table/index DDL and SQL foreign-key actions, schema DDL, views, trigger DDL and dispatch, CREATE TABLE, ALTER TABLE column and constraint changes, transaction before-image rollback and SQL savepoints, and VACUUM, live TCP sessions, TLS transport, password/SCRAM authentication and authorization, metrics updates including Prometheus export, liveness/readiness health reporting through the server request router, live CLI doctor checks with safe repair behavior, truthful CLI status reporting, live CLI branch diff/merge/checkout operations, live CLI inspection values, live CLI snapshot creation/listing and vacuum reporting, live CLI backup creation and restore dry-run validation, live CLI database creation and deletion, the documented SQL compatibility contract, full/snapshot/incremental/differential backup validation, logical replication, durable replica state, guarded failover promotion, durable fencing epochs, replication peer-token authentication, recovery resource checks, migration/schema-diff behavior including executable migration SQL serialization and changed-migration detection, atomic branch checkout, branch state/diff/merge behavior, foreign-key integrity lookup behavior, compound/null/boolean check-constraint evaluation, lock conflict/wait/timeout behavior, complete standalone and transaction-manager deadlock cycle detection with victim rollback, concurrency mutex initialization, Rails schema-builder SQL generation including defaults and foreign-key conventions, engine block-based table creation, adapter schema dumps preserving primary keys and defaults, ActiveRecord hash-form ordering and populated-table migration round trips, bounded multi-process workload supervision with child reaping, combined production traffic/deadline/cancellation/capacity/deadlock soak validation, release configuration, release tag/changelog preflight, upgrade guards, benchmark execution, executor query deadlines, wire-level cancellation of in-flight requests, standalone SCRAM verification, actual WAL checkpoint sizing, typed deserialization-corruption detection, preservation of stored false values when defaults are present, filesystem fault injection with descriptor cleanup, compaction/reopen validation, process-level replication failover with stale-writer fencing, and live TCP network-partition catch-up and promotion (268 examples, 0 failures at the latest audit).
34
-
35
- ## Known limitations
36
-
37
- - The concurrent workload and regression suite verify parallel appends, point
38
- reads, scans, durable reopen reads, multi-process server clients, and a
39
- process-level replication failover drill. This does not certify universal
40
- capacity; embedded ownership remains exclusive and multi-process clients must
41
- use the server.
42
-
43
- - multi-row SQL `VALUES` inserts execute through the parser, binder, planner,
44
- and executor, including per-row conflict handling and Rails insert IDs
45
- - incremental backups capture WAL mutations after a verified base LSN; differential backups capture the verified base-relative WAL delta and restore through the same validated delta path
46
- - replication is limited to the explicit logical row-mutation envelope API;
47
- process replacement, live TCP partition/catch-up, and same-host fencing are
48
- tested, while true multi-host partitions and split-brain recovery remain
49
- deployment work
50
- - automatic failover is limited to synchronized candidates; fencing requires a shared durable fence path and still needs multi-host split-brain validation
51
- - the SQLite compatibility profile covers the documented common application
52
- surface, but it is not SQLite file-format or extension compatibility
53
- - production CA lifecycle, independent security review, performance targets,
54
- and large-scale workload behavior still require dedicated validation
55
- - branch state application and live target-branch merges are supported through the engine reconciliation hook
56
-
57
- ## Unsupported SQL
58
-
59
- The supported dialect is defined in [docs/sql/compatibility.md](sql/compatibility.md) and enforced by parser/execution integration coverage. RubyDB provides a tested common SQLite-style profile, not complete SQLite file-format, extension, pragma, or error compatibility.
60
-
61
- ## Durability guarantees
62
-
63
- WAL framing, reopen behavior, subprocess crash-recovery scenarios, actual checkpoint sizing, fail-closed visibility-map loading, filesystem fault injection, compaction/reopen behavior, networked replication failover, and verified restore drills are tested. This is not a substitute for real quota, power-loss, and filesystem-specific validation.
64
-
65
- ## Transaction guarantees
66
-
67
- Transaction commit/rollback, MVCC visibility, snapshot isolation behavior, and vacuum safe-point handling are covered by focused tests. High-contention workload validation remains outstanding.
68
-
69
- ## Isolation guarantees
70
-
71
- Read committed, repeatable read, and serializable conflict behavior are covered by focused tests; broader workload and distributed-isolation validation remains outstanding.
72
-
73
- ## Backup guarantees
74
-
75
- Full compressed backups restore into a fresh directory and checksum tampering is detected. Incremental and differential backups validate the base, checksum the change set, and apply WAL row mutations during restore. Production-scale backup-chain and filesystem fault-injection testing remains outstanding.
76
-
77
- ## Replication guarantees
78
-
79
- Logical row-mutation streaming, LSN deduplication, acknowledgments, acknowledged-LSN reporting, synchronized-candidate promotion verification, process replacement, live TCP partition/catch-up, and stale-primary rejection through durable fencing epochs are tested. Multi-host fencing and split-brain recovery still require dedicated validation.
80
-
81
- ## TLS guarantees
82
-
83
- TLS 1.2 or newer is enforced when enabled. Certificate/key parsing, CA-file validation, TLS client connections, and the end-to-end SSL transport path are tested. Mutual TLS is configurable through client certificates; certificate rotation and production CA lifecycle procedures remain operational work.
84
-
85
- ## Security model
86
-
87
- Password and SCRAM-SHA-256 authentication, server-signature verification, read/write authorization, and startup validation of incomplete auth configuration are tested through the server/session path.
88
-
89
- ## Performance characteristics
90
-
91
- The repository includes a deterministic storage benchmark (`RUBYDB_BENCHMARK_ITERATIONS=100 ruby -Ilib benchmarks/basic_workload.rb`) and a concurrent write/durability workload (`ruby benchmarks/concurrent_workload.rb`). Both emit machine-readable JSON. Production-scale throughput, WAL, checkpoint, and concurrency targets still require workload-specific baselines.
92
-
93
- ## Deployment requirements
94
-
95
- The project currently needs:
96
-
97
- - a validated Ruby support matrix
98
- - RubyGems publication credentials (`RUBYGEMS_API_KEY`) configured as a protected
99
- GitHub Actions secret before pushing a release tag
100
- - a documented operator runbook for backups, upgrades, rollback, and incident preservation
101
- - explicit durability and crash-recovery validation
102
- - secure-by-default configuration development
103
- - a narrower set of supported features documented honestly
104
-
105
- ## Remaining roadmap items
106
-
107
- Main remaining work is to complete the sequence laid out in the repository design:
108
-
109
- 1. production CA lifecycle and certificate rotation
110
- 2. broader SQL compatibility contract and workload benchmarks
111
- 3. production-grade failover fencing and broader SQL compatibility
112
- 4. operator runbooks and deployment validation
113
-
114
- ## Bottom line
115
-
116
- RubyDB has a validated production-oriented foundation, but it does not yet meet
117
- the bar for unrestricted production use with real data outside its documented
118
- feature set and deployment-specific validation.
1
+ # Production-readiness audit
2
+
3
+ ## Status
4
+
5
+ RubyDB has verified production-oriented foundations, but it is not yet a general-purpose production database. Supported behavior is backed by the RSpec suite in `spec/`.
6
+
7
+ Latest local audit: 270 examples, 0 failures. Hosted CI and deployment-specific
8
+ evidence remain separate release gates.
9
+
10
+ The feature inventory below was written during an earlier checkpoint and its
11
+ parenthetical historical count is not the current total. Use the latest audit
12
+ line above and rerun the suite for release evidence.
13
+
14
+ ## Implemented features
15
+
16
+ The repository contains substantial scaffolding for:
17
+
18
+ - page-based storage
19
+ - buffer pool caching
20
+ - file and page management
21
+ - WAL, recovery, and checkpoint modules
22
+ - transactions and MVCC abstractions
23
+ - SQL parser/planner infrastructure
24
+ - server, protocol, and connection components
25
+ - authentication and authorization modules
26
+ - backup and replication APIs
27
+ - monitoring and metrics interfaces
28
+ - Rails integration adapters
29
+ - ordered, locked migrations with durable version/checksum tracking
30
+
31
+ ## Tested features
32
+
33
+ The current suite verifies storage reopen, subprocess crash recovery, MVCC isolation/vacuum, durable visibility version-history traversal, constraints including ON DELETE/ON UPDATE referential actions and nullable values, indexes including deep B-tree splits, SQL execution including idempotent database/table/index DDL and SQL foreign-key actions, schema DDL, views, trigger DDL and dispatch, CREATE TABLE, ALTER TABLE column and constraint changes, transaction before-image rollback and SQL savepoints, and VACUUM, live TCP sessions, TLS transport, password/SCRAM authentication and authorization, metrics updates including Prometheus export, liveness/readiness health reporting through the server request router, live CLI doctor checks with safe repair behavior, truthful CLI status reporting, live CLI branch diff/merge/checkout operations, live CLI inspection values, live CLI snapshot creation/listing and vacuum reporting, live CLI backup creation and restore dry-run validation, live CLI database creation and deletion, the documented SQL compatibility contract, full/snapshot/incremental/differential backup validation, logical replication, durable replica state, guarded failover promotion, durable fencing epochs, replication peer-token authentication, recovery resource checks, migration/schema-diff behavior including executable migration SQL serialization and changed-migration detection, atomic branch checkout, branch state/diff/merge behavior, foreign-key integrity lookup behavior, compound/null/boolean check-constraint evaluation, lock conflict/wait/timeout behavior, complete standalone and transaction-manager deadlock cycle detection with victim rollback, concurrency mutex initialization, Rails schema-builder SQL generation including defaults and foreign-key conventions, engine block-based table creation, adapter schema dumps preserving primary keys and defaults, ActiveRecord hash-form ordering and populated-table migration round trips, bounded multi-process workload supervision with child reaping, combined production traffic/deadline/cancellation/capacity/deadlock soak validation, release configuration, release tag/changelog preflight, upgrade guards, benchmark execution, executor query deadlines, wire-level cancellation of in-flight requests, standalone SCRAM verification, actual WAL checkpoint sizing, typed deserialization-corruption detection, preservation of stored false values when defaults are present, filesystem fault injection with descriptor cleanup, compaction/reopen validation, process-level replication failover with stale-writer fencing, and live TCP network-partition catch-up and promotion (268 examples, 0 failures at the latest audit).
34
+
35
+ ## Known limitations
36
+
37
+ - The concurrent workload and regression suite verify parallel appends, point
38
+ reads, scans, durable reopen reads, multi-process server clients, and a
39
+ process-level replication failover drill. This does not certify universal
40
+ capacity; embedded ownership remains exclusive and multi-process clients must
41
+ use the server.
42
+
43
+ - multi-row SQL `VALUES` inserts execute through the parser, binder, planner,
44
+ and executor, including per-row conflict handling and Rails insert IDs
45
+ - incremental backups capture WAL mutations after a verified base LSN; differential backups capture the verified base-relative WAL delta and restore through the same validated delta path
46
+ - replication is limited to the explicit logical row-mutation envelope API;
47
+ process replacement, live TCP partition/catch-up, and same-host fencing are
48
+ tested, while true multi-host partitions and split-brain recovery remain
49
+ deployment work
50
+ - automatic failover is limited to synchronized candidates; fencing requires a shared durable fence path and still needs multi-host split-brain validation
51
+ - the SQLite compatibility profile covers the documented common application
52
+ surface, but it is not SQLite file-format or extension compatibility
53
+ - production CA lifecycle, independent security review, performance targets,
54
+ and large-scale workload behavior still require dedicated validation
55
+ - branch state application and live target-branch merges are supported through the engine reconciliation hook
56
+
57
+ ## Unsupported SQL
58
+
59
+ The supported dialect is defined in [docs/sql/compatibility.md](sql/compatibility.md) and enforced by parser/execution integration coverage. RubyDB provides a tested common SQLite-style profile, not complete SQLite file-format, extension, pragma, or error compatibility.
60
+
61
+ ## Durability guarantees
62
+
63
+ WAL framing, reopen behavior, subprocess crash-recovery scenarios, actual checkpoint sizing, fail-closed visibility-map loading, filesystem fault injection, compaction/reopen behavior, networked replication failover, and verified restore drills are tested. This is not a substitute for real quota, power-loss, and filesystem-specific validation.
64
+
65
+ ## Transaction guarantees
66
+
67
+ Transaction commit/rollback, MVCC visibility, snapshot isolation behavior, and vacuum safe-point handling are covered by focused tests. High-contention workload validation remains outstanding.
68
+
69
+ ## Isolation guarantees
70
+
71
+ Read committed, repeatable read, and serializable conflict behavior are covered by focused tests; broader workload and distributed-isolation validation remains outstanding.
72
+
73
+ ## Backup guarantees
74
+
75
+ Full compressed backups restore into a fresh directory and checksum tampering is detected. Incremental and differential backups validate the base, checksum the change set, and apply WAL row mutations during restore. Production-scale backup-chain and filesystem fault-injection testing remains outstanding.
76
+
77
+ ## Replication guarantees
78
+
79
+ Logical row-mutation streaming, LSN deduplication, acknowledgments, acknowledged-LSN reporting, synchronized-candidate promotion verification, process replacement, live TCP partition/catch-up, and stale-primary rejection through durable fencing epochs are tested. Multi-host fencing and split-brain recovery still require dedicated validation.
80
+
81
+ ## TLS guarantees
82
+
83
+ TLS 1.2 or newer is enforced when enabled. Certificate/key parsing, CA-file validation, TLS client connections, and the end-to-end SSL transport path are tested. Mutual TLS is configurable through client certificates; certificate rotation and production CA lifecycle procedures remain operational work.
84
+
85
+ ## Security model
86
+
87
+ Password and SCRAM-SHA-256 authentication, server-signature verification, read/write authorization, and startup validation of incomplete auth configuration are tested through the server/session path.
88
+
89
+ ## Performance characteristics
90
+
91
+ The repository includes a deterministic storage benchmark (`RUBYDB_BENCHMARK_ITERATIONS=100 ruby -Ilib benchmarks/basic_workload.rb`) and a concurrent write/durability workload (`ruby benchmarks/concurrent_workload.rb`). Both emit machine-readable JSON. Production-scale throughput, WAL, checkpoint, and concurrency targets still require workload-specific baselines.
92
+
93
+ ## Deployment requirements
94
+
95
+ The project currently needs:
96
+
97
+ - a validated Ruby support matrix
98
+ - RubyGems publication credentials (`RUBYGEMS_API_KEY`) configured as a protected
99
+ GitHub Actions secret before pushing a release tag
100
+ - a documented operator runbook for backups, upgrades, rollback, and incident preservation
101
+ - explicit durability and crash-recovery validation
102
+ - secure-by-default configuration development
103
+ - a narrower set of supported features documented honestly
104
+
105
+ ## Remaining roadmap items
106
+
107
+ Main remaining work is to complete the sequence laid out in the repository design:
108
+
109
+ 1. production CA lifecycle and certificate rotation
110
+ 2. broader SQL compatibility contract and workload benchmarks
111
+ 3. production-grade failover fencing and broader SQL compatibility
112
+ 4. operator runbooks and deployment validation
113
+
114
+ ## Bottom line
115
+
116
+ RubyDB has a validated production-oriented foundation, but it does not yet meet
117
+ the bar for unrestricted production use with real data outside its documented
118
+ feature set and deployment-specific validation.