rubydb 0.1.3 → 0.1.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (420) hide show
  1. checksums.yaml +4 -4
  2. data/.github/PULL_REQUEST_TEMPLATE.md +15 -15
  3. data/.github/workflows/benchmark.yml +26 -26
  4. data/.github/workflows/compatibility.yml +63 -63
  5. data/.github/workflows/fuzz.yml +33 -33
  6. data/.github/workflows/lint.yml +21 -21
  7. data/.github/workflows/operations.yml +24 -24
  8. data/.github/workflows/production-validation.yml +111 -111
  9. data/.github/workflows/release.yml +77 -77
  10. data/.github/workflows/security.yml +39 -37
  11. data/.github/workflows/test.yml +26 -26
  12. data/.github/workflows/workload.yml +58 -58
  13. data/.gitignore +12 -5
  14. data/.rubocop.yml +50 -44
  15. data/.standard.yml +9 -14
  16. data/ARCHITECTURE.md +21 -21
  17. data/CHANGELOG.md +49 -27
  18. data/CODE_OF_CONDUCT.md +13 -13
  19. data/CONTRIBUTING.md +29 -29
  20. data/GOVERNANCE.md +16 -16
  21. data/Gemfile +18 -17
  22. data/Gemfile.lock +125 -71
  23. data/README.md +249 -162
  24. data/ROADMAP.md +27 -27
  25. data/Rakefile +71 -71
  26. data/SECURITY.md +54 -54
  27. data/SUPPORT.md +14 -14
  28. data/adapters/activerecord/Gemfile +11 -11
  29. data/adapters/activerecord/lib/active_record/connection_adapters/rubydb_adapter.rb +877 -880
  30. data/adapters/activerecord/rubydb-activerecord.gemspec +21 -21
  31. data/adapters/activerecord/spec/rubydb_adapter_integration_spec.rb +155 -143
  32. data/adapters/ruby/README.md +18 -18
  33. data/adapters/sequel/README.md +11 -11
  34. data/config/monitoring/prometheus-alerts.yml +39 -39
  35. data/config/production.yml +36 -36
  36. data/docs/README.md +75 -71
  37. data/docs/architecture/concurrency.md +14 -14
  38. data/docs/architecture/current-state.md +125 -125
  39. data/docs/architecture/execution-engine.md +25 -25
  40. data/docs/architecture/indexes.md +19 -19
  41. data/docs/architecture/mvcc.md +19 -19
  42. data/docs/architecture/overview.md +13 -13
  43. data/docs/architecture/pages.md +11 -11
  44. data/docs/architecture/production-roadmap.md +82 -82
  45. data/docs/architecture/query-planner.md +20 -20
  46. data/docs/architecture/recovery.md +18 -18
  47. data/docs/architecture/sql-engine.md +12 -12
  48. data/docs/architecture/storage-engine.md +14 -14
  49. data/docs/architecture/transactions.md +10 -10
  50. data/docs/architecture/wal.md +28 -28
  51. data/docs/cli-cheatsheet.md +98 -98
  52. data/docs/cli.md +275 -275
  53. data/docs/contributing/architecture.md +9 -9
  54. data/docs/contributing/benchmarking.md +14 -14
  55. data/docs/contributing/development.md +16 -16
  56. data/docs/contributing/release-process.md +49 -49
  57. data/docs/contributing/testing.md +16 -16
  58. data/docs/debugging.md +229 -229
  59. data/docs/developer/branching.md +10 -10
  60. data/docs/developer/database-diff.md +10 -10
  61. data/docs/developer/local-development.md +17 -17
  62. data/docs/developer/snapshots.md +9 -9
  63. data/docs/developer/temporal-data.md +10 -10
  64. data/docs/developer-guide.md +297 -297
  65. data/docs/getting-started/first-database.md +16 -16
  66. data/docs/getting-started/first-query.md +13 -13
  67. data/docs/getting-started/installation.md +19 -19
  68. data/docs/getting-started/local-to-production.md +300 -300
  69. data/docs/getting-started/quickstart.md +17 -17
  70. data/docs/getting-started/rails.md +16 -16
  71. data/docs/hardening_backlog.md +93 -93
  72. data/docs/lessons-learned.md +112 -112
  73. data/docs/operations/backups.md +33 -33
  74. data/docs/operations/disaster-recovery.md +31 -31
  75. data/docs/operations/failover.md +30 -30
  76. data/docs/operations/monitoring.md +25 -25
  77. data/docs/operations/production-guide.md +295 -295
  78. data/docs/operations/production-runbook.md +45 -45
  79. data/docs/operations/replication.md +33 -33
  80. data/docs/operations/restore.md +6 -6
  81. data/docs/operations/runbook.md +34 -34
  82. data/docs/operations/upgrades.md +14 -14
  83. data/docs/operations/workload-testing.md +17 -17
  84. data/docs/production-readiness.md +118 -118
  85. data/docs/production_validation.md +150 -150
  86. data/docs/rails/active-record.md +11 -11
  87. data/docs/rails/compatibility-guide.md +90 -90
  88. data/docs/rails/database-yml.md +92 -92
  89. data/docs/rails/installation.md +17 -17
  90. data/docs/rails/migrations.md +17 -17
  91. data/docs/rails/production.md +82 -82
  92. data/docs/rails/troubleshooting.md +18 -18
  93. data/docs/release.md +25 -25
  94. data/docs/server/architecture.md +10 -10
  95. data/docs/server/authentication.md +10 -10
  96. data/docs/server/configuration.md +16 -16
  97. data/docs/server/connection-pooling.md +10 -10
  98. data/docs/server/deployment.md +10 -10
  99. data/docs/server/protocol.md +12 -12
  100. data/docs/sql/compatibility-guide.md +82 -82
  101. data/docs/sql/compatibility.md +39 -39
  102. data/docs/sql/data-types.md +10 -10
  103. data/docs/sql/functions.md +9 -9
  104. data/docs/sql/joins.md +9 -9
  105. data/docs/sql/operators.md +9 -9
  106. data/docs/sql/sqlite-compatibility.md +21 -21
  107. data/docs/sql/syntax.md +10 -10
  108. data/docs/sql/transactions.md +10 -10
  109. data/docs/troubleshooting.md +244 -244
  110. data/lessons/01-foundations.md +73 -0
  111. data/lessons/02-local-development.md +121 -0
  112. data/lessons/03-embedded-rubydb.md +99 -0
  113. data/lessons/04-rails-complex-apps.md +138 -0
  114. data/lessons/05-rubydb-production-server.md +237 -0
  115. data/lessons/06-postgresql-massive-apps.md +96 -0
  116. data/lessons/07-hybrid-microservices.md +94 -0
  117. data/lessons/08-migrations-backups-recovery.md +86 -0
  118. data/lessons/09-observability-security-scale.md +87 -0
  119. data/lessons/10-release-readiness.md +125 -0
  120. data/lib/rubydb/backup/archive.rb +332 -334
  121. data/lib/rubydb/backup/backup.rb +400 -401
  122. data/lib/rubydb/backup/incremental.rb +349 -353
  123. data/lib/rubydb/backup/restore.rb +289 -290
  124. data/lib/rubydb/backup/snapshot.rb +265 -267
  125. data/lib/rubydb/backup/verification.rb +276 -279
  126. data/lib/rubydb/branching/branch.rb +181 -181
  127. data/lib/rubydb/branching/branch_manager.rb +307 -311
  128. data/lib/rubydb/branching/branch_metadata.rb +140 -140
  129. data/lib/rubydb/branching/checkout.rb +165 -166
  130. data/lib/rubydb/branching/copy_on_write.rb +272 -272
  131. data/lib/rubydb/branching/diff.rb +137 -138
  132. data/lib/rubydb/branching/merge.rb +282 -285
  133. data/lib/rubydb/build_info.rb +15 -15
  134. data/lib/rubydb/catalog/catalog.rb +391 -391
  135. data/lib/rubydb/catalog/column.rb +112 -112
  136. data/lib/rubydb/catalog/constraint.rb +180 -180
  137. data/lib/rubydb/catalog/database.rb +184 -184
  138. data/lib/rubydb/catalog/index.rb +97 -97
  139. data/lib/rubydb/catalog/schema.rb +103 -103
  140. data/lib/rubydb/catalog/sequence.rb +90 -90
  141. data/lib/rubydb/catalog/system_catalog.rb +698 -698
  142. data/lib/rubydb/catalog/table.rb +178 -178
  143. data/lib/rubydb/catalog/trigger.rb +102 -102
  144. data/lib/rubydb/catalog/view.rb +66 -66
  145. data/lib/rubydb/cli/application.rb +163 -163
  146. data/lib/rubydb/cli/commands/backup.rb +80 -81
  147. data/lib/rubydb/cli/commands/branch.rb +72 -72
  148. data/lib/rubydb/cli/commands/checkout.rb +54 -54
  149. data/lib/rubydb/cli/commands/create.rb +58 -58
  150. data/lib/rubydb/cli/commands/diff.rb +76 -77
  151. data/lib/rubydb/cli/commands/doctor.rb +74 -74
  152. data/lib/rubydb/cli/commands/drop.rb +57 -57
  153. data/lib/rubydb/cli/commands/init.rb +101 -102
  154. data/lib/rubydb/cli/commands/inspect.rb +95 -95
  155. data/lib/rubydb/cli/commands/merge.rb +63 -63
  156. data/lib/rubydb/cli/commands/migrate.rb +62 -62
  157. data/lib/rubydb/cli/commands/restart.rb +42 -39
  158. data/lib/rubydb/cli/commands/restore.rb +121 -121
  159. data/lib/rubydb/cli/commands/shell.rb +365 -365
  160. data/lib/rubydb/cli/commands/snapshot.rb +79 -79
  161. data/lib/rubydb/cli/commands/start.rb +88 -82
  162. data/lib/rubydb/cli/commands/status.rb +96 -92
  163. data/lib/rubydb/cli/commands/stop.rb +47 -47
  164. data/lib/rubydb/cli/commands/vacuum.rb +58 -58
  165. data/lib/rubydb/cli/formatter.rb +221 -221
  166. data/lib/rubydb/cli/output.rb +168 -168
  167. data/lib/rubydb/client/client.rb +302 -304
  168. data/lib/rubydb/client/connection.rb +414 -415
  169. data/lib/rubydb/client/connection_pool.rb +168 -168
  170. data/lib/rubydb/client/connection_url.rb +96 -96
  171. data/lib/rubydb/client/prepared_statement.rb +60 -60
  172. data/lib/rubydb/client/result.rb +123 -123
  173. data/lib/rubydb/client/statement.rb +52 -52
  174. data/lib/rubydb/client/transaction.rb +130 -130
  175. data/lib/rubydb/concurrency/concurrency.rb +19 -19
  176. data/lib/rubydb/concurrency/deadlock_detector.rb +148 -150
  177. data/lib/rubydb/concurrency/latch.rb +101 -101
  178. data/lib/rubydb/concurrency/lock_graph.rb +163 -165
  179. data/lib/rubydb/concurrency/mutex.rb +181 -183
  180. data/lib/rubydb/concurrency/rw_lock.rb +180 -180
  181. data/lib/rubydb/concurrency/scheduler.rb +248 -250
  182. data/lib/rubydb/concurrency/worker_pool.rb +145 -143
  183. data/lib/rubydb/configuration/config.rb +170 -170
  184. data/lib/rubydb/configuration/defaults.rb +179 -179
  185. data/lib/rubydb/configuration/environment.rb +152 -152
  186. data/lib/rubydb/configuration/parser.rb +185 -185
  187. data/lib/rubydb/configuration/validation.rb +221 -221
  188. data/lib/rubydb/constants.rb +74 -74
  189. data/lib/rubydb/constraints/check.rb +181 -181
  190. data/lib/rubydb/constraints/constraint.rb +101 -101
  191. data/lib/rubydb/constraints/foreign_key.rb +130 -130
  192. data/lib/rubydb/constraints/not_null.rb +64 -64
  193. data/lib/rubydb/constraints/primary_key.rb +99 -99
  194. data/lib/rubydb/constraints/unique.rb +106 -108
  195. data/lib/rubydb/constraints/validator.rb +349 -350
  196. data/lib/rubydb/errors/authentication_error.rb +10 -10
  197. data/lib/rubydb/errors/authorization_error.rb +23 -23
  198. data/lib/rubydb/errors/client_error.rb +10 -10
  199. data/lib/rubydb/errors/configuration_error.rb +10 -10
  200. data/lib/rubydb/errors/connection_error.rb +10 -10
  201. data/lib/rubydb/errors/constraint_error.rb +23 -23
  202. data/lib/rubydb/errors/corruption_error.rb +10 -10
  203. data/lib/rubydb/errors/database_error.rb +10 -10
  204. data/lib/rubydb/errors/error.rb +20 -20
  205. data/lib/rubydb/errors/execution_error.rb +10 -10
  206. data/lib/rubydb/errors/parser_error.rb +10 -10
  207. data/lib/rubydb/errors/recovery_error.rb +10 -10
  208. data/lib/rubydb/errors/replication_error.rb +10 -10
  209. data/lib/rubydb/errors/server_error.rb +6 -6
  210. data/lib/rubydb/errors/storage_error.rb +10 -10
  211. data/lib/rubydb/errors/transaction_error.rb +10 -10
  212. data/lib/rubydb/execution/aggregate_executor.rb +134 -138
  213. data/lib/rubydb/execution/delete_executor.rb +110 -112
  214. data/lib/rubydb/execution/distinct_executor.rb +131 -135
  215. data/lib/rubydb/execution/executor.rb +1195 -1192
  216. data/lib/rubydb/execution/expression.rb +191 -193
  217. data/lib/rubydb/execution/index_scan.rb +142 -142
  218. data/lib/rubydb/execution/insert_executor.rb +215 -217
  219. data/lib/rubydb/execution/join_executor.rb +243 -249
  220. data/lib/rubydb/execution/limit_executor.rb +83 -85
  221. data/lib/rubydb/execution/optimizer.rb +227 -215
  222. data/lib/rubydb/execution/plan.rb +355 -353
  223. data/lib/rubydb/execution/planner.rb +542 -536
  224. data/lib/rubydb/execution/predicate.rb +235 -235
  225. data/lib/rubydb/execution/scan.rb +49 -49
  226. data/lib/rubydb/execution/sequential_scan.rb +63 -63
  227. data/lib/rubydb/execution/sort_executor.rb +180 -185
  228. data/lib/rubydb/execution/update_executor.rb +160 -162
  229. data/lib/rubydb/functions/aggregate.rb +70 -70
  230. data/lib/rubydb/functions/date_functions.rb +274 -278
  231. data/lib/rubydb/functions/function.rb +85 -85
  232. data/lib/rubydb/functions/json_functions.rb +231 -215
  233. data/lib/rubydb/functions/numeric_functions.rb +346 -346
  234. data/lib/rubydb/functions/scalar.rb +52 -52
  235. data/lib/rubydb/functions/string_functions.rb +383 -383
  236. data/lib/rubydb/functions/system_functions.rb +258 -246
  237. data/lib/rubydb/history/as_of.rb +238 -238
  238. data/lib/rubydb/history/change.rb +105 -105
  239. data/lib/rubydb/history/history.rb +131 -131
  240. data/lib/rubydb/history/history_manager.rb +228 -229
  241. data/lib/rubydb/history/temporal_query.rb +202 -202
  242. data/lib/rubydb/history/timeline.rb +144 -144
  243. data/lib/rubydb/indexes/btree.rb +186 -186
  244. data/lib/rubydb/indexes/btree_cursor.rb +258 -258
  245. data/lib/rubydb/indexes/btree_node.rb +384 -385
  246. data/lib/rubydb/indexes/hash_index.rb +150 -150
  247. data/lib/rubydb/indexes/index.rb +71 -71
  248. data/lib/rubydb/indexes/index_manager.rb +408 -406
  249. data/lib/rubydb/indexes/index_scan.rb +466 -470
  250. data/lib/rubydb/migrations/migration.rb +253 -254
  251. data/lib/rubydb/migrations/migration_lock.rb +146 -146
  252. data/lib/rubydb/migrations/migration_manager.rb +187 -176
  253. data/lib/rubydb/migrations/migration_version.rb +71 -71
  254. data/lib/rubydb/migrations/schema_diff.rb +211 -211
  255. data/lib/rubydb/migrations/schema_version.rb +64 -64
  256. data/lib/rubydb/monitoring/events.rb +155 -160
  257. data/lib/rubydb/monitoring/health.rb +216 -222
  258. data/lib/rubydb/monitoring/logger.rb +188 -193
  259. data/lib/rubydb/monitoring/metrics.rb +363 -359
  260. data/lib/rubydb/monitoring/performance.rb +176 -176
  261. data/lib/rubydb/monitoring/statistics.rb +168 -170
  262. data/lib/rubydb/mvcc/garbage_collector.rb +199 -199
  263. data/lib/rubydb/mvcc/mvcc.rb +16 -16
  264. data/lib/rubydb/mvcc/snapshot.rb +146 -147
  265. data/lib/rubydb/mvcc/vacuum.rb +180 -180
  266. data/lib/rubydb/mvcc/version.rb +106 -106
  267. data/lib/rubydb/mvcc/version_store.rb +396 -398
  268. data/lib/rubydb/mvcc/visibility.rb +107 -109
  269. data/lib/rubydb/protocol/capabilities.rb +125 -125
  270. data/lib/rubydb/protocol/decoder.rb +142 -145
  271. data/lib/rubydb/protocol/encoder.rb +131 -136
  272. data/lib/rubydb/protocol/handshake.rb +306 -305
  273. data/lib/rubydb/protocol/message.rb +121 -121
  274. data/lib/rubydb/protocol/parameter_binder.rb +101 -0
  275. data/lib/rubydb/protocol/protocol.rb +276 -277
  276. data/lib/rubydb/protocol/version.rb +54 -54
  277. data/lib/rubydb/rails/adapter.rb +245 -239
  278. data/lib/rubydb/rails/connection.rb +312 -314
  279. data/lib/rubydb/rails/database_statements.rb +122 -122
  280. data/lib/rubydb/rails/migration.rb +131 -131
  281. data/lib/rubydb/rails/quoting.rb +109 -109
  282. data/lib/rubydb/rails/result.rb +123 -120
  283. data/lib/rubydb/rails/schema_statements.rb +339 -339
  284. data/lib/rubydb/rails/transaction.rb +105 -105
  285. data/lib/rubydb/rails/type.rb +126 -126
  286. data/lib/rubydb/recovery/checkpoint.rb +261 -257
  287. data/lib/rubydb/recovery/consistency.rb +457 -467
  288. data/lib/rubydb/recovery/corruption_detector.rb +5 -5
  289. data/lib/rubydb/recovery/crash_recovery.rb +381 -387
  290. data/lib/rubydb/recovery/recovery_manager.rb +204 -206
  291. data/lib/rubydb/recovery/redo.rb +235 -237
  292. data/lib/rubydb/recovery/undo.rb +204 -206
  293. data/lib/rubydb/replication/failover.rb +5 -5
  294. data/lib/rubydb/replication/fencing.rb +63 -63
  295. data/lib/rubydb/replication/primary.rb +461 -450
  296. data/lib/rubydb/replication/replica.rb +382 -384
  297. data/lib/rubydb/replication/replication_log.rb +194 -200
  298. data/lib/rubydb/replication/replication_manager.rb +307 -308
  299. data/lib/rubydb/replication/replication_slot.rb +293 -295
  300. data/lib/rubydb/replication/replication_stream.rb +198 -201
  301. data/lib/rubydb/rubydb.rb +564 -560
  302. data/lib/rubydb/security/access_control.rb +252 -254
  303. data/lib/rubydb/security/audit_log.rb +209 -213
  304. data/lib/rubydb/security/authentication.rb +302 -302
  305. data/lib/rubydb/security/authorization.rb +282 -282
  306. data/lib/rubydb/security/credentials.rb +192 -196
  307. data/lib/rubydb/security/password.rb +205 -215
  308. data/lib/rubydb/security/permissions.rb +74 -74
  309. data/lib/rubydb/security/role.rb +99 -101
  310. data/lib/rubydb/security/user.rb +86 -86
  311. data/lib/rubydb/server/connection.rb +383 -366
  312. data/lib/rubydb/server/connection_pool.rb +193 -193
  313. data/lib/rubydb/server/lifecycle.rb +227 -228
  314. data/lib/rubydb/server/listener.rb +139 -136
  315. data/lib/rubydb/server/request_handler.rb +277 -276
  316. data/lib/rubydb/server/server.rb +363 -364
  317. data/lib/rubydb/server/session.rb +371 -369
  318. data/lib/rubydb/server/worker.rb +206 -210
  319. data/lib/rubydb/server/worker_pool.rb +168 -168
  320. data/lib/rubydb/sql/ast/alter_table.rb +169 -169
  321. data/lib/rubydb/sql/ast/begin_transaction.rb +47 -47
  322. data/lib/rubydb/sql/ast/commit.rb +37 -37
  323. data/lib/rubydb/sql/ast/constraint.rb +92 -83
  324. data/lib/rubydb/sql/ast/create_database.rb +41 -41
  325. data/lib/rubydb/sql/ast/create_index.rb +61 -61
  326. data/lib/rubydb/sql/ast/create_schema.rb +52 -52
  327. data/lib/rubydb/sql/ast/create_table.rb +187 -187
  328. data/lib/rubydb/sql/ast/delete.rb +54 -54
  329. data/lib/rubydb/sql/ast/drop_database.rb +41 -41
  330. data/lib/rubydb/sql/ast/drop_index.rb +41 -41
  331. data/lib/rubydb/sql/ast/drop_schema.rb +49 -49
  332. data/lib/rubydb/sql/ast/drop_table.rb +49 -49
  333. data/lib/rubydb/sql/ast/explain.rb +64 -64
  334. data/lib/rubydb/sql/ast/expression.rb +617 -604
  335. data/lib/rubydb/sql/ast/insert.rb +66 -66
  336. data/lib/rubydb/sql/ast/node.rb +42 -42
  337. data/lib/rubydb/sql/ast/rollback.rb +63 -63
  338. data/lib/rubydb/sql/ast/savepoint.rb +59 -59
  339. data/lib/rubydb/sql/ast/select.rb +88 -88
  340. data/lib/rubydb/sql/ast/set_operation.rb +22 -20
  341. data/lib/rubydb/sql/ast/trigger.rb +35 -29
  342. data/lib/rubydb/sql/ast/update.rb +88 -88
  343. data/lib/rubydb/sql/ast/vacuum.rb +19 -19
  344. data/lib/rubydb/sql/ast/view.rb +38 -32
  345. data/lib/rubydb/sql/ast/with.rb +32 -32
  346. data/lib/rubydb/sql/grammar.rb +86 -86
  347. data/lib/rubydb/sql/keywords.rb +156 -156
  348. data/lib/rubydb/sql/lexer.rb +209 -214
  349. data/lib/rubydb/sql/operators.rb +100 -100
  350. data/lib/rubydb/sql/parser.rb +1167 -1170
  351. data/lib/rubydb/sql/planner/analyzer.rb +283 -302
  352. data/lib/rubydb/sql/planner/binder.rb +537 -550
  353. data/lib/rubydb/sql/planner/type_checker.rb +427 -431
  354. data/lib/rubydb/sql/token.rb +210 -210
  355. data/lib/rubydb/storage/buffer_frame.rb +44 -44
  356. data/lib/rubydb/storage/buffer_pool.rb +155 -155
  357. data/lib/rubydb/storage/database_lock.rb +74 -74
  358. data/lib/rubydb/storage/deserializer.rb +332 -342
  359. data/lib/rubydb/storage/engine.rb +2347 -2330
  360. data/lib/rubydb/storage/file_manager.rb +191 -187
  361. data/lib/rubydb/storage/free_space_map.rb +79 -81
  362. data/lib/rubydb/storage/page.rb +92 -94
  363. data/lib/rubydb/storage/page_allocator.rb +852 -855
  364. data/lib/rubydb/storage/page_header.rb +63 -67
  365. data/lib/rubydb/storage/page_manager.rb +127 -131
  366. data/lib/rubydb/storage/record.rb +58 -58
  367. data/lib/rubydb/storage/row.rb +78 -78
  368. data/lib/rubydb/storage/serializer.rb +51 -51
  369. data/lib/rubydb/storage/storage_layout.rb +151 -151
  370. data/lib/rubydb/storage/storage_manager.rb +114 -114
  371. data/lib/rubydb/storage/tuple.rb +458 -461
  372. data/lib/rubydb/storage/visibility_map.rb +964 -973
  373. data/lib/rubydb/transactions/commit_manager.rb +219 -220
  374. data/lib/rubydb/transactions/isolation.rb +98 -98
  375. data/lib/rubydb/transactions/lock.rb +76 -76
  376. data/lib/rubydb/transactions/lock_manager.rb +359 -362
  377. data/lib/rubydb/transactions/savepoint.rb +142 -143
  378. data/lib/rubydb/transactions/transaction.rb +214 -215
  379. data/lib/rubydb/transactions/transaction_id.rb +84 -84
  380. data/lib/rubydb/transactions/transaction_log.rb +256 -257
  381. data/lib/rubydb/transactions/transaction_manager.rb +434 -435
  382. data/lib/rubydb/types/bigint.rb +36 -36
  383. data/lib/rubydb/types/blob.rb +37 -37
  384. data/lib/rubydb/types/boolean.rb +34 -34
  385. data/lib/rubydb/types/date.rb +39 -39
  386. data/lib/rubydb/types/decimal.rb +48 -48
  387. data/lib/rubydb/types/float.rb +34 -34
  388. data/lib/rubydb/types/integer.rb +36 -36
  389. data/lib/rubydb/types/json.rb +41 -41
  390. data/lib/rubydb/types/null.rb +34 -34
  391. data/lib/rubydb/types/smallint.rb +36 -36
  392. data/lib/rubydb/types/text.rb +37 -37
  393. data/lib/rubydb/types/time.rb +46 -46
  394. data/lib/rubydb/types/timestamp.rb +39 -39
  395. data/lib/rubydb/types/type.rb +119 -119
  396. data/lib/rubydb/types/uuid.rb +47 -47
  397. data/lib/rubydb/types/varchar.rb +37 -37
  398. data/lib/rubydb/version.rb +32 -32
  399. data/lib/rubydb/wal/archive.rb +190 -193
  400. data/lib/rubydb/wal/checkpoint.rb +181 -183
  401. data/lib/rubydb/wal/lsn.rb +94 -94
  402. data/lib/rubydb/wal/reader.rb +259 -260
  403. data/lib/rubydb/wal/record.rb +105 -105
  404. data/lib/rubydb/wal/segment.rb +193 -193
  405. data/lib/rubydb/wal/wal.rb +480 -452
  406. data/lib/rubydb/wal/writer.rb +236 -236
  407. data/lib/rubydb.rb +7 -7
  408. data/packaging/docker/docker-compose.failover.yml +43 -43
  409. data/packaging/homebrew/rubydb.rb +19 -19
  410. data/rubydb.gemspec +60 -57
  411. data/scripts/benchmark +7 -7
  412. data/scripts/durability_drill +37 -37
  413. data/scripts/fuzz +63 -63
  414. data/scripts/release +45 -40
  415. data/scripts/release_check +43 -43
  416. data/scripts/replication_failover_drill +268 -250
  417. data/scripts/replication_network_failover_drill +287 -255
  418. data/scripts/restore_drill +45 -45
  419. data/scripts/security +45 -0
  420. metadata +54 -1
@@ -1,45 +1,45 @@
1
- # RubyDB production runbook
2
-
3
- ## Before deployment
4
-
5
- 1. Pin the RubyDB commit and Ruby version. Run `bundle exec rspec`, the
6
- production soak, the durability drill, and the network failover drill.
7
- 2. Place the database, WAL, lock file, replication state, and backups on
8
- protected storage owned by the service account. Keep backups on a separate
9
- failure domain.
10
- 3. Enable TLS, use a private bind address, configure authentication and a
11
- replication token from a secret manager, and set connection, frame, query,
12
- worker, and disk-space limits.
13
- 4. Install the alert rules in
14
- `config/monitoring/prometheus-alerts.yml`. Test readiness, metrics, backup,
15
- and restore from the deployment environment.
16
-
17
- ## Backup and restore
18
-
19
- Run a full backup before upgrades and retain the matching WAL chain and
20
- manifest. Restore into a new inactive directory, run the restore drill, reopen
21
- the database, compare row counts/checksums, and only then switch traffic.
22
- Never overwrite the only copy during a restore.
23
-
24
- ## Failover
25
-
26
- Stop writes, confirm the replica is synchronized, fence the old primary using
27
- the shared durable fence path, promote exactly one replica, and verify client
28
- writes and replication status. Automatic election is not enabled until the
29
- deployment provides an independently tested fencing lease. A network partition
30
- must never result in two writable primaries.
31
-
32
- ## TLS and secret rotation
33
-
34
- Stage a new certificate/key and replication token in the secret manager,
35
- validate them on an inactive instance, restart or drain one peer at a time,
36
- verify TLS and replication authentication, then revoke the old credentials
37
- after the overlap window. Do not place secrets in repository files, command
38
- arguments, URLs, or logs.
39
-
40
- ## Upgrade and incident response
41
-
42
- Run the upgrade-compatibility suite against a copy of production backups.
43
- Preserve the database, WAL, replication state, logs, metrics, and timestamps
44
- before repair, vacuum, restore, or promotion. If checksums, fencing, or
45
- replication diverge, stop writes and escalate; do not guess at a repair.
1
+ # RubyDB production runbook
2
+
3
+ ## Before deployment
4
+
5
+ 1. Pin the RubyDB commit and Ruby version. Run `bundle exec rspec`, the
6
+ production soak, the durability drill, and the network failover drill.
7
+ 2. Place the database, WAL, lock file, replication state, and backups on
8
+ protected storage owned by the service account. Keep backups on a separate
9
+ failure domain.
10
+ 3. Enable TLS, use a private bind address, configure authentication and a
11
+ replication token from a secret manager, and set connection, frame, query,
12
+ worker, and disk-space limits.
13
+ 4. Install the alert rules in
14
+ `config/monitoring/prometheus-alerts.yml`. Test readiness, metrics, backup,
15
+ and restore from the deployment environment.
16
+
17
+ ## Backup and restore
18
+
19
+ Run a full backup before upgrades and retain the matching WAL chain and
20
+ manifest. Restore into a new inactive directory, run the restore drill, reopen
21
+ the database, compare row counts/checksums, and only then switch traffic.
22
+ Never overwrite the only copy during a restore.
23
+
24
+ ## Failover
25
+
26
+ Stop writes, confirm the replica is synchronized, fence the old primary using
27
+ the shared durable fence path, promote exactly one replica, and verify client
28
+ writes and replication status. Automatic election is not enabled until the
29
+ deployment provides an independently tested fencing lease. A network partition
30
+ must never result in two writable primaries.
31
+
32
+ ## TLS and secret rotation
33
+
34
+ Stage a new certificate/key and replication token in the secret manager,
35
+ validate them on an inactive instance, restart or drain one peer at a time,
36
+ verify TLS and replication authentication, then revoke the old credentials
37
+ after the overlap window. Do not place secrets in repository files, command
38
+ arguments, URLs, or logs.
39
+
40
+ ## Upgrade and incident response
41
+
42
+ Run the upgrade-compatibility suite against a copy of production backups.
43
+ Preserve the database, WAL, replication state, logs, metrics, and timestamps
44
+ before repair, vacuum, restore, or promotion. If checksums, fencing, or
45
+ replication diverge, stop writes and escalate; do not guess at a repair.
@@ -1,33 +1,33 @@
1
- # Replication operations
2
-
3
- RubyDB replication is logical WAL row-mutation replication. Configure the
4
- primary and replica with separate data directories, explicit network addresses,
5
- and the same high-entropy `replication_auth_token`. Restrict the listener with
6
- firewall rules and use TLS or a trusted private network; the token is peer
7
- authentication, not a replacement for transport encryption.
8
-
9
- ## Bootstrap and catch-up
10
-
11
- Start the primary first, then start the replica with its replica identifier and
12
- primary address. The primary sends catalog schema before row replay. Verify the
13
- replica reaches `SYNCED`, and compare received and replayed LSNs before routing
14
- reads to it.
15
-
16
- Monitor replication status continuously. A replica with a received/replayed
17
- gap is not eligible for promotion. Preserve the replication log and replica
18
- state files during an incident; do not manually edit LSN state.
19
-
20
- ## Token rotation
21
-
22
- Drain replication, stop or disconnect the replica, update the protected token
23
- on both sides, restart the primary listener if required, and reconnect the
24
- replica. Confirm accepted handshakes and zero unexpected authentication
25
- failures before returning the replica to service. Never place tokens in source
26
- control or command histories.
27
-
28
- ## Failure handling
29
-
30
- On a network partition, assume the old primary may still be reachable until it
31
- is fenced. Do not promote a lagging replica and do not run two writable primaries
32
- without a verified fencing lease. Re-bootstrap a divergent replica rather than
33
- guessing which WAL records are safe to apply.
1
+ # Replication operations
2
+
3
+ RubyDB replication is logical WAL row-mutation replication. Configure the
4
+ primary and replica with separate data directories, explicit network addresses,
5
+ and the same high-entropy `replication_auth_token`. Restrict the listener with
6
+ firewall rules and use TLS or a trusted private network; the token is peer
7
+ authentication, not a replacement for transport encryption.
8
+
9
+ ## Bootstrap and catch-up
10
+
11
+ Start the primary first, then start the replica with its replica identifier and
12
+ primary address. The primary sends catalog schema before row replay. Verify the
13
+ replica reaches `SYNCED`, and compare received and replayed LSNs before routing
14
+ reads to it.
15
+
16
+ Monitor replication status continuously. A replica with a received/replayed
17
+ gap is not eligible for promotion. Preserve the replication log and replica
18
+ state files during an incident; do not manually edit LSN state.
19
+
20
+ ## Token rotation
21
+
22
+ Drain replication, stop or disconnect the replica, update the protected token
23
+ on both sides, restart the primary listener if required, and reconnect the
24
+ replica. Confirm accepted handshakes and zero unexpected authentication
25
+ failures before returning the replica to service. Never place tokens in source
26
+ control or command histories.
27
+
28
+ ## Failure handling
29
+
30
+ On a network partition, assume the old primary may still be reachable until it
31
+ is fenced. Do not promote a lagging replica and do not run two writable primaries
32
+ without a verified fencing lease. Re-bootstrap a divergent replica rather than
33
+ guessing which WAL records are safe to apply.
@@ -1,6 +1,6 @@
1
- ## Restore verification
2
-
3
- Before treating a backup as recoverable, run the verification path that extracts
4
- it into an isolated temporary directory, opens the database with RubyDB, and
5
- compares the restored catalog and table row counts with the manifest. The
6
- verification removes its temporary database after the check.
1
+ ## Restore verification
2
+
3
+ Before treating a backup as recoverable, run the verification path that extracts
4
+ it into an isolated temporary directory, opens the database with RubyDB, and
5
+ compares the restored catalog and table row counts with the manifest. The
6
+ verification removes its temporary database after the check.
@@ -1,34 +1,34 @@
1
- # RubyDB operator runbook
2
-
3
- ## Start and health check
4
-
5
- Run the server with an explicit production configuration:
6
-
7
- ```sh
8
- RUBYDB_ENV=production rubydb -c config/production.yml start
9
- ```
10
-
11
- Confirm the process is listening, then run `rubydb status --json`. Treat a missing PID file, an invalid configuration, or a failed health check as a failed deployment; do not bypass startup validation.
12
-
13
- ## Backup and restore
14
-
15
- Create a verified full backup before upgrades or maintenance. Retain the manifest and WAL chain together. Test restoration into a separate directory before replacing a live data directory. Never restore over an active database.
16
-
17
- ## Upgrade and rollback
18
-
19
- Stop writes, create and verify a full backup, restore it into a staging directory, and run migration status plus application smoke queries. Promote the staged directory only after validation. Roll back by stopping RubyDB and switching back to the retained pre-upgrade directory; preserve the failed directory for diagnosis.
20
-
21
- ## Incident boundaries
22
-
23
- On corruption, checksum failure, repeated crash recovery, or replication divergence, stop writes and preserve the data directory and logs. Do not delete WAL, run vacuum, or force promotion until a verified backup and incident copy exist. Automatic failover is intentionally disabled and promotion requires an operator decision.
24
-
25
- ## Security operations
26
-
27
- Keep production credentials and TLS private keys outside the repository with restrictive permissions. Rotate certificates by staging the replacement, validating the chain, and restarting during a planned window. Keep TLS enabled and use peer verification when a trusted CA is available.
28
-
29
- For replication, configure the same high-entropy `replication_auth_token` on
30
- each primary and replica. A peer with a missing or incorrect token is rejected;
31
- rotate the token by draining replicas, updating the protected configuration on
32
- both sides, and reconnecting them. The replication listener is not a substitute
33
- for a private network or TLS termination, so restrict its port at the firewall
34
- and validate certificate/private-network controls during deployment.
1
+ # RubyDB operator runbook
2
+
3
+ ## Start and health check
4
+
5
+ Run the server with an explicit production configuration:
6
+
7
+ ```sh
8
+ RUBYDB_ENV=production rubydb -c config/production.yml start
9
+ ```
10
+
11
+ Confirm the process is listening, then run `rubydb status --json`. Treat a missing PID file, an invalid configuration, or a failed health check as a failed deployment; do not bypass startup validation.
12
+
13
+ ## Backup and restore
14
+
15
+ Create a verified full backup before upgrades or maintenance. Retain the manifest and WAL chain together. Test restoration into a separate directory before replacing a live data directory. Never restore over an active database.
16
+
17
+ ## Upgrade and rollback
18
+
19
+ Stop writes, create and verify a full backup, restore it into a staging directory, and run migration status plus application smoke queries. Promote the staged directory only after validation. Roll back by stopping RubyDB and switching back to the retained pre-upgrade directory; preserve the failed directory for diagnosis.
20
+
21
+ ## Incident boundaries
22
+
23
+ On corruption, checksum failure, repeated crash recovery, or replication divergence, stop writes and preserve the data directory and logs. Do not delete WAL, run vacuum, or force promotion until a verified backup and incident copy exist. Automatic failover is intentionally disabled and promotion requires an operator decision.
24
+
25
+ ## Security operations
26
+
27
+ Keep production credentials and TLS private keys outside the repository with restrictive permissions. Rotate certificates by staging the replacement, validating the chain, and restarting during a planned window. Keep TLS enabled and use peer verification when a trusted CA is available.
28
+
29
+ For replication, configure the same high-entropy `replication_auth_token` on
30
+ each primary and replica. A peer with a missing or incorrect token is rejected;
31
+ rotate the token by draining replicas, updating the protected configuration on
32
+ both sides, and reconnecting them. The replication listener is not a substitute
33
+ for a private network or TLS termination, so restrict its port at the firewall
34
+ and validate certificate/private-network controls during deployment.
@@ -1,14 +1,14 @@
1
- # Upgrades
2
-
3
- RubyDB upgrades must be treated as storage migrations, not as an in-place binary replacement.
4
-
5
- Before upgrading:
6
-
7
- 1. Stop writes and confirm the server is healthy.
8
- 2. Create and verify a full backup, including WAL when available.
9
- 3. Record the RubyDB version, storage page size, and backup manifest checksum.
10
- 4. Test the new release against a restored copy of the backup.
11
-
12
- The current storage format is version 1. RubyDB refuses to open an unknown page format or a database whose page size differs from the configured page size. There is no automatic on-disk format conversion yet; a future incompatible format must provide an explicit export/import migration.
13
-
14
- After upgrading, reopen the restored or migrated database, run the migration status command, verify application queries, and retain the pre-upgrade backup until the rollback window closes. Never delete the original data directory as part of an upgrade.
1
+ # Upgrades
2
+
3
+ RubyDB upgrades must be treated as storage migrations, not as an in-place binary replacement.
4
+
5
+ Before upgrading:
6
+
7
+ 1. Stop writes and confirm the server is healthy.
8
+ 2. Create and verify a full backup, including WAL when available.
9
+ 3. Record the RubyDB version, storage page size, and backup manifest checksum.
10
+ 4. Test the new release against a restored copy of the backup.
11
+
12
+ The current storage format is version 1. RubyDB refuses to open an unknown page format or a database whose page size differs from the configured page size. There is no automatic on-disk format conversion yet; a future incompatible format must provide an explicit export/import migration.
13
+
14
+ After upgrading, reopen the restored or migrated database, run the migration status command, verify application queries, and retain the pre-upgrade backup until the rollback window closes. Never delete the original data directory as part of an upgrade.
@@ -1,17 +1,17 @@
1
- # Workload testing
2
-
3
- Run the deterministic concurrent workload from the repository root:
4
-
5
- ```sh
6
- RUBYDB_WORKLOAD_THREADS=4 RUBYDB_WORKLOAD_OPERATIONS=250 ruby benchmarks/concurrent_workload.rb
7
- ```
8
-
9
- It performs concurrent inserts against one engine, verifies the final row count, closes the database, reopens it, and verifies durable reads. It prints one JSON result and exits non-zero on any operation or durability failure.
10
-
11
- Use a temporary database by default. To retain an artifact for inspection, provide an explicit path:
12
-
13
- ```sh
14
- RUBYDB_WORKLOAD_PATH=/var/lib/rubydb/workload.rdb RUBYDB_WORKLOAD_THREADS=16 RUBYDB_WORKLOAD_OPERATIONS=10000 ruby benchmarks/concurrent_workload.rb
15
- ```
16
-
17
- Run this in a staging environment on the target filesystem before deployment. Establish acceptable latency, throughput, WAL growth, disk-space, recovery-time, and backup/restore targets for the expected workload; do not treat a successful smoke workload as capacity certification.
1
+ # Workload testing
2
+
3
+ Run the deterministic concurrent workload from the repository root:
4
+
5
+ ```sh
6
+ RUBYDB_WORKLOAD_THREADS=4 RUBYDB_WORKLOAD_OPERATIONS=250 ruby benchmarks/concurrent_workload.rb
7
+ ```
8
+
9
+ It performs concurrent inserts against one engine, verifies the final row count, closes the database, reopens it, and verifies durable reads. It prints one JSON result and exits non-zero on any operation or durability failure.
10
+
11
+ Use a temporary database by default. To retain an artifact for inspection, provide an explicit path:
12
+
13
+ ```sh
14
+ RUBYDB_WORKLOAD_PATH=/var/lib/rubydb/workload.rdb RUBYDB_WORKLOAD_THREADS=16 RUBYDB_WORKLOAD_OPERATIONS=10000 ruby benchmarks/concurrent_workload.rb
15
+ ```
16
+
17
+ Run this in a staging environment on the target filesystem before deployment. Establish acceptable latency, throughput, WAL growth, disk-space, recovery-time, and backup/restore targets for the expected workload; do not treat a successful smoke workload as capacity certification.
@@ -1,118 +1,118 @@
1
- # Production-readiness audit
2
-
3
- ## Status
4
-
5
- RubyDB has verified production-oriented foundations, but it is not yet a general-purpose production database. Supported behavior is backed by the RSpec suite in `spec/`.
6
-
7
- Latest local audit: 270 examples, 0 failures. Hosted CI and deployment-specific
8
- evidence remain separate release gates.
9
-
10
- The feature inventory below was written during an earlier checkpoint and its
11
- parenthetical historical count is not the current total. Use the latest audit
12
- line above and rerun the suite for release evidence.
13
-
14
- ## Implemented features
15
-
16
- The repository contains substantial scaffolding for:
17
-
18
- - page-based storage
19
- - buffer pool caching
20
- - file and page management
21
- - WAL, recovery, and checkpoint modules
22
- - transactions and MVCC abstractions
23
- - SQL parser/planner infrastructure
24
- - server, protocol, and connection components
25
- - authentication and authorization modules
26
- - backup and replication APIs
27
- - monitoring and metrics interfaces
28
- - Rails integration adapters
29
- - ordered, locked migrations with durable version/checksum tracking
30
-
31
- ## Tested features
32
-
33
- The current suite verifies storage reopen, subprocess crash recovery, MVCC isolation/vacuum, durable visibility version-history traversal, constraints including ON DELETE/ON UPDATE referential actions and nullable values, indexes including deep B-tree splits, SQL execution including idempotent database/table/index DDL and SQL foreign-key actions, schema DDL, views, trigger DDL and dispatch, CREATE TABLE, ALTER TABLE column and constraint changes, transaction before-image rollback and SQL savepoints, and VACUUM, live TCP sessions, TLS transport, password/SCRAM authentication and authorization, metrics updates including Prometheus export, liveness/readiness health reporting through the server request router, live CLI doctor checks with safe repair behavior, truthful CLI status reporting, live CLI branch diff/merge/checkout operations, live CLI inspection values, live CLI snapshot creation/listing and vacuum reporting, live CLI backup creation and restore dry-run validation, live CLI database creation and deletion, the documented SQL compatibility contract, full/snapshot/incremental/differential backup validation, logical replication, durable replica state, guarded failover promotion, durable fencing epochs, replication peer-token authentication, recovery resource checks, migration/schema-diff behavior including executable migration SQL serialization and changed-migration detection, atomic branch checkout, branch state/diff/merge behavior, foreign-key integrity lookup behavior, compound/null/boolean check-constraint evaluation, lock conflict/wait/timeout behavior, complete standalone and transaction-manager deadlock cycle detection with victim rollback, concurrency mutex initialization, Rails schema-builder SQL generation including defaults and foreign-key conventions, engine block-based table creation, adapter schema dumps preserving primary keys and defaults, ActiveRecord hash-form ordering and populated-table migration round trips, bounded multi-process workload supervision with child reaping, combined production traffic/deadline/cancellation/capacity/deadlock soak validation, release configuration, release tag/changelog preflight, upgrade guards, benchmark execution, executor query deadlines, wire-level cancellation of in-flight requests, standalone SCRAM verification, actual WAL checkpoint sizing, typed deserialization-corruption detection, preservation of stored false values when defaults are present, filesystem fault injection with descriptor cleanup, compaction/reopen validation, process-level replication failover with stale-writer fencing, and live TCP network-partition catch-up and promotion (268 examples, 0 failures at the latest audit).
34
-
35
- ## Known limitations
36
-
37
- - The concurrent workload and regression suite verify parallel appends, point
38
- reads, scans, durable reopen reads, multi-process server clients, and a
39
- process-level replication failover drill. This does not certify universal
40
- capacity; embedded ownership remains exclusive and multi-process clients must
41
- use the server.
42
-
43
- - multi-row SQL `VALUES` inserts execute through the parser, binder, planner,
44
- and executor, including per-row conflict handling and Rails insert IDs
45
- - incremental backups capture WAL mutations after a verified base LSN; differential backups capture the verified base-relative WAL delta and restore through the same validated delta path
46
- - replication is limited to the explicit logical row-mutation envelope API;
47
- process replacement, live TCP partition/catch-up, and same-host fencing are
48
- tested, while true multi-host partitions and split-brain recovery remain
49
- deployment work
50
- - automatic failover is limited to synchronized candidates; fencing requires a shared durable fence path and still needs multi-host split-brain validation
51
- - the SQLite compatibility profile covers the documented common application
52
- surface, but it is not SQLite file-format or extension compatibility
53
- - production CA lifecycle, independent security review, performance targets,
54
- and large-scale workload behavior still require dedicated validation
55
- - branch state application and live target-branch merges are supported through the engine reconciliation hook
56
-
57
- ## Unsupported SQL
58
-
59
- The supported dialect is defined in [docs/sql/compatibility.md](sql/compatibility.md) and enforced by parser/execution integration coverage. RubyDB provides a tested common SQLite-style profile, not complete SQLite file-format, extension, pragma, or error compatibility.
60
-
61
- ## Durability guarantees
62
-
63
- WAL framing, reopen behavior, subprocess crash-recovery scenarios, actual checkpoint sizing, fail-closed visibility-map loading, filesystem fault injection, compaction/reopen behavior, networked replication failover, and verified restore drills are tested. This is not a substitute for real quota, power-loss, and filesystem-specific validation.
64
-
65
- ## Transaction guarantees
66
-
67
- Transaction commit/rollback, MVCC visibility, snapshot isolation behavior, and vacuum safe-point handling are covered by focused tests. High-contention workload validation remains outstanding.
68
-
69
- ## Isolation guarantees
70
-
71
- Read committed, repeatable read, and serializable conflict behavior are covered by focused tests; broader workload and distributed-isolation validation remains outstanding.
72
-
73
- ## Backup guarantees
74
-
75
- Full compressed backups restore into a fresh directory and checksum tampering is detected. Incremental and differential backups validate the base, checksum the change set, and apply WAL row mutations during restore. Production-scale backup-chain and filesystem fault-injection testing remains outstanding.
76
-
77
- ## Replication guarantees
78
-
79
- Logical row-mutation streaming, LSN deduplication, acknowledgments, acknowledged-LSN reporting, synchronized-candidate promotion verification, process replacement, live TCP partition/catch-up, and stale-primary rejection through durable fencing epochs are tested. Multi-host fencing and split-brain recovery still require dedicated validation.
80
-
81
- ## TLS guarantees
82
-
83
- TLS 1.2 or newer is enforced when enabled. Certificate/key parsing, CA-file validation, TLS client connections, and the end-to-end SSL transport path are tested. Mutual TLS is configurable through client certificates; certificate rotation and production CA lifecycle procedures remain operational work.
84
-
85
- ## Security model
86
-
87
- Password and SCRAM-SHA-256 authentication, server-signature verification, read/write authorization, and startup validation of incomplete auth configuration are tested through the server/session path.
88
-
89
- ## Performance characteristics
90
-
91
- The repository includes a deterministic storage benchmark (`RUBYDB_BENCHMARK_ITERATIONS=100 ruby -Ilib benchmarks/basic_workload.rb`) and a concurrent write/durability workload (`ruby benchmarks/concurrent_workload.rb`). Both emit machine-readable JSON. Production-scale throughput, WAL, checkpoint, and concurrency targets still require workload-specific baselines.
92
-
93
- ## Deployment requirements
94
-
95
- The project currently needs:
96
-
97
- - a validated Ruby support matrix
98
- - RubyGems publication credentials (`RUBYGEMS_API_KEY`) configured as a protected
99
- GitHub Actions secret before pushing a release tag
100
- - a documented operator runbook for backups, upgrades, rollback, and incident preservation
101
- - explicit durability and crash-recovery validation
102
- - secure-by-default configuration development
103
- - a narrower set of supported features documented honestly
104
-
105
- ## Remaining roadmap items
106
-
107
- Main remaining work is to complete the sequence laid out in the repository design:
108
-
109
- 1. production CA lifecycle and certificate rotation
110
- 2. broader SQL compatibility contract and workload benchmarks
111
- 3. production-grade failover fencing and broader SQL compatibility
112
- 4. operator runbooks and deployment validation
113
-
114
- ## Bottom line
115
-
116
- RubyDB has a validated production-oriented foundation, but it does not yet meet
117
- the bar for unrestricted production use with real data outside its documented
118
- feature set and deployment-specific validation.
1
+ # Production-readiness audit
2
+
3
+ ## Status
4
+
5
+ RubyDB has verified production-oriented foundations, but it is not yet a general-purpose production database. Supported behavior is backed by the RSpec suite in `spec/`.
6
+
7
+ Latest local audit: 270 examples, 0 failures. Hosted CI and deployment-specific
8
+ evidence remain separate release gates.
9
+
10
+ The feature inventory below was written during an earlier checkpoint and its
11
+ parenthetical historical count is not the current total. Use the latest audit
12
+ line above and rerun the suite for release evidence.
13
+
14
+ ## Implemented features
15
+
16
+ The repository contains substantial scaffolding for:
17
+
18
+ - page-based storage
19
+ - buffer pool caching
20
+ - file and page management
21
+ - WAL, recovery, and checkpoint modules
22
+ - transactions and MVCC abstractions
23
+ - SQL parser/planner infrastructure
24
+ - server, protocol, and connection components
25
+ - authentication and authorization modules
26
+ - backup and replication APIs
27
+ - monitoring and metrics interfaces
28
+ - Rails integration adapters
29
+ - ordered, locked migrations with durable version/checksum tracking
30
+
31
+ ## Tested features
32
+
33
+ The current suite verifies storage reopen, subprocess crash recovery, MVCC isolation/vacuum, durable visibility version-history traversal, constraints including ON DELETE/ON UPDATE referential actions and nullable values, indexes including deep B-tree splits, SQL execution including idempotent database/table/index DDL and SQL foreign-key actions, schema DDL, views, trigger DDL and dispatch, CREATE TABLE, ALTER TABLE column and constraint changes, transaction before-image rollback and SQL savepoints, and VACUUM, live TCP sessions, TLS transport, password/SCRAM authentication and authorization, metrics updates including Prometheus export, liveness/readiness health reporting through the server request router, live CLI doctor checks with safe repair behavior, truthful CLI status reporting, live CLI branch diff/merge/checkout operations, live CLI inspection values, live CLI snapshot creation/listing and vacuum reporting, live CLI backup creation and restore dry-run validation, live CLI database creation and deletion, the documented SQL compatibility contract, full/snapshot/incremental/differential backup validation, logical replication, durable replica state, guarded failover promotion, durable fencing epochs, replication peer-token authentication, recovery resource checks, migration/schema-diff behavior including executable migration SQL serialization and changed-migration detection, atomic branch checkout, branch state/diff/merge behavior, foreign-key integrity lookup behavior, compound/null/boolean check-constraint evaluation, lock conflict/wait/timeout behavior, complete standalone and transaction-manager deadlock cycle detection with victim rollback, concurrency mutex initialization, Rails schema-builder SQL generation including defaults and foreign-key conventions, engine block-based table creation, adapter schema dumps preserving primary keys and defaults, ActiveRecord hash-form ordering and populated-table migration round trips, bounded multi-process workload supervision with child reaping, combined production traffic/deadline/cancellation/capacity/deadlock soak validation, release configuration, release tag/changelog preflight, upgrade guards, benchmark execution, executor query deadlines, wire-level cancellation of in-flight requests, standalone SCRAM verification, actual WAL checkpoint sizing, typed deserialization-corruption detection, preservation of stored false values when defaults are present, filesystem fault injection with descriptor cleanup, compaction/reopen validation, process-level replication failover with stale-writer fencing, and live TCP network-partition catch-up and promotion (268 examples, 0 failures at the latest audit).
34
+
35
+ ## Known limitations
36
+
37
+ - The concurrent workload and regression suite verify parallel appends, point
38
+ reads, scans, durable reopen reads, multi-process server clients, and a
39
+ process-level replication failover drill. This does not certify universal
40
+ capacity; embedded ownership remains exclusive and multi-process clients must
41
+ use the server.
42
+
43
+ - multi-row SQL `VALUES` inserts execute through the parser, binder, planner,
44
+ and executor, including per-row conflict handling and Rails insert IDs
45
+ - incremental backups capture WAL mutations after a verified base LSN; differential backups capture the verified base-relative WAL delta and restore through the same validated delta path
46
+ - replication is limited to the explicit logical row-mutation envelope API;
47
+ process replacement, live TCP partition/catch-up, and same-host fencing are
48
+ tested, while true multi-host partitions and split-brain recovery remain
49
+ deployment work
50
+ - automatic failover is limited to synchronized candidates; fencing requires a shared durable fence path and still needs multi-host split-brain validation
51
+ - the SQLite compatibility profile covers the documented common application
52
+ surface, but it is not SQLite file-format or extension compatibility
53
+ - production CA lifecycle, independent security review, performance targets,
54
+ and large-scale workload behavior still require dedicated validation
55
+ - branch state application and live target-branch merges are supported through the engine reconciliation hook
56
+
57
+ ## Unsupported SQL
58
+
59
+ The supported dialect is defined in [docs/sql/compatibility.md](sql/compatibility.md) and enforced by parser/execution integration coverage. RubyDB provides a tested common SQLite-style profile, not complete SQLite file-format, extension, pragma, or error compatibility.
60
+
61
+ ## Durability guarantees
62
+
63
+ WAL framing, reopen behavior, subprocess crash-recovery scenarios, actual checkpoint sizing, fail-closed visibility-map loading, filesystem fault injection, compaction/reopen behavior, networked replication failover, and verified restore drills are tested. This is not a substitute for real quota, power-loss, and filesystem-specific validation.
64
+
65
+ ## Transaction guarantees
66
+
67
+ Transaction commit/rollback, MVCC visibility, snapshot isolation behavior, and vacuum safe-point handling are covered by focused tests. High-contention workload validation remains outstanding.
68
+
69
+ ## Isolation guarantees
70
+
71
+ Read committed, repeatable read, and serializable conflict behavior are covered by focused tests; broader workload and distributed-isolation validation remains outstanding.
72
+
73
+ ## Backup guarantees
74
+
75
+ Full compressed backups restore into a fresh directory and checksum tampering is detected. Incremental and differential backups validate the base, checksum the change set, and apply WAL row mutations during restore. Production-scale backup-chain and filesystem fault-injection testing remains outstanding.
76
+
77
+ ## Replication guarantees
78
+
79
+ Logical row-mutation streaming, LSN deduplication, acknowledgments, acknowledged-LSN reporting, synchronized-candidate promotion verification, process replacement, live TCP partition/catch-up, and stale-primary rejection through durable fencing epochs are tested. Multi-host fencing and split-brain recovery still require dedicated validation.
80
+
81
+ ## TLS guarantees
82
+
83
+ TLS 1.2 or newer is enforced when enabled. Certificate/key parsing, CA-file validation, TLS client connections, and the end-to-end SSL transport path are tested. Mutual TLS is configurable through client certificates; certificate rotation and production CA lifecycle procedures remain operational work.
84
+
85
+ ## Security model
86
+
87
+ Password and SCRAM-SHA-256 authentication, server-signature verification, read/write authorization, and startup validation of incomplete auth configuration are tested through the server/session path.
88
+
89
+ ## Performance characteristics
90
+
91
+ The repository includes a deterministic storage benchmark (`RUBYDB_BENCHMARK_ITERATIONS=100 ruby -Ilib benchmarks/basic_workload.rb`) and a concurrent write/durability workload (`ruby benchmarks/concurrent_workload.rb`). Both emit machine-readable JSON. Production-scale throughput, WAL, checkpoint, and concurrency targets still require workload-specific baselines.
92
+
93
+ ## Deployment requirements
94
+
95
+ The project currently needs:
96
+
97
+ - a validated Ruby support matrix
98
+ - RubyGems publication credentials (`RUBYGEMS_API_KEY`) configured as a protected
99
+ GitHub Actions secret before pushing a release tag
100
+ - a documented operator runbook for backups, upgrades, rollback, and incident preservation
101
+ - explicit durability and crash-recovery validation
102
+ - secure-by-default configuration development
103
+ - a narrower set of supported features documented honestly
104
+
105
+ ## Remaining roadmap items
106
+
107
+ Main remaining work is to complete the sequence laid out in the repository design:
108
+
109
+ 1. production CA lifecycle and certificate rotation
110
+ 2. broader SQL compatibility contract and workload benchmarks
111
+ 3. production-grade failover fencing and broader SQL compatibility
112
+ 4. operator runbooks and deployment validation
113
+
114
+ ## Bottom line
115
+
116
+ RubyDB has a validated production-oriented foundation, but it does not yet meet
117
+ the bar for unrestricted production use with real data outside its documented
118
+ feature set and deployment-specific validation.