rubydb 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (436) hide show
  1. checksums.yaml +7 -0
  2. data/.gitattributes +20 -0
  3. data/.github/ISSUE_TEMPLATE/bug.yml +0 -0
  4. data/.github/ISSUE_TEMPLATE/feature.yml +0 -0
  5. data/.github/ISSUE_TEMPLATE/performance.yml +0 -0
  6. data/.github/ISSUE_TEMPLATE/security.yml +0 -0
  7. data/.github/PULL_REQUEST_TEMPLATE.md +15 -0
  8. data/.github/dependabot.yml +0 -0
  9. data/.github/workflows/benchmark.yml +26 -0
  10. data/.github/workflows/compatibility.yml +63 -0
  11. data/.github/workflows/fuzz.yml +33 -0
  12. data/.github/workflows/lint.yml +21 -0
  13. data/.github/workflows/operations.yml +24 -0
  14. data/.github/workflows/production-validation.yml +111 -0
  15. data/.github/workflows/release.yml +77 -0
  16. data/.github/workflows/security.yml +37 -0
  17. data/.github/workflows/test.yml +26 -0
  18. data/.github/workflows/workload.yml +58 -0
  19. data/.gitignore +43 -0
  20. data/.rspec +4 -0
  21. data/.rubocop.yml +44 -0
  22. data/.ruby-version +1 -0
  23. data/.standard.yml +14 -0
  24. data/ARCHITECTURE.md +21 -0
  25. data/CHANGELOG.md +31 -0
  26. data/CODE_OF_CONDUCT.md +13 -0
  27. data/CONTRIBUTING.md +29 -0
  28. data/GOVERNANCE.md +16 -0
  29. data/Gemfile +17 -0
  30. data/Gemfile.lock +71 -0
  31. data/LICENSE +60 -0
  32. data/README.md +162 -0
  33. data/ROADMAP.md +27 -0
  34. data/Rakefile +71 -0
  35. data/SECURITY.md +54 -0
  36. data/SUPPORT.md +14 -0
  37. data/adapters/activerecord/Gemfile +11 -0
  38. data/adapters/activerecord/README.md +9 -0
  39. data/adapters/activerecord/lib/active_record/connection_adapters/rubydb_adapter.rb +887 -0
  40. data/adapters/activerecord/rubydb-activerecord.gemspec +27 -0
  41. data/adapters/activerecord/spec/rubydb_adapter_integration_spec.rb +151 -0
  42. data/adapters/ruby/README.md +18 -0
  43. data/adapters/sequel/README.md +11 -0
  44. data/bin/rubydb +19 -0
  45. data/bin/rubydb-server +126 -0
  46. data/config/development.yml +0 -0
  47. data/config/monitoring/prometheus-alerts.yml +39 -0
  48. data/config/production.yml +36 -0
  49. data/config/test.yml +0 -0
  50. data/docs/README.md +71 -0
  51. data/docs/architecture/concurrency.md +14 -0
  52. data/docs/architecture/current-state.md +125 -0
  53. data/docs/architecture/execution-engine.md +25 -0
  54. data/docs/architecture/indexes.md +19 -0
  55. data/docs/architecture/mvcc.md +19 -0
  56. data/docs/architecture/overview.md +13 -0
  57. data/docs/architecture/pages.md +11 -0
  58. data/docs/architecture/production-roadmap.md +82 -0
  59. data/docs/architecture/query-planner.md +20 -0
  60. data/docs/architecture/recovery.md +18 -0
  61. data/docs/architecture/sql-engine.md +12 -0
  62. data/docs/architecture/storage-engine.md +14 -0
  63. data/docs/architecture/transactions.md +10 -0
  64. data/docs/architecture/wal.md +28 -0
  65. data/docs/cli-cheatsheet.md +98 -0
  66. data/docs/cli.md +275 -0
  67. data/docs/contributing/architecture.md +9 -0
  68. data/docs/contributing/benchmarking.md +14 -0
  69. data/docs/contributing/development.md +16 -0
  70. data/docs/contributing/release-process.md +49 -0
  71. data/docs/contributing/testing.md +16 -0
  72. data/docs/debugging.md +229 -0
  73. data/docs/developer/branching.md +10 -0
  74. data/docs/developer/database-diff.md +10 -0
  75. data/docs/developer/local-development.md +17 -0
  76. data/docs/developer/snapshots.md +9 -0
  77. data/docs/developer/temporal-data.md +10 -0
  78. data/docs/developer-guide.md +297 -0
  79. data/docs/getting-started/first-database.md +16 -0
  80. data/docs/getting-started/first-query.md +13 -0
  81. data/docs/getting-started/installation.md +19 -0
  82. data/docs/getting-started/local-to-production.md +300 -0
  83. data/docs/getting-started/quickstart.md +17 -0
  84. data/docs/getting-started/rails.md +16 -0
  85. data/docs/hardening_backlog.md +93 -0
  86. data/docs/lessons-learned.md +112 -0
  87. data/docs/operations/backups.md +33 -0
  88. data/docs/operations/disaster-recovery.md +31 -0
  89. data/docs/operations/failover.md +30 -0
  90. data/docs/operations/monitoring.md +25 -0
  91. data/docs/operations/production-guide.md +295 -0
  92. data/docs/operations/production-runbook.md +45 -0
  93. data/docs/operations/replication.md +33 -0
  94. data/docs/operations/restore.md +6 -0
  95. data/docs/operations/runbook.md +34 -0
  96. data/docs/operations/upgrades.md +14 -0
  97. data/docs/operations/workload-testing.md +17 -0
  98. data/docs/production-readiness.md +118 -0
  99. data/docs/production_validation.md +150 -0
  100. data/docs/rails/active-record.md +11 -0
  101. data/docs/rails/compatibility-guide.md +90 -0
  102. data/docs/rails/database-yml.md +92 -0
  103. data/docs/rails/installation.md +17 -0
  104. data/docs/rails/migrations.md +17 -0
  105. data/docs/rails/production.md +82 -0
  106. data/docs/rails/troubleshooting.md +18 -0
  107. data/docs/release.md +59 -0
  108. data/docs/server/architecture.md +10 -0
  109. data/docs/server/authentication.md +10 -0
  110. data/docs/server/configuration.md +16 -0
  111. data/docs/server/connection-pooling.md +10 -0
  112. data/docs/server/deployment.md +10 -0
  113. data/docs/server/protocol.md +12 -0
  114. data/docs/sql/compatibility-guide.md +82 -0
  115. data/docs/sql/compatibility.md +39 -0
  116. data/docs/sql/data-types.md +10 -0
  117. data/docs/sql/functions.md +9 -0
  118. data/docs/sql/joins.md +9 -0
  119. data/docs/sql/operators.md +9 -0
  120. data/docs/sql/sqlite-compatibility.md +21 -0
  121. data/docs/sql/syntax.md +10 -0
  122. data/docs/sql/transactions.md +10 -0
  123. data/docs/troubleshooting.md +244 -0
  124. data/exe/rubydb +23 -0
  125. data/lib/rubydb/backup/archive.rb +334 -0
  126. data/lib/rubydb/backup/backup.rb +401 -0
  127. data/lib/rubydb/backup/incremental.rb +353 -0
  128. data/lib/rubydb/backup/restore.rb +290 -0
  129. data/lib/rubydb/backup/snapshot.rb +267 -0
  130. data/lib/rubydb/backup/verification.rb +279 -0
  131. data/lib/rubydb/branching/branch.rb +181 -0
  132. data/lib/rubydb/branching/branch_manager.rb +311 -0
  133. data/lib/rubydb/branching/branch_metadata.rb +140 -0
  134. data/lib/rubydb/branching/checkout.rb +166 -0
  135. data/lib/rubydb/branching/copy_on_write.rb +272 -0
  136. data/lib/rubydb/branching/diff.rb +138 -0
  137. data/lib/rubydb/branching/merge.rb +285 -0
  138. data/lib/rubydb/build_info.rb +15 -0
  139. data/lib/rubydb/catalog/catalog.rb +391 -0
  140. data/lib/rubydb/catalog/column.rb +112 -0
  141. data/lib/rubydb/catalog/constraint.rb +180 -0
  142. data/lib/rubydb/catalog/database.rb +184 -0
  143. data/lib/rubydb/catalog/index.rb +97 -0
  144. data/lib/rubydb/catalog/schema.rb +103 -0
  145. data/lib/rubydb/catalog/sequence.rb +90 -0
  146. data/lib/rubydb/catalog/system_catalog.rb +698 -0
  147. data/lib/rubydb/catalog/table.rb +178 -0
  148. data/lib/rubydb/catalog/trigger.rb +102 -0
  149. data/lib/rubydb/catalog/view.rb +66 -0
  150. data/lib/rubydb/cli/application.rb +163 -0
  151. data/lib/rubydb/cli/commands/backup.rb +81 -0
  152. data/lib/rubydb/cli/commands/branch.rb +72 -0
  153. data/lib/rubydb/cli/commands/checkout.rb +54 -0
  154. data/lib/rubydb/cli/commands/create.rb +58 -0
  155. data/lib/rubydb/cli/commands/diff.rb +77 -0
  156. data/lib/rubydb/cli/commands/doctor.rb +74 -0
  157. data/lib/rubydb/cli/commands/drop.rb +57 -0
  158. data/lib/rubydb/cli/commands/init.rb +102 -0
  159. data/lib/rubydb/cli/commands/inspect.rb +95 -0
  160. data/lib/rubydb/cli/commands/merge.rb +63 -0
  161. data/lib/rubydb/cli/commands/migrate.rb +62 -0
  162. data/lib/rubydb/cli/commands/restart.rb +39 -0
  163. data/lib/rubydb/cli/commands/restore.rb +121 -0
  164. data/lib/rubydb/cli/commands/shell.rb +365 -0
  165. data/lib/rubydb/cli/commands/snapshot.rb +79 -0
  166. data/lib/rubydb/cli/commands/start.rb +82 -0
  167. data/lib/rubydb/cli/commands/status.rb +92 -0
  168. data/lib/rubydb/cli/commands/stop.rb +47 -0
  169. data/lib/rubydb/cli/commands/vacuum.rb +58 -0
  170. data/lib/rubydb/cli/formatter.rb +221 -0
  171. data/lib/rubydb/cli/output.rb +168 -0
  172. data/lib/rubydb/client/client.rb +304 -0
  173. data/lib/rubydb/client/connection.rb +415 -0
  174. data/lib/rubydb/client/connection_pool.rb +168 -0
  175. data/lib/rubydb/client/connection_url.rb +96 -0
  176. data/lib/rubydb/client/prepared_statement.rb +60 -0
  177. data/lib/rubydb/client/result.rb +123 -0
  178. data/lib/rubydb/client/statement.rb +52 -0
  179. data/lib/rubydb/client/transaction.rb +130 -0
  180. data/lib/rubydb/concurrency/concurrency.rb +19 -0
  181. data/lib/rubydb/concurrency/deadlock_detector.rb +150 -0
  182. data/lib/rubydb/concurrency/latch.rb +101 -0
  183. data/lib/rubydb/concurrency/lock_graph.rb +165 -0
  184. data/lib/rubydb/concurrency/mutex.rb +183 -0
  185. data/lib/rubydb/concurrency/rw_lock.rb +180 -0
  186. data/lib/rubydb/concurrency/scheduler.rb +250 -0
  187. data/lib/rubydb/concurrency/worker_pool.rb +143 -0
  188. data/lib/rubydb/configuration/config.rb +170 -0
  189. data/lib/rubydb/configuration/defaults.rb +179 -0
  190. data/lib/rubydb/configuration/environment.rb +152 -0
  191. data/lib/rubydb/configuration/parser.rb +185 -0
  192. data/lib/rubydb/configuration/validation.rb +221 -0
  193. data/lib/rubydb/constants.rb +74 -0
  194. data/lib/rubydb/constraints/check.rb +181 -0
  195. data/lib/rubydb/constraints/constraint.rb +101 -0
  196. data/lib/rubydb/constraints/foreign_key.rb +130 -0
  197. data/lib/rubydb/constraints/not_null.rb +64 -0
  198. data/lib/rubydb/constraints/primary_key.rb +99 -0
  199. data/lib/rubydb/constraints/unique.rb +108 -0
  200. data/lib/rubydb/constraints/validator.rb +350 -0
  201. data/lib/rubydb/errors/authentication_error.rb +10 -0
  202. data/lib/rubydb/errors/authorization_error.rb +23 -0
  203. data/lib/rubydb/errors/client_error.rb +10 -0
  204. data/lib/rubydb/errors/configuration_error.rb +10 -0
  205. data/lib/rubydb/errors/connection_error.rb +10 -0
  206. data/lib/rubydb/errors/constraint_error.rb +23 -0
  207. data/lib/rubydb/errors/corruption_error.rb +10 -0
  208. data/lib/rubydb/errors/database_error.rb +10 -0
  209. data/lib/rubydb/errors/error.rb +20 -0
  210. data/lib/rubydb/errors/execution_error.rb +10 -0
  211. data/lib/rubydb/errors/parser_error.rb +10 -0
  212. data/lib/rubydb/errors/recovery_error.rb +10 -0
  213. data/lib/rubydb/errors/replication_error.rb +10 -0
  214. data/lib/rubydb/errors/server_error.rb +6 -0
  215. data/lib/rubydb/errors/storage_error.rb +10 -0
  216. data/lib/rubydb/errors/transaction_error.rb +10 -0
  217. data/lib/rubydb/execution/aggregate_executor.rb +138 -0
  218. data/lib/rubydb/execution/delete_executor.rb +112 -0
  219. data/lib/rubydb/execution/distinct_executor.rb +135 -0
  220. data/lib/rubydb/execution/executor.rb +1192 -0
  221. data/lib/rubydb/execution/expression.rb +193 -0
  222. data/lib/rubydb/execution/index_scan.rb +142 -0
  223. data/lib/rubydb/execution/insert_executor.rb +217 -0
  224. data/lib/rubydb/execution/join_executor.rb +249 -0
  225. data/lib/rubydb/execution/limit_executor.rb +85 -0
  226. data/lib/rubydb/execution/optimizer.rb +215 -0
  227. data/lib/rubydb/execution/plan.rb +361 -0
  228. data/lib/rubydb/execution/planner.rb +546 -0
  229. data/lib/rubydb/execution/predicate.rb +235 -0
  230. data/lib/rubydb/execution/scan.rb +49 -0
  231. data/lib/rubydb/execution/sequential_scan.rb +63 -0
  232. data/lib/rubydb/execution/sort_executor.rb +185 -0
  233. data/lib/rubydb/execution/update_executor.rb +162 -0
  234. data/lib/rubydb/functions/aggregate.rb +70 -0
  235. data/lib/rubydb/functions/date_functions.rb +278 -0
  236. data/lib/rubydb/functions/function.rb +85 -0
  237. data/lib/rubydb/functions/json_functions.rb +215 -0
  238. data/lib/rubydb/functions/numeric_functions.rb +346 -0
  239. data/lib/rubydb/functions/scalar.rb +52 -0
  240. data/lib/rubydb/functions/string_functions.rb +383 -0
  241. data/lib/rubydb/functions/system_functions.rb +246 -0
  242. data/lib/rubydb/history/as_of.rb +238 -0
  243. data/lib/rubydb/history/change.rb +105 -0
  244. data/lib/rubydb/history/history.rb +131 -0
  245. data/lib/rubydb/history/history_manager.rb +229 -0
  246. data/lib/rubydb/history/temporal_query.rb +202 -0
  247. data/lib/rubydb/history/timeline.rb +144 -0
  248. data/lib/rubydb/indexes/btree.rb +186 -0
  249. data/lib/rubydb/indexes/btree_cursor.rb +258 -0
  250. data/lib/rubydb/indexes/btree_node.rb +385 -0
  251. data/lib/rubydb/indexes/hash_index.rb +150 -0
  252. data/lib/rubydb/indexes/index.rb +71 -0
  253. data/lib/rubydb/indexes/index_manager.rb +406 -0
  254. data/lib/rubydb/indexes/index_scan.rb +470 -0
  255. data/lib/rubydb/migrations/migration.rb +254 -0
  256. data/lib/rubydb/migrations/migration_lock.rb +146 -0
  257. data/lib/rubydb/migrations/migration_manager.rb +176 -0
  258. data/lib/rubydb/migrations/migration_version.rb +71 -0
  259. data/lib/rubydb/migrations/schema_diff.rb +211 -0
  260. data/lib/rubydb/migrations/schema_version.rb +64 -0
  261. data/lib/rubydb/monitoring/events.rb +160 -0
  262. data/lib/rubydb/monitoring/health.rb +222 -0
  263. data/lib/rubydb/monitoring/logger.rb +193 -0
  264. data/lib/rubydb/monitoring/metrics.rb +359 -0
  265. data/lib/rubydb/monitoring/performance.rb +176 -0
  266. data/lib/rubydb/monitoring/statistics.rb +170 -0
  267. data/lib/rubydb/mvcc/garbage_collector.rb +199 -0
  268. data/lib/rubydb/mvcc/mvcc.rb +16 -0
  269. data/lib/rubydb/mvcc/snapshot.rb +147 -0
  270. data/lib/rubydb/mvcc/vacuum.rb +180 -0
  271. data/lib/rubydb/mvcc/version.rb +106 -0
  272. data/lib/rubydb/mvcc/version_store.rb +398 -0
  273. data/lib/rubydb/mvcc/visibility.rb +109 -0
  274. data/lib/rubydb/protocol/capabilities.rb +125 -0
  275. data/lib/rubydb/protocol/decoder.rb +145 -0
  276. data/lib/rubydb/protocol/encoder.rb +136 -0
  277. data/lib/rubydb/protocol/handshake.rb +305 -0
  278. data/lib/rubydb/protocol/message.rb +121 -0
  279. data/lib/rubydb/protocol/protocol.rb +277 -0
  280. data/lib/rubydb/protocol/version.rb +54 -0
  281. data/lib/rubydb/rails/adapter.rb +239 -0
  282. data/lib/rubydb/rails/connection.rb +314 -0
  283. data/lib/rubydb/rails/database_statements.rb +122 -0
  284. data/lib/rubydb/rails/migration.rb +131 -0
  285. data/lib/rubydb/rails/quoting.rb +109 -0
  286. data/lib/rubydb/rails/result.rb +120 -0
  287. data/lib/rubydb/rails/schema_statements.rb +339 -0
  288. data/lib/rubydb/rails/transaction.rb +105 -0
  289. data/lib/rubydb/rails/type.rb +126 -0
  290. data/lib/rubydb/recovery/checkpoint.rb +257 -0
  291. data/lib/rubydb/recovery/consistency.rb +467 -0
  292. data/lib/rubydb/recovery/corruption_detector.rb +5 -0
  293. data/lib/rubydb/recovery/crash_recovery.rb +387 -0
  294. data/lib/rubydb/recovery/recovery_manager.rb +206 -0
  295. data/lib/rubydb/recovery/redo.rb +237 -0
  296. data/lib/rubydb/recovery/undo.rb +206 -0
  297. data/lib/rubydb/replication/failover.rb +5 -0
  298. data/lib/rubydb/replication/fencing.rb +63 -0
  299. data/lib/rubydb/replication/primary.rb +450 -0
  300. data/lib/rubydb/replication/replica.rb +384 -0
  301. data/lib/rubydb/replication/replication_log.rb +200 -0
  302. data/lib/rubydb/replication/replication_manager.rb +308 -0
  303. data/lib/rubydb/replication/replication_slot.rb +295 -0
  304. data/lib/rubydb/replication/replication_stream.rb +201 -0
  305. data/lib/rubydb/rubydb.rb +560 -0
  306. data/lib/rubydb/security/access_control.rb +254 -0
  307. data/lib/rubydb/security/audit_log.rb +213 -0
  308. data/lib/rubydb/security/authentication.rb +302 -0
  309. data/lib/rubydb/security/authorization.rb +282 -0
  310. data/lib/rubydb/security/credentials.rb +196 -0
  311. data/lib/rubydb/security/password.rb +215 -0
  312. data/lib/rubydb/security/permissions.rb +74 -0
  313. data/lib/rubydb/security/role.rb +101 -0
  314. data/lib/rubydb/security/user.rb +86 -0
  315. data/lib/rubydb/server/connection.rb +366 -0
  316. data/lib/rubydb/server/connection_pool.rb +193 -0
  317. data/lib/rubydb/server/lifecycle.rb +228 -0
  318. data/lib/rubydb/server/listener.rb +136 -0
  319. data/lib/rubydb/server/request_handler.rb +276 -0
  320. data/lib/rubydb/server/server.rb +364 -0
  321. data/lib/rubydb/server/session.rb +369 -0
  322. data/lib/rubydb/server/worker.rb +210 -0
  323. data/lib/rubydb/server/worker_pool.rb +168 -0
  324. data/lib/rubydb/sql/ast/alter_table.rb +169 -0
  325. data/lib/rubydb/sql/ast/begin_transaction.rb +47 -0
  326. data/lib/rubydb/sql/ast/commit.rb +37 -0
  327. data/lib/rubydb/sql/ast/constraint.rb +83 -0
  328. data/lib/rubydb/sql/ast/create_database.rb +41 -0
  329. data/lib/rubydb/sql/ast/create_index.rb +61 -0
  330. data/lib/rubydb/sql/ast/create_schema.rb +52 -0
  331. data/lib/rubydb/sql/ast/create_table.rb +187 -0
  332. data/lib/rubydb/sql/ast/delete.rb +54 -0
  333. data/lib/rubydb/sql/ast/drop_database.rb +41 -0
  334. data/lib/rubydb/sql/ast/drop_index.rb +41 -0
  335. data/lib/rubydb/sql/ast/drop_schema.rb +49 -0
  336. data/lib/rubydb/sql/ast/drop_table.rb +49 -0
  337. data/lib/rubydb/sql/ast/explain.rb +64 -0
  338. data/lib/rubydb/sql/ast/expression.rb +604 -0
  339. data/lib/rubydb/sql/ast/insert.rb +82 -0
  340. data/lib/rubydb/sql/ast/node.rb +42 -0
  341. data/lib/rubydb/sql/ast/rollback.rb +63 -0
  342. data/lib/rubydb/sql/ast/savepoint.rb +59 -0
  343. data/lib/rubydb/sql/ast/select.rb +88 -0
  344. data/lib/rubydb/sql/ast/set_operation.rb +20 -0
  345. data/lib/rubydb/sql/ast/trigger.rb +29 -0
  346. data/lib/rubydb/sql/ast/update.rb +88 -0
  347. data/lib/rubydb/sql/ast/vacuum.rb +19 -0
  348. data/lib/rubydb/sql/ast/view.rb +32 -0
  349. data/lib/rubydb/sql/ast/with.rb +32 -0
  350. data/lib/rubydb/sql/grammar.rb +86 -0
  351. data/lib/rubydb/sql/keywords.rb +156 -0
  352. data/lib/rubydb/sql/lexer.rb +214 -0
  353. data/lib/rubydb/sql/operators.rb +100 -0
  354. data/lib/rubydb/sql/parser.rb +1185 -0
  355. data/lib/rubydb/sql/planner/analyzer.rb +302 -0
  356. data/lib/rubydb/sql/planner/binder.rb +550 -0
  357. data/lib/rubydb/sql/planner/type_checker.rb +431 -0
  358. data/lib/rubydb/sql/token.rb +210 -0
  359. data/lib/rubydb/storage/buffer_frame.rb +44 -0
  360. data/lib/rubydb/storage/buffer_pool.rb +155 -0
  361. data/lib/rubydb/storage/database_lock.rb +74 -0
  362. data/lib/rubydb/storage/deserializer.rb +342 -0
  363. data/lib/rubydb/storage/engine.rb +2338 -0
  364. data/lib/rubydb/storage/file_manager.rb +187 -0
  365. data/lib/rubydb/storage/free_space_map.rb +81 -0
  366. data/lib/rubydb/storage/page.rb +94 -0
  367. data/lib/rubydb/storage/page_allocator.rb +855 -0
  368. data/lib/rubydb/storage/page_header.rb +67 -0
  369. data/lib/rubydb/storage/page_manager.rb +131 -0
  370. data/lib/rubydb/storage/record.rb +58 -0
  371. data/lib/rubydb/storage/row.rb +78 -0
  372. data/lib/rubydb/storage/serializer.rb +51 -0
  373. data/lib/rubydb/storage/storage_layout.rb +151 -0
  374. data/lib/rubydb/storage/storage_manager.rb +114 -0
  375. data/lib/rubydb/storage/tuple.rb +461 -0
  376. data/lib/rubydb/storage/visibility_map.rb +984 -0
  377. data/lib/rubydb/transactions/commit_manager.rb +220 -0
  378. data/lib/rubydb/transactions/isolation.rb +98 -0
  379. data/lib/rubydb/transactions/lock.rb +76 -0
  380. data/lib/rubydb/transactions/lock_manager.rb +362 -0
  381. data/lib/rubydb/transactions/savepoint.rb +143 -0
  382. data/lib/rubydb/transactions/transaction.rb +215 -0
  383. data/lib/rubydb/transactions/transaction_id.rb +84 -0
  384. data/lib/rubydb/transactions/transaction_log.rb +257 -0
  385. data/lib/rubydb/transactions/transaction_manager.rb +435 -0
  386. data/lib/rubydb/types/bigint.rb +36 -0
  387. data/lib/rubydb/types/blob.rb +37 -0
  388. data/lib/rubydb/types/boolean.rb +34 -0
  389. data/lib/rubydb/types/date.rb +39 -0
  390. data/lib/rubydb/types/decimal.rb +48 -0
  391. data/lib/rubydb/types/float.rb +34 -0
  392. data/lib/rubydb/types/integer.rb +36 -0
  393. data/lib/rubydb/types/json.rb +41 -0
  394. data/lib/rubydb/types/null.rb +34 -0
  395. data/lib/rubydb/types/smallint.rb +36 -0
  396. data/lib/rubydb/types/text.rb +37 -0
  397. data/lib/rubydb/types/time.rb +46 -0
  398. data/lib/rubydb/types/timestamp.rb +39 -0
  399. data/lib/rubydb/types/type.rb +119 -0
  400. data/lib/rubydb/types/uuid.rb +47 -0
  401. data/lib/rubydb/types/varchar.rb +37 -0
  402. data/lib/rubydb/version.rb +33 -0
  403. data/lib/rubydb/wal/archive.rb +193 -0
  404. data/lib/rubydb/wal/checkpoint.rb +183 -0
  405. data/lib/rubydb/wal/lsn.rb +94 -0
  406. data/lib/rubydb/wal/reader.rb +260 -0
  407. data/lib/rubydb/wal/record.rb +105 -0
  408. data/lib/rubydb/wal/segment.rb +193 -0
  409. data/lib/rubydb/wal/wal.rb +452 -0
  410. data/lib/rubydb/wal/writer.rb +236 -0
  411. data/lib/rubydb.rb +7 -0
  412. data/packaging/docker/Dockerfile +36 -0
  413. data/packaging/docker/docker-compose.failover.yml +43 -0
  414. data/packaging/docker/entrypoint.sh +15 -0
  415. data/packaging/homebrew/rubydb.rb +19 -0
  416. data/packaging/systemd/rubydb.service +22 -0
  417. data/rubydb.gemspec +57 -0
  418. data/scripts/benchmark +7 -0
  419. data/scripts/durability_drill +37 -0
  420. data/scripts/format +0 -0
  421. data/scripts/fuzz +63 -0
  422. data/scripts/lint +0 -0
  423. data/scripts/release +50 -0
  424. data/scripts/release_check +43 -0
  425. data/scripts/replication_failover_drill +250 -0
  426. data/scripts/replication_network_failover_drill +255 -0
  427. data/scripts/restore_drill +45 -0
  428. data/scripts/security +0 -0
  429. data/scripts/setup +0 -0
  430. data/scripts/test +0 -0
  431. data/scripts/test-chaos +0 -0
  432. data/scripts/test-crash +0 -0
  433. data/scripts/test-integration +0 -0
  434. data/scripts/test-unit +0 -0
  435. data/visibility_map.json +1 -0
  436. metadata +538 -0
@@ -0,0 +1,25 @@
1
+ # Monitoring and alerting
2
+
3
+ Expose the RubyDB health and Prometheus endpoints through the deployment’s
4
+ authenticated monitoring path. Use liveness to decide whether a process
5
+ exists; use readiness to decide whether it is safe to route traffic.
6
+
7
+ At minimum, collect:
8
+
9
+ - request count, error count, latency, and active connections;
10
+ - WAL bytes/segments, checkpoint failures, flush failures, and recovery-required
11
+ commit acknowledgements;
12
+ - table/index counts, row counts, storage bytes, and vacuum/compaction duration;
13
+ - replication role, state, received/replayed LSN, lag, authentication failures,
14
+ and fencing epoch;
15
+ - process memory, CPU, open files, and filesystem free space.
16
+
17
+ Alert on readiness failure, repeated request errors, checkpoint or WAL flush
18
+ failures, any recovery-required acknowledgement, filesystem space below the
19
+ service threshold, replication lag beyond the application RPO, authentication
20
+ failures, and a fencing epoch change.
21
+
22
+ Dashboards must show rates and percentiles over time, not only current gauges.
23
+ Set thresholds from staging workload measurements and review them after every
24
+ capacity or schema change. A green liveness check alone is not evidence that
25
+ the database can accept traffic.
@@ -0,0 +1,295 @@
1
+ # RubyDB production operations guide
2
+
3
+ This guide describes a controlled production deployment for applications that
4
+ fit RubyDB’s documented feature surface. It is an operational companion to
5
+ the [production runbook](production-runbook.md), [disaster recovery guide](disaster-recovery.md),
6
+ and [monitoring guide](monitoring.md). It does not turn the current project
7
+ into a universal PostgreSQL, MySQL, or SQLite replacement.
8
+
9
+ ## Deployment decision
10
+
11
+ Choose embedded mode only when one process owns the database directory and the
12
+ application accepts process-local availability. Use server mode when web,
13
+ worker, migration, or administrative processes need concurrent access. Put
14
+ the database directory on storage with documented durability and rename/
15
+ flush semantics. Do not place it on an untested shared filesystem.
16
+
17
+ Before launch, validate the application’s schema, generated SQL, migrations,
18
+ backup/restore path, concurrency profile, and failure behavior against the
19
+ exact RubyDB version and configuration that will be deployed.
20
+
21
+ ## Reference topology
22
+
23
+ ```text
24
+ clients/web/workers -> TLS -> RubyDB server -> private database volume
25
+ |-> WAL/checkpoints
26
+ |-> verified backup destination
27
+ |-> metrics/log sink
28
+ optional replica --------------------^
29
+ ```
30
+
31
+ A replica is not a backup. A backup is not a fencing system. An application
32
+ load balancer health check is not proof that a primary is safe to write. Keep
33
+ these responsibilities separate.
34
+
35
+ ## Complete first deployment
36
+
37
+ The following is a reference deployment for a Rails or regular Ruby
38
+ application. Replace paths, hostnames, users, and limits with values approved
39
+ by your infrastructure team.
40
+
41
+ ### 1. Install and provision the server
42
+
43
+ Pin the RubyDB gem version on the database host. Do not use an unpinned
44
+ prerelease in a production deployment:
45
+
46
+ ```sh
47
+ gem install rubydb -v 0.1.0
48
+ useradd --system --home-dir /var/lib/rubydb --shell /usr/sbin/nologin rubydb
49
+ install -d -o rubydb -g rubydb -m 0700 /var/lib/rubydb/data
50
+ install -d -o rubydb -g rubydb -m 0750 /var/log/rubydb
51
+ install -d -o root -g rubydb -m 0750 /etc/rubydb
52
+ ```
53
+
54
+ Use a persistent local volume for `/var/lib/rubydb/data`. Keep backups on a
55
+ separate failure domain. The server account should not own application source,
56
+ certificate private keys, or unrelated host data.
57
+
58
+ ### 2. Configure the server
59
+
60
+ Start from the repository’s `config/production.yml` and place a reviewed copy
61
+ at `/etc/rubydb/production.yml`. Supply required secrets through the service
62
+ manager or secret store. At minimum, configure:
63
+
64
+ ```text
65
+ RUBYDB_HOST=0.0.0.0
66
+ RUBYDB_PORT=7432
67
+ RUBYDB_DATA_DIR=/var/lib/rubydb/data
68
+ RUBYDB_LOG_DIR=/var/log/rubydb
69
+ RUBYDB_USERNAME=app_rw
70
+ RUBYDB_PASSWORD=<secret-manager-value>
71
+ RUBYDB_SSL_ENABLED=true
72
+ RUBYDB_SSL_CERT_FILE=/etc/rubydb/tls/server.crt
73
+ RUBYDB_SSL_KEY_FILE=/etc/rubydb/tls/server.key
74
+ RUBYDB_SSL_CA_FILE=/etc/rubydb/tls/ca.crt
75
+ RUBYDB_SSL_VERIFY_PEER=true
76
+ ```
77
+
78
+ The server’s TLS private key and application password must be readable only by
79
+ the service or secret-management mechanism. Restrict port `7432` to the Rails
80
+ and worker network; do not expose it to the public internet.
81
+
82
+ ### 3. Start and verify the server
83
+
84
+ Run it under systemd, a supervised container, or an equivalent process
85
+ manager. The CLI configuration options must appear before the command:
86
+
87
+ ```sh
88
+ sudo -u rubydb env RUBYDB_USERNAME=app_rw RUBYDB_PASSWORD='from-secret-store' \
89
+ rubydb --config /etc/rubydb/production.yml --env production start
90
+ ```
91
+
92
+ In a systemd unit, use an `EnvironmentFile` protected with mode `0600` or a
93
+ native secret integration, then use:
94
+
95
+ ```ini
96
+ ExecStart=/usr/local/bin/rubydb --config /etc/rubydb/production.yml --env production start
97
+ Restart=on-failure
98
+ ```
99
+
100
+ Verify from the application network, not only from the database host:
101
+
102
+ ```sh
103
+ rubydb --config /etc/rubydb/production.yml --env production status --json
104
+ rubydb --config /etc/rubydb/production.yml --env production doctor --json
105
+ ```
106
+
107
+ Then run an authenticated TLS smoke query using the Ruby client and the same
108
+ URL the application will use:
109
+
110
+ ```sh
111
+ RUBYDB_URL='rubydbs://app_rw:URL_ENCODED_PASSWORD@db.example.com:7432/app?verify_peer=true&ca_file=%2Fetc%2Frubydb%2Ftls%2Fca.crt' \
112
+ ruby -rrubydb -e 'c=RubyDB::Client::Client.new(url: ENV.fetch("RUBYDB_URL")); p c.query("SELECT 1").to_hash; c.disconnect'
113
+ ```
114
+
115
+ ### 4. Configure a Rails application
116
+
117
+ In the Rails application’s `config/database.yml`, select server mode and read
118
+ the connection string from the deployment environment:
119
+
120
+ ```yaml
121
+ production:
122
+ adapter: rubydb
123
+ embedded: false
124
+ url: <%= ENV.fetch("RUBYDB_URL") %>
125
+ pool: <%= ENV.fetch("RAILS_MAX_THREADS", "5") %>
126
+ ```
127
+
128
+ Set `RUBYDB_URL` through the platform secret store:
129
+
130
+ ```text
131
+ RUBYDB_URL=rubydbs://app_rw:URL_ENCODED_PASSWORD@db.example.com:7432/app?verify_peer=true&ca_file=%2Fetc%2Frubydb%2Ftls%2Fca.crt
132
+ ```
133
+
134
+ Deploy the Rails application, then run the migration once using a controlled
135
+ release job, not concurrently from every web process:
136
+
137
+ ```sh
138
+ RAILS_ENV=production bundle exec rails db:migrate
139
+ RAILS_ENV=production bundle exec rails runner 'puts User.count'
140
+ ```
141
+
142
+ The web and job processes connect to the server endpoint. They never mount or
143
+ open `/var/lib/rubydb/data`.
144
+
145
+ ### 5. Configure a regular Ruby application
146
+
147
+ The regular client uses the same environment value:
148
+
149
+ ```ruby
150
+ require "rubydb"
151
+
152
+ client = RubyDB::Client::Client.new(url: ENV.fetch("RUBYDB_URL"))
153
+ begin
154
+ client.query("SELECT 1")
155
+ client.query("INSERT INTO audit_events (event_name) VALUES (?)", ["boot"])
156
+ ensure
157
+ client.disconnect
158
+ end
159
+ ```
160
+
161
+ Use application-level idempotency for retried writes. A network timeout does
162
+ not by itself prove that a commit was rolled back.
163
+
164
+ ### 6. Accept traffic gradually
165
+
166
+ Run a read/write smoke test, migration status check, backup, and representative
167
+ workload before routing all traffic. Watch p95/p99 latency, errors, lock waits,
168
+ active connections, WAL growth, checkpoint age, disk space, and memory during a
169
+ canary period. Keep the previous application version and verified database
170
+ backup available until the rollback window closes.
171
+
172
+ The Rails URL example and URL option reference are also maintained in [Rails
173
+ database configuration](../rails/database-yml.md).
174
+
175
+ ## Configuration and service identity
176
+
177
+ Run the server as a dedicated least-privilege account. Give it access only to
178
+ the database, WAL, temporary, certificate, and backup paths it needs. Store
179
+ passwords, peer tokens, private keys, and API credentials in a secret manager.
180
+ Do not put secrets in YAML committed to the repository, process arguments, or
181
+ logs.
182
+
183
+ Pin the RubyDB version and configuration for each deployment. Review changes to
184
+ durability mode, WAL retention, checkpoint thresholds, memory, connection
185
+ limits, request deadlines, lock timeouts, and TLS/authentication as production
186
+ changes. Keep a configuration checksum in the deployment record.
187
+
188
+ ## Readiness checklist
189
+
190
+ Before accepting traffic:
191
+
192
+ * database directory is on approved storage with sufficient space and inodes;
193
+ * service account and file permissions are verified;
194
+ * TLS certificate, key, CA, hostname, and expiration are checked;
195
+ * authentication and authorization deny an unauthenticated test client;
196
+ * health and readiness checks exercise a real request path;
197
+ * connection, request, lock, and shutdown timeouts are bounded;
198
+ * schema/migrations have completed and version/checksum is recorded;
199
+ * full backup has been created and restored into a fresh directory;
200
+ * monitoring, alert routing, and log retention are active;
201
+ * rollback and restore owners are named; and
202
+ * a representative smoke query and write have passed.
203
+
204
+ ## Capacity and resource limits
205
+
206
+ Set explicit limits for connections, request duration, lock waits, result size,
207
+ memory, worker count, WAL size, backup space, and file descriptors. Size the
208
+ application pool below the server limit, leaving room for migrations,
209
+ replication, health checks, and administration. A pool that equals the server
210
+ limit can starve the control plane.
211
+
212
+ Alert before exhaustion, not after it. Watch CPU, RSS, open files, disk bytes,
213
+ free inodes, WAL bytes, checkpoint age/duration, active transactions, lock
214
+ waits, pool utilization, errors, cancellations, and p95/p99 latency.
215
+
216
+ ## Backup policy
217
+
218
+ Define RPO and RTO with the application owner. At minimum, maintain verified
219
+ full backups, protect them from the database host, encrypt them at rest and in
220
+ transit, retain multiple generations, and record manifests/checksums. If using
221
+ incremental or differential backups, retain their verified base and ordered
222
+ chain.
223
+
224
+ A successful backup command is not proof of recoverability. Regularly restore
225
+ to an isolated directory, validate checksums and schema, run representative
226
+ queries, compare critical row counts, and record elapsed restore time. Run a
227
+ restore drill after format, backup, storage, or release changes.
228
+
229
+ ## Upgrade procedure
230
+
231
+ 1. Read the release notes, format compatibility, and migration notes.
232
+ 2. Create and verify a new full backup.
233
+ 3. Test the new version against a restored production-like copy.
234
+ 4. Run schema and application smoke tests, including populated-table writes.
235
+ 5. Drain or fence writes according to the deployment topology.
236
+ 6. Upgrade one controlled instance and verify health, WAL, and metrics.
237
+ 7. Re-enable traffic gradually and watch errors and latency.
238
+ 8. Keep the rollback binary and backup available until the validation window
239
+ closes.
240
+
241
+ Never roll back by pointing an older binary at a directory whose format or
242
+ metadata it cannot read. Use the documented restore/rollback path.
243
+
244
+ ## Failover procedure
245
+
246
+ RubyDB’s safe failover model requires a synchronized candidate and a durable
247
+ fencing decision. A manual operator sequence is:
248
+
249
+ 1. declare the incident and stop or isolate application writes;
250
+ 2. verify the primary’s last acknowledged LSN and fence epoch;
251
+ 3. confirm the candidate’s applied LSN and integrity;
252
+ 4. fence the old primary at the process, host, storage, or network layer;
253
+ 5. promote only after fencing is observable and durable;
254
+ 6. point clients at the new primary and run smoke writes/reads;
255
+ 7. monitor replication and stale-writer rejection; and
256
+ 8. recover the old primary as a replica only after its state is understood.
257
+
258
+ Automatic election requires an independently validated quorum, fencing,
259
+ partition behavior, stale-primary rejection, and recovery procedure. Do not
260
+ enable election based solely on a successful same-host test.
261
+
262
+ ## Incident response
263
+
264
+ Contain first: stop unsafe writes, protect the database directory, and record
265
+ the timeline. Preserve logs, metrics, WAL, metadata, configuration, process
266
+ state, and backup manifests. Use [troubleshooting](../troubleshooting.md) and
267
+ [debugging](../debugging.md) for evidence collection.
268
+
269
+ Classify the event as availability, durability, correctness, security, or
270
+ capacity. Assign an incident owner and a recovery owner. Communicate whether
271
+ commit outcomes are known, unknown, or confirmed rolled back. After recovery,
272
+ verify application invariants rather than relying only on process health.
273
+
274
+ ## TLS and secret rotation
275
+
276
+ Stage new certificates and CA material, validate the chain and hostname with a
277
+ test client, then switch through an atomic deployment/configuration change.
278
+ Maintain an overlap window only if clients support it. Confirm old material is
279
+ no longer accepted before revoking it. Rotate database passwords and peer
280
+ tokens through the secret manager; audit access and avoid printing values.
281
+
282
+ ## Maintenance
283
+
284
+ Schedule vacuum, compaction, checkpoint, index maintenance, and backups with
285
+ awareness of active readers and write load. Measure before and after. Use a
286
+ copy for repair or compaction experiments. Verify reopen, checksums, row counts,
287
+ indexes, and application smoke queries after maintenance.
288
+
289
+ ## Production evidence
290
+
291
+ The release record should contain the RubyDB commit, Ruby/Rails versions,
292
+ configuration checksum, schema/migration version, backup manifest, restore
293
+ drill result, benchmark/soak result, monitoring link, security review status,
294
+ and known limitations. See [production readiness](../production-readiness.md)
295
+ for the project-level boundaries.
@@ -0,0 +1,45 @@
1
+ # RubyDB production runbook
2
+
3
+ ## Before deployment
4
+
5
+ 1. Pin the RubyDB commit and Ruby version. Run `bundle exec rspec`, the
6
+ production soak, the durability drill, and the network failover drill.
7
+ 2. Place the database, WAL, lock file, replication state, and backups on
8
+ protected storage owned by the service account. Keep backups on a separate
9
+ failure domain.
10
+ 3. Enable TLS, use a private bind address, configure authentication and a
11
+ replication token from a secret manager, and set connection, frame, query,
12
+ worker, and disk-space limits.
13
+ 4. Install the alert rules in
14
+ `config/monitoring/prometheus-alerts.yml`. Test readiness, metrics, backup,
15
+ and restore from the deployment environment.
16
+
17
+ ## Backup and restore
18
+
19
+ Run a full backup before upgrades and retain the matching WAL chain and
20
+ manifest. Restore into a new inactive directory, run the restore drill, reopen
21
+ the database, compare row counts/checksums, and only then switch traffic.
22
+ Never overwrite the only copy during a restore.
23
+
24
+ ## Failover
25
+
26
+ Stop writes, confirm the replica is synchronized, fence the old primary using
27
+ the shared durable fence path, promote exactly one replica, and verify client
28
+ writes and replication status. Automatic election is not enabled until the
29
+ deployment provides an independently tested fencing lease. A network partition
30
+ must never result in two writable primaries.
31
+
32
+ ## TLS and secret rotation
33
+
34
+ Stage a new certificate/key and replication token in the secret manager,
35
+ validate them on an inactive instance, restart or drain one peer at a time,
36
+ verify TLS and replication authentication, then revoke the old credentials
37
+ after the overlap window. Do not place secrets in repository files, command
38
+ arguments, URLs, or logs.
39
+
40
+ ## Upgrade and incident response
41
+
42
+ Run the upgrade-compatibility suite against a copy of production backups.
43
+ Preserve the database, WAL, replication state, logs, metrics, and timestamps
44
+ before repair, vacuum, restore, or promotion. If checksums, fencing, or
45
+ replication diverge, stop writes and escalate; do not guess at a repair.
@@ -0,0 +1,33 @@
1
+ # Replication operations
2
+
3
+ RubyDB replication is logical WAL row-mutation replication. Configure the
4
+ primary and replica with separate data directories, explicit network addresses,
5
+ and the same high-entropy `replication_auth_token`. Restrict the listener with
6
+ firewall rules and use TLS or a trusted private network; the token is peer
7
+ authentication, not a replacement for transport encryption.
8
+
9
+ ## Bootstrap and catch-up
10
+
11
+ Start the primary first, then start the replica with its replica identifier and
12
+ primary address. The primary sends catalog schema before row replay. Verify the
13
+ replica reaches `SYNCED`, and compare received and replayed LSNs before routing
14
+ reads to it.
15
+
16
+ Monitor replication status continuously. A replica with a received/replayed
17
+ gap is not eligible for promotion. Preserve the replication log and replica
18
+ state files during an incident; do not manually edit LSN state.
19
+
20
+ ## Token rotation
21
+
22
+ Drain replication, stop or disconnect the replica, update the protected token
23
+ on both sides, restart the primary listener if required, and reconnect the
24
+ replica. Confirm accepted handshakes and zero unexpected authentication
25
+ failures before returning the replica to service. Never place tokens in source
26
+ control or command histories.
27
+
28
+ ## Failure handling
29
+
30
+ On a network partition, assume the old primary may still be reachable until it
31
+ is fenced. Do not promote a lagging replica and do not run two writable primaries
32
+ without a verified fencing lease. Re-bootstrap a divergent replica rather than
33
+ guessing which WAL records are safe to apply.
@@ -0,0 +1,6 @@
1
+ ## Restore verification
2
+
3
+ Before treating a backup as recoverable, run the verification path that extracts
4
+ it into an isolated temporary directory, opens the database with RubyDB, and
5
+ compares the restored catalog and table row counts with the manifest. The
6
+ verification removes its temporary database after the check.
@@ -0,0 +1,34 @@
1
+ # RubyDB operator runbook
2
+
3
+ ## Start and health check
4
+
5
+ Run the server with an explicit production configuration:
6
+
7
+ ```sh
8
+ RUBYDB_ENV=production rubydb -c config/production.yml start
9
+ ```
10
+
11
+ Confirm the process is listening, then run `rubydb status --json`. Treat a missing PID file, an invalid configuration, or a failed health check as a failed deployment; do not bypass startup validation.
12
+
13
+ ## Backup and restore
14
+
15
+ Create a verified full backup before upgrades or maintenance. Retain the manifest and WAL chain together. Test restoration into a separate directory before replacing a live data directory. Never restore over an active database.
16
+
17
+ ## Upgrade and rollback
18
+
19
+ Stop writes, create and verify a full backup, restore it into a staging directory, and run migration status plus application smoke queries. Promote the staged directory only after validation. Roll back by stopping RubyDB and switching back to the retained pre-upgrade directory; preserve the failed directory for diagnosis.
20
+
21
+ ## Incident boundaries
22
+
23
+ On corruption, checksum failure, repeated crash recovery, or replication divergence, stop writes and preserve the data directory and logs. Do not delete WAL, run vacuum, or force promotion until a verified backup and incident copy exist. Automatic failover is intentionally disabled and promotion requires an operator decision.
24
+
25
+ ## Security operations
26
+
27
+ Keep production credentials and TLS private keys outside the repository with restrictive permissions. Rotate certificates by staging the replacement, validating the chain, and restarting during a planned window. Keep TLS enabled and use peer verification when a trusted CA is available.
28
+
29
+ For replication, configure the same high-entropy `replication_auth_token` on
30
+ each primary and replica. A peer with a missing or incorrect token is rejected;
31
+ rotate the token by draining replicas, updating the protected configuration on
32
+ both sides, and reconnecting them. The replication listener is not a substitute
33
+ for a private network or TLS termination, so restrict its port at the firewall
34
+ and validate certificate/private-network controls during deployment.
@@ -0,0 +1,14 @@
1
+ # Upgrades
2
+
3
+ RubyDB upgrades must be treated as storage migrations, not as an in-place binary replacement.
4
+
5
+ Before upgrading:
6
+
7
+ 1. Stop writes and confirm the server is healthy.
8
+ 2. Create and verify a full backup, including WAL when available.
9
+ 3. Record the RubyDB version, storage page size, and backup manifest checksum.
10
+ 4. Test the new release against a restored copy of the backup.
11
+
12
+ The current storage format is version 1. RubyDB refuses to open an unknown page format or a database whose page size differs from the configured page size. There is no automatic on-disk format conversion yet; a future incompatible format must provide an explicit export/import migration.
13
+
14
+ After upgrading, reopen the restored or migrated database, run the migration status command, verify application queries, and retain the pre-upgrade backup until the rollback window closes. Never delete the original data directory as part of an upgrade.
@@ -0,0 +1,17 @@
1
+ # Workload testing
2
+
3
+ Run the deterministic concurrent workload from the repository root:
4
+
5
+ ```sh
6
+ RUBYDB_WORKLOAD_THREADS=4 RUBYDB_WORKLOAD_OPERATIONS=250 ruby benchmarks/concurrent_workload.rb
7
+ ```
8
+
9
+ It performs concurrent inserts against one engine, verifies the final row count, closes the database, reopens it, and verifies durable reads. It prints one JSON result and exits non-zero on any operation or durability failure.
10
+
11
+ Use a temporary database by default. To retain an artifact for inspection, provide an explicit path:
12
+
13
+ ```sh
14
+ RUBYDB_WORKLOAD_PATH=/var/lib/rubydb/workload.rdb RUBYDB_WORKLOAD_THREADS=16 RUBYDB_WORKLOAD_OPERATIONS=10000 ruby benchmarks/concurrent_workload.rb
15
+ ```
16
+
17
+ Run this in a staging environment on the target filesystem before deployment. Establish acceptable latency, throughput, WAL growth, disk-space, recovery-time, and backup/restore targets for the expected workload; do not treat a successful smoke workload as capacity certification.
@@ -0,0 +1,118 @@
1
+ # Production-readiness audit
2
+
3
+ ## Status
4
+
5
+ RubyDB has verified production-oriented foundations, but it is not yet a general-purpose production database. Supported behavior is backed by the RSpec suite in `spec/`.
6
+
7
+ Latest local audit: 270 examples, 0 failures. Hosted CI and deployment-specific
8
+ evidence remain separate release gates.
9
+
10
+ The feature inventory below was written during an earlier checkpoint and its
11
+ parenthetical historical count is not the current total. Use the latest audit
12
+ line above and rerun the suite for release evidence.
13
+
14
+ ## Implemented features
15
+
16
+ The repository contains substantial scaffolding for:
17
+
18
+ - page-based storage
19
+ - buffer pool caching
20
+ - file and page management
21
+ - WAL, recovery, and checkpoint modules
22
+ - transactions and MVCC abstractions
23
+ - SQL parser/planner infrastructure
24
+ - server, protocol, and connection components
25
+ - authentication and authorization modules
26
+ - backup and replication APIs
27
+ - monitoring and metrics interfaces
28
+ - Rails integration adapters
29
+ - ordered, locked migrations with durable version/checksum tracking
30
+
31
+ ## Tested features
32
+
33
+ The current suite verifies storage reopen, subprocess crash recovery, MVCC isolation/vacuum, durable visibility version-history traversal, constraints including ON DELETE/ON UPDATE referential actions and nullable values, indexes including deep B-tree splits, SQL execution including idempotent database/table/index DDL and SQL foreign-key actions, schema DDL, views, trigger DDL and dispatch, CREATE TABLE, ALTER TABLE column and constraint changes, transaction before-image rollback and SQL savepoints, and VACUUM, live TCP sessions, TLS transport, password/SCRAM authentication and authorization, metrics updates including Prometheus export, liveness/readiness health reporting through the server request router, live CLI doctor checks with safe repair behavior, truthful CLI status reporting, live CLI branch diff/merge/checkout operations, live CLI inspection values, live CLI snapshot creation/listing and vacuum reporting, live CLI backup creation and restore dry-run validation, live CLI database creation and deletion, the documented SQL compatibility contract, full/snapshot/incremental/differential backup validation, logical replication, durable replica state, guarded failover promotion, durable fencing epochs, replication peer-token authentication, recovery resource checks, migration/schema-diff behavior including executable migration SQL serialization and changed-migration detection, atomic branch checkout, branch state/diff/merge behavior, foreign-key integrity lookup behavior, compound/null/boolean check-constraint evaluation, lock conflict/wait/timeout behavior, complete standalone and transaction-manager deadlock cycle detection with victim rollback, concurrency mutex initialization, Rails schema-builder SQL generation including defaults and foreign-key conventions, engine block-based table creation, adapter schema dumps preserving primary keys and defaults, ActiveRecord hash-form ordering and populated-table migration round trips, bounded multi-process workload supervision with child reaping, combined production traffic/deadline/cancellation/capacity/deadlock soak validation, release configuration, release tag/changelog preflight, upgrade guards, benchmark execution, executor query deadlines, wire-level cancellation of in-flight requests, standalone SCRAM verification, actual WAL checkpoint sizing, typed deserialization-corruption detection, preservation of stored false values when defaults are present, filesystem fault injection with descriptor cleanup, compaction/reopen validation, process-level replication failover with stale-writer fencing, and live TCP network-partition catch-up and promotion (268 examples, 0 failures at the latest audit).
34
+
35
+ ## Known limitations
36
+
37
+ - The concurrent workload and regression suite verify parallel appends, point
38
+ reads, scans, durable reopen reads, multi-process server clients, and a
39
+ process-level replication failover drill. This does not certify universal
40
+ capacity; embedded ownership remains exclusive and multi-process clients must
41
+ use the server.
42
+
43
+ - multi-row SQL `VALUES` inserts execute through the parser, binder, planner,
44
+ and executor, including per-row conflict handling and Rails insert IDs
45
+ - incremental backups capture WAL mutations after a verified base LSN; differential backups capture the verified base-relative WAL delta and restore through the same validated delta path
46
+ - replication is limited to the explicit logical row-mutation envelope API;
47
+ process replacement, live TCP partition/catch-up, and same-host fencing are
48
+ tested, while true multi-host partitions and split-brain recovery remain
49
+ deployment work
50
+ - automatic failover is limited to synchronized candidates; fencing requires a shared durable fence path and still needs multi-host split-brain validation
51
+ - the SQLite compatibility profile covers the documented common application
52
+ surface, but it is not SQLite file-format or extension compatibility
53
+ - production CA lifecycle, independent security review, performance targets,
54
+ and large-scale workload behavior still require dedicated validation
55
+ - branch state application and live target-branch merges are supported through the engine reconciliation hook
56
+
57
+ ## Unsupported SQL
58
+
59
+ The supported dialect is defined in [docs/sql/compatibility.md](sql/compatibility.md) and enforced by parser/execution integration coverage. RubyDB provides a tested common SQLite-style profile, not complete SQLite file-format, extension, pragma, or error compatibility.
60
+
61
+ ## Durability guarantees
62
+
63
+ WAL framing, reopen behavior, subprocess crash-recovery scenarios, actual checkpoint sizing, fail-closed visibility-map loading, filesystem fault injection, compaction/reopen behavior, networked replication failover, and verified restore drills are tested. This is not a substitute for real quota, power-loss, and filesystem-specific validation.
64
+
65
+ ## Transaction guarantees
66
+
67
+ Transaction commit/rollback, MVCC visibility, snapshot isolation behavior, and vacuum safe-point handling are covered by focused tests. High-contention workload validation remains outstanding.
68
+
69
+ ## Isolation guarantees
70
+
71
+ Read committed, repeatable read, and serializable conflict behavior are covered by focused tests; broader workload and distributed-isolation validation remains outstanding.
72
+
73
+ ## Backup guarantees
74
+
75
+ Full compressed backups restore into a fresh directory and checksum tampering is detected. Incremental and differential backups validate the base, checksum the change set, and apply WAL row mutations during restore. Production-scale backup-chain and filesystem fault-injection testing remains outstanding.
76
+
77
+ ## Replication guarantees
78
+
79
+ Logical row-mutation streaming, LSN deduplication, acknowledgments, acknowledged-LSN reporting, synchronized-candidate promotion verification, process replacement, live TCP partition/catch-up, and stale-primary rejection through durable fencing epochs are tested. Multi-host fencing and split-brain recovery still require dedicated validation.
80
+
81
+ ## TLS guarantees
82
+
83
+ TLS 1.2 or newer is enforced when enabled. Certificate/key parsing, CA-file validation, TLS client connections, and the end-to-end SSL transport path are tested. Mutual TLS is configurable through client certificates; certificate rotation and production CA lifecycle procedures remain operational work.
84
+
85
+ ## Security model
86
+
87
+ Password and SCRAM-SHA-256 authentication, server-signature verification, read/write authorization, and startup validation of incomplete auth configuration are tested through the server/session path.
88
+
89
+ ## Performance characteristics
90
+
91
+ The repository includes a deterministic storage benchmark (`RUBYDB_BENCHMARK_ITERATIONS=100 ruby -Ilib benchmarks/basic_workload.rb`) and a concurrent write/durability workload (`ruby benchmarks/concurrent_workload.rb`). Both emit machine-readable JSON. Production-scale throughput, WAL, checkpoint, and concurrency targets still require workload-specific baselines.
92
+
93
+ ## Deployment requirements
94
+
95
+ The project currently needs:
96
+
97
+ - a validated Ruby support matrix
98
+ - RubyGems publication credentials (`RUBYGEMS_API_KEY`) configured as a protected
99
+ GitHub Actions secret before pushing a release tag
100
+ - a documented operator runbook for backups, upgrades, rollback, and incident preservation
101
+ - explicit durability and crash-recovery validation
102
+ - secure-by-default configuration development
103
+ - a narrower set of supported features documented honestly
104
+
105
+ ## Remaining roadmap items
106
+
107
+ Main remaining work is to complete the sequence laid out in the repository design:
108
+
109
+ 1. production CA lifecycle and certificate rotation
110
+ 2. broader SQL compatibility contract and workload benchmarks
111
+ 3. production-grade failover fencing and broader SQL compatibility
112
+ 4. operator runbooks and deployment validation
113
+
114
+ ## Bottom line
115
+
116
+ RubyDB has a validated production-oriented foundation, but it does not yet meet
117
+ the bar for unrestricted production use with real data outside its documented
118
+ feature set and deployment-specific validation.