rubydb 0.1.4 → 0.1.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (420) hide show
  1. checksums.yaml +4 -4
  2. data/.github/PULL_REQUEST_TEMPLATE.md +15 -15
  3. data/.github/workflows/benchmark.yml +26 -26
  4. data/.github/workflows/compatibility.yml +63 -63
  5. data/.github/workflows/fuzz.yml +33 -33
  6. data/.github/workflows/lint.yml +21 -21
  7. data/.github/workflows/operations.yml +24 -24
  8. data/.github/workflows/production-validation.yml +111 -111
  9. data/.github/workflows/release.yml +77 -77
  10. data/.github/workflows/security.yml +39 -37
  11. data/.github/workflows/test.yml +26 -26
  12. data/.github/workflows/workload.yml +58 -58
  13. data/.gitignore +12 -5
  14. data/.rubocop.yml +50 -44
  15. data/.standard.yml +9 -14
  16. data/ARCHITECTURE.md +21 -21
  17. data/CHANGELOG.md +43 -27
  18. data/CODE_OF_CONDUCT.md +13 -13
  19. data/CONTRIBUTING.md +29 -29
  20. data/GOVERNANCE.md +16 -16
  21. data/Gemfile +18 -17
  22. data/Gemfile.lock +125 -71
  23. data/README.md +249 -162
  24. data/ROADMAP.md +27 -27
  25. data/Rakefile +71 -71
  26. data/SECURITY.md +54 -54
  27. data/SUPPORT.md +14 -14
  28. data/adapters/activerecord/Gemfile +11 -11
  29. data/adapters/activerecord/lib/active_record/connection_adapters/rubydb_adapter.rb +873 -879
  30. data/adapters/activerecord/rubydb-activerecord.gemspec +20 -20
  31. data/adapters/activerecord/spec/rubydb_adapter_integration_spec.rb +143 -143
  32. data/adapters/ruby/README.md +18 -18
  33. data/adapters/sequel/README.md +11 -11
  34. data/config/monitoring/prometheus-alerts.yml +39 -39
  35. data/config/production.yml +36 -36
  36. data/docs/README.md +75 -71
  37. data/docs/architecture/concurrency.md +14 -14
  38. data/docs/architecture/current-state.md +125 -125
  39. data/docs/architecture/execution-engine.md +25 -25
  40. data/docs/architecture/indexes.md +19 -19
  41. data/docs/architecture/mvcc.md +19 -19
  42. data/docs/architecture/overview.md +13 -13
  43. data/docs/architecture/pages.md +11 -11
  44. data/docs/architecture/production-roadmap.md +82 -82
  45. data/docs/architecture/query-planner.md +20 -20
  46. data/docs/architecture/recovery.md +18 -18
  47. data/docs/architecture/sql-engine.md +12 -12
  48. data/docs/architecture/storage-engine.md +14 -14
  49. data/docs/architecture/transactions.md +10 -10
  50. data/docs/architecture/wal.md +28 -28
  51. data/docs/cli-cheatsheet.md +98 -98
  52. data/docs/cli.md +275 -275
  53. data/docs/contributing/architecture.md +9 -9
  54. data/docs/contributing/benchmarking.md +14 -14
  55. data/docs/contributing/development.md +16 -16
  56. data/docs/contributing/release-process.md +49 -49
  57. data/docs/contributing/testing.md +16 -16
  58. data/docs/debugging.md +229 -229
  59. data/docs/developer/branching.md +10 -10
  60. data/docs/developer/database-diff.md +10 -10
  61. data/docs/developer/local-development.md +17 -17
  62. data/docs/developer/snapshots.md +9 -9
  63. data/docs/developer/temporal-data.md +10 -10
  64. data/docs/developer-guide.md +297 -297
  65. data/docs/getting-started/first-database.md +16 -16
  66. data/docs/getting-started/first-query.md +13 -13
  67. data/docs/getting-started/installation.md +19 -19
  68. data/docs/getting-started/local-to-production.md +300 -300
  69. data/docs/getting-started/quickstart.md +17 -17
  70. data/docs/getting-started/rails.md +16 -16
  71. data/docs/hardening_backlog.md +93 -93
  72. data/docs/lessons-learned.md +112 -112
  73. data/docs/operations/backups.md +33 -33
  74. data/docs/operations/disaster-recovery.md +31 -31
  75. data/docs/operations/failover.md +30 -30
  76. data/docs/operations/monitoring.md +25 -25
  77. data/docs/operations/production-guide.md +295 -295
  78. data/docs/operations/production-runbook.md +45 -45
  79. data/docs/operations/replication.md +33 -33
  80. data/docs/operations/restore.md +6 -6
  81. data/docs/operations/runbook.md +34 -34
  82. data/docs/operations/upgrades.md +14 -14
  83. data/docs/operations/workload-testing.md +17 -17
  84. data/docs/production-readiness.md +118 -118
  85. data/docs/production_validation.md +150 -150
  86. data/docs/rails/active-record.md +11 -11
  87. data/docs/rails/compatibility-guide.md +90 -90
  88. data/docs/rails/database-yml.md +92 -92
  89. data/docs/rails/installation.md +17 -17
  90. data/docs/rails/migrations.md +17 -17
  91. data/docs/rails/production.md +82 -82
  92. data/docs/rails/troubleshooting.md +18 -18
  93. data/docs/release.md +25 -25
  94. data/docs/server/architecture.md +10 -10
  95. data/docs/server/authentication.md +10 -10
  96. data/docs/server/configuration.md +16 -16
  97. data/docs/server/connection-pooling.md +10 -10
  98. data/docs/server/deployment.md +10 -10
  99. data/docs/server/protocol.md +12 -12
  100. data/docs/sql/compatibility-guide.md +82 -82
  101. data/docs/sql/compatibility.md +39 -39
  102. data/docs/sql/data-types.md +10 -10
  103. data/docs/sql/functions.md +9 -9
  104. data/docs/sql/joins.md +9 -9
  105. data/docs/sql/operators.md +9 -9
  106. data/docs/sql/sqlite-compatibility.md +21 -21
  107. data/docs/sql/syntax.md +10 -10
  108. data/docs/sql/transactions.md +10 -10
  109. data/docs/troubleshooting.md +244 -244
  110. data/lessons/01-foundations.md +73 -0
  111. data/lessons/02-local-development.md +121 -0
  112. data/lessons/03-embedded-rubydb.md +99 -0
  113. data/lessons/04-rails-complex-apps.md +138 -0
  114. data/lessons/05-rubydb-production-server.md +237 -0
  115. data/lessons/06-postgresql-massive-apps.md +96 -0
  116. data/lessons/07-hybrid-microservices.md +94 -0
  117. data/lessons/08-migrations-backups-recovery.md +86 -0
  118. data/lessons/09-observability-security-scale.md +87 -0
  119. data/lessons/10-release-readiness.md +125 -0
  120. data/lib/rubydb/backup/archive.rb +332 -334
  121. data/lib/rubydb/backup/backup.rb +400 -401
  122. data/lib/rubydb/backup/incremental.rb +349 -353
  123. data/lib/rubydb/backup/restore.rb +289 -290
  124. data/lib/rubydb/backup/snapshot.rb +265 -267
  125. data/lib/rubydb/backup/verification.rb +276 -279
  126. data/lib/rubydb/branching/branch.rb +181 -181
  127. data/lib/rubydb/branching/branch_manager.rb +307 -311
  128. data/lib/rubydb/branching/branch_metadata.rb +140 -140
  129. data/lib/rubydb/branching/checkout.rb +165 -166
  130. data/lib/rubydb/branching/copy_on_write.rb +272 -272
  131. data/lib/rubydb/branching/diff.rb +137 -138
  132. data/lib/rubydb/branching/merge.rb +282 -285
  133. data/lib/rubydb/build_info.rb +15 -15
  134. data/lib/rubydb/catalog/catalog.rb +391 -391
  135. data/lib/rubydb/catalog/column.rb +112 -112
  136. data/lib/rubydb/catalog/constraint.rb +180 -180
  137. data/lib/rubydb/catalog/database.rb +184 -184
  138. data/lib/rubydb/catalog/index.rb +97 -97
  139. data/lib/rubydb/catalog/schema.rb +103 -103
  140. data/lib/rubydb/catalog/sequence.rb +90 -90
  141. data/lib/rubydb/catalog/system_catalog.rb +698 -698
  142. data/lib/rubydb/catalog/table.rb +178 -178
  143. data/lib/rubydb/catalog/trigger.rb +102 -102
  144. data/lib/rubydb/catalog/view.rb +66 -66
  145. data/lib/rubydb/cli/application.rb +163 -163
  146. data/lib/rubydb/cli/commands/backup.rb +80 -81
  147. data/lib/rubydb/cli/commands/branch.rb +72 -72
  148. data/lib/rubydb/cli/commands/checkout.rb +54 -54
  149. data/lib/rubydb/cli/commands/create.rb +58 -58
  150. data/lib/rubydb/cli/commands/diff.rb +76 -77
  151. data/lib/rubydb/cli/commands/doctor.rb +74 -74
  152. data/lib/rubydb/cli/commands/drop.rb +57 -57
  153. data/lib/rubydb/cli/commands/init.rb +101 -102
  154. data/lib/rubydb/cli/commands/inspect.rb +95 -95
  155. data/lib/rubydb/cli/commands/merge.rb +63 -63
  156. data/lib/rubydb/cli/commands/migrate.rb +62 -62
  157. data/lib/rubydb/cli/commands/restart.rb +42 -39
  158. data/lib/rubydb/cli/commands/restore.rb +121 -121
  159. data/lib/rubydb/cli/commands/shell.rb +365 -365
  160. data/lib/rubydb/cli/commands/snapshot.rb +79 -79
  161. data/lib/rubydb/cli/commands/start.rb +88 -82
  162. data/lib/rubydb/cli/commands/status.rb +96 -92
  163. data/lib/rubydb/cli/commands/stop.rb +47 -47
  164. data/lib/rubydb/cli/commands/vacuum.rb +58 -58
  165. data/lib/rubydb/cli/formatter.rb +221 -221
  166. data/lib/rubydb/cli/output.rb +168 -168
  167. data/lib/rubydb/client/client.rb +302 -304
  168. data/lib/rubydb/client/connection.rb +414 -415
  169. data/lib/rubydb/client/connection_pool.rb +168 -168
  170. data/lib/rubydb/client/connection_url.rb +96 -96
  171. data/lib/rubydb/client/prepared_statement.rb +60 -60
  172. data/lib/rubydb/client/result.rb +123 -123
  173. data/lib/rubydb/client/statement.rb +52 -52
  174. data/lib/rubydb/client/transaction.rb +130 -130
  175. data/lib/rubydb/concurrency/concurrency.rb +19 -19
  176. data/lib/rubydb/concurrency/deadlock_detector.rb +148 -150
  177. data/lib/rubydb/concurrency/latch.rb +101 -101
  178. data/lib/rubydb/concurrency/lock_graph.rb +163 -165
  179. data/lib/rubydb/concurrency/mutex.rb +181 -183
  180. data/lib/rubydb/concurrency/rw_lock.rb +180 -180
  181. data/lib/rubydb/concurrency/scheduler.rb +248 -250
  182. data/lib/rubydb/concurrency/worker_pool.rb +145 -143
  183. data/lib/rubydb/configuration/config.rb +170 -170
  184. data/lib/rubydb/configuration/defaults.rb +179 -179
  185. data/lib/rubydb/configuration/environment.rb +152 -152
  186. data/lib/rubydb/configuration/parser.rb +185 -185
  187. data/lib/rubydb/configuration/validation.rb +221 -221
  188. data/lib/rubydb/constants.rb +74 -74
  189. data/lib/rubydb/constraints/check.rb +181 -181
  190. data/lib/rubydb/constraints/constraint.rb +101 -101
  191. data/lib/rubydb/constraints/foreign_key.rb +130 -130
  192. data/lib/rubydb/constraints/not_null.rb +64 -64
  193. data/lib/rubydb/constraints/primary_key.rb +99 -99
  194. data/lib/rubydb/constraints/unique.rb +106 -108
  195. data/lib/rubydb/constraints/validator.rb +349 -350
  196. data/lib/rubydb/errors/authentication_error.rb +10 -10
  197. data/lib/rubydb/errors/authorization_error.rb +23 -23
  198. data/lib/rubydb/errors/client_error.rb +10 -10
  199. data/lib/rubydb/errors/configuration_error.rb +10 -10
  200. data/lib/rubydb/errors/connection_error.rb +10 -10
  201. data/lib/rubydb/errors/constraint_error.rb +23 -23
  202. data/lib/rubydb/errors/corruption_error.rb +10 -10
  203. data/lib/rubydb/errors/database_error.rb +10 -10
  204. data/lib/rubydb/errors/error.rb +20 -20
  205. data/lib/rubydb/errors/execution_error.rb +10 -10
  206. data/lib/rubydb/errors/parser_error.rb +10 -10
  207. data/lib/rubydb/errors/recovery_error.rb +10 -10
  208. data/lib/rubydb/errors/replication_error.rb +10 -10
  209. data/lib/rubydb/errors/server_error.rb +6 -6
  210. data/lib/rubydb/errors/storage_error.rb +10 -10
  211. data/lib/rubydb/errors/transaction_error.rb +10 -10
  212. data/lib/rubydb/execution/aggregate_executor.rb +134 -138
  213. data/lib/rubydb/execution/delete_executor.rb +110 -112
  214. data/lib/rubydb/execution/distinct_executor.rb +131 -135
  215. data/lib/rubydb/execution/executor.rb +1182 -1188
  216. data/lib/rubydb/execution/expression.rb +191 -193
  217. data/lib/rubydb/execution/index_scan.rb +142 -142
  218. data/lib/rubydb/execution/insert_executor.rb +215 -217
  219. data/lib/rubydb/execution/join_executor.rb +243 -249
  220. data/lib/rubydb/execution/limit_executor.rb +83 -85
  221. data/lib/rubydb/execution/optimizer.rb +227 -215
  222. data/lib/rubydb/execution/plan.rb +355 -353
  223. data/lib/rubydb/execution/planner.rb +542 -536
  224. data/lib/rubydb/execution/predicate.rb +235 -235
  225. data/lib/rubydb/execution/scan.rb +49 -49
  226. data/lib/rubydb/execution/sequential_scan.rb +63 -63
  227. data/lib/rubydb/execution/sort_executor.rb +180 -185
  228. data/lib/rubydb/execution/update_executor.rb +160 -162
  229. data/lib/rubydb/functions/aggregate.rb +70 -70
  230. data/lib/rubydb/functions/date_functions.rb +274 -278
  231. data/lib/rubydb/functions/function.rb +85 -85
  232. data/lib/rubydb/functions/json_functions.rb +231 -215
  233. data/lib/rubydb/functions/numeric_functions.rb +346 -346
  234. data/lib/rubydb/functions/scalar.rb +52 -52
  235. data/lib/rubydb/functions/string_functions.rb +383 -383
  236. data/lib/rubydb/functions/system_functions.rb +258 -246
  237. data/lib/rubydb/history/as_of.rb +238 -238
  238. data/lib/rubydb/history/change.rb +105 -105
  239. data/lib/rubydb/history/history.rb +131 -131
  240. data/lib/rubydb/history/history_manager.rb +228 -229
  241. data/lib/rubydb/history/temporal_query.rb +202 -202
  242. data/lib/rubydb/history/timeline.rb +144 -144
  243. data/lib/rubydb/indexes/btree.rb +186 -186
  244. data/lib/rubydb/indexes/btree_cursor.rb +258 -258
  245. data/lib/rubydb/indexes/btree_node.rb +384 -385
  246. data/lib/rubydb/indexes/hash_index.rb +150 -150
  247. data/lib/rubydb/indexes/index.rb +71 -71
  248. data/lib/rubydb/indexes/index_manager.rb +408 -406
  249. data/lib/rubydb/indexes/index_scan.rb +466 -470
  250. data/lib/rubydb/migrations/migration.rb +253 -254
  251. data/lib/rubydb/migrations/migration_lock.rb +146 -146
  252. data/lib/rubydb/migrations/migration_manager.rb +187 -176
  253. data/lib/rubydb/migrations/migration_version.rb +71 -71
  254. data/lib/rubydb/migrations/schema_diff.rb +211 -211
  255. data/lib/rubydb/migrations/schema_version.rb +64 -64
  256. data/lib/rubydb/monitoring/events.rb +155 -160
  257. data/lib/rubydb/monitoring/health.rb +216 -222
  258. data/lib/rubydb/monitoring/logger.rb +188 -193
  259. data/lib/rubydb/monitoring/metrics.rb +363 -359
  260. data/lib/rubydb/monitoring/performance.rb +176 -176
  261. data/lib/rubydb/monitoring/statistics.rb +168 -170
  262. data/lib/rubydb/mvcc/garbage_collector.rb +199 -199
  263. data/lib/rubydb/mvcc/mvcc.rb +16 -16
  264. data/lib/rubydb/mvcc/snapshot.rb +146 -147
  265. data/lib/rubydb/mvcc/vacuum.rb +180 -180
  266. data/lib/rubydb/mvcc/version.rb +106 -106
  267. data/lib/rubydb/mvcc/version_store.rb +396 -398
  268. data/lib/rubydb/mvcc/visibility.rb +107 -109
  269. data/lib/rubydb/protocol/capabilities.rb +125 -125
  270. data/lib/rubydb/protocol/decoder.rb +142 -145
  271. data/lib/rubydb/protocol/encoder.rb +131 -136
  272. data/lib/rubydb/protocol/handshake.rb +306 -305
  273. data/lib/rubydb/protocol/message.rb +121 -121
  274. data/lib/rubydb/protocol/parameter_binder.rb +101 -0
  275. data/lib/rubydb/protocol/protocol.rb +276 -277
  276. data/lib/rubydb/protocol/version.rb +54 -54
  277. data/lib/rubydb/rails/adapter.rb +245 -239
  278. data/lib/rubydb/rails/connection.rb +312 -314
  279. data/lib/rubydb/rails/database_statements.rb +122 -122
  280. data/lib/rubydb/rails/migration.rb +131 -131
  281. data/lib/rubydb/rails/quoting.rb +109 -109
  282. data/lib/rubydb/rails/result.rb +117 -117
  283. data/lib/rubydb/rails/schema_statements.rb +339 -339
  284. data/lib/rubydb/rails/transaction.rb +105 -105
  285. data/lib/rubydb/rails/type.rb +126 -126
  286. data/lib/rubydb/recovery/checkpoint.rb +261 -257
  287. data/lib/rubydb/recovery/consistency.rb +457 -467
  288. data/lib/rubydb/recovery/corruption_detector.rb +5 -5
  289. data/lib/rubydb/recovery/crash_recovery.rb +381 -387
  290. data/lib/rubydb/recovery/recovery_manager.rb +204 -206
  291. data/lib/rubydb/recovery/redo.rb +235 -237
  292. data/lib/rubydb/recovery/undo.rb +204 -206
  293. data/lib/rubydb/replication/failover.rb +5 -5
  294. data/lib/rubydb/replication/fencing.rb +63 -63
  295. data/lib/rubydb/replication/primary.rb +461 -450
  296. data/lib/rubydb/replication/replica.rb +382 -384
  297. data/lib/rubydb/replication/replication_log.rb +194 -200
  298. data/lib/rubydb/replication/replication_manager.rb +307 -308
  299. data/lib/rubydb/replication/replication_slot.rb +293 -295
  300. data/lib/rubydb/replication/replication_stream.rb +198 -201
  301. data/lib/rubydb/rubydb.rb +564 -560
  302. data/lib/rubydb/security/access_control.rb +252 -254
  303. data/lib/rubydb/security/audit_log.rb +209 -213
  304. data/lib/rubydb/security/authentication.rb +302 -302
  305. data/lib/rubydb/security/authorization.rb +282 -282
  306. data/lib/rubydb/security/credentials.rb +192 -196
  307. data/lib/rubydb/security/password.rb +205 -215
  308. data/lib/rubydb/security/permissions.rb +74 -74
  309. data/lib/rubydb/security/role.rb +99 -101
  310. data/lib/rubydb/security/user.rb +86 -86
  311. data/lib/rubydb/server/connection.rb +383 -366
  312. data/lib/rubydb/server/connection_pool.rb +193 -193
  313. data/lib/rubydb/server/lifecycle.rb +227 -228
  314. data/lib/rubydb/server/listener.rb +139 -136
  315. data/lib/rubydb/server/request_handler.rb +277 -276
  316. data/lib/rubydb/server/server.rb +363 -364
  317. data/lib/rubydb/server/session.rb +371 -369
  318. data/lib/rubydb/server/worker.rb +206 -210
  319. data/lib/rubydb/server/worker_pool.rb +168 -168
  320. data/lib/rubydb/sql/ast/alter_table.rb +169 -169
  321. data/lib/rubydb/sql/ast/begin_transaction.rb +47 -47
  322. data/lib/rubydb/sql/ast/commit.rb +37 -37
  323. data/lib/rubydb/sql/ast/constraint.rb +92 -83
  324. data/lib/rubydb/sql/ast/create_database.rb +41 -41
  325. data/lib/rubydb/sql/ast/create_index.rb +61 -61
  326. data/lib/rubydb/sql/ast/create_schema.rb +52 -52
  327. data/lib/rubydb/sql/ast/create_table.rb +187 -187
  328. data/lib/rubydb/sql/ast/delete.rb +54 -54
  329. data/lib/rubydb/sql/ast/drop_database.rb +41 -41
  330. data/lib/rubydb/sql/ast/drop_index.rb +41 -41
  331. data/lib/rubydb/sql/ast/drop_schema.rb +49 -49
  332. data/lib/rubydb/sql/ast/drop_table.rb +49 -49
  333. data/lib/rubydb/sql/ast/explain.rb +64 -64
  334. data/lib/rubydb/sql/ast/expression.rb +617 -604
  335. data/lib/rubydb/sql/ast/insert.rb +66 -66
  336. data/lib/rubydb/sql/ast/node.rb +42 -42
  337. data/lib/rubydb/sql/ast/rollback.rb +63 -63
  338. data/lib/rubydb/sql/ast/savepoint.rb +59 -59
  339. data/lib/rubydb/sql/ast/select.rb +88 -88
  340. data/lib/rubydb/sql/ast/set_operation.rb +22 -20
  341. data/lib/rubydb/sql/ast/trigger.rb +35 -29
  342. data/lib/rubydb/sql/ast/update.rb +88 -88
  343. data/lib/rubydb/sql/ast/vacuum.rb +19 -19
  344. data/lib/rubydb/sql/ast/view.rb +38 -32
  345. data/lib/rubydb/sql/ast/with.rb +32 -32
  346. data/lib/rubydb/sql/grammar.rb +86 -86
  347. data/lib/rubydb/sql/keywords.rb +156 -156
  348. data/lib/rubydb/sql/lexer.rb +209 -214
  349. data/lib/rubydb/sql/operators.rb +100 -100
  350. data/lib/rubydb/sql/parser.rb +1167 -1170
  351. data/lib/rubydb/sql/planner/analyzer.rb +283 -302
  352. data/lib/rubydb/sql/planner/binder.rb +537 -550
  353. data/lib/rubydb/sql/planner/type_checker.rb +427 -431
  354. data/lib/rubydb/sql/token.rb +210 -210
  355. data/lib/rubydb/storage/buffer_frame.rb +44 -44
  356. data/lib/rubydb/storage/buffer_pool.rb +155 -155
  357. data/lib/rubydb/storage/database_lock.rb +74 -74
  358. data/lib/rubydb/storage/deserializer.rb +332 -342
  359. data/lib/rubydb/storage/engine.rb +2347 -2330
  360. data/lib/rubydb/storage/file_manager.rb +191 -187
  361. data/lib/rubydb/storage/free_space_map.rb +79 -81
  362. data/lib/rubydb/storage/page.rb +92 -94
  363. data/lib/rubydb/storage/page_allocator.rb +852 -855
  364. data/lib/rubydb/storage/page_header.rb +63 -67
  365. data/lib/rubydb/storage/page_manager.rb +127 -131
  366. data/lib/rubydb/storage/record.rb +58 -58
  367. data/lib/rubydb/storage/row.rb +78 -78
  368. data/lib/rubydb/storage/serializer.rb +51 -51
  369. data/lib/rubydb/storage/storage_layout.rb +151 -151
  370. data/lib/rubydb/storage/storage_manager.rb +114 -114
  371. data/lib/rubydb/storage/tuple.rb +458 -461
  372. data/lib/rubydb/storage/visibility_map.rb +964 -973
  373. data/lib/rubydb/transactions/commit_manager.rb +219 -220
  374. data/lib/rubydb/transactions/isolation.rb +98 -98
  375. data/lib/rubydb/transactions/lock.rb +76 -76
  376. data/lib/rubydb/transactions/lock_manager.rb +359 -362
  377. data/lib/rubydb/transactions/savepoint.rb +142 -143
  378. data/lib/rubydb/transactions/transaction.rb +214 -215
  379. data/lib/rubydb/transactions/transaction_id.rb +84 -84
  380. data/lib/rubydb/transactions/transaction_log.rb +256 -257
  381. data/lib/rubydb/transactions/transaction_manager.rb +434 -435
  382. data/lib/rubydb/types/bigint.rb +36 -36
  383. data/lib/rubydb/types/blob.rb +37 -37
  384. data/lib/rubydb/types/boolean.rb +34 -34
  385. data/lib/rubydb/types/date.rb +39 -39
  386. data/lib/rubydb/types/decimal.rb +48 -48
  387. data/lib/rubydb/types/float.rb +34 -34
  388. data/lib/rubydb/types/integer.rb +36 -36
  389. data/lib/rubydb/types/json.rb +41 -41
  390. data/lib/rubydb/types/null.rb +34 -34
  391. data/lib/rubydb/types/smallint.rb +36 -36
  392. data/lib/rubydb/types/text.rb +37 -37
  393. data/lib/rubydb/types/time.rb +46 -46
  394. data/lib/rubydb/types/timestamp.rb +39 -39
  395. data/lib/rubydb/types/type.rb +119 -119
  396. data/lib/rubydb/types/uuid.rb +47 -47
  397. data/lib/rubydb/types/varchar.rb +37 -37
  398. data/lib/rubydb/version.rb +32 -32
  399. data/lib/rubydb/wal/archive.rb +190 -193
  400. data/lib/rubydb/wal/checkpoint.rb +181 -183
  401. data/lib/rubydb/wal/lsn.rb +94 -94
  402. data/lib/rubydb/wal/reader.rb +259 -260
  403. data/lib/rubydb/wal/record.rb +105 -105
  404. data/lib/rubydb/wal/segment.rb +193 -193
  405. data/lib/rubydb/wal/wal.rb +480 -452
  406. data/lib/rubydb/wal/writer.rb +236 -236
  407. data/lib/rubydb.rb +7 -7
  408. data/packaging/docker/docker-compose.failover.yml +43 -43
  409. data/packaging/homebrew/rubydb.rb +19 -19
  410. data/rubydb.gemspec +60 -57
  411. data/scripts/benchmark +7 -7
  412. data/scripts/durability_drill +37 -37
  413. data/scripts/fuzz +63 -63
  414. data/scripts/release +45 -40
  415. data/scripts/release_check +43 -43
  416. data/scripts/replication_failover_drill +268 -250
  417. data/scripts/replication_network_failover_drill +287 -255
  418. data/scripts/restore_drill +45 -45
  419. data/scripts/security +45 -0
  420. metadata +54 -1
data/docs/README.md CHANGED
@@ -1,71 +1,75 @@
1
- # RubyDB documentation index
2
-
3
- Start here:
4
-
5
- - [Developer guide](developer-guide.md)
6
- - [Troubleshooting](troubleshooting.md)
7
- - [Debugging playbook](debugging.md)
8
- - [Production operations guide](operations/production-guide.md)
9
- - [Quick start](getting-started/quickstart.md)
10
- - [Local development to production](getting-started/local-to-production.md)
11
- - [First database](getting-started/first-database.md)
12
- - [First query](getting-started/first-query.md)
13
- - [Rails installation](rails/installation.md)
14
- - [Rails production guidance](rails/production.md)
15
- - [SQLite compatibility profile](sql/sqlite-compatibility.md)
16
- - [SQL compatibility contract](sql/compatibility.md)
17
- - [SQL compatibility guide](sql/compatibility-guide.md)
18
- - [Rails compatibility guide](rails/compatibility-guide.md)
19
- - [Production validation](production_validation.md)
20
- - [Production-readiness audit](production-readiness.md)
21
- - [Production runbook](operations/production-runbook.md)
22
- - [CLI guide](cli.md)
23
- - [CLI cheat sheet](cli-cheatsheet.md)
24
- - [Disaster recovery](operations/disaster-recovery.md)
25
- - [Monitoring and alerting](operations/monitoring.md)
26
- - [Workload testing](operations/workload-testing.md)
27
- - [Release checklist](release.md)
28
- - [Lessons learned](lessons-learned.md)
29
-
30
- Architecture and development:
31
-
32
- - [Current-state audit](architecture/current-state.md)
33
- - [Production roadmap](architecture/production-roadmap.md)
34
- - [Architecture overview](architecture/overview.md)
35
- - [Storage engine](architecture/storage-engine.md)
36
- - [Transactions](architecture/transactions.md)
37
- - [MVCC](architecture/mvcc.md)
38
- - [WAL](architecture/wal.md)
39
- - [Query planner](architecture/query-planner.md)
40
- - [Testing guide](contributing/testing.md)
41
- - [Release process](contributing/release-process.md)
42
- - [Branching](developer/branching.md)
43
-
44
- The root [README](../README.md) is the public project overview. The current
45
- production claim is intentionally bounded by the tested features and the
46
- deployment-specific validation described above.
47
-
48
- ## Documentation map
49
-
50
- Use the guides for the complete workflow and the smaller pages for focused
51
- reference:
52
-
53
- * `getting-started/` gets a new Ruby or Rails application running.
54
- * `sql/` defines syntax, types, expressions, transactions, joins, functions,
55
- and compatibility boundaries.
56
- * `architecture/` explains storage, WAL, recovery, MVCC, indexes, planning,
57
- execution, server ownership, and replication.
58
- * `developer/` covers local development, snapshots, branches, temporal data,
59
- database diffs, and implementation workflows.
60
- * `rails/` covers installation, configuration, migrations, production use,
61
- compatibility, and troubleshooting.
62
- * `server/` covers deployment, authentication, TLS, pooling, protocol, and
63
- operational behavior.
64
- * `operations/` covers backups, restore drills, monitoring, workload tests,
65
- production procedures, and incident response.
66
- * `contributing/` and the root policy files cover testing, release, governance,
67
- support, security, and contribution requirements.
68
-
69
- If a topic page and an implementation disagree, treat executable tests and the
70
- documented compatibility contract as the source of truth, then open an issue
71
- to reconcile the documentation.
1
+ # RubyDB documentation index
2
+
3
+ Start here:
4
+
5
+ - [Developer guide](developer-guide.md)
6
+ - [Troubleshooting](troubleshooting.md)
7
+ - [Debugging playbook](debugging.md)
8
+ - [Production operations guide](operations/production-guide.md)
9
+ - [Quick start](getting-started/quickstart.md)
10
+ - [Local development to production](getting-started/local-to-production.md)
11
+ - [First database](getting-started/first-database.md)
12
+ - [First query](getting-started/first-query.md)
13
+ - [Rails installation](rails/installation.md)
14
+ - [Rails production guidance](rails/production.md)
15
+ - [SQLite compatibility profile](sql/sqlite-compatibility.md)
16
+ - [SQL compatibility contract](sql/compatibility.md)
17
+ - [SQL compatibility guide](sql/compatibility-guide.md)
18
+ - [Rails compatibility guide](rails/compatibility-guide.md)
19
+ - [Python adapter](../adapters/python/README.md)
20
+ - [Production validation](production_validation.md)
21
+ - [Production-readiness audit](production-readiness.md)
22
+ - [Production runbook](operations/production-runbook.md)
23
+ - [CLI guide](cli.md)
24
+ - [CLI cheat sheet](cli-cheatsheet.md)
25
+ - [Disaster recovery](operations/disaster-recovery.md)
26
+ - [Monitoring and alerting](operations/monitoring.md)
27
+ - [Workload testing](operations/workload-testing.md)
28
+ - [Release checklist](release.md)
29
+ - [Lessons learned](lessons-learned.md)
30
+ - [Ten-lesson production journey](../lessons/01-foundations.md)
31
+
32
+ Architecture and development:
33
+
34
+ - [Current-state audit](architecture/current-state.md)
35
+ - [Production roadmap](architecture/production-roadmap.md)
36
+ - [Architecture overview](architecture/overview.md)
37
+ - [Storage engine](architecture/storage-engine.md)
38
+ - [Transactions](architecture/transactions.md)
39
+ - [MVCC](architecture/mvcc.md)
40
+ - [WAL](architecture/wal.md)
41
+ - [Query planner](architecture/query-planner.md)
42
+ - [Testing guide](contributing/testing.md)
43
+ - [Release process](contributing/release-process.md)
44
+ - [Branching](developer/branching.md)
45
+
46
+ The root [README](../README.md) is the public project overview. The current
47
+ production claim is intentionally bounded by the tested features and the
48
+ deployment-specific validation described above.
49
+
50
+ ## Documentation map
51
+
52
+ Use the guides for the complete workflow and the smaller pages for focused
53
+ reference:
54
+
55
+ * `getting-started/` gets a new Ruby or Rails application running.
56
+ * `sql/` defines syntax, types, expressions, transactions, joins, functions,
57
+ and compatibility boundaries.
58
+ * `architecture/` explains storage, WAL, recovery, MVCC, indexes, planning,
59
+ execution, server ownership, and replication.
60
+ * `developer/` covers local development, snapshots, branches, temporal data,
61
+ database diffs, and implementation workflows.
62
+ * `rails/` covers installation, configuration, migrations, production use,
63
+ compatibility, and troubleshooting.
64
+ * `server/` covers deployment, authentication, TLS, pooling, protocol, and
65
+ operational behavior.
66
+ * `operations/` covers backups, restore drills, monitoring, workload tests,
67
+ production procedures, and incident response.
68
+ * `../lessons/` is a guided beginner-to-production journey that combines local
69
+ RubyDB, Rails, PostgreSQL, and hybrid microservice decisions.
70
+ * `contributing/` and the root policy files cover testing, release, governance,
71
+ support, security, and contribution requirements.
72
+
73
+ If a topic page and an implementation disagree, treat executable tests and the
74
+ documented compatibility contract as the source of truth, then open an issue
75
+ to reconcile the documentation.
@@ -1,14 +1,14 @@
1
- # Concurrency architecture
2
-
3
- Threads coordinate through locks, transaction ownership, the lock manager,
4
- buffer-pool synchronization, and server worker/connection limits. Transactions
5
- have connection-scoped state in server mode; the embedded owner remains one
6
- process.
7
-
8
- The lock manager detects wait-for cycles and selects a victim. The transaction
9
- manager rolls the victim back, releases its locks, and returns a retryable
10
- deadlock error. Timeouts and cancellation are distinct: a timeout stops a
11
- request deadline, while wire cancellation actively marks an in-flight request.
12
-
13
- Validate concurrency with the production soak, multi-process workload, latency
14
- percentiles, deadlock checks, resource limits, and durable reopen verification.
1
+ # Concurrency architecture
2
+
3
+ Threads coordinate through locks, transaction ownership, the lock manager,
4
+ buffer-pool synchronization, and server worker/connection limits. Transactions
5
+ have connection-scoped state in server mode; the embedded owner remains one
6
+ process.
7
+
8
+ The lock manager detects wait-for cycles and selects a victim. The transaction
9
+ manager rolls the victim back, releases its locks, and returns a retryable
10
+ deadlock error. Timeouts and cancellation are distinct: a timeout stops a
11
+ request deadline, while wire cancellation actively marks an in-flight request.
12
+
13
+ Validate concurrency with the production soak, multi-process workload, latency
14
+ percentiles, deadlock checks, resource limits, and durable reopen verification.
@@ -1,125 +1,125 @@
1
- # RubyDB current-state audit
2
-
3
- > This document is a dated architecture audit, not a statement that every
4
- > finding remains open. For current validation evidence, see
5
- > [production validation](../production_validation.md) and the latest CI runs.
6
-
7
- ## Executive summary
8
-
9
- RubyDB is a broad database project with integrated storage, WAL, MVCC,
10
- transactions, catalog, security, server, replication, backup, Rails, and
11
- monitoring paths. The implementation now has substantial focused validation,
12
- but several deployment-level capabilities remain environment-specific or
13
- intentionally bounded.
14
-
15
- The primary risk is not that the code is absent; it is that multiple components exist without being integrated into a consistent, correct, durable engine. Existing code needs additional hardening before it can safely store user data.
16
-
17
- ## Repository audit
18
-
19
- ### Runtime and package state
20
-
21
- - Ruby runtime in the development container: Ruby 3.4.7
22
- - Package metadata targets Ruby >= 3.3.0; CI covers Ruby 3.3 and 3.4 on Linux,
23
- macOS, and Windows.
24
- - The top-level library entrypoint and gem packaging are covered by
25
- `require "rubydb"` and release-artifact tests.
26
- - The project uses Bundler and RSpec as the base test stack.
27
- - The repository is organized around a large `lib/rubydb` module tree and includes docs, examples, chaos, and fuzz directories.
28
-
29
- ### Architectural reality
30
-
31
- The project already contains modules for:
32
-
33
- - storage engine and page manager
34
- - buffer pool and file management
35
- - WAL and recovery components
36
- - transactions and MVCC
37
- - catalog, indexes, and constraints
38
- - SQL parser and planner
39
- - server and protocol layers
40
- - security and authorization
41
- - replication, backups, and monitoring
42
- - Rails integration
43
-
44
- The remaining risk is uneven validation breadth: many capabilities are proven
45
- by focused tests, but multi-host operations, capacity limits, and complete
46
- external-database dialect compatibility are not yet certified.
47
-
48
- ## High-risk findings
49
-
50
- ### 1. Library loading (resolved)
51
-
52
- The standard Ruby consumer path is now tested:
53
-
54
- ```ruby
55
- require "rubydb"
56
- ```
57
-
58
- Release metadata and the local gem build verify that this loads correctly.
59
-
60
- ### 2. Storage layer (partially resolved)
61
-
62
- The storage engine now has page validation, corruption detection, WAL/recovery,
63
- fsync paths, restart tests, subprocess crash tests, filesystem fault injection,
64
- and compaction/reopen coverage. Real disk-quota and power-loss validation on
65
- deployment filesystems remains open.
66
-
67
- ### 3. Transaction and MVCC semantics (validated foundation)
68
-
69
- The transaction manager and visibility map exist, transaction rollback restores
70
- update/delete before-images, and read-committed, repeatable-read, serializable
71
- conflict, deadlock, and concurrency workload paths have regression coverage.
72
- Distributed isolation and deployment-specific contention limits remain open.
73
-
74
- ### 4. SQL engine is a documented subset
75
-
76
- The parser, planner, and executor are covered through end-to-end tests for the
77
- documented RubyDB SQL subset, including joins, aggregates, CTEs, subqueries,
78
- set operations, upserts, and window functions. A common SQLite-style profile
79
- is tested for new Ruby/Rails applications; full PostgreSQL/MySQL/SQLite dialect
80
- and file-format compatibility remains intentionally out of scope for this
81
- phase.
82
-
83
- ### 5. Security (validated foundation)
84
-
85
- Authentication, TLS, SCRAM, authorization, framing limits, deadlines, and
86
- resource-limit startup checks are tied to server/query execution and tested.
87
- An independent security review and production certificate/secret lifecycle
88
- validation remain open.
89
-
90
- ### 6. Production posture is intentionally bounded
91
-
92
- The README and production documents now distinguish tested behavior from
93
- deployment work. RubyDB still needs environment-specific capacity, filesystem,
94
- multi-host failover, certificate lifecycle, and independent security validation
95
- before broader production claims are appropriate.
96
-
97
- ## Current production posture
98
-
99
- RubyDB is best understood as:
100
-
101
- - a serious Ruby database foundation with repeatable local validation,
102
- - suitable for controlled workloads within the documented feature set,
103
- - not a drop-in general-purpose database or automatic high-availability service.
104
-
105
- The correct short-term operating posture is to treat the project as a pre-production, high-potential codebase that needs disciplined validation and narrower, correctness-first milestones.
106
-
107
- ## Immediate action items
108
-
109
- 1. Run the hosted Ruby/Rails/OS compatibility matrix and retain its evidence.
110
- 2. Validate real disk quota, power-loss, and filesystem behavior on deployment targets.
111
- 3. Validate multi-host partition, split-brain, fencing, and operator failover procedures.
112
- 4. Complete an independent security review and certificate/secret rotation drill.
113
- 5. Implement or intentionally reject unsupported features with explicit errors.
114
- 6. Establish workload-specific capacity baselines and release sign-off records.
115
-
116
- The latest local audit passed 270 examples with zero failures. Hosted matrix,
117
- container, filesystem quota/power-loss, multi-host fencing, and independent
118
- security-review evidence must still be retained for a production deployment.
119
-
120
- ## Conclusion
121
-
122
- RubyDB has a substantial production-oriented foundation and repeatable local
123
- validation. Operators must stay within the documented feature set and complete
124
- deployment-specific capacity, filesystem, security, and multi-host failover
125
- validation before entrusting critical data to it.
1
+ # RubyDB current-state audit
2
+
3
+ > This document is a dated architecture audit, not a statement that every
4
+ > finding remains open. For current validation evidence, see
5
+ > [production validation](../production_validation.md) and the latest CI runs.
6
+
7
+ ## Executive summary
8
+
9
+ RubyDB is a broad database project with integrated storage, WAL, MVCC,
10
+ transactions, catalog, security, server, replication, backup, Rails, and
11
+ monitoring paths. The implementation now has substantial focused validation,
12
+ but several deployment-level capabilities remain environment-specific or
13
+ intentionally bounded.
14
+
15
+ The primary risk is not that the code is absent; it is that multiple components exist without being integrated into a consistent, correct, durable engine. Existing code needs additional hardening before it can safely store user data.
16
+
17
+ ## Repository audit
18
+
19
+ ### Runtime and package state
20
+
21
+ - Ruby runtime in the development container: Ruby 3.4.7
22
+ - Package metadata targets Ruby >= 3.3.0; CI covers Ruby 3.3 and 3.4 on Linux,
23
+ macOS, and Windows.
24
+ - The top-level library entrypoint and gem packaging are covered by
25
+ `require "rubydb"` and release-artifact tests.
26
+ - The project uses Bundler and RSpec as the base test stack.
27
+ - The repository is organized around a large `lib/rubydb` module tree and includes docs, examples, chaos, and fuzz directories.
28
+
29
+ ### Architectural reality
30
+
31
+ The project already contains modules for:
32
+
33
+ - storage engine and page manager
34
+ - buffer pool and file management
35
+ - WAL and recovery components
36
+ - transactions and MVCC
37
+ - catalog, indexes, and constraints
38
+ - SQL parser and planner
39
+ - server and protocol layers
40
+ - security and authorization
41
+ - replication, backups, and monitoring
42
+ - Rails integration
43
+
44
+ The remaining risk is uneven validation breadth: many capabilities are proven
45
+ by focused tests, but multi-host operations, capacity limits, and complete
46
+ external-database dialect compatibility are not yet certified.
47
+
48
+ ## High-risk findings
49
+
50
+ ### 1. Library loading (resolved)
51
+
52
+ The standard Ruby consumer path is now tested:
53
+
54
+ ```ruby
55
+ require "rubydb"
56
+ ```
57
+
58
+ Release metadata and the local gem build verify that this loads correctly.
59
+
60
+ ### 2. Storage layer (partially resolved)
61
+
62
+ The storage engine now has page validation, corruption detection, WAL/recovery,
63
+ fsync paths, restart tests, subprocess crash tests, filesystem fault injection,
64
+ and compaction/reopen coverage. Real disk-quota and power-loss validation on
65
+ deployment filesystems remains open.
66
+
67
+ ### 3. Transaction and MVCC semantics (validated foundation)
68
+
69
+ The transaction manager and visibility map exist, transaction rollback restores
70
+ update/delete before-images, and read-committed, repeatable-read, serializable
71
+ conflict, deadlock, and concurrency workload paths have regression coverage.
72
+ Distributed isolation and deployment-specific contention limits remain open.
73
+
74
+ ### 4. SQL engine is a documented subset
75
+
76
+ The parser, planner, and executor are covered through end-to-end tests for the
77
+ documented RubyDB SQL subset, including joins, aggregates, CTEs, subqueries,
78
+ set operations, upserts, and window functions. A common SQLite-style profile
79
+ is tested for new Ruby/Rails applications; full PostgreSQL/MySQL/SQLite dialect
80
+ and file-format compatibility remains intentionally out of scope for this
81
+ phase.
82
+
83
+ ### 5. Security (validated foundation)
84
+
85
+ Authentication, TLS, SCRAM, authorization, framing limits, deadlines, and
86
+ resource-limit startup checks are tied to server/query execution and tested.
87
+ An independent security review and production certificate/secret lifecycle
88
+ validation remain open.
89
+
90
+ ### 6. Production posture is intentionally bounded
91
+
92
+ The README and production documents now distinguish tested behavior from
93
+ deployment work. RubyDB still needs environment-specific capacity, filesystem,
94
+ multi-host failover, certificate lifecycle, and independent security validation
95
+ before broader production claims are appropriate.
96
+
97
+ ## Current production posture
98
+
99
+ RubyDB is best understood as:
100
+
101
+ - a serious Ruby database foundation with repeatable local validation,
102
+ - suitable for controlled workloads within the documented feature set,
103
+ - not a drop-in general-purpose database or automatic high-availability service.
104
+
105
+ The correct short-term operating posture is to treat the project as a pre-production, high-potential codebase that needs disciplined validation and narrower, correctness-first milestones.
106
+
107
+ ## Immediate action items
108
+
109
+ 1. Run the hosted Ruby/Rails/OS compatibility matrix and retain its evidence.
110
+ 2. Validate real disk quota, power-loss, and filesystem behavior on deployment targets.
111
+ 3. Validate multi-host partition, split-brain, fencing, and operator failover procedures.
112
+ 4. Complete an independent security review and certificate/secret rotation drill.
113
+ 5. Implement or intentionally reject unsupported features with explicit errors.
114
+ 6. Establish workload-specific capacity baselines and release sign-off records.
115
+
116
+ The latest local audit passed 270 examples with zero failures. Hosted matrix,
117
+ container, filesystem quota/power-loss, multi-host fencing, and independent
118
+ security-review evidence must still be retained for a production deployment.
119
+
120
+ ## Conclusion
121
+
122
+ RubyDB has a substantial production-oriented foundation and repeatable local
123
+ validation. Operators must stay within the documented feature set and complete
124
+ deployment-specific capacity, filesystem, security, and multi-host failover
125
+ validation before entrusting critical data to it.
@@ -1,25 +1,25 @@
1
- # Execution engine
2
-
3
- The execution pipeline is:
4
-
5
- `SQL text -> lexer -> parser/AST -> binder -> planner -> executor -> result`
6
-
7
- The planner selects scans, index scans, joins, filters, aggregates, sorting,
8
- limits, set operations, CTEs, subqueries, and window operations supported by
9
- the documented dialect. DML is routed through transaction-aware executors and
10
- the engine's durable mutation path.
11
-
12
- Unsupported syntax must produce an explicit parser or execution error. New
13
- operators and functions require type-checking, null behavior, transaction
14
- coverage, and compatibility documentation.
15
-
16
- ## Execution invariants
17
-
18
- The executor must consume a bound plan, check cancellation and deadlines at
19
- bounded points, and return either a complete result or an error that leaves the
20
- transaction in a documented state. DML goes through the same transaction and
21
- WAL path as the Ruby API; a server response must never bypass durable commit.
22
-
23
- When diagnosing a result, compare parser output, bound parameters, selected
24
- plan, row visibility, and final serialization. See the [developer guide](../developer-guide.md)
25
- and [debugging playbook](../debugging.md).
1
+ # Execution engine
2
+
3
+ The execution pipeline is:
4
+
5
+ `SQL text -> lexer -> parser/AST -> binder -> planner -> executor -> result`
6
+
7
+ The planner selects scans, index scans, joins, filters, aggregates, sorting,
8
+ limits, set operations, CTEs, subqueries, and window operations supported by
9
+ the documented dialect. DML is routed through transaction-aware executors and
10
+ the engine's durable mutation path.
11
+
12
+ Unsupported syntax must produce an explicit parser or execution error. New
13
+ operators and functions require type-checking, null behavior, transaction
14
+ coverage, and compatibility documentation.
15
+
16
+ ## Execution invariants
17
+
18
+ The executor must consume a bound plan, check cancellation and deadlines at
19
+ bounded points, and return either a complete result or an error that leaves the
20
+ transaction in a documented state. DML goes through the same transaction and
21
+ WAL path as the Ruby API; a server response must never bypass durable commit.
22
+
23
+ When diagnosing a result, compare parser output, bound parameters, selected
24
+ plan, row visibility, and final serialization. See the [developer guide](../developer-guide.md)
25
+ and [debugging playbook](../debugging.md).
@@ -1,19 +1,19 @@
1
- # Indexes
2
-
3
- RubyDB provides B-tree indexes for supported table columns, including unique
4
- indexes. The executor can use indexes for eligible lookups while preserving
5
- correctness through table visibility and transaction rules.
6
-
7
- Index metadata is persisted with the catalog. Index creation, DML maintenance,
8
- rollback, deep splits, reopen, and failure handling must remain consistent. If
9
- index recovery fails, startup must fail visibly; never silently rebuild or claim
10
- an index is healthy without verification.
11
-
12
- ## Change checklist
13
-
14
- Index changes require coverage for empty and populated tables, duplicate and
15
- null keys, deep splits, deletes, rollback, crash/replay, compaction, and
16
- concurrent readers. Compare an index lookup with a table scan before and after
17
- reopen. A required-index persistence error must abort the owning operation and
18
- be observable by the caller. See [troubleshooting](../troubleshooting.md) for
19
- the safe response to suspected index corruption.
1
+ # Indexes
2
+
3
+ RubyDB provides B-tree indexes for supported table columns, including unique
4
+ indexes. The executor can use indexes for eligible lookups while preserving
5
+ correctness through table visibility and transaction rules.
6
+
7
+ Index metadata is persisted with the catalog. Index creation, DML maintenance,
8
+ rollback, deep splits, reopen, and failure handling must remain consistent. If
9
+ index recovery fails, startup must fail visibly; never silently rebuild or claim
10
+ an index is healthy without verification.
11
+
12
+ ## Change checklist
13
+
14
+ Index changes require coverage for empty and populated tables, duplicate and
15
+ null keys, deep splits, deletes, rollback, crash/replay, compaction, and
16
+ concurrent readers. Compare an index lookup with a table scan before and after
17
+ reopen. A required-index persistence error must abort the owning operation and
18
+ be observable by the caller. See [troubleshooting](../troubleshooting.md) for
19
+ the safe response to suspected index corruption.
@@ -1,19 +1,19 @@
1
- # MVCC support status
2
-
3
- RubyDB currently provides safe READ COMMITTED behavior for the storage engine.
4
- Uncommitted row versions are visible to their owning transaction and hidden
5
- from other transactions. Committed versions become visible after the durable
6
- COMMIT WAL record is written. Deleted physical records remain on pages until
7
- vacuum/compaction and are excluded from ordinary scans.
8
-
9
- REPEATABLE READ uses persisted historical row versions and transaction
10
- snapshots. SERIALIZABLE uses snapshot validation over tracked row read/write
11
- keys and conservative table predicates. A newer committed version intersecting
12
- the dependency set aborts commit with a serialization failure. Table-level
13
- predicate tracking prevents phantoms at the cost of false-positive conflicts;
14
- exact index-range predicate locking is a future optimization.
15
-
16
- The version store is persisted atomically and supports safe-point vacuuming.
17
- Vacuum never removes active versions and retains the newest committed base
18
- version needed by an active reader; older committed history is removable only
19
- when its commit ID precedes the oldest active transaction.
1
+ # MVCC support status
2
+
3
+ RubyDB currently provides safe READ COMMITTED behavior for the storage engine.
4
+ Uncommitted row versions are visible to their owning transaction and hidden
5
+ from other transactions. Committed versions become visible after the durable
6
+ COMMIT WAL record is written. Deleted physical records remain on pages until
7
+ vacuum/compaction and are excluded from ordinary scans.
8
+
9
+ REPEATABLE READ uses persisted historical row versions and transaction
10
+ snapshots. SERIALIZABLE uses snapshot validation over tracked row read/write
11
+ keys and conservative table predicates. A newer committed version intersecting
12
+ the dependency set aborts commit with a serialization failure. Table-level
13
+ predicate tracking prevents phantoms at the cost of false-positive conflicts;
14
+ exact index-range predicate locking is a future optimization.
15
+
16
+ The version store is persisted atomically and supports safe-point vacuuming.
17
+ Vacuum never removes active versions and retains the newest committed base
18
+ version needed by an active reader; older committed history is removable only
19
+ when its commit ID precedes the oldest active transaction.
@@ -1,13 +1,13 @@
1
- # Architecture overview
2
-
3
- RubyDB separates logical query processing from durable state changes. SQL is
4
- lexed into tokens, parsed into an AST, bound against catalog metadata, planned,
5
- and executed against the storage engine. DML is coordinated with transactions
6
- and WAL before durable acknowledgement.
7
-
8
- The catalog defines tables, columns, constraints, views, and indexes. The page
9
- manager and buffer pool provide durable storage. Recovery replays valid WAL and
10
- rejects malformed or uncertain state. Server/client mode isolates application
11
- processes from the exclusive embedded owner.
12
-
13
- Read the [current-state audit](current-state.md) before relying on any feature.
1
+ # Architecture overview
2
+
3
+ RubyDB separates logical query processing from durable state changes. SQL is
4
+ lexed into tokens, parsed into an AST, bound against catalog metadata, planned,
5
+ and executed against the storage engine. DML is coordinated with transactions
6
+ and WAL before durable acknowledgement.
7
+
8
+ The catalog defines tables, columns, constraints, views, and indexes. The page
9
+ manager and buffer pool provide durable storage. Recovery replays valid WAL and
10
+ rejects malformed or uncertain state. Server/client mode isolates application
11
+ processes from the exclusive embedded owner.
12
+
13
+ Read the [current-state audit](current-state.md) before relying on any feature.
@@ -1,11 +1,11 @@
1
- # Pages and file layout
2
-
3
- RubyDB stores fixed-size pages containing validated headers, checksums, and
4
- typed records. Page allocation, metadata, table data, and index pages are
5
- managed by the storage layer; callers must not edit database files directly.
6
-
7
- The page format is versioned. Unknown formats, invalid page sizes, malformed
8
- headers, and checksum failures fail closed during open or recovery. Keep the
9
- database, WAL, catalog metadata, and lock files together when copying or
10
- restoring a database. See [storage format](../../spec/storage/format.md) and
11
- [upgrade guidance](../operations/upgrades.md).
1
+ # Pages and file layout
2
+
3
+ RubyDB stores fixed-size pages containing validated headers, checksums, and
4
+ typed records. Page allocation, metadata, table data, and index pages are
5
+ managed by the storage layer; callers must not edit database files directly.
6
+
7
+ The page format is versioned. Unknown formats, invalid page sizes, malformed
8
+ headers, and checksum failures fail closed during open or recovery. Keep the
9
+ database, WAL, catalog metadata, and lock files together when copying or
10
+ restoring a database. See [storage format](../../spec/storage/format.md) and
11
+ [upgrade guidance](../operations/upgrades.md).