rubydb 0.1.3 → 0.1.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (420) hide show
  1. checksums.yaml +4 -4
  2. data/.github/PULL_REQUEST_TEMPLATE.md +15 -15
  3. data/.github/workflows/benchmark.yml +26 -26
  4. data/.github/workflows/compatibility.yml +63 -63
  5. data/.github/workflows/fuzz.yml +33 -33
  6. data/.github/workflows/lint.yml +21 -21
  7. data/.github/workflows/operations.yml +24 -24
  8. data/.github/workflows/production-validation.yml +111 -111
  9. data/.github/workflows/release.yml +77 -77
  10. data/.github/workflows/security.yml +39 -37
  11. data/.github/workflows/test.yml +26 -26
  12. data/.github/workflows/workload.yml +58 -58
  13. data/.gitignore +12 -5
  14. data/.rubocop.yml +50 -44
  15. data/.standard.yml +9 -14
  16. data/ARCHITECTURE.md +21 -21
  17. data/CHANGELOG.md +49 -27
  18. data/CODE_OF_CONDUCT.md +13 -13
  19. data/CONTRIBUTING.md +29 -29
  20. data/GOVERNANCE.md +16 -16
  21. data/Gemfile +18 -17
  22. data/Gemfile.lock +125 -71
  23. data/README.md +249 -162
  24. data/ROADMAP.md +27 -27
  25. data/Rakefile +71 -71
  26. data/SECURITY.md +54 -54
  27. data/SUPPORT.md +14 -14
  28. data/adapters/activerecord/Gemfile +11 -11
  29. data/adapters/activerecord/lib/active_record/connection_adapters/rubydb_adapter.rb +877 -880
  30. data/adapters/activerecord/rubydb-activerecord.gemspec +21 -21
  31. data/adapters/activerecord/spec/rubydb_adapter_integration_spec.rb +155 -143
  32. data/adapters/ruby/README.md +18 -18
  33. data/adapters/sequel/README.md +11 -11
  34. data/config/monitoring/prometheus-alerts.yml +39 -39
  35. data/config/production.yml +36 -36
  36. data/docs/README.md +75 -71
  37. data/docs/architecture/concurrency.md +14 -14
  38. data/docs/architecture/current-state.md +125 -125
  39. data/docs/architecture/execution-engine.md +25 -25
  40. data/docs/architecture/indexes.md +19 -19
  41. data/docs/architecture/mvcc.md +19 -19
  42. data/docs/architecture/overview.md +13 -13
  43. data/docs/architecture/pages.md +11 -11
  44. data/docs/architecture/production-roadmap.md +82 -82
  45. data/docs/architecture/query-planner.md +20 -20
  46. data/docs/architecture/recovery.md +18 -18
  47. data/docs/architecture/sql-engine.md +12 -12
  48. data/docs/architecture/storage-engine.md +14 -14
  49. data/docs/architecture/transactions.md +10 -10
  50. data/docs/architecture/wal.md +28 -28
  51. data/docs/cli-cheatsheet.md +98 -98
  52. data/docs/cli.md +275 -275
  53. data/docs/contributing/architecture.md +9 -9
  54. data/docs/contributing/benchmarking.md +14 -14
  55. data/docs/contributing/development.md +16 -16
  56. data/docs/contributing/release-process.md +49 -49
  57. data/docs/contributing/testing.md +16 -16
  58. data/docs/debugging.md +229 -229
  59. data/docs/developer/branching.md +10 -10
  60. data/docs/developer/database-diff.md +10 -10
  61. data/docs/developer/local-development.md +17 -17
  62. data/docs/developer/snapshots.md +9 -9
  63. data/docs/developer/temporal-data.md +10 -10
  64. data/docs/developer-guide.md +297 -297
  65. data/docs/getting-started/first-database.md +16 -16
  66. data/docs/getting-started/first-query.md +13 -13
  67. data/docs/getting-started/installation.md +19 -19
  68. data/docs/getting-started/local-to-production.md +300 -300
  69. data/docs/getting-started/quickstart.md +17 -17
  70. data/docs/getting-started/rails.md +16 -16
  71. data/docs/hardening_backlog.md +93 -93
  72. data/docs/lessons-learned.md +112 -112
  73. data/docs/operations/backups.md +33 -33
  74. data/docs/operations/disaster-recovery.md +31 -31
  75. data/docs/operations/failover.md +30 -30
  76. data/docs/operations/monitoring.md +25 -25
  77. data/docs/operations/production-guide.md +295 -295
  78. data/docs/operations/production-runbook.md +45 -45
  79. data/docs/operations/replication.md +33 -33
  80. data/docs/operations/restore.md +6 -6
  81. data/docs/operations/runbook.md +34 -34
  82. data/docs/operations/upgrades.md +14 -14
  83. data/docs/operations/workload-testing.md +17 -17
  84. data/docs/production-readiness.md +118 -118
  85. data/docs/production_validation.md +150 -150
  86. data/docs/rails/active-record.md +11 -11
  87. data/docs/rails/compatibility-guide.md +90 -90
  88. data/docs/rails/database-yml.md +92 -92
  89. data/docs/rails/installation.md +17 -17
  90. data/docs/rails/migrations.md +17 -17
  91. data/docs/rails/production.md +82 -82
  92. data/docs/rails/troubleshooting.md +18 -18
  93. data/docs/release.md +25 -25
  94. data/docs/server/architecture.md +10 -10
  95. data/docs/server/authentication.md +10 -10
  96. data/docs/server/configuration.md +16 -16
  97. data/docs/server/connection-pooling.md +10 -10
  98. data/docs/server/deployment.md +10 -10
  99. data/docs/server/protocol.md +12 -12
  100. data/docs/sql/compatibility-guide.md +82 -82
  101. data/docs/sql/compatibility.md +39 -39
  102. data/docs/sql/data-types.md +10 -10
  103. data/docs/sql/functions.md +9 -9
  104. data/docs/sql/joins.md +9 -9
  105. data/docs/sql/operators.md +9 -9
  106. data/docs/sql/sqlite-compatibility.md +21 -21
  107. data/docs/sql/syntax.md +10 -10
  108. data/docs/sql/transactions.md +10 -10
  109. data/docs/troubleshooting.md +244 -244
  110. data/lessons/01-foundations.md +73 -0
  111. data/lessons/02-local-development.md +121 -0
  112. data/lessons/03-embedded-rubydb.md +99 -0
  113. data/lessons/04-rails-complex-apps.md +138 -0
  114. data/lessons/05-rubydb-production-server.md +237 -0
  115. data/lessons/06-postgresql-massive-apps.md +96 -0
  116. data/lessons/07-hybrid-microservices.md +94 -0
  117. data/lessons/08-migrations-backups-recovery.md +86 -0
  118. data/lessons/09-observability-security-scale.md +87 -0
  119. data/lessons/10-release-readiness.md +125 -0
  120. data/lib/rubydb/backup/archive.rb +332 -334
  121. data/lib/rubydb/backup/backup.rb +400 -401
  122. data/lib/rubydb/backup/incremental.rb +349 -353
  123. data/lib/rubydb/backup/restore.rb +289 -290
  124. data/lib/rubydb/backup/snapshot.rb +265 -267
  125. data/lib/rubydb/backup/verification.rb +276 -279
  126. data/lib/rubydb/branching/branch.rb +181 -181
  127. data/lib/rubydb/branching/branch_manager.rb +307 -311
  128. data/lib/rubydb/branching/branch_metadata.rb +140 -140
  129. data/lib/rubydb/branching/checkout.rb +165 -166
  130. data/lib/rubydb/branching/copy_on_write.rb +272 -272
  131. data/lib/rubydb/branching/diff.rb +137 -138
  132. data/lib/rubydb/branching/merge.rb +282 -285
  133. data/lib/rubydb/build_info.rb +15 -15
  134. data/lib/rubydb/catalog/catalog.rb +391 -391
  135. data/lib/rubydb/catalog/column.rb +112 -112
  136. data/lib/rubydb/catalog/constraint.rb +180 -180
  137. data/lib/rubydb/catalog/database.rb +184 -184
  138. data/lib/rubydb/catalog/index.rb +97 -97
  139. data/lib/rubydb/catalog/schema.rb +103 -103
  140. data/lib/rubydb/catalog/sequence.rb +90 -90
  141. data/lib/rubydb/catalog/system_catalog.rb +698 -698
  142. data/lib/rubydb/catalog/table.rb +178 -178
  143. data/lib/rubydb/catalog/trigger.rb +102 -102
  144. data/lib/rubydb/catalog/view.rb +66 -66
  145. data/lib/rubydb/cli/application.rb +163 -163
  146. data/lib/rubydb/cli/commands/backup.rb +80 -81
  147. data/lib/rubydb/cli/commands/branch.rb +72 -72
  148. data/lib/rubydb/cli/commands/checkout.rb +54 -54
  149. data/lib/rubydb/cli/commands/create.rb +58 -58
  150. data/lib/rubydb/cli/commands/diff.rb +76 -77
  151. data/lib/rubydb/cli/commands/doctor.rb +74 -74
  152. data/lib/rubydb/cli/commands/drop.rb +57 -57
  153. data/lib/rubydb/cli/commands/init.rb +101 -102
  154. data/lib/rubydb/cli/commands/inspect.rb +95 -95
  155. data/lib/rubydb/cli/commands/merge.rb +63 -63
  156. data/lib/rubydb/cli/commands/migrate.rb +62 -62
  157. data/lib/rubydb/cli/commands/restart.rb +42 -39
  158. data/lib/rubydb/cli/commands/restore.rb +121 -121
  159. data/lib/rubydb/cli/commands/shell.rb +365 -365
  160. data/lib/rubydb/cli/commands/snapshot.rb +79 -79
  161. data/lib/rubydb/cli/commands/start.rb +88 -82
  162. data/lib/rubydb/cli/commands/status.rb +96 -92
  163. data/lib/rubydb/cli/commands/stop.rb +47 -47
  164. data/lib/rubydb/cli/commands/vacuum.rb +58 -58
  165. data/lib/rubydb/cli/formatter.rb +221 -221
  166. data/lib/rubydb/cli/output.rb +168 -168
  167. data/lib/rubydb/client/client.rb +302 -304
  168. data/lib/rubydb/client/connection.rb +414 -415
  169. data/lib/rubydb/client/connection_pool.rb +168 -168
  170. data/lib/rubydb/client/connection_url.rb +96 -96
  171. data/lib/rubydb/client/prepared_statement.rb +60 -60
  172. data/lib/rubydb/client/result.rb +123 -123
  173. data/lib/rubydb/client/statement.rb +52 -52
  174. data/lib/rubydb/client/transaction.rb +130 -130
  175. data/lib/rubydb/concurrency/concurrency.rb +19 -19
  176. data/lib/rubydb/concurrency/deadlock_detector.rb +148 -150
  177. data/lib/rubydb/concurrency/latch.rb +101 -101
  178. data/lib/rubydb/concurrency/lock_graph.rb +163 -165
  179. data/lib/rubydb/concurrency/mutex.rb +181 -183
  180. data/lib/rubydb/concurrency/rw_lock.rb +180 -180
  181. data/lib/rubydb/concurrency/scheduler.rb +248 -250
  182. data/lib/rubydb/concurrency/worker_pool.rb +145 -143
  183. data/lib/rubydb/configuration/config.rb +170 -170
  184. data/lib/rubydb/configuration/defaults.rb +179 -179
  185. data/lib/rubydb/configuration/environment.rb +152 -152
  186. data/lib/rubydb/configuration/parser.rb +185 -185
  187. data/lib/rubydb/configuration/validation.rb +221 -221
  188. data/lib/rubydb/constants.rb +74 -74
  189. data/lib/rubydb/constraints/check.rb +181 -181
  190. data/lib/rubydb/constraints/constraint.rb +101 -101
  191. data/lib/rubydb/constraints/foreign_key.rb +130 -130
  192. data/lib/rubydb/constraints/not_null.rb +64 -64
  193. data/lib/rubydb/constraints/primary_key.rb +99 -99
  194. data/lib/rubydb/constraints/unique.rb +106 -108
  195. data/lib/rubydb/constraints/validator.rb +349 -350
  196. data/lib/rubydb/errors/authentication_error.rb +10 -10
  197. data/lib/rubydb/errors/authorization_error.rb +23 -23
  198. data/lib/rubydb/errors/client_error.rb +10 -10
  199. data/lib/rubydb/errors/configuration_error.rb +10 -10
  200. data/lib/rubydb/errors/connection_error.rb +10 -10
  201. data/lib/rubydb/errors/constraint_error.rb +23 -23
  202. data/lib/rubydb/errors/corruption_error.rb +10 -10
  203. data/lib/rubydb/errors/database_error.rb +10 -10
  204. data/lib/rubydb/errors/error.rb +20 -20
  205. data/lib/rubydb/errors/execution_error.rb +10 -10
  206. data/lib/rubydb/errors/parser_error.rb +10 -10
  207. data/lib/rubydb/errors/recovery_error.rb +10 -10
  208. data/lib/rubydb/errors/replication_error.rb +10 -10
  209. data/lib/rubydb/errors/server_error.rb +6 -6
  210. data/lib/rubydb/errors/storage_error.rb +10 -10
  211. data/lib/rubydb/errors/transaction_error.rb +10 -10
  212. data/lib/rubydb/execution/aggregate_executor.rb +134 -138
  213. data/lib/rubydb/execution/delete_executor.rb +110 -112
  214. data/lib/rubydb/execution/distinct_executor.rb +131 -135
  215. data/lib/rubydb/execution/executor.rb +1195 -1192
  216. data/lib/rubydb/execution/expression.rb +191 -193
  217. data/lib/rubydb/execution/index_scan.rb +142 -142
  218. data/lib/rubydb/execution/insert_executor.rb +215 -217
  219. data/lib/rubydb/execution/join_executor.rb +243 -249
  220. data/lib/rubydb/execution/limit_executor.rb +83 -85
  221. data/lib/rubydb/execution/optimizer.rb +227 -215
  222. data/lib/rubydb/execution/plan.rb +355 -353
  223. data/lib/rubydb/execution/planner.rb +542 -536
  224. data/lib/rubydb/execution/predicate.rb +235 -235
  225. data/lib/rubydb/execution/scan.rb +49 -49
  226. data/lib/rubydb/execution/sequential_scan.rb +63 -63
  227. data/lib/rubydb/execution/sort_executor.rb +180 -185
  228. data/lib/rubydb/execution/update_executor.rb +160 -162
  229. data/lib/rubydb/functions/aggregate.rb +70 -70
  230. data/lib/rubydb/functions/date_functions.rb +274 -278
  231. data/lib/rubydb/functions/function.rb +85 -85
  232. data/lib/rubydb/functions/json_functions.rb +231 -215
  233. data/lib/rubydb/functions/numeric_functions.rb +346 -346
  234. data/lib/rubydb/functions/scalar.rb +52 -52
  235. data/lib/rubydb/functions/string_functions.rb +383 -383
  236. data/lib/rubydb/functions/system_functions.rb +258 -246
  237. data/lib/rubydb/history/as_of.rb +238 -238
  238. data/lib/rubydb/history/change.rb +105 -105
  239. data/lib/rubydb/history/history.rb +131 -131
  240. data/lib/rubydb/history/history_manager.rb +228 -229
  241. data/lib/rubydb/history/temporal_query.rb +202 -202
  242. data/lib/rubydb/history/timeline.rb +144 -144
  243. data/lib/rubydb/indexes/btree.rb +186 -186
  244. data/lib/rubydb/indexes/btree_cursor.rb +258 -258
  245. data/lib/rubydb/indexes/btree_node.rb +384 -385
  246. data/lib/rubydb/indexes/hash_index.rb +150 -150
  247. data/lib/rubydb/indexes/index.rb +71 -71
  248. data/lib/rubydb/indexes/index_manager.rb +408 -406
  249. data/lib/rubydb/indexes/index_scan.rb +466 -470
  250. data/lib/rubydb/migrations/migration.rb +253 -254
  251. data/lib/rubydb/migrations/migration_lock.rb +146 -146
  252. data/lib/rubydb/migrations/migration_manager.rb +187 -176
  253. data/lib/rubydb/migrations/migration_version.rb +71 -71
  254. data/lib/rubydb/migrations/schema_diff.rb +211 -211
  255. data/lib/rubydb/migrations/schema_version.rb +64 -64
  256. data/lib/rubydb/monitoring/events.rb +155 -160
  257. data/lib/rubydb/monitoring/health.rb +216 -222
  258. data/lib/rubydb/monitoring/logger.rb +188 -193
  259. data/lib/rubydb/monitoring/metrics.rb +363 -359
  260. data/lib/rubydb/monitoring/performance.rb +176 -176
  261. data/lib/rubydb/monitoring/statistics.rb +168 -170
  262. data/lib/rubydb/mvcc/garbage_collector.rb +199 -199
  263. data/lib/rubydb/mvcc/mvcc.rb +16 -16
  264. data/lib/rubydb/mvcc/snapshot.rb +146 -147
  265. data/lib/rubydb/mvcc/vacuum.rb +180 -180
  266. data/lib/rubydb/mvcc/version.rb +106 -106
  267. data/lib/rubydb/mvcc/version_store.rb +396 -398
  268. data/lib/rubydb/mvcc/visibility.rb +107 -109
  269. data/lib/rubydb/protocol/capabilities.rb +125 -125
  270. data/lib/rubydb/protocol/decoder.rb +142 -145
  271. data/lib/rubydb/protocol/encoder.rb +131 -136
  272. data/lib/rubydb/protocol/handshake.rb +306 -305
  273. data/lib/rubydb/protocol/message.rb +121 -121
  274. data/lib/rubydb/protocol/parameter_binder.rb +101 -0
  275. data/lib/rubydb/protocol/protocol.rb +276 -277
  276. data/lib/rubydb/protocol/version.rb +54 -54
  277. data/lib/rubydb/rails/adapter.rb +245 -239
  278. data/lib/rubydb/rails/connection.rb +312 -314
  279. data/lib/rubydb/rails/database_statements.rb +122 -122
  280. data/lib/rubydb/rails/migration.rb +131 -131
  281. data/lib/rubydb/rails/quoting.rb +109 -109
  282. data/lib/rubydb/rails/result.rb +123 -120
  283. data/lib/rubydb/rails/schema_statements.rb +339 -339
  284. data/lib/rubydb/rails/transaction.rb +105 -105
  285. data/lib/rubydb/rails/type.rb +126 -126
  286. data/lib/rubydb/recovery/checkpoint.rb +261 -257
  287. data/lib/rubydb/recovery/consistency.rb +457 -467
  288. data/lib/rubydb/recovery/corruption_detector.rb +5 -5
  289. data/lib/rubydb/recovery/crash_recovery.rb +381 -387
  290. data/lib/rubydb/recovery/recovery_manager.rb +204 -206
  291. data/lib/rubydb/recovery/redo.rb +235 -237
  292. data/lib/rubydb/recovery/undo.rb +204 -206
  293. data/lib/rubydb/replication/failover.rb +5 -5
  294. data/lib/rubydb/replication/fencing.rb +63 -63
  295. data/lib/rubydb/replication/primary.rb +461 -450
  296. data/lib/rubydb/replication/replica.rb +382 -384
  297. data/lib/rubydb/replication/replication_log.rb +194 -200
  298. data/lib/rubydb/replication/replication_manager.rb +307 -308
  299. data/lib/rubydb/replication/replication_slot.rb +293 -295
  300. data/lib/rubydb/replication/replication_stream.rb +198 -201
  301. data/lib/rubydb/rubydb.rb +564 -560
  302. data/lib/rubydb/security/access_control.rb +252 -254
  303. data/lib/rubydb/security/audit_log.rb +209 -213
  304. data/lib/rubydb/security/authentication.rb +302 -302
  305. data/lib/rubydb/security/authorization.rb +282 -282
  306. data/lib/rubydb/security/credentials.rb +192 -196
  307. data/lib/rubydb/security/password.rb +205 -215
  308. data/lib/rubydb/security/permissions.rb +74 -74
  309. data/lib/rubydb/security/role.rb +99 -101
  310. data/lib/rubydb/security/user.rb +86 -86
  311. data/lib/rubydb/server/connection.rb +383 -366
  312. data/lib/rubydb/server/connection_pool.rb +193 -193
  313. data/lib/rubydb/server/lifecycle.rb +227 -228
  314. data/lib/rubydb/server/listener.rb +139 -136
  315. data/lib/rubydb/server/request_handler.rb +277 -276
  316. data/lib/rubydb/server/server.rb +363 -364
  317. data/lib/rubydb/server/session.rb +371 -369
  318. data/lib/rubydb/server/worker.rb +206 -210
  319. data/lib/rubydb/server/worker_pool.rb +168 -168
  320. data/lib/rubydb/sql/ast/alter_table.rb +169 -169
  321. data/lib/rubydb/sql/ast/begin_transaction.rb +47 -47
  322. data/lib/rubydb/sql/ast/commit.rb +37 -37
  323. data/lib/rubydb/sql/ast/constraint.rb +92 -83
  324. data/lib/rubydb/sql/ast/create_database.rb +41 -41
  325. data/lib/rubydb/sql/ast/create_index.rb +61 -61
  326. data/lib/rubydb/sql/ast/create_schema.rb +52 -52
  327. data/lib/rubydb/sql/ast/create_table.rb +187 -187
  328. data/lib/rubydb/sql/ast/delete.rb +54 -54
  329. data/lib/rubydb/sql/ast/drop_database.rb +41 -41
  330. data/lib/rubydb/sql/ast/drop_index.rb +41 -41
  331. data/lib/rubydb/sql/ast/drop_schema.rb +49 -49
  332. data/lib/rubydb/sql/ast/drop_table.rb +49 -49
  333. data/lib/rubydb/sql/ast/explain.rb +64 -64
  334. data/lib/rubydb/sql/ast/expression.rb +617 -604
  335. data/lib/rubydb/sql/ast/insert.rb +66 -66
  336. data/lib/rubydb/sql/ast/node.rb +42 -42
  337. data/lib/rubydb/sql/ast/rollback.rb +63 -63
  338. data/lib/rubydb/sql/ast/savepoint.rb +59 -59
  339. data/lib/rubydb/sql/ast/select.rb +88 -88
  340. data/lib/rubydb/sql/ast/set_operation.rb +22 -20
  341. data/lib/rubydb/sql/ast/trigger.rb +35 -29
  342. data/lib/rubydb/sql/ast/update.rb +88 -88
  343. data/lib/rubydb/sql/ast/vacuum.rb +19 -19
  344. data/lib/rubydb/sql/ast/view.rb +38 -32
  345. data/lib/rubydb/sql/ast/with.rb +32 -32
  346. data/lib/rubydb/sql/grammar.rb +86 -86
  347. data/lib/rubydb/sql/keywords.rb +156 -156
  348. data/lib/rubydb/sql/lexer.rb +209 -214
  349. data/lib/rubydb/sql/operators.rb +100 -100
  350. data/lib/rubydb/sql/parser.rb +1167 -1170
  351. data/lib/rubydb/sql/planner/analyzer.rb +283 -302
  352. data/lib/rubydb/sql/planner/binder.rb +537 -550
  353. data/lib/rubydb/sql/planner/type_checker.rb +427 -431
  354. data/lib/rubydb/sql/token.rb +210 -210
  355. data/lib/rubydb/storage/buffer_frame.rb +44 -44
  356. data/lib/rubydb/storage/buffer_pool.rb +155 -155
  357. data/lib/rubydb/storage/database_lock.rb +74 -74
  358. data/lib/rubydb/storage/deserializer.rb +332 -342
  359. data/lib/rubydb/storage/engine.rb +2347 -2330
  360. data/lib/rubydb/storage/file_manager.rb +191 -187
  361. data/lib/rubydb/storage/free_space_map.rb +79 -81
  362. data/lib/rubydb/storage/page.rb +92 -94
  363. data/lib/rubydb/storage/page_allocator.rb +852 -855
  364. data/lib/rubydb/storage/page_header.rb +63 -67
  365. data/lib/rubydb/storage/page_manager.rb +127 -131
  366. data/lib/rubydb/storage/record.rb +58 -58
  367. data/lib/rubydb/storage/row.rb +78 -78
  368. data/lib/rubydb/storage/serializer.rb +51 -51
  369. data/lib/rubydb/storage/storage_layout.rb +151 -151
  370. data/lib/rubydb/storage/storage_manager.rb +114 -114
  371. data/lib/rubydb/storage/tuple.rb +458 -461
  372. data/lib/rubydb/storage/visibility_map.rb +964 -973
  373. data/lib/rubydb/transactions/commit_manager.rb +219 -220
  374. data/lib/rubydb/transactions/isolation.rb +98 -98
  375. data/lib/rubydb/transactions/lock.rb +76 -76
  376. data/lib/rubydb/transactions/lock_manager.rb +359 -362
  377. data/lib/rubydb/transactions/savepoint.rb +142 -143
  378. data/lib/rubydb/transactions/transaction.rb +214 -215
  379. data/lib/rubydb/transactions/transaction_id.rb +84 -84
  380. data/lib/rubydb/transactions/transaction_log.rb +256 -257
  381. data/lib/rubydb/transactions/transaction_manager.rb +434 -435
  382. data/lib/rubydb/types/bigint.rb +36 -36
  383. data/lib/rubydb/types/blob.rb +37 -37
  384. data/lib/rubydb/types/boolean.rb +34 -34
  385. data/lib/rubydb/types/date.rb +39 -39
  386. data/lib/rubydb/types/decimal.rb +48 -48
  387. data/lib/rubydb/types/float.rb +34 -34
  388. data/lib/rubydb/types/integer.rb +36 -36
  389. data/lib/rubydb/types/json.rb +41 -41
  390. data/lib/rubydb/types/null.rb +34 -34
  391. data/lib/rubydb/types/smallint.rb +36 -36
  392. data/lib/rubydb/types/text.rb +37 -37
  393. data/lib/rubydb/types/time.rb +46 -46
  394. data/lib/rubydb/types/timestamp.rb +39 -39
  395. data/lib/rubydb/types/type.rb +119 -119
  396. data/lib/rubydb/types/uuid.rb +47 -47
  397. data/lib/rubydb/types/varchar.rb +37 -37
  398. data/lib/rubydb/version.rb +32 -32
  399. data/lib/rubydb/wal/archive.rb +190 -193
  400. data/lib/rubydb/wal/checkpoint.rb +181 -183
  401. data/lib/rubydb/wal/lsn.rb +94 -94
  402. data/lib/rubydb/wal/reader.rb +259 -260
  403. data/lib/rubydb/wal/record.rb +105 -105
  404. data/lib/rubydb/wal/segment.rb +193 -193
  405. data/lib/rubydb/wal/wal.rb +480 -452
  406. data/lib/rubydb/wal/writer.rb +236 -236
  407. data/lib/rubydb.rb +7 -7
  408. data/packaging/docker/docker-compose.failover.yml +43 -43
  409. data/packaging/homebrew/rubydb.rb +19 -19
  410. data/rubydb.gemspec +60 -57
  411. data/scripts/benchmark +7 -7
  412. data/scripts/durability_drill +37 -37
  413. data/scripts/fuzz +63 -63
  414. data/scripts/release +45 -40
  415. data/scripts/release_check +43 -43
  416. data/scripts/replication_failover_drill +268 -250
  417. data/scripts/replication_network_failover_drill +287 -255
  418. data/scripts/restore_drill +45 -45
  419. data/scripts/security +45 -0
  420. metadata +54 -1
@@ -1,297 +1,297 @@
1
- # RubyDB developer guide
2
-
3
- This is the implementation guide for contributors who need to change RubyDB
4
- safely. It explains the repository, the request path, the storage invariants,
5
- the test strategy, and the debugging workflow. Read it together with the
6
- topic specifications under `spec/`; the specifications are the behavioral
7
- contract, while this guide explains how the pieces fit together.
8
-
9
- ## 1. Scope and support promise
10
-
11
- RubyDB is a Ruby-native relational database with two ownership modes:
12
-
13
- * embedded mode, where one Ruby process owns a database directory; and
14
- * server mode, where the server owns the directory and application processes
15
- use the client protocol.
16
-
17
- The supported surface is the behavior exercised by the test suite and listed
18
- in [SQL compatibility](sql/compatibility.md). Similar syntax does not imply
19
- complete PostgreSQL, MySQL, or SQLite compatibility. A change that broadens
20
- syntax must also specify its semantics, errors, types, transactions, and
21
- adapter behavior.
22
-
23
- ## 2. Repository orientation
24
-
25
- The important directories are:
26
-
27
- | Path | Responsibility |
28
- | --- | --- |
29
- | `lib/rubydb` | Engine, SQL, storage, transactions, server, client, adapters |
30
- | `spec` | Unit, integration, protocol, SQL, recovery, and adapter contracts |
31
- | `benchmarks` | Repeatable throughput and concurrency measurements |
32
- | `scripts` | Soak tests, durability drills, fuzzing, and release checks |
33
- | `config` | Example configuration and monitoring rules |
34
- | `examples` | Runnable Ruby and Rails applications |
35
- | `docs` | User, operator, developer, and compatibility documentation |
36
- | `packaging` | Container and deployment examples |
37
- | `exe` and `bin` | Public command-line entry points |
38
-
39
- Start with `lib/rubydb.rb` and follow the public API into the engine. For a
40
- feature, locate the nearest existing spec before editing implementation code.
41
- Avoid adding a second abstraction when an existing subsystem already owns the
42
- invariant.
43
-
44
- ## 3. Local setup
45
-
46
- Use a supported Ruby version from the project CI matrix and install the locked
47
- dependencies:
48
-
49
- ```sh
50
- bundle install
51
- bundle exec rspec
52
- bundle exec rubocop
53
- ```
54
-
55
- Use a temporary database for experiments. Never use a production path in a
56
- test or run a destructive command against an unknown directory:
57
-
58
- ```ruby
59
- require "tmpdir"
60
- require "rubydb"
61
-
62
- Dir.mktmpdir("rubydb-dev-") do |dir|
63
- engine = RubyDB::Storage::Engine.new(File.join(dir, "dev.rdb"))
64
- engine.execute("CREATE TABLE items (id INTEGER PRIMARY KEY, name TEXT)")
65
- engine.close
66
- end
67
- ```
68
-
69
- The server/client path is required when testing process boundaries. Multiple
70
- independent embedded owners must never open the same database directory.
71
-
72
- ## 4. The request lifecycle
73
-
74
- A typical SQL request follows this sequence:
75
-
76
- ```text
77
- Ruby API or client
78
- -> connection/session and authentication
79
- -> SQL tokenizer/parser
80
- -> binder and type/parameter validation
81
- -> planner and executor
82
- -> transaction/MVCC read or write set
83
- -> table/index mutation
84
- -> WAL append and durable commit
85
- -> result encoding and protocol response
86
- ```
87
-
88
- Keep failures at the layer that owns them. Parser errors should not be turned
89
- into storage errors. A failed WAL append must not report a committed schema or
90
- row change. A client timeout must not silently convert an unknown commit into
91
- a rollback; the caller must be able to determine whether a request was
92
- committed before retrying.
93
-
94
- ## 5. Storage and durability invariants
95
-
96
- The storage directory contains data pages, metadata, WAL state, and recovery
97
- artifacts. Changes must preserve these invariants:
98
-
99
- 1. A page has valid framing and checksum before it is trusted.
100
- 2. WAL records are validated before replay and are applied in LSN order.
101
- 3. A commit acknowledgement is emitted only after the configured durability
102
- point has succeeded.
103
- 4. Schema publication and its dependent indexes are visible atomically.
104
- 5. Recovery is idempotent: replaying a committed record does not duplicate a
105
- row or corrupt an index.
106
- 6. Torn, truncated, or corrupted input fails closed with a useful error.
107
- 7. Temporary files are not mistaken for a completed checkpoint or backup.
108
-
109
- When changing page layout or record encoding, update the format specification,
110
- versioning/upgrade path, compatibility tests, and recovery fixtures. Do not
111
- silently reinterpret old bytes. If a format cannot be read, return an explicit
112
- upgrade or corruption error and preserve the original files for diagnosis.
113
-
114
- ## 6. WAL, checkpoints, and recovery
115
-
116
- The WAL is the source of truth between checkpoints. A checkpoint copies safe
117
- state to durable pages and advances the recovery boundary only after all
118
- required data and metadata have been flushed. Recovery should:
119
-
120
- 1. open the directory read-only where possible;
121
- 2. validate metadata and the last known checkpoint;
122
- 3. scan WAL frames, stopping only at a valid end boundary;
123
- 4. reject checksum, sequence, length, or transaction inconsistencies;
124
- 5. replay committed work and discard incomplete transactions;
125
- 6. rebuild or validate indexes before accepting writes; and
126
- 7. publish a recovery result with the replayed LSN and warnings.
127
-
128
- Tests must cover normal reopen, a process terminated during a write, an
129
- interrupted checkpoint, truncated WAL, invalid checksums, missing metadata,
130
- full-disk behavior, and restore into a new directory. Fault injection belongs
131
- around filesystem calls, not only around Ruby methods, because failures occur
132
- at `fsync`, rename, allocation, and close boundaries.
133
-
134
- ## 7. Transactions, MVCC, and locking
135
-
136
- Every statement executes in a transaction context, whether it is explicit or
137
- implicit. The context owns the snapshot, read view, write set, lock state,
138
- savepoints, and commit result. A transaction must not leak locks or snapshots
139
- when it raises, times out, is cancelled, or loses its connection.
140
-
141
- MVCC readers use a stable visibility point. Writers create new versions and
142
- retain the before-image needed by active readers and rollback. Vacuum may
143
- reclaim a version only after the global safe point has passed it. A new feature
144
- must define behavior for:
145
-
146
- * read committed, repeatable read, and serializable transactions;
147
- * concurrent update of the same row;
148
- * unique and foreign-key conflicts;
149
- * deadlock detection and victim rollback;
150
- * lock wait timeouts and request cancellation; and
151
- * commit acknowledgement followed by client disconnect.
152
-
153
- Deadlock resolution must abort a complete victim transaction, release all of
154
- its locks, and leave other transactions able to progress. Never fix a deadlock
155
- by globally disabling locking or by releasing a lock without undoing the
156
- corresponding write set.
157
-
158
- ## 8. SQL implementation workflow
159
-
160
- For a new statement or expression:
161
-
162
- 1. write the syntax and semantic contract in `spec/sql`;
163
- 2. add parser acceptance and rejection examples;
164
- 3. add binder/type/nullability behavior;
165
- 4. add planner and executor tests;
166
- 5. test transaction, constraint, index, and error interactions;
167
- 6. test the Ruby API and server protocol path; and
168
- 7. test the ActiveRecord-generated SQL when the feature is adapter-visible.
169
-
170
- Prefer parameter binding to string interpolation. Every expression needs
171
- defined behavior for `NULL`, booleans, numeric coercion, text comparison,
172
- collation, and invalid input. Every DDL operation needs idempotence or an
173
- explicit error contract. Every optimizer rewrite must have a semantic
174
- equivalence test against the non-optimized execution path.
175
-
176
- The documented common SQLite-style profile is intentionally narrower than
177
- SQLite itself. Do not label a feature “SQLite compatible” until its syntax,
178
- results, types, error behavior, and migration behavior are tested.
179
-
180
- ## 9. Tables, indexes, and constraints
181
-
182
- Table mutations and index mutations are one logical operation. If an index
183
- write fails, the statement must fail visibly and the transaction must either
184
- roll back or retain a recoverable pending state; it must not acknowledge a row
185
- that cannot be found by a required index.
186
-
187
- For each index type, test empty and populated tables, duplicate keys, `NULL`,
188
- deep splits, reopen, recovery replay, deletion/merge, and concurrent readers.
189
- For each constraint, test direct SQL, prepared parameters, ActiveRecord
190
- inserts/updates, rollback, and the exact error class/message contract where
191
- callers may depend on it.
192
-
193
- ## 10. Server, client, and wire protocol
194
-
195
- The server is the single owner of the database directory. A client session
196
- handles authentication, capability negotiation, request IDs, deadlines,
197
- cancellation, result framing, and connection shutdown. Protocol changes must
198
- be backward-compatible or carry an explicit protocol version and rejection
199
- path.
200
-
201
- Wire cancellation is part of correctness. A cancelled request must stop
202
- execution at safe checkpoints, release transaction resources, and return a
203
- definitive cancellation response. If cancellation races with commit, the
204
- server must preserve the commit outcome and expose enough request identity for
205
- the client to query status rather than blindly retrying.
206
-
207
- Test malformed lengths, unknown message types, duplicate request IDs, partial
208
- frames, client disconnects, server shutdown, timeout races, authentication
209
- failure, TLS negotiation, and cancellation during a long scan.
210
-
211
- ## 11. Rails and adapter work
212
-
213
- The ActiveRecord adapter translates Rails schema and query APIs into RubyDB
214
- SQL. Adapter code must preserve Rails expectations for quoting, bind
215
- parameters, affected-row counts, last-insert IDs, transactions, savepoints,
216
- schema introspection, migration versions, and exceptions.
217
-
218
- When changing adapter behavior, run the example app and test at least:
219
-
220
- * `where`, ordering, limits, scopes, joins, and aggregate relations;
221
- * eager loading, nested associations, and inverse relationships;
222
- * connection pools with multiple threads;
223
- * schema dump/load and populated-table migrations;
224
- * rollback, retry, and migration checksum behavior; and
225
- * every supported Ruby/Rails combination in CI.
226
-
227
- The adapter is not proof of complete SQLite or PostgreSQL compatibility. An
228
- application should run its own generated-SQL and migration suite before a
229
- cutover.
230
-
231
- ## 12. Testing pyramid
232
-
233
- Use the smallest test that proves the invariant:
234
-
235
- * unit specs for parsing, encoding, types, and isolated algorithms;
236
- * component specs for storage, WAL, indexes, transactions, and protocol;
237
- * integration specs for engine-to-SQL and server-to-client behavior;
238
- * process specs for crash recovery, pooling, and failover;
239
- * workload tests for sustained concurrency and resource limits; and
240
- * example applications for real Rails and Ruby workflows.
241
-
242
- Useful commands include:
243
-
244
- ```sh
245
- bundle exec rspec
246
- bundle exec rspec spec/sql spec/storage spec/transactions
247
- ruby scripts/durability_drill
248
- ruby scripts/production_soak
249
- ruby scripts/fuzz
250
- RUBYDB_BENCHMARK_ITERATIONS=100 ruby -Ilib benchmarks/basic_workload.rb
251
- ```
252
-
253
- Record the Ruby version, commit, OS, command, seed, database configuration,
254
- and artifact paths for every non-trivial run. A green unit suite does not
255
- replace a restore drill, multi-process soak, or deployment test.
256
-
257
- ## 13. Debugging a failing change
258
-
259
- Start with [Debugging RubyDB](debugging.md). Reproduce on a fresh temporary
260
- directory, reduce the schema and SQL, rerun with the reported seed, and save
261
- logs plus the WAL metadata. Compare embedded and server/client execution when
262
- ownership or protocol is suspected. Use [Troubleshooting](troubleshooting.md)
263
- for operator symptoms and evidence-preserving recovery steps.
264
-
265
- Never “repair” a failing test by deleting WAL, disabling checksums, loosening a
266
- constraint, or turning off synchronization. Preserve the failing directory,
267
- copy it, and investigate the copy.
268
-
269
- ## 14. Performance and capacity work
270
-
271
- Benchmark one variable at a time and report latency percentiles, throughput,
272
- concurrency, payload shape, cache size, WAL/checkpoint settings, and storage
273
- medium. Track p50, p95, p99, error rate, lock wait time, WAL growth, checkpoint
274
- duration, memory, file descriptors, and CPU.
275
-
276
- Capacity limits are workload-specific. A benchmark result is not a guarantee
277
- for a Rails application with different indexes, query shapes, or connection
278
- pool settings. Add a regression threshold only after the benchmark is stable
279
- across repeated runs and environments.
280
-
281
- ## 15. Safe contribution checklist
282
-
283
- Before opening a pull request:
284
-
285
- 1. Explain the invariant and compatibility impact.
286
- 2. Add focused tests, including failure paths.
287
- 3. Run the relevant focused suite and the full suite.
288
- 4. Run formatting/lint and `git diff --check`.
289
- 5. Update the appropriate spec and user/operator documentation.
290
- 6. Note migrations, format changes, recovery implications, and rollback plan.
291
- 7. Include benchmark or soak evidence for hot-path and concurrency changes.
292
- 8. Remove debug output, secrets, temporary data, and generated artifacts.
293
-
294
- See [Contributing](../CONTRIBUTING.md), [testing](contributing/testing.md),
295
- and [release process](contributing/release-process.md) for the repository
296
- workflow.
297
-
1
+ # RubyDB developer guide
2
+
3
+ This is the implementation guide for contributors who need to change RubyDB
4
+ safely. It explains the repository, the request path, the storage invariants,
5
+ the test strategy, and the debugging workflow. Read it together with the
6
+ topic specifications under `spec/`; the specifications are the behavioral
7
+ contract, while this guide explains how the pieces fit together.
8
+
9
+ ## 1. Scope and support promise
10
+
11
+ RubyDB is a Ruby-native relational database with two ownership modes:
12
+
13
+ * embedded mode, where one Ruby process owns a database directory; and
14
+ * server mode, where the server owns the directory and application processes
15
+ use the client protocol.
16
+
17
+ The supported surface is the behavior exercised by the test suite and listed
18
+ in [SQL compatibility](sql/compatibility.md). Similar syntax does not imply
19
+ complete PostgreSQL, MySQL, or SQLite compatibility. A change that broadens
20
+ syntax must also specify its semantics, errors, types, transactions, and
21
+ adapter behavior.
22
+
23
+ ## 2. Repository orientation
24
+
25
+ The important directories are:
26
+
27
+ | Path | Responsibility |
28
+ | --- | --- |
29
+ | `lib/rubydb` | Engine, SQL, storage, transactions, server, client, adapters |
30
+ | `spec` | Unit, integration, protocol, SQL, recovery, and adapter contracts |
31
+ | `benchmarks` | Repeatable throughput and concurrency measurements |
32
+ | `scripts` | Soak tests, durability drills, fuzzing, and release checks |
33
+ | `config` | Example configuration and monitoring rules |
34
+ | `examples` | Runnable Ruby and Rails applications |
35
+ | `docs` | User, operator, developer, and compatibility documentation |
36
+ | `packaging` | Container and deployment examples |
37
+ | `exe` and `bin` | Public command-line entry points |
38
+
39
+ Start with `lib/rubydb.rb` and follow the public API into the engine. For a
40
+ feature, locate the nearest existing spec before editing implementation code.
41
+ Avoid adding a second abstraction when an existing subsystem already owns the
42
+ invariant.
43
+
44
+ ## 3. Local setup
45
+
46
+ Use a supported Ruby version from the project CI matrix and install the locked
47
+ dependencies:
48
+
49
+ ```sh
50
+ bundle install
51
+ bundle exec rspec
52
+ bundle exec rubocop
53
+ ```
54
+
55
+ Use a temporary database for experiments. Never use a production path in a
56
+ test or run a destructive command against an unknown directory:
57
+
58
+ ```ruby
59
+ require "tmpdir"
60
+ require "rubydb"
61
+
62
+ Dir.mktmpdir("rubydb-dev-") do |dir|
63
+ engine = RubyDB::Storage::Engine.new(File.join(dir, "dev.rdb"))
64
+ engine.execute("CREATE TABLE items (id INTEGER PRIMARY KEY, name TEXT)")
65
+ engine.close
66
+ end
67
+ ```
68
+
69
+ The server/client path is required when testing process boundaries. Multiple
70
+ independent embedded owners must never open the same database directory.
71
+
72
+ ## 4. The request lifecycle
73
+
74
+ A typical SQL request follows this sequence:
75
+
76
+ ```text
77
+ Ruby API or client
78
+ -> connection/session and authentication
79
+ -> SQL tokenizer/parser
80
+ -> binder and type/parameter validation
81
+ -> planner and executor
82
+ -> transaction/MVCC read or write set
83
+ -> table/index mutation
84
+ -> WAL append and durable commit
85
+ -> result encoding and protocol response
86
+ ```
87
+
88
+ Keep failures at the layer that owns them. Parser errors should not be turned
89
+ into storage errors. A failed WAL append must not report a committed schema or
90
+ row change. A client timeout must not silently convert an unknown commit into
91
+ a rollback; the caller must be able to determine whether a request was
92
+ committed before retrying.
93
+
94
+ ## 5. Storage and durability invariants
95
+
96
+ The storage directory contains data pages, metadata, WAL state, and recovery
97
+ artifacts. Changes must preserve these invariants:
98
+
99
+ 1. A page has valid framing and checksum before it is trusted.
100
+ 2. WAL records are validated before replay and are applied in LSN order.
101
+ 3. A commit acknowledgement is emitted only after the configured durability
102
+ point has succeeded.
103
+ 4. Schema publication and its dependent indexes are visible atomically.
104
+ 5. Recovery is idempotent: replaying a committed record does not duplicate a
105
+ row or corrupt an index.
106
+ 6. Torn, truncated, or corrupted input fails closed with a useful error.
107
+ 7. Temporary files are not mistaken for a completed checkpoint or backup.
108
+
109
+ When changing page layout or record encoding, update the format specification,
110
+ versioning/upgrade path, compatibility tests, and recovery fixtures. Do not
111
+ silently reinterpret old bytes. If a format cannot be read, return an explicit
112
+ upgrade or corruption error and preserve the original files for diagnosis.
113
+
114
+ ## 6. WAL, checkpoints, and recovery
115
+
116
+ The WAL is the source of truth between checkpoints. A checkpoint copies safe
117
+ state to durable pages and advances the recovery boundary only after all
118
+ required data and metadata have been flushed. Recovery should:
119
+
120
+ 1. open the directory read-only where possible;
121
+ 2. validate metadata and the last known checkpoint;
122
+ 3. scan WAL frames, stopping only at a valid end boundary;
123
+ 4. reject checksum, sequence, length, or transaction inconsistencies;
124
+ 5. replay committed work and discard incomplete transactions;
125
+ 6. rebuild or validate indexes before accepting writes; and
126
+ 7. publish a recovery result with the replayed LSN and warnings.
127
+
128
+ Tests must cover normal reopen, a process terminated during a write, an
129
+ interrupted checkpoint, truncated WAL, invalid checksums, missing metadata,
130
+ full-disk behavior, and restore into a new directory. Fault injection belongs
131
+ around filesystem calls, not only around Ruby methods, because failures occur
132
+ at `fsync`, rename, allocation, and close boundaries.
133
+
134
+ ## 7. Transactions, MVCC, and locking
135
+
136
+ Every statement executes in a transaction context, whether it is explicit or
137
+ implicit. The context owns the snapshot, read view, write set, lock state,
138
+ savepoints, and commit result. A transaction must not leak locks or snapshots
139
+ when it raises, times out, is cancelled, or loses its connection.
140
+
141
+ MVCC readers use a stable visibility point. Writers create new versions and
142
+ retain the before-image needed by active readers and rollback. Vacuum may
143
+ reclaim a version only after the global safe point has passed it. A new feature
144
+ must define behavior for:
145
+
146
+ * read committed, repeatable read, and serializable transactions;
147
+ * concurrent update of the same row;
148
+ * unique and foreign-key conflicts;
149
+ * deadlock detection and victim rollback;
150
+ * lock wait timeouts and request cancellation; and
151
+ * commit acknowledgement followed by client disconnect.
152
+
153
+ Deadlock resolution must abort a complete victim transaction, release all of
154
+ its locks, and leave other transactions able to progress. Never fix a deadlock
155
+ by globally disabling locking or by releasing a lock without undoing the
156
+ corresponding write set.
157
+
158
+ ## 8. SQL implementation workflow
159
+
160
+ For a new statement or expression:
161
+
162
+ 1. write the syntax and semantic contract in `spec/sql`;
163
+ 2. add parser acceptance and rejection examples;
164
+ 3. add binder/type/nullability behavior;
165
+ 4. add planner and executor tests;
166
+ 5. test transaction, constraint, index, and error interactions;
167
+ 6. test the Ruby API and server protocol path; and
168
+ 7. test the ActiveRecord-generated SQL when the feature is adapter-visible.
169
+
170
+ Prefer parameter binding to string interpolation. Every expression needs
171
+ defined behavior for `NULL`, booleans, numeric coercion, text comparison,
172
+ collation, and invalid input. Every DDL operation needs idempotence or an
173
+ explicit error contract. Every optimizer rewrite must have a semantic
174
+ equivalence test against the non-optimized execution path.
175
+
176
+ The documented common SQLite-style profile is intentionally narrower than
177
+ SQLite itself. Do not label a feature “SQLite compatible” until its syntax,
178
+ results, types, error behavior, and migration behavior are tested.
179
+
180
+ ## 9. Tables, indexes, and constraints
181
+
182
+ Table mutations and index mutations are one logical operation. If an index
183
+ write fails, the statement must fail visibly and the transaction must either
184
+ roll back or retain a recoverable pending state; it must not acknowledge a row
185
+ that cannot be found by a required index.
186
+
187
+ For each index type, test empty and populated tables, duplicate keys, `NULL`,
188
+ deep splits, reopen, recovery replay, deletion/merge, and concurrent readers.
189
+ For each constraint, test direct SQL, prepared parameters, ActiveRecord
190
+ inserts/updates, rollback, and the exact error class/message contract where
191
+ callers may depend on it.
192
+
193
+ ## 10. Server, client, and wire protocol
194
+
195
+ The server is the single owner of the database directory. A client session
196
+ handles authentication, capability negotiation, request IDs, deadlines,
197
+ cancellation, result framing, and connection shutdown. Protocol changes must
198
+ be backward-compatible or carry an explicit protocol version and rejection
199
+ path.
200
+
201
+ Wire cancellation is part of correctness. A cancelled request must stop
202
+ execution at safe checkpoints, release transaction resources, and return a
203
+ definitive cancellation response. If cancellation races with commit, the
204
+ server must preserve the commit outcome and expose enough request identity for
205
+ the client to query status rather than blindly retrying.
206
+
207
+ Test malformed lengths, unknown message types, duplicate request IDs, partial
208
+ frames, client disconnects, server shutdown, timeout races, authentication
209
+ failure, TLS negotiation, and cancellation during a long scan.
210
+
211
+ ## 11. Rails and adapter work
212
+
213
+ The ActiveRecord adapter translates Rails schema and query APIs into RubyDB
214
+ SQL. Adapter code must preserve Rails expectations for quoting, bind
215
+ parameters, affected-row counts, last-insert IDs, transactions, savepoints,
216
+ schema introspection, migration versions, and exceptions.
217
+
218
+ When changing adapter behavior, run the example app and test at least:
219
+
220
+ * `where`, ordering, limits, scopes, joins, and aggregate relations;
221
+ * eager loading, nested associations, and inverse relationships;
222
+ * connection pools with multiple threads;
223
+ * schema dump/load and populated-table migrations;
224
+ * rollback, retry, and migration checksum behavior; and
225
+ * every supported Ruby/Rails combination in CI.
226
+
227
+ The adapter is not proof of complete SQLite or PostgreSQL compatibility. An
228
+ application should run its own generated-SQL and migration suite before a
229
+ cutover.
230
+
231
+ ## 12. Testing pyramid
232
+
233
+ Use the smallest test that proves the invariant:
234
+
235
+ * unit specs for parsing, encoding, types, and isolated algorithms;
236
+ * component specs for storage, WAL, indexes, transactions, and protocol;
237
+ * integration specs for engine-to-SQL and server-to-client behavior;
238
+ * process specs for crash recovery, pooling, and failover;
239
+ * workload tests for sustained concurrency and resource limits; and
240
+ * example applications for real Rails and Ruby workflows.
241
+
242
+ Useful commands include:
243
+
244
+ ```sh
245
+ bundle exec rspec
246
+ bundle exec rspec spec/sql spec/storage spec/transactions
247
+ ruby scripts/durability_drill
248
+ ruby scripts/production_soak
249
+ ruby scripts/fuzz
250
+ RUBYDB_BENCHMARK_ITERATIONS=100 ruby -Ilib benchmarks/basic_workload.rb
251
+ ```
252
+
253
+ Record the Ruby version, commit, OS, command, seed, database configuration,
254
+ and artifact paths for every non-trivial run. A green unit suite does not
255
+ replace a restore drill, multi-process soak, or deployment test.
256
+
257
+ ## 13. Debugging a failing change
258
+
259
+ Start with [Debugging RubyDB](debugging.md). Reproduce on a fresh temporary
260
+ directory, reduce the schema and SQL, rerun with the reported seed, and save
261
+ logs plus the WAL metadata. Compare embedded and server/client execution when
262
+ ownership or protocol is suspected. Use [Troubleshooting](troubleshooting.md)
263
+ for operator symptoms and evidence-preserving recovery steps.
264
+
265
+ Never “repair” a failing test by deleting WAL, disabling checksums, loosening a
266
+ constraint, or turning off synchronization. Preserve the failing directory,
267
+ copy it, and investigate the copy.
268
+
269
+ ## 14. Performance and capacity work
270
+
271
+ Benchmark one variable at a time and report latency percentiles, throughput,
272
+ concurrency, payload shape, cache size, WAL/checkpoint settings, and storage
273
+ medium. Track p50, p95, p99, error rate, lock wait time, WAL growth, checkpoint
274
+ duration, memory, file descriptors, and CPU.
275
+
276
+ Capacity limits are workload-specific. A benchmark result is not a guarantee
277
+ for a Rails application with different indexes, query shapes, or connection
278
+ pool settings. Add a regression threshold only after the benchmark is stable
279
+ across repeated runs and environments.
280
+
281
+ ## 15. Safe contribution checklist
282
+
283
+ Before opening a pull request:
284
+
285
+ 1. Explain the invariant and compatibility impact.
286
+ 2. Add focused tests, including failure paths.
287
+ 3. Run the relevant focused suite and the full suite.
288
+ 4. Run formatting/lint and `git diff --check`.
289
+ 5. Update the appropriate spec and user/operator documentation.
290
+ 6. Note migrations, format changes, recovery implications, and rollback plan.
291
+ 7. Include benchmark or soak evidence for hot-path and concurrency changes.
292
+ 8. Remove debug output, secrets, temporary data, and generated artifacts.
293
+
294
+ See [Contributing](../CONTRIBUTING.md), [testing](contributing/testing.md),
295
+ and [release process](contributing/release-process.md) for the repository
296
+ workflow.
297
+
@@ -1,16 +1,16 @@
1
- # First database
2
-
3
- ```ruby
4
- require "rubydb"
5
-
6
- path = "tmp/first-database.rdb"
7
- engine = RubyDB::Storage::Engine.new(path)
8
- engine.execute("CREATE TABLE accounts (id INTEGER PRIMARY KEY, email TEXT UNIQUE NOT NULL)")
9
- engine.execute("INSERT INTO accounts (id, email) VALUES (1, 'ada@example.test')")
10
- puts engine.execute("SELECT id, email FROM accounts").inspect
11
- engine.close
12
- ```
13
-
14
- The engine creates durable storage and WAL state under the selected path. Close
15
- the engine cleanly and keep the database, WAL, and lock files together. A single
16
- embedded path has one owner; use a server for multiple processes.
1
+ # First database
2
+
3
+ ```ruby
4
+ require "rubydb"
5
+
6
+ path = "tmp/first-database.rdb"
7
+ engine = RubyDB::Storage::Engine.new(path)
8
+ engine.execute("CREATE TABLE accounts (id INTEGER PRIMARY KEY, email TEXT UNIQUE NOT NULL)")
9
+ engine.execute("INSERT INTO accounts (id, email) VALUES (1, 'ada@example.test')")
10
+ puts engine.execute("SELECT id, email FROM accounts").inspect
11
+ engine.close
12
+ ```
13
+
14
+ The engine creates durable storage and WAL state under the selected path. Close
15
+ the engine cleanly and keep the database, WAL, and lock files together. A single
16
+ embedded path has one owner; use a server for multiple processes.
@@ -1,13 +1,13 @@
1
- # First query
2
-
3
- RubyDB executes the documented SQL subset through its lexer, parser, planner,
4
- and executor:
5
-
6
- ```ruby
7
- rows = engine.execute("SELECT id, email FROM accounts WHERE id = 1 ORDER BY id LIMIT 10")
8
- puts rows.inspect
9
- ```
10
-
11
- Use the Rails connection or client binding APIs for external values. Do not
12
- interpolate untrusted input into SQL. See [SQL compatibility](../sql/compatibility.md)
13
- for supported statements and explicit boundaries.
1
+ # First query
2
+
3
+ RubyDB executes the documented SQL subset through its lexer, parser, planner,
4
+ and executor:
5
+
6
+ ```ruby
7
+ rows = engine.execute("SELECT id, email FROM accounts WHERE id = 1 ORDER BY id LIMIT 10")
8
+ puts rows.inspect
9
+ ```
10
+
11
+ Use the Rails connection or client binding APIs for external values. Do not
12
+ interpolate untrusted input into SQL. See [SQL compatibility](../sql/compatibility.md)
13
+ for supported statements and explicit boundaries.