rubydb 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (436) hide show
  1. checksums.yaml +7 -0
  2. data/.gitattributes +20 -0
  3. data/.github/ISSUE_TEMPLATE/bug.yml +0 -0
  4. data/.github/ISSUE_TEMPLATE/feature.yml +0 -0
  5. data/.github/ISSUE_TEMPLATE/performance.yml +0 -0
  6. data/.github/ISSUE_TEMPLATE/security.yml +0 -0
  7. data/.github/PULL_REQUEST_TEMPLATE.md +15 -0
  8. data/.github/dependabot.yml +0 -0
  9. data/.github/workflows/benchmark.yml +26 -0
  10. data/.github/workflows/compatibility.yml +63 -0
  11. data/.github/workflows/fuzz.yml +33 -0
  12. data/.github/workflows/lint.yml +21 -0
  13. data/.github/workflows/operations.yml +24 -0
  14. data/.github/workflows/production-validation.yml +111 -0
  15. data/.github/workflows/release.yml +77 -0
  16. data/.github/workflows/security.yml +37 -0
  17. data/.github/workflows/test.yml +26 -0
  18. data/.github/workflows/workload.yml +58 -0
  19. data/.gitignore +43 -0
  20. data/.rspec +4 -0
  21. data/.rubocop.yml +44 -0
  22. data/.ruby-version +1 -0
  23. data/.standard.yml +14 -0
  24. data/ARCHITECTURE.md +21 -0
  25. data/CHANGELOG.md +31 -0
  26. data/CODE_OF_CONDUCT.md +13 -0
  27. data/CONTRIBUTING.md +29 -0
  28. data/GOVERNANCE.md +16 -0
  29. data/Gemfile +17 -0
  30. data/Gemfile.lock +71 -0
  31. data/LICENSE +60 -0
  32. data/README.md +162 -0
  33. data/ROADMAP.md +27 -0
  34. data/Rakefile +71 -0
  35. data/SECURITY.md +54 -0
  36. data/SUPPORT.md +14 -0
  37. data/adapters/activerecord/Gemfile +11 -0
  38. data/adapters/activerecord/README.md +9 -0
  39. data/adapters/activerecord/lib/active_record/connection_adapters/rubydb_adapter.rb +887 -0
  40. data/adapters/activerecord/rubydb-activerecord.gemspec +27 -0
  41. data/adapters/activerecord/spec/rubydb_adapter_integration_spec.rb +151 -0
  42. data/adapters/ruby/README.md +18 -0
  43. data/adapters/sequel/README.md +11 -0
  44. data/bin/rubydb +19 -0
  45. data/bin/rubydb-server +126 -0
  46. data/config/development.yml +0 -0
  47. data/config/monitoring/prometheus-alerts.yml +39 -0
  48. data/config/production.yml +36 -0
  49. data/config/test.yml +0 -0
  50. data/docs/README.md +71 -0
  51. data/docs/architecture/concurrency.md +14 -0
  52. data/docs/architecture/current-state.md +125 -0
  53. data/docs/architecture/execution-engine.md +25 -0
  54. data/docs/architecture/indexes.md +19 -0
  55. data/docs/architecture/mvcc.md +19 -0
  56. data/docs/architecture/overview.md +13 -0
  57. data/docs/architecture/pages.md +11 -0
  58. data/docs/architecture/production-roadmap.md +82 -0
  59. data/docs/architecture/query-planner.md +20 -0
  60. data/docs/architecture/recovery.md +18 -0
  61. data/docs/architecture/sql-engine.md +12 -0
  62. data/docs/architecture/storage-engine.md +14 -0
  63. data/docs/architecture/transactions.md +10 -0
  64. data/docs/architecture/wal.md +28 -0
  65. data/docs/cli-cheatsheet.md +98 -0
  66. data/docs/cli.md +275 -0
  67. data/docs/contributing/architecture.md +9 -0
  68. data/docs/contributing/benchmarking.md +14 -0
  69. data/docs/contributing/development.md +16 -0
  70. data/docs/contributing/release-process.md +49 -0
  71. data/docs/contributing/testing.md +16 -0
  72. data/docs/debugging.md +229 -0
  73. data/docs/developer/branching.md +10 -0
  74. data/docs/developer/database-diff.md +10 -0
  75. data/docs/developer/local-development.md +17 -0
  76. data/docs/developer/snapshots.md +9 -0
  77. data/docs/developer/temporal-data.md +10 -0
  78. data/docs/developer-guide.md +297 -0
  79. data/docs/getting-started/first-database.md +16 -0
  80. data/docs/getting-started/first-query.md +13 -0
  81. data/docs/getting-started/installation.md +19 -0
  82. data/docs/getting-started/local-to-production.md +300 -0
  83. data/docs/getting-started/quickstart.md +17 -0
  84. data/docs/getting-started/rails.md +16 -0
  85. data/docs/hardening_backlog.md +93 -0
  86. data/docs/lessons-learned.md +112 -0
  87. data/docs/operations/backups.md +33 -0
  88. data/docs/operations/disaster-recovery.md +31 -0
  89. data/docs/operations/failover.md +30 -0
  90. data/docs/operations/monitoring.md +25 -0
  91. data/docs/operations/production-guide.md +295 -0
  92. data/docs/operations/production-runbook.md +45 -0
  93. data/docs/operations/replication.md +33 -0
  94. data/docs/operations/restore.md +6 -0
  95. data/docs/operations/runbook.md +34 -0
  96. data/docs/operations/upgrades.md +14 -0
  97. data/docs/operations/workload-testing.md +17 -0
  98. data/docs/production-readiness.md +118 -0
  99. data/docs/production_validation.md +150 -0
  100. data/docs/rails/active-record.md +11 -0
  101. data/docs/rails/compatibility-guide.md +90 -0
  102. data/docs/rails/database-yml.md +92 -0
  103. data/docs/rails/installation.md +17 -0
  104. data/docs/rails/migrations.md +17 -0
  105. data/docs/rails/production.md +82 -0
  106. data/docs/rails/troubleshooting.md +18 -0
  107. data/docs/release.md +59 -0
  108. data/docs/server/architecture.md +10 -0
  109. data/docs/server/authentication.md +10 -0
  110. data/docs/server/configuration.md +16 -0
  111. data/docs/server/connection-pooling.md +10 -0
  112. data/docs/server/deployment.md +10 -0
  113. data/docs/server/protocol.md +12 -0
  114. data/docs/sql/compatibility-guide.md +82 -0
  115. data/docs/sql/compatibility.md +39 -0
  116. data/docs/sql/data-types.md +10 -0
  117. data/docs/sql/functions.md +9 -0
  118. data/docs/sql/joins.md +9 -0
  119. data/docs/sql/operators.md +9 -0
  120. data/docs/sql/sqlite-compatibility.md +21 -0
  121. data/docs/sql/syntax.md +10 -0
  122. data/docs/sql/transactions.md +10 -0
  123. data/docs/troubleshooting.md +244 -0
  124. data/exe/rubydb +23 -0
  125. data/lib/rubydb/backup/archive.rb +334 -0
  126. data/lib/rubydb/backup/backup.rb +401 -0
  127. data/lib/rubydb/backup/incremental.rb +353 -0
  128. data/lib/rubydb/backup/restore.rb +290 -0
  129. data/lib/rubydb/backup/snapshot.rb +267 -0
  130. data/lib/rubydb/backup/verification.rb +279 -0
  131. data/lib/rubydb/branching/branch.rb +181 -0
  132. data/lib/rubydb/branching/branch_manager.rb +311 -0
  133. data/lib/rubydb/branching/branch_metadata.rb +140 -0
  134. data/lib/rubydb/branching/checkout.rb +166 -0
  135. data/lib/rubydb/branching/copy_on_write.rb +272 -0
  136. data/lib/rubydb/branching/diff.rb +138 -0
  137. data/lib/rubydb/branching/merge.rb +285 -0
  138. data/lib/rubydb/build_info.rb +15 -0
  139. data/lib/rubydb/catalog/catalog.rb +391 -0
  140. data/lib/rubydb/catalog/column.rb +112 -0
  141. data/lib/rubydb/catalog/constraint.rb +180 -0
  142. data/lib/rubydb/catalog/database.rb +184 -0
  143. data/lib/rubydb/catalog/index.rb +97 -0
  144. data/lib/rubydb/catalog/schema.rb +103 -0
  145. data/lib/rubydb/catalog/sequence.rb +90 -0
  146. data/lib/rubydb/catalog/system_catalog.rb +698 -0
  147. data/lib/rubydb/catalog/table.rb +178 -0
  148. data/lib/rubydb/catalog/trigger.rb +102 -0
  149. data/lib/rubydb/catalog/view.rb +66 -0
  150. data/lib/rubydb/cli/application.rb +163 -0
  151. data/lib/rubydb/cli/commands/backup.rb +81 -0
  152. data/lib/rubydb/cli/commands/branch.rb +72 -0
  153. data/lib/rubydb/cli/commands/checkout.rb +54 -0
  154. data/lib/rubydb/cli/commands/create.rb +58 -0
  155. data/lib/rubydb/cli/commands/diff.rb +77 -0
  156. data/lib/rubydb/cli/commands/doctor.rb +74 -0
  157. data/lib/rubydb/cli/commands/drop.rb +57 -0
  158. data/lib/rubydb/cli/commands/init.rb +102 -0
  159. data/lib/rubydb/cli/commands/inspect.rb +95 -0
  160. data/lib/rubydb/cli/commands/merge.rb +63 -0
  161. data/lib/rubydb/cli/commands/migrate.rb +62 -0
  162. data/lib/rubydb/cli/commands/restart.rb +39 -0
  163. data/lib/rubydb/cli/commands/restore.rb +121 -0
  164. data/lib/rubydb/cli/commands/shell.rb +365 -0
  165. data/lib/rubydb/cli/commands/snapshot.rb +79 -0
  166. data/lib/rubydb/cli/commands/start.rb +82 -0
  167. data/lib/rubydb/cli/commands/status.rb +92 -0
  168. data/lib/rubydb/cli/commands/stop.rb +47 -0
  169. data/lib/rubydb/cli/commands/vacuum.rb +58 -0
  170. data/lib/rubydb/cli/formatter.rb +221 -0
  171. data/lib/rubydb/cli/output.rb +168 -0
  172. data/lib/rubydb/client/client.rb +304 -0
  173. data/lib/rubydb/client/connection.rb +415 -0
  174. data/lib/rubydb/client/connection_pool.rb +168 -0
  175. data/lib/rubydb/client/connection_url.rb +96 -0
  176. data/lib/rubydb/client/prepared_statement.rb +60 -0
  177. data/lib/rubydb/client/result.rb +123 -0
  178. data/lib/rubydb/client/statement.rb +52 -0
  179. data/lib/rubydb/client/transaction.rb +130 -0
  180. data/lib/rubydb/concurrency/concurrency.rb +19 -0
  181. data/lib/rubydb/concurrency/deadlock_detector.rb +150 -0
  182. data/lib/rubydb/concurrency/latch.rb +101 -0
  183. data/lib/rubydb/concurrency/lock_graph.rb +165 -0
  184. data/lib/rubydb/concurrency/mutex.rb +183 -0
  185. data/lib/rubydb/concurrency/rw_lock.rb +180 -0
  186. data/lib/rubydb/concurrency/scheduler.rb +250 -0
  187. data/lib/rubydb/concurrency/worker_pool.rb +143 -0
  188. data/lib/rubydb/configuration/config.rb +170 -0
  189. data/lib/rubydb/configuration/defaults.rb +179 -0
  190. data/lib/rubydb/configuration/environment.rb +152 -0
  191. data/lib/rubydb/configuration/parser.rb +185 -0
  192. data/lib/rubydb/configuration/validation.rb +221 -0
  193. data/lib/rubydb/constants.rb +74 -0
  194. data/lib/rubydb/constraints/check.rb +181 -0
  195. data/lib/rubydb/constraints/constraint.rb +101 -0
  196. data/lib/rubydb/constraints/foreign_key.rb +130 -0
  197. data/lib/rubydb/constraints/not_null.rb +64 -0
  198. data/lib/rubydb/constraints/primary_key.rb +99 -0
  199. data/lib/rubydb/constraints/unique.rb +108 -0
  200. data/lib/rubydb/constraints/validator.rb +350 -0
  201. data/lib/rubydb/errors/authentication_error.rb +10 -0
  202. data/lib/rubydb/errors/authorization_error.rb +23 -0
  203. data/lib/rubydb/errors/client_error.rb +10 -0
  204. data/lib/rubydb/errors/configuration_error.rb +10 -0
  205. data/lib/rubydb/errors/connection_error.rb +10 -0
  206. data/lib/rubydb/errors/constraint_error.rb +23 -0
  207. data/lib/rubydb/errors/corruption_error.rb +10 -0
  208. data/lib/rubydb/errors/database_error.rb +10 -0
  209. data/lib/rubydb/errors/error.rb +20 -0
  210. data/lib/rubydb/errors/execution_error.rb +10 -0
  211. data/lib/rubydb/errors/parser_error.rb +10 -0
  212. data/lib/rubydb/errors/recovery_error.rb +10 -0
  213. data/lib/rubydb/errors/replication_error.rb +10 -0
  214. data/lib/rubydb/errors/server_error.rb +6 -0
  215. data/lib/rubydb/errors/storage_error.rb +10 -0
  216. data/lib/rubydb/errors/transaction_error.rb +10 -0
  217. data/lib/rubydb/execution/aggregate_executor.rb +138 -0
  218. data/lib/rubydb/execution/delete_executor.rb +112 -0
  219. data/lib/rubydb/execution/distinct_executor.rb +135 -0
  220. data/lib/rubydb/execution/executor.rb +1192 -0
  221. data/lib/rubydb/execution/expression.rb +193 -0
  222. data/lib/rubydb/execution/index_scan.rb +142 -0
  223. data/lib/rubydb/execution/insert_executor.rb +217 -0
  224. data/lib/rubydb/execution/join_executor.rb +249 -0
  225. data/lib/rubydb/execution/limit_executor.rb +85 -0
  226. data/lib/rubydb/execution/optimizer.rb +215 -0
  227. data/lib/rubydb/execution/plan.rb +361 -0
  228. data/lib/rubydb/execution/planner.rb +546 -0
  229. data/lib/rubydb/execution/predicate.rb +235 -0
  230. data/lib/rubydb/execution/scan.rb +49 -0
  231. data/lib/rubydb/execution/sequential_scan.rb +63 -0
  232. data/lib/rubydb/execution/sort_executor.rb +185 -0
  233. data/lib/rubydb/execution/update_executor.rb +162 -0
  234. data/lib/rubydb/functions/aggregate.rb +70 -0
  235. data/lib/rubydb/functions/date_functions.rb +278 -0
  236. data/lib/rubydb/functions/function.rb +85 -0
  237. data/lib/rubydb/functions/json_functions.rb +215 -0
  238. data/lib/rubydb/functions/numeric_functions.rb +346 -0
  239. data/lib/rubydb/functions/scalar.rb +52 -0
  240. data/lib/rubydb/functions/string_functions.rb +383 -0
  241. data/lib/rubydb/functions/system_functions.rb +246 -0
  242. data/lib/rubydb/history/as_of.rb +238 -0
  243. data/lib/rubydb/history/change.rb +105 -0
  244. data/lib/rubydb/history/history.rb +131 -0
  245. data/lib/rubydb/history/history_manager.rb +229 -0
  246. data/lib/rubydb/history/temporal_query.rb +202 -0
  247. data/lib/rubydb/history/timeline.rb +144 -0
  248. data/lib/rubydb/indexes/btree.rb +186 -0
  249. data/lib/rubydb/indexes/btree_cursor.rb +258 -0
  250. data/lib/rubydb/indexes/btree_node.rb +385 -0
  251. data/lib/rubydb/indexes/hash_index.rb +150 -0
  252. data/lib/rubydb/indexes/index.rb +71 -0
  253. data/lib/rubydb/indexes/index_manager.rb +406 -0
  254. data/lib/rubydb/indexes/index_scan.rb +470 -0
  255. data/lib/rubydb/migrations/migration.rb +254 -0
  256. data/lib/rubydb/migrations/migration_lock.rb +146 -0
  257. data/lib/rubydb/migrations/migration_manager.rb +176 -0
  258. data/lib/rubydb/migrations/migration_version.rb +71 -0
  259. data/lib/rubydb/migrations/schema_diff.rb +211 -0
  260. data/lib/rubydb/migrations/schema_version.rb +64 -0
  261. data/lib/rubydb/monitoring/events.rb +160 -0
  262. data/lib/rubydb/monitoring/health.rb +222 -0
  263. data/lib/rubydb/monitoring/logger.rb +193 -0
  264. data/lib/rubydb/monitoring/metrics.rb +359 -0
  265. data/lib/rubydb/monitoring/performance.rb +176 -0
  266. data/lib/rubydb/monitoring/statistics.rb +170 -0
  267. data/lib/rubydb/mvcc/garbage_collector.rb +199 -0
  268. data/lib/rubydb/mvcc/mvcc.rb +16 -0
  269. data/lib/rubydb/mvcc/snapshot.rb +147 -0
  270. data/lib/rubydb/mvcc/vacuum.rb +180 -0
  271. data/lib/rubydb/mvcc/version.rb +106 -0
  272. data/lib/rubydb/mvcc/version_store.rb +398 -0
  273. data/lib/rubydb/mvcc/visibility.rb +109 -0
  274. data/lib/rubydb/protocol/capabilities.rb +125 -0
  275. data/lib/rubydb/protocol/decoder.rb +145 -0
  276. data/lib/rubydb/protocol/encoder.rb +136 -0
  277. data/lib/rubydb/protocol/handshake.rb +305 -0
  278. data/lib/rubydb/protocol/message.rb +121 -0
  279. data/lib/rubydb/protocol/protocol.rb +277 -0
  280. data/lib/rubydb/protocol/version.rb +54 -0
  281. data/lib/rubydb/rails/adapter.rb +239 -0
  282. data/lib/rubydb/rails/connection.rb +314 -0
  283. data/lib/rubydb/rails/database_statements.rb +122 -0
  284. data/lib/rubydb/rails/migration.rb +131 -0
  285. data/lib/rubydb/rails/quoting.rb +109 -0
  286. data/lib/rubydb/rails/result.rb +120 -0
  287. data/lib/rubydb/rails/schema_statements.rb +339 -0
  288. data/lib/rubydb/rails/transaction.rb +105 -0
  289. data/lib/rubydb/rails/type.rb +126 -0
  290. data/lib/rubydb/recovery/checkpoint.rb +257 -0
  291. data/lib/rubydb/recovery/consistency.rb +467 -0
  292. data/lib/rubydb/recovery/corruption_detector.rb +5 -0
  293. data/lib/rubydb/recovery/crash_recovery.rb +387 -0
  294. data/lib/rubydb/recovery/recovery_manager.rb +206 -0
  295. data/lib/rubydb/recovery/redo.rb +237 -0
  296. data/lib/rubydb/recovery/undo.rb +206 -0
  297. data/lib/rubydb/replication/failover.rb +5 -0
  298. data/lib/rubydb/replication/fencing.rb +63 -0
  299. data/lib/rubydb/replication/primary.rb +450 -0
  300. data/lib/rubydb/replication/replica.rb +384 -0
  301. data/lib/rubydb/replication/replication_log.rb +200 -0
  302. data/lib/rubydb/replication/replication_manager.rb +308 -0
  303. data/lib/rubydb/replication/replication_slot.rb +295 -0
  304. data/lib/rubydb/replication/replication_stream.rb +201 -0
  305. data/lib/rubydb/rubydb.rb +560 -0
  306. data/lib/rubydb/security/access_control.rb +254 -0
  307. data/lib/rubydb/security/audit_log.rb +213 -0
  308. data/lib/rubydb/security/authentication.rb +302 -0
  309. data/lib/rubydb/security/authorization.rb +282 -0
  310. data/lib/rubydb/security/credentials.rb +196 -0
  311. data/lib/rubydb/security/password.rb +215 -0
  312. data/lib/rubydb/security/permissions.rb +74 -0
  313. data/lib/rubydb/security/role.rb +101 -0
  314. data/lib/rubydb/security/user.rb +86 -0
  315. data/lib/rubydb/server/connection.rb +366 -0
  316. data/lib/rubydb/server/connection_pool.rb +193 -0
  317. data/lib/rubydb/server/lifecycle.rb +228 -0
  318. data/lib/rubydb/server/listener.rb +136 -0
  319. data/lib/rubydb/server/request_handler.rb +276 -0
  320. data/lib/rubydb/server/server.rb +364 -0
  321. data/lib/rubydb/server/session.rb +369 -0
  322. data/lib/rubydb/server/worker.rb +210 -0
  323. data/lib/rubydb/server/worker_pool.rb +168 -0
  324. data/lib/rubydb/sql/ast/alter_table.rb +169 -0
  325. data/lib/rubydb/sql/ast/begin_transaction.rb +47 -0
  326. data/lib/rubydb/sql/ast/commit.rb +37 -0
  327. data/lib/rubydb/sql/ast/constraint.rb +83 -0
  328. data/lib/rubydb/sql/ast/create_database.rb +41 -0
  329. data/lib/rubydb/sql/ast/create_index.rb +61 -0
  330. data/lib/rubydb/sql/ast/create_schema.rb +52 -0
  331. data/lib/rubydb/sql/ast/create_table.rb +187 -0
  332. data/lib/rubydb/sql/ast/delete.rb +54 -0
  333. data/lib/rubydb/sql/ast/drop_database.rb +41 -0
  334. data/lib/rubydb/sql/ast/drop_index.rb +41 -0
  335. data/lib/rubydb/sql/ast/drop_schema.rb +49 -0
  336. data/lib/rubydb/sql/ast/drop_table.rb +49 -0
  337. data/lib/rubydb/sql/ast/explain.rb +64 -0
  338. data/lib/rubydb/sql/ast/expression.rb +604 -0
  339. data/lib/rubydb/sql/ast/insert.rb +82 -0
  340. data/lib/rubydb/sql/ast/node.rb +42 -0
  341. data/lib/rubydb/sql/ast/rollback.rb +63 -0
  342. data/lib/rubydb/sql/ast/savepoint.rb +59 -0
  343. data/lib/rubydb/sql/ast/select.rb +88 -0
  344. data/lib/rubydb/sql/ast/set_operation.rb +20 -0
  345. data/lib/rubydb/sql/ast/trigger.rb +29 -0
  346. data/lib/rubydb/sql/ast/update.rb +88 -0
  347. data/lib/rubydb/sql/ast/vacuum.rb +19 -0
  348. data/lib/rubydb/sql/ast/view.rb +32 -0
  349. data/lib/rubydb/sql/ast/with.rb +32 -0
  350. data/lib/rubydb/sql/grammar.rb +86 -0
  351. data/lib/rubydb/sql/keywords.rb +156 -0
  352. data/lib/rubydb/sql/lexer.rb +214 -0
  353. data/lib/rubydb/sql/operators.rb +100 -0
  354. data/lib/rubydb/sql/parser.rb +1185 -0
  355. data/lib/rubydb/sql/planner/analyzer.rb +302 -0
  356. data/lib/rubydb/sql/planner/binder.rb +550 -0
  357. data/lib/rubydb/sql/planner/type_checker.rb +431 -0
  358. data/lib/rubydb/sql/token.rb +210 -0
  359. data/lib/rubydb/storage/buffer_frame.rb +44 -0
  360. data/lib/rubydb/storage/buffer_pool.rb +155 -0
  361. data/lib/rubydb/storage/database_lock.rb +74 -0
  362. data/lib/rubydb/storage/deserializer.rb +342 -0
  363. data/lib/rubydb/storage/engine.rb +2338 -0
  364. data/lib/rubydb/storage/file_manager.rb +187 -0
  365. data/lib/rubydb/storage/free_space_map.rb +81 -0
  366. data/lib/rubydb/storage/page.rb +94 -0
  367. data/lib/rubydb/storage/page_allocator.rb +855 -0
  368. data/lib/rubydb/storage/page_header.rb +67 -0
  369. data/lib/rubydb/storage/page_manager.rb +131 -0
  370. data/lib/rubydb/storage/record.rb +58 -0
  371. data/lib/rubydb/storage/row.rb +78 -0
  372. data/lib/rubydb/storage/serializer.rb +51 -0
  373. data/lib/rubydb/storage/storage_layout.rb +151 -0
  374. data/lib/rubydb/storage/storage_manager.rb +114 -0
  375. data/lib/rubydb/storage/tuple.rb +461 -0
  376. data/lib/rubydb/storage/visibility_map.rb +984 -0
  377. data/lib/rubydb/transactions/commit_manager.rb +220 -0
  378. data/lib/rubydb/transactions/isolation.rb +98 -0
  379. data/lib/rubydb/transactions/lock.rb +76 -0
  380. data/lib/rubydb/transactions/lock_manager.rb +362 -0
  381. data/lib/rubydb/transactions/savepoint.rb +143 -0
  382. data/lib/rubydb/transactions/transaction.rb +215 -0
  383. data/lib/rubydb/transactions/transaction_id.rb +84 -0
  384. data/lib/rubydb/transactions/transaction_log.rb +257 -0
  385. data/lib/rubydb/transactions/transaction_manager.rb +435 -0
  386. data/lib/rubydb/types/bigint.rb +36 -0
  387. data/lib/rubydb/types/blob.rb +37 -0
  388. data/lib/rubydb/types/boolean.rb +34 -0
  389. data/lib/rubydb/types/date.rb +39 -0
  390. data/lib/rubydb/types/decimal.rb +48 -0
  391. data/lib/rubydb/types/float.rb +34 -0
  392. data/lib/rubydb/types/integer.rb +36 -0
  393. data/lib/rubydb/types/json.rb +41 -0
  394. data/lib/rubydb/types/null.rb +34 -0
  395. data/lib/rubydb/types/smallint.rb +36 -0
  396. data/lib/rubydb/types/text.rb +37 -0
  397. data/lib/rubydb/types/time.rb +46 -0
  398. data/lib/rubydb/types/timestamp.rb +39 -0
  399. data/lib/rubydb/types/type.rb +119 -0
  400. data/lib/rubydb/types/uuid.rb +47 -0
  401. data/lib/rubydb/types/varchar.rb +37 -0
  402. data/lib/rubydb/version.rb +33 -0
  403. data/lib/rubydb/wal/archive.rb +193 -0
  404. data/lib/rubydb/wal/checkpoint.rb +183 -0
  405. data/lib/rubydb/wal/lsn.rb +94 -0
  406. data/lib/rubydb/wal/reader.rb +260 -0
  407. data/lib/rubydb/wal/record.rb +105 -0
  408. data/lib/rubydb/wal/segment.rb +193 -0
  409. data/lib/rubydb/wal/wal.rb +452 -0
  410. data/lib/rubydb/wal/writer.rb +236 -0
  411. data/lib/rubydb.rb +7 -0
  412. data/packaging/docker/Dockerfile +36 -0
  413. data/packaging/docker/docker-compose.failover.yml +43 -0
  414. data/packaging/docker/entrypoint.sh +15 -0
  415. data/packaging/homebrew/rubydb.rb +19 -0
  416. data/packaging/systemd/rubydb.service +22 -0
  417. data/rubydb.gemspec +57 -0
  418. data/scripts/benchmark +7 -0
  419. data/scripts/durability_drill +37 -0
  420. data/scripts/format +0 -0
  421. data/scripts/fuzz +63 -0
  422. data/scripts/lint +0 -0
  423. data/scripts/release +50 -0
  424. data/scripts/release_check +43 -0
  425. data/scripts/replication_failover_drill +250 -0
  426. data/scripts/replication_network_failover_drill +255 -0
  427. data/scripts/restore_drill +45 -0
  428. data/scripts/security +0 -0
  429. data/scripts/setup +0 -0
  430. data/scripts/test +0 -0
  431. data/scripts/test-chaos +0 -0
  432. data/scripts/test-crash +0 -0
  433. data/scripts/test-integration +0 -0
  434. data/scripts/test-unit +0 -0
  435. data/visibility_map.json +1 -0
  436. metadata +538 -0
@@ -0,0 +1,19 @@
1
+ # Indexes
2
+
3
+ RubyDB provides B-tree indexes for supported table columns, including unique
4
+ indexes. The executor can use indexes for eligible lookups while preserving
5
+ correctness through table visibility and transaction rules.
6
+
7
+ Index metadata is persisted with the catalog. Index creation, DML maintenance,
8
+ rollback, deep splits, reopen, and failure handling must remain consistent. If
9
+ index recovery fails, startup must fail visibly; never silently rebuild or claim
10
+ an index is healthy without verification.
11
+
12
+ ## Change checklist
13
+
14
+ Index changes require coverage for empty and populated tables, duplicate and
15
+ null keys, deep splits, deletes, rollback, crash/replay, compaction, and
16
+ concurrent readers. Compare an index lookup with a table scan before and after
17
+ reopen. A required-index persistence error must abort the owning operation and
18
+ be observable by the caller. See [troubleshooting](../troubleshooting.md) for
19
+ the safe response to suspected index corruption.
@@ -0,0 +1,19 @@
1
+ # MVCC support status
2
+
3
+ RubyDB currently provides safe READ COMMITTED behavior for the storage engine.
4
+ Uncommitted row versions are visible to their owning transaction and hidden
5
+ from other transactions. Committed versions become visible after the durable
6
+ COMMIT WAL record is written. Deleted physical records remain on pages until
7
+ vacuum/compaction and are excluded from ordinary scans.
8
+
9
+ REPEATABLE READ uses persisted historical row versions and transaction
10
+ snapshots. SERIALIZABLE uses snapshot validation over tracked row read/write
11
+ keys and conservative table predicates. A newer committed version intersecting
12
+ the dependency set aborts commit with a serialization failure. Table-level
13
+ predicate tracking prevents phantoms at the cost of false-positive conflicts;
14
+ exact index-range predicate locking is a future optimization.
15
+
16
+ The version store is persisted atomically and supports safe-point vacuuming.
17
+ Vacuum never removes active versions and retains the newest committed base
18
+ version needed by an active reader; older committed history is removable only
19
+ when its commit ID precedes the oldest active transaction.
@@ -0,0 +1,13 @@
1
+ # Architecture overview
2
+
3
+ RubyDB separates logical query processing from durable state changes. SQL is
4
+ lexed into tokens, parsed into an AST, bound against catalog metadata, planned,
5
+ and executed against the storage engine. DML is coordinated with transactions
6
+ and WAL before durable acknowledgement.
7
+
8
+ The catalog defines tables, columns, constraints, views, and indexes. The page
9
+ manager and buffer pool provide durable storage. Recovery replays valid WAL and
10
+ rejects malformed or uncertain state. Server/client mode isolates application
11
+ processes from the exclusive embedded owner.
12
+
13
+ Read the [current-state audit](current-state.md) before relying on any feature.
@@ -0,0 +1,11 @@
1
+ # Pages and file layout
2
+
3
+ RubyDB stores fixed-size pages containing validated headers, checksums, and
4
+ typed records. Page allocation, metadata, table data, and index pages are
5
+ managed by the storage layer; callers must not edit database files directly.
6
+
7
+ The page format is versioned. Unknown formats, invalid page sizes, malformed
8
+ headers, and checksum failures fail closed during open or recovery. Keep the
9
+ database, WAL, catalog metadata, and lock files together when copying or
10
+ restoring a database. See [storage format](../../spec/storage/format.md) and
11
+ [upgrade guidance](../operations/upgrades.md).
@@ -0,0 +1,82 @@
1
+ # RubyDB production roadmap
2
+
3
+ ## Current baseline
4
+
5
+ The project has a meaningful repository skeleton and substantial design intent. It already includes modules for storage, indexing, transactions, MVCC, SQL parts, server, replication, and backup flows. However, the implementation still lacks enough end-to-end proof for durable, crash-safe operation.
6
+
7
+ ## Prioritized roadmap
8
+
9
+ ### Phase 0 — Repository stabilization
10
+
11
+ - fix library loading and Ruby compatibility metadata
12
+ - verify the project loads via `require "rubydb"`
13
+ - establish a truthful project health baseline
14
+ - add characterization tests for current public behavior
15
+
16
+ ### Phase 1 — Storage durability
17
+
18
+ - define a stable on-disk page format and file format
19
+ - add checksums, header validation, and corruption detection
20
+ - ensure WAL durability precedes page flush semantics
21
+ - add real restart tests and crash tests
22
+
23
+ ### Phase 2 — Transaction correctness
24
+
25
+ - complete transaction lifecycle handling
26
+ - prove commit/rollback semantics under restart
27
+ - validate isolation semantics and visibility rules
28
+ - add deadlock and lock manager tests
29
+
30
+ ### Phase 3 — Catalog and indexes
31
+
32
+ - persist catalog metadata consistently
33
+ - maintain indexes correctly during DML
34
+ - verify index consistency after crash and rollback
35
+ - document supported SQL compatibility targets honestly
36
+
37
+ ### Phase 4 — SQL execution and optimizer
38
+
39
+ - validate end-to-end SQL execution
40
+ - ensure binder, planner, and executor are integrated
41
+ - add integration tests for basic DML and queries
42
+ - implement `EXPLAIN` only with real, honest plan output
43
+
44
+ ### Phase 5 — Server and security
45
+
46
+ - harden server lifecycle and malformed-client handling
47
+ - enforce connection limits and timeouts
48
+ - check authorization before execution
49
+ - validate auth flows and access controls
50
+
51
+ ### Phase 6 — Backup, recovery, and replication
52
+
53
+ - prove full backup + restore works from a clean environment
54
+ - validate point-in-time or incremental recovery only when implemented correctly
55
+ - only add replication once WAL recovery is demonstrably correct
56
+
57
+ ### Phase 7 — Monitoring and production readiness
58
+
59
+ - add structured logging and health/readiness checks
60
+ - expose operational metrics
61
+ - create a final production-readiness audit with limitations and unsupported features
62
+
63
+ ## Hard gates before declaring production readiness
64
+
65
+ RubyDB should not claim production readiness until it demonstrates all of the following:
66
+
67
+ - updated data survives restart
68
+ - crash recovery is validated with subprocess termination tests
69
+ - transactions preserve isolation semantics
70
+ - WAL ordering is durable and replayed correctly
71
+ - index and table data stay consistent
72
+ - authZ/authN is enforced over real operations
73
+ - backup and restore work in a clean environment
74
+ - server operations handle failures safely without whole-process crashes
75
+
76
+ ## Decision rule
77
+
78
+ The repository must prefer correctness over feature breadth. A smaller well-tested engine is more valuable than a broad but unsafe one.
79
+
80
+ ## Long-term target
81
+
82
+ The final target is not a fake "PostgreSQL clone" or a marketing demo. The target is a Ruby-first, SQLite-like development experience paired with durable, crash-safe relational database semantics that can be validated under real workloads.
@@ -0,0 +1,20 @@
1
+ # Query planner
2
+
3
+ The planner binds identifiers to catalog columns, validates expressions, and
4
+ builds executable plans. It supports sequential/index scans, safe inner-join
5
+ reordering, outer joins, filters, grouping, sorting, limits, set operations,
6
+ subqueries, CTEs, and window operations in the documented SQL surface.
7
+
8
+ Plans must preserve SQL null, ordering, grouping, and transaction visibility
9
+ semantics. `EXPLAIN` reports the actual selected plan; it is not a performance
10
+ promise. Use the benchmark and workload harnesses to establish capacity on the
11
+ target hardware.
12
+
13
+ ## Plan review
14
+
15
+ For every rewrite, compare the optimized and baseline results on empty,
16
+ duplicate, null, and concurrent data. Include joins with unmatched rows,
17
+ grouping, limits, and repeated execution with bound parameters. Measure plan
18
+ selection, execution, lock wait, and I/O separately. A plan that is faster but
19
+ changes cardinality or ordering is incorrect; add the minimized query as a
20
+ regression spec before merging.
@@ -0,0 +1,18 @@
1
+ # Recovery architecture
2
+
3
+ Recovery validates the database format, page headers, checksums, metadata, and
4
+ WAL before making state available. Valid committed WAL records may be replayed;
5
+ malformed or ambiguous records fail closed and require operator investigation.
6
+
7
+ Recovery procedures work from a preserved source or verified backup. Restore to
8
+ a new directory, run dry-run and checksum checks, reopen the restored engine,
9
+ compare schema/row evidence, and only then switch application traffic. See the
10
+ [disaster recovery](../operations/disaster-recovery.md) procedure.
11
+
12
+ ## Failure handling
13
+
14
+ Recovery is allowed to replay only validated, committed records. Truncated
15
+ frames, invalid checksums, impossible LSNs, and incomplete metadata must stop
16
+ startup with an actionable error. Preserve the original directory and WAL;
17
+ repair or compaction experiments belong on a copy. The [debugging playbook](../debugging.md)
18
+ lists the evidence to collect.
@@ -0,0 +1,12 @@
1
+ # SQL engine
2
+
3
+ RubyDB implements a documented RubyDB SQL subset with a tested common
4
+ SQLite-style application profile. Covered features include CRUD, constraints,
5
+ joins, grouping/aggregates, transactions/savepoints, CTEs, subqueries, set
6
+ operations, upserts, windows, indexes, views, and maintenance statements listed
7
+ in [SQL compatibility](../sql/compatibility.md).
8
+
9
+ This is not complete PostgreSQL, MySQL, or SQLite dialect/file-format
10
+ compatibility. Unsupported syntax must fail explicitly. Applications migrating
11
+ from another engine must run their own schema, query, migration, and error
12
+ behavior suite.
@@ -0,0 +1,14 @@
1
+ # Storage engine
2
+
3
+ The storage engine owns the database path, page manager, buffer pool, catalog,
4
+ indexes, WAL, and recovery lifecycle. It serializes durable changes and exposes
5
+ Ruby-native operations used by the SQL executor and adapters.
6
+
7
+ Embedded ownership is exclusive and enforced with an operating-system lock.
8
+ Multiple processes must use the server. On failure, callers should preserve the
9
+ original directory and recover into a new destination; do not delete WAL or
10
+ overwrite the source during investigation.
11
+
12
+ Storage changes require reopen, crash, fault-injection, corruption, compaction,
13
+ and backup/restore coverage. The [lessons learned](../lessons-learned.md) page
14
+ explains why these are separate guarantees.
@@ -0,0 +1,10 @@
1
+ # Transactions
2
+
3
+ Transactions group mutations into a commit or rollback boundary. WAL commit and
4
+ flush ordering precede durable acknowledgement. MVCC determines visibility,
5
+ while lock management protects conflicting writes and resolves wait-for cycles.
6
+
7
+ Supported controls include `BEGIN`, `COMMIT`, `ROLLBACK`, savepoints, and
8
+ rollback to savepoint. A deadlock victim is rolled back and must retry the whole
9
+ application unit as appropriate. Test isolation and recovery together; a green
10
+ single-thread transaction test does not prove concurrent correctness.
@@ -0,0 +1,28 @@
1
+ # RubyDB WAL format
2
+
3
+ Each WAL segment is a sequence of independently framed records:
4
+
5
+ ```text
6
+ 4-byte unsigned big-endian payload length
7
+ 16-byte ASCII SHA-256 checksum prefix
8
+ UTF-8 JSON payload
9
+ ```
10
+
11
+ The checksum covers the JSON payload only. A record's LSN offset is the byte
12
+ offset of its frame header within the segment, so readers can advance exactly
13
+ one frame at a time. If the final frame is incomplete, recovery stops at that
14
+ tail and retains all complete preceding records.
15
+
16
+ WAL data is flushed and fsynced before the storage engine flushes data pages
17
+ when synchronous WAL mode is enabled (the default). A successful `write` is
18
+ therefore durable only after the WAL sync completes; callers must still use
19
+ the engine's transaction/close APIs to coordinate higher-level durability.
20
+
21
+ ## Transaction recovery records
22
+
23
+ Mutation records include the transaction ID. UPDATE records include the
24
+ previous column values, and DELETE records include the previous row values and
25
+ columns. Recovery replays mutations for transactions with a durable COMMIT
26
+ record. Transactions that have a BEGIN but no COMMIT are undone in reverse WAL
27
+ order. Recovery uses the existing row version for DELETE undo and does not
28
+ create a replacement row with a new identifier.
@@ -0,0 +1,98 @@
1
+ # RubyDB CLI cheat sheet
2
+
3
+ Replace paths and names before running. Put global options before the command.
4
+
5
+ ## Global
6
+
7
+ ```sh
8
+ rubydb --help
9
+ rubydb --version
10
+ rubydb --config config/production.yml --env production --no-color status --json
11
+ ```
12
+
13
+ ## Database lifecycle
14
+
15
+ ```sh
16
+ rubydb init --name app --dir data
17
+ rubydb create --database data/app.rdb
18
+ rubydb status --json
19
+ rubydb doctor --quick --json
20
+ rubydb inspect --database data/app.rdb --stats --wal
21
+ rubydb drop --database data/app.rdb # prompts
22
+ rubydb drop --database data/app.rdb --force # destructive
23
+ ```
24
+
25
+ ## Server
26
+
27
+ ```sh
28
+ rubydb --env production start --host 127.0.0.1 --port 7432 \
29
+ --data-dir /var/lib/rubydb/data --log-dir /var/log/rubydb \
30
+ --pid-file /run/rubydb.pid
31
+ rubydb stop --pid-file /run/rubydb.pid
32
+ rubydb restart --pid-file /run/rubydb.pid
33
+ ```
34
+
35
+ ## SQL shell
36
+
37
+ ```sh
38
+ rubydb shell --database app --table
39
+ rubydb shell --database app --json
40
+ ```
41
+
42
+ Inside shell: `.help` `.tables` `.table users` `.schema users` `.status`
43
+ `.begin` `.commit` `.rollback` `.explain SELECT ...` `.exit`.
44
+
45
+ ## Migrations
46
+
47
+ ```sh
48
+ rubydb migrate --database data/app.rdb --path db/migrate --dry-run
49
+ rubydb migrate --database data/app.rdb --path db/migrate
50
+ rubydb migrate --database data/app.rdb --path db/migrate --down --steps 1
51
+ ```
52
+
53
+ ## Backups and restore
54
+
55
+ ```sh
56
+ rubydb backup --database data/app.rdb --dir backups --type full --compress
57
+ rubydb backup --database data/app.rdb --dir backups --type incremental
58
+ rubydb backup --database data/app.rdb --dir backups --type differential
59
+ rubydb restore --dir backups --latest --dry-run
60
+ rubydb restore --dir backups --latest
61
+ rubydb restore --dir backups --backup NAME --point-in-time 2026-09-09T12:00:00Z
62
+ ```
63
+
64
+ ## Snapshots and branches
65
+
66
+ ```sh
67
+ rubydb snapshot --database data/app.rdb --dir snapshots --name before-change
68
+ rubydb snapshot --database data/app.rdb --dir snapshots --list
69
+ rubydb branch --database data/app.rdb --branch-dir branches --list
70
+ rubydb branch --database data/app.rdb --branch-dir branches --create feature --from main
71
+ rubydb checkout --database data/app.rdb --branch-dir branches feature
72
+ rubydb diff --database data/app.rdb --branch-dir branches main feature --summary
73
+ rubydb merge --database data/app.rdb --branch-dir branches feature --into main
74
+ ```
75
+
76
+ ## Maintenance and release checks
77
+
78
+ ```sh
79
+ rubydb vacuum --database data/app.rdb --dry-run
80
+ rubydb vacuum --database data/app.rdb --full --analyze
81
+ ruby scripts/durability_drill
82
+ ruby scripts/replication_failover_drill
83
+ ruby scripts/replication_network_failover_drill
84
+ ruby scripts/release
85
+ ```
86
+
87
+ ## Safety rules
88
+
89
+ - Back up and verify before `migrate`, `merge`, `checkout`, `vacuum --full`, or
90
+ any restore.
91
+ - Treat `drop --force`, `create --force`, restore `--force`, and process `--force`
92
+ stop as destructive operations.
93
+ - Preserve database, WAL, lock, metadata, logs, and configuration together.
94
+ - Use one embedded owner; use server/client mode for multiple processes.
95
+ - Never put passwords, tokens, private keys, or API keys in commands committed
96
+ to source control or in captured output.
97
+ - A successful CLI exit is not a universal production certification; retain the
98
+ workload, restore, failover, CI, and security evidence.
data/docs/cli.md ADDED
@@ -0,0 +1,275 @@
1
+ # RubyDB CLI guide
2
+
3
+ The `rubydb` command manages local databases, server processes, migrations,
4
+ backups, snapshots, branches, inspection, maintenance, and health checks. Run
5
+ `rubydb --help` or `rubydb <command> --help` for the installed command's live
6
+ options.
7
+
8
+ ## Command shape
9
+
10
+ ```text
11
+ rubydb [global options] <command> [command options]
12
+ ```
13
+
14
+ Global options must appear before the command:
15
+
16
+ ```sh
17
+ rubydb --version
18
+ rubydb --no-color --env production status --json
19
+ rubydb --config config/production.yml --env production doctor --json
20
+ ```
21
+
22
+ Global options are `--config FILE`, `--env ENV`, `--verbose`, `--quiet`,
23
+ `--no-color`, `--help`, and `--version`. Commands return zero on success and a
24
+ non-zero status on failure. Keep `RUBYDB_DEBUG=1` available when a failed
25
+ command needs a Ruby backtrace; never include secrets in captured logs.
26
+
27
+ ## Initialize or create a database
28
+
29
+ `init` creates a named database under a data directory and installs the default
30
+ RubyDB metadata/users/sessions tables:
31
+
32
+ ```sh
33
+ rubydb init --name app --dir data
34
+ ```
35
+
36
+ `create` creates an empty database file:
37
+
38
+ ```sh
39
+ rubydb create --name app --dir data
40
+ rubydb create --database /var/lib/rubydb/app.rdb
41
+ ```
42
+
43
+ `--force` is destructive for an existing database. Use it only for a disposable
44
+ development directory after verifying the path. It is not a production reset
45
+ procedure.
46
+
47
+ ## Start, stop, restart, and status
48
+
49
+ Start a foreground server for development or a service manager:
50
+
51
+ ```sh
52
+ rubydb --env production start \
53
+ --host 127.0.0.1 \
54
+ --port 7432 \
55
+ --data-dir /var/lib/rubydb/data \
56
+ --log-dir /var/log/rubydb \
57
+ --pid-file /run/rubydb.pid
58
+ ```
59
+
60
+ Use `--daemon` only when an external service manager is not supervising the
61
+ process. Prefer systemd, a container supervisor, or an equivalent process
62
+ manager for production. Stop gracefully first:
63
+
64
+ ```sh
65
+ rubydb stop --pid-file /run/rubydb.pid
66
+ rubydb restart --pid-file /run/rubydb.pid
67
+ ```
68
+
69
+ `--force` sends a hard termination and can interrupt active work; reserve it
70
+ for a confirmed hung process after preserving logs and checking recovery needs.
71
+
72
+ Inspect local status:
73
+
74
+ ```sh
75
+ rubydb --env production status
76
+ rubydb --env production status --json > status.json
77
+ ```
78
+
79
+ Use JSON status in monitoring, but combine it with the server readiness endpoint,
80
+ Prometheus metrics, process state, disk-space checks, and application probes.
81
+
82
+ ## Doctor and inspect
83
+
84
+ `doctor` runs health/integrity checks. Start with a read-only quick check:
85
+
86
+ ```sh
87
+ rubydb doctor --quick --json
88
+ rubydb doctor --json > doctor.json
89
+ ```
90
+
91
+ `doctor --fix` may perform supported repairs. Take and verify a backup first,
92
+ stop application writes, record the output, and review every proposed action:
93
+
94
+ ```sh
95
+ rubydb doctor --fix
96
+ ```
97
+
98
+ `inspect` reports storage internals without replacing a backup or integrity
99
+ verification:
100
+
101
+ ```sh
102
+ rubydb inspect --database data/app.rdb --stats --wal
103
+ rubydb inspect --database data/app.rdb --pages --indexes
104
+ rubydb inspect --database data/app.rdb --table users
105
+ ```
106
+
107
+ Use `inspect --wal` to correlate WAL growth with checkpoint/backup incidents.
108
+ Do not edit files based on an inspection result; use the documented recovery
109
+ procedure.
110
+
111
+ ## SQL shell
112
+
113
+ Open the configured local database shell:
114
+
115
+ ```sh
116
+ rubydb shell --database app
117
+ ```
118
+
119
+ Output modes include `--table`, `--json`, and `--csv`. The shell supports these
120
+ dot-commands:
121
+
122
+ | Command | Purpose |
123
+ | --- | --- |
124
+ | `.help`, `.?` | Show shell help |
125
+ | `.exit`, `.quit` | Exit safely |
126
+ | `.status` | Show connection/database status |
127
+ | `.tables` | List tables |
128
+ | `.table NAME` | Describe columns and constraints |
129
+ | `.schema NAME` | Show a table definition |
130
+ | `.db NAME` | Switch configured local database |
131
+ | `.begin`, `.txn` | Begin a transaction |
132
+ | `.commit` | Commit the active transaction |
133
+ | `.rollback` | Roll back the active transaction |
134
+ | `.explain SQL` | Show the selected query plan |
135
+ | `.clear`, `.cls` | Clear the terminal |
136
+
137
+ Example:
138
+
139
+ ```text
140
+ users> .tables
141
+ users> SELECT id, email FROM users ORDER BY id LIMIT 20;
142
+ users> .begin
143
+ users> UPDATE users SET active = TRUE WHERE id = 1;
144
+ users> .rollback
145
+ users> .exit
146
+ ```
147
+
148
+ Use bound parameters from application/client APIs for untrusted values. The
149
+ shell is an administrative tool, not a connection-pool substitute. For
150
+ multiple application processes, use the managed server and client libraries;
151
+ never open one embedded path concurrently from independent processes.
152
+
153
+ ## Migrations
154
+
155
+ Run migrations from the default `db/migrate` directory:
156
+
157
+ ```sh
158
+ rubydb migrate --database data/app.rdb --path db/migrate
159
+ rubydb migrate --database data/app.rdb --path db/migrate --dry-run
160
+ ```
161
+
162
+ To roll back, specify the number of steps and use a verified backup boundary:
163
+
164
+ ```sh
165
+ rubydb migrate --database data/app.rdb --path db/migrate --down --steps 1
166
+ ```
167
+
168
+ `--version VERSION` targets a migration version. Before applying migrations in
169
+ production, stop or coordinate writes, create a verified backup, test against a
170
+ populated restore, run schema dump/load and application smoke queries, and keep
171
+ the pre-migration directory until rollback closes. Applied migration checksum
172
+ changes fail closed.
173
+
174
+ ## Backups and restore
175
+
176
+ Create a verified full backup:
177
+
178
+ ```sh
179
+ rubydb backup --database data/app.rdb --dir backups --type full --compress
180
+ ```
181
+
182
+ The supported types are `full`, `incremental`, and `differential`. Use
183
+ `--no-verify` only for controlled diagnostics; a release or production backup
184
+ should verify. Retain the manifest, checksum, database version, and required
185
+ WAL chain together.
186
+
187
+ List or dry-run a restore before changing a live path:
188
+
189
+ ```sh
190
+ rubydb restore --dir backups --latest --dry-run
191
+ rubydb restore --database data/app.rdb --dir backups --backup BACKUP_NAME --dry-run
192
+ ```
193
+
194
+ Restore to a new inactive directory whenever possible. `--force` is destructive
195
+ and must not overwrite the only source. After restore, reopen the engine, verify
196
+ schema and row evidence, run application smoke queries, and record RPO/RTO.
197
+ Point-in-time restore uses `--point-in-time TIME` only when the required WAL
198
+ chain is present and verified.
199
+
200
+ ## Snapshots
201
+
202
+ Snapshots are useful for staging, inspection, and branch workflows:
203
+
204
+ ```sh
205
+ rubydb snapshot --database data/app.rdb --dir snapshots --name before-migration
206
+ rubydb snapshot --database data/app.rdb --dir snapshots --list
207
+ rubydb snapshot --database data/app.rdb --dir snapshots --restore before-migration
208
+ rubydb snapshot --database data/app.rdb --dir snapshots --delete old-snapshot
209
+ ```
210
+
211
+ Keep snapshot metadata/checksums and do not treat snapshots as the only backup.
212
+ Restore into a separate destination before routing traffic.
213
+
214
+ ## Branches and database diff
215
+
216
+ List or create a branch:
217
+
218
+ ```sh
219
+ rubydb branch --database data/app.rdb --branch-dir branches --list
220
+ rubydb branch --database data/app.rdb --branch-dir branches --create feature-x --from main
221
+ rubydb checkout --database data/app.rdb --branch-dir branches feature-x
222
+ rubydb checkout --database data/app.rdb --branch-dir branches --create experiment
223
+ ```
224
+
225
+ Review changes before merging:
226
+
227
+ ```sh
228
+ rubydb diff --database data/app.rdb --branch-dir branches main feature-x
229
+ rubydb diff --database data/app.rdb --branch-dir branches --summary main feature-x
230
+ rubydb diff --database data/app.rdb --branch-dir branches --table users main feature-x
231
+ rubydb merge --database data/app.rdb --branch-dir branches feature-x --into main
232
+ ```
233
+
234
+ `merge` supports the configured strategy and `--no-commit`. Take a verified
235
+ backup, review the diff, test the target with representative data, and preserve
236
+ the pre-merge state. Branches are not replication or backup replacements.
237
+
238
+ ## Vacuum and maintenance
239
+
240
+ Preview maintenance first:
241
+
242
+ ```sh
243
+ rubydb vacuum --database data/app.rdb --dry-run
244
+ rubydb vacuum --database data/app.rdb --table users
245
+ rubydb vacuum --database data/app.rdb --full --analyze
246
+ ```
247
+
248
+ Run full vacuum only during an approved maintenance window. Long-running
249
+ transactions may retain versions and prevent space reclamation. Monitor free
250
+ space, WAL/checkpoint behavior, query latency, and active transaction age.
251
+
252
+ ## Production command sequences
253
+
254
+ ### Pre-deployment
255
+
256
+ ```sh
257
+ rubydb --config config/production.yml --env production doctor --quick --json
258
+ rubydb --config config/production.yml --env production status --json
259
+ rubydb backup --database /var/lib/rubydb/app.rdb --dir /var/backups/rubydb --type full --compress
260
+ rubydb migrate --database /var/lib/rubydb/app.rdb --path db/migrate --dry-run
261
+ ```
262
+
263
+ ### Incident preservation
264
+
265
+ Stop writes, preserve the database/WAL/log/configuration set, copy it to an
266
+ incident directory, and then use `doctor --quick`, `inspect --stats --wal`, and
267
+ the restore procedure. Do not use `drop --force`, `create --force`, or vacuum as
268
+ an attempted corruption repair.
269
+
270
+ ### Release evidence
271
+
272
+ Archive command output with the RubyDB commit/version, Ruby version, OS,
273
+ database path class, workload, backup name/checksum, restore time, and operator.
274
+ The CLI is one part of the production evidence; CI, workload, failover, and
275
+ security review records are also required.
@@ -0,0 +1,9 @@
1
+ # Architecture contribution guide
2
+
3
+ Before changing a core subsystem, identify its ownership, durability boundary,
4
+ failure behavior, and public compatibility contract. Changes to storage, WAL,
5
+ recovery, transactions, replication, protocol, or adapters need focused tests
6
+ and an end-to-end regression.
7
+
8
+ Document invariants in the relevant architecture page and add a lesson when a
9
+ new failure mode is discovered. Prefer explicit errors over silent fallback.
@@ -0,0 +1,14 @@
1
+ # Benchmarking guide
2
+
3
+ Run deterministic and concurrent workloads from the repository root:
4
+
5
+ ```sh
6
+ RUBYDB_BENCHMARK_ITERATIONS=100 ruby -Ilib benchmarks/basic_workload.rb
7
+ RUBYDB_SOAK_THREADS=16 RUBYDB_SOAK_OPERATIONS=2000 ruby benchmarks/concurrent_soak.rb
8
+ RUBYDB_PRODUCTION_SOAK_CLIENTS=16 RUBYDB_PRODUCTION_SOAK_OPERATIONS=2000 ruby benchmarks/production_soak.rb
9
+ ```
10
+
11
+ Record commit, Ruby version, OS, CPU/memory/storage, dataset size, seed,
12
+ throughput, p50/p95/p99 latency, WAL growth, and recovery time. Benchmarks are
13
+ not universal capacity certification; compare against application-specific
14
+ limits and repeat after schema or runtime changes.
@@ -0,0 +1,16 @@
1
+ # Development guide
2
+
3
+ Install dependencies with `bundle install`. Run focused specs while developing,
4
+ then run `bundle exec rspec` and `bundle exec rubocop` before opening a pull
5
+ request. Keep test databases in temporary directories and close engines in
6
+ `ensure` blocks.
7
+
8
+ Never use production data or secrets in local tests. Changes that affect a
9
+ stored format, SQL behavior, protocol, migration, or release process must update
10
+ the corresponding documentation and changelog.
11
+
12
+ For the complete change workflow, read the [developer guide](../developer-guide.md),
13
+ [debugging playbook](../debugging.md), and [testing guide](testing.md). Every
14
+ bug fix should include a regression test and an explanation of its invariant.
15
+ Run fault, concurrency, or recovery tests when the change crosses a durable
16
+ boundary; a unit test alone is not sufficient evidence.