rubydb 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (436) hide show
  1. checksums.yaml +7 -0
  2. data/.gitattributes +20 -0
  3. data/.github/ISSUE_TEMPLATE/bug.yml +0 -0
  4. data/.github/ISSUE_TEMPLATE/feature.yml +0 -0
  5. data/.github/ISSUE_TEMPLATE/performance.yml +0 -0
  6. data/.github/ISSUE_TEMPLATE/security.yml +0 -0
  7. data/.github/PULL_REQUEST_TEMPLATE.md +15 -0
  8. data/.github/dependabot.yml +0 -0
  9. data/.github/workflows/benchmark.yml +26 -0
  10. data/.github/workflows/compatibility.yml +63 -0
  11. data/.github/workflows/fuzz.yml +33 -0
  12. data/.github/workflows/lint.yml +21 -0
  13. data/.github/workflows/operations.yml +24 -0
  14. data/.github/workflows/production-validation.yml +111 -0
  15. data/.github/workflows/release.yml +77 -0
  16. data/.github/workflows/security.yml +37 -0
  17. data/.github/workflows/test.yml +26 -0
  18. data/.github/workflows/workload.yml +58 -0
  19. data/.gitignore +43 -0
  20. data/.rspec +4 -0
  21. data/.rubocop.yml +44 -0
  22. data/.ruby-version +1 -0
  23. data/.standard.yml +14 -0
  24. data/ARCHITECTURE.md +21 -0
  25. data/CHANGELOG.md +31 -0
  26. data/CODE_OF_CONDUCT.md +13 -0
  27. data/CONTRIBUTING.md +29 -0
  28. data/GOVERNANCE.md +16 -0
  29. data/Gemfile +17 -0
  30. data/Gemfile.lock +71 -0
  31. data/LICENSE +60 -0
  32. data/README.md +162 -0
  33. data/ROADMAP.md +27 -0
  34. data/Rakefile +71 -0
  35. data/SECURITY.md +54 -0
  36. data/SUPPORT.md +14 -0
  37. data/adapters/activerecord/Gemfile +11 -0
  38. data/adapters/activerecord/README.md +9 -0
  39. data/adapters/activerecord/lib/active_record/connection_adapters/rubydb_adapter.rb +887 -0
  40. data/adapters/activerecord/rubydb-activerecord.gemspec +27 -0
  41. data/adapters/activerecord/spec/rubydb_adapter_integration_spec.rb +151 -0
  42. data/adapters/ruby/README.md +18 -0
  43. data/adapters/sequel/README.md +11 -0
  44. data/bin/rubydb +19 -0
  45. data/bin/rubydb-server +126 -0
  46. data/config/development.yml +0 -0
  47. data/config/monitoring/prometheus-alerts.yml +39 -0
  48. data/config/production.yml +36 -0
  49. data/config/test.yml +0 -0
  50. data/docs/README.md +71 -0
  51. data/docs/architecture/concurrency.md +14 -0
  52. data/docs/architecture/current-state.md +125 -0
  53. data/docs/architecture/execution-engine.md +25 -0
  54. data/docs/architecture/indexes.md +19 -0
  55. data/docs/architecture/mvcc.md +19 -0
  56. data/docs/architecture/overview.md +13 -0
  57. data/docs/architecture/pages.md +11 -0
  58. data/docs/architecture/production-roadmap.md +82 -0
  59. data/docs/architecture/query-planner.md +20 -0
  60. data/docs/architecture/recovery.md +18 -0
  61. data/docs/architecture/sql-engine.md +12 -0
  62. data/docs/architecture/storage-engine.md +14 -0
  63. data/docs/architecture/transactions.md +10 -0
  64. data/docs/architecture/wal.md +28 -0
  65. data/docs/cli-cheatsheet.md +98 -0
  66. data/docs/cli.md +275 -0
  67. data/docs/contributing/architecture.md +9 -0
  68. data/docs/contributing/benchmarking.md +14 -0
  69. data/docs/contributing/development.md +16 -0
  70. data/docs/contributing/release-process.md +49 -0
  71. data/docs/contributing/testing.md +16 -0
  72. data/docs/debugging.md +229 -0
  73. data/docs/developer/branching.md +10 -0
  74. data/docs/developer/database-diff.md +10 -0
  75. data/docs/developer/local-development.md +17 -0
  76. data/docs/developer/snapshots.md +9 -0
  77. data/docs/developer/temporal-data.md +10 -0
  78. data/docs/developer-guide.md +297 -0
  79. data/docs/getting-started/first-database.md +16 -0
  80. data/docs/getting-started/first-query.md +13 -0
  81. data/docs/getting-started/installation.md +19 -0
  82. data/docs/getting-started/local-to-production.md +300 -0
  83. data/docs/getting-started/quickstart.md +17 -0
  84. data/docs/getting-started/rails.md +16 -0
  85. data/docs/hardening_backlog.md +93 -0
  86. data/docs/lessons-learned.md +112 -0
  87. data/docs/operations/backups.md +33 -0
  88. data/docs/operations/disaster-recovery.md +31 -0
  89. data/docs/operations/failover.md +30 -0
  90. data/docs/operations/monitoring.md +25 -0
  91. data/docs/operations/production-guide.md +295 -0
  92. data/docs/operations/production-runbook.md +45 -0
  93. data/docs/operations/replication.md +33 -0
  94. data/docs/operations/restore.md +6 -0
  95. data/docs/operations/runbook.md +34 -0
  96. data/docs/operations/upgrades.md +14 -0
  97. data/docs/operations/workload-testing.md +17 -0
  98. data/docs/production-readiness.md +118 -0
  99. data/docs/production_validation.md +150 -0
  100. data/docs/rails/active-record.md +11 -0
  101. data/docs/rails/compatibility-guide.md +90 -0
  102. data/docs/rails/database-yml.md +92 -0
  103. data/docs/rails/installation.md +17 -0
  104. data/docs/rails/migrations.md +17 -0
  105. data/docs/rails/production.md +82 -0
  106. data/docs/rails/troubleshooting.md +18 -0
  107. data/docs/release.md +59 -0
  108. data/docs/server/architecture.md +10 -0
  109. data/docs/server/authentication.md +10 -0
  110. data/docs/server/configuration.md +16 -0
  111. data/docs/server/connection-pooling.md +10 -0
  112. data/docs/server/deployment.md +10 -0
  113. data/docs/server/protocol.md +12 -0
  114. data/docs/sql/compatibility-guide.md +82 -0
  115. data/docs/sql/compatibility.md +39 -0
  116. data/docs/sql/data-types.md +10 -0
  117. data/docs/sql/functions.md +9 -0
  118. data/docs/sql/joins.md +9 -0
  119. data/docs/sql/operators.md +9 -0
  120. data/docs/sql/sqlite-compatibility.md +21 -0
  121. data/docs/sql/syntax.md +10 -0
  122. data/docs/sql/transactions.md +10 -0
  123. data/docs/troubleshooting.md +244 -0
  124. data/exe/rubydb +23 -0
  125. data/lib/rubydb/backup/archive.rb +334 -0
  126. data/lib/rubydb/backup/backup.rb +401 -0
  127. data/lib/rubydb/backup/incremental.rb +353 -0
  128. data/lib/rubydb/backup/restore.rb +290 -0
  129. data/lib/rubydb/backup/snapshot.rb +267 -0
  130. data/lib/rubydb/backup/verification.rb +279 -0
  131. data/lib/rubydb/branching/branch.rb +181 -0
  132. data/lib/rubydb/branching/branch_manager.rb +311 -0
  133. data/lib/rubydb/branching/branch_metadata.rb +140 -0
  134. data/lib/rubydb/branching/checkout.rb +166 -0
  135. data/lib/rubydb/branching/copy_on_write.rb +272 -0
  136. data/lib/rubydb/branching/diff.rb +138 -0
  137. data/lib/rubydb/branching/merge.rb +285 -0
  138. data/lib/rubydb/build_info.rb +15 -0
  139. data/lib/rubydb/catalog/catalog.rb +391 -0
  140. data/lib/rubydb/catalog/column.rb +112 -0
  141. data/lib/rubydb/catalog/constraint.rb +180 -0
  142. data/lib/rubydb/catalog/database.rb +184 -0
  143. data/lib/rubydb/catalog/index.rb +97 -0
  144. data/lib/rubydb/catalog/schema.rb +103 -0
  145. data/lib/rubydb/catalog/sequence.rb +90 -0
  146. data/lib/rubydb/catalog/system_catalog.rb +698 -0
  147. data/lib/rubydb/catalog/table.rb +178 -0
  148. data/lib/rubydb/catalog/trigger.rb +102 -0
  149. data/lib/rubydb/catalog/view.rb +66 -0
  150. data/lib/rubydb/cli/application.rb +163 -0
  151. data/lib/rubydb/cli/commands/backup.rb +81 -0
  152. data/lib/rubydb/cli/commands/branch.rb +72 -0
  153. data/lib/rubydb/cli/commands/checkout.rb +54 -0
  154. data/lib/rubydb/cli/commands/create.rb +58 -0
  155. data/lib/rubydb/cli/commands/diff.rb +77 -0
  156. data/lib/rubydb/cli/commands/doctor.rb +74 -0
  157. data/lib/rubydb/cli/commands/drop.rb +57 -0
  158. data/lib/rubydb/cli/commands/init.rb +102 -0
  159. data/lib/rubydb/cli/commands/inspect.rb +95 -0
  160. data/lib/rubydb/cli/commands/merge.rb +63 -0
  161. data/lib/rubydb/cli/commands/migrate.rb +62 -0
  162. data/lib/rubydb/cli/commands/restart.rb +39 -0
  163. data/lib/rubydb/cli/commands/restore.rb +121 -0
  164. data/lib/rubydb/cli/commands/shell.rb +365 -0
  165. data/lib/rubydb/cli/commands/snapshot.rb +79 -0
  166. data/lib/rubydb/cli/commands/start.rb +82 -0
  167. data/lib/rubydb/cli/commands/status.rb +92 -0
  168. data/lib/rubydb/cli/commands/stop.rb +47 -0
  169. data/lib/rubydb/cli/commands/vacuum.rb +58 -0
  170. data/lib/rubydb/cli/formatter.rb +221 -0
  171. data/lib/rubydb/cli/output.rb +168 -0
  172. data/lib/rubydb/client/client.rb +304 -0
  173. data/lib/rubydb/client/connection.rb +415 -0
  174. data/lib/rubydb/client/connection_pool.rb +168 -0
  175. data/lib/rubydb/client/connection_url.rb +96 -0
  176. data/lib/rubydb/client/prepared_statement.rb +60 -0
  177. data/lib/rubydb/client/result.rb +123 -0
  178. data/lib/rubydb/client/statement.rb +52 -0
  179. data/lib/rubydb/client/transaction.rb +130 -0
  180. data/lib/rubydb/concurrency/concurrency.rb +19 -0
  181. data/lib/rubydb/concurrency/deadlock_detector.rb +150 -0
  182. data/lib/rubydb/concurrency/latch.rb +101 -0
  183. data/lib/rubydb/concurrency/lock_graph.rb +165 -0
  184. data/lib/rubydb/concurrency/mutex.rb +183 -0
  185. data/lib/rubydb/concurrency/rw_lock.rb +180 -0
  186. data/lib/rubydb/concurrency/scheduler.rb +250 -0
  187. data/lib/rubydb/concurrency/worker_pool.rb +143 -0
  188. data/lib/rubydb/configuration/config.rb +170 -0
  189. data/lib/rubydb/configuration/defaults.rb +179 -0
  190. data/lib/rubydb/configuration/environment.rb +152 -0
  191. data/lib/rubydb/configuration/parser.rb +185 -0
  192. data/lib/rubydb/configuration/validation.rb +221 -0
  193. data/lib/rubydb/constants.rb +74 -0
  194. data/lib/rubydb/constraints/check.rb +181 -0
  195. data/lib/rubydb/constraints/constraint.rb +101 -0
  196. data/lib/rubydb/constraints/foreign_key.rb +130 -0
  197. data/lib/rubydb/constraints/not_null.rb +64 -0
  198. data/lib/rubydb/constraints/primary_key.rb +99 -0
  199. data/lib/rubydb/constraints/unique.rb +108 -0
  200. data/lib/rubydb/constraints/validator.rb +350 -0
  201. data/lib/rubydb/errors/authentication_error.rb +10 -0
  202. data/lib/rubydb/errors/authorization_error.rb +23 -0
  203. data/lib/rubydb/errors/client_error.rb +10 -0
  204. data/lib/rubydb/errors/configuration_error.rb +10 -0
  205. data/lib/rubydb/errors/connection_error.rb +10 -0
  206. data/lib/rubydb/errors/constraint_error.rb +23 -0
  207. data/lib/rubydb/errors/corruption_error.rb +10 -0
  208. data/lib/rubydb/errors/database_error.rb +10 -0
  209. data/lib/rubydb/errors/error.rb +20 -0
  210. data/lib/rubydb/errors/execution_error.rb +10 -0
  211. data/lib/rubydb/errors/parser_error.rb +10 -0
  212. data/lib/rubydb/errors/recovery_error.rb +10 -0
  213. data/lib/rubydb/errors/replication_error.rb +10 -0
  214. data/lib/rubydb/errors/server_error.rb +6 -0
  215. data/lib/rubydb/errors/storage_error.rb +10 -0
  216. data/lib/rubydb/errors/transaction_error.rb +10 -0
  217. data/lib/rubydb/execution/aggregate_executor.rb +138 -0
  218. data/lib/rubydb/execution/delete_executor.rb +112 -0
  219. data/lib/rubydb/execution/distinct_executor.rb +135 -0
  220. data/lib/rubydb/execution/executor.rb +1192 -0
  221. data/lib/rubydb/execution/expression.rb +193 -0
  222. data/lib/rubydb/execution/index_scan.rb +142 -0
  223. data/lib/rubydb/execution/insert_executor.rb +217 -0
  224. data/lib/rubydb/execution/join_executor.rb +249 -0
  225. data/lib/rubydb/execution/limit_executor.rb +85 -0
  226. data/lib/rubydb/execution/optimizer.rb +215 -0
  227. data/lib/rubydb/execution/plan.rb +361 -0
  228. data/lib/rubydb/execution/planner.rb +546 -0
  229. data/lib/rubydb/execution/predicate.rb +235 -0
  230. data/lib/rubydb/execution/scan.rb +49 -0
  231. data/lib/rubydb/execution/sequential_scan.rb +63 -0
  232. data/lib/rubydb/execution/sort_executor.rb +185 -0
  233. data/lib/rubydb/execution/update_executor.rb +162 -0
  234. data/lib/rubydb/functions/aggregate.rb +70 -0
  235. data/lib/rubydb/functions/date_functions.rb +278 -0
  236. data/lib/rubydb/functions/function.rb +85 -0
  237. data/lib/rubydb/functions/json_functions.rb +215 -0
  238. data/lib/rubydb/functions/numeric_functions.rb +346 -0
  239. data/lib/rubydb/functions/scalar.rb +52 -0
  240. data/lib/rubydb/functions/string_functions.rb +383 -0
  241. data/lib/rubydb/functions/system_functions.rb +246 -0
  242. data/lib/rubydb/history/as_of.rb +238 -0
  243. data/lib/rubydb/history/change.rb +105 -0
  244. data/lib/rubydb/history/history.rb +131 -0
  245. data/lib/rubydb/history/history_manager.rb +229 -0
  246. data/lib/rubydb/history/temporal_query.rb +202 -0
  247. data/lib/rubydb/history/timeline.rb +144 -0
  248. data/lib/rubydb/indexes/btree.rb +186 -0
  249. data/lib/rubydb/indexes/btree_cursor.rb +258 -0
  250. data/lib/rubydb/indexes/btree_node.rb +385 -0
  251. data/lib/rubydb/indexes/hash_index.rb +150 -0
  252. data/lib/rubydb/indexes/index.rb +71 -0
  253. data/lib/rubydb/indexes/index_manager.rb +406 -0
  254. data/lib/rubydb/indexes/index_scan.rb +470 -0
  255. data/lib/rubydb/migrations/migration.rb +254 -0
  256. data/lib/rubydb/migrations/migration_lock.rb +146 -0
  257. data/lib/rubydb/migrations/migration_manager.rb +176 -0
  258. data/lib/rubydb/migrations/migration_version.rb +71 -0
  259. data/lib/rubydb/migrations/schema_diff.rb +211 -0
  260. data/lib/rubydb/migrations/schema_version.rb +64 -0
  261. data/lib/rubydb/monitoring/events.rb +160 -0
  262. data/lib/rubydb/monitoring/health.rb +222 -0
  263. data/lib/rubydb/monitoring/logger.rb +193 -0
  264. data/lib/rubydb/monitoring/metrics.rb +359 -0
  265. data/lib/rubydb/monitoring/performance.rb +176 -0
  266. data/lib/rubydb/monitoring/statistics.rb +170 -0
  267. data/lib/rubydb/mvcc/garbage_collector.rb +199 -0
  268. data/lib/rubydb/mvcc/mvcc.rb +16 -0
  269. data/lib/rubydb/mvcc/snapshot.rb +147 -0
  270. data/lib/rubydb/mvcc/vacuum.rb +180 -0
  271. data/lib/rubydb/mvcc/version.rb +106 -0
  272. data/lib/rubydb/mvcc/version_store.rb +398 -0
  273. data/lib/rubydb/mvcc/visibility.rb +109 -0
  274. data/lib/rubydb/protocol/capabilities.rb +125 -0
  275. data/lib/rubydb/protocol/decoder.rb +145 -0
  276. data/lib/rubydb/protocol/encoder.rb +136 -0
  277. data/lib/rubydb/protocol/handshake.rb +305 -0
  278. data/lib/rubydb/protocol/message.rb +121 -0
  279. data/lib/rubydb/protocol/protocol.rb +277 -0
  280. data/lib/rubydb/protocol/version.rb +54 -0
  281. data/lib/rubydb/rails/adapter.rb +239 -0
  282. data/lib/rubydb/rails/connection.rb +314 -0
  283. data/lib/rubydb/rails/database_statements.rb +122 -0
  284. data/lib/rubydb/rails/migration.rb +131 -0
  285. data/lib/rubydb/rails/quoting.rb +109 -0
  286. data/lib/rubydb/rails/result.rb +120 -0
  287. data/lib/rubydb/rails/schema_statements.rb +339 -0
  288. data/lib/rubydb/rails/transaction.rb +105 -0
  289. data/lib/rubydb/rails/type.rb +126 -0
  290. data/lib/rubydb/recovery/checkpoint.rb +257 -0
  291. data/lib/rubydb/recovery/consistency.rb +467 -0
  292. data/lib/rubydb/recovery/corruption_detector.rb +5 -0
  293. data/lib/rubydb/recovery/crash_recovery.rb +387 -0
  294. data/lib/rubydb/recovery/recovery_manager.rb +206 -0
  295. data/lib/rubydb/recovery/redo.rb +237 -0
  296. data/lib/rubydb/recovery/undo.rb +206 -0
  297. data/lib/rubydb/replication/failover.rb +5 -0
  298. data/lib/rubydb/replication/fencing.rb +63 -0
  299. data/lib/rubydb/replication/primary.rb +450 -0
  300. data/lib/rubydb/replication/replica.rb +384 -0
  301. data/lib/rubydb/replication/replication_log.rb +200 -0
  302. data/lib/rubydb/replication/replication_manager.rb +308 -0
  303. data/lib/rubydb/replication/replication_slot.rb +295 -0
  304. data/lib/rubydb/replication/replication_stream.rb +201 -0
  305. data/lib/rubydb/rubydb.rb +560 -0
  306. data/lib/rubydb/security/access_control.rb +254 -0
  307. data/lib/rubydb/security/audit_log.rb +213 -0
  308. data/lib/rubydb/security/authentication.rb +302 -0
  309. data/lib/rubydb/security/authorization.rb +282 -0
  310. data/lib/rubydb/security/credentials.rb +196 -0
  311. data/lib/rubydb/security/password.rb +215 -0
  312. data/lib/rubydb/security/permissions.rb +74 -0
  313. data/lib/rubydb/security/role.rb +101 -0
  314. data/lib/rubydb/security/user.rb +86 -0
  315. data/lib/rubydb/server/connection.rb +366 -0
  316. data/lib/rubydb/server/connection_pool.rb +193 -0
  317. data/lib/rubydb/server/lifecycle.rb +228 -0
  318. data/lib/rubydb/server/listener.rb +136 -0
  319. data/lib/rubydb/server/request_handler.rb +276 -0
  320. data/lib/rubydb/server/server.rb +364 -0
  321. data/lib/rubydb/server/session.rb +369 -0
  322. data/lib/rubydb/server/worker.rb +210 -0
  323. data/lib/rubydb/server/worker_pool.rb +168 -0
  324. data/lib/rubydb/sql/ast/alter_table.rb +169 -0
  325. data/lib/rubydb/sql/ast/begin_transaction.rb +47 -0
  326. data/lib/rubydb/sql/ast/commit.rb +37 -0
  327. data/lib/rubydb/sql/ast/constraint.rb +83 -0
  328. data/lib/rubydb/sql/ast/create_database.rb +41 -0
  329. data/lib/rubydb/sql/ast/create_index.rb +61 -0
  330. data/lib/rubydb/sql/ast/create_schema.rb +52 -0
  331. data/lib/rubydb/sql/ast/create_table.rb +187 -0
  332. data/lib/rubydb/sql/ast/delete.rb +54 -0
  333. data/lib/rubydb/sql/ast/drop_database.rb +41 -0
  334. data/lib/rubydb/sql/ast/drop_index.rb +41 -0
  335. data/lib/rubydb/sql/ast/drop_schema.rb +49 -0
  336. data/lib/rubydb/sql/ast/drop_table.rb +49 -0
  337. data/lib/rubydb/sql/ast/explain.rb +64 -0
  338. data/lib/rubydb/sql/ast/expression.rb +604 -0
  339. data/lib/rubydb/sql/ast/insert.rb +82 -0
  340. data/lib/rubydb/sql/ast/node.rb +42 -0
  341. data/lib/rubydb/sql/ast/rollback.rb +63 -0
  342. data/lib/rubydb/sql/ast/savepoint.rb +59 -0
  343. data/lib/rubydb/sql/ast/select.rb +88 -0
  344. data/lib/rubydb/sql/ast/set_operation.rb +20 -0
  345. data/lib/rubydb/sql/ast/trigger.rb +29 -0
  346. data/lib/rubydb/sql/ast/update.rb +88 -0
  347. data/lib/rubydb/sql/ast/vacuum.rb +19 -0
  348. data/lib/rubydb/sql/ast/view.rb +32 -0
  349. data/lib/rubydb/sql/ast/with.rb +32 -0
  350. data/lib/rubydb/sql/grammar.rb +86 -0
  351. data/lib/rubydb/sql/keywords.rb +156 -0
  352. data/lib/rubydb/sql/lexer.rb +214 -0
  353. data/lib/rubydb/sql/operators.rb +100 -0
  354. data/lib/rubydb/sql/parser.rb +1185 -0
  355. data/lib/rubydb/sql/planner/analyzer.rb +302 -0
  356. data/lib/rubydb/sql/planner/binder.rb +550 -0
  357. data/lib/rubydb/sql/planner/type_checker.rb +431 -0
  358. data/lib/rubydb/sql/token.rb +210 -0
  359. data/lib/rubydb/storage/buffer_frame.rb +44 -0
  360. data/lib/rubydb/storage/buffer_pool.rb +155 -0
  361. data/lib/rubydb/storage/database_lock.rb +74 -0
  362. data/lib/rubydb/storage/deserializer.rb +342 -0
  363. data/lib/rubydb/storage/engine.rb +2338 -0
  364. data/lib/rubydb/storage/file_manager.rb +187 -0
  365. data/lib/rubydb/storage/free_space_map.rb +81 -0
  366. data/lib/rubydb/storage/page.rb +94 -0
  367. data/lib/rubydb/storage/page_allocator.rb +855 -0
  368. data/lib/rubydb/storage/page_header.rb +67 -0
  369. data/lib/rubydb/storage/page_manager.rb +131 -0
  370. data/lib/rubydb/storage/record.rb +58 -0
  371. data/lib/rubydb/storage/row.rb +78 -0
  372. data/lib/rubydb/storage/serializer.rb +51 -0
  373. data/lib/rubydb/storage/storage_layout.rb +151 -0
  374. data/lib/rubydb/storage/storage_manager.rb +114 -0
  375. data/lib/rubydb/storage/tuple.rb +461 -0
  376. data/lib/rubydb/storage/visibility_map.rb +984 -0
  377. data/lib/rubydb/transactions/commit_manager.rb +220 -0
  378. data/lib/rubydb/transactions/isolation.rb +98 -0
  379. data/lib/rubydb/transactions/lock.rb +76 -0
  380. data/lib/rubydb/transactions/lock_manager.rb +362 -0
  381. data/lib/rubydb/transactions/savepoint.rb +143 -0
  382. data/lib/rubydb/transactions/transaction.rb +215 -0
  383. data/lib/rubydb/transactions/transaction_id.rb +84 -0
  384. data/lib/rubydb/transactions/transaction_log.rb +257 -0
  385. data/lib/rubydb/transactions/transaction_manager.rb +435 -0
  386. data/lib/rubydb/types/bigint.rb +36 -0
  387. data/lib/rubydb/types/blob.rb +37 -0
  388. data/lib/rubydb/types/boolean.rb +34 -0
  389. data/lib/rubydb/types/date.rb +39 -0
  390. data/lib/rubydb/types/decimal.rb +48 -0
  391. data/lib/rubydb/types/float.rb +34 -0
  392. data/lib/rubydb/types/integer.rb +36 -0
  393. data/lib/rubydb/types/json.rb +41 -0
  394. data/lib/rubydb/types/null.rb +34 -0
  395. data/lib/rubydb/types/smallint.rb +36 -0
  396. data/lib/rubydb/types/text.rb +37 -0
  397. data/lib/rubydb/types/time.rb +46 -0
  398. data/lib/rubydb/types/timestamp.rb +39 -0
  399. data/lib/rubydb/types/type.rb +119 -0
  400. data/lib/rubydb/types/uuid.rb +47 -0
  401. data/lib/rubydb/types/varchar.rb +37 -0
  402. data/lib/rubydb/version.rb +33 -0
  403. data/lib/rubydb/wal/archive.rb +193 -0
  404. data/lib/rubydb/wal/checkpoint.rb +183 -0
  405. data/lib/rubydb/wal/lsn.rb +94 -0
  406. data/lib/rubydb/wal/reader.rb +260 -0
  407. data/lib/rubydb/wal/record.rb +105 -0
  408. data/lib/rubydb/wal/segment.rb +193 -0
  409. data/lib/rubydb/wal/wal.rb +452 -0
  410. data/lib/rubydb/wal/writer.rb +236 -0
  411. data/lib/rubydb.rb +7 -0
  412. data/packaging/docker/Dockerfile +36 -0
  413. data/packaging/docker/docker-compose.failover.yml +43 -0
  414. data/packaging/docker/entrypoint.sh +15 -0
  415. data/packaging/homebrew/rubydb.rb +19 -0
  416. data/packaging/systemd/rubydb.service +22 -0
  417. data/rubydb.gemspec +57 -0
  418. data/scripts/benchmark +7 -0
  419. data/scripts/durability_drill +37 -0
  420. data/scripts/format +0 -0
  421. data/scripts/fuzz +63 -0
  422. data/scripts/lint +0 -0
  423. data/scripts/release +50 -0
  424. data/scripts/release_check +43 -0
  425. data/scripts/replication_failover_drill +250 -0
  426. data/scripts/replication_network_failover_drill +255 -0
  427. data/scripts/restore_drill +45 -0
  428. data/scripts/security +0 -0
  429. data/scripts/setup +0 -0
  430. data/scripts/test +0 -0
  431. data/scripts/test-chaos +0 -0
  432. data/scripts/test-crash +0 -0
  433. data/scripts/test-integration +0 -0
  434. data/scripts/test-unit +0 -0
  435. data/visibility_map.json +1 -0
  436. metadata +538 -0
@@ -0,0 +1,27 @@
1
+ # frozen_string_literal: true
2
+
3
+ Gem::Specification.new do |spec|
4
+ spec.name = "rubydb-activerecord"
5
+ spec.version = "0.1.0"
6
+ spec.authors = ["Aldane Hutchinson"]
7
+ spec.email = ["aldanehutchinson5@gmail.com"]
8
+
9
+ spec.summary = "ActiveRecord adapter for RubyDB"
10
+ spec.description = "ActiveRecord adapter for the RubyDB database"
11
+ spec.homepage = "https://github.com/aldanedev-create/rubydb"
12
+ spec.license = "MIT"
13
+ spec.required_ruby_version = ">= 3.2.0"
14
+
15
+ spec.metadata["homepage_uri"] = spec.homepage
16
+ spec.metadata["source_code_uri"] = spec.homepage
17
+
18
+ spec.files = Dir.glob("lib/**/*.rb") + %w[README.md]
19
+ spec.require_paths = ["lib"]
20
+
21
+ spec.add_dependency "activerecord", ">= 7.1", "< 8.1"
22
+ spec.add_dependency "rubydb", "~> 0.1.0"
23
+
24
+ spec.add_development_dependency "rake", "~> 13.2"
25
+ spec.add_development_dependency "rspec", "~> 3.13"
26
+ spec.add_development_dependency "rubocop", "~> 1.60"
27
+ end
@@ -0,0 +1,151 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "tmpdir"
4
+
5
+ $LOAD_PATH.unshift(File.expand_path("../../../lib", __dir__))
6
+ $LOAD_PATH.unshift(File.expand_path("../lib", __dir__))
7
+
8
+ require "rubydb"
9
+ require "active_record"
10
+ require "active_record/connection_adapters/rubydb_adapter"
11
+
12
+ RSpec.describe ActiveRecord::ConnectionAdapters::RubyDBAdapter do
13
+ def migration_version
14
+ "#{ActiveRecord::VERSION::MAJOR}.#{ActiveRecord::VERSION::MINOR}"
15
+ end
16
+
17
+ let(:model) do
18
+ Class.new(ActiveRecord::Base) do
19
+ self.table_name = "accounts"
20
+ end
21
+ end
22
+
23
+ around do |example|
24
+ Dir.mktmpdir("rubydb-active-record") do |directory|
25
+ engine = RubyDB::Storage::Engine.new(File.join(directory, "accounts.rdb"), auto_cleanup: false, auto_vacuum: false)
26
+ ActiveRecord::Base.establish_connection(adapter: "rubydb", engine: engine)
27
+ example.run
28
+ ensure
29
+ ActiveRecord::Base.connection_pool.disconnect! if ActiveRecord::Base.connected?
30
+ engine&.close
31
+ end
32
+ end
33
+
34
+ it "creates and finds an ActiveRecord model through the embedded engine" do
35
+ connection = ActiveRecord::Base.connection
36
+ connection.execute("CREATE TABLE accounts (id INTEGER PRIMARY KEY, email VARCHAR(255) NOT NULL, active BOOLEAN NOT NULL)")
37
+
38
+ created = model.create!(id: 1, email: "ada@example.test", active: true)
39
+ loaded = model.find(created.id)
40
+
41
+ expect(loaded.attributes).to include("id" => 1, "email" => "ada@example.test", "active" => true)
42
+ end
43
+
44
+ it "runs a Rails migration that creates a table, adds a column, and adds an index" do
45
+ migration = Class.new(ActiveRecord::Migration[migration_version]) do
46
+ def change
47
+ create_table :projects do |table|
48
+ table.string :name, null: false
49
+ end
50
+ add_column :projects, :active, :boolean, default: true, null: false
51
+ add_index :projects, :name, unique: true
52
+ end
53
+ end
54
+
55
+ migration.new.migrate(:up)
56
+ connection = ActiveRecord::Base.connection
57
+
58
+ expect(connection.table_exists?(:projects)).to be(true)
59
+ expect(connection.columns(:projects).map(&:name)).to include("id", "name", "active")
60
+ expect(connection.indexes(:projects)).to include(an_object_having_attributes(name: "idx_projects_name", unique: true))
61
+ schema = connection.dump_schema
62
+ expect(schema).to include('t.boolean "active", default: true, null: false')
63
+ expect(schema).to include('add_index "projects", ["name"], unique: true')
64
+ expect(schema).not_to include('t.integer "id"')
65
+
66
+ migration.new.migrate(:down)
67
+ expect(connection.table_exists?(:projects)).to be(false)
68
+ end
69
+
70
+ it "executes an ActiveRecord association join with qualified filtering" do
71
+ stub_const("RubydbAccount", Class.new(ActiveRecord::Base) do
72
+ self.table_name = "accounts"
73
+ has_many :rubydb_projects, class_name: "RubydbProject", foreign_key: :account_id
74
+ end)
75
+ stub_const("RubydbProject", Class.new(ActiveRecord::Base) do
76
+ self.table_name = "projects"
77
+ belongs_to :rubydb_account, class_name: "RubydbAccount", foreign_key: :account_id
78
+ end)
79
+
80
+ connection = ActiveRecord::Base.connection
81
+ connection.execute("CREATE TABLE accounts (id INTEGER PRIMARY KEY, email VARCHAR(255) NOT NULL)")
82
+ connection.execute("CREATE TABLE projects (id INTEGER PRIMARY KEY, account_id INTEGER NOT NULL, name VARCHAR(255) NOT NULL)")
83
+ account = RubydbAccount.create!(id: 1, email: "ada@example.test")
84
+ RubydbProject.create!(id: 10, account_id: account.id, name: "RubyDB")
85
+
86
+ projects = RubydbProject.joins(:rubydb_account).where(accounts: { email: "ada@example.test" }).to_a
87
+
88
+ expect(projects.map(&:attributes)).to include(hash_including("id" => 10, "name" => "RubyDB", "account_id" => 1))
89
+ end
90
+
91
+ it "loads nested associations and ordered join results from populated tables" do
92
+ stub_const("RubydbAuthor", Class.new(ActiveRecord::Base) do
93
+ self.table_name = "authors"
94
+ has_many :rubydb_books, class_name: "RubydbBook", foreign_key: :author_id
95
+ end)
96
+ stub_const("RubydbBook", Class.new(ActiveRecord::Base) do
97
+ self.table_name = "books"
98
+ belongs_to :rubydb_author, class_name: "RubydbAuthor", foreign_key: :author_id
99
+ has_many :rubydb_reviews, class_name: "RubydbReview", foreign_key: :book_id
100
+ end)
101
+ stub_const("RubydbReview", Class.new(ActiveRecord::Base) do
102
+ self.table_name = "reviews"
103
+ belongs_to :rubydb_book, class_name: "RubydbBook", foreign_key: :book_id
104
+ end)
105
+
106
+ connection = ActiveRecord::Base.connection
107
+ connection.execute("CREATE TABLE authors (id INTEGER PRIMARY KEY, email VARCHAR(255) NOT NULL)")
108
+ connection.execute("CREATE TABLE books (id INTEGER PRIMARY KEY, author_id INTEGER NOT NULL, title VARCHAR(255) NOT NULL)")
109
+ connection.execute("CREATE TABLE reviews (id INTEGER PRIMARY KEY, book_id INTEGER NOT NULL, rating INTEGER NOT NULL)")
110
+ author = RubydbAuthor.create!(id: 1, email: "ada@example.test")
111
+ other = RubydbAuthor.create!(id: 2, email: "other@example.test")
112
+ first = RubydbBook.create!(id: 10, author_id: author.id, title: "A")
113
+ second = RubydbBook.create!(id: 11, author_id: author.id, title: "B")
114
+ RubydbBook.create!(id: 12, author_id: other.id, title: "C")
115
+ RubydbReview.create!(id: 20, book_id: first.id, rating: 5)
116
+ RubydbReview.create!(id: 21, book_id: second.id, rating: 4)
117
+
118
+ loaded = RubydbAuthor.includes(rubydb_books: :rubydb_reviews)
119
+ .where(email: "ada@example.test").to_a
120
+ joined_titles = RubydbBook.joins(:rubydb_author)
121
+ .where(rubydb_authors: { email: "ada@example.test" })
122
+ .order(title: :desc).pluck(:title)
123
+
124
+ expect(loaded.first.rubydb_books.map(&:title)).to contain_exactly("A", "B")
125
+ expect(loaded.first.rubydb_books.flat_map { |book| book.rubydb_reviews.map(&:rating) })
126
+ .to contain_exactly(5, 4)
127
+ expect(joined_titles).to eq(["B", "A"])
128
+ end
129
+
130
+ it "round-trips a migration on a populated table" do
131
+ connection = ActiveRecord::Base.connection
132
+ connection.execute("CREATE TABLE accounts (id INTEGER PRIMARY KEY, email VARCHAR(255) NOT NULL)")
133
+ connection.execute("INSERT INTO accounts (id, email) VALUES (1, 'ada@example.test')")
134
+
135
+ migration = Class.new(ActiveRecord::Migration[migration_version]) do
136
+ def change
137
+ add_column :accounts, :status, :string, default: "new", null: false
138
+ add_index :accounts, :status
139
+ end
140
+ end
141
+
142
+ migration.new.migrate(:up)
143
+ loaded = connection.select_one("SELECT status FROM accounts WHERE id = 1")
144
+
145
+ expect(loaded["status"] || loaded[:status]).to eq("new")
146
+ expect(connection.indexes(:accounts)).to include(an_object_having_attributes(columns: ["status"]))
147
+
148
+ migration.new.migrate(:down)
149
+ expect(connection.columns(:accounts).map(&:name)).not_to include("status")
150
+ end
151
+ end
@@ -0,0 +1,18 @@
1
+ # RubyDB Ruby adapter
2
+
3
+ The Ruby adapter is the direct Ruby API for embedded and client/server use.
4
+ For one owning process, open an embedded engine:
5
+
6
+ ```ruby
7
+ require "rubydb"
8
+
9
+ engine = RubyDB::Storage::Engine.new("tmp/app.rdb")
10
+ engine.execute("CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT)")
11
+ engine.execute("INSERT INTO users (id, name) VALUES (1, 'Aldane')")
12
+ engine.close
13
+ ```
14
+
15
+ An embedded path has exclusive ownership. Multiple application processes must
16
+ connect to a managed RubyDB server through `RubyDB::Client::Client`. Validate
17
+ the exact RubyDB SQL subset, backup, restore, workload, and recovery procedure
18
+ before using important data. See the root README and `docs/production-readiness.md`.
@@ -0,0 +1,11 @@
1
+ # RubyDB Sequel adapter
2
+
3
+ The Sequel adapter is an integration surface for applications that use Sequel.
4
+ It should be configured with a RubyDB connection and kept within the SQL and
5
+ schema features documented by RubyDB. Use one embedded owner or point all
6
+ processes at a RubyDB server; never open the same embedded path independently
7
+ from multiple workers.
8
+
9
+ Before production use, run the application's Sequel schema, query, transaction,
10
+ pool, migration, backup, restore, and failure tests. Compatibility is feature-
11
+ specific and is not a claim of complete PostgreSQL, MySQL, or SQLite behavior.
data/bin/rubydb ADDED
@@ -0,0 +1,19 @@
1
+ #!/usr/bin/env ruby
2
+ # frozen_string_literal: true
3
+
4
+ # RubyDB CLI - Main entry point for database management commands
5
+
6
+ require_relative "../lib/rubydb"
7
+ require_relative "../lib/rubydb/cli/application"
8
+
9
+ # Load the CLI application and run it
10
+ begin
11
+ RubyDB::CLI::Application.start(ARGV)
12
+ rescue Interrupt
13
+ puts "\nInterrupted. Exiting..."
14
+ exit 0
15
+ rescue StandardError => e
16
+ $stderr.puts "Error: #{e.message}"
17
+ $stderr.puts e.backtrace if ENV["RUBYDB_DEBUG"]
18
+ exit 1
19
+ end
data/bin/rubydb-server ADDED
@@ -0,0 +1,126 @@
1
+ #!/usr/bin/env ruby
2
+ # frozen_string_literal: true
3
+
4
+ # RubyDB Server - Production database server
5
+
6
+ require "optparse"
7
+ require_relative "../lib/rubydb"
8
+ require_relative "../lib/rubydb/server"
9
+
10
+ # Parse command line options
11
+ options = {
12
+ host: "localhost",
13
+ port: 7432,
14
+ database: nil,
15
+ config: nil,
16
+ daemonize: false,
17
+ pid_file: nil,
18
+ log_file: nil,
19
+ debug: false
20
+ }
21
+
22
+ OptionParser.new do |opts|
23
+ opts.banner = "Usage: rubydb-server [options]"
24
+
25
+ opts.on("-h", "--host HOST", "Bind to host (default: localhost)") do |h|
26
+ options[:host] = h
27
+ end
28
+
29
+ opts.on("-p", "--port PORT", Integer, "Bind to port (default: 7432)") do |p|
30
+ options[:port] = p
31
+ end
32
+
33
+ opts.on("-d", "--database PATH", "Path to database file") do |db|
34
+ options[:database] = db
35
+ end
36
+
37
+ opts.on("-c", "--config FILE", "Configuration file path") do |config|
38
+ options[:config] = config
39
+ end
40
+
41
+ opts.on("-D", "--daemonize", "Run as daemon") do
42
+ options[:daemonize] = true
43
+ end
44
+
45
+ opts.on("--pid-file FILE", "PID file location") do |pid|
46
+ options[:pid_file] = pid
47
+ end
48
+
49
+ opts.on("--log-file FILE", "Log file location") do |log|
50
+ options[:log_file] = log
51
+ end
52
+
53
+ opts.on("--debug", "Enable debug mode") do
54
+ options[:debug] = true
55
+ end
56
+
57
+ opts.on("-v", "--version", "Print version") do
58
+ require_relative "../lib/rubydb/version"
59
+ puts "RubyDB Server v#{RubyDB::VERSION}"
60
+ exit 0
61
+ end
62
+
63
+ opts.on("-h", "--help", "Print this help") do
64
+ puts opts
65
+ exit 0
66
+ end
67
+ end.parse!
68
+
69
+ # Validate required options
70
+ if options[:database].nil? && options[:config].nil?
71
+ $stderr.puts "Error: --database or --config is required"
72
+ exit 1
73
+ end
74
+
75
+ # Start the server
76
+ begin
77
+ if options[:daemonize]
78
+ Process.daemon(true, true) if Process.respond_to?(:daemon)
79
+ end
80
+
81
+ # Write PID file if specified
82
+ if options[:pid_file]
83
+ File.write(options[:pid_file], Process.pid.to_s)
84
+ end
85
+
86
+ # Load configuration
87
+ config = if options[:config]
88
+ RubyDB::Configuration.load(options[:config])
89
+ else
90
+ RubyDB::Configuration.new do |c|
91
+ c.host = options[:host]
92
+ c.port = options[:port]
93
+ c.database = options[:database]
94
+ c.debug = options[:debug]
95
+ end
96
+ end
97
+
98
+ # Initialize and start server
99
+ server = RubyDB::Server.new(config)
100
+
101
+ trap("INT") do
102
+ puts "\nShutting down RubyDB server..."
103
+ server.stop
104
+ exit 0
105
+ end
106
+
107
+ trap("TERM") do
108
+ puts "Terminating RubyDB server..."
109
+ server.stop
110
+ exit 0
111
+ end
112
+
113
+ puts "RubyDB Server v#{RubyDB::VERSION} starting..."
114
+ puts "Listening on #{config.host}:#{config.port}"
115
+ puts "Database: #{config.database}"
116
+ puts "Press Ctrl+C to stop"
117
+
118
+ server.start
119
+
120
+ rescue StandardError => e
121
+ $stderr.puts "Fatal error: #{e.message}"
122
+ $stderr.puts e.backtrace if options[:debug]
123
+ exit 1
124
+ ensure
125
+ File.delete(options[:pid_file]) if options[:pid_file] && File.exist?(options[:pid_file])
126
+ end
File without changes
@@ -0,0 +1,39 @@
1
+ groups:
2
+ - name: rubydb-production
3
+ rules:
4
+ - alert: RubyDBNotReady
5
+ expr: rubydb_health_ready == 0
6
+ for: 2m
7
+ labels:
8
+ severity: critical
9
+ annotations:
10
+ summary: RubyDB is not ready to serve traffic
11
+ description: Readiness has been false for more than two minutes.
12
+ - alert: RubyDBConnectionsNearLimit
13
+ expr: rubydb_connections_active / rubydb_connections_max > 0.85
14
+ for: 10m
15
+ labels:
16
+ severity: warning
17
+ annotations:
18
+ summary: RubyDB connection capacity is nearly exhausted
19
+ - alert: RubyDBRequestFailures
20
+ expr: increase(rubydb_requests_failed_total[5m]) > 0
21
+ for: 5m
22
+ labels:
23
+ severity: warning
24
+ annotations:
25
+ summary: RubyDB requests are failing
26
+ - alert: RubyDBReplicationLag
27
+ expr: rubydb_replication_lag_seconds > 10
28
+ for: 2m
29
+ labels:
30
+ severity: critical
31
+ annotations:
32
+ summary: RubyDB replica lag exceeds the promotion threshold
33
+ - alert: RubyDBDiskSpaceLow
34
+ expr: rubydb_disk_free_ratio < 0.15
35
+ for: 5m
36
+ labels:
37
+ severity: critical
38
+ annotations:
39
+ summary: RubyDB storage is running low
@@ -0,0 +1,36 @@
1
+ server:
2
+ host: <%= ENV.fetch("RUBYDB_HOST", "127.0.0.1").to_json %>
3
+ port: <%= ENV.fetch("RUBYDB_PORT", "7432") %>
4
+ max_connections: <%= ENV.fetch("RUBYDB_MAX_CONNECTIONS", "100") %>
5
+ min_workers: <%= ENV.fetch("RUBYDB_MIN_WORKERS", "2") %>
6
+ max_workers: <%= ENV.fetch("RUBYDB_MAX_WORKERS", "20") %>
7
+ worker_queue_size: <%= ENV.fetch("RUBYDB_WORKER_QUEUE_SIZE", "1000") %>
8
+ read_timeout: <%= ENV.fetch("RUBYDB_READ_TIMEOUT", "30") %>
9
+ write_timeout: <%= ENV.fetch("RUBYDB_WRITE_TIMEOUT", "30") %>
10
+ idle_timeout: <%= ENV.fetch("RUBYDB_IDLE_TIMEOUT", "300") %>
11
+ max_request_size: <%= ENV.fetch("RUBYDB_MAX_REQUEST_SIZE", "10485760") %>
12
+ daemonize: false
13
+ pid_file: <%= ENV.fetch("RUBYDB_PID_FILE", "rubydb.pid").to_json %>
14
+
15
+ storage:
16
+ data_dir: <%= ENV.fetch("RUBYDB_DATA_DIR", "/var/lib/rubydb/data").to_json %>
17
+ fsync: true
18
+ wal_enabled: true
19
+
20
+ logging:
21
+ log_dir: <%= ENV.fetch("RUBYDB_LOG_DIR", "/var/log/rubydb").to_json %>
22
+ log_level: <%= ENV.fetch("RUBYDB_LOG_LEVEL", "info").to_json %>
23
+
24
+ auth:
25
+ method: password
26
+ credentials:
27
+ username: <%= ENV.fetch("RUBYDB_USERNAME").to_json %>
28
+ password: <%= ENV.fetch("RUBYDB_PASSWORD").to_json %>
29
+
30
+ ssl:
31
+ enabled: <%= ENV.fetch("RUBYDB_SSL_ENABLED", "true") %>
32
+ cert_file: <%= ENV.fetch("RUBYDB_SSL_CERT_FILE", "/etc/rubydb/tls/server.crt").to_json %>
33
+ key_file: <%= ENV.fetch("RUBYDB_SSL_KEY_FILE", "/etc/rubydb/tls/server.key").to_json %>
34
+ ca_file: <%= ENV.fetch("RUBYDB_SSL_CA_FILE", "/etc/rubydb/tls/ca.crt").to_json %>
35
+ verify_peer: <%= ENV.fetch("RUBYDB_SSL_VERIFY_PEER", "false") %>
36
+ min_version: TLS1_2
data/config/test.yml ADDED
File without changes
data/docs/README.md ADDED
@@ -0,0 +1,71 @@
1
+ # RubyDB documentation index
2
+
3
+ Start here:
4
+
5
+ - [Developer guide](developer-guide.md)
6
+ - [Troubleshooting](troubleshooting.md)
7
+ - [Debugging playbook](debugging.md)
8
+ - [Production operations guide](operations/production-guide.md)
9
+ - [Quick start](getting-started/quickstart.md)
10
+ - [Local development to production](getting-started/local-to-production.md)
11
+ - [First database](getting-started/first-database.md)
12
+ - [First query](getting-started/first-query.md)
13
+ - [Rails installation](rails/installation.md)
14
+ - [Rails production guidance](rails/production.md)
15
+ - [SQLite compatibility profile](sql/sqlite-compatibility.md)
16
+ - [SQL compatibility contract](sql/compatibility.md)
17
+ - [SQL compatibility guide](sql/compatibility-guide.md)
18
+ - [Rails compatibility guide](rails/compatibility-guide.md)
19
+ - [Production validation](production_validation.md)
20
+ - [Production-readiness audit](production-readiness.md)
21
+ - [Production runbook](operations/production-runbook.md)
22
+ - [CLI guide](cli.md)
23
+ - [CLI cheat sheet](cli-cheatsheet.md)
24
+ - [Disaster recovery](operations/disaster-recovery.md)
25
+ - [Monitoring and alerting](operations/monitoring.md)
26
+ - [Workload testing](operations/workload-testing.md)
27
+ - [Release checklist](release.md)
28
+ - [Lessons learned](lessons-learned.md)
29
+
30
+ Architecture and development:
31
+
32
+ - [Current-state audit](architecture/current-state.md)
33
+ - [Production roadmap](architecture/production-roadmap.md)
34
+ - [Architecture overview](architecture/overview.md)
35
+ - [Storage engine](architecture/storage-engine.md)
36
+ - [Transactions](architecture/transactions.md)
37
+ - [MVCC](architecture/mvcc.md)
38
+ - [WAL](architecture/wal.md)
39
+ - [Query planner](architecture/query-planner.md)
40
+ - [Testing guide](contributing/testing.md)
41
+ - [Release process](contributing/release-process.md)
42
+ - [Branching](developer/branching.md)
43
+
44
+ The root [README](../README.md) is the public project overview. The current
45
+ production claim is intentionally bounded by the tested features and the
46
+ deployment-specific validation described above.
47
+
48
+ ## Documentation map
49
+
50
+ Use the guides for the complete workflow and the smaller pages for focused
51
+ reference:
52
+
53
+ * `getting-started/` gets a new Ruby or Rails application running.
54
+ * `sql/` defines syntax, types, expressions, transactions, joins, functions,
55
+ and compatibility boundaries.
56
+ * `architecture/` explains storage, WAL, recovery, MVCC, indexes, planning,
57
+ execution, server ownership, and replication.
58
+ * `developer/` covers local development, snapshots, branches, temporal data,
59
+ database diffs, and implementation workflows.
60
+ * `rails/` covers installation, configuration, migrations, production use,
61
+ compatibility, and troubleshooting.
62
+ * `server/` covers deployment, authentication, TLS, pooling, protocol, and
63
+ operational behavior.
64
+ * `operations/` covers backups, restore drills, monitoring, workload tests,
65
+ production procedures, and incident response.
66
+ * `contributing/` and the root policy files cover testing, release, governance,
67
+ support, security, and contribution requirements.
68
+
69
+ If a topic page and an implementation disagree, treat executable tests and the
70
+ documented compatibility contract as the source of truth, then open an issue
71
+ to reconcile the documentation.
@@ -0,0 +1,14 @@
1
+ # Concurrency architecture
2
+
3
+ Threads coordinate through locks, transaction ownership, the lock manager,
4
+ buffer-pool synchronization, and server worker/connection limits. Transactions
5
+ have connection-scoped state in server mode; the embedded owner remains one
6
+ process.
7
+
8
+ The lock manager detects wait-for cycles and selects a victim. The transaction
9
+ manager rolls the victim back, releases its locks, and returns a retryable
10
+ deadlock error. Timeouts and cancellation are distinct: a timeout stops a
11
+ request deadline, while wire cancellation actively marks an in-flight request.
12
+
13
+ Validate concurrency with the production soak, multi-process workload, latency
14
+ percentiles, deadlock checks, resource limits, and durable reopen verification.
@@ -0,0 +1,125 @@
1
+ # RubyDB current-state audit
2
+
3
+ > This document is a dated architecture audit, not a statement that every
4
+ > finding remains open. For current validation evidence, see
5
+ > [production validation](../production_validation.md) and the latest CI runs.
6
+
7
+ ## Executive summary
8
+
9
+ RubyDB is a broad database project with integrated storage, WAL, MVCC,
10
+ transactions, catalog, security, server, replication, backup, Rails, and
11
+ monitoring paths. The implementation now has substantial focused validation,
12
+ but several deployment-level capabilities remain environment-specific or
13
+ intentionally bounded.
14
+
15
+ The primary risk is not that the code is absent; it is that multiple components exist without being integrated into a consistent, correct, durable engine. Existing code needs additional hardening before it can safely store user data.
16
+
17
+ ## Repository audit
18
+
19
+ ### Runtime and package state
20
+
21
+ - Ruby runtime in the development container: Ruby 3.4.7
22
+ - Package metadata targets Ruby >= 3.3.0; CI covers Ruby 3.3 and 3.4 on Linux,
23
+ macOS, and Windows.
24
+ - The top-level library entrypoint and gem packaging are covered by
25
+ `require "rubydb"` and release-artifact tests.
26
+ - The project uses Bundler and RSpec as the base test stack.
27
+ - The repository is organized around a large `lib/rubydb` module tree and includes docs, examples, chaos, and fuzz directories.
28
+
29
+ ### Architectural reality
30
+
31
+ The project already contains modules for:
32
+
33
+ - storage engine and page manager
34
+ - buffer pool and file management
35
+ - WAL and recovery components
36
+ - transactions and MVCC
37
+ - catalog, indexes, and constraints
38
+ - SQL parser and planner
39
+ - server and protocol layers
40
+ - security and authorization
41
+ - replication, backups, and monitoring
42
+ - Rails integration
43
+
44
+ The remaining risk is uneven validation breadth: many capabilities are proven
45
+ by focused tests, but multi-host operations, capacity limits, and complete
46
+ external-database dialect compatibility are not yet certified.
47
+
48
+ ## High-risk findings
49
+
50
+ ### 1. Library loading (resolved)
51
+
52
+ The standard Ruby consumer path is now tested:
53
+
54
+ ```ruby
55
+ require "rubydb"
56
+ ```
57
+
58
+ Release metadata and the local gem build verify that this loads correctly.
59
+
60
+ ### 2. Storage layer (partially resolved)
61
+
62
+ The storage engine now has page validation, corruption detection, WAL/recovery,
63
+ fsync paths, restart tests, subprocess crash tests, filesystem fault injection,
64
+ and compaction/reopen coverage. Real disk-quota and power-loss validation on
65
+ deployment filesystems remains open.
66
+
67
+ ### 3. Transaction and MVCC semantics (validated foundation)
68
+
69
+ The transaction manager and visibility map exist, transaction rollback restores
70
+ update/delete before-images, and read-committed, repeatable-read, serializable
71
+ conflict, deadlock, and concurrency workload paths have regression coverage.
72
+ Distributed isolation and deployment-specific contention limits remain open.
73
+
74
+ ### 4. SQL engine is a documented subset
75
+
76
+ The parser, planner, and executor are covered through end-to-end tests for the
77
+ documented RubyDB SQL subset, including joins, aggregates, CTEs, subqueries,
78
+ set operations, upserts, and window functions. A common SQLite-style profile
79
+ is tested for new Ruby/Rails applications; full PostgreSQL/MySQL/SQLite dialect
80
+ and file-format compatibility remains intentionally out of scope for this
81
+ phase.
82
+
83
+ ### 5. Security (validated foundation)
84
+
85
+ Authentication, TLS, SCRAM, authorization, framing limits, deadlines, and
86
+ resource-limit startup checks are tied to server/query execution and tested.
87
+ An independent security review and production certificate/secret lifecycle
88
+ validation remain open.
89
+
90
+ ### 6. Production posture is intentionally bounded
91
+
92
+ The README and production documents now distinguish tested behavior from
93
+ deployment work. RubyDB still needs environment-specific capacity, filesystem,
94
+ multi-host failover, certificate lifecycle, and independent security validation
95
+ before broader production claims are appropriate.
96
+
97
+ ## Current production posture
98
+
99
+ RubyDB is best understood as:
100
+
101
+ - a serious Ruby database foundation with repeatable local validation,
102
+ - suitable for controlled workloads within the documented feature set,
103
+ - not a drop-in general-purpose database or automatic high-availability service.
104
+
105
+ The correct short-term operating posture is to treat the project as a pre-production, high-potential codebase that needs disciplined validation and narrower, correctness-first milestones.
106
+
107
+ ## Immediate action items
108
+
109
+ 1. Run the hosted Ruby/Rails/OS compatibility matrix and retain its evidence.
110
+ 2. Validate real disk quota, power-loss, and filesystem behavior on deployment targets.
111
+ 3. Validate multi-host partition, split-brain, fencing, and operator failover procedures.
112
+ 4. Complete an independent security review and certificate/secret rotation drill.
113
+ 5. Implement or intentionally reject unsupported features with explicit errors.
114
+ 6. Establish workload-specific capacity baselines and release sign-off records.
115
+
116
+ The latest local audit passed 270 examples with zero failures. Hosted matrix,
117
+ container, filesystem quota/power-loss, multi-host fencing, and independent
118
+ security-review evidence must still be retained for a production deployment.
119
+
120
+ ## Conclusion
121
+
122
+ RubyDB has a substantial production-oriented foundation and repeatable local
123
+ validation. Operators must stay within the documented feature set and complete
124
+ deployment-specific capacity, filesystem, security, and multi-host failover
125
+ validation before entrusting critical data to it.
@@ -0,0 +1,25 @@
1
+ # Execution engine
2
+
3
+ The execution pipeline is:
4
+
5
+ `SQL text -> lexer -> parser/AST -> binder -> planner -> executor -> result`
6
+
7
+ The planner selects scans, index scans, joins, filters, aggregates, sorting,
8
+ limits, set operations, CTEs, subqueries, and window operations supported by
9
+ the documented dialect. DML is routed through transaction-aware executors and
10
+ the engine's durable mutation path.
11
+
12
+ Unsupported syntax must produce an explicit parser or execution error. New
13
+ operators and functions require type-checking, null behavior, transaction
14
+ coverage, and compatibility documentation.
15
+
16
+ ## Execution invariants
17
+
18
+ The executor must consume a bound plan, check cancellation and deadlines at
19
+ bounded points, and return either a complete result or an error that leaves the
20
+ transaction in a documented state. DML goes through the same transaction and
21
+ WAL path as the Ruby API; a server response must never bypass durable commit.
22
+
23
+ When diagnosing a result, compare parser output, bound parameters, selected
24
+ plan, row visibility, and final serialization. See the [developer guide](../developer-guide.md)
25
+ and [debugging playbook](../debugging.md).