rubydb 0.1.0

data/CHANGELOG.md

@@ -0,0 +1,31 @@
+# Changelog
+
+All notable changes to RubyDB are documented here. Versions follow
+[Semantic Versioning](https://semver.org/).
+
+## Unreleased
+
+- Added full developer, troubleshooting, debugging, production operations,
+  Rails compatibility, and SQL compatibility guides with repository maps,
+  incident evidence procedures, safe recovery guidance, and deployment
+  checklists.
+- Added RubyDB connection URLs (`rubydb://` and TLS-enabled `rubydbs://`) for
+  regular Ruby clients and Rails `database.yml`, including percent-encoded
+  credentials and documented TLS/query options.
+- Added a beginner-friendly local-to-production guide covering local RubyDB,
+  PostgreSQL migration, RubyDB server deployment, environment URLs, Render-style
+  hosting, data transfer, smoke tests, and production checklists.
+- Added a documentation index and lessons-learned guide covering durability,
+  ownership, replication fencing, cancellation, deadlocks, Rails adapters,
+  operations, evidence, and security review boundaries.
+- Added a full CLI guide and cheat sheet covering lifecycle, server operation,
+  shell, migrations, backup/restore, snapshots, branches, inspection, doctor,
+  vacuum, maintenance, and release workflows.
+- Clarified the tested common SQLite-style profile and production limits.
+
+## 0.1.0 - 2026-09-07
+
+- Initial public release candidate.
+- Includes the embedded engine, ActiveRecord adapter, backup/recovery, WAL,
+  TLS, authentication, and validated production-hardening checkpoints.
+- See `docs/production_validation.md` for tested scope and operational limits.

data/CODE_OF_CONDUCT.md

@@ -0,0 +1,13 @@
+# Contributor Code of Conduct
+
+RubyDB contributors are expected to make participation respectful, practical,
+and safe. Harassment, discrimination, threats, personal attacks, doxxing,
+deliberate disruption, and unwanted sexual attention are not acceptable.
+
+Project maintainers may moderate comments, discussions, issues, and pull
+requests when conduct violates this standard. They should explain the reason,
+apply the rule consistently, and protect people who report concerns.
+
+Report a conduct concern privately to the repository maintainers through a
+GitHub private security/contact channel. Do not include production secrets or
+customer data. Reports are handled confidentially as far as practical.

data/CONTRIBUTING.md

@@ -0,0 +1,29 @@
+# Contributing to RubyDB
+
+RubyDB is a correctness-first database project. Small, reviewable changes with
+tests and documentation are preferred over broad changes with unverified
+claims.
+
+## Development
+
+```sh
+bundle install
+bundle exec rspec
+bundle exec rubocop
+```
+
+Use a temporary database for tests. Do not commit database files, credentials,
+generated private keys, coverage output, or release artifacts. Follow the
+architecture and testing guides in `docs/`.
+
+## Pull requests
+
+Explain the invariant being changed, the failure mode it prevents, and the
+rollback or compatibility impact. Add regression coverage for parser,
+execution, storage, transaction, protocol, adapter, or operational changes.
+Run the relevant workload, crash, backup/restore, and security checks. Update
+the compatibility contract when behavior changes.
+
+Never describe a simulated fault test as proof of physical power-loss,
+multi-host, or independent security-review results. See the
+[lessons learned](docs/lessons-learned.md) guide.

data/GOVERNANCE.md

@@ -0,0 +1,16 @@
+# RubyDB governance
+
+RubyDB is maintained through reviewed changes in the public repository. The
+maintainers are responsible for release decisions, security coordination,
+compatibility claims, and accepting changes that affect data safety.
+
+Changes to storage format, WAL, recovery, transaction isolation, replication,
+authentication, protocol framing, or release automation require focused tests,
+documentation, and an explicit operational-impact review.
+
+Security vulnerabilities should be reported privately as described in
+`SECURITY.md`. Public issues are appropriate for reproducible bugs that do not
+expose an unpatched vulnerability.
+
+The project follows semantic versioning while compatibility is still evolving.
+Release tags must pass the protected CI and RubyGems release checks.

data/Gemfile

@@ -0,0 +1,17 @@
+# frozen_string_literal: true
+
+source 'https://rubygems.org'
+
+gemspec
+
+gem 'rake', '~> 13.2'
+gem 'rspec', '~> 3.13'
+gem 'rubocop', '~> 1.65', require: false
+gem 'rubocop-performance', '~> 1.21', require: false
+gem 'standard', '~> 1.39', require: false
+gem 'simplecov', '~> 0.22', require: false
+
+# Optional development dependencies
+gem 'pry', '~> 0.14', group: :development
+gem 'pry-byebug', '~> 3.10', group: :development
+gem 'memory_profiler', '~> 1.0', group: :development

data/Gemfile.lock

@@ -0,0 +1,71 @@
+PATH
+  remote: .
+  specs:
+    rubydb (0.1.0)
+
+GEM
+  remote: https://rubygems.org/
+  specs:
+    ast (2.4.2)
+    base64 (0.2.0)
+    json (2.7.2)
+    language_server-protocol (3.17.0.3)
+    parallel (1.24.0)
+    parser (3.3.1.0)
+      ast (~> 2.4.1)
+      racc
+    racc (1.8.0)
+    rainbow (3.1.1)
+    rake (13.2.1)
+    regexp_parser (2.9.2)
+    rexml (3.2.6)
+    rspec (3.13.0)
+      rspec-core (~> 3.13.0)
+      rspec-expectations (~> 3.13.0)
+      rspec-mocks (~> 3.13.0)
+    rspec-core (3.13.0)
+      rspec-support (~> 3.13.0)
+    rspec-expectations (3.13.0)
+      diff-lcs (>= 1.2.0, < 2.0)
+      rspec-support (~> 3.13.0)
+    rspec-mocks (3.13.0)
+      diff-lcs (>= 1.2.0, < 2.0)
+      rspec-support (~> 3.13.0)
+    rspec-support (3.13.1)
+    rubocop (1.63.4)
+      json (~> 2.3)
+      language_server-protocol (>= 3.17.0)
+      parallel (~> 1.10)
+      parser (>= 3.3.0.2)
+      rainbow (>= 2.2.2, < 4.0)
+      regexp_parser (>= 1.8, < 3.0)
+      rexml (>= 3.2.5, < 4.0)
+      rubocop-ast (>= 1.31.1, < 2.0)
+      ruby-progressbar (~> 1.7)
+      unicode-display_width (>= 2.4.0, < 3.0)
+    rubocop-ast (1.31.3)
+      parser (>= 3.3.0.2)
+    rubocop-performance (1.21.0)
+      rubocop (>= 1.48.1, < 2.0)
+      rubocop-ast (>= 1.31.1, < 2.0)
+    ruby-progressbar (1.13.0)
+    standard (1.35.1)
+      language_server-protocol (~> 3.17.0.2)
+      rubocop (~> 1.63.0)
+      rubocop-performance (~> 1.21.0)
+    unicode-display_width (2.5.0)
+
+PLATFORMS
+  arm64-darwin-23
+  ruby
+
+DEPENDENCIES
+  rake (~> 13.2)
+  rspec (~> 3.13)
+  rubocop (~> 1.60)
+  rubocop-performance (~> 1.20)
+  rubydb!
+  standard (~> 1.35)
+
+BUNDLED WITH
+   2.5.9

data/LICENSE

@@ -0,0 +1,60 @@
+# MIT License
+
+Copyright (c) Aldane Hutchinson
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+
+---
+
+## Additional Disclaimer for Alpha Status
+
+**⚠️ IMPORTANT: RubyDB is currently in ALPHA development.**
+
+This software is provided "AS IS" without warranty of any kind. It is not yet
+production ready. Use in development and testing environments only. Data loss
+is possible. The API is subject to change without notice.
+
+---
+
+## Third-Party Licenses
+
+RubyDB includes dependencies with their own licenses:
+
+| Dependency | License |
+|------------|---------|
+| concurrent-ruby | MIT |
+| json | Ruby License |
+| zlib | zlib License |
+| openssl | Ruby License |
+| digest | Ruby License |
+| fileutils | Ruby License |
+| activerecord | MIT |
+| rake | MIT |
+| rspec | MIT |
+| rubocop | MIT |
+
+All dependencies are compatible with the MIT License.
+
+---
+
+## Contributing Agreement
+
+By contributing to this project, you agree that your contributions will be
+licensed under the MIT License. You also agree that your contributions are
+your original work and you have the right to license them.

data/README.md

@@ -0,0 +1,162 @@
+# RubyDB
+
+RubyDB is a Ruby-native relational database with an embedded engine, a
+client/server mode, a Ruby client, and an ActiveRecord adapter.
+
+> **Status: alpha.** RubyDB is suitable for experimentation, development,
+> controlled embedded workloads, and applications that stay within the
+> documented and tested feature set. It provides a tested common SQLite-style
+> profile, but is not a drop-in replacement for PostgreSQL, MySQL, or SQLite.
+
+## What works today
+
+The repository contains implementation and automated coverage for:
+
+- SQL tables, CRUD, joins, grouping and aggregates, ordering, transactions,
+  savepoints, conflict handling, and documented maintenance statements
+- typed values, primary/foreign keys, unique and check constraints, and B-tree
+  indexes
+- durable storage, WAL-backed commits, recovery, snapshots, branching, and
+  MVCC paths
+- Ruby API, client/server protocol, connection pooling, configuration, and
+  operational tooling
+- ActiveRecord integration, Rails migrations, and a runnable Rails example
+
+These features are not a guarantee of compatibility with every application.
+Run the test suite and validate your own schema, queries, workload, backup,
+restore, and failure scenarios before using RubyDB for important data.
+
+## Why “complete PostgreSQL/MySQL/SQLite compatibility” matters
+
+That requirement is only necessary when RubyDB is intended to be a drop-in
+replacement for an existing application using one of those databases.
+
+It includes much more than accepting similar `SELECT` statements:
+
+- dialect-specific SQL syntax, functions, operators, casts, and error behavior
+- query semantics for joins, `NULL`, ordering, grouping, subqueries, CTEs,
+  unions, upserts, and window functions
+- data types, indexes, constraints, generated values, and transaction behavior
+- migration behavior and ActiveRecord adapter mappings
+- client protocol, connection behavior, locking, limits, and operational tools
+
+A new Ruby or Rails application does not need complete compatibility. It can
+use RubyDB's documented SQL and adapter behavior directly. Compatibility is
+needed to move an existing PostgreSQL, MySQL, or SQLite application without
+rewriting queries and without discovering semantic differences in production.
+
+RubyDB currently targets a documented RubyDB SQL subset plus tested Rails
+operations. The compatibility documents describe the supported statements;
+unsupported or unverified dialect features must not be assumed to work.
+
+## Quick start
+
+Install the prerelease gem:
+
+```sh
+gem install rubydb --pre
+```
+
+For local development from this repository:
+
+```sh
+bundle install
+bundle exec rspec
+```
+
+## Ruby usage
+
+RubyDB can run embedded in a single owning process:
+
+```ruby
+require "rubydb"
+
+engine = RubyDB::Storage::Engine.new("tmp/example.rdb")
+engine.execute("CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT NOT NULL)")
+engine.execute("INSERT INTO users (name) VALUES ('Aldane')")
+puts engine.execute("SELECT * FROM users").inspect
+engine.close
+```
+
+For multiple application processes, use RubyDB's server/client mode and point
+clients at the managed server. Do not open the same embedded database path
+from multiple independent processes. A regular Ruby application can use a
+RubyDB connection URL supplied by its environment:
+
+```ruby
+client = RubyDB::Client::Client.new(url: ENV.fetch("RUBYDB_URL"))
+client.query("SELECT 1")
+client.disconnect
+```
+
+Use the `rubydb://` or TLS-enabled `rubydbs://` format documented in the Rails
+configuration guide. RubyDB URLs are not PostgreSQL URLs.
+
+## Rails example
+
+The small Rails 7.2 application in
+[`examples/rails_app`](examples/rails_app) runs a real migration, model query,
+and browser form through `rubydb-activerecord`.
+
+```sh
+cd examples/rails_app
+bundle install
+bundle exec ruby bin/rails db:migrate
+bundle exec ruby bin/rails server -b 127.0.0.1 -p 3001
+```
+
+Open <http://127.0.0.1:3001/>. The example uses an embedded database under
+`tmp/`; set `RUBYDB_DATABASE` to choose another path. See the adapter and Rails
+documentation for network configuration, migrations, production deployment,
+backups, restore drills, and monitoring.
+
+For an even smaller end-to-end smoke test, see the tiny GitHub-style app in
+[`examples/github_clone`](examples/github_clone). It covers repositories,
+issues, commits, Rails associations, foreign keys, indexes, seed data, and a
+browser page backed by RubyDB.
+
+## Compatibility policy
+
+RubyDB does not claim complete PostgreSQL, MySQL, or SQLite compatibility until
+each compatibility area has both an implementation and repeatable validation.
+The project must validate at least:
+
+1. parser and execution behavior for the documented dialect surface
+2. type, constraint, transaction, locking, and error semantics
+3. ActiveRecord queries, joins, eager loading, associations, and migrations
+4. sustained concurrency, cancellation, recovery, backup/restore, and failover
+5. supported Ruby, Rails, operating-system, and client/server combinations
+
+Until then, compatibility should be treated as feature-specific, not implied
+by the presence of an adapter.
+
+## Documentation
+
+- [Documentation index](docs/README.md)
+- [Developer guide](docs/developer-guide.md)
+- [Troubleshooting guide](docs/troubleshooting.md)
+- [Debugging playbook](docs/debugging.md)
+- [Production operations guide](docs/operations/production-guide.md)
+- [Lessons learned](docs/lessons-learned.md)
+- [Getting started](docs/getting-started/quickstart.md)
+- [Local development to production](docs/getting-started/local-to-production.md)
+- [SQL compatibility](docs/sql/compatibility.md)
+- [SQL compatibility guide](docs/sql/compatibility-guide.md)
+- [SQLite compatibility profile](docs/sql/sqlite-compatibility.md)
+- [SQL syntax](docs/sql/syntax.md)
+- [Rails installation](docs/rails/installation.md)
+- [Rails production guidance](docs/rails/production.md)
+- [Rails compatibility guide](docs/rails/compatibility-guide.md)
+- [Production readiness](docs/production-readiness.md)
+- [Operations and workload testing](docs/operations/workload-testing.md)
+- [Production runbook](docs/operations/production-runbook.md)
+- [CLI guide](docs/cli.md)
+- [CLI cheat sheet](docs/cli-cheatsheet.md)
+- [Release checklist](docs/release.md)
+- [Security policy](SECURITY.md)
+- [Contributing and testing](CONTRIBUTING.md)
+- [Roadmap](ROADMAP.md)
+
+## License
+
+RubyDB is released under the [MIT License](LICENSE).

data/ROADMAP.md

@@ -0,0 +1,27 @@
+# RubyDB roadmap
+
+## Current phase: validated foundation
+
+RubyDB has a durable storage/WAL/recovery foundation, transactions and MVCC,
+the documented SQL engine, server/client protocol, Rails adapter coverage,
+backups, logical replication, monitoring, fuzzing, and release automation.
+The latest local audit passed 270 examples with zero failures.
+
+## Next gates
+
+1. Retain hosted Ruby/Rails/OS matrix evidence for each release.
+2. Run physical filesystem quota, power-loss, corruption, restore, and compaction
+   drills on deployment targets.
+3. Validate multi-host replication partitions, independent fencing, stale
+   primary rejection, and operator promotion procedures.
+4. Complete certificate/secret rotation validation and an independent security
+   review.
+5. Establish workload-specific latency, throughput, capacity, RPO, and RTO
+   baselines.
+
+## Longer term
+
+Expand the documented SQLite-style profile and only then consider broader
+PostgreSQL/MySQL compatibility where implementation and semantic validation can
+be maintained. Add automatic election only with an independently tested fencing
+authority. Preserve correctness and explicit unsupported errors as hard gates.

data/Rakefile

@@ -0,0 +1,71 @@
+# frozen_string_literal: true
+
+require 'bundler/gem_tasks'
+require 'rspec/core/rake_task'
+require 'rbconfig'
+require 'digest'
+require 'fileutils'
+
+RSpec::Core::RakeTask.new(:spec)
+
+task default: %i[spec rubocop]
+
+namespace :build do
+  desc 'Build the gem and write a SHA-512 checksum for the exact artifact'
+  task checksum: :build do
+    gem_path = Dir['pkg/rubydb-*.gem'].max_by { |path| File.mtime(path) }
+    abort 'No built gem found in pkg/' unless gem_path
+
+    FileUtils.mkdir_p('checksums')
+    checksum_path = File.join('checksums', "#{File.basename(gem_path)}.sha512")
+    File.write(checksum_path, "#{Digest::SHA512.file(gem_path).hexdigest}  #{File.basename(gem_path)}\n")
+    puts "Wrote #{checksum_path}"
+  end
+end
+
+desc 'Run all tests'
+task test: :spec
+
+desc 'Run unit tests only'
+RSpec::Core::RakeTask.new(:'test:unit') do |t|
+  t.pattern = 'test/unit/**/*_spec.rb'
+end
+
+desc 'Run integration tests only'
+RSpec::Core::RakeTask.new(:'test:integration') do |t|
+  t.pattern = 'test/integration/**/*_spec.rb'
+end
+
+begin
+  require 'rubocop/rake_task'
+  RuboCop::RakeTask.new
+rescue LoadError
+  # rubocop not installed
+end
+
+namespace :test do
+  desc 'Run concurrency tests'
+  RSpec::Core::RakeTask.new(:concurrency) do |t|
+    t.pattern = 'test/concurrency/**/*_spec.rb'
+  end
+
+  desc 'Run crash tests'
+  RSpec::Core::RakeTask.new(:crash) do |t|
+    t.pattern = 'test/crash/**/*_spec.rb'
+  end
+
+  desc 'Run chaos tests'
+  RSpec::Core::RakeTask.new(:chaos) do |t|
+    t.pattern = 'test/chaos/**/*_spec.rb'
+  end
+end
+
+desc 'Run benchmarks'
+task :benchmark do
+  Dir['benchmarks/**/*.rb'].each { |f| load f }
+end
+
+desc 'Run the concurrent workload and durability verification'
+task :workload do
+  abort 'Concurrent workload failed' unless system(RbConfig.ruby, 'benchmarks/concurrent_workload.rb')
+end

data/SECURITY.md

@@ -0,0 +1,54 @@
+# RubyDB security policy
+
+RubyDB is alpha software with a documented, tested feature set. Security
+controls are implemented and tested for the server protocol, TLS transport,
+password/SCRAM authentication, authorization, bounded frames, query deadlines,
+resource limits, replication tokens, path validation, backup manifests, and
+fencing. A deployment must still complete an independent security review
+before it handles sensitive or regulated data.
+
+## Secure deployment requirements
+
+- Keep the database directory, WAL, backups, TLS private keys, authentication
+  credentials, and replication state on protected storage with least-privilege
+  filesystem ownership.
+- Bind the server and replication listeners to explicit private addresses. Do
+  not expose them directly to the public internet; restrict ingress with a
+  firewall or private network.
+- Enable TLS with TLS 1.2 or newer, validate the server certificate, and use a
+  trusted CA or mutual TLS where the deployment requires peer identity.
+- Use high-entropy credentials and a separate high-entropy replication token.
+  Never put secrets in source control, URLs, shell history, logs, or bug
+  reports.
+- Configure maximum connections, request/frame sizes, query deadlines, and
+  filesystem thresholds for the host. Treat readiness failure and recovery-
+  required acknowledgements as incidents.
+- Run backups with encryption approved by the organization, retain the
+  manifest and WAL chain together, and perform restore drills into a new
+  inactive directory.
+
+## Rotation and incident response
+
+Stage replacement certificates and credentials, validate them on a non-serving
+instance, then rotate during a planned window. Drain clients or replication
+peers, change both sides of a replication token, reconnect, and verify the
+authentication-failure metric returns to baseline. Retain the old certificate
+only for the documented overlap window and remove it afterward.
+
+For suspected compromise, corruption, checksum failure, replication
+divergence, or fencing anomalies: stop writes, isolate the affected node,
+preserve the database/WAL/log files and timestamps, and create a verified copy
+before attempting repair, vacuum, restore, or promotion. Automatic election is
+disabled; never run two writable primaries without an externally verified
+fencing lease.
+
+## Reporting a vulnerability
+
+Do not disclose unpatched vulnerabilities in a public issue. Use a private
+GitHub security advisory for the repository or contact the maintainer listed in
+the gem metadata with reproduction steps, affected versions, impact, and a
+safe disclosure timeline. Do not include production secrets or customer data.
+
+Security scans run in GitHub Actions, but scan results do not replace review of
+deployment configuration, network policy, secret storage, certificate
+lifecycle, or the host filesystem.

data/SUPPORT.md

@@ -0,0 +1,14 @@
+# RubyDB support policy
+
+RubyDB currently supports the documented RubyDB SQL surface, the tested common
+SQLite-style profile, the Ruby client/server path, and the ActiveRecord adapter
+combinations listed by CI. It does not promise complete PostgreSQL, MySQL, or
+SQLite dialect, file-format, extension, or wire compatibility.
+
+When requesting help, include RubyDB commit/version, Ruby and Rails versions,
+OS, topology, configuration shape without secrets, a minimal reproduction, and
+the exact error. Include workload and recovery evidence for data-safety issues.
+
+For corruption, checksum, replication divergence, or fencing anomalies, stop
+writes and preserve the database, WAL, logs, and timestamps before attempting
+repair. Report unpatched security issues privately through `SECURITY.md`.

data/adapters/activerecord/Gemfile

@@ -0,0 +1,11 @@
+# frozen_string_literal: true
+
+source "https://rubygems.org"
+
+gemspec
+
+rails_version = ENV.fetch("RUBYDB_RAILS_VERSION", "7.2")
+gem "activerecord", "~> #{rails_version}.0"
+gem "rake", "~> 13.2"
+gem "rspec", "~> 3.13"
+gem "rubocop", "~> 1.60"

data/adapters/activerecord/README.md

@@ -0,0 +1,9 @@
+# RubyDB ActiveRecord adapter
+
+The `rubydb-activerecord` gem connects Rails applications to RubyDB through
+ActiveRecord. It supports embedded RubyDB engines and network connections.
+
+For an executable Rails 7.2 example, see
+[`examples/rails_app`](../../examples/rails_app). The example includes a
+database configuration, migration, model/controller, browser form, and a
+smoke test that exercises the adapter against a real RubyDB engine.