rubydb 0.1.0

data/docs/sql/compatibility-guide.md

@@ -0,0 +1,82 @@
+# SQL compatibility guide
+
+This page is a developer reference for evaluating SQL support. The normative
+statement list is [SQL compatibility](compatibility.md); the SQLite-style
+application profile is [here](sqlite-compatibility.md). RubyDB does not claim
+complete PostgreSQL, MySQL, or SQLite dialect compatibility.
+
+## Compatibility has four layers
+
+An SQL feature is production-ready only when all four layers agree:
+
+1. syntax is parsed and rejected with useful errors;
+2. binding and type rules are defined, including `NULL`;
+3. planner/executor results and transaction semantics are correct; and
+4. Ruby, client/server, and ActiveRecord paths behave consistently.
+
+A parser accepting a statement is not sufficient. Test duplicate rows,
+ordering, aggregates, joins, constraints, rollback, savepoints, indexes,
+timeouts, and recovery for every feature that mutates or reads data.
+
+## Query review checklist
+
+For each query shape, record schema, indexes, bind types, expected rows,
+expected ordering, transaction isolation, and error behavior. Test empty,
+single-row, duplicate, `NULL`, and large-result cases. Explicitly order results
+when the application requires order; relational results are not inherently
+ordered.
+
+Review:
+
+* projections and aliases;
+* boolean precedence and `NULL` truth tables;
+* inner, left, and multi-table joins;
+* grouping, aggregate input, and `HAVING`;
+* limits and offsets at boundaries;
+* conflict/upsert behavior and affected-row counts;
+* casts, defaults, and constraint errors; and
+* snapshot visibility during concurrent writes.
+
+## Application migration from SQLite
+
+Start with a schema dump and a representative query/migration inventory. Run
+the application test suite against RubyDB, not only a few hand-written
+queries. Review pragmas, virtual tables, FTS, extensions, custom functions,
+file-format assumptions, recursive queries, date/time behavior, collations,
+and error handling. These areas are not included merely because the common
+profile looks SQLite-like.
+
+Never point RubyDB at an existing SQLite file and expect it to open. Migrate
+through an explicit export/import process with checksums, row counts, and
+application-level verification.
+
+## SQL change workflow
+
+Add a contract example under `spec/sql`, parser tests, execution tests, failure
+tests, and adapter coverage when relevant. Update the compatibility page and
+release notes. Add a regression for every bug. If semantics are intentionally
+different from another database, document the difference with a migration
+recommendation.
+
+## Safe examples
+
+```sql
+CREATE TABLE IF NOT EXISTS users (
+  id INTEGER PRIMARY KEY,
+  email TEXT NOT NULL UNIQUE,
+  active BOOLEAN DEFAULT TRUE
+);
+
+INSERT INTO users (email, active) VALUES (?, ?);
+
+SELECT active, COUNT(*) AS total
+FROM users
+WHERE email IS NOT NULL
+GROUP BY active
+HAVING COUNT(*) > 0
+ORDER BY active;
+```
+
+Use bound parameters from Ruby or ActiveRecord. Do not construct SQL by
+concatenating user input.
+

data/docs/sql/compatibility.md

@@ -0,0 +1,39 @@
+# RubyDB SQL compatibility
+
+RubyDB supports a deliberately documented SQL dialect. It includes a tested
+SQLite-style compatibility profile for common new Ruby/Rails applications (see
+`sqlite-compatibility.md`), but this is not a claim of complete SQLite,
+PostgreSQL, or MySQL compatibility.
+
+Supported and tested statements include:
+
+- `SELECT` with projections, `WHERE`, ordering, limits, offsets, built-in
+  expressions, and `COUNT`, `SUM`, `AVG`, `MIN`, and `MAX` aggregates (including
+  `DISTINCT` aggregate arguments) with
+  `GROUP BY` and `HAVING`
+- window ranking functions `ROW_NUMBER`, `RANK`, and `DENSE_RANK`, plus
+  partition-wide and explicit `ROWS`-framed aggregate windows using
+  `OVER (PARTITION BY ... ORDER BY ...)`
+- `UNION`, `UNION ALL`, `INTERSECT`, and `EXCEPT` for compatible SELECT
+  projections
+- non-correlated scalar subqueries, `IN (SELECT ...)`, and `EXISTS (SELECT ...)`
+  predicates
+- materialized non-recursive and bounded recursive `WITH` common table
+  expressions
+- `INSERT`, `UPDATE`, and `DELETE`, including multi-row `VALUES`,
+  `ON CONFLICT DO NOTHING`, and targeted `ON CONFLICT (...) DO UPDATE SET ...`
+  with `excluded.column`
+- `CREATE TABLE` and `DROP TABLE`, including primary keys, unique constraints,
+  checks, foreign keys, and referential actions
+- `CREATE INDEX`/`CREATE UNIQUE INDEX` and `DROP INDEX`
+- `ALTER TABLE` add/drop columns and constraints
+- `CREATE`/`DROP DATABASE`, `CREATE`/`DROP SCHEMA`, and `CREATE`/`DROP VIEW`
+- supported trigger metadata DDL: `CREATE TRIGGER ... EXECUTE FUNCTION ...`
+- `BEGIN`, `COMMIT`, `ROLLBACK`, `SAVEPOINT`, `ROLLBACK TO SAVEPOINT`, and
+  `RELEASE SAVEPOINT`
+- `EXPLAIN`, `EXPLAIN ANALYZE`, and `VACUUM`
+
+Unsupported syntax must fail with a parser or execution error. RubyDB does not
+promise arbitrary SQL extensions, PostgreSQL wire-level compatibility, or
+feature parity with another database engine. The executable contract is the
+integration coverage under `spec/`.

data/docs/sql/data-types.md

@@ -0,0 +1,10 @@
+# SQL data types
+
+The documented RubyDB types are `INTEGER`, `BIGINT`, `SMALLINT`, `FLOAT`,
+`DECIMAL`, `BOOLEAN`, `TEXT`, `VARCHAR`, `BLOB`, `DATE`, `TIME`, `TIMESTAMP`,
+`JSON`, and `UUID`. Type conversion, nullability, defaults, and constraint
+behavior are validated through the SQL and Rails suites.
+
+RubyDB does not claim storage or coercion compatibility with every other SQL
+engine. Preserve types explicitly in migrations and test application boundary
+values, nulls, booleans, timestamps, decimals, JSON, and binary data.

data/docs/sql/functions.md

@@ -0,0 +1,9 @@
+# SQL functions
+
+The tested function surface includes `COUNT`, `SUM`, `AVG`, `MIN`, `MAX`,
+`LOWER`, `UPPER`, `LENGTH`, `SUBSTR`, `CONCAT`, `COALESCE`, and `NULLIF`, plus
+the documented window ranking and aggregate functions.
+
+Function null behavior and return types are part of the compatibility contract.
+Do not assume a PostgreSQL, MySQL, or SQLite extension exists because a function
+has the same name; unsupported functions must be reported explicitly.

data/docs/sql/joins.md

@@ -0,0 +1,9 @@
+# SQL joins
+
+RubyDB supports qualified `INNER`, `LEFT [OUTER]`, `RIGHT`, `FULL [OUTER]`, and
+`CROSS JOIN` forms with `ON` predicates in the documented dialect. Outer joins
+preserve unmatched rows with null-extended columns.
+
+Use explicit table qualification when columns are ambiguous. Validate join
+cardinality, null behavior, ordering, grouping, and transaction visibility with
+representative data. Join reordering is limited to safe inner-join plans.

data/docs/sql/operators.md

@@ -0,0 +1,9 @@
+# SQL operators and predicates
+
+The documented surface includes arithmetic, comparison, boolean `AND`/`OR` and
+`NOT`, `IS NULL`/`IS NOT NULL`, `BETWEEN`, `IN`, `EXISTS`, `LIKE`, `ILIKE`, and
+three-valued null behavior covered by the SQL tests.
+
+Use bound parameters for user input. Test false, null, empty string, numeric
+precision, and mixed-type values explicitly; similar syntax across SQL engines
+does not guarantee identical semantics.

data/docs/sql/sqlite-compatibility.md

@@ -0,0 +1,21 @@
+# SQLite compatibility profile
+
+RubyDB now maintains an explicit SQLite-style compatibility profile for new
+Ruby and Rails applications. The profile is tested by
+`spec/sqlite_compatibility_spec.rb` and covers the common application surface:
+
+- `CREATE TABLE IF NOT EXISTS`, integer primary keys with `AUTOINCREMENT`,
+  defaults, `NOT NULL`, and `UNIQUE`
+- CRUD, parameter binding through the Rails connection, transactions, and
+  rollback
+- `WHERE`, `IS NULL`/boolean predicates, ordering, `LIMIT`/`OFFSET`, joins,
+  grouped aggregates, and `HAVING`
+- targeted `ON CONFLICT ... DO UPDATE` upserts
+- ActiveRecord schema inspection through the embedded adapter
+
+This is useful for applications written against the documented RubyDB surface,
+but it is not a complete replacement for the SQLite library. SQLite pragmas,
+virtual tables, FTS, recursive query edge cases, extension APIs, file-format
+compatibility, and every SQLite function/error behavior remain unsupported or
+unverified. An existing SQLite application must run its own migration and
+query suite before migration.

data/docs/sql/syntax.md

@@ -0,0 +1,10 @@
+# SQL syntax
+
+Statements are parsed by RubyDB's lexer and parser and executed through the
+planner. The supported statements include `SELECT`, `INSERT`, `UPDATE`,
+`DELETE`, table/index/schema/view DDL, transactions/savepoints, `EXPLAIN`, and
+`VACUUM` as listed in [compatibility](compatibility.md).
+
+Use semicolons for multiple statements only where the client path supports
+them. Identifiers may be quoted with double quotes or backticks. Unsupported
+syntax fails with a parser or execution error; it is never silently ignored.

data/docs/sql/transactions.md

@@ -0,0 +1,10 @@
+# SQL transactions
+
+RubyDB supports `BEGIN`, `COMMIT`, `ROLLBACK`, `SAVEPOINT`, `ROLLBACK TO
+SAVEPOINT`, and `RELEASE SAVEPOINT`. Commits coordinate MVCC, locks, row
+mutation, WAL, and durable acknowledgement.
+
+Keep transactions short enough for the workload. Long-running readers can
+retain old versions and delay vacuum; deadlock victims must retry the
+application unit. Validate transaction behavior after restart and under
+concurrency.

data/docs/troubleshooting.md

@@ -0,0 +1,244 @@
+# RubyDB troubleshooting guide
+
+This guide is for developers and operators diagnosing a live or test
+deployment. Preserve evidence before attempting recovery. If data is
+important, stop writes, copy the database directory and WAL to protected
+storage, record the RubyDB version and commit, and work on a copy.
+
+## First response
+
+Collect the smallest useful incident bundle:
+
+```sh
+ruby -v
+bundle exec ruby -Ilib exe/rubydb --version
+bundle exec ruby -Ilib exe/rubydb status --config config/rubydb.yml
+bundle exec ruby -Ilib exe/rubydb doctor --config config/rubydb.yml
+```
+
+Also record the operating system, deployment topology, database path (without
+credentials), configuration checksum, recent migrations, request IDs, error
+logs, disk/free-inode state, process list, and whether the failure affects
+embedded mode, server mode, or both. Redact passwords, tokens, private keys,
+certificate contents, and customer values.
+
+Do not delete WAL files, run vacuum, force promotion, or retry an unknown
+commit until the state and evidence are preserved.
+
+## Startup and configuration
+
+### The database will not open
+
+Check that the path is the intended directory, is readable and writable by the
+service account, and is not already owned by another embedded process. Inspect
+the status and doctor output. If the path contains an incomplete checkpoint,
+use the documented recovery flow and keep the original directory unchanged.
+
+Common causes are a wrong working directory, a missing parent directory,
+permissions, a stale lock, an unsupported format version, and an incomplete
+restore. A stale lock must be investigated against the process owner; never
+remove it merely because startup is inconvenient.
+
+### The server starts and immediately exits
+
+Run the same command in the foreground with verbose logging. Validate the
+configuration, TLS files, certificate/key pairing, authentication settings,
+listen address, and port availability. Check the service manager’s stdout and
+stderr rather than only its health endpoint. `RUBYDB_DEBUG=1` can expose a
+development backtrace; do not enable verbose debug output on a public service
+without reviewing sensitive data exposure.
+
+### The server is healthy but clients cannot connect
+
+Confirm the client endpoint, port, TLS mode, CA path, SNI/hostname, and
+authentication credentials. Test from the same network namespace as the
+application. A listening socket proves only that a process has bound a port;
+the readiness check must also verify the database owner and request path.
+
+## SQL and query failures
+
+### A statement is rejected
+
+Capture the exact SQL shape with values redacted and determine whether the
+failure is parser, binder, planner, executor, or constraint related. Compare
+the statement against [SQL compatibility](sql/compatibility.md) and the
+[SQLite-style profile](sql/sqlite-compatibility.md). Test a minimal statement
+with one table, then add joins, predicates, grouping, and constraints one at a
+time.
+
+Do not assume that syntax accepted by SQLite, PostgreSQL, or MySQL has the
+same semantics in RubyDB. Unsupported dialect features should be rewritten or
+tracked as compatibility work, not hidden behind a generic fallback.
+
+### Results are wrong or unstable
+
+Reproduce without the optimizer if that diagnostic mode exists, then compare
+the plan and row versions. Add explicit ordering when order is part of the
+application contract. Check `NULL` predicates, implicit casts, duplicate join
+keys, grouping columns, transaction snapshot, and stale statistics. Preserve
+the schema, seed data, SQL, and expected result as a regression spec.
+
+### A query is slow
+
+Record query shape, row count, indexes, bind values, plan, duration, lock wait,
+and whether the delay is execution or checkpoint/WAL pressure. Test with and
+without the suspected index and inspect the scan cardinality. Do not add
+indexes blindly: each index adds write, storage, recovery, and vacuum cost.
+
+## Transactions, locks, and concurrency
+
+### Requests hang
+
+Separate network wait, lock wait, disk wait, and executor work. Check active
+transactions, lock owners, waiters, deadlines, and cancellation logs. Set a
+bounded request/lock timeout in staging and capture a thread dump. A timeout
+must release resources and report whether commit was known.
+
+### Deadlock detected
+
+Keep the deadlock graph, victim transaction ID, SQL fingerprints, and lock
+order. Confirm that the victim rolled back all writes and released every lock.
+Retry only idempotent application work. Fix the application’s lock ordering or
+transaction size; do not disable deadlock detection.
+
+### Data disappears inside a transaction
+
+Check snapshot timing, savepoints, rollback paths, and whether the read and
+write use the same connection. In server mode, a transaction is connection
+scoped unless the API says otherwise. A connection-pool checkout must not
+reuse a connection with an open transaction.
+
+### High concurrency causes errors
+
+Reduce workers and payload size, then increase one at a time. Monitor memory,
+file descriptors, WAL growth, checkpoint time, lock waits, cancellation rate,
+and p99 latency. Embedded mode has one process owner; use server mode for
+multiple application processes. Use the production soak scripts before
+changing limits.
+
+## WAL, recovery, and corruption
+
+### Recovery takes too long
+
+Measure WAL size, last checkpoint LSN, frame count, page count, and storage
+latency. A large WAL may indicate a blocked checkpoint, a long reader, or
+insufficient checkpoint scheduling. Preserve the directory, then run a copy
+through inspection and compaction. Never truncate WAL by hand.
+
+### Checksum or framing validation fails
+
+Treat this as a durability incident. Stop writes, preserve all database and WAL
+files, capture filesystem and process termination evidence, and attempt restore
+from the newest verified backup in a separate directory. Compare checksums and
+LSNs. If the backup also fails, escalate with the complete evidence bundle.
+
+### An index is inconsistent
+
+Do not make application decisions from a suspect index. Run the supported
+inspection/recovery or rebuild operation on a copy, compare indexed and table
+scans, and verify after reopen. The repair result must be durable before the
+copy is promoted. A failed index persistence operation must remain visible as
+an error.
+
+### Disk is full
+
+Stop growth safely: pause writes if necessary, preserve logs, and identify
+database, WAL, backup, and temporary-file usage. Free capacity through the
+host’s approved procedure, then verify the filesystem and resume with a
+controlled write. Do not delete WAL, active checkpoints, or the newest backup
+to make room.
+
+## Backup, restore, and upgrades
+
+### Backup succeeds but restore fails
+
+Check the backup manifest, checksum, base identity, LSN range, compression,
+RubyDB version, and free space in the target directory. Restore to a fresh
+directory and run integrity, schema, row-count, and application smoke checks.
+Keep the failed restore for diagnosis.
+
+### Incremental/differential chain is rejected
+
+Restore the verified base first and apply deltas in order. Confirm that the
+declared base checksum and LSN match. Missing or reordered pieces must fail
+closed; do not bypass validation.
+
+### An upgrade will not start
+
+Read the upgrade guard and format version. Take a verified backup, test the
+upgrade on a copy with a representative workload, and retain a rollback plan.
+Do not mix binary versions against a live embedded directory unless the
+compatibility contract explicitly allows it.
+
+## Replication and failover
+
+### Replica is behind
+
+Compare primary and replica acknowledged LSN, WAL retention, connection state,
+authentication, and apply errors. Check that the replica is not accepting
+writes. Reconnect/catch up through the supported protocol and verify row and
+LSN consistency before promotion.
+
+### Promotion is rejected
+
+Promotion should require a synchronized candidate and a valid fencing epoch.
+Resolve lag, stale metadata, or fence ownership first. Never force promotion
+because an application health check is red; a stale primary may still be able
+to write.
+
+### Suspected split brain
+
+Fence both writers at the deployment layer, stop application writes, preserve
+both logs and WALs, and identify the highest acknowledged commit/fence epoch.
+Do not merge divergent database directories manually. The current RubyDB
+workflow is explicitly fenced/manual unless a deployment has separately
+validated its election and fencing system across hosts.
+
+## Rails and migrations
+
+Check the Rails/Ruby versions, adapter configuration, connection pool size,
+database ownership mode, generated SQL, bind values, and migration version.
+Run `db:migrate:status`, inspect schema state, and compare the schema dump with
+the intended model. For a populated-table migration, test on a copy and plan
+backfill, locking, rollback, and deployment sequencing.
+
+Frequent causes include unsupported generated SQL, an embedded path shared by
+web and job processes, pool size exceeding server capacity, a migration
+checksum mismatch, or a schema dump that uses a feature outside RubyDB’s
+documented profile. See [Rails compatibility](rails/compatibility-guide.md)
+and [Rails troubleshooting](rails/troubleshooting.md).
+
+## TLS, authentication, and authorization
+
+Verify the server certificate chain, hostname, expiration, key permissions,
+CA bundle, and client/server TLS settings. Rotate certificates by staging the
+new chain, testing a client, switching atomically, and retaining the old chain
+only for the documented overlap period. Rotate secrets through the secret
+manager, not source control or command-line history.
+
+Authentication success does not imply authorization. Check the authenticated
+identity, role, operation, database, and audit record. Treat repeated failures
+as a security event and preserve timestamps and request IDs.
+
+## Resource exhaustion
+
+Track open files, memory, threads, CPU, disk bytes, inodes, WAL size, active
+transactions, pool utilization, and queue depth. Apply limits at the service
+and application layers. Reduce concurrency before raising limits, and confirm
+that cancellation and shutdown drain work without corrupting state.
+
+## Evidence and escalation
+
+An actionable report includes:
+
+* exact command/API and a minimal reproduction;
+* RubyDB commit/version, Ruby/Rails versions, OS and filesystem;
+* topology, configuration names, and relevant metrics;
+* sanitized logs with request/transaction/LSN identifiers;
+* database/WAL/backup checksums and sizes;
+* expected versus actual result; and
+* what was already attempted and whether it changed state.
+
+See [Debugging RubyDB](debugging.md) for collection commands and safe
+instrumentation.
+

data/exe/rubydb

@@ -0,0 +1,23 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+# RubyDB executable - This is the gem's main executable that gets installed
+# when the gem is installed. It delegates to bin/rubydb.
+
+# This file is the entry point that RubyGems uses when the gem is installed.
+# It allows users to run `rubydb` from anywhere after `gem install rubydb`.
+
+require_relative "../lib/rubydb"
+require_relative "../lib/rubydb/cli/application"
+
+# Load and execute the CLI application
+begin
+  RubyDB::CLI::Application.start(ARGV)
+rescue Interrupt
+  puts "\nInterrupted. Exiting..."
+  exit 0
+rescue StandardError => e
+  $stderr.puts "Error: #{e.message}"
+  $stderr.puts e.backtrace if ENV["RUBYDB_DEBUG"]
+  exit 1
+end