pgdd 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 2ec7b3e45b4b1e40622b867cceb8e28fdc9e4d5ac2cb19e4a75c95215581eb10
+  data.tar.gz: b4ad296661b2b1ae8a5775bb05c49801c10f569ac6d58354b242c1e0f9d518fa
+SHA512:
+  metadata.gz: ff4f072c2529929e3da0f1737b16fc744831c8a794c3bac453928925618b8375af5f124eac886644e14fea3fdb53ebee696ac3aaec376b0d216244d2d5713fcf
+  data.tar.gz: 13590142d6fa9cc19f849b6cfdb6c22f99c00f59d2d3d346d7ee37390508e7ec18b82228c80fc0eb011d4356e73dd5b5032eb9525fdeaaf9736f8eed70af28c8

data/CHANGELOG.md

@@ -0,0 +1,13 @@
+## [Unreleased]
+
+## [0.1.0] - 2026-10-22
+
+Initial release, converted from the single-file `data_dictionary_v3.rb` script.
+
+- `pgdd` command and `pgdd:generate` rake task.
+- A generated `README.md` in the output directory, from `lib/pgdd/readme_template.md`: what the files are, how to read the diagrams, and the command that produced the copy.
+- Markdown index page and one page per table: columns, indexes, foreign keys, referencing tables, check constraints, triggers, migrations and model details.
+- Graphviz ERD per table (focus table in full, related tables by key columns) and one for the whole schema; dark theme, transparent background, boxes and edges link to the pages.
+- Polymorphic associations drawn as dashed edges from the models' `has_one`/`has_many ..., as:` declarations.
+- Migrations that changed each table, found by reading `db/migrate`: schema DSL calls, SQL in `execute`, model backfills with an explicit table name, and renames.
+- Several databases (`--database`, `--all-databases`), SQLite and PostgreSQL specifics, exact or estimated row counts.

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2026 Dewayne VanHoozer
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,109 @@
+# pgdd
+
+As-built database documentation for a Rails application, with entity relationship diagrams.
+
+`pgdd` boots your application, reads its live database and its ActiveRecord models, and writes Markdown you can commit next to the code: an index page, one page per table, and a Graphviz diagram per table plus one of the whole schema. Nothing is hand-written and nothing comes from a design document. It records what the application actually has.
+
+From the database it reads tables, views, columns, keys, indexes, foreign keys, check constraints, triggers, comments, enum types, extensions, row counts and sizes. From the models it reads associations, enums, validations, STI hierarchies, store accessors, attachments, encrypted and virtual attributes. From `db/migrate` it works out which migrations changed each table.
+
+PostgreSQL gets the fullest treatment (enum types, triggers, planner row estimates, materialized views). SQLite is supported, and anything else ActiveRecord can connect to gets the parts the adapter exposes.
+
+## Installation
+
+Add it to the application's Gemfile, in the development group:
+
+```ruby
+group :development do
+  gem 'pgdd'
+end
+```
+
+The diagrams are rendered with Graphviz, so `dot` must be on the `PATH` (`brew install graphviz`, `apt install graphviz`). Without it, use `--erd-format dot` to write the DOT source instead, or `--no-erd`.
+
+Requires Ruby 3.2+ and Rails 6.1+. The gem has no runtime dependencies of its own; it runs inside the application's Rails.
+
+## Usage
+
+From the application's root:
+
+```bash
+bundle exec pgdd --output docs/database
+```
+
+Or as a rake task, which does the same thing inside the loaded application:
+
+```bash
+bin/rails "pgdd:generate[docs/database]"
+PGDD_OPTIONS="--count --all-databases" bin/rails pgdd:generate
+```
+
+Without `--output`, only the index page is written, to STDOUT.
+
+| Option                  | Effect                                                                                     |
+| ----------------------- | ------------------------------------------------------------------------------------------ |
+| `-o`, `--output DIR`    | Write the index, one page per table and the diagrams to DIR                                |
+| `-e`, `--environment`   | Rails environment (default: `RAILS_ENV` or `development`)                                  |
+| `-d`, `--database NAME` | Report this database config; repeatable (default: the primary only)                        |
+| `-a`, `--all-databases` | Report every database config (cache, queue, cable, ...)                                    |
+| `-c`, `--count`         | Exact row counts with `SELECT COUNT(*)` instead of planner estimates (can be slow)          |
+| `-i`, `--internal`      | Include `schema_migrations` and `ar_internal_metadata`                                     |
+| `-f`, `--erd-format`    | `svg` (default), `png`, `pdf`, or `dot` for the source                                     |
+| `--no-erd`              | Skip the diagrams                                                                          |
+
+`pgdd` only reads. The whole run happens inside `ActiveRecord::Base.while_preventing_writes`.
+
+## What it writes
+
+| File                 | Content                                                                                                                                                        |
+| -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `README.md`          | A guide to the directory: what the files are, how to read the diagrams, and the exact command that generated this copy. Rendered from the gem's `readme_template.md` |
+| `INDEX.md`           | The index: each database with its adapter, version and schema version; every table with a link to its page; enum types; models without a table; a data element index of every column across all tables |
+| `<table>.md`         | Columns, indexes, foreign keys, the tables that reference it, check constraints, triggers, the migrations that changed it, and the models backed by it           |
+| `<table>.svg`        | The table's entity relationship diagram, embedded in its page                                                                                                  |
+| `schema.svg`         | Every table in one diagram                                                                                                                                     |
+
+With more than one database, page and diagram names are prefixed: `queue.solid_queue_jobs.md`.
+
+The Markdown passes markdownlint and prettier as written: table pipes are aligned, and there are no hard-wrapped tables.
+
+### The diagrams
+
+Each table's diagram draws that table in full. Its related tables are collapsed to their primary key and the columns that take part in the relationship, so a table with a dozen children stays readable instead of becoming several screens tall. The full schema of a related table is one click away: every table name and column in a diagram links to that table's page. `schema.svg` draws every table the compact way.
+
+- Solid edges are foreign keys that exist in the database. Crow's foot on the many side, bar and ring on the parent key.
+- Dashed edges are polymorphic associations the models declare (`has_one`/`has_many ..., as:`), which Rails cannot back with a database foreign key. A single bar on the child end means the inverse is `has_one`.
+- Hovering shows table and column comments, and for a dashed edge the model the type column must name.
+
+Dark theme, transparent background, so they look right on both GitHub themes.
+
+**The links inside an SVG do not work on GitHub.** GitHub shows an SVG as a static image, embedded or on its own, and clicking an image opens the file. The links work when the SVG is opened from a checkout. For navigation that works everywhere, each page's Relations section opens with a linked list of the tables it references and the tables that reference it, polymorphic ones included.
+
+### Migrations
+
+Each page lists every migration that changed the table, oldest first, with a link to the file and what it did (`create_table`, `add_column availability`, `ALTER TABLE (SQL)`). The list comes from reading the migration files, not replaying them. It covers the schema DSL calls, including calls inside loops over a `%i[...]` list of tables, SQL inside `execute` strings and heredocs, backfills through a migration-local model with an explicit `table_name`, and `rename_table`, so a renamed table's page includes the migrations under its old name. Only the forward direction is read; `down` methods and `dir.down` blocks are ignored. A migration that changes data only through application models it does not name by table will not appear.
+
+## Library use
+
+The command is a thin wrapper. Inside a booted application:
+
+```ruby
+app    = Pgdd::Collector.new(root: Rails.root.to_s, count_rows: true).call   # Pgdd::App
+report = Pgdd::Report.new(app, erd_format: 'svg', output_dir: 'docs/database')
+report.pages                                                                 # { 'README.md' => '...', 'INDEX.md' => '...', 'users.md' => '...' }
+Pgdd::ErdWriter.new(app.databases.first, models: app.models).diagrams        # { 'users' => 'digraph ...', 'schema' => '...' }
+```
+
+`Pgdd::App` and everything under it are plain `Data` values, so `Report` and `ErdWriter` need no database and are tested without one. `Pgdd::MigrationScanner.tables(source)` works on any migration source string.
+
+## Development
+
+```bash
+bundle install
+bundle exec rake test
+```
+
+The tests cover the scanner, the report and the diagram builder with hand-built values. Running the command end to end needs a Rails application; point it at one with `bundle exec bin/pgdd path/to/app --output /tmp/dd`.
+
+## License
+
+MIT. See [LICENSE.txt](LICENSE.txt).

data/bin/pgdd

@@ -0,0 +1,10 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+# Runs from a checkout (bin on the PATH) as well as from the installed gem
+lib = File.expand_path('../lib', __dir__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+
+require 'pgdd'
+
+exit Pgdd::CLI.run(ARGV)

data/bundler_audit_output.txt

@@ -0,0 +1,10 @@
+Updating ruby-advisory-db ...
+From https://github.com/rubysec/ruby-advisory-db
+ * branch            master     -> FETCH_HEAD
+Already up to date.
+Updated ruby-advisory-db
+ruby-advisory-db:
+  advisories:	1266 advisories
+  last updated:	2026-10-21 00:11:45 +0200
+  commit:	5fd86bbd406995ea4ed2df84651b209e28072031
+No vulnerabilities found

data/fasterer_output.txt

@@ -0,0 +1 @@
+11 files inspected, 0 offenses detected

data/flay_output.txt

@@ -0,0 +1 @@
+Flay: no structural duplication detected (mass >= 150)

data/flog_output.txt

@@ -0,0 +1,29 @@
+Flog warnings (20.0–50.0):
+  42.2: Pgdd::Report#database_summary
+  39.0: Pgdd::Report::markdown_table
+  36.5: Pgdd::Report#related_tables
+  36.2: Pgdd::ErdWriter#key_columns
+  35.9: Pgdd::ErdWriter::node
+  34.6: Pgdd::Report#orphan_models
+  34.5: Pgdd::ModelInspector#associations
+  34.3: Pgdd::Report#data_element_index
+  34.0: Pgdd::MigrationScanner::structural_changes
+  33.5: Pgdd::CLI#parse_options
+  29.8: Pgdd::Report#table_summary
+  29.7: Pgdd::CLI#run
+  27.4: Pgdd::ErdWriter#relations
+  27.1: Pgdd::ErdWriter#polymorphic_key
+  25.6: Pgdd::ErdWriter::edge
+  24.3: Pgdd::Report::human_size
+  24.2: Pgdd::Report#database_section
+  23.5: Pgdd::Report#model_detail
+  22.8: Pgdd::ErdWriter#table_dot
+  22.4: Pgdd::Report#column_keys
+  22.2: Pgdd::Report::format_version
+  22.0: Pgdd::MigrationScanner::forward_source
+  22.0: Pgdd::Collector#call
+  22.0: Pgdd::Report#model_extras
+  21.7: Pgdd::Report#table_facts
+
+Flog failures (>= 50.0):
+  (none)

data/lib/pgdd/cli.rb

@@ -0,0 +1,118 @@
+# frozen_string_literal: true
+
+module Pgdd
+  # The pgdd command: parses options, boots the application unless it is
+  # already loaded (bin/rails runner, a rake task), collects, and writes
+  class CLI
+    DEFAULTS = { environment: nil, output: nil, count_rows: false, include_internal: false, databases: [],
+                 all_databases: false, erd: true, erd_format: 'svg' }.freeze
+
+    # Runs with argv and returns the process exit status
+    def self.run(argv, out: $stdout, err: $stderr)
+      new(out: out, err: err).run(argv)
+    end
+
+    def initialize(out: $stdout, err: $stderr)
+      @out = out
+      @err = err
+    end
+
+    def run(argv)
+      options = parse_options(argv)
+      return show(options[:help]) if options[:help]
+
+      root = rails_root(options)
+      return fail_with("not a Rails application: #{root}") unless File.exist?(File.join(root, 'config', 'environment.rb'))
+
+      boot(root, options[:environment])
+      ActiveRecord::Base.logger = nil
+      app = ActiveRecord::Base.while_preventing_writes do
+        Collector.new(root: root, **options.slice(:count_rows, :include_internal, :databases, :all_databases)).call
+      end
+      write(app, options, command_line(argv))
+    rescue OptionParser::ParseError, ErdWriter::Error => e
+      fail_with(e.message)
+    rescue Errno::ENOENT => e
+      raise unless e.message.end_with?('- dot')
+
+      fail_with('Graphviz dot not found; install graphviz or use --erd-format dot')
+    end
+
+    def show(text)
+      @out.puts text
+      0
+    end
+
+    def fail_with(message)
+      @err.puts "ERROR: #{message}"
+      1
+    end
+
+    # Rails.root when running inside the application, else the argument
+    def rails_root(options)
+      defined?(::Rails.root) && ::Rails.root ? ::Rails.root.to_s : options[:root]
+    end
+
+    # The pages and diagrams into the output directory, or the index to STDOUT
+    def write(app, options, command)
+      dir        = options[:output]
+      erd_format = options[:erd] && dir ? options[:erd_format] : nil
+      report     = Report.new(app, erd_format: erd_format, output_dir: dir, command: command)
+
+      if dir
+        FileUtils.mkdir_p(dir)
+        report.pages.each { |name, markdown| File.write(File.join(dir, name), markdown) }
+        write_erds(app, report, dir, erd_format) if erd_format
+      else
+        @out.print report.index_markdown
+      end
+      0
+    end
+
+    # The invocation as typed, for the generated README
+    def command_line(argv)
+      (['pgdd'] + argv.map { |a| a.match?(/[\s"']/) ? a.inspect : a }).join(' ')
+    end
+
+    # The option hash for argv; :root is the Rails root argument, expanded
+    def parse_options(argv)
+      options = DEFAULTS.transform_values(&:dup)
+
+      parser = OptionParser.new do |opts|
+        opts.banner = 'Usage: pgdd [options] [RAILS_ROOT]'
+        opts.on('-e', '--environment ENV', 'Rails environment (default: RAILS_ENV or development)') { |v| options[:environment] = v }
+        opts.on('-o', '--output DIR', "Write #{Report::INDEX}, a page per table and the diagrams to DIR",
+                "(without it only #{Report::INDEX} is written, to STDOUT)") { |v| options[:output] = File.expand_path(v) }
+        opts.on('-d', '--database NAME', 'Report this database config (repeatable; default: the primary only)') { |v| options[:databases] << v }
+        opts.on('-a', '--all-databases', 'Report every database config (cache, queue, cable, ...)') { options[:all_databases] = true }
+        opts.on('-c', '--count', 'Exact row counts with SELECT COUNT(*) (can be slow)') { options[:count_rows] = true }
+        opts.on('-i', '--internal', 'Include schema_migrations and ar_internal_metadata') { options[:include_internal] = true }
+        opts.on('-f', '--erd-format FORMAT', ErdWriter::FORMATS, "ERD format: #{ErdWriter::FORMATS.join(', ')} (default: svg; needs Graphviz dot unless dot)") { |v| options[:erd_format] = v }
+        opts.on('--no-erd', 'Skip the ERD diagrams') { options[:erd] = false }
+        opts.on('-v', '--version', 'Show the version') { options[:help] = "pgdd #{VERSION}" }
+        opts.on('-h', '--help', 'Show this help') { options[:help] = opts.help }
+      end
+
+      options[:root] = File.expand_path(parser.parse(argv).first || '.')
+      options
+    end
+
+    # Loads the application unless we are already inside it
+    def boot(root, environment)
+      return if defined?(::Rails.application) && ::Rails.application
+
+      ENV['RAILS_ENV'] = environment if environment
+      ENV['RAILS_ENV'] ||= 'development'
+      Dir.chdir(root)
+      require File.join(root, 'config', 'environment')
+    end
+
+    # Diagrams share the report's file stems so the pages and the links inside the SVGs line up
+    def write_erds(app, report, dir, format)
+      app.databases.reject(&:error).flat_map do |db|
+        models = app.models.select { |m| m.database == db.name }
+        ErdWriter.new(db, models: models, name_for: ->(table_name) { report.basename(db.name, table_name) }).write(dir, format: format)
+      end
+    end
+  end
+end

data/lib/pgdd/collector.rb

@@ -0,0 +1,109 @@
+# frozen_string_literal: true
+
+module Pgdd
+  # Reads the already-booted Rails application's databases and models into
+  # an App. Only reads: callers wrap it in while_preventing_writes.
+  class Collector
+    def initialize(root:, count_rows: false, include_internal: false, databases: [], all_databases: false)
+      @root             = root
+      @count_rows       = count_rows
+      @include_internal = include_internal
+      @databases        = databases
+      @all_databases    = all_databases
+    end
+
+    def call
+      configs = database_configs(@databases, all: @all_databases)
+      missing = configs.to_h { |c| [c.name, missing_sqlite_file(c.adapter, c.database)&.delete_prefix("#{@root}/")] }.compact
+      models  = inspect_models(configs.map(&:name), missing)
+      dbs     = inspect_databases(configs, missing)
+
+      App.new(name: app_name, root: @root, environment: Rails.env.to_s, rails_version: Rails.version,
+              ruby_version: RUBY_VERSION, databases: dbs, models: models)
+    end
+
+    def app_name
+      app_class = Rails.application.class
+      app_class.respond_to?(:module_parent_name) ? app_class.module_parent_name : app_class.parent_name
+    end
+
+    # Models on the named databases. Inspecting a model can open its
+    # connection, so a model on a missing database is reported, not inspected
+    def inspect_models(names, missing)
+      ModelInspector.models.filter_map do |m|
+        inspector = ModelInspector.new(m, root: @root)
+        db_name   = inspector.database_name
+        next unless names.include?(db_name)
+        next Model.new(name: m.name, table_name: m.table_name, database: db_name, error: "database file not found: #{missing[db_name]}") if missing[db_name]
+
+        inspector.call
+      end
+    end
+
+    # A Database per config; one that cannot be read is returned with its error
+    def inspect_databases(configs, missing)
+      database_connections(configs).map do |name, klass|
+        missing[name] ? missing_database(name, klass, missing[name]) : inspect_database(name, klass)
+      rescue StandardError => e
+        Database.new(name: name, error: "#{e.class}: #{e.message.lines.first&.strip}")
+      end
+    end
+
+    def missing_database(name, klass, file)
+      config = klass.connection_db_config
+      Database.new(name: name, adapter: config.adapter, location: config.database, error: "database file not found: #{file}")
+    end
+
+    def inspect_database(name, klass)
+      with_connection(klass) do |connection|
+        DatabaseInspector.new(connection, name: name, root: @root, count_rows: @count_rows, include_internal: @include_internal).call
+      end
+    end
+
+    # The configs to report: the ones named, or all of them, or just the primary
+    def database_configs(only = [], all: false)
+      configs = ActiveRecord::Base.configurations.configs_for(env_name: Rails.env)
+      configs = configs.reject { |c| c.respond_to?(:replica?) && c.replica? }
+      return configs.select { |c| only.include?(c.name) } if only.any?
+      return configs if all
+
+      primary = ActiveRecord::Base.connection_db_config.name
+      configs.select { |c| c.name == primary }
+    end
+
+    # [[config_name, connection_class], ...]; a secondary database gets an
+    # abstract class of its own, reused if one was already made for it
+    def database_connections(configs)
+      primary = ActiveRecord::Base.connection_db_config.name
+
+      configs.map do |config|
+        next [config.name, ActiveRecord::Base] if config.name == primary
+
+        [config.name, connection_class_for(config.name)]
+      end
+    end
+
+    def connection_class_for(config_name)
+      const = :"Connection#{config_name.to_s.gsub(/\W/, '_').capitalize}"
+      return self.class.const_get(const, false) if self.class.const_defined?(const, false)
+
+      klass = self.class.const_set(const, Class.new(ActiveRecord::Base) { self.abstract_class = true })
+      klass.establish_connection(config_name.to_sym)
+      klass
+    end
+
+    # Connecting to a missing SQLite file would create an empty one
+    def missing_sqlite_file(adapter, database)
+      return nil unless adapter.to_s.start_with?('sqlite') && database && !database.start_with?(':memory:', 'file:')
+
+      path = File.expand_path(database, @root)
+      path unless File.exist?(path)
+    end
+
+    def with_connection(klass, &)
+      return klass.with_connection(&) if klass.respond_to?(:with_connection)
+
+      yield klass.connection
+    end
+  end
+end