activerecord-pg-format-db-structure 0.1.4 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b1360637cb5c24ce89e734dd5bd544c80cd29e019102e2ea7268e49ab32f6f71
-  data.tar.gz: 4ff8913438ffb2991e1c23d734cd8b5b8e0327b1286ca483c5d46e2b422ac7c4
+  metadata.gz: 2a0b4f6730ac7a280afddd46b476a8443e4cbbb3eac4004b92692223437f432c
+  data.tar.gz: b23c3a6ed71d6c46efa516d21f4c546dbdd3d2638f7c60046af0ff015d576f4d
 SHA512:
-  metadata.gz: 5b95bb982ba987fed4a4ae273a337e3d00ee738012bf6c913a81c4d87b2c0e1a3e5ba5caa55bb829df6c242143ffff59e8cecfaab1ab4c517da2232d081023e1
-  data.tar.gz: 9e6b856118e64bf1de2370206057082f9e34cfea208c201c71f0cc6cdd52ad6c9bdd1c06b0c0ea5e4183b6a9711a126724a9c6398471d1e5fd32ec3c689a4bd7
+  metadata.gz: f74568c5dcdecd2b0fe20d0823167e97a59da8593d534f877370b8e91b333a1a1c397c37913d5c81e91d8ccdfd254fc594f930ea1a0dab1ada589815ed8c38b5
+  data.tar.gz: 0eec46264e79d1ff729e12bef71bf6fd90291dbaff13ed88ae259f4e26b15cf7584ddb91a581d05d00df6c5b1ce4add028b4dc5f1318b32f89ef34dbbb8b0ddb

data/CHANGELOG.md

@@ -1,5 +1,13 @@
 ## [Unreleased]
 
+## [0.2.0] - 2025-02-15
+
+- Remove preprocessors (no longer relevant now that we don't reuse the source string in the output)
+
+## [0.1.5] - 2025-02-08
+
+- Sort table columns
+
 ## [0.1.4] - 2025-02-08
 
 - Sort schema migrations inserts

data/README.md

@@ -1,6 +1,11 @@
 # activerecord-pg-format-db-structure
+[![Gem Version](https://img.shields.io/gem/v/activerecord-pg-format-db-structure)](https://rubygems.org/gems/activerecord-pg-format-db-structure)
+![GitHub Actions Workflow Status](https://img.shields.io/github/actions/workflow/status/ReifyAB/activerecord-pg-format-db-structure/main.yml)
 
-Automatically cleans up your `structure.sql` file after each rails migration.
+
+Automatically cleans up your PostgreSQL `structure.sql` file after each rails migration.
+
+Say good-bye to small those small diffs you get between coworkers!
 
 By default, it will:
 
@@ -9,9 +14,13 @@ By default, it will:
 * Inline table constraints
 * Move index creation below their corresponding tables
 * Group `ALTER TABLE` statements into a single statement per table
-* Removes unnecessary whitespace
+* Sorts table column declarations (primary key / foreign keys / data / timestamp / constraints)
+* Sorts `schema_migrations` inserts
+* Format and indent the entire file consistently
+
+It can also optionally inline foreign key declarations (see below).
 
-The task will transform this raw `structure.sql`:
+As an example, the task will transform this raw `structure.sql`:
 
 <details>
 
@@ -179,7 +188,7 @@ INSERT INTO "schema_migrations" (version) VALUES
 ```
 </details>
 
-into this much more compact and normalized version:
+into this normalize (and much more compatch & readable) version:
 
 ```sql
 
@@ -225,8 +234,9 @@ INSERT INTO schema_migrations (version) VALUES
 ;
 ```
 
-which is a lot more compact, easier to read, and reduces the risk of
-getting random diffs between machines after each migration.
+The goal is to make your `structure.sql` file easier to read and to
+reduce the risk of getting random diffs between machines after each
+migration.
 
 Those transformations are made by manipulating the SQL AST directly
 using [pg_query](https://github.com/pganalyze/pg_query), and each
@@ -240,7 +250,7 @@ You can also add your own transforms (see below).
 Add the following to your Gemfile:
 
 ```ruby
-gem 'activerecord-clean-db-structure'
+gem 'activerecord-pg-format-db-structure'
 ```
 
 ## Usage
@@ -253,10 +263,6 @@ If you want to configure which transforms to use, you can configure the library
 
 ```ruby
 Rails.application.configure do
-  config.activerecord_pg_format_db_structure.preprocessors = [
-    ActiveRecordPgFormatDbStructure::Preprocessors::RemoveWhitespaces
-  ]
-
   config.activerecord_pg_format_db_structure.transforms = [
     ActiveRecordPgFormatDbStructure::Transforms::RemoveCommentsOnExtensions,
     ActiveRecordPgFormatDbStructure::Transforms::SortSchemaMigrations,
@@ -265,7 +271,8 @@ Rails.application.configure do
     ActiveRecordPgFormatDbStructure::Transforms::InlineSerials,
     ActiveRecordPgFormatDbStructure::Transforms::InlineConstraints,
     ActiveRecordPgFormatDbStructure::Transforms::MoveIndicesAfterCreateTable,
-    ActiveRecordPgFormatDbStructure::Transforms::GroupAlterTableStatements
+    ActiveRecordPgFormatDbStructure::Transforms::GroupAlterTableStatements,
+    ActiveRecordPgFormatDbStructure::Transforms::SortTableColumns,
   ]
 
   config.activerecord_pg_format_db_structure.deparser = ActiveRecordPgFormatDbStructure::Deparser
@@ -282,12 +289,6 @@ formatted = ActiveRecordPgFormatDbStructure::Formatter.new.format(structure)
 File.write("db/structure.sql", formatted)
 ```
 
-## Preprocessors
-
-### RemoveWhitespaces
-
-Remove unnecessary comment, whitespase and empty lines.
-
 ## Transformers
 
 ### RemoveCommentsOnExtensions
@@ -339,6 +340,49 @@ table.
 
 Should be run after other operations that inline alter statements.
 
+### SortTableColumns
+
+Sort table columns, by order of priority and alphabetically:
+
+ 1. primary key
+ 2. foreign keys
+ 3. generic columns
+ 4. timestamps
+ 5. constraints
+
+Note that you can define your own ordering by replacing the default `priority_mapping`:
+
+```ruby
+ActiveRecordPgFormatDbStructure::Transforms::SortTableColumns.priority_mapping = lambda do |sortable_entry|
+  case sortable_entry
+  in is_column: true, is_primary_key: true, name:
+    [0, name]
+  in is_column: true, is_foreign_key: true, name:
+    [1, name]
+  in is_column: true, is_timestamp: false, name:
+    [2, name]
+  in is_column: true, is_timestamp: true, name:
+    [3, name]
+  in is_constraint: true, name:
+    [5, name]
+  end
+end
+```
+
+where `sortable_entry` is an instance of:
+
+```ruby
+SORTABLE_ENTRY = Data.define(
+  :name,
+  :is_column,
+  :is_constraint,
+  :is_primary_key,
+  :is_foreign_key,
+  :is_timestamp,
+  :raw_entry
+)
+```
+
 ## Deparser
 
 Returns an SQL string from raw PgQuery statements.

data/lib/activerecord-pg-format-db-structure/formatter.rb

@@ -6,23 +6,17 @@ require_relative "../activerecord-pg-format-db-structure"
 module ActiveRecordPgFormatDbStructure
   # Formats & normalizes in place the given SQL string
   class Formatter
-    attr_reader :preprocessors, :transforms, :deparser
+    attr_reader :transforms, :deparser
 
     def initialize(
-      preprocessors: DEFAULT_PREPROCESSORS,
       transforms: DEFAULT_TRANSFORMS,
       deparser: DEFAULT_DEPARSER
     )
-      @preprocessors = preprocessors
       @transforms = transforms
       @deparser = deparser
     end
 
     def format(source)
-      preprocessors.each do |preprocessor|
-        preprocessor.new(source).preprocess!
-      end
-
       raw_statements = PgQuery.parse(source).tree.stmts
 
       transforms.each do |transform|

data/lib/activerecord-pg-format-db-structure/railtie.rb

@@ -4,7 +4,6 @@ module ActiveRecordPgFormatDbStructure
   # Setup for Rails
   class Railtie < Rails::Railtie
     config.activerecord_pg_format_db_structure = ActiveSupport::OrderedOptions.new
-    config.activerecord_pg_format_db_structure.preprocessors = DEFAULT_PREPROCESSORS.dup
     config.activerecord_pg_format_db_structure.transforms = DEFAULT_TRANSFORMS.dup
     config.activerecord_pg_format_db_structure.deparser = DEFAULT_DEPARSER
 

data/lib/activerecord-pg-format-db-structure/transforms/sort_table_columns.rb

@@ -0,0 +1,220 @@
+# frozen_string_literal: true
+
+require_relative "base"
+
+module ActiveRecordPgFormatDbStructure
+  module Transforms
+    # Sort table columns
+    class SortTableColumns < Base
+      SORTABLE_ENTRY = Data.define(
+        :name,
+        :is_column,
+        :is_constraint,
+        :is_primary_key,
+        :is_foreign_key,
+        :is_timestamp,
+        :raw_entry
+      )
+
+      class << self
+        attr_accessor :priority_mapping
+      end
+
+      self.priority_mapping = lambda do |sortable_entry|
+        case sortable_entry
+        in is_column: true, is_primary_key: true, name:
+          [0, name]
+        in is_column: true, is_foreign_key: true, name:
+          [1, name]
+        in is_column: true, is_timestamp: false, name:
+          [2, name]
+        in is_column: true, is_timestamp: true, name:
+          [3, name]
+        in is_constraint: true, name:
+          [5, name]
+          # :nocov:
+          # non-reachable else
+          # :nocov:
+        end
+      end
+
+      def transform!
+        foreign_keys = extract_foreign_keys
+        primary_keys = extract_primary_keys
+        raw_statements.each do |raw_statement|
+          next unless raw_statement.stmt.to_h in create_stmt: { relation: { schemaname:, relname: } }
+
+          raw_statement.stmt.create_stmt.table_elts.sort_by! do |table_elt|
+            elt_priority(
+              table_elt:,
+              primary_keys: primary_keys.fetch({ schemaname:, relname: }, Set.new),
+              foreign_keys: foreign_keys.fetch({ schemaname:, relname: }, Set.new)
+            )
+          end
+        end
+      end
+
+      private
+
+      def extract_primary_keys
+        primary_keys = {}
+        raw_statements.each do |raw_statement|
+          case raw_statement.stmt.to_h
+          in alter_table_stmt: {
+            objtype: :OBJECT_TABLE,
+            relation: {
+              schemaname:,
+              relname:
+            }
+          }
+            primary_keys[{ schemaname:, relname: }] ||= Set.new
+            raw_statement.stmt.alter_table_stmt.cmds.each do |cmd|
+              extract_primary_keys_from_alter_table_cmd(cmd:).each do |key|
+                primary_keys[{ schemaname:, relname: }] << key
+              end
+            end
+          in create_stmt: { relation: { schemaname:, relname: } }
+            primary_keys[{ schemaname:, relname: }] ||= Set.new
+            raw_statement.stmt.create_stmt.table_elts.each do |table_elt|
+              extract_primary_keys_from_table_elt(table_elt:).each do |key|
+                primary_keys[{ schemaname:, relname: }] << key
+              end
+            end
+          else
+          end
+        end
+        primary_keys
+      end
+
+      def extract_primary_keys_from_alter_table_cmd(cmd:)
+        if cmd.to_h in {
+          alter_table_cmd: {
+            subtype: :AT_AddConstraint,
+            def: {
+              constraint: {
+                contype: :CONSTR_PRIMARY
+              }
+            }
+          }
+        }
+          cmd.alter_table_cmd.def.constraint.keys.map do |key|
+            key.string.sval
+          end
+        else
+          []
+        end
+      end
+
+      def extract_primary_keys_from_table_elt(table_elt:)
+        case table_elt.to_h
+        in column_def: { constraints: [*, {constraint: {contype: :CONSTR_PRIMARY}}, *] }
+          [table_elt.column_def.colname]
+        in constraint: { contype: :CONSTR_PRIMARY }
+          table_elt.constraint.keys.map do |key|
+            key.string.sval
+          end
+        else
+          []
+        end
+      end
+
+      def extract_foreign_keys
+        foreign_keys = {}
+        raw_statements.each do |raw_statement|
+          case raw_statement.stmt.to_h
+          in alter_table_stmt: {
+            objtype: :OBJECT_TABLE,
+            relation: {
+              schemaname:,
+              relname:
+            }
+          }
+            foreign_keys[{ schemaname:, relname: }] ||= Set.new
+            raw_statement.stmt.alter_table_stmt.cmds.each do |cmd|
+              extract_foreign_keys_from_alter_table_cmd(cmd:).each do |key|
+                foreign_keys[{ schemaname:, relname: }] << key
+              end
+            end
+          in create_stmt: { relation: { schemaname:, relname: } }
+            foreign_keys[{ schemaname:, relname: }] ||= Set.new
+            raw_statement.stmt.create_stmt.table_elts.each do |table_elt|
+              extract_foreign_keys_from_table_elt(table_elt:).each do |key|
+                foreign_keys[{ schemaname:, relname: }] << key
+              end
+            end
+          else
+          end
+        end
+        foreign_keys
+      end
+
+      def extract_foreign_keys_from_alter_table_cmd(cmd:)
+        if cmd.to_h in {
+          alter_table_cmd: {
+            subtype: :AT_AddConstraint,
+            def: {
+              constraint: {
+                contype: :CONSTR_FOREIGN
+              }
+            }
+          }
+        }
+          cmd.alter_table_cmd.def.constraint.fk_attrs.map do |fk_attr|
+            fk_attr.string.sval
+          end
+        else
+          []
+        end
+      end
+
+      def extract_foreign_keys_from_table_elt(table_elt:)
+        case table_elt.to_h
+        in column_def: { constraints: [*, {constraint: {contype: :CONSTR_FOREIGN}}, *] }
+          [table_elt.column_def.colname]
+        in constraint: { contype: :CONSTR_FOREIGN }
+          table_elt.constraint.fk_attrs.map do |fk_attr|
+            fk_attr.string.sval
+          end
+        else
+          []
+        end
+      end
+
+      def elt_priority(table_elt:, primary_keys:, foreign_keys:)
+        case table_elt.to_h
+        in column_def: _
+          self.class.priority_mapping[
+            SORTABLE_ENTRY.new(
+              name: table_elt.column_def.colname,
+              is_column: true,
+              is_constraint: false,
+              is_primary_key: primary_keys.include?(table_elt.column_def.colname),
+              is_timestamp: timestamp?(table_elt.column_def),
+              is_foreign_key: foreign_keys.include?(table_elt.column_def.colname),
+              raw_entry: table_elt
+            )
+          ]
+        in constraint: _
+          self.class.priority_mapping[
+            SORTABLE_ENTRY.new(
+              name: table_elt.constraint.conname,
+              is_column: false,
+              is_constraint: true,
+              is_primary_key: false,
+              is_timestamp: false,
+              is_foreign_key: false,
+              raw_entry: table_elt
+            )
+          ]
+          # :nocov:
+          # non-reachable else
+          # :nocov:
+        end
+      end
+
+      def timestamp?(table_elt)
+        table_elt.type_name.names.any? { |name| name.to_h in string: { sval: "timestamp" } }
+      end
+    end
+  end
+end

data/lib/activerecord-pg-format-db-structure/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module ActiveRecordPgFormatDbStructure
-  VERSION = "0.1.4"
+  VERSION = "0.2.0"
 end

data/lib/activerecord-pg-format-db-structure.rb

@@ -3,7 +3,6 @@
 require_relative "activerecord-pg-format-db-structure/version"
 
 require_relative "activerecord-pg-format-db-structure/deparser"
-require_relative "activerecord-pg-format-db-structure/preprocessors/remove_whitespaces"
 require_relative "activerecord-pg-format-db-structure/transforms/remove_comments_on_extensions"
 require_relative "activerecord-pg-format-db-structure/transforms/inline_serials"
 require_relative "activerecord-pg-format-db-structure/transforms/inline_primary_keys"
@@ -12,12 +11,9 @@ require_relative "activerecord-pg-format-db-structure/transforms/move_indices_af
 require_relative "activerecord-pg-format-db-structure/transforms/inline_constraints"
 require_relative "activerecord-pg-format-db-structure/transforms/group_alter_table_statements"
 require_relative "activerecord-pg-format-db-structure/transforms/sort_schema_migrations"
+require_relative "activerecord-pg-format-db-structure/transforms/sort_table_columns"
 
 module ActiveRecordPgFormatDbStructure
-  DEFAULT_PREPROCESSORS = [
-    Preprocessors::RemoveWhitespaces
-  ].freeze
-
   DEFAULT_TRANSFORMS = [
     Transforms::RemoveCommentsOnExtensions,
     Transforms::SortSchemaMigrations,
@@ -26,7 +22,8 @@ module ActiveRecordPgFormatDbStructure
     Transforms::InlineSerials,
     Transforms::InlineConstraints,
     Transforms::MoveIndicesAfterCreateTable,
-    Transforms::GroupAlterTableStatements
+    Transforms::GroupAlterTableStatements,
+    Transforms::SortTableColumns
   ].freeze
 
   DEFAULT_DEPARSER = Deparser

metadata

@@ -1,13 +1,13 @@
 --- !ruby/object:Gem::Specification
 name: activerecord-pg-format-db-structure
 version: !ruby/object:Gem::Version
-  version: 0.1.4
+  version: 0.2.0
 platform: ruby
 authors:
 - Jell
 bindir: exe
 cert_chain: []
-date: 2025-02-08 00:00:00.000000000 Z
+date: 2025-02-15 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: pg_query
@@ -42,7 +42,6 @@ files:
 - lib/activerecord-pg-format-db-structure/deparser.rb
 - lib/activerecord-pg-format-db-structure/formatter.rb
 - lib/activerecord-pg-format-db-structure/indenter.rb
-- lib/activerecord-pg-format-db-structure/preprocessors/remove_whitespaces.rb
 - lib/activerecord-pg-format-db-structure/railtie.rb
 - lib/activerecord-pg-format-db-structure/tasks/clean_db_structure.rake
 - lib/activerecord-pg-format-db-structure/transforms/base.rb
@@ -54,6 +53,7 @@ files:
 - lib/activerecord-pg-format-db-structure/transforms/move_indices_after_create_table.rb
 - lib/activerecord-pg-format-db-structure/transforms/remove_comments_on_extensions.rb
 - lib/activerecord-pg-format-db-structure/transforms/sort_schema_migrations.rb
+- lib/activerecord-pg-format-db-structure/transforms/sort_table_columns.rb
 - lib/activerecord-pg-format-db-structure/version.rb
 homepage: https://github.com/ReifyAB/activerecord-pg-format-db-structure
 licenses:

data/lib/activerecord-pg-format-db-structure/preprocessors/remove_whitespaces.rb

@@ -1,27 +0,0 @@
-# frozen_string_literal: true
-
-module ActiveRecordPgFormatDbStructure
-  module Preprocessors
-    # Remove whitespace and SQL comments from an SQL string
-    class RemoveWhitespaces
-      attr_reader :source
-
-      def initialize(source)
-        @source = source
-      end
-
-      def preprocess!
-        # Remove trailing whitespace
-        source.gsub!(/[ \t]+$/, "")
-        source.gsub!(/\A\n/, "")
-        source.gsub!(/\n\n\z/, "\n")
-
-        # Remove useless comment lines
-        source.gsub!(/^--\n/, "")
-
-        # Remove useless, version-specific parts of comments
-        source.gsub!(/^-- (.*); Schema: ([\w.]+|-); Owner: -.*/, '-- \1')
-      end
-    end
-  end
-end