ridgepole 1.0.1 → 3.1.5

data/README.md

@@ -5,21 +5,41 @@ Ridgepole is a tool to manage DB schema.
 It defines DB schema using [Rails DSL](http://guides.rubyonrails.org/migrations.html#types-of-schema-dumps), and updates DB schema according to DSL.
 (like Chef/Puppet)
 
-[![Gem Version](https://badge.fury.io/rb/ridgepole.svg)](http://badge.fury.io/rb/ridgepole)
-[![Build Status](https://github.com/ridgepole/ridgepole/workflows/test/badge.svg?branch=1.0)](https://github.com/ridgepole/ridgepole/actions)
-[![Coverage Status](https://coveralls.io/repos/github/ridgepole/ridgepole/badge.svg?branch=1.0)](https://coveralls.io/github/ridgepole/ridgepole?branch=1.0)
-
-**Notice**
-
-* ActiveRecord 6.1 is supported in ridgepole v0.9, but the ActiveRecord dump has been changed, so there is a difference between ActiveRecord 5.x/6.0 format.
-  * **If you use ActiveRecord 6.1, please modify Schemafile format**.
-  * cf. https://github.com/ridgepole/ridgepole/pull/323
-* `DROP TABLE` is skipped by default in v1.0 and later versions.
-  * If you want to `DROP TABLE`, please pass `--drop-table`.
-  * cf. https://github.com/ridgepole/ridgepole/pull/363
-* In Rails 7.0, the output of dumper is different from Rails 6.
-  * cf. https://github.com/rails/rails/issues/43909
-  * cf. https://github.com/rails/rails/commit/c2a6f618d22cca4d9b7be7fa7652e7aac509350c#diff-55f41513f027a3d219629f475f03c2d1105ca55c5093d691e1b3dc4710c6cc0b
+[![Gem Version](https://badge.fury.io/rb/ridgepole.svg)](https://badge.fury.io/rb/ridgepole)
+[![test](https://github.com/ridgepole/ridgepole/actions/workflows/test.yml/badge.svg)](https://github.com/ridgepole/ridgepole/actions/workflows/test.yml)
+[![codecov](https://codecov.io/gh/ridgepole/ridgepole/graph/badge.svg)](https://codecov.io/gh/ridgepole/ridgepole)
+
+> [!warning]
+> The order of columns when exporting has changed in Rails 8.1. https://github.com/rails/rails/pull/53281
+>
+> If you do not want to sort the columns, use `--disable-sort-columns` option.
+
+> [!note]
+> * ridgepole v3.1.0
+>   * Support Rails 8.1 (cf. https://github.com/ridgepole/ridgepole/pull/589)
+> * ridgepole v3.0.0
+>   * Support Rails 8.0 (cf. https://github.com/ridgepole/ridgepole/pull/504)
+> * ridgepole v2.0.0
+>   * Support Trilogy (cf. https://github.com/ridgepole/ridgepole/pull/447)
+>   * Support Rails 7.1 (cf. https://github.com/ridgepole/ridgepole/pull/441)
+>   * Drop support AcriveRecord 6.0 (cf. https://github.com/ridgepole/ridgepole/pull/440)
+> * Drop support ActiveRecord 5.x in ridgepole v1.2.0.
+> * Partitioning is no longer supported in ridgepole v1.1.0.
+> * ActiveRecord 7.x has some incompatible changes. If you get unintended differences in `datetime` columns consider changing `precision`:
+>   * Add `precision: nil` for columns that previously had no `precision` specified (cf. https://github.com/ridgepole/ridgepole/issues/381)
+>   * Remove `precision: 6` from columns that previously had `precision: 6` specified explicitly (cf. https://github.com/ridgepole/ridgepole/issues/406)
+> * For ActiveRecord 7.x series, please use AcriveRecord 7.0.2 or higher / Ridgepole 1.0.3 or higher.
+>   * cf. https://github.com/ridgepole/ridgepole/pull/380
+> * ActiveRecord 6.1 is supported in ridgepole v0.9, but the ActiveRecord dump has been changed, so there is a difference between ActiveRecord 5.x/6.0 format.
+>   * **If you use ActiveRecord 6.1, please modify Schemafile format**.
+>   * cf. https://github.com/ridgepole/ridgepole/pull/323
+> * `DROP TABLE` is skipped by default in v1.0 and later versions.
+>   * If you want to `DROP TABLE`, please pass `--drop-table`.
+>   * cf. https://github.com/ridgepole/ridgepole/pull/363
+> * In Rails 7.0, the output of dumper is different from Rails 6.
+>   * cf. https://github.com/rails/rails/issues/43909
+>   * cf. https://github.com/rails/rails/commit/c2a6f618d22cca4d9b7be7fa7652e7aac509350c#diff-55f41513f027a3d219629f475f03c2d1105ca55c5093d691e1b3dc4710c6cc0b
+> * SQLite does not support.
 
 ## Installation
 
@@ -29,7 +49,7 @@ Add this line to your application's Gemfile:
 
 And then execute:
 
-    $ bundle
+    $ bundle install
 
 Or install it yourself as:
 
@@ -71,13 +91,15 @@ Usage: ridgepole [options]
         --dump-with-default-fk-name
         --index-removed-drop-column
         --drop-table
+        --drop-table-only
         --mysql-change-table-options
         --mysql-change-table-comment
         --check-relation-type DEF_PK
         --ignore-table-comment
         --skip-column-comment-change
-        --create-table-with-index
         --allow-pk-change
+        --create-table-with-index
+        --disable-sort-columns
         --mysql-dump-auto-increment
     -r, --require LIBS
         --log-file LOG_FILE
@@ -85,6 +107,7 @@ Usage: ridgepole [options]
         --debug
         --[no-]color
     -v, --version
+    -h, --help
 ```
 
 ## Usage
@@ -166,6 +189,13 @@ create_table "user_comments", force: :cascade, renamed_from: "comments" do |t|
 end
 ```
 
+> [!note]
+> When using `renamed_from` on a table, Ridgepole will only perform the rename operation. All other changes to that table (columns, indexes, foreign keys, etc.) will not be detected during the same migration.
+>
+> If you need to rename AND modify a table, do it in two separate steps:
+> 1. First migration: Add `renamed_from` to rename the table
+> 2. Second migration: Remove `renamed_from` and apply your desired changes
+
 ## Foreign Key
 ```ruby
 create_table "parent", force: :cascade do |t|
@@ -181,6 +211,31 @@ add_index "child", ["parent_id"], name: "par_ind", using: :btree
 add_foreign_key "child", "parent", name: "child_ibfk_1"
 ```
 
+## CHECK Constraint
+
+```ruby
+create_table "products", force: :cascade do |t|
+  t.string  "name", null: false
+  t.decimal "price", precision: 10, scale: 2
+  t.integer "quantity"
+
+  t.check_constraint "price > 0", name: "price_check"
+  t.check_constraint "quantity >= 0", name: "quantity_check"
+end
+```
+
+> [!important]
+> **Matching Schemafile to Database Output**
+>
+> CHECK constraint expressions may be formatted differently by your database when exported. For example:
+> - DSL: `"(price > 0)"`
+> - Database export: `` "`price` > 0" `` (MySQL) or `"((price > 0))"` (PostgreSQL)
+>
+> To avoid unnecessary migrations where constraints are dropped and recreated:
+> Please modify your Schemafile to match exactly what the SchemaDumper outputs. This approach maintains simplicity and prevents migration issues instead of maintaining complex normalization logic in Ridgepole.
+>
+> Run `ridgepole --export` to see the exact format your database uses, then update your Schemafile accordingly.
+
 ## Ignore Column/Index/FK
 
 ```ruby
@@ -210,6 +265,19 @@ activerecord 5.0.0 and activerecord-mysql-awesome dumps a collation rather than
 
 See `mysql> show character set;` to find charset / collation pair for your system.
 
+## Generated Column (MySQL)
+
+There should be NO extra white spaces in the expression (such as after comma).
+Quotes in expression may cause the operations failure with MySQL 8.0.
+
+```ruby
+create_table "users", force: :cascade do |t|
+  t.string   "last_name"
+  t.string   "first_name"
+  t.virtual  "full_name", type: :string, as: "concat(`last_name`,' ',`first_name`)", stored: true
+end
+```
+
 ## Execute
 ```ruby
 create_table "authors", force: :cascade do |t|
@@ -292,7 +360,7 @@ end
 ```
 
 ```sh
-$ ridgepole -a -c database.yml --check-relation-type bigint # default primary key type (e.g. `<5.1`: integer, `>=5.1`: bigint for MySQL)
+$ ridgepole -a -c database.yml --check-relation-type bigint # default primary key type (e.g. bigint for MySQL)
 Apply `Schemafile`
 ...
 [WARNING] Relation column type is different.
@@ -301,19 +369,56 @@ Apply `Schemafile`
 ...
 ```
 
+## Define a partial index in PostgreSQL
+
+Partial indexes in PostgreSQL are normalized so differences are always detected.
+
+```ruby
+create_table "users", id: :serial, force: :cascade do |t|
+  t.text "email"
+  t.text "name"
+  t.index ["email"], name: "idx_users_email", unique: true, where: "email is not null"
+end
+```
+
+```sh
+% ridgepole -a -c database.yml --dry-run --verbose
+Apply `Schemafile` (dry-run)
+# Parse DSL
+# ...
+# Compare definitions
+#   users
+      :options=>
+       {:name=>"idx_users_email",
+        :unique=>true,
+-       :where=>"(email IS NOT NULL)"}}},
++       :where=>"email is not null"}}},
+  :options=>{:id=>:serial}}
+```
+
+Use a normalized WHERE clause to avoid detecting differences.
+
+```ruby
+#t.index ["email"], name: "idx_users_email", unique: true, where: "email is not null"
+t.index ["email"], name: "idx_users_email", unique: true, where: "(email IS NOT NULL)"
+```
+
+see https://github.com/ridgepole/ridgepole/issues/568
+
 ## Run tests
 
 
 ```sh
-docker-compose up -d
+docker compose up -d
 bundle install
 bundle exec appraisal install
-bundle exec appraisal activerecord-7.0 rake
-# POSTGRESQL=1 bundle exec appraisal activerecord-7.0 rake
-# MYSQL57=1 bundle exec appraisal activerecord-7.0 rake
+bundle exec appraisal activerecord-8.1 rake
+# POSTGRESQL=1 bundle exec appraisal activerecord-8.1 rake
+# MYSQL80=1 bundle exec appraisal activerecord-8.1 rake
 ```
 
-**Notice:** Ruby 2.6 or above/mysql-client/postgresql-client is required.
+> [!note]
+> mysql-client/postgresql-client is required for testing.
 
 ## Demo
 
@@ -322,6 +427,6 @@ bundle exec appraisal activerecord-7.0 rake
 
 ## Example project
 
-* https://github.com/ridgepole/ridgepole-example
-  * https://github.com/ridgepole/ridgepole-example/pull/1
-  * https://github.com/ridgepole/ridgepole-example/pull/2
+* https://github.com/winebarrel/ridgepole-example
+  * https://github.com/winebarrel/ridgepole-example/pull/1
+  * https://github.com/winebarrel/ridgepole-example/pull/2

data/bin/ridgepole

@@ -86,8 +86,7 @@ ARGV.options do |opt|
   opt.on('',   '--dry-run') { options[:dry_run] = true }
   opt.on('',   '--table-options OPTIONS') { |v| options[:table_options] = v }
   opt.on('',   '--table-hash-options OPTIONS') do |v|
-    # NOTE: Ruby2.4 doesn't support `symbolize_names: true`
-    hash = YAML.safe_load(v).deep_symbolize_keys
+    hash = YAML.safe_load(v, symbolize_names: true)
 
     case hash[:id]
     when String
@@ -137,13 +136,16 @@ ARGV.options do |opt|
   opt.on('',   '--dump-with-default-fk-name') { options[:dump_with_default_fk_name] = true }
   opt.on('',   '--index-removed-drop-column') { options[:index_removed_drop_column] = true }
   opt.on('',   '--drop-table') { options[:force_drop_table] = true }
+  opt.on('',   '--drop-table-only') { options[:drop_table_only] = true }
   opt.on('',   '--mysql-change-table-options') { options[:mysql_change_table_options] = true }
   opt.on('',   '--mysql-change-table-comment') { options[:mysql_change_table_comment] = true }
+  opt.on('',   '--pg-change-table-comment') { options[:postgresql_change_table_comment] = true }
   opt.on('',   '--check-relation-type DEF_PK') { |v| options[:check_relation_type] = v }
   opt.on('',   '--ignore-table-comment') { options[:ignore_table_comment] = true }
   opt.on('',   '--skip-column-comment-change') { options[:skip_column_comment_change] = true }
   opt.on('',   '--allow-pk-change') { options[:allow_pk_change] = true }
   opt.on('',   '--create-table-with-index') { options[:create_table_with_index] = true }
+  opt.on('',   '--disable-sort-columns') { options[:disable_sort_columns] = true }
 
   opt.on('',   '--mysql-dump-auto-increment') do
     options[:mysql_dump_auto_increment] = true
@@ -155,10 +157,14 @@ ARGV.options do |opt|
   opt.on('', '--debug') { options[:debug] = true }
   opt.on('', '--[no-]color') { |v| options[:color] = v }
 
-  opt.on('-v', '--version') do
+  opt.on_tail('-v', '--version') do
     puts opt.ver
     exit
   end
+  opt.on_tail('-h', '--help') do
+    puts opt
+    exit
+  end
 
   opt.parse!
 
@@ -181,7 +187,12 @@ begin
   client = Ridgepole::Client.new(Ridgepole::Config.load(config, env, spec_name), options) if config
 
   ActiveRecord::Base.logger = logger
-  ActiveSupport::LogSubscriber.colorize_logging = options[:color]
+
+  if ActiveRecord.gem_version < Gem::Version.new('8.2.0.alpha')
+    ActiveSupport::LogSubscriber.colorize_logging = options[:color]
+  else
+    ActiveSupport.colorize_logging = options[:color]
+  end
 
   case mode
   when :export
@@ -261,15 +272,13 @@ begin
         else
           File.open(diff_file)
         end
-      elsif Gem::Version.new(Psych::VERSION) >= Gem::Version.new('3.1.0.pre1') # Ruby 2.6
+      else
         YAML.safe_load(
           diff_file,
           permitted_classes: [],
           permitted_symbols: [],
           aliases: true
         )
-      else
-        YAML.safe_load(diff_file, [], [], true)
       end
     end
 

data/lib/ridgepole/cli/config.rb

@@ -13,15 +13,13 @@ module Ridgepole
                           parse_config_file(config)
                         elsif (expanded = File.expand_path(config)) && File.exist?(expanded)
                           parse_config_file(expanded)
-                        elsif Gem::Version.new(Psych::VERSION) >= Gem::Version.new('3.1.0.pre1') # Ruby 2.6
+                        else
                           YAML.safe_load(
                             ERB.new(config).result,
                             permitted_classes: [],
                             permitted_symbols: [],
                             aliases: true
                           )
-                        else
-                          YAML.safe_load(ERB.new(config).result, [], [], true)
                         end
 
         parsed_config = parse_database_url(config) unless parsed_config.is_a?(Hash)
@@ -39,17 +37,12 @@ module Ridgepole
 
       def parse_config_file(path)
         yaml = ERB.new(File.read(path)).result
-
-        if Gem::Version.new(Psych::VERSION) >= Gem::Version.new('3.1.0.pre1') # Ruby 2.6
-          YAML.safe_load(
-            yaml,
-            permitted_classes: [],
-            permitted_symbols: [],
-            aliases: true
-          )
-        else
-          YAML.safe_load(yaml, [], [], true)
-        end
+        YAML.safe_load(
+          yaml,
+          permitted_classes: [],
+          permitted_symbols: [],
+          aliases: true
+        )
       end
 
       def parse_database_url(config)
@@ -66,7 +59,7 @@ module Ridgepole
 
         query_hash =
           if uri.query
-            uri.query.split('&').map { |pair| pair.split('=') }.to_h
+            uri.query.split('&').to_h { |pair| pair.split('=') }
           else
             {}
           end

data/lib/ridgepole/client.rb

@@ -10,12 +10,13 @@ module Ridgepole
       # XXX: If the required processing in class method?
       @options[:index_removed_drop_column] = true if !@options.key?(:index_removed_drop_column) && (Ridgepole::DefaultsLimit.adapter == :postgresql)
 
-      Ridgepole::ExecuteExpander.expand_execute(ActiveRecord::Base.connection)
+      Ridgepole::ExecuteExpander.expand_execute(ActiveRecord::Base.connection, @options)
       @dumper = Ridgepole::Dumper.new(@options)
       @parser = Ridgepole::DSLParser.new(@options)
       @diff = Ridgepole::Diff.new(@options)
 
       require 'ridgepole/ext/abstract_mysql_adapter/dump_auto_increment' if @options[:mysql_dump_auto_increment]
+      require 'ridgepole/ext/schema_dumper/disable_sort_columns' if @options[:disable_sort_columns]
     end
 
     def dump(&block)
@@ -29,7 +30,7 @@ module Ridgepole
 
       logger.verbose_info('# Parse DSL')
       expected_definition, expected_execute = @parser.parse(dsl, opts)
-      expected_definition.each do |_table, definition|
+      expected_definition.each_value do |definition|
         merge_table_options(definition)
       end
       logger.verbose_info('# Load tables')

data/lib/ridgepole/default_limit.rb

@@ -2,16 +2,20 @@
 
 module Ridgepole
   class DefaultsLimit
+    DEFAULT_LIMIT_FOR_MYSQL = {
+      integer: 4,
+      bigint: 8,
+      float: 24,
+      string: 255,
+      text: 65_535,
+      binary: 65_535,
+    }.tap do |limits|
+      limits[:boolean] = 1 if ActiveRecord.gem_version < Gem::Version.new('8.1.0.beta1')
+    end.freeze
+
     DEFAULTS_LIMITS = {
-      mysql2: {
-        boolean: 1,
-        integer: 4,
-        bigint: 8,
-        float: 24,
-        string: 255,
-        text: 65_535,
-        binary: 65_535,
-      },
+      mysql2: DEFAULT_LIMIT_FOR_MYSQL,
+      trilogy: DEFAULT_LIMIT_FOR_MYSQL,
     }.freeze
 
     class << self