active_record_doctor 1.9.0 → 1.10.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: e69330f83b83c3adcd1397554bc916ee163acf3797b8be5b8f96f1960e2d8a3e
-  data.tar.gz: 8f6e09d3af78bb9aaf01374437653b25070c2c328a0ffeaedc800378a10074f6
+  metadata.gz: 1e1bf7642b1c7b471fbf78956825067ff070d261d4c47a182d1aaf00f10b98b7
+  data.tar.gz: 6e6fc746327d6db30c282d7964ae778b254107281d9626a251f8593d6ce85db4
 SHA512:
-  metadata.gz: 4b9d75665c0524cd5ad18c5aa21aaf59b3e7a75b63c88fb332cd3f600de4f2a9fa073b985f5155b81d54f177b25da5fba5dde494fa6eed28e16086a955082012
-  data.tar.gz: 94c14b73d06c92803b196fde5ca0572c9545b2ef8b89bc91e88755c5aed9c4e51087ab59a53a94f1fbcacf3f7a58a25cac38dfb4f81d7096ee4d03c7796632b4
+  metadata.gz: a0a2d71947728b9d6a130901d94d1ed8c2ede9dde209e91897a8beb8b757da2fea854b89a3a3f6223d658c203e75e687edbee2af37b2203ddb24b28e831439e8
+  data.tar.gz: f0fe87dd6acf379ef946bc513504342c3977bf5c4d7f71eca79290c693b318f0a838c31b95111f68635579cdfa26249d5a95fd456817b6a3c32023c1b4fe0fd1

data/README.md

@@ -11,6 +11,7 @@ can detect:
 * missing non-`NULL` constraints - [`active_record_doctor:missing_non_null_constraint`](#detecting-missing-non-null-constraints)
 * missing presence validations - [`active_record_doctor:missing_presence_validation`](#detecting-missing-presence-validations)
 * incorrect presence validations on boolean columns - [`active_record_doctor:incorrect_boolean_presence_validation`](#detecting-incorrect-presence-validations-on-boolean-columns)
+* mismatches between model length validations and database validation constraints - [`active_record_doctor:incorrect_length_validation`](#detecting-incorrect-length-validation)
 * incorrect values of `dependent` on associations - [`active_record_doctor:incorrect_dependent_option`](#detecting-incorrect-dependent-option-on-associations)
 * primary keys having short integer types - [`active_record_doctor:short_primary_key_type`](#detecting-primary-keys-having-short-integer-types)
 * mismatched foreign key types - [`active_record_doctor:mismatched_foreign_key_type`](#detecting-mismatched-foreign-key-types)
@@ -90,7 +91,8 @@ build steps -- it returns a non-zero exit status if any errors were reported.
 
 ### Obtaining Help
 
-If you'd like to obtain help on a specific detector then use the `help` sub-task:
+If you'd like to obtain help on a specific detector then use the `help`
+sub-task:
 
 ```
 bundle exec rake active_record_doctor:extraneous_indexes:help
@@ -101,7 +103,7 @@ configuration options, their meaning, and whether they're global or local.
 
 ### Configuration
 
-`active_record_doctor` can be configured to better suite your project's needs.
+`active_record_doctor` can be configured to better suit your project's needs.
 For example, if it complains about a model that you want ignored then you can
 add that model to the configuration file.
 
@@ -173,8 +175,10 @@ three-step process:
 
 Supported configuration options:
 
+- `enabled` - set to `false` to disable the detector altogether
 - `ignore_tables` - tables whose foreign keys should not be checked
-- `ignore_columns` - columns, written as table.column, that should not be checked.
+- `ignore_columns` - columns, written as table.column, that should not be
+  checked.
 
 ### Removing Extraneous Indexes
 
@@ -215,10 +219,15 @@ reported.
 Note that a unique index can _never be replaced by a non-unique one_. For
 example, if there's a unique index on `users.login` and a non-unique index on
 `users.login, users.domain` then the tool will _not_ suggest dropping
-`users.login` as it could violate the uniqueness assumption.
+`users.login` as it could violate the uniqueness assumption. However, a unique
+index on `users.login, user.domain` might be replaceable with `users.login` as
+the uniqueness of the latter implies the uniqueness of the former (if a given
+`login` can appear only once then it can be present in only one `login, domain`
+pair).
 
 Supported configuration options:
 
+- `enabled` - set to `false` to disable the detector altogether
 - `ignore_tables` - tables whose indexes should never be reported as extraneous.
 - `ignore_columns` - indexes that should never be reported as extraneous.
 
@@ -227,7 +236,8 @@ Supported configuration options:
 If you soft-delete some models (e.g. with `paranoia`) then you need to modify
 your indexes to include only non-deleted rows. Otherwise they will include
 logically non-existent rows. This will make them larger and slower to use. Most
-of the time they should only cover columns satisfying `deleted_at IS NULL`.
+of the time they should only cover columns satisfying `deleted_at IS NULL` (to
+cover existing records) or `deleted_at IS NOT NULL` (to cover deleted records).
 
 `active_record_doctor` can automatically detect indexes on tables with a
 `deleted_at` column. Just run:
@@ -242,9 +252,12 @@ appropriate migrations. You need to do that manually.
 
 Supported configuration options:
 
+- `enabled` - set to `false` to disable the detector altogether
 - `ignore_tables` - tables whose indexes should not be checked.
-- `ignore_columns` - specific columns, written as table.column, that should not be reported as unindexed.
-- `ignore_indexes` - specific indexes that should not be reported as excluding a timestamp column.
+- `ignore_columns` - specific columns, written as table.column, that should not
+  be reported as unindexed.
+- `ignore_indexes` - specific indexes that should not be reported as excluding a
+  timestamp column.
 - `column_names` - deletion timestamp column names.
 
 ### Detecting Missing Foreign Key Constraints
@@ -277,8 +290,10 @@ end
 
 Supported configuration options:
 
+- `enabled` - set to `false` to disable the detector altogether
 - `ignore_tables` - tables whose columns should not be checked.
-- `ignore_columns` - columns, written as table.column, that should not be checked.
+- `ignore_columns` - columns, written as table.column, that should not be
+  checked.
 
 ### Detecting Models Referencing Undefined Tables
 
@@ -310,13 +325,15 @@ this check as part of your Continuous Integration pipeline.
 
 Supported configuration options:
 
-- `ignore_models` - models whose underlying tables should not be checked for existence.
+- `enabled` - set to `false` to disable the detector altogether
+- `ignore_models` - models whose underlying tables should not be checked for
+  existence.
 
 ### Detecting Uniqueness Validations not Backed by an Index
 
-A model-level uniqueness validations should be backed by a database index in
-order to be robust. Otherwise you risk inserting duplicate values under heavy
-load.
+Model-level uniqueness validations and `has_one` associations should be backed
+by a database index in order to be robust. Otherwise you risk inserting
+duplicate values under heavy load.
 
 In order to detect such validations run:
 
@@ -334,13 +351,16 @@ This means that you should create a unique index on `users.email`.
 
 Supported configuration options:
 
+- `enabled` - set to `false` to disable the detector altogether
 - `ignore_models` - models whose uniqueness validators should not be checked.
-- `ignore_columns` - specific validators, written as Model(column1, column2, ...), that should not be checked.
+- `ignore_columns` - specific validators, written as Model(column1, ...), that
+  should not be checked.
 
 ### Detecting Missing Non-`NULL` Constraints
 
 If there's an unconditional presence validation on a column then it should be
-marked as non-`NULL`-able at the database level.
+marked as non-`NULL`-able at the database level or should have a `IS NOT NULL`
+constraint.
 
 In order to detect columns whose presence is required but that are marked
 `null: true` in the database run the following command:
@@ -362,8 +382,10 @@ This validator skips models whose corresponding database tables don't exist.
 
 Supported configuration options:
 
+- `enabled` - set to `false` to disable the detector altogether
 - `ignore_tables` - tables whose columns should not be checked.
-- `ignore_columns` - columns, written as table.column, that should not be checked.
+- `ignore_columns` - columns, written as table.column, that should not be
+  checked.
 
 ### Detecting Missing Presence Validations
 
@@ -389,8 +411,10 @@ This validator skips models whose corresponding database tables don't exist.
 
 Supported configuration options:
 
+- `enabled` - set to `false` to disable the detector altogether
 - `ignore_models` - models whose underlying tables' columns should not be checked.
-- `ignore_columns` - specific attributes, written as Model.attribute, that should not be checked.
+- `ignore_attributes` - specific attributes, written as Model.attribute, that
+  should not be checked.
 
 ### Detecting Incorrect Presence Validations on Boolean Columns
 
@@ -416,8 +440,43 @@ This validator skips models whose corresponding database tables don't exist.
 
 Supported configuration options:
 
+- `enabled` - set to `false` to disable the detector altogether
 - `ignore_models` - models whose validators should not be checked.
-- `ignore_columns` - attributes, written as Model.attribute, whose validators should not be checked.
+- `ignore_columns` - attributes, written as Model.attribute, whose validators
+  should not be checked.
+
+### Detecting Incorrect Length Validations
+
+String length can be enforced by both the database and the application. If
+there's a database limit then it's a good idea to add a model validation to
+ensure user-friendly error messages. Similarly, if there's a model validator
+without the corresponding database constraint then it's a good idea to add one
+to avoid saving invalid models.
+
+In order to detect columns whose length isn't validated properly run:
+
+```
+bundle exec rake active_record_doctor:incorrect_length_validation
+```
+
+The output of the command looks like this:
+
+```
+set the maximum length in the validator of User.email (currently 32) and the database limit on users.email (currently 64) to the same value
+add a length validator on User.address to enforce a maximum length of 64 defined on users.address
+```
+
+The first message means the validator on `User.email` is checking for a
+different maximum than the database limit on `users.email`. The second message
+means there's a database limit on `users.address` without the corresponding
+model validation.
+
+Supported configuration options:
+
+- `enabled` - set to `false` to disable the detector altogether
+- `ignore_models` - models whose validators should not be checked.
+- `ignore_columns` - attributes, written as Model.attribute, whose validators
+  should not be checked.
 
 ### Detecting Incorrect `dependent` Option on Associations
 
@@ -448,8 +507,10 @@ use `dependent: :destroy` or similar on Post.comments - the associated model has
 
 Supported configuration options:
 
+- `enabled` - set to `false` to disable the detector altogether
 - `ignore_models` - models whose associations should not be checked.
-- `ignore_columns` - associations, written as Model.association, that should not be checked.
+- `ignore_columns` - associations, written as Model.association, that should not
+  be checked.
 
 ### Detecting Primary Keys Having Short Integer Types
 
@@ -485,6 +546,7 @@ as all rows need to be rewritten.
 
 Supported configuration options:
 
+- `enabled` - set to `false` to disable the detector altogether
 - `ignore_tables` - tables whose primary keys should not be checked.
 
 ### Detecting Mismatched Foreign Key Types
@@ -508,8 +570,10 @@ companies.user_id references a column of different type - foreign keys should be
 
 Supported configuration options:
 
+- `enabled` - set to `false` to disable the detector altogether
 - `ignore_tables` - tables whose foreign keys should not be checked.
-- `ignore_columns` - foreign keys, written as table.column, that should not be checked.
+- `ignore_columns` - foreign keys, written as table.column, that should not be
+  checked.
 
 ## Ruby and Rails Compatibility Policy
 

data/lib/active_record_doctor/config/default.rb

@@ -10,50 +10,67 @@ ActiveRecordDoctor.configure do
   ]
 
   detector :extraneous_indexes,
+    enabled: true,
     ignore_tables: [],
     ignore_indexes: []
 
   detector :incorrect_boolean_presence_validation,
+    enabled: true,
+    ignore_models: [],
+    ignore_attributes: []
+
+  detector :incorrect_length_validation,
+    enabled: true,
     ignore_models: [],
     ignore_attributes: []
 
   detector :incorrect_dependent_option,
+    enabled: true,
     ignore_models: [],
     ignore_associations: []
 
   detector :mismatched_foreign_key_type,
+    enabled: true,
     ignore_tables: [],
     ignore_columns: []
 
   detector :missing_foreign_keys,
+    enabled: true,
     ignore_tables: [],
     ignore_columns: []
 
   detector :missing_non_null_constraint,
+    enabled: true,
     ignore_tables: [],
     ignore_columns: []
 
   detector :missing_presence_validation,
+    enabled: true,
     ignore_models: [],
     ignore_attributes: []
 
   detector :missing_unique_indexes,
+    enabled: true,
     ignore_models: [],
     ignore_columns: []
 
   detector :short_primary_key_type,
+    enabled: true,
     ignore_tables: []
 
   detector :undefined_table_references,
+    enabled: true,
     ignore_models: []
 
   detector :unindexed_deleted_at,
+    enabled: true,
     ignore_tables: [],
     ignore_columns: [],
     ignore_indexes: [],
     column_names: ["deleted_at", "discarded_at"]
 
   detector :unindexed_foreign_keys,
+    enabled: true,
     ignore_tables: [],
     ignore_columns: []
 end

data/lib/active_record_doctor/detectors/base.rb

@@ -4,8 +4,14 @@ module ActiveRecordDoctor
   module Detectors
     # Base class for all active_record_doctor detectors.
     class Base
+      BASE_CONFIG = {
+        enabled: {
+          description: "set to false to disable the detector altogether"
+        }
+      }.freeze
+
       class << self
-        attr_reader :description, :config
+        attr_reader :description
 
         def run(config, io)
           new(config, io).run
@@ -15,6 +21,10 @@ module ActiveRecordDoctor
           name.demodulize.underscore.to_sym
         end
 
+        def config
+          @config.merge(BASE_CONFIG)
+        end
+
         def locals_and_globals
           locals = []
           globals = []
@@ -37,7 +47,7 @@ module ActiveRecordDoctor
       def run
         @problems = []
 
-        detect
+        detect if config(:enabled)
         @problems.each do |problem|
           @io.puts(message(**problem))
         end
@@ -100,22 +110,6 @@ module ActiveRecordDoctor
         end
       end
 
-      def table_exists?(table_name)
-        if ActiveRecord::VERSION::STRING >= "5.1"
-          connection.table_exists?(table_name)
-        else
-          connection.data_source_exists?(table_name)
-        end
-      end
-
-      def tables_and_views
-        if connection.respond_to?(:data_sources)
-          connection.data_sources
-        else
-          connection.tables
-        end
-      end
-
       def primary_key(table_name)
         primary_key_name = connection.primary_key(table_name)
         return nil if primary_key_name.nil?
@@ -131,16 +125,48 @@ module ActiveRecordDoctor
         @views ||=
           if connection.respond_to?(:views)
             connection.views
-          elsif connection.adapter_name == "PostgreSQL"
-            ActiveRecord::Base.connection.execute(<<-SQL).map { |tuple| tuple.fetch("relname") }
-              SELECT c.relname FROM pg_class c WHERE c.relkind IN ('m', 'v')
+          elsif postgresql?
+            ActiveRecord::Base.connection.select_values(<<-SQL)
+              SELECT relname FROM pg_class WHERE relkind IN ('m', 'v')
+            SQL
+          elsif connection.adapter_name == "Mysql2"
+            ActiveRecord::Base.connection.select_values(<<-SQL)
+              SHOW FULL TABLES WHERE table_type = 'VIEW'
             SQL
           else
             # We don't support this Rails/database combination yet.
-            nil
+            []
           end
       end
 
+      def not_null_check_constraint_exists?(table, column)
+        check_constraints(table).any? do |definition|
+          definition =~ /\A#{column.name} IS NOT NULL\z/i ||
+            definition =~ /\A#{connection.quote_column_name(column.name)} IS NOT NULL\z/i
+        end
+      end
+
+      def check_constraints(table_name)
+        # ActiveRecord 6.1+
+        if connection.respond_to?(:supports_check_constraints?) && connection.supports_check_constraints?
+          connection.check_constraints(table_name).select(&:validated?).map(&:expression)
+        elsif postgresql?
+          definitions =
+            connection.select_values(<<-SQL)
+              SELECT pg_get_constraintdef(oid, true)
+              FROM pg_constraint
+              WHERE contype = 'c'
+                AND convalidated
+                AND conrelid = #{connection.quote(table_name)}::regclass
+            SQL
+
+          definitions.map { |definition| definition[/CHECK \((.+)\)/m, 1] }
+        else
+          # We don't support this Rails/database combination yet.
+          []
+        end
+      end
+
       def models(except: [])
         ActiveRecord::Base.descendants.reject do |model|
           model.name.start_with?("HABTM_") || except.include?(model.name)
@@ -150,6 +176,10 @@ module ActiveRecordDoctor
       def underscored_name
         self.class.underscored_name
       end
+
+      def postgresql?
+        ["PostgreSQL", "PostGIS"].include?(connection.adapter_name)
+      end
     end
   end
 end

data/lib/active_record_doctor/detectors/extraneous_indexes.rb

@@ -35,18 +35,20 @@ module ActiveRecordDoctor
       def subindexes_of_multi_column_indexes
         tables(except: config(:ignore_tables)).each do |table|
           indexes = indexes(table)
-          maximal_indexes = indexes.select { |index| maximal?(indexes, index) }
 
           indexes.each do |index|
-            next if maximal_indexes.include?(index)
+            next if config(:ignore_indexes).include?(index.name)
 
-            replacement_indexes = maximal_indexes.select do |maximum_index|
-              cover?(maximum_index, index)
-            end.map(&:name).sort
+            replacement_indexes = indexes.select do |other_index|
+              index != other_index && replaceable_with?(index, other_index)
+            end
 
-            next if config(:ignore_indexes).include?(index.name)
+            next if replacement_indexes.empty?
 
-            problem!(extraneous_index: index.name, replacement_indexes: replacement_indexes)
+            problem!(
+              extraneous_index: index.name,
+              replacement_indexes: replacement_indexes.map(&:name).sort
+            )
           end
         end
       end
@@ -55,33 +57,35 @@ module ActiveRecordDoctor
         tables(except: config(:ignore_tables)).each do |table|
           indexes(table).each do |index|
             next if config(:ignore_indexes).include?(index.name)
-            next if index.columns != ["id"]
+
+            primary_key = connection.primary_key(table)
+            next if index.columns != [primary_key] || index.where
 
             problem!(extraneous_index: index.name, replacement_indexes: nil)
           end
         end
       end
 
-      def maximal?(indexes, index)
-        indexes.all? do |another_index|
-          index == another_index || !cover?(another_index, index)
-        end
-      end
-
-      # Does lhs cover rhs?
-      def cover?(lhs, rhs)
-        return false unless compatible_options?(lhs, rhs)
+      def replaceable_with?(index1, index2)
+        return false if index1.type != index2.type
+        return false if index1.using != index2.using
+        return false if index1.where != index2.where
+        return false if opclasses(index1) != opclasses(index2)
 
-        case [lhs.unique, rhs.unique]
+        case [index1.unique, index2.unique]
         when [true, true]
-          lhs.columns == rhs.columns
-        when [false, true]
+          (index2.columns - index1.columns).empty?
+        when [true, false]
           false
         else
-          prefix?(rhs, lhs)
+          prefix?(index1, index2)
         end
       end
 
+      def opclasses(index)
+        index.respond_to?(:opclasses) ? index.opclasses : nil
+      end
+
       def prefix?(lhs, rhs)
         lhs.columns.count <= rhs.columns.count &&
           rhs.columns[0...lhs.columns.count] == lhs.columns
@@ -90,25 +94,6 @@ module ActiveRecordDoctor
       def indexes(table_name)
         super.select { |index| index.columns.is_a?(Array) }
       end
-
-      def compatible_options?(lhs, rhs)
-        lhs.type == rhs.type &&
-          lhs.using == rhs.using &&
-          lhs.where == rhs.where &&
-          same_opclasses?(lhs, rhs)
-      end
-
-      def same_opclasses?(lhs, rhs)
-        if ActiveRecord::VERSION::STRING >= "5.2"
-          rhs.columns.all? do |column|
-            lhs_opclass = lhs.opclasses.is_a?(Hash) ? lhs.opclasses[column] : lhs.opclasses
-            rhs_opclass = rhs.opclasses.is_a?(Hash) ? rhs.opclasses[column] : rhs.opclasses
-            lhs_opclass == rhs_opclass
-          end
-        else
-          true
-        end
-      end
     end
   end
 end

data/lib/active_record_doctor/detectors/incorrect_boolean_presence_validation.rb

@@ -26,8 +26,7 @@ module ActiveRecordDoctor
 
       def detect
         models(except: config(:ignore_models)).each do |model|
-          next if model.table_name.nil?
-          next unless table_exists?(model.table_name)
+          next unless model.table_exists?
 
           connection.columns(model.table_name).each do |column|
             next if config(:ignore_attributes).include?("#{model.name}.#{column.name}")

data/lib/active_record_doctor/detectors/incorrect_dependent_option.rb

@@ -18,22 +18,31 @@ module ActiveRecordDoctor
 
       private
 
-      def message(model:, association:, problem:)
+      def message(model:, association:, problem:, associated_models:)
+        associated_models.sort!
+
+        models_part =
+          if associated_models.length == 1
+            "model #{associated_models[0]} has"
+          else
+            "models #{associated_models.join(', ')} have"
+          end
+
         # rubocop:disable Layout/LineLength
         case problem
         when :suggest_destroy
-          "use `dependent: :destroy` or similar on #{model}.#{association} - the associated model has callbacks that are currently skipped"
+          "use `dependent: :destroy` or similar on #{model}.#{association} - the associated #{models_part} callbacks that are currently skipped"
         when :suggest_delete
-          "use `dependent: :delete` or similar on #{model}.#{association} - the associated model has no callbacks and can be deleted without loading"
+          "use `dependent: :delete` or similar on #{model}.#{association} - the associated #{models_part} no callbacks and can be deleted without loading"
         when :suggest_delete_all
-          "use `dependent: :delete_all` or similar on #{model}.#{association} - associated models have no validations and can be deleted in bulk"
+          "use `dependent: :delete_all` or similar on #{model}.#{association} - associated #{models_part} no validations and can be deleted in bulk"
         end
         # rubocop:enable Layout/LineLength
       end
 
       def detect
         models(except: config(:ignore_models)).each do |model|
-          next if model.table_name.nil?
+          next unless model.table_exists?
 
           associations = model.reflect_on_all_associations(:has_many) +
                          model.reflect_on_all_associations(:has_one) +
@@ -42,7 +51,16 @@ module ActiveRecordDoctor
           associations.each do |association|
             next if config(:ignore_associations).include?("#{model.name}.#{association.name}")
 
-            if callback_action(association) == :invoke && deletable?(association.klass)
+            associated_models =
+              if association.polymorphic?
+                models_having(as: association.name)
+              else
+                [association.klass]
+              end
+
+            deletable_models, destroyable_models = associated_models.partition { |klass| deletable?(klass) }
+
+            if callback_action(association) == :invoke && destroyable_models.empty? && deletable_models.present?
               suggestion =
                 case association.macro
                 when :has_many then :suggest_delete_all
@@ -50,14 +68,27 @@ module ActiveRecordDoctor
                 else raise("unsupported association type #{association.macro}")
                 end
 
-              problem!(model: model.name, association: association.name, problem: suggestion)
-            elsif callback_action(association) == :skip && !deletable?(association.klass)
-              problem!(model: model.name, association: association.name, problem: :suggest_destroy)
+              problem!(model: model.name, association: association.name, problem: suggestion,
+                       associated_models: deletable_models.map(&:name))
+            elsif callback_action(association) == :skip && destroyable_models.present?
+              problem!(model: model.name, association: association.name, problem: :suggest_destroy,
+                       associated_models: destroyable_models.map(&:name))
             end
           end
         end
       end
 
+      def models_having(as:)
+        models.select do |model|
+          associations = model.reflect_on_all_associations(:has_one) +
+                         model.reflect_on_all_associations(:has_many)
+
+          associations.any? do |association|
+            association.options[:as] == as
+          end
+        end
+      end
+
       def callback_action(reflection)
         case reflection.options[:dependent]
         when :delete, :delete_all then :skip