active_record_doctor 1.9.0 → 1.11.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: e69330f83b83c3adcd1397554bc916ee163acf3797b8be5b8f96f1960e2d8a3e
-  data.tar.gz: 8f6e09d3af78bb9aaf01374437653b25070c2c328a0ffeaedc800378a10074f6
+  metadata.gz: fef82b1493488683e11bc72273e142479cb47234651a8aa772a83121960f9309
+  data.tar.gz: c99afc25d8307ff0814e0459790f3d7fcf0b015d9f37f118870a9a1cfa2cf460
 SHA512:
-  metadata.gz: 4b9d75665c0524cd5ad18c5aa21aaf59b3e7a75b63c88fb332cd3f600de4f2a9fa073b985f5155b81d54f177b25da5fba5dde494fa6eed28e16086a955082012
-  data.tar.gz: 94c14b73d06c92803b196fde5ca0572c9545b2ef8b89bc91e88755c5aed9c4e51087ab59a53a94f1fbcacf3f7a58a25cac38dfb4f81d7096ee4d03c7796632b4
+  metadata.gz: ab75ff192c31cfc10cbd5c4106ca0033cc4d408c1e4977a417f76c53f2a2e73e37e5706162cd4807c700808e171756ddda022af6658b60f460d78109f9bea2ee
+  data.tar.gz: 075625a08ac5f192a68ed6c99409a4acf85cadb77ec117d6933186c2bf4d07e5aaf08928e9f30d5dd96b263e8e530a4a4da848ef5a4dfc82d5dae0df2475f7d2

data/README.md

@@ -11,6 +11,7 @@ can detect:
 * missing non-`NULL` constraints - [`active_record_doctor:missing_non_null_constraint`](#detecting-missing-non-null-constraints)
 * missing presence validations - [`active_record_doctor:missing_presence_validation`](#detecting-missing-presence-validations)
 * incorrect presence validations on boolean columns - [`active_record_doctor:incorrect_boolean_presence_validation`](#detecting-incorrect-presence-validations-on-boolean-columns)
+* mismatches between model length validations and database validation constraints - [`active_record_doctor:incorrect_length_validation`](#detecting-incorrect-length-validation)
 * incorrect values of `dependent` on associations - [`active_record_doctor:incorrect_dependent_option`](#detecting-incorrect-dependent-option-on-associations)
 * primary keys having short integer types - [`active_record_doctor:short_primary_key_type`](#detecting-primary-keys-having-short-integer-types)
 * mismatched foreign key types - [`active_record_doctor:mismatched_foreign_key_type`](#detecting-mismatched-foreign-key-types)
@@ -90,7 +91,8 @@ build steps -- it returns a non-zero exit status if any errors were reported.
 
 ### Obtaining Help
 
-If you'd like to obtain help on a specific detector then use the `help` sub-task:
+If you'd like to obtain help on a specific detector then use the `help`
+sub-task:
 
 ```
 bundle exec rake active_record_doctor:extraneous_indexes:help
@@ -101,7 +103,7 @@ configuration options, their meaning, and whether they're global or local.
 
 ### Configuration
 
-`active_record_doctor` can be configured to better suite your project's needs.
+`active_record_doctor` can be configured to better suit your project's needs.
 For example, if it complains about a model that you want ignored then you can
 add that model to the configuration file.
 
@@ -173,8 +175,10 @@ three-step process:
 
 Supported configuration options:
 
+- `enabled` - set to `false` to disable the detector altogether
 - `ignore_tables` - tables whose foreign keys should not be checked
-- `ignore_columns` - columns, written as table.column, that should not be checked.
+- `ignore_columns` - columns, written as table.column, that should not be
+  checked.
 
 ### Removing Extraneous Indexes
 
@@ -215,10 +219,15 @@ reported.
 Note that a unique index can _never be replaced by a non-unique one_. For
 example, if there's a unique index on `users.login` and a non-unique index on
 `users.login, users.domain` then the tool will _not_ suggest dropping
-`users.login` as it could violate the uniqueness assumption.
+`users.login` as it could violate the uniqueness assumption. However, a unique
+index on `users.login, user.domain` might be replaceable with `users.login` as
+the uniqueness of the latter implies the uniqueness of the former (if a given
+`login` can appear only once then it can be present in only one `login, domain`
+pair).
 
 Supported configuration options:
 
+- `enabled` - set to `false` to disable the detector altogether
 - `ignore_tables` - tables whose indexes should never be reported as extraneous.
 - `ignore_columns` - indexes that should never be reported as extraneous.
 
@@ -227,7 +236,8 @@ Supported configuration options:
 If you soft-delete some models (e.g. with `paranoia`) then you need to modify
 your indexes to include only non-deleted rows. Otherwise they will include
 logically non-existent rows. This will make them larger and slower to use. Most
-of the time they should only cover columns satisfying `deleted_at IS NULL`.
+of the time they should only cover columns satisfying `deleted_at IS NULL` (to
+cover existing records) or `deleted_at IS NOT NULL` (to cover deleted records).
 
 `active_record_doctor` can automatically detect indexes on tables with a
 `deleted_at` column. Just run:
@@ -242,9 +252,12 @@ appropriate migrations. You need to do that manually.
 
 Supported configuration options:
 
+- `enabled` - set to `false` to disable the detector altogether
 - `ignore_tables` - tables whose indexes should not be checked.
-- `ignore_columns` - specific columns, written as table.column, that should not be reported as unindexed.
-- `ignore_indexes` - specific indexes that should not be reported as excluding a timestamp column.
+- `ignore_columns` - specific columns, written as table.column, that should not
+  be reported as unindexed.
+- `ignore_indexes` - specific indexes that should not be reported as excluding a
+  timestamp column.
 - `column_names` - deletion timestamp column names.
 
 ### Detecting Missing Foreign Key Constraints
@@ -277,8 +290,10 @@ end
 
 Supported configuration options:
 
+- `enabled` - set to `false` to disable the detector altogether
 - `ignore_tables` - tables whose columns should not be checked.
-- `ignore_columns` - columns, written as table.column, that should not be checked.
+- `ignore_columns` - columns, written as table.column, that should not be
+  checked.
 
 ### Detecting Models Referencing Undefined Tables
 
@@ -310,13 +325,15 @@ this check as part of your Continuous Integration pipeline.
 
 Supported configuration options:
 
-- `ignore_models` - models whose underlying tables should not be checked for existence.
+- `enabled` - set to `false` to disable the detector altogether
+- `ignore_models` - models whose underlying tables should not be checked for
+  existence.
 
 ### Detecting Uniqueness Validations not Backed by an Index
 
-A model-level uniqueness validations should be backed by a database index in
-order to be robust. Otherwise you risk inserting duplicate values under heavy
-load.
+Model-level uniqueness validations and `has_one` associations should be backed
+by a database index in order to be robust. Otherwise you risk inserting
+duplicate values under heavy load.
 
 In order to detect such validations run:
 
@@ -334,13 +351,16 @@ This means that you should create a unique index on `users.email`.
 
 Supported configuration options:
 
+- `enabled` - set to `false` to disable the detector altogether
 - `ignore_models` - models whose uniqueness validators should not be checked.
-- `ignore_columns` - specific validators, written as Model(column1, column2, ...), that should not be checked.
+- `ignore_columns` - specific validators, written as Model(column1, ...), that
+  should not be checked.
 
 ### Detecting Missing Non-`NULL` Constraints
 
 If there's an unconditional presence validation on a column then it should be
-marked as non-`NULL`-able at the database level.
+marked as non-`NULL`-able at the database level or should have a `IS NOT NULL`
+constraint.
 
 In order to detect columns whose presence is required but that are marked
 `null: true` in the database run the following command:
@@ -362,8 +382,10 @@ This validator skips models whose corresponding database tables don't exist.
 
 Supported configuration options:
 
+- `enabled` - set to `false` to disable the detector altogether
 - `ignore_tables` - tables whose columns should not be checked.
-- `ignore_columns` - columns, written as table.column, that should not be checked.
+- `ignore_columns` - columns, written as table.column, that should not be
+  checked.
 
 ### Detecting Missing Presence Validations
 
@@ -389,8 +411,10 @@ This validator skips models whose corresponding database tables don't exist.
 
 Supported configuration options:
 
+- `enabled` - set to `false` to disable the detector altogether
 - `ignore_models` - models whose underlying tables' columns should not be checked.
-- `ignore_columns` - specific attributes, written as Model.attribute, that should not be checked.
+- `ignore_attributes` - specific attributes, written as Model.attribute, that
+  should not be checked.
 
 ### Detecting Incorrect Presence Validations on Boolean Columns
 
@@ -416,8 +440,43 @@ This validator skips models whose corresponding database tables don't exist.
 
 Supported configuration options:
 
+- `enabled` - set to `false` to disable the detector altogether
 - `ignore_models` - models whose validators should not be checked.
-- `ignore_columns` - attributes, written as Model.attribute, whose validators should not be checked.
+- `ignore_columns` - attributes, written as Model.attribute, whose validators
+  should not be checked.
+
+### Detecting Incorrect Length Validations
+
+String length can be enforced by both the database and the application. If
+there's a database limit then it's a good idea to add a model validation to
+ensure user-friendly error messages. Similarly, if there's a model validator
+without the corresponding database constraint then it's a good idea to add one
+to avoid saving invalid models.
+
+In order to detect columns whose length isn't validated properly run:
+
+```
+bundle exec rake active_record_doctor:incorrect_length_validation
+```
+
+The output of the command looks like this:
+
+```
+set the maximum length in the validator of User.email (currently 32) and the database limit on users.email (currently 64) to the same value
+add a length validator on User.address to enforce a maximum length of 64 defined on users.address
+```
+
+The first message means the validator on `User.email` is checking for a
+different maximum than the database limit on `users.email`. The second message
+means there's a database limit on `users.address` without the corresponding
+model validation.
+
+Supported configuration options:
+
+- `enabled` - set to `false` to disable the detector altogether
+- `ignore_models` - models whose validators should not be checked.
+- `ignore_columns` - attributes, written as Model.attribute, whose validators
+  should not be checked.
 
 ### Detecting Incorrect `dependent` Option on Associations
 
@@ -448,8 +507,10 @@ use `dependent: :destroy` or similar on Post.comments - the associated model has
 
 Supported configuration options:
 
+- `enabled` - set to `false` to disable the detector altogether
 - `ignore_models` - models whose associations should not be checked.
-- `ignore_columns` - associations, written as Model.association, that should not be checked.
+- `ignore_associations` - associations, written as Model.association, that should not
+  be checked.
 
 ### Detecting Primary Keys Having Short Integer Types
 
@@ -485,6 +546,7 @@ as all rows need to be rewritten.
 
 Supported configuration options:
 
+- `enabled` - set to `false` to disable the detector altogether
 - `ignore_tables` - tables whose primary keys should not be checked.
 
 ### Detecting Mismatched Foreign Key Types
@@ -508,8 +570,10 @@ companies.user_id references a column of different type - foreign keys should be
 
 Supported configuration options:
 
+- `enabled` - set to `false` to disable the detector altogether
 - `ignore_tables` - tables whose foreign keys should not be checked.
-- `ignore_columns` - foreign keys, written as table.column, that should not be checked.
+- `ignore_columns` - foreign keys, written as table.column, that should not be
+  checked.
 
 ## Ruby and Rails Compatibility Policy
 

data/lib/active_record_doctor/config/default.rb

@@ -10,50 +10,67 @@ ActiveRecordDoctor.configure do
   ]
 
   detector :extraneous_indexes,
+    enabled: true,
     ignore_tables: [],
     ignore_indexes: []
 
   detector :incorrect_boolean_presence_validation,
+    enabled: true,
+    ignore_models: [],
+    ignore_attributes: []
+
+  detector :incorrect_length_validation,
+    enabled: true,
     ignore_models: [],
     ignore_attributes: []
 
   detector :incorrect_dependent_option,
+    enabled: true,
     ignore_models: [],
     ignore_associations: []
 
   detector :mismatched_foreign_key_type,
+    enabled: true,
     ignore_tables: [],
     ignore_columns: []
 
   detector :missing_foreign_keys,
+    enabled: true,
     ignore_tables: [],
     ignore_columns: []
 
   detector :missing_non_null_constraint,
+    enabled: true,
     ignore_tables: [],
     ignore_columns: []
 
   detector :missing_presence_validation,
+    enabled: true,
     ignore_models: [],
     ignore_attributes: []
 
   detector :missing_unique_indexes,
+    enabled: true,
     ignore_models: [],
     ignore_columns: []
 
   detector :short_primary_key_type,
+    enabled: true,
     ignore_tables: []
 
   detector :undefined_table_references,
+    enabled: true,
     ignore_models: []
 
   detector :unindexed_deleted_at,
+    enabled: true,
     ignore_tables: [],
     ignore_columns: [],
     ignore_indexes: [],
     column_names: ["deleted_at", "discarded_at"]
 
   detector :unindexed_foreign_keys,
+    enabled: true,
     ignore_tables: [],
     ignore_columns: []
 end

data/lib/active_record_doctor/detectors/base.rb

@@ -4,17 +4,27 @@ module ActiveRecordDoctor
   module Detectors
     # Base class for all active_record_doctor detectors.
     class Base
+      BASE_CONFIG = {
+        enabled: {
+          description: "set to false to disable the detector altogether"
+        }
+      }.freeze
+
       class << self
-        attr_reader :description, :config
+        attr_reader :description
 
-        def run(config, io)
-          new(config, io).run
+        def run(*args, **kwargs, &block)
+          new(*args, **kwargs, &block).run
         end
 
         def underscored_name
           name.demodulize.underscore.to_sym
         end
 
+        def config
+          @config.merge(BASE_CONFIG)
+        end
+
         def locals_and_globals
           locals = []
           globals = []
@@ -28,23 +38,36 @@ module ActiveRecordDoctor
         end
       end
 
-      def initialize(config, io)
+      def initialize(config:, logger:, io:)
         @problems = []
         @config = config
+        @logger = logger
         @io = io
       end
 
       def run
-        @problems = []
+        log(underscored_name) do
+          @problems = []
 
-        detect
-        @problems.each do |problem|
-          @io.puts(message(**problem))
-        end
+          if config(:enabled)
+            detect
+          else
+            log("disabled; skipping")
+          end
 
-        success = @problems.empty?
-        @problems = nil
-        success
+          @problems.each do |problem|
+            @io.puts(message(**problem))
+          end
+
+          success = @problems.empty?
+          if success
+            log("No problems found")
+          else
+            log("Found #{@problems.count} problem(s)")
+          end
+          @problems = nil
+          success
+        end
       end
 
       private
@@ -69,7 +92,16 @@ module ActiveRecordDoctor
         raise("#message should be implemented by a subclass")
       end
 
+      def log(message, &block)
+        @logger.log(message, &block)
+      end
+
       def problem!(**attrs)
+        log("Problem found") do
+          attrs.each do |key, value|
+            log("#{key}: #{value.inspect}")
+          end
+        end
         @problems << attrs
       end
 
@@ -81,74 +113,202 @@ module ActiveRecordDoctor
         @connection ||= ActiveRecord::Base.connection
       end
 
-      def indexes(table_name, except: [])
-        connection.indexes(table_name).reject do |index|
-          except.include?(index.name)
-        end
+      def indexes(table_name)
+        connection.indexes(table_name)
       end
 
-      def tables(except: [])
-        tables =
-          if ActiveRecord::VERSION::STRING >= "5.1"
-            connection.tables
-          else
-            connection.data_sources
-          end
+      def primary_key(table_name)
+        primary_key_name = connection.primary_key(table_name)
+        return nil if primary_key_name.nil?
 
-        tables.reject do |table|
-          except.include?(table)
-        end
+        column(table_name, primary_key_name)
       end
 
-      def table_exists?(table_name)
-        if ActiveRecord::VERSION::STRING >= "5.1"
-          connection.table_exists?(table_name)
-        else
-          connection.data_source_exists?(table_name)
+      def column(table_name, column_name)
+        connection.columns(table_name).find { |column| column.name == column_name }
+      end
+
+      def not_null_check_constraint_exists?(table, column)
+        check_constraints(table).any? do |definition|
+          definition =~ /\A#{column.name} IS NOT NULL\z/i ||
+            definition =~ /\A#{connection.quote_column_name(column.name)} IS NOT NULL\z/i
         end
       end
 
-      def tables_and_views
-        if connection.respond_to?(:data_sources)
-          connection.data_sources
+      def check_constraints(table_name)
+        # ActiveRecord 6.1+
+        if connection.respond_to?(:supports_check_constraints?) && connection.supports_check_constraints?
+          connection.check_constraints(table_name).select(&:validated?).map(&:expression)
+        elsif postgresql?
+          definitions =
+            connection.select_values(<<-SQL)
+              SELECT pg_get_constraintdef(oid, true)
+              FROM pg_constraint
+              WHERE contype = 'c'
+                AND convalidated
+                AND conrelid = #{connection.quote(table_name)}::regclass
+            SQL
+
+          definitions.map { |definition| definition[/CHECK \((.+)\)/m, 1] }
         else
-          connection.tables
+          # We don't support this Rails/database combination yet.
+          []
         end
       end
 
-      def primary_key(table_name)
-        primary_key_name = connection.primary_key(table_name)
-        return nil if primary_key_name.nil?
+      def models
+        ActiveRecord::Base.descendants
+      end
 
-        column(table_name, primary_key_name)
+      def underscored_name
+        self.class.underscored_name
       end
 
-      def column(table_name, column_name)
-        connection.columns(table_name).find { |column| column.name == column_name }
+      def postgresql?
+        ["PostgreSQL", "PostGIS"].include?(connection.adapter_name)
       end
 
-      def views
-        @views ||=
-          if connection.respond_to?(:views)
-            connection.views
-          elsif connection.adapter_name == "PostgreSQL"
-            ActiveRecord::Base.connection.execute(<<-SQL).map { |tuple| tuple.fetch("relname") }
-              SELECT c.relname FROM pg_class c WHERE c.relkind IN ('m', 'v')
-            SQL
+      def each_model(except: [], abstract: nil, existing_tables_only: false)
+        log("Iterating over Active Record models") do
+          models.each do |model|
+            case
+            when model.name.start_with?("HABTM_")
+              log("#{model.name} - has-belongs-to-many model; skipping")
+            when except.include?(model.name)
+              log("#{model.name} - ignored via the configuration; skipping")
+            when abstract && !model.abstract_class?
+              log("#{model.name} - non-abstract model; skipping")
+            when abstract == false && model.abstract_class?
+              log("#{model.name} - abstract model; skipping")
+            when existing_tables_only && (model.table_name.nil? || !model.table_exists?)
+              log("#{model.name} - backed by a non-existent table #{model.table_name}; skipping")
+            else
+              log(model.name) do
+                yield(model)
+              end
+            end
+          end
+        end
+      end
+
+      def each_index(table_name, except: [], multicolumn_only: false)
+        indexes = connection.indexes(table_name)
+
+        message =
+          if multicolumn_only
+            "Iterating over multi-column indexes on #{table_name}"
           else
-            # We don't support this Rails/database combination yet.
-            nil
+            "Iterating over indexes on #{table_name}"
           end
+
+        log(message) do
+          indexes.each do |index|
+            case
+            when except.include?(index.name)
+              log("#{index.name} - ignored via the configuration; skipping")
+            when multicolumn_only && !index.columns.is_a?(Array)
+              log("#{index.name} - single-column index; skipping")
+            else
+              log("Index #{index.name} on #{table_name}") do
+                yield(index, indexes)
+              end
+            end
+          end
+        end
       end
 
-      def models(except: [])
-        ActiveRecord::Base.descendants.reject do |model|
-          model.name.start_with?("HABTM_") || except.include?(model.name)
+      def each_attribute(model, except: [], type: nil)
+        log("Iterating over attributes of #{model.name}") do
+          connection.columns(model.table_name).each do |column|
+            case
+            when except.include?("#{model.name}.#{column.name}")
+              log("#{model.name}.#{column.name} - ignored via the configuration; skipping")
+            when type && !Array(type).include?(column.type)
+              log("#{model.name}.#{column.name} - ignored due to the #{column.type} type; skipping")
+            else
+              log("#{model.name}.#{column.name}") do
+                yield(column)
+              end
+            end
+          end
         end
       end
 
-      def underscored_name
-        self.class.underscored_name
+      def each_column(table_name, only: nil, except: [])
+        log("Iterating over columns of #{table_name}") do
+          connection.columns(table_name).each do |column|
+            case
+            when except.include?("#{table_name}.#{column.name}")
+              log("#{column.name} - ignored via the configuration; skipping")
+            when only.nil? || only.include?(column.name)
+              log(column.name.to_s) do
+                yield(column)
+              end
+            end
+          end
+        end
+      end
+
+      def each_foreign_key(table_name)
+        log("Iterating over foreign keys on #{table_name}") do
+          connection.foreign_keys(table_name).each do |foreign_key|
+            log("#{foreign_key.name} - #{foreign_key.from_table}(#{foreign_key.options[:column]}) to #{foreign_key.to_table}(#{foreign_key.options[:primary_key]})") do # rubocop:disable Layout/LineLength
+              yield(foreign_key)
+            end
+          end
+        end
+      end
+
+      def each_table(except: [])
+        tables =
+          if ActiveRecord::VERSION::STRING >= "5.1"
+            connection.tables
+          else
+            connection.data_sources
+          end
+
+        log("Iterating over tables") do
+          tables.each do |table|
+            case
+            when except.include?(table)
+              log("#{table} - ignored via the configuration; skipping")
+            else
+              log(table) do
+                yield(table)
+              end
+            end
+          end
+        end
+      end
+
+      def each_association(model, except: [], type: [:has_many, :has_one, :belongs_to], has_scope: nil, through: nil)
+        type = Array(type)
+
+        log("Iterating over associations on #{model.name}") do
+          associations = []
+          type.each do |type1|
+            associations.concat(model.reflect_on_all_associations(type1))
+          end
+
+          associations.each do |association|
+            case
+            when except.include?("#{model.name}.#{association.name}")
+              log("#{model.name}.#{association.name} - ignored via the configuration; skipping")
+            when through && !association.is_a?(ActiveRecord::Reflection::ThroughReflection)
+              log("#{model.name}.#{association.name} - is not a through association; skipping")
+            when through == false && association.is_a?(ActiveRecord::Reflection::ThroughReflection)
+              log("#{model.name}.#{association.name} - is a through association; skipping")
+            when has_scope && association.scope.nil?
+              log("#{model.name}.#{association.name} - doesn't have a scope; skipping")
+            when has_scope == false && association.scope
+              log("#{model.name}.#{association.name} - has a scope; skipping")
+            else
+              log("#{association.macro} :#{association.name}") do
+                yield(association)
+              end
+            end
+          end
+        end
       end
     end
   end