packwerk 1.0.1 → 1.1.3

data/README.md

@@ -1,9 +1,16 @@
 # Packwerk [![Build Status](https://github.com/Shopify/packwerk/workflows/CI/badge.svg)](https://github.com/Shopify/packwerk/actions?query=workflow%3ACI)
 
+## NOTE: Packwerk is considered to be feature-complete for Shopify's uses. We are currently accepting bug fixes only, and it is not being actively developed. Please fork this project if you are interested in adding new features.
+
+> "I know who you are and because of that I know what you do."  
+> This knowledge is a dependency that raises the cost of change.  
+
+-- _Sandi Metz, Practical Object-Oriented Design in Ruby_
+
 Packwerk is a Ruby gem used to enforce boundaries and modularize Rails applications.
 
 Packwerk can be used to:
-* Combine group of files into packages
+* Combine groups of files into packages
 * Define package-level constant visibility (i.e. have publicly accessible constants)
 * Enforce privacy (inbound) and dependency (outbound) boundaries between packages
 * Help existing codebases to become more modular without obstructing development

data/USAGE.md

@@ -63,14 +63,35 @@ Packwerk reads from the `packwerk.yml` configuration file in the root directory.
 |----------------------|-------------------------------------------|--------------|
 | include              | **/*.{rb,rake,erb}                        | list of patterns for folder paths to include |
 | exclude              | {bin,node_modules,script,tmp,vendor}/**/* | list of patterns for folder paths to exclude |
-| package_paths        | **/                                       | patterns to find package configuration files, see: Defining packages |
+| package_paths        | **/                                       | a single pattern or a list of patterns to find package configuration files, see: [Defining packages](#Defining-packages) |
 | load_paths           | All application autoload paths            | list of load paths |
 | custom_associations  | N/A                                       | list of custom associations, if any |
 
+### Using a custom ERB parser
+
+You can specify a custom ERB parser if needed. For example, if you're using `<%graphql>` tags from https://github.com/github/graphql-client in your ERBs, you can use a custom parser subclass to comment them out so that Packwerk can parse the rest of the file:
+
+```ruby
+class CustomParser < Packwerk::Parsers::Erb
+  def parse_buffer(buffer, file_path:)
+    preprocessed_source = buffer.source
+
+    # Comment out <%graphql ... %> tags. They won't contain any object
+    # references anyways.
+    preprocessed_source = preprocessed_source.gsub(/<%graphql/, "<%#")
+
+    preprocessed_buffer = Parser::Source::Buffer.new(file_path)
+    preprocessed_buffer.source = preprocessed_source
+    super(preprocessed_buffer, file_path: file_path)
+  end
+end
+
+Packwerk::Parsers::Factory.instance.erb_parser_class = CustomParser
+```
 
 ### Inflections
 
-Packwerk requires custom inflections to be defined in `inflections.yml` instead of the traditional `inflections.rb`. This is because Packwerk accounts for custom inflections, such as acronyms, when resolving constants. Additionally, Packwerk interprets Active Record associations as references to constants. For example, `has_many :birds` is reference to the `Birds` constant.
+Packwerk requires custom inflections to be defined in `inflections.yml` instead of the traditional `inflections.rb`. This is because Packwerk accounts for custom inflections, such as acronyms, when resolving constants. Additionally, Packwerk interprets Active Record associations as references to constants. For example, `has_many :birds` is a reference to the `Bird` constant.
 
 In order to make your custom inflections compatible with Active Support and Packwerk, you must create a `config/inflections.yml` file and point `ActiveSupport::Inflector` to that file.
 

data/dev.yml

@@ -3,7 +3,7 @@ name: packwerk
 type: ruby
 
 up:
-  - ruby: 2.6.6
+  - ruby: 3.0.0
   - bundler
 
 commands:

data/exe/packwerk

@@ -3,4 +3,4 @@
 
 require "packwerk"
 
-Packwerk::Cli.new(style: Packwerk::OutputStyles::Coloured).run(ARGV.dup)
+Packwerk::Cli.new(style: Packwerk::OutputStyles::Coloured.new).run(ARGV.dup)

data/gemfiles/Gemfile-rails-6-0

@@ -0,0 +1,22 @@
+# frozen_string_literal: true
+
+source("https://rubygems.org")
+
+gemspec path: ".."
+
+# Specify the same dependency sources as the application Gemfile
+
+gem("spring")
+gem("rails", '~> 6.0.0')
+gem("constant_resolver", require: false)
+gem("sorbet-runtime", require: false)
+gem("rubocop-performance", require: false)
+gem("rubocop-sorbet", require: false)
+gem("mocha", require: false)
+gem("rubocop-shopify", require: false)
+gem("tapioca", require: false)
+
+group :development do
+  gem("byebug", require: false)
+  gem("minitest-focus", require: false)
+end

data/lib/packwerk.rb

@@ -4,6 +4,7 @@
 require "sorbet-runtime"
 require "active_support"
 require "constant_resolver"
+require "fileutils"
 
 require "packwerk/offense"
 
@@ -14,7 +15,6 @@ require "packwerk/cli"
 require "packwerk/configuration"
 require "packwerk/const_node_inspector"
 require "packwerk/constant_discovery"
-require "packwerk/constant_name_inspector"
 require "packwerk/dependency_checker"
 require "packwerk/deprecated_references"
 require "packwerk/files_for_processing"
@@ -28,7 +28,9 @@ require "packwerk/graph"
 require "packwerk/inflector"
 require "packwerk/node_processor"
 require "packwerk/node_visitor"
-require "packwerk/output_styles"
+require "packwerk/output_style"
+require "packwerk/output_styles/plain"
+require "packwerk/output_styles/coloured"
 require "packwerk/package"
 require "packwerk/package_set"
 require "packwerk/parsers"

data/lib/packwerk/application_load_paths.rb

@@ -0,0 +1,68 @@
+# typed: strict
+# frozen_string_literal: true
+
+require "bundler"
+
+module Packwerk
+  module ApplicationLoadPaths
+    class << self
+      extend T::Sig
+
+      sig { returns(T::Array[String]) }
+      def extract_relevant_paths
+        assert_application_booted
+        all_paths = extract_application_autoload_paths
+        relevant_paths = filter_relevant_paths(all_paths)
+        assert_load_paths_present(relevant_paths)
+        relative_path_strings(relevant_paths)
+      end
+
+      sig { void }
+      def assert_application_booted
+        raise "The application needs to be booted to extract load paths" unless defined?(::Rails)
+      end
+
+      sig { returns(T::Array[String]) }
+      def extract_application_autoload_paths
+        Rails.application.railties
+          .select { |railtie| railtie.is_a?(Rails::Engine) }
+          .push(Rails.application)
+          .flat_map do |engine|
+            paths = (engine.config.autoload_paths + engine.config.eager_load_paths + engine.config.autoload_once_paths)
+            paths.map(&:to_s).uniq
+          end
+      end
+
+      sig do
+        params(all_paths: T::Array[String], bundle_path: Pathname, rails_root: Pathname)
+          .returns(T::Array[Pathname])
+      end
+      def filter_relevant_paths(all_paths, bundle_path: Bundler.bundle_path, rails_root: Rails.root)
+        bundle_path_match = bundle_path.join("**")
+        rails_root_match = rails_root.join("**")
+
+        all_paths
+          .map { |path| Pathname.new(path).expand_path }
+          .select { |path| path.fnmatch(rails_root_match.to_s) } # path needs to be in application directory
+          .reject { |path| path.fnmatch(bundle_path_match.to_s) } # reject paths from vendored gems
+      end
+
+      sig { params(paths: T::Array[Pathname], rails_root: Pathname).returns(T::Array[String]) }
+      def relative_path_strings(paths, rails_root: Rails.root)
+        paths
+          .map { |path| path.relative_path_from(rails_root).to_s }
+          .uniq
+      end
+
+      sig { params(paths: T::Array[T.untyped]).void }
+      def assert_load_paths_present(paths)
+        if paths.empty?
+          raise <<~EOS
+            We could not extract autoload paths from your Rails app. This is likely a configuration error.
+            Packwerk will not work correctly without any autoload paths.
+          EOS
+        end
+      end
+    end
+  end
+end

data/lib/packwerk/application_validator.rb

@@ -9,15 +9,15 @@ require "yaml"
 require "packwerk/package_set"
 require "packwerk/graph"
 require "packwerk/inflector"
+require "packwerk/application_load_paths"
 
 module Packwerk
   class ApplicationValidator
-    def initialize(config_file_path:, application_load_paths:, configuration:)
+    def initialize(config_file_path:, configuration:)
       @config_file_path = config_file_path
       @configuration = configuration
 
-      # Load paths should be from the application
-      @application_load_paths = application_load_paths.sort.uniq
+      @application_load_paths = ApplicationLoadPaths.extract_relevant_paths
     end
 
     Result = Struct.new(:ok?, :error_value)
@@ -32,19 +32,10 @@ module Packwerk
         check_acyclic_graph,
         check_package_manifest_paths,
         check_valid_package_dependencies,
-        check_root_package_exist,
+        check_root_package_exists,
       ]
 
-      results.reject!(&:ok?)
-
-      if results.empty?
-        Result.new(true)
-      else
-        Result.new(
-          false,
-          results.map(&:error_value).join("\n===\n")
-        )
-      end
+      merge_results(results)
     end
 
     def check_autoload_path_cache
@@ -65,51 +56,37 @@ module Packwerk
     def check_package_manifests_for_privacy
       privacy_settings = package_manifests_settings_for("enforce_privacy")
 
-      autoload_paths = @configuration.load_paths
-
       resolver = ConstantResolver.new(
         root_path: @configuration.root_path,
-        load_paths: autoload_paths
+        load_paths: @configuration.load_paths
       )
 
-      errors = []
+      results = []
 
-      privacy_settings.each do |filepath, setting|
+      privacy_settings.each do |config_file_path, setting|
         next unless setting.is_a?(Array)
+        constants = setting
 
-        setting.each do |constant|
-          # make sure the constant can be loaded
-          constant.constantize # rubocop:disable Sorbet/ConstantsFromStrings
-          context = resolver.resolve(constant)
+        assert_constants_can_be_loaded(constants)
 
-          unless context
-            errors << "#{constant}, listed in #{filepath.inspect}, could not be resolved"
-            next
-          end
-
-          expected_filename = constant.underscore + ".rb"
+        constant_locations = constants.map { |c| [c, resolver.resolve(c)&.location] }
 
-          # We don't support all custom inflections yet, so we may accidentally resolve constants to the
-          # file that defines their parent namespace. This restriction makes sure that we don't.
-          next if context.location.end_with?(expected_filename)
-
-          errors << "Explicitly private constants need to have their own files.\n"\
-            "#{constant}, listed in #{filepath.inspect}, was resolved to #{context.location.inspect}.\n"\
-            "It should be in something like #{expected_filename.inspect}"
+        constant_locations.each do |name, location|
+          results << if location
+            check_private_constant_location(name, location, config_file_path)
+          else
+            private_constant_unresolvable(name, config_file_path)
+          end
         end
       end
 
-      if errors.empty?
-        Result.new(true)
-      else
-        Result.new(false, errors.join("\n---\n"))
-      end
+      merge_results(results, separator: "\n---\n")
     end
 
     def check_package_manifest_syntax
       errors = []
 
-      package_manifests(package_glob).each do |f|
+      package_manifests.each do |f|
         hash = YAML.load_file(f)
         next unless hash
 
@@ -124,26 +101,26 @@ module Packwerk
 
         if hash.key?("enforce_privacy")
           unless [TrueClass, FalseClass, Array].include?(hash["enforce_privacy"].class)
-            errors << "Invalid 'enforce_privacy' option in #{f.inspect}: #{hash['enforce_privacy'].inspect}"
+            errors << "Invalid 'enforce_privacy' option in #{f.inspect}: #{hash["enforce_privacy"].inspect}"
           end
         end
 
         if hash.key?("enforce_dependencies")
           unless [TrueClass, FalseClass].include?(hash["enforce_dependencies"].class)
-            errors << "Invalid 'enforce_dependencies' option in #{f.inspect}: #{hash['enforce_dependencies'].inspect}"
+            errors << "Invalid 'enforce_dependencies' option in #{f.inspect}: #{hash["enforce_dependencies"].inspect}"
           end
         end
 
         if hash.key?("public_path")
           unless hash["public_path"].is_a?(String)
-            errors << "'public_path' option must be a string in #{f.inspect}: #{hash['public_path'].inspect}"
+            errors << "'public_path' option must be a string in #{f.inspect}: #{hash["public_path"].inspect}"
           end
         end
 
         next unless hash.key?("dependencies")
         next if hash["dependencies"].is_a?(Array)
 
-        errors << "Invalid 'dependencies' option in #{f.inspect}: #{hash['dependencies'].inspect}"
+        errors << "Invalid 'dependencies' option in #{f.inspect}: #{hash["dependencies"].inspect}"
       end
 
       if errors.empty?
@@ -170,14 +147,12 @@ module Packwerk
     def check_inflection_file
       inflections_file = @configuration.inflections_file
 
-      test_inflections = ActiveSupport::Inflector::Inflections.new
-
-      Packwerk::Inflections::Default.apply_to(test_inflections)
-      Packwerk::Inflections::Custom.new(inflections_file).apply_to(test_inflections)
+      application_inflections = ActiveSupport::Inflector.inflections
+      packwerk_inflections = Packwerk::Inflector.from_file(inflections_file).inflections
 
       results = %i(plurals singulars uncountables humans acronyms).map do |type|
-        expected = ActiveSupport::Inflector.inflections.public_send(type).to_set
-        actual = test_inflections.public_send(type).to_set
+        expected = application_inflections.public_send(type).to_set
+        actual = packwerk_inflections.public_send(type).to_set
 
         if expected == actual
           Result.new(true)
@@ -195,24 +170,16 @@ module Packwerk
         end
       end
 
-      errors = results.reject(&:ok?)
-
-      if errors.empty?
-        Result.new(true)
-      else
-        Result.new(
-          false,
-          "Inflections specified in #{inflections_file} don't line up with application!\n" +
-            errors.map(&:error_value).join("\n")
-        )
-      end
+      merge_results(
+        results,
+        separator: "\n",
+        errors_headline: "Inflections specified in #{inflections_file} don't line up with application!\n"
+      )
     end
 
     def check_acyclic_graph
-      packages = Packwerk::PackageSet.load_all_from(".")
-
-      edges = packages.flat_map do |package|
-        package.dependencies.map { |dependency| [package, packages.fetch(dependency)] }
+      edges = package_set.flat_map do |package|
+        package.dependencies.map { |dependency| [package, package_set.fetch(dependency)] }
       end
       dependency_graph = Packwerk::Graph.new(*edges)
 
@@ -227,7 +194,7 @@ module Packwerk
       cycle_strings = dependency_graph.cycles.map do |cycle|
         cycle_strings = cycle.map(&:to_s)
         cycle_strings << cycle.first.to_s
-        "\t- #{cycle_strings.join(' → ')}"
+        "\t- #{cycle_strings.join(" → ")}"
       end
 
       if dependency_graph.acyclic?
@@ -299,7 +266,7 @@ module Packwerk
       end
     end
 
-    def check_root_package_exist
+    def check_root_package_exists
       root_package_path = File.join(@configuration.root_path, "package.yml")
       all_packages_manifests = package_manifests(package_glob)
 
@@ -318,8 +285,7 @@ module Packwerk
     private
 
     def package_manifests_settings_for(setting)
-      package_manifests(package_glob)
-        .map { |f| [f, (YAML.load_file(File.join(f)) || {})[setting]] }
+      package_manifests.map { |f| [f, (YAML.load_file(File.join(f)) || {})[setting]] }
     end
 
     def format_yaml_strings(list)
@@ -330,12 +296,17 @@ module Packwerk
       @configuration.package_paths || "**"
     end
 
-    def package_manifests(glob_pattern)
-      PackageSet.package_paths(@configuration.root_path, glob_pattern).map(&:to_s)
+    def package_manifests(glob_pattern = package_glob)
+      PackageSet.package_paths(@configuration.root_path, glob_pattern)
+        .map { |f| File.realpath(f) }
     end
 
     def relative_paths(paths)
-      paths.map { |path| Pathname.new(path).relative_path_from(@configuration.root_path) }
+      paths.map { |path| relative_path(path) }
+    end
+
+    def relative_path(path)
+      Pathname.new(path).relative_path_from(@configuration.root_path)
     end
 
     def invalid_package_path?(path)
@@ -345,5 +316,54 @@ module Packwerk
       package_path = File.join(@configuration.root_path, path, Packwerk::PackageSet::PACKAGE_CONFIG_FILENAME)
       !File.file?(package_path)
     end
+
+    def assert_constants_can_be_loaded(constants)
+      constants.each(&:constantize)
+      nil
+    end
+
+    def private_constant_unresolvable(name, config_file_path)
+      explicit_filepath = (name.start_with?("::") ? name[2..-1] : name).underscore + ".rb"
+
+      Result.new(
+        false,
+        "'#{name}', listed in #{config_file_path}, could not be resolved.\n"\
+        "This is probably because it is an autovivified namespace - a namespace module that doesn't have a\n"\
+        "file explicitly defining it. Packwerk currently doesn't support declaring autovivified namespaces as\n"\
+        "private. Add a #{explicit_filepath} file to explicitly define the constant."
+      )
+    end
+
+    def check_private_constant_location(name, location, config_file_path)
+      declared_package = package_set.package_from_path(relative_path(config_file_path))
+      constant_package = package_set.package_from_path(location)
+
+      if constant_package == declared_package
+        Result.new(true)
+      else
+        Result.new(
+          false,
+          "'#{name}' is declared as private in the '#{declared_package}' package but appears to be "\
+          "defined\nin the '#{constant_package}' package. Packwerk resolved it to #{location}."
+        )
+      end
+    end
+
+    def package_set
+      @package_set ||= Packwerk::PackageSet.load_all_from(@configuration.root_path, package_pathspec: package_glob)
+    end
+
+    def merge_results(results, separator: "\n===\n", errors_headline: "")
+      results.reject!(&:ok?)
+
+      if results.empty?
+        Result.new(true)
+      else
+        Result.new(
+          false,
+          errors_headline + results.map(&:error_value).join(separator)
+        )
+      end
+    end
   end
 end