zeitwerk 2.5.2 → 2.6.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: edfc3d6e8c62e95f5a008e6625b5b7f2ff7e0067e3d311ec352e1bf6c6922c27
-  data.tar.gz: 9087c6cc6524a8fd2b6843c283496e29cc5d9157421f6ba7994feb362f383333
+  metadata.gz: d7c15f85768f681d78dd0a96b23e5bb92fe8955fe74d9ae0cefeb1b0a55b76f1
+  data.tar.gz: 212a4b236b7d67c9413bd583bef540c0c9c090f5d0457af38b0681405bba3244
 SHA512:
-  metadata.gz: 4d31bf02d352d61024f553f21b14093bc4d8cfc9276c106e8a67b499044aaf12bdd076288c89828b342a1980f163706b62c82d9bd21c15cdc4cadacf117e5299
-  data.tar.gz: 89cc85a2efeb95760de195fc2acf854256dba89cdee2208992c33ba5d8cb78ad71adaddc757026660dbc4a2d562e1efa623572b91d2107e9cf30020ea83524ab
+  metadata.gz: 4feeb02e4ec695895035a39d1a46d45bd8eb77a1d72e732c28387ff16f4c3e2ac10735195fa246179cfce2fe072ada05c0f661b599fd4bc44763d300e485b12d
+  data.tar.gz: 0bd284cdfd10b1f27c7ad31ecd79c7bce9d76be46d3644941259ac7ee22ed425baac0cbcc340653547c873aecca6d5833c908e3009f58c4f0376be5402c9645b

data/README.md

@@ -47,6 +47,7 @@
   - [Edge cases](#edge-cases)
   - [Beware of circular dependencies](#beware-of-circular-dependencies)
   - [Reopening third-party namespaces](#reopening-third-party-namespaces)
+  - [Encodings](#encodings)
   - [Rules of thumb](#rules-of-thumb)
   - [Debuggers](#debuggers)
     - [debug.rb](#debugrb)
@@ -58,6 +59,7 @@
 - [Motivation](#motivation)
   - [Kernel#require is brittle](#kernelrequire-is-brittle)
   - [Rails autoloading was brittle](#rails-autoloading-was-brittle)
+- [Awards](#awards)
 - [Thanks](#thanks)
 - [License](#license)
 
@@ -237,13 +239,17 @@ should define `Geolocatable`, not `Concerns::Geolocatable`.
 <a id="markdown-implicit-namespaces" name="implicit-namespaces"></a>
 ### Implicit namespaces
 
-Directories without a matching Ruby file get modules autovivified automatically by Zeitwerk. For example, in
+If a namespace is just a simple module with no code, you do not need to define it in a file: Directories without a matching Ruby file get modules created automatically on your behalf.
+
+For example, if a project has an `admin` directory:
 
 ```
 app/controllers/admin/users_controller.rb -> Admin::UsersController
 ```
 
-`Admin` is autovivified as a module on demand, you do not need to define an `Admin` class or module in an `admin.rb` file explicitly.
+and does not have a file called `admin.rb`, Zeitwerk automatically creates an `Admin` module on your behalf the first time `Admin` is used.
+
+For this to happen, the directory has to contain non-ignored Ruby files, directly or recursively, otherwise it is ignored. This condition is evaluated again on reloads.
 
 <a id="markdown-explicit-namespaces" name="explicit-namespaces"></a>
 ### Explicit namespaces
@@ -349,8 +355,6 @@ lib/my_gem/foo.rb     # MyGem::Foo
 
 Neither a gemspec nor a version file are technically required, this helper works as long as the code is organized using that standard structure.
 
-If the entry point of your gem lives in a subdirectory of `lib` because it is reopening a namespace defined somewhere else, please use the generic API to setup the loader, and make sure you check the section [_Reopening third-party namespaces_](https://github.com/fxn/zeitwerk#reopening-third-party-namespaces) down below.
-
 Conceptually, `for_gem` translates to:
 
 ```ruby
@@ -363,8 +367,6 @@ loader.inflector = Zeitwerk::GemInflector.new(__FILE__)
 loader.push_dir(__dir__)
 ```
 
-except that this method returns the same object in subsequent calls from the same file, in the unlikely case the gem wants to be able to reload.
-
 If the main module references project constants at the top-level, Zeitwerk has to be ready to load them. Their definitions, in turn, may reference other project constants. And this is recursive. Therefore, it is important that the `setup` call happens above the main module definition:
 
 ```ruby
@@ -381,6 +383,26 @@ module MyGem
 end
 ```
 
+Loaders returned by `Zeitwerk::Loader.for_gem` issue warnings if `lib` has extra Ruby files or directories.
+
+For example, if the gem has Rails generators under `lib/generators`, by convention that directory defines a `Generators` Ruby module. If `generators` is just a container for non-autoloadable code and templates, not acting as a project namespace, you need to setup things accordingly.
+
+If the warning is legit, just tell the loader to ignore the offending file or directory:
+
+```ruby
+loader.ignore("#{__dir__}/generators")
+```
+
+Otherwise, there's a flag to say the extra stuff is OK:
+
+```ruby
+Zeitwerk::Loader.for_gem(warn_on_extra_files: false)
+```
+
+This method is idempotent when invoked from the same file, to support gems that want to reload (unlikely).
+
+If the entry point of your gem lives in a subdirectory of `lib` because it is reopening a namespace defined somewhere else, please use the generic API to setup the loader, and make sure you check the section [_Reopening third-party namespaces_](https://github.com/fxn/zeitwerk#reopening-third-party-namespaces) down below.
+
 <a id="markdown-autoloading" name="autoloading"></a>
 ### Autoloading
 
@@ -437,7 +459,7 @@ However, that can be overridden with `force`:
 loader.eager_load(force: true) # database adapters are eager loaded
 ```
 
-Which may be handy if the project eager loads in the test suite to [ensure project layour compliance](#testing-compliance).
+Which may be handy if the project eager loads in the test suite to [ensure project layout compliance](#testing-compliance).
 
 The `force` flag does not affect ignored files and directories, those are still ignored.
 
@@ -480,9 +502,14 @@ Reloading removes the currently loaded classes and modules and resets the loader
 
 It is important to highlight that this is an instance method. Don't worry about project dependencies managed by Zeitwerk, their loaders are independent.
 
-In order for reloading to be thread-safe, you need to implement some coordination. For example, a web framework that serves each request with its own thread may have a globally accessible RW lock. When a request comes in, the framework acquires the lock for reading at the beginning, and the code in the framework that calls `loader.reload` needs to acquire the lock for writing.
+Reloading is not thread-safe:
 
-On reloading, client code has to update anything that would otherwise be storing a stale object. For example, if the routing layer of a web framework stores controller class objects or instances in internal structures, on reload it has to refresh them somehow, possibly reevaluating routes.
+* You should not reload while another thread is reloading.
+* You should not autoload while another thread is reloading.
+
+In order to reload in a thread-safe manner, frameworks need to implement some coordination. For example, a web framework that serves each request with its own thread may have a globally accessible read/write lock: When a request comes in, the framework acquires the lock for reading at the beginning, and releases it at the end. On the other hand, the code in the framework responsible for the call to `Zeitwerk::Loader#reload` needs to acquire the lock for writing.
+
+On reloading, client code has to update anything that would otherwise be storing a stale object. For example, if the routing layer of a web framework stores reloadable controller class objects or instances in internal structures, on reload it has to refresh them somehow, possibly reevaluating routes.
 
 <a id="markdown-inflection" name="inflection"></a>
 ### Inflection
@@ -962,6 +989,18 @@ loader.setup
 
 With that, when Zeitwerk scans the file system and reaches the gem directories `lib/active_job` and `lib/active_job/queue_adapters`, it detects the corresponding modules already exist and therefore understands it does not have to manage them. The loader just descends into those directories. Eventually will reach `lib/active_job/queue_adapters/awesome_queue.rb`, and since `ActiveJob::QueueAdapters::AwesomeQueue` is unknown, Zeitwerk will manage it. Which is what happens regularly with the files in your gem. On reload, the namespaces are safe, won't be reloaded. The loader only reloads what it manages, which in this case is the adapter itself.
 
+<a id="markdown-encodings" name="encodings"></a>
+### Encodings
+
+Zeitwerk supports projects whose files and file system are in UTF-8. The encoding of the file system can be checked this way:
+
+```
+% ruby -e "puts Encoding.find('filesystem')"
+UTF-8
+```
+
+The test suite passes on Windows with codepage `Windows-1252` if all the involved absolute paths are ASCII. Other supersets of ASCII may work too, but you have to try.
+
 <a id="markdown-rules-of-thumb" name="rules-of-thumb"></a>
 ### Rules of thumb
 
@@ -1054,6 +1093,11 @@ With Zeitwerk, you just name things following conventions and done. Things are a
 
 Autoloading in Rails was based on `const_missing` up to Rails 5. That callback lacks fundamental information like the nesting or the resolution algorithm being used. Because of that, Rails autoloading was not able to match Ruby's semantics, and that introduced a [series of issues](https://guides.rubyonrails.org/v5.2/autoloading_and_reloading_constants.html#common-gotchas). Zeitwerk is based on a different technique and fixed Rails autoloading starting with Rails 6.
 
+<a id="markdown-awards" name="awards"></a>
+## Awards
+
+Zeitwerk has been awarded an "Outstanding Performance Award" Fukuoka Ruby Award 2022.
+
 <a id="markdown-thanks" name="thanks"></a>
 ## Thanks
 

data/lib/zeitwerk/error.rb

@@ -5,6 +5,9 @@ module Zeitwerk
   end
 
   class ReloadingDisabledError < Error
+    def initialize
+      super("can't reload, please call loader.enable_reloading before setup")
+    end
   end
 
   class NameError < ::NameError

data/lib/zeitwerk/gem_loader.rb

@@ -0,0 +1,65 @@
+# frozen_string_literal: true
+
+module Zeitwerk
+  # @private
+  class GemLoader < Loader
+    # Users should not create instances directly, the public interface is
+    # `Zeitwerk::Loader.for_gem`.
+    private_class_method :new
+
+    # @private
+    # @sig (String, bool) -> Zeitwerk::GemLoader
+    def self._new(root_file, warn_on_extra_files:)
+      new(root_file, warn_on_extra_files: warn_on_extra_files)
+    end
+
+    # @sig (String, bool) -> void
+    def initialize(root_file, warn_on_extra_files:)
+      super()
+
+      @tag                 = File.basename(root_file, ".rb")
+      @inflector           = GemInflector.new(root_file)
+      @root_file           = File.expand_path(root_file)
+      @lib                 = File.dirname(root_file)
+      @warn_on_extra_files = warn_on_extra_files
+
+      push_dir(@lib)
+    end
+
+    # @sig () -> void
+    def setup
+      warn_on_extra_files if @warn_on_extra_files
+      super
+    end
+
+    private
+
+    # @sig () -> void
+    def warn_on_extra_files
+      expected_namespace_dir = @root_file.delete_suffix(".rb")
+
+      ls(@lib) do |basename, abspath|
+        next if abspath == @root_file
+        next if abspath == expected_namespace_dir
+
+        basename_without_ext = basename.delete_suffix(".rb")
+        cname = inflector.camelize(basename_without_ext, abspath)
+        ftype = dir?(abspath) ? "directory" : "file"
+
+        warn(<<~EOS)
+          WARNING: Zeitwerk defines the constant #{cname} after the #{ftype}
+
+              #{abspath}
+
+          To prevent that, please configure the loader to ignore it:
+
+              loader.ignore("\#{__dir__}/#{basename}")
+
+          Otherwise, there is a flag to silence this warning:
+
+              Zeitwerk::Loader.for_gem(warn_on_extra_files: false)
+        EOS
+      end
+    end
+  end
+end

data/lib/zeitwerk/kernel.rb

@@ -3,11 +3,11 @@
 module Kernel
   module_function
 
-  # We are going to decorate Kernel#require with two goals.
+  # Zeitwerk's main idea is to define autoloads for project constants, and then
+  # intercept them when triggered in this thin `Kernel#require` wrapper.
   #
-  # First, by intercepting Kernel#require calls, we are able to autovivify
-  # modules on required directories, and also do internal housekeeping when
-  # managed files are loaded.
+  # That allows us to complete the circle, invoke callbacks, autovivify modules,
+  # define autoloads for just autoloaded namespaces, update internal state, etc.
   #
   # On the other hand, if you publish a new version of a gem that is now managed
   # by Zeitwerk, client code can reference directly your classes and modules and
@@ -17,7 +17,7 @@ module Kernel
   #
   # We cannot decorate with prepend + super because Kernel has already been
   # included in Object, and changes in ancestors don't get propagated into
-  # already existing ancestor chains.
+  # already existing ancestor chains on Ruby < 3.0.
   alias_method :zeitwerk_original_require, :require
 
   # @sig (String) -> true | false
@@ -29,7 +29,6 @@ module Kernel
         required
       else
         loader.on_dir_autoloaded(path)
-        $LOADED_FEATURES << path
         true
       end
     else

data/lib/zeitwerk/loader/config.rb

@@ -131,7 +131,6 @@ module Zeitwerk::Loader::Config
 
   # Sets a tag for the loader, useful for logging.
   #
-  # @param tag [#to_s]
   # @sig (#to_s) -> void
   def tag=(tag)
     @tag = tag.to_s

data/lib/zeitwerk/loader/helpers.rb

@@ -15,18 +15,50 @@ module Zeitwerk::Loader::Helpers
 
   # @sig (String) { (String, String) -> void } -> void
   def ls(dir)
-    Dir.each_child(dir) do |basename|
+    children = Dir.children(dir)
+
+    # The order in which a directory is listed depends on the file system.
+    #
+    # Since client code may run in different platforms, it seems convenient to
+    # order directory entries. This provides consistent eager loading across
+    # platforms, for example.
+    children.sort!
+
+    children.each do |basename|
       next if hidden?(basename)
 
       abspath = File.join(dir, basename)
       next if ignored_paths.member?(abspath)
 
+      if dir?(abspath)
+        next unless has_at_least_one_ruby_file?(abspath)
+      else
+        next unless ruby?(abspath)
+      end
+
       # We freeze abspath because that saves allocations when passed later to
       # File methods. See #125.
       yield basename, abspath.freeze
     end
   end
 
+  # @sig (String) -> bool
+  def has_at_least_one_ruby_file?(dir)
+    to_visit = [dir]
+
+    while dir = to_visit.shift
+      ls(dir) do |_basename, abspath|
+        if dir?(abspath)
+          to_visit << abspath
+        else
+          return true
+        end
+      end
+    end
+
+    false
+  end
+
   # @sig (String) -> bool
   def ruby?(path)
     path.end_with?(".rb")
@@ -37,7 +69,7 @@ module Zeitwerk::Loader::Helpers
     File.directory?(path)
   end
 
-  # @sig String -> bool
+  # @sig (String) -> bool
   def hidden?(basename)
     basename.start_with?(".")
   end

data/lib/zeitwerk/loader.rb

@@ -13,6 +13,9 @@ module Zeitwerk
     include Helpers
     include Config
 
+    MUTEX = Mutex.new
+    private_constant :MUTEX
+
     # Maps absolute paths for which an autoload has been set ---and not
     # executed--- to their corresponding parent class or module and constant
     # name.
@@ -123,7 +126,13 @@ module Zeitwerk
     # @sig () -> void
     def unload
       mutex.synchronize do
-        abspaths_of_unloaded_crefs = Set.new
+        # We are going to keep track of the files that were required by our
+        # autoloads to later remove them from $LOADED_FEATURES, thus making them
+        # loadable by Kernel#require again.
+        #
+        # Directories are not stored in $LOADED_FEATURES, keeping track of files
+        # is enough.
+        unloaded_files = Set.new
 
         autoloads.each do |abspath, (parent, cname)|
           if parent.autoload?(cname)
@@ -133,25 +142,28 @@ module Zeitwerk
             # and the constant path would escape unloadable_cpath? This is just
             # defensive code to clean things up as much as we are able to.
             unload_cref(parent, cname)
-            abspaths_of_unloaded_crefs.add(abspath)
+            unloaded_files.add(abspath) if ruby?(abspath)
           end
         end
 
         to_unload.each do |cpath, (abspath, (parent, cname))|
           unless on_unload_callbacks.empty?
-            value = parent.const_get(cname)
-            run_on_unload_callbacks(cpath, value, abspath)
+            begin
+              value = cget(parent, cname)
+            rescue ::NameError
+              # Perhaps the user deleted the constant by hand, or perhaps an
+              # autoload failed to define the expected constant but the user
+              # rescued the exception.
+            else
+              run_on_unload_callbacks(cpath, value, abspath)
+            end
           end
 
           unload_cref(parent, cname)
-          abspaths_of_unloaded_crefs.add(abspath)
+          unloaded_files.add(abspath) if ruby?(abspath)
         end
 
-        unless abspaths_of_unloaded_crefs.empty?
-          # We remove these abspaths from $LOADED_FEATURES because, otherwise,
-          # `require`'s idempotence would prevent newly defined autoloads from
-          # loading them again.
-          #
+        unless unloaded_files.empty?
           # Bootsnap decorates Kernel#require to speed it up using a cache and
           # this optimization does not check if $LOADED_FEATURES has the file.
           #
@@ -163,7 +175,7 @@ module Zeitwerk
           # Rails applications may depend on bootsnap, so for unloading to work
           # in that setting it is preferable that we restrict our API choice to
           # one of those methods.
-          $LOADED_FEATURES.reject! { |file| abspaths_of_unloaded_crefs.member?(file) }
+          $LOADED_FEATURES.reject! { |file| unloaded_files.member?(file) }
         end
 
         autoloads.clear
@@ -188,14 +200,12 @@ module Zeitwerk
     # @raise [Zeitwerk::Error]
     # @sig () -> void
     def reload
-      if reloading_enabled?
-        unload
-        recompute_ignored_paths
-        recompute_collapse_dirs
-        setup
-      else
-        raise ReloadingDisabledError, "can't reload, please call loader.enable_reloading before setup"
-      end
+      raise ReloadingDisabledError unless reloading_enabled?
+
+      unload
+      recompute_ignored_paths
+      recompute_collapse_dirs
+      setup
     end
 
     # Eager loads all files in the root directories, recursively. Files do not
@@ -228,7 +238,7 @@ module Zeitwerk
               if cref = autoloads[abspath]
                 cget(*cref)
               end
-            elsif dir?(abspath) && !root_dirs.key?(abspath)
+            elsif !root_dirs.key?(abspath)
               if collapse?(abspath)
                 queue << [namespace, abspath]
               else
@@ -281,10 +291,6 @@ module Zeitwerk
       # @sig #call | #debug | nil
       attr_accessor :default_logger
 
-      # @private
-      # @sig Mutex
-      attr_accessor :mutex
-
       # This is a shortcut for
       #
       #   require "zeitwerk"
@@ -296,10 +302,13 @@ module Zeitwerk
       # except that this method returns the same object in subsequent calls from
       # the same file, in the unlikely case the gem wants to be able to reload.
       #
-      # @sig () -> Zeitwerk::Loader
-      def for_gem
+      # This method returns a subclass of Zeitwerk::Loader, but the exact type
+      # is private, client code can only rely on the interface.
+      #
+      # @sig (bool) -> Zeitwerk::GemLoader
+      def for_gem(warn_on_extra_files: true)
         called_from = caller_locations(1, 1).first.path
-        Registry.loader_for_gem(called_from)
+        Registry.loader_for_gem(called_from, warn_on_extra_files: warn_on_extra_files)
       end
 
       # Broadcasts `eager_load` to all loaders.
@@ -318,8 +327,6 @@ module Zeitwerk
       end
     end
 
-    self.mutex = Mutex.new
-
     private # -------------------------------------------------------------------------------------
 
     # @sig (String, Module) -> void
@@ -330,7 +337,7 @@ module Zeitwerk
             basename.delete_suffix!(".rb")
             cname = inflector.camelize(basename, abspath).to_sym
             autoload_file(parent, cname, abspath)
-          elsif dir?(abspath)
+          else
             # In a Rails application, `app/models/concerns` is a subdirectory of
             # `app/models`, but both of them are root directories.
             #
@@ -458,7 +465,7 @@ module Zeitwerk
 
     # @sig (String) -> void
     def raise_if_conflicting_directory(dir)
-      self.class.mutex.synchronize do
+      MUTEX.synchronize do
         Registry.loaders.each do |loader|
           next if loader == self
           next if loader.ignores?(dir)

data/lib/zeitwerk/registry.rb

@@ -10,12 +10,11 @@ module Zeitwerk
       # @sig Array[Zeitwerk::Loader]
       attr_reader :loaders
 
-      # Registers loaders created with `for_gem` to make the method idempotent
-      # in case of reload.
+      # Registers gem loaders to let `for_gem` be idempotent in case of reload.
       #
       # @private
       # @sig Hash[String, Zeitwerk::Loader]
-      attr_reader :loaders_managing_gems
+      attr_reader :gem_loaders_by_root_file
 
       # Maps absolute paths to the loaders responsible for them.
       #
@@ -77,7 +76,7 @@ module Zeitwerk
       # @sig (Zeitwerk::Loader) -> void
       def unregister_loader(loader)
         loaders.delete(loader)
-        loaders_managing_gems.delete_if { |_, l| l == loader }
+        gem_loaders_by_root_file.delete_if { |_, l| l == loader }
         autoloads.delete_if { |_, l| l == loader }
         inceptions.delete_if { |_, (_, l)| l == loader }
       end
@@ -87,14 +86,8 @@ module Zeitwerk
       #
       # @private
       # @sig (String) -> Zeitwerk::Loader
-      def loader_for_gem(root_file)
-        loaders_managing_gems[root_file] ||= begin
-          Loader.new.tap do |loader|
-            loader.tag = File.basename(root_file, ".rb")
-            loader.inflector = GemInflector.new(root_file)
-            loader.push_dir(File.dirname(root_file))
-          end
-        end
+      def loader_for_gem(root_file, warn_on_extra_files:)
+        gem_loaders_by_root_file[root_file] ||= GemLoader._new(root_file, warn_on_extra_files: warn_on_extra_files)
       end
 
       # @private
@@ -137,9 +130,9 @@ module Zeitwerk
       end
     end
 
-    @loaders               = []
-    @loaders_managing_gems = {}
-    @autoloads             = {}
-    @inceptions            = {}
+    @loaders                  = []
+    @gem_loaders_by_root_file = {}
+    @autoloads                = {}
+    @inceptions               = {}
   end
 end

data/lib/zeitwerk/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Zeitwerk
-  VERSION = "2.5.2"
+  VERSION = "2.6.0"
 end

data/lib/zeitwerk.rb

@@ -3,6 +3,7 @@
 module Zeitwerk
   require_relative "zeitwerk/real_mod_name"
   require_relative "zeitwerk/loader"
+  require_relative "zeitwerk/gem_loader"
   require_relative "zeitwerk/registry"
   require_relative "zeitwerk/explicit_namespace"
   require_relative "zeitwerk/inflector"

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: zeitwerk
 version: !ruby/object:Gem::Version
-  version: 2.5.2
+  version: 2.6.0
 platform: ruby
 authors:
 - Xavier Noria
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2021-12-27 00:00:00.000000000 Z
+date: 2022-06-13 00:00:00.000000000 Z
 dependencies: []
 description: |2
       Zeitwerk implements constant autoloading with Ruby semantics. Each gem
@@ -26,6 +26,7 @@ files:
 - lib/zeitwerk/error.rb
 - lib/zeitwerk/explicit_namespace.rb
 - lib/zeitwerk/gem_inflector.rb
+- lib/zeitwerk/gem_loader.rb
 - lib/zeitwerk/inflector.rb
 - lib/zeitwerk/kernel.rb
 - lib/zeitwerk/loader.rb
@@ -58,7 +59,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.2.22
+rubygems_version: 3.3.3
 signing_key:
 specification_version: 4
 summary: Efficient and thread-safe constant autoloader