zeitwerk 2.5.4 → 2.6.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 3640f4751b86923c0dc81be1aed0580b1e2bdef59c6d3cc06c0c736d03b78c3b
-  data.tar.gz: 262a7697cfca8fa8d9630687ba8439dd9df404c052b71e5b91fb498c4114e5db
+  metadata.gz: cf4acabb8b402560402158aced0f30ae6bae5120dc65d719f4fb15c1385a2454
+  data.tar.gz: 0a297687455cf8c3924a64c5e36223c35b0f69712c807ee9bc70748f8c42ff37
 SHA512:
-  metadata.gz: 7decde169af7a300ae8962339a8886cede3cfdd75ade4d7428c39a53f3dad96c194b7e938938811dce8208f70dfcf2cc29735c03384eaa776dbc2985fea87325
-  data.tar.gz: 99c16812c1ad437860029642af2d1c1ffc00962369aa1f6dc770b1f64d102b09f046c3ad3ae2dc44d50479314a14b8c9ddb2d446a619407ca05178aff43890de
+  metadata.gz: 1d6666a098ae430f452edcd8b5e458541f20b20c4ee5903c4d18fe19cdce5afe81d6311a1bff5f9412b18ae57274e9b13132f8d59d9189b0ccd6cf7711337d6e
+  data.tar.gz: a131561faa463e489f7e9c4898bc8e212b2f6814ba3ce634b7ef62e25467f38b88e4c63106e8d53ee933681e1f0dc196244d16f13c299d2ca0771ee4031655a1

data/README.md

@@ -47,6 +47,8 @@
   - [Edge cases](#edge-cases)
   - [Beware of circular dependencies](#beware-of-circular-dependencies)
   - [Reopening third-party namespaces](#reopening-third-party-namespaces)
+  - [Introspection](#introspection)
+  - [Encodings](#encodings)
   - [Rules of thumb](#rules-of-thumb)
   - [Debuggers](#debuggers)
     - [debug.rb](#debugrb)
@@ -58,6 +60,7 @@
 - [Motivation](#motivation)
   - [Kernel#require is brittle](#kernelrequire-is-brittle)
   - [Rails autoloading was brittle](#rails-autoloading-was-brittle)
+- [Awards](#awards)
 - [Thanks](#thanks)
 - [License](#license)
 
@@ -237,13 +240,17 @@ should define `Geolocatable`, not `Concerns::Geolocatable`.
 <a id="markdown-implicit-namespaces" name="implicit-namespaces"></a>
 ### Implicit namespaces
 
-Directories without a matching Ruby file get modules autovivified automatically by Zeitwerk. For example, in
+If a namespace is just a simple module with no code, you do not need to define it in a file: Directories without a matching Ruby file get modules created automatically on your behalf.
+
+For example, if a project has an `admin` directory:
 
 ```
 app/controllers/admin/users_controller.rb -> Admin::UsersController
 ```
 
-`Admin` is autovivified as a module on demand, you do not need to define an `Admin` class or module in an `admin.rb` file explicitly.
+and does not have a file called `admin.rb`, Zeitwerk automatically creates an `Admin` module on your behalf the first time `Admin` is used.
+
+For this to happen, the directory has to contain non-ignored Ruby files with extension `.rb`, directly or recursively, otherwise it is ignored. This condition is evaluated again on reloads.
 
 <a id="markdown-explicit-namespaces" name="explicit-namespaces"></a>
 ### Explicit namespaces
@@ -349,8 +356,6 @@ lib/my_gem/foo.rb     # MyGem::Foo
 
 Neither a gemspec nor a version file are technically required, this helper works as long as the code is organized using that standard structure.
 
-If the entry point of your gem lives in a subdirectory of `lib` because it is reopening a namespace defined somewhere else, please use the generic API to setup the loader, and make sure you check the section [_Reopening third-party namespaces_](https://github.com/fxn/zeitwerk#reopening-third-party-namespaces) down below.
-
 Conceptually, `for_gem` translates to:
 
 ```ruby
@@ -363,8 +368,6 @@ loader.inflector = Zeitwerk::GemInflector.new(__FILE__)
 loader.push_dir(__dir__)
 ```
 
-except that this method returns the same object in subsequent calls from the same file, in the unlikely case the gem wants to be able to reload.
-
 If the main module references project constants at the top-level, Zeitwerk has to be ready to load them. Their definitions, in turn, may reference other project constants. And this is recursive. Therefore, it is important that the `setup` call happens above the main module definition:
 
 ```ruby
@@ -381,6 +384,26 @@ module MyGem
 end
 ```
 
+Loaders returned by `Zeitwerk::Loader.for_gem` issue warnings if `lib` has extra Ruby files or directories.
+
+For example, if the gem has Rails generators under `lib/generators`, by convention that directory defines a `Generators` Ruby module. If `generators` is just a container for non-autoloadable code and templates, not acting as a project namespace, you need to setup things accordingly.
+
+If the warning is legit, just tell the loader to ignore the offending file or directory:
+
+```ruby
+loader.ignore("#{__dir__}/generators")
+```
+
+Otherwise, there's a flag to say the extra stuff is OK:
+
+```ruby
+Zeitwerk::Loader.for_gem(warn_on_extra_files: false)
+```
+
+This method is idempotent when invoked from the same file, to support gems that want to reload (unlikely).
+
+If the entry point of your gem lives in a subdirectory of `lib` because it is reopening a namespace defined somewhere else, please use the generic API to setup the loader, and make sure you check the section [_Reopening third-party namespaces_](https://github.com/fxn/zeitwerk#reopening-third-party-namespaces) down below.
+
 <a id="markdown-autoloading" name="autoloading"></a>
 ### Autoloading
 
@@ -480,9 +503,14 @@ Reloading removes the currently loaded classes and modules and resets the loader
 
 It is important to highlight that this is an instance method. Don't worry about project dependencies managed by Zeitwerk, their loaders are independent.
 
-In order for reloading to be thread-safe, you need to implement some coordination. For example, a web framework that serves each request with its own thread may have a globally accessible RW lock. When a request comes in, the framework acquires the lock for reading at the beginning, and the code in the framework that calls `loader.reload` needs to acquire the lock for writing.
+Reloading is not thread-safe:
+
+* You should not reload while another thread is reloading.
+* You should not autoload while another thread is reloading.
 
-On reloading, client code has to update anything that would otherwise be storing a stale object. For example, if the routing layer of a web framework stores controller class objects or instances in internal structures, on reload it has to refresh them somehow, possibly reevaluating routes.
+In order to reload in a thread-safe manner, frameworks need to implement some coordination. For example, a web framework that serves each request with its own thread may have a globally accessible read/write lock: When a request comes in, the framework acquires the lock for reading at the beginning, and releases it at the end. On the other hand, the code in the framework responsible for the call to `Zeitwerk::Loader#reload` needs to acquire the lock for writing.
+
+On reloading, client code has to update anything that would otherwise be storing a stale object. For example, if the routing layer of a web framework stores reloadable controller class objects or instances in internal structures, on reload it has to refresh them somehow, possibly reevaluating routes.
 
 <a id="markdown-inflection" name="inflection"></a>
 ### Inflection
@@ -956,12 +984,48 @@ require "active_job"
 require "active_job/queue_adapters"
 
 require "zeitwerk"
-loader = Zeitwerk::Loader.for_gem
+# By passign the flag, we acknowledge the extra directory lib/active_job
+# has to be managed by the loader and no warning has to be issued for it.
+loader = Zeitwerk::Loader.for_gem(warn_on_extra_files: false)
 loader.setup
 ```
 
 With that, when Zeitwerk scans the file system and reaches the gem directories `lib/active_job` and `lib/active_job/queue_adapters`, it detects the corresponding modules already exist and therefore understands it does not have to manage them. The loader just descends into those directories. Eventually will reach `lib/active_job/queue_adapters/awesome_queue.rb`, and since `ActiveJob::QueueAdapters::AwesomeQueue` is unknown, Zeitwerk will manage it. Which is what happens regularly with the files in your gem. On reload, the namespaces are safe, won't be reloaded. The loader only reloads what it manages, which in this case is the adapter itself.
 
+<a id="markdown-introspection" name="introspection"></a>
+### Introspection
+
+The method `Zeitwerk::Loader#dirs` returns an array with the absolute paths of the root directories as strings:
+
+```ruby
+loader = Zeitwerk::Loader.new
+loader.push_dir(Pathname.new("/foo"))
+loader.dirs # => ["/foo"]
+```
+
+This method accepts an optional `namespaces` keyword argument. If truthy, the method returns a hash table instead. Keys are the absolute paths of the root directories as strings. Values are their corresponding namespaces, class or module objects:
+
+```ruby
+loader = Zeitwerk::Loader.new
+loader.push_dir(Pathname.new("/foo"))
+loader.push_dir(Pathname.new("/bar"), namespace: Bar)
+loader.dirs(namespaces: true) # => { "/foo" => Object, "/bar" => Bar }
+```
+
+These collections are read-only. Please add to them with `Zeitwerk::Loader#push_dir`.
+
+<a id="markdown-encodings" name="encodings"></a>
+### Encodings
+
+Zeitwerk supports projects whose files and file system are in UTF-8. The encoding of the file system can be checked this way:
+
+```
+% ruby -e "puts Encoding.find('filesystem')"
+UTF-8
+```
+
+The test suite passes on Windows with codepage `Windows-1252` if all the involved absolute paths are ASCII. Other supersets of ASCII may work too, but you have to try.
+
 <a id="markdown-rules-of-thumb" name="rules-of-thumb"></a>
 ### Rules of thumb
 
@@ -1054,6 +1118,11 @@ With Zeitwerk, you just name things following conventions and done. Things are a
 
 Autoloading in Rails was based on `const_missing` up to Rails 5. That callback lacks fundamental information like the nesting or the resolution algorithm being used. Because of that, Rails autoloading was not able to match Ruby's semantics, and that introduced a [series of issues](https://guides.rubyonrails.org/v5.2/autoloading_and_reloading_constants.html#common-gotchas). Zeitwerk is based on a different technique and fixed Rails autoloading starting with Rails 6.
 
+<a id="markdown-awards" name="awards"></a>
+## Awards
+
+Zeitwerk has been awarded an "Outstanding Performance Award" Fukuoka Ruby Award 2022.
+
 <a id="markdown-thanks" name="thanks"></a>
 ## Thanks
 

data/lib/zeitwerk/error.rb

@@ -5,6 +5,9 @@ module Zeitwerk
   end
 
   class ReloadingDisabledError < Error
+    def initialize
+      super("can't reload, please call loader.enable_reloading before setup")
+    end
   end
 
   class NameError < ::NameError

data/lib/zeitwerk/gem_loader.rb

@@ -0,0 +1,65 @@
+# frozen_string_literal: true
+
+module Zeitwerk
+  # @private
+  class GemLoader < Loader
+    # Users should not create instances directly, the public interface is
+    # `Zeitwerk::Loader.for_gem`.
+    private_class_method :new
+
+    # @private
+    # @sig (String, bool) -> Zeitwerk::GemLoader
+    def self._new(root_file, warn_on_extra_files:)
+      new(root_file, warn_on_extra_files: warn_on_extra_files)
+    end
+
+    # @sig (String, bool) -> void
+    def initialize(root_file, warn_on_extra_files:)
+      super()
+
+      @tag                 = File.basename(root_file, ".rb")
+      @inflector           = GemInflector.new(root_file)
+      @root_file           = File.expand_path(root_file)
+      @lib                 = File.dirname(root_file)
+      @warn_on_extra_files = warn_on_extra_files
+
+      push_dir(@lib)
+    end
+
+    # @sig () -> void
+    def setup
+      warn_on_extra_files if @warn_on_extra_files
+      super
+    end
+
+    private
+
+    # @sig () -> void
+    def warn_on_extra_files
+      expected_namespace_dir = @root_file.delete_suffix(".rb")
+
+      ls(@lib) do |basename, abspath|
+        next if abspath == @root_file
+        next if abspath == expected_namespace_dir
+
+        basename_without_ext = basename.delete_suffix(".rb")
+        cname = inflector.camelize(basename_without_ext, abspath)
+        ftype = dir?(abspath) ? "directory" : "file"
+
+        warn(<<~EOS)
+          WARNING: Zeitwerk defines the constant #{cname} after the #{ftype}
+
+              #{abspath}
+
+          To prevent that, please configure the loader to ignore it:
+
+              loader.ignore("\#{__dir__}/#{basename}")
+
+          Otherwise, there is a flag to silence this warning:
+
+              Zeitwerk::Loader.for_gem(warn_on_extra_files: false)
+        EOS
+      end
+    end
+  end
+end

data/lib/zeitwerk/loader/config.rb

@@ -5,18 +5,19 @@ require "securerandom"
 
 module Zeitwerk::Loader::Config
   # Absolute paths of the root directories. Stored in a hash to preserve
-  # order, easily handle duplicates, and also be able to have a fast lookup,
-  # needed for detecting nested paths.
+  # order, easily handle duplicates, have a fast lookup needed for detecting
+  # nested paths, and store custom namespaces as values.
   #
-  #   "/Users/fxn/blog/app/assets"   => true,
-  #   "/Users/fxn/blog/app/channels" => true,
+  #   "/Users/fxn/blog/app/assets"   => Object,
+  #   "/Users/fxn/blog/app/channels" => Object,
+  #   "/Users/fxn/blog/adapters"     => ActiveJob::QueueAdapters,
   #   ...
   #
   # This is a private collection maintained by the loader. The public
   # interface for it is `push_dir` and `dirs`.
   #
   # @private
-  # @sig Hash[String, true]
+  # @sig Hash[String, Module]
   attr_reader :root_dirs
 
   # @sig #camelize
@@ -131,18 +132,25 @@ module Zeitwerk::Loader::Config
 
   # Sets a tag for the loader, useful for logging.
   #
-  # @param tag [#to_s]
   # @sig (#to_s) -> void
   def tag=(tag)
     @tag = tag.to_s
   end
 
-  # Absolute paths of the root directories. This is a read-only collection,
-  # please push here via `push_dir`.
+  # If `namespaces` is falsey (default), returns an array with the absolute
+  # paths of the root directories as strings. If truthy, returns a hash table
+  # instead. Keys are the absolute paths of the root directories as strings,
+  # values are their corresponding namespaces, class or module objects.
   #
-  # @sig () -> Array[String]
-  def dirs
-    root_dirs.keys.freeze
+  # These are read-only collections, please add to them with `push_dir`.
+  #
+  # @sig () -> Array[String] | Hash[String, Module]
+  def dirs(namespaces: false)
+    if namespaces
+      root_dirs.clone
+    else
+      root_dirs.keys
+    end.freeze
   end
 
   # You need to call this method before setup in order to be able to reload.

data/lib/zeitwerk/loader/helpers.rb

@@ -15,18 +15,50 @@ module Zeitwerk::Loader::Helpers
 
   # @sig (String) { (String, String) -> void } -> void
   def ls(dir)
-    Dir.each_child(dir) do |basename|
+    children = Dir.children(dir)
+
+    # The order in which a directory is listed depends on the file system.
+    #
+    # Since client code may run in different platforms, it seems convenient to
+    # order directory entries. This provides consistent eager loading across
+    # platforms, for example.
+    children.sort!
+
+    children.each do |basename|
       next if hidden?(basename)
 
       abspath = File.join(dir, basename)
       next if ignored_paths.member?(abspath)
 
+      if dir?(abspath)
+        next unless has_at_least_one_ruby_file?(abspath)
+      else
+        next unless ruby?(abspath)
+      end
+
       # We freeze abspath because that saves allocations when passed later to
       # File methods. See #125.
       yield basename, abspath.freeze
     end
   end
 
+  # @sig (String) -> bool
+  def has_at_least_one_ruby_file?(dir)
+    to_visit = [dir]
+
+    while dir = to_visit.shift
+      ls(dir) do |_basename, abspath|
+        if dir?(abspath)
+          to_visit << abspath
+        else
+          return true
+        end
+      end
+    end
+
+    false
+  end
+
   # @sig (String) -> bool
   def ruby?(path)
     path.end_with?(".rb")
@@ -37,7 +69,7 @@ module Zeitwerk::Loader::Helpers
     File.directory?(path)
   end
 
-  # @sig String -> bool
+  # @sig (String) -> bool
   def hidden?(basename)
     basename.start_with?(".")
   end

data/lib/zeitwerk/loader.rb

@@ -13,6 +13,9 @@ module Zeitwerk
     include Helpers
     include Config
 
+    MUTEX = Mutex.new
+    private_constant :MUTEX
+
     # Maps absolute paths for which an autoload has been set ---and not
     # executed--- to their corresponding parent class or module and constant
     # name.
@@ -144,15 +147,16 @@ module Zeitwerk
         end
 
         to_unload.each do |cpath, (abspath, (parent, cname))|
-          # We have to check cdef? in this condition. Reason is, constants whose
-          # file does not define them have to be kept in to_unload as explained
-          # in the implementation of on_file_autoloaded.
-          #
-          # If the constant is not defined, on_unload should not be triggered
-          # for it.
-          if !on_unload_callbacks.empty? && cdef?(parent, cname)
-            value = parent.const_get(cname)
-            run_on_unload_callbacks(cpath, value, abspath)
+          unless on_unload_callbacks.empty?
+            begin
+              value = cget(parent, cname)
+            rescue ::NameError
+              # Perhaps the user deleted the constant by hand, or perhaps an
+              # autoload failed to define the expected constant but the user
+              # rescued the exception.
+            else
+              run_on_unload_callbacks(cpath, value, abspath)
+            end
           end
 
           unload_cref(parent, cname)
@@ -196,14 +200,12 @@ module Zeitwerk
     # @raise [Zeitwerk::Error]
     # @sig () -> void
     def reload
-      if reloading_enabled?
-        unload
-        recompute_ignored_paths
-        recompute_collapse_dirs
-        setup
-      else
-        raise ReloadingDisabledError, "can't reload, please call loader.enable_reloading before setup"
-      end
+      raise ReloadingDisabledError unless reloading_enabled?
+
+      unload
+      recompute_ignored_paths
+      recompute_collapse_dirs
+      setup
     end
 
     # Eager loads all files in the root directories, recursively. Files do not
@@ -236,7 +238,7 @@ module Zeitwerk
               if cref = autoloads[abspath]
                 cget(*cref)
               end
-            elsif dir?(abspath) && !root_dirs.key?(abspath)
+            elsif !root_dirs.key?(abspath)
               if collapse?(abspath)
                 queue << [namespace, abspath]
               else
@@ -289,10 +291,6 @@ module Zeitwerk
       # @sig #call | #debug | nil
       attr_accessor :default_logger
 
-      # @private
-      # @sig Mutex
-      attr_accessor :mutex
-
       # This is a shortcut for
       #
       #   require "zeitwerk"
@@ -304,10 +302,13 @@ module Zeitwerk
       # except that this method returns the same object in subsequent calls from
       # the same file, in the unlikely case the gem wants to be able to reload.
       #
-      # @sig () -> Zeitwerk::Loader
-      def for_gem
+      # This method returns a subclass of Zeitwerk::Loader, but the exact type
+      # is private, client code can only rely on the interface.
+      #
+      # @sig (bool) -> Zeitwerk::GemLoader
+      def for_gem(warn_on_extra_files: true)
         called_from = caller_locations(1, 1).first.path
-        Registry.loader_for_gem(called_from)
+        Registry.loader_for_gem(called_from, warn_on_extra_files: warn_on_extra_files)
       end
 
       # Broadcasts `eager_load` to all loaders.
@@ -326,8 +327,6 @@ module Zeitwerk
       end
     end
 
-    self.mutex = Mutex.new
-
     private # -------------------------------------------------------------------------------------
 
     # @sig (String, Module) -> void
@@ -338,7 +337,7 @@ module Zeitwerk
             basename.delete_suffix!(".rb")
             cname = inflector.camelize(basename, abspath).to_sym
             autoload_file(parent, cname, abspath)
-          elsif dir?(abspath)
+          else
             # In a Rails application, `app/models/concerns` is a subdirectory of
             # `app/models`, but both of them are root directories.
             #
@@ -466,7 +465,7 @@ module Zeitwerk
 
     # @sig (String) -> void
     def raise_if_conflicting_directory(dir)
-      self.class.mutex.synchronize do
+      MUTEX.synchronize do
         Registry.loaders.each do |loader|
           next if loader == self
           next if loader.ignores?(dir)

data/lib/zeitwerk/registry.rb

@@ -10,12 +10,11 @@ module Zeitwerk
       # @sig Array[Zeitwerk::Loader]
       attr_reader :loaders
 
-      # Registers loaders created with `for_gem` to make the method idempotent
-      # in case of reload.
+      # Registers gem loaders to let `for_gem` be idempotent in case of reload.
       #
       # @private
       # @sig Hash[String, Zeitwerk::Loader]
-      attr_reader :loaders_managing_gems
+      attr_reader :gem_loaders_by_root_file
 
       # Maps absolute paths to the loaders responsible for them.
       #
@@ -77,7 +76,7 @@ module Zeitwerk
       # @sig (Zeitwerk::Loader) -> void
       def unregister_loader(loader)
         loaders.delete(loader)
-        loaders_managing_gems.delete_if { |_, l| l == loader }
+        gem_loaders_by_root_file.delete_if { |_, l| l == loader }
         autoloads.delete_if { |_, l| l == loader }
         inceptions.delete_if { |_, (_, l)| l == loader }
       end
@@ -87,14 +86,8 @@ module Zeitwerk
       #
       # @private
       # @sig (String) -> Zeitwerk::Loader
-      def loader_for_gem(root_file)
-        loaders_managing_gems[root_file] ||= begin
-          Loader.new.tap do |loader|
-            loader.tag = File.basename(root_file, ".rb")
-            loader.inflector = GemInflector.new(root_file)
-            loader.push_dir(File.dirname(root_file))
-          end
-        end
+      def loader_for_gem(root_file, warn_on_extra_files:)
+        gem_loaders_by_root_file[root_file] ||= GemLoader._new(root_file, warn_on_extra_files: warn_on_extra_files)
       end
 
       # @private
@@ -137,9 +130,9 @@ module Zeitwerk
       end
     end
 
-    @loaders               = []
-    @loaders_managing_gems = {}
-    @autoloads             = {}
-    @inceptions            = {}
+    @loaders                  = []
+    @gem_loaders_by_root_file = {}
+    @autoloads                = {}
+    @inceptions               = {}
   end
 end

data/lib/zeitwerk/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Zeitwerk
-  VERSION = "2.5.4"
+  VERSION = "2.6.1"
 end

data/lib/zeitwerk.rb

@@ -3,6 +3,7 @@
 module Zeitwerk
   require_relative "zeitwerk/real_mod_name"
   require_relative "zeitwerk/loader"
+  require_relative "zeitwerk/gem_loader"
   require_relative "zeitwerk/registry"
   require_relative "zeitwerk/explicit_namespace"
   require_relative "zeitwerk/inflector"

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: zeitwerk
 version: !ruby/object:Gem::Version
-  version: 2.5.4
+  version: 2.6.1
 platform: ruby
 authors:
 - Xavier Noria
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2022-01-27 00:00:00.000000000 Z
+date: 2022-09-30 00:00:00.000000000 Z
 dependencies: []
 description: |2
       Zeitwerk implements constant autoloading with Ruby semantics. Each gem
@@ -26,6 +26,7 @@ files:
 - lib/zeitwerk/error.rb
 - lib/zeitwerk/explicit_namespace.rb
 - lib/zeitwerk/gem_inflector.rb
+- lib/zeitwerk/gem_loader.rb
 - lib/zeitwerk/inflector.rb
 - lib/zeitwerk/kernel.rb
 - lib/zeitwerk/loader.rb