zeitwerk 2.5.0.beta2 → 2.5.0.beta6

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 64ea17faefd07265c96bd8c49c9a4e14170612f85f868c03df91aa4511e473e3
-  data.tar.gz: fb98b0f841e49016039d73310d50563e8e6c3bbcf4c16f2d4441b241cb266647
+  metadata.gz: 74d74bec1f4963cc5d04f037e14723408ae5d1f8c4df16f31f2901402e3230ed
+  data.tar.gz: 3ebd7f860583cc7a560360b2e766193dc0c74c5cd902c7cb632a614e32dc68e7
 SHA512:
-  metadata.gz: 3dfdc023489a4b7306c2bbac6f56234ff4a80887820967ff691fd9b13609ea256ba5bca301870d5e5b35d26c09ece01158a984d3093fa63d0de0bd3d9cdff458
-  data.tar.gz: 4860a04e9e489251ce43e56f37c7213ac3dda7c301f9dff1826d7e5f2567b992bfac35f645dab67a356bb8a4024c29d0fd8431f91896f7a254bf747864dd8ef0
+  metadata.gz: f0bec92696285efb5ad1f76094b71a4de5a38a0665b493737c945623042b4d744be2c58f4006ed5c8c4abe0ccb70854c94ba63c3b7eeef1a5ae47086a956668d
+  data.tar.gz: a1392b3b73dcd85769f793aeca9a9d10cb50c70259d88761574c1e62e201b24f32d7122eda587af6017dea93fca27ce9463e5f0dae7b98523d3edba2c13802eb

data/README.md

@@ -19,19 +19,24 @@
   - [Implicit namespaces](#implicit-namespaces)
   - [Explicit namespaces](#explicit-namespaces)
   - [Collapsing directories](#collapsing-directories)
+  - [Testing compliance](#testing-compliance)
 - [Usage](#usage)
   - [Setup](#setup)
     - [Generic](#generic)
     - [for_gem](#for_gem)
   - [Autoloading](#autoloading)
   - [Eager loading](#eager-loading)
+    - [Eager load exclusions](#eager-load-exclusions)
+    - [Global eager load](#global-eager-load)
   - [Reloading](#reloading)
   - [Inflection](#inflection)
     - [Zeitwerk::Inflector](#zeitwerkinflector)
     - [Zeitwerk::GemInflector](#zeitwerkgeminflector)
     - [Custom inflector](#custom-inflector)
-  - [The on_load callback](#the-on_load-callback)
-  - [The on_unload callback](#the-on_unload-callback)
+  - [Callbacks](#callbacks)
+    - [The on_setup callback](#the-on_setup-callback)
+    - [The on_load callback](#the-on_load-callback)
+    - [The on_unload callback](#the-on_unload-callback)
     - [Technical details](#technical-details)
   - [Logging](#logging)
     - [Loader tag](#loader-tag)
@@ -40,15 +45,19 @@
     - [Use case: The adapter pattern](#use-case-the-adapter-pattern)
     - [Use case: Test files mixed with implementation files](#use-case-test-files-mixed-with-implementation-files)
   - [Edge cases](#edge-cases)
+  - [Beware of circular dependencies](#beware-of-circular-dependencies)
   - [Reopening third-party namespaces](#reopening-third-party-namespaces)
   - [Rules of thumb](#rules-of-thumb)
   - [Debuggers](#debuggers)
+    - [debug.rb](#debugrb)
     - [Break](#break)
     - [Byebug](#byebug)
 - [Pronunciation](#pronunciation)
 - [Supported Ruby versions](#supported-ruby-versions)
 - [Testing](#testing)
 - [Motivation](#motivation)
+  - [Kernel#require is brittle](#kernelrequire-is-brittle)
+  - [Rails autoloading was brittle](#rails-autoloading-was-brittle)
 - [Thanks](#thanks)
 - [License](#license)
 
@@ -157,6 +166,16 @@ class HttpCrawler
 end
 ```
 
+The first example needs a custom [inflection](https://github.com/fxn/zeitwerk#inflection) rule:
+
+```ruby
+loader.inflector.inflect("max_retries" => "MAX_RETRIES")
+```
+
+Otherwise, Zeitwerk would expect the file to define `MaxRetries`.
+
+In the second example, no custom rule is needed.
+
 <a id="markdown-root-directories-and-root-namespaces" name="root-directories-and-root-namespaces"></a>
 ### Root directories and root namespaces
 
@@ -275,6 +294,23 @@ To illustrate usage of glob patterns, if `actions` in the example above is part
 loader.collapse("#{__dir__}/*/actions")
 ```
 
+<a id="markdown-testing-compliance" name="testing-compliance"></a>
+### Testing compliance
+
+When a managed file is loaded, Zeitwerk verifies the expected constant is defined. If it is not, `Zeitwerk::NameError` is raised.
+
+So, an easy way to ensure compliance in the test suite is to eager load the project:
+
+```ruby
+begin
+  loader.eager_load(force: true)
+rescue Zeitwerk::NameError => e
+  flunk e.message
+else
+  assert true
+end
+```
+
 <a id="markdown-usage" name="usage"></a>
 ## Usage
 
@@ -377,7 +413,16 @@ Zeitwerk instances are able to eager load their managed files:
 loader.eager_load
 ```
 
-That skips [ignored files and directories](#ignoring-parts-of-the-project), and you can also tell Zeitwerk that certain files or directories are autoloadable, but should not be eager loaded:
+That skips [ignored files and directories](#ignoring-parts-of-the-project).
+
+In gems, the method needs to be invoked after the main namespace has been defined, as shown in [Synopsis](https://github.com/fxn/zeitwerk#synopsis).
+
+Eager loading is synchronized and idempotent.
+
+<a id="markdown-eager-load-exclusions" name="eager-load-exclusions"></a>
+#### Eager load exclusions
+
+ You can tell Zeitwerk that certain files or directories are autoloadable, but should not be eager loaded:
 
 ```ruby
 db_adapters = "#{__dir__}/my_gem/db_adapters"
@@ -386,13 +431,20 @@ loader.setup
 loader.eager_load # won't eager load the database adapters
 ```
 
-In gems, the method needs to be invoked after the main namespace has been defined, as shown in [Synopsis](https://github.com/fxn/zeitwerk#synopsis).
+However, that can be overridden with `force`:
 
-Eager loading is synchronized and idempotent.
+```ruby
+loader.eager_load(force: true) # database adapters are eager loaded
+```
+
+Which may be handy if the project eager loads in the test suite to [ensure project layour compliance](#testing-compliance).
 
-If eager loading a file does not define the expected class or module, Zeitwerk raises `Zeitwerk::NameError`, which is a subclass of `NameError`.
+The `force` flag does not affect ignored files and directories, those are still ignored.
 
-If you want to eager load yourself and all dependencies using Zeitwerk, you can broadcast the `eager_load` call to all instances:
+<a id="markdown-global-eager-load" name="global-eager-load"></a>
+#### Global eager load
+
+If you want to eager load yourself and all dependencies that use Zeitwerk, you can broadcast the `eager_load` call to all instances:
 
 ```ruby
 Zeitwerk::Loader.eager_load_all
@@ -402,6 +454,8 @@ This may be handy in top-level services, like web applications.
 
 Note that thanks to idempotence `Zeitwerk::Loader.eager_load_all` won't eager load twice if any of the instances already eager loaded.
 
+This method does not accept the `force` flag, since in general it wouldn't be a good idea to force eager loading in 3rd party code.
+
 <a id="markdown-reloading" name="reloading"></a>
 ### Reloading
 
@@ -551,8 +605,26 @@ class MyGem::Inflector < Zeitwerk::GemInflector
 end
 ```
 
+<a id="markdown-callbacks" name="callbacks"></a>
+### Callbacks
+
+<a id="markdown-the-on_setup-callback" name="the-on_setup-callback"></a>
+#### The on_setup callback
+
+The `on_setup` callback is fired on setup and on each reload:
+
+```ruby
+loader.on_setup do
+  # Ready to autoload here.
+end
+```
+
+Multiple `on_setup` callbacks are supported, and they run in order of definition.
+
+If `setup` was already executed, the callback is fired immediately.
+
 <a id="markdown-the-on_load-callback" name="the-on_load-callback"></a>
-### The on_load callback
+#### The on_load callback
 
 The usual place to run something when a file is loaded is the file itself. However, sometimes you'd like to be called, and this is possible with the `on_load` callback.
 
@@ -610,8 +682,10 @@ There are use cases for this last catch-all callback, but they are rare. If you
 
 If both types of callbacks are defined, the specific ones run first.
 
+Since `on_load` callbacks are executed right after files are loaded, even if the loading context seems to be far away, in practice **the block is subject to [circular dependencies](#beware-of-circular-dependencies)**. As a rule of thumb, as far as loading order and its interdependencies is concerned, you have to program as if the block was executed at the bottom of the file just loaded.
+
 <a id="markdown-the-on_unload-callback" name="the-on_unload-callback"></a>
-### The on_unload callback
+#### The on_unload callback
 
 When reloading is enabled, you may occasionally need to execute something before a certain autoloaded class or module is unloaded. The `on_unload` callback allows you to do that.
 
@@ -822,6 +896,26 @@ Trip = Struct.new { ... } # NOT SUPPORTED
 
 This only affects explicit namespaces, those idioms work well for any other ordinary class or module.
 
+<a id="markdown-beware-of-circular-dependencies" name="beware-of-circular-dependencies"></a>
+### Beware of circular dependencies
+
+In Ruby, you can't have certain top-level circular dependencies. Take for example:
+
+```ruby
+# c.rb
+class C < D
+end
+
+# d.rb
+class D
+  C
+end
+```
+
+In order to define `C`, you need to load `D`. However, the body of `D` refers to `C`.
+
+Circular dependencies like those do not work in plain Ruby, and therefore do not work in projects managed by Zeitwerk either.
+
 <a id="markdown-reopening-third-party-namespaces" name="reopening-third-party-namespaces"></a>
 ### Reopening third-party namespaces
 
@@ -876,6 +970,11 @@ With that, when Zeitwerk scans the file system and reaches the gem directories `
 <a id="markdown-debuggers" name="debuggers"></a>
 ### Debuggers
 
+<a id="markdown-debugrb" name="debugrb"></a>
+#### debug.rb
+
+The new [debug.rb](https://github.com/ruby/debug) gem and Zeitwerk seem to be compatible, as far as I can tell. This is the new debugger that is going to ship with Ruby 3.1.
+
 <a id="markdown-break" name="break"></a>
 #### Break
 
@@ -894,7 +993,11 @@ Zeitwerk and [Byebug](https://github.com/deivid-rodriguez/byebug) are incompatib
 <a id="markdown-supported-ruby-versions" name="supported-ruby-versions"></a>
 ## Supported Ruby versions
 
-Zeitwerk works with MRI 2.4.4 and above.
+Zeitwerk works with CRuby 2.5 and above.
+
+On TruffleRuby all is good except for thread-safety. Right now, in TruffleRuby `Module#autoload` does not block threads accessing a constant that is being autoloaded. CRuby prevents such access to avoid concurrent threads from seeing partial evaluations of the corresponding file. Zeitwerk inherits autoloading thread-safety from this property. This is not an issue if your project gets eager loaded, or if you lazy load in single-threaded environments. (See https://github.com/oracle/truffleruby/issues/2431.)
+
+JRuby 9.3.0.0 is almost there. As of this writing, the test suite of Zeitwerk passes on JRuby except for three tests. (See https://github.com/jruby/jruby/issues/6781.)
 
 <a id="markdown-testing" name="testing"></a>
 ## Testing
@@ -925,9 +1028,19 @@ and run `bin/test`.
 <a id="markdown-motivation" name="motivation"></a>
 ## Motivation
 
-Since `require` has global side-effects, and there is no static way to verify that you have issued the `require` calls for code that your file depends on, in practice it is very easy to forget some. That introduces bugs that depend on the load order. Zeitwerk provides a way to forget about `require` in your own code, just name things following conventions and done.
+<a id="markdown-kernelrequire-is-brittle" name="kernelrequire-is-brittle"></a>
+### Kernel#require is brittle
+
+Since `require` has global side-effects, and there is no static way to verify that you have issued the `require` calls for code that your file depends on, in practice it is very easy to forget some. That introduces bugs that depend on the load order.
+
+Also, if the project has namespaces, setting things up and getting client code to load things in a consistent way needs discipline. For example, `require "foo/bar"` may define `Foo`, instead of reopen it. That may be a broken window, giving place to superclass mismatches or partially-defined namespaces.
+
+With Zeitwerk, you just name things following conventions and done. Things are available everywhere, and descend is always orderly. Without effort and without broken windows.
+
+<a id="markdown-rails-autoloading-was-brittle" name="rails-autoloading-was-brittle"></a>
+### Rails autoloading was brittle
 
-On the other hand, autoloading in Rails is based on `const_missing`, which lacks fundamental information like the nesting and the resolution algorithm that was being used. Because of that, Rails autoloading is not able to match Ruby's semantics and that introduces a series of gotchas. The original goal of this project was to bring a better autoloading mechanism for Rails 6.
+Autoloading in Rails was based on `const_missing` up to Rails 5. That callback lacks fundamental information like the nesting or the resolution algorithm being used. Because of that, Rails autoloading was not able to match Ruby's semantics, and that introduced a [series of issues](https://guides.rubyonrails.org/v5.2/autoloading_and_reloading_constants.html#common-gotchas). Zeitwerk is based on a different technique and fixed Rails autoloading starting with Rails 6.
 
 <a id="markdown-thanks" name="thanks"></a>
 ## Thanks

data/lib/zeitwerk/autoloads.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 module Zeitwerk
   # @private
   class Autoloads

data/lib/zeitwerk/error.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 module Zeitwerk
   class Error < StandardError
   end

data/lib/zeitwerk/explicit_namespace.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 module Zeitwerk
   # Centralizes the logic for the trace point used to detect the creation of
   # explicit namespaces, needed to descend into matching subdirectories right

data/lib/zeitwerk/kernel.rb

@@ -3,7 +3,7 @@
 module Kernel
   module_function
 
-  # We are going to decorate Kerner#require with two goals.
+  # We are going to decorate Kernel#require with two goals.
   #
   # First, by intercepting Kernel#require calls, we are able to autovivify
   # modules on required directories, and also do internal housekeeping when

data/lib/zeitwerk/loader/callbacks.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 module Zeitwerk::Loader::Callbacks
   include Zeitwerk::RealModName
 
@@ -9,16 +11,19 @@ module Zeitwerk::Loader::Callbacks
     cref  = autoloads.delete(file)
     cpath = cpath(*cref)
 
+    # If reloading is enabled, we need to put this constant for unloading
+    # regardless of what cdef? says. In Ruby < 3.1 the internal state is not
+    # fully cleared. Module#constants still includes it, and you need to
+    # remove_const. See https://github.com/ruby/ruby/pull/4715.
     to_unload[cpath] = [file, cref] if reloading_enabled?
     Zeitwerk::Registry.unregister_autoload(file)
 
-    if logger && cdef?(*cref)
-      log("constant #{cpath} loaded from file #{file}")
-    elsif !cdef?(*cref)
+    if cdef?(*cref)
+      log("constant #{cpath} loaded from file #{file}") if logger
+      run_on_load_callbacks(cpath, cget(*cref), file) unless on_load_callbacks.empty?
+    else
       raise Zeitwerk::NameError.new("expected file #{file} to define constant #{cpath}, but didn't", cref.last)
     end
-
-    run_on_load_callbacks(cpath, cget(*cref), file) unless on_load_callbacks.empty?
   end
 
   # Invoked from our decorated Kernel#require when a managed directory is

data/lib/zeitwerk/loader/config.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 require "set"
 require "securerandom"
 
@@ -54,6 +56,12 @@ module Zeitwerk::Loader::Config
   # @sig Set[String]
   attr_reader :eager_load_exclusions
 
+  # User-oriented callbacks to be fired on setup and on reload.
+  #
+  # @private
+  # @sig Array[{ () -> void }]
+  attr_reader :on_setup_callbacks
+
   # User-oriented callbacks to be fired when a constant is loaded.
   #
   # @private
@@ -81,6 +89,7 @@ module Zeitwerk::Loader::Config
     @collapse_dirs          = Set.new
     @eager_load_exclusions  = Set.new
     @reloading_enabled      = false
+    @on_setup_callbacks     = []
     @on_load_callbacks      = {}
     @on_unload_callbacks    = {}
     @logger                 = self.class.default_logger
@@ -188,6 +197,17 @@ module Zeitwerk::Loader::Config
     end
   end
 
+  # Configure a block to be called after setup and on each reload.
+  # If setup was already done, the block runs immediately.
+  #
+  # @sig () { () -> void } -> void
+  def on_setup(&block)
+    mutex.synchronize do
+      on_setup_callbacks << block
+      block.call if @setup
+    end
+  end
+
   # Configure a block to be invoked once a certain constant path is loaded.
   # Supports multiple callbacks, and if there are many, they are executed in
   # the order in which they were defined.
@@ -247,17 +267,10 @@ module Zeitwerk::Loader::Config
 
   # @private
   # @sig (String) -> bool
-  def manages?(dir)
-    dir = dir + "/"
-    ignored_paths.each do |ignored_path|
-      return false if dir.start_with?(ignored_path + "/")
+  def ignores?(abspath)
+    ignored_paths.any? do |ignored_path|
+      ignored_path == abspath || (dir?(ignored_path) && abspath.start_with?(ignored_path + "/"))
     end
-
-    root_dirs.each_key do |root_dir|
-      return true if root_dir.start_with?(dir) || dir.start_with?(root_dir + "/")
-    end
-
-    false
   end
 
   private

data/lib/zeitwerk/loader/helpers.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 module Zeitwerk::Loader::Helpers
   private
 

data/lib/zeitwerk/loader.rb

@@ -110,6 +110,8 @@ module Zeitwerk
           set_autoloads_in_dir(root_dir, namespace)
         end
 
+        on_setup_callbacks.each(&:call)
+
         @setup = true
       end
     end
@@ -143,7 +145,7 @@ module Zeitwerk
             # Could happen if loaded with require_relative. That is unsupported,
             # and the constant path would escape unloadable_cpath? This is just
             # defensive code to clean things up as much as we are able to.
-            unload_cref(parent, cname)  if cdef?(parent, cname)
+            unload_cref(parent, cname)
             unloaded_files.add(abspath) if ruby?(abspath)
           end
         end
@@ -154,7 +156,7 @@ module Zeitwerk
             run_on_unload_callbacks(cpath, value, abspath)
           end
 
-          unload_cref(parent, cname)  if cdef?(parent, cname)
+          unload_cref(parent, cname)
           unloaded_files.add(abspath) if ruby?(abspath)
         end
 
@@ -208,25 +210,28 @@ module Zeitwerk
     # Eager loads all files in the root directories, recursively. Files do not
     # need to be in `$LOAD_PATH`, absolute file names are used. Ignored files
     # are not eager loaded. You can opt-out specifically in specific files and
-    # directories with `do_not_eager_load`.
+    # directories with `do_not_eager_load`, and that can be overridden passing
+    # `force: true`.
     #
-    # @sig () -> void
-    def eager_load
+    # @sig (true | false) -> void
+    def eager_load(force: false)
       mutex.synchronize do
         break if @eager_loaded
 
         log("eager load start") if logger
 
+        honour_exclusions = !force
+
         queue = []
         actual_root_dirs.each do |root_dir, namespace|
-          queue << [namespace, root_dir] unless excluded_from_eager_load?(root_dir)
+          queue << [namespace, root_dir] unless honour_exclusions && excluded_from_eager_load?(root_dir)
         end
 
         while to_eager_load = queue.shift
           namespace, dir = to_eager_load
 
           ls(dir) do |basename, abspath|
-            next if excluded_from_eager_load?(abspath)
+            next if honour_exclusions && excluded_from_eager_load?(abspath)
 
             if ruby?(abspath)
               if cref = autoloads.cref_for(abspath)
@@ -454,12 +459,21 @@ module Zeitwerk
     def raise_if_conflicting_directory(dir)
       self.class.mutex.synchronize do
         Registry.loaders.each do |loader|
-          if loader != self && loader.manages?(dir)
-            require "pp"
-            raise Error,
-              "loader\n\n#{pretty_inspect}\n\nwants to manage directory #{dir}," \
-              " which is already managed by\n\n#{loader.pretty_inspect}\n"
-            EOS
+          next if loader == self
+          next if loader.ignores?(dir)
+
+          dir = dir + "/"
+          loader.root_dirs.each do |root_dir, _namespace|
+            next if ignores?(root_dir)
+
+            root_dir = root_dir + "/"
+            if dir.start_with?(root_dir) || root_dir.start_with?(dir)
+              require "pp" # Needed for pretty_inspect, even in Ruby 2.5.
+              raise Error,
+                "loader\n\n#{pretty_inspect}\n\nwants to manage directory #{dir.chop}," \
+                " which is already managed by\n\n#{loader.pretty_inspect}\n"
+              EOS
+            end
           end
         end
       end
@@ -480,7 +494,13 @@ module Zeitwerk
 
     # @sig (Module, Symbol) -> void
     def unload_cref(parent, cname)
+      # Let's optimistically remove_const. The way we use it, this is going to
+      # succeed always if all is good.
       parent.__send__(:remove_const, cname)
+    rescue ::NameError
+      # There are a few edge scenarios in which this may happen. If the constant
+      # is gone, that is OK, anyway.
+    else
       log("#{cpath(parent, cname)} unloaded") if logger
     end
   end

data/lib/zeitwerk/real_mod_name.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 module Zeitwerk::RealModName
   UNBOUND_METHOD_MODULE_NAME = Module.instance_method(:name)
   private_constant :UNBOUND_METHOD_MODULE_NAME

data/lib/zeitwerk/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Zeitwerk
-  VERSION = "2.5.0.beta2"
+  VERSION = "2.5.0.beta6"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: zeitwerk
 version: !ruby/object:Gem::Version
-  version: 2.5.0.beta2
+  version: 2.5.0.beta6
 platform: ruby
 authors:
 - Xavier Noria
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2021-08-20 00:00:00.000000000 Z
+date: 2021-10-14 00:00:00.000000000 Z
 dependencies: []
 description: |2
       Zeitwerk implements constant autoloading with Ruby semantics. Each gem