bootsnap 1.6.0 → 1.15.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: '029d63ba428f470d2cd2ef194d857e20689e7c8e377e2eaaaab83570f366034a'
-  data.tar.gz: 36a55f9d12dddef6ea3c4739f586c9236d7f4bc677a8b6747dfd7465d46eeca2
+  metadata.gz: 68bf895082a1d27386514357e36c1b53972f7f2e22f50b2f9b77cde53fec9110
+  data.tar.gz: dcc033dc6522aa59d853f7840d228430aa5591892d063d91c061fb1c7edcb820
 SHA512:
-  metadata.gz: 131ec17c4e4912f387c18778250e689467ebfcc80e91eb884237307af1a57a3c89d4fb20c772fbc0330123a796d631861a9cf145bd32c54183077bbc97d7fa6f
-  data.tar.gz: d0c92454c8a5d8b16a25908fd0ae134fc817c5977958bde8424baca2bb6c96e60eb648c9f770bf7c7f58a3dd98871d4e7b0e3a0950c269100fded45ca9fddfa3
+  metadata.gz: b640bf89d0448536c17f88352e97aa4b5a321cec6abfcd011d1f442ec9c9170b0b08e0329e398919140da1b0d1e65d343b761be392d25eb75540425c42d766cb
+  data.tar.gz: 97080b27ccca5dd741bfccc0c3cd412c7fcea917fad2e79961ef2a7c18a1dd5478fc33f2a4d9f59a5f8765696760aef76799f401b1cf13b9a68d7ffc0d0a326b

data/CHANGELOG.md

@@ -1,5 +1,179 @@
 # Unreleased
 
+# 1.15.0
+
+* Add a readonly mode, for environments in which the updated cache wouldn't be persisted. See #428 and #423.
+
+# 1.14.0
+
+* Require Ruby 2.6.
+* Add a way to skip directories during load path scanning.
+  If you have large non-ruby directories in the middle of your load path, it can severely slow down scanning.
+  Typically this is a problem with `node_modules`. See #277.
+* Fix `Bootsnap.unload_cache!`, it simply wouldn't work at all becaue of a merge mistake. See #421.
+
+# 1.13.0
+
+* Stop decorating `Kernel.load`. This used to be very useful in development because the Rails "classic" autoloader
+  was using `Kernel.load` in dev and `Kernel.require` in production. But Zeitwerk is now the default, and it doesn't
+  use `Kernel.load` at all.
+
+  People still using the classic autoloader might want to stick to `bootsnap 1.12`.
+
+* Add `Bootsnap.unload_cache!`. Applications can call it at the end of their boot sequence when they know
+  no more code will be loaded to reclaim a bit of memory.
+
+# 1.12.0
+
+* `bootsnap precompile` CLI will now also precompile `Rakefile` and `.rake` files.
+
+* Stop decorating `Module#autoload` as it was only useful for supporting Ruby 2.2 and older.
+
+* Remove `uname` and other patform specific version from the cache keys. `RUBY_PLATFORM + RUBY_REVISION` should be
+  enough to ensure bytecode compatibility. This should improve caching for alpine based setups. See #409.
+
+# 1.11.1
+
+* Fix the `can't modify frozen Hash` error on load path cache mutation. See #411.
+
+# 1.11.0
+
+* Drop dependency on `fileutils`.
+
+* Better respect `Kernel#require` duck typing. While it almost never comes up in practice, `Kernel#require`
+  follow a fairly intricate duck-typing protocol on its argument implemented as `rb_get_path(VALUE)` in MRI.
+  So when applicable we bind `rb_get_path` and use it for improved compatibility. See #396 and #406.
+
+* Get rid of the `Kernel.require_relative` decorator by resolving `$LOAD_PATH` members to their real path.
+  This way we handle symlinks in `$LOAD_PATH` much more efficiently. See #402 for the detailed explanation.
+  
+* Drop support for Ruby 2.3 (to allow getting rid of the `Kernel.require_relative` decorator).
+
+# 1.10.3
+
+* Fix Regexp and Date type support in YAML compile cache. (#400)
+
+* Improve the YAML compile cache to support `UTF-8` symbols. (#398, #399)
+  [The default `MessagePack` symbol serializer assumes all symbols are ASCII](https://github.com/msgpack/msgpack-ruby/pull/211),
+  because of this, non-ASCII compatible symbol would be restored with `ASCII_8BIT` encoding (AKA `BINARY`).
+  Bootsnap now properly cache them in `UTF-8`.
+
+  Note that the above only apply for actual YAML symbols (e..g `--- :foo`).
+  The issue is still present for string keys parsed with `YAML.load_file(..., symbolize_names: true)`, that is a bug
+  in `msgpack` that will hopefully be solved soon, see: https://github.com/msgpack/msgpack-ruby/pull/246
+
+* Entirely disable the YAML compile cache if `Encoding.default_internal` is set to an encoding not supported by `msgpack`. (#398)
+  `Psych` coerce strings to `Encoding.default_internal`, but `MessagePack` doesn't. So in this scenario we can't provide
+  YAML caching at all without returning the strings in the wrong encoding.
+  This never came up in practice but might as well be safe.
+
+# 1.10.2
+
+* Reduce the `Kernel.require` extra stack frames some more. Now bootsnap should only add one extra frame per `require` call.
+
+* Better check `freeze` option support in JSON compile cache.
+  Previously `JSON.load_file(..., freeze: true)` would be cached even when the msgpack version is missing support for it.
+
+# 1.10.1
+
+* Fix `Kernel#autoload`'s fallback path always being executed.
+
+* Consider `unlink` failing with `ENOENT` as a success.
+
+# 1.10.0
+
+* Delay requiring `FileUtils`. (#285)
+  `FileUtils` can be installed as a gem, so it's best to wait for bundler to have setup the load path before requiring it.
+
+* Improve support of Psych 4. (#392)
+  Since `1.8.0`, `YAML.load_file` was no longer cached when Psych 4 was used. This is because `load_file` loads
+  in safe mode by default, so the Bootsnap cache could defeat that safety.
+  Now when precompiling YAML files, Bootsnap first try to parse them in safe mode, and if it can't fallback to unsafe mode,
+  and the cache contains a flag that records whether it was generated in safe mode or not.
+  `YAML.unsafe_load_file` will use safe caches just fine, but `YAML.load_file` will fallback to uncached YAML parsing
+  if the cache was generated using unsafe parsing.
+
+* Minimize the Kernel.require extra stack frames. (#393)
+  This should reduce the noise generated by bootsnap on `LoadError`.
+
+# 1.9.4
+
+* Ignore absolute paths in the loaded feature index. (#385)
+  This fixes a compatibility issue with Zeitwerk when Zeitwerk is loaded before bootsnap. It also should
+  reduce the memory usage and improve load performance of Zeitwerk managed files.
+
+* Automatically invalidate the load path cache whenever the Ruby version change. (#387)
+  This is to avoid issues in case the same installation path is re-used for subsequent ruby patch releases.
+
+# 1.9.3
+
+* Only disable the compile cache for source files impacted by [Ruby 3.0.3 [Bug 18250]](https://bugs.ruby-lang.org/issues/18250).
+  This should keep the performance loss to a minimum.
+
+# 1.9.2
+
+* Disable compile cache if [Ruby 3.0.3's ISeq cache bug](https://bugs.ruby-lang.org/issues/18250) is detected.
+  AKA `iseq.rb:13 to_binary: wrong argument type false (expected Symbol)`
+* Fix `Kernel.load` behavior: before `load 'a'` would load `a.rb` (and other tried extensions) and wouldn't load `a` unless `development_mode: true`, now only `a` would be loaded and files with extensions wouldn't be.
+
+# 1.9.1
+
+* Removed a forgotten debug statement in JSON precompilation.
+
+# 1.9.0
+
+* Added a compilation cache for `JSON.load_file`. (#370)
+
+# 1.8.1
+
+* Fixed support for older Psych. (#369)
+
+# 1.8.0
+
+* Improve support for Psych 4. (#368)
+
+# 1.7.7
+
+* Fix `require_relative` in evaled code on latest ruby 3.1.0-dev. (#366)
+
+# 1.7.6
+
+* Fix reliance on `set` to be required.
+* Fix `Encoding::UndefinedConversionError` error for Rails applications when precompiling cache. (#364)
+
+# 1.7.5
+
+* Handle a regression of Ruby 2.7.3 causing Bootsnap to call the deprecated `untaint` method. (#360)
+* Gracefully handle read-only file system as well as other errors preventing to persist the load path cache. (#358)
+
+# 1.7.4
+
+* Stop raising errors when encountering various file system errors. The cache is now best effort,
+  if somehow it can't be saved, bootsnap will gracefully fallback to the original operation (e.g. `Kernel.require`).
+  (#353, #177, #262)
+
+# 1.7.3
+
+* Disable YAML precompilation when encountering YAML tags. (#351)
+
+# 1.7.2
+
+* Fix compatibility with msgpack < 1. (#349)
+
+# 1.7.1
+
+* Warn Ruby 2.5 users if they turn ISeq caching on. (#327, #244)
+* Disable ISeq caching for the whole 2.5.x series again.
+* Better handle hashing of Ruby strings. (#318)
+
+# 1.7.0
+
+* Fix detection of YAML files in gems.
+* Adds an instrumentation API to monitor cache misses.
+* Allow to control the behavior of `require 'bootsnap/setup'` using environment variables.
+* Deprecate the `disable_trace` option.
+* Deprecate the `ActiveSupport::Dependencies` (AKA Classic autoloader) integration. (#344) 
+
 # 1.6.0
 
 * Fix a Ruby 2.7/3.0 issue with `YAML.load_file` keyword arguments. (#342)

data/LICENSE.txt

@@ -1,6 +1,6 @@
 The MIT License (MIT)
 
-Copyright (c) 2017 Shopify, Inc.
+Copyright (c) 2017-present Shopify, Inc.
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

data/README.md

@@ -1,6 +1,6 @@
 # Bootsnap [![Actions Status](https://github.com/Shopify/bootsnap/workflows/ci/badge.svg)](https://github.com/Shopify/bootsnap/actions)
 
-Bootsnap is a library that plugs into Ruby, with optional support for `ActiveSupport` and `YAML`,
+Bootsnap is a library that plugs into Ruby, with optional support for `YAML`,
 to optimize and cache expensive computations. See [How Does This Work](#how-does-this-work).
 
 #### Performance
@@ -11,7 +11,7 @@ to optimize and cache expensive computations. See [How Does This Work](#how-does
 - The core Shopify platform -- a rather large monolithic application -- boots about 75% faster,
   dropping from around 25s to 6.5s.
 * In Shopify core (a large app), about 25% of this gain can be attributed to `compile_cache_*`
-  features; 75% to path caching, and ~1% to `disable_trace`. This is fairly representative.
+  features; 75% to path caching. This is fairly representative.
 
 ## Usage
 
@@ -29,7 +29,7 @@ If you are using Rails, add this to `config/boot.rb` immediately after `require
 require 'bootsnap/setup'
 ```
 
-Note that bootsnap writes to `tmp/cache` (or the path specified by `ENV['BOOTSNAP_CACHE_DIR']`), 
+Note that bootsnap writes to `tmp/cache` (or the path specified by `ENV['BOOTSNAP_CACHE_DIR']`),
 and that directory *must* be writable. Rails will fail to
 boot if it is not. If this is unacceptable (e.g. you are running in a read-only container and
 unwilling to mount in a writable tmpdir), you should remove this line or wrap it in a conditional.
@@ -41,7 +41,7 @@ getting progressively slower, this is almost certainly the cause.**
 It's technically possible to simply specify `gem 'bootsnap', require: 'bootsnap/setup'`, but it's
 important to load Bootsnap as early as possible to get maximum performance improvement.
 
-You can see how this require works [here](https://github.com/Shopify/bootsnap/blob/master/lib/bootsnap/setup.rb).
+You can see how this require works [here](https://github.com/Shopify/bootsnap/blob/main/lib/bootsnap/setup.rb).
 
 If you are not using Rails, or if you are but want more control over things, add this to your
 application setup immediately after `require 'bundler/setup'` (i.e. as early as possible: the sooner
@@ -52,17 +52,15 @@ require 'bootsnap'
 env = ENV['RAILS_ENV'] || "development"
 Bootsnap.setup(
   cache_dir:            'tmp/cache',          # Path to your cache
+  ignore_directories:   ['node_modules'],     # Directory names to skip.
   development_mode:     env == 'development', # Current working environment, e.g. RACK_ENV, RAILS_ENV, etc
   load_path_cache:      true,                 # Optimize the LOAD_PATH with a cache
-  autoload_paths_cache: true,                 # Optimize ActiveSupport autoloads with cache
-  disable_trace:        true,                 # Set `RubyVM::InstructionSequence.compile_option = { trace_instruction: false }`
   compile_cache_iseq:   true,                 # Compile Ruby code into ISeq cache, breaks coverage reporting.
-  compile_cache_yaml:   true                  # Compile YAML into a cache
+  compile_cache_yaml:   true,                 # Compile YAML into a cache
+  readonly:             true,                 # Use the caches but don't update them on miss or stale entries.
 )
 ```
 
-**Note that `disable_trace` will break debuggers and tracing.**
-
 **Protip:** You can replace `require 'bootsnap'` with `BootLib::Require.from_gem('bootsnap',
 'bootsnap')` using [this trick](https://github.com/Shopify/bootsnap/wiki/Bootlib::Require). This
 will help optimize boot time further if you have an extremely large `$LOAD_PATH`.
@@ -72,12 +70,43 @@ speeds up the loading of individual source files, Spring keeps a copy of a pre-b
 on hand to completely skip parts of the boot process the next time it's needed. The two tools work
 well together, and are both included in a newly-generated Rails applications by default.
 
+### Environment variables
+
+`require 'bootsnap/setup'` behavior can be changed using environment variables:
+
+- `BOOTSNAP_CACHE_DIR` allows to define the cache location.
+- `DISABLE_BOOTSNAP` allows to entirely disable bootsnap.
+- `DISABLE_BOOTSNAP_LOAD_PATH_CACHE` allows to disable load path caching.
+- `DISABLE_BOOTSNAP_COMPILE_CACHE` allows to disable ISeq and YAML caches.
+- `BOOTSNAP_READONLY` configure bootsnap to not update the cache on miss or stale entries.
+- `BOOTSNAP_LOG` configure bootsnap to log all caches misses to STDERR.
+- `BOOTSNAP_IGNORE_DIRECTORIES` a comma separated list of directories that shouldn't be scanned.
+  Useful when you have large directories of non-ruby files inside `$LOAD_PATH`.
+  It default to ignore any directory named `node_modules`.
+
 ### Environments
 
 All Bootsnap features are enabled in development, test, production, and all other environments according to the configuration in the setup. At Shopify, we use this gem safely in all environments without issue.
 
 If you would like to disable any feature for a certain environment, we suggest changing the configuration to take into account the appropriate ENV var or configuration according to your needs.
 
+### Instrumentation
+
+Bootsnap cache misses can be monitored though a callback:
+
+```ruby
+Bootsnap.instrumentation = ->(event, path) { puts "#{event} #{path}" }
+```
+
+`event` is either `:miss` or `:stale`. You can also call `Bootsnap.log!` as a shortcut to
+log all events to STDERR.
+
+To turn instrumentation back off you can set it to nil:
+
+```ruby
+Bootsnap.instrumentation = nil
+```
+
 ## How does this work?
 
 Bootsnap optimizes methods to cache results of expensive computations, and can be grouped
@@ -85,8 +114,6 @@ into two broad categories:
 
 * [Path Pre-Scanning](#path-pre-scanning)
     * `Kernel#require` and `Kernel#load` are modified to eliminate `$LOAD_PATH` scans.
-    * `ActiveSupport::Dependencies.{autoloadable_module?,load_missing_constant,depend_on}` are
-      overridden to eliminate scans of `ActiveSupport::Dependencies.autoload_paths`.
 * [Compilation caching](#compilation-caching)
     * `RubyVM::InstructionSequence.load_iseq` is implemented to cache the result of ruby bytecode
       compilation.
@@ -125,10 +152,6 @@ open y/foo.rb
 ...
 ```
 
-Exactly the same strategy is employed for methods that traverse
-`ActiveSupport::Dependencies.autoload_paths` if the `autoload_paths_cache` option is given to
-`Bootsnap.setup`.
-
 The following diagram flowcharts the overrides that make the `*_path_cache` features work.
 
 ![Flowchart explaining
@@ -144,7 +167,7 @@ The only directories considered "stable" are things under the Ruby install prefi
 "volatile".
 
 In addition to the [`Bootsnap::LoadPathCache::Cache`
-source](https://github.com/Shopify/bootsnap/blob/master/lib/bootsnap/load_path_cache/cache.rb),
+source](https://github.com/Shopify/bootsnap/blob/main/lib/bootsnap/load_path_cache/cache.rb),
 this diagram may help clarify how entry resolution works:
 
 ![How path searching works](https://cloud.githubusercontent.com/assets/3074765/25388270/670b5652-299b-11e7-87fb-975647f68981.png)
@@ -215,9 +238,9 @@ Bootsnap writes a cache file containing a 64 byte header followed by the cache c
 is a cache key including several fields:
 
 * `version`, hardcoded in bootsnap. Essentially a schema version;
-* `ruby_platform`, A hash of `RUBY_PLATFORM` (e.g. x86_64-linux-gnu) variable and glibc version (on Linux) or OS version (`uname -v` on BSD, macOS)
+* `ruby_platform`, A hash of `RUBY_PLATFORM` (e.g. x86_64-linux-gnu) variable.
 * `compile_option`, which changes with `RubyVM::InstructionSequence.compile_option` does;
-* `ruby_revision`, the version of Ruby this was compiled with;
+* `ruby_revision`, A hash of `RUBY_REVISION`, the exact version of Ruby;
 * `size`, the size of the source file;
 * `mtime`, the last-modification timestamp of the source file when it was compiled; and
 * `data_size`, the number of bytes following the header, which we need to read it into a buffer.

data/exe/bootsnap

@@ -1,5 +1,5 @@
 #!/usr/bin/env ruby
 # frozen_string_literal: true
 
-require 'bootsnap/cli'
+require "bootsnap/cli"
 exit Bootsnap::CLI.new(ARGV).run