bootsnap 1.13.0 → 1.24.6

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 40e594cf459a80b9e118cf8cdb6d9f306228fe682e3c2af3a1c2b70432f47890
-  data.tar.gz: c120e3c24944e9c0568615a0b6ec23d422e949755339c8ffa118f0265e85755b
+  metadata.gz: c89c653bbd5db473e06179c6606268271c8c540a588cca938662f168d64f2d39
+  data.tar.gz: 449c08c79a258dea90e56646c9d62a06e1d696c065ce96c38a7f50ef2fb83f7b
 SHA512:
-  metadata.gz: 45c52711bd2fbf273f93d27ea4a8ee71d04c81b7444f0f49aa9b672e131544103b74d3326afe5228122dc52844b681c1e18bbb59fe429b2a2216e182e90ac5e7
-  data.tar.gz: ad78b4da53ffc8d6653f74ed777bae1987704de2a76062bbebc2b1d3977baeca900b36066994957adf3cefbda446b4393dd18e41637d1a39f9adac42878b6732
+  metadata.gz: 2ea80faffec0206b9bb7b3d19c1419b2e7eefc2dc5f4eb03352862b83afa07ed293d0b15f04fc069d89dac1cd4d4636770626e90e6129ffad77f0cfcb8c3f676
+  data.tar.gz: 82dcb1a4c934a9de29a8aa935ffe2d1a7c6e23ebf36ea743c3d29a4d9a1d8bd42696cdefd37dacdd21a4102f88f0b605d1cc6faef9c15ccdc6d99eef15f4483e

data/CHANGELOG.md

@@ -1,5 +1,148 @@
 # Unreleased
 
+# 1.24.6
+
+* Fix detection of Ruby bug #22023 on some patch versions of Ruby 3.4, and properly apply the workaround.
+
+# 1.24.5
+
+* No longer load the config file by default when setup is done manually. This is so cli applications like homebrew
+  don't mistakenly load another app's boostnap config.
+
+# 1.24.4
+
+* Fix several compatibility issues with Ruby `4.0.4`, particularly the `should not compile with coverage` error. See #547.
+* Fix `Bootsnap.enable_frozen_string_literal` to work even when coverage is enabled. Unfortunately only possible on Ruby `4.0.4+`.
+  On older rubies if coverage is enabled a warning will be issued and the feature won't work.
+* Reduced cache files header size from 64 to 32 bytes, and got rid of the random padding element.
+* Avoid leaking a private method in `Object` when testing for Parse.y bugs.
+
+# 1.24.3
+
+* Fix the `1.24.2` workaround to parse Ruby files with UTF-8 even when the `LANG` environment variable
+  is unset or set to `C`.
+
+# 1.24.2
+
+* Workaround two Ruby bugs in `RubyVM::InstructionSequence.compile_file`, that were causing
+  files to be loaded with the old Ruby parser instead of Prism, causing issues with some pattern matching syntax.
+  Ref: https://bugs.ruby-lang.org/issues/22023
+
+# 1.24.1
+
+* Fix encoding of Ruby source files loaded when `BOOTSNAP_READONLY` is set.
+  Files would incorectly be loaded in `ASCII-8BIT` causing literal strings outside
+  the pure ASCII range to have `ASCII-8BIT` encoding instead of `UTF-8`.
+  This bug was introduced in `1.24.0`.
+
+# 1.24.0
+
+* Added a hook API to customize Ruby compilation.
+
+# 1.23.0
+
+* Require Ruby 2.7.
+* Fix support for absolute paths in `BOOTSNAP_IGNORE_DIRECTORIES`.
+
+# 1.22.0
+
+* Better fix for the `opendir` crash.
+* Add `bootsnap/rake` for cleaning the bootsnap cache as part of `rake clobber`.
+
+# 1.21.1
+
+* Prevent a Ruby crash while scanning load path if `opendir` fails without setting `errno`.
+  According to the C spec this should not happen, but according to user reports, it did.
+
+# 1.21.0
+
+* Fix the `require` decorator to handle `Bootsnap.unload_cache!` being called.
+* Minor optimization: Eagerly clear cache buffers to appease the GC.
+
+# 1.20.1
+
+* Handle broken symlinks in load path scanning code.
+  Should fix `Errno::ENOENT fstatat` issues some users have encountered after upgrading to 1.20.0.
+
+# 1.20.0
+
+* Optimized load path scanning with a C extension. Should be about 2x faster on supported platforms.
+
+# 1.19.0
+
+* Remove JSON parsing cache. Recent versions of the `json` gem are as fast as `msgpack` if not faster.
+
+# 1.18.6
+
+* Fix cgroup CPU limits detection in CLI.
+
+# 1.18.5
+
+* Attempt to detect a QEMU bug that can cause `bootsnap precompile` to hang forever when building ARM64 docker images
+  from x86_64 machines. See #495.
+* Improve CLI to detect cgroup CPU limits and avoid spawning too many worker processes.
+
+# 1.18.4
+
+* Allow using bootsnap without bundler. See #488.
+* Fix startup failure if the cache directory points to a broken symlink.
+
+# 1.18.3
+
+* Fix the cache corruption issue in the revalidation feature. See #474.
+  The cache revalidation feature remains opt-in for now, until it is more battle tested.
+
+# 1.18.2
+
+* Disable stale cache entries revalidation by default as it seems to cause cache corruption issues. See #471 and #474.
+  Will be re-enabled in a future version once the root cause is identified.
+* Fix a potential compilation issue on some systems. See #470.
+
+# 1.18.1
+
+* Handle `EPERM` errors when opening files with `O_NOATIME`.
+
+# 1.18.0
+
+* `Bootsnap.instrumentation` now receive `:hit` events.
+* Add `Bootsnap.log_stats!` to print hit rate statistics on process exit. Can also be enabled with `BOOTSNAP_STATS=1`.
+* Revalidate stale cache entries by digesting the source content.
+  This should significantly improve performance in environments where `mtime` isn't preserved (e.g. CI systems doing a git clone, etc).
+  See #468.
+* Open source files and cache entries with `O_NOATIME` when available to reduce disk accesses. See #469.
+* `bootsnap precompile --gemfile` now look for `.rb` files in the whole gem and not just the `lib/` directory. See #466.
+
+# 1.17.1
+
+* Fix a compatibility issue with the `prism` library that ships with Ruby 3.3. See #463.
+* Improved the `Kernel#require` decorator to not cause a method redefinition warning. See #461.
+
+# 1.17.0
+
+* Ensure `$LOAD_PATH.dup` is Ractor shareable to fix a conflict with `did_you_mean`.
+* Allow to ignore directories using absolute paths.
+* Support YAML and JSON CompileCache on TruffleRuby.
+* Support LoadPathCache on TruffleRuby.
+
+# 1.16.0
+
+* Use `RbConfig::CONFIG["rubylibdir"]` instead of `RbConfig::CONFIG["libdir"]` to check for stdlib files. See #431.
+* Fix the cached version of `YAML.load_file` being slightly more permissive than the default `Psych` one. See #434.
+  `Date` and `Time` values are now properly rejected, as well as aliases.
+  If this causes a regression in your application, it is recommended to load *trusted* YAML files with `YAML.unsafe_load_file`.
+
+# 1.15.0
+
+* Add a readonly mode, for environments in which the updated cache wouldn't be persisted. See #428 and #423.
+
+# 1.14.0
+
+* Require Ruby 2.6.
+* Add a way to skip directories during load path scanning.
+  If you have large non-ruby directories in the middle of your load path, it can severely slow down scanning.
+  Typically this is a problem with `node_modules`. See #277.
+* Fix `Bootsnap.unload_cache!`, it simply wouldn't work at all because of a merge mistake. See #421.
+
 # 1.13.0
 
 * Stop decorating `Kernel.load`. This used to be very useful in development because the Rails "classic" autoloader
@@ -17,7 +160,7 @@
 
 * Stop decorating `Module#autoload` as it was only useful for supporting Ruby 2.2 and older.
 
-* Remove `uname` and other patform specific version from the cache keys. `RUBY_PLATFORM + RUBY_REVISION` should be
+* Remove `uname` and other platform specific version from the cache keys. `RUBY_PLATFORM + RUBY_REVISION` should be
   enough to ensure bytecode compatibility. This should improve caching for alpine based setups. See #409.
 
 # 1.11.1
@@ -34,7 +177,7 @@
 
 * Get rid of the `Kernel.require_relative` decorator by resolving `$LOAD_PATH` members to their real path.
   This way we handle symlinks in `$LOAD_PATH` much more efficiently. See #402 for the detailed explanation.
-  
+
 * Drop support for Ruby 2.3 (to allow getting rid of the `Kernel.require_relative` decorator).
 
 # 1.10.3
@@ -160,7 +303,7 @@
 * Adds an instrumentation API to monitor cache misses.
 * Allow to control the behavior of `require 'bootsnap/setup'` using environment variables.
 * Deprecate the `disable_trace` option.
-* Deprecate the `ActiveSupport::Dependencies` (AKA Classic autoloader) integration. (#344) 
+* Deprecate the `ActiveSupport::Dependencies` (AKA Classic autoloader) integration. (#344)
 
 # 1.6.0
 
@@ -180,12 +323,12 @@
 
 # 1.4.9
 
-* [Windows support](https://github.com/Shopify/bootsnap/pull/319)
-* [Fix potential crash](https://github.com/Shopify/bootsnap/pull/322)
+* [Windows support](https://github.com/rails/bootsnap/pull/319)
+* [Fix potential crash](https://github.com/rails/bootsnap/pull/322)
 
 # 1.4.8
 
-* [Prevent FallbackScan from polluting exception cause](https://github.com/Shopify/bootsnap/pull/314)
+* [Prevent FallbackScan from polluting exception cause](https://github.com/rails/bootsnap/pull/314)
 
 # 1.4.7
 
@@ -198,7 +341,7 @@
   required if a different file with the same name was already being required
 
   Example:
-  
+
       require 'foo'
       require 'foo.en'
 
@@ -252,7 +395,7 @@
 
 # 1.3.0
 
-* Handle cases where load path entries are symlinked (https://github.com/Shopify/bootsnap/pull/136)
+* Handle cases where load path entries are symlinked (https://github.com/rails/bootsnap/pull/136)
 
 # 1.2.1
 

data/LICENSE.txt

@@ -1,6 +1,7 @@
 The MIT License (MIT)
 
-Copyright (c) 2017-present Shopify, Inc.
+Copyright (c) 2017-2025 Shopify, Inc.
+Copyright (c) 2025-present Rails Foundation
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

data/README.md

@@ -1,6 +1,6 @@
-# Bootsnap [![Actions Status](https://github.com/Shopify/bootsnap/workflows/ci/badge.svg)](https://github.com/Shopify/bootsnap/actions)
+# Bootsnap [![Actions Status](https://github.com/rails/bootsnap/workflows/ci/badge.svg)](https://github.com/rails/bootsnap/actions)
 
-Bootsnap is a library that plugs into Ruby, with optional support for `YAML`,
+Bootsnap is a library that plugs into Ruby, with optional support for `YAML` and `JSON`,
 to optimize and cache expensive computations. See [How Does This Work](#how-does-this-work).
 
 #### Performance
@@ -41,7 +41,7 @@ getting progressively slower, this is almost certainly the cause.**
 It's technically possible to simply specify `gem 'bootsnap', require: 'bootsnap/setup'`, but it's
 important to load Bootsnap as early as possible to get maximum performance improvement.
 
-You can see how this require works [here](https://github.com/Shopify/bootsnap/blob/main/lib/bootsnap/setup.rb).
+You can see how this require works [here](https://github.com/rails/bootsnap/blob/main/lib/bootsnap/setup.rb).
 
 If you are not using Rails, or if you are but want more control over things, add this to your
 application setup immediately after `require 'bundler/setup'` (i.e. as early as possible: the sooner
@@ -52,31 +52,39 @@ require 'bootsnap'
 env = ENV['RAILS_ENV'] || "development"
 Bootsnap.setup(
   cache_dir:            'tmp/cache',          # Path to your cache
+  ignore_directories:   ['node_modules'],     # Directory names to skip.
   development_mode:     env == 'development', # Current working environment, e.g. RACK_ENV, RAILS_ENV, etc
   load_path_cache:      true,                 # Optimize the LOAD_PATH with a cache
   compile_cache_iseq:   true,                 # Compile Ruby code into ISeq cache, breaks coverage reporting.
-  compile_cache_yaml:   true                  # Compile YAML into a cache
+  compile_cache_yaml:   true,                 # Compile YAML into a cache
+  readonly:             true,                 # Use the caches but don't update them on miss or stale entries.
 )
 ```
 
 **Protip:** You can replace `require 'bootsnap'` with `BootLib::Require.from_gem('bootsnap',
-'bootsnap')` using [this trick](https://github.com/Shopify/bootsnap/wiki/Bootlib::Require). This
+'bootsnap')` using [this trick](https://github.com/rails/bootsnap/wiki/Bootlib::Require). This
 will help optimize boot time further if you have an extremely large `$LOAD_PATH`.
 
 Note: Bootsnap and [Spring](https://github.com/rails/spring) are orthogonal tools. While Bootsnap
 speeds up the loading of individual source files, Spring keeps a copy of a pre-booted Rails process
 on hand to completely skip parts of the boot process the next time it's needed. The two tools work
-well together, and are both included in a newly-generated Rails applications by default.
+well together.
 
 ### Environment variables
 
 `require 'bootsnap/setup'` behavior can be changed using environment variables:
 
 - `BOOTSNAP_CACHE_DIR` allows to define the cache location.
+- `BOOTSNAP_CONFIG` allows to change the default config location (`config/bootsnap.rb`).
 - `DISABLE_BOOTSNAP` allows to entirely disable bootsnap.
 - `DISABLE_BOOTSNAP_LOAD_PATH_CACHE` allows to disable load path caching.
 - `DISABLE_BOOTSNAP_COMPILE_CACHE` allows to disable ISeq and YAML caches.
+- `BOOTSNAP_READONLY` configure bootsnap to not update the cache on miss or stale entries.
 - `BOOTSNAP_LOG` configure bootsnap to log all caches misses to STDERR.
+- `BOOTSNAP_STATS` log hit rate statistics on exit. Can't be used if `BOOTSNAP_LOG` is enabled.
+- `BOOTSNAP_IGNORE_DIRECTORIES` a comma separated list of directories that shouldn't be scanned.
+  Useful when you have large directories of non-ruby files inside `$LOAD_PATH`.
+  It defaults to ignore any directory named `node_modules`.
 
 ### Environments
 
@@ -92,8 +100,8 @@ Bootsnap cache misses can be monitored though a callback:
 Bootsnap.instrumentation = ->(event, path) { puts "#{event} #{path}" }
 ```
 
-`event` is either `:miss` or `:stale`. You can also call `Bootsnap.log!` as a shortcut to
-log all events to STDERR.
+`event` is either `:hit`, `:miss`, `:stale` or `:revalidated`.
+You can also call `Bootsnap.log!` as a shortcut to log all events to STDERR.
 
 To turn instrumentation back off you can set it to nil:
 
@@ -113,6 +121,7 @@ into two broad categories:
       compilation.
     * `YAML.load_file` is modified to cache the result of loading a YAML object in MessagePack format
       (or Marshal, if the message uses types unsupported by MessagePack).
+    * `JSON.load_file` is modified to cache the result of loading a JSON object in MessagePack format
 
 ### Path Pre-Scanning
 
@@ -161,7 +170,7 @@ The only directories considered "stable" are things under the Ruby install prefi
 "volatile".
 
 In addition to the [`Bootsnap::LoadPathCache::Cache`
-source](https://github.com/Shopify/bootsnap/blob/main/lib/bootsnap/load_path_cache/cache.rb),
+source](https://github.com/rails/bootsnap/blob/main/lib/bootsnap/load_path_cache/cache.rb),
 this diagram may help clarify how entry resolution works:
 
 ![How path searching works](https://cloud.githubusercontent.com/assets/3074765/25388270/670b5652-299b-11e7-87fb-975647f68981.png)
@@ -179,13 +188,13 @@ result too, raising a `LoadError` without touching the filesystem at all.
 
 Ruby has complex grammar and parsing it is not a particularly cheap operation. Since 1.9, Ruby has
 translated ruby source to an internal bytecode format, which is then executed by the Ruby VM. Since
-2.3.0, Ruby [exposes an API](https://ruby-doc.org/core-2.3.0/RubyVM/InstructionSequence.html) that
+2.3.0, Ruby [exposes an API](https://docs.ruby-lang.org/en/master/RubyVM/InstructionSequence.html) that
 allows caching that bytecode. This allows us to bypass the relatively-expensive compilation step on
 subsequent loads of the same file.
 
-We also noticed that we spend a lot of time loading YAML documents during our application boot, and
-that MessagePack and Marshal are *much* faster at deserialization than YAML, even with a fast
-implementation. We use the same strategy of compilation caching for YAML documents, with the
+We also noticed that we spend a lot of time loading YAML and JSON documents during our application boot, and
+that MessagePack and Marshal are *much* faster at deserialization than YAML and JSON, even with a fast
+implementation. We use the same strategy of compilation caching for YAML and JSON documents, with the
 equivalent of Ruby's "bytecode" format being a MessagePack document (or, in the case of YAML
 documents with types unsupported by MessagePack, a Marshal stream).
 
@@ -228,14 +237,15 @@ This may look worse at a glance, but underlies a large performance difference.
 useful. [This ruby patch](https://bugs.ruby-lang.org/issues/13378) optimizes them out when coupled
 with bootsnap.)*
 
-Bootsnap writes a cache file containing a 64 byte header followed by the cache contents. The header
+Bootsnap writes a cache file containing a 32 byte header followed by the cache contents. The header
 is a cache key including several fields:
 
-* `version`, hardcoded in bootsnap. Essentially a schema version;
-* `ruby_platform`, A hash of `RUBY_PLATFORM` (e.g. x86_64-linux-gnu) variable.
-* `compile_option`, which changes with `RubyVM::InstructionSequence.compile_option` does;
-* `ruby_revision`, A hash of `RUBY_REVISION`, the exact version of Ruby;
+* `ruby_version_digest`, a digest of:
+  * The `RUBY_DESCRIPTION` constant e.g. `"ruby 4.0.2 (2026-03-17 revision d3da9fec82) +PRISM [arm64-darwin25]"`.
+  * Bootsnap's cache version. Hardcoded in bootsnap. Essentially a schema version;
+  * The content of `RubyVM::InstructionSequence.compile_option`.
 * `size`, the size of the source file;
+* `digest`, a fnv1a_64 hash of the source file;
 * `mtime`, the last-modification timestamp of the source file when it was compiled; and
 * `data_size`, the number of bytes following the header, which we need to read it into a buffer.
 
@@ -312,6 +322,38 @@ open    /c/nope.bundle -> -1
 # (nothing!)
 ```
 
+## Custom Compilers
+
+Bootsnap allows substituing the default Ruby compiler by another one.
+This can be configured from the bootsnap config file (defaults to `config/bootsnap.rb`).
+
+The main use case is to programmatically enable frozen string literals for your project without impacting dependencies:
+
+```ruby
+Bootsnap.enable_frozen_string_literal(app_only: true)
+```
+
+But it can also be used for more fine grained logic, or to implement all sort of Ruby code preprocessing:
+
+```ruby
+# config/bootsnap.rb
+gems_root = File.join(Bundler.bundle_path.cleanpath, "")
+app_root =  File.join(Dir.pwd, "")
+Bootsnap::CompileCache::ISeq.compiler_selector = ->(path) do
+  # Enable `frozen_string_literal: true` for app code, but not gems.
+
+  if path.start_with?(app_root) && !path.start_with?(gems_root)
+    Bootsnap::CompileCache::ISeq::FROZEN_STRING_LITERAL
+  else
+    Bootsnap::CompileCache::ISeq::DEFAULT
+  end
+end
+```
+
+*Important note*: This feature is only fully supported by Ruby `4.0.4` and newer.
+On older rubies the feature work except if the `Coverage` module is enabled, in which case a warning will be emitted
+and the default Ruby compiler will be used.
+
 ## Precompilation
 
 In development environments the bootsnap compilation cache is generated on the fly when source files are loaded.
@@ -322,9 +364,23 @@ To do so you can use the `bootsnap precompile` command.
 Example:
 
 ```bash
-$ bundle exec bootsnap precompile --gemfile app/ lib/
+$ bundle exec bootsnap precompile --gemfile app/ lib/ config/
 ```
 
+## Known issues
+
+### QEMU environments
+
+When building cross-platform Docker images, QEMU is often used for emulation and can be the source of a limitation that causes forked processes to hang. While Bootsnap includes automatic detection for this issue (as of [PR #501](https://github.com/rails/bootsnap/pull/501)), the detection may not always be sufficient.
+
+If you encounter hangs during precompilation in QEMU-based environments (such as when using Docker buildx for cross-platform builds), you can work around this by disabling parallelization with the `-j 0` option:
+
+```bash
+$ bundle exec bootsnap precompile -j 0 --gemfile app/ lib/ config/
+```
+
+See [Issue #495](https://github.com/rails/bootsnap/issues/495) for more details about this QEMU-related issue.
+
 ## When not to use Bootsnap
 
 *Alternative engines*: Bootsnap is pretty reliant on MRI features, and parts are disabled entirely on alternative ruby