bootsnap 1.6.0 → 1.7.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: '029d63ba428f470d2cd2ef194d857e20689e7c8e377e2eaaaab83570f366034a'
-  data.tar.gz: 36a55f9d12dddef6ea3c4739f586c9236d7f4bc677a8b6747dfd7465d46eeca2
+  metadata.gz: 7b27cdcd835cd67efea9340d2e09899a3fa553ded267992672183d3bce9cd7de
+  data.tar.gz: 73d228cbe15a69fb1a26d76f6375fe564f4de4db6e7f111e6a4d0ec31e09b0a1
 SHA512:
-  metadata.gz: 131ec17c4e4912f387c18778250e689467ebfcc80e91eb884237307af1a57a3c89d4fb20c772fbc0330123a796d631861a9cf145bd32c54183077bbc97d7fa6f
-  data.tar.gz: d0c92454c8a5d8b16a25908fd0ae134fc817c5977958bde8424baca2bb6c96e60eb648c9f770bf7c7f58a3dd98871d4e7b0e3a0950c269100fded45ca9fddfa3
+  metadata.gz: e967771cc3387e0297d1654a824f7406e2db1395766dc6df5d2069b78129a71737a497e4681c8988f62ce308e646f967137929266004dbc0e3fbf212da53d3f0
+  data.tar.gz: ba6e782a7c4da7d8762cea5501359077b66b68f6b0cbb00e8533557cfd5194ef71bc5016c2273a8ba6ad0aa8e5f434f449c3d0cc8252fde215032807cfd75a9f

data/CHANGELOG.md

@@ -1,5 +1,27 @@
 # Unreleased
 
+# 1.7.3
+
+* Disable YAML precompilation when encountering YAML tags. (#351)
+
+# 1.7.2
+
+* Fix compatibility with msgpack < 1. (#349)
+
+# 1.7.1
+
+* Warn Ruby 2.5 users if they turn ISeq caching on. (#327, #244)
+* Disable ISeq caching for the whole 2.5.x series again.
+* Better handle hashing of Ruby strings. (#318)
+
+# 1.7.0
+
+* Fix detection of YAML files in gems.
+* Adds an instrumentation API to monitor cache misses.
+* Allow to control the behavior of `require 'bootsnap/setup'` using environment variables.
+* Deprecate the `disable_trace` option.
+* Deprecate the `ActiveSupport::Dependencies` (AKA Classic autoloader) integration. (#344) 
+
 # 1.6.0
 
 * Fix a Ruby 2.7/3.0 issue with `YAML.load_file` keyword arguments. (#342)

data/README.md

@@ -1,6 +1,6 @@
 # Bootsnap [![Actions Status](https://github.com/Shopify/bootsnap/workflows/ci/badge.svg)](https://github.com/Shopify/bootsnap/actions)
 
-Bootsnap is a library that plugs into Ruby, with optional support for `ActiveSupport` and `YAML`,
+Bootsnap is a library that plugs into Ruby, with optional support for `YAML`,
 to optimize and cache expensive computations. See [How Does This Work](#how-does-this-work).
 
 #### Performance
@@ -11,7 +11,7 @@ to optimize and cache expensive computations. See [How Does This Work](#how-does
 - The core Shopify platform -- a rather large monolithic application -- boots about 75% faster,
   dropping from around 25s to 6.5s.
 * In Shopify core (a large app), about 25% of this gain can be attributed to `compile_cache_*`
-  features; 75% to path caching, and ~1% to `disable_trace`. This is fairly representative.
+  features; 75% to path caching. This is fairly representative.
 
 ## Usage
 
@@ -29,7 +29,7 @@ If you are using Rails, add this to `config/boot.rb` immediately after `require
 require 'bootsnap/setup'
 ```
 
-Note that bootsnap writes to `tmp/cache` (or the path specified by `ENV['BOOTSNAP_CACHE_DIR']`), 
+Note that bootsnap writes to `tmp/cache` (or the path specified by `ENV['BOOTSNAP_CACHE_DIR']`),
 and that directory *must* be writable. Rails will fail to
 boot if it is not. If this is unacceptable (e.g. you are running in a read-only container and
 unwilling to mount in a writable tmpdir), you should remove this line or wrap it in a conditional.
@@ -54,15 +54,11 @@ Bootsnap.setup(
   cache_dir:            'tmp/cache',          # Path to your cache
   development_mode:     env == 'development', # Current working environment, e.g. RACK_ENV, RAILS_ENV, etc
   load_path_cache:      true,                 # Optimize the LOAD_PATH with a cache
-  autoload_paths_cache: true,                 # Optimize ActiveSupport autoloads with cache
-  disable_trace:        true,                 # Set `RubyVM::InstructionSequence.compile_option = { trace_instruction: false }`
   compile_cache_iseq:   true,                 # Compile Ruby code into ISeq cache, breaks coverage reporting.
   compile_cache_yaml:   true                  # Compile YAML into a cache
 )
 ```
 
-**Note that `disable_trace` will break debuggers and tracing.**
-
 **Protip:** You can replace `require 'bootsnap'` with `BootLib::Require.from_gem('bootsnap',
 'bootsnap')` using [this trick](https://github.com/Shopify/bootsnap/wiki/Bootlib::Require). This
 will help optimize boot time further if you have an extremely large `$LOAD_PATH`.
@@ -72,12 +68,39 @@ speeds up the loading of individual source files, Spring keeps a copy of a pre-b
 on hand to completely skip parts of the boot process the next time it's needed. The two tools work
 well together, and are both included in a newly-generated Rails applications by default.
 
+### Environment variables
+
+`require 'bootsnap/setup'` behavior can be changed using environment variables:
+
+- `BOOTSNAP_CACHE_DIR` allows to define the cache location.
+- `DISABLE_BOOTSNAP` allows to entirely disable bootsnap.
+- `DISABLE_BOOTSNAP_LOAD_PATH_CACHE` allows to disable load path caching.
+- `DISABLE_BOOTSNAP_COMPILE_CACHE` allows to disable ISeq and YAML caches.
+- `BOOTSNAP_LOG` configure bootsnap to log all caches misses to STDERR.
+
 ### Environments
 
 All Bootsnap features are enabled in development, test, production, and all other environments according to the configuration in the setup. At Shopify, we use this gem safely in all environments without issue.
 
 If you would like to disable any feature for a certain environment, we suggest changing the configuration to take into account the appropriate ENV var or configuration according to your needs.
 
+### Instrumentation
+
+Bootsnap cache misses can be monitored though a callback:
+
+```ruby
+Bootsnap.instrumentation = ->(event, path) { puts "#{event} #{path}" }
+```
+
+`event` is either `:miss` or `:stale`. You can also call `Bootsnap.log!` as a shortcut to
+log all events to STDERR.
+
+To turn instrumentation back off you can set it to nil:
+
+```ruby
+Bootsnap.instrumentation = nil
+```
+
 ## How does this work?
 
 Bootsnap optimizes methods to cache results of expensive computations, and can be grouped
@@ -85,8 +108,6 @@ into two broad categories:
 
 * [Path Pre-Scanning](#path-pre-scanning)
     * `Kernel#require` and `Kernel#load` are modified to eliminate `$LOAD_PATH` scans.
-    * `ActiveSupport::Dependencies.{autoloadable_module?,load_missing_constant,depend_on}` are
-      overridden to eliminate scans of `ActiveSupport::Dependencies.autoload_paths`.
 * [Compilation caching](#compilation-caching)
     * `RubyVM::InstructionSequence.load_iseq` is implemented to cache the result of ruby bytecode
       compilation.
@@ -125,10 +146,6 @@ open y/foo.rb
 ...
 ```
 
-Exactly the same strategy is employed for methods that traverse
-`ActiveSupport::Dependencies.autoload_paths` if the `autoload_paths_cache` option is given to
-`Bootsnap.setup`.
-
 The following diagram flowcharts the overrides that make the `*_path_cache` features work.
 
 ![Flowchart explaining

data/ext/bootsnap/bootsnap.c

@@ -14,6 +14,7 @@
 #include "bootsnap.h"
 #include "ruby.h"
 #include <stdint.h>
+#include <stdbool.h>
 #include <sys/types.h>
 #include <errno.h>
 #include <fcntl.h>
@@ -34,6 +35,10 @@
 
 #define MAX_CREATE_TEMPFILE_ATTEMPT 3
 
+#ifndef RB_UNLIKELY
+  #define RB_UNLIKELY(x) (x)
+#endif
+
 /*
  * An instance of this key is written as the first 64 bytes of each cache file.
  * The mtime and size members track whether the file contents have changed, and
@@ -88,15 +93,19 @@ static VALUE rb_mBootsnap_CompileCache;
 static VALUE rb_mBootsnap_CompileCache_Native;
 static VALUE rb_eBootsnap_CompileCache_Uncompilable;
 static ID uncompilable;
+static ID instrumentation_method;
+static VALUE sym_miss;
+static VALUE sym_stale;
+static bool instrumentation_enabled = false;
 
 /* Functions exposed as module functions on Bootsnap::CompileCache::Native */
+static VALUE bs_instrumentation_enabled_set(VALUE self, VALUE enabled);
 static VALUE bs_compile_option_crc32_set(VALUE self, VALUE crc32_v);
 static VALUE bs_rb_fetch(VALUE self, VALUE cachedir_v, VALUE path_v, VALUE handler, VALUE args);
 static VALUE bs_rb_precompile(VALUE self, VALUE cachedir_v, VALUE path_v, VALUE handler);
 
 /* Helpers */
-static uint64_t fnv1a_64(const char *str);
-static void bs_cache_path(const char * cachedir, const char * path, char (* cache_path)[MAX_CACHEPATH_SIZE]);
+static void bs_cache_path(const char * cachedir, const VALUE path, char (* cache_path)[MAX_CACHEPATH_SIZE]);
 static int bs_read_key(int fd, struct bs_cache_key * key);
 static int cache_key_equal(struct bs_cache_key * k1, struct bs_cache_key * k2);
 static VALUE bs_fetch(char * path, VALUE path_v, char * cache_path, VALUE handler, VALUE args);
@@ -148,7 +157,15 @@ Init_bootsnap(void)
   current_ruby_platform = get_ruby_platform();
 
   uncompilable = rb_intern("__bootsnap_uncompilable__");
+  instrumentation_method = rb_intern("_instrument");
+
+  sym_miss = ID2SYM(rb_intern("miss"));
+  rb_global_variable(&sym_miss);
 
+  sym_stale = ID2SYM(rb_intern("stale"));
+  rb_global_variable(&sym_stale);
+
+  rb_define_module_function(rb_mBootsnap, "instrumentation_enabled=", bs_instrumentation_enabled_set, 1);
   rb_define_module_function(rb_mBootsnap_CompileCache_Native, "coverage_running?", bs_rb_coverage_running, 0);
   rb_define_module_function(rb_mBootsnap_CompileCache_Native, "fetch", bs_rb_fetch, 4);
   rb_define_module_function(rb_mBootsnap_CompileCache_Native, "precompile", bs_rb_precompile, 3);
@@ -158,6 +175,13 @@ Init_bootsnap(void)
   umask(current_umask);
 }
 
+static VALUE
+bs_instrumentation_enabled_set(VALUE self, VALUE enabled)
+{
+  instrumentation_enabled = RTEST(enabled);
+  return enabled;
+}
+
 /*
  * Bootsnap's ruby code registers a hook that notifies us via this function
  * when compile_option changes. These changes invalidate all existing caches.
@@ -186,7 +210,7 @@ bs_compile_option_crc32_set(VALUE self, VALUE crc32_v)
  *   - 32 bits doesn't feel collision-resistant enough; 64 is nice.
  */
 static uint64_t
-fnv1a_64_iter(uint64_t h, const char *str)
+fnv1a_64_iter_cstr(uint64_t h, const char *str)
 {
   unsigned char *s = (unsigned char *)str;
 
@@ -199,7 +223,21 @@ fnv1a_64_iter(uint64_t h, const char *str)
 }
 
 static uint64_t
-fnv1a_64(const char *str)
+fnv1a_64_iter(uint64_t h, const VALUE str)
+{
+  unsigned char *s = (unsigned char *)RSTRING_PTR(str);
+  unsigned char *str_end = (unsigned char *)RSTRING_PTR(str) + RSTRING_LEN(str);
+
+  while (s < str_end) {
+    h ^= (uint64_t)*s++;
+    h += (h << 1) + (h << 4) + (h << 5) + (h << 7) + (h << 8) + (h << 40);
+  }
+
+  return h;
+}
+
+static uint64_t
+fnv1a_64(const VALUE str)
 {
   uint64_t h = (uint64_t)0xcbf29ce484222325ULL;
   return fnv1a_64_iter(h, str);
@@ -220,7 +258,7 @@ get_ruby_revision(void)
   } else {
     uint64_t hash;
 
-    hash = fnv1a_64(StringValueCStr(ruby_revision));
+    hash = fnv1a_64(ruby_revision);
     return (uint32_t)(hash >> 32);
   }
 }
@@ -240,19 +278,19 @@ get_ruby_platform(void)
   VALUE ruby_platform;
 
   ruby_platform = rb_const_get(rb_cObject, rb_intern("RUBY_PLATFORM"));
-  hash = fnv1a_64(RSTRING_PTR(ruby_platform));
+  hash = fnv1a_64(ruby_platform);
 
 #ifdef _WIN32
   return (uint32_t)(hash >> 32) ^ (uint32_t)GetVersion();
 #elif defined(__GLIBC__)
-  hash = fnv1a_64_iter(hash, gnu_get_libc_version());
+  hash = fnv1a_64_iter_cstr(hash, gnu_get_libc_version());
   return (uint32_t)(hash >> 32);
 #else
   struct utsname utsname;
 
   /* Not worth crashing if this fails; lose extra cache invalidation potential */
   if (uname(&utsname) >= 0) {
-    hash = fnv1a_64_iter(hash, utsname.version);
+    hash = fnv1a_64_iter_cstr(hash, utsname.version);
   }
 
   return (uint32_t)(hash >> 32);
@@ -267,13 +305,13 @@ get_ruby_platform(void)
  * The path will look something like: <cachedir>/12/34567890abcdef
  */
 static void
-bs_cache_path(const char * cachedir, const char * path, char (* cache_path)[MAX_CACHEPATH_SIZE])
+bs_cache_path(const char * cachedir, const VALUE path, char (* cache_path)[MAX_CACHEPATH_SIZE])
 {
   uint64_t hash = fnv1a_64(path);
   uint8_t first_byte = (hash >> (64 - 8));
   uint64_t remainder = hash & 0x00ffffffffffffff;
 
-  sprintf(*cache_path, "%s/%02x/%014llx", cachedir, first_byte, remainder);
+  sprintf(*cache_path, "%s/%02"PRIx8"/%014"PRIx64, cachedir, first_byte, remainder);
 }
 
 /*
@@ -319,7 +357,7 @@ bs_rb_fetch(VALUE self, VALUE cachedir_v, VALUE path_v, VALUE handler, VALUE arg
   char cache_path[MAX_CACHEPATH_SIZE];
 
   /* generate cache path to cache_path */
-  bs_cache_path(cachedir, path, &cache_path);
+  bs_cache_path(cachedir, path_v, &cache_path);
 
   return bs_fetch(path, path_v, cache_path, handler, args);
 }
@@ -346,7 +384,7 @@ bs_rb_precompile(VALUE self, VALUE cachedir_v, VALUE path_v, VALUE handler)
   char cache_path[MAX_CACHEPATH_SIZE];
 
   /* generate cache path to cache_path */
-  bs_cache_path(cachedir, path, &cache_path);
+  bs_cache_path(cachedir, path_v, &cache_path);
 
   return bs_precompile(path, path_v, cache_path, handler);
 }
@@ -386,7 +424,8 @@ open_current_file(char * path, struct bs_cache_key * key, const char ** errno_pr
 }
 
 #define ERROR_WITH_ERRNO -1
-#define CACHE_MISSING_OR_INVALID -2
+#define CACHE_MISS -2
+#define CACHE_STALE -3
 
 /*
  * Read the cache key from the given fd, which must have position 0 (e.g.
@@ -394,15 +433,16 @@ open_current_file(char * path, struct bs_cache_key * key, const char ** errno_pr
  *
  * Possible return values:
  *   - 0 (OK, key was loaded)
- *   - CACHE_MISSING_OR_INVALID (-2)
  *   - ERROR_WITH_ERRNO (-1, errno is set)
+ *   - CACHE_MISS (-2)
+ *   - CACHE_STALE (-3)
  */
 static int
 bs_read_key(int fd, struct bs_cache_key * key)
 {
   ssize_t nread = read(fd, key, KEY_SIZE);
   if (nread < 0)        return ERROR_WITH_ERRNO;
-  if (nread < KEY_SIZE) return CACHE_MISSING_OR_INVALID;
+  if (nread < KEY_SIZE) return CACHE_STALE;
   return 0;
 }
 
@@ -412,7 +452,8 @@ bs_read_key(int fd, struct bs_cache_key * key)
  *
  * Possible return values:
  *   - 0 (OK, key was loaded)
- *   - CACHE_MISSING_OR_INVALID (-2)
+ *   - CACHE_MISS (-2)
+ *   - CACHE_STALE (-3)
  *   - ERROR_WITH_ERRNO (-1, errno is set)
  */
 static int
@@ -423,7 +464,7 @@ open_cache_file(const char * path, struct bs_cache_key * key, const char ** errn
   fd = open(path, O_RDONLY);
   if (fd < 0) {
     *errno_provenance = "bs_fetch:open_cache_file:open";
-    if (errno == ENOENT) return CACHE_MISSING_OR_INVALID;
+    if (errno == ENOENT) return CACHE_MISS;
     return ERROR_WITH_ERRNO;
   }
   #ifdef _WIN32
@@ -478,7 +519,7 @@ fetch_cached_data(int fd, ssize_t data_size, VALUE handler, VALUE args, VALUE *
     goto done;
   }
   if (nread != data_size) {
-    ret = CACHE_MISSING_OR_INVALID;
+    ret = CACHE_STALE;
     goto done;
   }
 
@@ -672,14 +713,22 @@ bs_fetch(char * path, VALUE path_v, char * cache_path, VALUE handler, VALUE args
 
   /* Open the cache key if it exists, and read its cache key in */
   cache_fd = open_cache_file(cache_path, &cached_key, &errno_provenance);
-  if (cache_fd == CACHE_MISSING_OR_INVALID) {
+  if (cache_fd == CACHE_MISS || cache_fd == CACHE_STALE) {
     /* This is ok: valid_cache remains false, we re-populate it. */
+    if (RB_UNLIKELY(instrumentation_enabled)) {
+      rb_funcall(rb_mBootsnap, instrumentation_method, 2, cache_fd == CACHE_MISS ? sym_miss : sym_stale, path_v);
+    }
   } else if (cache_fd < 0) {
     goto fail_errno;
   } else {
     /* True if the cache existed and no invalidating changes have occurred since
      * it was generated. */
     valid_cache = cache_key_equal(&current_key, &cached_key);
+    if (RB_UNLIKELY(instrumentation_enabled)) {
+      if (!valid_cache) {
+        rb_funcall(rb_mBootsnap, instrumentation_method, 2, sym_stale, path_v);
+      }
+    }
   }
 
   if (valid_cache) {
@@ -688,10 +737,10 @@ bs_fetch(char * path, VALUE path_v, char * cache_path, VALUE handler, VALUE args
       cache_fd, (ssize_t)cached_key.data_size, handler, args,
       &output_data, &exception_tag, &errno_provenance
     );
-    if (exception_tag != 0)                   goto raise;
-    else if (res == CACHE_MISSING_OR_INVALID) valid_cache = 0;
-    else if (res == ERROR_WITH_ERRNO)         goto fail_errno;
-    else if (!NIL_P(output_data))             goto succeed; /* fast-path, goal */
+    if (exception_tag != 0) goto raise;
+    else if (res == CACHE_MISS || res == CACHE_STALE) valid_cache = 0;
+    else if (res == ERROR_WITH_ERRNO) goto fail_errno;
+    else if (!NIL_P(output_data)) goto succeed; /* fast-path, goal */
   }
   close(cache_fd);
   cache_fd = -1;
@@ -778,7 +827,7 @@ bs_precompile(char * path, VALUE path_v, char * cache_path, VALUE handler)
 
   /* Open the cache key if it exists, and read its cache key in */
   cache_fd = open_cache_file(cache_path, &cached_key, &errno_provenance);
-  if (cache_fd == CACHE_MISSING_OR_INVALID) {
+  if (cache_fd == CACHE_MISS || cache_fd == CACHE_STALE) {
     /* This is ok: valid_cache remains false, we re-populate it. */
   } else if (cache_fd < 0) {
     goto fail;

data/ext/bootsnap/extconf.rb

@@ -1,19 +1,24 @@
 # frozen_string_literal: true
 require("mkmf")
-$CFLAGS << ' -O3 '
-$CFLAGS << ' -std=c99'
 
-# ruby.h has some -Wpedantic fails in some cases
-# (e.g. https://github.com/Shopify/bootsnap/issues/15)
-unless ['0', '', nil].include?(ENV['BOOTSNAP_PEDANTIC'])
-  $CFLAGS << ' -Wall'
-  $CFLAGS << ' -Werror'
-  $CFLAGS << ' -Wextra'
-  $CFLAGS << ' -Wpedantic'
+if RUBY_ENGINE == 'ruby'
+  $CFLAGS << ' -O3 '
+  $CFLAGS << ' -std=c99'
 
-  $CFLAGS << ' -Wno-unused-parameter' # VALUE self has to be there but we don't care what it is.
-  $CFLAGS << ' -Wno-keyword-macro' # hiding return
-  $CFLAGS << ' -Wno-gcc-compat' # ruby.h 2.6.0 on macos 10.14, dunno
-end
+  # ruby.h has some -Wpedantic fails in some cases
+  # (e.g. https://github.com/Shopify/bootsnap/issues/15)
+  unless ['0', '', nil].include?(ENV['BOOTSNAP_PEDANTIC'])
+    $CFLAGS << ' -Wall'
+    $CFLAGS << ' -Werror'
+    $CFLAGS << ' -Wextra'
+    $CFLAGS << ' -Wpedantic'
+
+    $CFLAGS << ' -Wno-unused-parameter' # VALUE self has to be there but we don't care what it is.
+    $CFLAGS << ' -Wno-keyword-macro' # hiding return
+    $CFLAGS << ' -Wno-gcc-compat' # ruby.h 2.6.0 on macos 10.14, dunno
+  end
 
-create_makefile("bootsnap/bootsnap")
+  create_makefile("bootsnap/bootsnap")
+else
+  File.write("Makefile", dummy_makefile($srcdir).join(""))
+end

data/lib/bootsnap.rb

@@ -8,25 +8,60 @@ require_relative('bootsnap/compile_cache')
 module Bootsnap
   InvalidConfiguration = Class.new(StandardError)
 
+  class << self
+    attr_reader :logger
+  end
+
+  def self.log!
+    self.logger = $stderr.method(:puts)
+  end
+
+  def self.logger=(logger)
+    @logger = logger
+    if logger.respond_to?(:debug)
+      self.instrumentation = ->(event, path) { @logger.debug("[Bootsnap] #{event} #{path}") }
+    else
+      self.instrumentation = ->(event, path) { @logger.call("[Bootsnap] #{event} #{path}") }
+    end
+  end
+
+  def self.instrumentation=(callback)
+    @instrumentation = callback
+    self.instrumentation_enabled = !!callback
+  end
+
+  def self._instrument(event, path)
+    @instrumentation.call(event, path)
+  end
+
   def self.setup(
     cache_dir:,
     development_mode: true,
     load_path_cache: true,
-    autoload_paths_cache: true,
-    disable_trace: false,
+    autoload_paths_cache: nil,
+    disable_trace: nil,
     compile_cache_iseq: true,
     compile_cache_yaml: true
   )
-    if autoload_paths_cache && !load_path_cache
-      raise(InvalidConfiguration, "feature 'autoload_paths_cache' depends on feature 'load_path_cache'")
+    unless autoload_paths_cache.nil?
+      warn "[DEPRECATED] Bootsnap's `autoload_paths_cache:` option is deprecated and will be removed. " \
+        "If you use Zeitwerk this option is useless, and if you are still using the classic autoloader " \
+        "upgrading is recommended."
     end
 
-    setup_disable_trace if disable_trace
+    unless disable_trace.nil?
+      warn "[DEPRECATED] Bootsnap's `disable_trace:` option is deprecated and will be removed. " \
+        "If you use Ruby 2.5 or newer this option is useless, if not upgrading is recommended."
+    end
+
+    if compile_cache_iseq && !iseq_cache_supported?
+      warn "Ruby 2.5 has a bug that break code tracing when code is loaded from cache. It is recommened " \
+        "to turn `compile_cache_iseq` off on Ruby 2.5"
+    end
 
     Bootsnap::LoadPathCache.setup(
       cache_path:       cache_dir + '/bootsnap/load-path-cache',
       development_mode: development_mode,
-      active_support:   autoload_paths_cache
     ) if load_path_cache
 
     Bootsnap::CompileCache.setup(
@@ -36,14 +71,51 @@ module Bootsnap
     )
   end
 
-  def self.setup_disable_trace
-    if Gem::Version.new(RUBY_VERSION) >= Gem::Version.new('2.5.0')
-      warn(
-        "from #{caller_locations(1, 1)[0]}: The 'disable_trace' method is not allowed with this Ruby version. " \
-        "current: #{RUBY_VERSION}, allowed version: < 2.5.0",
+  def self.iseq_cache_supported?
+    return @iseq_cache_supported if defined? @iseq_cache_supported
+
+    ruby_version = Gem::Version.new(RUBY_VERSION)
+    @iseq_cache_supported = ruby_version < Gem::Version.new('2.5.0') || ruby_version >= Gem::Version.new('2.6.0')
+  end
+
+  def self.default_setup
+    env = ENV['RAILS_ENV'] || ENV['RACK_ENV'] || ENV['ENV']
+    development_mode = ['', nil, 'development'].include?(env)
+
+    unless ENV['DISABLE_BOOTSNAP']
+      cache_dir = ENV['BOOTSNAP_CACHE_DIR']
+      unless cache_dir
+        config_dir_frame = caller.detect do |line|
+          line.include?('/config/')
+        end
+
+        unless config_dir_frame
+          $stderr.puts("[bootsnap/setup] couldn't infer cache directory! Either:")
+          $stderr.puts("[bootsnap/setup]   1. require bootsnap/setup from your application's config directory; or")
+          $stderr.puts("[bootsnap/setup]   2. Define the environment variable BOOTSNAP_CACHE_DIR")
+
+          raise("couldn't infer bootsnap cache directory")
+        end
+
+        path = config_dir_frame.split(/:\d+:/).first
+        path = File.dirname(path) until File.basename(path) == 'config'
+        app_root = File.dirname(path)
+
+        cache_dir = File.join(app_root, 'tmp', 'cache')
+      end
+
+
+      setup(
+        cache_dir:            cache_dir,
+        development_mode:     development_mode,
+        load_path_cache:      !ENV['DISABLE_BOOTSNAP_LOAD_PATH_CACHE'],
+        compile_cache_iseq:   !ENV['DISABLE_BOOTSNAP_COMPILE_CACHE'] && iseq_cache_supported?,
+        compile_cache_yaml:   !ENV['DISABLE_BOOTSNAP_COMPILE_CACHE'],
       )
-    else
-      RubyVM::InstructionSequence.compile_option = { trace_instruction: false }
+
+      if ENV['BOOTSNAP_LOG']
+        log!
+      end
     end
   end
 end

data/lib/bootsnap/cli.rb

@@ -60,7 +60,7 @@ module Bootsnap
 
           # Gems that include YAML files usually don't put them in `lib/`.
           # So we look at the gem root.
-          gem_pattern = %r{^#{Regexp.escape(Bundler.bundle_path.to_s)}/?(?:bundler/)?gem\/[^/]+}
+          gem_pattern = %r{^#{Regexp.escape(Bundler.bundle_path.to_s)}/?(?:bundler/)?gems\/[^/]+}
           gem_paths = $LOAD_PATH.map { |p| p[gem_pattern] }.compact.uniq
           precompile_yaml_files(gem_paths, exclude: gem_exclude)
         end
@@ -121,7 +121,7 @@ module Bootsnap
         if !exclude || !exclude.match?(path)
           list_files(path, '**/*.{yml,yaml}').each do |yaml_file|
             # We ignore hidden files to not match the various .ci.yml files
-            if !yaml_file.include?('/.') && (!exclude || !exclude.match?(yaml_file))
+            if !File.basename(yaml_file).start_with?('.') && (!exclude || !exclude.match?(yaml_file))
               @work_pool.push(:yaml, yaml_file)
             end
           end

data/lib/bootsnap/compile_cache/yaml.rb

@@ -8,8 +8,7 @@ module Bootsnap
         attr_accessor(:msgpack_factory, :cache_dir, :supported_options)
 
         def input_to_storage(contents, _)
-          raise(Uncompilable) if contents.index("!ruby/object")
-          obj = ::YAML.load(contents)
+          obj = strict_load(contents)
           msgpack_factory.dump(obj)
         rescue NoMethodError, RangeError
           # The object included things that we can't serialize
@@ -27,6 +26,13 @@ module Bootsnap
           ::YAML.load(data, **(kwargs || {}))
         end
 
+        def strict_load(payload, *args)
+          ast = ::YAML.parse(payload)
+          return ast unless ast
+          strict_visitor.create(*args).visit(ast)
+        end
+        ruby2_keywords :strict_load if respond_to?(:ruby2_keywords, true)
+
         def precompile(path, cache_dir: YAML.cache_dir)
           Bootsnap::CompileCache::Native.precompile(
             cache_dir,
@@ -51,22 +57,25 @@ module Bootsnap
           # see: https://github.com/msgpack/msgpack-ruby/pull/122
           factory = MessagePack::Factory.new
           factory.register_type(0x00, Symbol)
-          factory.register_type(
-            MessagePack::Timestamp::TYPE, # or just -1
-            Time,
-            packer: MessagePack::Time::Packer,
-            unpacker: MessagePack::Time::Unpacker
-          )
 
-          marshal_fallback = {
-            packer: ->(value) { Marshal.dump(value) },
-            unpacker: ->(payload) { Marshal.load(payload) },
-          }
-          {
-            Date => 0x01,
-            Regexp => 0x02,
-          }.each do |type, code|
-            factory.register_type(code, type, marshal_fallback)
+          if defined? MessagePack::Timestamp
+            factory.register_type(
+              MessagePack::Timestamp::TYPE, # or just -1
+              Time,
+              packer: MessagePack::Time::Packer,
+              unpacker: MessagePack::Time::Unpacker
+            )
+
+            marshal_fallback = {
+              packer: ->(value) { Marshal.dump(value) },
+              unpacker: ->(payload) { Marshal.load(payload) },
+            }
+            {
+              Date => 0x01,
+              Regexp => 0x02,
+            }.each do |type, code|
+              factory.register_type(code, type, marshal_fallback)
+            end
           end
 
           self.msgpack_factory = factory
@@ -83,6 +92,17 @@ module Bootsnap
           end
           self.supported_options.freeze
         end
+
+        def strict_visitor
+          self::NoTagsVisitor ||= Class.new(Psych::Visitors::ToRuby) do
+            def visit(target)
+              if target.tag
+                raise Uncompilable, "YAML tags are not supported: #{target.tag}"
+              end
+              super
+            end
+          end
+        end
       end
 
       module Patch
@@ -96,7 +116,7 @@ module Bootsnap
           begin
             ::Bootsnap::CompileCache::Native.fetch(
               Bootsnap::CompileCache::YAML.cache_dir,
-              path,
+              File.realpath(path),
               ::Bootsnap::CompileCache::YAML,
               kwargs,
             )

data/lib/bootsnap/load_path_cache.rb

@@ -28,10 +28,9 @@ module Bootsnap
     CACHED_EXTENSIONS = DLEXT2 ? [DOT_RB, DLEXT, DLEXT2] : [DOT_RB, DLEXT]
 
     class << self
-      attr_reader(:load_path_cache, :autoload_paths_cache,
-        :loaded_features_index, :realpath_cache)
+      attr_reader(:load_path_cache, :loaded_features_index, :realpath_cache)
 
-      def setup(cache_path:, development_mode:, active_support: true)
+      def setup(cache_path:, development_mode:)
         unless supported?
           warn("[bootsnap/setup] Load path caching is not supported on this implementation of Ruby") if $VERBOSE
           return
@@ -45,18 +44,6 @@ module Bootsnap
         @load_path_cache = Cache.new(store, $LOAD_PATH, development_mode: development_mode)
         require_relative('load_path_cache/core_ext/kernel_require')
         require_relative('load_path_cache/core_ext/loaded_features')
-
-        if active_support
-          # this should happen after setting up the initial cache because it
-          # loads a lot of code. It's better to do after +require+ is optimized.
-          require('active_support/dependencies')
-          @autoload_paths_cache = Cache.new(
-            store,
-            ::ActiveSupport::Dependencies.autoload_paths,
-            development_mode: development_mode
-          )
-          require_relative('load_path_cache/core_ext/active_support')
-        end
       end
 
       def supported?

data/lib/bootsnap/load_path_cache/cache.rb

@@ -10,8 +10,8 @@ module Bootsnap
       def initialize(store, path_obj, development_mode: false)
         @development_mode = development_mode
         @store = store
-        @mutex = defined?(::Mutex) ? ::Mutex.new : ::Thread::Mutex.new # TODO: Remove once Ruby 2.2 support is dropped.
-        @path_obj = path_obj.map! { |f| File.exist?(f) ? File.realpath(f) : f }
+        @mutex = Mutex.new
+        @path_obj = path_obj.map! { |f| PathScanner.os_path(File.exist?(f) ? File.realpath(f) : f.dup) }
         @has_relative_paths = nil
         reinitialize
       end
@@ -178,25 +178,42 @@ module Bootsnap
 
       if DLEXT2
         def search_index(f)
-          try_index("#{f}#{DOT_RB}") || try_index("#{f}#{DLEXT}") || try_index("#{f}#{DLEXT2}") || try_index(f)
+          try_index(f + DOT_RB) || try_index(f + DLEXT) || try_index(f + DLEXT2) || try_index(f)
         end
 
         def maybe_append_extension(f)
-          try_ext("#{f}#{DOT_RB}") || try_ext("#{f}#{DLEXT}") || try_ext("#{f}#{DLEXT2}") || f
+          try_ext(f + DOT_RB) || try_ext(f + DLEXT) || try_ext(f + DLEXT2) || f
         end
       else
         def search_index(f)
-          try_index("#{f}#{DOT_RB}") || try_index("#{f}#{DLEXT}") || try_index(f)
+          try_index(f + DOT_RB) || try_index(f + DLEXT) || try_index(f)
         end
 
         def maybe_append_extension(f)
-          try_ext("#{f}#{DOT_RB}") || try_ext("#{f}#{DLEXT}") || f
+          try_ext(f + DOT_RB) || try_ext(f + DLEXT) || f
         end
       end
 
-      def try_index(f)
-        if (p = @index[f])
-          "#{p}/#{f}"
+      s = rand.to_s.force_encoding(Encoding::US_ASCII).freeze
+      if s.respond_to?(:-@)
+        if (-s).equal?(s) && (-s.dup).equal?(s)
+          def try_index(f)
+            if (p = @index[f])
+              -(File.join(p, f).freeze)
+            end
+          end
+        else
+          def try_index(f)
+            if (p = @index[f])
+              -File.join(p, f).untaint
+            end
+          end
+        end
+      else
+        def try_index(f)
+          if (p = @index[f])
+            File.join(p, f)
+          end
         end
       end
 

data/lib/bootsnap/load_path_cache/loaded_features_index.rb

@@ -26,7 +26,7 @@ module Bootsnap
     class LoadedFeaturesIndex
       def initialize
         @lfi = {}
-        @mutex = defined?(::Mutex) ? ::Mutex.new : ::Thread::Mutex.new # TODO: Remove once Ruby 2.2 support is dropped.
+        @mutex = Mutex.new
 
         # In theory the user could mutate $LOADED_FEATURES and invalidate our
         # cache. If this ever comes up in practice — or if you, the

data/lib/bootsnap/load_path_cache/path_scanner.rb

@@ -33,10 +33,10 @@ module Bootsnap
           requirables = []
           walk(path, nil) do |relative_path, absolute_path, is_directory|
             if is_directory
-              dirs << relative_path
+              dirs << os_path(relative_path)
               !contains_bundle_path || !absolute_path.start_with?(BUNDLE_PATH)
             elsif relative_path.end_with?(*REQUIRABLE_EXTENSIONS)
-              requirables << relative_path
+              requirables << os_path(relative_path)
             end
           end
           [requirables, dirs]
@@ -45,7 +45,7 @@ module Bootsnap
         def walk(absolute_dir_path, relative_dir_path, &block)
           Dir.foreach(absolute_dir_path) do |name|
             next if name.start_with?('.')
-            relative_path = relative_dir_path ? "#{relative_dir_path}/#{name}" : name.freeze
+            relative_path = relative_dir_path ? File.join(relative_dir_path, name) : name
 
             absolute_path = "#{absolute_dir_path}/#{name}"
             if File.directory?(absolute_path)
@@ -57,6 +57,17 @@ module Bootsnap
             end
           end
         end
+
+        if RUBY_VERSION >= '3.1'
+          def os_path(path)
+            path.freeze
+          end
+        else
+          def os_path(path)
+            path.force_encoding(Encoding::US_ASCII) if path.ascii_only?
+            path.freeze
+          end
+        end
       end
     end
   end

data/lib/bootsnap/load_path_cache/store.rb

@@ -12,8 +12,7 @@ module Bootsnap
 
       def initialize(store_path)
         @store_path = store_path
-        # TODO: Remove conditional once Ruby 2.2 support is dropped.
-        @txn_mutex =  defined?(::Mutex) ? ::Mutex.new : ::Thread::Mutex.new
+        @txn_mutex = Mutex.new
         @dirty = false
         load_data
       end
@@ -63,12 +62,18 @@ module Bootsnap
 
       def load_data
         @data = begin
-          MessagePack.load(File.binread(@store_path))
-                # handle malformed data due to upgrade incompatibility
-                rescue Errno::ENOENT, MessagePack::MalformedFormatError, MessagePack::UnknownExtTypeError, EOFError
-                  {}
-                rescue ArgumentError => e
-                  e.message =~ /negative array size/ ? {} : raise
+          File.open(@store_path, encoding: Encoding::BINARY) do |io|
+            MessagePack.load(io)
+          end
+        # handle malformed data due to upgrade incompatibility
+        rescue Errno::ENOENT, MessagePack::MalformedFormatError, MessagePack::UnknownExtTypeError, EOFError
+          {}
+        rescue ArgumentError => error
+          if error.message =~ /negative array size/
+            {}
+          else
+            raise
+          end
         end
       end
 
@@ -80,7 +85,9 @@ module Bootsnap
         exclusive_write = File::Constants::CREAT | File::Constants::EXCL | File::Constants::WRONLY
         # `encoding:` looks redundant wrt `binwrite`, but necessary on windows
         # because binary is part of mode.
-        File.binwrite(tmp, MessagePack.dump(@data), mode: exclusive_write, encoding: Encoding::BINARY)
+        File.open(tmp, mode: exclusive_write, encoding: Encoding::BINARY) do |io|
+          MessagePack.dump(@data, io, freeze: true)
+        end
         FileUtils.mv(tmp, @store_path)
       rescue Errno::EEXIST
         retry

data/lib/bootsnap/setup.rb

@@ -1,39 +1,4 @@
 # frozen_string_literal: true
 require_relative('../bootsnap')
 
-env = ENV['RAILS_ENV'] || ENV['RACK_ENV'] || ENV['ENV']
-development_mode = ['', nil, 'development'].include?(env)
-
-cache_dir = ENV['BOOTSNAP_CACHE_DIR']
-unless cache_dir
-  config_dir_frame = caller.detect do |line|
-    line.include?('/config/')
-  end
-
-  unless config_dir_frame
-    $stderr.puts("[bootsnap/setup] couldn't infer cache directory! Either:")
-    $stderr.puts("[bootsnap/setup]   1. require bootsnap/setup from your application's config directory; or")
-    $stderr.puts("[bootsnap/setup]   2. Define the environment variable BOOTSNAP_CACHE_DIR")
-
-    raise("couldn't infer bootsnap cache directory")
-  end
-
-  path = config_dir_frame.split(/:\d+:/).first
-  path = File.dirname(path) until File.basename(path) == 'config'
-  app_root = File.dirname(path)
-
-  cache_dir = File.join(app_root, 'tmp', 'cache')
-end
-
-ruby_version = Gem::Version.new(RUBY_VERSION)
-iseq_cache_enabled = ruby_version < Gem::Version.new('2.5.0') || ruby_version >= Gem::Version.new('2.6.0')
-
-Bootsnap.setup(
-  cache_dir:            cache_dir,
-  development_mode:     development_mode,
-  load_path_cache:      true,
-  autoload_paths_cache: true, # assume rails. open to PRs to impl. detection
-  disable_trace:        false,
-  compile_cache_iseq:   iseq_cache_enabled,
-  compile_cache_yaml:   true,
-)
+Bootsnap.default_setup

data/lib/bootsnap/version.rb

@@ -1,4 +1,4 @@
 # frozen_string_literal: true
 module Bootsnap
-  VERSION = "1.6.0"
+  VERSION = "1.7.3"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: bootsnap
 version: !ruby/object:Gem::Version
-  version: 1.6.0
+  version: 1.7.3
 platform: ruby
 authors:
 - Burke Libbey
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2021-01-27 00:00:00.000000000 Z
+date: 2021-03-22 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler
@@ -121,7 +121,6 @@ files:
 - lib/bootsnap/load_path_cache.rb
 - lib/bootsnap/load_path_cache/cache.rb
 - lib/bootsnap/load_path_cache/change_observer.rb
-- lib/bootsnap/load_path_cache/core_ext/active_support.rb
 - lib/bootsnap/load_path_cache/core_ext/kernel_require.rb
 - lib/bootsnap/load_path_cache/core_ext/loaded_features.rb
 - lib/bootsnap/load_path_cache/loaded_features_index.rb

data/lib/bootsnap/load_path_cache/core_ext/active_support.rb

@@ -1,107 +0,0 @@
-# frozen_string_literal: true
-module Bootsnap
-  module LoadPathCache
-    module CoreExt
-      module ActiveSupport
-        def self.without_bootsnap_cache
-          prev = Thread.current[:without_bootsnap_cache] || false
-          Thread.current[:without_bootsnap_cache] = true
-          yield
-        ensure
-          Thread.current[:without_bootsnap_cache] = prev
-        end
-
-        def self.allow_bootsnap_retry(allowed)
-          prev = Thread.current[:without_bootsnap_retry] || false
-          Thread.current[:without_bootsnap_retry] = !allowed
-          yield
-        ensure
-          Thread.current[:without_bootsnap_retry] = prev
-        end
-
-        module ClassMethods
-          def autoload_paths=(o)
-            super
-            Bootsnap::LoadPathCache.autoload_paths_cache.reinitialize(o)
-          end
-
-          def search_for_file(path)
-            return super if Thread.current[:without_bootsnap_cache]
-            begin
-              Bootsnap::LoadPathCache.autoload_paths_cache.find(path)
-            rescue Bootsnap::LoadPathCache::ReturnFalse
-              nil # doesn't really apply here
-            rescue Bootsnap::LoadPathCache::FallbackScan
-              nil # doesn't really apply here
-            end
-          end
-
-          def autoloadable_module?(path_suffix)
-            Bootsnap::LoadPathCache.autoload_paths_cache.load_dir(path_suffix)
-          end
-
-          def remove_constant(const)
-            CoreExt::ActiveSupport.without_bootsnap_cache { super }
-          end
-
-          def require_or_load(*)
-            CoreExt::ActiveSupport.allow_bootsnap_retry(true) do
-              super
-            end
-          end
-
-          # If we can't find a constant using the patched implementation of
-          # search_for_file, try again with the default implementation.
-          #
-          # These methods call search_for_file, and we want to modify its
-          # behaviour.  The gymnastics here are a bit awkward, but it prevents
-          # 200+ lines of monkeypatches.
-          def load_missing_constant(from_mod, const_name)
-            CoreExt::ActiveSupport.allow_bootsnap_retry(false) do
-              super
-            end
-          rescue NameError => e
-            raise(e) if e.instance_variable_defined?(Bootsnap::LoadPathCache::ERROR_TAG_IVAR)
-            e.instance_variable_set(Bootsnap::LoadPathCache::ERROR_TAG_IVAR, true)
-
-            # This function can end up called recursively, we only want to
-            # retry at the top-level.
-            raise(e) if Thread.current[:without_bootsnap_retry]
-            # If we already had cache disabled, there's no use retrying
-            raise(e) if Thread.current[:without_bootsnap_cache]
-            # NoMethodError is a NameError, but we only want to handle actual
-            # NameError instances.
-            raise(e) unless e.class == NameError
-            # We can only confidently handle cases when *this* constant fails
-            # to load, not other constants referred to by it.
-            raise(e) unless e.name == const_name
-            # If the constant was actually loaded, something else went wrong?
-            raise(e) if from_mod.const_defined?(const_name)
-            CoreExt::ActiveSupport.without_bootsnap_cache { super }
-          end
-
-          # Signature has changed a few times over the years; easiest to not
-          # reiterate it with version polymorphism here...
-          def depend_on(*)
-            super
-          rescue LoadError => e
-            raise(e) if e.instance_variable_defined?(Bootsnap::LoadPathCache::ERROR_TAG_IVAR)
-            e.instance_variable_set(Bootsnap::LoadPathCache::ERROR_TAG_IVAR, true)
-
-            # If we already had cache disabled, there's no use retrying
-            raise(e) if Thread.current[:without_bootsnap_cache]
-            CoreExt::ActiveSupport.without_bootsnap_cache { super }
-          end
-        end
-      end
-    end
-  end
-end
-
-module ActiveSupport
-  module Dependencies
-    class << self
-      prepend(Bootsnap::LoadPathCache::CoreExt::ActiveSupport::ClassMethods)
-    end
-  end
-end