bootsnap 1.0.0 → 1.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 77b33ef59e01f0ddaa1de86a002ce286018b870d
-  data.tar.gz: 958ec1e45c50803261866ade91f9300dd6975d03
+  metadata.gz: fbdf0cc19b09b396571d9d38cb037cebfcdab7b2
+  data.tar.gz: f07f07ec071dbffe3a48e1ae324457729a11ca61
 SHA512:
-  metadata.gz: bcb190169610158178bff1fbf16ed6d43a4f986d06c7e07c16c5199aa658512749490b43e42b1e2e431a1099aa5ebd50e5905a35a0170f75e2cc2d468037eca1
-  data.tar.gz: 17fe5e20e585fce572bc39a47febe146abba4ec0aea70934c9d39f88fd271f158b3dd0fc3b0fe103e3060c97ade3bd890cf9e74d28f7ba4fb6b7836a401dac14
+  metadata.gz: 9db4ab87163f7c05a7b5c62fe17617149ec5acdc1caf186013c334222a4ad0a2087c5992b44fcf424201dd5b12448ce25826ccb4f99f495e450437aa90d0a0ae
+  data.tar.gz: 84714b27f6a3be1650807af151309dc789390944c1a317ec9c246067c2f4299184c182aa878c8eb5715da989c1c768a3d86992af02ab9d974f1d26d09bb2d170

data/CHANGELOG.md

@@ -1,3 +1,10 @@
+# 1.1.0
+
+* Add `bootsnap/setup`
+* Support jruby (without compile caching features)
+* Better deoptimization when Coverage is enabled
+* Consider `Bundler.bundle_path` to be stable
+
 # 1.0.0
 
 * (none)

data/README.md

@@ -1,11 +1,11 @@
-# Bootsnap ![CI status](https://travis-ci.org/Shopify/bootsnap.svg?branch=master)
+# Bootsnap [![Build Status](https://travis-ci.org/Shopify/bootsnap.svg?branch=master)](https://travis-ci.org/Shopify/bootsnap)
 
 **Beta-quality. See [the last section of this README](#trustworthiness).**
 
-Bootsnap is a library that plugs into a number of Ruby and (optionally) `ActiveSupport` and `YAML`
-methods to optimize and cache expensive computations. See [the How Does This Work section](#how-does-this-work) for more information.
+Bootsnap is a library that plugs into Ruby, with optional support for `ActiveSupport` and `YAML`,
+to optimize and cache expensive computations. See [How Does This Work](#how-does-this-work).
 
-#### Quick Performance Overview
+#### Performance
 - [Discourse](https://github.com/discourse/discourse) reports a boot time reduction of approximately 50%, from roughly 6 to 3 seconds on one machine;
 - One of our smaller internal apps also sees a reduction of 50%, from 3.6 to 1.8 seconds;
 - The core Shopify platform -- a rather large monolithic application -- boots about 75% faster, dropping from around 25s to 6.5s.
@@ -20,19 +20,27 @@ Add `bootsnap` to your `Gemfile`:
 gem 'bootsnap'
 ```
 
-Next, add this to your boot setup immediately after `require 'bundler/setup'` (i.e. as early as
-possible: the sooner this is loaded, the sooner it can start optimizing things)
+If you are using rails, add this to `config/boot.rb` immediately after `require 'bundler/setup'`:
+
+```ruby
+require 'bootsnap/setup'
+```
+
+If you are not using rails, or if you are but want more control over things, add this to your
+application setup immediately after `require 'bundler/setup'` (i.e. as early as possible: the sooner
+this is loaded, the sooner it can start optimizing things)
 
 ```ruby
 require 'bootsnap'
+env = ENV['RAILS_ENV'] || "development"
 Bootsnap.setup(
-  cache_dir:            'tmp/cache', # Path to your cache
-  development_mode:     ENV['MY_ENV'] == 'development',
-  load_path_cache:      true,        # Should we optimize the LOAD_PATH with a cache?
-  autoload_paths_cache: true,        # Should we optimize ActiveSupport autoloads with cache?
-  disable_trace:        false,       # Sets `RubyVM::InstructionSequence.compile_option = { trace_instruction: false }`
-  compile_cache_iseq:   true,        # Should compile Ruby code into ISeq cache?
-  compile_cache_yaml:   true         # Should compile YAML into a cache?
+  cache_dir:            'tmp/cache',          # Path to your cache
+  development_mode:     env == 'development', # Current working environment, e.g. RACK_ENV, RAILS_ENV, etc
+  load_path_cache:      true,                 # Optimize the LOAD_PATH with a cache
+  autoload_paths_cache: true,                 # Optimize ActiveSupport autoloads with cache
+  disable_trace:        true,                 # (Alpha) Set `RubyVM::InstructionSequence.compile_option = { trace_instruction: false }`
+  compile_cache_iseq:   true,                 # Compile Ruby code into ISeq cache, breaks coverage reporting.
+  compile_cache_yaml:   true                  # Compile YAML into a cache
 )
 ```
 
@@ -42,8 +50,7 @@ will help optimize boot time further if you have an extremely large `$LOAD_PATH`
 
 ## How does this work?
 
-Bootsnap is a library that plugs into a number of Ruby and (optionally) `ActiveSupport` and `YAML`
-methods. These methods are modified to cache results of expensive computations, and can be grouped
+Bootsnap optimizes methods to cache results of expensive computations, and can be grouped
 into two broad categories:
 
 * [Path Pre-Scanning](#path-pre-scanning)
@@ -103,7 +110,8 @@ entries do not expire -- once their contents has been scanned, it is assumed to
 
 The only directories considered "stable" are things under the Ruby install prefix
 (`RbConfig::CONFIG['prefix']`, e.g. `/usr/local/ruby` or `~/.rubies/x.y.z`), and things under the
-`Gem.path` (e.g. `~/.gem/ruby/x.y.z`). Everything else is considered "volatile".
+`Gem.path` (e.g. `~/.gem/ruby/x.y.z`) or `Bundler.bundle_path`. Everything else is considered
+"volatile".
 
 In addition to the [`Bootsnap::LoadPathCache::Cache`
 source](https://github.com/Shopify/bootsnap/blob/master/lib/bootsnap/load_path_cache/cache.rb),
@@ -262,7 +270,7 @@ open    /c/nope.bundle -> -1
 We use the `*_path_cache` features in production and haven't experienced any issues in a long time.
 
 The `compile_cache_*` features work well for us in development on macOS. It should work on Linux,
-and we intend to deploy it in production, but haven't we haven't yet.
+and we intend to deploy it in production, but we haven't yet.
 
 `disable_trace` should be completely safe, but we don't really use it because some people like to
 use tools that make use of `trace` instructions.

data/bootsnap.gemspec

@@ -18,19 +18,22 @@ Gem::Specification.new do |spec|
   spec.files = `git ls-files -z`.split("\x0").reject do |f|
     f.match(%r{^(test|spec|features)/})
   end
-  spec.bindir        = "exe"
-  spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
   spec.require_paths = ["lib"]
-  spec.extensions    = ['ext/bootsnap/extconf.rb']
 
-  spec.required_ruby_version = '>= 2.3.0'
+  spec.required_ruby_version = '>= 2.0.0'
+
+  if RUBY_PLATFORM =~ /java/
+    spec.platform = 'java'
+  else
+    spec.platform    = Gem::Platform::RUBY
+    spec.extensions  = ['ext/bootsnap/extconf.rb']
+  end
 
   spec.add_development_dependency "bundler", '~> 1'
   spec.add_development_dependency 'rake', '~> 10.0'
   spec.add_development_dependency 'rake-compiler', '~> 0'
   spec.add_development_dependency "minitest", "~> 5.0"
   spec.add_development_dependency "mocha", "~> 1.2"
-  spec.add_development_dependency "ffi-xattr", "~> 0.1.2"
 
   spec.add_runtime_dependency "msgpack", "~> 1.0"
 end

data/ext/bootsnap/bootsnap.c

@@ -18,7 +18,9 @@
 #include <errno.h>
 #include <fcntl.h>
 #include <sys/stat.h>
+#ifndef _WIN32
 #include <sys/utsname.h>
+#endif
 
 /* 1000 is an arbitrary limit; FNV64 plus some slashes brings the cap down to
  * 981 for the cache dir */
@@ -109,6 +111,15 @@ struct s2o_data;
 struct i2o_data;
 struct i2s_data;
 
+/* https://bugs.ruby-lang.org/issues/13667 */
+extern VALUE rb_get_coverages(void);
+static VALUE
+bs_rb_coverage_running(VALUE self)
+{
+  VALUE cov = rb_get_coverages();
+  return RTEST(cov) ? Qtrue : Qfalse;
+}
+
 /*
  * Ruby C extensions are initialized by calling Init_<extname>.
  *
@@ -129,6 +140,7 @@ Init_bootsnap(void)
 
   uncompilable = rb_intern("__bootsnap_uncompilable__");
 
+  rb_define_module_function(rb_mBootsnap_CompileCache_Native, "coverage_running?", bs_rb_coverage_running, 0);
   rb_define_module_function(rb_mBootsnap_CompileCache_Native, "fetch", bs_rb_fetch, 3);
   rb_define_module_function(rb_mBootsnap_CompileCache_Native, "compile_option_crc32=", bs_compile_option_crc32_set, 1);
 }
@@ -176,6 +188,9 @@ fnv1a_64(const char *str)
 static uint32_t
 get_os_version(void)
 {
+  #ifdef _WIN32
+  return (uint32_t)GetVersion();
+  #else
   uint64_t hash;
   struct utsname utsname;
 
@@ -185,6 +200,7 @@ get_os_version(void)
   hash = fnv1a_64(utsname.version);
 
   return (uint32_t)(hash >> 32);
+  #endif
 }
 
 /*
@@ -265,6 +281,9 @@ open_current_file(char * path, struct bs_cache_key * key)
 
   fd = open(path, O_RDONLY);
   if (fd < 0) return fd;
+  #ifdef _WIN32
+  setmode(fd, O_BINARY);
+  #endif
 
   if (fstat(fd, &statbuf) < 0) {
     close(fd);
@@ -316,11 +335,14 @@ open_cache_file(const char * path, struct bs_cache_key * key)
 {
   int fd, res;
 
-  fd = open(path, O_RDWR, 0644);
+  fd = open(path, O_RDONLY);
   if (fd < 0) {
     if (errno == ENOENT) return CACHE_MISSING_OR_INVALID;
     return ERROR_WITH_ERRNO;
   }
+  #ifdef _WIN32
+  setmode(fd, O_BINARY);
+  #endif
 
   res = bs_read_key(fd, key);
   if (res < 0) {
@@ -393,7 +415,11 @@ mkpath(char * file_path, mode_t mode)
   char * p;
   for (p = strchr(file_path + 1, '/'); p; p = strchr(p + 1, '/')) {
     *p = '\0';
+    #ifdef _WIN32
+    if (mkdir(file_path) == -1) {
+    #else
     if (mkdir(file_path, mode) == -1) {
+    #endif
       if (errno != EEXIST) {
         *p = '/';
         return -1;
@@ -428,6 +454,9 @@ atomic_write_cache_file(char * path, struct bs_cache_key * key, VALUE data)
     fd = open(tmp_path, O_WRONLY | O_CREAT, 0644);
     if (fd < 0) return -1;
   }
+  #ifdef _WIN32
+  setmode(fd, O_BINARY);
+  #endif
 
   key->data_size = RSTRING_LEN(data);
   nwrite = write(fd, key, KEY_SIZE);

data/lib/bootsnap/compile_cache/iseq.rb

@@ -12,12 +12,6 @@ module Bootsnap
         RubyVM::InstructionSequence.compile_file(path).to_binary
       rescue SyntaxError
         raise Uncompilable, 'syntax error'
-      rescue RuntimeError => e
-        if e.message == 'should not compile with coverage'
-          raise Uncompilable, 'coverage is enabled'
-        else
-          raise
-        end
       end
 
       def self.storage_to_output(binary)
@@ -37,6 +31,9 @@ module Bootsnap
 
       module InstructionSequenceMixin
         def load_iseq(path)
+          # Having coverage enabled prevents iseq dumping/loading.
+          return nil if defined?(Coverage) && Bootsnap::CompileCache::Native.coverage_running?
+
           Bootsnap::CompileCache::Native.fetch(
             Bootsnap::CompileCache::ISeq.cache_dir,
             path.to_s,
@@ -47,18 +44,6 @@ module Bootsnap
             puts "unmatched platform for file #{path}"
           end
           raise
-        rescue Errno::ERANGE
-          STDERR.puts <<~EOF
-            \x1b[31mError loading ISeq from cache for \x1b[1;34m#{path}\x1b[0;31m!
-            You can likely fix this by running:
-              \x1b[1;32mxattr -c #{path}
-            \x1b[0;31m...but, first, please make sure \x1b[1;34m@burke\x1b[0;31m knows you ran into this bug!
-            He will want to see the results of:
-              \x1b[1;32m/bin/ls -l@ #{path}
-            \x1b[0;31mand:
-              \x1b[1;32mxattr -p user.aotcc.key #{path}\x1b[0m
-          EOF
-          raise
         end
 
         def compile_option=(hash)

data/lib/bootsnap/compile_cache/yaml.rb

@@ -1,3 +1,5 @@
+require 'bootsnap/bootsnap'
+
 module Bootsnap
   module CompileCache
     module YAML

data/lib/bootsnap/compile_cache.rb

@@ -1,14 +1,13 @@
-require_relative 'compile_cache/iseq'
-require_relative 'compile_cache/yaml'
-
 module Bootsnap
   module CompileCache
     def self.setup(cache_dir:, iseq:, yaml:)
       if iseq
+        require_relative 'compile_cache/iseq'
         Bootsnap::CompileCache::ISeq.install!(cache_dir)
       end
 
       if yaml
+        require_relative 'compile_cache/yaml'
         Bootsnap::CompileCache::YAML.install!(cache_dir)
       end
     end

data/lib/bootsnap/load_path_cache/cache.rb

@@ -48,7 +48,7 @@ module Bootsnap
       def find(feature)
         reinitialize if (@has_relative_paths && dir_changed?) || stale?
         feature = feature.to_s
-        return feature if feature.start_with?(SLASH)
+        return feature if absolute_path?(feature)
         return File.expand_path(feature) if feature.start_with?('./')
         @mutex.synchronize do
           x = search_index(feature)
@@ -85,6 +85,16 @@ module Bootsnap
         end
       end
 
+      if RbConfig::CONFIG['host_os'] =~ /mswin|mingw|cygwin/
+        def absolute_path?(path)
+          path[1] == ':'
+        end
+      else
+        def absolute_path?(path)
+          path.start_with?(SLASH)
+        end
+      end
+
       def unshift_paths(sender, *paths)
         return unless sender == @path_obj
         @mutex.synchronize { unshift_paths_locked(*paths) }

data/lib/bootsnap/load_path_cache/path.rb

@@ -99,6 +99,8 @@ module Bootsnap
         @stability ||= begin
           if Gem.path.detect { |p| expanded_path.start_with?(p.to_s) }
             STABLE
+          elsif expanded_path.start_with?(Bundler.bundle_path.to_s)
+            STABLE
           elsif expanded_path.start_with?(RUBY_LIBDIR) && !expanded_path.start_with?(RUBY_SITEDIR)
             STABLE
           else

data/lib/bootsnap/setup.rb

@@ -0,0 +1,47 @@
+require_relative '../bootsnap'
+
+env = ENV['RAILS_ENV'] || ENV['RACK_ENV'] || ENV['ENV']
+development_mode = ['', nil, 'development'].include?(env)
+
+# only enable on 'ruby' (MRI), POSIX (darin, linux, *bsd), and >= 2.3.0
+enable_cc = \
+  RUBY_ENGINE == 'ruby' && \
+  RUBY_PLATFORM =~ /darwin|linux|bsd/ && \
+  RUBY_VERSION                    # "1.9.3"
+    .split('.')                   # ["1", "9", "3"]
+    .map(&:to_i)                  # [1, 9, 3]
+    .zip([2, 3, -1])              # [[1, 2], [9, 3], [3, -1]]
+    .map { |a, b| a <=> b }       # [-1, 1, 1]
+    .detect { |e| !e.zero? }      # -1
+    .==(1)                        # false
+
+cache_dir = ENV['BOOTSNAP_CACHE_DIR']
+unless cache_dir
+  config_dir_frame = caller.detect do |line|
+    line.include?('/config/')
+  end
+
+  unless config_dir_frame
+    $stderr.puts "[bootsnap/setup] couldn't infer cache directory! Either:"
+    $stderr.puts "[bootsnap/setup]   1. require bootsnap/setup from your application's config directory; or"
+    $stderr.puts "[bootsnap/setup]   2. Define the environment variable BOOTSNAP_CACHE_DIR"
+
+    raise "couldn't infer bootsnap cache directory"
+  end
+
+  path = config_dir_frame.split(/:\d+:/).first
+  path = File.dirname(path) until File.basename(path) == 'config'
+  app_root = File.dirname(path)
+
+  cache_dir = File.join(app_root, 'tmp', 'cache')
+end
+
+Bootsnap.setup(
+  cache_dir:            cache_dir,
+  development_mode:     development_mode,
+  load_path_cache:      true,
+  autoload_paths_cache: true, # assume rails. open to PRs to impl. detection
+  disable_trace:        false,
+  compile_cache_iseq:   enable_cc,
+  compile_cache_yaml:   enable_cc
+)

data/lib/bootsnap/version.rb

@@ -1,3 +1,3 @@
 module Bootsnap
-  VERSION = "1.0.0"
+  VERSION = "1.1.0"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: bootsnap
 version: !ruby/object:Gem::Version
-  version: 1.0.0
+  version: 1.1.0
 platform: ruby
 authors:
 - Burke Libbey
 autorequire: 
-bindir: exe
+bindir: bin
 cert_chain: []
-date: 2017-06-01 00:00:00.000000000 Z
+date: 2017-06-19 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler
@@ -80,20 +80,6 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '1.2'
-- !ruby/object:Gem::Dependency
-  name: ffi-xattr
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: 0.1.2
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: 0.1.2
 - !ruby/object:Gem::Dependency
   name: msgpack
   requirement: !ruby/object:Gem::Requirement
@@ -146,6 +132,7 @@ files:
 - lib/bootsnap/load_path_cache/path.rb
 - lib/bootsnap/load_path_cache/path_scanner.rb
 - lib/bootsnap/load_path_cache/store.rb
+- lib/bootsnap/setup.rb
 - lib/bootsnap/version.rb
 homepage: https://github.com/Shopify/bootsnap
 licenses:
@@ -159,7 +146,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
-      version: 2.3.0
+      version: 2.0.0
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="