bootsnap 0.2.9 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: fe31f9a445fa33edd58c5f58e0f661c0e08d2004
-  data.tar.gz: e01345ff505baf0ce9aa6a104325a93092d0ab53
+  metadata.gz: 3725f53309883dce83894e47eedbb4d8322eabd5
+  data.tar.gz: 81e14b1d48fc4431f6ac9bb2729cfb4a69aa0f5c
 SHA512:
-  metadata.gz: c319a87aa7d1db72c3535e0af539cd8629852238b4b88bcc1b4a7cf57ea978707aa82b879730815e293c5944b4c40d8b2531a8639780491853c58b66de01191e
-  data.tar.gz: d2d6042b09014425a6d4feb3d1abe28c1a1eaeddd1c29fff8752ba907322349129d065c38109b6d460438b667a72b04626165a59cde12f7af509c9043ba254e5
+  metadata.gz: 5a2b45cb55ceffdb5751561b3a35c476ea29bcd3d3a2a68a46521240d35cc1e61cf9ac90265db751b09967655969c7a685725878b5f80a61fd38a0efd7024458
+  data.tar.gz: 7f7975023cb1856d5ccada63e8be2e3ab94a751d4b80121821f2b01df6cbb94f5172041020ebb9300b63760da3fc2776e3b9b56160b95b4b0c07f82fbce07860

data/.travis.yml

@@ -0,0 +1,5 @@
+os: osx
+language: ruby
+rvm: ruby-2.4.0
+before_script: rake
+script: bin/testunit

data/CHANGELOG.md

@@ -0,0 +1,9 @@
+# 0.3.0
+
+* Migrate CompileCache from xattr as a cache backend to a cache directory
+    * Adds support for Linux and FreeBSD
+
+# 0.2.15
+
+* Support more versions of ActiveSupport (`depend_on`'s signature varies; don't reiterate it)
+* Fix bug in handling autoloaded modules that raise NoMethodError

data/README.md

@@ -1,10 +1,15 @@
-# Bootsnap
+# Bootsnap ![CI status](https://travis-ci.org/Shopify/bootsnap.svg?branch=master)
 
 **Beta-quality. See [the last section of this README](#trustworthiness).**
 
 Bootsnap is a library that plugs into a number of Ruby and (optionally) `ActiveSupport` and `YAML`
 methods to optimize and cache expensive computations. See [the How Does This Work section](#how-does-this-work) for more information.
 
+#### Quick Performance Overview
+- [Discourse](https://github.com/discourse/discourse) reports a boot time reduction of approximately 50%, from roughly 6 to 3 seconds on one machine;
+- One of our smaller internal apps also sees a reduction of 50%, from 3.6 to 1.8 seconds;
+- The core Shopify platform -- a rather large monolithic application -- boots about 75% faster, dropping from around 25s to 6.5s.
+
 ## Usage
 
 Add `bootsnap` to your `Gemfile`:
@@ -112,7 +117,8 @@ result too, raising a `LoadError` without touching the filesystem at all.
 
 ### Compilation Caching
 
-*(A simpler implementation of this concept can be found in [yomikomu](https://github.com/ko1/yomikomu)).*
+*(A more readable implementation of this concept can be found in
+[yomikomu](https://github.com/ko1/yomikomu)).*
 
 Ruby has complex grammar and parsing it is not a particularly cheap operation. Since 1.9, Ruby has
 translated ruby source to an internal bytecode format, which is then executed by the Ruby VM. Since
@@ -126,9 +132,8 @@ implementation. We use the same strategy of compilation caching for YAML documen
 equivalent of Ruby's "bytecode" format being a MessagePack document (or, in the case of YAML
 documents with types unsupported by MessagePack, a Marshal stream).
 
-These compilation results are stored using `xattr`s on the source files. This is likely to change in
-the future, as it has some limitations (notably precluding Linux support except where the user feels
-like changing mount flags). However, this is a very performant implementation. 
+These compilation results are stored in a cache directory, with filenames generated by taking a hash
+of the full expanded path of the input file (FNV1a-64).
 
 Whereas before, the sequence of syscalls generated to `require` a file would look like:
 
@@ -150,32 +155,36 @@ With bootsnap, we get:
 ```
 open      /c/foo.rb -> n
 fstat64   n
-fgetxattr n
-fgetxattr n
+close     n
+open      /c/foo.rb -> n
+fstat64   n
+open      (cache) -> m
+read      m
+read      m
+close     m
 close     n
 ```
 
-Bootsnap writes two `xattrs` attached to each file read:
+This may look worse at a glance, but underlies a large performance difference.
 
-* `user.aotcc.value`, the binary compilation result; and
-* `user.aotcc.key`, a cache key to determine whether `user.aotcc.value` is still valid.
+*(The first three syscalls in both listings -- `open`, `fstat64`, `close` -- are not inherently
+useful. [This ruby patch](https://bugs.ruby-lang.org/issues/13378) optimizes them out when coupled
+with bootsnap.)*
 
-The key includes several fields:
+Bootsnap writes a cache file containing a 64 byte header followed by the cache contents. The header
+is a cache key including several fields:
 
 * `version`, hardcoded in bootsnap. Essentially a schema version;
+* `os_version`, A hash of the current kernel version (on macOS, BSD) or glibc version (on Linux);
 * `compile_option`, which changes with `RubyVM::InstructionSequence.compile_option` does;
-* `data_size`, the number of bytes in `user.aotcc.value`, which we need to read it into a buffer
-  using `fgetxattr(2)`;
-* `ruby_revision`, the version of Ruby this was compiled with; and
-* `mtime`, the last-modification timestamp of the source file when it was compiled.
+* `ruby_revision`, the version of Ruby this was compiled with;
+* `size`, the size of the source file;
+* `mtime`, the last-modification timestamp of the source file when it was compiled; and
+* `data_size`, the number of bytes following the header, which we need to read it into a buffer.
 
 If the key is valid, the result is loaded from the value. Otherwise, it is regenerated and clobbers
 the current cache.
 
-This diagram may help illustrate how it works:
-
-![Compilation Caching](https://burkelibbey.s3.amazonaws.com/bootsnap-compile-cache.png)
-
 ### Putting it all together
 
 Imagine we have this file structure:
@@ -219,8 +228,13 @@ With bootsnap, we get:
 ```
 open      /c/foo.rb -> n
 fstat64   n
-fgetxattr n
-fgetxattr n
+close     n
+open      /c/foo.rb -> n
+fstat64   n
+open      (cache) -> m
+read      m
+read      m
+close     m
 close     n
 ```
 
@@ -245,8 +259,8 @@ open    /c/nope.bundle -> -1
 
 We use the `*_path_cache` features in production and haven't experienced any issues in a long time.
 
-The `compile_cache_*` features work well for us in development on macOS, but probably don't work on
-Linux at all.
+The `compile_cache_*` features work well for us in development on macOS. It should work on Linux,
+and we intend to deploy it in production, but haven't we haven't yet.
 
 `disable_trace` should be completely safe, but we don't really use it because some people like to
 use tools that make use of `trace` instructions.
@@ -256,5 +270,5 @@ use tools that make use of `trace` instructions.
 | `load_path_cache` | everywhere |
 | `autoload_path_cache` | everywhere |
 | `disable_trace` | nowhere, but it's safe unless you need tracing |
-| `compile_cache_iseq` | development, unlikely to work on Linux |
-| `compile_cache_yaml` | development, unlikely to work on Linux |
+| `compile_cache_iseq` | development, but probably safe to use everywhere |
+| `compile_cache_yaml` | development, but probably safe to use everywhere |

data/bootsnap.gemspec

@@ -23,12 +23,14 @@ Gem::Specification.new do |spec|
   spec.require_paths = ["lib"]
   spec.extensions    = ['ext/bootsnap/extconf.rb']
 
+  spec.required_ruby_version = '>= 2.3.0'
+
   spec.add_development_dependency "bundler", '~> 1'
   spec.add_development_dependency 'rake', '~> 10.0'
   spec.add_development_dependency 'rake-compiler', '~> 0'
   spec.add_development_dependency "minitest", "~> 5.0"
   spec.add_development_dependency "mocha", "~> 1.2"
+  spec.add_development_dependency "ffi-xattr", "~> 0.1.2"
 
   spec.add_runtime_dependency "msgpack", "~> 1.0"
-  spec.add_runtime_dependency "snappy", "~> 0.0.15"
 end

data/dev.yml

@@ -1,3 +1,6 @@
+env:
+  BOOTSNAP_PEDANTIC: '1'
+
 up:
   - ruby: 2.3.3
   - bundler