bootsnap 1.1.0-java

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 23456864a83d7cb5c0628ac3b4ab63ed73012bf1
+  data.tar.gz: 9abcf27c44220fd9d3330ee154c6a39f3e4854d2
+SHA512:
+  metadata.gz: 6661871e089bf414b5e828408eeddb2def5acb8d88d9d1cb2692140768ea9891ff1c5e0011544a935105dcdfd0e4a863e6060156404e30b73a3613bb6a493256
+  data.tar.gz: b1a51caa8730f9be7f16e6a67b308b824f1fcc4817f933abebf8bf1b479cacad3ab66bc9b11e6ab91043eb79d1b60f736f0265f5e0d750db16b63e759df7eef1

data/.gitignore

@@ -0,0 +1,17 @@
+/.bundle/
+/.yardoc
+/Gemfile.lock
+/_yardoc/
+/coverage/
+/doc/
+/pkg/
+/spec/reports/
+/tmp/
+*.bundle
+*.so
+*.o
+*.a
+*.gem
+*.db
+mkmf.log
+.rubocop-*

data/.rubocop.yml

@@ -0,0 +1,7 @@
+inherit_from:
+  - http://shopify.github.io/ruby-style-guide/rubocop.yml
+
+AllCops:
+  Exclude:
+    - 'vendor/**/*'
+

data/.travis.yml

@@ -0,0 +1,5 @@
+os: osx
+language: ruby
+rvm: ruby-2.4.0
+before_script: rake
+script: bin/testunit

data/CHANGELOG.md

@@ -0,0 +1,31 @@
+# 1.1.0
+
+* Add `bootsnap/setup`
+* Support jruby (without compile caching features)
+* Better deoptimization when Coverage is enabled
+* Consider `Bundler.bundle_path` to be stable
+
+# 1.0.0
+
+* (none)
+
+# 0.3.2
+
+* Minor performance savings around checking validity of cache in the presence of relative paths.
+* When coverage is enabled, skips optimization instead of exploding.
+
+# 0.3.1
+
+* Don't whitelist paths under `RbConfig::CONFIG['prefix']` as stable; instead use `['libdir']` (#41).
+* Catch `EOFError` when reading load-path-cache and regenerate cache.
+* Support relative paths in load-path-cache.
+
+# 0.3.0
+
+* Migrate CompileCache from xattr as a cache backend to a cache directory
+    * Adds support for Linux and FreeBSD
+
+# 0.2.15
+
+* Support more versions of ActiveSupport (`depend_on`'s signature varies; don't reiterate it)
+* Fix bug in handling autoloaded modules that raise NoMethodError

data/CONTRIBUTING.md

@@ -0,0 +1,21 @@
+# Contributing to Bootsnap
+
+We love receiving pull requests!
+
+## Standards
+
+* PR should explain what the feature does, and why the change exists.
+* PR should include any carrier specific documentation explaining how it works.
+* Code _must_ be tested, including both unit and remote tests where applicable.
+* Be consistent. Write clean code that follows [Ruby community standards](https://github.com/bbatsov/ruby-style-guide).
+* Code should be generic and reusable.
+
+If you're stuck, ask questions!
+
+## How to contribute
+
+1. Fork it ( https://github.com/Shopify/bootsnap/fork )
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am 'Add some feature'`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. Create a new Pull Request

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in bootsnap.gemspec
+gemspec

data/LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) 2017 Shopify
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,284 @@
+# Bootsnap [![Build Status](https://travis-ci.org/Shopify/bootsnap.svg?branch=master)](https://travis-ci.org/Shopify/bootsnap)
+
+**Beta-quality. See [the last section of this README](#trustworthiness).**
+
+Bootsnap is a library that plugs into Ruby, with optional support for `ActiveSupport` and `YAML`,
+to optimize and cache expensive computations. See [How Does This Work](#how-does-this-work).
+
+#### Performance
+- [Discourse](https://github.com/discourse/discourse) reports a boot time reduction of approximately 50%, from roughly 6 to 3 seconds on one machine;
+- One of our smaller internal apps also sees a reduction of 50%, from 3.6 to 1.8 seconds;
+- The core Shopify platform -- a rather large monolithic application -- boots about 75% faster, dropping from around 25s to 6.5s.
+
+## Usage
+
+This gem works on MacOS and Linux.
+
+Add `bootsnap` to your `Gemfile`:
+
+```ruby
+gem 'bootsnap'
+```
+
+If you are using rails, add this to `config/boot.rb` immediately after `require 'bundler/setup'`:
+
+```ruby
+require 'bootsnap/setup'
+```
+
+If you are not using rails, or if you are but want more control over things, add this to your
+application setup immediately after `require 'bundler/setup'` (i.e. as early as possible: the sooner
+this is loaded, the sooner it can start optimizing things)
+
+```ruby
+require 'bootsnap'
+env = ENV['RAILS_ENV'] || "development"
+Bootsnap.setup(
+  cache_dir:            'tmp/cache',          # Path to your cache
+  development_mode:     env == 'development', # Current working environment, e.g. RACK_ENV, RAILS_ENV, etc
+  load_path_cache:      true,                 # Optimize the LOAD_PATH with a cache
+  autoload_paths_cache: true,                 # Optimize ActiveSupport autoloads with cache
+  disable_trace:        true,                 # (Alpha) Set `RubyVM::InstructionSequence.compile_option = { trace_instruction: false }`
+  compile_cache_iseq:   true,                 # Compile Ruby code into ISeq cache, breaks coverage reporting.
+  compile_cache_yaml:   true                  # Compile YAML into a cache
+)
+```
+
+**Protip:** You can replace `require 'bootsnap'` with `BootLib::Require.from_gem('bootsnap',
+'bootsnap')` using [this trick](https://github.com/Shopify/bootsnap/wiki/Bootlib::Require). This
+will help optimize boot time further if you have an extremely large `$LOAD_PATH`.
+
+## How does this work?
+
+Bootsnap optimizes methods to cache results of expensive computations, and can be grouped
+into two broad categories:
+
+* [Path Pre-Scanning](#path-pre-scanning)
+    * `Kernel#require` and `Kernel#load` are modified to eliminate `$LOAD_PATH` scans.
+    * `ActiveSupport::Dependencies.{autoloadable_module?,load_missing_constant,depend_on}` are
+      overridden to eliminate scans of `ActiveSupport::Dependencies.autoload_paths`.
+* [Compilation caching](#compilation-caching)
+    * `RubyVM::InstructionSequence.load_iseq` is implemented to cache the result of ruby bytecode
+      compilation.
+    * `YAML.load_file` is modified to cache the result of loading a YAML object in MessagePack format
+      (or Marshal, if the message uses types unsupported by MessagePack).
+
+### Path Pre-Scanning
+
+*(This work is a minor evolution of [bootscale](https://github.com/byroot/bootscale)).*
+
+Upon initialization of bootsnap or modification of the path (e.g. `$LOAD_PATH`),
+`Bootsnap::LoadPathCache` will fetch a list of requirable entries from a cache, or, if necessary,
+perform a full scan and cache the result.
+
+Later, when we run (e.g.) `require 'foo'`, ruby *would* iterate through every item on our
+`$LOAD_PATH` `['x', 'y', ...]`,  looking for `x/foo.rb`, `y/foo.rb`, and so on. Bootsnap instead
+looks at all the cached requirables for each `$LOAD_PATH` entry and substitutes the full expanded
+path of the match ruby would have eventually chosen.
+
+If you look at the syscalls generated by this behaviour, the net effect is that what would
+previously look like this:
+
+```
+open  x/foo.rb # (fail)
+# (imagine this with 500 $LOAD_PATH entries instead of two)
+open  y/foo.rb # (success)
+close y/foo.rb
+open  y/foo.rb
+...
+```
+
+becomes this:
+
+```
+open y/foo.rb
+...
+```
+
+Exactly the same strategy is employed for methods that traverse
+`ActiveSupport::Dependencies.autoload_paths` if the `autoload_paths_cache` option is given to
+`Bootsnap.setup`.
+
+The following diagram flowcharts the overrides that make the `*_path_cache` features work.
+
+![Flowchart explaining
+Bootsnap](https://cloud.githubusercontent.com/assets/3074765/24532120/eed94e64-158b-11e7-9137-438d759b2ac8.png)
+
+Bootsnap classifies path entries into two categories: stable and volatile. Volatile entries are
+scanned each time the application boots, and their caches are only valid for 30 seconds. Stable
+entries do not expire -- once their contents has been scanned, it is assumed to never change.
+
+The only directories considered "stable" are things under the Ruby install prefix
+(`RbConfig::CONFIG['prefix']`, e.g. `/usr/local/ruby` or `~/.rubies/x.y.z`), and things under the
+`Gem.path` (e.g. `~/.gem/ruby/x.y.z`) or `Bundler.bundle_path`. Everything else is considered
+"volatile".
+
+In addition to the [`Bootsnap::LoadPathCache::Cache`
+source](https://github.com/Shopify/bootsnap/blob/master/lib/bootsnap/load_path_cache/cache.rb),
+this diagram may help clarify how entry resolution works:
+
+![How path searching works](https://cloud.githubusercontent.com/assets/3074765/25388270/670b5652-299b-11e7-87fb-975647f68981.png)
+
+
+It's also important to note how expensive `LoadError`s can be. If ruby invokes
+`require 'something'`, but that file isn't on `$LOAD_PATH`, it takes `2 *
+$LOAD_PATH.length` filesystem accesses to determine that. Bootsnap caches this
+result too, raising a `LoadError` without touching the filesystem at all.
+
+### Compilation Caching
+
+*(A more readable implementation of this concept can be found in
+[yomikomu](https://github.com/ko1/yomikomu)).*
+
+Ruby has complex grammar and parsing it is not a particularly cheap operation. Since 1.9, Ruby has
+translated ruby source to an internal bytecode format, which is then executed by the Ruby VM. Since
+2.2, Ruby [exposes an API](https://ruby-doc.org/core-2.3.0/RubyVM/InstructionSequence.html) that
+allows caching that bytecode. This allows us to bypass the relatively-expensive compilation step on
+subsequent loads of the same file.
+
+We also noticed that we spend a lot of time loading YAML documents during our application boot, and
+that MessagePack and Marshal are *much* faster at deserialization than YAML, even with a fast
+implementation. We use the same strategy of compilation caching for YAML documents, with the
+equivalent of Ruby's "bytecode" format being a MessagePack document (or, in the case of YAML
+documents with types unsupported by MessagePack, a Marshal stream).
+
+These compilation results are stored in a cache directory, with filenames generated by taking a hash
+of the full expanded path of the input file (FNV1a-64).
+
+Whereas before, the sequence of syscalls generated to `require` a file would look like:
+
+```
+open    /c/foo.rb -> m
+fstat64 m
+close   m
+open    /c/foo.rb -> o
+fstat64 o
+fstat64 o
+read    o
+read    o
+...
+close   o
+```
+
+With bootsnap, we get:
+
+```
+open      /c/foo.rb -> n
+fstat64   n
+close     n
+open      /c/foo.rb -> n
+fstat64   n
+open      (cache) -> m
+read      m
+read      m
+close     m
+close     n
+```
+
+This may look worse at a glance, but underlies a large performance difference.
+
+*(The first three syscalls in both listings -- `open`, `fstat64`, `close` -- are not inherently
+useful. [This ruby patch](https://bugs.ruby-lang.org/issues/13378) optimizes them out when coupled
+with bootsnap.)*
+
+Bootsnap writes a cache file containing a 64 byte header followed by the cache contents. The header
+is a cache key including several fields:
+
+* `version`, hardcoded in bootsnap. Essentially a schema version;
+* `os_version`, A hash of the current kernel version (on macOS, BSD) or glibc version (on Linux);
+* `compile_option`, which changes with `RubyVM::InstructionSequence.compile_option` does;
+* `ruby_revision`, the version of Ruby this was compiled with;
+* `size`, the size of the source file;
+* `mtime`, the last-modification timestamp of the source file when it was compiled; and
+* `data_size`, the number of bytes following the header, which we need to read it into a buffer.
+
+If the key is valid, the result is loaded from the value. Otherwise, it is regenerated and clobbers
+the current cache.
+
+### Putting it all together
+
+Imagine we have this file structure:
+
+```
+/
+├── a
+├── b
+└── c
+    └── foo.rb
+```
+
+And this `$LOAD_PATH`:
+
+```
+["/a", "/b", "/c"]
+```
+
+When we call `require 'foo'` without bootsnap, Ruby would generate this sequence of syscalls:
+
+
+```
+open    /a/foo.rb -> -1
+open    /b/foo.rb -> -1
+open    /c/foo.rb -> n
+close   n
+open    /c/foo.rb -> m
+fstat64 m
+close   m
+open    /c/foo.rb -> o
+fstat64 o
+fstat64 o
+read    o
+read    o
+...
+close   o
+```
+
+With bootsnap, we get:
+
+```
+open      /c/foo.rb -> n
+fstat64   n
+close     n
+open      /c/foo.rb -> n
+fstat64   n
+open      (cache) -> m
+read      m
+read      m
+close     m
+close     n
+```
+
+If we call `require 'nope'` without bootsnap, we get:
+
+```
+open    /a/nope.rb -> -1
+open    /b/nope.rb -> -1
+open    /c/nope.rb -> -1
+open    /a/nope.bundle -> -1
+open    /b/nope.bundle -> -1
+open    /c/nope.bundle -> -1
+```
+
+...and if we call `require 'nope'` *with* bootsnap, we get...
+
+```
+# (nothing!)
+```
+
+## Trustworthiness
+
+We use the `*_path_cache` features in production and haven't experienced any issues in a long time.
+
+The `compile_cache_*` features work well for us in development on macOS. It should work on Linux,
+and we intend to deploy it in production, but we haven't yet.
+
+`disable_trace` should be completely safe, but we don't really use it because some people like to
+use tools that make use of `trace` instructions.
+
+| feature | where we're using it |
+|-|-|
+| `load_path_cache` | everywhere |
+| `autoload_path_cache` | everywhere |
+| `disable_trace` | nowhere, but it's safe unless you need tracing |
+| `compile_cache_iseq` | development, but probably safe to use everywhere |
+| `compile_cache_yaml` | development, but probably safe to use everywhere |

data/Rakefile

@@ -0,0 +1,11 @@
+require 'rake/extensiontask'
+
+gemspec = Gem::Specification.load('bootsnap.gemspec')
+Rake::ExtensionTask.new do |ext|
+  ext.name = 'bootsnap'
+  ext.ext_dir = 'ext/bootsnap'
+  ext.lib_dir = 'lib/bootsnap'
+  ext.gem_spec = gemspec
+end
+
+task(default: :compile)

data/bin/console

@@ -0,0 +1,14 @@
+#!/usr/bin/env ruby
+
+require "bundler/setup"
+require "bootsnap"
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+# (If you use this, don't forget to add pry to your Gemfile!)
+# require "pry"
+# Pry.start
+
+require "irb"
+IRB.start(__FILE__)

data/bin/setup

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+set -euo pipefail
+IFS=$'\n\t'
+set -vx
+
+bundle install
+
+# Do any other automated setup that you need to do here

data/bin/testunit

@@ -0,0 +1,8 @@
+#!/bin/bash
+
+if [ $# -eq 0 ]; then
+  ruby -I"test" -e 'Dir.glob("./test/**/*_test.rb").each { |f| require f }' -- "$@"
+else
+  path=$1
+  ruby -I"test" -e "require '${path#test/}'" -- "$@"
+fi