bootsnap 1.4.0 → 1.9.1

Sign up to get free protection for your applications and to get access to all the features.
Files changed (45) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +126 -0
  3. data/README.md +68 -13
  4. data/exe/bootsnap +5 -0
  5. data/ext/bootsnap/bootsnap.c +285 -87
  6. data/ext/bootsnap/extconf.rb +20 -14
  7. data/lib/bootsnap/bundler.rb +1 -0
  8. data/lib/bootsnap/cli/worker_pool.rb +135 -0
  9. data/lib/bootsnap/cli.rb +281 -0
  10. data/lib/bootsnap/compile_cache/iseq.rb +24 -7
  11. data/lib/bootsnap/compile_cache/json.rb +79 -0
  12. data/lib/bootsnap/compile_cache/yaml.rb +145 -39
  13. data/lib/bootsnap/compile_cache.rb +25 -3
  14. data/lib/bootsnap/explicit_require.rb +1 -0
  15. data/lib/bootsnap/load_path_cache/cache.rb +44 -9
  16. data/lib/bootsnap/load_path_cache/change_observer.rb +5 -1
  17. data/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb +30 -6
  18. data/lib/bootsnap/load_path_cache/core_ext/loaded_features.rb +11 -0
  19. data/lib/bootsnap/load_path_cache/loaded_features_index.rb +43 -11
  20. data/lib/bootsnap/load_path_cache/path.rb +3 -2
  21. data/lib/bootsnap/load_path_cache/path_scanner.rb +53 -27
  22. data/lib/bootsnap/load_path_cache/realpath_cache.rb +5 -5
  23. data/lib/bootsnap/load_path_cache/store.rb +28 -14
  24. data/lib/bootsnap/load_path_cache.rb +10 -16
  25. data/lib/bootsnap/setup.rb +2 -33
  26. data/lib/bootsnap/version.rb +2 -1
  27. data/lib/bootsnap.rb +96 -17
  28. metadata +18 -29
  29. data/.gitignore +0 -17
  30. data/.rubocop.yml +0 -20
  31. data/.travis.yml +0 -24
  32. data/CODE_OF_CONDUCT.md +0 -74
  33. data/CONTRIBUTING.md +0 -21
  34. data/Gemfile +0 -8
  35. data/README.jp.md +0 -231
  36. data/Rakefile +0 -12
  37. data/bin/ci +0 -10
  38. data/bin/console +0 -14
  39. data/bin/setup +0 -8
  40. data/bin/test-minimal-support +0 -7
  41. data/bin/testunit +0 -8
  42. data/bootsnap.gemspec +0 -45
  43. data/dev.yml +0 -10
  44. data/lib/bootsnap/load_path_cache/core_ext/active_support.rb +0 -100
  45. data/shipit.rubygems.yml +0 -4
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 40694ca2512ff7861584a4f9cee05e424c3d9319f3b36c7f4f45f9196d2d9f19
4
- data.tar.gz: '087c4f2dafdd51a6a3d0a69d31aadabfe6ef90bf2ab12b861f2a288d7b16c04a'
3
+ metadata.gz: 00c1b5b2c37c6a351bb7656ef17ba7f7e6143fd773ceebd6ee09d9a55e25b75a
4
+ data.tar.gz: d6d6d1d68d97e4f2c50c61f0b928714cf94f29ecab6604bb0092517779212738
5
5
  SHA512:
6
- metadata.gz: ecbabde06c5f871702a000128e4dbc15d60fe0e16ab7adb4a8bb40b1c3b5b5f54b39c6927dc5aea9e49242d7e7f5deb596c42c44f8c2b8f998f835ec5b058b47
7
- data.tar.gz: d566ffe7c3048f57c3c4fcd925d5e8ff5a722fc2622628e26571361e7a0fbdb5cbe8c724d45f2fc8ba09d0de614c5fad87f1875ecc99a08a34aab3afa7665fe1
6
+ metadata.gz: e1ade83cee1e4a917317cecd34107493424a5174b8bf92cc74db47743d4d84931de24533979480a23aa4ddb496cae8ce1f3cf30a2fbed3996b885a68138d16f0
7
+ data.tar.gz: 23e4f6c77307b30526b880bfd12983b7e66c9539556acd3bc05a2fa3fd6884a142c83dfaa590846651afad243c0bf69c96d994ca8492e5539d706ba3e6cf713f
data/CHANGELOG.md CHANGED
@@ -1,3 +1,129 @@
1
+ # Unreleased
2
+
3
+ # 1.9.1
4
+
5
+ * Removed a forgotten debug statement in JSON precompilation.
6
+
7
+ # 1.9.0
8
+
9
+ * Added a compilation cache for `JSON.load_file`. (#370)
10
+
11
+ # 1.8.1
12
+
13
+ * Fixed support for older Psych. (#369)
14
+
15
+ # 1.8.0
16
+
17
+ * Improve support for Pysch 4. (#368)
18
+
19
+ # 1.7.7
20
+
21
+ * Fix `require_relative` in evaled code on latest ruby 3.1.0-dev. (#366)
22
+
23
+ # 1.7.6
24
+
25
+ * Fix reliance on `set` to be required.
26
+ * Fix `Encoding::UndefinedConversionError` error for Rails applications when precompiling cache. (#364)
27
+
28
+ # 1.7.5
29
+
30
+ * Handle a regression of Ruby 2.7.3 causing Bootsnap to call the deprecated `untaint` method. (#360)
31
+ * Gracefully handle read-only file system as well as other errors preventing to persist the load path cache. (#358)
32
+
33
+ # 1.7.4
34
+
35
+ * Stop raising errors when encoutering various file system errors. The cache is now best effort,
36
+ if somehow it can't be saved, bootsnapp will gracefully fallback to the original operation (e.g. `Kernel.require`).
37
+ (#353, #177, #262)
38
+
39
+ # 1.7.3
40
+
41
+ * Disable YAML precompilation when encountering YAML tags. (#351)
42
+
43
+ # 1.7.2
44
+
45
+ * Fix compatibility with msgpack < 1. (#349)
46
+
47
+ # 1.7.1
48
+
49
+ * Warn Ruby 2.5 users if they turn ISeq caching on. (#327, #244)
50
+ * Disable ISeq caching for the whole 2.5.x series again.
51
+ * Better handle hashing of Ruby strings. (#318)
52
+
53
+ # 1.7.0
54
+
55
+ * Fix detection of YAML files in gems.
56
+ * Adds an instrumentation API to monitor cache misses.
57
+ * Allow to control the behavior of `require 'bootsnap/setup'` using environment variables.
58
+ * Deprecate the `disable_trace` option.
59
+ * Deprecate the `ActiveSupport::Dependencies` (AKA Classic autoloader) integration. (#344)
60
+
61
+ # 1.6.0
62
+
63
+ * Fix a Ruby 2.7/3.0 issue with `YAML.load_file` keyword arguments. (#342)
64
+ * `bootsnap precompile` CLI use multiple processes to complete faster. (#341)
65
+ * `bootsnap precompile` CLI also precompile YAML files. (#340)
66
+ * Changed the load path cache directory from `$BOOTSNAP_CACHE_DIR/bootsnap-load-path-cache` to `$BOOTSNAP_CACHE_DIR/bootsnap/load-path-cache` for ease of use. (#334)
67
+ * Changed the compile cache directory from `$BOOTSNAP_CACHE_DIR/bootsnap-compile-cache` to `$BOOTSNAP_CACHE_DIR/bootsnap/compile-cache` for ease of use. (#334)
68
+
69
+ # 1.5.1
70
+
71
+ * Workaround a Ruby bug in InstructionSequence.compile_file. (#332)
72
+
73
+ # 1.5.0
74
+
75
+ * Add a command line to statically precompile the ISeq cache. (#326)
76
+
77
+ # 1.4.9
78
+
79
+ * [Windows support](https://github.com/Shopify/bootsnap/pull/319)
80
+ * [Fix potential crash](https://github.com/Shopify/bootsnap/pull/322)
81
+
82
+ # 1.4.8
83
+
84
+ * [Prevent FallbackScan from polluting exception cause](https://github.com/Shopify/bootsnap/pull/314)
85
+
86
+ # 1.4.7
87
+
88
+ * Various performance enhancements
89
+ * Fix race condition in heavy concurrent load scenarios that would cause bootsnap to raise
90
+
91
+ # 1.4.6
92
+
93
+ * Fix bug that was erroneously considering that files containing `.` in the names were being
94
+ required if a different file with the same name was already being required
95
+
96
+ Example:
97
+
98
+ require 'foo'
99
+ require 'foo.en'
100
+
101
+ Before bootsnap was considering `foo.en` to be the same file as `foo`
102
+
103
+ * Use glibc as part of the ruby_platform cache key
104
+
105
+ # 1.4.5
106
+
107
+ * MRI 2.7 support
108
+ * Fixed concurrency bugs
109
+
110
+ # 1.4.4
111
+
112
+ * Disable ISeq cache in `bootsnap/setup` by default in Ruby 2.5
113
+
114
+ # 1.4.3
115
+
116
+ * Fix some cache permissions and umask issues after switch to mkstemp
117
+
118
+ # 1.4.2
119
+
120
+ * Fix bug when removing features loaded by relative path from `$LOADED_FEATURES`
121
+ * Fix bug with propagation of `NameError` up from nested calls to `require`
122
+
123
+ # 1.4.1
124
+
125
+ * Don't register change observers to frozen objects.
126
+
1
127
  # 1.4.0
2
128
 
3
129
  * When running in development mode, always fall back to a full path scan on LoadError, making
data/README.md CHANGED
@@ -1,12 +1,17 @@
1
- # Bootsnap [![Build Status](https://travis-ci.org/Shopify/bootsnap.svg?branch=master)](https://travis-ci.org/Shopify/bootsnap)
1
+ # Bootsnap [![Actions Status](https://github.com/Shopify/bootsnap/workflows/ci/badge.svg)](https://github.com/Shopify/bootsnap/actions)
2
2
 
3
- Bootsnap is a library that plugs into Ruby, with optional support for `ActiveSupport` and `YAML`,
3
+ Bootsnap is a library that plugs into Ruby, with optional support for `YAML`,
4
4
  to optimize and cache expensive computations. See [How Does This Work](#how-does-this-work).
5
5
 
6
6
  #### Performance
7
- - [Discourse](https://github.com/discourse/discourse) reports a boot time reduction of approximately 50%, from roughly 6 to 3 seconds on one machine;
7
+
8
+ - [Discourse](https://github.com/discourse/discourse) reports a boot time reduction of approximately
9
+ 50%, from roughly 6 to 3 seconds on one machine;
8
10
  - One of our smaller internal apps also sees a reduction of 50%, from 3.6 to 1.8 seconds;
9
- - The core Shopify platform -- a rather large monolithic application -- boots about 75% faster, dropping from around 25s to 6.5s.
11
+ - The core Shopify platform -- a rather large monolithic application -- boots about 75% faster,
12
+ dropping from around 25s to 6.5s.
13
+ * In Shopify core (a large app), about 25% of this gain can be attributed to `compile_cache_*`
14
+ features; 75% to path caching. This is fairly representative.
10
15
 
11
16
  ## Usage
12
17
 
@@ -24,6 +29,15 @@ If you are using Rails, add this to `config/boot.rb` immediately after `require
24
29
  require 'bootsnap/setup'
25
30
  ```
26
31
 
32
+ Note that bootsnap writes to `tmp/cache` (or the path specified by `ENV['BOOTSNAP_CACHE_DIR']`),
33
+ and that directory *must* be writable. Rails will fail to
34
+ boot if it is not. If this is unacceptable (e.g. you are running in a read-only container and
35
+ unwilling to mount in a writable tmpdir), you should remove this line or wrap it in a conditional.
36
+
37
+ **Note also that bootsnap will never clean up its own cache: this is left up to you. Depending on your
38
+ deployment strategy, you may need to periodically purge `tmp/cache/bootsnap*`. If you notice deploys
39
+ getting progressively slower, this is almost certainly the cause.**
40
+
27
41
  It's technically possible to simply specify `gem 'bootsnap', require: 'bootsnap/setup'`, but it's
28
42
  important to load Bootsnap as early as possible to get maximum performance improvement.
29
43
 
@@ -40,8 +54,6 @@ Bootsnap.setup(
40
54
  cache_dir: 'tmp/cache', # Path to your cache
41
55
  development_mode: env == 'development', # Current working environment, e.g. RACK_ENV, RAILS_ENV, etc
42
56
  load_path_cache: true, # Optimize the LOAD_PATH with a cache
43
- autoload_paths_cache: true, # Optimize ActiveSupport autoloads with cache
44
- disable_trace: true, # (Alpha) Set `RubyVM::InstructionSequence.compile_option = { trace_instruction: false }`
45
57
  compile_cache_iseq: true, # Compile Ruby code into ISeq cache, breaks coverage reporting.
46
58
  compile_cache_yaml: true # Compile YAML into a cache
47
59
  )
@@ -56,12 +68,39 @@ speeds up the loading of individual source files, Spring keeps a copy of a pre-b
56
68
  on hand to completely skip parts of the boot process the next time it's needed. The two tools work
57
69
  well together, and are both included in a newly-generated Rails applications by default.
58
70
 
71
+ ### Environment variables
72
+
73
+ `require 'bootsnap/setup'` behavior can be changed using environment variables:
74
+
75
+ - `BOOTSNAP_CACHE_DIR` allows to define the cache location.
76
+ - `DISABLE_BOOTSNAP` allows to entirely disable bootsnap.
77
+ - `DISABLE_BOOTSNAP_LOAD_PATH_CACHE` allows to disable load path caching.
78
+ - `DISABLE_BOOTSNAP_COMPILE_CACHE` allows to disable ISeq and YAML caches.
79
+ - `BOOTSNAP_LOG` configure bootsnap to log all caches misses to STDERR.
80
+
59
81
  ### Environments
60
82
 
61
83
  All Bootsnap features are enabled in development, test, production, and all other environments according to the configuration in the setup. At Shopify, we use this gem safely in all environments without issue.
62
84
 
63
85
  If you would like to disable any feature for a certain environment, we suggest changing the configuration to take into account the appropriate ENV var or configuration according to your needs.
64
86
 
87
+ ### Instrumentation
88
+
89
+ Bootsnap cache misses can be monitored though a callback:
90
+
91
+ ```ruby
92
+ Bootsnap.instrumentation = ->(event, path) { puts "#{event} #{path}" }
93
+ ```
94
+
95
+ `event` is either `:miss` or `:stale`. You can also call `Bootsnap.log!` as a shortcut to
96
+ log all events to STDERR.
97
+
98
+ To turn instrumentation back off you can set it to nil:
99
+
100
+ ```ruby
101
+ Bootsnap.instrumentation = nil
102
+ ```
103
+
65
104
  ## How does this work?
66
105
 
67
106
  Bootsnap optimizes methods to cache results of expensive computations, and can be grouped
@@ -69,8 +108,6 @@ into two broad categories:
69
108
 
70
109
  * [Path Pre-Scanning](#path-pre-scanning)
71
110
  * `Kernel#require` and `Kernel#load` are modified to eliminate `$LOAD_PATH` scans.
72
- * `ActiveSupport::Dependencies.{autoloadable_module?,load_missing_constant,depend_on}` are
73
- overridden to eliminate scans of `ActiveSupport::Dependencies.autoload_paths`.
74
111
  * [Compilation caching](#compilation-caching)
75
112
  * `RubyVM::InstructionSequence.load_iseq` is implemented to cache the result of ruby bytecode
76
113
  compilation.
@@ -109,10 +146,6 @@ open y/foo.rb
109
146
  ...
110
147
  ```
111
148
 
112
- Exactly the same strategy is employed for methods that traverse
113
- `ActiveSupport::Dependencies.autoload_paths` if the `autoload_paths_cache` option is given to
114
- `Bootsnap.setup`.
115
-
116
149
  The following diagram flowcharts the overrides that make the `*_path_cache` features work.
117
150
 
118
151
  ![Flowchart explaining
@@ -199,7 +232,7 @@ Bootsnap writes a cache file containing a 64 byte header followed by the cache c
199
232
  is a cache key including several fields:
200
233
 
201
234
  * `version`, hardcoded in bootsnap. Essentially a schema version;
202
- * `os_version`, A hash of the current kernel version (on macOS, BSD) or glibc version (on Linux);
235
+ * `ruby_platform`, A hash of `RUBY_PLATFORM` (e.g. x86_64-linux-gnu) variable and glibc version (on Linux) or OS version (`uname -v` on BSD, macOS)
203
236
  * `compile_option`, which changes with `RubyVM::InstructionSequence.compile_option` does;
204
237
  * `ruby_revision`, the version of Ruby this was compiled with;
205
238
  * `size`, the size of the source file;
@@ -278,3 +311,25 @@ open /c/nope.bundle -> -1
278
311
  ```
279
312
  # (nothing!)
280
313
  ```
314
+
315
+ ## Precompilation
316
+
317
+ In development environments the bootsnap compilation cache is generated on the fly when source files are loaded.
318
+ But in production environments, such as docker images, you might need to precompile the cache.
319
+
320
+ To do so you can use the `bootsnap precompile` command.
321
+
322
+ Example:
323
+
324
+ ```bash
325
+ $ bundle exec bootsnap precompile --gemfile app/ lib/
326
+ ```
327
+
328
+ ## When not to use Bootsnap
329
+
330
+ *Alternative engines*: Bootsnap is pretty reliant on MRI features, and parts are disabled entirely on alternative ruby
331
+ engines.
332
+
333
+ *Non-local filesystems*: Bootsnap depends on `tmp/cache` (or whatever you set its cache directory
334
+ to) being on a relatively fast filesystem. If you put it on a network mount, bootsnap is very likely
335
+ to slow your application down quite a lot.
data/exe/bootsnap ADDED
@@ -0,0 +1,5 @@
1
+ #!/usr/bin/env ruby
2
+ # frozen_string_literal: true
3
+
4
+ require 'bootsnap/cli'
5
+ exit Bootsnap::CLI.new(ARGV).run