bootsnap 1.8.1 → 1.10.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 4565db3033321cbf9866e33e0d42021d76f164974154bfb62dc9bcb02082763c
4
- data.tar.gz: 48424a914d66bebe2029e2bbdb86aef1c0d684a9a773f219a54b4c2675aec0b4
3
+ metadata.gz: 3dfd9cfb2cde26fc31c82206b73f3940b4b46903a3fa73d0f3f1a4c4a90c6984
4
+ data.tar.gz: 6dd8acfb2676a2585eb320ed50ba0ddb9f9c847fb415bcda824c7ebff0cce752
5
5
  SHA512:
6
- metadata.gz: 05732a91a5437872eb83b96fcf1fa2db302acf599a53ca5a25c5ef9b628eef780051173f5081261a2491798ea71a005af180504da19707a09c52692eb01d65c3
7
- data.tar.gz: b19e9db48806a6cee08f9de5268c81fb235d577e8771f6acb238843a507856cb120fce3d63bec7d4664d4c88ea60f01ebb7acd8545ffa10291a3157dd25ebf89
6
+ metadata.gz: 219a84be718e4ac5681621739b6f9400d77a16dee631bdc286adad87c0ddf99cc581da7281150add34075c08c59832df2a468524656d2f1ac36ac1613be051bb
7
+ data.tar.gz: 0e407a034fd081c9e506637e0997d0c2cc0eea312be798f6fd0d573173699f906f1b43a1f3d37ff64ee6efb2f9257228c6c4e7663b7337bfc49d99e5089fccb9
data/CHANGELOG.md CHANGED
@@ -1,12 +1,87 @@
1
1
  # Unreleased
2
2
 
3
+ # 1.10.3
4
+
5
+ * Fix Regexp and Date type support in YAML compile cache. (#400)
6
+
7
+ * Improve the YAML compile cache to support `UTF-8` symbols. (#398, #399)
8
+ [The default `MessagePack` symbol serializer assumes all symbols are ASCII](https://github.com/msgpack/msgpack-ruby/pull/211),
9
+ because of this, non-ASCII compatible symbol would be restored with `ASCII_8BIT` encoding (AKA `BINARY`).
10
+ Bootsnap now properly cache them in `UTF-8`.
11
+
12
+ Note that the above only apply for actual YAML symbols (e..g `--- :foo`).
13
+ The issue is still present for string keys parsed with `YAML.load_file(..., symbolize_names: true)`, that is a bug
14
+ in `msgpack` that will hopefully be solved soon, see: https://github.com/msgpack/msgpack-ruby/pull/246
15
+
16
+ * Entirely disable the YAML compile cache if `Encoding.default_internal` is set to an encoding not supported by `msgpack`. (#398)
17
+ `Psych` coerce strings to `Encoding.default_internal`, but `MessagePack` doesn't. So in this scenario we can't provide
18
+ YAML caching at all without returning the strings in the wrong encoding.
19
+ This never came up in practice but might as well be safe.
20
+
21
+ # 1.10.2
22
+
23
+ * Reduce the `Kernel.require` extra stack frames some more. Now bootsnap should only add one extra frame per `require` call.
24
+
25
+ * Better check `freeze` option support in JSON compile cache.
26
+ Previously `JSON.load_file(..., freeze: true)` would be cached even when the msgpack version is missing support for it.
27
+
28
+ # 1.10.1
29
+
30
+ * Fix `Kernel#autoload`'s fallback path always being executed.
31
+
32
+ * Consider `unlink` failing with `ENOENT` as a success.
33
+
34
+ # 1.10.0
35
+
36
+ * Delay requiring `FileUtils`. (#285)
37
+ `FileUtils` can be installed as a gem, so it's best to wait for bundler to have setup the load path before requiring it.
38
+
39
+ * Improve support of Psych 4. (#392)
40
+ Since `1.8.0`, `YAML.load_file` was no longer cached when Psych 4 was used. This is because `load_file` loads
41
+ in safe mode by default, so the Bootsnap cache could defeat that safety.
42
+ Now when precompiling YAML files, Bootsnap first try to parse them in safe mode, and if it can't fallback to unsafe mode,
43
+ and the cache contains a flag that records whether it was generated in safe mode or not.
44
+ `YAML.unsafe_load_file` will use safe caches just fine, but `YAML.load_file` will fallback to uncached YAML parsing
45
+ if the cache was generated using unsafe parsing.
46
+
47
+ * Minimize the Kernel.require extra stack frames. (#393)
48
+ This should reduce the noise generated by bootsnap on `LoadError`.
49
+
50
+ # 1.9.4
51
+
52
+ * Ignore absolute paths in the loaded feature index. (#385)
53
+ This fixes a compatibility issue with Zeitwerk when Zeitwerk is loaded before bootsnap. It also should
54
+ reduce the memory usage and improve load performance of Zeitwerk managed files.
55
+
56
+ * Automatically invalidate the load path cache whenever the Ruby version change. (#387)
57
+ This is to avoid issues in case the same installation path is re-used for subsequent ruby patch releases.
58
+
59
+ # 1.9.3
60
+
61
+ * Only disable the compile cache for source files impacted by [Ruby 3.0.3 [Bug 18250]](https://bugs.ruby-lang.org/issues/18250).
62
+ This should keep the performance loss to a minimum.
63
+
64
+ # 1.9.2
65
+
66
+ * Disable compile cache if [Ruby 3.0.3's ISeq cache bug](https://bugs.ruby-lang.org/issues/18250) is detected.
67
+ AKA `iseq.rb:13 to_binary: wrong argument type false (expected Symbol)`
68
+ * Fix `Kernel.load` behavior: before `load 'a'` would load `a.rb` (and other tried extensions) and wouldn't load `a` unless `development_mode: true`, now only `a` would be loaded and files with extensions wouldn't be.
69
+
70
+ # 1.9.1
71
+
72
+ * Removed a forgotten debug statement in JSON precompilation.
73
+
74
+ # 1.9.0
75
+
76
+ * Added a compilation cache for `JSON.load_file`. (#370)
77
+
3
78
  # 1.8.1
4
79
 
5
80
  * Fixed support for older Psych. (#369)
6
81
 
7
82
  # 1.8.0
8
83
 
9
- * Improve support for Pysch 4. (#368)
84
+ * Improve support for Psych 4. (#368)
10
85
 
11
86
  # 1.7.7
12
87
 
@@ -24,8 +99,8 @@
24
99
 
25
100
  # 1.7.4
26
101
 
27
- * Stop raising errors when encoutering various file system errors. The cache is now best effort,
28
- if somehow it can't be saved, bootsnapp will gracefully fallback to the original operation (e.g. `Kernel.require`).
102
+ * Stop raising errors when encountering various file system errors. The cache is now best effort,
103
+ if somehow it can't be saved, bootsnap will gracefully fallback to the original operation (e.g. `Kernel.require`).
29
104
  (#353, #177, #262)
30
105
 
31
106
  # 1.7.3
data/LICENSE.txt CHANGED
@@ -1,6 +1,6 @@
1
1
  The MIT License (MIT)
2
2
 
3
- Copyright (c) 2017 Shopify, Inc.
3
+ Copyright (c) 2017-present Shopify, Inc.
4
4
 
5
5
  Permission is hereby granted, free of charge, to any person obtaining a copy
6
6
  of this software and associated documentation files (the "Software"), to deal
data/README.md CHANGED
@@ -41,7 +41,7 @@ getting progressively slower, this is almost certainly the cause.**
41
41
  It's technically possible to simply specify `gem 'bootsnap', require: 'bootsnap/setup'`, but it's
42
42
  important to load Bootsnap as early as possible to get maximum performance improvement.
43
43
 
44
- You can see how this require works [here](https://github.com/Shopify/bootsnap/blob/master/lib/bootsnap/setup.rb).
44
+ You can see how this require works [here](https://github.com/Shopify/bootsnap/blob/main/lib/bootsnap/setup.rb).
45
45
 
46
46
  If you are not using Rails, or if you are but want more control over things, add this to your
47
47
  application setup immediately after `require 'bundler/setup'` (i.e. as early as possible: the sooner
@@ -161,7 +161,7 @@ The only directories considered "stable" are things under the Ruby install prefi
161
161
  "volatile".
162
162
 
163
163
  In addition to the [`Bootsnap::LoadPathCache::Cache`
164
- source](https://github.com/Shopify/bootsnap/blob/master/lib/bootsnap/load_path_cache/cache.rb),
164
+ source](https://github.com/Shopify/bootsnap/blob/main/lib/bootsnap/load_path_cache/cache.rb),
165
165
  this diagram may help clarify how entry resolution works:
166
166
 
167
167
  ![How path searching works](https://cloud.githubusercontent.com/assets/3074765/25388270/670b5652-299b-11e7-87fb-975647f68981.png)
data/exe/bootsnap CHANGED
@@ -1,5 +1,5 @@
1
1
  #!/usr/bin/env ruby
2
2
  # frozen_string_literal: true
3
3
 
4
- require 'bootsnap/cli'
4
+ require "bootsnap/cli"
5
5
  exit Bootsnap::CLI.new(ARGV).run
@@ -2,7 +2,7 @@
2
2
  * Suggested reading order:
3
3
  * 1. Skim Init_bootsnap
4
4
  * 2. Skim bs_fetch
5
- * 3. The rest of everything
5
+ * 3. The rest of everyrything
6
6
  *
7
7
  * Init_bootsnap sets up the ruby objects and binds bs_fetch to
8
8
  * Bootsnap::CompileCache::Native.fetch.
@@ -75,7 +75,7 @@ struct bs_cache_key {
75
75
  STATIC_ASSERT(sizeof(struct bs_cache_key) == KEY_SIZE);
76
76
 
77
77
  /* Effectively a schema version. Bumping invalidates all previous caches */
78
- static const uint32_t current_version = 3;
78
+ static const uint32_t current_version = 4;
79
79
 
80
80
  /* hash of e.g. "x86_64-darwin17", invalidating when ruby is recompiled on a
81
81
  * new OS ABI, etc. */
@@ -91,8 +91,7 @@ static mode_t current_umask;
91
91
  static VALUE rb_mBootsnap;
92
92
  static VALUE rb_mBootsnap_CompileCache;
93
93
  static VALUE rb_mBootsnap_CompileCache_Native;
94
- static VALUE rb_eBootsnap_CompileCache_Uncompilable;
95
- static ID uncompilable;
94
+ static VALUE rb_cBootsnap_CompileCache_UNCOMPILABLE;
96
95
  static ID instrumentation_method;
97
96
  static VALUE sym_miss;
98
97
  static VALUE sym_stale;
@@ -120,10 +119,8 @@ static uint32_t get_ruby_platform(void);
120
119
  * exception.
121
120
  */
122
121
  static int bs_storage_to_output(VALUE handler, VALUE args, VALUE storage_data, VALUE * output_data);
123
- static VALUE prot_storage_to_output(VALUE arg);
124
122
  static VALUE prot_input_to_output(VALUE arg);
125
123
  static void bs_input_to_output(VALUE handler, VALUE args, VALUE input_data, VALUE * output_data, int * exception_tag);
126
- static VALUE prot_input_to_storage(VALUE arg);
127
124
  static int bs_input_to_storage(VALUE handler, VALUE args, VALUE input_data, VALUE pathval, VALUE * storage_data);
128
125
  struct s2o_data;
129
126
  struct i2o_data;
@@ -151,12 +148,12 @@ Init_bootsnap(void)
151
148
  rb_mBootsnap = rb_define_module("Bootsnap");
152
149
  rb_mBootsnap_CompileCache = rb_define_module_under(rb_mBootsnap, "CompileCache");
153
150
  rb_mBootsnap_CompileCache_Native = rb_define_module_under(rb_mBootsnap_CompileCache, "Native");
154
- rb_eBootsnap_CompileCache_Uncompilable = rb_define_class_under(rb_mBootsnap_CompileCache, "Uncompilable", rb_eStandardError);
151
+ rb_cBootsnap_CompileCache_UNCOMPILABLE = rb_const_get(rb_mBootsnap_CompileCache, rb_intern("UNCOMPILABLE"));
152
+ rb_global_variable(&rb_cBootsnap_CompileCache_UNCOMPILABLE);
155
153
 
156
154
  current_ruby_revision = get_ruby_revision();
157
155
  current_ruby_platform = get_ruby_platform();
158
156
 
159
- uncompilable = rb_intern("__bootsnap_uncompilable__");
160
157
  instrumentation_method = rb_intern("_instrument");
161
158
 
162
159
  sym_miss = ID2SYM(rb_intern("miss"));
@@ -426,6 +423,7 @@ open_current_file(char * path, struct bs_cache_key * key, const char ** errno_pr
426
423
  #define ERROR_WITH_ERRNO -1
427
424
  #define CACHE_MISS -2
428
425
  #define CACHE_STALE -3
426
+ #define CACHE_UNCOMPILABLE -4
429
427
 
430
428
  /*
431
429
  * Read the cache key from the given fd, which must have position 0 (e.g.
@@ -507,14 +505,14 @@ fetch_cached_data(int fd, ssize_t data_size, VALUE handler, VALUE args, VALUE *
507
505
  if (data_size > 100000000000) {
508
506
  *errno_provenance = "bs_fetch:fetch_cached_data:datasize";
509
507
  errno = EINVAL; /* because wtf? */
510
- ret = -1;
508
+ ret = ERROR_WITH_ERRNO;
511
509
  goto done;
512
510
  }
513
511
  data = ALLOC_N(char, data_size);
514
512
  nread = read(fd, data, data_size);
515
513
  if (nread < 0) {
516
514
  *errno_provenance = "bs_fetch:fetch_cached_data:read";
517
- ret = -1;
515
+ ret = ERROR_WITH_ERRNO;
518
516
  goto done;
519
517
  }
520
518
  if (nread != data_size) {
@@ -525,6 +523,10 @@ fetch_cached_data(int fd, ssize_t data_size, VALUE handler, VALUE args, VALUE *
525
523
  storage_data = rb_str_new(data, data_size);
526
524
 
527
525
  *exception_tag = bs_storage_to_output(handler, args, storage_data, output_data);
526
+ if (*output_data == rb_cBootsnap_CompileCache_UNCOMPILABLE) {
527
+ ret = CACHE_UNCOMPILABLE;
528
+ goto done;
529
+ }
528
530
  ret = 0;
529
531
  done:
530
532
  if (data != NULL) xfree(data);
@@ -737,7 +739,15 @@ bs_fetch(char * path, VALUE path_v, char * cache_path, VALUE handler, VALUE args
737
739
  &output_data, &exception_tag, &errno_provenance
738
740
  );
739
741
  if (exception_tag != 0) goto raise;
740
- else if (res == CACHE_MISS || res == CACHE_STALE) valid_cache = 0;
742
+ else if (res == CACHE_UNCOMPILABLE) {
743
+ /* If fetch_cached_data returned `Uncompilable` we fallback to `input_to_output`
744
+ This happens if we have say, an unsafe YAML cache, but try to load it in safe mode */
745
+ if (bs_read_contents(current_fd, current_key.size, &contents, &errno_provenance) < 0) goto fail_errno;
746
+ input_data = rb_str_new(contents, current_key.size);
747
+ bs_input_to_output(handler, args, input_data, &output_data, &exception_tag);
748
+ if (exception_tag != 0) goto raise;
749
+ goto succeed;
750
+ } else if (res == CACHE_MISS || res == CACHE_STALE) valid_cache = 0;
741
751
  else if (res == ERROR_WITH_ERRNO) goto fail_errno;
742
752
  else if (!NIL_P(output_data)) goto succeed; /* fast-path, goal */
743
753
  }
@@ -754,7 +764,7 @@ bs_fetch(char * path, VALUE path_v, char * cache_path, VALUE handler, VALUE args
754
764
  if (exception_tag != 0) goto raise;
755
765
  /* If input_to_storage raised Bootsnap::CompileCache::Uncompilable, don't try
756
766
  * to cache anything; just return input_to_output(input_data) */
757
- if (storage_data == uncompilable) {
767
+ if (storage_data == rb_cBootsnap_CompileCache_UNCOMPILABLE) {
758
768
  bs_input_to_output(handler, args, input_data, &output_data, &exception_tag);
759
769
  if (exception_tag != 0) goto raise;
760
770
  goto succeed;
@@ -772,12 +782,20 @@ bs_fetch(char * path, VALUE path_v, char * cache_path, VALUE handler, VALUE args
772
782
  exception_tag = bs_storage_to_output(handler, args, storage_data, &output_data);
773
783
  if (exception_tag != 0) goto raise;
774
784
 
775
- /* If output_data is nil, delete the cache entry and generate the output
776
- * using input_to_output */
777
- if (NIL_P(output_data)) {
785
+ if (output_data == rb_cBootsnap_CompileCache_UNCOMPILABLE) {
786
+ /* If storage_to_output returned `Uncompilable` we fallback to `input_to_output` */
787
+ bs_input_to_output(handler, args, input_data, &output_data, &exception_tag);
788
+ if (exception_tag != 0) goto raise;
789
+ } else if (NIL_P(output_data)) {
790
+ /* If output_data is nil, delete the cache entry and generate the output
791
+ * using input_to_output */
778
792
  if (unlink(cache_path) < 0) {
779
- errno_provenance = "bs_fetch:unlink";
780
- goto fail_errno;
793
+ /* If the cache was already deleted, it might be that another process did it before us.
794
+ * No point raising an error */
795
+ if (errno != ENOENT) {
796
+ errno_provenance = "bs_fetch:unlink";
797
+ goto fail_errno;
798
+ }
781
799
  }
782
800
  bs_input_to_output(handler, args, input_data, &output_data, &exception_tag);
783
801
  if (exception_tag != 0) goto raise;
@@ -856,7 +874,7 @@ bs_precompile(char * path, VALUE path_v, char * cache_path, VALUE handler)
856
874
 
857
875
  /* If input_to_storage raised Bootsnap::CompileCache::Uncompilable, don't try
858
876
  * to cache anything; just return false */
859
- if (storage_data == uncompilable) {
877
+ if (storage_data == rb_cBootsnap_CompileCache_UNCOMPILABLE) {
860
878
  goto fail;
861
879
  }
862
880
  /* If storage_data isn't a string, we can't cache it */
@@ -919,7 +937,7 @@ struct i2s_data {
919
937
  };
920
938
 
921
939
  static VALUE
922
- prot_storage_to_output(VALUE arg)
940
+ try_storage_to_output(VALUE arg)
923
941
  {
924
942
  struct s2o_data * data = (struct s2o_data *)arg;
925
943
  return rb_funcall(data->handler, rb_intern("storage_to_output"), 2, data->storage_data, data->args);
@@ -934,7 +952,7 @@ bs_storage_to_output(VALUE handler, VALUE args, VALUE storage_data, VALUE * outp
934
952
  .args = args,
935
953
  .storage_data = storage_data,
936
954
  };
937
- *output_data = rb_protect(prot_storage_to_output, (VALUE)&s2o_data, &state);
955
+ *output_data = rb_protect(try_storage_to_output, (VALUE)&s2o_data, &state);
938
956
  return state;
939
957
  }
940
958
 
@@ -963,22 +981,6 @@ try_input_to_storage(VALUE arg)
963
981
  return rb_funcall(data->handler, rb_intern("input_to_storage"), 2, data->input_data, data->pathval);
964
982
  }
965
983
 
966
- static VALUE
967
- rescue_input_to_storage(VALUE arg, VALUE e)
968
- {
969
- return uncompilable;
970
- }
971
-
972
- static VALUE
973
- prot_input_to_storage(VALUE arg)
974
- {
975
- struct i2s_data * data = (struct i2s_data *)arg;
976
- return rb_rescue2(
977
- try_input_to_storage, (VALUE)data,
978
- rescue_input_to_storage, Qnil,
979
- rb_eBootsnap_CompileCache_Uncompilable, 0);
980
- }
981
-
982
984
  static int
983
985
  bs_input_to_storage(VALUE handler, VALUE args, VALUE input_data, VALUE pathval, VALUE * storage_data)
984
986
  {
@@ -988,6 +990,6 @@ bs_input_to_storage(VALUE handler, VALUE args, VALUE input_data, VALUE pathval,
988
990
  .input_data = input_data,
989
991
  .pathval = pathval,
990
992
  };
991
- *storage_data = rb_protect(prot_input_to_storage, (VALUE)&i2s_data, &state);
993
+ *storage_data = rb_protect(try_input_to_storage, (VALUE)&i2s_data, &state);
992
994
  return state;
993
995
  }
@@ -1,21 +1,23 @@
1
1
  # frozen_string_literal: true
2
+
2
3
  require("mkmf")
3
4
 
4
- if RUBY_ENGINE == 'ruby'
5
- $CFLAGS << ' -O3 '
6
- $CFLAGS << ' -std=c99'
5
+ if RUBY_ENGINE == "ruby"
6
+ $CFLAGS << " -O3 "
7
+ $CFLAGS << " -std=c99"
7
8
 
8
9
  # ruby.h has some -Wpedantic fails in some cases
9
10
  # (e.g. https://github.com/Shopify/bootsnap/issues/15)
10
- unless ['0', '', nil].include?(ENV['BOOTSNAP_PEDANTIC'])
11
- $CFLAGS << ' -Wall'
12
- $CFLAGS << ' -Werror'
13
- $CFLAGS << ' -Wextra'
14
- $CFLAGS << ' -Wpedantic'
11
+ unless ["0", "", nil].include?(ENV["BOOTSNAP_PEDANTIC"])
12
+ $CFLAGS << " -Wall"
13
+ $CFLAGS << " -Werror"
14
+ $CFLAGS << " -Wextra"
15
+ $CFLAGS << " -Wpedantic"
15
16
 
16
- $CFLAGS << ' -Wno-unused-parameter' # VALUE self has to be there but we don't care what it is.
17
- $CFLAGS << ' -Wno-keyword-macro' # hiding return
18
- $CFLAGS << ' -Wno-gcc-compat' # ruby.h 2.6.0 on macos 10.14, dunno
17
+ $CFLAGS << " -Wno-unused-parameter" # VALUE self has to be there but we don't care what it is.
18
+ $CFLAGS << " -Wno-keyword-macro" # hiding return
19
+ $CFLAGS << " -Wno-gcc-compat" # ruby.h 2.6.0 on macos 10.14, dunno
20
+ $CFLAGS << " -Wno-compound-token-split-by-macro"
19
21
  end
20
22
 
21
23
  create_makefile("bootsnap/bootsnap")
@@ -1,4 +1,5 @@
1
1
  # frozen_string_literal: true
2
+
2
3
  module Bootsnap
3
4
  extend(self)
4
5
 
@@ -63,6 +63,7 @@ module Bootsnap
63
63
  loop do
64
64
  job, *args = Marshal.load(@pipe_out)
65
65
  return if job == :exit
66
+
66
67
  @jobs.fetch(job).call(*args)
67
68
  end
68
69
  rescue IOError
data/lib/bootsnap/cli.rb CHANGED
@@ -1,10 +1,10 @@
1
1
  # frozen_string_literal: true
2
2
 
3
- require 'bootsnap'
4
- require 'bootsnap/cli/worker_pool'
5
- require 'optparse'
6
- require 'fileutils'
7
- require 'etc'
3
+ require "bootsnap"
4
+ require "bootsnap/cli/worker_pool"
5
+ require "optparse"
6
+ require "fileutils"
7
+ require "etc"
8
8
 
9
9
  module Bootsnap
10
10
  class CLI
@@ -21,51 +21,58 @@ module Bootsnap
21
21
 
22
22
  attr_reader :cache_dir, :argv
23
23
 
24
- attr_accessor :compile_gemfile, :exclude, :verbose, :iseq, :yaml, :jobs
24
+ attr_accessor :compile_gemfile, :exclude, :verbose, :iseq, :yaml, :json, :jobs
25
25
 
26
26
  def initialize(argv)
27
27
  @argv = argv
28
- self.cache_dir = ENV.fetch('BOOTSNAP_CACHE_DIR', 'tmp/cache')
28
+ self.cache_dir = ENV.fetch("BOOTSNAP_CACHE_DIR", "tmp/cache")
29
29
  self.compile_gemfile = false
30
30
  self.exclude = nil
31
31
  self.verbose = false
32
32
  self.jobs = Etc.nprocessors
33
33
  self.iseq = true
34
34
  self.yaml = true
35
+ self.json = true
35
36
  end
36
37
 
37
38
  def precompile_command(*sources)
38
- require 'bootsnap/compile_cache/iseq'
39
- require 'bootsnap/compile_cache/yaml'
39
+ require "bootsnap/compile_cache/iseq"
40
+ require "bootsnap/compile_cache/yaml"
41
+ require "bootsnap/compile_cache/json"
40
42
 
41
43
  fix_default_encoding do
42
- Bootsnap::CompileCache::ISeq.cache_dir = self.cache_dir
44
+ Bootsnap::CompileCache::ISeq.cache_dir = cache_dir
43
45
  Bootsnap::CompileCache::YAML.init!
44
- Bootsnap::CompileCache::YAML.cache_dir = self.cache_dir
46
+ Bootsnap::CompileCache::YAML.cache_dir = cache_dir
47
+ Bootsnap::CompileCache::JSON.init!
48
+ Bootsnap::CompileCache::JSON.cache_dir = cache_dir
45
49
 
46
50
  @work_pool = WorkerPool.create(size: jobs, jobs: {
47
51
  ruby: method(:precompile_ruby),
48
52
  yaml: method(:precompile_yaml),
53
+ json: method(:precompile_json),
49
54
  })
50
55
  @work_pool.spawn
51
56
 
52
57
  main_sources = sources.map { |d| File.expand_path(d) }
53
58
  precompile_ruby_files(main_sources)
54
59
  precompile_yaml_files(main_sources)
60
+ precompile_json_files(main_sources)
55
61
 
56
62
  if compile_gemfile
57
63
  # Some gems embed their tests, they're very unlikely to be loaded, so not worth precompiling.
58
- gem_exclude = Regexp.union([exclude, '/spec/', '/test/'].compact)
64
+ gem_exclude = Regexp.union([exclude, "/spec/", "/test/"].compact)
59
65
  precompile_ruby_files($LOAD_PATH.map { |d| File.expand_path(d) }, exclude: gem_exclude)
60
66
 
61
- # Gems that include YAML files usually don't put them in `lib/`.
67
+ # Gems that include JSON or YAML files usually don't put them in `lib/`.
62
68
  # So we look at the gem root.
63
69
  gem_pattern = %r{^#{Regexp.escape(Bundler.bundle_path.to_s)}/?(?:bundler/)?gems\/[^/]+}
64
70
  gem_paths = $LOAD_PATH.map { |p| p[gem_pattern] }.compact.uniq
65
71
  precompile_yaml_files(gem_paths, exclude: gem_exclude)
72
+ precompile_json_files(gem_paths, exclude: gem_exclude)
66
73
  end
67
74
 
68
- if exitstatus = @work_pool.shutdown
75
+ if (exitstatus = @work_pool.shutdown)
69
76
  exit(exitstatus)
70
77
  end
71
78
  end
@@ -82,7 +89,7 @@ module Bootsnap
82
89
  if dir_sort
83
90
  def list_files(path, pattern)
84
91
  if File.directory?(path)
85
- Dir[File.join(path, pattern), sort: false]
92
+ Dir[File.join(path, pattern), sort: false]
86
93
  elsif File.exist?(path)
87
94
  [path]
88
95
  else
@@ -92,7 +99,7 @@ module Bootsnap
92
99
  else
93
100
  def list_files(path, pattern)
94
101
  if File.directory?(path)
95
- Dir[File.join(path, pattern)]
102
+ Dir[File.join(path, pattern)]
96
103
  elsif File.exist?(path)
97
104
  [path]
98
105
  else
@@ -119,9 +126,9 @@ module Bootsnap
119
126
 
120
127
  load_paths.each do |path|
121
128
  if !exclude || !exclude.match?(path)
122
- list_files(path, '**/*.{yml,yaml}').each do |yaml_file|
129
+ list_files(path, "**/*.{yml,yaml}").each do |yaml_file|
123
130
  # We ignore hidden files to not match the various .ci.yml files
124
- if !File.basename(yaml_file).start_with?('.') && (!exclude || !exclude.match?(yaml_file))
131
+ if !File.basename(yaml_file).start_with?(".") && (!exclude || !exclude.match?(yaml_file))
125
132
  @work_pool.push(:yaml, yaml_file)
126
133
  end
127
134
  end
@@ -131,18 +138,41 @@ module Bootsnap
131
138
 
132
139
  def precompile_yaml(*yaml_files)
133
140
  Array(yaml_files).each do |yaml_file|
134
- if CompileCache::YAML.precompile(yaml_file, cache_dir: cache_dir)
141
+ if CompileCache::YAML.precompile(yaml_file)
135
142
  STDERR.puts(yaml_file) if verbose
136
143
  end
137
144
  end
138
145
  end
139
146
 
147
+ def precompile_json_files(load_paths, exclude: self.exclude)
148
+ return unless json
149
+
150
+ load_paths.each do |path|
151
+ if !exclude || !exclude.match?(path)
152
+ list_files(path, "**/*.json").each do |json_file|
153
+ # We ignore hidden files to not match the various .config.json files
154
+ if !File.basename(json_file).start_with?(".") && (!exclude || !exclude.match?(json_file))
155
+ @work_pool.push(:json, json_file)
156
+ end
157
+ end
158
+ end
159
+ end
160
+ end
161
+
162
+ def precompile_json(*json_files)
163
+ Array(json_files).each do |json_file|
164
+ if CompileCache::JSON.precompile(json_file)
165
+ STDERR.puts(json_file) if verbose
166
+ end
167
+ end
168
+ end
169
+
140
170
  def precompile_ruby_files(load_paths, exclude: self.exclude)
141
171
  return unless iseq
142
172
 
143
173
  load_paths.each do |path|
144
174
  if !exclude || !exclude.match?(path)
145
- list_files(path, '**/*.rb').each do |ruby_file|
175
+ list_files(path, "**/*.rb").each do |ruby_file|
146
176
  if !exclude || !exclude.match?(ruby_file)
147
177
  @work_pool.push(:ruby, ruby_file)
148
178
  end
@@ -153,7 +183,7 @@ module Bootsnap
153
183
 
154
184
  def precompile_ruby(*ruby_files)
155
185
  Array(ruby_files).each do |ruby_file|
156
- if CompileCache::ISeq.precompile(ruby_file, cache_dir: cache_dir)
186
+ if CompileCache::ISeq.precompile(ruby_file)
157
187
  STDERR.puts(ruby_file) if verbose
158
188
  end
159
189
  end
@@ -180,7 +210,7 @@ module Bootsnap
180
210
  end
181
211
 
182
212
  def cache_dir=(dir)
183
- @cache_dir = File.expand_path(File.join(dir, 'bootsnap/compile-cache'))
213
+ @cache_dir = File.expand_path(File.join(dir, "bootsnap/compile-cache"))
184
214
  end
185
215
 
186
216
  def exclude_pattern(pattern)
@@ -195,24 +225,24 @@ module Bootsnap
195
225
  opts.separator "GLOBAL OPTIONS"
196
226
  opts.separator ""
197
227
 
198
- help = <<~EOS
228
+ help = <<~HELP
199
229
  Path to the bootsnap cache directory. Defaults to tmp/cache
200
- EOS
201
- opts.on('--cache-dir DIR', help.strip) do |dir|
230
+ HELP
231
+ opts.on("--cache-dir DIR", help.strip) do |dir|
202
232
  self.cache_dir = dir
203
233
  end
204
234
 
205
- help = <<~EOS
235
+ help = <<~HELP
206
236
  Print precompiled paths.
207
- EOS
208
- opts.on('--verbose', '-v', help.strip) do
237
+ HELP
238
+ opts.on("--verbose", "-v", help.strip) do
209
239
  self.verbose = true
210
240
  end
211
241
 
212
- help = <<~EOS
242
+ help = <<~HELP
213
243
  Number of workers to use. Default to number of processors, set to 0 to disable multi-processing.
214
- EOS
215
- opts.on('--jobs JOBS', '-j', help.strip) do |jobs|
244
+ HELP
245
+ opts.on("--jobs JOBS", "-j", help.strip) do |jobs|
216
246
  self.jobs = Integer(jobs)
217
247
  end
218
248
 
@@ -221,25 +251,30 @@ module Bootsnap
221
251
  opts.separator ""
222
252
  opts.separator " precompile [DIRECTORIES...]: Precompile all .rb files in the passed directories"
223
253
 
224
- help = <<~EOS
254
+ help = <<~HELP
225
255
  Precompile the gems in Gemfile
226
- EOS
227
- opts.on('--gemfile', help) { self.compile_gemfile = true }
256
+ HELP
257
+ opts.on("--gemfile", help) { self.compile_gemfile = true }
228
258
 
229
- help = <<~EOS
259
+ help = <<~HELP
230
260
  Path pattern to not precompile. e.g. --exclude 'aws-sdk|google-api'
231
- EOS
232
- opts.on('--exclude PATTERN', help) { |pattern| exclude_pattern(pattern) }
261
+ HELP
262
+ opts.on("--exclude PATTERN", help) { |pattern| exclude_pattern(pattern) }
233
263
 
234
- help = <<~EOS
264
+ help = <<~HELP
235
265
  Disable ISeq (.rb) precompilation.
236
- EOS
237
- opts.on('--no-iseq', help) { self.iseq = false }
266
+ HELP
267
+ opts.on("--no-iseq", help) { self.iseq = false }
238
268
 
239
- help = <<~EOS
269
+ help = <<~HELP
240
270
  Disable YAML precompilation.
241
- EOS
242
- opts.on('--no-yaml', help) { self.yaml = false }
271
+ HELP
272
+ opts.on("--no-yaml", help) { self.yaml = false }
273
+
274
+ help = <<~HELP
275
+ Disable JSON precompilation.
276
+ HELP
277
+ opts.on("--no-json", help) { self.json = false }
243
278
  end
244
279
  end
245
280
  end