bootsnap 0.2.15 → 0.3.0.pre

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: b0de1ff4a8bdd4478f422a07fba3981778828604
-  data.tar.gz: d9b7d5cc09e58be7bc31382be47d72870c60bf7f
+  metadata.gz: '082f8dd8be53486b79ff5248ad426cc28263778b'
+  data.tar.gz: 5bc6a79f2054aa8b869704cdb9aeab685eabb400
 SHA512:
-  metadata.gz: f153d3e9591eb2fb8d2efecb958cbe21f61d93040f891ef63809285418cf76b3883d5490b3aa60d656bd066f96d7489317b13833bdf465058f7a339547177f6c
-  data.tar.gz: be2f217b2481c5d3eb44226d9861d2890dc7901143ab056e0d1f2fbd5e4f152ef497fb1cd67481ae44c4723dc5b0c93a6f6f6faebd705f4ce321aa0e5949bd2f
+  metadata.gz: ad062964bbc28d997901a1f6016289a4c6671071f9e1c4640820fe9545427136e1268f6588b350e7d058d85641604b0004225a744792fcce9b24bfcb9e5a3172
+  data.tar.gz: 33c77f2e085ba965e1a9fccb65b5afd42a4db0485dbfca7aa5d3e3920497ff41d0a3ca0ca4b6706df55814d29f015056e6d862568fe8b48ad5194f6eb80e26dc

data/README.md

@@ -2,9 +2,6 @@
 
 **Beta-quality. See [the last section of this README](#trustworthiness).**
 
-**`compile_cache_*` features are only supported on MacOS (extremely likely to error on Linux;
-won't even compile on other platforms).**
-
 Bootsnap is a library that plugs into a number of Ruby and (optionally) `ActiveSupport` and `YAML`
 methods to optimize and cache expensive computations. See [the How Does This Work section](#how-does-this-work) for more information.
 
@@ -120,7 +117,8 @@ result too, raising a `LoadError` without touching the filesystem at all.
 
 ### Compilation Caching
 
-*(A simpler implementation of this concept can be found in [yomikomu](https://github.com/ko1/yomikomu)).*
+*(A more readable implementation of this concept can be found in
+[yomikomu](https://github.com/ko1/yomikomu)).*
 
 Ruby has complex grammar and parsing it is not a particularly cheap operation. Since 1.9, Ruby has
 translated ruby source to an internal bytecode format, which is then executed by the Ruby VM. Since
@@ -134,9 +132,8 @@ implementation. We use the same strategy of compilation caching for YAML documen
 equivalent of Ruby's "bytecode" format being a MessagePack document (or, in the case of YAML
 documents with types unsupported by MessagePack, a Marshal stream).
 
-These compilation results are stored using `xattr`s on the source files. This is likely to change in
-the future, as it has some limitations (notably precluding Linux support except where the user feels
-like changing mount flags). However, this is a very performant implementation. 
+These compilation results are stored in a cache directory, with filenames generated by taking a hash
+of the full expanded path of the input file (FNV1a-64).
 
 Whereas before, the sequence of syscalls generated to `require` a file would look like:
 
@@ -158,32 +155,36 @@ With bootsnap, we get:
 ```
 open      /c/foo.rb -> n
 fstat64   n
-fgetxattr n
-fgetxattr n
+close     n
+open      /c/foo.rb -> n
+fstat64   n
+open      (cache) -> m
+read      m
+read      m
+close     m
 close     n
 ```
 
-Bootsnap writes two `xattrs` attached to each file read:
+This may look worse at a glance, but underlies a large performance difference.
 
-* `user.aotcc.value`, the binary compilation result; and
-* `user.aotcc.key`, a cache key to determine whether `user.aotcc.value` is still valid.
+*(The first three syscalls in both listings -- `open`, `fstat64`, `close` -- are not inherently
+useful. [This ruby patch](https://bugs.ruby-lang.org/issues/13378) optimizes them out when coupled
+with bootsnap.)*
 
-The key includes several fields:
+Bootsnap writes a cache file containing a 64 byte header followed by the cache contents. The header
+is a cache key including several fields:
 
 * `version`, hardcoded in bootsnap. Essentially a schema version;
+* `os_version`, A hash of the current kernel version (on macOS, BSD) or glibc version (on Linux);
 * `compile_option`, which changes with `RubyVM::InstructionSequence.compile_option` does;
-* `data_size`, the number of bytes in `user.aotcc.value`, which we need to read it into a buffer
-  using `fgetxattr(2)`;
-* `ruby_revision`, the version of Ruby this was compiled with; and
-* `mtime`, the last-modification timestamp of the source file when it was compiled.
+* `ruby_revision`, the version of Ruby this was compiled with;
+* `size`, the size of the source file;
+* `mtime`, the last-modification timestamp of the source file when it was compiled; and
+* `data_size`, the number of bytes following the header, which we need to read it into a buffer.
 
 If the key is valid, the result is loaded from the value. Otherwise, it is regenerated and clobbers
 the current cache.
 
-This diagram may help illustrate how it works:
-
-![Compilation Caching](https://burkelibbey.s3.amazonaws.com/bootsnap-compile-cache.png)
-
 ### Putting it all together
 
 Imagine we have this file structure:
@@ -227,8 +228,13 @@ With bootsnap, we get:
 ```
 open      /c/foo.rb -> n
 fstat64   n
-fgetxattr n
-fgetxattr n
+close     n
+open      /c/foo.rb -> n
+fstat64   n
+open      (cache) -> m
+read      m
+read      m
+close     m
 close     n
 ```
 
@@ -253,8 +259,8 @@ open    /c/nope.bundle -> -1
 
 We use the `*_path_cache` features in production and haven't experienced any issues in a long time.
 
-The `compile_cache_*` features work well for us in development on macOS, but probably don't work on
-Linux at all.
+The `compile_cache_*` features work well for us in development on macOS. It should work on Linux,
+and we intend to deploy it in production, but haven't we haven't yet.
 
 `disable_trace` should be completely safe, but we don't really use it because some people like to
 use tools that make use of `trace` instructions.
@@ -264,5 +270,5 @@ use tools that make use of `trace` instructions.
 | `load_path_cache` | everywhere |
 | `autoload_path_cache` | everywhere |
 | `disable_trace` | nowhere, but it's safe unless you need tracing |
-| `compile_cache_iseq` | development, unlikely to work on Linux |
-| `compile_cache_yaml` | development, unlikely to work on Linux |
+| `compile_cache_iseq` | development, but probably safe to use everywhere |
+| `compile_cache_yaml` | development, but probably safe to use everywhere |

data/ext/bootsnap/bootsnap.c

@@ -1,60 +1,78 @@
 #include "bootsnap.h"
+#include "ruby.h"
+#include <stdint.h>
 #include <sys/types.h>
-#include <sys/xattr.h>
-#include <sys/stat.h>
 #include <errno.h>
-#include <unistd.h>
 #include <fcntl.h>
-#include <stdbool.h>
-#include <utime.h>
+#include <sys/stat.h>
 
-#ifdef __APPLE__
-// Used for the OS Directives to define the os_version constant
-#include <Availability.h>
-#define _ENOATTR ENOATTR
+#ifdef __linux__
+#include <gnu/libc-version.h>
 #else
-#define _ENOATTR ENODATA
+#include <sys/sysctl.h>
 #endif
 
-/* 
- * TODO:
- * - test on linux or reject on non-darwin
- * - source files over 4GB will likely break things (meh)
- */
+#define MAX_CACHEPATH_SIZE 1000
+#define MAX_CACHEDIR_SIZE  981
+
+#define KEY_SIZE 64
+
+struct bs_cache_key {
+  uint32_t version;
+  uint32_t os_version;
+  uint32_t compile_option;
+  uint32_t ruby_revision;
+  uint64_t size;
+  uint64_t mtime;
+  uint64_t data_size; /* not used for equality */
+  uint8_t  pad[24];
+} __attribute__((packed));
+
+#define STATIC_ASSERT(X)            STATIC_ASSERT2(X,__LINE__)
+#define STATIC_ASSERT2(X,L)         STATIC_ASSERT3(X,L)
+#define STATIC_ASSERT3(X,L)         STATIC_ASSERT_MSG(X,at_line_##L)
+#define STATIC_ASSERT_MSG(COND,MSG) typedef char static_assertion_##MSG[(!!(COND))*2-1]
+
+STATIC_ASSERT(sizeof(struct bs_cache_key) == KEY_SIZE);
+
+static const uint32_t current_version = 2;
+
+static uint32_t current_os_version;
+static uint32_t current_ruby_revision;
+static uint32_t current_compile_option_crc32 = 0;
 
 static VALUE rb_mBootsnap;
 static VALUE rb_mBootsnap_CompileCache;
 static VALUE rb_mBootsnap_CompileCache_Native;
 static VALUE rb_eBootsnap_CompileCache_Uncompilable;
-static uint32_t current_ruby_revision;
-static uint32_t current_compile_option_crc32 = 0;
 static ID uncompilable;
 
-struct stats {
-  uint64_t hit;
-  uint64_t unwritable;
-  uint64_t uncompilable;
-  uint64_t miss;
-  uint64_t fail;
-  uint64_t retry;
-};
-static struct stats stats = {
-  .hit = 0,
-  .unwritable = 0,
-  .uncompilable = 0,
-  .miss = 0,
-  .fail = 0,
-  .retry = 0,
-};
+/* Real API */
+static VALUE bs_compile_option_crc32_set(VALUE self, VALUE crc32_v);
+static VALUE bs_rb_fetch(VALUE self, VALUE cachedir_v, VALUE path_v, VALUE handler);
+
+/* Helpers */
+static uint64_t fnv_64a(const char *str);
+static void bs_cache_path(const char * cachedir, const char * path, char ** cache_path);
+static int bs_read_key(int fd, struct bs_cache_key * key);
+static int cache_key_equal(struct bs_cache_key * k1, struct bs_cache_key * k2);
+static VALUE bs_fetch(char * path, VALUE path_v, char * cache_path, VALUE handler);
+static int open_current_file(char * path, struct bs_cache_key * key);
+static int fetch_cached_data(int fd, ssize_t data_size, VALUE handler, VALUE * output_data, int * exception_tag);
+static VALUE prot_exception_for_errno(VALUE err);
+static uint32_t get_os_version(void);
 
-struct xattr_key {
-  uint8_t  version;
-  uint8_t  os_version;
-  uint32_t compile_option;
-  uint32_t data_size;
-  uint32_t ruby_revision;
-  uint64_t mtime;
-} __attribute__((packed));
+static int bs_storage_to_output(VALUE handler, VALUE storage_data, VALUE * output_data);
+static VALUE prot_storage_to_output(VALUE arg);
+static VALUE prot_input_to_output(VALUE arg);
+static void bs_input_to_output(VALUE handler, VALUE input_data, VALUE * output_data, int * exception_tag);
+static VALUE prot_input_to_storage(VALUE arg);
+static int bs_input_to_storage(VALUE handler, VALUE input_data, VALUE pathval, VALUE * storage_data);
+
+struct s2o_data {
+  VALUE handler;
+  VALUE storage_data;
+};
 
 struct i2o_data {
   VALUE handler;
@@ -67,59 +85,6 @@ struct i2s_data {
   VALUE pathval;
 };
 
-struct s2o_data {
-  VALUE handler;
-  VALUE storage_data;
-};
-
-static const uint8_t current_version = 11;
-static const char * xattr_key_name = "user.aotcc.key";
-static const char * xattr_data_name = "user.aotcc.value";
-static const size_t xattr_key_size = sizeof (struct xattr_key);
-
-#ifdef __MAC_10_15 // Mac OS 10.15 (future)
-static const int os_version = 15;
-#elif __MAC_10_14 // Mac OS 10.14 (future)
-static const int os_version = 14;
-#elif __MAC_10_13 // Mac OS 10.13 (future)
-static const int os_version = 13;
-#elif __MAC_10_12 // Mac OS X Sierra
-static const int os_version = 12;
-#elif __MAC_10_11 // Mac OS X El Capitan
-static const int os_version = 11;
-# else
-static const int os_version = 0;
-#endif
-
-#ifdef __APPLE__
-#define GETXATTR_TRAILER ,0,0
-#define SETXATTR_TRAILER ,0
-#define REMOVEXATTR_TRAILER ,0
-#else
-#define GETXATTR_TRAILER
-#define SETXATTR_TRAILER
-#define REMOVEXATTR_TRAILER
-#endif
-
-/* forward declarations */
-static int bs_fetch_data(int fd, size_t size, VALUE handler, VALUE * storage_data, int * exception_tag);
-static int bs_update_key(int fd, uint32_t data_size, uint64_t current_mtime);
-static int bs_open(const char * path, bool * writable);
-static int bs_get_cache(int fd, struct xattr_key * key);
-static size_t bs_read_contents(int fd, size_t size, char ** contents);
-static int bs_close_and_unclobber_times(int * fd, const char * path, time_t atime, time_t mtime);
-static VALUE bs_fetch(VALUE self, VALUE pathval, VALUE handler);
-static VALUE bs_compile_option_crc32_set(VALUE self, VALUE crc32val);
-static VALUE prot_exception_for_errno(VALUE err);
-static VALUE prot_input_to_output(VALUE arg);
-static void bs_input_to_output(VALUE handler, VALUE input_data, VALUE * output_data, int * exception_tag);
-static VALUE prot_input_to_storage(VALUE arg);
-static int bs_input_to_storage(VALUE handler, VALUE input_data, VALUE pathval, VALUE * storage_data);
-static VALUE prot_storage_to_output(VALUE arg);
-static int bs_storage_to_output(VALUE handler, VALUE storage_data, VALUE * output_data);
-static int logging_enabled();
-static VALUE bs_stats(VALUE self);
-
 void
 Init_bootsnap(void)
 {
@@ -127,339 +92,356 @@ Init_bootsnap(void)
   rb_mBootsnap_CompileCache = rb_define_module_under(rb_mBootsnap, "CompileCache");
   rb_mBootsnap_CompileCache_Native = rb_define_module_under(rb_mBootsnap_CompileCache, "Native");
   rb_eBootsnap_CompileCache_Uncompilable = rb_define_class_under(rb_mBootsnap_CompileCache, "Uncompilable", rb_eStandardError);
+
   current_ruby_revision = FIX2INT(rb_const_get(rb_cObject, rb_intern("RUBY_REVISION")));
+  current_os_version = get_os_version();
 
   uncompilable = rb_intern("__bootsnap_uncompilable__");
 
-  rb_define_module_function(rb_mBootsnap_CompileCache_Native, "fetch", bs_fetch, 2);
-  rb_define_module_function(rb_mBootsnap_CompileCache_Native, "stats", bs_stats, 0);
+  rb_define_module_function(rb_mBootsnap_CompileCache_Native, "fetch", bs_rb_fetch, 3);
   rb_define_module_function(rb_mBootsnap_CompileCache_Native, "compile_option_crc32=", bs_compile_option_crc32_set, 1);
 }
 
 static VALUE
-bs_stats(VALUE self)
+bs_compile_option_crc32_set(VALUE self, VALUE crc32_v)
 {
-  VALUE ret = rb_hash_new();
-  rb_hash_aset(ret, ID2SYM(rb_intern("hit")), INT2NUM(stats.hit));
-  rb_hash_aset(ret, ID2SYM(rb_intern("miss")), INT2NUM(stats.miss));
-  rb_hash_aset(ret, ID2SYM(rb_intern("unwritable")), INT2NUM(stats.unwritable));
-  rb_hash_aset(ret, ID2SYM(rb_intern("uncompilable")), INT2NUM(stats.uncompilable));
-  rb_hash_aset(ret, ID2SYM(rb_intern("fail")), INT2NUM(stats.fail));
-  rb_hash_aset(ret, ID2SYM(rb_intern("retry")), INT2NUM(stats.retry));
-  return ret;
+  Check_Type(crc32_v, T_FIXNUM);
+  current_compile_option_crc32 = FIX2UINT(crc32_v);
+  return Qnil;
 }
 
-static VALUE
-bs_compile_option_crc32_set(VALUE self, VALUE crc32val)
+static uint64_t
+fnv_64a(const char *str)
 {
-  Check_Type(crc32val, T_FIXNUM);
-  current_compile_option_crc32 = FIX2UINT(crc32val);
-  return Qnil;
+  unsigned char *s = (unsigned char *)str;
+  uint64_t h = ((uint64_t)0xcbf29ce484222325ULL);
+
+  while (*s) {
+    h ^= (uint64_t)*s++;
+    h += (h << 1) + (h << 4) + (h << 5) + (h << 7) + (h << 8) + (h << 40);
+  }
+
+  return h;
 }
 
-#define CHECK_C(ret, func) \
-  do { if ((int)(ret) == -1) FAIL((func), errno); } while(0);
+/* On Darwin and FreeBSD, this is the kernel version, since that tends to vary
+ * with whole-system upgrades. What we probably care about more is the libc
+ * version, which is what we explicitly ask for on linux.
+ * (and KERN_OSRELEASE came back empty for me one one linux box, so...?)
+ */
+static uint32_t
+get_os_version(void)
+{
+  size_t len;
+  uint64_t hash;
+#ifdef __linux__
+  const char * version;
+  version = gnu_get_libc_version();
+  hash = fnv_64a(version);
+#else
+  char * version;
+  int mib[2] = {CTL_KERN, KERN_OSRELEASE};
+  if (sysctl(mib, 2, NULL, &len, NULL, 0) < 0) return 0;
+  version = malloc(sizeof(char) * len);
+  if (sysctl(mib, 2, version, &len, NULL, 0) < 0) return 0;
+  hash = fnv_64a(version);
+  free(version);
+#endif
+  return (uint32_t)(hash >> 32);
+}
 
-#define FAIL(func, err) \
-  do { \
-    int state; \
-    exception = rb_protect(prot_exception_for_errno, INT2FIX(err), &state); \
-    if (state) exception = rb_eStandardError; \
-    goto fail; \
-  } while(0);
+static void
+bs_cache_path(const char * cachedir, const char * path, char ** cache_path)
+{
+  uint64_t hash = fnv_64a(path);
 
-#define CHECK_RB0() \
-  do { if (exception_tag != 0) goto raise; } while (0);
+  uint8_t first_byte = (hash >> (64 - 8));
+  uint64_t remainder = hash & 0x00ffffffffffffff;
 
-#define CHECK_RB(body) \
-  do { (body); CHECK_RB0(); } while (0);
+  sprintf(*cache_path, "%s/%02x/%014llx", cachedir, first_byte, remainder);
+}
 
-#define SUCCEED(final) \
-  do { \
-    output_data = final; \
-    goto cleanup; \
-  } while(0);
+static int
+cache_key_equal(struct bs_cache_key * k1, struct bs_cache_key * k2)
+{
+  return (
+    k1->version        == k2->version        &&
+    k1->os_version     == k2->os_version     &&
+    k1->compile_option == k2->compile_option &&
+    k1->ruby_revision  == k2->ruby_revision  &&
+    k1->size           == k2->size           &&
+    k1->mtime          == k2->mtime
+  );
+}
 
 static VALUE
-bs_fetch(VALUE self, VALUE pathval, VALUE handler)
+bs_rb_fetch(VALUE self, VALUE cachedir_v, VALUE path_v, VALUE handler)
 {
-  const char * path;
-
-  VALUE exception;
-  int exception_tag;
-
-  int fd, ret, retry;
-  bool valid_cache;
-  bool writable;
-  uint32_t data_size;
-  struct xattr_key cache_key;
-  struct stat statbuf;
-  char * contents;
+  Check_Type(cachedir_v, T_STRING);
+  Check_Type(path_v, T_STRING);
 
-  VALUE input_data;   /* data read from source file, e.g. YAML or ruby source */
-  VALUE storage_data; /* compiled data, e.g. msgpack / binary iseq */
-  VALUE output_data;  /* return data, e.g. ruby hash or loaded iseq */
-
-  /* don't leak memory */
-#define return   error!
-#define rb_raise error!
-
-  retry = 0;
-begin:
-  output_data = Qnil;
-  contents = 0;
-
-  /* Blow up if we can't turn our argument into a char* */
-  Check_Type(pathval, T_STRING);
-  path = RSTRING_PTR(pathval);
-
-  /* open the file, get its mtime and read the cache key xattr */
-  CHECK_C(fd          = bs_open(path, &writable),     "open");
-  CHECK_C(              fstat(fd, &statbuf),             "fstat");
-  CHECK_C(valid_cache = bs_get_cache(fd, &cache_key), "fgetxattr");
-
-  /* `valid_cache` is true if the cache key isn't trivially invalid, e.g. built
-   * with a different RUBY_REVISION */
-  if (valid_cache && cache_key.mtime == (uint64_t)statbuf.st_mtime) {
-    /* if the mtimes match, assume the cache is valid. fetch the cached data. */
-    ret = bs_fetch_data(fd, (size_t)cache_key.data_size, handler, &output_data, &exception_tag);
-    if (ret == -1 && errno == _ENOATTR) {
-      /* the key was present, but the data was missing. remove the key, and
-       * start over */
-      CHECK_C(fremovexattr(fd, xattr_key_name REMOVEXATTR_TRAILER), "fremovexattr");
-      goto retry;
-    }
-    CHECK_RB0();
-    CHECK_C(ret, "fgetxattr/fetch-data");
-    if (!NIL_P(output_data)) {
-      stats.hit++;
-      SUCCEED(output_data); /* this is the fast-path to shoot for */
-    }
-    valid_cache = false; /* invalid cache; we'll want to regenerate it */
+  if (RSTRING_LEN(cachedir_v) > MAX_CACHEDIR_SIZE) {
+    rb_raise(rb_eArgError, "cachedir too long");
   }
 
-  /* read the contents of the file and crc32 it to compare with the cache key */
-  CHECK_C(bs_read_contents(fd, statbuf.st_size, &contents), "read") /* contents must be xfree'd */
-
-  /* we need to pass this char* to ruby-land */
-  input_data = rb_str_new_static(contents, statbuf.st_size);
+  char * cachedir = RSTRING_PTR(cachedir_v);
+  char * path     = RSTRING_PTR(path_v);
+  char cache_path[MAX_CACHEPATH_SIZE];
 
-  /* if we didn't have write permission to the file, bail now -- everything
-   * that follows is about generating and writing the cache. Let's just convert
-   * the input format to the output format and return */
-  if (!writable) {
-    stats.unwritable++;
-    CHECK_RB(bs_input_to_output(handler, input_data, &output_data, &exception_tag));
-    SUCCEED(output_data);
+  { /* generate cache path to cache_path */
+    char * tmp = (char *)&cache_path;
+    bs_cache_path(cachedir, path, &tmp);
   }
 
-  /* Now, we know we have write permission, and can update the xattrs.
-   * Additionally, we know the cache is currently missing or absent, and needs
-   * to be updated. */
-  stats.miss++;
+  return bs_fetch(path, path_v, cache_path, handler);
+}
 
-  /* First, convert the input format to the storage format by calling into the
-   * handler. */
-  CHECK_RB(exception_tag = bs_input_to_storage(handler, input_data, pathval, &storage_data));
-  if (storage_data == uncompilable) {
-    /* The handler can raise Bootsnap::CompileCache::Uncompilable. When it does this,
-     * we just call the input_to_output handler method, bypassing the storage format. */
-    CHECK_RB(bs_input_to_output(handler, input_data, &output_data, &exception_tag));
-    stats.uncompilable++;
-    SUCCEED(output_data);
-  }
+static int
+open_current_file(char * path, struct bs_cache_key * key)
+{
+  struct stat statbuf;
+  int fd;
 
-  /* we can only really write strings to xattrs */
-  if (!RB_TYPE_P(storage_data, T_STRING)) {
-    goto invalid_type_storage_data;
-  }
+  fd = open(path, O_RDONLY);
+  if (fd < 0) return fd;
 
-  /* xattrs can't exceed 64MB */
-  if (RB_TYPE_P(storage_data, T_STRING) && RSTRING_LEN(storage_data) > 64 * 1024 * 1024) {
-    if (logging_enabled()) {
-      fprintf(stderr, "[OPT_AOT_LOG] warning: compiled artifact is over 64MB, which is too large to store in an xattr.%s\n", path);
-    }
-    CHECK_RB(bs_input_to_output(handler, input_data, &output_data, &exception_tag));
-    SUCCEED(output_data);
+  if (fstat(fd, &statbuf) < 0) {
+    close(fd);
+    return -1;
   }
 
-  data_size = (uint32_t)RSTRING_LEN(storage_data);
-
-  /* update the cache, but don't leave it in an invalid state even briefly: remove the key first. */
-  fremovexattr(fd, xattr_key_name REMOVEXATTR_TRAILER);
-  CHECK_C(fsetxattr(fd, xattr_data_name, RSTRING_PTR(storage_data), (size_t)data_size, 0 SETXATTR_TRAILER), "fsetxattr");
-  CHECK_C(bs_update_key(fd, data_size, statbuf.st_mtime), "fsetxattr");
+  key->version        = current_version;
+  key->os_version     = current_os_version;
+  key->compile_option = current_compile_option_crc32;
+  key->ruby_revision  = current_ruby_revision;
+  key->size           = (uint64_t)statbuf.st_size;
+  key->mtime          = (uint64_t)statbuf.st_mtime;
 
-  /* updating xattrs bumps mtime, so we set them back after */
-  CHECK_C(bs_close_and_unclobber_times(&fd, path, statbuf.st_atime, statbuf.st_mtime), "close/utime");
+  return fd;
+}
 
-  /* convert the data we just stored into the output format */
-  CHECK_RB(exception_tag = bs_storage_to_output(handler, storage_data, &output_data));
+#define ERROR_WITH_ERRNO -1
+#define CACHE_MISSING_OR_INVALID -2
 
-  /* if the storage data was broken, remove the cache and run input_to_output */
-  if (output_data == Qnil) {
-    /* deletion here is best effort; no need to fail if it does */
-    fremovexattr(fd, xattr_key_name REMOVEXATTR_TRAILER);
-    fremovexattr(fd, xattr_data_name REMOVEXATTR_TRAILER);
-    CHECK_RB(bs_input_to_output(handler, input_data, &output_data, &exception_tag));
-  }
+/*
+ * @return CACHE_MISSING_OR_INVALID
+ * @return ERROR_WITH_ERRNO(-1) and errno
+ */
+static int
+bs_read_key(int fd, struct bs_cache_key * key)
+{
+  ssize_t nread = read(fd, key, KEY_SIZE);
+  if (nread < 0)        return ERROR_WITH_ERRNO;
+  if (nread < KEY_SIZE) return CACHE_MISSING_OR_INVALID;
+  return 0;
+}
 
-  SUCCEED(output_data);
+/*
+ * @return CACHE_MISSING_OR_INVALID
+ * @return ERROR_WITH_ERRNO(-1) and errno
+ */
+static int
+open_cache_file(const char * path, struct bs_cache_key * key)
+{
+  int fd, res;
 
-#undef return
-#undef rb_raise
-#define CLEANUP \
-  if (contents != 0) xfree(contents); \
-  if (fd > 0) close(fd);
+  fd = open(path, O_RDWR, 0644);
+  if (fd < 0) {
+    if (errno == ENOENT) return CACHE_MISSING_OR_INVALID;
+    return ERROR_WITH_ERRNO;
+  }
 
-  __builtin_unreachable();
-cleanup:
-  CLEANUP;
-  return output_data;
-fail:
-  CLEANUP;
-  stats.fail++;
-  rb_exc_raise(exception);
-  __builtin_unreachable();
-invalid_type_storage_data:
-  CLEANUP;
-  stats.fail++;
-  Check_Type(storage_data, T_STRING);
-  __builtin_unreachable();
-retry:
-  CLEANUP;
-  stats.retry++;
-  if (retry == 1) {
-    rb_raise(rb_eRuntimeError, "internal error in bootsnap");
-    __builtin_unreachable();
+  res = bs_read_key(fd, key);
+  if (res < 0) {
+    close(fd);
+    return res;
   }
-  retry = 1;
-  goto begin;
-raise:
-  CLEANUP;
-  stats.fail++;
-  rb_jump_tag(exception_tag);
-  __builtin_unreachable();
+
+  return fd;
 }
 
 static int
-bs_fetch_data(int fd, size_t size, VALUE handler, VALUE * output_data, int * exception_tag)
+fetch_cached_data(int fd, ssize_t data_size, VALUE handler, VALUE * output_data, int * exception_tag)
 {
+  char * data;
+  ssize_t nread;
   int ret;
-  ssize_t nbytes;
-  void * xattr_data;
-  VALUE storage_data;
 
-  *output_data = Qnil;
-  *exception_tag = 0;
+  VALUE storage_data;
 
-  xattr_data = ALLOC_N(uint8_t, size);
-  nbytes = fgetxattr(fd, xattr_data_name, xattr_data, size GETXATTR_TRAILER);
-  if (nbytes == -1) {
-    ret = -1;
-    goto done;
+  if (data_size > 100000000000) {
+    errno = EINVAL; /* because wtf? */
+    return -1;
   }
-  if (nbytes != (ssize_t)size) {
-    errno = EIO; /* lies but whatever */
+  data = ALLOC_N(char, data_size);
+  nread = read(fd, data, data_size);
+  if (nread < 0) {
     ret = -1;
     goto done;
   }
-  storage_data = rb_str_new_static(xattr_data, nbytes);
-  ret = bs_storage_to_output(handler, storage_data, output_data);
-  if (ret != 0) {
-    *exception_tag = ret;
-    errno = 0;
+  if (nread != data_size) {
+    ret = CACHE_MISSING_OR_INVALID;
+    goto done;
   }
+
+  storage_data = rb_str_new_static(data, data_size);
+
+  *exception_tag = bs_storage_to_output(handler, storage_data, output_data);
+  ret = 0;
 done:
-  xfree(xattr_data);
+  xfree(data);
   return ret;
 }
 
-static int
-bs_update_key(int fd, uint32_t data_size, uint64_t current_mtime)
+static ssize_t
+bs_read_contents(int fd, size_t size, char ** contents)
 {
-  struct xattr_key xattr_key;
-
-  xattr_key = (struct xattr_key){
-    .version        = current_version,
-    .os_version     = os_version,
-    .data_size      = data_size,
-    .compile_option = current_compile_option_crc32,
-    .ruby_revision  = current_ruby_revision,
-    .mtime          = current_mtime,
-  };
-
-  return fsetxattr(fd, xattr_key_name, &xattr_key, (size_t)xattr_key_size, 0 SETXATTR_TRAILER);
+  *contents = ALLOC_N(char, size);
+  return read(fd, *contents, size);
 }
 
-/*
- * Open the file O_RDWR if possible, or O_RDONLY if that throws EACCES.
- * Set +writable+ to indicate which mode was used.
- */
 static int
-bs_open(const char * path, bool * writable)
+mkpath(char * file_path, mode_t mode)
 {
-  int fd;
-
-  *writable = true;
-  fd = open(path, O_RDWR);
-  if (fd == -1 && errno == EACCES) {
-    *writable = false;
-    if (logging_enabled()) {
-      fprintf(stderr, "[OPT_AOT_LOG] warning: unable to cache because no write permission to %s\n", path);
+  /* It would likely be more efficient to count back until we
+   * find a component that *does* exist, but this will only run
+   * at most 256 times, so it seems not worthwhile to change. */
+  char * p;
+  for (p = strchr(file_path + 1, '/'); p; p = strchr(p + 1, '/')) {
+    *p = '\0';
+    if (mkdir(file_path, mode) == -1) {
+      if (errno != EEXIST) {
+        *p = '/';
+        return -1;
+      }
     }
-    fd = open(path, O_RDONLY);
+    *p = '/';
   }
-  return fd;
+  return 0;
 }
 
-/*
- * Fetch the cache key from the relevant xattr into +key+.
- * Returns:
- *   0:  invalid/no cache
- *   1:  valid cache
- *   -1: fgetxattr failed, errno is set
- */
 static int
-bs_get_cache(int fd, struct xattr_key * key)
+atomic_write_cache_file(char * path, struct bs_cache_key * key, VALUE data)
 {
-  ssize_t nbytes;
+  char template[MAX_CACHEPATH_SIZE + 20];
+  char * dest;
+  char * tmp_path;
+  int fd;
+  ssize_t nwrite;
 
-  nbytes = fgetxattr(fd, xattr_key_name, (void *)key, xattr_key_size GETXATTR_TRAILER);
-  if (nbytes == -1 && errno != _ENOATTR) {
+  dest = strncpy(template, path, MAX_CACHEPATH_SIZE);
+  strcat(dest, ".tmp.XXXXXX");
+
+  tmp_path = mktemp(template);
+  fd = open(tmp_path, O_WRONLY | O_CREAT, 0644);
+  if (fd < 0) {
+    if (mkpath(path, 0755) < 0) return -1;
+    fd = open(tmp_path, O_WRONLY | O_CREAT, 0644);
+    if (fd < 0) return -1;
+  }
+
+  key->data_size = RSTRING_LEN(data);
+  nwrite = write(fd, key, KEY_SIZE);
+  if (nwrite < 0) return -1;
+  if (nwrite != KEY_SIZE) {
+    errno = EIO; /* Lies but whatever */
     return -1;
   }
 
-  return (nbytes == (ssize_t)xattr_key_size && \
-      key->version == current_version && \
-      key->os_version == os_version && \
-      key->compile_option == current_compile_option_crc32 && \
-      key->ruby_revision == current_ruby_revision);
-}
+  nwrite = write(fd, RSTRING_PTR(data), RSTRING_LEN(data));
+  if (nwrite < 0) return -1;
+  if (nwrite != RSTRING_LEN(data)) {
+    errno = EIO; /* Lies but whatever */
+    return -1;
+  }
 
-/*
- * Read an entire file into a char*
- * contents must be freed with xfree() when done.
- */
-static size_t
-bs_read_contents(int fd, size_t size, char ** contents)
-{
-  *contents = ALLOC_N(char, size);
-  return read(fd, *contents, size);
+  close(fd);
+  return rename(tmp_path, path);
 }
 
-static int
-bs_close_and_unclobber_times(int * fd, const char * path, time_t atime, time_t mtime)
+static VALUE
+bs_fetch(char * path, VALUE path_v, char * cache_path, VALUE handler)
 {
-  struct utimbuf times = {
-    .actime = atime,
-    .modtime = mtime,
-  };
-  if (close(*fd) == -1) {
-    return -1;
+  struct bs_cache_key cached_key, current_key;
+  char * contents = NULL;
+  int cache_fd = -1, current_fd = -1;
+  int res, valid_cache, exception_tag = 0;
+
+  VALUE input_data;   /* data read from source file, e.g. YAML or ruby source */
+  VALUE storage_data; /* compiled data, e.g. msgpack / binary iseq */
+  VALUE output_data;  /* return data, e.g. ruby hash or loaded iseq */
+
+  VALUE exception;
+
+  current_fd = open_current_file(path, &current_key);
+  if (current_fd < 0) goto fail_errno;
+
+  cache_fd = open_cache_file(cache_path, &cached_key);
+  if (cache_fd < 0 && cache_fd != CACHE_MISSING_OR_INVALID) goto fail_errno;
+
+  valid_cache = cache_key_equal(&current_key, &cached_key);
+
+  if (valid_cache) {
+    /* Fetch the cache data and return it if we're able to load it successfully */
+    res = fetch_cached_data(cache_fd, (ssize_t)cached_key.data_size, handler, &output_data, &exception_tag);
+    if (exception_tag != 0)                   goto raise;
+    else if (res == CACHE_MISSING_OR_INVALID) valid_cache = 0;
+    else if (res == ERROR_WITH_ERRNO)         goto fail_errno;
+    else if (!NIL_P(output_data))             goto succeed; /* fast-path, goal */
+  }
+  close(cache_fd);
+  cache_fd = -1;
+  /* Cache is stale, invalid, or missing. Regenerate and write it out. */
+
+  if (bs_read_contents(current_fd, current_key.size, &contents) < 0) goto fail_errno;
+  input_data = rb_str_new_static(contents, current_key.size);
+
+  exception_tag = bs_input_to_storage(handler, input_data, path_v, &storage_data);
+  if (exception_tag != 0) goto raise;
+  if (storage_data == uncompilable) {
+    bs_input_to_output(handler, input_data, &output_data, &exception_tag);
+    if (exception_tag != 0) goto raise;
+    goto succeed;
   }
-  *fd = 0;
-  return utime(path, &times);
+  if (!RB_TYPE_P(storage_data, T_STRING)) goto invalid_type_storage_data;
+
+  res = atomic_write_cache_file(cache_path, &current_key, storage_data);
+  if (res < 0) goto fail_errno;
+
+  exception_tag = bs_storage_to_output(handler, storage_data, &output_data);
+  if (exception_tag != 0) goto raise;
+
+  if (NIL_P(output_data)) {
+    if (unlink(cache_path) < 0) goto fail_errno;
+    bs_input_to_output(handler, input_data, &output_data, &exception_tag);
+    if (exception_tag != 0) goto raise;
+  }
+  // output_data is now the correct return; cascade into succeed:
+
+#define CLEANUP \
+  if (contents != NULL) xfree(contents);   \
+  if (current_fd >= 0)  close(current_fd); \
+  if (cache_fd >= 0)    close(cache_fd);
+
+succeed:
+  CLEANUP;
+  return output_data;
+fail_errno:
+  CLEANUP;
+  exception = rb_protect(prot_exception_for_errno, INT2FIX(errno), &res);
+  if (res) exception = rb_eStandardError;
+  rb_exc_raise(exception);
+  __builtin_unreachable();
+raise:
+  CLEANUP;
+  rb_jump_tag(exception_tag);
+  __builtin_unreachable();
+invalid_type_storage_data:
+  CLEANUP;
+  Check_Type(storage_data, T_STRING);
+  __builtin_unreachable();
+
+#undef CLEANUP
 }
 
 static VALUE
@@ -474,11 +456,28 @@ prot_exception_for_errno(VALUE err)
   return rb_eStandardError;
 }
 
+
+/*****************************************************************************/
+/********************* Handler Wrappers **************************************/
+/*****************************************************************************/
+
 static VALUE
-prot_input_to_output(VALUE arg)
+prot_storage_to_output(VALUE arg)
 {
-  struct i2o_data * data = (struct i2o_data *)arg;
-  return rb_funcall(data->handler, rb_intern("input_to_output"), 1, data->input_data);
+  struct s2o_data * data = (struct s2o_data *)arg;
+  return rb_funcall(data->handler, rb_intern("storage_to_output"), 1, data->storage_data);
+}
+
+static int
+bs_storage_to_output(VALUE handler, VALUE storage_data, VALUE * output_data)
+{
+  int state;
+  struct s2o_data s2o_data = {
+    .handler      = handler,
+    .storage_data = storage_data,
+  };
+  *output_data = rb_protect(prot_storage_to_output, (VALUE)&s2o_data, &state);
+  return state;
 }
 
 static void
@@ -491,6 +490,13 @@ bs_input_to_output(VALUE handler, VALUE input_data, VALUE * output_data, int * e
   *output_data = rb_protect(prot_input_to_output, (VALUE)&i2o_data, exception_tag);
 }
 
+static VALUE
+prot_input_to_output(VALUE arg)
+{
+  struct i2o_data * data = (struct i2o_data *)arg;
+  return rb_funcall(data->handler, rb_intern("input_to_output"), 1, data->input_data);
+}
+
 static VALUE
 try_input_to_storage(VALUE arg)
 {
@@ -526,36 +532,3 @@ bs_input_to_storage(VALUE handler, VALUE input_data, VALUE pathval, VALUE * stor
   *storage_data = rb_protect(prot_input_to_storage, (VALUE)&i2s_data, &state);
   return state;
 }
-
-static VALUE
-prot_storage_to_output(VALUE arg)
-{
-  struct s2o_data * data = (struct s2o_data *)arg;
-  return rb_funcall(data->handler, rb_intern("storage_to_output"), 1, data->storage_data);
-}
-
-static int
-bs_storage_to_output(VALUE handler, VALUE storage_data, VALUE * output_data)
-{
-  int state;
-  struct s2o_data s2o_data = {
-    .handler      = handler,
-    .storage_data = storage_data,
-  };
-  *output_data = rb_protect(prot_storage_to_output, (VALUE)&s2o_data, &state);
-  return state;
-}
-
-/* default no if empty, yes if present, no if "0" */
-static int
-logging_enabled()
-{
-  char * log = getenv("OPT_AOT_LOG");
-  if (log == 0) {
-    return 0;
-  } else if (log[0] == '0') {
-    return 0;
-  } else {
-    return 1;
-  }
-}

data/ext/bootsnap/bootsnap.h

@@ -1,8 +1,6 @@
 #ifndef BOOTSNAP_H
 #define BOOTSNAP_H 1
 
-#include <stdint.h>
-#include <sys/types.h>
-#include "ruby.h"
+/* doesn't expose anything */
 
 #endif /* BOOTSNAP_H */

data/lib/bootsnap.rb

@@ -27,6 +27,7 @@ module Bootsnap
     ) if load_path_cache
 
     Bootsnap::CompileCache.setup(
+      cache_dir: cache_dir + '/bootsnap-compile-cache',
       iseq: compile_cache_iseq,
       yaml: compile_cache_yaml
     )

data/lib/bootsnap/compile_cache.rb

@@ -3,13 +3,13 @@ require_relative 'compile_cache/yaml'
 
 module Bootsnap
   module CompileCache
-    def self.setup(iseq:, yaml:)
+    def self.setup(cache_dir:, iseq:, yaml:)
       if iseq
-        Bootsnap::CompileCache::ISeq.install!
+        Bootsnap::CompileCache::ISeq.install!(cache_dir)
       end
 
       if yaml
-        Bootsnap::CompileCache::YAML.install!
+        Bootsnap::CompileCache::YAML.install!(cache_dir)
       end
     end
   end

data/lib/bootsnap/compile_cache/iseq.rb

@@ -4,6 +4,10 @@ require 'zlib'
 module Bootsnap
   module CompileCache
     module ISeq
+      class << self
+        attr_accessor :cache_dir
+      end
+
       def self.input_to_storage(_, path)
         RubyVM::InstructionSequence.compile_file(path).to_binary
       rescue SyntaxError
@@ -28,6 +32,7 @@ module Bootsnap
       module InstructionSequenceMixin
         def load_iseq(path)
           Bootsnap::CompileCache::Native.fetch(
+            Bootsnap::CompileCache::ISeq.cache_dir,
             path.to_s,
             Bootsnap::CompileCache::ISeq
           )
@@ -62,7 +67,8 @@ module Bootsnap
         Bootsnap::CompileCache::Native.compile_option_crc32 = crc
       end
 
-      def self.install!
+      def self.install!(cache_dir)
+        Bootsnap::CompileCache::ISeq.cache_dir = cache_dir
         Bootsnap::CompileCache::ISeq.compile_option_updated
         class << RubyVM::InstructionSequence
           prepend InstructionSequenceMixin

data/lib/bootsnap/compile_cache/yaml.rb

@@ -30,7 +30,7 @@ module Bootsnap
         ::YAML.load(data)
       end
 
-      def self.install!
+      def self.install!(cache_dir)
         require 'yaml'
         require 'msgpack'
 
@@ -44,6 +44,7 @@ module Bootsnap
         klass = class << ::YAML; self; end
         klass.send(:define_method, :load_file) do |path|
           Bootsnap::CompileCache::Native.fetch(
+            cache_dir,
             path.to_s,
             Bootsnap::CompileCache::YAML
           )

data/lib/bootsnap/version.rb

@@ -1,3 +1,3 @@
 module Bootsnap
-  VERSION = "0.2.15"
+  VERSION = "0.3.0.pre"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: bootsnap
 version: !ruby/object:Gem::Version
-  version: 0.2.15
+  version: 0.3.0.pre
 platform: ruby
 authors:
 - Burke Libbey
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2017-05-23 00:00:00.000000000 Z
+date: 2017-05-25 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler
@@ -162,9 +162,9 @@ required_ruby_version: !ruby/object:Gem::Requirement
       version: 2.3.0
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
-  - - ">="
+  - - ">"
     - !ruby/object:Gem::Version
-      version: '0'
+      version: 1.3.1
 requirements: []
 rubyforge_project: 
 rubygems_version: 2.6.10