atomic_cache 0.2.3.rc1 → 0.4.1.rc1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: da4480539afcbe1c7c15e34dfbd77c7da40761853fc270bab1a0ad6d778b253e
-  data.tar.gz: c0deb89b6a828430252b6ae85d1727aa5282837b355013f661be6f100406125b
+  metadata.gz: db2b157ed872e6cb90af8cf2ef61a97f237485bc763998464eb4faf488d26c11
+  data.tar.gz: 48e13e212479454f2ece192c81c8f755a4aa039a2cf5930cf1cdfea2eaae438d
 SHA512:
-  metadata.gz: 1aacd6f8ac2f420ac3b87e3d3031533f0cd3268cd6cb567a22e990c860274a6640800c4cfd4a88481b90af5c40ef60101d699a2a2df348fbe60e8e4639dd6901
-  data.tar.gz: c405dbd590e3f3f095a25ca1189687548cdf09380bba0b390060560b555730eade8528f897665d99d50a2ac0783ca00bd96e4029d852fbd79ffe14cd38daee0c
+  metadata.gz: 5fb9aba2eaeb20e9e3206c80039941b005c7f6e82c05bfddd59482783b028706f706a6843a8ea8261d18895732035d1deeabda178e5437620f33ca9abf88b54e
+  data.tar.gz: 06511e8a0ec9b33de005994f6938877f85b053c03072f52b6cc0dbd22a99389dfb36245f7002d59ea364d1ea583eec0b440c6bd2a5e1b84604434474275d6fe2

data/README.md

@@ -23,7 +23,7 @@ In a nutshell:
 class Foo < ActiveRecord::Base
   include AtomicCache::GlobalLMTCacheConcern
 
-  cache_class(:custom_foo)  # optional
+  force_cache_class(:custom_foo)  # optional
   cache_version(5)          # optional
 
   def active_foos(ids)

data/docs/MODEL_SETUP.md

@@ -7,13 +7,13 @@ class Foo < ActiveRecord::Base
 end
 ```
 
-### cache_class
+### force_cache_class
 By default the cache identifier for a class is set to the name of a class (ie. `self.to_s`).  In some cases it makes sense to set a custom value for the cache identifier.  In cases where a custom cache identifier is set, it's important that the identifier remain unique across the project.
 
 ```ruby
 class SuperDescriptiveDomainModelAbstractFactoryImplManager < ActiveRecord::Base
   include AtomicCache::GlobalLMTCacheConcern
-  cache_class('sddmafim')
+  force_cache_class('sddmafim')
 end
 ```
 

data/docs/PROJECT_SETUP.md

@@ -23,9 +23,10 @@ AtomicCache::DefaultConfig.configure do |config|
   config.metrics = Datadog::Statsd.new('localhost', 8125, namespace: 'cache.atomic')
 
   # note: these values can also be set in an env file for env-specific settings
-  config.namespace      = 'atom'
-  config.cache_storage  = AtomicCache::Storage::SharedMemory.new
-  config.key_storage    = AtomicCache::Storage::SharedMemory.new
+  config.namespace        = 'atom'
+  config.default_options  = { generate_ttl_ms: 500 }
+  config.cache_storage    = AtomicCache::Storage::SharedMemory.new
+  config.key_storage      = AtomicCache::Storage::SharedMemory.new
 end
 ```
 
@@ -36,7 +37,7 @@ Note that `Datadog::Statsd` is not _required_.  Adding it, however, will enable
   * `key_storage` - Storage adapter for key manager (see below)
 
 #### Optional
-  * `default_options` - Default options for every fetch call.  See [fetch options](/Ibotta/atomic_cache/blob/main/docs/USAGE.md#fetch).
+  * `default_options` - Override default options for every fetch call, unless specified at call site.  See [fetch options](/Ibotta/atomic_cache/blob/main/docs/USAGE.md#fetch).
   * `logger` - Logger instance.  Used for debug and warn logs. Defaults to nil.
   * `timestamp_formatter` - Proc to format last modified time for storage. Defaults to timestamp (`Time.to_i`)
   * `metrics` - Metrics instance. Defaults to nil.
@@ -45,6 +46,49 @@ Note that `Datadog::Statsd` is not _required_.  Adding it, however, will enable
 #### ★ Best Practice ★
 Keep the global namespace short.  For example, memcached has a limit of 250 characters for key length.
 
+#### More Complex Rails Configuration
+
+In any real-world project, the need to run multiple caching strategies or setups is likely to arise. In those cases, it's often advantageous
+to keep a DRY setup, with multiple caching clients sharing the same config. Because Rails initializers run after the environment-specific
+config files, a sane way to manage this is to keep client network settings int he config files, then reference them from the initializer.
+
+```ruby
+# config/environments/staging
+config.memcache_hosts = [ "staging.host.cache.amazonaws.com" ]
+config.cache_store_options = {
+  expires_in: 15.minutes,
+  compress: true,
+  # ...
+}
+
+# config/environments/production
+config.memcache_hosts = [ "prod1.host.cache.amazonaws.com", "prod2.host.cache.amazonaws.com" ]
+config.cache_store_options = {
+  expires_in: 1.hour,
+  compress: true,
+  # ...
+}
+
+# config/initializers/cache.rb
+AtomicCache::DefaultConfig.configure do |config|
+  if Rails.env.development? || Rails.env.test?
+    config.cache_storage  = AtomicCache::Storage::SharedMemory.new
+    config.key_storage    = AtomicCache::Storage::SharedMemory.new
+
+  elsif Rails.env.staging? || Rails.env.production?
+    # Your::Application.config will be loaded by config/environments/*
+    memcache_hosts = Your::Application.config.memcache_hosts
+    options = Your::Application.config.cache_store_options
+    
+    dc = Dalli::Client.new(memcache_hosts, options)
+    config.cache_storage  = AtomicCache::Storage::Dalli.new(dc)
+    config.key_storage    = AtomicCache::Storage::Dalli.new(dc)
+  end
+  
+  # other AtomicCache configuration...
+end
+```
+
 ## Storage Adapters
 
 ### InstanceMemory & SharedMemory

data/docs/USAGE.md

@@ -38,17 +38,6 @@ The ideal `generate_ttl_ms` time is just slightly longer than the average genera
 
 If metrics are enabled, the `<namespace>.generate.run` can be used to determine the min/max/average generate time for a particular cache and the `generate_ttl_ms` tuned using that.
 
-#### `quick_retry_ms`
-_`false` to disable. Defaults to false._
-
-In the case where another process is computing the new cache value, before falling back to the last known value, if `quick_retry_ms` has a value the atomic client will check the new cache once after the given duration (in milliseconds).
-
-The danger with `quick_retry_ms` is that when enabled it applies a delay to all fall-through requests at the cost of only benefitting some customers.  As the average generate block duration increases, the effectiveness of `quick_retry_ms` decreases because there is less of a likelihood that a customer will get a fresh value.  Consider the graph below.  For example, a cache with an average generate duration of 200ms, configured with a `quick_retry_ms` of 50ms (red) will only likely get a fresh value for 25% of customers.
-
-`quick_retry_ms` is most effective for caches that are quick to generate but whose values are slow to change.  `quick_retry_ms` is least effective for caches that are slow to update but quick to change.
-
-![quick_retry_ms graph](https://github.com/Ibotta/atomic_cache/raw/ca473f28e179da8c24f638eeeeb48750bc8cbe64/docs/img/quick_retry_graph.png)
-
 #### `max_retries` & `backoff_duration_ms`
 _`max_retries` defaults to 5._
 _`backoff_duration_ms` defaults to 50ms._

data/lib/atomic_cache/atomic_cache_client.rb

@@ -6,7 +6,6 @@ require 'active_support/core_ext/hash'
 module AtomicCache
   class AtomicCacheClient
 
-    DEFAULT_quick_retry_ms = false
     DEFAULT_MAX_RETRIES = 5
     DEFAULT_GENERATE_TIME_MS = 30000 # 30 seconds
     BACKOFF_DURATION_MS = 50
@@ -27,13 +26,11 @@ module AtomicCache
       raise ArgumentError.new("`storage` required but none given") unless @storage.present?
     end
 
-
     # Attempts to fetch the given keyspace, using an optional block to generate
     # a new value when the cache is expired
     #
     # @param keyspace [AtomicCache::Keyspace] the keyspace to fetch
     # @option options [Numeric] :generate_ttl_ms (30000) Max generate duration in ms
-    # @option options [Numeric] :quick_retry_ms (false) Short duration to check back before using last known value
     # @option options [Numeric] :max_retries (5) Max times to rety in waiting case
     # @option options [Numeric] :backoff_duration_ms (50) Duration in ms to wait between retries
     # @yield Generates a new value when cache is expired
@@ -45,6 +42,7 @@ module AtomicCache
       value = @storage.read(key, options) if key.present?
       if !value.nil?
         metrics(:increment, 'read.present', tags: tags)
+        log(:debug, "Read value from key: '#{key}'")
         return value
       end
 
@@ -57,15 +55,14 @@ module AtomicCache
         return new_value unless new_value.nil?
       end
 
-      # quick check back to see if the other process has finished
-      # or fall back to the last known value
-      value = quick_retry(keyspace, options, tags) || last_known_value(keyspace, options, tags)
+      # attempt to fall back to the last known value
+      value = last_known_value(keyspace, options, tags)
       return value if value.present?
 
       # wait for the other process if a last known value isn't there
       if key.present?
         return time('wait.run', tags: tags) do
-          wait_for_new_value(key, options, tags)
+          wait_for_new_value(keyspace, options, tags)
         end
       end
 
@@ -109,24 +106,6 @@ module AtomicCache
       nil
     end
 
-    def quick_retry(keyspace, options, tags)
-      key = @timestamp_manager.current_key(keyspace)
-      duration = option(:quick_retry_ms, options, DEFAULT_quick_retry_ms)
-
-      if duration.present? and key.present?
-        sleep(duration.to_f / 1000)
-        value = @storage.read(key, options)
-
-        if !value.nil?
-          metrics(:increment, 'empty-cache-retry.present', tags: tags)
-          return value
-        end
-        metrics(:increment, 'empty-cache-retry.not-present', tags: tags)
-      end
-
-      nil
-    end
-
     def last_known_value(keyspace, options, tags)
       lkk = @timestamp_manager.last_known_key(keyspace)
 
@@ -136,6 +115,7 @@ module AtomicCache
         # last known key may have expired
         if !lkv.nil?
           metrics(:increment, 'last-known-value.present', tags: tags)
+          log(:debug, "Read value from last known value key: '#{lkk}'")
           return lkv
         end
 
@@ -149,7 +129,7 @@ module AtomicCache
       nil
     end
 
-    def wait_for_new_value(key, options, tags)
+    def wait_for_new_value(keyspace, options, tags)
       max_retries = option(:max_retries, options, DEFAULT_MAX_RETRIES)
       max_retries.times do |attempt|
         metrics_tags = tags.clone.push("attempt:#{attempt}")
@@ -160,6 +140,8 @@ module AtomicCache
         backoff_duration_ms = option(:backoff_duration_ms, options, backoff_duration_ms)
         sleep((backoff_duration_ms.to_f / 1000) * attempt)
 
+        # re-fetch the key each time, to make sure we're actually getting the latest key with the correct LMT
+        key = @timestamp_manager.current_key(keyspace)
         value = @storage.read(key, options)
         if !value.nil?
           metrics(:increment, 'wait.present', tags: metrics_tags)
@@ -168,7 +150,7 @@ module AtomicCache
       end
 
       metrics(:increment, 'wait.give-up')
-      log(:warn, "Giving up fetching cache key `#{key}`. Exceeded max retries (#{max_retries}).")
+      log(:warn, "Giving up waiting. Exceeded max retries (#{max_retries}).")
       nil
     end
 

data/lib/atomic_cache/concerns/global_lmt_cache_concern.rb

@@ -26,7 +26,7 @@ module AtomicCache
         end
       end
 
-      def cache_class(kls)
+      def force_cache_class(kls)
         ATOMIC_CACHE_CONCERN_MUTEX.synchronize do
           @atomic_cache_class = kls
         end

data/lib/atomic_cache/storage/memory.rb

@@ -16,7 +16,7 @@ module AtomicCache
 
       def add(raw_key, new_value, ttl, user_options={})
         store_op(raw_key, user_options) do |key, options|
-          return false if store.has_key?(key)
+          return false if store.has_key?(key) && !ttl_expired?(store[key])
           write(key, new_value, ttl, user_options)
         end
       end
@@ -29,8 +29,7 @@ module AtomicCache
           unmarshaled = unmarshal(entry[:value], user_options)
           return unmarshaled if entry[:ttl].nil? or entry[:ttl] == false
 
-          life = Time.now - entry[:written_at]
-          if (life >= entry[:ttl])
+          if ttl_expired?(entry)
             store.delete(key)
             nil
           else
@@ -54,6 +53,12 @@ module AtomicCache
 
       protected
 
+      def ttl_expired?(entry)
+        return false unless entry
+        life = Time.now - entry[:written_at]
+        life >= entry[:ttl]
+      end
+
       def write(key, value, ttl=nil, user_options)
         store[key] = {
           value: marshal(value, user_options),

data/lib/atomic_cache/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module AtomicCache
-  VERSION = "0.2.3.rc1"
+  VERSION = "0.4.1.rc1"
 end

data/spec/atomic_cache/atomic_cache_client_spec.rb

@@ -138,17 +138,6 @@ describe 'AtomicCacheClient' do
             timestamp_manager.lock(keyspace, 100)
           end
 
-          it 'waits for a short duration to see if the other thread generated the value' do
-            timestamp_manager.promote(keyspace, last_known_key: 'lkk', timestamp: 1420090000)
-            key_storage.set('lkk', 'old:value')
-            new_value = 'value from another thread'
-            allow(cache_storage).to receive(:read)
-              .with(timestamp_manager.current_key(keyspace), anything)
-              .and_return(nil, new_value)
-
-            expect(subject.fetch(keyspace, quick_retry_ms: 5) { 'value' }).to eq(new_value)
-          end
-
           context 'when the last known value is present' do
             it 'returns the last known value' do
               timestamp_manager.promote(keyspace, last_known_key: 'lkk', timestamp: 1420090000)
@@ -191,17 +180,6 @@ describe 'AtomicCacheClient' do
       end
 
       context 'and when a block is NOT given' do
-        it 'waits for a short duration to see if the other thread generated the value' do
-          timestamp_manager.promote(keyspace, last_known_key: 'asdf', timestamp: 1420090000)
-          new_value = 'value from another thread'
-          allow(cache_storage).to receive(:read)
-            .with(timestamp_manager.current_key(keyspace), anything)
-            .and_return(nil, new_value)
-
-          result = subject.fetch(keyspace, quick_retry_ms: 50)
-          expect(result).to eq(new_value)
-        end
-
         it 'returns nil if nothing is present' do
           expect(subject.fetch(keyspace)).to eq(nil)
         end

data/spec/atomic_cache/concerns/global_lmt_cache_concern_spec.rb

@@ -104,12 +104,12 @@ describe 'AtomicCacheConcern' do
       class Foo2
         include AtomicCache::GlobalLMTCacheConcern
         cache_version(3)
-        cache_class('foo')
+        force_cache_class('foo')
       end
       Foo2
     end
 
-    it 'uses the given version and cache_class become part of the cache keyspace' do
+    it 'uses the given version and force_cache_class become part of the cache keyspace' do
       subject.expire_cache
       expect(key_storage.store).to have_key(:'foo:v3:lmt')
     end

data/spec/atomic_cache/integration/integration_spec.rb

@@ -0,0 +1,137 @@
+# frozen_string_literal: true
+
+require 'spec_helper'
+
+describe 'Integration -' do
+  let(:key_storage) { AtomicCache::Storage::SharedMemory.new }
+  let(:cache_storage) { AtomicCache::Storage::SharedMemory.new }
+  let(:keyspace) { AtomicCache::Keyspace.new(namespace: 'int.waiting') }
+  let(:timestamp_manager) { AtomicCache::LastModTimeKeyManager.new(keyspace: keyspace, storage: key_storage) }
+
+  before(:each) do
+    key_storage.reset
+    cache_storage.reset
+  end
+
+  describe 'fallback:' do
+    let(:generating_client) { AtomicCache::AtomicCacheClient.new(storage: cache_storage, timestamp_manager: timestamp_manager) }
+    let(:fallback_client) { AtomicCache::AtomicCacheClient.new(storage: cache_storage, timestamp_manager: timestamp_manager) }
+
+    it 'falls back to the old value when a lock is present' do
+      old_time = Time.local(2021, 1, 1, 15, 30, 0)
+      new_time = Time.local(2021, 1, 1, 16, 30, 0)
+
+      # prime cache with an old value
+
+      Timecop.freeze(old_time) do
+        generating_client.fetch(keyspace) { "old value" }
+      end
+      timestamp_manager.last_modified_time = new_time
+
+      # start generating process for new time
+      generating_thread = ClientThread.new(generating_client, keyspace)
+      generating_thread.start
+      sleep 0.05
+
+      value = fallback_client.fetch(keyspace)
+      generating_thread.terminate
+
+      expect(value).to eq("old value")
+    end
+  end
+
+  describe 'waiting:' do
+    let(:generating_client) { AtomicCache::AtomicCacheClient.new(storage: cache_storage, timestamp_manager: timestamp_manager) }
+    let(:waiting_client) { AtomicCache::AtomicCacheClient.new(storage: cache_storage, timestamp_manager: timestamp_manager) }
+
+    it 'waits for a key when no last know value is available' do
+      generating_thread = ClientThread.new(generating_client, keyspace)
+      generating_thread.start
+      waiting_thread = ClientThread.new(waiting_client, keyspace)
+      waiting_thread.start
+
+      generating_thread.generate
+      sleep 0.05
+      waiting_thread.fetch
+      sleep 0.05
+      generating_thread.complete
+      sleep 0.05
+
+      generating_thread.terminate
+      waiting_thread.terminate
+
+      expect(generating_thread.result).to eq([1, 2, 3])
+      expect(waiting_thread.result).to eq([1, 2, 3])
+    end
+  end
+end
+
+
+# Avert your eyes:
+# this class allows atomic client interaction to happen asynchronously so that
+# the waiting behavior of the client can be tested simultaneous to controlling how
+# long the 'generate' behavior takes
+#
+# It works by accepting an incoming 'message' which it places onto one of two queues
+class ClientThread
+  attr_reader :result
+
+  # idea: maybe make the return value set when the thread is initialized
+  def initialize(client, keyspace)
+    @keyspace = keyspace
+    @client = client
+    @msg_queue = Queue.new
+    @generate_queue = Queue.new
+    @result = nil
+  end
+
+  def start
+    @thread = Thread.new(&method(:run))
+  end
+
+  def fetch
+    @msg_queue << :fetch
+  end
+
+  def generate
+    @msg_queue << :generate
+  end
+
+  def complete
+    @generate_queue << :complete
+  end
+
+  def terminate
+    @msg_queue << :terminate
+  end
+
+  private
+
+  def run
+    loop do
+      msg = @msg_queue.pop
+      sleep 0.001; next unless msg
+
+      case msg
+      when :terminate
+        Thread.stop
+      when :generate
+        do_generate
+      when :fetch
+        @result = @client.fetch(@keyspace)
+      end
+    end
+  end
+
+  def do_generate
+    @client.fetch(@keyspace) do
+      loop do
+        msg = @generate_queue.pop
+        sleep 0.001; next unless msg
+        break if msg == :complete
+      end
+      @result = [1, 2, 3] # generated value
+      @result
+    end
+  end
+end

data/spec/atomic_cache/storage/memory_spec.rb

@@ -17,17 +17,34 @@ shared_examples 'memory storage' do
       expect(result).to eq(true)
     end
 
-    it 'does not write the key if it exists' do
-      entry = { value: Marshal.dump('foo'), ttl: 100, written_at: 100 }
+    # SharedMemory.new.add("foo", ttl: 100)
+
+    it 'does not write the key if it exists but expiration time is NOT up' do
+      entry = { value: Marshal.dump('foo'), ttl: 5000, written_at: Time.local(2021, 1, 1, 12, 0, 0) }
       subject.store[:key] = entry
 
-      result = subject.add('key', 'value', 200)
-      expect(result).to eq(false)
+      Timecop.freeze(Time.local(2021, 1, 1, 12, 0, 1)) do
+        result = subject.add('key', 'value', 5000)
+        expect(result).to eq(false)
+      end
 
       # stored values should not have changed
       expect(subject.store).to have_key(:key)
       expect(Marshal.load(subject.store[:key][:value])).to eq('foo')
-      expect(subject.store[:key][:ttl]).to eq(100)
+    end
+
+    it 'does write the key if it exists and expiration time IS up' do
+      entry = { value: Marshal.dump('foo'), ttl: 50, written_at: Time.local(2021, 1, 1, 12, 0, 0) }
+      subject.store[:key] = entry
+
+      Timecop.freeze(Time.local(2021, 1, 1, 12, 30, 0)) do
+        result = subject.add('key', 'value', 50)
+        expect(result).to eq(true)
+      end
+
+      # stored values should not have changed
+      expect(subject.store).to have_key(:key)
+      expect(Marshal.load(subject.store[:key][:value])).to eq('value')
     end
   end
 

data/spec/atomic_cache/storage/shared_memory_spec.rb

@@ -3,7 +3,7 @@
 require 'spec_helper'
 require_relative 'memory_spec'
 
-describe 'InstanceMemory' do
+describe 'SharedMemory' do
   subject { AtomicCache::Storage::SharedMemory.new }
   it_behaves_like 'memory storage'
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: atomic_cache
 version: !ruby/object:Gem::Version
-  version: 0.2.3.rc1
+  version: 0.4.1.rc1
 platform: ruby
 authors:
 - Ibotta Developers
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2021-06-23 00:00:00.000000000 Z
+date: 2021-07-07 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler
@@ -171,7 +171,8 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '0.1'
-description: desc
+description: A gem which prevents the thundering herd problem through a distributed
+  lock
 email: osscompliance@ibotta.com
 executables: []
 extensions: []
@@ -184,7 +185,6 @@ files:
 - docs/MODEL_SETUP.md
 - docs/PROJECT_SETUP.md
 - docs/USAGE.md
-- docs/img/quick_retry_graph.png
 - lib/atomic_cache.rb
 - lib/atomic_cache/atomic_cache_client.rb
 - lib/atomic_cache/concerns/global_lmt_cache_concern.rb
@@ -200,6 +200,7 @@ files:
 - spec/atomic_cache/atomic_cache_client_spec.rb
 - spec/atomic_cache/concerns/global_lmt_cache_concern_spec.rb
 - spec/atomic_cache/default_config_spec.rb
+- spec/atomic_cache/integration/integration_spec.rb
 - spec/atomic_cache/key/keyspace_spec.rb
 - spec/atomic_cache/key/last_mod_time_key_manager_spec.rb
 - spec/atomic_cache/storage/dalli_spec.rb
@@ -229,5 +230,9 @@ requirements: []
 rubygems_version: 3.0.8
 signing_key: 
 specification_version: 4
-summary: summary
+summary: In a nutshell:* The key of every cached value includes a timestamp* Once
+  a cache key is written to, it is never written over* When a newer version of a cached
+  value is available, it is written to a new key* When a new value is being generated
+  for a new key only 1 process is allowed to do so at a time* While the new value
+  is being generated, other processes read one key older than most recent
 test_files: []