lock_and_cache 2.1.1 → 2.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 19f9d5d8d273859b3802392c2c716bf2a60b897f
-  data.tar.gz: 45727c9b74dd4fb11b6e8e6fa9dd2655ae01f231
+  metadata.gz: b4e54a34f87ae8639c3bca086131b48998817736
+  data.tar.gz: 0063344db4cbe9d2cb01b2e192908810b2277822
 SHA512:
-  metadata.gz: f27d00caceafde847297680ca8e12c81d2d1f006ff93c50a1f711117d75bd0a1a73a6d118caf29a9cafe7adf5084d6ab66843a2af70e57fec2bd4323a228ad15
-  data.tar.gz: 7b66cc2ed3a4f4da75dfec0b22afb299163e999f5791cb59ecc931883910412d7a34845790d6abf2e9e4654b3e5821d2deb3098df1796e090d5a71364dda8161
+  metadata.gz: f2fca5d12d2e52b3c3616d4ed529b994299e30c20d203f2084f31220930842264c3bbdcbbac5c8d8caba98ce086ca3d9092b21e03d1bf39d91a04511fd49bb69
+  data.tar.gz: a22749c51adebc1e425a498ee3e4695bafb3512f3b5685d039ab2e04f3312957525f009c906060f172c6f225ff103017adeaf903af14231e2bad8ffae310f598

data/CHANGELOG

@@ -1,3 +1,11 @@
+2.2.0 / 2015-11-15
+
+* Enhancements
+
+  * Increase default heartbeat expires to 32 seconds from 2 (which was too strict IMO)
+  * Allow setting heartbeat_expires: globally (LockAndCache.heartbeat_expires=) or per call
+  * Provide LockAndCache.locked?()
+
 2.1.1 / 2015-10-26
 
 * Bug fixes

data/README.md

@@ -9,11 +9,27 @@
 
 Lock and cache using redis!
 
+Most caching libraries don't do locking, meaning that >1 process can be calculating a cached value at the same time. Since you presumably cache things because they cost CPU, database reads, or money, doesn't it make sense to lock while caching?
+
+## Quickstart
+
+```ruby
+LockAndCache.storage = Redis.new
+
+LockAndCache.lock_and_cache(:stock_price, {company: 'MSFT', date: '2015-05-05'}, expires: 10, nil_expires: 1) do
+  # get yer stock quote
+  # if 50 processes call this at the same time, only 1 will call the stock quote service
+  # the other 49 will wait on the lock, then get the cached value
+  # the value will expire in 10 seconds
+  # but if the value you get back is nil, that will expire after 1 second
+end
+```
+
 ## Sponsor
 
-<p><a href="http://faraday.io"><img src="https://s3.amazonaws.com/photos.angel.co/startups/i/175701-a63ebd1b56a401e905963c64958204d4-medium_jpg.jpg" alt="Faraday logo"/></a></p>
+<p><a href="http://faraday.io"><img src="http://cdn2.hubspot.net/hubfs/515497/img/logo.svg" alt="Faraday logo"/></a></p>
 
-We use [`lock_and_cache`](https://rubygems.org/gems/lock_and_cache) for [big data-driven marketing at Faraday](http://faraday.io).
+We use [`lock_and_cache`](https://github.com/seamusabshere/lock_and_cache) for [data-driven marketing at Faraday](http://faraday.io).
 
 ## TOC
 
@@ -23,6 +39,7 @@ We use [`lock_and_cache`](https://rubygems.org/gems/lock_and_cache) for [big dat
 
   - [Theory](#theory)
   - [Practice](#practice)
+    - [Setup](#setup)
     - [Locking](#locking)
     - [Caching](#caching)
       - [Standalone mode](#standalone-mode)
@@ -44,22 +61,18 @@ We use [`lock_and_cache`](https://rubygems.org/gems/lock_and_cache) for [big dat
 
 `lock_and_cache`...
 
-1. returns cached value if found
-2. acquires a lock
-3. returns cached value if found (just in case it was calculated while we were waiting for a lock)
-4. calculates and caches the value
-5. releases the lock
-6. returns the value
+1. <span style="color: red;">returns cached value</span> (if exists)
+2. <span style="color: green;">acquires a lock</span>
+3. <span style="color: red;">returns cached value</span> (just in case it was calculated while we were waiting for a lock)
+4. <span style="color: red;">calculates and caches the value</span>
+5. <span style="color: green;">releases the lock</span>
+6. <span style="color: red;">returns the value</span>
 
-As you can see, most caching libraries only take care of (1) and (4).
+As you can see, most caching libraries only take care of (1) and (4) (well, and (5) of course).
 
 ## Practice
 
-### Locking
-
-Based on [antirez's Redlock algorithm](http://redis.io/topics/distlock).
-
-Above and beyond Redlock, a 2-second heartbeat is used that will clear the lock if a process is killed. This is implemented using lock extensions.
+### Setup
 
 ```ruby
 LockAndCache.storage = Redis.new
@@ -67,79 +80,102 @@ LockAndCache.storage = Redis.new
 
 It will use this redis for both locking and storing cached values.
 
+### Locking
+
+Based on [antirez's Redlock algorithm](http://redis.io/topics/distlock).
+
+Above and beyond Redlock, a 32-second heartbeat is used that will clear the lock if a process is killed. This is implemented using lock extensions.
+
 ### Caching
 
-(be sure to set up storage as above)
+This gem is a simplified, improved version of https://github.com/seamusabshere/cache_method. In that library, you could only cache a method call.
 
-#### Standalone mode
+In this library, you have two options: providing the whole cache key every time (standalone) or letting the library pull information about its context.
 
 ```ruby
-LockAndCache.lock_and_cache('stock_price') do
-  # get yer stock quote
+# standalone example
+LockAndCache.lock_and_cache(:stock_price, {company: 'MSFT', date: '2015-05-05'}, expires: 10) do
+  # ...
+end
+
+# context example
+def stock_price(date)
+  lock_and_cache(date, expires: 10) do
+    # ...
+  end
+end
+def lock_and_cache_key
+  company
 end
 ```
 
-But that's probably not very useful without parameters
+#### Standalone mode
 
 ```ruby
-LockAndCache.lock_and_cache('stock_price', company: 'MSFT', date: '2015-05-05') do
+LockAndCache.lock_and_cache(:stock_price, company: 'MSFT', date: '2015-05-05') do
   # get yer stock quote
 end
 ```
 
-And you probably want an expiry
+You probably want an expiry
 
 ```ruby
-LockAndCache.lock_and_cache('stock_price', {company: 'MSFT', date: '2015-05-05'}, expires: 10) do
+LockAndCache.lock_and_cache(:stock_price, {company: 'MSFT', date: '2015-05-05'}, expires: 10) do
   # get yer stock quote
 end
 ```
 
 Note how we separated options (`{expires: 10}`) from a hash that is part of the cache key (`{company: 'MSFT', date: '2015-05-05'}`).
 
-You can clear a cache:
+One other crazy thing: `nil_expires` - for when you want to check more often if the external stock price service returned nil
+
+```ruby
+LockAndCache.lock_and_cache(:stock_price, {company: 'MSFT', date: '2015-05-05'}, expires: 10, nil_expires: 1) do
+  # get yer stock quote
+end
+```
+
+Clear it with
 
 ```ruby
-LockAndCache.lock_and_cache('stock_price', company: 'MSFT', date: '2015-05-05')
+LockAndCache.clear :stock_price, company: 'MSFT', date: '2015-05-05'
 ```
 
-One other crazy thing: let's say you want to check more often if the external stock price service returned nil
+Check locks with
 
 ```ruby
-LockAndCache.lock_and_cache('stock_price', {company: 'MSFT', date: '2015-05-05'}, expires: 10, nil_expires: 1) do
-  # get yer stock quote
-end
+LockAndCache.locked? :stock_price, company: 'MSFT', date: '2015-05-05'
 ```
 
 #### Context mode
 
 "Context mode" simply adds the class name, method name, and context key (the results of `#id` or `#lock_and_cache_key`) of the caller to the cache key.
 
-(This gem evolved from https://github.com/seamusabshere/cache_method, where you always cached a method call...)
-
 ```ruby
 class Stock
   include LockAndCache
 
-  def initialize(ticker_symbol)
+  def initialize(company)
     [...]
   end
 
-  def price(date)
-    lock_and_cache(date, expires: 10) do # <------ see how the cache key depends on the method args?
-      # do the work
+  def stock_price(date)
+    lock_and_cache(date, expires: 10) do
+      # the cache key will be StockQuote (the class) + get (the method name) + id (the instance identifier) + date (the arg you specified)
     end
   end
 
   def lock_and_cache_key # <---------- if you don't define this, it will try to call #id
-    ticker_symbol
+    company
   end
 end
 ```
 
-The key will be `{ StockQuote, :get, $id, $date, }`. In other words, it auto-detects the class, method, context key ... and you add other args if you want.
+The cache key will be StockQuote (the class) + get (the method name) + id (the instance identifier) + date (the arg you specified).
+
+In other words, it auto-detects the class, method, context key ... and you add other args if you want.
 
-Here's how to clear a cache in context mode:
+Clear it with
 
 ```ruby
 blog.lock_and_cache_clear(:get, date)

data/lib/lock_and_cache/action.rb

@@ -35,6 +35,9 @@ module LockAndCache
     def perform
       debug = (ENV['LOCK_AND_CACHE_DEBUG'] == 'true')
       max_lock_wait = options.fetch 'max_lock_wait', LockAndCache.max_lock_wait
+      heartbeat_expires = options.fetch('heartbeat_expires', LockAndCache.heartbeat_expires).to_f.ceil
+      raise "heartbeat_expires must be >= 2 seconds" unless heartbeat_expires >= 2
+      heartbeat_frequency = (heartbeat_expires / 2).ceil
       Thread.exclusive { $stderr.puts "[lock_and_cache] A1 #{key.debug} #{Base64.encode64(digest).strip} #{Digest::SHA1.hexdigest digest}" } if debug
       if storage.exists(digest) and (existing = storage.get(digest)).is_a?(String)
         return ::Marshal.load(existing)
@@ -45,7 +48,7 @@ module LockAndCache
       lock_info = nil
       begin
         Timeout.timeout(max_lock_wait, TimeoutWaitingForLock) do
-          until lock_info = lock_manager.lock(lock_digest, LockAndCache::LOCK_HEARTBEAT_EXPIRES*1000)
+          until lock_info = lock_manager.lock(lock_digest, heartbeat_expires*1000)
             Thread.exclusive { $stderr.puts "[lock_and_cache] C1 #{key.debug} #{Base64.encode64(digest).strip} #{Digest::SHA1.hexdigest digest}" } if debug
             sleep rand
           end
@@ -63,10 +66,10 @@ module LockAndCache
               loop do
                 Thread.exclusive { $stderr.puts "[lock_and_cache] heartbeat1 #{key.debug} #{Base64.encode64(digest).strip} #{Digest::SHA1.hexdigest digest}" } if debug
                 break if done
-                sleep LockAndCache::LOCK_HEARTBEAT_PERIOD
+                sleep heartbeat_frequency
                 break if done
                 Thread.exclusive { $stderr.puts "[lock_and_cache] heartbeat2 #{key.debug} #{Base64.encode64(digest).strip} #{Digest::SHA1.hexdigest digest}" } if debug
-                lock_manager.lock lock_digest, LockAndCache::LOCK_HEARTBEAT_EXPIRES*1000, extend: lock_info
+                lock_manager.lock lock_digest, heartbeat_expires*1000, extend: lock_info
               end
             end
             retval = blk.call

data/lib/lock_and_cache/version.rb

@@ -1,3 +1,3 @@
 module LockAndCache
-  VERSION = '2.1.1'
+  VERSION = '2.2.0'
 end

data/lib/lock_and_cache.rb

@@ -10,17 +10,13 @@ require_relative 'lock_and_cache/version'
 require_relative 'lock_and_cache/action'
 require_relative 'lock_and_cache/key'
 
-# Lock and cache methods using redis!
+# Lock and cache using redis!
 #
-# I bet you're caching, but are you locking?
+# Most caching libraries don't do locking, meaning that >1 process can be calculating a cached value at the same time. Since you presumably cache things because they cost CPU, database reads, or money, doesn't it make sense to lock while caching?
 module LockAndCache
   DEFAULT_MAX_LOCK_WAIT = 60 * 60 * 24 # 1 day in seconds
 
-  # @private
-  LOCK_HEARTBEAT_EXPIRES = 2
-
-  # @private
-  LOCK_HEARTBEAT_PERIOD = 1
+  DEFAULT_HEARTBEAT_EXPIRES = 32 # 32 seconds
 
   class TimeoutWaitingForLock < StandardError; end
 
@@ -51,7 +47,7 @@ module LockAndCache
   #
   # @note Standalone mode. See also "context mode," where you mix LockAndCache into a class and call it from within its methods.
   #
-  # @note A single hash arg is treated as a cached key. `LockAndCache.lock_and_cache(foo: :bar, expires: 100)` will be treated as a cache key of `foo: :bar, expires: 100` (which is probably wrong!!!). `LockAndCache.lock_and_cache({foo: :bar}, expires: 100)` will be treated as a cache key of `foo: :bar` and options `expires: 100`. This is the opposite of context mode and is true because we don't have any context to set the cache key from otherwise.
+  # @note A single hash arg is treated as a cache key, e.g. `LockAndCache.lock_and_cache(foo: :bar, expires: 100)` will be treated as a cache key of `foo: :bar, expires: 100` (which is probably wrong!!!). Try `LockAndCache.lock_and_cache({ foo: :bar }, expires: 100)` instead. This is the opposite of context mode.
   def LockAndCache.lock_and_cache(*key_parts_and_options, &blk)
     options = (key_parts_and_options.last.is_a?(Hash) && key_parts_and_options.length > 1) ? key_parts_and_options.pop : {}
     raise "need a cache key" unless key_parts_and_options.length > 0
@@ -68,6 +64,14 @@ module LockAndCache
     key.clear
   end
 
+  # Check if a key is locked
+  #
+  # @note Standalone mode. See also "context mode," where you mix LockAndCache into a class and call it from within its methods.
+  def LockAndCache.locked?(*key_parts)
+    key = LockAndCache::Key.new key_parts
+    key.locked?
+  end
+
   # @param seconds [Numeric] Maximum wait time to get a lock
   #
   # @note Can be overridden by putting `max_lock_wait:` in your call to `#lock_and_cache`
@@ -80,6 +84,20 @@ module LockAndCache
     @max_lock_wait || DEFAULT_MAX_LOCK_WAIT
   end
 
+  # @param seconds [Numeric] How often a process has to heartbeat in order to keep a lock
+  #
+  # @note Can be overridden by putting `heartbeat_expires:` in your call to `#lock_and_cache`
+  def LockAndCache.heartbeat_expires=(seconds)
+    memo = seconds.to_f
+    raise "heartbeat_expires must be greater than 2 seconds" unless memo >= 2
+    @heartbeat_expires = memo
+  end
+
+  # @private
+  def LockAndCache.heartbeat_expires
+    @heartbeat_expires || DEFAULT_HEARTBEAT_EXPIRES
+  end
+
   # @private
   def LockAndCache.lock_manager
     @lock_manager
@@ -87,7 +105,7 @@ module LockAndCache
 
   # Check if a method is locked on an object.
   #
-  # @note Subject mode - this is expected to be called on an object that has LockAndCache mixed in. See also standalone mode.
+  # @note Subject mode - this is expected to be called on an object whose class has LockAndCache mixed in. See also standalone mode.
   def lock_and_cache_locked?(method_id, *key_parts)
     key = LockAndCache::Key.new key_parts, context: self, method_id: method_id
     key.locked?
@@ -95,7 +113,7 @@ module LockAndCache
 
   # Clear a lock and cache given exactly the method and exactly the same arguments
   #
-  # @note Subject mode - this is expected to be called on an object that has LockAndCache mixed in. See also standalone mode.
+  # @note Subject mode - this is expected to be called on an object whose class has LockAndCache mixed in. See also standalone mode.
   def lock_and_cache_clear(method_id, *key_parts)
     key = LockAndCache::Key.new key_parts, context: self, method_id: method_id
     key.clear
@@ -103,15 +121,15 @@ module LockAndCache
 
   # Lock and cache a method given key parts.
   #
-  # This is the defining characteristic of context mode: the cache key will automatically include the class name of the object calling it (the context!) and the name of the method it is called from.
+  # The cache key will automatically include the class name of the object calling it (the context!) and the name of the method it is called from.
   #
   # @param key_parts_and_options [*] Parts that you want to include in the lock and cache key. If the last element is a Hash, it will be treated as options.
   #
   # @return The cached value (possibly newly calculated).
   #
-  # @note Subject mode - this is expected to be called on an object that has LockAndCache mixed in. See also standalone mode.
+  # @note Subject mode - this is expected to be called on an object whose class has LockAndCache mixed in. See also standalone mode.
   #
-  # @note A single hash arg is treated as an options hash. `lock_and_cache(expires: 100)` will be treated as options `expires: 100`. This is the opposite of standalone mode and true because we want to support people constructing cache keys from the context (context) PLUS an arbitrary hash of stuff.
+  # @note A single hash arg is treated as an options hash, e.g. `lock_and_cache(expires: 100)` will be treated as options `expires: 100`. This is the opposite of standalone mode.
   def lock_and_cache(*key_parts_and_options, &blk)
     options = key_parts_and_options.last.is_a?(Hash) ? key_parts_and_options.pop : {}
     key = LockAndCache::Key.new key_parts_and_options, context: self, caller: caller

data/spec/lock_and_cache_spec.rb

@@ -95,7 +95,7 @@ class Sleeper
   end
 
   def poke
-    lock_and_cache do
+    lock_and_cache heartbeat_expires: 2 do
       sleep
     end
   end
@@ -295,6 +295,16 @@ describe LockAndCache do
       end.to raise_error(/need/)
     end
 
+    it 'allows checking locks' do
+      expect(LockAndCache.locked?(:sleeper)).to be_falsey
+      t = Thread.new do
+        LockAndCache.lock_and_cache(:sleeper) { sleep 1 }
+      end
+      sleep 0.2
+      expect(LockAndCache.locked?(:sleeper)).to be_truthy
+      t.join
+    end
+
     it 'allows clearing' do
       count = 0
       expect(LockAndCache.lock_and_cache('hello') { count += 1 }).to eq(1)
@@ -304,6 +314,15 @@ describe LockAndCache do
       expect(count).to eq(2)
     end
 
+    it 'allows clearing (complex keys)' do
+      count = 0
+      expect(LockAndCache.lock_and_cache('hello', {world: 1}, expires: 100) { count += 1 }).to eq(1)
+      expect(count).to eq(1)
+      LockAndCache.clear('hello', world: 1)
+      expect(LockAndCache.lock_and_cache('hello', {world: 1}, expires: 100) { count += 1 }).to eq(2)
+      expect(count).to eq(2)
+    end
+
     it 'allows multi-part keys' do
       count = 0
       expect(LockAndCache.lock_and_cache(['hello', 1, { target: 'world' }]) { count += 1 }).to eq(1)

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: lock_and_cache
 version: !ruby/object:Gem::Version
-  version: 2.1.1
+  version: 2.2.0
 platform: ruby
 authors:
 - Seamus Abshere
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2015-10-26 00:00:00.000000000 Z
+date: 2015-11-15 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activesupport