moneta 1.0.0 → 1.5.1

data/SPEC.md

@@ -67,6 +67,67 @@ Closes the store
 
 Feature detection. Adapters MUST return <code>:create</code> and <code>:increment</code> if these methods are supported.
 
+### `each_key => Enumerator` and `each_key(&block) => Object`
+
+Enumerates over the keys in the store. This method is not supported by all
+stores. When not supported, this method MUST raise a `NotImplementedError`,
+regardless of whether a block is supplied. When supported, this method allows
+traversal of all keys in the store. The method behaves differently depending on
+whether a block is supplied.  In either case, for each key, `k` in the
+traversal, `key?(k)` MUST return `true`; and for each key, `k` for which
+`key?(k)` returns `true`, `k` MUST be traversed by `each_key`. Keys MAY be
+traversed in any order. Mutation of the store while traversing keys MAY be
+allowed. Querying the store (calling `fetch`, `key?`, etc.) while traversing
+MUST be allowed.
+
+* If no block is supplied, `each_key` MUST return an `Enumerator` that can be
+  used to traverse each key (e.g. by calling `each`). Calling methods on the
+  `Enumerator` such as `each` with a block MUST return the store object.
+
+* If a block is supplied, that block MUST be called once with each traversed key
+  as the only argument.  When called in this way, `each_key` MUST return the
+  store object.
+
+### `values_at(*keys[Array<Object>], **options[Hash]) => Array<Object>`
+
+Returns an array containing the values associated with the given keys, in the
+same order as the supplied keys.  If a key is not present in the
+key-value-store, `nil` MUST be returned in its place.  For each key, and each
+value, the same restrictions apply as apply to individual keys passed to, and
+values received from the store in the specification of `[]` (see above).  The
+adapter MAY perform this operation atomically.
+
+### `fetch_values(*keys[Array<Object>], **options[Hash], &defaults) => Array<Object>`
+
+Behaves identically to `values_at`, except that it MUST accept an optional
+block. When supplied, the block will be called successively with each supplied
+key that is not present in the store.  The return value of the block call MUST
+be used in place of `nil` in returned the array of values.  As with `fetch`
+(above), the adapter MUST NOT store the return value of the block call in the
+key-value-store.  The adapter MAY perform this operation atomically.
+
+### `slice(*keys[Array<Object>], **options[Hash]) => <Array(Object, Object)>`
+
+Returns a collection of key-value pairs corresponding to those supplied keys
+which are present in the key-value store, and their associated values.  A key
+MUST be present in the return value if and only if it was supplied in the `keys`
+parameter and it is present in the key-value store.  For each key, and each
+value, the same restrictions apply as apply to individual keys passed to, and
+values received from the store in the specification of `[]` (see above).  The
+adapter MAY perform this operation atomically.
+
+### `merge!(pairs[<Array(Object, Object)>], options[Hash] => {}, &block) => self`
+
+Stores the pairs in the key-value-store, and returns the store object.  This
+method MUST behave identically to successively calling `[]=` with each key-value
+pair and the options hash; except that the adapter MAY perform this operation
+atomically, and the method MUST accept an optional block, which MUST be called
+for each key that is to be overwritten.  When the block is provided, it MUST be
+called before overwriting any existing values with the key, old value and
+supplied value, and the return value of the block MUST be used in place of the
+supplied value. `merge!` MUST also be aliased as `update`.
+
+
 ## Additional Options Hashes
 
 The following methods may all take an additional Hash as a final argument. This allows the client to send additional options which can be specified by the adapter (and which may be specified by extensions to this specification). The methods MUST NOT modify the supplied option hash.
@@ -78,6 +139,15 @@ The following methods may all take an additional Hash as a final argument. This
 * key?
 * increment
 * clear
+* merge!
+
+Additionally, the following methods accept options as keyword arguments, after
+non-keyword arguments.  These keyword arguments are treated as a hash,
+equivalent to supplying a hash to the above methods.
+
+* values_at
+* fetch_values
+* slice
 
 In the case of methods with optional arguments, the Hash MUST be provided as the final argument. Keys in this Hash MUST be Symbols.
 

data/feature_matrix.yaml

@@ -0,0 +1,227 @@
+---
+group: Persistent stores
+features: [ persist ]
+notes:
+  poor multiprocess performance: >
+    Sqlite/YAML/PStore are multiprocess safe, but the performance suffers badly
+    since the whole database file must be locked for writing. Use a key/value
+    server if you want multiprocess concurrency!
+backends:
+  - adapter: Mongo
+    platforms: [ MRI, JRuby ]
+    gems: mongo
+    features: [ threadsafe, multiprocess, increment, create, expires, each_key, bulk_read, bulk_write ]
+    description: "[MongoDB](http://www.mongodb.org/) database"
+  - adapter: Redis
+    platforms: [ MRI, JRuby ]
+    gems: redis
+    features: [ threadsafe, multiprocess, increment, create, expires, each_key, bulk_read, bulk_write ]
+    description: "[Redis](http://redis.io/) database"
+  - adapter: ActiveRecord
+    platforms: [ MRI, JRuby ]
+    gems: activerecord
+    features: [ threadsafe, multiprocess, increment, create, each_key, bulk_read, bulk_write ]
+    description: "[ActiveRecord](https://rubygems.org/gems/activerecord) ORM"
+  - adapter: File
+    platforms: [ MRI, JRuby ]
+    features: [ threadsafe, multiprocess, increment, create, each_key ]
+    description: "File store"
+  - adapter: LMDB
+    platforms: [ MRI ]
+    gems: lmdb
+    features: [ threadsafe, multiprocess, increment, create, each_key, bulk_read, bulk_write ]
+    description: "[Symas Lightning Memory-Mapped Database (LMDB)](http://symas.com/mdb/)"
+  - adapter: Sequel
+    platforms: [ MRI, JRuby ]
+    gems: sequel
+    features: [ threadsafe, multiprocess, increment, create, each_key, bulk_read, bulk_write ]
+    description: "[Sequel](http://sequel.rubyforge.org/) ORM"
+  - adapter: TokyoTyrant
+    platforms: [ MRI, JRuby ]
+    gems: tokyotyrant or ruby-tokyotyrant
+    features: [ multiprocess, increment, create, bulk_read ]
+    description: "[TokyoTyrant](http://fallabs.com/tokyotyrant/) database"
+  - adapter: PStore
+    platforms: [ MRI ]
+    features: [ multiprocess, increment, create, each_key, bulk_read, bulk_write ]
+    description: "[PStore](http://ruby-doc.org/stdlib/libdoc/pstore/rdoc/PStore.html) store"
+    notes:
+      multiprocess: poor multiprocess performance
+  - adapter: YAML
+    platforms: [ MRI, JRuby ]
+    features: [ multiprocess, increment, create, each_key ]
+    description: "[YAML](http://www.ruby-doc.org/stdlib/libdoc/yaml/rdoc/YAML/Store.html) store"
+    notes:
+      multiprocess: poor multiprocess performance
+  - adapter: Sqlite
+    platforms: [ MRI ]
+    gems: sqlite3
+    features: [ multiprocess, increment, create, bulk_read, bulk_write, each_key ]
+    description: "[Sqlite3](http://sqlite.org/) database"
+    unknown: [ threadsafe ]
+    notes:
+      multiprocess: poor multiprocess performance
+  - adapter: Daybreak
+    platforms: [ MRI, JRuby ]
+    gems: daybreak
+    features: [ multiprocess, increment, create, each_key, bulk_write ]
+    description: "Incredibly fast pure-ruby key/value store [Daybreak](http://propublica.github.com/daybreak/)"
+  - adapter: DBM
+    platforms: [ MRI ]
+    features: [ increment, create, each_key, bulk_read, bulk_write ]
+    description: "[Berkeley DB using DBM interface or NDBM (Depends on Ruby environment)](http://www.ruby-doc.org/stdlib/libdoc/dbm/rdoc/DBM.html)"
+  - adapter: GDBM
+    platforms: [ MRI, JRuby ]
+    gems: ffi-gdbm on JRuby
+    features: [ increment, create, each_key, bulk_read, bulk_write ]
+    description: "[GDBM](http://www.ruby-doc.org/stdlib/libdoc/gdbm/rdoc/GDBM.html) database"
+  - adapter: LevelDB
+    platforms: [ MRI ]
+    gems: leveldb
+    features: [ increment, create, each_key, bulk_read, bulk_write ]
+    description: "[LevelDB](http://code.google.com/p/leveldb/) database"
+  - adapter: SDBM
+    platforms: [ MRI ]
+    features: [ increment, create, each_key, bulk_read, bulk_write ]
+    description: "[SDBM](http://www.ruby-doc.org/stdlib/libdoc/sdbm/rdoc/SDBM.html) database"
+  - adapter: TDB
+    platforms: [ MRI ]
+    gems: tdb
+    features: [ increment, create, each_key ]
+    description: "[TDB](http://tdb.samba.org/) database"
+  - adapter: KyotoCabinet
+    platforms: [ MRI ]
+    gems: kyotocabinet-ruby or kyotocabinet-ruby-reanimated
+    features: [ increment, create, each_key, bulk_read, bulk_write ]
+    description: "[KyotoCabinet](http://fallabs.com/kyotocabinet/) database"
+  - adapter: TokyoCabinet
+    platforms: [ MRI ]
+    gems: tokyocabinet
+    features: [ increment, create, each_key ]
+    description: "[TokyoCabinet](http://fallabs.com/tokyocabinet/) database"
+  - adapter: DataMapper
+    platforms: [ MRI ]
+    gems: dm-core, dm-migrations
+    features: [ threadsafe, multiprocess, create ]
+    description: "[DataMapper](http://datamapper.org/) ORM"
+  - adapter: Couch
+    platforms: [ MRI, JRuby ]
+    gems: faraday, multi_json
+    features: [ multiprocess, create, each_key, bulk_read, bulk_write ]
+    description: "[CouchDB](http://couchdb.apache.org/) database"
+  - adapter: HBase
+    platforms: [ Unstable ]
+    gems: hbaserb
+    features: [ multiprocess, increment ]
+    unknown: [threadsafe]
+    description: "[HBase](http://hbase.apache.org/) database"
+  - adapter: Cassandra
+    platforms: [ MRI, JRuby ]
+    gems: cassandra
+    features: [ multiprocess, expires, each_key, bulk_read, bulk_write ]
+    unknown: [threadsafe]
+    description: "[Cassandra](http://cassandra.apache.org/) distributed database"
+  - adapter: LocalMemCache
+    platforms: [ MRI ]
+    gems: localmemcache
+    features: [ threadsafe, multiprocess ]
+    description: "[LocalMemCache](http://localmemcache.rubyforge.org/) database"
+  - adapter: Fog
+    platforms: [ MRI, JRuby ]
+    gems: fog
+    features: [multiprocess]
+    unknown: [threadsafe]
+    description: "[Fog](http://fog.io/) cloud store"
+  - adapter: Riak
+    platforms: [ Unstable ]
+    gems: riak-client
+    features: [multiprocess]
+    description: "[Riak](http://docs.basho.com/) database"
+---
+group: Non-persistent stores
+notes:
+  memcached protocol: >
+    There are some servers which use the memcached protocol but which are persistent
+    (e.g. [MemcacheDB](http://memcachedb.org/),
+    [Kai](http://sourceforge.net/apps/mediawiki/kai),
+    [IronCache](http://dev.iron.io/cache/reference/memcache/),
+    [Roma](https://github.com/roma/roma/tree),
+    [Flare](http://labs.gree.jp/Top/OpenSource/Flare-en.html) and
+    [Kumofs](https://github.com/etolabo/kumofs))
+  memcached backend: >
+    This feature is only available if the dalli backend is selected
+  in-memory multiprocess caveat: >
+    Store is multi-process safe because it is an in-memory store, values are not
+    shared between multiple processes
+backends:
+  - adapter: MemcachedDalli
+    platforms: [ MRI, JRuby ]
+    gems: dalli
+    features: [ threadsafe, multiprocess, increment, create, expires, bulk_read, bulk_write ]
+    description: "[Memcached](http://memcached.org/) database with Dalli library"
+    notes:
+      persist: memcached protocol
+  - adapter: Memcached
+    platforms: [ MRI ]
+    gems: dalli or memcached
+    features: [ multiprocess, increment, create, expires ]
+    unknown: [ threadsafe, bulk_read, bulk_write, JRuby ]
+    description: "[Memcached](http://memcached.org/) database"
+    notes:
+      persist: memcached protocol
+      JRuby: memcached backend
+      threadsafe: memcached backend
+      bulk_read: memcached backend
+      bulk_write: memcached backend
+  - adapter: MemcachedNative
+    platforms: [ MRI ]
+    gems: memcached
+    features: [ multiprocess, increment, create, expires ]
+    description: "Memcached database with native library"
+    notes:
+      persist: memcached protocol
+  - adapter: Cookie
+    platforms: [ MRI, JRuby ]
+    features: [ multiprocess, increment, create, expires, each_key ]
+    description: "Cookie in memory store"
+    notes:
+      multiprocess: in-memory multiprocess caveat
+  - adapter: LRUHash
+    platforms: [ MRI, JRuby ]
+    features: [ multiprocess, increment, create, each_key ]
+    description: "LRU memory store"
+    notes:
+      multiprocess: in-memory multiprocess caveat
+  - adapter: Memory
+    platforms: [ MRI, JRuby ]
+    features: [ multiprocess, increment, create, each_key, bulk_read, bulk_write ]
+    description: Memory store
+    notes:
+      multiprocess: in-memory multiprocess caveat
+  - adapter: "Null"
+    platforms: [ MRI, JRuby ]
+    features: [ threadsafe, multiprocess ]
+    description: "No database"
+---
+group: Network clients
+notes:
+  depends on server: Depends on server
+backends:
+  - adapter: Client
+    platforms: [ MRI, JRuby ]
+    features: [multiprocess]
+    unknown: [ increment, create, expires, persist, each_key ]
+    description: "Moneta client adapter"
+    notes:
+      increment: depends on server
+      create: depends on server
+      expires: depends on server
+      persist: depends on server
+      each_key: depends on server
+  - adapter: RestClient
+    platforms: [ MRI, JRuby ]
+    features: [ multiprocess ]
+    unknown: [ persist ]
+    description: "Moneta REST client adapter"
+    notes:
+      persist: depends on server

data/lib/action_dispatch/middleware/session/moneta_store.rb

@@ -3,6 +3,7 @@ require 'action_dispatch/middleware/session/abstract_store'
 
 module ActionDispatch
   module Session
+    # A Rails cache backed by any Moneta store
     class MonetaStore < Rack::Session::Moneta
       include Compatibility
       include StaleSessionCheck

data/lib/active_support/cache/moneta_store.rb

@@ -12,14 +12,14 @@ module ActiveSupport
       def increment(key, amount = 1, options = nil)
         options = merged_options(options)
         instrument(:increment, key, amount: amount) do
-          @store.increment(namespaced_key(key, options), amount, moneta_options(options))
+          @store.increment(normalize_key(key, options), amount, moneta_options(options))
         end
       end
 
       def decrement(key, amount = 1, options = nil)
         options = merged_options(options)
         instrument(:decrement, key, amount: amount) do
-          @store.increment(namespaced_key(key, options), -amount, moneta_options(options))
+          @store.increment(normalize_key(key, options), -amount, moneta_options(options))
         end
       end
 
@@ -30,11 +30,79 @@ module ActiveSupport
         end
       end
 
+      # This prevents underlying Moneta transformers from erroring on raw values
+      def exist?(name, options = {})
+        super
+      rescue
+        super(name, options.merge(raw: true))
+      end
+
+      # These are the rails 5.2 versions of these methods, which call into the
+      # lower-level read_multi_entries and write_multi_entries methods.  We
+      # define them here only if the superclass versions don't use the *_entries
+      # methods.
+      unless [:read_multi_entries, :write_multi_entries].all? { |m| superclass.private_instance_methods.include? m }
+        def fetch_multi(*names)
+          raise ArgumentError, "Missing block: `Cache#fetch_multi` requires a block." \
+            unless block_given?
+
+          options = names.extract_options!
+          options = merged_options(options)
+
+          instrument :read_multi, names, options do |payload|
+            read_multi_entries(names, options).tap do |results|
+              payload[:hits] = results.keys
+              payload[:super_operation] = :fetch_multi
+
+              writes = {}
+
+              (names - results.keys).each do |name|
+                results[name] = writes[name] = yield(name)
+              end
+
+              write_multi writes, options
+            end
+          end
+        end
+
+        def read_multi(*names)
+          options = names.extract_options!
+          options = merged_options(options)
+
+          instrument :read_multi, names, options do |payload|
+            read_multi_entries(names, options).tap do |results|
+              payload[:hits] = results.keys
+            end
+          end
+        end
+
+        def write_multi(hash, options = nil)
+          options = merged_options(options)
+
+          instrument :write_multi, hash, options do
+            entries = hash.each_with_object({}) do |(name, value), memo|
+              memo[normalize_key(name, options)] = \
+                Entry.new(value, options.merge(version: normalize_version(name, options)))
+            end
+
+            write_multi_entries entries, options
+          end
+        end
+      end
+
       protected
 
+      def make_entry(value)
+        case value
+        when ActiveSupport::Cache::Entry, nil
+          value
+        else
+          ActiveSupport::Cache::Entry.new(value)
+        end
+      end
+
       def read_entry(key, options)
-        entry = @store.load(key, moneta_options(options))
-        entry && (ActiveSupport::Cache::Entry === entry ? entry : ActiveSupport::Cache::Entry.new(entry))
+        make_entry(@store.load(key, moneta_options(options, false)))
       end
 
       def write_entry(key, entry, options)
@@ -48,11 +116,39 @@ module ActiveSupport
         true
       end
 
+      def read_multi_entries(names, options)
+        keys = names.map { |name| normalize_key(name, options) }
+        entries = @store
+          .values_at(*keys, **moneta_options(options, false))
+          .map(&method(:make_entry))
+
+        names.zip(keys, entries).map do |name, key, entry|
+          next if entry == nil
+          delete_entry(key, options) if entry.expired?
+          next if entry.expired? || entry.mismatched?(normalize_version(name, options))
+
+          [name, entry.value]
+        end.compact.to_h
+      end
+
+      def write_multi_entries(hash, options)
+        pairs = if options[:raw]
+                  hash.transform_values { |entry| entry.value.to_s }
+                else
+                  hash
+                end
+
+        @store.merge!(pairs, moneta_options(options))
+        hash
+      end
+
       private
 
-      def moneta_options(options)
+      def moneta_options(options, include_expires = true)
         new_options = options ? options.dup : {}
-        new_options[:expires] = new_options.delete(:expires_in).to_i if new_options.include?(:expires_in)
+        if new_options.include?(:expires_in) and include_expires
+          new_options[:expires] = new_options.delete(:expires_in).to_r
+        end
         new_options
       end
     end

data/lib/moneta/adapter.rb

@@ -0,0 +1,52 @@
+module Moneta
+  # Adapter base class
+  # @api public
+  class Adapter
+    include Defaults
+    include Config
+
+    attr_reader :backend
+
+    class << self
+      # Define a block used to build this adapter's backend.  The block will
+      # receive as keyword arguments any options passed to the adapter during
+      # initialization that are not config settings.
+      #
+      # If the adapter is initialized with a `:backend` option, this will be used
+      # instead, and the block won't be called.
+      #
+      # @param [Boolean] required
+      # @yield [**options] options passed to the adapter's initialize method
+      # @yieldreturn [Object] The backend to use
+      def backend(required: true, &block)
+        raise "backend block already set" if class_variables(false).include?(:@@backend_block)
+        class_variable_set(:@@backend_block, block)
+        class_variable_set(:@@backend_required, true) if required
+      end
+
+      def backend_block
+        class_variable_get(:@@backend_block) if class_variable_defined?(:@@backend_block)
+      end
+
+      def backend_required?
+        class_variable_defined?(:@@backend_required)
+      end
+    end
+
+    # @param [Hash] options
+    def initialize(options = {})
+      set_backend(**configure(**options))
+    end
+
+    private
+
+    def set_backend(backend: nil, **options)
+      @backend = backend ||
+        if backend_block = self.class.backend_block
+          instance_exec(**options, &backend_block)
+        end
+
+      raise ArgumentError, 'backend needs to be set - refer to adapter documentation' if !@backend && self.class.backend_required?
+    end
+  end
+end