juno 0.2.8 → 0.3.0

data/.travis.yml

@@ -9,7 +9,7 @@ services:
   - riak
   - couchdb
   - redis-server
-#  - cassandra
+  - cassandra
   - memcached
   - mongodb
 before_install:

data/Gemfile

@@ -42,8 +42,9 @@ gem 'sequel'
 gem 'dalli'
 gem 'riak-client'
 gem 'hashery'
-#gem 'cassandra'
+gem 'cassandra'
 #gem 'localmemcache'
+alternatives :mri => 'leveldb-ruby'
 alternatives :mri => 'tokyocabinet'
 alternatives :mri => 'memcached', :jruby => 'jruby-memcached'
 alternatives :mri => 'sqlite3', :jruby => %w(jdbc-sqlite3 activerecord-jdbc-adapter activerecord-jdbcsqlite3-adapter)

data/README.md

@@ -1,5 +1,5 @@
 Juno: A unified interface for key/value stores
-================================================
+==============================================
 
 [![Build Status](https://secure.travis-ci.org/minad/juno.png?branch=master)](http://travis-ci.org/minad/juno) [![Dependency Status](https://gemnasium.com/minad/juno.png?travis)](https://gemnasium.com/minad/juno) [![Code Climate](https://codeclimate.com/badge.png)](https://codeclimate.com/github/minad/juno)
 
@@ -14,8 +14,10 @@ Juno is very feature rich:
 * Custom serialization via `Juno::Transformer` proxy (Marshal/JSON/YAML and many more)
 * Custom key transformation via `Juno::Transformer` proxy
 * Value compression via `Juno::Transformer` proxy (Zlib, Snappy, QuickLZ, LZO)
-* Expiration for all stores (Added via proxy if not supported natively)
-* Integration with Rack as session store and Rack-Cache
+* Expiration for all stores (Added via proxy `Juno::Expires` if not supported natively)
+* Integration with [Rack](http://rack.github.com/) as cookie and session store and [Rack-Cache](https://github.com/rtomayko/rack-cache)
+
+Juno is tested thoroughly using [Travis-CI](http://travis-ci.org/minad/juno).
 
 Supported backends
 ------------------
@@ -23,44 +25,40 @@ Supported backends
 Out of the box, it supports the following backends:
 
 * Memory:
-    * In-memory store (:Memory)
-    * LRU hash (:LRUHash)
-    * LocalMemCache (:LocalMemCache)
-    * Memcached store (:Memcached, :MemcachedNative and :MemcachedDalli)
+    * In-memory store (`:Memory`)
+    * LRU hash (`:LRUHash`)
+    * LocalMemCache (`:LocalMemCache`)
+    * Memcached store (`:Memcached`, `:MemcachedNative` and `:MemcachedDalli`)
 * Relational Databases:
-    * DataMapper (:DataMapper)
-    * ActiveRecord (:ActiveRecord)
-    * Sequel (:Sequel)
-    * Sqlite3 (:Sqlite)
+    * DataMapper (`:DataMapper`)
+    * ActiveRecord (`:ActiveRecord`)
+    * Sequel (`:Sequel`)
+    * Sqlite3 (`:Sqlite`)
 * Filesystem:
-    * PStore (:PStore)
-    * YAML store (:YAML)
-    * Filesystem directory store (:File)
-    * Filesystem directory store which spreads files in subdirectories using md5 hash (:HashFile)
+    * PStore (`:PStore`)
+    * YAML store (`:YAML`)
+    * Filesystem directory store (`:File`)
+    * Filesystem directory store which spreads files in subdirectories using md5 hash (`:HashFile`)
 * Key/value databases:
-    * Berkeley DB (:DBM)
-    * GDBM (:GDBM)
-    * SDBM (:SDBM)
-    * Redis (:Redis)
-    * Riak (:Riak)
-    * TokyoCabinet (:TokyoCabinet)
-    * Cassandra (:Cassandra)
+    * Berkeley DB (`:DBM`)
+    * GDBM (`:GDBM`)
+    * SDBM (`:SDBM`)
+    * Redis (`:Redis`)
+    * Riak (`:Riak`)
+    * TokyoCabinet (`:TokyoCabinet`)
+    * Cassandra (`:Cassandra`)
+    * LevelDB (`:LevelDB`)
 * Document databases:
-    * CouchDB (:Couch)
-    * MongoDB (:Mongo)
+    * CouchDB (`:Couch`)
+    * MongoDB (`:Mongo`)
 * Other
-    * Fog cloud storage which supports Amazon S3, Rackspace, etc. (:Fog)
-    * Storage which doesn't store anything (:Null)
-
-Supported serializers:
+    * Fog cloud storage which supports Amazon S3, Rackspace, etc. (`:Fog`)
+    * Storage which doesn't store anything (`:Null`)
 
-* Marshal
-* YAML
-* JSON (via multi_json)
-* MessagePack
-* BSON
-* Ox
-* BERT
+Some of the backends are not exactly based on key/value stores, e.g. the relational ones. These
+are useful if you already use the corresponding backend in your application. You get a key/value
+store for free then without installing any additional services and you still have the possibility
+to upgrade to a real key/value store.
 
 Proxies
 -------
@@ -69,16 +67,45 @@ In addition it supports proxies (Similar to [Rack middlewares](http://rack.githu
 add additional features to storage backends:
 
 * `Juno::Proxy` proxy base class
-* `Juno::Expires` to add expiration support to stores which don't support it natively
-* `Juno::Stack` to stack multiple stores (Read returns result from first where the key is found, writes go to all stores)
-* `Juno::Transformer` transforms keys and values (Marshal, YAML, JSON, Base64, MD5, ...)
-* `Juno::Cache` combine two stores, one as backend and one as cache (e.g. Juno::Adapters::File + Juno::Adapters::Memory)
-* `Juno::Lock` to make store thread safe
-* `Juno::Logger` to log database accesses
+* `Juno::Expires` to add expiration support to stores which don't support it natively. Add it in the builder using `use :Expires`.
+* `Juno::Stack` to stack multiple stores (Read returns result from first where the key is found, writes go to all stores). Add it in the builder using `use :Stack`.
+* `Juno::Transformer` transforms keys and values (Marshal, YAML, JSON, Base64, MD5, ...). Add it in the builder using `use :Transformer`.
+* `Juno::Cache` combine two stores, one as backend and one as cache (e.g. `Juno::Adapters::File` + `Juno::Adapters::Memory`). Add it in the builder using `use :Cache`.
+* `Juno::Lock` to make store thread safe. Add it in the builder using `use :Lock`.
+* `Juno::Logger` to log database accesses. Add it in the builder using `use :Logger`.
 
 The Juno API is purposely extremely similar to the Hash API. In order so support an
 identical API across stores, it does not support iteration or partial matches.
 
+Supported serializers and compressors (`Juno::Transformer`)
+-----------------------------------------------------------
+
+Supported serializers:
+
+* BEncode (`:bencode`)
+* BERT (`:bert`)
+* BSON (`:bson`)
+* JSON (`:json`)
+* Marshal (`:marshal`)
+* MessagePack (`:msgpack`)
+* Ox (`:ox`)
+* TNetStrings (`:tnet`)
+* YAML (`:yaml`)
+
+Supported value compressors:
+
+* LZMA (`:lzma`)
+* LZO (`:lzo`)
+* Snappy (`:snappy`)
+* QuickLZ (`:quicklz`)
+* Zlib (`:zlib`)
+
+Special transformers:
+
+* Digests (MD5, Shas, ...)
+* Add prefix to keys (`:prefix`)
+* HMAC to verify values (`:hmac`, useful for `Rack::JunoCookies`)
+
 Links
 -----
 
@@ -175,9 +202,9 @@ end
 Framework Integration
 ---------------------
 
-Inspired by [redis-store](https://github.com/jodosha/redis-store) there exist integration classes for Rack and Rack-Cache.
+Inspired by [redis-store](https://github.com/jodosha/redis-store) there exist integration classes for [Rack](http://rack.github.com/) and [Rack-Cache](https://github.com/rtomayko/rack-cache).
 
-Use Juno as a Rack session store:
+Use Juno as a [Rack](http://rack.github.com/) session store:
 
 ~~~ ruby
 require 'rack/session/juno'
@@ -191,7 +218,7 @@ use Rack::Session::Juno do
 end
 ~~~
 
-Use Juno as a Rack-Cache store:
+Use Juno as a [Rack-Cache](https://github.com/rtomayko/rack-cache) store:
 
 ~~~ ruby
 require 'rack/cache/juno'
@@ -210,20 +237,23 @@ use Rack::Cache,
       :entity_store => 'juno://named_entitystore'
 ~~~
 
-Use Juno to store cookies in Rack:
+Use Juno to store cookies in [Rack](http://rack.github.com/). It uses the `Juno::Adapters::Cookie`. You might
+wonder what the purpose of this store or Rack middleware is: It makes it possible
+to use all the transformers on the cookies (e.g. `:prefix`, `:marshal` and `:hmac` for value verification).
 
 ~~~ ruby
 require 'rack/juno_cookies'
 
 use Rack::JunoCookies, :domain => 'example.com', :path => '/path'
-run lambda { |env|
+run lambda do |env|
   req = Rack::Request.new(env)
-  req.cookies #=> is now a Juno store!!
+  req.cookies #=> is now a Juno store!
+  env['rack.request.cookie_hash'] #=> is now a Juno store!
   req.cookies['key'] #=> retrieves 'key'
   req.cookies['key'] = 'value' #=> sets 'key'
   req.cookies.delete('key') #=> removes 'key'
-  [200,{},[]]
-}
+  [200, {}, []]
+end
 ~~~
 
 Alternatives

data/SPEC.md

@@ -65,20 +65,7 @@ The following methods may all take an additional Hash as a final argument. This
 
 In the case of methods with optional arguments, the Hash MUST be provided as the final argument. Keys in this Hash MUST be Symbols.
 
-# Key Equality
-
-Adapters MUST consider keys as equal to one another if and only if the value of <code>Marshal.dump(keya)</code> is the same (byte-for-byte) as <code>Marshal.dump(keyb)</code>. This does not mean that adapters are required to use <code>Marshal.dump</code> to calculate the key to use for a given key specified by the consumer of the adapter. However, if an adapter does not, it MUST guarantee that the value returned for every key is identical to the value that would be returned if it did a byte-for-byte comparison of the result of <code>Marshal.dump</code> for every operation involving a key.
-
-# Storage and Serialization
-
-In a Juno-compliant adapter, any Ruby object that can be serialized using Ruby's marshalling system may be used for keys or values.
-
-Adapters MAY use the marshalling system to serialize Ruby objects. Adapters MUST NOT return an Object from a fetch operation that existed on the heap prior to the fetch operation. The intention of this requirement is to prevent adapters that use the heap for persistence to store direct references to Objects passed into the <code>store</code> or <code>[]=</code> methods.
-
 # Atomicity
 
 The base Juno specification does not specify any atomicity guarantees. However, extensions to this spec may specify extensions that define additional guarantees for any of the defined operations.
 
-# Expiry
-
-The base Juno specification does not specify any mechanism for time-based expiry. However, extensions to this spec may specify mechanisms (using <code>store</code> to provide expiration semantics.

data/lib/juno.rb

@@ -19,6 +19,7 @@ module Juno
     autoload :File,            'juno/adapters/file'
     autoload :Fog,             'juno/adapters/fog'
     autoload :GDBM,            'juno/adapters/gdbm'
+    autoload :LevelDB,         'juno/adapters/leveldb'
     autoload :LocalMemCache,   'juno/adapters/localmemcache'
     autoload :LRUHash,         'juno/adapters/lruhash'
     autoload :Memcached,       'juno/adapters/memcached'
@@ -73,7 +74,7 @@ module Juno
     transformer = { :key => [key_serializer], :value => [value_serializer], :prefix => options.delete(:prefix) }
     transformer[:key] << :prefix if transformer[:prefix]
     transformer[:value] << (Symbol === compress ? compress : :zlib) if compress
-    raise 'Name must be Symbol' unless Symbol === name
+    raise ArgumentError, 'Name must be Symbol' unless Symbol === name
     case name
     when :Sequel, :ActiveRecord, :Couch
       # Sequel accept only base64 keys and values

data/lib/juno/adapters/cassandra.rb

@@ -13,42 +13,66 @@ module Juno
       # @param [Hash] options
       #
       # Options:
-      # * :keyspace - Cassandra keyspace (default Juno)
-      # * :column_family - Cassandra column family (default :Juno)
+      # * :keyspace - Cassandra keyspace (default 'juno')
+      # * :column_family - Cassandra column family (default 'juno')
       # * :host - Server host name (default 127.0.0.1)
       # * :port - Server port (default 9160)
       def initialize(options = {})
-        options[:keyspace] ||= 'Juno'
-        options[:host]     ||= '127.0.0.1'
-        options[:port]     ||=  9160
-        @column_family = options[:column_family] || :Juno
-        @client = ::Cassandra.new(options[:keyspace], "#{options[:host]}:#{options[:port]}")
+        options[:host] ||= '127.0.0.1'
+        options[:port] ||=  9160
+        keyspace = (options[:keyspace] ||= 'juno')
+        @cf = (options[:column_family] || 'juno').to_sym
+        @client = ::Cassandra.new('system', "#{options[:host]}:#{options[:port]}")
+        unless @client.keyspaces.include?(keyspace)
+          cf_def = ::Cassandra::ColumnFamily.new(:keyspace => keyspace, :name => @cf.to_s)
+          ks_def = ::Cassandra::Keyspace.new(:name => keyspace,
+                                             :strategy_class => 'SimpleStrategy',
+                                             :strategy_options => { 'replication_factor' => '1' },
+                                             :replication_factor => 1,
+                                             :cf_defs => [cf_def])
+          # Wait for keyspace to be created (issue #24)
+          10.times do
+            begin
+              @client.add_keyspace(ks_def)
+            rescue Exception => ex
+              puts "Cassandra: #{ex.message}"
+            end
+            break if @client.keyspaces.include?(keyspace)
+            sleep 0.1
+          end
+        end
+        @client.keyspace = keyspace
       end
 
       def key?(key, options = {})
-        @client.exists?(@column_family, key)
+        @client.exists?(@cf, key)
       end
 
       def load(key, options = {})
-        value = @client.get(@column_family, key)
-        value ? value['value'] : nil
+        value = @client.get(@cf, key)
+        if value
+          if options.include?(:expires)
+            store(key, value['value'], options)
+          else
+            value['value']
+          end
+        end
       end
 
       def delete(key, options = {})
         if value = load(key, options)
-          @client.remove(@column_family, key)
+          @client.remove(@cf, key)
           value
         end
       end
 
       def store(key, value, options = {})
-        @client.insert(@column_family, key,
-                       {'value' => value}, :ttl => options[:expires])
+        @client.insert(@cf, key, {'value' => value}, :ttl => options[:expires])
         value
       end
 
       def clear(options = {})
-        @client.each_key(@column_family) do |key|
+        @client.each_key(@cf) do |key|
           delete(key)
         end
         self

data/lib/juno/adapters/couch.rb

@@ -12,7 +12,7 @@ module Juno
       # Options:
       # * :db - Couch database
       def initialize(options = {})
-        raise 'Option :db is required' unless options[:db]
+        raise ArgumentError, 'Option :db is required' unless options[:db]
         @db = ::CouchRest.database!(options[:db])
       end
 
@@ -23,13 +23,13 @@ module Juno
       end
 
       def load(key, options = {})
-        @db.get(key)['data']
+        @db.get(key)['value']
       rescue RestClient::ResourceNotFound
         nil
       end
 
       def store(key, value, options = {})
-        doc = {'_id' => key, 'data' => value}
+        doc = {'_id' => key, 'value' => value}
         begin
           doc['_rev'] = @db.get(key)['_rev']
         rescue RestClient::ResourceNotFound
@@ -43,7 +43,7 @@ module Juno
       def delete(key, options = {})
         value = @db.get(key)
         @db.delete_doc('_id' => value['_id'], '_rev' => value['_rev'])
-        value['data']
+        value['value']
       rescue RestClient::ResourceNotFound
         nil
       end

data/lib/juno/adapters/datamapper.rb

@@ -21,7 +21,7 @@ module Juno
       # * :repository - Repository name (default :juno)
       # * :table - Table name (default :juno)
       def initialize(options = {})
-        raise 'Option :setup is required' unless options[:setup]
+        raise ArgumentError, 'Option :setup is required' unless options[:setup]
         @repository = options.delete(:repository) || :juno
         Store.storage_names[@repository] = (options.delete(:table) || :juno).to_s
         ::DataMapper.setup(@repository, options[:setup])

data/lib/juno/adapters/dbm.rb

@@ -12,7 +12,7 @@ module Juno
       # Options:
       # * :file - Database file
       def initialize(options = {})
-        raise 'Option :file is required' unless options[:file]
+        raise ArgumentError, 'Option :file is required' unless options[:file]
         @memory = ::DBM.new(options[:file])
       end
 

data/lib/juno/adapters/file.rb

@@ -6,9 +6,9 @@ module Juno
     # @api public
     class File < Base
       def initialize(options = {})
-        raise 'Option :dir is required' unless @dir = options[:dir]
+        raise ArgumentError, 'Option :dir is required' unless @dir = options[:dir]
         FileUtils.mkpath(@dir)
-        raise "#{@dir} is not a dir" unless ::File.directory?(@dir)
+        raise "#{@dir} is not a directory" unless ::File.directory?(@dir)
       end
 
       def key?(key, options = {})

data/lib/juno/adapters/fog.rb

@@ -13,7 +13,7 @@ module Juno
       # * :dir - Fog directory
       # * Other options passed to Fog::Storage#new
       def initialize(options = {})
-        raise 'Option :dir is required' unless dir = options.delete(:dir)
+        raise ArgumentError, 'Option :dir is required' unless dir = options.delete(:dir)
         storage = ::Fog::Storage.new(options)
         @directory = storage.directories.create(:key => dir)
       end

data/lib/juno/adapters/gdbm.rb

@@ -12,7 +12,7 @@ module Juno
       # Options:
       # * :file - Database file
       def initialize(options = {})
-        raise 'Option :file is required' unless options[:file]
+        raise ArgumentError, 'Option :file is required' unless options[:file]
         @memory = ::GDBM.new(options[:file])
       end
 

data/lib/juno/adapters/leveldb.rb

@@ -0,0 +1,35 @@
+require 'leveldb'
+
+module Juno
+  module Adapters
+    # LevelDB backend
+    # @api public
+    class LevelDB < Memory
+      # Constructor
+      #
+      # @param [Hash] options
+      #
+      # Options:
+      # * :dir - Database path
+      # * All other options passed to LevelDB::DB#new
+      def initialize(options = {})
+        raise ArgumentError, 'Option :dir is required' unless options[:dir]
+        @memory = ::LevelDB::DB.new(options[:dir])
+      end
+
+      def key?(key, options = {})
+        @memory.includes?(key)
+      end
+
+      def clear(options = {})
+        @memory.each {|k,v| delete(k, options) }
+        self
+      end
+
+      def close
+        @memory.close
+        nil
+      end
+    end
+  end
+end