dalli 0.9.2 → 0.9.3

Sign up to get free protection for your applications and to get access to all the features.

Potentially problematic release.


This version of dalli might be problematic. Click here for more details.

data/History.md CHANGED
@@ -1,6 +1,15 @@
1
1
  Dalli Changelog
2
2
  =====================
3
3
 
4
+ 0.9.3
5
+ -----
6
+
7
+ - Rails 2.3 support (beanieboi)
8
+ - Rails SessionStore support
9
+ - Passenger integration
10
+ - memcache-client upgrade docs, see Upgrade.md
11
+
12
+
4
13
  0.9.2
5
14
  ----
6
15
 
data/README.md CHANGED
@@ -27,6 +27,7 @@ So a few notes. Dalli:
27
27
  2. contains explicit "chokepoint" methods which handle all requests; these can be hooked into by monitoring tools (NewRelic, Rack::Bug, etc) to track memcached usage.
28
28
  3. comes with hooks to replace memcache-client in Rails.
29
29
  4. is approx 700 lines of Ruby. memcache-client is approx 1250 lines.
30
+ 5. supports SASL for use in managed environments, e.g. Heroku.
30
31
 
31
32
 
32
33
  Installation and Usage
@@ -44,23 +45,55 @@ Remember, Dalli **requires** memcached 1.4+. You can check the version with `me
44
45
  The test suite requires memcached 1.4.3+ with SASL enabled (./configure --enable-sasl). Currently only supports the PLAIN mechanism.
45
46
 
46
47
 
47
- Usage with Rails
48
+ Usage with Rails 3.0
48
49
  ---------------------------
49
50
 
50
51
  In your Gemfile:
51
52
 
52
53
  gem 'dalli'
53
54
 
54
- In `config/environments/production.rb`. Note that we are also setting a reasonable default for maximum cache entry lifetime (one day), enabling compression for large values, and namespacing all entries for this rails app. Remove the namespace if you have multiple apps which share cached values.
55
+ In `config/environments/production.rb`:
56
+
57
+ config.cache_store = :dalli_store
58
+
59
+ A more comprehensive example (note that we are setting a reasonable default for maximum cache entry lifetime (one day), enabling compression for large values, and namespacing all entries for this rails app. Remove the namespace if you have multiple apps which share cached values):
55
60
 
56
- require 'active_support/cache/dalli_store'
57
61
  config.cache_store = :dalli_store, 'cache-1.example.com', 'cache-2.example.com',
58
62
  :namespace => NAME_OF_RAILS_APP, :expires_in => 1.day, :compress => true, :compress_threshold => 64.kilobytes
59
63
 
60
64
 
65
+ Usage with Rails 2.3.x
66
+ ----------------------------
67
+
68
+ In `config/environment.rb`:
69
+
70
+ config.gem 'dalli'
71
+
72
+ In `config/environments/production.rb`:
73
+
74
+ require 'active_support/cache/dalli_store23'
75
+ config.cache_store = :dalli_store
76
+
77
+
78
+ Usage with Passenger
79
+ ------------------------
80
+
81
+ Put this at the bottom of `config/environment.rb`:
82
+
83
+ if defined?(PhusionPassenger)
84
+ PhusionPassenger.on_event(:starting_worker_process) do |forked|
85
+ # Only works with DalliStore
86
+ Rails.cache.reset if forked
87
+ end
88
+ end
89
+
90
+
61
91
  Features and Changes
62
92
  ------------------------
63
93
 
94
+ Dalli is **NOT** 100% API compatible with memcache-client. If you have code which uses the MemCache
95
+ API directly, it will likely need small tweaks. Method parameters and return values changed slightly.
96
+
64
97
  memcache-client allowed developers to store either raw or marshalled values with each API call. I feel this is needless complexity; Dalli allows you to control marshalling per-Client with the `:marshal => false` flag but you cannot explicitly set the raw flag for each API call. By default, marshalling is enabled.
65
98
 
66
99
  I've removed support for key namespaces and automatic pruning of keys longer than 250 characters. ActiveSupport::Cache implements these features so there is little need for Dalli to reinvent them.
@@ -76,6 +109,12 @@ Helping Out
76
109
  If you have a fix you wish to provide, please fork the code, fix in your local project and then send a pull request on github. Please ensure that you include a test which verifies your fix and update History.md with a one sentence description of your fix so you get credit as a contributor.
77
110
 
78
111
 
112
+ Thanks
113
+ ------------
114
+
115
+ Brian Mitchell - for his remix-stash project which was helpful when implementing and testing the binary protocol support.
116
+
117
+
79
118
  Author
80
119
  ----------
81
120
 
data/Rakefile CHANGED
@@ -4,4 +4,9 @@ Rake::TestTask.new(:test) do |test|
4
4
  test.pattern = 'test/**/test_*.rb'
5
5
  end
6
6
 
7
+ Rake::TestTask.new(:bench) do |test|
8
+ test.libs << 'test'
9
+ test.pattern = 'test/benchmark_test.rb'
10
+ end
11
+
7
12
  task :default => :test
data/TODO.md CHANGED
@@ -3,3 +3,4 @@ TODO
3
3
 
4
4
  * More of the memcached instruction set. This will be done based on user demand. Email me if the API is missing a feature you'd like to use.
5
5
  * Better API documentation
6
+ * Detect and fail on older memcached servers (pre-1.4)
@@ -0,0 +1,44 @@
1
+ Upgrading from memcache-client
2
+ ========
3
+
4
+ Dalli is not meant to be 100% compatible with memcache-client, there are serveral differences in the API.
5
+
6
+
7
+ Marshalling
8
+ ---------------
9
+
10
+ Dalli has removed support for specifying the marshalling behavior for each operation.
11
+
12
+ Take this typical operation:
13
+
14
+ cache = MemCache.new
15
+ cache.set('abc', 123)
16
+
17
+ Technically 123 is an Integer and presumably you want `cache.get('abc')` to return an Integer. Since memcached stores values as binary blobs, Dalli will serialize the value to a binary blob for storage. When you get() the value back, Ruby will deserialize it properly to an Integer and all will be well. Without marshalling, Dalli will convert values to Strings and so get() would return a String, not an Integer.
18
+
19
+ The memcache-client API allowed you to control marshalling on a per-method basis using a boolean 'raw' parameter to several of the API methods:
20
+
21
+ cache = MemCache.new
22
+ cache.set('abc', 123, 0, true)
23
+ cache.get('abc', true) => '123'
24
+
25
+ cache.set('abc', 123, 0)
26
+ cache.get('abc') => 123
27
+
28
+ Note that the last 'raw' parameter is set to true in the first two API calls and so `get` returns a string, not an integer. In the second example, we don't provide the raw parameter. Since it defaults to false, it works exactly like Dalli.
29
+
30
+ If the code specifies raw as false, you can simply remove that parameter. If the code is using raw = true, you will need to use the :marshal option to create a Dalli::Client instance that does not perform marshalling:
31
+
32
+ cache = Dalli::Client.new(servers, :marshal => false)
33
+
34
+ If the code is mixing marshal modes (performing operations where raw is both true and false), you will need to use two different client instances.
35
+
36
+
37
+ Return Values
38
+ ----------------
39
+
40
+ In memcache-client, `set(key, value)` normally returns "STORED\r\n". This is an artifact of the text protocol used in earlier versions of memcached. Code that checks the return value will need to be updated. Dalli raises errors for exceptional cases but otherwise returns true or false depending on whether the operation succeeded or not. These methods are affected:
41
+
42
+ set
43
+ add
44
+ replace
@@ -19,6 +19,7 @@ Gem::Specification.new do |s|
19
19
  "Gemfile",
20
20
  "dalli.gemspec",
21
21
  "Performance.md",
22
+ "Upgrade.md",
22
23
  ]
23
24
  s.homepage = %q{http://github.com/mperham/dalli}
24
25
  s.rdoc_options = ["--charset=UTF-8"]
@@ -27,7 +28,7 @@ Gem::Specification.new do |s|
27
28
  s.test_files = Dir.glob("test/**/*")
28
29
  s.add_development_dependency(%q<shoulda>, [">= 0"])
29
30
  s.add_development_dependency(%q<mocha>, [">= 0"])
30
- s.add_development_dependency(%q<rails>, [">= 3.0.0.rc2"])
31
+ s.add_development_dependency(%q<rails>, [">= 3.0.0"])
31
32
  s.add_development_dependency(%q<memcache-client>, [">= 1.8.5"])
32
33
  end
33
34
 
@@ -0,0 +1,76 @@
1
+ require 'active_support/cache'
2
+ require 'action_dispatch/middleware/session/abstract_store'
3
+ require 'dalli'
4
+
5
+ # Dalli-based session store for Rails 3.0. Use like so:
6
+ #
7
+ # require 'action_dispatch/middleware/session/dalli_store'
8
+ # config.session_store ActionDispatch::Session::DalliStore, ['cache-1', 'cache-2'], :expire_after => 2.weeks
9
+ module ActionDispatch
10
+ module Session
11
+ class DalliStore < AbstractStore
12
+ def initialize(app, options = {})
13
+ # Support old :expires option
14
+ options[:expire_after] ||= options[:expires]
15
+
16
+ super
17
+
18
+ @default_options = {
19
+ :namespace => 'rack:session',
20
+ :memcache_server => 'localhost:11211',
21
+ :expires_in => options[:expire_after]
22
+ }.merge(@default_options)
23
+
24
+ @pool = options[:cache] || begin
25
+ Dalli::Client.new(
26
+ @default_options[:memcache_server], @default_options)
27
+ end
28
+ @namespace = @default_options[:namespace]
29
+
30
+ super
31
+ end
32
+
33
+ def reset
34
+ @pool.reset
35
+ end
36
+
37
+ private
38
+
39
+ def session_key(sid)
40
+ # Dalli does not support namespaces directly so we have
41
+ # to roll our own.
42
+ @namespace ? "#{@namespace}:#{sid}" : sid
43
+ end
44
+
45
+ def get_session(env, sid)
46
+ begin
47
+ session = @pool.get(session_key(sid)) || {}
48
+ rescue Dalli::DalliError => de
49
+ Rails.logger.warn("Session::DalliStore: #{$!.message}")
50
+ session = {}
51
+ end
52
+ [sid, session]
53
+ end
54
+
55
+ def set_session(env, sid, session_data)
56
+ options = env['rack.session.options']
57
+ expiry = options[:expire_after] || 0
58
+ @pool.set(session_key(sid), session_data, expiry)
59
+ sid
60
+ rescue Dalli::DalliError
61
+ Rails.logger.warn("Session::DalliStore: #{$!.message}")
62
+ false
63
+ end
64
+
65
+ def destroy(env)
66
+ if sid = current_session_id(env)
67
+ @pool.delete(session_key(sid))
68
+ end
69
+ rescue Dalli::DalliError
70
+ Rails.logger.warn("Session::DalliStore: #{$!.message}")
71
+ false
72
+ end
73
+
74
+ end
75
+ end
76
+ end
@@ -111,6 +111,10 @@ module ActiveSupport
111
111
  @data.stats
112
112
  end
113
113
 
114
+ def reset
115
+ @pool.reset
116
+ end
117
+
114
118
  protected
115
119
  # Read an entry from the cache.
116
120
  def read_entry(key, options) # :nodoc:
@@ -0,0 +1,172 @@
1
+ begin
2
+ require 'dalli'
3
+ rescue LoadError => e
4
+ $stderr.puts "You don't have dalli installed in your application: #{e.message}"
5
+ raise e
6
+ end
7
+ require 'digest/md5'
8
+
9
+ module ActiveSupport
10
+ module Cache
11
+ # A cache store implementation which stores data in Memcached:
12
+ # http://www.danga.com/memcached/
13
+ #
14
+ # DalliStore implements the Strategy::LocalCache strategy which implements
15
+ # an in memory cache inside of a block.
16
+ class DalliStore < Store
17
+ module Response # :nodoc:
18
+ STORED = "STORED\r\n"
19
+ NOT_STORED = "NOT_STORED\r\n"
20
+ EXISTS = "EXISTS\r\n"
21
+ NOT_FOUND = "NOT_FOUND\r\n"
22
+ DELETED = "DELETED\r\n"
23
+ end
24
+
25
+ ESCAPE_KEY_CHARS = /[\x00-\x20%\x7F-\xFF]/
26
+
27
+ def self.build_mem_cache(*addresses)
28
+ addresses = addresses.flatten
29
+ options = addresses.extract_options!
30
+ addresses = ["localhost"] if addresses.empty?
31
+ Dalli::Client.new(addresses, options)
32
+ end
33
+
34
+ # Creates a new DalliStore object, with the given memcached server
35
+ # addresses. Each address is either a host name, or a host-with-port string
36
+ # in the form of "host_name:port". For example:
37
+ #
38
+ # ActiveSupport::Cache::DalliStore.new("localhost", "server-downstairs.localnetwork:8229")
39
+ #
40
+ # If no addresses are specified, then DalliStore will connect to
41
+ # localhost port 11211 (the default memcached port).
42
+ #
43
+ def initialize(*addresses)
44
+ if addresses.first.respond_to?(:get)
45
+ @data = addresses.first
46
+ else
47
+ @data = self.class.build_mem_cache(*addresses)
48
+ end
49
+
50
+ extend Strategy::LocalCache
51
+ end
52
+
53
+ # Reads multiple keys from the cache using a single call to the
54
+ # servers for all keys. Options can be passed in the last argument.
55
+ def read_multi(*names)
56
+ keys_to_names = names.inject({}){|map, name| map[escape_key(name)] = name; map}
57
+ cache_keys = {}
58
+ # map keys to servers
59
+ names.each do |key|
60
+ cache_key = escape_key key
61
+ cache_keys[cache_key] = key
62
+ end
63
+
64
+ values = @data.get_multi keys_to_names.keys
65
+ results = {}
66
+ values.each do |key, value|
67
+ results[cache_keys[key]] = Marshal.load value
68
+ end
69
+ results
70
+ end
71
+
72
+ # Increment a cached value. This method uses the memcached incr atomic
73
+ # operator and can only be used on values written with the :raw option.
74
+ # Calling it on a value not stored with :raw will initialize that value
75
+ # to zero.
76
+ def increment(key, amount = 1) # :nodoc:
77
+ log("incrementing", key, amount)
78
+
79
+ response = @data.incr(escape_key(key), amount)
80
+ response == Response::NOT_FOUND ? nil : response
81
+ rescue Dalli::DalliError
82
+ nil
83
+ end
84
+ # Decrement a cached value. This method uses the memcached decr atomic
85
+ # operator and can only be used on values written with the :raw option.
86
+ # Calling it on a value not stored with :raw will initialize that value
87
+ # to zero.
88
+ def decrement(key, amount = 1) # :nodoc:
89
+ log("decrement", key, amount)
90
+ response = @data.decr(escape_key(key), amount)
91
+ response == Response::NOT_FOUND ? nil : response
92
+ rescue Dalli::DalliError
93
+ nil
94
+ end
95
+
96
+ def reset
97
+ @pool.reset
98
+ end
99
+
100
+ # Clear the entire cache on all memcached servers. This method should
101
+ # be used with care when using a shared cache.
102
+ def clear
103
+ @data.flush_all
104
+ end
105
+
106
+ # Get the statistics from the memcached servers.
107
+ def stats
108
+ @data.stats
109
+ end
110
+
111
+ # Read an entry from the cache.
112
+ def read(key, options = nil) # :nodoc:
113
+ super
114
+ value = @data.get(escape_key(key))
115
+ return nil if value.nil?
116
+ value = Marshal.load value
117
+ value
118
+ rescue Dalli::DalliError => e
119
+ logger.error("DalliError (#{e}): #{e.message}")
120
+ nil
121
+ end
122
+
123
+ # Writes a value to the cache.
124
+ #
125
+ # Possible options:
126
+ # - +:unless_exist+ - set to true if you don't want to update the cache
127
+ # if the key is already set.
128
+ # - +:expires_in+ - the number of seconds that this value may stay in
129
+ # the cache. See ActiveSupport::Cache::Store#write for an example.
130
+ def write(key, value, options = nil)
131
+ super
132
+ method = options && options[:unless_exist] ? :add : :set
133
+ # memcache-client will break the connection if you send it an integer
134
+ # in raw mode, so we convert it to a string to be sure it continues working.
135
+ value = Marshal.dump value
136
+ @data.send(method, escape_key(key), value, expires_in(options))
137
+ rescue Dalli::DalliError => e
138
+ logger.error("DalliError (#{e}): #{e.message}")
139
+ false
140
+ end
141
+
142
+ def delete(key, options = nil) # :nodoc:
143
+ super
144
+ @data.delete(escape_key(key))
145
+ rescue Dalli::DalliError => e
146
+ logger.error("DalliError (#{e}): #{e.message}")
147
+ false
148
+ end
149
+
150
+ def exist?(key, options = nil) # :nodoc:
151
+ # Doesn't call super, cause exist? in memcache is in fact a read
152
+ # But who cares? Reading is very fast anyway
153
+ # Local cache is checked first, if it doesn't know then memcache itself is read from
154
+ !read(key, options).nil?
155
+ end
156
+
157
+ def delete_matched(matcher, options = nil) # :nodoc:
158
+ # don't do any local caching at present, just pass
159
+ # through and let the error happen
160
+ super
161
+ raise "Not supported by Memcache"
162
+ end
163
+
164
+ private
165
+ def escape_key(key)
166
+ key = key.to_s.gsub(ESCAPE_KEY_CHARS){|match| "%#{match.getbyte(0).to_s(16).upcase}"}
167
+ key = "#{key[0, 213]}:md5:#{Digest::MD5.hexdigest(key)}" if key.size > 250
168
+ key
169
+ end
170
+ end
171
+ end
172
+ end
@@ -20,11 +20,8 @@ module Dalli
20
20
  # Default: true.
21
21
  #
22
22
  def initialize(servers=nil, options={})
23
- @ring = Dalli::Ring.new(
24
- Array(env_servers || servers).map do |s|
25
- Dalli::Server.new(s)
26
- end, options
27
- )
23
+ @servers = servers
24
+ @options = options
28
25
  self.extend(Dalli::Marshal) unless options[:marshal] == false
29
26
  end
30
27
 
@@ -38,11 +35,11 @@ module Dalli
38
35
  end
39
36
 
40
37
  def get_multi(*keys)
41
- @ring.lock do
38
+ ring.lock do
42
39
  keys.flatten.each do |key|
43
40
  perform(:getkq, key)
44
41
  end
45
- values = @ring.servers.inject({}) { |hash, s| hash.merge!(s.request(:noop)); hash }
42
+ values = ring.servers.inject({}) { |hash, s| hash.merge!(s.request(:noop)); hash }
46
43
  values.inject(values) { |memo, (k,v)| memo[k] = deserialize(v); memo }
47
44
  end
48
45
  end
@@ -91,11 +88,12 @@ module Dalli
91
88
 
92
89
  def flush(delay=0)
93
90
  time = -delay
94
- @ring.servers.map { |s| s.request(:flush, time += delay) }
91
+ ring.servers.map { |s| s.request(:flush, time += delay) }
95
92
  end
96
93
 
97
- def flush_all
98
- flush(0)
94
+ # deprecated, please use #flush.
95
+ def flush_all(delay=0)
96
+ flush(delay)
99
97
  end
100
98
 
101
99
  ##
@@ -129,16 +127,27 @@ module Dalli
129
127
  end
130
128
 
131
129
  def stats
132
- @ring.servers.inject({}) { |memo, s| memo["#{s.hostname}:#{s.port}"] = s.request(:stats); memo }
130
+ ring.servers.inject({}) { |memo, s| memo["#{s.hostname}:#{s.port}"] = s.request(:stats); memo }
133
131
  end
134
132
 
135
133
  def close
136
- @ring.servers.map { |s| s.close }
137
- @ring = nil
134
+ if @ring
135
+ @ring.servers.map { |s| s.close }
136
+ @ring = nil
137
+ end
138
138
  end
139
+ alias_method :reset, :close
139
140
 
140
141
  private
141
142
 
143
+ def ring
144
+ @ring ||= Dalli::Ring.new(
145
+ Array(env_servers || @servers).map do |s|
146
+ Dalli::Server.new(s)
147
+ end, @options
148
+ )
149
+ end
150
+
142
151
  def serialize(value)
143
152
  value.to_s
144
153
  end
@@ -155,7 +164,7 @@ module Dalli
155
164
  def perform(op, *args)
156
165
  key = args.first
157
166
  validate_key(key)
158
- server = @ring.server_for_key(key)
167
+ server = ring.server_for_key(key)
159
168
  server.request(op, *args)
160
169
  end
161
170