memcache-client 1.7.4 → 1.7.5

Sign up to get free protection for your applications and to get access to all the features.
data/FAQ.rdoc CHANGED
@@ -23,9 +23,9 @@ You can increase the timeout or disable them completely with the following confi
23
23
  MemCache.new ['server1', 'server2'], { :timeout => 1.0 } # 1 second timeout
24
24
 
25
25
 
26
- == The new memcache-client is slower than 1.5.0!?
26
+ == Isn't Evan Weaver's memcached gem faster?
27
27
 
28
- Yes, in the simplest case, 1.6.x+ is slower than 1.5.0. If you just have a single memcached
29
- server, the latest memcache-client will be slower. This is because 1.5.0 does not
30
- timeout network operations. For reliability purposes, memcache-client 1.6.x+ enables
31
- timeouts by default. Most of this performance penalty is now gone in 1.7.3 and later.
28
+ The latest version of memcached-client is anywhere from 33% to 100% slower than memcached in various benchmarks. Keep in mind this means that 10,000 get requests take 1.8 sec instead of 1.2 seconds.
29
+ In practice, memcache-client is unlikely to be a bottleneck in your system but there is always going
30
+ to be an overhead to pure Ruby. Evan's memcached gem is a thin wrapper around a C library, which
31
+ makes it very fast but also difficult to install for some people.
@@ -1,3 +1,12 @@
1
+ = 1.7.5 (2009-09-09)
2
+
3
+ * Fix ruby warnings (josh)
4
+ * Make megabyte value size limit optional since Tokyo Tyrant can accept values larger than 1MB.
5
+ Use :check_size => false to disable the size check. (jsl)
6
+ * Ruby 1.9 support for recent I/O changes.
7
+ * Fix duplicate value marshalling on server error. (rajiv)
8
+ * Added option :autofix_keys (disabled by default) to replace long keys with md5 hashes (sd)
9
+
1
10
  = 1.7.4 (2009-06-09)
2
11
 
3
12
  * Fix issue with raising timeout errors.
@@ -32,8 +32,9 @@ multiple memcached servers, you should install the RubyInline gem for ultimate p
32
32
 
33
33
  == Using memcache-client with Rails
34
34
 
35
- Rails 2.1+ includes memcache-client out of the box. See ActiveSupport::Cache::MemCacheStore
36
- and the Rails.cache method for more details.
35
+ Rails 2.1+ includes memcache-client 1.5.0 out of the box. See ActiveSupport::Cache::MemCacheStore
36
+ and the Rails.cache method for more details. Rails 2.3+ will use the latest memcache-client
37
+ gem installed.
37
38
 
38
39
 
39
40
  == Questions?
data/Rakefile CHANGED
@@ -26,7 +26,9 @@ Rake::RDocTask.new do |rd|
26
26
  rd.rdoc_dir = 'doc'
27
27
  end
28
28
 
29
- Rake::TestTask.new
29
+ Rake::TestTask.new do |t|
30
+ t.warning = true
31
+ end
30
32
 
31
33
  task :default => :test
32
34
 
@@ -15,19 +15,21 @@ class MemCache
15
15
  ##
16
16
  # The version of MemCache you are using.
17
17
 
18
- VERSION = '1.7.4'
18
+ VERSION = '1.7.5'
19
19
 
20
20
  ##
21
21
  # Default options for the cache object.
22
22
 
23
23
  DEFAULT_OPTIONS = {
24
- :namespace => nil,
25
- :readonly => false,
26
- :multithread => true,
27
- :failover => true,
28
- :timeout => 0.5,
29
- :logger => nil,
30
- :no_reply => false,
24
+ :namespace => nil,
25
+ :readonly => false,
26
+ :multithread => true,
27
+ :failover => true,
28
+ :timeout => 0.5,
29
+ :logger => nil,
30
+ :no_reply => false,
31
+ :check_size => true,
32
+ :autofix_keys => false
31
33
  }
32
34
 
33
35
  ##
@@ -50,6 +52,19 @@ class MemCache
50
52
 
51
53
  attr_reader :multithread
52
54
 
55
+ ##
56
+ # Whether to try to fix keys that are too long and will be truncated by
57
+ # using their SHA1 hash instead.
58
+ # The hash is only used on keys longer than 250 characters, or containing spaces,
59
+ # to avoid impacting performance unnecesarily.
60
+ #
61
+ # In theory, your code should generate correct keys when calling memcache,
62
+ # so it's your responsibility and you should try to fix this problem at its source.
63
+ #
64
+ # But if that's not possible, enable this option and memcache-client will give you a hand.
65
+
66
+ attr_reader :autofix_keys
67
+
53
68
  ##
54
69
  # The servers this client talks to. Play at your own peril.
55
70
 
@@ -84,19 +99,21 @@ class MemCache
84
99
  #
85
100
  # Valid options for +opts+ are:
86
101
  #
87
- # [:namespace] Prepends this value to all keys added or retrieved.
88
- # [:readonly] Raises an exception on cache writes when true.
89
- # [:multithread] Wraps cache access in a Mutex for thread safety. Defaults to true.
90
- # [:failover] Should the client try to failover to another server if the
91
- # first server is down? Defaults to true.
92
- # [:timeout] Time to use as the socket read timeout. Defaults to 0.5 sec,
93
- # set to nil to disable timeouts (this is a major performance penalty in Ruby 1.8,
94
- # "gem install SystemTimer' to remove most of the penalty).
95
- # [:logger] Logger to use for info/debug output, defaults to nil
96
- # [:no_reply] Don't bother looking for a reply for write operations (i.e. they
97
- # become 'fire and forget'), memcached 1.2.5 and later only, speeds up
98
- # set/add/delete/incr/decr significantly.
99
- #
102
+ # [:namespace] Prepends this value to all keys added or retrieved.
103
+ # [:readonly] Raises an exception on cache writes when true.
104
+ # [:multithread] Wraps cache access in a Mutex for thread safety. Defaults to true.
105
+ # [:failover] Should the client try to failover to another server if the
106
+ # first server is down? Defaults to true.
107
+ # [:timeout] Time to use as the socket read timeout. Defaults to 0.5 sec,
108
+ # set to nil to disable timeouts.
109
+ # [:logger] Logger to use for info/debug output, defaults to nil
110
+ # [:no_reply] Don't bother looking for a reply for write operations (i.e. they
111
+ # become 'fire and forget'), memcached 1.2.5 and later only, speeds up
112
+ # set/add/delete/incr/decr significantly.
113
+ # [:check_size] Raises a MemCacheError if the value to be set is greater than 1 MB, which
114
+ # is the maximum key size for the standard memcached server. Defaults to true.
115
+ # [:autofix_keys] If a key is longer than 250 characters or contains spaces,
116
+ # use an SHA1 hash instead, to prevent collisions on truncated keys.
100
117
  # Other options are ignored.
101
118
 
102
119
  def initialize(*args)
@@ -120,14 +137,16 @@ class MemCache
120
137
  end
121
138
 
122
139
  opts = DEFAULT_OPTIONS.merge opts
123
- @namespace = opts[:namespace]
124
- @readonly = opts[:readonly]
125
- @multithread = opts[:multithread]
126
- @timeout = opts[:timeout]
127
- @failover = opts[:failover]
128
- @logger = opts[:logger]
129
- @no_reply = opts[:no_reply]
130
- @mutex = Mutex.new if @multithread
140
+ @namespace = opts[:namespace]
141
+ @readonly = opts[:readonly]
142
+ @multithread = opts[:multithread]
143
+ @autofix_keys = opts[:autofix_keys]
144
+ @timeout = opts[:timeout]
145
+ @failover = opts[:failover]
146
+ @logger = opts[:logger]
147
+ @no_reply = opts[:no_reply]
148
+ @check_size = opts[:check_size]
149
+ @mutex = Mutex.new if @multithread
131
150
 
132
151
  logger.info { "memcache-client #{VERSION} #{Array(servers).inspect}" } if logger
133
152
 
@@ -312,12 +331,14 @@ class MemCache
312
331
 
313
332
  def set(key, value, expiry = 0, raw = false)
314
333
  raise MemCacheError, "Update of readonly cache" if @readonly
315
- with_server(key) do |server, cache_key|
316
334
 
317
- value = Marshal.dump value unless raw
335
+ value = Marshal.dump value unless raw
336
+ with_server(key) do |server, cache_key|
318
337
  logger.debug { "set #{key} to #{server.inspect}: #{value.to_s.size}" } if logger
319
338
 
320
- raise MemCacheError, "Value too large, memcached can only store 1MB of data per key" if value.to_s.size > ONE_MB
339
+ if @check_size && value.to_s.size > ONE_MB
340
+ raise MemCacheError, "Value too large, memcached can only store 1MB of data per key"
341
+ end
321
342
 
322
343
  command = "set #{cache_key} 0 #{expiry} #{value.to_s.size}#{noreply}\r\n#{value}\r\n"
323
344
 
@@ -359,10 +380,9 @@ class MemCache
359
380
  (value, token) = gets(key, raw)
360
381
  return nil unless value
361
382
  updated = yield value
383
+ value = Marshal.dump updated unless raw
362
384
 
363
385
  with_server(key) do |server, cache_key|
364
-
365
- value = Marshal.dump updated unless raw
366
386
  logger.debug { "cas #{key} to #{server.inspect}: #{value.to_s.size}" } if logger
367
387
  command = "cas #{cache_key} 0 #{expiry} #{value.to_s.size} #{token}#{noreply}\r\n#{value}\r\n"
368
388
 
@@ -392,8 +412,8 @@ class MemCache
392
412
 
393
413
  def add(key, value, expiry = 0, raw = false)
394
414
  raise MemCacheError, "Update of readonly cache" if @readonly
415
+ value = Marshal.dump value unless raw
395
416
  with_server(key) do |server, cache_key|
396
- value = Marshal.dump value unless raw
397
417
  logger.debug { "add #{key} to #{server}: #{value ? value.to_s.size : 'nil'}" } if logger
398
418
  command = "add #{cache_key} 0 #{expiry} #{value.to_s.size}#{noreply}\r\n#{value}\r\n"
399
419
 
@@ -413,8 +433,8 @@ class MemCache
413
433
  # If +raw+ is true, +value+ will not be Marshalled.
414
434
  def replace(key, value, expiry = 0, raw = false)
415
435
  raise MemCacheError, "Update of readonly cache" if @readonly
436
+ value = Marshal.dump value unless raw
416
437
  with_server(key) do |server, cache_key|
417
- value = Marshal.dump value unless raw
418
438
  logger.debug { "replace #{key} to #{server}: #{value ? value.to_s.size : 'nil'}" } if logger
419
439
  command = "replace #{cache_key} 0 #{expiry} #{value.to_s.size}#{noreply}\r\n#{value}\r\n"
420
440
 
@@ -621,6 +641,10 @@ class MemCache
621
641
  # requested.
622
642
 
623
643
  def make_cache_key(key)
644
+ if @autofix_keys and (key =~ /\s/ or (key.length + (namespace.nil? ? 0 : namespace.length)) > 250)
645
+ key = "#{Digest::SHA1.hexdigest(key)}-autofixed"
646
+ end
647
+
624
648
  if namespace.nil? then
625
649
  key
626
650
  else
@@ -1040,23 +1064,23 @@ class MemCache
1040
1064
  class BufferedIO < Net::BufferedIO # :nodoc:
1041
1065
  BUFSIZE = 1024 * 16
1042
1066
 
1043
- # An implementation similar to this is in *trunk* for 1.9. When it
1044
- # gets released, this method can be removed when using 1.9
1045
- def rbuf_fill
1046
- begin
1047
- @rbuf << @io.read_nonblock(BUFSIZE)
1048
- rescue Errno::EWOULDBLOCK
1049
- retry unless @read_timeout
1050
- if IO.select([@io], nil, nil, @read_timeout)
1051
- retry
1052
- else
1053
- raise Timeout::Error, 'IO timeout'
1067
+ if RUBY_VERSION < '1.9.1'
1068
+ def rbuf_fill
1069
+ begin
1070
+ @rbuf << @io.read_nonblock(BUFSIZE)
1071
+ rescue Errno::EWOULDBLOCK
1072
+ retry unless @read_timeout
1073
+ if IO.select([@io], nil, nil, @read_timeout)
1074
+ retry
1075
+ else
1076
+ raise Timeout::Error, 'IO timeout'
1077
+ end
1054
1078
  end
1055
1079
  end
1056
1080
  end
1057
1081
 
1058
- def setsockopt *args
1059
- @io.setsockopt *args
1082
+ def setsockopt(*args)
1083
+ @io.setsockopt(*args)
1060
1084
  end
1061
1085
 
1062
1086
  def gets
@@ -18,7 +18,8 @@ require File.dirname(__FILE__) + '/../lib/memcache' if not defined?(MemCache)
18
18
  class MemCache
19
19
 
20
20
  attr_writer :namespace
21
-
21
+ attr_writer :autofix_keys
22
+
22
23
  end
23
24
 
24
25
  class FakeSocket
@@ -269,7 +270,7 @@ class TestMemCache < Test::Unit::TestCase
269
270
  @cache.cache_get(server, 'my_namespace:key')
270
271
  end
271
272
 
272
- assert_match /#{Regexp.quote 'No connection to server (NOT CONNECTED)'}/, e.message
273
+ assert_match(/#{Regexp.quote 'No connection to server (NOT CONNECTED)'}/, e.message)
273
274
 
274
275
  assert !server.alive?
275
276
  end
@@ -327,7 +328,7 @@ class TestMemCache < Test::Unit::TestCase
327
328
  @cache.cache_get_multi server, 'my_namespace:key'
328
329
  end
329
330
 
330
- assert_match /#{Regexp.quote 'No connection to server (NOT CONNECTED)'}/, e.message
331
+ assert_match(/#{Regexp.quote 'No connection to server (NOT CONNECTED)'}/, e.message)
331
332
 
332
333
  assert !server.alive?
333
334
  end
@@ -563,7 +564,7 @@ class TestMemCache < Test::Unit::TestCase
563
564
  @cache.get 'key'
564
565
  end
565
566
 
566
- assert_match /^No connection to server/, e.message
567
+ assert_match(/^No connection to server/, e.message)
567
568
  end
568
569
 
569
570
  def test_get_no_servers
@@ -717,6 +718,36 @@ class TestMemCache < Test::Unit::TestCase
717
718
  assert_equal 'key', @cache.make_cache_key('key')
718
719
  end
719
720
 
721
+ def test_make_cache_key_without_autofix
722
+ @cache.autofix_keys = false
723
+
724
+ key = "keys with more than two hundred and fifty characters can cause problems, because they get truncated and start colliding with each other. It's not a common occurrence, but when it happens is very hard to debug. the autofix option takes care of that for you"
725
+ hash = Digest::SHA1.hexdigest(key)
726
+ @cache.namespace = nil
727
+ assert_equal key, @cache.make_cache_key(key)
728
+
729
+ @cache.autofix_keys = true
730
+
731
+ @cache.namespace = "my_namespace"
732
+ assert_equal 'my_namespace:key', @cache.make_cache_key('key')
733
+ @cache.namespace = nil
734
+ assert_equal 'key', @cache.make_cache_key('key')
735
+
736
+ key = "keys with more than two hundred and fifty characters can cause problems, because they get truncated and start colliding with each other. It's not a common occurrence, but when it happens is very hard to debug. the autofix option takes care of that for you"
737
+ hash = Digest::SHA1.hexdigest(key)
738
+ @cache.namespace = "my_namespace"
739
+ assert_equal "my_namespace:#{hash}-autofixed", @cache.make_cache_key(key)
740
+ @cache.namespace = nil
741
+ assert_equal "#{hash}-autofixed", @cache.make_cache_key(key)
742
+
743
+ key = "a short key with spaces"
744
+ hash = Digest::SHA1.hexdigest(key)
745
+ @cache.namespace = "my_namespace"
746
+ assert_equal "my_namespace:#{hash}-autofixed", @cache.make_cache_key(key)
747
+ @cache.namespace = nil
748
+ assert_equal "#{hash}-autofixed", @cache.make_cache_key(key)
749
+ end
750
+
720
751
  def test_servers
721
752
  server = FakeServer.new
722
753
  @cache.servers = []
@@ -776,6 +807,38 @@ class TestMemCache < Test::Unit::TestCase
776
807
  assert_equal 'Update of readonly cache', e.message
777
808
  end
778
809
 
810
+ def test_check_size_on
811
+ cache = MemCache.new :check_size => true
812
+
813
+ server = FakeServer.new
814
+ server.socket.data.write "STORED\r\n"
815
+ server.socket.data.rewind
816
+
817
+ cache.servers = []
818
+ cache.servers << server
819
+
820
+ e = assert_raise MemCache::MemCacheError do
821
+ cache.set 'key', 'v' * 1048577
822
+ end
823
+
824
+ assert_equal 'Value too large, memcached can only store 1MB of data per key', e.message
825
+ end
826
+
827
+ def test_check_size_off
828
+ cache = MemCache.new :check_size => false
829
+
830
+ server = FakeServer.new
831
+ server.socket.data.write "STORED\r\n"
832
+ server.socket.data.rewind
833
+
834
+ cache.servers = []
835
+ cache.servers << server
836
+
837
+ assert_nothing_raised do
838
+ cache.set 'key', 'v' * 1048577
839
+ end
840
+ end
841
+
779
842
  def test_set_too_big
780
843
  server = FakeServer.new
781
844
 
@@ -790,7 +853,7 @@ class TestMemCache < Test::Unit::TestCase
790
853
  @cache.set 'key', 'v'
791
854
  end
792
855
 
793
- assert_match /object too large for cache/, e.message
856
+ assert_match(/object too large for cache/, e.message)
794
857
  end
795
858
 
796
859
  def test_prepend
@@ -977,7 +1040,7 @@ class TestMemCache < Test::Unit::TestCase
977
1040
  @cache.flush_all
978
1041
  end
979
1042
 
980
- assert_match /flush_all\r\n/, socket.written.string
1043
+ assert_match(/flush_all\r\n/, socket.written.string)
981
1044
  end
982
1045
 
983
1046
  def test_flush_all_for_real
@@ -1033,8 +1096,8 @@ class TestMemCache < Test::Unit::TestCase
1033
1096
  end
1034
1097
 
1035
1098
  output = server.socket.written.string
1036
- assert_match /set my_namespace:test/, output
1037
- assert_match /test value/, output
1099
+ assert_match(/set my_namespace:test/, output)
1100
+ assert_match(/test value/, output)
1038
1101
  end
1039
1102
 
1040
1103
  def test_basic_unthreaded_operations_should_work
@@ -1056,8 +1119,8 @@ class TestMemCache < Test::Unit::TestCase
1056
1119
  end
1057
1120
 
1058
1121
  output = server.socket.written.string
1059
- assert_match /set my_namespace:test/, output
1060
- assert_match /test value/, output
1122
+ assert_match(/set my_namespace:test/, output)
1123
+ assert_match(/test value/, output)
1061
1124
  end
1062
1125
 
1063
1126
  def util_setup_fake_server
@@ -1122,9 +1185,9 @@ class TestMemCache < Test::Unit::TestCase
1122
1185
  assert cache.decr('c', 5) > 14
1123
1186
  assert_equal 11, cache.get('b')
1124
1187
  d = cache.get('d', true)
1125
- assert_match /\Aab*\Z/, d
1188
+ assert_match(/\Aab*\Z/, d)
1126
1189
  e = cache.get('e', true)
1127
- assert_match /\Ay*x\Z/, e
1190
+ assert_match(/\Ay*x\Z/, e)
1128
1191
  end
1129
1192
  end
1130
1193
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: memcache-client
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.7.4
4
+ version: 1.7.5
5
5
  platform: ruby
6
6
  authors:
7
7
  - Eric Hodel
@@ -11,7 +11,7 @@ autorequire:
11
11
  bindir: bin
12
12
  cert_chain: []
13
13
 
14
- date: 2009-06-09 00:00:00 -05:00
14
+ date: 2009-09-09 00:00:00 -05:00
15
15
  default_executable:
16
16
  dependencies: []
17
17
 
@@ -31,7 +31,7 @@ files:
31
31
  - Rakefile
32
32
  - lib/memcache.rb
33
33
  - lib/continuum_native.rb
34
- has_rdoc: false
34
+ has_rdoc: true
35
35
  homepage: http://github.com/mperham/memcache-client
36
36
  licenses: []
37
37
 
@@ -55,7 +55,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
55
55
  requirements: []
56
56
 
57
57
  rubyforge_project: seattlerb
58
- rubygems_version: 1.3.2
58
+ rubygems_version: 1.3.5
59
59
  signing_key:
60
60
  specification_version: 3
61
61
  summary: A Ruby library for accessing memcached.