mperham-memcache-client 1.6.4 → 1.6.5

Sign up to get free protection for your applications and to get access to all the features.
data/History.txt CHANGED
@@ -1,3 +1,15 @@
1
+ = 1.6.5 (2009-02-27)
2
+
3
+ * Change memcache-client to multithreaded by default. The mutex does not add significant
4
+ overhead and it is far too easy, now that Sinatra, Rails and Merb are all thread-safe, to
5
+ use memcache-client in a thread-unsafe manner. Remove some unnecessary mutexing and add
6
+ a test to verify heavily multithreaded usage does not act unexpectedly.
7
+
8
+ * Add optional support for the SystemTimer gem when running on Ruby 1.8.x. This gem is
9
+ highly recommended - it ensures timeouts actually work and halves the overhead of using
10
+ timeouts. Using this gem, Ruby 1.8.x is actually faster in my performance tests
11
+ than Ruby 1.9.x. Just "gem install SystemTimer" and it should be picked up automatically.
12
+
1
13
  = 1.6.4 (2009-02-19)
2
14
 
3
15
  * Remove native code altogether. The speedup was only 10% on Ruby 1.8.6 and did not work
data/README.rdoc CHANGED
@@ -16,22 +16,24 @@ Just install the gem:
16
16
 
17
17
  With one server:
18
18
 
19
- CACHE = MemCache.new 'localhost:11211', :namespace => 'my_namespace'
19
+ CACHE = MemCache.new 'localhost:11211'
20
20
 
21
21
  Or with multiple servers:
22
22
 
23
- CACHE = MemCache.new %w[one.example.com:11211 two.example.com:11211],
24
- :namespace => 'my_namespace'
23
+ CACHE = MemCache.new %w[one.example.com:11211 two.example.com:11211]
24
+
25
+
26
+ == Tuning memcache-client
27
+
28
+ The MemCache.new method takes a number of options which can be useful at times. Please
29
+ read the source comments there for an overview.
25
30
 
26
- See MemCache.new for details. Please note memcache-client is not thread-safe
27
- by default. You should create a separate instance for each thread in your
28
- process.
29
31
 
30
32
  == Using memcache-client with Rails
31
33
 
32
- There's no need to use memcache-client in a Rails application. Rails 2.1+ includes
33
- a basic caching library which can be used with memcached. See ActiveSupport::Cache::Store
34
- for more details.
34
+ Rails 2.1+ includes memcache-client out of the box. See ActiveSupport::Cache::MemCacheStore
35
+ and the Rails.cache method for more details.
36
+
35
37
 
36
38
  == Questions?
37
39
 
data/lib/memcache.rb CHANGED
@@ -2,10 +2,28 @@ $TESTING = defined?($TESTING) && $TESTING
2
2
 
3
3
  require 'socket'
4
4
  require 'thread'
5
- require 'timeout'
6
5
  require 'zlib'
7
6
  require 'digest/sha1'
8
7
 
8
+ begin
9
+ # Try to use the SystemTimer gem instead of Ruby's timeout library
10
+ # when running on something that looks like Ruby 1.8.x. See:
11
+ # http://ph7spot.com/articles/system_timer
12
+ # We don't want to bother trying to load SystemTimer on jruby and
13
+ # ruby 1.9+.
14
+ if !defined?(RUBY_ENGINE)
15
+ require 'system_timer'
16
+ MemCacheTimer = SystemTimer
17
+ else
18
+ require 'timeout'
19
+ MemCacheTimer = Timeout
20
+ end
21
+ rescue LoadError => e
22
+ puts "[memcache-client] Could not load SystemTimer gem, falling back to Ruby's slower/unsafe timeout library: #{e.message}"
23
+ require 'timeout'
24
+ MemCacheTimer = Timeout
25
+ end
26
+
9
27
  ##
10
28
  # A Ruby client library for memcached.
11
29
  #
@@ -15,7 +33,7 @@ class MemCache
15
33
  ##
16
34
  # The version of MemCache you are using.
17
35
 
18
- VERSION = '1.6.4'
36
+ VERSION = '1.6.5'
19
37
 
20
38
  ##
21
39
  # Default options for the cache object.
@@ -23,7 +41,7 @@ class MemCache
23
41
  DEFAULT_OPTIONS = {
24
42
  :namespace => nil,
25
43
  :readonly => false,
26
- :multithread => false,
44
+ :multithread => true,
27
45
  :failover => true,
28
46
  :timeout => 0.5,
29
47
  :logger => nil,
@@ -55,7 +73,7 @@ class MemCache
55
73
  attr_reader :servers
56
74
 
57
75
  ##
58
- # Socket timeout limit with this client, defaults to 0.25 sec.
76
+ # Socket timeout limit with this client, defaults to 0.5 sec.
59
77
  # Set to nil to disable timeouts.
60
78
 
61
79
  attr_reader :timeout
@@ -79,12 +97,14 @@ class MemCache
79
97
  #
80
98
  # [:namespace] Prepends this value to all keys added or retrieved.
81
99
  # [:readonly] Raises an exception on cache writes when true.
82
- # [:multithread] Wraps cache access in a Mutex for thread safety.
100
+ # [:multithread] Wraps cache access in a Mutex for thread safety. Defaults to true.
83
101
  # [:failover] Should the client try to failover to another server if the
84
102
  # first server is down? Defaults to true.
85
- # [:timeout] Time to use as the socket read timeout. Defaults to 0.25 sec,
86
- # set to nil to disable timeouts (this is a major performance penalty in Ruby 1.8).
103
+ # [:timeout] Time to use as the socket read timeout. Defaults to 0.5 sec,
104
+ # set to nil to disable timeouts (this is a major performance penalty in Ruby 1.8,
105
+ # "gem install SystemTimer' to remove most of the penalty).
87
106
  # [:logger] Logger to use for info/debug output, defaults to nil
107
+ #
88
108
  # Other options are ignored.
89
109
 
90
110
  def initialize(*args)
@@ -160,9 +180,6 @@ class MemCache
160
180
  weight ||= DEFAULT_WEIGHT
161
181
  Server.new self, host, port, weight
162
182
  else
163
- if server.multithread != @multithread then
164
- raise ArgumentError, "can't mix threaded and non-threaded servers"
165
- end
166
183
  server
167
184
  end
168
185
  end
@@ -220,6 +237,8 @@ class MemCache
220
237
  # cache["a"] = 1
221
238
  # cache["b"] = 2
222
239
  # cache.get_multi "a", "b" # => { "a" => 1, "b" => 2 }
240
+ #
241
+ # Note that get_multi assumes the values are marshalled.
223
242
 
224
243
  def get_multi(*keys)
225
244
  raise MemCacheError, 'No active servers' unless active?
@@ -353,7 +372,6 @@ class MemCache
353
372
  raise MemCacheError, "Update of readonly cache" if @readonly
354
373
 
355
374
  begin
356
- @mutex.lock if @multithread
357
375
  @servers.each do |server|
358
376
  with_socket_management(server) do |socket|
359
377
  socket.write "flush_all\r\n"
@@ -364,8 +382,6 @@ class MemCache
364
382
  end
365
383
  rescue IndexError => err
366
384
  handle_error nil, err
367
- ensure
368
- @mutex.unlock if @multithread
369
385
  end
370
386
  end
371
387
 
@@ -621,7 +637,7 @@ class MemCache
621
637
 
622
638
  block.call(socket)
623
639
 
624
- rescue SocketError => err
640
+ rescue SocketError, Timeout::Error => err
625
641
  logger.warn { "Socket failure: #{err.message}" } if logger
626
642
  server.mark_dead(err)
627
643
  handle_error(server, err)
@@ -754,7 +770,6 @@ class MemCache
754
770
 
755
771
  attr_reader :status
756
772
 
757
- attr_reader :multithread
758
773
  attr_reader :logger
759
774
 
760
775
  ##
@@ -769,9 +784,6 @@ class MemCache
769
784
  @port = port.to_i
770
785
  @weight = weight.to_i
771
786
 
772
- @multithread = memcache.multithread
773
- @mutex = Mutex.new
774
-
775
787
  @sock = nil
776
788
  @retry = nil
777
789
  @status = 'NOT CONNECTED'
@@ -801,7 +813,6 @@ class MemCache
801
813
  # Returns the connected socket object on success or nil on failure.
802
814
 
803
815
  def socket
804
- @mutex.lock if @multithread
805
816
  return @sock if @sock and not @sock.closed?
806
817
 
807
818
  @sock = nil
@@ -824,8 +835,6 @@ class MemCache
824
835
  end
825
836
 
826
837
  return @sock
827
- ensure
828
- @mutex.unlock if @multithread
829
838
  end
830
839
 
831
840
  ##
@@ -833,13 +842,10 @@ class MemCache
833
842
  # object. The server is not considered dead.
834
843
 
835
844
  def close
836
- @mutex.lock if @multithread
837
845
  @sock.close if @sock && !@sock.closed?
838
846
  @sock = nil
839
847
  @retry = nil
840
848
  @status = "NOT CONNECTED"
841
- ensure
842
- @mutex.unlock if @multithread
843
849
  end
844
850
 
845
851
  ##
@@ -868,26 +874,26 @@ end
868
874
  class TCPTimeoutSocket
869
875
 
870
876
  def initialize(host, port, timeout)
871
- Timeout::timeout(MemCache::Server::CONNECT_TIMEOUT, SocketError) do
877
+ MemCacheTimer.timeout(MemCache::Server::CONNECT_TIMEOUT) do
872
878
  @sock = TCPSocket.new(host, port)
873
879
  @len = timeout
874
880
  end
875
881
  end
876
882
 
877
883
  def write(*args)
878
- Timeout::timeout(@len, SocketError) do
884
+ MemCacheTimer.timeout(@len) do
879
885
  @sock.write(*args)
880
886
  end
881
887
  end
882
888
 
883
889
  def gets(*args)
884
- Timeout::timeout(@len, SocketError) do
890
+ MemCacheTimer.timeout(@len) do
885
891
  @sock.gets(*args)
886
892
  end
887
893
  end
888
894
 
889
895
  def read(*args)
890
- Timeout::timeout(@len, SocketError) do
896
+ MemCacheTimer.timeout(@len) do
891
897
  @sock.read(*args)
892
898
  end
893
899
  end
@@ -10,9 +10,10 @@ rescue LoadError => e
10
10
  puts "Some tests require flexmock, please run `gem install flexmock`"
11
11
  end
12
12
 
13
+ Thread.abort_on_exception = true
13
14
  $TESTING = true
14
15
 
15
- require File.dirname(__FILE__) + '/../lib/memcache'
16
+ require File.dirname(__FILE__) + '/../lib/memcache' if not defined?(MemCache)
16
17
 
17
18
  class MemCache
18
19
 
@@ -79,7 +80,7 @@ end
79
80
 
80
81
  class FakeServer
81
82
 
82
- attr_reader :host, :port, :socket, :weight, :multithread, :status
83
+ attr_accessor :host, :port, :socket, :weight, :multithread, :status
83
84
 
84
85
  def initialize(socket = nil)
85
86
  @closed = false
@@ -87,7 +88,7 @@ class FakeServer
87
88
  @port = 11211
88
89
  @socket = socket || FakeSocket.new
89
90
  @weight = 1
90
- @multithread = false
91
+ @multithread = true
91
92
  @status = "CONNECTED"
92
93
  end
93
94
 
@@ -117,9 +118,9 @@ class TestMemCache < Test::Unit::TestCase
117
118
 
118
119
  def test_performance
119
120
  requirement(memcached_running?, 'A real memcached server must be running for performance testing') do
120
- host = Socket.gethostname
121
121
 
122
- cache = MemCache.new(['localhost:11211',"#{host}:11211"])
122
+ cache = MemCache.new(['localhost:11211',"127.0.0.1:11211"])
123
+ cache.flush_all
123
124
  cache.add('a', 1, 120)
124
125
  with = xprofile 'get' do
125
126
  1000.times do
@@ -129,7 +130,7 @@ class TestMemCache < Test::Unit::TestCase
129
130
  puts ''
130
131
  puts "1000 gets with socket timeout: #{with} sec"
131
132
 
132
- cache = MemCache.new(['localhost:11211',"#{host}:11211"], :timeout => nil)
133
+ cache = MemCache.new(['localhost:11211',"127.0.0.1:11211"], :timeout => nil)
133
134
  cache.add('a', 1, 120)
134
135
  without = xprofile 'get' do
135
136
  1000.times do
@@ -335,8 +336,10 @@ class TestMemCache < Test::Unit::TestCase
335
336
 
336
337
  def test_multithread_error
337
338
  server = FakeServer.new
339
+ server.multithread = false
340
+
341
+ @cache = MemCache.new(['localhost:1'], :multithread => false)
338
342
 
339
- # Write two messages to the socket to test failover
340
343
  server.socket.data.write "bogus response\r\nbogus response\r\n"
341
344
  server.socket.data.rewind
342
345
 
@@ -972,5 +975,37 @@ class TestMemCache < Test::Unit::TestCase
972
975
  return server
973
976
  end
974
977
 
978
+ def test_crazy_multithreaded_access
979
+ requirement(memcached_running?, 'A real memcached server must be running for performance testing') do
980
+
981
+ cache = MemCache.new(['localhost:11211', '127.0.0.1:11211'])
982
+ cache.flush_all
983
+ workers = []
984
+
985
+ # Have a bunch of threads perform a bunch of operations at the same time.
986
+ # Verify the result of each operation to ensure the request and response
987
+ # are not intermingled between threads.
988
+ 10.times do
989
+ workers << Thread.new do
990
+ 100.times do
991
+ cache.set('a', 9)
992
+ cache.set('b', 11)
993
+ cache.add('c', 10, 0, true)
994
+ assert_equal "NOT_STORED\r\n", cache.add('a', 11)
995
+ assert_equal({ 'a' => 9, 'b' => 11 }, cache.get_multi(['a', 'b']))
996
+ inc = cache.incr('c', 10)
997
+ assert_equal 0, inc % 5
998
+ assert inc > 14
999
+ assert cache.decr('c', 5) > 14
1000
+ assert_equal 11, cache.get('b')
1001
+ end
1002
+ end
1003
+ end
1004
+
1005
+ workers.each { |w| w.join }
1006
+ cache.flush_all
1007
+ end
1008
+ end
1009
+
975
1010
  end
976
1011
 
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: mperham-memcache-client
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.6.4
4
+ version: 1.6.5
5
5
  platform: ruby
6
6
  authors:
7
7
  - Eric Hodel
@@ -11,7 +11,7 @@ autorequire:
11
11
  bindir: bin
12
12
  cert_chain: []
13
13
 
14
- date: 2009-02-13 00:00:00 -08:00
14
+ date: 2009-02-25 00:00:00 -08:00
15
15
  default_executable:
16
16
  dependencies: []
17
17