memcache-client 1.7.8 → 1.8.0

Sign up to get free protection for your applications and to get access to all the features.
data/FAQ.rdoc CHANGED
@@ -25,7 +25,8 @@ You can increase the timeout or disable them completely with the following confi
25
25
 
26
26
  == Isn't Evan Weaver's memcached gem faster?
27
27
 
28
- The latest version of memcached-client is anywhere from 33% to 100% slower than memcached in various benchmarks. Keep in mind this means that 10,000 get requests take 1.8 sec instead of 1.2 seconds.
28
+ The latest version of memcached-client is anywhere from 33% to 100% slower than memcached in various
29
+ benchmarks. Keep in mind this means that 10,000 get requests take 1.8 sec instead of 1.2 seconds.
29
30
  In practice, memcache-client is unlikely to be a bottleneck in your system but there is always going
30
31
  to be an overhead to pure Ruby. memcache-client does have the advantage of built-in integration into
31
32
  Rails and should work on non-MRI platforms: JRuby, MacRuby, etc.
@@ -1,3 +1,9 @@
1
+ = 1.8.0 (2010-03-05)
2
+
3
+ * Add support for EventMachine-based connections.
4
+ * Add support for raw values in get_multi
5
+ * Add memcached_top binary for gathering server statistics
6
+
1
7
  = 1.7.8 (2010-02-03)
2
8
 
3
9
  * Fix issue where autofix_keys logic did not account for namespace length. (menno)
@@ -36,6 +36,22 @@ Rails 2.1+ includes memcache-client 1.5.0 out of the box. See ActiveSupport::Ca
36
36
  and the Rails.cache method for more details. Rails 2.3+ will use the latest memcache-client
37
37
  gem installed.
38
38
 
39
+ == Using memcache-client with EventMachine
40
+
41
+ memcache-client 1.8.0 added support for native EventMachine connections using
42
+ Ruby 1.9. If you are using an EventMachine-based application (e.g. thin), you can
43
+ activate the EventMachine support like so:
44
+
45
+ require 'memcache'
46
+ require 'memcache/event_machine'
47
+
48
+ EM.run do
49
+ Fiber.new do
50
+ m = MemCache.new('localhost:11211')
51
+ m.set 'abc', 'xyz'
52
+ m.get 'abc'
53
+ end.resume
54
+ end
39
55
 
40
56
  == Questions?
41
57
 
data/Rakefile CHANGED
@@ -14,6 +14,7 @@ begin
14
14
  s.has_rdoc = true
15
15
  s.files = FileList["[A-Z]*", "{lib,test}/**/*", 'performance.txt']
16
16
  s.test_files = FileList["test/test_*.rb"]
17
+ s.executables = ['memcached_top']
17
18
  end
18
19
  Jeweler::GemcutterTasks.new
19
20
  rescue LoadError
@@ -29,6 +30,7 @@ end
29
30
 
30
31
  Rake::TestTask.new do |t|
31
32
  t.warning = true
33
+ t.libs = ['lib', 'test']
32
34
  end
33
35
 
34
36
  task :default => :test
@@ -1,4 +1,5 @@
1
- ---
2
- :patch: 8
1
+ ---
2
+ :build:
3
3
  :major: 1
4
- :minor: 7
4
+ :minor: 8
5
+ :patch: 0
@@ -0,0 +1,56 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ require 'optparse'
4
+ require 'ostruct'
5
+ require 'socket'
6
+
7
+ @options = OpenStruct.new
8
+ @options.hostname = 'localhost'
9
+ @options.port = 11211
10
+
11
+ op = OptionParser.new do |opts|
12
+ opts.banner = "View memcached server statistics\nUsage: #{$0} [options]"
13
+ opts.separator "General Options:"
14
+ opts.on("-h HOSTNAME", "--hostname=HOSTNAME", "Hostname [default: localhost]") do |h|
15
+ @options.hostname = h
16
+ end
17
+ opts.on("-p PORT", "--port=PORT", Integer, "Port [default: 11211]") do |p|
18
+ @options.port = p
19
+ end
20
+ opts.on_tail("--help", "Show this message") do
21
+ puts opts
22
+ exit
23
+ end
24
+ end
25
+ op.parse!
26
+
27
+ def stats_data
28
+ data = ''
29
+ sock = TCPSocket.new(@options.hostname, @options.port)
30
+ sock.print("stats\r\n")
31
+ sock.flush
32
+ # memcached does not close the socket once it is done writing
33
+ # the stats data. We need to read line by line until we detect
34
+ # the END line and then stop/close on our side.
35
+ stats = sock.gets
36
+ while true
37
+ data += stats
38
+ break if stats.strip == 'END'
39
+ stats = sock.gets
40
+ end
41
+ sock.close
42
+ data
43
+ end
44
+
45
+ def parse(stats_data)
46
+ stats = []
47
+ stats_data.each_line do |line|
48
+ stats << "#{$1}: #{$2}" if line =~ /STAT (\w+) (\S+)/
49
+ end
50
+ stats.sort
51
+ end
52
+
53
+ stats = parse(stats_data)
54
+ stats.each do |stat|
55
+ puts stat
56
+ end
@@ -1,3 +1,4 @@
1
+ # encoding: utf-8
1
2
  $TESTING = defined?($TESTING) && $TESTING
2
3
 
3
4
  require 'socket'
@@ -295,11 +296,16 @@ class MemCache
295
296
  # cache["b"] = 2
296
297
  # cache.get_multi "a", "b" # => { "a" => 1, "b" => 2 }
297
298
  #
298
- # Note that get_multi assumes the values are marshalled.
299
+ # Note that get_multi assumes the values are marshalled. You can pass
300
+ # in :raw => true to bypass value marshalling.
301
+ #
302
+ # cache.get_multi('a', 'b', ..., :raw => true)
299
303
 
300
304
  def get_multi(*keys)
301
305
  raise MemCacheError, 'No active servers' unless active?
302
306
 
307
+ opts = keys.last.is_a?(Hash) ? keys.pop : {}
308
+
303
309
  keys.flatten!
304
310
  key_count = keys.length
305
311
  cache_keys = {}
@@ -313,13 +319,13 @@ class MemCache
313
319
  end
314
320
 
315
321
  results = {}
316
-
322
+ raw = opts[:raw] || false
317
323
  server_keys.each do |server, keys_for_server|
318
324
  keys_for_server_str = keys_for_server.join ' '
319
325
  begin
320
326
  values = cache_get_multi server, keys_for_server_str
321
327
  values.each do |key, value|
322
- results[cache_keys[key]] = Marshal.load value
328
+ results[cache_keys[key]] = raw ? value : Marshal.load(value)
323
329
  end
324
330
  rescue IndexError => e
325
331
  # Ignore this server and try the others
@@ -0,0 +1,149 @@
1
+ # Extensions for using memcache-client with EventMachine
2
+
3
+ raise "memcache/event_machine requires Ruby 1.9" if RUBY_VERSION < '1.9'
4
+
5
+ require 'fiber'
6
+
7
+ class MemCache::Server
8
+
9
+ alias :blocking_socket :socket
10
+
11
+ def socket
12
+ # Support plain old TCP socket connections if the user
13
+ # has not setup EM. Much easier to deal with in irb,
14
+ # script/console, etc.
15
+ return blocking_socket if !EM.reactor_running?
16
+
17
+ return @sock if @sock and not @sock.closed?
18
+
19
+ @sock = nil
20
+
21
+ # If the host was dead, don't retry for a while.
22
+ return if @retry and @retry > Time.now
23
+
24
+ fiber = Fiber.current
25
+ @sock = EM::SocketConnection.connect(@host, @port, @timeout)
26
+ yielding = true
27
+ @sock.callback do
28
+ @status = 'CONNECTED'
29
+ @retry = nil
30
+ yielding = false
31
+ fiber.resume if Fiber.current != fiber
32
+ end
33
+ @sock.errback do
34
+ yielding = false
35
+ fiber.resume if Fiber.current != fiber
36
+ end
37
+ Fiber.yield if yielding
38
+ @sock
39
+ end
40
+
41
+ end
42
+
43
+ module EM
44
+ module SocketConnection
45
+ include EM::Deferrable
46
+
47
+ def self.connect(host, port, timeout)
48
+ EM.connect(host, port, self) do |conn|
49
+ conn.pending_connect_timeout = timeout
50
+ end
51
+ end
52
+
53
+ def initialize
54
+ @connected = false
55
+ @index = 0
56
+ @buf = ''
57
+ end
58
+
59
+ def closed?
60
+ !@connected
61
+ end
62
+
63
+ def close
64
+ @connected = false
65
+ close_connection(true)
66
+ end
67
+
68
+ def write(buf)
69
+ send_data(buf)
70
+ end
71
+
72
+ def read(size)
73
+ if can_read?(size)
74
+ yank(size)
75
+ else
76
+ fiber = Fiber.current
77
+ @size = size
78
+ @callback = proc { |data|
79
+ fiber.resume(data)
80
+ }
81
+ # TODO Can leak fiber if the connection dies while
82
+ # this fiber is yielded, waiting for data
83
+ Fiber.yield
84
+ end
85
+ end
86
+
87
+ SEP = "\r\n"
88
+
89
+ def gets
90
+ while true
91
+ # Read to ensure we have some data in the buffer
92
+ line = read(2)
93
+ # Reset the buffer index to zero
94
+ @buf = @buf.slice(@index..-1)
95
+ @index = 0
96
+ if eol = @buf.index(SEP)
97
+ line << yank(eol + SEP.size)
98
+ break
99
+ else
100
+ # EOL not in the current buffer
101
+ line << yank(@buf.size)
102
+ end
103
+ end
104
+ line
105
+ end
106
+
107
+ def can_read?(size)
108
+ @buf.size >= @index + size
109
+ end
110
+
111
+ # EM callbacks
112
+
113
+ def receive_data(data)
114
+ @buf << data
115
+
116
+ if @callback and can_read?(@size)
117
+ callback = @callback
118
+ data = yank(@size)
119
+ @callback = @size = nil
120
+ callback.call(data)
121
+ end
122
+ end
123
+
124
+ def post_init
125
+ @connected = true
126
+ succeed
127
+ end
128
+
129
+ def unbind
130
+ @connected = false
131
+ end
132
+
133
+ private
134
+
135
+ BUFFER_SIZE = 4096
136
+
137
+ def yank(len)
138
+ data = @buf.slice(@index, len)
139
+ @index += len
140
+ @index = @buf.size if @index > @buf.size
141
+ if @index >= BUFFER_SIZE
142
+ @buf = @buf.slice(@index..-1)
143
+ @index = 0
144
+ end
145
+ data
146
+ end
147
+
148
+ end
149
+ end
@@ -1,13 +1,8 @@
1
- HERE = File.dirname(__FILE__)
2
- $LOAD_PATH.unshift "#{HERE}/../lib"
3
- #$LOAD_PATH << "/Library/Ruby/Gems/1.8/gems/activesupport-2.2.2/lib/active_support/vendor/memcache-client-1.5.1"
4
-
5
1
  require 'benchmark'
6
- require 'rubygems'
7
2
  require 'test/unit'
8
3
 
9
4
  $TESTING = true
10
- require 'memcache' if not defined?(MemCache)
5
+ require 'memcache'
11
6
 
12
7
  class TestBenchmark < Test::Unit::TestCase
13
8
 
@@ -34,12 +29,24 @@ class TestBenchmark < Test::Unit::TestCase
34
29
  @key5 = "Medium2"*8
35
30
  @key6 = "Long3"*40
36
31
  end
32
+
33
+ def test_em
34
+ return if RUBY_VERSION < '1.9'
35
+ require 'eventmachine'
36
+ require 'memcache/event_machine'
37
+ puts "with EventMachine"
38
+ EM.run do
39
+ Fiber.new do
40
+ test_benchmark
41
+ EM.stop
42
+ end.resume
43
+ end
44
+ end
37
45
 
38
46
  def test_benchmark
39
47
  Benchmark.bm(31) do |x|
40
48
 
41
49
  n = 2500
42
- # n = 1000
43
50
 
44
51
  @m = MemCache.new(*@opts)
45
52
  x.report("set:plain:memcache-client") do
@@ -0,0 +1,62 @@
1
+ # encoding: ascii-8bit
2
+ require 'test/unit'
3
+ require 'memcache'
4
+
5
+ class TestEventMachine < Test::Unit::TestCase
6
+
7
+ def test_live_server
8
+ return puts("Skipping EventMachine test, not Ruby 1.9") if RUBY_VERSION < '1.9'
9
+ return puts("Skipping EventMachine test, no live server") if !live_server?
10
+
11
+ require 'eventmachine'
12
+ require 'memcache/event_machine'
13
+ ex = nil
14
+ within_em do
15
+ begin
16
+ m = MemCache.new(['127.0.0.1:11211', 'localhost:11211'])
17
+ key1 = 'foo'
18
+ key2 = 'bar'*50
19
+ key3 = '£∞'*50
20
+ value1 = 'abc'
21
+ value2 = 'xyz'*1000
22
+ value3 = '∞§¶•ª'*1000
23
+
24
+ 1000.times do
25
+ assert_equal "STORED\r\n", m.set(key1, value1)
26
+ assert_equal "STORED\r\n", m.set(key2, value2)
27
+ assert_equal "STORED\r\n", m.set(key3, value3)
28
+ assert_equal value1, m.get(key1)
29
+ assert_equal value2, m.get(key2)
30
+ assert_equal value3, m.get(key3)
31
+ assert_equal "DELETED\r\n", m.delete(key1)
32
+ assert_equal "STORED\r\n", m.set(key1, value2)
33
+ assert_equal value2, m.get(key1)
34
+ assert_equal "STORED\r\n", m.set(key2, value3)
35
+ assert_equal value3, m.get(key2)
36
+ assert_equal "STORED\r\n", m.set(key3, value1)
37
+ assert_equal value1, m.get(key3)
38
+ assert_equal({ key1 => value2, key2 => value3, key3 => value1 },
39
+ m.get_multi(key1, key2, key3))
40
+ end
41
+ rescue Exception => exp
42
+ puts exp.message
43
+ ex = exp
44
+ ensure
45
+ EM.stop
46
+ end
47
+ end
48
+ raise ex if ex
49
+ end
50
+
51
+ private
52
+
53
+ def within_em(&block)
54
+ EM.run do
55
+ Fiber.new(&block).resume
56
+ end
57
+ end
58
+
59
+ def live_server?
60
+ TCPSocket.new('localhost', 11211) rescue nil
61
+ end
62
+ end
@@ -2,6 +2,9 @@
2
2
  require 'logger'
3
3
  require 'stringio'
4
4
  require 'test/unit'
5
+ $TESTING = true
6
+ require 'memcache'
7
+
5
8
  require 'rubygems'
6
9
  begin
7
10
  gem 'flexmock'
@@ -11,9 +14,6 @@ rescue LoadError => e
11
14
  end
12
15
 
13
16
  Thread.abort_on_exception = true
14
- $TESTING = true
15
-
16
- require File.dirname(__FILE__) + '/../lib/memcache' if not defined?(MemCache)
17
17
 
18
18
  class MemCache
19
19
 
@@ -598,6 +598,28 @@ class TestMemCache < Test::Unit::TestCase
598
598
  assert_equal expected.sort, values.sort
599
599
  end
600
600
 
601
+ def test_get_multi_raw
602
+ server = FakeServer.new
603
+ server.socket.data.write "VALUE my_namespace:key 0 10\r\n"
604
+ server.socket.data.write "0123456789\r\n"
605
+ server.socket.data.write "VALUE my_namespace:keyb 0 10\r\n"
606
+ server.socket.data.write "9876543210\r\n"
607
+ server.socket.data.write "END\r\n"
608
+ server.socket.data.rewind
609
+
610
+ @cache.servers = []
611
+ @cache.servers << server
612
+
613
+ values = @cache.get_multi 'key', 'keyb', :raw => true
614
+
615
+ assert_equal "get my_namespace:key my_namespace:keyb\r\n",
616
+ server.socket.written.string
617
+
618
+ expected = { 'key' => '0123456789', 'keyb' => '9876543210' }
619
+
620
+ assert_equal expected.sort, values.sort
621
+ end
622
+
601
623
  def test_get_raw
602
624
  server = FakeServer.new
603
625
  server.socket.data.write "VALUE my_namespace:key 0 10\r\n"
metadata CHANGED
@@ -1,7 +1,12 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: memcache-client
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.7.8
4
+ prerelease: false
5
+ segments:
6
+ - 1
7
+ - 8
8
+ - 0
9
+ version: 1.8.0
5
10
  platform: ruby
6
11
  authors:
7
12
  - Eric Hodel
@@ -11,14 +16,14 @@ autorequire:
11
16
  bindir: bin
12
17
  cert_chain: []
13
18
 
14
- date: 2010-02-03 00:00:00 -06:00
15
- default_executable:
19
+ date: 2010-03-05 00:00:00 -06:00
20
+ default_executable: memcached_top
16
21
  dependencies: []
17
22
 
18
23
  description: A Ruby library for accessing memcached.
19
24
  email: mperham@gmail.com
20
- executables: []
21
-
25
+ executables:
26
+ - memcached_top
22
27
  extensions: []
23
28
 
24
29
  extra_rdoc_files:
@@ -33,9 +38,11 @@ files:
33
38
  - VERSION.yml
34
39
  - lib/continuum_native.rb
35
40
  - lib/memcache.rb
41
+ - lib/memcache/event_machine.rb
36
42
  - lib/memcache_util.rb
37
43
  - performance.txt
38
44
  - test/test_benchmark.rb
45
+ - test/test_event_machine.rb
39
46
  - test/test_mem_cache.rb
40
47
  has_rdoc: true
41
48
  homepage: http://github.com/mperham/memcache-client
@@ -50,21 +57,24 @@ required_ruby_version: !ruby/object:Gem::Requirement
50
57
  requirements:
51
58
  - - ">="
52
59
  - !ruby/object:Gem::Version
60
+ segments:
61
+ - 0
53
62
  version: "0"
54
- version:
55
63
  required_rubygems_version: !ruby/object:Gem::Requirement
56
64
  requirements:
57
65
  - - ">="
58
66
  - !ruby/object:Gem::Version
67
+ segments:
68
+ - 0
59
69
  version: "0"
60
- version:
61
70
  requirements: []
62
71
 
63
72
  rubyforge_project:
64
- rubygems_version: 1.3.5
73
+ rubygems_version: 1.3.6
65
74
  signing_key:
66
75
  specification_version: 3
67
76
  summary: A Ruby library for accessing memcached.
68
77
  test_files:
69
78
  - test/test_benchmark.rb
79
+ - test/test_event_machine.rb
70
80
  - test/test_mem_cache.rb