memcached 0.5 → 0.6

data/CHANGELOG

@@ -1,2 +1,4 @@
 
+v0.6. Better documentation; benchmark suite; improve buffered IO API; remove namespace accessor in favor of generic options hash; patch up extconf.rb to handle unusual library situations; increase test coverage.
+
 v0.5. First release.

data/Manifest

@@ -1,6 +1,5 @@
 CHANGELOG
 ext/extconf.rb
-ext/libmemcached.h
 ext/libmemcached.i
 ext/libmemcached_wrap.c
 lib/memcached/behaviors.rb
@@ -11,9 +10,12 @@ lib/memcached.rb
 LICENSE
 Manifest
 README
-test/benchmark/benchmark_test.rb
+test/benchmark/benchmark.rb
+test/benchmark/benchmark_set_get.rb
+test/benchmark/profile.rb
 test/setup.rb
 test/teardown.rb
 test/test_helper.rb
 test/unit/binding_test.rb
 test/unit/memcached_test.rb
+TODO

data/README

@@ -7,14 +7,10 @@ An interface to the libmemcached C client.
 
 Copyright 2008 Cloudburst, LLC. Licensed under the AFL 3. See the included LICENSE file. Portions copyright 2008 TangentOrg, Brian Aker, licensed under the BSD license, and used with permission.
 
-The public certificate for the gem is here[http://rubyforge.org/frs/download.php/25331/evan_weaver-original-public_cert.pem]. 
+The public certificate for this gem is here[http://rubyforge.org/frs/download.php/25331/evan_weaver-original-public_cert.pem]. 
 
 If you like this software, please {make a donation}[http://blog.evanweaver.com/donate/], or {recommend Evan}[http://www.workingwithrails.com/person/7739-evan-weaver] at Working with Rails.
 
-== Status
-
-Very new. Risky in production.
-
 == Features
 
 * clean API
@@ -22,13 +18,13 @@ Very new. Risky in production.
 * multiple hashing modes, including consistent hashing
 * ludicrous speed, including optional non-blocking IO
 
-== Requirements
+The <b>memcached</b> library wraps the pure-C libmemcached client via SWIG. Both libraries are very new; use caution.
 
-* {libmemcached 0.13}[http://tangent.org/552/libmemcached.html]
+== Installation
 
-Other versions of libmemcached are not supported.
+You need {libmemcached 0.13}[http://tangent.org/552/libmemcached.html]. Other versions of libmemcached are not supported. You also need {memcached itself}[http://www.danga.com/memcached/] if you want to test against a local server.
 
-Extract the tarball, then for Linux, run:
+Download and extract the {libmemcached tarball}[http://download.tangent.org/libmemcached-0.13.tar.gz]. Then for Linux, run:
   ./configure
   make && sudo make install
   
@@ -36,9 +32,96 @@ For OS X/MacPorts, run:
   ./configure --prefix=/opt/local
   make && sudo make install
   
+Now install the gem:
+  sudo gem install memcached --no-rdoc --no-ri
+  
 == Usage
 
-Whatever!
+Start a local memcached server:
+  $ memcached -p 11211 &
+
+Now, in Ruby, require the library and instantiate a Memcached object at a global level:
+
+  require 'memcached'
+  $cache = Memcached.new("127.0.0.1:11211")
+  
+Now you can set things and get things:
+ 
+  value = 'hello'
+  $cache.set 'test', value
+  $cache.get 'test' #=> "hello"
+
+You can set with an expiration timeout:
+
+  value = 'hello'
+  $cache.set 'test', value, 1
+  sleep(2)
+  $cache.get 'test' #=> raises Memcached::NotFound
+
+You can get multiple values at once:
+
+  value = 'hello'
+  $cache.set 'test', value
+  $cache.get ['test', 'test2'] #=> ["hello", Memcached::NotFound]
+  
+You can set a counter and increment it:
+
+  start = 1
+  $cache.set 'counter', start, 0, false
+  $cache.increment 'counter' #=> 2
+  $cache.increment 'counter' #=> 3
+  $cache.get('counter', false).to_i #=> 3
+
+You can get some server stats:
+
+  $cache.stats #=> {..., :bytes_written=>[62], :version=>["1.2.4"] ...}
+  
+Note that the API is not the same as that of <b>Ruby-MemCache</b> or <b>memcache-client</b>. In particular, <tt>nil</tt> is a valid record value. Memcached#get does not return <tt>nil</tt> on failure, rather it raises <b>Memcached::NotFound</b>. This is consistent with the behavior of memcached itself. For example:
+
+  $cache.set 'test', nil
+  $cache.get 'test' #=> nil
+  $cache.delete 'test'
+  $cache.get 'test' #=> raises Memcached::NotFound
+
+A compatibility wrapper for legacy applications is in progress.
+  
+== Threading
+
+<b>memcached</b> is threadsafe, but each thread requires its own Memcached instance. Create a global Memcached, and then call Memcached#clone each time you spawn a thread.
+
+  thread = Thread.new do
+    cache = $cache.clone
+    # Perform operations on cache, not $cache
+    cache.set 'example', 1
+    cache.get 'example'
+    # ...
+  end  
+
+  # Join the thread so that exceptions don't get lost
+  thread.join
+
+== Benchmarks
+
+<b>memcached</b> is much, much faster than <b>memcache-client</b>:
+
+                                       user     system      total        real
+  set:ruby:noblock:memcached       0.050000   0.000000   0.050000 (  0.054205)
+  set:ruby:memcached               0.130000   0.160000   0.290000 (  0.338529)
+  set:ruby:memcache-client         3.690000   0.130000   3.820000 (  4.030806)
+  ...
+  get:ruby:memcached               0.110000   0.020000   0.130000 (  0.316446)
+  get:ruby:memcache-client         3.730000   0.120000   3.850000 (  4.040446)
+  ...
+  missing:ruby:memcached           0.160000   0.090000   0.250000 (  0.401891)
+  missing:ruby:memcache-client     3.650000   0.090000   3.740000 (  3.880192)
+  ...
+  mixed:ruby:noblock:memcached     0.190000   0.230000   0.420000 (  0.667130)
+  mixed:ruby:memcached             0.250000   0.250000   0.500000 (  0.663093)
+  mixed:ruby:memcache-client       7.410000   0.210000   7.620000 (  7.942145)
+
+These benchmarks were run on x86-64 Linux. You can easily run your own benchmarks, as long as you have memcached itself on your system:
+  $ ruby -e 'system("ruby #{File.dirname(`gem which memcached`.split("\n").
+    last)}/../test/benchmark/benchmark.rb")'
 
 == Reporting problems
 
@@ -48,4 +131,5 @@ Patches and contributions are very welcome. Please note that contributors are re
 
 == Further resources
 
-Whatever!
+* {Memcached wiki}[http://www.socialtext.net/memcached/index.cgi]
+* {Libmemcached homepage}[http://tangent.org/552/libmemcached.html]

data/TODO

@@ -0,0 +1,11 @@
+
+Can happen any time:
+
+* real multiget
+* look for memory leaks
+
+Waiting on libmemcached 0.14:
+
+* verify version
+* CAS support
+* passing missing increment/decrement tests

data/ext/extconf.rb

@@ -12,5 +12,13 @@ if ENV['DEBUG']
 end
 
 dir_config 'libmemcached'
-find_library 'memcached', 'memcached_server_add'
+
+# XXX There's probably a better way to do this
+unless find_library 'memcached', 'memcached_server_add', *ENV['LD_LIBRARY_PATH'].to_s.split(":")
+  raise "shared library 'libmemcached' not found"
+end
+unless find_header 'libmemcached/memcached.h', *ENV['INCLUDE_PATH'].to_s.split(":")
+  raise "header file 'libmemcached/memcached.h' not  found"
+end
+
 create_makefile 'libmemcached'

data/ext/libmemcached.i

@@ -55,7 +55,7 @@
   free($1);
 };
 
-%include "libmemcached.h"
+%include "/opt/local/include/libmemcached/memcached.h"
 
 // Manual wrappers
 
@@ -94,3 +94,14 @@ void memcached_repair_server_st(memcached_st *in_ptr, memcached_server_st *host)
     host->write_ptr= 0;
 };
 %}
+
+// Stub some bogus methods left in the 0.13 memcached.h header
+%{
+memcached_return memcached_mdelete(memcached_st *ptr, char **key, size_t *key_length, unsigned int number_of_keys, time_t expiration) {
+  return;
+};
+memcached_return memcached_mdelete_by_key(memcached_st *ptr, char *master_key, size_t master_key_length,
+ char **key, size_t *key_length, unsigned int number_of_keys, time_t expiration) {
+  return;
+};
+%}

data/ext/libmemcached_wrap.c

@@ -1940,6 +1940,15 @@ void memcached_repair_server_st(memcached_st *in_ptr, memcached_server_st *host)
     host->write_ptr= 0;
 };
 
+
+memcached_return memcached_mdelete(memcached_st *ptr, char **key, size_t *key_length, unsigned int number_of_keys, time_t expiration) {
+  return;
+};
+memcached_return memcached_mdelete_by_key(memcached_st *ptr, char *master_key, size_t master_key_length,
+ char **key, size_t *key_length, unsigned int number_of_keys, time_t expiration) {
+  return;
+};
+
 swig_class cMemcachedServerSt;
 
 SWIGINTERN VALUE
@@ -8222,6 +8231,135 @@ fail:
 }
 
 
+SWIGINTERN VALUE
+_wrap_memcached_mdelete(int argc, VALUE *argv, VALUE self) {
+  memcached_st *arg1 = (memcached_st *) 0 ;
+  char **arg2 = (char **) 0 ;
+  size_t *arg3 = (size_t *) 0 ;
+  unsigned int arg4 ;
+  time_t arg5 ;
+  memcached_return result;
+  void *argp1 = 0 ;
+  int res1 = 0 ;
+  void *argp2 = 0 ;
+  int res2 = 0 ;
+  void *argp3 = 0 ;
+  int res3 = 0 ;
+  unsigned int val4 ;
+  int ecode4 = 0 ;
+  VALUE vresult = Qnil;
+  
+  if ((argc < 5) || (argc > 5)) {
+    rb_raise(rb_eArgError, "wrong # of arguments(%d for 5)",argc); SWIG_fail;
+  }
+  res1 = SWIG_ConvertPtr(argv[0], &argp1,SWIGTYPE_p_memcached_st, 0 |  0 );
+  if (!SWIG_IsOK(res1)) {
+    SWIG_exception_fail(SWIG_ArgError(res1), "in method '" "memcached_mdelete" "', argument " "1"" of type '" "memcached_st *""'"); 
+  }
+  arg1 = (memcached_st *)(argp1);
+  res2 = SWIG_ConvertPtr(argv[1], &argp2,SWIGTYPE_p_p_char, 0 |  0 );
+  if (!SWIG_IsOK(res2)) {
+    SWIG_exception_fail(SWIG_ArgError(res2), "in method '" "memcached_mdelete" "', argument " "2"" of type '" "char **""'"); 
+  }
+  arg2 = (char **)(argp2);
+  res3 = SWIG_ConvertPtr(argv[2], &argp3,SWIGTYPE_p_size_t, 0 |  0 );
+  if (!SWIG_IsOK(res3)) {
+    SWIG_exception_fail(SWIG_ArgError(res3), "in method '" "memcached_mdelete" "', argument " "3"" of type '" "size_t *""'"); 
+  }
+  arg3 = (size_t *)(argp3);
+  ecode4 = SWIG_AsVal_unsigned_SS_int(argv[3], &val4);
+  if (!SWIG_IsOK(ecode4)) {
+    SWIG_exception_fail(SWIG_ArgError(ecode4), "in method '" "memcached_mdelete" "', argument " "4"" of type '" "unsigned int""'");
+  } 
+  arg4 = (unsigned int)(val4);
+  {
+    if (NIL_P(argv[4]))
+    arg5 = (time_t)-1;
+    else
+    arg5 = NUM2LONG(rb_funcall(argv[4], rb_intern("tv_sec"), 0));
+  }
+  result = (memcached_return)memcached_mdelete(arg1,arg2,arg3,arg4,arg5);
+  vresult = SWIG_From_int((int)(result));
+  return vresult;
+fail:
+  return Qnil;
+}
+
+
+SWIGINTERN VALUE
+_wrap_memcached_mdelete_by_key(int argc, VALUE *argv, VALUE self) {
+  memcached_st *arg1 = (memcached_st *) 0 ;
+  char *arg2 = (char *) 0 ;
+  size_t arg3 ;
+  char **arg4 = (char **) 0 ;
+  size_t *arg5 = (size_t *) 0 ;
+  unsigned int arg6 ;
+  time_t arg7 ;
+  memcached_return result;
+  void *argp1 = 0 ;
+  int res1 = 0 ;
+  int res2 ;
+  char *buf2 = 0 ;
+  int alloc2 = 0 ;
+  size_t val3 ;
+  int ecode3 = 0 ;
+  void *argp4 = 0 ;
+  int res4 = 0 ;
+  void *argp5 = 0 ;
+  int res5 = 0 ;
+  unsigned int val6 ;
+  int ecode6 = 0 ;
+  VALUE vresult = Qnil;
+  
+  if ((argc < 7) || (argc > 7)) {
+    rb_raise(rb_eArgError, "wrong # of arguments(%d for 7)",argc); SWIG_fail;
+  }
+  res1 = SWIG_ConvertPtr(argv[0], &argp1,SWIGTYPE_p_memcached_st, 0 |  0 );
+  if (!SWIG_IsOK(res1)) {
+    SWIG_exception_fail(SWIG_ArgError(res1), "in method '" "memcached_mdelete_by_key" "', argument " "1"" of type '" "memcached_st *""'"); 
+  }
+  arg1 = (memcached_st *)(argp1);
+  res2 = SWIG_AsCharPtrAndSize(argv[1], &buf2, NULL, &alloc2);
+  if (!SWIG_IsOK(res2)) {
+    SWIG_exception_fail(SWIG_ArgError(res2), "in method '" "memcached_mdelete_by_key" "', argument " "2"" of type '" "char *""'");
+  }
+  arg2 = buf2;
+  ecode3 = SWIG_AsVal_size_t(argv[2], &val3);
+  if (!SWIG_IsOK(ecode3)) {
+    SWIG_exception_fail(SWIG_ArgError(ecode3), "in method '" "memcached_mdelete_by_key" "', argument " "3"" of type '" "size_t""'");
+  } 
+  arg3 = (size_t)(val3);
+  res4 = SWIG_ConvertPtr(argv[3], &argp4,SWIGTYPE_p_p_char, 0 |  0 );
+  if (!SWIG_IsOK(res4)) {
+    SWIG_exception_fail(SWIG_ArgError(res4), "in method '" "memcached_mdelete_by_key" "', argument " "4"" of type '" "char **""'"); 
+  }
+  arg4 = (char **)(argp4);
+  res5 = SWIG_ConvertPtr(argv[4], &argp5,SWIGTYPE_p_size_t, 0 |  0 );
+  if (!SWIG_IsOK(res5)) {
+    SWIG_exception_fail(SWIG_ArgError(res5), "in method '" "memcached_mdelete_by_key" "', argument " "5"" of type '" "size_t *""'"); 
+  }
+  arg5 = (size_t *)(argp5);
+  ecode6 = SWIG_AsVal_unsigned_SS_int(argv[5], &val6);
+  if (!SWIG_IsOK(ecode6)) {
+    SWIG_exception_fail(SWIG_ArgError(ecode6), "in method '" "memcached_mdelete_by_key" "', argument " "6"" of type '" "unsigned int""'");
+  } 
+  arg6 = (unsigned int)(val6);
+  {
+    if (NIL_P(argv[6]))
+    arg7 = (time_t)-1;
+    else
+    arg7 = NUM2LONG(rb_funcall(argv[6], rb_intern("tv_sec"), 0));
+  }
+  result = (memcached_return)memcached_mdelete_by_key(arg1,arg2,arg3,arg4,arg5,arg6,arg7);
+  vresult = SWIG_From_int((int)(result));
+  if (alloc2 == SWIG_NEWOBJ) free((char*)buf2);
+  return vresult;
+fail:
+  if (alloc2 == SWIG_NEWOBJ) free((char*)buf2);
+  return Qnil;
+}
+
+
 SWIGINTERN VALUE
 _wrap_memcached_fetch_execute(int argc, VALUE *argv, VALUE self) {
   memcached_st *arg1 = (memcached_st *) 0 ;
@@ -9146,6 +9284,8 @@ SWIGEXPORT void Init_libmemcached(void) {
   rb_define_module_function(mLibmemcached, "memcached_append_by_key", _wrap_memcached_append_by_key, -1);
   rb_define_module_function(mLibmemcached, "memcached_cas_by_key", _wrap_memcached_cas_by_key, -1);
   rb_define_module_function(mLibmemcached, "memcached_delete_by_key", _wrap_memcached_delete_by_key, -1);
+  rb_define_module_function(mLibmemcached, "memcached_mdelete", _wrap_memcached_mdelete, -1);
+  rb_define_module_function(mLibmemcached, "memcached_mdelete_by_key", _wrap_memcached_mdelete_by_key, -1);
   rb_define_module_function(mLibmemcached, "memcached_fetch_execute", _wrap_memcached_fetch_execute, -1);
   rb_define_module_function(mLibmemcached, "memcached_result_free", _wrap_memcached_result_free, -1);
   rb_define_module_function(mLibmemcached, "memcached_result_create", _wrap_memcached_result_create, -1);

data/lib/memcached.rb

@@ -4,3 +4,20 @@ require 'memcached/integer'
 require 'memcached/exceptions'
 require 'memcached/behaviors'
 require 'memcached/memcached'
+
+=begin rdoc
+The generated SWIG module for accessing libmemcached's C API.
+
+Includes the full set of libmemcached static methods (as defined in <tt>$INCLUDE_PATH/libmemcached/memcached.h</tt>), and classes for the available structs:
+
+* Libmemcached::MemcachedResultSt
+* Libmemcached::MemcachedServerSt
+* Libmemcached::MemcachedSt
+* Libmemcached::MemcachedStatSt
+* Libmemcached::MemcachedStringSt
+
+A number of SWIG typemaps and C helper methods are also defined in <tt>ext/libmemcached.i</tt>.
+
+=end
+module Libmemcached
+end

data/lib/memcached/behaviors.rb

@@ -1,6 +1,8 @@
 
 class Memcached
 
+#:stopdoc:
+
   def self.load_constants(prefix, hash = {}, offset = 0)
     Libmemcached.constants.grep(/^#{prefix}/).each do |const_name|
       hash[const_name[prefix.length..-1].downcase.to_sym] = Libmemcached.const_get(const_name) + offset
@@ -21,9 +23,12 @@ class Memcached
   DISTRIBUTION_VALUES = {}
   BEHAVIOR_VALUES.merge!(load_constants("MEMCACHED_DISTRIBUTION_", DISTRIBUTION_VALUES, 2))
 
+#:startdoc:
+
   private
   
-  def set_behavior(behavior, value)    
+  # Set a behavior option for this Memcached instance. Accepts a Symbol <tt>behavior</tt> and either <tt>true</tt>, <tt>false</tt>, or a Symbol for <tt>value</tt>. Arguments are validated and converted into integers for the struct setter method.
+  def set_behavior(behavior, value) #:doc:
     raise ArgumentError, "No setting #{behavior.inspect}" unless b_id = BEHAVIORS[behavior]    
     raise ArgumentError, "No setting value #{value.inspect}" unless v_id = BEHAVIOR_VALUES[value]
     

data/lib/memcached/exceptions.rb

@@ -1,12 +1,55 @@
 
 class Memcached
 
+=begin rdoc
+
+Superclass for all Memcached runtime exceptions. 
+
+Subclasses correspond one-to-one with server response strings or libmemcached errors. For example, raising <b>Memcached::NotFound</b> means that the server returned <tt>"NOT_FOUND\r\n"</tt>.
+
+== Subclasses
+
+* Memcached::AKeyLengthOfZeroWasProvided
+* Memcached::ATimeoutOccurred
+* Memcached::ActionNotSupported
+* Memcached::ActionQueued
+* Memcached::ClientError
+* Memcached::ConnectionBindFailure
+* Memcached::ConnectionDataDoesNotExist
+* Memcached::ConnectionDataExists
+* Memcached::ConnectionFailure
+* Memcached::ConnectionSocketCreateFailure
+* Memcached::CouldNotOpenUnixSocket
+* Memcached::Failure
+* Memcached::FetchWasNotCompleted
+* Memcached::HostnameLookupFailure
+* Memcached::MemoryAllocationFailure
+* Memcached::NoServersDefined
+* Memcached::NotFound
+* Memcached::NotStored
+* Memcached::PartialRead
+* Memcached::ProtocolError
+* Memcached::ReadFailure
+* Memcached::ServerDelete
+* Memcached::ServerEnd
+* Memcached::ServerError
+* Memcached::ServerValue
+* Memcached::SomeErrorsWereReported
+* Memcached::StatValue
+* Memcached::SystemError
+* Memcached::UnknownReadFailure
+* Memcached::WriteFailure
+
+=end
   class Error < RuntimeError
   end
   
-  class NotImplemented < StandardError
+  # Raised if a method depends on functionality not yet completed in libmemcached. 
+  class NotImplemented < NoMethodError
   end
 
+#:stopdoc:
+
   class << self
     private
     def camelize(string)
@@ -14,17 +57,19 @@ class Memcached
     end   
   end
   
-  @@exceptions = []
-  @@empty_struct = Libmemcached::MemcachedSt.new
-  Libmemcached.memcached_create(@@empty_struct)
+  EXCEPTIONS = []
+  EMPTY_STRUCT = Libmemcached::MemcachedSt.new
+  Libmemcached.memcached_create(EMPTY_STRUCT)
   
   # Generate exception classes
-  Libmemcached::MEMCACHED_MAXIMUM_RETURN.times do |exception_index|    
-    description = Libmemcached.memcached_strerror(@@empty_struct, exception_index)
+  Libmemcached::MEMCACHED_MAXIMUM_RETURN.times do |index|
+    description = Libmemcached.memcached_strerror(EMPTY_STRUCT, index)
     exception_class = eval("class #{camelize(description)} < Error; self; end")
-    @@exceptions << exception_class
+    EXCEPTIONS << exception_class
   end
   
   # Verify library version
-  # XXX Impossible with current libmemcached  
+  # XXX Waiting on libmemcached 0.14
+  
+#:startdoc:
 end