redis 2.0.0 → 2.2.1

data/.gitignore

@@ -0,0 +1,8 @@
+nohup.out
+redis/*
+rdsrv
+pkg/*
+coverage/*
+.idea
+*.rdb
+*.swp

data/CHANGELOG.md

@@ -0,0 +1,49 @@
+# 2.2.2 (unreleased)
+
+# 2.2.1
+
+* Internal API: Client#call and family are now called with a single array
+  argument, since splatting a large number of arguments (100K+) results in a
+  stack overflow on 1.9.2.
+
+* The `INFO` command can optionally take a subcommand. When the subcommand is
+  `COMMANDSTATS`, the client will properly format the returned statistics per
+  command. Subcommands for `INFO` are available since Redis v2.3.0 (unstable).
+
+* Change `IO#syswrite` back to the buffered `IO#write` since some Rubies do
+  short writes for large (1MB+) buffers and some don't (see issue #108).
+
+# 2.2.0
+
+* Added method `Redis#without_reconnect` that ensures the client will not try
+  to reconnect when running the code inside the specified block.
+
+* Thread-safe by default. Thread safety can be explicitly disabled by passing
+  `:thread_safe => false` as argument.
+
+* Commands called inside a MULTI/EXEC no longer raise error replies, since a
+  successful EXEC means the commands inside the block were executed.
+
+* MULTI/EXEC blocks are pipelined.
+
+* Don't disconnect on error replies.
+
+* Use `IO#syswrite` instead of `IO#write` because write buffering is not
+  necessary.
+
+* Connect to a unix socket by passing the `:path` option as argument.
+
+* The timeout value is coerced into a float, allowing sub-second timeouts.
+
+* Accept both `:with_scores` _and_ `:withscores` as argument to sorted set
+  commands.
+
+* Use [hiredis](https://github.com/pietern/hiredis-rb) (v0.3 or higher) by
+  requiring "redis/connection/hiredis".
+
+* Use [em-synchrony](https://github.com/igrigorik/em-synchrony) by requiring
+  "redis/connection/synchrony".
+
+# 2.1.1
+
+See commit log.

data/README.md

@@ -0,0 +1,208 @@
+# redis-rb
+
+A Ruby client library for the [Redis](http://redis.io) key-value store.
+
+## A note about versions
+
+Versions *1.0.x* target all versions of Redis. You have to use this one if you are using Redis < 1.2.
+
+Version *2.0* is a big refactoring of the previous version and makes little effort to be
+backwards-compatible when it shouldn't. It does not support Redis' original protocol, favoring the
+new, binary-safe one. You should be using this version if you're running Redis 1.2+.
+
+## Information about Redis
+
+Redis is a key-value store with some interesting features:
+
+1. It's fast.
+2. Keys are strings but values are typed. Currently Redis supports strings, lists, sets, sorted sets and hashes. [Atomic operations](http://redis.io/commands) can be done on all of these types.
+
+See [the Redis homepage](http://redis.io) for more information.
+
+## Getting started
+
+You can connect to Redis by instantiating the `Redis` class:
+
+    require "redis"
+
+    redis = Redis.new
+
+This assumes Redis was started with default values listening on `localhost`, port 6379. If you need to connect to a remote server or a different port, try:
+
+    redis = Redis.new(:host => "10.0.1.1", :port => 6380)
+
+To connect to Redis listening on a unix socket, try:
+
+    redis = Redis.new(:path => "/tmp/redis.sock")
+
+Once connected, you can start running commands against Redis:
+
+    >> redis.set "foo", "bar"
+    => "OK"
+
+    >> redis.get "foo"
+    => "bar"
+
+    >> redis.sadd "users", "albert"
+    => true
+
+    >> redis.sadd "users", "bernard"
+    => true
+
+    >> redis.sadd "users", "charles"
+    => true
+
+How many users?
+
+    >> redis.scard "users"
+    => 3
+
+Is `albert` a user?
+
+    >> redis.sismember "users", "albert"
+    => true
+
+Is `isabel` a user?
+
+    >> redis.sismember "users", "isabel"
+    => false
+
+Handle groups:
+
+    >> redis.sadd "admins", "albert"
+    => true
+
+    >> redis.sadd "admins", "isabel"
+    => true
+
+Users who are also admins:
+
+    >> redis.sinter "users", "admins"
+    => ["albert"]
+
+Users who are not admins:
+
+    >> redis.sdiff "users", "admins"
+    => ["bernard", "charles"]
+
+Admins who are not users:
+
+    >> redis.sdiff "admins", "users"
+    => ["isabel"]
+
+All users and admins:
+
+    >> redis.sunion "admins", "users"
+    => ["albert", "bernard", "charles", "isabel"]
+
+
+## Storing objects
+
+Redis only stores strings as values. If you want to store an object inside a key, you can use a serialization/deseralization mechanism like JSON:
+
+    >> redis.set "foo", [1, 2, 3].to_json
+    => OK
+
+    >> JSON.parse(redis.get("foo"))
+    => [1, 2, 3]
+
+## Executing multiple commands atomically
+
+You can use `MULTI/EXEC` to run arbitrary commands in an atomic fashion:
+
+    redis.multi do
+      redis.set "foo", "bar"
+      redis.incr "baz"
+    end
+
+## Multithreaded Operation
+
+Starting with redis-rb 2.2.0, the client is thread-safe by default. To use
+earlier versions safely in a multithreaded environment, be sure to initialize
+the client with `:thread_safe => true`. Thread-safety can be explicitly
+disabled for versions 2.2 and up by initializing the client with `:thread_safe
+=> false`.
+
+See the tests and benchmarks for examples.
+
+## Alternate drivers
+
+Non-default connection drivers are only used when they are explicitly required.
+By default, redis-rb uses Ruby's socket library to talk with Redis.
+
+### hiredis
+
+Using redis-rb with hiredis-rb (v0.3 or higher) as backend is done by requiring
+`redis/connection/hiredis` before requiring `redis`. This will make redis-rb
+pick up hiredis as default driver automatically. This driver optimizes for
+speed, at the cost of portability. Since hiredis is a C extension, JRuby is not
+supported (by default). Use hiredis when you have large array replies (think
+`LRANGE`, `SMEMBERS`, `ZRANGE`, etc.) and/or large pipelines of commands.
+
+Using redis-rb with hiredis from a Gemfile:
+
+    gem "hiredis", "~> 0.3.1"
+    gem "redis", "~> 2.2.0", :require => ["redis/connection/hiredis", "redis"]
+
+### synchrony
+
+This driver adds support for
+[em-synchrony](https://github.com/igrigorik/em-synchrony). Using the synchrony
+backend from redis-rb is done by requiring `redis/connection/synchrony` before
+requiring `redis`. This driver makes redis-rb work with EventMachine's
+asynchronous I/O, while not changing the exposed API. The hiredis gem needs to
+be available as well, because the synchrony driver uses hiredis for parsing the
+Redis protocol.
+
+Using redis-rb with synchrony from a Gemfile:
+
+    gem "hiredis", "~> 0.3.1"
+    gem "em-synchrony"
+    gem "redis", "~> 2.2.0", :require => ["redis/connection/synchrony", "redis"]
+
+## Testing
+
+This library (v2.2) is tested against the following interpreters:
+
+* MRI 1.8.7 (drivers: Ruby, hiredis)
+* MRI 1.9.2 (drivers: Ruby, hiredis, em-synchrony)
+* JRuby 1.6 (drivers: Ruby)
+* Rubinius 1.2 (drivers: Ruby, hiredis)
+
+## Known issues
+
+* Ruby 1.9 doesn't raise on socket timeouts in `IO#read` but rather retries the
+  read operation. This means socket timeouts don't work on 1.9 when using the
+  pure Ruby I/O code. Use hiredis when you want use socket timeouts on 1.9.
+
+* Ruby 1.8 *does* raise on socket timeouts in `IO#read`, but prints a warning
+  that using `IO#read` for non blocking reads is obsolete. This is wrong, since
+  the read is in fact blocking, but `EAGAIN` (which is returned on socket
+  timeouts) is interpreted as if the read was non blocking. Use hiredis to
+  prevent seeing this warning.
+
+## More info
+
+Check the [Redis Command Reference](http://redis.io/commands) or check the tests to find out how to use this client.
+
+## Contributors
+
+(ordered chronologically with more than 5 commits, see `git shortlog -sn` for
+all contributors)
+
+* Ezra Zygmuntowicz
+* Taylor Weibley
+* Matthew Clark
+* Brian McKinney
+* Luca Guidi
+* Salvatore Sanfillipo
+* Chris Wanstrath
+* Damian Janowski
+* Michel Martens
+* Nick Quaranto
+* Pieter Noordhuis
+* Ilya Grigorik
+
+## Contributing
+
+[Fork the project](http://github.com/ezmobius/redis-rb) and send pull requests. You can also ask for help at `#redis-rb` on Freenode.

data/Rakefile

@@ -3,31 +3,7 @@ require 'rake/gempackagetask'
 require 'rake/testtask'
 
 $:.unshift File.join(File.dirname(__FILE__), 'lib')
-require 'redis'
-
-GEM = 'redis'
-GEM_NAME = 'redis'
-GEM_VERSION = Redis::VERSION
-AUTHORS = ['Ezra Zygmuntowicz', 'Taylor Weibley', 'Matthew Clark', 'Brian McKinney', 'Salvatore Sanfilippo', 'Luca Guidi', 'Michel Martens', 'Damian Janowski']
-EMAIL = "ez@engineyard.com"
-HOMEPAGE = "http://github.com/ezmobius/redis-rb"
-SUMMARY = "Ruby client library for Redis, the key value storage server"
-
-spec = Gem::Specification.new do |s|
-  s.name = GEM
-  s.version = GEM_VERSION
-  s.platform = Gem::Platform::RUBY
-  s.has_rdoc = true
-  s.extra_rdoc_files = ["LICENSE"]
-  s.summary = SUMMARY
-  s.description = s.summary
-  s.authors = AUTHORS
-  s.email = EMAIL
-  s.homepage = HOMEPAGE
-  s.require_path = 'lib'
-  s.autorequire = GEM
-  s.files = %w(LICENSE README.markdown Rakefile) + Dir.glob("{lib,tasks,spec}/**/*")
-end
+require 'redis/version'
 
 REDIS_DIR = File.expand_path(File.join("..", "test"), __FILE__)
 REDIS_CNF = File.join(REDIS_DIR, "test.conf")
@@ -40,76 +16,204 @@ task :run => [:start, :test, :stop]
 
 desc "Start the Redis server"
 task :start do
-  unless File.exists?(REDIS_PID)
-    system "redis-server #{REDIS_CNF}"
+  redis_running = \
+  begin
+    File.exists?(REDIS_PID) && Process.kill(0, File.read(REDIS_PID).to_i)
+  rescue Errno::ESRCH
+    FileUtils.rm REDIS_PID
+    false
   end
+
+  system "redis-server #{REDIS_CNF}" unless redis_running
 end
 
 desc "Stop the Redis server"
 task :stop do
   if File.exists?(REDIS_PID)
-    system "kill #{File.read(REDIS_PID)}"
-    system "rm #{REDIS_PID}"
+    Process.kill "INT", File.read(REDIS_PID).to_i
+    FileUtils.rm REDIS_PID
   end
 end
 
-Rake::TestTask.new(:test) do |t|
-  t.pattern = 'test/**/*_test.rb'
+def isolated(&block)
+  pid = fork { yield }
+  Process.wait(pid)
 end
 
-Rake::GemPackageTask.new(spec) do |pkg|
-  pkg.gem_spec = spec
-end
+desc "Run the test suite"
+task :test => ["test:ruby", "test:hiredis", "test:synchrony"]
 
-desc "install the gem locally"
-task :install => [:package] do
-  sh %{gem install pkg/#{GEM}-#{GEM_VERSION}}
-end
+namespace :test do
+  desc "Run tests against the Ruby driver"
+  task :ruby do
+    require "cutest"
 
-desc "create a gemspec file"
-task :gemspec do
-  File.open("#{GEM}.gemspec", "w") do |file|
-    file.puts spec.to_ruby
+    isolated do
+      Cutest.run(Dir["./test/**/*_test.rb"])
+    end
   end
-end
 
-desc "Generate YARDoc"
-task :yardoc do
-  require "yard"
+  desc "Run tests against the hiredis driver"
+  task :hiredis do
+    require "cutest"
+
+    isolated do
+      begin
+        require "redis/connection/hiredis"
+
+        puts
+        puts "Running tests against hiredis v#{Hiredis::VERSION}"
 
-  opts = ["--title", "A Ruby client for Redis"]
+        Cutest.run(Dir["./test/**/*_test.rb"])
+      rescue LoadError
+        puts "Skipping tests against hiredis"
+      end
+    end
+  end
+
+  desc "Run tests against the em-synchrony driver"
+  task :synchrony do
+    require "cutest"
 
-  YARD::CLI::Yardoc.run(*opts)
+    # Synchrony needs 1.9
+    next if RUBY_VERSION < "1.9"
+
+    isolated do
+      begin
+        require "redis/connection/synchrony"
+
+        puts
+        puts "Running tests against em-synchrony"
+
+        threaded_tests = ['./test/thread_safety_test.rb']
+        Cutest.run(Dir['./test/**/*_test.rb'] - threaded_tests)
+      rescue LoadError
+        puts "Skipping tests against em-synchrony"
+      end
+    end
+  end
 end
 
-namespace :commands do
-  def redis_commands
-    $redis_commands ||= begin
-      require "nokogiri"
+task :doc => ["doc:generate", "doc:prepare"]
+
+namespace :doc do
+  task :generate do
+    require "shellwords"
+
+    `rm -rf doc`
+
+    current_branch = `git branch`[/^\* (.*)$/, 1]
+
+    begin
+      tags = `git tag -l`.split("\n").sort.reverse
 
-      doc = Nokogiri::HTML(open("http://code.google.com/p/redis/wiki/CommandReference"))
+      tags.each do |tag|
+        `git checkout -q #{tag} 2>/dev/null`
 
-      commands = {}
+        unless $?.success?
+          $stderr.puts "Need a clean working copy. Please git-stash away."
+          exit 1
+        end
+
+        puts tag
+
+        `mkdir -p doc/#{tag}`
+
+        files = `git ls-tree -r HEAD lib`.split("\n").map do |line|
+          line[/\t(.*)$/, 1]
+        end
+
+        opts = [
+          "--title", "A Ruby client for Redis",
+          "--output", "doc/#{tag}",
+          "--no-cache",
+          "--no-save",
+          "-q",
+          *files
+        ]
 
-      doc.xpath("//ul/li").each do |node|
-        node.at_xpath("./a").text.split("/").each do |name|
-          if name =~ /^[A-Z]+$/
-            commands[name.downcase] = node.at_xpath("./tt").text
+        `yardoc #{Shellwords.shelljoin opts}`
+      end
+    ensure
+      `git checkout -q #{current_branch}`
+    end
+  end
+
+  task :prepare do
+    versions = `git tag -l`.split("\n").grep(/^v/).sort
+    latest_version = versions.last
+
+    File.open("doc/.htaccess", "w") do |file|
+      file.puts "RedirectMatch 302 ^/?$ /#{latest_version}"
+    end
+
+    File.open("doc/robots.txt", "w") do |file|
+      file.puts "User-Agent: *"
+
+      (versions - [latest_version]).each do |version|
+        file.puts "Disallow: /#{version}"
+      end
+    end
+
+    google_analytics = <<-EOS
+    <script type="text/javascript">
+
+      var _gaq = _gaq || [];
+      _gaq.push(['_setAccount', 'UA-11356145-2']);
+      _gaq.push(['_trackPageview']);
+
+      (function() {
+        var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+        ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+        var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+      })();
+
+    </script>
+    EOS
+
+    Dir["doc/**/*.html"].each do |path|
+      lines = IO.readlines(path)
+
+      File.open(path, "w") do |file|
+        lines.each do |line|
+          if line.include?("</head>")
+            file.write(google_analytics)
           end
+
+          file.write(line)
         end
       end
+    end
+  end
+
+  task :deploy do
+    system "rsync --del -avz doc/ redis-rb.keyvalue.org:deploys/redis-rb.keyvalue.org/"
+  end
+end
+
+namespace :commands do
+  def redis_commands
+    $redis_commands ||= doc.keys.map do |key|
+      key.split(" ").first.downcase
+    end.uniq
+  end
 
-      commands
+  def doc
+    $doc ||= begin
+      require "open-uri"
+      require "json"
+
+      JSON.parse(open("https://github.com/antirez/redis-doc/raw/master/commands.json").read)
     end
   end
 
-  task :doc do
-    source = File.read("lib/redis.rb")
+  def document(file)
+    source = File.read(file)
 
-    redis_commands.each do |name, text|
-      source.sub!(/(?:^ *#.*\n)*^( *)def #{name}(\(|$)/) do
-        indent, extra_args = $1, $2
-        comment = "#{indent}# #{text.strip}"
+    doc.each do |name, command|
+      source.sub!(/(?:^ *# .*\n)*(^ *#\n(^ *# .+?\n)*)*^( *)def #{name.downcase}(\(|$)/) do
+        extra_comments, indent, extra_args = $1, $3, $4
+        comment = "#{indent}# #{command["summary"].strip}."
 
         IO.popen("par p#{2 + indent.size} 80", "r+") do |io|
           io.puts comment
@@ -117,32 +221,38 @@ namespace :commands do
           comment = io.read
         end
 
-        "#{comment}#{indent}def #{name}#{extra_args}"
+        "#{comment}#{extra_comments}#{indent}def #{name.downcase}#{extra_args}"
       end
     end
 
-    File.open("lib/redis.rb", "w") { |f| f.write(source) }
+    File.open(file, "w") { |f| f.write(source) }
+  end
+
+  task :doc do
+    document "lib/redis.rb"
+    document "lib/redis/distributed.rb"
   end
 
   task :verify do
     require "redis"
+    require "stringio"
 
-    Dir["test/**/*_test.rb"].each { |f| require f }
+    require "./test/helper"
 
-    log = StringIO.new
+    OPTIONS[:logger] = Logger.new("./tmp/log")
 
-    RedisTest::OPTIONS[:logger] = Logger.new(log)
+    Rake::Task["test:ruby"].invoke
 
     redis = Redis.new
 
-    Test::Unit::AutoRunner.run
-
     report = ["Command", "\033[0mDefined?\033[0m", "\033[0mTested?\033[0m"]
 
     yes, no = "\033[1;32mYes\033[0m", "\033[1;31mNo\033[0m"
 
+    log = File.read("./tmp/log")
+
     redis_commands.sort.each do |name, _|
-      defined, tested = redis.respond_to?(name), log.string[">> #{name.upcase}"]
+      defined, tested = redis.respond_to?(name), log[">> #{name.upcase}"]
 
       next if defined && tested
 

data/TODO.md

@@ -0,0 +1,4 @@
+# 2.3
+
+* Remove the splat from the prototypes of `Client#call`, `Client#call_loop`,
+  `Pipeline#call` and `Subscription#call`.

data/benchmarking/logging.rb

@@ -0,0 +1,62 @@
+# Run with
+#
+#   $ ruby -Ilib benchmarking/logging.rb
+#
+
+begin
+  require "bench"
+rescue LoadError
+  $stderr.puts "`gem install bench` and try again."
+  exit 1
+end
+
+require "redis"
+require "logger"
+
+def log(level, namespace = nil)
+  logger = (namespace || Kernel).const_get(:Logger).new("/dev/null")
+  logger.level = (namespace || Logger).const_get(level)
+  logger
+end
+
+def stress(redis)
+  redis.flushdb
+
+  n = (ARGV.shift || 2000).to_i
+
+  n.times do |i|
+    key = "foo:#{i}"
+    redis.set key, i
+    redis.get key
+  end
+end
+
+default = Redis.new
+
+logging_redises = [
+  Redis.new(:logger => log(:DEBUG)),
+  Redis.new(:logger => log(:INFO)),
+]
+
+begin
+  require "log4r"
+
+  logging_redises += [
+    Redis.new(:logger => log(:DEBUG, Log4r)),
+    Redis.new(:logger => log(:INFO, Log4r)),
+  ]
+rescue LoadError
+  $stderr.puts "Log4r not installed. `gem install log4r` if you want to compare it against Ruby's Logger (spoiler: it's much faster)."
+end
+
+benchmark "Default options (no logger)" do
+  stress(default)
+end
+
+logging_redises.each do |redis|
+  benchmark "#{redis.client.logger.class} on #{Logger::SEV_LABEL[redis.client.logger.level]}" do
+    stress(redis)
+  end
+end
+
+run 10