cql-rb 1.0.0.pre5 → 1.0.0.pre6

data/README.md

@@ -18,72 +18,85 @@ The native transport protocol (sometimes called binary protocol, or CQL protocol
 
 # Quick start
 
-    require 'cql'
+```ruby
+require 'cql'
 
-    client = Cql::Client.connect(host: 'cassandra.example.com')
-    client.use('system')
-    rows = client.execute('SELECT keyspace_name, columnfamily_name FROM schema_columnfamilies')
-    rows.each do |row|
-      puts "The keyspace #{row['keyspace_name']} has a table called #{row['columnfamily_name']}""
-    end
+client = Cql::Client.connect(host: 'cassandra.example.com')
+client.use('system')
+rows = client.execute('SELECT keyspace_name, columnfamily_name FROM schema_columnfamilies')
+rows.each do |row|
+  puts "The keyspace #{row['keyspace_name']} has a table called #{row['columnfamily_name']}""
+end
+```
 
 when you're done you can call `#close` to disconnect from Cassandra. You can connect to multiple Cassandra nodes by passing multiple comma separated host names to the `:host` option.
 
 ## Changing keyspaces
 
-    client.use('measurements')
+```ruby
+client.use('measurements')
+```
 
 or using CQL:
 
-    client.execute('USE measurements')
+```ruby
+client.execute('USE measurements')
+```
 
 ## Running queries
 
 You run CQL statements by passing them to `#execute`. Most statements don't have any result and the call will return nil.
 
-    client.execute("INSERT INTO events (id, date, description) VALUES (23462, '2013-02-24T10:14:23+0000', 'Rang bell, ate food')")
-
-    client.execute("UPDATE events SET description = 'Oh, my' WHERE id = 13126")
+```ruby
+client.execute("INSERT INTO events (id, date, description) VALUES (23462, '2013-02-24T10:14:23+0000', 'Rang bell, ate food')")
 
+client.execute("UPDATE events SET description = 'Oh, my' WHERE id = 13126")
+```
 
 If the CQL statement passed to `#execute` returns a result (e.g. it's a `SELECT` statement) the call returns an enumerable of rows:
 
-    rows = client.execute('SELECT date, description FROM events')
-    rows.each do |row|
-      row.each do |key, value|
-        puts "#{key} = #{value}"
-      end
-    end
+```ruby
+rows = client.execute('SELECT date, description FROM events')
+rows.each do |row|
+  row.each do |key, value|
+    puts "#{key} = #{value}"
+  end
+end
+```
 
 The enumerable also has an accessor called `metadata` which returns a description of the rows and columns:
 
-    rows = client.execute('SELECT date, description FROM events'
-    rows.metadata['date'].type # => :date
+```ruby
+rows = client.execute('SELECT date, description FROM events'
+rows.metadata['date'].type # => :date
+```
 
 ## Creating keyspaces and tables
 
 There is no special facility for creating keyspaces and tables, they are created by executing CQL:
 
-    keyspace_definition = <<-KSDEF
-      CREATE KEYSPACE measurements
-      WITH replication = {
-        'class': 'SimpleStrategy',
-        'replication_factor': 3
-      }
-    KSDEF
-
-    table_definition = <<- TABLEDEF
-      CREATE TABLE events (
-        id INT,
-        date DATE,
-        comment VARCHAR,
-        PRIMARY KEY (id)
-      )
-    TABLEDEF
-
-    client.execute(keyspace_definition)
-    client.use(measurements)
-    client.execute(table_definition)
+```ruby
+keyspace_definition = <<-KSDEF
+  CREATE KEYSPACE measurements
+  WITH replication = {
+    'class': 'SimpleStrategy',
+    'replication_factor': 3
+  }
+KSDEF
+
+table_definition = <<-TABLEDEF
+  CREATE TABLE events (
+    id INT,
+    date DATE,
+    comment VARCHAR,
+    PRIMARY KEY (id)
+  )
+TABLEDEF
+
+client.execute(keyspace_definition)
+client.use(measurements)
+client.execute(table_definition)
+```
 
 You can also `ALTER` keyspaces and tables.
 
@@ -91,8 +104,10 @@ You can also `ALTER` keyspaces and tables.
 
 The driver supports prepared statements. Use `#prepare` to create a statement object, and then call `#execute` on that object to run a statement. You must supply values for all bound parameters when you call `#execute`.
 
-    statement = client.prepare('SELECT date, description FROM events WHERE id = ?')
-    rows = statement.execute(1235)
+```ruby
+statement = client.prepare('SELECT date, description FROM events WHERE id = ?')
+rows = statement.execute(1235)
+```
 
 A prepared statement can be run many times, but the CQL parsing will only be done once. Use prepared statements for queries you run over and over again.
 
@@ -104,7 +119,9 @@ At this time prepared statements are local to a single connection. Even if you c
 
 The `#execute` method supports setting the desired consistency level for the statement:
 
-    client.execute('SELECT * FROM peers', :local_quorum)
+```ruby
+client.execute('SELECT * FROM peers', :local_quorum)
+```
 
 The possible values are: 
 
@@ -129,11 +146,9 @@ Read more about CQL3 in the [CQL3 syntax documentation](https://github.com/apach
 
 # Known bugs & limitations
 
-* If any connection raises an error the whole IO reactor shuts down.
+* There are still edge cases around connection errors, and there is no automatic reconnection.
 * JRuby 1.6.8 is not supported, although it should be. The only known issue is that connection failures aren't handled gracefully.
 * No automatic peer discovery.
-* You can't specify consistency level when executing prepared statements.
-* Authentication is not supported.
 * Compression is not supported.
 * Large results are buffered in memory until the whole response has been loaded, the protocol makes it possible to start to deliver rows to the client code as soon as the metadata is loaded, but this is not supported yet.
 * There is no cluster introspection utilities (like the `DESCRIBE` commands in `cqlsh`).

data/lib/cql.rb

@@ -5,6 +5,7 @@ module Cql
 end
 
 require 'cql/uuid'
+require 'cql/byte_buffer'
 require 'cql/future'
 require 'cql/io'
 require 'cql/protocol'

data/lib/cql/byte_buffer.rb

@@ -0,0 +1,138 @@
+# encoding: utf-8
+
+module Cql
+  class ByteBuffer
+    def initialize(initial_bytes='')
+      @read_buffer = ''
+      @write_buffer = ''
+      @offset = 0
+      @length = 0
+      append(initial_bytes) unless initial_bytes.empty?
+    end
+
+    attr_reader :length
+    alias_method :size, :length
+    alias_method :bytesize, :length
+
+    def empty?
+      length == 0
+    end
+
+    def append(bytes)
+      bytes = bytes.to_s
+      unless bytes.ascii_only?
+        bytes = bytes.dup.force_encoding(::Encoding::BINARY)
+      end
+      retag = @write_buffer.empty?
+      @write_buffer << bytes
+      @write_buffer.force_encoding(::Encoding::BINARY) if retag
+      @length += bytes.bytesize
+      self
+    end
+    alias_method :<<, :append
+
+    def discard(n)
+      raise RangeError, "#{n} bytes to discard but only #{@length} available" if @length < n
+      @offset += n
+      @length -= n
+      self
+    end
+
+    def read(n)
+      raise RangeError, "#{n} bytes required but only #{@length} available" if @length < n
+      if @offset >= @read_buffer.bytesize
+        swap_buffers
+      end
+      if @offset + n > @read_buffer.bytesize
+        s = read(@read_buffer.bytesize - @offset)
+        s << read(n - s.bytesize)
+        s
+      else
+        s = @read_buffer[@offset, n]
+        @offset += n
+        @length -= n
+        s
+      end
+    end
+
+    def read_int
+      raise RangeError, "4 bytes required to read an int, but only #{@length} available" if @length < 4
+      if @offset >= @read_buffer.bytesize
+        swap_buffers
+      end
+      if @read_buffer.bytesize >= @offset + 4
+        i0 = @read_buffer.getbyte(@offset + 0)
+        i1 = @read_buffer.getbyte(@offset + 1)
+        i2 = @read_buffer.getbyte(@offset + 2)
+        i3 = @read_buffer.getbyte(@offset + 3)
+        @offset += 4
+        @length -= 4
+      else
+        i0 = read_byte
+        i1 = read_byte
+        i2 = read_byte
+        i3 = read_byte
+      end
+      (i0 << 24) | (i1 << 16) | (i2 << 8) | i3
+    end
+
+    def read_short
+      raise RangeError, "2 bytes required to read a short, but only #{@length} available" if @length < 2
+      if @offset >= @read_buffer.bytesize
+        swap_buffers
+      end
+      if @read_buffer.bytesize >= @offset + 2
+        i0 = @read_buffer.getbyte(@offset + 0)
+        i1 = @read_buffer.getbyte(@offset + 1)
+        @offset += 2
+        @length -= 2
+      else
+        i0 = read_byte
+        i1 = read_byte
+      end
+      (i0 << 8) | i1
+    end
+
+    def read_byte(signed=false)
+      raise RangeError, "No bytes available to read byte" if empty?
+      if @offset >= @read_buffer.bytesize
+        swap_buffers
+      end
+      b = @read_buffer.getbyte(@offset)
+      b = (b & 0x7f) - (b & 0x80) if signed
+      @offset += 1
+      @length -= 1
+      b
+    end
+
+    def eql?(other)
+      self.to_str.eql?(other.to_str)
+    end
+    alias_method :==, :eql?
+
+    def hash
+      to_str.hash
+    end
+
+    def dup
+      self.class.new(to_str)
+    end
+
+    def to_str
+      (@read_buffer + @write_buffer)[@offset, @length]
+    end
+    alias_method :to_s, :to_str
+
+    def inspect
+      %(#<#{self.class.name}: #{to_str.inspect}>)
+    end
+
+    private
+
+    def swap_buffers
+      @offset -= @read_buffer.bytesize
+      @read_buffer = @write_buffer
+      @write_buffer = ''
+    end
+  end
+end

data/lib/cql/future.rb

@@ -1,5 +1,8 @@
 # encoding: utf-8
 
+require 'thread'
+
+
 module Cql
   FutureError = Class.new(CqlError)
 

data/lib/cql/io/node_connection.rb

@@ -1,7 +1,6 @@
 # encoding: utf-8
 
 require 'socket'
-require 'resolv-replace'
 
 
 module Cql
@@ -13,8 +12,8 @@ module Cql
         @connected_future = Future.new
         @io = nil
         @addrinfo = nil
-        @write_buffer = ''
-        @read_buffer = ''
+        @write_buffer = ByteBuffer.new
+        @read_buffer = ByteBuffer.new
         @current_frame = Protocol::ResponseFrame.new(@read_buffer)
         @response_tasks = [nil] * 128
         @event_listeners = Hash.new { |h, k| h[k] = [] }
@@ -29,7 +28,7 @@ module Cql
           @io = Socket.new(address_family, socket_type, 0)
           @io.connect_nonblock(@sockaddr)
         rescue Errno::EINPROGRESS
-          # ok
+          # NOTE not connected yet, this is expected
         rescue SystemCallError, SocketError => e
           fail_connection!(e)
         end
@@ -72,22 +71,6 @@ module Cql
         @io && (!@write_buffer.empty? || connecting?)
       end
 
-      def perform_request(request, future)
-        stream_id = next_stream_id
-        Protocol::RequestFrame.new(request, stream_id).write(@write_buffer)
-        @response_tasks[stream_id] = future
-      rescue => e
-        case e
-        when CqlError
-          error = e
-        else
-          error = IoError.new(e.message)
-          error.set_backtrace(e.backtrace)
-        end
-        @response_tasks.delete(stream_id)
-        future.fail!(error)
-      end
-
       def handle_read
         new_bytes = @io.read_nonblock(2**16)
         @current_frame << new_bytes
@@ -107,11 +90,30 @@ module Cql
         force_close(e)
       end
 
+      def perform_request(request, future)
+        stream_id = next_stream_id
+        Protocol::RequestFrame.new(request, stream_id).write(@write_buffer)
+        @response_tasks[stream_id] = future
+      rescue => e
+        case e
+        when CqlError
+          error = e
+        else
+          error = IoError.new(e.message)
+          error.set_backtrace(e.backtrace)
+        end
+        @response_tasks.delete(stream_id)
+        future.fail!(error)
+      end
+
       def handle_write
-        succeed_connection! if connecting?
-        if !@write_buffer.empty?
-          bytes_written = @io.write_nonblock(@write_buffer)
-          @write_buffer.slice!(0, bytes_written)
+        if connecting?
+          handle_connecting
+        else
+          if !@write_buffer.empty?
+            bytes_written = @io.write_nonblock(@write_buffer)
+            @write_buffer.discard(bytes_written)
+          end
         end
       rescue => e
         force_close(e)
@@ -124,8 +126,9 @@ module Cql
           @io.connect_nonblock(@sockaddr)
           succeed_connection!
         end
+      rescue Errno::EALREADY, Errno::EINPROGRESS
+        # NOTE still not connected
       rescue Errno::EISCONN
-        # ok
         succeed_connection!
       rescue SystemCallError, SocketError => e
         fail_connection!(e)
@@ -136,7 +139,7 @@ module Cql
           begin
             @io.close
           rescue SystemCallError
-            # nothing to do, it wasn't open
+            # NOTE nothing to do, it wasn't open
           end
           if connecting?
             succeed_connection!

data/lib/cql/protocol.rb

@@ -26,7 +26,7 @@ module Cql
 
       BYTES_FORMAT = 'C*'.freeze
       TWO_INTS_FORMAT = 'NN'.freeze
-      HEADER_FORMAT = 'c4N'.freeze
+      HEADER_FORMAT = 'c4'.freeze
     end
 
     module Constants

data/lib/cql/protocol/decoding.rb

@@ -6,14 +6,13 @@ module Cql
       extend self
 
       def read_byte!(buffer)
-        raise DecodingError, 'No byte available to decode' if buffer.empty?
-        b = buffer.slice!(0, 1)
-        b.getbyte(0)
+        buffer.read_byte
+      rescue RangeError => e
+        raise DecodingError, e.message, e.backtrace
       end
 
       def read_varint!(buffer, length=buffer.length, signed=true)
-        raise DecodingError, "Length #{length} specifed but only #{buffer.bytesize} bytes given" if buffer.bytesize < length
-        bytes = buffer.slice!(0, length)
+        bytes = buffer.read(length)
         n = 0
         bytes.each_byte do |b|
           n = (n << 8) | b
@@ -22,62 +21,72 @@ module Cql
           n -= 2**(bytes.length * 8)
         end
         n
+      rescue RangeError => e
+        raise DecodingError, e.message, e.backtrace
       end
 
       def read_decimal!(buffer, length=buffer.length)
-        raise DecodingError, "Length #{length} specifed but only #{buffer.bytesize} bytes given" if buffer.bytesize < length
         size = read_int!(buffer)
-        number_bytes = buffer.slice!(0, length - 4)
-        number_string = read_varint!(number_bytes).to_s
+        number_string = read_varint!(buffer, length - 4).to_s
         fraction_string = number_string[0, number_string.length - size] << DECIMAL_POINT << number_string[number_string.length - size, number_string.length]
         BigDecimal.new(fraction_string)
+      rescue RangeError => e
+        raise DecodingError, e.message, e.backtrace
       end
 
       def read_long!(buffer)
-        raise DecodingError, "Need eight bytes to decode long, only #{buffer.bytesize} bytes given" if buffer.bytesize < 8
-        top, bottom = buffer.slice!(0, 8).unpack(Formats::TWO_INTS_FORMAT)
+        top, bottom = buffer.read(8).unpack(Formats::TWO_INTS_FORMAT)
         (top << 32) | bottom
+      rescue RangeError => e
+        raise DecodingError, e.message, e.backtrace
       end
 
       def read_double!(buffer)
-        raise DecodingError, "Need eight bytes to decode double, only #{buffer.bytesize} bytes given" if buffer.bytesize < 8
-        buffer.slice!(0, 8).unpack(Formats::DOUBLE_FORMAT).first
+        buffer.read(8).unpack(Formats::DOUBLE_FORMAT).first
+      rescue RangeError => e
+        raise DecodingError, "Not enough bytes available to decode a double: #{e.message}", e.backtrace
       end
 
       def read_float!(buffer)
-        raise DecodingError, "Need four bytes to decode float, only #{buffer.bytesize} bytes given" if buffer.bytesize < 4
-        buffer.slice!(0, 4).unpack(Formats::FLOAT_FORMAT).first
+        buffer.read(4).unpack(Formats::FLOAT_FORMAT).first
+      rescue RangeError => e
+        raise DecodingError, "Not enough bytes available to decode a float: #{e.message}", e.backtrace
       end
 
       def read_int!(buffer)
-        raise DecodingError, "Need four bytes to decode an int, only #{buffer.bytesize} bytes given" if buffer.bytesize < 4
-        buffer.slice!(0, 4).unpack(Formats::INT_FORMAT).first
+        buffer.read_int
+      rescue RangeError => e
+        raise DecodingError, "Not enough bytes available to decode an int: #{e.message}", e.backtrace
       end
 
       def read_short!(buffer)
-        raise DecodingError, "Need two bytes to decode a short, only #{buffer.bytesize} bytes given" if buffer.bytesize < 2
-        buffer.slice!(0, 2).unpack(Formats::SHORT_FORMAT).first
+        buffer.read_short
+      rescue RangeError => e
+        raise DecodingError, "Not enough bytes available to decode a short: #{e.message}", e.backtrace
       end
 
       def read_string!(buffer)
         length = read_short!(buffer)
-        raise DecodingError, "String length is #{length}, but only #{buffer.bytesize} bytes given" if buffer.bytesize < length
-        string = buffer.slice!(0, length)
+        string = buffer.read(length)
         string.force_encoding(::Encoding::UTF_8)
         string
+      rescue RangeError => e
+        raise DecodingError, "Not enough bytes available to decode a string: #{e.message}", e.backtrace
       end
 
       def read_long_string!(buffer)
         length = read_int!(buffer)
-        raise DecodingError, "String length is #{length}, but only #{buffer.bytesize} bytes given" if buffer.bytesize < length
-        string = buffer.slice!(0, length)
+        string = buffer.read(length)
         string.force_encoding(::Encoding::UTF_8)
         string
+      rescue RangeError => e
+        raise DecodingError, "Not enough bytes available to decode a long string: #{e.message}", e.backtrace
       end
 
       def read_uuid!(buffer)
-        raise DecodingError, "UUID requires 16 bytes, but only #{buffer.bytesize} bytes given" if buffer.bytesize < 16
         Uuid.new(read_varint!(buffer, 16, false))
+      rescue RangeError => e
+        raise DecodingError, "Not enough bytes available to decode a UUID: #{e.message}", e.backtrace
       end
 
       def read_string_list!(buffer)
@@ -90,19 +99,17 @@ module Cql
       def read_bytes!(buffer)
         size = read_int!(buffer)
         return nil if size & 0x80000000 == 0x80000000
-        raise DecodingError, "Byte array length is #{size}, but only #{buffer.bytesize} bytes given" if buffer.bytesize < size
-        bytes = buffer.slice!(0, size)
-        bytes.force_encoding(::Encoding::BINARY)
-        bytes
+        buffer.read(size)
+      rescue RangeError => e
+        raise DecodingError, "Not enough bytes available to decode a bytes: #{e.message}", e.backtrace
       end
 
       def read_short_bytes!(buffer)
         size = read_short!(buffer)
         return nil if size & 0x8000 == 0x8000
-        raise DecodingError, "Byte array length is #{size}, but only #{buffer.bytesize} bytes given" if buffer.bytesize < size
-        bytes = buffer.slice!(0, size)
-        bytes.force_encoding(::Encoding::BINARY)
-        bytes
+        buffer.read(size)
+      rescue RangeError => e
+        raise DecodingError, "Not enough bytes available to decode a short bytes: #{e.message}", e.backtrace
       end
 
       def read_option!(buffer)
@@ -116,10 +123,11 @@ module Cql
 
       def read_inet!(buffer)
         size = read_byte!(buffer)
-        raise DecodingError, "Inet requires #{size} bytes, but only #{buffer.bytesize} bytes given" if buffer.bytesize < size
-        ip_addr = IPAddr.new_ntoh(buffer.slice!(0, size))
+        ip_addr = IPAddr.new_ntoh(buffer.read(size))
         port = read_int!(buffer)
         [ip_addr, port]
+      rescue RangeError => e
+        raise DecodingError, "Not enough bytes available to decode an INET: #{e.message}", e.backtrace
       end
 
       def read_consistency!(buffer)