cql-rb 1.2.0.pre0 → 1.2.0.pre1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 5c27b19f205bb59c5f8fe37faa6259064d85c761
-  data.tar.gz: 74615d268c49624381e8c8088fb173834e3c0313
+  metadata.gz: c7cc1fd494fd2616c7b129e191cf7440ede77f05
+  data.tar.gz: c82c1161c92f0bb69c156a4a6abcba240dc76395
 SHA512:
-  metadata.gz: 7e8bd6f3e31b0b6e201baca992267b12502e5b78b71ef64378d4f52b791a8a2036c049650c5c4e7f17a0a1b4814d942a423c802742e5173071430d59d1b811ab
-  data.tar.gz: 86539fa464d45531f30aea8198d70a726d20d6bf3247d2b25fe76d1e49a37b568c465772059af4adfd637a7371406f7410389ae824d592cd6b18c43bafcfff56
+  metadata.gz: 1acb48539528d929819760af28715539fd372564b6cd158e9d3fa858a0da9a76e8c9abb06ef076b21b3198ec19a241a97a5d66520c1d6799b850513fcec5e19d
+  data.tar.gz: eaefa65e49c8769b8ec6398bc20db5ab1b808162eb8e796cf4a1286de7f65c808ab9ec3ba4eacd88387a132c3b95d1967492b5a52145419b37a1c2b6ff4323ba

data/README.md

@@ -3,6 +3,8 @@
 [![Build Status](https://travis-ci.org/iconara/cql-rb.png?branch=master)](https://travis-ci.org/iconara/cql-rb)
 [![Coverage Status](https://coveralls.io/repos/iconara/cql-rb/badge.png)](https://coveralls.io/r/iconara/cql-rb)
 
+_Please note that this is the readme for the development version and that some features described here might not yet have been released._
+
 # Requirements
 
 Cassandra 1.2 or later with the native transport protocol turned on and a modern Ruby. It's tested continuously in Travis with Ruby 1.9.3, 2.0, JRuby 1.7 and Rubinius 2.0.
@@ -42,6 +44,8 @@ The full [API documentation](http://rubydoc.info/gems/cql-rb/frames) is availabl
 
 ## Changing keyspaces
 
+You can specify a keyspace to change to immediately after connection by passing the `:keyspace` option to `Client.connect`, but you can also use the `#use` method, or `#execute`:
+
 ```ruby
 client.use('measurements')
 ```
@@ -54,7 +58,7 @@ client.execute('USE measurements')
 
 ## Running queries
 
-You run CQL statements by passing them to `#execute`. Most statements don't have any result and the call will return nil.
+You run CQL statements by passing them to `#execute`.
 
 ```ruby
 client.execute("INSERT INTO events (id, date, description) VALUES (23462, '2013-02-24T10:14:23+0000', 'Rang bell, ate food')")
@@ -183,7 +187,7 @@ Read more about CQL3 in the [CQL3 syntax documentation](https://github.com/apach
 
 # Cassandra 2.0
 
-Cassandra 2.0 introduced a new version of the native protocol with some new features like argument interpolation in non-prepared statements, result set cursors, a new authentication mechanism and the `SERIAL` consistency. These features are not yet supported, but the driver will work with Cassandra 2.0 using the earlier protocol.
+Cassandra 2.0 introduced a new version of the native protocol with some new features like argument interpolation in non-prepared statements, result set cursors, a new authentication mechanism and the `SERIAL` consistency. These features are not yet supported, but the driver will work with Cassandra 2.0 using the earlier protocol. Support for all of the features of the new protocol is being worked on. If there is a particular feature that you would want to see implemented, open an issue and describe your use case. This helps with prioritizing what should be implemented first.
 
 # Troubleshooting
 
@@ -231,6 +235,8 @@ There's a known issue with collections that get too big. The protocol uses a sho
 
 Please open an issue. It should be working, but it's hard to write tests for, so there may be edge cases that aren't covered.
 
+If you are using DataStax Enterprise and authentication it is unfortunately not supported yet. DataStax backported the authentication from Cassandra 2.0 into DSE, even though it only uses Cassandra 1.2. The authentication mechanism in Cassandra 2.0 is very different and you will have to wait until support for Cassandra 2.0 is added to cql-rb before it will work with DSE.
+
 ## I'm connecting to port 9160 and it doesn't work
 
 Port 9160 is the old Thrift interface, the binary protocol runs on 9042. This is also the default port for cql-rb, so unless you've changed the port in `cassandra.yaml`, don't override the port.
@@ -239,6 +245,34 @@ Port 9160 is the old Thrift interface, the binary protocol runs on 9042. This is
 
 Open an issue and someone will try to help you out. Please include the gem version, Casandra version and Ruby version, and explain as much about what you're doing as you can, preferably the smallest piece of code that reliably triggers the problem. The more information you give, the better the chances you will get help.
 
+# Performance tips
+
+## Use prepared statements
+
+When you use prepared statements you don't have to smash strings together to create a chunk of CQL to send to the server. Avoiding creating many and large strings in Ruby can be a performance gain in itself. Not sending the query every time, but only the actual data also decreases the traffic over the network, and it decreases the time it takes for the server to handle the request since it doesn't have to parse CQL. Prepared statements are also very convenient, so there is really no reason not to use them.
+
+## Use JRuby
+
+If you want to be serious about Ruby performance you have to use JRuby. The cql-rb client is completely thread safe, and the CQL protocol is pipelined by design so you can spin up as many threads as you like and your requests per second will scale more or less linearly (up to what your cores, network and Cassandra cluster can deliver, obviously).
+
+Applications using cql-rb and JRuby can do over 10,000 write requests per second from a single EC2 m1.large if tuned correctly.
+
+## Try batching
+
+Batching in Cassandra isn't always as good as in other (non-distributed) databases. Since rows are distributed accross the cluster the coordinator node must still send the individual pieces of a batch to other nodes, and you could have done that yourself instead. Batches also mean that in most cases you need to smash strings together to create a big CQL string, so you increase the size of your requests, using up more bandwidth and making the server have to do more CQL parsing. Prepared statements are almost always a better choice for performance.
+
+Cassandra 2.0 introduced a new form of batches where you can send a batch of prepared statement executions as one request, when support for that arrives in cql-rb, this advice should be reconsidered.
+
+## Try compression
+
+If your requests or responses are big, compression can help decrease the amound of traffic over the network, which is often a good thing. If your requests and responses are small, compression often doesn't do anything. You should benchmark and see what works for you. The Snappy compressor that comes with cql-rb uses very little CPU, so most of the time it doesn't hurt to leave it on.
+
+In read-heavy applications requests are often small, and need no compression, but responses can be big. In these situations you can modify the compressor used to turn off compression for requests completely. The Snappy compressor that comes with cql-rb will not compress frames less than 64 bytes, for example, and you can change this threshold when you create the compressor.
+
+# Try experimental features
+
+To get maximum performance you can't wait for a request to complete before sending the next. At it's core cql-rb embraces this completely and uses non-blocking IO and a completely asynchronous model for the request processing. The synchronous API that you use is just a thin façade on top that exists for convenience. If you need to scale to thousands of requests per second, have a look at the client code and look at the asynchronous core, it works very much like the public API, _but using it they should be considererd **experimental**_. Experimental in this context does not mean buggy, it is the core of cql-rb after all, but it means that you cannot rely on it being backwards compatible.
+
 # Changelog & versioning
 
 Check out the [releases on GitHub](https://github.com/iconara/cql-rb/releases). Version numbering follows the [semantic versioning](http://semver.org/) scheme.
@@ -252,8 +286,8 @@ Prereleases will be stable, in the sense that they will have finished and proper
 * JRuby 1.6 is not officially supported, although 1.6.8 should work, if you're stuck in JRuby 1.6.8 try and see if it works for you.
 * Windows is not supported (there is experimental support in the [`windows` branch](https://github.com/iconara/cql-rb/tree/windows_support)).
 * Large results are buffered in memory until the whole response has been loaded, the protocol makes it possible to start to deliver rows to the client code as soon as the metadata is loaded, but this is not supported yet.
-* There is no cluster introspection utilities (like the `DESCRIBE` commands in `cqlsh`).
-* New features in v2 of the protocol are not supported
+* There is no cluster introspection utilities (like the `DESCRIBE` commands in `cqlsh`) -- but it's not clear whether that will ever be added, it would be useful, but it is also something that another gem could add on top.
+* New features in v2 of the protocol are not supported -- this is planned and in progress
 
 Also check out the [issues](https://github.com/iconara/cql-rb/issues) for open bugs.
 
@@ -263,7 +297,7 @@ Also check out the [issues](https://github.com/iconara/cql-rb/issues) for open b
 
 # Copyright
 
-Copyright 2013 Theo Hultberg/Iconara and contributors
+Copyright 2013–2014 Theo Hultberg/Iconara and contributors
 
 _Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License You may obtain a copy of the License at_
 

data/lib/cql/client/asynchronous_prepared_statement.rb

@@ -47,7 +47,8 @@ module Cql
             @raw_metadata = response.metadata
             @metadata = ResultMetadata.new(@raw_metadata)
           end
-          @logger.debug('Statement prepared on new connection')
+          hex_id = response.id.each_byte.map { |x| x.to_s(16).rjust(2, '0') }.join('')
+          @logger.debug('Statement %s prepared on node %s (%s:%d)' % [hex_id, connection[:host_id].to_s, connection.host, connection.port])
         end
         f.map { self }
       end

data/lib/cql/protocol/responses/prepared_result_response.rb

@@ -32,7 +32,8 @@ module Cql
       end
 
       def to_s
-        %(RESULT PREPARED #{id.each_byte.map { |x| x.to_s(16) }.join('')} #@metadata)
+        hex_id = @id.each_byte.map { |x| x.to_s(16).rjust(2, '0') }.join('')
+        %(RESULT PREPARED #{hex_id} #@metadata)
       end
 
       private

data/lib/cql/version.rb

@@ -1,5 +1,5 @@
 # encoding: utf-8
 
 module Cql
-  VERSION = '1.2.0.pre0'.freeze
+  VERSION = '1.2.0.pre1'.freeze
 end

data/spec/cql/client/asynchronous_prepared_statement_spec.rb

@@ -44,7 +44,7 @@ module Cql
       def handle_request(connection, request, timeout)
         case request
         when Protocol::PrepareRequest
-          statement_id = [rand(2**31)].pack('c*')
+          statement_id = Array.new(16) { [rand(255)].pack('c') }.join('')
           connection[:last_prepared_statement_id] = statement_id
           Protocol::PreparedResultResponse.new(statement_id, raw_metadata, nil)
         when Protocol::ExecuteRequest
@@ -170,10 +170,11 @@ module Cql
             execute_request.values.should == [11, 'hello']
           end
 
-          it 'logs a message' do
+          it 'logs a message with the host ID, host address and its own ID' do
+            new_connection[:host_id] = Uuid.new('a4a70900-24e1-11df-8924-001ff3591711')
             logger.stub(:debug)
             statement.execute(11, 'hello')
-            logger.should have_received(:debug).with(/Statement prepared/).once
+            logger.should have_received(:debug).with(/Statement [0-9a-f]{32} prepared on node a4a70900-24e1-11df-8924-001ff3591711 \(h3.example.com:1234\)/).once
           end
         end
 

data/spec/cql/protocol/responses/prepared_result_response_spec.rb

@@ -29,7 +29,7 @@ module Cql
 
       describe '#to_s' do
         it 'returns a string with the ID and metadata' do
-          response = described_class.new("\xCAH\x7F\x1Ez\x82\xD2<N\x8A\xF35Qq\xA5/", [['ks', 'tbl', 'col', :varchar]], nil)
+          response = described_class.new("\xCAH\x7F\x1Ez\x82\xD2<N\x8A\xF35Qq\x00/", [['ks', 'tbl', 'col', :varchar]], nil)
           response.to_s.should match(/^RESULT PREPARED [0-9a-f]{32} \[\["ks", "tbl", "col", :varchar\]\]$/)
         end
       end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: cql-rb
 version: !ruby/object:Gem::Version
-  version: 1.2.0.pre0
+  version: 1.2.0.pre1
 platform: ruby
 authors:
 - Theo Hultberg
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2014-01-04 00:00:00.000000000 Z
+date: 2014-01-14 00:00:00.000000000 Z
 dependencies: []
 description: A pure Ruby CQL3 driver for Cassandra
 email: