net_tcp_client 1.0.2 → 2.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 1955b43fbf000cce65133a3015187ae3cb5e0c58
-  data.tar.gz: 9a59c68c30a5fd7f544ff28ce45e348b185ee55e
+  metadata.gz: 9d6906da5ebe4657343ec1175664ee2f42b359eb
+  data.tar.gz: c816880d83f7d557390b00cbdfb5ed7c6a308512
 SHA512:
-  metadata.gz: b87f0dfdf9a5662411bbde05c2c9ddb04ee74f0d7bd915a1d67db404165fd5398bcc0c4df28043846038571586ed2d842446da95170f437790e5c31e8e807840
-  data.tar.gz: e3e3810825b828d2e474f6785f047f0ca64a301ce628fc5cfc0e696e22315b477a5413639bdc4b51634032a77d9a8b54c2c4ed6134db6fc92066cd96acce508d
+  metadata.gz: 5d1aafb91baf376f68f291d5d18bc9d5569522cc2b093baf337f525b8060d3e632f6799ee64d8da12fc4db533f01a29059ec63144403da29e0f9884cd0f1a623
+  data.tar.gz: 32605f07229206ee29216ccfb5a2ecc74fe9c8c6d32459b7ec973fcd31f4541ab4ca710f148e6804254838265af364737449265908407f227100a96b761950af

data/README.md

@@ -1,43 +1,184 @@
 # net_tcp_client
-![](https://img.shields.io/gem/v/net_tcp_client.svg) ![](https://img.shields.io/travis/rocketjob/net_tcp_client.svg) ![](https://img.shields.io/gem/dt/net_tcp_client.svg) ![](https://img.shields.io/badge/status-production%20ready-blue.svg)
+[![Gem Version](https://img.shields.io/gem/v/net_tcp_client.svg)](https://rubygems.org/gems/net_tcp_client) [![Build Status](https://travis-ci.org/rocketjob/net_tcp_client.svg?branch=master)](https://travis-ci.org/rocketjob/net_tcp_client) [![Downloads](https://img.shields.io/gem/dt/net_tcp_client.svg)](https://rubygems.org/gems/net_tcp_client) [![License](https://img.shields.io/badge/license-Apache%202.0-brightgreen.svg)](http://opensource.org/licenses/Apache-2.0) ![](https://img.shields.io/badge/status-Production%20Ready-blue.svg) [![Gitter chat](https://img.shields.io/badge/IRC%20(gitter)-Support-brightgreen.svg)](https://gitter.im/rocketjob/support)
 
-Net::TCPClient is a TCP Socket Client with built-in timeouts, retries, and logging
+Net::TCPClient is a TCP Socket Client with automated failover, load balancing, retries and built-in timeouts.
 
 * http://github.com/rocketjob/net_tcp_client
 
 ## Introduction
 
-Net::TCPClient implements resilience features that many developers wish was
+Net::TCPClient implements high availability and resilience features that many developers wish was
 already included in the standard Ruby libraries.
 
-With so many "client" libraries to servers such us memcache, MongoDB, Redis, etc.
-their focus is on the communication formats and messaging interactions. As a result
-adding resilience is usually an after thought.
-
-More importantly the way that each client implements connection failure handling
-varies dramatically. The purpose of this library is to extract the best
-of all the socket error handling out there and create a consistent way of dealing
-with connection failures.
-
 Another important feature is that the _connect_ and _read_ API's use timeout's to
 prevent a network issue from "hanging" the client program.
 
-## Net::TCPClient API
+## Features
+
+* Automated failover to another server.
+* Load balancing across multiple servers.
+* SSL and non-ssl connections.
+* Connect Timeout.
+* Read Timeout.
+* Write Timeout.
+* Fails over / load balances across all servers under a single DNS entry.
+* Logging.
+    * Optional trace level logging of all data sent or received.
+* Uses non blocking timeouts, instead of using threads such as used by the Timeout class.
+* Additional exceptions to distinguish between connection failures and timeouts.
+* Handshake callbacks.
+    * After a new connection has been established callbacks can be used
+      for handshakes such as authentication before data is sent.
+
+### Example
+
+~~~ruby
+require 'net/tcp_client'
+
+Net::TCPClient.connect(server: 'mydomain:3300') do |client|
+  client.send('Update the database')
+  response = client.read(20)
+  puts "Received: #{response}"
+end
+~~~
+
+Enable SSL encryption:
+
+~~~ruby
+require 'net/tcp_client'
 
-Net::TCPClient is a drop in replacement for TCPSocket when used as a client.
+Net::TCPClient.connect(server: 'mydomain:3300', ssl: true) do |client|
+  client.send('Update the database')
+  response = client.read(20)
+  puts "Received: #{response}"
+end
+~~~
 
-The initializer is the only deviation since it accepts several new options
-that support automatic failover, re-connect and messaging retries.
+## High Availability
 
-## Example
+Net::TCPClient automatically tries each server in turn, should it fail to connect, or
+if the connection is lost the next server is tried immediately.
 
-Connect to a server at address `localhost`, and on port `3300`.
+Net::TCPClient detects DNS entries that have multiple IP Addresses associated with them and
+adds each of the ip addresses for the single DNS name to the list of servers to try to connect to.
 
-Specify a custom retry interval and retry counts during connection.
+If a server is unavailable, cannot connect, or the connection is lost, the next server is immediately
+tried. Once all servers have been exhausted, it will keep trying to connect, starting with the
+first server again.
 
-```ruby
-require 'net/tcp_client'
+When a connection is first established, and every time a connection is lost, Net::TCPClient
+uses connection policies to determine which server to connect to.
+
+## Load Balancing
+
+Using the connection policies client TCP connections can be balanced across multiple servers.
+
+## Connection Policies
+
+#### Ordered
+
+Servers are tried in the order they were supplied.
+
+~~~ruby
+tcp_client = Net::TCPClient.new(
+  servers: ['server1:3300', 'server2:3300', 'server3:3600']
+)
+~~~
+
+The servers will tried in the following order:
+`server1`, `server2`, `server3`
+
+`:ordered` is the default, but can be explicitly defined follows:
+
+~~~ruby
+tcp_client = Net::TCPClient.new(
+  servers: ['server1:3300', 'server2:3300', 'server3:3600'],
+  policy:  :ordered
+)
+~~~
+
+#### Random
+
+Servers are tried in a Random order.
+
+~~~ruby
+tcp_client = Net::TCPClient.new(
+  servers: ['server1:3300', 'server2:3300', 'server3:3600'],
+  policy:  :ordered
+)
+~~~
+
+No server is tried again until all of the others have been tried first.
+
+Example run, the servers could be tried in the following order:
+`server3`, `server1`, `server2`
+
+#### Custom defined order
+
+Supply your own custom order / load balancing algorithm for connecting to servers:
+
+Example:
+
+~~~ruby
+tcp_client = Net::TCPClient.new(
+  servers: ['server1:3300', 'server2:3300', 'server3:3600'],
+  policy:  -> addresses, count do
+    # Return nil after the last address has been tried so that retry logic can take over
+    if count <= address.size
+      addresses.sample
+    end
+  end
+)
+~~~
+
+The above example returns addresses in random order without checking if a host name has been used before.
+
+It is important to check the count so that once all servers have been tried, it should return nil so that
+the retry logic can take over. Otherwise it will constantly try to connect to the servers without
+the retry delays etc.
+
+Example run, the servers could be tried in the following order:
+`server3`, `server1`, `server3`
 
+### Automatic Retry
+
+If a connection cannot be established to any servers in the list Net::TCPClient will retry from the
+first server. This retry behavior can be controlled using the following options:
+
+* `connect_retry_count` [Fixnum]
+    * Number of times to retry connecting when a connection fails
+    * Default: 10
+
+* `connect_retry_interval` [Float]
+    * Number of seconds between connection retry attempts after the first failed attempt
+    * Default: 0.5
+
+* `retry_count` [Fixnum]
+    * Number of times to retry when calling #retry_on_connection_failure
+    * This is independent of :connect_retry_count which still applies with
+    * connection failures. This retry controls upto how many times to retry the
+    * supplied block should a connection failure occur during the block
+    * Default: 3
+
+#### Note
+
+A server will only be retried again using the retry controls above once all other servers in the
+list have been exhausted.
+
+This means that if a connection is lost to a server that it will try to connect to a different server,
+not the same server unless it is the only server in the list.
+
+## Tuning
+
+If there are multiple servers in the list it is important to keep the `connect_timeout` low otherwise
+it can take a long time to find the next available server.
+
+## Retry on connection loss
+
+To transparently handle when a connection is lost after it has been established
+wrap calls that can be retried with `retry_on_connection_failure`.
+
+~~~ruby
 Net::TCPClient.connect(
   server:                 'localhost:3300',
   connect_retry_interval: 0.1,
@@ -45,12 +186,57 @@ Net::TCPClient.connect(
 ) do |client|
   # If the connection is lost, create a new one and retry the send
   client.retry_on_connection_failure do
-    client.send('Update the database')
+    client.send('How many users available?')
+    response = client.read(20)
+    puts "Received: #{response}"
   end
-  response = client.read(20)
-  puts "Received: #{response}"
 end
-```
+~~~
+
+If the connection is lost during either the `send` or the `read` above the
+entire block will be re-tried once the connection has been re-stablished.
+
+## Callbacks
+
+Any time a connection has been established a callback can be called to handle activities such as:
+
+* Initialize per connection session sequence numbers.
+* Pass authentication information to the server.
+* Perform a handshake with the server.
+
+#### Authentication example:
+
+~~~ruby
+tcp_client = Net::TCPClient.new(
+  servers: ['server1:3300', 'server2:3300', 'server3:3600'],
+  on_connect: -> do |client|
+    client.send('My username and password')
+    result = client.read(2)
+    raise "Authentication failed" if result != 'OK'
+  end
+)
+~~~
+
+#### Per connection sequence number example:
+
+~~~ruby
+tcp_client = Net::TCPClient.new(
+  servers: ['server1:3300', 'server2:3300', 'server3:3600'],
+  on_connect: -> do |client|
+    # Set the sequence number to 0
+    user_data = 0
+  end
+)
+
+tcp_client.retry_on_connection_failure do
+  # Send with the sequence number
+  tcp_client.send("#{tcp_client.user_data} hello")
+  result = tcp_client.receive(30)
+
+  # Increment sequence number after every call to the server
+  tcp_client.user_data += 1
+end
+~~~
 
 ## Project Status
 
@@ -64,34 +250,46 @@ test on a daily basis, including connections over the internet between remote da
 
     gem install net_tcp_client
 
-Although not required, it is recommended to use [Semantic Logger](http://rocketjob.github.io/semantic_logger) for logging purposes:
+To enable logging add [Semantic Logger](http://rocketjob.github.io/semantic_logger):
 
     gem install semantic_logger
 
 Or, add the following lines to you `Gemfile`:
 
-```ruby
-    gem 'semantic_logger'
-    gem 'net_tcp_client'
-```
+~~~ruby
+gem 'semantic_logger'
+gem 'net_tcp_client'
+~~~
 
 To configure a stand-alone application for Semantic Logger:
 
-```ruby
+~~~ruby
 require 'semantic_logger'
 
 # Set the global default log level
 SemanticLogger.default_level = :trace
 
 # Log to a file, and use the colorized formatter
-SemanticLogger.add_appender('development.log', &SemanticLogger::Appender::Base.colorized_formatter)
-```
+SemanticLogger.add_appender(file_name: 'development.log', formatter: :color)
+~~~
 
 If running Rails, see: [Semantic Logger Rails](http://rocketjob.github.io/semantic_logger/rails.html)
 
-Without Semantic Logger present a Ruby logger can be passed into Net::TCPClient.
+### Support
 
-### Upgrading from ResilientSocket
+Join the [Gitter chat session](https://gitter.im/rocketjob/support) if you have any questions.
+
+Issues / bugs can be reported via [Github issues](https://github.com/rocketjob/net_tcp_client/issues).
+
+### Upgrading to V2
+
+The following breaking changes have been made with V2:
+* The Connection timeout default is now 10 seconds, was 30 seconds.
+* To enable logging, add gem semantic_logger.
+    * The :logger option has been removed.
+* Deprecated option and attribute :server_selector has been removed.
+
+### Upgrading from ResilientSocket ![](https://img.shields.io/gem/dt/resilient_socket.svg)
 
 ResilientSocket::TCPClient has been renamed to Net::TCPClient.
 The API is exactly the same, just with a new namespace. Please upgrade to the new
@@ -101,8 +299,8 @@ with `Net::TCPClient` in your code.
 ## Supports
 
 Tested and supported on the following Ruby platforms:
-- Ruby 1.9.3, 2.0, 2.1, 2.2 and above
-- JRuby 1.7, 9.0 and above
+- Ruby 2.1, 2.2, 2.3 and above
+- JRuby 1.7.23, 9.0 and above
 - Rubinius 2.5 and above
 
 There is a soft dependency on [Semantic Logger](http://github.com/rocketjob/semantic_logger). It will use SemanticLogger only if
@@ -112,14 +310,12 @@ it is already available, otherwise any other standard Ruby logger can be used.
 
 Be sure to place the `semantic_logger` gem dependency before `net_tcp_client` in your Gemfile.
 
-## Versioning
-
-This project adheres to [Semantic Versioning](http://semver.org/).
-
 ## Author
 
 [Reid Morrison](https://github.com/reidmorrison)
 
+[Contributors](https://github.com/rocketjob/net_tcp_client/graphs/contributors)
+
 ## Versioning
 
 This project uses [Semantic Versioning](http://semver.org/).

data/lib/net/tcp_client.rb

@@ -1,6 +1,22 @@
 require 'socket'
+# Load SemanticLogger if available
+begin
+  require 'semantic_logger'
+rescue LoadError
+end
 require 'net/tcp_client/version'
+require 'net/tcp_client/address'
 require 'net/tcp_client/exceptions'
 require 'net/tcp_client/tcp_client'
-# Use the builtin logger only if semantic logger is not already loaded
-require 'net/tcp_client/logging'
+
+# @formatter:off
+module Net
+  class TCPClient
+    module Policy
+      autoload :Base,    'net/tcp_client/policy/base.rb'
+      autoload :Custom,  'net/tcp_client/policy/custom.rb'
+      autoload :Ordered, 'net/tcp_client/policy/ordered.rb'
+      autoload :Random,  'net/tcp_client/policy/random.rb'
+    end
+  end
+end

data/lib/net/tcp_client/address.rb

@@ -0,0 +1,53 @@
+require 'socket'
+require 'ipaddr'
+module Net
+  class TCPClient
+    # Host name, ip address and port to connect to
+    class Address
+      attr_accessor :host_name, :ip_address, :port
+
+      # Returns [Array<String>] ip addresses for the supplied DNS entry
+      # Returns dns_name if it is already an IP Address
+      def self.ip_addresses(dns_name)
+        ips = []
+        Socket.getaddrinfo(dns_name, nil, Socket::AF_INET, Socket::SOCK_STREAM).each do |s|
+          ips << s[3] if s[0] == 'AF_INET'
+        end
+        ips.uniq
+      end
+
+      # Returns [Array<Net::TCPClient::Address>] addresses for a given DNS / host name.
+      # The Addresses will contain the resolved ip address, host name, and port number.
+      #
+      # Note:
+      #   Multiple ip addresses will be returned when a DNS entry has multiple ip addresses associated with it.
+      def self.addresses(dns_name, port)
+        ip_addresses(dns_name).collect { |ip| new(dns_name, ip, port) }
+      end
+
+      # Returns [Array<Net::TCPClient::Address>] addresses for a list of DNS / host name's
+      # that are paired with their numbers
+      #
+      # server_name should be either a host_name, or ip address combined with a port:
+      #   "host_name:1234"
+      #   "192.168.1.10:80"
+      def self.addresses_for_server_name(server_name)
+        dns_name, port = server_name.split(':')
+        port           = port.to_i
+        raise(ArgumentError, "Invalid host_name: #{server_name.inspect}. Must be formatted as 'host_name:1234' or '192.168.1.10:80'") unless dns_name && (port > 0)
+        addresses(dns_name, port)
+      end
+
+      def initialize(host_name, ip_address, port)
+        @host_name  = host_name
+        @ip_address = ip_address
+        @port       = port.to_i
+      end
+
+      def to_s
+        "#{host_name}[#{ip_address}]:#{port}"
+      end
+    end
+
+  end
+end

data/lib/net/tcp_client/exceptions.rb

@@ -1,14 +1,16 @@
 module Net
   class TCPClient
 
-    class ConnectionTimeout < ::SocketError;
+    class ConnectionTimeout < ::SocketError
     end
-    class ReadTimeout < ::SocketError;
+    class ReadTimeout < ::SocketError
+    end
+    class WriteTimeout < ::SocketError
     end
 
     # Raised by ResilientSocket whenever a Socket connection failure has occurred
     class ConnectionFailure < ::SocketError
-      # Returns the hostname and port against which the connection failure occurred
+      # Returns the host name and port against which the connection failure occurred
       attr_reader :server
 
       # Returns the original exception that caused the connection failure