net_tcp_client 1.0.0 → 1.0.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +59 -35
- data/Rakefile +3 -3
- data/lib/net/tcp_client.rb +1 -0
- data/lib/net/tcp_client/exceptions.rb +5 -4
- data/lib/net/tcp_client/logging.rb +18 -16
- data/lib/net/tcp_client/tcp_client.rb +73 -78
- data/lib/net/tcp_client/version.rb +2 -2
- data/lib/net_tcp_client.rb +1 -0
- data/test/simple_tcp_server.rb +31 -32
- data/test/tcp_client_test.rb +60 -71
- data/test/test_helper.rb +15 -0
- metadata +8 -5
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: f5b976dfea9f0cdbd07d3f25bfae55c230d26fd9
|
4
|
+
data.tar.gz: 10240a3b20378e17b9af022fef1e3bc7f27a29c2
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 71c89932e7a8a378d4386eb5885e66224afc9b25d22f4edda43a0382398cb434570dbfb140cdb706067e35cb728307a1b3030bf1a24fc9a93b411b5abd872f3b
|
7
|
+
data.tar.gz: 5eca04bf84b8e02ef677b2069c962b3429bf8906aa4f67ce7c66ba3e60d7c7be32095c2719b99df4f0ad67bf2804fa708c211781aaa9e21e82ecd1485a1d60c7
|
data/README.md
CHANGED
@@ -1,13 +1,13 @@
|
|
1
|
-
net_tcp_client
|
2
|
-
|
1
|
+
# net_tcp_client
|
2
|
+
![](https://img.shields.io/gem/v/net_tcp_client.svg) ![](https://img.shields.io/travis/rocketjob/net_tcp_client.svg) ![](https://img.shields.io/gem/dt/net_tcp_client.svg) ![](https://img.shields.io/badge/status-production%20ready-blue.svg)
|
3
3
|
|
4
4
|
Net::TCPClient is a TCP Socket Client with built-in timeouts, retries, and logging
|
5
5
|
|
6
|
-
* http://github.com/
|
6
|
+
* http://github.com/rocketjob/net_tcp_client
|
7
7
|
|
8
8
|
## Introduction
|
9
9
|
|
10
|
-
Net::TCPClient implements resilience features that
|
10
|
+
Net::TCPClient implements resilience features that many developers wish was
|
11
11
|
already included in the standard Ruby libraries.
|
12
12
|
|
13
13
|
With so many "client" libraries to servers such us memcache, MongoDB, Redis, etc.
|
@@ -52,50 +52,74 @@ Net::TCPClient.connect(
|
|
52
52
|
end
|
53
53
|
```
|
54
54
|
|
55
|
-
##
|
55
|
+
## Project Status
|
56
56
|
|
57
|
-
|
57
|
+
### Production Ready
|
58
58
|
|
59
|
-
|
60
|
-
|
61
|
-
|
62
|
-
|
63
|
-
## Production Use
|
64
|
-
|
65
|
-
Net::TCPClient was built for and is being used in a high performance, highly concurrent
|
66
|
-
production environment. The resilient capabilities of Net::TCPClient are put to the
|
67
|
-
test on a daily basis, especially with connections over the internet between
|
68
|
-
remote data centers.
|
59
|
+
Net::TCPClient is actively being used in a high performance, highly concurrent
|
60
|
+
production environments. The resilient capabilities of Net::TCPClient are put to the
|
61
|
+
test on a daily basis, including connections over the internet between remote data centers.
|
69
62
|
|
70
63
|
## Installation
|
71
64
|
|
72
65
|
gem install net_tcp_client
|
73
66
|
|
74
|
-
|
67
|
+
Although not required, it is recommended to use [Semantic Logger](http://rocketjob.github.io/semantic_logger) for logging purposes:
|
75
68
|
|
76
|
-
|
77
|
-
* Home: <https://github.com/reidmorrison/net_tcp_client>
|
78
|
-
* Bugs: <http://github.com/reidmorrison/net_tcp_client/issues>
|
79
|
-
* Gems: <http://rubygems.org/gems/net_tcp_client>
|
69
|
+
gem install semantic_logger
|
80
70
|
|
81
|
-
|
71
|
+
Or, add the following lines to you `Gemfile`:
|
82
72
|
|
83
|
-
|
73
|
+
```ruby
|
74
|
+
gem 'semantic_logger'
|
75
|
+
gem 'net_tcp_client'
|
76
|
+
```
|
77
|
+
|
78
|
+
To configure a stand-alone application for Semantic Logger:
|
79
|
+
|
80
|
+
```ruby
|
81
|
+
require 'semantic_logger'
|
82
|
+
|
83
|
+
# Set the global default log level
|
84
|
+
SemanticLogger.default_level = :trace
|
85
|
+
|
86
|
+
# Log to a file, and use the colorized formatter
|
87
|
+
SemanticLogger.add_appender('development.log', &SemanticLogger::Appender::Base.colorized_formatter)
|
88
|
+
```
|
84
89
|
|
85
|
-
[
|
90
|
+
If running Rails, see: [Semantic Logger Rails](http://rocketjob.github.io/semantic_logger/rails.html)
|
86
91
|
|
87
|
-
|
92
|
+
Without Semantic Logger present a Ruby logger can be passed into Net::TCPClient.
|
88
93
|
|
89
|
-
|
94
|
+
### Upgrading from ResilientSocket
|
90
95
|
|
91
|
-
|
92
|
-
|
93
|
-
|
96
|
+
ResilientSocket::TCPClient has been renamed to Net::TCPClient.
|
97
|
+
The API is exactly the same, just with a new namespace. Please upgrade to the new
|
98
|
+
`net_tcp_client` gem and replace all occurrences of `ResilientSocket::TCPClient`
|
99
|
+
with `Net::TCPClient` in your code.
|
94
100
|
|
95
|
-
|
101
|
+
## Supports
|
96
102
|
|
97
|
-
|
98
|
-
|
99
|
-
|
100
|
-
|
101
|
-
|
103
|
+
Tested and supported on the following Ruby platforms:
|
104
|
+
- Ruby 1.9.3, 2.0, 2.1, 2.2 and above
|
105
|
+
- JRuby 1.7, 9.0 and above
|
106
|
+
- Rubinius 2.5 and above
|
107
|
+
|
108
|
+
There is a soft dependency on [Semantic Logger](http://github.com/rocketjob/semantic_logger). It will use SemanticLogger only if
|
109
|
+
it is already available, otherwise any other standard Ruby logger can be used.
|
110
|
+
|
111
|
+
### Note
|
112
|
+
|
113
|
+
Be sure to place the `semantic_logger` gem dependency before `net_tcp_client` in your Gemfile.
|
114
|
+
|
115
|
+
## Versioning
|
116
|
+
|
117
|
+
This project adheres to [Semantic Versioning](http://semver.org/).
|
118
|
+
|
119
|
+
## Author
|
120
|
+
|
121
|
+
[Reid Morrison](https://github.com/reidmorrison)
|
122
|
+
|
123
|
+
## Versioning
|
124
|
+
|
125
|
+
This project uses [Semantic Versioning](http://semver.org/).
|
data/Rakefile
CHANGED
@@ -5,17 +5,17 @@ $LOAD_PATH.unshift File.expand_path("../lib", __FILE__)
|
|
5
5
|
require 'net/tcp_client/version'
|
6
6
|
|
7
7
|
task :gem do
|
8
|
-
system
|
8
|
+
system 'gem build net_tcp_client.gemspec'
|
9
9
|
end
|
10
10
|
|
11
11
|
task :publish => :gem do
|
12
12
|
system "git tag -a v#{Net::TCPClient::VERSION} -m 'Tagging #{Net::TCPClient::VERSION}'"
|
13
|
-
system
|
13
|
+
system 'git push --tags'
|
14
14
|
system "gem push net_tcp_client-#{Net::TCPClient::VERSION}.gem"
|
15
15
|
system "rm net_tcp_client-#{Net::TCPClient::VERSION}.gem"
|
16
16
|
end
|
17
17
|
|
18
|
-
desc
|
18
|
+
desc 'Run Test Suite'
|
19
19
|
task :test do
|
20
20
|
Rake::TestTask.new(:functional) do |t|
|
21
21
|
t.test_files = FileList['test/*_test.rb']
|
data/lib/net/tcp_client.rb
CHANGED
@@ -1,9 +1,10 @@
|
|
1
|
-
require 'socket'
|
2
1
|
module Net
|
3
2
|
class TCPClient
|
4
3
|
|
5
|
-
class ConnectionTimeout < ::SocketError;
|
6
|
-
|
4
|
+
class ConnectionTimeout < ::SocketError;
|
5
|
+
end
|
6
|
+
class ReadTimeout < ::SocketError;
|
7
|
+
end
|
7
8
|
|
8
9
|
# Raised by ResilientSocket whenever a Socket connection failure has occurred
|
9
10
|
class ConnectionFailure < ::SocketError
|
@@ -26,7 +27,7 @@ module Net
|
|
26
27
|
# Original Exception if any, otherwise nil
|
27
28
|
def initialize(message, server, cause=nil)
|
28
29
|
@server = server
|
29
|
-
@cause
|
30
|
+
@cause = cause
|
30
31
|
super(message)
|
31
32
|
end
|
32
33
|
end
|
@@ -18,7 +18,7 @@ module Net
|
|
18
18
|
else
|
19
19
|
# Return a nil logger
|
20
20
|
require 'logger'
|
21
|
-
logger
|
21
|
+
logger = Logger.new($null)
|
22
22
|
logger.level = Logger::FATAL
|
23
23
|
logger.extend(InstanceMethods)
|
24
24
|
logger
|
@@ -64,7 +64,7 @@ module Net
|
|
64
64
|
def format_log_message(level, message=nil, payload=nil, exception=nil, duration=nil, &block)
|
65
65
|
if exception.nil? && payload && payload.is_a?(Exception)
|
66
66
|
exception = payload
|
67
|
-
payload
|
67
|
+
payload = nil
|
68
68
|
end
|
69
69
|
|
70
70
|
if block && (result = block.call)
|
@@ -85,8 +85,8 @@ module Net
|
|
85
85
|
tags_str = tags.collect { |tag| "[#{tag}]" }.join(" ") + " " if tags && (tags.size > 0)
|
86
86
|
|
87
87
|
message = message.to_s.dup
|
88
|
-
message <<
|
89
|
-
message <<
|
88
|
+
message << ' -- ' << payload.inspect if payload
|
89
|
+
message << ' -- Exception: ' << "#{exception.class}: #{exception.message}\n#{(exception.backtrace || []).join("\n")}" if exception
|
90
90
|
|
91
91
|
duration_str = duration ? "(#{'%.1f' % duration}ms) " : ''
|
92
92
|
|
@@ -95,26 +95,27 @@ module Net
|
|
95
95
|
|
96
96
|
# Measure the supplied block and log the message
|
97
97
|
def benchmark(level, message, params, &block)
|
98
|
-
start
|
98
|
+
start = Time.now
|
99
99
|
begin
|
100
|
-
rc
|
100
|
+
rc = block.call(params) if block
|
101
101
|
exception = params[:exception]
|
102
102
|
rc
|
103
103
|
rescue Exception => exc
|
104
104
|
exception = exc
|
105
105
|
ensure
|
106
|
-
end_time
|
106
|
+
end_time = Time.now
|
107
107
|
# Extract options after block completes so that block can modify any of the options
|
108
108
|
log_exception = params[:log_exception] || :partial
|
109
109
|
on_exception_level = params[:on_exception_level]
|
110
110
|
min_duration = params[:min_duration] || 0.0
|
111
111
|
payload = params[:payload]
|
112
112
|
metric = params[:metric]
|
113
|
-
duration =
|
114
|
-
|
115
|
-
|
116
|
-
|
117
|
-
|
113
|
+
duration =
|
114
|
+
if block_given?
|
115
|
+
1000.0 * (end_time - start)
|
116
|
+
else
|
117
|
+
params[:duration] || raise('Mandatory block missing when :duration option is not supplied')
|
118
|
+
end
|
118
119
|
|
119
120
|
# Add scoped payload
|
120
121
|
if self.payload
|
@@ -128,8 +129,8 @@ module Net
|
|
128
129
|
level = on_exception_level if on_exception_level
|
129
130
|
when :partial
|
130
131
|
# On exception change the log level
|
131
|
-
level
|
132
|
-
message
|
132
|
+
level = on_exception_level if on_exception_level
|
133
|
+
message = "#{message} -- Exception: #{exception.class}: #{exception.message}"
|
133
134
|
logged_exception = nil
|
134
135
|
else
|
135
136
|
logged_exception = nil
|
@@ -162,7 +163,8 @@ module Net
|
|
162
163
|
def push_tags *tags
|
163
164
|
# Need to flatten and reject empties to support calls from Rails 4
|
164
165
|
new_tags = tags.flatten.collect(&:to_s).reject(&:empty?)
|
165
|
-
t
|
166
|
+
t = Thread.current[:semantic_logger_tags]
|
167
|
+
|
166
168
|
Thread.current[:semantic_logger_tags] = t.nil? ? new_tags : t.concat(new_tags)
|
167
169
|
new_tags
|
168
170
|
end
|
@@ -173,7 +175,7 @@ module Net
|
|
173
175
|
end
|
174
176
|
|
175
177
|
def with_payload(payload)
|
176
|
-
current_payload
|
178
|
+
current_payload = self.payload
|
177
179
|
Thread.current[:semantic_logger_payload] = current_payload ? current_payload.merge(payload) : payload
|
178
180
|
yield
|
179
181
|
ensure
|
@@ -1,6 +1,4 @@
|
|
1
|
-
require 'socket'
|
2
1
|
module Net
|
3
|
-
|
4
2
|
# Make Socket calls resilient by adding timeouts, retries and specific
|
5
3
|
# exception categories
|
6
4
|
#
|
@@ -50,7 +48,7 @@ module Net
|
|
50
48
|
attr_accessor :read_timeout, :connect_timeout, :connect_retry_count,
|
51
49
|
:retry_count, :connect_retry_interval, :server_selector, :close_on_error
|
52
50
|
|
53
|
-
# Returns [
|
51
|
+
# Returns [true|false] Whether send buffering is enabled for this connection
|
54
52
|
attr_reader :buffered
|
55
53
|
|
56
54
|
# Returns the logger being used by the TCPClient instance
|
@@ -237,15 +235,18 @@ module Net
|
|
237
235
|
@close_on_error = true if @close_on_error.nil?
|
238
236
|
@logger = params.delete(:logger)
|
239
237
|
|
240
|
-
|
241
|
-
|
242
|
-
|
238
|
+
if server = params.delete(:server)
|
239
|
+
@servers = [server]
|
240
|
+
end
|
241
|
+
if servers = params.delete(:servers)
|
242
|
+
@servers = servers
|
243
243
|
end
|
244
|
+
raise(ArgumentError, 'Missing mandatory :server or :servers') unless @servers
|
244
245
|
|
245
|
-
# If a logger is supplied
|
246
|
+
# If a logger is supplied then extend it with the SemanticLogger API
|
246
247
|
@logger = Logging.new_logger(logger, "#{self.class.name} #{@servers.inspect}", params.delete(:log_level))
|
247
248
|
|
248
|
-
|
249
|
+
raise(ArgumentError, "Invalid options: #{params.inspect}") if params.size > 0
|
249
250
|
|
250
251
|
# Connect to the Server
|
251
252
|
connect
|
@@ -277,50 +278,17 @@ module Net
|
|
277
278
|
# and create a new connection
|
278
279
|
def connect
|
279
280
|
@socket.close if @socket && !@socket.closed?
|
280
|
-
|
281
|
-
|
282
|
-
when @server_selector.is_a?(Proc)
|
283
|
-
connect_to_server(@server_selector.call(@servers))
|
284
|
-
|
285
|
-
when @server_selector == :ordered
|
286
|
-
# Try each server in sequence
|
287
|
-
exception = nil
|
288
|
-
@servers.find do |server|
|
289
|
-
begin
|
290
|
-
connect_to_server(server)
|
291
|
-
exception = nil
|
292
|
-
true
|
293
|
-
rescue Net::TCPClient::ConnectionFailure => exc
|
294
|
-
exception = exc
|
295
|
-
false
|
296
|
-
end
|
297
|
-
end
|
298
|
-
# Raise Exception once it has also failed to connect to all servers
|
299
|
-
raise(exception) if exception
|
300
|
-
|
301
|
-
when @server_selector == :random
|
302
|
-
# Pick each server randomly, trying each server until one can be connected to
|
303
|
-
# If no server can be connected to a Net::TCPClient::ConnectionFailure is raised
|
304
|
-
servers_to_try = @servers.uniq
|
305
|
-
exception = nil
|
306
|
-
servers_to_try.size.times do |i|
|
307
|
-
server = servers_to_try[rand(servers_to_try.size)]
|
308
|
-
servers_to_try.delete(server)
|
309
|
-
begin
|
310
|
-
connect_to_server(server)
|
311
|
-
exception = nil
|
312
|
-
rescue Net::TCPClient::ConnectionFailure => exc
|
313
|
-
exception = exc
|
314
|
-
end
|
315
|
-
end
|
316
|
-
# Raise Exception once it has also failed to connect to all servers
|
317
|
-
raise(exception) if exception
|
318
|
-
|
319
|
-
else
|
320
|
-
raise ArgumentError.new("Invalid or unknown value for parameter :server_selector => #{@server_selector}")
|
321
|
-
end
|
322
|
-
else
|
281
|
+
case
|
282
|
+
when @servers.size == 1
|
323
283
|
connect_to_server(@servers.first)
|
284
|
+
when @server_selector.is_a?(Proc)
|
285
|
+
connect_to_server(@server_selector.call(@servers))
|
286
|
+
when @server_selector == :ordered
|
287
|
+
connect_to_servers_in_order(@servers)
|
288
|
+
when @server_selector == :random
|
289
|
+
connect_to_servers_in_order(@servers.sample(@servers.size))
|
290
|
+
else
|
291
|
+
raise ArgumentError.new("Invalid or unknown value for parameter :server_selector => #{@server_selector}")
|
324
292
|
end
|
325
293
|
|
326
294
|
# Invoke user supplied Block every time a new connection has been established
|
@@ -336,9 +304,10 @@ module Net
|
|
336
304
|
# For a description of the errors, see Socket#write
|
337
305
|
#
|
338
306
|
def write(data)
|
339
|
-
|
307
|
+
data = data.to_s
|
308
|
+
logger.trace('#write ==> sending', data)
|
340
309
|
stats = {}
|
341
|
-
logger.benchmark_debug(
|
310
|
+
logger.benchmark_debug('#write ==> complete', stats) do
|
342
311
|
begin
|
343
312
|
stats[:bytes_sent] = @socket.write(data)
|
344
313
|
rescue SystemCallError => exception
|
@@ -388,40 +357,21 @@ module Net
|
|
388
357
|
# before calling _connect_ or _retry_on_connection_failure_ to create
|
389
358
|
# a new connection
|
390
359
|
#
|
391
|
-
def read(length, buffer=nil, timeout=
|
360
|
+
def read(length, buffer = nil, timeout = read_timeout)
|
392
361
|
result = nil
|
393
362
|
logger.benchmark_debug("#read <== read #{length} bytes") do
|
394
|
-
|
395
|
-
# Block on data to read for @read_timeout seconds
|
396
|
-
ready = begin
|
397
|
-
ready = IO.select([@socket], nil, [@socket], timeout || @read_timeout)
|
398
|
-
rescue IOError => exception
|
399
|
-
logger.warn "#read Connection failure while waiting for data: #{exception.class}: #{exception.message}"
|
400
|
-
close if close_on_error
|
401
|
-
raise Net::TCPClient::ConnectionFailure.new("#{exception.class}: #{exception.message}", @server, exception)
|
402
|
-
rescue Exception
|
403
|
-
# Close the connection on any other exception since the connection
|
404
|
-
# will now be in an inconsistent state
|
405
|
-
close if close_on_error
|
406
|
-
raise
|
407
|
-
end
|
408
|
-
unless ready
|
409
|
-
close if close_on_error
|
410
|
-
logger.warn "#read Timeout waiting for server to reply"
|
411
|
-
raise Net::TCPClient::ReadTimeout.new("Timedout after #{timeout || @read_timeout} seconds trying to read from #{@server}")
|
412
|
-
end
|
413
|
-
end
|
363
|
+
wait_for_data(timeout)
|
414
364
|
|
415
365
|
# Read data from socket
|
416
366
|
begin
|
417
367
|
result = buffer.nil? ? @socket.read(length) : @socket.read(length, buffer)
|
418
|
-
logger.trace(
|
368
|
+
logger.trace('#read <== received', result)
|
419
369
|
|
420
370
|
# EOF before all the data was returned
|
421
371
|
if result.nil? || (result.length < length)
|
422
372
|
close if close_on_error
|
423
373
|
logger.warn "#read server closed the connection before #{length} bytes were returned"
|
424
|
-
raise Net::TCPClient::ConnectionFailure.new(
|
374
|
+
raise Net::TCPClient::ConnectionFailure.new('Connection lost while reading data', @server, EOFError.new('end of file reached'))
|
425
375
|
end
|
426
376
|
rescue SystemCallError, IOError => exception
|
427
377
|
close if close_on_error
|
@@ -560,9 +510,9 @@ module Net
|
|
560
510
|
retries = 0
|
561
511
|
logger.benchmark_info "Connected to #{server}" do
|
562
512
|
host_name, port = server.split(":")
|
563
|
-
port
|
513
|
+
port = port.to_i
|
564
514
|
|
565
|
-
address
|
515
|
+
address = Socket.getaddrinfo(host_name, nil, Socket::AF_INET)
|
566
516
|
socket_address = Socket.pack_sockaddr_in(port, address[0][3])
|
567
517
|
|
568
518
|
begin
|
@@ -600,5 +550,50 @@ module Net
|
|
600
550
|
@server = server
|
601
551
|
end
|
602
552
|
|
553
|
+
# Try connecting to each server in the order supplied
|
554
|
+
# The next server is tried if it cannot connect to the current one
|
555
|
+
# After the last server a ConnectionFailure will be raised
|
556
|
+
def connect_to_servers_in_order(servers)
|
557
|
+
exception = nil
|
558
|
+
servers.find do |server|
|
559
|
+
begin
|
560
|
+
connect_to_server(server)
|
561
|
+
exception = nil
|
562
|
+
true
|
563
|
+
rescue Net::TCPClient::ConnectionFailure => exc
|
564
|
+
exception = exc
|
565
|
+
false
|
566
|
+
end
|
567
|
+
end
|
568
|
+
# Raise Exception once it has also failed to connect to all servers
|
569
|
+
raise(exception) if exception
|
570
|
+
end
|
571
|
+
|
572
|
+
# Return once data is ready to be ready
|
573
|
+
# Raises Net::TCPClient::ReadTimeout if the timeout is exceeded
|
574
|
+
def wait_for_data(timeout)
|
575
|
+
return if timeout == -1
|
576
|
+
|
577
|
+
ready = false
|
578
|
+
begin
|
579
|
+
ready = IO.select([@socket], nil, [@socket], timeout)
|
580
|
+
rescue IOError => exception
|
581
|
+
logger.warn "#read Connection failure while waiting for data: #{exception.class}: #{exception.message}"
|
582
|
+
close if close_on_error
|
583
|
+
raise Net::TCPClient::ConnectionFailure.new("#{exception.class}: #{exception.message}", @server, exception)
|
584
|
+
rescue Exception
|
585
|
+
# Close the connection on any other exception since the connection
|
586
|
+
# will now be in an inconsistent state
|
587
|
+
close if close_on_error
|
588
|
+
raise
|
589
|
+
end
|
590
|
+
|
591
|
+
unless ready
|
592
|
+
close if close_on_error
|
593
|
+
logger.warn "#read Timeout after #{timeout} seconds"
|
594
|
+
raise Net::TCPClient::ReadTimeout.new("Timedout after #{timeout} seconds trying to read from #{@server}")
|
595
|
+
end
|
596
|
+
end
|
597
|
+
|
603
598
|
end
|
604
599
|
end
|
@@ -0,0 +1 @@
|
|
1
|
+
require 'net/tcp_client'
|
data/test/simple_tcp_server.rb
CHANGED
@@ -1,4 +1,3 @@
|
|
1
|
-
require 'rubygems'
|
2
1
|
require 'socket'
|
3
2
|
require 'bson'
|
4
3
|
require 'semantic_logger'
|
@@ -8,7 +7,7 @@ require 'semantic_logger'
|
|
8
7
|
def read_bson_document(io)
|
9
8
|
bytebuf = BSON::ByteBuffer.new
|
10
9
|
# Read 4 byte size of following BSON document
|
11
|
-
bytes
|
10
|
+
bytes = io.read(4)
|
12
11
|
return unless bytes
|
13
12
|
# Read BSON document
|
14
13
|
sz = bytes.unpack("V")[0]
|
@@ -22,33 +21,34 @@ end
|
|
22
21
|
# Simple single threaded server for testing purposes using a local socket
|
23
22
|
# Sends and receives BSON Messages
|
24
23
|
class SimpleTCPServer
|
25
|
-
|
24
|
+
include SemanticLogger::Loggable
|
25
|
+
|
26
|
+
attr_accessor :thread, :server
|
27
|
+
|
26
28
|
def initialize(port = 2000)
|
27
29
|
start(port)
|
28
30
|
end
|
29
31
|
|
30
32
|
def start(port)
|
31
|
-
|
32
|
-
|
33
|
-
|
34
|
-
@thread = Thread.new do
|
33
|
+
self.server = TCPServer.open(port)
|
34
|
+
self.thread = Thread.new do
|
35
35
|
loop do
|
36
|
-
|
36
|
+
logger.debug 'Waiting for a client to connect'
|
37
37
|
|
38
38
|
# Wait for a client to connect
|
39
|
-
on_request(
|
39
|
+
on_request(server.accept)
|
40
40
|
end
|
41
41
|
end
|
42
42
|
end
|
43
43
|
|
44
44
|
def stop
|
45
|
-
if
|
46
|
-
|
47
|
-
|
48
|
-
|
45
|
+
if thread
|
46
|
+
thread.kill
|
47
|
+
thread.join
|
48
|
+
self.thread = nil
|
49
49
|
end
|
50
50
|
begin
|
51
|
-
|
51
|
+
server.close if server
|
52
52
|
rescue IOError
|
53
53
|
end
|
54
54
|
end
|
@@ -58,50 +58,49 @@ class SimpleTCPServer
|
|
58
58
|
def on_message(message)
|
59
59
|
case message['action']
|
60
60
|
when 'test1'
|
61
|
-
{
|
61
|
+
{'result' => 'test1'}
|
62
62
|
when 'sleep'
|
63
63
|
sleep message['duration'] || 1
|
64
|
-
{
|
64
|
+
{'result' => 'sleep'}
|
65
65
|
when 'fail'
|
66
66
|
if message['attempt'].to_i >= 2
|
67
|
-
{
|
67
|
+
{'result' => 'fail'}
|
68
68
|
else
|
69
69
|
nil
|
70
70
|
end
|
71
71
|
else
|
72
|
-
{
|
72
|
+
{'result' => "Unknown action: #{message['action']}"}
|
73
73
|
end
|
74
74
|
end
|
75
75
|
|
76
76
|
# Called for each client connection
|
77
77
|
# In a real server each request would be handled in a separate thread
|
78
78
|
def on_request(client)
|
79
|
-
|
79
|
+
logger.debug 'Client connected, waiting for data from client'
|
80
80
|
|
81
|
-
while(request = read_bson_document(client)) do
|
82
|
-
|
83
|
-
@logger.trace 'Request', request
|
81
|
+
while (request = read_bson_document(client)) do
|
82
|
+
logger.debug 'Received request', request
|
84
83
|
break unless request
|
85
84
|
|
86
85
|
if reply = on_message(request)
|
87
|
-
|
88
|
-
|
86
|
+
logger.debug 'Sending Reply'
|
87
|
+
logger.trace 'Reply', reply
|
89
88
|
client.print(BSON.serialize(reply))
|
90
89
|
else
|
91
|
-
|
92
|
-
|
90
|
+
logger.debug 'Closing client since no reply is being sent back'
|
91
|
+
server.close
|
93
92
|
client.close
|
94
|
-
|
95
|
-
|
96
|
-
|
93
|
+
logger.debug 'Server closed'
|
94
|
+
#thread.kill
|
95
|
+
logger.debug 'thread killed'
|
97
96
|
start(2000)
|
98
|
-
|
97
|
+
logger.debug 'Server Restarted'
|
99
98
|
break
|
100
99
|
end
|
101
100
|
end
|
102
101
|
# Disconnect from the client
|
103
102
|
client.close
|
104
|
-
|
103
|
+
logger.debug 'Disconnected from the client'
|
105
104
|
end
|
106
105
|
|
107
106
|
end
|
@@ -111,4 +110,4 @@ if $0 == __FILE__
|
|
111
110
|
SemanticLogger.add_appender(STDOUT)
|
112
111
|
server = SimpleTCPServer.new(2000)
|
113
112
|
server.thread.join
|
114
|
-
end
|
113
|
+
end
|
data/test/tcp_client_test.rb
CHANGED
@@ -1,42 +1,31 @@
|
|
1
|
-
# Allow test to be run in-place without requiring a gem install
|
2
|
-
$LOAD_PATH.unshift File.dirname(__FILE__) + '/../lib'
|
3
|
-
$LOAD_PATH.unshift File.dirname(__FILE__)
|
4
|
-
|
5
|
-
require 'rubygems'
|
6
|
-
require 'test/unit'
|
7
|
-
require 'shoulda'
|
8
1
|
require 'socket'
|
9
|
-
|
10
|
-
|
11
|
-
|
12
|
-
SemanticLogger.default_level = :trace
|
13
|
-
SemanticLogger.add_appender('test.log')
|
2
|
+
require_relative 'test_helper'
|
3
|
+
require_relative 'simple_tcp_server'
|
14
4
|
|
15
5
|
# Unit Test for Net::TCPClient
|
16
|
-
class TCPClientTest < Test
|
17
|
-
|
18
|
-
|
19
|
-
|
20
|
-
|
21
|
-
exception = assert_raise Net::TCPClient::ConnectionFailure do
|
6
|
+
class TCPClientTest < Minitest::Test
|
7
|
+
describe Net::TCPClient do
|
8
|
+
describe 'without server' do
|
9
|
+
it 'raises an exception when cannot reach server after 5 retries' do
|
10
|
+
exception = assert_raises Net::TCPClient::ConnectionFailure do
|
22
11
|
Net::TCPClient.new(
|
23
|
-
:
|
24
|
-
:
|
25
|
-
:
|
12
|
+
server: 'localhost:3300',
|
13
|
+
connect_retry_interval: 0.1,
|
14
|
+
connect_retry_count: 5)
|
26
15
|
end
|
27
16
|
assert_match /After 5 connection attempts to host 'localhost:3300': Errno::ECONNREFUSED/, exception.message
|
28
17
|
end
|
29
18
|
|
30
|
-
|
19
|
+
it 'times out on connect' do
|
31
20
|
# Create a TCP Server, but do not respond to connections
|
32
21
|
server = TCPServer.open(2001)
|
33
22
|
|
34
|
-
exception =
|
23
|
+
exception = assert_raises Net::TCPClient::ConnectionTimeout do
|
35
24
|
1000.times do
|
36
25
|
Net::TCPClient.new(
|
37
|
-
:
|
38
|
-
:
|
39
|
-
:
|
26
|
+
server: 'localhost:2001',
|
27
|
+
connect_timeout: 0.5,
|
28
|
+
connect_retry_count: 3
|
40
29
|
)
|
41
30
|
end
|
42
31
|
end
|
@@ -46,47 +35,47 @@ class TCPClientTest < Test::Unit::TestCase
|
|
46
35
|
|
47
36
|
end
|
48
37
|
|
49
|
-
|
50
|
-
|
51
|
-
@server
|
38
|
+
describe "with server" do
|
39
|
+
before do
|
40
|
+
@server = SimpleTCPServer.new(2000)
|
52
41
|
@server_name = 'localhost:2000'
|
53
42
|
end
|
54
43
|
|
55
|
-
|
44
|
+
after do
|
56
45
|
@server.stop if @server
|
57
46
|
end
|
58
47
|
|
59
|
-
|
60
|
-
|
48
|
+
describe 'without client connection' do
|
49
|
+
it 'times out on first receive and then successfully reads the response' do
|
61
50
|
@read_timeout = 3.0
|
62
51
|
# Need a custom client that does not auto close on error:
|
63
|
-
@client
|
64
|
-
:
|
65
|
-
:
|
66
|
-
:
|
52
|
+
@client = Net::TCPClient.new(
|
53
|
+
server: @server_name,
|
54
|
+
read_timeout: @read_timeout,
|
55
|
+
close_on_error: false
|
67
56
|
)
|
68
57
|
|
69
|
-
request = {
|
58
|
+
request = {'action' => 'sleep', 'duration' => @read_timeout + 0.5}
|
70
59
|
@client.write(BSON.serialize(request))
|
71
60
|
|
72
|
-
exception =
|
61
|
+
exception = assert_raises Net::TCPClient::ReadTimeout do
|
73
62
|
# Read 4 bytes from server
|
74
63
|
@client.read(4)
|
75
64
|
end
|
76
65
|
assert_equal false, @client.close_on_error
|
77
|
-
assert @client.alive?,
|
66
|
+
assert @client.alive?, 'The client connection is not alive after the read timed out with close_on_error: false'
|
78
67
|
assert_match /Timedout after #{@read_timeout} seconds trying to read from #{@server_name}/, exception.message
|
79
68
|
reply = read_bson_document(@client)
|
80
69
|
assert_equal 'sleep', reply['result']
|
81
70
|
@client.close
|
82
71
|
end
|
83
72
|
|
84
|
-
|
73
|
+
it 'support infinite timeout' do
|
85
74
|
@client = Net::TCPClient.new(
|
86
|
-
:
|
87
|
-
:
|
75
|
+
server: @server_name,
|
76
|
+
connect_timeout: -1
|
88
77
|
)
|
89
|
-
request = {
|
78
|
+
request = {'action' => 'test1'}
|
90
79
|
@client.write(BSON.serialize(request))
|
91
80
|
reply = read_bson_document(@client)
|
92
81
|
assert_equal 'test1', reply['result']
|
@@ -94,51 +83,51 @@ class TCPClientTest < Test::Unit::TestCase
|
|
94
83
|
end
|
95
84
|
end
|
96
85
|
|
97
|
-
|
98
|
-
|
86
|
+
describe 'with client connection' do
|
87
|
+
before do
|
99
88
|
@read_timeout = 3.0
|
100
|
-
@client
|
101
|
-
:
|
102
|
-
:
|
89
|
+
@client = Net::TCPClient.new(
|
90
|
+
server: @server_name,
|
91
|
+
read_timeout: @read_timeout
|
103
92
|
)
|
104
93
|
assert @client.alive?
|
105
94
|
assert_equal true, @client.close_on_error
|
106
95
|
end
|
107
96
|
|
108
|
-
def
|
97
|
+
def after
|
109
98
|
if @client
|
110
99
|
@client.close
|
111
100
|
assert !@client.alive?
|
112
101
|
end
|
113
102
|
end
|
114
103
|
|
115
|
-
|
116
|
-
request = {
|
104
|
+
it 'sends and receives data' do
|
105
|
+
request = {'action' => 'test1'}
|
117
106
|
@client.write(BSON.serialize(request))
|
118
107
|
reply = read_bson_document(@client)
|
119
108
|
assert_equal 'test1', reply['result']
|
120
109
|
end
|
121
110
|
|
122
|
-
|
123
|
-
request = {
|
111
|
+
it 'timeouts on receive' do
|
112
|
+
request = {'action' => 'sleep', 'duration' => @read_timeout + 0.5}
|
124
113
|
@client.write(BSON.serialize(request))
|
125
114
|
|
126
|
-
exception =
|
115
|
+
exception = assert_raises Net::TCPClient::ReadTimeout do
|
127
116
|
# Read 4 bytes from server
|
128
117
|
@client.read(4)
|
129
118
|
end
|
130
|
-
# Due to :
|
119
|
+
# Due to close_on_error: true, a timeout will close the connection
|
131
120
|
# to prevent use of a socket connection in an inconsistent state
|
132
121
|
assert_equal false, @client.alive?
|
133
122
|
assert_match /Timedout after #{@read_timeout} seconds trying to read from #{@server_name}/, exception.message
|
134
123
|
end
|
135
124
|
|
136
|
-
|
125
|
+
it 'retries on connection failure' do
|
137
126
|
attempt = 0
|
138
|
-
reply
|
139
|
-
request = {
|
127
|
+
reply = @client.retry_on_connection_failure do
|
128
|
+
request = {'action' => 'fail', 'attempt' => (attempt+=1)}
|
140
129
|
@client.write(BSON.serialize(request))
|
141
|
-
# Note: Do not put the read in this block if it
|
130
|
+
# Note: Do not put the read in this block if it never sends the
|
142
131
|
# same request twice to the server
|
143
132
|
read_bson_document(@client)
|
144
133
|
end
|
@@ -147,15 +136,15 @@ class TCPClientTest < Test::Unit::TestCase
|
|
147
136
|
|
148
137
|
end
|
149
138
|
|
150
|
-
|
151
|
-
|
139
|
+
describe 'without client connection' do
|
140
|
+
it 'connects to second server when the first is down' do
|
152
141
|
client = Net::TCPClient.new(
|
153
|
-
:
|
154
|
-
:
|
142
|
+
servers: ['localhost:1999', @server_name],
|
143
|
+
read_timeout: 3
|
155
144
|
)
|
156
145
|
assert_equal @server_name, client.server
|
157
146
|
|
158
|
-
request = {
|
147
|
+
request = {'action' => 'test1'}
|
159
148
|
client.write(BSON.serialize(request))
|
160
149
|
reply = read_bson_document(client)
|
161
150
|
assert_equal 'test1', reply['result']
|
@@ -163,19 +152,19 @@ class TCPClientTest < Test::Unit::TestCase
|
|
163
152
|
client.close
|
164
153
|
end
|
165
154
|
|
166
|
-
|
155
|
+
it 'calls on_connect after connection' do
|
167
156
|
client = Net::TCPClient.new(
|
168
|
-
:
|
169
|
-
:
|
170
|
-
:
|
157
|
+
server: @server_name,
|
158
|
+
read_timeout: 3,
|
159
|
+
on_connect: Proc.new do |socket|
|
171
160
|
# Reset user_data on each connection
|
172
|
-
socket.user_data = {
|
161
|
+
socket.user_data = {sequence: 1}
|
173
162
|
end
|
174
163
|
)
|
175
164
|
assert_equal @server_name, client.server
|
176
165
|
assert_equal 1, client.user_data[:sequence]
|
177
166
|
|
178
|
-
request = {
|
167
|
+
request = {'action' => 'test1'}
|
179
168
|
client.write(BSON.serialize(request))
|
180
169
|
reply = read_bson_document(client)
|
181
170
|
assert_equal 'test1', reply['result']
|
@@ -187,4 +176,4 @@ class TCPClientTest < Test::Unit::TestCase
|
|
187
176
|
end
|
188
177
|
|
189
178
|
end
|
190
|
-
end
|
179
|
+
end
|
data/test/test_helper.rb
ADDED
@@ -0,0 +1,15 @@
|
|
1
|
+
# Allow test to be run in-place without requiring a gem install
|
2
|
+
$LOAD_PATH.unshift File.dirname(__FILE__) + '/../lib'
|
3
|
+
|
4
|
+
# Configure Rails Environment
|
5
|
+
ENV['RAILS_ENV'] = 'test'
|
6
|
+
|
7
|
+
require 'minitest/autorun'
|
8
|
+
require 'minitest/reporters'
|
9
|
+
require 'semantic_logger'
|
10
|
+
require 'net/tcp_client'
|
11
|
+
|
12
|
+
Minitest::Reporters.use! Minitest::Reporters::SpecReporter.new
|
13
|
+
|
14
|
+
SemanticLogger.default_level = :trace
|
15
|
+
SemanticLogger.add_appender('test.log', &SemanticLogger::Appender::Base.colorized_formatter)
|
metadata
CHANGED
@@ -1,16 +1,16 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: net_tcp_client
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.0.
|
4
|
+
version: 1.0.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Reid Morrison
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2015-11-03 00:00:00.000000000 Z
|
12
12
|
dependencies: []
|
13
|
-
description: Net::TCPClient implements resilience features that
|
13
|
+
description: Net::TCPClient implements resilience features that many developers wish
|
14
14
|
was already included in the standard Ruby libraries.
|
15
15
|
email:
|
16
16
|
- reidmo@gmail.com
|
@@ -26,9 +26,11 @@ files:
|
|
26
26
|
- lib/net/tcp_client/logging.rb
|
27
27
|
- lib/net/tcp_client/tcp_client.rb
|
28
28
|
- lib/net/tcp_client/version.rb
|
29
|
+
- lib/net_tcp_client.rb
|
29
30
|
- test/simple_tcp_server.rb
|
30
31
|
- test/tcp_client_test.rb
|
31
|
-
|
32
|
+
- test/test_helper.rb
|
33
|
+
homepage: https://github.com/rocketjob/net_tcp_client
|
32
34
|
licenses:
|
33
35
|
- Apache License V2.0
|
34
36
|
metadata: {}
|
@@ -48,7 +50,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
48
50
|
version: '0'
|
49
51
|
requirements: []
|
50
52
|
rubyforge_project:
|
51
|
-
rubygems_version: 2.
|
53
|
+
rubygems_version: 2.4.5.1
|
52
54
|
signing_key:
|
53
55
|
specification_version: 4
|
54
56
|
summary: Net::TCPClient is a TCP Socket Client with built-in timeouts, retries, and
|
@@ -56,3 +58,4 @@ summary: Net::TCPClient is a TCP Socket Client with built-in timeouts, retries,
|
|
56
58
|
test_files:
|
57
59
|
- test/simple_tcp_server.rb
|
58
60
|
- test/tcp_client_test.rb
|
61
|
+
- test/test_helper.rb
|