net_tcp_client 1.0.0 → 1.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +59 -35
- data/Rakefile +3 -3
- data/lib/net/tcp_client.rb +1 -0
- data/lib/net/tcp_client/exceptions.rb +5 -4
- data/lib/net/tcp_client/logging.rb +18 -16
- data/lib/net/tcp_client/tcp_client.rb +73 -78
- data/lib/net/tcp_client/version.rb +2 -2
- data/lib/net_tcp_client.rb +1 -0
- data/test/simple_tcp_server.rb +31 -32
- data/test/tcp_client_test.rb +60 -71
- data/test/test_helper.rb +15 -0
- metadata +8 -5
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: f5b976dfea9f0cdbd07d3f25bfae55c230d26fd9
|
4
|
+
data.tar.gz: 10240a3b20378e17b9af022fef1e3bc7f27a29c2
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 71c89932e7a8a378d4386eb5885e66224afc9b25d22f4edda43a0382398cb434570dbfb140cdb706067e35cb728307a1b3030bf1a24fc9a93b411b5abd872f3b
|
7
|
+
data.tar.gz: 5eca04bf84b8e02ef677b2069c962b3429bf8906aa4f67ce7c66ba3e60d7c7be32095c2719b99df4f0ad67bf2804fa708c211781aaa9e21e82ecd1485a1d60c7
|
data/README.md
CHANGED
@@ -1,13 +1,13 @@
|
|
1
|
-
net_tcp_client
|
2
|
-
|
1
|
+
# net_tcp_client
|
2
|
+
   
|
3
3
|
|
4
4
|
Net::TCPClient is a TCP Socket Client with built-in timeouts, retries, and logging
|
5
5
|
|
6
|
-
* http://github.com/
|
6
|
+
* http://github.com/rocketjob/net_tcp_client
|
7
7
|
|
8
8
|
## Introduction
|
9
9
|
|
10
|
-
Net::TCPClient implements resilience features that
|
10
|
+
Net::TCPClient implements resilience features that many developers wish was
|
11
11
|
already included in the standard Ruby libraries.
|
12
12
|
|
13
13
|
With so many "client" libraries to servers such us memcache, MongoDB, Redis, etc.
|
@@ -52,50 +52,74 @@ Net::TCPClient.connect(
|
|
52
52
|
end
|
53
53
|
```
|
54
54
|
|
55
|
-
##
|
55
|
+
## Project Status
|
56
56
|
|
57
|
-
|
57
|
+
### Production Ready
|
58
58
|
|
59
|
-
|
60
|
-
|
61
|
-
|
62
|
-
|
63
|
-
## Production Use
|
64
|
-
|
65
|
-
Net::TCPClient was built for and is being used in a high performance, highly concurrent
|
66
|
-
production environment. The resilient capabilities of Net::TCPClient are put to the
|
67
|
-
test on a daily basis, especially with connections over the internet between
|
68
|
-
remote data centers.
|
59
|
+
Net::TCPClient is actively being used in a high performance, highly concurrent
|
60
|
+
production environments. The resilient capabilities of Net::TCPClient are put to the
|
61
|
+
test on a daily basis, including connections over the internet between remote data centers.
|
69
62
|
|
70
63
|
## Installation
|
71
64
|
|
72
65
|
gem install net_tcp_client
|
73
66
|
|
74
|
-
|
67
|
+
Although not required, it is recommended to use [Semantic Logger](http://rocketjob.github.io/semantic_logger) for logging purposes:
|
75
68
|
|
76
|
-
|
77
|
-
* Home: <https://github.com/reidmorrison/net_tcp_client>
|
78
|
-
* Bugs: <http://github.com/reidmorrison/net_tcp_client/issues>
|
79
|
-
* Gems: <http://rubygems.org/gems/net_tcp_client>
|
69
|
+
gem install semantic_logger
|
80
70
|
|
81
|
-
|
71
|
+
Or, add the following lines to you `Gemfile`:
|
82
72
|
|
83
|
-
|
73
|
+
```ruby
|
74
|
+
gem 'semantic_logger'
|
75
|
+
gem 'net_tcp_client'
|
76
|
+
```
|
77
|
+
|
78
|
+
To configure a stand-alone application for Semantic Logger:
|
79
|
+
|
80
|
+
```ruby
|
81
|
+
require 'semantic_logger'
|
82
|
+
|
83
|
+
# Set the global default log level
|
84
|
+
SemanticLogger.default_level = :trace
|
85
|
+
|
86
|
+
# Log to a file, and use the colorized formatter
|
87
|
+
SemanticLogger.add_appender('development.log', &SemanticLogger::Appender::Base.colorized_formatter)
|
88
|
+
```
|
84
89
|
|
85
|
-
[
|
90
|
+
If running Rails, see: [Semantic Logger Rails](http://rocketjob.github.io/semantic_logger/rails.html)
|
86
91
|
|
87
|
-
|
92
|
+
Without Semantic Logger present a Ruby logger can be passed into Net::TCPClient.
|
88
93
|
|
89
|
-
|
94
|
+
### Upgrading from ResilientSocket
|
90
95
|
|
91
|
-
|
92
|
-
|
93
|
-
|
96
|
+
ResilientSocket::TCPClient has been renamed to Net::TCPClient.
|
97
|
+
The API is exactly the same, just with a new namespace. Please upgrade to the new
|
98
|
+
`net_tcp_client` gem and replace all occurrences of `ResilientSocket::TCPClient`
|
99
|
+
with `Net::TCPClient` in your code.
|
94
100
|
|
95
|
-
|
101
|
+
## Supports
|
96
102
|
|
97
|
-
|
98
|
-
|
99
|
-
|
100
|
-
|
101
|
-
|
103
|
+
Tested and supported on the following Ruby platforms:
|
104
|
+
- Ruby 1.9.3, 2.0, 2.1, 2.2 and above
|
105
|
+
- JRuby 1.7, 9.0 and above
|
106
|
+
- Rubinius 2.5 and above
|
107
|
+
|
108
|
+
There is a soft dependency on [Semantic Logger](http://github.com/rocketjob/semantic_logger). It will use SemanticLogger only if
|
109
|
+
it is already available, otherwise any other standard Ruby logger can be used.
|
110
|
+
|
111
|
+
### Note
|
112
|
+
|
113
|
+
Be sure to place the `semantic_logger` gem dependency before `net_tcp_client` in your Gemfile.
|
114
|
+
|
115
|
+
## Versioning
|
116
|
+
|
117
|
+
This project adheres to [Semantic Versioning](http://semver.org/).
|
118
|
+
|
119
|
+
## Author
|
120
|
+
|
121
|
+
[Reid Morrison](https://github.com/reidmorrison)
|
122
|
+
|
123
|
+
## Versioning
|
124
|
+
|
125
|
+
This project uses [Semantic Versioning](http://semver.org/).
|
data/Rakefile
CHANGED
@@ -5,17 +5,17 @@ $LOAD_PATH.unshift File.expand_path("../lib", __FILE__)
|
|
5
5
|
require 'net/tcp_client/version'
|
6
6
|
|
7
7
|
task :gem do
|
8
|
-
system
|
8
|
+
system 'gem build net_tcp_client.gemspec'
|
9
9
|
end
|
10
10
|
|
11
11
|
task :publish => :gem do
|
12
12
|
system "git tag -a v#{Net::TCPClient::VERSION} -m 'Tagging #{Net::TCPClient::VERSION}'"
|
13
|
-
system
|
13
|
+
system 'git push --tags'
|
14
14
|
system "gem push net_tcp_client-#{Net::TCPClient::VERSION}.gem"
|
15
15
|
system "rm net_tcp_client-#{Net::TCPClient::VERSION}.gem"
|
16
16
|
end
|
17
17
|
|
18
|
-
desc
|
18
|
+
desc 'Run Test Suite'
|
19
19
|
task :test do
|
20
20
|
Rake::TestTask.new(:functional) do |t|
|
21
21
|
t.test_files = FileList['test/*_test.rb']
|
data/lib/net/tcp_client.rb
CHANGED
@@ -1,9 +1,10 @@
|
|
1
|
-
require 'socket'
|
2
1
|
module Net
|
3
2
|
class TCPClient
|
4
3
|
|
5
|
-
class ConnectionTimeout < ::SocketError;
|
6
|
-
|
4
|
+
class ConnectionTimeout < ::SocketError;
|
5
|
+
end
|
6
|
+
class ReadTimeout < ::SocketError;
|
7
|
+
end
|
7
8
|
|
8
9
|
# Raised by ResilientSocket whenever a Socket connection failure has occurred
|
9
10
|
class ConnectionFailure < ::SocketError
|
@@ -26,7 +27,7 @@ module Net
|
|
26
27
|
# Original Exception if any, otherwise nil
|
27
28
|
def initialize(message, server, cause=nil)
|
28
29
|
@server = server
|
29
|
-
@cause
|
30
|
+
@cause = cause
|
30
31
|
super(message)
|
31
32
|
end
|
32
33
|
end
|
@@ -18,7 +18,7 @@ module Net
|
|
18
18
|
else
|
19
19
|
# Return a nil logger
|
20
20
|
require 'logger'
|
21
|
-
logger
|
21
|
+
logger = Logger.new($null)
|
22
22
|
logger.level = Logger::FATAL
|
23
23
|
logger.extend(InstanceMethods)
|
24
24
|
logger
|
@@ -64,7 +64,7 @@ module Net
|
|
64
64
|
def format_log_message(level, message=nil, payload=nil, exception=nil, duration=nil, &block)
|
65
65
|
if exception.nil? && payload && payload.is_a?(Exception)
|
66
66
|
exception = payload
|
67
|
-
payload
|
67
|
+
payload = nil
|
68
68
|
end
|
69
69
|
|
70
70
|
if block && (result = block.call)
|
@@ -85,8 +85,8 @@ module Net
|
|
85
85
|
tags_str = tags.collect { |tag| "[#{tag}]" }.join(" ") + " " if tags && (tags.size > 0)
|
86
86
|
|
87
87
|
message = message.to_s.dup
|
88
|
-
message <<
|
89
|
-
message <<
|
88
|
+
message << ' -- ' << payload.inspect if payload
|
89
|
+
message << ' -- Exception: ' << "#{exception.class}: #{exception.message}\n#{(exception.backtrace || []).join("\n")}" if exception
|
90
90
|
|
91
91
|
duration_str = duration ? "(#{'%.1f' % duration}ms) " : ''
|
92
92
|
|
@@ -95,26 +95,27 @@ module Net
|
|
95
95
|
|
96
96
|
# Measure the supplied block and log the message
|
97
97
|
def benchmark(level, message, params, &block)
|
98
|
-
start
|
98
|
+
start = Time.now
|
99
99
|
begin
|
100
|
-
rc
|
100
|
+
rc = block.call(params) if block
|
101
101
|
exception = params[:exception]
|
102
102
|
rc
|
103
103
|
rescue Exception => exc
|
104
104
|
exception = exc
|
105
105
|
ensure
|
106
|
-
end_time
|
106
|
+
end_time = Time.now
|
107
107
|
# Extract options after block completes so that block can modify any of the options
|
108
108
|
log_exception = params[:log_exception] || :partial
|
109
109
|
on_exception_level = params[:on_exception_level]
|
110
110
|
min_duration = params[:min_duration] || 0.0
|
111
111
|
payload = params[:payload]
|
112
112
|
metric = params[:metric]
|
113
|
-
duration =
|
114
|
-
|
115
|
-
|
116
|
-
|
117
|
-
|
113
|
+
duration =
|
114
|
+
if block_given?
|
115
|
+
1000.0 * (end_time - start)
|
116
|
+
else
|
117
|
+
params[:duration] || raise('Mandatory block missing when :duration option is not supplied')
|
118
|
+
end
|
118
119
|
|
119
120
|
# Add scoped payload
|
120
121
|
if self.payload
|
@@ -128,8 +129,8 @@ module Net
|
|
128
129
|
level = on_exception_level if on_exception_level
|
129
130
|
when :partial
|
130
131
|
# On exception change the log level
|
131
|
-
level
|
132
|
-
message
|
132
|
+
level = on_exception_level if on_exception_level
|
133
|
+
message = "#{message} -- Exception: #{exception.class}: #{exception.message}"
|
133
134
|
logged_exception = nil
|
134
135
|
else
|
135
136
|
logged_exception = nil
|
@@ -162,7 +163,8 @@ module Net
|
|
162
163
|
def push_tags *tags
|
163
164
|
# Need to flatten and reject empties to support calls from Rails 4
|
164
165
|
new_tags = tags.flatten.collect(&:to_s).reject(&:empty?)
|
165
|
-
t
|
166
|
+
t = Thread.current[:semantic_logger_tags]
|
167
|
+
|
166
168
|
Thread.current[:semantic_logger_tags] = t.nil? ? new_tags : t.concat(new_tags)
|
167
169
|
new_tags
|
168
170
|
end
|
@@ -173,7 +175,7 @@ module Net
|
|
173
175
|
end
|
174
176
|
|
175
177
|
def with_payload(payload)
|
176
|
-
current_payload
|
178
|
+
current_payload = self.payload
|
177
179
|
Thread.current[:semantic_logger_payload] = current_payload ? current_payload.merge(payload) : payload
|
178
180
|
yield
|
179
181
|
ensure
|
@@ -1,6 +1,4 @@
|
|
1
|
-
require 'socket'
|
2
1
|
module Net
|
3
|
-
|
4
2
|
# Make Socket calls resilient by adding timeouts, retries and specific
|
5
3
|
# exception categories
|
6
4
|
#
|
@@ -50,7 +48,7 @@ module Net
|
|
50
48
|
attr_accessor :read_timeout, :connect_timeout, :connect_retry_count,
|
51
49
|
:retry_count, :connect_retry_interval, :server_selector, :close_on_error
|
52
50
|
|
53
|
-
# Returns [
|
51
|
+
# Returns [true|false] Whether send buffering is enabled for this connection
|
54
52
|
attr_reader :buffered
|
55
53
|
|
56
54
|
# Returns the logger being used by the TCPClient instance
|
@@ -237,15 +235,18 @@ module Net
|
|
237
235
|
@close_on_error = true if @close_on_error.nil?
|
238
236
|
@logger = params.delete(:logger)
|
239
237
|
|
240
|
-
|
241
|
-
|
242
|
-
|
238
|
+
if server = params.delete(:server)
|
239
|
+
@servers = [server]
|
240
|
+
end
|
241
|
+
if servers = params.delete(:servers)
|
242
|
+
@servers = servers
|
243
243
|
end
|
244
|
+
raise(ArgumentError, 'Missing mandatory :server or :servers') unless @servers
|
244
245
|
|
245
|
-
# If a logger is supplied
|
246
|
+
# If a logger is supplied then extend it with the SemanticLogger API
|
246
247
|
@logger = Logging.new_logger(logger, "#{self.class.name} #{@servers.inspect}", params.delete(:log_level))
|
247
248
|
|
248
|
-
|
249
|
+
raise(ArgumentError, "Invalid options: #{params.inspect}") if params.size > 0
|
249
250
|
|
250
251
|
# Connect to the Server
|
251
252
|
connect
|
@@ -277,50 +278,17 @@ module Net
|
|
277
278
|
# and create a new connection
|
278
279
|
def connect
|
279
280
|
@socket.close if @socket && !@socket.closed?
|
280
|
-
|
281
|
-
|
282
|
-
when @server_selector.is_a?(Proc)
|
283
|
-
connect_to_server(@server_selector.call(@servers))
|
284
|
-
|
285
|
-
when @server_selector == :ordered
|
286
|
-
# Try each server in sequence
|
287
|
-
exception = nil
|
288
|
-
@servers.find do |server|
|
289
|
-
begin
|
290
|
-
connect_to_server(server)
|
291
|
-
exception = nil
|
292
|
-
true
|
293
|
-
rescue Net::TCPClient::ConnectionFailure => exc
|
294
|
-
exception = exc
|
295
|
-
false
|
296
|
-
end
|
297
|
-
end
|
298
|
-
# Raise Exception once it has also failed to connect to all servers
|
299
|
-
raise(exception) if exception
|
300
|
-
|
301
|
-
when @server_selector == :random
|
302
|
-
# Pick each server randomly, trying each server until one can be connected to
|
303
|
-
# If no server can be connected to a Net::TCPClient::ConnectionFailure is raised
|
304
|
-
servers_to_try = @servers.uniq
|
305
|
-
exception = nil
|
306
|
-
servers_to_try.size.times do |i|
|
307
|
-
server = servers_to_try[rand(servers_to_try.size)]
|
308
|
-
servers_to_try.delete(server)
|
309
|
-
begin
|
310
|
-
connect_to_server(server)
|
311
|
-
exception = nil
|
312
|
-
rescue Net::TCPClient::ConnectionFailure => exc
|
313
|
-
exception = exc
|
314
|
-
end
|
315
|
-
end
|
316
|
-
# Raise Exception once it has also failed to connect to all servers
|
317
|
-
raise(exception) if exception
|
318
|
-
|
319
|
-
else
|
320
|
-
raise ArgumentError.new("Invalid or unknown value for parameter :server_selector => #{@server_selector}")
|
321
|
-
end
|
322
|
-
else
|
281
|
+
case
|
282
|
+
when @servers.size == 1
|
323
283
|
connect_to_server(@servers.first)
|
284
|
+
when @server_selector.is_a?(Proc)
|
285
|
+
connect_to_server(@server_selector.call(@servers))
|
286
|
+
when @server_selector == :ordered
|
287
|
+
connect_to_servers_in_order(@servers)
|
288
|
+
when @server_selector == :random
|
289
|
+
connect_to_servers_in_order(@servers.sample(@servers.size))
|
290
|
+
else
|
291
|
+
raise ArgumentError.new("Invalid or unknown value for parameter :server_selector => #{@server_selector}")
|
324
292
|
end
|
325
293
|
|
326
294
|
# Invoke user supplied Block every time a new connection has been established
|
@@ -336,9 +304,10 @@ module Net
|
|
336
304
|
# For a description of the errors, see Socket#write
|
337
305
|
#
|
338
306
|
def write(data)
|
339
|
-
|
307
|
+
data = data.to_s
|
308
|
+
logger.trace('#write ==> sending', data)
|
340
309
|
stats = {}
|
341
|
-
logger.benchmark_debug(
|
310
|
+
logger.benchmark_debug('#write ==> complete', stats) do
|
342
311
|
begin
|
343
312
|
stats[:bytes_sent] = @socket.write(data)
|
344
313
|
rescue SystemCallError => exception
|
@@ -388,40 +357,21 @@ module Net
|
|
388
357
|
# before calling _connect_ or _retry_on_connection_failure_ to create
|
389
358
|
# a new connection
|
390
359
|
#
|
391
|
-
def read(length, buffer=nil, timeout=
|
360
|
+
def read(length, buffer = nil, timeout = read_timeout)
|
392
361
|
result = nil
|
393
362
|
logger.benchmark_debug("#read <== read #{length} bytes") do
|
394
|
-
|
395
|
-
# Block on data to read for @read_timeout seconds
|
396
|
-
ready = begin
|
397
|
-
ready = IO.select([@socket], nil, [@socket], timeout || @read_timeout)
|
398
|
-
rescue IOError => exception
|
399
|
-
logger.warn "#read Connection failure while waiting for data: #{exception.class}: #{exception.message}"
|
400
|
-
close if close_on_error
|
401
|
-
raise Net::TCPClient::ConnectionFailure.new("#{exception.class}: #{exception.message}", @server, exception)
|
402
|
-
rescue Exception
|
403
|
-
# Close the connection on any other exception since the connection
|
404
|
-
# will now be in an inconsistent state
|
405
|
-
close if close_on_error
|
406
|
-
raise
|
407
|
-
end
|
408
|
-
unless ready
|
409
|
-
close if close_on_error
|
410
|
-
logger.warn "#read Timeout waiting for server to reply"
|
411
|
-
raise Net::TCPClient::ReadTimeout.new("Timedout after #{timeout || @read_timeout} seconds trying to read from #{@server}")
|
412
|
-
end
|
413
|
-
end
|
363
|
+
wait_for_data(timeout)
|
414
364
|
|
415
365
|
# Read data from socket
|
416
366
|
begin
|
417
367
|
result = buffer.nil? ? @socket.read(length) : @socket.read(length, buffer)
|
418
|
-
logger.trace(
|
368
|
+
logger.trace('#read <== received', result)
|
419
369
|
|
420
370
|
# EOF before all the data was returned
|
421
371
|
if result.nil? || (result.length < length)
|
422
372
|
close if close_on_error
|
423
373
|
logger.warn "#read server closed the connection before #{length} bytes were returned"
|
424
|
-
raise Net::TCPClient::ConnectionFailure.new(
|
374
|
+
raise Net::TCPClient::ConnectionFailure.new('Connection lost while reading data', @server, EOFError.new('end of file reached'))
|
425
375
|
end
|
426
376
|
rescue SystemCallError, IOError => exception
|
427
377
|
close if close_on_error
|
@@ -560,9 +510,9 @@ module Net
|
|
560
510
|
retries = 0
|
561
511
|
logger.benchmark_info "Connected to #{server}" do
|
562
512
|
host_name, port = server.split(":")
|
563
|
-
port
|
513
|
+
port = port.to_i
|
564
514
|
|
565
|
-
address
|
515
|
+
address = Socket.getaddrinfo(host_name, nil, Socket::AF_INET)
|
566
516
|
socket_address = Socket.pack_sockaddr_in(port, address[0][3])
|
567
517
|
|
568
518
|
begin
|
@@ -600,5 +550,50 @@ module Net
|
|
600
550
|
@server = server
|
601
551
|
end
|
602
552
|
|
553
|
+
# Try connecting to each server in the order supplied
|
554
|
+
# The next server is tried if it cannot connect to the current one
|
555
|
+
# After the last server a ConnectionFailure will be raised
|
556
|
+
def connect_to_servers_in_order(servers)
|
557
|
+
exception = nil
|
558
|
+
servers.find do |server|
|
559
|
+
begin
|
560
|
+
connect_to_server(server)
|
561
|
+
exception = nil
|
562
|
+
true
|
563
|
+
rescue Net::TCPClient::ConnectionFailure => exc
|
564
|
+
exception = exc
|
565
|
+
false
|
566
|
+
end
|
567
|
+
end
|
568
|
+
# Raise Exception once it has also failed to connect to all servers
|
569
|
+
raise(exception) if exception
|
570
|
+
end
|
571
|
+
|
572
|
+
# Return once data is ready to be ready
|
573
|
+
# Raises Net::TCPClient::ReadTimeout if the timeout is exceeded
|
574
|
+
def wait_for_data(timeout)
|
575
|
+
return if timeout == -1
|
576
|
+
|
577
|
+
ready = false
|
578
|
+
begin
|
579
|
+
ready = IO.select([@socket], nil, [@socket], timeout)
|
580
|
+
rescue IOError => exception
|
581
|
+
logger.warn "#read Connection failure while waiting for data: #{exception.class}: #{exception.message}"
|
582
|
+
close if close_on_error
|
583
|
+
raise Net::TCPClient::ConnectionFailure.new("#{exception.class}: #{exception.message}", @server, exception)
|
584
|
+
rescue Exception
|
585
|
+
# Close the connection on any other exception since the connection
|
586
|
+
# will now be in an inconsistent state
|
587
|
+
close if close_on_error
|
588
|
+
raise
|
589
|
+
end
|
590
|
+
|
591
|
+
unless ready
|
592
|
+
close if close_on_error
|
593
|
+
logger.warn "#read Timeout after #{timeout} seconds"
|
594
|
+
raise Net::TCPClient::ReadTimeout.new("Timedout after #{timeout} seconds trying to read from #{@server}")
|
595
|
+
end
|
596
|
+
end
|
597
|
+
|
603
598
|
end
|
604
599
|
end
|
@@ -0,0 +1 @@
|
|
1
|
+
require 'net/tcp_client'
|
data/test/simple_tcp_server.rb
CHANGED
@@ -1,4 +1,3 @@
|
|
1
|
-
require 'rubygems'
|
2
1
|
require 'socket'
|
3
2
|
require 'bson'
|
4
3
|
require 'semantic_logger'
|
@@ -8,7 +7,7 @@ require 'semantic_logger'
|
|
8
7
|
def read_bson_document(io)
|
9
8
|
bytebuf = BSON::ByteBuffer.new
|
10
9
|
# Read 4 byte size of following BSON document
|
11
|
-
bytes
|
10
|
+
bytes = io.read(4)
|
12
11
|
return unless bytes
|
13
12
|
# Read BSON document
|
14
13
|
sz = bytes.unpack("V")[0]
|
@@ -22,33 +21,34 @@ end
|
|
22
21
|
# Simple single threaded server for testing purposes using a local socket
|
23
22
|
# Sends and receives BSON Messages
|
24
23
|
class SimpleTCPServer
|
25
|
-
|
24
|
+
include SemanticLogger::Loggable
|
25
|
+
|
26
|
+
attr_accessor :thread, :server
|
27
|
+
|
26
28
|
def initialize(port = 2000)
|
27
29
|
start(port)
|
28
30
|
end
|
29
31
|
|
30
32
|
def start(port)
|
31
|
-
|
32
|
-
|
33
|
-
|
34
|
-
@thread = Thread.new do
|
33
|
+
self.server = TCPServer.open(port)
|
34
|
+
self.thread = Thread.new do
|
35
35
|
loop do
|
36
|
-
|
36
|
+
logger.debug 'Waiting for a client to connect'
|
37
37
|
|
38
38
|
# Wait for a client to connect
|
39
|
-
on_request(
|
39
|
+
on_request(server.accept)
|
40
40
|
end
|
41
41
|
end
|
42
42
|
end
|
43
43
|
|
44
44
|
def stop
|
45
|
-
if
|
46
|
-
|
47
|
-
|
48
|
-
|
45
|
+
if thread
|
46
|
+
thread.kill
|
47
|
+
thread.join
|
48
|
+
self.thread = nil
|
49
49
|
end
|
50
50
|
begin
|
51
|
-
|
51
|
+
server.close if server
|
52
52
|
rescue IOError
|
53
53
|
end
|
54
54
|
end
|
@@ -58,50 +58,49 @@ class SimpleTCPServer
|
|
58
58
|
def on_message(message)
|
59
59
|
case message['action']
|
60
60
|
when 'test1'
|
61
|
-
{
|
61
|
+
{'result' => 'test1'}
|
62
62
|
when 'sleep'
|
63
63
|
sleep message['duration'] || 1
|
64
|
-
{
|
64
|
+
{'result' => 'sleep'}
|
65
65
|
when 'fail'
|
66
66
|
if message['attempt'].to_i >= 2
|
67
|
-
{
|
67
|
+
{'result' => 'fail'}
|
68
68
|
else
|
69
69
|
nil
|
70
70
|
end
|
71
71
|
else
|
72
|
-
{
|
72
|
+
{'result' => "Unknown action: #{message['action']}"}
|
73
73
|
end
|
74
74
|
end
|
75
75
|
|
76
76
|
# Called for each client connection
|
77
77
|
# In a real server each request would be handled in a separate thread
|
78
78
|
def on_request(client)
|
79
|
-
|
79
|
+
logger.debug 'Client connected, waiting for data from client'
|
80
80
|
|
81
|
-
while(request = read_bson_document(client)) do
|
82
|
-
|
83
|
-
@logger.trace 'Request', request
|
81
|
+
while (request = read_bson_document(client)) do
|
82
|
+
logger.debug 'Received request', request
|
84
83
|
break unless request
|
85
84
|
|
86
85
|
if reply = on_message(request)
|
87
|
-
|
88
|
-
|
86
|
+
logger.debug 'Sending Reply'
|
87
|
+
logger.trace 'Reply', reply
|
89
88
|
client.print(BSON.serialize(reply))
|
90
89
|
else
|
91
|
-
|
92
|
-
|
90
|
+
logger.debug 'Closing client since no reply is being sent back'
|
91
|
+
server.close
|
93
92
|
client.close
|
94
|
-
|
95
|
-
|
96
|
-
|
93
|
+
logger.debug 'Server closed'
|
94
|
+
#thread.kill
|
95
|
+
logger.debug 'thread killed'
|
97
96
|
start(2000)
|
98
|
-
|
97
|
+
logger.debug 'Server Restarted'
|
99
98
|
break
|
100
99
|
end
|
101
100
|
end
|
102
101
|
# Disconnect from the client
|
103
102
|
client.close
|
104
|
-
|
103
|
+
logger.debug 'Disconnected from the client'
|
105
104
|
end
|
106
105
|
|
107
106
|
end
|
@@ -111,4 +110,4 @@ if $0 == __FILE__
|
|
111
110
|
SemanticLogger.add_appender(STDOUT)
|
112
111
|
server = SimpleTCPServer.new(2000)
|
113
112
|
server.thread.join
|
114
|
-
end
|
113
|
+
end
|
data/test/tcp_client_test.rb
CHANGED
@@ -1,42 +1,31 @@
|
|
1
|
-
# Allow test to be run in-place without requiring a gem install
|
2
|
-
$LOAD_PATH.unshift File.dirname(__FILE__) + '/../lib'
|
3
|
-
$LOAD_PATH.unshift File.dirname(__FILE__)
|
4
|
-
|
5
|
-
require 'rubygems'
|
6
|
-
require 'test/unit'
|
7
|
-
require 'shoulda'
|
8
1
|
require 'socket'
|
9
|
-
|
10
|
-
|
11
|
-
|
12
|
-
SemanticLogger.default_level = :trace
|
13
|
-
SemanticLogger.add_appender('test.log')
|
2
|
+
require_relative 'test_helper'
|
3
|
+
require_relative 'simple_tcp_server'
|
14
4
|
|
15
5
|
# Unit Test for Net::TCPClient
|
16
|
-
class TCPClientTest < Test
|
17
|
-
|
18
|
-
|
19
|
-
|
20
|
-
|
21
|
-
exception = assert_raise Net::TCPClient::ConnectionFailure do
|
6
|
+
class TCPClientTest < Minitest::Test
|
7
|
+
describe Net::TCPClient do
|
8
|
+
describe 'without server' do
|
9
|
+
it 'raises an exception when cannot reach server after 5 retries' do
|
10
|
+
exception = assert_raises Net::TCPClient::ConnectionFailure do
|
22
11
|
Net::TCPClient.new(
|
23
|
-
:
|
24
|
-
:
|
25
|
-
:
|
12
|
+
server: 'localhost:3300',
|
13
|
+
connect_retry_interval: 0.1,
|
14
|
+
connect_retry_count: 5)
|
26
15
|
end
|
27
16
|
assert_match /After 5 connection attempts to host 'localhost:3300': Errno::ECONNREFUSED/, exception.message
|
28
17
|
end
|
29
18
|
|
30
|
-
|
19
|
+
it 'times out on connect' do
|
31
20
|
# Create a TCP Server, but do not respond to connections
|
32
21
|
server = TCPServer.open(2001)
|
33
22
|
|
34
|
-
exception =
|
23
|
+
exception = assert_raises Net::TCPClient::ConnectionTimeout do
|
35
24
|
1000.times do
|
36
25
|
Net::TCPClient.new(
|
37
|
-
:
|
38
|
-
:
|
39
|
-
:
|
26
|
+
server: 'localhost:2001',
|
27
|
+
connect_timeout: 0.5,
|
28
|
+
connect_retry_count: 3
|
40
29
|
)
|
41
30
|
end
|
42
31
|
end
|
@@ -46,47 +35,47 @@ class TCPClientTest < Test::Unit::TestCase
|
|
46
35
|
|
47
36
|
end
|
48
37
|
|
49
|
-
|
50
|
-
|
51
|
-
@server
|
38
|
+
describe "with server" do
|
39
|
+
before do
|
40
|
+
@server = SimpleTCPServer.new(2000)
|
52
41
|
@server_name = 'localhost:2000'
|
53
42
|
end
|
54
43
|
|
55
|
-
|
44
|
+
after do
|
56
45
|
@server.stop if @server
|
57
46
|
end
|
58
47
|
|
59
|
-
|
60
|
-
|
48
|
+
describe 'without client connection' do
|
49
|
+
it 'times out on first receive and then successfully reads the response' do
|
61
50
|
@read_timeout = 3.0
|
62
51
|
# Need a custom client that does not auto close on error:
|
63
|
-
@client
|
64
|
-
:
|
65
|
-
:
|
66
|
-
:
|
52
|
+
@client = Net::TCPClient.new(
|
53
|
+
server: @server_name,
|
54
|
+
read_timeout: @read_timeout,
|
55
|
+
close_on_error: false
|
67
56
|
)
|
68
57
|
|
69
|
-
request = {
|
58
|
+
request = {'action' => 'sleep', 'duration' => @read_timeout + 0.5}
|
70
59
|
@client.write(BSON.serialize(request))
|
71
60
|
|
72
|
-
exception =
|
61
|
+
exception = assert_raises Net::TCPClient::ReadTimeout do
|
73
62
|
# Read 4 bytes from server
|
74
63
|
@client.read(4)
|
75
64
|
end
|
76
65
|
assert_equal false, @client.close_on_error
|
77
|
-
assert @client.alive?,
|
66
|
+
assert @client.alive?, 'The client connection is not alive after the read timed out with close_on_error: false'
|
78
67
|
assert_match /Timedout after #{@read_timeout} seconds trying to read from #{@server_name}/, exception.message
|
79
68
|
reply = read_bson_document(@client)
|
80
69
|
assert_equal 'sleep', reply['result']
|
81
70
|
@client.close
|
82
71
|
end
|
83
72
|
|
84
|
-
|
73
|
+
it 'support infinite timeout' do
|
85
74
|
@client = Net::TCPClient.new(
|
86
|
-
:
|
87
|
-
:
|
75
|
+
server: @server_name,
|
76
|
+
connect_timeout: -1
|
88
77
|
)
|
89
|
-
request = {
|
78
|
+
request = {'action' => 'test1'}
|
90
79
|
@client.write(BSON.serialize(request))
|
91
80
|
reply = read_bson_document(@client)
|
92
81
|
assert_equal 'test1', reply['result']
|
@@ -94,51 +83,51 @@ class TCPClientTest < Test::Unit::TestCase
|
|
94
83
|
end
|
95
84
|
end
|
96
85
|
|
97
|
-
|
98
|
-
|
86
|
+
describe 'with client connection' do
|
87
|
+
before do
|
99
88
|
@read_timeout = 3.0
|
100
|
-
@client
|
101
|
-
:
|
102
|
-
:
|
89
|
+
@client = Net::TCPClient.new(
|
90
|
+
server: @server_name,
|
91
|
+
read_timeout: @read_timeout
|
103
92
|
)
|
104
93
|
assert @client.alive?
|
105
94
|
assert_equal true, @client.close_on_error
|
106
95
|
end
|
107
96
|
|
108
|
-
def
|
97
|
+
def after
|
109
98
|
if @client
|
110
99
|
@client.close
|
111
100
|
assert !@client.alive?
|
112
101
|
end
|
113
102
|
end
|
114
103
|
|
115
|
-
|
116
|
-
request = {
|
104
|
+
it 'sends and receives data' do
|
105
|
+
request = {'action' => 'test1'}
|
117
106
|
@client.write(BSON.serialize(request))
|
118
107
|
reply = read_bson_document(@client)
|
119
108
|
assert_equal 'test1', reply['result']
|
120
109
|
end
|
121
110
|
|
122
|
-
|
123
|
-
request = {
|
111
|
+
it 'timeouts on receive' do
|
112
|
+
request = {'action' => 'sleep', 'duration' => @read_timeout + 0.5}
|
124
113
|
@client.write(BSON.serialize(request))
|
125
114
|
|
126
|
-
exception =
|
115
|
+
exception = assert_raises Net::TCPClient::ReadTimeout do
|
127
116
|
# Read 4 bytes from server
|
128
117
|
@client.read(4)
|
129
118
|
end
|
130
|
-
# Due to :
|
119
|
+
# Due to close_on_error: true, a timeout will close the connection
|
131
120
|
# to prevent use of a socket connection in an inconsistent state
|
132
121
|
assert_equal false, @client.alive?
|
133
122
|
assert_match /Timedout after #{@read_timeout} seconds trying to read from #{@server_name}/, exception.message
|
134
123
|
end
|
135
124
|
|
136
|
-
|
125
|
+
it 'retries on connection failure' do
|
137
126
|
attempt = 0
|
138
|
-
reply
|
139
|
-
request = {
|
127
|
+
reply = @client.retry_on_connection_failure do
|
128
|
+
request = {'action' => 'fail', 'attempt' => (attempt+=1)}
|
140
129
|
@client.write(BSON.serialize(request))
|
141
|
-
# Note: Do not put the read in this block if it
|
130
|
+
# Note: Do not put the read in this block if it never sends the
|
142
131
|
# same request twice to the server
|
143
132
|
read_bson_document(@client)
|
144
133
|
end
|
@@ -147,15 +136,15 @@ class TCPClientTest < Test::Unit::TestCase
|
|
147
136
|
|
148
137
|
end
|
149
138
|
|
150
|
-
|
151
|
-
|
139
|
+
describe 'without client connection' do
|
140
|
+
it 'connects to second server when the first is down' do
|
152
141
|
client = Net::TCPClient.new(
|
153
|
-
:
|
154
|
-
:
|
142
|
+
servers: ['localhost:1999', @server_name],
|
143
|
+
read_timeout: 3
|
155
144
|
)
|
156
145
|
assert_equal @server_name, client.server
|
157
146
|
|
158
|
-
request = {
|
147
|
+
request = {'action' => 'test1'}
|
159
148
|
client.write(BSON.serialize(request))
|
160
149
|
reply = read_bson_document(client)
|
161
150
|
assert_equal 'test1', reply['result']
|
@@ -163,19 +152,19 @@ class TCPClientTest < Test::Unit::TestCase
|
|
163
152
|
client.close
|
164
153
|
end
|
165
154
|
|
166
|
-
|
155
|
+
it 'calls on_connect after connection' do
|
167
156
|
client = Net::TCPClient.new(
|
168
|
-
:
|
169
|
-
:
|
170
|
-
:
|
157
|
+
server: @server_name,
|
158
|
+
read_timeout: 3,
|
159
|
+
on_connect: Proc.new do |socket|
|
171
160
|
# Reset user_data on each connection
|
172
|
-
socket.user_data = {
|
161
|
+
socket.user_data = {sequence: 1}
|
173
162
|
end
|
174
163
|
)
|
175
164
|
assert_equal @server_name, client.server
|
176
165
|
assert_equal 1, client.user_data[:sequence]
|
177
166
|
|
178
|
-
request = {
|
167
|
+
request = {'action' => 'test1'}
|
179
168
|
client.write(BSON.serialize(request))
|
180
169
|
reply = read_bson_document(client)
|
181
170
|
assert_equal 'test1', reply['result']
|
@@ -187,4 +176,4 @@ class TCPClientTest < Test::Unit::TestCase
|
|
187
176
|
end
|
188
177
|
|
189
178
|
end
|
190
|
-
end
|
179
|
+
end
|
data/test/test_helper.rb
ADDED
@@ -0,0 +1,15 @@
|
|
1
|
+
# Allow test to be run in-place without requiring a gem install
|
2
|
+
$LOAD_PATH.unshift File.dirname(__FILE__) + '/../lib'
|
3
|
+
|
4
|
+
# Configure Rails Environment
|
5
|
+
ENV['RAILS_ENV'] = 'test'
|
6
|
+
|
7
|
+
require 'minitest/autorun'
|
8
|
+
require 'minitest/reporters'
|
9
|
+
require 'semantic_logger'
|
10
|
+
require 'net/tcp_client'
|
11
|
+
|
12
|
+
Minitest::Reporters.use! Minitest::Reporters::SpecReporter.new
|
13
|
+
|
14
|
+
SemanticLogger.default_level = :trace
|
15
|
+
SemanticLogger.add_appender('test.log', &SemanticLogger::Appender::Base.colorized_formatter)
|
metadata
CHANGED
@@ -1,16 +1,16 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: net_tcp_client
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.0.
|
4
|
+
version: 1.0.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Reid Morrison
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2015-11-03 00:00:00.000000000 Z
|
12
12
|
dependencies: []
|
13
|
-
description: Net::TCPClient implements resilience features that
|
13
|
+
description: Net::TCPClient implements resilience features that many developers wish
|
14
14
|
was already included in the standard Ruby libraries.
|
15
15
|
email:
|
16
16
|
- reidmo@gmail.com
|
@@ -26,9 +26,11 @@ files:
|
|
26
26
|
- lib/net/tcp_client/logging.rb
|
27
27
|
- lib/net/tcp_client/tcp_client.rb
|
28
28
|
- lib/net/tcp_client/version.rb
|
29
|
+
- lib/net_tcp_client.rb
|
29
30
|
- test/simple_tcp_server.rb
|
30
31
|
- test/tcp_client_test.rb
|
31
|
-
|
32
|
+
- test/test_helper.rb
|
33
|
+
homepage: https://github.com/rocketjob/net_tcp_client
|
32
34
|
licenses:
|
33
35
|
- Apache License V2.0
|
34
36
|
metadata: {}
|
@@ -48,7 +50,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
48
50
|
version: '0'
|
49
51
|
requirements: []
|
50
52
|
rubyforge_project:
|
51
|
-
rubygems_version: 2.
|
53
|
+
rubygems_version: 2.4.5.1
|
52
54
|
signing_key:
|
53
55
|
specification_version: 4
|
54
56
|
summary: Net::TCPClient is a TCP Socket Client with built-in timeouts, retries, and
|
@@ -56,3 +58,4 @@ summary: Net::TCPClient is a TCP Socket Client with built-in timeouts, retries,
|
|
56
58
|
test_files:
|
57
59
|
- test/simple_tcp_server.rb
|
58
60
|
- test/tcp_client_test.rb
|
61
|
+
- test/test_helper.rb
|