sshkit 1.8.1 → 1.9.0.rc1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 51882b5069d3847d45a2bf42f22e40929f4bd2a8
4
- data.tar.gz: cc62423ab261347458924b9e33ca5a53d0359f8f
3
+ metadata.gz: abfe247587ee2e726dbc24ea5de49ac889398eeb
4
+ data.tar.gz: 68b375157b3a48703d6f7cc7ae46888a4f41e1b8
5
5
  SHA512:
6
- metadata.gz: a9108ac1a2b8b19233d68ab546fce120a9755e8a59709bb739c1e497f08ba4c7289a230fa1dfd59a975444579b6bf4fff7126db82ced5d37978fcde4089bb79d
7
- data.tar.gz: 115a396201a2b3e7d83b200c8db05f1cf5e5c401cd0ab8d6a921a37972f6b30f766bd842d87aa29fe0a2cc5e1f99d33ff4ae6b21098fb280902b8ae3d857c86d
6
+ metadata.gz: 1ebb765665fa9b9d82277a9346e1a94a3e65cf818095976d970d8a928c692eaf2214c4fc39b5781c0b9ada96d6743cda52010aa91dc4b71a05953d890b34729a
7
+ data.tar.gz: d0c45809d3ad62dd5c3639c3a435c9d8b796757ff6b8cd58a802c82b15493fd1c8320ec72c551c1179e2b2fb5e7ac83acce2bbf97fb5968788a7e135331abb32
@@ -1,6 +1,7 @@
1
1
  language: ruby
2
2
  rvm:
3
- - 2.2.2
4
- - 2.1.6
3
+ - 2.3.0
4
+ - 2.2.4
5
+ - 2.1.8
5
6
  - 2.0.0
6
- script: "rake test:units"
7
+ script: "rake test:units lint"
@@ -8,6 +8,44 @@ appear at the top.
8
8
  * Add your entries below here, remember to credit yourself however you want
9
9
  to be credited!
10
10
 
11
+ ## 1.9.0.rc1
12
+
13
+ ### Potentially breaking changes
14
+
15
+ * The SSHKit DSL is no longer automatically included when you `require` it.
16
+ **This means you must now explicitly `include SSHKit::DSL`.**
17
+ See [PR #219](https://github.com/capistrano/sshkit/pull/219) for details.
18
+ @beatrichartz
19
+ * `SSHKit::Backend::Printer#test` now always returns true
20
+ [PR #312](https://github.com/capistrano/sshkit/pull/312) @mikz
21
+
22
+ ### New features
23
+
24
+ * `SSHKit::Formatter::Abstract` now accepts an optional Hash of options
25
+ [PR #308](https://github.com/capistrano/sshkit/pull/308) @mattbrictson
26
+ * Add `SSHKit::Backend.current` so that Capistrano plugin authors can refactor
27
+ helper methods and still have easy access to the currently-executing Backend
28
+ without having to use global variables.
29
+ * Add `SSHKit.config.default_runner` options that allows to override default command runner.
30
+ This option also accepts a name of the custom runner class.
31
+ * The ConnectionPool has been rewritten in this release to be more efficient
32
+ and have a cleaner internal API. You can still completely disable the pool
33
+ by setting `SSHKit::Backend::Netssh.pool.idle_timeout = 0`.
34
+ @mattbrictson @byroot [PR #328](https://github.com/capistrano/sshkit/pull/328)
35
+
36
+ ### Bug fixes
37
+
38
+ * make sure working directory for commands is properly cleared after `within` blocks
39
+ [PR #307](https://github.com/capistrano/sshkit/pull/307)
40
+ @steved
41
+ * display more accurate string for commands with spaces being output in `Formatter::Pretty`
42
+ [PR #304](https://github.com/capistrano/sshkit/pull/304)
43
+ @steved
44
+ [PR #319](https://github.com/capistrano/sshkit/pull/319) @mattbrictson
45
+ * Fix a race condition experienced in JRuby that could cause multi-server
46
+ deploys to fail. [PR #322](https://github.com/capistrano/sshkit/pull/322)
47
+ @mattbrictson
48
+
11
49
  ## 1.8.1
12
50
 
13
51
  * Change license to MIT, thanks to all the patient contributors who gave
@@ -86,6 +124,7 @@ appear at the top.
86
124
  * Removed dependency on the `colorize` gem. SSHKit now implements its own ANSI color logic, with no external dependencies. Note that SSHKit now only supports the `:bold` or plain modes. Other modes will be gracefully ignored. [#263](https://github.com/capistrano/sshkit/issues/263)
87
125
  * New API for setting the formatter: `use_format`. This differs from `format=` in that it accepts options or arguments that will be passed to the formatter's constructor. The `format=` syntax will be deprecated in a future release. [#295](https://github.com/capistrano/sshkit/issues/295)
88
126
  * SSHKit now immediately raises a `NameError` if you try to set a formatter that does not exist. [#295](https://github.com/capistrano/sshkit/issues/295)
127
+ * Fix error message when the formatter does not exist. [#301](https://github.com/capistrano/sshkit/pull/301)
89
128
 
90
129
  ## 1.7.1
91
130
 
@@ -28,6 +28,16 @@ ruby versions.
28
28
  **The Travis build does not run the functional tests,
29
29
  so make sure all the tests pass locally before creating your PR.**
30
30
 
31
+ ## Coding guidelines
32
+
33
+ This project uses [RuboCop](https://github.com/bbatsov/rubocop) to enforce standard Ruby coding
34
+ guidelines. Currently we run RuboCop's lint rules only, which check for readability issues
35
+ like indentation, ambiguity, and useless/unreachable code.
36
+
37
+ * Test that your contributions pass with `rake lint`
38
+ * The linter is also run as part of the full test suite with `rake`
39
+ * Note the Travis build will fail and your PR cannot be merged if the linter finds errors
40
+
31
41
  ## Changelog
32
42
 
33
43
  Most changes should have an accompanying entry in the [CHANGELOG](CHANGELOG.md), unless they
@@ -366,12 +366,14 @@ known test cases, it works. The key thing is that `if` is not mapped to
366
366
  Into the `Rakefile` simply put something like:
367
367
 
368
368
  ```ruby
369
- require 'sshkit/dsl'
369
+ require 'sshkit'
370
370
 
371
371
  SSHKit.config.command_map[:rake] = "./bin/rake"
372
372
 
373
373
  desc "Deploy the site, pulls from Git, migrate the db and precompile assets, then restart Passenger."
374
374
  task :deploy do
375
+ include SSHKit::DSL
376
+
375
377
  on "example.com" do |host|
376
378
  within "/opt/sites/example.com" do
377
379
  execute :git, :pull
data/README.md CHANGED
@@ -3,8 +3,9 @@
3
3
  **SSHKit** is a toolkit for running commands in a structured way on one or
4
4
  more servers.
5
5
 
6
- [![Build Status](https://travis-ci.org/capistrano/sshkit.png?branch=master)](https://travis-ci.org/capistrano/sshkit)
7
- [![Dependency Status](https://gemnasium.com/leehambley/sshkit.png)](https://gemnasium.com/leehambley/sshkit)
6
+ [![Gem Version](https://badge.fury.io/rb/sshkit.svg)](https://rubygems.org/gems/sshkit)
7
+ [![Build Status](https://travis-ci.org/capistrano/sshkit.svg?branch=master)](https://travis-ci.org/capistrano/sshkit)
8
+ [![Dependency Status](https://gemnasium.com/capistrano/sshkit.svg)](https://gemnasium.com/capistrano/sshkit)
8
9
 
9
10
  ## How might it work?
10
11
 
@@ -12,7 +13,7 @@ The typical use-case looks something like this:
12
13
 
13
14
  ```ruby
14
15
  require 'sshkit'
15
- require 'sshkit/dsl'
16
+ include SSHKit::DSL
16
17
 
17
18
  on %w{1.example.com 2.example.com}, in: :sequence, wait: 5 do |host|
18
19
  within "/opt/sites/example.com" do
@@ -65,7 +66,7 @@ you can pass the `strip: false` option: `capture(:ls, '-l', strip: false)`
65
66
  #### Transferring files
66
67
 
67
68
  All backends also support the `upload!` and `download!` methods for transferring files.
68
- For the remote backed, the file is tranferred with scp.
69
+ For the remote backend, the file is tranferred with scp.
69
70
 
70
71
  ```ruby
71
72
  on '1.example.com' do
@@ -442,24 +443,22 @@ ENV['SSHKIT_COLOR'] = 'TRUE'
442
443
 
443
444
  Want custom output formatting? Here's what you have to do:
444
445
 
445
- 1. Write a new formatter class in the `SSHKit::Formatter` module. As an example, check out the default [pretty](https://github.com/capistrano/sshkit/blob/master/lib/sshkit/formatters/pretty.rb) formatter.
446
+ 1. Write a new formatter class in the `SSHKit::Formatter` module. Your class should subclass `SSHKit::Formatter::Abstract` to inherit conveniences and common behavior. For a basic an example, check out the [Pretty](https://github.com/capistrano/sshkit/blob/master/lib/sshkit/formatters/pretty.rb) formatter.
446
447
  1. Set the output format as described above. E.g. if your new formatter is called `FooBar`:
447
448
 
448
449
  ```ruby
449
450
  SSHKit.config.use_format :foobar
450
451
  ```
451
452
 
452
- If your formatter class takes a second `options` argument in its constructor, you can pass options to it like this:
453
+ All formatters that extend from `SSHKit::Formatter::Abstract` accept an options Hash as a constructor argument. You can pass options to your formatter like this:
453
454
 
454
455
  ```ruby
455
456
  SSHKit.config.use_format :foobar, :my_option => "value"
456
457
  ```
457
458
 
458
- Which will call your constructor:
459
+ You can then access these options using the `options` accessor within your formatter code.
459
460
 
460
- ```ruby
461
- SSHKit::Formatter::FooBar.new($stdout, :my_option => "value")
462
- ```
461
+ For a much more full-featured formatter example that makes use of options, check out the [Airbrussh repository](https://github.com/mattbrictson/airbrussh/).
463
462
 
464
463
  ## Output Verbosity
465
464
 
data/Rakefile CHANGED
@@ -2,8 +2,9 @@
2
2
 
3
3
  require 'bundler/gem_tasks'
4
4
  require 'rake/testtask'
5
+ require 'rubocop/rake_task'
5
6
 
6
- task :default => :test
7
+ task :default => [:test, :lint]
7
8
 
8
9
  desc "Run all tests"
9
10
  task :test => ['test:units', 'test:functional']
@@ -26,6 +27,11 @@ Rake::Task["test:functional"].enhance do
26
27
  warn "Remember there are still some VMs running, kill them with `vagrant halt` if you are finished using them."
27
28
  end
28
29
 
30
+ desc 'Run RuboCop lint checks'
31
+ RuboCop::RakeTask.new(:lint) do |task|
32
+ task.options = ['--lint']
33
+ end
34
+
29
35
  task "release:rubygem_push" do
30
36
  # Delay loading Chandler until this point, since it requires Ruby >= 2.1,
31
37
  # which may not be available in all environments (e.g. Travis).
@@ -10,6 +10,7 @@ require_relative 'configuration'
10
10
  require_relative 'coordinator'
11
11
 
12
12
  require_relative 'deprecation_logger'
13
+ require_relative 'dsl'
13
14
 
14
15
  require_relative 'exception'
15
16
 
@@ -35,4 +36,4 @@ require_relative 'backends/connection_pool'
35
36
  require_relative 'backends/printer'
36
37
  require_relative 'backends/netssh'
37
38
  require_relative 'backends/local'
38
- require_relative 'backends/skipper'
39
+ require_relative 'backends/skipper'
@@ -4,6 +4,19 @@ module SSHKit
4
4
 
5
5
  MethodUnavailableError = Class.new(SSHKit::StandardError)
6
6
 
7
+ # The Backend instance that is running in the current thread. If no Backend
8
+ # is running, returns `nil` instead.
9
+ #
10
+ # Example:
11
+ #
12
+ # on(:local) do
13
+ # self == SSHKit::Backend.current # => true
14
+ # end
15
+ #
16
+ def self.current
17
+ Thread.current["sshkit_backend"]
18
+ end
19
+
7
20
  class Abstract
8
21
 
9
22
  extend Forwardable
@@ -12,7 +25,10 @@ module SSHKit
12
25
  attr_reader :host
13
26
 
14
27
  def run
28
+ Thread.current["sshkit_backend"] = self
15
29
  instance_exec(@host, &@block)
30
+ ensure
31
+ Thread.current["sshkit_backend"] = nil
16
32
  end
17
33
 
18
34
  def initialize(host, &block)
@@ -121,8 +137,16 @@ module SSHKit
121
137
  command(args, options).tap { |cmd| execute_command(cmd) }
122
138
  end
123
139
 
140
+ def pwd_path
141
+ if @pwd.nil? || @pwd.empty?
142
+ nil
143
+ else
144
+ File.join(@pwd)
145
+ end
146
+ end
147
+
124
148
  def command(args, options)
125
- SSHKit::Command.new(*[*args, options.merge({in: @pwd.nil? ? nil : File.join(@pwd), env: @env, host: @host, user: @user, group: @group})])
149
+ SSHKit::Command.new(*[*args, options.merge({in: pwd_path, env: @env, host: @host, user: @user, group: @group})])
126
150
  end
127
151
 
128
152
  end
@@ -1,120 +1,159 @@
1
+ require "monitor"
1
2
  require "thread"
2
3
 
3
- # Since we call to_s on new_connection_args and use that as a hash
4
- # We need to make sure the memory address of the object is not used as part of the key
5
- # Otherwise identical objects with different memory address won't get a hash hit.
6
- # In the case of proxy commands, this can lead to proxy processes leaking
7
- # And in severe cases can cause deploys to fail due to default file descriptor limits
8
- # An alternate solution would be to use a different means of generating hash keys
9
- module Net; module SSH; module Proxy
10
- class Command
11
- def inspect
12
- @command_line_template
13
- end
4
+ # Since we call to_s on new connection arguments and use that as a cache key, we
5
+ # need to make sure the memory address of the object is not used as part of the
6
+ # key. Otherwise identical objects with different memory address won't reuse the
7
+ # cache.
8
+ #
9
+ # In the case of proxy commands, this can lead to proxy processes leaking, and
10
+ # in severe cases can cause deploys to fail due to default file descriptor
11
+ # limits. An alternate solution would be to use a different means of generating
12
+ # hash keys.
13
+ #
14
+ require "net/ssh/proxy/command"
15
+ class Net::SSH::Proxy::Command
16
+ # Ensure a stable string value is used, rather than memory address.
17
+ def inspect
18
+ @command_line_template
14
19
  end
15
- end;end;end
16
-
17
- module SSHKit
18
-
19
- module Backend
20
-
21
- class ConnectionPool
20
+ end
22
21
 
23
- attr_accessor :idle_timeout
22
+ # The ConnectionPool caches connections and allows them to be reused, so long as
23
+ # the reuse happens within the `idle_timeout` period. Timed out connections are
24
+ # closed, forcing a new connection to be used in that case.
25
+ #
26
+ # Additionally, a background thread is started to check for abandoned
27
+ # connections that have timed out without any attempt at being reused. These
28
+ # are eventually closed as well and removed from the cache.
29
+ #
30
+ # If `idle_timeout` set to `false`, `0`, or `nil`, no caching is performed, and
31
+ # a new connection is created and then immediately closed each time. The default
32
+ # timeout is 30 (seconds).
33
+ #
34
+ # There is a single public method: `with`. Example usage:
35
+ #
36
+ # pool = SSHKit::Backend::ConnectionPool.new
37
+ # pool.with(Net::SSH.method(:start), "host", "username") do |connection|
38
+ # # do stuff with connection
39
+ # end
40
+ #
41
+ class SSHKit::Backend::ConnectionPool
42
+ attr_accessor :idle_timeout
43
+
44
+ def initialize(idle_timeout=30)
45
+ @idle_timeout = idle_timeout
46
+ @caches = {}
47
+ @caches.extend(MonitorMixin)
48
+ @timed_out_connections = Queue.new
49
+ Thread.new { run_eviction_loop }
50
+ end
24
51
 
25
- def initialize
26
- self.idle_timeout = 30
27
- @mutex = Mutex.new
28
- @pool = {}
29
- end
52
+ # Creates a new connection or reuses a cached connection (if possible) and
53
+ # yields the connection to the given block. Connections are created by
54
+ # invoking the `connection_factory` proc with the given `args`. The arguments
55
+ # are used to construct a key used for caching.
56
+ def with(connection_factory, *args)
57
+ cache = find_cache(args)
58
+ conn = cache.pop || begin
59
+ connection_factory.call(*args)
60
+ end
61
+ yield(conn)
62
+ ensure
63
+ cache.push(conn) unless conn.nil?
64
+ end
30
65
 
31
- def checkout(*new_connection_args, &block)
32
- entry = nil
33
- key = new_connection_args.to_s
34
- if idle_timeout
35
- prune_expired
36
- entry = find_live_entry(key)
37
- end
38
- entry || create_new_entry(new_connection_args, key, &block)
39
- end
66
+ # Immediately remove all cached connections, without closing them. This only
67
+ # exists for unit test purposes.
68
+ def flush_connections
69
+ caches.synchronize { caches.clear }
70
+ end
40
71
 
41
- def checkin(entry)
42
- if idle_timeout
43
- prune_expired
44
- entry.expires_at = Time.now + idle_timeout
45
- @mutex.synchronize do
46
- @pool[entry.key] ||= []
47
- @pool[entry.key] << entry
48
- end
49
- end
50
- end
72
+ # Immediately close all cached connections and empty the pool.
73
+ def close_connections
74
+ caches.synchronize do
75
+ caches.values.each(&:clear)
76
+ caches.clear
77
+ process_deferred_close
78
+ end
79
+ end
51
80
 
52
- def close_connections
53
- @mutex.synchronize do
54
- @pool.values.flatten.map(&:connection).uniq.each do |conn|
55
- if conn.respond_to?(:closed?) && conn.respond_to?(:close)
56
- conn.close unless conn.closed?
57
- end
58
- end
59
- @pool.clear
60
- end
61
- end
81
+ private
62
82
 
63
- def flush_connections
64
- @mutex.synchronize { @pool.clear }
65
- end
83
+ attr_reader :caches, :timed_out_connections
66
84
 
67
- private
68
-
69
- def prune_expired
70
- @mutex.synchronize do
71
- @pool.each_value do |entries|
72
- entries.collect! do |entry|
73
- if entry.expired?
74
- entry.close unless entry.closed?
75
- nil
76
- else
77
- entry
78
- end
79
- end.compact!
80
- end
81
- end
82
- end
85
+ def cache_enabled?
86
+ idle_timeout && idle_timeout > 0
87
+ end
83
88
 
84
- def find_live_entry(key)
85
- @mutex.synchronize do
86
- return nil unless @pool.key?(key)
87
- while (entry = @pool[key].shift)
88
- return entry if entry.live?
89
- end
90
- end
91
- nil
92
- end
89
+ # Look up a Cache that matches the given connection arguments.
90
+ def find_cache(args)
91
+ if cache_enabled?
92
+ key = args.to_s
93
+ caches[key] || thread_safe_find_or_create_cache(key)
94
+ else
95
+ NilCache.new(method(:silently_close_connection))
96
+ end
97
+ end
93
98
 
94
- def create_new_entry(args, key, &block)
95
- Entry.new block.call(*args), key
99
+ # Cache creation needs to happen in a mutex, because otherwise a race
100
+ # condition might cause two identical caches to be created for the same key.
101
+ def thread_safe_find_or_create_cache(key)
102
+ caches.synchronize do
103
+ caches[key] ||= begin
104
+ Cache.new(idle_timeout, method(:silently_close_connection_later))
96
105
  end
106
+ end
107
+ end
97
108
 
98
- Entry = Struct.new(:connection, :key) do
99
- attr_accessor :expires_at
100
-
101
- def live?
102
- !expired? && !closed?
103
- end
109
+ # Loops indefinitely to close connections and to find abandoned connections
110
+ # that need to be closed.
111
+ def run_eviction_loop
112
+ loop do
113
+ process_deferred_close
104
114
 
105
- def expired?
106
- expires_at && Time.now > expires_at
107
- end
115
+ # Periodically sweep all Caches to evict stale connections
116
+ sleep([idle_timeout, 5].min)
117
+ caches.values.each(&:evict)
118
+ end
119
+ end
108
120
 
109
- def close
110
- connection.respond_to?(:close) && connection.close
111
- end
121
+ # Immediately close any connections that are pending closure.
122
+ # rubocop:disable Lint/HandleExceptions
123
+ def process_deferred_close
124
+ until timed_out_connections.empty?
125
+ connection = timed_out_connections.pop(true)
126
+ silently_close_connection(connection)
127
+ end
128
+ rescue ThreadError
129
+ # Queue#pop(true) raises ThreadError if the queue is empty.
130
+ # This could only happen if `close_connections` is called at the same time
131
+ # the background eviction thread has woken up to close connections. In any
132
+ # case, it is not something we need to care about, since an empty queue is
133
+ # perfectly OK.
134
+ end
135
+ # rubocop:enable Lint/HandleExceptions
112
136
 
113
- def closed?
114
- connection.respond_to?(:closed?) && connection.closed?
115
- end
116
- end
137
+ # Adds the connection to a queue that is processed asynchronously by a
138
+ # background thread. The connection will eventually be closed.
139
+ def silently_close_connection_later(connection)
140
+ timed_out_connections << connection
141
+ end
117
142
 
118
- end
143
+ # Close the given `connection` immediately, assuming it responds to a `close`
144
+ # method. If it doesn't, or if `nil` is provided, it is silently ignored. Any
145
+ # `StandardError` is also silently ignored. Returns `true` if the connection
146
+ # was closed; `false` if it was already closed or could not be closed due to
147
+ # an error.
148
+ def silently_close_connection(connection)
149
+ return false unless connection.respond_to?(:close)
150
+ return false if connection.respond_to?(:closed?) && connection.closed?
151
+ connection.close
152
+ true
153
+ rescue StandardError
154
+ false
119
155
  end
120
156
  end
157
+
158
+ require "sshkit/backends/connection_pool/cache"
159
+ require "sshkit/backends/connection_pool/nil_cache"