net-ssh 8.0.0.rc3 → 8.0.0.rc4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 03f0ed4c48532bcfa337a176d655407155043de284d1428cd51a4bf2b7cb2f58
-  data.tar.gz: 49e8bbef56ac3139f5bb05cfad70afb33dda44227e9ce17f4424bd5aed749b95
+  metadata.gz: 475db99174313b32d90dc650a358dd0cbec01ae59b0daca8f2e02eeb97a8cb9e
+  data.tar.gz: 9cdb86ac170fa3b3ca90357a106bc9742210dd1eae61e7525f4b36f71342a590
 SHA512:
-  metadata.gz: 56ec747abdff1eb6a0cbfb03adb3f880246372afe864f421c4241881671b9ce49dc1dc37a30c9e75e52526284b7c05caab1f14d367f94a1cdd422e1cf4aea93d
-  data.tar.gz: df092fc1f9da4c85eaf3b305890e0635b9d16d72b3bf05711180a9c28652d97a987be3011bbf7610363d0a9412dafa056c527172519a92c2bf77e7f228bcade8
+  metadata.gz: 6ada5144a2d5b86aae33b317a42a915b9107731b921fd01bfac3678da06499e9e1fd9c5f4bb310787f345ff8279445c82b00fb129c1837341d40fe7f9eea1779
+  data.tar.gz: 0cf535957b80c52960df55d93073efff00aa7cf89a9e7323a4dc38e59833ae0bf3a2e419c032c282ffe38bb14234486436c4ff151931dff73d0330229a83508c

data/CHANGES.txt

@@ -1,3 +1,17 @@
+=== 8.0.0 rc4
+
+  * Security: RSA signatures using SHA-1 (`ssh-rsa`, `ssh-rsa-cert-v01@openssh.com`) are no longer offered by default for host keys or public key authentication, as in OpenSSH since 8.8; RSA keys keep working via rsa-sha2-256/512. For servers that only support SHA-1 add `HostKeyAlgorithms +ssh-rsa` / `PubkeyAcceptedAlgorithms +ssh-rsa` in ssh_config (or `host_key: '+ssh-rsa'` / `pubkey_algorithms: %w[+ssh-rsa]`), or pass `append_all_supported_algorithms: true` [#1039]
+  * Security: ssh_config `PubkeyAcceptedAlgorithms` (and its old name `PubkeyAcceptedKeyTypes`) is now honoured, including `+`, `-` and `^`, and `pubkey_algorithms` now applies to all key types instead of only RSA, adding `rsa-sha2-512`; DSA (`ssh-dss`) keys are no longer offered by default, as in OpenSSH since 7.0 — add `+ssh-dss` or pass `append_all_supported_algorithms: true` to re-enable [#923]
+  * Security: ssh_config `Ciphers`, `MACs`, `KexAlgorithms` and `HostKeyAlgorithms` lists starting with `+`, `-` or `^` now append to, remove from or prepend to the default list, as in OpenSSH, instead of silently enabling every supported algorithm including weak ones that are off by default; every listed item is honoured and `-` accepts wildcards. Pass `append_all_supported_algorithms: true` to also get the full supported list [#925]
+  * Support agent-backed FIDO/U2F security keys (sk-ecdsa-sha2-nistp256@openssh.com, sk-ssh-ed25519@openssh.com) for publickey auth [#1007]
+  * Port forwarding no longer buffers a whole stream in memory when one side reads slowly: like OpenSSH, window is granted only as data is written to the local socket, and the local socket is read only while the remote window has room. An application that writes everything to a forwarded port before reading anything now blocks once the window fills, as it would with `ssh -L` or a direct TCP connection; the old unbounded buffering hid this [#1037, #1003]
+  * Data forwarded to a local socket just before the remote end sends EOF or closes the channel is no longer truncated or dropped: it is written out before the socket is shut down or closed, without blocking the event loop until the local application reads [#1037]
+  * Multi-factor authentication works when the server's AuthenticationMethods order differs from net-ssh's: after each partial success the allowed methods are tried again, as OpenSSH does [#873, #872]
+  * `HostKeyError#host`, `#port` and `#ip` now return the connected host, port and IP (`ip` is nil through a ProxyCommand) instead of always nil [#963]
+  * OpenSSH-format private keys with Windows (CRLF) line endings now load [#922, #921]
+  * Send CHANNEL_EOF before CHANNEL_CLOSE when closing a session's remaining channels, as OpenSSH does; fixes hangs at close with servers that wait for EOF, such as some SFTP servers [#672]
+  * ssh_config: trailing `# comments` are now ignored, as in OpenSSH; a `#` inside quotes or in the middle of a word is kept [#990]
+
 === 8.0.0 rc3
 
   * Fix "Calling Net::SSH::Buffer methods on HostKeyEntries PubKey is deprecated" warnings printed on every connection to a host in known_hosts [#1026, #1027]

data/README.md

@@ -39,13 +39,17 @@ Unsecure algoritms will definitely be removed in Net::SSH 8.*.
 
 | Name                 | Support               | Details  |
 |----------------------|-----------------------|----------|
-| ssh-rsa              | OK                    |          |
+| rsa-sha2-512         | OK                    |          |
+| rsa-sha2-256         | OK                    |          |
+| ssh-rsa              | Disabled by default in 8.0 | SHA-1 signatures, off by default as in OpenSSH 8.8; enable with `HostKeyAlgorithms +ssh-rsa` or `host_key: '+ssh-rsa'` |
 | ssh-ed25519          | OK                    | Backed by Ruby OpenSSL raw Ed25519 APIs (`openssl >= 3.2.0`) |
 | ecdsa-sha2-nistp521  | OK                    | [using weak elliptic curves](https://safecurves.cr.yp.to/) |
 | ecdsa-sha2-nistp384  | OK                    | [using weak elliptic curves](https://safecurves.cr.yp.to/) |
 | ecdsa-sha2-nistp256  | OK                    | [using weak elliptic curves](https://safecurves.cr.yp.to/) |
 | ssh-dss              | Deprecated in 6.0     | unsecure, will be removed in 8.0 |
 
+RSA public key authentication likewise uses rsa-sha2-512 or rsa-sha2-256 by default; ssh-rsa and ssh-dss keys are only offered when enabled, e.g. with `PubkeyAcceptedAlgorithms +ssh-rsa` or `pubkey_algorithms: %w[+ssh-rsa]`.
+
 ### Key Exchange
 
 | Name                                 | Support               | Details  |
@@ -194,6 +198,11 @@ Ed25519 public key auth is backed by Ruby OpenSSL. The `bcrypt_pbkdf` gem is onl
 gem install bcrypt_pbkdf
 ```
 
+### FIDO/U2F security keys
+
+`sk-ecdsa-sha2-nistp256@openssh.com` and `sk-ssh-ed25519@openssh.com` keys work when held in `ssh-agent`.
+Loading `sk-*` private key files from disk is not supported.
+
 ## RUBY SUPPORT
 
 * See [net-ssh.gemspec](https://github.com/net-ssh/net-ssh/blob/master/net-ssh.gemspec) for current versions ruby requirements

data/lib/net/ssh/authentication/ed25519.rb

@@ -74,7 +74,7 @@ module Net
           end
 
           def self.read(datafull, password)
-            datafull = datafull.strip
+            datafull = datafull.strip.encode(universal_newline: true)
             raise ArgumentError.new("Expected #{MBEGIN} at start of private key") unless datafull.start_with?(MBEGIN)
             raise ArgumentError.new("Expected #{MEND} at end of private key") unless datafull.end_with?(MEND)
 

data/lib/net/ssh/authentication/methods/abstract.rb

@@ -20,9 +20,7 @@ module Net
           # this.
           attr_reader :key_manager
 
-          # So far only affects algorithms used for rsa keys, but can be
-          # extended to other keys, e.g after reading of
-          # PubkeyAcceptedAlgorithms option from ssh_config file is implemented.
+          # public key algorithms to be used
           attr_reader :pubkey_algorithms
 
           # Instantiates a new authentication method.
@@ -31,11 +29,7 @@ module Net
             @key_manager = options[:key_manager]
             @options = options
             @prompt = options[:password_prompt]
-            @pubkey_algorithms = options[:pubkey_algorithms] \
-              || %w[rsa-sha2-256-cert-v01@openssh.com
-                    ssh-rsa-cert-v01@openssh.com
-                    rsa-sha2-256
-                    ssh-rsa]
+            @pubkey_algorithms = session.transport.pubkey_algorithms
             self.logger = session.logger
           end
 

data/lib/net/ssh/authentication/methods/publickey.rb

@@ -15,6 +15,8 @@ module Net
           def authenticate(next_service, username, password = nil)
             return false unless key_manager
 
+            debug { "pubkey_algorithms: #{pubkey_algorithms.join(', ')}" }
+
             key_manager.each_identity do |identity|
               return true if authenticate_with(identity, next_service, username)
             end
@@ -123,7 +125,7 @@ module Net
                   end
                 end
               end
-            elsif authenticate_with_alg(identity, next_service, username, type)
+            elsif pubkey_algorithms.include?(type) && authenticate_with_alg(identity, next_service, username, type)
               # success
               return true
             end

data/lib/net/ssh/authentication/security_key.rb

@@ -0,0 +1,70 @@
+require 'net/ssh/authentication/pub_key_fingerprint'
+
+module Net
+  module SSH
+    module Authentication
+      # FIDO/U2F security keys held by ssh-agent. The agent does the signing (incl. user
+      # presence); net-ssh only parses and re-serialises the public key blob (see PROTOCOL.u2f).
+      module SecurityKey
+        SK_ECDSA_SHA2_NISTP256 = "sk-ecdsa-sha2-nistp256@openssh.com"
+        SK_SSH_ED25519 = "sk-ssh-ed25519@openssh.com"
+        TYPES = [SK_ECDSA_SHA2_NISTP256, SK_SSH_ED25519].freeze
+
+        def self.understands?(type)
+          TYPES.include?(type)
+        end
+
+        # Public key of an agent-held security key; signing is delegated to the agent.
+        class PubKey
+          include Net::SSH::Authentication::PubKeyFingerprint
+
+          attr_reader :ssh_type, :curve, :public_key_data, :application
+
+          # Wire formats:
+          #   sk-ecdsa-sha2-nistp256@openssh.com: string curve, string Q, string application
+          #   sk-ssh-ed25519@openssh.com:         string public_key, string application
+          def self.read_keyblob(type, buffer)
+            case type
+            when SK_ECDSA_SHA2_NISTP256
+              curve = buffer.read_string
+              public_key_data = buffer.read_string
+              application = buffer.read_string
+              new(type, public_key_data, application, curve: curve)
+            when SK_SSH_ED25519
+              public_key_data = buffer.read_string
+              application = buffer.read_string
+              new(type, public_key_data, application)
+            else
+              raise NotImplementedError, "unsupported security key type `#{type}'"
+            end
+          end
+
+          def initialize(ssh_type, public_key_data, application, curve: nil)
+            @ssh_type = ssh_type
+            @public_key_data = public_key_data
+            @application = application
+            @curve = curve
+          end
+
+          def to_blob
+            buffer = Net::SSH::Buffer.new
+            buffer.write_string(ssh_type)
+            buffer.write_string(curve) if curve
+            buffer.write_string(public_key_data)
+            buffer.write_string(application)
+            buffer.to_s
+          end
+
+          def ssh_signature_type
+            ssh_type
+          end
+
+          # Only used to de-duplicate identities, so a stable key-specific string suffices.
+          def to_pem
+            "#{ssh_type} #{[to_blob].pack('m0')}"
+          end
+        end
+      end
+    end
+  end
+end

data/lib/net/ssh/authentication/session.rb

@@ -69,27 +69,22 @@ module Net
 
           attempted = []
 
-          @auth_methods.each do |name|
-            next unless @allowed_auth_methods.include?(name)
-
-            attempted << name
-
-            debug { "trying #{name}" }
-            begin
-              auth_class = Methods.const_get(name.split(/\W+/).map { |p| p.capitalize }.join)
-              method = auth_class.new(self,
-                                      key_manager: key_manager, password_prompt: options[:password_prompt],
-                                      pubkey_algorithms: options[:pubkey_algorithms] || nil)
-            rescue NameError
-              debug {"Mechanism #{name} was requested, but isn't a known type.  Ignoring it."}
-              next
+          # After a partial success (sshd AuthenticationMethods) restart from the most preferred method the server now allows, as OpenSSH does.
+          loop do
+            @partial_success = false
+
+            @auth_methods.each do |name|
+              next unless @allowed_auth_methods.include?(name)
+
+              attempted << name
+              return true if try_authenticate(name, next_service, username, password, key_manager)
+              break if @partial_success
             end
 
-            return true if method.authenticate(next_service, username, password)
-          rescue Net::SSH::Authentication::DisallowedMethod
+            break unless @partial_success
           end
 
-          error { "all authorization methods failed (tried #{attempted.join(', ')})" }
+          error { "all authorization methods failed (tried #{attempted.uniq.join(', ')})" }
           return false
         ensure
           key_manager.finish if key_manager
@@ -109,6 +104,7 @@ module Net
 
             when USERAUTH_FAILURE
               @allowed_auth_methods = packet[:authentications].split(/,/)
+              @partial_success ||= packet[:partial_success]
               debug { "allowed methods: #{packet[:authentications]}" }
               return packet
 
@@ -136,6 +132,22 @@ module Net
 
         private
 
+        def try_authenticate(method_name, next_service, username, password, key_manager)
+          debug { "trying #{method_name}" }
+          begin
+            auth_class = Methods.const_get(method_name.split(/\W+/).map { |p| p.capitalize }.join)
+            method = auth_class.new(self,
+                                    key_manager: key_manager, password_prompt: options[:password_prompt])
+          rescue NameError
+            debug {"Mechanism #{method_name} was requested, but isn't a known type.  Ignoring it."}
+            return false
+          end
+
+          method.authenticate(next_service, username, password)
+        rescue Net::SSH::Authentication::DisallowedMethod
+          false
+        end
+
         # Returns an array of paths to the key files usually defined
         # by system default.
         def default_keys

data/lib/net/ssh/buffer.rb

@@ -2,6 +2,7 @@ require 'net/ssh/transport/openssl'
 
 require 'net/ssh/authentication/certificate'
 require 'net/ssh/authentication/ed25519_loader'
+require 'net/ssh/authentication/security_key'
 
 module Net
   module SSH
@@ -338,6 +339,9 @@ module Net
           key = Net::SSH::Authentication::ED25519::PubKey.read_keyblob(self)
         when /^ecdsa\-sha2\-(\w*)$/
           key = OpenSSL::PKey::EC.read_keyblob($1, self)
+        when Net::SSH::Authentication::SecurityKey::SK_ECDSA_SHA2_NISTP256,
+          Net::SSH::Authentication::SecurityKey::SK_SSH_ED25519
+          key = Net::SSH::Authentication::SecurityKey::PubKey.read_keyblob(type, self)
         else
           raise NotImplementedError, "unsupported key type `#{type}'"
         end

data/lib/net/ssh/buffered_io.rb

@@ -166,6 +166,11 @@ module Net
     #    http://github.com/net-ssh/net-ssh/tree/portfwfix
     #
     module ForwardedBufferedIo
+      def self.extended(object) # :nodoc:
+        object.instance_variable_set(:@on_sent, nil)
+        object.instance_variable_set(:@shutdown_write_when_drained, false)
+      end
+
       def fill(n = 8192)
         begin
           super(n)
@@ -182,11 +187,25 @@ module Net
         end
       end
 
+      # Internal and experimental: registers a block called with the number of
+      # bytes each #send_pending wrote to the socket.
+      def on_sent(&block) # :nodoc:
+        @on_sent = block
+      end
+
       def send_pending
         begin
-          super
-        rescue Errno::ECONNRESET => e
-          debug { "connection was reset => shallowing exception:#{e}" }
+          pending = output.length
+          result = super
+          @on_sent.call(pending - output.length) if @on_sent && output.length < pending
+          shutdown_write_if_drained
+          result
+        rescue Errno::EPIPE, Errno::ECONNRESET => e
+          # Like OpenSSH's chan_write_failed: the peer is gone, so drop what it
+          # will never read and close, letting the forwarded channel close too.
+          debug { "write failed => discarding #{output.length} bytes and closing:#{e}" }
+          output.clear!
+          close
           return 0
         rescue IOError => e
           if e.message =~ /closed/ then
@@ -197,6 +216,24 @@ module Net
           end
         end
       end
+
+      # Internal: shuts down the write side once pending output has been sent,
+      # as OpenSSH does on CHANNEL_EOF (chan_rcvd_ieof, chan_obuf_empty).
+      def shutdown_write_when_drained # :nodoc:
+        @shutdown_write_when_drained = true
+        shutdown_write_if_drained
+      end
+
+      private
+
+      def shutdown_write_if_drained
+        return unless @shutdown_write_when_drained && output.empty? && !closed?
+
+        @shutdown_write_when_drained = false
+        shutdown(Socket::SHUT_WR)
+      rescue Errno::EPIPE, Errno::ENOTCONN => e
+        debug { "shutdown failed => shallowing exception:#{e}" }
+      end
     end
   end
 end

data/lib/net/ssh/config.rb

@@ -31,6 +31,7 @@ module Net
     # * PreferredAuthentications => maps to the :auth_methods option
     # * ProxyCommand => maps to the :proxy option
     # * ProxyJump => maps to the :proxy option
+    # * PubkeyAcceptedAlgorithms (or PubkeyAcceptedKeyTypes) => maps to :pubkey_algorithms
     # * PubKeyAuthentication => maps to the :auth_methods option
     # * RekeyLimit => :rekey_limit
     # * StrictHostKeyChecking => :verify_host_key
@@ -95,6 +96,7 @@ module Net
             next if value.nil?
 
             key.downcase!
+            value = uncomment(value)
             value = unquote(value)
 
             value = case value.strip
@@ -230,7 +232,7 @@ module Net
           user: :user,
           checkhostip: :check_host_ip
         }.freeze
-        def translate_config_key(hash, key, value, settings)
+        def translate_config_key(hash, key, value, settings) # rubocop:disable Metrics/AbcSize
           case key
           when :stricthostkeychecking
             hash[:verify_host_key] = translate_verify_host_key(value)
@@ -276,6 +278,8 @@ module Net
             if (proxy = setup_proxy(*value))
               hash[:proxy] = proxy
             end
+          when :pubkeyacceptedalgorithms, :pubkeyacceptedkeytypes
+            hash[:pubkey_algorithms] = value.split(/,/)
           when :pubkeyauthentication
             if value
               (hash[:auth_methods] << 'publickey').uniq!
@@ -399,6 +403,11 @@ module Net
           !condition_matches.empty? && condition_matches.all?
         end
 
+        # As in OpenSSH's argv_split, "#" starts a comment only at the start of a token and outside quotes.
+        def uncomment(string)
+          string[/\A(?:\s*(?!#)(?:\\.|"[^"]*"?|'[^']*'?|[^\s"'])+)*/]
+        end
+
         def unquote(string)
           string =~ /^"(.*)"$/ ? Regexp.last_match(1) : string
         end

data/lib/net/ssh/connection/channel.rb

@@ -118,6 +118,8 @@ module Net
 
           @local_maximum_packet_size = max_pkt_size
           @local_window_size = @local_maximum_window_size = max_win_size
+          @local_consumed = 0
+          @grant_window_on_consume = false
 
           @on_confirm_open = on_confirm_open
 
@@ -254,6 +256,26 @@ module Net
           output.append(data.to_s)
         end
 
+        # Internal and experimental, used by port forwarding: from now on window
+        # is granted only for bytes reported via #consumed, so a slow consumer
+        # stalls the remote sender instead of buffering everything it sends.
+        # Must be called before any data is received.
+        def grant_window_on_consume! # :nodoc:
+          raise ArgumentError, "grant_window_on_consume! must be called before any data is received" if local_window_size != local_maximum_window_size
+
+          @grant_window_on_consume = true
+        end
+
+        # Internal and experimental: reports that +bytes+ of received data have
+        # been processed, letting the remote end send that much more.
+        def consumed(bytes) # :nodoc:
+          raise ArgumentError, "consumed requires grant_window_on_consume!" unless @grant_window_on_consume
+          raise ArgumentError, "consumed bytes must not be negative" if bytes.negative?
+
+          @local_consumed += bytes
+          adjust_window_for_consumed
+        end
+
         # Returns true if the channel exists in the channel list of the session,
         # and false otherwise. This can be used to determine whether a channel has
         # been closed or not.
@@ -598,7 +620,12 @@ module Net
         # type, and data.
         def do_extended_data(type, data)
           update_local_window_size(data.length)
-          @on_extended_data.call(self, type, data) if @on_extended_data
+          if @on_extended_data
+            @on_extended_data.call(self, type, data)
+          elsif @grant_window_on_consume
+            # Nothing will ever consume it, so don't let it shrink the window.
+            consumed(data.length)
+          end
         end
 
         # Invokes the #on_eof callback when the server indicates that no
@@ -651,6 +678,8 @@ module Net
         # server telling it that the window size has grown.
         def update_local_window_size(size)
           @local_window_size -= size
+          return adjust_window_for_consumed if @grant_window_on_consume
+
           if local_window_size < local_maximum_window_size / 2
             connection.send_message(
               Buffer.from(:byte, CHANNEL_WINDOW_ADJUST, :long, remote_id, :long, LOCAL_WINDOW_SIZE_INCREMENT)
@@ -663,6 +692,21 @@ module Net
           end
         end
 
+        # Mirrors OpenSSH's channel_check_window: once the window is below half,
+        # give back what was consumed, growing the maximum window the same way
+        # #update_local_window_size does.
+        def adjust_window_for_consumed
+          return if @local_consumed.zero? || local_window_size >= local_maximum_window_size / 2
+          return if remote_id.nil? || local_closed? || remote_closed?
+
+          growth = (GOOD_LOCAL_MAXIUMUM_WINDOW_SIZE - local_maximum_window_size).clamp(0, LOCAL_WINDOW_SIZE_INCREMENT)
+          adjust = @local_consumed + growth
+          connection.send_message(Buffer.from(:byte, CHANNEL_WINDOW_ADJUST, :long, remote_id, :long, adjust))
+          @local_window_size += adjust
+          @local_maximum_window_size += growth
+          @local_consumed = 0
+        end
+
         # Gets an +Array+ of local environment variables in the remote process'
         # environment.
         # A variable name can either be described by a +Regexp+ or +String+.

data/lib/net/ssh/connection/session.rb

@@ -24,7 +24,7 @@ module Net
       #     # 'ssh' is an instance of Net::SSH::Connection::Session
       #     ssh.exec! "/etc/init.d/some_process start"
       #   end
-      class Session
+      class Session # rubocop:disable Metrics/ClassLength
         include Loggable
         include Constants
 
@@ -74,6 +74,8 @@ module Net
           @channel_id_counter = -1
           @channels = Hash.new(NilChannel.new(self))
           @listeners = { transport.socket => nil }
+          @read_paused = {}
+          @draining = {}
           @pending_requests = []
           @channel_open_handlers = {}
           @on_global_request = {}
@@ -120,12 +122,19 @@ module Net
         # connection.
         def close
           info { "closing remaining channels (#{channels.length} open)" }
-          channels.each { |id, channel| channel.close }
+
+          channels.each do |id, channel|
+            channel.eof!
+            channel.close
+          end
+
           begin
             loop(0.1) { channels.any? }
           rescue Net::SSH::Disconnect
             raise unless channels.empty?
           end
+          @draining.each_key(&:close)
+          @draining.clear
           transport.close
         end
 
@@ -146,10 +155,13 @@ module Net
         # a +true+ value for +include_invisible+, then those will be counted.
         #
         # This can be useful for determining whether the event loop should continue
-        # to be run.
+        # to be run. A forwarded socket still writing out data that arrived before
+        # its channel closed also counts as busy.
         #
         #   ssh.loop { ssh.busy? }
         def busy?(include_invisible = false)
+          return true if @draining.any?
+
           if include_invisible
             channels.any?
           else
@@ -255,7 +267,7 @@ module Net
         def ev_do_calculate_rw_wait(wait)
           r = listeners.keys
           w = r.select { |w2| w2.respond_to?(:pending_write?) && w2.pending_write? }
-          [r, w, io_select_wait(wait)]
+          [r.reject { |io| @read_paused.key?(io) }, w, io_select_wait(wait)]
         end
 
         # This is called internally as part of #process.
@@ -281,6 +293,7 @@ module Net
 
           Array(writers).each do |writer|
             writer.send_pending
+            close_if_drained(writer) if @draining.key?(writer)
           end
         end
 
@@ -485,9 +498,36 @@ module Net
         # Removes the given io object from the listeners collection, so that the
         # event loop will no longer monitor it.
         def stop_listening_to(io)
+          @read_paused.delete(io)
+          @draining.delete(io)
           listeners.delete(io)
         end
 
+        # Internal and experimental, used by port forwarding: stops the event
+        # loop from reading +io+ while it keeps flushing its pending output.
+        def pause_reading(io) # :nodoc:
+          raise ArgumentError, "the transport socket can't be paused" if io.equal?(transport.socket)
+
+          @read_paused[io] = true if listeners.key?(io)
+        end
+
+        def resume_reading(io) # :nodoc:
+          @read_paused.delete(io)
+        end
+
+        def reading_paused?(io) # :nodoc:
+          @read_paused.key?(io)
+        end
+
+        # Internal and experimental, used by port forwarding: stops reading +io+
+        # but keeps flushing its pending output, then closes it and stops
+        # listening, as OpenSSH's chan_rcvd_oclose does.
+        def close_when_drained(io) # :nodoc:
+          pause_reading(io)
+          @draining[io] = true
+          close_if_drained(io)
+        end
+
         # Returns a reference to the Net::SSH::Service::Forward service, which can
         # be used for forwarding ports over SSH.
         def forward
@@ -565,6 +605,13 @@ module Net
           end
         end
 
+        def close_if_drained(io)
+          return if io.pending_write? && !io.closed? && !closed? && listeners.key?(io)
+
+          io.close unless io.closed?
+          stop_listening_to(io)
+        end
+
         def channel_closed(channel)
           channel.remote_closed!
           channel.close

data/lib/net/ssh/errors.rb

@@ -62,19 +62,21 @@ module Net
         @data && @data[:fingerprint]
       end
 
-      # Returns the host name for the remote host, as reported by the socket.
+      # Returns the host name that was connected to.
       def host
-        @data && @data[:peer] && @data[:peer][:host]
+        session&.host
       end
 
-      # Returns the port number for the remote host, as reported by the socket.
+      # Returns the port number that was connected to.
       def port
-        @data && @data[:peer] && @data[:peer][:port]
+        session&.port
       end
 
-      # Returns the IP address of the remote host, as reported by the socket.
+      # Returns the IP address of the remote host, as reported by the socket,
+      # or nil when connected through a ProxyCommand.
       def ip
-        @data && @data[:peer] && @data[:peer][:ip]
+        peer_ip = session&.socket&.peer_ip
+        peer_ip unless peer_ip == Net::SSH::Transport::PacketStream::PROXY_COMMAND_HOST_IP
       end
 
       # Returns the key itself, as reported by the remote host.
@@ -87,6 +89,12 @@ module Net
       def remember_host!
         @callback.call
       end
+
+      private
+
+      def session
+        @data && @data[:session]
+      end
     end
 
     # Raised when the cached key for a particular host does not match the

data/lib/net/ssh/key_factory.rb

@@ -2,6 +2,7 @@ require 'net/ssh/transport/openssl'
 require 'net/ssh/prompt'
 
 require 'net/ssh/authentication/ed25519_loader'
+require 'net/ssh/authentication/security_key'
 
 module Net
   module SSH
@@ -92,7 +93,7 @@ module Net
           blob = nil
           begin
             blob = fields.shift
-          end while !blob.nil? && !/^(ssh-(rsa|dss|ed25519)|ecdsa-sha2-nistp\d+)(-cert-v01@openssh\.com)?$/.match(blob)
+          end while !blob.nil? && !public_key_type?(blob)
           blob = fields.shift
 
           raise Net::SSH::Exception, "public key at #{filename} is not valid" if blob.nil?
@@ -104,6 +105,11 @@ module Net
 
         private
 
+        def public_key_type?(type)
+          /^(ssh-(rsa|dss|ed25519)|ecdsa-sha2-nistp\d+)(-cert-v01@openssh\.com)?$/.match?(type) ||
+            Net::SSH::Authentication::SecurityKey.understands?(type)
+        end
+
         # rubocop:disable Style/Documentation, Lint/DuplicateMethods
         class KeyType
           def self.read(key_data, passphrase)

data/lib/net/ssh/service/forward.rb

@@ -320,6 +320,11 @@ module Net
           session.listen_to(client)
           channel[:socket] = client
 
+          # Grant window only for data the local end has accepted, so a slow
+          # reader stalls the remote sender instead of filling our memory.
+          channel.grant_window_on_consume!
+          client.on_sent { |bytes| channel.consumed(bytes) }
+
           channel.on_data do |ch, data|
             debug { "data:#{data.length} on #{type} forwarded channel" }
             ch[:socket].enqueue(data)
@@ -327,36 +332,36 @@ module Net
 
           channel.on_eof do |ch|
             debug { "eof #{type} on #{type} forwarded channel" }
-            begin
-              ch[:socket].send_pending
-              ch[:socket].shutdown Socket::SHUT_WR
-            rescue IOError => e
-              if e.message =~ /closed/ then
-                debug { "epipe in on_eof => shallowing exception:#{e}" }
-              else
-                raise
-              end
-            rescue Errno::EPIPE => e
-              debug { "epipe in on_eof => shallowing exception:#{e}" }
-            rescue Errno::ENOTCONN => e
-              debug { "enotconn in on_eof => shallowing exception:#{e}" }
-            end
+            ch[:socket].shutdown_write_when_drained
           end
 
           channel.on_close do |ch|
             debug { "closing #{type} forwarded channel" }
-            ch[:socket].close if !client.closed?
-            session.stop_listening_to(ch[:socket])
+            session.close_when_drained(ch[:socket])
           end
 
           channel.on_process do |ch|
-            if ch[:socket].closed?
+            socket = ch[:socket]
+            if socket.closed?
               ch.info { "#{type} forwarded connection closed" }
+              session.stop_listening_to(socket)
               ch.close
-            elsif ch[:socket].available > 0
-              data = ch[:socket].read_available(8192)
-              ch.debug { "read #{data.length} bytes from client, sending over #{type} forwarded connection" }
-              ch.send_data(data)
+            else
+              if socket.available > 0
+                data = socket.read_available(8192)
+                ch.debug { "read #{data.length} bytes from client, sending over #{type} forwarded connection" }
+                ch.send_data(data)
+              end
+
+              # Like OpenSSH's channel_pre_open, only read more once the remote
+              # window can take what is already queued; writes to the socket keep
+              # flowing. ch.output is still unsent here because Channel#process
+              # runs this block before enqueue_pending_output.
+              if ch.output.length + socket.available < ch.remote_window_size.to_i
+                session.resume_reading(socket)
+              else
+                session.pause_reading(socket)
+              end
             end
           end
         end

data/lib/net/ssh/transport/algorithms.rb

@@ -8,6 +8,7 @@ require 'net/ssh/transport/kex'
 require 'net/ssh/transport/kex/curve25519_sha256_loader'
 require 'net/ssh/transport/server_version'
 require 'net/ssh/authentication/ed25519_loader'
+require 'net/ssh/authentication/security_key'
 
 module Net
   module SSH
@@ -19,7 +20,7 @@ module Net
       #
       # You will never instantiate or reference this directly. It is used
       # internally by the transport layer.
-      class Algorithms
+      class Algorithms # rubocop:disable Metrics/ClassLength
         include Loggable
         include Constants
 
@@ -31,11 +32,8 @@ module Net
                        ecdsa-sha2-nistp521
                        ecdsa-sha2-nistp384
                        ecdsa-sha2-nistp256
-                       ssh-rsa-cert-v01@openssh.com
-                       ssh-rsa-cert-v00@openssh.com
-                       ssh-rsa
-                       rsa-sha2-256
-                       rsa-sha2-512],
+                       rsa-sha2-512
+                       rsa-sha2-256],
 
           kex: %w[ecdh-sha2-nistp521
                   ecdh-sha2-nistp384
@@ -52,7 +50,16 @@ module Net
 
           hmac: %w[hmac-sha2-512-etm@openssh.com hmac-sha2-256-etm@openssh.com
                    hmac-sha2-512 hmac-sha2-256
-                   hmac-sha1]
+                   hmac-sha1],
+
+          pubkey_algorithms: %w[ssh-ed25519-cert-v01@openssh.com
+                                ecdsa-sha2-nistp256-cert-v01@openssh.com ecdsa-sha2-nistp384-cert-v01@openssh.com
+                                ecdsa-sha2-nistp521-cert-v01@openssh.com
+                                rsa-sha2-512-cert-v01@openssh.com rsa-sha2-256-cert-v01@openssh.com
+                                ssh-ed25519
+                                ecdsa-sha2-nistp256 ecdsa-sha2-nistp384 ecdsa-sha2-nistp521] +
+                             [Authentication::SecurityKey::SK_SSH_ED25519, Authentication::SecurityKey::SK_ECDSA_SHA2_NISTP256] +
+                             %w[rsa-sha2-512 rsa-sha2-256]
         }.freeze
 
         if Net::SSH::Transport::ChaCha20Poly1305CipherLoader::LOADED
@@ -76,7 +83,8 @@ module Net
 
         # Define all algorithms, with the deprecated, supported by Net::SSH.
         ALGORITHMS = {
-          host_key: DEFAULT_ALGORITHMS[:host_key] + %w[ssh-dss],
+          host_key: DEFAULT_ALGORITHMS[:host_key] +
+                    %w[ssh-rsa-cert-v01@openssh.com ssh-rsa-cert-v00@openssh.com ssh-rsa ssh-dss],
 
           kex: DEFAULT_ALGORITHMS[:kex] +
                %w[diffie-hellman-group-exchange-sha1
@@ -98,6 +106,8 @@ module Net
                    hmac-md5 hmac-md5-96
                    none],
 
+          pubkey_algorithms: DEFAULT_ALGORITHMS[:pubkey_algorithms] + %w[ssh-rsa-cert-v01@openssh.com ssh-rsa ssh-dss],
+
           compression: %w[none zlib@openssh.com zlib],
           language: %w[]
         }.freeze
@@ -269,7 +279,9 @@ module Net
 
           ALGORITHMS.each do |algorithm, supported|
             algorithms[algorithm] = compose_algorithm_list(
-              supported, options[algorithm] || DEFAULT_ALGORITHMS[algorithm],
+              supported,
+              DEFAULT_ALGORITHMS[algorithm],
+              options[algorithm] || DEFAULT_ALGORITHMS[algorithm],
               options[:append_all_supported_algorithms]
             )
           end
@@ -287,7 +299,11 @@ module Net
             # existing known key for the host has preference.
 
             existing_keys = session.host_keys
-            host_keys = existing_keys.flat_map { |key| key.respond_to?(:ssh_types) ? key.ssh_types : [key.ssh_type] }.uniq
+            host_keys = existing_keys.flat_map { |key| key.respond_to?(:ssh_types) ? key.ssh_types : [key.ssh_type] }
+            # A known ssh-rsa key also verifies with rsa-sha2-*, and must not re-enable disabled SHA-1 algorithms.
+            disabled_sha1 = %w[ssh-rsa ssh-rsa-cert-v01@openssh.com ssh-rsa-cert-v00@openssh.com] - algorithms[:host_key]
+            host_keys = host_keys.flat_map { |type| type == 'ssh-rsa' ? %w[rsa-sha2-512 rsa-sha2-256 ssh-rsa] : [type] }
+            host_keys = (host_keys - disabled_sha1).uniq
             algorithms[:host_key].each do |name|
               host_keys << name unless host_keys.include?(name)
             end
@@ -296,21 +312,27 @@ module Net
         end
 
         # Composes the list of algorithms by taking supported algorithms and matching with supplied options.
-        def compose_algorithm_list(supported, option, append_all_supported_algorithms = false)
+        def compose_algorithm_list(supported, defaults, option, append_all_supported_algorithms = false)
           return supported.dup unless option
 
           list = []
           option = Array(option).compact.uniq
 
-          if option.first && option.first.start_with?('+', '-')
-            list = supported.dup
-
-            appends = option.select { |opt| opt.start_with?('+') }.map { |opt| opt[1..-1] }
-            deletions = option.select { |opt| opt.start_with?('-') }.map { |opt| opt[1..-1] }
+          if option.first && option.first.start_with?('+', '-', '^')
+            ops = { '^' => [], '+' => [], '-' => [] }
+            cur_list = nil
+            option.each do |cur_opt|
+              if ops.key?(cur_opt[0])
+                cur_list = ops[cur_opt[0]]
+                cur_opt = cur_opt[1..-1]
+              end
+              cur_list << cur_opt
+            end
 
-            list.concat(appends)
+            list = ops['^'] + defaults + ops['+']
+            list.concat(supported) if append_all_supported_algorithms
 
-            deletions.each do |opt|
+            ops['-'].each do |opt|
               if opt.include?('*')
                 opt_escaped = Regexp.escape(opt)
                 algo_re = /\A#{opt_escaped.gsub('\*', '[A-Za-z\d\-@\.]*')}\z/

data/lib/net/ssh/transport/session.rb

@@ -107,6 +107,10 @@ module Net
           end
         end
 
+        def pubkey_algorithms
+          algorithms[:pubkey_algorithms]
+        end
+
         # Returns the host (and possibly IP address) in a format compatible with
         # SSH known-host files.
         def host_as_string

data/lib/net/ssh/version.rb

@@ -56,7 +56,7 @@ module Net
 
       # The prerelease component of this version of the Net::SSH library
       # nil allowed
-      PRE   = "rc3"
+      PRE   = "rc4"
 
       # The current version of the Net::SSH library as a Version instance
       CURRENT = new(*[MAJOR, MINOR, TINY, PRE].compact)

data/lib/net/ssh.rb

@@ -174,11 +174,13 @@ module Net
     # * :properties => a hash of key/value pairs to add to the new connection's
     #   properties (see Net::SSH::Connection::Session#properties)
     # * :proxy => a proxy instance (see Proxy) to use when connecting
-    # * :pubkey_algorithms => the public key authentication algorithms to use for
-    #   this connection. Valid values are 'rsa-sha2-256-cert-v01@openssh.com',
-    #   'ssh-rsa-cert-v01@openssh.com', 'rsa-sha2-256', 'ssh-rsa'. Currently, this
-    #   option is only used for RSA public key authentication and ignored for other
-    #   types.
+    # * :pubkey_algorithms => the public key authentication algorithms to offer,
+    #   in order of preference (e.g. %w[ssh-ed25519 rsa-sha2-512 rsa-sha2-256]).
+    #   Keys whose type isn't listed are skipped. Like PubkeyAcceptedAlgorithms,
+    #   entries starting with +, - or ^ modify the default list. ssh-rsa (SHA-1),
+    #   ssh-rsa-cert-v01@openssh.com and ssh-dss are supported but not offered
+    #   unless listed here (e.g. '+ssh-rsa') or :append_all_supported_algorithms
+    #   is set.
     # * :rekey_blocks_limit => the max number of blocks to process before rekeying
     # * :rekey_limit => the max number of bytes to process before rekeying
     # * :rekey_packet_limit => the max number of packets to process before rekeying

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: net-ssh
 version: !ruby/object:Gem::Version
-  version: 8.0.0.rc3
+  version: 8.0.0.rc4
 platform: ruby
 authors:
 - Jamis Buck
@@ -224,6 +224,7 @@ files:
 - lib/net/ssh/authentication/methods/publickey.rb
 - lib/net/ssh/authentication/pageant.rb
 - lib/net/ssh/authentication/pub_key_fingerprint.rb
+- lib/net/ssh/authentication/security_key.rb
 - lib/net/ssh/authentication/session.rb
 - lib/net/ssh/buffer.rb
 - lib/net/ssh/buffered_io.rb