Summary

There is a possibility for denial of service by memory exhaustion when net-imap reads server responses. At any time while the client is connected, a malicious server can send can send a "literal" byte count, which is automatically read by the client's receiver thread. The response reader immediately allocates memory for the number of bytes indicated by the server response.

This should not be an issue when securely connecting to trusted IMAP servers that are well-behaved. It can affect insecure connections and buggy, untrusted, or compromised servers (for example, connecting to a user supplied hostname).

Details

The IMAP protocol allows "literal" strings to be sent in responses, prefixed with their size in curly braces (e.g. {1234567890}). When Net::IMAP receives a response containing a literal string, it calls IO#read with that size. When called with a size, IO#read immediately allocates memory to buffer the entire string before processing continues. The server does not need to send any more data. There is no limit on the size of literals that will be accepted.

Fix

Upgrade

Users should upgrade to net-imap 0.5.7 or later. A configurable max_response_size limit has been added to Net::IMAP's response reader. The max_response_size limit has also been backported to net-imap 0.2.5, 0.3.9, and 0.4.20.

To set a global value for max_response_size, users must upgrade to net-imap ~> 0.4.20, or > 0.5.7.

Configuration

To avoid backward compatibility issues for secure connections to trusted well-behaved servers, the default max_response_size for net-imap 0.5.7 is very high (512MiB), and the default max_response_size for net-imap ~> 0.4.20, ~> 0.3.9, and 0.2.5 is nil (unlimited).

When connecting to untrusted servers or using insecure connections, a much lower max_response_size should be used.

# Set the global max_response_size (only ~> v0.4.20, > 0.5.7)
Net::IMAP.config.max_response_size = 256 << 10 # 256 KiB

# Set when creating the connection
imap = Net::IMAP.new(hostname, ssl: true,
                     max_response_size: 16 << 10) # 16 KiB

# Set after creating the connection
imap.max_response_size = 256 << 20 # 256 KiB
# flush currently waiting read, to ensure the new setting is loaded
imap.noop

Please Note: max_response_size only limits the size per response. It does not prevent a flood of individual responses and it does not limit how many unhandled responses may be stored on the responses hash. Users are responsible for adding response handlers to prune excessive unhandled responses.

Compatibility with lower max_response_size

A lower max_response_size may cause a few commands which legitimately return very large responses to raise an exception and close the connection. The max_response_size could be temporarily set to a higher value, but paginated or limited versions of commands should be used whenever possible. For example, to fetch message bodies:

imap.max_response_size = 256 << 20 # 256 KiB
imap.noop # flush currently waiting read

# fetch a message in 252KiB chunks
size = imap.uid_fetch(uid, "RFC822.SIZE").first.rfc822_size
limit = 252 << 10
message = ((0..size)limit).each_with_object("") {|offset, str|
  str << imap.uid_fetch(uid,
  "BODY.PEEK[]<#{offset}.#{limit}>").first.message(offset:)
}

imap.max_response_size = 16 << 20 # 16 KiB
imap.noop # flush currently waiting read

References

• PR to introduce max_response_size: https://github.com/ruby/net-imap/pull/442
  • Specific commit: 0ae8576c1 - lib/net/imap/response_reader.rb
• Backport to 0.4: https://github.com/ruby/net-imap/pull/445
• Backport to 0.3: https://github.com/ruby/net-imap/pull/446
• Backport to 0.2: https://github.com/ruby/net-imap/pull/447

Summary

There is a possibility for denial of service by memory exhaustion in net-imap's response parser. At any time while the client is connected, a malicious server can send can send highly compressed uid-set data which is automatically read by the client's receiver thread. The response parser uses Range#to_a to convert the uid-set data into arrays of integers, with no limitation on the expanded size of the ranges.

Details

IMAP's uid-set and sequence-set formats can compress ranges of numbers, for example: "1,2,3,4,5" and "1:5" both represent the same set. When Net::IMAP::ResponseParser receives APPENDUID or COPYUID response codes, it expands each uid-set into an array of integers. On a 64 bit system, these arrays will expand to 8 bytes for each number in the set. A malicious IMAP server may send specially crafted APPENDUID or COPYUID responses with very large uid-set ranges.

The Net::IMAP client parses each server response in a separate thread, as soon as each responses is received from the server. This attack works even when the client does not handle the APPENDUID or COPYUID responses.

Malicious inputs:

# 40 bytes expands to ~1.6GB:
"* OK [COPYUID 1 1:99999999 1:99999999]\r\n"

# Worst *valid* input scenario (using uint32 max),
# 44 bytes expands to 64GiB:
"* OK [COPYUID 1 1:4294967295 1:4294967295]\r\n"

# Numbers must be non-zero uint32, but this isn't validated.  Arrays
# larger than UINT32_MAX can be created.  For example, the following
# would theoretically expand to almost 800 exabytes:
"* OK [COPYUID 1 1:99999999999999999999 1:99999999999999999999]\r\n"

Simple way to test this:

require "net/imap"

def test(size)
  input = "A004 OK [COPYUID 1 1:#{size} 1:#{size}] too large?\n"
  parser = Net::IMAP::ResponseParser.new
  parser.parse input
end

test(99_999_999)

Fixes

Preferred Fix, minor API changes

Upgrade to v0.4.19, v0.5.6, or higher, and configure:

# globally
Net::IMAP.config.parser_use_deprecated_uidplus_data = false
# per-client
imap = Net::IMAP.new(hostname, ssl: true,
                               parser_use_deprecated_uidplus_data: false)
imap.config.parser_use_deprecated_uidplus_data = false

This replaces UIDPlusData with AppendUIDData and CopyUIDData. These classes store their UIDs as Net::IMAP::SequenceSet objects (not expanded into arrays of integers). Code that does not handle APPENDUID or COPYUID responses will not notice any difference. Code that does handle these responses may need to be updated. See the documentation for UIDPlusData, AppendUIDData and CopyUIDData.

For v0.3.8, this option is not available. For v0.4.19, the default value is true. For v0.5.6, the default value is :up_to_max_size. For v0.6.0, the only allowed value will be false (UIDPlusData will be removed from v0.6).

Mitigation, backward compatible API

Upgrade to v0.3.8, v0.4.19, v0.5.6, or higher.

For backward compatibility, uid-set can still be expanded into an array, but a maximum limit will be applied.

Assign config.parser_max_deprecated_uidplus_data_size to set the maximum UIDPlusData UID set size. When config.parser_use_deprecated_uidplus_data == true, larger sets will crash. When config.parser_use_deprecated_uidplus_data == :up_to_max_size, larger sets will use AppendUIDData or CopyUIDData.

For v0.3,8, this limit is hard-coded to 10,000, and larger sets will always raise Net::IMAP::ResponseParseError. For v0.4.19, the limit defaults to 1000. For v0.5.6, the limit defaults to 100. For v0.6.0, the limit will be ignored (UIDPlusData will be removed from v0.6).

Please Note: unhandled responses

If the client does not add response handlers to prune unhandled responses, a malicious server can still eventually exhaust all

client memory, by repeatedly sending malicious responses. However, net-imap has always retained unhandled responses, and it has always been necessary for long-lived connections to prune these responses. This is not significantly different from connecting to a trusted server with a long-lived connection. To limit the maximum number of retained responses, a simple handler might look something like the following:

limit = 1000
imap.add_response_handler do |resp|
  next unless resp.respond_to?(:name) && resp.respond_to?(:data)
  name = resp.name
  code = resp.data.code&.name if resp.data.respond_to?(:code)
  if Net::IMAP::VERSION > "0.4.0"
    imap.responses(name) { _1.slice!(0...-limit) }
    imap.responses(code) { _1.slice!(0...-limit) }
  else
    imap.responses(name).slice!(0...-limit)
    imap.responses(code).slice!(0...-limit)
  end
end