em-imap 0.1

data/LICENSE.MIT

@@ -0,0 +1,19 @@
+Copyright (c) 2011 Conrad Irwin <conrad@rapportive.com>
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,179 @@
+An [EventMachine](http://eventmachine.org/) based [IMAP](http://tools.ietf.org/html/rfc3501) client.
+
+## Installation
+
+    gem install em-imap
+
+## Usage
+
+This document tries to introduce concepts of IMAP alongside the facilities of the library that handle them, to give you an idea of how to perform basic IMAP operations. IMAP is more fully explained in [RFC3501](http://tools.ietf.org/html/rfc3501), and the details of the library are of course in the source code.
+
+### Connecting
+
+Before you can communicate with an IMAP server, you must first connect to it. There are three connection parameters, the hostname, the port number, and whether to use SSL/TLS. As with every method in EM::IMAP, `EM::IMAP.connect` returns a [deferrable](http://eventmachine.rubyforge.org/docs/DEFERRABLES.html) enhanced by the [deferrable\_gratification](https://github.com/samstokes/deferrable_gratification) library.
+
+For example, to connect to Gmail's IMAP server, you can use the following snippet:
+
+    require 'rubygems'
+    require 'em-imap'
+
+    EM::run do
+      client = EM::IMAP.connect('imap.gmail.com', 993, true)
+      client.errback do |error|
+        puts "Connecting failed: #{error}"
+      end.callback do |hello_response|
+        puts "Connecting succeeded!"
+      end.bothback do
+        EM::stop
+      end
+    end
+
+### Authenticating
+
+There are two authentication mechanisms in IMAP, `LOGIN` and `AUTHENTICATE`, exposed as two methods on the EM::IMAP client, `.login(username, password)` and `.authenticate(mechanism, *args)`. Again these methods both return deferrables, and the cleanest way to tie deferrables together is to use the [`.bind!`](http://samstokes.github.com/deferrable_gratification/doc/DeferrableGratification/Combinators.html#bind!-instance_method) method from deferrable\_gratification.
+
+Extending our previous example to also log in to Gmail:
+
+    client = EM::IMAP.connect('imap.gmail.com', 993, true)
+    client.bind! do
+      client.login("conrad.irwin@gmail.com", ENV["GMAIL_PASSWORD"])
+    end.callback do
+      puts "Connected and logged in!"
+    end.errback do |error|
+      puts "Connecting or logging in failed: #{error}"
+    end
+
+The `.authenticate` method is more advanced and uses the same extensible mechanism as [Net::IMAP](http://www.ruby-doc.org/stdlib/libdoc/net/imap/rdoc/classes/Net/IMAP.html). The two mechanisms supported by default are `'LOGIN'` and [`'CRAM-MD5'`](http://www.ietf.org/rfc/rfc2195.txt), other mechanisms are provided by gems like [gmail\_xoauth](https://github.com/nfo/gmail_xoauth).
+
+### Mailbox-level IMAP
+
+Once the authentication has completed successfully, you can perform IMAP commands that don't require a currently selected mailbox. For example to get a list of the names of all Gmail mailboxes (including labels):
+
+    client = EM::IMAP.connect('imap.gmail.com', 993, true)
+    client.bind! do
+      client.login("conrad.irwin@gmail.com", ENV["GMAIL_PASSWORD"])
+    end.bind! do
+      client.list
+    end.callback do |list|
+      puts list.map(&:name)
+    end.errback do |error|
+      puts "Connecting, logging in or listing failed: #{error}"
+    end
+
+The useful commands available to you at this point are `.list`, `.create(mailbox)`, `.delete(mailbox)`, `.rename(old_mailbox, new_mailbox)`, `.status(mailbox)`. `.select(mailbox)` and `.examine(mailbox)` are discussed in the next section, and `.subscribe(mailbox)`, `.unsubscribe(mailbox)`, `.lsub` and `.append(mailbox, message, flags?, date_time)` are unlikely to be useful to you immediately. For a full list of IMAP commands, and detailed considerations, please refer to [RFC3501](http://tools.ietf.org/html/rfc3501).
+
+### Message-level IMAP
+
+In order to do useful things which actual messages, you need to first select a mailbox to interact with. There are two commands for doing this, `.select(mailbox)`, and `.examine(mailbox)`. They are the same except that `.examine` opens a mailbox in read-only mode; so that no changes are made (i.e. performing commands doesn't mark emails as read).
+
+For example to search for all emails relevant to em-imap in Gmail:
+
+    client = EM::IMAP.connect('imap.gmail.com', 993, true)
+    client.bind! do
+      client.login("conrad.irwin@gmail.com", ENV["GMAIL_PASSWORD"])
+    end.bind! do
+      client.select('[Google Mail]/All Mail')
+    end.bind! do
+      client.search('ALL', 'SUBJECT', 'em-imap')
+    end.callback do |results|
+      puts results
+    end.errback do |error|
+      puts "Something failed: #{error}"
+    end
+
+Once you have a list of message sequence numbers, as returned by search, you can actually read the emails with `.fetch`:
+
+    client = EM::IMAP.connect('imap.gmail.com', 993, true)
+    client.bind! do
+      client.login("conrad.irwin@gmail.com", ENV["GMAIL_PASSWORD"])
+    end.bind! do
+      client.select('[Google Mail]/All Mail')
+    end.bind! do
+      client.search('ALL', 'SUBJECT', 'em-imap')
+    end.bind! do |results|
+      client.fetch(results, 'BODY[TEXT]')
+    end.callback do |emails|
+      puts emails.map{|email| email.attr['BODY[TEXT]'] }
+    end.errback do |error|
+      puts "Something failed: #{error}"
+    end
+
+The useful commands available to you at this point are `.search(*args)`, `.expunge`, `.fetch(messages, attributes)`, `.store(messages, name, values)` and `.copy(messages, mailbox)`. If you'd like to work with UIDs instead of sequence numbers, there are UID based alternatives: `.uid_search`, `.uid_fetch`, `.uid_store` and `.uid_copy`. The `.close` command and `.check` command are unlikely to be useful to you immediately.
+
+### Untagged responses
+
+IMAP has the notion of untagged responses (aka. unsolicited responses). The idea is that sometimes when you run a command you'd like to be updated on the state of the mailbox with which you are interacting, even though notification isn't always required. To listen for these responses, the deferrables returned by each client method have a `.listen(&block)` method. All responses received by the server, up to and including the response that completes the current command will be passed to your block.
+
+For example, we could insert a listener into the above example to find out some interesting numbers:
+
+    end.bind! do
+      client.select('[Google Mail]/All Mail').listen do |response|
+        case response.name
+        when "EXISTS"
+          puts "There are #{response.data} total emails in All Mail"
+        when "RECENT"
+          puts "There are #{response.data} new emails in All Mail"
+        end
+      end
+    end.bind! do
+
+One IMAP command that exists solely to receive such unsolicited responses is IDLE. The IDLE command blocks the connection so that no other commands can use it, so before you can send further commands you must `stop` the IDLE command:
+
+    idler = client.idle
+
+    idler.listen do |response|
+      if (response.name == "EXISTS" rescue nil)
+        puts "Ooh, new emails!"
+        idler.stop
+        idler.callback do
+          # ... process new emails
+        end
+      end
+    end.errback do |e|
+      puts "Idler recieved an error: #{e}"
+    end
+
+### Concurrency
+
+IMAP is an explicitly concurrent protocol: clients MAY send commands without waiting for the previous command to complete, and servers MAY send any untagged response at any time.
+
+If you want to receive server responses at any time, you can call `.add_response_handler(&block)` on the client. This returns a deferrable like the IDLE command, on which you can call `stop` to stop receiving responses (which will cause the deferrable to succeed). You should also listen on the `errback` of this deferrable so that you know when the connection is closed:
+
+    handler = client.add_response_handler do |response|
+      puts "Server says: #{response}"
+    end.errback do |e|
+      puts "Connection closed?: #{e}"
+    end
+    EM::Timer.new(600){ handler.stop }
+
+If you want to send commands without waiting for previous replies, you can also do so. em-imap handles the few cases where this is not permitted (for example, during an IDLE command) by queueing the command until the connection becomes available again. If you do this, bear in mind that any blocks that are listening on the connection may receive responses from multiple commands interleaved.
+
+    client = EM::Imap.connect('imap.gmail.com', 993, true).callback do
+      logger_in = client.login('conrad.irwin@gmail.com', ENV["GMAIL_PASSWORD"])
+      selecter = client.select('[Google Mail]/All Mail')
+      searcher = client.search('from:conrad@rapportive.com').callback do |results|
+        puts results
+      end
+
+      logger_in.errback{ |e| selecter.fail e }
+      selecter.errback{ |e| searcher.fail e }
+      searcher.errback{ |e| "Something failed: #{e}" }
+    end
+
+## TODO
+
+em-imap is still very much a work-in-progress, and the API will change as time goes by.
+
+Before version 1, at least the following changes should be made:
+
+1. Stop using Net::IMAP in quite so many bizarre ways, probably clearer to copy-paste the code and rename relevant classes (particular NoResponseError..)
+2. Find a nicer API for some commands (maybe some objects to represent mailboxes, and/or messages?)
+3. Document argument serialization.
+4. Support SORT and THREAD.
+5. Put the in-line documentation into a real format.
+
+## Meta-foo
+
+Em-imap is made available under the MIT license, see LICENSE.MIT for details
+
+Patches and pull-requests are welcome.

data/lib/em-imap.rb

@@ -0,0 +1,40 @@
+require 'net/imap'
+require 'set'
+
+require 'rubygems'
+require 'eventmachine'
+require 'deferrable_gratification'
+
+$:.unshift File.dirname( __FILE__ )
+require 'em-imap/listener'
+require 'em-imap/continuation_synchronisation'
+require 'em-imap/command_sender'
+require 'em-imap/response_parser'
+require 'em-imap/connection'
+
+require 'em-imap/authenticators'
+require 'em-imap/client'
+$:.shift
+
+module EventMachine
+  module IMAP
+    # Connect to the specified IMAP server, using ssl if applicable.
+    #
+    # Returns a deferrable that will succeed or fail based on the
+    # success of the connection setup phase.
+    #
+    def self.connect(host, port, ssl=false)
+      Client.new(EventMachine::IMAP::Connection.connect(host, port, ssl))
+    end
+
+    class Command < Listener
+      attr_accessor :tag, :cmd, :args
+      def initialize(tag, cmd, args=[], &block)
+        super(&block)
+        self.tag = tag
+        self.cmd = cmd
+        self.args = args
+      end
+    end
+  end
+end

data/lib/em-imap/authenticators.rb

@@ -0,0 +1,28 @@
+module EventMachine
+  # Makes Net::IMAP.add_authenticator accessible through EM::IMAP and instances thereof.
+  # Also provides the authenticator method to EM::IMAP::Client to get authenticators
+  # for use in the authentication exchange.
+  #
+  module IMAP
+    def self.add_authenticator(klass)
+      Net::IMAP.add_authenticator(*args)
+    end
+
+    module Authenticators
+      def add_authenticator(*args)
+        EventMachine::IMAP.add_authenticator(*args)
+      end
+
+      private
+
+      def authenticator(type, *args)
+        raise ArgumentError, "Unknown auth type - '#{type}'" unless imap_authenticators[type]
+        imap_authenticators[type].new(*args)
+      end
+
+      def imap_authenticators
+        Net::IMAP.send :class_variable_get, :@@authenticators
+      end
+    end
+  end
+end

data/lib/em-imap/client.rb

@@ -0,0 +1,493 @@
+module EventMachine
+  module IMAP
+    # TODO: Anything that accepts or returns a mailbox name should have UTF7 support.
+    class Client
+      include EM::Deferrable
+      DG.enhance!(self)
+
+      include IMAP::Authenticators
+
+      def initialize(connection)
+        @connection = connection.errback{ |e| fail e }.callback{ |response| succeed response }
+      end
+
+      def disconnect
+        @connection.close_connection
+      end
+
+      ## 6.1 Client Commands - Any State.
+
+      # Ask the server which capabilities it supports.
+      #
+      # Succeeds with an array of capabilities.
+      #
+      def capability
+        one_data_response("CAPABILITY").transform{ |response| response.data }
+      end
+
+      # Actively do nothing.
+      #
+      # This is useful as a keep-alive, or to persuade the server to send
+      # any untagged responses your listeners would like.
+      #
+      # Succeeds with nil.
+      #
+      def noop
+        tagged_response("NOOP")
+      end
+
+      # Logout and close the connection.
+      #
+      # This will cause any other listeners or commands that are still active
+      # to fail, and render this client unusable.
+      #
+      def logout
+        command = tagged_response("LOGOUT").errback do |e|
+          if e.is_a? Net::IMAP::ByeResponseError
+            # RFC 3501 says the server MUST send a BYE response and then close the connection.
+            disconnect
+            command.succeed
+          end
+        end
+      end
+
+      ## 6.2 Client Commands - Not Authenticated State
+
+      # This would start tls negotiations, until this is implemented,
+      # simply pass true as the first parameter to EM::IMAP.connect.
+      #
+      def starttls
+        raise NotImplementedError
+      end
+
+      # Authenticate using a custom authenticator.
+      #
+      # By default there are two custom authenticators available:
+      #
+      #  'LOGIN', username, password
+      #  'CRAM-MD5', username, password (see RFC 2195)
+      #
+      # Though you can add new mechanisms using EM::IMAP.add_authenticator,
+      # see for example the gmail_xoauth gem.
+      #  
+      def authenticate(auth_type, *args)
+        # Extract these first so that any exceptions can be raised
+        # before the command is created.
+        auth_type = auth_type.upcase
+        auth_handler = authenticator(auth_type, *args)
+
+        tagged_response('AUTHENTICATE', auth_type).tap do |command|
+          @connection.send_authentication_data(auth_handler, command)
+        end
+      end
+
+      # Authenticate with a username and password.
+      #
+      # NOTE: this SHOULD only work over a tls connection.
+      #
+      # If the password is wrong, the command will fail with a
+      # Net::IMAP::NoResponseError.
+      #
+      def login(username, password)
+        tagged_response("LOGIN", username, password)
+      end
+
+      ## 6.3 Client Commands - Authenticated State
+
+      # Select a mailbox for performing commands against. 
+      #
+      # This will generate untagged responses that you can subscribe to
+      # by adding a block to the listener with .listen, for more detail,
+      # see RFC 3501, section 6.3.1.
+      #
+      def select(mailbox)
+        tagged_response("SELECT", to_utf7(mailbox))
+      end
+
+      # Select a mailbox for performing read-only commands.
+      #
+      # This is exactly the same as select, except that no operation may
+      # cause a change to the state of the mailbox or its messages.
+      #
+      def examine(mailbox)
+        tagged_response("EXAMINE", to_utf7(mailbox))
+      end
+
+      # Create a new mailbox with the given name.
+      #
+      def create(mailbox)
+        tagged_response("CREATE", to_utf7(mailbox))
+      end
+
+      # Delete the mailbox with this name.
+      #
+      def delete(mailbox)
+        tagged_response("DELETE", to_utf7(mailbox))
+      end
+
+      # Rename the mailbox with this name.
+      #
+      def rename(oldname, newname)
+        tagged_response("RENAME", to_utf7(oldname), to_utf7(newname))
+      end
+
+      # Add this mailbox to the list of subscribed mailboxes.
+      #
+      def subscribe(mailbox)
+        tagged_response("SUBSCRIBE", to_utf7(mailbox))
+      end
+
+      # Remove this mailbox from the list of subscribed mailboxes.
+      #
+      def unsubscribe(mailbox)
+        tagged_response("UNSUBSCRIBE", to_utf7(mailbox))
+      end
+
+      # List all available mailboxes.
+      #
+      # @param: refname, an optional context in which to list.
+      # @param: mailbox, a  which mailboxes to return.
+      #
+      # Succeeds with a list of Net::IMAP::MailboxList structs, each of which has:
+      #   .name, the name of the mailbox (in UTF8)
+      #   .delim, the delimeter (normally "/")
+      #   .attr, A list of attributes, e.g. :Noselect, :Haschildren, :Hasnochildren.
+      #
+      def list(refname="", pattern="*")
+        list_internal("LIST", refname, pattern)
+      end
+
+      # List all subscribed mailboxes.
+      #
+      # This is the same as list, but restricted to mailboxes that have been subscribed to.
+      #
+      def lsub(refname, pattern)
+        list_internal("LSUB", refname, pattern)
+      end
+
+      # Get the status of a mailbox.
+      #
+      # This provides similar information to the untagged responses you would
+      # get by running SELECT or EXAMINE without doing so.
+      #
+      # @param mailbox, a mailbox to query
+      # @param attrs, a list of attributes to query for (valid values include
+      #        MESSAGES, RECENT, UIDNEXT, UIDVALIDITY and UNSEEN — RFC3501#6.3.8)
+      #
+      # Succeeds with a hash of attribute name to value returned by the server.
+      #
+      def status(mailbox, attrs=['MESSAGES', 'RECENT', 'UIDNEXT', 'UIDVALIDITY', 'UNSEEN'])
+        attrs = [attrs] if attrs.is_a?(String)
+        one_data_response("STATUS", to_utf7(mailbox), attrs).transform do |response|
+          response.data.attr
+        end
+      end
+
+      # Add a message to the mailbox.
+      #
+      # @param mailbox, the mailbox to add to,
+      # @param message, the full text (including headers) of the email to add.
+      # @param flags, A list of flags to set on the email.
+      # @param date_time, The time to be used as the internal date of the email.
+      #
+      # The tagged response with which this command succeeds contains the UID
+      # of the email that was appended.
+      #
+      def append(mailbox, message, flags=nil, date_time=nil)
+        args = [to_utf7(mailbox)]
+        args << flags if flags
+        args << date_time if date_time
+        args << Net::IMAP::Literal.new(message)
+        tagged_response("APPEND", *args)
+      end
+
+      # 6.4 Client Commands - Selected State
+      
+      # Checkpoint the current mailbox.
+      #
+      # This is an implementation-defined operation, when in doubt, NOOP
+      # should be used instead.
+      #
+      def check
+        tagged_response("CHECK")
+      end
+
+      # Unselect the current mailbox.
+      #
+      # As a side-effect, permanently removes any messages that have the
+      # \Deleted flag. (Unless the mailbox was selected using the EXAMINE,
+      # in which case no side effects occur).
+      #
+      def close
+        tagged_response("CLOSE")
+      end
+      
+      # Permanently remove any messages with the \Deleted flag from the current
+      # mailbox.
+      #
+      # Succeeds with a list of message sequence numbers that were deleted.
+      #
+      # NOTE: If you're planning to EXPUNGE and then SELECT a new mailbox,
+      # and you don't care which messages are removed, consider using
+      # CLOSE instead.
+      #
+      def expunge
+        multi_data_response("EXPUNGE").transform do |untagged_responses|
+          untagged_responses.map(&:data)
+        end
+      end
+
+      # Search for messages in the current mailbox.
+      #
+      # @param *args The arguments to search, these can be strings, arrays or ranges
+      #              specifying sub-groups of search arguments or sets of messages.
+      #
+      #              If you want to use non-ASCII characters, then the first two
+      #              arguments should be 'CHARSET', 'UTF-8', though not all servers
+      #              support this.
+      #
+      # @succeed A list of message sequence numbers.
+      #
+      def search(*args)
+        search_internal(["SEARCH"], *args)
+      end
+
+      # The same as search, but succeeding with a list of UIDs not sequence numbers.
+      #
+      def uid_search(*args)
+        search_internal(["UID", "SEARCH"], *args)
+      end
+
+      # SORT and THREAD (like SEARCH) from http://tools.ietf.org/search/rfc5256
+      #
+      def sort(sort_keys, *args)
+        raise NotImplementedError
+      end
+
+      def uid_sort(sort_keys, *args)
+        raise NotImplementedError
+      end
+
+      def thread(algorithm, *args)
+        raise NotImplementedError
+      end
+
+      def uid_thread(algorithm, *args)
+        raise NotImplementedError
+      end
+
+      # Get the contents of, or information about, a message.
+      # 
+      # @param seq, a message or sequence of messages (a number, a range or an array of numbers)
+      # @param attr, the name of the attribute to fetch, or a list of attributes.
+      #
+      # Possible attribute names (see RFC 3501 for a full list):
+      #
+      #  ALL: Gets all header information,
+      #  FULL: Same as ALL with the addition of the BODY,
+      #  FAST: Same as ALL without the message envelope.
+      #
+      #  BODY: The body
+      #  BODY[<section>] A particular section of the body
+      #  BODY[<section>]<<start>,<length>> A substring of a section of the body.
+      #  BODY.PEEK: The body (but doesn't change the \Recent flag)
+      #  FLAGS: The flags
+      #  INTERNALDATE: The internal date
+      #  UID: The unique identifier
+      #
+      def fetch(seq, attr="FULL")
+        fetch_internal("FETCH", seq, attr)
+      end
+
+      # The same as fetch, but keyed of UIDs instead of sequence numbers.
+      #
+      def uid_fetch(seq, attr="FULL")
+        fetch_internal("UID", "FETCH", seq, attr)
+      end
+
+      # Update the flags on a message.
+      #
+      # @param seq, a message or sequence of messages (a number, a range, or an array of numbers)
+      # @param name, any of FLAGS FLAGS.SILENT, replace the flags
+      #                     +FLAGS, +FLAGS.SILENT, add the following flags
+      #                     -FLAGS, -FLAGS.SILENT, remove the following flags
+      #             The .SILENT versions suppress the server's responses.
+      # @param value, a list of flags (symbols)
+      #
+      def store(seq, name, value)
+        store_internal("STORE", seq, name, value)
+      end
+
+      # The same as store, but keyed off UIDs instead of sequence numbers.
+      #
+      def uid_store(seq, name, value)
+        store_internal("UID", "STORE", seq, name, value)
+      end
+
+      # Copy the specified messages to another mailbox.
+      #
+      def copy(seq, mailbox)
+        tagged_response("COPY", Net::IMAP::MessageSet.new(seq), to_utf7(mailbox))
+      end
+
+      # The same as copy, but keyed off UIDs instead of sequence numbers.
+      #
+      def uid_copy(seq, mailbox)
+        tagged_response("UID", "COPY", Net::IMAP::MessageSet.new(seq), to_utf7(mailbox))
+      end
+
+      # The IDLE command allows you to wait for any untagged responses
+      # that give status updates about the contents of a mailbox.
+      #
+      # Until you call stop on the idler, no further commands can be sent
+      # over this connection.
+      #
+      # idler = connection.idle do |untagged_response|
+      #   case untagged_response.name
+      #   #...
+      #   end
+      # end
+      #
+      # EM.timeout(60) { idler.stop }
+      #
+      def idle(&block)
+        send_command("IDLE").tap do |command|
+          @connection.prepare_idle_continuation(command)
+          command.listen(&block) if block_given?
+        end
+      end
+
+      def add_response_handler(&block)
+        @connection.add_response_handler(&block)
+      end
+
+      private
+
+      # Convert a string to the modified UTF-7 required by IMAP for mailbox naming.
+      # See RFC 3501 Section 5.1.3 for more information.
+      #
+      def to_utf7(text)
+        Net::IMAP.encode_utf7(text)
+      end
+
+      # Convert a string from the modified UTF-7 required by IMAP for mailbox naming.
+      # See RFC 3501 Section 5.1.3 for more information.
+      #
+      def to_utf8(text)
+        Net::IMAP.decode_utf7(text)
+      end
+      
+      # Send a command that should return a deferrable that succeeds with
+      # a tagged_response.
+      #
+      def tagged_response(cmd, *args)
+        # We put in an otherwise unnecessary transform to hide the listen
+        # method from callers for consistency with other types of responses.
+        send_command(cmd, *args)
+      end
+
+      # Send a command that should return a deferrable that succeeds with
+      # a single untagged response with the same name as the command.
+      #
+      def one_data_response(cmd, *args)
+        multi_data_response(cmd, *args).transform do |untagged_responses|
+          untagged_responses.last
+        end
+      end
+
+      # Send a command that should return a deferrable that succeeds with
+      # multiple untagged responses with the same name as the command.
+      #
+      def multi_data_response(cmd, *args)
+        collect_untagged_responses(cmd, cmd, *args)
+      end
+
+      # Send a command that should return a deferrable that succeeds with
+      # multiple untagged responses with the given name.
+      #
+      def collect_untagged_responses(name, *command)
+        untagged_responses = []
+
+        send_command(*command).listen do |response|
+          if response.is_a?(Net::IMAP::UntaggedResponse) && response.name == name
+            untagged_responses << response
+
+          # If we observe another tagged response completeing, then we can be
+          # sure that the previous untagged responses were not relevant to this command.
+          elsif response.is_a?(Net::IMAP::TaggedResponse)
+            untagged_responses = []
+
+          end
+        end.transform do |tagged_response|
+          untagged_responses
+        end
+      end
+
+      def send_command(cmd, *args)
+        @connection.send_command(cmd, *args)
+      end
+
+      # Extract more useful data from the LIST and LSUB commands, see #list for details. 
+      def list_internal(cmd, refname, pattern)
+        multi_data_response(cmd, to_utf7(refname), to_utf7(pattern)).transform do |untagged_responses|
+          untagged_responses.map(&:data).map do |data|
+            data.dup.tap do |new_data|
+              new_data.name = to_utf8(data.name)
+            end
+          end
+        end
+      end
+
+      # From Net::IMAP
+      def fetch_internal(cmd, set, attr)
+        case attr
+        when String then
+          attr = Net::IMAP::RawData.new(attr)
+        when Array then
+          attr = attr.map { |arg|
+            arg.is_a?(String) ? Net::IMAP::RawData.new(arg) : arg
+          }
+        end
+
+        set = Net::IMAP::MessageSet.new(set)
+
+        multi_data_response(cmd, set, attr).transform do |untagged_responses|
+          untagged_responses.map(&:data)
+        end
+      end
+
+      # Ensure that the flags are symbols, and that the message set is a message set.
+      def store_internal(cmd, set, attr, flags)
+        flags = flags.map(&:to_sym)
+        set = Net::IMAP::MessageSet.new(set)
+        collect_untagged_responses('FETCH', cmd, set, attr, flags).transform do |untagged_responses|
+          untagged_responses.map(&:data)
+        end
+      end
+
+      def search_internal(command, *args)
+        command += normalize_search_criteria(args)
+        one_data_response(*command).transform{ |untagged_response| untagged_response.data }
+      end
+
+      # Recursively find all the message sets in the arguments and convert them so that
+      # Net::IMAP can serialize them.
+      def normalize_search_criteria(args)
+        args.map do |arg|
+          case arg
+          when "*", -1, Range
+            Net::IMAP::MessageSet.new(arg)
+          when Array
+            if Array.all?{ |item| item.is_a?(Number) || item.is_a?(Range) }
+              Net::IMAP::MessageSet.new(arg)
+            else
+              normalize_search_criteria(arg)
+            end
+          else
+            arg
+          end
+        end
+      end
+    end
+  end
+end