rgrove-larch 1.0.0.12 → 1.0.0.13

data/HISTORY

@@ -6,7 +6,10 @@ Version 1.0.1 (?)
   * Much more robust handling of unexpected server disconnects and dropped
     connections.
   * Added --all option to copy all folders recursively.
+  * Added --all-subscribed option to copy all subscribed folders recursively.
   * Added --dry-run option to simulate changes without actually making them.
+  * Added --exclude and --exclude-file options to specify folders that should
+    not be copied.
   * Added --ssl-certs option to specify a bundle of trusted SSL certificates.
   * Added --ssl-verify option to verify server SSL certificates.
   * Added a new "insane" logging level, which will output all IMAP commands and

data/README.rdoc

@@ -8,7 +8,7 @@ Larch is particularly well-suited for copying email to, from, or between Gmail
 accounts.
 
 *Author*::    Ryan Grove (mailto:ryan@wonko.com)
-*Version*::   1.0.0 (2009-03-17)
+*Version*::   ? (?)
 *Copyright*:: Copyright (c) 2009 Ryan Grove. All rights reserved.
 *License*::   GPL 2.0 (http://opensource.org/licenses/gpl-2.0.php)
 *Website*::   http://github.com/rgrove/larch
@@ -21,6 +21,45 @@ Install Larch via RubyGems:
 
 == Usage
 
+  larch --from <uri> --to <uri> [options]
+
+  Required:
+       --from, -f <s>:   URI of the source IMAP server.
+         --to, -t <s>:   URI of the destination IMAP server.
+
+  Copy Options:
+                --all:   Copy all folders recursively
+     --all-subscribed:   Copy all subscribed folders recursively
+       --exclude <s+>:   List of mailbox names/patterns that shouldn't be
+                         copied
+   --exclude-file <s>:   Filename containing mailbox names/patterns that
+                         shouldn't be copied
+    --from-folder <s>:   Source folder to copy from (default: INBOX)
+      --from-pass <s>:   Source server password (default: prompt)
+      --from-user <s>:   Source server username (default: prompt)
+      --to-folder <s>:   Destination folder to copy to (default: INBOX)
+        --to-pass <s>:   Destination server password (default: prompt)
+        --to-user <s>:   Destination server username (default: prompt)
+
+  General Options:
+        --dry-run, -n:   Don't actually make any changes.
+          --fast-scan:   Use a faster (but less accurate) method to scan
+                         mailboxes. This may result in messages being
+                         re-copied.
+    --max-retries <i>:   Maximum number of times to retry after a recoverable
+                         error (default: 3)
+   --no-create-folder:   Don't create destination folders that don't already
+                         exist
+      --ssl-certs <s>:   Path to a trusted certificate bundle to use to
+                         verify server SSL certificates
+         --ssl-verify:   Verify server SSL certificates
+  --verbosity, -V <s>:   Output verbosity: debug, info, warn, error, or fatal
+                         (default: info)
+        --version, -v:   Print version and exit
+           --help, -h:   Show this message
+
+== Examples
+
 Larch is run from the command line. At a minimum, you must specify a source
 server and a destination server in the form of IMAP URIs. You may also specify
 one or more options:
@@ -37,22 +76,45 @@ the necessary usernames and passwords, then sync the contents of the source's
 
   larch --from imap://mail.example.com --to imap://imap.gmail.com
 
-To connect using SSL, specify a URI beginning with <tt>imaps://</tt>:
+To <b>connect using SSL</b>, specify a URI beginning with <tt>imaps://</tt>:
 
   larch --from imaps://mail.example.com --to imaps://imap.gmail.com
 
-If you'd like to sync a folder other than +INBOX+, specify the source and
-destination folders using <tt>--from-folder</tt> and <tt>--to-folder</tt>:
+If you'd like to <b>sync a specific folder</b> other than +INBOX+, specify the
+source and destination folders using <tt>--from-folder</tt> and
+<tt>--to-folder</tt>:
 
   larch --from imaps://mail.example.com --to imaps://imap.gmail.com \
     --from-folder "Sent Mail" --to-folder "Sent Mail"
 
-By default Larch will create the specified folder if it doesn't already exist on
-the destination server. To prevent this, add the <tt>--no-create-folder</tt>
+To <b>sync all folders</b>, use the <tt>--all</tt> option (or
+<tt>--all-subscribed</tt> if you only want to <b>sync subscribed folders</b>):
+
+  larch --from imaps://mail.example.com --to imaps://imap.gmail.com --all
+
+By default Larch will create folders on the destination server if they don't
+already exist. To prevent this, add the <tt>--no-create-folder</tt> option:
+
+  larch --from imaps://mail.example.com --to imaps://imap.gmail.com --all \
+    --no-create-folder
+
+You can <b>prevent Larch from syncing one or more folders</b> by using the
+<tt>--exclude</tt> option:
+
+  larch --from imaps://mail.example.com --to imaps://imap.gmail.com --all \
+    --exclude Spam Trash Drafts "[Gmail]/*"
+
+If your exclusion list is long or complex, create a text file with one exclusion
+pattern per line and tell Larch to load it with the <tt>--exclude-file</tt>
 option:
 
-  larch --from imaps://mail.example.com --to imaps://imap.gmail.com \
-    --from-folder "Sent Mail" --to-folder "Sent Mail" --no-create-folder
+  larch --from imaps://mail.example.com --to imaps://imap.gmail.com --all \
+    --exclude-file exclude.txt
+
+The wildcard characters <tt>*</tt> and <tt>?</tt> are supported in exclusion
+lists. You can also use a regular expression by enclosing a pattern in
+forward slashes, so the previous example could be achieved with the
+pattern <tt>/(Spam|Trash|Drafts|\[Gmail\]\/.*)/</tt>
 
 == Server Compatibility
 
@@ -69,15 +131,22 @@ servers:
 
 == Known Issues
 
-Larch uses Ruby's Net::IMAP standard library for all IMAP operations. While
-Net::IMAP is generally a very solid library, it contains a bug that can cause a
-deadlock to occur if a connection drops unexpectedly (either due to network
-issues or because the server closed the connection without warning) when the
-server has already begun sending a response and Net::IMAP is waiting to receive
-more data.
-
-If this happens, Net::IMAP will continue waiting forever without passing control
-back to Larch, and you will need to manually kill and restart Larch.
+* Larch uses Ruby's Net::IMAP standard library for all IMAP operations. While
+  Net::IMAP is generally a very solid library, it contains a bug that can
+  cause a deadlock to occur if a connection drops unexpectedly (either due to
+  network issues or because the server closed the connection without warning)
+  when the server has already begun sending a response and Net::IMAP is
+  waiting to receive more data.
+
+  If this happens, Net::IMAP will continue waiting forever without passing
+  control back to Larch, and you will need to manually kill and restart Larch.
+
+* The Ruby package on Debian, Ubuntu, and some other Debian-based Linux
+  distributions doesn't include the OpenSSL standard library. If you see an
+  error like <tt>uninitialized constant Larch::IMAP::OpenSSL (NameError)</tt>
+  when running Larch, you may need to install the <tt>libopenssl-ruby</tt>
+  package. Please feel free to complain to the maintainer of your distribution's
+  Ruby packages.
 
 == Support
 

data/bin/larch

@@ -24,6 +24,9 @@ EOS
 
     text "\nCopy Options:"
     opt :all,              "Copy all folders recursively", :short => :none
+    opt :all_subscribed,   "Copy all subscribed folders recursively", :short => :none
+    opt :exclude,          "List of mailbox names/patterns that shouldn't be copied", :short => :none, :type => :strings, :multi => true
+    opt :exclude_file,     "Filename containing mailbox names/patterns that shouldn't be copied", :short => :none, :type => :string
     opt :from_folder,      "Source folder to copy from", :short => :none, :default => 'INBOX'
     opt :from_pass,        "Source server password (default: prompt)", :short => :none, :type => :string
     opt :from_user,        "Source server username (default: prompt)", :short => :none, :type => :string
@@ -48,20 +51,36 @@ EOS
     end
   end
 
-  if options[:all_given] && (options[:from_folder_given] || options[:to_folder_given])
-    Trollop.die :all, "may not be used with --from-folder or --to-folder"
-  end
-
   unless Logger::LEVELS.has_key?(options[:verbosity].to_sym)
     Trollop.die :verbosity, "must be one of: #{Logger::LEVELS.keys.join(', ')}"
   end
 
+  if options[:exclude_file_given]
+    filename = options[:exclude_file]
+
+    Trollop.die :exclude_file, ": file not found: #{filename}" unless File.file?(filename)
+    Trollop.die :exclude_file, ": file cannot be read: #{filename}" unless File.readable?(filename)
+  end
+
+  # Prevent conflicting options from being used.
+  if options[:all_given]
+    [:all_subscribed, :from_folder, :to_folder].each do |o|
+      Trollop.die :all, "may not be used with --#{o.to_s.gsub('_', '-')}" if options["#{o}_given".to_sym]
+    end
+  end
+
+  if options[:all_subscribed_given]
+    [:all, :from_folder, :to_folder].each do |o|
+      Trollop.die :all_subscribed, "may not be used with --#{o.to_s.gsub('_', '-')}" if options["#{o}_given".to_sym]
+    end
+  end
+
   # Create URIs.
   uri_from = URI(options[:from])
   uri_to   = URI(options[:to])
 
-  # --all option overrides folders specified in URIs
-  if options[:all_given]
+  # --all and --all-subscribed options override folders specified in URIs
+  if options[:all_given] || options[:all_subscribed_given]
     uri_from.path = ''
     uri_to.path   = ''
   end
@@ -85,7 +104,11 @@ EOS
   uri_to.password   ||= CGI.escape(ask("Destination password (#{uri_to.host}): ") {|q| q.echo = false })
 
   # Go go go!
-  init(options[:verbosity])
+  init(
+    options[:verbosity],
+    options[:exclude] ? options[:exclude].flatten : [],
+    options[:exclude_file]
+  )
 
   Net::IMAP.debug = true if @log.level == :insane
 
@@ -117,6 +140,8 @@ EOS
 
   if options[:all_given]
     copy_all(imap_from, imap_to)
+  elsif options[:all_subscribed_given]
+    copy_all(imap_from, imap_to, true)
   else
     copy_folder(imap_from, imap_to)
   end

data/lib/larch/errors.rb

@@ -1,6 +1,5 @@
 module Larch
   class Error < StandardError; end
-  class WatchdogException < Exception; end
 
   class IMAP
     class Error < Larch::Error; end

data/lib/larch/imap.rb

@@ -119,9 +119,9 @@ class IMAP
   end
 
   # Gets a Larch::IMAP::Mailbox instance representing the specified mailbox. If
-  # the mailbox doesn't exist and the +:create_mailbox+ option is +false+, or if
-  # +:create_mailbox+ is +true+ and mailbox creation fails, a
-  # Larch::IMAP::MailboxNotFoundError will be raised.
+  # the mailbox doesn't exist and the <tt>:create_mailbox</tt> option is
+  # +false+, or if <tt>:create_mailbox</tt> is +true+ and mailbox creation
+  # fails, a Larch::IMAP::MailboxNotFoundError will be raised.
   def mailbox(name, delim = '/')
     retries = 0
 
@@ -160,8 +160,8 @@ class IMAP
     @uri.port || (ssl? ? 993 : 143)
   end
 
-  # Connect if necessary, execute the given block, retry up to 3 times if a
-  # recoverable error occurs, die if an unrecoverable error occurs.
+  # Connect if necessary, execute the given block, retry if a recoverable error
+  # occurs, die if an unrecoverable error occurs.
   def safely
     safe_connect
 

data/lib/larch/version.rb

@@ -1,6 +1,6 @@
 module Larch
   APP_NAME      = 'Larch'
-  APP_VERSION   = '1.0.0.12'
+  APP_VERSION   = '1.0.0.13'
   APP_AUTHOR    = 'Ryan Grove'
   APP_EMAIL     = 'ryan@wonko.com'
   APP_URL       = 'http://github.com/rgrove/larch/'

data/lib/larch.rb

@@ -17,11 +17,26 @@ require 'larch/version'
 module Larch
 
   class << self
-    attr_reader :log
+    attr_reader :log, :exclude
 
-    def init(log_level = :info)
+    EXCLUDE_COMMENT = /#.*$/
+    EXCLUDE_REGEX   = /^\s*\/(.*)\/\s*/
+    GLOB_PATTERNS   = {'*' => '.*', '?' => '.'}
+
+    def init(log_level = :info, exclude = [], exclude_file = nil)
       @log = Logger.new(log_level)
 
+      @exclude = exclude.map do |e|
+        if e =~ EXCLUDE_REGEX
+          Regexp.new($1, Regexp::IGNORECASE)
+        else
+          glob_to_regex(e.strip)
+        end
+      end
+
+      load_exclude_file(exclude_file) if exclude_file
+
+      # Stats
       @copied = 0
       @failed = 0
       @total  = 0
@@ -29,24 +44,28 @@ module Larch
 
     # Recursively copies all messages in all folders from the source to the
     # destination.
-    def copy_all(imap_from, imap_to)
+    def copy_all(imap_from, imap_to, subscribed_only = false)
       raise ArgumentError, "imap_from must be a Larch::IMAP instance" unless imap_from.is_a?(IMAP)
       raise ArgumentError, "imap_to must be a Larch::IMAP instance" unless imap_to.is_a?(IMAP)
 
+      @copied = 0
+      @failed = 0
+      @total  = 0
+
       imap_from.each_mailbox do |mailbox_from|
-        @copied = 0
-        @failed = 0
-        @total  = 0
+        next if excluded?(mailbox_from.name)
+        next if subscribed_only && !mailbox_from.subscribed?
 
         mailbox_to = imap_to.mailbox(mailbox_from.name, mailbox_from.delim)
-        copy_messages(imap_from, mailbox_from, imap_to, mailbox_to)
         mailbox_to.subscribe if mailbox_from.subscribed?
 
-        summary
+        copy_messages(imap_from, mailbox_from, imap_to, mailbox_to)
       end
 
     rescue => e
       @log.fatal e.message
+
+    ensure
       summary
     end
 
@@ -60,8 +79,13 @@ module Larch
       @failed = 0
       @total  = 0
 
-      copy_messages(imap_from, imap_from.mailbox(imap_from.uri_mailbox || 'INBOX'),
-          imap_to, imap_to.mailbox(imap_to.uri_mailbox || 'INBOX'))
+      from_name = imap_from.uri_mailbox || 'INBOX'
+      to_name   = imap_to.uri_mailbox || 'INBOX'
+
+      return if excluded?(from_name) || excluded?(to_name)
+
+      copy_messages(imap_from, imap_from.mailbox(from_name), imap_to,
+          imap_to.mailbox(to_name))
 
       imap_from.disconnect
       imap_to.disconnect
@@ -85,6 +109,8 @@ module Larch
       raise ArgumentError, "imap_to must be a Larch::IMAP instance" unless imap_to.is_a?(IMAP)
       raise ArgumentError, "mailbox_to must be a Larch::IMAP::Mailbox instance" unless mailbox_to.is_a?(IMAP::Mailbox)
 
+      return if excluded?(mailbox_from.name) || excluded?(mailbox_to.name)
+
       @log.info "copying messages from #{imap_from.host}/#{mailbox_from.name} to #{imap_to.host}/#{mailbox_to.name}"
 
       imap_from.connect
@@ -118,6 +144,49 @@ module Larch
         end
       end
     end
+
+    def excluded?(name)
+      name = name.downcase
+
+      @exclude.each do |e|
+        return true if (e.is_a?(Regexp) ? !!(name =~ e) : File.fnmatch?(e, name))
+      end
+
+      return false
+    end
+
+    def glob_to_regex(str)
+      str.gsub!(/(.)/) {|c| GLOB_PATTERNS[$1] || Regexp.escape(c) }
+      Regexp.new("^#{str}$", Regexp::IGNORECASE)
+    end
+
+    def load_exclude_file(filename)
+      @exclude ||= []
+      lineno = 0
+
+      File.open(filename, 'rb') do |f|
+        f.each do |line|
+          lineno += 1
+
+          # Strip comments.
+          line.sub!(EXCLUDE_COMMENT, '')
+          line.strip!
+
+          # Skip empty lines.
+          next if line.empty?
+
+          if line =~ EXCLUDE_REGEX
+            @exclude << Regexp.new($1, Regexp::IGNORECASE)
+          else
+            @exclude << glob_to_regex(line)
+          end
+        end
+      end
+
+    rescue => e
+      raise Larch::IMAP::FatalError, "error in exclude file at line #{lineno}: #{e}"
+    end
+
   end
 
-end
+end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification 
 name: rgrove-larch
 version: !ruby/object:Gem::Version 
-  version: 1.0.0.12
+  version: 1.0.0.13
 platform: ruby
 authors: 
 - Ryan Grove