larch 1.1.0.dev.20091126 → 1.1.0.dev.20091203
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/README.rdoc +52 -6
- data/lib/larch/config.rb +2 -0
- data/lib/larch/imap/mailbox.rb +19 -17
- data/lib/larch/imap.rb +17 -20
- data/lib/larch/version.rb +1 -1
- data/lib/larch.rb +13 -14
- metadata +2 -2
data/README.rdoc
CHANGED
@@ -22,7 +22,7 @@ Latest stable release:
|
|
22
22
|
|
23
23
|
Latest development version:
|
24
24
|
|
25
|
-
gem install larch
|
25
|
+
gem install larch --pre
|
26
26
|
|
27
27
|
== Usage
|
28
28
|
|
@@ -213,8 +213,47 @@ servers:
|
|
213
213
|
The following servers do not work well with Larch:
|
214
214
|
|
215
215
|
* BlitzMail - Buggy server implementation; fails to properly quote or escape
|
216
|
-
some IMAP responses, which can cause
|
217
|
-
character that will never arrive.
|
216
|
+
some IMAP responses, which can cause Net::IMAP to hang waiting for a
|
217
|
+
terminating character that will never arrive.
|
218
|
+
|
219
|
+
=== Gmail Quirks
|
220
|
+
|
221
|
+
Gmail's IMAP implementation is quirky. Larch does its best to work around these
|
222
|
+
quirks whenever possible, but here are a few things to watch out for:
|
223
|
+
|
224
|
+
==== "Some messages could not be FETCHed" error
|
225
|
+
|
226
|
+
This error indicates that a message on Gmail is corrupt, and Gmail itself is
|
227
|
+
unable to read it. The message will continue to show up in the mailbox, but all
|
228
|
+
attempts to access it via IMAP, POP, or the Gmail web interface will result in
|
229
|
+
errors. Larch will try to skip these messages and continue processing others
|
230
|
+
if possible.
|
231
|
+
|
232
|
+
It's not clear how this corruption occurs or exactly what kind of corruption
|
233
|
+
causes these errors, although in every case I'm aware of, the corrupt message
|
234
|
+
has originated outside of Gmail (Gmail itself does not corrupt the message).
|
235
|
+
There is currently no known solution for this problem apart from deleting the
|
236
|
+
corrupted messages.
|
237
|
+
|
238
|
+
==== Maximum folder name length of 40 characters
|
239
|
+
|
240
|
+
Folders in Gmail IMAP are really just labels, and a nested folder hierarchy like
|
241
|
+
"Foo\Bar\Baz" is actually just a single label with "\" characters in it. Since
|
242
|
+
Gmail limits label names to 40 characters, it's impossible to create a nested
|
243
|
+
folder hierarchy with a combined name length of more than 40 characters
|
244
|
+
including delimiters.
|
245
|
+
|
246
|
+
If you try to copy a too-long folder hierarchy to Gmail using Larch, Gmail will
|
247
|
+
return a "Folder name is not allowed" error. To work around this, reorganize
|
248
|
+
your mail into a shallower hierarchy on the source server to avoid hitting the
|
249
|
+
folder name length limit.
|
250
|
+
|
251
|
+
==== Folder names cannot contain leading or trailing whitespace
|
252
|
+
|
253
|
+
Most IMAP servers allow folder names to contain leading and trailing whitespace,
|
254
|
+
such as " folder ". Gmail does not. When copying folders to Gmail, Larch will
|
255
|
+
automatically remove leading and trailing whitespace in folder names to prevent
|
256
|
+
errors.
|
218
257
|
|
219
258
|
== Known Issues
|
220
259
|
|
@@ -223,10 +262,14 @@ The following servers do not work well with Larch:
|
|
223
262
|
cause a deadlock to occur if a connection drops unexpectedly (either due to
|
224
263
|
network issues or because the server closed the connection without warning)
|
225
264
|
when the server has already begun sending a response and Net::IMAP is
|
226
|
-
waiting to receive more data.
|
265
|
+
waiting to receive more data. If this happens, Net::IMAP will continue waiting
|
266
|
+
forever without passing control back to Larch, and you will need to manually
|
267
|
+
kill and restart Larch.
|
227
268
|
|
228
|
-
|
229
|
-
|
269
|
+
Net::IMAP in Ruby 1.8 has also been known to hang when it can't parse a server
|
270
|
+
response, either because the response itself is malformed or because of a
|
271
|
+
bug in Net::IMAP's parser. This is rare, but it happens. Unfortunately there's
|
272
|
+
nothing Larch can do about this.
|
230
273
|
|
231
274
|
* The Ruby package on Debian, Ubuntu, and some other Debian-based Linux
|
232
275
|
distributions doesn't include the OpenSSL standard library. If you see an
|
@@ -241,6 +284,9 @@ The Larch mailing list is the best place for questions, comments, and discussion
|
|
241
284
|
about Larch. You can join the list or view the archives at
|
242
285
|
http://groups.google.com/group/larch
|
243
286
|
|
287
|
+
First-time senders to the list are moderated to prevent spam, so there may be a
|
288
|
+
delay before your first message shows up.
|
289
|
+
|
244
290
|
== Credit
|
245
291
|
|
246
292
|
The Larch::IMAP class borrows heavily from Sup[http://sup.rubyforge.org] by
|
data/lib/larch/config.rb
CHANGED
data/lib/larch/imap/mailbox.rb
CHANGED
@@ -16,17 +16,15 @@ class Mailbox
|
|
16
16
|
def initialize(imap, name, delim, subscribed, *attr)
|
17
17
|
raise ArgumentError, "must provide a Larch::IMAP instance" unless imap.is_a?(Larch::IMAP)
|
18
18
|
|
19
|
+
@attr = attr.flatten
|
20
|
+
@delim = delim
|
21
|
+
@flags = []
|
19
22
|
@imap = imap
|
23
|
+
@last_scan = nil
|
20
24
|
@name = name
|
21
25
|
@name_utf7 = Net::IMAP.encode_utf7(@name)
|
22
|
-
@delim = delim
|
23
|
-
@flags = []
|
24
26
|
@perm_flags = []
|
25
27
|
@subscribed = subscribed
|
26
|
-
@attr = attr.flatten
|
27
|
-
|
28
|
-
@last_scan = nil
|
29
|
-
@mutex = Mutex.new
|
30
28
|
|
31
29
|
# Valid mailbox states are :closed (no mailbox open), :examined (mailbox
|
32
30
|
# open and read-only), or :selected (mailbox open and read-write).
|
@@ -52,6 +50,8 @@ class Mailbox
|
|
52
50
|
# Create private convenience methods (debug, info, warn, etc.) to make
|
53
51
|
# logging easier.
|
54
52
|
Logger::LEVELS.each_key do |level|
|
53
|
+
next if Mailbox.private_method_defined?(level)
|
54
|
+
|
55
55
|
Mailbox.class_eval do
|
56
56
|
define_method(level) do |msg|
|
57
57
|
Larch.log.log(level, "#{@imap.username}@#{@imap.host}: #{@name}: #{msg}")
|
@@ -82,7 +82,7 @@ class Mailbox
|
|
82
82
|
next true if flag == :Recent
|
83
83
|
|
84
84
|
unless @flags.include?(flag) || @perm_flags.include?(:*) || @perm_flags.include?(flag)
|
85
|
-
|
85
|
+
info "flag not supported on destination: #{flag}"
|
86
86
|
true
|
87
87
|
end
|
88
88
|
end
|
@@ -167,14 +167,15 @@ class Mailbox
|
|
167
167
|
|
168
168
|
# Resets the mailbox state.
|
169
169
|
def reset
|
170
|
-
@
|
170
|
+
@state = :closed
|
171
171
|
end
|
172
172
|
|
173
173
|
# Fetches message headers from this mailbox.
|
174
174
|
def scan
|
175
|
-
|
175
|
+
now = Time.now.to_i
|
176
|
+
return if @last_scan && (now - @last_scan) < SCAN_INTERVAL
|
176
177
|
first_scan = @last_scan.nil?
|
177
|
-
@
|
178
|
+
@last_scan = now
|
178
179
|
|
179
180
|
# Compare the mailbox's current status with its last known status.
|
180
181
|
begin
|
@@ -314,7 +315,8 @@ class Mailbox
|
|
314
315
|
rescue => e
|
315
316
|
# Set this mailbox's uidnext value to the last known good UID that was
|
316
317
|
# stored in the database, plus 1. This will allow Larch to resume where
|
317
|
-
# the error occurred on the next attempt rather than having to start
|
318
|
+
# the error occurred on the next attempt rather than having to start
|
319
|
+
# over.
|
318
320
|
@db_mailbox.update(:uidnext => last_good_uid + 1) if last_good_uid
|
319
321
|
raise
|
320
322
|
end
|
@@ -329,7 +331,7 @@ class Mailbox
|
|
329
331
|
return false if subscribed? && !force
|
330
332
|
|
331
333
|
@imap.safely { @imap.conn.subscribe(@name_utf7) } unless @imap.options[:dry_run]
|
332
|
-
@
|
334
|
+
@subscribed = true
|
333
335
|
@db_mailbox.update(:subscribed => 1)
|
334
336
|
|
335
337
|
true
|
@@ -345,7 +347,7 @@ class Mailbox
|
|
345
347
|
return false unless subscribed? || force
|
346
348
|
|
347
349
|
@imap.safely { @imap.conn.unsubscribe(@name_utf7) } unless @imap.options[:dry_run]
|
348
|
-
@
|
350
|
+
@subscribed = false
|
349
351
|
@db_mailbox.update(:subscribed => 0)
|
350
352
|
|
351
353
|
true
|
@@ -397,13 +399,13 @@ class Mailbox
|
|
397
399
|
|
398
400
|
@imap.safely do
|
399
401
|
begin
|
400
|
-
@
|
402
|
+
@state = :closed
|
401
403
|
|
402
404
|
debug "examining mailbox"
|
403
405
|
@imap.conn.examine(@name_utf7)
|
404
406
|
refresh_flags
|
405
407
|
|
406
|
-
@
|
408
|
+
@state = :examined
|
407
409
|
|
408
410
|
rescue Net::IMAP::NoResponseError => e
|
409
411
|
raise Error, "unable to examine mailbox: #{e.message}"
|
@@ -425,13 +427,13 @@ class Mailbox
|
|
425
427
|
|
426
428
|
@imap.safely do
|
427
429
|
begin
|
428
|
-
@
|
430
|
+
@state = :closed
|
429
431
|
|
430
432
|
debug "selecting mailbox"
|
431
433
|
@imap.conn.select(@name_utf7)
|
432
434
|
refresh_flags
|
433
435
|
|
434
|
-
@
|
436
|
+
@state = :selected
|
435
437
|
|
436
438
|
rescue Net::IMAP::NoResponseError => e
|
437
439
|
raise Error, "unable to select mailbox: #{e.message}" unless create
|
data/lib/larch/imap.rb
CHANGED
@@ -56,7 +56,6 @@ class IMAP
|
|
56
56
|
|
57
57
|
@conn = nil
|
58
58
|
@mailboxes = {}
|
59
|
-
@mutex = Mutex.new
|
60
59
|
@quirks = {
|
61
60
|
:gmail => false
|
62
61
|
}
|
@@ -69,6 +68,8 @@ class IMAP
|
|
69
68
|
# Create private convenience methods (debug, info, warn, etc.) to make
|
70
69
|
# logging easier.
|
71
70
|
Logger::LEVELS.each_key do |level|
|
71
|
+
next if IMAP.private_method_defined?(level)
|
72
|
+
|
72
73
|
IMAP.class_eval do
|
73
74
|
define_method(level) do |msg|
|
74
75
|
Larch.log.log(level, "#{username}@#{host}: #{msg}")
|
@@ -185,7 +186,7 @@ class IMAP
|
|
185
186
|
|
186
187
|
raise unless (retries += 1) <= @options[:max_retries]
|
187
188
|
|
188
|
-
|
189
|
+
warn "#{e.class.name}: #{e.message} (reconnecting)"
|
189
190
|
|
190
191
|
reset
|
191
192
|
sleep 1 * retries
|
@@ -198,7 +199,7 @@ class IMAP
|
|
198
199
|
|
199
200
|
raise unless (retries += 1) <= @options[:max_retries]
|
200
201
|
|
201
|
-
|
202
|
+
warn "#{e.class.name}: #{e.message} (will retry)"
|
202
203
|
|
203
204
|
sleep 1 * retries
|
204
205
|
retry
|
@@ -252,10 +253,8 @@ class IMAP
|
|
252
253
|
|
253
254
|
# Resets the connection and mailbox state.
|
254
255
|
def reset
|
255
|
-
@
|
256
|
-
|
257
|
-
@mailboxes.each_value {|mb| mb.reset }
|
258
|
-
end
|
256
|
+
@conn = nil
|
257
|
+
@mailboxes.each_value {|mb| mb.reset }
|
259
258
|
end
|
260
259
|
|
261
260
|
def safe_connect
|
@@ -277,7 +276,7 @@ class IMAP
|
|
277
276
|
# verification errors.
|
278
277
|
raise if e.is_a?(OpenSSL::SSL::SSLError) && e.message =~ /certificate verify failed/
|
279
278
|
|
280
|
-
|
279
|
+
warn "#{e.class.name}: #{e.message} (will retry)"
|
281
280
|
|
282
281
|
reset
|
283
282
|
sleep 1 * retries
|
@@ -289,7 +288,7 @@ class IMAP
|
|
289
288
|
end
|
290
289
|
|
291
290
|
def unsafe_connect
|
292
|
-
|
291
|
+
debug "connecting..."
|
293
292
|
|
294
293
|
exception = nil
|
295
294
|
|
@@ -322,7 +321,7 @@ class IMAP
|
|
322
321
|
@conn.authenticate(method, username, password)
|
323
322
|
end
|
324
323
|
|
325
|
-
|
324
|
+
debug "authenticated using #{method}"
|
326
325
|
|
327
326
|
rescue Net::IMAP::BadResponseError, Net::IMAP::NoResponseError => e
|
328
327
|
debug "#{method} auth failed: #{e.message}"
|
@@ -343,18 +342,16 @@ class IMAP
|
|
343
342
|
all = safely { @conn.list('', '*') } || []
|
344
343
|
subscribed = safely { @conn.lsub('', '*') } || []
|
345
344
|
|
346
|
-
|
347
|
-
|
348
|
-
@mailboxes.delete_if {|k, v| !all.any?{|mb| Net::IMAP.decode_utf7(mb.name) == k}}
|
345
|
+
# Remove cached mailboxes that no longer exist.
|
346
|
+
@mailboxes.delete_if {|k, v| !all.any?{|mb| Net::IMAP.decode_utf7(mb.name) == k}}
|
349
347
|
|
350
|
-
|
351
|
-
|
352
|
-
|
353
|
-
|
348
|
+
# Update cached mailboxes.
|
349
|
+
all.each do |mb|
|
350
|
+
name = Net::IMAP.decode_utf7(mb.name)
|
351
|
+
name = 'INBOX' if name.downcase == 'inbox'
|
354
352
|
|
355
|
-
|
356
|
-
|
357
|
-
end
|
353
|
+
@mailboxes[name] ||= Mailbox.new(self, name, mb.delim,
|
354
|
+
subscribed.any?{|s| s.name == mb.name}, mb.attr)
|
358
355
|
end
|
359
356
|
|
360
357
|
# Remove mailboxes that no longer exist from the database.
|
data/lib/larch/version.rb
CHANGED
data/lib/larch.rb
CHANGED
@@ -1,7 +1,3 @@
|
|
1
|
-
# Prepend this file's directory to the include path if it's not there already.
|
2
|
-
$:.unshift(File.dirname(File.expand_path(__FILE__)))
|
3
|
-
$:.uniq!
|
4
|
-
|
5
1
|
require 'cgi'
|
6
2
|
require 'digest/md5'
|
7
3
|
require 'fileutils'
|
@@ -23,7 +19,7 @@ require 'larch/version'
|
|
23
19
|
module Larch
|
24
20
|
|
25
21
|
class << self
|
26
|
-
attr_reader :config, :db, :
|
22
|
+
attr_reader :config, :db, :exclude, :log
|
27
23
|
|
28
24
|
EXCLUDE_COMMENT = /#.*$/
|
29
25
|
EXCLUDE_REGEX = /^\s*\/(.*)\/\s*/
|
@@ -37,15 +33,7 @@ module Larch
|
|
37
33
|
@log = Logger.new(@config[:verbosity])
|
38
34
|
@db = open_db(@config[:database])
|
39
35
|
|
40
|
-
|
41
|
-
if e =~ EXCLUDE_REGEX
|
42
|
-
Regexp.new($1, Regexp::IGNORECASE)
|
43
|
-
else
|
44
|
-
glob_to_regex(e.strip)
|
45
|
-
end
|
46
|
-
end
|
47
|
-
|
48
|
-
load_exclude_file(@config[:exclude_file]) if @config[:exclude_file]
|
36
|
+
parse_exclusions
|
49
37
|
|
50
38
|
Net::IMAP.debug = true if @log.level == :insane
|
51
39
|
|
@@ -256,6 +244,17 @@ module Larch
|
|
256
244
|
raise Larch::IMAP::FatalError, "error in exclude file at line #{lineno}: #{e}"
|
257
245
|
end
|
258
246
|
|
247
|
+
def parse_exclusions
|
248
|
+
@exclude = @config[:exclude].map do |e|
|
249
|
+
if e =~ EXCLUDE_REGEX
|
250
|
+
Regexp.new($1, Regexp::IGNORECASE)
|
251
|
+
else
|
252
|
+
glob_to_regex(e.strip)
|
253
|
+
end
|
254
|
+
end
|
255
|
+
|
256
|
+
load_exclude_file(@config[:exclude_file]) if @config[:exclude_file]
|
257
|
+
end
|
259
258
|
end
|
260
259
|
|
261
260
|
end
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: larch
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.1.0.dev.
|
4
|
+
version: 1.1.0.dev.20091203
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Ryan Grove
|
@@ -9,7 +9,7 @@ autorequire:
|
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
11
|
|
12
|
-
date: 2009-
|
12
|
+
date: 2009-12-03 00:00:00 -08:00
|
13
13
|
default_executable:
|
14
14
|
dependencies:
|
15
15
|
- !ruby/object:Gem::Dependency
|