inbox-sync 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/.gitignore ADDED
@@ -0,0 +1,21 @@
1
+ *.gem
2
+ *.log
3
+ *.rbc
4
+ .bundle
5
+ .config
6
+ .yardoc
7
+ .rvmrc
8
+ .rbenv-version
9
+ Gemfile.lock
10
+ InstalledFiles
11
+ _yardoc
12
+ coverage
13
+ doc/
14
+ lib/bundler/man
15
+ pkg
16
+ rdoc
17
+ spec/reports
18
+ test/tmp
19
+ test/version_tmp
20
+ tmp
21
+ test.rb
data/Gemfile ADDED
@@ -0,0 +1,7 @@
1
+ source "http://rubygems.org"
2
+
3
+ # Specify your gem's dependencies in inbox-sync.gemspec
4
+ gemspec
5
+
6
+ gem 'bundler', '~>1.1'
7
+ gem 'rake', '~>0.9.2'
data/LICENSE ADDED
@@ -0,0 +1,22 @@
1
+ Copyright (c) 2012 Kelly Redding
2
+
3
+ MIT License
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining
6
+ a copy of this software and associated documentation files (the
7
+ "Software"), to deal in the Software without restriction, including
8
+ without limitation the rights to use, copy, modify, merge, publish,
9
+ distribute, sublicense, and/or sell copies of the Software, and to
10
+ permit persons to whom the Software is furnished to do so, subject to
11
+ the following conditions:
12
+
13
+ The above copyright notice and this permission notice shall be
14
+ included in all copies or substantial portions of the Software.
15
+
16
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
17
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
18
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
19
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
20
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
21
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
22
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,185 @@
1
+ # InboxSync
2
+
3
+ Move messages from one inbox to another. Useful when server-side email forwarding is not an option. (TODO) Can apply filters to messages as they are being moved. Run on-demand, on a schedule, or as a daemon.
4
+
5
+ ## Installation
6
+
7
+ Add this line to your application's Gemfile:
8
+
9
+ gem 'inbox-sync'
10
+
11
+ And then execute:
12
+
13
+ $ bundle
14
+
15
+ Or install it yourself as:
16
+
17
+ $ gem install inbox-sync
18
+
19
+ # How does it work?
20
+
21
+ InboxSync uses IMAP to query a source inbox, process its messages, append them to a destination inbox, and archive them on the source. It logs each step in the process and will send notification emails when something goes wrong.
22
+
23
+ (TODO) InboxSync provides a framework for defining destination filters for post-sync mail processing (ie moving/archiving, copying/labeling, deletion, etc).
24
+
25
+ InboxSync provides a basic ruby runner class to handle polling the source on an interval and running the configured sync(s). You can call it in any number of ways: in a script, from a cron, as a daemon, or as part of a larger system.
26
+
27
+ ## Usage
28
+
29
+ It should be fairly straight-forward: create and configure a sync then run it. This will move all messages in the `source` inbox to the `dest` inbox.
30
+
31
+ ### Create your Sync
32
+
33
+ ```ruby
34
+ sync = InboxSync.new
35
+ ```
36
+
37
+ ### Configure it
38
+
39
+ ```ruby
40
+ # manually set configs
41
+ sync.config.source.host = 'imap.source-host.com'
42
+
43
+ # or use a more DSL like approach
44
+ sync.config.source.login.user 'me'
45
+ sync.config.source.login.pw 'secret'
46
+
47
+ # or use a configure block, if you like
48
+ sync.configure do
49
+ dest.host 'imap.dest-host.com'
50
+ dest.login 'me', 'secret'
51
+ end
52
+ ```
53
+
54
+ ### Run it
55
+
56
+ ```ruby
57
+ InboxSync.run(sync, :interval => 5)
58
+ ```
59
+
60
+ ## Sync Definition
61
+
62
+ ### `source`
63
+
64
+ IMAP settings for the source inbox.
65
+
66
+ * *host*: eg. `'imap.some-domain.com'`.
67
+ * *port*: defaults to `143`.
68
+ * *ssl*: whether to use SSL. defaults to `false`.
69
+ * *login*: credentials (user, pw).
70
+ * *inbox*: name of the inbox folder. defaults to `'INBOX'`
71
+ * *expunge*: whether to expunge the inbox before and after processing. defaults to `true`.
72
+
73
+ ### `dest`
74
+
75
+ IMAP settings for the destination inbox. Has the some attributes and defaults as the `source`.
76
+
77
+ ### `notify`
78
+
79
+ SMTP settings to send notifications with.
80
+
81
+ * *host*: eg. `'smtp.some-domain.com'`.
82
+ * *port*: defaults to `25`.
83
+ * *tls*: whethe to use TLS encryption. defaults to `false`.
84
+ * *helo*: the helo domain to send with.
85
+ * *login*: credentials (user, pw).
86
+ * *authtype*: defaults to `:login`.
87
+ * *from_addr*: address to send the notifications from.
88
+ * *to_addr*: address(es) to send the notifications to.
89
+
90
+ ### `archive_folder`
91
+
92
+ The (optional) folder on the source to create and archive (move) source inbox messages to when processing is complete. Defaults to `"Archived"`. Set to `nil` to disable archiving on the source and delete the messages after processing.
93
+
94
+ ### `logger`
95
+
96
+ A logger to use. Defaults to ruby's `Logger` on `STDOUT`.
97
+
98
+ ## Running
99
+
100
+ InboxSync provides a `Runner` class that will loop indefinitely, running syncs every `:interval` seconds. Stick it in a daemon, a rake task, a CLI, or whatever depending on how you want to invoke it. Here is an example using it in a basic ruby script:
101
+
102
+ ```ruby
103
+ require 'inbox-sync'
104
+
105
+ sync = InboxSync.new.configure do
106
+ source.host 'imap.gmail.com'
107
+ source.port 993
108
+ source.ssl 'Yes'
109
+ source.login 'joetest@kellyredding.com', 'joetest1'
110
+
111
+ dest.host 'imap.gmail.com'
112
+ dest.port 993
113
+ dest.ssl 'Yes'
114
+ dest.login 'suetest@kellyredding.com', 'suetest1'
115
+
116
+ notify.host 'smtp.gmail.com'
117
+ notify.port 587
118
+ notify.tls 'Yes'
119
+ notify.helo 'gmail.com'
120
+ notify.login 'joetest@kellyredding.com', 'joetest1'
121
+ notify.to_addr 'joetest@kellyredding.com'
122
+ notify.to_addr 'suetest@kellyredding.com'
123
+
124
+ logger Logger.new('log/inbox-sync.log')
125
+ end
126
+
127
+ InboxSync.run(sync, :interval => 20)
128
+ ```
129
+
130
+ The `InboxSync.run` method is just a macro for creating a runner and calling its `start` method.
131
+
132
+ ```ruby
133
+ InboxSync::Runner.new(sync, :interval => 5).start
134
+ ```
135
+
136
+ By default, it will log to `STDOUT` but accepts a `:logger` option to override this.
137
+
138
+ ```ruby
139
+ InboxSync.run(sync, {
140
+ :interval => 5,
141
+ :logger => Logger.new('/path/to/log.log')
142
+ })
143
+ ```
144
+
145
+ You can pass any number of syncs to run. Each `:interval` period, it will run them sequentially:
146
+
147
+ ```ruby
148
+ InboxSync.run(sync1, sync2, sync3, :interval => 5)
149
+ ```
150
+
151
+ If you pass no `:interval` option (or pass a negative value for it), the runner will run the sync(s) once and then exit instead of running the syncs indefinitely on the interval.
152
+
153
+ ```ruby
154
+ InboxSync.run(sync)
155
+ ```
156
+
157
+ The runner traps `SIGINT` and `SIGQUIT` and will shutdown nicely once any in-progress syncs have finished.
158
+
159
+ ## Filter Framework
160
+
161
+ TODO
162
+
163
+ ## Error Handling
164
+
165
+ InboxSync generates detailed logs of both running its syncs and processing sync mail items. If a mail fails to append (ie rejected by the dest IMAP), InboxSync will attempt to strip the mail to its most basic (ie plain/text) form and will retry the append.
166
+
167
+ In addtion, InboxSync will notify via email when something goes wrong with a sync. You configure `notify` settings when defining your syncs. These settings determine where/how notifications are sent out. There are two types a notifications InboxSync will send: `RunSyncError` and `SyncMailItemError`.
168
+
169
+ In any case, if an `archive_folder` is set, no source messages will be permanently deleted and are always available there for reference.
170
+
171
+ ### `RunSyncError` notification
172
+
173
+ This notification is sent when there is a problem running a sync in general. For example, the sync can't connect to the source to read its mail items or the runner itself has a runtime exception. This notification lets you know that something went wrong and that mail items aren't being sync'd. It also details the exception that happened with a full backtrace.
174
+
175
+ ### `SyncMailItemError` notification
176
+
177
+ This notification is sent wnen there is a problem syncing a specific mail item. For example the destination rejects the append or there was a problem archiving the mail item at the source. It lets you know there was a problem and gives you some info about the email that had a problem. It also details the exception that happened with a full backtrace.
178
+
179
+ ## Contributing
180
+
181
+ 1. Fork it
182
+ 2. Create your feature branch (`git checkout -b my-new-feature`)
183
+ 3. Commit your changes (`git commit -am 'Added some feature'`)
184
+ 4. Push to the branch (`git push origin my-new-feature`)
185
+ 5. Create new Pull Request
data/Rakefile ADDED
@@ -0,0 +1,8 @@
1
+ #!/usr/bin/env rake
2
+
3
+ require 'assert/rake_tasks'
4
+ Assert::RakeTasks.for(:test)
5
+
6
+ require 'bundler/gem_tasks'
7
+
8
+ task :default => :build
@@ -0,0 +1,23 @@
1
+ # -*- encoding: utf-8 -*-
2
+ require File.expand_path('../lib/inbox-sync/version', __FILE__)
3
+
4
+ Gem::Specification.new do |gem|
5
+ gem.name = "inbox-sync"
6
+ gem.version = InboxSync::VERSION
7
+ gem.description = %q{Move messages from one inbox to another}
8
+ gem.summary = %q{Move messages from one inbox to another}
9
+
10
+ gem.authors = ["Kelly Redding"]
11
+ gem.email = ["kelly@kellyredding.com"]
12
+ gem.homepage = "http://github.com/kellyredding/inbox-sync"
13
+
14
+ gem.files = `git ls-files`.split("\n")
15
+ gem.test_files = `git ls-files -- {test,spec,features}/*`.split("\n")
16
+ gem.executables = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
17
+ gem.require_paths = ["lib"]
18
+
19
+ gem.add_development_dependency("assert")
20
+
21
+ gem.add_dependency("ns-options", ["~> 0.4.1"])
22
+ gem.add_dependency("mail", ["~> 2.4"])
23
+ end
@@ -0,0 +1,29 @@
1
+ require 'ns-options'
2
+
3
+ module InboxSync; end
4
+ class InboxSync::Config
5
+
6
+ class Credentials
7
+ include NsOptions::Proxy
8
+
9
+ opt :user, :required => true
10
+ opt :pw, :required => true
11
+
12
+ def initialize(*args)
13
+ the_args = args.flatten
14
+ if the_args.size == 1
15
+ self.apply(args.last)
16
+ else
17
+ self.user, self.pw = the_args
18
+ end
19
+ end
20
+
21
+ def validate!
22
+ if !required_set?
23
+ raise ArgumentError, "some required configs are missing"
24
+ end
25
+ end
26
+
27
+ end
28
+
29
+ end
@@ -0,0 +1,28 @@
1
+ require 'ns-options'
2
+ require 'ns-options/boolean'
3
+ require 'inbox-sync/config/credentials'
4
+
5
+ module InboxSync; end
6
+ class InboxSync::Config
7
+
8
+ class IMAPConfig
9
+ include NsOptions::Proxy
10
+
11
+ opt :host, :required => true
12
+ opt :port, :default => 143, :required => true
13
+ opt :ssl, NsOptions::Boolean, :default => false, :required => true
14
+ opt :login, Credentials, :required => true, :default => {}
15
+ opt :inbox, :default => "INBOX", :required => true
16
+ opt :expunge, NsOptions::Boolean, :default => true, :required => true
17
+
18
+ def validate!
19
+ if !required_set?
20
+ raise ArgumentError, "some required configs are missing"
21
+ end
22
+
23
+ login.validate!
24
+ end
25
+
26
+ end
27
+
28
+ end
@@ -0,0 +1,30 @@
1
+ require 'ns-options'
2
+ require 'ns-options/boolean'
3
+ require 'inbox-sync/config/credentials'
4
+
5
+ module InboxSync; end
6
+ class InboxSync::Config
7
+
8
+ class SMTPConfig
9
+ include NsOptions::Proxy
10
+
11
+ opt :host, :required => true
12
+ opt :port, :default => 25, :required => true
13
+ opt :tls, NsOptions::Boolean, :default => false, :required => true
14
+ opt :helo, :required => true
15
+ opt :login, Credentials, :required => true, :default => {}
16
+ opt :authtype, :default => :login, :required => true
17
+ opt :from_addr, :required => true
18
+ opt :to_addr, :required => true
19
+
20
+ def validate!
21
+ if !required_set?
22
+ raise ArgumentError, "some required configs are missing"
23
+ end
24
+
25
+ login.validate!
26
+ end
27
+
28
+ end
29
+
30
+ end
@@ -0,0 +1,30 @@
1
+ require 'logger'
2
+ require 'ns-options'
3
+ require 'inbox-sync/config/imap_config'
4
+ require 'inbox-sync/config/smtp_config'
5
+
6
+ module InboxSync
7
+
8
+ class Config
9
+ include NsOptions::Proxy
10
+
11
+ opt :source, IMAPConfig, :required => true, :default => {}
12
+ opt :dest, IMAPConfig, :required => true, :default => {}
13
+ opt :notify, SMTPConfig, :required => true, :default => {}
14
+
15
+ opt :archive_folder, :default => 'Archived'
16
+ opt :logger, Logger, :required => true, :default => STDOUT
17
+
18
+ def validate!
19
+ if !required_set?
20
+ raise ArgumentError, "some required configs are missing"
21
+ end
22
+
23
+ source.validate!
24
+ dest.validate!
25
+ notify.validate!
26
+ end
27
+
28
+ end
29
+
30
+ end
@@ -0,0 +1,78 @@
1
+ require 'mail'
2
+
3
+ module InboxSync
4
+
5
+ class MailItem
6
+
7
+ def self.find(imap)
8
+ imap.uid_search(['ALL']).
9
+ map do |uid|
10
+ [uid, imap.uid_fetch(uid, ['RFC822', 'INTERNALDATE']).first]
11
+ end.
12
+ map do |uid_meta|
13
+ self.new(
14
+ uid_meta.first,
15
+ uid_meta.last.attr['RFC822'],
16
+ uid_meta.last.attr["INTERNALDATE"]
17
+ )
18
+ end
19
+ end
20
+
21
+ attr_reader :uid, :meta, :message
22
+
23
+ def initialize(uid, rfc822, internal_date)
24
+ @uid = uid
25
+ @meta = {
26
+ 'RFC822' => rfc822,
27
+ 'INTERNALDATE' => internal_date
28
+ }
29
+ @message = ::Mail.new(rfc822)
30
+ end
31
+
32
+ def name
33
+ "[#{@uid}] #{@message.from}: #{@message.subject.inspect} (#{time_s(@message.date)})"
34
+ end
35
+
36
+ # Returns a stripped down version of the mail item
37
+ # The stripped down versions is just the 'text/plain' part of multipart
38
+ # mail items. If the original mail item was not multipart, then the
39
+ # stripped down version is the same as the original.
40
+ # This implies that stripped down mail items have no attachments.
41
+
42
+ def stripped
43
+ @stripped ||= strip_down(MailItem.new(
44
+ self.uid,
45
+ self.meta['RFC822'],
46
+ self.meta["INTERNALDATE"]
47
+ ))
48
+ end
49
+
50
+ def inspect
51
+ "#<#{self.class}:#{'0x%x' % (self.object_id << 1)}: @uid=#{@uid.inspect}, from=#{@message.from.inspect}, subject=#{@message.subject.inspect}, 'INTERNALDATE'=#{@meta['INTERNALDATE'].inspect}>"
52
+ end
53
+
54
+ private
55
+
56
+ def time_s(datetime)
57
+ datetime.strftime("%a %b %-d %Y, %I:%M %p")
58
+ end
59
+
60
+ def strip_down(mail_item)
61
+ message = mail_item.message
62
+ if message.multipart?
63
+ message.parts.delete_if do |part|
64
+ !part.content_type.match(/text\/plain/)
65
+ end
66
+ message.parts.first.body = strip_down_body_s(message.parts.first.body)
67
+ mail_item.meta['RFC822'] = message.to_s
68
+ end
69
+ mail_item
70
+ end
71
+
72
+ def strip_down_body_s(body_s)
73
+ "**[inbox-sync] stripped down to just plain text part**\n\n#{body_s}"
74
+ end
75
+
76
+ end
77
+
78
+ end
@@ -0,0 +1,47 @@
1
+ require 'mail'
2
+
3
+ module InboxSync; end
4
+ module InboxSync::Notice
5
+
6
+ class Base
7
+
8
+ attr_reader :mail
9
+
10
+ def initialize(smtp, config)
11
+ @smtp = smtp
12
+ @config = config
13
+
14
+ @mail = ::Mail.new
15
+ @mail.from = self.from
16
+ @mail.to = self.to
17
+ @mail.subject = self.subject
18
+ @mail.body = self.body
19
+ end
20
+
21
+ def from; @config.from_addr; end
22
+ def to; @config.to_addr; end
23
+
24
+ def subject(msg="notice")
25
+ "[inbox-sync] #{msg}"
26
+ end
27
+
28
+ def body
29
+ raise RuntimeError, "subclass `Notice::Base` and define your body"
30
+ end
31
+
32
+ def send
33
+ @smtp.start(helo, user, pw, authtype) do |smtp|
34
+ smtp.send_message(@mail.to_s, from, to)
35
+ end
36
+ end
37
+
38
+ protected
39
+
40
+ def helo; @config.helo; end
41
+ def user; @config.login.user; end
42
+ def pw; @config.login.pw; end
43
+ def authtype; @config.authtype; end
44
+
45
+ end
46
+
47
+ end
@@ -0,0 +1,44 @@
1
+ require 'inbox-sync/notice/base'
2
+
3
+ module InboxSync; end
4
+ module InboxSync::Notice
5
+
6
+ class RunSyncError < Base
7
+
8
+ BODY = %{
9
+ :sync_name
10
+
11
+ An error happened while running this sync. The error has
12
+ been logged but no mail items from this sync's source are
13
+ being sync'd. The runner will continue to attempt this
14
+ sync so mails like this will continue until the problem
15
+ is fixed.
16
+
17
+ Error
18
+ =====
19
+ :error_message (:error_name)
20
+ :error_backtrace
21
+ }.strip.freeze
22
+
23
+ def initialize(smtp, config, data={})
24
+ @error = data[:error]
25
+ @sync = data[:sync]
26
+
27
+ super(smtp, config)
28
+ end
29
+
30
+ def subject
31
+ super("sync run error (#{@sync.uid})")
32
+ end
33
+
34
+ def body
35
+ @body ||= BODY.
36
+ gsub(':sync_name', @sync.name).
37
+ gsub(':error_message', @error.message).
38
+ gsub(':error_name', @error.class.name).
39
+ gsub(':error_backtrace', @error.backtrace.join("\n "))
40
+ end
41
+
42
+ end
43
+
44
+ end
@@ -0,0 +1,45 @@
1
+ require 'inbox-sync/notice/base'
2
+
3
+ module InboxSync; end
4
+ module InboxSync::Notice
5
+
6
+ class SyncMailItemError < Base
7
+
8
+ BODY = %{
9
+ :sync_name
10
+ :mail_item_name
11
+
12
+ An error happened while syncing this mail item. The error
13
+ has been logged and the mail item has been archived on the
14
+ source. The sync will continue processing new mail items.
15
+
16
+ Error
17
+ =====
18
+ :error_message (:error_name)
19
+ :error_backtrace
20
+ }.strip.freeze
21
+
22
+ def initialize(smtp, config, data={})
23
+ @error = data[:error]
24
+ @mail_item = data[:mail_item]
25
+ @sync = data[:sync]
26
+
27
+ super(smtp, config)
28
+ end
29
+
30
+ def subject
31
+ super("mail item sync error (#{@mail_item.uid}, #{@sync.uid})")
32
+ end
33
+
34
+ def body
35
+ @body ||= BODY.
36
+ gsub(':sync_name', @sync.name).
37
+ gsub(':mail_item_name', @mail_item.name).
38
+ gsub(':error_message', @error.message).
39
+ gsub(':error_name', @error.class.name).
40
+ gsub(':error_backtrace', @error.backtrace.join("\n "))
41
+ end
42
+
43
+ end
44
+
45
+ end