gitlab-mail_room 0.0.10 → 0.0.11
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +88 -9
- data/lib/mail_room/mailbox.rb +42 -3
- data/lib/mail_room/mailbox_watcher.rb +7 -1
- data/lib/mail_room/microsoft_graph.rb +7 -0
- data/lib/mail_room/microsoft_graph/connection.rb +216 -0
- data/lib/mail_room/version.rb +1 -1
- data/mail_room.gemspec +2 -0
- data/spec/lib/mailbox_spec.rb +48 -0
- data/spec/lib/mailbox_watcher_spec.rb +54 -38
- data/spec/lib/microsoft_graph/connection_spec.rb +183 -0
- data/spec/spec_helper.rb +10 -0
- metadata +34 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: aed4391d3a022966a3f2adda086e9b2e059d3ce639755c088fb865a3fa694ffc
|
4
|
+
data.tar.gz: 8c4f102bb223f73f914365d6c77dc7793b2149dbc47ce015e4c888de527d595d
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 3018e959345d18859c40aae953bfcad1a779136f27cd9284da9a1c2edc44f6fcf5436aba4c2911ba0bb36147e4c066293768539802d4ff040a5136a16530a7cf
|
7
|
+
data.tar.gz: afd251026ab38a1d54ac2e9fa88252a68f72971caa657b736e936f9151382879e05f2c90aa1a9026efac739cfa01a3604b6f48f542f3d098af8c20a0a59be5eb
|
data/README.md
CHANGED
@@ -20,7 +20,14 @@ The fork is useful as we can post quick fixes to our own fork and release fixes
|
|
20
20
|
|
21
21
|
## README
|
22
22
|
|
23
|
-
mail_room is a configuration based process that will
|
23
|
+
mail_room is a configuration based process that will listen for incoming
|
24
|
+
e-mail and execute a delivery method when a new message is
|
25
|
+
received. mail_room supports the following methods for receiving e-mail:
|
26
|
+
|
27
|
+
* IMAP
|
28
|
+
* [Microsoft Graph API](https://docs.microsoft.com/en-us/graph/api/resources/mail-api-overview?view=graph-rest-1.0)
|
29
|
+
|
30
|
+
Examples of delivery methods include:
|
24
31
|
|
25
32
|
* POST to a delivery URL (Postback)
|
26
33
|
* Queue a job to Sidekiq or Que for later processing (Sidekiq or Que)
|
@@ -117,6 +124,20 @@ You will also need to install `faraday` or `letter_opener` if you use the `postb
|
|
117
124
|
:host: 127.0.0.1
|
118
125
|
:port: 26379
|
119
126
|
:worker: EmailReceiverWorker
|
127
|
+
-
|
128
|
+
:email: "user7@outlook365.com"
|
129
|
+
:password: "password"
|
130
|
+
:name: "inbox"
|
131
|
+
:inbox_method: microsoft_graph
|
132
|
+
:inbox_options:
|
133
|
+
:tenant_id: 12345
|
134
|
+
:client_id: ABCDE
|
135
|
+
:client_secret: YOUR-SECRET-HERE
|
136
|
+
:poll_interval: 60
|
137
|
+
:delivery_method: sidekiq
|
138
|
+
:delivery_options:
|
139
|
+
:redis_url: redis://localhost:6379
|
140
|
+
:worker: EmailReceiverWorker
|
120
141
|
```
|
121
142
|
|
122
143
|
**Note:** If using `delete_after_delivery`, you also probably want to use
|
@@ -134,6 +155,72 @@ the server is running. Otherwise, it returns a 500 status code.
|
|
134
155
|
|
135
156
|
This feature is not included in upstream `mail_room` and is specific to GitLab.
|
136
157
|
|
158
|
+
## inbox_method
|
159
|
+
|
160
|
+
By default, IMAP mode is assumed for reading a mailbox.
|
161
|
+
|
162
|
+
### IMAP Server Configuration ##
|
163
|
+
|
164
|
+
You can set per-mailbox configuration for the IMAP server's `host` (default: 'imap.gmail.com'), `port` (default: 993), `ssl` (default: true), and `start_tls` (default: false).
|
165
|
+
|
166
|
+
If you want to set additional options for IMAP SSL you can pass a YAML hash to match [SSLContext#set_params](http://docs.ruby-lang.org/en/2.2.0/OpenSSL/SSL/SSLContext.html#method-i-set_params). If you set `verify_mode` to `:none` it'll replace with the appropriate constant.
|
167
|
+
|
168
|
+
If you're seeing the error `Please log in via your web browser: https://support.google.com/mail/accounts/answer/78754 (Failure)`, you need to configure your Gmail account to allow less secure apps to access it: https://support.google.com/accounts/answer/6010255.
|
169
|
+
|
170
|
+
### Microsoft Graph configuration
|
171
|
+
|
172
|
+
To use the Microsoft Graph API instead of IMAP to read e-mail, you will
|
173
|
+
need to create an application in the Azure Active Directory. See the
|
174
|
+
[Microsoft instructions](https://docs.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app) for more details:
|
175
|
+
|
176
|
+
1. Sign in to the [Azure portal](https://portal.azure.com).
|
177
|
+
1. Search for and select `Azure Active Directory`.
|
178
|
+
1. Under `Manage`, select `App registrations` > `New registration`.
|
179
|
+
1. Enter a `Name` for your application, such as `MailRoom`. Users of your app might see this name, and you can change it later.
|
180
|
+
1. If `Supported account types` is listed, select the appropriate option.
|
181
|
+
1. Leave `Redirect URI` blank. This is not needed.
|
182
|
+
1. Select `Register`.
|
183
|
+
1. Under `Manage`, select `Certificates & secrets`.
|
184
|
+
1. Under `Client secrets`, select `New client secret`, and enter a name.
|
185
|
+
1. Under `Expires`, select `Never`, unless you plan on updating the credentials every time it expires.
|
186
|
+
1. Select `Add`. Record the secret value in a safe location for use in a later step.
|
187
|
+
1. Under `Manage`, select `API Permissions` > `Add a permission`. Select `Microsoft Graph`.
|
188
|
+
1. Select `Application permissions`.
|
189
|
+
1. Under the `Mail` node, select `Mail.ReadWrite`, and then select Add permissions.
|
190
|
+
1. If `User.Read` is listed in the permission list, you can delete this.
|
191
|
+
1. Click `Grant admin consent` for these permissions.
|
192
|
+
|
193
|
+
#### Restrict mailbox access
|
194
|
+
|
195
|
+
Note that for MailRoom to work as a service account, this application
|
196
|
+
must have the `Mail.ReadWrite` to read/write mail in *all*
|
197
|
+
mailboxes. However, while this appears to be security risk,
|
198
|
+
we can configure an application access policy to limit the
|
199
|
+
mailbox access for this account. [Follow these instructions](https://docs.microsoft.com/en-us/graph/auth-limit-mailbox-access)
|
200
|
+
to setup PowerShell and configure this policy.
|
201
|
+
|
202
|
+
#### MailRoom config for Microsoft Graph
|
203
|
+
|
204
|
+
In the MailRoom configuration, set `inbox_method` to `microsoft_graph`.
|
205
|
+
You will also need:
|
206
|
+
|
207
|
+
* The client and tenant ID from the `Overview` section in the Azure app page
|
208
|
+
* The client secret created earlier
|
209
|
+
|
210
|
+
Fill in `inbox_options` with these values:
|
211
|
+
|
212
|
+
```yaml
|
213
|
+
:inbox_method: microsoft_graph
|
214
|
+
:inbox_options:
|
215
|
+
:tenant_id: 12345
|
216
|
+
:client_id: ABCDE
|
217
|
+
:client_secret: YOUR-SECRET-HERE
|
218
|
+
:poll_interval: 60
|
219
|
+
```
|
220
|
+
|
221
|
+
By default, MailRoom will poll for new messages every 60 seconds. `poll_interval` configures the number of
|
222
|
+
seconds to poll. Setting the value to 0 or under will default to 60 seconds.
|
223
|
+
|
137
224
|
## delivery_method ##
|
138
225
|
|
139
226
|
### postback ###
|
@@ -284,14 +371,6 @@ If you'd prefer not to wait that long, you can pass `idle_timeout` in seconds fo
|
|
284
371
|
|
285
372
|
This setting allows configuration of the IMAP search command sent to the server. This still defaults 'UNSEEN'. You may find that 'NEW' works better for you.
|
286
373
|
|
287
|
-
## IMAP Server Configuration ##
|
288
|
-
|
289
|
-
You can set per-mailbox configuration for the IMAP server's `host` (default: 'imap.gmail.com'), `port` (default: 993), `ssl` (default: true), and `start_tls` (default: false).
|
290
|
-
|
291
|
-
If you want to set additional options for IMAP SSL you can pass a YAML hash to match [SSLContext#set_params](http://docs.ruby-lang.org/en/2.2.0/OpenSSL/SSL/SSLContext.html#method-i-set_params). If you set `verify_mode` to `:none` it'll replace with the appropriate constant.
|
292
|
-
|
293
|
-
If you're seeing the error `Please log in via your web browser: https://support.google.com/mail/accounts/answer/78754 (Failure)`, you need to configure your Gmail account to allow less secure apps to access it: https://support.google.com/accounts/answer/6010255.
|
294
|
-
|
295
374
|
## Running in Production ##
|
296
375
|
|
297
376
|
I suggest running with either upstart or init.d. Check out this wiki page for some example scripts for both: https://github.com/tpitale/mail_room/wiki/Init-Scripts-for-Running-mail_room
|
data/lib/mail_room/mailbox.rb
CHANGED
@@ -1,11 +1,14 @@
|
|
1
1
|
require "mail_room/delivery"
|
2
2
|
require "mail_room/arbitration"
|
3
3
|
require "mail_room/imap"
|
4
|
+
require "mail_room/microsoft_graph"
|
4
5
|
|
5
6
|
module MailRoom
|
6
7
|
# Mailbox Configuration fields
|
7
8
|
MAILBOX_FIELDS = [
|
8
9
|
:email,
|
10
|
+
:inbox_method,
|
11
|
+
:inbox_options,
|
9
12
|
:password,
|
10
13
|
:host,
|
11
14
|
:port,
|
@@ -42,7 +45,9 @@ module MailRoom
|
|
42
45
|
# 29 minutes, as suggested by the spec: https://tools.ietf.org/html/rfc2177
|
43
46
|
IMAP_IDLE_TIMEOUT = 29 * 60 # 29 minutes in in seconds
|
44
47
|
|
45
|
-
|
48
|
+
IMAP_CONFIGURATION = [:name, :email, :password, :host, :port].freeze
|
49
|
+
MICROSOFT_GRAPH_CONFIGURATION = [:name, :email].freeze
|
50
|
+
MICROSOFT_GRAPH_INBOX_OPTIONS = [:tenant_id, :client_id, :client_secret].freeze
|
46
51
|
|
47
52
|
# Default attributes for the mailbox configuration
|
48
53
|
DEFAULTS = {
|
@@ -122,14 +127,32 @@ module MailRoom
|
|
122
127
|
{ email: self.email, name: self.name }
|
123
128
|
end
|
124
129
|
|
130
|
+
def imap?
|
131
|
+
!microsoft_graph?
|
132
|
+
end
|
133
|
+
|
134
|
+
def microsoft_graph?
|
135
|
+
self[:inbox_method].to_s == 'microsoft_graph'
|
136
|
+
end
|
137
|
+
|
125
138
|
def validate!
|
139
|
+
if microsoft_graph?
|
140
|
+
validate_microsoft_graph!
|
141
|
+
else
|
142
|
+
validate_imap!
|
143
|
+
end
|
144
|
+
end
|
145
|
+
|
146
|
+
private
|
147
|
+
|
148
|
+
def validate_imap!
|
126
149
|
if self[:idle_timeout] > IMAP_IDLE_TIMEOUT
|
127
150
|
raise IdleTimeoutTooLarge,
|
128
151
|
"Please use an idle timeout smaller than #{29*60} to prevent " \
|
129
152
|
"IMAP server disconnects"
|
130
153
|
end
|
131
154
|
|
132
|
-
|
155
|
+
IMAP_CONFIGURATION.each do |k|
|
133
156
|
if self[k].nil?
|
134
157
|
raise ConfigurationError,
|
135
158
|
"Field :#{k} is required in Mailbox: #{inspect}"
|
@@ -137,7 +160,23 @@ module MailRoom
|
|
137
160
|
end
|
138
161
|
end
|
139
162
|
|
140
|
-
|
163
|
+
def validate_microsoft_graph!
|
164
|
+
raise ConfigurationError, "Missing inbox_options in Mailbox: #{inspect}" unless self.inbox_options.is_a?(Hash)
|
165
|
+
|
166
|
+
MICROSOFT_GRAPH_CONFIGURATION.each do |k|
|
167
|
+
if self[k].nil?
|
168
|
+
raise ConfigurationError,
|
169
|
+
"Field :#{k} is required in Mailbox: #{inspect}"
|
170
|
+
end
|
171
|
+
end
|
172
|
+
|
173
|
+
MICROSOFT_GRAPH_INBOX_OPTIONS.each do |k|
|
174
|
+
if self[:inbox_options][k].nil?
|
175
|
+
raise ConfigurationError,
|
176
|
+
"inbox_options field :#{k} is required in Mailbox: #{inspect}"
|
177
|
+
end
|
178
|
+
end
|
179
|
+
end
|
141
180
|
|
142
181
|
def parsed_arbitration_options
|
143
182
|
arbitration_klass::Options.new(self)
|
@@ -62,8 +62,14 @@ module MailRoom
|
|
62
62
|
end
|
63
63
|
|
64
64
|
private
|
65
|
+
|
65
66
|
def connection
|
66
|
-
@connection ||=
|
67
|
+
@connection ||=
|
68
|
+
if @mailbox.microsoft_graph?
|
69
|
+
::MailRoom::MicrosoftGraph::Connection.new(@mailbox)
|
70
|
+
else
|
71
|
+
::MailRoom::IMAP::Connection.new(@mailbox)
|
72
|
+
end
|
67
73
|
end
|
68
74
|
end
|
69
75
|
end
|
@@ -0,0 +1,216 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'json'
|
4
|
+
require 'oauth2'
|
5
|
+
|
6
|
+
module MailRoom
|
7
|
+
module MicrosoftGraph
|
8
|
+
class Connection < MailRoom::Connection
|
9
|
+
SCOPE = 'https://graph.microsoft.com/.default'
|
10
|
+
NEXT_PAGE_KEY = '@odata.nextLink'
|
11
|
+
DEFAULT_POLL_INTERVAL_S = 60
|
12
|
+
|
13
|
+
TooManyRequestsError = Class.new(RuntimeError)
|
14
|
+
|
15
|
+
attr_accessor :token, :throttled_count
|
16
|
+
|
17
|
+
def initialize(mailbox)
|
18
|
+
super
|
19
|
+
|
20
|
+
reset
|
21
|
+
setup
|
22
|
+
end
|
23
|
+
|
24
|
+
def wait
|
25
|
+
process_mailbox
|
26
|
+
|
27
|
+
@throttled_count = 0
|
28
|
+
wait_for_new_messages
|
29
|
+
rescue TooManyRequestsError => e
|
30
|
+
@throttled_count += 1
|
31
|
+
|
32
|
+
@mailbox.logger.warn({ context: @mailbox.context, action: 'Too many requests, backing off...', backoff_s: backoff_secs, error: e.message, error_backtrace: e.backtrace })
|
33
|
+
|
34
|
+
backoff
|
35
|
+
rescue OAuth2::Error, IOError => e
|
36
|
+
@mailbox.logger.warn({ context: @mailbox.context, action: 'Disconnected. Resetting...', error: e.message, error_backtrace: e.backtrace })
|
37
|
+
|
38
|
+
reset
|
39
|
+
setup
|
40
|
+
end
|
41
|
+
|
42
|
+
private
|
43
|
+
|
44
|
+
def wait_for_new_messages
|
45
|
+
sleep poll_interval
|
46
|
+
end
|
47
|
+
|
48
|
+
def backoff
|
49
|
+
sleep backoff_secs
|
50
|
+
end
|
51
|
+
|
52
|
+
def backoff_secs
|
53
|
+
[60 * 10, 2**throttled_count].min
|
54
|
+
end
|
55
|
+
|
56
|
+
def reset
|
57
|
+
@token = nil
|
58
|
+
@throttled_count = 0
|
59
|
+
end
|
60
|
+
|
61
|
+
def setup
|
62
|
+
@mailbox.logger.info({ context: @mailbox.context, action: 'Retrieving OAuth2 token...' })
|
63
|
+
|
64
|
+
@token = client.client_credentials.get_token({ scope: SCOPE })
|
65
|
+
end
|
66
|
+
|
67
|
+
def client
|
68
|
+
@client ||= OAuth2::Client.new(client_id, client_secret,
|
69
|
+
site: 'https://login.microsoftonline.com',
|
70
|
+
authorize_url: "/#{tenant_id}/oauth2/v2.0/authorize",
|
71
|
+
token_url: "/#{tenant_id}/oauth2/v2.0/token",
|
72
|
+
auth_scheme: :basic_auth)
|
73
|
+
end
|
74
|
+
|
75
|
+
def inbox_options
|
76
|
+
mailbox.inbox_options
|
77
|
+
end
|
78
|
+
|
79
|
+
def tenant_id
|
80
|
+
inbox_options[:tenant_id]
|
81
|
+
end
|
82
|
+
|
83
|
+
def client_id
|
84
|
+
inbox_options[:client_id]
|
85
|
+
end
|
86
|
+
|
87
|
+
def client_secret
|
88
|
+
inbox_options[:client_secret]
|
89
|
+
end
|
90
|
+
|
91
|
+
def poll_interval
|
92
|
+
@poll_interval ||= begin
|
93
|
+
interval = inbox_options[:poll_interval].to_i
|
94
|
+
|
95
|
+
if interval.positive?
|
96
|
+
interval
|
97
|
+
else
|
98
|
+
DEFAULT_POLL_INTERVAL_S
|
99
|
+
end
|
100
|
+
end
|
101
|
+
end
|
102
|
+
|
103
|
+
def process_mailbox
|
104
|
+
return unless @new_message_handler
|
105
|
+
|
106
|
+
@mailbox.logger.info({ context: @mailbox.context, action: 'Processing started' })
|
107
|
+
|
108
|
+
new_messages.each do |msg|
|
109
|
+
success = @new_message_handler.call(msg)
|
110
|
+
handle_delivered(msg) if success
|
111
|
+
end
|
112
|
+
end
|
113
|
+
|
114
|
+
def handle_delivered(msg)
|
115
|
+
mark_as_read(msg)
|
116
|
+
delete_message(msg) if @mailbox.delete_after_delivery
|
117
|
+
end
|
118
|
+
|
119
|
+
def delete_message(msg)
|
120
|
+
token.delete(msg_url(msg.uid))
|
121
|
+
end
|
122
|
+
|
123
|
+
def mark_as_read(msg)
|
124
|
+
token.patch(msg_url(msg.uid),
|
125
|
+
headers: { 'Content-Type' => 'application/json' },
|
126
|
+
body: { isRead: true }.to_json)
|
127
|
+
end
|
128
|
+
|
129
|
+
def new_messages
|
130
|
+
messages_for_ids(new_message_ids)
|
131
|
+
end
|
132
|
+
|
133
|
+
# Yields a page of message IDs at a time
|
134
|
+
def new_message_ids
|
135
|
+
url = unread_messages_url
|
136
|
+
|
137
|
+
Enumerator.new do |block|
|
138
|
+
loop do
|
139
|
+
messages, next_page_url = unread_messages(url: url)
|
140
|
+
messages.each { |msg| block.yield msg }
|
141
|
+
|
142
|
+
break unless next_page_url
|
143
|
+
|
144
|
+
url = next_page_url
|
145
|
+
end
|
146
|
+
end
|
147
|
+
end
|
148
|
+
|
149
|
+
def unread_messages(url:)
|
150
|
+
body = get(url)
|
151
|
+
|
152
|
+
return [[], nil] unless body
|
153
|
+
|
154
|
+
all_unread = body['value'].map { |msg| msg['id'] }
|
155
|
+
to_deliver = all_unread.select { |uid| @mailbox.deliver?(uid) }
|
156
|
+
@mailbox.logger.info({ context: @mailbox.context, action: 'Getting new messages',
|
157
|
+
unread: { count: all_unread.count, ids: all_unread },
|
158
|
+
to_be_delivered: { count: to_deliver.count, ids: to_deliver } })
|
159
|
+
[to_deliver, body[NEXT_PAGE_KEY]]
|
160
|
+
rescue TypeError, JSON::ParserError => e
|
161
|
+
log_exception('Error parsing JSON response', e)
|
162
|
+
[[], nil]
|
163
|
+
end
|
164
|
+
|
165
|
+
# Returns the JSON response
|
166
|
+
def get(url)
|
167
|
+
response = token.get(url, { raise_errors: false })
|
168
|
+
|
169
|
+
case response.status
|
170
|
+
when 429
|
171
|
+
raise TooManyRequestsError
|
172
|
+
when 400..599
|
173
|
+
raise OAuth2::Error, response
|
174
|
+
end
|
175
|
+
|
176
|
+
return unless response.body
|
177
|
+
|
178
|
+
body = JSON.parse(response.body)
|
179
|
+
|
180
|
+
raise TypeError, 'Response did not contain value hash' unless body.is_a?(Hash) && body.key?('value')
|
181
|
+
|
182
|
+
body
|
183
|
+
end
|
184
|
+
|
185
|
+
def messages_for_ids(message_ids)
|
186
|
+
message_ids.each_with_object([]) do |id, arr|
|
187
|
+
response = token.get(rfc822_msg_url(id))
|
188
|
+
|
189
|
+
arr << ::MailRoom::Message.new(uid: id, body: response.body)
|
190
|
+
end
|
191
|
+
end
|
192
|
+
|
193
|
+
def base_url
|
194
|
+
"https://graph.microsoft.com/v1.0/users/#{mailbox.email}/mailFolders/#{mailbox.name}/messages"
|
195
|
+
end
|
196
|
+
|
197
|
+
def unread_messages_url
|
198
|
+
"#{base_url}?$filter=isRead eq false"
|
199
|
+
end
|
200
|
+
|
201
|
+
def msg_url(id)
|
202
|
+
# Attempting to use the base_url fails with "The OData request is not supported"
|
203
|
+
"https://graph.microsoft.com/v1.0/users/#{mailbox.email}/messages/#{id}"
|
204
|
+
end
|
205
|
+
|
206
|
+
def rfc822_msg_url(id)
|
207
|
+
# Attempting to use the base_url fails with "The OData request is not supported"
|
208
|
+
"#{msg_url(id)}/$value"
|
209
|
+
end
|
210
|
+
|
211
|
+
def log_exception(message, exception)
|
212
|
+
@mailbox.logger.warn({ context: @mailbox.context, message: message, exception: exception.to_s })
|
213
|
+
end
|
214
|
+
end
|
215
|
+
end
|
216
|
+
end
|
data/lib/mail_room/version.rb
CHANGED
data/mail_room.gemspec
CHANGED
@@ -18,6 +18,7 @@ Gem::Specification.new do |gem|
|
|
18
18
|
gem.require_paths = ["lib"]
|
19
19
|
|
20
20
|
gem.add_dependency "net-imap", ">= 0.2.1"
|
21
|
+
gem.add_dependency "oauth2", "~> 1.4.4"
|
21
22
|
|
22
23
|
gem.add_development_dependency "rake"
|
23
24
|
gem.add_development_dependency "rspec", "~> 3.9"
|
@@ -33,4 +34,5 @@ Gem::Specification.new do |gem|
|
|
33
34
|
gem.add_development_dependency "redis-namespace"
|
34
35
|
gem.add_development_dependency "pg"
|
35
36
|
gem.add_development_dependency "charlock_holmes"
|
37
|
+
gem.add_development_dependency "webmock"
|
36
38
|
end
|
data/spec/lib/mailbox_spec.rb
CHANGED
@@ -3,6 +3,17 @@ require 'spec_helper'
|
|
3
3
|
describe MailRoom::Mailbox do
|
4
4
|
let(:sample_message) { MailRoom::Message.new(uid: 123, body: 'a message') }
|
5
5
|
|
6
|
+
context 'with IMAP configuration' do
|
7
|
+
subject { build_mailbox }
|
8
|
+
|
9
|
+
describe '#imap?' do
|
10
|
+
it 'configured as an IMAP inbox' do
|
11
|
+
expect(subject.imap?).to be true
|
12
|
+
expect(subject.microsoft_graph?).to be false
|
13
|
+
end
|
14
|
+
end
|
15
|
+
end
|
16
|
+
|
6
17
|
describe "#deliver" do
|
7
18
|
context "with arbitration_method of noop" do
|
8
19
|
it 'arbitrates with a Noop instance' do
|
@@ -125,5 +136,42 @@ describe MailRoom::Mailbox do
|
|
125
136
|
expect { build_mailbox({:host => nil}) }.to raise_error(MailRoom::ConfigurationError)
|
126
137
|
end
|
127
138
|
end
|
139
|
+
|
140
|
+
context "with Microsoft Graph configuration" do
|
141
|
+
let(:options) do
|
142
|
+
{
|
143
|
+
arbitration_method: 'redis',
|
144
|
+
}.merge(REQUIRED_MICROSOFT_GRAPH_DEFAULTS)
|
145
|
+
end
|
146
|
+
|
147
|
+
subject { build_mailbox(options) }
|
148
|
+
|
149
|
+
def delete_inbox_option(key)
|
150
|
+
options[:inbox_options] = options[:inbox_options].dup.delete(key)
|
151
|
+
end
|
152
|
+
|
153
|
+
it 'allows password omission' do
|
154
|
+
expect { subject }.not_to raise_error
|
155
|
+
end
|
156
|
+
|
157
|
+
it 'configured as a Microsoft Graph inbox' do
|
158
|
+
expect(subject.imap?).to be false
|
159
|
+
expect(subject.microsoft_graph?).to be true
|
160
|
+
end
|
161
|
+
|
162
|
+
it 'raises an error when the inbox options are not present' do
|
163
|
+
options.delete(:inbox_options)
|
164
|
+
|
165
|
+
expect { subject }.to raise_error(MailRoom::ConfigurationError)
|
166
|
+
end
|
167
|
+
|
168
|
+
%i[tenant_id client_id client_secret].each do |item|
|
169
|
+
it "raises an error when the #{item} is not present" do
|
170
|
+
delete_inbox_option(item)
|
171
|
+
|
172
|
+
expect { subject }.to raise_error(MailRoom::ConfigurationError)
|
173
|
+
end
|
174
|
+
end
|
175
|
+
end
|
128
176
|
end
|
129
177
|
end
|
@@ -1,61 +1,77 @@
|
|
1
1
|
require 'spec_helper'
|
2
2
|
|
3
3
|
describe MailRoom::MailboxWatcher do
|
4
|
-
|
5
|
-
|
6
|
-
|
7
|
-
|
8
|
-
|
9
|
-
|
4
|
+
context 'with IMAP configured' do
|
5
|
+
let(:mailbox) {build_mailbox}
|
6
|
+
|
7
|
+
describe '#running?' do
|
8
|
+
it 'is false by default' do
|
9
|
+
watcher = MailRoom::MailboxWatcher.new(mailbox)
|
10
|
+
expect(watcher.running?).to eq(false)
|
11
|
+
end
|
10
12
|
end
|
11
|
-
end
|
12
13
|
|
13
|
-
|
14
|
-
|
15
|
-
|
14
|
+
describe '#run' do
|
15
|
+
let(:imap) {stub(:login => true, :select => true)}
|
16
|
+
let(:watcher) {MailRoom::MailboxWatcher.new(mailbox)}
|
16
17
|
|
17
|
-
|
18
|
-
|
19
|
-
|
18
|
+
before :each do
|
19
|
+
Net::IMAP.stubs(:new).returns(imap) # prevent connection
|
20
|
+
end
|
20
21
|
|
21
|
-
|
22
|
-
|
22
|
+
it 'loops over wait while running' do
|
23
|
+
connection = MailRoom::IMAP::Connection.new(mailbox)
|
23
24
|
|
24
|
-
|
25
|
+
MailRoom::IMAP::Connection.stubs(:new).returns(connection)
|
25
26
|
|
26
|
-
|
27
|
-
|
28
|
-
|
27
|
+
watcher.expects(:running?).twice.returns(true, false)
|
28
|
+
connection.expects(:wait).once
|
29
|
+
connection.expects(:on_new_message).once
|
29
30
|
|
30
|
-
|
31
|
-
|
31
|
+
watcher.run
|
32
|
+
watcher.watching_thread.join # wait for finishing run
|
33
|
+
end
|
32
34
|
end
|
33
|
-
end
|
34
35
|
|
35
|
-
|
36
|
-
|
37
|
-
|
36
|
+
describe '#quit' do
|
37
|
+
let(:imap) {stub(:login => true, :select => true)}
|
38
|
+
let(:watcher) {MailRoom::MailboxWatcher.new(mailbox)}
|
38
39
|
|
39
|
-
|
40
|
-
|
41
|
-
|
40
|
+
before :each do
|
41
|
+
Net::IMAP.stubs(:new).returns(imap) # prevent connection
|
42
|
+
end
|
43
|
+
|
44
|
+
it 'closes and waits for the connection' do
|
45
|
+
connection = MailRoom::IMAP::Connection.new(mailbox)
|
46
|
+
connection.stubs(:wait)
|
47
|
+
connection.stubs(:quit)
|
42
48
|
|
43
|
-
|
44
|
-
connection = MailRoom::IMAP::Connection.new(mailbox)
|
45
|
-
connection.stubs(:wait)
|
46
|
-
connection.stubs(:quit)
|
49
|
+
MailRoom::IMAP::Connection.stubs(:new).returns(connection)
|
47
50
|
|
48
|
-
|
51
|
+
watcher.run
|
52
|
+
|
53
|
+
expect(watcher.running?).to eq(true)
|
54
|
+
|
55
|
+
connection.expects(:quit)
|
56
|
+
|
57
|
+
watcher.quit
|
58
|
+
|
59
|
+
expect(watcher.running?).to eq(false)
|
60
|
+
end
|
61
|
+
end
|
62
|
+
end
|
49
63
|
|
50
|
-
|
64
|
+
context 'with Microsoft Graph configured' do
|
65
|
+
let(:mailbox) { build_mailbox(REQUIRED_MICROSOFT_GRAPH_DEFAULTS) }
|
51
66
|
|
52
|
-
|
67
|
+
subject { described_class.new(mailbox) }
|
53
68
|
|
54
|
-
|
69
|
+
it 'initializes a Microsoft Graph connection' do
|
70
|
+
connection = stub(on_new_message: nil)
|
55
71
|
|
56
|
-
|
72
|
+
MailRoom::MicrosoftGraph::Connection.stubs(:new).returns(connection)
|
57
73
|
|
58
|
-
expect(
|
74
|
+
expect(subject.send(:connection)).to eq(connection)
|
59
75
|
end
|
60
76
|
end
|
61
77
|
end
|
@@ -0,0 +1,183 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'securerandom'
|
4
|
+
require 'spec_helper'
|
5
|
+
require 'json'
|
6
|
+
require 'webmock/rspec'
|
7
|
+
|
8
|
+
describe MailRoom::MicrosoftGraph::Connection do
|
9
|
+
let(:tenant_id) { options[:inbox_options][:tenant_id] }
|
10
|
+
let(:options) do
|
11
|
+
{
|
12
|
+
delete_after_delivery: true,
|
13
|
+
expunge_deleted: true
|
14
|
+
}.merge(REQUIRED_MICROSOFT_GRAPH_DEFAULTS)
|
15
|
+
end
|
16
|
+
let(:mailbox) { build_mailbox(options) }
|
17
|
+
let(:base_url) { 'https://graph.microsoft.com/v1.0/users/user@example.com/mailFolders/inbox/messages' }
|
18
|
+
let(:message_base_url) { 'https://graph.microsoft.com/v1.0/users/user@example.com/messages' }
|
19
|
+
|
20
|
+
before do
|
21
|
+
WebMock.enable!
|
22
|
+
end
|
23
|
+
|
24
|
+
context '#wait' do
|
25
|
+
let(:connection) { described_class.new(mailbox) }
|
26
|
+
let(:uid) { 1 }
|
27
|
+
let(:access_token) { SecureRandom.hex }
|
28
|
+
let(:refresh_token) { SecureRandom.hex }
|
29
|
+
let(:expires_in) { Time.now + 3600 }
|
30
|
+
let(:unread_messages_body) { '' }
|
31
|
+
let(:status) { 200 }
|
32
|
+
let!(:stub_token) do
|
33
|
+
stub_request(:post, "https://login.microsoftonline.com/#{tenant_id}/oauth2/v2.0/token").to_return(
|
34
|
+
body: { 'access_token' => access_token, 'refresh_token' => refresh_token, 'expires_in' => expires_in }.to_json,
|
35
|
+
headers: { 'Content-Type' => 'application/json' }
|
36
|
+
)
|
37
|
+
end
|
38
|
+
let!(:stub_unread_messages_request) do
|
39
|
+
stub_request(:get, "#{base_url}?$filter=isRead%20eq%20false").to_return(
|
40
|
+
status: status,
|
41
|
+
body: unread_messages_body.to_json,
|
42
|
+
headers: { 'Content-Type' => 'application/json' }
|
43
|
+
)
|
44
|
+
end
|
45
|
+
|
46
|
+
before do
|
47
|
+
connection.stubs(:wait_for_new_messages)
|
48
|
+
end
|
49
|
+
|
50
|
+
describe 'poll interval' do
|
51
|
+
it 'defaults to 60 seconds' do
|
52
|
+
expect(connection.send(:poll_interval)).to eq(60)
|
53
|
+
end
|
54
|
+
|
55
|
+
context 'interval set to 10' do
|
56
|
+
let(:options) do
|
57
|
+
{
|
58
|
+
inbox_method: :microsoft_graph,
|
59
|
+
inbox_options: {
|
60
|
+
tenant_id: '98776',
|
61
|
+
client_id: '12345',
|
62
|
+
client_secret: 'MY-SECRET',
|
63
|
+
poll_interval: '10'
|
64
|
+
}
|
65
|
+
}
|
66
|
+
end
|
67
|
+
|
68
|
+
it 'sets the poll interval to 10' do
|
69
|
+
expect(connection.send(:poll_interval)).to eq(10)
|
70
|
+
end
|
71
|
+
end
|
72
|
+
end
|
73
|
+
|
74
|
+
context 'with a single message' do
|
75
|
+
let(:message_id) { SecureRandom.hex }
|
76
|
+
let(:unread_messages_body) { { value: ['id' => message_id] } }
|
77
|
+
let(:message_url) { "#{message_base_url}/#{message_id}" }
|
78
|
+
let(:message_body) { 'hello world' }
|
79
|
+
|
80
|
+
it 'requests message ID' do
|
81
|
+
stub_get = stub_request(:get, "#{message_url}/$value").to_return(
|
82
|
+
status: 200,
|
83
|
+
body: message_body
|
84
|
+
)
|
85
|
+
stub_patch = stub_request(:patch, message_url).with(body: { "isRead": true }.to_json)
|
86
|
+
stub_delete = stub_request(:delete, message_url)
|
87
|
+
message_count = 0
|
88
|
+
|
89
|
+
connection.on_new_message do |message|
|
90
|
+
message_count += 1
|
91
|
+
expect(message.uid).to eq(message_id)
|
92
|
+
expect(message.body).to eq(message_body)
|
93
|
+
end
|
94
|
+
|
95
|
+
connection.wait
|
96
|
+
|
97
|
+
assert_requested(stub_token)
|
98
|
+
assert_requested(stub_unread_messages_request)
|
99
|
+
assert_requested(stub_get)
|
100
|
+
assert_requested(stub_patch)
|
101
|
+
assert_requested(stub_delete)
|
102
|
+
expect(message_count).to eq(1)
|
103
|
+
end
|
104
|
+
end
|
105
|
+
|
106
|
+
context 'with multiple pages of messages' do
|
107
|
+
let(:message_ids) { [SecureRandom.hex, SecureRandom.hex] }
|
108
|
+
let(:next_page_url) { 'https://graph.microsoft.com/v1.0/nextPage' }
|
109
|
+
let(:unread_messages_body) { { value: ['id' => message_ids.first], '@odata.nextLink' => next_page_url } }
|
110
|
+
let(:message_body) { 'hello world' }
|
111
|
+
|
112
|
+
it 'requests message ID' do
|
113
|
+
stub_request(:get, next_page_url).to_return(
|
114
|
+
status: 200,
|
115
|
+
body: { value: ['id' => message_ids[1]] }.to_json
|
116
|
+
)
|
117
|
+
|
118
|
+
stubs = []
|
119
|
+
message_ids.each do |message_id|
|
120
|
+
rfc822_msg_url = "#{message_base_url}/#{message_id}/$value"
|
121
|
+
stubs << stub_request(:get, rfc822_msg_url).to_return(
|
122
|
+
status: 200,
|
123
|
+
body: message_body
|
124
|
+
)
|
125
|
+
|
126
|
+
msg_url = "#{message_base_url}/#{message_id}"
|
127
|
+
stubs << stub_request(:patch, msg_url).with(body: { "isRead": true }.to_json)
|
128
|
+
stubs << stub_request(:delete, msg_url)
|
129
|
+
end
|
130
|
+
|
131
|
+
message_count = 0
|
132
|
+
|
133
|
+
connection.on_new_message do |message|
|
134
|
+
expect(message.uid).to eq(message_ids[message_count])
|
135
|
+
expect(message.body).to eq(message_body)
|
136
|
+
message_count += 1
|
137
|
+
end
|
138
|
+
|
139
|
+
connection.wait
|
140
|
+
|
141
|
+
stubs.each { |stub| assert_requested(stub) }
|
142
|
+
expect(message_count).to eq(2)
|
143
|
+
end
|
144
|
+
end
|
145
|
+
|
146
|
+
context 'too many requests' do
|
147
|
+
let(:status) { 429 }
|
148
|
+
|
149
|
+
it 'backs off' do
|
150
|
+
connection.expects(:backoff)
|
151
|
+
|
152
|
+
connection.on_new_message {}
|
153
|
+
connection.wait
|
154
|
+
|
155
|
+
expect(connection.throttled_count).to eq(1)
|
156
|
+
end
|
157
|
+
end
|
158
|
+
|
159
|
+
context 'invalid JSON response' do
|
160
|
+
let(:body) { 'this is something' }
|
161
|
+
|
162
|
+
it 'ignores the message and logs a warning' do
|
163
|
+
mailbox.logger.expects(:warn)
|
164
|
+
|
165
|
+
connection.on_new_message {}
|
166
|
+
connection.wait
|
167
|
+
end
|
168
|
+
end
|
169
|
+
|
170
|
+
context '500 error' do
|
171
|
+
let(:status) { 500 }
|
172
|
+
|
173
|
+
it 'resets the state and logs a warning' do
|
174
|
+
connection.expects(:reset)
|
175
|
+
connection.expects(:setup)
|
176
|
+
mailbox.logger.expects(:warn)
|
177
|
+
|
178
|
+
connection.on_new_message {}
|
179
|
+
connection.wait
|
180
|
+
end
|
181
|
+
end
|
182
|
+
end
|
183
|
+
end
|
data/spec/spec_helper.rb
CHANGED
@@ -27,6 +27,16 @@ REQUIRED_MAILBOX_DEFAULTS = {
|
|
27
27
|
:password => "password123"
|
28
28
|
}
|
29
29
|
|
30
|
+
REQUIRED_MICROSOFT_GRAPH_DEFAULTS = {
|
31
|
+
password: nil,
|
32
|
+
inbox_method: :microsoft_graph,
|
33
|
+
inbox_options: {
|
34
|
+
tenant_id: '98776',
|
35
|
+
client_id: '12345',
|
36
|
+
client_secret: 'MY-SECRET',
|
37
|
+
}.freeze
|
38
|
+
}.freeze
|
39
|
+
|
30
40
|
def build_mailbox(options = {})
|
31
41
|
MailRoom::Mailbox.new(REQUIRED_MAILBOX_DEFAULTS.merge(options))
|
32
42
|
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: gitlab-mail_room
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.0.
|
4
|
+
version: 0.0.11
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Tony Pitale
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2021-03-
|
11
|
+
date: 2021-03-30 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: net-imap
|
@@ -24,6 +24,20 @@ dependencies:
|
|
24
24
|
- - ">="
|
25
25
|
- !ruby/object:Gem::Version
|
26
26
|
version: 0.2.1
|
27
|
+
- !ruby/object:Gem::Dependency
|
28
|
+
name: oauth2
|
29
|
+
requirement: !ruby/object:Gem::Requirement
|
30
|
+
requirements:
|
31
|
+
- - "~>"
|
32
|
+
- !ruby/object:Gem::Version
|
33
|
+
version: 1.4.4
|
34
|
+
type: :runtime
|
35
|
+
prerelease: false
|
36
|
+
version_requirements: !ruby/object:Gem::Requirement
|
37
|
+
requirements:
|
38
|
+
- - "~>"
|
39
|
+
- !ruby/object:Gem::Version
|
40
|
+
version: 1.4.4
|
27
41
|
- !ruby/object:Gem::Dependency
|
28
42
|
name: rake
|
29
43
|
requirement: !ruby/object:Gem::Requirement
|
@@ -192,6 +206,20 @@ dependencies:
|
|
192
206
|
- - ">="
|
193
207
|
- !ruby/object:Gem::Version
|
194
208
|
version: '0'
|
209
|
+
- !ruby/object:Gem::Dependency
|
210
|
+
name: webmock
|
211
|
+
requirement: !ruby/object:Gem::Requirement
|
212
|
+
requirements:
|
213
|
+
- - ">="
|
214
|
+
- !ruby/object:Gem::Version
|
215
|
+
version: '0'
|
216
|
+
type: :development
|
217
|
+
prerelease: false
|
218
|
+
version_requirements: !ruby/object:Gem::Requirement
|
219
|
+
requirements:
|
220
|
+
- - ">="
|
221
|
+
- !ruby/object:Gem::Version
|
222
|
+
version: '0'
|
195
223
|
description: mail_room will proxy email (gmail) from IMAP to a delivery method
|
196
224
|
email:
|
197
225
|
- tpitale@gmail.com
|
@@ -236,6 +264,8 @@ files:
|
|
236
264
|
- lib/mail_room/mailbox.rb
|
237
265
|
- lib/mail_room/mailbox_watcher.rb
|
238
266
|
- lib/mail_room/message.rb
|
267
|
+
- lib/mail_room/microsoft_graph.rb
|
268
|
+
- lib/mail_room/microsoft_graph/connection.rb
|
239
269
|
- lib/mail_room/version.rb
|
240
270
|
- logfile.log
|
241
271
|
- mail_room.gemspec
|
@@ -257,6 +287,7 @@ files:
|
|
257
287
|
- spec/lib/mailbox_spec.rb
|
258
288
|
- spec/lib/mailbox_watcher_spec.rb
|
259
289
|
- spec/lib/message_spec.rb
|
290
|
+
- spec/lib/microsoft_graph/connection_spec.rb
|
260
291
|
- spec/spec_helper.rb
|
261
292
|
homepage: http://github.com/tpitale/mail_room
|
262
293
|
licenses: []
|
@@ -300,4 +331,5 @@ test_files:
|
|
300
331
|
- spec/lib/mailbox_spec.rb
|
301
332
|
- spec/lib/mailbox_watcher_spec.rb
|
302
333
|
- spec/lib/message_spec.rb
|
334
|
+
- spec/lib/microsoft_graph/connection_spec.rb
|
303
335
|
- spec/spec_helper.rb
|