mailcannon 0.0.8 → 0.1.0.pre.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.gitignore +1 -0
- data/Gemfile +1 -2
- data/Rakefile +1 -1
- data/Readme.md +39 -4
- data/env.sample.sh +5 -4
- data/lib/mailcannon/adapters/sendgrid_web.rb +48 -12
- data/lib/mailcannon/envelope.rb +12 -3
- data/lib/mailcannon/envelope_bag.rb +9 -0
- data/lib/mailcannon/envelope_bag_map_reduce.rb +106 -0
- data/lib/mailcannon/envelope_bag_statistic.rb +5 -0
- data/lib/mailcannon/reduces/envelope_bag_map.js +3 -0
- data/lib/mailcannon/reduces/envelope_bag_reduce.js +116 -0
- data/lib/mailcannon/sendgrid_event.rb +18 -0
- data/lib/mailcannon/version.rb +4 -3
- data/lib/mailcannon/workers/barrel.rb +2 -16
- data/lib/mailcannon/workers/envelope_bag_reduce_job.rb +11 -0
- data/lib/mailcannon.rb +5 -5
- data/mailcannon.gemspec +0 -2
- data/spec/integration/1k_spec.rb +16 -2
- data/spec/integration/full_stack_stats_spec.rb +118 -0
- data/spec/integration/xsmtpapi_spec.rb +4 -1
- data/spec/mailcannon/adapters/sendgrid_spec.rb +17 -4
- data/spec/mailcannon/envelope_bag_map_reduce_spec.rb +211 -0
- data/spec/mailcannon/envelope_bag_statistic_spec.rb +41 -0
- data/spec/mailcannon/envelope_spec.rb +5 -0
- data/spec/mailcannon/workers/envelope_bag_reduce_job_spec.rb +17 -0
- data/templates/config/mailcannon.yml +10 -1
- metadata +18 -38
- data/lib/mailcannon/airbrake.rb +0 -18
- data/lib/mailcannon/librato.rb +0 -15
- data/spec/mailcannon/airbrake_spec.rb +0 -23
- data/spec/mailcannon/librato_spec.rb +0 -24
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 81d99b16a82ec2314d0e8326bc3c2f93eed2fc03
|
4
|
+
data.tar.gz: 248dba656dc1d563a28bc4384e8566be3a986fa5
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: cf38c7f8ad0596607206f3b69d953f0da7736eee72b271f65736138c3fd4958c2e894ab5cbc1292b975100d88ff9343b3bc06083af0476ec4e77e8e9bbe1064c
|
7
|
+
data.tar.gz: 15135c7aa6a189aff2ce6243123a1d0b8ff68d35f043790db63547167905609245f2b85ba0df1176807bc8605deb85a5f61894e1b841e2d73c0dcc466964bfac
|
data/.gitignore
CHANGED
data/Gemfile
CHANGED
@@ -3,10 +3,9 @@ source 'https://rubygems.org'
|
|
3
3
|
gem 'redis'
|
4
4
|
gem 'activemodel'
|
5
5
|
gem 'mongoid', '>=3.1.6'
|
6
|
-
gem 'sidekiq'
|
6
|
+
gem 'sidekiq', '2.17.7'
|
7
7
|
gem 'sendgrid_webapi', '0.0.3'
|
8
8
|
gem 'librato-metrics'
|
9
|
-
gem 'airbrake'
|
10
9
|
gem 'rubysl', platform: :rbx
|
11
10
|
gem 'jruby-openssl', platform: :jruby
|
12
11
|
gem 'yajl-ruby', :platforms=>[:rbx,:ruby]
|
data/Rakefile
CHANGED
data/Readme.md
CHANGED
@@ -5,13 +5,13 @@
|
|
5
5
|
MailCannon
|
6
6
|
==========
|
7
7
|
|
8
|
-
Although this is a **WORK IN PROGRESS**, we're
|
8
|
+
Although this is a **WORK IN PROGRESS**, we're getting great results on **production** at [Resultados Digitais](http://resultadosdigitais.com.br/).
|
9
9
|
|
10
10
|
This Gem relies heavily on both [Sidekiq](https://github.com/mperham/sidekiq) and Celluloid Gems, you are encouraged to use it anywhere with Ruby (a http interface is on the Roadmap ).
|
11
11
|
|
12
12
|
This Gem provides a worker ready for deploy cooked with [MongoDB](http://www.mongodb.org/) + [Mongoid](https://github.com/mongoid/mongoid) + [Sidekiq](https://github.com/mperham/sidekiq) + [Rubinius](http://rubini.us/) (feel free to use on MRI and jRuby as well).
|
13
13
|
|
14
|
-
For production deployment, you should take a
|
14
|
+
For production deployment, you should take a look at both [MailCannon Outpost](https://github.com/lucasmartins/mailcannon-outpost) and [MailCannon Monitor](https://github.com/lucasmartins/mailcannon-monitor) projects.
|
15
15
|
|
16
16
|
Install
|
17
17
|
=======
|
@@ -68,7 +68,7 @@ envelope = MailCannon::Envelope.create(
|
|
68
68
|
subject: 'Test',
|
69
69
|
mail: MailCannon::Mail.new(text: 'you will see this when no HTML reader is available', html: 'this should be an HTML'))
|
70
70
|
envelope_bag.push envelope
|
71
|
-
envelope_bag.post! # this will sent using the 'hot-account'.
|
71
|
+
envelope_bag.post! # this will be sent using the 'hot-account'.
|
72
72
|
```
|
73
73
|
|
74
74
|
### Configuration file
|
@@ -87,17 +87,52 @@ Edit the file to meet your environemnt needs.
|
|
87
87
|
|
88
88
|
Check the [specs](https://github.com/lucasmartins/mailcannon/tree/master/spec) to see the testing example, it will surely make it clearer.
|
89
89
|
|
90
|
+
### Statistics & MapReduce
|
91
|
+
|
92
|
+
MailCannon provides statistics calculation/reduce for the events related to an `Envelope`, like `open`,`click`,`spam`, etc. Assuming you have your Outpost running properly (running reduce jobs), you can access the data through the `envelope.stats` method to get the following hash:
|
93
|
+
|
94
|
+
```ruby
|
95
|
+
{
|
96
|
+
"posted"=>{"count"=>0.0, "targets"=>[]},
|
97
|
+
"processed"=>{"count"=>0.0, "targets"=>[]},
|
98
|
+
"delivered"=>{"count"=>1.0, "targets"=>["1"]},
|
99
|
+
"open"=>{"count"=>1.0, "targets"=>["2"]},
|
100
|
+
"click"=>{"count"=>0.0, "targets"=>[]},
|
101
|
+
"deferred"=>{"count"=>0.0, "targets"=>[]},
|
102
|
+
"spam_report"=>{"count"=>0.0, "targets"=>[]},
|
103
|
+
"spam"=>{"count"=>0.0, "targets"=>[]},
|
104
|
+
"unsubscribe"=>{"count"=>0.0, "targets"=>[]},
|
105
|
+
"drop"=>{"count"=>0.0, "targets"=>[]},
|
106
|
+
"bounce"=>{"count"=>1.0, "targets"=>["3"]}
|
107
|
+
}
|
108
|
+
```
|
109
|
+
|
110
|
+
You can trigger the reduce operation directly with `envelope.reduce_statistics`.
|
111
|
+
|
112
|
+
**Targets** are your __glue_id__ to link this data inside your own application, we use it as the "Contact#id" so we can show witch `Contact` has received, read, or clicked the email.
|
113
|
+
|
114
|
+
Repeating events on the same target will increase the array: `"click"=>{"count"=>3.0, "targets"=>["3","3","3"]}`
|
115
|
+
|
116
|
+
|
90
117
|
Docs
|
91
118
|
====
|
92
119
|
You should check the [factories](https://github.com/lucasmartins/mailcannon/tree/master/spec/factories) to learn what you need to build your objects, and the [tests](https://github.com/lucasmartins/mailcannon/tree/master/spec/mailcannon) to learn how to use them. But hey, we have docs [right here](http://rdoc.info/github/lucasmartins/mailcannon/master/frames).
|
93
120
|
|
121
|
+
Roadmap
|
122
|
+
=======
|
123
|
+
|
124
|
+
- Statistics (Map&Reduce awesomeness);
|
125
|
+
- Memory optimization (focused on MailCannon Outpost);
|
126
|
+
- HTTP (webservice) interface - so you don't need to be coding Ruby to use it!;
|
127
|
+
- New service adapter (Mandrill?);
|
128
|
+
|
94
129
|
Contribute
|
95
130
|
==========
|
96
131
|
|
97
132
|
Just fork [MailCannon](https://github.com/lucasmartins/mailcannon), add your feature+spec, and make a pull request. Do not mess up with the version file though.
|
98
133
|
|
99
134
|
**NOTICE**: The project is at embrionary stage, breaking changes will apply.
|
100
|
-
|
135
|
+
|
101
136
|
Support
|
102
137
|
=======
|
103
138
|
|
data/env.sample.sh
CHANGED
@@ -1,8 +1,9 @@
|
|
1
1
|
export REDIS_URL='redis://localhost:6379'
|
2
2
|
export SENDGRID_USERNAME=''
|
3
3
|
export SENDGRID_PASSWORD=''
|
4
|
-
export AIRBRAKE_TOKEN=''
|
5
|
-
export LIBRATO_SOURCE='staging'
|
6
|
-
export LIBRATO_USER=''
|
7
|
-
export LIBRATO_TOKEN=''
|
4
|
+
#export AIRBRAKE_TOKEN=''
|
5
|
+
#export LIBRATO_SOURCE='staging'
|
6
|
+
#export LIBRATO_USER=''
|
7
|
+
#export LIBRATO_TOKEN=''
|
8
8
|
export MONGODB_URL=mongodb://localhost:27017:mailcannon_development
|
9
|
+
export MONGODB_PORT=27017
|
@@ -29,7 +29,7 @@ module MailCannon::Adapter::SendgridWeb
|
|
29
29
|
rescue Exception => e
|
30
30
|
logger.error "Unable to read auth config from Envelope or Bag, using default auth options from ENV"
|
31
31
|
return default_auth
|
32
|
-
end
|
32
|
+
end
|
33
33
|
end
|
34
34
|
end
|
35
35
|
|
@@ -55,27 +55,63 @@ module MailCannon::Adapter::SendgridWeb
|
|
55
55
|
self.save!
|
56
56
|
end
|
57
57
|
|
58
|
-
def
|
59
|
-
name_placeholder = MailCannon.config['default_name_placeholder'] || '%name%'
|
58
|
+
def build_to_subs(placeholder, to_key)
|
60
59
|
selected_hash_array = []
|
61
|
-
self.to.map {|h| selected_hash_array.push h[
|
62
|
-
{'sub'=>{"#{
|
60
|
+
self.to.map {|h| selected_hash_array.push h[to_key]||h[to_key.to_sym]||''}
|
61
|
+
{'sub'=>{"#{placeholder}"=>selected_hash_array}}
|
62
|
+
end
|
63
|
+
|
64
|
+
def build_name_subs
|
65
|
+
placeholder = MailCannon.config['default_name_placeholder'] || '%name%'
|
66
|
+
build_to_subs(placeholder, 'name')
|
67
|
+
end
|
68
|
+
|
69
|
+
def build_email_subs
|
70
|
+
placeholder = MailCannon.config['default_email_placeholder'] || '%email%'
|
71
|
+
build_to_subs(placeholder, 'email')
|
72
|
+
end
|
73
|
+
|
74
|
+
|
75
|
+
def build_unique_args
|
76
|
+
unique_args = {}
|
77
|
+
if MailCannon.config['add_envelope_id_to_unique_args']
|
78
|
+
unique_args.merge!({'envelope_id'=>self.id})
|
79
|
+
end
|
80
|
+
if MailCannon.config['add_envelope_bag_id_to_unique_args'] && self.envelope_bag
|
81
|
+
unique_args.merge!({'envelope_bag_id'=>self.envelope_bag.id})
|
82
|
+
end
|
83
|
+
unique_args
|
63
84
|
end
|
64
85
|
|
65
86
|
def build_xsmtpapi(recipients,subs)
|
66
87
|
xsmtpapi = self.xsmtpapi || {}
|
67
|
-
to = []
|
68
88
|
recipients.symbolize_keys!
|
69
|
-
recipients[:to]
|
70
|
-
h.symbolize_keys!
|
71
|
-
to.push h[:email]
|
72
|
-
end
|
89
|
+
to = extract_values(recipients[:to],:email)
|
73
90
|
xsmtpapi.merge!({'to' => to}) if to
|
74
|
-
xsmtpapi = xsmtpapi
|
75
|
-
xsmtpapi = xsmtpapi
|
91
|
+
xsmtpapi = merge_subs_hash(xsmtpapi,subs)
|
92
|
+
xsmtpapi = merge_subs_hash(xsmtpapi,build_name_subs)
|
93
|
+
xsmtpapi = merge_subs_hash(xsmtpapi,build_email_subs)
|
94
|
+
xsmtpapi.merge!({'unique_args' => build_unique_args })
|
76
95
|
return xsmtpapi
|
77
96
|
end
|
78
97
|
|
98
|
+
def extract_values(values,key)
|
99
|
+
extract=[]
|
100
|
+
values.each do |h|
|
101
|
+
h.symbolize_keys!
|
102
|
+
extract.push h[key]
|
103
|
+
end
|
104
|
+
extract
|
105
|
+
end
|
106
|
+
|
107
|
+
def merge_subs_hash(xsmtpapi,subs)
|
108
|
+
if subs!=nil && subs.is_a?(Hash)
|
109
|
+
xsmtpapi.deep_merge(subs)
|
110
|
+
else
|
111
|
+
xsmtpapi
|
112
|
+
end
|
113
|
+
end
|
114
|
+
|
79
115
|
def validate_xsmtpapi!
|
80
116
|
return true
|
81
117
|
if self.to.size>1
|
data/lib/mailcannon/envelope.rb
CHANGED
@@ -4,10 +4,12 @@ class MailCannon::Envelope
|
|
4
4
|
include Mongoid::Timestamps
|
5
5
|
include MailCannon::Adapter::SendgridWeb
|
6
6
|
|
7
|
-
embeds_one :mail
|
8
|
-
embeds_many :stamps
|
9
7
|
belongs_to :envelope_bag
|
10
8
|
|
9
|
+
embeds_one :mail
|
10
|
+
embeds_many :stamps
|
11
|
+
has_many :sendgrid_events
|
12
|
+
|
11
13
|
field :from, type: String
|
12
14
|
field :from_name, type: String
|
13
15
|
field :to, type: Array # of hashes. [{email: '', name: ''},...]
|
@@ -19,7 +21,6 @@ class MailCannon::Envelope
|
|
19
21
|
field :xsmtpapi, type: Hash # this will mostly be used by MailCannon itself. http://sendgrid.com/docs/API_Reference/SMTP_API/index.html
|
20
22
|
field :auth, type: Hash # {user: 'foo', password: 'bar'}, some Adapters might need an token:secret pair, which you can translete into user:password pair.
|
21
23
|
field :jid, type: String
|
22
|
-
|
23
24
|
|
24
25
|
validates :from, :to, :subject, presence: true
|
25
26
|
validates_associated :mail
|
@@ -65,6 +66,14 @@ class MailCannon::Envelope
|
|
65
66
|
false
|
66
67
|
end
|
67
68
|
end
|
69
|
+
|
70
|
+
def processed?
|
71
|
+
if self.stamps.where(code: 1).count > 0
|
72
|
+
true
|
73
|
+
else
|
74
|
+
false
|
75
|
+
end
|
76
|
+
end
|
68
77
|
|
69
78
|
private
|
70
79
|
def schedule_send_job
|
@@ -2,11 +2,20 @@
|
|
2
2
|
class MailCannon::EnvelopeBag
|
3
3
|
include Mongoid::Document
|
4
4
|
include Mongoid::Timestamps
|
5
|
+
include MailCannon::EnvelopeBagMapReduce
|
5
6
|
|
6
7
|
has_many :envelopes, autosave: true
|
7
8
|
field :integration_code, type: String # Used to link your own app models to the Bag.
|
8
9
|
field :auth, type: Hash # {user: 'foo', password: 'bar'}, some Adapters might need an token:secret pair, which you can translete into user:password pair. This config will be overriden by the Envelope.auth if present.
|
9
10
|
|
11
|
+
def stats
|
12
|
+
begin
|
13
|
+
MailCannon::EnvelopeBagStatistic.find(self.id).value
|
14
|
+
rescue Mongoid::Errors::DocumentNotFound => e
|
15
|
+
raise "You haven't run envelope.reduce_statistics yet, no data available!"
|
16
|
+
end
|
17
|
+
end
|
18
|
+
|
10
19
|
def push(envelope)
|
11
20
|
self.envelopes.push envelope
|
12
21
|
end
|
@@ -0,0 +1,106 @@
|
|
1
|
+
module MailCannon::EnvelopeBagMapReduce
|
2
|
+
module ClassMethods
|
3
|
+
|
4
|
+
def find_gem_root_path
|
5
|
+
path = ""
|
6
|
+
begin
|
7
|
+
spec = Gem::Specification.find_by_name("mailcannon")
|
8
|
+
path = spec.gem_dir
|
9
|
+
rescue LoadError => e
|
10
|
+
raise unless e.message =~ /mailcannon/
|
11
|
+
puts 'mailcannon gem path not found'
|
12
|
+
end
|
13
|
+
path
|
14
|
+
end
|
15
|
+
|
16
|
+
def reduce_statistics_for_envelope_bag(id)
|
17
|
+
events = change_events_status_for_envelope_bag(id, nil, :lock)
|
18
|
+
result = events.map_reduce(self.js_map, self.js_reduce).out(merge: "mail_cannon_envelope_bag_statistics")
|
19
|
+
set_events_to(events,:processed)
|
20
|
+
{raw: result.raw, count: events.count}
|
21
|
+
end
|
22
|
+
|
23
|
+
def statistics_for_envelope(id)
|
24
|
+
self.stats
|
25
|
+
end
|
26
|
+
|
27
|
+
def js_map
|
28
|
+
root_path = find_gem_root_path
|
29
|
+
root_path += "/" unless root_path.empty?
|
30
|
+
@js_map ||= File.read("#{root_path}lib/mailcannon/reduces/envelope_bag_map.js")
|
31
|
+
#TODO cache this
|
32
|
+
@js_map
|
33
|
+
end
|
34
|
+
|
35
|
+
def js_reduce
|
36
|
+
root_path = find_gem_root_path
|
37
|
+
root_path += "/" unless root_path.empty?
|
38
|
+
@js_reduce ||= File.read("#{root_path}lib/mailcannon/reduces/envelope_bag_reduce.js")
|
39
|
+
#TODO cache this
|
40
|
+
@js_reduce
|
41
|
+
end
|
42
|
+
|
43
|
+
# [from|to]sym = :new, :lock, :processed
|
44
|
+
def change_events_status_for_envelope_bag(id, from_sym, to_sym)
|
45
|
+
from_status = processed_status_for(from_sym)
|
46
|
+
to_status = processed_status_for(to_sym)
|
47
|
+
if from_sym
|
48
|
+
query = MailCannon::SendgridEvent.where(envelope_bag_id: id, processed: from_status)
|
49
|
+
else
|
50
|
+
query = MailCannon::SendgridEvent.where(envelope_bag_id: id)
|
51
|
+
end
|
52
|
+
if query.kind_of?(Mongoid::Criteria)
|
53
|
+
query.update_all(processed: to_status)
|
54
|
+
else
|
55
|
+
query.processed=to_status
|
56
|
+
query.save
|
57
|
+
end
|
58
|
+
query
|
59
|
+
end
|
60
|
+
|
61
|
+
#private
|
62
|
+
def set_events_to(events,status)
|
63
|
+
status = processed_status_for(status)
|
64
|
+
if events.kind_of?(Mongoid::Criteria)
|
65
|
+
events.update_all(processed: status)
|
66
|
+
else
|
67
|
+
events.processed=status
|
68
|
+
events.save
|
69
|
+
end
|
70
|
+
end
|
71
|
+
|
72
|
+
def processed_status_for(input)
|
73
|
+
status_map = {
|
74
|
+
lock: false,
|
75
|
+
processed: true,
|
76
|
+
new: nil
|
77
|
+
}
|
78
|
+
if [false,true,nil].include?(input)
|
79
|
+
status_map = status_map.invert
|
80
|
+
raise "Unexpected input(#{input})" unless status_map.has_key?(input)
|
81
|
+
end
|
82
|
+
status_map[input]
|
83
|
+
end
|
84
|
+
|
85
|
+
end
|
86
|
+
|
87
|
+
module InstanceMethods
|
88
|
+
def reduce_statistics
|
89
|
+
self.class.reduce_statistics_for_envelope_bag(self.id)
|
90
|
+
end
|
91
|
+
|
92
|
+
def statistics
|
93
|
+
self.class.statistics_for_envelope(self.id)
|
94
|
+
end
|
95
|
+
|
96
|
+
def change_events_status(from_sym, to_sym)
|
97
|
+
self.set_processed_status_for_envelope(self.id, from_sym, to_sym)
|
98
|
+
end
|
99
|
+
end
|
100
|
+
|
101
|
+
def self.included(receiver)
|
102
|
+
receiver.extend ClassMethods
|
103
|
+
receiver.send :include, InstanceMethods
|
104
|
+
end
|
105
|
+
end
|
106
|
+
|
@@ -0,0 +1,116 @@
|
|
1
|
+
function (key, values) {
|
2
|
+
var result = {
|
3
|
+
'posted': {
|
4
|
+
'count': 0,
|
5
|
+
'targets': []
|
6
|
+
},
|
7
|
+
'processed': {
|
8
|
+
'count': 0,
|
9
|
+
'targets': []
|
10
|
+
},
|
11
|
+
'delivered': {
|
12
|
+
'count': 0,
|
13
|
+
'targets': []
|
14
|
+
},
|
15
|
+
'open': {
|
16
|
+
'count': 0,
|
17
|
+
'targets': []
|
18
|
+
},
|
19
|
+
'click': {
|
20
|
+
'count': 0,
|
21
|
+
'targets': []
|
22
|
+
},
|
23
|
+
'deferred': {
|
24
|
+
'count': 0,
|
25
|
+
'targets': []
|
26
|
+
},
|
27
|
+
'spam_report': {
|
28
|
+
'count': 0,
|
29
|
+
'targets': []
|
30
|
+
},
|
31
|
+
'spam': {
|
32
|
+
'count': 0,
|
33
|
+
'targets': []
|
34
|
+
},
|
35
|
+
'unsubscribe': {
|
36
|
+
'count': 0,
|
37
|
+
'targets': []
|
38
|
+
},
|
39
|
+
'drop': {
|
40
|
+
'count': 0,
|
41
|
+
'targets': []
|
42
|
+
},
|
43
|
+
'hard_bounce': {
|
44
|
+
'count': 0,
|
45
|
+
'targets': []
|
46
|
+
},
|
47
|
+
'soft_bounce': {
|
48
|
+
'count': 0,
|
49
|
+
'targets': []
|
50
|
+
},
|
51
|
+
'unknown': {
|
52
|
+
'count': 0,
|
53
|
+
'targets': []
|
54
|
+
}
|
55
|
+
};
|
56
|
+
|
57
|
+
values.forEach(function(value) {
|
58
|
+
switch (value['event']) {
|
59
|
+
case 'posted':
|
60
|
+
result['posted']['count']++;
|
61
|
+
result['posted']['targets'].push(value['target_id']);
|
62
|
+
break;
|
63
|
+
case 'processed':
|
64
|
+
result['processed']['count']++;
|
65
|
+
result['processed']['targets'].push(value['target_id']);
|
66
|
+
break;
|
67
|
+
case 'delivered':
|
68
|
+
result['delivered']['count']++;
|
69
|
+
result['delivered']['targets'].push(value['target_id']);
|
70
|
+
break;
|
71
|
+
case 'open':
|
72
|
+
result['open']['count']++;
|
73
|
+
result['open']['targets'].push(value['target_id']);
|
74
|
+
break;
|
75
|
+
case 'click':
|
76
|
+
result['click']['count']++;
|
77
|
+
result['click']['targets'].push(value['target_id']);
|
78
|
+
break;
|
79
|
+
case 'deferred':
|
80
|
+
result['deferred']['count']++;
|
81
|
+
result['deferred']['targets'].push(value['target_id']);
|
82
|
+
break;
|
83
|
+
case 'spam_report':
|
84
|
+
result['spam_report']['count']++;
|
85
|
+
result['spam_report']['targets'].push(value['target_id']);
|
86
|
+
break;
|
87
|
+
case 'spam':
|
88
|
+
result['spam']['count']++;
|
89
|
+
result['spam']['targets'].push(value['target_id']);
|
90
|
+
break;
|
91
|
+
case 'unsubscribe':
|
92
|
+
result['unsubscribe']['count']++;
|
93
|
+
result['unsubscribe']['targets'].push(value['target_id']);
|
94
|
+
break;
|
95
|
+
case 'drop':
|
96
|
+
result['drop']['count']++;
|
97
|
+
result['drop']['targets'].push(value['target_id']);
|
98
|
+
break;
|
99
|
+
case 'bounce':
|
100
|
+
if(value['type'] == "bounce"){
|
101
|
+
result['hard_bounce']['count']++;
|
102
|
+
result['hard_bounce']['targets'].push(value['target_id']);
|
103
|
+
}
|
104
|
+
else {
|
105
|
+
result['soft_bounce']['count']++;
|
106
|
+
result['soft_bounce']['targets'].push(value['target_id']);
|
107
|
+
}
|
108
|
+
break;
|
109
|
+
default:
|
110
|
+
result['unknown']['count']++;
|
111
|
+
result['unknown']['targets'].push(value['target_id']);
|
112
|
+
break;
|
113
|
+
}
|
114
|
+
});
|
115
|
+
return result;
|
116
|
+
}
|
@@ -0,0 +1,18 @@
|
|
1
|
+
class MailCannon::SendgridEvent
|
2
|
+
include Mongoid::Document
|
3
|
+
field :envelope_id, type: String
|
4
|
+
field :envelope_bag_id, type: String
|
5
|
+
field :email, type: String
|
6
|
+
field :timestamp, type: String
|
7
|
+
field :unique_arg, type: String
|
8
|
+
field :event, type: String
|
9
|
+
field :type, type: String
|
10
|
+
field :processed, type: Boolean, default: nil
|
11
|
+
|
12
|
+
belongs_to :envelope
|
13
|
+
belongs_to :envelope_bag
|
14
|
+
|
15
|
+
def self.insert_bulk(tha_huge_string)
|
16
|
+
MailCannon::SendgridEvent.collection.insert(tha_huge_string)
|
17
|
+
end
|
18
|
+
end
|
data/lib/mailcannon/version.rb
CHANGED
@@ -4,23 +4,10 @@ class MailCannon::Barrel
|
|
4
4
|
|
5
5
|
def perform(envelope_id)
|
6
6
|
envelope_id = envelope_id['$oid'] if envelope_id['$oid']
|
7
|
-
|
8
|
-
shoot_with_librato!(envelope_id)
|
9
|
-
else
|
10
|
-
shoot!(envelope_id)
|
11
|
-
end
|
7
|
+
shoot!(envelope_id)
|
12
8
|
end
|
13
9
|
|
14
10
|
private
|
15
|
-
def shoot_with_librato!(envelope_id)
|
16
|
-
MailCannon::Librato.authenticate
|
17
|
-
aggregator = Librato::Metrics::Aggregator.new
|
18
|
-
aggregator.time 'mailcannon.shoot' do
|
19
|
-
shoot!(envelope_id)
|
20
|
-
end
|
21
|
-
aggregator.submit
|
22
|
-
end
|
23
|
-
|
24
11
|
def shoot!(envelope_id)
|
25
12
|
logger.info "sending MailCannon::Envelope.find('#{envelope_id}')"
|
26
13
|
begin
|
@@ -34,8 +21,7 @@ class MailCannon::Barrel
|
|
34
21
|
rescue Mongoid::Errors::DocumentNotFound
|
35
22
|
logger.error "unable to find the document MailCannon::Envelope.find('#{envelope_id}')"
|
36
23
|
rescue Exception => e
|
37
|
-
logger.error "unable to send MailCannon::Envelope.find('#{envelope_id}')
|
38
|
-
Airbrake.notify(e, parameters: {'envelope_id'=>envelope_id}, cgi_data: ENV.to_hash) if MailCannon::Airbrake.available?
|
24
|
+
logger.error "unable to send MailCannon::Envelope.find('#{envelope_id}') #{e.backtrace}"
|
39
25
|
end
|
40
26
|
end
|
41
27
|
end
|
data/lib/mailcannon.rb
CHANGED
@@ -7,8 +7,6 @@ require 'sidekiq'
|
|
7
7
|
require 'sendgrid_webapi'
|
8
8
|
require 'yajl-ruby' if RUBY_PLATFORM=='ruby'
|
9
9
|
require 'jruby-openssl' if RUBY_PLATFORM=='jruby'
|
10
|
-
require 'librato/metrics'
|
11
|
-
require 'airbrake'
|
12
10
|
|
13
11
|
Encoding.default_internal = "utf-8"
|
14
12
|
Encoding.default_external = "utf-8"
|
@@ -16,14 +14,16 @@ Encoding.default_external = "utf-8"
|
|
16
14
|
module MailCannon
|
17
15
|
require_relative 'mailcannon/adapter'
|
18
16
|
require_relative 'mailcannon/adapters/sendgrid_web'
|
17
|
+
require_relative 'mailcannon/envelope_bag_map_reduce'
|
18
|
+
require_relative 'mailcannon/envelope_bag_statistic'
|
19
19
|
require_relative 'mailcannon/envelope_bag'
|
20
|
-
require_relative 'mailcannon/envelope'
|
20
|
+
require_relative 'mailcannon/envelope'
|
21
21
|
require_relative 'mailcannon/mail'
|
22
22
|
require_relative 'mailcannon/stamp'
|
23
23
|
require_relative 'mailcannon/event'
|
24
|
+
require_relative 'mailcannon/sendgrid_event'
|
24
25
|
require_relative 'mailcannon/workers/barrel'
|
25
|
-
require_relative 'mailcannon/
|
26
|
-
require_relative 'mailcannon/airbrake'
|
26
|
+
require_relative 'mailcannon/workers/envelope_bag_reduce_job'
|
27
27
|
require_relative 'mailcannon/version'
|
28
28
|
|
29
29
|
# To be used with caution
|
data/mailcannon.gemspec
CHANGED
@@ -36,8 +36,6 @@ Gem::Specification.new do |s|
|
|
36
36
|
s.add_dependency 'sidekiq'
|
37
37
|
s.add_dependency 'sendgrid_webapi'
|
38
38
|
s.add_dependency 'json-schema'
|
39
|
-
s.add_dependency 'librato-metrics'
|
40
|
-
s.add_dependency 'airbrake'
|
41
39
|
|
42
40
|
s.add_development_dependency "vcr"
|
43
41
|
s.add_development_dependency "rspec"
|
data/spec/integration/1k_spec.rb
CHANGED
@@ -1,10 +1,24 @@
|
|
1
1
|
require "spec_helper"
|
2
|
+
require 'benchmark'
|
3
|
+
|
2
4
|
|
3
5
|
describe "shoot 1k emails!" do
|
4
|
-
let(:
|
6
|
+
let!(:envelope_a) { build(:envelope_multi_1k) }
|
5
7
|
it "sends http request for Sendgrid web API" do
|
6
8
|
VCR.use_cassette('mailcannon_integration_1k') do
|
7
|
-
|
9
|
+
Sidekiq::Testing.inline! do
|
10
|
+
bm = Benchmark.measure do
|
11
|
+
envelope_a.post!
|
12
|
+
end
|
13
|
+
puts "1k test real time: #{bm.real}"
|
14
|
+
expect(envelope_a.reload.processed?).to be_true
|
15
|
+
|
16
|
+
# Travis has been showing unstable performance, not feasible to include performance tests.
|
17
|
+
# The performance varies from machine to machine, specially when using dedicated servers for each service.
|
18
|
+
if ENV['PERFORMANCE_TEST']
|
19
|
+
expect(bm.real<0.2).to be_true
|
20
|
+
end
|
21
|
+
end
|
8
22
|
end
|
9
23
|
end
|
10
24
|
end
|