aiwilliams-mlist 0.1.4 → 0.1.5

Sign up to get free protection for your applications and to get access to all the features.
data/CHANGELOG CHANGED
@@ -1,19 +1,34 @@
1
+ *0.1.5 [Solid Basics] (January 17, 2009)
2
+
3
+ * No longer storing the list label in the message subject field, thereby supporting cleaner viewing of threads [aiwilliams]
4
+ * Improved handling of incoming email subjects by leaving labels alone that aren't obviously the label of the list, thereby allowing for subjects like "[Ann] My New Thing" [aiwilliams]
5
+ * Notifying subscribers when the list indicates that they are currently blocked from posting messages to a list they are subscribed to [aiwilliams]
6
+ * Added MList::Manager module to better define the interface of list managers. This needs to be included into list manager implementations. [aiwilliams]
7
+ * MList::List implementors may now answer the footer content to be appended to the bottom of the text/plain part of messages. [aiwilliams]
8
+ * List footers are stripped from text/plain part of messages before being delivered. [aiwilliams]
9
+ * Observers of MList models which are defined in client applications now work without special instruction. [aiwilliams]
10
+ * A first pass implementation for converting html to text using Hpricot. [aiwilliams]
11
+ * Better thread tree, supporting message navigation within a thread through a linked list kind of approach. [aiwilliams]
12
+ * Better parent message associating using in-reply-to, then references, then subject. [aiwilliams]
13
+ * MList.version is hash of {:major => 0, :minor => 0, :patch => 0}, with a to_s of 'MList 0.0.0'. [aiwilliams]
14
+ * Fixed bug where original email source content was last in TMail::Mail#to_s usage. [aiwilliams]
15
+
1
16
  *0.1.4 [] (January 7, 2009)
2
17
 
3
- * Fixed bug where default email server was not allowing for settings
4
- * Made subject for reply place 're:' in front of the list label
5
- * Added really simple tree support. Really simple.
18
+ * Fixed bug where default email server was not allowing for settings [aiwilliams]
19
+ * Made subject for reply place 're:' in front of the list label [aiwilliams]
20
+ * Added really simple tree support. Really simple. [aiwilliams]
6
21
 
7
22
  *0.1.3 [] (January 7, 2009)
8
23
 
9
- * Generating message id as UUID
10
- * Allowing setting of domain for message id
11
- * Fixed bug in storing message id
24
+ * Generating message id as UUID [aiwilliams]
25
+ * Allowing setting of domain for message id [aiwilliams]
26
+ * Fixed bug in storing message id [aiwilliams]
12
27
 
13
28
  *0.1.2 [] (January 7, 2009)
14
29
 
15
- * Added references header when creating a reply post.
16
- * Including display_name in from address of EmailPost.
17
- * Improved extraction of text when it is nested inside parts.
30
+ * Added references header when creating a reply post. [aiwilliams]
31
+ * Including display_name in from address of EmailPost. [aiwilliams]
32
+ * Improved extraction of text when it is nested inside parts. [aiwilliams]
18
33
 
19
34
  *0.1.1 [First Working Release] (January 5, 2009)
data/README CHANGED
@@ -32,34 +32,49 @@ as they seem. Alas, I go boldly forward.
32
32
 
33
33
  ==== Thread Trees
34
34
 
35
- We need to implement this at the database level. For now, each MList::Message
36
- simply points to it's parent. I believe the implementation will ultimately be
37
- extracted from awesome_nested_set. That's a bit of work, so for now, the
38
- MList::Thread provides a couple of methods, roots and children, to support the
39
- simplest display of the tree.
35
+ A Thread answers a tree of messages where each message in the tree knows it's
36
+ parent, children, previous and next message, and whether it is the root or a leaf
37
+ in the tree. The messages are actually delegates, wrappers around the real
38
+ message instances from the thread.
39
+
40
+ This approach makes a few assumptions:
41
+
42
+ # You want the tree because you will be displaying it # Moving through a tree
43
+ message by message should 'feel' right; next is determined by walking the tree
44
+ depth first.
45
+
46
+ It may or may not prove useful someday to have this knowledge in the form of
47
+ something like awesome_nested_set.
40
48
 
41
49
  ==== Extracting 'from' IP address from emails is not implemented
42
50
 
43
51
  http://compnetworking.about.com/od/workingwithipaddresses/qt/ipaddressemail.htm
44
-
45
- ==== Observing MList ActiveRecord subsclasses (ie, MList::Message, MList::Thread)
46
52
 
47
- ActiveRecord observers are reloaded at each request in development mode. They
48
- will be registered with the MList models each time. Since the MList models are
49
- required once at initialization, there will always only be one instance of the
50
- model class, and therefore, many instances of the observer class (all but the
51
- most recent invalid, since they were undefined) registered with it.
53
+ ==== Deleting messages
54
+
55
+ Mail list servers don't typically deal with this, do they? When an email is sent,
56
+ it's sent! If you delete one, how can a reply to it be accurately injected into a
57
+ partially broken tree?
58
+
59
+ Since MList is designed to integrate with applications that want to distribute
60
+ messages, those applications may want to delete messages. I feel, at the time of
61
+ writing this, that the feature of deleting is NOT a problem that MList should
62
+ attempt to address in great detail. The models are ActiveRecord descendants, so
63
+ they can be deleted with #destroy, et. al. MList will not prevent that from
64
+ happening. This has implications, of course.
52
65
 
53
- There are a number of ways to solve this, the best being the one that makes
54
- things 'just work' such that I can delete this part of the document. For now, do
55
- the following in environment.rb:
66
+ # MList::Thread#tree will likely break if messages are gone.
56
67
 
57
- * remove the observer from the Rails' config.active_record.observers list
58
- * after the Rails::Initializer.run block, "require 'myobserver'"
59
- * after that require line, "Myobserver.instance"
68
+ In my own usage, I have opted for adding a column to my mlist_messages table,
69
+ deleted_at. The application will make decisions based on whether that column has
70
+ a value or not. I would suggest you implement something similar, always favoring
71
+ leaving the records in place (paranoid delete). Since MList::MailList#messages
72
+ (and other such has_many associations) don't know about this column, they will
73
+ always answer collections which still include those messages.
60
74
 
61
- This will load the observer once, thereby only registering it once with the MList
62
- class you are observing.
75
+ When an MList::MailList is destroyed, all it's MList::Messages and MList::Threads
76
+ will be deleted (:dependent => :delete_all). If no other MList::Messages are
77
+ referencing the MList::Email records, they will also be deleted.
63
78
 
64
79
  == SYNOPSIS:
65
80
 
@@ -73,6 +88,7 @@ You love Ruby. You want MList.
73
88
 
74
89
  You'll need some gems.
75
90
 
91
+ * hpricot
76
92
  * uuid (macaddr also)
77
93
  * tmail
78
94
  * activesupport
data/Rakefile CHANGED
@@ -17,7 +17,7 @@ begin
17
17
  s.name = 'mlist'
18
18
  s.summary = 'A Ruby mailing list library designed to be integrated into other applications.'
19
19
  s.email = 'adam@thewilliams.ws'
20
- s.files = FileList["[A-Z]*", "{lib}/**/*"].exclude("tmp,**/tmp")
20
+ s.files = FileList["[A-Z]*", "{lib,rails}/**/*"].exclude("tmp")
21
21
  s.homepage = "http://github.com/aiwilliams/mlist"
22
22
  s.description = s.summary
23
23
  s.authors = ['Adam Williams']
data/TODO ADDED
@@ -0,0 +1,36 @@
1
+ #html_to_text
2
+
3
+ ADDRESS - Address
4
+ BLOCKQUOTE - Block quotation
5
+
6
+ DIV - Generic block-level container
7
+ DL - Definition list
8
+ FIELDSET - Form control group
9
+ FORM - Interactive form
10
+ H1 - Level-one heading
11
+ H2 - Level-two heading
12
+ H3 - Level-three heading
13
+ H4 - Level-four heading
14
+ H5 - Level-five heading
15
+ H6 - Level-six heading
16
+ HR - Horizontal rule
17
+ OL - Ordered list
18
+ P - Paragraph
19
+ PRE - Preformatted text
20
+
21
+ TABLE - Table
22
+ output TRs as lines, csv the TDs
23
+
24
+ UL - Unordered list
25
+
26
+ DD - Definition description
27
+ DT - Definition term
28
+ LI - List item
29
+
30
+
31
+ a (Links) - Link to Somewhere[1] ---- [1] http://
32
+ b (Bold) - *bold*
33
+ i (Italic) - _italic_
34
+ strong - *strong*
35
+ em (emphasis) - _emphasis_
36
+ u (underline) - probably do nothing, maybe something like em
@@ -1,4 +1,4 @@
1
1
  ---
2
2
  :minor: 1
3
- :patch: 4
3
+ :patch: 5
4
4
  :major: 0
@@ -1,5 +1,8 @@
1
+ require 'delegate'
2
+
1
3
  require 'uuid'
2
4
  require 'tmail'
5
+ require 'hpricot'
3
6
  require 'activesupport'
4
7
  require 'activerecord'
5
8
 
@@ -14,7 +17,17 @@ require 'mlist/email_subscriber'
14
17
  require 'mlist/server'
15
18
  require 'mlist/thread'
16
19
 
20
+ require 'mlist/manager'
21
+
17
22
  module MList
23
+ mattr_reader :version
24
+ @@version = YAML.load_file(File.join(File.dirname(__FILE__), '..', "VERSION.yml"))
25
+ class << @@version
26
+ def to_s
27
+ @to_s ||= [self[:major], self[:minor], self[:patch]].join('.')
28
+ end
29
+ end
30
+
18
31
  class DoubleDeliveryError < StandardError
19
32
  def initialize(message)
20
33
  super("A message should never be delivered more than once. An attempt was made to deliver this message:\n#{message.inspect}")
@@ -28,33 +28,9 @@ module MList
28
28
  tmail.header_string('to') =~ /mlist-/
29
29
  end
30
30
 
31
- # Extracts the parent message identifier from the source, using
32
- # in-reply-to first, then references. Brackets around the identifier are
33
- # removed.
34
- #
35
- # If you provide an MList::MailList, it will be searched for a message
36
- # having the same subject as a last resort.
37
- #
38
- def parent_identifier(mail_list = nil)
39
- identifier = tmail.header_string('in-reply-to') || begin
40
- references = tmail['references']
41
- references.ids.first if references
42
- end
43
-
44
- if identifier
45
- remove_brackets(identifier)
46
- elsif mail_list && subject =~ /(^|[^\w])re:/i
47
- parent_message = mail_list.messages.find(:first,
48
- :select => 'identifier',
49
- :conditions => ['mlist_messages.subject = ?', remove_regard(subject)],
50
- :order => 'created_at asc')
51
- parent_message.identifier if parent_message
52
- end
53
- end
54
-
55
31
  def tmail=(tmail)
56
32
  @tmail = tmail
57
- write_attribute(:source, tmail.to_s)
33
+ write_attribute(:source, tmail.port.read_all)
58
34
  end
59
35
 
60
36
  def tmail
@@ -65,11 +41,15 @@ module MList
65
41
  # methods, excluding those overridden by this Class and the [] method (an
66
42
  # ActiveRecord method).
67
43
  def method_missing(symbol, *args, &block) # :nodoc:
68
- if symbol.to_s !~ /=\Z/ && symbol != :[] && symbol != :source && tmail.respond_to?(symbol)
44
+ if symbol.to_s !~ /=\Z/ && symbol != :[] && symbol != :source && tmail.respond_to?(symbol)
69
45
  tmail.__send__(symbol, *args, &block)
70
46
  else
71
47
  super
72
48
  end
73
49
  end
50
+
51
+ def respond_to?(method)
52
+ super || (method.to_s !~ /=\Z/ && tmail.respond_to?(method))
53
+ end
74
54
  end
75
55
  end
@@ -57,6 +57,10 @@ module MList
57
57
  @attributes['subject'] || (reply_to_message ? "Re: #{reply_to_message.subject}" : nil)
58
58
  end
59
59
 
60
+ def to_s
61
+ to_tmail.to_s
62
+ end
63
+
60
64
  def to_tmail
61
65
  raise ActiveRecord::RecordInvalid.new(self) unless valid?
62
66
 
@@ -5,6 +5,9 @@ module MList
5
5
  # in processing email coming to a list - that is, whatever you include this
6
6
  # into may re-define behavior appropriately.
7
7
  #
8
+ # Your 'subscriber' instances MUST respond to :email_address. They may
9
+ # optionally respond to :display_name.
10
+ #
8
11
  module List
9
12
 
10
13
  # Answers whether this list is active or not. All lists are active all the
@@ -14,53 +17,116 @@ module MList
14
17
  true
15
18
  end
16
19
 
17
- def host
18
- address.match(/@(.*)\Z/)[1]
20
+ # Answers whether the subscriber is blocked from posting or not. This will
21
+ # not be asked when the list is not active (answers _active?_ as false).
22
+ #
23
+ def blocked?(subscriber)
24
+ false
25
+ end
26
+
27
+ # Answers the footer content for this list. Default implementation is very
28
+ # simple right now. Expect improvements in the future.
29
+ #
30
+ def footer_content(message)
31
+ %Q{The "#{label}" mailing list\nTo post messages, send email to #{post_url}}
19
32
  end
20
33
 
34
+ # Answer a suitable label for the list, which will be used in various
35
+ # parts of content that is delivered to subscribers, etc.
36
+ #
37
+ def label
38
+ raise 'answer the list label'
39
+ end
40
+
41
+ # Answers the headers that are to be included in the emails delivered for
42
+ # this list. Any entries that have a nil value will not be included in the
43
+ # delivered email.
44
+ #
21
45
  def list_headers
22
46
  {
23
47
  'list-id' => list_id,
24
- 'list-archive' => (archive_url rescue nil),
25
- 'list-subscribe' => (subscribe_url rescue nil),
26
- 'list-unsubscribe' => (unsubscribe_url rescue nil),
27
- 'list-owner' => (owner_url rescue nil),
28
- 'list-help' => (help_url rescue nil),
48
+ 'list-archive' => archive_url,
49
+ 'list-subscribe' => subscribe_url,
50
+ 'list-unsubscribe' => unsubscribe_url,
51
+ 'list-owner' => owner_url,
52
+ 'list-help' => help_url,
29
53
  'list-post' => post_url
30
54
  }
31
55
  end
32
56
 
57
+ # Answers a unique, never changing value for this list.
58
+ #
33
59
  def list_id
34
60
  raise 'answer a unique, never changing value'
35
61
  end
36
62
 
37
- def name
38
- address.match(/\A(.*?)@/)[1]
63
+ # The web address where an archive of this list may be found, nil if there
64
+ # is no archive.
65
+ #
66
+ def archive_url
67
+ nil
68
+ end
69
+
70
+ # The web address of the list help site, nil if this is not supported.
71
+ #
72
+ def help_url
73
+ nil
74
+ end
75
+
76
+ # The email address of the list owner, nil if this is not supported.
77
+ #
78
+ def owner_url
79
+ nil
39
80
  end
40
81
 
82
+ # The email address where posts should be sent. Defaults to the address of
83
+ # the list.
84
+ #
41
85
  def post_url
42
86
  address
43
87
  end
44
88
 
45
- # A list is responsible for answering the recipient subscribers.
89
+ # The web url where subscriptions to this list may be created, nil if this
90
+ # is not supported.
91
+ #
92
+ def subscribe_url
93
+ nil
94
+ end
46
95
 
96
+ # The web url where subscriptions to this list may be deleted, nil if this
97
+ # is not supported.
98
+ #
99
+ def unsubscribe_url
100
+ nil
101
+ end
102
+
103
+ # A list is responsible for answering the recipient subscribers.
104
+ #
47
105
  # The subscriber of the incoming message is provided if the list would
48
106
  # like to exclude it from the returned list. It is not assumed that it
49
107
  # will be included or excluded, thereby allowing the list to decide. This
50
108
  # default implementation does not include the sending subscriber in the
51
109
  # list of recipients.
52
110
  #
53
- # Your 'subscriber' instance MUST respond to :email_address. They may
54
- # optionally respond to :display_name.
55
- #
56
111
  def recipients(subscriber)
57
112
  subscribers.reject {|s| s.email_address == subscriber.email_address}
58
113
  end
59
114
 
115
+ # A list must answer the subscriber who's email address is that of the one
116
+ # provided. The default implementation will pick the first instance that
117
+ # answers subscriber.email_address == email_address. Your implementation
118
+ # should probably select just one record.
119
+ #
60
120
  def subscriber(email_address)
61
121
  subscribers.detect {|s| s.email_address == email_address}
62
122
  end
63
123
 
124
+ # A list must answer whether there is a subscriber who's email address is
125
+ # that of the one provided. This is checked before the subscriber is
126
+ # requested in order to allow for the lightest weight check possible; that
127
+ # is, your implementation could avoid loading the actual subscriber
128
+ # instance.
129
+ #
64
130
  def subscriber?(email_address)
65
131
  !subscriber(email_address).nil?
66
132
  end
@@ -69,6 +135,15 @@ module MList
69
135
  # certain events occur during the processing of email sent to a list.
70
136
  #
71
137
  module Callbacks
138
+
139
+ # Called when an email is a post to the list by a subscriber whom the
140
+ # list claims is blocked (answers true to _blocked?(subscriber)_). This
141
+ # will not be called if the list is inactive (answers false to
142
+ # _active?_);
143
+ #
144
+ def blocked_subscriber_post(email, subscriber)
145
+ end
146
+
72
147
  def bounce(email)
73
148
  end
74
149
 
@@ -18,8 +18,11 @@ module MList
18
18
  mail_list
19
19
  end
20
20
 
21
+ include MList::Util::EmailHelpers
22
+
21
23
  belongs_to :manager_list, :polymorphic => true
22
24
 
25
+ before_destroy :delete_unreferenced_email
23
26
  has_many :messages, :class_name => 'MList::Message', :dependent => :delete_all
24
27
  has_many :threads, :class_name => 'MList::Thread', :dependent => :delete_all
25
28
 
@@ -39,14 +42,13 @@ module MList
39
42
  :mail_list => self,
40
43
  :subscriber => email.subscriber,
41
44
  :recipients => list.recipients(email.subscriber),
42
- :email => MList::Email.new(:tmail => email.to_tmail)
45
+ :email => MList::Email.new(:source => email.to_s)
43
46
  ), :search_parent => false
44
47
  end
45
48
 
46
49
  # Processes the email received by the MList::Server.
47
50
  #
48
- def process_email(email)
49
- subscriber = list.subscriber(email.from_address)
51
+ def process_email(email, subscriber)
50
52
  recipients = list.recipients(subscriber)
51
53
  process_message messages.build(
52
54
  :mail_list => self,
@@ -56,6 +58,46 @@ module MList
56
58
  )
57
59
  end
58
60
 
61
+ # Answers the provided subject with superfluous 're:' and this list's
62
+ # labels removed.
63
+ #
64
+ # clean_subject('[List Label] Re: The new Chrome Browser from Google') => 'Re: The new Chrome Browser from Google'
65
+ # clean_subject('Re: [List Label] Re: The new Chrome Browser from Google') => 'Re: The new Chrome Browser from Google'
66
+ #
67
+ def clean_subject(string)
68
+ without_label = string.gsub(subject_prefix_regex, '')
69
+ if without_label =~ REGARD_RE
70
+ "Re: #{remove_regard(without_label)}"
71
+ else
72
+ without_label
73
+ end
74
+ end
75
+
76
+ def find_parent_message(email)
77
+ if in_reply_to = email.header_string('in-reply-to')
78
+ message = messages.find(:first,
79
+ :conditions => ['identifier = ?', remove_brackets(in_reply_to)])
80
+ return message if message
81
+ end
82
+
83
+ if email.references
84
+ reference_identifiers = email.references.collect {|rid| remove_brackets(rid)}
85
+ message = messages.find(:first,
86
+ :conditions => ['identifier in (?)', reference_identifiers],
87
+ :order => 'created_at desc')
88
+ return message if message
89
+ end
90
+
91
+ if email.subject =~ REGARD_RE
92
+ message = messages.find(:first,
93
+ :conditions => ['subject = ?', remove_regard(clean_subject(email.subject))],
94
+ :order => 'created_at asc')
95
+ return message if message
96
+ end
97
+ end
98
+
99
+ # The MList::List instance of the list manager.
100
+ #
59
101
  def list
60
102
  @list ||= manager_list
61
103
  end
@@ -75,17 +117,59 @@ module MList
75
117
  !message.recipients.blank?
76
118
  end
77
119
 
120
+ # Distinct footer start marker. It is important to realize that changing
121
+ # this could be problematic.
122
+ #
123
+ FOOTER_BLOCK_START = "-~----~~----~----~----~----~---~~-~----~------~--~-~-"
124
+
125
+ # Distinct footer end marker. It is important to realize that changing
126
+ # this could be problematic.
127
+ #
128
+ FOOTER_BLOCK_END = "--~--~---~-----~--~----~-----~~~----~---~---~--~----~"
129
+
78
130
  private
131
+ FOOTER_BLOCK_START_RE = %r[#{FOOTER_BLOCK_START}]
132
+ FOOTER_BLOCK_END_RE = %r[#{FOOTER_BLOCK_END}]
133
+
79
134
  # http://mail.python.org/pipermail/mailman-developers/2006-April/018718.html
80
135
  def bounce_headers
81
- {'sender' => "mlist-#{address}",
82
- 'errors-to' => "mlist-#{address}"}
136
+ # tmail would not correctly quote the label in the sender header, which would break smtp delivery
137
+ {'sender' => "<mlist-#{address}>", 'errors-to' => "#{label} <mlist-#{address}>"}
138
+ end
139
+
140
+ def delete_unreferenced_email
141
+ conditions = %Q{
142
+ mlist_emails.id in (
143
+ select me.id from mlist_emails me left join mlist_messages mm on mm.email_id = me.id
144
+ where mm.mail_list_id = #{id}
145
+ ) AND mlist_emails.id not in (
146
+ select meb.id from mlist_emails meb left join mlist_messages mmb on mmb.email_id = meb.id
147
+ where mmb.mail_list_id != #{id}
148
+ )}
149
+ MList::Email.delete_all(conditions)
150
+ end
151
+
152
+ def strip_list_footers(content)
153
+ if content =~ FOOTER_BLOCK_START_RE
154
+ in_footer_block = false
155
+ content = normalize_new_lines(content)
156
+ content = content.split("\n").reject do |line|
157
+ if in_footer_block
158
+ in_footer_block = line !~ FOOTER_BLOCK_END_RE
159
+ true
160
+ else
161
+ in_footer_block = line =~ FOOTER_BLOCK_START_RE
162
+ end
163
+ end.join("\n").rstrip
164
+ end
165
+ content
83
166
  end
84
167
 
85
168
  # http://www.jamesshuggins.com/h/web1/list-email-headers.htm
86
169
  def list_headers
87
- headers = list.list_headers
170
+ headers = list.list_headers.dup
88
171
  headers['x-beenthere'] = address
172
+ headers['x-mlist-version'] = MList.version.to_s
89
173
  headers.update(bounce_headers)
90
174
  headers.delete_if {|k,v| v.nil?}
91
175
  end
@@ -118,36 +202,55 @@ module MList
118
202
  def prepare_delivery(message, options)
119
203
  message.identifier = outgoing_server.generate_message_id
120
204
  message.created_at = options[:delivery_time]
205
+ message.subject = clean_subject(message.subject)
121
206
  returning(message.delivery) do |delivery|
122
207
  delivery.date = message.created_at
123
208
  delivery.message_id = message.identifier
124
209
  delivery.mailer = message.mailer
125
210
  delivery.headers = list_headers
126
- delivery.subject = list_subject(message)
211
+ delivery.subject = list_subject(message.subject)
127
212
  delivery.to = address
128
213
  delivery.bcc = message.recipients.collect(&:email_address)
129
214
  delivery.reply_to = "#{label} <#{post_url}>"
215
+ prepare_list_footer(delivery, message)
130
216
  end
131
217
  end
132
218
 
133
- def list_subject(message)
134
- prefix = "[#{label}]"
135
- subject = message.subject.dup
136
- if subject =~ /re: /i
137
- subject.gsub!(%r{(re:\s*)}i, '')
138
- prefix = 'Re: ' + prefix
219
+ def prepare_list_footer(delivery, message)
220
+ text_plain_part = delivery.text_plain_part
221
+ return unless text_plain_part
222
+
223
+ content = strip_list_footers(text_plain_part.body)
224
+ content << "\n\n" unless content.end_with?("\n\n")
225
+ content << list_footer(message)
226
+ text_plain_part.body = content
227
+ end
228
+
229
+ def list_footer(message)
230
+ content = list.footer_content(message)
231
+ "#{FOOTER_BLOCK_START}\n#{content}\n#{FOOTER_BLOCK_END}"
232
+ end
233
+
234
+ def list_subject(string)
235
+ list_subject = string.dup
236
+ if list_subject =~ REGARD_RE
237
+ "Re: #{subject_prefix} #{remove_regard(list_subject)}"
238
+ else
239
+ "#{subject_prefix} #{list_subject}"
139
240
  end
140
- subject.gsub!(/\[.*?\]/, '')
141
- subject.strip!
142
- "#{prefix} #{subject}"
143
241
  end
144
242
 
145
243
  def find_thread(message, options)
146
- if options[:search_parent]
147
- message.parent_identifier = message.email.parent_identifier(self)
148
- message.parent = messages.find_by_identifier(message.parent_identifier)
149
- end
244
+ message.parent = find_parent_message(message.email) if message.email && options[:search_parent]
150
245
  message.parent ? message.parent.thread : threads.build
151
246
  end
247
+
248
+ def subject_prefix_regex
249
+ @subject_prefix_regex ||= Regexp.new(Regexp.escape(subject_prefix) + ' ')
250
+ end
251
+
252
+ def subject_prefix
253
+ @subject_prefix ||= "[#{label}]"
254
+ end
152
255
  end
153
256
  end
@@ -0,0 +1,30 @@
1
+ require 'mlist/manager/notifier'
2
+
3
+ module MList
4
+
5
+ # The interface of list managers.
6
+ #
7
+ # A module is provided instead of a base class to allow implementors to
8
+ # subclass whatever they like. Practically speaking, they can create an
9
+ # ActiveRecord subclass.
10
+ #
11
+ module Manager
12
+
13
+ # Answers an enumeration of MList::List implementations to which the given
14
+ # email should be published.
15
+ #
16
+ def lists(email)
17
+ raise 'implement in your list manager'
18
+ end
19
+
20
+ # Answers the MList::Manager::Notifier of this list manager. Includers of
21
+ # this module may initialize the @notifier instance variable with their
22
+ # own implementation/subclass to generate custom content for the different
23
+ # notices.
24
+ #
25
+ def notifier
26
+ @notifier ||= MList::Manager::Notifier.new
27
+ end
28
+ end
29
+
30
+ end
@@ -2,6 +2,8 @@ module MList
2
2
  module Manager
3
3
 
4
4
  class Database
5
+ include ::MList::Manager
6
+
5
7
  def create_list(address, attributes = {})
6
8
  attributes = {
7
9
  :address => address,
@@ -16,7 +18,7 @@ module MList
16
18
  end
17
19
 
18
20
  def no_lists_found(email)
19
- # your application may care
21
+ # TODO: Move to notifier
20
22
  end
21
23
 
22
24
  class List < ActiveRecord::Base
@@ -24,6 +26,10 @@ module MList
24
26
 
25
27
  has_many :subscribers, :dependent => :delete_all
26
28
 
29
+ def label
30
+ self[:label]
31
+ end
32
+
27
33
  def list_id
28
34
  "#{self.class.name}#{id}"
29
35
  end
@@ -0,0 +1,31 @@
1
+ module MList
2
+ module Manager
3
+
4
+ # Constructs the notices that are sent to list subscribers. Applications
5
+ # may subclass this to customize the content of a notice delivery.
6
+ #
7
+ class Notifier
8
+
9
+ # Answers the delivery that will be sent to a subscriber when an
10
+ # MList::List indicates that the distribution of an email from that
11
+ # subscriber has been blocked.
12
+ #
13
+ def subscriber_blocked(list, email, subscriber)
14
+ delivery = MList::Util::TMailBuilder.new(TMail::Mail.new)
15
+ delivery.write_header('x-mlist-loop', 'notice')
16
+ delivery.write_header('x-mlist-notice', 'subscriber_blocked')
17
+ delivery.to = subscriber.email_address
18
+ delivery.from = "mlist-#{list.address}"
19
+ prepare_subscriber_blocked_content(list, email, subscriber, delivery)
20
+ delivery
21
+ end
22
+
23
+ protected
24
+ def prepare_subscriber_blocked_content(list, email, subscriber, delivery)
25
+ delivery.set_content_type('text/plain')
26
+ delivery.body = %{Although you are a subscriber to this list, your message cannot be posted at this time. Please contact the administrator of the list.}
27
+ end
28
+ end
29
+
30
+ end
31
+ end
@@ -10,6 +10,8 @@ module MList
10
10
  belongs_to :mail_list, :class_name => 'MList::MailList', :counter_cache => :messages_count
11
11
  belongs_to :thread, :class_name => 'MList::Thread', :counter_cache => :messages_count
12
12
 
13
+ after_destroy :delete_unreferenced_email
14
+
13
15
  # A temporary storage of recipient subscribers, obtained from
14
16
  # MList::Lists. This list is not available when a message is reloaded.
15
17
  #
@@ -73,13 +75,29 @@ module MList
73
75
  text_to_html(text_for_reply)
74
76
  end
75
77
 
76
- # Answers the subject with all prefixes removed.
78
+ def parent_with_identifier_capture=(parent)
79
+ if parent
80
+ self.parent_without_identifier_capture = parent
81
+ self.parent_identifier = parent.identifier
82
+ else
83
+ self.parent_without_identifier_capture = nil
84
+ self.parent_identifier = nil
85
+ end
86
+ end
87
+ alias_method_chain :parent=, :identifier_capture
88
+
89
+ # Answers the subject with 'Re:' prefixed. Note that it is the
90
+ # responsibility of the MList::MailList to perform any processing of the
91
+ # persisted subject (ie, cleaning up labels, etc).
92
+ #
93
+ # message.subject = '[List Label] Re: The new Chrome Browser from Google'
94
+ # message.subject_for_reply => 'Re: [List Label] The new Chrome Browser from Google'
77
95
  #
78
96
  # message.subject = 'Re: [List Label] Re: The new Chrome Browser from Google'
79
- # message.subject_for_reply => 'Re: The new Chrome Browser from Google'
97
+ # message.subject_for_reply => 'Re: [List Label] The new Chrome Browser from Google'
80
98
  #
81
99
  def subject_for_reply
82
- "Re: #{remove_regard(subject)}"
100
+ subject =~ REGARD_RE ? subject : "Re: #{subject}"
83
101
  end
84
102
 
85
103
  # Answers the subscriber from which this message comes.
@@ -113,5 +131,10 @@ module MList
113
131
  @subscriber = self.subscriber_address = self.subscriber_type = self.subscriber_id = nil
114
132
  end
115
133
  end
134
+
135
+ private
136
+ def delete_unreferenced_email
137
+ email.destroy unless MList::Message.count(:conditions => "email_id = #{email_id} and id != #{id}") > 0
138
+ end
116
139
  end
117
140
  end
@@ -1,10 +1,11 @@
1
1
  module MList
2
2
  class Server
3
- attr_reader :list_manager, :email_server
3
+ attr_reader :list_manager, :email_server, :notifier
4
4
 
5
5
  def initialize(config)
6
6
  @list_manager = config[:list_manager]
7
7
  @email_server = config[:email_server]
8
+ @notifier = MList::Manager::Notifier.new
8
9
  @email_server.receiver(self)
9
10
  end
10
11
 
@@ -24,23 +25,38 @@ module MList
24
25
  end
25
26
 
26
27
  protected
27
- def process_bounce(list, email)
28
- list.bounce(email)
29
- end
30
-
31
28
  def process_post(lists, email)
32
29
  lists.each do |list|
33
30
  next if email.been_here?(list)
34
31
  if list.subscriber?(email.from_address)
35
- if list.active?
36
- mail_list(list).process_email(email)
37
- else
38
- list.inactive_post(email)
39
- end
32
+ publish_if_list_active(list, email)
40
33
  else
41
34
  list.non_subscriber_post(email)
42
35
  end
43
36
  end
44
37
  end
38
+
39
+ def publish_if_list_active(list, email)
40
+ if list.active?
41
+ subscriber = list.subscriber(email.from_address)
42
+ publish_unless_blocked(list, email, subscriber)
43
+ else
44
+ list.inactive_post(email)
45
+ end
46
+ end
47
+
48
+ def publish_unless_blocked(list, email, subscriber)
49
+ if list.blocked?(subscriber)
50
+ notice_delivery = notifier.subscriber_blocked(list, email, subscriber)
51
+ email_server.deliver(notice_delivery.tmail)
52
+ else
53
+ mail_list(list).process_email(email, subscriber)
54
+ end
55
+ end
56
+
57
+ def process_bounce(list, email)
58
+ list.bounce(email)
59
+ end
60
+
45
61
  end
46
62
  end
@@ -5,34 +5,94 @@ module MList
5
5
  belongs_to :mail_list, :class_name => 'MList::MailList', :counter_cache => :threads_count
6
6
  has_many :messages, :class_name => 'MList::Message', :dependent => :delete_all
7
7
 
8
- def children(message)
9
- messages.select {|m| m.parent == message}
10
- end
11
-
12
8
  def first?(message)
13
- messages.first == message
9
+ tree_order.first == message
14
10
  end
15
11
 
16
12
  def last?(message)
17
- messages.last == message
13
+ tree_order.last == message
18
14
  end
19
15
 
20
16
  def next(message)
21
- i = messages.index(message)
17
+ i = tree_order.index(message)
22
18
  messages[i + 1] unless messages.size < i
23
19
  end
24
20
 
25
21
  def previous(message)
26
- i = messages.index(message)
22
+ i = tree_order.index(message)
27
23
  messages[i - 1] if i > 0
28
24
  end
29
25
 
30
- def roots
31
- messages.select {|m| m.parent.nil?}
32
- end
33
-
34
26
  def subject
35
27
  messages.first.subject
36
28
  end
29
+
30
+ # Answers a tree of messages.
31
+ #
32
+ # The nodes of the tree are decorated to act like a linked list, providing
33
+ # pointers to _next_ and _previous_ in the tree.
34
+ #
35
+ def tree
36
+ return nil if messages.size == 0
37
+ build_tree unless @tree
38
+ @tree
39
+ end
40
+
41
+ def tree_order
42
+ build_tree unless @tree
43
+ @tree_order
44
+ end
45
+
46
+ private
47
+ def build_tree
48
+ nodes = messages.collect do |m|
49
+ m.parent = messages.detect {|pm| pm.id == m.parent_id}
50
+ Node.new(m)
51
+ end
52
+
53
+ nodes.each do |node|
54
+ if parent_node = nodes.detect {|n| n == node.parent}
55
+ node.parent_node = parent_node
56
+ parent_node.children << node
57
+ end
58
+ end
59
+
60
+ @tree_order = []
61
+ previous_node = nil
62
+ nodes.first.visit do |node|
63
+ @tree_order << node
64
+ if previous_node
65
+ node.previous = previous_node
66
+ previous_node.next = node
67
+ end
68
+ previous_node = node
69
+ end
70
+
71
+ @tree = @tree_order.first
72
+ end
73
+
74
+ class Node < DelegateClass(Message)
75
+ attr_accessor :parent_node, :previous, :next
76
+ attr_reader :children
77
+
78
+ def initialize(message)
79
+ super
80
+ @children = []
81
+ end
82
+
83
+ def leaf?
84
+ children.empty?
85
+ end
86
+
87
+ def root?
88
+ parent_node.nil?
89
+ end
90
+
91
+ def visit(&visitor)
92
+ visitor.call self
93
+ children.each {|c| c.visit(&visitor)}
94
+ end
95
+ end
96
+
37
97
  end
38
98
  end
@@ -1,6 +1,83 @@
1
1
  module MList
2
2
  module Util
3
3
 
4
+ class HtmlTextExtraction
5
+ def initialize(html)
6
+ @doc = Hpricot(html)
7
+ end
8
+
9
+ def execute
10
+ @text, @anchors = '', []
11
+ @doc.each_child do |node|
12
+ extract_text_from_node(node) if Hpricot::Elem::Trav === node
13
+ end
14
+ @text.strip!
15
+ unless @anchors.empty?
16
+ refs = []
17
+ @anchors.each_with_index do |href, i|
18
+ refs << "[#{i+1}] #{href}"
19
+ end
20
+ @text << "\n\n--\n#{refs.join("\n")}"
21
+ end
22
+ @text
23
+ end
24
+
25
+ def extract_text_from_node(node)
26
+ case node.name
27
+ when 'head'
28
+ when 'h1', 'h2', 'h3', 'h4', 'h5', 'h6'
29
+ @text << node.inner_text
30
+ @text << "\n\n"
31
+ when 'br'
32
+ @text << "\n"
33
+ when 'ol'
34
+ node.children_of_type('li').each_with_index do |li, i|
35
+ @text << " #{i+1}. #{li.inner_text}"
36
+ @text << "\n\n"
37
+ end
38
+ when 'ul'
39
+ node.children_of_type('li').each do |li|
40
+ @text << " * #{li.inner_text.strip}"
41
+ @text << "\n\n"
42
+ end
43
+ when 'strong'
44
+ @text << "*#{node.inner_text}*"
45
+ when 'em'
46
+ @text << "_#{node.inner_text}_"
47
+ when 'dl'
48
+ node.traverse_element('dt', 'dd') do |dt_dd|
49
+ extract_text_from_node(dt_dd)
50
+ end
51
+ when 'a'
52
+ @anchors << node['href']
53
+ extract_text_from_text_node(node)
54
+ @text << "[#{@anchors.size}]"
55
+ when 'p', 'dt', 'dd'
56
+ extract_text_from_children(node)
57
+ @text.rstrip!
58
+ @text << "\n\n"
59
+ else
60
+ extract_text_from_children(node)
61
+ end
62
+ end
63
+
64
+ def extract_text_from_children(elem)
65
+ elem.each_child do |node|
66
+ case node
67
+ when Hpricot::Text::Trav
68
+ extract_text_from_text_node(node)
69
+ when Hpricot::Elem::Trav
70
+ extract_text_from_node(node)
71
+ end
72
+ end
73
+ end
74
+
75
+ def extract_text_from_text_node(node)
76
+ text = @text.end_with?("\n") ? node.inner_text.lstrip : node.inner_text
77
+ @text << text.gsub(/\s{2,}/, ' ').sub(/\n/, '')
78
+ end
79
+ end
80
+
4
81
  module EmailHelpers
5
82
  def sanitize_header(charset, name, *values)
6
83
  header_sanitizer(name).call(charset, *values)
@@ -10,12 +87,15 @@ module MList
10
87
  Util.default_header_sanitizers[name]
11
88
  end
12
89
 
90
+ def html_to_text(html)
91
+ HtmlTextExtraction.new(html).execute
92
+ end
93
+
13
94
  def normalize_new_lines(text)
14
95
  text.to_s.gsub(/\r\n?/, "\n")
15
96
  end
16
97
 
17
98
  BRACKETS_RE = /\A<(.*?)>\Z/
18
-
19
99
  def bracket(string)
20
100
  string.blank? || string =~ BRACKETS_RE ? string : "<#{string}>"
21
101
  end
@@ -24,9 +104,12 @@ module MList
24
104
  string =~ BRACKETS_RE ? $1 : string
25
105
  end
26
106
 
107
+ REGARD_RE = /(^|[^\w])re: /i
27
108
  def remove_regard(string)
28
- stripped = string.strip
29
- stripped =~ /\A.*re:\s+(\[.*\]\s*)?(.*?)\Z/i ? $2 : stripped
109
+ while string =~ REGARD_RE
110
+ string = string.sub(REGARD_RE, ' ')
111
+ end
112
+ string.strip
30
113
  end
31
114
 
32
115
  def text_to_html(text)
@@ -19,8 +19,8 @@ module MList
19
19
  when 'text/html'
20
20
  tmail.body.strip
21
21
  when 'multipart/alternative'
22
- text_part = tmail.parts.detect {|part| part.content_type == 'text/html'}
23
- text_part.body.strip if text_part
22
+ html_part = tmail.parts.detect {|part| part.content_type == 'text/html'}
23
+ html_part.body.strip if html_part
24
24
  end
25
25
  end
26
26
 
@@ -33,19 +33,45 @@ module MList
33
33
  end
34
34
 
35
35
  def text
36
- returning('') {|content| extract_text_content(tmail, content)}
36
+ text_content = ''
37
+ extract_text_content(tmail, text_content)
38
+ return text_content unless text_content.blank?
39
+
40
+ html_content = ''
41
+ extract_html_content(tmail, html_content)
42
+ return html_to_text(html_content) unless html_content.blank?
43
+
44
+ return nil
45
+ end
46
+
47
+ # Answers the first text/plain part it can find, the tmail itself if
48
+ # it's content type is text/plain.
49
+ #
50
+ def text_plain_part(part = tmail)
51
+ case part.content_type
52
+ when 'text/plain'
53
+ part
54
+ when 'multipart/alternative'
55
+ part.parts.detect {|part| text_plain_part(part)}
56
+ end
37
57
  end
38
58
 
39
59
  private
60
+ def extract_html_content(part, collector)
61
+ case part.content_type
62
+ when 'text/html'
63
+ collector << part.body.strip
64
+ when 'multipart/alternative', 'multipart/mixed', 'multipart/related'
65
+ part.parts.each {|part| extract_html_content(part, collector)}
66
+ end
67
+ end
68
+
40
69
  def extract_text_content(part, collector)
41
70
  case part.content_type
42
71
  when 'text/plain'
43
72
  collector << part.body.strip
44
- when 'multipart/alternative'
45
- text_part = part.parts.detect {|part| part.content_type == 'text/plain'}
46
- collector << text_part.body.strip if text_part
47
- when 'multipart/mixed', 'multipart/related'
48
- part.parts.each {|mixed_part| extract_text_content(mixed_part, collector)}
73
+ when 'multipart/alternative', 'multipart/mixed', 'multipart/related'
74
+ part.parts.each {|part| extract_text_content(part, collector)}
49
75
  end
50
76
  end
51
77
  end
@@ -0,0 +1,28 @@
1
+ require 'dispatcher' unless defined?(::Dispatcher)
2
+
3
+ Dispatcher.module_eval do
4
+ # Provides the mechinism to support applications that want to observe MList
5
+ # models.
6
+ #
7
+ # ActiveRecord observers are reloaded at each request in development mode.
8
+ # They will be registered with the MList models each time. Since the MList
9
+ # models are required once at initialization, there will always only be one
10
+ # instance of the model class, and therefore, many instances of the observer
11
+ # class registered with it; all but the most recent are invalid, since they
12
+ # were undefined when the dispatcher reloaded the application.
13
+ #
14
+ # Why not an initializer "to_prepare" block? Simply because we must clear
15
+ # the observers in the ActiveRecord classes before the
16
+ # ActiveRecord::Base.instantiate_observers call is made by the prepare block
17
+ # that we cannot get in front of with the initializer approach. Also, it
18
+ # lessens the configuration burden of the MList client application.
19
+ #
20
+ # Should we ever have observers in MList, this will likely need more careful
21
+ # attention.
22
+ #
23
+ def reload_application_with_plugin_record_support
24
+ ActiveRecord::Base.send(:subclasses).each(&:delete_observers)
25
+ reload_application_without_plugin_record_support
26
+ end
27
+ alias_method_chain :reload_application, :plugin_record_support
28
+ end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: aiwilliams-mlist
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.4
4
+ version: 0.1.5
5
5
  platform: ruby
6
6
  authors:
7
7
  - Adam Williams
@@ -9,7 +9,7 @@ autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
11
 
12
- date: 2009-02-09 00:00:00 -08:00
12
+ date: 2009-02-17 00:00:00 -08:00
13
13
  default_executable:
14
14
  dependencies: []
15
15
 
@@ -25,6 +25,7 @@ files:
25
25
  - CHANGELOG
26
26
  - Rakefile
27
27
  - README
28
+ - TODO
28
29
  - VERSION.yml
29
30
  - lib/mlist
30
31
  - lib/mlist/email.rb
@@ -41,6 +42,8 @@ files:
41
42
  - lib/mlist/mail_list.rb
42
43
  - lib/mlist/manager
43
44
  - lib/mlist/manager/database.rb
45
+ - lib/mlist/manager/notifier.rb
46
+ - lib/mlist/manager.rb
44
47
  - lib/mlist/message.rb
45
48
  - lib/mlist/server.rb
46
49
  - lib/mlist/thread.rb
@@ -53,6 +56,7 @@ files:
53
56
  - lib/mlist/util.rb
54
57
  - lib/mlist.rb
55
58
  - lib/pop_ssl.rb
59
+ - rails/init.rb
56
60
  has_rdoc: false
57
61
  homepage: http://github.com/aiwilliams/mlist
58
62
  post_install_message: