mail 1.1.0 → 1.2.1

data/CHANGELOG.rdoc

@@ -1,6 +1,19 @@
+== Wed Nov  4 23:24:32 UTC 2009 Mikel Lindsaar <raasdnil@gmail.com>
+
+* Created commit 2b5d608: Closes Issue #2 - Empty header field values not parsing <mikel>
+* Version bumb to 1.2.1
+
+== Wed Nov  4 12:54:43 UTC 2009 Mikel Lindsaar <raasdnil@gmail.com>
+
+* Renamed Mail::Message.encode! to Mail::Message.ready_to_send!, deprecated :encode! <mikel>
+* Rewrote encoding and decoding methods on all classes.  Adds a lot of boiler plate code, but allows us to
+  be really precise in each field type that needs custom encoding.  Now all encoding is done by the field_type
+  itself.  Need to follow through on the body. <mikel>
+* Bump version to 1.2.0 due to changes of :encoded, :decoded behaviour <mikel>
+
 == Tue Nov  3 00:59:45 UTC 2009 Mikel Lindsaar <raasdnil@gmail.com>
 
-* Tested mail against entire Enron set (2.3gb) and the Trec 2005 set (0.5gb), ~ half a million emails without crashing
+* Tested mail against entire Enron set (2.3gb) and the Trec 2005 set (0.5gb), ~ half a million emails without crashing <jlindley>
 * Some headers only can appear once, enforce during header parse assignment. <jlindley>
 * Convert empty bodies into empty arrays instead of nil. <jlindley>
 * Handle blank content dispositions. <jlindley>

data/README.rdoc

@@ -49,7 +49,8 @@ and you can manually create any other type of MIME email.
 
 Next TODO:
 
-* Add multilingual MIME support for mixed ASCII and multibyte headers
+* Improve MIME support for character sets in headers, currently works, mostly, needs
+  refinement.
 * Add IMAP wrapper
 
 == Testing Policy
@@ -64,13 +65,10 @@ It also means you can be sure Mail will behave correctly.
 
 == API Policy
 
-Right now Mail is still under development for a 1.0.0 release.
+No API removals within a single point release.  All removals to be depreciated with
+warnings for at least one MINOR point release before removal.
 
-The API will change between now and 1.0.0.  How much?  Not sure.  Basically though
-once 1.0.0 is released, no API removals within a single point release.  All removals
-to be depreciated with warnings for at least one point release before removal.
-
-Also, all private or protected methods to be declared as such.
+Also, all private or protected methods to be declared as such - though this is still I/P.
 
 == Installation
 
@@ -89,7 +87,34 @@ may or may not be a problem for you.
 
 If you want to install mail manually, you can download the gem from github and do:
 
- # gem install mail-1.0.0.gem
+ # gem install mail-1.2.1.gem
+ 
+== Encodings
+
+If you didn't know, handling encodings in Emails is not as straight forward as you
+would hope.
+
+I have tried to simplify it some:
+
+1. All objects that can render into an email, have an :encoded method.  Encoded will
+   return the object as a complete string ready to send in the mail system, that is,
+   it will include the header field and value and CRLF at the end and wrapped as
+   needed.
+   
+2. All objects that can render into an email, have a :decoded method.  Decoded will
+   return the object's "value" only as a string.  This means it will not include
+   the header fields (like 'To:' or 'Subject:').
+   
+3. By default, calling :to_s on an object will call it's encoded method, that is, make
+   it ready to send in an email.
+   
+4. Structured fields that have parameter values that can be encoded (e.g. Content-Type) will
+   provide decoded parameter values when you call the parameter names as methods against
+   the object.
+   
+5. Structured fields that have parameter values that can be encoded (e.g. Content-Type) will
+   provide encoded parameter values when you call the parameter names through the 
+   object.parameters['<parameter_name>'] method call.
  
 == Contributing
 
@@ -417,11 +442,10 @@ Of course... Mail will round trip an attachment as well
 
 == Excerpts from TREC Spam Corpus 2005
 
-The spec fixture files in spec/fixtures/emails/from_trec_2005
-are from the 2005 TREC Public Spam Corpus. They remain copyrighted
-under the terms of that project and license agreement. They are used
-in this project to verify and describe the development of this
-email parser implementation.
+The spec fixture files in spec/fixtures/emails/from_trec_2005 are from the 
+2005 TREC Public Spam Corpus. They remain copyrighted under the terms of
+that project and license agreement. They are used in this project to verify 
+and describe the development of this email parser implementation.
 
 http://plg.uwaterloo.ca/~gvcormac/treccorpus/
 

data/lib/mail/configuration.rb

@@ -85,6 +85,10 @@ module Mail
       @tls || false
     end
     
+    def param_encode_language(value = nil)
+      value ? @encode_language = value : @encode_language ||= 'en'
+    end
+    
   end
 
 end

data/lib/mail/elements/address.rb

@@ -82,6 +82,7 @@ module Mail
     def display_name
       parse unless @parsed
       @display_name ||= get_display_name
+      Encodings.decode_encode(@display_name, @output_type) if @display_name
     end
     
     # Provides a way to assign a display name to an already made Mail::Address object.
@@ -154,6 +155,16 @@ module Mail
       parse unless @parsed
       "#<#{self.class}:#{self.object_id} Address: |#{to_s}| >"
     end
+    
+    def encoded
+      @output_type = :encode
+      format
+    end
+    
+    def decoded
+      @output_type = :decode
+      format
+    end
 
     private
     

data/lib/mail/encodings/base64.rb

@@ -12,6 +12,7 @@ module Mail
       def self.encode(str)
         RubyVer.encode_base64( str )
       end
+      
     end
   end
 end

data/lib/mail/encodings/encodings.rb

@@ -2,26 +2,153 @@
 module Mail
   module Encodings
     
+    include Mail::Patterns
+    
+    # Is the encoding we want defined?
+    # 
+    # Example:
+    # 
+    #  Encodings.defined?(:base64) #=> true
     def Encodings.defined?( str )
       string = str.to_s.split(/[_-]/).map { |v| v.capitalize }.join('')
       RubyVer.has_constant?(Mail::Encodings, string)
     end
     
+    # Gets a defined encoding type, QuotedPrintable or Base64 for now.
+    # 
+    # Each encoding needs to be defined as a Mail::Encodings::ClassName for 
+    # this to work, allows us to add other encodings in the future.
+    # 
+    # Example:
+    # 
+    #  Encodings.get_encoding(:base64) #=> Mail::Encodings::Base64
     def Encodings.get_encoding( str )
       string = str.to_s.split(/[_-]/).map { |v| v.capitalize }.join('')
       RubyVer.get_constant(Mail::Encodings, string)
     end
+
+    # Encodes a parameter value using URI Escaping, note the language field 'en' can
+    # be set using Mail::Configuration, like so:
+    # 
+    #  Mail.defaults.do
+    #    param_encode_language 'jp'
+    #  end
+    #
+    # The character set used for encoding will either be the value of $KCODE for 
+    # Ruby < 1.9 or the encoding on the string passed in.
+    # 
+    # Example:
+    # 
+    #  Mail::Encodings.param_encode("This is fun") #=> "us-ascii'en'This%20is%20fun"
+    def Encodings.param_encode(str)
+      RubyVer.param_encode(str)
+    end
+
+    # Decodes a parameter value using URI Escaping.
+    # 
+    # Example:
+    # 
+    #  Mail::Encodings.param_decode("This%20is%20fun", 'us-ascii') #=> "This is fun"
+    #
+    #  str = Mail::Encodings.param_decode("This%20is%20fun", 'iso-8559-1')
+    #  str.encoding #=> 'ISO-8859-1'      ## Only on Ruby 1.9
+    #  str #=> "This is fun"
+    def Encodings.param_decode(str, encoding)
+      RubyVer.param_decode(str, encoding)
+    end
     
-    def Encodings.b_encode(str, encoding = nil)
-      RubyVer.b_encode(str, encoding)
+    # Decodes or encodes a string as needed for either Base64 or QP encoding types in
+    # the =?<encoding>?[QB]?<string>?=" format.
+    # 
+    # The output type needs to be :decode to decode the input string or :encode to
+    # encode the input string.  The character set used for encoding will either be
+    # the value of $KCODE for Ruby < 1.9 or the encoding on the string passed in.
+    # 
+    # On encoding, will only send out Base64 encoded strings.
+    def Encodings.decode_encode(str, output_type)
+      case
+      when output_type == :decode
+        Encodings.value_decode(str)
+      else
+        if str.ascii_only?
+          str
+        else
+          Encodings.b_value_encode(str, find_encoding(str))
+        end
+      end
     end
     
-    def Encodings.q_encode(str, encoding = nil)
-      RubyVer.q_encode(str, encoding)
+    # Decodes a given string as Base64 or Quoted Printable, depending on what
+    # type it is.
+    # 
+    # 
+    def Encodings.value_decode(str)
+      case
+      when str =~ /\=\?.+?\?B\?.+?\?\=/
+        Encodings.b_value_decode(str)
+      when str =~ /\=\?.+?\?Q\?.+?\?\=/
+        Encodings.q_value_decode(str)
+      else
+        str
+      end
+    end
+
+    # Encode a string with Base64 Encoding and returns it ready to be inserted
+    # as a value for a field, that is, in the =?<charset>?B?<string>?= format
+    #
+    # Example:
+    # 
+    #  Encodings.b_value_encode('This is あ string', 'UTF-8') 
+    #  #=> "=?UTF-8?B?VGhpcyBpcyDjgYIgc3RyaW5n?="
+    def Encodings.b_value_encode(str, encoding = nil)
+      string, encoding = RubyVer.b_value_encode(str, encoding)
+      string.split("\n").map do |str|
+        "=?#{encoding}?B?#{str.chomp}?="
+      end.join(" ")
     end
     
-    def Encodings.param_decode(str, encoding)
-      RubyVer.param_decode(str, encoding)
+    # Decodes a Base64 string from the "=?UTF-8?B?VGhpcyBpcyDjgYIgc3RyaW5n?=" format
+    # 
+    # Example:
+    # 
+    #  Encodings.b_value_encode("=?UTF-8?B?VGhpcyBpcyDjgYIgc3RyaW5n?=") 
+    #  #=> 'This is あ string'
+    def Encodings.b_value_decode(str)
+      string = str.split(Mail::Patterns::WSP)
+      string.flatten.map do |s|
+        RubyVer.b_value_decode(s)
+      end.join('')
+    end
+    
+    # Encode a string with Quoted-Printable Encoding and returns it ready to be inserted
+    # as a value for a field, that is, in the =?<charset>?Q?<string>?= format
+    #
+    # Example:
+    # 
+    #  Encodings.q_value_encode('This is あ string', 'UTF-8') 
+    #  #=> "=?UTF-8?Q?This_is_=E3=81=82_string?="
+    def Encodings.q_value_encode(str, encoding = nil)
+      string, encoding = RubyVer.q_value_encode(str, encoding)
+      "=?#{encoding}?Q?#{string.chomp}?="
+    end
+    
+    # Decodes a Quoted-Printable string from the "=?UTF-8?Q?This_is_=E3=81=82_string?=" format
+    # 
+    # Example:
+    # 
+    #  Encodings.b_value_encode("=?UTF-8?Q?This_is_=E3=81=82_string?=") 
+    #  #=> 'This is あ string'
+    def Encodings.q_value_decode(str)
+      string = str.split(Mail::Patterns::WSP)
+      string.flatten.map do |s|
+        RubyVer.q_value_decode(s)
+      end.join('')
+    end
+    
+    private
+    
+    def Encodings.find_encoding(str)
+      RUBY_VERSION >= '1.9' ? str.encoding : $KCODE
     end
     
   end

data/lib/mail/encodings/quoted_printable.rb

@@ -5,7 +5,7 @@ module Mail
     
       # Decode the string from Quoted-Printable
       def self.decode(str)
-        str.unpack("M*").first
+        str.unpack("M*").first.gsub('_', ' ')
       end
 
       def self.encode(str)

data/lib/mail/field.rb

@@ -69,7 +69,7 @@ module Mail
       case
       when name =~ /:/ && value.blank?   # Field.new("field-name: field data")
         name, value = split(name)
-        create_field(name, value)
+        create_field(name, value.to_s)
       when name !~ /:/ && value.blank?  # Field.new("field-name")
         create_field(name, nil)
       else                              # Field.new("field-name", "value")
@@ -111,8 +111,8 @@ module Mail
     end
     
     def <=>( other )
-      self_order = FIELD_ORDER.rindex(self.name.downcase) || 100
-      other_order = FIELD_ORDER.rindex(other.name.downcase) || 100
+      self_order = FIELD_ORDER.rindex(self.name.to_s.downcase) || 100
+      other_order = FIELD_ORDER.rindex(other.name.to_s.downcase) || 100
       self_order <=> other_order
     end
     
@@ -132,7 +132,7 @@ module Mail
     private
     
     def split(raw_field)
-      match_data = raw_field.match(/^(#{FIELD_NAME})\s*:\s*(#{FIELD_BODY})$/)
+      match_data = raw_field.match(/^(#{FIELD_NAME})\s*:\s*(#{FIELD_BODY})?$/)
       [match_data[1].to_s.strip, match_data[2].to_s.strip]
     rescue
       STDERR.puts "WARNING: Could not parse (and so ignorning) '#{raw_field}'"
@@ -149,64 +149,64 @@ module Mail
     def new_field(name, value)
       # Could do this with constantize and make it "as DRY as", but a simple case 
       # statement is, well, simpler... 
-      case name.downcase
-      when /^to$/
+      case name
+      when /^to$/i
         ToField.new(name, value)
-      when /^cc$/
+      when /^cc$/i
         CcField.new(name, value)
-      when /^bcc$/
+      when /^bcc$/i
         BccField.new(name, value)
-      when /^message-id$/
+      when /^message-id$/i
         MessageIdField.new(name, value)
-      when /^in-reply-to$/
+      when /^in-reply-to$/i
         InReplyToField.new(name, value)
-      when /^references$/
+      when /^references$/i
         ReferencesField.new(name, value)
-      when /^subject$/
+      when /^subject$/i
         SubjectField.new(name, value)
-      when /^comments$/
+      when /^comments$/i
         CommentsField.new(name, value)
-      when /^keywords$/
+      when /^keywords$/i
         KeywordsField.new(name, value)
-      when /^date$/
+      when /^date$/i
         DateField.new(name, value)
-      when /^from$/
+      when /^from$/i
         FromField.new(name, value)
-      when /^sender$/
+      when /^sender$/i
         SenderField.new(name, value)
-      when /^reply-to$/
+      when /^reply-to$/i
         ReplyToField.new(name, value)
-      when /^resent-date$/
+      when /^resent-date$/i
         ResentDateField.new(name, value)
-      when /^resent-from$/
+      when /^resent-from$/i
         ResentFromField.new(name, value)
-      when /^resent-sender$/ 
+      when /^resent-sender$/i 
         ResentSenderField.new(name, value)
-      when /^resent-to$/
+      when /^resent-to$/i
         ResentToField.new(name, value)
-      when /^resent-cc$/
+      when /^resent-cc$/i
         ResentCcField.new(name, value)
-      when /^resent-bcc$/
+      when /^resent-bcc$/i
         ResentBccField.new(name, value)
-      when /^resent-message-id$/
+      when /^resent-message-id$/i
         ResentMessageIdField.new(name, value)
-      when /^return-path$/
+      when /^return-path$/i
         ReturnPathField.new(name, value)
-      when /^received$/
+      when /^received$/i
         ReceivedField.new(name, value)
-      when /^mime-version$/
+      when /^mime-version$/i
         MimeVersionField.new(name, value)
-      when /^content-transfer-encoding$/
+      when /^content-transfer-encoding$/i
         ContentTransferEncodingField.new(name, value)
-      when /^content-description$/
+      when /^content-description$/i
         ContentDescriptionField.new(name, value)
-      when /^content-disposition$/
+      when /^content-disposition$/i
         ContentDispositionField.new(name, value)
-      when /^content-type$/
+      when /^content-type$/i
         ContentTypeField.new(name, value)
-      when /^content-id$/
+      when /^content-id$/i
         ContentIdField.new(name, value)
-      when /^content-location$/
+      when /^content-location$/i
         ContentLocationField.new(name, value)
       else 
         OptionalField.new(name, value)

data/lib/mail/fields/bcc_field.rb

@@ -31,9 +31,18 @@ module Mail
     include Mail::CommonAddress
     
     FIELD_NAME = 'bcc'
+    CAPITALIZED_FIELD = 'Bcc'
     
     def initialize(*args)
-      super(FIELD_NAME, strip_field(FIELD_NAME, args.last))
+      super(CAPITALIZED_FIELD, strip_field(FIELD_NAME, args.last))
+    end
+    
+    def encoded
+      do_encode(CAPITALIZED_FIELD)
+    end
+    
+    def decoded
+      do_decode
     end
     
   end