ruby-msg 1.2.17

data/FIXES

@@ -0,0 +1,34 @@
+FIXES
+
+recent fixes based on importing results into evolution
+
+1. was running into some issue with base64 encoded message/rfc822 attachments displaying
+   as empty. encoding them as plain solved the issue (odd).
+
+2. problem with a large percentage of emails, not displaying as mime. turned out to be
+   all received from blackberry. further, turned out there was 2 content-type headers,
+	 "Content-Type", which I add, and "Content-type". normally my override works, but I
+	 need to handle it case insensitvely it would appear. more tricky, whats the story
+	 with these. fixing that will probably fix that whole class of issues there.
+	 evolution was renaming my second content type as X-Invalid-Content-Type or something.
+
+3. another interesting one. had content-transfer-encoding set in the transport message
+   headers. it was set to base64. i didn't override that, so evolution "decoded" my
+	 plaintext message into complete garbage.
+	 fix - delete content-transfer-encoding.
+
+4. added content-location and content-id output in the mime handling of attachments
+   to get some inline html/image mails to work properly.
+   further, the containing mime content-type must be multipart/related, not multipart/mixed,
+	 at least for evolution, in order for the images to appear inline.
+   could still improve in this area. if someone drags and drops in an image, it may
+	 be inline in the rtf version, but exchanges generates crappy html such that the image
+	 doesn't display inline. maybe i should correct the html output in these cases as i'm
+	 throwing away the rtf version.
+
+5. note you may need wingdings installed. i had a lot of L and J appear in messages from
+   outlook users. turns out its smilies in wingdings. i think its only if word is used
+	 as email editor and has autotext messing things up.
+
+6. still unsure about how to do my "\r" handling.
+

data/README

@@ -0,0 +1,121 @@
+#summary ruby-msg - A library for reading Outlook msg files, and for converting them to RFC2822 emails.
+
+= Introduction =
+
+Generally, the goal of the project is the conversion of .msg files into proper rfc2822
+emails, independent of outlook, or any platform dependencies etc. 
+In fact its currently pure ruby, so it should be easy to get started with.
+
+It draws on `msgconvert.pl`, but tries to take a cleaner and more complete approach.
+Neither are complete yet, however, but I think that this project provides a clean foundation upon which to work on a good converter for msg files for use in outlook migrations etc.
+
+I am happy to accept patches, give commit bits etc.
+
+Please let me know how it works for you, any feedback would be welcomed.
+
+= Usage =
+
+Higher level access to the msg, can be had through the top level data accessors.
+
+{{{
+require 'msg'
+
+msg = Msg.load open(filename)
+
+# access to the 3 main data stores, if you want to poke with the msg
+# internals
+msg.recipients
+# => [#<Recipient:'\'Marley, Bob\' <bob.marley@gmail.com>'>]
+msg.attachments
+# => [#<Attachment filename='blah1.tif'>, #<Attachment filename='blah2.tif'>]
+msg.properties
+# => #<Properties ... normalized_subject='Testing' ... 
+# creation_time=#<DateTime: 2454042.45074714,0,2299161> ...>
+}}}
+
+To completely abstract away all msg peculiarities, convert the msg to a mime object.
+The message as a whole, and some of its main parts support conversion to mime objects.
+
+{{{
+msg.attachments.first.to_mime
+# => #<Mime content_type='application/octet-stream'>
+mime = msg.to_mime
+puts mime.to_tree
+# =>
+- #<Mime content_type='multipart/mixed'>
+  |- #<Mime content_type='multipart/alternative'>
+  |  |- #<Mime content_type='text/plain'>
+  |  \- #<Mime content_type='text/html'>
+  |- #<Mime content_type='application/octet-stream'>
+  \- #<Mime content_type='application/octet-stream'>
+
+# convert mime object to serialised form,
+# inclusive of attachments etc. (not ideal in memory, but its wip).
+puts mime.to_s
+}}}
+
+You can also access the underlying ole object, and see all the gory details of how msgs are serialised:
+
+{{{
+puts msg.ole.root.to_tree
+# =>
+- #<OleDir:"Root Entry" size=3840 time="2006-11-03T00:52:53Z">
+  |- #<OleDir:"__nameid_version1.0" size=0 time="2006-11-03T00:52:53Z">
+  |  |- #<OleDir:"__substg1.0_00020102" size=16 data="CCAGAAAAAADAAA...">
+  |  |- #<OleDir:"__substg1.0_00030102" size=64 data="DoUAAAYAAABShQ...">
+  |  |- #<OleDir:"__substg1.0_00040102" size=0 data="">
+  |  |- #<OleDir:"__substg1.0_10010102" size=16 data="UoUAAAYAAQAQhQ...">
+  |  |- #<OleDir:"__substg1.0_10090102" size=8 data="GIUAAAYABgA=">
+  |  |- #<OleDir:"__substg1.0_100A0102" size=8 data="BoUAAAYABwA=">
+  |  |- #<OleDir:"__substg1.0_100F0102" size=8 data="A4UAAAYABAA=">
+  |  |- #<OleDir:"__substg1.0_10110102" size=8 data="AYUAAAYAAwA=">
+  |  |- #<OleDir:"__substg1.0_10120102" size=8 data="DoUAAAYAAAA=">
+  |  \- #<OleDir:"__substg1.0_101E0102" size=8 data="VIUAAAYAAgA=">
+  |- #<OleDir:"__substg1.0_001A001E" size=8 data="SVBNLk5vdGU=">
+  ...
+  |- #<OleDir:"__substg1.0_8002001E" size=4 data="MTEuMA==">
+  |- #<OleDir:"__properties_version1.0" size=800 data="AAAAAAAAAAABAA...">
+  \- #<OleDir:"__recip_version1.0_#00000000" size=0 time="2006-11-03T00:52:53Z">
+
+     |- #<OleDir:"__substg1.0_0FF60102" size=4 data="AAAAAA==">
+     |- #<OleDir:"__substg1.0_3001001E" size=4 data="YXNkZg==">
+     |- #<OleDir:"__substg1.0_5FF6001E" size=4 data="YXNkZg==">
+     \- #<OleDir:"__properties_version1.0" size=152 data="AAAAAAAAAAAeAA...">
+}}}
+
+= Further Details =
+
+Named properties have recently been implemented, and Msg::Properties now allows associated guids. Keys are represented by Msg::Properties::Key, which contains the relevant code.
+
+You can now write code like:
+{{{
+props = msg.properties
+
+props[0x0037] # access subject by mapi code
+props[0x0037, Msg::Properties::PS_MAPI] # equivalent, with explicit GUID.
+key = Msg::Properties::Key.new 0x0037 # => 0x0037
+props[key] # same again
+
+# keys support being converted to symbols, and then use a symbolic lookup
+key.to_sym # => :subject
+props[:subject] # as above
+props.subject   # still good
+}}}
+
+Under the hood, there is complete support for named properties:
+{{{
+# to get the categories as set by outlook
+props['Keywords', Msg::Properties::PS_PUBLIC_STRINGS]
+# => ["Business", "Competition", "Favorites"]
+
+# and as a fallback, the symbolic lookup will automatically use named properties,
+# which can be seen:
+props.resolve :keywords
+# => #<Key {00020329-0000-0000-c000-000000000046}/"Keywords">
+
+# which allows this to work:
+props.keywords # as above
+}}}
+
+With some more work, the property storage model should be able to reach feature
+completion.

data/Rakefile

@@ -0,0 +1,66 @@
+require 'rake/rdoctask'
+require 'rake/testtask'
+require 'rake/packagetask'
+require 'rake/gempackagetask'
+
+require 'rbconfig'
+require 'fileutils'
+
+$: << './lib'
+require 'msg.rb'
+
+PKG_NAME = 'ruby-msg'
+PKG_VERSION = Msg::VERSION
+
+task :default => [:test]
+
+Rake::TestTask.new(:test) do |t|
+	t.test_files = FileList["test/test_*.rb"]
+	t.warning = true
+	t.verbose = true
+end
+
+# RDocTask wasn't working for me
+desc 'Build the rdoc HTML Files'
+task :rdoc do
+	system "rdoc -S -N -m Msg -w 2 -t '#{PKG_NAME} documentation' lib"
+end
+
+=begin
+Rake::PackageTask.new(PKG_NAME, PKG_VERSION) do |p|
+	p.need_tar_gz = true
+	p.package_dir = 'build'
+	p.package_files.include("Rakefile", "README")
+	p.package_files.include("contrib/*.c")
+	p.package_files.include("test/test_*.rb", "test/*.doc", "lib/*.rb", "lib/ole/storage.rb")
+end
+=end
+
+spec = Gem::Specification.new do |s|
+	s.name = PKG_NAME
+	s.version = PKG_VERSION
+	s.summary = %q{Ruby Msg library.}
+	s.description = %q{A library for reading Outlook msg files, and for converting them to RFC2822 emails.}
+	s.authors = ["Charles Lowe"]
+	s.email = %q{aquasync@gmail.com}
+	s.homepage = %q{http://code.google.com/p/ruby-msg}
+	#s.rubyforge_project = %q{ruby-msg}
+
+	s.executables = ['msgtool', 'oletool']
+	s.files  = Dir.glob('data/*.yaml') + ['Rakefile', 'README', 'FIXES']
+	s.files += Dir.glob("lib/**/*.rb")
+	s.files += Dir.glob("test/test_*.rb") + Dir.glob("test/*.doc")
+	s.files += Dir.glob("bin/*")
+	
+	s.has_rdoc = true
+
+	s.autorequire = 'msg'
+end
+
+Rake::GemPackageTask.new(spec) do |p|
+	p.gem_spec = spec
+	p.need_tar = true
+	p.need_zip = false
+	p.package_dir = 'build'
+end
+

data/bin/msgtool

@@ -0,0 +1,63 @@
+#! /usr/bin/ruby
+
+require 'optparse'
+require 'rubygems'
+require 'msg'
+require 'time'
+
+def munge_headers mime, opts
+	opts[:header_defaults].each do |s|
+		key, val = s.match(/(.*?):\s+(.*)/)[1..-1]
+		mime.headers[key] = [val] if mime.headers[key].empty?
+	end
+end
+
+def msgtool
+	opts = {:verbose => false, :action => :convert, :header_defaults => []}
+	op = OptionParser.new do |op|
+		op.banner = "Usage: msgtool [options] [files]"
+		op.separator ''
+		op.on('-c', '--convert', 'Convert msg files (default)') { opts[:action] = :convert }
+		op.on('-m', '--convert-mbox', 'Convert msg files for mbox usage') { opts[:action] = :convert_mbox }
+		op.on('-d', '--header-default STR', 'Provide a default value for top level mail header') { |hd| opts[:header_defaults] << hd }
+		op.separator ''
+		op.on('-v', '--[no-]verbose', 'Run verbosely') { |v| opts[:verbose] = v }
+		op.on_tail('-h', '--help', 'Show this message') { puts op; exit }
+	end
+	msgs = op.parse ARGV
+	if msgs.empty?
+		puts 'Must specify 1 or more msg files.'
+		puts op
+		exit 1
+	end
+	# just shut up and convert a message to eml
+	Msg::Log.level = Ole::Log.level = opts[:verbose] ? Logger::WARN : Logger::FATAL
+	case opts[:action]
+	when :convert
+		msgs.each do |filename|
+			msg = Msg.open filename
+			mime = msg.to_mime
+			munge_headers mime, opts
+			puts mime.to_s
+		end
+	when :convert_mbox
+		msgs.each do |filename|
+			msg = Msg.open filename
+			# could use something from the msg in our from line if we wanted
+			puts "From msgtool@ruby-msg #{Time.now.rfc2822}"
+			mime = msg.to_mime
+			munge_headers mime, opts
+			mime.to_s.each do |line|
+				# we do the append > style mbox quoting (mboxrd i think its called), as it
+				# is the only one that can be robuslty un-quoted. evolution doesn't use this!
+				if line =~ /^>*From /o
+					print '>' + line
+				else
+					print line
+				end
+			end
+		end
+	end
+end
+
+msgtool

data/bin/oletool

@@ -0,0 +1,35 @@
+#! /usr/bin/ruby
+
+require 'optparse'
+require 'rubygems'
+require 'ole/storage'
+
+def oletool
+	opts = {:verbose => false, :action => :tree}
+	op = OptionParser.new do |op|
+		op.banner = "Usage: oletool [options] [files]"
+		op.separator ''
+		op.on('-t', '--tree', 'Dump ole trees for files (default)') { opts[:action] = :tree }
+		op.on('-r', '--repack', 'Repack the ole files in canonical form') { opts[:action] = :repack }
+		op.separator ''
+		op.on('-v', '--[no-]verbose', 'Run verbosely') { |v| opts[:verbose] = v }
+		op.on_tail('-h', '--help', 'Show this message') { puts op; exit }
+	end
+	files = op.parse ARGV
+	if files.empty?
+		puts 'Must specify 1 or more msg files.'
+		puts op
+		exit 1
+	end
+	Ole::Log.level = opts[:verbose] ? Logger::WARN : Logger::FATAL
+	files.each do |file|
+		case opts[:action]
+		when :tree
+			Ole::Storage.open(file) { |ole| puts ole.root.to_tree }
+		when :repack
+			Ole::Storage.open(file, &:repack)
+		end
+	end
+end
+
+oletool