sup 0.13.2.1 → 0.14.0

data/CONTRIBUTORS

@@ -11,13 +11,14 @@ Eric Sherman <hyperbolist at the gmail dot coms>
 Tero Tilus <tero at the tilus dot nets>
 Ben Walton <bwalton at the artsci.utoronto dot cas>
 Mike Stipicevic <stipim at the rpi dot edus>
+Clint Byrum <clint at the ubuntu dot coms>
 Marcus Williams <marcus-sup at the bar-coded dot nets>
 Lionel Ott <white.magic at the gmx dot des>
 Gaudenz Steinlin <gaudenz at the soziologie dot chs>
 Damien Leone <damien.leone at the fensalir dot frs>
 Ingmar Vanhassel <ingmar at the exherbo dot orgs>
 Mark Alexander <marka at the pobox dot coms>
-Eric Weikl <eric.weikl at the tngtech dot coms>
+Eric Weikl <eric.weikl at the gmx dot nets>
 Christopher Warrington <chrisw at the rice dot edus>
 W. Trevor King <wking at the drexel dot edus>
 Richard Brown <rbrown at the exherbo dot orgs>
@@ -26,9 +27,11 @@ Marc Hartstein <marc.hartstein at the alum.vassar dot edus>
 Israel Herraiz <israel.herraiz at the gmail dot coms>
 Bo Borgerson <gigabo at the gmail dot coms>
 Michael Hamann <michael at the content-space dot des>
+Jonathan Lassoff <jof at the thejof dot coms>
 William Erik Baxter <web at the superscript dot coms>
 Grant Hollingworth <grant at the antiflux dot orgs>
 Markus Klinik <markus.klinik at the gmx dot des>
+Ico Doornekamp <ico at the pruts dot nls>
 Adeodato Simó <dato at the net.com.org dot ess>
 Daniel Schoepe <daniel.schoepe at the googlemail dot coms>
 Jason Petsod <jason at the petsod dot orgs>
@@ -44,21 +47,22 @@ Andrew Pimlott <andrew at the pimlott dot nets>
 Jeff Balogh <its.jeff.balogh at the gmail dot coms>
 Matías Aguirre <matiasaguirre at the gmail dot coms>
 Kornilios Kourtis <kkourt at the cslab.ece.ntua dot grs>
-Giorgio Lando <patroclo7 at the gmail dot coms>
 Kevin Riggle <kevinr at the free-dissociation dot coms>
+Giorgio Lando <patroclo7 at the gmail dot coms>
 Benoît PIERRE <benoit.pierre at the gmail dot coms>
+Matthieu Rakotojaona <matthieu.rakotojaona at the gmail dot coms>
 Alvaro Herrera <alvherre at the alvh.no-ip dot orgs>
 Steven Lawrance <stl at the koffein dot nets>
 Jonah <Jonah at the GoodCoffee dot cas>
 ian <itaylor at the uark dot edus>
+Todd Eisenberger <teisenbe at the andrew.cmu dot edus>
 Adam Lloyd <adam at the alloy-d dot nets>
-Gregor Hoffleit <gregor at the sam.mediasupervision dot des>
 MichaelRevell <mikearevell at the gmail dot coms>
-Todd Eisenberger <teisenbe at the andrew.cmu dot edus>
+Per Andersson <avtobiff at the gmail dot coms>
+Gregor Hoffleit <gregor at the sam.mediasupervision dot des>
 Steven Walter <swalter at the monarch.(none)>
 Jon M. Dugan <jdugan at the es dot nets>
-Jonathan Lassoff <jof at the thejof dot coms>
-Matthieu Rakotojaona <matthieu.rakotojaona at the gmail dot coms>
-Stefan Lundström <lundst at the snabb.(none)>
 Matthias Vallentin <vallentin at the icir dot orgs>
+Stefan Lundström <lundst at the snabb.(none)>
+Horacio Sanson <horacio at the skillupjapan.co dot jps>
 Kirill Smelkov <kirr at the landau.phys.spbu dot rus>

data/History.txt

@@ -1,6 +1,12 @@
-== 0.13.2.1 / 2013-10-29
-
-* SBU1: security release
+== 0.14.0 / 2013-08-15
+
+* CJK compatability
+* Psych over Syck
+* Ruby 1.8 deprecated
+* Thread safety
+* No more Iconv, but using built in Ruby encodings. Better UTF-8
+  handling.
+* GPGME 2.0 support
 
 == 0.13.2 / 2013-06-26
 

data/ReleaseNotes

@@ -1,24 +1,21 @@
-Release 0.13.2.1:
+Release 0.14.0:
 
-Security advisory (#SBU1) for Sup
+CJK-compatability, Psych usage, thread safety, GPGME 2.0 support. Sup is now
+Ruby 1.9 based, and apart from RMail - ready for Ruby 2.0.0.
 
-We have been notified of an potential exploit in the somewhat careless
-way Sup treats attachment metadata in received e-mails. The issues
-should now be fixed and I have released Sup 0.13.2.1 and 0.14.1.1 which
-incorporates these fixes. Please upgrade immediately and also ensure
-that your mime-decode or mime-view hooks are secure [0], [1].
+Sup now uses Psych as a YAML parser (default by Ruby) and your previous
+configuration files (~/.sup/*.yaml) may need to be migrated or re-created for
+them to work with the new sup. A migration script is included for this.
 
-This is specifically related to using quotes (',") around filename or
-content_type which is already escaped using Ruby Shellwords.escape -
-this means that the string (content_type, filename) is intended to be
-used _without_ any further quotes. Please make sure that if you use
-.mailcap (non OSX systems), you do not quote the string.
+Check https://github.com/sup-heliotrope/sup/wiki/Migration-0.13-to-0.14 for
+the latest instructions.
 
-Credit goes to: joernchen of Phenoelit (http://phenoelit.de) who
-discovered and suggested fixes for these issues.
+First back up your ~/.sup directory and index, after installing the new sup
+run:
 
-[0] https://github.com/sup-heliotrope/sup/wiki/Viewing-Attachments
-[1] https://github.com/sup-heliotrope/sup/wiki/Secure-usage-of-Sup
+$ sup-psych-ify-config-files
+
+to migrate your files. You should now be all set for buisness.
 
 
 Release 0.13.2:

data/bin/sup

@@ -1,4 +1,5 @@
 #!/usr/bin/env ruby
+# encoding: utf-8
 
 $:.unshift File.join(File.dirname(__FILE__), *%w[.. lib])
 
@@ -8,8 +9,6 @@ require 'ncursesw'
 
 no_gpgme = false
 begin
-  # gpgme broke its API in 2.0, so make sure we have the old version for now.
-  gem 'gpgme', '=1.0.8'
   require 'gpgme'
 rescue LoadError
   no_gpgme = true
@@ -158,7 +157,11 @@ begin
 
   $die = false
   trap("TERM") { |x| $die = true }
-  trap("WINCH") { |x| BufferManager.sigwinch_happened! }
+  trap("WINCH") do |x|
+   ::Thread.new do
+     BufferManager.sigwinch_happened!
+   end
+  end
 
   if(s = Redwood::SourceManager.source_for DraftManager.source_name)
     DraftManager.source = s
@@ -290,8 +293,8 @@ begin
       b.mode.load_in_background if new
     when :search
       completions = LabelManager.all_labels.map { |l| "label:#{LabelManager.string_for l}" }
-      completions = completions.each { |l| l.force_encoding 'UTF-8' if l.methods.include?(:encoding) }
-      completions += ["from:", "to:", "after:", "before:", "date:", "limit:", "AND", "OR", "NOT"]
+      completions = completions.each { |l| l.fix_encoding }
+      completions += Index::COMPL_PREFIXES
       query = BufferManager.ask_many_with_completions :search, "Search all messages (enter for saved searches): ", completions
       unless query.nil?
         if query.empty?
@@ -304,7 +307,7 @@ begin
       SearchResultsMode.spawn_from_query "is:unread"
     when :list_labels
       labels = LabelManager.all_labels.map { |l| LabelManager.string_for l }
-      labels = labels.each { |l| l.force_encoding 'UTF-8' if l.methods.include?(:encoding) }
+      labels = labels.each { |l| l.fix_encoding }
 
       user_label = bm.ask_with_completions :label, "Show threads with label (enter for listing): ", labels
       unless user_label.nil?
@@ -412,7 +415,7 @@ unless Redwood::exceptions.empty?
 We are very sorry. It seems that an error occurred in Sup. Please
 accept our sincere apologies. Please submit the contents of
 #{BASE_DIR}/exception-log.txt and a brief report of the
-circumstances to https://github.com/sup-heliotrope/sup/issues so that 
+circumstances to https://github.com/sup-heliotrope/sup/issues so that
 we might address this problem. Thank you!
 
 Sincerely,

data/bin/sup-psych-ify-config-files

@@ -0,0 +1,16 @@
+#!/usr/bin/env ruby
+
+$:.unshift File.join(File.dirname(__FILE__), *%w[.. lib])
+
+require "sup"
+require "fileutils"
+
+Redwood.start
+
+fn = Redwood::SOURCE_FN
+FileUtils.cp fn, "#{fn}.syck_bak"
+
+Redwood::SourceManager.load_sources fn
+Redwood::SourceManager.save_sources fn, true
+
+Redwood.finish

data/lib/sup.rb

@@ -1,11 +1,8 @@
-require 'rubygems'
+# encoding: utf-8
 
-require 'syck'
+require 'rubygems'
 require 'yaml'
-if YAML.const_defined? :ENGINE
-  YAML::ENGINE.yamler = 'syck'
-end
-
+YAML::ENGINE.yamler = 'psych'
 require 'zlib'
 require 'thread'
 require 'fileutils'
@@ -28,18 +25,23 @@ end
 class Module
   def yaml_properties *props
     props = props.map { |p| p.to_s }
-    vars = props.map { |p| "@#{p}" }
-    klass = self
-    path = klass.name.gsub(/::/, "/")
 
-    klass.instance_eval do
-      define_method(:to_yaml_properties) { vars }
-      define_method(:to_yaml_type) { "!#{Redwood::YAML_DOMAIN},#{Redwood::YAML_DATE}/#{path}" }
+    path = name.gsub(/::/, "/")
+    yaml_tag "!#{Redwood::YAML_DOMAIN},#{Redwood::YAML_DATE}/#{path}"
+
+    define_method :init_with do |coder|
+      initialize(*coder.map.values_at(*props))
     end
 
-    YAML.add_domain_type("#{Redwood::YAML_DOMAIN},#{Redwood::YAML_DATE}", path) do |type, val|
-      klass.new(*props.map { |p| val[p] })
+    define_method :encode_with do |coder|
+      coder.map = props.inject({}) do |hash, key|
+        hash[key] = instance_variable_get("@#{key}")
+        hash
+      end
     end
+
+    # Legacy
+    Psych.load_tags["!#{Redwood::LEGACY_YAML_DOMAIN},#{Redwood::YAML_DATE}/#{path}"] = self
   end
 end
 
@@ -58,7 +60,8 @@ module Redwood
   SEARCH_FN  = File.join(BASE_DIR, "searches.txt")
   LOG_FN     = File.join(BASE_DIR, "log")
 
-  YAML_DOMAIN = "masanjin.net"
+  YAML_DOMAIN = "supmua.org"
+  LEGACY_YAML_DOMAIN = "masanjin.net"
   YAML_DATE = "2006-10-01"
 
   ## record exceptions thrown in threads nicely
@@ -268,7 +271,7 @@ EOM
     else
       require 'etc'
       require 'socket'
-      name = Etc.getpwnam(ENV["USER"]).gecos.split(/,/).first rescue nil
+      name = Etc.getpwnam(ENV["USER"]).gecos.split(/,/).first.force_encoding($encoding).fix_encoding rescue nil
       name ||= ENV["USER"]
       email = ENV["USER"] + "@" +
         begin
@@ -280,8 +283,8 @@ EOM
       config = {
         :accounts => {
           :default => {
-            :name => name,
-            :email => email,
+            :name => name.fix_encoding,
+            :email => email.fix_encoding,
             :alternates => [],
             :sendmail => "/usr/sbin/sendmail -oem -ti",
             :signature => File.join(ENV["HOME"], ".signature"),
@@ -314,6 +317,7 @@ require "sup/logger/singleton"
 ## determine encoding and character set
 $encoding = Locale.current.charset
 $encoding = "UTF-8" if $encoding == "utf8"
+$encoding = "UTF-8" if $encoding == "UTF8"
 if $encoding
   debug "using character set encoding #{$encoding.inspect}"
 else
@@ -321,6 +325,16 @@ else
   $encoding = "UTF-8"
 end
 
+# test encoding
+teststr = "test"
+teststr.encode('UTF-8')
+begin
+  teststr.encode($encoding)
+rescue Encoding::ConverterNotFoundError
+  warn "locale encoding is invalid, defaulting to utf-8"
+  $encoding = "UTF-8"
+end
+
 require "sup/buffer"
 require "sup/keymap"
 require "sup/mode"

data/lib/sup/account.rb

@@ -50,8 +50,9 @@ class AccountManager
       [:name, :sendmail, :signature, :gpgkey].each { |k| hash[k] ||= @default_account.send(k) }
     end
     hash[:alternates] ||= []
+    fail "alternative emails are not an array: #{hash[:alternates]}" unless hash[:alternates].kind_of? Array
 
-    [:name, :signature].each { |x| hash[x].force_encoding Encoding::UTF_8 if hash[x].respond_to? :encoding }
+    [:name, :signature].each { |x| hash[x] ? hash[x].fix_encoding : nil }
 
     a = Account.new hash
     @accounts[a] = true

data/lib/sup/buffer.rb

@@ -1,3 +1,5 @@
+# encoding: utf-8
+
 require 'etc'
 require 'thread'
 
@@ -126,14 +128,11 @@ class Buffer
     @w.attrset Colormap.color_for(opts[:color] || :none, opts[:highlight])
     s ||= ""
     maxl = @width - x # maximum display width width
-    stringl = maxl    # string "length"
 
     # fill up the line with blanks to overwrite old screen contents
     @w.mvaddstr y, x, " " * maxl unless opts[:no_fill]
 
-    ## the next horribleness is thanks to ruby's lack of widechar support
-    stringl += 1 while stringl < s.length && s[0 ... stringl].display_length < maxl
-    @w.mvaddstr y, x, s[0 ... stringl]
+    @w.mvaddstr y, x, s.slice_by_display_length(maxl)
   end
 
   def clear
@@ -450,7 +449,7 @@ EOS
 
   def ask_with_completions domain, question, completions, default=nil
     ask domain, question, default do |s|
-      s.force_encoding 'UTF-8' if s.methods.include?(:encoding)
+      s.fix_encoding
       completions.select { |x| x =~ /^#{Regexp::escape s}/iu }.map { |x| [x, x] }
     end
   end
@@ -467,9 +466,9 @@ EOS
           raise "william screwed up completion: #{partial.inspect}"
         end
 
-      prefix.force_encoding 'UTF-8' if prefix.methods.include?(:encoding)
-      target.force_encoding 'UTF-8' if target.methods.include?(:encoding)
-      completions.select { |x| x =~ /^#{Regexp::escape target}/i }.map { |x| [prefix + x, x] }
+      prefix.fix_encoding
+      target.fix_encoding
+      completions.select { |x| x =~ /^#{Regexp::escape target}/iu }.map { |x| [prefix + x, x] }
     end
   end
 
@@ -477,12 +476,12 @@ EOS
     ask domain, question, default do |partial|
       prefix, target = partial.split_on_commas_with_remainder
       target ||= prefix.pop || ""
-      target.force_encoding 'UTF-8' if target.methods.include?(:encoding)
+      target.fix_encoding
 
       prefix = prefix.join(", ") + (prefix.empty? ? "" : ", ")
-      prefix.force_encoding 'UTF-8' if prefix.methods.include?(:encoding)
+      prefix.fix_encoding
 
-      completions.select { |x| x =~ /^#{Regexp::escape target}/i }.sort_by { |c| [ContactManager.contact_for(c) ? 0 : 1, c] }.map { |x| [prefix + x, x] }
+      completions.select { |x| x =~ /^#{Regexp::escape target}/iu }.sort_by { |c| [ContactManager.contact_for(c) ? 0 : 1, c] }.map { |x| [prefix + x, x] }
     end
   end
 
@@ -495,7 +494,7 @@ EOS
         if dir
           [[s.sub(full, dir), "~#{name}"]]
         else
-          users.select { |u| u =~ /^#{Regexp::escape name}/ }.map do |u|
+          users.select { |u| u =~ /^#{Regexp::escape name}/u }.map do |u|
             [s.sub("~#{name}", "~#{u}"), "~#{u}"]
           end
         end
@@ -554,6 +553,7 @@ EOS
 
     completions = (recent + contacts).flatten.uniq
     completions += HookManager.run("extra-contact-addresses") || []
+
     answer = BufferManager.ask_many_emails_with_completions domain, question, completions, default
 
     if answer
@@ -622,7 +622,7 @@ EOS
       tf.deactivate
       draw_screen :sync => false, :status => status, :title => title
     end
-    tf.value.tap { |x| x.force_encoding Encoding::UTF_8 if x && x.respond_to?(:encoding) }
+    tf.value.tap { |x| x }
   end
 
   def ask_getch question, accept=nil

data/lib/sup/contact.rb

@@ -1,3 +1,5 @@
+# encoding: utf-8
+
 module Redwood
 
 class ContactManager
@@ -54,7 +56,7 @@ class ContactManager
   def is_aliased_contact? person; !@p2a[person].nil? end
 
   def save
-    File.open(@fn, "w") do |f|
+    File.open(@fn, "w:UTF-8") do |f|
       @p2a.sort_by { |(p, a)| [p.full_address, a] }.each do |(p, a)|
         f.puts "#{a || ''}: #{p.full_address}"
       end

data/lib/sup/crypto.rb

@@ -1,6 +1,4 @@
 begin
-  # gpgme broke its API in 2.0, so make sure we have the old version for now.
-  gem 'gpgme', '=1.0.8'
   require 'gpgme'
 rescue LoadError
 end
@@ -64,28 +62,39 @@ EOS
     @gpgme_present =
       begin
         begin
-          GPGME.check_version({:protocol => GPGME::PROTOCOL_OpenPGP})
+          begin
+            GPGME.check_version({:protocol => GPGME::PROTOCOL_OpenPGP})
+          rescue TypeError
+            GPGME.check_version(nil)
+          end
           true
         rescue GPGME::Error
           false
+        rescue ArgumentError
+          # gpgme 2.0.0 raises this due to the hash->string conversion
+          false
         end
       rescue NameError
         false
       end
 
     unless @gpgme_present
-      @not_working_reason = ['gpgme gem not present', 
+      @not_working_reason = ['gpgme gem not present',
         'Install the gpgme gem in order to use signed and encrypted emails']
       return
     end
 
     # if gpg2 is available, it will start gpg-agent if required
     if (bin = `which gpg2`.chomp) =~ /\S/
-      GPGME.set_engine_info GPGME::PROTOCOL_OpenPGP, bin, nil
+      if GPGME.respond_to?('set_engine_info')
+        GPGME.set_engine_info GPGME::PROTOCOL_OpenPGP, bin, nil
+      else
+        GPGME.gpgme_set_engine_info GPGME::PROTOCOL_OpenPGP, bin, nil
+      end
     else
       # check if the gpg-options hook uses the passphrase_callback
       # if it doesn't then check if gpg agent is present
-      gpg_opts = HookManager.run("gpg-options", 
+      gpg_opts = HookManager.run("gpg-options",
                                {:operation => "sign", :options => {}}) || {}
       if gpg_opts[:passphrase_callback].nil?
         if ENV['GPG_AGENT_INFO'].nil?
@@ -110,22 +119,29 @@ EOS
   end
 
   def have_crypto?; @not_working_reason.nil? end
+  def not_working_reason; @not_working_reason end
 
   def sign from, to, payload
     return unknown_status(@not_working_reason) unless @not_working_reason.nil?
 
     gpg_opts = {:protocol => GPGME::PROTOCOL_OpenPGP, :armor => true, :textmode => true}
     gpg_opts.merge!(gen_sign_user_opts(from))
-    gpg_opts = HookManager.run("gpg-options", 
+    gpg_opts = HookManager.run("gpg-options",
                                {:operation => "sign", :options => gpg_opts}) || gpg_opts
 
     begin
-      sig = GPGME.detach_sign(format_payload(payload), gpg_opts)
+      if GPGME.respond_to?('detach_sign')
+        sig = GPGME.detach_sign(format_payload(payload), gpg_opts)
+      else
+        crypto = GPGME::Crypto.new
+        gpg_opts[:mode] = GPGME::SIG_MODE_DETACH
+        sig = crypto.sign(format_payload(payload), gpg_opts).read
+      end
     rescue GPGME::Error => exc
       raise Error, gpgme_exc_msg(exc.message)
     end
 
-    # if the key (or gpg-agent) is not available GPGME does not complain 
+    # if the key (or gpg-agent) is not available GPGME does not complain
     # but just returns a zero length string. Let's catch that
     if sig.length == 0
       raise Error, gpgme_exc_msg("GPG failed to generate signature: check that gpg-agent is running and your key is available.")
@@ -145,7 +161,7 @@ EOS
 
     gpg_opts = {:protocol => GPGME::PROTOCOL_OpenPGP, :armor => true, :textmode => true}
     if sign
-      gpg_opts.merge!(gen_sign_user_opts(from)) 
+      gpg_opts.merge!(gen_sign_user_opts(from))
       gpg_opts.merge!({:sign => true})
     end
     gpg_opts = HookManager.run("gpg-options",
@@ -153,12 +169,18 @@ EOS
     recipients = to + [from]
     recipients = HookManager.run("gpg-expand-keys", { :recipients => recipients }) || recipients
     begin
-      cipher = GPGME.encrypt(recipients, format_payload(payload), gpg_opts)
+      if GPGME.respond_to?('encrypt')
+        cipher = GPGME.encrypt(recipients, format_payload(payload), gpg_opts)
+      else
+        crypto = GPGME::Crypto.new
+        gpg_opts[:recipients] = recipients
+        cipher = crypto.encrypt(format_payload(payload), gpg_opts).read
+      end
     rescue GPGME::Error => exc
       raise Error, gpgme_exc_msg(exc.message)
     end
 
-    # if the key (or gpg-agent) is not available GPGME does not complain 
+    # if the key (or gpg-agent) is not available GPGME does not complain
     # but just returns a zero length string. Let's catch that
     if cipher.length == 0
       raise Error, gpgme_exc_msg("GPG failed to generate cipher text: check that gpg-agent is running and your key is available.")
@@ -262,7 +284,11 @@ EOS
                                {:operation => "decrypt", :options => gpg_opts}) || gpg_opts
     ctx = GPGME::Ctx.new(gpg_opts)
     cipher_data = GPGME::Data.from_str(format_payload(payload))
-    plain_data = GPGME::Data.empty
+    if GPGME::Data.respond_to?('empty')
+      plain_data = GPGME::Data.empty
+    else
+      plain_data = GPGME::Data.empty!
+    end
     begin
       ctx.decrypt_verify(cipher_data, plain_data)
     rescue GPGME::Error => exc
@@ -275,7 +301,7 @@ EOS
     end
     plain_data.seek(0, IO::SEEK_SET)
     output = plain_data.read
-    output.force_encoding Encoding::ASCII_8BIT if output.respond_to? :force_encoding
+    output.transcode(Encoding::ASCII_8BIT, output.encoding)
 
     ## TODO: test to see if it is still necessary to do a 2nd run if verify
     ## fails.
@@ -290,7 +316,7 @@ EOS
       # Look for Charset, they are put before the base64 crypted part
       charsets = payload.body.split("\n").grep(/^Charset:/)
       if !charsets.empty? and charsets[0] =~ /^Charset: (.+)$/
-        output = Iconv.easy_decode($encoding, $1, output)
+        output.transcode($encoding, $1)
       end
       msg.body = output
     else
@@ -314,7 +340,7 @@ EOS
       msg = RMail::Parser.read output
       if msg.header.content_type =~ %r{^multipart/} && !msg.multipart?
         output = "MIME-Version: 1.0\n" + output
-        output.force_encoding Encoding::ASCII_8BIT if output.respond_to? :force_encoding
+        output.fix_encoding
         msg = RMail::Parser.read output
       end
     end
@@ -330,7 +356,7 @@ private
 
   def gpgme_exc_msg msg
     err_msg = "Exception in GPGME call: #{msg}"
-    info err_msg
+    #info err_msg
     err_msg
   end
 
@@ -362,7 +388,7 @@ private
       else
         first_sig = "Unknown error or empty signature"
       end
-    rescue EOFError 
+    rescue EOFError
       from_key = nil
       first_sig = "No public key available for #{signature.fingerprint}"
     end