sup 0.3 → 0.4

data/HACKING

@@ -1,20 +1,42 @@
-Running Sup locally
--------------------
+Running Sup from your git checkout
+----------------------------------
+
 Invoke it like this:
 
-ruby -I lib -w bin/sup
+  ruby -I lib -w bin/sup
 
 You'll have to install all gems mentioned in the Rakefile (look for the line
 setting p.extra_deps). If you're on a Debian or Debian-based system (e.g.
 Ubuntu), you'll have to make sure you have a complete Ruby installation,
-especially libssl-ruby.
+especially libssl-ruby. You will need libruby-devel, gcc, and make installed
+to build certain gems like Ferret. Gem install does not do a good job of
+detecting when these things are missing and the build fails.
+
+Rubygems also is particularly aggressive about picking up libraries from
+installed gems. If you do have Sup installed as a gem, please examine
+backtraces to make sure you're loading files from the repository and NOT from
+the installed gem before submitting any bug reports.
 
 Coding standards
 ----------------
 
-- Don't wrap code unless it really benefits from it. The days of
-  80-column displays are long over. But do wrap comments and other
-  text at whatever Emacs meta-Q does.
-- I like poetry mode.
+- Don't wrap code unless it really benefits from it.
+- Do wrap comments at 72 characters.
+- Old lisp-style comment differentiations:
+   # one for comments on the same line as a line of code
+   ## two for comments on their own line, except:
+   ### three for comments that demarcate large sections of code (rare)
 - Use {} for one-liner blocks and do/end for multi-line blocks.
-
+- I like poetry mode. Don't use parentheses unless you must.
+- The one exception to poetry mode is if-statements that have an assignment in
+  the condition. To make it clear this is not a comparison, surround the
+  condition by parentheses. E.g.:
+    if a == b                    if(a = some.computation)
+      ...             BUT          ... something with a
+    end                          end
+- and/or versus ||/&&. In Ruby, "and" and "or" bind very loosely---even
+  more loosely than function application. This makes them ideal for
+  end-of-line short-circuit control in poetry mode. So, use || and &&
+  for ordinary logical comparisons, and "and" and "or" for end-of-line
+  flow control. E.g.:
+    x = a || b or raise "neither is true"

data/History.txt

@@ -1,3 +1,10 @@
+== 0.4 / 2008-01-23
+* GPG support for signing and encrypting outgoing mail
+* New hooks: mime attachment, attribution line
+* Improved local charset detection using gettext library
+* Better quoted region detection
+* Many bugfixes and UI improvements
+
 == 0.3 / 2007-10-29
 * In-buffer search (finally!)
 * Subscribe to/unsubscribe from mailing list commands.

data/Manifest.txt

@@ -11,6 +11,7 @@ bin/sup-dump
 bin/sup-recover-sources
 bin/sup-sync
 bin/sup-sync-back
+bin/sup-tweak-labels
 doc/FAQ.txt
 doc/Hooks.txt
 doc/NewUserGuide.txt
@@ -24,6 +25,7 @@ lib/sup/contact.rb
 lib/sup/crypto.rb
 lib/sup/draft.rb
 lib/sup/hook.rb
+lib/sup/horizontal-selector.rb
 lib/sup/imap.rb
 lib/sup/index.rb
 lib/sup/keymap.rb

data/Rakefile

@@ -1,5 +1,3 @@
-# -*- ruby -*-
-
 require 'rubygems'
 require 'hoe'
 $:.unshift 'lib' # force loading from ./lib/ if it exists
@@ -9,7 +7,12 @@ class Hoe
   def extra_deps; @extra_deps.reject { |x| Array(x).first == "hoe" } end
 end # thanks to "Mike H"
 
-Hoe.new('sup', Redwood::VERSION) do |p|
+## allow people who use development versions by running "rake gem"
+## and installing the resulting gem it to be able to do this. (gem
+## versions must be in dotted-digit notation only).
+version = Redwood::VERSION == "git" ? "999" : Redwood::VERSION
+
+Hoe.new('sup', version) do |p|
   p.rubyforge_name = 'sup'
   p.author = "William Morgan"
   p.summary = 'A console-based email client with the best features of GMail, mutt, and emacs. Features full text search, labels, tagged operations, multiple buffers, recent contacts, and more.'
@@ -17,7 +20,7 @@ Hoe.new('sup', Redwood::VERSION) do |p|
   p.url = p.paragraphs_of('README.txt', 0).first.split(/\n/)[2].gsub(/^\s+/, "")
   p.changes = p.paragraphs_of('History.txt', 0..0).join("\n\n")
   p.email = "wmorgan-sup@masanjin.net"
-  p.extra_deps = [['ferret', '>= 0.10.13'], ['ncurses', '>= 0.9.1'], ['rmail', '>= 0.17'], 'highline', 'net-ssh', ['trollop', '>= 1.7'], 'lockfile', 'mime-types']
+  p.extra_deps = [['ferret', '>= 0.10.13'], ['ncurses', '>= 0.9.1'], ['rmail', '>= 0.17'], 'highline', 'net-ssh', ['trollop', '>= 1.7'], 'lockfile', 'mime-types', 'gettext']
 end
 
 rule 'ss?.png' => 'ss?-small.png' do |t|
@@ -45,4 +48,5 @@ task :upload_webpage_images => (SCREENSHOTS + SCREENSHOTS_SMALL) do |t|
   sh "scp -C #{t.prerequisites * ' '} wmorgan@rubyforge.org:/var/www/gforge-projects/sup/"
 end
 
-# vim: syntax=Ruby
+# vim: syntax=ruby
+# -*- ruby -*-

data/bin/sup

@@ -7,6 +7,20 @@ require 'fileutils'
 require 'trollop'
 require "sup"
 
+BIN_VERSION = "0.4"
+
+unless Redwood::VERSION == BIN_VERSION
+  $stderr.puts <<EOS
+
+Error: version mismatch!
+The sup executable is at version #{BIN_VERSION.inspect}.
+The sup libraries are at version #{Redwood::VERSION.inspect}.
+
+Is your development environment conflicting with rubygems?
+EOS
+  exit(-1)
+end
+
 $exceptions = []
 $opts = Trollop::options do
   version "sup v#{Redwood::VERSION}"
@@ -20,6 +34,7 @@ Options are:
 EOS
   opt :list_hooks, "List all hooks and descriptions thereof, and quit."
   opt :no_threads, "Turn of threading. Helps with debugging. (Necessarily disables background polling for new messages.)"
+  opt :no_initial_poll, "Don't poll for new messages when starting."
   opt :search, "Search for threads ", :type => String
 end
 
@@ -154,9 +169,8 @@ begin
           Ncurses::A_BOLD
     c.add :completion_character_color, Ncurses::COLOR_WHITE,
           Ncurses::COLOR_BLACK, Ncurses::A_BOLD
-    c.add :reply_mode_selected_color, Ncurses::COLOR_YELLOW, Ncurses::COLOR_BLACK, Ncurses::A_BOLD
-    c.add :reply_mode_unselected_color, Ncurses::COLOR_CYAN, Ncurses::COLOR_BLACK
-    c.add :reply_mode_label_color, Ncurses::COLOR_CYAN, Ncurses::COLOR_BLACK
+    c.add :horizontal_selector_selected_color, Ncurses::COLOR_YELLOW, Ncurses::COLOR_BLACK, Ncurses::A_BOLD
+    c.add :horizontal_selector_unselected_color, Ncurses::COLOR_CYAN, Ncurses::COLOR_BLACK
     c.add :search_highlight_color, Ncurses::COLOR_BLACK, Ncurses::COLOR_YELLOW, Ncurses::A_BOLD, :highlight => :search_highlight_color
   end
 
@@ -180,9 +194,9 @@ begin
         Redwood::log "fatal error loading from #{s}: #{e.message}"
       end
     end
-  end
+  end unless $opts[:no_initial_poll]
   
-  imode.load_threads :num => ibuf.content_height, :when_done => lambda { reporting_thread("poll after loading inbox") { sleep 1; PollManager.poll } unless $opts[:no_threads] }
+  imode.load_threads :num => ibuf.content_height, :when_done => lambda { reporting_thread("poll after loading inbox") { sleep 1; PollManager.poll } unless $opts[:no_threads] || $opts[:no_initial_poll] }
 
   unless $opts[:no_threads]
     PollManager.start
@@ -199,67 +213,77 @@ begin
     next unless c
     bm.erase_flash
 
-    unless bm.handle_input(c)
-      x = global_keymap.action_for c
-      case x
-      when :quit
-        break if bm.kill_all_buffers_safely
-      when :help
-        curmode = bm.focus_buf.mode
-        bm.spawn_unless_exists("<help for #{curmode.name}>") { HelpMode.new curmode, global_keymap }
-      when :roll_buffers
-        bm.roll_buffers
-      when :roll_buffers_backwards
-        bm.roll_buffers_backwards
-      when :kill_buffer
-        bm.kill_buffer_safely bm.focus_buf
-      when :list_buffers
-        bm.spawn_unless_exists("Buffer List") { BufferListMode.new }
-      when :list_contacts
-        b, new = bm.spawn_unless_exists("Contact List") { ContactListMode.new }
-        b.mode.load_in_background if new
-      when :search
-        query = BufferManager.ask :search, "search all messages: "
-        next unless query && query !~ /^\s*$/
-        SearchResultsMode.spawn_from_query query
-      when :list_labels
-        labels = LabelManager.listable_labels.map { |l| LabelManager.string_for l }
-        user_label = bm.ask_with_completions :label, "Show threads with label (enter for listing): ", labels
-        unless user_label.nil?
-          if user_label.empty?
-            bm.spawn_unless_exists("Label list") { LabelListMode.new } if user_label && user_label.empty?
-          else
-            LabelSearchResultsMode.spawn_nicely user_label
-          end
+    action =
+      begin
+        if bm.handle_input c
+          :nothing
+        else
+          bm.resolve_input_with_keymap c, global_keymap
         end
-      when :compose
-        ComposeMode.spawn_nicely
-      when :poll
-        reporting_thread("user-invoked poll") { PollManager.poll }
-      when :recall_draft
-        case Index.num_results_for :label => :draft
-        when 0
-          bm.flash "No draft messages."
-        when 1
-          m = nil
-          Index.each_id_by_date(:label => :draft) { |mid, builder| m = builder.call }
-          r = ResumeMode.new(m)
-          BufferManager.spawn "Edit message", r
-          r.edit_message
+      rescue InputSequenceAborted
+        :nothing
+      end
+
+    case action
+    when :quit
+      break if bm.kill_all_buffers_safely
+    when :help
+      curmode = bm.focus_buf.mode
+      bm.spawn_unless_exists("<help for #{curmode.name}>") { HelpMode.new curmode, global_keymap }
+    when :roll_buffers
+      bm.roll_buffers
+    when :roll_buffers_backwards
+      bm.roll_buffers_backwards
+    when :kill_buffer
+      bm.kill_buffer_safely bm.focus_buf
+    when :list_buffers
+      bm.spawn_unless_exists("Buffer List") { BufferListMode.new }
+    when :list_contacts
+      b, new = bm.spawn_unless_exists("Contact List") { ContactListMode.new }
+      b.mode.load_in_background if new
+    when :search
+      query = BufferManager.ask :search, "search all messages: "
+      next unless query && query !~ /^\s*$/
+      SearchResultsMode.spawn_from_query query
+    when :list_labels
+      labels = LabelManager.listable_labels.map { |l| LabelManager.string_for l }
+      user_label = bm.ask_with_completions :label, "Show threads with label (enter for listing): ", labels
+      unless user_label.nil?
+        if user_label.empty?
+          bm.spawn_unless_exists("Label list") { LabelListMode.new } if user_label && user_label.empty?
         else
-          b, new = BufferManager.spawn_unless_exists("All drafts") { LabelSearchResultsMode.new [:draft] }
-          b.mode.load_threads :num => b.content_height if new
+          LabelSearchResultsMode.spawn_nicely user_label
         end
-      when :nothing
-      when :redraw
-        bm.completely_redraw_screen
+      end
+    when :compose
+      ComposeMode.spawn_nicely
+    when :poll
+      reporting_thread("user-invoked poll") { PollManager.poll }
+    when :recall_draft
+      case Index.num_results_for :label => :draft
+      when 0
+        bm.flash "No draft messages."
+      when 1
+        m = nil
+        Index.each_id_by_date(:label => :draft) { |mid, builder| m = builder.call }
+        r = ResumeMode.new(m)
+        BufferManager.spawn "Edit message", r
+        r.edit_message
       else
-        bm.flash "Unknown keypress '#{c.to_character}' for #{bm.focus_buf.mode.name}."
+        b, new = BufferManager.spawn_unless_exists("All drafts") { LabelSearchResultsMode.new [:draft] }
+        b.mode.load_threads :num => b.content_height if new
       end
+    when :nothing, InputSequenceAborted
+    when :redraw
+      bm.completely_redraw_screen
+    else
+      bm.flash "Unknown keypress '#{c.to_character}' for #{bm.focus_buf.mode.name}."
     end
 
     bm.draw_screen
   end
+
+  bm.kill_all_buffers if SuicideManager.die?
 rescue Exception => e
   $exceptions << [e, "main"]
 ensure

data/bin/sup-config

@@ -96,7 +96,7 @@ def add_source
         $last_folder = fn
 
         fn = "/#{fn}" # lame
-        if srv =~ /^(\w+):(\d+)$/
+        if srv =~ /^(\S+):(\d+)$/
           host, port = $1, $2.to_i
         else
           host, port = srv, nil

data/bin/sup-sync

@@ -7,6 +7,9 @@ require "sup"
 
 class Float
   def to_s; sprintf '%.2f', self; end
+   def to_time_s
+     infinite? ? "unknown" : super
+   end
 end
 
 class Numeric

data/bin/sup-tweak-labels

@@ -0,0 +1,127 @@
+#!/usr/bin/env ruby
+
+require 'rubygems'
+require 'trollop'
+require "sup"
+
+class Float
+  def to_s; sprintf '%.2f', self; end
+   def to_time_s
+     infinite? ? "unknown" : super
+   end
+end
+
+class Numeric
+  def to_time_s
+    i = to_i
+    sprintf "%d:%02d:%02d", i / 3600, (i / 60) % 60, i % 60
+  end
+end
+
+def time
+  startt = Time.now
+  yield
+  Time.now - startt
+end
+
+opts = Trollop::options do
+  version "sup-tweak-labels (sup #{Redwood::VERSION})"
+  banner <<EOS
+Batch modification of message state for messages already in the index.
+
+Usage:
+  sup-tweak-labels [options] <source>*
+
+where <source>* is zero or more source URIs. Supported source URI schemes can
+be seen by running "sup-add --help".
+
+Options:
+EOS
+  opt :add, "One or more labels (comma-separated) to add to every message from the specified sources", :type => String
+  opt :remove, "One or more labels (comma-separated) to remove from every message from the specified sources, if those labels are present", :type => String
+
+  text <<EOS
+
+Other options:
+EOS
+  opt :verbose, "Print message ids as they're processed."
+  opt :all_sources, "Scan over all sources.", :short => :none
+  opt :dry_run, "Don't actually modify the index. Probably only useful with --verbose.", :short => "-n"
+  opt :version, "Show version information", :short => :none
+end
+
+add_labels = (opts[:add] || "").split(",").map { |l| l.intern }.uniq
+remove_labels = (opts[:remove] || "").split(",").map { |l| l.intern }.uniq
+
+Trollop::die "nothing to do: no labels to add or remove" if add_labels.empty? && remove_labels.empty?
+
+Redwood::start
+begin
+  index = Redwood::Index.new
+  index.load
+
+  source_ids = 
+    if opts[:all_sources]
+      index.sources
+    else
+      ARGV.map do |uri|
+        index.source_for uri or Trollop::die "Unknown source: #{uri}. Did you add it with sup-add first?"
+      end
+  end.map { |s| s.id }
+  Trollop::die "nothing to do: no sources" if source_ids.empty?
+
+  query = "+(" + source_ids.map { |id| "source_id:#{id}" }.join(" ") + ")"
+  if add_labels.empty?
+    ## if all we're doing is removing labels, we can further restrict the
+    ## query to only messages with those labels
+    query += " +(" + remove_labels.map { |l| "label:#{l}" }.join(" ") + ")"
+  end
+
+  results = index.ferret.search query, :limit => :all
+  num_total = results.total_hits
+
+  $stderr.puts "Found #{num_total} documents across #{source_ids.length} sources. Scanning..."
+
+  num_changed = num_scanned = 0
+  last_info_time = start_time = Time.now
+  results.hits.each do |hit|
+    num_scanned += 1
+    id = hit.doc
+
+    m = index.build_message id
+    old_labels = m.labels.clone
+
+    m.labels += add_labels
+    m.labels -= remove_labels
+    m.labels = m.labels.uniq
+
+    unless m.labels.sort_by { |s| s.to_s } == old_labels.sort_by { |s| s.to_s }
+      num_changed += 1
+      puts "#{m.id}: {#{old_labels.join ','}} => {#{m.labels.join ','}}" if opts[:verbose]
+      index.sync_message m unless opts[:dry_run]
+    end
+
+    if Time.now - last_info_time > 60
+      last_info_time = Time.now
+      elapsed = last_info_time - start_time
+      pctdone = 100.0 * num_scanned.to_f / num_total.to_f
+      remaining = (100.0 - pctdone) * (elapsed.to_f / pctdone)
+      $stderr.puts "## #{num_scanned} (#{pctdone}%) read; #{elapsed.to_time_s} elapsed; #{remaining.to_time_s} remaining"
+    end
+  end
+  $stderr.puts "Scanned #{num_scanned} / #{num_total} messages and changed #{num_changed}."
+
+  unless num_changed == 0
+    $stderr.puts "Optimizing index..."
+    index.ferret.optimize unless opts[:dry_run]
+  end
+
+rescue Exception => e
+  File.open("sup-exception-log.txt", "w") { |f| f.puts e.backtrace }
+  raise
+ensure
+  index.save
+  Redwood::finish
+  index.unlock
+end
+

data/doc/TODO

@@ -1,20 +1,10 @@
-for 0.4
+for 0.5
 -------
 _ mark thread as unread should remember the unread messages and mark
   only them as unread, just like gmail
-_ mark thread as unread should have a version within thread-view-mode
-  which then also closes the buffer
 _ bugfix: time zone parsing broken?
-_ forwards optionally include attachments
-_ attach messages
-_ flesh out gpg integration: sign & encrypt outgoing
-_ mbox: don't keep filehandles open, and protect all reads with
-  dotlockfile
-_ imap: share connection to the same server.
-_ bugfix: screwing with the headers when editing causes a crash
 _ need a better way to force an address to a particular name,
   for things like evite addresses
-_ pressing A in thread-view-mode should jump to next message
 _ imap "add all folders on this server" option in sup-add
 _ for new message flashes, add new message counts until keypress
 _ bugfix: missing sources should be handled better
@@ -22,13 +12,29 @@ _ search results: highlight relevant snippets and open to relevant
   portion of thread
 _ have "notes" (treated as emails to oneself, never sent) as
   first-class objects.
+_ make use of the username in URIs, and move account passwords to
+  a different file from sources.yaml. heck, allow encryption of that
+  file.
+
+for 0.4
+-------
+_ bugfix in drafts: the entire thread is currently discarded, rather
+  than just the one message. (need to distinguish per-message and
+  per-thread deletion in the update messages.)
+_ bugfix in completion: capitalization for contact names
+_ imap: cache headers
+_ imap: share connection to the same server.
+_ dispatch-and-kill version of mark thread as unread in thread-view-mode
+_ forward for one or more tagged messages should attach them
+_ mbox: don't keep filehandles open, and protect all reads with dotlockfile
+_ bugfix: screwing with the headers when editing causes a crash
 
 future
 ------
 _ ldbd support
 _ don't use a people.txt; store email addresses directly in the index. too many
   problems with email addresses that occur with multiple names.
-_ infix match instead of prefix match for tab completion (maybe!)
+_ infix match instead of prefix match for tab completion
 _ fix killed threads contributing to unread message count problem (prob. need
   to maintain all killed message ids and our own unread message count for
   inbox).
@@ -77,6 +83,11 @@ x gmail support: obsoleted by imap
 
 done
 ----
+x bugfix: threading broken
+x bugfix: thread ordering in thread-index-mode sometimes jumps around with 'M'
+x forwards optionally include attachments
+x flesh out gpg integration: sign & encrypt outgoing
+x pressing A in thread-view-mode should jump to next message
 x multi-thread dump upon crash
 x hook manager caches values of any proc "variables"
 x bugfix: remove delay on startup if a usual imap source exists