sup 0.1 → 0.2

data/History.txt

@@ -1,3 +1,12 @@
+== 0.2 / 2007-10-29
+* Complete hook system for user-inserted code.
+* GPG signature verification and decryption.
+* Automatically scold users who top-post.
+* Automatically warn when sending a message with words like
+  "attachment" in the body if there aren't actually any attachments to
+  the message.
+* Millions of bugfixes.
+
 == 0.1 / 2007-07-17
 * MIME attachment creation.
 * i18n support: character set conversion and rfc2047 header decoding.

data/Manifest.txt

@@ -12,15 +12,18 @@ bin/sup-recover-sources
 bin/sup-sync
 bin/sup-sync-back
 doc/FAQ.txt
+doc/Hooks.txt
+doc/NewUserGuide.txt
 doc/Philosophy.txt
 doc/TODO
-doc/UserGuide.txt
 lib/sup.rb
 lib/sup/account.rb
 lib/sup/buffer.rb
 lib/sup/colormap.rb
 lib/sup/contact.rb
+lib/sup/crypto.rb
 lib/sup/draft.rb
+lib/sup/hook.rb
 lib/sup/imap.rb
 lib/sup/index.rb
 lib/sup/keymap.rb
@@ -31,6 +34,7 @@ lib/sup/mbox.rb
 lib/sup/mbox/loader.rb
 lib/sup/mbox/ssh-file.rb
 lib/sup/mbox/ssh-loader.rb
+lib/sup/message-chunks.rb
 lib/sup/message.rb
 lib/sup/mode.rb
 lib/sup/modes/buffer-list-mode.rb

data/Rakefile

@@ -2,6 +2,7 @@
 
 require 'rubygems'
 require 'hoe'
+$:.unshift 'lib' # force loading from ./lib/ if it exists
 require 'sup'
 
 class Hoe
@@ -23,7 +24,7 @@ rule 'ss?.png' => 'ss?-small.png' do |t|
 end
 
 ## is there really no way to make a rule for this?
-WWW_FILES = %w(www/index.html README.txt doc/Philosophy.txt doc/FAQ.txt doc/UserGuide.txt www/main.css)
+WWW_FILES = %w(www/index.html README.txt doc/Philosophy.txt doc/FAQ.txt doc/NewUserGuide.txt www/main.css)
 
 SCREENSHOTS = FileList["www/ss?.png"]
 SCREENSHOTS_SMALL = []

data/bin/sup

@@ -2,6 +2,7 @@
 
 require 'rubygems'
 require 'ncurses'
+require 'curses'
 require 'fileutils'
 require 'trollop'
 require "sup"
@@ -16,9 +17,15 @@ Usage:
 
 Options are:
 EOS
+  opt :list_hooks, "List all hooks and descriptions thereof, and quit."
   opt :no_threads, "Turn of threading. Helps with debugging. (Necessarily disables background polling for new messages.)"
 end
 
+if $opts[:list_hooks]
+  Redwood::HookManager.print_hooks
+  exit
+end
+
 Thread.abort_on_exception = true # make debugging possible
 
 module Redwood
@@ -85,6 +92,7 @@ EOS
 end
 
 begin
+  extend CanSpawnComposeMode
   Redwood::start
   Index.load
 
@@ -120,6 +128,10 @@ begin
     c.add :alternate_patina_color, Ncurses::COLOR_BLACK, Ncurses::COLOR_BLUE
     c.add :missing_message_color, Ncurses::COLOR_BLACK, Ncurses::COLOR_RED
     c.add :attachment_color, Ncurses::COLOR_CYAN, Ncurses::COLOR_BLACK
+    c.add :cryptosig_valid_color, Ncurses::COLOR_YELLOW, Ncurses::COLOR_BLACK, Ncurses::A_BOLD
+    c.add :cryptosig_unknown_color, Ncurses::COLOR_CYAN, Ncurses::COLOR_BLACK
+    c.add :cryptosig_invalid_color, Ncurses::COLOR_YELLOW, Ncurses::COLOR_RED, Ncurses::A_BOLD
+    c.add :generic_notice_patina_color, Ncurses::COLOR_CYAN, Ncurses::COLOR_BLACK
     c.add :quote_patina_color, Ncurses::COLOR_YELLOW, Ncurses::COLOR_BLACK
     c.add :sig_patina_color, Ncurses::COLOR_YELLOW, Ncurses::COLOR_BLACK
     c.add :quote_color, Ncurses::COLOR_YELLOW, Ncurses::COLOR_BLACK
@@ -139,6 +151,9 @@ begin
           Ncurses::A_BOLD
     c.add :completion_character_color, Ncurses::COLOR_WHITE,
           Ncurses::COLOR_BLACK, Ncurses::A_BOLD
+    c.add :reply_mode_selected_color, Ncurses::COLOR_YELLOW, Ncurses::COLOR_BLACK, Ncurses::A_BOLD
+    c.add :reply_mode_unselected_color, Ncurses::COLOR_CYAN, Ncurses::COLOR_BLACK
+    c.add :reply_mode_label_color, Ncurses::COLOR_CYAN, Ncurses::COLOR_BLACK
   end
 
   log "initializing buffer manager"
@@ -217,28 +232,30 @@ begin
           bm.flash "Couldn't parse query."
         end
       when :list_labels
-        labels = LabelManager.listable_label_strings
+        labels = LabelManager.listable_labels.map { |l| LabelManager.string_for l }
         user_label = bm.ask_with_completions :label, "Show threads with label (enter for listing): ", labels
-        user_label = bm.spawn_modal("Label list", LabelListMode.new) if user_label && user_label.empty?
+        user_label =
+          case user_label
+          when nil, /^\s*$/
+            bm.spawn_modal("Label list", LabelListMode.new) if user_label && user_label.empty?
+          else
+            LabelManager.label_for user_label
+          end
         
-        label = LabelManager.label_for user_label if user_label
-        case label
+        case user_label
         when nil
         when :inbox
           BufferManager.raise_to_front InboxMode.instance.buffer
         else
           b = BufferManager.spawn_unless_exists("All threads with label '#{user_label}'") do
-            mode = LabelSearchResultsMode.new([label])
+            mode = LabelSearchResultsMode.new([user_label])
           end
           b.mode.load_threads :num => b.content_height
         end
 
       when :compose
-        mode = ComposeMode.new
-        bm.spawn "New Message", mode
-        mode.edit
+        spawn_compose_mode
       when :poll
-        #          bm.raise_to_front PollManager.buffer
         reporting_thread { PollManager.poll }
       when :recall_draft
         case Index.num_results_for :label => :draft
@@ -249,7 +266,7 @@ begin
           Index.each_id_by_date(:label => :draft) { |mid, builder| m = builder.call }
           r = ResumeMode.new(m)
           BufferManager.spawn "Edit message", r
-          r.edit
+          r.edit_message
         else
           b = BufferManager.spawn_unless_exists("All drafts") do
             mode = LabelSearchResultsMode.new [:draft]

data/bin/sup-add

@@ -93,7 +93,6 @@ begin
     end
 
     parsed_uri = URI(uri)
-    Trollop::die "no path component to uri: #{parsed_uri}" unless parsed_uri.path
 
     source = 
       case parsed_uri.scheme
@@ -109,6 +108,8 @@ begin
         Redwood::Maildir.new uri, nil, !$opts[:unusual], $opts[:archive], nil, labels
       when "mbox"
         Redwood::MBox::Loader.new uri, nil, !$opts[:unusual], $opts[:archive], nil, labels
+      when nil
+        Trollop::die "Sources must be specified with an URI"
       else
         Trollop::die "Unknown source type #{parsed_uri.scheme.inspect}"      
       end

data/bin/sup-sync-back

@@ -8,14 +8,17 @@ require "sup"
 
 ## save a message 'm' to an open file pointer 'fp'
 def save m, fp
-  m.source.each_raw_full_message_line(m.source_info) { |l| fp.print l }
+  m.source.each_raw_message_line(m.source_info) { |l| fp.print l }
+end
+def die msg
+  $stderr.puts "Error: #{msg}"
+  exit(-1)
 end
 
 opts = Trollop::options do
   version "sup-sync-back (sup #{Redwood::VERSION})"
   banner <<EOS
-Partially synchronizes a message source with the Sup index by deleting
-or moving any messages from the source that are marked as deleted or
+Drop or move messages from Sup sources that are marked as deleted or
 spam in the Sup index.
 
 Currently only works with mbox sources.
@@ -26,20 +29,30 @@ Usage:
 where <source>* is zero or more source URIs. If no sources are given,
 sync back all usual sources.
 
-You probably want to run sup-sync --changed after this command.
+You almost certainly want to run sup-sync --changed after this command.
+Running this does not change the index.
 
 Options include:
 EOS
-  opt :delete_deleted, "Delete deleted messages.", :default => false, :short => "d"
+  opt :drop_deleted, "Drop deleted messages.", :default => false, :short => "d"
   opt :move_deleted, "Move deleted messages to a local mbox file.", :type => String, :short => :none
-  opt :delete_spam, "Delete spam messages.", :default => false, :short => "s"
+  opt :drop_spam, "Drop spam messages.", :default => false, :short => "s"
   opt :move_spam, "Move spam messages to a local mbox file.", :type => String, :short => :none
+
+  opt :with_dotlockfile, "Specific dotlockfile location (mbox files only).", :default => "/usr/bin/dotlockfile", :short => :none
+  opt :dont_use_dotlockfile, "Don't use dotlockfile to lock mbox files. Dangerous if other processes modify them concurrently.", :default => false, :short => :none
+
   opt :verbose, "Print message ids as they're processed."
   opt :dry_run, "Don't actually modify the index. Probably only useful with --verbose.", :short => "-n"
   opt :version, "Show version information", :short => :none
 
-  conflicts :delete_deleted, :move_deleted
-  conflicts :delete_spam, :move_spam
+  conflicts :drop_deleted, :move_deleted
+  conflicts :drop_spam, :move_spam
+end
+
+unless opts[:drop_deleted] || opts[:move_deleted] || opts[:drop_spam] || opts[:move_spam]
+  puts "Nothing to do."
+  exit
 end
 
 Redwood::start
@@ -52,12 +65,14 @@ unless opts[:dry_run]
   spam_fp = File.open(opts[:move_spam], "a") if opts[:move_spam]
 end
 
+dotlockfile = opts[:with_dotlockfile] || "/usr/bin/dotlockfile"
+
 begin
   index.load
 
   sources = ARGV.map do |uri|
-    s = index.source_for(uri) or Trollop::die "Unknown source: #{uri}. Did you add it with sup-add first?"
-    s.is_a?(Redwood::MBox::Loader) or Trollop::die "#{uri} is not an mbox source."
+    s = index.source_for(uri) or die "unknown source: #{uri}. Did you add it with sup-add first?"
+    s.is_a?(Redwood::MBox::Loader) or die "#{uri} is not an mbox source."
     s
   end
 
@@ -65,17 +80,25 @@ begin
     sources = index.usual_sources.select { |s| s.is_a? Redwood::MBox::Loader } 
   end
 
+  unless sources.all? { |s| s.file_path.nil? } || File.executable?(dotlockfile) || opts[:dont_use_dotlockfile]
+    die <<EOS
+can't execute dotlockfile binary: #{dotlockfile}. Specify --with-dotlockfile
+if it's in a nonstandard location, or, if you want to live dangerously, try
+--dont-use-dotlockfile
+EOS
+  end
+
   modified_sources = []
   sources.each do |source|
     $stderr.puts "Scanning #{source}..."
 
-    unless ((opts[:delete_deleted] || opts[:move_deleted]) && index.has_any_from_source_with_label?(source, :deleted)) || ((opts[:delete_spam] || opts[:move_spam]) && index.has_any_from_source_with_label?(source, :spam))
+    unless ((opts[:drop_deleted] || opts[:move_deleted]) && index.has_any_from_source_with_label?(source, :deleted)) || ((opts[:drop_spam] || opts[:move_spam]) && index.has_any_from_source_with_label?(source, :spam))
       $stderr.puts "Nothing to do from this source; skipping"
       next
     end
 
     source.reset!
-    num_deleted = num_moved = num_scanned = 0
+    num_dropped = num_moved = num_scanned = 0
     
     out_fp = Tempfile.new "sup-sync-back-#{source.id}"
     Redwood::PollManager.add_messages_from source do |m, offset, entry|
@@ -85,9 +108,9 @@ begin
         labels = entry[:label].split.map { |x| x.intern }.to_boolean_h
 
         if labels.member? :deleted
-          if opts[:delete_deleted]
+          if opts[:drop_deleted]
             puts "Dropping deleted message #{source}##{offset}" if opts[:verbose]
-            num_deleted += 1
+            num_dropped += 1
           elsif opts[:move_deleted] && labels.member?(:deleted)
             puts "Moving deleted message #{source}##{offset}" if opts[:verbose]
             save m, deleted_fp unless opts[:dry_run]
@@ -95,9 +118,9 @@ begin
           end
 
         elsif labels.member? :spam
-          if opts[:delete_spam]
-            puts "Deleting spam message #{source}##{offset}" if opts[:verbose]
-            num_deleted += 1
+          if opts[:drop_spam]
+            puts "Dropping spam message #{source}##{offset}" if opts[:verbose]
+            num_dropped += 1
           elsif opts[:move_spam] && labels.member?(:spam)
             puts "Moving spam message #{source}##{offset}" if opts[:verbose]
             save m, spam_fp unless opts[:dry_run]
@@ -112,15 +135,21 @@ begin
 
       nil # don't actually add anything!
     end
-    $stderr.puts "Scanned #{num_scanned}, deleted #{num_deleted}, moved #{num_moved} messages from #{source}."
-    modified_sources << source if num_deleted > 0 || num_moved > 0
+    $stderr.puts "Scanned #{num_scanned}, dropped #{num_dropped}, moved #{num_moved} messages from #{source}."
+    modified_sources << source if num_dropped > 0 || num_moved > 0
     out_fp.close unless opts[:dry_run]
 
-    unless opts[:dry_run] || (num_deleted == 0 && num_moved == 0)
+    unless opts[:dry_run] || (num_dropped == 0 && num_moved == 0)
       deleted_fp.flush if deleted_fp
       spam_fp.flush if spam_fp
-      $stderr.puts "Moving #{out_fp.path} to #{source.file_path}"
-      FileUtils.mv out_fp.path, source.file_path
+      unless opts[:dont_use_dotlockfile]
+        puts "Locking #{source.file_path}..."
+        system "#{opts[:dotlockfile]} -l #{source.file_path}"
+        puts "Writing #{source.file_path}..."
+        FileUtils.cp out_fp.path, source.file_path
+        puts "Unlocking #{source.file_path}..."
+        system "#{opts[:dotlockfile]} -u #{source.file_path}"
+      end
     end
   end
 
@@ -137,7 +166,6 @@ rescue Exception => e
   File.open("sup-exception-log.txt", "w") { |f| f.puts e.backtrace }
   raise
 ensure
-  index.save
   Redwood::finish
   index.unlock
 end

data/doc/FAQ.txt

@@ -4,21 +4,22 @@ Sup FAQ
 Q: What is Sup?
 A: Sup is a console-based email client for people with a lot of email.
    It supports tagging, very fast full-text search, automatic contact-
-   list management, and more. If you're the type of person who treats
-   email as an extension of your long-term memory, Sup is for you.
+   list management, custom code insertion via a hook system, and more.
+   If you're the type of person who treats email as an extension of your
+   long-term memory, Sup is for you.
 
 Q: What does Sup stand for?
 A: "What's up?"
 
-Q: Sup looks a lot like a text-based Gmail.
-A: I stole a lot of their ideas. And improved upon them, of course.
+Q: Sup looks like a text-based Gmail.
+A: I stole their ideas. And improved them!
 
-Q: If you love Gmail so much, why not just use it?
-A: I hate ads, I hate using a mouse, and I hate non-programmability
-   and non-extensibility.
+Q: Why not just use Gmail?
+A: I wrote Sup because I hate ads, I hate using a mouse, and I hate
+   non-programmability and non-extensibility.
 
    Also, Gmail doesn't let you use a monospace font, which is just
-   plain lame.
+   lame.
 
    Also, Gmail encourages top-posting. THIS CANNOT BE TOLERATED!
 
@@ -43,17 +44,16 @@ Q: But I want to delete it for real, not just add a 'deleted' flag in
 A: Currently, for mbox sources, there is a batch deletion tool that
    will strip out all messages marked as spam or deleted.
 
-Q: I got some error message about needing to run sup-sync --changed
-   when I tried to read a message. What's that about?
-A: If messages have been moved, deleted, or altered in a source, Sup
-   may have to rebuild its index for that source. For example, for
-   mbox files, reading a single unread message changes the offsets of
-   every file on disk. Rather than rescanning every time, Sup assumes
-   sources don't change except by having new messages added. If that
-   assumption is violated, you'll have to sync the index.
+Q: How well does Sup play with other mail clients?
+A: Not well at all. If messages have been moved, deleted, or altered
+   due to some other client, Sup will have to rebuild its index for
+   that message source. For example, for mbox files, reading a single
+   unread message changes the offsets of every file on disk. Rather
+   than rescanning every time, Sup assumes sources don't change except
+   by having new messages added. If that assumption is violated,
+   you'll have to sync the index.
 
 Q: How do I back up my index?
-Q: How do I make a state dump?
 A: Since the contents of the messages are recoverable from their
    sources using sup-sync, all you need to back up is the message
    state. To do this, simply run:
@@ -66,13 +66,10 @@ A: Run:
      sup-sync [<source>+] --restored --restore <dumpfile>
    where <dumpfile> was created as above.
 
-Q: I upgraded Ferret and the index format changed. I need to
-   completely rebuild my index. How do I do this?
 Q: Ferret crashed and I can't read my index. Luckily I made a state
    dump. What should I do?
-A: First, you'll need a complete state dump. If you haven't made
-   one, you'll need to downgrade Ferret and make a state dump as
-   above. Then run these commands:
+Q: How do I rebuild the index completely?
+A: Run:
      rm -rf ~/.sup/ferret    # omg wtf
      sup-sync --all-sources --all --restore <dumpfile>
    Voila! A brand new index.
@@ -82,29 +79,24 @@ Q: I want to move messages from one source to another. (E.g., my
    some of those messages to local mbox files.) How do I do that while
    preserving message state?
 A: Move the messages from the source to the target using whatever tool
-   you'd like. Then (and this is the important part), run:
+   you'd like. Mutt's a good one. :) Then run:
      sup-sync --changed <source1> <source2>
 
-   If you sup-sync only one source at a time, depending on the order,
-   the messages may be treated as missing and then deleted from the
-   index, which means that their states will be lost when you sync the
-   other source.
+   Note that if you sup-sync only one source at a time, depending on
+   the order in which you do it, the messages may be treated as
+   missing and then deleted from the index, which means that their
+   states will be lost when you sync the other source. So do them both
+   in one go.
 
 Q: What are all these "Redwood" references I see in the code?
-A: That was Sup's original name. (Think pine, elm. Although I am a
-   Mutt user, I couldn't think of a good progression there.) But it was
-   taken by another project on RubyForge, and wasn't that original,
-   and was too long to type anyways.
+A: That was Sup's original name. (Think pine, elm. Although I was a
+   Mutt user, I couldn't think of a good progression there.) But it
+   was taken by another project on RubyForge, and wasn't that
+   original, and was too long to type anyways.
 
    Maybe one day I'll do a huge search-and-replace on the code, but it
    doesn't seem that important at this point.
 
-Q: How is Sup possible?
-A: Sup is only possible through the hard work of Dave Balmain, the
-   author of ferret, which is the search engine behind Sup. Ferret is
-   really a first-class piece of software, and it's due to the
-   tremendous amount of time and effort he's put in to it.
-
 Common Problems
 ---------------
 

data/doc/Hooks.txt

@@ -0,0 +1,38 @@
+Sup's Hook System
+-----------------
+
+Sup can be easily customized via its hook system, which allows custom
+user code to be injected into Sup's execution path by "hooking" the
+code onto pre-defined events. When those events occur, the code is
+executed.
+
+To see which hooks are available, simply run sup -l. Each hook sits in
+a file in ~/.sup/hooks/. Hooks are written in Ruby, and require no
+class or method definitions, just the executable code itself.
+
+Information passes from Sup to the hook code via Ruby variables
+(actually method calls), and from the hook code back to Sup via a
+return value. Each hook description lists the variables and return
+value expected, if any.
+
+Some example hooks:
+
+before-poll:
+  ## runs fetchmail before polling
+  if (@last_fetchmail_time || Time.now) < Time.now - 60
+    say "Running fetchmail..."
+    system "fetchmail >& /dev/null"
+    say "Done running fetchmail."
+  end
+  @last_fetchmail_time = Time.now
+
+
+mime-decode:
+  ## turn text/html attachments into plain text, unless they are part
+  ## of a multipart/alternative pair
+  unless sibling_types.member? "text/plain"
+    case content_type
+    when "text/html"
+      `/usr/bin/w3m -dump -T #{content_type} '#{filename}'`
+    end
+  end