sup 0.1 → 0.2

data/doc/{UserGuide.txt → NewUserGuide.txt}

@@ -13,11 +13,10 @@ cryptic messages. You can also press '?' at any point to get a list of
 keyboard commands, but in the absence of any email, these will be
 mostly useless. When you get bored, press 'q' to quit.
 
-To actually use Sup for email, we need to load messages into the
-index. The index is where Sup stores all message state (e.g. read or
-unread, any message labels), and all information necessary for
-searching and for threading messages. Sup only knows about messages in
-its index.
+To use Sup for email, we need to load messages into the index. The
+index is where Sup stores all message state (e.g. read or unread, any
+message labels), and all information necessary for searching and for
+threading messages. Sup only knows about messages in its index.
 
 We can add messages to the index by telling Sup about the "source"
 where the messages reside. Sources are things like IMAP folders, mbox
@@ -68,7 +67,7 @@ messages together into threads: each line in the inbox is a thread,
 and the number in parentheses is the number of messages in that
 thread. (If there's no number, there's just one message in the
 thread.) In Sup, most operations are on threads, not individual
-messages. The ideais that you rarely want to operate on a message
+messages. The idea is that you rarely want to operate on a message
 independent of its context. You typically want to view, archive, kill,
 or label all the messages in a thread at one time.
 
@@ -168,15 +167,15 @@ can figure out how to:
  - Star an individual message, not just a thread
 
 There's one last thing to be aware of when using Sup: how it interacts
-with other email programs. As I described at the beginning of the
-document, Sup stores data about messages in the index, but doesn't
-duplicate the message contents themselves. The messages remain on the
-source. If the index and the source every fall out of sync, e.g. due
-to another email client modifying the source, then Sup will be unable
-to operate on that source. For example, for mbox files, Sup stores a
-byte offset into the file for each message. If a message deleted from
-that file by another client, or even marked as read (yeah, mbox
-sucks), all succeeding offsets will be wrong.
+with other email programs. As I described above, Sup stores data about
+messages in the index, but doesn't duplicate the message contents
+themselves. The messages remain on the source. If the index and the
+source every fall out of sync, e.g. due to another email client
+modifying the source, then Sup will be unable to operate on that
+source. For example, for mbox files, Sup stores a byte offset into the
+file for each message. If a message deleted from that file by another
+client, or even marked as read (yeah, mbox sucks), all succeeding
+offsets will be wrong.
 
 That's the bad news. The good news is that Sup is pretty good at being
 able to detect this type of situation, and fixing it is just a matter
@@ -185,7 +184,7 @@ how to invoke sup-sync when it detects a problem. This is a
 complication you will almost certainly run in to if you use both Sup
 and another MUA on the same source, so it's good to be aware of it.
 
-Have fun, and let me know if you have any problems. --William
+Have fun, and let me know if you have any problems!
 
 Appending A: sup-add and sup-sync
 ---------------------------------
@@ -201,7 +200,8 @@ Instead of using sup-config to add a new source, you can manually run
 Before you add the source, you need make three decisions. The first is
 whether you want Sup to regularly poll this source for new messages.
 By default it will, but if this is a source that will never have new
-messages, you can specify --unusual. Sup polls only "usual" sources.
+messages, you can specify --unusual. Sup polls only "usual" sources
+when checking for new mail (unless you manually invoke sup-sync).
 
 The second is whether you want messages from the source to be
 automatically archived. An archived message will not show up in your
@@ -225,12 +225,12 @@ import except your actual inbox. You can also specify --read to mark
 all imported messages as read; the default is to preserve the
 read/unread status from the source.
 
-sup-sync will now load all the messages from the source into the
+Sup-sync will now load all the messages from the source into the
 index. Depending on the size of the source, this may take a while.
 Don't panic! It's a one-time process.
 
-Appending B: Handling high-volume mailing lists
------------------------------------------------
+Appendix B: Handling high-volume mailing lists
+----------------------------------------------
 
 Here's what I recommend:
 1. Use procmail to filter messages from the list into a distinct source.
@@ -238,8 +238,14 @@ Here's what I recommend:
    on, and with a label corresponding to the mailing list name.
    (E.g.: sup-add mbox:/home/me/Mail/ruby-talk -a -l ruby-talk)
 3. Voila! Sup will load new messages into the index but not into the
-   inbox, and you can browse the mailing list traffice at any point by
+   inbox, and you can browse the mailing list traffic at any point by
    searching for that label.
 
 
+Appendix C: Reading blogs with Sup
+----------------------------------
 
+Really, blog posts should be read like emails are read---you should be
+able to mark them as unread, flag them, label them, etc. Use rss2email
+to transform RSS feeds into emails, direct them all into a source, and
+add that source to Sup. Voila!

data/doc/TODO

@@ -1,12 +1,101 @@
-for 0.1
+for 0.2
 -------
+
+x bugfix: contacts.txt isn't parsed correctly when there are spaces in
+   aliases
+x bugfix: @ signs in names make sendmail die silently
+x bugfix: sent.mbox and >From
+x bugfix: tokenized email addresses (amazon.com, etc)
+x bugfix: trailing spaces in buffermanager.ask
+x bugfix: need to URL-unescape maildir folders
+x bugfix: downcasing tab completion
+x warnings: top-posting, missing attachment
+x hookability
+
+for 0.3
+-------
+_ mark thread as unread should remember the unread messages and mark
+   only them as unread, just like gmail
+_ mark thread as unread should have a version within thread-view-mode
+   which then also closes the buffer
+_ bugfix: time zone parsing broken?
+_ mailing list auto-subscribe/unsubscribe
+_ forwards optionally include attachments
+_ attach messages
+_ flesh out gpg integration: sign & encrypt outgoing
+_ mbox: don't keep filehandles open, and protect all reads with dotlockfile
+_ bugfix: screwing with the headers when editing causes a crash
+_ need a better way to force an address to a particular name,
+   for things like evite addresses
+_ pressing A in thread-view-mode should jump to next message
+_ imap "add all folders on this server" option in sup-add
+_ for new message flashes, add new message counts until keypress
+_ bugfix: missing sources should be handled better
+_ search results: highlight relevant snippets and open to relevant
+   portion of thread
+_ have "notes" (treated as emails to oneself, never sent) as
+   first-class objects.
+
+future
+------
+_ emlx support (some os x thing)
+_ tab completion for mid-text cursors
+_ bugfix: not horizontal scrolling for ncurses text field entry
+_ use trac or something. this file is getting a little silly.
+_ saved searches
+_ bugfix: sometimes, when one new message comes into an imap folder,
+  we don't catch it until a reload. but we do see a message
+  indicating they're loaded to inbox (imap only? hard to reproduce.)
+_ bugfix: ferret flakiness: just added message but can't find it.
+   possibly a message id tokenization issue.
+_ bugfix: read before thread-index has finished loading then hides the
+  thread?!? wtf. (on jamie) (? still valid ?)
+_ bugfix: display field width in index-mode needs to be determined
+   per-character rather than per-byte
+_ select all, starred, to me, etc
+_ undo
+_ Net::SMTP support
+_ ruby-talk:XXXX detection (via hooks?)
+_ more control character support in buffer line editing
+_ mboxz, mboxbz
+_ swappable keymappings
+_ bugfix: when returning from a shelling out, sometime ncurses is
+   crazy and refuses to interpret any keystrokes
+_ configurable colors
+_ better batch deletion (extend to non-mbox sources)
+_ annotations on messages
+_ pop support
+_ toggleble wrapping of text
+_ maybe: de-archived messages auto-added to inbox
+_ prune old entries from people.txt so that it doesn't grow without
+   bound
+_ maildir+ssh
+
+maybe
+-----
+_ split out threading & message chunk parsing to a separate library
+_ rangefilter on the initial inbox to only consider the most recent 1000 messages
+
+denied
+------
+x rss feed reading: use rss2email
+x gmail support: obsoleted by imap
+
+done
+----
+x bugfix: deadlock (on rubyforge) (? still valid ?)
+x bugfix: ferret corrupt index problem at index.c:901. see http://ferret.davebalmain.com/trac/ticket/279
+x tab completion for to: and cc: in compose-mode
+x individual labeling in thread-view-mode
+x translate aliases in queries on to: and from: fields
+x tab completion on labeling
 x bugfix: any interactive prompt after "No new messages." flash has an
    empty line above it.
 x detect other sup instances and do something intelligent (because
+   ferret crashes violently with more than one index writer open)
 x refactor all the *-search-results-mode classes
 x decode RFC 2047 ("encoded word") headers
   - see: http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/101949, http://dev.rubyonrails.org/ticket/6807
-   ferret crashes violently with more than one index writer open)
 x create attachments
 x add arbitrary labels to sources
 x improve sup-config
@@ -21,61 +110,6 @@ x bugfix: sup-add not prompting for old accounts, i think? possibly because
 x wide character support
 x i18n support
 x tab completion on labels
-
-for next release
-----------------
-_ use trac or something. this file is getting a little silly.
-_ gpg integration
-_ user-defined hooks
-_ saved searches
-_ bugfix: missing sources should be handled better
-_ tab completion for contacts
-_ bugfix: screwing the headers when editing causes a crash
-_ bugfix: sometimes, when one new message comes into an imap folder,
-   we don't catch it until a reload. but we do see a message
-   indicating they're loaded to inbox (imap only? hard to reproduce.)
-_ bugfix: need a better way to force an address to a particular name,
-   for things like evite addresses
-_ bugfix: ferret flakiness: just added message but can't find it
-_ add new message counts until keypress
-_ bugfix: deadlock (on rubyforge)
-_ bugfix: ferret corrupt index problem at index.c:901. see http://ferret.davebalmain.com/trac/ticket/279
-_ bugfix: read before thread-index has finished loading then hides the
-   thread?!? wtf. (on jamie)_ bugfix: width in index-mode needs to be
-   determined per-character rather than per-byte
-_ search results: highlight relevant snippets and open to relevant
-   portion of thread
-_ rss feed reading
-_ forward attachments
-_ select all, starred, to me, etc
-_ undo
-_ gmail support
-_ warnings: top-posting, missing attachment, ruby-talk:XXXX detection
-_ Net::SMTP support
-_ more control character support in buffer line editing
-
-future
-------
-mboxz, mboxbz
-swappable keymappings
-bugfix: when returning from a shelling out, sometime ncurses is crazy
-   and refuses to interpret any keystrokes
-better batch deletion (extend to non-mbox sources)
-annotations on messages
-pop
-be able to mark individual messages as spam in thread-view-mode
-toggle wrapping
-maybe: de-archived messages auto-added to inbox
-prune old entries from contacts.txt so that it doesn't grow without bound
-
-maybe
------
-split out threading & message chunk parsing to a separate library
-filters
-rangefilter on the initial inbox to only consider the most recent 1000 messages
-
-done
-----
 x nice little startup config program
 x bugfix: triggering a pageup when cursor scrolling up jumps to the
    bottom of the page rather than the next line

data/lib/sup.rb

@@ -3,6 +3,7 @@ require 'yaml'
 require 'zlib'
 require 'thread'
 require 'fileutils'
+require 'curses'
 
 class Object
   ## this is for debugging purposes because i keep calling #id on the
@@ -31,7 +32,7 @@ class Module
 end
 
 module Redwood
-  VERSION = "0.1"
+  VERSION = "0.2"
 
   BASE_DIR   = ENV["SUP_BASE"] || File.join(ENV["HOME"], ".sup")
   CONFIG_FN  = File.join(BASE_DIR, "config.yaml")
@@ -43,6 +44,7 @@ module Redwood
   SENT_FN    = File.join(BASE_DIR, "sent.mbox")
   LOCK_FN    = File.join(BASE_DIR, "lock")
   SUICIDE_FN = File.join(BASE_DIR, "please-kill-yourself")
+  HOOK_DIR   = File.join(BASE_DIR, "hooks")
 
   YAML_DOMAIN = "masanjin.net"
   YAML_DATE = "2006-10-01"
@@ -111,6 +113,7 @@ module Redwood
     Redwood::UpdateManager.new
     Redwood::PollManager.new
     Redwood::SuicideManager.new Redwood::SUICIDE_FN
+    Redwood::CryptoManager.new
   end
 
   def finish
@@ -193,6 +196,10 @@ else
     :editor => ENV["EDITOR"] || "/usr/bin/vim -f -c 'setlocal spell spelllang=en_us' -c 'set filetype=mail'",
     :thread_by_subject => false,
     :edit_signature => false,
+    :ask_for_cc => true,
+    :ask_for_bcc => false,
+    :confirm_no_attachments => true,
+    :confirm_top_posting => true,
   }
   begin
     FileUtils.mkdir_p Redwood::BASE_DIR
@@ -203,8 +210,16 @@ else
 end
 
 require "sup/util"
+require "sup/hook"
+
+## we have to initialize this guy first, because other classes must
+## reference it in order to register hooks, and they do that at parse
+## time.
+Redwood::HookManager.new Redwood::HOOK_DIR
+
 require "sup/update"
 require "sup/suicide"
+require "sup/message-chunks"
 require "sup/message"
 require "sup/source"
 require "sup/mbox"
@@ -224,6 +239,7 @@ require "sup/contact"
 require "sup/tagger"
 require "sup/draft"
 require "sup/poll"
+require "sup/crypto"
 require "sup/modes/scroll-mode"
 require "sup/modes/text-mode"
 require "sup/modes/line-cursor-mode"

data/lib/sup/buffer.rb

@@ -1,6 +1,8 @@
 require 'etc'
 require 'thread'
+require 'ncurses'
 
+if defined? Ncurses
 module Ncurses
   def rows
     lame, lamer = [], []
@@ -14,6 +16,12 @@ module Ncurses
     lamer.first
   end
 
+  def curx
+    lame, lamer = [], []
+    stdscr.getyx lame, lamer
+    lamer.first
+  end
+
   def mutex; @mutex ||= Mutex.new; end
   def sync &b; mutex.synchronize(&b); end
 
@@ -27,12 +35,16 @@ module Ncurses
     end
   end
 
-  module_function :rows, :cols, :nonblocking_getch, :mutex, :sync
+  module_function :rows, :cols, :curx, :nonblocking_getch, :mutex, :sync
+
+  remove_const :KEY_ENTER
+  remove_const :KEY_CANCEL
 
   KEY_ENTER = 10
-  KEY_CANCEL = ?\a # ctrl-g
+  KEY_CANCEL = 7 # ctrl-g
   KEY_TAB = 9
 end
+end
 
 module Redwood
 
@@ -328,12 +340,29 @@ class BufferManager
 
   def ask_with_completions domain, question, completions, default=nil
     ask domain, question, default do |s|
-      completions.select { |x| x =~ /^#{s}/i }.map { |x| [x.downcase, x] }
+      completions.select { |x| x =~ /^#{s}/i }.map { |x| [x, x] }
+    end
+  end
+
+  def ask_many_with_completions domain, question, completions, default=nil, sep=" "
+    ask domain, question, default do |partial|
+      prefix, target = 
+        case partial#.gsub(/#{sep}+/, sep)
+        when /^\s*$/
+          ["", ""]
+        when /^(.+#{sep})$/
+          [$1, ""]
+        when /^(.*#{sep})?(.+?)$/
+          [$1 || "", $2]
+        else
+          raise "william screwed up completion: #{partial.inspect}"
+        end
+
+      completions.select { |x| x =~ /^#{target}/i }.map { |x| [prefix + x, x] }
     end
   end
 
-  ## returns an ARRAY of filenames!
-  def ask_for_filenames domain, question, default=nil
+  def ask_for_filename domain, question, default=nil
     answer = ask domain, question, default do |s|
       if s =~ /(~([^\s\/]*))/ # twiddle directory expansion
         full = $1
@@ -361,13 +390,53 @@ class BufferManager
         elsif File.directory?(answer)
           spawn_modal "file browser", FileBrowserMode.new(answer)
         else
-          [answer]
+          answer
         end
     end
 
-    answer || []
+    answer
+  end
+
+  ## returns an array of labels
+  def ask_for_labels domain, question, default_labels, forbidden_labels=[]
+    default_labels = default_labels - forbidden_labels - LabelManager::RESERVED_LABELS
+    default = default_labels.join(" ")
+    default += " " unless default.empty?
+
+    applyable_labels = (LabelManager.applyable_labels - forbidden_labels).map { |l| LabelManager.string_for l }.sort_by { |s| s.downcase }
+
+    answer = ask_many_with_completions domain, question, applyable_labels, default
+
+    return unless answer
+
+    user_labels = answer.split(/\s+/).map { |l| l.intern }
+    user_labels.each do |l|
+      if forbidden_labels.include?(l) || LabelManager::RESERVED_LABELS.include?(l)
+        BufferManager.flash "'#{l}' is a reserved label!"
+        return
+      end
+    end
+    user_labels
+  end
+
+  def ask_for_contacts domain, question, default_contacts=[]
+    default = default_contacts.map { |s| s.to_s }.join(" ")
+    default += " " unless default.empty?
+    
+    recent = Index.load_contacts(AccountManager.user_emails, :num => 10).map { |c| [c.longname, c.email] }
+    contacts = ContactManager.contacts.map { |c| [ContactManager.alias_for(c), c.longname, c.email] }
+
+    Redwood::log "recent: #{recent.inspect}"
+
+    completions = (recent + contacts).flatten.uniq.sort
+    answer = BufferManager.ask_many_with_completions domain, question, completions, default, /\s*,\s*/
+
+    if answer
+      answer.split_on_commas.map { |x| ContactManager.contact_for(x.downcase) || PersonManager.person_for(x) }
+    end
   end
 
+
   def ask domain, question, default=nil, &block
     raise "impossible!" if @asking
     @asking = true
@@ -393,27 +462,22 @@ class BufferManager
 
     while true
       c = Ncurses.nonblocking_getch
-      next unless c  # getch timeout
+      next unless c # getch timeout
       break unless tf.handle_input c # process keystroke
 
       if tf.new_completions?
         kill_buffer completion_buf if completion_buf
         
-        prefix_len =
-          if tf.value =~ /\/$/
-            0
-          else
-            File.basename(tf.value).length
-          end
+        shorts = tf.completions.map { |full, short| short }
+        prefix_len = shorts.shared_prefix.length
 
-        mode = CompletionMode.new tf.completions.map { |full, short| short }, :header => "Possible completions for \"#{tf.value}\": ", :prefix_len => prefix_len
+        mode = CompletionMode.new shorts, :header => "Possible completions for \"#{tf.value}\": ", :prefix_len => prefix_len
         completion_buf = spawn "<completions>", mode, :height => 10
 
         draw_screen :skip_minibuf => true
         tf.position_cursor
       elsif tf.roll_completions?
         completion_buf.mode.roll
-
         draw_screen :skip_minibuf => true
         tf.position_cursor
       end