vmail 0.0.1

data/.gitignore

@@ -0,0 +1,4 @@
+*swp
+login.yml
+gmail.yml
+*.log

data/NOTES

@@ -0,0 +1,541 @@
+
+interesting. the UID don't seem to be globally unique. two different
+messages in 2 different mailboxes can have same UID
+
+
+Mon Nov 22 23:12:49 EST 2010
+
+- next, do window sensing and recreation. let user expand the message
+  detail window and shrink it with some key like <cr>
+
+I need status windows that show how many messagesin a box, when it was
+last updated, etc
+
+This app is coming along well.
+
+
+Another reason to not allow a divergent offline store is that when the
+schema changes on an update, it's easier to have the user rebuild the
+locate version of the GMAIL 
+
+
+Another anomaly. Threading seems to reverse the UIDs. 
+
+
+Imap tasks should be batched so that we don't have the overhead of an
+imap login each time.
+
+The vertial 3 pan view should have a different set of columns in the
+message list window.
+
+Need command to reprocess text translation, also to display way eml.
+
+NEXT. Learn how to do a popup window. Try out other Vim plugins! Do some
+research online.
+
+OR, do replies.
+
+- work on email formatting
+
+Flag new messages. And position cursor correctly on a refresh.
+
+
+
+
+
+inputlist()
+
+
+winrestcmd() potentially useful
+
+1resize 20|vert 1resize 84|2resize 18|vert 2resize 84|3resize 39|vert 3resize 84|
+
+
+------------------------------------------------------------------------
+Thu Nov 25 16:55:07 EST 2010 {{{1
+
+Trying new schema and more unix-inspired approach. First, the schema is
+changed to allow unique messages to be shared across mailboxes via
+message_refs. The Mailbox model has been removed.
+
+Second, the tools should be smaller and less interdependent. There
+should be a tool just for getting a text list of UID for a mailbox, and
+then another that takes a UID and pulls down the raw eml, which I can
+then use for testing.
+
+The tool to insert into data should take two arguments, the mailbox name
+and the UID, and the raw eml into STDIN.
+
+Set up a test harness for all this email caching and data extraction
+with rspec or test unit
+
+------------------------------------------------------------------------
+Fri Nov 26 12:08:26 EST 2010 {{{1
+
+I need a text extract tool that works like iconv command line tool.
+
+Lots of little tools for my workbench.
+
+
+Now, I'm thinking of ditching the database. Use one document of
+concatenated EML with message and mailbox info incorporated.
+
+
+I tried using folds to present the email headers but I'm not sure it
+works. Too many key strokes to unfold, fold, move to next message, and
+unfold. Also, search doesn't work as expected since it searches through
+all the text.
+
+This suggests that using the database might be the best solution.
+
+
+OK I've experimented and found a reasonably fast way to do imap searchs
+
+e.g. 
+
+  ruby lib/gmail.rb FROM monit BEFORE 8-Aug-2010 SINCE 1-Jun-2010
+
+Now the question is how to get from output to load a specific message
+and then possibly reply to it.
+
+And later, we need to keep a contact list somehow. 
+
+I also forgot how an email is sent with sendmail. Maybe with
+authentication.
+
+------------------------------------------------------------------------
+Sun Nov 28 10:42:50 EST 2010
+
+Got threaded message list fetch working
+
+Marking messages for deletion or starring could be done very simply:
+Just add a mark '*' or 'D' to the beginning of lines that should be
+starred or deleted, then press a command key.
+
+Just put one commented line at top of generated list. This is so the
+viewer can refresh the list on demand when the user wants to. But the
+focus should be on generating these imap search result lists from the
+command line and saving the output to a file. Then the use can view the
+file in the viewer. Should the file serve as a virtual mailbox?
+
+User can alway do "search command | tee inbox" periodically and then
+view the inbox in the viewer when a new message arrives.
+
+There are even ways of setting up watches from the command to see if a
+new message arrives in a certain mailbox. These recipes should be
+written up in a tutorial on the website.
+
+
+Wrote a wrapper startup script that you invoke like so
+
+./wrapper inbox 20 all
+./wrapper inbox 20 subject instantwatcher
+
+This will generate the list file and open the viewer on it.
+
+I think this is the workflow I want. It keeps the viewer lightweight and
+prevents it from becoming a captive user interface.
+
+
+
+Remember to convert uid to_i before calling IMAP methods.
+
+Need to figure out a way to flag a line and show the change. Do we have
+to use Vim data structures?
+
+Also, be careful. The f command throws an error when there is no mailbox
+and search term information at the top of the message list.
+
+
+Make a view mode that lets one simple page through the messages and keep
+track of which message is visible.
+
+TODO Try d for trash and D for permanent Delete.
+
+
+http://stackoverflow.com/questions/1430866/gmail-threading-imap-and-ruby
+
+------------------------------------------------------------------------
+Tue Nov 30 07:30:19 EST 2010
+
+Start from command line, and then quit. Not meant to be an application
+that you keep open. At least not for my uses.
+
+But it would be better to start up the app in the Ruby wrapper
+
+If it's contained in vim, I can use append(lnum, list) to insert
+previous messages to the top of the buffer as the user scrolls back.
+
+So two things to accomplish this morning: 
+1. start the application in a Ruby wrapper
+2. select mailbox from viewer
+3. infinite paging back
+
+Done:
+1. Ruby wrapper
+2. Update the current mailbox
+3. Fast deletes, range flagging
+4. slightly better multipart displaying
+
+Won't do:
+- infinite paging 
+- starting mailbox from viewer
+
+TODO
+- send messages
+
+
+Unicode korean characters are wider than ascii characters, so this will
+mess up alignment, but I don't know what I can do about this.
+
+------------------------------------------------------------------------
+Wed Dec  1 15:39:39 EST 2010
+
+- got composing messages working, switching between full screen message
+  view and list view
+
+TODO
+- reply to messages
+
+contacts
+- import contacts by looking at sent mailbox. save contacts into a file
+  and use that for autocompletion
+
+split reply window veritcally: that's the way I wanted this interface
+
+Quoting emails:
+
+  http://en.wikipedia.org/wiki/Posting_style
+
+stackoverflow:
+
+  If you get the raw source of the messages you can relate messages to one
+  another by looking at the following headers:
+
+  Message-ID:
+  <097819EBC7F79F4A850C8F088D35927302442A82AA@xxxxxxxxxxxxxxxxxxx>
+  References: <4A8BE8F3.2060007@xxxxxxxxxxxxxx>
+  In-Reply-To: <4A8BE8F3.2060007@xxxxxxxxxxxxxxx>
+  link|flag
+  answered Aug 28 '09 at 7:23
+  Kees de Kooter
+  1,7341512
+
+
+TODO
+- get full email addresses including names
+- fix reply to quote header
+- address book
+- later: 
+  - append to text file
+  - display and save attachments
+Thurs
+  - start packaging
+- need a better way to refresh a stale IMAP connection
+  - try wrapping client calls with Timeout. Then call daemon to refresh
+    connection if things take too long.
+
+* Need to save mail list buffer or else it forgets what user has flagged
+or deleted.
+
+
+- will have to write out message list buffer
+
+- IMAP fails from parsing email from 
+
+  D, [2010-12-02T04:14:47.034816 #26464] DEBUG -- : error fetching uid
+  83682
+  D, [2010-12-02T04:14:47.034884 #26464] DEBUG -- : unknown token -
+  "\"[Christian" (Net::IMAP::ResponseParseError)
+
+
+It is vital to rewrite the message list buffer to file. This is a
+significant bug. Since new messages gotten via update disappear when
+coming back from full screen message view.
+
+Also, let use go forward and back while reading full messages.
+Or even in split view. Don't bind n since we need it for search. Use
+CTRL-n, CTRL-p? or leader n leader 
+
+Need better recovery from expired IMAP session. At least some kind of
+warning so that use retries the previous action that failed.
+
+TODO
+- gmail charset correction
+
+
+------------------------------------------------------------------------
+Tue Dec  7 09:36:01 EST 2010
+
+Instead of autocomplete mailbox chooser, have a generic search command
+line buffer that might autocomplete the mailbox for you. 
+
+This simplifies the user interface and makes it more coherent.
+
+You will always specify a mailbox and a command. Get the user used to
+doing searches like this.
+
+I'm also thinking of keep the viewer thin and not that featureful.
+I'll emphasize the command line features, like using search criteria to
+pipe emails into a text file.
+
+Change the architecture so that the daemon is fired up in a separate
+process.
+
+I don't think I need to download the RFC822 header, the envelope has
+this data. 
+
+Done. 
+
+Next, in the search window, I can keep a history of previous searches.
+And just execute the command that the user hits carriage return on.
+
+    ENVELOPE: !ruby/struct:Net::IMAP::Envelope
+      date: Tue, 7 Dec 2010 11:25:47 -0500
+      subject: "Error message: /titles.new"
+      from:
+      - !ruby/struct:Net::IMAP::Address
+        name:
+        route:
+        mailbox: error
+        host: instantwatcher.com
+      sender:
+      - !ruby/struct:Net::IMAP::Address
+        name:
+        route:
+        mailbox: error
+        host: instantwatcher.com
+      reply_to:
+      - !ruby/struct:Net::IMAP::Address
+        name:
+        route:
+        mailbox: error
+        host: instantwatcher.com
+      to:
+      - !ruby/struct:Net::IMAP::Address
+        name:
+        route:
+        mailbox: dhchoi
+        host: gmail.com
+      cc:
+      bcc:
+      in_reply_to:
+      message_id: <4cfe600b1cd1f_202540e109c8ee@ubuntu.tmail>
+
+
+
+
+(druby://127.0.0.1:61676) /Users/choi/.rvm/rubies/ruby-1.9.2-p0/lib/ruby/1.9.1/net/imap.rb:3235:in `parse_error': unknown token - "\"[Christian" (Net::IMAP::ResponseParseError)
+        from (druby://127.0.0.1:61676) /Users/choi/.rvm/rubies/ruby-1.9.2-p0/lib/ruby/1.9.1/net/imap.rb:3180:in `next_token'
+        from (druby://127.0.0.1:61676) /Users/choi/.rvm/rubies/ruby-1.9.2-p0/lib/ruby/1.9.1/net/imap.rb:3097:in `lookahead'
+        from (druby://127.0.0.1:61676) /Users/choi/.rvm/rubies/ruby-1.9.2-p0/lib/ruby/1.9.1/net/imap.rb:2995:in `nstring'
+        from (druby://127.0.0.1:61676) /Users/choi/.rvm/rubies/ruby-1.9.2-p0/lib/ruby/1.9.1/net/imap.rb:2183:in `envelope'
+        from (druby://127.0.0.1:61676) /Users/choi/.rvm/rubies/ruby-1.9.2-p0/lib/ruby/1.9.1/net/imap.rb:2170:in `envelope_data'
+        from (druby://127.0.0.1:61676) /Users/choi/.rvm/rubies/ruby-1.9.2-p0/lib/ruby/1.9.1/net/imap.rb:2145:in `msg_att'
+        from (druby://127.0.0.1:61676) /Users/choi/.rvm/rubies/ruby-1.9.2-p0/lib/ruby/1.9.1/net/imap.rb:2125:in `numeric_response'
+        from (druby://127.0.0.1:61676) /Users/choi/.rvm/rubies/ruby-1.9.2-p0/lib/ruby/1.9.1/net/imap.rb:2067:in `response_untagged'
+        from (druby://127.0.0.1:61676) /Users/choi/.rvm/rubies/ruby-1.9.2-p0/lib/ruby/1.9.1/net/imap.rb:2047:in `response'
+        from (druby://127.0.0.1:61676) /Users/choi/.rvm/rubies/ruby-1.9.2-p0/lib/ruby/1.9.1/net/imap.rb:1973:in `parse'
+        from (druby://127.0.0.1:61676) /Users/choi/.rvm/rubies/ruby-1.9.2-p0/lib/ruby/1.9.1/net/imap.rb:1124:in `get_response'
+        from (druby://127.0.0.1:61676) /Users/choi/.rvm/rubies/ruby-1.9.2-p0/lib/ruby/1.9.1/net/imap.rb:1036:in `receive_responses'
+        from (druby://127.0.0.1:61676) /Users/choi/.rvm/rubies/ruby-1.9.2-p0/lib/ruby/1.9.1/net/imap.rb:1023:in `block in initialize'
+        from run.rb:8:in `block in <main>'
+        from run.rb:7:in `open'
+        from run.rb:7:in `<main>'
+
+
+
+    # Sends a APPEND command to append the +message+ to the end of
+    # the +mailbox+. The optional +flags+ argument is an array of 
+    # flags to initially passing to the new message.  The optional
+    # +date_time+ argument specifies the creation time to assign to the 
+    # new message; it defaults to the current time.
+    # For example:
+    #
+    #   imap.append("inbox", <<EOF.gsub(/\n/, "\r\n"), [:Seen], Time.now)
+    #   Subject: hello
+    #   From: shugo@ruby-lang.org
+    #   To: shugo@ruby-lang.org
+    #   
+    #   hello world
+    #   EOF
+    #
+    # A Net::IMAP::NoResponseError is raised if the mailbox does
+    # not exist (it is not created automatically), or if the flags,
+    # date_time, or message arguments contain errors.
+
+
+
+
+Next, I think I need to restore the vertical mailbox selection mode, and
+then have the search window execute in the current mailbox.
+
+Finally, I need to eliminate the limit number (on search) and just to dynamic
+back-paging when a user gets to the top line, which will say "Get 40
+more messages" or something.
+
+Also, forget 2 pane, just toggle full screen message and list view
+
+Finally, I need to do autocompletion for contacts.
+
+
+
+done today:
+- backpaging
+- single window at a time
+- vertical mailbox selection
+- no more yaml
+
+OK next
+- autocompletion for contacts
+- extract and show attachments, bytesize
+  - can't show this without downloading RFC822.header
+- autocomplete better for mailbox
+- STTY adjustment `stty size` from the vim client and sent to back end?
+
+This url doesn't open 
+
+  http://twitter.com/#!/kanter/status/12398658028834816
+
+  Because of the # or !. Wrapping it in quotes double or singles
+  doesn't fix the prob. Need to use system and shellescape
+
+TODO
+- display current mailbox somewhere, and also search query
+- allow splitting the window to keep multiple messages open. Don't call only. 
+- move to - done
+- contacts!
+- character encodings
+
+echo winwidth(1) 
+  use this instead of system('stty size') to get window width
+  - then adjust the column sizes
+
+:set statusline=%<%f\ %h%m%r%=%-14.(%l,%c%V%)\ %P
+
+  - in .vmail/ dir?
+
+------------------------------------------------------------------------
+Thu Dec  9 10:13:53 EST 2010
+
+- got status line to show current mailbox. Next I just need to show the
+  current search query!
+- got bells to go away in most cases
+- need to work on autocomplete for addressbook
+- getting there!
+
+- ok i got statusline working in principle
+
+- next, do contacts
+- fix encoding issues
+- get ready to release. Monday seems doable.
+
+for contacts, use dictionary file option in vim
+
+
+DONE
+- contacts, proof of concept
+- encoding issues (some at least)
+- get reply template recipients right
+- forwarding (not with attachments)
+- auto-adjusting column widths depending on window width
+- for older message, replace time with year
+- signature
+- open hyperlink in browser (OS X only?)
+- open html in browser
+- when show_next_message, get message without flashing the list
+  - solved by restoring two pane view
+- when loading update, position cursor on first new message
+- spacebar in message window to fullscreen the message and toggle back
+- spam range
+- in list, show message sizes 
+- prevent the "8 more lines" and wait for enter. 2 is the threshhold
+- fix compose message, reply windowing
+- mapping improvements:
+  - straight to reply from list window
+- fixed addresses in recipients to a reply all 
+- position cursor conveniently on opening compose windows
+- show num additional when in sent msg list
+- save drafts on server to [Gmail]/Drafts
+- use unquote_and_convert for subject lines?
+- attachments extraction
+- encoding issues (korean) 
+  - plus use iconv for 1.8
+  - search for from kr
+  - can't rely on mail.encoding to be non-nil
+- if html version, command to open
+
+
+next:
+- create another bin wrapper to start both daemon and client from same
+  window. redirect streams from daemon to a log file
+  - can alter dep on command line opts
+- package gem
+  - vmail-server
+  - vmail (client)
+  - try running as one wrapper process
+- sending an attachment
+  - can specify a directory, then all the items in the directory get
+    attached and sent
+- forwarding attachments
+  - could be "attach: [uid] attachments to signal"
+- use :wq to send compose message, not ,d (easy to confuse with delete!)
+  - close! to cancel
+  - fewer mappings to remember this way
+- put instructions in message window; put in a top section 
+- map u update from message window      
+- follow mysql and use -u and -p flags on startup of server?
+  - omitting -p flag forces prompt
+  - get rid of config file
+- use temp files
+  - tempname()
+  - system() allows a parameters that is written to tmp file and passed
+    to stdin
+- help document should just be readme? or part of it.
+- print all selected messages into file (append)
+  - can pipe through a command line tool, so we don't need to clutter
+    viewer.vim with more mappings and functions
+- put "move to >" and "select mailbox >" in command window
+- make compat with ruby 1.8
+- help
+- readme
+- ,s s confusion? Star vs search
+- sort contacts by frequency, then take first 10 or so of any match
+  group
+  - enhance contacts auto fix with more advance vim script
+- mvim = allow using through VMAIL_EDITOR env variable
+
+later:
+- allow one daemon, multiple clients (select mailbox?)
+- remember searches?
+- archive function (shortcut for move to all mail)
+- crate to package app as a OS X app?
+  http://rubyconf2008.confreaks.com/crate-packaging-your-ruby-application.html
+  - need to bundle macvim
+- if move to unknown mailbox, create the mailbox, then do the move
+- some lingering encoding issues
+- something is wrong with cursorline sometimes
+- try flashing progress in echo line
+  - straight to switch mailbox, etc from message window
+- allow deliver command from any buffer or window, as long as headers are at top
+- range selection from : prompt
+- starring 
+  - do fast action like deletes
+- reload after window resize
+- sometimes update doesn't work - bug
+- message threads
+
+
+
+------------------------------------------------------------------------
+
+Threading
+how to find the message that a message is in reply-to
+
+imap.search ["HEADER", "MESSAGE-ID", "<757FC46C-9394-494E-91B4-B051F48419DA@prx.org>"]
+
+:! ls %
+
+  % is filename
+
+