sortah 0.5.0

data/.gitignore

@@ -0,0 +1,8 @@
+*.gem
+.bundle
+Gemfile.lock
+pkg/*
+.rspec
+tmp/*
+
+coverage/

data/.rvmrc

@@ -0,0 +1,2 @@
+RVM_STRING="rvm use ruby-1.9.2-p290@sortah --create --verbose"
+[ -e './.rvmrc_local' ] && source './.rvmrc_local' || $RVM_STRING

data/.travis.yml

@@ -0,0 +1,3 @@
+rvm:
+  - 1.9.2
+  - 1.9.3

data/CONTRIBUTION.md

@@ -0,0 +1,23 @@
+#Version rules
+
+Version 1.0 comes when I'm pretty sure there are no major outstanding bugs from
+release. After that, SemVer applies.
+
+Let me (jfredett) control the Version, please don't submit a version change in
+any pull request. This is merely out of concern for sanity.
+
+#Contribution
+
+Easy peasy, fork, pull request. Don't submit something with broken tests in it. 
+
+If you find a bug, filing an issue is cool, fixing it is cooler. I will love you
+forever if you fix it. 
+
+##Contributing sortah definitions:
+
+For now, there is no "sortah contrib" repo, if there are many people interested
+in sharing some common set of sortah definitions, then one will be created.
+
+For now, if you have something cool, make a page in the wiki.
+
+

data/FUTURE_PLANS.md

@@ -0,0 +1,73 @@
+#Ideas
+
+- proxy destinations
+
+Difficulty: middling
+
+To allow for non-filesystem storage locations (say, redis, elasticsearch, w/e),
+it would be nice to have "pseudo" destinations, like:
+
+    destination :redis do
+      RedisHandler.acquire_conn do |redis|
+        redis.put email.key, email.to_s
+      end
+    end
+
+    lens :key do
+      email.key = SHA1_of(email)
+    end
+
+    router :root, :lenses => [:key] do
+      send_to :redis
+    end
+
+- multi-target `send_to`
+
+Difficulty: easy
+
+It would be nice to say:
+
+    router do
+      send_to [:foo, :bar, :baz]
+    end
+
+and have a copy be sent to each destination, optionally, it could take a
+parameter, "linked", so that the others would only be soft-links to the
+canonical one (specifed by link), eg:
+
+    router do
+      send_to [:foo, :bar, :baz], :link => :foo
+    end
+
+In the above, :bar and :baz would be softlinks to :foo
+
+- contrib
+
+Difficulty: easy
+Prereq: Having a community.
+
+Set up an easy-to-use 'contrib' repo for community sortah libraries.
+
+- getmail integration
+
+Difficulty: easy | hard
+
+This comes in two flavors, the easy flavor is to just provide a wrapper
+for defining a getmailrc. Perhaps making it easy to define a gmail rc and
+also allowing for safe password injections (eg, not storing the password
+plaintext in the rc file).
+
+The harder version would be to just implement getmail as part of sortah.
+This would allow for fine-grained control over how sortah gets fired, allowing
+for better concurrency.
+
+
+
+
+
+
+
+
+
+
+

data/Gemfile

@@ -0,0 +1,10 @@
+source "http://rubygems.org"
+
+# development deps:
+gem 'ruby-debug19', :require => 'rubydebug'
+gem 'rspec'
+gem 'rake'
+gem 'pry', :require => 'pry'
+
+# Specify your gem's dependencies in sortah.gemspec
+gemspec

data/KNOWN_BUGS.md

@@ -0,0 +1,12 @@
+doing 
+
+    sortah.sort(email)
+    sortah.metadata#blahblahblah
+
+does not behave as expected (it doens't preserve metadata to the next line)
+this has something to do with how I pull the data in the Kernel patch. it works
+if you do
+
+    sortah.sort(email).metadata#blahblah
+
+

data/LICENSE

@@ -0,0 +1,27 @@
+                    Copyright (c) 2011, Joseph Fredette
+                            All rights reserved.
+
+Redistribution and use in source and binary forms, with or without modification,
+are permitted provided that the following conditions are met:
+
+Redistributions of source code must retain the above copyright notice, this list
+of conditions and the following disclaimer.
+
+Redistributions in binary form must reproduce the above copyright notice, this
+list of conditions and the following disclaimer in the documentation and/or
+other materials provided with the distribution.
+
+Neither the name of "sortah" nor the names of its contributors may be used to
+endorse or promote products derived from this software without specific prior
+written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR
+ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
+ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

data/README.md

@@ -0,0 +1,216 @@
+#sortah 
+
+##For sortin' your friggin' mail.
+
+--------------------------------------------------------------------------------
+
+[![Build Status](https://secure.travis-ci.org/jfredett/sortah.png)](http://travis-ci.org/jfredett/sortah)
+
+--------------------------------------------------------------------------------
+
+Sortah sort's mail. It provides a ruby [EDSL](# Embedded DSL) for manipulating
+email objects. The DSL allows the definition of three principle components:
+
+- Destinations
+
+A destination takes in an Email object, and returns a system path. This is where
+the email object passed into it will be saved. Ex:
+
+    destination :spam, "spam/" 
+    destination :ham, "/"
+
+These are -- in essence -- simple delcarations of the structure of your sorting
+system. They may take one of several forms. The examples above are relative to
+the mail directory, and will transparently manage organization south of that.
+They can also be absolute paths, eg:
+
+    destination :devnull, :abs => "/dev/null"
+
+which is regarded as an absolutely qualified path. It may also alias another
+path:
+
+    destination :tldr, :devnull 
+
+- Lenses
+
+Lenses are functions which produce a value given an input email. This value is
+interpreted as metadata to be used by the routers. Ex:
+
+    lens :spam_value do
+      x = 0
+      email.text.each_line do |line|
+        x += (line =~ /extension/) ? 1 : 0
+      end
+      x
+    end
+
+    lens :word_count do
+      email.text.split.size
+    end
+
+lenses can also depend on other lenses.
+
+    lens :spam_ratio :lenses => [:spam_value, :word_count] do
+      email.spam_value / email.word_count 
+    end
+
+You may specify the `pass_through` option to cause the lens to not set any 
+metadata. This is useful in two cases, updating old metadata, and interaction
+(typically creational) with other services. Eg:
+
+    lens :example_update, :pass_through => true do
+      email.spam_value = 1000000 if email.sender == "annoying_guy0022493@hotmail.com"
+    end
+
+    lens :example_interaction, :pass_through => true, lenses => [:spam_value] do
+      return unless email.spam_value >= 1000000
+      HTTParty.post "http://spamblacklist.net/spammer/new", :body => email.sender
+    end
+
+- Routers
+
+This is the core of the language, a router is an object which produce either a
+router object, or a destination. If it produces a destination, then the email is
+delivered to that destination. If it produces another router, then the email is
+passed along to the router produced. A router also 'depends' on lenses. These
+lenses get applied when the router is called. There is one router which is
+special, the "root" router, this is the first router which gets called. To
+declare it, simply declare a router without a name. Ex:
+
+    router :spam_filter, :lenses => [:spam_value] do
+      send_to :ham if email.spam_value < 10
+      send_to :spam 
+    end
+
+    router :root, :lenses => [:word_count] do
+      send_to :tldr if email.word_count > 100 
+      send_to :spam_filter 
+    end
+
+`send_to` will first search for a destination with the given name, if it cannot
+find one, it will send it search for the corresponding router. It also acts as
+`return` -- halting execution of the block when it is called. This is
+implemented via an exception, which means it _may_ cause performance issues on
+things like the JVM, YMMV.
+
+when defining a root router with lenses, you must specify ":root" as the title.
+
+## Common problems, and how to solve them:
+
+### Problem: Adding a mail to an external service, and then saving it.
+
+As a user of sortah, you want to set up filters to save all email from the
+address "searchable@somewhere.net" to the folder "foobar/", as well as register
+it with the external service "RubberBandSearch". 
+
+### Solution
+
+    destination :foobar, "foobar/"
+
+    lens :search_index , :pass_through => true do
+      #code to register the email in RubberBandSearch
+      email.indexed? = true
+    end
+
+    router :index_in_rubberband, :lenses => [:search_index] do
+      send_to :foobar
+    end
+
+    router :lenses => [:spam?] do
+      send_to :devnull if email.spam? 
+      send_to :index_in_rubberband 
+    end
+
+Here we've used a `pass_through` lens to do the actual indexing, and the router
+is left as more of a proxy to call the lens. 
+
+### Problem
+
+As a user of sortah, you want to maintain a whitelist of people who should have
+their own folders, and you want those people to be subsorted in some arbitrarily 
+deep parent folders, eg:
+
+    family/ 
+      mom/
+      dad/
+      uncle_timmy/ 
+    coworkers/
+      pointy_hair/
+      dilbert/
+      old_coworkers/
+        jim/
+    personal/
+      wife/
+      friends/
+        bob/
+        mike/
+        jack/
+
+etc. Further, you'd like to only maintain the above file (or something like it), and
+not have to write new sortah code every time you move jobs or make new friends.[1]
+
+[1] Ideally, this code would maintain a directory structure for you. But as of right
+now, sortah has no aspirations to do such a thing. Each edition which _moves_ files
+in the yaml definition file will simply create new folders, it is up to the author 
+of that yaml file to keep the directory coherent with the yaml file.
+
+## Solution
+
+First, define a yaml file like the following:
+
+    personal: 
+      - name: wife
+        sender:
+          - pretty-lady-who-feeds-me@scary.com
+    family: 
+      - name: mom
+        sender:
+          - mom@hotmail.com
+          - mom@gmail.com
+      - name: dad
+        sender:
+          - dad@work.org
+    nested:
+      - name: example
+        reply-to: some_list@place.com
+      - deeper-nesting:
+        - name: deeper-nested-example
+        - reply-to: somewhere_else@overtherainbow.biz.co.uk
+    #...
+
+This yaml file will represent the directory structure, as well as provide information
+about how to determine whether the email is from that person or not.
+
+Next, you could define a class `Contact`, which could be built with the following methods:
+
+    class Contact
+      # ... contains a definition for 'path' -- which is built from the yaml file.
+      
+      def destination 
+        destination name, path
+      end
+
+      def wants?(email)
+        search_fields.any? { |f,v| email[f] =~ /#{v}/ }
+      end
+
+      def search_fields
+        #these are the key/value pairs from the YAML file which are of the form:
+        #  email-field: content_string
+        #eg:
+        #  sender: 'me@place.net'
+        #  reply-to: 'mailing-list@majordomo.com'
+        #etc
+      end
+      # ...
+    end
+
+All of this code could be bound up in a router, eg:
+
+    router :contacts do
+      contacts = Contact.load_from_file('contacts.yml') 
+      contacts.select { |c| c.wants?(email) }.first.destination
+    end
+
+Much of this is left to pseudocode, but you can see how being able to use pure-ruby
+allows for complex routes to be expressed simply.

data/Rakefile

@@ -0,0 +1,14 @@
+require 'bundler/gem_tasks'
+require 'rspec/core/rake_task'
+
+task :default => :spec
+
+desc "Run specs"
+RSpec::Core::RakeTask.new
+
+
+task :c => :console
+desc "start up a irb console"
+task :console do
+  system 'bundle exec irb'
+end

data/TUTORIAL.md

@@ -0,0 +1,43 @@
+#How to use Sortah
+
+First, I reccommend the following setup. In your `$HOME` directory, create a file
+called '.sortah/', and beneath that, create 'rc', then execute the command 
+
+    ln -s .sortah/rc .sortahrc 
+
+from your `$HOME` directory to link the `.sortahrc` file to the "real" rc file.
+Next, I would initialize a git repo (or whatever VCS you prefer) in the
+`.sortah` directory, and add your rc to it.
+
+Next, create an rvmrc file in the sortah directory with a gemset of 'sortahrc'
+(or whatever you prefer), this is where you will marshall all your dependencies
+for sortah -- at the moment, it's only going to be one. You can use bundler for
+this if you like, but if you just need vanilla sortah, it may be worth just
+using `gem` to install sortah and eliminate the bundler overhead, YMMV.
+
+Now that we've done that, we can wire up our getmailrc to point to sortah, as
+follows:
+
+    [destination]
+    type = MDA_external
+    path = $HOME/.sortah/sortah.sh
+    arguments = ("--log-errors", )
+
+Where `path` should point to your `.sortah` directory. Next, we need to create
+the starter script, `sortah.sh`, this should look like:
+
+    #!/bin/sh
+    rvm 1.9.2@sortahrc exec sortah $@
+   
+then run
+
+    chmod a+x ~/.sortah/sortah.sh
+
+This wraps the sortah executable so that we can always call it in the context of
+the sortahrc gemset -- if you install this directly to your system, then this
+shouldn't be necessary.
+
+Once you've done this, getmail should automatically use sortah to sort your
+email, now you just need to write your sortah definitions in the `~/.sortah/rc`
+file!
+