hastie 0.2.0

data/.autotest

@@ -0,0 +1,9 @@
+Autotest.add_hook :initialize do |at|
+  at.clear_mappings
+  at.add_exception(/\.git/)
+  at.add_exception(/coverage/)
+  at.add_exception(/spec\/sandbox/)
+  at.add_mapping(%r{^spec/.*_spec}) {|filename,_| at.files_matching %r{#{filename}}}
+  at.add_mapping(%r{}) {|_,_| at.files_matching %r{spec/.*_spec}}
+end
+

data/.gitignore

@@ -0,0 +1,11 @@
+Gemfile.lock
+demo/*
+coverage/*
+spec/sandbox/*
+spec/reports/*
+.DS_Store
+.config
+.bundle
+doc/*
+pkg/*
+

data/.rspec

@@ -0,0 +1,2 @@
+--color
+--format=documentation

data/Gemfile

@@ -0,0 +1,10 @@
+source 'https://rubygems.org'
+
+gemspec
+
+gem "jekyll", :git => "git://github.com/vlandham/jekyll.git"
+
+group :test do
+  gem 'fakefs', require: "fakefs/safe"
+end
+

data/LICENSE

@@ -0,0 +1,21 @@
+Copyright (c) 2011 Jim Vallandingham
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+

data/README.textile

@@ -0,0 +1,136 @@
+h1. Hastie
+
+A tool to extend Jekyll for use in report creation.
+
+h2. Using Hastie
+
+Assuming you have hastie installed, what are the steps to start using it? These steps also assume you have a compatible jekyll installed as well as a jekyll blog framework, such as bioreports. See below for more info on jekyll and bioreports.
+
+h3. hastie config
+
+First, setup your @.hastie@ config file to point to your bioreports path
+
+<pre>
+hastie config /path/to/bioreports/source -a jfv -t textile
+</pre>
+
+This creates a @.hastie@ file in your home directory. The @-a@ flag indicates the analyst (your username) to set. The @-t@ flag tells hastie to generate textile documents for your reports.
+
+h3. hastie create
+
+Now you are ready to generate a new report. Navigate to the directory that you would like the report to be created in, and create it with @hastie create@:
+
+<pre>
+cd /path/to/report/base
+hastie create analysis_of_sample_x -r ada -p hym
+</pre>
+
+This will create a new directory called @report@ with your new report inside. The flags set meta data about the report. @-r@ for the researcher this report is for and @-p@ for the PI or lead researcher of the researcher specified by @-r@. These flags are optional. This data can be easily added later to your report if you forget.
+
+Inside your report directory will be a number of files and sub-directories to get your report started. Many of these files are copied directly from the server you supplied in your config file.
+
+h3. hastie watch
+
+Now start editing your report. It will be inside your @report@ directory, and be named @date_report_name.textile@. Watch changes to your report as you modify this file using @hastie watch@:
+
+<pre>
+cd report
+vim 2011-02-04_analysis_of_sample_x.textile
+hastie watch
+</pre>
+
+Hastie watch really just starts jekyll and builds your report locally. But this helps so that you don't have to remember any new commands. 
+
+h3. hastie publish
+
+Now you have some info in your report, and you would like to publish it to the main server. That's where @hastie publish@ comes in:
+
+<pre>
+# ctrl-c to kill hastie watch
+hastie publish
+</pre>
+
+This will copy your important files to the server in your @.hastie@ file and then run jekyll there. It will also check in this change to your server's git repository.
+
+And that's all there is to creating a new report! @create@, @watch@, and @publish@.
+
+h3. NOTE
+
+Users may get an error in @read_yaml@ about an *invalid byte sequence*. To fix this, modify your @.bashrc@ to include:
+
+<pre>
+export LC_ALL=en_US.UTF-8
+export LANG=en_US.UTF-8
+</pre>
+
+This is "documented in a jekyll issue":https://github.com/mojombo/jekyll/issues/188
+
+h2. What is Hastie?
+
+Hastie works with "Jekyll":https://github.com/metalhelix/jekyll to create a powerful and flexible system by which reports can be created by different individuals and uploaded to a central 'server'.
+
+h2. Why Hastie?
+
+Say you work in an office that generates data, and reports about that data, for clients (so, for example, a core Bioinformatics facility providing results to various research labs). 
+
+The reports you generate typically come as a mix of Word documents, Excel spreadsheets, and external data files. Everyone in your office creates these reports differently. They are stored in various locations, in various folders on networked drives. They have different levels of information, and generally are a hassle to create and update.
+
+Consumers of these reports (i.e. the researchers you generated them for) have to navigate confusing and disparate network paths to find these reports. They have to have Word, Excel, and whatever other programs you have (and the right versions of these programs) installed on the computer they are using, and have to have them all open at the same time to see their report and data. 
+
+In short, there is a high cost for both encoding and decoding this information. Its a lose lose situation. 
+
+Hastie is a component of a small suite of tools to facilitate faster encoding and decoding of this data. Attempting to lower the requirements of producing and consuming reports - so producers (analysts) can generate bigger, better, reports faster and more consistent with their co-workers, and consumers (researchers) can get to their data with less hand-holding and fewer hoops to jump through.
+
+
+h2. What part does Hastie play?
+
+Hastie serves as a simple interface between the creators of the reports and the report system itself. It takes care of all the back-end silliness, and lets you create and publish reports quickly and easily.
+
+Hastie is a command-line tool that communicates with the report server (really just a jekyll source directory). It creates a framework for building up a report, allows you to develop this report locally, with whatever tools you are most comfortable with, and then easily publish this report so that it can be shared with others.
+
+h2. Hastie's associates
+
+Hastie is one piece of the puzzle for better, faster, more consistent reports. You also need:
+* "Jekyll":https://github.com/metalhelix/jekyll - a static site generator that has lots of power behind it to allow customization galore. Jekyll is built for blogs - but with Hastie's help, it makes a mighty fine report system as well.
+** Hastie requires a custom build of Jekyll that includes some missing features to be fully functional.
+** Hopefully in the future, these features will be integrated back into Jekyll proper.
+* A Jekyll site framework. Jekyll allows for plugins and layouts and other components. Hastie also expects some additional files present in its 'Jekyll Sever' to work.
+** Check out "bioreports":https://github.com/metalhexlix/bioreports for an example of such a site framework.
+
+
+h2. Hastie features
+
+<pre>
+Usage: hastie [COMMAND] <OPTIONS>
+
+Hastie Commands:
+  version - Print version number and exit
+  help    - Print this help and exit
+  create  - Create new report directory framework with given name
+  watch   - Monitor current directory for changes.
+            Updates local copy of report
+  publish - Send report to server. Update published reports
+  update  - Update local report directory with changes from report server
+  config  - Create new hastie config file for communicating with server
+  create_server - Create new server directory framework with given name
+
+create options:
+  Use 'hastie create -h' for more information on start options
+
+publish options:
+  Use 'hastie publish -h' for more information on publish options
+
+update options:
+  Use 'hastie update -h' for more information on publish options
+
+config options:
+  Use 'hastie config -h' for more information on publish options
+
+create_server options:
+  Use 'hastie create_server -h' for more information on publish options
+</pre>
+
+
+h2. Whats in a name?
+
+In Strange Case of Dr. Jekyll and Mr. Hyde, "Dr. Hastie Lanyon":http://en.wikipedia.org/wiki/Strange_Case_of_Dr_Jekyll_and_Mr_Hyde#Dr_Hastie_Lanyon finds Jekyll's ideas *too fanciful*.

data/Rakefile

@@ -0,0 +1 @@
+require 'bundler/gem_tasks'

data/bin/hastie

@@ -0,0 +1,64 @@
+#!/usr/bin/env ruby
+# -*- mode: ruby -*-
+
+require 'hastie'
+
+module Hastie
+  module Executable
+    HELP = <<-HELP
+Usage: hastie [COMMAND] <OPTIONS>
+
+Hastie Commands:
+  version - Print version number and exit
+  help    - Print this help and exit
+  create  - Create new report directory framework with given name
+  watch   - Monitor current directory for changes.
+            Updates local copy of report
+  publish - Send report to server. Update published reports
+  update  - Update local report directory with changes from report server
+  config  - Create new hastie config file for communicating with server
+  create_server - Create new server directory framework with given name
+
+create options:
+  Use 'hastie create -h' for more information on start options
+
+publish options:
+  Use 'hastie publish -h' for more information on publish options
+
+update options:
+  Use 'hastie update -h' for more information on publish options
+
+config options:
+  Use 'hastie config -h' for more information on publish options
+
+create_server options:
+  Use 'hastie create_server -h' for more information on publish options
+
+    HELP
+
+    def self.execute(command, args)
+      case command
+      when "version"
+        require 'hastie/version'
+        puts Hastie::VERSION
+      when "create", "start"
+        Hastie::ReportGenerator.start ARGV
+      when "watch", "monitor"
+        Hastie::ReportWatcher.start ARGV
+      when "publish"
+        Hastie::ReportPublisher.start ARGV
+      when "update"
+        Hastie::ReportUpdater.start ARGV
+      when "config"
+        Hastie::ConfigGenerator.start ARGV
+      when "create_server","server_start"
+        Hastie::ServerGenerator.start ARGV
+      else
+        puts Hastie::Executable::HELP
+      end
+    end
+  end
+end
+
+command = ARGV.shift
+Hastie::Executable.execute command, ARGV

data/hastie.gemspec

@@ -0,0 +1,31 @@
+# -*- encoding: utf-8 -*-
+require File.expand_path('../lib/hastie/version', __FILE__)
+
+Gem::Specification.new do |s|
+  s.add_dependency "thor", "~> 0.14.6"
+  s.add_dependency "grit", "~> 2.4.1"
+  s.add_dependency "sinatra", "~> 1.3.2"
+  s.add_dependency "thin", "~> 1.3.1"
+  s.add_development_dependency "bundler", "~> 1.0"
+  s.add_development_dependency "rdoc", "~> 3.9"
+  s.add_development_dependency "rspec", "~> 2.3"
+  s.add_development_dependency "simplecov", "~> 0.4"
+  s.add_development_dependency "ZenTest", "~> 4.5.0"
+  # s.add_development_dependency "autotest-fsevent", "~> 0.2.5"
+  # s.add_development_dependency "autotest-growl", "~> 0.2.9"
+  s.authors = ['Jim Vallandingham']
+  s.description = %q{}
+  s.email = 'none@none.com'
+  s.executables = `git ls-files -- bin/*`.split("\n").map{|f| File.basename(f)}
+  s.extra_rdoc_files = ['LICENSE', 'README.textile']
+  s.files = `git ls-files`.split("\n")
+  s.homepage = 'http://github.com/vlandham/hastie'
+  s.name = 'hastie'
+  s.rdoc_options = ['--charset=UTF-8']
+  s.require_paths = ['lib']
+  s.required_rubygems_version = Gem::Requirement.new('>= 1.3.6')
+  s.summary = s.description
+  s.test_files = `git ls-files -- {test,spec,features}/*`.split("\n")
+  s.version = Hastie::VERSION.dup
+end
+

data/lib/hastie/config_file.rb

@@ -0,0 +1,42 @@
+require 'yaml'
+require 'thor/core_ext/hash_with_indifferent_access'
+
+module Hastie
+  class ConfigFile
+
+    # takes yaml filename
+    # outputs thor's hash with indifferent access of content
+    # if input file cannot be read, empty thor hash is returned
+    def self.load filename, root = nil
+      if !File.exists? filename
+        return
+      end
+      output = Thor::CoreExt::HashWithIndifferentAccess.new()
+      config = YAML.load(File.read(filename))
+      if config
+        if root
+          config = {root => config}
+        end
+        output = Thor::CoreExt::HashWithIndifferentAccess.new(config)
+      end
+      output
+    end
+
+    def self.write filename, data
+      output_data = data.to_hash.to_yaml
+      File.open(filename, 'w') {|file| file.puts output_data}
+    end
+
+    # very lazy. very limited
+    # only use for _reports.yml
+    def self.append filename, data
+      if !File.exists? filename
+        puts "ERROR: #{filename} not found"
+        return
+      end
+      File.open(filename, 'a') do |file|
+        file.puts "- #{data}"
+      end
+    end
+  end
+end

data/lib/hastie/config_generator.rb

@@ -0,0 +1,37 @@
+require 'fileutils'
+require 'thor/group'
+require 'hastie/constants'
+module Hastie
+  class ConfigGenerator < Thor::Group
+    include Thor::Actions
+
+    def self.banner
+      "hastie config [SERVER_ROOT] <OPTIONS>"
+    end
+
+    argument :server_root, :type => :string, :desc => "Root path of server location"
+    class_option :path, :aliases => "-p", :desc => "Root directory of where the config file will be written to", :default => File.expand_path("~")
+    class_option :name, :aliases => "-n", :desc => "Name of the config file", :default => ".hastie"
+    class_option :analyst, :aliases => "-a", :desc => "Analyst for reports generated with this config file"
+    class_option :type, :aliases => "-t", :desc => "Default format of reports to generate", :default => "textile"
+
+    def self.source_root
+      File.dirname(__FILE__)
+    end
+
+
+    def check_server_root
+      if !File.directory? server_root
+        say_status "error", "#{server_root} is not a directory", :red
+        say_status "error", "Please use full path to server directory", :red
+        exit(1)
+      end
+    end
+
+    def create_config_file
+      output_file = File.join(options[:path], options[:name])
+      template "templates/hastie_config.tt", output_file
+    end
+
+  end
+end

data/lib/hastie/constants.rb

@@ -0,0 +1,25 @@
+module Hastie
+  CONFIG_FILE = File.expand_path(File.join("~", ".hastie"))
+  SERVER_REPORTS_FILE = "_reports.yml"
+  SERVER_CONFIG_FILE = "_config.yml"
+  SERVER_PUBLISH_CONFIG_FILE = "_server_config.yml"
+
+  DATA_ROOT = "data"
+  REPORT_CONFIG_FILE = "report.yml"
+
+  def self.config_file
+    CONFIG_FILE
+  end
+
+  def self.report_config_name
+    REPORT_CONFIG_FILE
+  end
+
+  def self.watch_config_file
+    SERVER_CONFIG_FILE
+  end
+
+  def self.publish_config_file
+    SERVER_PUBLISH_CONFIG_FILE
+  end
+end

data/lib/hastie/report_generator.rb

@@ -0,0 +1,118 @@
+require 'fileutils'
+require 'hastie/config_file'
+require 'hastie/constants'
+require 'hastie/server_reader'
+
+module Hastie
+  class ReportGenerator < ServerReader
+    def self.banner
+      "hastie create [NAME] <OPTIONS>"
+    end
+
+    attr_accessor :report_id, :title, :analyst, :researcher, :pi
+
+    desc "Creates framework for new report"
+    argument :name, :type => :string, :desc => "The name of the new report. no spaces"
+    class_option :type, :aliases => "-t", :desc => "Type of report to generate"
+    class_option :analyst, :aliases => "-a", :desc => "Analyst generating the report"
+    class_option :researcher, :aliases => "-r", :desc => "Researcher the report is for"
+    class_option :pi, :aliases => "-p", :desc => "PI the researcher is under"
+    class_option :date, :aliases => "-d", :desc => "Date to use in report filename. Ex: 2011-11-29", :default => "#{Time.now.strftime('%Y-%m-%d')}"
+    class_option :output, :aliases => "-o", :desc => "Output Directory for report", :default => File.join(".", "report")
+
+    def setup_variables
+      options[:type] ||= "textile"
+      options[:date] ||= "#{Time.now.strftime('%Y-%m-%d')}"
+      options[:name] = File.basename(name)
+      self.title = options[:name].gsub("_", " ").capitalize
+      options[:title] = self.title
+      # report_id will be used internally in case the name turns
+      # out to be too loose to use
+      self.report_id = File.basename(name)
+      options[:report_id] = self.report_id
+      self.destination_root = options[:output]
+      say_status "note", "root: #{self.destination_root}"
+
+      options[:analyst] ||= "unknown"
+      self.analyst = options[:analyst] || "unknown"
+      options[:researcher] ||= "unknown"
+      options[:pi] ||= "unknown"
+      options[:data_dir] ||= data_dir
+    end
+
+    def check_name_availible
+      if options[:published_reports] and options[:published_reports].include? report_id
+        say_status "error", "Sorry, the #{report_id} is already a published report", :red
+        say_status "error", "Please run again with a different name", :red
+        exit(1)
+      end
+    end
+
+    def check_destination
+      if File.exists? self.destination_root
+        say_status "error", "#{self.destination_root} already exists.", :red
+        say_status "error", "please choose different output directory", :red
+        exit(1)
+      end
+    end
+
+    def create_report_file
+      say_status "create", "report: #{options[:report_id]}"
+      extension = determine_extension(options[:type])
+      options[:extension] = extension
+      template_file = "templates/report.#{extension}.tt"
+      report_filename = "#{options[:date]}-#{report_id}.#{extension}"
+      say_status  "note", "report file: #{report_filename}"
+      template template_file, report_filename
+      options[:report_file] = report_filename
+      options[:report_file_generated] = File.basename(report_filename, File.extname(report_filename)) + ".html"
+    end
+
+    def create_index_file
+      template_file = "templates/report_index.html.tt"
+      template template_file, "index.html"
+    end
+
+    def create_data_dir
+      create_file File.join(data_dir, ".gitignore"), :verbose => true
+    end
+
+    def fetch_static_files
+      options[:server]["static"] ||= []
+      options[:server]["static"].each do |static_file|
+        static_path = File.join(options[:server_root], static_file)
+        if File.exists? static_path
+          say_status "copy", "#{static_path} to #{File.basename(destination_root)}"
+          FileUtils.cp_r static_path, self.destination_root
+        end
+      end
+    end
+
+    def write_config_file
+      output_config_file = File.join(self.destination_root, Hastie.report_config_name)
+      say_status "write", "#{File.basename(output_config_file)}"
+      ConfigFile.write(output_config_file, options)
+    end
+
+    no_tasks do
+      def determine_extension report_type
+        extension = case report_type.to_sym
+                    when :markdown,:md
+                      "markdown"
+                    when :textile
+                      "textile"
+                    when :html,:htm
+                      "html"
+                    else
+                      say "WARNING: #{report_type} not a valid type. Defaulting to textile"
+                      "textile"
+                    end
+        extension
+      end
+
+      def data_dir
+        File.join(DATA_ROOT, options[:report_id])
+      end
+    end
+  end
+end

data/lib/hastie/report_publisher.rb

@@ -0,0 +1,162 @@
+require 'fileutils'
+require 'hastie/config_file'
+require 'hastie/constants'
+require 'hastie/server_reader'
+
+module Hastie
+  class ReportPublisher < ServerReader
+    def self.banner
+      "hastie publish <REPORT_DIR> <OPTIONS>"
+    end
+
+    attr_accessor :report_dir
+    desc "Publishes report to server"
+    argument :name, :type => :string, :default => ".", :desc => "The name report directory"
+
+    def read_report_file
+      self.report_dir = File.expand_path(name)
+      report_config_file = File.join(report_dir, Hastie.report_config_name)
+      if !File.exists? report_config_file
+        say "Cannot locate #{Hastie.report_config_name}."
+        say "Current directory is not a report."
+        exit(1)
+      end
+      # we get the report filename from this config file
+      local_config = ConfigFile.load(report_config_file, :local)
+      self.options = local_config.merge(self.options)
+    end
+
+    def check_options
+      all_valid = true
+      required_server_options = ["reports_dir"]
+      required_server_options.each do |option|
+        if !options[:server] or !options[:server][option]
+          say_status "error", "Missing #{option} option from server config", :red
+          all_valid = false
+        end
+      end
+
+      required_local_options = ["report_file", "report_id"]
+      required_local_options.each do |option|
+        if !options[:local] or !options[:local][option]
+          say_status "error", "Missing #{option} option from local config", :red
+          all_valid = false
+        end
+      end
+
+      if !all_valid
+        exit(1)
+      end
+    end
+
+    def set_destination_directory
+      self.destination_root = File.join(options[:server_root])
+      say_status "note", "root: #{self.destination_root}"
+    end
+
+    def copy_report_file
+      report_filename = options[:local]["report_file"]
+      local_report = File.join(report_dir, report_filename)
+      destination_report = File.join(options[:server_root], options[:server]["reports_dir"], report_filename)
+
+      if File.exists? destination_report
+        say_status "removing", destination_report, :yellow
+        command = "rm -f #{destination_report}"
+        pid = Process.fork do
+          exec(command)
+        end
+        Process.waitpid(pid)
+      end
+
+      if File.exists? local_report
+        say_status "publishing", report_filename
+        # command = "cp #{local_report} #{destination_report}"
+        # pid = Process.fork do
+        #   exec(command)
+        # end
+        # Process.waitpid(pid)
+        FileUtils.cp local_report, destination_report
+      else
+        say_status "error", "Report file not found: #{report_filename}", :red
+        exit(1)
+      end
+    end
+
+    def copy_data_directory
+      data_dir = File.join(report_dir, DATA_ROOT, options[:local]["report_id"])
+      destination_dir = File.join(options[:server_root], DATA_ROOT)
+
+      existing_destination_dir = File.join(destination_dir, options[:local]["report_id"])
+
+      if File.exists? existing_destination_dir
+        say_status "removing", existing_destination_dir, :yellow
+        command = "rm -rf #{existing_destination_dir}"
+        pid = Process.fork do
+          exec(command)
+        end
+        Process.waitpid(pid)
+      end
+
+      if File.exists? data_dir
+        say_status "publishing", data_dir
+        # command = "cp -r #{data_dir} #{destination_dir}"
+        # pid = Process.fork do
+        #   exec(command)
+        # end
+        # Process.waitpid(pid)
+        FileUtils.cp_r data_dir, destination_dir
+      else
+        say_status "warning", "report data directory not found #{data_dir}", :yellow
+      end
+    end
+
+    def add_to_reports_file
+      in_root do
+        say_status "note", "modifying #{SERVER_REPORTS_FILE}"
+        say_status "note", " to include #{options[:local]["report_id"]}"
+        server_report_file = SERVER_REPORTS_FILE
+        if !File.exists? server_report_file
+          say_status "error", "Cannot find #{SERVER_REPORTS_FILE} file in server directory:", :red
+        end
+        ConfigFile.append(server_report_file, options[:local]["report_id"])
+      end
+    end
+
+    def update_git_repo
+      in_root do
+        say_status "note", "updating git repository"
+        Grit::Git.git_timeout = 25
+        repo = Grit::Repo.new(".")
+        # ensure we are on the server branch
+        repo.git.native :checkout, {}, 'server'
+        repo = Grit::Repo.new(".")
+        if repo.head.name != "server"
+          say_status "error", "Remote git not on server branch", :red
+          say_status "error", "Please git checkout server", :red
+          say_status "error", "Current branch: #{repo.head.name}", :red
+          exit(1)
+        end
+        all_files = Dir.glob("./**")
+        repo.add(all_files)
+        repo.commit_all("update with report: #{options[:local]["report_id"]}")
+      end
+    end
+
+    def publish_with_jekyll
+      in_root do
+        say_status "publishing", "updating server reports", :yellow
+        config_file = File.expand_path(File.join(FileUtils.getwd, Hastie.watch_config_file))
+        server_config_file = File.expand_path(File.join(FileUtils.getwd, Hastie.publish_config_file))
+        if File.exists?(server_config_file)
+          config_file = server_config_file
+        end
+        say_status "config", config_file, :yellow
+        command = "jekyll --config #{config_file}"
+        pid = Process.fork do
+          exec(command)
+        end
+        Process.waitpid(pid)
+      end
+    end
+  end
+end