dkastner-rocco 0.8

data/CHANGES.md

@@ -0,0 +1,63 @@
+CHANGES
+=======
+
+0.6 (2011-03-05)
+----------------
+
+This release brought to you almost entirely
+by [mikewest](http://github.com/mikewest).
+
+### Features
+
+* Added `-t`/`--template` CLI option that allows you to specify a Mustache
+  template that ought be used when rendering the final documentation.
+  (Issue #16)
+
+* More variables in templates:
+  * `docs?`:    True if `docs` contains text of any sort, False if it's empty.
+  * `code?`:    True if `code` contains text of any sort, False if it's empty.
+  * `empty?`:   True if both `code` and `docs` are empty.  False otherwise.
+  * `header?`:  True if `docs` contains _only_ a HTML header.  False otherwise.
+
+* Test suite!  (Run `rake test`)
+
+* Autodetect file's language if Pygments is installed locally (Issue #19)
+
+* Autopopulate comment characters for known languages (Issue #20)
+
+* Correctly parse block comments (Issue #22)
+
+* Stripping encoding definitions from Ruby and Python files in the same
+  way we strip shebang lines (Issue #21)
+
+* Adjusting section IDs to contain descriptive test from headers.  A header
+  section's ID might be `section-Header_text_goes_here` for friendlier URLs.
+  Other section IDs will remain the same (`section-2` will stay
+  `section-2`). (Issue #28)
+
+### Bugs Fixed
+
+* Docco's CSS changed: we updated Rocco's HTML accordingly, and pinned
+  the CSS file to Docco's 0.3.0 tag.  (Issues #12 and #23)
+
+* Fixed code highlighting for shell scripts (among others) (Issue #13)
+
+* Fixed buggy regex for comment char stripping (Issue #15)
+
+* Specifying UTF-8 encoding for Pygments (Issue #10)
+
+* Extensionless file support (thanks to [Vasily Polovnyov][vast] for the
+  fix!) (Issue #24)
+
+* Fixing language support for Pygments webservice (Issue #11)
+
+* The source jumplist now generates correctly relative URLs (Issue #26)
+
+* Fixed an issue with using mustache's `template_path=` incorrectly.
+
+[vast]: https://github.com/vast
+
+0.5
+---
+
+Rocco 0.5 emerged from the hazy mists, complete and unfettered by history.

data/COPYING

@@ -0,0 +1,18 @@
+Copyright (c) 2010 Ryan Tomayko <http://tomayko.com/about>
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to
+deal in the Software without restriction, including without limitation the
+rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+sell copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER 
+IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README

@@ -0,0 +1,23 @@
+
+
+                  ___       ___       ___       ___       ___
+                 /\  \     /\  \     /\  \     /\  \     /\  \
+                /::\  \   /::\  \   /::\  \   /::\  \   /::\  \
+               /::\:\__\ /:/\:\__\ /:/\:\__\ /:/\:\__\ /:/\:\__\
+               \;:::/  / \:\/:/  / \:\ \/__/ \:\ \/__/ \:\/:/  /
+                |:\/__/   \::/  /   \:\__\    \:\__\    \::/  /
+                 \|__|     \/__/     \/__/     \/__/     \/__/
+
+
+
+    Rocco is  a quick-and-dirty,  literate-programming-style documentation
+    generator for Ruby. See the Rocco generated docs for more information:
+
+                    <http://rtomayko.github.com/rocco/>
+
+
+    Rocco is a port of,  and borrows heavily from, Docco  -- the original
+    quick-and-dirty,   hundred-line-long,   literate-programming-style
+    documentation generator in CoffeeScript:
+
+                    <http://jashkenas.github.com/docco/>

data/Rakefile

@@ -0,0 +1,115 @@
+$LOAD_PATH.unshift 'lib'
+
+require 'rake/testtask'
+require 'rake/clean'
+
+task :default => [:sup, :docs, :test]
+
+desc 'Holla'
+task :sup do
+  verbose do
+    lines = File.read('README').split("\n")[0,12]
+    lines.map! { |line| line[15..-1] }
+    puts lines.join("\n")
+  end
+end
+
+desc 'Run tests (default)'
+Rake::TestTask.new(:test) do |t|
+  t.test_files = FileList['test/suite.rb']
+  t.ruby_opts = ['-rubygems'] if defined? Gem
+end
+
+# Bring in Rocco tasks
+require 'rocco/tasks'
+Rocco::make 'docs/'
+
+desc 'Build rocco docs'
+task :docs => :rocco
+directory 'docs/'
+
+desc 'Build docs and open in browser for the reading'
+task :read => :docs do
+  sh 'open docs/lib/rocco.html'
+end
+
+# Make index.html a copy of rocco.html
+file 'docs/index.html' => 'docs/lib/rocco.html' do |f|
+  cp 'docs/lib/rocco.html', 'docs/index.html', :preserve => true
+end
+task :docs => 'docs/index.html'
+CLEAN.include 'docs/index.html'
+
+# Alias for docs task
+task :doc => :docs
+
+# GITHUB PAGES ===============================================================
+
+desc 'Update gh-pages branch'
+task :pages => ['docs/.git', :docs] do
+  rev = `git rev-parse --short HEAD`.strip
+  Dir.chdir 'docs' do
+    sh "git add *.html"
+    sh "git commit -m 'rebuild pages from #{rev}'" do |ok,res|
+      if ok
+        verbose { puts "gh-pages updated" }
+        sh "git push -q o HEAD:gh-pages"
+      end
+    end
+  end
+end
+
+# Update the pages/ directory clone
+file 'docs/.git' => ['docs/', '.git/refs/heads/gh-pages'] do |f|
+  sh "cd docs && git init -q && git remote add o ../.git" if !File.exist?(f.name)
+  sh "cd docs && git fetch -q o && git reset -q --hard o/gh-pages && touch ."
+end
+CLOBBER.include 'docs/.git'
+
+# PACKAGING =================================================================
+
+if defined?(Gem)
+  SPEC = eval(File.read('rocco.gemspec'))
+
+  def package(ext='')
+    "pkg/rocco-#{SPEC.version}" + ext
+  end
+
+  desc 'Build packages'
+  task :package => %w[.gem .tar.gz].map {|e| package(e)}
+
+  desc 'Build and install as local gem'
+  task :install => package('.gem') do
+    sh "gem install #{package('.gem')}"
+  end
+
+  directory 'pkg/'
+
+  file package('.gem') => %w[pkg/ rocco.gemspec] + SPEC.files do |f|
+    sh "gem build rocco.gemspec"
+    mv File.basename(f.name), f.name
+  end
+
+  file package('.tar.gz') => %w[pkg/] + SPEC.files do |f|
+    sh "git archive --format=tar HEAD | gzip > #{f.name}"
+  end
+end
+
+# GEMSPEC ===================================================================
+
+file 'rocco.gemspec' => FileList['{lib,test,bin}/**','Rakefile'] do |f|
+  version = File.read('lib/rocco.rb')[/VERSION = '(.*)'/] && $1
+  date = Time.now.strftime("%Y-%m-%d")
+  spec = File.
+    read(f.name).
+    sub(/s\.version\s*=\s*'.*'/, "s.version = '#{version}'")
+  parts = spec.split("  # = MANIFEST =\n")
+  files = `git ls-files`.
+    split("\n").sort.reject{ |file| file =~ /^\./ }.
+    map{ |file| "    #{file}" }.join("\n")
+  parts[1] = "  s.files = %w[\n#{files}\n  ]\n"
+  spec = parts.join("  # = MANIFEST =\n")
+  spec.sub!(/s.date = '.*'/, "s.date = '#{date}'")
+  File.open(f.name, 'w') { |io| io.write(spec) }
+  puts "#{f.name} #{version} (#{date})"
+end

data/bin/rocco

@@ -0,0 +1,100 @@
+#!/usr/bin/env ruby
+#/ Usage: rocco [-l <lang>] [-c <chars>] [-o <dir>] <file>...
+#/ Generate literate-programming-style documentation for Ruby source <file>s.
+#/
+#/ Options:
+#/   -l, --language=<lang>  The Pygments lexer to use to highlight code
+#/   -c, --comment-chars=<chars>
+#/                          The string to recognize as a comment marker
+#/   -o, --output=<dir>     Directory where generated HTML files are written
+#/   -t, --template=<path>  The file to use as template when rendering HTML
+#/   -d, --docblocks        Parse Docblock @annotations in comments
+#/       --help             Show this help message
+
+require 'optparse'
+require 'fileutils'
+
+# Write usage message to stdout and exit.
+def usage(stream=$stderr, status=1)
+  stream.puts File.readlines(__FILE__).
+    grep(/^#\//).
+    map { |line| line.sub(/^#. ?/, '') }.
+    join
+  exit status
+end
+
+# Like `Kernel#abort` but writes a note encouraging the user to consult
+# `rocco --help` for more information.
+def abort_with_note(message=nil)
+  $stderr.puts message if message
+  abort "See `rocco --help' for usage information."
+end
+
+# Parse command line options, aborting if anything goes wrong.
+output_dir = '.'
+sources = []
+options = {}
+ARGV.options { |o|
+  o.program_name = File.basename($0)
+  o.on("-o", "--output=DIR") { |dir| output_dir = dir }
+  o.on("-l", "--language=LANG") { |lang| options[:language] = lang }
+  o.on("-c", "--comment-chars=CHARS") { |chars| options[:comment_chars] = Regexp.escape(chars) }
+  o.on("-t", "--template=TEMPLATE") { |template| options[:template_file] = template }
+  o.on("-d", "--docblocks") { options[:docblocks] = true }
+  o.on_tail("-h", "--help") { usage($stdout, 0) }
+  o.parse!
+} or abort_with_note
+
+# Use http://pygments.appspot.com in case `pygmentize(1)` isn't available.
+if ! ENV['PATH'].split(':').any? { |dir| File.exist?("#{dir}/pygmentize") }
+  unless options[:webservice]
+    $stderr.puts "pygmentize not in PATH; using pygments.appspot.com instead"
+    options[:webservice] = true
+  end
+end
+
+# Eat sources from ARGV.
+sources << ARGV.shift while ARGV.any?
+
+# Make sure we have some files to work with.
+if sources.empty?
+  abort_with_note "#{File.basename($0)}: no input <file>s given"
+end
+
+# What a fucking mess. Most of this is duplicated in rocco.rb too.
+libdir = File.expand_path('../../lib', __FILE__).sub(/^#{Dir.pwd}\//, '')
+begin
+  require 'rdiscount'
+  require 'rocco'
+rescue LoadError
+  case $!.to_s
+  when /rdiscount/
+    if !defined?(Gem)
+      warn "warn: #$!. trying again with rubygems"
+      require 'rubygems'
+      retry
+    else
+      require 'bluecloth'
+      Markdown = BlueCloth
+      $LOADED_FEATURES << 'rdiscount.rb'
+      retry
+    end
+  when /rocco/
+    if !$:.include?(libdir)
+      warn "warn: #$!. trying again with #{libdir} on load path"
+      $:.unshift(libdir)
+      retry
+    end
+  end
+  raise
+end
+
+# Run each file through Rocco and write output.
+sources.each do |filename|
+  rocco = Rocco.new(filename, sources, options)
+  dest = filename.sub(Regexp.new("#{File.extname(filename)}$"),".html")
+  dest = File.join(output_dir, dest) if output_dir != '.'
+  puts "rocco: #{filename} -> #{dest}"
+  FileUtils.mkdir_p File.dirname(dest)
+  File.open(dest, 'wb') { |fd| fd.write(rocco.to_html) }
+end

data/lib/rocco.rb

@@ -0,0 +1,526 @@
+# **Rocco** is a Ruby port of [Docco][do], the quick-and-dirty,
+# hundred-line-long, literate-programming-style documentation generator.
+#
+# Rocco reads Ruby source files and produces annotated source documentation
+# in HTML format. Comments are formatted with [Markdown][md] and presented
+# alongside syntax highlighted code so as to give an annotation effect.
+# This page is the result of running Rocco against [its own source file][so].
+#
+# Most of this was written while waiting for [node.js][no] to build (so I
+# could use Docco!). Docco's gorgeous HTML and CSS are taken verbatim.
+# The main difference is that Rocco is written in Ruby instead of
+# [CoffeeScript][co] and may be a bit easier to obtain and install in
+# existing Ruby environments or where node doesn't run yet.
+#
+# Install Rocco with Rubygems:
+#
+#     gem install rocco
+#
+# Once installed, the `rocco` command can be used to generate documentation
+# for a set of Ruby source files:
+#
+#     rocco lib/*.rb
+#
+# The HTML files are written to the current working directory.
+#
+# [no]: http://nodejs.org/
+# [do]: http://jashkenas.github.com/docco/
+# [co]: http://coffeescript.org/
+# [md]: http://daringfireball.net/projects/markdown/
+# [so]: http://github.com/rtomayko/rocco/blob/master/lib/rocco.rb#commit
+
+#### Prerequisites
+
+# We'll need a Markdown library. [RDiscount][rd], if we're lucky. Otherwise,
+# issue a warning and fall back on using BlueCloth.
+#
+# [rd]: http://github.com/rtomayko/rdiscount
+begin
+  require 'rdiscount'
+rescue LoadError => boom
+  warn "WARNING: #{boom}. Trying bluecloth."
+  require 'bluecloth'
+  Markdown = BlueCloth
+end
+
+# We use [{{ mustache }}](http://defunkt.github.com/mustache/) for
+# HTML templating.
+require 'mustache'
+
+# We use `Net::HTTP` to highlight code via <http://pygments.appspot.com>
+require 'net/http'
+
+# Code is run through [Pygments](http://pygments.org/) for syntax
+# highlighting. If it's not installed, locally, use a webservice.
+if !ENV['PATH'].split(':').any? { |dir| File.executable?("#{dir}/pygmentize") }
+  warn "WARNING: Pygments not found. Using webservice."
+end
+
+#### Public Interface
+
+# `Rocco.new` takes a source `filename`, an optional list of source filenames
+# for other documentation sources, an `options` hash, and an optional `block`.
+# The `options` hash respects three members:
+#
+# * `:language`: specifies which Pygments lexer to use if one can't be
+#   auto-detected from the filename.  _Defaults to `ruby`_.
+#
+# * `:comment_chars`, which specifies the comment characters of the
+#   target language. _Defaults to `#`_.
+#
+# * `:template_file`, which specifies a external template file to use
+#   when rendering the final, highlighted file via Mustache.  _Defaults
+#   to `nil` (that is, Mustache will use `./lib/rocco/layout.mustache`)_.
+#
+class Rocco
+  VERSION = '0.6'
+
+  def initialize(filename, sources=[], options={}, &block)
+    @file       = filename
+    @sources    = sources
+
+    # When `block` is given, it must read the contents of the file using
+    # whatever means necessary and return it as a string. With no `block`,
+    # the file is read to retrieve data.
+    @data =
+      if block_given?
+        yield
+      else
+        File.read(filename)
+      end
+
+    defaults = {
+      :language      => 'ruby',
+      :comment_chars => '#',
+      :template_file => nil
+    }
+    @options = defaults.merge(options)
+
+    # If we detect a language
+    if detect_language() != "text"
+      # then assign the detected language to `:language`, and look for
+      # comment characters based on that language
+      @options[:language] = detect_language()
+      @options[:comment_chars] = generate_comment_chars()
+
+    # If we didn't detect a language, but the user provided one, use it
+    # to look around for comment characters to override the default.
+    elsif @options[:language] != defaults[:language]
+      @options[:comment_chars] = generate_comment_chars()
+
+    # If neither is true, then convert the default comment character string
+    # into the comment_char syntax (we'll discuss that syntax in detail when
+    # we get to `generate_comment_chars()` in a moment.
+    else
+      @options[:comment_chars] = {
+        :single => @options[:comment_chars],
+        :multi => nil
+      }
+    end
+
+    # Turn `:comment_chars` into a regex matching a series of spaces, the
+    # `:comment_chars` string, and the an optional space.  We'll use that
+    # to detect single-line comments.
+    @comment_pattern =
+      Regexp.new("^\\s*#{@options[:comment_chars][:single]}\s?")
+
+    # `parse()` the file contents stored in `@data`.  Run the result through
+    # `split()` and that result through `highlight()` to generate the final
+    # section list.
+    @sections = highlight(split(parse(@data)))
+  end
+
+  # The filename as given to `Rocco.new`.
+  attr_reader :file
+
+  # The merged options array
+  attr_reader :options
+
+  # A list of two-tuples representing each *section* of the source file. Each
+  # item in the list has the form: `[docs_html, code_html]`, where both
+  # elements are strings containing the documentation and source code HTML,
+  # respectively.
+  attr_reader :sections
+
+  # A list of all source filenames included in the documentation set. Useful
+  # for building an index of other files.
+  attr_reader :sources
+
+  # Generate HTML output for the entire document.
+  require 'rocco/layout'
+  def to_html
+    Rocco::Layout.new(self, @options[:template_file]).render
+  end
+
+  # Helper Functions
+  # ----------------
+
+  # Returns `true` if `pygmentize` is available locally, `false` otherwise.
+  def pygmentize?
+    @_pygmentize ||= ENV['PATH'].split(':').
+      any? { |dir| File.executable?("#{dir}/pygmentize") }
+  end
+
+  # If `pygmentize` is available, we can use it to autodetect a file's
+  # language based on its filename.  Filenames without extensions, or with
+  # extensions that `pygmentize` doesn't understand will return `text`.
+  # We'll also return `text` if `pygmentize` isn't available.
+  #
+  # We'll memoize the result, as we'll call this a few times.
+  def detect_language
+    @_language ||=
+      if pygmentize?
+        %x[pygmentize -N #{@file}].strip.split('+').first
+      else
+        "text"
+      end
+  end
+
+  # Given a file's language, we should be able to autopopulate the
+  # `comment_chars` variables for single-line comments.  If we don't
+  # have comment characters on record for a given language, we'll
+  # use the user-provided `:comment_char` option (which defaults to
+  # `#`).
+  #
+  # Comment characters are listed as:
+  #
+  #     { :single       => "//",
+  #       :multi_start  => "/**",
+  #       :multi_middle => "*",
+  #       :multi_end    => "*/" }
+  #
+  # `:single` denotes the leading character of a single-line comment.
+  # `:multi_start` denotes the string that should appear alone on a
+  # line of code to begin a block of documentation.  `:multi_middle`
+  # denotes the leading character of block comment content, and
+  # `:multi_end` is the string that ought appear alone on a line to
+  # close a block of documentation.  That is:
+  #
+  #     /**                 [:multi][:start]
+  #      *                  [:multi][:middle]
+  #      ...
+  #      *                  [:multi][:middle]
+  #      */                 [:multi][:end]
+  #
+  # If a language only has one type of comment, the missing type
+  # should be assigned `nil`.
+  #
+  # At the moment, we're only returning `:single`.  Consider this
+  # groundwork for block comment parsing.
+  C_STYLE_COMMENTS = {
+    :single => "//",
+    :multi  => { :start => "/**", :middle => "*", :end => "*/" },
+    :heredoc => nil
+  }
+  COMMENT_STYLES  = {
+    "bash"          =>  { :single => "#", :multi => nil },
+    "c"             =>  C_STYLE_COMMENTS,
+    "coffee-script" =>  {
+      :single => "#",
+      :multi  => { :start => "###", :middle => nil, :end => "###" },
+      :heredoc => nil
+    },
+    "cpp" =>  C_STYLE_COMMENTS,
+    "csharp" => C_STYLE_COMMENTS,
+    "css"           =>  {
+      :single => nil,
+      :multi  => { :start => "/**", :middle => "*", :end => "*/" },
+      :heredoc => nil
+    },
+    "html"           =>  {
+      :single => nil,
+      :multi => { :start => '<!--', :middle => nil, :end => '-->' },
+      :heredoc => nil
+    },
+    "java"          =>  C_STYLE_COMMENTS,
+    "js"            =>  C_STYLE_COMMENTS,
+    "lua"           =>  {
+      :single => "--",
+      :multi => nil,
+      :heredoc => nil
+    },
+    "php" => C_STYLE_COMMENTS,
+    "python"        =>  {
+      :single => "#",
+      :multi  => { :start => '"""', :middle => nil, :end => '"""' },
+      :heredoc => nil
+    },
+    "rb"            =>  {
+      :single => "#",
+      :multi  => { :start => '=begin', :middle => nil, :end => '=end' },
+      :heredoc => "<<-"
+    },
+    "scheme"        =>  { :single => ";;",  :multi => nil, :heredoc => nil },
+    "xml"           =>  {
+      :single => nil,
+      :multi => { :start => '<!--', :middle => nil, :end => '-->' },
+      :heredoc => nil
+    },
+  }
+
+  def generate_comment_chars
+    @_commentchar ||=
+      if COMMENT_STYLES[@options[:language]]
+        COMMENT_STYLES[@options[:language]]
+      else
+        { :single => @options[:comment_chars], :multi => nil, :heredoc => nil }
+      end
+  end
+
+  # Internal Parsing and Highlighting
+  # ---------------------------------
+
+  # Parse the raw file data into a list of two-tuples. Each tuple has the
+  # form `[docs, code]` where both elements are arrays containing the
+  # raw lines parsed from the input file, comment characters stripped.
+  def parse(data)
+    sections = []
+    docs, code = [], []
+    lines = data.split("\n")
+
+    # The first line is ignored if it is a shebang line.  We also ignore the
+    # PEP 263 encoding information in python sourcefiles, and the similar ruby
+    # 1.9 syntax.
+    lines.shift if lines[0] =~ /^\#\!/
+    lines.shift if lines[0] =~ /coding[:=]\s*[-\w.]+/ &&
+                   [ "python", "rb" ].include?(@options[:language])
+
+    # To detect both block comments and single-line comments, we'll set
+    # up a tiny state machine, and loop through each line of the file.
+    # This requires an `in_comment_block` boolean, and a few regular
+    # expressions for line tests.  We'll do the same for fake heredoc parsing.
+    in_comment_block = false
+    in_heredoc = false
+    single_line_comment, block_comment_start, block_comment_mid, block_comment_end =
+      nil, nil, nil, nil
+    if not @options[:comment_chars][:single].nil?
+      single_line_comment = Regexp.new("^\\s*#{Regexp.escape(@options[:comment_chars][:single])}\\s?")
+    end
+    if not @options[:comment_chars][:multi].nil?
+      block_comment_start = Regexp.new("^\\s*#{Regexp.escape(@options[:comment_chars][:multi][:start])}\\s*$")
+      block_comment_end   = Regexp.new("^\\s*#{Regexp.escape(@options[:comment_chars][:multi][:end])}\\s*$")
+      block_comment_one_liner = Regexp.new("^\\s*#{Regexp.escape(@options[:comment_chars][:multi][:start])}\\s*(.*?)\\s*#{Regexp.escape(@options[:comment_chars][:multi][:end])}\\s*$")
+      block_comment_start_with = Regexp.new("^\\s*#{Regexp.escape(@options[:comment_chars][:multi][:start])}\\s*(.*?)$")
+      block_comment_end_with = Regexp.new("\\s*(.*?)\\s*#{Regexp.escape(@options[:comment_chars][:multi][:end])}\\s*$")
+      if @options[:comment_chars][:multi][:middle]
+        block_comment_mid = Regexp.new("^\\s*#{Regexp.escape(@options[:comment_chars][:multi][:middle])}\\s?")
+      end
+    end
+    if not @options[:comment_chars][:heredoc].nil?
+      heredoc_start = Regexp.new("#{Regexp.escape(@options[:comment_chars][:heredoc])}(\\S+)$")
+    end
+    lines.each do |line|
+      # If we're currently in a comment block, check whether the line matches
+      # the _end_ of a comment block or the _end_ of a comment block with a
+      # comment.
+      if in_comment_block
+        if block_comment_end && line.match(block_comment_end)
+          in_comment_block = false
+        elsif block_comment_end_with && line.match(block_comment_end_with)
+          in_comment_block = false
+          docs << line.match(block_comment_end_with).captures.first.
+                        sub(block_comment_mid || '', '')
+        else
+          docs << line.sub(block_comment_mid || '', '')
+        end
+      # If we're currently in a heredoc, we're looking for the end of the
+      # heredoc, and everything it contains is code.
+      elsif in_heredoc
+        if line.match(Regexp.new("^#{Regexp.escape(in_heredoc)}$"))
+          in_heredoc = false
+        end
+        code << line
+      # Otherwise, check whether the line starts a heredoc. If so, note the end
+      # pattern, and the line is code.  Otherwise check whether the line matches
+      # the beginning of a block, or a single-line comment all on it's lonesome.
+      # In either case, if there's code, start a new section.
+      else
+        if heredoc_start && line.match(heredoc_start)
+          in_heredoc = $1
+          code << line
+        elsif block_comment_one_liner && line.match(block_comment_one_liner)
+          if code.any?
+            sections << [docs, code]
+            docs, code = [], []
+          end
+          docs << line.match(block_comment_one_liner).captures.first
+        elsif block_comment_start && line.match(block_comment_start)
+          in_comment_block = true
+          if code.any?
+            sections << [docs, code]
+            docs, code = [], []
+          end
+        elsif block_comment_start_with && line.match(block_comment_start_with)
+          in_comment_block = true
+          if code.any?
+            sections << [docs, code]
+            docs, code = [], []
+          end
+          docs << line.match(block_comment_start_with).captures.first
+        elsif single_line_comment && line.match(single_line_comment)
+          if code.any?
+            sections << [docs, code]
+            docs, code = [], []
+          end
+          docs << line.sub(single_line_comment || '', '')
+        else
+          code << line
+        end
+      end
+    end
+    sections << [docs, code] if docs.any? || code.any?
+    normalize_leading_spaces(sections)
+  end
+
+  # Normalizes documentation whitespace by checking for leading whitespace,
+  # removing it, and then removing the same amount of whitespace from each
+  # succeeding line.  That is:
+  #
+  #     def func():
+  #       """
+  #         Comment 1
+  #         Comment 2
+  #       """
+  #       print "omg!"
+  #
+  # should yield a comment block of `Comment 1\nComment 2` and code of
+  # `def func():\n  print "omg!"`
+  def normalize_leading_spaces(sections)
+    sections.map do |section|
+      if section.any? && section[0].any?
+        leading_space = section[0][0].match("^\s+")
+        if leading_space
+          section[0] =
+            section[0].map{ |line| line.sub(/^#{leading_space.to_s}/, '') }
+        end
+      end
+      section
+    end
+  end
+
+  # Take the list of paired *sections* two-tuples and split into two
+  # separate lists: one holding the comments with leaders removed and
+  # one with the code blocks.
+  def split(sections)
+    docs_blocks, code_blocks = [], []
+    sections.each do |docs,code|
+      docs_blocks << docs.join("\n")
+      code_blocks << code.map do |line|
+        tabs = line.match(/^(\t+)/)
+        tabs ? line.sub(/^\t+/, '  ' * tabs.captures[0].length) : line
+      end.join("\n")
+    end
+    [docs_blocks, code_blocks]
+  end
+
+  # Take a list of block comments and convert Docblock @annotations to
+  # Markdown syntax.
+  def docblock(docs)
+    docs.map do |doc|
+      doc.split("\n").map do |line|
+        line.match(/^@\w+/) ? line.sub(/^@(\w+)\s+/, '> **\1** ')+"  " : line
+      end.join("\n")
+    end
+  end
+
+  # Take the result of `split` and apply Markdown formatting to comments and
+  # syntax highlighting to source code.
+  def highlight(blocks)
+    docs_blocks, code_blocks = blocks
+
+    # Pre-process Docblock @annotations.
+    if @options[:docblocks]
+      docs_blocks = docblock(docs_blocks)
+    end
+
+    # Combine all docs blocks into a single big markdown document with section
+    # dividers and run through the Markdown processor. Then split it back out
+    # into separate sections.
+    markdown = docs_blocks.join("\n\n##### DIVIDER\n\n")
+    docs_html = Markdown.new(markdown, :smart).
+      to_html.
+      split(/\n*<h5>DIVIDER<\/h5>\n*/m)
+
+    # Combine all code blocks into a single big stream with section dividers and
+    # run through either `pygmentize(1)` or <http://pygments.appspot.com>
+    span, espan = '<span class="c.?">', '</span>'
+    if @options[:comment_chars][:single]
+      front = @options[:comment_chars][:single]
+      divider_input  = "\n\n#{front} DIVIDER\n\n"
+      divider_output = Regexp.new(
+        [ "\\n*",
+          span,
+          Regexp.escape(CGI.escapeHTML(front)),
+          ' DIVIDER',
+          espan,
+          "\\n*"
+        ].join, Regexp::MULTILINE
+      )
+    else
+      front = @options[:comment_chars][:multi][:start]
+      back  = @options[:comment_chars][:multi][:end]
+      divider_input  = "\n\n#{front}\nDIVIDER\n#{back}\n\n"
+      divider_output = Regexp.new(
+        [ "\\n*",
+          span, Regexp.escape(CGI.escapeHTML(front)), espan,
+          "\\n",
+          span, "DIVIDER", espan,
+          "\\n",
+          span, Regexp.escape(CGI.escapeHTML(back)), espan,
+          "\\n*"
+        ].join, Regexp::MULTILINE
+      )
+    end
+
+    code_stream = code_blocks.join(divider_input)
+
+    code_html =
+      if pygmentize?
+        highlight_pygmentize(code_stream)
+      else
+        highlight_webservice(code_stream)
+      end
+
+    # Do some post-processing on the pygments output to split things back
+    # into sections and remove partial `<pre>` blocks.
+    code_html = code_html.
+      split(divider_output).
+      map { |code| code.sub(/\n?<div class="highlight"><pre>/m, '') }.
+      map { |code| code.sub(/\n?<\/pre><\/div>\n/m, '') }
+
+    # Lastly, combine the docs and code lists back into a list of two-tuples.
+    docs_html.zip(code_html)
+  end
+
+  # We `popen` a read/write pygmentize process in the parent and
+  # then fork off a child process to write the input.
+  def highlight_pygmentize(code)
+    code_html = nil
+    open("|pygmentize -l #{@options[:language]} -O encoding=utf-8 -f html", 'r+') do |fd|
+      pid =
+        fork {
+          fd.close_read
+          fd.write code
+          fd.close_write
+          exit!
+        }
+      fd.close_write
+      code_html = fd.read
+      fd.close_read
+      Process.wait(pid)
+    end
+
+    code_html
+  end
+
+  # Pygments is not one of those things that's trivial for a ruby user to install,
+  # so we'll fall back on a webservice to highlight the code if it isn't available.
+  def highlight_webservice(code)
+    Net::HTTP.post_form(
+      URI.parse('http://pygments.appspot.com/'),
+      {'lang' => @options[:language], 'code' => code}
+    ).body
+  end
+end
+
+# And that's it.