kramdown 0.1.0

data/doc/tests.page

@@ -0,0 +1,44 @@
+---
+title: Tests and Benchmark
+---
+
+# Tests
+
+There exist several test suites for testing the correctness of a Markdown implementation. The
+original [Markdown Test Suite] is the standard which one needs to test against. The [PHP Markdown
+suite][MDTest] contains the original test suite and several more tests (some specifically geared
+towards the extension of the PHP Markdown Extra package). I have used the latter test tool to
+roughly verify that kramdown is able to parse standard Markdown. However, since the syntax used by
+kramdown varies slightly from standard Markdown most of the tests fail - which is fine. When looking
+at the differences one can see that the failures result from these differences.
+
+Besides using the above mentioned test suite kramdown comes with its own set of tests which is used
+to verify that the implementation matches the kramdown specification.
+
+If you believe you have found a bug in the implementation, please follow these steps:
+
+* Check the syntax page and see if the behaviour is not intended.
+
+* If the behaviour is not intended and it seems that kramdown should parse some text in another
+  fashion, please open a bug report and attach two files: one with the text and one with the HTML
+  conversion you think is correct.
+
+^
+
+# Benchmark
+
+kramdown comes with a small benchmark to test how fast it is in regard to four other Ruby Markdown
+implementations: Maruku, BlueFeather, BlueCloth and RDiscount. The first two are written using only
+Ruby, the latter two use the C discount library for the actual hard work (which makes them really
+fast but they do not provide additional syntax elements). As one can see below, kramdown is
+currently (November 2009) ~5x faster than Maruku, ~10x faster than BlueFeather but ~30x slower than
+BlueCloth and rdiscount:
+
+{::nokramdown:}
+<pre><code>
+{execute_cmd: {command: "ruby -Ilib -rubygems benchmark/benchmark.rb", process_output: false, escape_html: true}}
+</code></pre>
+{::nokramdown:}
+
+[Markdown Test Suite]: http://daringfireball.net/projects/downloads/MarkdownTest_1.0.zip
+[MDTest]: http://www.michelf.com/docs/projets/mdtest-1.0.zip

data/doc/virtual

@@ -0,0 +1,2 @@
+rdoc/index.html:
+  title: API Documentation

data/lib/kramdown.rb

@@ -0,0 +1,23 @@
+# -*- coding: utf-8 -*-
+#
+#--
+# Copyright (C) 2009 Thomas Leitner <t_leitner@gmx.at>
+#
+# This file is part of kramdown.
+#
+# kramdown is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#++
+#
+
+require 'kramdown/document'

data/lib/kramdown/converter.rb

@@ -0,0 +1,215 @@
+# -*- coding: utf-8 -*-
+#
+#--
+# Copyright (C) 2009 Thomas Leitner <t_leitner@gmx.at>
+#
+# This file is part of kramdown.
+#
+# kramdown is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#++
+#
+
+require 'rexml/parsers/baseparser'
+
+module Kramdown
+
+  # This module contains all available converters, i.e. classes that take a document and convert the
+  # document tree to a string in a specific format, for example, HTML.
+  module Converter
+
+    # Converts a Kramdown::Document to HTML.
+    class Html
+
+      # Initialize the HTML converter with the given Kramdown document +doc+.
+      def initialize(doc)
+        @doc = doc
+        @footnote_counter = @footnote_start = @doc.options[:footnote_nr]
+        @footnotes = []
+      end
+      private_class_method(:new, :allocate)
+
+      # Convert the Kramdown document +doc+ to HTML.
+      def self.convert(doc)
+        new(doc).convert(doc.tree)
+      end
+
+      # Convert the element tree +el+, setting the indentation level to +indent+.
+      def convert(el, indent = -2)
+        result = ''
+        el.children.each do |inner_el|
+          result << convert(inner_el, indent + 2)
+        end
+        send("convert_#{el.type}", el, result, indent)
+      end
+
+      def convert_blank(el, inner, indent)
+        "\n"
+      end
+
+      def convert_text(el, inner, indent)
+        escape_html(el.value, false)
+      end
+
+      def convert_p(el, inner, indent)
+        "#{' '*indent}<p#{options_for_element(el)}>#{inner}</p>\n"
+      end
+
+      def convert_codeblock(el, inner, indent)
+        result = escape_html(el.value)
+        if el.options[:attr] && el.options[:attr].has_key?('class') && el.options[:attr]['class'] =~ /\bshow-whitespaces\b/
+          result.gsub!(/(?:(^[ \t]+)|([ \t]+$)|([ \t]+))/) do |m|
+            suffix = ($1 ? '-l' : ($2 ? '-r' : ''))
+            m.scan(/./).map do |c|
+              case c
+              when "\t" then "<span class=\"ws-tab#{suffix}\">\t</span>"
+              when " " then "<span class=\"ws-space#{suffix}\">&sdot;</span>"
+              end
+            end.join('')
+          end
+        end
+        "#{' '*indent}<pre#{options_for_element(el)}><code>#{result}#{result =~ /\n\Z/ ? '' : "\n"}</code></pre>\n"
+      end
+
+      def convert_blockquote(el, inner, indent)
+        "#{' '*indent}<blockquote#{options_for_element(el)}>\n#{inner}#{' '*indent}</blockquote>\n"
+      end
+
+      def convert_header(el, inner, indent)
+        "#{' '*indent}<h#{el.options[:level]}#{options_for_element(el)}>#{inner}</h#{el.options[:level]}>\n"
+      end
+
+      def convert_hr(el, inner, indent)
+        "#{' '*indent}<hr />\n"
+      end
+
+      def convert_ul(el, inner, indent)
+        "#{' '*indent}<#{el.type}#{options_for_element(el)}>\n#{inner}#{' '*indent}</#{el.type}>\n"
+      end
+      alias :convert_ol :convert_ul
+
+      def convert_li(el, inner, indent)
+        output = ' '*indent << "<li" << options_for_element(el) << ">"
+        if el.options[:first_as_block]
+          output << "\n" << inner << ' '*indent
+        else
+          output << inner << (inner =~ /\n\Z/ ? ' '*indent : '')
+        end
+        output << "</li>\n"
+      end
+
+      def convert_html_raw(el, inner, indent)
+        el.value + (el.options[:type] == :block ? "\n" : '')
+      end
+
+      HTML_TAGS_WITH_BODY=['div']
+
+      def convert_html_element(el, inner, indent)
+        if @doc.options[:filter_html].include?(el.value)
+          inner.chomp + (el.options[:type] == :block ? "\n" : '')
+        elsif el.options[:type] == :span
+          "<#{el.value}#{options_for_element(el)}" << (!inner.empty? ? ">#{inner}</#{el.value}>" : " />")
+        else
+          output = ' '*indent << "<#{el.value}#{options_for_element(el)}"
+          if !inner.empty?
+            output << ">\n#{inner.chomp}\n"  << ' '*indent << "</#{el.value}>"
+          elsif HTML_TAGS_WITH_BODY.include?(el.value)
+            output << "></#{el.value}>"
+          else
+            output << " />"
+          end
+          output << "\n"
+        end
+      end
+
+      def convert_br(el, inner, indent)
+        "<br />"
+      end
+
+      def convert_a(el, inner, indent)
+        "<a#{options_for_element(el)}>#{inner}</a>"
+      end
+
+      def convert_img(el, inner, indent)
+        "<img#{options_for_element(el)} />"
+      end
+
+      def convert_codespan(el, inner, indent)
+        "<code#{options_for_element(el)}>#{escape_html(el.value)}</code>"
+      end
+
+      def convert_footnote(el, inner, indent)
+        number = @footnote_counter
+        @footnote_counter += 1
+        @footnotes << [el.options[:name], @doc.parse_infos[:footnotes][el.options[:name]]]
+        "<sup id=\"fnref:#{el.options[:name]}\"><a href=\"#fn:#{el.options[:name]}\" rel=\"footnote\">#{number}</a></sup>"
+      end
+
+      def convert_raw(el, inner, indent)
+        el.value
+      end
+
+      def convert_em(el, inner, indent)
+        "<#{el.type}#{options_for_element(el)}>#{inner}</#{el.type}>"
+      end
+      alias :convert_strong :convert_em
+
+      def convert_root(el, inner, indent)
+        inner << footnote_content
+      end
+
+
+      # Return a HTML list with the footnote content for the used footnotes.
+      def footnote_content
+        ol = Element.new(:ol)
+        ol.options[:attr] = {'start' => @footnote_start} if @footnote_start != 1
+        @footnotes.each do |name, data|
+          li = Element.new(:li, nil, {:attr => {:id => "fn:#{name}"}, :first_as_block => true})
+          li.children = Marshal.load(Marshal.dump(data[:content].children)) #TODO: probably remove this!!!!
+          ol.children << li
+
+          ref = Element.new(:raw, "<a href=\"#fnref:#{name}\" rev=\"footnote\">&#8617;</a>")
+          if li.children.last.type == :p
+            para = li.children.last
+          else
+            li.children << (para = Element.new(:p))
+          end
+          para.children << ref
+        end
+        (ol.children.empty? ? '' : "<div class=\"kramdown-footnotes\">\n#{convert(ol, 2)}</div>\n")
+      end
+
+      # Return the string with the attributes of the element +el+.
+      def options_for_element(el)
+        (el.options[:attr] || {}).map {|k,v| v.nil? ? '' : " #{k}=\"#{escape_html(v.to_s, false)}\"" }.sort.join('')
+      end
+
+      ESCAPE_MAP = {
+        '<' => '&lt;',
+        '>' => '&gt;',
+        '"' => '&quot;',
+        '&' => '&amp;'
+      }
+      ESCAPE_ALL_RE = Regexp.union(*ESCAPE_MAP.collect {|k,v| Regexp.escape(k)})
+      ESCAPE_ALL_NOT_ENTITIES_RE = Regexp.union(REXML::Parsers::BaseParser::REFERENCE_RE, ESCAPE_ALL_RE)
+
+      # Escape the special HTML characters in the string +str+. If +all+ is +true+ then all
+      # characters are escaped, if +all+ is +false+
+      def escape_html(str, all = true)
+        str.gsub(all ? ESCAPE_ALL_RE : ESCAPE_ALL_NOT_ENTITIES_RE) {|m| ESCAPE_MAP[m] || m}
+      end
+
+    end
+
+  end
+end

data/lib/kramdown/document.rb

@@ -0,0 +1,150 @@
+# -*- coding: utf-8 -*-
+#
+#--
+# Copyright (C) 2009 Thomas Leitner <t_leitner@gmx.at>
+#
+# This file is part of kramdown.
+#
+# kramdown is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#++
+#
+
+require 'kramdown/error'
+require 'kramdown/parser'
+require 'kramdown/converter'
+require 'kramdown/extension'
+
+module Kramdown
+
+  # The kramdown version.
+  VERSION = '0.1.0'
+
+  # The main interface to kramdown.
+  #
+  # This class provides a one-stop-shop for using kramdown to convert text into various output
+  # formats. Use it like this:
+  #
+  #   require 'kramdown'
+  #   doc = Kramdown::Document.new('This *is* some kramdown text')
+  #   puts doc.to_html
+  #
+  # The #to_html method is a shortcut for using the Converter::ToHtml class. If other converters are
+  # added later, there may be additional shortcut methods.
+  #
+  # The second argument to the #new method is an options hash for customizing the behaviour of
+  # kramdown.
+  class Document
+
+    # Currently available options are:
+    #
+    # [:auto_ids (used by the parser)]
+    #    A boolean value deciding whether automatic header ID generation is used. Default: +false+.
+    #    When using the +kdoptions+ extension, the string 'false' will be the value +false+, every
+    #    other non-empty string will be +true+.
+    # [:filter_html (used by the HTML converter)]
+    #    An array of HTML tag names that defines which tags should be filtered from the output. For
+    #    example, if the value contains +iframe+, then all HTML +iframe+ tags are filtered out and
+    #    only the body is displayed. Default: empty array. When using the +kdoptions+ extension, the
+    #    string value needs to hold the HTML tag names separated by one or more spaces.
+    # [:footnote_nr (used by the HTML converter)]
+    #    The initial number used for creating the link to the first footnote. Default: +1+. When
+    #    using the +kdoptions+ extension, the string value needs to be a valid number.
+    DEFAULT_OPTIONS={
+      :footnote_nr => 1,
+      :filter_html => [],
+      :auto_ids => false
+    }
+
+
+    # The element tree of the document. It is immediately available after the #new method has been
+    # called.
+    attr_accessor :tree
+
+    # The options hash which holds the options for parsing/converting the Kramdown document. It is
+    # possible that these values get changed during the parsing phase.
+    attr_accessor :options
+
+    # An array of warning messages. It is filled with warnings during the parsing phase (i.e. in
+    # #new) and the converting phase.
+    attr_reader :warnings
+
+    # Holds needed parse information like ALDs, link definitions and so on.
+    attr_reader :parse_infos
+
+    # Holds an instance of the extension class.
+    attr_reader :extension
+
+
+    # Create a new Kramdown document from the string +source+ and use the provided +options+ (see
+    # DEFAULT_OPTIONS for a list of available options). The +source+ is immediately parsed by the
+    # kramdown parser sothat after this call the output can be generated.
+    #
+    # The parameter +ext+ can be used to set a custom extension class. Note that the default
+    # kramdown extensions should be available in the custom extension class.
+    def initialize(source, options = {}, ext = nil)
+      @options = DEFAULT_OPTIONS.merge(options)
+      @warnings = []
+      @parse_infos = {}
+      @extension = extension || Kramdown::Extension.new
+      @tree = Parser::Kramdown.parse(source, self)
+    end
+
+    # Convert the document to HTML. Uses the Converter::ToHtml class for doing the conversion.
+    def to_html
+      Converter::Html.convert(self)
+    end
+
+    def inspect #:nodoc:
+      "<KD:Document: options=#{@options.inspect} tree=#{@tree.inspect} warnings=#{@warnings.inspect}>"
+    end
+
+  end
+
+
+  # Represents all elements in the parse tree.
+  #
+  # kramdown only uses this one class for representing all available elements in a parse tree
+  # (paragraphs, headers, emphasis, ...). The type of element can be set via the #type accessor.
+  class Element
+
+    # A symbol representing the element type. For example, +:p+ or +:blockquote+.
+    attr_accessor :type
+
+    # The value of the element. The interpretation of this field depends on the type of the element.
+    # Many elements don't use this field.
+    attr_accessor :value
+
+    # The options hash for the element. It is used for storing arbitray options as well as the
+    # *attributes* of the element under the <tt>:attr</tt> key.
+    attr_accessor :options
+
+    # The child elements of this element.
+    attr_accessor :children
+
+
+    # Create a new Element object of type +type+. The optional parameters +value+ and +options+ can
+    # also be set in this constructor for convenience.
+    def initialize(type, value = nil, options = {})
+      @type, @value, @options = type, value, options
+      @children = []
+    end
+
+    def inspect #:nodoc:
+      "<kd:#{@type}#{@value.nil? ? '' : ' ' + @value.inspect}#{options.empty? ? '' : ' ' + @options.inspect}#{@children.empty? ? '' : ' ' + @children.inspect}>"
+    end
+
+  end
+
+end
+

data/lib/kramdown/error.rb

@@ -0,0 +1,27 @@
+# -*- coding: utf-8 -*-
+#
+#--
+# Copyright (C) 2009 Thomas Leitner <t_leitner@gmx.at>
+#
+# This file is part of kramdown.
+#
+# kramdown is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#++
+#
+
+module Kramdown
+
+  class Error < RuntimeError; end
+
+end