redcarpet 1.17.0 → 2.0.0

data/README.markdown

@@ -1,61 +1,318 @@
-Ruby + Upskirt = Markdown that doesn't suck
-===========================================
+Redcarpet 2 is written with sugar, spice and everything nice
+============================================================
 
-Redcarpet is a Ruby wrapper for Upskirt. It is mostly based on Ryan
-Tomayko's RDiscount wrapper, and inspired by Rick Astley wearing a kilt.
+Redcarpet is Ruby library for Markdown processing that smells like
+butterflies and popcorn.
 
-Redcarpet is powered by the Upskirt library, which can be found at
+Redcarpet used to be a drop-in replacement for Redcloth. This is no longer the
+case since version 2 -- it now has its own API, but retains the old name. Yes,
+that does mean that Redcarpet 2 is not backwards-compatible with the 1.X
+versions.
 
-	https://www.github.com/tanoku/upskirt
+Redcarpet is powered by the Sundown library, which can be found at
 
-You might want to find out more about Upskirt to see what makes these Ruby
-bindings so awesome.
+    https://www.github.com/tanoku/sundown
 
-Credits
--------
+You might want to find out more about Sundown to see what makes this Ruby
+library so awesome.
 
-* Natacha Porté, lady of Markdown
-* Vicent Martí, wannabe
-* With special thanks to Ryan Tomayko
+This library is written by people
+-------------------------------------------------------
 
-Install
--------
+Redcarpet 2 has been rewritten from scratch by Vicent Martí (@tanoku). Why
+are you not following me on Twitter?
 
-Redcarpet is readily available as a Ruby gem:
+Redcarpet would not be possible without the Sundown library and its authors
+(Natacha Porté, Vicent Martí, and its many awesome contributors).
+
+You can totally install it as a Gem
+-----------------------------------
+
+Redcarpet is readily available as a Ruby gem. It will build some native
+extensions, but the parser is standalone and requires no installed libraries.
 
     $ [sudo] gem install redcarpet
 
-The Redcarpet source (including Upskirt as a submodule) is available at GitHub:
+The Redcarpet source (including Sundown as a submodule) is available at GitHub:
 
     $ git clone git://github.com/tanoku/redcarpet.git
 
-Usage
------
+And it's like *really* simple to use
+------------------------------------
+
+The core of the Redcarpet library is the `Redcarpet::Markdown` class. Each
+instance of the class is attached to a `Renderer` object; the Markdown class
+performs parsing of a document and uses the attached renderer to generate
+output.
+
+The `Markdown` object is encouraged to be instantiated once with the required
+settings, and reused between parses.
+
+    Markdown.new(renderer, extensions={})
+
+    Initializes a Markdown parser
+
+    renderer -  a renderer object, inheriting from Redcarpet::Render::Base.
+                If the given object has not been instantiated, the library
+                will do it with default arguments.
+
+    extensions - a hash containing the Markdown extensions which the parser
+                will identify. The following extensions are accepted:
+
+                :no_intra_emphasis - do not parse emphasis inside of words.
+                    Strings such as `foo_bar_baz` will not generate `<em>`
+                    tags.
+
+                :tables - parse tables, PHP-Markdown style
+
+                :fenced_code_blocks - parse fenced code blocks, PHP-Markdown
+                    style .Blocks delimited with 3 or more `~` or backticks
+                    will be considered as code, without the need to be
+                    indented. An optional language name may be added at the
+                    end of the opening fence for the code block
+
+                :autolink - parse links even when they are not enclosed in
+                    `<>` characters. Autolinks for the http, https and ftp
+                    protocols will be automatically detected. Email addresses
+                    are also handled, and http links without protocol, but
+                    starting with `www.`
+
+                :strikethrough - parse strikethrough, PHP-Markdown style
+                    Two `~` characters mark the start of a strikethrough,
+                    e.g. `this is ~~good~~ bad`
+
+                :lax_html_blocks - HTML blocks do not require to be surrounded
+                    by an empty line as in the Markdown standard.
+
+                :space_after_headers - A space is always required between the
+                    hash at the beginning of a header and its name, e.g.
+                    `#this is my header` would not be a valid header.
+
+                :superscript - parse superscripts after the `^` character;
+                    contiguous superscripts are nested together, and complex
+                    values can be enclosed in parenthesis,
+                    e.g. `this is the 2^(nd) time`
+
+    Example:
+
+        markdown = Redcarpet::Markdown.new(Redcarpet::Render::HTML,
+            :autolink => true, :space_after_headers => true)
+
+Rendering with the `Markdown` object is done through `Markdown#render`.
+Unlike in the RedCloth API, the text to render is passed as an argument
+and not stored inside the `Markdown` instance, to encourage reusability.
+
+    Markdown.render(text)
+
+    Render a Markdown document with the attached renderer
+
+    text - a Markdown document
+
+    Example:
+
+        markdown.render("This is *bongos*, indeed.")
+        #=> "<p>This is <em>bongos</em>, indeed</p>"
+
+
+Darling, I packed you a couple renderers for lunch 
+--------------------------------------------------
+
+Redcarpet comes with two built-in renderers, `Redcarpet::Render::HTML` and
+`Redcarpet::Render::XHTML`, which output HTML and XHTML, respectively. These
+renderers are actually implemented in C, and hence offer a brilliant
+performance, several degrees of magnitude faster than other Ruby Markdown
+solutions.
+
+All the rendering flags that previously applied only to HTML output have
+now been moved to the `Render::HTML` class, and may be enabled when
+instantiating the renderer:
+
+    Render::HTML.new(render_options={})
+
+    Initializes an HTML renderer. The following flags are available:
+
+        :filter_html - do not allow any user-inputted HTML in the output
+
+        :no_images - do not generate any `<img>` tags
+
+        :no_links - do not generate any `<a>` tags
+
+        :no_styles - do not generate any `<style>` tags
+
+        :safe_links_only - only generate links for protocols which are considered safe
+
+        :with_toc_data - add HTML anchors to each header in the output HTML,
+            to allow linking to each section.
+
+        :hard_wrap - insert HTML `<br>` tags inside on paragraphs where the origin
+            Markdown document had newlines (by default, Markdown ignores these
+            newlines).
+
+        :xhtml - output XHTML-conformant tags. This option is always enabled in the
+            `Render::XHTML` renderer.
+
+
+    Example:
+
+        rndr = Redcarpet::Render::HTML.new(:no_links => true, :hard_wrap => true)
+
+
+The `HTML` renderer has an alternate version, `Redcarpet::Render::HTML_TOC`,
+which will output a table of contents in HTML based on the headers of the
+Markdown document.
+
+Furthermore, the abstract base class `Redcarpet::Render::Base` can be used
+to write a custom renderer purely in Ruby, or extending an existing renderer.
+See the following section for more information.
+
+
+And you can even cook your own
+------------------------------
+
+Custom renderers are created by inheriting from an existing renderer. The
+built-in renderers, `HTML` and `XHTML` may be extended as such:
+
+~~~~~ ruby
+# create a custom renderer that allows highlighting of code blocks
+class HTMLwithAlbino < Redcarpet::Render::HTML
+  def block_code(code, language)
+    Albino.safe_colorize(code, language)
+  end
+end
+
+markdown = Redcarpet::Markdown.new(HTMLwithAlbino, :fenced_code_blocks => true)
+~~~~~
+
+But new renderers can also be created from scratch (see `lib/render_man.rb` for
+an example implementation of a Manpage renderer)
+
+~~~~~~ ruby
+class ManPage < Redcarpet::Render::Base
+    # you get the drill -- keep going from here
+end
+~~~~~
+
+The following instance methods may be implemented by the renderer:
+
+    # Block-level calls
+    # If the return value of the method is `nil`, the block
+    # will be skipped.
+    # If the method for a document element is not implemented,
+    # the block will be skipped.
+    # 
+    # Example:
+    #
+    #   class RenderWithoutCode < Redcarpet::Render::HTML
+    #     def block_code(code, language)
+    #       nil
+    #     end
+    #   end
+    #
+    block_code(code, language)
+    block_quote(quote)
+    block_html(raw_html)
+    header(text, header_level)
+    hrule()
+    list(contents, list_type)
+    list_item(text, list_type)
+    paragraph(text)
+    table(header, body)
+    table_row(content)
+    table_cell(content, alignment)
+
+    # Span-level calls
+    # A return value of `nil` will not output any data
+    # If the method for a document element is not implemented,
+    # the contents of the span will be copied verbatim
+    autolink(link, link_type)
+    codespan(code)
+    double_emphasis(text)
+    emphasis(text)
+    image(link, title, alt_text)
+    linebreak()
+    link(link, title, content)
+    raw_html(raw_html)
+    triple_emphasis(text)
+    strikethrough(text)
+    superscript(text)
+
+    # Low level rendering
+    entity(text)
+    normal_text(text)
+
+    # Header of the document
+    # Rendered before any another elements
+    doc_header()
+
+    # Footer of the document
+    # Rendered after all the other elements
+    doc_footer()
+
+    # Pre/post-process
+    # Special callback: preprocess or postprocess the whole
+    # document before or after the rendering process begins
+    preprocess(full_document)
+    postprocess(full_document)
+    
+
+Also, now our Pants are much smarter
+------------------------------------
+
+Redcarpet 2 comes with a standalone [SmartyPants](
+http://daringfireball.net/projects/smartypants/) implementation. It is fully
+compliant with the original implementation. It is the fastest SmartyPants
+parser there is, with a difference of several orders of magnitude.
+
+The SmartyPants parser can be found in `Redcarpet::Render::SmartyPants`. It has
+been implemented as a module, so it can be used standalone or as a mixin.
+
+When mixed with a Renderer class, it will override the `postprocess` method
+to perform SmartyPants replacements once the rendering is complete
+
+~~~~ ruby
+# Mixin
+class HTMLWithPants < Redcarpet::Render::HTML
+  include Redcarpet::Render::SmartyPants
+end
+
+# Standalone
+Redcarpet::Render::SmartyPants.render("<p>Oh SmartyPants, you're so crazy...</p>")
+~~~~~
+
+SmartyPants works on top of already-rendered HTML, and will ignore replacements
+inside the content of HTML tags and inside specific HTML blocks such as 
+`<code>` or `<pre>`.
+
+What? You really want to mix Markdown renderers?
+------------------------------------------------
+
+What a terrible idea! Markdown is already ill-specified enough; if you create
+software that is renderer-independent, the results will be completely unreliable!
+
+Each renderer has its own API and its own set of extensions: you should choose one
+(it doesn't have to be Redcarpet, though that would be great!), write your
+software accordingly, and force your users to install it. That's the
+only way to have reliable and predictable Markdown output on your program.
 
-Redcarpet implements the basic protocol popularized by RedCloth:
+Still, if major forces (let's say, tornadoes or other natural disasters) force you
+to keep a Markdown-compatibility later, Redcarpet also supports this:
 
-~~~~~~ {ruby}
-require 'redcarpet'
-markdown = Redcarpet.new("Hello World!")
-puts markdown.to_html
-~~~~~~
+    require 'redcarpet/compat'
 
-[Additional processing options](http://rdoc.info/github/tanoku/redcarpet/master/Redcarpet#autolink-instance_method) can be turned on when creating the
-Redcarpet object:
+Requiring the compatibility library will declare a `Markdown` class with the
+classical RedCloth API, e.g.
 
-~~~~~~ {ruby}
-markdown = Redcarpet.new("Hello World!", :smart, :filter_html)
-~~~~~~
+    Markdown.new('this is my text').to_html
 
-Note that by default, Redcarpet parses standard Markdown (with no extensions)
-and offers a sane subset of parse options which allow you to modify the rendering
-output and to enable MD extensions on a per-case basis.
+This class renders 100% standards compliant Markdown with 0 extensions. Nada.
+Don't even try to enable extensions with a compatibility layer, because
+that's a maintance nightmare and won't work.
 
-Redcarpet also offers a wrapper class, `RedcarpetCompat` with the same flags
-and behavior as the RDiscount library, which acts as a drop-in replacement.
+On a related topic: if your Markdown gem has a `lib/markdown.rb` file that
+monkeypatches the Markdown class, you're a terrible human being. Just saying.
 
-License
--------
+Boring legal stuff
+------------------
 
 Copyright (c) 2011, Vicent Martí
 

data/Rakefile

@@ -70,9 +70,6 @@ task :install => package('.gem') do
   sh "gem install #{package('.gem')}"
 end
 
-desc 'Update the gemspec'
-task :update_gem => file('redcarpet.gemspec')
-
 directory 'pkg/'
 
 file package('.gem') => %w[pkg/ redcarpet.gemspec] + $spec.files do |f|
@@ -91,9 +88,10 @@ def source_version
   line.match(/.*VERSION = '(.*)'/)[1]
 end
 
-file 'redcarpet.gemspec' => FileList['Rakefile','lib/redcarpet.rb'] do |f|
+task :update_gem do
   # read spec file and split out manifest section
-  spec = File.read(f.name)
+  GEMFILE = 'redcarpet.gemspec'
+  spec = File.read(GEMFILE)
   head, manifest, tail = spec.split("  # = MANIFEST =\n")
   head.sub!(/\.version = '.*'/, ".version = '#{source_version}'")
   head.sub!(/\.date = '.*'/, ".date = '#{Date.today.to_s}'")
@@ -107,26 +105,26 @@ file 'redcarpet.gemspec' => FileList['Rakefile','lib/redcarpet.rb'] do |f|
   # piece file back together and write...
   manifest = "  s.files = %w[\n#{files}\n  ]\n"
   spec = [head,manifest,tail].join("  # = MANIFEST =\n")
-  File.open(f.name, 'w') { |io| io.write(spec) }
-  puts "updated #{f.name}"
+  File.open(GEMFILE, 'w') { |io| io.write(spec) }
+  puts "updated #{GEMFILE}"
 end
 
-desc 'Gather required Upskirt sources into extension directory'
-task :gather => 'upskirt/src/markdown.h' do |t|
+desc 'Gather required Sundown sources into extension directory'
+task :gather => 'sundown/src/markdown.h' do |t|
   files =
     FileList[
-      'upskirt/src/{markdown,buffer,array,autolink}.h',
-      'upskirt/src/{markdown,buffer,array,autolink}.c',
-      'upskirt/html/{html,html_smartypants}.c',
-      'upskirt/html/html.h',
+      'sundown/src/{markdown,buffer,stack,autolink,html_blocks}.h',
+      'sundown/src/{markdown,buffer,stack,autolink}.c',
+      'sundown/html/{html,html_smartypants,houdini_html_e,houdini_href_e}.c',
+      'sundown/html/{html,houdini}.h',
     ]
   cp files, 'ext/redcarpet/',
     :preserve => true,
     :verbose => true
 end
 
-file 'upskirt/src/markdown.h' do |t|
-  abort "The Upskirt submodule is required."
+file 'sundown/src/markdown.h' do |t|
+  abort "The Sundown submodule is required."
 end
 
 

data/bin/redcarpet

@@ -10,4 +10,4 @@ if ARGV.include?('--help')
 end
 
 require 'redcarpet'
-STDOUT.write(Redcarpet.new(ARGF.read).to_html)
+STDOUT.write(Redcarpet::Markdown.new(Redcarpet::Render::HTML).render(ARGF.read))

data/ext/redcarpet/autolink.c

@@ -22,11 +22,11 @@
 #include <ctype.h>
 
 int
-is_safe_link(const char *link, size_t link_len)
+sd_autolink_issafe(const uint8_t *link, size_t link_len)
 {
-	static const size_t valid_uris_count = 4;
+	static const size_t valid_uris_count = 5;
 	static const char *valid_uris[] = {
-		"http://", "https://", "ftp://", "mailto://"
+		"/", "http://", "https://", "ftp://", "mailto:"
 	};
 
 	size_t i;
@@ -35,7 +35,7 @@ is_safe_link(const char *link, size_t link_len)
 		size_t len = strlen(valid_uris[i]);
 
 		if (link_len > len &&
-			strncasecmp(link, valid_uris[i], len) == 0 &&
+			strncasecmp((char *)link, valid_uris[i], len) == 0 &&
 			isalnum(link[len]))
 			return 1;
 	}
@@ -44,9 +44,16 @@ is_safe_link(const char *link, size_t link_len)
 }
 
 static size_t
-autolink_delim(char *data, size_t link_end, size_t offset, size_t size)
+autolink_delim(uint8_t *data, size_t link_end, size_t offset, size_t size)
 {
-	char cclose, copen = 0;
+	uint8_t cclose, copen = 0;
+	size_t i;
+
+	for (i = 0; i < link_end; ++i)
+		if (data[i] == '<') {
+			link_end = i;
+			break;
+		}
 
 	while (link_end > 0) {
 		if (strchr("?!.,", data[link_end - 1]) != NULL)
@@ -63,11 +70,6 @@ autolink_delim(char *data, size_t link_end, size_t offset, size_t size)
 			else
 				link_end--;
 		}
-
-		else if (data[link_end - 1] == '>') {
-			while (link_end > 0 && data[link_end] != '<')
-				link_end--;
-		}
 		else break;
 	}
 
@@ -125,29 +127,43 @@ autolink_delim(char *data, size_t link_end, size_t offset, size_t size)
 	return link_end;
 }
 
+static size_t
+check_domain(uint8_t *data, size_t size)
+{
+	size_t i, np = 0;
+
+	if (!isalnum(data[0]))
+		return 0;
+
+	for (i = 1; i < size - 1; ++i) {
+		if (data[i] == '.') np++;
+		else if (!isalnum(data[i]) && data[i] != '-') break;
+	}
+
+	/* a valid domain needs to have at least a dot.
+	 * that's as far as we get */
+	return np ? i : 0;
+}
+
 size_t
-ups_autolink__www(size_t *rewind_p, struct buf *link, char *data, size_t offset, size_t size)
+sd_autolink__www(size_t *rewind_p, struct buf *link, uint8_t *data, size_t offset, size_t size)
 {
 	size_t link_end;
-	int np = 0;
 
 	if (offset > 0 && !ispunct(data[-1]) && !isspace(data[-1]))
 		return 0;
 
-	if (size < 4 || memcmp(data, "www.", STRLEN("www.")) != 0)
+	if (size < 4 || memcmp(data, "www.", strlen("www.")) != 0)
 		return 0;
 
-	link_end = 0;
-	while (link_end < size && !isspace(data[link_end])) {
-		if (data[link_end] == '.')
-			np++;
-
-		link_end++;
-	}
+	link_end = check_domain(data, size);
 
-	if (np < 2)
+	if (link_end == 0)
 		return 0;
 
+	while (link_end < size && !isspace(data[link_end]))
+		link_end++;
+
 	link_end = autolink_delim(data, link_end, offset, size);
 
 	if (link_end == 0)
@@ -160,13 +176,13 @@ ups_autolink__www(size_t *rewind_p, struct buf *link, char *data, size_t offset,
 }
 
 size_t
-ups_autolink__email(size_t *rewind_p, struct buf *link, char *data, size_t offset, size_t size)
+sd_autolink__email(size_t *rewind_p, struct buf *link, uint8_t *data, size_t offset, size_t size)
 {
 	size_t link_end, rewind;
 	int nb = 0, np = 0;
 
 	for (rewind = 0; rewind < offset; ++rewind) {
-		char c = data[-rewind - 1];
+		uint8_t c = data[-rewind - 1];
 
 		if (isalnum(c))
 			continue;
@@ -181,7 +197,7 @@ ups_autolink__email(size_t *rewind_p, struct buf *link, char *data, size_t offse
 		return 0;
 
 	for (link_end = 0; link_end < size; ++link_end) {
-		char c = data[link_end];
+		uint8_t c = data[link_end];
 
 		if (isalnum(c))
 			continue;
@@ -209,9 +225,9 @@ ups_autolink__email(size_t *rewind_p, struct buf *link, char *data, size_t offse
 }
 
 size_t
-ups_autolink__url(size_t *rewind_p, struct buf *link, char *data, size_t offset, size_t size)
+sd_autolink__url(size_t *rewind_p, struct buf *link, uint8_t *data, size_t offset, size_t size)
 {
-	size_t link_end, rewind = 0;
+	size_t link_end, rewind = 0, domain_len;
 
 	if (size < 4 || data[1] != '/' || data[2] != '/')
 		return 0;
@@ -219,10 +235,15 @@ ups_autolink__url(size_t *rewind_p, struct buf *link, char *data, size_t offset,
 	while (rewind < offset && isalpha(data[-rewind - 1]))
 		rewind++;
 
-	if (!is_safe_link(data - rewind, size + rewind))
+	if (!sd_autolink_issafe(data - rewind, size + rewind))
+		return 0;
+	link_end = strlen("://");
+
+	domain_len = check_domain(data + link_end, size - link_end);
+	if (domain_len == 0)
 		return 0;
 
-	link_end = 0;
+	link_end += domain_len;
 	while (link_end < size && !isspace(data[link_end]))
 		link_end++;
 

data/ext/redcarpet/autolink.h

@@ -0,0 +1,36 @@
+/*
+ * Copyright (c) 2011, Vicent Marti
+ *
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose with or without fee is hereby granted, provided that the above
+ * copyright notice and this permission notice appear in all copies.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+ * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+ * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+ * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+ * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+ * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+ * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+ */
+
+#ifndef UPSKIRT_AUTOLINK_H
+#define UPSKIRT_AUTOLINK_H
+
+#include "buffer.h"
+
+extern int
+sd_autolink_issafe(const uint8_t *link, size_t link_len);
+
+extern size_t
+sd_autolink__www(size_t *rewind_p, struct buf *link, uint8_t *data, size_t offset, size_t size);
+
+extern size_t
+sd_autolink__email(size_t *rewind_p, struct buf *link, uint8_t *data, size_t offset, size_t size);
+
+extern size_t
+sd_autolink__url(size_t *rewind_p, struct buf *link, uint8_t *data, size_t offset, size_t size);
+
+#endif
+
+/* vim: set filetype=c: */