reverse_markdown 0.4.7 → 0.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: fb69e03d2dc1bfd473c2073a44a890d25a6c9a25
-  data.tar.gz: 91a7b52f84751dcaf6246024aae160a702fb6fcf
+  metadata.gz: d29227386b5418e906da5a4cb42ce616a2d3872b
+  data.tar.gz: 0f433297212a99d783573daac3fd3723e7d9f6a0
 SHA512:
-  metadata.gz: acd40208862365222a27f1f4430f6ce067e75b2235cd2d403ac7dc126b408352e934e31ac9380914b41a3fe03ce299fadecc1a78298c92a00e104812e7a5780a
-  data.tar.gz: d1f17eb2540c99d07065f4d8f78bdadee8d2aa88650070fd59ce4d62d560ec27f0e8521d400f7d920e3be4e26346acc2ee5a24e508a9f33abe867b9fb264713e
+  metadata.gz: 23ff8eb3e7a76e197b5526fcdf6c14d899e1ec600f1b40046d5584e2c40d62a9f22734656b5843f38ec5fa53dcf3a31f7b9baa7c2962e993eb07e5d4486f3afa
+  data.tar.gz: 2e46ce50817a0e735329dd2e73a9d47943e284a7cfb440362c8f8b6b1cbbb4d9945a6c492bafe8bd34843b7987fe1556410d29d4f6b34ce1d7936c570759f5dc

data/.gitignore

@@ -6,3 +6,4 @@
 Gemfile.lock
 pkg/*
 coverage/*
+TODO

data/.travis.yml

@@ -1,7 +1,7 @@
 rvm:
-  - 1.9.2
-  - 1.9.3
   - 2.0.0
+  - 2.1.0
+  - 2.1.1
 
 script: "bundle exec rake spec"
 
@@ -9,4 +9,3 @@ notifications:
   disabled: false
   recipients:
     - xijo@gmx.de
-    - code@harlantwood.net

data/LICENSE

@@ -0,0 +1,13 @@
+            DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
+                    Version 2, December 2004
+
+ Copyright (C) 2014 Johannes Opper <xijo@gmx.de>
+
+ Everyone is permitted to copy and distribute verbatim or modified
+ copies of this license document, and changing it is allowed as long
+ as the name is changed.
+
+            DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
+   TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
+
+  0. You just DO WHAT THE FUCK YOU WANT TO.

data/README.md

@@ -1,69 +1,108 @@
 # Summary
 
-Transform existing html into markdown in a simple way, for example if you want to import existings tags into your markdown based application.
+Transform html into markdown. Useful for example if you want to import html into your markdown based application.
 
-[![Build Status](https://secure.travis-ci.org/xijo/reverse_markdown.png?branch=master)](https://travis-ci.org/xijo/reverse_markdown) [![Gem Version](https://badge.fury.io/rb/reverse_markdown.png)](http://badge.fury.io/rb/reverse_markdown)
+[![Build Status](https://secure.travis-ci.org/xijo/reverse_markdown.png?branch=master)](https://travis-ci.org/xijo/reverse_markdown) [![Gem Version](https://badge.fury.io/rb/reverse_markdown.png)](http://badge.fury.io/rb/reverse_markdown) [![Code Climate](https://codeclimate.com/github/xijo/reverse_markdown.png)](https://codeclimate.com/github/xijo/reverse_markdown) [![Code Climate](https://codeclimate.com/github/xijo/reverse_markdown/coverage.png)](https://codeclimate.com/github/xijo/reverse_markdown)
 
-# Requirements
+## Migrate to 0.5.0
 
-Depends on [Nokogiri](http://nokogiri.org/).
+There were some breaking changes, please make sure you don't miss them:
 
-# Installation
+1. Only ruby versions 2.0.0 or a above are supported
+2. There is no `Mapper` class any more. Just use `ReverseMarkdown.convert(input, options)`
+3. Config option `github_style_code_blocks` changed its name to `github_flavored`
 
-Install the gem:
+Please open an issue and let me know about it if you have any trouble with the new version.
 
-  [sudo] gem install reverse_markdown
+## Requirements
 
-If you want to use it in Rails context, add it to your Gemfile:
+1. [Nokogiri](http://nokogiri.org/)
+2. Ruby 2.0.0 or higher
+
+## Installation
+
+Install the gem
+
+```sh
+[sudo] gem install reverse_markdown
+```
+
+or add it to your Gemfile
 
 ```ruby
 gem 'reverse_markdown'
 ```
 
-# Synopsis
+## Features
 
-Given you have html content as string or Nokogiri document or element just pass it over to the module like this:
+- Supports all the established html tags like `h1`, `h2`, `h3`, `h4`, `h5`, `h6`, `p`, `em`, `strong`, `i`, `b`, `blockquote`, `code`, `img`, `a`, `hr`, `li`, `ol`, `ul`, `table`, `tr`, `th`, `td`, `br`
+- Module based - if you miss a tag, just add it
+- Can deal with nested lists
+- Inline and block code is supported
+- Supports blockquote
 
-```ruby
-ReverseMarkdown.parse content
-````
 
-However, the old syntax is still supported:
+# Usage
+
+## Ruby
+
+You can convert html content as string or Nokogiri document:
 
 ```ruby
-ReverseMarkdown.parse_element content
-ReverseMarkdown.parse_string content
+input  = '<strong>feelings</strong>'
+result = ReverseMarkdown.convert input
+result.inspect # " **feelings** "
 ````
 
-You can also convert html files to markdown from the command line:
+## Commandline
+
+It's also possible to convert html files to markdown using the binary:
 
 ```sh
-$ reverse_markdown file.html > file.markdown
-$ cat file.html | reverse_markdown > file.markdown
+$ reverse_markdown file.html > file.md
+$ cat file.html | reverse_markdown > file.md
 ````
 
-Additionally there is a support for github-like multiline code which is indented with "`":
+## Configuration
+
+The following options are available:
+
+ - `ignore_unknown_tags` (default `true`) - bypass unknown tags instead of raising an `ReverseMarkdown::UnknownTagError`
+ - `github_flavored` (default `false`) - use [github flavored markdown](https://help.github.com/articles/github-flavored-markdown) (yet only code blocks are supported)
+
+### As options
+
+Just pass your chosen configuration options in after the input
 
 ```ruby
-ReverseMarkdown.parse_string(input, github_style_code_blocks: true)
-````
+ReverseMarkdown.convert(input, ignore_unknown_tags: false, github_flavored: true)
+```
+
+### Preconfigure
 
-# Tag support
+Or configure it block style on a initializer level
 
-Only basic html tags are supported right now. However, it should not be to difficult to add some. Feel free to contribute or notify me about missing stuff.
+```ruby
+ReverseMarkdown.config do |config|
+  config.ignore_unknown_tags = false
+  config.github_flavored     = true
+end
+```
 
-- supported tags: `h1`, `h2`, `h3`, `h4`, `h5`, `h6`, `p`, `em`, `strong`, `i`, `b`, `blockquote`, `code`, `img`, `a`, `hr`, `li`, `ol`, `ul`, `table`, `tr`, `th`, `td`
-- nested lists
-- inline and block code
 
-# See as well
+# Related stuff
 
-- [wmd-editor](http://wmd-editor.com) - Markdown flavored text editor
+- [html_massage](https://github.com/harlantwood/html_massage) - A gem by Harlan T. Wood to convert regular sites into markdown using reverse_markdown
+- [word-to-markdown](https://github.com/benbalter/word-to-markdown) - Convert word docs into markdown while using reverse_markdown, by Ben Balter
 - [markdown syntax](http://daringfireball.net/projects/markdown) - The markdown syntax specification
-- [html_massage](https://rubygems.org/gems/html_massage) - A gem by Harlan T. Wood to convert regular sites into markdown
+- [github flavored markdown](https://help.github.com/articles/github-flavored-markdown) - Githubs extension to markdown
+- [wmd-editor](http://wmd-editor.com) - Markdown flavored text editor
+
 
 # Thanks
 
 ..to Ben Woosley for his improvements to the first version.
 
 ..to Harlan T. Wood for his help with the newer versions.
+
+..and all contributors

data/bin/reverse_markdown

@@ -2,4 +2,4 @@
 # Usage: reverse_markdown [FILE]...
 # Usage: cat FILE | reverse_markdown
 require 'reverse_markdown'
-puts ReverseMarkdown.parse(ARGF.read)
+puts ReverseMarkdown.convert(ARGF.read)

data/lib/reverse_markdown/cleaner.rb

@@ -0,0 +1,66 @@
+module ReverseMarkdown
+  class Cleaner
+
+    def tidy(string)
+      clean_tag_borders(remove_leading_newlines(remove_newlines(remove_inner_whitespaces(string))))
+    end
+
+    def remove_newlines(string)
+      string.gsub(/\n{3,}/, "\n\n")
+    end
+
+    def remove_leading_newlines(string)
+      string.gsub /\A\n\n?/, ''
+    end
+
+    def remove_inner_whitespaces(string)
+      string.each_line.inject("") do |memo, line|
+        memo + preserve_border_whitespaces(line) do
+          line.strip.gsub(/[ \t]{2,}/, ' ')
+        end
+      end
+    end
+
+    # Find non-asterisk content that is enclosed by two or
+    # more asterisks. Ensure that only one whitespace occur
+    # in the border area.
+    # Same for underscores and brackets.
+    def clean_tag_borders(string)
+      result = string.gsub /\s?\*{2,}.*?\*{2,}\s?/ do |match|
+        preserve_border_whitespaces(match, default_border: ' ') do
+          match.strip.sub('** ', '**').sub(' **', '**')
+        end
+      end
+
+      result = result.gsub /\s?\_{2,}.*?\_{2,}\s?/ do |match|
+        preserve_border_whitespaces(match, default_border: ' ') do
+          match.strip.sub('__ ', '__').sub(' __', '__')
+        end
+      end
+
+      result.gsub /\s?\[.*?\]\s?/ do |match|
+        preserve_border_whitespaces(match) do
+          match.strip.sub('[ ', '[').sub(' ]', ']')
+        end
+      end
+    end
+
+    private
+
+    def preserve_border_whitespaces(string, default_border: '', &block)
+      string_start = present_or_default(string[/\A\s*/], default_border)
+      string_end   = present_or_default(string[/\s*\Z/], default_border)
+      result       = yield
+      string_start + result + string_end
+    end
+
+    def present_or_default(string, default)
+      if string.nil? || string.empty?
+        default
+      else
+        string
+      end
+    end
+
+  end
+end

data/lib/reverse_markdown/config.rb

@@ -0,0 +1,20 @@
+module ReverseMarkdown
+  class Config
+    attr_accessor :ignore_unknown_tags, :github_flavored
+
+    def initialize
+      reset
+    end
+
+    def apply(options = {})
+      options.each do |method, value|
+        __send__(:"#{method}=", value)
+      end
+    end
+
+    def reset
+      @ignore_unknown_tags = true
+      @github_flavored     = false
+    end
+  end
+end

data/lib/reverse_markdown/converters/a.rb

@@ -0,0 +1,19 @@
+module ReverseMarkdown
+  module Converters
+    class A < Base
+      def convert(node)
+        name  = treat_children(node)
+        href  = node['href']
+        title = extract_title(node)
+
+        if href.to_s.start_with?('#') || href.to_s.empty? || name.empty?
+          name
+        else
+          " [#{name}](#{href}#{title})"
+        end
+      end
+    end
+
+    register :a, A.new
+  end
+end

data/lib/reverse_markdown/converters/base.rb

@@ -0,0 +1,24 @@
+module ReverseMarkdown
+  module Converters
+    class Base
+      def treat_children(node)
+        node.children.inject('') do |memo, child|
+          memo << treat(child)
+        end
+      end
+
+      def treat(node)
+        ReverseMarkdown::Converters.lookup(node.name).convert(node)
+      end
+
+      def escape_keychars(string)
+        string.gsub(/[\*\_]/, '*' => '\*', '_' => '\_')
+      end
+
+      def extract_title(node)
+        title = escape_keychars(node['title'].to_s)
+        title.empty? ? '' : %[ "#{title}"]
+      end
+    end
+  end
+end

data/lib/reverse_markdown/converters/blockquote.rb

@@ -0,0 +1,13 @@
+module ReverseMarkdown
+  module Converters
+    class Blockquote < Base
+      def convert(node)
+        content = treat_children(node).strip
+        content = ReverseMarkdown.cleaner.remove_newlines(content)
+        '> ' << content.lines.join('> ')
+      end
+    end
+
+    register :blockquote, Blockquote.new
+  end
+end

data/lib/reverse_markdown/converters/br.rb

@@ -0,0 +1,11 @@
+module ReverseMarkdown
+  module Converters
+    class Br < Base
+      def convert(node)
+        "  \n"
+      end
+    end
+
+    register :br, Br.new
+  end
+end

data/lib/reverse_markdown/converters/code.rb

@@ -0,0 +1,11 @@
+module ReverseMarkdown
+  module Converters
+    class Code < Base
+      def convert(node)
+        "`#{node.text}`"
+      end
+    end
+
+    register :code, Code.new
+  end
+end

data/lib/reverse_markdown/converters/div.rb

@@ -0,0 +1,11 @@
+module ReverseMarkdown
+  module Converters
+    class Div < Base
+      def convert(node)
+        "\n" << treat_children(node) << "\n"
+      end
+    end
+
+    register :div, Div.new
+  end
+end

data/lib/reverse_markdown/converters/dump.rb

@@ -0,0 +1,9 @@
+module ReverseMarkdown
+  module Converters
+    class Dump < Base
+      def convert(node)
+        node.to_s
+      end
+    end
+  end
+end

data/lib/reverse_markdown/converters/em.rb

@@ -0,0 +1,21 @@
+module ReverseMarkdown
+  module Converters
+    class Em < Base
+      def convert(node)
+        content = treat_children(node)
+        if content.strip.empty? || already_italic?(node)
+          content
+        else
+          "_#{content}_"
+        end
+      end
+
+      def already_italic?(node)
+        node.ancestors('italic').size > 0 || node.ancestors('em').size > 0
+      end
+    end
+
+    register :em, Em.new
+    register :i,  Em.new
+  end
+end

data/lib/reverse_markdown/converters/h.rb

@@ -0,0 +1,17 @@
+module ReverseMarkdown
+  module Converters
+    class H < Base
+      def convert(node)
+        prefix = '#' * node.name[/\d/].to_i
+        ["\n", prefix, ' ', treat_children(node), "\n"].join
+      end
+    end
+
+    register :h1, H.new
+    register :h2, H.new
+    register :h3, H.new
+    register :h4, H.new
+    register :h5, H.new
+    register :h6, H.new
+  end
+end

data/lib/reverse_markdown/converters/hr.rb

@@ -0,0 +1,11 @@
+module ReverseMarkdown
+  module Converters
+    class Hr < Base
+      def convert(node)
+        "\n* * *\n"
+      end
+    end
+
+    register :hr, Hr.new
+  end
+end

data/lib/reverse_markdown/converters/img.rb

@@ -0,0 +1,14 @@
+module ReverseMarkdown
+  module Converters
+    class Img < Base
+      def convert(node)
+        alt   = node['alt']
+        src   = node['src']
+        title = extract_title(node)
+        " ![#{alt}](#{src}#{title})"
+      end
+    end
+
+    register :img, Img.new
+  end
+end

data/lib/reverse_markdown/converters/li.rb

@@ -0,0 +1,28 @@
+module ReverseMarkdown
+  module Converters
+    class Li < Base
+      def convert(node)
+        content     = treat_children(node)
+        indentation = indentation_for(node)
+        prefix      = prefix_for(node)
+        "#{indentation}#{prefix}#{content}\n"
+      end
+
+      def prefix_for(node)
+        if node.parent.name == 'ol'
+          index = node.parent.xpath('li').index(node)
+          "#{index.to_i + 1}. "
+        else
+          '- '
+        end
+      end
+
+      def indentation_for(node)
+        length = node.ancestors('ol').size + node.ancestors('ul').size
+        '  ' * (length - 1)
+      end
+    end
+
+    register :li, Li.new
+  end
+end

data/lib/reverse_markdown/converters/ol.rb

@@ -0,0 +1,12 @@
+module ReverseMarkdown
+  module Converters
+    class Ol < Base
+      def convert(node)
+        "\n" << treat_children(node)
+      end
+    end
+
+    register :ol, Ol.new
+    register :ul, Ol.new
+  end
+end

data/lib/reverse_markdown/converters/p.rb

@@ -0,0 +1,11 @@
+module ReverseMarkdown
+  module Converters
+    class P < Base
+      def convert(node)
+        "\n\n" << treat_children(node).strip << "\n\n"
+      end
+    end
+
+    register :p, P.new
+  end
+end

data/lib/reverse_markdown/converters/pass_through.rb

@@ -0,0 +1,14 @@
+module ReverseMarkdown
+  module Converters
+    class PassThrough < Base
+      def convert(node)
+        treat_children(node)
+      end
+    end
+
+    register :document, PassThrough.new
+    register :html,     PassThrough.new
+    register :body,     PassThrough.new
+    register :span,     PassThrough.new
+  end
+end

data/lib/reverse_markdown/converters/pre.rb

@@ -0,0 +1,17 @@
+require 'base64'
+
+module ReverseMarkdown
+  module Converters
+    class Pre < Base
+      def convert(node)
+        if ReverseMarkdown.config.github_flavored
+          "```\n" << node.text.strip << "\n```\n"
+        else
+          "\n\n    " << node.text.strip.lines.join("    ") << "\n\n"
+        end
+      end
+    end
+
+    register :pre, Pre.new
+  end
+end

data/lib/reverse_markdown/converters/strong.rb

@@ -0,0 +1,21 @@
+module ReverseMarkdown
+  module Converters
+    class Strong < Base
+      def convert(node)
+        content = treat_children(node)
+        if content.strip.empty? || already_strong?(node)
+          content
+        else
+          "**#{content}**"
+        end
+      end
+
+      def already_strong?(node)
+        node.ancestors('strong').size > 0 || node.ancestors('b').size > 0
+      end
+    end
+
+    register :strong, Strong.new
+    register :b,      Strong.new
+  end
+end

data/lib/reverse_markdown/converters/table.rb

@@ -0,0 +1,11 @@
+module ReverseMarkdown
+  module Converters
+    class Table < Base
+      def convert(node)
+        "\n\n" << treat_children(node) << "\n"
+      end
+    end
+
+    register :table, Table.new
+  end
+end

data/lib/reverse_markdown/converters/td.rb

@@ -0,0 +1,13 @@
+module ReverseMarkdown
+  module Converters
+    class Td < Base
+      def convert(node)
+        content = treat_children(node)
+        " #{content} |"
+      end
+    end
+
+    register :td, Td.new
+    register :th, Td.new
+  end
+end

data/lib/reverse_markdown/converters/text.rb

@@ -0,0 +1,30 @@
+module ReverseMarkdown
+  module Converters
+    class Text < Base
+      def convert(node, options = {})
+        if node.text.strip.empty?
+          treat_empty(node)
+        else
+          treat_text(node)
+        end
+      end
+
+      def treat_empty(node)
+        parent = node.parent.name.to_sym
+        if [:ol, :ul].include?(parent)  # Otherwise the identation is broken
+          ''
+        elsif node.text == ' '          # Regular whitespace text node
+          ' '
+        else
+          ''
+        end
+      end
+
+      def treat_text(node)
+        escape_keychars node.text.tr("\n\t", '').squeeze(' ')
+      end
+    end
+
+    register :text, Text.new
+  end
+end

data/lib/reverse_markdown/converters/tr.rb

@@ -0,0 +1,21 @@
+module ReverseMarkdown
+  module Converters
+    class Tr < Base
+      def convert(node)
+        content = treat_children(node).rstrip
+        result  = "|#{content}\n"
+        table_header_row?(node) ? result + underline_for(node) : result
+      end
+
+      def table_header_row?(node)
+        node.element_children.all? {|child| child.name.to_sym == :th}
+      end
+
+      def underline_for(node)
+        "| " + (['---'] * node.element_children.size).join(' | ') + " |\n"
+      end
+    end
+
+    register :tr, Tr.new
+  end
+end