reverse_asciidoctor 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: cb94b96658189613a9f4cdf21a3fa5d3eefbad17
+  data.tar.gz: 392dd9b562dd8a1d24193d4031f62916edda8fd2
+SHA512:
+  metadata.gz: a51d1071f41d2627def8bb06929bcb1308a673741067b29be58ad847dfb2d70ca7028e1cca1f52f9844310c378009395e61e0e0ffad1b8686a9ab3a21b7966a7
+  data.tar.gz: b829c7f3f60651a916c2fd23322406a581c7524825a804e6e26ea6953cb45cfe44f6ba316baeb2cebc92c9bd6388dfa976757f4ae1a0cd6bc487e279a2c0b611

data/Gemfile

@@ -0,0 +1,4 @@
+source "http://rubygems.org"
+
+# Specify your gem's dependencies in reverse_markdown.gemspec
+gemspec

data/LICENSE

@@ -0,0 +1,32 @@
+BSD 2-Clause License
+
+Copyright (c) 2018, Ribose
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+* Redistributions of source code must retain the above copyright notice, this
+  list of conditions and the following disclaimer.
+
+* Redistributions in binary form must reproduce the above copyright notice,
+  this list of conditions and the following disclaimer in the documentation
+  and/or other materials provided with the distribution.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+
+
+LICENSE OF https://github.com/xijo/reverse_markdown
+
+https://github.com/xijo/reverse_markdown, on which this gem is based, was
+licensed with the DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE, v2.

data/README.adoc

@@ -0,0 +1,186 @@
+= reverse_asciidoctor
+
+Based on https://github.com/xijo/reverse_markdown
+
+Transforms HTML into asciidoctor.
+
+== Requirements
+
+. http://nokogiri.org/[Nokogiri]
+. Ruby 1.9.3 or higher
+
+== Installation
+
+Install the gem
+
+[source,console]
+----
+[sudo] gem install reverse_asciidoctor
+----
+
+or add it to your Gemfile
+
+[source,ruby]
+----
+gem 'reverse_asciidoctor'
+----
+
+== Features
+
+As a port of reverse_markdown, reverse_asciidoctor shares its features:
+
+* Module based - if you miss a tag, just add it
+* Can deal with nested lists
+* Inline and block code is supported
+* Supports blockquote
+
+It supports the following html tags supported by reverse_markdown:
+
+* `a`
+* `blockquote`
+* `br`
+* `code`, `tt` (added: `kbd`, `samp`, `var`)
+* `div`, `article`
+* `em`, `i` (added: `cite`)
+* `h1`, `h2`, `h3`, `h4`, `h5`, `h6`, `hr`
+* `img`
+* `li`, `ol`, `ul` (added: `dir`)
+* `p`, `pre`
+* `strong`, `b`
+* `table`, `td`, `th`, `tr`
+
+NOTE:
+* reverse_asciidoctor does *not* support `del` or `strike`, because Asciidoctor out of the box does not
+* As with reverse_markdown, `pre` is only treated as sourcecode if it is contained in a `div@class = highlight-` element, or has a `@brush` attribute naming the language (Confluence).
+* The gem does not support `p@align`, because Asciidoctor doesn't
+
+In addition, it supports:
+
+* `aside`
+* `audio`, `video` (with `@src` attributes)
+* `figure`, `figcaption`
+* `mark`
+* `q`
+* `sub`, `sup`
+* `@id` anchors
+* `blockquote@cite`
+* `img/@width`, `img/@height`
+* `ol/@style`, `ol/@start`, `ol/@reversed`, `ul/@type`
+* `td/@colspan`, `td/@rowspan`, `td@/align`, `td@/valign`
+* `table/caption`, `table/@width`, `table/@frame` (partial), `table/@rules` (partial)
+* Lists and paragraphs within cells
+** Not tables within cells: Asciidoctor cannot deal with nested tabls
+
+It also supports MathML... sort of.
+
+* Asciidoctor supports AsciiMath and LaTeX for stem expressions. HTML uses MathML.
+The gem will recognise MathML expressions in HTML, and will wrap them in Asciidoctor
+`stem:[ ]` macros. The result of this gem is not actually legal Asciidoctor for stem: 
+Asciidoctor will presumably
+think this is AsciiMath in the `stem:[ ]` macro, try to pass it into MathJax as
+AsciiMath, and fail. But of course, MathJax has no problem with MathML, and some postprocessing
+on the Asciidoctor output can ensure that the MathML is treated by MathJax (or whatever else
+uses the output) as such; so this is still much better than nothing for stem processing.
+* An alternative would be to attempt to map MathML to either LaTeX or AsciiMath.
+** The self-description of https://github.com/learningobjectsinc/mathml-to-asciimath
+("subset"... "this module is not: comprehensive, performant") does not recommend it,
+when MathJax is entirely happy with MathML anyway.
+** https://github.com/transpect/mml2tex looks rather more robust, and is also used
+to export Word documents and their OOMML to LaTeX via MathML. But we'd still rather
+keep the MathML in place.
+
+The gem does not support:
+
+* `col`, `colgroup`
+* `source`, `picture`
+* `bdi`, `bdo`, `ruby`, `rt`, `rp`, `wbr`
+* `frame`, `frameset`, `iframe`, `noframes`, `noscript`, `script`, `input`, `output`, `progress`
+* `map`, `canvas`, `dialog`, `embed`, `object`, `param`, `svg`, `track`
+* `fieldset`, `button`, `datalist`, `form`, `label`, `legend`, `menu`, `menulist`, `optgroup`, `option`, `select`, `textarea`
+* `big`, `dfn`, `font`, `s`, `small`, `span`, `strike`, `u`
+* `center`
+* `data`, `meter`
+* `del`, `ins`
+* `footer`, `header`, `main`, `nav`, `details`, `section`, `summary`, `template`
+
+== Usage
+
+=== Ruby
+
+You can convert html content as string or Nokogiri document:
+
+[source,ruby]
+----
+input  = '<strong>feelings</strong>'
+result = ReverseAsciidoctor.convert input
+result.inspect # " *feelings* "
+----
+
+=== Commandline
+
+It's also possible to convert html files to markdown using the binary:
+
+[source,console]
+----
+$ bin/reverse_asciidoctor file.html > file.adoc
+$ cat file.html | bin/reverse_asciidoctor > file.adoc
+----
+
+In addition, the `bin/w2m` script (
+adapted from https://github.com/benbalter/word-to-markdown[Ben Balter's word-to-markdown])
+script extracts HTML from Word docx documents, and converts it to Asciidoc.
+
+[source,console]
+----
+$ bundle exec bin/w2m document.docx > document.adoc
+----
+
+The script presumes that LibreOffice has already been installed: it uses LibreOffice's
+export to XHTML. LibreOffice's export of XHTML is superior to the native Microsoft Word export
+to HTML: it exports lists (which Word keeps as paragraphs), and it exports OOMML into MathML.
+On the other hand, the LibreOffice export relies on 
+
+=== Configuration
+
+The following options are available:
+
+* `unknown_tags` (default `pass_through`) - how to handle unknown tags. Valid options are:
+** `pass_through` - Include the unknown tag completely into the result
+** `drop` - Drop the unknown tag and its content
+** `bypass` - Ignore the unknown tag but try to convert its content
+** `raise` - Raise an error to let you know
+* `tag_border` (default `' '`) - how to handle tag borders. valid options are:
+** `' '` - Add whitespace if there is none at tag borders.
+** `''` - Do not not add whitespace.
+
+==== As options
+
+Just pass your chosen configuration options in after the input. The given options will last for this operation only.
+
+[source,ruby]
+----
+ReverseAsciidoctor.convert(input, unknown_tags: :raise)
+----
+
+==== Preconfigure
+
+Or configure it block style on a initializer level. These configurations will last for all conversions until they are set to something different.
+
+[source,ruby]
+----
+ReverseAsciidoctor.config do |config|
+  config.unknown_tags     = :bypass
+  config.github_flavored  = true
+  config.tag_border  = ''
+end
+----
+
+
+== Related stuff
+
+* https://github.com/xijo/reverse_markdown[Xijo's original reverse_markdown gem]
+* https://github.com/xijo/reverse_markdown/wiki/Write-your-own-converter[Write custom converters] - Wiki entry about how to write your own converter
+* https://github.com/harlantwood/html_massage[html_massage] - A gem by Harlan T. Wood to convert regular sites into markdown using reverse_markdown
+* https://github.com/benbalter/word-to-markdown[word-to-markdown] - Convert word docs into markdown while using reverse_markdown, by Ben Balter
+* https://github.com/asciidocfx/HtmlToAsciidoc[HtmlToAsciidoc] - Javascript regexp-based converter of HTML to Asciidoctor
+* https://asciidoctor.org/docs/user-manual/[The Asciidoctor User Manual]

data/Rakefile

@@ -0,0 +1,14 @@
+require 'bundler/gem_tasks'
+
+if File.exist?('.codeclimate')
+  ENV["CODECLIMATE_REPO_TOKEN"] = File.read('.codeclimate').strip
+end
+
+require 'rspec/core/rake_task'
+RSpec::Core::RakeTask.new(:spec)
+task :default => :spec
+
+desc 'Open an irb session preloaded with this library'
+task :console do
+  sh 'irb -rubygems -I lib -r reverse_asciidoctor.rb'
+end

data/bin/reverse_asciidoctor

@@ -0,0 +1,14 @@
+#!/usr/bin/env ruby
+# Usage: reverse_asciidoctor [FILE]...
+# Usage: cat FILE | reverse_asciidoctor
+require 'reverse_asciidoctor'
+require 'optparse'
+
+options = {}
+OptionParser.new do |opts|
+  opts.banner = "Usage: reverse_asciidoctor [options] <file>"
+
+  opts.on('-u', '--unknown_tags [pass_through, drop, bypass, raise]', 'Unknown tag handling (default: pass_through)') { |v| ReverseMarkdown.config.unknown_tags = v }
+end.parse!
+
+puts ReverseAsciidoctor.convert(ARGF.read)

data/bin/w2m

@@ -0,0 +1,31 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require 'word-to-markdown'
+require 'reverse_asciidoctor'
+
+def scrub_whitespace(string)
+  string = string.dup
+  string.gsub!(' ', ' ')       # HTML encoded spaces
+  string.sub!(/\A[[:space:]]+/, '') # document leading whitespace
+  string.sub!(/[[:space:]]+\z/, '') # document trailing whitespace
+  string.gsub!(/([ ]+)$/, '')       # line trailing whitespace
+  string.gsub!(/\n\n\n\n/, "\n\n")  # Quadruple line breaks
+  string.delete!(' ')               # Unicode non-breaking spaces, injected as tabs
+  string
+end
+
+if ARGV.size != 1 || ARGV[0] == '--help'
+  puts 'Usage: bundle exec w2m path/to/document.docx'
+  exit 1
+end
+
+if ARGV[0] == '--version'
+  puts "WordToMarkdown v#{WordToMarkdown::VERSION}"
+  puts "LibreOffice v#{WordToMarkdown.soffice.version}" unless Gem.win_platform?
+else
+  doc = WordToMarkdown.new ARGV[0]
+  # puts doc.to_s
+  puts ReverseAsciidoctor.convert(scrub_whitespace(doc.document.html), WordToMarkdown::REVERSE_MARKDOWN_OPTIONS)
+end
+

data/lib/reverse_asciidoctor.rb

@@ -0,0 +1,70 @@
+require 'digest'
+require 'nokogiri'
+require 'reverse_asciidoctor/version'
+require 'reverse_asciidoctor/errors'
+require 'reverse_asciidoctor/cleaner'
+require 'reverse_asciidoctor/config'
+require 'reverse_asciidoctor/converters'
+require 'reverse_asciidoctor/converters/base'
+require 'reverse_asciidoctor/converters/a'
+require 'reverse_asciidoctor/converters/aside'
+require 'reverse_asciidoctor/converters/audio'
+require 'reverse_asciidoctor/converters/blockquote'
+require 'reverse_asciidoctor/converters/br'
+require 'reverse_asciidoctor/converters/bypass'
+require 'reverse_asciidoctor/converters/code'
+require 'reverse_asciidoctor/converters/div'
+require 'reverse_asciidoctor/converters/drop'
+require 'reverse_asciidoctor/converters/em'
+require 'reverse_asciidoctor/converters/figure'
+require 'reverse_asciidoctor/converters/h'
+require 'reverse_asciidoctor/converters/head'
+require 'reverse_asciidoctor/converters/hr'
+require 'reverse_asciidoctor/converters/ignore'
+require 'reverse_asciidoctor/converters/img'
+require 'reverse_asciidoctor/converters/mark'
+require 'reverse_asciidoctor/converters/li'
+require 'reverse_asciidoctor/converters/ol'
+require 'reverse_asciidoctor/converters/p'
+require 'reverse_asciidoctor/converters/pass_through'
+require 'reverse_asciidoctor/converters/pre'
+require 'reverse_asciidoctor/converters/q'
+require 'reverse_asciidoctor/converters/strong'
+require 'reverse_asciidoctor/converters/sup'
+require 'reverse_asciidoctor/converters/sub'
+require 'reverse_asciidoctor/converters/table'
+require 'reverse_asciidoctor/converters/td'
+require 'reverse_asciidoctor/converters/th'
+require 'reverse_asciidoctor/converters/text'
+require 'reverse_asciidoctor/converters/tr'
+require 'reverse_asciidoctor/converters/video'
+require 'reverse_asciidoctor/converters/math'
+
+module ReverseAsciidoctor
+
+  def self.convert(input, options = {})
+    root = case input
+      when String                  then Nokogiri::HTML(input).root
+      when Nokogiri::XML::Document then input.root
+      when Nokogiri::XML::Node     then input
+    end
+
+    root or return ''
+
+    config.with(options) do
+      result = ReverseAsciidoctor::Converters.lookup(root.name).convert(root)
+      cleaner.tidy(result)
+    end
+  end
+
+  def self.config
+    @config ||= Config.new
+    yield @config if block_given?
+    @config
+  end
+
+  def self.cleaner
+    @cleaner ||= Cleaner.new
+  end
+
+end

data/lib/reverse_asciidoctor/cleaner.rb

@@ -0,0 +1,85 @@
+module ReverseAsciidoctor
+  class Cleaner
+
+    def tidy(string)
+      result = remove_inner_whitespaces(string)
+      result = remove_newlines(result)
+      result = remove_leading_newlines(result)
+      result = clean_tag_borders(result)
+      clean_punctuation_characters(result)
+    end
+
+    def remove_newlines(string)
+      string.gsub(/\n{3,}/, "\n\n")
+    end
+
+    def remove_leading_newlines(string)
+      string.gsub(/\A\n+/, '')
+    end
+
+    def remove_inner_whitespaces(string)
+      string.each_line.inject("") do |memo, line|
+        memo + preserve_border_whitespaces(line) do
+          line.strip.gsub(/[ \t]{2,}/, ' ')
+        end
+      end
+    end
+
+    # Find non-asterisk content that is enclosed by two or
+    # more asterisks. Ensure that only one whitespace occurs
+    # in the border area.
+    # Same for underscores and brackets.
+    def clean_tag_borders(string)
+      result = string.gsub(/\s?\*{2,}.*?\*{2,}\s?/) do |match|
+        preserve_border_whitespaces(match, default_border: ReverseAsciidoctor.config.tag_border) do
+          match.strip.sub('** ', '**').sub(' **', '**')
+        end
+      end
+
+      result = result.gsub(/\s?\_{2,}.*?\_{2,}\s?/) do |match|
+        preserve_border_whitespaces(match, default_border: ReverseAsciidoctor.config.tag_border) do
+          match.strip.sub('__ ', '__').sub(' __', '__')
+        end
+      end
+
+      result = result.gsub(/\s?~{2,}.*?~{2,}\s?/) do |match|
+        preserve_border_whitespaces(match, default_border: ReverseAsciidoctor.config.tag_border) do
+          match.strip.sub('~~ ', '~~').sub(' ~~', '~~')
+        end
+      end
+
+      result.gsub(/\s?\[.*?\]\s?/) do |match|
+        preserve_border_whitespaces(match) do
+          match.strip.sub('[ ', '[').sub(' ]', ']')
+        end
+      end
+    end
+
+    def clean_punctuation_characters(string)
+      string.gsub(/(\*\*|~~|__)\s([\.!\?'"])/, "\\1".strip + "\\2")
+    end
+
+    private
+
+    def preserve_border_whitespaces(string, options = {}, &block)
+      return string if string =~ /\A\s*\Z/
+      default_border = options.fetch(:default_border, '')
+      # If the string contains part of a link so the characters [,],(,)
+      # then don't add any extra spaces
+      default_border = '' if string =~ /[\[\(\]\)]/
+      string_start   = present_or_default(string[/\A\s*/], default_border)
+      string_end     = present_or_default(string[/\s*\Z/], default_border)
+      result         = yield
+      string_start + result + string_end
+    end
+
+    def present_or_default(string, default)
+      if string.nil? || string.empty?
+        default
+      else
+        string
+      end
+    end
+
+  end
+end