RubyGems - coradoc - Versions diffs - 1.1.7 → 2.0.2 - Mend

coradoc 1.1.7 → 2.0.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (1368) hide show

data/lib/coradoc/input/html/README.adoc DELETED Viewed

@@ -1,308 +0,0 @@
-= AsciiDoc from HTML and Microsoft Word: formerly reverse_adoc
-== Purpose
-Transforms HTML and Microsoft Word into AsciiDoc.
-Based on https://github.com/xijo/reverse_markdown
-reverse_adoc used to be a separate Gem, but now it's part of Coradoc.
-== Installation
-Install the gem:
-[source,console]
-----
-[sudo] gem install coradoc
-----
-or add it to your `Gemfile`:
-[source,ruby]
-----
-gem 'coradoc'
-----
-== Command-line usage
-=== HTML to AsciiDoc: `reverse_adoc`
-Convert HTML files to AsciiDoc:
-[source,console]
-----
-$ reverse_adoc file.html > file.adoc
-$ cat file.html | reverse_adoc > file.adoc
-----
-=== Microsoft Word to AsciiDoc: `w2a`
-Convert Word `.doc` or `.docx` files to AsciiDoc:
-[source,console]
-----
-$ w2a file.docx > file.adoc
-----
-[source,console]
-----
-$ w2a input.docx -o output.adoc
-----
-Help:
-[source,console]
-----
-$ w2a -h
-Usage: w2a [options] <file>
-    -a, --mathml2asciimath           Convert MathML to AsciiMath
-    -o, --output=FILENAME            Output file to write to
-    -e, --external-images            Export images if data URI
-    -v, --version                    Version information
-    -h, --help                       Prints this help
-----
-NOTE: `w2a` requires LibreOffice to be installed. It uses LibreOffice's
-export to XHTML. LibreOffice's export of XHTML is superior to the native Microsoft Word export
-to HTML: it exports lists (which Word keeps as paragraphs), and it exports OOMML into MathML.
-On the other hand, the LibreOffice export relies on default styling being used in the
-document, and it may not cope with ordered lists or headings with customised appearance.
-For best results, reset the styles in the document you're converting to those in
-the default `Normal.dot` template.
-NOTE: `w2a` requires the command-line version of LibreOffice, `soffice`. As it turns out,
-LibreOffice v6 appears to render formulae in HTML as images instead of MathML expressions;
-use LibreOffice v5. If you have both LibreOffice v5 and LibreOffice v6 installed, make sure
-that your OS path searches for the LibreOffice v5 version of `soffice` first; e.g. on Mac,
-include something like `/Applications/LibreOffice5.4.7.2.app/Contents/MacOS` in your PATH
-environment.
-NOTE: Some information in OOMML is not preserved in the export to MathML from LibreOffice;
-in particular, font shifts such as double-struck fonts.
-The LibreOffice exporter does seem to drop some text (possibly associated with
-MathML); use with caution.
-NOTE: Adapted from `w2m` of
-https://github.com/benbalter/word-to-markdown[Ben Balter's word-to-markdown]
-=== Common options
-==== MathML to AsciiMath conversion
-If you wish to convert the MathML in the document to AsciiMath, run the script with the
-`--mathml2asciimath` option:
-[source,console]
-----
-$ w2a --mathml2asciimath document.docx > document.adoc
-----
-==== Extracting images
-Images referred by the HTML can be extracted into the destination output folder by using:
-[source,console]
-----
-$ reverse_adoc input.docx -o output/file.adoc -e
-$ reverse_adoc input.docx --output output/file.adoc --external-images
-----
-Word embedded images can be extracted into the destination output folder by using:
-[source,console]
-----
-$ w2a input.docx -o output/file.adoc -e
-$ w2a input.docx --output output/file.adoc --external-images
-----
-==== Handling unknown HTML tags
-The `--unknown_tags` option allows you to specify how to handle unknown tags
-(default `pass_through`).
-Valid options are:
-* `pass_through` - Include the unknown tag completely into the result
-* `drop` - Drop the unknown tag and its content
-* `bypass` - Ignore the unknown tag but try to convert its content
-* `raise` - Raise an error to let you know
-==== Tagging of borders
-Specify how to handle tag borders with the option `--tag_border` (default `' '`).
-Valid options are:
-* `' '` - Add whitespace if there is none at tag borders.
-* `''` - Do not not add whitespace.
-== Features
-=== General
-`reverse_adoc` shares features as a port of `reverse_markdown`:
-* Module based -- if you miss a tag, just add it
-* Can deal with nested lists
-* Inline and block code is supported
-* Supports blockquote
-It supports the following HTML tags (these are supported by `reverse_markdown`):
-* `a`
-* `blockquote`
-* `br`
-* `code`, `tt` (added: `kbd`, `samp`, `var`)
-* `div`, `article`
-* `em`, `i` (added: `cite`)
-* `h1`, `h2`, `h3`, `h4`, `h5`, `h6`, `hr`
-* `img`
-* `li`, `ol`, `ul` (added: `dir`)
-* `p`, `pre`
-* `strong`, `b`
-* `table`, `td`, `th`, `tr`
-[NOTE]
-====
-* reverse_adoc does *not* support `del` or `strike`, because Asciidoctor does not out of the box.
-* As with reverse_markdown, `pre` is only treated as sourcecode if it is contained in a `div@class = highlight-` element, or has a `@brush` attribute naming the language (Confluence).
-* The gem does not support `p@align`, because Asciidoctor doesn't
-====
-In addition, it supports:
-* `aside`
-* `audio`, `video` (with `@src` attributes)
-* `figure`, `figcaption`
-* `mark`
-* `q`
-* `sub`, `sup`
-* `@id` anchors
-* `blockquote@cite`
-* `img/@width`, `img/@height`
-* `ol/@style`, `ol/@start`, `ol/@reversed`, `ul/@type`
-* `td/@colspan`, `td/@rowspan`, `td@/align`, `td@/valign`
-* `table/caption`, `table/@width`, `table/@frame` (partial), `table/@rules` (partial)
-* Lists and paragraphs within cells
-** Not tables within cells: Asciidoctor cannot deal with nested tables
-The gem does not support:
-* `col`, `colgroup`
-* `source`, `picture`
-* `bdi`, `bdo`, `ruby`, `rt`, `rp`, `wbr`
-* `frame`, `frameset`, `iframe`, `noframes`, `noscript`, `script`, `input`, `output`, `progress`
-* `map`, `canvas`, `dialog`, `embed`, `object`, `param`, `svg`, `track`
-* `fieldset`, `button`, `datalist`, `form`, `label`, `legend`, `menu`, `menulist`, `optgroup`, `option`, `select`, `textarea`
-* `big`, `dfn`, `font`, `s`, `small`, `span`, `strike`, `u`
-* `center`
-* `data`, `meter`
-* `del`, `ins`
-* `footer`, `header`, `main`, `nav`, `details`, `section`, `summary`, `template`
-=== MathML support
-If you are using this gem in the context of https://www.metanorma.com[Metanorma],
-Metanorma AsciiDoc accepts MathML as a native mathematical format. So you do not need
-to convert the MathML to AsciiMath.
-The gem will optionally invoke the https://github.com/metanorma/mathml2asciimath
-gem, to convert MathML to AsciiMath. The conversion is not perfect, and will need to be
-post-edited; but it's a lot better than nothing.
-NOTE: Asciidoctor does not support MathML input. HTML uses MathML.
-The gem will recognize MathML expressions in HTML, and will wrap them in Asciidoctor
-`stem:[ ]` macros. The result of this gem is not actually legal Asciidoctor for `stem`:
-Asciidoctor will presumably
-think this is AsciiMath in the `stem:[ ]` macro, try to pass it into MathJax as
-AsciiMath, and fail. But of course, MathJax has no problem with MathML, and some postprocessing
-on the Asciidoctor output can ensure that the MathML is treated by MathJax (or whatever else
-uses the output) as such; so this is still much better than nothing for stem processing.
-=== Word cleanup
-This gem is routinely used in the Metanorma project to export Word documents to AsciiDoc.
-The HTML export from Word that the gem uses, from LibreOffice, is much cleaner than the
-native HTML 4 export from Word; but it has some infelicities which this gem cleans up:
-* The HTML export has trouble with subscripts, and routinely exports them as headings; the `w2a`
-script tries to clean them up.
-* The `w2a` cleans up spaces, but it does not strip them.
-* Spaces are removed from anchors and cross-references.
-* Double underscores are removed from anchors and cross-references.
-* Cross-references to `_GoBack` and to `_Toc` followed by numbers (used to construct tables of contents) are ignored.
-== Ruby library usage
-=== General
-Simple to use.
-[source,ruby]
-----
-require 'coradoc/input/html'
-result = Coradoc::Input::HTML.convert input
-result.inspect # " *feelings* "
-----
-=== Configure with options
-Just pass your chosen configuration options in after the input. The given options will last for this operation only.
-[source,ruby]
-----
-require 'coradoc/input/html'
-Coradoc::Input::HTML.convert(input, unknown_tags: :raise, mathml2asciimath: true)
-----
-=== Preconfigure using an initializer
-Or configure it block style on a initializer level. These configurations will last for all conversions until they are set to something different.
-[source,ruby]
-----
-require 'coradoc/input/html'
-Coradoc::Input::HTML.config do |config|
-  config.unknown_tags      = :bypass
-  config.mathml2asciimath  = true
-  config.tag_border  = ''
-end
-----
-=== Convert HTML to a Coradoc AST
-[source,ruby]
-----
-require 'coradoc/input/html'
-# Options can be supplied as keyword arguments
-Coradoc::Input::Html::HtmlConverter.to_coradoc("<b><i>Some input</i></b>")
-----
-== Related stuff
-* https://github.com/xijo/reverse_markdown[Xijo's original reverse_markdown gem]
-* https://github.com/xijo/reverse_markdown/wiki/Write-your-own-converter[Write custom converters] - Wiki entry about how to write your own converter
-* https://github.com/harlantwood/html_massage[html_massage] - A gem by Harlan T. Wood to convert regular sites into markdown using reverse_markdown
-* https://github.com/benbalter/word-to-markdown[word-to-markdown] - Convert word docs into markdown while using reverse_markdown, by Ben Balter
-* https://github.com/asciidocfx/HtmlToAsciidoc[HtmlToAsciidoc] - Javascript regexp-based converter of HTML to Asciidoctor
-* https://asciidoctor.org/docs/user-manual/[The Asciidoctor User Manual]

data/lib/coradoc/input/html/cleaner.rb DELETED Viewed

@@ -1,142 +0,0 @@
-module Coradoc
-  module Input
-    module Html
-      class Cleaner
-        def tidy(string)
-          if string.is_a? Hash
-            return string.transform_values { |i| tidy(i) }
-          end
-          result = HtmlConverter.track_time "Removing inner whitespace" do
-            remove_inner_whitespaces(String.new(string))
-          end
-          result = HtmlConverter.track_time "Removing newlines" do
-            remove_newlines(result)
-          end
-          result = HtmlConverter.track_time "Removing leading newlines" do
-            remove_leading_newlines(result)
-          end
-          result = HtmlConverter.track_time "Cleaning tag borders" do
-            clean_tag_borders(result)
-          end
-          result = HtmlConverter.track_time "Cleaning punctuation characters" do
-            clean_punctuation_characters(result)
-          end
-          result = remove_block_leading_newlines(result)
-          result = remove_section_attribute_newlines(result)
-        end
-        def remove_block_leading_newlines(string)
-          string.gsub("]\n****\n\n", "]\n****\n")
-        end
-        def remove_section_attribute_newlines(string)
-          string.gsub("]\n\n==", "]\n==")
-        end
-        def remove_newlines(string)
-          string.gsub(/\n{3,}/, "\n\n")
-        end
-        def remove_leading_newlines(string)
-          string.gsub(/\A\n+/, "")
-        end
-        def remove_inner_whitespaces(string)
-          unless string.nil?
-            string.gsub!(/\n stem:\[/, "\nstem:[")
-            string.gsub!(/(stem:\[([^\]]|\\\])*\])\n(?=\S)/, "\\1 ")
-            string.gsub!(/(stem:\[([^\]]|\\\])*\])\s+(?=[\^-])/, "\\1")
-          end
-          result = +""
-          string.each_line do |line|
-            result << preserve_border_whitespaces(line) do
-              line.strip.gsub(/[ \t]{2,}/, " ")
-            end
-          end
-          result
-        end
-        # Find non-asterisk content that is enclosed by two or
-        # more asterisks. Ensure that only one whitespace occurs
-        # in the border area.
-        # Same for underscores and brackets.
-        def clean_tag_borders(string)
-          # result = string.gsub(/\s?\*{2,}.*?\*{2,}\s?/) do |match|
-          # preserve_border_whitespaces(match, default_border: Coradoc::Input::HTML.config.tag_border) do
-          #   match.strip.sub("** ", "**").sub(" **", "**")
-          # end
-          # end
-          # result = string.gsub(/\s?_{2,}.*?_{2,}\s?/) do |match|
-          #   preserve_border_whitespaces(match, default_border: Coradoc::Input::HTML.config.tag_border) do
-          #     match.strip.sub("__ ", "__").sub(" __", "__")
-          #   end
-          # end
-          result = string.gsub(/\s?~{2,}.*?~{2,}\s?/) do |match|
-            preserve_border_whitespaces(match,
-                                        default_border: Coradoc::Input::Html.config.tag_border) do
-              match.strip.sub("~~ ", "~~").sub(" ~~", "~~")
-            end
-          end
-          result.gsub(/\s?\[.*?\]\s?/) do |match|
-            preserve_border_whitespaces(match) do
-              match.strip.sub("[ ", "[").sub(" ]", "]")
-            end
-          end
-        end
-        def clean_punctuation_characters(string)
-          string.gsub(/(\*\*|~~|__)\s([.!?'"])/, "\\1\\2")
-        end
-        # preprocesses HTML, rather than postprocessing it
-        def preprocess_word_html(string)
-          clean_headings(scrub_whitespace(string.dup))
-        end
-        def scrub_whitespace(string)
-          string.gsub!(/&nbsp;|&#xA0;|\u00a0/i, "&#xA0;") # HTML encoded spaces
-          string = Coradoc.strip_unicode(string) # Strip document-level leading and trailing whitespace
-          string.gsub!(/( +)$/, " ") # line trailing whitespace
-          string.gsub!(/\n\n\n\n/, "\n\n") # Quadruple line breaks
-          # string.delete!('?| ')               # Unicode non-breaking spaces, injected as tabs
-          string
-        end
-        # following added by me
-        def clean_headings(string)
-          string.gsub!(%r{<h([1-9])[^>]*></h\1>}, " ")
-          # I don't know why Libre Office is inserting them, but they need to go
-          string.gsub!(%r{<h([1-9])[^>]* style="vertical-align: super;[^>]*>(.+?)</h\1>},
-                       "<sup>\\2</sup>")
-          # I absolutely don't know why Libre Office is rendering superscripts as h1
-          string
-        end
-        private
-        def preserve_border_whitespaces(string, options = {})
-          return string if /\A\s*\Z/.match?(string)
-          default_border = options.fetch(:default_border, "")
-          # If the string contains part of a link so the characters [,],(,)
-          # then don't add any extra spaces
-          default_border = "" if /[\[(\])]/.match?(string)
-          string_start   = present_or_default(string[/\A\s*/], default_border)
-          string_end     = present_or_default(string[/\s*\Z/], default_border)
-          result         = yield
-          string_start + result + string_end
-        end
-        def present_or_default(string, default)
-          return default if string.nil? || string.empty?
-          string
-        end
-      end
-    end
-  end
-end

data/lib/coradoc/input/html/config.rb DELETED Viewed

@@ -1,77 +0,0 @@
-require "tmpdir"
-module Coradoc
-  module Input
-    module Html
-      class Config
-        def initialize
-          @unknown_tags     = :pass_through
-          @input_format     = :html
-          @mathml2asciimath = false
-          @external_images  = false
-          # Destination to save file and images
-          @destination      = nil
-          # Source of HTML
-          # @sourcedir        = nil
-          # Image counter, assuming there are max 999 images
-          @image_counter = 1
-          # pad with 0s
-          @image_counter_pattern = "%03d"
-          @em_delimiter     = "_".freeze
-          @strong_delimiter = "*".freeze
-          @inline_options   = {}
-          @tag_border       = " ".freeze
-          @split_sections   = nil
-          # Document width - used to compute table sizes.
-          # This is an assumption for screen size in input document.
-          # If column widths are specified in absolute values, then we
-          # have to convert them to relative values, as AsciiDoc only
-          # supports those.
-          @doc_width        = 1000
-          # Plugin system
-          @plugins          = []
-          # Debugging options
-          @track_time       = false
-        end
-        def with(options = {})
-          old_options = @inline_options
-          @inline_options = options
-          result = yield
-          @inline_options = old_options
-          result
-        end
-        def self.declare_option(option)
-          define_method(option) do
-            @inline_options[option] || instance_variable_get(:"@#{option}")
-          end
-          attr_writer option
-        end
-        declare_option :unknown_tags
-        declare_option :tag_border
-        declare_option :mathml2asciimath
-        declare_option :external_images
-        declare_option :destination
-        declare_option :sourcedir
-        declare_option :image_counter
-        declare_option :image_counter_pattern
-        declare_option :input_format
-        declare_option :split_sections
-        declare_option :doc_width
-        declare_option :plugins
-        declare_option :track_time
-      end
-    end
-  end
-end

data/lib/coradoc/input/html/converters/a.rb DELETED Viewed

@@ -1,52 +0,0 @@
-require "coradoc"
-module Coradoc
-  module Input
-    module Html
-      module Converters
-        class A < Base
-          def to_coradoc(node, state = {})
-            name  = treat_children(node, state)
-            href  = node["href"]
-            title = extract_title(node)
-            id = node["id"] || node["name"]
-            id = id&.gsub(/\s/, "")&.gsub(/__+/, "_")
-            id = nil if id&.empty?
-            return "" if /^_Toc\d+$|^_GoBack$/.match?(id)
-            return Coradoc::Element::Inline::Anchor.new(id) if id
-            if href.to_s.start_with?("#")
-              href = href.sub(/^#/, "").gsub(/\s/, "").gsub(/__+/, "_")
-              return Coradoc::Element::Inline::CrossReference.new(href, name)
-            end
-            return name if href.to_s.empty?
-            ambigous_characters = /[\w.?&#=%;\[\u{ff}-\u{10ffff}]/
-            if name&.strip == href
-              name = ""
-              right_constrain = textnode_after_start_with?(node,
-                                                           ambigous_characters)
-            end
-            out = []
-            out << " " if textnode_before_end_with?(node, ambigous_characters)
-            out << Coradoc::Element::Inline::Link.new(
-              path: href,
-              name: name.strip,
-              title: title.strip,
-              right_constrain: right_constrain,
-            )
-            out
-          end
-        end
-        register :a, A.new
-      end
-    end
-  end
-end

data/lib/coradoc/input/html/converters/aside.rb DELETED Viewed

@@ -1,16 +0,0 @@
-module Coradoc
-  module Input
-    module Html
-      module Converters
-        class Aside < Base
-          def to_coradoc(node, state = {})
-            content = treat_children(node, state)
-            Coradoc::Element::Block::Side.new(lines: content.lines)
-          end
-        end
-        register :aside, Aside.new
-      end
-    end
-  end
-end

data/lib/coradoc/input/html/converters/audio.rb DELETED Viewed

@@ -1,29 +0,0 @@
-module Coradoc
-  module Input
-    module Html
-      module Converters
-        class Audio < Base
-          def to_coradoc(node, _state = {})
-            src = node["src"]
-            id = node["id"]
-            title = extract_title(node)
-            attributes = Coradoc::Element::AttributeList.new
-            options = options(node)
-            attributes.add_named("options", options) if options.any?
-            Coradoc::Element::Audio.new(title, id: id, src: src,
-                                               attributes: attributes)
-          end
-          def options(node)
-            autoplay = node["autoplay"]
-            loop_attr = node["loop"]
-            controls = node["controls"]
-            [autoplay, loop_attr, controls].compact
-          end
-        end
-        register :audio, Audio.new
-      end
-    end
-  end
-end