reverse_asciidoctor 0.2.0 → 0.2.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 14f4831267dd9a4738994bd00b46eb45b4142441
-  data.tar.gz: ff0f1af49f22bbf573f02b0d3df20d725ab95df3
+  metadata.gz: 4401fdaa4562099df961e17489d733cfed7aa2b2
+  data.tar.gz: 0dd3c32893e3f3abe2b8a90ee0812966de9b3cbf
 SHA512:
-  metadata.gz: 2f9b24155a4091c855f8035353c1a192d339f2563e75fcfc93f52858462efc850a7ddeed813bbbc70c57524ae722ed19cba44401c808ac9c72fa8a776f968b84
-  data.tar.gz: '087d8216c45fffc860af770aff3ad1f81a473ba7c8e5b92bfbf9db9b60b59fb6f9bae6fe58b6a14756e7514232594363b9da168cb835e10ab5b836f048342183'
+  metadata.gz: d4e5b592c0a68ac39bf0846f43b8955720ad5457219942f40afe97cc74568d37f8a01072b2d52660923292ea516aa967b42bb2d649586548b1d9c95c74281422
+  data.tar.gz: ca556358cd939540a1c1e745eb2438b78038f4da121a05cce0da35e43ff5f0b8f1e3d44f7cf99eba8448874b65a64335427690bd4f606fe0f04a4feecdd52e8b

data/Gemfile

@@ -1,4 +1,6 @@
 source "http://rubygems.org"
 
+gem "mathml2asciimath", git: "https://github.com/riboseinc/mathml2asciimath"
+
 # Specify your gem's dependencies in reverse_markdown.gemspec
 gemspec

data/README.adoc

@@ -25,12 +25,66 @@ or add it to your Gemfile
 gem 'reverse_asciidoctor'
 ----
 
+== Usage
+
+=== Ruby
+
+You can convert html content as string or Nokogiri document:
+
+[source,ruby]
+----
+input  = '<strong>feelings</strong>'
+result = ReverseAsciidoctor.convert input
+result.inspect # " *feelings* "
+----
+
+=== Commandline
+
+It's also possible to convert html files to markdown using the binary:
+
+[source,console]
+----
+$ bin/reverse_asciidoctor file.html > file.adoc
+$ cat file.html | bin/reverse_asciidoctor > file.adoc
+----
+
+In addition, the `bin/w2a` script (adapted from the `bin/w2m` script in 
+https://github.com/benbalter/word-to-markdown[Ben Balter's word-to-markdown])
+extracts HTML from Word docx documents, and converts it to Asciidoc.
+
+[source,console]
+----
+$ bundle exec bin/w2a document.docx > document.adoc
+----
+
+The script presumes that LibreOffice has already been installed: it uses LibreOffice's
+export to XHTML. LibreOffice's export of XHTML is superior to the native Microsoft Word export
+to HTML: it exports lists (which Word keeps as paragraphs), and it exports OOMML into MathML.
+On the other hand, the LibreOffice export relies on default styling being used in the
+document, and it may not cope with ordered lists or headings with customised appearance.
+For best results, reset the styles in the document you're converting to those in
+the default Normal.dot template.
+
+If you wish to convert the MathML in the document to AsciiMath, run the script with the
+`--mathml2asciimath` option:
+
+[source,console]
+----
+$ bundle exec bin/w2a --mathml2asciimath document.docx > document.adoc
+----
+
+Note that some information in OOMML is not preserved in the export to MathML from LibreOffice;
+in particular, font shifts such as double-struck fonts.
+
+Note that the LibreOffice exporter does seem to drop some text (possibly associated with
+MathML); use with caution.
+
 == Features
 
 As a port of reverse_markdown, reverse_asciidoctor shares its features:
 
 * Module based - if you miss a tag, just add it
-* Can deal with nested lists
+* Can deal with nested lists 
 * Inline and block code is supported
 * Supports blockquote
 
@@ -43,7 +97,7 @@ It supports the following html tags supported by reverse_markdown:
 * `div`, `article`
 * `em`, `i` (added: `cite`)
 * `h1`, `h2`, `h3`, `h4`, `h5`, `h6`, `hr`
-* `img`
+* `img` 
 * `li`, `ol`, `ul` (added: `dir`)
 * `p`, `pre`
 * `strong`, `b`
@@ -60,7 +114,7 @@ In addition, it supports:
 
 * `aside`
 * `audio`, `video` (with `@src` attributes)
-* `figure`, `figcaption`
+* `figure`, `figcaption` 
 * `mark`
 * `q`
 * `sub`, `sup`
@@ -77,19 +131,15 @@ It also supports MathML... sort of.
 
 * Asciidoctor supports AsciiMath and LaTeX for stem expressions. HTML uses MathML.
 The gem will recognise MathML expressions in HTML, and will wrap them in Asciidoctor
-`stem:[ ]` macros. The result of this gem is not actually legal Asciidoctor for stem: 
+`stem:[ ]` macros. The result of this gem is not actually legal Asciidoctor for stem:
 Asciidoctor will presumably
 think this is AsciiMath in the `stem:[ ]` macro, try to pass it into MathJax as
 AsciiMath, and fail. But of course, MathJax has no problem with MathML, and some postprocessing
 on the Asciidoctor output can ensure that the MathML is treated by MathJax (or whatever else
 uses the output) as such; so this is still much better than nothing for stem processing.
-* An alternative would be to attempt to map MathML to either LaTeX or AsciiMath.
-** The self-description of https://github.com/learningobjectsinc/mathml-to-asciimath
-("subset"... "this module is not: comprehensive, performant") does not recommend it,
-when MathJax is entirely happy with MathML anyway.
-** https://github.com/transpect/mml2tex looks rather more robust, and is also used
-to export Word documents and their OOMML to LaTeX via MathML. But we'd still rather
-keep the MathML in place.
+** The gem will optionally invoke the https://github.com/riboseinc/mathml2asciimath
+gem, to convert MathML to AsciiMath. The conversion is not perfect, and will need to be
+post-edited; but it's a lot better than nothing.
 
 The gem does not support:
 
@@ -105,47 +155,8 @@ The gem does not support:
 * `del`, `ins`
 * `footer`, `header`, `main`, `nav`, `details`, `section`, `summary`, `template`
 
-== Usage
-
-=== Ruby
-
-You can convert html content as string or Nokogiri document:
-
-[source,ruby]
-----
-input  = '<strong>feelings</strong>'
-result = ReverseAsciidoctor.convert input
-result.inspect # " *feelings* "
-----
-
-=== Commandline
-
-It's also possible to convert html files to markdown using the binary:
-
-[source,console]
-----
-$ bin/reverse_asciidoctor file.html > file.adoc
-$ cat file.html | bin/reverse_asciidoctor > file.adoc
-----
-
-In addition, the `bin/w2a` script (adapted from the `bin/w2m` script in 
-https://github.com/benbalter/word-to-markdown[Ben Balter's word-to-markdown])
-extracts HTML from Word docx documents, and converts it to Asciidoc.
-
-[source,console]
-----
-$ bundle exec bin/w2a document.docx > document.adoc
-----
-
-The script presumes that LibreOffice has already been installed: it uses LibreOffice's
-export to XHTML. LibreOffice's export of XHTML is superior to the native Microsoft Word export
-to HTML: it exports lists (which Word keeps as paragraphs), and it exports OOMML into MathML.
-On the other hand, the LibreOffice export relies on default styling being used in the
-document, and it may not cope with ordered lists or headings with customised appearance.
-For best results, reset the styles in the document you're converting to those in
-the default Normal.dot template.
 
-=== Configuration
+== Configuration
 
 The following options are available:
 
@@ -157,17 +168,19 @@ The following options are available:
 * `tag_border` (default `' '`) - how to handle tag borders. valid options are:
 ** `' '` - Add whitespace if there is none at tag borders.
 ** `''` - Do not not add whitespace.
+* `mathml2asciimath` - if `true`, will use the https://github.com/riboseinc/mathml2asciimath gem
+to convert MathML to AsciiMath
 
-==== As options
+=== As options
 
 Just pass your chosen configuration options in after the input. The given options will last for this operation only.
 
 [source,ruby]
 ----
-ReverseAsciidoctor.convert(input, unknown_tags: :raise)
+ReverseAsciidoctor.convert(input, unknown_tags: :raise, mathml2asciimath: true)
 ----
 
-==== Preconfigure
+=== Preconfigure
 
 Or configure it block style on a initializer level. These configurations will last for all conversions until they are set to something different.
 
@@ -175,7 +188,7 @@ Or configure it block style on a initializer level. These configurations will la
 ----
 ReverseAsciidoctor.config do |config|
   config.unknown_tags     = :bypass
-  config.github_flavored  = true
+  config.mathml2asciimath  = true
   config.tag_border  = ''
 end
 ----

data/bin/reverse_asciidoctor

@@ -9,6 +9,7 @@ OptionParser.new do |opts|
   opts.banner = "Usage: reverse_asciidoctor [options] <file>"
 
   opts.on('-u', '--unknown_tags [pass_through, drop, bypass, raise]', 'Unknown tag handling (default: pass_through)') { |v| ReverseMarkdown.config.unknown_tags = v }
+  opts.on('-a', '--mathml2asciimath', 'Convert MathML to AsciiMath') { |v| ReverseMarkdown.config.mathml2asciimath = true }
 end.parse!
 
 puts ReverseAsciidoctor.convert(ARGF.read)

data/bin/w2a

@@ -12,20 +12,29 @@ def scrub_whitespace(string)
   string.gsub!(/([ ]+)$/, '')       # line trailing whitespace
   string.gsub!(/\n\n\n\n/, "\n\n")  # Quadruple line breaks
   string.delete!(' ')               # Unicode non-breaking spaces, injected as tabs
+  # following added by me
+  string.gsub!(%r{<h[1-9][^>]*></h1>}, " ") # I don't know why Libre Office is inserting them, but they need to go
+  string.gsub!(%r{<h1[^>]* style="vertical-align: super;[^>]*>([^<]+)</h1>},
+              "<sup>\\1</sup>")         # I absolutely don't know why Libre Office is rendering superscripts as h1
   string
 end
 
-if ARGV.size != 1 || ARGV[0] == '--help'
+if ARGV.size != 1 && ARGV[0] != "--mathml2asciimath" || ARGV[0] == '--help'
   puts 'Usage: bundle exec w2m path/to/document.docx'
   exit 1
 end
 
+if ARGV[0] == "--mathml2asciimath" && ARGV[1]
+  ReverseAsciidoctor.config.mathml2asciimath = true
+  ARGV[0] = ARGV[1]
+end
+
 if ARGV[0] == '--version'
   puts "WordToMarkdown v#{WordToMarkdown::VERSION}"
   puts "LibreOffice v#{WordToMarkdown.soffice.version}" unless Gem.win_platform?
 else
   doc = WordToMarkdown.new ARGV[0]
-  # puts doc.to_s
+  # puts scrub_whitespace(doc.document.html)
   puts ReverseAsciidoctor.convert(scrub_whitespace(doc.document.html), WordToMarkdown::REVERSE_MARKDOWN_OPTIONS)
 end
 

data/lib/reverse_asciidoctor/cleaner.rb

@@ -18,6 +18,11 @@ module ReverseAsciidoctor
     end
 
     def remove_inner_whitespaces(string)
+      unless string.nil?
+        string.gsub!(/\n stem:\[/, "\nstem:[")
+        string.gsub!(/(stem:\[([^\]]|\\\])*\])\n(?=\S)/, "\\1 ")
+        string.gsub!(/(stem:\[([^\]]|\\\])*\])\s+(?=[\^-])/, "\\1")
+      end
       string.each_line.inject("") do |memo, line|
         memo + preserve_border_whitespaces(line) do
           line.strip.gsub(/[ \t]{2,}/, ' ')

data/lib/reverse_asciidoctor/config.rb

@@ -1,9 +1,10 @@
 module ReverseAsciidoctor
   class Config
-    attr_accessor :unknown_tags, :tag_border
+    attr_accessor :unknown_tags, :tag_border, :mathml2asciimath
 
     def initialize
       @unknown_tags     = :pass_through
+      @mathml2asciimath     = false
       @em_delimiter     = '_'.freeze
       @strong_delimiter = '*'.freeze
       @inline_options   = {}
@@ -21,6 +22,10 @@ module ReverseAsciidoctor
       @inline_options[:unknown_tags] || @unknown_tags
     end
 
+    def mathml2asciimath
+      @inline_options[:mathml2asciimath] || @mathml2asciimath
+    end
+
     def tag_border
       @inline_options[:tag_border] || @tag_border
     end

data/lib/reverse_asciidoctor/converters/math.rb

@@ -1,11 +1,17 @@
-# This is cheating: we're injecting MathML into Asciidoctor, but
+# Unless run with ReverseAsciidoctor.config.mathml2asciimath,
+# this is cheating: we're injecting MathML into Asciidoctor, but
 # Asciidoctor only understands AsciiMath or LaTeX
 
+require "mathml2asciimath"
+
 module ReverseAsciidoctor
   module Converters
     class Math < Base
       def convert(node, state = {})
-        "stem:[" << node.to_s.gsub(/\n/, " ") << "]"
+        stem = node.to_s.gsub(/\n/, " ")
+        stem = MathML2AsciiMath.m2a(stem) if ReverseAsciidoctor.config.mathml2asciimath
+        stem = stem.gsub(/\[/, "\\[").gsub(/\]/, "\\]").gsub(/\(\(([^\)]+)\)\)/, "(\\1)") unless stem.nil?
+        " stem:[" << stem << "] "
       end
     end
 

data/lib/reverse_asciidoctor/version.rb

@@ -1,3 +1,3 @@
 module ReverseAsciidoctor
-  VERSION = '0.2.0'
+  VERSION = '0.2.1'
 end

data/reverse_asciidoctor.gemspec

@@ -19,6 +19,7 @@ Gem::Specification.new do |s|
 
   # specify any dependencies here; for example:
   s.add_dependency 'nokogiri'
+  s.add_dependency 'mathml2asciimath'
   s.add_development_dependency 'rspec'
   s.add_development_dependency 'simplecov'
   s.add_development_dependency 'rake'

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: reverse_asciidoctor
 version: !ruby/object:Gem::Version
-  version: 0.2.0
+  version: 0.2.1
 platform: ruby
 authors:
 - Ribose Inc.
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2018-04-17 00:00:00.000000000 Z
+date: 2018-05-03 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: nokogiri
@@ -24,6 +24,20 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
+- !ruby/object:Gem::Dependency
+  name: mathml2asciimath
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: rspec
   requirement: !ruby/object:Gem::Requirement
@@ -229,7 +243,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 2.6.14
+rubygems_version: 2.6.12
 signing_key: 
 specification_version: 4
 summary: Convert html code into asciidoctor.