asciimath 1.0.9 → 2.0.3

data/CHANGELOG.adoc

@@ -1,5 +1,51 @@
 = Asciimath Changelog
 
+== 2.0.3
+
+Enhancements::
+
+  * Issue #59: make escaping of non-ASCII characters in MathML and HTML output optional.
+
+Bug fixes::
+
+  * Issue #58: produce `<mo>` tags instead of `<mi>` tags for non-alphanumeric strings.
+  * Issue #62: resolve infinite recursion in LaTeX generator.
+
+== 2.0.2
+
+Enhancements::
+
+  * Issue #51: generate `accent` and `accentunder` attributes on `munder`, `mover`, and `munderover` tags when appropriate
+  * Issue #52: map `phi` to U+03D5 and `varphi` to U+03C6 by default.
+  * Issue #53: Add support for Roman style font command `rm`.
+
+Bug fixes::
+
+  * Issue #50: fix parsing error when unary and binary operators are missing an operand
+
+== 2.0.1
+
+Enhancements::
+
+  * Issue #40: add `ker` as standard symbol (@GarkGarcia)
+  * Issue #46: allow customisation of the parsing and rendering symbol tables (@GarkGarcia)
+
+Bug fixes::
+
+  * Issue #48: fix parsing of single column matrices (@ronaldtse)
+
+== 2.0.0
+
+Enhancements::
+
+  * The AsciiMath parser has been significantly improved and should now have reached feature parity with the original Javascript implementation.
+  * Issue #16: add initial LaTeX output support (@GarkGarcia)
+  * Issue #26: avoid generating `<mfenced>` tags in MathML output
+
+Bug fixes::
+
+  * Issue #14: add support for additional symbols and functions that are supported by Asciimath JS
+
 == 1.0.9
 
 Bug fixes::
@@ -60,4 +106,4 @@ Enhancements::
 
 Initial release::
 
-  * An AsciiMath parser and MathML converter written in Ruby
+  * An AsciiMath parser and MathML converter written in Ruby

data/Gemfile

@@ -2,3 +2,8 @@ source 'https://rubygems.org'
 
 # Specify your gem's dependencies in asciimath.gemspec
 gemspec
+
+gem "bundler", "> 0"
+gem "rake", "~> 13.0.0"
+gem "rspec", "~> 3.9.0"
+gem "nokogiri"

data/README.adoc

@@ -1,13 +1,14 @@
 # AsciiMath
+:uri-project: https://github.com/asciidoctor/asciimath
 ifndef::env-site[:status:]
 
-An http://asciimath.org[AsciiMath] parser and MathML generator written in pure Ruby.
+An http://asciimath.org[AsciiMath] parser and MathML/LaTeX generator written in pure Ruby.
 
 ifdef::status[]
 [discrete]
 ## Status
 
-image:https://travis-ci.org/pepijnve/asciimath.svg?branch=master["Linux Build Status", link="https://travis-ci.org/asciidoctor/asciimath"]
+image:{uri-project}/workflows/CI/badge.svg?branch=master["Build Status", link={uri-project}/actions?query=branch%3Amaster]
 image:https://img.shields.io/gem/v/asciimath.svg?label=gem%20version[Gem Version, link=https://rubygems.org/gems/asciimath]
 endif::status[]
 
@@ -48,15 +49,16 @@ parsed_expression = AsciiMath.parse(asciimath)
 
 The parsed expression is a set of nested Array and Hash objects.
 
-This expression can then be converted to MathML or HTML (experimental.
+This expression can then be converted to MathML, HTML (experimental) or LaTeX.
 
 [source,ruby]
 ----
 math_ml = parsed_expression.to_mathml
 html = parsed_expression.to_html
+latex = parsed_expression.to_latex
 ----
 
-The MathML or HTML code is returned as a String.
+The MathML, HTML or LaTeX code is returned as a String.
 
 ### Command line
 
@@ -78,8 +80,53 @@ asciimath mathml "an asciimath string"
 asciimath html "an asciimath string"
 ----
 
+.LaTeX Generation
+[source]
+----
+asciimath latex "an asciimath string"
+----
+
 This command will print out the generated code on stdout.
 
+## Extentions and Customization
+
+The parser can be extended by passing a custum tokenization table:
+
+[source, ruby]
+----
+my_tokens_table = AsciiMath::SymbolTableBuilder.new
+AsciiMath::Parser.add_default_parser_symbols(my_tokens_table)
+my_tokens_table.add('mysymbol', :mysymbol, :symbol)
+
+AsciiMath::parse("a + mysymbol + b", my_tokens_table.build)
+----
+
+Furthermore, the behaviour of the tokenizer be customized by altering the value 
+associated with a token in `AsciiMath::Tokenizer::DEFAULT_PARSE_SYMBOL_TABLE`:
+
+[source, ruby]
+----
+my_tokens_table = AsciiMath::SymbolTableBuilder.new
+AsciiMath::Parser.add_default_parser_symbols(my_tokens_table)
+my_tokens_table.add('alpha', :beta, :symbol)
+
+# Now "alpha + beta" is equivalent to "beta + beta"
+AsciiMath::parse("alpha + beta", my_tokens_table.build)
+----
+
+The same behaviour applies to each individual render (`MathMLBuilder`, 
+`HTMLBuilder` and `LatexBuilder`). By adding entries to a rendere's rendering 
+table (or modifying exisisting entries), users can customize it's output:
+
+[source, ruby]
+----
+my_rendering_table = AsciiMath::SymbolTableBuilder.new
+AsciiMath::MarkupBuilder.add_default_display_symbols(my_rendering_table)
+my_rendering_table.add('alpha', '\u03b2', :identifier)
+
+# Now "alpha + beta" is equivalent to "beta + beta"
+AsciiMath::parse("alpha + beta").to_mathml(my_rendering_table.build)
+----
 
 ## Notes on the HTML Output
 
@@ -95,6 +142,23 @@ Known issues are as follows:
 Rendering the HTML output correctly requires the inclusion of `style/math.css` in the html document.
 There is currently no specific required font for this output, it simply selects a `serif` font family - change the `@font-family` attribute in the `.math-inline` class to select something specific.
 
+## Notes on the LaTeX Output
+
+All LaTeX commands and environments used in the output are coverved by 
+https://ctan.org/pkg/amsmath[`amsmath`] and `amssymb`, with a few exceptions:
+
+ * `\color`
+ * `\cancel`
+ * `\mathscr`
+ * `\twoheadrightarrowtail`
+
+The `\color` command is supported by the 
+https://www.ctan.org/pkg/xcolor[`xcolor`] package, which is included in most 
+LaTeX distributions. The `\cancel` command is supported by the 
+https://www.ctan.org/pkg/cancel[cancel] package, also included in most LaTeX 
+distributions. The other commands are supported by the 
+https://ctan.org/pkg/stix[`stix`] package.
+
 ## Contributing
 
 . Fork it (https://github.com/pepijnve/asciimath/fork)

data/asciimath.gemspec

@@ -1,13 +1,14 @@
 # coding: utf-8
 lib = File.expand_path('../lib', __FILE__)
 $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+
 require 'asciimath/version'
 
 Gem::Specification.new do |spec|
   spec.name          = "asciimath"
   spec.version       = AsciiMath::VERSION
-  spec.authors       = ["Pepijn Van Eeckhoudt"]
-  spec.email         = ["pepijn@vaneeckhoudt.net"]
+  spec.authors       = ["Pepijn Van Eeckhoudt", "Gark Garcia"]
+  spec.email         = ["pepijn@vaneeckhoudt.net", "pablo-ecobar@riseup.net"]
   spec.summary       = %q{AsciiMath parser and converter}
   spec.description   = %q{A pure Ruby AsciiMath parsing and conversion library.}
   spec.homepage      = ""
@@ -17,8 +18,4 @@ Gem::Specification.new do |spec|
   spec.executables   = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
   spec.test_files    = spec.files.grep(%r{^(test|spec|features)/})
   spec.require_paths = ["lib"]
-
-  spec.add_development_dependency "bundler", "> 0"
-  spec.add_development_dependency "rake", "~> 10.0"
-  spec.add_development_dependency "rspec", "~> 3.1.0"
 end

data/dump_symbol_table.rb

@@ -0,0 +1,46 @@
+require_relative 'lib/asciimath'
+
+def escape_adoc(adoc)
+  case adoc
+    when nil
+      ''
+    when '+'
+      adoc
+    else
+      "++#{adoc.gsub('|', '\\|')}++"
+  end
+end
+
+puts "|==="
+puts '|AsciiMath |Symbol |MathML Value |LaTeX Value'
+puts
+
+AsciiMath::Parser::DEFAULT_PARSER_SYMBOL_TABLE.each_pair do |asciimath, value|
+  sym = value[:value]
+  unless sym.is_a?(Symbol)
+    next
+  end
+
+  mathml = AsciiMath::MathMLBuilder.default_display_symbol_table[sym]
+
+  if mathml
+    val = mathml[:value]
+  else
+    val = "Missing!!!!!"
+  end
+
+  latex = AsciiMath::LatexBuilder::SYMBOLS[sym] || "\\#{sym.to_s}"
+
+  codepoint = ""
+  if val.is_a?(String)
+    codepoint = val.codepoints.map do |cp|
+      cpstr = sprintf('U+%04X', cp)
+      "https://codepoints.net/#{cpstr}[#{cpstr}]"
+    end.join(' ')
+  end
+
+  puts "|#{escape_adoc(asciimath)} |:#{sym.to_s} |#{escape_adoc(val.to_s)} (#{codepoint}) |#{escape_adoc(latex)}"
+end
+
+puts "|==="
+puts

data/lib/asciimath.rb

@@ -1,4 +1,5 @@
-require File.join(File.dirname(__FILE__), 'asciimath/version')
-require File.join(File.dirname(__FILE__), 'asciimath/parser')
-require File.join(File.dirname(__FILE__), 'asciimath/mathml')
-require File.join(File.dirname(__FILE__), 'asciimath/html')
+require_relative 'asciimath/version'
+require_relative 'asciimath/parser'
+require_relative 'asciimath/mathml'
+require_relative 'asciimath/html'
+require_relative 'asciimath/latex'

data/lib/asciimath/ast.rb

@@ -0,0 +1,456 @@
+module AsciiMath
+  module AST
+    def expression(*e)
+      case e.length
+        when 0
+          nil
+        when 1
+          e[0]
+        else
+          Sequence.new(e)
+      end
+    end
+
+    def paren(lparen, e, rparen)
+      Paren.new(lparen, e, rparen)
+    end
+
+    def group(lparen, e, rparen)
+      Group.new(lparen, e, rparen)
+    end
+
+    def subsup(e, sub, sup)
+      SubSup.new(e, sub, sup)
+    end
+
+    def sub(e, sub)
+      SubSup.new(e, sub, nil)
+    end
+
+    def sup(e, sup)
+      SubSup.new(e, nil, sup)
+    end
+
+    def unary(operator, e)
+      UnaryOp.new(operator, e)
+    end
+
+    def binary(operator, e1, e2)
+      BinaryOp.new(operator, e1, e2)
+    end
+
+    def infix(e1, operator, e2)
+      InfixOp.new(operator, e1, e2)
+    end
+
+    def text(value)
+      Text.new(value)
+    end
+
+    def number(value)
+      Number.new(value)
+    end
+
+    def symbol(symbol, text)
+      Symbol.new(symbol, text)
+    end
+
+    def identifier(value)
+      Identifier.new(value)
+    end
+
+    def matrix(lparen, rows, rparen)
+      Matrix.new(lparen, rows, rparen)
+    end
+
+    def color(r, g, b, text)
+      Color.new(r, g, b, text)
+    end
+
+    class Node
+      attr_reader :parent
+
+      def initialize
+        @parent = nil
+      end
+
+      protected
+
+      attr_writer :parent
+    end
+
+    class InnerNode < Node
+      include Enumerable
+
+      def initialize
+        super
+        @children = []
+      end
+
+      def [](*args)
+        @children[*args]
+      end
+
+      def length
+        @children.length
+      end
+
+      def each(&block)
+        @children.each(&block)
+      end
+
+      protected
+
+      def child_nodes
+        @children
+      end
+      
+      def add(node)
+        node.parent.remove(node) if node.parent
+        node.parent = self
+        child_nodes << node
+      end
+
+      def remove(node)
+        node.parent = nil
+        child_nodes.delete(node)
+      end
+    end
+
+    class Sequence < InnerNode
+      def initialize(nodes)
+        super()
+        nodes.each { |node| add(node) }
+      end
+
+      def to_s
+        child_nodes.map { |node| node.to_s }.join(" ")
+      end
+
+      def ==(o)
+        o.class == self.class && o.child_nodes == child_nodes
+      end
+    end
+
+    class Paren < InnerNode
+      attr_reader :lparen
+      attr_reader :rparen
+
+      def initialize(lparen, e, rparen)
+        super()
+        @lparen = lparen
+        @rparen = rparen
+        add(e) if e
+      end
+
+      def expression
+        child_nodes[0]
+      end
+
+      def to_s
+        "#{lparen.nil? ? '' : lparen.text}#{expression}#{rparen.nil? ? '' : rparen.text}"
+      end
+
+      def ==(o)
+        o.class == self.class && o.lparen == lparen && o.expression == expression && o.rparen == rparen
+      end
+    end
+
+    class Group < InnerNode
+      attr_reader :lparen
+      attr_reader :rparen
+
+      def initialize(lparen, e, rparen)
+        super()
+        @lparen = lparen
+        @rparen = rparen
+        add(e) if e
+      end
+
+      def expression
+        child_nodes[0]
+      end
+
+      def to_s
+        "#{lparen.nil? ? '' : lparen.text}#{expression}#{rparen.nil? ? '' : rparen.text}"
+      end
+
+      def ==(o)
+        o.class == self.class && o.lparen == lparen && o.expression == expression && o.rparen == rparen
+      end
+    end
+
+    class SubSup < InnerNode
+      def initialize(e, sub, sup)
+        super()
+        add(e)
+        add(sub || Empty.new)
+        add(sup || Empty.new)
+      end
+
+      def base_expression
+        child_nodes[0]
+      end
+
+      def sub_expression
+        child = child_nodes[1]
+        child.is_a?(Empty) ? nil : child
+      end
+
+      def sup_expression
+        child = child_nodes[2]
+        child.is_a?(Empty) ? nil : child
+      end
+
+      def to_s
+        s = ""
+        s << base_expression.to_s
+        sub = sub_expression
+        if sub
+          s << "_" << sub.to_s
+        end
+        sup = sup_expression
+        if sup
+          s << "^" << sup.to_s
+        end
+        s
+      end
+
+      def ==(o)
+        o.class == self.class && o.base_expression == base_expression && o.sub_expression == sub_expression && o.sup_expression == sup_expression
+      end
+    end
+
+    class UnaryOp < InnerNode
+      def initialize(operator, e)
+        super()
+        add(operator)
+        add(e)
+      end
+
+      def operator
+        child_nodes[0]
+      end
+
+      def operand
+        child_nodes[1]
+      end
+
+      def to_s
+        "#{operator} #{operand}"
+      end
+
+      def ==(o)
+        o.class == self.class && o.operator == operator && o.operand == operand
+      end
+    end
+
+    class BinaryOp < InnerNode
+      def initialize(operator, e1, e2)
+        super()
+        add(operator)
+        add(e1)
+        add(e2)
+      end
+
+
+      def operator
+        child_nodes[0]
+      end
+
+      def operand1
+        child_nodes[1]
+      end
+
+      def operand2
+        child_nodes[2]
+      end
+
+      def to_s
+        "#{operator} #{operand1} #{operand2}"
+      end
+
+      def ==(o)
+        o.class == self.class && o.operator == operator && o.operand1 == operand1 && o.operand2 == operand2
+      end
+    end
+
+    class InfixOp < InnerNode
+      def initialize(operator, e1, e2)
+        super()
+        add(operator)
+        add(e1)
+        add(e2)
+      end
+
+
+      def operator
+        child_nodes[0]
+      end
+
+      def operand1
+        child_nodes[1]
+      end
+
+      def operand2
+        child_nodes[2]
+      end
+
+      def to_s
+        "#{operand1} #{operator} #{operand2}"
+      end
+
+      def ==(o)
+        o.class == self.class && o.operator == operator && o.operand1 == operand1 && o.operand2 == operand2
+      end
+    end
+
+    class ValueNode < Node
+      attr_reader :value
+
+      def initialize(value)
+        super()
+        @value = value
+      end
+
+      def to_s
+        value.to_s
+      end
+
+      def ==(o)
+        o.class == self.class && o.value == value
+      end
+    end
+
+    class Text < ValueNode
+      def initialize(value)
+        super(value.dup.freeze)
+      end
+
+      def to_s
+        '"' + super + '"'
+      end
+    end
+
+    class Number < ValueNode
+      def initialize(value)
+        super(value.dup.freeze)
+      end
+    end
+
+    class Symbol < ValueNode
+      attr_reader :text
+
+      def initialize(value, text)
+        super(value)
+        @text = text.dup.freeze
+      end
+
+      def ==(o)
+        super && o.text == text
+      end
+
+      def to_s
+        text
+      end
+    end
+
+    class Identifier < ValueNode
+      def initialize(value)
+        super(value.dup.freeze)
+      end
+    end
+
+    class Color < ValueNode
+      attr_reader :text
+
+      def initialize(r, g, b, text)
+        super({:r => r, :g => g, :b => b}.freeze)
+        @text = text.dup.freeze
+      end
+
+      def red
+        value[:r]
+      end
+
+      def green
+        value[:g]
+      end
+
+      def blue
+        value[:b]
+      end
+
+      def ==(o)
+        o.class == self.class &&
+            o.red == red &&
+            o.green == green &&
+            o.blue == blue &&
+            o.text == text
+      end
+
+      def to_hex_rgb
+        sprintf('#%02x%02x%02x', red, green, blue)
+      end
+
+      def to_s
+        text
+      end
+    end
+
+    class Matrix < InnerNode
+      attr_reader :lparen
+      attr_reader :rparen
+
+      def initialize(lparen, rows, rparen)
+        super()
+        @lparen = lparen
+        @rparen = rparen
+        rows.map { |row| MatrixRow.new(row) }.each { |row_seq| add(row_seq) }
+      end
+
+      def to_s
+        s = ""
+        s << (lparen.nil? ? '{:' : lparen.text)
+        s << child_nodes.map { |node| node.to_s }.join(",")
+        s << (rparen.nil? ? ':}' : rparen.text)
+      end
+
+      def ==(o)
+        o.class == self.class &&
+            o.lparen == lparen &&
+            o.child_nodes == child_nodes &&
+            o.rparen == rparen
+      end
+    end
+
+    class MatrixRow < InnerNode
+      def initialize(nodes)
+        super()
+        nodes.each { |node| add(node || Empty.new) }
+      end
+
+      def to_s
+        "(" + child_nodes.map { |node| node.to_s }.join(",") + ")"
+      end
+
+      def ==(o)
+        o.class == self.class && o.child_nodes == child_nodes
+      end
+    end
+
+    class Empty < Node
+      def initialize()
+        super
+      end
+
+      def to_s
+        ''
+      end
+
+      def ==(o)
+        o.class == self.class
+      end
+    end
+  end
+end