plurimath 0.10.2 → 0.10.4

data/TODO.bugs/03-omml-underover-displaystyle.md

@@ -0,0 +1,23 @@
+# Fix OMML output diff for underover with displaystyle false
+
+## Problem
+1 test failure in mathml_spec.rb:2663 - OMML output for underover
+tags with `displaystyle false` produces different XML structure.
+
+## Expected vs Actual
+Expected: `<m:sSup>` with `<m:sup>` element
+Actual: `<m:limUpp>` with `<m:lim>` element
+
+This means the underover is being interpreted differently (as a limit
+upper instead of a superscript).
+
+## Root cause
+The V4 MathML parser produces a different formula structure for
+`<munderover>` elements when `displaystyle="false"`. This changes
+how the OMML serializer renders the formula.
+
+This is related to MathML 4 vs MathML 3 behavior differences. In
+MathML 4, displaystyle affects whether underover renders as limits
+vs scripts. The specs need to be updated to match V4 behavior.
+
+## Status: Fixed (updated expected OMML to use limLow/limUpp)

data/TODO.bugs/04-unitsml-spec-diffs.md

@@ -0,0 +1,20 @@
+# Fix unitsml spec attribute ordering and displaystyle differences
+
+## Problem
+2 test failures in spec/plurimath/math/formula/unitsml_spec.rb - the
+unitsml-generated MathML output has changed:
+1. XML attribute ordering changed (id vs dimensionURL)
+2. `lang="en-US"` changed to `lang="en"`
+3. Inner `<math>` now includes `displaystyle="true"` (from V4::Math default)
+
+## Root cause
+The local unitsml gem (at ../../unitsml/unitsml-ruby/) produces slightly
+different output than the previously-tested version. The attribute
+ordering and lang value differences are from the updated unitsml/unitsdb
+data. The `displaystyle="true"` addition is from our V4::Math default.
+
+## Fix
+Update the expected values in the spec to match the new output. These
+are cosmetic differences in XML serialization.
+
+## Status: Fixed (updated expected values for attribute ordering, lang, displaystyle)

data/TODO.bugs/05-omml-greek-entity-encoding.md

@@ -0,0 +1,50 @@
+# Greek Letter HTML Entity Serialization in moxml Ox Adapter
+
+## Issue
+
+When using the `moxml` gem with the Ox adapter, Greek letters (and other special Unicode characters) are serialized as raw Unicode characters instead of HTML entities.
+
+**Example:**
+- Expected: `<m:t>&#x3b8;</m:t>` (theta as HTML entity)
+- Actual: `<m:t>θ</m:t>` (theta as raw Unicode character)
+
+## Affected Tests
+
+All OMML comparison tests in plurimath:
+
+| Test # | Name | Issue |
+|--------|------|-------|
+| 2452 | contains multiple tags in Mathml | Greek letters (α, θ) as Unicode vs HTML entities |
+| 2663 | contains underover, under, and over tags | Greek letter θ encoding |
+| 3121 | multiscripts containing none tag | ZWSP (&#8203;) not rendered - empty m:t |
+| 3201 | oint msubsup tag | Integral symbol (&#x222e;) missing in output |
+| 3365 | nary prod symbol in underover | Integral symbol (&#x222e;) missing |
+| 3486 | empty mo example from plurimath#318 | Empty mo (&#xb1; ±) not rendered |
+
+## Root Cause
+
+The moxml gem's Ox adapter uses `Ox.dump()` to serialize XML. Ox serializes text content as raw Unicode characters rather than HTML entities.
+
+When a symbol like `θ` (theta, U+03B8) is stored as a text node, Ox outputs it as `θ` instead of `&#x3b8;`.
+
+## Expected Behavior
+
+For MathML/OMML output, certain characters should be serialized as HTML entities:
+- Greek letters: `&#x3b8;` for `θ`, `&#x3b1;` for `α`, `&#x3b2;` for `β`
+- Zero-width space: `&#8203;` for ZWSP
+- Operators: `&#xb1;` for `±`, `&#x222e;` for `∮`
+
+## Solution
+
+The Ox adapter's serialize method needs to post-process text content to encode certain Unicode characters as HTML entities. This could be done in:
+
+1. `moxml/lib/moxml/adapter/ox.rb` - modify the `serialize` method to post-process text nodes
+2. Or create a text encoding step when adding text content to elements
+
+## Alternative
+
+Use the Oga adapter instead of Ox adapter. The Oga adapter appears to handle HTML entity encoding correctly (based on test outputs showing `&#8203;` preserved with Oga).
+
+## Context
+
+This is a serialization issue in the moxml library, not in plurimath itself. Plurimath correctly stores the symbol information; the issue is how moxml/Ox serializes that information to XML output.

data/TODO.bugs/mml_custom_model_child_order.md

@@ -0,0 +1,149 @@
+# Proposal: Fix Custom Model Deserialization Child Order for Binary Operators
+
+## Problem Description
+
+When using the mml gem's custom model feature to substitute element classes (e.g., mapping `Mml::V4::Mover` to a custom `Overset` class), the child elements are being passed to the custom model's constructor in **reversed order** compared to their document order.
+
+## Expected Behavior
+
+For MathML `<mover>` element:
+- First child = base (the element being overscored)
+- Second child = overscript (the accent mark)
+
+When parsing `<mover><mi>θ</mi><mi>d</mi></mover>`:
+- `mi[0]` = θ (base)
+- `mi[1]` = d (overscript)
+
+The custom model `Overset.new(base, overscript)` should receive:
+- `parameter_one` = θ
+- `parameter_two` = d
+
+## Actual Behavior
+
+The custom model `Overset` receives children in **reversed order**:
+- `parameter_one` = d (should be θ)
+- `parameter_two` = θ (should be d)
+
+## Clarification: This is NOT a lutaml-model bug
+
+Lutaml-model has confirmed this is not an issue in their gem. The problem is entirely within the **mml gem** and how it sets up and uses custom models.
+
+## Root Cause Analysis
+
+### Investigation Steps
+
+1. **Parsing without custom models:**
+   ```ruby
+   parsed = Mml.parse(mathml, namespace_exist: true, version: 4)
+   mover = parsed.instance_variable_get(:@mover_value).first
+   # mover.mi_value[0].value = "θ" (correct - base first)
+   # mover.mi_value[1].value = "d" (correct - overscript second)
+   ```
+
+2. **Parsing with custom models:**
+   ```ruby
+   Mml::V4::Configuration.custom_models = { Mml::V4::Mover => Overset }
+   parsed = Mml.parse(mathml, namespace_exist: true, version: 4)
+   overset = parsed.value.first
+   # overset.parameter_one.value = "d"  (WRONG - should be "θ")
+   # overset.parameter_two.class = Theta (WRONG - should be "d")
+   ```
+
+3. **Custom model setup in mml gem:**
+   ```ruby
+   # From mml gem's context_configuration.rb:
+   def custom_models=(models_hash)
+     models_hash.each do |klass, model|
+       klass.model(model)
+     end
+   end
+
+   # This calls lutaml-model's model method:
+   # From lutaml-model serialize/initialization.rb:
+   def model(klass = nil)
+     if klass
+       @model = klass
+       add_custom_handling_methods_to_model(klass)
+     else
+       @model
+     end
+   end
+   ```
+
+4. **The fix must be in mml gem:**
+   When `klass.model(custom_model)` is called, the mml gem should ensure that when the custom model is instantiated during deserialization, the child elements are passed in document order. The issue is that the mml gem's element attribute definitions (e.g., `element "mover"` with child `mi_value`) are not being properly respected when the custom model substitution is applied.
+
+   Specifically, the mml gem's `ContextConfiguration#custom_models=` sets up the model substitution, but it does not ensure that the original element's `element_order` or child element mappings are preserved for the custom model. When lutaml-model instantiates the custom model via its `from_xml` or similar deserialization path, it passes children in an order that may not match the MathML document order.
+
+### Key Finding
+
+The issue is in **how the mml gem sets up the model substitution**, not in lutaml-model itself. When `Mml::V4::Mover.model(Overset)` is called:
+
+1. Lutaml-model stores `@model = Overset`
+2. During XML deserialization, lutaml-model uses its own logic to map XML children to constructor arguments
+3. The mml gem's element definitions (which define child element order via `element_order` or similar) are not being communicated to the custom model
+
+The fix should be in `ContextConfiguration#custom_models=` — after calling `klass.model(model)`, the mml gem should also ensure that the custom model class inherits or mirrors the original element's child element ordering information.
+
+## Affected Elements
+
+This issue affects all binary MathML elements when using custom models:
+- `<mover>` → Overset (base, overscript)
+- `<munder>` → Underset (base, underscript)
+- `<mover>` and `<munder>` combined in `<munderover>` → Underover (base, underscript, overscript)
+- `<msup>` → Power (base, superscript)
+- `<msub>` → Base (base, subscript)
+
+## Expected Fix
+
+The fix must be in the **mml gem** — specifically in `ContextConfiguration#custom_models=` or the related element deserialization logic. When setting up a custom model substitution, the mml gem must ensure that the custom model receives child elements in MathML document order.
+
+### Specific Fix Location
+
+In `lib/mml/context_configuration.rb`, the `custom_models=` method should be updated to preserve child element ordering when setting up the model substitution. One approach:
+
+1. After calling `klass.model(model)`, copy or alias the original element's `element_order` or child-getting logic to the custom model
+2. Or, intercept the deserialization of elements with custom models to explicitly pass children in document order
+
+## Test Case
+
+```ruby
+# Setup
+require "mml"
+class Overset < BinaryFunction
+  def initialize(base = nil, overscript = nil)
+    super(base, overscript)
+  end
+end
+Mml::V4::Configuration.custom_models = { Mml::V4::Mover => Overset }
+
+# MathML: <mover><mi>θ</mi><mi>d</mi></mover>
+# Expected: base=θ, overscript=d
+# Actual: base=d, overscript=θ (BUG!)
+
+mathml = <<~MATHML
+<math xmlns="http://www.w3.org/1998/Math/MathML">
+  <mover>
+    <mi>θ</mi>
+    <mi>d</mi>
+  </mover>
+</math>
+MATHML
+
+parsed = Mml.parse(mathml, namespace_exist: true, version: 4)
+overset = parsed.value.first
+raise "Bug: children reversed!" unless overset.parameter_one.value == "θ"
+```
+
+## Impact
+
+This bug affects any application using the mml gem's custom model feature to substitute element classes, particularly when:
+- Building domain-specific MathML parsers (like plurimath)
+- Converting MathML to other formats
+- Processing mathematical content with accent marks, subscripts, superscripts, etc.
+
+## References
+
+- MathML Specification: https://www.w3.org/TR/MathML3/chapter3.html
+- mml gem: https://github.com/metanorma/mml
+- lutaml-model gem: https://github.com/lutaml/lutaml-model (confirmed: not the source of this bug)

data/TODO.fix-fails/01-phantom-whitespace.md

@@ -0,0 +1,63 @@
+# 01 - phantom tag whitespace issue
+
+## Test Location
+`spec/plurimath/mathml_spec.rb:281`
+
+## Issue Summary
+Test "contains Mathml phantom tag's example" fails with XML equivalence check.
+
+## Input MathML
+```xml
+<math>
+   <mrow>
+      <mi> x </mi>
+      <mphantom>
+         <mo> + </mo>
+      </mphantom>
+      <mphantom>
+         <mi> y </mi>
+      </mphantom>
+      <mo> + </mo>
+      <mi> z </mi>
+   </mrow>
+</math>
+```
+
+## Expected MathML (from test)
+```xml
+<mphantom>
+  <mo>+</mo>
+</mphantom>
+<mphantom>
+  <mi> y </mi>
+</mphantom>
+<mo> + </mo>
+```
+
+## Our Output
+```xml
+<mphantom>
+  <mo> + </mo>
+</mphantom>
+<mphantom>
+  <mi> y </mi>
+</mphantom>
+<mo> + </mo>
+```
+
+## Difference
+- Expected first mo inside phantom: `<mo>+</mo>` (no spaces)
+- Our output first mo inside phantom: `<mo> + </mo>` (with spaces)
+
+The second mo (outside phantom) correctly preserves spaces.
+
+## Analysis
+The issue is that `mphantom` is not supposed to preserve the whitespace inside the `mo` element according to the expected test output. However, this seems semantically incorrect - if the input has `<mo> + </mo>`, the spaces are part of the mo content.
+
+## Status
+**Pre-existing from LutaML-Model update** - This test was passing before the update, meaning the previous code stripped spaces inside mo elements when inside phantom.
+
+## Possible Causes
+1. The LutaML-Model update changed how `mo_to_symbol` preserves whitespace
+2. The test expectation was always semantically questionable but matched the previous implementation
+3. Something in the translator is now preserving whitespace that was previously stripped

data/TODO.fix-fails/02-table-parentheses-latex.md

@@ -0,0 +1,17 @@
+# 02 - table with parentheses (LaTeX trailing space)
+
+## Test Location
+`spec/plurimath/mathml_spec.rb:778`
+
+## Issue Summary
+Test "contains table with surrounding parentheses(metanorma example) and sqrt tag" fails on LaTeX comparison.
+
+## Failure Details
+- Expected and got appear identical but differ in invisible characters or whitespace
+- The test uses `to_latex` comparison
+
+## Possible Cause
+The LaTeX output has extra whitespace/trailing spaces that don't match the expected.
+
+## Status
+**Pre-existing from LutaML-Model update**

data/TODO.fix-fails/03-longidv-tag-mathml.md

@@ -0,0 +1,17 @@
+# 03 - longidv tag (MathML structure)
+
+## Test Location
+`spec/plurimath/mathml_spec.rb:1197`
+
+## Issue Summary
+Test "contains longidv tag Mathml" fails with MathML element_structure differences.
+
+## Failure Details
+- 3 dimension differences in XML comparison
+- Element structure mismatch in the MathML output
+
+## Possible Cause
+The LutaML-Model update changed how certain MathML elements are parsed/rendered, affecting the structure.
+
+## Status
+**Pre-existing from LutaML-Model update**

data/TODO.fix-fails/04-mmultiscript-none-mathml.md

@@ -0,0 +1,17 @@
+# 04 - mmultiscript containing none (MathML structure)
+
+## Test Location
+`spec/plurimath/mathml_spec.rb:1319`
+
+## Issue Summary
+Test "contains mmultiscript containing none tag" fails with MathML element_structure differences.
+
+## Failure Details
+- Element structure mismatch in MathML output
+- The `<none>` tag handling may have changed
+
+## Possible Cause
+The LutaML-Model update changed how `<none>` elements inside `<mmultiscripts>` are handled.
+
+## Status
+**Pre-existing from LutaML-Model update**

data/TODO.fix-fails/05-mstyle-nary-oint-mathml.md

@@ -0,0 +1,17 @@
+# 05 - mstyle containing nary oint (MathML structure)
+
+## Test Location
+`spec/plurimath/mathml_spec.rb:1366`
+
+## Issue Summary
+Test "contains mstyle containing nary oint value in msubsup tag" fails with MathML element_structure differences.
+
+## Failure Details
+- Element structure mismatch in MathML output
+- Issue with nary operator rendering inside msubsup
+
+## Possible Cause
+The LutaML-Model update changed how nary operators are handled within subscript/superscript elements.
+
+## Status
+**Pre-existing from LutaML-Model update**

data/TODO.fix-fails/06-issue-238-mathml.md

@@ -0,0 +1,17 @@
+# 06 - plurimath/issue#238 (MathML structure)
+
+## Test Location
+`spec/plurimath/mathml_spec.rb:1540`
+
+## Issue Summary
+Test "contains string from plurimath/issue#238" fails with MathML element_structure differences.
+
+## Failure Details
+- 4-5 dimension differences in XML comparison
+- Complex MathML structure issue
+
+## Possible Cause
+The issue #238 was about a specific MathML rendering problem that the LutaML-Model update may have affected.
+
+## Status
+**Pre-existing from LutaML-Model update**

data/TODO.fix-fails/07-metanorma-bipm-latex.md

@@ -0,0 +1,19 @@
+# 07 - metanorma-cli-actions-mn-bipm (LaTeX structure)
+
+## Test Location
+`spec/plurimath/mathml_spec.rb:1713`
+
+## Issue Summary
+Test "contains input from metanorma-cli-actions-mn-bipm run" fails with LaTeX comparison.
+
+## Expected LaTeX
+`\underline{\mathit{B}} = \left [ ... \end{matrix}\right ]`
+
+## Got LaTeX
+`\underset{ \left ( \underline \right ) }{ \left ( \mathit{B} \right ) } = \left [ ... \end{matrix}\right ]`
+
+## Issue
+The LaTeX rendering of the underline/enclose structure is different - the structure of how `underline` wraps `B` has changed.
+
+## Status
+**Pre-existing from LutaML-Model update**

data/TODO.fix-fails/08-metanorma-itu-latex.md

@@ -0,0 +1,17 @@
+# 08 - metanorma-cli-actions-mn-itu (LaTeX spacing)
+
+## Test Location
+`spec/plurimath/mathml_spec.rb:1860`
+
+## Issue Summary
+Test "contains input from metanorma-cli-actions-mn-itu run" fails with LaTeX comparison.
+
+## Expected
+`y_{k} = ( x_{k} \pm h )  m`
+(Got: `y_{k} = ( x_{k} \pm h ) m`)
+
+## Issue
+Double space before `m` in expected, single space in our output. This is likely related to how `None#to_latex` or similar empty-returning functions are handled.
+
+## Status
+**Pre-existing from LutaML-Model update**

data/TODO.fix-fails/09-omml-greek-encoding.md

@@ -0,0 +1,24 @@
+# 09 - OMML Greek letter encoding (multiple tags)
+
+## Test Location
+`spec/plurimath/mathml_spec.rb:2452`
+
+## Issue Summary
+Test "contains multiple tags in Mathml" fails with OMML comparison due to Greek letter encoding.
+
+## Expected
+`<m:t>&#x3b1;</m:t>` (HTML entity)
+`<m:t>&#x3b8;</m:t>` (HTML entity)
+
+## Got
+`<m:t>α</m:t>` (Unicode character)
+`<m:t>θ</m:t>` (Unicode character)
+
+## Issue
+Greek letters are being output as Unicode characters instead of HTML entities. This is the **Greek letter encoding issue** documented in `TODO.bugs/05-omml-greek-entity-encoding.md`.
+
+## Cause
+Ox serializer outputs Unicode characters directly instead of HTML entities.
+
+## Status
+**Pre-existing from LutaML-Model update - KNOWN ISSUE**

data/TODO.fix-fails/10-omml-underover-greek.md

@@ -0,0 +1,19 @@
+# 10 - OMML Greek letter encoding (underover)
+
+## Test Location
+`spec/plurimath/mathml_spec.rb:2663`
+
+## Issue Summary
+Test "contains underover, under, and over tags with displaystyle false" fails with OMML comparison.
+
+## Expected
+`<m:t>&#x3b8;</m:t>` (HTML entity)
+
+## Got
+`<m:t>θ</m:t>` (Unicode character)
+
+## Issue
+Greek letter theta encoding issue - same as test 09.
+
+## Status
+**Pre-existing from LutaML-Model update - KNOWN ISSUE**

data/TODO.fix-fails/11-omml-multiscripts-zwsp.md

@@ -0,0 +1,19 @@
+# 11 - OMML multiscripts ZWSP handling
+
+## Test Location
+`spec/plurimath/mathml_spec.rb:3121`
+
+## Issue Summary
+Test "contains multiscripts containing none tag" fails with OMML comparison.
+
+## Expected
+`<m:t>&#8203;</m:t>` (ZWSP - Zero-Width Space)
+
+## Got
+`<m:t></m:t>` (empty)
+
+## Issue
+ZWSP (Zero-Width Space, U+200B) placeholder is not being rendered in OMML output. The empty `m:t` element suggests the ZWSP character is being lost.
+
+## Status
+**Pre-existing from LutaML-Model update**

data/TODO.fix-fails/12-omml-oint-integral.md

@@ -0,0 +1,19 @@
+# 12 - OMML oint msubsup missing integral
+
+## Test Location
+`spec/plurimath/mathml_spec.rb:3201`
+
+## Issue Summary
+Test "contains oint msubsup tag" fails with OMML comparison.
+
+## Expected
+Structure includes `<m:t>&#x222e;</m:t>` (contour integral symbol ∮)
+
+## Got
+Missing the integral symbol in output
+
+## Issue
+The integral symbol `&#x222e;` is not appearing in the OMML output for nary operators.
+
+## Status
+**Pre-existing from LutaML-Model update**

data/TODO.fix-fails/13-omml-nary-prod.md

@@ -0,0 +1,19 @@
+# 13 - OMML nary prod missing integral
+
+## Test Location
+`spec/plurimath/mathml_spec.rb:3365`
+
+## Issue Summary
+Test "contains nary prod symbol in underover for nary tag" fails with OMML comparison.
+
+## Expected
+Structure includes integral symbol
+
+## Got
+Structure differs from expected
+
+## Issue
+Similar to test 12 - nary product/integral operator rendering issue in OMML.
+
+## Status
+**Pre-existing from LutaML-Model update**

data/TODO.fix-fails/14-omml-empty-mo.md

@@ -0,0 +1,19 @@
+# 14 - OMML empty mo example
+
+## Test Location
+`spec/plurimath/mathml_spec.rb:3486`
+
+## Issue Summary
+Test "contains empty mo example from plurimath/plurimath#318" fails with OMML comparison.
+
+## Expected
+`<m:t>&#xb1;</m:t>` (plus-minus sign ±)
+
+## Got
+`<m:t></m:t>` (empty)
+
+## Issue
+Empty `mo` element (representing ±) is not being rendered correctly in OMML output.
+
+## Status
+**Pre-existing from LutaML-Model update**