litdown 0.3.0__tar.gz

litdown-0.3.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Centre for Population Genomics
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

litdown-0.3.0/PKG-INFO

@@ -0,0 +1,186 @@
+Metadata-Version: 2.4
+Name: litdown
+Version: 0.3.0
+Summary: Convert scholarly full-text XML (JATS, Elsevier) to Markdown.
+Author: Toby Sargeant
+License: MIT
+Project-URL: Repository, https://github.com/populationgenomics/litdown
+Project-URL: Issues, https://github.com/populationgenomics/litdown/issues
+Keywords: jats,elsevier,pmc,markdown,mathml,latex
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Text Processing :: Markup :: XML
+Classifier: Topic :: Text Processing :: Markup :: Markdown
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: defusedxml>=0.7
+Dynamic: license-file
+
+# litdown
+
+[![Lint](https://github.com/populationgenomics/litdown/actions/workflows/lint.yaml/badge.svg)](https://github.com/populationgenomics/litdown/actions/workflows/lint.yaml)
+
+Convert scholarly full-text XML to Markdown with embedded LaTeX for inline and
+display math. Two dialects are supported behind a single `convert` entry point
+that sniffs the document root and dispatches:
+
+- **JATS** (`<article>`) — the format PubMed Central distributes.
+- **Elsevier** (`<full-text-retrieval-response>`) — the ScienceDirect Article
+  Retrieval API's `xocs`/`ja`/`ce` schema.
+
+The intended consumer is downstream LLM tooling — the markdown is plain text
+suitable for retrieval, summarisation, or analysis without round-tripping
+through a typesetter.
+
+## Spec target
+
+The **JATS** dialect is implemented against the **JATS Journal Archiving and
+Interchange Tag Set (Archiving), NISO Z39.96-2024 v1.4** — the format PMC
+distributes. PMC upconverts older content (NLM Archiving 1.x–3.x, JATS
+1.0–1.3) into 1.4 when serving the OA bucket, so a converter that handles 1.4
+covers the entire PMC corpus regardless of when the article was authored.
+
+This is **not** the Article Authoring tag set (more restrictive; intended as
+an authoring target, not a corpus). Article-Authoring-only content is a
+subset of Archiving content and works without code changes.
+
+The **Elsevier** dialect targets the `ce:`/`ja:`/`xocs:` schema returned by
+the ScienceDirect Article Retrieval API. Math is standard W3C MathML (shared
+with the JATS math path); tables are CALS (`tgroup`/`row`/`entry`); references
+parse the structured `sb:` (Siemens) model. An unrecognised root element
+raises `ValueError` rather than returning an empty string, so a caller passing
+the wrong bytes fails loudly.
+
+## Install
+
+```bash
+pip install -e .              # runtime
+pip install -e '.[dev]'       # runtime + pytest
+pip install -r requirements-dev.txt && pre-commit install   # contributing
+```
+
+Editable install. Provides a `litdown` console script.
+
+## Use
+
+CLI:
+
+```bash
+litdown article.xml > article.md
+litdown article.xml article.md
+```
+
+Library:
+
+```python
+from litdown import convert, mml_to_tex, render_mathml
+
+md = convert("article.xml")           # JATS or Elsevier XML path → markdown
+latex = mml_to_tex(math_element)      # MathML Element → LaTeX
+fragment = render_mathml(math_element, display=True)  # → "$$...$$"
+```
+
+## What's in the package
+
+```text
+litdown/
+  jats.py      JATS XML → Markdown
+  elsevier.py  Elsevier (ce:/ja:/xocs:) XML → Markdown
+  common.py    dialect-neutral leaves (tag helpers, table grid, inline wraps)
+  mathml.py    MathML → LaTeX
+```
+
+The MathML converter is the more battle-tested piece — it has been graded
+against the W3C MathML 3 Presentation test suite using both Pandoc and a
+Gemini blind-grading harness. The cases that survived grading are checked
+in under `tests/w3c_mml/` with their expected LaTeX in `tests/golden.json`;
+the regression suite re-runs the converter over them on every test run.
+
+## Tests and fixtures
+
+```bash
+pytest                                # full suite
+```
+
+Three test files:
+
+- `tests/test_mml_unit.py` — exhaustive per-element MathML cases.
+- `tests/test_jats_articles.py` — structural assertions over real PMC
+  articles in `tests/fixtures/<PMCID>/`, parametrised so adding a fixture
+  extends the suite automatically. Known per-fixture defects are
+  xfail-marked in a `KNOWN_BUGS` dict so the suite stays green; when a fix
+  lands the xfail flips to "unexpectedly passed" and forces the entry's
+  removal.
+- `tests/test_elsevier_articles.py` — structural assertions over Elsevier
+  articles committed as flat `*.xml` files under `tests/fixtures/elsevier/`
+  (math not dropped, CALS tables rendered, every cross-ref/float/reference
+  anchored). Vendor only CC-BY (`by/4.0`) articles; see
+  `docs/elsevier-dialect-plan.md` for how to harvest fixtures.
+
+### Fetching test fixtures
+
+PMC articles are not redistributed in this repository — each article has
+its own licence (a mix of CC-BY, CC-BY-NC variants, and others), and the
+publisher PDFs in particular carry more restrictive terms. The fixture
+directories are gitignored. To populate them:
+
+```bash
+python tools/fetch_pmc.py --manifest tests/fixtures/MANIFEST.txt
+```
+
+This reads `tests/fixtures/MANIFEST.txt` (one PMCID per line), pulls each
+article's JATS XML, publisher PDF, plain text, and referenced figure
+assets from the public `pmc-oa-opendata` S3 bucket, and caches them under
+`tests/fixtures/<PMCID>/`. Fetches are idempotent; re-running is cheap.
+
+The article-fixture tests skip cleanly when no fixtures are present, so
+`pytest` works against the MathML unit suite alone.
+
+## tools/
+
+Discovery and evaluation utilities — none are imported by the package or
+needed for normal use.
+
+| Script | Purpose |
+|---|---|
+| `fetch_pmc.py` | Cache a PMCID's JATS XML, publisher PDF, plain text and figure assets into `tests/fixtures/<PMCID>/`. Default `core` mode skips supplementary materials; pass `--all` to include them. |
+| `eval_articles.py` | Send fixture PDF + our markdown to Vertex AI Gemini and ask it to enumerate content-fidelity gaps. Findings appended to `eval_findings.jsonl`. Run ad-hoc, not in CI. Requires `LITDOWN_GCP_PROJECT` env var or `--project`. |
+| `test_mml.py` | Run our MathML converter against the W3C test suite and against the npm `mathml-to-latex` package; produce a per-test report. |
+| `grade_mml.py` | Blind A/B grade MathML disagreements against the W3C reference using Gemini. |
+| `build_grading_page.py`, `build_preview_page.py` | Build self-contained HTML pages for human review of the grading runs. |
+| `mml2tex_shim.js` | Node entry point used by `test_mml.py` to call the npm `mathml-to-latex` library. |
+
+## The discovery loop
+
+```text
+        fetch_pmc.py            (acquire fixture)
+              ↓
+        litdown.convert
+              ↓
+        eval_articles.py        (Gemini reads PDF + our markdown)
+              ↓
+        triage findings         → encode each as a structural test
+              ↓                    → fix the converter
+        re-run, repeat
+```
+
+The structural test suite is the regression net (deterministic, runs in
+CI). LLM eval is the discovery tool (non-deterministic, runs ad-hoc). Each
+real defect the eval surfaces should be added to
+`tests/test_jats_articles.py` once fixed, so it can never silently regress.
+
+## Known limitations
+
+- Tables typeset as images (older PLOS Genetics, BMJ, etc.) cannot be
+  reconstructed as markdown tables — the converter falls back to an image
+  link so content isn't lost, but downstream tools won't get structured
+  data without an OCR step.
+- The consortium author rendering for papers like gnomAD (PMC7334197)
+  emits the consortium *name* only; individual members listed in nested
+  `<contrib-group>` are dropped.
+- Some end-of-article metadata sections (Author contributions, Competing
+  interests, Funding, Data availability) live inside `<fn-group>` or
+  `<notes>` in `<back>`; these aren't currently rendered.
+- Soft hyphens / line-break artefacts in source XML are not normalised,
+  so words split across lines in the JATS source can render with stray
+  spaces ("si milarity").

litdown-0.3.0/README.md

@@ -0,0 +1,168 @@
+# litdown
+
+[![Lint](https://github.com/populationgenomics/litdown/actions/workflows/lint.yaml/badge.svg)](https://github.com/populationgenomics/litdown/actions/workflows/lint.yaml)
+
+Convert scholarly full-text XML to Markdown with embedded LaTeX for inline and
+display math. Two dialects are supported behind a single `convert` entry point
+that sniffs the document root and dispatches:
+
+- **JATS** (`<article>`) — the format PubMed Central distributes.
+- **Elsevier** (`<full-text-retrieval-response>`) — the ScienceDirect Article
+  Retrieval API's `xocs`/`ja`/`ce` schema.
+
+The intended consumer is downstream LLM tooling — the markdown is plain text
+suitable for retrieval, summarisation, or analysis without round-tripping
+through a typesetter.
+
+## Spec target
+
+The **JATS** dialect is implemented against the **JATS Journal Archiving and
+Interchange Tag Set (Archiving), NISO Z39.96-2024 v1.4** — the format PMC
+distributes. PMC upconverts older content (NLM Archiving 1.x–3.x, JATS
+1.0–1.3) into 1.4 when serving the OA bucket, so a converter that handles 1.4
+covers the entire PMC corpus regardless of when the article was authored.
+
+This is **not** the Article Authoring tag set (more restrictive; intended as
+an authoring target, not a corpus). Article-Authoring-only content is a
+subset of Archiving content and works without code changes.
+
+The **Elsevier** dialect targets the `ce:`/`ja:`/`xocs:` schema returned by
+the ScienceDirect Article Retrieval API. Math is standard W3C MathML (shared
+with the JATS math path); tables are CALS (`tgroup`/`row`/`entry`); references
+parse the structured `sb:` (Siemens) model. An unrecognised root element
+raises `ValueError` rather than returning an empty string, so a caller passing
+the wrong bytes fails loudly.
+
+## Install
+
+```bash
+pip install -e .              # runtime
+pip install -e '.[dev]'       # runtime + pytest
+pip install -r requirements-dev.txt && pre-commit install   # contributing
+```
+
+Editable install. Provides a `litdown` console script.
+
+## Use
+
+CLI:
+
+```bash
+litdown article.xml > article.md
+litdown article.xml article.md
+```
+
+Library:
+
+```python
+from litdown import convert, mml_to_tex, render_mathml
+
+md = convert("article.xml")           # JATS or Elsevier XML path → markdown
+latex = mml_to_tex(math_element)      # MathML Element → LaTeX
+fragment = render_mathml(math_element, display=True)  # → "$$...$$"
+```
+
+## What's in the package
+
+```text
+litdown/
+  jats.py      JATS XML → Markdown
+  elsevier.py  Elsevier (ce:/ja:/xocs:) XML → Markdown
+  common.py    dialect-neutral leaves (tag helpers, table grid, inline wraps)
+  mathml.py    MathML → LaTeX
+```
+
+The MathML converter is the more battle-tested piece — it has been graded
+against the W3C MathML 3 Presentation test suite using both Pandoc and a
+Gemini blind-grading harness. The cases that survived grading are checked
+in under `tests/w3c_mml/` with their expected LaTeX in `tests/golden.json`;
+the regression suite re-runs the converter over them on every test run.
+
+## Tests and fixtures
+
+```bash
+pytest                                # full suite
+```
+
+Three test files:
+
+- `tests/test_mml_unit.py` — exhaustive per-element MathML cases.
+- `tests/test_jats_articles.py` — structural assertions over real PMC
+  articles in `tests/fixtures/<PMCID>/`, parametrised so adding a fixture
+  extends the suite automatically. Known per-fixture defects are
+  xfail-marked in a `KNOWN_BUGS` dict so the suite stays green; when a fix
+  lands the xfail flips to "unexpectedly passed" and forces the entry's
+  removal.
+- `tests/test_elsevier_articles.py` — structural assertions over Elsevier
+  articles committed as flat `*.xml` files under `tests/fixtures/elsevier/`
+  (math not dropped, CALS tables rendered, every cross-ref/float/reference
+  anchored). Vendor only CC-BY (`by/4.0`) articles; see
+  `docs/elsevier-dialect-plan.md` for how to harvest fixtures.
+
+### Fetching test fixtures
+
+PMC articles are not redistributed in this repository — each article has
+its own licence (a mix of CC-BY, CC-BY-NC variants, and others), and the
+publisher PDFs in particular carry more restrictive terms. The fixture
+directories are gitignored. To populate them:
+
+```bash
+python tools/fetch_pmc.py --manifest tests/fixtures/MANIFEST.txt
+```
+
+This reads `tests/fixtures/MANIFEST.txt` (one PMCID per line), pulls each
+article's JATS XML, publisher PDF, plain text, and referenced figure
+assets from the public `pmc-oa-opendata` S3 bucket, and caches them under
+`tests/fixtures/<PMCID>/`. Fetches are idempotent; re-running is cheap.
+
+The article-fixture tests skip cleanly when no fixtures are present, so
+`pytest` works against the MathML unit suite alone.
+
+## tools/
+
+Discovery and evaluation utilities — none are imported by the package or
+needed for normal use.
+
+| Script | Purpose |
+|---|---|
+| `fetch_pmc.py` | Cache a PMCID's JATS XML, publisher PDF, plain text and figure assets into `tests/fixtures/<PMCID>/`. Default `core` mode skips supplementary materials; pass `--all` to include them. |
+| `eval_articles.py` | Send fixture PDF + our markdown to Vertex AI Gemini and ask it to enumerate content-fidelity gaps. Findings appended to `eval_findings.jsonl`. Run ad-hoc, not in CI. Requires `LITDOWN_GCP_PROJECT` env var or `--project`. |
+| `test_mml.py` | Run our MathML converter against the W3C test suite and against the npm `mathml-to-latex` package; produce a per-test report. |
+| `grade_mml.py` | Blind A/B grade MathML disagreements against the W3C reference using Gemini. |
+| `build_grading_page.py`, `build_preview_page.py` | Build self-contained HTML pages for human review of the grading runs. |
+| `mml2tex_shim.js` | Node entry point used by `test_mml.py` to call the npm `mathml-to-latex` library. |
+
+## The discovery loop
+
+```text
+        fetch_pmc.py            (acquire fixture)
+              ↓
+        litdown.convert
+              ↓
+        eval_articles.py        (Gemini reads PDF + our markdown)
+              ↓
+        triage findings         → encode each as a structural test
+              ↓                    → fix the converter
+        re-run, repeat
+```
+
+The structural test suite is the regression net (deterministic, runs in
+CI). LLM eval is the discovery tool (non-deterministic, runs ad-hoc). Each
+real defect the eval surfaces should be added to
+`tests/test_jats_articles.py` once fixed, so it can never silently regress.
+
+## Known limitations
+
+- Tables typeset as images (older PLOS Genetics, BMJ, etc.) cannot be
+  reconstructed as markdown tables — the converter falls back to an image
+  link so content isn't lost, but downstream tools won't get structured
+  data without an OCR step.
+- The consortium author rendering for papers like gnomAD (PMC7334197)
+  emits the consortium *name* only; individual members listed in nested
+  `<contrib-group>` are dropped.
+- Some end-of-article metadata sections (Author contributions, Competing
+  interests, Funding, Data availability) live inside `<fn-group>` or
+  `<notes>` in `<back>`; these aren't currently rendered.
+- Soft hyphens / line-break artefacts in source XML are not normalised,
+  so words split across lines in the JATS source can render with stray
+  spaces ("si milarity").

litdown-0.3.0/litdown/__init__.py

@@ -0,0 +1,57 @@
+"""litdown — convert scholarly full-text XML to Markdown.
+
+Ships two dialects behind a single :func:`convert` entry point, which
+sniffs the document root and dispatches:
+
+* **JATS** (``<article>``) — PMC / NLM full text, via :mod:`litdown.jats`.
+* **Elsevier** (``<full-text-retrieval-response>``) — the ScienceDirect
+  Article Retrieval API's ``xocs``/``ja``/``ce`` schema, via
+  :mod:`litdown.elsevier`.
+
+The :mod:`litdown.mathml` MathML→LaTeX converter and the dialect-neutral
+leaves in :mod:`litdown.common` are shared across both dialects.
+"""
+
+from __future__ import annotations
+
+import importlib.metadata
+import pathlib
+
+import defusedxml.ElementTree
+
+from litdown import common, elsevier, jats
+
+# Re-exported as the package's public API (see __all__): callers use
+# `from litdown import mml_to_tex, render_mathml`.
+from litdown.mathml import mml_to_tex, render_mathml
+
+try:
+    __version__ = importlib.metadata.version('litdown')
+except importlib.metadata.PackageNotFoundError:
+    # Package metadata not available — e.g. running directly from the
+    # source tree without an editable install.
+    __version__ = '0.0.0+unknown'
+
+__all__ = ['__version__', 'convert', 'mml_to_tex', 'render_mathml']
+
+
+def convert(xml_path: str | pathlib.Path) -> str:
+    """Convert a scholarly full-text XML file to Markdown.
+
+    Sniffs the root element's local name (a single cheap parse) and
+    dispatches to the matching dialect. An unrecognised root raises
+    rather than returning ``''`` — a silent empty string would mask
+    "wrong bytes" bugs in the caller, the exact failure mode the Elsevier
+    dialect was added to fix.
+    """
+    tree = defusedxml.ElementTree.parse(xml_path)
+    root = tree.getroot()
+    if root is None:
+        return ''
+
+    name = common.get_tag(root)
+    if name == 'article':
+        return jats.render(root)
+    if name == 'full-text-retrieval-response':
+        return elsevier.render(root)
+    raise ValueError(f'unrecognized root element: {root.tag}')

litdown-0.3.0/litdown/__main__.py

@@ -0,0 +1,27 @@
+"""CLI: ``python -m litdown article.xml [output.md]``."""
+
+from __future__ import annotations
+
+import sys
+
+import litdown
+
+
+def main(argv: list[str] | None = None) -> int:
+    args = sys.argv if argv is None else argv
+    if len(args) < 2:
+        print(f'Usage: {args[0]} <article.xml> [output.md]', file=sys.stderr)
+        return 1
+
+    md = litdown.convert(args[1])
+
+    if len(args) >= 3:
+        with open(args[2], 'w') as f:
+            f.write(md)
+    else:
+        print(md)
+    return 0
+
+
+if __name__ == '__main__':
+    sys.exit(main())

litdown-0.3.0/litdown/common.py

@@ -0,0 +1,161 @@
+"""Dialect-neutral leaves shared by litdown's XML dialects.
+
+These helpers carry no JATS- or Elsevier-specific knowledge; they're the
+bits both :mod:`litdown.jats` and :mod:`litdown.elsevier` would otherwise
+duplicate verbatim: namespace-stripping tag helpers, the xlink href
+accessor, table-cell escaping, the inline typographic leaf formatters, and
+the markdown-table grid builder (colspan/rowspan expansion + multi-row
+header collapse).
+
+The inline *dispatchers* are deliberately NOT shared — JATS and Elsevier
+diverge on cross-ref/link attribute handling enough that one config-driven
+function would be more tangled than two small ones. Each dialect keeps its
+own dispatcher and calls :func:`inline_wrap` for the shared leaf wrappings.
+"""
+
+from __future__ import annotations
+
+import xml.etree.ElementTree as ET
+
+XLINK_NS = 'http://www.w3.org/1999/xlink'
+MML_NS = 'http://www.w3.org/1998/Math/MathML'
+
+
+def get_tag(elem: ET.Element) -> str:
+    """Return an element's local tag name, stripping any ``{ns}`` prefix."""
+    tag = elem.tag
+    return tag.split('}', 1)[1] if '}' in tag else tag
+
+
+# Alias: tests and the Elsevier dialect spell it ``_local``; the JATS code
+# spells it ``get_tag``. Same function, two historical names.
+_local = get_tag
+
+
+def xlink_href(elem: ET.Element) -> str:
+    return elem.get(f'{{{XLINK_NS}}}href') or elem.get('href') or ''
+
+
+def md_escape_cell(text: str) -> str:
+    """Escape pipe characters inside a markdown table cell."""
+    return text.replace('|', '\\|')
+
+
+# ---------------------------------------------------------------------------
+# Inline leaf formatters
+# ---------------------------------------------------------------------------
+
+# Canonical inline-styling tag → markdown wrapping template. Each dialect
+# maps its own element names (e.g. JATS <sub> vs Elsevier <inf>) onto these
+# canonical keys before calling inline_wrap.
+_INLINE_WRAP = {
+    'italic': '*{}*',
+    'bold': '**{}**',
+    'sup': '<sup>{}</sup>',
+    'sub': '<sub>{}</sub>',
+    'underline': '<u>{}</u>',
+    'monospace': '`{}`',
+    'strike': '~~{}~~',
+}
+
+
+def inline_wrap(name: str, inner: str) -> str | None:
+    """Wrap ``inner`` markdown for a canonical inline-styling tag.
+
+    Returns ``None`` if ``name`` is not a recognised shared leaf, so the
+    caller can fall through to its dialect-specific handling.
+    """
+    tpl = _INLINE_WRAP.get(name)
+    return tpl.format(inner) if tpl is not None else None
+
+
+# ---------------------------------------------------------------------------
+# Markdown-table grid builder
+# ---------------------------------------------------------------------------
+
+
+def expand_rows(raw_rows: list[list[tuple[str, int, int]]]) -> list[list[str]]:
+    """Expand colspan/rowspan into a rectangular grid of strings.
+
+    Each input row is a list of ``(content, colspan, rowspan)`` tuples.
+
+    colspan > 1: content in the first slot, empty string in the rest
+                 (preserves the column label without duplicating it).
+    rowspan > 1: content repeated in each spanned row
+                 (keeps every row self-contained for an LLM reader).
+    """
+    if not raw_rows:
+        return []
+    occupied: dict[tuple[int, int], str] = {}
+    for row_idx, cells in enumerate(raw_rows):
+        col_idx = 0
+        for content, colspan, rowspan in cells:
+            # Advance past any cells already occupied by a rowspan above.
+            while (row_idx, col_idx) in occupied:
+                col_idx += 1
+            for dr in range(rowspan):
+                for dc in range(colspan):
+                    # Repeat content across rowspan; use "" for extra colspan slots.
+                    occupied[(row_idx + dr, col_idx + dc)] = content if dc == 0 else ''
+            col_idx += colspan
+
+    if not occupied:
+        return []
+    nrows = max(r for r, _ in occupied) + 1
+    ncols = max(c for _, c in occupied) + 1
+    return [[occupied.get((r, c), '') for c in range(ncols)] for r in range(nrows)]
+
+
+def render_grid(
+    header_rows_raw: list[list[tuple[str, int, int]]],
+    body_rows_raw: list[list[tuple[str, int, int]]],
+) -> str:
+    """Build a markdown table from normalized ``(content, colspan, rowspan)`` rows.
+
+    Shared by the JATS (XHTML) and Elsevier (CALS) table renderers: each
+    dialect translates its own spanning model into the normalized tuple
+    rows, then hands them here. Markdown tables only support a single
+    header row, so multi-row headers are collapsed column-by-column with
+    " / " joins (Nature / extended-data tables routinely use 2-4 levels).
+    """
+    header_rows = expand_rows(header_rows_raw)
+    body_rows = expand_rows(body_rows_raw)
+
+    all_rows = header_rows + body_rows
+    if not all_rows:
+        return ''
+
+    ncols = max(len(r) for r in all_rows)
+
+    def pad(row: list[str]) -> list[str]:
+        return row + [''] * (ncols - len(row))
+
+    def is_decorator(row: list[str]) -> bool:
+        text = ''.join(row).strip()
+        return text in {'', '<hr/>', '<hr />'}
+
+    real_header_rows = [pad(r) for r in header_rows if not is_decorator(r)]
+    if real_header_rows:
+        combined = []
+        for col in range(ncols):
+            seen: list[str] = []
+            for r in real_header_rows:
+                v = r[col].strip()
+                if v and v not in seen:
+                    seen.append(v)
+            combined.append(' / '.join(seen))
+    else:
+        combined = []
+
+    lines = []
+    if combined:
+        lines.append('| ' + ' | '.join(combined) + ' |')
+        lines.append('| ' + ' | '.join(['---'] * ncols) + ' |')
+    else:
+        lines.append('| ' + ' | '.join([''] * ncols) + ' |')
+        lines.append('| ' + ' | '.join(['---'] * ncols) + ' |')
+
+    for row in body_rows:
+        lines.append('| ' + ' | '.join(pad(row)) + ' |')
+
+    return '\n'.join(lines)