myform 0.7.0__tar.gz

myform-0.7.0/PKG-INFO

@@ -0,0 +1,189 @@
+Metadata-Version: 2.4
+Name: myform
+Version: 0.7.0
+Summary: A typed translator that reads MyST and GFM and writes MyST, GFM, rST, and Typst.
+Author: Robb Doering
+Author-email: Robb Doering <robb@doering.ai>
+License-Expression: MPL-2.0
+Classifier: Development Status :: 3 - Alpha
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Operating System :: OS Independent
+Classifier: Framework :: Pytest
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: Text Processing :: Markup :: Markdown
+Classifier: Typing :: Typed
+Requires-Dist: mdformat>=1.0,<2
+Requires-Dist: mdformat-deflist>=0.1.4
+Requires-Dist: mdformat-footnote>=0.1.2
+Requires-Dist: mdformat-front-matters>=2.0.0
+Requires-Dist: mdformat-gfm>=1.0.0
+Requires-Dist: mdit-py-plugins>=0.5.0
+Requires-Dist: more-itertools>=10.7.0
+Requires-Dist: my-basis>=0.8.4
+Requires-Dist: pydantic>=2.11.7
+Requires-Dist: regex>=2024.4.28
+Requires-Dist: srsly>=2.5.3
+Requires-Python: >=3.13
+Project-URL: Homepage, https://gitlab.com/doering-ai/libs/myform
+Project-URL: Repository, https://gitlab.com/doering-ai/libs/myform
+Project-URL: Issues, https://gitlab.com/doering-ai/libs/myform/-/issues
+Description-Content-Type: text/markdown
+
+# myform
+
+> **Status:** In development — the package has not had its first public release.
+> Expect breaking changes.
+
+![Pipeline Status](https://img.shields.io/gitlab/pipeline-status/doering-ai/libs/myform?branch=main) ![Test Coverage](https://img.shields.io/gitlab/pipeline-coverage/doering-ai/libs/myform?branch=main) [![License](https://img.shields.io/gitlab/license/doering-ai/libs/myform)](LICENSE) [![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit)](https://github.com/pre-commit/pre-commit) [![pyrefly](https://img.shields.io/endpoint?url=https://pyrefly.org/badge.json)](https://github.com/facebook/pyrefly) [![ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
+
+`myform` is a universal docs-format translator built around one absolute, typed document tree.
+It reads MyST Markdown and GitHub Flavored Markdown (GFM), then writes idiomatic MyST, GFM, Sphinx reStructuredText, or Typst.
+Formatting is the identity case: `myform.convert(text, format, format)` runs through the same reader, typed tree, and writer as a cross-format conversion.
+
+| Format                  | Read | Write |
+| :---------------------- | :--: | :---: |
+| MyST Markdown           | Yes  |  Yes  |
+| GFM                     | Yes  |  Yes  |
+| Sphinx reStructuredText |  No  |  Yes  |
+| Typst                   |  No  |  Yes  |
+
+The two Markdown identity conversions remain the compatibility rail.
+They produce byte-for-byte the same text as the existing mdformat pipelines, including the MyST opinions developed in this repository.
+
+## Develop from source
+
+Until the first `myform` release, install the checked-out source with `uv`:
+
+```sh
+uv sync --all-extras
+uv run myform --help
+```
+
+After the first public release, the package installation command will be:
+
+```sh
+pip install myform
+```
+
+## Translate documents
+
+```sh
+uv run myform convert guide.md --from myst --to rst -o guide.rst
+printf '# Hello\n' | uv run myform convert - --from gfm --to typst
+uv run myform convert guide.md --from myst --to gfm --report json >guide.gfm
+```
+
+Use `-` as the input path to read stdin.
+Output goes to stdout unless `-o/--output` names a file.
+Conversion reports always go to stderr, keeping document output safe for pipelines.
+Choose `--report summary` (the default), `--report json`, or `--report none`.
+
+## Format documents
+
+The `format` verb performs an identity conversion:
+
+```sh
+uv run myform format README.md
+uv run myform format README.md --format myst -o README.formatted.md
+```
+
+The Python API exposes the same contract:
+
+```python
+import myform
+
+rst = myform.convert(source, 'myst', 'rst')
+formatted = myform.convert(source, 'myst', 'myst')
+```
+
+## Configure translation
+
+`myform` discovers the nearest `myform.yaml`, `.myform.yaml`, or `[tool.myform]` table by walking upward from the target path.
+Within one directory, that order is also the precedence order.
+The nearest directory wins rather than merging several files.
+
+```yaml
+degrade:
+  mode: annotate
+  comments: true
+assist:
+  enabled: false
+typst:
+  postformat: off
+extensions: []
+```
+
+Every non-native conversion is represented in the conversion report.
+The default `annotate` degradation mode also leaves a provenance comment when the target supports comments; `silent` suppresses comments but not reports, while `strict` refuses carrying or dropping unsupported constructs.
+
+The optional LM assist is degradation-only and disabled by default, which makes zero network calls.
+When enabled, it uses the configured local OpenAI-compatible gateway and retains a deterministic fallback.
+
+The committed [`myform.schema.json`](myform.schema.json) and [`myst-options.schema.json`](myst-options.schema.json) schemas describe the translator and mdformat-facade configuration surfaces.
+Refresh or verify both with:
+
+```sh
+uv run myform-write-schema
+uv run myform-write-schema --check
+```
+
+## Use the MyST mdformat facade
+
+The original plugin remains available through mdformat's `myst` extension entry point:
+
+```sh
+uv run mdformat README.md
+```
+
+This repository began as a ground-up rewrite of [`mdformat-myst`](https://github.com/executablebooks/mdformat-myst), written to make uncompiled MyST documents look closer to the hand-formatted sources maintained by the MyST team.
+That facade remains useful on its own:
+
+1. [Targets](https://myst-parser.readthedocs.io/en/latest/syntax/cross-referencing.html#creating-explicit-targets) hug the headings or blocks they identify instead of being separated by an extra blank line.
+
+2. [Directive options](https://myst-parser.readthedocs.io/en/latest/syntax/roles-and-directives.html) prefer the readable `:key: value` prefix form and escalate to a YAML block only when necessary.
+
+3. Directive contents are formatted as first-class Markdown rather than treated as opaque text.
+
+4. Parser and renderer rules are explicit, decorator-registered examples of a modern mdformat extension.
+
+5. YAML values use `srsly`'s maintained `ruamel.yaml` integration.
+
+6. The optional `sembr` mode reflows paragraphs to [one sentence per line](https://sembr.org/) without changing their rendered output:
+
+   ```toml
+   [plugin.myst]
+   sembr = true
+   ```
+
+   Enable it per invocation with `mdformat --sembr <filename>`; `--no-sembr` overrides the TOML setting.
+
+The facade also preserves established skip-formatting comments:
+
+```md
+<!-- myform-ignore -->
+
+This next block is left exactly as written.
+```
+
+```md
+<!-- myform-ignore-start -->
+
+Every block in this range is preserved.
+
+<!-- myform-ignore-end -->
+```
+
+```md
+<!-- myform-ignore-file -->
+```
+
+The markers act at the document top level.
+Their strings remain configurable through `[plugin.myst]`.
+
+## Documentation
+
+- [Documentation front door](docs/index.md)
+- [API reference](docs/api.md)
+- [Universal translator specification](docs/spec/myform-spec.md)
+- [Typst ecosystem memo](docs/spec/typst-ecosystem-memo.md)
+- [Release procedure](RELEASING.md)

myform-0.7.0/README.md

@@ -0,0 +1,158 @@
+# myform
+
+> **Status:** In development — the package has not had its first public release.
+> Expect breaking changes.
+
+![Pipeline Status](https://img.shields.io/gitlab/pipeline-status/doering-ai/libs/myform?branch=main) ![Test Coverage](https://img.shields.io/gitlab/pipeline-coverage/doering-ai/libs/myform?branch=main) [![License](https://img.shields.io/gitlab/license/doering-ai/libs/myform)](LICENSE) [![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit)](https://github.com/pre-commit/pre-commit) [![pyrefly](https://img.shields.io/endpoint?url=https://pyrefly.org/badge.json)](https://github.com/facebook/pyrefly) [![ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
+
+`myform` is a universal docs-format translator built around one absolute, typed document tree.
+It reads MyST Markdown and GitHub Flavored Markdown (GFM), then writes idiomatic MyST, GFM, Sphinx reStructuredText, or Typst.
+Formatting is the identity case: `myform.convert(text, format, format)` runs through the same reader, typed tree, and writer as a cross-format conversion.
+
+| Format                  | Read | Write |
+| :---------------------- | :--: | :---: |
+| MyST Markdown           | Yes  |  Yes  |
+| GFM                     | Yes  |  Yes  |
+| Sphinx reStructuredText |  No  |  Yes  |
+| Typst                   |  No  |  Yes  |
+
+The two Markdown identity conversions remain the compatibility rail.
+They produce byte-for-byte the same text as the existing mdformat pipelines, including the MyST opinions developed in this repository.
+
+## Develop from source
+
+Until the first `myform` release, install the checked-out source with `uv`:
+
+```sh
+uv sync --all-extras
+uv run myform --help
+```
+
+After the first public release, the package installation command will be:
+
+```sh
+pip install myform
+```
+
+## Translate documents
+
+```sh
+uv run myform convert guide.md --from myst --to rst -o guide.rst
+printf '# Hello\n' | uv run myform convert - --from gfm --to typst
+uv run myform convert guide.md --from myst --to gfm --report json >guide.gfm
+```
+
+Use `-` as the input path to read stdin.
+Output goes to stdout unless `-o/--output` names a file.
+Conversion reports always go to stderr, keeping document output safe for pipelines.
+Choose `--report summary` (the default), `--report json`, or `--report none`.
+
+## Format documents
+
+The `format` verb performs an identity conversion:
+
+```sh
+uv run myform format README.md
+uv run myform format README.md --format myst -o README.formatted.md
+```
+
+The Python API exposes the same contract:
+
+```python
+import myform
+
+rst = myform.convert(source, 'myst', 'rst')
+formatted = myform.convert(source, 'myst', 'myst')
+```
+
+## Configure translation
+
+`myform` discovers the nearest `myform.yaml`, `.myform.yaml`, or `[tool.myform]` table by walking upward from the target path.
+Within one directory, that order is also the precedence order.
+The nearest directory wins rather than merging several files.
+
+```yaml
+degrade:
+  mode: annotate
+  comments: true
+assist:
+  enabled: false
+typst:
+  postformat: off
+extensions: []
+```
+
+Every non-native conversion is represented in the conversion report.
+The default `annotate` degradation mode also leaves a provenance comment when the target supports comments; `silent` suppresses comments but not reports, while `strict` refuses carrying or dropping unsupported constructs.
+
+The optional LM assist is degradation-only and disabled by default, which makes zero network calls.
+When enabled, it uses the configured local OpenAI-compatible gateway and retains a deterministic fallback.
+
+The committed [`myform.schema.json`](myform.schema.json) and [`myst-options.schema.json`](myst-options.schema.json) schemas describe the translator and mdformat-facade configuration surfaces.
+Refresh or verify both with:
+
+```sh
+uv run myform-write-schema
+uv run myform-write-schema --check
+```
+
+## Use the MyST mdformat facade
+
+The original plugin remains available through mdformat's `myst` extension entry point:
+
+```sh
+uv run mdformat README.md
+```
+
+This repository began as a ground-up rewrite of [`mdformat-myst`](https://github.com/executablebooks/mdformat-myst), written to make uncompiled MyST documents look closer to the hand-formatted sources maintained by the MyST team.
+That facade remains useful on its own:
+
+1. [Targets](https://myst-parser.readthedocs.io/en/latest/syntax/cross-referencing.html#creating-explicit-targets) hug the headings or blocks they identify instead of being separated by an extra blank line.
+
+2. [Directive options](https://myst-parser.readthedocs.io/en/latest/syntax/roles-and-directives.html) prefer the readable `:key: value` prefix form and escalate to a YAML block only when necessary.
+
+3. Directive contents are formatted as first-class Markdown rather than treated as opaque text.
+
+4. Parser and renderer rules are explicit, decorator-registered examples of a modern mdformat extension.
+
+5. YAML values use `srsly`'s maintained `ruamel.yaml` integration.
+
+6. The optional `sembr` mode reflows paragraphs to [one sentence per line](https://sembr.org/) without changing their rendered output:
+
+   ```toml
+   [plugin.myst]
+   sembr = true
+   ```
+
+   Enable it per invocation with `mdformat --sembr <filename>`; `--no-sembr` overrides the TOML setting.
+
+The facade also preserves established skip-formatting comments:
+
+```md
+<!-- myform-ignore -->
+
+This next block is left exactly as written.
+```
+
+```md
+<!-- myform-ignore-start -->
+
+Every block in this range is preserved.
+
+<!-- myform-ignore-end -->
+```
+
+```md
+<!-- myform-ignore-file -->
+```
+
+The markers act at the document top level.
+Their strings remain configurable through `[plugin.myst]`.
+
+## Documentation
+
+- [Documentation front door](docs/index.md)
+- [API reference](docs/api.md)
+- [Universal translator specification](docs/spec/myform-spec.md)
+- [Typst ecosystem memo](docs/spec/typst-ecosystem-memo.md)
+- [Release procedure](RELEASING.md)

myform-0.7.0/myform/Ignore.py

@@ -0,0 +1,191 @@
+"""Skip-formatting ("ignore") comments -- the mdformat #53 gap, filled for MyST.
+
+mdformat core has no way to exempt content from formatting. This module adds the family every
+other formatter ships, as configurable HTML comments (already valid CommonMark, so no new grammar):
+
+- ``<!-- myform-ignore -->`` preserves the *next* block verbatim;
+- ``<!-- myform-ignore-start -->`` ... ``<!-- myform-ignore-end -->`` a range;
+- ``<!-- myform-ignore-file -->`` the whole file.
+
+Markers act only at the **document top level**: one nested inside a list item or blockquote is left
+as an ordinary comment and does not suppress formatting -- acting on it would unbalance the spliced
+token tree and re-apply the container's prefix to the preserved source every run. The marker strings
+are configurable via `MystOptions`.
+
+The mechanism is a ``markdown-it`` core rule that runs after block parsing: the tokens of an ignored
+span are replaced by a single ``myst_ignored`` token carrying the span's *original source*, which
+`Renderer.render_myst_ignored` emits verbatim (an ``html_block`` carrier would ``lstrip`` the first
+line and de-indent preserved code). The replacement is ephemeral -- internal to one render -- and
+the emitted markdown is re-parsed fresh on the next run, so formatting stays idempotent (the marker
+comments are themselves preserved and re-detected).
+"""
+
+############
+### HEAD ###
+############
+### STANDARD
+### EXTERNAL
+import regex as re
+from markdown_it.token import Token
+from markdown_it.rules_core import StateCore
+
+### INTERNAL
+from .options import MystOptions
+
+############
+### DATA ###
+############
+#: Matches an HTML-comment block whose entire content is a single comment, capturing the inner text.
+COMMENT = re.compile(r'\s*<!--\s*(?P<body>.*?)\s*-->\s*\Z', re.DOTALL)
+
+
+############
+### BODY ###
+############
+class Ignore:
+    """The ignore-comment core rule and its token-surgery helpers."""
+
+    @staticmethod
+    def apply(state: StateCore) -> None:
+        """Core rule: replace ignore-marked spans with verbatim-source ``html_block`` tokens."""
+        opts = MystOptions.from_options(
+            state.md.options.get('mdformat', {}).get('plugin', {}).get('myst', {})
+        )
+        markers = (
+            opts.ignore_comment,
+            opts.ignore_start_comment,
+            opts.ignore_end_comment,
+            opts.ignore_file_comment,
+        )
+        # Fast path: nothing that could be a marker appears in the source at all.
+        if not any(marker in state.src for marker in markers):
+            return
+
+        tokens = state.tokens
+        src_lines = state.src.split('\n')
+
+        # File-level: any file marker (at any nesting) preserves the whole document, unchanged.
+        if any(Ignore._marker(tok, opts) == 'file' for tok in tokens):
+            state.tokens = [Ignore._verbatim(src_lines, 0, len(src_lines))]
+            return
+
+        new: list[Token] = []
+        i, n = 0, len(tokens)
+        while i < n:
+            tok = tokens[i]
+            kind = Ignore._marker(tok, opts)
+
+            # `next`/`start` markers act only at the document top level. A marker nested inside a
+            # container (list item, blockquote) is left as an ordinary comment: acting on it would
+            # unbalance the spliced token tree (crash) and re-apply the container prefix to the
+            # preserved source every run (unbounded ``> > x``). The verbatim carrier is always
+            # emitted at level 0, so container blocks it *contains* still round-trip (their source
+            # already carries the right prefixes).
+            if kind == 'next' and tok.level == 0:
+                new.append(tok)  # keep the marker comment itself
+                i += 1
+                # The block to preserve is the next token (markdown-it emits no blank tokens between
+                # blocks). Skip if it is another marker, is nested, or has no source map.
+                nxt = tokens[i] if i < n else None
+                if nxt and nxt.level == 0 and nxt.map and Ignore._marker(nxt, opts) is None:
+                    span, i = Ignore._consume_block(tokens, i)
+                    Ignore._emit_verbatim(new, src_lines, span[0], span[1])
+                continue
+
+            if kind == 'start' and tok.level == 0:
+                new.append(tok)
+                start_map = tok.map
+                start_line = start_map[1] if start_map else 0
+                # Match only a top-level end marker; a nested one would leave the splice unbalanced.
+                j = i + 1
+                while j < n and not (
+                    tokens[j].level == 0 and Ignore._marker(tokens[j], opts) == 'end'
+                ):
+                    j += 1
+                if j < n:  # matched end marker at top level
+                    end_map = tokens[j].map
+                    end_line = end_map[0] if end_map else len(src_lines)
+                    Ignore._emit_verbatim(new, src_lines, start_line, end_line)
+                    new.append(tokens[j])  # keep the end marker
+                    i = j + 1
+                else:  # unterminated range -> preserve to end of document
+                    Ignore._emit_verbatim(new, src_lines, start_line, len(src_lines))
+                    i = n
+                continue
+
+            new.append(tok)
+            i += 1
+
+        state.tokens = new
+
+    # -------------------
+    # `.` Helpers
+    # -------------------
+    @staticmethod
+    def _marker(token: Token, opts: MystOptions) -> str | None:
+        """Classify *token* as an ignore marker: ``next``/``start``/``end``/``file`` or ``None``."""
+        if token.type != 'html_block':
+            return None
+        match = COMMENT.fullmatch(token.content)
+        if not match:
+            return None
+        body = match['body']
+        if body == opts.ignore_file_comment:
+            return 'file'
+        if body == opts.ignore_start_comment:
+            return 'start'
+        if body == opts.ignore_end_comment:
+            return 'end'
+        if body == opts.ignore_comment:
+            return 'next'
+        return None
+
+    @staticmethod
+    def _consume_block(tokens: list[Token], i: int) -> tuple[list[int], int]:
+        """Return ``(source_span, next_index)`` for the block starting at ``tokens[i]``.
+
+        Walks the open/close nest to its matching close, taking the source span from the opener's
+        first line through the *maximum* end line of any token inside. This is required because the
+        plugin's ``myst_directive_open`` maps only the opener line (its closer sits on a separate
+        ``myst_directive_close`` token), unlike stock markdown-it blocks whose opener spans the
+        whole block; the max-end walk is correct for both.
+        """
+        start = tokens[i]
+        start_map = start.map or [0, 0]
+        first_line, last_line = start_map[0], start_map[1]
+        depth = start.nesting
+        j = i + 1
+        while depth > 0 and j < len(tokens):
+            child_map = tokens[j].map
+            if child_map:
+                last_line = max(last_line, child_map[1])
+            depth += tokens[j].nesting
+            j += 1
+        return [first_line, last_line], j
+
+    @staticmethod
+    def _emit_verbatim(new: list[Token], src_lines: list[str], start: int, end: int) -> None:
+        """Append a verbatim carrier for ``src_lines[start:end]`` only when it has content.
+
+        Guards the degenerate empty span (e.g. a bare unterminated ``ignore-start`` at EOF) so no
+        empty block is emitted, which would otherwise add a stray trailing blank line.
+        """
+        token = Ignore._verbatim(src_lines, start, end)
+        if token.content:
+            new.append(token)
+
+    @staticmethod
+    def _verbatim(src_lines: list[str], start_line: int, end_line: int) -> Token:
+        """Build a ``myst_ignored`` token holding ``src_lines[start_line:end_line]`` verbatim.
+
+        Always emitted at the document top level (``level == 0``); its renderer
+        (`Renderer.render_myst_ignored`) returns the source untouched -- unlike an ``html_block``
+        carrier, whose renderer would ``lstrip`` the first line and de-indent preserved code.
+        """
+        token = Token('myst_ignored', '', 0)
+        # ``strip('\n')`` drops the blank lines surrounding a range's content (between the markers)
+        # without touching leading *spaces*, so an indented code block keeps its indent.
+        token.content = '\n'.join(src_lines[start_line:end_line]).strip('\n')
+        token.block = True
+        token.map = [start_line, end_line]
+        return token